Just to clarify this, there is a single pot of building materials.  You can store up to 10 of each building material for use in upgrades - total.  So when you go to upgrade a building and you have 5 granite and use 4 granite, when you go to upgrade another building you will just have 1 granite available.  If this is not the case and for instance, you started the day with 5 granite, but only used 1 during the day but end the day with 2 granite, then you  should file a support ticket with Playdom.  Ever since the problem with receiving materials and gifts a few weeks ago, some people have complained of strange things like materials mysteriously disappearing.