They're called badges and the meaning varies with the app. For the app store, as a previous poster said, it means the number of updated apps available for you. On my calendar app it means the number of appointments left that day. You have to check the settings for each app because if you don't want badges you can turn them off.