The music playing app was renamed to Music (from iTunes). I don't remember exactly what version of iOS, but it should be changed on any device that's running iOS 5.x or later.
As Johanna says, neither the Music app nor the iTunes Store app can be deleted. They are most likely restricted or hiding in a folder somewhere.
But, I'm wondering if the person who gave/sold the iPad to you restored it as they should have. Because if they did then there should be no restrictions set and nothing but the native apps (until you add them yourself). If you got it with stuff on it, those apps will be associated with the original user's account, and you will not be able to update them. Not to mention (ok, I'm mentioning) many other problems that can crop up due to what ever settings and accounts they've set up previously.
