Sometimes when opening Mail this sound symbolizing a new mail has arrived gets played, but then there is no new email.

I have already refreshed, looked into each email inbox separately and just can't find it. It is driving me nuts.

Any ideas as to why this would happen?

Happens to me all the time.

My computer and email program are running most of the day. If the iPad grabs email before the computer (and with Push it almost always does), it dings and adds a badge to the Mail icon. Then the computer goes to the site and gets the email. About half the time it's junk mail, so the computer removes it from the inbox and places it into the Junk Folder. This causes another push notification to the iPad, which removes the email from it's inbox (but usually leaves the badge on the icon).

This is pretty typical for an IMAP mail server. Each app/device updates the mail folders online, which instantly (or eventually) updates email on all other devices. You get a lot of false alarms this way, but you almost never have to go through all your devices and delete the email again, and again, and. . .

Sounds like a reasonable explanation.

But would that still be the case, if the iPad is the only device connected go the internet, accessing my emails?

I am still waiting for the landline to be connected, so none of my laptop/pc is online reading my emails. Unless someone else has access to my email accounts.

No, if your iPad is the only thing accessing your email, that shouldn't happen. Though I suppose it might with some email providers. I only use Yahoo! (free), GMail, and MobileMe, so my experience is limited to those.

Depending on your notification setup, there is more than one thing that can make your iPad go ding.

Go into Settings >> Notifications and look over the settings for each app. Maybe it is something else trying to get your attention. Sound, badge, and alerts can all be turned on/off individually.