If you have a cellular model, no problemo. Those of us with Wifi, though, the time can drift, sometimes quite badly. IOS4 will never pick up time using the internets, only by phone towers. Syncing with iTunes will only set the time if it is more than a few minutes off. To force iTunes to set the time, you may have to flip the device's time way off before syncing.
Mine loses almost 10 seconds a day, so it doesn't take long for it to be wrong enough to break Google's time-based authenticator (like an RSA badge). If you want to see how far off you are, get the free "Emerald Time" app.