That is the way it works, which I honestly do find to be a bit of a flaw. If you don't go in and close out the apps all the way (by holding any one of them down until they "wiggle" at which point you see the little minus sign where you can delete them), they will just keep adding up! At some point, I am sure this will necessarily impact either the memory or the storage space or both.
It takes some getting used to, mostly because people have preconceptions about what they think Apple's multi-tasking implementation should be and misconceptions about how it actually works.
Only applications which are specifically updated to support multitasking will be using additional memory over say a 3.x implementation, however Apple have been extremely cautious about exactly "how and what" is kept running in the background. Typically, a true multi-tasked application has a much smaller memory footprint in the background compared to when it is in the foreground, so the net effect of backgrounding an application is quite small in terms of RAM. That's not to say it is nothing...
Applications which do not support any of the multitasking features will still appear in the multitasking bar, and people naturally assume these must be taking up memory. Actually it is better to think of the multitasking bar as a list of recently used programs. Some may be running in the background if they are multitasking enabled, the others are simply like a desktop shortcut, and will launch the app fresh if pushed again.
It took me a good week to get over the OCD "must close evertying" behaviour when I got the launch day iPhone4. I would urge all new IOS4 iPad users to try and get over their preconceptions about the bar and force yourself to try and use it for extended periods of time without constantly clearing out the icons. iOS has pretty decent memory management and you really shouldn't get to a state where you notice any negative effects from having lots of apps in the bar. If you do, by all means close some.
What you end up with is actually a quick launch bar with the things you use the most at the start... Quite useful I find...
Hope that helps.