For the most part I am ambivalent towards the multitasking feature added in the new OS. It really doesn't change how I use my device for the most part. I have however found one issue with memory usage that I think is relevant to the conversation.
I am a big Netflix user. I would guess that watching "Dexter" or "Weeds" probably consumes at least 75% of my usage right now. I have found that Netflix will get stuck loading when I try to run the app if I have too many "frozen" applications. I have seen this numerous times. I can try to run the app any number of times and it simply never gets past the loading screen. If however I double tap the home button and close all of the frozen tasks the app loads right up and works great. From my simple observation, those apps that are frozen are taking up resources. It could be that some apps have not been updated to play well with iOS 4.x and this might go away. At this time however I am not willing to say that frozen apps have no effect on system performance.