I hadn't rebooted for awhile so thought I'd do so. I held the wake/sleep button, then slid the red bar across when prompted. It then went off. So then I turned it on, entered my passcode, and was surprised to see my same apps running in the multi-task bar. Is it supposed to work this way? Do I need to shut them all down manually first for a reboot?

That is normal behavior.

The apps stay in a frozen state, even when the iPad is restarted. And yes, you have to close them down manually. There is no automatic way to do so.

Some don't freeze but instead restart.