The following applies only if you are using iTunes to sync your music: Instructions were tested on Mac OS X, Windows options may vary slightly in the location and name.
Go to iTunes on the computer. Under the Advanced menu you will find a Get Album Artwork item. If you select it iTunes will go to the iTunes store and attempt to find artwork for your songs. This is hit and miss with songs imported from CD, but works more often than not.
Of course it only works if the album is available in iTunes.
I believe there are third party album art hunting programs that will work with iTunes, but I don't know any names. You can try searching for them.
You can also manually add art to your iTunes library. Right click on a song and select Get Info. In the artwork tab you can add or change the cover. Of course you need the art first. Either scan (if you have a scanner) the CD cover, or do a Google image search to find an acceptable cover. Once it's on your computer you can either drag and drop or use the add option.
Unfortunately you can't add art to multiple songs. Some info can be changed this way, but not the artwork, probably for some reason only the program developers know.
Once you have the art in iTunes it is only a matter of syncing your iPad again.