OK - not quite sure why you're not seeing the 'X' when the icons jiggle. For sure, you can't delete the apps that came with the iPad - Safari, Mail etc - so those icons jiggle, but don't have the 'X', but bookmarks should.
So let's try something which fixes a lot of problems - power the iPad down completely by pressing and holding the Power button until the slider control appears, accept the offer to switch off, wait until the iPad screen is completely blank, then switch on again by pressing and holding the Power button until the white Apple logo appears and wait until it powers up.
Now try again by pressing any icon until they jiggle and see if the 'X' has re-appeared.
Tim