Hi, I'm wondering if there has been any luck narrowing down the cause of this.
The same thing is happening for me (iPad 1 3G/iOS5.01) - when I close an album the cover disappears so I'm just left with the album title and blank space above it.
Strangely though, it only seems to happen in landscape view and only with the albums in certain positions. For instance only the last album in the first row and the last two albums in the second row present this issue. The other albums can be opened and closed without the vanishing act.
If I switch from album to photo view and back to album view the covers will reappear and perform as normal from then on. Similarly when I hit the edit button then done, the covers will reappear the next time you close the album and will no longer disappear. They also instantly reappear if I drag them to a new position or drag another album across the top of the vanished one.
I also found that while the cover is missing if there is any change to the position of the album (e.g. through indirect reordering or a switch from landscape to portrait view) the title will disappear as well and all I have is a black space where it should be - and in this state the album is unable to be selected for opening, moving, or deletion.
Any insight to this would be appreciated. As the OP said, it's mainly just an annoyance but I'd like to know if it's something I can fix or not. I can't seem to find any other mention of this issue elsewhere.