A few years ago I would have described myself as "a migratory life form with a tropism for bookstores." Amazon weened me off of bookstores with books I could have delivered direct to my home. Then they broke me of the paper habit with their wonderful Kindle devices. They have the ebook market sewn up almost as tightly as Apple and the music market.
Sadly the truth is that less than 10% of the US population ever sets foot in a bookstore, and that was before Amazon. While I am sure Libraries account for a significant number of readers, the fact is very few adults read books. It will never be as big a market as music. While the same people who do read probably account for iPad sales, it will still be the smaller portion of the market.
So while Apple will probably be happy to keep the iBook store around as an option, they will be equally happy to have someone else providing the mainstream book sales.