Lots of people don't read books or read them very rarely. "The Da Vinci Code", by Dan Brown was and is somewhat of an exception to that statement. Lots of people read his book. Almost all of them enjoyed it. It has a fairly interesting premise, its paced fairly well, and when the villain is revealed the pieces all come clearly together.
The book as a mystery does have some drawbacks. Right now it is very popular to solve mysteries with science, for example the CSI franchise. Solving riddles as a means to discovery is old fashioned. However, religion is a really big topic in the US and Canada and as such the topic of the book is somewhat more important than the fashion the mystery is solved.
I read a study about th effect of television on mood. The study involved randomly calling subject on the phone asking them to rate their mood and then asking them what they were doing immediately before the phone call. It turns out that the average mood while watching tv is 'borderline depression'. The scientists theorized that this is because tv is an entirely passive medium. All you have to do is sit there and you are fed the story. Reading, on the other hand, resulted in a positive mood. The theory is that reading is something that has to be done actively. You have to read to get the story, not just sit there. You are doing something vs not doing anything.
This plays into why I think the movie is getting bad reviews. Like I said before, lots of non-readers have read "The Da Vinci Code". It is a pretty decent book, but besides that it is a huge success. People discovered along with the story, how fun reading actually is and beyond that how fun it is to discuss books with other people.
Now, these people who don't normally read may not normally enjoy reading for whatever the reason discovered a book and really enjoyed reading it and they really enjoy discussing it. Which is something that they wouldn't have particularly expected. Movies on the other hand are something that people do like. They like them a lot. they go expecting to be entertained. (Much like watching tv). Now if something like reading which is normally not liked or at least not done very much for pure enjoyment can be so good, people may be thinking, how much better will the movie be because movies are always enjoyable.
The expectations get set so high that when there are things that can't be done because of the format, it lessens the story and enjoyment of the movie. For example in a book, the author can just write what one of the characters is thinking or all of the characters for that matter. In a movie that is much more difficult and would require a narrator, weird conversations or excessive use of flashbacks.
Flashbacks in a book are better than flashbacks in a movie. It is perfectly clear that you are reading a flashback and in your mind you imagine them equally as well as the present time. In a movie however, you have to make it clear that it is a flashback or the audience will be confused (especially if they haven't read the book). One of the ways a flashback is signalled in a movie is to use a steady cam shot, a hazy focus or weird processing after the fact. Flashbacks just don't cut it in movies like they do in our imagination.
I think that many of the poor reviews are based on a few of these ideas and maybe a few others. Really the medium is the problem here, not the story.