It isn't a big problem. The difficulty with reading reviews - especially those on Amazon - is that it is too easy to accept the reviews at face value and panic. If all of the reviews say the same thing, then there's a problem. If it's just one review, it's a fad of the reviewer. I know this from personal experience, as my own books have had 'odd' reviews.
Ask yourself this: how many books have you sold? And how many of these complaints have you had?
My own take on the reviewing process can be found at http://www.ianhughesma.com/2011/08/26/the-problem-with-reviews/
I've come to the conclusion that all you can do is write the book that you would want to read. If a reviewer doesn't like the way you've done it, they've got the right to go away and write their own book. Chances are that they won't!
Don't worry, after a while the criticisms begin to bite less deeply and you can focus on writing rather than worrying!