HOME > Chowhound > Cookware >
Do you create unique foods?
TELL US

Amazon ratings- wisdom of the crowd?

h
hobbess Feb 24, 2010 12:25 AM

Does anybody see any problems with the following methodology:

If you want to buy a measuring cup or some other cookware product, go to Amazon and look through the items they sell in that category, and pick the one item in that category with the highest rating? Assuming the sample size was large enough, wouldn't that method tell you what the best products would be?

Or, do you think a single expert would be a better guide who's able to test a wide range of products in the same category against each other?

I've noticed that there are quite a few equipment recommendations from something like Cook's Illustrated that get really poor ratings on Amazon.

  1. Upload a photo (10 MB limit)
Delete
  1. tanuki soup Feb 24, 2010 01:31 AM

    For all such rating systems, I find that reading the actual contents of the reviews tells you much more than simply looking at the number of stars. You can usually identify the people who know what they're talking about and those who don't. I also think that the reviews giving a product 2 to 4 stars (rather than 1 or 5 stars) are the most informative because they tend to provide a more balanced analysis of the product's strengths and weaknesses.

    5 Replies
    1. re: tanuki soup
      Soop Feb 24, 2010 02:50 AM

      I agree. Also, it depends how many reviews it has. I'd research a 1* item with 1 review, but I'd be drawn to a 5* item with 345 reviews.

      1. re: Soop
        b
        Beckyleach Feb 24, 2010 06:30 AM

        Agreed. That said, I've been shopping on Amazon since Day 1 and the reviews--once they built up a bit over the years--have been a MOST reliable indicator--as long as I read through them all and take into account the fools, the chronically whiny, and those who give a product a bad rating because UPS left it next door. ;-)

        1. re: Beckyleach
          c
          CocoaNut Feb 26, 2010 12:07 PM

          :)))

      2. re: tanuki soup
        Indirect Heat Feb 25, 2010 07:26 PM

        I also like the 1 and 5 star reviews. Particularly with something like recipe books. Most of my favourite recipe books have 1* ratings that say something like, "This book is much too hard. All the recipes required real ingredients, and sometimes required 3 or more steps. Rachel Ray, this author is not."

        The crowd is definitely not wise. But using the reasons that people hate and love stuff will point you towards what you actually want or need.

        1. re: Indirect Heat
          Chemicalkinetics Feb 25, 2010 08:06 PM

          Hmm, I tend to buy any book which say "Rachel Ray, this author is not"

      3. a
        armagnac Feb 24, 2010 03:04 AM

        Cook's Illustrated publishes a magazine by that name and also produces the America's Test Kitchen and Cook's Country TV programs. They test competing products systematically against each other as well as against criteria such as effectiveness, safety, and price. Think of them as a specialized Consumer's Report for the home kitchen.

        None of the customers who post their opinions on amazon.com does this kind of systematic comparative testing, and if you try to make comparisons among them, good luck! I may look for the bad amazon.com reviews of a product to find out if there's any significant problem that Cooks Illustrated didn't pick up, if they tested it at all. If several users have the same complaint, that may be a red flag.

        Otherwise, I'd rely on the professionals. If you go with their recommendations, you may not get absolutely positively the very best in a category, which finally is a judgment call, but at least you shouldn't go wrong. I haven't, anyway.

        You can check out Cooks Illustrated's equipment reports, taste tests, recipes, how-to videos, and all the rest for free during a 2-week trial period, downloading as much content as you want, before deciding whether to pay. It's here:

        http://www.cooksillustrated.com/

        2 Replies
        1. re: armagnac
          Soop Feb 24, 2010 03:19 AM

          "None of the customers who post their opinions on amazon.com does this kind of systematic comparative testing, and if you try to make comparisons among them, good luck! I may look for the bad amazon.com reviews of a product to find out if there's any significant problem that Cooks Illustrated didn't pick up, if they tested it at all. If several users have the same complaint, that may be a red flag."

          However, I doubt Cooks Illustrated test it in a large number of real-world scenarios with different techniques and recipes etc.

          Cooks illustrated etc are good for straightforward comparisons, but both have their place. Also, sometimes, you might not find the specific item mentioned I suppose.
          But if there enough reviews,

          1. re: Soop
            a
            armagnac Feb 24, 2010 03:45 AM

            Real world situations do put different demands on equipment, no doubt. But there are infinite real-world situations, the amazon.com reviewers don't spell out exactly their own situations, and besides, it's not unknown for equipment to fail because the user didn't know how to use it, or was careless. So I'd say that the results of systematic tests conducted in a testing laboratory are inherently more reliable than you can realistically expect to get from amazon.com. There may be one user who you could really trust to tell you something important that the pros somehow didn't, but how can you find her among all the others?

            For free advice, better to ask and look for it at Chowhound, where at least there's interaction among the posters, including yourself, so you can ask follow-up questions. But when buying something at all expensive, better check what the professional testers have to say.

        2. Joe Berger Feb 24, 2010 05:46 AM

          Amazon reviews are helpful, but always keep a few things in mind. reviewers often post their very first impressions which may change significantly over time. reviewers often have no other similar product experience to compare it to. reviewers often give items as gifts, so their impression and that of the end user, may turn out dramatically different. be careful of some detailed all positive reviews who may be a business affiliate of that product.

          3 Replies
          1. re: Joe Berger
            kleine mocha Feb 26, 2010 09:46 AM

            I agree with those who say you must actually read the reviews. I just purchased a book called the Legume Bible (or possibly the Bean Bible, not in hand at the moment) after checking it out of the library. One reviewer who gave it a low star rating explaing it was because the book was not vegan or vegetarian!!

            1. re: kleine mocha
              Chemicalkinetics Feb 26, 2010 09:56 AM

              Agree. Obviously, it becomes very difficult to read all the reviews when a product has 30+ reviews. So it is good practice to read some 5 star reviews with high rating and some 1-2 star reviews with low rating. In fact, some time a 3-4 star is most balanced.

              1. re: Chemicalkinetics
                flourgirl Mar 3, 2010 12:17 PM

                kleine mocha - that is so funny! I have the same book (actually bought it from the library for $1) and I saw that same review. You really do have to read the reviews.

          2. s
            smkit Feb 24, 2010 05:54 AM

            I tend to rely on Amazon a bit more with kitchen equipment that is electric as Cook's Illustrated IMO can't keep up with all the models, brands etc. and give them a long enough test to see how durable they are. But for roasting pans, knives, cookware etc. I think they do a good job of testing.

            With Amazon I also feel there is a downward bias as people with problems will usually write in to warn people rather than people with good experiences singing the praise of an object.

            There is also very little way with Amazon to ensure the reviewers are using the items correctly. Buying cutting boards is really hard based upon Amazon reviews. I would almost bet that many are sending them through the dishwasher.

            But then again, Cook's Illustrated knocks Epicurean cutting boards because they 'clack' too much. Seriously? Does this affect the cutting?

            Anyhow my dog loves the Epicurean 'clack' because she knows when I might be cutting up some food that just may fall onto the floor.

            1. hobbybaker Feb 24, 2010 06:07 AM

              I sometimes wonder if CI checks/can check "durability". My example is the "All-Clad Slow Cooker" . CI reviewed it very highly, but many Amazon reviewers reported troubles and some of them are shortly after their warranty periods are gone. Many of Amazon (or cooking.com) reviews are written in a hype shortly after their purchases, but some of them are quite insightful and sometimes updated especially if they had troubles. As other people have pointed out, chow is the good place as there are always someone with good knowledge/experience who are kind enough to answer my follow-up questions.

              2 Replies
              1. re: hobbybaker
                Soop Feb 24, 2010 06:41 AM

                I always wait 6 months to a year after reviewing something like cookware on Amazon.

                1. re: Soop
                  hobbybaker Feb 24, 2010 08:58 AM

                  Hi, Soop - Then, you are the one who always got my "yes" to the helpful vote ;)

              2. grnidkjun Feb 24, 2010 06:53 AM

                What??? Opinions other than chowhounders Matter???? ;)

                I think you've gotten a lot of good advice here so no need for me to repeat it.

                1 Reply
                1. re: grnidkjun
                  s
                  shmulibaby Feb 24, 2010 07:05 AM

                  One other tip relating to "crowd wisdom": for items that have more than just a few reviews, I like to sort the reviews by "most helpful". This way you're putting the more useful reviews at the top and not wasting as much time reading the idiot ones.

                2. r
                  runwestierun Feb 24, 2010 07:32 AM

                  I rely very heavily on Amazon ratings and consider it my most reliable source of consumer information. I think the reviewers at CI are demographically similar, whereas I feel I get a wider cross section at Amazon. I bought a propane wok at Amazon, and one reviewer from Hawaii talked about cooking a luau for 50ppl with it. I don't think luaus are in the front of Christopher Kimball's brain. Also, at Amazon, you get people who are buying an item to replace a model of the same item that didn't work as well ("the old one was too heavy, this one is lighter and won't scratch the table). CI might not be aware of issues with older models not in their possession. Only once have I bought an item rated highly by CI and low at Amazon, and it was insignificant (plastic wrap), but I agreed with the Amazon consensus.

                  I especially rely on Amazon if a product is quite new on the market. I bought several SkyScouts (a video camera-looking thing that will identify a star for you using GPS) after reading the reviews. People were having trouble with them, but once one of the reviewers figured out to get away from metal buildings, it worked great for everyone. I really value Amazon's rating system. That coupled with their free shipping makes them my go-to retailer.

                  1 Reply
                  1. re: runwestierun
                    Joe Berger Feb 24, 2010 11:40 AM

                    I've also noticed how the Amazon reviews can start negative with a fatal flaw, and later turn positive once a solution is found. I recently needed to purchase a stove top coffee percolator and found a $20 Farberware that looked great but most hated because the top plastic dome knob would melt after a few uses. Then someone figured out you could buy a replacement GLASS knob, while others complained the glass knobs shatter easy. Finally this collective zeitgeist concluded that you buy the glass knob, and avoid screwing it on tight, then voilà- you now have the perfect peculator under $30 shipped - that's exactly what I did, and I could not be happier with my purchases - only thanks to those reviewers.

                  2. Chemicalkinetics Feb 24, 2010 08:15 AM

                    Hobbes,
                    I find the Amazon feedback is a great service. For disclosure, I am an active participator at it. I find the “Star-Rating” is good for skimming through products, but not as reliable as reading through the reviews. Sometime you have people who never bought the product and give a rating and sometime you have people give a poor rating because the shipping was late or the product does not fit their special need. I have read a review which the guy gave a poor rating because the DMT diamond sharpening stone is too small for sharpening his ice skater. Well, the product description is accurate, so the reviewer simply made the mistake of buying the wrong size. This star-rating is particularly unreliable when the total number of reviews is small. If you have 3 people rate a product, then a single 1-star rating can destroy the average rating. However, once you got about 30+ people rate the product than the Star Rating system works reasonably well.

                    I also value reviews which are based on long ownership. It is one thing to read, “I got this product yesterday in the mail and it is just great” and it is another thing “I have been using this for over 3 years and I have no complaint.” Finally, you can read the review and you can tell if the person is being very objective and knows a lot or being passionate but does not understand the product.

                    As you pointed out, Amazon rating is based on average population and average buyers are not experts. Not being an elitist, but I don't think most people fully understand their products. On the other hand, people know when their products are failing them: when a waffle maker won't heat up or a toaster oven has uneven heat. As such, the negative reviews on Amazon.com can be much more helpful than positive reviews.

                    1. c
                      CocoaNut Feb 24, 2010 08:33 AM

                      Wisdom of the collective crowd - including CI, CR, epinions, amazon and here. I've known (very recently) for CR to dub something a "recommended buy" but the "crowd" nix it. There are the obvious shills.... There's the ridiculous: item given a 1* due to it's timer "beep" being inaudible, but the reviewer admitted a hearing problem. Then there are those "problems" OR "assets" that just plain don't matter to me - this happens often with restaurant reviews.

                      So, the number of stars mean nothing without a bit of review assessment.... then hope you have made the right decision.

                      1. Candy Feb 24, 2010 08:41 AM

                        I sell cookware, and Cooks Illustrated. We rely more on Amazon's group ratings. I rarely pay attention to CI. In think they are a bit off the wall and biased in their ratings and often wrong.

                        3 Replies
                        1. re: Candy
                          Chemicalkinetics Feb 24, 2010 09:37 AM

                          Candy,

                          What do you mean by biased? "Biased" refers the judgement is not objective and leans in a certain direction, so in what way does CI lean? Leaning toward well known brand? Or leaning toward foreign cookware? I am just curious as I don't have CI subscription.

                          1. re: Chemicalkinetics
                            c
                            CocoaNut Feb 24, 2010 09:50 AM

                            From being a watcher of their PBS programming, I don't see that they're really biased. I've seen them choose both a more expensive product and poo-poo the more expensive product. I've also seen them praise an expensive piece of equipment, but also add a "rider" for a less expensive product -- ALL-Clad is a primary example. They love it!, but when reviewing pots and pans they always offer a more budget friendly alternative.

                            Taking into account the mass review sites, oftentimes, CI seems to be in closer accord than CR..... IMHO......

                          2. re: Candy
                            p
                            pabboy Mar 2, 2010 05:47 AM

                            How is Cooks Illustrated biased? Please provide examples.

                          3. r
                            RichardM Feb 25, 2010 05:26 AM

                            Just my $0.02 but businesses like CI or rating cos. like CR start by establishing criteria for evaluation. As with polling questions, those can have an affect on the results. Also as pointed out, judging the fitness for purpose doesn't consider attributes like durability. But then nothing is made to last very long anymore.

                            With user feedback like amazon, the issue is veracity. It is easy for manufacturers or distributors to have employees post in their spare time. This gets more difficult as time goes on and number of reviews grows. I agree with tanuki soup and others that reading the actual comments of those that rated 2 - 4 sarts is often more useful than the 5 star ratings that reflect the 'honeymoon' period of a new purchase.

                            1. h
                              housewolf Feb 25, 2010 04:22 PM

                              Something you need to watch out for on Amazon is that the reviews are sometimes/often for a different model of the product than the one you are looking at. An older or newer model of the product or a smiliar/related product by the same manufacturer. For instance, I was looking at the Waring Pro Belgian Waffle maker on Amazon recently, with hundreds of reviews. Well, it turns out that Waring has been selling a new model of that particlar appliance for only a few months, and there were only a couple reviews that were actually for the new model.

                              No matter who the "Pro"s are, be it CI or Consumer Reports or any other rating agency, they only review a subset of any particular item so imho you always need to take them with a grain of salt.

                              The best advice anyone ever gave me was "make sure there is a good return policy".

                              1 Reply
                              1. re: housewolf
                                Politeness Feb 27, 2010 07:50 AM

                                housewolf: "Something you need to watch out for on Amazon is that the reviews are sometimes/often for a different model of the product than the one you are looking at."

                                We encountered a very good example of that (not of cookware, however) when we were looking for a memory card reader for a digital camera late last year. One reviewer at Amazon wrote of one card reader (the one that we ended up buying, in fact):

                                "The slot identification is nearly impossible to read. The slot ID is 3.5 point type and the totally black reader hard to see and I have excellent vision."

                                A second reviewer applauded the first's observation, stating:

                                "Therefore, concerns like "the different slots aren't well labeled" are relatively weighty. I have an earlier sandisk reader and agree that poor labeling is annoying. I know from experience where the SD cards go, and of course CF cards are obvious, but in the occasions when I have to get data from something more esoteric, it *is* an annoyance."

                                http://www.amazon.com/review/R3UWSUO3...

                                The problem: the model that was being "reviewed" has only one slot. On that model, there is no "slot identification."

                                Another reviewer of the same product wrote:

                                "one problem with this reader is not with speed but with the three-leg stand.
                                it is glued to the card reader. the metal leg came off after a couple of days of using it. i threw the stand away.".

                                There never was any glue: the connection between the card reader and the three-leg stand is by a bar magnet in the stand, the reader and stand are packed separately in the box, and it is the purchaser who puts the reader on the stand (without glue). A reviewer who actually had used the product surely would know that.

                                So among a dozen reviews of the product on Amazon (there were fewer at the time we were making our purchase), there are (at least) two that are by reviewers who know nothing of the product. But a reader who has not used the product himself or herself would not know that the reviewers had proved their own ignorance of the item they were reviewing.

                              2. Zeldog Feb 26, 2010 08:02 PM

                                The main problem with ratings on Amazon or Yelp or similar sites is the reviews do not represent the opinions of everyone who used the products, but only those who are aware of the existence of the web site AND feel strongly enough about the product to take the time to post an opinion. I suspect this leads to an overestimation of negative opinions, since pissed off people are more likely to want to vent. As others have noted, the other problem is the opinions do not compare products but merely state whether the buyer was satisfied or not with a particular model.

                                That said, if a product's reviews are largely negative I would think twice before buying. Also if lots of reviews complained of the same problem. Other than that I don't have much faith in that sort of review.

                                3 Replies
                                1. re: Zeldog
                                  Chemicalkinetics Feb 26, 2010 08:17 PM

                                  Agree. I have read a review for a waffle maker and most of the them are positive, but there are also a fair amount of people complain about it. As it turns out, most of the complaints described the same thing: fused got blown. This leads me believe it is a real problem and not just a once in a blue moon thing.

                                  1. re: Zeldog
                                    q
                                    queencru Feb 27, 2010 03:59 AM

                                    When I look at the Amazon reviews, I am looking at a specific model. I want to know if that one has any problems. Sometimes a product can get generally good reviews but there will be one issue that is reported over and over again. If that's an issue that doesn't concern me, I might buy it anyway, but if it is an issue that will be a problem for me, I will look to other products. One thing Amazon does do is tell you other similar products you might be interested in, so sometimes you can do a good search of comparable items based on the one initial product.

                                    I've found the reviews to be pretty reliable overall. I've seen products come out that get 10-15 negative reviews within the first few days they're on the market, and usually that means the product isn't something I'd want to buy. Then there are other reviews that are 3-star that have lots of one stars from people who opened up the box, used the product, got confused, and wrote a review all within an hour. I tend to disregard those reviews.

                                    1. re: Zeldog
                                      m
                                      Mother of four Mar 2, 2010 11:17 AM

                                      Every time I buy something from Amazon, which is a lot, they send me an email to review the product. You don't have to find a web site to do it!
                                      I have been very impressed with the reviews, and for the most part find them very reliable.

                                    2. JonParker Feb 27, 2010 04:50 AM

                                      I once bought a coffee maker at Target because it was the right price, size, and looked good. After having it die after a week, getting it replaced, and having that one die after a weekend, I went to Amazon and looked at the reviews. They were uniformly negative because of the same issue. I checked the highest rated coffee makers, bought that one, and it made great coffee for four years.

                                      Since then I check Amazon for nearly any purchase, whether I'm buying it from them or not. It is important to read them, not just because you can become aware of potential problems, but because sometimes a product will get negative reviews over the inclusion or omission of a feature that's just not important to me. If everyone thinks a toaster oven is awful because it doesn't have an incinerate setting I can use my own judgment as to whether or not that's a dealbreaker for me.

                                      1. m
                                        munchiesblog Feb 27, 2010 04:59 AM

                                        I think that Amazon can be really useful but if you are going to trust the reviews you should first see how many there are. I'd say if there are less than 5 at least see if you can get some reviews from another site before buying. Otherwise they are totally helpful and for the most part trustworthy

                                        1. al b. darned Feb 27, 2010 09:13 AM

                                          As others have said, I read the Amazon reviews. Sometimes even the 5* ones have a clue as to a deal breaker.

                                          A case in point: I was in the market for a decent toaster oven. One reviewer of the Cuisinart model I was considering noted it was easy to clean the crumb tray by simply pulling the tray out of the back of the unit. Every other unit I ever owned or considered had the tray accessible from the front. While not a big deal to some, this was a major negative for me. Others noted the Cuisinart didn't shut off, but merely "dinged" at the end of the timing cycle. Unacceptable to me. I ended up with a Krups and have been happy with it for 3 1/2 years.

                                          1. d
                                            deep Mar 1, 2010 07:17 PM

                                            Just one thing that I don't think anyone here has mentioned: some amazon reviews are faked.

                                            It doesn't happen a lot, but since a lot of cookware seems to have a lower number of ratings than things like electronics it is something to watch out for. You can usually tell if someone is a shill for a company by looking at their other reviews. If they review nothing but products of a certain brand and do so with across the board five star well written raving reviews then they are probably being paid. Also, if one person has bought a large number of one item (such as multiple high end espresso machines) then that should also warn you that this person might be getting paid.

                                            I do think that amazon reviews can be useful in some instances, but be careful of putting too much stock into one persons review.

                                            2 Replies
                                            1. re: deep
                                              h
                                              hobbess Mar 2, 2010 11:04 PM

                                              I've noticed that for some product reviews on Amazon, the reviews are really divided- its a bimodal distribution where the reviewerss either really love it or really hate it. The review rating will be either one star or five star, with almost little feedback that gives it a two star, three star, or four star rating.

                                              I've always been curious why you'd see that distibution of review ratings. If people were divided about a product, I'd think you would see more of a Gausian(?) distribution.

                                              But, now, I wonder if the faked amazon reviews were responsible for those divergent ratings- where the company had to step in to salvage the brutal feedback on Amazon.

                                              1. re: hobbess
                                                hobbybaker Mar 3, 2010 09:45 AM

                                                Hi, hobbess. I like the voting system (helpful/not helpful) and the comment system for the reviews on Amazon. Also, "Verified Purchase" signs showing if those who actually bought the reviewed product from amazon or not is helpful to me. (Ofcourse, you cannot tell if the person bought the product at other places than Amazon or not. And we somehow feel if the person really used it or not before writing the review.) As you and deep have pointed out, there are sometimes strange or looks-like fake reviews there. However, I always notice that those reviews often get many "not helpful vote" and pushed down towards the bottom of all the reviews. I like the way people can prioritize and even avoid those strange reviews to save our time. One of the reviews on All-Clad 12 inch Fry-pan with lid was just like that. His review was very odd to me and many others. The way, his review was at the bottom based on the voting.

                                            Show Hidden Posts