It’s easy to fall under the spell of Amazon. The all-powerful e-commerce behemoth provides an unparalleled customer experience and has been establishing itself as the one-stop everything store for over 20 years. A vast network of consumers test, rate, and provide feedback to help inform their peers’ decisions. When it comes to selecting a product, shoppers tend to trust that quality is written in the stars, so to speak.
But it turns out those ratings and reviews aren’t quite as bulletproof as Amazon’s return policy or delivery estimates. A new study from the University of Colorado Boulder found very little correlation between user ratings on Amazon and quality or resale value.
Researchers compared the ratings of 1,272 products across 120 categories, with consumer reports, which tend to evaluate items more objectively.
“The likelihood that an item with a higher user rating performs objectively better than an item with a lower user rating is only 57 percent,” Bart de Langhe, author of the study and CU-Boulder marketing professor, said in a press release. “A correspondence of 50 percent would be random, so user ratings provide very little insight about objective product performance.”
According to de Langhe, consumers are often influenced by brand reputation and price-signaling when reviewing a product. He also cautions shoppers not to make purchasing decisions based on the average ratings of a product without examining the sample size.
“Oftentimes, there are just not enough ratings for a product or there is too much disagreement among reviewers,” he said. “In this case, consumers should not trust the average very much, but they do nonetheless.”
The study did find a slightly stronger correlation between positive Amazon ratings and product quality when the sample size of reviews was larger.
The findings come amid Amazon’s ongoing legal campaign to protect the legitimacy of its reviews. Last week, the company filed suit against five sites selling fake product reviews. Last fall, Amazon sued 1,000 Fiverr users offering to write bogus reviews on the e-commerce platform in exchange for money.
Customer reviews, Amazon says, are an essential part of its business.
“It is important for customers to know that fake and misleading reviews remain a very small fraction of the overall number of reviews; in fact, we remove these reviews as soon as they are identified and we introduced a review ranking system so that the most recent, helpful reviews appear first,” said an Amazon spokeswoman. “The vast majority of reviews on Amazon are authentic, helping millions of customers make informed buying decisions every day.”
Establishing legitimacy in an age of user-generated content is a constant battle. As the University of Boulder study shows, even genuine reviews can offer opaque insights into the quality of a product. De Langhe suggests shoppers take consumer reviews with a grain of salt.
“Accurately evaluating product performance is not an easy task,” he said. “Different alternatives need to be evaluated side by side under the same conditions using objective measurement instruments. You can’t assume that people follow such a scientific approach before they rate products online.”
