Out with the old and in with the new — that’s what a study comparing Consumer Reports to Decide.com concludes.

PricewaterhouseCoopers/AEV conducted research on the two recommendation agencies and found Consumer Reports several steps behind Decide.com in various areas like product availability and an up-to-date database. The study, which was sponsored by Seattle-based Decide, analyzed 100 products in an eight-day span last month and stacked them against each other.

Recommendations differed 31 percent of the time. One of the more notable statistics was that 12.5 percent of recommendations shown as “Best Buy,” or “Recommended,” by Consumer Reporters were incorrect based on newer versions of the item with stronger features or lower prices available at other online outlets.

A Decide.com product recommendation.

“Decide is to Consumer Reports what Yelp is to Zagat: an example of technology disrupting older ways of providing consumers advice,” said Mike Fridgen, Decide CEO, in a press release. “Our technology provides product recommendations in real-time by using big data, millions of reviews from across the Internet, in addition to experts.”

There were also vast differences in user reviews and pricing decisions. Decide features 30 times more user reviews that are updated in real time and factored in recommendations. On the other hand, Consumer Reports allows only subscriber reviews, which aren’t calculated into the recommendation.

The only category that Consumer Reports matched Decide on was in the “Comprehensive” section.

The report reminds us of last week’s Redfin study that showed how its competitors like Zillow and Trulia have outdated database information. With how fast technology moves today, providing the newest information available at all times is certainly becoming more and more crucial.

Decide.com was co-founded by University of Washington computer scientist Oren Etzioni, and it is backed by Maveron, Madrona and others.

Previously on GeekWire: Consumer Reports for geeks: New ‘Decide Score’ uncovers the best and worst products

Comments

  • Guest II

    Um, Consumer Reports has their own testing facilities and directly tests products in a scientific and unbiased manner. Does Decide.com do that? Until they do there is no comparison.

    • Guest

      Exactly.It’s like saying McDonalds is better than El Gauchos because McDonalds is cheaper, has more locations and more menu items. It’s a true statement, but it ignores the main selling point of the “inferior” establishment. Consumer Reports is useful because it provides a thorough, unbiased review done by experts. You can get user reviews and current pricing anywhere.

    • jamiesteven

      I’d argue that Decide’s aggregation of expert and user reviews make their ratings more useful than Consumer Reports. Expert reviewers go through many (if not more in some cases) tests that Consumer Reports does. CNET’s TV reviews are more comprehensive than CR, in my opinion.

    • cksasaki

      Seriously. While judging things on subjective factors Consumer Reports sometimes lacks, their other tests for safety, durability, and utility will go beyond a simple user rating. Will Decide.com track reliability ratings for manufacturers over multiple years?

      While I’m probably not going to use Consumer Reports for my next smart phone, I know I’ll be looking at their historic car reliability data for my next car.

      I’m also not convinced that their user ratings are any more reliable than the ratings found at Amazon or other sites. I wonder what some of those users are thinking…

  • Guest

    Congratulations to Decide! We have been boycotting Consumer Reports ever since they asked us for money to view their dot-org (!!) web site, so Decide is a better option for us.

    • Joe d’Coder

      Yeah, decide just wants our identification… much better. lol

  • http://www.facebook.com/cathy.sullivan.923 Cathy Sullivan

    The study was sponsored by Decide.com and was probably designed to make Decide.com appear better then Consumer Reports. If Decide.com gets their own testing lab and then produces a study like this, it will be more believable.

  • http://www.facebook.com/people/Scott-Milburn/1038096131 Scott Milburn

    I agree with the below comments that Consumer Reports provides a substantially different service – do Decide users or experts run a washing machine 1,000 times, or expose multiple paint samples to bright sun for a year, or ask more than 100,000 readers for input on the vacuum cleaners they bought in the last four years, or all the other things CR does in its testing?
    Not to mention, who sponsored this study? Decide? Oh, I can’t imagine there was ANY bias in what the study examined.

  • snowyegret

    This “study” of Decide, sponsored by Decide, does not constitute news. Please.

  • MikeFridgen

    Hi,
    this is Mike Fridgen, CEO of Decide.com. Consumer Reports pioneered the
    consumer advocacy movement and we are inspired to take it to the next level. Our
    company paid for the study conducted by PwC/AEV and we stand behind the
    results. You can access the methodology and results from here https://www.decide.com/decide-vs-consumer-reports Guest II, @Scott Milburn, @Cathy Sullivan Decide uses
    both expert (Consumer Reports, CNET, DP Reivew and many others) and millions of
    user reviews from across the web to build our unbiased and objective
    recommendations for consumers. We capture the insights you described and
    more. Check out some of the examples provided in the link above and see
    for yourself.

    • Guest

      Mike,

      This is Guest, the CEO of GeekWire’s comments section. You sponsored a study. The study concluded that you were superior to a competitor who did not sponsor that study. You “stand behind the results” as if that’s supposed to increase the legitimacy of the study. Were the study to have favored Consumer Reports, would you have posted it on your home page?

      In conclusion, please stop “standing behind” a partial survey and show me some independent (i.e., not sponsored) research that vouches for Decide.

      With love,
      Guest

  • http://www.facebook.com/ryanlburt Ryan Burt

    I agree not apples to apples comparison, but you have to admit great viral link bait!

  • Guest

    There’s merit to the apples/oranges points. One thing about CR is that they’re based on a high degree of trust and objectivity. They’re a non-profit shop (http://en.wikipedia.org/wiki/Consumers_Union) and they do independent testing.

    From what I can tell Decide.com has a different business model than that. They’re for profit and don’t have dedicated independent expert testing.

    Because CR is non-profit and Decide.com is for profit, I think part of the issue it a competitive study like this feels a bit unfair, like Decide.com is beating up CR in a mismatched fight. I don’t see CR dedicating resources to defending hard against this (and I hope they don’t as that takes resources away from the testing I prize with them).

    Competition is good. Certainly CR is lags on technology (I don’t use them for that for some of the reasons outlined) and I’d love them to step up their game there.

    But direct competitive marketing like this is aggressive in a way that can (and I think is) backfiring.

    I’d rather have seen decide.com differentiate in a non-confrontational way or just focus on someone else (like Amazon and their ratings systems).

    CR has done us a LOT of good over the years. They’ve uncovered safety issues that have saved lives. That’s why they have so much trust banked up. I don’t want to see a world without CR and a tack like this makes me think that Decide.com is gunning to put them out of business. And so I’m going to vote with my self-interest here and stand behind the organization I know has helped me. And if that means I have to stand against whoever is attacking them, then I guess that’s where I stand.

    This is why aggressive marketing like this can backfire. You can force people to take sides when they hadn’t planned on it. Until this article, I had no opinion on decide.com. Now, I have a negative one.

    • MikeFridgen

      Hi Guest, I appreciate your thoughtful comment and your
      feedback. Consumer Reports is an
      inspiration to us and we share a similar mission to provide the most objective,
      trustworthy advice. I hope you will give us another shot.

      Mike
      CEO, Decide

      • GuestAgain

        Thanks I do appreciate the comment. Like I say, I would rather you save the head-on marketing for someone that can fight back, like Amazon.

        Thanks again.

  • http://twitter.com/etzioni Oren Etzioni

    Some of you are naturally skeptical of any “sponsored study”, but put yourself for a second in Decide’s shoes. We have a new product that, by analyzing millions of reviews algorithmically, produces far better recommendations than Consumer Reports. We needed to find a way to make that case—hence the study, which was careful and methodical. Why not try a few examples for yourself?

  • http://tac.is/here tacanderson

    I don’t think the issue here is if the report was paid for by Decide or not but the complete lack of journalism here. It should have been mentioned that this was a paid for piece of research, and if the reporter wanted real bonus points they’d point out that the head of marketing for Decide used to work at AEV, and what about some links to the research in the article? I got nothing against Decide, PWC/AEV, [waves to friends] or the commissioned research, but a little bit of journalism or some basic due diligence here would have been a good thing.

  • updater

    Ummm, consumer reports still has a website and a company, decide.com is part of eBay in some unclear way. Apparently, decide.com wasn’t that great after all. It’s called staying power to all you young ones.

Job Listings on GeekWork

Find more jobs on GeekWork. Employers, post a job here.