The Problem With Yelp

Yelp reviews get a huge amount of mostly deserved criticism when people who don’t have a clue about a cuisine leave idiotic posts, or even deride restaurants for not being open late enough, not having off-street parking, or, in one review, having an illegally parked car towed out of their lot. One has to wonder how much these reviews affect the bottom line of a business. A study at UC Berkeley set out to answer that very question.

…limited empirical evidence links digital word-of-mouth to purchasing decisions. We implement a regression discontinuity design to estimate the effect of positive ratings on restaurant reservation availability. An extra half-star rating causes restaurants to sell out 19 percentage points (49%) more frequently, with larger impacts when alternate information is more scarce. These returns suggest that restaurateurs face incentives to leave fake reviews, but a rich set of robustness checks confirm that restaurants do not manipulate ratings in a confounding, discontinuous manner.

…We recover the true underlying average rating for each restaurant and use this measure to implement a regression discontinuity (RD) design. We match the Yelp rating data to a database of restaurant reservation availability and estimate the impact of crossing each 0.5-star threshold on reservation availability. Our results indicate that Yelp ratings have substantial effects on restaurant customer flows. These impacts appear largest for restaurants for which there is less information on quality available outside of Yelp.

For a new restaurant this effect can be huge. is one of the most trafficked websites on the internet, and without a large number of reviews to offset people with an obvious ax to grind, the site could make the difference between success or failure.

In related news, the NY Times reports that Yelp is running sting operations to catch businesses that are hiring people to write good reviews.

 The first eight businesses — including a moving company, two repair shops and a concern that organizes treasure hunts — will find themselves exposed…

For the next three months, their Yelp profile pages will feature a “consumer alert” that says: “We caught someone red-handed trying to buy reviews for this business.”

Potential customers will see the incriminating e-mails trying to hire a reviewer.

NPR also weighs in on the problem –

Online reviews of restaurants, travel deals, apps and just about anything you want to buy have become a powerful driver of consumer behavior. Unsurprisingly, they have also created a powerful incentive to cheat.

As readers of online reviews intuitively know, merchants and authors post glowing reviews of their own products, and harsh reviews of their competitors, all the while pretending to be authentic customers. Now and then, fake reviews come to light, but most of the time, we don’t know if that five-star restaurant review was posted by your neighbor down the block or the chef at the restaurant — writing under a pseudonym.

That’s because many fakers have things down to a science. “What you’re trying to do is to be indistinguishable from a real review,” said Dina Mayzlin, a marketing professor at the University of Southern California. “So it’ll be, by definition, very hard to tell the good fakes [apart] from the real reviews.”

The Federal Trade Commission has tried to crack down on fake reviews by imposing fines and penalties, but the incentive for cheating — especially when combined with the low odds of getting caught — remains high.

I’m not trying to say that Yelp is a cesspool of cheats and idiots. When I’m traveling, the first place I look for a restaurant is on Yelp. I find that businesses with a large aggregate of reviews are a fairly reliable indicator of quality.

NPR passes along some tips:

  1. Compare reviews not only within a site, but across different websites.
  2. Reviews by people who are verified by the site are more trustworthy than reviews by anonymous reviewers — especially when it comes to negative reviews.
  3. Read reviews less for whether they give a hotel or a restaurant one star or five stars, but more for the specific information they give about the experience.
  4. Reviews are very useful for information that experts or merchants might not think to provide — how late a swimming pool stays open could be useful if you are traveling with a family.
  5. Focus on aggregates, not outliers. You can’t trust a handful of bad reviews or glowing reviews, but trends are much harder to fake.

This is fine and good, but it won’t filter out reviews that are plain ignorant. I think I’ll start a section on stupid Yelp reviews. Feel free to pass some along and I’ll post them.

I’m a huge fan of “Real Actors Read Yelp Reviews”. Here’s are my current favorites for your viewing pleasure:

Your thoughts are welcome

  1. Michele says

    Your columns and postings are always great. Thanks for taking the time to do this work. On this topic, as someone who looks to Yelp as my first (but not only) stop for info on restaurants and other places when I’m traveling, I enjoyed this information. As someone who regularly writes Yelp reviews to try to share with others information and tips they might find useful before they visit a place, I appreciate this information. The NPR itemized tips are useful but also I’m surprised that folks can’t fiture that out for themselves when reading Yelp or any other review site? And once one uses Yelp for a bit, it is pretty easy to spot the fakes, even the good ones. It is also easy to learn to weight reviews by the track record of the Yeep doing the reviewing. I have a Yelp friend who had a great sign on her Yelp profile, “I think before I Yelp”. Those are the folks to pay attention to when letting Yelp reviews guide “where to go” decisions and it is relatively easy when one uses Yelp a good deal to figure out who those reviewers are. Thanks again for your work to create some of the best reviews and food news reporting in Portland media!

  2. says

    As a professional Concierge as an established Portland Hotel, my guests frequently ask directions to an obscure restaurant…A light always goes on in my head, and I always ask if they are meeting someone there or how they found out about that restaurant…more times that not they say…”Yelp says it awesome…” I discreetly support their decision but give them some other options with proven success records in my book. There is a fine line, but most often I do not agree with Yelp’s reviews. We Portlanders are saavy diners but I have seen yelp divulge some poor information which relflects poorly on PDX to our visitors. We are so lucky here in the “land-of plenty” to have such amazing, locally owned restaurants, and I take pride in making sure my guests have the best cuilinary (adventure)experience possible.

    I will say my guests all arrive in PDX extremely Tech Saavy, and already have a quiver of spots they want to dine at and often ask me for the inside line… Most often they have a copy of the NY Times, Bon Appetit, etc…I love it!!! Thank you whole heartedly Food Dude for working your mojo…I am in your corner always!!!

    Great article.

  3. says

    Hi, Food Dude!

    Thank you for writing your 2 cents’ worth about ‘crank’ reviews; it was an insightful read. I’ve been in the service industry for near on 20 years as waiter, ‘tender, busser, barback, coffee jerk (I pre-date the ‘barrista’ moniker), etc. Having said that, I don’t even think that that makes me necessarily fully qualified to review restos and the like. I do anyway. I also use Yelp to find new spots of interest, however, not exclusively, and, I hope, I’m savvy enough to read through the rubbish that sometimes is posted on the site. Actually, I would hope that most of us are!

    The ‘Real Actors…’ clips are hilarious. Thank you so much for posting them. I’ve just watched them all. #11 made me cry with laughter. If you were to post a section on ‘stupid Yelp reviews’, I’d be ‘all eyes’, as it were.

    Cheers and thanks, again!

Leave a Reply

Your email address will not be published. Required fields are marked *