The Problem With Yelp

Yelp reviews get a huge amount of mostly deserved criticism when people who don’t have a clue about a cuisine leave idiotic posts, or even deride restaurants for not being open late enough, not having off-street parking, or, in one review, having an illegally parked car towed out of their lot. One has to wonder how much these reviews affect the bottom line of a business. A study at UC Berkeley set out to answer that very question.

…limited empirical evidence links digital word-of-mouth to purchasing decisions. We implement a regression discontinuity design to estimate the effect of positive Yelp.com ratings on restaurant reservation availability. An extra half-star rating causes restaurants to sell out 19 percentage points (49%) more frequently, with larger impacts when alternate information is more scarce. These returns suggest that restaurateurs face incentives to leave fake reviews, but a rich set of robustness checks confirm that restaurants do not manipulate ratings in a confounding, discontinuous manner.

…We recover the true underlying average rating for each restaurant and use this measure to implement a regression discontinuity (RD) design. We match the Yelp rating data to a database of restaurant reservation availability and estimate the impact of crossing each 0.5-star threshold on reservation availability. Our results indicate that Yelp ratings have substantial effects on restaurant customer flows. These impacts appear largest for restaurants for which there is less information on quality available outside of Yelp.

For a new restaurant this effect can be huge. Yelp.com is one of the most trafficked websites on the internet, and without a large number of reviews to offset people with an obvious ax to grind, the site could make the difference between success or failure.

In related news, the NY Times reports that Yelp is running sting operations to catch businesses that are hiring people to write good reviews.

 The first eight businesses — including a moving company, two repair shops and a concern that organizes treasure hunts — will find themselves exposed…

For the next three months, their Yelp profile pages will feature a “consumer alert” that says: “We caught someone red-handed trying to buy reviews for this business.”

Potential customers will see the incriminating e-mails trying to hire a reviewer.

NPR also weighs in on the problem –

Online reviews of restaurants, travel deals, apps and just about anything you want to buy have become a powerful driver of consumer behavior. Unsurprisingly, they have also created a powerful incentive to cheat.

As readers of online reviews intuitively know, merchants and authors post glowing reviews of their own products, and harsh reviews of their competitors, all the while pretending to be authentic customers. Now and then, fake reviews come to light, but most of the time, we don’t know if that five-star restaurant review was posted by your neighbor down the block or the chef at the restaurant — writing under a pseudonym.

That’s because many fakers have things down to a science. “What you’re trying to do is to be indistinguishable from a real review,” said Dina Mayzlin, a marketing professor at the University of Southern California. “So it’ll be, by definition, very hard to tell the good fakes [apart] from the real reviews.”

The Federal Trade Commission has tried to crack down on fake reviews by imposing fines and penalties, but the incentive for cheating — especially when combined with the low odds of getting caught — remains high.

I’m not trying to say that Yelp is a cesspool of cheats and idiots. When I’m traveling, the first place I look for a restaurant is on Yelp. I find that businesses with a large aggregate of reviews are a fairly reliable indicator of quality.

NPR passes along some tips:

1. Compare reviews not only within a site, but across different websites.
2. Reviews by people who are verified by the site are more trustworthy than reviews by anonymous reviewers — especially when it comes to negative reviews.
3. Read reviews less for whether they give a hotel or a restaurant one star or five stars, but more for the specific information they give about the experience.
4. Reviews are very useful for information that experts or merchants might not think to provide — how late a swimming pool stays open could be useful if you are traveling with a family.
5. Focus on aggregates, not outliers. You can’t trust a handful of bad reviews or glowing reviews, but trends are much harder to fake.

This is fine and good, but it won’t filter out reviews that are plain ignorant. I think I’ll start a section on stupid Yelp reviews. Feel free to pass some along and I’ll post them.

I’m a huge fan of “Real Actors Read Yelp Reviews”. Here’s are my current favorites for your viewing pleasure: