Would you like to be able to give your point score to each review, based on your experiences?
I’ve asked this question before, but this site has a lot more readers now. My question is, should reviews include a final score, or rating?
The positives:
Makes it easy to search for somewhere to eat, based on rating.
Makes it easy to tell the outcome at a glance, when you don’t want to read the entire review.
Generates lots of discussion and argument on the site.
Look! Stars!
Negatives:
Some people never read the review, and form their opinion based on the ratings alone.
Keeps me awake at night trying to decide on a rating.
I like the idea that a balanced review will allow the reader to form their own opinion.
Generates lots of discussion and argument on the site.
I am sure you could come up with more positive and negatives. What do you think? Is a 0 to 5 review using a .5 step increment a broad enough range? Feel free to leave comments and vote in the poll on the sidebar.
-s says
I thought about this issue when we first started up salemfood. Ultimately I decided that a system of quantitative rankings based on my own personal preferences wouldn’t work. You run into problems across cuisines (e.g. it would take an extraordinary Mexican meal to get to 4 stars because my standards are very high, while 4 star Chinese would be something that didn’t come back up later that night.) You run into problems across locations — Wong’s King is the best Chinese place in Portland so it should get 4 stars, but does it stack up regionally? Nationally? Worldwide? Does it matter? Should I do ice skating scoring and not give out marks too high just in case something else is better?
Defining the scale and applying it was far too much for me to worry about. I think a thorough narrative, combined with a less strictly quatitative scoring (“recommended,” “it’s not my taste but it’s good for what it is,” “run far away as fast as possible”) is ultimately a less taxing solution.
I do wish I knew what Paul Zimmerman’s meal-scoring system is, because it sounds like he has a strict system of weights and measures that get considered. See this article. He’s got scores like 8.78 and 8.775. One must wonder how it works.
Jeff says
Have you considered a Zagat-like “multi-dimensional” rating system, so you don’t feel like you are boiling a complicated, well-thought-out review to a single number? I don’t mean that you have to use the “food/service/decor” triplet, of course. Invent your own system.
Take the Carlyle review. You gave it 2 stars because you thought it was reaching for something higher than it was able to achieve, even though you would never argue that they are putting out “worse” food or are a “worse” restaurant than, say, Pok Pok, which gets 3.5 stars. I’ll bet you could come up with a way of expressing that intent in a numerical rating system.
witzend says
I think Jeff makes a valid point. Perhaps rather than a number or letter grade (ala the “O”) you could use the “bad-fair-good-excellent” rating in say three catagories; food, service, and ambiance. I think that ultimately this is far more informative. If I were basing a dining choice on a number, why would I feel compelled to choose anything below a three? However, if I knew that the food at a restaurant was deemed “excellent”, in light of the fact that the service was “fair” and the decor was “bad”, I may still opt to try it out.
pascal says
I think a lot of folks on this site know my views on reviews…
If one has to do a restaurant review, the rating system just doesn’t pan out. Many times I read reviews, in different publications, that just do not reflect the rating. FD, it seems that you are so tormented about those ratings… I really think that people actually READ your reviews and do not search per ratings. They like your writing, your views and your tastes. One will often don’t agree with you, but in the end your readers trust your judgement. So why rate?
I really believe that if someone write a review with respect for the chef or / and owner as well as the staff, giving pertinent and knowledgeable opinion about food / service / decor / wine list (often forgotten), then not only it would benefit the reader but the management as well who might appreciate the comments and fix what is not working well. No one would get pissed, everybody wins!
Aaron says
I vote for a multi-dimensional system… Several options:
Food Quality/Consistency
Menu Innovation
Plate Presentation
Service/Decor
Value
If assigning a final value still bugs you, don’t even bother averaging them up or giving an “Overall” score — let the rest of the review speak for itself.
Food Dude says
I like the idea of breaking it down into categories. Ambience, Service, and food do paint a pretty good picture. Anyone else have input?
pascal says
WINE LIST, Dude, WINE LIST!
Betsy says
No more than three categories – otherwise, you’re a cataloguer, not a reviewer.
I’d argue that much of the classifications listed above – menu innovation, plate presentation, etc. – should be captured in the review itself, and won’t *always* apply to every single review. So why hamstring yourself?
nancy says
If you’re going to catagorize, I agree with Betsy: three only, per what you mentioned: Service, Atmopshere, Food. And personally, I like stars, as they are simply facet of a review.
That said, the NY Times stars without catagories, and it seems to work, as the narrative is always enough to let you know what you’re in for, and the star(s) are balanced accordingly. Of course, they tend to review mid- to high-range restaurants, and address the Pok Poks of the world in separate (unstarred) reviews. More food for thought.
-s says
I think for broad categories, Service/Atmosphere/Food sums it up pretty well (well, and wine list, to make Pascal happy, though I would think it should be left out of an overall tally, as knocking down the score of a taqueria because of their wine list would be rather tacky.)
Aaron’s comment made me think of Iron Chef America, which has taste, originality, and plating as the judging criteria.
The problem with a specific ‘menu innovation’ metric is that you require the reviewer to be an expert in that type of food, which is an unreasonable expectation for someone who isn’t making any money off the venture. You may eat pibil at a Mexican place up here and think it’s the most innovative thing ever even though it’s not, it’s just not served much up here. And I think that the metric also sets an expectation that if a place isn’t doing duck foam or whatever the Gourmet magazine writers are beating to death that month, it’s not innovative.
An interesting compromise could be an ‘originality’ or ‘uniqueness’ metric. An example would be Siam Society — in an area with quite a number of pretty good Thai places, it’s both good and they serve dishes that go beyond the standard American Thai fare.
OR…you could build originality into the overall ‘food’ score, along with taste and presentation.
MyNextMeal says
For me the amount of $-$$$$ a restaurant wants me to spend is the first/primary indicator of what kind of experience/food I expect to enjoy.
For me, a rating system within a more “objective” category of cost would help address the divide between the POK-POKs vs Higgins.
It would make sense to identify the 4-star dining places within the $ range. And the 4-star dining places within the $$$$ range.
onetontemplebell says
My personal preference would be for a multi dimensional rating system. Although I would rate a restaurant on Food, Atmosphere, Service, and Price. I like good food but many of my favorite restaurants and bars are my favorite because of atmosphere rather than mind blowing food. For example I enjoy the occasional trip to Montague at 1 in the morning with a group of 10 people and I would not rate that place very high on a food or service scale.
If you don’t want to rate it like that then just toss them all together. I honesty just scan the reviews, ignore the rating, and pay attention to comments. The well known Portland unevenness leads to two people who are equally excited about food coming away with radically different experiences at even the best places. I personally had horrible service at Carlye and mediocre food for the price but I know people whose opinion I highly respect who think it is the second coming of the foodie messiah. Given this unevenness I take the reviews and comments in aggregate and decide if I want to go somewhere.
Pam says
I agree with the 3 categoriers. Any more than that would get unwieldy, and it’s too difficult to boil down a multi-dimensional experience into just one category. I’d also dispense with a wine list rating, and just leave that as part of the review. I think the main question is how well does a restaurant achieve what it’s set out to do? If looked at that way, then both Pok Pok and Carlyle can be assessed within the same system.
Food Dude says
The problem with reviewing, is everything is so subjective. Is the definition of great ambiance a beautiful modern interior or a minimilist asian motif… or even a bright comfortable space with lots of windows. Some people prefer a patio over anything else.
You could pretty much say that about all 3 criteria. I’ve been to dinner with friends, who complained service was too slow, when I thought it was fine, and so forth. It’s a tough question.
What do you all think of breaking each score down into the three components, and then averaging them, showing all scores on the review?
Mateu says
Since the “pro-stars” crowd seems to be winning, I’ll speak up against them. -s makes a great point about the problem of reviewing the beverage menu, so I will go one further and argue that the relevant categories depend on the place. As onetontemplebell points out, Montage can seem like a 5 star idea at 2am, but a 2 star idea for lunch. There’s a pair of hole-in-the-wall taco shops in San Diego I’ll visit every chance I get; does that make them 5 stars? I’m a vegetarian; should I rate a great restaurant badly because they only have 1 veg dish?
If you want to be cited, sure, add the stars. If you want to be read, don’t.
adeu,
Mateu
Mateu says
Oh and by the way, on Firefox on Linux, the little voting buttons appear, but no text appears next to them. So I’m a-scared to vote!
Food Dude says
Frank Bruni, restaurant critic for the NY Times did an article on restaurant ratings last week. You can read it here (free membership required). I find myself pretty much agreeing with what he said.
Readers frequently ask me why I “havent reviewed xxxx”. Much of the time, I’ve been to the restaurant, but haven’t been happy with my meal, and rather than write a bad review that they can ill afford, I just pass it up. Exceptions are: chains, restaurants spending lots of money on advertising, restaurants opening with a good deal of PR/splash. To me, they are fair game.
I originally modeled my ratings on the NY Times scale, but made a few changes, as it didn’t make sense. It reads:
0 – poor-satisfactory
1 – good
2 – very good
3 – excellent
4 – extraordinary
Frankly, I find it confusing. In my mind, poor and satisfactory are too very different things. Very good and excellent are really subjective.
Mine is:
0 – poor – pass it by
1 – fair – you could do better
2 – average
3 – very good
4 – superior
mczlaw says
FD:
As you and I have discussed, I believe in a 1-10 scale system to give maximum stratification to the grades. Ideally, though always imperfectly, this reduces the chances that places of differing quality will end up with the same score.
On my scale, 0=unspeakably horrible in every respect; 5=average; and 10=extraordinary in every respect. The wealth of numbers between give the reviewer lots of flexibility to pair underlying analysis with the ultimate rating.
–mcz
nate says
I’m going to go against the grain here a bit and argue for a more simple rating system. Three ratings: Good, Bad, Ambivalent (may want to choose a different name). This allows you to go with your gut feeling and saves you some sleepless nights (hopefully). If you liked it and look forward to your next visit, it’s good. If you may be back, but are in no rush, or only under certain circumstances (for a particular dish or setting) then it’s ambivalent, and if you never intend to return, it’s bad.
As Pascal notes, most folks here probably read your entire review (or at the very least skim). I think the rating serves more to rule out real bombs or call out real gems. Any in-depth understanding of a place would require reading the full review. In the parallel world of film reviews, I think one of the reasons Siskel & Ebert (now Ebert & Roeper) became so popular was because their system was so simple. If a movie got two thumbs up, there was a pretty good chance it was at least decent; if it got two thumbs down, it was probably pretty poor; and if it was a split, well, your mileage may vary. Furthermore, the ratings never told you enough on their own to sell you completely one way or the other. Hence, you had to actually watch/listen/read their reviews to actually know anything about the film. This encouraged viewers/readers to gather more information before making their decision and, especially in the early days when the quality of reviews was better (IMHO), you could get a pretty good sense as to whether or not you wanted to see the film.
Sorry for the digression, but you can count me in favor of a simple rating system that encourages further exploration by the reader.