What do orange stars mean on Yelp

We are looking for the best Italian in town. Today's hungry people no longer rely solely on the judgment of friends or restaurant critics - you take a look at the relevant rating platforms on the Internet. With more than 500 hits, the first-placed restaurant gets the full number of points, as expected, but after that it gets confusing: The restaurant in sixth place has a better overall result than the fifth. And the sad place 521, almost at the bottom of the ranking, is taken by a restaurant with full points.

It is definitely an achievement that today everyone can find out from anywhere how crispy a pizza is, how family-friendly an employer is, or how comfortable a hotel bed is. Online rating portals help with consumption or even life decisions, but they can also confuse and sometimes even deceive users.

On Tuesday, the Federal Court of Justice in Karlsruhe announced a judgment in a case in which it could have gone that way. The review portal Yelp only includes user ratings that an algorithm considers helpful in the overall rating. He sorts out others as "not recommended", which can distort the overall grade - and against which a fitness studio operator complains. Similar to the not entirely trivial search for the best Italian, the Karlsruhe process also poses questions to consumers: What is behind an online rating? And to what extent can you trust her? A little guide.

Do all the reviews make it onto the page?

"It was ok, but not great." Such short and limited meaningful reviews can be found on Google, for example. Tripadvisor reviews, on the other hand, must be at least 200 characters long. Swearwords are taboo for many, the Kununu platform, which users can use to rate their employer, also ensures that no names are mentioned and that a review does not specifically identify a person.

Before a review is activated, many portals check the text in order to keep the number of manipulated reviews low. In doing so, they proceed differently, as an investigation by the "market watchdogs" at the consumer center showed almost two years ago. Some use algorithms, have suspicious reviews checked by human employees and their users report strange content. But not all portals tread this "ideal road", according to the study. For some, the sheer number of reviews has top priority. What is certain is that some reviews are sorted out from the outset by many providers. The portals are relatively secretive when it comes to when and according to which criteria their test algorithms and employees are sorted out. Revealing this could make their craft easier for fraudsters, according to Tripadvisor.

How is the overall grade created?

A survey among a number of large platforms shows that the overall rating usually corresponds to the mean value of all reviews submitted and approved. The grading scale of course differs depending on the platform - sometimes five circles are top, sometimes there are six suns.

Does a very good rating also mean a place at the top of the ranking?

Not necessarily. It also happens that a restaurant with a better overall rating is behind a restaurant with a lower rating in the ranking list. "In addition to the quality of the reviews, the topicality and quantity also flow into the ranking," says Tripadvisor. A restaurant with very good, but many old or only a few test results slips down, especially in a large city with many bars. With Holidaycheck, reviews that are older than two years are still visible, but no longer have any influence on the ranking.

How does the user know how the overall grade and the ranking are created?

It's not that easy to find out. When users write a review, they often receive information about what is allowed and what is not and what a meaningful review can look like. But how the overall grade and the ranking list come about is often difficult or impossible to see. Florian Stößel, legal expert at the Federation of German Consumer Organizations, criticizes this. He calls for a concrete legal framework that specifies how an evaluation and ranking are made. In addition, the portals should point this out more transparently, he thinks.

But even if users know how an overall rating was calculated, they can never be entirely sure whether individual reviews have been manipulated or bought. There are indeed indications that a review is not authentic - for example, many test results from restaurants and hotels in several locations within a short period of time. Of course, these clues also know the portals' algorithms - and they can still miss out on fake reviews. But it doesn't have to be a contradiction in terms to use platforms to orientate oneself when making a purchase decision and still remain healthy and suspicious

Correction: An earlier version of the article stated that Tripadvisor reviews must be at least 200 words long. That is not correct, they must be 200 characters long. We have corrected the error in the text.