Lost your reviews? It’s probably best keep that to yourself.

Posted on 05. Sep, 2012 by in Thoughts



Fake reviews, also known as deceptive opinion spam, is a growing problem. There’s a huge temptation for businesses of all sizes and types to use sock puppet profiles to publish fake reviews for their products and services as well as those of their competitors.

Review Centre speaking from imagination(?)

Review Centre has a fake review problem but is not alone...

 

Recently, there has been considerable unrest over Google’s removal of customer reviews from certain business owners. Often the businesses claim to have worked hard to get these reviews and that it’s unfair of Google to remove them “arbitrarily” and without explanation.

So, is Google being unfair or simply protecting trust in the reviews it chooses to promote?

Over time, reviews have become a cesspit of spammy goings-on. It’s not uncommon for major retailers to ask for an SEO company’s price for fake reviews during a pitch process. Presumably the seedier companies are happy to carry out this practise.

Sites such as Trip Advisor, Review Centre, Amazon and Google are all targeted by spam reviews and are often accused of doing little to weed them out. Take a look at the top performers in competitive markets on Review Centre or at any of the high profile Trip Advisor articles and you’ll see what I mean. Are we really supposed to believe that small online retailers who have achieved 500 five star reviews with almost other reviews have done this purely through their “fantastic service!!!!”? Often sites get lots of perfect reviews then one perfectly bad one, followed by loads of perfects again. I encourage you to have a look for yourself, you’ll see what I mean.

Google filtering spam reviews

It’s easy to recognise suspicious patterns in reviews but it can be difficult to tell for certain which individual reviews are fake and which are real just by reading them. However, if I can tell where fakes are appearing from the pattern of posting, an algorithm should be able to do it much more effectively. Imagine it was possible to also identify fake reviews just from the text with an accuracy of over 90%? Google would be all over that, right?

Google has removed many reviews from the likes of Review Centre and others from it’s aggregated review pages. Additionally there are numerous reports of businesses losing their Google reviews.

So what is Google doing? There’s been no official announcement but it seems reasonable to assume that Google has introduced new filters to weed out fake or suspicious reviews.

Detecting deceptive opinion spam

A fair amount of research has gone into the automatic detection of deceptive opinion spam, including this paper, Finding Deceptive Opinion Spam by Any Stretch of the Imagination. It’s worth noting that the New York Times reported that Google had requested the writer, Myle Ott’s resume after publication.

The researchers asked a group to write 400 positive reviews of 20 Chicago hotels. These were compared to the same number of truthful reviews. The average person could distinguish fake from real at rates no better than chance.

The texts were analysed for features which might give away the fact that they are either fake or truthful. For example, the real reviews contained more words which related directly to the hotel such as “bathroom” or “check-in” whereas the fake reviews contained more text which was telling a story or setting a scene, words like “my huband” and “business trip” were used more often. Fake reviews used more verbs, exclamation marks and extreme language such as “amazing” and “incredible” whereas the real ones used more nouns and were liberal with the use of adjectives and exclamations.

This keyword analysis, combined with analysis of word pairs allowed the researchers to produce an algorithm which identified the fake reviews with an accuracy of 89.8%.

By adding in other signals such as timings (five reviews in a day?), number of reviews posted by the user and so on, it’s not inconceivable that a company such as Google should be able to identify fakes to well over 90% accuracy.

Presumably different types of reviews (products, services etc.) would require different signals or weightings but it doesn’t take much to figure out that Google most likely has a refined set of signals allowing its algorithms to identify fake reviews.

Undoubtedly Google’s deceptive opinion spam filter isn’t 100% accurate but I’d guess that if you’ve lost 80% or even 20% of your reviews, it might not be wise to shout too loudly about it.

A fun online tool is available based on the above research: http://reviewskeptic.com/. In my tests (which were restaurant reviews rather than hotels) it was only 66% accurate – it detected 12 of 35 genuine reviews as fake. Unfortunately I don’t have any hotel reviews which I can confirm as genuine in order to do a proper test.

Trip Advisor Fake Reviews

Image from http://tripadvisor-warning.com/

Tags: , ,