The Limits of Spam Algorithms

By Deane Barker on April 27, 2012

Yelp, You Cost Me $2000 by Suppressing Genuine Reviews, Here’s How You Fix It: An interesting story of a false positive spam algorithm doing some damage.

Yelp flagged poor reviews of a moving company as spam, and hid them.  Turns out, they were legit – the moving company were not nice people – but since this guy never saw the reviews, he hired the company anyway.

Turns out, the behavior meant to indicate spam overlaid perfectly on the behavior of people trying to complain about this company.

Your algorithm typically hides entries by people who only post one review and who don’t otherwise engage in Yelp. Your assumption is that if a user only posts one review, posts no comments, has no friends etc. then most likely they are fake and trying to game the system.

[…] In each case the one star review was left by someone who would never normally leave a review… they were simply so outraged that they were motivated to signup to Yelp and try to warn others how bad this company is. None of them ever used Yelp again. Furthermore, they didn’t have the knowledge or inclination to try to make their Yelp profile look acceptable to Yelp’s automated suppression systems.