The cult of the one-★ review
Playing the game
Like any many fixed systems, the supposedly fair system of user reviews is horribly broken. The process was ripe for gaming. And gamed it has been. Companies optimize their apps and websites to get you to leave ★★★★★ reviews. EA, with their Dungeon Keeper on mobile platforms has optimized this to the extreme:
Clicking “5 Stars” take you to the store to give that ★★★★★ rating. While clicking “1-4 Stars”…
Emails, as far as I know, don’t show up as app store reviews. Instead, you get ushered away from app store to some back room where no one knows what was said. That, in theory, leads to more ★★★★★ reviews and a higher score. EA’s tactic may have gone unnoticed if it wasn’t for other issues with the game.
“Beating” the system
Less than satisfied users on the other hand, in response to all the ★★★★★ reviews, or perhaps just in anger and spite, leave ★ reviews and scathing comments. “I’d give it no stars if that was possible!” raves one review. “Doesn’t support feature X, thus ★.” says another.
In response to EA, the masses gathered to “correct” the injustice: an unfair playing field. Their response, ★ reviews. This behavior is called Griefing.
The test
Go to Amazon, or Best Buy, or the App Store or Google Play and read a few reviews. Play close attention to the ★ reviews.
Results
What did you find? I went to the Apple App Store (which is hard to navigate from a browser, but I digress) and looked at the first app in the first category: iBooks.
Here are the three reviews I saw:
Pay special attention to the middle review. Other than a thirst for violence, what sticks out? To me, it’s the desire to get a single thing changed (the new UI design to match iOS 7’s massive overhaul). So does this ruin the experience? I don’t know, I’m not the review’s author, shr00mie. Only shr00mie knows how shr00mie feels. And if we take shr00mie at his or her word, the app is ruined. Reading books is ruined.
The review has an agenda. The agenda is to get something changed. And the way the user has decided is to attempt griefing with a ★ review. This behavior exists anywhere a user can leave a review. Let’s check the Google Play store. Maybe it’s just Apple zealots, right?
This comes from the first app the store shows, NCAA® March Madness (so glad they got that ® mark in there. I was totally going to steal their mark!)
This was the first review of the app. In a way, the reviewer is correct, if the user only installed the app to watch women’s basketball, and it only shows men’s basketball, then it’s useless, and worthy of a ★ review. But the app’s description states that allows “unlimited access to live streaming video of all 67 games of the 2014 NCAA® Division I Men’s Basketball Tournament”. I’ll let you be the judge.
How I handle user reviews
Honestly, I’ve had my fair share of “does not support move to SD” reviews (on Android, the ability to run an app from an SD Card vs. internal memory). Does the app deserve a ★ review because you have too many apps installed on your phone?
I can not longer trust review scores. Between the “I ran this app for 2 minutes and it looks awesome ★★★★★” and the obvious attempts at review griefing, the signal to noise ratio is much to low for me.