How reading user reviews and triaging software defects are similar

User reviews saved e-commerce — the power of the Internet buying masses, crowd-sourced into a win-win situation for both retailers and consumers (B2C!).

The above is a lie (and also hard to read, filled with terrible e-buzzwords).

User reviews are difficult to parse, give a false sense of security to the buyer, and are one of the many profit-extraction tools that retailers employ. How many times have you seen a 4 or more star review (out of 5) end up being a complete dud? How about a poorly review app on your phone actually being quite good (Amazon app store, I’m looking at you).

I’ve finally gotten to a point where I feel comfortable parsing the reviews and discerning (potentially) valuable information. I try to end up with one of the following (mind-blowing) generalizations:

1. This product is worth purchasing
2. This product is not worth purchasing
3. Not enough information is available to make a decision

Often the journey is the reward, not the destination. How do I parse all of the reviews, both good (in content, not score) and bad? I want to know what’s genuinely wrong with the product, and if I’ll be okay with it. I’ve come up with a few rules that help me determine this:

1. Score doesn’t matter
2. Read most or all the bad reviews (within reason)
3. Read a random sampling of average and good reviews
4. Discard the overly positive lovefests and the snarky, curmudgeonly negative reviews

So how does this relate to software defects? Defects have a stigma attached to them. They are often an unfair “1 star” review by the person reporting the issue. When prioritizing work, these sour items have the possibility to poison the choice of the work that gets done. Each defect needs to fit into one of the three categories, conspicuously similar to user reviews:

1. This defect is worth pursuing at this time
2. This defect is not worth pursuing at this time
3. Not enough information is known to make a decision; research and reevaluate until #1 or #2.

What technique and tidbits can you learn by reading the list of defects as if it was an anonymous user review?

1. The priority of the defect as determined by the submitter may not represent the true priority within the project
2. Read all the comments, and similar defects, and weigh together
3. Deflect the overly negative opinions of your software; a single defect can change the opinion of the software, but shouldn’t change the team’s motivation
4. Likewise, don’t get trapped only hearing the positive feedback. I’ve yet to meet a project that has zero defects (although I did have a manager once who claimed his last project didn’t have any bugs and that was the standard I was going to be held to)

Often times, a sign of great software are the requests for enhancements by the end users. At a glance, user reviews don’t give an accurate picture of what’s right or wrong with a product. Likewise, defects (and enhancement request) don’t always tell the story of where your project needs to focus its energies.

User review and software defects have their value when evaluated together. Also, if you liked this post, be sure to give it a 5 star rating. ;)