5 Reputation Fallacies (And How to Avoid Them)

Designing a reputation system is hard. Do it right, and you’re likely to draw from disciplines as disparate as computer science, sociology, user experience design and behavioral economics. Do it wrong, and you could wreak horrible downstream effects on the morale, motivations and mindset of your community.

We often design web reputation systems by drawing from easily-available examples on the Web, but this is an impoverished approach; it often leads the designer to settle on a design solution before they’ve properly understood the context, and framed the design problem. This is cargo-cult design at its worst, and leads to such “common sense” fallacies as…

• “It’s The People, Dummy!”

When you think of reputation on the Web, do you just assume that it applies chiefly to people? This is only partially true. Reputation applies to things as well, and—in fact—it’s almost impossible to know a person’s reputation without evaluating the reputation of things.

• “One Reputation To Rule Them All”

Do you think that someone’s Ebay Seller Reputation should follow them onto Facebook? Should Slashdot karma matter over on Reddit? Some very smart people have fallen victim to this fallacy: the belief that one reputation is enough to accurately convey “the measure of a man.”

• “All I Need is Five Stars”

Ratings input mechanisms fall into, and out of, vogue. Once it was 5-Stars, then it was Digg-style upvoting, now Facebook’s ‘Like’ holds the crown. If you’re tempted to start with a ratings scheme in mind, and then back-design a system to justify it, proceed with caution. (Includes a bonus fallacy: “Of course I need a down-vote!”)

• “Competition is Always Good”

Feeling inspired by game-like elements? Levels, points and leaderboards? Just make sure that they influence the right kind of behavior, and don’t assume a level of community competition that may, or may not, be appropriate.

• “Negative Karma Will Out the Bad Guys”

What’s the best way to identify the bad actors in your community? Why, label them, of course! Wrong—you should employ negative karma sparingly, and display it almost never.

The authors of Building Web Reputation Systems (O’Reilly, 2010) will debunk these fallacies, and a couple of others besides, drawing on real-life examples of actual deployed reputation systems from some of our industry’s biggest names: EA, Google, Yahoo! and Ebay. You might be surprised at how pervasive, and persistent, these fallacies have been through the years. But don’t worry—we’ll also tell you how to avoid falling into the same old traps!