« October 2009 | Main | December 2009 »

November 18, 2009

Reputation is Identity

Reputation Wednesday is an ongoing series of essays about reputation-related matters. This week's entry discusses the ways that reputation can make for richer user identities on your site. It is lightly adapted from our draft of Chapter 8.

Imagine you're at a party, and your friend Ted wants you to meet his friend, Mary. He might very well say something like… "I want you to meet my friend, Mary. She's the brunette over by the buffet line." A fine, beginning, to be sure. It helps to know who you're dealing with. But now imagine that Ted ended there as well. He doesn't take you by the arm, walk you over to Mary, and introduce you face to face. Maybe he walks off to get another drink. Um… this does not bode well for your new friendship with Mary.

Sadly, until fairly recently, this has been the state of identity on much of the Web. When people were represented at all, they were often nothing more than a meager collection of sparse data elements: a username; maybe an avatar; just enough identifying characteristics that you might recognize them again later, but not much else.

With the advent of social on the web, things have improved. Perhaps the biggest improvement has been that now people's relationships formulate a sizable component of their identity and presence on most sites. Now, mutual friends or acquaintances can act as a natural entree to forming new relationships. So at least Ted now will go that extra step and walk you over to that buffet table for a proper introduction.

But, you still won't know much about Mary, will you? Once introductions are out of the way, what will you possibly have to talk about? The addition of reputation to your site will provide that much-needed final dimension to your users' identities: depth. Wouldn't it be nice to review a truly rich and deep view of Mary's identity on your site before deciding what you and she will or won't have in common?

Here are but a few reasons why user identities on your site will be stronger with reputation than they would be without.

  • Reputation is based on history and the simple act of recording those histories – a user's past actions, or voting history, or the history of their relationship to the site – provides you with a lot of content (and context) that you can present to other users. This is a much richer model of identity than just a display-name and an avatar.
  • Visible histories reveal shared affinities and allow users with common interests to find each other. If you are a Top Contributor in the Board Games section of a site, then like-minded folks can find you, follow you, or invite you to participate in their activities.

    You will, however, find contexts where this is not desirable. On a question-and-answer site like Yahoo! Answers, for instance, don't be surprised to find out that many users won't want their questions about gonorrhea or chlamydia to appear as part of their historical record. Err on the side of giving your users control over what appears, or give them the ability to hide their participation history altogether.

  • A past is hard to fake. Most site identities are cheap. In and of themselves, they just don't mean much. A couple of quick form-fields, a 'Submit' button and practically anyone (or no one– bots welcome!) can become a full-fledged member of most sites. It is much harder, however, to fake a history of interaction with a site for any duration of time.

    We don't mean to imply that it can't be done – harvesting 'deep' identities is practically an offshoot industry of the MMORPG world (See the figure above.) But it does provide a fairly high participatory hurdle to jump. When done properly, user karma can assure some level of commitment and engagement from your users. (Or at least allow you to ascertain those levels quickly.)

  • Reputation disambiguates identity conflicts. Hopefully, you've moved away from publicly identifying users on your site by their unique identifier. (You have read the Tripartite Identity Pattern, right?) But this introduces a whole new headache: identity spoofing. If your public namespace doesn't guarantee uniqueness (or even if it does– it'll be hard to guard against similar-appearing/l33t-speak equivalents and the like) then you'll have this problem.

    Once your community is at scale, trolls will take great delight in appropriating others' identities – assuming the same display name, uploading the same avatar – purely in an effort to disrupt conversations. It's not a perfect defense, but always associate a contributor's identity with his or her participation history or reputation to help mitigate these occurrences. You will, at least, have armed the community with the information they need to decide who's legit and who's an interloper.

These are some of the reasons that extending user identities with reputation is useful. Chapter 8 of Building Web Reputation Systems offers a series of considerations for how to do so most effectively.

November 11, 2009

5-Star Failure?

Reputation Wednesday is an ongoing series of essays about reputation-related matters. This week's entry confirms that poorly chosen reputation inputs will indeed yield poor results.

Pity the poor, beleaguered 5-Star rating. Not so very long ago, it was the belle of the online ratings ball: its widespread adoption by high-profile sites like Amazon, Yahoo!, and Netflix influenced a host of imitators, and—at one point—star-ratings were practically an a priori choice for site designers when considering how best to capture their users' opinions. Their no-brainer inclusion had almost reached cargo cult design status.

This has subsided in recent years, as stars have received stiff competition from hot, upstart mechanisms like "Digg-style" voting (what we, when contributing to the Yahoo! Pattern Library, rechristened as Vote to Promote.) And Facebook's "Like" action (which, I guess, was ahem, "inspired by" FriendFeed though, let us not forget that for a time, also flirted with Thumbs Up & Down rating of feed items.) Definitely, within the past 2 or 3 years, stars 'obvious' appeal as the ratings mechanism of choice is no longer so obvious.

Even more recently, 5-Star ratings fall from grace is almost complete. YouTube fired the first volley, declaring that, by and large, people on YouTube overwhelmingly give 5 stars to videos on that site. (For readers of this site, you'll recall that we blogged about similar J-Curve distributions that are prevalent on Yahoo! as well.)

And then the venerable Wall Street Journal declared that On the Internet, Everyone's a Critic But They're Not Very Critical:

One of the Web's little secrets is that when consumers write online reviews, they tend to leave positive ratings: The average grade for things online is about 4.3 stars out of five.
And, just like that, as quickly as 'stars are it' rose to prominence, 'stars are dead' is rapidly becoming the accepted wisdom. (Don't believe me? Read the comments when TechCrunch covered the YouTube discovery, and you'll see folks all-but-rushing to prop up a variety of their 'preferred rating mechanism' in stars' place.)

Are stars dead?

This is, of course, the wrong way to frame the question. Stars, thumbs, favorites, or sliders: any of these ratings input mechanisms are dead-on-arrival if they're not carefully considered within the context of use. 5-Star ratings require a little more cognitive investment than a simple 'I Like This' statement, so--before designing 5-star ratings into your system--consider the following.

Will it be clear to users what you're asking them to assess? It's not entirely surprising that YouTube's ratings overwhelmingly tend toward the positive. That's a long-observed and well understood phenomenon in the social sciences called Acquiescence Bias. It is "the tendency of a respondent to agree with a statement when in doubt." And 5-star ratings, in the case of YouTube, are nothing but doubt. What, exactly, is a fair and accurate quantitative assessment for a video on YouTube? The input mechanism does provide some clues, in the form of text hints for the various ratings levels (ranging from 'Poor' to 'Awesome!') but these are highly subjective and - themselves - way too open to interpretation.

Is a scale necessary? If the primary decision you're asking users to make is 'good vs. bad' or 'I liked it' or 'I didn't', then are multiple steps of decisioning really adding anything to their evaluation?

Are comparisons being made? Should I, as a user, rate videos in comparison to other similar videos on YouTube? What, exactly, distinguishes a 5-star football to the groin video from a 2-star? Am I rating against like videos? Or all videos on YouTube? (Or every video I've ever seen!?)

Have they watched the video? One way to encourage more-thoughtful ratings is to place the input mechanism at the proper juncture: make some attempt, at least, to ensure that the user is rating the thing only after having experienced it. YouTube's 5-star mechanism is fixed and always-present, encouraging drive-by ratings, premature ratings or just general sloppiness of assessment.

So, are stars inappropriate for YouTube, at least in the way that they've designed them? Probably, yes.

To wrap up, some quick links. Check out this elegant and innovative design that the folks at Steepster recently rolled out, and think about the ways it cleverly addresses all four of the concerns listed above.

And to see a really in-depth study of 5-star ratings used effectively, check out Using 5-Star Ratings from Christopher Allen & Shannon Appelcline's excellent series on Systems for Collective Choice.