Differences

This shows you the differences between two versions of the page.

chapter_7 [2009/11/18 18:20]
randy
chapter_7 [2009/12/01 14:37] (current)
randy submitted for publisher review
Line 1: Line 1:
===== Displaying Reputation ===== ===== Displaying Reputation =====
-You've designed a reputation model and decided how to collect your inputs. But your work doesn't end there. Far from it. Now you have decisions to make about how to use the reputations that your system is tabulating. In this chapter and the next, we'll discuss the many options for using reputation to improve the user experience of your site, enrich content quality, and provide incentives for your users to become better, more active participants. In this chapter specifically, we'll discuss options for whether to display reputation, whom to display it to, how to display it, and help you decided which display forms are right for your application.+In <html><a href="/doku.php?id=Chapter_6">Chapter_6</a>&nbsp;</html>we described how to create a custom reputation model by identifying the objects in you application, selecting appropriate inputs, and developing the processes you'll need to generate your reputations. But your work doesn't end there. Far from it. Now you have decisions to make about how to use the reputations that your system is tabulating. 
 + 
 +In this chapter and the next, we'll discuss the many options for using reputation to improve the user experience of your site, enrich content quality, and provide incentives for your users to become better, more active participants. In this chapter specifically, we'll discuss options for whether to display reputation, whom to display it to, how to display it, and help you decided which display forms are right for your application.
==== How to Use a Reputation: Three Questions ==== ==== How to Use a Reputation: Three Questions ====
-We'll walk you through a simple process for deciding on the best way to use a reputation, starting with three questions. For each of reputation you are creating to display or use, you should ask each of these questions before proceeding:+For each of reputation you are creating to display or use, you should ask each of these questions before proceeding:
  - Who will be able to see the reputation?   - Who will be able to see the reputation?
Line 15: Line 17:
    * And/or will this score be used to make other decisions about how the site flows or your business operates?     * And/or will this score be used to make other decisions about how the site flows or your business operates?
  - Is this reputation for a //content// item or a //person//? Each requires a fundamentally different approach.   - Is this reputation for a //content// item or a //person//? Each requires a fundamentally different approach.
-Though you may choose multiple answers from the list above for each reputation, try to keep it simple at first-don't try to do too much with a single reputation. Confounding the purposes of a reputation-by, for example, surfacing participation points in a public karma score-can encourage undesirable user behavior and may even backfire by discouraging participation. Read Chapters 8 and 9 completely for a solid understanding of the issues related to overloading a single reputation.+Though you may choose multiple answers from the list above for each reputation, try to keep it simple at first-don't try to do too much with a single reputation. Confounding the purposes of a reputation-by, for example, surfacing participation points in a public karma score-can encourage undesirable user behavior and may even backfire by discouraging participation. Read <html><a href="/doku.php?id=Chapter_7">Chapter_7</a>&nbsp;</html>and <html><a href="/doku.php?id=Chapter_8">Chapter_8</a>&nbsp;</html>completely for a solid understanding of the issues related to overloading a single reputation.
<note caution>Resist the temptation to treat a single reputation score as the cure-all for your user-generated content incentive ills. Remember the lesson of the FICO score in <html><a href="/doku.php?id=Chapter_1#Chap_1-FICO">Chap_1-FICO</a>&nbsp;</html>. <note caution>Resist the temptation to treat a single reputation score as the cure-all for your user-generated content incentive ills. Remember the lesson of the FICO score in <html><a href="/doku.php?id=Chapter_1#Chap_1-FICO">Chap_1-FICO</a>&nbsp;</html>.
Line 30: Line 32:
Compelling reasons exist to keep reputations hidden from users. In fact, in some circumstances, you may want to obscure the fact that you're tracking them at all. It may sound rather Machiavellian, but the truth of the matter is this: a community under public scrutiny behaves differently (and, in many ways, less honestly) than one in blissful ignorance. Compelling reasons exist to keep reputations hidden from users. In fact, in some circumstances, you may want to obscure the fact that you're tracking them at all. It may sound rather Machiavellian, but the truth of the matter is this: a community under public scrutiny behaves differently (and, in many ways, less honestly) than one in blissful ignorance.
-Several tradeoffs are involved. Displaying reputations takes up significant page real estate, requires user interface design and testing, and can compete with your content for the user's attention and understanding. Quickly, show Digg.com (<html><a href="#Figure_7-1">Figure_7-1</a>&nbsp;</html>) to 10 of your friends and ask them, What kind of site is this? News? Entertainment? Community? Odds are good that at least a few of them will answer: "This appears to be a contest of some sort."+Several trade offs are involved. Displaying reputations takes up significant page real estate, requires user interface design and testing, and can compete with your content for the user's attention and understanding. Quickly, show Digg.com (<html><a href="#Figure_7-1">Figure_7-1</a>&nbsp;</html>) to 10 of your friends and ask them, What kind of site is this? News? Entertainment? Community? Odds are good that at least a few of them will answer: “This appears to be some sort of //contest//.
-<html><a name="Figure_7-1"><center></html>// Figure_7-1: Digg's main page. //<html></center></a></html> +<html><a name="Figure_7-1"><center></html>// Figure_7-1: Digg's site design puts overt reputation scores front and center. //<html></center></a></html> 
-<html><center><img width="65%" src="http://buildingreputation.com/lib/exe/fetch.php?media=DiggMainPage.png"/></center></html>+<html><center><img width="65%" src="http://buildingreputation.com/lib/exe/fetch.php?media=Figure_7-1.png"/></center></html>
The impression that Digg makes is not a bad thing. It just demonstrates that Digg made a conscious decision to display content reputation prominently-in fact, the display of reputation is the central interaction mechanism on the site. It's practically impossible to interact with Digg, or get any use out of it, without some understanding of how community voting affects the selection and display of popular items on the site. (Digg is perhaps the most well-known example of a site that employs the vote-to-promote pattern. See <html><a href="/doku.php?id=Chapter_6">Chapter_6</a>&nbsp;</html>.) The impression that Digg makes is not a bad thing. It just demonstrates that Digg made a conscious decision to display content reputation prominently-in fact, the display of reputation is the central interaction mechanism on the site. It's practically impossible to interact with Digg, or get any use out of it, without some understanding of how community voting affects the selection and display of popular items on the site. (Digg is perhaps the most well-known example of a site that employs the vote-to-promote pattern. See <html><a href="/doku.php?id=Chapter_6">Chapter_6</a>&nbsp;</html>.)
-Juxtapose Digg's approach with that of Flickr. The popular photo-sharing and discovery service also makes use of reputation to surface quality content, but it does not display explicit reputations: rather, it prominently displays items that achieve a certain reputation and that can be browsed (daily, weekly, or monthly) in the "Explore" gallery (at [[http://www.flickr.com/explore|http://www.flickr.com/explore]]<html><a href="#Figure_7-2">Figure_7-2</a>&nbsp;</html>. The result is a very consistent and impressive display of high-quality photos with very little indication of how those photos are selected.+Juxtapose Digg's approach with that of Flickr. The popular photo-sharing and discovery service also makes use of reputation to surface quality content, but it does not display explicit reputations: rather, it prominently displays items that achieve a certain reputation and that can be browsed (daily, weekly, or monthly) in the “Explore” gallery (at [[http://www.flickr.com/explore|http://www.flickr.com/explore]]<html><a href="#Figure_7-2">Figure_7-2</a>&nbsp;</html>. The result is a very consistent and impressive display of high-quality photos with very little indication of how those photos are selected.
-<html><a name="Figure_7-2"><center></html>// Figure_7-2: Flickr's "Explore" gallery. //<html></center></a></html> +<html><a name="Figure_7-2"><center></html>// Figure_7-2: Flickr's Exploregallery is also based on reputation, but you never see a score associated with a photo. //<html></center></a></html> 
-<html><center><img width="65%" src="http://buildingreputation.com/lib/exe/fetch.php?media=FlickrExploreInteresting.png"/></center></html>+<html><center><img width="65%" src="http://buildingreputation.com/lib/exe/fetch.php?media=Figure_7-2.png"/></center></html>
-Flickr's interestingness algorithm determines which photos make it into the "Explore" gallery and which don't. The same algorithm lets users sort their own photos by interestingness.+Flickr's interestingness algorithm determines which photos make it into the “Explore” gallery and which don't. The same algorithm lets users sort their own photos by interestingness.
-Digg and Flickr represent two very different approaches to reputation display, but the results are very much the same. Theoretically, you can always glance at the front page of Digg, or Flickr's "Explore" gallery, to see where the good stuff is-what are people watching, commenting on, or interacting with the most on the site.+Digg and Flickr represent two very different approaches to reputation display, but the results are very much the same. Theoretically, you can always glance at the front page of Digg, or Flickr's “Explore” gallery, to see where the good stuff is-what are people watching, commenting on, or interacting with the most on the site.
How do you decide whether or not to display reputations on your site? And how prominently? Generally, follow the rule of least disclosure: do not display a reputation that doesn't add specific value to the objects being evaluated. How do you decide whether or not to display reputations on your site? And how prominently? Generally, follow the rule of least disclosure: do not display a reputation that doesn't add specific value to the objects being evaluated.
Line 52: Line 54:
<note caution>Avoid collecting reputation for display only. Orkut allowed users to rate other users explicitly on iconic criteria like “trusty,” “cool,” and “sexy” for no use other than display. This use of reputation caused all kinds of social backlash. <note caution>Avoid collecting reputation for display only. Orkut allowed users to rate other users explicitly on iconic criteria like “trusty,” “cool,” and “sexy” for no use other than display. This use of reputation caused all kinds of social backlash.
-People were either disappointed that they weren't rated "cool" by more people, or they were creeped out by people of the same gender calling them sexy. Eventually, Orkut removed the display of individual friends' ratings and kept only the aggregate scores.+People were either disappointed that they weren't rated “cool” by more people, or they were creeped out by people of the same gender calling them sexy. Eventually, Orkut removed the display of individual friends' ratings and kept only the aggregate scores.
</note> </note>
Line 61: Line 63:
Are you tracking a reputation primarily to keep users informed about how well they or their creations are performing in the community? Consider displaying that reputation only to its owner, as a personal communication between site and user. Are you tracking a reputation primarily to keep users informed about how well they or their creations are performing in the community? Consider displaying that reputation only to its owner, as a personal communication between site and user.
-<note tip>+<box blue 75% round>
** Personal Reputation Is Not Private ** ** Personal Reputation Is Not Private **
Line 68: Line 70:
In other words, reputations may be displayed in a personal context, but that's no guarantee that they're private. As a service provider, you should acknowledge that distinction and account for it in your terms of service. In other words, reputations may be displayed in a personal context, but that's no guarantee that they're private. As a service provider, you should acknowledge that distinction and account for it in your terms of service.
-</note>+</box>
Personal reputations are used extensively for applications like social bookmarking, lists of favorites, training recommendation systems, sorting and filtering news feeds, providing content quality and feedback, fine-grained experience point tracking, and other performance metrics. Most of the same user interface patterns used for displaying public reputation apply to personal ones too, but take care to ensure that each user knows when her reputations will and will not be displayed to others. Personal reputations are used extensively for applications like social bookmarking, lists of favorites, training recommendation systems, sorting and filtering news feeds, providing content quality and feedback, fine-grained experience point tracking, and other performance metrics. Most of the same user interface patterns used for displaying public reputation apply to personal ones too, but take care to ensure that each user knows when her reputations will and will not be displayed to others.
-<note tip>Keep a reputation personal when its owner gains some significant benefit from the reputation-when it either improves her experience of the site (that is, personalizes it) or provides a tool for increasing self-satisfaction. For example, by selecting news stories about various sports teams over time, a user might generate a geographic region reputation that can be used to target advertising displayed to the user. Clearly that reputation should not be public information, but it might be surfaced privately so that the user can correct it-"I'm a fan of Northern California sports teams, but I'm going to MIT and I really want ads for electronics stores in the Boston area."+<note tip>Keep a reputation personal when its owner gains some significant benefit from the reputation-when it either improves her experience of the site (that is, personalizes it) or provides a tool for increasing self-satisfaction. For example, by selecting news stories about various sports teams over time, a user might generate a geographic region reputation that can be used to target advertising displayed to the user. Clearly that reputation should not be public information, but it might be surfaced privately so that the user can correct it-“I'm a fan of Northern California sports teams, but I'm going to MIT and I really want ads for electronics stores in the Boston area.
</note> </note>
-<html><a name="Figure_7-3"><center></html>// Figure_7-3: Google Analytics. //<html></center></a></html> +<html><a name="Figure_7-3"><center></html>// Figure_7-3: Google's Analytics interface shows information that is clearly best kept between you and Google. It's personal. //<html></center></a></html> 
-<html><center><img width="65%" src="http://buildingreputation.com/lib/exe/fetch.php?media=GoogleAnalytics.png"/></center></html>+<html><center><img width="65%" src="http://buildingreputation.com/lib/exe/fetch.php?media=Figure_7-3.png"/></center></html>
Google Analytics (<html><a href="#Figure_7-3">Figure_7-3</a>&nbsp;</html>) is an example of rich personal reputation information. It provides detailed information about the performance of your web site, across a known range of score types, and it is only available to you, the site owner (or others you grant access to). While that information is invaluable to you in gauging the response of a community (in this case, the entire Web) to your content, exposing it to everyone would offer very little practical benefit. In fact, it would be a horrible idea. Google Analytics (<html><a href="#Figure_7-3">Figure_7-3</a>&nbsp;</html>) is an example of rich personal reputation information. It provides detailed information about the performance of your web site, across a known range of score types, and it is only available to you, the site owner (or others you grant access to). While that information is invaluable to you in gauging the response of a community (in this case, the entire Web) to your content, exposing it to everyone would offer very little practical benefit. In fact, it would be a horrible idea.
Line 99: Line 101:
Public reputations are used for hundreds of purposes on the Web: to compare items in a list on the basis of community member feedback, to evaluate particular targets for online transaction trustworthiness, to filter and display top-rated message board posts, to rank the best local Indonesian restaurants, to show today's gallery of the most interesting photos, to display leaderboards of the top-scoring reputation targets, and much more. Public reputations are used for hundreds of purposes on the Web: to compare items in a list on the basis of community member feedback, to evaluate particular targets for online transaction trustworthiness, to filter and display top-rated message board posts, to rank the best local Indonesian restaurants, to show today's gallery of the most interesting photos, to display leaderboards of the top-scoring reputation targets, and much more.
-Over time, public reputations can evolve to represent your community's understanding of its own zeitgeist. And there's the rub-depending on how you use public reputation, you can alienate users who aren't part of the in crowd. Yelp is all about public ratings and reviews of local restaurants, but it isn't used extensively by users over 50. Most of the reviews are written by twentysomethings [most "Yelpers" are between the ages of 26 and 35] who seem to be mostly interested in a restaurant's potential as a dating hangout.+Over time, public reputations can evolve to represent your community's understanding of its own zeitgeist. And there's the rub-depending on how you use public reputation, you can alienate users who aren't part of the in crowd. Yelp is all about public ratings and reviews of local restaurants, but it isn't used extensively by users over 50. Most of the reviews are written by twentysomethings [most “Yelpers” are between the ages of 26 and 35] who seem to be mostly interested in a restaurant's potential as a dating hangout.
<note tip>Public reputations are useful for allowing users to compare like items. Public karma reputations also serve as an effective extension of a person's identity. <note tip>Public reputations are useful for allowing users to compare like items. Public karma reputations also serve as an effective extension of a person's identity.
Line 108: Line 110:
Almost every web site with a large volume of user-generated content is using hidden reputation scores internally-as a means of tracking exactly who is saying what about a content item or another user. Almost every web site with a large volume of user-generated content is using hidden reputation scores internally-as a means of tracking exactly who is saying what about a content item or another user.
-  * When users click the "Spam" button in a webmail application, they contribute to a database of IP addresses for abusive mail servers.+  * When users click the “Spam” button in a webmail application, they contribute to a database of IP addresses for abusive mail servers.
  * Web crawlers constantly scan the Web to examine what sites link to what other sites and to calculate a hidden score such as Google's PageRank.   * Web crawlers constantly scan the Web to examine what sites link to what other sites and to calculate a hidden score such as Google's PageRank.
  * Yahoo! Answers tracks corporate reputation for users who are particularly good at identifying bad content and gives them more power to hide bad content quickly.   * Yahoo! Answers tracks corporate reputation for users who are particularly good at identifying bad content and gives them more power to hide bad content quickly.
Line 148: Line 150:
</note> </note>
-Recommender systems use reputation to make suggestions about similarities between user tastes: "People who like the same things as you do you also like . . ." and discovered taste similarities between items: "People who liked this item also like . . . " They use reputation in the form of confidence scores and typically display multiple entities in rank order when making recommendations. When the user selects a suggested item, that selection itself is also entered in the reputation system to further improve the quality of future results.+Recommender systems use reputation to make suggestions about similarities between user tastes: “People who like the same things as you do you also like…” and discovered taste similarities between items: “People who liked this item also like…” They use reputation in the form of confidence scores and typically display multiple entities in rank order when making recommendations. When the user selects a suggested item, that selection itself is also entered in the reputation system to further improve the quality of future results.
The specific reputation usage patterns related to ranking and sorting are quality-sort search results, leaderboards, related items, recommendations, search relevance (such as Google's PageRank), corporate community health metrics, and advertising performance metrics. The specific reputation usage patterns related to ranking and sorting are quality-sort search results, leaderboards, related items, recommendations, search relevance (such as Google's PageRank), corporate community health metrics, and advertising performance metrics.
Line 194: Line 196:
== Karma Is Complex, Built of Indirect Inputs == == Karma Is Complex, Built of Indirect Inputs ==
-Be careful with Karma - sometimes making things as simple and explicit as possible is the wrong choice for reputation:+Be careful with Karma-sometimes making things as simple and explicit as possible is the wrong choice for reputation:
  * Rating a user directly should be avoided. Typical implementations only require a user to click once to rate another user and are therefore prone to abuse. When direct evaluation karma models are combined with the common practice of streamlining user registration processes (on many sites //opening a new account// is an easier operation than changing the password on an existing account), they get out of hand quickly. See the example of Orkut in <html><a href="/doku.php?id=Chapter_7#Chap_7-Display_Numbered_Levels">Chap_7-Display_Numbered_Levels</a>&nbsp;</html>.   * Rating a user directly should be avoided. Typical implementations only require a user to click once to rate another user and are therefore prone to abuse. When direct evaluation karma models are combined with the common practice of streamlining user registration processes (on many sites //opening a new account// is an easier operation than changing the password on an existing account), they get out of hand quickly. See the example of Orkut in <html><a href="/doku.php?id=Chapter_7#Chap_7-Display_Numbered_Levels">Chap_7-Display_Numbered_Levels</a>&nbsp;</html>.
  * Asking people to evaluate others directly is socially awkward. Don't put users in the position of lying about their friends.   * Asking people to evaluate others directly is socially awkward. Don't put users in the position of lying about their friends.
  * Using multiple inputs presents a broader picture of the target user's value.   * Using multiple inputs presents a broader picture of the target user's value.
-  * Economics research into "revealed preference," or what people actually do, as opposed to what they say, indicates that actions provide a more accurate picture of value than elicited ratings.+  * Economics research into “revealed preference,or what people actually do, as opposed to what they say, indicates that actions provide a more accurate picture of value than elicited ratings.
== Karma Calculations Are Often Opaque == == Karma Calculations Are Often Opaque ==
Line 210: Line 212:
  * Publicly displayed karma should be rare because, as with content reputation, users are easily confused by the display of many reputations on the same page or within the same context.   * Publicly displayed karma should be rare because, as with content reputation, users are easily confused by the display of many reputations on the same page or within the same context.
  * Publicly displayed karma should be rare because it can create the wrong incentives for your community. Avoid sorting users by karma. See <html><a href="/doku.php?id=Chapter_7#Chap_7-Leaderboards_Considered_Harmful">Chap_7-Leaderboards_Considered_Harmful</a>&nbsp;</html>.   * Publicly displayed karma should be rare because it can create the wrong incentives for your community. Avoid sorting users by karma. See <html><a href="/doku.php?id=Chapter_7#Chap_7-Leaderboards_Considered_Harmful">Chap_7-Leaderboards_Considered_Harmful</a>&nbsp;</html>.
-  * If you do display it publicly, make karma visually distinct from any nearby content reputation. Yahoo!'s EU message board displays the karma of a post's author as a colored medallion, with the message rated with stars. But consider this: Slashdot's message board doesn't display the karma of post authors to anyone. Even the display of a user's own karma is vague: "positive," "good," or "excellent." After originally displaying karma publicly as a number, over time Slashdot has shifted to an increasingly opaque display of karma.+  * If you do display it publicly, make karma visually distinct from any nearby content reputation. Yahoo!'s EU message board displays the karma of a post's author as a colored medallion, with the message rated with stars. But consider this: Slashdot's message board doesn't display the karma of post authors to anyone. Even the display of a user's own karma is vague: “positive,” “good,or “excellent.After originally displaying karma publicly as a number, over time Slashdot has shifted to an increasingly opaque display of karma.
  * Public displayed karma should be rare because it isn't expected. When Yahoo! Shopping added Top Reviewer karma to encourage review creation, they displayed a Top Reviewer badge with each review and rushed it out for the Christmas 2006 season. After the New Year had passed, user testing revealed that most users didn't even notice the badges. When they //did// notice them, many thought they meant either that the //item// was top rated or that the user was a //paid shill// for the product manufacturer or Yahoo!.   * Public displayed karma should be rare because it isn't expected. When Yahoo! Shopping added Top Reviewer karma to encourage review creation, they displayed a Top Reviewer badge with each review and rushed it out for the Christmas 2006 season. After the New Year had passed, user testing revealed that most users didn't even notice the badges. When they //did// notice them, many thought they meant either that the //item// was top rated or that the user was a //paid shill// for the product manufacturer or Yahoo!.
Line 218: Line 220:
Yahoo! holds reputation for karma scores to a higher standard than reputation for content. Be very careful in applying terminology and labels to people, for several reasons: Yahoo! holds reputation for karma scores to a higher standard than reputation for content. Be very careful in applying terminology and labels to people, for several reasons:
-  * Avoid labels that might appear as attacks. They set a hostile tone that will be amplified in users' responses. This caution applies both to overly positive labels (such as "hotshot" or "top" designations) or negative ones (such as "newbie" or "rookie"). +  * Avoid labels that might appear as attacks. They set a hostile tone that will be amplified in users' responses. This caution applies both to overly positive labels (such as “hotshot” or “top” designations) or negative ones (such as “newbie” or “rookie” ). 
-  * Avoid labels that introduce legal risks. What if a site labeled members of a health forum "experts," and these "experts" then gave out bad advice?+  * Avoid labels that introduce legal risks. What if a site labeled members of a health forum “experts,and these “experts” then gave out bad advice?
These are rules of thumb that may not necessarily apply to a given context. In role-playing games, for example, publicly shared simple karma is displayed in terms of experience levels, which are inherently competitive. These are rules of thumb that may not necessarily apply to a given context. In role-playing games, for example, publicly shared simple karma is displayed in terms of experience levels, which are inherently competitive.
Line 267: Line 269:
  * Be careful how you display percentages if the input claim type isn't suitable for normalized output of the aggregated results. For example, consider the displaying the results of a series of thumb votes; though you can display the thumb graphic that got the majority of votes, you'll probably still want to display either the raw votes for each or the percentages of the total up votes and down votes.   * Be careful how you display percentages if the input claim type isn't suitable for normalized output of the aggregated results. For example, consider the displaying the results of a series of thumb votes; though you can display the thumb graphic that got the majority of votes, you'll probably still want to display either the raw votes for each or the percentages of the total up votes and down votes.
<html><a name="Figure_7-4"><center></html>// Figure_7-4: Content example: normalized percentages with summary count. //<html></center></a></html> <html><a name="Figure_7-4"><center></html>// Figure_7-4: Content example: normalized percentages with summary count. //<html></center></a></html>
-<html><center><img width="50%" src="http://buildingreputation.com/lib/exe/fetch.php?media=Ch08-ThumbsPercentage.png"/></center></html>+<html><center><img width="50%" src="http://buildingreputation.com/lib/exe/fetch.php?media=Figure_7-4.png"/></center></html>
<html><a href="#Figure_7-4">Figure_7-4</a>&nbsp;</html>displays content reputation as the percentage of thumbs-up ratings given on Yahoo! Television for a television episode. Notice that the simple average calculation requires that the total number of votes be included in the display to allow users to evaluate the reliability of the score. <html><a href="#Figure_7-4">Figure_7-4</a>&nbsp;</html>displays content reputation as the percentage of thumbs-up ratings given on Yahoo! Television for a television episode. Notice that the simple average calculation requires that the total number of votes be included in the display to allow users to evaluate the reliability of the score.
  * Consider that a graphical sliding scale or thermometer view will makes the reputation easier to understand at a glance. If necessary, also display the numeric value alongside the graphic.   * Consider that a graphical sliding scale or thermometer view will makes the reputation easier to understand at a glance. If necessary, also display the numeric value alongside the graphic.
<html><a name="Figure_7-5"><center></html>// Figure_7-5: Karma example: percentage bars with named levels. //<html></center></a></html> <html><a name="Figure_7-5"><center></html>// Figure_7-5: Karma example: percentage bars with named levels. //<html></center></a></html>
-<html><center><img width="65%" src="http://buildingreputation.com/lib/exe/fetch.php?media=Ch08-WowReputationBars.png"/></center></html>+<html><center><img width="65%" src="http://buildingreputation.com/lib/exe/fetch.php?media=Figure_7-5.png"/></center></html>
<html><a href="#Figure_7-5">Figure_7-5</a>&nbsp;</html>shows a number of Okefarflung's karma scores as percentage bars, each representing his reputation with various political factions on Worlds of Warcraft. Printed over each bar is one of the current named levels (see <html><a href="/doku.php?id=Chapter_7#Chap_7-Display_Named_Levels">Chap_7-Display_Named_Levels</a>&nbsp;</html>) that his current reputation falls in. <html><a href="#Figure_7-5">Figure_7-5</a>&nbsp;</html>shows a number of Okefarflung's karma scores as percentage bars, each representing his reputation with various political factions on Worlds of Warcraft. Printed over each bar is one of the current named levels (see <html><a href="/doku.php?id=Chapter_7#Chap_7-Display_Named_Levels">Chap_7-Display_Named_Levels</a>&nbsp;</html>) that his current reputation falls in.
Line 278: Line 280:
<ul><li>Percentage displays of normalized scores are universally understood.</li><li>No more precise display is possible.</li><li>Is Web 2.0 API and spreadsheet friendly.</li><li>Implementation is trivial. This is often the primary reason this approach is considered.</li></ul> <ul><li>Percentage displays of normalized scores are universally understood.</li><li>No more precise display is possible.</li><li>Is Web 2.0 API and spreadsheet friendly.</li><li>Implementation is trivial. This is often the primary reason this approach is considered.</li></ul>
</td><td align="left"> </td><td align="left">
-<ul><li>Percentages aren't accurate for very small sample sizes and therefore can be misleading. One yes vote shouldn't be expressed as "100.00% of votes tallied are in favor . . ." Consider suppressing percentage display until a reasonable number of inputs have accumulated, adjusting the score, or at least displaying the number of inputs alongside the average.</li><li>As with accuracy, precision entails various challenges: displaying too many decimal digits can lead users to make unwarranted assumptions about accuracy. Also, if the input was from level-based or nonlinear normalization or irregular distributions, average scores can be skewed.</li><li>Lots of numbers on a page can seem impersonal, especially when they're associated with people.</li></ul>+<ul><li>Percentages aren't accurate for very small sample sizes and therefore can be misleading. One yes vote shouldn't be expressed as “100.00% of votes tallied are in favor…” Consider suppressing percentage display until a reasonable number of inputs have accumulated, adjusting the score, or at least displaying the number of inputs alongside the average.</li><li>As with accuracy, precision entails various challenges: displaying too many decimal digits can lead users to make unwarranted assumptions about accuracy. Also, if the input was from level-based or nonlinear normalization or irregular distributions, average scores can be skewed.</li><li>Lots of numbers on a page can seem impersonal, especially when they're associated with people.</li></ul>
</td></tr></tbody></table></html><html><a name='Chap_7-Points_and_Accumulators'></a></html> </td></tr></tbody></table></html><html><a name='Chap_7-Points_and_Accumulators'></a></html>
=== Points and Accumulators === === Points and Accumulators ===
Line 289: Line 291:
  * Display counts of actions collected from many users, such as voting and favorites.   * Display counts of actions collected from many users, such as voting and favorites.
-<html><a name="Figure_7-6"><center></html>// Figure_7-6: Content example: Digg's diggs and comment counts. //<html></center></a></html> +<html><a name="Figure_7-6"><center></html>// Figure_7-6: Content example: Digg shows the number of times an item has been Digged.Another example is the count of comments for an item. //<html></center></a></html> 
-<html><center><img width="65%" src="http://buildingreputation.com/lib/exe/fetch.php?media=Ch08-DiggEntry.png"/></center></html>+<html><center><img width="65%" src="http://buildingreputation.com/lib/exe/fetch.php?media=Figure_7-6.png"/></center></html>
<html><a href="#Figure_7-6">Figure_7-6</a>&nbsp;</html>shows an entry from Digg.com, which displays two different accumulators: The number of //Diggs// and //Comments//. Note the //Share// and //Bury// buttons. Though these effect the chance that an entity is displayed on the home page, the counts for these actions are not displayed to the users. <html><a href="#Figure_7-6">Figure_7-6</a>&nbsp;</html>shows an entry from Digg.com, which displays two different accumulators: The number of //Diggs// and //Comments//. Note the //Share// and //Bury// buttons. Though these effect the chance that an entity is displayed on the home page, the counts for these actions are not displayed to the users.
  * Publicly display points when you wish to encourage users to take actions that increase or decrease the value for an entity.   * Publicly display points when you wish to encourage users to take actions that increase or decrease the value for an entity.
<html><a name="Figure_7-7"><center></html>// Figure_7-7: Karma example: Yahoo! Answers awards points mostly for participation. //<html></center></a></html> <html><a name="Figure_7-7"><center></html>// Figure_7-7: Karma example: Yahoo! Answers awards points mostly for participation. //<html></center></a></html>
-<html><center><img width="35%" src="http://buildingreputation.com/lib/exe/fetch.php?media=Ch07-Fig_7-7-PointsAnswers.png"/></center></html>+<html><center><img width="35%" src="http://buildingreputation.com/lib/exe/fetch.php?media=Figure_7-7.png"/></center></html>
<html><a href="#Figure_7-7">Figure_7-7</a>&nbsp;</html>shows a typical participation-points enabled website, in this case Yahoo! Answers. Points are granted for a very wide range of activities including loggin in, creating content, and evaluating other's contributions. Note that this mini-profile also displays a numbered level (see <html><a href="/doku.php?id=Chapter_7#Chap_7-Display_Numbered_Levels">Chap_7-Display_Numbered_Levels</a>&nbsp;</html>) to simplify comparison between users. The number of points accumulated in such systems can get pretty large. <html><a href="#Figure_7-7">Figure_7-7</a>&nbsp;</html>shows a typical participation-points enabled website, in this case Yahoo! Answers. Points are granted for a very wide range of activities including loggin in, creating content, and evaluating other's contributions. Note that this mini-profile also displays a numbered level (see <html><a href="/doku.php?id=Chapter_7#Chap_7-Display_Numbered_Levels">Chap_7-Display_Numbered_Levels</a>&nbsp;</html>) to simplify comparison between users. The number of points accumulated in such systems can get pretty large.
Line 302: Line 304:
<ul><li>Explicitly displayed point amounts that the user can influence can be a powerful motivator for some users to participate.</li><li>Is easy to understand in ranked lists.</li><li>Implementation is trivial.</li></ul> <ul><li>Explicitly displayed point amounts that the user can influence can be a powerful motivator for some users to participate.</li><li>Is easy to understand in ranked lists.</li><li>Implementation is trivial.</li></ul>
</td><td align="left"> </td><td align="left">
-<ul><li>First-Mover Effect - If your accumulator has no cap, awards effectively deflate over time as the leading entities continue to accumulate points and increase their lead. New users become frustrated that they can't catch up, and new - often more interesting- entities receive less attention. Consider either caps and/or decay for your point system.</li><li>Encourages the minimum effort for the most maximum benefit behavior: The system tells you exactly how many points are associated with your actions in real time. Yahoo! Answers gives 10 points for an answer chosen as the best, and 1 point each to users who rate other people's answers. Too bad that writing the best answer takes more than 10 times as long as it does to click a thumb icon 10 times.</li><li>If you do cap your points, when the most of your users reach that cap, you will need to add new activities to justify moving the cap to move higher. For example, online role-playing games typically extend the level-cap along with expanded content for the users to explore.</li></ul>+<ul><li>First-Mover Effect-If your accumulator has no cap, awards effectively deflate over time as the leading entities continue to accumulate points and increase their lead. New users become frustrated that they can't catch up, and new-often more interesting- entities receive less attention. Consider either caps and/or decay for your point system.</li><li>Encourages the minimum effort for the most maximum benefit behavior: The system tells you exactly how many points are associated with your actions in real time. Yahoo! Answers gives 10 points for an answer chosen as the best, and 1 point each to users who rate other people's answers. Too bad that writing the best answer takes more than 10 times as long as it does to click a thumb icon 10 times.</li><li>If you do cap your points, when the most of your users reach that cap, you will need to add new activities to justify moving the cap to move higher. For example, online role-playing games typically extend the level-cap along with expanded content for the users to explore.</li></ul>
</td></tr></tbody></table></html> </td></tr></tbody></table></html>
=== Statistical Evidence === === Statistical Evidence ===
Line 315: Line 317:
  * Use this display format when a variety of data points would provide a well-rounded view of an entity's worth or performance   * Use this display format when a variety of data points would provide a well-rounded view of an entity's worth or performance
-<html><a name="Figure_7-8"><center></html>// Figure_7-8: Content example: YouTube's statistics and data. //<html></center></a></html> +<html><a name="Figure_7-8"><center></html>// Figure_7-8: Content Example: with YouTube's very powerful Statistics and Datayou can track a video's rise in popularity on the site. (Sociologist and researcher Cameron Marlow calls it an Epidemiology Interface.) //<html></center></a></html> 
-<html><center><img width="65%" src="http://buildingreputation.com/lib/exe/fetch.php?media=StatEvidenceYouTube.png"/></center></html>+<html><center><img width="65%" src="http://buildingreputation.com/lib/exe/fetch.php?media=Figure_7-8.png"/></center></html>
<html><a href="#Figure_7-8">Figure_7-8</a>&nbsp;</html>shows YouTube.com's many different statistics associated with each video, each subject to different subjective interpretation. For example, the number of times a video is //Favorited// can be compared to the total number of //Views// to determine relative popularity. <html><a href="#Figure_7-8">Figure_7-8</a>&nbsp;</html>shows YouTube.com's many different statistics associated with each video, each subject to different subjective interpretation. For example, the number of times a video is //Favorited// can be compared to the total number of //Views// to determine relative popularity.
  * Use statistical evidence in displays of counts of actions collected from many users, such as voting and favorites.   * Use statistical evidence in displays of counts of actions collected from many users, such as voting and favorites.
-<html><a name="Figure_7-9"><center></html>// Figure_7-9: Karma example: Yahoo! Answers enhanced point and level information with statistical detail. //<html></center></a></html> +<html><a name="Figure_7-9"><center></html>// Figure_7-9: Karma example: answers enhanced point and level information with statistical detail. //<html></center></a></html> 
-<html><center><img width="65%" src="http://buildingreputation.com/lib/exe/fetch.php?media=StatisticalEvAnswers.png"/></center></html>+<html><center><img width="65%" src="http://buildingreputation.com/lib/exe/fetch.php?media=Figure_7-9.png"/></center></html>
Yahoo! Answers provides a categorical breakdown of statistics by contributor, as shown in <html><a href="#Figure_7-9">Figure_7-9</a>&nbsp;</html>. This allows readers to notice if the user is an answer-person (as shown here) or a question-person or something else. Yahoo! Answers provides a categorical breakdown of statistics by contributor, as shown in <html><a href="#Figure_7-9">Figure_7-9</a>&nbsp;</html>. This allows readers to notice if the user is an answer-person (as shown here) or a question-person or something else.
  * Optionally, you might extend statistical evidence to include even more information about how a particular score was derived.   * Optionally, you might extend statistical evidence to include even more information about how a particular score was derived.
<html><a name="Figure_7-10"><center></html>// Figure_7-10: Yahoo! Answers displays the sources for statistical evidence. //<html></center></a></html> <html><a name="Figure_7-10"><center></html>// Figure_7-10: Yahoo! Answers displays the sources for statistical evidence. //<html></center></a></html>
-<html><center><img width="80%" src="http://buildingreputation.com/lib/exe/fetch.php?media=StatEvidenceWhoStarred.png"/></center></html>+<html><center><img width="80%" src="http://buildingreputation.com/lib/exe/fetch.php?media=Figure_7-10.png"/></center></html>
-<html><a href="#Figure_7-10">Figure_7-10</a>&nbsp;</html>shows how Yahoo! Answers displays not only how many people have “starred” a question (that is, found it interesting); it also shows exactly who starred it. However, displaying that information can have negative consequences: it may create an expectation of social reciprocity (for example, your friends might become upset if you opted not to endorse their contributions).+<html><a href="#Figure_7-10">Figure_7-10</a>&nbsp;</html>shows how Yahoo! Answers displays not only how many people have “starred” a question (that is, found it interesting); it also shows exactly who starred it. However, displaying that information can have negative consequences: among other things, it may create an expectation of social reciprocity (for example, your friends might become upset if you opted not to endorse their contributions).
<html><table align="center" border="1" class="inline"><thead><tr><td align="center">Pros</td><td align="center">Cons</td></tr></thead><tbody><tr><td align="left"> <html><table align="center" border="1" class="inline"><thead><tr><td align="center">Pros</td><td align="center">Cons</td></tr></thead><tbody><tr><td align="left">
<ul><li>Does not attempt to mediate or frame the experience for users: Lets them decide which reputation elements are relevant for their purposes</li></ul> <ul><li>Does not attempt to mediate or frame the experience for users: Lets them decide which reputation elements are relevant for their purposes</li></ul>
Line 357: Line 359:
  * Assign numbered level if the reputation will be displayed in a rank-ordered sort a list of entities.   * Assign numbered level if the reputation will be displayed in a rank-ordered sort a list of entities.
<html><a name="Figure_7-11"><center></html>// Figure_7-11: Content example: stars and bars-iconic numbered levels. //<html></center></a></html> <html><a name="Figure_7-11"><center></html>// Figure_7-11: Content example: stars and bars-iconic numbered levels. //<html></center></a></html>
-<html><center><img width="50%" src="http://buildingreputation.com/lib/exe/fetch.php?media=Ch08-StarsAndBars.png"/></center></html>+<html><center><img width="50%" src="http://buildingreputation.com/lib/exe/fetch.php?media=Figure_7-11.png"/></center></html>
-<html><a href="#Figure_7-11">Figure_7-11</a>&nbsp;</html>shows a typical Stars-and-Bars display pattern for ratings and reviews. Stars and Bars are numbered levels, which happen to be displayed as graphics. In this example, each has a numbered level of 0 to 5. Though each review's ratings are useful when displayed along side the entity, the average of the overall score is used to rank-order results on search results pages.+<html><a href="#Figure_7-11">Figure_7-11</a>&nbsp;</html>shows a typical Stars-and-Bars display pattern for ratings and reviews. Stars and Bars are numbered levels, which happen to be displayed as graphics. In this example, each has a numbered level of 0 to 5. Though each review's ratings are useful when displayed alongside the entity, the average of the overall score is used to rank-order results on search results pages.
  * It is typical to use numbered levels to display aggregate reputation if the //inputs// were also numbered levels. Did you input stars? Output stars.   * It is typical to use numbered levels to display aggregate reputation if the //inputs// were also numbered levels. Did you input stars? Output stars.
<html><a name="Figure_7-12"><center></html>// Figure_7-12: Karma example: Orkut profile-an accumulator and iconic number levels. //<html></center></a></html> <html><a name="Figure_7-12"><center></html>// Figure_7-12: Karma example: Orkut profile-an accumulator and iconic number levels. //<html></center></a></html>
-<html><center><img width="50%" src="http://buildingreputation.com/lib/exe/fetch.php?media=Ch07-Fig_7-12-orkutreputation.png"/></center></html>+<html><center><img width="50%" src="http://buildingreputation.com/lib/exe/fetch.php?media=Figure_7-12.png"/></center></html>
<html><a href="#Figure_7-12">Figure_7-12</a>&nbsp;</html>is the Karma ratings from Orkut.com. The Fans indicator is an accumulator (see <html><a href="/doku.php?id=Chapter_7#Chap_7-Points_and_Accumulators">Chap_7-Points_and_Accumulators</a>&nbsp;</html>), and the Trusty, Cool, and Sexy ratings are numeric levels. The user's simply click on the smiling faces, ice cubes, and hearts next to their friends profiles to influence their scores. Many sites don't allow direct karma ratings such as these with good reason (see <html><a href="/doku.php?id=Chapter_7#Chap_7-Displaying_Karma">Chap_7-Displaying_Karma</a>&nbsp;</html>.) <html><a href="#Figure_7-12">Figure_7-12</a>&nbsp;</html>is the Karma ratings from Orkut.com. The Fans indicator is an accumulator (see <html><a href="/doku.php?id=Chapter_7#Chap_7-Points_and_Accumulators">Chap_7-Points_and_Accumulators</a>&nbsp;</html>), and the Trusty, Cool, and Sexy ratings are numeric levels. The user's simply click on the smiling faces, ice cubes, and hearts next to their friends profiles to influence their scores. Many sites don't allow direct karma ratings such as these with good reason (see <html><a href="/doku.php?id=Chapter_7#Chap_7-Displaying_Karma">Chap_7-Displaying_Karma</a>&nbsp;</html>.)
  * If you need to display more than 10 levels, use numbered levels. Consider using numbered levels instead of named levels if you display more than five levels.   * If you need to display more than 10 levels, use numbered levels. Consider using numbered levels instead of named levels if you display more than five levels.
<html><a name="Figure_7-13"><center></html>// Figure_7-13: Karma example: Experience levels and guild rank (sortable). //<html></center></a></html> <html><a name="Figure_7-13"><center></html>// Figure_7-13: Karma example: Experience levels and guild rank (sortable). //<html></center></a></html>
-<html><center><img width="65%" src="http://buildingreputation.com/lib/exe/fetch.php?media=Ch08-WowGuildRoster.png"/></center></html>+<html><center><img width="65%" src="http://buildingreputation.com/lib/exe/fetch.php?media=Figure_7-13.png"/></center></html>
-<html><a href="#Figure_7-13">Figure_7-13</a>&nbsp;</html>displays two forms, out of many, of numbered levels for the game Worlds of Warcraft. The user controls a character who's name is shown in the Members column, the first numbered level is labeled "Level" and ranges from 1 to 80, representing the amount of time and skill the user has dedicated to this character. The Guild Rank is a reverse-rank numbered level that represents the status of the user in the guild - this score is assigned by the guild master, who has the lowest guild rank.+<html><a href="#Figure_7-13">Figure_7-13</a>&nbsp;</html>displays two forms, out of many, of numbered levels for the game //World of Warcraft//. The user controls a character who's name is shown in the Members column, the first numbered level is labeled “Level” and ranges from 1 to 80, representing the amount of time and skill the user has dedicated to this character. The Guild Rank is a reverse-rank numbered level that represents the status of the user in the guild-this score is assigned by the guild master, who has the lowest guild rank.
<html><table align="center" border="1" class="inline"><thead><tr><td align="center">Pros</td><td align="center">Cons</td></tr></thead><tbody><tr><td align="left"> <html><table align="center" border="1" class="inline"><thead><tr><td align="center">Pros</td><td align="center">Cons</td></tr></thead><tbody><tr><td align="left">
<ul><li>Is easy to read.</li><li>Accommodates unlimited values. You can always add more levels at the top.</li><li>In ranked lists, relative value is easy to see.</li></ul> <ul><li>Is easy to read.</li><li>Accommodates unlimited values. You can always add more levels at the top.</li><li>In ranked lists, relative value is easy to see.</li></ul>
</td><td align="left"> </td><td align="left">
-<ul><li>Numeric format doesn't convey limits or global value. Is level 20 good? What about 40? Often requires "What's this?" user interface elements to explain levels to new users.</li><li>Lots of numbers on a page can seem impersonal, especially when they're associated with people.</li><li>For karma, numbered levels can be perceived as fostering an undesirable competitive spirit.</li></ul>+<ul><li>Numeric format doesn't convey limits or global value. Is level 20 good? What about 40? Often requires “What's this?user interface elements to explain levels to new users.</li><li>Lots of numbers on a page can seem impersonal, especially when they're associated with people.</li><li>For karma, numbered levels can be perceived as fostering an undesirable competitive spirit.</li></ul>
</td></tr></tbody></table></html><html><a name='Chap_7-Display_Named_Levels'></a></html> </td></tr></tbody></table></html><html><a name='Chap_7-Display_Named_Levels'></a></html>
== Named Levels == == Named Levels ==
In a named levels display pattern, a short, readable string of characters is substituted for a level number. In a named levels display pattern, a short, readable string of characters is substituted for a level number.
-The name adds semantic meaning to each level so that users can more easily recognize the entity's reputation when the reputation is displayed separately. Is the user a "silver contributor" or is the beef prime, choice, select, or standard?+The name adds semantic meaning to each level so that users can more easily recognize the entity's reputation when the reputation is displayed separately. Is the user a “silver contributor” or is the beef prime, choice, select, or standard?
Line 385: Line 387:
  * Named levels are useful when the number of labels is five or less, so that each level can have a name that accurately expresses its meaning.   * Named levels are useful when the number of labels is five or less, so that each level can have a name that accurately expresses its meaning.
<html><a name="Table_7-1"><center></html>// Table_7-1: Content example: USDA meat grades //<html></center></a><table align="center" border="1" class="inline"><thead><tr><td align="center">Species</td><td align="center">Quality Grades</td></tr></thead><tbody><tr><td align="center">Beef</td><td align="center">prime, choice, select, standard, utility, cutter, canner</td></tr><tr><td align="center">Lamb and yearling mutton</td><td align="center">prime, choice, good, utility, cull</td></tr><tr><td align="center">Mutton</td><td align="center">choice, good, utility, cull</td></tr><tr><td align="center">Veal and calf</td><td align="center">prime, choice, good, standard, utility</td></tr></tbody></table></html><html><a name="Figure_7-14"><center></html>// Figure_7-14: Content example: USDA prime, choice, and select stamps. //<html></center></a></html> <html><a name="Table_7-1"><center></html>// Table_7-1: Content example: USDA meat grades //<html></center></a><table align="center" border="1" class="inline"><thead><tr><td align="center">Species</td><td align="center">Quality Grades</td></tr></thead><tbody><tr><td align="center">Beef</td><td align="center">prime, choice, select, standard, utility, cutter, canner</td></tr><tr><td align="center">Lamb and yearling mutton</td><td align="center">prime, choice, good, utility, cull</td></tr><tr><td align="center">Mutton</td><td align="center">choice, good, utility, cull</td></tr><tr><td align="center">Veal and calf</td><td align="center">prime, choice, good, standard, utility</td></tr></tbody></table></html><html><a name="Figure_7-14"><center></html>// Figure_7-14: Content example: USDA prime, choice, and select stamps. //<html></center></a></html>
-<html><center><img width="65%" src="http://buildingreputation.com/lib/exe/fetch.php?media=Ch07-Fig_7-14-USDAMeatGrades.png"/></center></html>+<html><center><img width="65%" src="http://buildingreputation.com/lib/exe/fetch.php?media=Figure_7-14.png"/></center></html>
-<html><a href="#Table_7-1">Table_7-1</a>&nbsp;</html>and <html><a href="#Figure_7-14">Figure_7-14</a>&nbsp;</html>show the meat grading levels use by the United States Department of Agriculture.The labels are descriptive, representing existing industry terms, and several are shared across different animal species - providing consumers a consistent standard for comparison.+<html><a href="#Table_7-1">Table_7-1</a>&nbsp;</html>and <html><a href="#Figure_7-14">Figure_7-14</a>&nbsp;</html>show the meat grading levels use by the United States Department of Agriculture.The labels are descriptive, representing existing industry terms, and several are shared across different animal species-providing consumers a consistent standard for comparison.
  * Named levels are particularly useful numeric levels are too impersonal or encourage undesired competition.   * Named levels are particularly useful numeric levels are too impersonal or encourage undesired competition.
-  * If you're considering using numeric levels, but find that the top and bottom levels should feel closer together than the numeric distance between them would otherwise indicate - consider using named levels instead. This is especially useful with karma scores so that new participants don't get stuck with a demeaning level indicator, like "Level 1 of 10"+  * If you're considering using numeric levels, but find that the top and bottom levels should feel closer together than the numeric distance between them would otherwise indicate-consider using named levels instead. This is especially useful with karma scores so that new participants don't get stuck with a demeaning level indicator, like “Level 1 of 10”
-<html><a name="Figure_7-15"><center></html>// Figure_7-15: Karma example: The WikiAnswers contributor levels design has seen several awkward expansions. //<html></center></a></html> +<html><a name="Figure_7-15"><center></html>// Figure_7-15: Karma example: The contributor levels on WikiAnswers have seen several awkward expansions. //<html></center></a></html> 
-<html><center><img width="65%" src="http://buildingreputation.com/lib/exe/fetch.php?media=Ch08-WikiAnswersContributorLevels.png"/></center></html>+<html><center><img width="65%" src="http://buildingreputation.com/lib/exe/fetch.php?media=Figure_7-15.png"/></center></html>
-<html><a href="#Figure_7-15">Figure_7-15</a>&nbsp;</html>displays the current named levels used by WikiAnswers.com for user contributions. The original three categories were Bronze, Silver, and Gold - named after competitive medals. They are granted when non-linearly increasing thresholds are met. Over time, the system has been expanded on three separate occasions to reward the nearly compulsive contributions of a handful of users.+<html><a href="#Figure_7-15">Figure_7-15</a>&nbsp;</html>displays the current named levels used by WikiAnswers.com for user contributions. The original three categories were Bronze, Silver, and Gold-named after competitive medals. They are granted when non-linearly increasing thresholds are met. Over time, the system has been expanded on three separate occasions to reward the nearly compulsive contributions of a handful of users.
<html><table align="center" border="1" class="inline"><thead><tr><td align="center">Pros</td><td align="center">Cons</td></tr></thead><tbody><tr><td align="left"> <html><table align="center" border="1" class="inline"><thead><tr><td align="center">Pros</td><td align="center">Cons</td></tr></thead><tbody><tr><td align="left">
<ul><li>Hiding level numbers allows for more expressiveness.</li><li>Level names can be thematically appropriate to, and vary by, your application(s).</li><li>Common hierarchies work well-for example, poor, average, good, and excellent.</li><li>This pattern is usually stronger when the named levels are displayed alongside other ratings, such as stars, points, and raw scores, to clarify them.</li></ul> <ul><li>Hiding level numbers allows for more expressiveness.</li><li>Level names can be thematically appropriate to, and vary by, your application(s).</li><li>Common hierarchies work well-for example, poor, average, good, and excellent.</li><li>This pattern is usually stronger when the named levels are displayed alongside other ratings, such as stars, points, and raw scores, to clarify them.</li></ul>
Line 409: Line 411:
** Using Ranked Lists ** ** Using Ranked Lists **
-  * Use leaderboards //for content// liberally. Provide filtered views of the boards to slice and dice by time ("Popular Today/This Week/All Time") or by reputation type ("Most Viewed/Top Rated").+  * Use leaderboards //for content// liberally. Provide filtered views of the boards to slice and dice by time (“Popular Today/This Week/All Time” ) or by reputation type (“Most Viewed/Top Rated” ).
<html><a name="Figure_7-16"><center></html>// Figure_7-16: Content example: YouTube's most-viewed videos. //<html></center></a></html> <html><a name="Figure_7-16"><center></html>// Figure_7-16: Content example: YouTube's most-viewed videos. //<html></center></a></html>
-<html><center><img width="65%" src="http://buildingreputation.com/lib/exe/fetch.php?media=YouTubeLeaderboard.png"/></center></html>+<html><center><img width="65%" src="http://buildingreputation.com/lib/exe/fetch.php?media=Figure_7-16.png"/></center></html>
<html><a href="#Figure_7-16">Figure_7-16</a>&nbsp;</html>shows YouTube's leaderboard ranking for most-viewed videos as a grid. With numbers this high, it's hard for potential reputation abusers to push inappropriate content onto the first page. Note that there are several leaderboards, one each for Today, This Week, This Month, and All Time. <html><a href="#Figure_7-16">Figure_7-16</a>&nbsp;</html>shows YouTube's leaderboard ranking for most-viewed videos as a grid. With numbers this high, it's hard for potential reputation abusers to push inappropriate content onto the first page. Note that there are several leaderboards, one each for Today, This Week, This Month, and All Time.
  * Use leaderboards //for people// sparingly, and only in contexts that are competitive by nature. Consider giving people leaderboards narrow scope (for example, only ranking me against my friends, to keep the comparisons fun and the stakes low).   * Use leaderboards //for people// sparingly, and only in contexts that are competitive by nature. Consider giving people leaderboards narrow scope (for example, only ranking me against my friends, to keep the comparisons fun and the stakes low).
-<html><a name="Figure_7-17"><center></html>// Figure_7-17: Karma example: Yahoo! Answers //<html></center></a></html> +<html><a name="Figure_7-17"><center></html>// Figure_7-17: Karma example: Yahoo! Answers leaderboard //<html></center></a></html> 
-<html><center><img width="65%" src="http://buildingreputation.com/lib/exe/fetch.php?media=YahooAnswersLeaderboard.png"/></center></html>+<html><center><img width="65%" src="http://buildingreputation.com/lib/exe/fetch.php?media=Figure_7-17.png"/></center></html>
<html><a href="#Figure_7-17">Figure_7-17</a>&nbsp;</html>displays Yahoo! Answer's leaderboard. The original version of this page was based solely on the number of points accumulated by participation, users quickly figured out which actions produced the most points for the least effort. When the user's best-answer percentage was eventually added to the profile display, it was discovered that the top-ranked users all had quality scores of less than 10%! <html><a href="#Figure_7-17">Figure_7-17</a>&nbsp;</html>displays Yahoo! Answer's leaderboard. The original version of this page was based solely on the number of points accumulated by participation, users quickly figured out which actions produced the most points for the least effort. When the user's best-answer percentage was eventually added to the profile display, it was discovered that the top-ranked users all had quality scores of less than 10%!
Line 432: Line 434:
  * Use top-X leaderboards //for content// to highlight only the best of the best contributions in your community.   * Use top-X leaderboards //for content// to highlight only the best of the best contributions in your community.
<html><a name="Figure_7-18"><center></html>// Figure_7-18: Content example: BillBoard's Hot 100. //<html></center></a></html> <html><a name="Figure_7-18"><center></html>// Figure_7-18: Content example: BillBoard's Hot 100. //<html></center></a></html>
-<html><center><img width="65%" src="http://buildingreputation.com/lib/exe/fetch.php?media=BillboardHot100.png"/></center></html>+<html><center><img width="65%" src="http://buildingreputation.com/lib/exe/fetch.php?media=Figure_7-18.png"/></center></html>
<html><a href="#Figure_7-18">Figure_7-18</a>&nbsp;</html>shows a Top-X display for content: the BillBoard's Hot 100's list of top recordings. The artists themselves have very little, if any, direct influence over their song's rank on this list. <html><a href="#Figure_7-18">Figure_7-18</a>&nbsp;</html>shows a Top-X display for content: the BillBoard's Hot 100's list of top recordings. The artists themselves have very little, if any, direct influence over their song's rank on this list.
  * Use top-X designations //for people// sparingly, and only in contexts that are competitive by nature. Because available categories in a top-X system are bounded, they will have greater perceived value in the community.   * Use top-X designations //for people// sparingly, and only in contexts that are competitive by nature. Because available categories in a top-X system are bounded, they will have greater perceived value in the community.
-<html><a name="Figure_7-19"><center></html>// Figure_7-19: Karma example: Amazon's top reviewers. //<html></center></a></html> +<html><a name="Figure_7-19"><center></html>// Figure_7-19: Karma example: Amazon's top reviewer rankings. //<html></center></a></html> 
-<html><center><img width="65%" src="http://buildingreputation.com/lib/exe/fetch.php?media=AmazonTopReviewers.png"/></center></html>+<html><center><img width="65%" src="http://buildingreputation.com/lib/exe/fetch.php?media=Figure_7-19.png"/></center></html>
-<html><a href="#Figure_7-19">Figure_7-19</a>&nbsp;</html>displays the //new// index of Top-X karma for Amazon.com review writers. The very high number of reviews written by each of these leaders creates value both for Amazon and the reviewers themselves. Authors and publishers seek them out to review/endorse their book - sometimes for a nominal fee. The original version of this reputation system, now known as "Classic Reviewer Rank", suffered deeply from first-mover effects (see <html><a href="/doku.php?id=Chapter_3#Chap_3-First_Mover_Effects">Chap_3-First_Mover_Effects</a>&nbsp;</html>) and other problems detailed in this book. This eventually lead to the creation of the new model, as pictured.+<html><a href="#Figure_7-19">Figure_7-19</a>&nbsp;</html>displays the //new// index of Top-X karma for Amazon.com review writers. The very high number of reviews written by each of these leaders creates value both for Amazon and the reviewers themselves. Authors and publishers seek them out to review/endorse their book-sometimes for a nominal fee. The original version of this reputation system, now known as “Classic Reviewer Rank” , suffered deeply from first-mover effects (see <html><a href="/doku.php?id=Chapter_3#Chap_3-First_Mover_Effects">Chap_3-First_Mover_Effects</a>&nbsp;</html>) and other problems detailed in this book. This eventually lead to the creation of the new model, as pictured.
<html><table align="center" border="1" class="inline"><thead><tr><td align="center">Pros</td><td align="center">Cons</td></tr></thead><tbody><tr><td align="left"> <html><table align="center" border="1" class="inline"><thead><tr><td align="center">Pros</td><td align="center">Cons</td></tr></thead><tbody><tr><td align="left">
<ul><li>Highly motivating for top performers. The prestige of earning a top-10 or top-100 designation may make contributors work twice as hard to keep it.</li><li>Yields a small, bounded set of entities to promote as high quality.</li></ul> <ul><li>Highly motivating for top performers. The prestige of earning a top-10 or top-100 designation may make contributors work twice as hard to keep it.</li><li>Yields a small, bounded set of entities to promote as high quality.</li></ul>
Line 447: Line 449:
==== Practitioner's Tips ==== ==== Practitioner's Tips ====
<html><a name='Chap_7-Leaderboards_Considered_Harmful'></a></html> <html><a name='Chap_7-Leaderboards_Considered_Harmful'></a></html>
-=== Be Careful with Leaderboards === +=== Leaderboards Considered Harmful === 
-It's too soon in the evolution of social media applications to make absolute statements about the design of social media sites, but one thing is clear: it's usually a bad idea to rank the members of your community and pit them against one another. Like a genie, a leaderboard promises riches (comparisons! incentives! user engagement!) that often lead to undesired consequences.+It's still too early to speak in absolutes about the design of social-media sites, but one fact is becoming abundantly clear: ranking the members of your community-and pitting them one-against-the-other in a competitive fashion-is typically a bad idea. Like the fabled //djinni// of yore, leaderboards on your site promise riches (comparisons! incentives! user engagement!!) but often lead to undesired consequences.
The thought process involved in creating leaderboards typically goes something like this: there's an activity on your site that you'd like to promote; a number of people are engaged in that activity who should be recognized; and a whole bunch of other people won't jump in without a kick in the pants. Leaderboards seem like the perfect solution. Active contributors will get their recognition: placement at the top of the ranks. The also-rans will find incentive: to emulate leaders and climb the boards. The thought process involved in creating leaderboards typically goes something like this: there's an activity on your site that you'd like to promote; a number of people are engaged in that activity who should be recognized; and a whole bunch of other people won't jump in without a kick in the pants. Leaderboards seem like the perfect solution. Active contributors will get their recognition: placement at the top of the ranks. The also-rans will find incentive: to emulate leaders and climb the boards.
-And that activity you're trying to promote? Site usage should swell with all those earnest, motivated users plugging away, right? It's the classic win-win-win scenario. In practice, the use of a leaderboard display pattern has rarely played out in a straightforward way. Here are just a few reasons why leaderboards are hard to get right.+And that activity you're trying to promote? Site usage should swell with all those earnest, motivated users plugging away, right? It's the classic win-win-win scenario. In practice, employing this pattern has rarely been this straightforward. Here are just a few reasons why leaderboards are hard to get right.
== What Do You Measure? == == What Do You Measure? ==
-Many leaderboards make the mistake of basing standings only on what is easy to measure. Unfortunately, what's easy to measure often tells you nothing at all about what is good. Leaderboards tend to fare well in very competitive contexts, because there's a convenient correlation between measurability and quality. (It's called "performance"-number of wins versus losses within overall attempts.)+Many leaderboards make the mistake of basing standings only on what is easy to measure. Unfortunately, what's easy to measure often tells you nothing at all about what is //good//. Leaderboards tend to fare well in very competitive contexts, because there's a convenient correlation between measurability and quality. (It's called “performance” -number of wins versus losses within overall attempts.)
But how do you measure quality in a user-generated video community? Or a site for ratings and reviews? It should have very little to do with the quantities of simple activity that a person generates (the number of times an action is repeated, a comment given or a review posted). But such measurements-discrete, countable, and objective-are exactly what leaderboards excel at. But how do you measure quality in a user-generated video community? Or a site for ratings and reviews? It should have very little to do with the quantities of simple activity that a person generates (the number of times an action is repeated, a comment given or a review posted). But such measurements-discrete, countable, and objective-are exactly what leaderboards excel at.
Line 462: Line 464:
== Whatever You Do Measure Will Be Taken Way Too Seriously == == Whatever You Do Measure Will Be Taken Way Too Seriously ==
-Even if you succeed in leavening your leaderboard with metrics for quality (perhaps you weigh community votes or count send-to-a-friend actions), be aware that-because a leaderboard singles out these factors for praise and reward-your community will hold them in high esteem too. Leaderboards have a kind of "Code of Hammurabi" effect on community values: what's written becomes the law of the land. You'll likely notice that effect in the activities that people will-and won't-engage in on your site. So tread carefully-are you really that much smarter than your community, that you alone should dictate its character?+Even if you succeed in leavening your leaderboard with metrics for quality (perhaps you weigh community votes or count send-to-a-friend actions), be aware that-because a leaderboard singles out these factors for praise and reward-your community will hold them in high esteem too. Leaderboards have a kind of “Code of Hammurabi” effect on community values: what's written becomes the law of the land. You'll likely notice that effect in the activities that people will-and won't-engage in on your site. So tread carefully-are you really that much smarter than your community, that you alone should dictate its character?
-== If It Looks Like a Leaderboard and Quacks Like a Leaderboard . . . ==+== If It Looks Like a Leaderboard and Quacks Like a Leaderboard... ==
Even sites that don't display overt leaderboards may veer too closely into the realm of comparative statistics. Consider Twitter and its prominent display of community members' stats. Even sites that don't display overt leaderboards may veer too closely into the realm of comparative statistics. Consider Twitter and its prominent display of community members' stats.
-<html><a name="Figure_7-20"><center></html>// Figure_7-20: You'd be utterly forgiven if you signed into Twitter and mistook the dashboard for a scoreboard! //<html></center></a></html> +<html><a name="Figure_7-20"><center></html>// Figure_7-20: You'd be completely forgiven if you signed into Twitter and mistook this dashboard for a scoreboard! //<html></center></a></html> 
-<html><center><img width="50%" src="http://buildingreputation.com/lib/exe/fetch.php?media=Ch08-TwitterFollowers.png"/></center></html>+<html><center><img width="50%" src="http://buildingreputation.com/lib/exe/fetch.php?media=Figure_7-20.png"/></center></html>
-The problem may not lie with the //existence// of the stats but in the prominence of their display. (See <html><a href="#Figure_7-20">Figure_7-20</a>&nbsp;</html>) They give Twitter the appearance of a community that values popularity and the sheer size of a participant's social network. Is it any wonder, then, that a whole host of community-created leaderboards have sprung up to automate just such comparisons? Twitterholic, Twitterank, Favrd, and a whole host of others are the natural extension of this value-by-numbers approach.+The problem may not lie with the //existence// of the stats but in the prominence of their display. (<html><a href="#Figure_7-20">Figure_7-20</a>&nbsp;</html>) They give Twitter the appearance of a community that values popularity and the sheer size of a participant's social network. Is it any wonder, then, that a whole host of community-created leaderboards have sprung up to automate just such comparisons? Twitterholic, Twitterank, Favrd, and a whole host of others are the natural extension of this value-by-numbers approach.
== Leaderboards Are Powerful and Capricious == == Leaderboards Are Powerful and Capricious ==
-In the earliest days of Orkut (Google's also-ran entry in social networking), the product managers put a fun little widget at the top of the site: a country counter, showing where members were from. Cute and harmless, right? Google had no way of knowing, however, that seemingly the entire population of Brazil would make it a point of national pride to push their country to the top of that list. Brazilian blogger Naitze Teng wrote: "Communities dedicated to raising the number of Brazilians on Orkut were following the numbers closely, planning gatherings and flash mobs to coincide with the inevitable. When it was reported that Brazilians had outnumbered Americans registered on Orkut, parties . . . were thrown in celebration."+In the earliest days of Orkut (Google's also-ran entry in social networking), the product managers put a fun little widget at the top of the site: a country counter, showing where members were from. Cute and harmless, right? Google had no way of knowing, however, that seemingly the entire population of Brazil would make it a point of national pride to push their country to the top of that list. Brazilian blogger Naitze Teng wrote: “Communities dedicated to raising the number of Brazilians on Orkut were following the numbers closely, planning gatherings and flash mobs to coincide with the inevitable. When it was reported that Brazilians had outnumbered Americans registered on Orkut, parties... were thrown in celebration.
-Brazil maintained its number one position on Orkut (as of this writing, 51% of Orkut users are Brazilian; the United States and India are tied for a distant second with 17% apiece). Orkut is basically a Brazilian social network. That's not a bad "problem" for Google to have, but it's probably not an outcome that Google would have expected from a device like a leaderboard widget.+Brazil has maintained its number one position on Orkut (as of this writing, 51% of Orkut users are Brazilian; the United States and India are tied for a distant second with 17% apiece). Orkut today is basically a Brazilian social network. That's not a bad “problem” for Google to have, but it's probably not an outcome that they would have expected from such a simple, small and insignificant thing as a leaderboard widget.
Line 483: Line 485:
The most insidious artifact of a leaderboard community may be that the very presence of a leaderboard changes the community dynamic and calls into question the motivations for every action that users take. If that sounds a bit extreme, consider Twitter: friend counts and followers have become the coins of that realm. When you get a notification of a new follower, aren't you just a little more likely to believe that it's just someone fishing around for a reciprocal follow? Sad, but true. And this despite the fact that Twitter itself never has officially featured a leaderboard-it merely made the statistics known and provided an API to get at them. In doing so, it may have let the genie out of the bottle. The most insidious artifact of a leaderboard community may be that the very presence of a leaderboard changes the community dynamic and calls into question the motivations for every action that users take. If that sounds a bit extreme, consider Twitter: friend counts and followers have become the coins of that realm. When you get a notification of a new follower, aren't you just a little more likely to believe that it's just someone fishing around for a reciprocal follow? Sad, but true. And this despite the fact that Twitter itself never has officially featured a leaderboard-it merely made the statistics known and provided an API to get at them. In doing so, it may have let the genie out of the bottle.
-<note>“Leaderboards Considered Harmful” first appeared as an essay in//Designing Social Interfaces//(O'Reilly, 2009) by Christian Crumlish and Erin Malone, also available online at [[http://www.designingsocialinterfaces.com/|DesigningSocialInterfaces.com]].+<note tip>“Leaderboards Considered Harmful” first appeared as an essay in//Designing Social Interfaces//(O'Reilly, 2009) by Christian Crumlish and Erin Malone, also available online at [[http://www.designingsocialinterfaces.com/|DesigningSocialInterfaces.com]].
</note> </note>
 +
 +=== Going Beyond Displaying Reputation ===
 +This entire chapter has focused on the explicit display of reputation, usually directly to users. Though important, this isn't usually the most valuable use for this information. <html><a href="/doku.php?id=Chapter_8">Chapter_8</a>&nbsp;</html>describes using reputation to modify the utility of an application-to separate the best entities from the pack, and to help identify and destroy the most harmful ones.
 +
chapter_7.txt · Last modified: 2009/12/01 14:37 by randy
www.chimeric.de Creative Commons License Valid CSS Driven by DokuWiki do yourself a favour and use a real browser - get firefox!! Recent changes RSS feed Valid XHTML 1.0