===== Planning Your System's Design ===== ==== Asking the Right Questions ==== In reputation system design-as with //most// endeavors in life-it turns out that you'll get much better answers out of the process if you spend a little time upfront considering the right questions. This is that point in the process where we'll pause to do just that. We'll explore some very simple questions-Why are we doing this? What do we hope to get out of it? How will we know we've succeeded? The answers to these questions will undoubtedly be not quite so simple. Community sites on the Web vary wildly in their makeup: different cultures, customs, business models and rules for behavior. Designing a successful reputation system means designing a system that's successful for //your// particular set of circumstances. Which means that you'll need to have a fairly well-tuned understanding of your community and the social outcomes that you hope to affect. We'll get you started. Some careful planning and consideration //now// will save you a world of heartache later. The questions we'll help you answer in this chapter are: - What are your goals for your application? * Is generating revenue based on your content a primary focus of your business? * Will user activity be a critical measure of success? * What about loyalty (keeping the same users coming back month after month)? - What is your Content Control Pattern? * Who is going to create, review, and moderate your site's content and/or community? * What services do staff and professional feed content or external services provide? * What roles do the users play? Based on the answer to these questions, you will find out how comprehensive a reputation system you need. In some cases this will be none at all! Each content service pattern includes recommendations and examples of incentive models to consider for your system. - How do the incentive models that are appropriate for my goals and Content Control Pattern work? * Are you depending on //altruistic, commercial,// or //egocentric// incentives or some combination of them? * Which of these work best with karma? Which abhor it? Understanding how the various incentive models have been demonstrated to work in similar environments will narrow down the choices you make as you proceed to Chapter_7 . === What are your Goals? Start Selfish === Marney Beard, a long-time project manager at Sun Microsystems, had a wonderful statement about participating in any team endeavor. Marney used to say "It's alright to //start// selfish. As long as you don't //end// there." (This was actually advice that Marney first gave her kids, but it turns out to be equally applicable in the compromise-laden world of product development.) So, following Marney's excellent advice, we encourage you to-for a moment-take a very selfish and self-centered view of your plans for a rich reputation system for your website. Yes, ultimately, your system will be a balance between your goals, your community's desires, and the tolerances and motivations of everyone that comes to the site. But-for now-let's just talk about //you//. Why do you want to implement a reputation system for your site? Are you primarily interested in... == Business Goals == Both content and people reputations can be used to strengthen one or more aspects of your business. There is no shame in admitting this! As long as we're starting selfish, let's get downright crass. How can imbuing your site with an historical sense of people and content reputation help your bottom line? ** User Engagement ** Perhaps you'd like to deepen user engagement-either the amount of time that users spend in your community, or the number and breadth of activities that they partake in. For a host of reasons, more-engaged users are more-valuable users, both to your community and to you, from a business sense. Offering an incentive system to users may persuade them to try more of your offering than their own natural curiosity alone would. Offline marketers have long been aware of this fact-such promotions as sweepstakes, contests and giveaways are all designed to influence customer behavior and deepen engagement. User engagement can be defined in many different ways. Eric T. Peterson offers up a pretty good list of basic metrics for engagement. He posits that those users are most engaged who: * view critical content on your site * have //returned// to your site recently (multiple visits) * return //directly// to your website some of the time (not via a link) * have long sessions, with some regularity, on your site * have subscribed to at least one of your site's available data feeds List adapted from http://blog.webanalyticsdemystified.com/weblog/2006/12/how-do-you-calculate-engagement-part-i.html This list is good, though it is definitely skewed toward an advertiser's or a content-publishers view of engagement on the Web. It's also loaded with subjective measures-what constitutes a "long" session? Which content is "critical?" But that's fine! We want subjective-at this point, we want to tailor our reputation approach to exactly what we hope to get out of it! So what would a good set of metrics be to determine //community engagement// on your site? Again, the best answer to that question //for you// will be intimately tied to the goals that you're trying to achieve. Tara Hunt, as part of a longer discussion on Metrics for Healthy Communities offers the following suggestions: * the rate of attrition in your community, especially with new members * the average length of time it takes for a newbie to become a regular contributor * multiple community crossover-if your members are part of many communities, how do they interact with your site? Flickr photos? Twittering? Etc.? * the number giving as well as the receiving actions-eg. readers receive, posters are giving (advice, knowledge, etc.). * community participation in gardening, policing and keeping the community a nicer place (eg. people who report content as spam, people who edit the wiki for better layout, etc.) List adapted from http://www.horsepigcow.com/2007/10/03/metrics-for-healthy-communities/ This is a good list, but still highly subjective. Once you decide //how// you'd like your users to engage with your site and community, then you'll need to determine how to measure that engagement. // Figure_6-1: No matter how you measure it, Bernadette e. is one active and engaged Yelper.' // {{Ch06-EngagedYelpUser.png}} ** Speaking of Metrics... ** Later, in Chapter_10 , we'll ask you to evaluate your site's performance against the goals that you define in this chapter. Obviously, this exercise will be greatly aided if you have actual data to compare from before you roll out your reputation system to afterward. It's a good idea to anticipate the metrics that will help you in performance evaluation, and make sure //now// that your site or application is properly instrumented to provide that data. And, of course, to ensure that data is being logged appropriately and saved for the time when you'll need it for decision-making, tweaking and tuning of your system. After all, there's nothing quite like doing a 'before and after' comparison, only to realize that you weren't keeping proper data //before//... ** Establishing Loyalty ** Or maybe you're interested in building brand loyalty with your visitors. Perhaps you'd like to establish a relationship with them that extends beyond the boundaries of one visit or one log-in. Yahoo! Fantasy Sports employs a fun reputation system, enhanced with nicely-illustrated trophies for achieving milestones (a winning season in a league) for various sports. // Figure_6-2: "Boca Joe" has played a variety of fantasy sports on Yahoo! since 2002. Do you suppose the reputation he's earned on the site helps brings him back each year? // {{Ch06-YahooFantasySportsProfile.png}} This simple feature serves many purposes: the trophies are fun and engaging; they may serve as incentive for community members to excel within a sport; they help extend a user's identity and give them a way to express their own unique set of interests and biases to the community; but they are //also// an effective way of establishing a continuing bond with Fantasy Sports players-one that persists from from season to season and sport to sport. Any time a Yahoo! Fantasy Sports user is considering a switch to a competing service (Fantasy Sports is big business! there are any number of very capable competitors in the space), the presence of these trophies provides tangible evidence of the //switching cost// for doing so: a reputation reset. ** Coaxing Out Shy Advertisers ** Perhaps you are concerned about your site's ability to attract advertisers. User-generated content is a hot Internet trend that's almost become synonymous with Web 2.0, but it has also been slow to attract advertisers. Particularly those big, traditional (but deep-pocketed!) companies who worry about the effect of displaying their own brand in close proximity to the Wild-West frontier ethos that's sometimes evident on sites like YouTube or Flickr. Once again, reputation systems offer a way out of this conundrum. By tracking the high-quality contributors and contributions on your site, you can guarantee to advertisers that the content their brand will be associated with will meet certain standards of quality. In fact, you can even craft your system to reward particular //aspects// of contribution-perhaps, for instance, you keep a 'Clean Contributor' reputation that takes into account a contributor's typical profanity level (and also considers Abuse reports against the same author.) Without some form of quality-and-legality-based filtering, there's simply //no way// that a prominent and respected advertiser like Johnson's would associate their brand with YouTube's (typically anything-goes) user-contributed videos. // Figure_6-3: The Johnson's Baby Channel on YouTube exhibits a higher-than-average level of trust in the quality of user submissions. // {{Ch06-SponsoredUGC.png}} Of course, another way to allay advertisers' fears is by generally improving the quality (both real and perceived) of content generated by the members of your community... ** Improving Content Quality ** Reputation systems //really// shine at helping you make value judgements about the relative quality of content that users submit to your site. Chapter 9 will focus on the myriad techniques for filtering out bad content and encouraging good quality contributions. There are any number of business benefits to keeping the perceived quality of content high on your site. We've already touched on a few (increased attraction to advertisers, the Virtuous Circle of user contributions.) But there are also some real, tangible cost benefits you may not have considered. === Content Control Patterns === [TBD] Define the taxonomy. Use a decision chart to point them to the correct pattern. Unpack each element, such as "Staff Evaluate". Deal with the common case that a site may need multiple content control patterns: i.e. the category hierarchy is created, evaluated and removed by staff, but the elements uploaded are created by the users with the staff doing all evaluation and removal. In that case both patterns apply - and since the posited category tree is a //Web 1.0// content control pattern, it can effectively be ignored when considering the selection of reputation models. == Content Control Pattern //Web 1.0//: Staff Create, Evaluate, and Remove == No reputation required, since users do nothing. Probably not worth the expense to build a content reputation system solely for staff - except maybe for searching a very large DB. == Content Control Pattern //Bug Report//: Staff Create and Evaluate, Users Remove == In this pattern, the users are attempting to petition for a change in the editorially created and reviewed database. This pattern is similar to the Chap_6-User-Submission-Pattern , though the moderation process is typically less expressive than a review. An example of this process is voting to hide content as inappropriate or out of context. [Example site TBD] Generally does not require reputation. == Content Control Pattern //Reviews//: Staff Create and Remove, Users Evaluate == First generation ratings are reviews: Early Amazon, Movie/Local/Product sites. Simple rate-only sites: Y!News most-emailed - Y!Buzz? Altruism Incentives. No karma required. Blogs - BoingBoing == Content Control Pattern //[TBD]//: Staff Create, Users Evaluate and Remove == [TBD] == Content Control Pattern //Submit-Publish//: Users Create, Staff Evaluate and Remove == Some recipe sites, early share your news, contest sites, etc. Need more examples here. Altruism. No Karma. == Content Control Pattern //[TBD]//: Users Create and Remove, Staff Evaluate == [TBD] Public Library of Science? == Content Control Pattern //Basic Social Media//: Users Create and Evaluate, Staff Remove == Most big sites. DCMA, COPA and COPPA are example reasons most sites stop their content openness here. Y!Answers lives here and adds just a little bit of User moderation. This eventually doesn't scale - then reputation is your only hope. Slashdot, Fark, YouTube == Content Control Pattern //The Full Monty//: Users Create, Evaluate, and Remove == Wiki's live here. Lots of reputation needs - consider Karma. Y!Answers === Incentives for user participation, quality, and moderation === == //Predictably Irrational// == Chapter 4 of the book //Predictably Irrational//, Duke University Professor of Behavioural Economics Dan Ariely is entitled //The Cost of Social Norms: Why We Are Happy to Do Things but Not When We are Paid to Do Them//. In it he describes a view of two separate and incentive exchanges for doing work and the norms that set the rules for them: he calls them social norms and market norms. Social norms govern doing work for other people because they asked you to - often because doing the favor makes you feel good. Ariely says these exchanges "wrapped up in our social nature and our need for community. They are usually warm and fuzzy." Market norms, on the other hand, are cold and mediated by wages, prices and cash: "There's nothing warm and fuzzy about it." It is the land of "you get what you pay for." Don't cross the streams: these two sets of norms don't mix well. Ariely gives several examples of confusion when these incentive models mix; First he describes a hypothetical scene after a family home-cooked holiday dinner where he offers to pay his mother $400.00 and the outrage that would cause (including that the social damage would be take a very long time to repair). The second less purely hypothetical and more common example is about dating and sex - a guy takes a girl out on a series of expensive dates. Should he expect increased social interaction - maybe at least a passionate kiss? "On the fourth date he casually mentions how much this romance is costing him. Now he's crossed the line Violation! ... He should have known you can't mix social and market norms-especially in this case-without implying that the lady is a tramp." //Predictably Irrational// then goes on to detail experiments that verify that there are significant differences between social and market exchanges, at least when it comes to very small units of work, much like the kinds of user created content tasks that the designers of web sites are looking to incentivize. The tasks were trivial - use a mouse to drag a circle into a square on a computer screen as many times as possible in five minutes. Three groups were tested - one was not offered any compensation, one was offered $0.50 the last was offered $5.00 for their time. Though the group paid $5 did more work than the one paid 50 cents,//the group that did the most work were those who were not offered any money at all!// When the money was substituted with a gift of the same value (Snickers Bar and a box of Godiva Chocolate), the work distinction went away - it seems that gifts behave in the domain of social norms and everyone worked as hard as those that were uncompensated. But! When the price sticker is left on the chocolates so that the subjects can see the monetary value of the reward, the market norms again applied and the striking difference in work results returns - volunteers again worked harder than those who received priced chocolates. == Incentives and Reputation == When considering how a reputation system might help with your content control pattern, be careful to consider the appropriate incentives for your users and the tasks you are asking them to do on your behalf. But also remember that you have particular goals for your application - sometimes this may lead selecting a different reputation model - try to accommodate both sets of needs. Ariely talked about two categories of norms - social and market, but for reputation systems we talk about three main groups of online incentive behaviors: //altruistic// (for the good of others), //commercial// (to generate revenue), and //egocentric// (for self gratification). Interestingly these map somewhat against social norms (altruism & egocentric) and market norms (commercial & egocentric). Notice that egocentric is listed under both norms, something usually to be avoided. This is because market-like reputation systems (points and virtual currencies) are being used to create successful work incentives for these users. In effect, egocentric crosses the two exchanges by creating a entirely new virtual world where these norms can co-exist in ways that we would find socially repugnant in the real world - in these worlds bragging is good! == Altruistic (Sharing) Incentives == The altruistic incentives represent the giving nature of the user - they have something to share - a story, a comment, a photo, an evaluation, etc. and they feel compelled to share it on your site. Their incentives are internal - even if they include a feeling of obligation to another user, a friend, or even loyalty (or hatred) for your brand. The altruistic or sharing patterns are: //Tit-for-Tat/Pay-it-Forward// (because someone else did it for me first), //Friendship// (because I care about those who will consume this), //Know-it-all/Crusader/Opinionated// (because I know something everyone else needs to know). This list is incomplete - if you know of others, please contribute them to the website for this book: [LINK]BuildingReptuation.com. When considering reputation models that create altruism incentives, remember that this is the realm of social norms - it is all about sharing, not about accumulating commercial value or karma points. Avoid aggrandizing the creator of altruistically created content - they don't want their contributions to be counted, recognized, ranked, evaluated, compensated, or rewarded in any significant way. Comparing their work to anyone else's is a significant discouragement to participation. ** Tit-for-tat/Pay It Forward ** Tit-for-tat is being used here to mean that a user decides to contribute based on a feeling of obligation to pay back a favor because they received benefit from either the site or from the other users of the site. In early social sites with content control patterns like //Reviews//, where users only evaluate staff provided content, there were no site-provided incentives to participate and users most often indicated that they contributed because a review on the site helped them. //Pay It Forward// is the title of a book written by Catherine Ryan Hyde that was made into a motion picture released in 2000 that introduced the concept of//pay it forward//: improve the state of the world by doing a unconditional and unrequested deed of kindness to another - with the hope that the recipient would do the same thing for one or more other people creating a never ending and always expanding world of altruism. We mention it here so that you might consider it as a model for corporate reputation systems that track altruistic contributions as an indicator of community health. ** Friendship ** In fall of 2004, when the Yahoo! 360Â° social network first introduced the vitality stream (now known as the //personal news feed// and improperly attributed as being invented by Facebook who copied it years later) it included events that were generated whenever your friends wrote a review of a restaurant or hotel for Yahoo! Local. From the day the service was opened to the public, Yahoo! Local saw a 45% sustained lift in the number of reviews written daily. The knowledge that friends would be notified when you wrote a review - in effect notifying them both of where you went and what you thought - became a much stronger altruistic motivator than Tit-for-tat. In the sited example above, a Yahoo! 360Â° user was more than fifty (50!) times more likely to write a review than a typical Yahoo! Local user. There's really no reputation system involved here, you simply make sure that friends can see the things that each other contribute to the site - either through news feed events or when a user searches for an item, or whenever they happen to encounter something a friend did before. ** Know-it-all/Crusader/Opinionated ** Lastly, there are people who have an internal motivation based on passion. Some of the passion is temporary, such has having a terrible customer experience and wishing to share their frustration with the anonymous masses and perhaps exact some minor revenge on the business in question. Some of the passion is the result of deeply held beliefs, such as religion or politics. Some is topical expertise and others are just killing time. In any case, people seem to have a lot to say that is of very mixed commercial value. Look at the comments on a YouTube video, but not for very long, your eyes might bleed. This group of altruistic motivations is a mixed bag - there are great contributions in here, but there's also a lot of junk as we mentioned in Chap_1-lots_of_crap . If you have reason to believe that a large portion of your most influential community members will be motivated by controversial ideas, carefully consider the costs of your evaluation and removal choices for your content control pattern. A large out-of-control community can be worse than having no community at all. // : Helpful has changed meanings to something akin to agreement in this movie review // {{Ch06-HelpfulBecomesAgreement.png}} Go to any movie review site and look at the reviews for //Fahrenheit 9/11// and //The Passion of the Christ// and look how people respond to each other's reviews. If the reviews have "Was this review helpful?" voting, look at the reviews with the highest total votes and how polarized they are. Clearly the word //helpful// in this context now means //agreement// with the author. == Commercial Incentives == Commercial incentives fall squarely in Ariely's market norms - people are doing this for the money, though the money may come in the form of //direct// payment from the user to the content creator. Advertisers have a nearly scientific understanding of the significant commercial value of something they call branding. Likewise, influential bloggers know they are posting to build their brand, often to become perceived as subject matter experts which may lead to improved standing - speaking engagements, consulting contracts, improved permanent positions at universities or prominent corporations, or even a book deal. A few may actually be directly paid to produce their online content, but more are capturing //indirect// commercial value. Reputation models in content control patterns with commercial incentives requires a much stronger sense of user identity and will need strong and distinctive profile thats provide links to their valuable contributions and content. An expert in, say the area of textile design will want, as part of their personal brand, want to share a set of links to content that they think is particularly noteworthy to their fans. But, don't confuse the need to support a strong profile/brand for the contributor with the need for a strong or prominent karma system. When a new brand is being introduced to a market, be it a new kind of dish soap or a new topical blogger, a karma system that favors established participants can be a disincentive. The community decides how to treat newcomers - with open arms or with suspicion. An example of the latter is eBay - where every new seller must "pay their dues" and bend over backward to get their first dozen or so positive evaluations before the market at large will embrace them as a trustworthy vendor. The goals of your application should help you decide if you need karma in your commercial incentive model. One possible rule of thumb: If users are going to be passing money directly to other people they don't know, consider adding karma to help establish trust. ** Direct Revenue ** By direct revenue we mean: whenever a someone forks over hard earned cash related to the consumption of the content contributors work and it ends up, even with an intermediary or two, and some or all of it ends up in the contributors hands. This could be the result of a subscription, short term contract, a goods or services transaction, on-content advertising like Google's AdSense, or even a PayPal based tip jar. When it comes to real money, people get serious about issues of trust, and reputation systems play a critical role in helping establish trust. Without a doubt, the most well known - and studied - reputation system for online direct revenue business is eBay's buyer and seller feedback (karma) system. Without a way for strangers to gage the trustworthiness of the other party, there could be no online auction market at all. When considering reputation systems for applications with direct revenue incentives, step back and make sure that you might not be better off with an either altruistic or egocentric model - despite what you may have been taught in school, money is //not// always the best motivator and is a pretty big barrier to entry for consumers. The ill-fated Google Answers failed because it had a user-to-user direct revenue incentive model where competing sites, such as WikiAnswers provided similar results for free financed, ironically, by using AdSense to monetize answer pages which are indexed by, you guessed it, Google! ** The Zero Price Effect: Free is disproportionately better than cheap! ** In //Predictably Irrational,// Dan Ariely details an series of experiments to show that people have an irrational urge to chose a free item over a unusually low-priced but higher quality item: First he offered people a single choice between buying a 1-cent Hershey's Kiss and a 15-cent Lint truffle and most people bought the higher quality truffle. But, when he dropped the price both items by one penny, making the Kiss //free//, a dramatic majority of a new group of buyers instead selected the Kiss! He calls this the Zero Price Effect. For designing incentive systems it provokes two thoughts: * Don't delude yourself that sufficiently low-pricing in a user-to-user direct revenue incentive design will overcome the Zero Price Effect. * Even if you give away user contributions for free you can still have direct revenue: charge advertisers/sponsors instead of the consumers directly. ** Branding - Professional Promotion ** The indirect form of commercial incentives can be lumped under the word //branding//, the process of professional promotion of people, goods, or organizations. The advertiser's half of the direct revenue incentive model lives here as well. The goal is to expose the audience to your message and eventually capture value in the form of a sale, subscriber, or job. There is typically a large number of steps between branding activities: writing blog posts, running ads, creating widgets to be embedded on other sites, participating in online discussions, attending conferences, etc. and the desired effects. Reputation systems are one way to have users close the loop with direct feedback as well as help measure the success of these activities. // : Awe.sm turns URL-shortening into a reputation system, measuring how many people clicked your URL as well as how many shared it with others // {{Ch06-AweSmReputation.png}} Take the simple act of sharing a URL on a social site, such as Twitter - without a reputation system you have no idea how many people followed your recommendation, nor how many other people shared it as well. The URL-shortening service Awe.sm provides both features - it tracks how many people clicked on your URL and how many different people shared the URL with others. For contributors that are building their brand, public karma systems are a double-edged sword - when they are at the top of their perspective markets, their karma can be seen as a big indicator of trustworthiness. For new entrants, most karma scores can't differentiate between inexperience (or just coming to the web-game later than peers) and incompetence. This problem can be addressed by making sure to including the ability to include time-limited scores in the mix. For example, B.F. Skinner is a world renowned and respected behaviourist, but that does me no good when I'm looking for a Educational Psychology thesis advisor - he's dead. == Egocentric Incentives == Found online most often in computer games, egocentric incentives are tied to deeply wired self-centered motives. Some of these incentive systems tap into motivations are described in behavioral psychology with terms such as a //classical/operant conditioning// (where animals were trained to respond to food-related stimulus) and schedules of reinforcement: //fixed, continuous//, and //variable//, //interval// and //ratio//- all leading to the conclusion that people can be influenced to repetitively perform simple tasks by providing periodic rewards, even if as simple as a pleasing sound. Care must be taken with considering the powerful effects of using reputation to drive egocentric incentives. Looking at an individual's animals behavior, in a social vacuum, as many of the classical behaviorists did does not reflect how we very-social humans reflect our egocentric behaviors //to each other//. Humans have make teams and compete in tournaments. We follow leader boards comparing our selves - even our companies' - performance to others. Even if it doesn't help another soul or generate us any revenue, we often want to feel //recognition// for our accomplishments. Even if we don't seek accolades from our peers, we want to be able to //demonstrate mastery// of something - to hear that little feedback from a system that says "You did it! Good Job!" When creating reputation systems utilizing egocentric incentives, a user profile becomes a critical requirement of your system. The user needs some place to show off their accomplishments - even if only to themselves. Almost by definition, egocentric incentives are one or more forms of karma. Even if you select a simple system of granting trophies for achievements, the users will compare their collections to each other. New norms will appear that look more like market norms than social norms: people will trade favors to both advance their karma, people will attempt to cheat to get an advantage, and those who feel they can't compete will //opt-out// all together! Yes, egocentric incentives and karma provide very powerful motivations, but these are almost antithetical to altruistic ones. There have been many systems that have over-designed to reinforce the egocentric gaming contingent and ended up with an almost entirely experts-only customer base: consider just about any online role playing game that survived more than three years. For example, Worlds of Warcraft must continually produce new content for it's highest-level users in order to retain them and their revenue stream, all but abandoning improvements to increase its acquisition of new users - stunting growth. When new users do arrive (usually the result of a marketing promotion), they end up playing alone because all the veterans are too advanced and distracted by the new content to bother to go through the leveling process again. ** Recognition ** [TBD] ** The Quest for Mastery ** [TBD] It's all a game. === Consider Your Community === Introducing a reputation system into your community will almost certainly affect its character and behavior in some way. Some of these effects will be positive (hopefully! I mean, that's why you're reading this book, right?) But there are potentially negative side-effects to be aware of as well. It is almost impossible to predict exactly what community effects will result from implementing a reputation system because-and we bet you can guess what we're going to say here!-it is so bound to the particular context that you are designing for. But here are a number of community factors to consider early in the process. == What are people here to do? == This may seem like a simple question, but it's one that often goes unasked: what, exactly, is the purpose of this community? What are the actions, activities and engagements that they expect when they come to your community? == Is this a new community? Or an established one? == Many of the models and techniques that we're covering are equally applicable whether your community is a brand-new, aspiring one or has been around awhile and already enjoys a certain dynamic. However, it may be slightly more difficult to introduce a robust reputation system into an existing and thriving community than it would have been to bake reputation in from the beginning. Why is this? * In an established community, there are already shared mores and customs. Whether the community rules have been formalized or not, there are indeed expectations for how to participate in the community. As well as an understanding of what types of actions and behaviors are viewed as transgressive. The more-established and more strongly held these community values are, the better a job of matching your reputation systems inputs and rewards to those values you must do. * With some communities, you may already be running into problems of //scale// that you would not encounter with an early-stage or brand new site. There might already be an overwhelming amount of conversation (or noise, depending on how you look at it.) And you could be faced with migration issues: * Should you 'grandfather' old content, and just leave it out of the new system? * Will you expect people to go back and retroactively grade old content? (In all likelihood, they won't.) * In particular, changes to //an existing reputation system// may be difficult. ////Whether as trivial and opaque as tweaking some of the parameters that determine a video's "popularity" or as visible and significant as introducing a new level designation for top performers, you are likely to encounter resistance (or at the very least, curiosity) from your community. You are, in effect, changing the rules for your community, so expect the typical reaction: some will welcome the changes, others (typically, those who benefited most under the old rules) will denounce them.//// We don't mean to imply, however, that designing a reputation system for a new "green-field" community is an easy task: in these circumstances, rather than identifying the characteristics of your community that you'd like to enhance (or leave unharmed), your task will be to //imagine// the community effects that you're hoping to influence, then make smart decisions to affect those outcomes. In any event, it is always worthwhile to consider... == What //size// is the community? == == What is your community's character? What would you like it to be? == Is your community a friendly, welcoming place? Helpful? Collaborative? Argumentative or spirited? Downright //combative//? Communities can display a whole range of behaviors and it can be dangerous to generalize too much about any specific community, but you must at least consider the overall character of the community that your reputation system seeks to influence. ** The Competitive Spectrum ** Though it's not the only aspect worth considering, a very telling one is the level of perceived competitiveness in your community: the individual goals of community members, and to what degree those goals coexist peacefully, or conflict; the actions that community-members engage in, and to what degree those actions may impinge on the experiences of other community-members; and to what degree inter-person comparisons or contests are desired. Consider your community for a moment, and where it might fall along the competitive spectrum in Figure_6-4 . // Figure_6-4: Community Competitive Spectrum // {{Ch06-CompetitiveSpectrum.jpg}} In general, more competitive a group of people in a community are, the more appropriate it is to compare those people (and the artifacts that they generate) against each other. Read that last bit again, and carefully! A common mistake that product architects (especially for social web experiences) make is //assuming// that a higher level of competitiveness exists than what's really there. Because reputation systems and their attendant incentive systems are often intended to emulate the principles of engaging game designs, designers often gravitate toward the aggressively competitive-and comparative-end of the spectrum. Even the intermediate stages along the spectrum can be deceiving. For example, where would you place a community like Match.com or Yahoo! Personals along the spectrum? Perhaps your first instinct was to say "I would place a dating site firmly at the 'Competitive' stage of the spectrum." I mean, people are competing for attention, right? And dates? Remember, though, the entire context for reputation in this example. Most importantly, remember the desires of the person //doing the evaluating// on these sites. A visitor to a dating site probably doesn't want a competition, nor does she view it that way: for her, this site may be a more collaborative endeavor. She's looking for a potential dating partner across her own particular set of criteria and needs. Not necessarily 'the best person on the site.' === Consider Your Users === Communities are, of course, made up of people, and these self-same people will come to your site with a number of prejudices, goals (some fuzzy and vague, others crystal clear in their heads) and ambitions. We've already asked you to consider your reputation goals from a selfish (business-driven) perspective. AND from the standpoint of a good community manager, in considering the effects that reputation decisions may have on community-level behaviors. Now, we'll dig deeper into what motivates those individuals, those //people// that make up your community. Let's begin with an anecdote... == Why do people participate? == In 2007, as part of our research into appropriate reputation signifiers and metaphors at Yahoo!, we brought a number of users into our research labs, and showed them a handful of different ways to represent a person's reputation within a community (Figure_6-5 . The fictitious community that we constructed for the purposes of the study was one based on Yahoo! Local, and the tasks that subjects were given all related, in one way or another, to writing reviews of local goods and services. The people that we brought in were almost all active review-writers, both on Yahoo! and at other popular review-driven web sites. The review-writing was, of course, purely tangential to what we really wanted to learn: given a number of different ways of displaying a person's reputation, which ones would our subjects gravitate toward? Are 'Points-based' reputations perceived any differently than 'Levels?' (Or badges, or 'Top X', or...) // Figure_6-5: Yahoo! tested a variety of patterns for representing people reputation, to mixed results. // {{Ch06-AVarietyOfRepRepresentations.jpg}} In the end, what we learned about our users preferences for different patterns of presentation was both unsurprising and somewhat underwhelming: as a group, there didn't seem to be any preference for one mode of presentation over any other. In fact, most everyone found //something// to dislike about almost every single pattern. Some patterns (Points, Levels and any combination thereof) were deemed 'to cold,' or too much like being graded. Others ('Reviews HOTSHOT!') were seen as too boastful, or cocky. And aspiring to any of these achievements was not something that most of our subjects expressed any interest in: "They seem like people with too much time on their hands" was a common refrain. But the problems our users had with all of these patterns, and the //questions// they had about each were invaluable, because they gave us a lot of insight into the motivations of these prolific review-writers. And helped us make some assumptions about exactly what drove these folks to contribute online in the first place. Our excellent user researcher Beverly Tseng Freeman summarized these motivations thusly:

Self-interest	Interest in Others	Interest in the Object Being Rated
Have a voice	Help others	"Fill a void"
Practice or hone your craft	Share a unique perspective	"Set the record straight" (Provide a dissenting opinion.)
Chronicle life experiences	Feel connected to others	Reward a business
Be seen in a favorable light	Entertain others	Punish the business
Express yourself

A successful reputation system should acknowledge this range of motivations, and find ways to reward contributors in a way appropriate to any of them. == Extrinsic vs Intrinsic Motivators == Regardless of the //specific// reasons that a person may or may not contribute to your site (which will vary widely, and are probably fairly unique to the individual) it can be very useful to think of the incentives that you are offering your users in terms of the reward that they provide: are the rewards associated with doing a certain task //intrinsic//? That is, is the action of doing the task //also// the reward for having done it? Intrinsic motivation is a powerful thing and is the force behind many of the Web's most thriving user-contributed communities. Or are you providing //extrinsic// rewards? Extrinsic rewards attempt to replace the intrinsic satisfaction of an activity with some external reward. (Money is perhaps the mother-of-all extrinsic rewards.) Any number of extrinsic motivators have been attempted on the Web. Perhaps you've considered some of these yourself: * A 'Rewards' system, that provides prizes in exchange for continuing contributions * A prize for registering to a new service * Immaterial benefits, such as displayable badges or trophies for top contributors Extrinsic rewards are not objectively better or worse than intrinsic motivators: they're just different, and providing one or the other will possibly yeild very different community effects. Think of it this way: which do you enjoy more? Your intrinsically motivated hobby, or your for-pay job? Which do you feel more effective at? Where do you do your best work? (No, really, we're asking-we expect that the answers will be different for different people!) Perhaps the most dangerous thing you can do to your web community is to (inadvertently or knowingly) attempt to replace intrinsic motivations (people contributing for the love of it) with extrinsic ones (paying your contributors, or doling out other types of rewards.) These types of changes can have a debilitating effect on the community's character. === Reputation as Incentive for user Behavior === Making the link between reputation systems and incentive. Why does anyone care how many Yahoo! Answers Points they, or anyone else have? How do we align the goals we set for our business and our users with a reputation in order to produce a virtuous circle? What are the advantages to using a reputation-based incentive system? - use when the existing intrinsic rewards for the goal are weak or non-existent - use to encourage users to perform tasks (including learning and orientation) tasks of increasing complexity - use to increase the quantity and/or quality of content and user contributions - use to establish a competitive context for participation - use to unlock enhanced functionality for users - use to identify particular users for specific attention and action, either an award or discipline == Use Positive Reputation to Get Users to Do Good Stuff for You == Using Reputation to encouraging desirable behavior is harder than it is to discourage undesirable actions - because it usually requires the reputation scores to be surfaced to the user, and often to other users or even the public. ** How Positive Reputation Works ** Users perform actions, and at specific junctions receive extrinsic reward for doing so. ** Typical Positive Reputation Pattern ** Choose one - (Render choked without this para) * **Points** * Blah * **Trophies** * Blah * **Levels** * Blah * **Virtual Currencies** * Blah ** ** Public, positive reputation is extrinsic motivation! It devalues intrinsic, altruistic, social-norm-driven contribution. Avoid displacing existing intrinsic motivations with public positive reputation. There will be an explosion. Yes. We say this twice in the same chapter. It's that important. == Use Negative Reputation to Identify and Discourage the Bad Stuff == Negative Reputation is internal and more similar things again. Specific designs would be good here. ==== Planning Checklist, Step 1: Goals Identified and Reputation Patterns Selected ==== At this point, you should have a clean list of business goals and for your users. You should also have a general idea of which of the design patterns you will build and extend to design your reputation system.