Building Web Reputation Systems

Edit the page and hit Save. See Formatting Syntax for Wiki syntax. Please edit the page only if you can improve it. If you want to test some things, learn to make your first steps on the playground.

** Preface **

==== What is This Book About? ====
Today's Web is the product of over a //billion// hands and minds. Around the clock and around the globe, a world full of people are pumping out contributions small and large: full-length features on Vimeo; video shorts on YouTube; comments on Blogger; discussions on Yahoo! Groups; and tagged-and-titled Del.icio.us bookmarks. User-generated content and robust crowd participation have become the hallmarks of Web 2.0.

But the booming popularity of social media has created a whole new set of challenges for those who create websites and online communities, as well as for the users who consume those sites:

* problems of //scale// (how to manage-and present-an overwhelming inflow of user contributions)
  * problems of //quality// (how to tell the good stuff from the bad)
  * problems of //engagement// (how to reward contributors in a way that keeps them coming back)
  * and problems of //moderation// (how to “stamp out” the worst stuff in a quick and efficient fashion)
Reputation systems can provide an effective solution to all of these problems.

<WRAP blue-box>
** Reputation and Karma-Two Simple Definitions **

* **Reputation**
    * Information used to make a value judgment about an object or person

* **Karma**
    * The reputation(s) for a user
</WRAP>
What is reputation in an online community? In its broadest sense, reputation is //information used to make a value judgment about an object or a person.// There are potentially many components to reputation. For example, //karma// is a reputation score for a user in a community, it may be comprised of many components, such as:

* How long has this person been a member of the community?
  * What types of activities has she engaged in?
  * How well has she performed at them?
  * What do other people think about this person?
Having access to a person's reputation might help you make better informed judgments. Judgements like…

* Can I trust this person?
  * Should I transact with this person?
  * Is it worth my time to listen to this person?
The same principles of reputation can also be applied to that person's individual contributions-the video, photo or textual content that they've authored. Content reputation can answer questions such as:

* What's the best video in this category?
  * What are the most interesting photos submitted today?
  * Should I bother reading this blog-entry?
  * Which comments in a thread are truly worth my time?
For the site operator, reputation is at its most powerful when karma (people reputation) is used in concert with content reputation to increase your understanding of your online community, and the relative value of its participants and contents. For the community itself, reputation is //relevance//. To the site, reputation represents //value// and //return on investment//.

So What does a //reputation system// do? It powers the whole process: it monitors the community; makes note of the actions that community-members take; assesses the community's response to those actions; and keeps a running tally of the history of it all. (Optionally, the system may then //show the results of these calculations back// to users, enabling them to make judgments about the people and things they encounter.)

Reputation systems are the underlying mechanisms behind some of the best-known consumer websites. All of the following are powered-in whole or in part-by reputation systems:

* Amazon's product reviews are probably the most well-known example of content reputation. Their Top Reviewers program (karma) tracks reputable reviews and trusted reviewers. Both content and user reputation combine to provide context for potential product buyers to evaluate products any reviewer bias. [[|http://www.amazon.com/gp/customer-reviews/top-reviewers.html]]
  * Ebay's Merchant Feedback scores are based on the number of successful transactions that a seller or buyer has completed. It is aggregated entirely out of hundreds or thousands of individual purchase transaction ratings. [[|http://pages.ebay.com/help/feedback/scores-reputation.html]]
  * Built out of a deep per-post user rating and classification system, Slashdot's author karma is an oft-referenced program used to surface good content and silence trolls (troublemakers) and spammers. [[|http://slashdot.org/faq/com-mod.shtml#cm700]]
  * Xbox Live's very successful achievements reward users for beating mini-goals within games and cumulatively add to community members' reputations. [[|http://www.xbox.com/en-US/games/tips/achievements.htm]]
But, of course, developing a successful reputation system is not easy. Our experience is that reputation is one part each of engineering, social science, psychology and economics. Social media architects and designers must pay careful attention to multiple factors: the context in which the community is situated; the incentives they'd like to provide contributors (which often demand a nuanced understanding of the actions they want to encourage and discourage); and the effects that decisions made early in the design process may have on the community spirit, downstream.

//Building Web Reputation Systems// provides a complete, soup-to-nuts process for designing and developing your own community's reputation program. In a guided fashion, you will start with the hard questions:

* How can reputation enhance my business? The community?
  * What are the right types of behaviors to encourage?
  * What are the right objects (people, things?) to accrue reputation?
  * What types of actions should count towards a reputation?
…and then you'll learn how to build //reputation models// to accurately reflect all of these inputs. To do so, you'll need a //reputation grammar//, which we've developed and applied to many successful reputation systems used across the web.

We also provide well-diagrammed reputation patterns for common Web needs (“Digg-style” voting systems, ratings & reviews, simple karma) and some extended reputation system reviews and a case study based on actual reputation deployments at industry-leading social sites including Yahoo!, Flickr and eBay.

==== Why Write a Book About Reputation? ====
We wrote this book because we saw how critical reputation has become to the survival and growth of the web. Though there are many academic research papers on specific generational algorithms and social effects of reputation systems, we couldn't find a single book that put it all in context-describing it as a separate domain of knowledge, complete with a grammar, emerging best-use patterns, and recurring anti-patterns. Until now.

Our extensive experience creating and deploying several reputation systems across a wide range of social media at Yahoo! provided much of our base of knowledge for this book. We add a comprehensive review of other existing applications to venture forth into unexplained territory to offer a working definition of web reputation systems: a survivor's guide for those who would follow. We hope this book will improve the quality of conversation about reputation systems, and help improve the quality of products that are using them.

==== Who Should Read This Book ====
This book and the supporting blog and wiki ([[|http://buildingreputation.com]]) are targeted at several audiences:

* **Primary audience**
    * Anyone who is //building, operating, or participating// in a web site or online application that leverages user generated content: social networking, ratings, votes, reviews, comments, posts, uploads, games, etc. You'll need to understand reputation systems in order to maximize your user experience, search engine performance, and streamline your operational expenses.

* **A 
//must have// reference for**
    * System Architects, Product Managers, Community Support Staff, or User Interface Designers tasked with designing a reputation system for an online community. (Or any variant thereof: a social collaboration tool; voting mechanism, or a ratings and reviews experience.)

* **Also useful to**
    * Game designers, who deal with reward and incentive systems for social games; or any technologist in a role that requires some familiarity with reputation systems, who'd like to become conversant in their design and operations.
Successful social media design is hard. Many companies and organizations understand that. Rewarding points for user participation may generate activity and build audience, but site owners should be wary of possible negative impact on content quality. They need this book to help them choose the right reputation model for their site, and to learn how to help them evaluate the health of their deployed social collaboration system.

This book assumes that the reader has a rudimentary understanding of application logic, can read basic flow-charts, creates content for the web, and has direct experience interacting with other users on socially active web sites. No programming or web engineering experience is required, though some of the more technical aspects of the book will be greatly aided by some passing familiarity with programmatic thinking.

==== Organization of This Book ====
This book is broken into three main sections; Reputation Defined and Illustrated, Extended Elements and Applied Examples, and Building Web Reputation Systems. It was written to be consumed sequentially, but readers who are already experienced developing web sites heavy in user-generated content may be able to move more quickly through Section I.

=== Section I: Reputation Defined and Illustrated ===
<html><a href="/doku.php?id=Chapter_1">Chapter_1</a>&nbsp;</html>briefly introduces reputation basic concepts and terminology and sets reputation systems in their proper historical context. With this context in mind, <html><a href="/doku.php?id=Chapter_2">Chapter_2</a>&nbsp;</html>goes on define a complete grammar for thinking about reputation and provides a graphical language to clearly describe reputation systems for analysis and development.

=== Section II: Extended Elements and Applied Examples ===
The primitive reputation grammar elements described in Section I are to be combined in <html><a href="/doku.php?id=Chapter_3">Chapter_3</a>&nbsp;</html>into higher-level //building blocks// that can be used as-is for simple reputation systems, or combined to create more complex ones. In <html><a href="/doku.php?id=Chapter_4">Chapter_4</a>&nbsp;</html>we show how the grammar and blocks we've described can be used to elucidate many of the reputation systems currently deployed across the web, including Digg, Ebay, and Flickr. In this sections, practitioners tips become a regular feature of most chapters-the lessons we've learned are not to be missed-even by the casual reader.

=== Section III: Building Web Reputation Systems ===
The last section, comprising more than half the book, goes beyond cookie-cutter reputation needs to offer detailed advice on designing, building, deploying, and operating a custom reputation system. The project begins, //not// by drawing a model or screen mockups, but by answering the three big questions posed in <html><a href="/doku.php?id=Chapter_5">Chapter_5</a>&nbsp;</html>that define and limit your reputation choices. With those answers, <html><a href="/doku.php?id=Chapter_6">Chapter_6</a>&nbsp;</html>helps to identify your application's objects, methods, and inputs powering your reputation. With all of this in mind, you can start drawing the reputation model. <html><a href="/doku.php?id=Chapter_7">Chapter_7</a>&nbsp;</html>helps you decide how to display reputation, and <html><a href="/doku.php?id=Chapter_8">Chapter_8</a>&nbsp;</html>describes other common uses for reputation, such as providing search relevance. At this point, the reputation model is designed, and the screen mocks are ready. Next up is implementation, testing, and deployment for which strategies and tips are covered in <html><a href="/doku.php?id=Chapter_3">Chapter_3</a>&nbsp;</html>. We wrap it all up with a case study in <html><a href="/doku.php?id=Chapter_10">Chapter_10</a>&nbsp;</html>: The Yahoo! Answer's Community Moderation Model. This chapter ties the entire book together, revisiting each of the steps outlined in detail with a single concrete example. It is a successful reputation model that provides inspiration, many insights, and several notes of caution.

<html><a href="/doku.php?id=Appendix_A">Appendix_A</a>&nbsp;</html>is intended for technical readers, such as system architects and platform engineers. It describes the //Reputation Framework//-execution environment for multiple reputation models. The Yahoo! Reputation Platform is described in detail as an example of a framework that can scale to tens of thousands of transactions-per-second-but had to make some functional trade-offs that are quite illustrative.

=== Role-Based Reading (For Those in a Hurry) ===
Here are a few alternate chapter reading list recommendations, based on your professional role.

* **[Product | UX | Game] Designers and Application Product Managers**
    * We wrote this book primarily for you-Chapters 1 through 10 are all important. If you must skim, be sure to read all of the practitioners tips, warnings, notes, and sidebars to make sure you aren't missing something important. User experience folks should pay extra attention to the pros and cons in Chapters 7 and 8.

* **System Architects, Software Engineers, Platform Engineers**
    * Assuming you're reading this book as part of a plan to deploy a reputation system - read Chapters 1 and 2 completely-the definitions are important to later sections. Skim Chapter 3 but read all the practitioners tips, and pay close attention to the last half of Chapter 4. In Chapter 5, familiarize yourself with the Content Control Patterns and the limiting effects they have on reputation systems. Chapters 6, 9, and 10 are all worth your full attention. Also look at Appendix A and consider if you need a reputation framework.

* **Community Support Staff, [Program | Project] Managers, Operations Staff**
    * If you're involved in a support role with reputation systems, read Chapter 1 and review the definitions in Chapter 2. In Chapter 3 be sure to read the practitioners tips and likewise the advice about why reputation sometimes fails at the end of Chapter 4. Chapters 7 and 8 provide patterns for how reputation faces the users and the company and explain when (and when not) to use them. You're probably in a role that is detailed in Chapter 9-if so, read it. Chapter 10 may be the most important chapter in the book for you-nothing like a practical example to get oriented.

==== Conventions Used in This Book ====
The following typographical conventions are used in this book:

* **Italic**
    * Indicates new terms, URLs, email addresses, filenames, and file extensions.

* **
Constant width
**
    * Used for program listings, as well as within paragraphs to refer to program elements such as variable or function names, databases, data types, environment variables, statements, and keywords.

* **
Constant width bold
**
    * Shows commands or other text that should be typed literally by the user.

* **
Constant width italic
**
    * Shows text that should be replaced with user-supplied values or by values determined by context.
<WRAP tip>This icon signifies a tip, suggestion, or general note.

</WRAP>
<WRAP warning>This icon indicates a warning or caution.

</WRAP>

==== SafariÂ® Books Online ====
<WRAP tip>When you see a SafariÂ® Books Online icon on the cover of your favorite technology book, that means the book is available online through the Oâ€™Reilly Network Safari Bookshelf.

</WRAP>
Safari offers a solution thatâ€™s better than e-books. Itâ€™s a virtual library that lets you easily search thousands of top tech books, cut and paste code samples, download chapters, and find quick answers when you need the most accurate, current information. Try it for free at [[http://safari.oreilly.com|]].

==== How to Contact Us ====
Please address comments and questions concerning this book to the publisher:

Oâ€™Reilly Media, Inc.1005 Gravenstein Highway NorthSebastopol, CA 95472800-998-9938 (in the United States or Canada)707-829-0515 (international or local)707 829-0104 (fax)We have a web page for this book, where we list errata, examples, and any additional information. You can access this page at:

[[http://www.oreilly.com/catalog/<catalog page>|]]
To comment or ask technical questions about this book, send email to:

bookquestions@oreilly.com
For more information about our books, conferences, Resource Centers, and the Oâ€™Reilly Network, see our web site at:

[[http://www.oreilly.com|]]

==== Acknowledgements ====
As with any book that reports so much personal experience with a topic, there are more people to thank than we even know or recall-to any we've missed, know that we are grateful for the lessons you helped us learn, even if we're forgetful of your names.

We are first-time authors, so our editorial and publishing supporting cast come foremost to mind:

Mary Tressler, our Editor at O'Reilly and our mentor-you helped us learn the ropes and were always supportive when we stumbled.

Havi Hoffman, head of Yahoo! Press-you believed in this project from the beginning, and despite all logistical and legal challenges, you made it happen along with the unbounded support of your fellow Yahoos: Douglas Crockford, Christian Crumlish, and Neal Sample. Without all of you, there'd be no book at all.

Cate DeHeer, at [[http://dutchgirl.com|DutchGirl.com]]our main copy editor-you unified our voices and made us both sound great without losing our personality.

Sanders Kleinfeld, Marlowe Shaeffer, Adam Witwer, and the rest of the support staff at O'Reilly-you made it all go as smoothly as possible.

The Yahoo! Reputation Platform team, in its various incarnations: Alex Chen, Matthias Eichstaedt, Yvonne French, Jason Harmon, Chip Morningstar, Dmitri Smirnov, Farhad Tanzad, Mammad Zadeh-you all helped define, implement, operate and refine one of the world's finest platforms that provided us with most of the grammar and technical lessons used in this book.

The Yahoo! reputation-enabled product managers: Micah Alpern, Frederique Dame, Miles Libby, Cheralyn Watson, Ori Zaltzman, and so many others-when others scoffed, you were visionary and saw reputation as an unique opportunity to improve your product. So many of the socially oriented stories we've used here are a direct result of your pioneering work.

Our author-mentors: Douglas Crockford, Christian Crumlish, Amy Jo Kim, and Erin Malone-you all helped us understand just what it takes (and what it doesn't) to be an author.

The readers/commentors on our blog, wiki, and manuscript-by letting us know what you think as we went along, you significantly improved the first edition of this book. For those of you who comment after this is published-thank you so much for helping us keep this information up-to-date and accurate. Web publishing FTW!

=== From Randy ===
First and foremost, I'd like to thank my partner on this project, Bryce Glass-who presented the idea about us writing a book about reputation together just at the time I was feeling the desire to write something, but too timid to do it on my own. I knew immediately that this was a great idea, and that he would be the perfect coauthor-I had some the product and engineering experience and he really understood the UX design issues as well as being world-class at creating wonderfully simple images to communicating complex concepts. Truly our talents combined produced a book that is greater than the sum of it's parts.

Without the explicit encouragement from my wonderful wife, Pamela, this book would never have been started. I began working on it while being nominally unemployed, and at the worst of the 2008 economic downturn. Though I had enough contract work to just barely meet expenses, I could have just continued my search for full time employment and simply deferred the opportunity to write down my experiences in favor of a steady paycheck. While I was dithering, unsure about taking on mantle of authorship, she said “You should go for it!” Her faith in and support for me is an inspiration.

To my parents-Frank and Kathy Farmer: for your constant encouragement to dig ever-deeper into whatever topic I was interested in, I am forever grateful. I hope that sharing my knowledge with others will help others along a similar path.

Reeve, Cassi, Amanda, and Alice Farmer-you are my pride and joy, and the reason I keep striving to improve the world you will inherit.

I'd also like to acknowledge folks who personally influenced me in significant ways that eventually led me here:

* Thomas Hartsig Sr., formerly head of the Macomb Intermediate School District Computer Based Instruction group. Tom had the foresight to hire untested high-school programmers to create educational software in the late 1970s. At the MISD I learned that anyone can build a good reputation through hard work and inspiration.
  * Steve Arnold, former head of Lucasfilm Games/LucasArts, and everyone I worked there with during the early 1980s. Nothing convinces you that anything is possible like working for George Lucas.
  * Phil Salin, free-market economist, who encouraged me to create reputation systems for his lifelong project The American Information Exchange in the pre-web 1990s. If he'd only survived and we'd timed it a bit better, we could have been eBay.
  * Mark Hull, who hired me into Yahoo! first to create the business plan to build and leverage a reputation platform, then to co-design the Yahoo! 360° social network, and to help found the Community Platforms group, where the reputation platform would eventually be built.
  * Scott Moore and Han Qu, who helped me clarify the Content Control Patterns-thanks guys!

=== From Bryce ===
I, too, would like to thank my coauthor, Randy Farmer. His enthusiasm for, and absolute grasp of, social media and online communities was a large part of what drew me to Yahoo!'s Community Platforms team. Listening to Randy expound upon web-based reputation (or identity, or incentives, or context, or…) has been that rarest of professional treats for me: working on projects with Randy is like auditing a //really good// graduate-level course in social software design. Most inspiring of all, though, Randy is your enthusiasm and zest for learning-you don't just work at this stuff, you //love// it, and your energy is contagious.

The real, untold hero of this book-for me-has been my wife LeeAnn. What a long, strange trip it's been, honey! In the eighteen-some months since we first drafted the proposal, I've been laid off, unemployed, re-employed and spent //way// too many evenings and weekends stealing away for “a couple of hours to write.” //You//, meanwhile have been carrying our next child (with grace) while caring for our wonderful son Edison and three bratty dogs. I'll never forget the sacrifices you've made this year to help me see this book through-next year is all about you, I promise.

It truly does take a village, so I'd like to thank my loving parents, Larry and JoAnne Glass, as well as my wonderful inlaws, James and Diane Dick, for the //many// weekends of babysitting that freed Daddy up for... yes, more writing.

I'd also like to thank two of our early manuscript reviewers. Clay Spinuzzi and Bill Hart-Davidson, you both brought analytical eyes and rhetoricians' minds to our draft, and I especially appreciate your help. (Bill-you in particular gave us so much good feedback that we had to defer much of it. Are you available for our second edition?)

I'd also like to thank several past-and-present Yahoos: the aforementioned Christian Crumlish-you've been a great champion of our book, and a great friend as well; my first manager at Yahoo!, Erin Malone-thank you, thank you, thank you for your friendship, mentoring and assigning me to work with the Reputation Platform team!; Matt Leacock, who supported that platform before me, and is an all-around amazing UX designer and longtime friend; and finally my last manager at Yahoo!, Amanda Linden, who threw her unabashed support and approval behind the book and my involvement in it.

And finally, I'd like to thank my new team at Manta Media, Inc. Particularly my manager Marty Vian and my design compatriot David Roe. I'm afraid that you all inherited a slightly flawed and damaged designer-one with this baggage-laden //book// attached-and, whether you've realized it or not, you have all been supportive in the extreme in helping me get it to the finish line.

Edit summary

Note: By editing this page you agree to license your content under the following license: CC Attribution-Share Alike 4.0 International