« March 2010 | Main | May 2010 »

April 06, 2010

Don't Display Negative Karma Redux: Unvarnished

It's Reputation Wednesday again, and the entire subject area of reputation systems seems to be heating up. For example there's been a lot of chatter about Unvarnished.

Update 4/13/2010: The Register is reporting that an eBayer is being sued in the amount of $15,000 for leaving negative feedback - more fodder for thought...

Unvarnished is a public karma system for real-world identities which will reportedly accept [and protect] negative anonymous comments, presumably from former co-workers.

This has generated a lot of chatter, mostly negative from folks like Evelyn Rusli at TechCrunch:Unvarnished: A Clean, Well-Lighted Place For Defamation
Today, Unvarnished makes its beta debut. It’s essentially Yelp for LinkedIn: any user can create an online profile for a professional and submit anonymous reviews. You can claim your profile, but unlike LinkedIn, you have to accept every post, warts and all. And once the profile is up there’s no taking it down.

I asked co-founder, Peter Kazanjy, “Will you ever give users the option to take down their profile?” Kazanjy’s reply: “No, because if we did that, everyone would take their profile down”
...and... CNet's Molly Wood writes in Unvarnished: Person reviews or trollfest?
Because let's be clear. Though Unvarnished may be billed as a natural extension of trends that started with LinkedIn, Yelp, and even Facebook, MySpace, and message boards, there's nothing about this site that, in my opinion, doesn't lead almost immediately to rank nastiness.

After a long conversation with co-founder Peter Kazanjy, formerly of VMWare, I'm convinced that the founders (the others come from eBay and LinkedIn) really do think they're creating a site that will maintain a professional veneer, be well moderated by its users, and won't descend into personal attacks. I just don't agree.
...and perhaps a bit more positive - Craig Newmark says in Trust and reputation systems: redistributing power and influence
The most prominent experiment in directly measuring trust is Unvarnished, very recently launched in beta form. You rate what trust you have in specific individuals, and they might rate you. Unvarnished is pretty controversial, and is already attracting a lot of legal speculation. They're trying to address all the problems related to the trustworthiness of the information they receive, and if so, might become very successful.

Unvarnished Against the grain

This service breaks several tenants of online karma (people reputation) as outlined in Building Web Reputation Systems (wiki):

  1. Don't Display Negative Karma!(from The Dollhouse Mafia post)
    We said it there best: "Avoid negative public karma, really."
  2. Karma is Complex, Built of Indirect Input (Chapter 7 of our book draft)
    The reason for using indirect input is to establish a clear context of evaluation - eBay requires you complete a transaction in their system before you rate a seller. There is no way for unvarnished to tie the negative comments to an actual context (co-worker).
  3. There is a real problem with the incentives model for all the participants (we spend one half a chapter on incentives and motivation for user generated content).
    As we warn there, mixing ego-based motivation (i.e. revenge) with commercial incentives (personal brand building) is usually toxic. Unvarnished is especially problematic with the ability to leave anonymous comments. Doesn't anyone remember F*ckedCompany.com? Having been the target of comments like "Sieg Heil, Randy!" I can tell you one possible outcome for Unvarnished: Deadpool.
  4. Scanning our post summarizing Karma best practices suggests quite a few places Unvarnished might want to look at closely when creating and displaying their karma.

A colleague that worked for Wink.com, an identity aggregation site told me that they would get people angry at the fact that a profile had been assembled on their behalf on Wink, even if it was only built by a search engine—they would often demand it's removal, even though it only contained public data. Identity and privacy are sensitive topics.

The one thing I'm sure of, from my experience building online communities for over 35 years, the founders of Unvarnished will discover that the use-patterns will look nothing like what they've planned for or predicted. They have bitten off something in an area that is fraught with peril, and so far (in the press, at least) haven't shown any understanding how significantly different business reviews are from public user karma, especially when people's livelihoods are at stake.

[BTW, I've signed up for the beta at getunvarnished.com - so if you're already a member, push the magic button that requests a review from me. :-)]

April 02, 2010

[MIT] Online Reputation Systems: How to Design One That Does What You Need

There is a top-notch post on creating reputation systems over at the MIT Sloan Management Review: Online Reputation Systems: How to Design One That Does What You Need by Chrysanthos Dellarocas, an associate professor of management at Boston University School of Management and an affiliated researcher at MIT's Center for Collective Intelligence

It's fairly lengthy, and aligns very closely with what we cover in greater detail in our book. Here I've excerpted only the section headers to peak your interest. Go read it, bookmark it, and if you have an MIT Sloan account, please leave a comment. BTW, if anyone has the author's address, we'd love to send professor Dellarocas a free copy...

How can a website attract the contributors it needs?

  • Key Decision #1: What are the key business objectives of your reputation system?
    • Build Trust
    • Promote Quality
    • Facilitate Member Matching
    • Sustain Loyalty
  • Key decision #2: What information should be included in your user’s reputation profile?
    • Which actions are most relevant to the reputation system’s users?
    • Which user behaviors are desirable?
    • For which behaviors is it possible to obtain reliable information?
  • Key Decision #3: How should reputation information be aggregated and displayed?
    Reputation mechanisms employ a variety of methods for displaying outputs. These fall into three large categories:
    • Raw activity statistics. Examples: number of reviews posted, number of transactions completed.
    • Scores and distinctions. Examples: star ratings (such as Amazon reviews), numerical scores (eBay’s feedback score, TopCoder’s user rating), numbered levels or named member tiers (World of Warcraft’s player levels, Slashdot’s “moderator” and “meta-moderator” tiers) or achievement badges (eBay power seller, Amazon Top Reviewer).
    • Leaderboards and other methods of displaying relative user rankings. Examples: the list of top Amazon reviewers; Epinions’ author popularity ranking.

April 01, 2010

Guest Post over on CommunityGuy.com: There’s a Whole Lotta Crap Out There

We've guest posted over at CommunityGuy.com outlining a content-quality scale.


Content at the higher end of the scale should be rewarded, trumpeted, and showcased.
Stuff on the lower registers will either be ignored, hidden or reported to the authorities.

Read the post CommunityGuy.com