« Polish & Predictability | Main | Ratings Bias Effects »

Abusing FICO

Today's NY Times has a reputation-related article that I find distressing—Another Hurdle for the Jobless: Credit Inquiries:

Once reserved for government jobs or payroll positions that could involve significant sums of money, credit checks are now fast, cheap and used for all manner of work. Employers, often winnowing a big pool of job applicants in days of nearly 10 percent unemployment, view the credit check as a valuable tool for assessing someone’s judgment.
This is, basically, the tale of a formalized reputation score (your FICO, or credit score) that has run amok, and may have outgrown its original intent.

Factors in a FICO credit score

In Chapter 1 of Building Web 2.0 Reputations Systems, we discuss the differences between local reputations and global reputations. Local reputations are highly specific to the context in which they're earned and typically have greater value as a decision-making device when they're evaluated within that context. This makes a certain sense when you think about it: John may have earned the 'Top Selling Realtor' award at his agency for three years running, but—when pulled over by the police for erratic driving—he'd be foolish to proffer up his lucite trophies as evidence of his upstanding, sober character. The one reputation has no currency in the other context.

There is a certain gray area, however, where local reputations do have some fungibility between contexts. Where the perceived differences between contexts are minimal enough that the reputation earned in one should still count for something in the other. So if John takes his Top Realtor awards with him to a regional real-estate conference? Well, they still kinda count for something. The context differs in scope—geography—but the applied domain—real estate—is the same. People in the new context can understand the reputation well enough to evaluate it and appreciate the effort (the inputs) that went into earning it.

Or perhaps John puts his awards on his LinkedIn profile to impress local small business owners with the chamber of commerce. The domain's not real estate anymore, but enough other facets of the contexts (local geography, a shared 'business professional' context) give his reputation some meaning to that audience. So locally-earned reputations can be immensely valuable when applied in related contexts.

An extremely limited number of reputations find utility in a great number of contexts. We call these reputations global reputations. [Note: this terminology is largely of our own devise, and we're not convinced that local vs. global carries exactly the right connotations. We welcome your suggestions.] Global reputations may, in fact, be an illusion. Our tendency seems to be to want to believe that some objective measure of a person's self-worth can be tabulated, stored and transferred with ease.

This, of course, is not the case. The further the interpretation of a reputation strays afield from the context in which it was earned (and the purposes for which it was originally created), then the less reliable that reputation is. This is the crux of the problem with FICO. Think for a moment about the original intent of your credit report. It is a tool for credit lenders to predict how valuable a customer you may or may not be to them. It is a highly-contextual and specific score, and it considers inputs specific to that evaluation: do you pay your bills on time? do you use the credit you've been given? how much? how frequently? do you have a lot of credit? too much?

The answers to these questions have great utility to credit-lenders, and rolling them up into an easily accessible score has definitely contributed to FICO's wild success as a credit-lending device. Unfortunately, this level of abstraction also gives the illusion that the FICO score has great portability between contexts. It's a numerical score, so it looks like an objective measure. The temptation is simply too great to believe that a high credit score equates to good citizenship, trustworthiness, high moral fiber and love of God & country.

I think you can see how misled and ridiculous this belief is, but as evidence that it's a fundamentally flawed assumption, consider the following: even within its own context, FICO is basically a subjective measure. It was created to inform the purposes of a specific set of evaluators—credit lenders. Remember, when a credit underwriter evaluates your score, the question that they're really hoping to answer is: will I make money off of this customer? Only as a facet or component of that question do they care about the related one: is this person responsible with their money?

A high credit score doesn't necessarily mean what many folks seem to think it means about a person's financial stability or decision-making prowess. There are any number of extremely wise patterns of behavior that a person might engage in that could result in a poor (or no) credit score: saving most of your money; or paying for purchases only in cash; never taking out a line of credit. These behaviors might make you a less-than-optimal credit customer, but would anyone truly argue that they make you a bad person? Or financially irresponsible? Of course not.

So, judging the quality of a person's character on their credit report? That's a fool's errand.

The Times article points out the truly insidious problem with basing hiring determinations on a person's credit score:

“How do you get out from under it?” asked Matthew W. Finkin, a law professor at the University of Illinois, who fears that the unemployed and debt-ridden could form a luckless class. “You can’t re-establish your credit if you can’t get a job, and you can’t get a job if you’ve got bad credit.”
This is exactly the danger that our draft chapter points out. This mis-application of your credit rating creates a feedback loop. This is a situation in which the inputs into the system (in this case, your employment) are dependent in some part upon the output from the system. Why are feedback loops bad? Well, as the Times points out—feedback loops are self-perpetuating and, once started, nigh-impossible to break. And, much like in music production (Neil Young notwithstanding) feedback loops are generally to be avoided because they muddy the fidelity of the signal. (We'll be talking more about feedback loops in the yet-to-be-drafted Chapter 10 of our book.)

Remember, a credit score—like any reputation—was created to serve a purpose. Generally, to provide information to make determinations within a specific context. Straying too far from that context, or applying it in ways that feed back into the system, erodes confidence in the system itself. (It's out of scope for—though perfectly aligned with—this discussion, but some blame the misapplication of FICO into the field of mortgage approvals for playing a significant role in the subprime mortgage collapse.)


TrackBack URL for this entry: