Desperate Measures

In physics, there’s a concept called the “observer effect,” which basically means that to accurately measure something you very often change it a little through the actions of your measurement. We can’t really be “passive observers” most of the time; in order to study something, we have to interact with it at least a little. Think of checking tire pressure – the very act of attaching the pressure gauge to the valve lets out just a smidge of air. So by measuring the air pressure you also changed it, if only a bit.

This isn’t just a physics thing. If you study animals in their natural habitat, you have to disrupt that habitat, if only in a minor way. You have to sneak in a camera or something, and get your smell on things and scare away a bird that might have been the next meal of the tiger you’re trying to study, etc.

And it’s not just a hard science thing, either. In fact, the problem is much, much worse in the realm of human interaction. Because in addition to everything else, humans can usually observe that they’re being measured – especially if you tell them!

Let’s talk about resumes for a moment. Most people that use resumes have the core idea of them completely wrong. Most people think that a resume is an impartial record of their professional life. They think of a resume almost as if it was created by an outside observer who’s been watching their career. If that was actually how resumes got made, that would be fine – but it isn’t. Because if that were how resumes were made (by impartial, outside observers) then employers could use them as a measure of a candidate’s ability and experience.

But that’s not what they are at all. They’re self-created marketing documents, and that changes things a lot. You see, what employers generally want to see on resumes is generally pretty known (or at least discoverable). Their standards of measure are public information, for the most part. And that means, for the most part, that they’re bad standards of measure.

Let me go deeper into this tangent for a moment. Let’s say an eccentric billionaire walks into a sleepy town one day (without announcing anything about his wealth), and asks around until he finds the person who has worked hardest to become a painter. Discovering an artist toiling away on brilliance in poverty-level conditions, the billionaire bequeaths a million dollars on this person in a public display of support, speaking about how much the billionaire values the artistic pursuits and why they’re so important. The billionaire then says that he’ll return to the town next year.

What would you expect that town to look like, a year hence? The place would probably be overrun with artists! But here’s the problem – that very first artist was pursuing art out of a genuine love of it. You could be sure of this because the artist had no expectation of a sudden windfall. Meanwhile, every other artist in the city now has their motives sullied by the fact that they know that there’s a major reward lurking if they paint hard enough. Incentives change behavior. The people of the city know that the measure of “best artist” would be rewarded, and so they changed their behaviors appropriately in response to the measure. That means that the measure is no longer the measure the billionaire wants it to be – it’s no longer a measure of “who loves art the most,” but rather “who wants a million dollars the most.” Since those two measures will reflect different people, the point is lost.

So, back up to resumes. If a measurement becomes a target you can adjust to, it stops being a good measure. That is extra true when it comes to measurements that are very easy to change and very hard to quantify in the first place, which is like 99% of what shows up on resumes.

I can read a resume, but what I’m reading is a piece of paper written entirely by a person who wants a specific thing that I have to offer, and who knew in advance exactly what I wanted to see. I’m not saying that all people who turn in resumes are shameless liars or anything (though certainly a few are, and their resumes will look the same as those of the people who aren’t), but I’m saying that even if the resume is 100% truthful and accurate, even doing the actual stuff you’ve written about was informed by the fact that someone else would want to read it, not because you wanted to do it or it had value.

For example, I’ve talked to a number of people who worked at Facebook, and I heard this story many times: it was a good idea to work at Facebook for two years, even though Facebook was a terrible employer and everyone hated working there, because it looked so good on your resume. Many of these people did nothing of value while they were there. They “phoned it in” and did just enough to not get fired for two years (or close enough to round up) and then bounced, knowing that future employers would look at their resume and go “Ooooh, Facebook! How impressive!” No lying involved – the resume was accurate – but the actions of the person in question were deadweight loss to everyone involved.

See, that’s the other problem with targeting a measure – in most cases, you have to do wasteful things in order to do it. In the example of the town full of artists, a LOT of time is being wasted. I don’t think “art” is a waste or anything. But I think that garbage art that you don’t even want to make and no one else wants to see that you’re creating in a frenzy just because one of you out of hundreds will get a million dollars for doing so is just about the definition of wasteful. All of those people, absent the million-dollar prize, would have been doing significantly more valuable things with their year. In fact, if you added up the value of all the things those people could have built or accomplished in that year, the total would certainly be more than a million dollars.

Of course, in some cases point of measuring things in an observable way is to encourage changes in behavior. Measuring grades in school is meant not only to be an observation of ability but also an encouragement to improve ability. And of course… it doesn’t work. For exactly all the reasons I’ve stated. What grades measure is precise. “How well you’re learning” or “how smart you are” are very imprecise things that only very loosely correlate to your grades in school, but they do make sure that children spend all of their time and energy not actually learning, but trying to get good grades. Actual knowledge retention of the things children are graded on is abysmal (remember that show “Are You Smarter Than a Fifth Grader?”), because grades don’t measure how well you retain stuff, just how well you pass tests.

So here’s the upshot of all this rambling: as soon as the subject of your measurement knows that they’re being measured (assuming that some reward is also tied to the measurement), they will bend towards appeasing the yardstick. If you aren’t fully aware of that when you try to measure, you’re wasting your time – and probably everyone else’s, too. In a real way, you’re inflicting harm. Of course, our understanding and operation in the world requires that we have data and that requires measurement. At some point, you simply have to. But you have to know that this effect exists. You have to know that you cannot be an impartial observer – your observation will change the outcomes. The easier it is to bend to the yardstick, the more of exactly that you will get.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s