How Much?

Imagine you’re conducting a very narrow “focus group,” with only ten members. What you’re trying to discover is whether or not you should sell chocolate or vanilla cookies.

You ask each of the ten people whether they prefer chocolate or vanilla and have them check off a box. 9 vote chocolate and one votes vanilla. So you should sell chocolate, right?

Well, maybe.

Let’s say that those 9 people have a very slight preference for chocolate; they’ll buy a chocolate cookie over a vanilla one if both are in front of them, but if they want a cookie and no chocolate option exists they’re happy to buy vanilla. Almost never would they choose not to buy a cookie at all if there wasn’t a chocolate one. And on top of that, they’re also not big cookie fans to begin with; they desire maybe 1-2 packs of cookies a year.

Meanwhile, the Vanilla Guy is obsessed. If a store carries only chocolate cookies and no vanilla, he won’t even shop in that store any more. And on top of that, he buys ten packs of vanilla cookies a week – a major consumer.

Assuming these ten people were a representative sample of the population, the company should absolutely make and sell vanilla cookies. The 10% of the population that prefers them buys more in two weeks than the other 90% of the population buys in a year, meaning that even though the market is niche, it’s deep.

The problem with a lot of the measurement of choices is that it doesn’t reflect this kind of depth. Most choice-measurement is very binary. Even most attempts to measure strength of conviction are pretty feeble – if you give someone a five-point scale instead of a binary choice it’s mostly junk, based on impulse and emotion rather than actual conviction in the long term. It won’t translate to action.

If I really wanted to measure overall depth of opinion (and not just how many people were on each side of a debate issue), I’d want something to truly measure depth. And since talk is cheap, I’d charge for it.

What people will pay money for is very, very different than what people will vocally support. Talk is cheap. Not only does it cost nothing to say you want to help some particular social cause – in many cases you actually benefit just by saying so, in the sense that you gain social capital with your peers. Ask people whether or not they support preserving a national park and most will say yes. Ask them to sign a petition and you’ll get a smaller number; ask them to donate a dollar and you’ll get fewer still.

Imagine asking the chocolate/vanilla question again, but this time giving 4 choices instead of two:

  1. I prefer chocolate.
  2. I prefer vanilla.
  3. I prefer chocolate so much that I’ll give you $10 to demonstrate my conviction.
  4. I prefer vanilla so much that I’ll give you $10 to demonstrate my conviction.

If you asked the original group this question, your focus group would have actually given you the correct answer – sell vanilla cookies. You’d have gotten 9 answers of “Number 1” and one answer of “Number 4.” If you weighted the answers based on dollar gains, you’d see the real answer.

If you’re uncomfortable with money being used as a proxy for strength of conviction (and I could see why), then you can replace $10 with anything that isn’t nothing. For instance, the four choices could be:

  1. I prefer chocolate.
  2. I prefer vanilla.
  3. I prefer chocolate so much that I’ll sit here for an extra hour to demonstrate my conviction.
  4. I prefer vanilla so much that I’ll sit here for an extra hour to demonstrate my conviction.

or they could be:

  1. I prefer chocolate.
  2. I prefer vanilla.
  3. I prefer chocolate so much that I’ll let you stab me with a needle to demonstrate my conviction.
  4. I prefer vanilla so much that I’ll let you stab me with a needle to demonstrate my conviction.

or whatever you want. The point isn’t to use any particular cost, it’s just to impose some cost on an answer. Otherwise, the answer should be taken as the mildest possible version, with the understanding that a single very strong preference might be more influential on the potential outcome of a decision than even a large number of mild preferences.

Consider that carefully any time you have to base a decision (even in part) on what people say they prefer. Try to find another way to figure out how much they prefer it – because talk is cheap.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s