Market Research:Belief in a Pseudo Science

Probability is an interesting thing. It's relatively easy to evaluate the probability of a particular event happening but, when it comes down to it, we humans are much more concerned with our own experience.

If we experience an unlikely event we tend to massively over-estimate the likelihood of that event happening again.

For example, last week I was hit on the head by a golf ball. There I was waiting for someone else in my group to play, minding my own business, when an errant drive from an adjacent hole landed a direct blow to my skull.

Fortunately, this was a good inch above either of the points where it could have killed me (the temple and the eye).

The probability of me being hit was tiny. I was playing golf with friends who I play with only once or twice a year. We had never played the course before. Every single shot we had hit up to that point had contributed to us being at that part of the course at that particular moment. And if I had selected a six iron instead of a seven iron as I teed off on the par 3 I would have been on the green not just short of it.

Objectively, millions of people play golf, hardly any serious injuries or deaths are reported, so the probability of me being hit again are as tiny as they always were.

But my mind is already working on ways it can avoid a repeat occurrence: should I always wear a hat - the solid peak would afford some protection to my face. My Oakley sunglasses have lenses that are tested to be impact resistance and arms that cover the temple area - should I wear them like safety glasses every time I play (even when there's no sun).

Rationally I know that I should carry on as before, but emotionally I want to 'learn' from the experience.

Probability gets distorted in other ways in our perceptions too. The artificial learning we do from stories mean that sensational news of child abductions or air disasters feel like much more significant risks than their probabilities would suggest they should do.

When it comes down to it, we aren't as objective and scientific in evaluating factors that affect our own lives as we're inclined to tell ourselves.

All of which brings me to market research.

As I explain in my forthcoming book Consumer.ology, market research is a pseudo science. It isn't a proven demonstrable, repeatable science; it's something that people feel makes sense and that they choose to believe in.

Once belief is established - usually because people like the idea of whatever is being suggested - the human minds lack of objectivity kicks in. We can select the evidence that reinforces our beliefs and tacitly ignore that which conflicts.

If you consider anything that other people believe in and you don't you will see this pattern; the tricky part is seeing it in something you have, up to this point, chosen to believe in yourself.

I will leave Consumer.ology to explain why the vast majority of market research is fundamentally flawed (and what you should do instead). But for now I'll ask you to consider the probability that market research will be right, and why you shouldn't let a misguided sense of probability determine your belief in it.

The standard defence to my assertion that science can't support the notion that asking consumers what they think is valid any more than it can support the idea that the position of the planets and your birthday will determine what kind of day you're going to have, is to say that, "We've used market research on lots of occasions and it guided us to the right decision."

What is the probability that a piece of market research will produce a result that is correct?

Broadly speaking there are three possible outcomes from conducting market research:

1. The results suggest you should take a particular course of action that turns out to be profitable.
   
2. The results are ambiguous.
   
3. The results suggest you should take a particular course of action that turns out to be unprofitable.

Already, the probability of research being wrong is one third.

However, virtually no research is unbiased. The very moment that you ask a question you are pointing the spotlight in a particular direction. That direction is informed by what the business believes is important and often by what the business has decided it wants to do and is simply looking to justify.

Since the business should know what's important the likelihood is that it is looking for confirmation to do something that it intends to do. This will affect both the questions it asks and the way it interprets the results. Research becomes an exercise of confirmatory self-justification. Unless the company is clueless this should be right more often than it's wrong.

So now the probability of research being wrong is dropping further - perhaps as low as one in ten.

That makes it much easier to unconsciously dismiss these occasions as aberrations: some glitch in the works that can be over-looked because we 'know' research works from the other nine occasions.

But I would argue that we should look at the question of research validity from two different directions.

Firstly, since research does fail (and there are countless examples that have found their way into the public domain), we should seek to understand why.

Secondly, we should look at the science underlying the notion of asking people questions: does it make sense that people can explain what they think or what they will do?

It's tempting to say "Of course" because we all like to believe that we know we think ourselves and that our actions are consciously determined.

But...

After examining all the evidence from psychology and neuroscience I have concluded that people can't explain what they think or what they will do with any useful degree of accuracy.

Consumer.ology explains why this is the case and what you should do instead. But for now, whilst you wait for it to be published in the UK this September (November in the US), I would urge you to start being more objective about the market research you've experienced. The probability is that you'll be shocked.

Tweet