Market Research Misleads Society

In the summer of 2011 England experienced a number of riots.  Here is one disturbing video of the type of behaviour that was taking place in a number of major cities.  I should warn you that it's not comfortable viewing:

This week a report was produced by the London School of Economics (LSE) and the Guardian newspaper that investigated the causes of the riots that swept through England in the summer.  It has received extensive publicity.

The research is dangerously misleading.

I don’t know quite how widely broadcast these disturbances were around the world, but it made for a lot of ugly television here in the UK, as mobs looted from shops and burnt cars and buildings.  The apparent trigger for this activity was the police shooting of someone who was believed to be armed and didn't respond to the commands of armed police (a blank firing gun was found at the scene).

The report by the LSE and Guardian newspaper is based on 270 interviews with rioters.

The main conclusion provided from the report, at least in so far as the headlines written and debate triggered, is that anti-police sentiment was a significant factor in the riots.  According to reports, “85% of people interviewed cited anger at policing practices as a key factor in why the violence happened.”

Knowing that interviews were used and reading the interview topic guide (available here http://image.guardian.co.uk/sys-files/Guardian/documents/2011/11/29/RIOT-QUESTIONNAIRE.pdf) it is, frankly, no surprise that this conclusion was reached. 

The research is an exercise in retrospective self-justification and post-rationalisation.

Let’s for a moment consider some of the complex interlinked factors that may have contributed to the behaviour of the rioters:

• A lack of self-control, the by-product of underdeveloped levels of conscious restraint in comparison to unconscious desires to copy other people, enhance status, attain a feeling of power, or not to lose face with friends.
  
• A lack of respect for authority, possibly caused by deficient parenting (let’s not forget most people didn’t go out and riot).
  
• A misguided belief that the police wouldn’t take action, so that free-stuff was available without consequences.
  
• Social proof that everybody like them was doing it because of the wall-to-wall media coverage that, for some time, described the behaviour of the rioters as a protest.
  The almost immediate media desire to ascribe rational drivers to the actions of rioters to help fill up the audio component of rolling news programmes when they had such dramatic footage that they knew would make compulsive viewing.
  
• Low intelligence leading to low earning potential, combined with a strong desire for consumption driven by a celebrity-focused media and powerful brand marketing building desire for unaffordable consumer goods.

What, do you suppose, is the likelihood that any of these factors would be identified by an individual who took part in the riots, when questioned about them many weeks after they took place?

The hypotheses that I’ve listed have one thing in common.  They reflect the role of the unconscious mind in human behaviour: processes that studies have proven repeatedly people are unable to access consciously with any reliability.  On the other hand, the survey invites conscious post-rationalisation of a rioter’s behaviour: we're all fairly adept at justifying our actions in this way, it's something we learn to do from an early age. 

Conscious post-rationalisation makes a number of things extremely likely. For instance, people’s psychologically healthy need to see themselves in a positive light will shape how they interpret events: so, guess what: “The police were to blame (not me).”

Availability bias will make it very likely that one, frequently discussed or media reported ‘cause’ will rise to the surface as a consistent factor: in the sequence of events where the police shooting was the trigger, the role of the police is foremost in people’s minds.   The correlated event becomes the cause and, because most people don’t understand statistical thinking, it feels like a satisfactory explanation.

Priming will influence the answers obtained: the people interviewed were primed by the questions used to consider the possibility that the police were to blame (the third topic raised, after questions about when they first heard about the riots and what they first thought:

“Speaking generally about the riots, ask the participant why they thought people were getting involved.
Rioting specifically? Looting? Attacking police?”

In one report about the research a Guardian article comments, “Again and again rioters from different parts of the country described the police as a gang.”  Such terminology sounds vivid and compelling; until you see that the following question was asked:

“Tell the participant some people have said the riots were partly down to ‘gangs’. What do they think?
What do they think of the term ‘gang’? How do they think ‘gangs’ were involved? What were the ‘gang’s involved? What were the names? What did they do?”

It’s not too hard to chart the progression of self-justification that has been unwittingly shaped by the questions asked and the natural mental biases of the respondents. 

At stake is not just the provision of misinformation into a debate that may lead to a wasteful investment in reforms that go on to have no impact.  There is also the irresponsibility of pedaling invalid ‘analysis’ of what happened that appears to justify the actions of those involved.  Now we have a situation where everyone is hearing that the police were in some way to blame for the riots taking place: the risk is that more people adopt an anti-police perspective and a subsequent trigger event drives more people to participate in lawless, antisocial behaviour.

Amongst the list of all the people involved in the research there was not one psychologist.  It is apparent that no one considered the first question that any research project should ask: can people answer these questions with any degree of psychological validity?  The answer would have been a clear and unequivocal ‘No’.  Instead, the questions were asked, over a million words of response have been collected and recorded, and the analysis is now being banded about as a legitimate insight into what occurred in England in the summer.

I hope you share my concern about the potential risks of this kind of research.  I don't expect the media to embrace my concerns, The Guardian evidently believes it's on to a good thing, with pages and pages of coverage and the protracted publication of different elements of the research.  However, I do hope that you can consider your own organisations research and the potential for spurious research 'insights' to become facts around which costly initiatives are launched.

Philip Graves

Tweet

Read the Full Account of The Market Research Myth