Web thinktank-international.blogspot.com

Wednesday, September 15, 2010

Dan Ariely has seen some really bad qual research

Dan Ariely has continued his unrelenting attack on the focus group in this month's Research World. I put the article down wondering; 'who the hell conducted the qual research he's seen?!'

One can only assume poor Dan has seen some truly awful and worryingly unsophisticated qualitative research. In the interview he says "Let's stop doing as many focus groups... ... let's stop getting cold calculated reactions to decisions that have an emotional component". It's difficult to argue with the second bit of this. Moving away from rational reactions to things that people react to emotionally 'in the real world' has been one of the great challenges of research for as long as it has existed.

It's the linking of this so definitely with focus groups that puzzled me and got me exploring the other places he's put the boot into groups.

On his blog he talks about how clients "rely on the intuition of about 10-12 lay people with no relevant training who ultimately have no idea what they’re talking about". This feels like a very old fashioned and strangely simplistic view of groups. It's certainly not what groups are used for here in the UK, and one hopes the US.

What is said in the group is only part of the story. The value of groups comes from two things, neither of which comes instantly from the potentially rationalized reasons given by respondents:

1. Non verbal communication the moderator picks up on, the energy in the room. What creates a spark? Often this will be a spark which respondents then struggle to rationalize or in so doing actually miss what really got them going. No matter, a good researcher will have picked up on this spark (a shift in the temperature of the room) during the group and can then explore it's meaning later (which brings us on to number 2).

2. The analysis. No client should expect to come away from a night behind the mirror with the answer (although they should have been inspired to perhaps look at the problem a fresh in light of what they heard). Answers don't come directly from the mouths of respondents. Respondents are a great resource but we're not relying on them to save the day. Answers come out of the analysis process. The time taken by researchers thinking about what occurred in the group (and this goes further than just what was said), what this means to the client, and how it can be developed into actionable insights.

As qual researchers we recognise that clients can have an "insatiable need for a story" (another of Dan's concerns regarding focus groups). Calming this insatiable need is something good quallies are adept at too. We've all had a client who comes along to one group, hears one juicy and compelling story from an interesting respondent and think they've got the answer. However, they haven't. Good researchers are aware of this and have the skill to ensure that the client soon is too.

Dan does a great job of highlighting the pitfalls of bad focus groups and what behavioural economics can bring to the party. I just can't agree with his extrapolation that this means we should abandon focus groups. Surely a wiser conclusion would be to suggest we abandon doing focus groups badly. Otherwise we're throwing the baby out with the bath water.

Therefore I would like to take this opportunity to invite Dan to come and watch qualitative research being done well, and then used intelligently. I'm fairly confident you'll find they aren't a "waste of money" but have a role to play in understanding consumers better; just as quant research, ethnography and behavioural economics do. All have value but none are the definitive answer.

Come on Dan, there's free sandwiches*

*Or we can charge you £10 for them if it will make them taste better.