The new series of X Factor has just hit our screens and most of us love (even though we may not admit it!) to watch the contestants’ performances every Saturday night. What we don’t see is all the hard work – rehearsing with the musicians, the lighting, the costumes, the camera work – that goes into creating each Saturday night’s performance. The other vital ingredient that makes a successful performance sparkle is the ‘X’ Factor – the intangible quality that some contestants have and others don’t.  This has little to do with technical talent and everything to do with the contestants’ personal story, their experience and their character. As viewers of the show, we are presented with an edited version of each contestant, typically showcased in a two minute song, briefly introduced with their personal story.

Research is often presented in a similar way; we share nuggets of the ‘story’ and impactful findings but we don’t often share all the hard work that goes into making our research successful.

Qualitative researchers love to talk to our clients about our fieldwork methodologies, our outputs, our respondents, our samples, our timings, our modes of engagement, our deliverables – the list goes on. We very rarely talk about our analysis processes – how we do it, how it will add value, how we know it is robust, how we know it is objective. Analyzing our qualitative research findings is often the most intensive phase of our projects – it is the crucial link to transform hours of fieldwork into meaningful insight.  Often, more time is invested in analysis than in any other part of the research process – and rightly so. Most qualitative researchers would probably agree that it can be the most satisfying part of our role and yet curiously, it is the phase we talk about least.

Our approach to analysis is what gives our research the ‘X’ Factor.  This short blog post discusses why analysis is often neglected in research design and suggests ways of ensuring that analysis is as much a part of the research process as collecting the data.

Analysis: the elephant in the room

There are a few reasons why this crucial part of our methodology is neglected when qualitative researchers talk to clients about what we offer.  Firstly, fieldwork approaches are often considered to be the ‘sexy’ part of our methodology, the part of our work where we engage and immerse ourselves (and often our clients) in the lives of consumers – fieldwork attracts the spotlight!  Secondly, we assume that our clients know how we carry out qualitative analysis and as a result we assume that they are not interested.  Thirdly, because it is difficult! Explaining the methods we use to ensure that our qualitative research is robust, objective and truly adds value is tricky and when you are working across multiple and diverse international markets, it becomes even more challenging.  As new qualitative methods such as using social media and online communities become more mainstream, these analysis processes will become increasingly complex.

Our qualitative research projects lead to reams of transcripts, videos, vox pops, online discussions and other outputs.  At first glance, making sense of this can appear a very daunting task.  In our experience, every researcher (and client) approaches a fresh research project with a set of assumptions, values and beliefs. Some of them are very much at the forefront of our minds and some are buried away at the back, making it difficult to acknowledge them, and therefore even more difficult to challenge them. Some of these assumptions are helpful and others are not.  But how do we know the difference?

Minimizing bias and maximizing judgement

Below are some thoughts on how we can ensure that analysis limits bias and helps us to feel more able to openly discuss our approach and thinking when trying to ‘sell’ our research.

Ideally, the analysis process will allow space to ensure that judgement is maximized and biased assumptions are minimized.  Some of our methods for minimizing bias and maximizing judgement are described below.

• Convene an ‘assumption amnesty’ – before the fieldwork takes place, where all researchers (and clients) share and log their assumptions ahead of the research process.  This log should be revisited before, during and after the analysis phase.  Assumptions should be challenged and can be translated into hypotheses where appropriate.  The entire process of qualitative research can support businesses on transformational journeys, simply by posing challenging questions and tackling underlying assumptions.  This is where qualitative research works best.
• Analyze in diverse teams – different academic and professional backgrounds will help with the process of challenging assumptions and will also bring new ideas to the table.
• Don’t ignore hunches – as described above, judgement and bias are often confused.    Successful analysts of qualitative research will be comfortable when applying their judgement.  After all, our clients come back to us repeatedly, precisely because we are industry-focused researchers with an understanding of the nuances of the technology sector. Great research is about attention to detail, expertise, impartiality and rigour but it’s also about resourcefulness, creativity, new ideas and passion.  Analysis in qualitative research is a process of making sense of complexity.  Having an idea of ‘sense’ is based on previous experiences and judgements, so it is important to acknowledge this.
• Be reluctant to reduce – we sometimes think that applying a ‘model’ to our research somehow validates it, removes it from judgements, and makes it more objective.  It could just as easily be argued that the opposite is the case.  There is a temptation within any type of research to reduce analysis processes to a framework or model.   Shoehorning primary fieldwork into predetermined models actually reinforces – and in some cases validates – our predefined assumptions and values, some of which we should be rejecting.   Models and frameworks work best when they are used to ensure consistency and quality, for example across multiple markets.  But they should be used with caution when they assume particular value sets or take things for granted.

 Making sense of the complexity  

 Therefore, in a similar way to the ‘hard work’  (hours of rehearsal, the singing lessons, the camera tricks etc.) overlooked in a contestant’s X Factor story, analysis is often the elephant in the room when it comes to talking about qualitative research methodologies. Especially for non-researchers (including many clients) there currently remains an air of mystique around the analysis process.

Qualitative research should be loaded with creativity and judgment while still retaining objectivity.  To tread this line successfully, it is necessary to retain objectivity by owning up to the assumptions and letting them go while being comfortable with drawing on previous experiences, using our judgement to make ‘sense’ out of the complexity.  The key to successful analysis of qualitative data is not always about applying grandiose theories, but about making sure that analysis models are applicable, appropriate and challenge any preconceived assumptions.

While analysis remains a topic to be brushed over very quickly during most discussions with clients, GfK TechQual wants to bring about change, making analysis a real talking point.  With an approach that helps to maximize objectivity and minimize bias, analysis can lose its air of mystique and can become something we – and our clients – feel confident to talk about.

No related posts.

Tags: analysis, impactful, Methodology, qualitative, qualitative research, qualitative researchers, Research, research findings, story