Methods matter - illustrating quality in qualitative analysis and the role of CAQDAS

By Christina Silver on Jan 06, 2018 at 11:28 AM in CAQDAS commentary

by Christina Silver, 6th January 2018

Professor Debra Jackson’s post about the Journal of Child & Family Studies’ intention to from now on only review and publish quantitative papers and the discussion it prompted on Twitter indicates how important it is for qualitative researchers to fully describe their methods and illustrate the quality of their analysis. Using dedicated CAQDAS packages to facilitate analysis won’t necessarily result in higher quality outputs, but they can be used to illustrate process and rigour, and thereby have an important role to play. 

 

Methods matter - illustrating quality in qualitative analysis and the role of CAQDAS

Nonsensical paradigm wars

I’ve never understood the polarisation of qualitative and quantitative methods. Whilst it’s true that some of us are more highly trained in, or simply prefer one or other set of methods, it’s not true that one or other is better or more appropriate in all circumstances. Methods should be chosen or constructed to best suit to the research question at hand. Sometimes that will be quantitative methods, sometimes qualitative, and sometimes mixed. 

 

Undervalued methods

Many qualitative researchers feel their methods are undervalued in comparison to quantitative methods, and decisions by journals to only publish quantitative papers will inevitably fuel this perception. But I’ve had many discussions with quantitative researchers who suggest their methods are not valued either. Academic and applied expertise in general is currently being questioned by politicians and the media - something that was debated at the Social Research Association’s annual conference in December, entitled “Social Research in a Sceptical Age”

 

Methods matter - illustrating quality in qualitative analysis and the role of CAQDAS

 

Methods matter

One problem is the restrictions journals place on how research papers are written. Some are more formulaic and prescriptive than others, but there’s often insufficient space within one paper to adequately describe and discuss methods as well as findings. We’re all under pressure to illustrate impact, and many therefore need to prioritise the substantive elements of research rather than the methodological. But illustrating the value of empirical findings is based on the methods that are employed to generate them. Methods matter. 

 

The way we write-up research

One way to get over the restrictions of journal paper requirements and the often resulting suppression of methods is to plan from the outset a series of publications. Perhaps starting with one that describes in detail the issue being investigated, the research design, methodology and the analytic strategies and tactics. Then subsequent substantive papers can reference the methodological publication, thereby both freeing up space to concentrate on sets of findings and ensuring that the underlying methods can be adequately scrutinised. 

 

No excuses— using CAQDAS to illustrate quality

For too long what qualitative analysts actually do to get from their data to their interpretation has been masked in too much mystery. But this neither should, nor need, be the case. Since dedicated software designed for the purpose became available - way back in the 1980’s - we’ve been able to illustrate our processes in ways that are concrete. But not enough researchers do so. In an environment where our expertise and methods are increasingly scrutinised it’s important to spend the effort showing what we do and how we do it. And CAQDAS packages - whichever one you choose - are powerful platforms for doing so. 

 

Because as soon as you use one of these software programs to manage and analyse qualitative data you are able to show others what you’ve done, how you’ve done it, and the basis of your interpretation in the data. And that’s a powerful and concrete way of illustrating quality.  

  

A few useful references on this topic

Bringer, J. D., Johnston, L. H., & Brackenridge, C. H. (2004). Maximizing Transparency in a Doctoral Thesis1: The Complexities of Writing About the Use of QSR*NVIVO Within a Grounded Theory Study. Qualitative Research, 4(2), 247–265. http://doi.org/10.1177/1468794104044434

Davidson, J., Paulus, T., & Jackson, K. (2016). Speculating on the Future of Digital Tools for Qualitative Research. Qualitative Inquiry, 22(7), 606–610. http://doi.org/10.1177/1077800415622505

Jackson, K. (2014). Qualitative methods, transparency, and qualitative data analysis software: Toward an understanding of transparency in motion. Available from ProQuest Digital Dissertations. (AAT 3621346) Google Scholar

Lu, C.-J. J., & Shulman, S. W. (2008). Rigor and flexibility in computer-based qualitative research: Introducing the Coding Analysis Toolkit. International Journal of Multiple Research Approaches, 2(1), 105–117. http://doi.org/10.5172/mra.455.2.1.105

Odena, O. (2013). Using software to tell a trustworthy, convincing and useful story. International Journal of Social Research Methodology, 16(5), 355–372. http://doi.org/10.1080/13645579.2012.706019

Oswald, A. G. (2017). Improving outcomes with Qualitative Data Analysis Software: A reflective journey. Qualitative Social Work, 1473325017744860. http://doi.org/10.1177/1473325017744860

Salmona, M., & Kaczynski, D. (2016). Don ’ t Blame the Software : Using Qualitative Data Analysis Software Successfully in Doctoral Research. Forum: Qualitative Social Research, 17(3), Art. 11. http://www.qualitative-research.net/index.php/fqs/article/view/2505

Sin, C. H. (2007). Using Software to Open up the “Black Box” of Qualitative Data Analysis in Evaluations: The Experience of a Multi-Site Team Using NUD*IST Version 6. Evaluation, 13(1), 110–120. http://doi.org/http://dx.doi.org/10.1177/1356389007073684

Wickham, M., & Woods, M. (2005). Reflecting on the Strategic Use of CAQDAS to Manage and Report on the Qualitative Research Process. The Qualitative Report, 10(4), 687–702. Retrieved from http://www.nova.edu/ssss/QR/QR10-4/wickham.pdf

Woods, M., Paulus, T., Atkins, D. P., & Macklin, R. (2015). Advancing Qualitative Research Using Qualitative Data Analysis Software (QDAS)? Reviewing Potential Versus Practice in Published Studies using ATLAS.ti and NVivo, 1994–2013. Social Science Computer Review, 0894439315596311. http://doi.org/10.1177/0894439315596311

Zhao, P., Li, P., Ross, K., & Dennis, B. (2016). Methodological Tool or Methodology ? Beyond Instrumentality and Efficiency with Qualitative Data Analysis Software, Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, [S.l.], v. 17, n. 2, apr. 2016. http://www.qualitative-research.net/index.php/fqs/article/view/2597/3984







Testimonials

The 3 day NVivo course was based around Five-Level QDA and was an excellent balance between the technicalities of using analytical software and its real-world use as a research tool. Christina grounded the training in how to conduct high quality post-graduate research as a means to enhance good research practice, allow effective data analysis, and support research project management. This is far preferable to an approach focusing largely or wholly on the technical details of software, with little relation to conducing actual academic research. Although the 3 days was very intensive it comes highly recommended.
Simon Cox
PhD Student, Nottingham Business School, Chief Officer, NHS Scarborough and Ryedale Clinical Commissioning Group