Case Study - Extending the use of unstructured data analysis technology (NVivo software) from qualitative researchers to the entire university community.

Dr Susana Monserrat-Revillo, School of Sport, Exercise and Health Sciences (SSEHS)

Abstract   

Since 2017, the author has been teaching NVivo software on campus, initially to PhD students and later to PG and UG students. Initially the focus was on teaching NVivo for qualitative research, which is a minority in many major fields but in 2020, the author shifted the focus to teaching NVivo for writing literature reviews (LR). This extended the learning of this software to the entire university community, including quantitative and qualitative researchers/students. Over the last 3-4 years, some UG and PGT students using NVivo for LR have produced high-quality literature reviews, making the process quicker and very beneficial for students with learning difficulties, which lead the author to submit a research proposal to analyse the benefits of using this software among the whole community of students. 

1. Background

  • The author has been teaching NVivo software on campus (天堂视频 and London) since 2017. Initially to PhD students (basic and advanced level) but later extended her teaching on NVivo Fundamentals to PG students (SSEHS) and recently to UG students (SSEHS). 
  • From 2017 to 2020 the focus of her teaching was on how to use NVivo for qualitative research, main purpose of the software, which is only applicable to research that collects data using methods such as interviews, case studies, focus groups or ethnography. It should be noted that qualitative research remains a minority in many major 
  • In 2020, following personal experience and feedback from PhD students (see section 5 below), the sessions focused on teaching NVivo for writing literature reviews rather than for qualitative data analysis. Thus, learning about this technology was extended to the entire research community, quantitative and qualitative researchers/students, because although it is a technology widely used in qualitative industry settings, it is still in the minority in academia. At this point, the author felt that a short version of the seminar could be very beneficial to PGT and UG students doing a LR for their dissertations, and some seminars were offered for the first time to PG and UG students at SSEHS, mandatory for the former and optional for the latter (with an attendance in 2023/24 of  166 PGT Sport Management students (module PSP114 Research Methods and Skills for Sport Managers), 80 PGT Sport and Exercise Psychology students (PSP510 Qualitative Research Methods and PSP003 Qualitative Research modules), while it was offered on a voluntary basis to 475 UG students (module PSC700700 Final Project), of which 35 attended an in-person session.  
  • Over the last three or four years, it has been observed that some of the UG and PGT students who used NVIVO for their dissertations were able to produce high quality literature reviews. Several of them commented to the author that the process was much quicker and more efficient. It was also noted that it was very beneficial for students with learning difficulties such as dyslexia or ADHD, since, as a tutor to some of them, the author was able to see its usefulness for their dissertations. After a short teaching seminar (2-3 h), some students wondered why they had not learned it earlier in the course, as it would have helped them considerably for other long essays or course-type work, prior to their dissertation. 
  • As there is little academic research on the benefits of using NVivo (or any other CAQDAS software) for anything other than qualitative data analysis, I have submitted this year a proposal to the TIA Awards at LU with the aim of gathering evidence on the benefits and setbacks of using NVivo for LRs for dissertations, no matter the field or methods used. Some authors (Beekhuyzen, 2007; O'Neill et al., 2018; Rylee & Cavanagh, 2022) described the steps of the process based on personal experience, or provided detailed guidelines for coding and analysing journal papers from a theoretical point of view (Bandara et al., 2015), but there has been little reflection on the benefits of using CAQDAS to conduct the literature review based on a sufficiently large sample of participants or the quality of the output.

2. Methodology

The initial NVivo courses taught by the author until 2020 were based on the typical structure of how NVivo is taught mainly for data analysis, with a short final section on how to apply it to LRs. From 2020, the focus of the courses taught to all students including PhD and PGT and UG students was changed to focus from the outset on the application of NVivo for LRs, after considering that journal papers are some form of unstructured data, so that the sessions aimed at all 3 types of students focus on learning how to use this software for the non-structured text found in papers, applicable to all students on campus (100%) as opposed to the usual 12% of students who apply a qualitative methodology in their dissertations/research projects. Thus, all students doing a dissertation would benefit from the advantages of this software/technology to read, organise the papers, collate and evaluate the information and analyse and interpret it efficiently and effectively. 

3. Issues

The main barrier encountered has been the lack of knowledge and reluctance on the part of a large part of the scientific community on campus regarding the usefulness of using technology such as NVIVO, taking into account the low penetration of this type of software among staff, both those who identify themselves as qualitative researchers (many of whom continue to analyse qualitative data manually) and researchers who self-identify as quantitative, as there is a certain disdain for anything related to qualitative methodology, which is somehow considered "unscientific".  The author has encountered reluctance to support the expansion of technologies such as NVivo among colleagues and staff at all levels: colleagues involved in Doctoral School workshops; module leaders of PGT courses (i.e. PSP114 - Research Methods and Skills for Sport Managers), and at UG level (i.e. PSC700 - Final Year Project), for different reasons: not knowing the benefits, not having used it before, not understanding how it works, seeming too complicated, not considering it necessary for UG students (when they may be those who need it most).

4. Benefits

Having personally used a mixed methods approach in my doctoral thesis and in my current research gives me a good understanding of the limits and advantages of both quantitative and qualitative approaches, as well as the existence of some misunderstandings between researchers/staff from both "camps". This makes me aware of the need to be persistent and patient in incorporating changes that bring the two approaches closer together and also involve the introduction of technology. Moreover, being aware of the low penetration rate of the technology among many staff members who are not used to it, either because of age or lack of experience, has allowed me to explain to students at all levels (UG, PGT or PGR) that they need to be convinced of its usefulness, as they may have to justify it (e.g. in their annual reports), as not many thesis or dissertation supervisors are familiar with it or do not actively recommend it.

5. Evidence of Success 

NVivo can assist the researcher in reading and interpreting a large number of journal articles and other texts (O'Neill et al., 2018) in a more effective and efficient manner, and this was reflected in the feedback from the Doctoral College workshops given by the award candidate in recent years: the relevant feedback received by the author after some Doctoral College’s workshops can be found here highlighted in yellow: and

6. How Can Other Academics Reproduce This? 

There is a perception that NVivo is quite complex and difficult to learn, but my initial experience teaching it at various levels for qualitative data analysis (6.5 h blocks for PhD students and 3 h for PGT) has shown me that 2 or 3 h are sufficient to be able to apply the basic features of the software to LRs, if properly guided. NVivo is software that is already available on campus and can be applied to LRs in any field, not only in the social sciences, but there is a need for greater awareness. This year a project has been submitted to the TIA Awards (together with Dr Janine Coates) with the aim of gathering evidence on the benefits of using NVivo for LRs (measuring the outcome and its effectiveness), the results of which are intended to be publicised in a number of ways: creating a toolkit highlighting its benefits and linking them to current LR-related resources already available on campus, such as , 鈥痑苍诲 , from the Library and the Academic Language Support Service; presenting at the LU Learning and Teaching Conference; signposting resources available on the LinkedIn Learning platform, and in the future, creating short courses on this content.

7. Reflections

There have been some unexpected or collateral discoveries from teaching NVivo for LRs. Some students with dyslexia or ADHD, or even some international students for whom English was not their native language, told me that they wished they had learned it earlier, rather than in the last year/at the end of the semester, as it would have helped them greatly in organising their readings and collating information for other coursework, such as essays and reports, given the extra difficulty they have in managing unstructured texts. Therefore, it remains to be studied how to integrate NVivo with the tools already available for students with learning difficulties (or non-native speakers) to help them with all kinds of coursework. 

8. References

Bandara, W., Furtmueller, E., Gorbacheva, E., Miskon, S., & Beekhuyzen, J. (2015). Achieving Rigor in Literature Reviews: Insights from Qualitative Data Analysis and Tool-Support. Communications of the Association for Information Systems, 37. 

Beekhuyzen, J. (2007). Putting the Pieces of the puzzle together: Using NVivo for a Literature Review. New Zealand. 

O’Neill, M., Booth, S., & Lamb, J. (2018). Using NVivoTM for Literature Reviews: The Eight Step Pedagogy (N7+1). The Qualitative Report. 

Rylee, T. L., & Cavanagh, S. J. (2022). Using NVivo as a methodological tool for a literature review on nursing innovation: A step-by-step approach. Health Services and Outcomes Research Methodology, 22(4), 454–468.  

Thelwall, M., Nevill, T. (2021). Is research with qualitative data more prevalent and impactful now? Interviews, case studies, focus groups and ethnographies. Library and Information Science Research, 43.