by Christina Silver, Director. CAQDAS Networking Project (also posted on the QDAS blog)
I got a bunch of interesting comments on my LinkedIn post asking who’d be interested in what I think about AI in qualitative research, and why, so here goes….
There are 6 sections. I could actually write a post the length of this one on each topic, but instead I’ve prioritised some “headlines” as I currently see them under these headings:
- Background: from whence my thoughts derive
- What I think of the development of AI tools for Qual
- What I think of the capabilities of AI tools for Qual
- What I think about the implications of AI on Qual
- How I will – and will not – use GenAI
- The evolving relationship between methods and tools
It’s quite a long post, so you might need coffee…
If you can’t be bothered to read it, or if you prefer a listening-experience, you can head over to my #CAQDASchat with Christina podcast and listen while you walk the dog, or do the washing up. Or if you really want to, you can watch me tell this story on YouTube, instead… 😉
Oh, and I should say, I’ve written this entirely myself. Although I’m not against the use of GenAI for certain writing tasks1, I like writing, and so I’ve written this myself. Even though it took me many hours over several weeks to pull it together. I hope you think it was worth my time writing it, and that it’s worth yours reading it.
1 see one reason why here in response to a piece posted on LinkedIn about university professors now refusing to mark students’ writing that has been in any way facilitated by Generative-AI.
Background: from whence my thoughts derive
I’m personally fascinated by, and professionally engaged in, the rise and implications of GenAI for qualitative research, and in how various communities are responding. My thinking on these matters is continually evolving, but my main interest, and the context from which this piece is written, is in the relationship between qualitative methods and tools.
If you know any of my work, you’ll know I’m an advocate of methods driving the use of tools – or, in the language of the Five-Level QDA method, the CAQDAS pedagogy I developed with Nick Woolf – that strategies drive tactics. What that means is that qualitative researchers use tools (whether manual, digital or AI) to enact methods in the service of accomplishing research objectives, rather than the other way round (i.e. finding a way to use tools, just because they’re there)2.
2 Yes, I know tools can inform methods, and that’s okay, but I don’t think tools should drive methods. The distinction is subtle but important.
Professional positionality: awareness-raising and capacity-building at the CNP
My role at the CAQDAS Networking Project (CNP) involves developing a broad and deep understanding of what’s happening in the Computer-Assisted Qualitative Data Analysis space so the project can continue to raise awareness and build capacity in methodologically appropriate use of tools. This requires a balanced approach that takes account of the range of tools and their capabilities, as well as diverse perspectives around their development and use. As such, my approach remains – as it has always been – pragmatic and pluralistic3.
3 If you want to know more about this approach, look out for this forthcoming chapter by me: Silver C (under review) The Five-Level QDA Method in the Gen-AI Era: Rethinking Qualitative Pedagogy and Practice. Invited chapter for D. Morgan and S. Friese (Eds.) Qualitative Data Analysis with Artificial Intelligence: Theory, Methods and Practice.)
Sometimes I’m challenged for taking this approach – I’m frequently asked which is the “best” tool, and encouraged to share my ‘favourites’. But honestly, I don’t believe there is a “best” tool, just like I don’t believe there is a “best” method(ology)4. Instead, I believe there are only appropriate methods for the research questions at hand, that need to be enacted through the most appropriate use of tools.
4 see my “rant on that image” post about the Quant-Qual paradigm wars
What constitutes the most appropriate use of tools is something I can certainly facilitate researchers to think about, but at the end of the day, it’s the responsibility of those doing each project to decide – and justify – what methods and tools are appropriate. I would never deem to suggest that I’m font of all such knowledge. No one is. I know a lot about qualitative methods and tools, but you are the expert in your project.
My inverted hype-cycle: from scepticism to pragmatism
Pretty immediately after OpenAI released ChatGPT in November 2022 hype about what the capabilities of Large Language Models (LLMs) meant for qualitative research began. My initial reaction was a mix of scepticism, concern and intrigue.
- scepticism about whether these tools could actually do what it was being claimed they could
- concern about what these technologies would mean for the field of qualitative research
- intrigue about how researchers would react to the capabilities and how developers would adopt them
It was clear I not only needed to get to grips with what was happening and how it might impact the field, but that it was my responsibility to do so.
But it would be dishonest to say that I was initially excited. As such, I didn’t follow the typical “hype cycle” proposed by Gartner. Rather than starting out with “a peak of inflated expectations” after the technology trigger, followed by the “trough of disillusionment” before over time going through a “slope of enlightenment” before reaching a “plateau of productivity”, my experience began with a “trough of disillusionment”, followed by an inverted “slope of enlightenment” on the way to a “plateau of productivity” that I am yet to reach. There has been no “peak of inflated expectations” for me.
Fig. I. Comparing the classic “Gartner Hype Cycle” following a technological trigger, with my own experience with Generative-AI for Qualitative Analysis since November 2022
So pragmatism prevailed. It was obvious very quickly that qualitative researchers were using these tools. Whether me, you or anyone else likes it or not, this is a reality. We therefore must discuss it. Every angle of it. And in a respectful and open way.
Let’s play nicely, please
A lot of people have their heads in the sand about what’s happening, or are up-in-arms about it, criticising students and fellow researchers who are experimenting with and/or using Generative-AI tools for aspects of the qualitative research process. That’s not helpful. Ignoring what’s happening or demonising those who do use Generative-AI is in my belief, actually harmful.
That’s not to say I advocate wholesale use of Generative-AI for qualitative research for its own sake. There’s a lot of rubbish being produced out there right now in the name of qualitative research, and many inappropriate and unethical uses of these technologies. This is equally dangerous.
My point is that the community – in all its diversity – must come together to debate the issues in open, collegiate and meaningful ways. We should call out the charlatans, absolutely. But to demonise those who are using Generative-AI, without being open to understanding what they’re doing, why they’re doing it, and how, belies a form of dogmatism that disappoints me.
I’ve actually been impressed with the openness of those attending my sessions, who do not all attend as advocates, as you may expect. Typically, there’s a range of “gut feelings” amongst groups, that I capture at the beginning by asking them. Some of you will have seen the examples I’ve occasionally posted on LinkedIn. Here’s the results of circa 550 participants attending various of my sessions in the past few months. These are researchers attending sessions about the use of Generative-AI for qualitative research. It’s a testament to the practitioners in the field that those who are sceptical attend such sessions.
My Qual-AI experiences: adopting a participant-observer role
So how did I get to this position? Since early 2023 I’ve designed and delivered dozens and dozens and dozens of awareness-raising, training and capacity-building sessions, of varying formats and lengths, on topics related to AI-assisted qualitative research. Not just via the CAQDAS Networking Projectbut also in open workshops for other providers, including the Social Research Association, Research Accelerator,Instats, and my own company QDAS, and for numerous bespoke clients and conferences etc. I must collate a list!
These experiences have allowed me to engage with several thousand researchers and students around the world working on qualitative projects in a variety of contexts. While sharing information about tools and fostering critical thinking about their impact on methods, I’ve learnt a lot about the evolving state of the debate concerning Generative-AI in qualitative research spaces.
I’ve spoken to and taught students and emerging researchers, established researches and academics, government researchers and policy makers, industry and commercial analysts and qualitative practitioners.
To be able to design and deliver these sessions I’ve of course had to learn about LLMs, Generative-AI and the tools that harness these capabilities for qualitative research applications. This involves continually experimenting with tools as they evolve, so I can think about and discuss how they may or may not be useful for different types of qualitative research.
This is actually nothing new for me. I’ve been doing this since 1998 when I first began working with the CAQDAS Networking Project. What’s different now is the scope and pace of development, and the nature of debate in different communities of practice, and on social media. Way more tools are out there that I need to get a handle on, and the speed of development is staggering in comparison to what we’ve seen over the past 40-odd years since the CAQDAS field emerged.
This blows my mind and is sometimes overwhelming and exhausting, but I also thoroughly enjoy playing with tools, talking with developers and thinking about the implications. I’m continually grateful for how open most CAQDAS developers are with me about what they’re doing in the space.
As such I’ve essentially been a “participant-observer” in this space over the past two and a half years. It’s been a privilege to work with so many researchers around the world who have shared their experiences and thoughts so openly, and asked questions that have got me thinking in different ways.
What I think of the development of AI tools for Qual
Although my focus and specialism is in the use of tools for qualitative data analysis (whether AI or otherwise), I don’t believe this conversation can be had outside of a broader discussion about the development of “AI”. So that’s where I’m starting: here are my “headlines” on this topic as I currently am thinking about them.
Knowing what we’re talking about: What even is “AI”?
Discussion about “AI” for qualitative research – and decisions about whether to use tools – can only be had in the context of understanding what AI is, how it works and what the implications of its development and use are.
The multiple definitions of “AI”, including those stemming from technical, marketing and critical perspectives are – in my experience – not well-understood in qualitative communities. How can we engage meaningfully with what’s happening without exploring these definitions, where they come from, who advocates for them, and how they shape our thinking? Some of the more influential authors on my thinking in this regard include Simon Lindgren, Emily Bender & Alex Hanna, and Karen Hao.
So how do I understand “AI”? In all these myriads of ways and contexts. In relation to my main area of specialism: computer-assisted qualitative analysis, it’s super important to remember that “AI” tools are not new. For example, we’ve had supervised and unsupervised machine-learning tools in this space for years. The current focus on Generative-AI tools, must be understood in that historical context. So if you come on one of my sessions, expect a history lesson.
And terminology in this space is not static. This is something I find particularly interesting and illuminating when I reflect on what’s happening in the Qualitative-AI space. It seems to me that most qualitative researchers, when referring to “AI” nowadays mean Generative-AI that derives from LLMs. But this is not the only form of AI that has impacted our space. We should critically reflect on the difference in uptake and discussion about “traditional” forms of “AI”, that have been around in our field since at least 1999 – more than a quarter of a century ago – in comparison to “generative” forms of “AI”. I have more to say on the “qual-at-scale” discussion in this context, but not here and now.
Instead, I want to finish this part of my reflections by emphasising that in my experience, a lot of qualitative researchers who are considering the use of Generative-AI tools in their projects, do not have a detailed understanding of how LLMs work. Many of them are surprised and ‘enlightened’ when they find out. This is fundamental foundational knowledge to know, in deciding whether and how it might or might not be appropriate to use the tools that harness these capabilities for qualitative analysis.
It concerns me that researchers are using these tools without such knowledge. How can anyone make an informed decision about whether, how and when it might be appropriate and useful, without this knowledge? This is an aspect of the “hype” we need to address. Which brings me on to another neglected area in the space…
Ethics are everywhere, yes, everywhere
The more I’ve learned about the ethics surrounding the development and use of Generative-AI, the more considered I’ve become in how I choose to use it – or not. In my awareness-raising role, this is something you won’t find me skipping over.
Fig. 2 Badge saying “Ethics are Everywhere” that was in the delegate pack of the inaugural International Creative Research Methods Conference organised by Helen Kara, September 2023
In my experience, qualitative researchers tend to think about data security and privacy when considering the ethics of the use of Generative-AI for qualitative research – very important of course (more below), but the ethics involved are much, much broader than that, and as a community we should be discussing them a whole lot more.
As social researchers, how can we decide whether and how to use Generative-AI without taking into account the social and environmental implications?
So, if you come to one of my sessions, you’ll be considering a variety of broader ethical considerations, including:
- the data that LLMs are trained on and how it was obtained and recompensed. Asking questions like, are the models you’re using trained on data that was fairly acquired, and that are fit for the purposes you intend to put them to?
- the environmental implications of developing, running and using LLMs and the inequitable distribution across the globe in terms of the environmental consequences of using them. Do you think about this before you quickly “ask ChatGPT”? I think we should.
- the exploitation of humans involved in e.g. fine-tuning models. Does this sit okay with you when you’re sat at home prompting an LLM to do a task you could easily do yourself?
The point is that the development and use of tools are not benign, and as social researchers we must ask ourselves whether the consequences of GenAI use aligns with our values. This is an ongoing dialogue, and as the field continues developing there may be different ways to respond to the ethical implications.
But this is a conversation that is not being had enough in the qualitative field. Why? I’m not sure, but what I do know is that it is data-related ethical concerns that occupy many qualitative researcher’s minds…
Data security and privacy is a top concern: and so it should be
In my experience, this is what most qualitative researchers are most concerned about. And they are right to be concerned. Bespoke GenAI tools5 for qualitative research usually assure data security and privacy – because such tools are largely developed by companies that have either been in this space for years (“established CAQDAS-packages”) or are new companies focused on developing tools for aspects of the qualitative workflow leaning mainly or entirely on GenAI capabilities.
5 by bespoke tools I mean those that are designed specifically for qualitative research purposes. Currently (in addition to in-house tools developed by organisations for their own purposes) these fall into 2 groups: established CAQDAS-package integrations and New online Apps
I generally trust these companies with regards the safeguards they say they put in place re data security and privacy. Why? Because not only do I know most of them personally (and many of the established ones for decades), I also know that the implications of a data breach are pretty major for them, so I don’t believe they would cut corners in this regard.
However, when using CAQDAS-packages (established or new) that use 3rd party models, often developed by the big tech companies, we also have to trust that the agreements the CAQDAS-developers have with those 3rd party companies will be upheld by those 3rd party companies. This I am less inclined to be trusting of.
Different CAQDAS-packages use different models. They also differ in how open they are about the models they use, for which tasks, and in whether they enable users to choose between models. I urge anyone considering their use to investigate this thoroughly to ensure the safeguards they have in place meet your ethical needs (IRB etc.).
For these reasons, there are certain type of data I wouldn’t upload for GenAI processing (see below for more on what I will and won’t do), unless I know exactly where data is going for which tasks. This said, I do think things will change with models (also see below).
We need to discuss what “informed consent” now means
Although most qualitative researchers I come across are most concerned about data privacy and security, there is less discussion about our responsibilities to the participants who provide such data when we use GenAI to facilitate qualitative analysis.
It’s obvious that a bunch of people are uploading qualitative data gathered from humans without thinking about or asking those people for their consent to do so. I’ve heard researchers say things along the lines of:
- “it doesn’t matter if I use ChatGPT, or NVivo, or highlighter pens, it’s up to us as researchers, how we do it, it’s just a tool”
- “if I ask participants if they consent to me using AI to analyse their interviews, they’ll say no, so I’m not going to do that, because then I can’t get my project done”
- “if we don’t really understand how it works, how can we expect our participants to, so its’ just better to do what we have always done, assure anonymity, that’s enough”.
Such comments concern me greatly. I don’t agree. What ‘informed consent’ now means needs urgently to be discussed in the qualitative community.
But those of you reading this who are horrified by the above, don’t just slam these researchers, engage with them, understand where they are coming from, what underlies their positions, explain to them why it matters.
This isn’t something that’s being discussed much as far as I’ve seen. In my sessions I always raise this question, encouraging researchers to put themselves in the shoes of their participants via a series of exercises. But as a community of practice, we need to discuss this, urgently.
What I think of the capabilities of AI tools for Qual
I’ve written elsewhere about the broad genres of AI tools for Qual, and the capabilities that derive from them, in relation to the qualitative research process. And the specifics about which tool does what, how well, and for what purpose, is the topic for a workshop or maybe a subsequent post.
Here I want to make a few more high-level comments about the potential role of GenAI, focusing on either questions that researchers are asking me a lot about right now, or topics which I believe are particularly important to discuss.
We didn’t ask for this: what’s the problem GenAI seeks to solve?
When I ask students and researchers – which I often do – why they’re thinking of using GenAI tools for their qualitative projects, I get a variety of responses. Amongst the top five are usually ‘to save time’, ‘to do better analysis’ and ‘to not be left behind’. Each trouble me, but here I want to speak to the final one: fear of missing out, or FOMO as it’s referred to on social media.
If you’ve spent time on LinkedIn recently, and follow any of the people I do, you’ll easily be sucked into the impression that there’s a tide of AI occurring and that if you’re not using it, you’re being left on the shelf. Jumping on the bandwagon is understandable given the amount of hype we encounter. That ‘everyone is using it’ is an oft heard sentiment but is it really true? And if it is true, is this the reason to use it oneself? I think not.
When groups of researchers and computer scientists first began developing software to manage qualitative materials and facilitate the ‘messy’ process of analysis, back in the 1980s, they did so to try and solve an identified problem. Until recently, this has been the case throughout the CAQDAS field. As each new product entered the field, its developers expressed the gap they were filling, or an unmet need they were addressing.
Then in November 2022, suddenly a new capability was thrust upon us and qualitative researchers and developers alike began scrabbling around to work out how to use it. Although I’m pragmatic about the fact that this is the situation we find ourselves in, I’m not convinced this is the best way to develop tools or foster methodological innovation.
Tools are powerful. I love tools. But we mustn’t use them in a methodological vacuum. This brings me on to talk about types of tools.
To use general-purpose or bespoke tools? There is no question
I get asked a lot “why pay for a software program or subscription when I can just use ChatGPT, or another general-purpose tool”? There are lots of reasons. Among the most important is that bespoke tools are just that: bespoke. The discerning developers have researched which AI models to a better job at certain tasks, and they have optimised their tools accordingly.
Without getting into the detail, here’s an analogy that illustrates the point. Think of it as being a bit like a post-office sorting room: letters, parcels, and postcards come into the office, and they’re sorted according to type and destination, by trained postal staff, and they’re then sent out accordingly, perhaps on a train, in a van or by foot, to get to where they need to go within the timeframe specified by the postage stamps, the fragile parcels being given special treatment. The sorting job is what the CAQDAS developers are doing with our prompts, to optimise their fulfilment for our analytic purposes and needs.
None of that happens when using general-purpose tools. There is no sorting office.
There are a number of implications of this, but I’ll just highlight two of the most important. First, the vastly reduced likelihood of “hallucinations” (not the best word as it anthropomorphises LLMs which is unhelpful, but this is the established term for it) when using bespoke tools: CAQDAS packages (established or new) are typically programmed not to give an answer if there is no relevant data. Second, usually bespoke tools provide direct links from summarised responses to the data that the responses are based on – fundamentally necessary for the tasks of qualitative analysis so we can review, validate, reflect, interpret6.
6 have I mentioned interpretation and my thoughts on whether LLMs can do it? See here.
As such, I’m generally impressed with how CAQDAS developers are integrating GenAI tools into their products, in particular those that give researchers choices about how tools work, or the models that are used.
But I would like to see more transparency concerning the methodological underpinnings as well as the technicalities of implementation. This is not the place to outline the specifics – come to a workshop or check out the reviews on the CAQDAS website (which we’re working hard to update at the moment)
What I can say here is that the way different developers are harnessing GenAI (whether into established CAQDAS-packages or in new Aps) reveals a lot about their perceptions of qualitative methods and the needs of the community. I encourage them to discuss this more openly. Some are doing so – but I’d like to see a lot more of it.
Opening up the “black-box” is one way developers that have the good of the field at their heart, set themselves apart from the charlatans – of which there are many, so my advice is to beware the “do qual analysis in minutes” discourses. Despite what some may have you believe, it’s not a click-of-a-button task. Never has been. Never will be.
Methodological Matters: no one-size-fits-all response
Whether GenAI is methodologically appropriate for qualitative analysis is not a question that has a simple yes/no answer. For some purposes it may have a place and be incredibly useful, for others it is wholly inappropriate. The nuance behind making these decisions is often not sufficiently reflected in discussions. There’s a lot of black-and-white “it’s great vs. it’s awful” assertions that are truly unhelpful.
As a community we need to do better – we must discuss the developments and their implications openly and with respect for alternative views. This relates not only to the tools, but also to perceptions of what ‘qualitative research’ is, and what ‘qualitative data analysis’ entails. These are not homogenous endeavours. There’s a methodological spectrum and different ways of working with qualitative materials are appropriate all the way along it.
I’m not a fan of the “GenAI doesn’t align with qualitative principles” mantra. Not certain types of qualitative work, no, but others, yes. The purists have a valid point in the context of their type of work, but dogma about what is “real” qualitative work is naive and patronising.
What I think about the implications of the use of AI tools for Qual
There are many implications of all this on the field, and we are yet to see how it will all play out and what the lasting impacts will be. Whatever your thoughts, we are no doubt in the midst of a fast-moving and disruptive era. Whilst disruption of the status quo can sometimes lead to innovation and in the long-term benefits, we need to be careful how we engage with tools and with one another.
A deepening schism in the field
There seems to be a deepening schism in the field as a result of GenAI, because its potential implications are the most profound and divisive than we’ve seen since the first CAQDAS packages emerged. Advocates and sceptics each have strong views, and many valid points to make. In the middle are a large group of researchers and teachers of methods who are unsure about the capabilities and the implications of these developments.
Such differing perspectives reflect what was seen when dedicated CAQDAS-packages first emerged in the 1980s, but the divergence between ‘camps’ is both more obvious, and more divisive now. When I opened the 2014 CAQDAS conference, I spoke about the ebb-and-flow of the convergence and divergence of the field over time. We’re now firmly in an era of divergence.
There are also differences in how qualitative researchers working in different sectors are responding and using GenAI tools for qualitative work – academics are typically most sceptical and resistant. Other sectors (e.g. industry (market research) and government social research) appear to be embracing GenAI more (although not exclusively).
It will be interesting to see what happens in coming years, perhaps we’ll see divergence between sectors, as some adopt GenAI and others are more reluctant to do so.
Either way, what I hope for is more open dialogue, and less demonising.
Is coding dead? I don’t think so.
Among one of the most powerful things the advocates say is that we no longer need to code qualitative materials to do qualitative analysis. The ability to engage in a dialogue, or ‘chat’ with qualitative texts using a conversational interface is a different way of engaging with material, and for some it’s a game changer.
Although I can see its value for some purposes, I don’t believe we have seen, or are near to seeing, a ‘paradigm shift’ in how qualitative researchers do qualitative analysis. Why?
First, because it’s not the case that most qualitative researchers are using these tools – either at all, or instead of other tools. For a paradigm shift to take place requires more than simply a new capability, that a few people are using or advocating.
Second, using GenAI interfaces to ‘chat’ with qualitative data is not a new analytic method in itself, but just a different way of doing it. In the language of the Five-Level QDA method, it’s a new tactic, not a new strategy or methodology (which would be required for a paradigm shift).
Third, just because it’s possible, doesn’t mean other tactics – for example coding – are redundant. I agree qualitative analysis can happen without coding – after all this is not a new idea; we’ve long had qualitative methods that don’t rely on coding. We can learn something from the history here too. When CAQDAS packages emerged in the mid-1980s we didn’t suddenly see everyone throw away their highlighter pens. In fact, there are still many researchers who do qualitative analysis using pen-and-paper methods – these are the tactics they choose to enact their analytic strategies.
Using GenAI to ‘chat’ with data via the conversational interface is a different tactic, but I’m not convinced that it’s a new analytic strategy or that it is – or will – take over from the humble highlighter pen, the use of general-purpose software (Word, Excel etc.) or the advanced qualitative coding functionality in the established CAQDAS-packages that many of us have been using.
For me, one reason is that I like coding. I like considering deeply what bits of data mean in the context of other bits of data. And although I can see how GenAI chatting can be useful in some circumstances, we do lose something by not gathering instances of concepts via the coding process. I’m not yet ready to let go of that.
This is not to say I can’t see a place for ‘chatting’ with qualitative data. In my sessions we look at its potential in a variety of ways, and it can be useful. But for me its use is most powerful when used in combination with, integrated coding, not wholly instead of it.
Maybe we’ll see qualitative coding become less dominant as a tactic for enacting methods as time goes by, but we are far from that right now, and likely, if the history of the past 40 years is anything to go by, we’ll not see every qualitative researcher – or maybe even most – do their analysis using these tools.
Methods teachers must step-up to the plate
I want to finish up this part by talking about something that I think is of utmost importance right now, and that’s the role of methods teachers. A lot of hype around GenAI suggests that it can save us a whole load of time, generate deeper insights than we mere humans can, and do a bunch of mundane tasks that we would rather not do ourselves. Hence much narrative is around replacing. This is neither helpful nor true.
There’s also a lot of concern about how the next generation of researchers will learn how to do analysis and whether students are ‘cheating’. I do have serious concerns about how some “researchers” are using tools and talking about qualitative analysis in the context of GenAI: what I call “qualitative deepfake” is real and is a serious threat to the profession, the status of knowledge, and the robustness of societal decisions made on the back of qualitative research.
As a community we need to call this out. There are many self-proclaimed “experts” who’ve popped up out of nowhere, sharing ‘cheat sheets’ and proclaiming qualitative analysis can be done in minutes. Not so.
And so, I believe the teachers of qualitative methods must integrate discussion of GenAI into curricula: to neglect to teach current and future students about GenAI-assisted QDA is irresponsible. I hear many personal misgivings about the role of GenAI in qualitative analysis from methods teachers, and suggestions that there’s a ‘lack of time in the curriculum’, not just for teaching about GenAI but qualitative software tools more generally. These are not good reasons to neglect such teaching, and I do wonder if these are just excuse based on personal opinions rather than the good of students.
It doesn’t have to be a choice between teaching methods or tools, even if there is limited time in the curriculum. It is possible to teach methods and tools together and even to teach methods via the use of tools.
Whether we like CAQDAS-packages or AI tools or not, and whether we use those tools ourselves or not, as teachers of methods I believe we do a major disservice to our students by not teaching them about tools and how to think about use them appropriately. In the AI era that is WAY MORE important than ever before because, like it or not, people are using AI. If we don’t teach students how to do so appropriately then how can we expect them to use it with integrity, or make an informed choice not to?
I know from personal experience this is possible within limited timeframes. If you’re a teacher and you don’t know how to do this, be in touch and I will be happy to help.
How I will – and won’t – use AI tools
As a result of all the above, I’m selective in how I use GenAI tools and encourage others to consider their implications of their use very carefully.
Here’s another little analogy: We make many choices every day that affect our individual environmental impact on the world, like how diligent we are about recycling, whether we grab another plastic water bottle from the shop shelf when we’ve forgotten our reusable one, or reduce the amount of red meat we eat on a weekly basis. So too, our everyday decisions about whether and how to use GenAI, and for which tasks, are individual moral judgements.
My primary use of GenAI up to now has been to explore what it does in order to inform and fulfil my awareness-raising and capacity-building remit. This has involved a range of experimentations, with a wide variety of tools, using a range of amounts and types of data, considering how GenAI tools can facilitate the enactment of different qualitative methods. In addition to these experiments, it has included using it for some of my own research. But that’s for another post.
- I would only use GenAI for qualitative analysis via bespoke CAQDAS tools and will not upload research data to general-purpose chatbots. Why? Because they’re not designed for qualitative data analysis purposes and therefore are not the best choice for such tasks. Also I am not trusting of the big tech companies.
- But, although I do believe that the CAQDAS developers have in place what they believe to be robust agreements with the model platforms, I’m not sure I believe that all of those 3rd party businesses will respect those agreements. I would therefore think very carefully before using even these GenAI tools with certain types of participant data.
- I therefore like the ability to choose which models to use, and I look forward to a future which sees more such choices and also models that are developed fairly and are more fit for purpose than the big tech companies’ models.
- I will not use GenAI to create images, music or other creative artifacts for any purpose. Why? Because they’re generated from training data that was highly likely to have been stolen without recompense. In addition, they look rubbish. I do not like looking at AI-generated images, I do not like listening to AI bots delivering podcasts, I do not like listening to AI-generated music. And I really don’t understand the point.
- However, if I am prepared to use general-purpose LLMs for some tasks, and when I do, I am willing to have my prompts become part of the ongoing training data set. Why? Because if I am going to teach how to use them appropriately, and use them myself for some tasks, I necessarily want better models and therefore should be open to contributing to making them better.
- I will not use Chatbots to search the internet for routine questions or tasks that a simple Google search (or similar) can do. Why? Because asking ChatGPT (or equivalent) a question costs more energy than a google search. And because it’s not designed for the task.
- I will continue to familiarise with the developing capabilities of LLMs and tools harnessing GenAI for qualitative research purposes so I can continue raising awareness about their capabilities and consequences to facilitate researchers make informed choices about their own use. This means I will use them.
- But I minimise the re-generation tasks – I don’t recreate what I already have unless I have to for teaching purposes, so I have a bank of already created examples that I show, rather than redoing each task each time I teach.
- And I would not use GenAI for qualitative analysis without the explicit informed consent from participants, and I would only use GenAI for commissioned research where the commissioning body requested it.
The evolving relationship between methods and tools: we have agency, let’s use it
I’ve spent my career thinking about the relationship between qualitative methods and tools. How are qualitative methods enacted through the use of tools? How do tools influence the development of qualitative methods? Where does the balance between the two lie in different contexts? These are amongst the questions that have occupied my mind for 30-odd years, and now, in this era of Generative-AI that has been thrust upon us, it continues.
More urgently than perhaps ever before, so not only will I continue to raise awareness and encourage critical reflection about the use of tools, I will urge the community at large do engage in the debate. Because it matters.
If it matters to you to, then whatever your position in the debate, you can have a role at shaping what happens from this point on. Share your concerns, discuss the possibilities, shape how CAQDAS developers harness capabilities.
But I won’t push the use of Generative-AI over and above any other tool. I meet researchers where they are in their process, thinking and tool adoption, and take pride and enjoyment in discussing the issues to facilitate them to make informed decisions that work for them.
Although I am spending a lot of time discussing and teaching about GenAI in Qual spaces, I continue to do a lot of work with researchers using other tools. Because other tools still have a place, and are often more appropriate.
And often I take a digital detox completely, and get down with data in with my highlighter pens and scissors in the garden, to connect with them through my senses. If you want to work with me to discuss any tools for qualitative analysis, subscribe to my Linktree to be alerted when new sessions are scheduled. And if you want to hear more about what I do in my garden when the tech isn’t in the room, why not check out a workshop I’m designing with Anuja Cabraal and Daniel Turner in Edinburgh and have some of what we like to call “qualitative fun”
AI use: I wrote this entirely myself.
