My thoughts on AI in qualitative research.

I recently read an article in the online periodical Inside Higher Ed entitled "Can we trust AI in qualitative research?" by Andrew Gillen. This title immediately caught my attention because I have experience with using AI in my qualitative research and have similar concerns with one caveat - AI systems will most likely get better over time. AI will improve despite its many problematic characteristics; the potential for "hallucinations," built-in biases and even overt racism, copyright infringement issues, and the black box problem of how AI "makes decisions." Not to mention the increasing environmental impact of using AI. Anyone who has been following the development of AI companies like Perplexity, OpenAI, and others are no doubt familiar with many of these arguments for limiting the development of AI. Even if none of those issues were at play, there is still a vital and persistent argument about the role of human creativity and intelligence in the face of such technological power. If AI is indeed "intelligent" (the jury is still out about this)then what of human intelligence? Will humans be made obsolete? Will white collar knowledge workers be laid off en masse? Or, as techno-optimists opine, will the advent of artificial general intelligence usher in a diamond age of untold prosperity, creativity, and increased leisure? The answers to those questions will be answered in an overwhelmingly supportive capatalist political environment where the big technology companies that already contral these systems will use their billions in venture capital to expand this technology and convince the average person that it is in the best interest of progress. But I digress. In the here and now, what of using AI in qualitative research? Is this an evil practice that undermines the very validity that qualitative research has gained in the last 30 years? Perhaps. Is there any validity to the use of AI in our research? Possibly - if deployed with intentional guardraisl and robust training. In 2023-2024 I was one of those doctoral students who wrestled with a dissertation committee to deploy ChatGPT as a part of my qualitative data analysis. I went into the dissertation proposal process seeking to be fully transparent with my goal to utilize AI in my research. My goal was not to use AI wholesale to complete my data analysis. I was, and am, completely committed to the manual, iterative, and cyclical practices of generating codes and themes, developing thick, saturated descriptions, and using the researcher's positionality to make meaning out of the data set. My goal was to engage in my own manual data analysis first, and then use ChatGPT as a secondary collaborative or confirmation coder. The results were interesting to say the least. Manually, I broke down a series of interview transcripts to ascertain codes and themes using techniques and guidance from Saldaña's seminal work, Qualitative Data Analysis. After I was satisfied with that work over multiple cycles, then I applied ChatGPT. I broke apart the interview transcripts according to their corresponding research questions. Then, I cut and pasted the transcript chunk into ChatGPT with the prompt, "Qualitatively analyze this interview segment and tell me the codes and themes you find." I wanted to compare what the AI generated to what I had previously generated through my own processes of meaning-making. The results were eerily similar. ChatGPT created more codes than I did, however, many of the extra codes it generated could be easily combined and condensed into the same themes I'd already generated. The themes it generated were nearly identical to the ones I created. While I found that to be fascinating in some sense, I also found it to be a bit off-putting due to its implications. I went into this exercise with an unquestioned sense of techno-inevitability and came out with a renewed sense of caution. What the AI responses lacked was nuance, position, and the innate complexity of human embodiment that I brought to the analysis. The AI wasn't aware of the body language of my interviewees. AI didn't understand the larger theoretical concepts I applied to the data. AI lacked an overarching understanding of the real-world context in which I was asking interview questions, my inherent biases and points of view, and the fragments of self through which I interpreted results. I'll say this as it pertains to qualitative research and AI. Researchers and the faculty that train them should absolutely emphasize developing the nuanced skills of data analysis and iterative meaning making that center the real world experiences of the researcher to develop thick, rich, nuanced analyses. When we jump straight to AI to answer our questions, we rob ourselves of agency, creativity, and critical thinking. While it does seem that the developent of AI will continue apace, let's keep it at arm's length as we continue to explore the human researcher's role in qualitative research.

Comments

Post a Comment

Popular posts from this blog

The AI impact we're not talking about in higher education

 Much has already been written about the potential implications of artificial intelligence (AI) being used by students in higher education. Much of what has been written is focused on the harrowing dangers of students who use AI to help write admissions essays, complete homework assignments, and fake their way through college. Faculty members vacillate between outright forbidding the use of AI on homework assignments and trying to find clever ways to incorporate AI into assignments (knowing that students are most likely using these tools anyway). Even as I type this post, Grammarly, an AI writing companion, is correcting my spelling mistakes and recommending style and grammar edits.  These issues will increasingly require faculty, administrators, researchers, and others to grapple with a time-honored question: What is the purpose of an education, anyway? Again, much has been written about this, and a few different camps are emerging across the higher education landscape. Some ...
Read more

Confessions of an Enneagram Type 1

I took the Enneagram survey this week. The Enneagram is an assessment tool developed by Oscar Ichazo to better understand ourselves. There are nine Enneagram types. Each type categorizes you with certain personality strengths and areas for growth. After taking the quick and efficient Enneagram test at https://enneagramtest.com/, my results tallied up to a Type 1 - the Perfectionist... Type 1's are motivated to improve themselves and their lives. They can be perceived by others as ambitious. They are in general very responsible and others can easily depend on them. There's nothing wrong with being a perfectionist, BUT, it does come with its own set of challenges. Perfectionists are known to be detail-oriented, organized, and have a reputation for getting $*#! done. Those describe me fairly well. At work, I keep all my tasks and projects organized on a giant white board that I categorize by projects broken down into small, bite-sized chunks. I relish marking items off this li...
Read more

Bringing learning to life with experiential education

 I've worked in the experiential education realm my entire career. My specific focus is on outdoor and travel education. I've had the privilege to facilitate challenge courses, lead backpacking, kayaking, rafting, canoeing, cycling, sailing, and walking adventures, and introduce students to life in other countries like England, Italy, Portugal, Costa Rica, and Mexico. I never get tired of it. Experiential education is the "learning by doing" we all crave deep inside. It is active, hands-on, engaged, reflective, and iterative. It allows us to experiment, to create prototypes and test them, and to learn by failing and trying again. It can be applied to almost any academic discipline. (I say almost because I can't imagine what an experiential lesson on nuclear particle physics might look like...)  In collaboration with faculty members across my campus, I've combined my background and training in outdoor education to help them bring subjects such as biological ill...
Read more