[Critical AI’s TEACHING INSIGHTS series welcomes writing on topics of potential interest to educators and other readers inside and outside of the academy. The below post describes a ChatGPT writing test performed by Jane Rosenzweig, Director of the Harvard College Writing Center.]
Since the semester began, I’ve had several conversations with instructors in different disciplines at different institutions who are encouraging their students to consult ChatGPT for feedback on essays before turning them in. Some of these “consult ChatGPT” assignments include guidance for students about what kind of feedback to ask for and how to ask. In other cases, students are given only general prompting advice (be specific, better prompts get better output). But what happens when a novice writer uploads an essay to ChatGPT for feedback and then has to decide what to do with that feedback? I was curious to see how this might work in practice, so I decided to test the scenario in which students would ask the chatbot for editing advice.
I took a paragraph I had written in a recent article and added three obvious problems to it before feeding it to GPT 3.5. I chose 3.5 because I’ve seen a number of assignments that encourage students to use the free version of ChatGPT rather than paying for GPT4. My test passage includes a misplaced semi-colon, repetition, and a particularly awkward passive voice sentence:
Soon after ChatGPT was released; an artificial intelligence researcher from one of the big tech companies told me that I should not worry nor should I be concerned about how the technology would affect how students learn to write. In two years, she assured me, only aspiring professional writers would enroll in writing classes. The writing by others would no longer be necessary.
For my first test, I asked ChatGPT to edit the passage without prompting it to look for specific problems. I didn’t tell it what audience I was writing for, and I didn’t tell it what errors to look for. Not surprisingly, I got back an edit that looks like what you’d get if you pasted in some text and asked the chatbot to revise it to sound “smart.”
ChatGPT did catch the semi-colon, but also added “according to her” and changed “in two years” to “within a mere two years.” These are not edits I would make, and they are not edits that my editor at the Los Angeles Times made in the original piece. Students could learn something from talking through these edits, but they would need human guidance to understand what to do with them.
You can get different edits from ChatGPT if you know what to ask for—and if you know how to ask. I put the same passage in and asked the chatbot to edit for passive voice. I was not clear enough in that prompt (I was curious to see what it would do), so it put the whole passage into passive voice.
That can be fixed! I tried again, and this time I specifically told the chatbot to “eliminate” passive voice. This round, the passive voice was eliminated. But the chatbot also changed “told me” to “reassured me,” while leaving the awkward repetition of “worry or be concerned about.”
What if you ask the bot to edit for specific problems that you want to make sure you avoid, but also to edit for other problems? In the next round, I asked the chatbot to eliminate the passive voice and edit for “other writing problems.” The semi-colon was fixed, and this time, the repetition was also fixed. Mysteriously, though, “assured” became “confidently stated.” Will students doing this type of assignment accept that the chatbot is correct even when the edit is random—or actually bad?
Students could ask ChatGPT why it made those particular edits, and some instructors are designing assignments that encourage or require this step. In my example, students would learn from the chatbot that passive voice was eliminated (which is what we told the bot to do). They would also learn that repetition was eliminated, which is a helpful edit. The other explanations are vague. Sentences were rephrased for “clarity” and “flow.”
Is the problem here that we used GPT 3.5 instead of GPT4, as someone suggested when I posted a thread about my experiment to social media? I tried the same prompt with GPT4, and this time, the repetition was not corrected.
This was just one experiment, and I was only testing one type of assignment—one that encourages students to try out ChatGPT without offering guidance or explicit learning goals. Other instructors are using generative AI in their courses in more scaffolded ways, and the results will, no doubt, be quite different. Nevertheless, my experiment reinforced two of my concerns about teaching with generative AI. First, before asking our students to bring automated language models into the writing process, we need to know what kinds of output they are likely to get—and what they can or will do with it. Second, before designing writing assignments that ask students to use generative AI, we should have a clear sense of what we want them to learn from those exercises. A successful assignment will look different when the goal is to teach students how to think critically about generative AI than it will when the goal is to teach students how to write clearly and eloquently.
It is crucial that we teach our students to think critically about generative AI–and asking them to engage with AI tools in different contexts across different disciplines will be an important part of that process. But rather than simply asking students to turn to the chatbot for “feedback” or for any other step of the writing process, we should be helping our students understand how LLMs are trained, what types of data they are trained on, what we don’t know about that data, and how bias is baked into these systems. We should be talking with our students about how these systems generate output and encouraging them to think about what it means to rely on predictive text generators in different contexts—and what the effects of widespread usage will be on the environment and society. In the context of writing assignments, we should be encouraging our students to explore the connection between writing and thinking, what it means to have their own ideas, and what skills they will need to have in order to critically engage with AI output as “feedback” on something they’ve written or as writing they encounter in other contexts.*
I asked ChatGPT one final question as part of my experiment.
*For resources on how to develop this teaching see, for example, “TEACHING CRITICAL AI LITERACY: Advice for the New Semester.”