So I finally tried ChatGPT. And this is how it went.
I went through the hassle of setting up a new email address and played with the free version of OpenAI. I wanted it to get to do something it’s supposed to be good at. To digest what I had learned so far about large language models.
My biggest concerns are data privacy, bias, and hallucination. Large language models aren’t secure, aren’t neutral, and make stuff up. So unless it doesn’t matter whether what it says is true, you need to know where to check what ChatGPT tells you. And it will reinforce the biases in the data it has got (Sway is brilliant at explaining this). If you read one article about the issues involved, I’d recommend the UK Society of Authors advice to writers. If you want to use it in the drafting stage or to process information, other people can help you get started (the Thesis Whisperer and Nature for academics, the CIEP and APA for copy editors, and the ATA for translators). The model is only as good as the data it gets, and it doesn’t cite its sources. And it takes energy. The environmental impact is hard to calculate.
I knew all that without trying it for myself. But it was time to take the plunge.
At first, I was thrilled with how OpenAI handled a lot of information, fast. As soon as I asked it to summarize all the links and notes I’d collected about it, it could see how some didn’t relate to the question I was asking.
But the text it generated was pretty long too, and reading and regenerating it took real time. Its first output felt like a school essay. When I asked it to make it sound more like an academic article abstract, it just used higher-register words like “paramount,” and “imperative.” I was soon drowning in verbal diarrhoea. So I gave it word limits and got it to give me bullet points, which was better.
Then I asked it to rewrite its summary about itself like Chaucer, which was fun, but maybe less useful (tacking a letter “e” on the end of words doesn’t make it olde Englyshe). Regenerating the output made it worse: the text flowed more poorly, it got increasingly doggerel-like, and information got lost.
After one hour, I made myself stop and log out. This thing can save time but it can also be a colossal waste of time. I won’t inflict the whole “Chaucer” draft on you, but here are 10 unedited lines:
Whan thaten writers ande linguists han yspoken,
Of large language models, here be reasons token,
Concernes they han, of privacie and such,
Editinge limitations, biases as much…
To generate ideas, when blocke befall,
With prompts ande suggestions, to answer the call…
Yet caution be advised, to relye not alone,
On these models vast, bute to make truth known,
Verify information, fact-checke with care,
Retain one’s voice, unique ande rare…
I think the real Chaucer deserves a word here. He says that if we expect to find some useful truth in our writing, we need to separate the wheat from the chaff:
For Seint Paul seith that al that writen is,
To oure doctrine it is ywrite, ywis;
Taketh the fruyt, and lat the chaf be stille.
(Canterbury Tales, The Nun’s Priest’s Tale, VII.3431 ff.)
And I found that quote by googling. When I asked ChatGPT to find me an image to go with this text, it told me to use a search engine (DALL.E is for another day). Without using ChatGPT, I summarized my ideas faster, and sounded more like me, in that same time. But if I need to process a lot of new information or find out how to use a term or tool, I’ll give it another whirl. I can see how it would be useful if you’re drafting something or entering uncharted territory.
Thank you to my colleagues in the ITI, NEaT, MET, SENSE, EASE, and Akavan erityisalat (the Finnish trade union for academic specialists, including linguists) for our conversations about using large language models over the last month or so. And thank you to the wonderful few of you who I spent a weekend retreat with, creating the space to write this. Half way through my draft, I stopped for lunch, yoga, and to talk about it. When I finish, we’ll heat the sauna. A chat tool can’t support my writing like that.
Organizing your thoughts takes more than feeding chunks of language into a model. Writing takes more than regenerating other people’s words without crediting them. I won’t stop learning about or using this tool, but I won’t stop doing a lot of other things that help me write.
Have you tried it? Is it useful? Does it help you write better? I’d like to know. We need each other to sift the wheat from the chaff.
Illustration from the Ellesmere Manuscript. Held by© MS EL 26 C 9, The Huntington Library, San Marino, California.
Ridge Writing Retreats 