Can you teach yourself to detect AI writing? Maybe

Celeste Rodriguez Louro, University of Western Australia

AI is quietly reshaping writing. It’s not just that prose is changing — the very structure of written language is genuinely changing too.

It’s not just a new technology.

It’s not a fad.

It’s an undeniable revolution.

Did this article’s introduction seem AI generated? What made you feel that way?

The truth is, it was not AI generated; I wrote it to emulate synthetic prose, the kind you get when a large language model (LLM) organises words into text.

That kind of prose suddenly seems to be everywhere: on low-quality websites built to accumulate advertising clicks, in social media captions chasing engagement, and even book manuscripts. Crime novelist Jerry Falade recently lost a book deal worth more than $2 million after his own literary agents said they could no longer faithfully determine how his manuscript had come together. Hachette pulled the horror novel Shy Girl in March this year under similar suspicion.

Before generative AI, we could generally assume that written text had been composed by a human. This is no longer the case. So how can we spot AI-written text?

There is no single giveaway. Instead, there are patterns: particular words, sentence structure and expressive habits that appear repeatedly in AI-generated prose. LLMs are also updated rapidly and efficiently so what gives away synthetic text today may no longer do a few months from now.

Quiet, quietly and genuinely

The word quietly is a significant giveaway. Search interest in the word has climbed steadily on Google Trends since December 2021, though this measures searches rather than appearances in writing, so it is suggestive rather than proof.

What did change over that period is how many people began writing with the help of LLMs, and these models do seem to reach for quiet and quietly more than the average human writer.

Genuinely is a similar story, and a more anecdotal one. I notice it overused in AI output, Claude included, more than I would expect from a person writing casually. No one has run a rigorous study on that specific word yet.

The documented version of this pattern is vocabulary that has spiked in scientific writing since 2022: delve, meticulous, underscore, boast and intricate all appear far more often in scanned PubMed abstracts than before, in a pattern researchers have tied directly to ChatGPT-style phrasing.

None of these words is proof on its own. But when words like quietly, genuinely, delve and meticulous appear alongside the same polished sentence framing, it starts sounding like a shared house style among LLMs, rather than an individual voice.

A house style of its own

The “not X, but Y” construction is another part of that house style, alongside a fondness for the rule of three.

It also favours taking a modest claim and then immediately escalating it. For example: “This article presents a useful perspective on language. It fundamentally changes how we think about what it means to write”. Human writers have always used it, but generative AI produces the pattern with remarkable consistency.

Then there’s the em dash, the earliest and most mocked AI tell. It never held up particularly well. One writer ran the same prompt through six chatbots and got eight em dashes from ChatGPT in 573 words, but none from Gemini or Meta AI.

This reaffirms the lesson that, while individual clues are unreliable, their accumulation matters.

Read more: Slanguage: Why AI’s stylistic negation — ‘it’s not X, it’s Y’ — is both annoying and doesn’t work

Lengthy text

Another clue to generative AI use is text length.

A reporter for The Atlantic describes how, after a driver crashed into her in Johannesburg, his frantic and incoherent behaviour at the scene gave way to a lengthy text written in polished prose only half an hour later. When she later contacted a mechanic whose texts had previously been filled with shorthand, his reply came back in the same distinctive AI voice.

Human language is shaped by the pressures of effort and time. Generative AI removes much of that cost. Realistically, a person might text: “Sorry, running late. Traffic is awful. Be there in 20”.

AI can swiftly turn that into a paragraph explaining the unexpected traffic congestion, expressing sincere regret and thanking the recipient for their patience. Nothing in that longer version is necessarily wrong. It’s simply doing far more linguistic work than the situation generally requires when time is of the essence.

The dangers of AI detection

Some people believe AI detection software can identify AI-generated text. However, these systems are far from perfect and can worsen existing biases.

A 2023 Stanford study found that seven popular AI detectors falsely identified an average of 61% of essays written by non-native English speakers as AI generated, with one tool flagging 97% of them.

AI detection has evolved since then, and more recent research presents a more complicated picture: current detectors can perform better in some settings, yet remain vulnerable to evasion and still produce false positives.

At the same time, the question is shifting beyond detection altogether: the European Union is now demanding that all AI-generated content is labelled or watermarked.

Are LLMs changing the way we write?

A new study in Nature Human Behaviour, analysing more than 880,000 Reddit posts, news articles and academic pieces, found that the spread of LLMs was associated with less variation in writing style.

Knowing who wrote something matters because language lands differently depending on who it comes from. Is the person trustworthy? Knowledgeable? Funny? Younger? Are they real?

The other reason is dexterity. Humans are excellent at problem solving and creativity, and those abilities appear in the peculiarities of individual language: an unexpected word, a unique comparison, a sentence structure that may reappear across writers but not en masse, as is the case with synthetic text.

If the prose across our media begins to sound as though it has been cut from the same automatic, uncreative, underwhelming and repetitive mould, we lose individuality, nuance, and our trust in the authenticity of the written text and its writer.

This article is republished from The Conversation under a Creative Commons license. Read the original article.