EN PT ID

AI Detection False Positives 2026: What the Data Shows

November 17, 2026 · 11 min read · Industry Analysis

On 29 July 2026, Jerry Falade's debut crime novel disappeared. Roughly five weeks earlier, a two-book deal with Macmillan's Minotaur Books had been won in a fourteen-way auction, reported at $2.4 million. What ended it was not a finding that a machine wrote the book. It was that his agents at Europa Content could no longer substantiate what they had been told.

The agency's stated reason was procedural rather than moral: it had vetted the book with Falade for AI use during submission, and it could no longer verify that account. Nobody said the book was machine-written. The manuscript did not fail a test. It failed to be provable.

And nobody who owed Falade anything ran a detector. The scores that circulated came from third parties with no duty to him, run on an unpublished manuscript, tallying stylistic tics — 18 "not X, but Y" constructions and 46 em dashes, by his account. The people with obligations to him ran nothing.

For anyone publishing on X, Bluesky or LinkedIn, this is not only a publishing story. It is a description of what happens when a machine-generated accusation lands on a human's public output and there is no evidence trail to answer it with. In 2026 the accusation is cheap, fast, and anonymous. The archive is the only thing that answers it.

What Actually Happened, Precisely

Four cases from the past year make the pattern visible, because each one turned on a different question: did a detector run, who ran it, and did anyone with a duty to the author stand behind them.

CaseWhat happenedDetectorHow it was resolved
Falade — Minotaur / Europa ContentTwo-book deal worth $2.4M withdrawn 29 July 2026None run by the agency; a third-party report was citedNot resolved. Withdrawn as unprovable, not as guilty
Nazir — Commonwealth Short Story PrizeWon overall from 7,806 entries in July 2026Pangram returned 100% machine-generatedThe Foundation reviewed drafts, notes and timestamps and concluded AI was not used
Ballard — HachetteUS edition cancelled and UK printing withdrawn in March 2026A public video analysis, viewed more than a million timesNot resolved. First withdrawal of its kind by a major house
Wolfe — Simon & SchusterBookTok bestseller stayed on sale after a similar claimPangram measured roughly 60% AI-generated or AI-assisted, per The AtlanticThe publisher stood behind the author and the book

Read that table row by row and the pattern is not "detectors are accurate". It is: a detector score did not decide any of these cases. In two, nobody with an obligation ran a detector at all. In one, the detector result was investigated and set aside. In the fourth, the outcome turned on whether the people who owed the author chose to stand behind them.

The Falade case is the sharpest illustration, because his own position was technically strong. He had time-stamped drafts, written notes, dated WhatsApp messages documenting the book's development, and a document version history that shows the manuscript evolving. He assembled that file without knowing he would need it. No institution examined it.

Three Numbers That Should Change How You Read a Detector Score

Detector marketing treats accuracy as a single number. It is not. Every accuracy figure hides a base rate, and the base rate in the wild is what decides whether an accusation is fair.

MeasurementSourceNumberWhat it means in practice
Human reader versus machine classifier, same taskSuvanto et al., ICAART 202649.9% human accuracy against 98.14% for the best classifier"I can just tell" is not a detection method
False positives on non-native English writingLiang et al., Patterns 202361.22% average false positive rate across 7 detectors and 91 human TOEFL essaysPlain vocabularies get charged more
Selling books versus revenue, 2023–2026Chakrabarty et al., 202619.2-fold growth in books with observed sales against 8.9-fold growth in revenueVolume is not quality, and dilution is measurable

The humans were at chance

The DET12 study, published at ICAART in January 2026, is the most useful controlled test of detector-versus-human judgement available. The corpus was built from twelve public-domain detective novels — six by Agatha Christie, six by Dorothy L. Sayers — cut into 8,068 short human excerpts, then rewritten by GPT-4.1 into 8,068 machine counterparts. One hundred and nineteen evaluators were each shown five human-or-machine pairs with ninety seconds and no internet access. Of 595 judgements, 297 were correct: 49.9% accuracy, indistinguishable from a coin flip.

On the same data, a naive Bayes classifier reached 97.93%, a random forest 93.10%, and the authors' own interpretable linear model 98.14%. The gap is the whole crisis in one line: the person whose professional identity is built on reading closely performed at chance, and a bag-of-words model from the 1960s performed at ninety-eight per cent.

Earlier, at the scale of an academic institution, the same arithmetic ran on students. Turnitin launched its detector in April 2023 and processed more than 38 million submissions within four months, initially claiming roughly a 1% document-level false positive rate and later acknowledging something closer to 4% at sentence level. Vanderbilt University did the multiplication that publishing has not: with about 75,000 papers submitted in 2022, a 1% false positive rate implies roughly 750 students wrongly implicated in a single year. Vanderbilt disabled the detector in August 2023.

The bias is measurable, not hypothetical

The mechanism behind the Vanderbilt number was identified in a Stanford paper published in Patterns in July 2023. Seven widely used detectors were tested on 91 human-authored TOEFL essays and 88 US eighth-grade essays. The detectors were near-perfect on the eighth-graders. On the TOEFL essays they misclassified more than half the writing as machine-generated, for an average false positive rate of 61.22%.

The reason is not mysterious. These systems key on perplexity — how statistically surprising a text's word choices are. A writer with a constrained lexical range produces low-perplexity prose. So does a language model. The detector cannot tell a machine from a person with fewer words available, and it resolves the ambiguity against the person.

That is a direct, testable description of who gets charged. It is the same axis that showed up in the Falade case, where a Nigerian doctoral candidate writing crime fiction in his second language had his unpublished manuscript scored on its em dashes by someone outside the transaction.

The Fine-Tuning Problem: About $81 Buys Away Every Tell

There is a reason the previous generation of detectors looked reliable, and it is not that they were. It is that they were fitted to the cheapest, most careless style of machine writing — and that style is disappearing.

A preregistered study by Chakrabarty, Ginsburg and Dhillon set MFA-trained writers against ChatGPT, Claude and Gemini, each producing excerpts of up to 450 words emulating fifty award-winning authors. A panel of 159 readers — 28 MFA candidates and 131 general readers — assessed the results blind across 3,840 pairwise comparisons. With ordinary prompting, experts were emphatic: they disfavoured machine text on stylistic fidelity at an odds ratio of 0.16 and on quality at 0.13.

Then the researchers fine-tuned GPT-4o on each author's complete works, and every finding inverted. Expert readers now favoured the machine on fidelity at an odds ratio of 8.16 and on quality at 1.87. General readers shifted further still, to 16.65 and 5.42.

The detection numbers inverted with them. Human writing was never misclassified at the study's 0.9 threshold. In-context machine output was flagged 97% of the time by Pangram and 91% by GPTZero. Fine-tuned machine output was flagged 3% of the time by Pangram and not at all by GPTZero.

The reason is stylistic, and it is specific. Cliché density correlated with the detector score at a Pearson coefficient of 0.60 and mediated 16.4% of the effect of detectability on reader preference. After fine-tuning, that mediation collapsed to a statistically insignificant 1.3% — evidence that fine-tuning eliminates the stylistic signature rather than masking it.

And the price of doing so is trivial. Model fine-tuning and inference cost between $22.25 and $272.50 per author, with a median of $77.88, plus about $3 of inference to generate 100,000 words. That is roughly 0.3% of the $25,000 a professional writer would charge for the same manuscript.

Every tell an editor has learned to recognise — clichés, elaborate synonyms, flattened idiom, Americanisms, absent subtext — is an artefact of prompting a general model. None of them is a property of machine origin. Roughly eighty dollars removes them. The industry has built its defences against the version of the problem that is about to stop existing, while the people who keep tripping those defences are the ones whose prose happens to resemble cheap machine output.

What Detectors Actually Key On

The DET12 authors built an interpretable classifier specifically so they could open it up. What they found is worth reading carefully, because it explains both the accuracy and the failure mode.

The dominant signal was lexical variety. Where Christie or Sayers wrote "said", the rewrite reached for "remarked", "responded" or "mentioned"; "cried" became "exclaimed". Measured information-theoretically, the machine text carried 9.164 bits of unigram entropy against 8.875 for the human originals — roughly 22% greater variation in word choice. The machine was not writing worse. It was writing with a bigger thesaurus and less restraint, and that alone gives it away.

Four further categories appeared in the hand annotation:

The uncomfortable corollary is that some of the classifier's power is an artefact of one class being written by British novelists a century ago and the other by an American model last year. The classifier has learned, among other things, that British spelling and dated idiom mean human.

Read that in the other direction and you get the sentence that matters most for anyone who publishes for a living: a detector trained on cheap machine prose will systematically penalise a writer whose English is correct but plain. The suspicion will track register and accent at least as much as it tracks machine use.

A Detection-Dispute Workflow for Creators

You do not need to become a forensics expert to survive an accusation. You need a record that was built before the accusation existed. Five steps.

1. Do not amplify

A score travelling through your replies is now part of the record in a way you did not choose. Do not retweet the accusation, do not argue style in the thread, and do not delete anything you would want to point at later. Answer the accusation from your own archive, once, in one place.

2. Freeze the original

Capture the post as it stood, before any edit, before any deletion. Hash the file. The point is not paranoia — it is that the single most common failure in these cases is that the author changed the text after the accusation and thereby destroyed the version that could answer it.

3. Separate the source from the accusation

Two files, always. One holds what was published. The other holds what was said about it. Never let an annotation about a detector score live inside the captured post, because the moment it does, the archive is no longer a copy of anything.

4. Log the detector run as metadata

Record the tool name, the date, the threshold, the number of characters sampled and the score. Without those five fields, a detector result is not reproducible and is worth nothing — to you and to anyone reviewing it later.

5. Answer with timestamps, not adjectives

"This is not AI slop, it is me" is a claim about authorship that you cannot support. "Here is the draft from 14 March, here is the version history, here is the capture from the morning it posted" is a claim about process, and process is what the Commonwealth Foundation used, what authors with disclosure contracts use, and what the Authors Guild's Human Authored mark is built on.

# 1. Capture the live post page exactly as a reader saw it
curl -sL -A "Mozilla/5.0" "https://x.com/HANDLE/status/POST_ID" \
  -o archive/x/HANDLE/POST_ID/page.html

# 2. Record the capture facts in an immutable sidecar
mkdir -p archive/x/HANDLE/POST_ID
printf '%s\n' \
  "source: https://x.com/HANDLE/status/POST_ID" \
  "platform: x" \
  "captured_at: 2026-11-17T08:31:12Z" \
  "sha256: PAGE_SHA256" \
  > archive/x/HANDLE/POST_ID/provenance.yml
# 3. Re-fetch later and compare against the stored copy
curl -sL -A "Mozilla/5.0" "https://x.com/HANDLE/status/POST_ID" \
  -o archive/x/HANDLE/POST_ID/page-later.html
diff -q archive/x/HANDLE/POST_ID/page.html \
        archive/x/HANDLE/POST_ID/page-later.html \
  && echo "UNCHANGED"

# 4. Keep any detector run as an annotation, never inside the source text
printf '%s\n' \
  "tool: TOOL_NAME" \
  "run_at: 2026-11-17T09:02:00Z" \
  "threshold: 0.90" \
  "sample_chars: 2800" \
  "score: 0.97" \
  "note: triage signal only, not an authorship finding" \
  > archive/x/HANDLE/POST_ID/detector-run.yml

Two more things worth having in the same folder. The re-fetch comparison in step 2 is what lets you show that a post was edited after an accusation, or that it was not edited at all. And a clean JSON copy of the public profile is a cheap, dated artefact of what your account looked like at the time of the dispute.

# 5. ThreadGrab's public endpoint returns clean JSON you can archive
curl -s "https://threadgrab.com/api/profile/HANDLE" \
  -H "Accept: application/json" \
  | python3 -c "import sys,json; d=json.load(sys.stdin); \
print(d.get('username'), d.get('follower_count'))"

None of this requires you to become suspicious of your own writing. It requires the opposite: a durable record of your process, kept cheaply and routinely, so that a dispute is a lookup rather than a reconstruction.

Which Evidence Actually Holds Up

If you are asked to defend a post, or if you are the one making the accusation, this is the evidence hierarchy that the 2026 cases actually followed.

EvidenceWeightWhy
Your own capture timestamps and file hashesStrongIndependent of any detector; proves what existed and when
Dated drafts, notes and version historyStrongThe Commonwealth Foundation's method — process, not output
Publisher or agent standing behind the authorDecisive in practiceKept the Wolfe book on sale; the Falade deal died without it
Third-party detector score on unpublished textWeakNo consent, no duty, no reproducibility
A platform report buttonSignal onlyHuman-driven and unaudited; a tally, not a verdict
An editor's confident intuitionVery weak49.9% accuracy in the only controlled test

Note the ordering. The strongest items are all procedural and all under your control. The weakest item — the detector score — is the one that arrives first and attracts the most attention.

The Four Fields to Archive

If you adopt one practice from this article, make it this. Every captured post gets four fields, plus one annotation file that may or may not exist.

That is the same four-layer structure that a provenance-first social archive uses, and it is the direct answer to the failure that killed Falade's deal. His agents could not substantiate what they had been told because the chain of custody had no verifiable record attached to it. The record is not hard to keep. It is just not kept by default.

The wider industry is converging on the same idea from the policy side. The Authors Guild's Human Authored certification, expanded in March 2026 to all authors publishing in the United States and the United Kingdom with the Society of Authors, requires third-party identity verification, a signed licence and a $10 per-title fee — and does not analyse the manuscript at all. South Korea went further and built a disclosure taxonomy: on 29 September 2026 the Korean Publishers Association and the Writers Association of Korea announced voluntary three-level principles, from "Human Authored Work" through "AI-Assisted Work" to "AI-Generated Publication", with the author declaring their level on submission.

Both systems rest on the author's word rather than on a classifier, and both are widely regarded as weak. The weakness is instructive: a disclosure regime is only as good as the archive that supports it.

FAQ

Does a detector score prove that a post was written by AI?

No. It is a probabilistic signal produced by a classifier, and its error characteristics depend on the base rate of machine text in whatever population you tested. At a 1% prevalence, a classifier that is 98% accurate still produces a flagged pile where roughly two thirds of the accused are innocent. Present it as triage, never as proof.

Can I safely run a detector on my own archive?

Yes, as long as you record the tool name, the run date, the threshold, the number of characters sampled and the score. Without those five fields the result cannot be reproduced or reviewed later. Keep the result in a separate annotation file so that your capture stays a faithful copy of what was published.

What should I do if a detector flags one of my posts?

Treat it as a signal to check your own process, not a verdict on your authorship. Do not amplify the accusation, do not edit the post, do not argue style in the thread. Pull your dated drafts and your capture record and publish them once, in one place, on your own terms.

Are AI detectors biased against non-native English writers?

Yes, and the effect is large and measured. A Stanford study testing seven detectors against 91 human TOEFL essays recorded an average false positive rate of 61.22%, while the same detectors were near-perfect on US eighth-grade essays. The cause is perplexity: a constrained vocabulary looks statistically like machine output, and the detector resolves the ambiguity against the writer.

Does an AI-content label solve the false-positive problem?

No, because a label and a detector are different instruments with different evidentiary weight. A platform label is an attribution claim made by the platform's own classifier; a detector score is a measurement anyone can run on any text. Both are fallible, and neither substitutes for a dated process record.

Will archiving my posts actually protect me?

Yes, and only in a specific way: it lets you answer a question about process with process evidence. A dated capture, a file hash and a version history show what existed and when, which is exactly what the Commonwealth Foundation used when it investigated the Nazir case. Without them, a dispute about authorship cannot be resolved at all, which is how a $2.4M deal ended in five weeks.

Bottom Line

The 2026 evidence does not support using detector scores to decide whether a named person wrote something. The humans who read for a living were at chance on the only controlled test. The classifiers that beat them were fitted to the cheapest machine prose, and roughly eighty dollars of fine-tuning removes that prose. The tools that measured false positives at 61% on non-native English and 750 wrongly accused students at one university were measuring something real about how the instruments work.

What survived every one of those cases was procedural. Drafts with dates. Timestamps on captures. A version history. A publisher who chose to stand behind the author. The people who lost in 2026 did not lose because they were caught — they lost because the absence of a record was itself treated as evidence.

So keep the cheap defence ready before you need it. Capture the post, hash the file, write four fields to a sidecar, and move on. It costs one command and about thirty seconds, and it converts the worst day of your publishing life from a matter of your word against a machine into a lookup.

Archive the post, not the accusation

ThreadGrab saves public X threads, Bluesky posts and LinkedIn articles as clean Markdown and JSON, with the capture time attached. When a detector score arrives, the archive is what you answer with.

Try ThreadGrab →

Primary sources