Sounds like AI, measured

The complaint arrived in one line: our writing is typically AI, with em-dashes and typical AI structures. Every product in the studio has copy that agents wrote, so the complaint applied everywhere at once, and the obvious response (tell the agents to sound human) had already been tried on several surfaces and lasted about two paragraphs per draft. The problem with "sounds like AI" is that it is a verdict with no number in it. You cannot fail a draft on it, and you cannot tell whether a rewrite helped.

So the first move was to stop arguing about it and measure it. Two decisions made that possible. Pick a human control and score it with the same code you score yourself with. And express every tell as a rate per hundred sentences, so a 40-key dictionary and a 3,000-sentence corpus can be compared on one axis.

The control

The control was six Paul Graham essays, written between 2004 and 2021, unambiguously by a person. Run through the same meter they measure 1.1 em dashes per hundred sentences. The product's letters, prose libraries and site copy measured 24 to 30. Roughly one sentence in four carried a dash. That is about a twentyfold outlier, and the number is what converted the complaint into a defect: an em dash is legitimate punctuation, and one in every fourth sentence is a rate no human writer sustains by choice.

site-copy        29.5
prose-libs       27.4
blog-stories     26.5
email-letters    24.6
coach-prompts    21.8
complete-report  15.0
digest/weekly    14.4
campaign-drip    13.8
courses           6.0

control: 6 Paul Graham essays   1.1

The meter is unglamorous. It walks a list of surfaces (a label and a set of files each), pulls string literals out of TypeScript and values out of JSON, strips comments (this repo's comments carry more em dashes than its copy does), drops anything that looks like a class name or a URL, and splits the rest into sentences. Sentences are the denominator of every number it prints, so the filter that keeps `text-sm` out of the sentence count matters more than any pattern.

The biggest surface was invisible

The scorecard's first version listed `site-copy` as the English i18n dictionary for the localised site tree. The English front door reads a different dictionary, in a directory the meter never walked. So the homepage reported as 44 sentences when its own copy was 223 keys the meter had not seen, and a campaign-copy rewrite measured clean while the strings a signed-in reader actually gets were untouched. That surface, once added, was the largest in the repo: 34 files, 2,372 sentences, 21.2 em dashes per hundred.

A surface that is missing is worse than a surface that scores badly, because it scores zero and zero reads like good news. The rule we took from it: enumerate surfaces from the filesystem, not from memory, and put a structural test on the list (every declared surface resolves to files, every surface yields more than ten sentences) so a renamed directory fails loudly instead of vanishing from the report.

Lexical tells die; structural tells survive

The vocabulary everyone associates with generated prose was already gone. Across every English surface in the repo there was one "tapestry", one "testament", one "game-changer" and three "journey"s, all of them inside prompt text rather than copy a reader sees. A banned-word list had done that work months earlier and it held. Word lists are cheap to enforce, and the tell they describe ("delves" at 28 times its pre-LLM frequency, in the published excess-vocabulary studies) has mostly stopped being useful.

What remained was structure. The em dash was a house habit every surface shared. A second construction separated the surfaces far more cleanly: the trailing participial clause, a finished sentence with ", angling the bishop at the long diagonal" bolted on. It ran at 9.0 per hundred sentences in the opening guides and 8.0 in the opening stories, against 0.0 to 0.6 on every hand-written surface and near zero in the control. Those two corpora are exactly the two written wholesale by agent desks. The construction is fine English; the density is a fingerprint, precise enough to name which files a model wrote.

Then the first fix shipped, the letters came down to 0.9 em dashes per hundred against the control's 1.1, and the owner read the letters we send and still heard a machine. He quoted one back: "Maybe your move is fine. Maybe it's the leak." Nothing had looked for it, because nobody had written it down. A tell list is only ever the tells someone thought of. The second vintage came from reading real sent copy rather than imagining what a model might do: two consecutive sentences both opening on a hedge, the tricolon used as cadence (4.8 per hundred in the letters, 5.5 in the stories), "here's the thing" openers, a modal stacked on an evidential verb ("might suggest"), and the "not X. Y is what matters." seesaw that had evaded the original pattern by splitting itself across a full stop.

Budgets, not bans, plus a rhythm floor

Every one of those constructions is legitimate English a good writer uses occasionally. Banning them produces prose that is differently robotic. So the gate carries budgets per hundred sentences, chosen from the measured corpus: far enough above what a human reaches that good prose cannot trip them, far enough below what our copy measured that the gate bites. Two constructions get a budget of zero, because there is no dose of them that reads as craft.

BUDGETS = {
  emPer100Sentences:          5,    // control 1.1; product measured 24-30
  participialPer100Sentences: 2,    // agent-written corpora ran 8-9
  contrastivePer100Sentences: 3,    // "not X, it's Y" in its disguises
  minLengthCv:                0.35, // the rhythm floor (see below)
  smallSampleSentences:       40,   // below this, absolute caps, not rates
}

TELL_BUDGETS = {
  speculativeAnaphoraPer100Sentences: 0,  // "Maybe X. Maybe Y."
  signpostOpenersPer100Sentences:     0,  // "Here's the thing"
  tricolonsPer100Sentences:           4,  // one per 25 sentences
  anaphoraPairsPer100Sentences:       3,  // same two opening words, twice
  questionsPer100Sentences:           6,
  hedgeStacksPer100Sentences:         2,  // "might suggest"
  splitContrastivesPer100Sentences:   2,  // the seesaw across a full stop
  ratherThanPer100Sentences:          3,
  summarisingClosersPer100Sentences:  4,  // a last sentence with no fact in it
}

// plus LEXICAL_PATTERNS (delve, tapestry, journey, unlock, game-changer,
// the excess-vocabulary adjectives...) at a budget of zero

The rhythm floor is the one that survives a punctuation cleanup. Take every sentence's length in words, compute the standard deviation divided by the mean, and refuse anything under 0.35. A writer varies; a generator settles into a house length, and once the dashes are gone that sameness is what is left to hear. The complaint it emits is specific enough to act on: sentence lengths vary by only so many percent of their mean, every sentence is about N words, break two long ones and let one run short.

The same number wears two verdicts. On a single 40-sentence chapter the participial rate has almost no resolution, so there it is advisory; on a corpus of 2,600 sentences it blocks. The pattern, the cap and the stoplist are defined once and imported by both callers, so the number the scorecard prints and the number the gate enforces cannot come from two regexes.

Rewrite cheap, but lock the facts

Once a corpus has a measured rate and a budget, the rewrite is an edit job, and an edit job can go to a small model. What makes that safe is that the corpora were fact-checked prose: numbers from a live position API, lines verified move by move. A re-registering pass must prove it invented nothing, and that check is mechanical.

factLock(original, candidate):
  for each extractor in [digits, SAN moves, capitalised runs]:
    have  = set(extractor(original))
    added = extractor(candidate) - have
    if added: complain("<kind> in the rewrite that are not in the source")

// direction is deliberate: the rewrite may DROP a repeated "52%";
// it may never INTRODUCE a number, move or name, because an introduced
// number is a number nobody measured.

About 1.8M tokens of a small model went through that lock across the rewrite and introduced zero invented facts. The lock has a known false positive, kept on purpose: it cannot tell a proper noun from a capitalised sentence opener, so splitting a sentence at a new word reports an invented name. The cost falls on the writer, who splits somewhere else. The alternative is a check loose enough to let a swapped player name through, and on fact-checked prose a rejected rewrite is the cheaper error.

The loop is the write-then-verify shape we use for every generated artefact: score the draft, quote each offending sentence in the complaint, hand the complaints back as the correction turn. "Avoid AI writing" produces two good paragraphs and a relapse. "These three sentences end in a participial clause: ..." produces three fixed sentences.

Ship the gate, not the rewrite

The rewrite is the gate's first customer. The deliverable is a command that any generator runs on its own output before the file is written, with exit 0 as the finish line, because the corpus will be regenerated again and without the gate the next generation drifts straight back to the house voice.

npx tsx scripts/gate-corpus.mts --against origin/main data/stories/<key>.json

file                       sents  em   em/100s  partic  part/100s
data/stories/<key>.json    <n>    <em>  <rate>   <p>     <rate>

1/1 within budget.                         # exit 0

# non-zero exit prints the correction brief:
REJECTED data/stories/<other>.json
The draft was rejected. Fix exactly these and change nothing else:
1. [participial-tail] 4 trailing participial clauses across 58 sentences
   (budget 1). Split each into two sentences, or join with "and".
   offending text: ...

One wrinkle in shipping a gate over an existing corpus: a rule invented today cannot retroactively fail files generated before it existed. The moment the tricolon budget switched on, more than half of the 95 story files exceeded it. Failing them would mean either a regeneration bill nobody had authorised or a budget quietly loosened until the corpus passed, and the second is worse because it makes the rule meaningless. So the second-vintage tells block on new letters at compose time and land as advisories on the old corpus, and the test prints that debt on every run so it stays visible until the next regeneration clears it.

Re-measure after

                 em dash        participial
blog-stories     23.8 -> 0.7    6.2 -> 0.7
courses           6.0 -> 0.2    6.8 -> 0.3
app-copy         21.2 -> 1.3    (502 dashes -> 33, contrastive 1.8 -> 0.4)
site-copy        29.5 -> 2.1    (by hand)
coach-prompts    21.8 -> 1.1    (by hand)
campaign-drip    13.8 -> 1.4    (by hand)
email-letters         -> 0.9    control 1.1

The 33 surviving dashes in the app copy are deliberate: chart and title separators, byline attribution, two prose dashes kept because the beat is the point. The account screen had forty confirmations all shaped "Saved - back on."; they did not all become "Saved. X." either, because swapping one uniformity for another is the same tell.

Then re-measure the translations, because a model asked to preserve style reproduces the rate it sees and rounds up. Every changed key was re-translated, and the dash habit had been mirrored into the translations and then some: 18 keys carried a dash in a translation where the English never had one (15 German, 2 French, 1 Spanish).

What we got wrong

• The meter missed the biggest surface, above. It also asserted real match counts against live copy as its own sanity check, so cleaning the copy made the guard fail on success. Pattern sanity now runs on a synthetic fixture built to contain every tell, one that never gets fixed; the live surfaces carry structural checks only.
• The contrastive regex required a literal "not", so the homepage hero's "isn't on the board, it's here" ran the construction and scored 0.0. A scorecard of zeroes must never be able to read as good news; every pattern now has a positive specimen in the fixture.
• This blog's own de-slop pass, a month earlier, had a counted list too: em dashes 65 to 2, hedges 29 to 3, first-person singular 23 to 0, and the set came out 16% shorter with nothing added to fill the gaps. The tell was aphoristic closers, nearly every paragraph landing on a balanced quotable line. Round one promoted them into standalone one-sentence paragraphs, which was worse: extracting a tic into its own line emphasises it. Round two deleted them and let paragraphs end on their last real piece of information. The summarising-closer counter in the gate is that lesson as code, flagged only when a last sentence both opens on a summarising move and carries no number, move or link.

What the number does and does not say

A rate per hundred sentences measures a habit. It does not measure whether the sentence was worth writing. A letter can break every tell rule zero times and still be the same letter as last week with new numbers in it, and that turned out to be the complaint the owner actually had; it needed a different instrument (opener reuse and n-gram overlap after the reader's own facts are masked out) and is its own post. The gate also only speaks for the surfaces on its list, which is why the list is tested and why the scorecard is read before anyone claims a surface is clean.

What the loop does establish is narrower than taste. A style directive with no generator, no measurement and no gate is a preference, and before this there was no file anyone could name as holding the voice. Now there is one: the budgets live in a constant, the product's copy sits within a factor of two of a human control on the constructions we know how to count, a draft that drifts back fails a command with a non-zero exit, and the rewrite that got it there changed no number, move or name.