Tiny prose alive not coherent
2026-09-04, v11/scripts/chat_audit.py over all of chat_logs/. Scored the
18 checkpoints she rated by ear that day (8 keep, 8 cut with enough replies to
score) against six metrics. Result, as AUC = P(kept model ranks above cut):
| metric | kept | cut | AUC |
|---|---|---|---|
| self-contained | 68.7% | 81.3% | 0.20 |
| responsive (shares a content word with the cue) | 23.2% | 33.8% | 0.20 |
| asks back | 15.2% | 20.2% | 0.41 |
| near-dup looping | 0.0% | 2.1% | 0.38 |
| median words | 18.2 | 11.2 | 0.63 |
Both separating metrics run BACKWARDS. The mechanism is visible by eye.g2long (CUT, 88.6% self-contained) loops: "I shall have to go to a hotel
to-morrow" / "I shall go to the hotel to-morrow"; "But you'll be sorry" /
"I don't see why you should be sorry". g4ask (KEPT, 59.1%) misidentifies who
it is talking to — Mr. Dunn, Cayley, Mr. Lipscombe — but is alive: "The deuce
you are. I was thinking of nothing but the moon."
The only way to never name someone offstage is to say nothing specific, and the
only way to score high on cue-responsiveness is to parrot the cue. Both metrics
reward emptiness.
Caveat: 8 vs 8, z ~ 2.0, seven metrics tested — statistically marginal. The
qualitative contrast is the strong evidence, not the AUC.
Why: this retires the self-containedness filter (9,895 "fully clean"
conversations, sized 2026-08-30, never built) — it would have selected forg2long. It also corrects the reading of her own words: she said "as long as
her replies mostly make sense", and I translated that into "no offstage
referents", which is not what she meant. The axis she judges on is ALIVE vs
DEAD, not coherent vs confused.
How to apply: do not filter or score on proper nouns, offstage referents, or
cue overlap. When proposing a metric, check it against the keep/cut set inchat_audit.py first — a metric that cannot separate her verdict is not
measuring what she hears. Supersedes the remedy in
tiny-prose-register-is-the-defect. See also
tiny-prose-voice-concentration-flat, feedback-smaller-steps.