Tiny prose corpus design

Decided 2026-08-17. Full write-up in prose/notes/corpus-design.md.

Pretrain is a language, SFT is a voice.

• Pretraining installs 18th/19th c. English narrative prose. It does not

install voice at 243M params — volume washes it out. So filtering pretrain
data by author sex costs ~half the available tokens to buy a property
pretraining cannot hold. Rounds 1–2 conflated the two jobs; that's what
capped the corpus at 198 authors.

• The female voice lives in SFT only, and the real criterion is **speaker sex,

not author sex**. A woman's line written by Dickens counts; a man's line
written by Gaskell doesn't. This expands the SFT pool (female speakers in
the male canon, and in the male-authored BL shelf) while tightening the test.

• Register band for pretrain is ~1700–1900 English narrative prose. 18th c. is

in deliberately — Fielding, Richardson, Radcliffe et al. — because 18th/19th
c. English are close, and there are few enough 18th c. novels that they land
as single-digit % of tokens.

Two cleanliness bars. Pretrain: is it narrative prose and is the OCR
readable. SFT: near-perfect dialogue extraction, which is what all of
dialogue_parse exists for. Applying the SFT bar to pretrain data is what made
the BL shelf feel impossible.

Why it matters now: corpus_v10 is 95M tokens for a 243M model — ~50x under
the conservative Chinchilla point. Anchor collapse, register drift, and the
tiny-prose-dpo-round1 null result are all consistent with a starved model.
Volume is the priority and nothing gets traded for it.

Related: tiny-prose-goals-constraints, tiny-prose-side-project,
tiny-prose-corpus-retrieval