Tiny prose base is male
2026-09-04. Benched the bare anneal checkpoint (anneal_e_bl_mixed, the
base EVERY SFT variant was initialised from) against Claude Sonnet on the same
openers as the SFT models. It holds a conversation, has more personality than
the SFT models, and is unmistakably male:
Confound it, why can't you leave me alone? Pardon me, sit down and drink a glass of wine with me. Capital sports, aren't they? / I say, your sister's awfully pretty
Mechanism, and it is by design, documented in the code.split_exchanges.py: pool_anneal.jsonl = "no known female speaker" —
every exchange containing a woman is routed away to the SFT pools.build_anneal_variants.py: "Not the gender tilt... the anneal pool stays as it
is and every female exchange is left for SFT." Measured: pool_anneal 0%
female / 31% male, book_pool_anneal 0% female / 21.5% male.
So the base spends 7.37B tokens learning to talk from a corpus with every
identifiable woman removed, and SFT then gets ~250 iterations over ~1M tokens
to make him a woman. Measured the same day: reply length, register and
verbosity are identical between base and SFT (median 18w vs 16w, mean 22w
both, >40w 13% both). SFT is not teaching turn shape. It supplies gender.
Why: this explains the whole month at once. Every flat lever — DPO, SFT
iters, corpus slices, gold weight, grading rounds, Warner voice concentration —
was rearranging a thin layer on top of a base whose character was already set.
It also explains g4voc/g4voc2 getting opposite verdicts on a 0.05% data
difference: at that layer there is little real signal to separate.
The two-bar design's insight ("voice comes from a female SPEAKER, not a female
author") is sound. The budget is what failed: 8,108 female exchanges at SFT
cannot outvote 7.37B tokens of male-voiced base.
How to apply: the untried structural experiment is a female-voiced
anneal — anneal ON pool_sft_candidates.jsonl / pool_sft_top.jsonl instead
of holding them out. An anneal is 1,500 steps, not a pretrain. Prefer that over
any further SFT-side change. See tiny-prose-voice-concentration-flat,
tiny-prose-volume-is-not-the-bottleneck, tiny-prose-corpus-design.