melissa kate
<< Little Jane

1. Build the anneal corpus (local, fast)

python prose/scripts/build_anneal_corpus.py

2. Round 8

python prose/cloud/train.py \
--corpus prose/data/corpus.txt \
--output-dir prose/models/model_v8 \
--n-layer 12 --n-head 12 --n-embd 768 \
--max-iters 60000 --eval-interval 500 \
--min-lr-frac 0.0 \
--anneal-data prose/data/anneal.encoded.npy

3. Round 10 (243.57M params, 94.97M tokens, ~31.5h on the M5 Max)

python prose/scripts/train_tokenizer.py # -> data/tokenizer_v10.model
python prose/scripts/build_anneal_corpus.py \
--tokenizer prose/data/tokenizer_v10.model \
--output prose/data/anneal_v10.encoded.npy

python prose/cloud/train.py \
--corpus prose/data/corpus.txt \
--tokenizer prose/data/tokenizer_v10.model \
--encoded-cache prose/data/corpus_v10.encoded.npy \
--output-dir prose/models/model_v10 \
--n-layer 18 --n-head 16 --n-embd 1024 --block-size 512 \
--max-iters 27500 --eval-interval 500 \
--min-lr-frac 0.0 \
--anneal-data prose/data/anneal_v10.encoded.npy

SFT

Build dataset with hand-graded data weighted higher.

NO --chat-unattr-weight 1 (that's the old habit): the anneal already trains

on chat_named + chat_unattributed, so those rows are duplicate signal at SFT

time. Dropping unattr takes the dataset 14,922 -> 6,424 rows and lifts graded

data from 10% to 27% of training TOKENS. Fewer rows, ~3x the new signal.

python prose/scripts/build_sft_dataset.py --min-female-frac 0.5

caffeinate python prose/cloud/sft.py \
--resume prose/models/model_v10/final.pt \
--data prose/data/sft_dataset.jsonl \
--output-dir prose/models/model_v10_sft7 \
--warmup-iters 25 --epochs 3 --eval-interval 20 --keep-all

DPO

python prose/cloud/dpo.py \
--resume prose/models/model_v10_sft8/best.pt \
--ref-logprobs prose/json/ref_logprobs.jsonl \
--output-dir prose/models/model_v10_dpo1

grade conversations - create new conversation sets for SFT

change the model based on the last best run > --ckpt prose/models/model_v8_sft5/best.pt

V10

3 turns

python prose/cloud/grade_conversations.py \
--ckpt prose/models/model_v10_sft4/best.pt \
--output prose/json/graded_conversations.jsonl \
--prompts shared/prompts/other_prompts.jsonl \
shared/prompts/followup_prompts.jsonl \
shared/prompts/victorian_greetings.jsonl \
--candidates 6 --repetition-penalty 1.15 --top-p 0.9 \
--web

flow

python prose/cloud/chat_continuous.py \
--ckpt prose/models/model_v10_sft7/best.pt \
--output prose/json/continuous_conversations.jsonl \
--prompts shared/prompts/conversation_starters_clean.jsonl \
shared/prompts/followup_prompts.jsonl \
shared/prompts/glue_pairs.jsonl \
shared/prompts/greetings.jsonl \
shared/prompts/modern_prompts.jsonl \
shared/prompts/other_prompts.jsonl \
shared/prompts/victorian_greetings.jsonl \
shared/prompts/victorian_knowledge.jsonl \
--candidates 6 --repetition-penalty 1.15 --top-p 0.9 \
--web --shuffle

python prose/cloud/chat_continuous.py \
--ckpt prose/models/model_v10_sft9/best.pt \
--output prose/json/continuous_conversations.jsonl \
--prompts shared/prompts/greetings.jsonl \
shared/prompts/modern_prompts.jsonl \
shared/prompts/other_prompts.jsonl \
shared/prompts/weak_spots.jsonl \
--candidates 4 --repetition-penalty 1.15 --top-p 0.9 \
--web --shuffle

V9

python prose/cloud/chat_continuous.py --ckpt prose/models/model_v9_sft4/best.pt --shuffle --prompts shared/prompts/other_prompts.jsonl shared/prompts/followup_prompts.jsonl shared/prompts/victorian_greetings.jsonl

V8

python prose/cloud/chat_continuous.py --ckpt prose/models/model_v8_sft5/best.pt --shuffle --prompts shared/prompts/other_prompts.jsonl shared/prompts/followup_prompts.jsonl shared/prompts/victorian_greetings.jsonl

python3 -m http.server 8000 -d prose/live

[sft] done — best val 2.260 → prose/models/model_v8_sft3/best.pt
[sft] done — best val 2.225 → prose/models/model_v8_sft4/best.pt
[sft] done — best val 2.220 → prose/models/model_v8_sft5/best.pt
==========
[sft] done — best val 2.407 → prose/models/model_v9_sft/best.pt
[sft] done — best val 2.342 → prose/models/model_v9_sft4/best.pt
==========
v10 pretrain — corpus val 3.508 (best == final iter; anneal val 3.75 → 2.80)
[sft] v10_sft val 3.339 @ iter 0 → best 2.526 @ 200 (BUGGY template, deleted)
[sft] v10_sft2 val 2.657 @ iter 0 → best 2.612 @ 100 (template fixed; 14,922 rows)
[sft] v10_sft4 val 2.531 @ iter 0 → best 2.352 @ 125 (6,424 rows, graded 27% of tokens)
^ the template fix alone moved iter-0 val 3.339 → 2.657 with zero training

==========
[build] chat_named 7,490 rows × weight 1 = 7,490
[build] chat_unattributed 13,050 rows × weight 1 = 13,050
[build] graded_single 65 rows × weight 3 = 195
[build] graded_convo 343 rows × weight 5 = 1,715
------------------------------------------------------------

[build] continuous_short 220 rows × weight 5 = 1,100
[build] continuous_long 938 rows × weight 2 = 1,876

[build] continuous_short 222 rows × weight 5 = 1,110
[build] continuous_long 977 rows × weight 2 = 1,954

conversations graded 343 continuous 212
assistant turns graded 1008 continuous 1139

graded turns 663 2026-08-01 → 2026-08-06
candidates 3290 (5.0/turn)
keepers (≤0.5) 1205 (37%)
hand edits 353
checkpoint prose/models/model_v10_sft4/best.pt, prose/models/model_v10_sft5/best.pt

preference pairs 3576 ~1502 live at 42% (cache covers 401 — 262 turns stale)
-> past the 1200 live-pair mark for a DPO round

conversations graded 343 continuous 396
assistant turns graded 1008 continuous 1811

graded turns 1363 2026-08-01 → 2026-08-12
candidates 4745 (3.5/turn)
keepers (≤0.5) 2383 (50%)
hand edits 706
checkpoint prose/models/model_v10_sft4/best.pt, prose/models/model_v10_sft5/best.pt, prose/models/model_v10_sft6/best.pt, prose/models/model_v10_sft6/iter0280.pt, prose/models/model_v10_sft7/best.pt, prose/models/model_v10_sft8/best.pt

preference pairs 4191 ~1760 live at 42% (no cache — run ref_logprobs.py)
-> past the 1200 live-pair mark for a DPO round

READ THE STATS ABOVE WITH CARE

different tokenizers, different corpora, and different val splits; and every
model before v10_sft2 was trained on a chat template with three bugs in it
(see the 2026-08-01 fixes in sft.py). Within a block, rebuilding the dataset
reshuffles the split too.

low-loss behaviour — re-emitting a phrase already in the context is a very
confident prediction — so a model drifting into "I'm not a little glad to be
here. I'm not a little glad to be here." scores BETTER on val. The physical
weighting (graded ×5/×3, continuous ×2) leaks duplicate rows across the split
and rewards memorisation on top of that. Every new-best val in the v10_sft3
run came with a repetition loop; the nicest sample (iter 75) scored worse.

--keep-all is for (saves iter0020.pt, iter0040.pt, ... ~1GB each).

(baseline ≈ 2.1 batches per kept turn).

warmup 50 / 175 iters = 93.7 iter-equivalents
warmup 25 / 175 iters = 95.0 ← shorter warmup shifts the curve, doesn't add budget
warmup 25 / 100 iters = 53.7 ← this one is a real cut (the anti-repetition run)
Note get_lr floors at min_lr_frac=0.1 of peak — sft.py does not expose a flag
to change it, so SFT never decays to zero the way the pretrain does.

  1. gutenberg_fetch_6.py → gutenberg_new/ =
  2. clean_all.py --input-dir prose/data/gutenberg_new → then move to gutenberg/
  3. build_prose_corpus.py → rebuilds corpus.txt
  4. clean_blbooks.py → independent, any time +
  5. gate_oov.py --rebuild-vocab --dry-run → must follow 3