melissa kate

Little Jane

A 245M-parameter Victorian chat model, trained from nothing

Posts

• compare/ 2026-10-05

  Jane from scratch, against the same corpus fine-tuned onto a modern base.

• Jane from scratch vs Qwen2.5-0.5B on the same corpus 2026-10-05

  2026-10-05. Settled. Read this before proposing to re-run it.

• Data Generation Strategy 2026-10-02

  Current State

• Continual Learning Pipeline for v11 2026-10-02

  Overview

• Continual Learning - Round 1 Results 2026-10-02

  Summary

• v11 SFT Setup 2026-10-01

  This folder contains the complete SFT pipeline for the v11 model line.

• 1. Build the anneal corpus (local, fast) 2026-08-30

  python prose/scripts/build_anneal_corpus.py

• The SFT loop 2026-08-27

  Grade → build → run → read. One round is three commands and about half an

• v12 — corpus rebuild, BL mining, six-way anneal sweep (2026-08-25) 2026-08-25

  What changed

• v10 (sft9/sft10) vs v11 — same probes 2026-08-23

  v10: temp 0.8 top_k 40 top_p 0.9 rep 1.15

• tiny-prose todo 2026-08-22

  v11 base + anneal done. sft.py works. Remaining work is all DATA.

• tiny-prose running cost ledger 2026-08-20

  Rough tracking of cloud spend across the project. Update as we go.

• Corpus design: pretrain is a language, SFT is a voice 2026-08-17

  The single most important rule in this project, and the one the fetch rounds

• v10 SFT checkpoint comparison 2026-08-01

  Six prompts, one reply each, from best.pt of every v10 SFT run.

• compare-v8_sft5-vp_sft 2026-07-26

  ((venv) ) melissa@Melissas-MacBook-Pro tiny-chat % python prose/cloud/chat_continuous.py --ckpt prose/models/model_v8_sft5/best.pt --prompts shared/prompts/foll

• What I Actually Learned Training a Tiny Transformer From Scratch 2026-07-07

  A journal of the real mechanics, not the textbook version.

• Scaling Plan — Phase 2 2026-07-07

  After the May 2026 experiments (see LEARNINGS.md), the path forward is clear:

• Fine-tuning a "Han responder" (Qwen2.5-7B-Instruct + MLX-LM LoRA) 2026-07-07

  Goal: bake Han's voice into a local model's weights so it answers terse-and-dry

• Han — Voice Rubric 2026-07-07

  _Auto-generated from the gold pairs. Edit freely — this is what the reply suggestions imitate, and what graders score against._

• Settings prompts 2026-07-07

  Interior: Smoky jazz club, back booth

• tiny-chat roadmap 2026-07-07

  Long-running side project to train a sardonic-detective tiny LM from scratch

• tiny-chat — project notes / handoff 2026-07-07

  Context capture so this work is portable across machines (the Cowork chat itself

Notes

• Doppelganger discoveries model "How Doppelganger's branching works — discoveries, not routes; losing is allowed; the phone is the second chance"
• Han finetune v1 results "Results of the first Han LoRA fine-tune end-to-end experiment — what worked, the characteristic failure mode, and the planned next iteration."
• Han finetune v2 results "Second Han LoRA fine-tune — Qwen2.5-7B-Instruct on Modal A10G. Clear step-change over v1: ~3x better raw hit rate, anchor-phrase amplification gone."
• Modal stack baseline "Working Modal-based training + inference stack for the Han generator. Operational baseline — what fits where, what it costs, the gotchas hit on first build."
• Tiny prose alive not coherent Her ear prefers models that are LESS self-contained and LESS cue-responsive — those metrics measure blandness, not sense
• Tiny prose base is male The anneal pool excludes female speakers BY CONSTRUCTION — the base is a male-voiced Victorian gentleman with personality, and SFT supplies gender, not conversation
• Tiny prose character is topic Jane sounded boring because the corpus is 17% money / 15% marriage / 0.7% gothic — and because sampling temp had been lowered
• Tiny prose corpus design "Tiny-prose corpus rule — pretrain teaches a register (volume, author sex irrelevant), SFT teaches the voice (female SPEAKER, always); two different cleanliness bars"
• Tiny prose corpus retrieval "Melissa feels hand-writing SFT turns is 'too much of my hand behind the curtain'; corpus retrieval (quote_index.py) is the sanctioned third source"
• Tiny prose decoding config "Decoding config is a first-class lever for Little Jane and lives in the checkpoint; rep penalty 1.15 + learned </s> stop, both added 2026-08-26"
• Tiny prose dpo round1 "DPO round 1 on Little Jane (2026-08-13) produced no gain by three independent measures; checkpoint deleted. Don't re-run without more distinct graded turns."
• Tiny prose opener coverage "Gold's opener set IS Jane's opener distribution; shared/prompts is only 9% used, and the thinnest intents are exhausted in the library too"
• Tiny prose register is the defect Every selection mechanism aims at 17-25 word expository prose while the target is 1-2 sentences; gold is the most verbose source in the mix
• Tiny prose side project "From-scratch TinyGPT ('Little Jane') on a Victorian corpus. v11 base retrained 2026-08-24 (7.37B tokens, val 2.7379). Best SFT as of 2026-08-27 is v11/models/sft_cfg/g4voc2. The loop is documented in prose/notes/sft-loop.md; grading is the remaining headroom."
• Tiny prose voice concentration flat Concentrating one author's voice (Warner at 14.3% of supervised words) did not make her sound more like a person — mixture levers keep coming back flat
• Tiny prose volume is not the bottleneck Chatterbox is the control — 3.8x Jane's data, worse at conversation; the constraint is register/shape, not volume