Han finetune v2 results
Han LoRA fine-tune v2 (June 2026): ported off MLX/Mac onto Modal cloud. Trained Qwen2.5-7B-Instruct (fp16) with LoRA on top 16 of 28 blocks, batch 8 / lr 1e-5 / max-steps 600 with early-stop patience 2. Ran full 600 steps in ~12.5 min for ~$0.23 on a single A10G. Final eval_loss 0.9104 at epoch 2.21 — basically tied with the MLX 3B-16L v1 floor (0.910) but on a 5x larger base, and eval was still nudging down at the end (headroom for a longer v3 if wanted).
Voice quality: clear step-change over v1. On the 6-prompt acid-test panel (favor / barista / helpful-bait / soft-core / landlord / coworker):
- Raw single-shot Han hit rate: ~67% (12/18 candidates) vs v1's ~21% Sonnet keep rate
- Best-of-3 with ≥1 keeper: 100% (6/6 prompts)
- Anchor-phrase amplification: gone — every line is its own; no v1-style "hit the bricks" repetition
- Qwen3 template bugs: none (Qwen2.5 family, as expected)
Standout lines (no parroting of training data): "You're asking a fish for help with your aquarium" (refusal-by-metaphor), "who's the unlucky guy?" (ring-box noir without sentimentality), "steal his lunch back. keep the receipt." (peak deadpan).
Remaining failure modes (rare, filtered by best-of-N):
- Helpful-eager leak — "Sure thing buddy, I'd love some company. What time do you get here?" (would never)
- Sentimentality slip — "she's in your life for a reason. don't lose her" (not Han)
- Advice-giving — "you should get your own handwriting sample"
- Brief persona slip — "raise it again and I'll find somewhere to live" (model briefly thinks the rent is its own)
These are the same failure modes FINETUNE_HAN.md's "Watch for" section called out — best-of-N + grader pick-best (the original han-finetune-v1-results dog-food plan) cleanly removes them in practice.
Why: Validates the move to Modal was the right call — both speed (12 min vs hours of MLX babysitting) and quality (bigger base genuinely fights helpfulness leaks better). The portfolio-shape stack (train + serve in cloud/, scale-to-zero endpoint) works end-to-end. See modal-stack-baseline for operational details.
How to apply: v2 supersedes v1 as the production Han generator. If the user wants to iterate further: bump max-steps to 1000 (loss was still dropping), or wire up the best-of-N grader pick-best loop for synth data generation against the deployed endpoint instead of Ollama. Don't go back to MLX/3B for serious work — Mac mini local is now scratch-only.