Data Generation Strategy

Current State

Graded data: ~3,929 high-quality Victorian conversations

• cues_remapped.jsonl (900)
• graded_convs.jsonl (294)
• graded_batch400.jsonl (122 edit)
• handpicked.jsonl (97)
• danglers_person.jsonl (1,132)
• cues_batch1.reviewed.jsonl (299)

Problem: Need more data to continue improving model

Solution: Use r2 to Generate New Training Data

Approach 1: Ouroboros (Self-Play)

Generate conversations using r2, then manually review/grade them.

Steps:

1. Use r2 to generate responses to modern prompts
2. Create conversation pairs (user prompt → r2 response)
3. Manually review and grade the generated conversations
4. Add to graded data pool
5. Train r3 on expanded dataset

Pros:

• Can generate unlimited data
• r2 already knows Victorian style
• Control prompt variety

Cons:

• Requires manual review
• May amplify r2's weaknesses
• Time-consuming

Approach 2: Prompt Expansion

Take existing prompts and generate variations.

Steps:

1. Take prompts from graded data
2. Generate 3-5 variations per prompt
3. Use r2 to generate responses to variations
4. Review and grade variations
5. Add to graded data pool

Example:

• Original: "Good morning, how are you today?"
• Variations:
• "Good afternoon, how are you?"
• "How do you do this fine day?"
• "I hope you're well this morning."

Pros:

• Expands prompt coverage
• Maintains quality of original prompts
• Easier than full ouroboros

Cons:

• Still requires review
• Limited by original prompt quality

Approach 3: Cue-Based Generation

Use Victorian-style cues to generate responses.

Steps:

1. Create cue templates (e.g., "I received a letter", "The weather is...")
2. Use r2 to generate responses to cues
3. Review and grade
4. Add to graded data pool

Pros:

• Focused on Victorian topics
• Easy to control content
• Can target specific scenarios

Cons:

• May be too narrow
• Requires cue creation

Recommended Strategy

Phase 1: Prompt Expansion (Quick Win)

• Take 100-200 existing prompts
• Generate 3-5 variations each
• Use r2 to generate responses
• Review 50-100 best variations
• Add to graded data (~300-500 new examples)

Phase 2: Targeted Ouroboros

• Identify weak areas (e.g., specific topics)
• Generate prompts for those areas
• Use r2 to generate responses
• Review and grade
• Add to graded data

Phase 3: Continuous Improvement

• Each round: generate + review 100-200 new examples
• Train next round on expanded dataset
• Repeat

Implementation

Quick Prompt Expansion Script

cd /Users/melissa/Code/tiny-chat

# Take random prompts from existing graded data
# Generate variations
# Use r2 to generate responses
# Save to new JSONL file
# Review and grade

Ouroboros Script

cd /Users/melissa/Code/tiny-chat

# Generate conversation pairs using r2
# Save to JSONL file
# Review and grade
# Add to graded data pool

Metrics to Track

• Prompt variety: Number of unique prompt types
• Response quality: % of responses that sound Victorian
• Conciseness: Average response length
• Topic coverage: Distribution of topics in conversations

Next Steps

1. Immediate: Generate 100-200 prompt variations
2. Short-term: Review and grade 50-100 variations
3. Medium-term: Train r3 on expanded dataset
4. Long-term: Continuous data generation pipeline