diff --git a/examples/skill-rl/README.md b/examples/skill-rl/README.md new file mode 100644 index 000000000..e3c220a46 --- /dev/null +++ b/examples/skill-rl/README.md @@ -0,0 +1,239 @@ +# Skill-RL: Train Models to Follow Markdown Instructions + +Train small language models to perfectly execute structured markdown skills using reinforcement learning. + +## The Problem + +Large models (70B+) can follow complex markdown instructions reasonably well. Smaller models (3B-7B) struggle with: +- Multi-step procedures +- Conditional logic ("if X, then do Y") +- Output format compliance +- Tool calling sequences + +**Solution:** Use ART to train small models on specific skills until they match or exceed large model performance. + +## What is a "Skill"? + +A skill is a structured markdown document that defines: +1. **When to apply** — trigger conditions +2. **How to execute** — step-by-step procedure +3. **Expected output** — format and content requirements +4. **Verification** — how to check success + +Example skill structure: +```markdown +# SKILL.md — Git Commit Message Generator + +## Trigger +User asks to write a commit message for staged changes. + +## Procedure +1. Run `git diff --staged` to see changes +2. Identify the TYPE: feat|fix|docs|refactor|test|chore +3. Summarize the change in <50 chars for the subject +4. Add body paragraphs if change is complex +5. Reference issue numbers if mentioned + +## Output Format +``` +(): + + + +