grpo-rl-training

SKILL.md

GRPO/RL Training with TRL

Expert-level guidance for implementing Group Relative Policy Optimization (GRPO) using the Transformer Reinforcement Learning (TRL) library. This skill provides battle-tested patterns, critical insights, and production-ready workflows for fine-tuning language models with custom reward functions.

When to Use This Skill

Use GRPO training when you need to:

Enforce specific output formats (e.g., XML tags, JSON, structured reasoning)

Teach verifiable tasks with objective correctness metrics (math, coding, fact-checking)

Improve reasoning capabilities by rewarding chain-of-thought patterns

Align models to domain-specific behaviors without labeled preference data

Optimize for multiple objectives simultaneously (format + correctness + style)

Do NOT use GRPO for:

Simple supervised fine-tuning tasks (use SFT instead)

Tasks without clear reward signals

When you already have high-quality preference pairs (use DPO/PPO instead)