hugging-face-model-trainer

SKILL.md

Cloud-based language model training with TRL on Hugging Face Jobs, supporting SFT, DPO, GRPO, and reward modeling with automatic Hub persistence.

Covers four training methods (SFT for instruction tuning, DPO for preference alignment, GRPO for online RL, reward modeling for RLHF) with production-ready example scripts and cost estimation tools

Submit training jobs via hf_jobs() MCP tool with inline UV scripts (PEP 723 format); no local GPU required, results automatically saved to Hugging Face Hub

Includes real-time monitoring via Trackio, dataset validation before training, GGUF conversion for local deployment (Ollama, llama.cpp), and comprehensive hardware selection guidance (t4-small to a100-large)

Critical prerequisites: paid HF account, HF_TOKEN in job secrets, dataset format validation, timeout set to 1-2+ hours (default 30 min insufficient), and push_to_hub=True configuration to prevent data loss in ephemeral environment

TRL Training on Hugging Face Jobs

Overview

Train language models using TRL (Transformer Reinforcement Learning) on fully managed Hugging Face infrastructure. No local GPU setup required—models train on cloud GPUs and results are automatically saved to the Hugging Face Hub.

TRL provides multiple training methods:

SFT (Supervised Fine-Tuning) - Standard instruction tuning

DPO (Direct Preference Optimization) - Alignment from preference data

GRPO (Group Relative Policy Optimization) - Online RL training

Reward Modeling - Train reward models for RLHF

For detailed TRL method documentation:

hf_doc_search("your query", product="trl")
hf_doc_fetch("https://huggingface.co/docs/trl/sft_trainer")  # SFT
hf_doc_fetch("https://huggingface.co/docs/trl/dpo_trainer")  # DPO
# etc.