back
loading skill details...
Attach judges to AI Config variations for automatic LLM-as-a-judge evaluation. Create custom judges, configure sampling rates, and monitor quality scores.
AI Config Online Evaluations Attach judges to AI Config variations for automatic quality scoring using LLM-as-a-judge methodology. Judges evaluate responses and return scores between 0.0 and 1.0. Prerequisites LaunchDarkly account with AI Configs enabled API access token with write permissions Existing AI Config with variations (use aiconfig-create skill) For automatic metric recording and the consolidated judge-result API: Python AI SDK v0.20.0+ or Node.js AI SDK v0.20.0+ API Key Detection Check environment variables - LAUNCHDARKLY_API_KEY, LAUNCHDARKLY_API_TOKEN, LD_API_KEY Check MCP config - Claude: ~/.claude/config.json -> mcpServers.launchdarkly.env.LAUNCHDARKLY_API_KEY Prompt user - Only if detection fails Core Concepts
don't have the plugin yet? install it then click "run inline in claude" again.