langsmith-evaluator

Item: langsmith-evaluator
Rating: 3.2
Author: Implexa

INVOKE THIS SKILL when building evaluation pipelines for LangSmith. Covers three core components: (1) Creating Evaluators - LLM-as-Judge, custom code; (2)…

view source

installs

stars

karma

SkillRank score ↗

3.2/ 10

evaluated by implexa, claude-haiku-4-5 · 2026-07-11

langsmith-evaluator exposes environment variable setup and a single CLI command for listing evaluators, but omits intent, procedure steps, decision logic, and output contracts entirely.

structure

2.0

trigger phrases

2.0

procedure

2.0

edge cases

1.0

documentation

3.0

strengths

SKILL.md

LANGSMITH_API_KEY=lsv2_pt_your_api_key_here          # REQUIRED
LANGSMITH_PROJECT=your-project-name                   # Check this to know which project has traces
LANGSMITH_WORKSPACE_ID=your-workspace-id              # Optional: for org-scoped keys
OPENAI_API_KEY=your_openai_key                        # For LLM as Judge

Authentication is REQUIRED: either set the LANGSMITH_API_KEY environment variable, or pass the --api-key flag to CLI commands (preferred):

langsmith evaluator list --api-key $LANGSMITH_API_KEY

related skills

semantically similar in the cross-vendor index

skills.sh

82% match

langsmith-dataset

INVOKE THIS SKILL when creating evaluation datasets, uploading datasets to LangSmith, or managing existing datasets. Covers dataset types (final_response,…

don't have the plugin yet? install it then click "run inline in claude" again.