chunking-strategy

SKILL.md

Optimal chunking strategies for RAG systems and document processing pipelines.

Five strategy levels from fixed-size to advanced methods (late chunking, contextual retrieval), each suited to different document types and complexity

Includes recursive character chunking with hierarchical separators, structure-aware chunking for Markdown/code/PDFs, and embedding-based semantic chunking with configurable thresholds

Provides evaluation framework covering retrieval precision, recall, end-to-end accuracy, processing latency, and resource usage

Best practices emphasize starting simple (512 tokens, 10-20% overlap), testing with representative documents, and iterating based on document characteristics and retrieval performance

Chunking Strategy for RAG Systems

Overview

Provides chunking strategies for RAG systems, vector databases, and document processing. Recommends chunk sizes, overlap percentages, and boundary detection methods; validates semantic coherence; evaluates retrieval metrics.

When to Use

Use when building or optimizing RAG systems, vector search pipelines, document chunking workflows, or performance-tuning existing systems with poor retrieval quality.

Instructions

Choose Chunking Strategy

Select based on document type and use case: