site-crawlability

SKILL.md

SEO Technical: Crawlability

Guides crawlability improvements: robots, X-Robots-Tag, site structure, and internal linking.

When invoking: On first use, if helpful, open with 1–2 sentences on what this skill covers and why it matters, then provide the main output. On subsequent use or when the user asks to skip, go directly to the main output.

Scope (Technical SEO)

Redirect chains & loops: Fix multi-hop redirects; point directly to final URL

Broken links (4xx): Fix broken internal/external links; 301 or remove

Site architecture: Logical hierarchy; pages within 3–4 clicks from homepage

Orphan pages: Add internal links to pages with no incoming links

Pagination: Prefer pagination over infinite scroll for crawlability

Crawl budget: Reduce waste on duplicates, redirects, low-value URLs (see below)

AI crawler optimization: SSR for critical content; URL management; reduce 404/redirect waste (see below)

Initial Assessment

Check for project context first: If .claude/project-context.md or .cursor/project-context.md exists, read it for site structure.