Learn how to encode your internal link graph and heading hierarchy into HTTP response headers. This allows crawlers to rebuild your site structure without parsing HTML, achieving crawl speeds of 660 requests per second. The technique uses Cloudflare Workers and base64url encoding.
SEO Disciplines
Internal Linking
Discover how to find internal linking opportunities using Screaming Frog's semantic similarity feature. Configure Screaming Frog with an LLM like ChatGPT and an API key, then adjust the semantic similarity threshold to identify similar content. Run a crawl and find pages that are close matches but not exact, revealing valuable linking opportunities.
Keyword clustering and intent at scale
Learn more →Discover how to audit, fix, and rollback internal links on your WordPress site using Claude AI. This tool identifies orphan posts, broken links, and weak anchor text. The plugin runs locally, ensuring no data leaves your machine. Install in two minutes.
Discover how ChatGPT Deep Research uses internal links to understand websites. By analyzing WebSocket traffic, see how it extracts HTML links, creates structured references, and exposes the full link graph, including navigation and footer links, to the model.
BERTlinker uses AI to find internal linking opportunities based on meaning, not just keywords. Upload a crawl file from Screaming Frog, Sitebulb, Ahrefs, or any other crawler and get a formatted Excel report to improve SEO. Free for up to 1,000 pages.