Render Modes

The crawler supports four render modes. The mode controls how pages are fetched and what data is captured.

HTML mode (default first pass)

Raw HTTP GET fetch only. No JavaScript execution. This is the fastest mode and simulates what a non-rendering crawler sees.

Data captured:

• HTTP status code and response headers
• Raw HTML body
• Parsed metadata (title, canonical, meta robots, description)
• Internal and external links (from <a href> tags in raw HTML)
• Structured data blocks (JSON-LD in raw HTML)
• hreflang attributes
• Security headers
• Content hash and word count

This mode is used for the initial crawl of all pages. It identifies pages that may need rendered checking.

Rendered mode

Performs the raw fetch (pass 1) and then loads the same URL in a headless Chromium instance (pass 2) via Cloudflare Browser Rendering. After the page loads and settles (network idle + 500ms), the DOM is captured.

Additional data captured (vs HTML mode):

• Post-hydration title, canonical, meta robots, description (may differ from raw)
• Links only present in rendered DOM
• Structured data injected by JavaScript
• Content blocks only present after rendering
• Render time and JS errors

The two passes are diffed field-by-field. Any discrepancy between raw and rendered values generates a render-parity finding.

When rendered mode is used:

• Explicitly configured at the project level
• Auto-triggered when HTML-mode crawl detects signs of heavy client-side rendering:
  • Very low raw HTML word count relative to expected content
  • Client-side routing patterns detected in HTML
  • Next.js, Nuxt, Gatsby, or similar frameworks detected

Browser rendering sessions are reused across pages via Durable Objects to minimize per-page overhead. A fresh session is created only when the previous session has timed out or errored.

Validation mode

A targeted re-crawl of specific URLs in rendered mode. Used when:

• A user requests “verify this fix” for a specific finding
• A deployment is detected and key pages need re-checking
• A scheduled validation run for high-importance pages

Validation crawls don’t follow links — they only fetch the explicitly specified URLs.

Delta mode

Re-crawls only URLs where the content hash changed since the last crawl. Used for:

• Frequent lightweight crawls on large sites (catching recent changes)
• Verifying that a deployed fix affected the right pages

Delta mode compares the content hash of each URL’s raw HTML against the hash stored from the last full crawl. URLs with matching hashes are skipped. This dramatically reduces crawl time for large sites where most pages don’t change between crawls.

Configuring render mode

At the project level: Project → Settings → Crawl → Render mode

• html — HTML-only, no browser rendering
• auto — HTML first, then render pages that appear to need it (recommended)
• rendered — render all pages (slowest, most thorough)

For most sites, auto is the right choice. Use rendered when you know every page uses client-side rendering and you want parity data on all pages.

Performance characteristics

Mode	Pages/minute	Cost	Coverage
HTML	~100	Minimal	Raw HTML only
Auto	~60	Low-moderate	Full where needed
Rendered	~30	Moderate	Full on all pages
Validation	Variable	Low (targeted)	Only specified URLs
Delta	Variable	Low	Changed pages only

Render times are approximate and depend on page complexity, third-party resources, and edge node load.