Competitor Intelligence Overview
The competitor intelligence module crawls your competitors’ sites and builds a keyword footprint from their pages — revealing what they rank for, how they’re structured, and where you have gaps.
What it reveals
Section titled “What it reveals”Keyword gaps — keywords your competitors rank for that you don’t. These are your most direct growth opportunities: topics the market is searching for where you have no presence but competitors do.
Content structure — how competitors organize their content, what topics they cover across which URL structures, and which page types appear to drive the most ranking signals.
Keyword landscape — a materialized view of keyword overlap across all your competitors, showing where you dominate, where you’re behind, and where no one has a strong presence (uncontested territory worth claiming).
How competitor crawls work
Section titled “How competitor crawls work”When you add a competitor, VisibilityIQ crawls up to 20 of their pages using a respectful crawl agent (VisibilityIQ/1.0) that observes their robots.txt. It extracts keywords from titles, headings, meta descriptions, and URLs — not from their analytics or private data.
The crawl is intentionally shallow (20 pages maximum, 1-second delay between requests, no more than once per 7 days per competitor). This is ethical competitive research, not aggressive scraping.
What it doesn’t do
Section titled “What it doesn’t do”Competitor intelligence does not:
- Access any private data (analytics, rankings, ad performance)
- Crawl pages blocked by
robots.txt - Show you individual competitor traffic numbers (those come from VisibilityIQ’s organic database, not direct crawl)
- Deep-crawl competitor sites (maximum 20 pages per crawl job)
Keyword extraction is based on what’s publicly visible in HTML — the same signals search engines use.
When to use it
Section titled “When to use it”Competitor intelligence is most useful when:
- You’re launching or expanding into a new topic area and want to understand what’s already ranking
- You’re auditing your content strategy against what’s actually being searched for
- You want to identify content gaps quickly without manual research
- You’re prioritizing which new pages to create
It’s not a replacement for original keyword research — it’s a fast way to validate and expand your list based on what’s actually working for competitors in your space.
Getting started
Section titled “Getting started”See Adding competitors to set up your first competitor and Keyword gap analysis to understand the gap analysis view.