AI Visibility Checks

AI crawler access — whether GPTBot, ClaudeBot, PerplexityBot, and Google-Extended can reach and read your content — is a core VisibilityIQ audit dimension. This guide shows how to access AI-specific findings via the API.

Fetching AI crawler access findings

Filter by checkId prefix to get only AI-access-related findings:

# All AI crawler access findings
curl "https://visibilityiq365.com/api/v1/projects/proj_abc123/issues?scope=site&checkId=ai-access.*" \
  -H "Authorization: Bearer vis_live_YOUR_KEY_HERE"

Or filter by severity to get high-priority AI access issues:

curl "https://visibilityiq365.com/api/v1/projects/proj_abc123/issues?severity=high&checkId=ai-access.*" \
  -H "Authorization: Bearer vis_live_YOUR_KEY_HERE"

Reading AI access evidence

The evidence object for AI crawler findings includes:

{
  "checkId": "ai-access.robots-block-gptbot",
  "severity": "high",
  "confidence": "high",
  "scope": "site",
  "evidence": {
    "userAgent": "GPTBot",
    "directive": "Disallow: /",
    "robotsTxtUrl": "https://example.com/robots.txt",
    "googlebot": "Allowed",
    "affectedPageCount": 1842,
    "rawDirectives": [
      "User-agent: GPTBot",
      "Disallow: /"
    ]
  },
  "guidance": {
    "summary": "GPTBot (OpenAI's training crawler) is blocked from your entire site while Googlebot is allowed.",
    "whyItMatters": "Blocking GPTBot prevents your content from appearing in ChatGPT responses and OpenAI's search results.",
    "autoApplyEligible": true,
    "riskLevel": "review",
    "generatedArtifact": {
      "filename": "robots.txt",
      "mimeType": "text/plain",
      "contents": "# Generated by VisibilityIQ\nUser-agent: *\nAllow: /\n\nUser-agent: GPTBot\nAllow: /\n",
      "placement": "Site root (https://example.com/robots.txt)",
      "sourceFindingIds": ["ai-access.robots-block-gptbot"]
    }
  }
}

Note the generatedArtifact — for robots.txt findings, VisibilityIQ generates the corrected robots.txt file content directly in the finding. You can download this from the dashboard or retrieve it via the API and deploy it programmatically.

JS-only content findings

For pages where content is only available after JavaScript execution:

{
  "checkId": "ai-access.js-only-content",
  "severity": "high",
  "confidence": "medium",
  "scope": "page",
  "pageUrl": "https://example.com/pricing",
  "evidence": {
    "rawBodyLength": 142,
    "renderedBodyLength": 3847,
    "contentAbsentInRaw": [
      "pricing-table",
      "plan-comparison"
    ],
    "aiBotsAffected": ["GPTBot", "ClaudeBot", "PerplexityBot", "Google-Extended"]
  }
}

llms.txt findings

{
  "checkId": "ai-access.llms-txt-missing",
  "severity": "medium",
  "confidence": "high",
  "scope": "site",
  "evidence": {
    "checkedUrl": "https://example.com/llms.txt",
    "statusCode": 404
  },
  "guidance": {
    "summary": "Your site doesn't have an llms.txt file, which helps AI systems understand how to work with your content.",
    "generatedArtifact": {
      "filename": "llms.txt",
      "mimeType": "text/plain",
      "contents": "# example.com\n\n> Technical SEO platform for developers\n\n## Docs\n\n- Docs: https://example.com/docs\n- API: https://example.com/api\n\n## Optional\n\n- Contact: support@example.com\n",
      "placement": "Site root (https://example.com/llms.txt)"
    }
  }
}

Getting the AI readiness summary

To get a high-level view of your AI-surface readiness across all checks:

curl "https://visibilityiq365.com/api/v1/projects/proj_abc123/ai-readiness" \
  -H "Authorization: Bearer vis_live_YOUR_KEY_HERE"

Response:

{
  "ok": true,
  "data": {
    "score": 62,
    "tier": "partial",
    "findings": {
      "robotsBlocking": {
        "agents": ["GPTBot", "ClaudeBot"],
        "pagesBlocked": 1842
      },
      "jsOnlyContent": {
        "pagesAffected": 14,
        "criticalPagesAffected": 3
      },
      "llmsTxt": {
        "present": false,
        "generated": true
      }
    }
  }
}

The tier field: blocked (major bots blocked), partial (some issues), ready (no significant AI access issues).