AIVisibilityObservation

AIVisibilityObservation captures two distinct things under the “AI visibility” umbrella: (1) whether AI crawlers can physically access your content, and (2) whether your content is being cited in AI-generated answers (AI Overviews, Perplexity, ChatGPT search).

These are related but separate concerns. A site can block GPTBot entirely (access problem) or can be accessible to GPTBot but never cited (citation/quality problem).

TypeScript interface

interface AIVisibilityObservation {
  url: string;
  projectId: string;
  observedAt: string;

  // Bot access (from robots.txt analysis)
  robotsAccessByAgent: {
    googlebot: BotAccess;
    bingbot: BotAccess;
    gptbot: BotAccess;
    claudebot: BotAccess;
    perplexitybot: BotAccess;
    googleExtended: BotAccess;
    oaiSearchbot: BotAccess;
    bytespider: BotAccess;
  };

  // Content accessibility
  contentRequiresJs: boolean;         // primary content absent in raw HTML
  rawBodyWordCount: number;
  renderedBodyWordCount: number | null;
  contentDeltaWords: number | null;   // rendered - raw (positive = JS adds content)

  // llms.txt
  llmsTxtPresent: boolean;
  llmsTxtUrl: string | null;
  llmsTxtValid: boolean;              // parseable and has required sections
  llmsTxtWordCount: number | null;

  // AI Overview citations (from AI Visibility Tracker connector — Tier 2)
  aiOverviewCitations: AIOCitation[];
  perplexityCitations: Citation[];
  chatGptCitations: Citation[];

  // Crawl-delay for AI agents
  gptbotCrawlDelay: number | null;    // seconds, from robots.txt
  claudebotCrawlDelay: number | null;
}

type BotAccess = 'allowed' | 'blocked' | 'unknown';

interface AIOCitation {
  keyword: string;
  citedUrl: string;
  citationType: 'primary' | 'supporting';
  detectedAt: string;
}

interface Citation {
  keyword: string;
  citedUrl: string;
  position: number;
  detectedAt: string;
}

Access vs. citation

The robots access fields capture what your robots.txt allows or denies. The citation fields capture actual citation behavior in AI-generated responses — these require the AI Visibility Tracker connector (Tier 2, not yet available in all plans).

Even with full access, content that’s only available via JavaScript is effectively invisible to AI crawlers that don’t render JavaScript. contentRequiresJs = true with high contentDeltaWords is a strong signal that AI crawlers can’t read your primary content even though robots.txt allows them.

Findings from this model

The audit engine generates these findings from AIVisibilityObservation:

Finding	Condition
ai-access.robots-block-gptbot	robotsAccessByAgent.gptbot = 'blocked'
ai-access.js-only-content	contentRequiresJs = true AND contentDeltaWords > 500
ai-access.llms-txt-missing	llmsTxtPresent = false
ai-access.crawl-delay-ai	gptbotCrawlDelay > 60 OR claudebotCrawlDelay > 60

These findings include a generatedArtifact in the guidance payload — the corrected robots.txt or a generated llms.txt appropriate to the site.