How to Measure AI Search Visibility When ChatGPT ‘Fans Out’ Queries (and Why Rankings Don’t Map 1:1)

Measure three outcomes (don’t collapse them into “rank”):

1. Retrieved/used: your content influences the answer (often invisible).
2. Mentioned: your brand/entity appears in the text.
3. Cited: your domain/URL is linked or explicitly attributed.

“Invisible influence” happens—facts can be used without a citation—but it’s also the easiest metric to overclaim.

Example seed prompt: “best project management software for agencies.” When I tested prompt sets, that one question quickly fanned out into pricing, time tracking + billing, permissions/client portals, migration from Asana/Trello, and utilization reporting. You can rank #2 for the seed keyword and still miss citations if the answer is built from “agency utilization reporting” sources you don’t cover.

Define your measurement unit and KPIs (what to report weekly)

Use a repeatable unit: topic + controlled prompt set + facet coverage, not “one keyword.” Pick a topic stakeholders care about, then write 10–20 prompts that reflect likely fan-out facets (how-to, comparison, constraints, troubleshooting).

Start with 1–2 primary KPIs you can trend weekly:

• Inclusion rate: % of prompts where your domain/pages appear in sources used/cited (in practice: citations; optionally a documented “used” proxy).
• Citation share of voice (SOV): your citations ÷ total citations across the prompt set.
• Answer presence for target entities: % of prompts where your brand/product/person/location is mentioned.

Add secondary KPIs only if you’ll act on them: URL-level coverage (which pages win), consistency across runs (variance), and source diversity vs. concentration (unique domains selected).

Example (toy numbers): 12 prompts, cited in 3 → 25% inclusion rate. 24 total citations, 4 to you → 16.7% citation SOV. If 3 repeats move “cited prompts” from 2–5, report the range.

Step-by-step workflow: measure AI visibility under fan-out (tool-agnostic)

Step 1: Build a controlled prompt set. Start from a seed query, then write 10–20 prompts covering the facets you expect to appear (definitions/selection, comparisons, constraints, implementation). Version it (v1, v1.1); in my experience, prompt drift breaks trendlines faster than real visibility changes.

Step 2: Set sampling rules. Keep assumptions stable: 2–3 runs per prompt, finish the full set within 1–2 days, and hold geo/device/account state/model constant. You’re building a repeatable instrument, not proving “the truth.”

Step 3: Log and normalize. Capture answer text (mentions), cited links (citations), and entities. Normalize to domain (strip tracking params) and canonical URL when possible.

Step 4: Score and roll up. Use a fast rubric: Mention 0/1, Citation 0/1, Inclusion proxy = 1 if cited (optionally “probable used” only when you can document a distinctive paraphrase across runs). Roll up per prompt → topic → domain/page, then report weekly deltas (entered/exited prompts, citations gained/lost). Document prompt versions and page changes so you don’t overclaim causality.

Conclusion: How to align stakeholders on what ‘good’ looks like

Under query fan-out, track topics and facets, not single keywords, and keep used vs. mentioned vs. cited separate. A defensible weekly report shows trend direction: inclusion rate and citation SOV on a stable prompt set, plus which prompts and pages drove movement.

For the next week, pick one topic, draft 10–20 facet-driven prompts, run 2–3 samples per prompt, and baseline inclusion rate + citation SOV. Then change one thing (add a missing facet section or tighten passage clarity) and look for deltas in the prompt cluster that should be affected.

Diagnose missed visibility: the three most common failure modes

1. Facet mismatch: you cover the seed query but not the fanned-out intent.

• Detect: specific prompt clusters never cite/mention you.
• Fix: add a tight section for that facet (constraints, comparisons, implementation steps).

1. Passage mismatch: the info exists, but it’s not extractable.

• Fix: clearer H2/H3s, explicit definitions, and short Q→A blocks inside the page (not a new FAQ page).

1. Trust/access issues: you’re harder to select.

• Check: HTTPS validity, indexability/crawlability basics, and metadata that matches the page.

Some misses are provider/model policy or source availability. Treat that as a constraint and measure what you can repeat.

Sources

1. OpenAI — SearchGPT (prototype) announcement
2. Google — About AI Overviews and AI Mode (Search Help)