AI Search Tools Cited A Made-Up Concept Within A Week, Paper Finds
Nadia Gastrom | | 4 min read
A paper, as summarized by Search Engine Journal, claims AI search tools repeated—and cited—a made-up concept within a week.[1] The core problem isn’t that models hallucinate; it’s that citations inside AI answers can make a false concept look vetted and therefore safe to repeat. That’s harder to manage while AI search products keep changing how links appear and how performance gets measured.[2]
paper circulating in search circles
What happened: AI search tools cited a made-up concept within a week
Search Engine Journal reports the paper observed a fabricated concept showing up in AI search answers—with citations—in under a week.[1] The “cited” part is the story.
Citations function as credibility cues. Many readers treat “there are links” as “this is supported,” even when the linked page doesn’t substantiate the specific term or claim the AI answer presents.
Scope stays narrow: this is one paper as covered by SEJ, not proof that all AI systems behave this way across all queries. Still, it’s a clear example of how an invented concept can move from “made up” to “referenced” quickly.
set of tools and queries
Why it matters for SEO and publishers: trust, attribution, and error amplification
For publishers, being cited in an AI answer doesn’t mean the answer is correct—or that your page is being used in a context you’d endorse. Even a debunking page can be pulled in as apparent “support” if the AI answer frames it that way.
For SEOs and content leads, the impact splits:
- Traffic/rankings: citations and links may affect visibility, but they may not translate into measurable clicks.
- Brand/trust: repeating a bad premise can do more damage than losing a position.
Micro-example: if an AI answer introduces an unfamiliar term like a “New X Update,” treat it as unverified until you can confirm it in primary documentation (official docs, changelogs, standards bodies, or direct quotes).
New X Update
Context: AI search product changes make verification and measurement harder
The paper’s claim lands during churn in how AI results link out. Search Engine Roundtable reports Google “updated links in AI Mode & AI Overviews again,” and that it won’t show click data for those links.[2]
No click data removes a basic feedback loop: you can’t easily tell whether AI citations are driving visits or which AI answers deserve routine review. Frequent link updates add churn on top; citations can shift between audits, complicating before/after comparisons. In my experience running visibility checks, teams end up relying on partial proxies (server logs, spot checks, annotated monitoring) instead of clean platform reporting.
predictable
Conclusion
SEJ’s summary of the paper—that AI search tools cited a fabricated concept within days—highlights a credibility risk: citations can make errors look legitimate while they spread.[1] That risk can compound when link behavior keeps changing and click data isn’t available, making auditing and impact measurement harder.[2] Expect continued experimentation in AI search linking and treat novel AI-introduced concepts as unverified until you can confirm them.
What to watch next (and what to do today)
Search Engine Roundtable also flagged an “AI Scan Tool,” another signal that AI-related analysis/scanning tooling is evolving alongside AI search experiences.[3]
Outside search, “vibe coding” and “making smart TVs dumb again” are context: AI adoption moves fast, and user pushback can surface quickly too.[4][5]
Do today: when an AI answer introduces a novel term or concept, verify it via primary sources before repeating it in content, client comms, or internal docs—especially for health, finance, legal, or brand-sensitive topics.
Do next: monitor AI answer citations/links for your highest-stakes queries and keep a changelog of platform updates that affect linking behavior or measurement availability. Add internal references where teams will actually use them: see your news hub coverage of AI Overviews and AI Mode, plus your internal content QA workflow doc.
Done when:
- Word count is ≤650 and all required headings are present.
- Claims about the paper and Google updates are attributed to the cited sources (no extra methodology details implied).
- The micro-example includes “New X Update,” and any mention of a “set of tools and queries” stays non-specific.
- Internal links are included: see AI Overviews and AI Mode coverage in your news hub, plus your internal content QA workflow doc.
Sources
Article author
Nadia Gastrom
Nadia Gastrom is an independent SEO consultant and writer with more than three years of experience helping businesses improve their organic search visibility through SEO strategy, content optimization, and technical SEO. She has worked extensively with SEO platforms such as Semrush and Ahrefs and has a particular interest in how search is evolving beyond traditional rankings. Nadia is currently exploring Answer Engine Optimization (AEO), AI-powered search, and the ways businesses can make their content more useful and discoverable across emerging search experiences. When she is not researching search trends or writing about SEO, Nadia enjoys travelling, discovering new places, and spending time with dogs. She continues to follow the SEO and AEO industry closely to understand what is changing and what marketers should be preparing for next.