Why we removed llms.txt from our AI Visibility methodology

The case that llms.txt is foundational AEO hygiene was the consensus through 2024. Every emerging AI Visibility tool (Profound, Athena Intelligence, ScrunchAI, Otterly, Peec AI) scored it. Every guide recommended shipping it. The case against it arrived in late 2025 and is empirical: SE Ranking's November 2025 study of nearly 300,000 domains tested llms.txt presence against how often AI models cite a domain and found no measurable effect. Google's own AI optimization guide says Google Search ignores it. No major LLM provider has committed to acting on it in production. On June 21, 2026, we removed llms.txt from the scored portion of the 10-Point AI Citation Audit and replaced Check 2 with brand mention frequency, which correlates with AI visibility at 0.66 to 0.71 in Ahrefs' December 2025 study of 75,000 brands. This article explains the decision: what we changed, what we kept, why methodology-transparency requires updating weights when evidence moves, and why this puts us at odds with other AEO tools.

What was llms.txt supposed to do?

llms.txt is a proposed web standard introduced in late 2024 by Jeremy Howard. The intent: give AI crawlers a structured, machine-readable summary of a site, similar in spirit to robots.txt for traditional search crawlers but designed for LLMs. The file lives at the root (https://yourdomain.com/llms.txt), uses a Markdown-derived format, and typically declares the site's identity, key pages, author information, and topic taxonomy. A companion llms-full.txt extends the format with deeper page-by-page descriptions.

The pitch was compelling. Brands wanted a clean way to tell AI systems "this is who we are, these are the pages that matter, here's the context." Mintlify, Anthropic's documentation, Stripe's developer docs, and several other prominent technical brands shipped llms.txt files. IDE-agent tools (Cursor, Continue, Cline, and various MCP servers) built support for the standard. Adoption grew through 2025.

The implicit promise: AI assistants would read these files and use them to inform citation decisions. A brand that shipped a clean llms.txt would be more likely to be understood, attributed, and ultimately cited. AEO tools picked up the convention; by mid-2025, "llms.txt presence" appeared in scoring rubrics across the category.

What did the research find?

The implicit promise didn't hold. The most rigorous test to date was published by SE Ranking in November 2025: a study of nearly 300,000 domains that measured how often each domain is cited by AI models. The methodology combined Spearman correlation with an XGBoost prediction model and SHAP analysis, and tested llms.txt presence as one of the variables.

The result: no measurable effect of llms.txt presence on AI citation rates. Removing the variable made the prediction model more accurate, which means the file added noise, not signal.

Other signals do show up in the data. Ahrefs' December 2025 study of 75,000 brands found branded web mentions correlate with AI visibility at 0.66 to 0.71. SE Ranking's November 2025 study of 129,000 domains found that sites with more than 32,000 referring domains are 3.5 times more likely to be cited by ChatGPT than sites with up to 200. llms.txt shows nothing comparable.

The finding was not an outlier. Adoption was small to begin with. In February 2025, SEO consultant Chris Green crawled the Majestic Million and found about 15 sites with a valid llms.txt file. By November 2025, SE Ranking found the file on 10.13% of its sample of nearly 300,000 domains, while the HTTP Archive's 2025 Web Almanac found it on about 2% of sites. Across all three, citation behavior was indifferent to whether the file existed.

What did Google publicly say?

The signal got even stronger when Google said the quiet part out loud. In June 2025, Google's John Mueller wrote, "FWIW no AI system currently uses llms.txt." Gary Illyes of Google said in July 2025 that Google doesn't support the standard and isn't planning to, as Search Engine Roundtable reported. Google's AI optimization guide now says Google Search ignores the file. Mueller compared llms.txt to the discredited keywords meta tag, a useful-sounding standard with no demonstrable impact on ranking or citation.

Other LLM providers were quieter but consistent. As of September 2026, we have found no public commitment from OpenAI, Anthropic, Meta or Mistral to use llms.txt in any production retrieval system. We followed up with what's publicly disclosed and found no statement from any major LLM provider claiming llms.txt informs their citation behavior.

The standard remains useful in one specific domain: IDE-agent attribution. Cursor, Continue, Cline, and various MCP servers do read llms.txt when pointed at documentation sites. A brand that runs an SDK or developer-facing product still benefits from shipping the file because those tools read it directly. That's a real use case. But it's developer tooling, not consumer AI search, and it's not what the AEO tools were claiming to score.

Why does this break the consensus?

Most AI Visibility scoring tools (Profound, Athena Intelligence, ScrunchAI, Otterly, Peec AI) were built before the SE Ranking research published. They added llms.txt detection to their scoring on the assumption that AI assistants would eventually act on the standard. The assumption was reasonable in early 2025. By late 2025 the evidence said otherwise.

The methodology-honest response is to drop the check. The methodology-convenient response is to keep it because everyone else does. Most tools kept it.

That kept-it position has a real consequence: brands that ship llms.txt get inflated AEO scores that don't predict actual citation lift. They spend hours on a file that gives them a higher number on a vendor scorecard but doesn't move their citation rate in ChatGPT or Perplexity. The score becomes vendor-pleasing rather than reality-tracking.

We didn't want to be in that position. The 10-Point AI Citation Framework is published openly, with weight tables visible on every report. If we left llms.txt in the scoring after the evidence said it didn't work, anyone reading the SE Ranking study and our report would notice the contradiction immediately. Methodology transparency forces methodology honesty.

What did we change in the methodology?

On June 21, 2026, we shipped v0.3.0 of the 10-Point AI Citation Audit. The specific changes:

Check 2 renamed. From "llms.txt presence + quality" to "Brand mention frequency."

Check 2 scoring rubric. The new rubric, in force since June 2026 and set out in brand mention frequency: the #1 predictor, measures weighted brand-name mentions on the open web in the last 90 days. Tier 1 sources (Forbes, WSJ, NYT, Bloomberg, FT, vertical-leading trade press) weighted 3×. Tier 2 (mid-tier trade publications, podcasts with named transcripts, Substack writers) weighted 1.5×. Tier 3 (HARO/Connectively/Featured placements, niche blogs) weighted 1×. Tier 4 (directory listings) weighted 0.25× to avoid double-counting Check 3 (Directory citation footprint).

Per-LLM formula re-weighting. Check 2 weight increased across all five LLMs in June 2026, and the current weight tables are published in the 10-Point AI Citation Framework. Previous range was 5-10% (llms.txt); new range is 10-20% (brand mention frequency). The increased weight reflects the published evidence on brand mentions: Check 2 now carries weight commensurate with its predictive power. Source pool for the re-weighting was pulled primarily from Check 4 (schema) and Check 8 (backlinks), both of which were over-weighted in v0.2.0 relative to the brand-authority evidence.

Veto logic unchanged. Check 1 (crawler accessibility) remains the only veto. A site with llms.txt and a robots.txt block on OAI-SearchBot still scores zero on ChatGPT. The veto rule itself was revised in September 2026, and the current rule is set out in the 10-Point AI Citation Framework.

Process unchanged. All ten checks still run. The audit structure didn't change. The veto logic didn't change. The per-LLM scoring formula structure didn't change. We just swapped the second check for one with evidence behind it and adjusted weights accordingly.

Source: Data for AI Search methodology change of June 2026, documented in the 10-Point AI Citation Framework.

What did we keep?

We kept llms.txt as a hygiene flag on every audit report. The check still runs. The file is still detected. We still report whether it exists.

It just doesn't contribute to the score.

The hygiene flag exists because llms.txt is genuinely useful for IDE-agent attribution. Developer-tooling brands, SDK publishers, and documentation-heavy sites still benefit from shipping it. We surface the file's presence (or absence) on every report so brands can make an informed call about whether to ship it for IDE-agent reasons. We just don't pretend the file moves ChatGPT or Perplexity citation rates because the evidence says it doesn't.

This is the distinction between "we measure this signal" and "this signal moves the score." We measure both. The score only reflects what the evidence supports.

Why is brand mention frequency a better signal?

The mechanic is intuitive once you understand how AI assistants choose what to cite. They are not search engines. They are inference systems trained on a massive corpus of text. When ChatGPT or Claude encounters a buyer question ("best luxury real estate agent in Pacific Palisades"), it does not retrieve the top ten Google results and select one. It infers, from its training corpus and from any real-time retrieval, which brand entities are most strongly associated with the category. Brand mention frequency across the open web is the cleanest signal of that association.

A backlink is one type of mention. But a mention is broader. A brand mentioned in a Forbes article without a hyperlink still increases the model's confidence that the brand exists in the category. A brand mentioned in a podcast transcript, a Substack newsletter, a HARO placement, a trade publication contributed essay, or a public industry directory, even without a backlink, still trains the model's category map.

The branded web mentions that Ahrefs measured capture all of these. It's not a measure of links; it's a measure of brand presence across the open web, exactly the signal AI assistants weight heavily when deciding who to attribute.

Brand mention frequency is also harder to game cheaply. Paid placements can produce mentions, but the source-tier weighting (Forbes vs. niche blog) makes paid-mention gaming expensive and detectable. A brand cannot purchase its way to a high Check 2 score the way it could ship an llms.txt file in twenty minutes.

How does this compare to what other AEO tools do?

Most AEO tools still score llms.txt and have not publicly addressed the SE Ranking findings. We surveyed the category as of June 2026:

• Profound: llms.txt still scored. Closed methodology, no public weight disclosure.
• Athena Intelligence (AthenaHQ): llms.txt still scored. Closed methodology.
• ScrunchAI: llms.txt still scored. Closed methodology.
• Otterly: llms.txt still scored. Closed methodology.
• Peec AI: llms.txt still scored. Closed methodology.
• BrandRank: llms.txt still scored.
• Goodie: llms.txt partial scoring.
• Data for AI Search: llms.txt reported as hygiene flag only. Brand mention frequency is Check 2.

We don't claim our methodology is more accurate than every closed-methodology competitor. We can't. Closed methodologies aren't testable. We claim our methodology is more transparent: every weight is published, every check has a defined rubric, the rationale for every change ships with the change. Brands can audit our audit. When new research shifts the evidence, we update the weights and document why.

If Profound, Athena, ScrunchAI, or any other tool drops llms.txt scoring after this article publishes, that's a win for the category. We'd rather be early on the right answer than late on the convenient one.

What if llms.txt starts working later?

That is possible, because the standard is still under development. Anthropic, OpenAI, Google, Meta, or Mistral could announce production support for llms.txt at any time. If that happens, and if subsequent empirical research shows citation lift, we'll add it back to the scoring with the appropriate weight.

The framework is versioned (v0.3.0 as of July 2026), and the current weights are published in the 10-Point AI Citation Framework. The changelog ships with every report. A future v0.4.0 that re-adds llms.txt scoring will document why, link to the supporting research, and publish the new weights. The same transparency rule that justified removing the check governs adding it back.

What we won't do: re-add the check because the category vendor consensus pressures us back to it. The decision rule is "evidence moves the weights, not vendor convention."

Frequently asked questions

Should we still ship llms.txt?

If you run a developer-tooling product, an SDK, or a documentation-heavy site that IDE agents (Cursor, Continue, Cline, MCP servers) might consume, yes, ship it. The standard is genuinely useful in that context. If you're a marketing site, a service business, or any non-developer-facing brand, the cost-benefit is poor. The file costs 20 minutes to ship and provides no measurable citation lift on consumer AI assistants. Spend the 20 minutes on a HARO pitch instead.

What's the practical impact on our score if we already shipped llms.txt?

None. We still detect it and report it; we just don't add points for it. Brands that previously scored 5-10 points from Check 2 on llms.txt will see those points reallocated to brand mention frequency. Brands with strong open-web mention coverage will score higher under v0.3.0 of the 10-Point AI Citation Audit, in force since June 2026; brands relying heavily on llms.txt without underlying brand mentions will score lower. The change reflects what actually predicts citation.

Is this a contrarian position for marketing reasons?

No. We'd rather not be contrarian. We removed llms.txt from scoring because the empirical evidence said it didn't work, and methodology transparency required acting on the evidence. If the evidence had said llms.txt worked, we'd have kept it and probably increased its weight. The decision rule isn't contrarianism; it's "follow the data."

How can I check brand mention frequency for my own brand?

Run a manual query: "YourBrandName" -site:yourbrand.com on Google with a 90-day date filter. Count the results. Group by source tier. The same logic the audit applies. Tools like Mention.com and Ahrefs' Brand Monitoring give automated tracking. The Data for AI Search audit produces this count automatically and weights it appropriately for the per-platform formulas.

What's the next methodology change you anticipate?

Original data publication (Check 7) helps in our audits as of July 2026, run with the 10-Point AI Citation Audit, though less than brand mentions. We're watching whether that signal strengthens as AI assistants increasingly cite primary research. If published research shows it does, we'll re-weight. Schema-markup signals (Check 4) are also under review. Recent research suggests FAQPage schema specifically may be over-weighted relative to broader schema completeness. Both are candidates for v0.4.0 if the evidence supports a change.

---

The 10-Point AI Citation Audit is open methodology. Every weight is published. Every change is documented. See the 10-Point AI Citation Framework for the full methodology document. Companion guides: What is AEO? and What is GEO?.