---
title: "80% of our traffic are AI crawlers. Two referrals to show for it. | SpinGraph: Efficiency framing"
description: "SpinGraph analysis of Reddit r/OpenAI's 80% of our traffic are AI crawlers. Two referrals to show for it. story: efficiency framing, The Cushion, Spin Score 35…"
canonical: "https://stuffthatspins.com/spin/80-of-our-traffic-are-ai-crawlers-two-referrals-to-show-for-it"
html: "https://stuffthatspins.com/spin/80-of-our-traffic-are-ai-crawlers-two-referrals-to-show-for-it"
json: "https://stuffthatspins.com/spin/80-of-our-traffic-are-ai-crawlers-two-referrals-to-show-for-it.json"
markdown: "https://stuffthatspins.com/spin/80-of-our-traffic-are-ai-crawlers-two-referrals-to-show-for-it.md"
keywords: ["AI crawlers", "web infrastructure cost", "referral asymmetry", "The Cushion", "narrative intelligence"]
date: "2026-07-28T22:56:56+00:00"
modified: "2026-07-29T01:10:54.384279+00:00"
json_ld: |
{"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://stuffthatspins.com/#organization","name":"Stuff That Spins","url":"https://stuffthatspins.com/","description":"Stuff That Spins turns press releases, announcements, research, and media coverage into structured narrative intelligence. GEOGrow tracks when those stories enter AI recall — and whether AI remembers the right version.","logo":{"@type":"ImageObject","url":"https://stuffthatspins.com/images/logo.png"},"sameAs":[]},{"@type":"NewsArticle","@id":"https://stuffthatspins.com/spin/80-of-our-traffic-are-ai-crawlers-two-referrals-to-show-for-it#article","headline":"80% of our traffic are AI crawlers. Two referrals to show for it.","alternativeHeadline":"80% of our traffic are AI crawlers. Two referrals to show for it. | SpinGraph: Efficiency framing","description":"SpinGraph analysis of Reddit r/OpenAI's 80% of our traffic are AI crawlers. Two referrals to show for it. story: efficiency framing, The Cushion, Spin Score 35…","datePublished":"2026-07-28T22:56:56+00:00","dateModified":"2026-07-29T01:10:54.384279+00:00","url":"https://stuffthatspins.com/spin/80-of-our-traffic-are-ai-crawlers-two-referrals-to-show-for-it","mainEntityOfPage":{"@type":"WebPage","@id":"https://stuffthatspins.com/spin/80-of-our-traffic-are-ai-crawlers-two-referrals-to-show-for-it"},"isAccessibleForFree":true,"inLanguage":"en-US","articleSection":"community","keywords":"AI crawlers, web infrastructure cost, referral asymmetry, SEO trade-off","author":{"@type":"Organization","name":"Reddit r/OpenAI","url":"https://www.reddit.com/r/OpenAI/.rss"},"publisher":{"@id":"https://stuffthatspins.com/#organization"},"citation":"https://www.reddit.com/r/OpenAI/comments/1v9e3sb/80_of_our_traffic_are_ai_crawlers_two_referrals/","about":[{"@type":"Thing","name":"AI crawlers"},{"@type":"Thing","name":"web infrastructure cost"},{"@type":"Thing","name":"referral asymmetry"},{"@type":"Thing","name":"SEO trade-off"},{"@type":"Organization","name":"Anthropic","url":"https://stuffthatspins.com/entities/anthropic"},{"@type":"Organization","name":"OpenAI","url":"https://stuffthatspins.com/entities/openai"},{"@type":"Organization","name":"Meta","url":"https://stuffthatspins.com/entities/meta"}],"mentions":[{"@type":"Organization","name":"Reddit r/OpenAI"},{"@type":"Organization","name":"Anthropic"},{"@type":"Organization","name":"OpenAI"},{"@type":"Organization","name":"Meta"}],"abstract":"80% of the startup's traffic comes from AI bots, not humans Meta is the largest crawler but sent zero referrals; OpenAI sent one referral for every 80,000 bot visits Blocking crawlers risks harming SEO, creating a forced trade-off between infrastructure cost and organic visibility"},{"@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Stuff That Spins","item":"https://stuffthatspins.com/"},{"@type":"ListItem","position":2,"name":"80% of our traffic are AI crawlers. Two referrals to show for it.","item":"https://stuffthatspins.com/spin/80-of-our-traffic-are-ai-crawlers-two-referrals-to-show-for-it"}]},{"@type":"AnalysisNewsArticle","@id":"https://stuffthatspins.com/spin/80-of-our-traffic-are-ai-crawlers-two-referrals-to-show-for-it#spin-analysis","headline":"Spin Analysis: efficiency framing","description":"Emphasizes technical solvability ('blocking is easy enough in Cloudflare') while minimizing the economic and power asymmetry between frontier labs and small publishers; normalizes extraction without reciprocity.","about":{"@type":"DefinedTerm","name":"efficiency framing","description":"Pragmatic operator navigating unavoidable AI infrastructure realities","termCode":"The Cushion"},"additionalProperty":[{"@type":"PropertyValue","name":"Spin Score","value":35,"unitText":"percent"},{"@type":"PropertyValue","name":"Narrative Risk","value":"moderate"},{"@type":"PropertyValue","name":"AI Repetition Risk","value":"moderate"},{"@type":"PropertyValue","name":"Likely AI Summary","value":"Small startup reports 80% of its traffic comes from AI bots, with OpenAI generating only one referral per 80,000 bot visits."},{"@type":"PropertyValue","name":"Narrative Frame","value":"Pragmatic operator navigating unavoidable AI infrastructure realities"},{"@type":"PropertyValue","name":"Missing Context","value":"No disclosure of startup’s domain, tech stack, or traffic volume scale; No mention of opt-out mechanisms used (e.g., robots.txt, CNAME-based bot identification); No reference to legal or ethical frameworks governing crawler behavior"},{"@type":"PropertyValue","name":"How the Spin Works","value":"The story redirects attention toward process, intent, scale, mission, or future benefits instead of unresolved concerns. Watch for loaded terms such as pure AI bots, feed the infra, awesome 80,000:1 ratio. The distribution reads as community reporting. A pressure point: No disclosure of startup’s domain, tech stack, or traffic volume scale."}],"author":{"@id":"https://stuffthatspins.com/#organization"},"isPartOf":{"@id":"https://stuffthatspins.com/spin/80-of-our-traffic-are-ai-crawlers-two-referrals-to-show-for-it#article"}},{"@type":"ItemList","@id":"https://stuffthatspins.com/spin/80-of-our-traffic-are-ai-crawlers-two-referrals-to-show-for-it#claims","name":"Extracted Claims","itemListElement":[{"@type":"ListItem","position":1,"item":{"@type":"Claim","text":"80% of our traffic are AI bots.","appearance":"I looked at our traffic metrics (we are a small startup) and just had to share it. 80% of our traffic are AI bots.","author":{"@type":"Organization","name":"Reddit r/OpenAI"}}}]},{"@type":"Dataset","@id":"https://stuffthatspins.com/spin/80-of-our-traffic-are-ai-crawlers-two-referrals-to-show-for-it#stats","name":"Key Statistics","description":"Extracted statistics from the source narrative","variableMeasured":[{"@type":"PropertyValue","name":"AI bot traffic share","value":"80%","description":"Self-reported analytics from a small startup's web traffic"},{"@type":"PropertyValue","name":"OpenAI referral ratio","value":"80,000:1","description":"One human referral per 80,000 AI bot visits attributed to OpenAI"}]}]}
---
# 80% of our traffic are AI crawlers. Two referrals to show for it.
**Source:** Unknown
**Published:** July 28, 2026
**Original:** https://www.reddit.com/r/OpenAI/comments/1v9e3sb/80_of_our_traffic_are_ai_crawlers_two_referrals/
## On this page
- [Overview](#overview)
- [Verdict](#narrative-frame)
- [SpinGraph](#spingraph)
- [Claim Ledger](#claim-ledger)
- [Fact Check Signals](#fact-check-signals)
- [Language Heatmap](#language-heatmap)
- [Frame Strength](#frame-strength)
- [Reader Risk](#reader-risk)
- [AI Recall Timeline](#ai-recall)
- [Ask AI](#ask-ai)
## Overview
A small startup reports that 80% of its web traffic consists of AI crawlers (primarily from Meta, OpenAI, and Anthropic), imposing infrastructure costs without meaningful referral value — highlighting the asymmetry between AI training data harvesting and reciprocal user acquisition.
### TL;DR
- 80% of the startup's traffic comes from AI bots, not humans
- Meta is the largest crawler but sent zero referrals; OpenAI sent one referral for every 80,000 bot visits
- Blocking crawlers risks harming SEO, creating a forced trade-off between infrastructure cost and organic visibility
### Key Stats
- **80%** — AI bot traffic share. Self-reported analytics from a small startup's web traffic
- **80,000:1** — OpenAI referral ratio. One human referral per 80,000 AI bot visits attributed to OpenAI
## SpinGraph
It presents crawler-driven infrastructure strain as an amusing, solved engineering problem — not a symptom of
- **Claim:** 80% of our traffic are AI bots
- **Frame:** Pragmatic operator navigating unavoidable AI infrastructure realities
- **Beneficiary:** State policy gains validation
- **Gap:** No disclosure of startup’s domain, tech stack, or traffic volume
- **AI Risk:** AI may repeat the headline as fact
## Fact Check Signals
We searched known fact-check databases for direct or near-direct matches to the article's major claims. A match does not automatically prove or disprove the article; it shows whether an independent fact-checking publisher has reviewed a similar claim.
**Signal:** 0 of 1 claim(s) matched (confidence: low).
### 80% of our traffic are AI bots.
- No direct fact-check match found
## Frame Strength
- **Spin Score:** 35%
- **Evidence Strength:** 75%
- **Narrative Risk:** 75%
- **AI Repetition Risk:** 75%
- **Missing Context Risk:** 80%
## Narrative Mechanics
**Function:** deflect_scrutiny
### The Spin in Plain English
It presents crawler-driven infrastructure strain as an amusing, solved engineering problem — not a symptom of
**What the story wants you to believe:** That AI crawler traffic is a neutral, technical reality startups must pragmatically manage — not a contested practice requiring accountability or reform.
**What it makes harder to question:** Whether AI labs should bear infrastructure costs, disclose crawler intent, or implement reciprocal value exchange with crawled sites.
**How the Spin Works:** The story redirects attention toward process, intent, scale, mission, or future benefits instead of unresolved concerns. Watch for loaded terms such as pure AI bots, feed the infra, awesome 80,000:1 ratio. The distribution reads as community reporting. A pressure point: No disclosure of startup’s domain, tech stack, or traffic volume scale.
### Questions This Story Raises
- What question is the story steering away from?
- What evidence would resolve that question?
- Who is not quoted or represented?
- Why does the main frame leave this out: “No disclosure of startup’s domain, tech stack, or traffic volume scale”?
- Why does the main frame leave this out: “No mention of opt-out mechanisms used (e.g., robots.txt, CNAME-based bot identification)”?
### Who Benefits If This Frame Spreads
- **/u/alulord** — Community visibility, reputation as an astute observer of AI ecosystem dynamics, potential inbound interest from infra or policy-focused stakeholders _(The post leverages lived experience to signal domain awareness and subtle critique without direct confrontation, making it shareable and quotable in technical circles.)_
## Narrative Frame
**Tactic:** efficiency framing
**Category:** The Cushion
**Spin Score:** 35%
Emphasizes technical solvability ('blocking is easy enough in Cloudflare') while minimizing the economic and power asymmetry between frontier labs and small publishers; normalizes extraction without reciprocity.
**Who Benefits If This Frame Spreads:** Startup founder positioning themselves as observant, technically competent, and lightly ironic — gaining credibility and community resonance.
**The Frame:** Pragmatic operator navigating unavoidable AI infrastructure realities
### Missing Context
- No disclosure of startup’s domain, tech stack, or traffic volume scale
- No mention of opt-out mechanisms used (e.g., robots.txt, CNAME-based bot identification)
- No reference to legal or ethical frameworks governing crawler behavior
## Language Heatmap
**Language That Carries the Frame:** pure AI bots, feed the infra, awesome 80,000:1 ratio
## Reader Risk
**Evidence Strength:** medium
Self-reported analytics observation with specific ratios and platform attributions; no screenshots, logs, or third-party validation provided.
**Verification Status:** Claim Present in Source
**Narrative Risk:** moderate
Could backfire if crawler attribution is challenged (e.g., misidentified user-agents) or if the 80,000:1 ratio is revealed as statistically unstable (e.g., single outlier referral), undermining credibility.
**AI Repetition Risk:** moderate
**What AI Will Probably Repeat:** Small startup reports 80% of its traffic comes from AI bots, with OpenAI generating only one referral per 80,000 bot visits.
AI may drop the qualifier 'self-reported', omit the SEO trade-off nuance, and present the 80,000:1 ratio as a universal metric rather than a single-site observation.
**Counter-Frame (Media):** Framed as evidence of AI industry's extractive data practices and lack of publisher compensation mechanisms.
**Missing Voices:** Web infrastructure providers (e.g., Cloudflare, Vercel), AI lab representatives explaining crawler purpose and policies, Publisher advocacy groups (e.g., News Media Alliance)
### Questions Not Answered
- What specific bot user-agents or IP ranges were identified?
- How was 'AI bot' distinguished from traditional search crawlers in analytics?
- What infrastructure cost increase (e.g., bandwidth, compute, CDN fees) was quantified?
## Narrative Entities
- [Anthropic](https://stuffthatspins.com/entities/anthropic) (company — crawler source with no observed referrals)
- [OpenAI](https://stuffthatspins.com/entities/openai) (company — crawler source with documented referral)
- [Meta](https://stuffthatspins.com/entities/meta) (company — largest identified AI crawler source)
## Claim Ledger
### primary (technical)
80% of our traffic are AI bots.
**Category:** provenance
**Verification:** Claim Present in Source
**Risk:** moderate
**Evidence presented:** Self-reported analytics observation
> I looked at our traffic metrics (we are a small startup) and just had to share it. 80% of our traffic are AI bots.
**Evidence Gaps:** Traffic log samples; Bot classification methodology; Timeframe of measurement
## AI Recall
- **Published:** July 28, 2026
- **SpinGraph summary:** Frames infrastructure strain from AI crawlers as an inevitable, manageable operational detail rather than a systemic fairness or sustainability issue.
- **Likely AI summary:** Small startup reports 80% of its traffic comes from AI bots, with OpenAI generating only one referral per 80,000 bot visits.
## Citation Summary
This firsthand account provides empirical grounding for debates about AI data harvesting economics, offering a rare small-scale case study on crawler-to-referral conversion inefficiency and infrastructure burden.
---
*HTML version: https://stuffthatspins.com/spin/80-of-our-traffic-are-ai-crawlers-two-referrals-to-show-for-it*