Botify just analyzed 7 billion log files and surfaced a finding that should reshape how every marketing team thinks about AI search: OpenAI’s crawl of the web has tripled since GPT-5 launched in August 2025. The full study is worth reading in Chris Long’s report on the Botify blog, but the headline alone forces a question most marketers have been ignoring: which OpenAI bot is doing the crawling, and what does each one actually do?
If you’ve been treating “ChatGPT” as one bot, you’re missing three different optimization opportunities, and probably losing visibility in the one that matters most.
Why “ChatGPT” Is Not One Bot
OpenAI operates three distinct crawlers, each triggered by a different event, each serving a different purpose. Conflating them is like treating Googlebot, Adsbot, and Mediapartners-Google as the same crawler. They share an origin, not a job.
Botify’s data shows the three bots growing at very different rates: GPTBot (training) up 2.9x, OAI-SearchBot (search and grounding) up 3.5x, and ChatGPT-User (user-initiated fetches) actually down 28%. The asymmetry is not a coincidence. It tells you where OpenAI is investing, and where your content needs to show up.
The Three Crawlers, Defined
| Bot | Trigger | Purpose | What to optimize for |
|---|---|---|---|
| ChatGPT-User | A user asks ChatGPT to look at a specific page | Fetch a single URL on demand | Page-level clarity, fast load, structured data |
| GPTBot | OpenAI’s scheduled training crawl | Feed foundational model knowledge | Topic depth, authority, evergreen quality |
| OAI-SearchBot | Real-time search to ground a ChatGPT answer | Find current, citable sources for live answers | Topical authority, freshness, internal coherence |
You can identify each one in your server logs by user-agent string. If you have not started monitoring these separately, you do not know what kind of attention your content is getting.
The Paradox in the Data
The most counterintuitive finding from Botify’s study: ChatGPT-User events are down 28% while OAI-SearchBot is up 3.5x. People are clicking “browse this page” inside ChatGPT less, but ChatGPT is going to the web more. How?
The most likely explanation is that OpenAI is now building its own index. In May 2025, the ratio of search-to-train activity was 0.95 (slightly more training than searching). By April 2026, that ratio had flipped to 1.14 (more searching than training). OpenAI is no longer borrowing the web through one-off user fetches. It is constructing its own continuous, real-time view of the web, and grounding answers from that view.
For marketers, the implication is direct: you are now being evaluated by a search index that operates on different rules than Google’s, and being evaluated three times more often than you were a year ago.
What This Means for Content Strategy
The era of optimizing individual high-volume pages and hoping the cluster takes care of itself is over. When AI engines decide whom to cite, they are not pulling one page. They are evaluating whether your domain demonstrates breadth and depth on a topic. A single excellent page surrounded by thin coverage is read as a one-off. A coherent cluster of related pages is read as authority.
This is the shift our team at Ambient Array has been preparing clients for since GPT-5 launched. Our approach is built around the simple observation that the unit of authority is now the topic, not the page. We help clients map the topical territory they need to own and then build coverage that makes them the obvious source for AI engines to cite.
Where Topic Modeler Fits
Topic Modeler is the tool we built to do this systematically. It maps the topical neighborhood around your business, identifies coverage gaps relative to the queries AI engines are answering, and produces a content plan that closes those gaps. The Content Space Expander module shows you specifically which sub-topics you are missing, and which ones competitors already own. If you want to see how the workflow comes together end to end, our Power SEO and AI Authority Builder service walks through the full architecture.
The Practical Next Step
Three things to do this week:
- Pull your server logs and separate visits by user-agent. Count ChatGPT-User, GPTBot, and OAI-SearchBot independently. If one of them is dramatically lower than the others, you have a specific gap to close.
- Pick your three highest-revenue topics. Audit how many distinct pages on your site cover each one. If the answer is one or two, you are not building topical authority, you are building isolated pages.
- Decide whether you want to map your topical territory yourself or whether you want help. Either way, the next twelve months will reward the brands that treat AI search as a measurement-first discipline, not a content-volume race.
The brands that figure this out first will own the answers. That is the whole game now.
Want to see where your topical authority stands today? Schedule a topical authority assessment with our team. We will map your current coverage, identify the gaps that matter most, and show you what closing them looks like.