What is OAI-SearchBot, and how is it different from GPTBot?

· Edited by Chris Dolan

OAI-SearchBot is the crawler OpenAI uses to find pages for the search results shown in ChatGPT. GPTBot is a different crawler, which collects content that may be used to train OpenAI's models. A site can allow one and refuse the other, because OpenAI treats them as separate settings. Every statement below was checked against OpenAI's own pages on 9 October 2026.

The practical point is this: blocking GPTBot asks OpenAI to keep your pages out of training, and blocking OAI-SearchBot keeps them out of ChatGPT's search answers. The two choices are separate, and a robots.txt file that refuses both makes both.

What OAI-SearchBot is

OpenAI's overview of its crawlers lists four user agents: OAI-SearchBot, GPTBot, ChatGPT-User and OAI-AdsBot. It describes OAI-SearchBot as the one used to surface websites in the search results of ChatGPT's search features.

ChatGPT search is what makes this matter. OpenAI launched it on 31 October 2024, and an update to the launch announcement dated 5 February 2025 says: “ChatGPT search is now available to everyone in regions where ChatGPT is available”. OpenAI's help article on searching the web with ChatGPT tells site owners what it asks of them: “allow OAI-Searchbot to crawl the site”, and let requests from its published addresses through your host or content delivery network. The article presents this as what makes a site eligible. It does not promise placement.

OAI-SearchBot and GPTBot: search results versus model training

The two crawlers answer to different rules in your robots.txt file, and OpenAI's overview says of those rules: “Each setting is independent of the others”.

  • GPTBot crawls content that may be used to train OpenAI's generative models. Disallowing it tells OpenAI not to use the site's content for training.
  • OAI-SearchBot is for search. OpenAI says a site that opts out of it will not be shown in ChatGPT search answers, though it can still appear as a navigational link.

The overview adds one detail that surprises people: when a site allows both, OpenAI may use the results of a single crawl for both purposes, to avoid fetching the same pages twice.

Where ChatGPT-User fits

ChatGPT-User is not a crawler in the usual sense. It fetches a page when a person using ChatGPT, or a custom GPT, asks for something that needs that page. OpenAI states that it is not used to crawl the web automatically and that it plays no part in deciding whether content appears in search.

It also behaves differently towards robots.txt. The overview says: “Because these actions are initiated by a user, robots.txt rules may not apply.” A rule written for ChatGPT-User may therefore not stop a visit that a person asked for. Our guide to what AI search shows about your business explains why visits made for a user are counted separately from crawling.

Blocking one and allowing the other in robots.txt

To stay out of training and stay eligible for ChatGPT search, give each crawler its own group:

User-agent: GPTBot
Disallow: /

User-agent: OAI-SearchBot
Allow: /

Three things are worth knowing before you rely on it.

  • It is a request. The standard that defines robots.txt, RFC 9309, says of its rules: “These rules are not a form of access authorization.” The same standard makes the crawler's name case-insensitive, so OAI-SearchBot and oai-searchbot are one rule.
  • It takes time. OpenAI says its search systems can take about 24 hours to adjust after a robots.txt change.
  • A blocked page can still be named. OpenAI's FAQ for publishers says that when it learns of a disallowed page from elsewhere, “we may surface just the link and page title in ChatGPT Atlas”. The FAQ also notes that a noindex tag works only on a page the crawler is allowed to fetch.

A robots.txt block is also the first thing to rule out when a site is missing from AI answers. The technical accessibility section of our audit checklist starts there.

How to tell a real OpenAI visit from a fake one

Any program can call itself OAI-SearchBot, so the name in a log line settles nothing. OpenAI publishes the addresses each of its agents uses as a separate file, and the file for this crawler is searchbot.json. A request that names OAI-SearchBot and comes from an address outside that file should be treated as an imitation.

Fetch the file when you need it. The lists change, and the overview warns that the version number in the user-agent string may change too. On 9 October 2026 the examples read OAI-SearchBot/1.4 and GPTBot/1.4.

What changed in 2025 and 2026

OpenAI's crawler page carries no change history, so the dates below come from trade press that compared versions.

  • 9 December 2025. OpenAI edited its description of ChatGPT-User. PPC Land reported: “OpenAI removed language indicating this crawler would comply with robots.txt rules.” The current wording, quoted above, is the result.
  • 23 April 2026. Search Engine Journal reported that the page now listed OAI-AdsBot, “bringing the documented bot count to four”. That agent visits only pages submitted as ads on ChatGPT.

Older articles that describe three OpenAI bots, or say all of them obey robots.txt, were written before these edits.

Sources

All eight were read on 9 October 2026.

  1. Overview of OpenAI Crawlers, OpenAI developer documentation (no date shown).
  2. Introducing ChatGPT search, OpenAI, 31 October 2024, with an update of 5 February 2025.
  3. Searching the web with ChatGPT, OpenAI Help Center.
  4. RFC 9309: Robots Exclusion Protocol, IETF, September 2022.
  5. Publishers and Developers FAQ, OpenAI Help Center.
  6. searchbot.json, OpenAI's published addresses for OAI-SearchBot.
  7. OpenAI revises ChatGPT crawler documentation with significant policy changes, PPC Land, 9 December 2025.
  8. OpenAI's Crawler Docs Now List OAI-AdsBot For ChatGPT Ads, Search Engine Journal, 23 April 2026.