OAI-SearchBot (OpenAI Search Crawler)

OAI-SearchBot is OpenAI's web crawler for search. OpenAI uses OAI-SearchBot to surface websites in search results within ChatGPT's search features.

For publishers and website owners, OAI-SearchBot is particularly relevant to ChatGPT Search because it provides a specific robots.txt control for managing whether OpenAI's search crawler can access website content.

OAI-SearchBot should not be confused with GPTBot or ChatGPT-User. OpenAI documents these as separate user agents with different purposes.

What Does OAI-SearchBot Do?

OAI-SearchBot crawls public web content for OpenAI's search products. OpenAI describes it specifically as a crawler used to surface websites in search results in ChatGPT's search features.

This makes OAI-SearchBot part of the technical discovery layer behind OpenAI search experiences rather than a general-purpose label for every request made by ChatGPT.

Website owners can use the OAI-SearchBot robots.txt token to communicate whether the crawler is permitted to access particular sections of a website.

OAI-SearchBot and ChatGPT Search

ChatGPT Search allows ChatGPT to incorporate current information from the web and provide source links as part of supported search experiences.

OAI-SearchBot is the crawler website owners should manage when deciding whether their public content can participate in OpenAI's automatic search crawling.

OpenAI recommends allowing OAI-SearchBot in robots.txt and allowing requests from its published search crawler IP ranges to help ensure a site can appear in search results.

How to Allow OAI-SearchBot in Robots.txt

A website that wants to allow OAI-SearchBot across its public site can use a robots.txt rule such as:

The file should be published as the site's robots.txt file and interpreted according to the applicable Robots Exclusion Protocol rules.

More granular rules can also be used when a publisher wants to permit crawling of some paths while excluding others.

How to Block OAI-SearchBot

A publisher can communicate that OAI-SearchBot should not crawl the site by using:

OpenAI states that sites opted out of OAI-SearchBot will not be shown in ChatGPT search answers, although a site may still appear as a navigational link in certain circumstances.

OpenAI also notes that changes to OAI-SearchBot robots.txt settings can take approximately 24 hours to be reflected by its systems.

Can a Blocked Page Still Appear as a Link?

Blocking OAI-SearchBot does not necessarily mean that every reference to the URL becomes impossible.

OpenAI explains that if it learns about a disallowed page through another source, such as a third-party search provider or links discovered on other pages, it may still have enough information to surface a navigational link or page title in supported experiences.

This illustrates an important difference between preventing content crawling and preventing a URL from being known.

OAI-SearchBot vs. GPTBot

OAI-SearchBot and GPTBot are separate OpenAI crawler controls.

OpenAI explicitly states that these settings are independent. A publisher can therefore allow OAI-SearchBot while blocking GPTBot.

This configuration communicates that the website permits OpenAI's search crawler while opting out of GPTBot crawling for generative AI foundation model training.

OAI-SearchBot vs. ChatGPT-User

ChatGPT-User is another separate OpenAI user agent. OpenAI uses it for certain actions initiated by users in ChatGPT and Custom GPTs.

Unlike OAI-SearchBot, ChatGPT-User is not used to automatically crawl the web and is not the control used to determine whether content may appear in Search.

Because ChatGPT-User requests are initiated by users, OpenAI notes that robots.txt rules may not apply to them in the same way as automatic crawling.

OAI-SearchBot and AI Crawler Accessibility

Allowing OAI-SearchBot in robots.txt is only one component of AI Crawler Accessibility.

A website may allow OAI-SearchBot in robots.txt while another technical layer prevents the crawler from retrieving the page.

Potential barriers include:

• CDN security rules;
• web application firewalls;
• bot-management systems;
• CAPTCHA challenges;
• JavaScript-based human verification;
• authentication requirements;
• IP restrictions;
• geographic restrictions;
• rate limiting;
• 403 Forbidden responses;
• 429 Too Many Requests responses; and
• server errors.

A complete OAI-SearchBot accessibility audit should therefore test actual technical access rather than reviewing robots.txt alone.

OAI-SearchBot IP Ranges

OpenAI publishes IP ranges for OAI-SearchBot. These can help engineering and security teams identify or allow legitimate crawler traffic when infrastructure-level controls require IP-based rules.

The current list is maintained by OpenAI at OpenAI's OAI-SearchBot IP range file.

Because crawler infrastructure can change, website owners should rely on OpenAI's maintained list rather than copying a static set of IP addresses into permanent documentation.

OAI-SearchBot User-Agent

OAI-SearchBot identifies itself through an HTTP user-agent containing the OAI-SearchBot token.

OpenAI notes that version numbers in its example user-agent strings may change. Technical rules should therefore be designed around the documented crawler identity rather than assuming one permanent version number.

OpenAI may also include a robots.txt marker when OAI-SearchBot fetches a robots.txt file, helping website owners distinguish those requests in logs.

Does OAI-SearchBot Index Websites?

It is more accurate to describe OAI-SearchBot using OpenAI's own language: it is the crawler for search and is used to surface websites in search results in ChatGPT's search features.

Website owners should avoid assuming that OpenAI uses a traditional Google-like indexing architecture simply because it operates a search crawler.

Crawling, indexing, retrieval, source selection, and answer generation are separate concepts.

This distinction is particularly important when discussing AI Search Indexing.

Does Allowing OAI-SearchBot Guarantee ChatGPT Visibility?

No. Allowing OAI-SearchBot removes one potential technical barrier to OpenAI's search crawler, but it does not guarantee that a website will be selected for a particular search answer.

Search visibility can depend on the user's query, available sources, relevance, retrieval and ranking systems, freshness, source selection, and other factors.

Does OAI-SearchBot Access Guarantee a Citation?

No. Crawler access and AI Citations occur at different stages.

OAI-SearchBot may be technically able to access a website while that website is not selected as a source for a particular user query.

This is why teams should measure actual citation performance rather than using crawler accessibility as a proxy for citation visibility.

OAI-SearchBot and AI Visibility

OAI-SearchBot accessibility can form part of the technical foundation for AI Visibility in ChatGPT Search.

The technical layer asks whether OpenAI's search crawler can access the intended public content. The measurement layer asks whether the brand, website, products, and pages actually appear in relevant AI-generated answers.

OAI-SearchBot and Citation Monitoring

Once crawler access is correctly configured, teams can monitor whether their domains and URLs actually become sources in AI-generated answers.

AI Citation Monitoring helps teams measure source appearances instead of assuming that technical accessibility translates directly into citations.

This separates two different questions:

How to Audit OAI-SearchBot Access

1. Review robots.txt. Check whether OAI-SearchBot is explicitly or indirectly blocked from important public content.
2. Check important paths. Verify that product, feature, documentation, article, category, and other strategically important URLs are accessible.
3. Review CDN and WAF rules. Confirm that infrastructure security does not block legitimate OpenAI crawler traffic.
4. Check bot protection. Look for CAPTCHA, JavaScript challenges, behavioral detection, or other automated systems that may reject crawlers.
5. Inspect HTTP responses. Pay particular attention to 403, 429, 5xx, redirect, and authentication responses.
6. Use OpenAI's published IP ranges where appropriate. Reference the current official search crawler IP list rather than a manually maintained static list.
7. Separate Search from training policy. Review OAI-SearchBot and GPTBot independently.
8. Monitor actual ChatGPT visibility. Technical accessibility alone cannot show whether content is appearing in relevant answers.

Common OAI-SearchBot Mistakes

• confusing OAI-SearchBot with GPTBot;
• blocking GPTBot and assuming ChatGPT Search is therefore blocked;
• allowing OAI-SearchBot in robots.txt while blocking it at the CDN or WAF;
• assuming every ChatGPT request uses OAI-SearchBot;
• confusing ChatGPT-User with automatic search crawling;
• using outdated crawler IP addresses;
• assuming crawler access guarantees a ChatGPT citation;
• assuming robots.txt alone determines AI Search visibility; and
• treating OAI-SearchBot as a model-training crawler.

OAI-SearchBot and LLM SEO

Managing OAI-SearchBot correctly can be one technical component of LLM SEO, AEO, and Generative Engine Optimization (GEO).

The objective is not simply to "allow AI bots." It is to make an informed decision about which systems should access public content and then ensure the intended search crawlers are not accidentally blocked by technical infrastructure.

From OAI-SearchBot Access to ChatGPT Search Performance

OAI-SearchBot sits near the beginning of the ChatGPT Search discovery workflow. Once the technical access layer is working, teams still need to understand which prompts trigger their brand, which pages are cited, which competitors appear, and how visibility changes over time.

Using Ansvisor's ChatGPT Visibility Tracker, teams can monitor downstream ChatGPT visibility after the technical crawler-access layer.

Ansvisor's AI Search Intelligence Platform extends that measurement across prompts, answers, citations, competitors, sources, opportunities, and actions.