
For its first two years, Claude answered only from what it learned in training. Anthropic added web search in March 2025 and opened it to every plan worldwide that May. Claude search matters to brands for a plain reason. When a buyer asks Claude for a vendor shortlist, the cited pages shape the answer, and a site Claude can’t read can’t be one of them.
Anthropic’s help center says Claude calls a search tool when a topic benefits from current information. Every response that uses it includes citations, so readers can check the sources.
With web search on, Claude can also fetch a specific URL you paste. Research, on paid plans, runs many searches for a single question and depends on web search being enabled. In workspaces on Team and Enterprise plans, an owner switches web search on for everyone first.
Anthropic’s crawler documentation lists three bots and what blocking each one does:
| Bot | Purpose | If you block it |
|---|---|---|
| Claude-SearchBot | Indexes content to improve search results | Your visibility and accuracy in search answers may drop |
| Claude-User | Fetches pages when a user’s question needs them | You may be less visible in user-directed web search |
| ClaudeBot | Collects content that may be used for training | It signals that your future content should be excluded from training |
Anthropic says its bots honor robots.txt, support Crawl-delay and won’t try to bypass CAPTCHAs. Rules apply per subdomain, so each one needs its own file. Our page on ClaudeBot and Claude-SearchBot covers the setup in detail.
Developers add the same capability to their own products through the web search tool in the Claude API. Claude decides when to search, the API runs the searches and the reply cites its sources. Citations are always on for this tool.
Builders can restrict results to an allowed list of domains or exclude a blocked list, and cap the number of searches per request. Your page can therefore be cited inside Claude’s own apps and inside third-party products built on the API.
All three search the web and cite sources, but each reads the web through its own pipes. ChatGPT search depends on access for OAI-SearchBot. Gemini grounds answers in Google Search, as our Gemini search page explains. Claude’s index depends in part on Claude-SearchBot.
So one page can be cited by one assistant and missing from the other two. Track each platform on its own before you draw conclusions.
Not every Claude reply uses search. For stable topics, Claude answers from its training data, which has a cutoff date and shows no sources.
That gives you two routes into an answer. Search can cite you today, as long as Claude can fetch your pages. Training shapes what Claude says about your category when it doesn’t search. Blocking ClaudeBot signals that your future content should stay out of that data.
A site that blocks ClaudeBot but allows Claude-SearchBot keeps the first route and gives up the second. That can be a sound choice. Just make it on purpose.
These steps overlap heavily with other engines. Our guide on increasing AI mentions and citations goes further.
Ansvisor tracks Claude on Enterprise plans. The Claude AI visibility tracker runs your prompts in Claude on a schedule and records which brands it names and which URLs it cites. You see Claude next to ChatGPT, Gemini, Perplexity, Copilot and both Google AI surfaces. Site Audit also checks that robots.txt doesn’t lock Claude-SearchBot out.
Each prompt you win brings more mentions, more citations and more buyers. Ansvisor keeps finding the next one.
Claude search is the web search capability built into Anthropic’s Claude. When a question benefits from current information, Claude calls a search tool and reads the results. It replies with citations, so you can check each source yourself.
Yes. Web search has been available on all Claude plans worldwide since May 2025, and with it on, Claude can also fetch URLs you share. In the newer Claude app experience, Claude searches on its own when it helps. Team and Enterprise owners enable it for the workspace first.
Yes. Anthropic says every web search response includes citations. In the developer API, citations are always on for the web search tool. Each one carries the source URL, the page title and a short excerpt of the cited text.
Allow Claude-SearchBot and Claude-User in robots.txt and at your firewall. Keep core content in plain HTML and publish pages that answer specific questions with concrete facts. Mentions on the third-party pages Claude already cites for your topic also help.
Yes, by running a fixed prompt set in Claude on a schedule and recording mentions and citations. Ansvisor offers Claude visibility tracking on Enterprise plans, alongside ChatGPT, Gemini, Perplexity, Copilot and both Google AI surfaces.
See how AI answers describe your brand, find out why competitors get picked, and close the gaps with AI agents that work alongside your team.
14-day free trial or 50% off your first month with code SAVE50.
Continue exploring key AI visibility concepts.
Measure and improve how often your brand appears in AI-generated answers.
Learn more →Strategies for increasing visibility in answer engines and AI summaries.
Learn more →Optimizing content for AI-powered discovery experiences.
Learn more →Understand how OpenAI retrieves and synthesizes information.
Learn more →AI-generated summaries that appear directly in Google Search.
Learn more →Explore how Perplexity cites and presents sources.
Learn more →References and sources used by AI systems to support answers.
Learn more →Measure the quality and influence of cited sources.
Learn more →How easily AI systems can discover and reuse your content.
Learn more →New terms are added regularly.
Help us improve the page or suggest a new term →
Co-founder at Ansvisor
Cihan Geyik is the co-founder of Ansvisor, an open-source AI Visibility platform for AI Search. With more than 15 years of experience in digital marketing and growth, he writes about AI visibility, AI search, AEO, GEO, citations, and answer engines. He focuses on helping brands understand and improve their presence across ChatGPT, Gemini, Google AI Overviews, Bing AI Mode and other AI answer engines.
Add Ansvisor as a preferred source and Google will show you more of our AI search, AEO and GEO guides.
© 2026 Ansvisor. All rights reserved. Ansvisor is an open-source AI Search Execution Platform for AI Visibility. Ansvisor is a brand owned and operated by Empler AI Inc.
The open-source AI Search Execution Platform. Track how AI answers talk about your brand and act on it.
Open source · MIT license© 2026 Ansvisor, a brand of Empler AI Inc. · 1401 PE Ave. Unit 105, Wilmington, DE 19806, United States
Cookie preferences
We use cookies to keep the site working and, with your OK, to measure traffic and improve our marketing. Read our Cookies & Tracking policy.