AI & Infrastructure
PerplexityBot / Perplexity-User

PerplexityBot / Perplexity-User

PerplexityBot is Perplexity’s search crawler, which finds and links websites in Perplexity answers. Perplexity-User is a separate fetcher that visits a page when a user’s question needs it and generally ignores robots.txt.
October 5, 2026
Cihan Geyik
Table of Content

Perplexity shows numbered source links next to its answers, so it can send real visits to the sites it reads. Two user agents decide whether it can read yours. PerplexityBot crawls the web ahead of time for Perplexity search. Perplexity-User fetches a page live when someone’s question needs it. They follow different rules, and treating them as one bot is the most common mistake in Perplexity crawler policy.

PerplexityBot follows your robots.txt. Perplexity-User generally doesn’t. Perplexity’s own documentation says the user-triggered fetcher generally ignores robots.txt, because a person asked for the page.

What is PerplexityBot?

PerplexityBot is Perplexity’s search crawler. Its documented job is to surface and link websites in Perplexity search results. Perplexity states that it is not used to crawl content for AI foundation models.

The full user agent string, as listed in Perplexity’s crawler documentation, is:

Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; PerplexityBot/1.0; +https://perplexity.ai/perplexitybot)

In firewall and log rules, match on the PerplexityBot token rather than the whole string. Version numbers in user agents can change.

What is Perplexity-User?

Perplexity-User supports user actions inside Perplexity. When someone asks a question that needs a specific page, it may visit that page and link it in the answer.

Perplexity says it is not used for web crawling or to collect training data. Its string follows the same pattern:

Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Perplexity-User/1.0; +https://perplexity.ai/perplexity-user)

PerplexityBot vs Perplexity-User

PerplexityBotPerplexity-User
RoleCrawls ahead of time for Perplexity searchFetches a page when a user’s question needs it
TriggerAutomaticA user request
robots.txtControls it; changes can take up to 24 hoursGenerally ignored, per Perplexity
Model trainingNot used for it, per PerplexityNot used for it, per Perplexity
Published IP listperplexitybot.jsonperplexity-user.json

How to allow or block PerplexityBot in robots.txt

Address the bot by its token in your robots.txt file. This keeps PerplexityBot out of one private section and leaves the rest open:

User-agent: PerplexityBot
Disallow: /internal/
Allow: /

To block it everywhere, use Disallow: / on its own. Perplexity says it can take up to 24 hours for its systems to reflect robots.txt changes.

Watch the fallback rule as well. Under RFC 9309, a crawler with no group of its own follows the User-agent: * group. A blanket Disallow: / under * blocks PerplexityBot too, unless you give it a named group.

Blocking PerplexityBot is a visibility decision. Perplexity says the bot doesn’t collect training data, so blocking it won’t keep you out of a model. It mainly lowers your chance of being surfaced and linked in Perplexity answers.

Does Perplexity-User respect robots.txt?

Generally, no. Perplexity’s documentation says that because a user requested the fetch, Perplexity-User generally ignores robots.txt rules.

A User-agent: Perplexity-User group can still record your preference. Don’t treat it as a lock. To keep content away from the fetcher, use real access controls: a login, firewall rules or bot management at your CDN.

There’s a trade-off. Blocking the fetcher also stops Perplexity from reading your page when a user asks about it directly. The answer then gets built from what other sites say about you.

How to verify Perplexity traffic in your logs

Anyone can put "PerplexityBot" in a user agent. Before you allow or block by name, match the request IP against Perplexity’s published lists:

https://www.perplexity.com/perplexitybot.json
https://www.perplexity.com/perplexity-user.json

If you run a web application firewall, Perplexity recommends matching both the user agent and these IP ranges. Refresh the rules from the JSON files regularly, since the ranges change.

A quick first pass on a server log looks like this:

grep -E "PerplexityBot|Perplexity-User" access.log | cut -d" " -f1 | sort | uniq -c | sort -rn

It lists each requesting IP with its hit count, ready to compare with the published ranges.

Verification has a second purpose. In August 2025, Cloudflare reported undeclared Perplexity crawling with a generic browser user agent after sites blocked its bots. Perplexity disputed the findings. Logs checked against published IPs let you judge your own site on evidence.

How do Perplexity’s bots compare with OpenAI’s and Anthropic’s?

ProviderSearch crawlerTraining crawlerUser-triggered fetcher
PerplexityPerplexityBotNone documentedPerplexity-User
OpenAIOAI-SearchBotGPTBotChatGPT-User
AnthropicClaude-SearchBotClaudeBotClaude-User

The pattern holds across providers: search, training and live fetching are separate decisions with separate tokens. For the other two, see our pages on OAI-SearchBot and ClaudeBot and Claude-SearchBot.

Does allowing PerplexityBot get you cited?

Access gets you in the door. Nothing more.

robots.txt allows the bot → Firewall lets it through → Page is crawled → Page is retrieved for a question → Page is cited

Every step after the second depends on relevance and on the other sources competing for the same question. A page can pass every technical check and still lose the citation to a clearer, more specific source. The only way to know whether Perplexity search cites you is to track real answers.

Should you allow PerplexityBot and Perplexity-User?

For most brands that want to be recommended, yes. Both agents feed answers that link back to their sources, and Perplexity says neither is used for model training.

  • SaaS and ecommerce sites usually allow both. Product, pricing and comparison pages are exactly what buyers ask Perplexity about.
  • Publishers with paid content can keep paid pages behind a login. That stops both agents there and leaves free articles open.
  • Sites worried about server load can rate-limit verified Perplexity IPs before reaching for a full block.
  • Pages that must stay private need authentication. No robots.txt rule protects them from a user-triggered fetch.

How to audit your Perplexity crawler setup

  1. Read robots.txt for PerplexityBot, Perplexity-User and the * group.
  2. Check CDN and firewall rules for both agents, including any one-click "block AI bots" setting.
  3. Search server logs for both user agents and confirm the IPs match the published lists.
  4. Look for 403, 429 and 5xx responses served to verified Perplexity IPs.
  5. Confirm core content is in the HTML, since most AI crawlers don’t run JavaScript.
  6. Track whether Perplexity actually cites your pages for the prompts you care about.

Steps two to five are the core of AI crawler accessibility. Our checklist on checking if AI crawlers can access your website covers them for every major bot.

How Ansvisor helps

Ansvisor’s Site Audit checks your robots.txt and AI bot access, so a stray rule doesn’t lock PerplexityBot out. The Perplexity visibility tracker then shows whether Perplexity names and cites you across your prompts. You also see which of your pages it cites and which competitors it picks instead.

Perplexity tracking is included from the Growth plan, next to ChatGPT, Gemini, Microsoft Copilot, Google AI Overviews and Google AI Mode. Start a 14-day free trial to see your Perplexity citations.

PerplexityBot, Perplexity-User, Perplexity Crawler, Perplexity User Agent, Perplexity Bot, Perplexity Search Crawler

Let AI Search Work for Your Growth

Each prompt you win brings more mentions, more citations and more buyers. Ansvisor keeps finding the next one.

AI Answers Mentions Citations Visibility Score AI Share of Voice AI Traffic Revenue
Win Back Search VisibilityTry Ansvisor 14-day Free →✓ Zero risk. Cancel anytime.

FAQ

Frequently asked questions.

What is PerplexityBot?

PerplexityBot is Perplexity’s search crawler. Perplexity says it is designed to surface and link websites in Perplexity search results and is not used to crawl content for AI foundation models. You control it with robots.txt.

What is Perplexity-User?

Perplexity-User is a fetcher that visits a page when a Perplexity user asks a question that needs it, then links the page in the answer. Perplexity says it is not used for web crawling or for training AI foundation models.

Does Perplexity-User respect robots.txt?

Generally not. Perplexity’s documentation says that because a user requested the fetch, Perplexity-User generally ignores robots.txt rules. To keep content away from it, use a login or firewall rules, knowing that this also stops Perplexity from reading the page for users.

How do I block or allow PerplexityBot?

Add a “User-agent: PerplexityBot” group to robots.txt with “Disallow: /” to block it or “Allow: /” to permit it. Perplexity says changes can take up to 24 hours. If you use a firewall, allow both the user agent and Perplexity’s published IP ranges.

Does allowing PerplexityBot guarantee citations in Perplexity?

No. Access only makes your pages eligible to be found. Whether Perplexity cites a page depends on the question and on the other sources available. Treat crawler access as a prerequisite and measure citations directly with a tracker.

Help us grow the AI Visibility Glossary

New terms are added regularly.

Help us improve the page or suggest a new term →
About the Author
Cihan Geyik

Cihan Geyik

Co-founder at Ansvisor

Cihan Geyik is the co-founder of Ansvisor, an open-source AI Visibility platform for AI Search. With more than 15 years of experience in digital marketing and growth, he writes about AI visibility, AI search, AEO, GEO, citations, and answer engines. He focuses on helping brands understand and improve their presence across ChatGPT, Gemini, Google AI Overviews, Bing AI Mode and other AI answer engines.

Google Preferred Sources
Get more Ansvisor insights in Google

Add Ansvisor as a preferred source and Google will show you more of our AI search, AEO and GEO guides.