PerplexityBot / Perplexity-User

PerplexityBot and Perplexity-User are two web agents operated by Perplexity, but they serve different purposes. PerplexityBot is used to discover and surface websites in Perplexity search results, while Perplexity-User retrieves web pages in response to requests initiated by Perplexity users.

This distinction matters for AI Crawler Accessibility because automated search crawling and user-triggered retrieval have different behaviors and different relationships with robots.txt.

What Is PerplexityBot?

PerplexityBot is Perplexity's web crawler for search. Perplexity describes it as a crawler designed to surface and link websites in Perplexity search results.

This means PerplexityBot is relevant to whether public website content can be discovered and considered within Perplexity's search ecosystem.

Perplexity also states that PerplexityBot is not used to crawl content for AI foundation models.

What Is Perplexity-User?

Perplexity-User is a separate user-triggered fetcher. It may retrieve a web page when a Perplexity user's request requires access to that page.

Unlike PerplexityBot, Perplexity-User is not used for automatic web crawling. Perplexity also states that it is not used to collect content for training AI foundation models.

This makes Perplexity-User conceptually closer to an on-demand web fetcher than a conventional search indexing crawler.

PerplexityBot vs. Perplexity-User

PerplexityBot and Robots.txt

Website owners can define crawler-specific instructions for PerplexityBot in their robots.txt file.

A website that wants PerplexityBot to access its public content can use:

This explicitly communicates that PerplexityBot is permitted to crawl the public site under the applicable Robots Exclusion Protocol rules.

How to Block PerplexityBot

A publisher that does not want PerplexityBot crawling the site can specify:

This decision should be evaluated differently from blocking a crawler used exclusively for model training. PerplexityBot exists to help surface and link websites in Perplexity search results.

Does Perplexity-User Follow Robots.txt?

Perplexity documents Perplexity-User differently from its automated crawler.

Because a user initiates the fetch, Perplexity states that Perplexity-User generally ignores robots.txt rules.

A robots.txt directive such as the following should therefore not be treated as a reliable access-control mechanism:

The directive may express a publisher preference, but sensitive or private content should never depend on robots.txt for protection.

Why Does Perplexity-User Generally Ignore Robots.txt?

Perplexity distinguishes a user-requested page fetch from automatic web crawling.

When Perplexity-User accesses a page, the request occurs because a user asked Perplexity to retrieve information that requires that resource.

This behavior makes it particularly important to distinguish crawler policy from actual technical access control.

Can You Block Perplexity-User?

If a website must technically prevent Perplexity-User from retrieving content, robots.txt should not be relied on as the enforcement layer.

Actual access can instead be controlled through mechanisms such as:

• authentication;
• authorization;
• server-level access controls;
• CDN rules;
• web application firewall rules;
• verified bot-management policies; and
• other network or application security controls.

However, blocking user-triggered retrieval also means Perplexity may be unable to retrieve that content when a user specifically requests it.

PerplexityBot Is Not a Training Crawler

One of the most important distinctions for publishers is that Perplexity does not describe PerplexityBot as a crawler for foundation-model training.

Its documented purpose is to surface and link websites in Perplexity search results.

Treating every AI-related bot as a training crawler can therefore lead to incorrect robots.txt decisions.

PerplexityBot and AI Search Indexing

PerplexityBot is relevant to AI Search Indexing because its search crawling helps Perplexity discover content that can be surfaced and linked in its search experience.

However, crawler access should not be treated as proof that a page will appear for any particular query.

Discovery, indexing, retrieval, source selection, and answer generation are distinct stages.

PerplexityBot and Perplexity Citations

Perplexity is particularly relevant to AI Citations because its search experience surfaces links to sources used in answers.

PerplexityBot accessibility can help make a site available to Perplexity's search systems, but it does not guarantee that the site will be selected as a source for a particular prompt.

Does Blocking PerplexityBot Affect Perplexity Visibility?

Blocking a search crawler can reduce the crawler's ability to discover and retrieve content through its normal automated crawling process.

Because PerplexityBot is specifically designed to surface and link websites in Perplexity search results, publishers seeking AI Visibility in Perplexity should evaluate the consequences before intentionally restricting it.

This is fundamentally different from blocking a training-only crawler, where the decision may not directly govern a search discovery channel.

Does Allowing PerplexityBot Guarantee Visibility?

No. Allowing the crawler removes one potential technical barrier, but visibility depends on downstream search and answer systems.

A page may be crawlable and still not be selected for a particular user query.

Actual Perplexity visibility therefore needs to be measured separately from crawler configuration.

PerplexityBot and Crawler Access

A robots.txt configuration is only one layer of Crawler Access.

Even when PerplexityBot is permitted by robots.txt, other systems can prevent successful retrieval.

These can include:

• CDN security policies;
• web application firewalls;
• bot-management systems;
• CAPTCHA challenges;
• authentication requirements;
• rate limiting;
• IP restrictions;
• geographic restrictions;
• HTTP 403 responses;
• HTTP 429 responses; and
• server errors.

How to Verify PerplexityBot Traffic

User-agent strings can be spoofed. Seeing the word PerplexityBot in a server log does not by itself prove that the request came from Perplexity.

Perplexity publishes IP ranges for PerplexityBot that technical teams can use as an additional verification signal:

PerplexityBot IP ranges

Because crawler infrastructure can change, teams should use the current published list rather than permanently copying individual IP addresses into documentation.

How to Verify Perplexity-User Traffic

Perplexity also publishes separate IP information for Perplexity-User:

Perplexity-User IP ranges

This separation reinforces that PerplexityBot and Perplexity-User are distinct agents with different roles.

PerplexityBot vs. OAI-SearchBot

PerplexityBot and OAI-SearchBot have broadly comparable search-oriented roles within their respective platforms.

PerplexityBot helps Perplexity surface and link websites in search results, while OAI-SearchBot is OpenAI's crawler for surfacing websites in ChatGPT search features.

Neither should automatically be categorized as a foundation-model training crawler.

PerplexityBot vs. Claude-SearchBot

PerplexityBot and Claude-SearchBot are also both search-oriented crawlers.

Their exact architectures and downstream systems differ, but both demonstrate why AI crawler governance should distinguish search crawlers from training crawlers.

Perplexity-User vs. Other User-Triggered Agents

Perplexity-User belongs to a growing category of web agents that retrieve content in response to a human request rather than continuously crawling the web.

Similar user-triggered agents may exist across other AI platforms, but their robots.txt behavior should not be assumed to be identical.

Each provider's current documentation should be reviewed independently when defining crawler and access policies.

PerplexityBot and AI Crawler Accessibility

For Perplexity Search, crawler accessibility can be viewed as a technical eligibility layer.

The important distinction is that technical accessibility creates an opportunity for discovery. It does not create a guaranteed citation or visibility outcome.

Should Websites Allow PerplexityBot?

There is no universal crawler policy for every organization, but the purpose of PerplexityBot should be considered before blocking it.

For organizations that want their public content discoverable in Perplexity Search, allowing the search crawler is generally aligned with that objective.

Organizations with different content-access requirements may choose more restrictive policies for specific paths or entire sites.

The key is to recognize that blocking PerplexityBot is not simply an opt-out from foundation-model training.

Common Perplexity Crawler Mistakes

• treating PerplexityBot and Perplexity-User as the same agent;
• describing PerplexityBot as a foundation-model training crawler;
• assuming Perplexity-User is an automatic indexing crawler;
• relying on robots.txt as a security mechanism for Perplexity-User;
• blocking PerplexityBot without considering the potential search visibility impact;
• assuming allowing PerplexityBot guarantees citations;
• trusting a user-agent string without considering request verification;
• allowing a crawler in robots.txt while blocking it through infrastructure;
• confusing crawler access with source selection; and
• treating every AI crawler as if it serves the same purpose.

How to Audit PerplexityBot and Perplexity-User

1. Review PerplexityBot rules. Determine whether robots.txt permits access to public content intended for Perplexity Search.
2. Check important paths. Review product pages, articles, documentation, category pages, and other strategically important URLs.
3. Separate PerplexityBot from Perplexity-User. Do not apply assumptions about automatic crawling to user-triggered retrieval.
4. Review actual access controls. Sensitive content should use authentication or other enforceable security mechanisms rather than robots.txt.
5. Inspect infrastructure. Check CDN, WAF, bot-management, rate limits, and HTTP responses.
6. Verify crawler requests. Use Perplexity's current published IP information alongside user-agent data where appropriate.
7. Measure Perplexity visibility. Do not infer citations or visibility from crawler configuration alone.

Perplexity Crawlers and LLM SEO

Understanding PerplexityBot and Perplexity-User is one technical component of LLM SEO, AEO, and Generative Engine Optimization (GEO).

The objective is not simply to allow as many AI bots as possible. It is to understand what each agent does, decide which forms of access support the organization's goals, and then measure the downstream results.

From PerplexityBot Access to Perplexity Visibility

Crawler accessibility is an input. Perplexity visibility and citations are measurable outcomes.

Ansvisor's Perplexity Visibility Tracker helps teams monitor how brands appear across relevant Perplexity prompts and answers.

Ansvisor's Citation Monitoring can then be used to analyze which domains and URLs actually appear as sources.

The broader Ansvisor AI Search Intelligence Platform connects visibility, prompts, citations, competitors, opportunities, and actions across the AI Search workflow.