Perplexity is using stealth, undeclared crawlers to evade website no-crawl directives
2025-08-04
Perplexity is repeatedly modifying their user agent and changing IPs and ASNs to hide their crawling activity, in direct conflict with explicit no-crawl preferences expressed by websites....
