Perplexity is using stealth, undeclared crawlers to evade website no-crawl directives
blog.cloudflare.com
Perplexity is repeatedly modifying their user agent and changing IPs and ASNs to hide their crawling activity, in direct conflict with explicit no-crawl preferences expressed by websites.
  • Ptsf@lemmy.worldEnglish
    6·
    26 days ago

    Or just under the unauthorized access provision of the computer fraud and abuse act since they’re directly evading access controls.