Crawlers
Tuesday, March 3, 2026 5:30:00 PM
Perplexity's robots and crawlers
Research into Perplexity's approach to scraping web content
Find out how we classify Perplexity's crawler scrape web content.
Friday, February 13, 2026 5:30:00 PM
Meta's robots and crawlers
Research into Meta's approach to scraping web content
Find out how we classify Meta's crawlers and AI content scrapers.