AI Crawler & Speed Gate
If robots.txt blocks AI search bots or mobile TTFB stalls, your site is filtered out before retrieval begins.
Requirement: Open AI crawler policy & LCP tuning
User-agent: OAI-SearchBot
Allow: /
User-agent: PerplexityBot
Allow: /