Loading page
Number of observed HTTP requests from documented AI crawler user agents in a site's connected server logs.
Crawler Occurrences measure requests attributed to documented AI user agents such as GPTBot, ClaudeBot, or PerplexityBot in the logs a site delivers. They show which page was requested, when, by which classified crawler, and which HTTP response group was returned. A high count is not evidence that a platform values, indexed, trained on, retrieved, or cited that page. Missing delivery, provider filters, quotas, and bounded page detail can reduce the visible count, so coverage metadata must be read with the metric.
Qwairy classifies observable AI user agents in connected server logs and consolidates them into retained daily and monthly snapshots. See exact observed totals, frequency, page groups, HTTP responses, and explicit coverage caveats without retaining raw traffic events.
Automated web client operated by an AI company for model training, AI search, or user-requested retrieval.
OpenAI's web crawler that collects public web content to train future GPT models; site owners control its access via robots.txt.
Anthropic's web crawler that collects public web content to train and improve Claude models; site owners control its access via robots.txt.
Page-level view combining observed crawler requests, AI referral sessions, search data, and citation signals when those sources are connected.