Why does Crawl Insights show no data or gaps?

Gaps in Crawl Insights usually come from a usage limit or from the connection itself, not from bots stopping their visits.

Your monthly bot visit limit was reached

Crawl Insights has a monthly limit of bot visits per company, not per project:

• Starter: 4M, Pro: 10M, Advanced: 25M, Enterprise: custom
• Essential: 10M, Growth: 25M, Scale: 45M, Comprehensive: custom

Once the limit is reached, no new data is stored for the rest of that month. Storage resumes the next month.

Cloudflare Workers free plan

The Workers Free plan allows 100,000 requests per day, and every request to your site counts, not only AI bots. Once the limit is hit, data collection pauses for the rest of the day. By default Cloudflare then shows visitors a 1027 error page. To avoid this, set the route mode to Fail Open in your Cloudflare dashboard, or upgrade to a Workers Paid plan.

You changed your hosting or CDN

To switch to a new provider:

1. Go to Crawl Insights → Settings.
2. Disconnect the current source in the danger zone.
3. Connect the new provider.

Disconnecting does not delete your historical data.

Please note: only requests from known AI crawlers and agents are stored, so other traffic will never appear.

More detail: https://docs.peec.ai/crawl-insights