Get started (28)

Last updated Apr 23, 2026 This guide instructs you on how to:

  • View AI crawlers that are interacting with pages in your domain (a Cloudflare zone).
  • Use AI Crawl Control to block individual crawlers from accessing your content.

Prerequisites

  1. Sign up for a Cloudflare account ↗︎.
  2. Connect your domain to Cloudflare.
  3. Make sure your domain is proxying traffic through Cloudflare.

1. Monitor AI crawler activity at a glance

  1. Go to AI Crawl Control. Go to AI Crawl Control ↗
  2. Review the snapshot of your AI crawler activity in the Overview tab.
  3. Use the filters to view activity by specific date ranges, crawlers, operators, hostnames, or paths.

2. Block specific AI crawlers

To block specific AI crawlers:

  1. Go to AI Crawl Control. Go to AI Crawl Control ↗
  2. Review which AI crawlers are accessing your domain in the Crawlers tab.
  3. In the Action column, select Block.

Quality of AI crawler detection

On the free plan, AI Crawl Control identifies AI crawlers based on their user agent strings ↗︎. This enables AI Crawl Control to detect well-known, self-identifying AI crawlers.

Upgrade your plan to enable a more thorough detection using Cloudflare's Bot Management detection ID field.

To block specific AI crawlers:

  1. Go to AI Crawl Control. Go to AI Crawl Control ↗
  2. Review which AI crawlers are accessing your domain in the Crawlers tab.
  3. In the Action column, select Block.

For more information, refer to Manage AI crawlers.

You can also create more complex rules when taking action on AI crawlers, using Cloudflare WAF. For more information on creating more specific rules, refer to Create a custom rule in the dashboard.

3. Explore detailed metrics

For more detailed analytics, use the Metrics tab.

  1. Go to AI Crawl Control. Go to AI Crawl Control ↗
  2. Review detailed breakdowns by date range, crawler, operator, status code, hostname, or path in the Metrics tab.

Note that on free plans, the Metrics tab only displays metrics for the past 24 hours.

Plan comparison

All plans Enterprise plans with Bot Management
AI crawler detection via user agent strings ↗︎ Advanced AI crawler detection via Bot Management detection ID
Maximum 24-hour analytics window Configurable analytics timeframes
Allow/block controls Allow/block controls, and the ability to charge AI crawlers using pay per crawl

Next steps

Refer to the following related resources:

Was this helpful?

YesNo

On this page

Get started (28) - 图1Docs

{"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/ai-crawl-control/get-started/#page","headline":"Get started","description":"Learn how to set up AI Crawl Control.","url":"https://developers.cloudflare.com/ai-crawl-control/get-started/","inLanguage":"en","image":"https://developers.cloudflare.com/ai-crawl-control/get-started/og.png?v=328d2d25df6d422f","dateModified":"2026-04-23","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}}

description: View metrics on AI crawler interactions with your site. title: Analyze AI traffic image: https://developers.cloudflare.com/ai-crawl-control/features/analyze-ai-traffic/og.png?v=9ebf5a8c17c79d1e

Skip to content

Documentation Index
Fetch the complete documentation index at: https://developers.cloudflare.com/ai-crawl-control/llms.txt
Use this file to discover all available pages before exploring further.