---
title: Bot & Crawler Insights in Cloudflare Radar
description: Expanded visibility into crawler, referral, and bot activity with new Radar API endpoints
image: https://developers.cloudflare.com/changelog-preview.png
---

[Skip to content](#%5Ftop) 

# Changelog

New updates and improvements at Cloudflare.

[ Subscribe to RSS ](https://developers.cloudflare.com/changelog/rss/index.xml) [ View RSS feeds ](https://developers.cloudflare.com/fundamentals/new-features/available-rss-feeds/) 

![hero image](https://developers.cloudflare.com/_astro/hero.CVYJHPAd_26AMqX.svg) 

[ ← Back to all posts ](https://developers.cloudflare.com/changelog/) 

## Bot & Crawler Insights in Cloudflare Radar

Jul 01, 2025 

[ Radar ](https://developers.cloudflare.com/radar/) 

#### Web crawlers insights

[**Radar**](https://developers.cloudflare.com/radar/) now offers expanded insights into web crawlers, giving you greater visibility into aggregated trends in crawl and refer activity.

We have introduced the following endpoints:

* [/bots/crawlers/summary/{dimension}](https://developers.cloudflare.com/api/resources/radar/subresources/bots/subresources/web%5Fcrawlers/methods/summary/): Returns an overview of crawler HTTP request distributions across key dimensions.
* [/bots/crawlers/timeseries\_groups/{dimension}](https://developers.cloudflare.com/api/resources/radar/subresources/bots/subresources/web%5Fcrawlers/methods/timeseries%5Fgroups/): Provides time-series data on crawler request distributions across the same dimensions.

These endpoints allow analysis across the following dimensions:

* `user_agent`: Parsed data from the `User-Agent` header.
* `referer`: Parsed data from the `Referer` header.
* `crawl_refer_ratio`: Ratio of HTML page crawl requests to HTML page referrals by platform.

#### Broader bot insights

In addition to crawler-specific insights, Radar now provides a broader set of bot endpoints:

* [/bots/](https://developers.cloudflare.com/api/resources/radar/subresources/bots/): Lists all bots.
* [/bots/{bot\_slug}](https://developers.cloudflare.com/api/resources/radar/subresources/bots/methods/get/): Returns detailed metadata for a specific bot.
* [/bots/timeseries](https://developers.cloudflare.com/api/resources/radar/subresources/bots/methods/timeseries/): Time-series data for bot activity.
* [/bots/summary/{dimension}](https://developers.cloudflare.com/api/resources/radar/subresources/bots/methods/summary/): Returns an overview of bot HTTP request distributions across key dimensions.
* [/bots/timeseries\_groups/{dimension}](https://developers.cloudflare.com/api/resources/radar/subresources/bots/methods/timeseries%5Fgroups/): Provides time-series data on bot request distributions across the same dimensions.

These endpoints support filtering and breakdowns by:

* `bot`: Bot name.
* `bot_operator`: The organization or entity operating the bot.
* `bot_category`: Classification of bot type.

The previously available `verified_bots` endpoints have now been deprecated in favor of this set of bot insights APIs. While current data still focuses on verified bots, we plan to expand support for unverified bot traffic in the future.

Learn more about the new Radar bot and crawler insights in our [blog post ↗](https://blog.cloudflare.com/ai-search-crawl-refer-ratio-on-radar).