Skip to content
Logs
Visit Logs on GitHub
Set theme to dark (⇧+D)

Bot Management Dashboard

This tutorial explains how to activate Cloudflare Bot Management and view related dashboards in the Elastic, Google Data Studio, Looker, Splunk, and Sumo Logic analytics platforms.

Overview

Bot Management is a Cloudflare Enterprise feature. To enable it, contact your Cloudflare Account Team.

If you haven’t used Cloudflare Logs before, visit our Logs documentation for more details. Contact your Cloudflare Customer Account Team to enable logs for your account.

Prerequisites

Before getting started, make sure that you:

Task 1 - Configure Cloudflare to Detect Bot Traffic

Before proceeding, make sure that you’ve enabled Cloudflare Bot Management for your Enterprise account. To start:

  1. In the Cloudflare dashboard, create a Firewall rule. Learn more.
  2. For Rule name, enter Bot management - Generic.
  3. Under Expression Preview, click Edit Expression, then copy and paste the following expression: (bot score lt 30 and not verified_bot).
    • This rule only selects requests with a bot score less than 30 and excludes good bots. Requests with a score under 30 are considered bad bots. Your Firewall rule can be as granular as required. For example, applying conditions only for a specific URL, like login or sign-up pages.
  4. Choose an action: Log or Challenge (Captcha).
    • Cloudflare recommends starting with the action of Log and running it for several days in order to identify which requests fall under the rule above to check if any false positives are registered. You can refine the rule expression further, based on your findings. Once you complete testing, switch the rule action to Challenge (Captcha).
  5. Click Save.

Create Cloudflare Firewall rule to identify bad bots

Task 2 - View Bot Traffic Dashboards

Viewing dashboards for existing analytics integrations

The Cloudflare Bot Management Dashboard is already available in Elastic, Google Data Studio, Looker, Splunk, and Sumo Logic. If you use any of those platforms, you do not need to do anything else to view these existing dashboards.

See the Analytics Integrations and Analyze log data with Google Cloud sections for details.

Viewing dashboards for other platforms

If you use a platform that is not part of the integrations mentioned above, you will need to design your own dashboards.

The following sub-sections describe how to identify bot requests and perform the correct calculations using data from your Cloudflare logs.

Identify bot requests

Combining the following two fields reveals if the request is a bot request:

EdgePathingSrc = 'filterBasedFirewall' AND EdgePathingStatus = 'captchaNew'

Identify valid requests

Valid requests are those for which the user can solve the CAPTCHA. In this case, EdgePathingStatus = 'captchaSucc'.

The following combination of fields reveals which CAPTCHAs were solved:

EdgePathingSrc = 'filterBasedFirewall' AND EdgePathingStatus = 'captchaSucc'

Calculate number of bad bots

To start, make sure to exclude solved CAPTCHAs from calculations:

Bad Bots = SUM(EdgePathingSrc = 'filterBasedFirewall' AND EdgePathingStatus = 'captchaNew') - SUM(EdgePathingSrc = 'filterBasedFirewall' AND EdgePathingStatus = 'captchaSucc' )

If you see a significant increase in CAPTCHA conversion rate, contact your Cloudflare Account Team for further investigation.

Understand the Bot Management Dashboard

The Bot Management Dashboard consists of three columns to help you better understand and monitor traffic behavior. You can also analyze the ratio between all traffic and bot traffic.

The columns are:

  1. Global Traffic - Shows all requests

  2. Solved CAPTCHAs - Shows requests with solved CAPTCHAs

  3. Bad Bots - Shows confirmed bad bot requests, which exclude solved CAPTCHAs

Example dashboards

The following images demonstrate some of the dashboards available.

Elastic (Learn more) Cloudflare Bot Management in Elastic

Google Data Studio (Learn more) Cloudflare Bot Management in Google Data Studio page 1 Cloudflare Bot Management in Google Data Studio page 2

Graylog (Learn more) Cloudflare Bot Management in Graylog

Looker (Learn more) Cloudflare Bot Management in Looker

Splunk (Learn more) Cloudflare Bot Management in Splunk page 1 Cloudflare Bot Management in Splunk page 2

Sumo Logic (Learn more) Cloudflare Bot Management in Sumo Logic