This tutorial describes how to get Cloudflare logs from Amazon S3 into Splunk using the . To learn how to use Logpush to send logs to AWS S3, refer to the . Alternatively, you can use to get logs to your directly and skip Task 1.
Before sending your Cloudflare log data to Splunk, make sure that you:
- Have an existing Splunk Enterprise or Cloud account
- Have a Cloudflare Enterprise account with Cloudflare Logs enabled
- Configure or
- Consult the for the Cloudflare App
Task 1 - Connect AWS S3 to Splunk
To start receiving Cloudflare log data, you need to connect AWS S3 to Splunk as follows:
Log in to your Splunk instance > Apps > Find More Apps.
Search for Splunk Add-on for Amazon Web Services.
Once installed, restart and reopen your Splunk instance.
In Configurations, click Create New Input > S3 Access Logs > Generic S3, and enter the following:
AWS Account: Enter the read-only AWS account you created for the Splunk instance.
Assume role: Optional
S3 bucket: From the drop down menu, select the S3 bucket containing the Cloudflare logs.
S3 Key Prefix: Leave empty.
Source Type: Enter _cloudflare:json_or if disabled, see Step 7 below.
Index: Enter _cloudflare._You can use an existing index or create a new one as described in Step 8 below.
If the field is inactive and you can’t update the default value aws:s3:accesslogs,_as shown in the screenshot above,_update the Source Type manually by going to Settings > Data Inputs > Select AWS S3 and open your current AWS S3 connection. Scroll down and select More Settings. Manually update field Source type to the value _cloudflare:json_and click Save.
Now, logs should be loading into Splunk. You can verify this under Splunk Add-on for AWS > Search. In the search box, type:
Next, select the desired time interval and and click Search.
If everything is configured correctly, you should be able to see Cloudflare logs as shown in the screenshot below.
Task 2 - Install and Configure the Cloudflare App for Splunk
- Login to your Splunk instance.
- Under Apps > Find More Apps search for Cloudflare App for Splunk.
- Click Install.
Once installed, you need to configure the application. To do this, a set up page is included with the application:
- Open the set up screen by clicking the Apps dropdown, then Manage Apps > Cloudflare Set Up.
- Enter the Index name where the Cloudflare JSON logs are stored. This value must be entered in the format index=index_name. By default the value is set to index=cloudflare.
- Optional: Enable . This option is disabled by default.
The Cloudflare App is now installed and the dashboards should be populating with data.
Post Installation Notes
You can change the Index Name after initial configuration by accessing the app Set up page by clicking on the Apps dropdown, then Manage Apps > Cloudflare App for Splunk > Set up.
Also, you can find the Index Name manually by visiting Settings > Advanced search > Search macros.
The Cloudflare App for Splunk comes with a custom Cloudflare Data Model which has an acceleration time frame of 1 day but is not accelerated by default. If you enable , we recommend that the Data Model is only accelerated for 1 or 7 days to ensure there are no adverse effects within your Splunk environment.
You can enable or disable acceleration after the initial configuration by accessing the app Set up page by clicking the Apps dropdown, then Manage Apps > Cloudflare Set Up.
Task 3 - View the Dashboards
You can analyze Cloudflare logs with the thirteen (13) dashboards listed below.
You can use filters within these dashboards to help narrow the analysis by date and time, device type, country, user agent, client IP, hostname, and more to further help with debugging and tracing.
About the Dashboards
The following dashboards outlined below are available as part of the Cloudflare App for Splunk.
Cloudflare - Snapshot
Cloudflare - Reliability
Summary and Detailed: Get insights on the availability of your websites and applications. Metrics include origin response error ratio, origin response status over time, percentage of 3xx/4xx/5xx errors over time, and more.
Cloudflare - Security
WAF: Get insights on threat identification and mitigation by our Web Application Firewall, including events like SQL injections, XSS, and more. Use this data to fine tune the firewall to target obvious threats and prevent false positives.
Cloudflare - Performance
Requests and Cache and Bandwidth: Identify and address performance issues and caching misconfigurations. Metrics include total vs. cached bandwidth, saved bandwidth, total requests, cache ratio, top uncached requests, and more.
Hostname, Content Type, Request Methods, Connection Type: Get insights into your most popular hostnames, most requested content types, breakdown of request methods, and connection type.
All dashboard have a set of filters that you can apply to the entire dashboard, as shown in the following example. Filters are applied across the entire dashboard.
You can use filters to drill down and examine the data at a granular level. Filters include client country, client device type, client IP, client request host, client request URI, client request user agent, edge response status, origin IP, and origin response status.
The default time interval is set to 24 hours. Note that for correct calculations filter will need to exclude Worker subrequests (WorkerSubrequest = false) and purge requests (ClientRequestMethod is not PURGE).
Time Range (EdgeStartTimestamp)
Client Device type
Client Request Host
Client Request URI
Client Request User Agent
Edge response status
Origin Response Status
Client Request Method
The Splunk Cloudflare App relies on data from the Cloudflare Enterprise Logs fields outlined below. Depending on which fields you have enabled, certain dashboards might not populate fully.
If that is the case, verify and test the Cloudflare App filters below each dashboard (these filters are the same across all dashboards). You can delete any filters that you don’t need, even if such filters include data fields already contained in your logs.
Also, you could compare the list of fields you are getting in Cloudflare Logs with the fields listed in Splunk > Settings > Data Model > Cloudflare.
The available fields are: