Enable S3-compatible endpoints
Cloudflare Logpush now supports S3-compatible destinations in an API-only beta, including:
- Digital Ocean Spaces
- Backblaze B2
- Alibaba Cloud OSS
- JD Cloud Object Storage Service
- Oracle Cloud Object Storage
- Linode Object Storage
- On-premise Ceph Object Gateway
For more information about Logpush and the current production APIs, see the Cloudflare Logpush documentation.
To set up S3-compatible endpoints:
- Create a job with the appropriate endpoint URL and authentication parameters
- Then, enable the job to begin pushing logs
See below for detailed instructions.
1. Create a job
To create a job, make a POST
request to the Logpush jobs endpoint with the following fields:
name
(optional) - Use your domain name as the job name.destination_conf
- A log destination consisting of an endpoint name, bucket name, bucket path, region, access-key-id, and secret-access-key in the following string format:
"s3://<BUCKET-NAME>/<BUCKET-PATH>?region=<REGION>&access-key-id=<ACCESS-KEY-ID>&secret-access-key=<SECRET-ACCESS-KEY>&endpoint=<ENDPOINT-URL>"
dataset
- the category of logs you want to receive; eitherhttp_requests
(default),spectrum_events
orfirewall_events
logpull_options
(optional) - To configure fields, sample rate, and timestamp format, see Logpush API options
Example request using cURL:
curl -s -X POST \https://api.cloudflare.com/client/v4/zones/<ZONE_ID>/logpush/jobs \-d '{"name":"<DOMAIN_NAME>","destination_conf":"s3://<BUCKET-NAME>/<BUCKET-PATH>?region=<REGION>&access-key-id=<ACCESS-KEY-ID>&secret-access-key=<SECRET-ACCESS-KEY>&endpoint=<ENDPOINT-URL>", "logpull_options":"fields=RayID,EdgeStartTimestamp×tamps=rfc3339", "dataset":"http_requests"}' | jq .
Response:
{ "errors": [], "messages": [], "result": { "id": 100, "dataset": "http_requests", "enabled": false, "name": "<DOMAIN_NAME>", "logpull_options": "fields=RayID,EdgeStartTimestamp×tamps=rfc3339", "destination_conf": "s3://<BUCKET-NAME>/<BUCKET-PATH>?region=<REGION>&access-key-id=<ACCESS-KEY-ID>&secret-access-key=<SECRET-ACCESS-KEY>&endpoint=<ENDPOINT-URL>", "last_complete": null, "last_error": null, "error_message": null }, "success": true}
2. Enable (update) a job
To enable a job, make a PUT
request to the Logpush jobs endpoint. You’ll use the job ID returned from the previous step in the URL, and send {"enabled": true}
in the request body.
Example request using cURL:
curl -s -X PUT \https://api.cloudflare.com/client/v4/zones/<ZONE_ID>/logpush/jobs/100 -d'{"enabled":true}' | jq .
Response:
{ "errors": [], "messages": [], "result": { "id": 100, "dataset": "http_requests", "enabled": true, "name": "<DOMAIN_NAME>", "logpull_options": "fields=RayID,EdgeStartTimestamp×tamps=rfc3339", "destination_conf": "s3://<BUCKET-NAME>/<BUCKET-PATH>?region=<REGION>&access-key-id=<ACCESS-KEY-ID>&secret-access-key=<SECRET-ACCESS-KEY>&endpoint=<ENDPOINT-URL>", "last_complete": null, "last_error": null, "error_message": null }, "success": true}