Changelog

Each Basin Pipelines stream can now ingest up to 1 GB/s, increased from 5 MB/s.

The higher per-stream limit gives high-volume application events, telemetry, and logs more room to grow without splitting ingestion across streams solely to stay within the previous limit.

For the full list of stream, sink, and pipeline limits, refer to Basin Pipelines limits.

Basin, formerly the Cloudflare Data Platform, is now generally available. Basin brings an end-to-end analytics platform to the Developer Platform, enabling you to collect data from a variety of sources, such as apps, infrastructure, devices, and other Cloudflare services, then query it to answer analytical questions.

Basin Pipelines, formerly Cloudflare Pipelines, ingests events from Workers, HTTP endpoints, and Cloudflare Logpush. It transforms events with SQL and ingests them into Iceberg tables or files on R2. With Basin Pipelines you can:

• Ingest application and device events through HTTP endpoints or Workers bindings.
• Filter and reshape Cloudflare logs before storing them as Iceberg tables, Parquet, or JSON.
• Catch schema mismatches with typed bindings and investigate dropped events in the dashboard.

Basin Catalog, formerly R2 Data Catalog, manages and automatically maintains Apache Iceberg tables ↗︎ to keep them fast, cost-efficient, and accessible to any compatible query engine. With Basin Catalog you can:

• Connect DuckDB, Spark, Snowflake, or PyIceberg to the same tables.
• Maintain growing tables with compaction, snapshot expiration, and manifest optimization.
• Share analytical data across tools and clouds without paying egress fees.

Basin SQL, formerly R2 SQL, is a serverless, distributed SQL engine for querying large Apache Iceberg tables in Basin Catalog without managing or scaling compute. With Basin SQL you can:

• Summarize and rank data with standard and approximate aggregates, grouping sets, and window functions.
• Combine and inspect datasets with joins, subqueries, common table expressions, set operations, schema discovery, and EXPLAIN.
• Transform strings, timestamps, JSON, and complex values with more than 190 functions.

To get started with creating an end-to-end data pipeline, run:

npx wrangler basin pipelines setup

yarn wrangler basin pipelines setup

pnpm wrangler basin pipelines setup

Or get started by referring to the Basin getting started guide.

R2 Data Catalog now provides table-level maintenance visibility and manual compaction queueing in the Cloudflare dashboard. These updates make it easier to understand when maintenance is eligible to run, inspect completed operations, and request maintenance without leaving the table view.

To view table maintenance details:

The updated dashboard includes:

• Maintenance tab — View compaction and snapshot expiration settings, schedules, and next eligibility alongside the table's Schema and Metadata tabs.
• Recent runs — Review a paginated audit log with job status, duration, and expandable details for manifest rewrites, compaction, and snapshot expiration. Expanded rows include operation metrics for each maintenance operation.
• Manual queueing — Select Queue maintenance to request compaction during normal scheduler polling. The dashboard checks permissions and explains when another maintenance job conflicts with the request or the daily accepted-request limit has been reached.
• Updated catalog layout — Find catalog metrics in the Metrics tab, use the renamed Explorer tab to browse data, and switch between table details using tabs instead of a scroll-to-section sidebar.
• Improved schema browser — For accounts with the schema browser enabled, select a namespace to open its tables in the right pane while also expanding the namespace tree. The tree can now be collapsed to provide more space for table details.

For more information about compaction and snapshot expiration, refer to Table maintenance.

Billing is now enabled for R2 Data Catalog on non-enterprise accounts. R2 Data Catalog usage beyond the included free tier will appear on your next invoice.

R2 Data Catalog charges based on two dimensions, in addition to standard R2 storage and operations:

• Catalog operations: $9.00 / million operations for metadata requests such as creating tables, reading table metadata, and updating table properties.
• Compaction: $0.005 / GB processed and $2.00 / million objects processed. These charges only apply when automatic compaction is turned on for a table.

Each dimension includes a monthly free tier: 1 million catalog operations, 10 GB of compaction data processed, and 1 million compaction objects processed.

For example, a single Iceberg table with 50 GB of data, 500,000 catalog operations per month, and compaction turned on that processes 20 GB across 200,000 files would be billed as follows:

Standard R2 storage charges ($0.015 / GB-month) apply separately for the 50 GB of data stored.

For full pricing details and billing examples, refer to R2 Data Catalog pricing.

Billing is now enabled for Cloudflare Pipelines on non-enterprise accounts. Pipelines usage beyond the included free tier will appear on your next invoice.

Pipelines charges based on two usage dimensions. Ingress into a Pipeline stream remains free regardless of volume:

• SQL transforms: $0.04 / GB for stateless transforms (filter, reshape, unnest, cast, compute).
• Sinks (egress): $0.03 / GB for JSON output, $0.06 / GB for Parquet or Iceberg output.

Workers Paid plans include 50 GB / month for both SQL transforms and sinks. Standard R2 storage and operations charges apply for data written to R2 buckets, and R2 Data Catalog charges apply when writing to Iceberg tables.

For example, a pipeline that ingests 500 GB of event data per month, uses a SQL transform to filter and reshape it, and writes 300 GB to an R2 Data Catalog Iceberg table would be billed as follows:

For full pricing details and billing examples, refer to Pipelines pricing.

Billing is now enabled for R2 SQL on non-enterprise accounts. R2 SQL usage beyond the included free tier will appear on your next invoice.

R2 SQL charges based on a single dimension:

• Data scanned: $0.0025 / GB ($2.50 / TB) of compressed data read from R2 to execute your query.

All plans include 10 GB of data scanned per month. Each query is billed for a minimum of 10 MB of data scanned. R2 SQL pricing is additive to standard R2 storage and operations and R2 Data Catalog charges. R2 does not charge for egress, so there is no additional data transfer cost.

For example, a user who stores 500 GB of Parquet data in R2 Data Catalog and runs queries that scan a total of 50 GB of compressed data during the month would be billed as follows:

For full pricing details and billing examples, refer to R2 SQL pricing.

R2 Data Catalog now accepts read-only API tokens, so query engines and clients that only read data no longer need a read-write token. Previously, every catalog operation required an Admin Read & Write token, which granted read-only clients more access than they needed.

You can now authenticate your Iceberg engine based on your workload:

• Read-only operations (such as listing namespaces, loading tables, and querying data) work with an Admin Read only token (R2 Data Catalog read and R2 storage read).
• Write operations (such as creating or dropping tables and committing transactions) continue to require an Admin Read & Write token.

This lets you follow the principle of least privilege — for example, using a read-write token for the pipeline that writes to your tables and read-only tokens for engines like R2 SQL, DuckDB, or PyIceberg that query them.

Note that credentials vended by the catalog inherit the R2 storage permissions of the token used to authenticate. To ensure read-only access to your underlying data, scope the R2 storage permission to read-only as well.

For details on choosing and creating the right token, refer to Authenticate your Iceberg engine.

R2 Data Catalog, a managed Apache Iceberg ↗︎ catalog built into R2, now automatically optimizes manifest files as part of compaction.

Manifest files track the data files that make up an Iceberg table. As a table accumulates many small or fragmented manifests, query engines must read more metadata during query planning, which slows down queries even before any data is scanned.

When compaction runs, R2 Data Catalog now rewrites and clusters manifest files by partition as a best-effort pre-step. This consolidates fragmented manifests, reduces the number of manifests a query engine must open, and lowers metadata I/O overhead. Tables that are already well-clustered are skipped, so the operation only runs when it provides a benefit.

This happens automatically for tables with compaction enabled — no configuration changes are required.

For more information, refer to Table maintenance.

You can now query your R2 Data Catalog tables with R2 SQL directly from the Cloudflare dashboard, without installing a CLI or wiring up a client. This makes it easy to explore your Apache Iceberg ↗︎ data, validate queries, and inspect results in one place.

To get started, go to R2 Data Catalog ↗︎ in the Cloudflare dashboard and select Query data to launch the built-in SQL editor. From there you can:

• Write and run queries interactively — Iterate on R2 SQL directly in the browser with syntax highlighting and autocomplete, instead of re-running commands through Wrangler or the REST API.
• Explore your data — Explore your namespaces and tables alongside the editor so you can discover what's queryable without leaving the page or using other tools.
• Understand results and performance — View result sets with per-query statistics, export them, and get helpful EXPLAIN outputs to see exactly how a query runs.

R2 Data Catalog is a managed Apache Iceberg ↗︎ catalog built directly into your R2 bucket. Iceberg tracks your data through a tree of metadata files, so every insert, update, and delete must go through a catalog transaction. Manually adding, modifying, or deleting objects outside the catalog can leave pointers referencing files that no longer exist, corrupting the table into an inconsistent state that is difficult to recover from.

To help prevent this, the R2 dashboard and Wrangler now warn you when you attempt a manual delete operation on a Data Catalog-enabled bucket.

When you try to delete objects from a bucket that has R2 Data Catalog enabled, the dashboard displays a warning explaining that the operation could leave the catalog in an invalid state, with a link to the documentation for deleting data correctly. You can cancel the operation or choose to proceed anyway.

Wrangler now checks whether a bucket is Data Catalog-enabled before running a delete and warns you before continuing:

Data Catalog is enabled for this bucket. 
Proceeding may leave the data catalog in an invalid state. Continue?

To learn how to safely manage and delete data in your tables, refer to the R2 Data Catalog documentation.

R2 SQL now supports window functions, SELECT DISTINCT, set operations, and additional aggregates, making it easier to write analytical queries without preprocessing your data elsewhere.

R2 SQL is Cloudflare's serverless, distributed SQL engine for querying Apache Iceberg ↗︎ tables stored in R2 Data Catalog.

• Window functions — ROW_NUMBER, RANK, DENSE_RANK, PERCENT_RANK, CUME_DIST, NTILE, LAG, LEAD, FIRST_VALUE, LAST_VALUE, NTH_VALUE, and aggregates with an OVER (...) clause, including PARTITION BY and explicit frames
• QUALIFY — filter rows based on a window function result
• DISTINCT — SELECT DISTINCT, DISTINCT ON (...), and the DISTINCT modifier on aggregates such as COUNT(DISTINCT ...)
• Set operations — UNION, UNION ALL, INTERSECT, and EXCEPT
• Grouping extensions — GROUPING SETS, ROLLUP, and CUBE
• Exact aggregates — MEDIAN, PERCENTILE_CONT, ARRAY_AGG, and STRING_AGG

SELECT customer_id, region,
       ROW_NUMBER() OVER (PARTITION BY region ORDER BY total_amount DESC) AS rank_in_region
FROM my_namespace.sales_data

SELECT customer_id, region, total_amount
FROM my_namespace.sales_data
QUALIFY ROW_NUMBER() OVER (PARTITION BY region ORDER BY total_amount DESC) <= 3

SELECT customer_id FROM my_namespace.sales_data
EXCEPT
SELECT customer_id FROM my_namespace.archived_sales

The named WINDOW clause is not supported — inline the OVER (...) specification at each call site. For the full syntax reference, refer to the SQL reference. For supported features and performance guidance, refer to Limitations and best practices.

R2 SQL now supports set operations (UNION, INTERSECT, EXCEPT) and SELECT DISTINCT, expanding the range of analytical queries you can run directly on Apache Iceberg ↗︎ tables in R2 Data Catalog.

Combine the results of multiple SELECT statements:

• UNION — returns all rows from both queries, removing duplicates
• UNION ALL — returns all rows from both queries, including duplicates
• INTERSECT — returns only rows that appear in both queries
• EXCEPT — returns rows from the first query that do not appear in the second

-- Find zones that had either firewall blocks OR high-risk requests
SELECT zone_id FROM my_namespace.firewall_events WHERE action = 'block'
UNION
SELECT zone_id FROM my_namespace.http_requests WHERE risk_score > 0.8

-- Find zones with both firewall blocks AND high traffic
SELECT zone_id FROM my_namespace.firewall_events WHERE action = 'block'
INTERSECT
SELECT zone_id FROM my_namespace.http_requests
GROUP BY zone_id
HAVING COUNT(*) > 10000

-- Find enterprise zones that have not been compacted
SELECT zone_id FROM my_namespace.zones WHERE plan = 'enterprise'
EXCEPT
SELECT zone_id FROM my_namespace.compaction_history

Eliminate duplicate rows from query results:

SELECT DISTINCT region, department
FROM my_namespace.sales_data
WHERE total_amount > 1000
ORDER BY region, department
LIMIT 100

For large datasets where approximate results are acceptable, approx_distinct() remains a faster alternative for counting unique values.

For the full syntax reference, refer to the SQL reference. For performance guidance, refer to Limitations and best practices.

The pipeline field inside the pipelines binding configuration in your Wrangler configuration file has been renamed to stream. The old field is deprecated but still accepted.

Update your configuration to use stream to avoid the deprecation warning.

Before (deprecated):

{
  "$schema": "./node_modules/wrangler/config-schema.json",
  "pipelines": [
    {
      "binding": "MY_PIPELINE",
      "pipeline": "<STREAM_ID>"
    }
  ]
}

[[pipelines]]
binding = "MY_PIPELINE"
pipeline = "<STREAM_ID>"

After:

{
  "$schema": "./node_modules/wrangler/config-schema.json",
  "pipelines": [
    {
      "binding": "MY_PIPELINE",
      "stream": "<STREAM_ID>"
    }
  ]
}

[[pipelines]]
binding = "MY_PIPELINE"
stream = "<STREAM_ID>"

No other changes are required. The binding name, TypeScript types, and runtime API (env.MY_PIPELINE.send(...)) remain the same.

For more information on configuring pipeline bindings, refer to Writing to streams.

R2 Data Catalog is a managed Apache Iceberg ↗︎ data catalog built directly into R2 buckets, queryable by any Iceberg-compatible engine such as Spark, Snowflake, and DuckDB. R2 Data Catalog now has published pricing for catalog operations and table compaction, in addition to standard R2 storage and operations.

Billing is not yet enabled. We will provide at least 30 days notice before we start charging for R2 Data Catalog usage.

Pricing is based on two dimensions:

• Catalog operations: $9.00 / million operations for metadata requests such as creating tables, reading table metadata, and updating table properties.
• Compaction: $0.005 / GB processed and $2.00 / million objects processed. These charges only apply when automatic compaction is turned on for a table.

Both dimensions include a monthly free tier: 1 million catalog operations, 10 GB of compaction data processed, and 1 million compaction objects processed.

For full pricing details and billing examples, refer to R2 Data Catalog pricing.

R2 Data Catalog is a managed Apache Iceberg ↗︎ data catalog built directly into your R2 bucket. It exposes a standard Iceberg REST catalog interface so you can connect query engines like Spark, Snowflake, DuckDB, and R2 SQL to your data in R2.

R2 Data Catalog now has a dedicated section in the Cloudflare dashboard, replacing the previous settings panel embedded in R2 bucket configuration. The new experience includes:

• Catalog overview — View all your catalogs in one place with catalog request counts, bucket sizes, and table maintenance status at a glance.
• Guided setup wizard — Create a catalog in three steps: choose or create an R2 bucket, configure table maintenance (compaction and snapshot expiration), and review. The wizard creates the bucket and generates a service credential automatically.
• Settings management — A dedicated settings page for each catalog with sections for general configuration, table maintenance, service credentials, and disabling the catalog. You can now enable and configure snapshot expiration directly from the dashboard.
• Built-in metrics — Five charts on each catalog's metrics tab: bytes compacted, files compacted, catalog requests, storage size, and snapshots expired.

To get started, go to R2 Data Catalog in the Cloudflare dashboard or refer to the getting started guide and manage catalogs documentation.

Cloudflare Pipelines is a streaming data platform that ingests events, transforms them with SQL, and writes to R2 as JSON, Parquet, or Apache Iceberg ↗︎ tables. Pipelines now has published pricing based on two usage dimensions: the volume of data processed by SQL transforms and the volume of data delivered to sinks. Ingress into a Pipeline stream is free.

Billing is not yet enabled. We will provide at least 30 days notice before we start charging for Pipelines usage.

Pipelines pricing model is designed to charge per GB based on what you use:

• Streams (ingress): Free, regardless of volume.
• SQL transforms: $0.04 / GB for stateless transforms (filter, reshape, unnest, cast, compute).
• Sinks: $0.03 / GB for JSON, $0.06 / GB for Parquet or Iceberg output.

Workers Free plans include 1 GB / month for each dimension. Workers Paid plans include 50 GB / month.

For full pricing details and billing examples, refer to Pipelines pricing.

R2 SQL is a serverless, distributed query engine that runs SQL against Apache Iceberg ↗︎ tables stored in R2 Data Catalog. R2 SQL now has published pricing based on a single dimension: the volume of compressed data scanned to execute your queries. At $2.50 / TB ($0.0025 / GB), R2 SQL is priced at half the cost of AWS Athena and less than half of Google BigQuery on-demand.

Billing is not yet enabled. We will provide at least 30 days notice before we start charging for R2 SQL usage.

Data scanned is measured on compressed bytes read from R2 object storage. This matches what you see in your R2 bucket — if a Parquet file is 100 MB on disk, scanning that file bills for 100 MB. Each query has a minimum billing increment of 10 MB.

All plans include 10 GB of data scanned per month. Standard R2 storage and operations and R2 Data Catalog charges apply separately.

For full pricing details and billing examples, refer to R2 SQL pricing.

R2 SQL is Cloudflare's serverless, distributed SQL engine for querying Apache Iceberg ↗︎ tables stored in R2 Data Catalog. R2 SQL runs directly on Cloudflare's global network with no infrastructure to manage, so you can analyze data in R2 without exporting it to an external warehouse.

R2 SQL now supports joining multiple Iceberg tables in a single query. You can combine tables with JOINs, filter with subqueries, and define multi-table CTEs to build complex analytical queries.

• JOINs — INNER JOIN, LEFT JOIN, RIGHT JOIN, FULL OUTER JOIN, CROSS JOIN, and implicit joins (comma-separated FROM with conditions in WHERE)
• Subqueries — IN / NOT IN, EXISTS / NOT EXISTS, scalar subqueries in SELECT / WHERE / HAVING, and derived tables (subqueries in FROM)
• Multi-table CTEs — WITH clauses can reference different tables and include JOINs
• Self-joins — join a table with itself using different aliases
• Multi-way joins — join three or more tables in a single query

SELECT z.domain, z.plan, COUNT(*) AS request_count
FROM my_namespace.zones z
INNER JOIN my_namespace.http_requests h ON z.zone_id = h.zone_id
WHERE z.plan = 'enterprise'
GROUP BY z.domain, z.plan
ORDER BY request_count DESC
LIMIT 20

SELECT z.domain, z.plan
FROM my_namespace.zones z
WHERE EXISTS (
    SELECT 1 FROM my_namespace.firewall_events f
    WHERE f.zone_id = z.zone_id AND f.action = 'block'
)
ORDER BY z.domain
LIMIT 20

WITH top_zones AS (
    SELECT zone_id, COUNT(*) AS req_count
    FROM my_namespace.http_requests
    GROUP BY zone_id
    ORDER BY req_count DESC
    LIMIT 50
),
zone_threats AS (
    SELECT zone_id, COUNT(*) AS threat_count
    FROM my_namespace.firewall_events
    WHERE risk_score > 0.5
    GROUP BY zone_id
)
SELECT tz.zone_id, tz.req_count, COALESCE(zt.threat_count, 0) AS threat_count
FROM top_zones tz
LEFT JOIN zone_threats zt ON tz.zone_id = zt.zone_id
ORDER BY tz.req_count DESC
LIMIT 20

For the full syntax reference, refer to the SQL reference. For performance guidance with joins, refer to Limitations and best practices.

R2 Data Catalog is a managed Apache Iceberg data catalog built directly into your R2 bucket that allows you to connect query engines like R2 SQL, Spark, Snowflake, and DuckDB to your data in R2.

You can now query analytics for your R2 Data Catalog warehouses via Cloudflare's GraphQL Analytics API. Two new datasets are available:

• r2CatalogDataOperationsAdaptiveGroups tracks Iceberg REST API requests made to your catalog, including operation type, request duration, HTTP status, and request body bytes. Use this to monitor request volume and latency across warehouses, namespaces, and tables.
• r2CatalogTableMaintenanceAdaptiveGroups tracks table maintenance jobs such as compaction and snapshot expiration. Use this to monitor job success rates, files processed, bytes read and written, and job duration.

Both datasets support filtering by warehouse name, namespace, table name, and time range. They also include percentile aggregations for duration metrics.

For detailed schema information and example queries, refer to the R2 Data Catalog metrics and analytics documentation.

Cloudflare Pipelines ingests streaming data via Workers or HTTP endpoints, transforms it with SQL, and writes it to R2 as Apache Iceberg tables. R2 Data Catalog manages those Iceberg tables, compaction, and compatibility with query engines like R2 SQL, Spark, and DuckDB.

You can now create and manage both products using Terraform, supported in the Cloudflare Terraform provider v5.19.0 ↗︎.

This adds four new resources that let you define your entire data pipeline as infrastructure-as-code: a data catalog, a stream for ingestion, a sink that writes to R2 Data Catalog or R2, and a pipeline that connects them with SQL.

The new Terraform resources are:

• cloudflare_r2_data_catalog ↗︎ — enable the data catalog on an R2 bucket
• cloudflare_pipeline_stream ↗︎ — create a stream that receives events via HTTP or Worker bindings
• cloudflare_pipeline_sink ↗︎ — create a sink that writes to R2 Data Catalog or R2
• cloudflare_pipeline ↗︎ — create a pipeline with SQL connecting a stream to a sink

Here is a minimal example that creates a stream, an R2 Data Catalog sink, and a pipeline:

resource "cloudflare_pipeline_stream" "my_stream" {
  account_id = var.cloudflare_account_id
  name       = "my_stream"
  format     = { type = "json" }
  schema = {
    fields = [{
      name     = "value"
      type     = "json"
      required = true
    }]
  }
  http           = { enabled = true, authentication = false, cors = {} }
  worker_binding = { enabled = false }
}

resource "cloudflare_pipeline_sink" "my_sink" {
  account_id = var.cloudflare_account_id
  name       = "my_sink"
  type       = "r2_data_catalog"
  format     = { type = "parquet" }
  schema     = { fields = [] }
  config = {
    account_id = var.cloudflare_account_id
    bucket     = "my-pipeline-bucket"
    table_name = "my_table"
    token      = var.catalog_token
  }
}

resource "cloudflare_pipeline" "my_pipeline" {
  account_id = var.cloudflare_account_id
  name       = "my_pipeline"
  sql        = "INSERT INTO ${cloudflare_pipeline_sink.my_sink.name} SELECT * FROM ${cloudflare_pipeline_stream.my_stream.name}"
}

For a full end-to-end example that includes R2 bucket creation, data catalog setup, and scoped API token provisioning, refer to the Pipelines Terraform documentation.

R2 Data Catalog, a managed Apache Iceberg ↗︎ catalog built into R2, now removes unreferenced data files during automatic snapshot expiration. This improvement reduces storage costs and eliminates the need to run manual maintenance jobs to reclaim space from deleted data.

Previously, snapshot expiration only cleaned up Iceberg metadata files such as manifests and manifest lists. Data files that were no longer referenced by active snapshots remained in R2 storage until you manually ran remove_orphan_files or expire_snapshots through an engine like Spark. This required extra operational overhead and left stale data files consuming storage.

Snapshot expiration now handles both metadata and data file cleanup automatically. When a snapshot is expired, any data files that are no longer referenced by retained snapshots are removed from R2 storage.

# Enable catalog-level snapshot expiration
npx wrangler r2 bucket catalog snapshot-expiration enable my-bucket \
  --older-than-days 7 \
  --retain-last 10

For more information, refer to the table maintenance documentation.

R2 SQL is Cloudflare's serverless, distributed, analytics query engine for querying Apache Iceberg ↗︎ tables stored in R2 Data Catalog.

R2 SQL now supports functions for querying JSON data stored in Apache Iceberg tables, an easier way to parse query plans with EXPLAIN FORMAT JSON, and querying tables without partition keys stored in R2 Data Catalog.

JSON functions extract and manipulate JSON values directly in SQL without client-side processing:

SELECT
  json_get_str(doc, 'name') AS name,
  json_get_int(doc, 'user', 'profile', 'level') AS level,
  json_get_bool(doc, 'active') AS is_active
FROM my_namespace.sales_data
WHERE json_contains(doc, 'email')

For a full list of available functions, refer to JSON functions.

EXPLAIN FORMAT JSON returns query execution plans as structured JSON for programmatic analysis and observability integrations:

npx wrangler r2 sql query "${WAREHOUSE}" "EXPLAIN FORMAT JSON SELECT * FROM logpush.requests LIMIT 10;"

┌──────────────────────────────────────┐
│ plan                                 │
├──────────────────────────────────────┤
│ {                                    │
│   "name": "CoalescePartitionsExec",  │
│   "output_partitions": 1,            │
│   "rows": 10,                        │
│   "size_approx": "310B",             │
│   "children": [                      │
│     {                                │
│       "name": "DataSourceExec",      │
│       "output_partitions": 4,        │
│       "rows": 28951,                 │
│       "size_approx": "900.0KB",      │
│       "table": "logpush.requests",   │
│       "files": 7,                    │
│       "bytes": 900019,               │
│       "projection": [                │
│         "__ingest_ts",               │
│         "CPUTimeMs",                 │
│         "DispatchNamespace",         │
│         "Entrypoint",                │
│         "Event",                     │
│         "EventTimestampMs",          │
│         "EventType",                 │
│         "Exceptions",                │
│         "Logs",                      │
│         "Outcome",                   │
│         "ScriptName",                │
│         "ScriptTags",                │
│         "ScriptVersion",             │
│         "WallTimeMs"                 │
│       ],                             │
│       "limit": 10                    │
│     }                                │
│   ]                                  │
│ }                                    │
└──────────────────────────────────────┘

For more details, refer to EXPLAIN.

Unpartitioned Iceberg tables can now be queried directly, which is useful for smaller datasets or data without natural time dimensions. For tables with more than 1000 files, partitioning is still recommended for better performance.

Refer to Limitations and best practices for the latest guidance on using R2 SQL.

R2 SQL now supports an expanded SQL grammar so you can write richer analytical queries without exporting data. This release adds CASE expressions, column aliases, arithmetic in clauses, 163 scalar functions, 33 aggregate functions, EXPLAIN, Common Table Expressions (CTEs),and full struct/array/map access. R2 SQL is Cloudflare's serverless, distributed, analytics query engine for querying Apache Iceberg ↗︎ tables stored in R2 Data Catalog. This page documents the supported SQL syntax.

• Column aliases — SELECT col AS alias now works in all clauses
• CASE expressions — conditional logic directly in SQL (searched and simple forms)
• Scalar functions — 163 new functions across math, string, datetime, regex, crypto, encoding, and type inspection categories
• Aggregate functions — statistical (variance, stddev, correlation, regression), bitwise, boolean, and positional aggregates join the existing basic and approximate functions
• Complex types — query struct fields with bracket notation, use 46 array functions, and extract map keys/values
• Common table expressions (CTEs) — use WITH ... AS to define named temporary result sets. Chained CTEs are supported. All CTEs must reference the same single table.
• Full expression support — arithmetic, type casting (CAST, TRY_CAST, :: shorthand), and EXTRACT in SELECT, WHERE, GROUP BY, HAVING, and ORDER BY

SELECT source,
    CASE
        WHEN AVG(price) > 30 THEN 'premium'
        WHEN AVG(price) > 10 THEN 'mid-tier'
        ELSE 'budget'
    END AS tier,
    round(stddev(price), 2) AS price_volatility,
    approx_percentile_cont(price, 0.95) AS p95_price
FROM my_namespace.sales_data
GROUP BY source

SELECT product_name,
    pricing['price'] AS price,
    array_to_string(tags, ', ') AS tag_list
FROM my_namespace.products
WHERE array_has(tags, 'Action')
ORDER BY pricing['price'] DESC
LIMIT 10

WITH monthly AS (
    SELECT date_trunc('month', sale_timestamp) AS month,
        department,
        COUNT(*) AS transactions,
        round(AVG(total_amount), 2) AS avg_amount
    FROM my_namespace.sales_data
    WHERE sale_timestamp BETWEEN '2025-01-01T00:00:00Z' AND '2025-12-31T23:59:59Z'
    GROUP BY date_trunc('month', sale_timestamp), department
),
ranked AS (
    SELECT month, department, transactions, avg_amount,
        CASE
            WHEN avg_amount > 1000 THEN 'high-value'
            WHEN avg_amount > 500 THEN 'mid-value'
            ELSE 'standard'
        END AS tier
    FROM monthly
    WHERE transactions > 100
)
SELECT * FROM ranked
ORDER BY month, avg_amount DESC

For the full function reference and syntax details, refer to the SQL reference. For limitations and best practices, refer to Limitations and best practices.

Cloudflare Pipelines ingests streaming data via Workers or HTTP endpoints, transforms it with SQL, and writes it to R2 as Apache Iceberg tables. Today we are shipping three improvements to help you understand why streaming events get dropped, catch data quality issues early, and set up Pipelines faster.

When stream events don't match the expected schema, Pipelines accepts them during ingestion but drops them when attempting to deliver them to the sink. To help you identify the root cause of these issues, we are introducing a new dashboard and metrics that surface dropped events with detailed error messages.

Dropped events can also be queried programmatically via the new pipelinesUserErrorsAdaptiveGroups GraphQL dataset. The dataset breaks down failures by specific error type (missing_field, type_mismatch, parse_failure, or null_value) so you can trace issues back to the source.

query GetPipelineUserErrors(
	$accountTag: String!
	$pipelineId: String!
	$datetimeStart: Time!
	$datetimeEnd: Time!
) {
	viewer {
		accounts(filter: { accountTag: $accountTag }) {
			pipelinesUserErrorsAdaptiveGroups(
				limit: 100
				filter: {
					pipelineId: $pipelineId
					datetime_geq: $datetimeStart
					datetime_leq: $datetimeEnd
				}
				orderBy: [count_DESC]
			) {
				count
				dimensions {
					errorFamily
					errorType
				}
			}
		}
	}
}

For the full list of dimensions, error types, and additional query examples, refer to User error metrics.

Sending data to a Pipeline from a Worker previously used a generic Pipeline<PipelineRecord> type, which meant schema mismatches (wrong field names, incorrect types) were only caught at runtime as dropped events.

Running wrangler types now generates schema-specific TypeScript types for your Pipeline bindings. TypeScript catches missing required fields and incorrect field types at compile time, before your code is deployed.

declare namespace Cloudflare {
	type EcommerceStreamRecord = {
		user_id: string;
		event_type: string;
		product_id?: string;
		amount?: number;
	};
	interface Env {
		STREAM: import("cloudflare:pipelines").Pipeline<Cloudflare.EcommerceStreamRecord>;
	}
}

For more information, refer to Typed Pipeline bindings.

Setting up a new Pipeline previously required multiple manual steps: creating an R2 bucket, enabling R2 Data Catalog, generating an API token, and configuring format, compression, and rolling policies individually.

The wrangler pipelines setup command now offers a Simple setup mode that applies recommended defaults and automatically creates the R2 bucket and enables R2 Data Catalog if they do not already exist. Validation errors during setup prompt you to retry inline rather than restarting the entire process.

For a full walkthrough, refer to the Getting started guide.

R2 Data Catalog now supports automatic snapshot expiration for Apache Iceberg tables.

In Apache Iceberg, a snapshot is metadata that represents the state of a table at a given point in time. Every mutation creates a new snapshot which enable powerful features like time travel queries and rollback capabilities but will accumulate over time.

Without regular cleanup, these accumulated snapshots can lead to:

• Metadata overhead
• Slower table operations
• Increased storage costs.

Snapshot expiration in R2 Data Catalog automatically removes old table snapshots based on your configured retention policy, improving performance and storage costs.

# Enable catalog-level snapshot expiration
# Expire snapshots older than 7 days, always retain at least 10 recent snapshots
npx wrangler r2 bucket catalog snapshot-expiration enable my-bucket \
  --older-than-days 7 \
  --retain-last 10

Snapshot expiration uses two parameters to determine which snapshots to remove:

• --older-than-days: age threshold in days
• --retain-last: minimum snapshot count to retain

Both conditions must be met before a snapshot is expired, ensuring you always retain recent snapshots even if they exceed the age threshold.

This feature complements automatic compaction, which optimizes query performance by combining small data files into larger ones. Together, these automatic maintenance operations keep your Iceberg tables performant and cost-efficient without manual intervention.

For more information, refer to Table maintenance or Manage catalogs.