Manage catalogs

Learn how to:

• Enable and disable Basin Catalog on your buckets.
• Enable and disable table maintenance features like compaction and snapshot expiration.
• Authenticate Iceberg engines using API tokens.

Enabling the catalog on a bucket turns on the REST catalog interface and provides a Catalog URI and Warehouse name required by Iceberg clients. Once enabled, you can create and manage Iceberg tables in that bucket.

npx wrangler basin catalog enable <BUCKET_NAME>

yarn wrangler basin catalog enable <BUCKET_NAME>

pnpm wrangler basin catalog enable <BUCKET_NAME>

When you disable the catalog on a bucket, it immediately stops serving requests from the catalog interface. Any Iceberg table references stored in that catalog become inaccessible until you re-enable it.

npx wrangler basin catalog disable <BUCKET_NAME>

yarn wrangler basin catalog disable <BUCKET_NAME>

pnpm wrangler basin catalog disable <BUCKET_NAME>

Compaction improves query performance by combining the many small files created during data ingestion into fewer, larger files according to the set target file size. For more information about compaction and why it is valuable, refer to About compaction.

npx wrangler basin catalog compaction enable <BUCKET_NAME> --target-size 128 --token <API_TOKEN>

yarn wrangler basin catalog compaction enable <BUCKET_NAME> --target-size 128 --token <API_TOKEN>

pnpm wrangler basin catalog compaction enable <BUCKET_NAME> --target-size 128 --token <API_TOKEN>

npx wrangler basin catalog compaction enable <BUCKET_NAME> <NAMESPACE> <TABLE> --target-size 128

yarn wrangler basin catalog compaction enable <BUCKET_NAME> <NAMESPACE> <TABLE> --target-size 128

pnpm wrangler basin catalog compaction enable <BUCKET_NAME> <NAMESPACE> <TABLE> --target-size 128

Once enabled, compaction applies retroactively to all existing tables (for catalog-level compaction) or the specified table (for table-level compaction).

Disabling compaction will prevent the process from running for all tables (catalog level) or a specific table (table level). You can re-enable it at any time.

npx wrangler basin catalog compaction disable <BUCKET_NAME>

yarn wrangler basin catalog compaction disable <BUCKET_NAME>

pnpm wrangler basin catalog compaction disable <BUCKET_NAME>

npx wrangler basin catalog compaction disable <BUCKET_NAME> <NAMESPACE> <TABLE>

yarn wrangler basin catalog compaction disable <BUCKET_NAME> <NAMESPACE> <TABLE>

pnpm wrangler basin catalog compaction disable <BUCKET_NAME> <NAMESPACE> <TABLE>

Snapshot expiration automatically removes old table snapshots and any unreferenced data files to reduce metadata overhead and storage costs. You can configure:

• Max snapshot age - Snapshots older than this duration are expired. Specify a value followed by a unit (d for days, h for hours, m for minutes, s for seconds). For example, 7d expires snapshots older than 7 days.
• Min snapshots to keep - The minimum number of snapshots to retain, regardless of age.

npx wrangler basin catalog snapshot-expiration enable <BUCKET_NAME> --token <API_TOKEN> --older-than-days 7 --retain-last 10

yarn wrangler basin catalog snapshot-expiration enable <BUCKET_NAME> --token <API_TOKEN> --older-than-days 7 --retain-last 10

pnpm wrangler basin catalog snapshot-expiration enable <BUCKET_NAME> --token <API_TOKEN> --older-than-days 7 --retain-last 10

npx wrangler basin catalog snapshot-expiration enable <BUCKET_NAME> <NAMESPACE> <TABLE> --older-than-days 2 --retain-last 5

yarn wrangler basin catalog snapshot-expiration enable <BUCKET_NAME> <NAMESPACE> <TABLE> --older-than-days 2 --retain-last 5

pnpm wrangler basin catalog snapshot-expiration enable <BUCKET_NAME> <NAMESPACE> <TABLE> --older-than-days 2 --retain-last 5

Disabling snapshot expiration prevents the process from running for all tables (catalog level) or a specific table (table level). You can re-enable snapshot expiration at any time.

npx wrangler basin catalog snapshot-expiration disable <BUCKET_NAME>

yarn wrangler basin catalog snapshot-expiration disable <BUCKET_NAME>

pnpm wrangler basin catalog snapshot-expiration disable <BUCKET_NAME>

npx wrangler basin catalog snapshot-expiration disable <BUCKET_NAME> <NAMESPACE> <TABLE>

yarn wrangler basin catalog snapshot-expiration disable <BUCKET_NAME> <NAMESPACE> <TABLE>

pnpm wrangler basin catalog snapshot-expiration disable <BUCKET_NAME> <NAMESPACE> <TABLE>

To connect your Iceberg engine to Basin Catalog, you must provide a Cloudflare API token with both Basin Catalog permissions and R2 storage permissions. Iceberg engines interact with Basin Catalog to perform table operations. The catalog also provides engines with SigV4 credentials, which are required to access the underlying data files stored in R2.

Basin Catalog supports both read-only and read-write tokens:

• Read-only operations (for example, listing namespaces, loading tables, and querying data) require a token with read access to Basin Catalog and R2 storage.
• Write operations (for example, creating or dropping tables and committing transactions) require a token with read and write access to Basin Catalog and R2 storage.

Use a read-only token for query engines and clients that only read data (such as Basin SQL, DuckDB, or PyIceberg readers), and a read-write token for engines and pipelines that create tables or write data.

Create an R2 API token with the permissions matching your workload:

• Admin Read & Write — for engines and pipelines that read and write data. Includes read and write access to both Basin Catalog and R2 storage.
• Admin Read only — for query engines and clients that only read data. Includes read access to both Basin Catalog and R2 storage.

Providing the resulting token value to your Iceberg engine gives it the ability to manage catalog metadata and handle data operations (reads or writes to R2).

To create an API token programmatically for use with Basin Catalog, you need to specify both Basin Catalog and R2 storage permission groups in your Access Policy.

Use read and write permission groups for engines and pipelines that create tables or write data:

[
	{
		"id": "f267e341f3dd4697bd3b9f71dd96247f",
		"effect": "allow",
		"resources": {
			"com.cloudflare.edge.r2.bucket.4793d734c0b8e484dfc37ec392b5fa8a_default_my-bucket": "*",
			"com.cloudflare.edge.r2.bucket.4793d734c0b8e484dfc37ec392b5fa8a_eu_my-eu-bucket": "*"
		},
		"permission_groups": [
			{
				"id": "d229766a2f7f4d299f20eaa8c9b1fde9",
				"name": "Workers R2 Data Catalog Write"
			},
			{
				"id": "2efd5506f9c8494dacb1fa10a3e7d5b6",
				"name": "Workers R2 Storage Bucket Item Write"
			}
		]
	}
]

Use read permission groups for query engines and clients that only read data:

[
	{
		"id": "f267e341f3dd4697bd3b9f71dd96247f",
		"effect": "allow",
		"resources": {
			"com.cloudflare.edge.r2.bucket.4793d734c0b8e484dfc37ec392b5fa8a_default_my-bucket": "*",
			"com.cloudflare.edge.r2.bucket.4793d734c0b8e484dfc37ec392b5fa8a_eu_my-eu-bucket": "*"
		},
		"permission_groups": [
			{
				"id": "45db74139a62490b9b60eb7c4f34994b",
				"name": "Workers R2 Data Catalog Read"
			},
			{
				"id": "6a018a9f2fc74eb6b293b0c548f38b39",
				"name": "Workers R2 Storage Bucket Item Read"
			}
		]
	}
]

To learn more about how to create API tokens for Basin Catalog using the API, including required permission groups and usage examples, refer to the Create API tokens via API documentation.

Local Uploads writes object data to a nearby location, then asynchronously copies it to your bucket. Data is queryable immediately and remains strongly consistent. This can significantly improve latency of writes from Apache Iceberg clients outside of the region of the respective Basin Catalog bucket.

To enable R2 Local Uploads, you can use the following Wrangler command:

npx wrangler r2 bucket local-uploads enable <BUCKET_NAME>

yarn wrangler r2 bucket local-uploads enable <BUCKET_NAME>

pnpm wrangler r2 bucket local-uploads enable <BUCKET_NAME>

• Basin Catalog does not currently support R2 buckets in a non-default jurisdiction.