List Sinks

client.Pipelines.Sinks.List(ctx, params) (*V4PagePaginationArray[SinkListResponse], error)

GET/accounts/{account_id}/pipelines/v1/sinks

List/Filter Sinks in Account.

Security

API Token

The preferred authorization scheme for interacting with the Cloudflare API. Create a token.

Example:Authorization: Bearer Sn3lZJTBX6kkg7OdcBUAxOO963GEIyGQqnFTOFYY

API Email + API Key

The previous authorization scheme for interacting with the Cloudflare API, used in conjunction with a Global API key.

Example:X-Auth-Email: user@example.com

The previous authorization scheme for interacting with the Cloudflare API. When possible, use API tokens instead of Global API keys.

Example:X-Auth-Key: 144c9defac04969c7bfad8efaa8ea194

Accepted Permissions (at least one required)

Pipelines WritePipelines Read

ParametersExpand Collapse

params SinkListParams

AccountID param.Field[string]

Path param: Specifies the public ID of the account.

Page param.Field[float64]optional

Query param

PerPage param.Field[float64]optional

Query param

PipelineID param.Field[string]optional

Query param

ReturnsExpand Collapse

type SinkListResponse struct{…}

ID string

Indicates a unique identifier for this sink.

CreatedAt Time

formatdate-time

ModifiedAt Time

formatdate-time

Name string

Defines the name of the Sink.

maxLength128

minLength1

Type SinkListResponseType

Specifies the type of sink.

One of the following:

const SinkListResponseTypeR2 SinkListResponseType = "r2"

const SinkListResponseTypeR2DataCatalog SinkListResponseType = "r2_data_catalog"

Config SinkListResponseConfigoptional

Defines the configuration of the R2 Sink.

One of the following:

type SinkListResponseConfigCloudflarePipelinesR2TablePublic struct{…}

R2 Sink public configuration.

AccountID string

Cloudflare Account ID for the bucket

Bucket string

R2 Bucket to write to

FileNaming SinkListResponseConfigCloudflarePipelinesR2TablePublicFileNamingoptional

Controls filename prefix/suffix and strategy.

Prefix stringoptional

The prefix to use in file name. i.e prefix-.parquet

Strategy SinkListResponseConfigCloudflarePipelinesR2TablePublicFileNamingStrategyoptional

Filename generation strategy.

One of the following:

const SinkListResponseConfigCloudflarePipelinesR2TablePublicFileNamingStrategySerial SinkListResponseConfigCloudflarePipelinesR2TablePublicFileNamingStrategy = "serial"

const SinkListResponseConfigCloudflarePipelinesR2TablePublicFileNamingStrategyUUID SinkListResponseConfigCloudflarePipelinesR2TablePublicFileNamingStrategy = "uuid"

const SinkListResponseConfigCloudflarePipelinesR2TablePublicFileNamingStrategyUUIDV7 SinkListResponseConfigCloudflarePipelinesR2TablePublicFileNamingStrategy = "uuid_v7"

const SinkListResponseConfigCloudflarePipelinesR2TablePublicFileNamingStrategyUlid SinkListResponseConfigCloudflarePipelinesR2TablePublicFileNamingStrategy = "ulid"

Suffix stringoptional

This will overwrite the default file suffix. i.e .parquet, use with caution

Jurisdiction stringoptional

Jurisdiction this bucket is hosted in

Partitioning SinkListResponseConfigCloudflarePipelinesR2TablePublicPartitioningoptional

Data-layout partitioning for sinks.

TimePattern stringoptional

The pattern of the date string

Path stringoptional

Subpath within the bucket to write to

RollingPolicy SinkListResponseConfigCloudflarePipelinesR2TablePublicRollingPolicyoptional

Rolling policy for file sinks (when & why to close a file and open a new one).

FileSizeBytes int64optional

Files will be rolled after reaching this number of bytes

formatuint64

minimum0

InactivitySeconds int64optional

Number of seconds of inactivity to wait before rolling over to a new file

formatuint64

minimum1

IntervalSeconds int64optional

Number of seconds to wait before rolling over to a new file

formatuint64

minimum1

type SinkListResponseConfigCloudflarePipelinesR2DataCatalogTablePublic struct{…}

R2 Data Catalog Sink public configuration.

AccountID string

Cloudflare Account ID

formaturi

Bucket string

The R2 Bucket that hosts this catalog

TableName string

Table name

Namespace stringoptional

Table namespace

RollingPolicy SinkListResponseConfigCloudflarePipelinesR2DataCatalogTablePublicRollingPolicyoptional

Rolling policy for file sinks (when & why to close a file and open a new one).

FileSizeBytes int64optional

Files will be rolled after reaching this number of bytes

formatuint64

minimum0

InactivitySeconds int64optional

Number of seconds of inactivity to wait before rolling over to a new file

formatuint64

minimum1

IntervalSeconds int64optional

Number of seconds to wait before rolling over to a new file

formatuint64

minimum1

Format SinkListResponseFormatoptional

One of the following:

type SinkListResponseFormatJson struct{…}

Type SinkListResponseFormatJsonType

DecimalEncoding SinkListResponseFormatJsonDecimalEncodingoptional

One of the following:

const SinkListResponseFormatJsonDecimalEncodingNumber SinkListResponseFormatJsonDecimalEncoding = "number"

const SinkListResponseFormatJsonDecimalEncodingString SinkListResponseFormatJsonDecimalEncoding = "string"

const SinkListResponseFormatJsonDecimalEncodingBytes SinkListResponseFormatJsonDecimalEncoding = "bytes"

TimestampFormat SinkListResponseFormatJsonTimestampFormatoptional

One of the following:

const SinkListResponseFormatJsonTimestampFormatRfc3339 SinkListResponseFormatJsonTimestampFormat = "rfc3339"

const SinkListResponseFormatJsonTimestampFormatUnixMillis SinkListResponseFormatJsonTimestampFormat = "unix_millis"

Unstructured booloptional

type SinkListResponseFormatParquet struct{…}

Type SinkListResponseFormatParquetType

Compression SinkListResponseFormatParquetCompressionoptional

One of the following:

const SinkListResponseFormatParquetCompressionUncompressed SinkListResponseFormatParquetCompression = "uncompressed"

const SinkListResponseFormatParquetCompressionSnappy SinkListResponseFormatParquetCompression = "snappy"

const SinkListResponseFormatParquetCompressionGzip SinkListResponseFormatParquetCompression = "gzip"

const SinkListResponseFormatParquetCompressionZstd SinkListResponseFormatParquetCompression = "zstd"

const SinkListResponseFormatParquetCompressionLz4 SinkListResponseFormatParquetCompression = "lz4"

RowGroupBytes int64optional

formatint64

minimum0

Schema SinkListResponseSchemaoptional

Fields []SinkListResponseSchemaFieldoptional

One of the following:

type SinkListResponseSchemaFieldsInt32 struct{…}

Type SinkListResponseSchemaFieldsInt32Type

MetadataKey stringoptional

Name stringoptional

Required booloptional

SqlName stringoptional

type SinkListResponseSchemaFieldsInt64 struct{…}

Type SinkListResponseSchemaFieldsInt64Type

MetadataKey stringoptional

Name stringoptional

Required booloptional

SqlName stringoptional

type SinkListResponseSchemaFieldsFloat32 struct{…}

Type SinkListResponseSchemaFieldsFloat32Type

MetadataKey stringoptional

Name stringoptional

Required booloptional

SqlName stringoptional

type SinkListResponseSchemaFieldsFloat64 struct{…}

Type SinkListResponseSchemaFieldsFloat64Type

MetadataKey stringoptional

Name stringoptional

Required booloptional

SqlName stringoptional

type SinkListResponseSchemaFieldsBool struct{…}

Type SinkListResponseSchemaFieldsBoolType

MetadataKey stringoptional

Name stringoptional

Required booloptional

SqlName stringoptional

type SinkListResponseSchemaFieldsString struct{…}

Type SinkListResponseSchemaFieldsStringType

MetadataKey stringoptional

Name stringoptional

Required booloptional

SqlName stringoptional

type SinkListResponseSchemaFieldsBinary struct{…}

Type SinkListResponseSchemaFieldsBinaryType

MetadataKey stringoptional

Name stringoptional

Required booloptional

SqlName stringoptional

type SinkListResponseSchemaFieldsTimestamp struct{…}

Type SinkListResponseSchemaFieldsTimestampType

MetadataKey stringoptional

Name stringoptional

Required booloptional

SqlName stringoptional

Unit SinkListResponseSchemaFieldsTimestampUnitoptional

One of the following:

const SinkListResponseSchemaFieldsTimestampUnitSecond SinkListResponseSchemaFieldsTimestampUnit = "second"

const SinkListResponseSchemaFieldsTimestampUnitMillisecond SinkListResponseSchemaFieldsTimestampUnit = "millisecond"

const SinkListResponseSchemaFieldsTimestampUnitMicrosecond SinkListResponseSchemaFieldsTimestampUnit = "microsecond"

const SinkListResponseSchemaFieldsTimestampUnitNanosecond SinkListResponseSchemaFieldsTimestampUnit = "nanosecond"

type SinkListResponseSchemaFieldsJson struct{…}

Type SinkListResponseSchemaFieldsJsonType

MetadataKey stringoptional

Name stringoptional

Required booloptional

SqlName stringoptional

type SinkListResponseSchemaFieldsStruct struct{…}

type SinkListResponseSchemaFieldsList struct{…}

Format SinkListResponseSchemaFormatoptional

One of the following:

type SinkListResponseSchemaFormatJson struct{…}

Type SinkListResponseSchemaFormatJsonType

DecimalEncoding SinkListResponseSchemaFormatJsonDecimalEncodingoptional

One of the following:

const SinkListResponseSchemaFormatJsonDecimalEncodingNumber SinkListResponseSchemaFormatJsonDecimalEncoding = "number"

const SinkListResponseSchemaFormatJsonDecimalEncodingString SinkListResponseSchemaFormatJsonDecimalEncoding = "string"

const SinkListResponseSchemaFormatJsonDecimalEncodingBytes SinkListResponseSchemaFormatJsonDecimalEncoding = "bytes"

TimestampFormat SinkListResponseSchemaFormatJsonTimestampFormatoptional

One of the following:

const SinkListResponseSchemaFormatJsonTimestampFormatRfc3339 SinkListResponseSchemaFormatJsonTimestampFormat = "rfc3339"

const SinkListResponseSchemaFormatJsonTimestampFormatUnixMillis SinkListResponseSchemaFormatJsonTimestampFormat = "unix_millis"

Unstructured booloptional

type SinkListResponseSchemaFormatParquet struct{…}

Type SinkListResponseSchemaFormatParquetType

Compression SinkListResponseSchemaFormatParquetCompressionoptional

One of the following:

const SinkListResponseSchemaFormatParquetCompressionUncompressed SinkListResponseSchemaFormatParquetCompression = "uncompressed"

const SinkListResponseSchemaFormatParquetCompressionSnappy SinkListResponseSchemaFormatParquetCompression = "snappy"

const SinkListResponseSchemaFormatParquetCompressionGzip SinkListResponseSchemaFormatParquetCompression = "gzip"

const SinkListResponseSchemaFormatParquetCompressionZstd SinkListResponseSchemaFormatParquetCompression = "zstd"

const SinkListResponseSchemaFormatParquetCompressionLz4 SinkListResponseSchemaFormatParquetCompression = "lz4"

RowGroupBytes int64optional

formatint64

minimum0

Inferred booloptional

List Sinks

package main

import (
  "context"
  "fmt"

  "github.com/cloudflare/cloudflare-go"
  "github.com/cloudflare/cloudflare-go/option"
  "github.com/cloudflare/cloudflare-go/pipelines"
)

func main() {
  client := cloudflare.NewClient(
    option.WithAPIToken("Sn3lZJTBX6kkg7OdcBUAxOO963GEIyGQqnFTOFYY"),
  )
  page, err := client.Pipelines.Sinks.List(context.TODO(), pipelines.SinkListParams{
    AccountID: cloudflare.F("0123105f4ecef8ad9ca31a8372d0c353"),
  })
  if err != nil {
    panic(err.Error())
  }
  fmt.Printf("%+v\n", page)
}

{
  "result": [
    {
      "id": "01234567890123457689012345678901",
      "created_at": "2019-12-27T18:11:19.117Z",
      "modified_at": "2019-12-27T18:11:19.117Z",
      "name": "my_sink",
      "type": "r2",
      "config": {
        "account_id": "account_id",
        "bucket": "bucket",
        "file_naming": {
          "prefix": "prefix",
          "strategy": "serial",
          "suffix": "suffix"
        },
        "jurisdiction": "jurisdiction",
        "partitioning": {
          "time_pattern": "year=%Y/month=%m/day=%d/hour=%H"
        },
        "path": "path",
        "rolling_policy": {
          "file_size_bytes": 0,
          "inactivity_seconds": 1,
          "interval_seconds": 1
        }
      },
      "format": {
        "type": "json",
        "decimal_encoding": "number",
        "timestamp_format": "rfc3339",
        "unstructured": true
      },
      "schema": {
        "fields": [
          {
            "type": "int32",
            "metadata_key": "metadata_key",
            "name": "name",
            "required": true,
            "sql_name": "sql_name"
          }
        ],
        "format": {
          "type": "json",
          "decimal_encoding": "number",
          "timestamp_format": "rfc3339",
          "unstructured": true
        },
        "inferred": true
      }
    }
  ],
  "result_info": {
    "count": 1,
    "page": 0,
    "per_page": 10,
    "total_count": 1
  },
  "success": true
}

Returns Examples

{
  "result": [
    {
      "id": "01234567890123457689012345678901",
      "created_at": "2019-12-27T18:11:19.117Z",
      "modified_at": "2019-12-27T18:11:19.117Z",
      "name": "my_sink",
      "type": "r2",
      "config": {
        "account_id": "account_id",
        "bucket": "bucket",
        "file_naming": {
          "prefix": "prefix",
          "strategy": "serial",
          "suffix": "suffix"
        },
        "jurisdiction": "jurisdiction",
        "partitioning": {
          "time_pattern": "year=%Y/month=%m/day=%d/hour=%H"
        },
        "path": "path",
        "rolling_policy": {
          "file_size_bytes": 0,
          "inactivity_seconds": 1,
          "interval_seconds": 1
        }
      },
      "format": {
        "type": "json",
        "decimal_encoding": "number",
        "timestamp_format": "rfc3339",
        "unstructured": true
      },
      "schema": {
        "fields": [
          {
            "type": "int32",
            "metadata_key": "metadata_key",
            "name": "name",
            "required": true,
            "sql_name": "sql_name"
          }
        ],
        "format": {
          "type": "json",
          "decimal_encoding": "number",
          "timestamp_format": "rfc3339",
          "unstructured": true
        },
        "inferred": true
      }
    }
  ],
  "result_info": {
    "count": 1,
    "page": 0,
    "per_page": 10,
    "total_count": 1
  },
  "success": true
}