Create Sink

client.Pipelines.Sinks.New(ctx, params) (*SinkNewResponse, error)

POST/accounts/{account_id}/pipelines/v1/sinks

Create a new Sink.

Security

API Token

The preferred authorization scheme for interacting with the Cloudflare API. Create a token.

Example:Authorization: Bearer Sn3lZJTBX6kkg7OdcBUAxOO963GEIyGQqnFTOFYY

API Email + API Key

The previous authorization scheme for interacting with the Cloudflare API, used in conjunction with a Global API key.

Example:X-Auth-Email: user@example.com

The previous authorization scheme for interacting with the Cloudflare API. When possible, use API tokens instead of Global API keys.

Example:X-Auth-Key: 144c9defac04969c7bfad8efaa8ea194

Accepted Permissions (at least one required)

Pipelines Write

ParametersExpand Collapse

params SinkNewParams

AccountID param.Field[string]

Path param: Specifies the public ID of the account.

Name param.Field[string]

Body param: Defines the name of the Sink.

maxLength128

minLength1

Type param.Field[SinkNewParamsType]

Body param: Specifies the type of sink.

const SinkNewParamsTypeR2 SinkNewParamsType = "r2"

const SinkNewParamsTypeR2DataCatalog SinkNewParamsType = "r2_data_catalog"

Config param.Field[SinkNewParamsConfig]optional

Body param: Defines the configuration of the R2 Sink.

type SinkNewParamsConfigCloudflarePipelinesR2Table struct{…}

AccountID string

Cloudflare Account ID for the bucket

Bucket string

R2 Bucket to write to

Credentials SinkNewParamsConfigCloudflarePipelinesR2TableCredentials

AccessKeyID string

Cloudflare Account ID for the bucket

formatvar-str

SecretAccessKey string

Cloudflare Account ID for the bucket

formatvar-str

FileNaming SinkNewParamsConfigCloudflarePipelinesR2TableFileNamingoptional

Controls filename prefix/suffix and strategy.

Prefix stringoptional

The prefix to use in file name. i.e prefix-.parquet

Strategy SinkNewParamsConfigCloudflarePipelinesR2TableFileNamingStrategyoptional

Filename generation strategy.

One of the following:

const SinkNewParamsConfigCloudflarePipelinesR2TableFileNamingStrategySerial SinkNewParamsConfigCloudflarePipelinesR2TableFileNamingStrategy = "serial"

const SinkNewParamsConfigCloudflarePipelinesR2TableFileNamingStrategyUUID SinkNewParamsConfigCloudflarePipelinesR2TableFileNamingStrategy = "uuid"

const SinkNewParamsConfigCloudflarePipelinesR2TableFileNamingStrategyUUIDV7 SinkNewParamsConfigCloudflarePipelinesR2TableFileNamingStrategy = "uuid_v7"

const SinkNewParamsConfigCloudflarePipelinesR2TableFileNamingStrategyUlid SinkNewParamsConfigCloudflarePipelinesR2TableFileNamingStrategy = "ulid"

Suffix stringoptional

This will overwrite the default file suffix. i.e .parquet, use with caution

Jurisdiction stringoptional

Jurisdiction this bucket is hosted in

Partitioning SinkNewParamsConfigCloudflarePipelinesR2TablePartitioningoptional

Data-layout partitioning for sinks.

TimePattern stringoptional

The pattern of the date string

Path stringoptional

Subpath within the bucket to write to

RollingPolicy SinkNewParamsConfigCloudflarePipelinesR2TableRollingPolicyoptional

Rolling policy for file sinks (when & why to close a file and open a new one).

FileSizeBytes int64optional

Files will be rolled after reaching this number of bytes

formatuint64

minimum0

InactivitySeconds int64optional

Number of seconds of inactivity to wait before rolling over to a new file

formatuint64

minimum1

IntervalSeconds int64optional

Number of seconds to wait before rolling over to a new file

formatuint64

minimum1

type SinkNewParamsConfigCloudflarePipelinesR2DataCatalogTable struct{…}

R2 Data Catalog Sink

Token string

Authentication token

formatvar-str

AccountID string

Cloudflare Account ID

formaturi

Bucket string

The R2 Bucket that hosts this catalog

TableName string

Table name

Namespace stringoptional

Table namespace

RollingPolicy SinkNewParamsConfigCloudflarePipelinesR2DataCatalogTableRollingPolicyoptional

Rolling policy for file sinks (when & why to close a file and open a new one).

FileSizeBytes int64optional

Files will be rolled after reaching this number of bytes

formatuint64

minimum0

InactivitySeconds int64optional

Number of seconds of inactivity to wait before rolling over to a new file

formatuint64

minimum1

IntervalSeconds int64optional

Number of seconds to wait before rolling over to a new file

formatuint64

minimum1

Format param.Field[SinkNewParamsFormat]optional

Body param

type SinkNewParamsFormatJson struct{…}

Type SinkNewParamsFormatJsonType

DecimalEncoding SinkNewParamsFormatJsonDecimalEncodingoptional

One of the following:

const SinkNewParamsFormatJsonDecimalEncodingNumber SinkNewParamsFormatJsonDecimalEncoding = "number"

const SinkNewParamsFormatJsonDecimalEncodingString SinkNewParamsFormatJsonDecimalEncoding = "string"

const SinkNewParamsFormatJsonDecimalEncodingBytes SinkNewParamsFormatJsonDecimalEncoding = "bytes"

TimestampFormat SinkNewParamsFormatJsonTimestampFormatoptional

One of the following:

const SinkNewParamsFormatJsonTimestampFormatRfc3339 SinkNewParamsFormatJsonTimestampFormat = "rfc3339"

const SinkNewParamsFormatJsonTimestampFormatUnixMillis SinkNewParamsFormatJsonTimestampFormat = "unix_millis"

Unstructured booloptional

type SinkNewParamsFormatParquet struct{…}

Type SinkNewParamsFormatParquetType

Compression SinkNewParamsFormatParquetCompressionoptional

One of the following:

const SinkNewParamsFormatParquetCompressionUncompressed SinkNewParamsFormatParquetCompression = "uncompressed"

const SinkNewParamsFormatParquetCompressionSnappy SinkNewParamsFormatParquetCompression = "snappy"

const SinkNewParamsFormatParquetCompressionGzip SinkNewParamsFormatParquetCompression = "gzip"

const SinkNewParamsFormatParquetCompressionZstd SinkNewParamsFormatParquetCompression = "zstd"

const SinkNewParamsFormatParquetCompressionLz4 SinkNewParamsFormatParquetCompression = "lz4"

RowGroupBytes int64optional

formatint64

minimum0

Schema param.Field[SinkNewParamsSchema]optional

Body param

Fields []SinkNewParamsSchemaFieldoptional

One of the following:

type SinkNewParamsSchemaFieldsInt32 struct{…}

Type SinkNewParamsSchemaFieldsInt32Type

MetadataKey stringoptional

Name stringoptional

Required booloptional

SqlName stringoptional

type SinkNewParamsSchemaFieldsInt64 struct{…}

Type SinkNewParamsSchemaFieldsInt64Type

MetadataKey stringoptional

Name stringoptional

Required booloptional

SqlName stringoptional

type SinkNewParamsSchemaFieldsFloat32 struct{…}

Type SinkNewParamsSchemaFieldsFloat32Type

MetadataKey stringoptional

Name stringoptional

Required booloptional

SqlName stringoptional

type SinkNewParamsSchemaFieldsFloat64 struct{…}

Type SinkNewParamsSchemaFieldsFloat64Type

MetadataKey stringoptional

Name stringoptional

Required booloptional

SqlName stringoptional

type SinkNewParamsSchemaFieldsBool struct{…}

Type SinkNewParamsSchemaFieldsBoolType

MetadataKey stringoptional

Name stringoptional

Required booloptional

SqlName stringoptional

type SinkNewParamsSchemaFieldsString struct{…}

Type SinkNewParamsSchemaFieldsStringType

MetadataKey stringoptional

Name stringoptional

Required booloptional

SqlName stringoptional

type SinkNewParamsSchemaFieldsBinary struct{…}

Type SinkNewParamsSchemaFieldsBinaryType

MetadataKey stringoptional

Name stringoptional

Required booloptional

SqlName stringoptional

type SinkNewParamsSchemaFieldsTimestamp struct{…}

Type SinkNewParamsSchemaFieldsTimestampType

MetadataKey stringoptional

Name stringoptional

Required booloptional

SqlName stringoptional

Unit SinkNewParamsSchemaFieldsTimestampUnitoptional

One of the following:

const SinkNewParamsSchemaFieldsTimestampUnitSecond SinkNewParamsSchemaFieldsTimestampUnit = "second"

const SinkNewParamsSchemaFieldsTimestampUnitMillisecond SinkNewParamsSchemaFieldsTimestampUnit = "millisecond"

const SinkNewParamsSchemaFieldsTimestampUnitMicrosecond SinkNewParamsSchemaFieldsTimestampUnit = "microsecond"

const SinkNewParamsSchemaFieldsTimestampUnitNanosecond SinkNewParamsSchemaFieldsTimestampUnit = "nanosecond"

type SinkNewParamsSchemaFieldsJson struct{…}

Type SinkNewParamsSchemaFieldsJsonType

MetadataKey stringoptional

Name stringoptional

Required booloptional

SqlName stringoptional

type SinkNewParamsSchemaFieldsStruct struct{…}

type SinkNewParamsSchemaFieldsList struct{…}

Format SinkNewParamsSchemaFormatoptional

One of the following:

type SinkNewParamsSchemaFormatJson struct{…}

Type SinkNewParamsSchemaFormatJsonType

DecimalEncoding SinkNewParamsSchemaFormatJsonDecimalEncodingoptional

One of the following:

const SinkNewParamsSchemaFormatJsonDecimalEncodingNumber SinkNewParamsSchemaFormatJsonDecimalEncoding = "number"

const SinkNewParamsSchemaFormatJsonDecimalEncodingString SinkNewParamsSchemaFormatJsonDecimalEncoding = "string"

const SinkNewParamsSchemaFormatJsonDecimalEncodingBytes SinkNewParamsSchemaFormatJsonDecimalEncoding = "bytes"

TimestampFormat SinkNewParamsSchemaFormatJsonTimestampFormatoptional

One of the following:

const SinkNewParamsSchemaFormatJsonTimestampFormatRfc3339 SinkNewParamsSchemaFormatJsonTimestampFormat = "rfc3339"

const SinkNewParamsSchemaFormatJsonTimestampFormatUnixMillis SinkNewParamsSchemaFormatJsonTimestampFormat = "unix_millis"

Unstructured booloptional

type SinkNewParamsSchemaFormatParquet struct{…}

Type SinkNewParamsSchemaFormatParquetType

Compression SinkNewParamsSchemaFormatParquetCompressionoptional

One of the following:

const SinkNewParamsSchemaFormatParquetCompressionUncompressed SinkNewParamsSchemaFormatParquetCompression = "uncompressed"

const SinkNewParamsSchemaFormatParquetCompressionSnappy SinkNewParamsSchemaFormatParquetCompression = "snappy"

const SinkNewParamsSchemaFormatParquetCompressionGzip SinkNewParamsSchemaFormatParquetCompression = "gzip"

const SinkNewParamsSchemaFormatParquetCompressionZstd SinkNewParamsSchemaFormatParquetCompression = "zstd"

const SinkNewParamsSchemaFormatParquetCompressionLz4 SinkNewParamsSchemaFormatParquetCompression = "lz4"

RowGroupBytes int64optional

formatint64

minimum0

Inferred booloptional

ReturnsExpand Collapse

type SinkNewResponse struct{…}

ID string

Indicates a unique identifier for this sink.

CreatedAt Time

formatdate-time

ModifiedAt Time

formatdate-time

Name string

Defines the name of the Sink.

maxLength128

minLength1

Type SinkNewResponseType

Specifies the type of sink.

One of the following:

const SinkNewResponseTypeR2 SinkNewResponseType = "r2"

const SinkNewResponseTypeR2DataCatalog SinkNewResponseType = "r2_data_catalog"

Config SinkNewResponseConfigoptional

R2 Data Catalog Sink

One of the following:

type SinkNewResponseConfigCloudflarePipelinesR2Table struct{…}

AccountID string

Cloudflare Account ID for the bucket

Bucket string

R2 Bucket to write to

Credentials SinkNewResponseConfigCloudflarePipelinesR2TableCredentials

AccessKeyID string

Cloudflare Account ID for the bucket

formatvar-str

SecretAccessKey string

Cloudflare Account ID for the bucket

formatvar-str

FileNaming SinkNewResponseConfigCloudflarePipelinesR2TableFileNamingoptional

Controls filename prefix/suffix and strategy.

Prefix stringoptional

The prefix to use in file name. i.e prefix-.parquet

Strategy SinkNewResponseConfigCloudflarePipelinesR2TableFileNamingStrategyoptional

Filename generation strategy.

One of the following:

const SinkNewResponseConfigCloudflarePipelinesR2TableFileNamingStrategySerial SinkNewResponseConfigCloudflarePipelinesR2TableFileNamingStrategy = "serial"

const SinkNewResponseConfigCloudflarePipelinesR2TableFileNamingStrategyUUID SinkNewResponseConfigCloudflarePipelinesR2TableFileNamingStrategy = "uuid"

const SinkNewResponseConfigCloudflarePipelinesR2TableFileNamingStrategyUUIDV7 SinkNewResponseConfigCloudflarePipelinesR2TableFileNamingStrategy = "uuid_v7"

const SinkNewResponseConfigCloudflarePipelinesR2TableFileNamingStrategyUlid SinkNewResponseConfigCloudflarePipelinesR2TableFileNamingStrategy = "ulid"

Suffix stringoptional

This will overwrite the default file suffix. i.e .parquet, use with caution

Jurisdiction stringoptional

Jurisdiction this bucket is hosted in

Partitioning SinkNewResponseConfigCloudflarePipelinesR2TablePartitioningoptional

Data-layout partitioning for sinks.

TimePattern stringoptional

The pattern of the date string

Path stringoptional

Subpath within the bucket to write to

RollingPolicy SinkNewResponseConfigCloudflarePipelinesR2TableRollingPolicyoptional

Rolling policy for file sinks (when & why to close a file and open a new one).

FileSizeBytes int64optional

Files will be rolled after reaching this number of bytes

formatuint64

minimum0

InactivitySeconds int64optional

Number of seconds of inactivity to wait before rolling over to a new file

formatuint64

minimum1

IntervalSeconds int64optional

Number of seconds to wait before rolling over to a new file

formatuint64

minimum1

type SinkNewResponseConfigCloudflarePipelinesR2DataCatalogTable struct{…}

R2 Data Catalog Sink

Token string

Authentication token

formatvar-str

AccountID string

Cloudflare Account ID

formaturi

Bucket string

The R2 Bucket that hosts this catalog

TableName string

Table name

Namespace stringoptional

Table namespace

RollingPolicy SinkNewResponseConfigCloudflarePipelinesR2DataCatalogTableRollingPolicyoptional

Rolling policy for file sinks (when & why to close a file and open a new one).

FileSizeBytes int64optional

Files will be rolled after reaching this number of bytes

formatuint64

minimum0

InactivitySeconds int64optional

Number of seconds of inactivity to wait before rolling over to a new file

formatuint64

minimum1

IntervalSeconds int64optional

Number of seconds to wait before rolling over to a new file

formatuint64

minimum1

Format SinkNewResponseFormatoptional

One of the following:

type SinkNewResponseFormatJson struct{…}

Type SinkNewResponseFormatJsonType

DecimalEncoding SinkNewResponseFormatJsonDecimalEncodingoptional

One of the following:

const SinkNewResponseFormatJsonDecimalEncodingNumber SinkNewResponseFormatJsonDecimalEncoding = "number"

const SinkNewResponseFormatJsonDecimalEncodingString SinkNewResponseFormatJsonDecimalEncoding = "string"

const SinkNewResponseFormatJsonDecimalEncodingBytes SinkNewResponseFormatJsonDecimalEncoding = "bytes"

TimestampFormat SinkNewResponseFormatJsonTimestampFormatoptional

One of the following:

const SinkNewResponseFormatJsonTimestampFormatRfc3339 SinkNewResponseFormatJsonTimestampFormat = "rfc3339"

const SinkNewResponseFormatJsonTimestampFormatUnixMillis SinkNewResponseFormatJsonTimestampFormat = "unix_millis"

Unstructured booloptional

type SinkNewResponseFormatParquet struct{…}

Type SinkNewResponseFormatParquetType

Compression SinkNewResponseFormatParquetCompressionoptional

One of the following:

const SinkNewResponseFormatParquetCompressionUncompressed SinkNewResponseFormatParquetCompression = "uncompressed"

const SinkNewResponseFormatParquetCompressionSnappy SinkNewResponseFormatParquetCompression = "snappy"

const SinkNewResponseFormatParquetCompressionGzip SinkNewResponseFormatParquetCompression = "gzip"

const SinkNewResponseFormatParquetCompressionZstd SinkNewResponseFormatParquetCompression = "zstd"

const SinkNewResponseFormatParquetCompressionLz4 SinkNewResponseFormatParquetCompression = "lz4"

RowGroupBytes int64optional

formatint64

minimum0

Schema SinkNewResponseSchemaoptional

Fields []SinkNewResponseSchemaFieldoptional

One of the following:

type SinkNewResponseSchemaFieldsInt32 struct{…}

Type SinkNewResponseSchemaFieldsInt32Type

MetadataKey stringoptional

Name stringoptional

Required booloptional

SqlName stringoptional

type SinkNewResponseSchemaFieldsInt64 struct{…}

Type SinkNewResponseSchemaFieldsInt64Type

MetadataKey stringoptional

Name stringoptional

Required booloptional

SqlName stringoptional

type SinkNewResponseSchemaFieldsFloat32 struct{…}

Type SinkNewResponseSchemaFieldsFloat32Type

MetadataKey stringoptional

Name stringoptional

Required booloptional

SqlName stringoptional

type SinkNewResponseSchemaFieldsFloat64 struct{…}

Type SinkNewResponseSchemaFieldsFloat64Type

MetadataKey stringoptional

Name stringoptional

Required booloptional

SqlName stringoptional

type SinkNewResponseSchemaFieldsBool struct{…}

Type SinkNewResponseSchemaFieldsBoolType

MetadataKey stringoptional

Name stringoptional

Required booloptional

SqlName stringoptional

type SinkNewResponseSchemaFieldsString struct{…}

Type SinkNewResponseSchemaFieldsStringType

MetadataKey stringoptional

Name stringoptional

Required booloptional

SqlName stringoptional

type SinkNewResponseSchemaFieldsBinary struct{…}

Type SinkNewResponseSchemaFieldsBinaryType

MetadataKey stringoptional

Name stringoptional

Required booloptional

SqlName stringoptional

type SinkNewResponseSchemaFieldsTimestamp struct{…}

Type SinkNewResponseSchemaFieldsTimestampType

MetadataKey stringoptional

Name stringoptional

Required booloptional

SqlName stringoptional

Unit SinkNewResponseSchemaFieldsTimestampUnitoptional

One of the following:

const SinkNewResponseSchemaFieldsTimestampUnitSecond SinkNewResponseSchemaFieldsTimestampUnit = "second"

const SinkNewResponseSchemaFieldsTimestampUnitMillisecond SinkNewResponseSchemaFieldsTimestampUnit = "millisecond"

const SinkNewResponseSchemaFieldsTimestampUnitMicrosecond SinkNewResponseSchemaFieldsTimestampUnit = "microsecond"

const SinkNewResponseSchemaFieldsTimestampUnitNanosecond SinkNewResponseSchemaFieldsTimestampUnit = "nanosecond"

type SinkNewResponseSchemaFieldsJson struct{…}

Type SinkNewResponseSchemaFieldsJsonType

MetadataKey stringoptional

Name stringoptional

Required booloptional

SqlName stringoptional

type SinkNewResponseSchemaFieldsStruct struct{…}

type SinkNewResponseSchemaFieldsList struct{…}

Format SinkNewResponseSchemaFormatoptional

One of the following:

type SinkNewResponseSchemaFormatJson struct{…}

Type SinkNewResponseSchemaFormatJsonType

DecimalEncoding SinkNewResponseSchemaFormatJsonDecimalEncodingoptional

One of the following:

const SinkNewResponseSchemaFormatJsonDecimalEncodingNumber SinkNewResponseSchemaFormatJsonDecimalEncoding = "number"

const SinkNewResponseSchemaFormatJsonDecimalEncodingString SinkNewResponseSchemaFormatJsonDecimalEncoding = "string"

const SinkNewResponseSchemaFormatJsonDecimalEncodingBytes SinkNewResponseSchemaFormatJsonDecimalEncoding = "bytes"

TimestampFormat SinkNewResponseSchemaFormatJsonTimestampFormatoptional

One of the following:

const SinkNewResponseSchemaFormatJsonTimestampFormatRfc3339 SinkNewResponseSchemaFormatJsonTimestampFormat = "rfc3339"

const SinkNewResponseSchemaFormatJsonTimestampFormatUnixMillis SinkNewResponseSchemaFormatJsonTimestampFormat = "unix_millis"

Unstructured booloptional

type SinkNewResponseSchemaFormatParquet struct{…}

Type SinkNewResponseSchemaFormatParquetType

Compression SinkNewResponseSchemaFormatParquetCompressionoptional

One of the following:

const SinkNewResponseSchemaFormatParquetCompressionUncompressed SinkNewResponseSchemaFormatParquetCompression = "uncompressed"

const SinkNewResponseSchemaFormatParquetCompressionSnappy SinkNewResponseSchemaFormatParquetCompression = "snappy"

const SinkNewResponseSchemaFormatParquetCompressionGzip SinkNewResponseSchemaFormatParquetCompression = "gzip"

const SinkNewResponseSchemaFormatParquetCompressionZstd SinkNewResponseSchemaFormatParquetCompression = "zstd"

const SinkNewResponseSchemaFormatParquetCompressionLz4 SinkNewResponseSchemaFormatParquetCompression = "lz4"

RowGroupBytes int64optional

formatint64

minimum0

Inferred booloptional

Create Sink

package main

import (
  "context"
  "fmt"

  "github.com/cloudflare/cloudflare-go"
  "github.com/cloudflare/cloudflare-go/option"
  "github.com/cloudflare/cloudflare-go/pipelines"
)

func main() {
  client := cloudflare.NewClient(
    option.WithAPIToken("Sn3lZJTBX6kkg7OdcBUAxOO963GEIyGQqnFTOFYY"),
  )
  sink, err := client.Pipelines.Sinks.New(context.TODO(), pipelines.SinkNewParams{
    AccountID: cloudflare.F("0123105f4ecef8ad9ca31a8372d0c353"),
    Name: cloudflare.F("my_sink"),
    Type: cloudflare.F(pipelines.SinkNewParamsTypeR2),
  })
  if err != nil {
    panic(err.Error())
  }
  fmt.Printf("%+v\n", sink.ID)
}

{
  "result": {
    "id": "01234567890123457689012345678901",
    "created_at": "2019-12-27T18:11:19.117Z",
    "modified_at": "2019-12-27T18:11:19.117Z",
    "name": "my_sink",
    "type": "r2",
    "config": {
      "account_id": "account_id",
      "bucket": "bucket",
      "credentials": {
        "access_key_id": "access_key_id",
        "secret_access_key": "secret_access_key"
      },
      "file_naming": {
        "prefix": "prefix",
        "strategy": "serial",
        "suffix": "suffix"
      },
      "jurisdiction": "jurisdiction",
      "partitioning": {
        "time_pattern": "year=%Y/month=%m/day=%d/hour=%H"
      },
      "path": "path",
      "rolling_policy": {
        "file_size_bytes": 0,
        "inactivity_seconds": 1,
        "interval_seconds": 1
      }
    },
    "format": {
      "type": "json",
      "decimal_encoding": "number",
      "timestamp_format": "rfc3339",
      "unstructured": true
    },
    "schema": {
      "fields": [
        {
          "type": "int32",
          "metadata_key": "metadata_key",
          "name": "name",
          "required": true,
          "sql_name": "sql_name"
        }
      ],
      "format": {
        "type": "json",
        "decimal_encoding": "number",
        "timestamp_format": "rfc3339",
        "unstructured": true
      },
      "inferred": true
    }
  },
  "success": true
}

Returns Examples

{
  "result": {
    "id": "01234567890123457689012345678901",
    "created_at": "2019-12-27T18:11:19.117Z",
    "modified_at": "2019-12-27T18:11:19.117Z",
    "name": "my_sink",
    "type": "r2",
    "config": {
      "account_id": "account_id",
      "bucket": "bucket",
      "credentials": {
        "access_key_id": "access_key_id",
        "secret_access_key": "secret_access_key"
      },
      "file_naming": {
        "prefix": "prefix",
        "strategy": "serial",
        "suffix": "suffix"
      },
      "jurisdiction": "jurisdiction",
      "partitioning": {
        "time_pattern": "year=%Y/month=%m/day=%d/hour=%H"
      },
      "path": "path",
      "rolling_policy": {
        "file_size_bytes": 0,
        "inactivity_seconds": 1,
        "interval_seconds": 1
      }
    },
    "format": {
      "type": "json",
      "decimal_encoding": "number",
      "timestamp_format": "rfc3339",
      "unstructured": true
    },
    "schema": {
      "fields": [
        {
          "type": "int32",
          "metadata_key": "metadata_key",
          "name": "name",
          "required": true,
          "sql_name": "sql_name"
        }
      ],
      "format": {
        "type": "json",
        "decimal_encoding": "number",
        "timestamp_format": "rfc3339",
        "unstructured": true
      },
      "inferred": true
    }
  },
  "success": true
}