Fork me on GitHub

Azure Blob Sink Configuration

The complete Azure Blob sink configuration file

azure.retry.backoff.initial.delay.ms

  • Default value: 1000
  • Type: LONG
  • Valid values: [0,…]
  • Importance: MEDIUM

Initial retry delay in milliseconds. The default value is 1000

azure.retry.backoff.max.attempts

  • Default value: 6
  • Type: INT
  • Valid values: [0,…]
  • Importance: MEDIUM

Retry max attempts. The default value is 6

azure.retry.backoff.max.delay.ms

  • Default value: 32000
  • Type: LONG
  • Valid values: [0,…]
  • Importance: MEDIUM

Maximum retry delay in milliseconds. The default value is 32000

azure.storage.connection.string

  • Default value: none
  • Type: STRING
  • Valid values: no restrictions
  • Importance: HIGH

Azure Storage connection string.

azure.storage.container.name

  • Default value: none
  • Type: STRING
  • Valid values: non-empty string
  • Importance: HIGH

The Azure Blob container name to store output files in.

azure.user.agent

  • Default value: Azure Blob Sink/3.4.3-SNAPSHOT (GPN: Aiven;)
  • Type: STRING
  • Valid values: no restrictions
  • Importance: LOW

A custom user agent used while contacting Azure

file.compression.type

  • Default value: none
  • Type: STRING
  • Valid values: io.aiven.kafka.connect.azure.sink.AzureBlobSinkConfig$2@13a5fe33
  • Importance: MEDIUM

The compression type used for files put on Azure Blob. The supported values are: ‘none’, ‘gzip’, ‘snappy’, ‘zstd’.

file.max.records

  • Default value: none
  • Type: INT
  • Valid values: io.aiven.kafka.connect.azure.sink.AzureBlobSinkConfig$3@5f9d02cb
  • Importance: MEDIUM

The maximum number of records to put in a single file. Must be a non-negative integer number. 0 is interpreted as “unlimited”, which is the default.

file.name.prefix

  • Default value: none
  • Type: STRING
  • Valid values: io.aiven.kafka.connect.azure.sink.AzureBlobSinkConfig$1@63753b6d
  • Importance: MEDIUM

The prefix to be added to the name of each file put on Azure Blob.

file.name.template

  • Default value: none
  • Type: STRING
  • Valid values: io.aiven.kafka.connect.common.config.validators.FilenameTemplateValidator@6b09bb57
  • Importance: MEDIUM

The template for file names on Azure Blob. Supports {{ variable }} placeholders for substituting variables. Currently supported variables are topic, partition, and start_offset (the offset of the first record in the file). Only some combinations of variables are valid, which currently are:

  • topic, partition, start_offset.

file.name.timestamp.source

  • Default value: WALLCLOCK
  • Type: STRING
  • Valid values: io.aiven.kafka.connect.azure.sink.AzureBlobSinkConfig$5@6536e911
  • Importance: LOW

Specifies the timestamp variable source. Default is wall-clock.

file.name.timestamp.timezone

  • Default value: Z
  • Type: STRING
  • Valid values: io.aiven.kafka.connect.azure.sink.AzureBlobSinkConfig$4@520a3426
  • Importance: LOW

Specifies the timezone in which the dates and time for the timestamp variable will be treated. Use standard shot and long names. Default is UTC

format.output.envelope

  • Default value: true
  • Type: BOOLEAN
  • Valid values: no restrictions
  • Importance: MEDIUM

Whether to enable envelope for entries with single field.

format.output.fields

  • Default value: [value]
  • Type: LIST
  • Valid values: io.aiven.kafka.connect.common.config.validators.OutputFieldsValidator@18eed359
  • Importance: MEDIUM

Fields to put into output files. The supported values are: ‘key’, ‘value’, ‘offset’, ‘timestamp’, ‘headers’.

format.output.fields.value.encoding

  • Default value: base64
  • Type: STRING
  • Valid values: io.aiven.kafka.connect.common.config.validators.OutputFieldsEncodingValidator@3e9b1010
  • Importance: MEDIUM

The type of encoding for the value field. The supported values are: ‘none’, ‘base64’.

format.output.type

  • Default value: csv
  • Type: STRING
  • Valid values: ‘avro’, ‘csv’, ‘json’, ‘jsonl’, ‘parquet’
  • Importance: MEDIUM

The format type of output contentThe supported values are: ‘avro’, ‘csv’, ‘json’, ‘jsonl’, ‘parquet’.