Skip to main content

Module sink

Module sink 

Source
Expand description

File sink connector implementing SinkConnector.

Supports two modes:

  • Append: writes directly to the output file, flushing on pre-commit.
  • Rolling: creates a new .tmp file per epoch, atomically renamed on commit.

For bulk formats (Parquet), mid-epoch rotation is disabled — files are written as a single unit on pre_commit().

Structs§

FileSink
File sink connector with append or rolling mode.