Skip to main content

Module iceberg

Module iceberg 

Source
Expand description

Apache Iceberg sink connector implementation.

IcebergSink implements SinkConnector, writing Arrow RecordBatch data to Iceberg tables with exactly-once semantics tied to checkpoint epochs via Iceberg’s atomic transaction commits.

pre_commit() moves the buffer into staged_batches. commit_epoch() writes Parquet data files and commits via Iceberg Transaction::fast_append(). rollback_epoch() discards staged data without side effects.

Structs§

IcebergSink
Apache Iceberg sink connector.