Expand description
Delta Lake sink connector implementation.
DeltaLakeSink implements SinkConnector, writing Arrow RecordBatch
data to Delta Lake tables with ACID transactions and at-least-once delivery
(exactly-once opt-in via delivery.guarantee = 'exactly-once').
§Write Strategies
- Append mode: Arrow-to-Parquet zero-copy writes for immutable streams
- Overwrite mode: Replace partition contents for recomputation
- Upsert mode: CDC MERGE via Z-set changelog integration
Exactly-once semantics use epoch-to-Delta-version mapping: each LaminarDB
epoch maps to exactly one Delta Lake transaction via txn application
transaction metadata in the Delta log.
§Ring Architecture
- Ring 0: No sink code. Data arrives via SPSC channel (~5ns push).
- Ring 1: Batch buffering, Parquet writes, Delta log commits.
- Ring 2: Schema management, configuration, health checks.
Structs§
- Delta
Lake Sink - Delta Lake sink connector.