Skip to main content

Module changelog

Module changelog 

Source
Expand description

Sink-agnostic changelog collapse for upsert sinks (Z-set / CDC → key-unique _op batch). Pulled in by upsert-capable sink features (e.g. delta-lake). Sink-agnostic changelog collapse for upsert sinks. A commit epoch can carry many changelog events for one merge key (every aggregate value change is a retract + insert), which an upsert writer rejects as a cardinality violation. collapse_changelog folds an epoch down to one row per key.

Functions§

collapse_changelog
Collapse a concatenated changelog epoch batch into a key-unique batch carrying a _op column of U (upsert) or D (delete), one row per merge_key.