Expand description
§Watermark-Bounded Sort Operator
Buffers events between watermark boundaries, emits sorted batch when watermark advances. Useful for producing ordered output (e.g., sorted Parquet files) from out-of-order streams.
§Memory Bounds
Only holds events in the (last_watermark, current_watermark) range,
which is bounded by max_out_of_orderness. A max_buffer_size safety
limit prevents unbounded growth.
§How It Works
- Events arrive out of order (within bounded disorder)
- Events are buffered until watermark advances
- On watermark advance, all events with timestamp <= new watermark are sorted and emitted as a batch
- Late events (timestamp <= last emitted watermark) are dropped
Structs§
- Watermark
Bounded Sort Operator - Watermark-bounded sort operator.