Expand description
Unified checkpoint coordination. Unified checkpoint coordinator.
Single orchestrator that replaces StreamCheckpointManager,
PipelineCheckpointManager, and the persistence side of DagRecoveryManager.
Lives in Ring 2 (control plane). Reuses the existing
DagCheckpointCoordinator for barrier logic.
§Checkpoint Cycle
- Barrier propagation —
dag_coordinator.trigger_checkpoint() - Operator snapshot —
dag_coordinator.finalize_checkpoint()→ operator states - Source snapshot —
source.checkpoint()for each source - Sink pre-commit —
sink.pre_commit(epoch)for each exactly-once sink - Manifest persist —
store.save(&manifest)(atomic write) - Sink commit —
sink.commit_epoch(epoch)for each exactly-once sink - On ANY failure at 6 —
sink.rollback_epoch()on remaining sinks
Structs§
- Adaptive
Checkpoint Config - Configuration for adaptive checkpoint intervals.
- Checkpoint
Config - Unified checkpoint configuration.
- Checkpoint
Coordinator - Unified checkpoint coordinator.
- Checkpoint
Result - Result of a checkpoint attempt.
- Checkpoint
Stats - Checkpoint performance statistics.
- Duration
Histogram - Fixed-size ring buffer for checkpoint duration percentile tracking.
- Unaligned
Checkpoint Config - Re-export from
laminar_core::checkpoint::unaligned. Configuration for unaligned checkpoints. - WalPrepare
Result - Result of WAL preparation for a checkpoint.
Enums§
- Checkpoint
Phase - Phase of the checkpoint lifecycle.
Functions§
- connector_
to_ source_ checkpoint - Converts a
ConnectorCheckpointback to aSourceCheckpoint. - dag_
snapshot_ to_ manifest_ operators - Converts DAG operator states (from
DagCheckpointSnapshot) to manifest format. - manifest_
operators_ to_ dag_ states - Converts manifest operator states back to DAG format for recovery.
- source_
to_ connector_ checkpoint - Converts a
SourceCheckpointto aConnectorCheckpoint.