Skip to main content

Module checkpoint_coordinator

Module checkpoint_coordinator 

Source
Expand description

Unified checkpoint coordination. Unified checkpoint coordinator.

Single orchestrator that replaces StreamCheckpointManager, PipelineCheckpointManager, and the persistence side of DagRecoveryManager. Lives in Ring 2 (control plane). Reuses the existing DagCheckpointCoordinator for barrier logic.

§Checkpoint Cycle

  1. Barrier propagation — dag_coordinator.trigger_checkpoint()
  2. Operator snapshot — dag_coordinator.finalize_checkpoint() → operator states
  3. Source snapshot — source.checkpoint() for each source
  4. Sink pre-commit — sink.pre_commit(epoch) for each exactly-once sink
  5. Manifest persist — store.save(&manifest) (atomic write)
  6. Sink commit — sink.commit_epoch(epoch) for each exactly-once sink
  7. On ANY failure at 6 — sink.rollback_epoch() on remaining sinks

Structs§

AdaptiveCheckpointConfig
Configuration for adaptive checkpoint intervals.
CheckpointConfig
Unified checkpoint configuration.
CheckpointCoordinator
Unified checkpoint coordinator.
CheckpointResult
Result of a checkpoint attempt.
CheckpointStats
Checkpoint performance statistics.
DurationHistogram
Fixed-size ring buffer for checkpoint duration percentile tracking.
UnalignedCheckpointConfig
Re-export from laminar_core::checkpoint::unaligned. Configuration for unaligned checkpoints.
WalPrepareResult
Result of WAL preparation for a checkpoint.

Enums§

CheckpointPhase
Phase of the checkpoint lifecycle.

Functions§

connector_to_source_checkpoint
Converts a ConnectorCheckpoint back to a SourceCheckpoint.
dag_snapshot_to_manifest_operators
Converts DAG operator states (from DagCheckpointSnapshot) to manifest format.
manifest_operators_to_dag_states
Converts manifest operator states back to DAG format for recovery.
source_to_connector_checkpoint
Converts a SourceCheckpoint to a ConnectorCheckpoint.