Module checkpoint_coordinator 

Unified checkpoint coordination. Unified checkpoint coordinator.

Single orchestrator for checkpoint lifecycle. Lives in Ring 2 (control plane).

The checkpoint manifest is the source of truth for source offsets. Kafka broker commits are advisory (for monitoring tools). On recovery, offsets restore from manifest, not consumer group state.

Checkpoint Cycle

1. Barrier injection — CheckpointBarrierInjector.trigger()
2. Operator snapshot — OperatorGraph.snapshot_state() → operator states
3. Source snapshot — source.checkpoint() for each source
4. Sink pre-commit — sink.pre_commit(epoch) for each exactly-once sink
5. Manifest persist — store.save(&manifest) (atomic write)
6. Sink commit — sink.commit_epoch(epoch) for each exactly-once sink
7. On ANY failure at 6 — sink.rollback_epoch() on remaining sinks

Structs

CheckpointConfig

Unified checkpoint configuration.

CheckpointCoordinator

Unified checkpoint coordinator.

CheckpointRequest

Parameters for a checkpoint operation.

CheckpointResult

Result of a checkpoint attempt.

CheckpointStats

Checkpoint performance statistics.

DurationHistogram

Fixed-size ring buffer for duration percentile tracking.

Enums

CheckpointPhase

Phase of the checkpoint lifecycle.

Functions

connector_to_source_checkpoint

Converts a ConnectorCheckpoint back to a SourceCheckpoint.

source_to_connector_checkpoint

Converts a SourceCheckpoint to a ConnectorCheckpoint.