pub struct CheckpointManifest {Show 19 fields
pub version: u32,
pub checkpoint_id: u64,
pub epoch: u64,
pub timestamp_ms: u64,
pub durable_phase: DurableCheckpointPhase,
pub participant_id: u64,
pub source_offsets: HashMap<String, ConnectorCheckpoint>,
pub table_offsets: HashMap<String, ConnectorCheckpoint>,
pub operator_states: HashMap<String, OperatorCheckpoint>,
pub table_store_checkpoint_path: Option<String>,
pub watermark: Option<i64>,
pub source_watermarks: HashMap<String, i64>,
pub source_names: Vec<String>,
pub sink_names: Vec<String>,
pub pipeline_identity: PipelineIdentity,
pub deployment_id: String,
pub partitioning_abi_version: u16,
pub vnode_count: u16,
pub state_checksum: Option<String>,
}Expand description
A point-in-time snapshot of all pipeline state.
Fields§
§version: u32Manifest format version (for future evolution).
checkpoint_id: u64Unique, monotonically increasing checkpoint ID; must equal epoch.
epoch: u64The same checkpoint ID retained in the persisted field named epoch.
timestamp_ms: u64Timestamp when checkpoint was created (millis since Unix epoch).
durable_phase: DurableCheckpointPhaseDurable publication state. This field is intentionally required.
participant_id: u64Writer identity (0 outside cluster mode).
source_offsets: HashMap<String, ConnectorCheckpoint>Per-source connector offsets (key: source name).
table_offsets: HashMap<String, ConnectorCheckpoint>Per-table source offsets for reference tables (key: table name).
operator_states: HashMap<String, OperatorCheckpoint>Per-operator checkpoint data (key: operator/node name).
Small state is inlined as base64. Large state is stored in a separate
state.bin file and this map holds only a reference marker.
table_store_checkpoint_path: Option<String>Path to the table store checkpoint, if any.
watermark: Option<i64>Global watermark at checkpoint time.
source_watermarks: HashMap<String, i64>Per-source watermarks (key: source name).
source_names: Vec<String>Sorted names of all registered sources at checkpoint time.
Used during recovery to detect topology changes (added/removed sources) and warn the operator.
sink_names: Vec<String>Sorted names of all registered sinks at checkpoint time.
pipeline_identity: PipelineIdentityRequired, deterministic identity of the logical topology and state ABI.
deployment_id: StringCreate-once checkpoint/decision-store incarnation. A storage reset rotates this value so surviving external sink cursors cannot be reused by a fresh checkpoint-id sequence.
partitioning_abi_version: u16Durable key encoding, hashing, and key-group mapping contract.
vnode_count: u16Virtual partition count for state key distribution.
state_checksum: Option<String>SHA-256 hex digest of the sidecar state.bin file (if any).
Written during checkpoint commit so that recovery can verify the sidecar hasn’t been corrupted or truncated on disk/S3.
Implementations§
Source§impl CheckpointManifest
impl CheckpointManifest
Sourcepub fn validate(
&self,
expected_key_group_count: KeyGroupCount,
) -> Vec<ManifestValidationError>
pub fn validate( &self, expected_key_group_count: KeyGroupCount, ) -> Vec<ManifestValidationError>
Validates manifest consistency before recovery.
expected_key_group_count is the runtime’s configured key-group count;
a manifest written with a different count can’t be safely restored
because state keys won’t map to the same shards.
Returns a list of issues found. An empty list means the manifest is valid. Every returned issue makes the manifest ineligible for recovery.
Sourcepub fn new(checkpoint_id: u64, epoch: u64) -> Self
pub fn new(checkpoint_id: u64, epoch: u64) -> Self
Creates a new manifest for an embedded or single-node runtime.
Validation rejects values where checkpoint_id and epoch differ.
Sourcepub fn new_with_key_group_count(
checkpoint_id: u64,
epoch: u64,
key_group_count: KeyGroupCount,
) -> Self
pub fn new_with_key_group_count( checkpoint_id: u64, epoch: u64, key_group_count: KeyGroupCount, ) -> Self
Creates a new manifest with an explicit stable key-group count.
Validation rejects values where checkpoint_id and epoch differ.
Trait Implementations§
Source§impl Clone for CheckpointManifest
impl Clone for CheckpointManifest
Source§fn clone(&self) -> CheckpointManifest
fn clone(&self) -> CheckpointManifest
1.0.0 (const: unstable) · Source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
source. Read moreSource§impl Debug for CheckpointManifest
impl Debug for CheckpointManifest
Source§impl<'de> Deserialize<'de> for CheckpointManifest
impl<'de> Deserialize<'de> for CheckpointManifest
Source§fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>where
__D: Deserializer<'de>,
fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>where
__D: Deserializer<'de>,
Source§impl PartialEq for CheckpointManifest
impl PartialEq for CheckpointManifest
Source§impl Serialize for CheckpointManifest
impl Serialize for CheckpointManifest
impl StructuralPartialEq for CheckpointManifest
Auto Trait Implementations§
impl Freeze for CheckpointManifest
impl RefUnwindSafe for CheckpointManifest
impl Send for CheckpointManifest
impl Sync for CheckpointManifest
impl Unpin for CheckpointManifest
impl UnsafeUnpin for CheckpointManifest
impl UnwindSafe for CheckpointManifest
Blanket Implementations§
impl<T> Allocation for T
§impl<T> ArchivePointee for T
impl<T> ArchivePointee for T
§type ArchivedMetadata = ()
type ArchivedMetadata = ()
§fn pointer_metadata(
_: &<T as ArchivePointee>::ArchivedMetadata,
) -> <T as Pointee>::Metadata
fn pointer_metadata( _: &<T as ArchivePointee>::ArchivedMetadata, ) -> <T as Pointee>::Metadata
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> DeserializeOwned for Twhere
T: for<'de> Deserialize<'de>,
§impl<T> Instrument for T
impl<T> Instrument for T
§fn instrument(self, span: Span) -> Instrumented<Self>
fn instrument(self, span: Span) -> Instrumented<Self>
§fn in_current_span(self) -> Instrumented<Self>
fn in_current_span(self) -> Instrumented<Self>
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self>
fn into_either(self, into_left: bool) -> Either<Self, Self>
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more§impl<T> IntoRequest<T> for T
impl<T> IntoRequest<T> for T
§fn into_request(self) -> Request<T>
fn into_request(self) -> Request<T>
T in a tonic::Request§impl<L> LayerExt<L> for L
impl<L> LayerExt<L> for L
§fn named_layer<S>(&self, service: S) -> Layered<<L as Layer<S>>::Service, S>where
L: Layer<S>,
fn named_layer<S>(&self, service: S) -> Layered<<L as Layer<S>>::Service, S>where
L: Layer<S>,
Layered].