Skip to main content

EngineMetrics

laminar_db::engine_metrics

Struct EngineMetrics 

Source 
pub struct EngineMetrics {
Show 51 fields
    pub events_ingested: IntCounter,
    pub events_emitted: IntCounter,
    pub events_dropped: IntCounter,
    pub cycles: IntCounter,
    pub batches: IntCounter,
    pub queries_compiled: IntCounter,
    pub queries_cached_plan: IntCounter,
    pub cycles_backpressured: IntCounter,
    pub mv_updates: IntCounter,
    pub mv_bytes_stored: IntGauge,
    pub pipeline_watermark: IntGauge,
    pub source_watermark_ms: IntGaugeVec,
    pub source_idle: IntGaugeVec,
    pub stream_watermark_ms: IntGaugeVec,
    pub input_buf_bytes: IntGaugeVec,
    pub shed_records_total: IntCounterVec,
    pub checkpoints_completed: IntCounter,
    pub checkpoints_failed: IntCounter,
    pub checkpoint_epoch: IntGauge,
    pub checkpoint_size_bytes: IntGauge,
    pub sink_write_failures: IntCounter,
    pub sink_write_timeouts: IntCounter,
    pub sink_task_channel_closed: IntCounter,
    pub sink_filter_rejected_rows: IntCounterVec,
    pub window_late_dropped: IntCounter,
    pub events_null_timestamp: IntCounter,
    pub temporal_filter_buffered: IntGauge,
    pub temporal_filter_inserts: IntCounter,
    pub temporal_filter_retracts: IntCounter,
    pub temporal_filter_dropped: IntCounter,
    pub cycle_duration: Histogram,
    pub checkpoint_duration: Histogram,
    pub checkpoint_pipeline_stall_duration: Histogram,
    pub checkpoint_barrier_local_duration: Histogram,
    pub checkpoint_aligned_resume_wait: Histogram,
    pub checkpoint_restorable_gate_wait: Histogram,
    pub checkpoint_unchanged_vnodes: IntCounter,
    pub coordinated_committer_lag_epochs: IntGauge,
    pub sink_precommit_duration: Histogram,
    pub lookup_cache_hits: IntCounterVec,
    pub lookup_cache_misses: IntCounterVec,
    pub lookup_source_errors: IntCounterVec,
    pub lookup_in_flight_rows: IntGaugeVec,
    pub placement_vnodes_per_domain: IntGaugeVec,
    pub placement_blast_radius_ratio: Gauge,
    pub pipeline_faults_total: IntCounter,
    pub pipeline_cycle_errors_total: IntCounter,
    pub pipeline_restarts_total: IntCounter,
    pub coordinated_recoveries_total: IntCounter,
    pub coordinated_recovery_failures_total: IntCounter,
    pub shuffle_delivery_loss_incidents_total: IntCounter,

}
Expand description

Pipeline metrics registered on an explicit prometheus Registry.

Constructed once at startup, Arc-shared into PipelineCallback, CheckpointCoordinator, and OperatorGraph.

Fields§

§events_ingested: IntCounter

Events ingested from sources.

§events_emitted: IntCounter

Events emitted to streams.

§events_dropped: IntCounter

Events dropped.

§cycles: IntCounter

Processing cycles completed.

§batches: IntCounter

Batches processed.

§queries_compiled: IntCounter

Queries using compiled PhysicalExpr.

§queries_cached_plan: IntCounter

Queries using cached logical plan.

§cycles_backpressured: IntCounter

Cycles skipped by backpressure.

§mv_updates: IntCounter

Materialized view updates.

§mv_bytes_stored: IntGauge

Approximate MV bytes stored.

§pipeline_watermark: IntGauge

Global pipeline watermark.

§source_watermark_ms: IntGaugeVec

Per-source watermark (epoch-ms). Label: source.

§source_idle: IntGaugeVec

1 if a source is idle (excluded from the watermark min), else 0. Label: source.

§stream_watermark_ms: IntGaugeVec

Per-stream watermark (epoch-ms). Label: stream.

§input_buf_bytes: IntGaugeVec

Per-stream input-port buffered bytes. Label: stream.

§shed_records_total: IntCounterVec

Per-stream rows shed by the ShedOldest policy. Label: stream.

§checkpoints_completed: IntCounter

Completed checkpoints.

§checkpoints_failed: IntCounter

Failed checkpoints.

§checkpoint_epoch: IntGauge

Current checkpoint epoch.

§checkpoint_size_bytes: IntGauge

Last checkpoint size in bytes.

§sink_write_failures: IntCounter

Sink write errors.

§sink_write_timeouts: IntCounter

Sink write timeouts.

§sink_task_channel_closed: IntCounter

Sink task channel closed.

§sink_filter_rejected_rows: IntCounterVec

Rows dropped because the sink’s WHERE filter failed to compile to a PhysicalExpr (fail-closed). Label: sink.

§window_late_dropped: IntCounter

Rows dropped at operator level past allowed_lateness (distinct from events_dropped, which is source-side).

§events_null_timestamp: IntCounter

Source rows dropped because the event-time column was null.

§temporal_filter_buffered: IntGauge

Rows currently buffered by temporal-filter operators.

§temporal_filter_inserts: IntCounter

Z-set inserts (+1) emitted by temporal-filter operators.

§temporal_filter_retracts: IntCounter

Z-set retractions (-1) emitted by temporal-filter operators.

§temporal_filter_dropped: IntCounter

Late / born-expired / beyond-horizon rows dropped un-emitted.

§cycle_duration: Histogram

Per-cycle processing duration.

§checkpoint_duration: Histogram

Checkpoint cycle duration.

§checkpoint_pipeline_stall_duration: Histogram

Pipeline stall per barrier: sink write fence, shuffle alignment, state capture, and the Aligned resume gate. Exactly-once mode also includes its inline durable tail; other modes resume at Aligned while that tail remains supervised in the background.

§checkpoint_barrier_local_duration: Histogram

Local barrier work while the pipeline is paused: sink fencing, shuffle alignment, state capture, and construction of the immutable durable-tail handoff.

§checkpoint_aligned_resume_wait: Histogram

Cluster-shuffle pause after local capture while waiting for the global Aligned release. Embedded and single-node runtimes do not observe this metric.

§checkpoint_restorable_gate_wait: Histogram

Time the leader’s restorable gate spends polling for vnode partials (failed gates that burn the timeout are observed too). When this dominates restorable latency at production cadence, the push-driven upload-completion-ack follow-up is worth building.

§checkpoint_unchanged_vnodes: IntCounter

Vnode partials written as references to an unchanged base instead of re-uploading state.

§coordinated_committer_lag_epochs: IntGauge

Sealed-but-not-yet-externally-committed epochs for coordinated sinks (designated-committer lag). Rising = the committer can’t keep up.

§sink_precommit_duration: Histogram

Sink pre-commit round-trip (2PC phase 1).

§lookup_cache_hits: IntCounterVec

On-demand lookup cache hits (served without a source fetch). Label: table.

§lookup_cache_misses: IntCounterVec

On-demand lookup cache misses (not in cache). Label: table.

§lookup_source_errors: IntCounterVec

On-demand lookup source fetch errors/timeouts. Label: table.

§lookup_in_flight_rows: IntGaugeVec

On-demand lookup rows awaiting a source fetch. Label: table.

§placement_vnodes_per_domain: IntGaugeVec

Vnodes owned per failure domain (cluster mode). Label: domain.

§placement_blast_radius_ratio: Gauge

Largest single domain’s share of all vnodes ([0, 1]) — the blast radius.

§pipeline_faults_total: IntCounter

Unexpected compute-thread exits (operator panics) that faulted the pipeline.

§pipeline_cycle_errors_total: IntCounter

Fatal cycle errors dropped-and-continued under at-least-once delivery.

§pipeline_restarts_total: IntCounter

Automatic recover-from-checkpoint restarts triggered by the fault supervisor.

§coordinated_recoveries_total: IntCounter

Leader-coordinated global restart-to-epoch rounds this node applied (cluster mode).

§coordinated_recovery_failures_total: IntCounter

Leader-coordinated recovery rounds abandoned (self-restore failed or restore quorum timed out).

§shuffle_delivery_loss_incidents_total: IntCounter

Shuffle delivery-loss incidents; each one fences an epoch and forces replay.

Implementations§

Source§

impl EngineMetrics

Source

pub fn new(registry: &Registry) -> Self

Register all engine metrics on the given registry. Startup only.

§Panics

Panics if metric registration fails (duplicate names).

Auto Trait Implementations§

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
§

impl<T> ArchivePointee for T

§

type ArchivedMetadata = ()

The archived version of the pointer metadata for this type.
§

fn pointer_metadata( _: &<T as ArchivePointee>::ArchivedMetadata, ) -> <T as Pointee>::Metadata

Converts some archived metadata to the pointer metadata for itself.
§

impl<U> As for U

§

fn as_<T>(self) -> T
where T: CastFrom<U>, U: Sized,

Casts self to type T. The semantics of numeric casting with the as operator are followed, so <T as As>::as_::<U> can be used in the same way as T as U for numeric conversions. Read more
§

impl<T> AsAny for T
where T: Any + Send + Sync,

§

fn any_ref(&self) -> &(dyn Any + Send + Sync + 'static)

Obtains a dyn Any reference to the object: Read more
§

fn as_any(self: Arc<T>) -> Arc<dyn Any + Send + Sync> ⓘ

Obtains an Arc<dyn Any> reference to the object: Read more
§

fn into_any(self: Box<T>) -> Box<dyn Any + Send + Sync>

Converts the object to Box<dyn Any>: Read more
§

fn type_name(&self) -> &'static str

Convenient wrapper for std::any::type_name, since Any does not provide it and Any::type_id is useless as a debugging aid (its Debug is just a mess of hex digits).
§

impl<T> AsAny for T
where T: Any,

§

fn as_any(&self) -> &(dyn Any + 'static)

§

fn as_any_mut(&mut self) -> &mut (dyn Any + 'static)

§

fn type_name(&self) -> &'static str

Gets the type name of self
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
§

impl<T> Conv for T

§

fn conv<T>(self) -> T
where Self: Into<T>,

Converts self into T using Into<T>. Read more
§

impl<T> Downcast for T
where T: AsAny + ?Sized,

§

fn is<T>(&self) -> bool
where T: AsAny,

Returns true if the boxed type is the same as T. Read more
§

fn downcast_ref<T>(&self) -> Option<&T>
where T: AsAny,

Forward to the method defined on the type Any.
§

fn downcast_mut<T>(&mut self) -> Option<&mut T>
where T: AsAny,

Forward to the method defined on the type Any.
§

impl<T> FmtForward for T

§

fn fmt_binary(self) -> FmtBinary<Self>
where Self: Binary,

Causes self to use its Binary implementation when Debug-formatted.
§

fn fmt_display(self) -> FmtDisplay<Self>
where Self: Display,

Causes self to use its Display implementation when Debug-formatted.
§

fn fmt_lower_exp(self) -> FmtLowerExp<Self>
where Self: LowerExp,

Causes self to use its LowerExp implementation when Debug-formatted.
§

fn fmt_lower_hex(self) -> FmtLowerHex<Self>
where Self: LowerHex,

Causes self to use its LowerHex implementation when Debug-formatted.
§

fn fmt_octal(self) -> FmtOctal<Self>
where Self: Octal,

Causes self to use its Octal implementation when Debug-formatted.
§

fn fmt_pointer(self) -> FmtPointer<Self>
where Self: Pointer,

Causes self to use its Pointer implementation when Debug-formatted.
§

fn fmt_upper_exp(self) -> FmtUpperExp<Self>
where Self: UpperExp,

Causes self to use its UpperExp implementation when Debug-formatted.
§

fn fmt_upper_hex(self) -> FmtUpperHex<Self>
where Self: UpperHex,

Causes self to use its UpperHex implementation when Debug-formatted.
§

fn fmt_list(self) -> FmtList<Self>
where &'a Self: for<'a> IntoIterator,

Formats each item in a sequence. Read more
Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

§

impl<T> FutureExt for T

§

fn with_context(self, otel_cx: Context) -> WithContext<Self>

Attaches the provided Context to this type, returning a WithContext wrapper. Read more
§

fn with_current_context(self) -> WithContext<Self>

Attaches the current Context to this type, returning a WithContext wrapper. Read more
§

impl<T> Instrument for T

§

fn instrument(self, span: Span) -> Instrumented<Self>

Instruments this type with the provided [Span], returning an Instrumented wrapper. Read more
§

fn in_current_span(self) -> Instrumented<Self>

Instruments this type with the current Span, returning an Instrumented wrapper. Read more
Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T> IntoEither for T

Source§

fn into_either(self, into_left: bool) -> Either<Self, Self>

Converts self into a Left variant of Either<Self, Self> if into_left is true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more
Source§

fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
where F: FnOnce(&Self) -> bool,

Converts self into a Left variant of Either<Self, Self> if into_left(&self) returns true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more
§

impl<T> IntoRequest<T> for T

§

fn into_request(self) -> Request<T>

Wrap the input message T in a tonic::Request
§

impl<Unshared, Shared> IntoShared<Shared> for Unshared
where Shared: FromUnshared<Unshared>,

§

fn into_shared(self) -> Shared

Creates a shared type from an unshared type.
§

impl<L> LayerExt<L> for L

§

fn named_layer<S>(&self, service: S) -> Layered<<L as Layer<S>>::Service, S>
where L: Layer<S>,

Applies the layer to a service and wraps it in [Layered].
§

impl<T> LayoutRaw for T

§

fn layout_raw(_: <T as Pointee>::Metadata) -> Result<Layout, LayoutError>

Returns the layout of the type.
§

impl<T> MaybeSend for T
where T: Send,

§

impl<T, N1, N2> Niching<NichedOption<T, N1>> for N2
where T: SharedNiching<N1, N2>, N1: Niching<T>, N2: Niching<T>,

§

unsafe fn is_niched(niched: *const NichedOption<T, N1>) -> bool

Returns whether the given value has been niched. Read more
§

fn resolve_niched(out: Place<NichedOption<T, N1>>)

Writes data to out indicating that a T is niched.
§

impl<T> Pipe for T
where T: ?Sized,

§

fn pipe<R>(self, func: impl FnOnce(Self) -> R) -> R
where Self: Sized,

Pipes by value. This is generally the method you want to use. Read more
§

fn pipe_ref<'a, R>(&'a self, func: impl FnOnce(&'a Self) -> R) -> R
where R: 'a,

Borrows self and passes that borrow into the pipe function. Read more
§

fn pipe_ref_mut<'a, R>(&'a mut self, func: impl FnOnce(&'a mut Self) -> R) -> R
where R: 'a,

Mutably borrows self and passes that borrow into the pipe function. Read more
§

fn pipe_borrow<'a, B, R>(&'a self, func: impl FnOnce(&'a B) -> R) -> R
where Self: Borrow<B>, B: 'a + ?Sized, R: 'a,

Borrows self, then passes self.borrow() into the pipe function. Read more
§

fn pipe_borrow_mut<'a, B, R>( &'a mut self, func: impl FnOnce(&'a mut B) -> R, ) -> R
where Self: BorrowMut<B>, B: 'a + ?Sized, R: 'a,

Mutably borrows self, then passes self.borrow_mut() into the pipe function. Read more
§

fn pipe_as_ref<'a, U, R>(&'a self, func: impl FnOnce(&'a U) -> R) -> R
where Self: AsRef<U>, U: 'a + ?Sized, R: 'a,

Borrows self, then passes self.as_ref() into the pipe function.
§

fn pipe_as_mut<'a, U, R>(&'a mut self, func: impl FnOnce(&'a mut U) -> R) -> R
where Self: AsMut<U>, U: 'a + ?Sized, R: 'a,

Mutably borrows self, then passes self.as_mut() into the pipe function.
§

fn pipe_deref<'a, T, R>(&'a self, func: impl FnOnce(&'a T) -> R) -> R
where Self: Deref<Target = T>, T: 'a + ?Sized, R: 'a,

Borrows self, then passes self.deref() into the pipe function.
§

fn pipe_deref_mut<'a, T, R>( &'a mut self, func: impl FnOnce(&'a mut T) -> R, ) -> R
where Self: DerefMut<Target = T> + Deref, T: 'a + ?Sized, R: 'a,

Mutably borrows self, then passes self.deref_mut() into the pipe function.
§

impl<T> Pointable for T

§

const ALIGN: usize

The alignment of pointer.
§

type Init = T

The type for initializers.
§

unsafe fn init(init: <T as Pointable>::Init) -> usize

Initializes a with the given initializer. Read more
§

unsafe fn deref<'a>(ptr: usize) -> &'a T

Dereferences the given pointer. Read more
§

unsafe fn deref_mut<'a>(ptr: usize) -> &'a mut T

Mutably dereferences the given pointer. Read more
§

unsafe fn drop(ptr: usize)

Drops the object pointed to by the given pointer. Read more
§

impl<T> Pointee for T

§

type Metadata = ()

The metadata type for pointers and references to this type.
§

impl<T> PolicyExt for T
where T: ?Sized,

§

fn and<P, B, E>(self, other: P) -> And<T, P>
where T: Sized + Policy<B, E>, P: Policy<B, E>,

Create a new Policy that returns [Action::Follow] only if self and other return Action::Follow. Read more
§

fn or<P, B, E>(self, other: P) -> Or<T, P>
where T: Sized + Policy<B, E>, P: Policy<B, E>,

Create a new Policy that returns [Action::Follow] if either self or other returns Action::Follow. Read more
Source§

impl<T> Same for T

Source§

type Output = T

Should always be Self
§

impl<T> Tap for T

§

fn tap(self, func: impl FnOnce(&Self)) -> Self

Immutable access to a value. Read more
§

fn tap_mut(self, func: impl FnOnce(&mut Self)) -> Self

Mutable access to a value. Read more
§

fn tap_borrow<B>(self, func: impl FnOnce(&B)) -> Self
where Self: Borrow<B>, B: ?Sized,

Immutable access to the Borrow<B> of a value. Read more
§

fn tap_borrow_mut<B>(self, func: impl FnOnce(&mut B)) -> Self
where Self: BorrowMut<B>, B: ?Sized,

Mutable access to the BorrowMut<B> of a value. Read more
§

fn tap_ref<R>(self, func: impl FnOnce(&R)) -> Self
where Self: AsRef<R>, R: ?Sized,

Immutable access to the AsRef<R> view of a value. Read more
§

fn tap_ref_mut<R>(self, func: impl FnOnce(&mut R)) -> Self
where Self: AsMut<R>, R: ?Sized,

Mutable access to the AsMut<R> view of a value. Read more
§

fn tap_deref<T>(self, func: impl FnOnce(&T)) -> Self
where Self: Deref<Target = T>, T: ?Sized,

Immutable access to the Deref::Target of a value. Read more
§

fn tap_deref_mut<T>(self, func: impl FnOnce(&mut T)) -> Self
where Self: DerefMut<Target = T> + Deref, T: ?Sized,

Mutable access to the Deref::Target of a value. Read more
§

fn tap_dbg(self, func: impl FnOnce(&Self)) -> Self

Calls .tap() only in debug builds, and is erased in release builds.
§

fn tap_mut_dbg(self, func: impl FnOnce(&mut Self)) -> Self

Calls .tap_mut() only in debug builds, and is erased in release builds.
§

fn tap_borrow_dbg<B>(self, func: impl FnOnce(&B)) -> Self
where Self: Borrow<B>, B: ?Sized,

Calls .tap_borrow() only in debug builds, and is erased in release builds.
§

fn tap_borrow_mut_dbg<B>(self, func: impl FnOnce(&mut B)) -> Self
where Self: BorrowMut<B>, B: ?Sized,

Calls .tap_borrow_mut() only in debug builds, and is erased in release builds.
§

fn tap_ref_dbg<R>(self, func: impl FnOnce(&R)) -> Self
where Self: AsRef<R>, R: ?Sized,

Calls .tap_ref() only in debug builds, and is erased in release builds.
§

fn tap_ref_mut_dbg<R>(self, func: impl FnOnce(&mut R)) -> Self
where Self: AsMut<R>, R: ?Sized,

Calls .tap_ref_mut() only in debug builds, and is erased in release builds.
§

fn tap_deref_dbg<T>(self, func: impl FnOnce(&T)) -> Self
where Self: Deref<Target = T>, T: ?Sized,

Calls .tap_deref() only in debug builds, and is erased in release builds.
§

fn tap_deref_mut_dbg<T>(self, func: impl FnOnce(&mut T)) -> Self
where Self: DerefMut<Target = T> + Deref, T: ?Sized,

Calls .tap_deref_mut() only in debug builds, and is erased in release builds.
§

impl<T> TryConv for T

§

fn try_conv<T>(self) -> Result<T, Self::Error>
where Self: TryInto<T>,

Attempts to convert self into T using TryInto<T>. Read more
Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = Infallible

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.
§

impl<KernelType, ArrowType> TryIntoArrow<ArrowType> for KernelType
where ArrowType: TryFromKernel<KernelType>,

§

fn try_into_arrow(self) -> Result<ArrowType, ArrowError>

§

impl<KernelType, ArrowType> TryIntoKernel<KernelType> for ArrowType
where KernelType: TryFromArrow<ArrowType>,

§

fn try_into_kernel(self) -> Result<KernelType, ArrowError>

§

impl<V, T> VZip<V> for T
where V: MultiLane<T>,

§

fn vzip(self) -> V

§

impl<T> WithSubscriber for T

§

fn with_subscriber<S>(self, subscriber: S) -> WithDispatch<Self>
where S: Into<Dispatch>,

Attaches the provided Subscriber to this type, returning a [WithDispatch] wrapper. Read more
§

fn with_current_subscriber(self) -> WithDispatch<Self>

Attaches the current default Subscriber to this type, returning a [WithDispatch] wrapper. Read more