Struct PipelineConfig 

pub struct PipelineConfig {

    pub max_poll_records: usize,
    pub channel_capacity: usize,
    pub fallback_poll_interval: Duration,
    pub checkpoint_interval: Option<Duration>,
    pub batch_window: Duration,
    pub barrier_alignment_timeout: Duration,
    pub delivery_guarantee: DeliveryGuarantee,
    pub cycle_budget_ns: u64,
    pub drain_budget_ns: u64,
    pub query_budget_ns: u64,
    pub background_budget_ns: u64,
    pub max_input_buf_batches: usize,
    pub max_input_buf_bytes: Option<usize>,
    pub backpressure_policy: BackpressurePolicy,
}

Configuration for the event-driven connector pipeline.

Fields

max_poll_records: usize

Maximum records per poll_batch() call.

channel_capacity: usize

Channel capacity for per-source mpsc sender → coordinator.

fallback_poll_interval: Duration

Fallback poll interval when a source returns data_ready_notify() == None.

checkpoint_interval: Option<Duration>

Checkpoint interval (if checkpointing is enabled).

This controls the timer-based checkpoint mode (at-least-once). For exactly-once semantics, use barrier-aligned checkpoints via PipelineCallback::checkpoint_with_barrier instead. Timer-based checkpoints capture offsets before operator state, which means on recovery the consumer replays from the offset and operators may re-process some records (at-least-once, not exactly-once).

batch_window: Duration

Coordinator micro-batch window.

After receiving the first event in a cycle, the coordinator sleeps for this duration to let more events accumulate in the channel before executing the SQL cycle. This bounds the number of SQL executions per second while preserving low latency.

During this window the coordinator does not drain the channel, so the bounded channel provides natural backpressure to source tasks (they block on tx.send().await when the channel fills).

Set to Duration::ZERO to execute immediately (no batching).

barrier_alignment_timeout: Duration

Maximum time to wait for all sources to align on a checkpoint barrier before cancelling the checkpoint.

delivery_guarantee: DeliveryGuarantee

End-to-end delivery guarantee for the pipeline.

Validated at startup: ExactlyOnce requires all sources to support replay, all sinks to support exactly-once, and checkpointing to be enabled. See DeliveryGuarantee for details.

cycle_budget_ns: u64

Maximum wall-clock time for a single processing cycle (nanoseconds). Logged at DEBUG when exceeded. Default: 10ms.

drain_budget_ns: u64

Maximum wall-clock time for the message drain phase (nanoseconds). The drain loop terminates early when this budget is exhausted. Default: 1ms.

query_budget_ns: u64

Maximum wall-clock time for per-query execution within a cycle (nanoseconds). When elapsed time exceeds this budget, remaining queries are deferred to the next cycle. At least one query always executes for forward progress. Default: 8ms.

background_budget_ns: u64

Maximum wall-clock time for background work (barriers, checkpoint, table polling) after SQL execution (nanoseconds). When exceeded, low-priority tasks (table polling) are skipped. Default: 5ms.

max_input_buf_batches: usize

Per-input-port batch cap. Default: 256.

max_input_buf_bytes: Option<usize>

Per-input-port byte cap. None = disabled.

backpressure_policy: BackpressurePolicy

What to do when either cap is exceeded.

Trait Implementations

impl Clone for PipelineConfig

fn clone(&self) -> PipelineConfig

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for PipelineConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for PipelineConfig

fn default() -> Self

Returns the “default value” for a type.