Module pipeline 

Thread-per-core connector pipeline. Thread-per-core connector pipeline.

Each source connector runs on a dedicated I/O thread with a single-threaded tokio runtime, pushing events through SPSC queues to CPU-pinned core threads. The TpcPipelineCoordinator drains core outboxes, runs SQL execution cycles, routes results to sinks, and manages checkpoint barriers.

Architecture

┌──────────┐     ┌──────────┐     ┌──────────┐
│ Source 0  │     │ Source 1  │     │ Source N  │   I/O threads
│ (thread)  │     │ (thread)  │     │ (thread)  │
└────┬─────┘     └────┬─────┘     └────┬─────┘
     │ SPSC           │ SPSC           │ SPSC
     ▼                ▼                ▼
┌──────────┐     ┌──────────┐     ┌──────────┐
│  Core 0   │     │  Core 1   │     │  Core M   │   Pinned threads
└────┬─────┘     └────┬─────┘     └────┬─────┘
     │ SPSC           │ SPSC           │ SPSC
     └───────┬────────┘────────────────┘
             ▼
    ┌─────────────────────┐
    │ TpcPipelineCoordinator │  tokio task: SQL exec, sinks, checkpoints
    └─────────────────────┘

Re-exports

pub use callback::PipelineCallback;

pub use callback::SourceRegistration;

pub use config::PipelineConfig;

pub use tpc_coordinator::TpcPipelineCoordinator;

pub use tpc_runtime::TpcRuntime;

Modules

callback

Pipeline callback trait and source registration types.

config

Pipeline configuration.

source_adapter

Source I/O thread adapter for thread-per-core mode.

tpc_coordinator

TPC-mode pipeline coordinator.

tpc_runtime

Thread-per-core runtime managing N CoreHandles and their source I/O threads.