Expand description
Simplified pipeline coordinator.
Sources push directly to the coordinator via crossfire::mpsc.
The coordinator runs on a dedicated single-threaded tokio runtime
(laminar-compute thread), isolating CPU-bound event processing
from IO tasks (Kafka poll, S3 checkpoint writes, HTTP) on the main
work-stealing runtime. SQL execution is delegated to PipelineCallback.
§Architecture
Source task (main tokio runtime)
│ connector.poll_batch().await
│
└──── MAsyncTx ────► StreamingCoordinator (dedicated compute thread)
│ callback.execute_cycle()
│ callback.write_to_sinks()
▼
SinksStructs§
- Streaming
Coordinator - Simplified pipeline coordinator — single tokio task, no core threads.