Skip to main content

Module shuffle

Module shuffle 

Source
Expand description

Cross-instance shuffle: message codec, credit flow, wire protocol. Cross-instance shuffle over Tonic gRPC client-streaming: one client-streaming call per peer pair carrying ShuffleMessage frames. Backpressure is the HTTP/2 flow-control window plus the bounded crossfire inbound queue on the consuming side. The real transport is compiled under cluster; the default build keeps a networking-free shim.

Re-exports§

pub use message::ShuffleMessage;
pub use routing::route_checkpointed_batch;
pub use routing::row_vnodes;
pub use routing::CheckpointRoutePlan;
pub use routing::LocalRoute;
pub use routing::RemoteRoute;
pub use routing::ShuffleRoutingError;
pub use routing::ROUTE_MAX_BATCH_BYTES;
pub use routing::ROUTE_MAX_BATCH_ROWS;
pub use routing::ROUTE_TARGET_BATCH_BYTES;
pub use transport::SHUFFLE_ADDR_KEY;
pub use transport::is_scope_cancelled;
pub use transport::ReceivedBatch;
pub use transport::ReceivedShuffle;
pub use transport::ShuffleBatchAdmission;
pub use transport::ShufflePeerId;
pub use transport::ShuffleReceiver;
pub use transport::ShuffleSender;

Modules§

message
Logical messages carried by the ordered shuffle transport.
routing
Row-to-vnode routing shared by checkpointed cluster shuffle paths.
transport
Cross-node shuffle transport over Tonic gRPC client-streaming.