Skip to main content

Module shuffle

Module shuffle 

Source
Expand description

Cross-instance shuffle: message codec, credit flow, wire protocol. Cross-instance shuffle over Tonic gRPC client-streaming: one client-streaming call per peer pair carrying ShuffleMessage frames. Backpressure is the HTTP/2 flow-control window plus the bounded crossfire inbound queue on the consuming side. The real transport is compiled under cluster; the default build keeps a networking-free shim.

Re-exports§

pub use message::ShuffleMessage;
pub use routing::logical_batch_bytes;
pub use routing::route_checkpointed_batch;
pub use routing::row_vnodes;
pub use routing::CheckpointRoutePlan;
pub use routing::LocalRoute;
pub use routing::RemoteRoute;
pub use routing::ShuffleRoutingError;
pub use routing::ROUTE_MAX_BATCH_BYTES;
pub use routing::ROUTE_MAX_BATCH_ROWS;
pub use routing::ROUTE_TARGET_BATCH_BYTES;
pub use transport::is_scope_cancelled;
pub use transport::ReceivedBatch;
pub use transport::ReceivedFrontierCut;
pub use transport::ReceivedShuffle;
pub use transport::ShuffleBatchAdmission;
pub use transport::ShufflePeerId;
pub use transport::ShuffleReceiver;
pub use transport::ShuffleSender;
pub use transport::is_shuffle_send_stalled;
pub use transport::shuffle_send_may_have_been_admitted;
pub use transport::ShufflePeerFlow;
pub use transport::SHUFFLE_ADDR_KEY;

Modules§

message
Logical messages carried by the ordered shuffle transport.
routing
Row-to-vnode routing shared by checkpointed cluster shuffle paths.
transport
Cross-node shuffle transport over Tonic gRPC client-streaming.

Structs§

ShuffleTopologyFence
Exact logical topology and catalog digest, independent of assignment and recovery counters. Installing this transport fence is a trusted control-path action, not an output or Release permit.

Constants§

SHUFFLE_NODE_BUDGET_BYTES
Data bytes a node’s shuffle endpoints hold in each direction, summed over its peers.