Expand description
Cross-instance shuffle: message codec, credit flow, wire protocol.
Cross-instance shuffle over Tonic gRPC client-streaming: one
client-streaming call per peer pair carrying ShuffleMessage frames.
Backpressure is the HTTP/2 flow-control window plus the bounded crossfire
inbound queue on the consuming side. The real transport is compiled under
cluster; the default build keeps a networking-free shim.
Re-exports§
pub use message::ShuffleMessage;pub use routing::logical_batch_bytes;pub use routing::route_checkpointed_batch;pub use routing::row_vnodes;pub use routing::CheckpointRoutePlan;pub use routing::LocalRoute;pub use routing::RemoteRoute;pub use routing::ShuffleRoutingError;pub use routing::ROUTE_MAX_BATCH_BYTES;pub use routing::ROUTE_MAX_BATCH_ROWS;pub use routing::ROUTE_TARGET_BATCH_BYTES;pub use transport::is_scope_cancelled;pub use transport::ReceivedBatch;pub use transport::ReceivedFrontierCut;pub use transport::ReceivedShuffle;pub use transport::ShuffleBatchAdmission;pub use transport::ShufflePeerId;pub use transport::ShuffleReceiver;pub use transport::ShuffleSender;pub use transport::is_shuffle_send_stalled;pub use transport::shuffle_send_may_have_been_admitted;pub use transport::ShufflePeerFlow;pub use transport::SHUFFLE_ADDR_KEY;
Modules§
- message
- Logical messages carried by the ordered shuffle transport.
- routing
- Row-to-vnode routing shared by checkpointed cluster shuffle paths.
- transport
- Cross-node shuffle transport over Tonic gRPC client-streaming.
Structs§
- Shuffle
Topology Fence - Exact logical topology and catalog digest, independent of assignment and recovery counters. Installing this transport fence is a trusted control-path action, not an output or Release permit.
Constants§
- SHUFFLE_
NODE_ BUDGET_ BYTES - Data bytes a node’s shuffle endpoints hold in each direction, summed over its peers.