Skip to main content

Module transport

Module transport 

Source
Expand description

Cross-node shuffle transport over Tonic gRPC client-streaming.

Each sender opens a client-streaming Shuffle RPC per peer and pushes a forward-only stream of ShuffleFrames; the receiver runs the ShuffleTransport service, attributes every stream to the peer announced in its leading Hello, and surfaces decoded ShuffleMessages on a bounded crossfire MPSC queue. Backpressure is the HTTP/2 flow-control window plus that bounded queue. See super::message for the per-frame payloads and crate::serialization for the Arrow IPC (de)serialization of VnodeData.

The real gRPC path is compiled under the cluster feature (which pulls in tonic/prost). A default build keeps the same public API via a networking-free shim so the types referenced by laminar-db/laminar-server signatures still compile without the cluster dependencies.

Structs§

ShuffleReceiver
Inbound side of the shuffle fabric: a Tonic ShuffleTransport server surfacing every received frame, attributed to its sending peer, on the bounded crossfire queue.
ShuffleSender
Lazy pool of outbound client-streaming calls, keyed by peer id.

Constants§

SHUFFLE_ADDR_KEY
Gossip KV key used by ShuffleReceiver::bind_with_kv to publish the listener’s socket address, and by ShuffleSender to discover peer addresses on first contact. Value: the bound socket address formatted via SocketAddr::to_string().

Type Aliases§

ShufflePeerId
Peer-local identifier on the wire. Matches cluster::discovery::NodeId’s inner type for seamless conversion.