Skip to main content

Module message

Module message 

Source
Expand description

Logical messages carried on a shuffle stream.

The wire encoding is gRPC/protobuf (proto/shuffle.proto, ShuffleFrame); the <-> conversion lives in super::transport. A VnodeData’s batch is Arrow IPC-encoded with a per-stage streaming encoder (see crate::serialization::BatchStreamEncoder): the schema rides only the first frame of each stage and later frames are schema-less continuations. This assumes a stage’s schema is stable for the life of a connection.

Enums§

ShuffleMessage
Logical message carried on a shuffle connection.

Constants§

MAX_PAYLOAD_BYTES
Maximum Arrow IPC payload accepted for a single VnodeData frame: 64 MiB.