Frame Layer: Binary Framing Protocol

Loading…

Frame Layer: Binary Framing Protocol

Relevant source files

• core/README.md
• core/src/constants.rs
• core/src/frame.rs
• core/src/frame/README.md
• core/src/frame/frame_codec.rs
• core/src/frame/frame_error.rs
• core/src/frame/frame_kind.rs
• core/src/frame/frame_mux_stream_decoder.rs
• core/src/frame/frame_stream_encoder.rs
• core/src/frame/frame_struct.rs

The Frame Layer is the foundational transport-agnostic layer of the Muxio library. It provides a binary-first protocol for multiplexing multiple logical streams over a single physical connection (e.g., TCP, WebSockets, or UDP). It handles serialization, chunking, sequencing, and out-of-order reassembly of binary data.

1. Data Structures & Constants

The core unit of communication is the Frame struct core/src/frame/frame_struct.rs:13-48 Every frame contains a fixed-size header followed by a variable-length payload.

Frame Structure

FrameKind Enum

The FrameKind core/src/frame/frame_kind.rs:5-12 determines the lifecycle state of a logical stream:

• Open (0): Initiates a stream; may contain initial data core/src/frame/frame_kind.rs6
• Data (1): Standard data transmission core/src/frame/frame_kind.rs7
• End (2): Graceful termination; no further frames will be sent core/src/frame/frame_kind.rs8
• Cancel (3): Immediate termination; remaining buffered frames should be discarded core/src/frame/frame_kind.rs9
• Ping (5) / Pong (4): Connectivity and latency checks core/src/frame/frame_kind.rs:10-11

Binary Layout

The protocol uses a fixed header size (21 bytes) core/src/constants.rs7 All multi-byte integers are encoded in Little-Endian format core/src/frame/frame_codec.rs:38-44

Sources: core/src/frame/frame_struct.rs:13-48 core/src/frame/frame_kind.rs:5-12 core/src/frame/frame_codec.rs:34-48 core/src/constants.rs:1-7

2. Encoding and Decoding Logic

The FrameCodec core/src/frame/frame_codec.rs18 provides the stateless logic for converting between Frame objects and raw bytes.

• encode(&Frame) -> Vec<u8>: Serializes the header and appends the payload. It pre-allocates the exact capacity needed (FRAME_HEADER_SIZE + payload.len()) to avoid multiple reallocations core/src/frame/frame_codec.rs:34-48
• decode(&[u8]) -> Result<DecodedFrame, FrameDecodeError>: Validates that the buffer is at least FRAME_HEADER_SIZE core/src/frame/frame_codec.rs:69-71 It returns a DecodedFrame which wraps the Frame and an optional error core/src/frame/frame_struct.rs:51-54

Error Types

Errors are categorized into FrameEncodeError core/src/frame/frame_error.rs:4-12 and FrameDecodeError core/src/frame/frame_error.rs:27-37 Key errors include:

• CorruptFrame: Data does not match the expected protocol format core/src/frame/frame_error.rs5 core/src/frame/frame_error.rs28
• IncompleteHeader: The buffer provided to the decoder is too small to read the header or the full payload core/src/frame/frame_error.rs36
• ReadAfterCancel / WriteAfterCancel: Attempting I/O on a stream that has been terminated via Cancel core/src/frame/frame_error.rs11 core/src/frame/frame_error.rs33

Sources: core/src/frame/frame_codec.rs:9-128 core/src/frame/frame_error.rs:1-54 core/src/frame/frame_struct.rs:51-54

3. Stream Management

FrameStreamEncoder

The FrameStreamEncoder core/src/frame/frame_stream_encoder.rs:11-23 converts a continuous stream of bytes into a sequence of Frame objects for a single stream_id.

1. Chunking : It accepts arbitrary byte slices via write_bytes and splits them into chunks of max_chunk_size core/src/frame/frame_stream_encoder.rs:63-91
2. State Tracking : It maintains the next_seq_id and transitions the next_kind from Open to Data after the first frame is emitted core/src/frame/frame_stream_encoder.rs:86-87
3. Callback Emission : Encoded bytes are passed to an on_emit closure, allowing integration with various transports (e.g., writing to a WebSocket) core/src/frame/frame_stream_encoder.rs:47-59

FrameMuxStreamDecoder

The FrameMuxStreamDecoder core/src/frame/frame_mux_stream_decoder.rs:31-34 is a stateful, multiplexed reassembler. Unlike the encoder, one decoder handles all active streams on a connection.

1. Buffering : It maintains a Vec<u8> for partial frames that haven’t fully arrived across the wire core/src/frame/frame_mux_stream_decoder.rs32 core/src/frame/frame_mux_stream_decoder.rs71
2. Reassembly : It uses a HashMap<u32, StreamReassembly> to track next_expected sequence IDs for every stream core/src/frame/frame_mux_stream_decoder.rs33 core/src/frame/frame_mux_stream_decoder.rs:36-41
3. Out-of-Order Handling : Frames arriving out of order are stored in a BTreeMap keyed by seq_id core/src/frame/frame_mux_stream_decoder.rs38 They are only yielded to the application via FrameDecoderIterator once all preceding frames for that stream have been processed core/src/frame/frame_mux_stream_decoder.rs:139-142

Sources: core/src/frame/frame_stream_encoder.rs:11-166 core/src/frame/frame_mux_stream_decoder.rs:31-158

4. System Flow Diagrams

Entity Mapping: Logical to Code

This diagram maps the conceptual framing protocol to the specific Rust entities implementing it.

Sources: core/src/frame/frame_mux_stream_decoder.rs:31-41 core/src/frame/frame_struct.rs:13-48 core/src/frame/frame_stream_encoder.rs:11-23

graph TD
    subgraph "Logical Protocol"
        ["Binary Stream"] --> ["Multiplexed Frames"]
        ["Multiplexed Frames"] --> ["Stream ID 100"]
        ["Multiplexed Frames"] --> ["Stream ID 200"]
end

    subgraph "Code Entity Space (core/src/frame/)"
        direction LR
        ["FrameMuxStreamDecoder"] -- "manages" --> ["StreamReassembly"]
        ["FrameCodec"] -- "creates" --> ["Frame"]
        ["FrameStreamEncoder"] -- "uses" --> ["FrameCodec"]
end

    ["Multiplexed Frames"] -- "processed by" --> ["FrameMuxStreamDecoder"]
    ["Frame"] -- "defines" --> ["Multiplexed Frames"]
    ["Stream ID 100"] -- "tracked by" --> ["StreamReassembly"]

sequenceDiagram
    participant App as "Application Layer"
    participant Enc as "FrameStreamEncoder"
    participant Codec as "FrameCodec"
    participant Net as "Physical Transport (TCP/WS)"
    participant MuxDec as "FrameMuxStreamDecoder"

    App->>Enc: write_bytes(data)
    Note over Enc: Chunks data by\nmax_chunk_size [core/src/frame/frame_stream_encoder.rs:73]
    Enc->>Codec: encode(Frame)
    Codec-->>Enc: Vec<u8> [core/src/frame/frame_codec.rs:34]
    Enc->>Net: on_emit(bytes) [core/src/frame/frame_stream_encoder.rs:56]
    
    Net->>MuxDec: read_bytes(bytes) [core/src/frame/frame_mux_stream_decoder.rs:70]
    MuxDec->>Codec: decode(chunk) [core/src/frame/frame_mux_stream_decoder.rs:94]
    Codec-->>MuxDec: DecodedFrame [core/src/frame/frame_codec.rs:121]
    
    alt Out of Order
        MuxDec->>MuxDec: Buffer in BTreeMap [core/src/frame/frame_mux_stream_decoder.rs:137]
    else In Order
        MuxDec-->>App: Iterator::next() -> Frame [core/src/frame/frame_mux_stream_decoder.rs:50]
    end

Data Flow: Encoding to Decoding

The following diagram illustrates the lifecycle of data as it passes through the Frame Layer.

Sources: core/src/frame/frame_stream_encoder.rs:63-91 core/src/frame/frame_mux_stream_decoder.rs:70-157 core/src/frame/frame_codec.rs:20-128

5. Summary of Key Functions

Sources: core/src/frame/frame_codec.rs:1-128 core/src/frame/frame_stream_encoder.rs:1-166 core/src/frame/frame_mux_stream_decoder.rs:1-158