Binary Framing Protocol

Relevant source files

• DRAFT.md
• README.md
• src/rpc/rpc_internals/rpc_session.rs
• src/rpc/rpc_internals/rpc_stream_decoder.rs

Purpose and Scope

The Binary Framing Protocol is the foundational layer of the muxio system that enables efficient stream multiplexing over a single connection. This protocol defines the low-level binary format for chunking data into discrete frames, assigning stream identifiers, and managing frame lifecycle (start, payload, end, cancel). It operates below the RPC layer and is completely agnostic to message semantics.

For information about how the RPC layer uses these frames for request/response correlation, see RPC Dispatcher. For details on RPC-level message structures, see Request and Response Types.

Sources: README.md:28-29 DRAFT.md:11-21

Overview

The binary framing protocol provides a minimal, low-overhead mechanism for transmitting multiple independent data streams over a single bidirectional connection. Each frame contains:

• A stream identifier to distinguish concurrent streams
• A frame type indicating the frame's purpose in the stream lifecycle
• A payload containing application data

This design enables interleaved transmission of frames from different streams without requiring separate network connections, while maintaining correct reassembly on the receiving end.

Sources: README.md:20-32 src/rpc/rpc_internals/rpc_session.rs:15-24

Frame Structure

Basic Frame Layout

Frame Structure Diagram

Each frame consists of a header section and a payload section. The header contains metadata for routing and lifecycle management, while the payload carries the actual application data.

Sources: src/rpc/rpc_internals/rpc_session.rs:61-116

Frame Header Fields

The header provides the minimal information needed to route frames to the correct stream decoder and manage stream lifecycle. Additional protocol-specific headers (such as RPC headers) are encoded within the frame payload itself.

Sources: src/rpc/rpc_internals/rpc_session.rs:66-68 src/rpc/rpc_internals/rpc_stream_decoder.rs:53-56

Frame Types (FrameKind)

The protocol defines four frame types that manage stream lifecycle:

Frame Type Enumeration

Frame Lifecycle State Machine

The Start frame initiates a stream and typically carries protocol headers (such as RPC headers) in its payload. Subsequent Payload frames carry chunked data. The stream terminates with either an End frame (normal completion) or a Cancel frame (abnormal termination).

Sources: src/rpc/rpc_internals/rpc_session.rs:98-100 src/rpc/rpc_internals/rpc_stream_decoder.rs:156-167

Stream Multiplexing

sequenceDiagram
    participant App as "Application"
    participant Session as "RpcSession"
    participant Encoder as "RpcStreamEncoder"
    
    App->>Session: init_request()
    Session->>Session: stream_id = next_stream_id
    Session->>Session: next_stream_id++
    Session->>Encoder: new(stream_id, ...)
    Encoder-->>Session: RpcStreamEncoder
    Session-->>App: encoder with stream_id
    
    Note over Session: Each request gets unique stream_id

Stream ID Allocation

Each stream is assigned a unique u32 identifier by the RpcSession. Stream IDs are allocated sequentially using an incrementing counter, ensuring that each initiated stream receives a distinct identifier.

Stream ID Allocation Flow

Sources: src/rpc/rpc_internals/rpc_session.rs:35-50

Concurrent Stream Handling

The FrameMuxStreamDecoder receives interleaved frames from multiple streams and demultiplexes them based on stream_id. Each stream maintains its own RpcStreamDecoder instance in a HashMap, enabling independent decoding state management.

Stream Demultiplexing Architecture

graph LR
    Input["Incoming Bytes"]
Mux["FrameMuxStreamDecoder"]
subgraph "Per-Stream Decoders"
        D1["RpcStreamDecoder\nstream_id: 1"]
D2["RpcStreamDecoder\nstream_id: 2"]
D3["RpcStreamDecoder\nstream_id: 3"]
end
    
    Events["RpcStreamEvent[]"]
Input --> Mux
 
   Mux -->|stream_id: 1| D1
 
   Mux -->|stream_id: 2| D2
 
   Mux -->|stream_id: 3| D3
    
 
   D1 --> Events
 
   D2 --> Events
 
   D3 --> Events

The session maintains a HashMap<u32, RpcStreamDecoder> where the key is the stream_id. When a frame arrives, the session looks up the appropriate decoder, creates one if it doesn't exist, and delegates frame processing to that decoder.

Sources: src/rpc/rpc_internals/rpc_session.rs:20-24 src/rpc/rpc_internals/rpc_session.rs:68-74

Frame Encoding and Decoding

Encoding Process

Frame encoding is performed by the RpcStreamEncoder, which chunks large payloads into multiple frames based on a configurable max_chunk_size:

Frame Encoding Process

graph TB
    Input["Application Data"]
Header["RPC Header\n(First Frame)"]
Chunker["Payload Chunker\n(max_chunk_size)"]
subgraph "Emitted Frames"
        F1["Frame 1: Start\n+ RPC Header\n+ Data Chunk 1"]
F2["Frame 2: Payload\n+ Data Chunk 2"]
F3["Frame 3: Payload\n+ Data Chunk 3"]
F4["Frame 4: End\n+ Data Chunk 4"]
end
    
    Transport["on_emit Callback"]
Input --> Header
 
   Input --> Chunker
    
 
   Header --> F1
 
   Chunker --> F1
 
   Chunker --> F2
 
   Chunker --> F3
 
   Chunker --> F4
    
 
   F1 --> Transport
 
   F2 --> Transport
 
   F3 --> Transport
 
   F4 --> Transport

The encoder emits frames via a callback function (on_emit), allowing the transport layer to send data immediately without buffering entire streams in memory.

Sources: src/rpc/rpc_internals/rpc_session.rs:35-50

Decoding Process

Frame decoding occurs in two stages:

1. Frame-level decoding by FrameMuxStreamDecoder: Parses incoming bytes into individual DecodedFrame structures
2. Stream-level decoding by RpcStreamDecoder: Reassembles frames into complete messages and emits RpcStreamEvents

sequenceDiagram
    participant Input as "Network Bytes"
    participant FrameDecoder as "FrameMuxStreamDecoder"
    participant Session as "RpcSession"
    participant StreamDecoder as "RpcStreamDecoder"
    participant Handler as "Event Handler"
    
    Input->>FrameDecoder: read_bytes(input)
    FrameDecoder->>FrameDecoder: Parse frame headers
    FrameDecoder-->>Session: DecodedFrame[]
    
    loop "For each frame"
        Session->>Session: Get/Create decoder by stream_id
        Session->>StreamDecoder: decode_rpc_frame(frame)
        StreamDecoder->>StreamDecoder: Parse RPC header\n(if Start frame)
        StreamDecoder->>StreamDecoder: Accumulate payload
        StreamDecoder-->>Session: RpcStreamEvent[]
        
        loop "For each event"
            Session->>Handler: on_rpc_stream_event(event)
        end
        
        alt "Frame is End or Cancel"
            Session->>Session: Remove stream decoder
        end
    end

Frame Decoding Sequence

The RpcSession.read_bytes() method orchestrates this process, managing decoder lifecycle and error propagation.

Sources: src/rpc/rpc_internals/rpc_session.rs:53-117 src/rpc/rpc_internals/rpc_stream_decoder.rs:53-186

RPC Frame Header Structure

While the core framing protocol is agnostic to payload contents, the RPC layer adds its own header structure within the frame payload. The first frame of each RPC stream (the Start frame) contains an RPC header followed by optional data:

RPC Header Layout

The total RPC frame header size is 15 + metadata_length bytes. This header is parsed by the RpcStreamDecoder when processing the first frame of a stream.

Sources: src/rpc/rpc_internals/rpc_stream_decoder.rs:60-125 src/rpc/rpc_internals/rpc_stream_decoder.rs:1-8

Frame Flow Through System Layers

Complete Frame Flow Diagram

This diagram illustrates how frames flow from application code through the framing protocol to the network, and how they are decoded and reassembled on the receiving end. The framing layer is responsible for the chunking and frame type management, while higher layers handle RPC semantics.

Sources: src/rpc/rpc_internals/rpc_session.rs:1-118 src/rpc/rpc_internals/rpc_stream_decoder.rs:1-187

stateDiagram-v2
    [*] --> AwaitHeader : New stream
    
    AwaitHeader --> AwaitHeader : Partial header (buffer incomplete)
    AwaitHeader --> AwaitPayload : Header complete (emit Header event)
    
    AwaitPayload --> AwaitPayload : Payload frame (emit PayloadChunk)
    AwaitPayload --> Done : End frame (emit End event)
    AwaitPayload --> [*] : Cancel frame (error + cleanup)
    
    Done --> [*] : Stream complete
    
    note right of AwaitHeader
        Buffer accumulates bytes
        until full RPC header
        can be parsed
    end note
    
    note right of AwaitPayload
        Each payload frame
        emits immediately,
        no buffering
    end note

Frame Reassembly State Machine

The RpcStreamDecoder maintains internal state to reassemble frames into complete messages:

RpcStreamDecoder State Machine

The decoder starts in AwaitHeader state, buffering bytes until the complete RPC header (including variable-length metadata) is received. Once the header is parsed, it transitions to AwaitPayload state, where subsequent frames are emitted as PayloadChunk events without additional buffering. The stream completes when an End frame is received or terminates abnormally on a Cancel frame.

Sources: src/rpc/rpc_internals/rpc_stream_decoder.rs:11-24 src/rpc/rpc_internals/rpc_stream_decoder.rs:59-183

Error Handling in Frame Processing

The framing protocol defines several error conditions that can occur during decoding:

When an error occurs during frame decoding, the RpcSession removes the stream decoder from its HashMap, emits an RpcStreamEvent::Error, and propagates the error to the caller. This ensures that corrupted streams don't affect other concurrent streams.

Sources: src/rpc/rpc_internals/rpc_session.rs:80-94 src/rpc/rpc_internals/rpc_stream_decoder.rs:165-166 src/rpc/rpc_internals/rpc_session.rs:98-100

Cleanup and Resource Management

Stream decoders are automatically cleaned up in the following scenarios:

1. Normal completion : When an End frame is received
2. Abnormal termination : When a Cancel frame is received
3. Decode errors : When frame decoding fails

The cleanup process removes the RpcStreamDecoder from the session's HashMap, freeing associated resources:

This design ensures that long-lived sessions don't accumulate memory for completed streams.

Sources: src/rpc/rpc_internals/rpc_session.rs:74-100

Key Design Characteristics

The binary framing protocol exhibits several important design properties:

Minimal Overhead : Frame headers contain only essential fields (stream_id and kind), minimizing bytes-on-wire for high-throughput scenarios.

Stream Independence : Each stream maintains separate decoding state, enabling true concurrent multiplexing without head-of-line blocking between streams.

Callback-Driven Architecture : Frame encoding emits bytes immediately via callbacks, avoiding the need to buffer entire messages in memory. Frame decoding similarly emits events immediately as frames complete.

Transport Agnostic : The protocol operates on &[u8] byte slices and emits bytes via callbacks, making no assumptions about the underlying transport (WebSocket, TCP, in-memory channels, etc.).

Runtime Agnostic : The core framing logic uses synchronous control flow with callbacks, requiring no specific async runtime and enabling integration with both Tokio and WASM environments.

Sources: README.md:30-35 DRAFT.md:48-52 src/rpc/rpc_internals/rpc_session.rs:35-50

Dismiss

Refresh this wiki

Enter email to refresh