04. Client protocol: sessions, packets, messages, responses, reliability, rate limits

Draft 0.2, 2026-11-13. Normative. The byte layouts are in 01-conventions.md §6 and this chapter does not change them. This chapter gives their semantics, the state machines on both sides, the order of checks, and the test vectors. Local names use the prefix PROTO_. 01 §14 is binding here: where this chapter and 01 §14 differ, 01 §14 wins.

The latency contract, from the client's side (01 §14.0, REQ-LAT-1). Every accepted packet of 1 to 10 messages gets the first record of every message, at every index 0–9 and of every type, within LAT_MAX_NS = 1 µs from its first bit in. This is a maximum, not a percentile, and it has no rate, size or position condition. A packet that the exchange cannot serve in time gets Reject BUSY within the same 1 µs (§3.5). A packet that is the session's next valid packet has no third outcome. The only window without a 1 µs answer is the handoff window of 01 §14.0 item 2 (H6). On AWS the 1 µs is measured inside the instance (01 §14.0 item 1).

0. Scope and constraints#

This chapter covers the order-entry path between a client and the home that runs the book:

• request packets from a client to the sequencer and to the data tier;
• response packets from the sequencer (and from the recovery service) to the client;
• the session life cycle, the keys, the admin channel that opens a BASE-mode session;
• reliability across loss, failover and epoch handoff;
• rate limits and the message fee classes.

It does not cover market data or drop copy (10-accounts-risk-md.md), the order stream between the sequencer and the replicas (07-replication.md), or the on-chain session transaction (09-chain.md).

How the protocol meets the hard constraints:

1. Decisions at a glance#

2. Transport#

2.1 Why UDP, and not TCP or kernel-bypass TCP#

UDP is the transport. The reasons, strongest first:

1. The protocol needs application-level sequencing anyway. A TCP connection ends at one box. When the sequencer fails over (≤ FAILOVER_TARGET_MS) or the book moves home every EPOCH_HOURS, the client must resend its unacknowledged packets to a different box and the new box must deduplicate them. That needs pkt_seq, resp_seq and resend rules at the application layer (§5). With those in place, TCP's own retransmission is a second, redundant reliability layer.
2. Head-of-line blocking. TCP delivers bytes in order. One lost segment stalls every later message until a retransmission arrives. The Linux minimum RTO is 200 ms by default (vendor: kernel default) and ≥ 1 ms when tuned. A stall of that size breaks LAT_MAX_NS for every message behind the loss. With UDP, the lane rejects the packets behind a gap in tens of nanoseconds and the client resends after one round trip (§5.1).
3. Lane cost. TCP needs per-connection send and receive state, ACK generation, window updates and timers. For MAX_SESSIONS_PER_LANE sessions this is extra state and extra packets (ACKs) on a core that has a budget of LANE_PASS1_NS + LANE_APPLY_NS per message. UDP needs one compare on pkt_seq.
4. Message boundaries. The MAC covers exactly one packet, and count fixes the length. On a byte stream, the lane would need a framing layer before it could check either.
5. Kernel-bypass TCP does not remove 2–4. Stacks such as Onload or TCPDirect remove the syscall cost. They keep the stream semantics, the head-of-line blocking and the per-connection state, and they run their timers outside the order stream.

IPv4, not IPv6: the IPv4 header is 20 bytes shorter. At 100M packets/s that is 16 Gbps per direction (computed). Every COTS NIC filter and the AWS ENA path support IPv4. IPv6 is a v2 item (§12).

The admin channel (§4.3) is TLS 1.3 over TCP. It is never on the 1 µs path, so the cost of TCP does not matter there.

2.2 Endpoints and ports#

Rules:

• The client sends a session's packets only to its lane's port. The lane is fixed for the life of the session and is encoded in the SessionId (§4.1).
• The NIC steers each packet by (destination address, destination port) to the lane's RX queue. On REF, 06-platform.md assigns lanes to NIC ports, and the key bundle carries the resulting address for each lane.
• On AWS, ENA steers by RSS only. A packet can land on the wrong lane, which forwards it to the owning lane over an SPSC ring (+100 ns, budget, from DESIGN §4). The client does nothing different.
• Responses always go to the session's bound address (§4.2), never to the source address of the packet that caused them. So a replayed or spoofed packet cannot redirect any response.
• Responses come from the order address of the box that sends them. After a failover the source changes. The client accepts responses from any box listed in its key bundle and selects the tag key by the source box (§4.4).

2.3 Sizes and MTU#

Decisions:

• The standard 1500-byte MTU is enough. No jumbo frames are needed, so every COTS NIC and every cloud network carries the protocol unchanged.
• No fragmentation. Clients set DF. The lane drops IPv4 fragments and packets with IPv4 options, and counts them (PROTO_CNT_MALFORMED).
• The UDP checksum is optional for the client (IPv4 allows 0). The MAC covers integrity. The NIC verifies a nonzero checksum in hardware and drops a packet that fails.
• Wire load (computed): 100M full requests/s is 100M × 258 B × 8 = 206 Gbps, which loads 4 × 100GbE to 52%. This matches DESIGN §4.

2.4 The dual send#

Every request packet goes to two places:

ASCII original

 client ──(1) sequencer copy ──► vyrn-seq lane l      (latency path; decides acceptance)
   │
   └────(2) data copy ─────────► vyrn-data node d     (payload path; feeds the replicas)
                                     │
                                     └──► replicas fetch by PktHash, vote when complete

Client obligations:

Why D3. A packet that the sequencer never answered may still have been accepted, and its data copy exists either way. If the client could reuse the pkt_seq for other content, the log would hold two packets under one name. So the client never does that. To change its mind, it sends a new packet that cancels or replaces.

When the copies differ. 07-replication.md detects it by PktHash: the replicas look up the payload named by the PacketRecord, and a missing or different payload starts its check. The outcomes, as the client sees them:

Note for 07: two transmissions of one pkt_seq have different PktHash values when only flags bit0 or epoch16 differ. Equivocation is therefore a difference in logical content (D3), not a difference in PktHash. See Requests to 01.

3. The validation pipeline in the lane#

3.1 Order of checks#

The lane runs these checks in this order on every received packet. The cheapest check runs first, and no check that can close a session runs before the checks that prove the packet is for this session, from its bound address, and in its epoch. All costs are budgets on REF at 5 GHz. Benchmark B2 (11-build.md) measures them. V0–V8 are the packet part of pass 1 (01 §14.2): together they fit the 45 ns per packet of LANE_PASS1_NS. Test D (§3.5) is computed at dequeue; a shed is taken right after V5.

Why this order:

• V3 before V6. A packet from an unbound address never costs a SipHash. That is the DoS defence (§3.4). It also gives 01 §12.5 its meaning: a bad MAC from an unbound address is dropped and counted at V3, and the session stays open. A bad MAC at V6 can only come from the bound address with the receiver's own epoch, so it closes the session (01 §14.8).
• V4 before V6. The tag key depends on the epoch. An honest client that is a few milliseconds late at a boundary sends a packet tagged with last epoch's key. If V6 ran first, that packet would fail the MAC and close an honest session. With V4 first, the client gets EPOCH_MISMATCH and switches keys. Sessions survive the boundary (01 §14.8), so this matters for every session at every handoff.
• V5 before V6. A far-out pkt_seq cannot come from an honest client. Dropping it costs one compare and closes nothing.
• V6 before every response that is not EPOCH_MISMATCH. The lane never answers an unauthenticated packet, except at V4 (where it cannot authenticate) and then only under the holdoff of §3.2.
• Acceptance before V9–V11. Once a packet passes V6 and V8, it is authenticated and it is the next packet. The lane accepts it and records it even if its contents are wrong. If the lane dropped it, the client would have to resend a different packet under the same pkt_seq, which D3 forbids. Every rejection after acceptance is an in-stream record.

Speculative work. The lane may parse and classify the messages while SipHash runs. It changes no state and forwards nothing to the BBO core until V6 passes. This hides most of the MAC time on the cross path.

Two passes (01 §14.2). Pass 1 runs M1–M8 on every message in index order and pushes each improve, cross, IOC, FOK and post-only-would-cross to the BBO ring as soon as it is classified, with its in-packet dependency list and pre-assigned handles. Pass 2 applies the joins and cancels in index order. A cross at index i waits only for the pass-1 work of messages 0 … i−1 (5 ns each), not for their apply. Worst case, index 9 of a full packet (budget, REF): cross fill 880 ns, join or cancel ack 965 ns (955 typical), against the 980 ns shed threshold (DEADLINE_GUARD_NS = 20, 01 §14.17.1). A pooled packet adds 20 ns for the account token. Index 0 is ~90 ns faster, and an ack's slack grows by ~9 ns per earlier index, so the SDK puts a packet's cancels first (index 0 has ~100 ns of slack). Message order is the client's choice, so this changes no semantics. Every index meets LAT_MAX_NS.

Two passes do not change outcomes. The records of every message are exactly those that applying the messages one at a time, in index order, would produce: the same fills, prices, quantities, STP outcomes, rejects and handles. The dependency list makes the BBO core treat each earlier op of the packet as resting at its own priority. A client can rely on the outcome of index order. It cannot rely on the arrival order of records across the B-packet and the L-packet (R-ACROSS, §7.2); inside one response packet, records are in index order (R-INDEX).

fn on_rx(&mut self, p: &RxPkt) {
    let late = self.must_shed(p);                                           // D, §3.5, at dequeue
    let n = p.data[8] as usize;
    if n == 0 || n > MAX_MSGS_PER_PKT || p.len != 32 + 16 * n { return self.cnt.malformed.inc(); }
    let sid = SessionId(le_u32(&p.data[0..4]));
    if !self.serves(sid.lane()) { return self.other_lane(p, sid); }          // §3.2, §4.7
    let s = &mut self.sessions[sid.slot()];
    if s.gen != sid.gen() || s.state == State::Free { return self.cnt.unknown_session.inc(); }
    if p.src != s.bound { return s.cnt.unbound_src.inc(); }
    if le_u16(&p.data[10..12]) != self.epoch.low16() { return self.oos(s, p, EPOCH_MISMATCH, self.epoch.0); }
    let seq = le_u32(&p.data[4..8]);
    if !s.seq_in_window(seq) { return s.cnt.seq_window.inc(); }
    if late { return self.busy(s, p, seq); }   // PacketRecord.flags bit0 SHED + Reject BUSY, ~20 ns
    let end = 16 + 16 * n;
    if siphash24(&s.k_seq, &p.data[..end]) != le_u64(&p.data[end..end + 8]) {
        return self.close(s, Close::BadMac { pkt_seq: seq });               // in-stream, final
    }
    if s.state == State::Closed { return self.oos(s, p, SESSION_CLOSED, s.close_cause as u32); }
    match seq.cmp(&s.expected) {
        Ordering::Less => self.duplicate(s, p, seq),                          // §5.2
        Ordering::Greater => self.oos(s, p, BAD_SEQ, s.expected),             // §5.1
        Ordering::Equal => self.accept(s, p, seq),                            // V9..V11
    }
}

3.2 Out-of-stream responses and the other-lane path#

An out-of-stream response answers a packet that the lane did not accept. It has:

• resp_seq = 0, count = 1, flags = 0, echo_pkt = the pkt_seq of the packet it answers;
• one Reject record with index 0xFF (the whole packet), the reason, the detail, and client_tag = 0;
• a tag with K_out(e, this box), and the session's bound address as destination.

Rules:

• The lane sends at most one out-of-stream response per (session, reason) per PROTO_OOS_HOLDOFF_US. Later ones in that window are only counted. BUSY is the exception: it has no holdoff, and every shed packet gets its own (§3.5).
• Out-of-stream responses are not retained, not resent, and not part of any state. Except for BUSY, they are not engine output: they depend on packets that the order stream does not contain. Their frame gate (01 §14.5) is empty, or for BUSY its PacketRecord with flags bit 0 (SHED) set (01 §6.6; bit 1 is ADVANCE, a count = 0 record that no client packet produces).
• The five out-of-stream reasons are EPOCH_MISMATCH, BAD_SEQ, SESSION_CLOSED, WRONG_LANE and BUSY (lane shed, index 0xFF). No other reason is ever sent out of stream.

Other-lane path. A packet whose session names another lane goes to a slow path with a budget of PROTO_WRONG_LANE_PPS per lane. Above that budget the lane drops and counts.

• Port steering (REF, CONSUMER). The client sent to the wrong port. The lane reads the owning lane's session slot (read-only; keys do not change while a session exists), runs V3–V6 against it, and if they pass, sends out-of-stream WRONG_LANE with detail = the correct LaneId. It never accepts the packet. A V6 failure here is dropped and counted: only the owning lane can close a session (INV-WRITER).
• RSS steering (AWS). The NIC put the packet on the wrong lane, and the client did nothing wrong. The lane forwards the packet to the owning lane over an SPSC ring (05-engine.md) and sends no response. The owning lane uses the original rx_ts.

3.3 Per-message checks#

For each message, in index order, the lane runs these checks. The first failure decides the reject reason. The order is fixed, so replicas reproduce the same reason.

Budget per message (01 §14.2): M1–M8 and the forward push are pass 1, LANE_PASS1_NS = 5 ns. M9 for a join or a cancel is pass 2, LANE_APPLY_NS = 9 ns. LANE_NS_PER_MSG = 30 is the average with every amortized cost (risk misses, overflow levels, settles) and is what admission (§9.1) uses; it is not a per-message latency term.

A rejected message changes no order state. It does consume its message token (M1), so a flood of invalid messages is limited like any other flood.

3.4 Spoofed sources and denial of service#

The lane alone cannot absorb a full-rate flood. One 100GbE port carries up to 148.8M minimum frames per second (computed), and at 5 ns per drop one lane handles 200M/s, but the flood still fills the lane's RX ring and delays real packets. So the edge does the first filtering, with fixed-function features of a merchant switch (C-NOCHIP):

With E1–E3, the only party that can send from a session's bound address is the owner of that address's link. So a bad MAC from the bound address is the owner's bug or compromise, and closing the session at the first failure is correct. It also limits a key-guessing attacker to one guess per session.

3.5 Deadline admission and BUSY (01 §14.1)#

Test D. When the lane dequeues a packet it computes age = now − rx_ts (TSC↔PHC mapping, 06-platform.md; on AWS rx_ts is the TSC at RX completion, 01 §14.11). It sheds the packet if either holds:

1. age + predicted_path_ns(p) > LAT_MAX_NS − DEADLINE_GUARD_NS. For a packet with forwarded ops, the prediction includes the BBO-ring backlog that the lane can see.
2. Serving p fully would leave a packet behind it unable to get its own BUSY in time: for some pending packet k (those in the RX ring, plus the most that E4 admits during p's work), age_k + L_p + PROTO_SHED_NS × k + PROTO_BUSY_TAIL_NS > LAT_MAX_NS − DEADLINE_GUARD_NS, where L_p is p's lane work and k its position behind p.

Why every BUSY arrives within 1 µs. Rule 2 holds the backlog where every pending packet can still be shed in time. A shed costs PROTO_SHED_NS = 20 ns, and E4 admits at most one packet per 25 ns, so the backlog cannot grow while the lane sheds. A lane BUSY waits for the logged cut like an ack, because its SHED record is frame-gated: PROTO_BUSY_TAIL_NS = 460 (cut 170 + LaneCut poll 20 + gate 10 + lane TX 20 + NIC TX 240). Rule 2 keeps every pending packet's dequeue age ≤ 500 ns (03 ARC_ADMIT_AGE_MAX_NS), so the worst case (budget, REF) is RX 320 + wait 180 + shed 20 + 460 = 980 ns, the threshold. The BBO core applies the same two rules to forwarded ops (05-engine.md). Its BUSY path, from the Shed step, is 10 + one BBO → lane hop 100 + lane 40

• NIC TX 240 = 390 ns, now 35 ns longer than the 355 ns fill path from a step. So a head that cannot start its step now is shed once its age + 390 + 15 > 980 (03 §6.0), and every BBO BUSY also arrives in time.

The two kinds of BUSY, as the client sees them:

A later message of the same packet that lists a shed op in its dependency list, or names the handle pre-assigned to it, is shed with it and gets its own Reject BUSY. Every shed decision is read from the log on replay, never recomputed, so INV-DET holds. A lane shed skips V6 because it changes no state: no packet is applied and nothing can close. The BUSY goes only to the bound address, tagged with K_out.

BUSY, gapless pkt_seq and deduplication.

• A lane BUSY never acknowledges anything. It carries resp_seq 0, so the cumulative rule of §5.1 is unchanged. Its detail = expected: every pkt_seq below it was accepted.
• After a lane shed of p, packets p + 1, … of the session are shed too while the lane sheds (each with its own BUSY, detail p), or are gaps once it stops (§5.1). Either way the client goes back to p. Go-back-N needs no special case.
• A resend of a shed p is a first acceptance, not a duplicate. G1 holds: p executes once.
• Test D runs before V8, so a duplicate can be shed too. Then detail > p tells the client that p was accepted earlier. It recovers p's responses with ResendReq (§5.3).
• What each lane class can expect at the reference workload is §3.6. BUSY answers overload within 1 µs. It is never a normal outcome, and a client counts it.

3.6 Lane classes and LANE_PACE (01 §14.15, §14.17.2)#

The lane map in the key bundle (§4.4) gives each account its class. A dedicated lane serves the accounts of one firm only (a large firm may hold several lanes). A pooled session belongs to the pool of 7 cores (lanes 42–48 on REF); its SessionId names its home pool core, and any pool core may process its packets (§4.7).

LANE_PACE, the SDK pacing rule for dedicated lanes. The SDK keeps one pacer per dedicated lane, shared by all of the firm's sessions on that lane, and sends packet n+1 on the lane no earlier than

t_send(n+1) ≥ t_send(n) + lane_ns(n) + PROTO_PACE_MARGIN_NS + PROTO_PACE_JITTER_NS
lane_ns(n)   = 45 + count(n) × (LANE_PASS1_NS + LANE_APPLY_NS) + PROTO_RESP_BUILD_NS   (01 §14.2)
             = 45 + 14 × count(n) + 20        (REF; the bundle publishes the table per target)

The margin (10 ns) pays for the lane's background slices (fill-event settlement, control ops, credit refills); the jitter term (25 ns) covers compression of the gap by the leaf switch. So at the NIC the gap is never shorter than the previous packet's lane work, and the packet never waits in the lane (02 §4.4.1). A 10-message packet then occupies a 240 ns slot, so one dedicated lane takes ≤ 4.2M full packets/s from a paced firm. Packets the pacer delays wait in the SDK; that delay is not part of the 1 µs, which starts at the exchange NIC. A firm whose sessions send from several hosts either gives each host its own lanes or coordinates one pacer. A firm that does not pace can make only its own packets wait; no other client can reach its lane.

What each class can expect for BUSY (02 REQ-LAT-5, 01 §14.17.2):

B3 measures every class and publishes the curves. Under every class, every packet still gets its first record or its BUSY within LAT_MAX_NS.

4. Sessions#

4.1 SessionId layout#

The exchange assigns every SessionId. Its bits have this meaning:

ASCII original

 31            20 19        12 11           0
┌────────────────┬────────────┬──────────────┐
│ generation u12 │ LaneId u8  │ slot u12     │
└────────────────┴────────────┴──────────────┘

• slot indexes the lane's session table directly (MAX_SESSIONS_PER_LANE = 4,096 = 2^12).
• LaneId names the lane, so the lane decides V2 with one compare and one load, with no hash table that an attacker could probe.
• generation increases each time a slot is reused, so a stale SessionId never matches a new session. It wraps after 4,096 reuses of one slot.
• A session never changes lane. An account that the operator moves gets new sessions (§4.7).
• The same SessionId is valid in every home. Every home runs the same lane count.

4.2 Session state#

The session's lane is the only writer of every field below (INV-WRITER). Control records from vyrn-admin or the chain reach the lane through the order stream, and the lane applies them.

Everything except ring_oldest, the lane's response ring, the counters and the out-of-stream holdoff is part of the replicated state. The ring is derived state: in-stream responses are a function of the order stream, so the warm standby vyrn-seq (01 §14.4) and the next home's sequencer rebuild it by replay. It is not committed in the state root. Sessions survive epoch boundaries (01 §14.8, H8).

4.3 State machine#

ASCII original

            OpenSession accepted (admin channel in BASE, chain tx in CHAIN)
                                  │
                                  ▼
                           ┌────────────┐  open record not in the final log
                           │  OPENING   │─────────────────────────────► gone (client opens again)
                           └─────┬──────┘
                                 │ lane applies SessionOpen; reply to the client once it is final
                                 ▼
       timeout: COD fires  ┌────────────┐  Logout or admin drain
       stale = true ┌──────│   ACTIVE   │──────────────────────┐
       next packet  └─────►│            │                      ▼
       clears it           └─────┬──────┘               ┌────────────┐
                                 │                      │  DRAINING  │── no resting orders,
          bad MAC from bound,    │                      └─────┬──────┘   or PROTO_DRAIN_MAX_MS
          equivocation, admin    │                            │                 │
          kill, lane move        ▼                            ▼                 ▼
                           ┌──────────────────────────────────────────────────────────┐
                           │ CLOSED: no packet accepted. Resting orders stay unless   │
                           │ COD; their later records still enter the session stream. │
                           └─────────────────────────┬────────────────────────────────┘
                                                     │ no resting orders, PROTO_CLOSED_KEEP_S
                                                     ▼
                                                   FREE (slot reusable; generation + 1)

When a session enters CLOSED, the lane emits a final in-stream record: Reject BAD_MAC with detail = the offending pkt_seq (cause BAD_MAC), or Reject SESSION_CLOSED with detail = the close cause (every other cause). Both have index 0xFF.

4.4 BASE mode: open over the admin channel#

In BASE mode the client opens a session over a TLS 1.3 connection to vyrn-admin, outside the order-entry path:

• Mutual TLS. Each account has a client certificate from the exchange's CA. TLS 1.2 and earlier are refused.
• The channel stays open for the life of the session. vyrn-admin pushes status and key bundles on it. If it drops, the client reconnects. The UDP session is not affected.
• The encoding of the admin messages belongs to vyrn-admin (11-build.md). This chapter fixes their fields.

struct OpenSessionReq {
    account: AccountId, books: Vec<BookId>, proto_version: u16,   // 1 for this chapter
    src: SocketAddrV4,                                            // becomes the bound address
    cod: Cod,                                                     // Off | On
    timeout_ms: u32,                                              // §4.8
    rate_tier: u8,                                                // §9, limits from 10
    attest: bool,                                                 // §7.5 Attest records
}
struct OpenSessionResp {
    session: SessionId, lane: LaneId, epoch: Epoch,
    bundle: KeyBundle, limits: Limits, active_from_step: StepId,
}
struct KeyBundle {
    epoch: Epoch, home: HomeCode, boundary_ts: Nanos,             // end of this epoch (TAI)
    seq_index: u8,                                                // box that is the sequencer now
    failover_order: [u8; N_MAX],                                  // order of successors
    boxes: [BoxKeys; N_MAX],                                      // every box of the home
    data: [SocketAddrV4; 8],                                      // D4 preference order
    next_home: HomeCode, lane_move: Option<LaneId>,               // §4.7
}
struct BoxKeys {
    index: u8, order: SocketAddrV4, recovery: SocketAddrV4,
    k_seq: [u8; 16], k_out: [u8; 16],
}
// Other operations: GetKeyBundle(epoch), Rebind(src), Logout, SetCod(Cod).
// Pushes: KeyBundle(e+1), DataTierStatus, SequencerStatus.

ASCII original

 client                    vyrn-admin                      sequencer lane      replicas
   │── TLS 1.3 (mTLS) ───────►│                                   │                 │
   │── OpenSessionReq ───────►│ pick slot, gen+1, derive keys     │                 │
   │                          │── SessionOpen control record ────►│ OPENING→ACTIVE  │
   │                          │                                   │── PacketRecord ►│
   │                          │◄───────────────── block with the record is final ───│
   │◄─ OpenSessionResp ───────│ (keys, lane, endpoints)           │                 │
   │── request pkt_seq 1 (UDP) ──────────────────────────────────►│                 │

Budget: an open takes ≤ 50 ms, most of it TLS and the finality of the open record. Opens are never on the 1 µs path.

vyrn-admin gives each box only the keys of that box. The replicas of the home also receive every box's k_seq, so they can check tag_seq on the data copies they use (§2.4). BASE mode trusts the operator with client keys. CHAIN mode does not (§4.5).

Rebind(src) changes the bound address. It reaches the lane as a control record and takes effect at that record's place in the order stream. From then on, V3 checks the new address and responses go to it.

4.5 CHAIN mode: open by transaction (summary)#

09-chain.md is normative for everything in this section.

• The wallet signs an OpenSession transaction (Ed25519 or secp256k1, 01 §4). It carries the account, an X25519 public key of the client, the source address to bind, the COD option, the timeout, the protocol version, the bond, and the hash-chain commitments for K_com.
• When the transaction is final, the state machine assigns the SessionId deterministically from the account's lane. The client reads it from the receipt.
• Each validator v and the client derive the per-epoch keys from X25519 between the client key and v's static key, then BLAKE3 derive_key with context "vyrn v1 session", the session and the epoch. That yields K_seq(e, v) and K_out(e, v).
• K_com(e, s) exists for each validator s that can be the sequencer in epoch e. Every validator except s holds it. A packet uses K_com(e, s) for the current sequencer s. So after a failover to s', the new sequencer does not hold the commit key that its own packets use, and forging still needs two byzantine validators.
• K_com values are disclosed after the epoch's last block is final (TESLA style), so anyone can verify every tag_com in the log afterwards.
• A client that cannot reach the sequencer may send the same packet to any validator's inclusion port. The state machine places it at a block boundary. Deduplication by (session, pkt_seq) applies there too.

4.6 Keys and rotation#

All three are 16-byte SipHash-2-4 keys. In BASE mode tag_com is 0 and the lane ignores it. In CHAIN mode the sequencer ignores tag_com, and the other validators check it before they vote.

Timeline around the boundary T between epoch e and e + 1:

 T − PROTO_KEY_PREISSUE_S   bundle for e+1 available (BASE: pushed over admin; CHAIN: derivable)
 T − 600 s … T − 1 s        EpochNotice countdown (§7.4)
 T                          client switches: epoch16 = e+1, K_seq(e+1, ·), new home's endpoints
 T + PROTO_KEY_GRACE_S      client may delete K_out(e, ·); the exchange has deleted K_seq(e, ·)

A client that has no bundle for e + 1 at T cannot send in e + 1. Its session stays ACTIVE and its timeout clock is paused (§4.8) until it fetches the bundle and sends.

Key load (computed): one key handles at most 100M packets/s × 28,800 s = 2.9 × 10^12 MACs per epoch, below 2^42. A forger gets one try per session, because a bad MAC from the bound address closes the session, and the chance that one try succeeds is 2^−64.

4.7 Lane assignment#

• vyrn-admin (BASE) or the state machine (CHAIN) puts every session of an account on the account's lane (01 §1: an account belongs to one lane per epoch). The top firms by volume get dedicated lanes; every other account is pooled, and its sessions name a home pool core (10 §1.3, 01 §14.15).
• A pool core serves every session whose LaneId is in the pool (serves() in §3.1), so a packet another pool core takes from the home core's queue is not WRONG_LANE. The account token keeps one core per account at a time, and each take is logged (AcctTake, 01 §14.17.4).
• An operator moves an account to another lane only at an epoch boundary, and only to rebalance load. The move appears in the bundle for e + 1 as lane_move.
• At the boundary, the account's sessions on the old lane go CLOSED (cause LANE_MOVED) and their resting orders end with Out cause LANE_MOVED. The client opens new sessions on the new lane.

This rule keeps Handle and SessionId lane-local with no translation table. Moves are rare, so their cost (one re-quote) is acceptable.

4.8 Heartbeats and timeouts#

Order-stream time. The lane measures the timeout on order-stream time, never on the wall clock (01 §11 rule 3). Lane time is the larger of the rx_ts of the last packet the lane processed and the time of the last TimerTick record. 05-engine.md and 07-replication.md provide a TimerTick for every lane at least once per BLOCK_NS, so an idle lane still sees time pass.

On timeout: the lane sets stale. If cod = On, it cancels the session's resting orders (§4.9). The session stays ACTIVE: the next accepted packet clears stale. UDP has no connection to lose, so a short outage on the client side costs its orders (if COD is on) and not its session.

The clock pauses from the end of an epoch until the new sequencer's first step, and during a failover, from the old sequencer's last record until the new one's first. The client cannot send during those windows, so they do not count against it.

Exchange silence, client side: if the client has unacknowledged packets and gets no in-stream response for PROTO_CLIENT_SILENCE_MS, it assumes a failover is under way (§5.6). It keeps resending with backoff, and it asks vyrn-admin for SequencerStatus if no notice arrives within PROTO_CLIENT_FAILOVER_MS.

4.9 Cancel-on-disconnect#

Each session has a COD option, set at open and changed with SetCod (it takes effect at its control record):

COD cancels only orders that this session entered, not orders of other sessions of the same account. Each canceled order gets Out with cause DISCONNECT in the session's stream. COD runs once per stale episode. It is a deterministic op at the TimerTick or control record that triggers it, so the replicas run it too.

5. Reliability#

5.1 Request direction: gapless pkt_seq, go-back-N#

• A session's packets carry pkt_seq = 1, 2, 3, … with no gap and no reuse (01 §6.1).
• The lane accepts only pkt_seq = expected. It never buffers a packet that arrives early. A lane with no reorder buffer has nothing to replay except the packets it accepted.
• Acknowledgement is cumulative. The lane accepts in order and allocates resp_seq in order, so any in-stream response with echo_pkt = p ≥ 1 proves that every pkt_seq ≤ p was accepted. An out-of-stream BAD_SEQ with detail x proves that every pkt_seq < x was accepted.
• Gap. If expected < pkt_seq ≤ expected + PROTO_SEQ_AHEAD and the MAC is good, the lane drops the packet and sends out-of-stream BAD_SEQ with detail = expected, under the holdoff. The client then resends every packet from expected to its highest sent pkt_seq, in order, with flags bit0 = 1 (go-back-N).
• Client timer. If the oldest unacknowledged packet has no acknowledgement after PROTO_REQ_RTO_US, the client resends from it with bit0 = 1. The timer doubles on each expiry up to PROTO_REQ_RTO_MAX_MS, and resets on progress. Set PROTO_REQ_RTO_US to at least 4 × the measured round trip.
• Window. The client keeps at most PROTO_SEQ_AHEAD packets unacknowledged. A packet beyond that is dropped at V5.

ASCII original

 client                                        lane (expected = 7)
   │── 7 ──────────────────────────────────────►│ accept, expected = 8
   │── 8 ────────────────────X (lost)           │
   │── 9 ──────────────────────────────────────►│ gap: drop, out-of-stream BAD_SEQ(8)
   │── 10 ─────────────────────────────────────►│ gap: drop (holdoff, counted only)
   │◄── resp(echo 7) ───────────────────────────│
   │◄── BAD_SEQ, detail 8 (resp_seq 0) ─────────│
   │── 8 (bit0) ── 9 (bit0) ── 10 (bit0) ──────►│ accept 8, 9, 10

Go-back-N costs one round trip of extra packets after a loss. Inside a colo, the round trip is a few microseconds and loss is rare, so this is cheaper than a reorder buffer in every lane.

5.2 Duplicates and the resend flag#

A packet with a good MAC and pkt_seq < expected is a duplicate. Flags bit0 says why the client sent it:

Idempotency. Each (session, pkt_seq) is accepted at most once, ever. expected is replicated state, so this holds on a new sequencer after a failover and in a new home after a handoff. A client may therefore resend any packet at any time, and the exchange executes it once or not at all.

A resend to the lane carries the same logical content (D3) and goes to the data tier too (D2). The lane does not compare a duplicate with the accepted packet. If the content differs, the replicas detect equivocation (§2.4).

5.3 Response direction: gapless resp_seq#

• In-stream response packets carry resp_seq = 1, 2, 3, … per session with no gap. Out-of-stream packets carry 0 and are outside this sequence.
• Two producers send a session's packets (§7.1), from different TX queues. So packets can arrive out of order by up to about 1 µs (budget). The client buffers out-of-order packets and processes them in resp_seq order.
• Gap. If resp_seq r is missing for PROTO_RESP_GAP_WAIT_US after a higher one arrived, the client sends ResendReq(r, r') for the missing range.
• Tail loss. If the last response is lost, nothing reveals the gap until the next response. A packet that holds only a Heartbeat gets an in-stream answer, so tail loss is found within one heartbeat interval at worst. A client that needs faster detection sends a Heartbeat sooner.
• The lane serves a ResendReq from its ring if the range is at most PROTO_LANE_RESEND_MAX packets and is still in the ring. Otherwise it rejects it with BAD_SEQ (§6.7), and the client uses the recovery service. RESEND_GONE (22) means that neither store holds the range (§5.4).

5.4 Retention: the lane ring and the recovery service#

01 §5 sets RESP_RETAIN_PKTS = 65,536 response packets per session. They cannot all live in the lanes:

(computed). So retention has two tiers (01 §14.6):

• Response ring, on the sequencer. One byte ring of RESP_HOT_RING_BYTES = 4 GiB per lane, in the lane's DRAM, shared by its sessions: ~8 s of in-stream responses at peak (budget), about 15M packets of 280 B. The lane appends each in-stream packet as it sends it, and the oldest bytes are overwritten first. Each session's ring_oldest follows. The standby rebuilds the ring by replay (§4.2).
• Recovery service, on a replica. One vyrn-replica per home runs the recovery role. It keeps the last RESP_RETAIN_PKTS packets of every session, built from its own replay, on NVMe. At peak the response stream is of the same order as the request stream (budget ≤ 26 GB/s; 06-platform.md sizes the drives). It keeps the stream of a CLOSED session for PROTO_CLOSED_KEEP_S.

Recovery request. The client sends to the recovery address of box b, port PROTO_RECOVERY_PORT, a request packet with:

• pkt_seq = 0 (never valid on the order port, so a recovery request can never be confused with order entry and never consumes a pkt_seq);
• count = 1, one ResendReq(from, to), the current epoch16;
• tag_seq with K_seq(e, b) and tag_com = 0.

The service runs V1–V6 with the session's bound address, then sends the packets in the range with bit0 = 1, tagged with K_out(e, b), at most PROTO_RECOVERY_PPS per session. A V6 failure here is dropped and counted and closes nothing: the service changes no state. A range newer than the replica's replay is cut at the newest packet it holds, and the rest gets an out-of-stream BAD_SEQ with detail = that newest resp_seq.

RESEND_GONE (22). Both stores are functions of the order stream, so each one knows the other's oldest resp_seq per session: the lane computes the recovery bound as next_resp − RESP_RETAIN_PKTS, and the recovery service tracks the ring's ring_oldest without storing its bytes. A store that does not hold a range answers BAD_SEQ with detail = the oldest resp_seq the other store holds, so the client goes there. Only a range that both stores have dropped gets Reject RESEND_GONE, detail = the oldest resp_seq either holds. The client then rebuilds its order state from the drop copy (10-accounts-risk-md.md).

5.5 Gaps in both directions: summary#

5.6 Sequencer failover#

 t0          old sequencer sends its last frame, then dies
 t0 + 2 ms   replicas detect silence (FAILOVER_SILENCE_MS) and finalize the longest prefix they hold
 ≤ t0+20 ms  box b, the home's warm standby vyrn-seq (01 §14.4), starts as sequencer at the next step
             each lane of b emits, for every live session, one in-stream EpochNotice
             with next_epoch = current epoch (the sequencer-change variant, §7.4), resp_seq = r
 client      on that notice: switch the order endpoint and K_seq to box b,
             then resend every unacknowledged packet, bit0 = 1, to b (and to the data tier)

What the client sees, case by case:

The rewind rule. Let r be the resp_seq of the sequencer-change notice. Every in-stream response that the client received from the old sequencer with resp_seq ≥ r is void: its records are not in the final log, and box b will reuse those resp_seq values. The client undoes the effect of the void records on its order state (§10) and treats their requests as unanswered.

A void response exists only if the old sequencer's frames were lost on both replication paths before it died, because every in-stream response leaves after its records left on both paths (rule R-FIRST, §7.2; 01 §14.5). A sequencer crash alone never voids a response: an acked order is never lost. DESIGN §0 point 3 gives the odds of the double loss. The protocol makes that case visible and exact. It does not make it common.

Until the notice arrives, the client keeps resending to the old box, and those packets are lost. If no notice arrives within PROTO_CLIENT_FAILOVER_MS, the client asks vyrn-admin (BASE) or chain state (CHAIN) for SequencerStatus.

5.7 Epoch handoff#

ASCII original

 old home (epoch e)                      client                        new home (epoch e+1)
  │── EpochNotice T−600 … T−1 s ────────►│                                  │
  │   last step S at T; then drops all   │── from T: epoch16 = e+1 ────────►│ stages packets
  │   packets                            │   new K_seq, new endpoints       │ (08-epochs.md)
  │── finalize S, cert(e, S), log tail ─────────────────────────────────────►│ state at S
  │                                      │◄── EpochNotice (epoch start) ────│ starts at S+1
  │                                      │◄── answers to staged packets ────│ auction phase
  │                                      │◄── uncross fills (liq 2) ────────│ then continuous

6. Messages#

6.1 Rules for every message#

• Bytes 2–3 are 0 in every v1 message, and so is every field that 01 §6.2 marks 0 (M3).
• client_tag is opaque. The exchange echoes it and never checks it for uniqueness. Replace has no client_tag; its records carry the stored tag of the order it replaces.
• Handles. The lane assigns a Handle when it accepts a New or a priority-losing Replace. An operation on a handle succeeds only if the slot is live, its generation matches, and the order belongs to the session's account. Any other case is UNKNOWN_HANDLE, so a client cannot learn anything about another account's orders. PROTO_HANDLE_ALL = 0xFFFF_FFFF is never a real handle (slot 0xFF_FFFF is never allocated).
• Handle reuse. 05-engine.md allocates slots so that a slot is reused no sooner than PROTO_HANDLE_REUSE_MIN_S after its order ended. The client never sends a handle older than that past the order's last record. With an 8-bit generation a much older handle could match a newer order of the same account.
• Effective point. Each message takes effect at one place in the total order: its LaneSeq for a lane op, its step for a BBO-core op. Messages of one packet take effect in index order. canceled qty and TOO_LATE are evaluated at that place.
• Order flags (01 §6.2): bit 0 side; bits 1–2 TIF (0 Day, 1 IOC, 2 FOK, 3 AUCTION_ONLY); bit 3 post-only; bits 4–5 STP mode; bits 6–7 zero.
• Invalid flag combinations get BAD_TYPE with detail PROTO_D_FLAGS: post-only with IOC or FOK (one must not take, the other must take or die), and post-only with AUCTION_ONLY (an auction has no maker).

6.2 New (0x01)#

Fields: flags, tick, qty, client_tag.

Validation, in M-order (§3.3):

Routing after validation:

Notes:

• IOC and FOK always go to the BBO core, even if they look non-marketable against the lane's snapshot. A snapshot can be stale in the unsafe direction after an improve, and an IOC must trade against the true book. They are charged as crosses (§9).
• A post-only improve can reach the BBO core and find that the true book crosses it. The BBO core rejects it with POST_ONLY_WOULD_CROSS. If a rescan (05-engine.md) finds a resting post-only order as the aggressor, that order ends with Out cause POST_ONLY, after its Ack.
• A taker gets one Fill per price level, best level first (01 §6.4). The remainder of a Day order rests at its own limit tick, and its Ack carries that tick.
• Day means the trading day. The trading day ends at the TYO boundary (00:00 UTC, EPOCH_BOUNDARIES_UTC). Resting Day orders survive the LDN and NYC handoffs and end with Out EXPIRED at the first step of the next TYO epoch (H9).

6.3 Cancel (0x02)#

Fields: handle, client_tag.

Cancel against fill: the cut orders them (DESIGN §3). Fills at earlier steps reduce the open qty first, so canceled qty is what was left. If nothing was left, the answer is TOO_LATE. Every Fill of the order that precedes the Cancel in the total order has a lower resp_seq than the CancelAck (R-HANDLE, §7.2).

6.4 Reduce (0x03)#

Fields: handle, reduce_by, client_tag.

• reduce_by = 0 → Reject BAD_QTY.
• Live order: canceled = min(reduce_by, open). The open qty and the order's total qty both fall by canceled. The order keeps its handle and its time priority. Record: CancelAck(handle, canceled, client_tag).
• Reduce to 0. If reduce_by ≥ open, the order ends. The record is the same CancelAck, with canceled = open. The client knows the order ended because its open qty, tracked in resp_seq order, is now 0 (§10).
• Ended order → TOO_LATE. Unknown handle → UNKNOWN_HANDLE. Both as for Cancel.

6.5 Replace (0x04)#

Fields: flags (the full order flags of the replacement; the side must match), handle, new tick, new qty.

qty is the new total quantity of the order, filled part included. Let total be the order's quantity after earlier Reduces, cum its filled quantity, and open = total − cum. Total-qty semantics mean that a fill racing the Replace can never make the account trade more than the new qty.

Lane checks, in this order. Any failure gives one Reject with the order's stored client_tag and leaves the old order untouched:

Then one of two cases:

• A priority-loss Replace whose new order goes to the BBO core has all its records, the CancelAck of the old order included, in the B-packet (§7.1). So the CancelAck always comes first.
• Outcomes that only the BBO core can decide (fills, STP, POST_ONLY_WOULD_CROSS against the true book, FOK_UNFILLABLE) happen after the old order is canceled. They are outcomes of the new order, not a rejection of the Replace.
• The client tells the cases apart by the handle: the same handle in the CancelAck and no Ack means a reduction. A new handle means priority was lost.

6.6 CancelAll (0x12)#

Fields: client_tag. Scope: every live order that this session entered in the packet's book and whose New or Replace precedes the CancelAll in the total order. Orders of other sessions of the same account are not touched (use vyrn-admin for an account-wide kill).

Records, all with the CancelAll's index and client_tag:

1. one CancelAck(handle, canceled qty) per canceled order, in ascending handle order;
2. then a summary CancelAck(PROTO_HANDLE_ALL, number of orders canceled). The summary is always sent, also when the count is 0, so the client knows the CancelAll is complete.

The CancelAll takes effect at once, at its effective point, with one AcctKill write (05-engine.md §4.7): from that point no covered order can trade. So its apply cost does not grow with the order count, and its first record leaves within LAT_MAX_NS like any message. Only the per-order records grow with the count. They follow in later L-packets with echo_pkt = p at the lane's record rate, and the summary comes last. CancelAll draws PROTO_CANCELALL_TOKENS message tokens, so it cannot be used to stall a lane. An order still at the BBO core at the CancelAll's effective point is canceled if it comes to rest; 05-engine.md defines how the lane learns that.

6.7 Heartbeat (0x10) and ResendReq (0x11)#

Heartbeat has no fields. It produces no record of its own. A packet whose messages produce no record gets an EpochNotice (R-ANSWER, §7.2), so a heartbeat sent alone is always answered. It consumes a pkt_seq like any packet and draws from the packet bucket.

ResendReq(from, to) asks the lane to resend in-stream packets from … to, inclusive:

A resent packet keeps its resp_seq, echo_pkt, count and records. Its epoch16 and tag are those of the epoch and the box that resends it. The resends are not in-stream records, so a packet that holds only ResendReq messages also gets an EpochNotice (R-ANSWER). Resends draw from the RESEND bucket (§9). The resent packets are copies and change no state.

6.8 STP#

STP prevents a trade between two orders of the same AccountId. Only the aggressor's mode applies (the order that is later in the total order, also in a rescan).

Mode 0 allows self-trades. 10-accounts-risk-md.md monitors them. STP in the opening auction uncross follows 08-epochs.md.

6.9 Book phases#

The AUCTION_PHASE detail is the current phase (PROTO_PHASE_*: 1 continuous, 2 auction, 3 halted). Resting orders stay in a halted book. 08-epochs.md owns the auction's timing and its uncross. 10-accounts-risk-md.md owns halts.

7. Responses#

7.1 Producers and the resp_seq allocator#

Two kinds of core send a session's response packets: its lane and the TX cores (TX-A, TX-B, TX-C on cores 3, 5 and 0, 01 §14.17.3). Only the lane allocates resp_seq; a TX core uses numbers the lane reserved for it.

Reservation. When the lane accepts a request packet with at least one BBO-routed message, it allocates resp_seq r for the B-packet before any other resp_seq of that packet, and passes r with the first message it pushes to the BBO core. The lane's own packet for the same request (the L-packet) gets r + 1. A request packet with no BBO-routed message gets only an L-packet.

ASCII original

 request pkt p:  [0] Cancel h1   [1] New cross   [2] New join
                      │               │               │
 lane:                │  reserve r ───┼──► BBO ring   │
                      ▼               │               ▼
                 L-packet r+1: CancelAck(0), Ack(2)   (after a cut includes them)
 BBO core → TX core:  └────────────► B-packet r: Fill(1) … Ack(1)   (≤ 1 µs path)

Rules:

• A B-packet is never empty: every BBO-routed message yields at least one record.
• The B-packet holds up to 16 records of the packet's BBO-routed messages, in index order. The TX core keeps one slot free for each later BBO-routed message of the packet (the lane passes their count with the first push), so every BBO-routed message has its first record in the B-packet, which leaves within LAT_MAX_NS. A message's records that do not fit are its later fills and terminal record. The lane settles the taker side from the fill events, so it knows which records did not fit. It sends them in its next packets with echo_pkt = p, before any later record of the session. The L-packet follows the same rule for lane-local messages.
• Maker fills (01 §14.14, §14.17.5). The TX cores send them in the step that made them, right after the taker's B-packet, with no hop through the maker's lane: +16 ns per maker packet on each TX core (PROTO_MAKER_PKT_NS, budget). The maker fills of the first steps of a sweep leave within LAT_MAX_NS of the first bit of the taker's packet; 01 puts this at about 40 makers at index 9, more at lower indices. The maker fills of later continuation steps leave within LAT_MAX_NS of the start of their own step. The taker's first fill is always within 1 µs. A maker fill is a notice to another client, not the response to the maker's own packet, so this exempts no response. The maker's lane still settles the balance from the fill event, off the latency path, and appends the packet to its response ring then.
• Maker range. For each session with resting orders, the lane keeps a credit of PROTO_MAKER_RANGE = 16 numbers in the session's maker-range line, which the lane writes and the TX core reads. The TX core takes the next number of the credit for each maker packet and writes its own used counter. The lane refills the credit by 16 when its applied fill events show 8 used. One step writes at most STEP_FILL_CAP = 64 fill records (01 §14.15), so it needs at most 4 maker packets of one session, and a longer sweep continues in further steps. The credit is therefore never exceeded, and every order hit gets its maker record from the TX core.
• Numbering rule (gapless, R-DET). A session's resp_seq order is the merge of the maker packets, in step order, with the lane-numbered packets (B-packet reservations, L-packets, unsolicited lane packets), each of which sits at a lane position q (a LaneSeq). A maker packet of step s comes before a lane packet at q exactly when cut_s[lane] < q. To place a packet at q, the lane writes (close_seq = q, n) to the maker-range line before it publishes q, so every step whose cut includes q shifts the rest of the credit up by n. The lane numbers its packet after it has applied the fill events of every step before the first cut that includes q; it already waits for that before a CancelAck (R-HANDLE). So the numbers are relative, not a fixed absolute block: an unused number of the credit is never a gap, and an order's fills always follow its Ack in resp_seq order.
• This keeps INV-WRITER: the lane is the only writer of next_resp, of the credit and of close_seq; the TX core is the only writer of used. Both fill paths stay inside CCD 0 as in DESIGN §4.

7.2 Rules for in-stream packets#

7.3 Records#

Reject versus Out: Reject means the message had no effect on the book. Out means an order that was in the matching step leaves the book. Reject STP and Reject FOK_UNFILLABLE are kills with no fill; Out STP and Out IOC_REMAINDER follow fills or a match attempt.

There is no FOK cause: an unfillable FOK is always Reject FOK_UNFILLABLE (01 §14.10).

7.4 EpochNotice#

Fields: bytes 4–7 next epoch (u32); bytes 8–11 the home word: byte 8 = HomeCode, byte 9 = the sequencer box index in that epoch (0xFF if not yet fixed), bytes 10–11 = 0; bytes 12–15 seconds to boundary = ⌊(boundary_ts − lane time) / 10^9⌋, saturating at 0. Lane time is order-stream time (§4.8), so the value is deterministic.

The client reads the variant from next epoch: equal to the current epoch means a sequencer change, which also starts the rewind rule (§5.6). The current epoch is the full epoch whose low 16 bits are the packet's epoch16.

7.5 What a record attests: execution, durability, finality, settlement#

01 §14.13 item 3 separates four facts. They come in this order. The protocol never merges them into one "confirmed" flag, and a client must not either.

Attest (0x88, 01 §14.10). An out-of-stream response packet (resp_seq 0, echo_pkt 0) with one record:

off size field
  0   1  type      0x88
  1   1  level     1 DURABLE, 2 FINAL, 3 SETTLED
  2   2  reserved  0
  4   4  upto      every in-stream record of this session with resp_seq ≤ upto has this level
  8   4  height    low 32 bits of the block Height that reached it
 12   4  reserved  0

• The lane sends it from idle time, never on the 1 µs path, at most once per level per BLOCK_NS per session, and only when upto moved. A lost Attest is replaced by the next one, so it is not retained or resent. Its frame gate is empty: it reports, it changes nothing.
• A client that wants it sets attest at open (OpenSessionReq, §4.4). Every client can get the same facts per fill from the drop copy, which labels each level separately (10).
• A record at level 1 or above is never voided by the rewind rule (§5.6): a void record is one whose frames no replica holds. A level-2 record is irreversible. In CHAIN only a Bust can undo a final fill, and only as settlement fails (09).

8. Reject reasons#

"In" means in-stream (accepted packet, consumes pkt_seq, retained, resendable). "Out" means out-of-stream (resp_seq 0, not accepted, the client resends the same logical content later).

9. Rate limits and fee classes#

9.1 Buckets#

Every bucket is a GCRA (a token bucket written as one timestamp). All values are integers in nanoseconds of rx_ts, so a replica that replays the PacketRecords gets the same decisions (INV-DET). The bucket state is part of the lane's replicated state.

struct Gcra { tat: Nanos, t_ns: u64, burst: u64 }        // t_ns = 10^9 / rate, set at open
impl Gcra {
    /// Takes `cost` tokens at order-stream time `now` unless fewer than `keep` would be left.
    fn take(&mut self, now: Nanos, cost: u64, keep: u64) -> bool {
        let base = max(self.tat, now);
        let next = base + cost * self.t_ns;              // checked in the real code (01 §11 rule 4)
        if next > now + (self.burst - keep) * self.t_ns { return false; }
        self.tat = next;
        true
    }
}

• Cancel reserve. New and priority-losing Replace take from MSG with keep = PROTO_CANCEL_RESERVE (a quarter of the burst). Cancel, Reduce and CancelAll use keep = 0. So a client at its limit can still cancel.
• BBO units. The BBO core spends BBO_CROSS_NS per cross and BBO_IMPROVE_NS per improve, a 5 : 1 ratio, so the BBO bucket counts improves as 1 unit and crosses as 5. One book can take 0.6 × 10^9 / 8 = 75M units/s at BBO_MAX_LOAD (computed), which covers CROSS_CAP crosses (50M units) plus 20M improves (DESIGN §3).
• Admission. vyrn-admin admits a session only if the committed MSG rates of the lane's sessions sum to ≤ 0.6 × 10^9 / LANE_NS_PER_MSG = 20M/s, and the committed BBO rates of the book sum to ≤ 70M units/s (budget). Admission does not make the 1 µs hold; test D does, at any load (§3.5). Admission keeps BUSY within each lane class's bound (§3.6, B3).
• 10-accounts-risk-md.md owns the tiers T0–T3 and their prices. The defaults above are this chapter's budget for T1.

9.2 Fee classes#

Each message gets one fee class from its routing. 10-accounts-risk-md.md owns the amounts and how they are settled. The lane accrues the count per class in the account's lane state.

A priority-loss Replace is charged CANCEL plus the class of its new order. A lane-level reject is charged JOIN because it used lane time. A RATE_LIMIT reject is free because the bucket already stops the flood.

10. The order as the client sees it#

10.1 Client state machine#

The client runs one machine per order, keyed by (pkt_seq, index) until the first record gives it a handle, and by handle after that. It feeds records in resp_seq order (§5.3). It tracks open = order qty − fills − canceled qty.

ASCII original

                    send New (pkt_seq p, index i)
                                │
                                ▼
                         ┌─────────────┐ no answer: resend p with bit0 = 1 (§5.1)
                         │   SENT      │◄──────────────┐
                         └──────┬──────┘───────────────┘
        Reject ─────────────────┼────────────────────────────────────► REJECTED
        Fill… with open = 0 ────┼────────────────────────────────────► FILLED
        Fill… then Out ─────────┼────────────────────────────────────► OUT
        Fill… then Ack, or Ack  │
                                ▼
                         ┌─────────────┐  maker Fill: open −= qty; open = 0 ──► FILLED
                         │    LIVE     │  Out (STP, POST_ONLY, EXPIRED, DISCONNECT,
                         │  (handle h) │       LANE_MOVED, ADMIN, AUCTION_UNFILLED) ──► OUT
                         └──┬───┬───┬──┘
              send Cancel   │   │   │ send Replace
                            ▼   │   ▼
                 ┌──────────────┐ │ ┌────────────────┐
                 │CANCEL_PENDING│ │ │REPLACE_PENDING │ Reject (not TOO_LATE): back to LIVE, h kept
                 └──────┬───────┘ │ └───────┬────────┘ CancelAck(h), no new handle: LIVE (reduced)
     CancelAck: CANCELED│         │         │          CancelAck(h) + records of h': h ends
     TOO_LATE: the fills│  send   │         │          (CANCELED), h' starts in SENT-like
     before it ended it │  Reduce ▼         │          state with its records
     (FILLED or OUT)    │  REDUCE_PENDING: CancelAck: open −= qty; open = 0 ──► CANCELED
                        ▼                   otherwise back to LIVE
             CANCELED / FILLED / OUT (terminal)

Maker fills can arrive in every pending state. They lower open there too. A CancelAck after them reports only what was left. Reject BUSY is not REJECTED: the order never reached the book, so the client may send it again (§3.5). Each terminal state is level 0 (EXECUTED) until an Attest or the drop copy raises it (§7.5).

Rewind. On a sequencer-change notice with resp_seq r, the client drops the effects of every record with resp_seq ≥ r that it got from the old sequencer, by replaying its own record log up to r − 1. Then it resends its unacknowledged packets (§5.6).

10.2 What the client can rely on#

11. Conformance test vectors#

The script below is pure Python 3 with only the standard library. It first checks SipHash-2-4 against the reference vector of the SipHash paper (key = bytes 00..0f, message = bytes 00..0e → 0xa129ca6149be45e5) and two more vectors of the same table, then builds the packets. Run it with python3 -I. 11-build.md turns every vector into a test of vyrn-wire.

Keys (test only, never production):

Scenario: epoch 1000 (epoch16 = 0x03e8), book 1. Session A = slot 7, lane 3, generation 1 (0x00103007). Session B = slot 9, lane 5, generation 1 (0x00105009), another account. Session C = slot 2, lane 0, generation 4 (0x00400002), CHAIN mode. The lane's snapshot bid is 100001 or better when V1 and V3 arrive, so both orders are joins and go in L-packets. Before V5, the only bids are A4 (8 @ 100001) and 1 lot @ 100000. V11 answers the first transmission of V12, which carried epoch16 = 999; V12 is the same logical content resent with epoch16 = 1000 (D3). V12 also gets an R-ANSWER EpochNotice as resp_seq 5 (not printed), so V15 is resp_seq 6. V16 reaches lane 3 when test D (§3.5) fails, so the lane sheds it: V17 is the out-of-stream BUSY (resp_seq 0, echo_pkt 6, detail = expected = 6), and nothing was applied. V18 is the client's resend of the same pkt_seq with the same logical content and bit0 = 1; the lane accepts it as a first acceptance.

#!/usr/bin/env python3
"""VYRN      04-protocol conformance vectors. Pure Python 3, standard library only."""
import struct

MASK = (1 << 64) - 1


def rotl(x: int, b: int) -> int:
    return ((x << b) | (x >> (64 - b))) & MASK


def siphash24(key: bytes, msg: bytes) -> int:
    """SipHash-2-4, 128-bit key, 64-bit tag (as an integer; on the wire it is little-endian)."""
    if len(key) != 16:
        raise ValueError(f"SipHash key must be 16 bytes, got {len(key)}")
    k0, k1 = struct.unpack("<QQ", key)
    v = [k0 ^ 0x736F6D6570736575, k1 ^ 0x646F72616E646F6D,
         k0 ^ 0x6C7967656E657261, k1 ^ 0x7465646279746573]

    def rnd() -> None:
        v[0] = (v[0] + v[1]) & MASK; v[1] = rotl(v[1], 13); v[1] ^= v[0]; v[0] = rotl(v[0], 32)
        v[2] = (v[2] + v[3]) & MASK; v[3] = rotl(v[3], 16); v[3] ^= v[2]
        v[0] = (v[0] + v[3]) & MASK; v[3] = rotl(v[3], 21); v[3] ^= v[0]
        v[2] = (v[2] + v[1]) & MASK; v[1] = rotl(v[1], 17); v[1] ^= v[2]; v[2] = rotl(v[2], 32)

    end = len(msg) - len(msg) % 8
    for i in range(0, end, 8):
        m = struct.unpack_from("<Q", msg, i)[0]
        v[3] ^= m; rnd(); rnd(); v[0] ^= m
    b = ((len(msg) & 0xFF) << 56) | int.from_bytes(msg[end:], "little")
    v[3] ^= b; rnd(); rnd(); v[0] ^= b
    v[2] ^= 0xFF
    for _ in range(4):
        rnd()
    return v[0] ^ v[1] ^ v[2] ^ v[3]


# Reference vectors from the SipHash paper (key 00..0f, message 00..len-1).
for n, want in [(15, 0xA129CA6149BE45E5), (0, 0x726FDB47DD0E0E31), (8, 0x93F5F5799A932462)]:
    got = siphash24(bytes(range(16)), bytes(range(n)))
    if got != want:
        raise SystemExit(f"SipHash-2-4 self-test FAILED for len {n}: {got:#018x}")
print("siphash24(00..0f, 00..0e) = 0xa129ca6149be45e5  reference vector OK (also len 0, 8)")

K_SEQ = bytes(range(0x00, 0x10))   # test keys only
K_OUT = bytes(range(0x10, 0x20))
K_COM = bytes(range(0x20, 0x30))


def sid(slot, lane, gen): return slot | lane << 12 | gen << 20          # section 4.1
def hdl(slot, gen): return slot | gen << 24                             # 01 section 3
def of(side, tif=0, post=0, stp=0): return side | tif << 1 | post << 3 | stp << 4
def rec(t, b1=0, w=0, a=0, b=0, c=0): return struct.pack("<BBHIII", t, b1, w, a, b, c)


def new(f, tick, qty, tag): return rec(0x01, f, 0, tick, qty, tag)
def cancel(h, tag): return rec(0x02, 0, 0, h, 0, tag)
def reduce(h, by, tag): return rec(0x03, 0, 0, h, by, tag)
def replace(f, h, tick, qty): return rec(0x04, f, 0, h, tick, qty)
def heartbeat(): return rec(0x10)
def resend_req(lo, hi): return rec(0x11, 0, 0, lo, hi, 0)
def cancel_all(tag): return rec(0x12, 0, 0, 0, 0, tag)
def ack(i, h, tick, tag): return rec(0x81, i, 0, h, tick, tag)
def fill(i, liq, h, tick, qty): return rec(0x82, i, liq, h, tick, qty)
def reject(i, why, detail, tag): return rec(0x83, i, why, detail, 0, tag)
def cancel_ack(i, h, qty, tag): return rec(0x84, i, 0, h, qty, tag)
def out(i, cause, h, qty, tag): return rec(0x85, i, cause, h, qty, tag)
def epoch_notice(e, home, secs): return rec(0x86, 0, 0, e, home, secs)


def request(s, seq, e16, book, msgs, k_com=None, flags=0):
    body = struct.pack("<IIBBHHH", s, seq, len(msgs), flags, e16, book, 0) + b"".join(msgs)
    tag_com = siphash24(k_com, body) if k_com else 0                    # 0 in BASE mode
    return body + struct.pack("<QQ", siphash24(K_SEQ, body), tag_com)


def response(s, rseq, e16, echo, recs, flags=0):
    body = struct.pack("<IIBBHI", s, rseq, len(recs), flags, e16, echo) + b"".join(recs)
    return body + struct.pack("<Q", siphash24(K_OUT, body))


def verify_request(pkt, key):
    n = pkt[8]
    end = 16 + 16 * n
    ok_len = 1 <= n <= 10 and len(pkt) == end + 16
    return ok_len and siphash24(key, pkt[:end]) == struct.unpack_from("<Q", pkt, end)[0]


def dump(name, pkt, req):
    print(f"\n{name} ({len(pkt)} B)")
    for off in range(0, len(pkt), 16):
        print(f"  {off:03x}: " + " ".join(f"{x:02x}" for x in pkt[off:off + 16]))
    tags = struct.unpack_from("<QQ", pkt, len(pkt) - 16) if req else struct.unpack_from("<Q", pkt, len(pkt) - 8)
    print("  " + "  ".join(f"{n}={t:#018x}" for n, t in zip(("tag_seq", "tag_com") if req else ("tag",), tags)))


E, B = 1000, 1                                   # epoch 1000 (epoch16 0x03e8), book 1
A, Bs, C = sid(7, 3, 1), sid(9, 5, 1), sid(2, 0, 4)
A1, A2, A3, A4, B1 = hdl(42, 1), hdl(43, 1), hdl(44, 1), hdl(45, 1), hdl(7, 3)
V = [
    ("V1 req  A: New buy Day 5 @ 100000, STP 1", request(A, 1, E, B, [new(of(0, stp=1), 100_000, 5, 0x11111111)]), True),
    ("V2 resp A: Ack (lane, L-packet)", response(A, 1, E, 1, [ack(0, A1, 100_000, 0x11111111)]), False),
    ("V3 req  A: Cancel A1, Reduce A2 by 2, Replace A3 -> buy 8 @ 100001",
     request(A, 2, E, B, [cancel(A1, 0x22222222), reduce(A2, 2, 0x33333333), replace(of(0, stp=1), A3, 100_001, 8)]), True),
    ("V4 resp A: CancelAck, CancelAck, CancelAck(old) + Ack(new), stored tag 0x44444444",
     response(A, 2, E, 2, [cancel_ack(0, A1, 5, 0x22222222), cancel_ack(1, A2, 2, 0x33333333),
                           cancel_ack(2, A3, 4, 0x44444444), ack(2, A4, 100_001, 0x44444444)]), False),
    ("V5 req  B: New sell IOC 10 @ 100000", request(Bs, 1, E, B, [new(of(1, tif=1), 100_000, 10, 0x55555555)]), True),
    ("V6 resp B: B-packet Fill 8 @ 100001, Fill 1 @ 100000, Out IOC_REMAINDER 1",
     response(Bs, 1, E, 1, [fill(0, 1, B1, 100_001, 8), fill(0, 1, B1, 100_000, 1), out(0, 1, B1, 1, 0x55555555)]), False),
    ("V7 resp A: maker Fill of A4 from the TX core, maker range (echo_pkt 0)", response(A, 3, E, 0, [fill(0xFF, 0, A4, 100_001, 8)]), False),
    ("V8 req  A: Heartbeat", request(A, 3, E, B, [heartbeat()]), True),
    ("V9 resp A: EpochNotice (answer): next 1001, LDN box 1, 3600 s",
     response(A, 4, E, 3, [epoch_notice(E + 1, 2 | 1 << 8, 3600)]), False),
    ("V10 req C: CHAIN mode, New sell post-only 3 @ 100002",
     request(C, 1, E, B, [new(of(1, post=1), 100_002, 3, 0x66666666)], k_com=K_COM), True),
    ("V11 resp A: out-of-stream Reject EPOCH_MISMATCH, detail 1000",
     response(A, 0, E, 4, [reject(0xFF, 15, E, 0)]), False),
    ("V12 req A: ResendReq(1, 1)", request(A, 4, E, B, [resend_req(1, 1)]), True),
    ("V13 resp A: resend of V2 (flags bit0, same resp_seq, new tag)",
     response(A, 1, E, 1, [ack(0, A1, 100_000, 0x11111111)], flags=1), False),
    ("V14 req A: CancelAll", request(A, 5, E, B, [cancel_all(0x77777777)]), True),
    ("V15 resp A: CancelAck x2, summary CancelAck(HANDLE_ALL, 2)",
     response(A, 6, E, 5, [cancel_ack(0, A2, 3, 0x77777777), cancel_ack(0, hdl(46, 1), 6, 0x77777777),
                           cancel_ack(0, 0xFFFFFFFF, 2, 0x77777777)]), False),
    ("V16 req A: New buy IOC 2 @ 100000 (arrives in overload)",
     request(A, 6, E, B, [new(of(0, tif=1), 100_000, 2, 0x88888888)]), True),
    ("V17 resp A: out-of-stream Reject BUSY (lane shed), index 0xFF, detail = expected 6",
     response(A, 0, E, 6, [reject(0xFF, 24, 6, 0)]), False),
    ("V18 req A: resend of V16, same pkt_seq and content, bit0 = 1",
     request(A, 6, E, B, [new(of(0, tif=1), 100_000, 2, 0x88888888)], flags=1), True),
]
for name, pkt, req in V:
    dump(name, pkt, req)

v1, v10 = V[0][1], V[9][1]
bad = bytearray(v1); bad[32] ^= 0x01                                    # one bit of tag_seq
for what, ok in [("V1 verifies with K_SEQ", verify_request(v1, K_SEQ)),
                 ("V1 fails with K_OUT", not verify_request(v1, K_OUT)),
                 ("V1 with one tag bit flipped fails (bound source: close session)",
                  not verify_request(bytes(bad), K_SEQ)),
                 ("V10 tag_com = SipHash(K_COM, bytes 0..32)",
                  struct.unpack_from("<Q", v10, 40)[0] == siphash24(K_COM, v10[:32])),
                 ("V13 tag differs from V2 tag", V[12][1][-8:] != V[1][1][-8:]),
                 ("V17 is out of stream (resp_seq 0), reason BUSY 24, echo_pkt 6",
                  struct.unpack_from("<I", V[16][1], 4)[0] == 0 and V[16][1][16:18] == b"\x83\xff"
                  and struct.unpack_from("<H", V[16][1], 18)[0] == 24
                  and struct.unpack_from("<I", V[16][1], 12)[0] == 6),
                 ("V18 has V16's logical content (D3) and verifies", V[17][1][16:32] == V[15][1][16:32]
                  and V[17][1][:9] == V[15][1][:9] and verify_request(V[17][1], K_SEQ))]:
    if not ok:
        raise SystemExit(f"FAILED: {what}")
    print(f"check OK: {what}")

Output of python3 -I vyrn_proto_vectors.py on 2026-11-13 (all checks pass):

siphash24(00..0f, 00..0e) = 0xa129ca6149be45e5  reference vector OK (also len 0, 8)

V1 req  A: New buy Day 5 @ 100000, STP 1 (48 B)
  000: 07 30 10 00 01 00 00 00 01 00 e8 03 01 00 00 00
  010: 01 10 00 00 a0 86 01 00 05 00 00 00 11 11 11 11
  020: ce a6 ae 0c 10 82 13 8e 00 00 00 00 00 00 00 00
  tag_seq=0x8e1382100caea6ce  tag_com=0x0000000000000000

V2 resp A: Ack (lane, L-packet) (40 B)
  000: 07 30 10 00 01 00 00 00 01 00 e8 03 01 00 00 00
  010: 81 00 00 00 2a 00 00 01 a0 86 01 00 11 11 11 11
  020: 7b 0a dd 3a 35 20 db 3c
  tag=0x3cdb20353add0a7b

V3 req  A: Cancel A1, Reduce A2 by 2, Replace A3 -> buy 8 @ 100001 (80 B)
  000: 07 30 10 00 02 00 00 00 03 00 e8 03 01 00 00 00
  010: 02 00 00 00 2a 00 00 01 00 00 00 00 22 22 22 22
  020: 03 00 00 00 2b 00 00 01 02 00 00 00 33 33 33 33
  030: 04 10 00 00 2c 00 00 01 a1 86 01 00 08 00 00 00
  040: 21 7c cc dc 1c 1c 91 3a 00 00 00 00 00 00 00 00
  tag_seq=0x3a911c1cdccc7c21  tag_com=0x0000000000000000

V4 resp A: CancelAck, CancelAck, CancelAck(old) + Ack(new), stored tag 0x44444444 (88 B)
  000: 07 30 10 00 02 00 00 00 04 00 e8 03 02 00 00 00
  010: 84 00 00 00 2a 00 00 01 05 00 00 00 22 22 22 22
  020: 84 01 00 00 2b 00 00 01 02 00 00 00 33 33 33 33
  030: 84 02 00 00 2c 00 00 01 04 00 00 00 44 44 44 44
  040: 81 02 00 00 2d 00 00 01 a1 86 01 00 44 44 44 44
  050: 17 d2 51 0e 46 ae 13 4a
  tag=0x4a13ae460e51d217

V5 req  B: New sell IOC 10 @ 100000 (48 B)
  000: 09 50 10 00 01 00 00 00 01 00 e8 03 01 00 00 00
  010: 01 03 00 00 a0 86 01 00 0a 00 00 00 55 55 55 55
  020: 23 7b 64 e3 77 6d 4d 3f 00 00 00 00 00 00 00 00
  tag_seq=0x3f4d6d77e3647b23  tag_com=0x0000000000000000

V6 resp B: B-packet Fill 8 @ 100001, Fill 1 @ 100000, Out IOC_REMAINDER 1 (72 B)
  000: 09 50 10 00 01 00 00 00 03 00 e8 03 01 00 00 00
  010: 82 00 01 00 07 00 00 03 a1 86 01 00 08 00 00 00
  020: 82 00 01 00 07 00 00 03 a0 86 01 00 01 00 00 00
  030: 85 00 01 00 07 00 00 03 01 00 00 00 55 55 55 55
  040: 42 85 5c 79 88 cf e3 f2
  tag=0xf2e3cf88795c8542

V7 resp A: maker Fill of A4 from the TX core, maker range (echo_pkt 0) (40 B)
  000: 07 30 10 00 03 00 00 00 01 00 e8 03 00 00 00 00
  010: 82 ff 00 00 2d 00 00 01 a1 86 01 00 08 00 00 00
  020: b6 25 c0 08 da 1d e7 d1
  tag=0xd1e71dda08c025b6

V8 req  A: Heartbeat (48 B)
  000: 07 30 10 00 03 00 00 00 01 00 e8 03 01 00 00 00
  010: 10 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  020: 15 e1 48 48 1e f2 ba bf 00 00 00 00 00 00 00 00
  tag_seq=0xbfbaf21e4848e115  tag_com=0x0000000000000000

V9 resp A: EpochNotice (answer): next 1001, LDN box 1, 3600 s (40 B)
  000: 07 30 10 00 04 00 00 00 01 00 e8 03 03 00 00 00
  010: 86 00 00 00 e9 03 00 00 02 01 00 00 10 0e 00 00
  020: 4d f2 47 1a ea 6b 28 80
  tag=0x80286bea1a47f24d

V10 req C: CHAIN mode, New sell post-only 3 @ 100002 (48 B)
  000: 02 00 40 00 01 00 00 00 01 00 e8 03 01 00 00 00
  010: 01 09 00 00 a2 86 01 00 03 00 00 00 66 66 66 66
  020: 2f 67 e7 30 b4 b6 e4 b2 9c a5 89 aa 10 23 6a 3a
  tag_seq=0xb2e4b6b430e7672f  tag_com=0x3a6a2310aa89a59c

V11 resp A: out-of-stream Reject EPOCH_MISMATCH, detail 1000 (40 B)
  000: 07 30 10 00 00 00 00 00 01 00 e8 03 04 00 00 00
  010: 83 ff 0f 00 e8 03 00 00 00 00 00 00 00 00 00 00
  020: 2d f4 0f bc 51 46 ce 1a
  tag=0x1ace4651bc0ff42d

V12 req A: ResendReq(1, 1) (48 B)
  000: 07 30 10 00 04 00 00 00 01 00 e8 03 01 00 00 00
  010: 11 00 00 00 01 00 00 00 01 00 00 00 00 00 00 00
  020: 44 52 2e b1 e1 40 0c 9c 00 00 00 00 00 00 00 00
  tag_seq=0x9c0c40e1b12e5244  tag_com=0x0000000000000000

V13 resp A: resend of V2 (flags bit0, same resp_seq, new tag) (40 B)
  000: 07 30 10 00 01 00 00 00 01 01 e8 03 01 00 00 00
  010: 81 00 00 00 2a 00 00 01 a0 86 01 00 11 11 11 11
  020: cb 0a 1c 2c f0 0b 2f 66
  tag=0x662f0bf02c1c0acb

V14 req A: CancelAll (48 B)
  000: 07 30 10 00 05 00 00 00 01 00 e8 03 01 00 00 00
  010: 12 00 00 00 00 00 00 00 00 00 00 00 77 77 77 77
  020: c4 17 c2 a9 56 95 1a ee 00 00 00 00 00 00 00 00
  tag_seq=0xee1a9556a9c217c4  tag_com=0x0000000000000000

V15 resp A: CancelAck x2, summary CancelAck(HANDLE_ALL, 2) (72 B)
  000: 07 30 10 00 06 00 00 00 03 00 e8 03 05 00 00 00
  010: 84 00 00 00 2b 00 00 01 03 00 00 00 77 77 77 77
  020: 84 00 00 00 2e 00 00 01 06 00 00 00 77 77 77 77
  030: 84 00 00 00 ff ff ff ff 02 00 00 00 77 77 77 77
  040: f5 85 d4 43 dd fd c5 b5
  tag=0xb5c5fddd43d485f5

V16 req A: New buy IOC 2 @ 100000 (arrives in overload) (48 B)
  000: 07 30 10 00 06 00 00 00 01 00 e8 03 01 00 00 00
  010: 01 02 00 00 a0 86 01 00 02 00 00 00 88 88 88 88
  020: fd 45 e4 3b dc 2a da 03 00 00 00 00 00 00 00 00
  tag_seq=0x03da2adc3be445fd  tag_com=0x0000000000000000

V17 resp A: out-of-stream Reject BUSY (lane shed), index 0xFF, detail = expected 6 (40 B)
  000: 07 30 10 00 00 00 00 00 01 00 e8 03 06 00 00 00
  010: 83 ff 18 00 06 00 00 00 00 00 00 00 00 00 00 00
  020: 70 0e fd 35 ea 4e de dc
  tag=0xdcde4eea35fd0e70

V18 req A: resend of V16, same pkt_seq and content, bit0 = 1 (48 B)
  000: 07 30 10 00 06 00 00 00 01 01 e8 03 01 00 00 00
  010: 01 02 00 00 a0 86 01 00 02 00 00 00 88 88 88 88
  020: 37 e2 a3 17 d7 27 2c 64 00 00 00 00 00 00 00 00
  tag_seq=0x642c27d717a3e237  tag_com=0x0000000000000000
check OK: V1 verifies with K_SEQ
check OK: V1 fails with K_OUT
check OK: V1 with one tag bit flipped fails (bound source: close session)
check OK: V10 tag_com = SipHash(K_COM, bytes 0..32)
check OK: V13 tag differs from V2 tag
check OK: V17 is out of stream (resp_seq 0), reason BUSY 24, echo_pkt 6
check OK: V18 has V16's logical content (D3) and verifies

Expected lane behaviour for the negative vector: the flipped tag_seq comes from A's bound address, so the lane closes session A at V6 and sends Reject BAD_MAC (index 0xFF, detail 1) as the final in-stream record. The same bytes from any other address are dropped at V3 and counted, and A stays open. The same bytes with epoch16 = 999 get EPOCH_MISMATCH at V4 and close nothing (01 §14.8). For V17, the expected client action is: keep expected = 6, resend from 6 (V18), and do not count V16 as acknowledged.

12. Versioning#

The wire carries no version field. The protocol version is fixed per session at open (proto_version in BASE, the open transaction in CHAIN). This keeps the header at PKT_HDR_BYTES and the lane never branches on a per-packet version. This chapter is version 1.

Code space: request types 0x01–0x0F are order messages, 0x10–0x1F control, 0x20–0x7F reserved. Response types are 0x80–0xFF; v1 uses 0x81–0x86.

Planned use of reserved fields (informative; not part of v1):

13. Interfaces to other chapters#

Consumes:

Provides:

14. Local definitions#

All values are budgets unless marked otherwise.

Types: the SessionId bit layout (§4.1); OpenSessionReq, OpenSessionResp, KeyBundle, BoxKeys (§4.4); Cod = Off | On; Gcra (§9.1); bucket ids 1–7 (§9.1); the B-packet and L-packet (§7.1); the Attest record (§7.5).

15. Requests to 01#

Resolved by 01 §14 and dropped from this list: the bad-MAC close rule (now with epoch16, §14.8), Out causes 5–8, Attest 0x88 and DURABLE (§14.10, §14.13), BBO-core Shed scope and the E4 policer (§14.1), maker fills within 1 µs (§14.14) and in long sweeps (§14.17.5), the maker credit bound (STEP_FILL_CAP, §14.15), PacketRecord.flags (§6.6), INV-FRAME-FIRST for every response (§14.5), Fill.liq = 2 (§14.10), the FOK cause (§14.10), RESP_RETAIN_PKTS ownership and the lane ring (§14.6), and BAD_BOOK (§14.10).

1. §4 vs §12.5. §4 still says "the first failed MAC on a session closes the session". Reword it to §12.5 plus §14.8: bound address and matching epoch16.
2. INV-DET says "every response". Out-of-stream responses other than BUSY depend on packets that the order stream does not hold. Restrict it to in-stream responses and BUSY.
3. PktHash. A resend (flags bit0) and a resend into a new epoch (epoch16) change the hashed bytes. Compute PktHash with flags bit0 and epoch16 set to 0, so that every transmission of one pkt_seq hashes the same and equivocation is exactly a PktHash mismatch. Until then, 07 compares logical content (D3).
4. Reserved sequence values. State in §6.1 and §6.3: pkt_seq = 0 is a recovery request (§5.4); resp_seq = 0 is an out-of-stream response (§3.2). Response flags bits 1–7 are 0.
5. EpochNotice. Define the home word (byte 8 HomeCode, byte 9 box index) and the sequencer-change variant (next epoch = current epoch).
6. BUSY stream. State in §14.1 that a lane BUSY is out of stream (resp_seq 0, no holdoff) and a BBO-core BUSY is in stream at the message index (§3.5).
7. Maker numbering. State in §14.14 that the 16-number range is a relative credit placed by the merge rule of §7.1, not a fixed block: a fixed block leaves gaps when a maker stops trading, and lets a fill number below its order's Ack.
8. Promote the SessionId layout, PROTO_HANDLE_ALL (and the reserved slot 0xFF_FFFF), the reject detail codes of §14, the Day definition (the trading day ends at 00:00 UTC), and a TimerTick record in §6.6 with the control records listed in §13.
9. LANE_PACE wording. §14.15 says a paced client leaves "at least predicted_path_ns" between packets. predicted_path_ns is the remaining path to the wire (~560–645 ns), which would cap a dedicated lane at ~1.6M packets/s and 1B msgs/s would not close. §3.6 paces by the lane's occupancy lane_ns plus margin and jitter; say so in §14.15.
10. Maker count. §14.17.5's ~40 makers at index 9 needs ≤ 7.5 ns per maker packet per TX core; PROTO_MAKER_PKT_NS = 16 gives 18 on three TX cores. State the cost the 40 assumes.
11. Pool sessions. Say in §14.15 that a pooled SessionId names its home pool core and that every pool core accepts the pool's sessions (§4.7), and that WRONG_LANE is never sent inside the pool.

16. Open questions#