npro: plan for the sansIO portStatus: agreed, 2026-10-03. Phases 0 and 1a are done. npro is the Rust port of libwebsockets' sansIO half (https://npro.rs). This plan puts into practice the C tree's porting guide
( The work is split into phases. Each phase is a series of commits, and each commit builds, passes its tests and has one purpose. A phase ends when its exit check passes, not when its code has been written. 0. Decisions needed before any codeThese are Andy's calls. Each one has a recommendation and the reason for it.
C's
Two items from the earlier list remain for the port to settle itself.
Each gets a test in the port:
- Fragment-order violations on tx are still only
1. What the C tree gives us to check againstWhat exists today on
They still include no request or response body framing, no ws close
started by the application, no ping, and no timers.
- Fuzz seeds: The port cannot be checked beyond what those cover. So phase 1 includes
adding transcripts in C first, each a case in
That is C work, committed to the C tree, and it lands before the Rust equivalent is claimed done. 1a. C work before the C tree is a clean basisFirst taken stock of on Resolved since the first lookEach item below is fixed on
Resolved at |
| C | Rust |
|---|---|
rx(bytes) -> consumed, empty = peer closed | conn.rx(now, &mut [u8]) -> Result<Rx<'_>, Error>, with rx.consumed and rx.event: Option<Event<'_>>. It takes &mut so that ws unmasks in place, and it returns at the first event, borrowing its payload from the input. The caller loops while progress is made ("a parse that consumed less is a held byte"). conn.rx_closed(now) is the peer closing |
tx(buf, max) -> n, more | conn.tx(now, &mut [u8], &mut impl TxSource) -> Tx, with written and more. The connection's own frames (status lines, 101, CLOSE, PONG) come from its state. The application's payload is pulled from TxSource into the buffer behind the frame header the connection has reserved. That is C's LWS_PRE framing in place turned into a pull, with no copy and no parking. This is "Open before it is done" item 4 |
deadline() | conn.deadline_passed(now). conn.next_deadline() -> Option<Instant> is a query, not a request |
want_write, want_read(on/off), close(phase) | conn.poll_request() -> Option<Request>: WantWrite, WantRead(bool), Close(Phase). Only the phases that exist for stage 1: quiesce, shutdown, stage, release |
transport(up/failed/gone) | conn.transport_up(now, TransportInfo), carrying alpn and tls-ness, which IO sets before up; conn.transport_failed(now) |
| callbacks (125 reasons) | Event variants returned from rx / poll_event(), and the application's calls made between them. Stage 1 needs about 15: request headers, body chunk, body done, response headers, response body, ws established, ws message fragment, pong, peer close, closed, writeable |
lws_service_set_now() | now: Instant is an argument of every entry point. Instant is a newtype of µs u64 and WallTime one of seconds. Monotonicity is enforced (debug assert plus saturating) |
lws_get_random() | a Random trait, handed in where drawn (&mut dyn or generic: to be settled in phase 1a) |
Buffers without alloc:
- The header table. C's
ahis a pool ofmax_http_header_data-byte tables with a fragment index. The RustHeaderTable<S: AsMut<[u8]>>has its capacity fixed byS:[u8; 4096]on a device, orBox<[u8]>withalloc. The fragment index is a fixed array of(offset, len)newtypes per known token. Header presence is explicit, as aPresentvariant, not a length. Unknown headers are records in the same storage. - Rx. The ws rx buffer does not exist as such: payload is delivered
from the input slice, in fragments, with
rx_buffer_sizeas the fragment cap. - Tx. The tx side owns nothing but its position.
The connection is four machines, each its own enum, with the role,
the side and the socket's usability: npro_core::state::Machines. As
built in phase 1b, which departs from the first form of this plan (per-role
enums carrying each state's data) for reasons given there:
Transport,Carrier,Live(the transaction) andClose, each one enum shared by every role, as in C; all of C's transport phases are declared. The fields are private, and only the event table writes them.Role(None,H1,Ws,RawSktso far) andSide; a role or side change is a transition like any other.- Events: C's
LWS_WSIEV_*are theEventenum. The transition function is onematchon the event, exhaustive, so an event cannot be added without its rows; within it, the rows are C's, in C's order, with_for C's"*"andANY. An event with no row is refused, as C refuses it (andLWS_WITH_STATE_CHECKaborts). TheANYrows are C's as they are:RESTART,CONN_FAILED,RETARGET, the close events. - Trace: every change is an
Edge, whoseDisplayis the line C'sLWS_WITH_STATE_TRACEwrites, less the connection's tag:LRS h1/S:HEADERS -> h1/S:H1_UPGRADE set_state ev=REQ_HDRS_COMPLETE. It needs no feature: writing lines is the IO side's business. That makes C's edge set diffable against the port's: README.sans-io-split.md rule 3, "the trace is the oracle". - Close invariants are refusals, as C's check makes them aborts: no
polite close phase entered with the socket unusable, no shutdown on a
raw socket, the close never going backwards, no live state while a
client restarts. The rows themselves keep
RETURNED_CLOSEto ws andSHUTDOWNto servers.
3. Phases
Phase 0: scaffold and gates (done, 2026-10-03)
Three commits:
- The workspace (crates/*, with the npro facade), its
[workspace.lints], clippy.toml, deny.toml, the README, and
scripts/ci.sh. CI runs fmt, clippy (with all features and with the
defaults), test, doc, the MSRV 1.85 check, the no_std build, cargo deny
and cargo audit.
- npro-test: a strict reader for lws-transcript/1, and copies of
the C transcripts with C-COMMIT and the C README beside them.
scripts/sync-c-oracle.sh refreshes them from a C checkout. C's fuzz
seeds join fuzz/seeds/ with the targets that use them.
Exit check, met: every gate passes at 1.85 and at stable, and the reader takes all 48 transcripts.
Phase 1a: substrate in the core crate (done, 2026-10-03)
- Random: the
Randomtrait, one draw perfill(), failing asUnavailable.SeededRandom, behind the additivereplayfeature, is C's stream; it is pinned by theapi-test-random-prngvector and by the ws-client transcript's key and first mask. sha1andbase64(encode only), with their RFC vectors and RFC 6455's accept value.- The incremental UTF-8 validator. It is checked exhaustively
against
core::str::from_utf8, and once, outside the tree, against C's ownlws_check_utf8(): 134M cases, none differing. time::Instant, in microseconds, withcore::time::Durationfor intervals. Wall time and deadlines come with their first users.- Ids and size newtypes move to the crates that define them: a
StreamIdbelongs to h2.
Property tests need no dependency. The generators are written over
SeededRandom, so a failing case is reproduced from its seed. Every
parser gets the one-shot vs fragmented oracle.
Exit check, met: the vectors pass, and clippy passes with
indexing_slicing, arithmetic_side_effects and as_conversions
denied.
Fuzzing (done, 2026-10-03)
Ahead of the parsers, so that each is fuzzed from its first commit:
crates/npro-fuzz holds a harness per target with an oracle, smoke-tested
in cargo test; fuzz/ holds the libFuzzer targets in a workspace of its
own; scripts/fuzz.sh runs them by hand and under sai, in CI and idle
time, with the corpora in a sai pool. The first targets are the
substrate's: utf8, sha1, base64, and the transcript reader.
fuzzing.md has the details.
Phase 1b: the connection machines (done, 2026-10-04)
npro_core::state: the four machines, the role, side and socket, theEventenum and C's event table rows for the stage-1 roles: h1 client and server, ws, and raw sockets. The setters are C's (lws_wsi_set_state_ev(),lws_wsi_role_transition_ev()): a live state ends the transport phase, a handshake-named state is the carrier's until it is established, a restart leaves the old socket and close behind.- The oracles, in
crates/npro-test/states/from C2dc2a33c(scripts/sync-c-states.shrefreshes them): C's table rows verbatim, and the 373 distinct edges C's whole ctest suite takes with the trace and the check on (248 tests, all passing; 202 of the edges are between stage-1 roles). - The tests,
crates/npro-test/tests/states.rs: - a second model of C's machines, in C's terms, reading its rows from the copy of C's table. Every state the port reaches from a birth (3,022 of them, walked exhaustively, which subsumes the property tests this plan asked for) is driven with every event, with and without each role a site can give, through both: 604,400 cases, agreeing on refusal, machines after, setter and trace line; - every one of C's 202 stage-1 edges is one the port takes; - the structural invariants hold in every reachable state.
Each was checked by planting bugs: a wrong row, a missing row, a setter keeping the transport or the dead socket, the close going backwards, no unusable-socket rule, tracing every edge. Every one fails the tests.
Exit check, met, with one difference listed for agreement. The port takes every edge C takes, and takes none C's table does not. The other direction of the plan's comparison, every edge the port can take seen in C's trace, does not hold and is not meant to: the port reaches 4,047 traced edges, C's suite exercises 202, and the rest are edges C's table allows that its tests do not reach (C's own comment: "statically present edges no test reaches"). Row-level equivalence with C's table replaces that direction.
Departure from section 2's first form, to agree: the machines are one
enum each, shared by the roles, as C's are, rather than per-role enums
carrying each state's data. C's machines cross roles: the close machine
is one ordered sequence that ws phases sit inside, and a role change
carries it (h1 to ws); the carrier's names are reused per transaction.
Keeping C's shape is what lets the port be held to C's table row by row.
Typed per-role views for the protocol crates (an H1Server that can only
be in its states) can sit on top when phase 1d has callers for them.
Phase 1c: h1 parsing
- The request and response header parser. It is byte-restartable,
with no recursion. Its limits come from config:
-
max_http_header_data, 4096 by default and capped at 32768; - the per-token limits; -WSI_TOKEN_COUNTfragments. - Name matching. This is a
matchon the lowercased name, not C's generated lextable trie. The trie is an implementation detail; the token set, the colon handling, and the method and version strings are the behaviour. - URI decoding and normalisation.
%XX, then the control-character refusal, then//,/./and/../, never above root, plus the urlargs splitting. The refusals are the same as C'sLPR_REFUSED: 403, or 414 / 431 past a limit, pinned by theh1-uri-*andh1-header-past-limittranscripts. - The chunked decoder, shared by client and server. It requires at least one hex digit, and extensions plus trailers are bounded at 4096 bytes per body, as C does.
- Tests:
- the one-shot vs randomly fragmented oracle, as a property test;
- fuzz targets
h1-request,h1-responseandchunked, seeded from C'sfuzz/fuzz-h1/seeds, added as fuzzing.md says; - a differential test against C on the same inputs, comparing the parsed token table and the verdict. This needs a small C harness built from the reference tree outside this repository; it is optional in CI.
Phase 1d: the h1 transaction, server and client
- Server (
lws_http_actionrules, in C's order): CL with TE gives 400; more than one Host gives 400; Expect gives 417 or a 100; TE other than a lonechunkedgives 501; CL is parsed strictly with overflow checked, and over the maximum gives 413; plus Connection keep-alive rules by version, the body states,DISCARD_BODY, the pipelining limit of 64, and the TXN_COMPLETED / drained handling. - A request with neither CL nor TE has no body, per RFC 9112 6.3 rule 7. C's server still reads a POST, PUT or PATCH body to the close, while its proxy path already treats it as zero-length. See section 1a, open item 3: C decides, and a transcript pins it, before the port claims it.
- Client:
- request composition in C's exact header order (the transcript pins
it);
- 1xx swallowed up to 8, with the header table rewound;
- CL and chunked bodies, and read-to-EOF when there is neither;
- the body pulled at the application's pace, never buffered;
- the kept-warm
IDLINGstate with its deadline. - Body bytes that arrived with the headers are kept. With the borrow-from-input rx this is structural: the rx simply does not consume them until the body event.
Exit check: h1-client-get and the first exchange of h1-ws-server
replay byte for byte, plus the new C transcripts of section 1 that cover
h1.
Phase 1e: ws
- The handshake.
- Server upgrade:
- the Connection token must include
upgrade; - Sec-WebSocket-Key must be present and under 128 bytes; - the subprotocol is the first known one, or the default; - the 101 is written in C's order, withUpgrade: WebSocketcapitalised as the transcript has it. - Client: 16 random bytes for the key, then the response is checked: status 101, Accept, Upgrade and Connection, and the offered protocol. - The frame parser. - Server and client are one parser parameterised by side, not two as in C. - It checks opcodes, RSV, continuation and FIN ordering, masking by side, control-frame length and fragmentation, and the 64-bit length top bit. - The configured frame and message maxima apply on both sides. - UTF-8 is validated incrementally, with close codes 1002 and 1007 as C uses them.
- Tx framing as a pull, the mask drawn per frame.
- The close machine:
-
WaitingToSendClose, thenAwaitingCloseAck; -ReturnedClose, where the peer's payload is echoed; -FlushingBeforeClose; - the received-code rewrite to 1002 per side; - 5 s CLOSE_SEND and CLOSE_ACK deadlines, 3 s on the client's reply; - the one-pending-pong rule, and the POLLOUT priority order: CLOSE, then PONG or the echoed CLOSE, then keepalive PING, then the application. - Keepalive: 40 / 50 s by default, with the ping payload being the
8-byte
now.
Exit check: ws-client (seeded) and h1-ws-server replay byte for
byte, as do the new ws transcripts. The fuzz target is seeded from
fuzz/fuzz-ws/seeds.
Phase 1f: permessage-deflate (feature pmd)
- Negotiation, the parameters with their C ranges, the zip-bomb cap of 256 MiB per message (configurable), and the drain budget.
- Parameters are chosen per direction by role, as C now does (
c85cd0c). - The fuzz target is seeded from
fuzz/fuzz-ws-pmd/seeds, which includesbomb-2mb-zeros.
Phase 1g: minimal npro-io
- std TCP, plain poll-style loop, no tls yet. Enough to run:
- autobahn's fuzzingclient against the ws server. This is a
conformance gate. It needs a Python tool, so Andy must say where it
may run;
- a differential run of the port's client against C's
minimal-http-serverand ws echo, and the reverse. - tls: rustls is the obvious candidate. It is a large dependency tree, so it is a separate decision, not taken here.
Later stages
Later stages follow the port guide's order and wait for their C transcripts: h2 (stage 2), quic and h3 (stage 3), and mqtt (stage 4). The guide's "Do not copy" list stays in force: a quic frame in flight is not pending output; mux parked rx; the kept-warm joiner's status.
4. Checks at a glance
| check | from | where it runs |
|---|---|---|
| transcripts byte for byte | C api-test-sansio | cargo test, every commit |
| state edge set vs C trace | C LWS_WITH_STATE_TRACE | cargo test, vendored edge file |
| one-shot vs fragmented parse | agent-context "Parsers" | proptest, every parser |
| fuzz, with an oracle per target | C's fuzz/fuzz-*/seeds, copied into fuzz/seeds/ | smoke tests in cargo test everywhere; libFuzzer in sai CI and idle time (fuzzing.md) |
| differential parse vs C | C tree built outside the repo | optional CI job |
| autobahn / h2spec / h3spec | conformance suites | per phase, once npro-io exists |
| lints, docs, deny, audit, MSRV | AGENTS.md | CI, every commit |
5. Not settled by this plan
- The core crate's name (decision 0.1).
- The mapping from each of C's callback reasons to an
Event. This plan lists the stage-1 set only. - Whether
Randomis&mut dynor a generic parameter. Generic avoids the vtable, but spreads a type parameter through every connection. - tls in
npro-io. - Where autobahn and the C differential builds run in CI.