Project homepage Mailing List  Warmcat.com  API Docs  Github Mirror 
    npro  
 Modern all-safe Rust Network Protocol library supporting h1, h2, h3, ws, wt sans-IO and with socket IO + tls
git clone https://npro.rs/repo/npro
 
root / crates / npro-test / h1 / requests / headers-tab-in-name.http
Author[]Andy Green <andy@warmcat.com> 2026-10-04 20:21 UTC
Committer[]Andy Green <andy@warmcat.com> 2026-10-05 06:32 UTC
Treee54799306caf56a025d304852898751db4d500c0   Raw Patch
 
docs: port plan: phase 1c is done, and how it departs from the plan
docs: port plan: phase 1c is done, and how it departs from the plan

What npro-h1 is, the two places it departs from C and why, and how it is
checked.  The differential test against C, which the plan had as a C
harness in an optional CI job, is a vendored oracle checked by every
cargo test, as the state machines' are; the checks table says so, and the
open question of where the C builds run is now where the oracles are
measured.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019kg5Eemy68ZaqDBcUJQG6J
diff --git a/docs/port-plan.md b/docs/port-plan.md index fd1bcb8..db7d6d3 100644 --- a/docs/port-plan.md +++ b/docs/port-plan.md @@ -1,6 +1,6 @@ # npro: plan for the sansIO port -Status: agreed, 2026-10-03. Phases 0, 1a and 1b are done. npro is the Rust port of +Status: agreed, 2026-10-03. Phases 0, 1a, 1b and 1c are done. npro is the Rust port of libwebsockets' sansIO half (https://npro.rs). This plan puts into practice the C tree's porting guide @@ -550,33 +550,67 @@ row by row. Typed per-role views for the protocol crates (an `H1Server` that can only be in its states) can sit on top when phase 1d has callers for them. -### Phase 1c: h1 parsing - -- **The request and response header parser.** It is byte-restartable, - with no recursion. Its limits come from config: - - `max_http_header_data`, 4096 by default and capped at 32768; - - the per-token limits; - - `WSI_TOKEN_COUNT` fragments. -- **Name matching.** This is a `match` on the lowercased name, not C's - generated lextable trie. The trie is an implementation detail; the - token set, the colon handling, and the method and version strings are - the behaviour. -- **URI decoding and normalisation.** `%XX`, then the control-character - refusal, then `//`, `/./` and `/../`, never above root, plus the - urlargs splitting. The refusals are the same as C's `LPR_REFUSED`: 403, - or 414 / 431 past a limit, pinned by the `h1-uri-*` and - `h1-header-past-limit` transcripts. -- **The chunked decoder**, shared by client and server. It requires at - least one hex digit, and extensions plus trailers are bounded at 4096 - bytes per body, as C does. -- **Tests**: - - the one-shot vs randomly fragmented oracle, as a property test; - - fuzz targets `h1-request`, `h1-response` and `chunked`, seeded from - C's `fuzz/fuzz-h1/seeds`, added as [fuzzing.md](fuzzing.md) says; - - a differential test against C on the same inputs, comparing the - parsed token table and the verdict. This needs a small C harness - built from the reference tree outside this repository; it is - optional in CI. +### Phase 1c: h1 parsing (done, 2026-10-04) + +The crate `npro-h1`, `no_std` with no dependencies, is C's +`lib/sansio/http/parsers.c`: + +- **`token`**: C's `enum lws_token_indexes`, its 97 tokens with C's + indices and spellings, with every header option on as C's default build + has it. Name matching is `lookup()` over the spellings: no spelling is + the start of another, so a name is matched exactly when C's lextable + reaches its terminal, and the trie itself is not ported. +- **`table`**: C's ah, `HeaderTable<S>` over caller-owned storage, at most + 32768 bytes. Its layout is C's byte for byte in what it uses up (each + value's NUL, each unknown header's eight byte record, the `?`'s unused + byte, C's 97 fragment slots), so a head fills it at the same byte as C's. + Presence is a `Slot::Present`, not a nonzero index. A client's own + request goes in with `create()`, and an interim response is dropped with + `snapshot()` and `rewind()`. +- **`head`**: C's `lws_parse()` and `lws_parse_urldecode()`, byte for byte + restartable. C's `parser_state`, `ues`, `ups`, `post_literal_equal`, + `lextable_pos` and `unk_pos` are enums. A refusal is a `Cause`, one per + way out of C's parser, and what a server answers for it (`LPR_REFUSED`'s + 400, 403, 414, 431, 501 or 505, or none for `LPR_FAIL`). Limits are the + table's size and per-token `Config::with_limit()`, C's `token_limits`. +- **`chunked`**: C's `lws_http_dechunk_framing()`, handing back the payload + where it lies in the input; **`fields`**: C's Content-Length and + lone-`chunked` Transfer-Encoding readers. + +Porting it found three bugs in C, fixed there first (lws 3c8459075 and +the two before it), so npro follows C as it now is: C marked "no name +begun" by the name's record being at offset 0, which a server's first +name is, so it began that name twice, losing nine bytes of every +request's ah, and took a LF as a request's second byte as a bare LF; C's +strict server took a header line starting with a bare CR into an unknown +header's name; and a repeated header's value kept its leading spaces, +where RFC 9110's OWS is not part of a value. + +**The tests**, in `crates/npro-test`: + +- **The differential test against C**, `tests/h1_c.rs`. The plan had it as + a C harness in an optional CI job; it is instead an oracle vendored like + the state machines': `scripts/sync-c-h1.sh` builds C static, compiles + `h1/c-heads.c` against its private headers to call `lws_parse()` and the + dechunker directly, and records what C makes of a corpus (C's fuzz-h1 + seeds and heads written to reach each branch, as a server and as a + client, and chunked bodies), with twelve variations of each, in C's + default configuration and a tight one: 7,007 cases. npro must reach + the same verdict and leave the same table, down to the bytes used. +- **The one-shot vs fragmented oracle** over the same heads, each split + three ways, and over every transcript's first request split at each + byte. +- **The transcripts**: the `h1-uri-*`, `h1-reqline-*` and + `h1-header-past-limit` requests come to the path, urlargs or refusal C's + `sansio-uri` app answered with, and the h1 client cases' heads and + framing headers read as C read them (`tests/h1_heads.rs`). +- **Fuzz targets** `h1-request`, `h1-response` and `chunked`, seeded from + C's `fuzz/fuzz-h1/seeds` and the corpus ([fuzzing.md](fuzzing.md)). + +**Exit check, met.** Every case of the vendored oracle agrees with C, +and planted bugs in the +parser and dechunker (a limit off by one, `+` left alone in the query, the +continuation's SP dropped) each fail it. ### Phase 1d: the h1 transaction, server and client @@ -678,7 +712,7 @@ not pending output; mux parked rx; the kept-warm joiner's status. | state edge set vs C trace | C `LWS_WITH_STATE_TRACE` | `cargo test`, vendored edge file | | one-shot vs fragmented parse | agent-context "Parsers" | proptest, every parser | | fuzz, with an oracle per target | C's `fuzz/fuzz-*/seeds`, copied into `fuzz/seeds/` | smoke tests in `cargo test` everywhere; libFuzzer in sai CI and idle time ([fuzzing.md](fuzzing.md)) | -| differential parse vs C | C tree built outside the repo | optional CI job | +| differential parse vs C | C's `lws_parse()` and dechunker over a corpus, `scripts/sync-c-h1.sh` | `cargo test`, vendored `c-heads.txt` | | autobahn / h2spec / h3spec | conformance suites | per phase, once `npro-io` exists | | lints, docs, deny, audit, MSRV | AGENTS.md | CI, every commit | @@ -690,4 +724,4 @@ not pending output; mux parked rx; the kept-warm joiner's status. - Whether `Random` is `&mut dyn` or a generic parameter. Generic avoids the vtable, but spreads a type parameter through every connection. - tls in `npro-io`. -- Where autobahn and the C differential builds run in CI. +- Where autobahn runs in CI, and whether the C oracles are measured there (`sync-c-states.sh` and `sync-c-h1.sh` are run by hand).
Page fetched 0s ago, creation time: 2ms (vhost etag hits: 0%, cache hits: 0%)