Project homepage Mailing List  Warmcat.com  API Docs  Github Mirror 
    npro  
 Modern all-safe Rust Network Protocol library supporting h1, h2, h3, ws, wt sans-IO and with socket IO + tls
git clone https://npro.rs/repo/npro
 
main main-devroot / fuzz / seeds / h1-request / leading-empty-2.http
Author[]Andy Green <andy@warmcat.com> 2026-10-06 09:19 UTC
Committer[]Andy Green <andy@warmcat.com> 2026-10-06 09:22 UTC
Treeb3dbaebdc1082ed6bd51c6597465fabaa0c57253   Raw Patch
 
docs: the IO model: driver, adapters, tls main main-dev
docs: the IO model: driver, adapters, tls

How npro meets sockets, tls and the user's scheduler, as discussed and
agreed on 2026-10-06.  The protocol crates never call IO: what could
block is a request they return, as C's want_write() and close() calls
become poll_request()'s values, so the same code runs under threads,
tokio, embassy or a loop of the user's own, and npro never spawns,
reads an ambient runtime, or bounds a connection by Send.

npro-io holds a no_std driver that keeps the ordering rules every
adapter would otherwise repeat, and the adapters as additive features:
std threads first, then mio, tokio and embassy.  tls is rustls inside
the driver behind a record-layer trait, never per adapter.

The port plan's phase 1g said a poll-style loop, but std has no poll();
that is the mio adapter, waiting on its dependency decision, and the
threads adapter comes first.  What was found about the rustls crypto
providers, and graviola's licence and platform limits, is recorded with
the decisions still open.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
diff --git a/docs/dependencies.md b/docs/dependencies.md index 410e98e..d35ef91 100644 --- a/docs/dependencies.md +++ b/docs/dependencies.md @@ -54,7 +54,7 @@ else is on it. A new workspace crate is added to it when it is created. | crates | may depend on | |---|---| | the protocol crates: npro-core, -h1, -ws, -h2, -h3, -quic, -wt, -mqtt, -http | nothing, except crates agreed below, each behind an opt-in feature, `no_std`, and with no build script | -| npro-io, the IO adapter | std, and what its tls decision admits | +| npro-io, the driver and the IO adapters | nothing without features; std, and the crates of each adapter's feature and of its tls decision ([io-model.md](io-model.md)) | | integrations with a runtime or framework, such as tokio | that runtime, in its own opt-in crate or feature, never a default | | npro-test, npro-fuzz, tools | what testing needs, admitted like anything else | | the libFuzzer targets in `fuzz/` | libFuzzer and what builds it, in a workspace of their own with its own `Cargo.lock` and `deny.toml`, so none of it enters the main workspace's graph | @@ -154,9 +154,15 @@ in the commit adding the libFuzzer targets. ## Open -- **tls**, for npro-io. rustls is the obvious candidate, but it needs a - crypto backend, and its usual ones (`ring`, `aws-lc-rs`) carry C and - assembly with build scripts. The alternatives are adapters to the - platform's tls or to OpenSSL. This is decided when npro-io needs it - (phase 1g and after, in [port-plan.md](port-plan.md)), and recorded - here. +- **tls**, for npro-io. rustls goes inside the driver, behind a + record-layer trait. rustls itself is `#![forbid(unsafe_code)]`; its + crypto provider is where the unsafe code is, and is the decision. What + was found about `aws-lc-rs`, `ring`, `rustls-rustcrypto` and `graviola` + is in [io-model.md](io-model.md). graviola leads for desktop and + server, but its licence (`Apache-2.0 OR ISC OR MIT-0`) is not MIT, and + it covers neither older CPUs nor microcontrollers. Decided when the tls + commit comes, and recorded here. +- **`mio`**, for npro-io's one-thread readiness loop (agreed in principle + 2026-10-06), with `libc`, which has a build script, and `windows-sys`. + Admitted, with what `cargo deny` shows they bring, when that adapter is + written. diff --git a/docs/io-model.md b/docs/io-model.md new file mode 100644 index 0000000..5ff198a --- /dev/null +++ b/docs/io-model.md @@ -0,0 +1,220 @@ +# The IO model + +How npro meets sockets, timers, tls and whatever scheduler the user has: +tokio, threads with blocking calls, embassy on a microcontroller, or an +event loop of their own. Discussed and agreed with Andy on 2026-10-06; +what is still open is listed at the end. + +## The rule + +**The protocol crates never call IO. Anything that could block is a +request the protocol returns, never an operation it calls.** + +Every protocol entry point is sync and non-blocking: it takes bytes and +`now` as values, does its work, and returns. That is the only kind of +function that both sync and async code can call, so the same protocol +code runs under any scheduler, and nothing in it is "coloured" sync or +async. + +C's sansIO half calls out to IO: `want_write()`, `close()`, the +deadline. npro turns those calls around: `conn.poll_request()` returns +`WantWrite`, `WantRead(bool)` or `Close(phase)`, and IO acts on the +value. An ops struct passed into the protocol would not give the same: +its `read` must be blocking, which an async caller cannot use without +parking a thread, or `async`, which a sync caller cannot use without a +runtime. A returned value also means no re-entry from a callback into +the library, which was a source of trouble in C. + +So npro promises, and no feature changes it: + +- it never spawns a task or a thread on the user's behalf; +- it never reads an ambient runtime or a global executor; +- it puts no `Send` or `'static` bound on a connection. A connection + is plain data, `Send` exactly when its storage and `Random` are. The + one place `Send` appears is the cross-context handle (below), in the + adapters that offer one. + +A protocol crate may take a trait only for a service that is sync, +cannot block, and does no IO: `Random`, and the tls record layer. Time +is a value, not a service. + +## Three layers + +1. **The protocol crates** (`npro-core`, `npro-h1`, `npro-ws`, ...): + `rx`, the `tx` pull, `deadline_passed`, `poll_request`, `poll_event`. + These exist. + +2. **The driver**, in `npro-io`, `no_std` without its `std` feature. It + holds a connection, its buffers and its tls, and keeps the rules + every adapter would otherwise have to get right on its own: the order + of rx, requests, tx and deadline; a hangup at any point; a drain that + says whether output was delivered or dropped; tls stepped by one + function whose first step may already finish the handshake. It does + no IO. It tells the adapter what it wants, as a set, and the adapter + tells it what happened. A first sketch, to be designed properly: + + ```rust + enum Want<'a> { + Io { read: Option<&'a mut [u8]>, write: Option<&'a [u8]>, until: Option<Instant> }, + Close, + } + // and back: read_done(now, n), write_done(now, n), timer(now), hangup(now) + ``` + + It must be a set, not one action at a time: a connection keeps + reading while it waits to write, or two peers each waiting to write + wedge. + +3. **Adapters**, which do the IO, each a short loop over the driver, + each behind its own `npro-io` feature. Features are additive: each + adds an adapter and changes nothing else. + + | adapter | feature | scale | needs | + |---|---|---|---| + | threads, blocking calls | `std` | a reader and a writer thread per connection; not for tens of thousands | std only | + | one-thread readiness loop | `mio` | many connections on one thread | `mio`, and through it `libc` (unix) and `windows-sys` | + | tokio | `tokio` | one task per connection, spawned by the user | tokio | + | embassy | `embassy` | connections fixed at build time (`pool_size`) | `embedded-io-async`, `embassy-time` | + + **Your own loop** is supported too, as C supports foreign loops + (libuv, libevent, glib, sd-event): drive the driver from it directly. + The driver is the lowest public level, and needs nothing from npro-io's + adapters. + +Notes for each adapter, from the discussion: + +- **threads**: the driver behind a `Mutex` shared by the two threads, + which is allowed in an adapter but not in a protocol crate. The writer + waits on a condvar for a write. The read timeout comes from `until`. + `shutdown()` on the socket wakes a reader blocked in `read()`. On + esp32 under esp-idf (std on FreeRTOS, sockets on lwIP) this runs + unchanged, but two thread stacks per connection cost RAM there; the + `mio` loop is the answer if that is too much. +- **mio**: std has no `poll()`. A readiness loop needs the OS's poll, + epoll or kqueue, and calling those is `unsafe`. mio does that in about + 170 marked places, and is what tokio stands on. npro's own code stays + `#![forbid(unsafe_code)]`. +- **tokio**: wait for readiness (`readable()`, `writable()`, a sleep + until `until`) and then do non-blocking `try_read` and `try_write`. + That way no buffer borrowed from the driver is held across an + `.await`, which is the same shape as a poll loop. The future is + `Send` when the connection is; a `&mut dyn TxSource` held across an + `.await` must be `dyn TxSource + Send`, or it is not. A compile-time + test asserts `Send`, so this cannot break silently. Plugging into + hyper or axum is a separate question: npro is an alternative stack, + not a hyper backend. +- **embassy**: buffers in `static` storage passed as `&'static mut`, not + inside the future, or every task becomes large. To check: whether + embassy-net's `TcpSocket::read_with` and `write_with` hand a closure + the socket's own ring buffer. If so, `rx` and the `tx` pull work on + it directly with no copy. + +### Writing from another task or thread + +`conn.send()` needs `&mut` access to the connection, so the +application's code has to run where the connection is: on the adapter's +thread, task or loop. In C this is `lws_callback_on_writable()` plus +`lws_cancel_service()`. An adapter may offer a `Handle`: a bounded +command queue plus a wakeup (a condvar, a waker, an eventfd). The +handle is where `Send + 'static` appears, and nowhere else. + +### What is written twice + +The user-facing conveniences, such as a blocking `client.get(url)` and +an async `client.get(url).await`, are written once per adapter, thin, +over the driver. Macro crates that generate both versions from one +source (`maybe_async` and others) are proc-macro dependencies, and are +not used; the duplication is kept small and visible instead. + +## tls + +rustls is itself sans-IO. It needs nothing from an event loop: its +`TimeProvider` takes wall time, and randomness comes from its crypto +provider. It goes inside the driver, once: + + socket -> rustls -> npro rx + npro tx pull -> rustls -> socket + +It is never wired in per adapter, so not `tokio-rustls`. The unbuffered +API (`UnbufferedClientConnection`, and its server equivalent) works on +caller-owned buffers, as the rest of npro does. + +rustls is one implementation behind a small record-layer trait in the +driver, not a fixed dependency, because the right tls differs by target. +rustls needs `alloc`, and around 16 KiB of record buffer per direction. + +### The crypto provider + +rustls itself carries no unsafe code; the providers are where it lives, +as in every tls stack, since constant-time and fast crypto ends up in +assembly or intrinsics. What was found on 2026-10-06: + +| provider | status | unsafe and build | platforms | +|---|---|---|---| +| `aws-lc-rs` | rustls' default; AWS | AWS-LC's C and assembly, compiled by its build script | wide | +| `ring` | its author stepped back in 2025; RUSTSEC-2025-0007 said it was unmaintained, then was withdrawn once the rustls team took on its maintenance. Commits are slow | mostly BoringSSL's C and assembly, compiled by its build script with a C compiler | wide | +| `rustls-rustcrypto` | outside the rustls project; its README says not for production, incomplete, not audited | pure Rust over the RustCrypto crates, but not free of unsafe: their fast paths are `unsafe` intrinsics or `asm!` behind runtime cpu detection (`sha2` 0.10.9: 29 uses; `cpufeatures`: 9), which some crates can be built without, slower. What remains even then is small and can't be avoided: `zeroize`'s volatile writes (15), so the compiler cannot drop a key wipe; `subtle`'s optimisation barrier (2); `getrandom`'s syscalls | anything Rust targets | +| `rustls-graviola` / `graviola` | by Joseph Birr-Pixton (ctz), rustls' original author; its README says it is "very new". 0.4.1, 2026-06-24 | formally verified assembly from AWS's s2n-bignum for the ECC, RSA and ML-KEM arithmetic, intrinsics for AES and SHA; no C compiler. Brings `cfg-if` and `getrandom` | **x86_64 and aarch64 only**, and only CPUs with the listed features (AES, SHA2, PMULL, NEON on aarch64, so no Raspberry Pi 4 or older; AVX2, ADX, BMI2, PCLMULQDQ and others on x86_64) | + +"Pure Rust" is not "no unsafe". Memory safety is also not the main +risk in crypto: timing side channels and correctness are, and that is +what rustls-rustcrypto's README warns about. The RustCrypto crates +underneath are widely used on their own, and on a microcontroller they +are likely to be in the tree anyway (`embedded-tls` uses them too). + +graviola is the best fit on desktop and server: no C, and verified +assembly where the arithmetic is. Its licence, `Apache-2.0 OR ISC OR +MIT-0`, is not MIT, and `deny.toml` admits only MIT, so it needs a +licence admitted as well as the crate. It does not cover CPUs without +those features, or any microcontroller. + +### esp32 + +Two different worlds, and the chip matters too: + +- **esp-idf**: std on FreeRTOS. The threads adapter works as it is. + tls can be rustls, since there is an allocator, but with a provider + that builds for the chip (graviola does not), or the IDF's own mbedTLS + with the chip's crypto hardware, through a separate binding crate that + nothing else depends on, as AGENTS.md allows for FFI. +- **esp-hal and embassy**: `no_std`, usually with no allocator. rustls + is out without `alloc`; `embedded-tls` is TLS 1.3 and client only, + over RustCrypto primitives. +- **The chip**: the RISC-V parts (C3, C6) build with the normal Rust + toolchain; Xtensa (the original esp32, S2, S3) needs Espressif's own + fork of the compiler, which complicates pinning the MSRV in CI. + +Device tests run on Andy's hardware, and their output comes back by +hand. + +## Proving the adapters behave the same + +The C transcripts that `npro-test` already replays against the protocol +crates are replayed through the driver, and then through each adapter's +loop over an in-memory transport, with identical results required. An +adapter that is not in that matrix is not claimed to behave the same. +Then real sockets: npro's client against C's servers, C's clients +against npro's servers, npro against itself, and autobahn both ways, over +plain TCP first and over tls after. These run locally until they pass, +then become sai tasks. + +## Decided + +- 2026-10-06: the driver lives in `npro-io`, which is `no_std` without + its `std` feature; adapters are further additive features of it. +- 2026-10-06: a `mio` readiness loop is supported, as well as threads, + tokio and embassy, and so is driving the driver from a loop of the + user's own. +- 2026-10-06: tests against C lws run locally until they pass, then + move into sai. + +## Open + +- The tls provider on desktop: graviola is the leading candidate, which + needs its licence admitted. What covers CPUs it does not, if anything. +- tls on esp32, which follows from which world and which chip. +- The driver's interface, designed by sketching all four adapter loops + against it, embassy's included, before it is fixed. +- Admitting `mio`, `libc` and `windows-sys` (and whatever `cargo deny` + shows they bring) when the `mio` adapter is written. `libc` has a + build script. diff --git a/docs/port-plan.md b/docs/port-plan.md index 1e5dba5..c73726f 100644 --- a/docs/port-plan.md +++ b/docs/port-plan.md @@ -364,15 +364,16 @@ page and the refusals the transcripts pin. - `npro-h1`: the h1 parser, chunked coding, and the h1 client and server transaction. - `npro-ws`: framing, handshake, close, and optional pmd. - - `npro-io`: std, sockets, the loop. It comes later, and tls later - still. + - `npro-io`: the driver, `no_std`, and the adapters (threads, mio, + tokio, embassy), each an additive feature, `std` among them. It + comes later, and tls later still ([io-model.md](io-model.md)). - `npro`: the facade. - `crates/npro-test` (`publish = false`): the transcript reader, the in-memory two-sided transport, the replay driver and the property-test helpers. - **Crate roots**: - every one carries `#![forbid(unsafe_code)]` and `#![no_std]`, - except `npro-io` and `npro-test`; + except `npro-test`, and `npro-io` when its `std` feature is on; - `alloc` only behind a feature; - `unexpected_cfgs` is denied in `[workspace.lints]`, along with `missing_docs` and the clippy set from AGENTS.md. @@ -851,7 +852,16 @@ and other extensions are not ported. ### Phase 1g: minimal `npro-io` -- std TCP, plain poll-style loop, no tls yet. Enough to run: +[io-model.md](io-model.md) is how npro meets sockets, tls and schedulers, +agreed 2026-10-06. For this phase: + +- the driver, `no_std`, replaying the transcripts over the in-memory + transport before any socket; +- the threads adapter (`std`), std TCP, no tls yet. The first form of + this plan said a poll-style loop, but std has no `poll()`; that is the + `mio` adapter, which waits for its dependency decision. The minimal + examples (ws echo server and client, http server and client) come with + it. Enough to run: - **autobahn**, both ways: `wstest -m fuzzingclient` against npro-io's ws echo server, and `-m fuzzingserver` against its echo client. This is a conformance gate. Where it runs is decided (2026-10-06): a sai @@ -867,8 +877,11 @@ and other extensions are not ported. compare with; - **a differential run** of the port's client against C's `minimal-http-server` and ws echo, and the reverse. -- tls: rustls is the obvious candidate. It is a large dependency tree, - so it is a separate decision, not taken here. +- the tests against C run locally until they pass, then become sai + tasks; +- tls: rustls inside the driver, behind a record-layer trait, once plain + TCP passes against C; the same tests again over tls. The crypto + provider is open: [io-model.md](io-model.md) has what was found. ### Later stages @@ -896,5 +909,6 @@ not pending output; mux parked rx; the kept-warm joiner's status. lists the stage-1 set only. - Whether `Random` is `&mut dyn` or a generic parameter. Generic avoids the vtable, but spreads a type parameter through every connection. -- tls in `npro-io`. +- tls in `npro-io`: the crypto provider, and tls on esp32 + ([io-model.md](io-model.md), "Open"). - Where autobahn runs in CI, and whether the C oracles are measured there (`sync-c-states.sh` and `sync-c-h1.sh` are run by hand).
Page fetched 0s ago, creation time: 4ms (vhost etag hits: 0%, cache hits: 0%)