Project homepage Mailing List  Warmcat.com  API Docs  Github Mirror 
    npro  
 Modern all-safe Rust Network Protocol library supporting h1, h2, h3, ws, wt sans-IO and with socket IO + tls
git clone https://npro.rs/repo/npro
 
root / crates / npro-test / transcripts / h1-connect-rejected-ua.json
Author[]Andy Green <andy@warmcat.com> 2026-10-03 19:45 UTC
Committer[]Andy Green <andy@warmcat.com> 2026-10-05 06:32 UTC
Tree03f17e6981ec8bb8f9c078083aa157b5d9595a4c   Raw Patch
 
fuzz: libFuzzer targets, run by sai on push and in idle time
fuzz: libFuzzer targets, run by sai on push and in idle time

The harnesses in npro-fuzz get libFuzzer targets, and sai runs them the
way it runs the C tree's.

fuzz/ is a cargo-fuzz workspace of its own, with its own Cargo.lock and
deny.toml, so nothing fuzzing needs enters the main workspace's graph.
Each target is one line handing libFuzzer's input to its harness, and
the crate keeps forbid(unsafe_code).

Admitted to the fuzz workspace only, with their register entries in
docs/dependencies.md: libfuzzer-sys (the fuzz_target! macro, and the
libFuzzer runtime it compiles from the C++ sources it carries, under
NCSA as well as MIT/Apache-2.0), arbitrary, which it depends on, and
what compiles libFuzzer in its build script: cc, find-msvc-tools,
jobserver, libc and shlex.  Build scripts are allowed for libfuzzer-sys
and libc only, neither touching the network.  arbitrary's package
carries its maintainer's publish.sh, never run by a build, let through
by its checksum.  The graph checked is Linux's, where fuzzing runs.
This route was chosen over linking the builder's libFuzzer through a
shim of npro's own, which would have needed unsafe code in npro.

scripts/fuzz.sh follows the C tree's fuzz/run.sh: it checks the fuzz
workspace with cargo deny and fetches it --locked before building, then
builds with cargo-fuzz, ASan and debug assertions, and runs each target
with its corpus and seeds.  Under sai it keeps the corpora in the pool,
replays the known bugs' reproducers in CI and fails while one still
crashes, sends findings only to sai-server, and in idle slices shares
SAI_IDLE_SECS between the targets in turn.  scripts/sai.sh gets a fuzz
profile, which runs libFuzzer with -verbosity=0 unless the builder
sets FUZZ_OPTS, as C's fuzz job does, so the log keeps each target's
start and final stats rather than a line per new unit; and .sai.json
the C tree's fuzz builder and a fuzz
configuration with two idle lanes and the "fuzz" pool, which belongs to
this repo and is not C's.

Checked here: every target runs clean, at 60k to 190k execs/s; with a
bug planted (utf8 accepting surrogates) a simulated sai run finds it,
puts the input and report in SAI_POOL_FINDINGS and nothing of the
report in the log, fails; replaying it as a known bug fails while the
bug is there and leaves ok-<sha1> once it is fixed; an idle slice
picks its targets and records whose turn is next in the pool.

docs/fuzzing.md says what each target checks, how to run and replay,
how it works under sai, how to add a target, and what is not done:
sai-server groups findings by stack frames and does not yet skip Rust's
panic machinery, so every panic in a target would be one bug there.
docs/sai.md has the builder's setup, the plan and README point at it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019kg5Eemy68ZaqDBcUJQG6J
diff --git a/.gitignore b/.gitignore index ea8c4bf..54bf757 100644 --- a/.gitignore +++ b/.gitignore @@ -1 +1,6 @@ /target +# what cargo fuzz run makes by default, used by hand in fuzz/ +/fuzz/target +/fuzz/corpus +/fuzz/artifacts +/fuzz/coverage diff --git a/.sai.json b/.sai.json index f57071a..12cedb2 100644 --- a/.sai.json +++ b/.sai.json @@ -38,6 +38,10 @@ "default": false, "build": [ "scripts/sai.sh ${cmake}" ] }, + "fuzz-linux-debian13/x86_64-amd/gcc": { + "default": false, + "build": [ "scripts/sai.sh ${cmake}" ] + }, "w11/x86_64-amd/msvc": { "default": false, "build": [ "cargo test --workspace --all-features --locked -j %SAI_PARALLEL%" ] @@ -111,6 +115,27 @@ "c-oracle": { "cmake": "oracle", "platforms": "none, linux-fedora44/x86_64-amd/gcc" + }, + + # libFuzzer over every fuzz target (scripts/fuzz.sh), 60s each, + # after replaying the reproducers of the bugs found so far. The + # corpora are in the "fuzz" pool, which sai keeps synced between + # every builder fuzzing npro, so coverage keeps advancing; pools + # belong to a repo, so this is not the C tree's "fuzz" pool. + # Findings go to sai-server through the pool, which shows them + # only to admins: they can be unfixed security bugs, and logs are + # public. + # + # Two idle tasks per platform carry on fuzzing the latest + # completed push in builders' idle time, where their conf allows + # it, sharing each slice (SAI_IDLE_SECS) between the targets in + # turn. docs/fuzzing.md has the details. + + "fuzz": { + "cmake": "fuzz 60", + "platforms": "none, fuzz-linux-debian13/x86_64-amd/gcc", + "idle": 2, + "pool": "fuzz" } } } diff --git a/README.md b/README.md index ce9ea39..72d0779 100644 --- a/README.md +++ b/README.md @@ -31,7 +31,9 @@ sansIO half is the specification, and the port is held to it by: which npro replays; - **the C state tables**: every state transition C allows, from `lib/sansio/wsi-state.c`, which npro's state enums must match; -- **fuzzing**, seeded from the C library's fuzz corpora; +- **fuzzing**, each target checked against an oracle, on every build + and in sai's idle time, seeded from the C library's fuzz corpora + ([docs/fuzzing.md](docs/fuzzing.md)); - **differential and conformance testing**, against C lws and the autobahn, h2spec and h3spec suites, as the IO crate makes them possible. diff --git a/docs/dependencies.md b/docs/dependencies.md index 16a4bf3..c2da44b 100644 --- a/docs/dependencies.md +++ b/docs/dependencies.md @@ -56,7 +56,8 @@ else is on it. A new workspace crate is added to it when it is created. | the protocol crates: npro-core, -h1, -ws, -h2, -h3, -quic, -wt, -mqtt, -http | nothing, except crates agreed below, each behind an opt-in feature, `no_std`, and with no build script | | npro-io, the IO adapter | std, and what its tls decision admits | | integrations with a runtime or framework, such as tokio | that runtime, in its own opt-in crate or feature, never a default | -| npro-test, fuzz targets, tools | what testing needs, admitted like anything else; fuzz targets live in their own workspace with their own lock file | +| npro-test, npro-fuzz, tools | what testing needs, admitted like anything else | +| the libFuzzer targets in `fuzz/` | libFuzzer and what builds it, in a workspace of their own with its own `Cargo.lock` and `deny.toml`, so none of it enters the main workspace's graph | | npro, the facade | the npro crates only | A protocol crate never depends on an IO crate, an async runtime, or a @@ -95,6 +96,8 @@ Removing a dependency is the same in reverse: take it out of ## The register +### The main workspace + Admitted: **none**. npro builds from its own sources and the Rust toolchain alone. @@ -102,6 +105,35 @@ toolchain alone. |---|---|---|---|---|---|---| | (none yet) | | | | | | | +### The fuzz workspace + +`fuzz/` builds the libFuzzer targets ([fuzzing.md](fuzzing.md)) and +nothing else; no npro crate depends on it. Its door is `fuzz/deny.toml`, +which `scripts/fuzz.sh` checks before it builds anything. It checks the +Linux graph only, since that is where fuzzing runs: on Windows, +`jobserver` would also bring `getrandom`, `r-efi` and `cfg-if`. + +The choice was between this, the standard Rust route, and linking the +builder's own libFuzzer runtime through a shim of npro's own. The shim +would have admitted no crates, but needed `unsafe` code in npro to turn +libFuzzer's pointer and length into a slice; npro took the crates, so +that none of its own code needs qualifying as safe. + +| crate | why | build script | unsafe | licence | +|---|---|---|---|---| +| `libfuzzer-sys` 0.4.13 | the `fuzz_target!` macro, and libFuzzer's runtime, whose C++ sources it carries | compiles those sources with `cc`; nothing else, and no network | the macro's entry point, taking libFuzzer's pointer and length | (MIT or Apache-2.0) and **NCSA**, the LLVM licence of libFuzzer's sources, admitted for this workspace only | +| `arbitrary` | a dependency of `libfuzzer-sys`, for targets taking structured input; npro's take bytes | none. It ships its maintainer's `publish.sh`, never run by a build, admitted by checksum | some | MIT or Apache-2.0 | +| `cc` | compiles libFuzzer, from `libfuzzer-sys`' build script | none | some | MIT or Apache-2.0 | +| `find-msvc-tools` | split out of `cc` | none | some | MIT or Apache-2.0 | +| `jobserver` | `cc`'s share of cargo's parallel jobs | none | some | MIT or Apache-2.0 | +| `libc` | `jobserver` and `cc` on unix | probes the rustc version, no network | the C bindings it is for | MIT or Apache-2.0 | +| `shlex` | `cc`'s parsing of compiler flags from the environment | none | a little | MIT or Apache-2.0 | + +Apart from `libfuzzer-sys` and `arbitrary`, which are linked into the +targets, they are build-time only: they run on the fuzz builder while the +targets build. All were admitted +in the commit adding the libFuzzer targets. + ## Decided, not yet admitted - **`miniz_oxide`**, with its dependency `adler2`, for permessage-deflate, diff --git a/docs/fuzzing.md b/docs/fuzzing.md new file mode 100644 index 0000000..249397d --- /dev/null +++ b/docs/fuzzing.md @@ -0,0 +1,149 @@ +# Fuzzing npro + +Everything in npro that reads bytes from outside is fuzzed, and each fuzz +target checks the code under test against an **oracle**: another way of +getting the same answer, or a property the answer must have. A target that +only waits for a panic finds out little about code that cannot panic; one +with an oracle finds wrong answers too. + +Fuzzing runs in three places: + +| where | what | when | +|---|---|---| +| `cargo test` | the smoke tests: every target over its seeds and 4000 inputs from a fixed seed | every build, every builder, Miri included | +| sai `fuzz` | the known bugs replayed, then libFuzzer for a minute on each target | every push | +| sai idle tasks | libFuzzer, the targets taking turns | builders' idle time, on the latest completed push | + +## The targets + +| target | the code under test | the oracle | +|---|---|---| +| `utf8` | `npro_core::utf8::Utf8Validator`, which checks ws text as it arrives | `core::str::from_utf8` over the whole text. Fed a byte at a time, the validator must refuse at exactly the first byte no well-formed text could have next; fed in pieces of 0 to 16 bytes, it must refuse in the piece holding that byte, and go on refusing | +| `sha1` | `npro_core::sha1::Sha1`, for the ws accept | the digest of the message fed in pieces equals the digest of it in one go | +| `base64` | `npro_core::base64::encode` | a strict decoder in the harness gets the input back; exactly `encoded_len` bytes are used and no more written; a buffer one byte short, or empty, is refused with nothing written | +| `transcript` | npro-test's transcript reader | whatever it accepts keeps its limits, and step times never go backwards | + +A target that splits its input to feed it in pieces takes the split from +the input's first byte, so libFuzzer explores the split like the rest of +the input. + +The h1 parser's targets come next, in phase 1c of +[port-plan.md](port-plan.md), seeded from the C library's corpora. + +## Where things are + +- `crates/npro-fuzz`: the harnesses, one safe `fn(&[u8])` per target, and + `Target`, which names them. Part of the main workspace, with no + dependencies outside it. Its `tests/smoke.rs` is the smoke tests. +- `fuzz/`: the libFuzzer targets, one line each, which hand libFuzzer's + input to a harness. A **workspace of its own**, with its own + `Cargo.lock` and `deny.toml`, so what fuzzing needs to build + (`libfuzzer-sys` and the crates that compile libFuzzer, see + [dependencies.md](dependencies.md)) never enters the main workspace. +- `fuzz/seeds/<target>/`: the hand-made seeds each corpus starts from + (its [README](../fuzz/seeds/README.md)). `transcript` also starts from + the vendored transcripts. +- `scripts/fuzz.sh`: builds the targets and runs libFuzzer over them, by + hand or under sai. It follows the C tree's `fuzz/run.sh`. + +## Running it by hand + +It needs nightly, cargo-fuzz, cargo-deny, a C++ compiler for libFuzzer's +runtime, and ideally `llvm-symbolizer` for readable reports: + +```sh +rustup toolchain install nightly --profile minimal +cargo install --locked cargo-fuzz cargo-deny +``` + +Then: + +```sh +scripts/fuzz.sh # 60s on every target +scripts/fuzz.sh 600 utf8 # 10 minutes on utf8 +CORPUS=~/npro-corpus scripts/fuzz.sh 3600 +``` + +It checks the fuzz workspace with `cargo deny` before building anything, +and builds with AddressSanitizer and debug assertions. `FUZZ_OPTS` passes +more flags to libFuzzer: `FUZZ_OPTS=-verbosity=0` leaves out its line for +every new unit and its periodic status, keeping the setup, the final +stats and any report. Builds and corpora +go under `target/fuzz/` (or `$CARGO_TARGET_DIR/fuzz/`) unless `BUILD` and +`CORPUS` say otherwise. + +### When it finds something + +A finding is an input that made a target panic: the code under test +panicked, or disagreed with the oracle. libFuzzer keeps it as +`target/fuzz/artifacts/crash-<sha1>`, and the run ends listing it. To +replay it under the fuzz build: + +```sh +target/fuzz/x86_64-unknown-linux-gnu/release/utf8 target/fuzz/artifacts/crash-... +``` + +The panic message says what disagreed, eg, +`utf8: fed a byte at a time it ends Complete, core says InvalidAt(1)`. +The same input reproduces it in a plain test too, which is where the fix's +regression test belongs: `npro_fuzz::Target::Utf8.run(&input)`. + +## Under sai + +The `fuzz` configuration in `.sai.json` runs `scripts/sai.sh fuzz 60` on +the C tree's fuzz builder, `fuzz-linux-debian13/x86_64-amd/gcc`, with +`"idle": 2` and `"pool": "fuzz"`. As in C's fuzz job, libFuzzer runs +there with `-verbosity=0`, so the log stays a few lines per target; a +`FUZZ_OPTS` set on the builder replaces it. [sai.md](sai.md) says what that builder +needs. sai gives the script: + +- **`SAI_POOL_DIR`**, the `fuzz` pool, which sai keeps synced between + every builder fuzzing npro. The corpora live there as + `corpus-<target>/`, named by SHA-1 as sai expects, so coverage keeps + advancing across jobs and builders. Pools belong to a repo: npro's + `fuzz` pool is not the C tree's. +- **`SAI_POOL_KNOWN`**, the reproducers of every bug found so far. The CI + run replays them before fuzzing, and fails while any still crashes. For + one that no longer does, it tells sai-server, which notes at which + commit. +- **`SAI_POOL_FINDINGS`**, where findings go: the input and libFuzzer's + report, which sai-server groups into bugs and shows **only to admins**. + A finding can be an unfixed security bug, and job logs are public, so + under sai the log gets only how each run went. +- **`SAI_IDLE_SECS`**, in an idle task: the slice's length, build + included. The time after the build is shared by as many targets as can + each have two minutes, taking turns across slices. An idle slice + reports what it finds, but does not fail. + +## Adding a target + +1. A harness in `crates/npro-fuzz/src/targets.rs`: what it drives, and + the oracle it checks against, reporting a disagreement with + `finding()`. It must not panic for any other reason. +2. A `Target` variant, its name, and its arm in `Target::run`. +3. A smoke test in `crates/npro-fuzz/tests/smoke.rs`, drawing inputs + that reach the code, which random bytes often do not. +4. Its seeds in `fuzz/seeds/<name>/`: for a protocol parser, C's corpus + for it, copied with where it came from. +5. A `[[bin]]` in `fuzz/Cargo.toml` and its one-line + `fuzz/fuzz_targets/<name>.rs`. + +`scripts/fuzz.sh` and sai pick it up from `cargo fuzz list`. A target +that needs a new crate is a dependency decision like any other +([dependencies.md](dependencies.md)). + +## Not done yet + +- **sai-server's grouping of Rust findings.** sai-server tells bugs + apart by the first three frames of the report's stack that are in the + code under test. It skips C's runtime and libFuzzer's frames, but not + Rust's: every panic's stack starts with `pthread_kill`, + `std::sys::pal::unix::abort_internal` and `std::process::abort`, so + every finding in a target is grouped as one bug. The frames that tell + findings apart come after `core::panicking::panic_fmt`. sai-server + needs to skip frames in `std::`, `core::`, `alloc::` (and `<std::`, + `<core::`, `<alloc::` impls), `libfuzzer_sys::`, `pthread_kill`, and + `npro_fuzz::targets::finding`, and to stop at `rust_fuzzer_test_input`. +- **Coverage reports** (`cargo fuzz coverage`) are not wired up. +- **Corpus minimizing** (`-merge=1`, and sai's pool replace) is not done; + the corpora only grow. diff --git a/docs/port-plan.md b/docs/port-plan.md index 2bc96b0..3422af1 100644 --- a/docs/port-plan.md +++ b/docs/port-plan.md @@ -457,8 +457,8 @@ Three commits: and cargo audit. - **`npro-test`**: a strict reader for `lws-transcript/1`, and copies of the C transcripts with `C-COMMIT` and the C README beside them. - `scripts/sync-c-oracle.sh` refreshes them from a C checkout. The fuzz - seeds join them when the first fuzz target does. + `scripts/sync-c-oracle.sh` refreshes them from a C checkout. C's fuzz + seeds join `fuzz/seeds/` with the targets that use them. **Exit check, met**: every gate passes at 1.85 and at stable, and the reader takes all 48 transcripts. @@ -487,6 +487,16 @@ parser gets the one-shot vs fragmented oracle. `indexing_slicing`, `arithmetic_side_effects` and `as_conversions` denied. +### Fuzzing (done, 2026-10-03) + +Ahead of the parsers, so that each is fuzzed from its first commit: +`crates/npro-fuzz` holds a harness per target with an oracle, smoke-tested +in `cargo test`; `fuzz/` holds the libFuzzer targets in a workspace of its +own; `scripts/fuzz.sh` runs them by hand and under sai, in CI and idle +time, with the corpora in a sai pool. The first targets are the +substrate's: `utf8`, `sha1`, `base64`, and the transcript reader. +[fuzzing.md](fuzzing.md) has the details. + ### Phase 1b: the connection machines - The four machines and the `WsiEvent` transition function, for the roles @@ -523,8 +533,8 @@ listed and agreed. bytes per body, as C does. - **Tests**: - the one-shot vs randomly fragmented oracle, as a property test; - - `cargo-fuzz` targets `h1-request`, `h1-response` and `chunked`, - seeded from `fuzz/fuzz-h1/seeds`; + - fuzz targets `h1-request`, `h1-response` and `chunked`, seeded from + C's `fuzz/fuzz-h1/seeds`, added as [fuzzing.md](fuzzing.md) says; - a differential test against C on the same inputs, comparing the parsed token table and the verdict. This needs a small C harness built from the reference tree outside this repository; it is @@ -629,7 +639,7 @@ not pending output; mux parked rx; the kept-warm joiner's status. | transcripts byte for byte | C `api-test-sansio` | `cargo test`, every commit | | state edge set vs C trace | C `LWS_WITH_STATE_TRACE` | `cargo test`, vendored edge file | | one-shot vs fragmented parse | agent-context "Parsers" | proptest, every parser | -| fuzz, seeded from C corpora | `fuzz/fuzz-*/seeds` | smoke over the seeds in CI; campaigns by hand | +| fuzz, with an oracle per target | C's `fuzz/fuzz-*/seeds`, copied into `fuzz/seeds/` | smoke tests in `cargo test` everywhere; libFuzzer in sai CI and idle time ([fuzzing.md](fuzzing.md)) | | differential parse vs C | C tree built outside the repo | optional CI job | | autobahn / h2spec / h3spec | conformance suites | per phase, once `npro-io` exists | | lints, docs, deny, audit, MSRV | AGENTS.md | CI, every commit | diff --git a/docs/sai.md b/docs/sai.md index 1240a6d..7f9af4b 100644 --- a/docs/sai.md +++ b/docs/sai.md @@ -22,6 +22,7 @@ unix platform has one step, `scripts/sai.sh ${cmake}`. Each configuration's | `miri-big-endian` | `miri s390x-unknown-linux-gnu` | fedora44 x86_64 | the tests pass on a big-endian machine, interpreted by Miri | | `miri-32bit` | `miri i686-unknown-linux-gnu` | fedora44 x86_64 | the tests pass where `usize` is 32 bits | | `c-oracle` | `oracle` | fedora44 x86_64 | C lws main-dev, built afresh, records the same transcripts npro holds | +| `fuzz` | `fuzz 60` | fuzz-debian13 x86_64 | no known bug still crashes, and a minute of libFuzzer on each fuzz target finds nothing new; two idle tasks carry on fuzzing in idle time, with the corpora in the `fuzz` pool ([fuzzing.md](fuzzing.md)) | How the C build's dimensions map here: - **The `cmake` option dimensions** become the `features` configuration. @@ -72,6 +73,7 @@ disk. | x86_64 and aarch64 Linux, riscv64, both macOS | `test` | rustup with stable (`--profile default`) | | fedora44 x86_64 | `gate`, `features`, `nostd`, `miri-*`, `c-oracle` | everything below | | freebsd/aarch64 | `test-freebsd` | `pkg install rust`, 1.85 or later; rustup has no FreeBSD aarch64 host | +| fuzz-debian13 x86_64 | `fuzz`, and its idle tasks | rustup with stable and nightly, cargo-fuzz, cargo-deny, a C++ compiler and llvm-symbolizer, and an `idle` object for the platform in the builder's configuration: see below | | w11/x86_64 | `test-windows` | rustup with stable for the MSVC host, and an `env` for the platform in the builder's configuration: see below | **Every builder with rustup**, as the `sai` user: @@ -106,6 +108,40 @@ Miri builds a standard library for each target it interprets. `miri setup` does it once ahead of time; otherwise every job on a fresh overlay spends several minutes on it. +**The fuzz builder**, the C tree's `fuzz-linux-debian13/x86_64-amd/gcc`, +which already has clang for C's fuzzing. As `sai`, after rustup: + +```sh +rustup toolchain install nightly --profile minimal # cargo-fuzz builds with nightly +cargo install --locked cargo-fuzz cargo-deny +``` + +and, as root, what libFuzzer's runtime is built with, and what turns the +sanitizer's addresses into function names: + +```sh +apt install g++ llvm # llvm provides llvm-symbolizer +``` + +`scripts/fuzz.sh` also finds a versioned `llvm-symbolizer-NN` if that is +all there is. Without one, reports have addresses but no function names, +and sai-server cannot tell one bug from another. + +For the idle tasks, the platform in the builder's configuration needs an +`idle` object (sai's `READMEs/README-idle.md`). It belongs to the +platform, so if C's fuzzing idle tasks already have one there, it serves +npro's too; otherwise, eg: + +```json +"idle": { + "share": 50, + "instances": 2, + "slice-secs": 900 +} +``` + +With 900 second slices, every target gets a turn in each slice. + **The Windows builder.** rustup's Windows installer is `rustup-init.exe`, from <https://rustup.rs>: <https://win.rustup.rs/x86_64> redirects to @@ -199,9 +235,12 @@ through with a cargo error. github.com. - `c-oracle` clones and fetches libwebsockets from libwebsockets.org, keeping its checkout in `$HOME/lws-oracle` (or `$LWS_ORACLE`). +- `fuzz` fetches the fuzz workspace's dependencies from crates.io, as + `fuzz/Cargo.lock` pins them (`cargo fetch --locked`), once per builder. + The pool is synced by sai-builder itself, not the job. -Nothing else fetches. The workspace has no dependencies, and every build -is `--locked`. +Nothing else fetches. The main workspace has no dependencies, and every +build is `--locked`. ### When a job fails on setup @@ -210,6 +249,7 @@ is `--locked`. | `rustc 1.75 is older than npro's rust-version 1.85`, or cargo's `` `resolver` setting `3` is not valid `` | the job found the distro's cargo: rustup is not installed for `sai` | install rustup as `sai`, as above | | `no such command: hack` (or `deny`, `audit`) | the cargo subcommand is not installed for `sai` | `cargo install --locked cargo-hack` (etc.) as `sai` | | `toolchain '1.85-…' is not installed` | the MSRV toolchain is missing | `rustup toolchain install 1.85 --profile minimal` as `sai` | +| `no such command: fuzz` | cargo-fuzz is not installed for `sai` | `cargo install --locked cargo-fuzz` as `sai` | | something installed earlier is missing again | it was installed inside a sai-virt overlay, not the base image | install it into the base image | | Miri spends minutes "preparing a sysroot" every job | Miri's std is rebuilt in each fresh overlay | `cargo +nightly miri setup --target …` in the base image | @@ -220,8 +260,6 @@ is `--locked`. board's silicon does not. rustup supports it as it is. - **esp32**, later: xtensa needs Espressif's Rust fork, installed with `espup`, once there is something to run on the device. -- **fuzzing**, later: cargo-fuzz needs nightly and clang, and joins the - fuzz builder with the h1 parser's fuzz targets. ## A possible sai change diff --git a/fuzz/Cargo.lock b/fuzz/Cargo.lock new file mode 100644 index 0000000..7da7fcb --- /dev/null +++ b/fuzz/Cargo.lock @@ -0,0 +1,106 @@ +# This file is automatically @generated by Cargo. +# It is not intended for manual editing. +version = 4 + +[[package]] +name = "arbitrary" +version = "1.4.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c3d036a3c4ab069c7b410a2ce876bd74808d2d0888a82667669f8e783a898bf1" + +[[package]] +name = "cc" +version = "1.6.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f74872d07caf508b30a21f6836e7d7016a2eaf7d9ff4f48deaa58cd8a0407630" +dependencies = [ + "find-msvc-tools", + "jobserver", + "libc", + "shlex", +] + +[[package]] +name = "cfg-if" +version = "1.0.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4e7648175b45a9a48536d676f68d918270699102aa8dab5496df06904c914600" + +[[package]] +name = "find-msvc-tools" +version = "0.1.14" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "aedcfb3409746eddb02b9e19ebda1c3394f759a152e48ee875a0844d1b955484" + +[[package]] +name = "getrandom" +version = "0.4.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "300e883d756b2e4ec94e02791f39b04b522276138852cfc41d9fb7e904106099" +dependencies = [ + "cfg-if", + "libc", + "r-efi", +] + +[[package]] +name = "jobserver" +version = "0.1.35" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1c00acbd29eabad4a2392fa0e921c874934dbbf4194312ad20f04a0ed67a3cb3" +dependencies = [ + "getrandom", + "libc", +] + +[[package]] +name = "libc" +version = "0.2.190" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ce5d3ddc6d3fa000eb1536d85e147bfe31aacaba692ed6a876f95cb7c855be78" + +[[package]] +name = "libfuzzer-sys" +version = "0.4.13" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a9fd2f41a1cba099f79a0b6b6c35656cf7c03351a7bae8ff0f28f25270f929d2" +dependencies = [ + "arbitrary", + "cc", +] + +[[package]] +name = "npro-core" +version = "0.0.2" + +[[package]] +name = "npro-fuzz" +version = "0.0.2" +dependencies = [ + "npro-core", + "npro-test", +] + +[[package]] +name = "npro-fuzz-targets" +version = "0.0.0" +dependencies = [ + "libfuzzer-sys", + "npro-fuzz", +] + +[[package]] +name = "npro-test" +version = "0.0.2" + +[[package]] +name = "r-efi" +version = "6.0.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f8dcc9c7d52a811697d2151c701e0d08956f92b0e24136cf4cf27b57a6a0d9bf" + +[[package]] +name = "shlex" +version = "2.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f8fadd59c855ef2080decdef8ff161eb6661b86933c9d82e5ba29dc602a55aba" diff --git a/fuzz/Cargo.toml b/fuzz/Cargo.toml new file mode 100644 index 0000000..c767bba --- /dev/null +++ b/fuzz/Cargo.toml @@ -0,0 +1,54 @@ +# The libFuzzer targets: one bin per npro_fuzz::Target, each a line handing +# libFuzzer's input to that target's harness in crates/npro-fuzz. +# +# A workspace of its own, with its own Cargo.lock and deny.toml, so that what +# fuzzing needs to build never enters the main workspace's dependency graph. +# Built by cargo-fuzz, with nightly: see docs/fuzzing.md. + +[package] +name = "npro-fuzz-targets" +version = "0.0.0" +publish = false +edition = "2024" +license = "MIT" + +[package.metadata] +cargo-fuzz = true + +[dependencies] +libfuzzer-sys = "=0.4.13" +npro-fuzz = { path = "../crates/npro-fuzz" } + +[[bin]] +name = "utf8" +path = "fuzz_targets/utf8.rs" +test = false +doc = false +bench = false + +[[bin]] +name = "sha1" +path = "fuzz_targets/sha1.rs" +test = false +doc = false +bench = false + +[[bin]] +name = "base64" +path = "fuzz_targets/base64.rs" +test = false +doc = false +bench = false + +[[bin]] +name = "transcript" +path = "fuzz_targets/transcript.rs" +test = false +doc = false +bench = false + +[lints.rust] +unsafe_code = "forbid" +unexpected_cfgs = "deny" + +[workspace] diff --git a/fuzz/deny.toml b/fuzz/deny.toml new file mode 100644 index 0000000..03379ae --- /dev/null +++ b/fuzz/deny.toml @@ -0,0 +1,77 @@ +# cargo deny for the fuzz workspace: what may enter what fuzzing builds. +# +# This is the same door as the main workspace's deny.toml, for a separate +# graph: these crates build the libFuzzer targets, and are never a dependency +# of anything npro ships. Each admitted crate has its entry in the fuzz +# section of docs/dependencies.md. scripts/fuzz.sh runs this check before it +# builds anything. +# +# cargo deny --manifest-path fuzz/Cargo.toml check --config fuzz/deny.toml + +[graph] +all-features = true +# fuzzing runs on Linux builders only; on Windows, jobserver would also bring +# getrandom, r-efi and cfg-if +targets = [ + "x86_64-unknown-linux-gnu", + "aarch64-unknown-linux-gnu", +] + +[advisories] +yanked = "deny" + +[licenses] +# NCSA: the LLVM licence of the libFuzzer sources libfuzzer-sys carries +allow = ["MIT", "NCSA"] +confidence-threshold = 0.9 + +[bans] +multiple-versions = "deny" +wildcards = "deny" +# path dependencies between the npro crates, none of them published +allow-wildcard-paths = true + +allow = [ + # the fuzz targets, and the npro crates they drive + "npro-fuzz-targets", + "npro-fuzz", + "npro-core", + "npro-test", + + # libFuzzer's runtime and the fuzz_target! macro + "libfuzzer-sys", + "arbitrary", + + # building libFuzzer's C++ sources, in libfuzzer-sys' build script + "cc", + "find-msvc-tools", + "jobserver", + "libc", + "shlex", +] + +[bans.build] +allow-build-scripts = [ + # compiles the libFuzzer runtime it carries, with cc + "libfuzzer-sys", + # probes the rustc version for cfgs + "libc", +] +executables = "deny" +interpreted = "deny" +include-dependencies = true +include-workspace = true +include-archives = true + +# arbitrary's package carries its maintainer's release script, which runs +# cargo publish and nothing else, and is never run by a build. Admitted by +# checksum, so a changed script is looked at again. +[[bans.build.bypass]] +crate = "arbitrary" +allow = [ + { path = "publish.sh", checksum = "752e221bdd960666b127df15effddd3d789ff3f1762498961fc79ae99f9a27f1" }, +] + +[sources] +unknown-registry = "deny" +unknown-git = "deny" diff --git a/fuzz/fuzz_targets/base64.rs b/fuzz/fuzz_targets/base64.rs new file mode 100644 index 0000000..038b22d --- /dev/null +++ b/fuzz/fuzz_targets/base64.rs @@ -0,0 +1,5 @@ +//! libFuzzer target for [`npro_fuzz::Target::Base64`]. + +#![no_main] + +libfuzzer_sys::fuzz_target!(|data: &[u8]| npro_fuzz::Target::Base64.run(data)); diff --git a/fuzz/fuzz_targets/sha1.rs b/fuzz/fuzz_targets/sha1.rs new file mode 100644 index 0000000..6b0ef15 --- /dev/null +++ b/fuzz/fuzz_targets/sha1.rs @@ -0,0 +1,5 @@ +//! libFuzzer target for [`npro_fuzz::Target::Sha1`]. + +#![no_main] + +libfuzzer_sys::fuzz_target!(|data: &[u8]| npro_fuzz::Target::Sha1.run(data)); diff --git a/fuzz/fuzz_targets/transcript.rs b/fuzz/fuzz_targets/transcript.rs new file mode 100644 index 0000000..aa7b669 --- /dev/null +++ b/fuzz/fuzz_targets/transcript.rs @@ -0,0 +1,5 @@ +//! libFuzzer target for [`npro_fuzz::Target::Transcript`]. + +#![no_main] + +libfuzzer_sys::fuzz_target!(|data: &[u8]| npro_fuzz::Target::Transcript.run(data)); diff --git a/fuzz/fuzz_targets/utf8.rs b/fuzz/fuzz_targets/utf8.rs new file mode 100644 index 0000000..c5f9f44 --- /dev/null +++ b/fuzz/fuzz_targets/utf8.rs @@ -0,0 +1,5 @@ +//! libFuzzer target for [`npro_fuzz::Target::Utf8`]. + +#![no_main] + +libfuzzer_sys::fuzz_target!(|data: &[u8]| npro_fuzz::Target::Utf8.run(data)); diff --git a/fuzz/seeds/README.md b/fuzz/seeds/README.md index a7fdc31..0604bac 100644 --- a/fuzz/seeds/README.md +++ b/fuzz/seeds/README.md @@ -14,6 +14,6 @@ The `transcript` target also starts from every transcript in Seeds are small and readable on purpose: each is a case worth starting from, named for what it is. What the fuzzer finds goes in its corpus, which is -kept between runs, not here. When the protocol crates arrive, their +kept between runs, not here ([docs/fuzzing.md](../../docs/fuzzing.md)). When the protocol crates arrive, their targets' seeds come from the C library's corpora (`fuzz/fuzz-*/seeds` in the C tree), copied with where they came from. diff --git a/scripts/fuzz.sh b/scripts/fuzz.sh new file mode 100755 index 0000000..b2efc1f --- /dev/null +++ b/scripts/fuzz.sh @@ -0,0 +1,305 @@ +#!/bin/sh +# +# Coverage-guided fuzzing of npro's fuzz targets with libFuzzer, by hand or +# under sai. docs/fuzzing.md says what the targets check, and how to read +# and replay what this finds. +# +# scripts/fuzz.sh # 60s per target, every target +# scripts/fuzz.sh 600 # 10 minutes per target +# scripts/fuzz.sh 600 utf8 sha1 # only the named targets +# BUILD=~/npro-fuzz scripts/fuzz.sh # build somewhere else +# CORPUS=~/corpus scripts/fuzz.sh # keep the corpora somewhere lasting +# +# The targets are the bins of the fuzz/ workspace, built by cargo-fuzz with +# nightly, AddressSanitizer and debug assertions. FUZZ_OPTS adds libFuzzer +# flags, eg, FUZZ_OPTS=-verbosity=0 to leave out its line for every new +# unit and its periodic status, as scripts/sai.sh does for sai's logs. +# +# Each target's corpus is <corpus>/corpus-<target>, grown from its seeds in +# fuzz/seeds/<target>/ (and, for transcript, the vendored transcripts). +# <corpus> is <build>/corpus unless CORPUS says otherwise. +# +# This follows the C tree's fuzz/run.sh, in how it works with sai: +# +# - Under a sai idle task (sai's READMEs/README-idle.md), SAI_IDLE_SECS is +# the length of the slice, build included. The time left after the build +# is shared by as many targets as can each have IDLE_MIN_TARGET_SECS, +# taking turns across slices. +# +# - Under a configuration naming a pool (README-pool.md), SAI_POOL_DIR is +# kept synced with every builder fuzzing npro, and the corpora live there. +# The first time, any corpora CORPUS already held are copied in. +# +# - With the pool come SAI_POOL_KNOWN, the reproducers of the bugs found so +# far, and SAI_POOL_FINDINGS, where findings go to sai-server +# (README-findings.md). A CI run (not an idle slice) first replays each +# target's known reproducers and fails if any still crash. Reports go +# only to sai-server, which shows them only to admins: they can be +# unfixed security bugs, and CI logs are public. An idle slice reports +# its findings that way, but does not fail. +# +# Findings (crash-*, leak-*, timeout-*, oom-*, slow-unit-*) are written into +# <build>/artifacts/ with each target's output in log-<target>.txt; those +# this run produced are listed at the end, and make it exit nonzero. + +set -eu + +REPO=$(CDPATH= cd -- "$(dirname -- "$0")/.." && pwd) +cd "$REPO" + +# rustup's tools, for the user the builder runs as +PATH="$HOME/.cargo/bin:$PATH" +export PATH + +. scripts/require.sh +require_rust +require_toolchain nightly "rustup toolchain install nightly --profile minimal" +require_cargo fuzz "cargo install --locked cargo-fuzz" +require_cargo deny "cargo install --locked cargo-deny" +require_cmd c++ "dnf install gcc-c++ (or apt install g++): libFuzzer's runtime is C++" +require_done + +BUILD="${BUILD:-${CARGO_TARGET_DIR:-$REPO/target}/fuzz}" +CORPUS="${CORPUS:-$BUILD/corpus}" +ARTIFACTS="$BUILD/artifacts" + +if [ -n "${SAI_POOL_DIR:-}" ] && [ -d "$SAI_POOL_DIR" ]; then + if [ "$CORPUS" != "$SAI_POOL_DIR" ] && [ -d "$CORPUS" ] && + [ ! -e "$CORPUS/.sai-pool-copied" ]; then + for d in "$CORPUS"/corpus-*; do + if [ -d "$d" ]; then + mkdir -p "$SAI_POOL_DIR/${d##*/}" + cp -Rn "$d/." "$SAI_POOL_DIR/${d##*/}/" + fi + done + touch "$CORPUS/.sai-pool-copied" + fi + CORPUS="$SAI_POOL_DIR" +fi + +SECS="${1:-60}" +case "$SECS" in + ''|*[!0-9]*) echo "usage: $0 [seconds per target] [target...]" >&2; exit 1 ;; +esac +if [ "$#" -gt 0 ]; then + shift +fi + +START=$(date +%s) +IDLE_SECS="${SAI_IDLE_SECS:-}" +case "$IDLE_SECS" in + ''|*[!0-9]*) IDLE_SECS="" ;; +esac +# least time a target gets in an idle slice, since each run begins by +# replaying its whole corpus +IDLE_MIN_TARGET_SECS=120 +# time an idle slice keeps back for runs starting and stopping, and reporting +IDLE_MARGIN_SECS=30 +IDLE_PER_TARGET_OVERHEAD_SECS=5 + +# the fuzz workspace's dependencies are let in only by name, like the main +# workspace's (docs/dependencies.md): check before building anything +echo "== cargo deny, fuzz workspace" +cargo deny --manifest-path fuzz/Cargo.toml check +# and build only what fuzz/Cargo.lock says +cargo +nightly fetch --locked --manifest-path fuzz/Cargo.toml + +echo "== build" +cargo +nightly fuzz build --fuzz-dir fuzz -O --debug-assertions \ + --target-dir "$BUILD" +host=$(rustc +nightly -vV | sed -n 's/^host: //p') +BIN="$BUILD/$host/release" + +ALL=$(cargo +nightly fuzz list --fuzz-dir fuzz) +if [ "$#" -gt 0 ]; then + TARGETS="$*" + for t in $TARGETS; do + case " $(echo $ALL) " in + *" $t "*) ;; + *) echo "no fuzz target $t; there are: $(echo $ALL)" >&2; exit 1 ;; + esac + done +else + TARGETS=$(echo $ALL) +fi + +# the sanitizer reports need function names, for people and for sai-server's +# grouping of findings into bugs +if [ -z "${ASAN_SYMBOLIZER_PATH:-}" ]; then + for s in llvm-symbolizer llvm-symbolizer-21 llvm-symbolizer-20 \ + llvm-symbolizer-19 llvm-symbolizer-18; do + if command -v "$s" >/dev/null 2>&1; then + ASAN_SYMBOLIZER_PATH=$(command -v "$s") + export ASAN_SYMBOLIZER_PATH + break + fi + done +fi + +mkdir -p "$ARTIFACTS" "$CORPUS" + +if [ -n "$IDLE_SECS" ]; then + set -- $TARGETS + n=$# + + left=$(( IDLE_SECS - ($(date +%s) - START) - IDLE_MARGIN_SECS )) + count=$(( left / (IDLE_MIN_TARGET_SECS + IDLE_PER_TARGET_OVERHEAD_SECS) )) + if [ "$count" -gt "$n" ]; then + count=$n + fi + if [ "$count" -lt 1 ]; then + echo "idle slice of ${IDLE_SECS}s has no time left after the build" + exit 0 + fi + SECS=$(( left / count - IDLE_PER_TARGET_OVERHEAD_SECS )) + + # whose turn it is, kept with the corpora so it lasts between slices + next=0 + if [ -r "$CORPUS/.idle-next" ]; then + read -r next < "$CORPUS/.idle-next" || next=0 + case "$next" in + ''|*[!0-9]*) next=0 ;; + esac + fi + next=$(( next % n )) + echo $(( (next + count) % n )) > "$CORPUS/.idle-next" + + TARGETS="" + i=0 + while [ "$i" -lt "$count" ]; do + k=$(( (next + i) % n + 1 )) + eval "TARGETS=\"\$TARGETS \${$k}\"" + i=$(( i + 1 )) + done + + echo "idle slice of ${IDLE_SECS}s: ${SECS}s each for$TARGETS" +fi + +# the seed directories of target $1 +seeds() { + echo "$REPO/fuzz/seeds/$1" + if [ "$1" = transcript ]; then + echo "$REPO/crates/npro-test/transcripts" + fi +} + +# so this run's findings can be told from earlier ones in $ARTIFACTS +STAMP="$ARTIFACTS/.run-stamp" +touch "$STAMP" + +rc=0 + +# send a finding to sai-server through the pool: $1 target, $2 the input, +# $3 the name to give it, $4 the report. Each is written under a hidden name +# first, so the builder never sends one half written. +sai_finding() { + d="$SAI_POOL_FINDINGS/$1" + mkdir -p "$d" + tail -c 1048576 "$4" > "$d/.$3.log" && mv "$d/.$3.log" "$d/$3.log" + cp "$2" "$d/.$3" && mv "$d/.$3" "$d/$3" +} + +# replay the known reproducers of target $1, failing for any that still +# crash, and telling sai-server about those that no longer do +replay_known() { + kdir="$SAI_POOL_KNOWN/$1" + [ -d "$kdir" ] || return 0 + + for f in "$kdir"/*; do + h=${f##*/} + case "$h" in + *[!0-9a-f]*|'') continue ;; + esac + [ ${#h} -eq 40 ] || continue + + rlog="$ARTIFACTS/replay-$1-$h.txt" + if "$BIN/$1" -timeout=60 "$f" > "$rlog" 2>&1; then + mkdir -p "$SAI_POOL_FINDINGS/$1" + : > "$SAI_POOL_FINDINGS/$1/.ok-$h" && + mv "$SAI_POOL_FINDINGS/$1/.ok-$h" \ + "$SAI_POOL_FINDINGS/$1/ok-$h" + else + echo "$1: known bug $h still crashes (report sent to sai)" + sai_finding "$1" "$f" "replay-$h" "$rlog" + rc=1 + fi + done +} + +# the first corpus dir receives new discoveries; the seed dirs are only read +run_target() { + # shellcheck disable=SC2046,SC2086 + "$BIN/$1" "$CORPUS/corpus-$1" $(seeds "$1") \ + -max_total_time="$SECS" \ + -print_final_stats=1 \ + -artifact_prefix="$ARTIFACTS/" \ + ${FUZZ_OPTS:-} +} + +for t in $TARGETS; do + echo + echo "=== $t: ${SECS}s ===" + mkdir -p "$CORPUS/corpus-$t" + log="$ARTIFACTS/log-$t.txt" + + if [ -n "${SAI_POOL_KNOWN:-}" ] && [ -n "${SAI_POOL_FINDINGS:-}" ] && + [ -z "$IDLE_SECS" ]; then + replay_known "$t" + fi + + TSTAMP="$ARTIFACTS/.target-stamp" + touch "$TSTAMP" + + if [ -t 1 ]; then + # interactive: live output, and a copy next to the artifacts + { run_target "$t"; echo $? > "$log.rc"; } 2>&1 | tee "$log" + [ "$(cat "$log.rc")" = 0 ] || rc=1 + rm -f "$log.rc" + else + # not a terminal, eg, a sai log: libFuzzer writes each status line + # in many small writes, which a log collector counting chunks + # charges for one by one; gather the output and emit it at once + run_target "$t" > "$log" 2>&1 || rc=1 + + if [ -n "${SAI_POOL_FINDINGS:-}" ]; then + # the reports go to sai-server; the public log only gets + # how it went + grep -aE '^(#[0-9]+[[:space:]]+(INITED|DONE)|Done [0-9]+ runs|stat::)' \ + "$log" || true + else + cat "$log" + fi + fi + + if [ -n "${SAI_POOL_FINDINGS:-}" ]; then + for f in $(find "$ARTIFACTS" -maxdepth 1 -type f \ + -newer "$TSTAMP" \( -name 'crash-*' \ + -o -name 'leak-*' -o -name 'timeout-*' \ + -o -name 'oom-*' -o -name 'slow-unit-*' \)); do + echo "$t: finding ${f##*/} (report sent to sai)" + sai_finding "$t" "$f" "${f##*/}" "$log" + done + fi +done + +# what this run found, by absolute path, so it can be collected from the log +# even when the run was somewhere else +FOUND=$(find "$ARTIFACTS" -maxdepth 1 -type f -newer "$STAMP" \ + \( -name 'crash-*' -o -name 'leak-*' -o -name 'timeout-*' \ + -o -name 'oom-*' -o -name 'slow-unit-*' \) | sort) + +echo +if [ -n "$FOUND" ]; then + echo "=== FINDINGS: replay each with $BIN/<target> <file> ===" + echo "$FOUND" + rc=1 +else + echo "=== no findings ===" +fi + +if [ -n "$IDLE_SECS" ] && [ -n "${SAI_POOL_FINDINGS:-}" ]; then + # an idle slice has reported what it found; it carries on next slice + rc=0 +fi + +exit $rc diff --git a/scripts/sai.sh b/scripts/sai.sh index 635de7c..40e4ecb 100755 --- a/scripts/sai.sh +++ b/scripts/sai.sh @@ -13,6 +13,8 @@ # big-endian machine without needing one # oracle C lws main-dev's transcripts, recorded afresh, compared # with the copies in crates/npro-test/transcripts +# fuzz [secs] libFuzzer over every fuzz target (scripts/fuzz.sh), in +# CI and in sai's idle time, with the corpora in a pool # # Each profile first checks for the tools it uses and lists any that are # missing with their install commands; docs/sai.md says what each builder @@ -132,8 +134,18 @@ oracle) exit 1 ;; +fuzz) + # libFuzzer's line for every new unit and its periodic status would + # swamp the job log, as in C's fuzz job: keep its start, the final + # stats and any report. FUZZ_OPTS set on the builder still wins. + FUZZ_OPTS="${FUZZ_OPTS:--verbosity=0}" + export FUZZ_OPTS + # fuzz.sh checks its own tools + exec scripts/fuzz.sh "$@" + ;; + *) - echo "usage: $0 gate | test | features | nostd | miri <target> | oracle" >&2 + echo "usage: $0 gate | test | features | nostd | miri <target> | oracle | fuzz [secs]" >&2 exit 1 ;; esac
Page fetched 0s ago, creation time: 5ms (vhost etag hits: 0%, cache hits: 0%)