Project homepage Mailing List  Warmcat.com  API Docs  Github Mirror 
    npro  
 Modern all-safe Rust Network Protocol library supporting h1, h2, h3, ws, wt sans-IO and with socket IO + tls
git clone https://npro.rs/repo/npro
 
root / crates / npro-test / h1 / requests / uri-absolute.http
Author[]Andy Green <andy@warmcat.com> 2026-10-06 04:19 UTC
Committer[]Andy Green <andy@warmcat.com> 2026-10-06 05:30 UTC
Treecf21d41664466bed365038030dc322693ed47708   Raw Patch
 
scripts: autobahn, in upstream's Docker image, and its builder
scripts: autobahn, in upstream's Docker image, and its builder

scripts/autobahn.sh runs Autobahn|Testsuite's wstest, which is Python 2
only, in upstream's crossbario/autobahn-testsuite image, pinned by digest
once the builder has pulled it: as fuzzingclient against an echo server
started on 9001, or as fuzzingserver against an echo client run once a
case, as C's scripts/autobahn-test-*.sh do by hand.  It writes the
configuration itself, with C's exclusions and why (the 12.x ones' reason
is still to be found), and fails unless every case is OK, NON-STRICT or
INFORMATIONAL, as C judges them.  It runs against C's minimal echo
examples now, for a baseline, and against npro-io's once phase 1g has
them.

docs/sai.md says how to set up the Debian 13 VM with Docker it runs on,
and no longer says the main workspace fetches nothing: since
permessage-deflate, cargo fetches miniz_oxide and adler2.

Not yet run: this was written where Docker is not available, so its first
run is the builder's.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019kg5Eemy68ZaqDBcUJQG6J
diff --git a/docs/sai.md b/docs/sai.md index 5af949b..0a2b9a4 100644 --- a/docs/sai.md +++ b/docs/sai.md @@ -239,8 +239,9 @@ through with a cargo error. `fuzz/Cargo.lock` pins them (`cargo fetch --locked`), once per builder. The pool is synced by sai-builder itself, not the job. -Nothing else fetches. The main workspace has no dependencies, and every -build is `--locked`. +Nothing else fetches but cargo itself: the main workspace's only +dependencies, `miniz_oxide` and `adler2` for npro-ws's `pmd` feature, come +from crates.io as `Cargo.lock` pins them, and every build is `--locked`. ### When a job fails on setup @@ -261,6 +262,57 @@ build is `--locked`. - **esp32**, later: xtensa needs Espressif's Rust fork, installed with `espup`, once there is something to run on the device. +## The autobahn builder + +Autobahn|Testsuite is the ws conformance gate (phase 1g of +[port-plan.md](port-plan.md)). Its `wstest` is Python 2 only, so it runs +in upstream's Docker image, `crossbario/autobahn-testsuite`, on a builder +of its own: a Debian 13 VM with Docker, like the one qir uses. +`scripts/autobahn.sh` runs it, both ways, against any echo server or +client. + +Setting one up: + +1. **The VM.** Debian 13 (trixie), x86_64; 2 vCPUs, 4GiB and 20GiB of + disk are plenty. Make it as you make the other builders, with the + user sai runs jobs as. If it is a sai-virt image, everything below goes + into the base image, not an overlay. +2. **Docker.** Debian's own package is enough: + + apt install docker.io + systemctl enable --now docker + usermod -aG docker sai # the user jobs run as + + Membership of `docker` is as good as root on that VM, which is a reason + for it to be a VM of its own. Rootless Podman, with the + `podman-docker` package giving a `docker` command, is the alternative + if that matters. +3. **The image, pinned.** As that user: + + docker pull crossbario/autobahn-testsuite + docker inspect --format '{{index .RepoDigests 0}}' crossbario/autobahn-testsuite + + The second prints `crossbario/autobahn-testsuite@sha256:…`; that goes + in `scripts/autobahn.sh` as its `image`, in a commit, so every run uses + exactly that image. Once it is pulled, jobs need no network for it. +4. **Rust,** as for every builder (above): rustup as that user. +5. **A first run against C,** to know the builder works before npro has + anything to test. As that user, in a libwebsockets checkout: + + apt install build-essential cmake git libssl-dev + mkdir build && cd build + cmake .. -DLWS_WITH_MINIMAL_EXAMPLES=1 && make -j4 + /path/to/npro/scripts/autobahn.sh server bin/lws-minimal-ws-server-echo -p 9001 + /path/to/npro/scripts/autobahn.sh client bin/lws-minimal-ws-client-echo -s 127.0.0.1 -p 9001 -u + + Each says how many cases ran and how many failed, and leaves the + reports in `./autobahn/reports`, `index.html` among them. Whatever C + fails is worth knowing in itself, and is the baseline npro is held to. +6. **Into sai,** as the other builders are, with a platform name such as + `linux-debian13/x86_64-amd/gcc`. The `.sai.json` configuration that + runs autobahn there comes with npro-io's echo server and client, in + phase 1g: until then there is nothing of npro's to run. + ## A possible sai change Nothing here needs sai changed. A neutral name for the configuration diff --git a/scripts/autobahn.sh b/scripts/autobahn.sh new file mode 100755 index 0000000..1d433e9 --- /dev/null +++ b/scripts/autobahn.sh @@ -0,0 +1,137 @@ +#!/usr/bin/env bash +# +# Autobahn|Testsuite against a ws echo server or client, both ways, with +# upstream's Docker image: wstest is Python 2 only, and the image is how +# upstream runs it. docs/sai.md says how to set up the builder it runs +# on; docs/port-plan.md (phase 1g) why it is a gate. +# +# scripts/autobahn.sh server <command starting an echo server on 9001> +# scripts/autobahn.sh client <command running one echo client connection> +# +# In "server" mode, the command is started in the background, and wstest's +# fuzzingclient runs every case against ws://127.0.0.1:9001. In "client" +# mode, wstest's fuzzingserver listens on 9001, and the command is run +# once a case, with the path to ask for appended as its last argument +# (/runCase?case=N&agent=AGENT), until it fails, the cases being over; then +# once with /updateReports?agent=AGENT. The echo examples of C, for a +# baseline, built with -DLWS_WITH_MINIMAL_EXAMPLES=1: +# +# scripts/autobahn.sh server bin/lws-minimal-ws-server-echo -p 9001 +# scripts/autobahn.sh client bin/lws-minimal-ws-client-echo \ +# -s 127.0.0.1 -p 9001 -u +# +# Reports go to ./autobahn/reports/{servers,clients}, index.html among +# them. The exit code is 0 only if every case not excluded below is OK, +# NON-STRICT or INFORMATIONAL, as C's scripts judge it. +# +# AUTOBAHN_IMAGE overrides the image; AUTOBAHN_AGENT the agent name. + +set -euo pipefail + +# Upstream's image, pinned: a digest, not a tag, so what runs is what was +# checked. Pin it on the builder's first run: docker pull +# crossbario/autobahn-testsuite, then docker inspect --format +# '{{index .RepoDigests 0}}' crossbario/autobahn-testsuite, and put that +# here. +image="${AUTOBAHN_IMAGE:-crossbario/autobahn-testsuite@sha256:PIN-ME}" +agent="${AUTOBAHN_AGENT:-npro}" +port=9001 + +# Cases excluded, and why. C's, until npro's own reasons replace them: +# 2.10, 2.11: several pings in flight; RFC 6455 does not require it, and +# C and npro keep one pending pong. +# 12.3.1, 12.3.2, 12.4.*, 12.5.*: excluded by C's client script for +# permessage-deflate with its client; the reason is not +# written down there, and is to be found before npro keeps +# them out. +exclude_server='"2.10", "2.11"' +exclude_client='"2.10", "2.11", "12.3.1", "12.3.2", "12.4.*", "12.5.*"' + +mode="${1:?usage: $0 server|client <command...>}" +shift +[ $# -gt 0 ] || { echo "$0: no command given" >&2; exit 2; } + +case "$image" in +*PIN-ME*) + echo "$0: pin the image's digest first: see the top of $0" >&2 + exit 2 + ;; +esac + +work="$PWD/autobahn" +mkdir -p "$work/config" "$work/reports" + +# wstest in the container, on the host's network so 127.0.0.1 is shared, +# as this user so the reports are ours +name="npro-autobahn-$$" +wstest() { + docker run --rm --name "$name" --network host --user "$(id -u):$(id -g)" \ + -v "$work/config:/config:ro" -v "$work/reports:/reports" \ + "$image" wstest "$@" +} + +# the cases that are not OK, NON-STRICT or INFORMATIONAL, from an index; +# "behavior": exactly, not "behaviorClose": +failures() { + grep '"behavior":' "$1" | grep -v -e '"OK"' -e '"NON-STRICT"' \ + -e '"INFORMATIONAL"' || true +} + +judge() { + local index="$1" what="$2" total bad + [ -s "$index" ] || { echo "$what: no report" >&2; return 1; } + total=$(grep -c '"behavior":' "$index" || true) + bad=$(failures "$index" | wc -l) + echo "$what: $total cases, $bad failed" + [ "$bad" -eq 0 ] +} + +case "$mode" in +server) + cat >"$work/config/fuzzingclient.json" <<EOF +{ + "outdir": "/reports/servers", + "servers": [ { "agent": "$agent", "url": "ws://127.0.0.1:$port" } ], + "cases": [ "*" ], + "exclude-cases": [ $exclude_server ], + "exclude-agent-cases": {} +} +EOF + "$@" & + server=$! + trap 'kill $server 2>/dev/null || true' EXIT + sleep 1 + kill -0 "$server" || { echo "$0: the server did not start" >&2; exit 3; } + wstest -m fuzzingclient -s /config/fuzzingclient.json + judge "$work/reports/servers/index.json" "autobahn's client, $agent's server" + ;; +client) + cat >"$work/config/fuzzingserver.json" <<EOF +{ + "url": "ws://127.0.0.1:$port", + "outdir": "/reports/clients", + "cases": [ "*" ], + "exclude-cases": [ $exclude_client ], + "exclude-agent-cases": {} +} +EOF + wstest -m fuzzingserver -s /config/fuzzingserver.json & + fuzzer=$! + # stopping the docker client need not stop its container + trap 'docker rm -f "$name" >/dev/null 2>&1 || true' EXIT + sleep 3 + kill -0 "$fuzzer" || { echo "$0: wstest did not start" >&2; exit 3; } + n=1 + while "$@" "/runCase?case=$n&agent=$agent"; do + n=$((n + 1)) + done + "$@" "/updateReports?agent=$agent" || true + sleep 2 + docker rm -f "$name" >/dev/null 2>&1 || true + judge "$work/reports/clients/index.json" "autobahn's server, $agent's client" + ;; +*) + echo "usage: $0 server|client <command...>" >&2 + exit 2 + ;; +esac
Page fetched 0s ago, creation time: 2ms (vhost etag hits: 0%, cache hits: 0%)