Project homepage Mailing List  Warmcat.com  API Docs  Github Mirror 
    npro  
 Modern all-safe Rust Network Protocol library supporting h1, h2, h3, ws, wt sans-IO and with socket IO + tls
git clone https://npro.rs/repo/npro
 
main main-devroot / crates / npro-test / h1 / requests / headers-pseudo.http
Author[]Andy Green <andy@warmcat.com> 2026-10-08 08:41 UTC
Committer[]Andy Green <andy@warmcat.com> 2026-10-08 08:58 UTC
Tree651eede341eb2b35ac6237bfb0a653e6a3f860e8   Raw Patch
 
docs: Secure Streams policy, a first draft main main-dev
docs: Secure Streams policy, a first draft

The SS policy becomes npro's one configuration language, taking over
what lwsws' lejp-conf does for vhosts, listeners, protocols and mounts,
as discussed with Andy on 2026-10-07 and 08.  It is a draft to iterate
on: what was decided is dated, what is open is listed.

Configuration binds declared parameters only, npro's own included.
Layers (a service's structure, its deployment, the front, and the
machine-local part) merge by namespace, not by file order.  The front
resolves them, and serves each instance a signed fragment for its
namespace, which the instance fetches when it starts and caches.  The
C SS policy's sections and every group of lejp-conf's keys are mapped
to where they land.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
diff --git a/docs/ss-policy.md b/docs/ss-policy.md new file mode 100644 index 0000000..41a7df3 --- /dev/null +++ b/docs/ss-policy.md @@ -0,0 +1,366 @@ +# Secure Streams policy + +**Draft, for iteration.** It records what was decided with Andy on +2026-10-07 and 2026-10-08, and marks what is still open. The stream and +service API it configures is a separate document, still to be written +(`ss-design.md`); [io-model.md](io-model.md) is the layer underneath. + +## What this replaces + +C lws has two configuration languages that grew apart: + +- **lwsws' lejp-conf** (`lib/io/lejp-conf.c`). It composes a server: + vhosts with listeners and tls, protocols enabled per vhost with their + options (pvo), and mounts that place them in the vhost's URL space + (`file://`, `cgi://`, proxy, redirect, `callback://`). +- **The Secure Streams policy** (`lib/secure-streams/policy-json.c`). It + describes streamtypes: endpoints, protocol, tls trust, retry, metadata. + Its server side is much smaller: a streamtype with `"server": true` is + one listener and one handler for every path, which is what lejp-conf + would call a single protocol plugin. + +npro has one: the SS policy, grown to do lejp-conf's job as well. SS is +npro's user API for client and server, and its policy is the only +configuration language. + +## Principles + +- **The policy is Rust types.** JSON (parsed by `npro-json`) is one way + to build them; Rust code is another. A `const` policy in code needs no + code generator, so C's `policy2c` and static-policy builds have no + equivalent: an embedded build simply has no JSON. +- **Most programs need no policy at all.** A stream made from a URL + derives what C's built-in `__default` streamtype gives: protocol and tls + from the scheme, the default port, the system's trust store, redirects + followed, headers read and written by name. A policy carries + deployment decisions (endpoints, trust, certs, retry, listeners, + routes) for when someone other than the programmer should make them. +- **Parameters only.** Configuration binds declared parameters; nothing + else can be changed from outside. +- **Merging is by namespace, never by file order.** There is no + equivalent of `conf.d`'s alphabetical loading. +- **Nothing is accepted and ignored.** An unknown key, a parameter + bound to the wrong type or out of its bounds, or a required parameter + left unbound, fails resolution, and the error names its path, for + example `vhosts."warmcat.com".routes[2].to: no endpoint + "mirror.wss"`. +- **npro is not an orchestrator.** It defines the policy, resolves it + and serves it to the processes that need it. Starting, restarting and + placing processes is systemd's job, or whatever the deployment uses. + +## Parameters + +A **parameter** is a declared, typed setting: its name, its type, its +default or that it is required, its bounds, and one line of +documentation. Configuration does nothing but bind parameters. + +Parameters are declared by: + +- **npro itself**, under the top-level `npro` namespace. Every + configuration struct in the core is declared this way: h1 header + limits, the ws rx buffer and the permessage-deflate message cap, + keepalive and other timeouts, tls ciphers, retry defaults. Bounds come + from C's ranges. Names are not settled; for illustration, + `npro.h1.header-bytes`, `npro.ws.pmd.max-message`, + `npro.tls.ciphers`. +- **each service**, in its structure (below): its options, and the bind + address of each of its named listeners, which is always a parameter. + +The declarations are written once in Rust, in a `macro_rules!` table +beside the configuration struct, with no serde and no proc-macros, and +`no_std` is unaffected. Two things come from them: + +- the decoder from the policy into the struct, so a declared parameter + can never be silently ignored; +- **the reference documentation of every parameter**, generated, so it + cannot drift from the code as lejp-conf's README has. + +### Scopes + +A parameter is bound at a scope: the whole process, a listener, a vhost, +a route, or an instance. The rules: + +- binding the same parameter twice in one scope is an error; +- a nested scope refines its parent for everything inside it: a route + inside a vhost, a vhost inside a listener, an instance inside its + deployment. + +Precedence is decided by the structure, never by which file was read +last. *(Open: agreed in discussion, not yet confirmed.)* + +### Exports and references + +Some facts belong to one layer and are needed by another: the front's +public host name, which a webrtc instance announces in its URLs; its +external address, which ICE needs. The owner **exports** them by name, +and a binding elsewhere **refers** to them: + +```json +"public-host": "${front.public-host}" +``` + +The dependency is then visible where it is used, and no layer reaches +into another's namespace. An unknown reference fails resolution. This +is the only substitution there is: lejp-conf's `"=NAME"` / `${NAME}` +preprocessor is not carried over. + +## The layers + +Taking a webrtc service as the example, which runs as its own process +and has both ws and udp listeners: + +| layer | written by | changes when | says | +|---|---|---|---| +| **structure** | the service's developer; it ships with the crate | the code changes | its named listeners and what each is ("signalling": ws, "media": datagram), with **no addresses**; its parameters, with their types, defaults and bounds; the client streamtypes it uses | +| **deployment** | whoever runs it | instances or machines change | the instances (`webrtc.eu1`, `webrtc.eu2`), and for each, the bind address of each named listener and its parameter values | +| **front** | whoever runs the public face | public names or certs change | listeners, tls, vhosts, routes, guards, the exports, and `npro.*` parameters at any scope | +| **local** | whoever runs the machine | the machine changes | the process's identity, where its authority is, the key that checks what the authority sends, and the process's own secrets | + +The deployment and front layers are read by the front; how the operator +splits them into files is up to the operator, since they merge by +namespace. The structure comes from the service: from its crate, for a +service linked into the front, or from the instance itself, for one in +another process (see "The handshake"). The local layer stays on its +machine and never travels. + +The instances are the namespace: `webrtc.eu1.media.bind`, +`webrtc.eu1.max-rooms`. A shorthand for "n instances, ports from a +base" may come later; instances are listed explicitly for now. + +### An example + +The structure, as `npro-webrtc` declares it (shown as JSON, which is +also how an instance sends it): + +```json +{ + "component": "npro-webrtc", "version": "0.3.0", + "listeners": { + "signalling": { "kind": "ws", "subprotocol": "lws-webrtc" }, + "media": { "kind": "datagram" } + }, + "params": { + "max-rooms": { "type": "u32", "default": 64, "min": 1, "max": 4096 }, + "public-host": { "type": "host", "required": true } + } +} +``` + +The deployment: + +```json +{ + "instances": { + "webrtc.eu1": { + "component": "npro-webrtc", + "bind": { "signalling": "unix:/run/npro/webrtc-eu1.sock", + "media": "[::]:3478" }, + "params": { "max-rooms": 128, "public-host": "${front.public-host}" } + } + } +} +``` + +The front: + +```json +{ + "exports": { "public-host": "rtc.example.com" }, + "listeners": { + "https": { "bind": "[::]:443", "tls": { "cert": "rtc" }, + "vhosts": ["rtc.example.com"] } + }, + "vhosts": { + "rtc.example.com": { + "params": { "npro.h1.keepalive": "5s" }, + "routes": [ + { "path": "/", "to": "www.http" }, + { "path": "/rtc", "to": "webrtc.eu1.signalling" } + ] + } + }, + "services": { + "www": { "type": "files", "params": { "root": "/var/www/rtc" } } + } +} +``` + +The route names `webrtc.eu1.signalling`, not an address: resolution +finds where that is from the deployment, so each address is written +once. `media` faces the public directly; a datagram listener cannot be +proxied by the front. + +## Resolution and publication + +**The front is the policy authority.** At its start, and when it is +told to reload, it resolves the layers. It then serves each instance its +**fragment**: everything bound in that instance's namespace, flattened, +with the exports it refers to and the `npro.*` parameters that apply to +it. + +**An instance fetches its fragment when it starts**, as an SS client +stream to its authority. A restarted instance, on any machine, always +starts with the latest configuration; there is no resolving at deploy +time, which could only produce configuration that had gone stale. + +- The instance **caches** the last fragment it accepted. If the + authority cannot be reached, it starts from the cache. With no cache + it fails to start, and systemd tries again. +- A new configuration reaches an instance when it next restarts. + *(Later: the front could tell a connected instance that its namespace + has changed, and the instance would exit cleanly, so systemd brings it + back with the new one.)* + +### The handshake + +The front links none of the separately built services, so it cannot know +their structure by itself. **The instance sends its structure when it +asks for its fragment**: "I am `webrtc.eu1`, built from npro-webrtc +0.3.0, and these are my listeners and parameters." The front resolves +against that, and answers with the fragment or with the resolution's +errors. So: + +- the front stays generic, and links only the services it hosts itself; +- an instance whose new binary declares a new parameter gets its + default, or a clear refusal if it is required and unbound; +- the front knows which configuration version each instance took, which, + with the health it sees by proxying, gives a status page such as + "eu1: healthy, policy v42" or "eu2: refused, `media.bind` unbound". + +*(Open: agreed in discussion as the direction, not yet confirmed.)* + +### Signing + +A fragment is configuration, and configuration is authority: it can +redirect endpoints and name certs. So: + +- **the front signs each fragment**, and the instance checks it with the + key in its local layer. The front has to hold the signing key, since + it resolves at run time. That adds no exposure: the front already + holds the tls keys and sees all the traffic; +- a fragment carries **its namespace and a version**, and an instance + refuses one for another namespace, or older than the one it has + cached, so a captured fragment cannot be replayed to roll an instance + back; +- fragments are signed, not encrypted, so **they carry no private + keys**. An instance's own secrets (the private key of its mutual tls + with the front, say) stay in its local layer, and a fragment refers to + them by name. + +The format (JWS, when npro has jose; or something smaller first) and the +signature algorithm (Ed25519, from whatever crypto the tls decision +admits) are open. + +### Transport to the authority + +On the same machine, a unix domain socket whose permissions admit only +the instance's user. Across machines, tcp with mutual tls, whose certs +are named in the local layer. The same channel, in the other direction, +is how the front proxies to the instance. + +### Health + +systemd keeps processes running. The front sees whether each instance +is healthy by proxying to it: failed connects and timeouts. It uses that +for a fast 503 on the routes of an instance that is down, for backing off +its reconnects with the instance's retry policy, and for the status page. +An optional active probe may come later. + +## From the C SS policy + +| C | npro | +|---|---| +| `release`, `product`, `schema-version` | the policy's header; the version that rollback protection checks is the fragment's, set by the front | +| `retry` | named retry schemes, as now; their limits (C: at most 8 backoff steps) become declared bounds | +| `certs` (base64 DER inline) | public certs may still be inline; private keys come only from the local layer or a file, never inline | +| `trust_stores` | kept, referenced by name | +| `s` (client streamtypes) | client streamtypes, declared in a structure by the service or program that uses them, with parameters the policy binds (endpoint, trust, retry) | +| `"server": true` streamtypes | listeners, vhosts, services and routes. C's one-listener server is the smallest case, and a short form of it stays | +| `metadata`, `${metadata}` | stream metadata, copied, never held by pointer, so a stream's state can cross a process boundary; headers by name are the default, which C has only with `direct_proto_str` | +| `options[]` (parsed, never used in C) | service parameters | +| overlay (modifies existing streamtypes only) | the layers | +| replacing the policy (destroys every stream) | restart into the new fragment | +| `fetch_policy` | fetching the fragment from the authority | +| static policy, `policy2c` | a `const` policy in Rust | +| `auth`, `metrics` | later; not designed here | + +## From lejp-conf + +Where each group of lejp-conf's keys lands. "Param" means a declared +parameter bound at the scope named. + +**Globals** + +| lejp-conf | npro | +|---|---| +| uid, gid, username, groupname, rlimit-nofile | not npro: systemd's `User=`, `Group=`, `LimitNOFILE=` | +| count-threads, count-async-threads | the runner's parameters | +| plugin-dir | gone: services are compiled in, or run as their own process | +| init-ssl | gone | +| server-string, timeout-secs, http-header-data | `npro.*` params, process scope | +| ip-limit-ah, ip-limit-wsi | `npro.*` limit params | +| default-alpn | listener tls | +| quic-* | `npro.quic.*` params | +| reject-service-keywords | a route guard | +| cpd-bypass | the client side's captive portal detection, a param | +| ws-pingpong-secs | gone (deprecated in C) | + +**Per vhost** + +| lejp-conf | npro | +|---|---| +| name | the vhost's key | +| port, interface, unix-socket, unix-socket-perms, noipv6, ipv6only, fo-listen-queue | listeners, which are separate from vhosts; a listener lists the vhosts it serves | +| host-ssl-key, -cert, -ca, -cert-grace-secs | listener tls, chosen per vhost by SNI; the private key from the local layer or a file | +| ciphers, tls13-ciphers, ecdh-curve, ssl-option-set/clear, alpn, client-cert-required, ignore-missing-cert | listener tls params | +| sts, redirect-http, allow-non-tls, allow-http-on-https, strict-host-check, sni-fallback | listener and vhost params | +| access-log, keepalive_timeout, h2-half-closed-long-poll, quic-mtu, quic-preferred-addresses | `npro.*` params, vhost scope | +| headers[] | the vhost's response headers | +| error-document-404 | the vhost's error documents | +| listen-accept-role, -protocol, apply-, fallback-listen-accept, onlyraw | a listener's kind, and its `Raw` endpoint | +| disable-no-protocol-ws-upgrades | a route rule | +| enable-client-ssl, client-ssl-* | client trust and certs, at the scope of the streams that use them | +| ws-protocols[] (pvo) | services, with their parameters | +| dht[] | a dht service | + +**Per mount** + +| lejp-conf | npro | +|---|---| +| mountpoint, exact-match, append-path | the route's path and how it matches; the longest matching path wins | +| origin `file://`, `gzip://` | the files service, with precompressed variants as its param | +| origin `http://`, `https://` | the proxy service: a route bridged to a client stream | +| origin `>http://`, `>https://` | a redirect route | +| origin `callback://` | a service's endpoint, by name | +| origin `cgi://`, cgi-* | open: a cgi service, or not carried over | +| default, extra-mimetypes, interpret, cache-* | the files service's params | +| auth-mask, basic-auth, interceptor-path | route guards | +| headers[], keepalive-timeout, no-ws-upgrades | route params | +| pmo[] | service params, route scope | + +**Not carried over**: the `"=NAME"` / `${NAME}` preprocessor (exports +and references cover what it was used for), and `conf.d` order. + +## Decided + +- 2026-10-07: the SS policy is npro's one configuration language, and + takes over lejp-conf's job. Services in their own processes are + fronted over unix domain sockets, or mutual tls across machines. +- 2026-10-08: parameters only; npro's own configuration is parameters + too, under the `npro` namespace. +- 2026-10-08: instances fetch their fragment from the front when they + start, and cache the last good one; there is no resolving at deploy + time. Fragments are signed, and checked with a key held locally. +- 2026-10-08: npro does not supervise processes; systemd does. npro + knows their health from proxying to them. + +## Open + +- The scope rule (same-scope duplicates are errors; nested scopes + refine). +- The handshake, with the instance sending its structure. +- Names for the `npro.*` parameters, and how they are grouped. +- The fragment's signature format and algorithm. +- Telling a running instance that its namespace changed. +- cgi. +- The JSON shapes above, which are illustrations, not a schema.
Page fetched 0s ago, creation time: 3ms (vhost etag hits: 0%, cache hits: 0%)