Project homepage Mailing List  Warmcat.com  API Docs  Github Mirror 
    npro  
 Modern all-safe Rust Network Protocol library supporting h1, h2, h3, ws, wt sans-IO and with socket IO + tls
git clone https://npro.rs/repo/npro
 
root / assets / windows-10.png
Author[]Andy Green <andy@warmcat.com> 2026-09-28 15:40 UTC
Committer[]Andy Green <andy@warmcat.com> 2026-10-04 04:09 UTC
Tree5050c4cb32aebc6f55acf9e77567d3494a256bfe   Raw Patch
 
push: add sai-push, to promote branches whose builds succeed
push: add sai-push, to promote branches whose builds succeed

sai-push follows sai-web feeds of events, as rss.json scoped to a watched
repository's fetch url, using the feed's long poll so it only hears from
sai-web when an event joins the feed or changes state.  When an event
succeeds on a branch one of the watch's rules matches, eg, main-dev, it
pushes that commit to the branch the rule maps it to, eg, main.

Rules are tried in order: the first whose wildcard "match" the branch
fits, and which it ends with the "branch-suffix" of, decides.  The
suffix is removed to name the target branch, and "force" says if the
push to it may be forced, so main-dev can force main while a
v5.0-stable-dev only ever fast-forwards v5.0-stable.

For each target branch, the newest successful non-ad-hoc event mapping
to it is promoted.  A promotion runs git, one command at a time, on a
bare repo per project in the repo cache dir: fetch the remote's
branches, check the commit is still on the source branch (not rewritten
since), check the target doesn't already have it, then push.  That last
check also stops an older success that comes back to the top, eg, when
a newer event's tasks are restarted, from rewinding a force-pushed
target.

Beyond what git can tell, an event whose notification arrived before
the one last promoted to a target is never promoted over it.  The last
event promoted to each target is kept in sai-push-state.json in the repo
cache, written by rename so it's always whole, so this holds across
restarts.

A watch can list mirrors, which get the same commit pushed to the same
branch after the primary remote, each checked first for already having
it.  A primary failure leaves the mirrors alone; a mirror failure
doesn't stop the others, and is retried later.  A mirror reached over
https, eg, github, can name a token file, eg, holding a fine-grained
access token limited to the mirror repos: git gets it by running
sai-push itself as its GIT_ASKPASS, so the token is never in a url, an
argv or the logs, and git's credential helpers are not used for it.
Credentials in a remote or mirror url, and token files anyone can read,
are refused.

The conf is checked whole before it's used: a member that's unknown or
in the wrong place (it says where it belongs), a value of the wrong
kind, or a missing schema, stops it starting with the line of each
problem, rather than lws_struct silently skipping it.  It warns about
http(s) remotes and mirrors it has no credentials for, and logs each
watch's remote and mirrors, or that it has none.

git is run from argv without a shell, with ssh in BatchMode, and the
project, branches and hash from the feed are validated first.  It's
started as root, creates the repo cache dir for the conf's "user", and
becomes that user inside lws_create_context(), before doing anything on
the network.  Feeds must be https except from loopback.

This needs lws with 864f9da72 ("lejp: pop an array's element parser when
the array ends, at any depth"): without it, the second of a watch's
"rules" and "mirrors" lists is silently dropped from the conf.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
diff --git a/CMakeLists.txt b/CMakeLists.txt index 1e2a70d..b9fb56b 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -62,6 +62,7 @@ set(CPACK_DEBIAN_BUILDER_PACKAGE_NAME "sai-builder") option(SAI_SERVER "Build the server + web part" ON) option(SAI_BUILDER "Build the builder part" ON) +option(SAI_PUSH "Build sai-push, which promotes branches that pass" ON) if (NOT SAI_SERVER AND NOT SAI_BUILDER) @@ -182,6 +183,9 @@ if (requirements) add_subdirectory(src/power) endif() endif() + if (SAI_PUSH AND NOT MSVC AND NOT WIN32) + add_subdirectory(src/push) + endif() if (SAI_BUILDER) add_subdirectory(src/builder) add_subdirectory(src/resource) diff --git a/README.md b/README.md index 9f527c8..784c509 100644 --- a/README.md +++ b/README.md @@ -244,6 +244,19 @@ at the start) and `<rss>`, so the body is still a valid document. At most 64 requests are held at once; past that, a request to wait is answered with a 503 and `retry-after`. +## sai-push: promoting branches that pass + +sai-push is an optional daemon, usually run on the git server host, that +follows sai-web's feed (as JSON, using the long poll above, so it only hears +from sai-web when something happened) and promotes commits that pass: eg, +when an event on `main-dev` succeeds, it pushes that commit to `main`, with +or without force depending on the branch, and optionally to the same branch +on mirrors such as github. + +See [READMEs/README-sai-push.md](READMEs/README-sai-push.md) for how it +decides what to push, setting it up, and all of its conf options with an +example. + ## Build flow and support for embedded ![build flow](READMEs/sai-build-test-flow.png) diff --git a/READMEs/README-sai-push.md b/READMEs/README-sai-push.md new file mode 100644 index 0000000..ce25b71 --- /dev/null +++ b/READMEs/README-sai-push.md @@ -0,0 +1,335 @@ +# sai-push + +## Overview + +sai-push is an optional daemon, usually run on the git server host, that +promotes commits once Sai says they're good: eg, when an event on +`main-dev` succeeds, it pushes that commit to `main`, and optionally to the +same branch on mirrors such as github. + +It follows sai-web's public feed of events (see "RSS feed of build events" +in the top level README), as JSON, using the feed's long poll: after the +first fetch, it holds a request open with sai-web and only hears back when +an event joins the feed or changes state, eg, from `building` to +`succeeded`. So it reacts within moments of a build finishing, without +polling. + +## How it decides what to push + +Each watch in the conf covers one repository, identified by the fetch url +its sai notifications give, and has rules mapping source branches to target +branches: + + - The first rule matching a branch decides: the branch must match the rule's + wildcard `match` (default `*`) and end in its `branch-suffix`. The suffix + is removed to name the target branch, eg, `main-dev` -> `main`. Branches + no rule matches are left alone. + + - For each target branch, the newest successful event on a source branch + mapping to it is the one promoted. That's the latest known-good commit, + even if newer events are still building or failed. Ad-hoc events (admin + scratch builds from the web UI) are never promoted. + + - An event whose notification arrived before the one last promoted to that + target is never promoted over it. This is remembered across restarts, + see "State" below. + +Before pushing, sai-push fetches the primary remote into a bare repo in its +cache dir and checks: + + - the commit is still on the source branch: if the source branch was + rewritten since, the commit isn't what the branch says is good any more, + and it's skipped + + - the target branch doesn't already have the commit: if it does, there's + nothing to do. So an older success coming back to the top, eg, because a + newer event's tasks were restarted, can't rewind the target, even where + force pushing is allowed. + +Then it pushes the commit to the target branch on the primary remote, +forced if the rule says so, and then to each mirror in turn, each checked +first for already having it. A target branch that doesn't exist yet is +created. + +If anything fails on the primary, the mirrors are not touched. If a mirror +fails, the other mirrors are still pushed. Failures are logged with git's +own output, and the same commit is tried again no sooner than 10 minutes +later, when the feed next changes or the long poll's 10 minute wait +expires. + +## Setting it up + +1) Build and install sai with the `SAI_PUSH` cmake option (on by default on + non-Windows), which installs `/usr/local/bin/sai-push`. It needs lws + built with `LWS_WITH_CLIENT`, `LWS_WITH_STRUCT_JSON` and `LWS_WITH_SPAWN`, + and recent enough to have + + - `864f9da72` "lejp: pop an array's element parser when the array ends, + at any depth": without it, a watch that has both `rules` and `mirrors` + silently loses whichever of them comes second in the conf + + and, in the lws front-end web server that proxies sai-web, + + - `fbedb4001` "proxy: a response still flowing keeps the parent's content + timeout off": without it, the front-end drops each held long poll + request after its `timeout_secs`, and sai-push goes round a fetch / + wait cycle that often instead of waiting quietly. + +2) Create a user just for sai-push, eg, `sai-push`, with a home dir and no + password. sai-push is started as root, and becomes this user before it + does anything on the network; git and ssh run as it. + +3) As that user, create an ssh key, and give it write access to the repos it + should manage on the git server, eg, in gitolite. + +4) Put the git server's host key in that user's `~/.ssh/known_hosts`, under + exactly the host name (and port, if not 22) the `remote` url uses. ssh + runs in BatchMode, so it will not ask, and an unknown host fails with + `Host key verification failed.` If the git server is this host, you can + take the key straight from its own host key file, rather than trusting + what comes over the network the first time, eg, + + ``` + # sudo -u sai-push sh -c 'awk "{print \"libwebsockets.org \" \$1 \" \" \$2}" /etc/ssh/ssh_host_ed25519_key.pub >> ~/.ssh/known_hosts' + ``` + + or log in once as that user with the same name as the `remote` url: + `sudo -u sai-push ssh git@libwebsockets.org`. + +5) For a github mirror, the best fit is a deploy key: an ssh key that + github accepts for just the one repository it's added to, with write + access if you allow it. It doesn't expire, so a headless daemon can't + quietly lose access to the mirror a month later, and it's revoked from + the repository's settings if the host is ever compromised. It works the + same way as the gitolite key, as an ssh mirror url, with no token file. + + - Make a key just for this, as the sai-push user, separate from the + gitolite one so either can be revoked alone (github also refuses a + key that's already in use anywhere else on github): + + ``` + # sudo -u sai-push ssh-keygen -t ed25519 -N "" -C "sai-push github libwebsockets" -f /home/sai-push/.ssh/github_libwebsockets + ``` + + - In the github web UI, as an admin of the repository: the repository's + **Settings** tab, then **Deploy keys** in the sidebar (eg, + `https://github.com/warmcat/libwebsockets/settings/keys`), then **Add + deploy key**. Give it a title, eg, `sai-push on libwebsockets.org`, + paste in the contents of the **public** key, + `/home/sai-push/.ssh/github_libwebsockets.pub`, tick **Allow write + access**, and **Add key**. + + - Tell ssh to use that key for github, in `/home/sai-push/.ssh/config` + (owned by sai-push, mode 0600): + + ``` + Host github.com + IdentityFile ~/.ssh/github_libwebsockets + IdentitiesOnly yes + ``` + + - Put github's host key in sai-push's `known_hosts`: check the + fingerprint ssh shows against the ones github publishes at + https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/githubs-ssh-key-fingerprints + before accepting it, with + + ``` + # sudo -u sai-push ssh -T git@github.com + ``` + + It should answer `Hi warmcat/libwebsockets! You've successfully + authenticated, but GitHub does not provide shell access.`: the + repository name there confirms it's the deploy key being used. + + - The mirror in the conf is then just `{ "url": + "ssh://git@github.com/warmcat/" }`. + + A deploy key covers only its one repository. To mirror another repo, + make it its own key and its own watch, and give it a host alias in the + ssh config, keeping github's host key entry by using `HostKeyAlias`: + + ``` + Host github-otherproject + HostName github.com + HostKeyAlias github.com + IdentityFile ~/.ssh/github_otherproject + IdentitiesOnly yes + ``` + + with that watch's mirror url `ssh://git@github-otherproject/warmcat/`. + + If the target branch is protected on github by a branch protection rule + or a ruleset, the deploy key must be allowed to push to it, and to force + push to it if the rule forces: rulesets can list "Deploy keys" as a + bypass actor. + + Alternatively, an https mirror can use a token, see `token-file` below, + eg, a github fine-grained personal access token restricted to just the + mirror repositories with "Contents: Read and write". But those expire: + when it does, the mirror pushes start failing, with git's message in the + log, until you make a new one and replace the file. + +6) Write the conf, `/etc/sai/push/conf`, see below. + +7) Install and start the systemd unit, `scripts/sai-push.service`: + + ``` + # cp scripts/sai-push.service /etc/systemd/system/ + # systemctl daemon-reload + # systemctl enable --now sai-push + ``` + + It orders itself after `sai-web.service`, for when sai-web is on the same + host, without depending on it: sai-push is often on the git server with + sai-web elsewhere, and retries the feed with backoff anyway. If the feed + url goes through a front-end web server on this host, add its unit to the + `After=` line too. + +## Command line + +|option|meaning| +|---|---| +|`-c <file>`|conf file to use, default `/etc/sai/push/conf`| +|`-d <loglevel>`|lws log level bitmap, eg, `-d 1039` adds info logging| + +## Configuration + +The conf is JSON, read from `/etc/sai/push/conf` unless `-c` says otherwise. +`#` comments to the end of the line are allowed. + +sai-push checks the whole conf before starting, and refuses to start if +anything in it isn't usable, logging each problem with its line number: + + - a member it doesn't know, eg, a misspelling, or one that's in the wrong + place, eg, `mirrors` outside the watch it belongs to: it says where a + misplaced member should go + - a value of the wrong kind, eg, `"force": "true"` (a string, rather than + `true`), or a single `{ }` where a list `[ { } ]` is needed + - a missing or wrong `schema`, or a file that isn't valid JSON + +It also warns, without refusing to start, about an http(s) `remote`, or an +http(s) mirror without a `token-file`, since git has no way to authenticate +pushes there unless the `user`'s own git config gives it credentials. At +startup it logs, for each watch, its rules, the remote it pushes to, and its +mirrors, or "no mirrors". + +### Example + +``` +# sai-push conf: /etc/sai/push/conf +{ + "schema": "sai-push", + + "user": "sai-push", + "repo-cache": "/var/cache/sai-push", + + "watches": [{ + "feed": "https://libwebsockets.org/sai/rss.xml", + "fetchurl": "https://libwebsockets.org/repo/libwebsockets", + "remote": "ssh://git@libwebsockets.org/", + + "rules": [ + # eg, v5.0-stable-dev -> v5.0-stable, fast-forward only + { "match": "*-stable-dev", "branch-suffix": "-dev", + "force": false }, + # anything else, eg, main-dev -> main, forced + { "branch-suffix": "-dev", "force": true } + ], + + "mirrors": [ + # a github deploy key for the repo, see setup step 5 + { "url": "ssh://git@github.com/warmcat/" } + ] + }] +} +``` + +With this, when an event for libwebsockets succeeds on `main-dev`, its commit +is force-pushed to `main` on `ssh://git@libwebsockets.org/libwebsockets` and +then on `ssh://git@github.com/warmcat/libwebsockets`; when one succeeds on +`v5.0-stable-dev`, its commit is pushed to `v5.0-stable` on both, but only +if that's a fast-forward. + +The same example, commented, is in `etc-sai-EXAMPLE/push/conf`. + +### Top level + +|member|required|meaning| +|---|---|---| +|`schema`|yes|must be `"sai-push"`| +|`user`|when started as root|the user sai-push becomes before going on the network. It must exist and not be root. git and ssh run as it, with its home dir as `HOME`, so its ssh keys, `known_hosts` and git config are the ones used. If sai-push is started as someone else, eg, to test it by hand, it stays as them and warns that it's ignoring this| +|`repo-cache`|yes|absolute path of a dir sai-push keeps a bare repo per project in, eg, `libwebsockets.git`, so each promotion only has to fetch what's new, plus its state file. It's created mode 0700 if needed, and given to `user` at startup| +|`watches`|yes|array of one or more watches, see below| + +### Watch + +Each watch covers one repository. + +|member|required|meaning| +|---|---|---| +|`feed`|yes|the sai-web feed url, eg, `https://libwebsockets.org/sai/rss.xml`. sai-push reads the same feed as JSON, so a url ending `rss.xml` has that changed to `rss.json`; it must end in one or the other. sai-push adds `?fetchurl=` for the watch itself, so the feed is scoped to the repository. It must be https, since sai-push acts on what the feed says, except that plain http is allowed from `localhost`, `127.0.0.1` or `::1`| +|`fetchurl`|yes|the repository's fetch url exactly as its sai notifications give it (`repository.fetchurl` in the notification JSON, shown as `sai:fetchurl` in the RSS feed). Only events with this fetch url are considered| +|`remote`|yes|the primary remote: a url prefix, the event's project name (eg, `libwebsockets`) is appended to it. It's where the source branches, eg, `main-dev`, are fetched from and checked, and the first place the target branches are pushed to. Usually ssh, using the `user`'s keys. It must not contain credentials| +|`rules`|yes|array of one or more rules, see below, tried in order| +|`mirrors`|no|array of mirrors, see below, which get the same pushes as `remote`, in order, after it| + +### Rule + +|member|required|meaning| +|---|---|---| +|`branch-suffix`|yes|the source branch must end with this, eg, `-dev`. It's removed to name the target branch. A branch that is only the suffix doesn't match| +|`match`|no|a pattern the whole source branch name (without `refs/heads/`) must match, with up to three `*` wildcards, eg, `*-stable-dev` or `v5.*-dev`. Default `*`, any branch| +|`force`|no|`true` if the push to the target branch may be a force push, eg, for a `main` that follows a rebased `main-dev`. Default `false`: the push must be a fast-forward, and is refused and logged otherwise. Mirrors are pushed with the same setting| + +### Mirror + +|member|required|meaning| +|---|---|---| +|`url`|yes|a url prefix, the event's project name is appended to it, eg, `ssh://git@github.com/warmcat/` for `ssh://git@github.com/warmcat/libwebsockets`. For github, ssh with a deploy key is recommended, see setup step 5; ssh uses the `user`'s ssh config and keys. It must not contain credentials: sai-push refuses to start if an https url has any| +|`token-file`|no|absolute path (`~` is not expanded) of a file holding a token git gives as the password when the (https) mirror asks, eg, a github fine-grained personal access token. Whitespace around the token in the file is ignored. Keep it with the `user`'s other credentials, eg, `/home/sai-push/.github-token`, owned by `user` and mode 0600. It must be readable by `user`, and must not be readable by everyone: sai-push refuses to start otherwise. git gets it by running sai-push itself as its `GIT_ASKPASS`, so the token is never in a url, an argv or the logs, and git's own credential helpers are not used for that mirror| + +## State + +`sai-push-state.json` in the `repo-cache` dir holds, for each target +branch, the last event promoted to it on the primary remote: its uuid, its +commit and the unix time its notification arrived. It's what stops an +older event being promoted over a newer one, including across restarts. +It's rewritten, by writing a new file and renaming it over the old one, +each time a promotion reaches the primary. + +If it's damaged, sai-push refuses to start rather than carry on without +it; removing it starts over, with only the git checks protecting the target +branches until the next promotion. + +## Logs + +sai-push logs to stderr, so under systemd, to the journal. Among other +things it logs + + - the rules and mirrors it understood from the conf, at startup + + - each feed update it acts on: `saip_feed_process: <fetchurl>: index ..., + N events` + + - each promotion started, and how each step ended, eg, `promoting to main + (force)`, `pushed <hash> to main on <remote>`, `main on <remote> already + has <hash>`, `<hash> is no longer on main-dev, not promoting it`, or `not + promoting <hash> ... it's older than event <uuid>, already promoted` + + - anything git says while doing it, a line at a time + +## Troubleshooting + +|symptom|cause| +|---|---| +|`git: Host key verification failed.`|the `user`'s `known_hosts` has no entry for the exact host name (and port) in the `remote` url. See step 4 above| +|`git: Permission denied (publickey)` or gitolite refusing access|the `user`'s ssh key isn't allowed write access to the repo on the git server| +|`git: ERROR: The key you are authenticating with has been marked as read only.`|the github deploy key was added without **Allow write access**: delete it and add it again with that ticked| +|`git: ERROR: Repository not found.` from github over ssh|ssh offered a key github doesn't have as a deploy key for that repo, eg, the gitolite key: check the `IdentityFile` / `IdentitiesOnly` lines in the `user`'s ssh config, and that `sudo -u sai-push ssh -T git@github.com` greets the right repo| +|a github push is refused by a protected branch rule|allow the deploy key (or the token's account) to push, and force push for a forced rule, or add it as a bypass actor, see setup step 5| +|a github https mirror push fails with an authentication error|the token is wrong or expired, or lacks "Contents: Read and write" on that repo| +|`! [rejected] ... (non-fast-forward)`|the rule doesn't allow a force push, and the target branch has moved on in a way the commit doesn't extend. Retried every 10 minutes while it's the newest success| +|`incomplete feed, retrying` roughly every `timeout_secs` of the front-end web server|the lws front-end proxying sai-web lacks `fbedb4001`, see step 1| +|`... doesn't belong there (...), it goes in ...`, `unknown member ...` or `... should be ...`, and sai-push won't start|the conf has a member in the wrong place, misspelled, or with the wrong kind of value, at the line given: see "Configuration"| +|a github or other mirror never gets pushed, with nothing logged about it|check the startup log has `mirrored to <url>` for it. If it says `no mirrors`, the `mirrors` list isn't inside the watch. If the conf is right, lws may lack `864f9da72`, see step 1| +|nothing is promoted although builds succeed|check the watch's `fetchurl` matches `sai:fetchurl` in the RSS feed exactly, and that a rule matches the branch| diff --git a/etc-sai-EXAMPLE/push/conf b/etc-sai-EXAMPLE/push/conf new file mode 100644 index 0000000..49c7621 --- /dev/null +++ b/etc-sai-EXAMPLE/push/conf @@ -0,0 +1,69 @@ +# sai-push conf: /etc/sai/push/conf +# +# sai-push follows sai-web's feed of events for each watch, and when an event +# for the watched repository succeeds on a branch one of the watch's rules +# matches, it pushes that commit to the branch the rule maps it to. +# +# See READMEs/README-sai-push.md for all the options and setting it up. +{ + "schema": "sai-push", + + # sai-push is started as root and becomes this user before doing + # anything on the network. Make a user just for this, whose ssh key + # the git server (eg, gitolite) accepts for pushing to the repos it + # should manage, and whose ~/.ssh/known_hosts already has the git + # server's host key (ssh runs with BatchMode, so it won't ask). + "user": "sai-push", + + # A bare repo per project is kept here, so each promotion only has to + # fetch what's new. It's created, and given to "user", at startup. + # sai-push-state.json in here remembers the last event promoted to + # each branch, so an older event is never promoted over it, even + # across restarts. + "repo-cache": "/var/cache/sai-push", + + "watches": [{ + # the sai rss feed (sai-push uses the same feed as JSON, + # rss.json). Must be https. + "feed": "https://libwebsockets.org/sai/rss.xml", + + # only events whose notification gave this fetch url count + "fetchurl": "https://libwebsockets.org/repo/libwebsockets", + + # the project name from the event is appended to make the url + # we fetch from and push to + "remote": "ssh://git@libwebsockets.org/", + + # Optional: other repos that get whatever we push to "remote", + # the same commit on the same branch. The project name is + # appended to "url" the same way. + # + # For github, use ssh with a deploy key for the repo, which + # doesn't expire, see READMEs/README-sai-push.md setup step 5. + # + # An https mirror can instead name a "token-file" holding a + # token with write access to just the mirror repos, eg, + # { "url": "https://github.com/warmcat/", + # "token-file": "/home/sai-push/.github-token" } + # owned by "user", mode 0600, absolute path ("~" isn't + # expanded). But tokens expire, and the pushes then fail + # until it's replaced. Never put credentials in the url. + "mirrors": [ + { "url": "ssh://git@github.com/warmcat/" } + ], + + # The first rule matching a branch that has an event succeed + # decides. "match" is a wildcard pattern on the branch + # (default "*"), the branch must end in "branch-suffix", which + # is removed to name the branch the commit is pushed to, and + # "force" says if that push may be a force push. Branches no + # rule matches are left alone. + "rules": [ + # eg, v5.0-stable-dev -> v5.0-stable, fast-forward only + { "match": "*-stable-dev", "branch-suffix": "-dev", + "force": false }, + # anything else, eg, main-dev -> main, forced + { "branch-suffix": "-dev", "force": true } + ] + }] +} diff --git a/scripts/sai-push.service b/scripts/sai-push.service new file mode 100644 index 0000000..e1d391a --- /dev/null +++ b/scripts/sai-push.service @@ -0,0 +1,20 @@ +[Unit] +Description=Sai push +# +# sai-push is no use until the sai-web whose feed it follows is up. When +# that's on this host, start after it (and after the front-end web server +# proxying it, if the feed url goes through that: add its unit here). +# Ordering only, not a dependency: sai-push is often on the git server +# host with sai-web elsewhere, and it retries the feed with backoff anyway. +# +After=network-online.target sai-web.service +Wants=network-online.target + +[Service] +# starts as root, becomes the conf's "user" before going on the network +ExecStart=/usr/local/bin/sai-push +Restart=on-failure +RestartSec=10 + +[Install] +WantedBy=multi-user.target diff --git a/src/push/CMakeLists.txt b/src/push/CMakeLists.txt new file mode 100644 index 0000000..18b0485 --- /dev/null +++ b/src/push/CMakeLists.txt @@ -0,0 +1,41 @@ +set(SUB "sai-push") +set(CPACK_DEBIAN_PUSH_PACKAGE_NAME "sai-push") + +set(SRCS + pu-sai.c + pu-feed.c + pu-git.c + pu-state.c + ../common/c-utils.c + ../common/struct-metadata.c +) + +set(requirements 1) +require_lws_config(LWS_WITH_CLIENT 1 requirements) +require_lws_config(LWS_WITH_STRUCT_JSON 1 requirements) +require_lws_config(LWS_WITH_SPAWN 1 requirements) + +if (requirements) + add_executable(${SUB} ${SRCS}) + add_dependencies(${SUB} sai_git_hash) + if (APPLE) + set_property(TARGET ${SUB} PROPERTY MACOSX_RPATH YES) + endif() + + if (SAI_LWS_INC_PATH) + target_include_directories(${SUB} PRIVATE ${SAI_LWS_INC_PATH}) + endif() + + target_link_libraries(${SUB} websockets ${SAI_LWS_LIB_PATH} ${LIBWEBSOCKETS_DEP_LIBS}) + + if (LWS_OPENSSL_LIBRARIES) + target_link_libraries(${SUB} ${LWS_OPENSSL_LIBRARIES}) + endif() + + if (SAI_EXT_PTHREAD_LIBRARIES) + target_link_libraries(${SUB} ${SAI_EXT_PTHREAD_LIBRARIES}) + endif() + + install(TARGETS ${SUB} + RUNTIME DESTINATION "${BIN_DIR}" COMPONENT push) +endif(requirements) diff --git a/src/push/pu-feed.c b/src/push/pu-feed.c new file mode 100644 index 0000000..15bc4dd --- /dev/null +++ b/src/push/pu-feed.c @@ -0,0 +1,411 @@ +/* + * Sai push - src/push/pu-feed.c + * + * Copyright (C) 2026 Andy Green <andy@warmcat.com> + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation: + * version 2.1 of the License. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, + * MA 02110-1301 USA + * + * Following a watch's sai-web feed: the first request gets the feed as it + * is, and each one after asks sai-web to hold it until the feed's index + * moves on from the one we last saw. Held responses have their headers at + * once and a newline every 10s until the feed follows, so a quiet connection + * is a dead one. + */ + +#include <libwebsockets.h> +#include <string.h> + +#include "pu-private.h" + +static const uint32_t saip_backoff_ms[] = { 1000, 2000, 5000, 10000, 30000, + 60000 }; + +static const lws_retry_bo_t saip_retry = { + .retry_ms_table = saip_backoff_ms, + .retry_ms_table_count = LWS_ARRAY_SIZE(saip_backoff_ms), + .conceal_count = LWS_RETRY_CONCEAL_ALWAYS, + .jitter_percent = 20, +}; + +static void +saip_feed_connect(lws_sorted_usec_list_t *sul) +{ + saip_watch_t *w = lws_container_of(sul, saip_watch_t, sul); + char path[768], uenc[3 * 96 + 1]; + struct lws_client_connect_info i; + int n; + + lws_urlencode(uenc, w->fetchurl, (int)sizeof(uenc)); + n = lws_snprintf(path, sizeof(path), "%s?fetchurl=%s", w->path, uenc); + if (w->index[0]) + lws_snprintf(path + n, sizeof(path) - (size_t)n, + "&wait=%d&index=%s", SAIP_WAIT_S, w->index); + + lwsac_free(&w->a.ac); + w->parse_done = w->parse_failed = w->handled = 0; + w->http_status = 0; + + memset(&i, 0, sizeof(i)); + i.context = saip.cx; + i.address = w->host; + i.port = w->port; + i.path = path; + i.host = w->host; + i.origin = w->host; + i.method = "GET"; + /* + * A held response is one long-lived stream: keep it the only thing + * on its connection + */ + i.alpn = "http/1.1"; + i.ssl_connection = w->tls ? LCCSCF_USE_SSL : 0; + i.protocol = protocol_saip_feed.name; + i.pwsi = &w->wsi; + i.opaque_user_data = w; + + lwsl_info("%s: %s%s\n", __func__, w->host, path); + + if (!lws_client_connect_via_info(&i)) + /* the failure went to CLIENT_CONNECTION_ERROR */ + lwsl_notice("%s: connect to %s failed\n", __func__, w->host); +} + +static void +saip_feed_schedule(saip_watch_t *w, int failed) +{ + uint32_t ms = 0; + + if (failed) + ms = lws_retry_get_delay_ms(saip.cx, &saip_retry, + &w->retry_count, NULL); + else + w->retry_count = 0; + + lws_sul_schedule(saip.cx, 0, &w->sul, saip_feed_connect, + (lws_usec_t)ms * LWS_US_PER_MS); +} + +void +saip_feed_start(saip_watch_t *w) +{ + saip_feed_schedule(w, 0); +} + +/* + * The response is over, one way or another: act on the feed if we got all + * of it, and ask again + */ +static void +saip_feed_done(saip_watch_t *w) +{ + if (w->handled) + return; + w->handled = 1; + + if (w->http_status != HTTP_STATUS_OK || !w->parse_done || + w->parse_failed || !w->a.dest) { + if (!w->http_status) + lwsl_notice("%s: %s%s: no response, retrying\n", + __func__, w->host, w->path); + else if (w->http_status != HTTP_STATUS_OK) + lwsl_notice("%s: %s%s: http %d, retrying\n", __func__, + w->host, w->path, w->http_status); + else + lwsl_notice("%s: %s%s: incomplete feed, retrying\n", + __func__, w->host, w->path); + + /* + * Start over with a plain fetch: whatever we missed, the feed + * as it is then covers it + */ + w->index[0] = '\0'; + lwsac_free(&w->a.ac); + saip_feed_schedule(w, 1); + + return; + } + + saip_feed_process(w, (sai_feed_t *)w->a.dest); + lwsac_free(&w->a.ac); + saip_feed_schedule(w, 0); +} + +static int +callback_saip_feed(struct lws *wsi, enum lws_callback_reasons reason, + void *user, void *in, size_t len) +{ + saip_watch_t *w = (saip_watch_t *)lws_get_opaque_user_data(wsi); + char buf[LWS_PRE + 4096], *px = buf + LWS_PRE; + int lenx = (int)sizeof(buf) - LWS_PRE, m; + + switch (reason) { + + case LWS_CALLBACK_CLIENT_CONNECTION_ERROR: + lwsl_notice("%s: %s: %s\n", __func__, w ? w->host : "?", + in ? (const char *)in : "connection error"); + if (w) { + w->wsi = NULL; + saip_feed_done(w); + } + break; + + case LWS_CALLBACK_ESTABLISHED_CLIENT_HTTP: + if (!w) + return -1; + + w->http_status = (int)lws_http_client_http_response(wsi); + + memset(&w->a, 0, sizeof(w->a)); + w->a.map_st[0] = lsm_schema_json_map_feed; + w->a.map_entries_st[0] = LWS_ARRAY_SIZE(lsm_schema_json_map_feed); + w->a.ac_block_size = 4096; + lws_struct_json_init_parse(&w->ctx, NULL, &w->a); + + lws_set_timeout(wsi, PENDING_TIMEOUT_USER_OK, + SAIP_RX_TIMEOUT_S); + break; + + case LWS_CALLBACK_RECEIVE_CLIENT_HTTP: + if (lws_http_client_read(wsi, &px, &lenx) < 0) + return -1; + break; + + case LWS_CALLBACK_RECEIVE_CLIENT_HTTP_READ: + if (!w) + return -1; + + /* anything, even a keepalive, means it's still alive */ + lws_set_timeout(wsi, PENDING_TIMEOUT_USER_OK, + SAIP_RX_TIMEOUT_S); + + if (w->http_status != HTTP_STATUS_OK || w->parse_done || + w->parse_failed) + break; + + m = lejp_parse(&w->ctx, (uint8_t *)in, (int)len); + if (m >= 0) + w->parse_done = 1; + else if (m != LEJP_CONTINUE) { + lwsl_notice("%s: feed JSON decode failed '%s'\n", + __func__, lejp_error_to_string(m)); + w->parse_failed = 1; + } + break; + + case LWS_CALLBACK_COMPLETED_CLIENT_HTTP: + if (!w) + break; + + saip_feed_done(w); + + /* we open a new connection for each request */ + return -1; + + case LWS_CALLBACK_CLOSED_CLIENT_HTTP: + if (!w) + break; + + w->wsi = NULL; + + /* + * A held response is close-delimited, so this may be how its + * end arrives... it was complete if the feed JSON was + */ + saip_feed_done(w); + break; + + default: + break; + } + + return lws_callback_http_dummy(wsi, reason, user, in, len); +} + +const struct lws_protocols protocol_saip_feed = { + .name = "sai-push-feed", + .callback = callback_saip_feed, +}; + +/* + * Project names go into a filesystem path and a remote url + */ +static int +saip_is_safe_project(const char *s) +{ + size_t n = 0; + + if (!s || !*s || *s == '.' || *s == '-') + return 0; + + for (; s[n]; n++) + if (!((s[n] >= '0' && s[n] <= '9') || + (s[n] >= 'a' && s[n] <= 'z') || + (s[n] >= 'A' && s[n] <= 'Z') || + s[n] == '.' || s[n] == '_' || s[n] == '-')) + return 0; + + return n <= 64; +} + +/* + * What the first rule matching the feed branch maps it to, or NULL if no + * rule wants it + */ +static const saip_rule_t * +saip_rule_for(saip_watch_t *w, const char *branch, char *dst, size_t dst_len) +{ + size_t bl = strlen(branch), sl; + + lws_start_foreach_dll(struct lws_dll2 *, p, w->rules.head) { + saip_rule_t *r = lws_container_of(p, saip_rule_t, list); + + sl = strlen(r->branch_suffix); + if (bl > sl && !strcmp(branch + bl - sl, r->branch_suffix) && + !lws_strcmp_wildcard(r->match, strlen(r->match), + branch, bl)) { + if (bl - sl >= dst_len) + return NULL; + memcpy(dst, branch, bl - sl); + dst[bl - sl] = '\0'; + + return r; + } + } lws_end_foreach_dll(p); + + return NULL; +} + +saip_target_t * +saip_target_get(saip_watch_t *w, const char *branch) +{ + saip_target_t *t; + + lws_start_foreach_dll(struct lws_dll2 *, p, w->targets.head) { + t = lws_container_of(p, saip_target_t, list); + if (!strcmp(t->branch, branch)) + return t; + } lws_end_foreach_dll(p); + + t = malloc(sizeof(*t)); + if (!t) + return NULL; + memset(t, 0, sizeof(*t)); + lws_strncpy(t->branch, branch, sizeof(t->branch)); + lws_dll2_add_tail(&t->list, &w->targets); + + return t; +} + +/* + * We have the feed as it is now. For each target branch, the newest event + * that succeeded on a feed branch mapping to it is the one that should be on + * it: that's the latest known-good commit, even if newer events are still + * building or failed, and older successes are superseded by it. + * + * The git job then only pushes it if it's still on the feed branch, and not + * already in the target branch, so an older success coming back to the top + * (eg, a newer event's tasks were restarted) can't rewind the target. And + * whatever git can tell, an event whose notification arrived before the one + * last promoted to the target is never promoted over it. + */ +void +saip_feed_process(saip_watch_t *w, sai_feed_t *f) +{ + lws_usec_t now = lws_now_usecs(); + char dst[65], seen[16][65]; + const saip_rule_t *r; + saip_target_t *t; + int ns = 0, n; + + lwsl_notice("%s: %s: index %s, %d events\n", __func__, w->fetchurl, + f->index, (int)f->items.count); + + lws_strncpy(w->index, f->index, sizeof(w->index)); + + /* the feed items are newest first */ + + lws_start_foreach_dll(struct lws_dll2 *, p, f->items.head) { + sai_feed_item_t *it = lws_container_of(p, sai_feed_item_t, + list); + + /* + * sai-web scoped the feed to our fetchurl already, but it + * costs nothing to be sure. Ad-hoc events are admin scratch + * builds, maybe with edited build scripts: never promoted. + */ + if (strcmp(it->fetchurl, w->fetchurl) || it->adhoc) + continue; + + if (strcmp(it->state_name, "succeeded")) + continue; + + r = saip_rule_for(w, it->branch, dst, sizeof(dst)); + if (!r) + continue; + + /* only the newest success for each target counts */ + for (n = 0; n < ns; n++) + if (!strcmp(seen[n], dst)) + break; + if (n < ns) + continue; + if (ns == (int)LWS_ARRAY_SIZE(seen)) + break; + lws_strncpy(seen[ns++], dst, sizeof(seen[0])); + + if (!saip_is_safe_project(it->project) || + !sai_is_safe_ref(it->branch) || !sai_is_safe_ref(dst) || + !sai_is_git_hash(it->hash) || + (strlen(it->hash) != 40 && strlen(it->hash) != 64)) { + lwsl_warn("%s: ignoring event %s with unexpected " + "project, branch or hash\n", __func__, + it->uuid); + continue; + } + + t = saip_target_get(w, dst); + if (!t) + continue; + + if (!strcmp(t->pushed, it->hash)) + continue; + + if (it->received < t->promoted_received) { + /* say so once, not every time the feed changes */ + if (strcmp(t->tried, it->hash)) { + lws_strncpy(t->tried, it->hash, + sizeof(t->tried)); + lwsl_user("%s: %s: not promoting %.12s (event " + "%s) to %s: it's older than event " + "%s, already promoted\n", __func__, + it->project, it->hash, it->uuid, dst, + t->promoted_uuid); + } + continue; + } + + if (!strcmp(t->tried, it->hash) && + now - t->tried_at < (lws_usec_t)SAIP_RETRY_S * + LWS_US_PER_SEC) + continue; + + lws_strncpy(t->tried, it->hash, sizeof(t->tried)); + t->tried_at = now; + + saip_job_queue(w, t, it, r->force); + + } lws_end_foreach_dll(p); +} diff --git a/src/push/pu-git.c b/src/push/pu-git.c new file mode 100644 index 0000000..daaf802 --- /dev/null +++ b/src/push/pu-git.c @@ -0,0 +1,505 @@ +/* + * Sai push - src/push/pu-git.c + * + * Copyright (C) 2026 Andy Green <andy@warmcat.com> + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation: + * version 2.1 of the License. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, + * MA 02110-1301 USA + * + * Promotions run as a series of git commands on a bare repo per project in + * the repo cache, one command at a time, one promotion at a time: jobs are + * rare, and this way nothing else ever touches a cache repo while git is. + * + * - git init --bare (does nothing if it's already there) + * - fetch all the primary remote's branches + * - the hash must still be on the feed branch, or it was rewritten since: + * then it's not what the feed branch says is good any more + * + * then for the primary remote, and each mirror in turn: + * + * - fetch the remote's target branch (the primary's came with the rest) + * - if the remote's target branch already has the hash, skip it: this also + * stops an older success rewinding a target that moved on + * - push the hash to the remote's target branch, forced if the rule says + * + * If the primary fails, the mirrors are left alone. If a mirror fails, the + * others are still done, and it's tried again later. + * + * The commands are given argv directly, without a shell, and everything in + * them from the feed was checked in saip_feed_process(). A remote with a + * token file gets it by git running us as its GIT_ASKPASS, so the token is + * never in an argv, a url or a log. + */ + +#include <libwebsockets.h> +#include <string.h> +#include <sys/wait.h> +#include <unistd.h> + +#include "pu-private.h" + +static const char * const step_names[] = { + "init", "fetch", "check feed branch", "fetch target branch", + "check target branch", "push", "done" +}; + +static void +saip_job_run(lws_sorted_usec_list_t *sul); + +static void +saip_job_kick(void) +{ + lws_sul_schedule(saip.cx, 0, &saip.sul_job, saip_job_run, 1); +} + +/* + * Remote 0 is the watch's primary, 1.. its mirrors in conf order + */ +static const char * +saip_job_remote(saip_job_t *j, int idx, const char **token_file) +{ + int n = 1; + + *token_file = NULL; + if (!idx) + return j->w->remote; + + lws_start_foreach_dll(struct lws_dll2 *, p, j->w->mirrors.head) { + saip_mirror_t *m = lws_container_of(p, saip_mirror_t, list); + + if (n++ == idx) { + *token_file = m->token_file; + return m->url; + } + } lws_end_foreach_dll(p); + + return NULL; +} + +void +saip_job_queue(saip_watch_t *w, saip_target_t *t, const sai_feed_item_t *it, + int force) +{ + saip_job_t *j; + + lws_start_foreach_dll(struct lws_dll2 *, p, saip.jobs.head) { + j = lws_container_of(p, saip_job_t, list); + if (j->t == t && !strcmp(j->hash, it->hash)) + return; /* already on it */ + } lws_end_foreach_dll(p); + + j = malloc(sizeof(*j)); + if (!j) { + lwsl_err("%s: OOM\n", __func__); + return; + } + memset(j, 0, sizeof(*j)); + + j->w = w; + j->t = t; + j->force = force; + j->received = it->received; + lws_strncpy(j->project, it->project, sizeof(j->project)); + lws_strncpy(j->src, it->branch, sizeof(j->src)); + lws_strncpy(j->dst, t->branch, sizeof(j->dst)); + lws_strncpy(j->hash, it->hash, sizeof(j->hash)); + lws_strncpy(j->uuid, it->uuid, sizeof(j->uuid)); + lws_snprintf(j->repo, sizeof(j->repo), "%s/%s.git", + saip.conf->repo_cache, j->project); + + lwsl_user("%s: %s: %.12s succeeded on %s, promoting to %s%s\n", + __func__, j->project, j->hash, j->src, j->dst, + j->force ? " (force)" : ""); + + lws_dll2_add_tail(&j->list, &saip.jobs); + if (saip.jobs.count == 1) + saip_job_kick(); +} + +static void +saip_job_log_line(saip_job_t *j) +{ + const char *tf, *r; + + j->line[j->line_len] = '\0'; + if (j->line_len) { + r = j->step >= SAIP_STEP_R_FETCH_DST ? + saip_job_remote(j, j->remote, &tf) : NULL; + lwsl_notice("%s: %s %s%s%s: git: %s\n", __func__, j->project, + step_names[j->step], r ? " " : "", r ? r : "", + j->line); + } + j->line_len = 0; +} + +/* + * The primary has the hash on the target now, whether we pushed it or it was + * already there: remember this event as the last one promoted, so no event + * older than it is ever promoted over it + */ +static void +saip_job_promoted(saip_job_t *j) +{ + saip_target_t *t = j->t; + + if (j->received < t->promoted_received) + return; + + t->promoted_received = j->received; + lws_strncpy(t->promoted_hash, j->hash, sizeof(t->promoted_hash)); + lws_strncpy(t->promoted_uuid, j->uuid, sizeof(t->promoted_uuid)); + saip_state_save(); +} + +/* on to the next remote, or finished */ + +static void +saip_job_next_remote(saip_job_t *j) +{ + const char *tf; + + j->remote++; + j->dst_fetched = 0; + + if (!saip_job_remote(j, j->remote, &tf)) { + if (!j->failed) + lws_strncpy(j->t->pushed, j->hash, + sizeof(j->t->pushed)); + j->step = SAIP_STEP_DONE; + + return; + } + + j->step = SAIP_STEP_R_FETCH_DST; +} + +static void +saip_git_reap(void *opaque, const lws_spawn_resource_us_t *res, siginfo_t *si, + int we_killed_him) +{ + saip_job_t *j = (saip_job_t *)opaque; + const char *tf, *r; + int code = -1; + + if (!j) + return; + + saip_job_log_line(j); + + if (si && si->si_code == CLD_EXITED) + code = si->si_status; + + if (we_killed_him) + lwsl_err("%s: %s: git %s timed out\n", __func__, j->project, + step_names[j->step]); + + r = saip_job_remote(j, j->remote, &tf); + + switch (j->step) { + case SAIP_STEP_ON_SRC: + if (code == 1) { + lwsl_user("%s: %s: %.12s is no longer on %s, not " + "promoting it\n", __func__, j->project, + j->hash, j->src); + j->step = SAIP_STEP_DONE; + goto next; + } + if (!code) { + /* remote 0's target came with the primary's fetch */ + j->remote = 0; + j->dst_fetched = 1; + j->step = SAIP_STEP_R_IN_DST; + goto next; + } + break; + + case SAIP_STEP_R_FETCH_DST: + /* + * Failing here is usually the target branch not existing on + * the remote yet. Either way, don't check against a stale + * copy from before: the push will say if there's a problem. + */ + j->dst_fetched = !code; + j->step = j->dst_fetched ? SAIP_STEP_R_IN_DST : + SAIP_STEP_R_PUSH; + goto next; + + case SAIP_STEP_R_IN_DST: + if (!code) { + lwsl_user("%s: %s: %s on %s already has %.12s\n", + __func__, j->project, j->dst, r, j->hash); + if (!j->remote) + saip_job_promoted(j); + saip_job_next_remote(j); + goto next; + } + /* + * Anything else, including the target branch not existing + * yet, means pushing it is up to us + */ + j->step = SAIP_STEP_R_PUSH; + goto next; + + case SAIP_STEP_R_PUSH: + if (!code) { + lwsl_user("%s: %s: pushed %.12s to %s on %s%s\n", + __func__, j->project, j->hash, j->dst, r, + j->force ? " (force)" : ""); + if (!j->remote) + saip_job_promoted(j); + saip_job_next_remote(j); + goto next; + } + + if (j->remote) { + /* a mirror failed: still do the others */ + lwsl_err("%s: %s: pushing %.12s to %s on mirror %s " + "failed (exit %d)\n", __func__, j->project, + j->hash, j->dst, r, code); + j->failed = 1; + saip_job_next_remote(j); + goto next; + } + break; + + default: + break; + } + + if (code) { + /* + * We'll try again when the feed next changes or the long poll + * times out, after SAIP_RETRY_S + */ + lwsl_err("%s: %s: promoting %.12s to %s failed at git %s " + "(exit %d)\n", __func__, j->project, j->hash, j->dst, + step_names[j->step], code); + j->step = SAIP_STEP_DONE; + goto next; + } + + j->step++; + +next: + /* we're inside lws spawn's reap here, start the next one after */ + saip_job_kick(); +} + +static int +saip_job_spawn(saip_job_t *j) +{ + char home[300], askpass[300], tokenfile[300], url[512], refspec[160], + tracking[160]; + const char *env[8], *argv[16], *prefix, *tf; + struct lws_spawn_piped_info info; + int n = 0, e = 0; + + prefix = saip_job_remote(j, j->step >= SAIP_STEP_R_FETCH_DST ? + j->remote : 0, &tf); + if (!prefix) + return 1; + lws_snprintf(url, sizeof(url), "%s%s", prefix, j->project); + + lws_snprintf(home, sizeof(home), "HOME=%s", saip.home); + env[e++] = home; + env[e++] = "PATH=/usr/local/bin:/usr/bin:/bin"; + env[e++] = "LANG=C"; + /* never wait for someone to type anything */ + env[e++] = "GIT_TERMINAL_PROMPT=0"; + env[e++] = "GIT_SSH_COMMAND=ssh -o BatchMode=yes"; + + argv[n++] = "git"; + + if (tf) { + /* + * git runs us to answer its username / password prompts, and + * we answer from the token file, see saip_askpass(). Clear + * any credential helpers the user's git config names, so + * nothing but that is asked, and nothing is stored. + */ + lws_snprintf(askpass, sizeof(askpass), "GIT_ASKPASS=%s", + saip.self); + lws_snprintf(tokenfile, sizeof(tokenfile), + "SAI_PUSH_TOKEN_FILE=%s", tf); + env[e++] = askpass; + env[e++] = tokenfile; + argv[n++] = "-c"; + argv[n++] = "credential.helper="; + } + env[e] = NULL; + + switch (j->step) { + case SAIP_STEP_INIT: + argv[n++] = "init"; + argv[n++] = "--bare"; + argv[n++] = "-q"; + argv[n++] = j->repo; + break; + + case SAIP_STEP_FETCH: + argv[n++] = "-C"; + argv[n++] = j->repo; + argv[n++] = "fetch"; + argv[n++] = "-q"; + argv[n++] = "--prune"; + argv[n++] = "--no-tags"; + argv[n++] = url; + argv[n++] = "+refs/heads/*:refs/sai-push/heads/*"; + break; + + case SAIP_STEP_R_FETCH_DST: + lws_snprintf(refspec, sizeof(refspec), + "+refs/heads/%s:refs/sai-push/mirror%d/%s", + j->dst, j->remote, j->dst); + argv[n++] = "-C"; + argv[n++] = j->repo; + argv[n++] = "fetch"; + argv[n++] = "-q"; + argv[n++] = "--no-tags"; + argv[n++] = url; + argv[n++] = refspec; + break; + + case SAIP_STEP_ON_SRC: + case SAIP_STEP_R_IN_DST: + if (j->step == SAIP_STEP_ON_SRC) + lws_snprintf(tracking, sizeof(tracking), + "refs/sai-push/heads/%s", j->src); + else if (!j->remote) + lws_snprintf(tracking, sizeof(tracking), + "refs/sai-push/heads/%s", j->dst); + else + lws_snprintf(tracking, sizeof(tracking), + "refs/sai-push/mirror%d/%s", j->remote, + j->dst); + argv[n++] = "-C"; + argv[n++] = j->repo; + argv[n++] = "merge-base"; + argv[n++] = "--is-ancestor"; + argv[n++] = j->hash; + argv[n++] = tracking; + break; + + case SAIP_STEP_R_PUSH: + lws_snprintf(refspec, sizeof(refspec), "%s%s:refs/heads/%s", + j->force ? "+" : "", j->hash, j->dst); + argv[n++] = "-C"; + argv[n++] = j->repo; + argv[n++] = "push"; + argv[n++] = "-q"; + argv[n++] = url; + argv[n++] = refspec; + break; + + default: + return 1; + } + + argv[n] = NULL; + + memset(&info, 0, sizeof(info)); + info.vh = saip.vh; + info.exec_array = argv; + info.env_array = env; + info.protocol_name = protocol_saip_git.name; + info.max_log_lines = 100; + info.timeout_us = (lws_usec_t)SAIP_GIT_TIMEOUT_S * + LWS_US_PER_SEC; + info.reap_cb = saip_git_reap; + info.opaque = j; + info.plsp = &saip.lsp; + + lwsl_info("%s: %s: git %s (%s)\n", __func__, j->project, + step_names[j->step], prefix); + + /* lws_spawn_piped() copies what it needs before returning */ + if (!lws_spawn_piped(&info)) { + lwsl_err("%s: %s: unable to spawn git\n", __func__, j->project); + return 1; + } + + return 0; +} + +static void +saip_job_run(lws_sorted_usec_list_t *sul) +{ + saip_job_t *j; + + while (saip.jobs.head) { + j = lws_container_of(saip.jobs.head, saip_job_t, list); + + if (j->step != SAIP_STEP_DONE) { + if (!saip_job_spawn(j)) + return; + /* couldn't even start it, give up on this one */ + } + + lws_dll2_remove(&j->list); + free(j); + } +} + +static int +callback_saip_git(struct lws *wsi, enum lws_callback_reasons reason, + void *user, void *in, size_t len) +{ + saip_job_t *j = (saip_job_t *)lws_get_opaque_user_data(wsi); + char buf[512]; + ssize_t n, m; + + switch (reason) { + case LWS_CALLBACK_RAW_RX_FILE: + n = read((int)(intptr_t)lws_get_socket_fd(wsi), buf, + sizeof(buf)); + if (n < 1) + return -1; + if (!j) + break; + + /* log what git says a line at a time */ + for (m = 0; m < n; m++) { + if (buf[m] == '\n' || buf[m] == '\r' || + j->line_len == sizeof(j->line) - 1) { + saip_job_log_line(j); + if (buf[m] == '\n' || buf[m] == '\r') + continue; + } + j->line[j->line_len++] = buf[m]; + } + break; + + case LWS_CALLBACK_RAW_CLOSE_FILE: + if (j) + saip_job_log_line(j); + /* + * lws spawn reaps the child when its last stdwsi closes, which + * calls saip_git_reap() and destroys the lsp + */ + if (saip.lsp) + lws_spawn_stdwsi_closed(saip.lsp, wsi); + break; + + default: + break; + } + + return 0; +} + +const struct lws_protocols protocol_saip_git = { + .name = "sai-push-git", + .callback = callback_saip_git, +}; diff --git a/src/push/pu-private.h b/src/push/pu-private.h new file mode 100644 index 0000000..aec3c9e --- /dev/null +++ b/src/push/pu-private.h @@ -0,0 +1,209 @@ +/* + * Sai push definitions src/push/pu-private.h + * + * Copyright (C) 2026 Andy Green <andy@warmcat.com> + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation: + * version 2.1 of the License. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, + * MA 02110-1301 USA + */ + +#include "../common/include/private.h" + +/* + * How long we ask sai-web to hold a request for (it caps it at 600 anyway) + */ +#define SAIP_WAIT_S 600 + +/* + * sai-web sends a keepalive newline every 10s on a held request, so this long + * without anything arriving means the connection is dead + */ +#define SAIP_RX_TIMEOUT_S 45 + +/* don't retry pushing the same hash to the same branch sooner than this */ +#define SAIP_RETRY_S 600 + +/* longest one git step may take before it's killed */ +#define SAIP_GIT_TIMEOUT_S 600 + +/* + * One promotion rule: a feed branch matching "match" (an lws_strcmp_wildcard + * pattern, default "*") and ending in branch_suffix is promoted to the + * branch named without the suffix, with a force push if force. The first + * matching rule in the watch decides. + */ + +typedef struct saip_rule { + lws_dll2_t list; + const char *match; + const char *branch_suffix; + int force; +} saip_rule_t; + +/* + * A mirror of the watched repo: whatever we push to the primary remote, we + * also push to the same branch here. The project name is appended to url. + * If token_file is given, it holds a token git gives as the password when + * the (https) remote asks, eg, a github fine-grained access token. + */ + +typedef struct saip_mirror { + lws_dll2_t list; + const char *url; + const char *token_file; +} saip_mirror_t; + +/* + * What we know about promoting to one target branch of a watch + */ + +typedef struct saip_target { + lws_dll2_t list; + char branch[65]; + char pushed[65]; /* hash we last pushed everywhere */ + char tried[65]; /* hash we last tried */ + lws_usec_t tried_at; + + /* + * The last event promoted to this branch on the primary, kept in the + * state file across restarts: an event whose notification arrived + * before this one's is never promoted over it + */ + uint64_t promoted_received; + char promoted_hash[65]; + char promoted_uuid[65]; +} saip_target_t; + +typedef struct saip_watch { + lws_dll2_t list; + + /* from the conf */ + + const char *feed; /* sai rss.xml or rss.json url */ + const char *fetchurl; /* events must have this */ + const char *remote; /* project name is appended */ + lws_dll2_owner_t rules; /* saip_rule_t */ + lws_dll2_owner_t mirrors; /* saip_mirror_t */ + + /* runtime */ + + char host[128]; + char path[256]; /* rss.json path, from '/' */ + int port; + int tls; + + struct lws *wsi; + lws_sorted_usec_list_t sul; /* next feed request */ + uint16_t retry_count; + + struct lejp_ctx ctx; + lws_struct_args_t a; /* the feed being parsed */ + int http_status; + char index[33]; /* from the last feed */ + + lws_dll2_owner_t targets; /* saip_target_t */ + + uint8_t parse_done:1; + uint8_t parse_failed:1; + uint8_t handled:1; +} saip_watch_t; + +enum { + SAIP_STEP_INIT, /* git init --bare, idempotent */ + SAIP_STEP_FETCH, /* fetch the primary remote's branches */ + SAIP_STEP_ON_SRC, /* the hash must be on the feed branch */ + + /* then these for the primary, and each mirror in turn */ + + SAIP_STEP_R_FETCH_DST, /* fetch the remote's target branch */ + SAIP_STEP_R_IN_DST, /* if the remote's target has it, skip it */ + SAIP_STEP_R_PUSH, /* push the hash to the remote's target */ + + SAIP_STEP_DONE +}; + +/* + * One promotion, run as a series of git commands, one at a time. Remote 0 + * is the watch's primary remote, 1.. are its mirrors in order. + */ + +typedef struct saip_job { + lws_dll2_t list; + saip_watch_t *w; + saip_target_t *t; + char project[65]; + char src[65]; /* feed branch, eg main-dev */ + char dst[65]; /* target branch, eg main */ + char hash[65]; + char uuid[65]; /* the event */ + uint64_t received; /* the event's notification */ + char repo[512]; /* bare repo in the cache */ + int force; + int step; + int remote; /* 0 primary, 1.. mirrors */ + uint8_t dst_fetched:1; /* remote's target is fetched */ + uint8_t failed:1; /* a mirror failed */ + char line[256]; /* partial output line */ + size_t line_len; +} saip_job_t; + +/* the conf, as parsed into its lwsac */ + +typedef struct saip_conf { + const char *user; + const char *repo_cache; + lws_dll2_owner_t watches; /* saip_watch_t */ +} saip_conf_t; + +typedef struct saip { + struct lws_context *cx; + struct lws_vhost *vh; + struct lwsac *ac_conf; + saip_conf_t *conf; + + char home[256]; /* for the git child env */ + char self[256]; /* our own path, for GIT_ASKPASS */ + lws_dll2_owner_t jobs; /* saip_job_t, head is running */ + struct lws_spawn_piped *lsp; + lws_sorted_usec_list_t sul_job; +} saip_t; + +extern saip_t saip; +extern const struct lws_protocols protocol_saip_feed, protocol_saip_git; + +int +saip_conf_load(const char *path); + +void +saip_feed_start(saip_watch_t *w); + +void +saip_feed_process(saip_watch_t *w, sai_feed_t *f); + +void +saip_job_queue(saip_watch_t *w, saip_target_t *t, const sai_feed_item_t *it, + int force); + +saip_target_t * +saip_target_get(saip_watch_t *w, const char *branch); + +int +saip_state_load(void); + +void +saip_state_save(void); + +int +saip_askpass(const char *prompt); diff --git a/src/push/pu-sai.c b/src/push/pu-sai.c new file mode 100644 index 0000000..3c80aeb --- /dev/null +++ b/src/push/pu-sai.c @@ -0,0 +1,821 @@ +/* + * Sai push - src/push/pu-sai.c + * + * Copyright (C) 2026 Andy Green <andy@warmcat.com> + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation: + * version 2.1 of the License. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, + * MA 02110-1301 USA + * + * sai-push follows sai-web feeds of events (the JSON form of the rss feed, + * using its long poll), and when an event for a watched repository succeeds + * on a branch a rule matches, eg, main-dev, it pushes that commit on to the + * branch the rule maps it to, eg, main. + * + * It's started as root, sets up its repo cache dir, then becomes the conf's + * "user" before it does anything on the network. That user is one that + * exists just for this, whose ssh keys the git server accepts for pushing to + * the repos it should manage, and whose known_hosts already has the git + * server's host key. + * + * Mirrors get the same pushes as the primary remote. A mirror reached by + * https with a token (eg, github with a fine-grained access token limited to + * the mirror repos) names a token file: git runs sai-push itself as its + * GIT_ASKPASS to read it, see saip_askpass(). + */ + +#include <libwebsockets.h> +#include <string.h> +#include <signal.h> +#include <fcntl.h> +#include <errno.h> +#include <unistd.h> +#include <pwd.h> +#include <limits.h> +#include <stdlib.h> +#include <sys/stat.h> + +#include "pu-private.h" + +saip_t saip; +static int interrupted; + +static const struct lws_protocols *pprotocols[] = { + &protocol_saip_feed, + &protocol_saip_git, + NULL +}; + +/* + * We read the JSON conf using lws_struct... instrument the related structures + */ + +static const lws_struct_map_t lsm_saip_rule[] = { + LSM_STRING_PTR (saip_rule_t, match, "match"), + LSM_STRING_PTR (saip_rule_t, branch_suffix, "branch-suffix"), + LSM_BOOLEAN (saip_rule_t, force, "force"), +}; + +static const lws_struct_map_t lsm_saip_mirror[] = { + LSM_STRING_PTR (saip_mirror_t, url, "url"), + LSM_STRING_PTR (saip_mirror_t, token_file, "token-file"), +}; + +static const lws_struct_map_t lsm_saip_watch[] = { + LSM_STRING_PTR (saip_watch_t, feed, "feed"), + LSM_STRING_PTR (saip_watch_t, fetchurl, "fetchurl"), + LSM_STRING_PTR (saip_watch_t, remote, "remote"), + LSM_LIST (saip_watch_t, rules, saip_rule_t, list, + NULL, lsm_saip_rule, "rules"), + LSM_LIST (saip_watch_t, mirrors, saip_mirror_t, list, + NULL, lsm_saip_mirror, "mirrors"), +}; + +static const lws_struct_map_t lsm_saip[] = { + LSM_STRING_PTR (saip_conf_t, user, "user"), + LSM_STRING_PTR (saip_conf_t, repo_cache, "repo-cache"), + LSM_LIST (saip_conf_t, watches, saip_watch_t, list, + NULL, lsm_saip_watch, "watches"), +}; + +static const lws_struct_map_t lsm_saip_schema[] = { + LSM_SCHEMA (saip_conf_t, NULL, lsm_saip, "sai-push"), +}; + +/* + * Work out where to connect for the watch's feed. It may be given as the + * rss.xml url people know, but we want the same feed as JSON. Only loopback + * feeds may be plain http: we act on what the feed says, so it must come + * from the sai we think it does. + */ +static int +saip_watch_feed_url(saip_watch_t *w) +{ + lws_parse_uri_t *u = lws_parse_uri_create(w->feed); + size_t l; + int bad; + + if (!u || !u->host[0] || u->unix_skt) { + lwsl_err("%s: can't parse feed url %s\n", __func__, w->feed); + lws_parse_uri_destroy(&u); + return 1; + } + + w->tls = !strcmp(u->scheme, "https"); + bad = !w->tls && (strcmp(u->scheme, "http") || + (strcmp(u->host, "localhost") && + strcmp(u->host, "127.0.0.1") && + strcmp(u->host, "::1"))); + + lws_strncpy(w->host, u->host, sizeof(w->host)); + w->port = u->port; + lws_snprintf(w->path, sizeof(w->path), "/%s", u->path); + lws_parse_uri_destroy(&u); + + if (bad) { + lwsl_err("%s: feed %s must be https\n", __func__, w->feed); + return 1; + } + + l = strlen(w->path); + if (l > 7 && !strcmp(w->path + l - 7, "rss.xml")) + /* the same feed, as JSON */ + memcpy(w->path + l - 7, "rss.json", 9); + else if (l < 8 || strcmp(w->path + l - 8, "rss.json")) { + lwsl_err("%s: feed %s should end in rss.xml or rss.json\n", + __func__, w->feed); + return 1; + } + + return 0; +} + +/* + * Credentials go in a token file, never in the url, which gets logged + */ +static int +saip_url_has_credentials(const char *url) +{ + const char *p = strstr(url, "://"), *at, *sl; + + if (!p || (strncmp(url, "https://", 8) && strncmp(url, "http://", 7))) + /* eg, git@host: for ssh, the user is not a secret */ + return 0; + + p += 3; + at = strchr(p, '@'); + sl = strchr(p, '/'); + + return at && (!sl || at < sl); +} + +static int +saip_mirror_check(saip_mirror_t *m) +{ + struct stat s; + + if (!m->url || !m->url[0]) { + lwsl_err("%s: each mirror needs a \"url\"\n", __func__); + return 1; + } + + if (saip_url_has_credentials(m->url)) { + /* not logging the url, it has the secret in it */ + lwsl_err("%s: a mirror url has credentials in it: put them " + "in a \"token-file\"\n", __func__); + return 1; + } + + if (!m->token_file) + return 0; + + if (m->token_file[0] != '/') { + lwsl_err("%s: mirror %s: \"token-file\" must be an absolute " + "path\n", __func__, m->url); + return 1; + } + + if (stat(m->token_file, &s)) { + lwsl_err("%s: mirror %s: can't stat %s: %s\n", __func__, + m->url, m->token_file, strerror(errno)); + return 1; + } + + if (s.st_mode & 0007) { + lwsl_err("%s: %s is readable by anyone, it should be readable " + "only by the sai-push user\n", __func__, + m->token_file); + return 1; + } + + return 0; +} + +/* + * lws_struct silently skips members its maps don't know, and a value of the + * wrong type, eg, "force": "true", just doesn't set the member. So a member + * put in the wrong place, misspelled or given the wrong kind of value, would + * leave part of the conf quietly unused. Before parsing the conf for real, + * walk it once checking every member against this list of where each one + * belongs and what kind of value it takes. + */ + +enum { + SAIP_CT_STR, + SAIP_CT_BOOL, + SAIP_CT_LIST, + SAIP_CT_OBJ, + SAIP_CT_OTHER /* numbers and null: nothing in the conf takes them */ +}; + +static const char * const saip_ct_names[] = { + "a string", "true or false", "a list, [ ... ]", "an object, { ... }", + "a number or null" +}; + +static const struct saip_conf_member { + const char *path; + uint8_t type; +} saip_conf_members[] = { + { "schema", SAIP_CT_STR }, + { "user", SAIP_CT_STR }, + { "repo-cache", SAIP_CT_STR }, + { "watches", SAIP_CT_LIST }, + { "watches[]", SAIP_CT_OBJ }, + { "watches[].feed", SAIP_CT_STR }, + { "watches[].fetchurl", SAIP_CT_STR }, + { "watches[].remote", SAIP_CT_STR }, + { "watches[].rules", SAIP_CT_LIST }, + { "watches[].rules[]", SAIP_CT_OBJ }, + { "watches[].rules[].match", SAIP_CT_STR }, + { "watches[].rules[].branch-suffix", SAIP_CT_STR }, + { "watches[].rules[].force", SAIP_CT_BOOL }, + { "watches[].mirrors", SAIP_CT_LIST }, + { "watches[].mirrors[]", SAIP_CT_OBJ }, + { "watches[].mirrors[].url", SAIP_CT_STR }, + { "watches[].mirrors[].token-file", SAIP_CT_STR }, +}; + +typedef struct saip_conf_check { + int problems; + uint8_t schema_seen; +} saip_conf_check_t; + +static const struct saip_conf_member * +saip_conf_member(const char *path) +{ + size_t n; + + for (n = 0; n < LWS_ARRAY_SIZE(saip_conf_members); n++) + if (!strcmp(saip_conf_members[n].path, path)) + return &saip_conf_members[n]; + + return NULL; +} + +static const char * +saip_last_name(const char *path) +{ + const char *p = strrchr(path, '.'); + + return p ? p + 1 : path; +} + +static signed char +saip_conf_check_cb(struct lejp_ctx *ctx, char reason) +{ + saip_conf_check_t *cc = (saip_conf_check_t *)ctx->user; + const struct saip_conf_member *m; + char path[sizeof(ctx->path)], *dot; + const char *name; + size_t l, n; + int got; + + lws_strncpy(path, ctx->path, sizeof(path)); + + switch (reason) { + case LEJPCB_PAIR_NAME: + if (saip_conf_member(path)) + return 0; + + /* + * Only complain about the outermost unknown member, not + * everything inside it as well + */ + dot = strrchr(path, '.'); + if (dot) { + *dot = '\0'; + if (!saip_conf_member(path)) + return 0; + } + + name = saip_last_name(ctx->path); + cc->problems++; + + /* is it a real member, just in the wrong place? */ + for (n = 0; n < LWS_ARRAY_SIZE(saip_conf_members); n++) + if (!strcmp(saip_last_name(saip_conf_members[n].path), + name)) { + lwsl_err("%s: line %d: \"%s\" doesn't belong " + "there (%s), it goes in %s\n", __func__, + ctx->line, name, ctx->path, + saip_conf_members[n].path); + return 0; + } + + lwsl_err("%s: line %d: unknown member \"%s\" (%s)\n", __func__, + ctx->line, name, ctx->path); + return 0; + + case LEJPCB_VAL_STR_END: + got = SAIP_CT_STR; + if (!strcmp(path, "schema")) { + cc->schema_seen = 1; + if (strcmp(ctx->buf, "sai-push")) { + lwsl_err("%s: line %d: \"schema\" must be " + "\"sai-push\"\n", __func__, ctx->line); + cc->problems++; + } + } + break; + + case LEJPCB_VAL_TRUE: + case LEJPCB_VAL_FALSE: + got = SAIP_CT_BOOL; + break; + + case LEJPCB_VAL_NUM_INT: + case LEJPCB_VAL_NUM_FLOAT: + case LEJPCB_VAL_NULL: + got = SAIP_CT_OTHER; + break; + + case LEJPCB_ARRAY_START: + /* the path already has the [] of the list's elements */ + l = strlen(path); + if (l >= 2 && !strcmp(path + l - 2, "[]")) + path[l - 2] = '\0'; + got = SAIP_CT_LIST; + break; + + case LEJPCB_OBJECT_START: + if (!path[0]) + return 0; /* the conf itself */ + got = SAIP_CT_OBJ; + break; + + default: + return 0; + } + + m = saip_conf_member(path); + if (!m || m->type == got) + /* if it's unknown, we already said so at its name */ + return 0; + + lwsl_err("%s: line %d: \"%s\" (%s) should be %s, not %s\n", __func__, + ctx->line, saip_last_name(path), path, + saip_ct_names[m->type], saip_ct_names[got]); + cc->problems++; + + return 0; +} + +static int +saip_conf_check(const char *path) +{ + saip_conf_check_t cc; + unsigned char buf[512]; + struct lejp_ctx ctx; + int n, m = LEJP_CONTINUE, fd; + + memset(&cc, 0, sizeof(cc)); + + fd = lws_open(path, O_RDONLY); + if (fd < 0) { + lwsl_err("%s: cannot open %s\n", __func__, path); + return 1; + } + + lejp_construct(&ctx, saip_conf_check_cb, &cc, NULL, 0); + sai_lejp_enable_comments(&ctx); + + do { + n = (int)read(fd, buf, sizeof(buf)); + if (n <= 0) + break; + m = lejp_parse(&ctx, buf, n); + } while (m == LEJP_CONTINUE); + + close(fd); + + if (m < 0) { + lwsl_err("%s: %s line %d: not valid JSON: %s\n", __func__, path, + ctx.line, lejp_error_to_string(m)); + cc.problems++; + } else if (!cc.schema_seen) { + lwsl_err("%s: %s needs \"schema\": \"sai-push\"\n", __func__, + path); + cc.problems++; + } + + lejp_destruct(&ctx); + + if (cc.problems) + lwsl_err("%s: %d problem%s in %s, not starting\n", __func__, + cc.problems, cc.problems == 1 ? "" : "s", path); + + return !!cc.problems; +} + +static int +saip_url_is_http(const char *url) +{ + return !strncmp(url, "https://", 8) || !strncmp(url, "http://", 7); +} + +int +saip_conf_load(const char *path) +{ + unsigned char buf[512]; + lws_struct_args_t a; + struct lejp_ctx ctx; + int n, m = LEJP_CONTINUE, fd; + saip_conf_t *c; + + if (saip_conf_check(path)) + return 1; + + memset(&a, 0, sizeof(a)); + a.map_st[0] = lsm_saip_schema; + a.map_entries_st[0] = LWS_ARRAY_SIZE(lsm_saip_schema); + a.ac_block_size = 1024; + + fd = lws_open(path, O_RDONLY); + if (fd < 0) { + lwsl_err("%s: cannot open %s\n", __func__, path); + return 1; + } + + lws_struct_json_init_parse(&ctx, NULL, &a); + sai_lejp_enable_comments(&ctx); + + do { + n = (int)read(fd, buf, sizeof(buf)); + if (n <= 0) + break; + m = lejp_parse(&ctx, buf, n); + } while (m == LEJP_CONTINUE); + + close(fd); + + if (m < 0 || !a.dest) { + lwsl_err("%s: %s line %d: JSON decode failed '%s'\n", __func__, + path, ctx.line, lejp_error_to_string(m)); + lejp_destruct(&ctx); + goto bail; + } + lejp_destruct(&ctx); + + c = (saip_conf_t *)a.dest; + + if (!c->repo_cache || c->repo_cache[0] != '/') { + lwsl_err("%s: \"repo-cache\" must be an absolute path\n", + __func__); + goto bail; + } + + if (!c->watches.count) { + lwsl_err("%s: no \"watches\"\n", __func__); + goto bail; + } + + lws_start_foreach_dll(struct lws_dll2 *, p, c->watches.head) { + saip_watch_t *w = lws_container_of(p, saip_watch_t, list); + + if (!w->feed || !w->fetchurl || !w->remote || + !w->rules.count) { + lwsl_err("%s: each watch needs \"feed\", \"fetchurl\", " + "\"remote\" and \"rules\"\n", __func__); + goto bail; + } + + if (saip_watch_feed_url(w)) + goto bail; + + lws_start_foreach_dll(struct lws_dll2 *, q, w->rules.head) { + saip_rule_t *r = lws_container_of(q, saip_rule_t, list); + + if (!r->branch_suffix || !r->branch_suffix[0]) { + lwsl_err("%s: each rule needs a " + "\"branch-suffix\"\n", __func__); + goto bail; + } + if (!r->match) + r->match = "*"; + + lwsl_notice("%s: %s: branches matching '%s' ending " + "'%s' go to the branch without it%s\n", + __func__, w->fetchurl, r->match, + r->branch_suffix, + r->force ? ", force pushed" : ""); + } lws_end_foreach_dll(q); + + if (saip_url_has_credentials(w->remote)) { + /* not logging the url, it has the secret in it */ + lwsl_err("%s: the remote url for %s has credentials in " + "it\n", __func__, w->fetchurl); + goto bail; + } + + /* + * git won't be asked anything interactively, so over http(s) + * it can only push with credentials it already has from the + * user's own git config, or a mirror's token-file + */ + if (saip_url_is_http(w->remote)) + lwsl_warn("%s: %s: remote %s is http(s), which can't " + "push unless the user's git config has " + "credentials for it: use ssh\n", __func__, + w->fetchurl, w->remote); + + lwsl_notice("%s: %s: pushing to %s\n", __func__, w->fetchurl, + w->remote); + + lws_start_foreach_dll(struct lws_dll2 *, q, w->mirrors.head) { + saip_mirror_t *mi = lws_container_of(q, saip_mirror_t, + list); + + if (saip_mirror_check(mi)) + goto bail; + + if (!mi->token_file && saip_url_is_http(mi->url)) + lwsl_warn("%s: %s: mirror %s is http(s) with no " + "\"token-file\", it can't push unless " + "the user's git config has " + "credentials for it: for github, use " + "ssh and a deploy key\n", __func__, + w->fetchurl, mi->url); + + lwsl_notice("%s: %s: mirrored to %s%s\n", __func__, + w->fetchurl, mi->url, + mi->token_file ? " (with token)" : ""); + } lws_end_foreach_dll(q); + + if (!w->mirrors.count) + lwsl_notice("%s: %s: no mirrors\n", __func__, + w->fetchurl); + + } lws_end_foreach_dll(p); + + /* the parsed conf lives in its lwsac for as long as we run */ + + saip.conf = c; + saip.ac_conf = a.ac; + + return 0; + +bail: + lwsac_free(&a.ac); + + return 1; +} + +/* + * While we're still root: find the user we'll become, and make sure the repo + * cache dir exists and belongs to them + */ +static int +saip_prepare_user(struct lws_context_creation_info *info) +{ + struct passwd *pw; + + if (geteuid()) { + /* eg, testing by hand: we just carry on as whoever we are */ + pw = getpwuid(geteuid()); + if (saip.conf->user && (!pw || strcmp(pw->pw_name, saip.conf->user))) + lwsl_warn("%s: not root, so staying as uid %u, not " + "becoming \"%s\"\n", __func__, + (unsigned int)geteuid(), saip.conf->user); + } else { + if (!saip.conf->user) { + lwsl_err("%s: started as root, the conf must give the " + "\"user\" to run as\n", __func__); + return 1; + } + + pw = getpwnam(saip.conf->user); + if (!pw || !pw->pw_uid) { + lwsl_err("%s: user \"%s\" unknown, or is root\n", + __func__, saip.conf->user); + return 1; + } + + /* lws drops to this in lws_create_context() */ + info->username = saip.conf->user; + } + + if (!pw) { + lwsl_err("%s: can't find our own user\n", __func__); + return 1; + } + + /* git and ssh in the children find config and keys under here */ + lws_strncpy(saip.home, pw->pw_dir, sizeof(saip.home)); + + if (mkdir(saip.conf->repo_cache, 0700) && errno != EEXIST) { + lwsl_err("%s: can't create %s: %s\n", __func__, + saip.conf->repo_cache, strerror(errno)); + return 1; + } + + if (!geteuid() && chown(saip.conf->repo_cache, pw->pw_uid, pw->pw_gid)) { + lwsl_err("%s: can't chown %s: %s\n", __func__, + saip.conf->repo_cache, strerror(errno)); + return 1; + } + + return 0; +} + +static void +sigint_handler(int sig) +{ + interrupted = 1; +} + +/* + * git runs us as its GIT_ASKPASS for a mirror with a token file, with the + * prompt as the argument and SAI_PUSH_TOKEN_FILE in the environment: we + * answer a username prompt with a placeholder (github ignores it for token + * auth) and a password prompt with the token, on stdout. + */ +int +saip_askpass(const char *prompt) +{ + const char *path = getenv("SAI_PUSH_TOKEN_FILE"); + char tok[512], *t = tok; + ssize_t r; + size_t n; + int fd; + + if (!strncmp(prompt, "Username", 8)) { + if (write(1, "x-access-token\n", 15) != 15) + return 1; + return 0; + } + + fd = lws_open(path, O_RDONLY); + if (fd < 0) + return 1; + r = read(fd, tok, sizeof(tok) - 1); + close(fd); + if (r <= 0) + return 1; + n = (size_t)r; /* leaves room for the '\n' */ + + /* just the token, whatever whitespace was around it in the file */ + while (n && (tok[n - 1] == '\n' || tok[n - 1] == '\r' || + tok[n - 1] == ' ' || tok[n - 1] == '\t')) + n--; + while (n && (*t == '\n' || *t == '\r' || *t == ' ' || *t == '\t')) { + t++; + n--; + } + if (!n) + return 1; + t[n++] = '\n'; + + return write(1, t, n) != (ssize_t)n; +} + +static int +saip_any_token_files(void) +{ + lws_start_foreach_dll(struct lws_dll2 *, q, saip.conf->watches.head) { + saip_watch_t *w = lws_container_of(q, saip_watch_t, list); + + lws_start_foreach_dll(struct lws_dll2 *, p, w->mirrors.head) { + if (lws_container_of(p, saip_mirror_t, + list)->token_file) + return 1; + } lws_end_foreach_dll(p); + } lws_end_foreach_dll(q); + + return 0; +} + +/* now we're the push user, it must be able to read its tokens */ + +static int +saip_check_token_access(void) +{ + lws_start_foreach_dll(struct lws_dll2 *, q, saip.conf->watches.head) { + saip_watch_t *w = lws_container_of(q, saip_watch_t, list); + + lws_start_foreach_dll(struct lws_dll2 *, p, w->mirrors.head) { + saip_mirror_t *m = lws_container_of(p, saip_mirror_t, + list); + + if (m->token_file && access(m->token_file, R_OK)) { + lwsl_err("%s: can't read %s as uid %u\n", + __func__, m->token_file, + (unsigned int)getuid()); + return 1; + } + } lws_end_foreach_dll(p); + } lws_end_foreach_dll(q); + + return 0; +} + +int +main(int argc, const char **argv) +{ + const char *p, *conf = "/etc/sai/push/conf"; + struct lws_context_creation_info info; + + /* git asking us for a token: answer and go, before anything else */ + if (argc == 2 && getenv("SAI_PUSH_TOKEN_FILE")) + return saip_askpass(argv[1]); + + memset(&info, 0, sizeof(info)); + lws_cmdline_option_handle_builtin(argc, argv, &info); + + lwsl_user("Sai Push - Copyright (C) 2026 Andy Green <andy@warmcat.com>\n"); + lwsl_user(" sai-push [-c <conf file>]\n"); + + if ((p = lws_cmdline_option(argc, argv, "-c"))) + conf = p; + + if (saip_conf_load(conf)) + return 1; + + if (saip_prepare_user(&info)) + goto bail; + + if (saip_state_load()) + goto bail; + + /* git runs us again as its askpass, it needs our full path */ + if (saip_any_token_files()) { + char rp[PATH_MAX]; + + if ((!realpath("/proc/self/exe", rp) && + !realpath(argv[0], rp)) || + strlen(rp) >= sizeof(saip.self)) { + lwsl_err("%s: can't find our own path for " + "GIT_ASKPASS\n", __func__); + goto bail; + } + lws_strncpy(saip.self, rp, sizeof(saip.self)); + } + + signal(SIGINT, sigint_handler); + signal(SIGTERM, sigint_handler); + + /* + * Not explicit vhosts, so lws drops to info.username inside + * lws_create_context(), before we open any connection + */ + info.port = CONTEXT_PORT_NO_LISTEN; + info.pprotocols = pprotocols; + info.options = LWS_SERVER_OPTION_DO_SSL_GLOBAL_INIT; + /* + * lws' own fds, a feed connection per watch, and the three stdio + * pipes of the one git child at a time + */ + info.fd_limit_per_thread = 32 + (unsigned int)saip.conf->watches.count; + + saip.cx = lws_create_context(&info); + if (!saip.cx) { + lwsl_err("%s: lws init failed\n", __func__); + goto bail; + } + saip.vh = lws_get_vhost_by_name(saip.cx, "default"); + + if (saip_check_token_access()) + goto bail_cx; + + lws_start_foreach_dll(struct lws_dll2 *, q, saip.conf->watches.head) { + saip_feed_start(lws_container_of(q, saip_watch_t, list)); + } lws_end_foreach_dll(q); + + while (!lws_service(saip.cx, 0) && !interrupted) + ; + +bail_cx: + lws_context_destroy(saip.cx); + +bail: + lws_start_foreach_dll_safe(struct lws_dll2 *, q, q1, saip.jobs.head) { + lws_dll2_remove(q); + free(lws_container_of(q, saip_job_t, list)); + } lws_end_foreach_dll_safe(q, q1); + + lws_start_foreach_dll(struct lws_dll2 *, q, saip.conf->watches.head) { + saip_watch_t *w = lws_container_of(q, saip_watch_t, list); + + lwsac_free(&w->a.ac); + lws_start_foreach_dll_safe(struct lws_dll2 *, t, t1, + w->targets.head) { + lws_dll2_remove(t); + free(lws_container_of(t, saip_target_t, list)); + } lws_end_foreach_dll_safe(t, t1); + } lws_end_foreach_dll(q); + + lwsac_free(&saip.ac_conf); + + return 0; +} diff --git a/src/push/pu-state.c b/src/push/pu-state.c new file mode 100644 index 0000000..2b29e83 --- /dev/null +++ b/src/push/pu-state.c @@ -0,0 +1,250 @@ +/* + * Sai push - src/push/pu-state.c + * + * Copyright (C) 2026 Andy Green <andy@warmcat.com> + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation: + * version 2.1 of the License. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, + * MA 02110-1301 USA + * + * For each target branch, the last event promoted to it on the primary + * remote is remembered in <repo-cache>/sai-push-state.json, so that across + * restarts too, an event whose notification arrived before that one's is + * never promoted over it. + */ + +#include <libwebsockets.h> +#include <string.h> +#include <fcntl.h> +#include <errno.h> +#include <unistd.h> + +#include "pu-private.h" + +typedef struct saip_state_entry { + lws_dll2_t list; + char fetchurl[96]; + char branch[65]; + char hash[65]; + char uuid[65]; + uint64_t received; +} saip_state_entry_t; + +typedef struct saip_state { + lws_dll2_owner_t entries; /* saip_state_entry_t */ +} saip_state_t; + +static const lws_struct_map_t lsm_saip_state_entry[] = { + LSM_CARRAY (saip_state_entry_t, fetchurl, "fetchurl"), + LSM_CARRAY (saip_state_entry_t, branch, "branch"), + LSM_CARRAY (saip_state_entry_t, hash, "hash"), + LSM_CARRAY (saip_state_entry_t, uuid, "uuid"), + LSM_UNSIGNED (saip_state_entry_t, received, "received"), +}; + +static const lws_struct_map_t lsm_saip_state[] = { + LSM_LIST (saip_state_t, entries, saip_state_entry_t, list, + NULL, lsm_saip_state_entry, "promoted"), +}; + +static const lws_struct_map_t lsm_saip_state_schema[] = { + LSM_SCHEMA (saip_state_t, NULL, lsm_saip_state, "sai-push-state"), +}; + +static void +saip_state_path(char *buf, size_t len, const char *suffix) +{ + lws_snprintf(buf, len, "%s/sai-push-state.json%s", + saip.conf->repo_cache, suffix); +} + +int +saip_state_load(void) +{ + unsigned char buf[512]; + lws_struct_args_t a; + struct lejp_ctx ctx; + int n, m = LEJP_CONTINUE, fd; + saip_state_t *st; + char path[512]; + + saip_state_path(path, sizeof(path), ""); + + fd = lws_open(path, O_RDONLY); + if (fd < 0) { + if (errno != ENOENT) { + lwsl_err("%s: can't open %s: %s\n", __func__, path, + strerror(errno)); + return 1; + } + + /* first run */ + return 0; + } + + memset(&a, 0, sizeof(a)); + a.map_st[0] = lsm_saip_state_schema; + a.map_entries_st[0] = LWS_ARRAY_SIZE(lsm_saip_state_schema); + a.ac_block_size = 1024; + + lws_struct_json_init_parse(&ctx, NULL, &a); + + do { + n = (int)read(fd, buf, sizeof(buf)); + if (n <= 0) + break; + m = lejp_parse(&ctx, buf, n); + } while (m == LEJP_CONTINUE); + + close(fd); + lejp_destruct(&ctx); + + if (m < 0 || !a.dest) { + /* + * Don't carry on without it: it's what stops us promoting an + * older event over a newer one + */ + lwsl_err("%s: %s is damaged ('%s'), remove it to start over\n", + __func__, path, lejp_error_to_string(m)); + lwsac_free(&a.ac); + + return 1; + } + + st = (saip_state_t *)a.dest; + + lws_start_foreach_dll(struct lws_dll2 *, p, st->entries.head) { + saip_state_entry_t *e = lws_container_of(p, saip_state_entry_t, + list); + saip_target_t *t = NULL; + + lws_start_foreach_dll(struct lws_dll2 *, q, + saip.conf->watches.head) { + saip_watch_t *w = lws_container_of(q, saip_watch_t, + list); + + if (!strcmp(w->fetchurl, e->fetchurl)) + t = saip_target_get(w, e->branch); + } lws_end_foreach_dll(q); + + if (!t) + /* not a watch we have any more, it'll be dropped */ + continue; + + t->promoted_received = e->received; + lws_strncpy(t->promoted_hash, e->hash, + sizeof(t->promoted_hash)); + lws_strncpy(t->promoted_uuid, e->uuid, + sizeof(t->promoted_uuid)); + + lwsl_notice("%s: %s %s: last promoted %.12s, event %s\n", + __func__, e->fetchurl, e->branch, e->hash, + e->uuid); + } lws_end_foreach_dll(p); + + lwsac_free(&a.ac); + + return 0; +} + +/* + * Written to a temp file and renamed over the old one, so there's always a + * whole state file, the old one or the new one + */ +void +saip_state_save(void) +{ + char path[512], tmp[520]; + lws_struct_serialize_t *js; + struct lwsac *ac = NULL; + saip_state_entry_t *e; + uint8_t buf[1024]; + saip_state_t st; + int fd, r, bad = 0; + size_t w; + + memset(&st, 0, sizeof(st)); + + lws_start_foreach_dll(struct lws_dll2 *, q, saip.conf->watches.head) { + saip_watch_t *wa = lws_container_of(q, saip_watch_t, list); + + lws_start_foreach_dll(struct lws_dll2 *, p, wa->targets.head) { + saip_target_t *t = lws_container_of(p, saip_target_t, + list); + + if (!t->promoted_received) + continue; + + e = lwsac_use_zero(&ac, sizeof(*e), 1024); + if (!e) + goto bail; + + lws_strncpy(e->fetchurl, wa->fetchurl, + sizeof(e->fetchurl)); + lws_strncpy(e->branch, t->branch, sizeof(e->branch)); + lws_strncpy(e->hash, t->promoted_hash, + sizeof(e->hash)); + lws_strncpy(e->uuid, t->promoted_uuid, + sizeof(e->uuid)); + e->received = t->promoted_received; + lws_dll2_add_tail(&e->list, &st.entries); + } lws_end_foreach_dll(p); + } lws_end_foreach_dll(q); + + saip_state_path(path, sizeof(path), ""); + saip_state_path(tmp, sizeof(tmp), ".tmp"); + + fd = lws_open(tmp, O_WRONLY | O_CREAT | O_TRUNC, 0600); + if (fd < 0) { + lwsl_err("%s: can't create %s: %s\n", __func__, tmp, + strerror(errno)); + goto bail; + } + + js = lws_struct_json_serialize_create(lsm_saip_state_schema, + LWS_ARRAY_SIZE(lsm_saip_state_schema), 0, &st); + if (!js) { + close(fd); + goto bail_unlink; + } + + do { + w = 0; + r = lws_struct_json_serialize(js, buf, sizeof(buf), &w); + if (r == LSJS_RESULT_ERROR || + (w && write(fd, buf, w) != (ssize_t)w)) + bad = 1; + } while (r == LSJS_RESULT_CONTINUE && !bad); + + lws_struct_json_serialize_destroy(&js); + + if (fsync(fd)) + bad = 1; + if (close(fd)) + bad = 1; + + if (bad || rename(tmp, path)) { + lwsl_err("%s: failed writing %s\n", __func__, path); + goto bail_unlink; + } + + lwsac_free(&ac); + + return; + +bail_unlink: + unlink(tmp); +bail: + lwsac_free(&ac); +}
Page fetched 0s ago, creation time: 8ms (vhost etag hits: 0%, cache hits: 0%)