# Contributing to Noust Noust is a Python 3.10+ CLI for deploying web apps on Linux servers. Automates Nginx/Apache, SSL, systemd, databases and backups; builds every deploy as a health-gated release with instant rollback; and serves an optional browser console (React SPA) over a JSON API. **Repository**: https://github.com/Perkybeet/noust | **License**: AGPL-3.0-or-later (from 2.1.0; releases up to 2.0.x were non-commercial source-available) ### The licence of a contribution Noust is free software under the AGPL-3.0-or-later. By contributing you agree that: 1. your contribution is licensed under the AGPL-3.0-or-later, like the rest of the project; 2. you have the right to submit it, and you say in its description where any part of it comes from when it is not your own, with its licence; 3. you also grant the author, Yago López Prado, a perpetual, irrevocable licence to use your contribution under other terms, so that Noust can also be offered under a separate commercial licence alongside the AGPL, which stays. For a licence on other terms, write to support@noust.dev. --- ## The four rules These are not style preferences. Each one exists because its absence produced a specific class of defect that shipped to users. Breaking one is how the project regresses. ### 1. Nothing runs a process except `CommandRunner` `src/noust/core/runner.py` is the only module allowed to import `subprocess`. Everything that calls nginx, systemctl, certbot, git, npm or a database client goes through it. ```python result = self.runner.run(["systemctl", "restart", unit], timeout=30, check=True) self.runner.stream(["npm", "install"], on_line=logger.substep, timeout=900) self.runner.capture_to_file(["pg_dump", db], destination, compress=True) ``` Argv only, never a shell. Timeouts are mandatory. Secrets go through `env=` or `input=`, never argv, because everything on a command line is visible in `ps` to every local user. `tests/conftest.py` makes real process execution fail in every test, so code that bypasses the runner cannot be tested and fails loudly instead. `tests/test_architecture.py` enforces the import rule. `--dry-run` is implemented here too, as `DryRunRunner`, and for file changes as `DryRunFileSystem` in `core/fs.py`. It is not wired per command: that is what left the flag honoured in three code paths and silently ignored in every destructive one. ### 2. `except Exception` is only allowed at an error boundary, and must log There were 302 of them, 149 silent. That is the mechanism by which five calls to methods that did not exist shipped for entire releases: every `AttributeError` became a cosmetic warning. Catch the specific exception. If you genuinely need a broad catch, it belongs in the CLI or API error boundary and it logs. mypy is the guard for this class of bug and it blocks CI. ### 3. There is one implementation of each thing The version lived in six hand-synchronised files, app-type detection had four implementations with contradicting precedence, and the web API was a second implementation of the whole product. When you find yourself writing something that already exists, use the existing one or move it somewhere both callers can reach. The web layer in particular is a **client** of the managers, never a parallel implementation. An endpoint translates HTTP to a manager call and back. ### 4. Guards go at the chokepoint A rule enforced in the caller is a rule with as many holes as there are callers. Unit ownership is checked in `ServiceManager`, not in the endpoints that use it. Read-only SQL is enforced by the database server, not by a keyword allowlist. Escaping is done by the template engine, not by remembering. --- ## Architecture ``` src/noust/ core/ runner.py the only place processes are executed fs.py the seam every file change goes through (DryRunFileSystem under --dry-run) paths.py every system path, unit name and marker in one place retired_names.py 4.0 refuses a server whose data 3.x never moved (the paths of before 3.0 are spelled here and nowhere else), and retires the links 3.x left (`noust setup retire-links`) legacy_names.py every name from before 4.0 that is still accepted (a header a script sends, a token in a CI secret, a file in a repository), each with why and until when; code asks it for the name, it never spells one legacy_archive.py the one reader of the backups and exports made before 4.0 appdir/ how an application's directory is laid out, below the managers and the deployers that both read it: releases.py (ReleaseManager: the only code that knows releases/, current, shared/), layout.py (which layout, where .env lives) and permissions.py (handing files to the app's account) deploy_events.py the deployment events every deployment publishes and its subscribers store.py SQLite persistence (WAL) with versioned migrations schema_fragments.py what a schema version written one fragment per area shares: apply it, and ask whether a store lacks any of it (v14 and v16, 4.1's, use it; each lane of a release fills its own `_SQL` and `_COLUMNS` and never another's) config.py layered config; secrets written 0600, redacted on the way out exceptions.py NoustError hierarchy, used for real channel_secrets.py the notification channels' credentials in config.yaml, sealed with at_rest.py under a key beside the file cdn_networks.py the networks of Fastly, Bunny and ArvanCloud, so the DNS check names the CDN sealing.py a central's secrets at rest: scrypt, then AES-256-CBC and HMAC-SHA256 through openssl, never a new dependency store_secrets.py which columns of the store hold a secret value and are emptied in a copy (an incident's snapshot, `store export`); a test fails on a column nobody ruled on compose_yaml.py a Compose document read the way Compose reads it (PyYAML takes `2222:22` for a base 60 number and `no` for false); every reader of a Compose file uses it logsafe.py every log record of the process reports its message on one line, so a value that came from a request cannot start another line (installed by importing logger.py) templates.py the one Jinja environment unit files, sites and pools are rendered with outbound.py where a request may go (`require_public_host`) and what its path may be (`path_segment`, `api_path`) timeutil.py the one clock: `utc_now_iso` (to the second or with its fraction, as each column was written), `local_now_iso`, `utc_now`; no module keeps a `_now` app_lookup.py "is there an application here" for what runs as the instance (the console's scoped answer is `web/api/deps.py` `known_app`); the command line's is `cli/lookup.py` web_gate.py the one door to the console's token manager for the layers that cannot import `web`, with the error that says how to install its dependencies validators/ names, environments, sources, ports, domains managers/ adapters: web server, systemd, certs, backups, databases, source, cron diagnose.py why an app is down: read-only probes, most likely cause first health.py the server-wide report behind `noust health` and /api/system/health siteconf/ the one nginx/Apache parser: lossless tree, structured model, edit operations, request routing (rule 3: nothing reads a site with a regex) stack_databases.py a Compose stack's databases: detect, dump, restore name_migration.py what a server still carries from before 4.0, and `noust setup rename`, which finishes the renaming without restarting anything (upstream_names.py is its nginx half) database/managed.py the engines Noust runs itself in containers beside the host's (another version, or an engine the host lacks): make, start, stop, upgrade, remove, from the catalog of official images pinned by digest (instance_catalog.py); once made, an instance is one more key (`postgresql@pg16`) to every database manager port_claims.py who holds a port: applications, the ports a Compose stack publishes, streams, databases, engines, SSH, the console and the kernel; every caller asks it app_identity.py the one answer to "which account does this app run as" site_topology.py, timeline.py what a site reaches, live; one timeline over a stretch internal_ca.py, ca_trust.py the server's own authority for names that only resolve inside a private network (`noust cert ca`): name-constrained, one machinery with the database engines' (`TlsAuthority`), its leaves kept as uploads, trust told per device console_exposure.py the console on a public name over TLS (`noust console expose`): nginx site, certificate, trusted proxy, and a record of exactly what it made so `unexpose` removes that and no more console_service.py the console as a process Noust starts and stops: the daemon, the unit `noust web enable` writes, the port it holds, the trusted proxies; the commands of `cli/commands/web.py` and the layers that look at the running console ask it backup_manager.py, rollback_manager.py a backup, and going back to one (the rollback points, and the safety backup taken first) server/platforms.py the one table of releases and processors Noust installs on; the installer and docs/PLATFORMS.md carry generated copies of it deployers/ strategies over a declarative pipeline (base/: `BaseDeployer`, a chain of classes, one module per phase of the pipeline), one per app type lifecycle.py the one "update an app"; release activation; resource limits migrate.py in-place to releases, explicit only, undone exactly on failure domains.py the one "names an app answers on": store, site, certificate, DNS check compose_relay.py zero-downtime for Compose: a twin of each web service while it is recreated compose_adopt.py `noust app adopt`: register a stack that already runs, touching nothing container.py `--type container`: a Dockerfile, a registry image or Railpack, run as a compose file Noust writes (helpers/container_compose.py), one image per release (helpers/container_image.py builds, pulls and prunes them), settings in core/container_store.py, changed through helpers/container_settings.py helpers/ health_gate.py, release_build.py, project_file.py (noust.yaml), hooks.py (which hooks run), site.py (the one writer and deleter of an app's site), compose_ports.py (the one port reader) fleet/ node enrollment: authorize.py installs the restricted key and prints the join code; keys.py the per-node keypair; tunnels.py the SSH tunnels (through CommandRunner); client.py a node's API over its own tunnel; nodes.py the store-backed registry a central keeps of them; enroll.py the guided enrollment from the central (a credential used once, never kept) central/ what a central is: role() (hub or server), setup.py (data directory, TLS, the allow-list), unlock.py the socket a sealed central takes a passphrase on monitor/ what watches the server: the collector and its samples, alert rules and the notifiers they reach, the process monitor and the reachability probe that `noust-monitor` runs recipes/ well-known applications as data, never code (`.yaml` and the assets they render or copy): the schema, the catalog, the Coolify and Dokploy importers and the deploy of a recipe integrations/ services outside the server: code hosts (forges/, github/, gitlab/) and the webhooks a registry or a forge calls terminal/ the console's shell in the browser and its one-off commands: tickets, sessions, recordings, and the limits around the most root-equivalent thing the console opens manifest/ a server written down: schema, plan and apply, as a client of the API (rule 3) pushdeploy/ a push as a way to deploy: a bare repository per application and who may push client/ Noust as a client of another Noust server, from a laptop or a pipeline, with a token and no SSH: the API, contexts and jobs templates/, completions/ data and no Python: the Jinja templates of nginx, Apache, PHP-FPM and systemd, and the generated shell completions web/ api/ thin layer over the managers; one error contract for every router; node_proxy.py forwards every node call, event and WebSocket over its tunnel (rule 3: a central re-implements nothing a node does) auth/ the console's sign-in, a package of the pieces `web/auth.py` was made of (`tokens.py` the token manager, `sessions.py` the SQLite store, `ratelimit.py`, `fleet.py` what a node admits from its central, `dependencies.py` what every route resolves); `__init__.py` imports each public name it had server.py security middleware, CSP, serves the console versioning.py /api/v1 as /api, X-Noust-API-Version, deprecated(); the compatibility promise is docs/api.md and scripts/api_compat.py (panel/openapi.baseline.json) events.py the /events SSE stream the console listens to jobs.py background jobs, persisted with their logs job_reconcile.py a job in a transient systemd unit is followed again after a restart gateway_client.py the console's own API called back as whoever asked (the manifest's plan and apply run on it): the engine is a client, never a second implementation manifest_jobs.py applying a manifest as a job, outside the deploy queue it feeds static/ the console's committed Vite build (generated from panel/, never edited) cli/ Click tree (app.py, commands/, including fleet.py, node.py, central.py); handlers hold no business logic panel/ Noust console source: React 19, TypeScript, Vite, TanStack Router/Query, Base UI, Tailwind v4; src/api/schema.gen.ts is generated from openapi.json e2e/ Playwright + axe + CSP gate against the real backend scripts/console_server.py the real API on a seeded, sandboxed store (development and E2E): the launcher of the package `scripts/console_server/`, which holds the sandbox, the machine the runner answers for, the seed of each page, the fleet and the Server area, one module each scripts/generate_sdk.py sdk/python from panel/openapi.json; sdk/terraform is the map the provider is written from (docs/sdk.md, docs/terraform.md); CI fails on a stale file terraform-provider-noust/ the Terraform and OpenTofu provider (Go, Plugin Framework): a client of the API like the SDK, with the operations table (internal/noust/operations.go) held to panel/openapi.json by its tests. Published from a repository of its own (docs/terraform-registry.md, scripts/prepare_terraform_registry.py) scripts/reproducible_build.py the wheel and the sdist of a commit, byte for byte (docs/reproducible-builds.md); CI and the release workflow build with it tests/support/ what the tests share and is not a test: simulated machines, factories, builders and the helpers of test modules (a test module is never imported by another, and nothing but `test_*.py`, `conftest.py` and `__init__.py` sits loose in `tests/`; `tests/test_tests_layout.py` holds both) tests/integration/run.py real deploys in a systemd container (Docker); not part of pytest. The 4.0 features' scenarios (how an application arrives: Procfile, scheduled updates, push, uploads, previews; and what protects its data: S3 backups, database instances, restoring a server) are in scenarios_delivery_and_data.py (loaded by run.py), and `--upgrade-from-3.3` (scenarios_upgrade.py) upgrades a 3.3.1 server that carries every legacy name; `--downgrade-to-3.3` goes back over a store the new version migrated. scenarios_real.py holds the ones that run the real software: the Compose recipes (`--scenario recipe_compose`) and a server a real Coolify runs, taken over (`--scenario coolify_takeover`), both opt-in because they pull gigabytes. Everything it creates on the machine's Docker carries the label `noust.test=1`; the rest runs in the harness container's own Docker (NOUST_IT_IMAGE, NOUST_IT_DOCKER_VOLUME and NOUST_IT_PREFIX name the image, the volume and the containers when the Docker is shared) tests/integration/install_matrix.py the installer on one container per release (run through `run.py --install-matrix`, inside the harness container); not part of pytest tests/integration/terraform_run.py the provider and the Python SDK against the live API of scripts/console_server.py, Go and Terraform in containers; not part of pytest tests/integration/identity.py SAML, OpenID Connect, LDAP and SCIM against Keycloak, OpenLDAP and a SCIM conformance client; not part of pytest tests/integration/network_run.py fail2ban, HTTP/3, forward-auth, prefix routes, a site's history, Coolify templates and the private certificate authority against the real programs and real clients (Debian server, Alpine client, AlmaLinux device); nginx with ModSecurity and the Core Rule Set blocking real attacks; certbot against Pebble (HTTP-01, DNS-01 through Noust's hooks and through the RFC 2136 plugin on a real BIND, and a wildcard); the time from `apt install` to a first deploy over HTTPS, measured and kept in tests/integration/results/first-deploy.json; not part of pytest tests/integration/fleet_run.py a central and a node over real SSH: enrollment, the restricted key, a server reached through a bastion and one that dials in (`fleet link`); not part of pytest tests/integration/toolchains_run.py the JVM, .NET, Elixir, Deno, Go, Rust, Rails and the Node frameworks (Nuxt, SvelteKit, Astro, Remix, Angular) built and served, and the runtimes downloaded and installed (Node, Python with uv, PHP with apt); not part of pytest tests/integration/forge_run.py a real Gitea and a real Forgejo: a deployment's commit status, a preview's pull request comment and the signed webhooks the receiver reads (docs/sources.md); one loopback port, `noust-test-w6x-*`, not part of pytest packaging/install.sh the one-line installer (POSIX sh, shellcheck-clean), published with a signed SHA256SUMS; its platform table is generated, never edited ``` ### Layers Imports go down, never up. `core` and `validators` are the base; `managers`, `deployers`, `fleet`, `monitor` and `pushdeploy` do the work on top of it; `web` and `cli` are the two ways in, and nothing under them imports them. `tests/test_layering.py` holds the rules, each with the list of places that still break it: the test fails when a new place appears and when a listed one is fixed and not taken off, so the lists can only get shorter. A command attaches itself to its group by importing the group (a module of its own), never the other way round, so no module imports another that imports it back when it loads. The graph is read from the syntax tree (`tests/support/import_graph_support.py`); an import under `TYPE_CHECKING` is not one at run time and does not count. | Layer | Must not | Places that still do | |---|---|---:| | `core`, `validators` | import `central`, `cli`, `deployers`, `fleet`, `integrations`, `managers`, `monitor`, `terminal`, `web` | 28 | | `deployers`, `fleet`, `managers`, `monitor`, `pushdeploy` | import `cli`, `web` | 11 | | `web` | import `cli` | 0 | | any module | import a module that imports it back when it loads | 0 | | any module | pass 3000 lines | 17 | The table is written from the test's own lists. A new module past 3000 lines is split before it is written, and the ones that are past it do not grow: new code goes in a module of its own beside them. The names an installed server reads do not move with the code: `noust.main:cli` (the console script of every copy installed before the command tree moved), `python -m noust`, and the three modules a transient unit runs with `python -m` (`managers/network_maintenance`, `managers/server/security_checks` and `managers/server/security_pending`). `tests/test_installed_names.py` fixes them; a move that touches one leaves a one-line module under the old name and records until when in `core/legacy_names.py`. ### Adding a deployer 1. Create `deployers/mytype.py` implementing the deployer interface. 2. Set `APP_TYPE`, `DISPLAY_NAME`, `DETECTION_FILES`, `DEFAULT_PORT`. 3. Implement `detect()`, `get_install_command()`, `get_build_command()`, `get_start_command()`. 4. Register with `DeployerRegistry.register(MyTypeDeployer)` at the end of the file. 5. Add a `detect()` test with a fake file tree, including the ambiguous cases. Build on `BaseDeployer` and the type gets releases, the health gate, domains, limits and deploy hooks for free. `MonorepoDeployer` and `DockerComposeDeployer` implement `AppDeployer` directly, which is why they deploy in place (`SUPPORTS_RELEASES` is read with `getattr`, defaulting to false); they share the hooks, the site writer and the domain capability (`helpers/hooks.py`, `helpers/site.py`) rather than re-implementing them. --- ## Deploy engine An application is on one of two layouts, recorded in `apps.layout`: - `inplace` (1.x): the service runs the tree every update rebuilds. - `releases`: `releases//` per deploy, `current` pointing at the active one, `.env` and persistent paths in `shared/`, a git cache in `repo/`. `BaseDeployer` works with three paths, and code must use the right one: - `app_path`: the application directory the store records. - `build_path`: where the code being deployed is built. The new release on releases, `app_path` in place. Everything that reads or builds the project uses it. - `runtime_path`: what the unit and the web server are given. `app_path/current` on releases, `app_path` in place, so one unit and one site serve every release. In place the three are the same directory, which is what keeps that layout exactly as it was. Outside a deploy, ask `core/appdir/layout.py`: `env_file_for(app)` for the `.env`, `code_path_for(app)` for the running code. Never join `app_path` with `".env"`. Rules: - **An in-place application is never converted implicitly.** `choose_layout()` keeps the layout an existing application has, a redeploy that asks for another is an error, and `ReleaseManager.activate()` refuses to replace a real `current` directory. Only `migrate.migrate()`, called by `noust app migrate` and `POST /api/apps/{d}/migrate`, moves an application onto releases. - **Every activation passes the same `HealthGate`**: deploy, update, rollback, migration and limits applied with a restart. When it fails, what served before is put back (the previous release, the in-place tree, the old limits) and the error carries the probes and the journal verbatim. - **`lifecycle.update_app` is the only update.** The CLI, the console's job and the webhook all call it. - **Nothing is written through a symlink found in a release or in `shared/`.** A repository is untrusted input. - **`app_path` is not always under the apps directory.** An adopted stack lives where it already ran (`/opt/shop`). Never build `apps_directory / app_name`: use the store's `app_path` (`lifecycle`, `BackupManager` and deletion already do), and never remove a directory Noust did not create unless the operator names it (`--remove-adopted-directory`). --- ## Access, audit and builds (3.1) - **Every API route declares a permission** in `web/permissions/routes_.py`; one global dependency enforces it and publishes it as `x-noust-permission`, and a route without one is refused (a test fails first). Roles (`viewer`, `operator`, `admin`, `security`, `auditor`) are sets of permissions in `web/permissions/roles.py`; accounts live in `core/accounts/`. The master access token is break-glass, not the normal way in. Root-equivalent actions can require a second person (`core/accounts/approvals.py`, guard on the router). - **One audit trail**: `noust.core.audit.record(event, actor=...)` with a closed catalog (`core/audit/catalog.py`; a test scans the code for undeclared events), an HMAC chain, and shipping to journald/syslog. CLI commands that change something are audited by `cli/app.py`. - **Builds never run as root.** Install, build and hooks run through `CommandRunner(..., sandbox=SandboxSpec(...))` as `noust-build` in a transient systemd unit, failing closed when the sandbox cannot be proven; migrations run as the app's user. Existing apps switch per app after a test build (`noust app sandbox test|enable`). - **Dry-run probes are declared by exact argv** (`core/runner.py` `READ_ONLY_PROBES` and each area's `probes.py`); a command is never read-only because one of its arguments looks harmless. - **The ENS profile** (`security.profile: ens-medium`) lives in one module, `core/ens/profile.py`; every area reads its defaults from there. ## Compose, hooks and the web server (3.2) Operator documentation is `docs/compose.md`. - **`noust.yaml` is untrusted input and the schema is closed.** `project_file.py` reads it (`hooks.pre_deploy`, `hooks.post_deploy`, `backup.databases`); an unknown key, `..`, a link or an absolute path outside a container is a `ValidationError` naming the field, never ignored. `hooks.resolve_hooks` is the one answer to "which hooks run": the operator's document (store `app_hooks`, root-equivalent to write) replaces the repository's whole, never merged. A hook is an argv (shlex, never a shell) through `CommandRunner`: in Compose a one-off container of the new image (`docker compose run --rm --no-deps`), elsewhere the release phase of the sandbox. `pre_deploy` failing aborts before any traffic moves; `post_deploy` failing is a deployment with `warnings` and the `deploy_hook_failed` notification. Prisma's automatic migration aborts on failure and does not run when hooks are declared. - **Schema changes are a decision.** A hook marked `migrates` (read against the migration tools' "nothing applied" wording) or Prisma applying a migration sets `deployments.schema_changed`; every way back (`rollback`, `releases rollback`, `update --commit`, files-only restore, the API and the console) goes through `lifecycle.require_schema_change_confirmed` and refuses without `schema_changed_ok`. Noust puts code back, never a database. The automatic go-back after a failed gate still happens and says so first. - **An operator's site is never rewritten or deleted** (`helpers/site.py`; the "Generated by Noust" marker decides). A domain becomes a site file in one place, `webserver.config_path` through `site_name_for`, because an adopted app's file may not be named after its domain (`shop` for `shop.example.com`). - **The relay** (`compose_relay.py`) shares its parts with blue/green and is opt-in through the same `zero-downtime` command. nginx reaches each relayed service through `/etc/nginx/noust-upstreams//.servers` (`core/paths.py`; upstream names are `noust_bg_`, and a site the operator wrote that still names the prefix of before 4.0 keeps working: `upstream_names.py` defines both and `noust setup doctor` says the line to change); an operator's site includes it itself and Noust prints the exact line. An interrupted update leaves a relay that the next update and `noust setup doctor` resolve. - **Adopting never changes what runs.** `compose_adopt.py` reads the project from the containers' labels (`apps.compose_project`, passed as `-p` everywhere), proves the stack with `up --dry-run --no-build`, creates and enables the unit without starting it. No Compose update cleans a checkout: it is brought to the branch and what git does not track (the `.env`, bind-mounted data) stays. - **A stack's databases are dumped before every Compose update**, in the pre-update backup, and a failed dump stops the update (`StackBackupError`): without it a migration cannot be undone. Passwords are read inside the container, never in argv. `backup_before_update` and `backup.databases: off` turn it off. - **`siteconf` is the one nginx/Apache parser.** `render(parse(text)) == text` for any text the server accepts; edit operations change only the bytes of the element they name; `route` applies the server's own location algorithm. The API (`structure`, `config/edit`, `route`, `topology`) writes nothing: the visual editor produces text and saves it through the one `PUT .../config` (rule 3 and 4). A site's config test runs inside the live `nginx.conf` with the enabled sites. - **One definition of a worker**: `compose_ports.is_headless_stack` (no TCP port published). Deploy, store, `diagnose`, the monitor and the summaries judge it by its containers. - **Accounts and the sandbox.** `app_identity.service_account` is the only answer to "as whom"; new apps get `noust-app-`, existing ones only by `noust app identity migrate`. The sandbox is the default for apps from before 3.1 through `sandbox_trial.trial_before_update`: it never breaks an update that worked. - **Monitor and jobs.** `app_unreachable`/`app_recovered` come from the monitor's `reachability` (three failures over a minute, the deploy's own `HealthCheck`); process samples (top 5 CPU and memory per minute, commands redacted) feed `GET /api/timeline`; a job records its transient unit (`jobs.unit`) so `job_reconcile` finishes it after a console restart. - **Store v13** is `core/schema_v13.py`; every new column has its own validated setter and survives a redeploy's full-row write. The harness (`tests/integration/run.py`) now has Docker inside; an optional scenario deploys a clean clone of a real production Compose application from a private repository, and nothing ever runs against the host's Docker or its containers. ## Fleet A central is a Noust whose job is several other Noust servers (`central.role`: `hub` never deploys anything itself, `server` also runs its own applications). It reaches each one over an SSH tunnel it opens outward and drives it through its own API: rule 3 again, so a central re-implements nothing a node does. `web/api/node_proxy.py` forwards every call, the event stream and the log and job WebSockets over that tunnel; nothing runs twice. - **The central never gets a shell.** `noust fleet authorize` runs on the node, as root, and installs the central's key for the unprivileged `noust-tunnel` account (no home, `nologin`, keys in a root-owned file), with a Noust `Match User` block in sshd (`AllowStreamLocalForwarding no`, `AllowTcpForwarding local`, `PermitOpen 127.0.0.1:`, `PermitTTY no`, `ForceCommand /usr/bin/false`) and the same limits repeated as key options (`restrict,port-forwarding,permitopen=...,permitlisten="127.0.0.1:1",command="/usr/bin/false"`, built in `fleet/authorize.py`). Both layers matter: `permitlisten` narrows TCP `ssh -R` but not Unix-socket forwarding, which with a key in root's `authorized_keys` (3.0) could create a socket as root at any path. `--ssh-user root` exists only behind an explicit confirmation. - **Each node sets a ceiling for its central.** `noust fleet authorize --access read|deploy|admin` (default `admin`) and `--allow-host-access` (SSH keys, sshd, firewall: off by default) are enforced by the node in the fleet admission (`fleet/policy.py` `permits()`), whatever the central claims; the central reads them from `GET /api/auth/fleet/self`. - **Fleet tokens are accepted only from loopback.** They travel through the tunnel, never across the network, and only a request presenting one may carry `X-Noust-Actor`, so a node's audit log names the operator behind the central rather than just the central's name. - **Elevation is the node's call, not the central's.** A node marks what needs sudo mode with the `x-noust-requires-elevation` OpenAPI extension; the central asks its own operator to confirm before forwarding such a call and vouches for it with `X-Noust-Elevated`, and the node refuses the call anyway if the central did not vouch - one source of truth, checked on both sides, so a compromised or outdated central cannot forward its way past a node's sudo mode. - **A host-key change is never accepted silently.** Each node's key is pinned to the central's own `known_hosts` the first time it is added; a change closes the tunnel and says why instead of trusting whatever key answers next. - **A guided enrollment keeps no credential.** `noust fleet enroll` and `POST /api/fleet/enroll` connect once with a password or a key, to the host key the operator confirmed, and do what the manual join does (`NodeEnroller` calls `NodeManager.add`). The credential reaches ssh through its environment or a file in a private, memory-backed directory removed on every way out, never an argument, a log line, a job result, an audit event or an approval (`ONE_TIME_FIELDS` leaves it out of the call's fingerprint). Noust is installed with the central's own copy of `install.sh`, sent over the connection: the server downloads no script. - **The console is published, never left open.** `noust console expose` refuses while any account can sign in without a second factor, under the ENS profile unless forced and given an allow-list, and on a name that is an application's or a site Noust did not write; there is no plain-HTTP form. `web._install_and_start` keeps the trusted proxy it recorded whenever the unit is written again. See `docs/CENTRAL.md` for running a central, sealing its secrets and adding a server, and `docs/PLATFORMS.md` for the platforms and the installer. ## Teams (4.0) Teams, projects and environments (`docs/teams.md`) are one more instance of rule 4. Applications, sites, databases and backup schedules belong to a project, and what a principal sees of them is decided in the data layer, not in the routes. - **Read through `ScopedStore`.** A router asks `noust.core.teams.scoped_store()` for apps, sites, databases, backup schedules, deployments and jobs; the store's own `get_app`, `list_sites` and the rest are for what runs as the instance. `tests/test_scope_architecture.py` fails on a router that reads one directly (with the few places allowed, each with its reason). - **Every route is classified** in `web/permissions/scopes.py` (resource, scoped, open, instance), like its permission in `routes_.py`. `enforce_scope` runs on the API router before the approvals guard; a resource outside the principal's teams is a 404 with the body of one that does not exist, never a 403. A new route under `/api/apps/{domain}` is covered by its prefix; a new area needs a rule. - **A new row gets an owner from the store** (`core/teams/ownership.py`): its application's, the one a job was queued for (`create_job(..., ownership=...)`), else the default. Never insert into `apps`, `sites`, `databases` or `backup_schedules` around `create_*`, and never write `project_id` or `environment_id` anywhere but a move (`TeamStore.move_app_to`). - **A manager that lists applications around what a request names reads through `ScopedStore`** too (the deploy keys do: `DeployKeyManager._visible`). Its scope is the ambient one, the request's inside a request and the instance's in the CLI and in a job, so one implementation serves both without showing a team another team's applications. A route whose body can ask for something the server owns (`install: true` installs a package as root) calls `web.api.scope.require_instance_scope`. A change to something shared by repository (rotating a deploy key that reaches another team's repositories) is refused in the manager, not the route, whoever calls it. - **A table another area adds for what an application or a database owns says how a team reaches it.** A column named `app_id`, `app_domain`, `domain`, `db_name`, `database` or `repository` in a `schema_v14.py` fragment (or an `ALTER` of an existing table) must be listed in `OWNED_TABLES` of `tests/test_scope_architecture.py` with the routes that reach it and how each is classified: a resource under `/api/apps/{domain}`, a list the handler filters, or the server's. A table of a server-wide thing (the console's own name) goes in `NOT_OWNED` with the reason. - **Team roles never reach the machine** (`web/permissions/team_roles.py`); a permission that is about the server is never added to `TENANT_PERMISSIONS`. --- ## Conventions - App name: the domain with dots as dashes (`shop.example.com` is `shop-example-com`). It names the directory `/var/www/apps/{app_name}/` and the unit `{app_name}.service`; units from before 0.14.1 carry a legacy prefix (`legacy_names.UNIT_PREFIX`) until the application's next deployment renames the unit as part of the restart it makes anyway. Noust's own units are `noust-web`, `noust-monitor`, `noust-previews`, and the cron and backup timers `noust-cron-*` and `noust-backup-*`; they carry the "Generated by Noust" marker (the first line of before 3.0 is still recognised until `noust setup rename` rewrites it). - Paths: `/etc/noust`, `/var/lib/noust` (store `noust.db`), `/var/backups/noust`, `/var/log/noust`. Ask `core/paths.py`, never spell one. Noust 4.0 moves nothing: a 2.x server upgrades to 3.3.x first, and `core/retired_names.py` refuses to start on a server that was not moved there. - Environment variables are `NOUST_*`, read through `paths.getenv`. - **The product has no other name.** What lives outside Noust's control and was made under the name of before 3.0 (a header a script sends, a token in a CI secret, a file in a customer's repository, a site an operator wrote, a backup or an export) is accepted on input and never written: `core/legacy_names.py` lists each name with why it is kept and until when (5.0; backups and exports for as long as the file exists), and `core/legacy_archive.py` reads the old archives. What Noust holds itself (a unit, a database account, a file in an application's tree, an image tag, a backup folder) is renamed the next time that thing is used, never in a step that restarts something outside a deployment, and `noust setup rename` finishes the rest. The paths of before 3.0 are spelled in `core/retired_names.py`, and in the package scripts that stop an upgrade over data nobody moved. Nowhere else: `tests/test_no_legacy_name.py` fails on the old name, in any case, in `src`, `docs`, `scripts` and the packaging, and a name that has to be recognised goes through `legacy_names`. The changelogs are history and are not rewritten. - Feedback items are numbered and commit bodies cite them as "item N". - Google-style docstrings on everything public, with Args/Returns/Raises. - Type hints everywhere, modern syntax (`X | None`, `list[str]`). - Actionable errors: `raise DeploymentError("what happened", details="how to fix it")`. - A pattern that judges text is applied with `fullmatch`, never `match` plus a `$`: `$` matches before a trailing newline, which is how `validate_app_name("shop\n")` was once accepted. `tests/test_regex_anchors.py` fails on a new one. Text a person or a repository supplies is judged at one place by an allowlist (`validators/`), and that place is attacked by `tests/test_adversarial_inputs.py`: add a class there when you add a validator. - Anything that opens a request to a host somebody typed goes through `core/outbound.py` (`require_public_host`), never its own resolver. A name that becomes part of a request to a platform's API is made one segment by `path_segment`, and the client checks the path it sends with `api_path`. - Code scanning alerts are fixed in one place, taught to CodeQL as a model that a test proves, or dismissed in GitHub with the reason written down; none is left open (a job fails while one of high severity is) and none is silenced in a line of code (`docs/code-scanning.md`). A function that returns a name only when it is acceptable goes in `.github/codeql/extensions/noust-python` with its proof in `tests/test_codeql_models.py`, once measured to remove an alert. - Comments explain **why**, never what. - **Start-up is a budget.** `noust --version` and `noust --help` import 16 Noust modules between them and `tests/test_cli_startup_budget.py` fails past 24. A package `__init__` that re-exports classes does it through `core/lazy_exports.py`: importing anything below a package runs its `__init__`, so imports at the top of it load everything the package offers. The root's `--help` and the shell completion read each command's line from `cli/command_summaries.py`; after changing a command's help or adding a command, run `python scripts/generate_command_summaries.py` (a test fails until you have). - No emojis in code, comments or commit messages. - Absolute paths in systemd units; `shutil.which()` or `/usr/bin/`, never nvm paths. - Noust requires root. There is no `sudo` inside argv and no `run_command_sudo`. --- ## Commands ```bash pip install -e ".[all,dev]" # development install pytest # tests pytest --cov=noust # with coverage pytest --shard 2/6 # one slice of the suite, by whole files (CI runs each slice on its own machine) pytest -m web # one area of the suite: the first word of a test file's name (tests/support/areas.py) ruff check src/noust tests # lint (blocking in CI) ruff format src/noust tests # format (blocking in CI) mypy # types (blocking in CI) python scripts/release.py --check # version consistency python scripts/security_model.py --write # regenerate docs/SECURITY-MODEL.md after changing a role, a header, a route or a guarantee python scripts/generate_command_summaries.py # regenerate cli/command_summaries.py after changing a command's help or adding a command ``` The integration harness deploys for real (nginx, systemd, Docker, a database) inside one privileged container, and is not part of pytest. On a machine whose Docker is shared, name what a run makes so it can be found and removed by that name and by the label `noust.test=1` (never a prune): ```bash export NOUST_IT_PREFIX=noust-test-me NOUST_IT_IMAGE=noust-test-me-image:latest \ NOUST_IT_DOCKER_VOLUME=noust-test-me-docker NOUST_IT_MEMORY=10g python tests/integration/run.py # every scenario but the optional ones python tests/integration/run.py --scenario procfile --keep # one (a substring); leave the container python tests/integration/run.py --reuse --skip-image-build --scenario procfile # again, in the kept container, in seconds python tests/integration/run.py --upgrade-from-3.3 # 3.3.1 with every legacy name -> this tree python tests/integration/run.py --install-matrix # the installer on each release (network) ``` The console needs Node 22 (the major in `panel/.nvmrc`); CI runs every one of these: ```bash cd panel && npm ci # the lock file, exactly; never `npm install` in CI npm run lint # eslint, zero warnings npm run typecheck # tsc, strict npm test # vitest unit + component tests (axe included) npm run check:api # schema.gen.ts matches openapi.json (npm run gen:api to fix) npm run build # writes src/noust/web/static: commit the result npm run check:bundle # the build's gzip weight against panel/bundle-budget.json npm run i18n:routes # rewrite src/i18n/routeNamespaces.gen.ts after a route or a page's texts change npm run e2e # Playwright: every page, both themes, zero axe/CSP violations npm run e2e:screens # screenshots of every route, both themes + 390px mobile ``` To work on the console against the real API, run a seeded, sandboxed backend and the Vite dev server, which proxies `/api`, `/events`, `/hooks` and `/ws` to it: ```bash python scripts/console_server.py --port 8080 # prints {"url", "token", ...}; sign in with the token cd panel && npm run dev # http://localhost:5173 ``` `console_server.py` runs the real FastAPI app under uvicorn over a store seeded by `tests/support/panel_factory.seed_console_state`, with every system path redirected into a temporary directory and a fake runner answering systemctl, journalctl, nginx and certbot, so nothing on the development machine is touched. `--totp` turns on the second sign-in factor; the E2E suite starts one per Playwright worker. --- ## Releasing The version has one source of truth: `[project].version` in `pyproject.toml`. ```bash python scripts/release.py 1.0.0 -m "Summary of the change" git commit -am "v1.0.0: Summary" git tag -s v1.0.0 -m "Release v1.0.0" git push && git push origin v1.0.0 ``` `scripts/release.py` propagates to `setup.py`, `src/noust/__init__.py`, the RPM spec in `rpm/`, the `.dsc` in `obs/` and both changelogs. CI runs `--check` and refuses to publish on a mismatch. Do not edit those files by hand. The README opens with a note about the newest minor release only: a minor replaces it, never stacks another above it. Earlier releases are in their changelogs (`docs/CHANGELOG-.md`). The tag is signed with the maintainer's SSH key (`git config gpg.format ssh`, and the same public key registered on GitHub as a signing key, so the tag shows as verified): `SECURITY.md` promises it from 4.0.0 on. A release that fixes a vulnerability is released with a changelog bullet that starts `SECURITY:` (`-m "SECURITY: what it fixes (GHSA-xxxx-xxxx-xxxx)"`). `release.yml` then titles the GitHub release "(security release)", which the update check reads so that every installation behind it, its `noust health` and its console say so. The same release renews `.well-known/security.txt`. GitHub Actions publishes to PyPI and OBS on tag push, and the central's container image to `ghcr.io/perkybeet/noust`. **OBS builds**: https://build.opensuse.org/package/show/home:Perkybeet/noust (15-30 min) A release is **announced only when every distribution serves it**. The GitHub release is created as a pre-release (servers' update checks and `releases/latest/download/install.sh` read the latest one), `release.yml` then waits in `obs-published` (`scripts/obs_published.py`, which reads the published index of each repository of `managers/server/platforms.py`) and only the `announce` job makes it the latest. Run `python scripts/obs_published.py vX.Y.Z` by hand to see who is behind. The console's Fleet > Updates shows the target version and how many servers are there, and the self-update (the console's and the fleet's "Update Noust") upgrade `noust`. The pictures of the console in the README and the guides (`docs/assets/console/`) are recorded in `SCREENS.json` with the release they show and a hash each, and `release.yml` refuses a tag whose pictures show an earlier minor release: before tagging a minor, run `cd panel && npm run docs:screens` (it retakes them from a seeded server and records them) and commit the folder. CI only notes the lag between releases, and fails on a picture changed by hand or one nobody recorded (`python scripts/docs_screens.py check`). Distribution names: `noust` on PyPI and OBS. `noust` declares `Conflicts` with the packages of before 3.0 that it replaced, never `Obsoletes`: a server on the 2.x line goes through 3.3 first, and the `%pre` and `preinst` scripts stop an upgrade over data nobody moved. The container is in `packaging/container/`. --- ## Packaging notes The OBS tarball is produced with `git archive HEAD`, so **only committed files ship** and the build environment has no network. That is why the console's Vite build is committed to `src/noust/web/static/`: Node never runs during packaging, and the `web/static/**/*` glob in `pyproject.toml` ships it in the wheel, sdist, deb and rpm. Frontend dependencies are build-time only; nothing but the build ships, and it loads nothing from a CDN. The CI `panel` job rebuilds and fails when the committed build differs from what `panel/` produces, so after any change under `panel/src` run `npm run build` and commit `src/noust/web/static`. When adding a dependency, declare it in all four places: `pyproject.toml`, `setup.py`, `obs/debian.control` and the RPM spec in `rpm/`. `tests/test_architecture.py` fails if an import is undeclared. Check the package exists on every target: `python3-inquirer` does not exist in Debian or Ubuntu, which is why interactive mode never worked there. | Import | Debian | RPM | |--------|--------|-----| | `click` | `python3-click` | `python3-click` | | `jinja2` | `python3-jinja2` | `python3-jinja2` (openSUSE: `python3-Jinja2`) | | `yaml` | `python3-yaml` | `python3-pyyaml` (openSUSE: `python3-PyYAML`) | | `rich` | `python3-rich` | `python3-rich` | | `questionary` | `python3-questionary` (Recommends: absent on Debian 12, Ubuntu 22.04) | `python3-questionary` | | `fastapi` | `python3-fastapi` | `python3-fastapi` | | `starlette` | `python3-starlette` | `python3-starlette` | | `pydantic` | `python3-pydantic` | `python3-pydantic` | | `uvicorn` | `python3-uvicorn` | `python3-uvicorn` | | `psutil` | `python3-psutil` | `python3-psutil` | Every model the API adds goes through `web/pydantic_compat.py`: Ubuntu 24.04 and Debian 12 ship pydantic 1.10, and a CI job pins it. The packaging scripts and the specs say nothing about the product's names from before 3.0 except where an upgrade over data nobody moved has to be stopped (Debian's `preinst`, the RPM `%pre`) and in the `Conflicts` lines; `tests/test_no_legacy_name.py` holds them to that. A real customer's name is as unwelcome in a fixture as the old name is in a script: `tests/test_no_real_customers.py` fails on it, so examples use `example.com`, `example.net`, `example.org` or an invented name. --- ## The panel **The design system is `docs/DESIGN.md`, and it is normative**: tokens, the seven page templates (`components/page`), components, patterns and copy rules. A page is built from a template and kit components; when something is missing the kit (and its gallery at `/__design`) grows first. ESLint rules and a ratchet (`styles/design-rules.test.ts`, baseline that only goes down) enforce it; read it before touching `panel/src`. The Noust console is a React single-page application. Its source lives in `panel/`; its build is committed to `src/noust/web/static/` and served by `server.py`: hashed chunks under `/assets` (cached immutable for a year) and `index.html` (`no-store`) for every GET outside `/api`, `/assets`, `/events`, `/ws`, `/hooks` and `/health`, where a miss answers JSON. The backend renders no pages: the console is a client of the JSON API, the `/events` stream (`machine`, `metrics`, `app`, `job`, `state`, `notice`) and the log and job WebSockets, like any script. Destructive endpoints depend on `require_elevated` (sudo mode, `api/deps.py`); the console answers the `403 elevation_required` with its "Confirm it's you" dialog and retries once. FastAPI's OpenAPI schema is exported to `panel/openapi.json` and compiled into `panel/src/api/schema.gen.ts`, so the console cannot call an endpoint that does not exist. The Content Security Policy is strict: `script-src 'self'; style-src 'self'`, no `unsafe-inline` anywhere, and `require-trusted-types-for 'script'`. That rules out inline `