xeno/Portfolio
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix: recover and fence atomic activation

Browse Source
This commit is contained in:
LucaP
2026-09-02 11:49:49 +03:00
parent 79e5d04395
commit a8fa09e21e
12 changed files with 1733 additions and 543 deletions
Download Patch File Download Diff File Expand all files Collapse all files

32
docs/deployment/nginx-runbook.md
View File

@@ -4,12 +4,14 @@ This runbook describes the provider-neutral release transaction. It is not autho
## Release-root contract
Provision one dedicated absolute directory such as `<release-root>`. Never use `/`, `/var`, `/srv`, a home directory, or an existing web root directly. The directory must be owned by the narrowly scoped deployment account and contain:
Provision one dedicated absolute directory such as `<release-root>`. Never use `/`, `/var`, `/srv`, a home directory, or an existing web root directly. The directory must be owned by a narrowly scoped activation service account and contain:
```text
<release-root>/
.portfolio-release-root exact bytes: portfolio-release-root/v1\n
.control/release-trust.json canonical audited signing trust policy
control/state.json canonical JSON, owner read/write only
inbox/ upload identity write-only handoff boundary
```
The initial state is the canonical form of:
@@ -22,11 +24,12 @@ The initial state is the canonical form of:
"latestLedgerHeadHash": "<protected-live-ledger-head>",
"latestLedgerRevision": 0,
"publisherFence": 0,
"releaseTrustPolicyDigest": "<sha256-of-protected-release-trust-policy>",
"schema": "portfolio-release-control-state/v1"
}
```
Provisioning must write that state through the protected control plane, not by copying an example with placeholder values. The deployment account needs write access only to this dedicated root. Nginx needs read/traverse access to `current`; it does not need permission to write releases or control state. `current` points to `releases/<release-id>/public`, while signed metadata and the SBOM remain in the immutable sibling `.release` directory and are therefore outside the served document root. Signing keys, the approval ledger, trust policy, server credentials, and private source remain outside the release package and public root.
Provisioning must write that state through the protected control plane, not by copying an example with placeholder values. The upload identity may create package files only in `inbox`; it must not own or be able to write the release root, `.control`, `control`, `releases`, receipts, or `current`. The activation service owns those protected paths and derives immutable release ownership from the provisioned root itself—UID/GID are never supplied by a release caller. Nginx needs read/traverse access to `current`; it does not need write permission. `current` points to `releases/<release-id>/public`, while signed metadata and the SBOM remain in the immutable sibling `.release` directory and are outside the served document root. Signing keys, the approval ledger, server credentials, and private source remain outside the release package and public root.
The scripts reject non-absolute roots, broad roots, missing sentinels, symlinked verifier modules, unknown canonical fields, and non-canonical JSON.
@@ -39,14 +42,12 @@ node scripts/build-release.mjs \
--dist <exact-dist-directory> \
--signed-manifest <signed-release-envelope.json> \
--sbom <sbom.cdx.json> \
--verifier-module <absolute-path-to-audited-sign-release.mjs> \
--trust-policy <external-canonical-trust-policy.json> \
--authority-root <release-root> \
--protected-state '<canonical-protected-state-json>' \
--now <trusted-rfc3339-time> \
--output <new-release.tar>
```
The builder verifies the real Ed25519 envelope through `verifySignedReleaseEnvelope`, validates the typed admission, requires the directory to equal the manifest closure exactly, and checks every byte count and SHA-256 hash. Its ustar output has sorted entries, fixed time/ownership/modes, no links, and exactly these metadata paths:
The production builder loads the single audited verifier and the digest-bound trust policy from the protected authority root. It does not accept verifier code, a policy path, or a caller-provided clock. It validates the typed admission, requires the directory to equal the manifest closure exactly, and checks every byte count and SHA-256 hash. Its ustar output has sorted entries, fixed time/ownership/modes, no links, and exactly these metadata paths:
```text
.release/package-index.json
@@ -68,13 +69,12 @@ The executable must expose that directory only through a non-public, authenticat
## Prepare without changing visibility
Use the deployment account's numeric UID/GID. Preparation verifies signature, exact predecessor, live ledger, fence, hashes, and time; extracts without using the system tar extractor; makes all release files `0444` and directories `0555`; runs the internal smoke; rechecks time and protected state; and writes an immutable readiness receipt. It does not create or change `current`.
Place the completed package in `<release-root>/inbox` through the upload identity. Preparation verifies signature, exact predecessor, live ledger, trust-policy digest, fence, hashes, and system time; extracts without using the system tar extractor; makes all release files `0444` and directories `0555` under activation-service ownership; runs the internal smoke; rechecks protected state; and writes a durable readiness receipt. It does not create or change `current`.
```sh
scripts/deploy/prepare-release.sh \
<release-root> <new-release.tar> <transaction-id> \
<absolute-verifier-module> <external-trust-policy.json> <absolute-smoke-command> \
<deployment-uid> <deployment-gid> <trusted-rfc3339-time>
<release-root> <release-root>/inbox/<new-release.tar> <transaction-id> \
<absolute-smoke-command>
```
Inspect the resulting release ID and readiness receipt before activation. A preparation failure leaves the existing `current` pointer untouched.
@@ -85,11 +85,10 @@ Supply the exact predecessor digest (`none` only for the first release) and the
```sh
scripts/deploy/activate-release.sh \
<release-root> <transaction-id> <expected-current-digest-or-none> <expected-next-fence> \
<absolute-verifier-module> <external-trust-policy.json> <trusted-rfc3339-time>
<release-root> <transaction-id> <expected-current-digest-or-none> <expected-next-fence>
```
Activation obtains an exclusive transaction lock, re-verifies the signed envelope against current protected state, rechecks receipt/ledger/predecessor/fence/expiry, writes a durable committing intent, creates the relative `current.next` symlink, and renames it atomically over `current`. Nothing else changes public visibility.
Activation obtains the activation-service transaction lock, re-verifies the signed envelope against current protected state, and immediately before publication rereads the exact receipt, ledger head/revision, trust-policy digest, predecessor, pointer, fence, and expiry. It writes a durable committing intent, creates the relative `current.next` symlink, and renames it atomically over `current`. Every protected control-plane writer must honor the same service lock/CAS protocol; direct state or pointer writes are forbidden.
- A failure before the rename leaves the predecessor visible.
- A crash after the rename is recovered by repeating the same command: the script observes the new pointer and finalizes the receipt/state without another visibility change.
@@ -104,16 +103,15 @@ Rollback does not replay the selected release's old manifest. The signing pipeli
```sh
scripts/deploy/rollback-release.sh \
<release-root> <selected-old-release-id> <newly-signed-rollback-release.tar> <transaction-id> \
<absolute-verifier-module> <external-trust-policy.json> <absolute-smoke-command> \
<deployment-uid> <deployment-gid> <current-manifest-digest> <expected-next-fence> <trusted-rfc3339-time>
<release-root> <selected-old-release-id> <release-root>/inbox/<newly-signed-rollback-release.tar> <transaction-id> \
<absolute-smoke-command> <current-manifest-digest> <expected-next-fence>
```
The script verifies that the rollback package uses a fresh manifest digest and exactly matches the selected old artifact closure, then performs the same readiness and atomic pointer transaction. Record both activation and rollback receipts in the release record.
## Orphan staging recovery
Only hidden directories named `<safe-transaction-id>.partial` beneath the validated dedicated root may be cleaned. Active transaction staging is retained. After protected-state recovery has confirmed that no transaction can resume:
On startup or retry, a durable `preparing` intent is recovered idempotently: the activation service proves that its target is not public, removes only that transaction's partial/release/receipt artifacts, clears the intent, and starts verification again. If the public pointer unexpectedly names a preparing release, automatic cleanup stops for privileged investigation. Only hidden directories named `<safe-transaction-id>.partial` beneath the validated dedicated root may otherwise be cleaned. Active transaction staging is retained. After protected-state recovery has confirmed that no transaction can resume:
```sh
scripts/deploy/prepare-release.sh <release-root> --cleanup-orphans