feat: package atomic reversible nginx releases
This commit is contained in:
133
docs/deployment/nginx-runbook.md
Normal file
133
docs/deployment/nginx-runbook.md
Normal file
@@ -0,0 +1,133 @@
|
||||
# Atomic Nginx release runbook
|
||||
|
||||
This runbook describes the provider-neutral release transaction. It is not authorization to connect to a server or publish the portfolio. Replace every angle-bracket placeholder only at the explicit deployment checkpoint.
|
||||
|
||||
## Release-root contract
|
||||
|
||||
Provision one dedicated absolute directory such as `<release-root>`. Never use `/`, `/var`, `/srv`, a home directory, or an existing web root directly. The directory must be owned by the narrowly scoped deployment account and contain:
|
||||
|
||||
```text
|
||||
<release-root>/
|
||||
.portfolio-release-root exact bytes: portfolio-release-root/v1\n
|
||||
control/state.json canonical JSON, owner read/write only
|
||||
```
|
||||
|
||||
The initial state is the canonical form of:
|
||||
|
||||
```json
|
||||
{
|
||||
"activeTransaction": null,
|
||||
"currentManifestDigest": null,
|
||||
"currentReleaseId": null,
|
||||
"latestLedgerHeadHash": "<protected-live-ledger-head>",
|
||||
"latestLedgerRevision": 0,
|
||||
"publisherFence": 0,
|
||||
"schema": "portfolio-release-control-state/v1"
|
||||
}
|
||||
```
|
||||
|
||||
Provisioning must write that state through the protected control plane, not by copying an example with placeholder values. The deployment account needs write access only to this dedicated root. Nginx needs read/traverse access to `current`; it does not need permission to write releases or control state. `current` points to `releases/<release-id>/public`, while signed metadata and the SBOM remain in the immutable sibling `.release` directory and are therefore outside the served document root. Signing keys, the approval ledger, trust policy, server credentials, and private source remain outside the release package and public root.
|
||||
|
||||
The scripts reject non-absolute roots, broad roots, missing sentinels, symlinked verifier modules, unknown canonical fields, and non-canonical JSON.
|
||||
|
||||
## Build a deterministic package
|
||||
|
||||
After the isolated privacy/signing stage has produced a canonical signed release envelope and SBOM:
|
||||
|
||||
```sh
|
||||
node scripts/build-release.mjs \
|
||||
--dist <exact-dist-directory> \
|
||||
--signed-manifest <signed-release-envelope.json> \
|
||||
--sbom <sbom.cdx.json> \
|
||||
--verifier-module <absolute-path-to-audited-sign-release.mjs> \
|
||||
--trust-policy <external-canonical-trust-policy.json> \
|
||||
--protected-state '<canonical-protected-state-json>' \
|
||||
--now <trusted-rfc3339-time> \
|
||||
--output <new-release.tar>
|
||||
```
|
||||
|
||||
The builder verifies the real Ed25519 envelope through `verifySignedReleaseEnvelope`, validates the typed admission, requires the directory to equal the manifest closure exactly, and checks every byte count and SHA-256 hash. Its ustar output has sorted entries, fixed time/ownership/modes, no links, and exactly these metadata paths:
|
||||
|
||||
```text
|
||||
.release/package-index.json
|
||||
.release/sbom.cdx.json
|
||||
.release/signed-release-manifest.json
|
||||
```
|
||||
|
||||
The remaining entries are exactly the signed deploy files. Building twice from identical inputs must produce identical archive bytes.
|
||||
|
||||
## Internal staged smoke command
|
||||
|
||||
Preparation requires an operator-provisioned absolute executable. It is called without a shell as:
|
||||
|
||||
```text
|
||||
<smoke-command> --release-dir <immutable-release-directory> --manifest-digest <digest>
|
||||
```
|
||||
|
||||
The executable must expose that directory only through a non-public, authenticated/internal Nginx location, request the staged index and representative hashed/media assets, verify the expected 200/status, headers, MIME/range behavior, and return non-zero on any mismatch. Its standard output becomes digest-bound smoke evidence. Do not point public Nginx at the staged directory.
|
||||
|
||||
## Prepare without changing visibility
|
||||
|
||||
Use the deployment account's numeric UID/GID. Preparation verifies signature, exact predecessor, live ledger, fence, hashes, and time; extracts without using the system tar extractor; makes all release files `0444` and directories `0555`; runs the internal smoke; rechecks time and protected state; and writes an immutable readiness receipt. It does not create or change `current`.
|
||||
|
||||
```sh
|
||||
scripts/deploy/prepare-release.sh \
|
||||
<release-root> <new-release.tar> <transaction-id> \
|
||||
<absolute-verifier-module> <external-trust-policy.json> <absolute-smoke-command> \
|
||||
<deployment-uid> <deployment-gid> <trusted-rfc3339-time>
|
||||
```
|
||||
|
||||
Inspect the resulting release ID and readiness receipt before activation. A preparation failure leaves the existing `current` pointer untouched.
|
||||
|
||||
## Fenced atomic activation
|
||||
|
||||
Supply the exact predecessor digest (`none` only for the first release) and the next fence printed by the verified admission:
|
||||
|
||||
```sh
|
||||
scripts/deploy/activate-release.sh \
|
||||
<release-root> <transaction-id> <expected-current-digest-or-none> <expected-next-fence> \
|
||||
<absolute-verifier-module> <external-trust-policy.json> <trusted-rfc3339-time>
|
||||
```
|
||||
|
||||
Activation obtains an exclusive transaction lock, re-verifies the signed envelope against current protected state, rechecks receipt/ledger/predecessor/fence/expiry, writes a durable committing intent, creates the relative `current.next` symlink, and renames it atomically over `current`. Nothing else changes public visibility.
|
||||
|
||||
- A failure before the rename leaves the predecessor visible.
|
||||
- A crash after the rename is recovered by repeating the same command: the script observes the new pointer and finalizes the receipt/state without another visibility change.
|
||||
- A second concurrent or stale invocation fails after the winner advances the pointer/fence.
|
||||
- Never edit `current`, `current.next`, control state, or receipts by hand.
|
||||
|
||||
After activation, run the independent public smoke suite. If it fails, use the signed rollback flow below.
|
||||
|
||||
## Controlled rollback
|
||||
|
||||
Rollback does not replay the selected release's old manifest. The signing pipeline must first create a new candidate whose predecessor is the currently committed manifest, whose ledger/approvals are current, and whose artifact closure is byte-identical to the selected immutable release. Package that new signed envelope as usual, then run:
|
||||
|
||||
```sh
|
||||
scripts/deploy/rollback-release.sh \
|
||||
<release-root> <selected-old-release-id> <newly-signed-rollback-release.tar> <transaction-id> \
|
||||
<absolute-verifier-module> <external-trust-policy.json> <absolute-smoke-command> \
|
||||
<deployment-uid> <deployment-gid> <current-manifest-digest> <expected-next-fence> <trusted-rfc3339-time>
|
||||
```
|
||||
|
||||
The script verifies that the rollback package uses a fresh manifest digest and exactly matches the selected old artifact closure, then performs the same readiness and atomic pointer transaction. Record both activation and rollback receipts in the release record.
|
||||
|
||||
## Orphan staging recovery
|
||||
|
||||
Only hidden directories named `<safe-transaction-id>.partial` beneath the validated dedicated root may be cleaned. Active transaction staging is retained. After protected-state recovery has confirmed that no transaction can resume:
|
||||
|
||||
```sh
|
||||
scripts/deploy/prepare-release.sh <release-root> --cleanup-orphans
|
||||
```
|
||||
|
||||
The cleanup path validates the absolute root and sentinel before traversal, rejects unexpected entries or symlinks, and never touches immutable releases, `current`, or paths outside `<release-root>/staging`.
|
||||
|
||||
## Release record
|
||||
|
||||
For every attempt record, without secrets:
|
||||
|
||||
- package SHA-256 and signed candidate/manifest digest;
|
||||
- build ID, release ID, predecessor, live-ledger head/revision, and fence;
|
||||
- readiness and activation receipt digests;
|
||||
- staged and public smoke outcomes;
|
||||
- selected rollback release and the new rollback manifest digest, when applicable;
|
||||
- operator authorization and UTC timestamps.
|
||||
Reference in New Issue
Block a user