Portfolio/docs/performance/lab-runbook.md

Portfolio performance lab runbook

This runbook separates executable developer evidence from the two physical release gates. A passing laptop/headless run never substitutes for the named hardware.

Automated build and transfer gate

Run npm run build:preview. The build ends with the explicitly named npm run check:budgets:transfer, which classifies every shipped file exactly once, measures Brotli transfer bytes (or raw bytes for already-compressed formats), verifies that the live Three.js scene remains behind a dynamic import, and applies the static section 14 thresholds. Unknown files fail closed. Every non-empty manifest media file is hash-checked and deterministically decoded: Sharp verifies image dimensions and ffprobe verifies video dimensions and duration against the manifest. Missing probe evidence, disagreement, or an unprobeable asset fails the check. Its report is marked transfer-only-non-gating with runtime evidence pending; this static build check is not the full performance budget gate.

Runtime budgets are not read from a hand-edited estimate. The authenticated test build records the actual production SceneResourceBag: owned texture dimensions/pixel layout/mipmap state, the byte lengths of unique geometry buffers, live render-target dimensions/attachments/samples, and the production framebuffer. A dedicated production-path capture derives peak image/scene and video concurrency from completed browser Resource Timing intervals. The evaluator recomputes decoded bytes, adds the fixed 32 MB allowance, and enforces the Low/High GPU, concurrency, and 2048-pixel mobile texture caps. Adding a production resource or eager request therefore changes the measured ledger.

Run npm run verify:runtime-budgets for the full executable budget gate. One Node orchestrator runs the authenticated Chromium production-path capture, writes the stable ignored test-results/runtime-budget-trace.v1.json, and immediately invokes the full checker with it. It loads the real High scene at a device DPR of 2 and verifies that the renderer actually reaches its 1.5 DPR cap. The checker rejects the former standalone evidence schema, a tier/profile mismatch, a missing DPR-cap observation, or framebuffer dimensions that do not match the observed viewport. Supplied totals are ignored and recomputed from raw allocations. Direct npm run check:budgets without --runtime-trace fails closed; use --transfer-only only through the explicitly non-gating static check.

npm run verify:phase3 runs this full runtime gate followed by the complete browser suite. It is the Phase 3 release verification command; build:preview alone is deliberately not presented as the full runtime gate.

High-tier-only chunks use the high-tier-*/environment-high-* bucket and are capped separately.

The current empty-media preview measures approximately 7.8 KB for HTML plus critical CSS, 61.8 KB for initial application JavaScript, and 114.2 KB for the deferred first live scene. These values are build observations, not permanent allowances.

Current-Chromium developer trace

Run:

npx playwright test tests/performance/drive-trace.spec.ts \
  --project=performance-desktop --project=performance-mobile --workers=1

The two 20-second traces use current Chromium, trusted CDP mouse/key/wheel input, the specified desktop/mobile network profiles, exact action offsets, fixed High/Low rendering tiers, the deterministic Toolkit cancellation barrier, and one fresh browser context per run. Each test writes drive-trace.v1.json under its ignored Playwright result directory.

Developer reports deliberately contain:

• evidenceKind: current-chromium-developer-machine-non-reference
• gatingReferenceResult: false
• individual budget booleans and measured values
• pendingPhysicalReferenceRuns: 5

They prove that the harness and interactions execute. They do not make a release claim, even if every number happens to pass. Instrumentation exists only in an authenticated test build, stays inside the page/test process, records no GPU or audio-device identity, and sends no telemetry.

Five-run physical reference gate

Use only the Pixel 6 8 GB / Chromium Mobile Low and M1 MacBook Air 8 GB / Chromium Desktop High reference classes, or hardware with the same CPU/GPU and RAM. Fix both displays at 60 Hz; require battery above 50%, power saving off, a cool start, no foreground competition, page zoom 100%, full motion, Save-Data off, and the exact network shaping from section 14. Record OS/browser/display versions in the private lab evidence, not in shipped artifacts.

For each profile, run five fresh temporary profiles from about:blank. Clear service workers, Cache Storage, HTTP cache, cookies, local/session storage, and IndexedDB before every run.

Reference mode is fail-closed. Create one private portfolio-reference-run-attestation/v1 JSON file per run with the exact pinned device, Chromium/OS/display, viewport, network, run index/ID, and every environmental assertion recorded. Keep device metadata outside the public repository. Compute the SHA-256 of the exact file bytes, then run one trace with all three variables set:

PORTFOLIO_REFERENCE_LAB=1 \
PORTFOLIO_REFERENCE_ATTESTATION_PATH=/private/evidence/desktop-run-1.json \
PORTFOLIO_REFERENCE_ATTESTATION_SHA256=<64-lowercase-hex-digest> \
npx playwright test tests/performance/drive-trace.spec.ts \
  --project=performance-desktop --workers=1

Reference mode rejects missing or extra fields, digest mismatch, wrong hardware/network/profile, failed setup assertions, or disagreement with the browser-observed viewport, visibility, reduced-motion, Save-Data, zoom, and Chromium version. The trace still emits an unsigned validated-reference-candidate-unsigned with gatingReferenceResult: false; ordinary developer runs are also unsigned and false. A warm-up median outside 16.667 ±1 ms or any per-run budget failure invalidates the candidate.

After independent device/evidence review, an authorized lab signer uses an Ed25519 private key held outside the repository to sign the exact candidate. Never copy the private key into the repository, result directory, command output, or trust policy:

npm run sign:performance -- \
  --key-id lab-release-2026 \
  --private-key /private/keys/reference-ed25519.pem \
  candidate.json signed-run-1.json

Signing binds the complete measurements, budget booleans, runtime/request ledgers, profile, attestation digest, run ID, and run index to a result digest. It is the only operation that changes gatingReferenceResult to true. The external release trust policy contains only the approved key ID and Ed25519 public key; it is selected by the release controller, never from a result file.

Aggregate exactly five reviewed JSON paths:

npm run aggregate:performance -- \
  --trust-policy /approved/policy/reference-trust-policy.json \
  signed-run-1.json signed-run-2.json signed-run-3.json signed-run-4.json signed-run-5.json

Aggregation verifies every result digest and Ed25519 signature against that explicit trusted policy. It rejects unsigned or tampered reports, a result-supplied/self-selected key, duplicate/replayed identities or digests, incomplete indexes, mixed profiles, and failed gates. Five unique indexes 1–5 are required; worst/median LCP remain at most 3.0/2.5 seconds and worst/median TBT at most 300/200 ms. Run and aggregate the other profile separately. Do not commit result files until their metadata has passed privacy review.

The former High DPR cap of 2 produced a conservative estimate above the 192 MB cap once the antialiased framebuffer, post target, route-sign texture, geometry, and fixed allowance were counted. High is therefore capped at DPR 1.5. The browser-produced ledger is now the release evidence rather than that former declaration.

Physical ten-minute protocol

tests/performance/physical-drive.protocol.ts is the versioned physical-drive.v1 action generator. A platform runner must consume it without changing timing or order. It uses only driveTouch touch and driveKeys key sources, round-half-up safe-viewport coordinates, 15×40 ms swipe moves plus a 400 ms pause, and the direction rule sampled at every cycle start.

The generator includes rotations, 30-second background periods, audio interruption, context loss/restore, the 389/390-second cycle exclusions, the 390-second Infrastructure barrier and 391-second production-epoch cancellation, live versus terminal-Static pause/resume branches, and Contact settlement at ten minutes. Mouse/pen fallback, scripted click/focus, altered offsets, or emulation in place of the physical devices invalidates evidence.

Physical iPhone/Pixel execution remains pending until those devices and platform automation drivers are available. Never mark those rows passed from desktop emulation.