diff --git a/docs/performance/lab-runbook.md b/docs/performance/lab-runbook.md index f5273e4..71c15ee 100644 --- a/docs/performance/lab-runbook.md +++ b/docs/performance/lab-runbook.md @@ -4,11 +4,13 @@ This runbook separates executable developer evidence from the two physical relea ## Automated build and transfer gate -Run `npm run build:preview`. The build ends with `npm run check:budgets`, which classifies every shipped file exactly once, measures Brotli transfer bytes (or raw bytes for already-compressed formats), verifies that the live Three.js scene remains behind a dynamic import, and applies the exact section 14 thresholds. Unknown files fail closed. Every non-empty manifest media file is hash-checked and deterministically decoded: Sharp verifies image dimensions and `ffprobe` verifies video dimensions and duration against the manifest. Missing probe evidence, disagreement, or an unprobeable asset fails the gate. +Run `npm run build:preview`. The build ends with the explicitly named `npm run check:budgets:transfer`, which classifies every shipped file exactly once, measures Brotli transfer bytes (or raw bytes for already-compressed formats), verifies that the live Three.js scene remains behind a dynamic import, and applies the static section 14 thresholds. Unknown files fail closed. Every non-empty manifest media file is hash-checked and deterministically decoded: Sharp verifies image dimensions and `ffprobe` verifies video dimensions and duration against the manifest. Missing probe evidence, disagreement, or an unprobeable asset fails the check. Its report is marked `transfer-only-non-gating` with runtime evidence pending; this static build check is not the full performance budget gate. Runtime budgets are not read from a hand-edited estimate. The authenticated test build records the actual production `SceneResourceBag`: owned texture dimensions/pixel layout/mipmap state, the byte lengths of unique geometry buffers, live render-target dimensions/attachments/samples, and the production framebuffer. A dedicated production-path capture derives peak image/scene and video concurrency from completed browser Resource Timing intervals. The evaluator recomputes decoded bytes, adds the fixed 32 MB allowance, and enforces the Low/High GPU, concurrency, and 2048-pixel mobile texture caps. Adding a production resource or eager request therefore changes the measured ledger. -Run `npm run capture:runtime-budgets`. It loads the real High scene at a device DPR of 2, verifies that the renderer actually reaches its 1.5 DPR cap, and writes `runtime-budget-trace.v1.json` beneath the ignored Playwright result directory. Then verify that exact capture with `npm run check:budgets -- --runtime-trace `. The checker rejects the former standalone evidence schema, a tier/profile mismatch, a missing DPR-cap observation, or framebuffer dimensions that do not match the observed viewport. Supplied totals are ignored and recomputed from raw allocations. +Run `npm run verify:runtime-budgets` for the full executable budget gate. One Node orchestrator runs the authenticated Chromium production-path capture, writes the stable ignored `test-results/runtime-budget-trace.v1.json`, and immediately invokes the full checker with it. It loads the real High scene at a device DPR of 2 and verifies that the renderer actually reaches its 1.5 DPR cap. The checker rejects the former standalone evidence schema, a tier/profile mismatch, a missing DPR-cap observation, or framebuffer dimensions that do not match the observed viewport. Supplied totals are ignored and recomputed from raw allocations. Direct `npm run check:budgets` without `--runtime-trace` fails closed; use `--transfer-only` only through the explicitly non-gating static check. + +`npm run verify:phase3` runs this full runtime gate followed by the complete browser suite. It is the Phase 3 release verification command; `build:preview` alone is deliberately not presented as the full runtime gate. High-tier-only chunks use the `high-tier-*`/`environment-high-*` bucket and are capped separately. diff --git a/package.json b/package.json index 85de25b..1d71057 100644 --- a/package.json +++ b/package.json @@ -18,11 +18,14 @@ "prepare:assets": "node scripts/prepare-assets.mjs", "verify:assets": "node scripts/prepare-assets.mjs --verify", "check:budgets": "node scripts/check-bundle-budgets.mjs dist", + "check:budgets:transfer": "node scripts/check-bundle-budgets.mjs dist --transfer-only", "capture:runtime-budgets": "playwright test tests/performance/runtime-budget-trace.spec.ts --project=runtime-budget-desktop --workers=1", + "verify:runtime-budgets": "node scripts/verify-runtime-budgets.mjs", + "verify:phase3": "npm run verify:runtime-budgets && npm run test:browser", "aggregate:performance": "node scripts/aggregate-drive-traces.mjs", "sign:performance": "node scripts/sign-drive-trace.mjs", "verify:approved-assets": "node scripts/materialize-empty-asset-set.mjs", - "build:preview": "npm run verify:approved-assets && npm run typecheck && npm run test:unit && npm run build:client && npm run build:ssr && npm run prerender -- --config config/site.preview.json && npm run verify:artifacts -- --config config/site.preview.json && npm run check:budgets" + "build:preview": "npm run verify:approved-assets && npm run typecheck && npm run test:unit && npm run build:client && npm run build:ssr && npm run prerender -- --config config/site.preview.json && npm run verify:artifacts -- --config config/site.preview.json && npm run check:budgets:transfer" }, "keywords": [], "author": "", diff --git a/scripts/check-bundle-budgets.mjs b/scripts/check-bundle-budgets.mjs index c131299..3c5ffb5 100644 --- a/scripts/check-bundle-budgets.mjs +++ b/scripts/check-bundle-budgets.mjs @@ -10,13 +10,16 @@ import { classifyArtifact, evaluateBudgetTotals, evaluateMediaAssetBudget, + resolveBudgetGateMode, validateRuntimeBudgetTraceReport, } from './lib/bundle-budgets.mjs'; const root = resolve(process.argv[2] ?? 'dist'); +const transferOnly = process.argv.includes('--transfer-only'); const runtimeTraceIndex = process.argv.indexOf('--runtime-trace'); const runtimeTracePath = runtimeTraceIndex === -1 ? null : process.argv[runtimeTraceIndex + 1]; if (runtimeTraceIndex !== -1 && runtimeTracePath === undefined) throw new Error('--runtime-trace requires one explicit drive-trace JSON path.'); +const gateMode = resolveBudgetGateMode({ transferOnly, runtimeTracePath }); if (!(await stat(root)).isDirectory()) throw new Error('Bundle budget target must be a directory.'); const listFiles = async (directory) => { @@ -121,8 +124,8 @@ if (!/import\([^)]*environment-/u.test(entrySource)) throw new Error('The live s if (/WebGLRenderer|WebGLProgram|ShaderMaterial/u.test(entrySource)) throw new Error('Three.js/live-scene code leaked into the reduced-motion initial chunk.'); let runtime = null; -if (runtimeTracePath !== null) { - const trace = await readJson(resolve(runtimeTracePath), 'Runtime drive trace'); +if (gateMode.kind === 'full') { + const trace = await readJson(resolve(gateMode.runtimeTracePath), 'Runtime drive trace'); runtime = validateRuntimeBudgetTraceReport(trace); } const violations = [...evaluateBudgetTotals(totals).violations, ...mediaViolations, ...(runtime?.violations ?? [])]; @@ -130,7 +133,9 @@ const report = { schema: 'portfolio-bundle-budget/v1', compression: 'brotli-quality-11-or-raw-already-compressed', totals, - runtime: runtime === null ? { status: 'pending-production-path-trace' } : { status: 'verified-production-path-trace', tier: runtime.tier }, + runtime: runtime === null + ? { status: 'transfer-only-non-gating', gating: false, runtimeEvidence: 'pending' } + : { status: 'verified-production-path-trace', gating: true, tier: runtime.tier }, records: records.sort((left, right) => left.path.localeCompare(right.path)), violations, }; diff --git a/scripts/lib/bundle-budgets.d.mts b/scripts/lib/bundle-budgets.d.mts index 86beb1a..b0c62f3 100644 --- a/scripts/lib/bundle-budgets.d.mts +++ b/scripts/lib/bundle-budgets.d.mts @@ -1,4 +1,8 @@ export const PERFORMANCE_BUDGETS: Readonly>; +export function resolveBudgetGateMode(options: { + transferOnly: boolean; + runtimeTracePath: string | null; +}): { kind: 'transfer-only' } | { kind: 'full'; runtimeTracePath: string }; export function classifyArtifact(path: string, source?: string): string; export function evaluateBudgetTotals(totals: Readonly>): { violations: Array<{ budget: string; actual: number; threshold: number }>; diff --git a/scripts/lib/bundle-budgets.mjs b/scripts/lib/bundle-budgets.mjs index 1107cd6..7bd2bfc 100644 --- a/scripts/lib/bundle-budgets.mjs +++ b/scripts/lib/bundle-budgets.mjs @@ -20,6 +20,13 @@ export const PERFORMANCE_BUDGETS = Object.freeze({ highTierGpuEstimate: 192 * 1024 * 1024, }); +export const resolveBudgetGateMode = ({ transferOnly, runtimeTracePath }) => { + if (transferOnly && runtimeTracePath !== null) throw new Error('Transfer-only mode cannot be combined with runtime evidence.'); + if (transferOnly) return { kind: 'transfer-only' }; + if (runtimeTracePath === null) throw new Error('A runtime trace is required for the full performance budget gate. Use --transfer-only only for a non-gating static build check.'); + return { kind: 'full', runtimeTracePath }; +}; + export const classifyArtifact = (path, source = '') => { if (path === 'index.html') return 'initial-html'; if (/^assets\/index-[^/]+\.css$/u.test(path)) return 'critical-css'; diff --git a/scripts/verify-runtime-budgets.mjs b/scripts/verify-runtime-budgets.mjs new file mode 100644 index 0000000..a2506fd --- /dev/null +++ b/scripts/verify-runtime-budgets.mjs @@ -0,0 +1,35 @@ +#!/usr/bin/env node +import { spawn } from 'node:child_process'; +import { access } from 'node:fs/promises'; +import { resolve } from 'node:path'; + +const repositoryRoot = process.cwd(); +const tracePath = resolve(repositoryRoot, 'test-results/runtime-budget-trace.v1.json'); + +const runNode = (args) => new Promise((resolveRun, rejectRun) => { + const child = spawn(process.execPath, args, { + cwd: repositoryRoot, + env: process.env, + stdio: 'inherit', + }); + child.once('error', rejectRun); + child.once('exit', (code, signal) => { + if (code === 0) resolveRun(); + else rejectRun(new Error(`Command failed${signal === null ? ` with exit code ${code}` : ` from signal ${signal}`}.`)); + }); +}); + +await runNode([ + resolve(repositoryRoot, 'node_modules/@playwright/test/cli.js'), + 'test', + 'tests/performance/runtime-budget-trace.spec.ts', + '--project=runtime-budget-desktop', + '--workers=1', +]); +await access(tracePath); +await runNode([ + resolve(repositoryRoot, 'scripts/check-bundle-budgets.mjs'), + 'dist', + '--runtime-trace', + tracePath, +]); diff --git a/tests/performance/budgets.test.ts b/tests/performance/budgets.test.ts index 621ada3..f27a8c8 100644 --- a/tests/performance/budgets.test.ts +++ b/tests/performance/budgets.test.ts @@ -8,9 +8,21 @@ import { validateRuntimeBudgetTraceReport, calculatePeakRequestConcurrency, PERFORMANCE_BUDGETS, + resolveBudgetGateMode, } from '../../scripts/lib/bundle-budgets.mjs'; describe('release budgets', () => { + it('fails closed unless the caller explicitly selects a runtime trace or transfer-only mode', () => { + expect(() => resolveBudgetGateMode({ transferOnly: false, runtimeTracePath: null })) + .toThrow(/runtime trace is required/u); + expect(resolveBudgetGateMode({ transferOnly: true, runtimeTracePath: null })) + .toEqual({ kind: 'transfer-only' }); + expect(resolveBudgetGateMode({ transferOnly: false, runtimeTracePath: '/tmp/runtime.json' })) + .toEqual({ kind: 'full', runtimeTracePath: '/tmp/runtime.json' }); + expect(() => resolveBudgetGateMode({ transferOnly: true, runtimeTracePath: '/tmp/runtime.json' })) + .toThrow(/cannot be combined/u); + }); + it('pins every exact section 14 threshold', () => { expect(PERFORMANCE_BUDGETS).toMatchObject({ initialHtmlCriticalCss: 100 * 1024, diff --git a/tests/performance/runtime-budget-trace.spec.ts b/tests/performance/runtime-budget-trace.spec.ts index a52e46f..ce2f1ca 100644 --- a/tests/performance/runtime-budget-trace.spec.ts +++ b/tests/performance/runtime-budget-trace.spec.ts @@ -1,4 +1,5 @@ import { writeFile } from 'node:fs/promises'; +import { resolve } from 'node:path'; import { expect, test } from '../helpers/serve-dist'; import { authorizedPerformanceUrl } from '../helpers/test-build-auth'; import { @@ -48,9 +49,13 @@ test('runtime-budget-trace.v1 captures the actual High production allocation at }; const evaluated = validateRuntimeBudgetTraceReport(report); expect(evaluated.violations).toEqual([]); - await writeFile(testInfo.outputPath('runtime-budget-trace.v1.json'), `${JSON.stringify({ + const serialized = `${JSON.stringify({ ...report, runtimeBudgetSummary: evaluated.tier, violations: evaluated.violations, - }, null, 2)}\n`); + }, null, 2)}\n`; + await Promise.all([ + writeFile(testInfo.outputPath('runtime-budget-trace.v1.json'), serialized), + writeFile(resolve('test-results/runtime-budget-trace.v1.json'), serialized), + ]); });