fix: bind performance evidence to production resources
This commit is contained in:
@@ -1,27 +0,0 @@
|
||||
{
|
||||
"schema": "portfolio-runtime-budget-evidence/v1",
|
||||
"peakRequests": { "imageScene": 0, "video": 0 },
|
||||
"tiers": [
|
||||
{
|
||||
"tier": "low",
|
||||
"textures": [
|
||||
{ "id": "route-sign-canvas", "width": 768, "height": 192, "bytesPerPixel": 4, "layers": 1, "mipmapped": true }
|
||||
],
|
||||
"geometryBytes": 1048576,
|
||||
"renderTargets": [
|
||||
{ "id": "default-framebuffer", "width": 488, "height": 1055, "bytesPerPixel": 4, "attachments": 2, "samples": 1 }
|
||||
]
|
||||
},
|
||||
{
|
||||
"tier": "high",
|
||||
"textures": [
|
||||
{ "id": "route-sign-canvas", "width": 768, "height": 192, "bytesPerPixel": 4, "layers": 1, "mipmapped": true }
|
||||
],
|
||||
"geometryBytes": 2097152,
|
||||
"renderTargets": [
|
||||
{ "id": "default-framebuffer", "width": 2160, "height": 1350, "bytesPerPixel": 4, "attachments": 2, "samples": 4 },
|
||||
{ "id": "post-processing-target", "width": 2160, "height": 1350, "bytesPerPixel": 4, "attachments": 2, "samples": 1 }
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -6,7 +6,11 @@ This runbook separates executable developer evidence from the two physical relea
|
||||
|
||||
Run `npm run build:preview`. The build ends with `npm run check:budgets`, which classifies every shipped file exactly once, measures Brotli transfer bytes (or raw bytes for already-compressed formats), verifies that the live Three.js scene remains behind a dynamic import, and applies the exact section 14 thresholds. Unknown files fail closed. Every non-empty manifest media file is hash-checked and deterministically decoded: Sharp verifies image dimensions and `ffprobe` verifies video dimensions and duration against the manifest. Missing probe evidence, disagreement, or an unprobeable asset fails the gate.
|
||||
|
||||
`config/performance-budget-evidence.json` is the auditable allocation declaration. The checker derives texture bytes as `width × height × bytes-per-pixel × layers × 4/3` for mipmaps, render-target bytes as `width × height × bytes-per-pixel × attachments × samples`, adds declared geometry buffer bytes and the fixed 32 MB allowance, and enforces the Low/High GPU caps. It also enforces peak request concurrency and the 2048-pixel mobile texture bound. High-tier-only chunks use the `high-tier-*`/`environment-high-*` bucket and are capped separately. Any scene allocation or request-scheduling change must update this declaration in the same change; incomplete allocations fail closed.
|
||||
Runtime budgets are not read from a hand-edited estimate. The authenticated test build records the actual production `SceneResourceBag`: owned texture dimensions/pixel layout/mipmap state, the byte lengths of unique geometry buffers, live render-target dimensions/attachments/samples, and the production framebuffer. A dedicated production-path capture derives peak image/scene and video concurrency from completed browser Resource Timing intervals. The evaluator recomputes decoded bytes, adds the fixed 32 MB allowance, and enforces the Low/High GPU, concurrency, and 2048-pixel mobile texture caps. Adding a production resource or eager request therefore changes the measured ledger.
|
||||
|
||||
Run `npm run capture:runtime-budgets`. It loads the real High scene at a device DPR of 2, verifies that the renderer actually reaches its 1.5 DPR cap, and writes `runtime-budget-trace.v1.json` beneath the ignored Playwright result directory. Then verify that exact capture with `npm run check:budgets -- --runtime-trace <runtime-budget-trace.v1.json>`. The checker rejects the former standalone evidence schema, a tier/profile mismatch, a missing DPR-cap observation, or framebuffer dimensions that do not match the observed viewport. Supplied totals are ignored and recomputed from raw allocations.
|
||||
|
||||
High-tier-only chunks use the `high-tier-*`/`environment-high-*` bucket and are capped separately.
|
||||
|
||||
The current empty-media preview measures approximately 7.8 KB for HTML plus critical CSS, 61.8 KB for initial application JavaScript, and 114.2 KB for the deferred first live scene. These values are build observations, not permanent allowances.
|
||||
|
||||
@@ -46,17 +50,30 @@ npx playwright test tests/performance/drive-trace.spec.ts \
|
||||
--project=performance-desktop --workers=1
|
||||
```
|
||||
|
||||
Reference mode rejects missing or extra fields, digest mismatch, wrong hardware/network/profile, failed setup assertions, or disagreement with the browser-observed viewport, visibility, reduced-motion, Save-Data, zoom, and Chromium version. Only a validated attestation can produce `gatingReferenceResult: true`; ordinary developer runs always remain false. A warm-up median outside 16.667 ±1 ms or any per-run budget failure invalidates the run.
|
||||
Reference mode rejects missing or extra fields, digest mismatch, wrong hardware/network/profile, failed setup assertions, or disagreement with the browser-observed viewport, visibility, reduced-motion, Save-Data, zoom, and Chromium version. The trace still emits an unsigned `validated-reference-candidate-unsigned` with `gatingReferenceResult: false`; ordinary developer runs are also unsigned and false. A warm-up median outside 16.667 ±1 ms or any per-run budget failure invalidates the candidate.
|
||||
|
||||
After independent device/evidence review, an authorized lab signer uses an Ed25519 private key held outside the repository to sign the exact candidate. Never copy the private key into the repository, result directory, command output, or trust policy:
|
||||
|
||||
```bash
|
||||
npm run sign:performance -- \
|
||||
--key-id lab-release-2026 \
|
||||
--private-key /private/keys/reference-ed25519.pem \
|
||||
candidate.json signed-run-1.json
|
||||
```
|
||||
|
||||
Signing binds the complete measurements, budget booleans, runtime/request ledgers, profile, attestation digest, run ID, and run index to a result digest. It is the only operation that changes `gatingReferenceResult` to true. The external release trust policy contains only the approved key ID and Ed25519 public key; it is selected by the release controller, never from a result file.
|
||||
|
||||
Aggregate exactly five reviewed JSON paths:
|
||||
|
||||
```bash
|
||||
npm run aggregate:performance -- run-1.json run-2.json run-3.json run-4.json run-5.json
|
||||
npm run aggregate:performance -- \
|
||||
--trust-policy /approved/policy/reference-trust-policy.json \
|
||||
signed-run-1.json signed-run-2.json signed-run-3.json signed-run-4.json signed-run-5.json
|
||||
```
|
||||
|
||||
Aggregation fails unless the set contains unique authenticated run IDs, digests, and indexes 1–5 for one profile, all five are marked reference evidence, every per-run gate passes, worst/median LCP are at most 3.0/2.5 seconds, and worst/median TBT are at most 300/200 ms. Run and aggregate the other profile separately. Do not commit result files until their metadata has passed privacy review.
|
||||
Aggregation verifies every result digest and Ed25519 signature against that explicit trusted policy. It rejects unsigned or tampered reports, a result-supplied/self-selected key, duplicate/replayed identities or digests, incomplete indexes, mixed profiles, and failed gates. Five unique indexes 1–5 are required; worst/median LCP remain at most 3.0/2.5 seconds and worst/median TBT at most 300/200 ms. Run and aggregate the other profile separately. Do not commit result files until their metadata has passed privacy review.
|
||||
|
||||
The former High DPR cap of 2 produced a conservative 242 MB estimate at 1440×900 once the antialiased framebuffer, post target, route-sign texture, geometry, and fixed allowance were counted. High is therefore capped at DPR 1.5. The executable declaration now derives 153,078,016 bytes (about 146 MB) for High and 39,508,160 bytes (about 37.7 MB) for Low, below the exact 192/96 MB limits.
|
||||
The former High DPR cap of 2 produced a conservative estimate above the 192 MB cap once the antialiased framebuffer, post target, route-sign texture, geometry, and fixed allowance were counted. High is therefore capped at DPR 1.5. The browser-produced ledger is now the release evidence rather than that former declaration.
|
||||
|
||||
## Physical ten-minute protocol
|
||||
|
||||
|
||||
@@ -18,7 +18,9 @@
|
||||
"prepare:assets": "node scripts/prepare-assets.mjs",
|
||||
"verify:assets": "node scripts/prepare-assets.mjs --verify",
|
||||
"check:budgets": "node scripts/check-bundle-budgets.mjs dist",
|
||||
"capture:runtime-budgets": "playwright test tests/performance/runtime-budget-trace.spec.ts --project=runtime-budget-desktop --workers=1",
|
||||
"aggregate:performance": "node scripts/aggregate-drive-traces.mjs",
|
||||
"sign:performance": "node scripts/sign-drive-trace.mjs",
|
||||
"verify:approved-assets": "node scripts/materialize-empty-asset-set.mjs",
|
||||
"build:preview": "npm run verify:approved-assets && npm run typecheck && npm run test:unit && npm run build:client && npm run build:ssr && npm run prerender -- --config config/site.preview.json && npm run verify:artifacts -- --config config/site.preview.json && npm run check:budgets"
|
||||
},
|
||||
|
||||
@@ -10,7 +10,8 @@ export default defineConfig({
|
||||
projects: [
|
||||
{ name: 'desktop-chromium', testMatch: /browser\/.*\.spec\.ts/u, use: { ...devices['Desktop Chrome'] } },
|
||||
{ name: 'mobile-chromium', testMatch: /browser\/.*\.spec\.ts/u, use: { ...devices['Pixel 5'] } },
|
||||
{ name: 'performance-desktop', testMatch: /performance\/drive-trace\.spec\.ts/u, use: { ...devices['Desktop Chrome'] } },
|
||||
{ name: 'performance-mobile', testMatch: /performance\/drive-trace\.spec\.ts/u, use: { ...devices['Desktop Chrome'], viewport: { width: 390, height: 844 }, isMobile: true, hasTouch: true } },
|
||||
{ name: 'performance-desktop', testMatch: /performance\/drive-trace\.spec\.ts/u, use: { ...devices['Desktop Chrome'], deviceScaleFactor: 2 } },
|
||||
{ name: 'performance-mobile', testMatch: /performance\/drive-trace\.spec\.ts/u, use: { ...devices['Desktop Chrome'], viewport: { width: 390, height: 844 }, deviceScaleFactor: 3, isMobile: true, hasTouch: true } },
|
||||
{ name: 'runtime-budget-desktop', testMatch: /performance\/runtime-budget-trace\.spec\.ts/u, use: { ...devices['Desktop Chrome'], deviceScaleFactor: 2 } },
|
||||
],
|
||||
});
|
||||
|
||||
@@ -2,10 +2,13 @@
|
||||
import { readFile } from 'node:fs/promises';
|
||||
import { aggregateFiveDriveTraces } from './lib/performance-aggregation.mjs';
|
||||
|
||||
const paths = process.argv.slice(2);
|
||||
const args = process.argv.slice(2);
|
||||
if (args[0] !== '--trust-policy' || typeof args[1] !== 'string') throw new Error('Pass --trust-policy followed by an explicit trusted public-key policy.');
|
||||
const policy = JSON.parse(await readFile(args[1], 'utf8'));
|
||||
const paths = args.slice(2);
|
||||
if (paths.length !== 5) throw new Error('Pass exactly five explicit drive-trace.v1 JSON paths.');
|
||||
const runs = [];
|
||||
for (const path of paths) runs.push(JSON.parse(await readFile(path, 'utf8')));
|
||||
const aggregate = aggregateFiveDriveTraces(runs);
|
||||
const aggregate = aggregateFiveDriveTraces(runs, policy);
|
||||
process.stdout.write(`${JSON.stringify(aggregate, null, 2)}\n`);
|
||||
if (!aggregate.passes) process.exitCode = 1;
|
||||
|
||||
@@ -10,10 +10,13 @@ import {
|
||||
classifyArtifact,
|
||||
evaluateBudgetTotals,
|
||||
evaluateMediaAssetBudget,
|
||||
evaluateRuntimeBudgetEvidence,
|
||||
validateRuntimeBudgetTraceReport,
|
||||
} from './lib/bundle-budgets.mjs';
|
||||
|
||||
const root = resolve(process.argv[2] ?? 'dist');
|
||||
const runtimeTraceIndex = process.argv.indexOf('--runtime-trace');
|
||||
const runtimeTracePath = runtimeTraceIndex === -1 ? null : process.argv[runtimeTraceIndex + 1];
|
||||
if (runtimeTraceIndex !== -1 && runtimeTracePath === undefined) throw new Error('--runtime-trace requires one explicit drive-trace JSON path.');
|
||||
if (!(await stat(root)).isDirectory()) throw new Error('Bundle budget target must be a directory.');
|
||||
|
||||
const listFiles = async (directory) => {
|
||||
@@ -38,7 +41,6 @@ const readJson = async (path, label) => {
|
||||
|
||||
const manifest = await readJson(resolve('content/public/asset-manifest.json'), 'Public asset manifest');
|
||||
const content = await readJson(resolve('content/public/portfolio-content.json'), 'Portfolio content');
|
||||
const runtimeEvidence = await readJson(resolve('config/performance-budget-evidence.json'), 'Runtime budget evidence');
|
||||
if (!Array.isArray(manifest?.assets)) throw new Error('Public asset manifest is invalid.');
|
||||
|
||||
const assetClassById = new Map();
|
||||
@@ -118,13 +120,17 @@ const entrySource = await readFile(resolve(root, entry.path), 'utf8');
|
||||
if (!/import\([^)]*environment-/u.test(entrySource)) throw new Error('The live scene is not separated behind a dynamic import.');
|
||||
if (/WebGLRenderer|WebGLProgram|ShaderMaterial/u.test(entrySource)) throw new Error('Three.js/live-scene code leaked into the reduced-motion initial chunk.');
|
||||
|
||||
const runtime = evaluateRuntimeBudgetEvidence(runtimeEvidence);
|
||||
const violations = [...evaluateBudgetTotals(totals).violations, ...mediaViolations, ...runtime.violations];
|
||||
let runtime = null;
|
||||
if (runtimeTracePath !== null) {
|
||||
const trace = await readJson(resolve(runtimeTracePath), 'Runtime drive trace');
|
||||
runtime = validateRuntimeBudgetTraceReport(trace);
|
||||
}
|
||||
const violations = [...evaluateBudgetTotals(totals).violations, ...mediaViolations, ...(runtime?.violations ?? [])];
|
||||
const report = {
|
||||
schema: 'portfolio-bundle-budget/v1',
|
||||
compression: 'brotli-quality-11-or-raw-already-compressed',
|
||||
totals,
|
||||
runtime: { peakRequests: runtimeEvidence.peakRequests, tiers: runtime.tiers },
|
||||
runtime: runtime === null ? { status: 'pending-production-path-trace' } : { status: 'verified-production-path-trace', tier: runtime.tier },
|
||||
records: records.sort((left, right) => left.path.localeCompare(right.path)),
|
||||
violations,
|
||||
};
|
||||
|
||||
@@ -13,3 +13,15 @@ export function evaluateRuntimeBudgetEvidence(evidence: Readonly<Record<string,
|
||||
tiers: Array<{ tier: string; textureBytes: number; geometryBytes: number; renderTargetBytes: number; runtimeAllowanceBytes: number; totalBytes: number }>;
|
||||
violations: Array<{ budget: string; actual: number; threshold: number; id?: string }>;
|
||||
};
|
||||
export function evaluateRuntimeTraceBudgetEvidence(evidence: Readonly<Record<string, unknown>>): {
|
||||
tier: { tier: string; textureBytes: number; geometryBytes: number; renderTargetBytes: number; runtimeAllowanceBytes: number; totalBytes: number };
|
||||
violations: Array<{ budget: string; actual: number; threshold: number; id?: string }>;
|
||||
};
|
||||
export function validateRuntimeBudgetTraceReport(report: Readonly<Record<string, unknown>>): {
|
||||
tier: { tier: string; textureBytes: number; geometryBytes: number; renderTargetBytes: number; runtimeAllowanceBytes: number; totalBytes: number };
|
||||
violations: Array<{ budget: string; actual: number; threshold: number; id?: string }>;
|
||||
};
|
||||
export function calculatePeakRequestConcurrency(entries: readonly Readonly<Record<string, unknown>>[]): {
|
||||
imageScene: number;
|
||||
video: number;
|
||||
};
|
||||
|
||||
@@ -133,3 +133,85 @@ export const evaluateRuntimeBudgetEvidence = (evidence) => {
|
||||
});
|
||||
return { tiers, violations };
|
||||
};
|
||||
|
||||
export const evaluateRuntimeTraceBudgetEvidence = (evidence) => {
|
||||
if (evidence?.schema !== 'portfolio-runtime-trace-evidence/v1' || evidence.allocation === null || typeof evidence.allocation !== 'object') {
|
||||
throw new Error('Runtime trace evidence must use portfolio-runtime-trace-evidence/v1.');
|
||||
}
|
||||
const evaluated = evaluateRuntimeBudgetEvidence({
|
||||
schema: 'portfolio-runtime-budget-evidence/v1',
|
||||
peakRequests: evidence.peakRequests,
|
||||
tiers: [
|
||||
evidence.allocation,
|
||||
evidence.allocation.tier === 'low'
|
||||
? { tier: 'high', textures: [], geometryBytes: 0, renderTargets: [] }
|
||||
: { tier: 'low', textures: [], geometryBytes: 0, renderTargets: [] },
|
||||
],
|
||||
});
|
||||
const tier = evaluated.tiers.find((item) => item.tier === evidence.allocation.tier);
|
||||
if (tier === undefined) throw new Error('Runtime trace tier could not be evaluated.');
|
||||
return {
|
||||
tier,
|
||||
violations: evaluated.violations,
|
||||
};
|
||||
};
|
||||
|
||||
export const validateRuntimeBudgetTraceReport = (report) => {
|
||||
if (report?.schema !== 'runtime-budget-trace.v1'
|
||||
|| report.evidenceKind !== 'authenticated-test-build-production-path'
|
||||
|| report.gatingReferenceResult !== false) {
|
||||
throw new Error('Runtime budget verification requires an authenticated production-path capture.');
|
||||
}
|
||||
const expectedProfile = report.tier === 'high' ? 'desktop-high' : report.tier === 'low' ? 'mobile-low' : null;
|
||||
if (expectedProfile === null || report.profile !== expectedProfile) {
|
||||
throw new Error('Runtime budget capture profile and tier disagree.');
|
||||
}
|
||||
const viewport = report.observed?.viewport;
|
||||
const devicePixelRatio = report.observed?.devicePixelRatio;
|
||||
const rendererPixelRatio = report.observed?.rendererPixelRatio;
|
||||
const expectedRendererPixelRatio = report.tier === 'high' ? 1.5 : 1.25;
|
||||
if (!finitePositiveInteger(viewport?.width) || !finitePositiveInteger(viewport?.height)
|
||||
|| !(Number.isFinite(devicePixelRatio) && devicePixelRatio >= expectedRendererPixelRatio)
|
||||
|| rendererPixelRatio !== expectedRendererPixelRatio) {
|
||||
throw new Error('Runtime budget capture did not exercise the named tier DPR cap.');
|
||||
}
|
||||
const evidence = report.runtimeBudgetEvidence;
|
||||
if (evidence?.allocation?.tier !== report.tier || !Array.isArray(evidence.allocation.renderTargets)) {
|
||||
throw new Error('Runtime budget capture allocation and tier disagree.');
|
||||
}
|
||||
const framebuffers = evidence.allocation.renderTargets.filter((target) => target?.id === 'default-framebuffer');
|
||||
const expectedWidth = Math.round(viewport.width * rendererPixelRatio);
|
||||
const expectedHeight = Math.round(viewport.height * rendererPixelRatio);
|
||||
if (framebuffers.length !== 1 || framebuffers[0].width !== expectedWidth || framebuffers[0].height !== expectedHeight) {
|
||||
throw new Error('Runtime budget capture framebuffer dimensions do not match the observed viewport and DPR cap.');
|
||||
}
|
||||
return evaluateRuntimeTraceBudgetEvidence(evidence);
|
||||
};
|
||||
|
||||
const requestClass = (entry) => {
|
||||
if (entry.initiatorType === 'video' || /\.(?:mp4|webm)(?:[?#]|$)/iu.test(entry.name)) return 'video';
|
||||
if (entry.initiatorType === 'img' || /\/assets\/(?:environment-|high-tier-)/u.test(entry.name)) return 'imageScene';
|
||||
return null;
|
||||
};
|
||||
|
||||
export const calculatePeakRequestConcurrency = (entries) => {
|
||||
if (!Array.isArray(entries)) throw new Error('Resource timing entries are required.');
|
||||
const peaks = { imageScene: 0, video: 0 };
|
||||
for (const kind of ['imageScene', 'video']) {
|
||||
const events = [];
|
||||
for (const entry of entries) {
|
||||
if (requestClass(entry) !== kind) continue;
|
||||
if (!Number.isFinite(entry.startTime) || !Number.isFinite(entry.responseEnd) || entry.responseEnd < entry.startTime) {
|
||||
throw new Error('Tracked resource timing is incomplete.');
|
||||
}
|
||||
events.push({ at: entry.startTime, delta: 1 }, { at: entry.responseEnd, delta: -1 });
|
||||
}
|
||||
events.sort((left, right) => left.at - right.at || left.delta - right.delta);
|
||||
let active = 0;
|
||||
for (const event of events) {
|
||||
active += event.delta;
|
||||
peaks[kind] = Math.max(peaks[kind], active);
|
||||
}
|
||||
}
|
||||
return peaks;
|
||||
};
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
export function aggregateFiveDriveTraces(runs: readonly unknown[]): {
|
||||
export function aggregateFiveDriveTraces(runs: readonly unknown[], trustPolicy: Readonly<Record<string, unknown>>): {
|
||||
schema: 'drive-trace-aggregation/v1';
|
||||
profile: string;
|
||||
runCount: 5;
|
||||
|
||||
@@ -1,11 +1,14 @@
|
||||
import { verifySignedReferenceRun } from './reference-evidence.mjs';
|
||||
|
||||
const median = (values) => {
|
||||
if (values.length === 0) throw new Error('At least one value is required.');
|
||||
const sorted = [...values].sort((left, right) => left - right);
|
||||
return sorted[Math.floor(sorted.length / 2)];
|
||||
};
|
||||
|
||||
export const aggregateFiveDriveTraces = (runs) => {
|
||||
export const aggregateFiveDriveTraces = (runs, trustPolicy) => {
|
||||
if (!Array.isArray(runs) || runs.length !== 5) throw new Error('Exactly five drive-trace.v1 runs are required.');
|
||||
for (const run of runs) verifySignedReferenceRun(run, trustPolicy);
|
||||
const profile = runs[0]?.profile;
|
||||
if (typeof profile !== 'string' || runs.some((run) => run?.schema !== 'drive-trace.v1' || run.profile !== profile)) {
|
||||
throw new Error('All runs must use one drive-trace.v1 profile.');
|
||||
|
||||
10
scripts/lib/reference-evidence.d.mts
Normal file
10
scripts/lib/reference-evidence.d.mts
Normal file
@@ -0,0 +1,10 @@
|
||||
import type { KeyLike } from 'node:crypto';
|
||||
|
||||
export function signReferenceCandidate(
|
||||
candidate: Readonly<Record<string, unknown>>,
|
||||
signer: { keyId: string; privateKey: KeyLike },
|
||||
): Record<string, unknown>;
|
||||
export function verifySignedReferenceRun(
|
||||
run: Readonly<Record<string, unknown>>,
|
||||
policy: { schema: string; keyId: string; publicKeyPem: string },
|
||||
): Record<string, unknown>;
|
||||
82
scripts/lib/reference-evidence.mjs
Normal file
82
scripts/lib/reference-evidence.mjs
Normal file
@@ -0,0 +1,82 @@
|
||||
import { createHash, createPublicKey, sign, verify } from 'node:crypto';
|
||||
|
||||
const domain = Buffer.from('portfolio-drive-trace-result/v1\0', 'utf8');
|
||||
|
||||
const canonicalize = (value) => {
|
||||
if (value === null || typeof value === 'boolean' || typeof value === 'string') return JSON.stringify(value);
|
||||
if (typeof value === 'number') {
|
||||
if (!Number.isFinite(value)) throw new Error('Reference evidence cannot contain non-finite numbers.');
|
||||
return JSON.stringify(value);
|
||||
}
|
||||
if (Array.isArray(value)) return `[${value.map(canonicalize).join(',')}]`;
|
||||
if (typeof value === 'object') {
|
||||
const entries = Object.entries(value).sort(([left], [right]) => left.localeCompare(right, 'en-US'));
|
||||
if (entries.some(([, item]) => item === undefined)) throw new Error('Reference evidence cannot contain undefined values.');
|
||||
return `{${entries.map(([key, item]) => `${JSON.stringify(key)}:${canonicalize(item)}`).join(',')}}`;
|
||||
}
|
||||
throw new Error('Reference evidence contains an unsupported value.');
|
||||
};
|
||||
|
||||
const signingPayload = (run) => {
|
||||
if (run === null || typeof run !== 'object' || Array.isArray(run)) throw new Error('Reference result must be an object.');
|
||||
const { referenceProof: _proof, ...payload } = run;
|
||||
return payload;
|
||||
};
|
||||
|
||||
const encodedPayload = (payload) => Buffer.concat([domain, Buffer.from(canonicalize(payload), 'utf8')]);
|
||||
|
||||
export const signReferenceCandidate = (candidate, signer) => {
|
||||
if (candidate?.schema !== 'drive-trace.v1' || candidate.evidenceKind !== 'validated-reference-candidate-unsigned'
|
||||
|| candidate.gatingReferenceResult !== false || candidate.referenceCandidateValidated !== true
|
||||
|| candidate.referenceBudgetsPass !== true || candidate.validReferenceCadence !== true
|
||||
|| !Object.values(candidate.budgetResults ?? {}).every((value) => value === true)) {
|
||||
throw new Error('Only a fully validated unsigned reference candidate can be signed.');
|
||||
}
|
||||
if (typeof signer?.keyId !== 'string' || !/^[a-z0-9][a-z0-9-]{2,79}$/u.test(signer.keyId) || signer.privateKey === undefined) {
|
||||
throw new Error('An external Ed25519 signing identity is required.');
|
||||
}
|
||||
const payload = {
|
||||
...candidate,
|
||||
evidenceKind: 'signed-physical-reference',
|
||||
gatingReferenceResult: true,
|
||||
};
|
||||
const bytes = encodedPayload(payload);
|
||||
const resultDigest = createHash('sha256').update(bytes).digest('hex');
|
||||
const signature = sign(null, bytes, signer.privateKey).toString('base64url');
|
||||
return {
|
||||
...payload,
|
||||
referenceProof: {
|
||||
schema: 'portfolio-reference-result-proof/v1',
|
||||
algorithm: 'Ed25519',
|
||||
encoding: 'base64url-nopad',
|
||||
keyId: signer.keyId,
|
||||
resultDigest,
|
||||
signature,
|
||||
},
|
||||
};
|
||||
};
|
||||
|
||||
export const verifySignedReferenceRun = (run, policy) => {
|
||||
if (policy?.schema !== 'portfolio-reference-trust-policy/v1'
|
||||
|| typeof policy.keyId !== 'string' || typeof policy.publicKeyPem !== 'string') {
|
||||
throw new Error('An explicit trusted reference public-key policy is required.');
|
||||
}
|
||||
const proof = run?.referenceProof;
|
||||
if (run?.schema !== 'drive-trace.v1' || run.evidenceKind !== 'signed-physical-reference'
|
||||
|| run.gatingReferenceResult !== true || proof?.schema !== 'portfolio-reference-result-proof/v1'
|
||||
|| proof.algorithm !== 'Ed25519' || proof.encoding !== 'base64url-nopad' || proof.keyId !== policy.keyId
|
||||
|| !/^[a-f0-9]{64}$/u.test(proof.resultDigest ?? '') || !/^[A-Za-z0-9_-]+$/u.test(proof.signature ?? '')) {
|
||||
throw new Error('Reference result signature metadata is invalid.');
|
||||
}
|
||||
const payload = signingPayload(run);
|
||||
const bytes = encodedPayload(payload);
|
||||
const digest = createHash('sha256').update(bytes).digest('hex');
|
||||
if (digest !== proof.resultDigest) throw new Error('Reference result digest mismatch.');
|
||||
let publicKey;
|
||||
try { publicKey = createPublicKey(policy.publicKeyPem); } catch { throw new Error('Trusted reference public key is invalid.'); }
|
||||
if (publicKey.asymmetricKeyType !== 'ed25519'
|
||||
|| !verify(null, bytes, publicKey, Buffer.from(proof.signature, 'base64url'))) {
|
||||
throw new Error('Reference result signature is not trusted.');
|
||||
}
|
||||
return payload;
|
||||
};
|
||||
18
scripts/sign-drive-trace.mjs
Normal file
18
scripts/sign-drive-trace.mjs
Normal file
@@ -0,0 +1,18 @@
|
||||
#!/usr/bin/env node
|
||||
import { createPrivateKey } from 'node:crypto';
|
||||
import { readFile, writeFile } from 'node:fs/promises';
|
||||
import { signReferenceCandidate } from './lib/reference-evidence.mjs';
|
||||
|
||||
const args = process.argv.slice(2);
|
||||
if (args.length !== 6 || args[0] !== '--key-id' || args[2] !== '--private-key') {
|
||||
throw new Error('Usage: sign-drive-trace --key-id ID --private-key PRIVATE_PEM INPUT_JSON OUTPUT_JSON');
|
||||
}
|
||||
const [, keyId, , privateKeyPath, inputPath, outputPath] = args;
|
||||
const [candidate, privateKeyPem] = await Promise.all([
|
||||
readFile(inputPath, 'utf8').then(JSON.parse),
|
||||
readFile(privateKeyPath, 'utf8'),
|
||||
]);
|
||||
const privateKey = createPrivateKey(privateKeyPem);
|
||||
if (privateKey.asymmetricKeyType !== 'ed25519') throw new Error('Reference results require an external Ed25519 private key.');
|
||||
const signed = signReferenceCandidate(candidate, { keyId, privateKey });
|
||||
await writeFile(outputPath, `${JSON.stringify(signed, null, 2)}\n`, { encoding: 'utf8', flag: 'wx', mode: 0o600 });
|
||||
@@ -24,6 +24,31 @@ export interface SceneResourceSnapshot {
|
||||
rafCount: 0 | 1;
|
||||
}
|
||||
|
||||
export interface GpuTextureAllocation {
|
||||
id: string;
|
||||
width: number;
|
||||
height: number;
|
||||
bytesPerPixel: number;
|
||||
layers: number;
|
||||
mipmapped: boolean;
|
||||
}
|
||||
|
||||
export interface GpuRenderTargetAllocation {
|
||||
id: string;
|
||||
width: number;
|
||||
height: number;
|
||||
bytesPerPixel: number;
|
||||
attachments: number;
|
||||
samples: number;
|
||||
}
|
||||
|
||||
export interface SceneGpuBudgetSnapshot {
|
||||
tier: 'low' | 'high';
|
||||
textures: readonly GpuTextureAllocation[];
|
||||
geometryBytes: number;
|
||||
renderTargets: readonly GpuRenderTargetAllocation[];
|
||||
}
|
||||
|
||||
const resourceKinds: readonly SceneResourceKind[] = [
|
||||
'geometry', 'material', 'texture', 'target', 'pass', 'renderer', 'other',
|
||||
];
|
||||
@@ -127,4 +152,67 @@ export class SceneResourceBag {
|
||||
rafCount: this.raf ? 1 : 0,
|
||||
};
|
||||
}
|
||||
|
||||
gpuBudgetSnapshot(input: {
|
||||
tier: 'low' | 'high';
|
||||
defaultFramebuffer: Omit<GpuRenderTargetAllocation, 'id'>;
|
||||
}): SceneGpuBudgetSnapshot {
|
||||
let geometryBytes = 0;
|
||||
const seenBuffers = new Set<ArrayBufferLike>();
|
||||
const textures: GpuTextureAllocation[] = [];
|
||||
const renderTargets: GpuRenderTargetAllocation[] = [{ id: 'default-framebuffer', ...input.defaultFramebuffer }];
|
||||
let textureIndex = 0;
|
||||
let targetIndex = 0;
|
||||
for (const [resource, kind] of this.resources) {
|
||||
if (kind === 'geometry') {
|
||||
const geometry = resource as DisposableResource & {
|
||||
attributes?: Record<string, { array?: ArrayBufferView; data?: { array?: ArrayBufferView } }>;
|
||||
index?: { array?: ArrayBufferView } | null;
|
||||
};
|
||||
const arrays = [
|
||||
...Object.values(geometry.attributes ?? {}).map((attribute) => attribute.array ?? attribute.data?.array),
|
||||
geometry.index?.array,
|
||||
].filter((array): array is ArrayBufferView => array !== undefined);
|
||||
for (const array of arrays) {
|
||||
if (seenBuffers.has(array.buffer)) continue;
|
||||
seenBuffers.add(array.buffer);
|
||||
geometryBytes += array.byteLength;
|
||||
}
|
||||
}
|
||||
if (kind === 'texture') {
|
||||
const texture = resource as DisposableResource & {
|
||||
image?: { width?: number; height?: number; data?: ArrayBufferView };
|
||||
generateMipmaps?: boolean;
|
||||
};
|
||||
const width = Number(texture.image?.width);
|
||||
const height = Number(texture.image?.height);
|
||||
if (!Number.isInteger(width) || width <= 0 || !Number.isInteger(height) || height <= 0) {
|
||||
throw new Error('Owned production texture is missing measurable dimensions.');
|
||||
}
|
||||
const dataBytes = texture.image?.data?.byteLength;
|
||||
const bytesPerPixel = dataBytes === undefined ? 4 : dataBytes / (width * height);
|
||||
if (!Number.isInteger(bytesPerPixel) || bytesPerPixel <= 0) throw new Error('Owned production texture has an unsupported pixel layout.');
|
||||
textures.push({ id: `owned-texture-${textureIndex}`, width, height, bytesPerPixel, layers: 1, mipmapped: texture.generateMipmaps !== false });
|
||||
textureIndex += 1;
|
||||
}
|
||||
if (kind === 'target') {
|
||||
const target = resource as DisposableResource & { width?: number; height?: number; samples?: number; depthBuffer?: boolean };
|
||||
const width = Number(target.width);
|
||||
const height = Number(target.height);
|
||||
if (!Number.isInteger(width) || width <= 0 || !Number.isInteger(height) || height <= 0) {
|
||||
throw new Error('Owned production render target is missing measurable dimensions.');
|
||||
}
|
||||
renderTargets.push({
|
||||
id: `owned-render-target-${targetIndex}`,
|
||||
width,
|
||||
height,
|
||||
bytesPerPixel: 4,
|
||||
attachments: target.depthBuffer === false ? 1 : 2,
|
||||
samples: Math.max(1, Number(target.samples) || 1),
|
||||
});
|
||||
targetIndex += 1;
|
||||
}
|
||||
}
|
||||
return { tier: input.tier, textures, geometryBytes, renderTargets };
|
||||
}
|
||||
}
|
||||
|
||||
@@ -136,6 +136,18 @@ export async function createProceduralSceneGeneration(
|
||||
Math.max(1, Math.floor(width * quality.pixelRatio)),
|
||||
Math.max(1, Math.floor(height * quality.pixelRatio)),
|
||||
);
|
||||
if (TEST_BUILD && context.quality !== 'medium') {
|
||||
window.__PORTFOLIO_PERFORMANCE__?.recordRuntimeAllocation(context.resources.gpuBudgetSnapshot({
|
||||
tier: context.quality,
|
||||
defaultFramebuffer: {
|
||||
width: Math.max(1, canvas.width),
|
||||
height: Math.max(1, canvas.height),
|
||||
bytesPerPixel: 4,
|
||||
attachments: 2,
|
||||
samples: context.quality === 'low' ? 1 : 4,
|
||||
},
|
||||
}));
|
||||
}
|
||||
};
|
||||
|
||||
if (context.signal.aborted) {
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
import type { SceneGpuBudgetSnapshot } from '@/scene/resources';
|
||||
|
||||
declare const __PORTFOLIO_TEST_BUILD__: boolean;
|
||||
declare const __PORTFOLIO_TEST_TOKEN_DIGEST__: string;
|
||||
|
||||
@@ -37,6 +39,7 @@ export interface ScriptedActionSample {
|
||||
|
||||
export interface SceneInterval { start: number; end: number }
|
||||
export interface SceneFrameSample { start: number; end: number }
|
||||
export type RuntimeAllocationSample = SceneGpuBudgetSnapshot;
|
||||
|
||||
export const nearestRankP95 = (samples: readonly number[]): number => {
|
||||
if (samples.length === 0) throw new Error('At least one sample is required.');
|
||||
@@ -115,7 +118,7 @@ export const evaluateEventToNextPaint = (
|
||||
if (entry.interactionId === 0) continue;
|
||||
groups.set(entry.interactionId, Math.max(groups.get(entry.interactionId) ?? 0, entry.duration));
|
||||
}
|
||||
if (groups.size !== 1) throw new Error('Event Timing groups are ambiguous.');
|
||||
if (groups.size !== 1) throw new Error(`Event Timing groups are ambiguous for ${action.id}: ${JSON.stringify(matching)}`);
|
||||
return [...groups.values()][0]!;
|
||||
});
|
||||
return { values, p95: nearestRankP95(values) };
|
||||
@@ -192,6 +195,7 @@ interface PerformanceSnapshot {
|
||||
cutoff: number;
|
||||
navigationBarrier: { armed: boolean; pendingEpoch: number | null; cancelledEpoch: number | null };
|
||||
actions: readonly ScriptedActionSample[];
|
||||
runtimeAllocation: RuntimeAllocationSample | null;
|
||||
}
|
||||
|
||||
interface PerformanceHooks {
|
||||
@@ -200,6 +204,7 @@ interface PerformanceHooks {
|
||||
endTrace(): void;
|
||||
recordSceneFrame(start: number, end: number): void;
|
||||
recordSceneState(state: string): void;
|
||||
recordRuntimeAllocation(allocation: RuntimeAllocationSample): void;
|
||||
armNavigationSettlementBarrier(): void;
|
||||
registerNavigationSettlement(epoch: number): void;
|
||||
holdNavigationSettlement(epoch: number): boolean;
|
||||
@@ -234,6 +239,7 @@ export const installAuthorizedPerformanceInstrumentation = async (): Promise<voi
|
||||
const sceneFrames: SceneFrameSample[] = [];
|
||||
const sceneIntervals: SceneInterval[] = [];
|
||||
let firstSceneFrame: number | null = null;
|
||||
let runtimeAllocation: RuntimeAllocationSample | null = null;
|
||||
let traceStart: number | null = null;
|
||||
let traceEnd: number | null = null;
|
||||
let runningIntervalStart: number | null = null;
|
||||
@@ -325,6 +331,13 @@ export const installAuthorizedPerformanceInstrumentation = async (): Promise<voi
|
||||
runningIntervalStart = null;
|
||||
}
|
||||
},
|
||||
recordRuntimeAllocation(allocation) {
|
||||
runtimeAllocation = Object.freeze({
|
||||
...allocation,
|
||||
textures: Object.freeze(allocation.textures.map((item) => Object.freeze({ ...item }))),
|
||||
renderTargets: Object.freeze(allocation.renderTargets.map((item) => Object.freeze({ ...item }))),
|
||||
});
|
||||
},
|
||||
armNavigationSettlementBarrier() {
|
||||
barrierArmed = true;
|
||||
barrierPendingEpoch = null;
|
||||
@@ -383,6 +396,7 @@ export const installAuthorizedPerformanceInstrumentation = async (): Promise<voi
|
||||
...record,
|
||||
eventEntries: Object.freeze([...record.eventEntries]),
|
||||
}))),
|
||||
runtimeAllocation,
|
||||
}),
|
||||
};
|
||||
Object.defineProperty(window, '__PORTFOLIO_PERFORMANCE__', {
|
||||
|
||||
@@ -1,36 +1,67 @@
|
||||
import { generateKeyPairSync } from 'node:crypto';
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import { aggregateFiveDriveTraces } from '../../scripts/lib/performance-aggregation.mjs';
|
||||
import { signReferenceCandidate } from '../../scripts/lib/reference-evidence.mjs';
|
||||
|
||||
const run = (lcp: number, tbt: number, runIndex: number, gatingReferenceResult = true) => ({
|
||||
const trusted = generateKeyPairSync('ed25519');
|
||||
const attacker = generateKeyPairSync('ed25519');
|
||||
const policy = {
|
||||
schema: 'portfolio-reference-trust-policy/v1',
|
||||
keyId: 'lab-release-2026',
|
||||
publicKeyPem: trusted.publicKey.export({ type: 'spki', format: 'pem' }).toString(),
|
||||
};
|
||||
|
||||
const candidate = (lcp: number, tbt: number, runIndex: number) => ({
|
||||
schema: 'drive-trace.v1',
|
||||
evidenceKind: 'validated-reference-candidate-unsigned',
|
||||
gatingReferenceResult: false,
|
||||
referenceCandidateValidated: true,
|
||||
profile: 'desktop-high',
|
||||
tier: 'high',
|
||||
referenceRun: { runId: `run-${runIndex}`, runIndex, attestationDigest: `${runIndex}`.repeat(64) },
|
||||
gatingReferenceResult,
|
||||
validReferenceCadence: true,
|
||||
referenceBudgetsPass: true,
|
||||
metrics: { lcp, tbt },
|
||||
budgetResults: { cls: true, scene: true },
|
||||
budgetResults: { cls: true, scene: true, runtimeResources: true, requestConcurrency: true },
|
||||
});
|
||||
|
||||
const signed = (lcp: number, tbt: number, runIndex: number) => signReferenceCandidate(
|
||||
candidate(lcp, tbt, runIndex),
|
||||
{ keyId: policy.keyId, privateKey: trusted.privateKey },
|
||||
);
|
||||
|
||||
describe('five-run drive trace aggregation', () => {
|
||||
it('uses worst and median gates across exactly five reference runs', () => {
|
||||
it('uses worst and median gates across exactly five trusted signed runs', () => {
|
||||
expect(aggregateFiveDriveTraces([
|
||||
run(2_000, 100, 1), run(2_100, 110, 2), run(2_500, 200, 3), run(2_900, 250, 4), run(3_000, 300, 5),
|
||||
])).toMatchObject({
|
||||
signed(2_000, 100, 1), signed(2_100, 110, 2), signed(2_500, 200, 3), signed(2_900, 250, 4), signed(3_000, 300, 5),
|
||||
], policy)).toMatchObject({
|
||||
runCount: 5,
|
||||
passes: true,
|
||||
summary: { lcpWorst: 3_000, lcpMedian: 2_500, tbtWorst: 300, tbtMedian: 200 },
|
||||
});
|
||||
});
|
||||
|
||||
it('rejects non-reference developer evidence and wrong run counts', () => {
|
||||
expect(aggregateFiveDriveTraces([
|
||||
run(2_000, 100, 1, false), run(2_000, 100, 2), run(2_000, 100, 3), run(2_000, 100, 4), run(2_000, 100, 5),
|
||||
]).passes).toBe(false);
|
||||
expect(() => aggregateFiveDriveTraces([run(2_000, 100, 1)])).toThrow(/five/u);
|
||||
it('rejects unsigned developer reports and wrong run counts', () => {
|
||||
expect(() => aggregateFiveDriveTraces([
|
||||
candidate(2_000, 100, 1), signed(2_000, 100, 2), signed(2_000, 100, 3), signed(2_000, 100, 4), signed(2_000, 100, 5),
|
||||
], policy)).toThrow(/signature/u);
|
||||
expect(() => aggregateFiveDriveTraces([signed(2_000, 100, 1)], policy)).toThrow(/five/u);
|
||||
});
|
||||
|
||||
it('rejects duplicate or incomplete authenticated reference identities', () => {
|
||||
const duplicate = [1, 2, 3, 4, 4].map((index) => run(2_000, 100, index));
|
||||
expect(() => aggregateFiveDriveTraces(duplicate)).toThrow(/run indexes 1 through 5/u);
|
||||
it('rejects self-selected keys, tampered measurements, and duplicate replay', () => {
|
||||
const selfSigned = signReferenceCandidate(candidate(2_000, 100, 1), { keyId: policy.keyId, privateKey: attacker.privateKey });
|
||||
expect(() => aggregateFiveDriveTraces([
|
||||
selfSigned, signed(2_000, 100, 2), signed(2_000, 100, 3), signed(2_000, 100, 4), signed(2_000, 100, 5),
|
||||
], policy)).toThrow(/signature/u);
|
||||
|
||||
const tampered = { ...signed(2_000, 100, 1), metrics: { lcp: 1, tbt: 1 } };
|
||||
expect(() => aggregateFiveDriveTraces([
|
||||
tampered, signed(2_000, 100, 2), signed(2_000, 100, 3), signed(2_000, 100, 4), signed(2_000, 100, 5),
|
||||
], policy)).toThrow(/digest|signature/u);
|
||||
|
||||
const replay = signed(2_000, 100, 4);
|
||||
expect(() => aggregateFiveDriveTraces([
|
||||
signed(2_000, 100, 1), signed(2_000, 100, 2), signed(2_000, 100, 3), replay, replay,
|
||||
], policy)).toThrow(/run indexes 1 through 5|duplicate/u);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -4,6 +4,9 @@ import {
|
||||
evaluateBudgetTotals,
|
||||
evaluateMediaAssetBudget,
|
||||
evaluateRuntimeBudgetEvidence,
|
||||
evaluateRuntimeTraceBudgetEvidence,
|
||||
validateRuntimeBudgetTraceReport,
|
||||
calculatePeakRequestConcurrency,
|
||||
PERFORMANCE_BUDGETS,
|
||||
} from '../../scripts/lib/bundle-budgets.mjs';
|
||||
|
||||
@@ -121,4 +124,83 @@ describe('release budgets', () => {
|
||||
tiers: [{ tier: 'low', textures: [], geometryBytes: 0, renderTargets: [] }],
|
||||
})).toThrow(/exactly one Low and High/u);
|
||||
});
|
||||
|
||||
it('counts peak concurrency from actual production resource timing intervals', () => {
|
||||
const peak = calculatePeakRequestConcurrency([
|
||||
{ name: '/assets/a.webp', initiatorType: 'img', startTime: 0, responseEnd: 50 },
|
||||
{ name: '/assets/b.webp', initiatorType: 'img', startTime: 10, responseEnd: 60 },
|
||||
{ name: '/assets/environment-a.js', initiatorType: 'script', startTime: 20, responseEnd: 70 },
|
||||
{ name: '/assets/a.webm', initiatorType: 'video', startTime: 0, responseEnd: 50 },
|
||||
{ name: '/assets/b.mp4', initiatorType: 'video', startTime: 10, responseEnd: 60 },
|
||||
]);
|
||||
expect(peak).toEqual({ imageScene: 3, video: 2 });
|
||||
expect(evaluateRuntimeTraceBudgetEvidence({
|
||||
schema: 'portfolio-runtime-trace-evidence/v1',
|
||||
peakRequests: peak,
|
||||
allocation: { tier: 'low', textures: [], geometryBytes: 0, renderTargets: [] },
|
||||
}).violations.map((item) => item.budget)).toEqual(expect.arrayContaining(['imageSceneConcurrency', 'videoConcurrency']));
|
||||
});
|
||||
|
||||
it('recomputes actual trace allocations and ignores a forged low total', () => {
|
||||
const result = evaluateRuntimeTraceBudgetEvidence({
|
||||
schema: 'portfolio-runtime-trace-evidence/v1',
|
||||
peakRequests: { imageScene: 0, video: 0 },
|
||||
allocation: {
|
||||
tier: 'low',
|
||||
textures: [{ id: 'actual-production-texture', width: 4_096, height: 4_096, bytesPerPixel: 4, layers: 1, mipmapped: true }],
|
||||
geometryBytes: 0,
|
||||
renderTargets: [{ id: 'default-framebuffer', width: 390, height: 844, bytesPerPixel: 4, attachments: 2, samples: 1 }],
|
||||
totalBytes: 1,
|
||||
},
|
||||
});
|
||||
|
||||
expect(result.tier.totalBytes).toBeGreaterThan(PERFORMANCE_BUDGETS.lowTierGpuEstimate!);
|
||||
expect(result.violations.map((item) => item.budget)).toEqual(expect.arrayContaining(['mobileTextureDimension', 'lowTierGpuEstimate']));
|
||||
});
|
||||
|
||||
it('accepts only a production-path High capture whose framebuffer matches the observed DPR cap', () => {
|
||||
const report = {
|
||||
schema: 'runtime-budget-trace.v1',
|
||||
evidenceKind: 'authenticated-test-build-production-path',
|
||||
gatingReferenceResult: false,
|
||||
profile: 'desktop-high',
|
||||
tier: 'high',
|
||||
observed: {
|
||||
viewport: { width: 1_440, height: 900 },
|
||||
devicePixelRatio: 2,
|
||||
rendererPixelRatio: 1.5,
|
||||
},
|
||||
runtimeBudgetEvidence: {
|
||||
schema: 'portfolio-runtime-trace-evidence/v1',
|
||||
peakRequests: { imageScene: 1, video: 0 },
|
||||
allocation: {
|
||||
tier: 'high',
|
||||
textures: [],
|
||||
geometryBytes: 0,
|
||||
renderTargets: [
|
||||
{ id: 'default-framebuffer', width: 2_160, height: 1_350, bytesPerPixel: 4, attachments: 2, samples: 4 },
|
||||
],
|
||||
},
|
||||
},
|
||||
};
|
||||
|
||||
expect(validateRuntimeBudgetTraceReport(report).violations).toEqual([]);
|
||||
expect(() => validateRuntimeBudgetTraceReport({
|
||||
...report,
|
||||
runtimeBudgetEvidence: {
|
||||
...report.runtimeBudgetEvidence,
|
||||
allocation: {
|
||||
...report.runtimeBudgetEvidence.allocation,
|
||||
renderTargets: [
|
||||
{ id: 'default-framebuffer', width: 1_440, height: 900, bytesPerPixel: 4, attachments: 2, samples: 4 },
|
||||
],
|
||||
},
|
||||
},
|
||||
})).toThrow(/framebuffer dimensions/u);
|
||||
expect(() => validateRuntimeBudgetTraceReport({
|
||||
schema: 'portfolio-runtime-budget-evidence/v1',
|
||||
peakRequests: { imageScene: 0, video: 0 },
|
||||
tiers: [],
|
||||
})).toThrow(/authenticated production-path/u);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -4,6 +4,7 @@ import type { CDPSession, Page } from '@playwright/test';
|
||||
import { expect, test } from '../helpers/serve-dist';
|
||||
import { authorizedPerformanceUrl } from '../helpers/test-build-auth';
|
||||
import { loadAuthenticatedReferenceRun, validateObservedReferenceEnvironment } from './reference-run';
|
||||
import { calculatePeakRequestConcurrency, evaluateRuntimeTraceBudgetEvidence } from '../../scripts/lib/bundle-budgets.mjs';
|
||||
import {
|
||||
calculateCls,
|
||||
calculateLabTbt,
|
||||
@@ -111,6 +112,7 @@ test('drive-trace.v1 runs on current Chromium with fail-closed reference authent
|
||||
reducedMotion: matchMedia('(prefers-reduced-motion: reduce)').matches ? 'reduce' as const : 'no-preference' as const,
|
||||
saveData: Boolean((navigator as Navigator & { connection?: { saveData?: boolean } }).connection?.saveData),
|
||||
pageZoom: visualViewport?.scale ?? 1,
|
||||
devicePixelRatio,
|
||||
}));
|
||||
validateObservedReferenceEnvironment(referenceRun, {
|
||||
...observed,
|
||||
@@ -182,6 +184,7 @@ test('drive-trace.v1 runs on current Chromium with fail-closed reference authent
|
||||
if (snapshot === undefined || snapshot.traceStart === null || snapshot.traceEnd === null || snapshot.firstSceneFrame === null) {
|
||||
throw new Error('The authenticated performance snapshot is incomplete.');
|
||||
}
|
||||
if (snapshot.runtimeAllocation === null) throw new Error('The production scene did not emit an actual allocation ledger.');
|
||||
const expectedPeriod = warmupMedian;
|
||||
const fcp = await page.evaluate(() => performance.getEntriesByName('first-contentful-paint')[0]?.startTime ?? null);
|
||||
const lcp = latestLcpBeforeCutoff(snapshot.lcp, snapshot.cutoff);
|
||||
@@ -193,6 +196,15 @@ test('drive-trace.v1 runs on current Chromium with fail-closed reference authent
|
||||
const lost = calculateLostPresentationShare(snapshot.pageRafs, expectedPeriod, Math.floor(20_000 / expectedPeriod) - 3);
|
||||
const lateShifts = unexpectedLateShifts(snapshot.layoutShifts, snapshot.firstSceneFrame, snapshot.traceEnd);
|
||||
const sceneCpuP95 = nearestRankP95(snapshot.sceneFrames.map((frame) => frame.end - frame.start));
|
||||
const resourceTimings = await page.evaluate(() => performance.getEntriesByType('resource').map((entry) => {
|
||||
const resource = entry as PerformanceResourceTiming;
|
||||
return { name: resource.name, initiatorType: resource.initiatorType, startTime: resource.startTime, responseEnd: resource.responseEnd };
|
||||
}));
|
||||
const peakRequests = calculatePeakRequestConcurrency(resourceTimings);
|
||||
const runtimeBudgetEvidence = { schema: 'portfolio-runtime-trace-evidence/v1', peakRequests, allocation: snapshot.runtimeAllocation };
|
||||
const runtimeBudget = evaluateRuntimeTraceBudgetEvidence(runtimeBudgetEvidence);
|
||||
const runtimeResourcePass = !runtimeBudget.violations.some((item) => ['mobileTextureDimension', 'lowTierGpuEstimate', 'highTierGpuEstimate'].includes(item.budget));
|
||||
const requestConcurrencyPass = !runtimeBudget.violations.some((item) => ['imageSceneConcurrency', 'videoConcurrency'].includes(item.budget));
|
||||
|
||||
const budgetResults = {
|
||||
lcp: lcp <= 3_000,
|
||||
@@ -204,6 +216,8 @@ test('drive-trace.v1 runs on current Chromium with fail-closed reference authent
|
||||
continuity: continuity.passes,
|
||||
lostPresentation: lost.hasSufficientCoverage && lost.share <= profile.lostShare,
|
||||
lateShifts: lateShifts.length === 0,
|
||||
runtimeResources: runtimeResourcePass,
|
||||
requestConcurrency: requestConcurrencyPass,
|
||||
};
|
||||
if (process.env.PORTFOLIO_REFERENCE_LAB === '1') {
|
||||
expect(Object.values(budgetResults).every(Boolean)).toBe(true);
|
||||
@@ -212,13 +226,16 @@ test('drive-trace.v1 runs on current Chromium with fail-closed reference authent
|
||||
|
||||
const report = {
|
||||
schema: 'drive-trace.v1',
|
||||
evidenceKind: referenceRun === null ? 'current-chromium-developer-machine-non-reference' : 'authenticated-physical-reference',
|
||||
gatingReferenceResult: referenceRun !== null,
|
||||
evidenceKind: referenceRun === null ? 'current-chromium-developer-machine-non-reference' : 'validated-reference-candidate-unsigned',
|
||||
gatingReferenceResult: false,
|
||||
referenceCandidateValidated: referenceRun !== null,
|
||||
...(referenceRun === null ? {} : { referenceRun: { runId: referenceRun.runId, runIndex: referenceRun.runIndex, attestationDigest: referenceRun.attestationDigest } }),
|
||||
profile: profile.name,
|
||||
tier: profile.tier,
|
||||
browser: testInfo.project.name,
|
||||
metrics: { warmupMedian, lcp, cls, tbt, postRunningLongTaskCount: postRunning.length, eventToNextPaintP95: eventToPaint.p95, sceneCpuP95, continuity, lostPresentationShare: lost.share, lateShiftCount: lateShifts.length },
|
||||
runtimeBudgetEvidence,
|
||||
runtimeBudgetSummary: runtimeBudget.tier,
|
||||
budgetResults,
|
||||
referenceBudgetsPass: validReferenceCadence && Object.values(budgetResults).every(Boolean),
|
||||
validReferenceCadence,
|
||||
|
||||
@@ -75,6 +75,7 @@ describe('authenticated reference runs', () => {
|
||||
reducedMotion: 'no-preference',
|
||||
saveData: false,
|
||||
pageZoom: 1,
|
||||
devicePixelRatio: 2,
|
||||
browserName: 'Chromium',
|
||||
browserVersion: '140.0.0',
|
||||
startedFromAboutBlank: true,
|
||||
@@ -85,9 +86,21 @@ describe('authenticated reference runs', () => {
|
||||
reducedMotion: 'no-preference',
|
||||
saveData: false,
|
||||
pageZoom: 1,
|
||||
devicePixelRatio: 2,
|
||||
browserName: 'Chromium',
|
||||
browserVersion: '140.0.0',
|
||||
startedFromAboutBlank: true,
|
||||
})).toThrow(/viewport/u);
|
||||
expect(() => validateObservedReferenceEnvironment(authenticated, {
|
||||
viewport: { width: 1_440, height: 900 },
|
||||
visible: true,
|
||||
reducedMotion: 'no-preference',
|
||||
saveData: false,
|
||||
pageZoom: 1,
|
||||
devicePixelRatio: 1,
|
||||
browserName: 'Chromium',
|
||||
browserVersion: '140.0.0',
|
||||
startedFromAboutBlank: true,
|
||||
})).toThrow(/pixel ratio/u);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -25,6 +25,7 @@ interface ObservedEnvironment {
|
||||
reducedMotion: 'no-preference' | 'reduce';
|
||||
saveData: boolean;
|
||||
pageZoom: number;
|
||||
devicePixelRatio: number;
|
||||
browserName: string;
|
||||
browserVersion: string;
|
||||
startedFromAboutBlank: boolean;
|
||||
@@ -98,5 +99,7 @@ export const loadAuthenticatedReferenceRun = async (profile: ReferenceProfile):
|
||||
export const validateObservedReferenceEnvironment = (reference: AuthenticatedReferenceRun, observed: ObservedEnvironment): void => {
|
||||
if (observed.viewport.width !== reference.viewport.width || observed.viewport.height !== reference.viewport.height) throw new Error('Observed viewport does not match the attestation.');
|
||||
if (!observed.visible || observed.reducedMotion !== 'no-preference' || observed.saveData || observed.pageZoom !== 1 || !observed.startedFromAboutBlank) throw new Error('Observed reference environment assertions failed.');
|
||||
const minimumDpr = reference.profile === 'desktop-high' ? 1.5 : 1.25;
|
||||
if (!Number.isFinite(observed.devicePixelRatio) || observed.devicePixelRatio < minimumDpr) throw new Error('Observed device pixel ratio cannot exercise the maximum tier cap.');
|
||||
if (observed.browserName !== 'Chromium' || observed.browserVersion !== reference.browserVersion) throw new Error('Observed browser does not match the attestation.');
|
||||
};
|
||||
|
||||
56
tests/performance/runtime-budget-trace.spec.ts
Normal file
56
tests/performance/runtime-budget-trace.spec.ts
Normal file
@@ -0,0 +1,56 @@
|
||||
import { writeFile } from 'node:fs/promises';
|
||||
import { expect, test } from '../helpers/serve-dist';
|
||||
import { authorizedPerformanceUrl } from '../helpers/test-build-auth';
|
||||
import {
|
||||
calculatePeakRequestConcurrency,
|
||||
validateRuntimeBudgetTraceReport,
|
||||
} from '../../scripts/lib/bundle-budgets.mjs';
|
||||
|
||||
test('runtime-budget-trace.v1 captures the actual High production allocation at its DPR cap', async ({ page, distUrl }, testInfo) => {
|
||||
test.setTimeout(30_000);
|
||||
const viewport = { width: 1_440, height: 900 };
|
||||
await page.setViewportSize(viewport);
|
||||
await page.emulateMedia({ reducedMotion: 'no-preference' });
|
||||
await page.goto(authorizedPerformanceUrl(distUrl, 'high'), { waitUntil: 'load', timeout: 10_000 });
|
||||
await expect(page.locator('.journey-world')).toHaveAttribute('data-scene-state', 'running', { timeout: 5_000 });
|
||||
await expect.poll(() => page.evaluate(() => window.__PORTFOLIO_PERFORMANCE__?.snapshot().runtimeAllocation ?? null))
|
||||
.not.toBeNull();
|
||||
|
||||
const snapshot = await page.evaluate(() => window.__PORTFOLIO_PERFORMANCE__?.snapshot());
|
||||
if (snapshot?.runtimeAllocation === null || snapshot?.runtimeAllocation === undefined) {
|
||||
throw new Error('The authenticated production scene did not emit an allocation ledger.');
|
||||
}
|
||||
const resourceTimings = await page.evaluate(() => performance.getEntriesByType('resource').map((entry) => {
|
||||
const resource = entry as PerformanceResourceTiming;
|
||||
return {
|
||||
name: resource.name,
|
||||
initiatorType: resource.initiatorType,
|
||||
startTime: resource.startTime,
|
||||
responseEnd: resource.responseEnd,
|
||||
};
|
||||
}));
|
||||
const report = {
|
||||
schema: 'runtime-budget-trace.v1',
|
||||
evidenceKind: 'authenticated-test-build-production-path',
|
||||
gatingReferenceResult: false,
|
||||
profile: 'desktop-high',
|
||||
tier: 'high',
|
||||
observed: {
|
||||
viewport,
|
||||
devicePixelRatio: await page.evaluate(() => devicePixelRatio),
|
||||
rendererPixelRatio: 1.5,
|
||||
},
|
||||
runtimeBudgetEvidence: {
|
||||
schema: 'portfolio-runtime-trace-evidence/v1',
|
||||
peakRequests: calculatePeakRequestConcurrency(resourceTimings),
|
||||
allocation: snapshot.runtimeAllocation,
|
||||
},
|
||||
};
|
||||
const evaluated = validateRuntimeBudgetTraceReport(report);
|
||||
expect(evaluated.violations).toEqual([]);
|
||||
await writeFile(testInfo.outputPath('runtime-budget-trace.v1.json'), `${JSON.stringify({
|
||||
...report,
|
||||
runtimeBudgetSummary: evaluated.tier,
|
||||
violations: evaluated.violations,
|
||||
}, null, 2)}\n`);
|
||||
});
|
||||
@@ -5,6 +5,7 @@ import { renderQualityFor } from '@/scene/render-quality';
|
||||
import { createChaseCamera } from '@/scene/world/environment';
|
||||
import { createCar } from '@/scene/world/car';
|
||||
import { routeLabelFor } from '@/scene/world/sign';
|
||||
import { evaluateRuntimeTraceBudgetEvidence } from '../../scripts/lib/bundle-budgets.mjs';
|
||||
|
||||
describe('SceneResourceBag', () => {
|
||||
it.each([
|
||||
@@ -90,6 +91,62 @@ describe('SceneResourceBag', () => {
|
||||
bag.dispose();
|
||||
expect(cancelFrame).toHaveBeenCalledOnce();
|
||||
});
|
||||
|
||||
it('derives allocation bytes from the actual owned geometry, texture, and render target', () => {
|
||||
const bag = new SceneResourceBag();
|
||||
const geometry = new THREE.BufferGeometry();
|
||||
geometry.setAttribute('position', new THREE.BufferAttribute(new Float32Array(12), 3));
|
||||
geometry.setIndex(new THREE.BufferAttribute(new Uint16Array(6), 1));
|
||||
bag.own('geometry', geometry);
|
||||
bag.own('texture', new THREE.DataTexture(new Uint8Array(4 * 8 * 4), 4, 8));
|
||||
bag.own('target', new THREE.WebGLRenderTarget(16, 8, { depthBuffer: true }));
|
||||
|
||||
const snapshot = bag.gpuBudgetSnapshot({
|
||||
tier: 'low',
|
||||
defaultFramebuffer: { width: 32, height: 16, bytesPerPixel: 4, attachments: 2, samples: 1 },
|
||||
});
|
||||
|
||||
expect(snapshot.geometryBytes).toBe(12 * 4 + 6 * 2);
|
||||
expect(snapshot.textures[0]).toMatchObject({ width: 4, height: 8, bytesPerPixel: 4, layers: 1 });
|
||||
expect(snapshot.renderTargets).toEqual(expect.arrayContaining([
|
||||
expect.objectContaining({ width: 16, height: 8, attachments: 2, samples: 1 }),
|
||||
expect.objectContaining({ id: 'default-framebuffer', width: 32, height: 16 }),
|
||||
]));
|
||||
});
|
||||
|
||||
it('changes the production ledger when an actual geometry allocation is added', () => {
|
||||
const bag = new SceneResourceBag();
|
||||
const before = bag.gpuBudgetSnapshot({ tier: 'high', defaultFramebuffer: { width: 1, height: 1, bytesPerPixel: 4, attachments: 2, samples: 1 } });
|
||||
const geometry = new THREE.BufferGeometry();
|
||||
geometry.setAttribute('position', new THREE.BufferAttribute(new Float32Array(30), 3));
|
||||
bag.own('geometry', geometry);
|
||||
const after = bag.gpuBudgetSnapshot({ tier: 'high', defaultFramebuffer: { width: 1, height: 1, bytesPerPixel: 4, attachments: 2, samples: 1 } });
|
||||
|
||||
expect(after.geometryBytes - before.geometryBytes).toBe(30 * 4);
|
||||
});
|
||||
|
||||
it('fails from actual oversized production texture and render-target allocations', () => {
|
||||
const textureBag = new SceneResourceBag();
|
||||
const canvas = document.createElement('canvas');
|
||||
canvas.width = 4_096;
|
||||
canvas.height = 4_096;
|
||||
textureBag.own('texture', new THREE.CanvasTexture(canvas));
|
||||
const textureResult = evaluateRuntimeTraceBudgetEvidence({
|
||||
schema: 'portfolio-runtime-trace-evidence/v1',
|
||||
peakRequests: { imageScene: 0, video: 0 },
|
||||
allocation: textureBag.gpuBudgetSnapshot({ tier: 'low', defaultFramebuffer: { width: 1, height: 1, bytesPerPixel: 4, attachments: 2, samples: 1 } }),
|
||||
});
|
||||
expect(textureResult.violations.map((item) => item.budget)).toEqual(expect.arrayContaining(['mobileTextureDimension', 'lowTierGpuEstimate']));
|
||||
|
||||
const targetBag = new SceneResourceBag();
|
||||
targetBag.own('target', new THREE.WebGLRenderTarget(4_096, 4_096, { depthBuffer: true }));
|
||||
const targetResult = evaluateRuntimeTraceBudgetEvidence({
|
||||
schema: 'portfolio-runtime-trace-evidence/v1',
|
||||
peakRequests: { imageScene: 0, video: 0 },
|
||||
allocation: targetBag.gpuBudgetSnapshot({ tier: 'low', defaultFramebuffer: { width: 1, height: 1, bytesPerPixel: 4, attachments: 2, samples: 1 } }),
|
||||
});
|
||||
expect(targetResult.violations.map((item) => item.budget)).toContain('lowTierGpuEstimate');
|
||||
});
|
||||
});
|
||||
|
||||
describe('scene render quality', () => {
|
||||
|
||||
Reference in New Issue
Block a user