Portfolio/docs/superpowers/specs/2026-09-04-portfolio-adaptive-soundtrack-design.md

Portfolio Adaptive Soundtrack Specification

Status: Approved in-chat design, pending written specification review
Date: 2026-09-04
Branch: codex/portfolio-audio-refinement
Worktree: /home/luca/Documents/Portfolio/.worktrees/portfolio-audio-refinement
Product: Pre-rendered single-page portfolio with an optional Three.js night-drive world and user-activated Web Audio soundtrack

1. Purpose

Replace the unstable scroll-to-soundtrack coupling with an adaptive playback system that preserves the approved authored recording. The result must sound like one continuous professionally produced score while responding gently to settled chapter changes and deliberate acceleration.

This is not a return to procedural composition. The approved 86 BPM, F minor recording remains the musical foundation. Procedural behavior is limited to intent stabilization, transport scheduling, subtle mix automation, and the existing engine/road texture concept.

The work is successful when tiny scrolling cannot trigger a musical transition, rapid navigator jumps produce at most one settled mood change, acceleration feels responsive without disturbing the composition, Start can idle indefinitely, and the completed main score continues into an unobtrusive loop instead of stopping.

2. Approved product decisions

• Use the authored-master plus adaptive-overlays architecture.
• Keep the approved 64-bar, approximately 178.604651-second main score unchanged as the dominant source.
• Use four broad musical moods rather than eight section-specific arrangements:
  • open: Start and Profile
  • shadow: Experience and Infrastructure
  • lift: Selected Builds and Additional Work
  • resolve: Toolkit and Contact
• Preserve intro-in -> first intro-idle pass -> repeating intro-idle -> intro-out -> main.
• After the main score, enter a subtle seamless afterglow loop indefinitely.
• If the visitor returns to Start after the main score begins, keep the main transport running and transition only to the open mood.
• Keep engine and road texture barely audible at cruise and raise them smoothly during acceleration.
• Give acceleration two response speeds:
  • immediate, restrained engine/air/filter movement;
  • a synchronized percussion/shimmer lift only after strong input is sustained.
• Do not pitch, time-stretch, change playback rate, seek, or restart the main recording in response to scrolling.
• Do not restore the rejected procedural music sequencer.

3. Current-state findings

3.1 Foundations to retain

• createAudioController remains the sole owner of explicit gesture activation, one-context construction, mute, volume, visibility suspension, interruption recovery, BFCache behavior, failure state, and teardown.
• The current Opus-first and MP3-fallback loading contract remains.
• The public asset manifest, exact asset-reference closure, hash verification, MIME handling, prerender materialization, and deferred delivery remain authoritative.
• The current intro runtime already contains approved sample-addressed regions and a verified eight-bar idle loop.
• The approved main recording and its listener-approved final two-second fade remain source material.
• The existing 48 kHz, 86 BPM, 4/4 transport constants and sample-aware bar math remain.
• Responsive sections continue to be measured through the existing batched heading reads and ResizeObserver path.
• Navigation epochs remain the authority for supersession and user interruption.
• The old engine/road design provides a useful starting concept: a low fundamental and harmonic, filtered road noise, an ignition envelope, and acceleration-controlled filtering and level.

3.2 Problems to replace

• The visual active section currently changes at one 35%-viewport line with no audio-specific hysteresis or dwell.
• AppShell.vue forwards an explicit navigation destination to audio before navigation arrives.
• Section identity and continuously changing acceleration are combined into one AudioJourneyState, so scroll-decay frames repeatedly cancel and retarget mix ramps.
• The intro exit uses a 120 ms timer and becomes irreversible when scheduled rather than when it becomes audible.
• Programmatic smooth scrolling and physical scrolling share the same raw acceleration signal.
• The current authored-track graph exposes only a single gain and high-shelf profile, so acceleration is neither isolated nor musically meaningful.
• The main recording is a one-shot. Direct activation in a late chapter can leave only one short phrase before silence.

3.3 Delivery constraints

• The four current web derivatives total 13,603,356 bytes and nearly consume the current 13 MiB deferred-audio allowance.
• The decoded intro and main buffers together require roughly 90 MiB. Full-length synchronized stems would add about 65 MiB each and are not acceptable for this portfolio.
• The local ACE-Step installation has no approved source stems for the master. Stem separation is not part of this design.
• Any new audio must be produced and retained under /home/luca/Projects/nightdrive-music until it passes the listening checkpoint.

4. System architecture

The system has five focused units joined through typed commands:

journey measurement + navigation epochs
                 |
                 v
        audio intent coordinator -----> committed chapter mood
                 |                      stable Start departure
                 +--------------------> smoothed drive energy
                                            |
                                            v
audio controller -> soundtrack transport -> adaptive mixer -> output
                         |                   |       |
                         |                   |       +-> drive texture
                         |                   +---------> drive-lift loop
                         +-----------------------------> intro/main/afterglow

4.1 Audio intent coordinator

Add one pure, independently testable coordinator between journey/navigation state and the audio graph. It owns:

• the current visual candidate section;
• the committed musical section and mood;
• candidate start time and signed boundary depth;
• the active navigation epoch and destination;
• whether explicit navigation is travelling or settled;
• the pending Start departure;
• the audio-specific drive-energy envelope;
• a monotonically increasing transition generation used to invalidate stale commands.

The coordinator produces separate outputs for discrete chapter intent and continuous drive energy. A drive-energy update must never imply a chapter change, and a chapter change must never synthesize acceleration.

4.2 Navigation settlement signal

Extend the existing navigation snapshot with an explicit settlement phase for the current epoch. An explicit navigation begins as travelling and becomes settled only through the existing scrollend, stable-frame fallback, or final timeout alignment path. Starting another navigation or receiving physical scroll intent invalidates the previous phase through the existing epoch change.

The visual destination and sign system may continue reacting immediately. Audio commits an explicit destination only when the same epoch is still current, its destination is the measured active section, and its phase is settled.

4.3 Section-boundary measurement

Reuse the existing batched heading geometry. Expose an audio-neutral signed boundary-depth measurement in viewport-height units rather than performing a second DOM read in the audio composable.

For forward travel, a candidate is eligible only after its heading is at least 0.06 viewport heights beyond the activation line, equivalent to 6vh. For reverse travel, the committed section must move at least the same distance back across the line.

A passive candidate must remain eligible and unchanged for 400 ms before commitment. The coordinator requests shared frames while that dwell is pending. Reversal, a different candidate, a navigation epoch change, document visibility loss, or a structural remeasurement clears the pending dwell. A resize or content-height change alone can never commit a new mood.

Skipping several sections produces one candidate: the final measured section after movement settles. Intermediate candidates never queue audio transitions.

4.4 Soundtrack transport

The transport owns one musical clock and these phases:

idle
  -> intro-in -> intro-idle-first-pass -> intro-idle-loop
  -> intro-out -> main -> afterglow-in -> afterglow-idle-loop

idle away from Start
  -> main at mapped eight-bar phrase -> afterglow-in -> afterglow-idle-loop

• The current intro sample boundaries remain authoritative.
• A stable Start departure schedules intro-out at the next safe bar boundary.
• Scheduled intro-out and main source handles remain cancellable until the transition boundary is reached. mainCommitted changes only at that boundary, never when nodes are prepared.
• Returning to Start before the boundary cancels the scheduled sources and automation, preserves the idle loop, and advances the transition generation.
• Main playback records one anchor time and offset. All later beat/bar calculations derive from that anchor.
• Direct activation away from Start begins at the existing section-mapped eight-bar phrase. Once started, main playback never seeks again.
• The one-bar afterglow-in starts at the beginning of the main score's final bar and sits beneath its approved tail. The eight-bar afterglow-idle region begins when the main source ends and then loops indefinitely; the transition bar is never repeated.
• Returning to Start after main commitment changes only the mood to open.

4.5 Adaptive mixer

The mixer receives infrequent committed mood commands and separately rate-limited drive-energy commands.

Mood changes start only after commitment and reach their targets over 1.4 seconds. Profiles may control only:

• master-bed trim within a narrow calibrated range;
• a gentle high-shelf or low-pass color no greater than approximately 1 dB at its effective band;
• drive-texture send;
• maximum drive-lift send.

Mood profiles do not manipulate playback position, rate, pitch, channel phase, or artificial stereo width. Their exact values are finalized through the in-site listening pass, remain data constants, and preserve open as the neutral reference.

The authored master receives fixed mix headroom before additive buses. No always-active compressor is placed across the approved recording. The maximum combined state is calibrated offline and in the browser so the graph remains below full scale at output volume 1.

4.6 Drive texture

Extract the useful engine/road concept into a module that receives the existing AudioContext and a destination bus. It exposes only drive-energy update and disposal operations.

• Engine: centered low fundamental and restrained harmonic.
• Ignition: one short envelope when the soundtrack is first activated.
• Road/air: deterministic, subtly decorrelated stereo texture with low-mid buildup removed.
• Cruise: always present but barely perceptible beneath the music.
• Response: AudioParam automation with a 220 ms attack, 900 ms release, and an energy dead zone of 0.08.
• Explicit navigation: a capped shaped envelope no greater than 0.35; it cannot activate the musical drive lift.
• Physical input: normalized energy above the dead zone may use the full response range.

The module creates no nodes per animation frame, uses no independent timer, and disconnects every owned node exactly once.

4.7 Drive-lift layer

Use one four-bar, approximately 11.162791-second authored loop containing restrained percussion and glassy non-dominant shimmer. It must be rhythmically useful but harmonically compatible with the full F minor score. A prominent melody, bass line, chord bed, fill, riser, or impact is prohibited.

The loop runs phase-aligned and muted from the main transport anchor. Physical drive energy above 0.62 for 450 ms arms entry. Gain begins on the next beat and reaches its mood-capped level over one beat. Energy below 0.32 for 800 ms arms exit, which fades over one beat. Separate entry and exit thresholds prevent chatter.

The layer is disabled during intro-idle. Its source may begin phase-aligned and muted before the main downbeat, but its gain cannot become audible until main playback begins. Programmatic navigation cannot arm it.

4.8 Typed command boundary

Replace the combined setJourneyState() graph call with two commands:

• setChapterIntent(intent): committed section, mood, Start state, entry phrase, and transition generation. This changes only on a stable semantic commitment.
• setDriveEnergy(sample): normalized energy and source classification of physical, navigation, or idle. This cannot alter transport phase or committed mood.

The audio controller stores the latest value of each command before activation and supplies both to the graph factory synchronously from the activating gesture. After activation, useAudioEngine watches them independently.

Drive-energy emission is limited to at most once every 33 ms and requires a change of at least 0.02, except that dead-zone crossings, drive-lift threshold crossings, and the final return to zero emit immediately. AudioParam attack and release automation supplies the smooth motion between emitted targets.

5. Authored audio assets

5.1 Afterglow loop

Produce one afterglow runtime containing a one-bar, approximately 2.790698-second transition followed by an eight-bar, approximately 22.325581-second low-energy loop. The transition begins beneath the main score's final bar; only the following eight-bar idle region repeats. Build it from, or condition it by, the approved score. It must:

• remain at 86 BPM, F minor, and 4/4;
• retain the master’s warm/glassy production identity;
• avoid an exposed lead or obvious eight-bar restart;
• use a non-looping transition bar that can emerge beneath the main fade and arrive at the idle loop's opening state;
• end with sufficient continuation audio for an exact-bar loop build;
• pass a three-repeat seam audition;
• remain compatible with all four subtle mood profiles and the drive-lift layer.

5.2 Drive-lift loop

Produce one four-bar loop following the constraints in section 4.7. Audition it only in context against representative master phrases from all four moods. If no candidate remains unobtrusive and compatible across the complete score, stop at the listening checkpoint and ask the user whether to omit the layer or continue production. Do not silently integrate a clashing asset or drop an approved requirement.

5.3 Production workflow

• Keep WAV masters, candidate renders, prompts, fixed seeds, source references, and measurements in /home/luca/Projects/nightdrive-music.
• Use the approved master and existing Nightdrive references for production density, timbre, groove, and spatial character, not melody or arrangement copying.
• Generate or edit only the short additions; do not regenerate the 64-bar master.
• Reject silence, clipping, invalid samples, mono collapse, unintended vocals, wrong duration, or an audible loop seam before listening review.
• Render the afterglow three times and inspect both seam evidence and the audio.
• Present one concise in-context audition containing the main-to-afterglow handoff, repeated afterglow seam, cruise texture, strong acceleration entry, and return to cruise.
• Musical approval remains a human listening decision. Technical measurements cannot establish musical quality.

5.4 Web delivery

After approval, add Opus and MP3 derivatives for afterglow and driveLift to the typed public soundtrack contract and asset manifest. Both are deferred and hash-bound like the existing intro and main sources.

Raise the deferred-audio transfer allowance from 13 MiB to a hard maximum of 16 MiB. Do not lower the approved intro/main encoding quality merely to create room. Exact closure continues to reject unreferenced files.

Decode the drive-lift asset after successful user activation without delaying intro playback. Load the afterglow after main playback is established and before its handoff window. Once the afterglow is running, release strong references to the completed main buffer and source nodes. Release the intro buffer after main commitment. Optional buffers and nodes are released on final teardown.

6. Data flow and transition rules

6.1 Before activation

• No AudioContext, fetch, decode, oscillator, or audio timer exists.
• The coordinator may track the latest stable location as pure state.
• Activation at Start selects the intro path.
• Activation elsewhere selects the mapped main phrase and committed mood for that section.

6.2 Passive physical scrolling

1. The journey system performs its existing single batched geometry measurement.
2. The visual section and scene may update immediately.
3. The audio coordinator evaluates boundary depth, candidate identity, layout revision, and time.
4. Only a candidate that satisfies six-percent hysteresis and 400 ms dwell becomes committed.
5. The committed mood is sent once to the mixer.
6. Physical acceleration is independently dead-banded and smoothed for drive texture and drive-lift gating.

6.3 Explicit navigation

1. A new navigation epoch invalidates every older pending audio destination and scheduled intent.
2. The master transport continues untouched while the page travels.
3. Raw programmatic scroll velocity is replaced with the capped navigation drive envelope.
4. Audio commits the destination only after that same epoch settles at the measured destination.
5. Rapid clicks therefore produce one final mood change. User wheel, touch, or scroll-key intent cancels explicit audio settlement and returns control to passive rules.

6.4 Layout changes

• Heading geometry remains derived from current content, not assumed section heights.
• ResizeObserver, font completion, image/video metadata, viewport resize, and orientation change increment a layout revision and clear only pending audio candidates.
• The last committed mood and the running musical transport remain unchanged.
• A later physical or explicit navigation can establish a new candidate against the new measurements.

7. Failure and degradation behavior

• Intro or main request/decode failure remains a terminal graph failure exposed as Audio unavailable through the existing control.
• Drive-lift request/decode failure disables only that layer. Authored music and drive texture continue.
• Afterglow request/decode failure allows the approved main fade to finish into silence. No uncaught error, emergency restart, or repeated retry is permitted.
• Drive-texture construction failure disables only engine/road response.
• A scheduled transition invalidated before its boundary stops unstarted sources, cancels its gain automation, and leaves the active source untouched.
• Visibility loss resets pending dwell and acceleration gates, suspends the existing graph, and produces no catch-up transition on return.
• Browser interruption, BFCache, mute, volume, and teardown retain the existing controller rules.
• Optional-layer failure is not promoted to the global Audio unavailable UI because the dominant approved soundtrack remains functional.
• Test-build audio fault hooks F10 through F13 remain available around the new browser soundtrack graph.

8. Focused verification strategy

Verification is intentionally proportional. Focused checks run during implementation; the compiled preview and full existing verification run once at completion.

8.1 Pure coordinator tests

• A tiny boundary crossing followed by reversal never commits.
• Remaining less than six percent past the boundary never commits regardless of time.
• An eligible unchanged passive candidate commits once after 400 ms.
• A structure-only remeasurement clears a candidate without changing the committed mood.
• A rapid sequence of explicit destinations commits only the latest settled epoch.
• User input cancels a travelling explicit destination.
• Returning to Start before the intro boundary cancels departure; returning after main commitment changes only to open.

8.2 Transport and graph tests

• Intro-out scheduling is bar-aligned and cancellable before its boundary.
• Main source position and anchor do not change on any mood command.
• Direct late-section activation begins at the mapped eight-bar phrase.
• Main-to-afterglow handoff occurs once and afterglow repeats indefinitely.
• Drive lift uses the shared phase, entry/exit hysteresis, and beat-length fades.
• Programmatic navigation cannot arm drive lift.
• Repeated drive updates create no additional persistent nodes.
• Optional asset failures degrade independently.
• Close/dispose stops and disconnects every owned source and node once.

8.3 Existing lifecycle checks

Run the focused gesture, repeated activation, visibility, interruption, mute, volume, BFCache, failure-injection, and teardown tests that protect createAudioController. Update adapters only where the graph interface intentionally splits chapter and drive commands.

8.4 Audio and browser acceptance

• Run the Nightdrive analyzer and FFmpeg loudness/true-peak inspection on the two new masters and web derivatives.
• Verify the afterglow rendered across three repetitions.
• Render the maximum combined browser mix at output volume 1 and verify finite samples and headroom below full scale.
• Perform one browser abuse pass: activate, micro-scroll near a boundary, reverse, jump Start to Contact to Toolkit rapidly, interrupt a jump, return to Start, sustain acceleration, release to cruise, and resize while audio is active.
• Perform one final in-site human listening checkpoint before declaring the soundtrack finished.
• Run npm run build:preview once after focused checks pass.

9. Scope boundaries

This work does not:

• regenerate or procedurally recreate the approved 64-bar master;
• introduce full-length stems or source separation;
• assign a unique composition to every section;
• alter native scroll behavior, visual section timing, scene composition, or content layout;
• add a new audio settings interface;
• make music dependent on WebGL or motion quality;
• use whole-track pitch, tempo, playback-rate, or time-stretch effects;
• copy the reference tracks’ melody or arrangement;
• perform unrelated refactoring or broaden the release and deployment scope.

10. Acceptance criteria

The implementation is accepted when all of the following are true:

1. Small scrolling that does not establish a stable new section produces no chapter transition or intro exit.
2. Rapid passive scrolling and rapid navigator changes never queue intermediate musical transitions.
3. Start loops cleanly for an arbitrary dwell and exits only after stable departure.
4. The main score plays continuously through all chapter changes and when returning to Start.
5. The main score enters a listener-approved seamless afterglow loop and can remain there indefinitely.
6. Four moods are perceptible only as subtle coherent shading, with no restart, seek, click, obvious volume dip, or abrupt tonal step.
7. Engine and road texture are barely audible at cruise, respond smoothly to physical acceleration, and remain restrained beneath the master.
8. Strong sustained physical acceleration can introduce the approved synchronized drive-lift layer; tiny input and programmatic navigation cannot.
9. Dynamic content height, viewport resize, and orientation changes do not directly change musical intent.
10. Gesture gating, mute, volume, visibility, interruption, BFCache, failure, and teardown behavior remain intact.
11. The combined output retains technical headroom, the exact asset manifest remains closed, and deferred audio does not exceed 16 MiB.
12. One final compiled-preview listening pass is approved by the user.