diff --git a/INTERACTION.md b/INTERACTION.md index f971692..a44ea19 100644 --- a/INTERACTION.md +++ b/INTERACTION.md @@ -30,9 +30,11 @@ Rust's canonical extraction pass supplies every recognised occurrence and its UT ## Audio lifecycle and preferences +Version 0.8 expands the silent startup to a roughly 3.5-second task/dependency choreography and adds an independent 280 ms foreground-return lifecycle. Both consume a bounded, read-only presentation from the existing GraphScene. Native focus is primary, with event-driven native rechecks when WebView2 omits a window notification. See [startup/return integration, marker inventory and unchanged sound gain staging](docs/STARTUP_AUDIO.md). The process-lifetime Rust claim prevents reloads from replaying the intro; startup completion does not disable future return detection. + Feedback accepts successful typed interactions with semantic role, bounded focus level and canonical identity. It lazily creates one AudioContext after a trusted gesture. Performance timestamps map to the audio clock; events over 80 ms stale are dropped. Native focus and document focus independently gate audio. Typing, hover and startup never sound. -NoteSynth creates original additive sine voices; note.ts centralizes the consonant pitch family and identity mapping. Subnetworks use C4/D4/E4/G4 with a 300 ms warm body; topics use C5/D5/E5/G5 with a 235 ms body; concepts use C6/D6/E6 with a 170 ms lighter body. Repetition varies detune within ±1.5 cents, attack 2.5–4 ms and level by at most 4%, without changing the fundamental pitch choice. Higher registers have lower gain. Back/close use short quiet envelopes; only three especially quiet neighbour arrivals are scheduled from shared signal timestamps. +NoteSynth creates original additive sine voices; note.ts centralizes the consonant pitch family and identity mapping. Subnetworks use C4/D4/E4/G4 with a 300 ms warm body; topics use C5/D5/E5/G5 with a 235 ms body; concepts use C6/D6/E6 with a 170 ms lighter body. Repetition varies detune within ±1.5 cents, attack 2.5–4 ms and level by at most 4%, without changing the fundamental pitch choice. Higher registers have lower gain. A cached, original band-passed noise contact ends after its 3.5 ms envelope without ending the tone. Back/close use short quiet envelopes; only three subordinate neighbour arrivals are scheduled from shared signal timestamps. VoiceBudget reserves real intervals, caps overlap at six and applies a 60 ms identical-contact gate. Direct actions may evict low-priority arrivals. Every oscillator, gain, filter and panner disconnects after completion or cancellation. Native blur cancels queued audio, with no replay on refocus. A compressor feeds a soft limiter and conservative master. OfflineAudioContext renders the real production graph for clipping/polyphony/cancellation tests; actual application captures tap the live output without a microphone. @@ -73,4 +75,4 @@ Quiz marking, card publication and scheduling are local Rust operations. The sch ## Current release verification -scripts/depth-desktop.mjs inspects the real development WebView with temporary synthetic files, real Rust commands and development-only instrumentation. It tests direct Canvas navigation, exact Back framing/layout, dirty edits, interrupted note loads, Reduced mode, deleted focus, small windows, 90 additional backlinks, duplicate subject UUIDs and idle rendering. No provider requests or production test globals are used. scripts/audio-validation.mjs renders all three real synthesized registers at 69% and 100% volume. Installed-app capture and release results are documented in VALIDATION.md. +scripts/depth-desktop.mjs inspects the real development WebView with temporary synthetic files, real Rust commands and development-only instrumentation. It tests direct Canvas navigation, exact Back framing/layout, dirty edits, interrupted note loads, Reduced mode, deleted focus, small windows, 90 additional backlinks, duplicate subject UUIDs and idle rendering. No provider requests or production test globals are used. scripts/audio-validation.mjs renders all three real synthesized registers at 35%, 69%, 70%, 100% and zero volume, including mute, arrival balance and source/node cleanup. scripts/startup-validation.mjs covers both startup and return lifecycles. scripts/record-startup.mjs and scripts/record-return.mjs capture genuine installed launches and Alt-Tab cycles; scripts/record-swarm-preview.mjs produces explicitly labelled browser fixture previews. scripts/swarm-release.mjs retains the historical 0.7.0 settings/navigation/live-audio harness. Actual completed checks and native capture limitations are documented in VALIDATION.md. diff --git a/VALIDATION.md b/VALIDATION.md index af3239b..d848678 100644 --- a/VALIDATION.md +++ b/VALIDATION.md @@ -1,4 +1,187 @@ -# Synapse 0.6.0 validation +# Synapse validation + +## 0.9.0 — 18 November 2026 + +Installed Synapse 0.9.0 played the v3 startup in **4461.7 ms** on a populated desktop. The recorded Windows Alt-Tab return lasted **281.5 ms**. Application source is `fb5d3b7a49f11e83f8ff3ed62108a69d0eca9379` on the local branch `codex/startup-v3`, based on the 0.8.1 native show/focus repair. The only subsequent change is this validation record. The release and its recordings stayed local. + +### Reference and installed presentation + +The supplied v3 HTML was replayed at 1× before implementation. Its construction ended at 4.0561 s and its sequence at 4.4561 s. The updated populated layout uses 0.900 s of opening orbit, ends construction at 4.060 s, and targets 4.460 s overall. The independent startup cleanup deadline is 4.900 s. The native show claim, 250 ms preparation budget and 400 ms return safety deadline were retained. + +The cold capture began **444.7 ms before** the startup mark. Both the cold and tray captures showed fourteen named crosshairs, 16 measured UI targets, six live graph nodes with six explicitly typed labels, and three existing connections. The schedule contained six kinds of work across the real sidebar and graph. Each graph label kept its node’s agent while the caret followed the letter advances. Individual retirement lasted 160 ms. In the cold capture the first agent was gone at 1.71 s and the last at 4.22 s. The coordinated outline pulse lasted 300 ms in the nodes’ subject colours. End-of-effect inspections found zero overlays, concealed elements, preparation gates or new audio contexts. + +The approved HTML and the installed cold-start MP4 were then played together at 1×. Seven paired frames covered the orbit, peel, selection handles, dimensions, typewriter carets, dragged copies, connection guides, departures and closing pulse. The live graph and sidebar determine where each job happens. The 1× side-by-side review is `Synapse-0.9.0-v3-comparison.mp4`. + +### Installed lifecycle results + +| Case | Observed result | +| --- | --- | +| Fresh visible process, populated vault | 4461.7 ms, 68.8 ms preparation. The recording includes the first presentation. | +| Hidden tray process, first foreground | Zero effects while hidden. The first foreground startup took 4465.5 ms, with 31.2 ms preparation. | +| Fresh process, welcome screen | 1821.0 ms with two agents, three UI targets and no graph jobs. | +| Reload after the first presentation | The native process reported `already-presented` and added zero effects. | +| Recorded Alt-Tab to a blank Notepad tab and back | One 281.5 ms return. All 47 active samples kept the workspace visible with zero agents. The graph Canvas hash stayed identical. The probe observed zero audio contexts and zero oscillator or buffer-source starts. | +| Real editor with an unsaved change | A 286.2 ms return preserved the complete CodeMirror document, caret/selection, 742 px scroll, editor focus, Edit mode, drawer geometry, scope, zoom and underlying Canvas hash. The next real keystroke replaced the selected 16 characters once, after the effect. | +| Native folder picker | Escape cancelled the picker and produced zero return effects. The following genuine Alt-Tab produced one 283.9 ms return. | + +Before the startup edits, a fresh baseline recorded a 283.6 ms browser return and the prior 283.9 ms installed 0.8.1 return was retained. The old and new renderers produced identical pixels at all nine sampled times from 0 through 279 ms in a controlled comparison. Native focus traces used real window switches. The WebView diagnostics observed events and compositor frames with focus emulation disabled. + +**Four native timing checks remain incomplete.** The desktop traces did not capture a first input during startup or during the return, a deliberate absence shorter than 500 ms, or a second qualifying foreground cycle within the 1500 ms cooldown. All four cases passed in the browser lifecycle suite. + +### Checks and package + +| Check | Result | +| --- | --- | +| Frontend, TypeScript and ESLint | 78 tests passed. Typechecking and lint passed. | +| Rust | 47 tests passed. The explicitly opt-in live-provider test remained ignored. | +| Browser lifecycle suite | All 58 passed, including Full, Reduced, Follow Windows, independent switches, input, dialog suppression, cooldown, large vaults, hidden show/focus, cleanup and CSS recovery. | +| Production build | Tauri built the unsigned Windows x64 NSIS installer with the existing hosted AI origin. | +| Extracted installer payload | The installed executable and three license/notice files matched the installer byte-for-byte. | +| Original owner state | The selected-vault and device files were restored byte-for-byte. Raw preference storage matched, including motion, sound and volume. Testing used a disposable vault and the original closed application state was restored. | + +The installer is `src-tauri/target/release/bundle/nsis/Synapse_0.9.0_x64-setup.exe` (5,981,544 bytes). Its SHA-256 is `45c2ae03b31f0e7621b8d12fbdd8c44dd71f2041f3afe69f4a318b80f024a49c`. The installed executable’s SHA-256 is `f4d963ca0e5a3bcf9af5b5adf58a6bd39acb3f5ffa3137eee57c4c1b3487a64f`. + +The private evidence directory is `.local/native-acceptance-0.9.0/`. The three native recording stems are `Synapse-0.9.0-native-cold-startup`, `Synapse-0.9.0-native-tray-first-foreground` and `Synapse-0.9.0-native-alt-tab-return`, each available as MP4 and GIF. The MP4s retain source timing within 0.51 ms and include short end holds. Their playback lengths are 5.29 s, 5.30 s and 1.11 s respectively. The 25 fps GIFs and all MP4s decoded successfully. The media use explicit video range to preserve the near-black workspace in Windows playback. The browser fixture report is separate at `.local/swarm-0.9.0/browser-report.json`. + +## 0.8.1 — 17 November 2026 + +Installed Synapse **0.8.1** fixes the native show/readiness failure and has actual cold-launch, tray-launch and operating-system Alt-Tab recordings. **Native acceptance remains incomplete:** input during either effect, a deliberately sub-500 ms focus bounce and rapid qualified cycles inside the 1500 ms cooldown have browser coverage but have not passed precisely timed native tests. Ordinary native returns, native dialogs and actual CodeMirror preservation passed. This is a local verification and repair pass; nothing was published. + +Final application source: **`54156ce8fe0914710d64ddc35fac77ac25a50a6d`**, on `codex/native-swarm-acceptance`. The starting source was `a8b95ec070ada730fa8817d1232b4bfe24ac4f51` / 0.8.0. The subsequent validation-document commit does not change the installed application. Package versions were updated consistently without dependency changes. + +| Check | Result | +| --- | --- | +| TypeScript typecheck and ESLint | Passed; ESLint also rerun after the final capture-helper edits | +| Frontend tests | 78 passed | +| Rust tests | 47 passed; one explicit live-provider test ignored | +| Rustfmt and Clippy, all targets, warnings denied | Passed | +| Browser lifecycle checks | 58 passed, including native show without either focus event and show without actual focus | +| Production build | Passed; unsigned x64 NSIS package, existing public AI origin preserved | +| Installed executable and three notice files | Match extracted final installer payload byte-for-byte | +| Native cold/tray/welcome, ordinary returns, dialogs and CodeMirror | Passed on that exact installed 0.8.1 executable | +| Precisely timed native interruption/bounce/cooldown cases | Incomplete; details below | + +### Native failure and repair + +The hidden WebView could already report DOM focus. Showing the window then produced neither a DOM focus event nor the expected Tauri focus event, leaving the preparation gate pending even after actual native focus/visibility became true. The prior DOM-event reconciliation alone could not recover that ordering. Failure evidence was retained before the repair. + +The native show path now emits `synapse-window-shown`. That notification wakes the bounded preparation gate and requests an authoritative native focus reconciliation; it does not fabricate focus or consume the launch claim. Rust continues to require actual visibility and focus before granting its one-use process claim. Unit and browser regressions cover both focused and unfocused show notifications. The final installed tray and cold launches passed without controller calls or synthetic focus events. + +An additional recording-harness issue was corrected: native CDP connections now use `noDefaults: true`, preventing Playwright's default focus emulation from contaminating native focus observations. Native UI input and Alt-Tab used the approved Windows Computer Use interface with a verified blank Notepad tab. CDP supplied passive instrumentation and compositor capture; OS-motion/DPI emulation and exact preference restoration are separately identified below. + +### Installed behaviour + +| Native case | Measured result | +| --- | --- | +| New normal visible process, populated vault | Startup **3502.1 ms**; nine agents, 16 UI targets, six actual nodes, three edges; preparation 74.7 ms | +| New hidden tray process, then first foreground | No startup while hidden; first eligible presentation **3504.9 ms**, same populated sample; preparation 36.7 ms | +| New process with no selected vault | Welcome startup **1479.8 ms**, two agents and three eligible targets | +| Frontend reload in an already presented process | `already-presented`; no full startup replay | +| Recorded genuine Alt-Tab return | Exactly one effect, **283.9 ms** | +| Alt-Tab with a focused, unsaved CodeMirror note | Exactly one effect, **285.2 ms** | +| Alt-Tab after native dialog checks | Exactly one effect, **283.9 ms**, proving suppression released | +| Empty graph, collapsed sidebar, startup disabled and return enabled | Exactly one return, **286.2 ms**; viewport/DPR emulated as identified below | + +Successful populated startups completed below the 3800 ms deadline, with no remaining preparation gate, concealed content, overlays or startup RAF/timer. No initial startup/return overlap or new AudioContext was observed. Intermediate and final decoded frames showed construction matching the real UI/graph and a clean handover, without a completed-interface-then-hide flash. The cold recording was armed **151.8 ms before** the startup mark. The welcome capture began about 7 ms after its mark, so it supplies native state/timing evidence rather than a complete first-frame welcome recording. + +The real editor test inserted a distinctive unsaved edit, selected the range between positions 69 and 103 and scrolled to **1183 px**. After actual Alt-Tab, the complete unsaved document, selection, scroll, focus, Edit mode, graph scope, zoom, drawer geometry and graph Canvas hash were unchanged. Forty-eight active return-frame samples showed zero cursors, zero concealed targets and no gate. A subsequent real `Q` input replaced the selection exactly once. That character check occurred after the effect; it does not substitute for input during the 280 ms effect. Only the disposable note was later saved. + +Real native folder-picker cancellation, successful selection of the disposable vault and export/save cancellation all accepted normal input and added zero return effects. Suppression also held through observed delayed focus restoration. The later genuine Alt-Tab return passed, with 47 active samples showing no cursors or hidden content. Injected dialog errors remain covered by the browser suite. + +Reduced motion skipped both native startup and native Alt-Tab returns. The startup and return toggles independently disabled their effects; return still ran when startup alone was disabled. Follow Windows respected reduced/no-preference media emulation, and explicit Full overrode emulated reduction. Actual Windows motion settings were not changed. Native layouts included 1440×900, a resized 1000×900 view with the real editor drawer, and a maximized 1920×1009 empty graph with collapsed navigation, all at DPR 1. A separate **emulated** 1000×700 / DPR 1.5 view also passed its return check. Physical monitor/DPI switching is not claimed. + +### Recordings and evidence + +All final captures use the installed 0.8.1 executable and synthetic content. MP4 encoding preserves original compositor timestamps within **0.5 ms**, with no interpolation or added soundtrack, and includes a short final hold. The MP4 durations are 4.33 s for cold startup, 4.34 s for tray startup and 1.12 s for return. GIFs are normal-speed 25 fps review copies and loop only for review. Both formats decoded successfully; intermediate construction/effect frames and final handovers were visually inspected. + +The session began on 0.8.0, so its private evidence directory retains that name: **`.local/native-acceptance-0.8.0/`**. Final report version fields and these media names identify the tested binary correctly: + +- `Synapse-0.8.1-native-cold-startup.{mp4,gif}` — actual new visible process, capture before animation start. +- `Synapse-0.8.1-native-tray-first-foreground.{mp4,gif}` — actual new hidden process, then native show. +- `Synapse-0.8.1-native-alt-tab-return.{mp4,gif}` — actual operating-system window switch and native focus transition. + +Underlying timestamps, frames, native events and marks are in `cold-final/report.json`, `tray-final/report.json` and `return-final/report.json`. The original return recorder reached its rAF sample cap before the final cycle; its compositor frames, focus events and performance marks still cover the recorded effect. Independent editor/post-dialog traces provide the active-frame cursor/visibility checks above. The helper now keeps a bounded recent history to avoid losing late evidence. + +`acceptance-report.json` verifies 19 case groups and explicitly records `acceptanceComplete: false`. Supporting files include `preview-media-validation.json`, `package-verification.json`, `editor-return-*`, `folder-cancel-close-*`, `folder-success-*`, `export-cancel-*`, `post-dialog-return-*`, preference/layout reports and restoration reports. The final 58-case browser report is `.local/swarm-0.8.1/browser-report.json`; the earlier version-hardcoded helper also wrote this run over `.local/swarm-0.8.0/browser-report.json`, so that old path no longer contains the historical 56-case run. All local captures and private backups remain ignored by Git. Reproduction instructions are in [STARTUP_AUDIO.md](docs/STARTUP_AUDIO.md). + +### Restoration and remaining limits + +The app has no supported isolated test profile. Before testing, the owner's device file, selected-vault record and exact preference string were backed up privately. Synapse was originally closed, with no owner editor session to preserve. Tests selected a disposable synthetic vault and did not invoke a provider or modify hosted funding/Calendar configuration. + +The original device and selected-vault files were restored **byte-for-byte**. The exact original localStorage preference string was restored and verified: schema v1, **Full motion, sound enabled, volume 0.69**, without adding the previously absent animation-toggle fields. Provider selection and consent were preserved. The test process and debugging port 9222 are closed; temporary emulation/capture sessions are closed. Only the disposable Notepad tab was removed, its restored original tabs remained unmodified, and Notepad was closed again. Restoration is recorded at `2026-11-17T08:39:12.340Z`. + +The remaining native cases are input during startup, input during return, an intentional away/back cycle shorter than 500 ms, and repeated qualified cycles within the 1500 ms cooldown. The native automation observation/action round trips could not reliably land those gestures inside their timing windows. An attempted real Tab interruption arrived after startup completed and is explicitly **not** counted as a pass. These cases retain automated browser coverage but require a future suitably timed native run before full native acceptance can be declared. No Computer Use policy stop occurred in this pass. Windows 10 and physical DPI changes are also unverified; the tested host is Windows 11 Pro 10.0.26300. + +Installer: `src-tauri/target/release/bundle/nsis/Synapse_0.8.1_x64-setup.exe` (5,979,260 bytes, unsigned). +SHA-256: `97b2cd6dffe2798ab78cea8e854fd5adff398c8702dc4b5cb41fc24f19c1afbc`. +Installed executable SHA-256: `4011d88563058c97eb5a9d3ceabeed224ba5080cf219cb3cd490506a48f5a9b5`. +The installed GPL v3 licence, copyright notice and third-party notices match the final installer payload. + +## 0.8.0 — 17 November 2026 + +Swarm Rebuild and the separate foreground-return effect are implemented, built and installed locally. **Native recording acceptance remains incomplete.** The supplied 0.8.0 GIF/MP4 previews are browser fixtures using the production controllers and GraphScene; they are not a new native process or a genuine Alt-Tab recording. No GitHub release was published. + +| Check | Result | +| ------------------------------------------------ | -------------------------------------------------------------------------- | +| TypeScript typecheck and ESLint | Passed | +| Frontend tests | 77 passed, including native-focus reconciliation and stale-query cleanup | +| Rust tests | 47 passed; one explicit live-provider test ignored | +| Rustfmt and Clippy, all targets, warnings denied | Passed | +| Browser lifecycle checks | 56 passed on the final source | +| Production build | Passed; unsigned x64 NSIS package built with the existing public AI origin | +| Installed executable and notices | Match the extracted installer payload byte-for-byte | +| Genuine native startup and Alt-Tab recordings | Not completed; see the native limitation below | + +The final browser suite measured **3504.8 ms** for the populated startup and **281.6 ms** for the standard return. Startup used nine agents, 18 UI targets, five actual nodes and four actual edges. The 300-node subnetwork fixture bounded its construction sample to six nodes and four edges. Checks cover dependency ordering, different entrances/durations/exits, matching graph handover positions, no completed-UI-then-hide flash, and no graph-sized selection box. A frozen-controller check confirms that the old short CSS deadline does not reveal content at 1300 ms and that independent fallbacks reveal it by 3900 ms. + +The suite also covers welcome/empty views, collapsed navigation, open drawers, 900×600 and 1920×1080 windows, emulated 100%/150%/200% DPR, Full/Reduced/Follow Windows, both animation toggles, slow restoration/fonts, late eligibility, storage failure, exceptions, Strict Mode, unmount, layout/font/DPR changes, first input and tray-style delayed presentation. Return checks verify one effect per qualifying simulated focus cycle, bounce/chatter/cooldown filtering, success/error dialog suppression with late focus restoration, no startup overlap, zero new AudioContexts, complete timer/RAF/overlay cleanup, and preserved graph state, unsaved textarea content, selection and scroll. Actual native CodeMirror and file-picker return checks remain outstanding. + +During the installed test, native `is_focused` and DOM focus were both true while no Tauri window-focus event had reached the controller; the hidden startup gate remained pending. The final source adds event-driven native focus rechecks on DOM transitions and lets DOM focus wake preparation while retaining the Rust claim as authority. It also discards a stale hidden-window reply arriving after foreground. Both regressions now pass in browser/unit tests. **The corrected focus path has not yet been reverified in an installed foreground session.** Windows Computer Use stopped because it could not determine the other browser window's URL confidently enough to enforce its policy, so desktop input ended. The incomplete native frames are not presented as a successful recording. + +The browser review clips were separately recorded at normal speed: startup **3502.3 ms**, return **283.8 ms**. The MP4s preserve original compositor timestamps within **0.5 ms**, contain no audio streams and include a short final hold. GIFs are 25 fps review copies and loop for review only. Construction, intermediate and final frames were visually inspected. These recordings are `.local/release-0.8.0/Synapse-0.8.0-browser-{startup,return}.{mp4,gif}`; reproduce them with `node scripts/record-swarm-preview.mjs`. The return preview uses a simulated focus lifecycle, not Alt-Tab. + +All runtime data came from a disposable synthetic vault. The original `device.json` and `last-vault.json` were restored byte-for-byte and checked again after installation. The exact existing preference record was never written during runtime testing (Full motion, sound enabled, 69% volume); reading missing animation toggles does not migrate storage. Personal AI selection, consent and funding were preserved. No provider calls or new audio tuning occurred. The testing process and debugging port are closed; Synapse was also closed before the runtime checks began. + +Installer: `src-tauri/target/release/bundle/nsis/Synapse_0.8.0_x64-setup.exe`. +SHA-256: `6748b3130827b95acb5e33128704fa46e6cdc4b2deaf5eafcdd1feede8e14dcc`. +Installed executable SHA-256: `9bee2a490173990dc4f3aa41f207a00d62165ab33d3987858aab50725240d6c2`. +GPL v3, copyright and third-party notices match the installer resources. + +Machine-readable evidence is local and ignored by Git: `.local/swarm-0.8.0/browser-report.json` (56 cases), `.local/release-0.8.0/package-verification.json`, `restored-owner-state.json`, both `browser-*/report.json` files and `preview-media-validation.json`. A resource-contended intermediate browser run correctly skipped preparation at its deadline; normal-motion fixtures now explicitly preload their fonts, while separate slow-font/deadline cases retain the failure-path checks. Windows 10, physical monitor/DPI changes and the final native foreground captures remain unverified. The installer remains unsigned. Historical 0.7.0 audio and native results below are not claimed as new 0.8.0 verification. + +## 0.7.0 — 16 November 2026 + +The installed production build adds the silent cursor-swarm startup and calibrates the original static chimes. All new desktop checks used temporary synthetic vaults. There were no live OpenAI requests, hosted-policy edits, repository visibility changes or new dependencies. The original vault selection, Personal AI selection, consent state, device settings and exact saved preference record (Full animations and 69% volume) were restored. The installed app was restarted normally without a debugging port. + +| Check | Result | +| -------------------------------------------------------- | ----------------------------------------------------- | +| TypeScript typecheck and ESLint | Passed | +| Frontend tests | 75 passed | +| Rust tests | 47 passed; one explicit live-provider test ignored | +| Rust formatting and Clippy, all targets, warnings denied | Passed | +| Hosted gateway | 22 local D1 tests passed with a mocked provider | +| Startup lifecycle in Chromium | 27 cases passed | +| Production audio graph | 150 OfflineAudioContext cases passed | +| Production build and NSIS installation | Passed; installed payload hash verified | +| Installed startup, nested navigation, settings and sound | Passed; real WebView footage and live output captured | + +Startup coverage includes Full, Reduced, Follow Windows, the independent checkbox, slow restoration and fonts, pointer/keyboard/touch/wheel/focus interruption, resize, DPR, late fonts/layout, modal/editor precedence, unmount, real React Strict Mode, background/hidden launch, native focus without a DOM focus event, once-per-launch, unavailable storage, induced failure and the CSS safety reveal. First-interaction handlers observed zero hidden regions and zero overlays without event prevention or replay. The fallback test deliberately stopped JavaScript cleanup and verified compositor visibility; a blocked renderer cannot execute a JavaScript deadline. + +The fresh installed launch with a restored vault selected 12 actual UI targets and completed in **1054.9 ms** from its accepted first foreground frame. The genuine no-vault welcome selected three eligible targets and completed in **1054.8 ms**. Both created zero AudioContexts, removed all hidden attributes/overlays and declined a replay after frontend reload. Actual native focus was established by the owner's Alt+Tab. The clips preserve original compositor timing, with a short hold of the final unchanged frame; no motion frames are interpolated. The GIF is a smaller 25 fps review copy. The measured times include preparation and remain below the 1200 ms deadline. + +The full audio matrix covers subnetwork/topic/concept at 35%, 69%, 70%, 100% and zero volume, including selection, individual arrival, three neighbours, dense overlap, cancellation before and during playback, mute, stale events, disposal and gating. The reference floors at 70% remain −20 dBFS peak for selection and −40 dBFS for individual arrivals. Individual arrivals stay below 70% of corresponding selection peaks and three simultaneous neighbours below 120%. The highest sample peak across all cases was **0.235457**, below the 0.241 bound. Samples were finite, silent cases were zero, tails cleared, reservations/voices ended, every instrumented node connection was released on disposal and one contact-noise buffer was cached per context. The contact's completion did not truncate the tonal body. + +[Startup integration, marker inventory and before/after audio measurements](docs/STARTUP_AUDIO.md) document the production path, active-window RMS, decay and spectrum. The static/glide comparison is matched by active-window RMS, within 0.000001 dB for the reference files. The owner rated the static cues **“Clear and comfortable.”** Static pitch remains the production default; the 12 ms / 25-cent glide exists only in the audition renderer. Speaker frequency response was not measured. + +Installed regression checks covered overview → subnetwork → topic → linked concepts and Back, saved volume, startup checkbox persistence, explicit Full overriding emulated OS reduction, Follow Windows respecting it, and reload without startup replay. Every settled Back canvas at the established 1600×1000 test viewport matched exactly. An additional native 1440×900 comparison differed at 36 of 1,296,000 pixels by at most one 8-bit channel step; this is a recorded rendering tolerance, not an exact-image pass at that viewport. No renderer errors occurred. + +The 12.82-second sound video uses the real app's output at 69% volume and 660 delivered compositor frames plus a final held-frame timestamp. All eight primary cues are present in the source and muxed audio. Their threshold onsets are 24–86 ms after the harness's pre-click markers; those markers precede Playwright's click dispatch, so this is not an input-latency measurement. Encoding preserves frame timestamps within 0.819 ms; source and muxed cue onsets match at the 5 ms analysis resolution. Actual capture cadence during the transitions was approximately 82–96 fps on this host, not a hardware-independent guarantee. + +The unsigned NSIS installer SHA-256 is `caaacb2d5b60e077265c384ec45372762b2da18b4cc0e1f669a4b13ed6f80f8d`. Its installed executable SHA-256 is `0e8c7f33bbcc177a58254a374794860a77963a88a5439c3a3bd6754868d3f548`, matching the extracted installer payload. GPL v3, copyright and complete third-party notices are installed. This update is built and installed locally; it does not publish a new GitHub release. + +Local evidence is excluded from Git: `.local/startup-0.7.0/browser-report.json`, `.local/sound-0.7.0/{before,after}/mix-report.json`, the matched comparison WAVs, `.local/release-0.7.0/{startup-cold,startup-welcome}/report.json`, installed regression/restoration reports, media validation and the actual MP4/GIF/WAV files. The tested host is Windows 11 Pro 10.0.26300. Windows 10 and physical multi-monitor DPI switching remain untested; DPR and OS motion changes were exercised through controlled browser emulation. The installer remains unsigned. + +## 0.6.0 — previous release Validation uses temporary synthetic vaults. No new live provider generation is performed for this release. The existing hosted service and Calendar remain unchanged. @@ -11,7 +194,7 @@ Validation uses temporary synthetic vaults. No new live provider generation is p | Hosted gateway | 22 local D1 tests passed with a mocked OpenAI provider | | Production audio | 54 OfflineAudioContext cases passed across three registers at 69% and maximum volume | | Development desktop | Real Rust/WebView checks passed; details below | -| Production build, NSIS installation and installed recording | Passed; publication awaits the history cleanup decision | +| Production build, NSIS installation and installed recording | Passed; history cleanup and clean public release subsequently completed | Desktop coverage includes overview/subnetwork/topic entry via the real Canvas, canonical linked notes, unchanged review scope, exact Back camera/layout/selection/drawer width/editor mode, close preserving scope, dirty-note cancel/discard, rapid retargeting, Reduced mode without packets, empty/deleted notes, 900×600 uncovered positioning, idle suspension, 90 real new backlinks and duplicate Physics subjects under distinct Neurons. No renderer errors occurred in the completed run. diff --git a/docs/STARTUP_AUDIO.md b/docs/STARTUP_AUDIO.md new file mode 100644 index 0000000..630e372 --- /dev/null +++ b/docs/STARTUP_AUDIO.md @@ -0,0 +1,79 @@ +# Startup, foreground return and sound — 0.9.0 + +Startup and foreground return are silent, disposable presentation layers. Neither unlocks audio, edits a note, invokes AI, moves a graph node or takes native focus. The sound synthesis and gain staging from 0.7.0 are unchanged. `useWorkspace.initialReady` separates actual restoration from construction: the 250 ms preparation budget begins after the workspace is ready, and a slow restore retains the existing restoration screen. + +## Integration and timing + +The v3 opening uses the approved mark with open circular terminals. On a populated desktop, fourteen named crosshairs orbit on ellipses of different sizes, phases and signed speeds. The mark draws above them. Each agent leaves its orbit when its first job is ready. The planner uses each measured target and a seeded schedule to calculate travel, writing pauses and independent finishing times. Each crosshair shrinks at its last working position for 160 ms, with the name badge fading in the first third of that time. + +The populated timeline starts with 900 ms of orbit. Construction ends at 4060 ms and the presentation ends at 4460 ms. Smaller layouts scale the crew and sparse layouts finish earlier. One 300 ms outline pulse follows construction in the existing subject colours. The JavaScript and CSS watchdogs release at 4900 ms. The separate pre-paint preparation budget is 250 ms. Hidden tray launch suspends preparation until foreground presentation. DOM focus and the native `synapse-window-shown` notification wake preparation. Rust checks native visibility and focus before granting the one-use launch claim. Late preparation reveals the interactive workspace. + +Pointer, keyboard, touch, wheel, scroll and composition input synchronously remove startup concealment before the application handler, without cancelling or replaying the input. Modal/editor precedence, unmount, backgrounding, resize, DPR, late fonts, actual layout changes and errors use one idempotent cleanup path. Only owned attributes are restored. Overlays are `aria-hidden`, have no focus targets and use `pointer-events: none`. Temporary observers, RAF, timers, sprites and task collections are released; no animation loop runs at idle. + +`src-tauri/src/startup.rs` creates a UUID and atomic one-use claim for the native process. Hidden/minimised/unfocused windows cannot consume a claim. Guarded session storage and a singleton provide additional protection, including unavailable storage and Strict Mode. Reloading, remounting or navigating cannot replay a consumed launch. Full overrides the OS reduction preference; Reduced bypasses both effects; Follow Windows uses the OS setting. Startup animation and Return animation are independent, default enabled when absent, and do not rewrite existing saved preferences on load. + +## Real graph presentation + +`GraphScene.presentation()` supplies settled projection, clipping, real node sprites, placed labels and connection curves. It examines up to 96 candidate nodes and 256 links, selecting up to six nodes and four edges for construction. The first node is boxed and its copies drag from that master. A writing agent types each selected visible graph label at its measured left edge using the live font, baseline and opacity. Connections begin at the source anchor, follow the pen to the destination and settle onto the live cubic curve. The closing outline follows each eligible visible diamond or junction within the candidate limit. The live graph takes over at the measured coordinates. + +## Foreground return + +`return.ts` has an independent lifecycle. After at least 500 ms away, it confirms foreground for 50 ms and respects a 1500 ms cooldown. `interaction/nativeFocus.ts` uses Tauri focus events as primary and rechecks native focus on DOM focus/blur/visibility transitions and native show notifications to cover missing WebView2 events. Stale asynchronous reads and duplicate notifications are discarded. Browser fixtures use a separate fallback. + +The approximately 280 ms overlay traces a few chrome edges, sends up to three signals along actual graph connections and briefly emphasizes receiving cores. CSS and JavaScript cleanup deadlines are 400 ms. All workspace content stays visible; there are no cursors, typing, duplication or camera changes. Retained editor focus is allowed; actual typing, scrolling, pointer input or IME composition takes precedence. + +`dialogs.ts` wraps native open/save calls with `withNativeDialog`. A depth counter suppresses the entire operation through success, cancellation or error, plus a bounded 1200 ms tail for late focus restoration. Blocking app modals and the permission-bearing reminder test are also suppressed. No permanent suppression or idle polling is used. + +## Marker inventory + +Snapshot selection examines at most 64 markers and chooses at most 20 UI targets. It rejects hidden, offscreen, clipped scrolling rows, zero-size, sensitive and actively edited regions and overlapping opacity targets. The sidebar surface affects its background only, so separately assembled children are never double-hidden. + +| Location | Markers | +| --------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Sidebar | Surface, separate brand icon and text, collapse control, vault heading, search, new note, individual navigation rows, actions and footer; compact rail uses fewer targets | +| Graph | Bounded actual node/edge sample from GraphScene; visible scope heading and count (`text`), empty message and controls (`box`) | +| Welcome | Mark, Synapse heading (`text`), Open vault and Preferences buttons | +| Existing drawer | Header and toolbar chrome only; editor/note bodies are never reconstructed | + +UI lettering uses the visible text range, measured glyph advances and complete Unicode graphemes. Explicit label markers identify eligible navigation captions. Empty search fields supply their placeholders. Typed previews use the decorative canvas and leave accessible text and editor contents intact. Clipped and overflowing UI captions use their live element at handover. + +## Production gain policy + +The signal path is event level × material level × register gain × quiet-cue policy, followed by a tone envelope/filter or the short contact envelope/filter, then common stereo pan → compressor → soft limiter → master × saved user volume. + +- Primary selection event: 0.8; selection material: 1. +- Arrival event: 0.35; arrival material: 0.8. The former extra 0.6 quiet attenuation is removed for arrivals. Dismissal/error cues retain their softer policy. Only three neighbours are scheduled; follow relays retain their 0.07 event level. Existing 60 ms identity gates, 80 ms stale rejection, six-voice budget and cancellation remain. +- Register gains: 0.39 / 0.38 / 0.36, with existing bounded 4% variation. Body durations remain 300 / 235 / 170 ms. Arrival bodies are capped at 100 ms. Static identity-derived pitches and ±1.5-cent repetition variation remain. +- Low-register partials: fundamental 1, second 0.3, third 0.15, fourth 0.012. Upper registers keep their restrained original partials. +- Original deterministic noise: one cached 8 ms buffer per context, band-pass centre 3200 Hz, Q 0.8, relative contact gain 0.12, 0.4 ms attack and zero by 3.5 ms. It shares voice level, panning and every output control. Only the tonal source ends the complete voice; ending the contact cannot disconnect the body. +- Compressor: −16 dB threshold, 4 dB knee, 3:1 ratio, 1 ms attack, 45 ms release. Tanh ceiling 0.32, master 0.75. The maximum-volume output bound remains 0.241 sample amplitude. Zero/mute immediately sets the output to zero and cancels active sources. +- New profiles default to 70%. Existing valid settings, including 69%, low values and zero, are read unchanged and are not rewritten on load. The schema remains v1; missing startup/return toggles mean enabled. + +## 0.7.0 measurements and audition (unchanged synthesis) + +The reference node is `preferences-preview`, using actual shared production event levels, centre pan, 48 kHz stereo OfflineAudioContext and the complete output bus. Both columns use 70% so the comparison does not conflate the new-profile default with synthesis changes. + +| Cue | Before peak / active RMS, dBFS | After peak / active RMS, dBFS | After 20 dB decay | +| --------------------------------- | ------------------------------ | ----------------------------- | ----------------- | +| Subnetwork select | −20.36 / −30.15 | −17.98 / −28.02 | 170 ms | +| Topic select | −22.13 / −32.39 | −19.07 / −28.91 | 125 ms | +| Concept select | −25.00 / −35.55 | −19.47 / −29.49 | 95 ms | +| Concept arrival (production role) | −53.03 / −61.77 | −26.18 / −34.78 | 90 ms | + +Active RMS uses the contiguous non-silent window above 1e−6, measured in one of the identical centre-pan channels. Decay uses 5 ms peak blocks. FFT bands are 0–500 / 500–2000 / 2000–6000 / 6000–24000 Hz. The subnetwork's 500–2000 Hz energy rises from 0.405% to 8.47%, while 91.52% remains below 500 Hz. The report includes all bands for every reference cue. These digital measurements are not a laptop-speaker frequency-response measurement. + +Accepted fixed-reference floors are −20 dBFS peak for selection and −40 dBFS for individual arrivals at 70%; low user volume is deliberately exempt. Individual arrivals must stay below 70% of the corresponding selection peak, and a worst-case three-neighbour overlap below 120%. Floors were preserved when initial tuning failed. The owner listened to the matched comparison on 16 November 2026 and rated the static cues **“Clear and comfortable.”** + +The audition-only 25-cent rise reaches normal pitch in 12 ms. It is injected by the test renderer and matched to each static cue's active RMS; there is no production glide setting or second musical note. `scripts/audio-audition.mjs` assembles static-then-glide pairs in subnetwork/topic/concept order, plus same-volume before/after selection/arrival examples. + +## Reproduce + +Run `node scripts/startup-validation.mjs` for browser lifecycle tests using the production GraphScene and both controllers, including real React Strict Mode, a 300-node scope and missing-native-focus/show regressions. A single case can be selected as the first argument and writes a separate report. Reports use the current package version. `npm test` includes native-focus reconciliation/race tests and saved-preference migration. Audio validation scripts remain available for sound changes; the historical measurements above are from the 0.7.0 audition. + +For passive installed capture, first close Synapse and prepare a disposable selected vault with the original state backed up. Set `SYNAPSE_NATIVE_EVIDENCE` to a new private evidence directory; otherwise the helpers use `.local/native-acceptance-`. Run `node scripts/native-startup.mjs visible cold-final` for a new normal launch or `node scripts/native-startup.mjs tray tray-final` for a hidden launch. The latter requires background operation to be enabled and a genuine native show action after capture is armed. Windows must actually grant foreground focus. The helper observes native state, startup marks, cleanup and compositor timestamps; a completed helper is not itself an acceptance assertion. Inspect the report and first captured timestamp before claiming complete footage. + +With the diagnostic session still available, `node scripts/native-observe.mjs record return-final` arms compositor capture for an actual operating-system Alt-Tab cycle. The `start ` and `stop ` commands collect passive before/after state and native event traces for dialogs, preferences and CodeMirror. They record actual editor text, selection and scroll, so use synthetic content only. The probe keeps a bounded recent frame history and stops any previous probe before starting another. All native CDP connections use `noDefaults: true`: Playwright's default focus emulation would make native focus observations invalid. CDP does not supply the input or call the animation controllers in these helpers. + +Set `FFMPEG` to the existing FFmpeg executable and run `node scripts/encode-native.mjs` to encode the three named captures. The encoder requires a captured frame before the measured animation starts, preserves source timestamps in MP4, checks encoded timing and silence, and produces 25 fps GIF review copies plus decoded inspection frames. The older `record-startup.mjs` and `record-return.mjs` helpers remain available, but the passive helpers above record native evidence. + +Back up device files, the selected vault and exact preference storage before native testing, and restore them afterward. The temporary debugging port is never enabled by the app or installer. Capture frames and machine-readable reports remain under ignored `.local/`. See `VALIDATION.md` for actual completed checks and any native capture limitations; the existence of a recording script is not evidence that a recording succeeded. diff --git a/docs/USER_GUIDE.md b/docs/USER_GUIDE.md index 65ee0f4..bf35a95 100644 --- a/docs/USER_GUIDE.md +++ b/docs/USER_GUIDE.md @@ -80,6 +80,10 @@ One critically damped controller moves the drawer, camera and logarithmic zoom t Preferences contains persistent Animations, Sound and Volume. Animations defaults to **Follow Windows**: disabling Windows animation effects makes camera and panel changes immediate. Choose **Full** for smooth Synapse motion independently of Windows, or **Reduced** for immediate movement and no connection currents. The setting applies consistently to the camera, panels, titles and CSS transitions, and takes effect immediately. +**Startup animation** plays once on the first eligible foreground presentation after a launch. On a populated desktop, fourteen named agents orbit the Synapse mark, then build the visible controls and graph. They drag copies, type labels and draw connections. Each agent fades when its work is done, and the node outlines pulse once. The full sequence takes about 4.46 seconds. Sparse workspaces finish sooner. You can click, type or scroll at any point to take control immediately. + +**Return animation** controls a separate, silent, roughly 280 ms accent when you Alt-Tab back after being away. The current workspace stays visible as a few chrome edges and graph connections briefly light up. It preserves your open note, unsaved text, caret, selection, scroll and graph view. Native file dialogs and brief focus bounces do not trigger it. Both checkboxes default to enabled when absent; Reduced skips both effects, Follow Windows respects the OS preference, and Full explicitly enables normal motion. New profiles start at 70% sound volume; existing saved settings and volume are preserved. + One lazy Web Audio instrument uses original additive synthesis. Subnetwork entries use C4–G4, topics C5–G5, and linked concepts C6–E6, with stable node-derived pitches and bounded attack, detune and level variation. Higher voices are shorter and quieter; deeper exploration stays in that highest register. Back/close and neighbour arrivals are restrained. Six-voice limits, compression/limiting, repeated-event suppression and native-focus cancellation remain in place. Typing, hover, startup and indexing stay silent. No recorded notification or external sound file is bundled. See [interaction architecture](../INTERACTION.md) and [executed validation](../VALIDATION.md) for timing, lifecycle and test boundaries. diff --git a/package-lock.json b/package-lock.json index 8ba107a..9d03c45 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,12 +1,12 @@ { "name": "synapse", - "version": "0.6.0", + "version": "0.9.0", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "synapse", - "version": "0.6.0", + "version": "0.9.0", "dependencies": { "@codemirror/autocomplete": "^6", "@codemirror/commands": "^6", diff --git a/package.json b/package.json index 06610f6..3b2a618 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "synapse", - "version": "0.6.0", + "version": "0.9.0", "private": true, "type": "module", "scripts": { diff --git a/scripts/audio-audition.mjs b/scripts/audio-audition.mjs new file mode 100644 index 0000000..3f05e57 --- /dev/null +++ b/scripts/audio-audition.mjs @@ -0,0 +1,32 @@ +import { readFile, writeFile } from "node:fs/promises"; +const root = ".local/sound-0.7.0"; +async function join(name, paths) { + const waves = await Promise.all( + paths.map((p) => readFile(`${root}/${p}.wav`)), + ); + const data = Buffer.concat(waves.map((b) => b.subarray(44))); + const header = Buffer.from(waves[0].subarray(0, 44)); + header.writeUInt32LE(data.length + 36, 4); + header.writeUInt32LE(data.length, 40); + await writeFile(`${root}/${name}.wav`, Buffer.concat([header, data])); +} +const roles = ["subnetwork", "topic", "concept"]; +await join( + "static-then-glide", + roles.flatMap((role) => [ + `after/${role}-select`, + `after/${role}-glide-matched`, + ]), +); +await join( + "before-then-after", + roles.flatMap((role) => [ + `before/${role}-select`, + `after/${role}-select`, + `before/${role}-arrival`, + `after/${role}-arrival`, + ]), +); +console.log( + "Wrote matched static/glide pairs and same-volume before/after cues.", +); diff --git a/scripts/audio-validation.mjs b/scripts/audio-validation.mjs index 1c06f3a..c54e3c3 100644 --- a/scripts/audio-validation.mjs +++ b/scripts/audio-validation.mjs @@ -1,65 +1,197 @@ -// Render the production voice graph in real Chromium Web Audio. No audio mocks. -// These numerical renders are separate from the installed-app audition. +// Real Chromium OfflineAudioContext, production event levels and complete bus. +// --baseline records the previous mix without applying the new audibility floors. +// Synthetic reference renders; the native app audition is recorded separately. import { chromium } from "@playwright/test"; import { createServer } from "vite"; import { mkdir, writeFile } from "node:fs/promises"; import assert from "node:assert/strict"; -const out = ".local/sound-0.6.0"; +const baseline = process.argv.includes("--baseline"); +const out = `.local/sound-0.7.0/${baseline ? "before" : "after"}`; await mkdir(out, { recursive: true }); const server = await createServer({ - server: { host: "127.0.0.1", port: 1430 }, + server: { host: "127.0.0.1", port: 1430, strictPort: true }, }); await server.listen(); const browser = await chromium.launch({ channel: "msedge", headless: true }); +function wav(channels, rate = 48000) { + const n = channels[0].length, + b = Buffer.alloc(44 + n * channels.length * 2); + b.write("RIFF"); + b.writeUInt32LE(b.length - 8, 4); + b.write("WAVEfmt ", 8); + b.writeUInt32LE(16, 16); + b.writeUInt16LE(1, 20); + b.writeUInt16LE(channels.length, 22); + b.writeUInt32LE(rate, 24); + b.writeUInt32LE(rate * channels.length * 2, 28); + b.writeUInt16LE(channels.length * 2, 32); + b.writeUInt16LE(16, 34); + b.write("data", 36); + b.writeUInt32LE(b.length - 44, 40); + for (let i = 0; i < n; i++) + for (let c = 0; c < channels.length; c++) + b.writeInt16LE( + Math.round(Math.max(-1, Math.min(1, channels[c][i])) * 32767), + 44 + (i * channels.length + c) * 2, + ); + return b; +} try { const page = await browser.newPage(); - await page.route("**/audio-check", (route) => - route.fulfill({ + await page.route("**/audio-check", (r) => + r.fulfill({ contentType: "text/html", - body: "Synapse production audio validation", + body: "Production audio validation", }), ); await page.goto("http://127.0.0.1:1430/audio-check"); const report = await page.evaluate(async () => { const { NoteSynth } = await import("/src/interaction/audio.ts"); const { nodeVoice } = await import("/src/interaction/note.ts"); - const peakOf = (samples) => - samples.reduce((peak, x) => Math.max(peak, Math.abs(x)), 0); - const cases = []; - for (const volume of [0.69, 1]) { - for (const role of ["subnetwork", "topic", "concept"]) { + const { actionLevel, ARRIVAL_LEVEL } = + await import("/src/interaction/soundLevels.ts"); + const roles = ["subnetwork", "topic", "concept"]; + const peakOf = (a) => a.reduce((p, x) => Math.max(p, Math.abs(x)), 0); + const db = (x) => (x > 0 ? 20 * Math.log10(x) : null); + // FFT of the entire decaying cue, zero padded. Report percent energy. + function spectrum(samples) { + const n = 32768, + re = new Float64Array(n), + im = new Float64Array(n); + re.set(samples.slice(0, n)); + for (let i = 1, j = 0; i < n; i++) { + let bit = n >> 1; + for (; j & bit; bit >>= 1) j ^= bit; + j ^= bit; + if (i < j) [re[i], re[j]] = [re[j], re[i]]; + } + for (let len = 2; len <= n; len <<= 1) { + const angle = (-2 * Math.PI) / len, + wr = Math.cos(angle), + wi = Math.sin(angle); + for (let i = 0; i < n; i += len) { + let ar = 1, + ai = 0; + for (let j = 0; j < len / 2; j++) { + const k = i + j, + l = k + len / 2, + tr = re[l] * ar - im[l] * ai, + ti = re[l] * ai + im[l] * ar; + re[l] = re[k] - tr; + im[l] = im[k] - ti; + re[k] += tr; + im[k] += ti; + [ar, ai] = [ar * wr - ai * wi, ar * wi + ai * wr]; + } + } + } + const bands = [0, 0, 0, 0]; + for (let i = 1; i < n / 2; i++) { + const hz = (i * 48000) / n, + band = hz < 500 ? 0 : hz < 2000 ? 1 : hz < 6000 ? 2 : 3; + bands[band] += re[i] ** 2 + im[i] ** 2; + } + const sum = bands.reduce((a, b) => a + b, 0); + return bands.map((x) => (sum ? x / sum : 0)); + } + function metrics(channels, spectral) { + const a = channels[0], + peak = Math.max(...channels.map(peakOf)); + let first = a.findIndex((x) => Math.abs(x) > 1e-6), + last = a.length - 1; + while (last >= 0 && Math.abs(a[last]) <= 1e-6) last--; + first = Math.max(0, first); + let energy = 0; + for (let i = first; i <= last; i++) energy += a[i] ** 2; + const activeRms = + last >= first ? Math.sqrt(energy / (last - first + 1)) : 0; + const blocks = Array.from( + { length: Math.floor(a.length / 240) }, + (_, i) => peakOf(a.slice(i * 240, (i + 1) * 240)), + ); + const maximum = Math.max(...blocks), + peakBlock = blocks.indexOf(maximum); + const decayBlock = blocks.findIndex( + (x, i) => i > peakBlock && x <= maximum / 10, + ); + return { + peak, + peakDbfs: db(peak), + activeRms, + activeRmsDbfs: db(activeRms), + activeMs: Math.max(0, last - first + 1) / 48, + decay20DbMs: + maximum && decayBlock >= 0 ? (decayBlock - peakBlock) * 5 : null, + spectralEnergy: spectral ? spectrum(a) : undefined, + bodyPeakAfterContact: peakOf(a.slice(48 * 40, 48 * 75)), + finite: channels.every((a) => a.every(Number.isFinite)), + tail: Math.max(...channels.map((a) => peakOf(a.slice(-4800)))), + }; + } + const cases = [], + clips = {}; + for (const volume of [0.35, 0.7, 0.69, 1, 0]) + for (const role of roles) { for (const name of [ - "single", + "select", + "arrival", "neighbours", "dense", "cancelled", + "mid-cancel", "muted", - "zero", "stale", "disposed", "gated", ]) { const context = new OfflineAudioContext(2, 38400, 48000); - const synth = new NoteSynth(context, name === "zero" ? 0 : volume); - const contact = { - id: "v0", - at: 0.02, - kind: "select", - level: 0.8, - subject: "s0", - role, + const connected = new Set(); + let noiseBuffers = 0; + const createBuffer = context.createBuffer.bind(context); + context.createBuffer = (...args) => { + noiseBuffers++; + return createBuffer(...args); }; - let accepted = 0; - const count = ["dense", "neighbours"].includes(name) - ? 30 - : name === "gated" - ? 2 - : 1; + for (const method of [ + "createGain", + "createBiquadFilter", + "createStereoPanner", + "createOscillator", + "createBufferSource", + "createDynamicsCompressor", + "createWaveShaper", + ]) { + const create = context[method].bind(context); + context[method] = (...args) => { + const node = create(...args), + connect = node.connect.bind(node), + disconnect = node.disconnect.bind(node); + node.connect = (...destinations) => { + connected.add(node); + return connect(...destinations); + }; + node.disconnect = (...destinations) => { + connected.delete(node); + return disconnect(...destinations); + }; + return node; + }; + } + const synth = new NoteSynth(context, volume); + const arrival = name === "arrival" || name === "neighbours"; if (name === "disposed") synth.dispose(); - for (let i = 0; i < count; i++) { + let accepted = 0; + const count = + name === "dense" + ? 30 + : name === "neighbours" + ? 3 + : name === "gated" + ? 2 + : 1; + for (let i = 0; i < count; i++) if ( synth.play({ - ...contact, id: `v${i}`, at: name === "stale" @@ -67,78 +199,209 @@ try { : name === "gated" ? 0.02 + i * 0.01 : 0.02, - subject: name === "gated" ? "same" : `s${i}`, - kind: name === "neighbours" ? "arrival" : "select", - level: name === "neighbours" ? 0.22 : 1, + kind: arrival ? "arrival" : "select", + level: arrival ? ARRIVAL_LEVEL : actionLevel("select"), + node: + name === "gated" + ? "preferences-preview" + : i + ? `neighbour-${i}` + : "preferences-preview", + role, + pan: 0, }) ) accepted++; - } if (name === "cancelled") synth.cancel(); if (name === "muted") synth.mute(true, volume); const maximumVoices = synth.activeVoices; - const buffer = await context.startRendering(); - const channels = [buffer.getChannelData(0), buffer.getChannelData(1)]; - const peak = Math.max(...channels.map(peakOf)); - const rms = Math.sqrt( - channels[0].reduce((sum, x) => sum + x * x, 0) / buffer.length, - ); + let afterCancel; + if (name === "mid-cancel") + context.suspend(0.06).then(() => { + synth.cancel(); + afterCancel = synth.activeVoices; + void context.resume(); + }); + const buffer = await context.startRendering(), + channels = [buffer.getChannelData(0), buffer.getChannelData(1)]; const result = { name, role, volume, - peak, - rms, + ...metrics(channels, name === "select" || name === "arrival"), accepted, maximumVoices, + afterCancel, remaining: synth.activeVoices, reservations: synth.budget.voices.size, - finite: channels.every((a) => a.every(Number.isFinite)), - tail: Math.max(...channels.map((a) => peakOf(a.slice(-4800)))), + connectedAfterVoices: connected.size, + noiseBuffers, }; - synth.dispose(); cases.push(result); + if (volume === 0.7 && (name === "select" || name === "arrival")) + clips[`${role}-${name}`] = channels.map((a) => Array.from(a)); + synth.dispose(); + result.connectedAfterDispose = connected.size; } } + // Audition only. Production oscillators remain at a static pitch. + const matched = []; + for (const role of roles) { + const context = new OfflineAudioContext(2, 38400, 48000), + create = context.createOscillator.bind(context); + context.createOscillator = () => { + const osc = create(), + start = osc.start.bind(osc); + osc.start = (at) => { + const pitch = osc.detune.value; + osc.detune.setValueAtTime(pitch - 25, at); + osc.detune.linearRampToValueAtTime(pitch, at + 0.012); + start(at); + }; + return osc; + }; + const synth = new NoteSynth(context, 0.7); + synth.play({ + id: "candidate", + at: 0.02, + kind: "select", + level: actionLevel("select"), + role, + node: "preferences-preview", + }); + const buffer = await context.startRendering(), + a = [buffer.getChannelData(0), buffer.getChannelData(1)]; + const before = metrics(a, true), + reference = cases.find( + (c) => c.volume === 0.7 && c.role === role && c.name === "select", + ); + const gain = reference.activeRms / before.activeRms; + const adjusted = a.map((c) => c.map((x) => x * gain)); + clips[`${role}-glide-matched`] = adjusted.map((a) => Array.from(a)); + matched.push({ + role, + gain, + static: reference.activeRmsDbfs, + candidate: metrics(adjusted, true), + }); + synth.dispose(); } return { - palette: ["subnetwork", "topic", "concept"].map((role) => - nodeVoice({ role }), + fixture: { + node: "preferences-preview", + select: actionLevel("select"), + arrival: ARRIVAL_LEVEL, + pan: 0, + sampleRate: 48000, + spectralBandsHz: ["0–500", "500–2000", "2000–6000", "6000–24000"], + }, + palette: roles.map((role) => + nodeVoice({ role, node: "preferences-preview" }), ), cases, + matched, + clips, }; }); + const { clips, ...measurements } = report; + await writeFile( + `${out}/mix-report.json`, + JSON.stringify(measurements, null, 2), + ); + for (const [name, channels] of Object.entries(clips)) + await writeFile(`${out}/${name}.wav`, wav(channels)); for (const c of report.cases) { assert.ok( - c.finite && c.peak < 0.241 && c.maximumVoices <= 6, + c.finite && c.peak <= 0.241 && c.maximumVoices <= 6, JSON.stringify(c), ); assert.equal(c.remaining, 0); assert.equal(c.reservations, 0); + assert.equal( + c.connectedAfterDispose, + 0, + "Every tone/contact/output node disconnects", + ); + assert.ok( + c.connectedAfterVoices <= 3, + "Only the three output bus nodes remain after a cue", + ); + if (!baseline && c.accepted) + assert.equal(c.noiseBuffers, 1, "One cached contact buffer per context"); assert.ok(c.tail < 1e-8); - if (["cancelled", "muted", "zero", "stale", "disposed"].includes(c.name)) + if ( + ["cancelled", "muted", "stale", "disposed"].includes(c.name) || + !c.volume + ) assert.equal(c.peak, 0); - if (["dense", "neighbours"].includes(c.name)) assert.equal(c.accepted, 6); + if (c.name === "dense") assert.equal(c.accepted, 6); + if (c.name === "neighbours") assert.equal(c.accepted, 3); if (c.name === "gated") assert.equal(c.accepted, 1); - if (c.name === "single") assert.ok(c.peak > 0.01); - } - for (const volume of [0.69, 1]) { - for (const role of ["subnetwork", "topic", "concept"]) { - const single = report.cases.find( - (c) => c.name === "single" && c.volume === volume && c.role === role, - ); - const neighbours = report.cases.find( - (c) => - c.name === "neighbours" && c.volume === volume && c.role === role, - ); - assert.ok(neighbours.peak < single.peak * 0.4); + if (c.name === "mid-cancel") assert.equal(c.afterCancel, 0); + if (!baseline && c.volume === 0.7) { + if (c.name === "select") + assert.ok(c.peak >= 0.1, `Select floor −20 dBFS: ${JSON.stringify(c)}`); + if (c.name === "arrival") + assert.ok( + c.peak >= 0.01, + `Arrival floor −40 dBFS: ${JSON.stringify(c)}`, + ); } + if (!baseline && c.volume && c.name === "select") + assert.ok( + c.bodyPeakAfterContact > c.peak * 0.05, + "Contact ending must leave the tone alive", + ); } - await writeFile( - `${out}/mix-report.json`, - JSON.stringify({ success: true, ...report }, null, 2), + if (!baseline) + for (const volume of [0.7, 0.69, 1]) + for (const role of ["subnetwork", "topic", "concept"]) { + const cue = (name) => + report.cases.find( + (c) => c.name === name && c.role === role && c.volume === volume, + ); + assert.ok( + cue("arrival").peak < cue("select").peak * 0.7, + `Individual arrival must remain subordinate: ${role}`, + ); + assert.ok( + cue("neighbours").peak < cue("select").peak * 1.2, + `Three arrivals must not dominate selection: ${role}`, + ); + } + console.log( + JSON.stringify( + { + success: true, + baseline, + cases: report.cases.length, + out, + reference: report.cases + .filter( + (c) => c.volume === 0.7 && ["select", "arrival"].includes(c.name), + ) + .map( + ({ + name, + role, + peakDbfs, + activeRmsDbfs, + decay20DbMs, + spectralEnergy, + }) => ({ + name, + role, + peakDbfs, + activeRmsDbfs, + decay20DbMs, + spectralEnergy, + }), + ), + }, + null, + 2, + ), ); - console.log(JSON.stringify(report, null, 2)); } finally { await browser.close(); await server.close(); diff --git a/scripts/encode-native.mjs b/scripts/encode-native.mjs new file mode 100644 index 0000000..08afe85 --- /dev/null +++ b/scripts/encode-native.mjs @@ -0,0 +1,235 @@ +// Preserve CDP's source timestamps; the only repeated frame is the final hold. +import { readFile, writeFile, stat } from "node:fs/promises"; +import { spawnSync } from "node:child_process"; +import assert from "node:assert/strict"; +import path from "node:path"; +const source = JSON.parse(await readFile("package.json", "utf8")); +const root = path.resolve( + process.argv[2] ?? + process.env.SYNAPSE_NATIVE_EVIDENCE ?? + `.local/native-acceptance-${source.version}`, +); +const ffmpeg = process.env.FFMPEG; +assert.ok(ffmpeg, "Set FFMPEG to the existing executable"); +function run(args) { + const r = spawnSync(ffmpeg, ["-hide_banner", ...args], { + encoding: "utf8", + windowsHide: true, + maxBuffer: 16 * 1024 * 1024, + }); + assert.equal(r.status, 0, r.stderr); + return r.stderr; +} +const reports = []; +for (const name of ["cold-final", "tray-final", "return-final"]) { + const dir = path.join(root, name); + const r = JSON.parse(await readFile(path.join(dir, "report.json"), "utf8")); + const effect = name === "return-final" ? "return" : "startup"; + const measure = r.measures + .filter((e) => e.name === `synapse.${effect}`) + .at(-1); + assert.equal( + r.version, + source.version, + `${name}: installed version differs from the requested release`, + ); + assert.ok( + measure && + r.marks.some( + (m) => + m.name === `synapse.${effect}:finish:complete` && + Math.abs(m.at - measure.startTime - measure.duration) < 2, + ), + `${name}: missing completed effect`, + ); + assert.equal(r.gate, null); + assert.equal(r.concealed + r.overlays + r.audio, 0); + if (effect === "startup") { + const info = r.marks.find((m) => m.name === "synapse.startup:start").detail; + assert.equal(info.agents, 14); + assert.equal(info.orbitEnd, 900); + assert.equal(info.constructionEnd, 4060); + assert.ok( + info.graphLabels > 0 && + measure.duration >= 4400 && + measure.duration <= 4900, + ); + } else { + assert.ok(measure.duration >= 260 && measure.duration <= 400); + assert.deepEqual( + r.audioStarts, + [], + "The recorded return must not start a sound", + ); + assert.equal(r.before.canvasHash, r.after.canvasHash); + assert.deepEqual(r.before.editor, r.after.editor); + assert.equal(r.after.measures.length - r.before.measures.length, 1); + assert.ok(r.native.some((s) => !s.focused) && r.native.at(-1)?.focused); + const active = r.frames.filter((f) => f.returning); + assert.ok( + active.length && + active.every((f) => !f.cursors && !f.concealed && !f.gate), + ); + } + const epoch = (r.timeOrigin + measure.startTime) / 1000; + const start = epoch - 0.08, + end = epoch + measure.duration / 1000 + 0.75; + const ordered = r.captureFrames + .toSorted((a, b) => a.time - b.time) + .filter((f, i, a) => !i || f.time !== a[i - 1].time); + assert.ok( + ordered[0].time < epoch, + `${name}: capture missed first animation frame`, + ); + const prior = ordered.filter((f) => f.time <= start).at(-1); + const timeline = [ + ...(prior ? [{ ...prior, time: start }] : []), + ...ordered.filter((f) => f.time > start && f.time < end), + ]; + const concat = ["ffconcat version 1.0"]; + timeline.forEach((f, i) => + concat.push( + `file '${f.file.replaceAll("\\", "/")}'`, + "option framerate 1000", + `duration ${Math.max(0.001, (timeline[i + 1]?.time ?? end) - f.time).toFixed(6)}`, + ), + ); + concat.push( + `file '${timeline.at(-1).file.replaceAll("\\", "/")}'`, + "option framerate 1000", + ); + const concatPath = path.join(dir, "clip.ffconcat"); + await writeFile(concatPath, concat.join("\n") + "\n"); + const label = + name === "cold-final" + ? "native-cold-startup" + : name === "tray-final" + ? "native-tray-first-foreground" + : "native-alt-tab-return"; + const stem = path.join(root, `Synapse-${r.version}-${label}`); + run([ + "-loglevel", + "error", + "-y", + "-f", + "concat", + "-safe", + "0", + "-i", + concatPath, + "-an", + // WebView2's JPEGs are full-range. Tag a real conversion to video range + // so Windows video playback preserves the near-black canvas and edges. + "-vf", + "scale=in_range=full:out_range=limited:in_color_matrix=bt601:out_color_matrix=bt709,format=yuv420p", + "-color_range", + "tv", + "-colorspace", + "bt709", + "-color_primaries", + "bt709", + "-color_trc", + "iec61966-2-1", + "-c:v", + "libx264", + "-preset", + "medium", + "-crf", + "18", + "-pix_fmt", + "yuv420p", + "-bf", + "0", + "-fps_mode", + "vfr", + "-enc_time_base", + "1:1000", + "-x264-params", + "fps=120/1:force-cfr=0", + "-level:v", + "5.2", + "-video_track_timescale", + "1000000", + "-movflags", + "+faststart", + `${stem}.mp4`, + ]); + run([ + "-loglevel", + "error", + "-y", + "-i", + `${stem}.mp4`, + "-filter_complex", + "fps=25,scale=1000:-2:flags=lanczos,split[a][b];[a]palettegen=stats_mode=diff[p];[b][p]paletteuse=dither=sierra2_4a:diff_mode=rectangle", + "-loop", + "0", + `${stem}.gif`, + ]); + const decoded = run([ + "-i", + `${stem}.mp4`, + "-vf", + "showinfo", + "-an", + "-f", + "null", + "NUL", + ]); + await writeFile(path.join(dir, "decode.log"), decoded); + assert.match(decoded, /yuv420p\(tv, bt709/); + const actual = [ + ...decoded.matchAll(/\bn:\s*\d+\s+pts:.*?pts_time:([-\d.]+)/g), + ].map((m) => Number(m[1])); + let time = 0; + const expected = []; + for (const line of concat) { + if (line.startsWith("file ")) expected.push(time); + if (line.startsWith("duration ")) time += Number(line.slice(9)); + } + assert.equal(actual.length, expected.length); + const maxTimingErrorMs = Math.max( + ...actual.map((t, i) => Math.abs(t - expected[i]) * 1000), + ); + assert.ok(maxTimingErrorMs <= 1.05); + assert.equal([...decoded.matchAll(/Stream #0:\d.*?Audio:/g)].length, 0); + run(["-loglevel", "error", "-i", `${stem}.gif`, "-f", "null", "NUL"]); + const times = + effect === "startup" + ? [0.48, 1.2, 1.8, 2.45, 3.2, 4.25, measure.duration / 1000 + 0.2] + : [0.14, 0.24, 0.7]; + for (const [i, t] of times.entries()) + run([ + "-loglevel", + "error", + "-y", + "-ss", + String(t), + "-i", + `${stem}.mp4`, + "-frames:v", + "1", + path.join(dir, `review-${i}.png`), + ]); + reports.push({ + name, + version: r.version, + native: true, + normalSpeed: true, + firstCapturedRelativeToStartMs: (ordered[0].time - epoch) * 1000, + animationMs: measure.duration, + frames: actual.length, + maxTimingErrorMs, + metadataDuration: /Duration: (\d+:\d+:\d+\.\d+)/.exec(decoded)?.[1], + mp4Bytes: (await stat(`${stem}.mp4`)).size, + gifBytes: (await stat(`${stem}.gif`)).size, + silent: true, + gifFps: 25, + stem, + }); +} +await writeFile( + path.join(root, "preview-media-validation.json"), + JSON.stringify(reports, null, 2), +); +console.log(JSON.stringify(reports, null, 2)); diff --git a/scripts/native-observe.mjs b/scripts/native-observe.mjs new file mode 100644 index 0000000..b2b650f --- /dev/null +++ b/scripts/native-observe.mjs @@ -0,0 +1,234 @@ +// Read-only native inspection and explicitly armed passive event collection. +import { chromium } from "@playwright/test"; +import { readFile, writeFile, mkdir } from "node:fs/promises"; +import { createHash } from "node:crypto"; +import path from "node:path"; +import { startProbe, stopProbe } from "./native-probe.mjs"; +const command = process.argv[2] ?? "state", + name = process.argv[3] ?? command; +const source = JSON.parse(await readFile("package.json", "utf8")); +const base = path.resolve( + process.env.SYNAPSE_NATIVE_EVIDENCE ?? + `.local/native-acceptance-${source.version}`, +); +await mkdir(base, { recursive: true }); +const browser = await chromium.connectOverCDP("http://127.0.0.1:9222", { + noDefaults: true, +}); +const page = browser.contexts()[0].pages()[0]; +const state = async () => { + const result = await page.evaluate(async () => { + const editor = document.querySelector(".cm-content")?.cmTile?.root?.view; + const graph = document.querySelector(".force-graph-container canvas"); + const selection = window.getSelection(); + return { + version: await window.__TAURI_INTERNALS__.invoke("plugin:app|version"), + nativeFocus: await window.__TAURI_INTERNALS__.invoke( + "plugin:window|is_focused", + { label: "main" }, + ), + visible: await window.__TAURI_INTERNALS__.invoke( + "plugin:window|is_visible", + { label: "main" }, + ), + focused: document.hasFocus(), + hidden: document.hidden, + width: innerWidth, + height: innerHeight, + dpr: devicePixelRatio, + heading: document.querySelector(".graph-heading h1")?.textContent, + zoom: document.querySelector(".graph-controls")?.textContent, + drawer: (() => { + const e = document.querySelector(".editor-shell"); + if (!e) return null; + const r = e.getBoundingClientRect(); + return { + x: r.x, + y: r.y, + width: r.width, + height: r.height, + inert: e.inert, + transform: getComputedStyle(e).transform, + }; + })(), + editor: editor + ? { + text: editor.state.doc.toString(), + selection: editor.state.selection.toJSON(), + scrollTop: editor.scrollDOM.scrollTop, + scrollLeft: editor.scrollDOM.scrollLeft, + focused: editor.hasFocus, + } + : null, + mode: document.querySelector(".editor-toolbar [aria-pressed=true]") + ?.textContent, + selectionText: selection?.toString(), + preferences: localStorage.getItem("synapse.preferences.v1"), + effectiveMotion: document + .querySelector("main.app") + ?.getAttribute("data-motion"), + gate: + document + .querySelector("[data-swarm-gate]") + ?.getAttribute("data-swarm-gate") ?? null, + hiddenCount: document.querySelectorAll( + '[data-swarm-hidden="true"],[data-swarm-surface-hidden="true"]', + ).length, + overlays: document.querySelectorAll(".swarm-overlay").length, + marks: performance + .getEntriesByType("mark") + .filter((e) => e.name.startsWith("synapse.")) + .map((e) => ({ name: e.name, at: e.startTime, detail: e.detail })), + measures: performance + .getEntriesByType("measure") + .filter((e) => e.name.startsWith("synapse.")) + .map((e) => e.toJSON()), + canvas: graph?.toDataURL(), + }; + }); + const { canvas, ...rest } = result; + return { + ...rest, + canvasHash: canvas + ? createHash("sha256").update(canvas).digest("hex") + : null, + }; +}; +if (command === "start") { + await startProbe(page); + await writeFile( + path.join(base, `${name}-before.json`), + JSON.stringify(await state(), null, 2), + ); + console.log(`Passive observation armed: ${name}`); +} else if (command === "stop") { + const probe = await stopProbe(page), + after = await state(); + await writeFile( + path.join(base, `${name}-after.json`), + JSON.stringify(after, null, 2), + ); + await writeFile( + path.join(base, `${name}-trace.json`), + JSON.stringify(probe, null, 2), + ); + const before = JSON.parse( + await readFile(path.join(base, `${name}-before.json`), "utf8"), + ); + console.log( + JSON.stringify( + { + name, + version: after.version, + addedMeasures: after.measures.slice(before.measures.length), + native: probe.native, + events: probe.events.map((e) => ({ + kind: e.kind, + key: e.key, + at: e.at, + focused: e.focused, + })), + gate: after.gate, + hidden: after.hiddenCount, + overlays: after.overlays, + canvasUnchanged: after.canvasHash === before.canvasHash, + editorPreserved: + JSON.stringify(after.editor) === JSON.stringify(before.editor), + heading: after.heading, + }, + null, + 2, + ), + ); +} else if (command === "record") { + await mkdir(path.join(base, name), { recursive: true }); + const before = await state(); + await startProbe(page); + const cdp = await browser.contexts()[0].newCDPSession(page), + frames = [], + writes = []; + cdp.on("Page.screencastFrame", (e) => { + void cdp.send("Page.screencastFrameAck", { sessionId: e.sessionId }); + const file = path.join( + base, + name, + `${String(frames.length).padStart(5, "0")}.jpg`, + ); + frames.push({ file, time: e.metadata.timestamp }); + writes.push(writeFile(file, Buffer.from(e.data, "base64"))); + }); + await cdp.send("Page.startScreencast", { + format: "jpeg", + quality: 95, + maxWidth: 1920, + maxHeight: 1200, + everyNthFrame: 1, + }); + console.log(`Native return recording armed: ${name}`); + const count = before.measures.filter( + (e) => e.name === "synapse.return", + ).length; + await page.waitForFunction( + (count) => performance.getEntriesByName("synapse.return").length > count, + count, + { timeout: 600000 }, + ); + await page.waitForTimeout(900); + await cdp.send("Page.stopScreencast"); + await Promise.all(writes); + const probe = await stopProbe(page), + after = await state(); + await writeFile( + path.join(base, name, "report.json"), + JSON.stringify( + { + native: true, + version: after.version, + before, + after, + captureFrames: frames, + ...probe, + }, + null, + 2, + ), + ); + console.log( + JSON.stringify( + { + name, + duration: after.measures.at(-1)?.duration, + finish: after.marks.at(-1)?.name, + native: probe.native, + frames: frames.length, + canvasUnchanged: before.canvasHash === after.canvasHash, + editorPreserved: + JSON.stringify(before.editor) === JSON.stringify(after.editor), + }, + null, + 2, + ), + ); +} else { + const result = await state(); + await writeFile( + path.join(base, `${name}.json`), + JSON.stringify(result, null, 2), + ); + console.log( + JSON.stringify( + { + ...result, + editor: result.editor + ? { + ...result.editor, + text: `${result.editor.text.length} characters`, + } + : null, + }, + null, + 2, + ), + ); +} +await browser.close(); diff --git a/scripts/native-probe.mjs b/scripts/native-probe.mjs new file mode 100644 index 0000000..5841f82 --- /dev/null +++ b/scripts/native-probe.mjs @@ -0,0 +1,230 @@ +// Passive diagnostics for the installed synthetic-vault session. No UI input. +export async function startProbe(page) { + await page.evaluate(() => { + window.nativeAcceptance?.stop?.(); + const q = (window.nativeAcceptance = { + frames: [], + events: [], + native: [], + audio: 0, + audioStarts: [], + }); + const snapshot = () => ({ + at: performance.now(), + focused: document.hasFocus(), + hidden: document.hidden, + gate: + document + .querySelector("[data-swarm-gate]") + ?.getAttribute("data-swarm-gate") ?? null, + concealed: document.querySelectorAll( + '[data-swarm-hidden="true"],[data-swarm-surface-hidden="true"]', + ).length, + startup: document.querySelectorAll(".swarm-startup").length, + returning: document.querySelectorAll(".swarm-return").length, + cursors: document.querySelectorAll(".swarm-cursor").length, + opacity: (() => { + const e = document.querySelector( + ".brand [data-swarm='text'],.welcome h1", + ); + return e ? getComputedStyle(e).opacity : null; + })(), + }); + q.originalAudio = window.AudioContext; + window.AudioContext = new Proxy(window.AudioContext, { + construct(target, args, receiver) { + q.audio++; + return Reflect.construct(target, args, receiver); + }, + }); + const audioSources = [AudioBufferSourceNode, OscillatorNode].map((type) => { + const descriptor = Object.getOwnPropertyDescriptor( + type.prototype, + "start", + ); + const start = type.prototype.start; + type.prototype.start = function (...args) { + q.audioStarts.push({ kind: type.name, at: performance.now() }); + return Reflect.apply(start, this, args); + }; + return { type, descriptor }; + }); + const record = (e) => { + q.events.push({ kind: e.type, key: e.key, ...snapshot() }); + }; + const events = [ + "focus", + "blur", + "visibilitychange", + "pointerdown", + "keydown", + "beforeinput", + "input", + "compositionstart", + ]; + events.forEach((name) => + (name === "visibilitychange" ? document : window).addEventListener( + name, + record, + true, + ), + ); + const frame = () => { + // Keep recent evidence even if the operator takes a while to return. + // Trim in batches so this passive observer does not shift on every frame. + if (q.frames.length >= 18000) q.frames.splice(0, 6000); + const sample = snapshot(); + q.frames.push(sample); + if (sample.startup && !q.schedule) { + const plan = globalThis[Symbol.for("synapse.startup.controller")]?.plan; + if (plan) + q.schedule = { + orbitEnd: plan.orbitEnd, + constructionEnd: plan.constructionEnd, + duration: plan.duration, + centre: plan.centre, + scale: plan.scale, + actors: plan.agents.map( + ({ id, name, radius, speed, departure, gone }) => ({ + id, + name, + radius, + speed, + departure, + gone, + }), + ), + tasks: plan.tasks.map( + ({ + id, + kind, + agent, + start, + work, + end, + typing, + anchor, + destination, + }) => ({ + id, + kind, + agent, + start, + work, + end, + characters: typing.length, + anchor, + destination, + }), + ), + }; + } + q.raf = requestAnimationFrame(frame); + }; + q.raf = requestAnimationFrame(frame); + let pending = false; + const poll = async () => { + if (pending) return; + pending = true; + const at = performance.now(); + try { + const [focused, visible] = await Promise.all([ + window.__TAURI_INTERNALS__.invoke("plugin:window|is_focused", { + label: "main", + }), + window.__TAURI_INTERNALS__.invoke("plugin:window|is_visible", { + label: "main", + }), + ]); + const last = q.native.at(-1); + if (!last || last.focused !== focused || last.visible !== visible) + q.native.push({ at, received: performance.now(), focused, visible }); + } finally { + pending = false; + } + }; + q.timer = setInterval(poll, 50); + void poll(); + q.stop = () => { + cancelAnimationFrame(q.raf); + clearInterval(q.timer); + events.forEach((name) => + (name === "visibilitychange" ? document : window).removeEventListener( + name, + record, + true, + ), + ); + window.AudioContext = q.originalAudio; + for (const { type, descriptor } of audioSources) { + if (descriptor) + Object.defineProperty(type.prototype, "start", descriptor); + else delete type.prototype.start; + } + }; + }); +} +export async function stopProbe(page) { + return page.evaluate(() => { + const q = window.nativeAcceptance; + q.stop(); + return { + frames: q.frames, + schedule: q.schedule, + events: q.events, + native: q.native, + audio: q.audio, + audioStarts: q.audioStarts, + timeOrigin: performance.timeOrigin, + marks: performance + .getEntriesByType("mark") + .filter((e) => e.name.startsWith("synapse.")) + .map((e) => ({ name: e.name, at: e.startTime, detail: e.detail })), + measures: performance + .getEntriesByType("measure") + .filter((e) => e.name.startsWith("synapse.")) + .map((e) => e.toJSON()), + gate: + document + .querySelector("[data-swarm-gate]") + ?.getAttribute("data-swarm-gate") ?? null, + concealed: document.querySelectorAll( + '[data-swarm-hidden="true"],[data-swarm-surface-hidden="true"]', + ).length, + overlays: document.querySelectorAll(".swarm-overlay").length, + controllers: Object.fromEntries( + Object.getOwnPropertySymbols(globalThis) + .filter((s) => + [ + "synapse.startup.controller", + "synapse.return.controller", + ].includes(s.description), + ) + .map((s) => [ + s.description, + Object.fromEntries( + Object.entries(globalThis[s]).filter(([k]) => + [ + "done", + "consumed", + "granted", + "nativeFocused", + "visible", + "suspended", + "readyAt", + "started", + "focused", + "seen", + "away", + "raf", + "timer", + "stable", + "deadline", + ].includes(k), + ), + ), + ]), + ), + }; + }); +} diff --git a/scripts/native-startup.mjs b/scripts/native-startup.mjs new file mode 100644 index 0000000..6c05e11 --- /dev/null +++ b/scripts/native-startup.mjs @@ -0,0 +1,134 @@ +// Actual installed launch; native foreground is supplied by Windows/user input. +import { chromium } from "@playwright/test"; +import { spawn } from "node:child_process"; +import { mkdir, writeFile, access, readFile } from "node:fs/promises"; +import path from "node:path"; +import assert from "node:assert/strict"; +import { startProbe, stopProbe } from "./native-probe.mjs"; +const mode = process.argv[2] ?? "tray", + name = process.argv[3] ?? mode; +const source = JSON.parse(await readFile("package.json", "utf8")); +const base = path.resolve( + process.env.SYNAPSE_NATIVE_EVIDENCE ?? + `.local/native-acceptance-${source.version}`, + ), + out = path.join(base, name); +await mkdir(out, { recursive: true }); +const executable = path.join( + process.env.LOCALAPPDATA, + "Synapse", + "synapse.exe", +); +const child = spawn(executable, mode === "visible" ? [] : ["--background"], { + env: { + ...process.env, + WEBVIEW2_ADDITIONAL_BROWSER_ARGUMENTS: "--remote-debugging-port=9222", + }, + detached: true, + stdio: "ignore", +}); +child.unref(); +for (let i = 0; i < 300; i++) { + try { + if ((await fetch("http://127.0.0.1:9222/json/version")).ok) break; + } catch { + /* The new WebView has not opened its diagnostics port yet. */ + } + await new Promise((r) => setTimeout(r, 25)); +} +const browser = await chromium.connectOverCDP("http://127.0.0.1:9222", { + noDefaults: true, + }), + context = browser.contexts()[0], + page = context.pages()[0]; +await page.waitForFunction(() => !!window.__TAURI_INTERNALS__?.invoke); +const invoke = (command, args = {}) => + page.evaluate( + ({ command, args }) => window.__TAURI_INTERNALS__.invoke(command, args), + { command, args }, + ); +const version = await invoke("plugin:app|version"); +assert.equal(page.url(), "http://tauri.localhost/"); +assert.equal( + version, + source.version, + "Installed Synapse version must match the source before recording", +); +try { + await access(path.join(base, "original-preferences.json")); +} catch { + await writeFile( + path.join(base, "original-preferences.json"), + JSON.stringify( + await page.evaluate(() => localStorage.getItem("synapse.preferences.v1")), + ), + ); +} +await startProbe(page); +const cdp = await context.newCDPSession(page), + frames = [], + writes = []; +cdp.on("Page.screencastFrame", (e) => { + void cdp.send("Page.screencastFrameAck", { sessionId: e.sessionId }); + const file = path.join(out, `${String(frames.length).padStart(5, "0")}.jpg`); + frames.push({ file, time: e.metadata.timestamp }); + writes.push(writeFile(file, Buffer.from(e.data, "base64"))); +}); +await cdp.send("Page.startScreencast", { + format: "jpeg", + quality: 95, + maxWidth: 1920, + maxHeight: 1200, + everyNthFrame: 1, +}); +console.log( + JSON.stringify({ + armed: true, + mode, + name, + version, + pid: child.pid, + visible: await invoke("plugin:window|is_visible", { label: "main" }), + focused: await invoke("plugin:window|is_focused", { label: "main" }), + }), +); +await page.waitForFunction( + () => + performance + .getEntriesByType("mark") + .some((e) => e.name.startsWith("synapse.startup:finish:")), + undefined, + { timeout: 600000 }, +); +await page.waitForTimeout(1100); +await cdp.send("Page.stopScreencast"); +await Promise.all(writes); +const probe = await stopProbe(page); +const report = { + native: true, + mode, + name, + version, + pid: child.pid, + captureFrames: frames, + ...probe, +}; +await writeFile(path.join(out, "report.json"), JSON.stringify(report, null, 2)); +console.log( + JSON.stringify( + { + name, + marks: probe.marks, + measures: probe.measures, + native: probe.native, + gate: probe.gate, + hidden: probe.concealed, + overlays: probe.overlays, + audio: probe.audio, + frames: frames.length, + }, + null, + 2, + ), +); +await browser.close(); diff --git a/scripts/record-return.mjs b/scripts/record-return.mjs new file mode 100644 index 0000000..b92dc24 --- /dev/null +++ b/scripts/record-return.mjs @@ -0,0 +1,174 @@ +// Observe one genuine native focus cycle. This script never synthesizes focus +// events; use Alt-Tab (human or Windows input automation) after it is armed. +import { chromium } from "@playwright/test"; +import { mkdir, writeFile, readFile } from "node:fs/promises"; +import { createHash } from "node:crypto"; +import path from "node:path"; +import assert from "node:assert/strict"; +const { version } = JSON.parse(await readFile("package.json", "utf8")); +const out = path.resolve(`.local/release-${version}/return`); +await mkdir(out, { recursive: true }); +const browser = await chromium.connectOverCDP("http://127.0.0.1:9222", { + noDefaults: true, + }), + context = browser.contexts()[0], + page = context.pages()[0]; +const invoke = (command, args = {}) => + page.evaluate( + ({ command, args }) => window.__TAURI_INTERNALS__.invoke(command, args), + { command, args }, + ); +assert.equal(await invoke("plugin:app|version"), version); +assert.equal(page.url(), "http://tauri.localhost/"); +await page.waitForFunction( + () => document.hasFocus() && !document.hidden, + undefined, + { timeout: 600000 }, +); +assert.equal(await invoke("plugin:window|is_focused", { label: "main" }), true); +await page.waitForTimeout(1600); +const state = () => + page.evaluate(() => { + const selection = window.getSelection(); + const nodePath = (node) => { + const path = []; + while (node?.parentNode && node !== document.body) { + path.unshift(Array.from(node.parentNode.childNodes).indexOf(node)); + node = node.parentNode; + } + return path; + }; + return { + heading: document.querySelector(".graph-heading h1")?.textContent, + editor: document.querySelector(".cm-content")?.textContent, + scroll: Array.from( + document.querySelectorAll(".cm-scroller,.hierarchy"), + ).map((e) => ({ top: e.scrollTop, left: e.scrollLeft })), + selection: selection + ? { + text: selection.toString(), + anchor: nodePath(selection.anchorNode), + anchorOffset: selection.anchorOffset, + focus: nodePath(selection.focusNode), + focusOffset: selection.focusOffset, + } + : null, + focusedElement: nodePath(document.activeElement), + preferences: localStorage.getItem("synapse.preferences.v1"), + }; + }); +const before = await state(); +const canvas = () => + page + .locator(".force-graph-container canvas") + .first() + .evaluate((c) => c.toDataURL()); +const beforeCanvas = await canvas(), + baseline = await page.evaluate( + () => performance.getEntriesByName("synapse.return").length, + ); +await page.evaluate(() => { + window.returnAudioContexts = 0; + const Original = window.AudioContext; + window.returnOriginalAudio = Original; + window.AudioContext = new Proxy(Original, { + construct(target, args, receiver) { + window.returnAudioContexts++; + return Reflect.construct(target, args, receiver); + }, + }); + window.returnFocusLog = []; + window.returnFocusListener = () => + window.returnFocusLog.push({ + at: performance.now(), + focused: document.hasFocus(), + hidden: document.hidden, + }); + window.addEventListener("focus", window.returnFocusListener); + window.addEventListener("blur", window.returnFocusListener); +}); +const cdp = await context.newCDPSession(page), + frames = [], + writes = []; +cdp.on("Page.screencastFrame", (e) => { + void cdp.send("Page.screencastFrameAck", { sessionId: e.sessionId }); + const file = path.join(out, `${String(frames.length).padStart(5, "0")}.jpg`); + frames.push({ file, time: e.metadata.timestamp }); + writes.push(writeFile(file, Buffer.from(e.data, "base64"))); +}); +await cdp.send("Page.startScreencast", { + format: "jpeg", + quality: 95, + maxWidth: 1800, + maxHeight: 1200, + everyNthFrame: 1, +}); +console.log( + "Return recording armed. Alt-Tab away for at least one second, then back.", +); +await page.waitForFunction( + (n) => performance.getEntriesByName("synapse.return").length > n, + baseline, + { timeout: 600000 }, +); +await page.waitForTimeout(900); +await cdp.send("Page.stopScreencast"); +await Promise.all(writes); +const result = await page.evaluate(() => ({ + timeOrigin: performance.timeOrigin, + marks: performance + .getEntriesByType("mark") + .filter((e) => e.name.startsWith("synapse.return")) + .map((e) => ({ name: e.name, time: e.startTime, detail: e.detail })), + measures: performance + .getEntriesByName("synapse.return") + .map((e) => e.toJSON()), + audioContexts: window.returnAudioContexts, + focus: window.returnFocusLog, + overlays: document.querySelectorAll(".swarm-overlay").length, + hidden: document.querySelectorAll("[data-swarm-hidden]").length, +})); +const after = await state(), + afterCanvas = await canvas(); +await page.evaluate(() => { + window.AudioContext = window.returnOriginalAudio; + window.removeEventListener("focus", window.returnFocusListener); + window.removeEventListener("blur", window.returnFocusListener); +}); +const measure = result.measures.at(-1); +const report = { + ...result, + native: true, + version, + frames, + before, + after, + canvasBefore: createHash("sha256").update(beforeCanvas).digest("hex"), + canvasAfter: createHash("sha256").update(afterCanvas).digest("hex"), + nativeFocused: await invoke("plugin:window|is_focused", { label: "main" }), +}; +await writeFile(path.join(out, "report.json"), JSON.stringify(report, null, 2)); +assert.equal(result.measures.length, baseline + 1); +assert.ok(measure.duration >= 250 && measure.duration <= 400); +assert.equal(result.overlays + result.hidden + result.audioContexts, 0); +assert.deepEqual(after, before); +assert.equal(afterCanvas, beforeCanvas); +assert.equal(report.nativeFocused, true); +assert.ok( + result.focus.some((e) => !e.focused) && result.focus.some((e) => e.focused), + "Genuine window focus cycle", +); +console.log( + JSON.stringify( + { + native: true, + duration: measure.duration, + frames: frames.length, + unchanged: true, + silent: true, + }, + null, + 2, + ), +); +await browser.close(); diff --git a/scripts/record-startup.mjs b/scripts/record-startup.mjs new file mode 100644 index 0000000..82e6c4f --- /dev/null +++ b/scripts/record-startup.mjs @@ -0,0 +1,216 @@ +// Capture the installed app's first foreground presentation from a new native +// process started to the tray. No replacement UI or frontend reload/replay. +// Requires a disposable vault and temporarily enabled background mode. The +// caller backs up/restores owner settings; this script never calls a provider. +import { chromium } from "@playwright/test"; +import { spawn } from "node:child_process"; +import { mkdir, writeFile, readFile } from "node:fs/promises"; +import path from "node:path"; +import assert from "node:assert/strict"; +const name = process.argv[2] ?? "restored"; +const existing = process.argv.includes("--existing"); +const visible = process.argv.includes("--visible"); +const waitShow = process.argv.includes("--wait-show"); +const executable = path.join( + process.env.LOCALAPPDATA, + "Synapse", + "synapse.exe", +); +const { version } = JSON.parse(await readFile("package.json", "utf8")); +const out = path.resolve(`.local/release-${version}/startup-${name}`); +await mkdir(out, { recursive: true }); +if (!existing) + spawn(executable, visible ? [] : ["--background"], { + env: { + ...process.env, + WEBVIEW2_ADDITIONAL_BROWSER_ARGUMENTS: "--remote-debugging-port=9222", + }, + detached: true, + stdio: "ignore", + }).unref(); +let ready = false; +for (let i = 0; i < 120; i++) { + try { + ready = (await fetch("http://127.0.0.1:9222/json/version")).ok; + } catch { + /* WebView is starting. */ + } + if (ready) break; + await new Promise((resolve) => setTimeout(resolve, 100)); +} +assert.ok(ready, "Installed WebView debug connection"); +// Default CDP attachment emulates focus and invalidates native acceptance. +const browser = await chromium.connectOverCDP("http://127.0.0.1:9222", { + noDefaults: true, +}); +const context = browser.contexts()[0], + page = context.pages()[0]; +await page.waitForFunction(() => !!window.__TAURI_INTERNALS__?.invoke); +const invoke = (command, args = {}) => + page.evaluate( + ({ command, args }) => window.__TAURI_INTERNALS__.invoke(command, args), + { command, args }, + ); +await page.waitForSelector(name === "welcome" ? ".welcome" : ".sidebar", { + state: "attached", +}); +assert.equal(page.url(), "http://tauri.localhost/"); +assert.equal(await invoke("plugin:app|version"), version); +if (!existing && !visible) + assert.equal( + await invoke("plugin:window|is_visible", { label: "main" }), + false, + ); +if (!visible) assert.equal(await page.locator(".swarm-overlay").count(), 0); +const prefs = await page.evaluate(() => + localStorage.getItem("synapse.preferences.v1"), +); +await page.evaluate(() => { + window.startupAudioContexts = 0; + if (window.originalAudioContext) + window.AudioContext = window.originalAudioContext; + window.originalAudioContext = window.AudioContext; + window.AudioContext = new Proxy(window.AudioContext, { + construct(target, args, receiver) { + window.startupAudioContexts++; + return Reflect.construct(target, args, receiver); + }, + }); +}); +const cdp = await context.newCDPSession(page), + frames = [], + writes = []; +cdp.on("Page.screencastFrame", (e) => { + void cdp.send("Page.screencastFrameAck", { sessionId: e.sessionId }); + const file = path.join(out, `${String(frames.length).padStart(5, "0")}.jpg`); + frames.push({ file, time: e.metadata.timestamp }); + writes.push(writeFile(file, Buffer.from(e.data, "base64"))); +}); +await cdp.send("Page.startScreencast", { + format: "jpeg", + quality: 95, + maxWidth: 1800, + maxHeight: 1200, + everyNthFrame: 1, +}); +const requested = Date.now() / 1000; +// The existing single-instance handler shows/focuses the hidden native window. +if (!existing && !visible && !waitShow) + spawn(executable, [], { detached: true, stdio: "ignore" }).unref(); +console.log("Recording armed; waiting for genuine native foreground."); +await page.waitForFunction( + () => + performance + .getEntriesByType("mark") + .some((e) => e.name.startsWith("synapse.startup:finish:")), + undefined, + { timeout: 600000 }, +); +await page.waitForTimeout(1200); +const result = await page.evaluate(() => ({ + timeOrigin: performance.timeOrigin, + at: performance.now(), + marks: performance + .getEntriesByType("mark") + .filter((e) => e.name.startsWith("synapse.startup")) + .map((e) => ({ name: e.name, time: e.startTime, detail: e.detail })), + measures: performance + .getEntriesByName("synapse.startup") + .map((e) => e.toJSON()), + overlay: document.querySelectorAll(".swarm-overlay").length, + hidden: document.querySelectorAll('[data-swarm-hidden="true"]').length, + audioContexts: window.startupAudioContexts, + heading: document.querySelector(".graph-heading h1, .welcome h1") + ?.textContent, + width: innerWidth, + height: innerHeight, + preferences: localStorage.getItem("synapse.preferences.v1"), +})); +await cdp.send("Page.stopScreencast"); +await Promise.all(writes); +await page.evaluate(() => { + window.AudioContext = window.originalAudioContext; +}); +const report = { + name, + requested, + ...result, + frames, + version: await invoke("plugin:app|version"), + nativeFocus: await invoke("plugin:window|is_focused", { label: "main" }), + replayClaim: await invoke("claim_startup"), +}; +await writeFile(path.join(out, "report.json"), JSON.stringify(report, null, 2)); +assert.ok( + result.marks.at(-1).name.endsWith("complete"), + JSON.stringify(result.marks), +); +assert.ok( + result.measures.at(-1).duration >= (name === "welcome" ? 500 : 4400) && + result.measures.at(-1).duration <= 4900, +); +assert.equal(result.overlay, 0); +assert.equal(result.hidden, 0); +assert.equal(result.audioContexts, 0); +assert.equal(result.preferences, prefs); +assert.equal(report.replayClaim.granted, false); +assert.ok(frames.length > 10); +const foreground = + (result.timeOrigin + result.measures.at(-1).startTime) / 1000; +const end = (result.timeOrigin + result.marks.at(-1).time) / 1000 + 0.7; +const start = foreground - 0.08; +const ordered = [...frames] + .sort((a, b) => a.time - b.time) + .filter((f, i, all) => !i || f.time !== all[i - 1].time); +const prior = ordered.filter((f) => f.time <= start).at(-1); +const timeline = [ + ...(prior ? [{ ...prior, time: start }] : []), + ...ordered.filter((f) => f.time > start && f.time < end), +]; +const concat = ["ffconcat version 1.0"]; +timeline.forEach((f, i) => { + concat.push( + `file '${f.file.replaceAll("\\", "/")}'`, + "option framerate 1000", + `duration ${Math.max(0.001, (timeline[i + 1]?.time ?? end) - f.time).toFixed(6)}`, + ); +}); +concat.push( + `file '${timeline.at(-1).file.replaceAll("\\", "/")}'`, + "option framerate 1000", +); +await writeFile(path.join(out, "frames.ffconcat"), concat.join("\n") + "\n"); +await page.reload(); +await page.waitForSelector(name === "welcome" ? ".welcome" : ".sidebar"); +await page.waitForTimeout(1300); +assert.equal(await page.locator(".swarm-overlay").count(), 0); +const replay = await page.evaluate(() => + performance + .getEntriesByType("mark") + .filter((e) => e.name.startsWith("synapse.startup")) + .map((e) => e.name), +); +assert.ok( + replay.some((n) => n.endsWith("already-presented")), + JSON.stringify(replay), +); +await writeFile( + path.join(out, "reload.json"), + JSON.stringify({ replay, cleaned: true }, null, 2), +); +console.log( + JSON.stringify( + { + name, + frames: frames.length, + duration: result.measures.at(-1).duration, + presentation: result.marks.find((m) => m.name.endsWith(":start"))?.detail, + silent: result.audioContexts === 0, + reloadedWithoutReplay: true, + out, + }, + null, + 2, + ), +); +await browser.close(); diff --git a/scripts/record-swarm-preview.mjs b/scripts/record-swarm-preview.mjs new file mode 100644 index 0000000..0f27ef1 --- /dev/null +++ b/scripts/record-swarm-preview.mjs @@ -0,0 +1,183 @@ +// Browser fixture review only. These clips are NOT native launch/Alt-Tab evidence. +import { chromium } from "@playwright/test"; +import { createServer } from "vite"; +import { mkdir, writeFile, readFile } from "node:fs/promises"; +import path from "node:path"; +import assert from "node:assert/strict"; +const { version } = JSON.parse(await readFile("package.json", "utf8")); +const cases = process.argv[2] ? [process.argv[2]] : ["startup", "return"]; +assert.ok(cases.every((name) => name === "startup" || name === "return")); +const server = await createServer({ + server: { host: "127.0.0.1", port: 1432, strictPort: true }, +}); +await server.listen(); +const browser = await chromium.launch({ channel: "msedge", headless: true }); +try { + for (const name of cases) { + const out = path.resolve(`.local/release-${version}/browser-${name}`); + await mkdir(out, { recursive: true }); + const context = await browser.newContext({ + viewport: { width: 1440, height: 900 }, + reducedMotion: "no-preference", + }); + const page = await context.newPage(); + await page.route("**/swarm-preview", (r) => + r.fulfill({ + contentType: "text/html", + body: ``, + }), + ); + await page.goto("http://127.0.0.1:1432/swarm-preview"); + await page.evaluate(async (name) => { + const { StartupController } = await import("/src/startup/controller.ts"); + const { ReturnController } = await import("/src/startup/return.ts"); + const { mountWorkspace } = await import("/scripts/startup-fixture.tsx"); + await Promise.all([ + document.fonts.load("400 24px Geist"), + document.fonts.load("650 18px Manrope"), + ]); + const startup = new StartupController(async () => ({ + id: `preview-${name}`, + foreground: true, + visible: true, + granted: true, + })); + const fixture = mountWorkspace(startup, { + ready: false, + reduced: false, + enabled: name === "startup", + blocked: false, + }); + window.preview = { startup, fixture, ReturnController }; + }, name); + await page.waitForSelector(".startup-restoring"); + if (name === "return") { + await page.evaluate(() => window.preview.fixture.ready()); + await page.waitForSelector(".graph-controls"); + await page.waitForTimeout(500); + await page.evaluate(() => { + const q = window.preview; + q.returns = new q.ReturnController(() => true, true); + q.returns.attach(document.querySelector("main")); + q.returns.update({ + ready: true, + reduced: false, + enabled: true, + blocked: false, + }); + q.returns.foreground(true); + q.returns.foreground(false); + }); + await page.waitForTimeout(700); + } + const cdp = await context.newCDPSession(page), + frames = [], + writes = []; + cdp.on("Page.screencastFrame", (e) => { + void cdp.send("Page.screencastFrameAck", { sessionId: e.sessionId }); + const file = path.join( + out, + `${String(frames.length).padStart(5, "0")}.jpg`, + ); + frames.push({ file, time: e.metadata.timestamp }); + writes.push(writeFile(file, Buffer.from(e.data, "base64"))); + }); + await cdp.send("Page.startScreencast", { + format: "jpeg", + quality: 95, + everyNthFrame: 1, + }); + await page.waitForTimeout(100); + await page.evaluate( + (name) => + name === "startup" + ? window.preview.fixture.ready() + : window.preview.returns.foreground(true), + name, + ); + await page.waitForFunction( + (name) => performance.getEntriesByName(`synapse.${name}`).length > 0, + name, + { timeout: 8000 }, + ); + await page.waitForTimeout(900); + await cdp.send("Page.stopScreencast"); + await Promise.all(writes); + const result = await page.evaluate( + (name) => ({ + timeOrigin: performance.timeOrigin, + measure: performance + .getEntriesByName(`synapse.${name}`) + .at(-1) + .toJSON(), + marks: performance + .getEntriesByType("mark") + .filter((e) => e.name.startsWith(`synapse.${name}`)) + .map((e) => ({ name: e.name, time: e.startTime, detail: e.detail })), + }), + name, + ); + assert.ok( + result.marks.at(-1).name.endsWith("complete"), + JSON.stringify(result), + ); + const start = (result.timeOrigin + result.measure.startTime) / 1000 - 0.08, + end = + (result.timeOrigin + + result.measure.startTime + + result.measure.duration) / + 1000 + + 0.75; + const ordered = [...frames] + .sort((a, b) => a.time - b.time) + .filter((f, i, a) => !i || f.time !== a[i - 1].time); + const prior = ordered.filter((f) => f.time <= start).at(-1); + const selected = [ + ...(prior ? [{ ...prior, time: start }] : []), + ...ordered.filter((f) => f.time > start && f.time < end), + ]; + const lines = ["ffconcat version 1.0"]; + selected.forEach((f, i) => + lines.push( + `file '${f.file.replaceAll("\\", "/")}'`, + "option framerate 1000", + `duration ${((selected[i + 1]?.time ?? end) - f.time).toFixed(6)}`, + ), + ); + lines.push( + `file '${selected.at(-1).file.replaceAll("\\", "/")}'`, + "option framerate 1000", + ); + await writeFile(path.join(out, "clip.ffconcat"), lines.join("\n") + "\n"); + await writeFile( + path.join(out, "report.json"), + JSON.stringify( + { + native: false, + fixture: true, + trigger: + name === "return" + ? "Simulated focus lifecycle; not Alt-Tab" + : "Browser fixture becoming ready; not a native launch", + ...result, + frames, + clip: { start, end, duration: end - start, frames: selected.length }, + }, + null, + 2, + ), + ); + console.log( + JSON.stringify({ + name, + native: false, + duration: result.measure.duration, + frames: selected.length, + }), + ); + await context.close(); + } +} finally { + await browser.close(); + await server.close(); +} diff --git a/scripts/startup-fixture.tsx b/scripts/startup-fixture.tsx new file mode 100644 index 0000000..f884ebb --- /dev/null +++ b/scripts/startup-fixture.tsx @@ -0,0 +1,235 @@ +// Browser-only lifecycle fixture, never imported by the production app. +import { StrictMode, useLayoutEffect, useRef, useState } from "react"; +import { createRoot } from "react-dom/client"; +import type { + StartupController, + StartupState, +} from "../src/startup/controller"; +import Graph from "../src/components/Graph"; +import { Exploration } from "../src/interaction/exploration"; +import { subjects, type Snapshot } from "../src/types"; +import "../src/styles.css"; +import "../src/startup/startup.css"; + +export function mountWorkspace( + controller: StartupController, + state: StartupState, + options: { + strict?: boolean; + collapsed?: boolean; + empty?: boolean; + welcome?: boolean; + drawer?: boolean; + dense?: boolean; + } = {}, +) { + const host = document.createElement("div"), + root = createRoot(host); + document.body.append(host); + const organization: Snapshot["organization"] = { + schema_version: 1, + neurons: [{ id: "study", name: "Study workspace", path: "study" }], + subjects: subjects.map((s) => ({ + id: s.id, + name: s.name, + color: s.color, + preset: s.id, + neuron_id: "study", + root: null, + legacy: false, + })), + location: { neuron_id: null, subject_id: options.dense ? "physics" : null }, + fallback_subject: "general", + }; + const notes = Array.from({ length: options.dense ? 300 : 15 }, (_, i) => ({ + id: `note-${i}`, + title: `Study note ${i}`, + subject: options.dense ? "physics" : subjects[i % 5].id, + subject_id: options.dense ? "physics" : subjects[i % 5].id, + neuron_id: "study", + revision: "fixture", + neighbors: 4, + })); + const snapshot: Snapshot = { + root: "fixture", + name: "Synapse demo", + organization, + notes: options.empty ? [] : notes, + edges: options.empty + ? [] + : notes + .slice(0, -1) + .map((n, i) => ({ source: n.id, target: notes[i + 1].id })), + links: [], + cards: [], + history: { schema_version: 1, cards: {} }, + warnings: [], + review_error: null, + }; + if (options.empty) snapshot.organization.subjects = []; + const exploration = new Exploration(); + exploration.sync(snapshot); + let makeReady = () => {}; + function Workspace() { + const main = useRef(null), + drawer = useRef(null); + const [ready, setReady] = useState(state.ready); + useLayoutEffect(() => { + makeReady = () => setReady(true); + return controller.attach(main.current!); + }, []); + useLayoutEffect(() => controller.update({ ...state, ready }), [ready]); + return ( +
+ {!ready ? ( +
+ Restoring your workspace… +
+ ) : options.welcome ? ( +
+

Synapse 👩🏽‍💻

+ +

Choose a folder of Markdown notes

+ +
+ ) : ( + <> + + {}} + query="" + left={options.collapsed ? 80 : 312} + drawer={drawer} + editorWidth={430} + auxiliaryOpen={options.drawer} + onOpen={() => {}} + onNewNote={() => {}} + /> + {options.drawer && ( +
+
+
+ Study note +
+