Record verified live setup deployment E2E results and remaining gates

Author: msukkari

packages/setupWizard/telemetry.html

@@ -1743,7 +1743,21 @@ <h3>Reo coexistence</h3>
 <p>The release notes and telemetry documentation should state that setup-wizard PostHog telemetry is always on when its code executes, list the categories collected, explain that PostHog creates a Person profile keyed only by a random pseudonymous session/deployment ID, explain the new-deployment ID handoff, and distinguish PostHog from Reo’s separate tracking and variable. They must not describe the project ingestion token as a secret.</p>
 <h2>Testing plan</h2>
 <h3>Implementation verification status</h3>
-<p><strong>Live deployment suite:</strong> <code>tests/e2e/liveDeployment.mjs</code> builds and installs the npm tarball, drives the real interactive CLI, and starts the released Sourcebot <code>v5.1.13</code> image with real PostgreSQL, Redis, migrations, backend, and web processes. It exercises public GitHub repository/organization/user scopes, GitLab projects, Gitea repositories, Gerrit projects, generic Git URLs, three local-clone layouts, automatic startup/interruption, and AI configuration. Each deployment uses an isolated Compose project and loopback port; only the test Compose networking/image settings and deployment telemetry opt-out are changed. Generated configuration, environment values, local mounts, and install-ID handoff remain intact. See <code>tests/e2e/liveDeployment.md</code> for reproduction and limits. A final clean rerun of all thirteen live scenarios is in progress; this does not replace the still-required full scenario/branch-coverage accounting.</p>
+<p><strong>Live deployment suite:</strong> <code>tests/e2e/liveDeployment.mjs</code> builds and installs the npm tarball, drives the real interactive CLI, and starts the released Sourcebot <code>v5.1.13</code> image with real PostgreSQL, Redis, migrations, backend, and web processes. It exercises public GitHub repository/organization/user scopes, GitLab projects, Gitea repositories, Gerrit projects, generic Git URLs, three local-clone layouts, automatic startup/interruption, and AI configuration. Each deployment uses an isolated Compose project and loopback port; only the test Compose networking/image settings and deployment telemetry opt-out are changed. Generated configuration, environment values, local mounts, and install-ID handoff remain intact. See <code>tests/e2e/liveDeployment.md</code> for reproduction and limits. The final clean rerun passed all thirteen scenarios against implementation/test commit <code>7f491c19</code> and package SHA-256 <code>c4c6157ceacf224e023bf8233dc79830b6683d05fcd614493071037f464eddab</code>. This does not replace the still-required full scenario/branch-coverage accounting.</p>
+<details>
+<summary>Live E2E evidence — thirteen scenarios passed; no Ask requests</summary>
+<ul>
+<li>GitHub: explicit repository, organization (all 16 repositories), and user (all 8 repositories). GitLab project, Gitea repository, Gerrit project, and generic remote Git each indexed successfully.</li>
+<li>Local repositories: a single root, two sibling clones represented by a wildcard, and two nested clones represented by separate connections. All expected repositories were indexed; distinct origins avoid Sourcebot's existing clone deduplication.</li>
+<li>Every scenario passed owner onboarding, authenticated search API and visible browser search results, HTTP readiness and working search after restart, and persistent install-ID equality with the wizard session.</li>
+<li>Automatic <code>docker compose up</code> launched by the CLI reached readiness; Ctrl+C exited with 130, retained exactly one completed event, and emitted no cancelled event after completion.</li>
+<li>All twelve AI providers were configured together using synthetic credentials, with exact generated/mounted model configuration and environment propagation verified. A separate Anthropic run verified the available development key without printing it. No Ask or inference calls were made.</li>
+<li>The real SDK's captured payloads passed the strict transport/schema/privacy contract, expected checkpoint order, source/provider counts, and identity assertions. This suite captures locally; the separate already-verified dev PostHog smoke remains ingestion/query evidence.</li>
+<li>All task-owned containers, volumes, networks, temporary installations, and credentials were removed. Redacted reports/screenshots are outside the repository. The task-downloaded Sourcebot image is retained for the separately requested Vertex-fix task to reuse and clean afterward.</li>
+</ul>
+<p>Harness corrections, not production regressions: wait for live autocomplete results; use distinct repository origins in multi-clone fixtures; assert the actual search summary instead of assuming a README result; and supply a credential-free Docker plugin/context configuration when isolating HOME. Cross-platform CI fixes also dispose Windows ConPTY resources, permit initial lockfile creation only in disposable Yarn test projects, and separate Node 24 CLI tests from the application's Node 20 job.</p>
+<p><strong>Remaining limits:</strong> no live authenticated Azure DevOps/Bitbucket or private/self-hosted-host access was available; fixture coverage is not equivalent to live integration verification. AI credential validity, inference, default credential discovery, and service-account-file access are not implied by startup. The pre-existing Google Vertex host-file/mount limitation is assigned to a separate user-requested task, which must validate its fix E2E and open a separate PR with a changelog edit. Full manifest/branch-coverage accounting and final CI evidence still gate overall feature completion; this PR remains draft.</p>
+</details>
 <p><strong>Live AI validation scope (user clarification):</strong> verify that the packaged wizard writes the selected provider/model configuration and credential references correctly, that the generated environment values reach the resulting Sourcebot container unchanged, and that Sourcebot starts successfully. Do not invoke Ask or make model-inference requests. A Pro license is not required for this test scope. Use available development credentials without printing them; where credentials are unavailable, distinguish configuration/startup coverage with synthetic values from actual credential validation. Continue real repository discovery, indexing, search, browser access, and restart tests independently.</p>
 <p>The feature is not release-complete until the full completion gate below is satisfied. Executable tests cover packed-package collectors, terminal paths, real SDK payloads, Docker branches, and actual container identity continuity. CI definitions are not evidence that Windows or other remote jobs have run.</p>
 <p>Implementation-discovered changes for the behavioral baseline allowlist: status 130 for Ctrl+C; cancellation-aware readiness cleanup; explicit final process exit after bounded telemetry shutdown; recoverable autocomplete fallback on network/decode failure with an 8-second timeout; and fixed, credential-free SDK shutdown-timeout diagnostics where PostHog itself emits them. Do not patch SDK internals or silence the global console. Empty search/catalog results are not failures; actual infrastructure errors can emit recoverable diagnostics while retaining manual fallback.</p>