ci(THU-884): collect Nightly DB diagnostics with alerts paused by ital0 · Pull Request #1303 · thunderbird/thunderbolt

ital0 · 2026-09-25T13:11:09Z

Summary

Keep the full scheduled Linux and macOS Nightly suites, existing projects, retries, and failure reports. Capture fixed database readiness and first-query labels in a small JSON file for each WebKit test attempt, including passing tests and retries.
Pause only the Nightly Linear/email/recovery notifier during the investigation. Its BetterStack heartbeat still runs after both test jobs complete, whether they pass or fail.
Remove the superseded one-case diagnostic job and config. The ordinary PR E2E workflow and other alert workflows are unchanged.

Reason

The original WebKit first-query rejection has not been reproduced. The installed PowerSync query path already awaits readiness, so this PR collects evidence without a speculative database fix. Uploaded diagnostic JSON contains only allowlisted outcomes, categories, error names, test status, and retry number; it contains no raw error text, stack, URL, or credentials.

Validation

bun run test passed; the second phase included the two new parser tests (318 pass, 0 fail).
bun run type-check, bun run e2e:check-collected (22 specs), actionlint, Prettier, the global Oxlint hook, and git diff --check passed.
A local WebKit request-only case passed and retained explicit missing labels. A local WebKit UI case produced readiness/query ready labels on failure and retry; it timed out later at the sign-in Continue click. No full CI Nightly run was dispatched from this PR.

Follow-up

Merge this PR to affect scheduled runs. After reviewing the diagnostic artifacts, restore the Nightly notifier and its heartbeat dependency manually, and remove the temporary instrumentation. There is no automatic cutoff in code.

- keep one Linear incident per workflow and report failed jobs - email first failures and recoveries from trusted main runs

- run Linux and macOS browser projects against isolated test services - keep Safari OPFS available with per-test persistent profiles - sanitize failure reports before artifact upload

- isolate one credential-free diagnostic from Nightly alerts - record safe readiness and first-query categories for the runner failure

- provide the gitignored browser worker asset before the isolated startup case

- keep the one startup case while starting the original server set

github-actions · 2026-09-25T13:12:26Z

Semgrep Security Scan

No security issues found.

github-actions · 2026-09-25T13:17:03Z

Preview environment destroyed 🧹

Stack preview-pr-1303 and its Cloudflare subdomain have been cleaned up.

github-actions · 2026-09-25T13:17:46Z

PR Metrics

Metric	Value
Lines changed (prod code)	`+159 / -56`
JS bundle size (gzipped)	🟢 645.3 KB → 645.1 KB (-143 B, -0.0%)
Test coverage	🟢 82.33% → 82.35% (+0.0%)
Performance (preview)	Preview not ready — Render deploy may have timed out
Accessibility	—
Best Practices	—
SEO	—

Updated Fri, 25 Sep 2026 15:27:19 GMT · run #3129

greptile-apps · 2026-09-25T13:42:34Z

Confidence Score: 5/5

[Medium risk] Adds database diagnostics collection to nightly test runs.

The PR appears safe to merge, though the testing guide should be restored before relying on it for reproduction or incident recovery.

Findings

Linux Nightly setup is incomplete ▶
Active alert runbook removed ▶

Summary

The PR retains scheduled cross-browser coverage while adding allowlisted database diagnostics for WebKit attempts and temporarily pausing the Nightly notifier. It also relocates the parser tests into e2e/ and shortens the testing guide. The guide should retain reproducible Linux setup and operational alert-recovery guidance.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  N[Nightly run] --> L[Linux Chromium and Firefox]
  N --> W[macOS WebKit]
  W --> D[Per-attempt database diagnostic JSON]
  L --> H[Heartbeat after both jobs]
  W --> H

_{Reviews (6) · Last reviewed commit: "docs: keep nightly testing guide focused..."}

github-actions

🔭 thunder-deep-review (advisory)

Complements the other bots — surfaces only what they did not flag. Never approves, never requests changes, never gates merge.
_{head: c2383316b558 · mode: single · deferred 0 item(s) already reported by other bots (best-effort dedup)}

- Keep the test matrix accurate as the suite grows - Align local startup checks with CI readiness

- Keep diagnostic branching out of the initialization sequence - Preserve finite labels and background readiness observation

github-actions

🔭 thunder-deep-review (advisory)

Reviewed the diff — no issues to report. ✅ Never approves, never requests changes, never gates merge.
_{head: c2a9858f1019 · mode: single · deferred 0 item(s) already reported by other bots (best-effort dedup)}

github-actions

🔭 thunder-deep-review (advisory)

Complements the other bots — surfaces only what they did not flag. Never approves, never requests changes, never gates merge.
_{head: 7cd119b6b37c · mode: single · deferred 0 item(s) already reported by other bots (best-effort dedup)}

Additional notes (couldn't anchor to a diff line)

.github/workflows/nightly.yml:191 — 🚫 Blocking — Dropping the notify job removes all failure alerting from Nightly
vite.config.ts:191 — 📐 Convention — forwardConsole doesn't appear to be a real Vite server option
e2e/db-diagnostic.ts — 📐 Convention — Label grammar is defined twice with nothing keeping the two copies in sync

…resh Main's THU-884 nightly fixes (#1302, #1303) rewrote and deleted parts of the testing doc our branch had moved to docs/internals/. Mirrored main's version at the new path rather than preserving the older port: - keep the deletion of the old docs/development/testing.md path - take main's note on the separate `bun test --cwd=e2e` step, its shorter Nightly E2E section, and its nightly-WebKit OPFS wording - drop the "CI failure and recovery alerts" section, which main removed now that alerts are paused - name e2e/db-diagnostic.test.ts in AGENTS.md's test-scope sentence, which main's new e2e step made stale

ital0 added 18 commits September 23, 2026 21:32

ci(THU-884): notify on workflow failure and recovery

83190d3

- keep one Linear incident per workflow and report failed jobs - email first failures and recoveries from trusted main runs

ci(THU-884): add nightly cross-browser E2E coverage

986ff35

- run Linux and macOS browser projects against isolated test services - keep Safari OPFS available with per-test persistent profiles - sanitize failure reports before artifact upload

chore(THU-884): match Resend URL exactly in notifier tests

4d8f3f7

ci(THU-884): gate notifier tests and narrow recovery conditions

ad09ab4

ci(THU-884): protect alert state and reject stale recovery

feb0bf2

ci(THU-884): report safe notifier errors and cover edge cases

04a4a2c

fix(THU-884): sign fixed mock OIDC identity claims

f60a828

ci(THU-884): scope nightly secrets and gate heartbeat on alerts

a275765

ci(THU-884): migrate nightly artifact sanitizer to Bun

e34a12f

style(THU-884): use const for signed incident state

02e8c1f

chore(THU-884): sync merged nightly base

144e26d

fix(THU-884): repair first Nightly failures

4f03d6b

chore(THU-884): retain rejection stack in artifact errors

ccb5c65

ci(THU-884): probe WebKit database startup on macOS

e29acb5

- isolate one credential-free diagnostic from Nightly alerts - record safe readiness and first-query categories for the runner failure

ci(THU-884): prepare PowerSync worker for diagnostic

655a288

- provide the gitignored browser worker asset before the isolated startup case

ci(THU-884): match Nightly server topology in diagnostic

000f176

- keep the one startup case while starting the original server set

ci(THU-884): collect Nightly database outcomes without alerts

25bc7ca

chore(THU-884): merge current main for diagnostic PR

efb6f7d

ital0 deployed to preview September 25, 2026 13:15 — with GitHub Actions Active

ital0 marked this pull request as ready for review September 25, 2026 13:29

ital0 added 2 commits September 25, 2026 13:33

chore(THU-884): remove stale Nightly notifier comment

0fe1704

docs(THU-884): remove stale Nightly investigation note

3f7bdab

ital0 deployed to preview September 25, 2026 13:42 — with GitHub Actions Active

greptile-apps Bot reviewed Sep 25, 2026

View reviewed changes

Comment thread src/hooks/use-app-initialization.ts Outdated

Comment thread scripts/db-diagnostic.test.ts Outdated

test: strengthen database diagnostics and remove ticket references

c238331

ital0 deployed to preview September 25, 2026 13:49 — with GitHub Actions Active

github-actions Bot reviewed Sep 25, 2026

View reviewed changes

Comment thread .github/workflows/nightly.yml

ital0 added 2 commits September 25, 2026 14:25

docs: clarify Nightly local readiness and alert behavior

f44591b

- Keep the test matrix accurate as the suite grows - Align local startup checks with CI readiness

refactor: isolate database startup diagnostics

c2a9858

- Keep diagnostic branching out of the initialization sequence - Preserve finite labels and background readiness observation

github-code-quality Bot found potential problems Sep 25, 2026

View reviewed changes

Comment thread src/hooks/db-diagnostic.test.ts Fixed

ital0 deployed to preview September 25, 2026 14:30 — with GitHub Actions Active

github-actions Bot reviewed Sep 25, 2026

View reviewed changes

chore: align database diagnostic observer and test placement

b1df8a6

github-code-quality Bot found potential problems Sep 25, 2026

View reviewed changes

Comment thread src/hooks/db-diagnostic.test.ts Fixed

ital0 deployed to preview September 25, 2026 14:58 — with GitHub Actions Active

chore: drop redundant database readiness return assertions

8975543

ital0 deployed to preview September 25, 2026 15:13 — with GitHub Actions Active

docs: keep nightly testing guide focused on reproduction

7cd119b

ital0 deployed to preview September 25, 2026 15:23 — with GitHub Actions Active

greptile-apps Bot reviewed Sep 25, 2026

View reviewed changes

Comment thread docs/development/testing.md

Comment thread docs/development/testing.md

github-actions Bot reviewed Sep 25, 2026

View reviewed changes

ital0 merged commit eca5363 into main Sep 25, 2026
36 checks passed

ital0 deleted the italomenezes/thu-884-notify-on-failure-job-for-scheduled-workflows-and-the branch September 25, 2026 15:56

Navigation Menu

ci(THU-884): collect Nightly DB diagnostics with alerts paused - #1303

ci(THU-884): collect Nightly DB diagnostics with alerts paused#1303
ital0 merged 26 commits into
mainfrom
italomenezes/thu-884-notify-on-failure-job-for-scheduled-workflows-and-the

ital0 commented Sep 25, 2026

Uh oh!

github-actions Bot commented Sep 25, 2026

Uh oh!

github-actions Bot commented Sep 25, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Sep 25, 2026 •

edited

Loading

Uh oh!

greptile-apps Bot commented Sep 25, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

github-actions Bot left a comment

Uh oh!

Uh oh!

Uh oh!

github-actions Bot left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions Bot left a comment

Uh oh!

Uh oh!

This branch was successfully deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

ital0 commented Sep 25, 2026

Summary

Reason

Validation

Follow-up

Uh oh!

github-actions Bot commented Sep 25, 2026

Semgrep Security Scan

Uh oh!

github-actions Bot commented Sep 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Sep 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Metrics

Uh oh!

greptile-apps Bot commented Sep 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Confidence Score: 5/5

Findings

Summary

Uh oh!

Uh oh!

Uh oh!

github-actions Bot left a comment

Choose a reason for hiding this comment

🔭 thunder-deep-review (advisory)

Uh oh!

Uh oh!

Uh oh!

github-actions Bot left a comment

Choose a reason for hiding this comment

🔭 thunder-deep-review (advisory)

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions Bot left a comment

Choose a reason for hiding this comment

🔭 thunder-deep-review (advisory)

Additional notes (couldn't anchor to a diff line)

Uh oh!

Uh oh!

This branch was successfully deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

github-actions Bot commented Sep 25, 2026 •

edited

Loading

github-actions Bot commented Sep 25, 2026 •

edited

Loading

greptile-apps Bot commented Sep 25, 2026 •

edited

Loading