feat(dev): speed up and add a live dashboard to local stack startup#20688
feat(dev): speed up and add a live dashboard to local stack startup#20688vbudhram wants to merge 1 commit into
Conversation
8008bd6 to
e552f0b
Compare
| persist-credentials: false | ||
| - name: Set up Node | ||
| # .nvmrc ships in the FxA repo, checked out into fxa-code/ just above | ||
| uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6 |
There was a problem hiding this comment.
This was failing for me in CI and reordering fixed it. Seems like we should use the nvmrc in FxA reitehr way.
| exit 1 | ||
| fi | ||
|
|
||
| # On an interactive TTY (the same gate the start dashboard uses), replace pm2's |
There was a problem hiding this comment.
I didn't have to add this, but watching all the pm2 and nx builds go by just seemed excessive. Now there is a simple live dashboard as the services come up. You can still see logs with FXA_START_PLAIN=1
There was a problem hiding this comment.
Pull request overview
This PR improves the local yarn start mza / yarn restart mza developer workflow by introducing a “dev-fast” Nx path (build-dev / start-dev / restart-dev) that avoids unnecessary full production library builds, and by adding a TTY-only live startup dashboard that summarizes per-service readiness while still capturing full raw output to artifacts/.
Changes:
- Add Nx
build-dev/start-dev/restart-devtarget defaults and wire per-packagebuild-dev/start-dev/restart-devscripts somzacan take a faster startup path. - Introduce a TTY dashboard (
_scripts/start-dashboard.js) forstart-devthat renders per-service status and writes full Nx output toartifacts/nx-start-dev.log. - Harden startup reliability (Docker preflight, labeled URL checks, Redis gate, dynamic db-patcher expected count) and adjust l10n GitHub Actions node setup to use the checked-out repo’s
.nvmrc.
Reviewed changes
Copilot reviewed 25 out of 25 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| packages/fxa-shared/scripts/build-ts-parallel.sh | Parallelizes ESM/CJS TypeScript builds for fxa-shared to reduce critical-path startup time. |
| packages/fxa-shared/package.json | Switches build-ts to the parallel script; adds build-dev/start-dev/restart-dev. |
| packages/fxa-settings/package.json | Adds build-dev; adds start-dev/restart-dev; improves readiness check labeling. |
| packages/fxa-react/package.json | Adds build-dev plus start-dev/restart-dev aliases. |
| packages/fxa-profile-server/package.json | Adds build-dev and start-dev/restart-dev; improves readiness check labeling. |
| packages/fxa-customs-server/package.json | Adds start-dev/restart-dev aliases. |
| packages/fxa-content-server/package.json | Adds build-dev, readiness labeling, and implicitDependencies for build-dev graph correctness. |
| packages/fxa-auth-server/package.json | Adds build-dev, start-dev/restart-dev, and Nx start-dev dependency tweaks (incl. gen-keys). |
| packages/fxa-auth-client/package.json | Adds build-dev script for dev-fast graph compatibility. |
| packages/123done/package.json | Adds start-dev/restart-dev aliases for dev-fast orchestration. |
| package-scripts.js | Routes start mza through _scripts/start-mza.sh and uses restart-dev for mza restarts. |
| nx.json | Adds build-dev, start-dev, and restart-dev target defaults/outputs for dev-fast execution. |
| libs/vendored/crypto-relier/project.json | Adds build-dev target so dist-loaded code is built in the dev-fast path. |
| libs/shared/l10n/project.json | Adds build-dev target for dev-fast coverage. |
| .github/workflows/l10n-gettext-extract.yml | Ensures setup-node reads .nvmrc from the checked-out FxA repo path. |
| _scripts/start-mza.sh | New wrapper for mza startup: preflight + infra + dev-fast services + quiet sync restart + total time. |
| _scripts/start-dashboard.test.js | Adds pure-logic tests for dashboard line parsing/state transitions. |
| _scripts/start-dashboard.js | Implements the live TTY startup dashboard and log capture for start-dev. |
| _scripts/pm2-all.sh | Adds dashboard entrypoint for start-dev, streams Nx output to logs, and improves readiness banner checks. |
| _scripts/check-url.sh | Enhances readiness polling output with labels/heartbeat and improved messaging. |
| _scripts/check-redis.sh | Adds a redis readiness gate (non-fatal) to reduce cold-start connection noise. |
| _scripts/check-pre-launch.sh | Avoids blocking prompts in non-TTY contexts when Nx cache is large. |
| _scripts/check-db-patcher.sh | Derives expected db-patcher count dynamically from migrations layout. |
| _scripts/check-build-dev-coverage.sh | Adds a guard to ensure dist-loaded libs in mza scope have a build-dev target. |
| _dev/pm2/start.sh | Adds Docker daemon preflight, quiet infra startup path, labeled URL checks, and redis/db-patcher gating. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| for i in $(seq 1 "$RETRY"); do | ||
| if [ "$(curl -s -o /dev/null -w "%{http_code}" "http://$TARGET")" == "$EXPECTED" ]; then |
| { project: 'fxa-auth-server', label: 'fxa-auth-server', url: 'http://localhost:9000/__heartbeat__', pm2: 'auth' }, | ||
| { project: 'fxa-profile-server', label: 'fxa-profile-server', url: 'http://localhost:1111/__heartbeat__', pm2: 'profile' }, | ||
| { project: 'fxa-content-server', label: 'fxa-content-server', url: 'http://localhost:3030', pm2: 'content' }, | ||
| { project: 'fxa-settings', label: 'fxa-settings', url: 'http://localhost:3000/settings/static/js/bundle.js', pm2: 'settings-react' }, | ||
| { project: '123done', label: '123done', url: 'http://localhost:8080', pm2: '123done' }, |
| if (j.targets && j.targets['build-dev']) ok=true; | ||
| if (j.scripts && j.scripts['build-dev']) ok=true; |
| #!/usr/bin/env node | ||
| 'use strict'; | ||
|
|
||
| // Live, in-place startup dashboard for `yarn start mza` (the start-dev fast |
There was a problem hiding this comment.
AI helped to build this live dashboard out. TLDR is there is a state machine to track the services startup state, rendering live output from nx and pm2 and service health polling
Since this is only on yarn start mza and a developer only improvement I think its probably ok.
| "command": "echo Build complete" | ||
| } | ||
| }, | ||
| "build-dev": { |
There was a problem hiding this comment.
This nx command is the stripped down version of build, skips compiling since dev mode builds from source anyways.
There was a problem hiding this comment.
Does this mean that compilation errors might not be caught until CI build step?
There was a problem hiding this comment.
Stack startup probably isn't the right place to catch those errors anyway... Perhaps adding pre-push hook would cover this gap?
Because: - Local stack startup (yarn start / yarn start mza) was slow and gave little feedback about what was happening during the boot sequence. This commit: - Speeds up the local stack startup and adds a live dashboard. - Shows a spinner during the quiet sync restart in yarn start mza. - Fixes the l10n-gettext-extract workflow's Node setup: the FxA repo is checked out into fxa-code/, so the root .nvmrc setup-node looked for was never present. Reorders setup-node after the checkout and points node-version-file at fxa-code/.nvmrc, keeping a single version source. - Bounds check-url.sh curl with connect/max timeouts so a stalled connection can't hang the readiness loop. - Polls fxa-content-server's bundle path so the dashboard can't report ready before webpack emits the bundle. - Recognizes build-dev declared under package.json nx.targets in the build-dev coverage guard.
vpomerleau
left a comment
There was a problem hiding this comment.
Tested the startup script and it was fast and clean 👍 🎉
✅ Stack ready in 151s.
| @@ -0,0 +1,70 @@ | |||
| #!/bin/bash | |||
There was a problem hiding this comment.
We chatted about this on Slack, we'll want to remove this file before merge since it is not called (was only for patch validation)
| "outputs": ["{projectRoot}/storybook-static"], | ||
| "cache": true | ||
| }, | ||
| "build-dev": { |
There was a problem hiding this comment.
This is wrong... We shouldn't be hard coding package specific files paths in the root nx.json file.
There was a problem hiding this comment.
@vbudhram Can we break this up into two (or more) PRs? We should have one commit for the live dashboard. And another commit for the stack start up fixes. I'm also a little skeptical we need these *-dev tasks... I have a branch in progress with NX upgrades that starts much faster, but didn't require a 'build-dev' or 'restart-dev' task. As far as I can tell the real speed up in this commit comes from the fact that you no longer require build to be dependency of start, which is actually a pretty small change in this sea of other changes. Also, all start tasks are technically 'dev' tasks because we don't use PM2 in prod...
Because
yarn start mzaran full production builds of the entire@fxa/*library graph, which the dev servers resolve fromsrcanyway, so most of that build work was wasted.This pull request
build-devfast path via separatestart-dev/restart-devnx targets (nx.json, per-packagepackage.json).yarn start mza/yarn restart mzause them; fullyarn start/yarn restartand CI keepbuildand stay race-free._scripts/start-dashboard.js(+start-dashboard.test.js): an in-place per-service dashboard (nx command, live activity line, phase, ready time). TTY-only;FXA_START_PLAIN=1/ non-TTY fall back to streaming. Raw output always lands inartifacts/nx-start-dev.log, and every failure points to where to look._scripts/start-mza.sh: quiets pm2 infra/launch/table noise and reports total startup time.REDIS_PASSWORD), dynamic db-patcher count, parallelfxa-sharedESM/CJS compile, and acheck-build-dev-coverage.shguard.Speed improvements
yarn start mzabuild graph@fxa/*build)yarn restart mzalib buildsyarn restart mzawall (warm)@fxa/*libs resolve fromsrcat dev runtime, sobuild-devskips them. Fullyarn start/yarn restart/ CI graphs are unchanged.Issue that this pull request solves
N/A — local dev-tooling improvement (no Jira ticket).
Checklist
Put an
xin the boxes that applyOther information
How to test:
yarn start mza(TTY) → expect the compact infra summary + live dashboard, then✅ Stack ready in Ns.yarn restart mza→ reloads viabuild-dev(~3 build tasks), not the full lib build.FXA_START_PLAIN=1 yarn start mza→ falls back to plain nx/pm2 streaming.Output