Skip to content

Latest commit

 

History

History
481 lines (460 loc) · 35.3 KB

TESTING.md

File metadata and controls

481 lines (460 loc) · 35.3 KB

Testing SpecThread

Playwright Test is the shared behavior-test runner. Linting, TypeScript checks, C# formatting, and builds remain separate checks.

Setup and commands

From the repository root with Node.js 24, npm 11, and the SDK in global.json:

npm ci
dotnet restore app/api --locked-mode
dotnet tool restore
npx playwright install chromium
npm run lint
npm run typecheck
dotnet format app/api --verify-no-changes --no-restore
npm test

On Linux CI install browsers with npx playwright install --with-deps chromium. The GitHub Actions workflow runs these same checks. It does not need Supabase credentials. Reinstall Chromium after updating Playwright.

CI caches npm packages from package-lock.json and NuGet packages from app/api/packages.lock.json; the API restore remains locked. If the verify job fails, its Playwright HTML report and test-results/ directory are uploaded as a GitHub Actions artifact for seven days when present.

A second CI job, api-image, builds the root Dockerfile that Render deploys and runs it as Render does: Production, port 8080, no database or issuer. It requires /health to answer {"status":"ok"}, anonymous /me to answer 401, the OpenAPI document not to be served, the process not to run as root, and the Supabase root CA to be in the image's trust store. Reproduce it with docker build --tag specthread-api:ci . and docker run --rm --publish 8080:8080 specthread-api:ci.

A third job, deploy-api, runs only on pushes to main after both pass. It is not a check: it applies migrations to Supabase and deploys the API (ADR-044, docs/DEPLOYMENT.md). After the Render deploy hook succeeds, it polls the public /health endpoint for up to five minutes and requires {"status":"ok"}. This checks service liveness, not that the exact requested commit is serving traffic.

The test web server and schema project require a running Docker engine and postgres:17. The first webServer entry calls scripts/build-api.mjs before starting the second entry (the API). Do not move that build into globalSetup: Playwright runs globalSetup after webServer startup, which locks the API DLL on Windows. scripts/start-test-web.mjs verifies EF initialization, creates an isolated password-protected database on a random loopback port, applies the versioned SQL migrations, and builds/starts Next.js with that database. No persistent volume is used. Global teardown also stops the web-test container on Windows, where process-tree termination may skip signal handlers. Auth tests use a random test secret, loopback base URL, and disabled live GitHub/dashboard credentials. They never use Supabase or real OAuth accounts. Schema runtime tests use a separate disposable database and simulated provider cancellation; successful real GitHub login remains a deployment check.

npm test -- --list         # Discover tests without starting apps
npm test -- --project=api  # API/EF tests (both managed servers still start)
npm test -- --headed      # Visible browser
npm run test:ui            # Interactive test UI
npm run test:report        # Latest HTML report
npm run build             # Standalone web production build
npm run build:api          # Standalone Release API build

The test web app runs in production mode, so Better Auth rate limits auth requests per client IP. The browser fixture in tests/e2e/fixtures.ts gives each test its own x-forwarded-for address, so a full run never exhausts one shared budget; headers a test sets explicitly take precedence. The test web app enables GitHub and Google with dummy client values, so both sign-in buttons render; nothing is sent to either provider. The GitHub stand-in counts the API requests it receives (GET /__github/requests?path=), which the release evidence test uses to check that stable answers are not asked again.

npm test builds the web app and the Release API before tests. It starts the web app at 127.0.0.1:3100, a test-only JWKS issuer (scripts/start-test-jwks.mjs) at 127.0.0.1:5101, and the API in Development at 127.0.0.1:5100 trusting that issuer. Auth tests also start extra API instances on 5102-5104, and product API tests on 5105. The Teams API tests use 5116, team management 5117, invitations 5118, documents 5119, requirement documents 5120, criteria versions 5121, follow-ups 5122, milestones 5123, evidence search and history 5124 (fake GitHub) and 5125, and evidence suggestions 5130 (fake GitHub) and 5131 (requirement documents moved from 5107 to 5109 when the branches were merged, because the proxy and GitHub stand-ins below already use those). scripts/start-test-web.mjs also starts the API on 5106 for browser tests. That instance uses the web test database and trusts the test web app as its token issuer. The web app reaches it through a test-only proxy on 5107 (scripts/test-api-proxy.mjs), set as SPECTHREAD_API_URL. The test API talks to a stand-in for GitHub on 5108 (scripts/test-github.mjs) with an app key generated for the run; tests describe what one user's GitHub token can see with tests/support/github.ts and never reach the real GitHub. The repository API tests use their own stand-in on 5109 and API instances on 5110 and 5111; the evidence API tests use 5113 and 5114, and the review API tests 5115. Test API instances blank the GitHub settings unless a test sets them, so a developer's user-secrets never reach them. They blank OTEL_EXPORTER_OTLP_ENDPOINT and OTEL_EXPORTER_OTLP_HEADERS the same way, so telemetry is off unless a test turns it on. The browser-test web app and its API (5106) do turn it on: scripts/start-test-web.mjs starts a fake OTLP receiver on 5126 (scripts/test-otlp.mjs), so the browser suite runs with telemetry on, and tests read back what arrived with tests/support/otlp.ts. The telemetry API tests start their own API on 5127 (and 5128 for a startup failure) sending to that receiver. A test calls failApi from tests/support/api-faults.ts to make the API fail, drop the connection, answer with a wrong body, or respond slowly for that test's own user only, so parallel tests are unaffected. Keep these ports free; existing servers are not reused. Do not run competing Next.js builds or development processes in this checkout: they share app/web/.next. Google Fonts currently require network access during the web build.

Current coverage

  • Chromium: home/dashboard navigation, keyboard skip link, login/signup navigation, GitHub buttons, labeled dashboard preview tabs and evidence expansion, shared sidebar route transitions, workspace and settings sidebar selection, the profile menu (signed out, loading, signed in, and failed log out), and scaffold/auth pages at mobile width. Desktop/mobile screenshots are saved inside the ignored test-results directory.

  • Landing page: navbar links to the About and How it works sections, redirects from the former About pages, public Privacy and Terms pages without the app sidebar, the hero animation with its pause control, evidence gathering and replay, the review decision on scroll, finished states under reduced motion, the signed-in Dashboard link, and narrow-screen layout and menu.

  • Route protection: signed-out redirects to login with the requested path for each route family and unknown URLs, forged and revoked sessions, public pages and their images, return to the requested page after email and social sign-in, and safeNextPath rejection of other origins, control characters, over-long values, and login/signup loops.

  • Route scaffold: navigation to the Teams area, representative static and dynamic placeholder pages that are still planned, a useful 404 for an unknown URL, and planned-page links that never point at an example project. Projects and Teams are real pages now and are covered by their own tests.

  • Projects (browser, real API on port 5106): the empty list for a new user, creating a project and landing on its overview, ordering by name, a blank name rejected with a message, keyboard submission, and the narrow layout. Project workspace: the details panel, recent requirements and the Requirements tab with rows seeded through SQL, the absence of a members page (ADR-031), a planned section inside the project frame, rename with a rejected blank name, archive after confirmation and the archived read-only state, a non-owner member seeing no rename or archive controls, and not-found for non-members, unknown IDs, and malformed IDs. The api project checks response parsing for projects, requirements, and members, name limits, UUID-shaped IDs, date formatting, and reading field messages from problem details.

  • Project failures (browser, through the test proxy): the list and every project section showing the error page for a 500, a dropped connection, a rejected token, and a malformed body, with "Try again" recovering; create, rename, and archive keeping the input and showing the right message for 400, 403, 404, 409, 500, a dropped connection, and a malformed success body; the loading message for slow calls, with the project frame kept while a tab loads; a form submitted after the session ended going to sign-in without creating anything; and a removed member seeing not-found. Not exercised: the API being slow enough to hit a platform timeout, and two owners renaming at the same moment.

  • Requirements (browser, real API through the test proxy): creating with ordered criteria, removing a criterion, blank title and blank criterion messages, editing with the version rising, a second save refused after someone else saved, archive after confirmation and the archived read-only state, an archived project hiding all change controls and refusing a form that was already open, not-found for other users, wrong-project addresses, and malformed IDs, keyboard entry and the narrow layout. Failures: create, save, and archive each with a 500, a dropped connection, a 404, and a malformed success body, keeping what was typed; API field messages shown next to the title and the right criterion; and a failing requirement page showing the error inside the project frame and recovering. The api project checks requirement parsing, input trimming and limits, and mapping problem details to fields. Reordering criteria: moving rows up and down on a new and on an existing requirement and reading the saved order; the first and last rows refusing to move further; text typed between moves staying with its row; reordering with the keyboard at 390px with focus kept on the pressed button; a criterion's message staying with its row after moves and removals; a failed save keeping the order; and moveItem in the api project. Not exercised: 50 criteria through the browser, two saves landing in the same instant, and what a screen reader announces.

  • GitHub repository connection (ADR-032). Browser: connecting, seeing the repository across the project, and disconnecting after confirmation; no linked GitHub account; a rejected GitHub sign-in; no installation; the choice kept after a refused attempt; a repository or installation that disappeared; a team Admin who is not the Owner connecting and disconnecting while the Owner sees the name on the project; Members and archived projects seeing it read-only; GitHub down, rate limiting, dropping connections, or answering wrongly while listing and while connecting; API failures while connecting and disconnecting; and the error page inside the project frame. API (schema suite): changes by the team Owner or an Admin only (a Member gets 403), 404 for non-members, refusing other users' installations and repositories and mismatched pairs, input validation, paging past 100 repositories and the 500 limit, 502 and 503 mapping, missing app credentials, anonymous 401, the user's token never being stored, and the migration's RLS, constraints, and rollback. Not exercised: the real GitHub, and an expired GitHub token being refreshed.

  • Evidence search and history (ADR-054). tests/schema/evidence-history.spec.ts (fake GitHub on 5124, API on 5125): additions and removals recorded newest first with who, when, and what the link was, outliving it; nothing recorded for a refused removal; title search by every word, case-insensitive, with state (merged too), author, and whether linked; a repo: qualifier in the words finding nothing; results from another repository dropped; validation of q; 409 without a repository; 404, 401; GitHub failures and rate limiting as 502; and the table's constraints, RLS, backfill from existing links, and rollback. Each test repository has its own name, because the fake GitHub finds a repository for search by name, as GitHub does. Browser (tests/e2e/evidence.spec.ts): searching, the results' state and author, linking from a result, "Linked by" on the link, the result marked Linked and offered again after removal, the change list on the page and the history page, no matches, and a search failing, refused, or answered wrongly.

  • Suggestions and moves (ADR-056). tests/schema/evidence-suggestions.spec.ts (fake GitHub on 5130, API on 5131): every form of closing reference, and plain mentions, other repositories, longer numbers, and unlinked issues not matching; descriptions not stored; a pending suggestion missing from the evidence list, count, review snapshot, and Evidence complete; confirm and reject with who and when, decided ones refused and never suggested again; linking by hand, edited references, refresh, and items gone from GitHub; members only, archived (also while waiting for the team lock), disconnected, and GitHub failures; a linked issue changing during a scan, and a truncated timeline dropping nothing; moves with both histories, outdated decisions, no re-suggestion, and moving back; refusals for the target, a duplicate, the limit, and no repository; the link or the target's links changing while a move waits for the lock; an earlier decision surviving a move; release comparison after a move; and the migration's constraints, RLS, and rollback. Browser (tests/e2e/evidence.spec.ts): finding, the label and reference, not counted, confirm, reject, the change list, staying rejected, a failed scan and a refused decision; moving with cancel, both requirements' changes and the history page, and a refused move. Not exercised: the real GitHub timeline, and timelines over 100 events.

  • Evidence (ADR-033). Browser: the hint when no repository is connected; linking by number, #number, and address; timeline order, state, and author; refused references with what was typed kept; refresh picking up closed and merged items and keeping deleted ones; unlinking; read-only evidence for archived requirements, archived projects, and a disconnected repository; keyboard use and the narrow layout; GitHub failing, dropping connections, answering wrongly, rejecting the app, or reporting the app uninstalled; API failures for link, refresh, and unlink; and the error page inside the project frame. API (schema suite): members only, 404 for outsiders and for evidence of another requirement, reference validation, duplicates, other repositories, the 50-link limit, archived and unconnected projects, no partial refresh after a failure, and the migration's RLS, constraints, and rollback. Tests that make the app or the installation token fail name their own installation or repository in the fault path, because every test shares those. Commits (ADR-034): linking by short SHA, full SHA, commit address, and pull-request commit address; changes and author shown for a commit; a pull request's totals and its commit list; duplicates under different spellings; unknown SHAs; answers from GitHub that are not the requested commit; refresh leaving commits untouched; and the migration's constraints and a rollback that removes commit links only. Checks (ADR-035): every GitHub conclusion and status mapped to its result; check runs and statuses merged and sorted; links outside GitHub dropped; no checks; checks GitHub will not let the app read, and only one of the two kinds readable; more than 100 checks; malformed answers and failures on either endpoint stopping a link or refresh with nothing saved; refresh updating results for pull requests and commits; and the results in words in the browser. Source (ADR-036): the link endpoint ignoring a source sent by the caller, a suggested link naming who confirmed it, and the migration's default, constraint, and rollback. Releases (ADR-037): linking by tag and by address, including tags with slashes, encoded characters, and only digits; the pre-release badge; which linked changes a release contains, using a squash-merged pull request's merge commit, with unmerged pull requests not compared and vanished commits not included; a change linked after a release; refresh recomputing contents and forgetting unlinked changes; a release deleted on GitHub keeping its snapshot; duplicates, unknown tags, other repositories, and malformed references; failures on each of the three GitHub calls saving nothing; and the migration's constraints and a rollback that removes release links only. The migration tests at the end of tests/schema/evidence.spec.ts roll back newest first and leave each migration rolled back, so their order matters. Not exercised: the real GitHub, a pull request with more than 100 commits, and the installation token being reused within one request.

  • Review decisions (ADR-038). tests/schema/reviews.spec.ts calls the API on port 5115 against Docker Postgres, with evidence links inserted as rows: decisions by two members read newest first, the stored evidence snapshot staying as it was after links change, no update or delete, the author recording their own decision (ADR-058), 404 for non-members and unknown requirements, 401 anonymous, validation of the choice, note, and version, 409 for a stale or future version and for an archived requirement or project, the latest decision and outdated on the requirement list (edit, link added, link removed, different link, link re-read, other requirements), and the migration's RLS, grants, constraints, foreign keys, and rollback. tests/e2e/reviews.spec.ts uses two users in separate browser contexts, the second added to the project in the database: the author deciding on their own requirement (also alone in a team), a Reviewer deciding without edit controls and an Observer seeing no form, each of the three decisions, the required reason, history, the decision on both lists, outdated after an edit and after a link is added but not after a removal that restores the reviewed set or a refresh from the fake GitHub, a requirement changed or archived while the form was open, keyboard use at 390px, every way the API can refuse or fail the action with the entered text kept, and failed or malformed loads with a retry. The parsers, the note rule, and reviewOutdated are unit tested in tests/api/project-data.spec.ts. The reviews migration stays applied in the other schema specs, so their migration-history counts include it. Not exercised: two reviewers deciding at the same moment.

  • Evidence count on requirement lists: tests/schema/reviews.spec.ts checks that the list's evidenceCount is 0 with nothing linked, counts issues, commits, and releases, follows a removed link, and leaves out other requirements' links. tests/e2e/evidence.spec.ts checks "No evidence", the plural and singular count on the Requirements tab and the overview, the row at 390px, and the error page for a list whose count is missing, text, or negative. The parser and wording are unit tested in tests/api/project-data.spec.ts.

  • API: health response without database credentials, development OpenAPI, and unknown routes (401 anonymous, 404 authenticated).

  • API JWT validation: valid tokens identify the user; missing, malformed, expired, wrong-issuer, wrong-audience, no-expiry, unpublished-key, HS256, and alg-none tokens are rejected; missing subjects are forbidden; a missing Auth:Issuer fails explicitly. The test issuer generates keys per run. The schema suite validates a real Better Auth ES256 token against the API after expiring a legacy EdDSA key.

  • Product API: team-owned creation/list/rename/archive/restore, Owner/Admin management, Member requirement editing, inherited member reads, revoked access after team removal, inactive legacy memberships, retired membership writes, isolated team project/archive lists, and active requirement counts. Ordered criteria/stale-version and archived-item checks remain. OpenAPI and anonymous checks run without a DB.

  • Documents (schema suite, API on 5119 against Docker Postgres): PDF, DOC, and DOCX upload; newest-first lists; byte-identical downloads with attachment and nosniff headers; idempotent archive; rejection of wrong types, mismatched contents, empty, oversized (400) and far oversized (413) files and long titles; 404 for non-members; 409 for archived projects; table constraints and RLS. Browser tests cover the Documentation tab, upload, download, archive with confirmation, the form's browser and server messages, a failed upload, archived projects, and a document opened under another project's address. The test API proxy forwards Content-Disposition so downloads keep their file names.

  • Requirement documents and the create-requirement story (ADR-049): the schema suite (API on 5120) links documents on create and replaces them on save, with kept links keeping who linked them and when; refuses another project's document, an unknown ID, a duplicate, and more than 20; refuses newly linking an archived document while one already linked stays linked; answers 404 to outsiders; and checks the table's keys, RLS, browser-role denial, and a rollback that keeps requirements and documents. tests/schema/product-api.spec.ts refuses a missing title or zero criteria on create and on save and checks createdBy and createdAt. Browser tests: the starting criterion row, the "at least one" message on a new and an existing requirement, the creation date on the page, the empty documents state linking to Documentation, ticking a document and opening it from the requirement, an archived link marked and then unticked, ticks kept after a refused save with the API's message beside them, and keyboard ticking at 390px. Not exercised: 20 documents through the browser.

  • Criteria versions and the accepted-criteria warning (ADR-051). tests/schema/criteria-versions.spec.ts (API on 5121): version 1 on create, a new version only when the ordered, trimmed criteria change, who and when, single versions and 404 for an unknown number, archived requirements still readable, 404 for outsiders and 401 anonymous; the 409 accepted_criteria_change without confirmation that changes nothing, the confirmed save, the accepted decision kept on version 1 with criteriaChanged on the list, no confirmation after a newer rejection; a catch-up version for criteria saved by the previous API; and the migrations' RLS, constraints (texts only, an empty array allowed), backfill of version 1 and of decisions on the current requirement version, and rollbacks, newest first. tests/e2e/reviews.spec.ts: the warning after an acceptance, its focus and text, Keep editing sending nothing, a title change saving without it, confirming to "Needs review" on the page and the list, the read-only version page with its decision, the Requirement history page, and an acceptance recorded while the form was open caught on save.

  • Reviewer and Observer roles (ADR-058). tests/schema/team-roles.spec.ts (API on 5133, evidence inserted as rows, so no GitHub): both roles read projects, requirements, evidence, history, follow-ups, documents, members, and decisions; both get 403 with the read-only message on every requirement, milestone, follow-up, evidence, suggestion, and document write, and nothing changes; a Reviewer decides, an Observer gets 403, and the author decides on their own requirement; the Owner and an Admin invite each role and acceptance grants it, Reviewers and Observers cannot invite or create projects, the Owner changes roles (a Reviewer made Member can create, then as Observer cannot), Admins cannot change roles but remove both new roles; and the migration's constraint and rollback (removes those memberships and invitations, keeps the others). tests/e2e: reviews.spec.ts (Reviewer and Observer views), team-invitations.spec.ts (role options per inviter and their descriptions), team-management.spec.ts (assigning Reviewer), and projects.spec.ts (the combined view grouped by team, including a team the user only belongs to, and a team heading opening only that team's projects). tests/api/project-data.spec.ts checks the role helpers and the grouping without a browser.

  • Requirement comments (ADR-059). tests/schema/comments.spec.ts (API on 5134): every role, Observers included, comments and replies, and the requirement and its decisions stay identical; validation (empty, 5,001 characters, NUL on create and edit, unknown parent, reply to a reply), 404 for non-members; only the author edits, the author, Owner, and Admins delete, Members and Observers get 403; a deleted comment keeps its place with no text, can't be edited or replied to, and the event list records each change newest first without old text; archived requirements refuse every write; and the migration's RLS, constraints, and rollback. tests/api/comments.spec.ts checks parsing, threading, permissions, notification recipients and email, event text, and the date-time format without a browser. tests/e2e/comments.spec.ts: an Observer posts a multi-line question with no review change, the Owner replies, edits, cancels and then confirms deleting the Observer's comment (the reply stays), the History page lists the four events without the deleted text, refused posts and edits keep the typed text, a 201 with an unreadable body still reads as posted, and the thread stays usable by keyboard at 390 px without overflow.

  • Follow-ups (ADR-052). tests/schema/follow-ups.spec.ts (API on 5122): links shown both ways with their history, no version change, removal recorded; refusals for no requirement, itself, another project, an unknown ID, a duplicate, the reverse link, and more than 20; archived requirements and projects (also while waiting for the team lock); 404 for outsiders, 401 anonymous; table constraints, RLS, and rollback. tests/e2e/follow-ups.spec.ts: adding, following the link both ways, removing, the change list on the page and the history page, and a failed and a refused add.

  • Milestones (ADR-053) and Evidence complete (ADR-057). tests/schema/milestones.spec.ts (API on 5123): a label set on create, set and cleared later without a version change, the length limit, archived and outsider refusals; the exact-label filter alone and with archived=true, the labels endpoint; every case of the Evidence complete rule and its missing parts, on the list and on the requirement, with issues, pull requests, commits, checks, and releases inserted in the database; the column's check and rollback. tests/e2e/milestones.spec.ts: a label on create and on the page, filtering, the counts with a seeded issue, merged pull request, release, and acceptance, the missing parts in the row and as Met/Missing on the requirement page, the empty state, 390px without horizontal scrolling, and a failed save keeping the label. Not exercised: 20 follow-ups through the browser, and an Evidence complete requirement fed by the fake GitHub. Parsing of evidenceMissing, including the previous API's response, is in tests/api/project-data.spec.ts.

  • Telemetry (ADR-055). tests/api/telemetry.spec.ts (API on 5127): spans named by route template with the caller's user.id; an invitation token in the path and query, a token in an unmatched path, and the bearer token never sent; the error log linked by trace ID; request metrics by route; /health not traced; the credentials header arriving decoded; an endpoint that is not an HTTP(S) URL stopping startup; and the web redaction and header parsing functions directly. tests/e2e/telemetry.spec.ts: an invitation link is one trace across web and API without its token; a signed-in page carries the user's ID on web and API spans, never their email, name, or session token. Each test starts its own trace with a traceparent header. Not exercised: Vercel's x-real-ip (read only on Vercel), real Grafana Cloud, and the API sending nothing when the endpoint is unset (nothing is registered then, and there is no receiver to observe the absence). The privacy page test in tests/e2e/landing.spec.ts checks the telemetry disclosure.

  • Last editor, archive view, and restore (ADR-050): tests/schema/product-api.spec.ts checks updatedBy after create and after another member's save; default and archived=true lists; an archived requirement readable but refusing saves and reviews; restore (member, idempotent, 404 for outsiders) keeping the version and last editor, then editing again; and the migration's backfill, empty-value fallback, foreign key, and rollback. tests/schema/reviews.spec.ts records a decision after a restore and refuses archive and restore in an archived project. Browser tests: the "Updated by" line, the Archived view and its empty state, the read-only page and edit notice, restoring, a failed restore, and a restore refused once the project is archived.

  • Teams: real API-backed creation, search, empty/error/retry and member-only access; reference Home, independent saved sidebar groups, personal favorites sorting first, keyboard team menu, and desktop/mobile screenshots. Required onboarding resumes across protected routes and closed browser sessions. API checks cover atomic first creation, concurrent submissions (one 201/one 409), completion surviving membership loss, and unbypassable project creation guard. Sidebar routes never expose JWTs. Web-test API 5106 shares the disposable DB and trusts the real Better Auth origin; schema API 5116 verifies roles, private preferences, RLS/grants, and rollback.

  • Team projects: compact real-data browser table, active/archive isolation, links to the existing project page, member-only page access, and mobile overflow checks. Archive restoration covers named confirmation/cancel, active-list refresh, Member read-only controls, and an Admin demoted while confirmation is open. Legacy migration tests preserve separate collaborator sets, owner fallback, timestamps, requirements/criteria, and existing teams. They verify required FK, onboarding backfill, and nondestructive association rollback/reapply. Member/settings checks cover role Save, read-only permissions, name/description validation, named removal/leave/transfer dialogs, cancellation/Escape/focus, post-transfer roles, lost access after leave, and concurrent mutation invariants. Invitation checks cover the Members dialog, Copy link, manager roles, the real invitation table, resend/revoke and unavailable states, focused mobile acceptance, wrong-account sign-out, and onboarding remaining required before acceptance. Schema tests cover hashed secrets, seven-day expiry, email normalization, current inviter permissions, verification/matching email, single-use/idempotent acceptance, concurrent accepts/revoke, cross-team isolation, RLS/grants, and rollback/reapply. Real Better Auth signup/verification with captured mail preserves the invitation destination and joins only the invited team without initial-team creation. Invitation tests use the loopback log sender or captured messages, never live mail.

  • EF: PostgreSQL provider initialization using dummy credentials at an unreachable local address, and explicit rejection when connection configuration is missing. These invoke the local dotnet-ef tool and never connect to Supabase.

  • Email/password and linking (schema suite, real Better Auth on Docker Postgres, captured emails): hashed credentials, verification required before sign-in, verify links sign in, duplicate sign-up and unknown-email reset look identical, single-use resets that revoke sessions, link-social for GitHub and Google, refusal to unlink the last method, and refusal to unlink when only a disabled provider would remain. Browser tests cover the forms, errors, 429, resend, reset pages, /account redirect, linking error messages, and narrow screens with mocked auth responses. Real Google/GitHub linking callbacks and Resend delivery are verified manually.

  • Auth: explicit configuration failures, email delivery and provider enablement rules, verified TLS options, restricted origins, CA PEM parsing/bundles, complete GitHub credential pairs, sign-in pages offering only enabled providers, session endpoint, HTTP rejection of unrelated origins, proxy header precedence, retryable HTTP/network failures, provider redirection, and a safe public error page. An active session redirects login and signup to Dashboard; an invalid cookie does not. The browser test creates a disposable user/session in the isolated web-test database and shares one random test secret across Playwright workers.

  • Schema: migration/rollback in Docker, Better Auth column compatibility, RLS, foreign keys, uniqueness, content constraints, and versioned SQL updates. Auth runtime tests additionally verify the rate-limit schema/RLS, simultaneous requests across independent auth instances, persistence across a fresh instance, window expiry, real OAuth-state cancellation, session joins, expired/revoked sessions, and isolated rate-limit rollback/reapply. Local join timing samples are attached to the Playwright report, without asserting a speedup ratio. A simulated successful GitHub callback verifies encrypted access/refresh tokens in PostgreSQL, usable token retrieval, prefixed legacy plaintext compatibility, and rejection of corrupt ciphertext. No real provider credentials are used.

Live OAuth and deployed TLS connectivity are not tested. Teams, onboarding, and team Projects/archive have browser coverage against the disposable database; global project flows remain API or scaffold checks owned by Claude. The health endpoint is liveness, not database readiness. EF initialization is not proof that live database credentials work.

Conventions

Use browser fixtures in tests/e2e and request fixtures in tests/api. Browser tests import test from tests/e2e/fixtures.ts, which signs each test in with a real seeded session by default. Signed-out tests set test.use({ signedIn: false }). Tests that log out must create their own session with createTestSession, because the default session is shared per worker. Use accessible locators and retrying assertions, not arbitrary sleeps. Browser-free TypeScript logic tests can get a separate Playwright project when such logic exists. Playwright cannot directly run C# unit tests; a different runner requires an explicit team decision. Add meaningful success and failure cases as features land.

CI mode rejects test.only, uses one worker, and retries twice. Local runs do not retry. Reports in playwright-report and artifacts in test-results are ignored by Git and ESLint. Traces/screenshots are retained for failures and can contain sensitive data when authenticated scenarios are later introduced.

Type checking generates Next.js route types and checks the web app plus the root Playwright configuration/tests. Playwright transpilation alone is not type checking. C# uses nullable checking and warnings as errors. There is no standalone JS/TS formatter configured; C# formatting uses dotnet format.

This development testing policy does not expand the product's limited verification feature. Passing tests are evidence, not automatic requirement acceptance.