Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,10 @@
DATABASE_URL=postgresql://user:password@ep-example-pooler.us-east-1.aws.neon.tech/neondb?sslmode=require
# Migrations only: the DIRECT string (same host without "-pooler").
DATABASE_URL_UNPOOLED=postgresql://user:password@ep-example.us-east-1.aws.neon.tech/neondb?sslmode=require
# Optional: websocket for a workspace that allows HTTPS but cannot reach
# Postgres over TCP. Leave empty for the default pg driver on deployments.
# WebSocket and storage clients honor HTTPS_PROXY and NO_PROXY (Node 24.5+).
DATABASE_TRANSPORT=

# Better Auth (ADR-006). Secret: 32+ random characters, e.g.
# openssl rand -base64 32
Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,7 @@ jobs:
cache: pnpm
- run: pnpm install --frozen-lockfile
- run: pnpm db:migrate
- run: pnpm db:verify --target app
- run: pnpm test:unit
- uses: actions/upload-artifact@v7
if: always()
Expand Down Expand Up @@ -129,6 +130,7 @@ jobs:
- run: pnpm install --frozen-lockfile
- run: pnpm exec playwright install --with-deps chromium
- run: pnpm db:migrate
- run: pnpm db:verify --target app
- run: pnpm build
- name: Scan client bundle for secrets
run: pnpm scan:bundle
Expand Down
3 changes: 2 additions & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -213,6 +213,7 @@ fails if this table and `package.json` differ.
| `pnpm db:migrate` | Apply migrations (uses `DATABASE_URL_UNPOOLED` when set) |
| `pnpm db:migrate:to` | Apply migrations to a named database: `local`, `branch` or `production` (ADR-049) |
| `pnpm db:whoami` | Show which database a target is and which migrations it has; no secrets printed |
| `pnpm db:verify` | Read-only financial schema and SELECT-access preflight for a named target; no repairs |
| `pnpm ai:check` | Ask each test model one question with the real `GEMINI_API_KEY` from `.env.local`; prints which answer. `pnpm ai:check users` asks the models that answer users, which spends their small daily allowance (ADR-059). `pnpm ai:check receipt` sends each test model a made-up receipt photo and says whether it read the total (ADR-067) |
| `pnpm batch` | Synthetic batches: `load`, `list`, `report`, `delete`, `promote`, `rollup`; `give` and `take-back` for developer accounts (`docs/SYNTHETIC.md`) |
| `pnpm nessie` | Nessie demo customers: `seed` a synthetic year into Nessie, `link` it to a user, `status`, and live `demo` changes (`docs/NESSIE_PLAN.md`) |
Expand Down Expand Up @@ -266,7 +267,7 @@ src/
<feature>/ built so far: accounts, agent, alerts, auth, bills, budgets,
cashflow, categorization, imports, insights,
landing, messaging, plans, recurring,
synthetic, transactions
synthetic, transactions, transfers
domain/ pure TypeScript logic, no I/O (most unit tests here)
server/ repositories, server actions ('server-only')
ui/ React components for this feature
Expand Down
22 changes: 22 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,3 +23,25 @@ pnpm dev
```

Open http://localhost:3000. Every command is listed in AGENTS.md section 5.

In a cloud workspace that allows HTTPS but cannot reach PostgreSQL over TCP,
set `DATABASE_TRANSPORT=websocket` in `.env.local`. The app then uses Neon's
WebSocket Pool; deployments still use `pg` when this variable is empty. The
WebSocket and storage clients honor the runtime's `HTTPS_PROXY` and `NO_PROXY`
settings on Node 24.5 or later. Migrations still use a direct TCP connection.

Local statement imports need the five storage variables in `.env.example`
from the Neon `storage-tests` branch. Put them in `.env.local` for the app and
`.env.storage-tests` for tests. Both files are ignored by Git; tests always use
local PostgreSQL and never the production storage bucket.

Before release, run `pnpm db:verify --target app` with the intended runtime
connection role. It checks financial schema and read access without changing
data; a successful health check or migration journal alone is insufficient.
See [database readiness](docs/DATABASE_READINESS.md) for targets, limitations
and the migrate/verify/release sequence.

Review recorded transfers from Transactions → **Review transfers**. Choose a
specific counterpart and confirm the classification; removing a pairing keeps
the existing labels. See [transfer review](docs/TRANSFER_REVIEW.md) for the
window, limits, Bills evidence rules and the separate normal-activity action.
30 changes: 30 additions & 0 deletions docs/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -194,6 +194,24 @@ Caching: pages that show workspace data render dynamically. No `unstable_cache`
or fetch caching of workspace data. Client state is keyed by workspace id and
cleared on workspace switch and sign-out.

### Web transfer review (built, ADR-066)

The Transactions subroute uses `features/transfers` for a bounded read-only
snapshot and explicit metadata writes. Every entry point authorizes before
financial reads; the repository scopes every row and account join by workspace.
Candidates use posted, safe integer, nonzero opposite cents on different owned
accounts within three calendar days. The UI requires a specific choice and
confirmation, including for an unambiguous suggestion.

Review writes and ordinary transaction update/delete acquire Bills' workspace
advisory lock (seed 53) before transaction locks sorted by ID. The shared
`lib/db/transaction-guards` module checks allocated evidence without feature
dependency cycles. Pair creation validates both expected states and updates
only flags, pair IDs and timestamps. Detachment retains flags; paired financial
edits are refused; deletion keeps its balance undo and detaches surviving owned
rows. Account deletion keeps its cascade; remaining orphans are reviewed
explicitly. Labels cannot change allocated Bills snapshots before unlinking.

## ARCH-05: Import pipeline

Governed by: ADR-009, ADR-010, ADR-012, ADR-020, ADR-025, ADR-044
Expand Down Expand Up @@ -248,6 +266,11 @@ Governed by: ADR-014, ADR-018, ADR-026, ADR-042
4. The headline, day panel and accessible labels come from the same
results, so the visual and the text can never disagree.

Liquid recorded transfer movements are kept as `balanceOnly` entries for the
calendar's opening and daily balance calculation. They never enter activity,
income, spending or recurring totals. Thus classification changes totals while
preserving the balance line, including transfers posted on different days.

## ARCH-07: AI layer

Governed by: ADR-019, ADR-045, ADR-059
Expand Down Expand Up @@ -304,6 +327,7 @@ Governed by: ADR-006, ADR-007, ADR-010, ADR-012, ADR-019, ADR-025
| Variable | Used by | Secret |
|---|---|---|
| `DATABASE_URL` | Drizzle at runtime, through `pg` (Neon's pooled connection string in production) | Yes |
| `DATABASE_TRANSPORT` | Optional: `websocket` uses Neon's serverless Pool and Drizzle's matching driver where TCP is unavailable. Empty or `tcp` keeps `pg`. Test runners always use `tcp` with local PostgreSQL | No |
| `DATABASE_URL_UNPOOLED` | Drizzle Kit migrations only (Neon's direct connection string); optional, falls back to `DATABASE_URL` | Yes |
| `BETTER_AUTH_SECRET` | Better Auth | Yes |
| `BETTER_AUTH_URL` | Better Auth | No |
Expand Down Expand Up @@ -381,6 +405,12 @@ deletes or rewrites data needs its own ADR.
- **`pnpm db:whoami [--target app|local|branch|production]`** prints the
endpoint id, the database it recognises, the migrations applied and
pending, and the synthetic batch count. It never prints a password.
- **`pnpm db:verify --target app|local|branch|production`** checks the
financial-page schema contract and zero-row SELECT access independently
of journal timestamps. CI runs it against its disposable service after
migrations. The [release procedure](DATABASE_READINESS.md) also requires
the intended runtime role and real-page smoke tests; the CLI does not gate
an independent auto-deployment or repair schema.

**Drizzle's rule:** it applies every migration newer than the newest one
recorded. If a migration is regenerated after it was applied somewhere, that
Expand Down
62 changes: 62 additions & 0 deletions docs/DATABASE_READINESS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
# Database readiness before release

Run `pnpm db:verify --target app|local|branch|production` from the repository
root. It prints the selected endpoint and database without a connection URL,
then checks this checkout's bounded financial-page contract. It does not
repair schema, apply migrations, read financial rows or change `/api/health`.

The contract covers Accounts, Transactions, statement-import references,
Categorization, Recurring payments, Budgets and Bills. Required columns come
from their current Drizzle tables, including transfer flags/pair IDs and Bills
evidence. Workspace/user references are limited to their IDs. Column types
and nullability, valid unconditional uniqueness and validated foreign-key
ownership/reference semantics must match. Constraint names and compatible
additional objects do not matter. This is not a complete database diff or a
validation of every auth, agent or calendar object.

Each required table is checked with a zero-row SELECT of the required columns
under the selected role. Write privileges, row-level-security policies,
defaults, arbitrary check expressions and stored data correctness are not
verified. No INSERT, UPDATE, DELETE or DDL probe runs. Checks use a read-only
repeatable-read transaction, a ten-second connection/client-query timeout,
five-second statement timeout and two-second lock timeout; the client closes
on success and failure. Permission, connection/query and required-schema
failures exit nonzero. A missing or inaccessible migration journal is a
diagnostic warning, as are unrecorded or newer migration timestamps.

For example, 0012 may have recovered `budget` while the original 0010 timestamp
is absent. Compatible schema passes with that journal warning. Conversely,
SELECT 1 and a fully populated journal do not make a missing table ready.
Use `pnpm db:whoami --target <name>` and the migration SQL to investigate;
do not replay migrations or rewrite history based on a warning alone.

## Release sequence

1. Identify the intended database and both runtime/migration connection roles.
`app` selects `DATABASE_URL`; `production` prefers
`DATABASE_URL_UNPOOLED`, which may have a more privileged role. Inspect
the secret-safe target display and use the existing named-target guard
for authorized production writes (ADR-049).
2. If a migration is needed, apply the reviewed migrations before traffic
reaches the new app version. Never infer production success from a local
migration. Migrations continue to require a direct TCP connection.
3. Run `pnpm db:verify --target app` using the intended runtime URL and app
role. A pass with the migration role alone does not establish app access.
A read-only check of the named deployment target is useful additionally.
In this HTTPS-only cloud workspace the verifier can use the user-selected
`DATABASE_TRANSPORT=websocket`; it honors inherited proxy/CA settings.
4. Release and smoke-test the real financial pages with the intended app
connection. Preserve any environment-blocked checks in the handoff.

CI's existing unit and E2E PostgreSQL jobs run the verifier after migration
with `--target app`, selecting their service on port 5432. `--target local`
selects Docker's port 5433. These are disposable CI databases, not production
verification. The CLI does not prevent a separate Vercel auto-deployment:
the team's release process must actually run this check before deployment.
No page gate, public endpoint, startup migration or offline-build database
dependency is introduced.

Fault tests create separate local databases. They never remove shared-suite
tables or production objects, and cover healthy SELECT 1 with missing schema,
column/type/nullability faults, ownership/uniqueness, read permissions,
compatible extras, journal discrepancies and command-level secret redaction.
27 changes: 27 additions & 0 deletions docs/DECISIONS.md
Original file line number Diff line number Diff line change
Expand Up @@ -2325,3 +2325,30 @@ Rules:
delivery are unchanged. No worker, messaging, AI, cron, job, notification
preferences or dashboard integration is implemented here. Those integrations
must explicitly adopt these invoice semantics if they consume the public API.

### ADR-066: Explicit web transfer review and reversible pair relationships

- **Status:** Accepted (2026-10-11), user authorized the supplied implementation plan.
- **Context:** Imported repayments can be counted twice. Transfer labels have
no provenance, and Bills evidence snapshots include the transfer flag.
- **Decision:** Review existing transactions on a Transactions subroute.
Suggest equal/opposite posted amounts on different owned accounts within
three calendar days (ADR-020); the user chooses and confirms every pair.
Review reads never classify automatically; this feature adds no import hook.
Pair exactly two rows atomically.
Remove pairing clears only the relationship and keeps each transfer label;
counting standalone rows as normal activity is a separate explicit action.
Legacy/orphan groups remain visible and require explicit detachment, with
no ownership inferred from pair IDs and no automatic repair on reads.
Classification changes no ledger fact or recorded balance. Serialize writes
with Bills' workspace advisory lock before deterministic transaction locks.
Reject stale/competing forms and flag changes on allocated payment evidence.
Paired financial fields/flags cannot be edited until detachment; deletion
preserves the existing balance undo and detaches surviving owned members.
- **Consequences:** Reuse existing transfer fields with no migration. Removing
pairing alone does not return amounts to spending/income. Bill evidence must
be explicitly unlinked before reclassification; labelled debt repayments
may gain/lose a relationship without changing that evidence. Default review
covers 90 days with explicit bounded older-date/account navigation.
The user approved the focused cashflow correction: liquid transfer amounts
move historical and daily balances without becoming spending or income.
Loading
Loading