diff --git a/.dockerignore b/.dockerignore index 5dbd41ab7..99850d380 100644 --- a/.dockerignore +++ b/.dockerignore @@ -16,3 +16,5 @@ docs coverage __tests__ e2e +src/generated +scripts/dist diff --git a/.env.example b/.env.example index b86a2105d..1b7b732c4 100644 --- a/.env.example +++ b/.env.example @@ -1,6 +1,15 @@ # Note: Copy these variables in your .env file -# DATABASE_URL=file:./dev.db +# Database. The bundled Postgres runs automatically and needs no +# configuration — it is not exposed outside the Docker network. +# +# To use your own Postgres instead, uncomment and set: +# DATABASE_URL=postgres://user:pass@db.example.internal:5432/jobsync +# Optional: also stop the then-unused bundled database container +# BUNDLED_DB=0 +# +# For local development (npm run dev) DATABASE_URL must be set: point it at +# any Postgres you can reach. See CONTRIBUTING.md. # If you are running it on a remote server/homelab, you must update timezone otherwise activities times may shift TZ=America/Edmonton diff --git a/.gitignore b/.gitignore index 69651172d..d621ce381 100644 --- a/.gitignore +++ b/.gitignore @@ -53,6 +53,10 @@ next-env.d.ts /data/ /jobsyncdb/ +# generated: the frozen SQLite snapshot client and the compiled boot scripts +/src/generated/ +/scripts/dist/ + /test-results/ /playwright-report/ /blob-report/ diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 267b75911..29c462cc5 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -44,6 +44,7 @@ This project follows a [Code of Conduct](./CODE_OF_CONDUCT.md). By participating - Node.js 20+ - npm - Git +- A Postgres database (any reachable Postgres 17+; Docker is one way to get one, not a requirement). Prefer one with the `pgvector` extension available — the bundled `pgvector/pgvector:pg18` image has it — since resume embeddings will need it ### Local Setup @@ -65,10 +66,22 @@ This project follows a [Code of Conduct](./CODE_OF_CONDUCT.md). By participating ```bash cp .env.example .env ``` -6. **Set up the database:** +6. **Point the app at a Postgres database.** Set `DATABASE_URL` in `.env`. Either use a Postgres you already have: + ``` + DATABASE_URL=postgres://user:password@localhost:5432/jobsync + ``` + or start only the bundled database from this repo and use it: + ```bash + docker compose -f docker-compose.yml -f docker-compose.lan-db.yml up -d db + ``` + ``` + DATABASE_URL=postgres://jobsync:jobsync@localhost:5432/jobsync + ``` + Then create the schema: ```bash npx prisma migrate dev ``` + The role needs `CREATEDB` so Prisma can make its temporary shadow database (the bundled one has it). Tests need no database: `npm test` mocks Prisma. 7. **Start the development server:** ```bash npm run dev diff --git a/Dockerfile b/Dockerfile index 5388e1ecc..9da2d488b 100644 --- a/Dockerfile +++ b/Dockerfile @@ -17,10 +17,13 @@ WORKDIR /app COPY --from=deps /app/node_modules ./node_modules COPY . . -ENV DATABASE_URL=file:/data/dev.db +# Two clients: the Postgres runtime client, and the frozen SQLite snapshot +# client that only the one-time import reads. Neither needs DATABASE_URL. +RUN npx -y prisma@6.19.0 generate \ + && npx -y prisma@6.19.0 generate --schema=prisma/schema.sqlite-snapshot.prisma -# Generate Prisma client -RUN npx prisma generate +# Boot scripts (wait-for-db, import-sqlite), compiled to plain CommonJS +RUN npx tsc -p scripts/tsconfig.json RUN npm run build @@ -45,6 +48,10 @@ COPY --from=builder /app/public ./public COPY --from=builder /app/prisma ./prisma COPY --from=builder --chown=nextjs:nodejs /app/.next/standalone ./ COPY --from=builder --chown=nextjs:nodejs /app/.next/static ./.next/static +COPY --from=builder /app/scripts/dist ./scripts/dist +# Nothing in the app imports the snapshot client, so Next's standalone trace +# never includes it or its SQLite engine binary: copy it explicitly. +COPY --from=builder /app/src/generated/sqlite-snapshot ./src/generated/sqlite-snapshot COPY docker-entrypoint.sh /app/docker-entrypoint.sh RUN chmod +x /app/docker-entrypoint.sh diff --git a/Jenkinsfile b/Jenkinsfile index 14cd2dc1e..f8bc46c7d 100644 --- a/Jenkinsfile +++ b/Jenkinsfile @@ -114,12 +114,18 @@ pipeline { sh ''' set -e - docker compose \ - -f docker-compose.yml \ - -f docker-compose.otel.yml \ - up -d \ - --force-recreate \ - --remove-orphans + # An explicit -f overrides COMPOSE_FILE, so pass none + # when .env already names the files. + if grep -qE '^[[:space:]]*COMPOSE_FILE=' .env 2>/dev/null; then + docker compose up -d --force-recreate --remove-orphans + else + docker compose \ + -f docker-compose.yml \ + -f docker-compose.otel.yml \ + up -d \ + --force-recreate \ + --remove-orphans + fi ''' } } diff --git a/README.md b/README.md index 193f97171..12cce5bd0 100644 --- a/README.md +++ b/README.md @@ -47,7 +47,7 @@ Track every application, get AI resume reviews and job matches, and discover new - **Contacts:** Keep recruiters, interviewers and references, linked to the jobs they touched. [Details](https://github.com/Gsync/jobsync/wiki/Contacts) - **Company Pages:** One page per company with its jobs, people and links, plus a watchlist. [Details](https://github.com/Gsync/jobsync/wiki/Companies) - **Question Bank:** Build a library of interview questions and answers, tagged by skill. -- **Data Backup & Restore:** Export everything as one zip, import it anywhere, and undo an import. [Details](https://github.com/Gsync/jobsync/wiki/Data-Backup) +- **Data Backup & Restore:** Export everything as one zip, import it anywhere, and undo an import. [Details](https://github.com/Gsync/jobsync/wiki/Backup) - **AI Assistant:** A docked chat that reviews resumes, scores job matches, writes cover letters and adds jobs you paste in. [Details](#ai-assistant-in-app-chat) - **AI Agent Integration (MCP):** Add jobs and questions from Claude Desktop and other agents. [Details](#mcp-server-ai-agent-integration) @@ -69,7 +69,7 @@ API keys for AI providers can be configured in **Settings** after signing in. ### Configuration (Optional) -Environment variables can be set in `docker-compose.yml`: +Environment variables are set in a `.env` file beside `docker-compose.yml` (copy `.env.example`). Do not edit `docker-compose.yml` itself — updates replace it. | Variable | Description | |---|---| @@ -77,10 +77,13 @@ Environment variables can be set in `docker-compose.yml`: | `AUTH_SECRET` | Auto-generated if not set. To set manually: `openssl rand -base64 32` | | `ENCRYPTION_KEY` | Encrypts the API keys you save in Settings. **Set your own** (`openssl rand -base64 32`) before adding any keys, and never change it afterwards — a new value makes every stored API key unrecoverable. | | `NEXTAUTH_URL` | The address you open JobSync at. Defaults to `http://localhost:3737`; on a homelab server, use the server's IP or hostname. | -| `MCP_ENABLED` | Set to `false` to turn off the [MCP server](#mcp-server-ai-agent-integration). Enabled by default in `docker-compose.yml`. | +| `MCP_ENABLED` | Set to `false` to turn off the [MCP server](#mcp-server-ai-agent-integration). Enabled by default. | +| `DATABASE_URL` | Leave unset to use the bundled Postgres. To use your own, set it to your server's address; optionally add `BUNDLED_DB=0` to stop the then-idle bundled container. The bundled image includes the `pgvector` extension, which an upcoming release uses for resume embeddings; your own server will need it then too. See [Upgrading to Postgres](https://github.com/Gsync/jobsync/wiki/Upgrading-To-Postgres). | ### Updating +**Upgrading from a SQLite release?** Read [Upgrading to Postgres](https://github.com/Gsync/jobsync/wiki/Upgrading-To-Postgres) first — the upgrade is one-way. + From the project directory, run the deploy script to pull the latest image and restart: ```sh @@ -226,7 +229,7 @@ We welcome contributions! Please read our [Contributing Guidelines](./CONTRIBUTI - Tailwind - Tiptap - Nivo -- Sqlite +- Postgres - Vercel AI-SDK - Ollama diff --git a/__tests__/activity.actions.spec.ts b/__tests__/activity.actions.spec.ts index e2763e716..a59c07614 100644 --- a/__tests__/activity.actions.spec.ts +++ b/__tests__/activity.actions.spec.ts @@ -117,9 +117,9 @@ describe("activity.actions", () => { userId: mockUser.id, endTime: { not: null }, OR: [ - { activityName: { contains: "TypeScript" } }, - { description: { contains: "TypeScript" } }, - { activityType: { label: { contains: "TypeScript" } } }, + { activityName: { contains: "TypeScript", mode: "insensitive" } }, + { description: { contains: "TypeScript", mode: "insensitive" } }, + { activityType: { label: { contains: "TypeScript", mode: "insensitive" } } }, ], }), }) @@ -128,9 +128,9 @@ describe("activity.actions", () => { expect.objectContaining({ where: expect.objectContaining({ OR: [ - { activityName: { contains: "TypeScript" } }, - { description: { contains: "TypeScript" } }, - { activityType: { label: { contains: "TypeScript" } } }, + { activityName: { contains: "TypeScript", mode: "insensitive" } }, + { description: { contains: "TypeScript", mode: "insensitive" } }, + { activityType: { label: { contains: "TypeScript", mode: "insensitive" } } }, ], }), }) @@ -147,7 +147,7 @@ describe("activity.actions", () => { const findManyCall = (prisma.activity.findMany as any).mock .calls[0][0]; expect(findManyCall.where.OR).toContainEqual({ - activityName: { contains: "Portfolio" }, + activityName: { contains: "Portfolio", mode: "insensitive" }, }); }); @@ -161,7 +161,7 @@ describe("activity.actions", () => { const findManyCall = (prisma.activity.findMany as any).mock .calls[0][0]; expect(findManyCall.where.OR).toContainEqual({ - description: { contains: "developer" }, + description: { contains: "developer", mode: "insensitive" }, }); }); @@ -175,7 +175,7 @@ describe("activity.actions", () => { const findManyCall = (prisma.activity.findMany as any).mock .calls[0][0]; expect(findManyCall.where.OR).toContainEqual({ - activityType: { label: { contains: "Learning" } }, + activityType: { label: { contains: "Learning", mode: "insensitive" } }, }); }); @@ -244,7 +244,7 @@ describe("activity.actions", () => { expect(prisma.activity.findMany).toHaveBeenCalledWith( expect.objectContaining({ - orderBy: { createdAt: "desc" }, + orderBy: [{ createdAt: "desc" }, { id: "desc" }], }) ); }); diff --git a/__tests__/backupImportGuards.spec.ts b/__tests__/backupImportGuards.spec.ts index b25992f21..da14523f2 100644 --- a/__tests__/backupImportGuards.spec.ts +++ b/__tests__/backupImportGuards.spec.ts @@ -106,6 +106,20 @@ describe("importBackup guards", () => { expect(syncSchedulerState).toHaveBeenCalled(); }); + it("strips NULs a SQLite-era backup can carry before inserting", async () => { + const data = BackupDataSchema.parse({ + Company: [{ id: "c1", label: "Acme\u0000 Corp", value: "acme corp", logoUrl: null }], + }); + const zip = new JSZip(); + zip.file("manifest.json", JSON.stringify(buildManifest(data, EMAIL))); + zip.file("data.json", JSON.stringify(data)); + await importBackup(await zip.generateAsync({ type: "nodebuffer" }), "user-1", EMAIL, true); + + expect(mockDb.company.create).toHaveBeenCalledWith({ + data: expect.objectContaining({ label: "Acme Corp" }), + }); + }); + it("wipes ChatConversation but never ApiKey or McpAccessToken", async () => { await importBackup(await emptyBackup(), "user-1", EMAIL, true); expect(mockDb.chatConversation.deleteMany).toHaveBeenCalled(); diff --git a/__tests__/backupRoundTrip.spec.ts b/__tests__/backupRoundTrip.spec.ts index 133c21ed7..8d622ba05 100644 --- a/__tests__/backupRoundTrip.spec.ts +++ b/__tests__/backupRoundTrip.spec.ts @@ -6,18 +6,19 @@ import { APP_CONSTANTS, CONTACT_ROLES, JOB_STAGES } from "@/lib/constants"; import { buildBackupZip } from "@/lib/backup/export"; import { importBackup } from "@/lib/backup/import"; import { listSnapshots, readSnapshot } from "@/lib/backup/snapshot"; -import { seedAccount } from "./helpers/backupTestDb"; +import { dropTestSchema, seedAccount } from "./helpers/backupTestDb"; // Async so the helper can be pulled in with a dynamic import — `require` is // not defined in a Vite-transformed ESM test file. const ctx = await vi.hoisted(async () => { const helpers = await import("./helpers/backupTestDb"); - const { url, dir } = helpers.makeTestDbUrl(); - helpers.pushSchema(url); - return { url, dir }; + const db = helpers.makeTestDb(); + if (db) helpers.pushSchema(db.url); + return db; }); vi.mock("@/lib/db", async () => { + if (!ctx) return { default: {} }; const { PrismaClient } = await import("@prisma/client"); return { default: new PrismaClient({ datasources: { db: { url: ctx.url } } }), @@ -27,13 +28,17 @@ vi.mock("@/lib/db", async () => { // Never start the cron in a test run. vi.mock("@/lib/scheduler", () => ({ syncSchedulerState: vi.fn() })); -const prisma = new PrismaClient({ datasources: { db: { url: ctx.url } } }); +// Only dereferenced inside the skipped-when-absent describe below +const prisma = ctx + ? new PrismaClient({ datasources: { db: { url: ctx.url } } }) + : (undefined as unknown as PrismaClient); let userId: string; let uploadsDir: string; const originalUploads = APP_CONSTANTS.UPLOADS_DIR; beforeAll(async () => { + if (!ctx) return; uploadsDir = path.join(ctx.dir, "uploads"); fs.mkdirSync(path.join(uploadsDir, "files", "resumes"), { recursive: true }); (APP_CONSTANTS as { UPLOADS_DIR: string }).UPLOADS_DIR = uploadsDir; @@ -43,7 +48,9 @@ beforeAll(async () => { }, 120_000); afterAll(async () => { + if (!ctx) return; (APP_CONSTANTS as { UPLOADS_DIR: string }).UPLOADS_DIR = originalUploads; + await dropTestSchema(prisma, ctx.schema); await prisma.$disconnect(); fs.rmSync(ctx.dir, { recursive: true, force: true }); }); @@ -229,7 +236,7 @@ async function seedFullAccount() { }); } -describe("backup round trip", () => { +describe.skipIf(!ctx)("backup round trip", () => { it("restores every row, relation and file over an existing account", async () => { const { buffer } = await buildBackupZip(userId, "owner@example.com"); diff --git a/__tests__/company.actions.spec.ts b/__tests__/company.actions.spec.ts index 0029a0c6f..066947891 100644 --- a/__tests__/company.actions.spec.ts +++ b/__tests__/company.actions.spec.ts @@ -79,7 +79,7 @@ describe("Company Actions", () => { where: { createdBy: mockUser.id }, skip: 0, take: 10, - orderBy: [{ jobsApplied: { _count: "desc" } }, { label: "asc" }], + orderBy: [{ jobsApplied: { _count: "desc" } }, { label: "asc" }, { id: "desc" }], }); expect(prisma.company.count).toHaveBeenCalledWith({ where: { createdBy: mockUser.id }, @@ -150,7 +150,7 @@ describe("Company Actions", () => { }, }, }, - orderBy: [{ jobsApplied: { _count: "desc" } }, { label: "asc" }], + orderBy: [{ jobsApplied: { _count: "desc" } }, { label: "asc" }, { id: "desc" }], }); expect(prisma.company.count).toHaveBeenCalledWith({ where: { createdBy: mockUser.id }, @@ -191,20 +191,20 @@ describe("Company Actions", () => { where: { createdBy: mockUser.id, OR: [ - { label: { contains: "Ama" } }, - { atsToken: { contains: "Ama" } }, + { label: { contains: "Ama", mode: "insensitive" } }, + { atsToken: { contains: "Ama", mode: "insensitive" } }, ], }, skip: 0, take: 10, - orderBy: [{ jobsApplied: { _count: "desc" } }, { label: "asc" }], + orderBy: [{ jobsApplied: { _count: "desc" } }, { label: "asc" }, { id: "desc" }], }); expect(prisma.company.count).toHaveBeenCalledWith({ where: { createdBy: mockUser.id, OR: [ - { label: { contains: "Ama" } }, - { atsToken: { contains: "Ama" } }, + { label: { contains: "Ama", mode: "insensitive" } }, + { atsToken: { contains: "Ama", mode: "insensitive" } }, ], }, }); @@ -237,8 +237,8 @@ describe("Company Actions", () => { where: { createdBy: mockUser.id, OR: [ - { label: { contains: "Ama" } }, - { atsToken: { contains: "Ama" } }, + { label: { contains: "Ama", mode: "insensitive" } }, + { atsToken: { contains: "Ama", mode: "insensitive" } }, ], }, skip: 0, @@ -268,14 +268,14 @@ describe("Company Actions", () => { }, }, }, - orderBy: [{ jobsApplied: { _count: "desc" } }, { label: "asc" }], + orderBy: [{ jobsApplied: { _count: "desc" } }, { label: "asc" }, { id: "desc" }], }); expect(prisma.company.count).toHaveBeenCalledWith({ where: { createdBy: mockUser.id, OR: [ - { label: { contains: "Ama" } }, - { atsToken: { contains: "Ama" } }, + { label: { contains: "Ama", mode: "insensitive" } }, + { atsToken: { contains: "Ama", mode: "insensitive" } }, ], }, }); @@ -291,7 +291,11 @@ describe("Company Actions", () => { expect(prisma.company.findMany).toHaveBeenCalledWith( expect.objectContaining({ where: { createdBy: mockUser.id, watched: true }, - orderBy: [{ watchedAt: "desc" }, { label: "asc" }], + orderBy: [ + { watchedAt: { sort: "desc", nulls: "last" } }, + { label: "asc" }, + { id: "desc" }, + ], }) ); }); @@ -307,7 +311,7 @@ describe("Company Actions", () => { where: { createdBy: mockUser.id }, skip: 0, take: 10, - orderBy: [{ jobsApplied: { _count: "desc" } }, { label: "asc" }], + orderBy: [{ jobsApplied: { _count: "desc" } }, { label: "asc" }, { id: "desc" }], }); }); }); @@ -327,6 +331,7 @@ describe("Company Actions", () => { expect(result).toEqual(mockCompanies); expect(prisma.company.findMany).toHaveBeenCalledWith({ where: { createdBy: mockUser.id }, + orderBy: [{ label: "asc" }, { id: "asc" }], }); }); @@ -352,6 +357,7 @@ describe("Company Actions", () => { expect(result).toEqual({ success: false, message: "Unexpected error" }); expect(prisma.company.findMany).toHaveBeenCalledWith({ where: { createdBy: mockUser.id }, + orderBy: [{ label: "asc" }, { id: "asc" }], }); }); }); diff --git a/__tests__/contact.actions.spec.ts b/__tests__/contact.actions.spec.ts index 462097150..4cb330f61 100644 --- a/__tests__/contact.actions.spec.ts +++ b/__tests__/contact.actions.spec.ts @@ -51,9 +51,9 @@ describe("contact actions", () => { it("searches name, email and title", async () => { await getContactList(1, 10, "pat"); expect(db.contact.findMany.mock.calls[0][0].where.OR).toEqual([ - { name: { contains: "pat" } }, - { email: { contains: "pat" } }, - { title: { contains: "pat" } }, + { name: { contains: "pat", mode: "insensitive" } }, + { email: { contains: "pat", mode: "insensitive" } }, + { title: { contains: "pat", mode: "insensitive" } }, ]); }); diff --git a/__tests__/contactRole.actions.spec.ts b/__tests__/contactRole.actions.spec.ts index bdbec9376..a13330c3d 100644 --- a/__tests__/contactRole.actions.spec.ts +++ b/__tests__/contactRole.actions.spec.ts @@ -44,7 +44,7 @@ describe("contactRole actions", () => { skip: 0, take: 10, include: { _count: { select: { jobContacts: true, contacts: true } } }, - orderBy: [{ label: "asc" }], + orderBy: [{ label: "asc" }, { id: "desc" }], }), ); }); @@ -57,7 +57,7 @@ describe("contactRole actions", () => { expect(db.contactRole.findMany).toHaveBeenCalledWith( expect.objectContaining({ - where: { createdBy: user.id, label: { contains: "recr" } }, + where: { createdBy: user.id, label: { contains: "recr", mode: "insensitive" } }, }), ); }); diff --git a/__tests__/coverLetter.actions.spec.ts b/__tests__/coverLetter.actions.spec.ts index 897646ebb..4f095cbfe 100644 --- a/__tests__/coverLetter.actions.spec.ts +++ b/__tests__/coverLetter.actions.spec.ts @@ -85,7 +85,7 @@ describe("coverLetterActions", () => { select: { Job: true }, }, }, - orderBy: { createdAt: "desc" }, + orderBy: [{ createdAt: "desc" }, { id: "desc" }], }); expect(prisma.coverLetter.count).toHaveBeenCalledWith({ where: { diff --git a/__tests__/dashboard.actions.spec.ts b/__tests__/dashboard.actions.spec.ts index c25ea81aa..ff91195ae 100644 --- a/__tests__/dashboard.actions.spec.ts +++ b/__tests__/dashboard.actions.spec.ts @@ -72,9 +72,10 @@ describe("Dashboard Actions", () => { Status: true, Location: true, }, - orderBy: { - appliedDate: "desc", - }, + orderBy: [ + { appliedDate: { sort: "desc", nulls: "last" } }, + { id: "desc" }, + ], take: APP_CONSTANTS.RECENT_NUM_JOBS_ACTIVITIES, }); }); diff --git a/__tests__/helpers/backupTestDb.ts b/__tests__/helpers/backupTestDb.ts index 4838c1ff0..f6f9070f1 100644 --- a/__tests__/helpers/backupTestDb.ts +++ b/__tests__/helpers/backupTestDb.ts @@ -11,9 +11,16 @@ import { JOB_STATUSES, } from "@/lib/constants"; -export function makeTestDbUrl(): { url: string; dir: string } { +// Opt-in: CI has no database, so without this variable the spec is skipped +export function makeTestDb(): { url: string; schema: string; dir: string } | null { + const base = process.env.BACKUP_TEST_DATABASE_URL; + if (!base) return null; + const schema = `backup_test_${randomUUID().replace(/-/g, "").slice(0, 12)}`; + const url = new URL(base); + // A schema of its own, so the run never touches the database's real tables + url.searchParams.set("schema", schema); const dir = fs.mkdtempSync(path.join(os.tmpdir(), "jobsync-backup-")); - return { url: `file:${path.join(dir, "test.db")}`, dir }; + return { url: url.toString(), schema, dir }; } // db push rather than migrate deploy: the round-trip asserts against the @@ -26,6 +33,13 @@ export function pushSchema(url: string): void { ); } +export async function dropTestSchema( + prisma: PrismaClient, + schema: string, +): Promise { + await prisma.$executeRawUnsafe(`DROP SCHEMA IF EXISTS "${schema}" CASCADE`); +} + // Mirrors what signup() seeds, so "empty account" means the same thing here as // it does in the app. export async function seedAccount( diff --git a/__tests__/importCopy.spec.ts b/__tests__/importCopy.spec.ts new file mode 100644 index 000000000..229579925 --- /dev/null +++ b/__tests__/importCopy.spec.ts @@ -0,0 +1,142 @@ +import { copyAll, type CopyClient } from "../scripts/import/copy"; +import type { ImportPlan } from "../scripts/import/order"; + +type Row = Record & { id: string }; + +function memoryClient(tables: Record, joins: Record = {}) { + const calls: { op: string; table: string; size?: number }[] = []; + const delegate = (name: string) => ({ + findMany: async ({ take, cursor }: { take: number; cursor?: { id: string } }) => { + const rows = [...(tables[name] ?? [])].sort((x, y) => (x.id < y.id ? -1 : 1)); + const start = cursor ? rows.findIndex((r) => r.id === cursor.id) + 1 : 0; + return rows.slice(start, start + take); + }, + count: async () => (tables[name] ?? []).length, + createMany: async ({ data, skipDuplicates }: { data: Row[]; skipDuplicates: boolean }) => { + expect(skipDuplicates).toBe(true); + calls.push({ op: "createMany", table: name, size: data.length }); + const existing = (tables[name] ??= []); + for (const row of data) if (!existing.some((r) => r.id === row.id)) existing.push({ ...row }); + }, + update: async ({ where, data }: { where: { id: string }; data: Row }) => { + calls.push({ op: "update", table: name }); + Object.assign(tables[name].find((r) => r.id === where.id)!, data); + }, + }); + const client: CopyClient = new Proxy( + { + $queryRawUnsafe: async (sql: string) => { + const table = /"(_\w+)"/.exec(sql)![1]; + if (sql.startsWith("SELECT COUNT")) return [{ n: BigInt((joins[table] ?? []).length) }]; + return joins[table] ?? []; + }, + $executeRawUnsafe: async (sql: string, ...values: string[]) => { + expect(sql).toContain("ON CONFLICT DO NOTHING"); + const table = /"(_\w+)"/.exec(sql)![1]; + calls.push({ op: "insertJoin", table, size: values.length / 2 }); + const existing = (joins[table] ??= []); + for (let i = 0; i < values.length; i += 2) { + const pair = { A: values[i], B: values[i + 1] }; + if (!existing.some((p) => p.A === pair.A && p.B === pair.B)) existing.push(pair); + } + return values.length / 2; + }, + } as Record, + { get: (target, key: string) => target[key] ?? (target[key] = delegate(key)) }, + ); + return { client, tables, joins, calls }; +} + +const plan: ImportPlan = { + order: ["User", "Resume", "Tag", "Job"], + joinTables: [{ table: "_JobToTag", a: "Job", b: "Tag" }], + deferred: [{ model: "User", field: "defaultResumeId" }], +}; + +const sourceData = () => + memoryClient( + { + user: [{ id: "u1", email: "a@b.c", defaultResumeId: "r1" }, { id: "u2", email: "d@e.f", defaultResumeId: null }], + resume: [{ id: "r1", title: "CV" }], + tag: [{ id: "t1", label: "Remote" }], + job: [ + { id: "j1", description: "one" }, + { id: "j2", description: "bad\u0000byte" }, + { id: "j3", description: "three" }, + ], + }, + { _JobToTag: [{ A: "j1", B: "t1" }, { A: "j2", B: "t1" }, { A: "j3", B: "t1" }] }, + ); + +describe("copyAll", () => { + it("copies every table and join table and reports matching counts", async () => { + const source = sourceData(); + const target = memoryClient({}); + const result = await copyAll(source.client, target.client, plan, () => {}); + expect(result.tables).toEqual([ + { name: "User", source: 2, target: 2 }, + { name: "Resume", source: 1, target: 1 }, + { name: "Tag", source: 1, target: 1 }, + { name: "Job", source: 3, target: 3 }, + { name: "_JobToTag", source: 3, target: 3 }, + ]); + }); + + it("inserts the deferred column as null, then sets it after Resume lands", async () => { + const source = sourceData(); + const target = memoryClient({}); + await copyAll(source.client, target.client, plan, () => {}); + const order = target.calls.map((c) => `${c.op}:${c.table}`); + expect(order.indexOf("update:user")).toBeGreaterThan(order.indexOf("createMany:resume")); + expect(target.calls.filter((c) => c.op === "update")).toHaveLength(1); + expect(target.tables.user.find((u) => u.id === "u1")!.defaultResumeId).toBe("r1"); + expect(target.tables.user.find((u) => u.id === "u2")!.defaultResumeId).toBeNull(); + }); + + it("chunks reads, writes and join inserts by the batch size", async () => { + const source = sourceData(); + const target = memoryClient({}); + await copyAll(source.client, target.client, plan, () => {}, 2); + expect(target.calls.filter((c) => c.table === "job").map((c) => c.size)).toEqual([2, 1]); + expect(target.calls.filter((c) => c.op === "insertJoin").map((c) => c.size)).toEqual([2, 1]); + }); + + it("sanitises strings and logs each value touched", async () => { + const source = sourceData(); + const target = memoryClient({}); + const log: string[] = []; + const result = await copyAll(source.client, target.client, plan, (line) => log.push(line)); + expect(target.tables.job.find((j) => j.id === "j2")!.description).toBe("badbyte"); + expect(result.sanitized).toBe(1); + expect(log).toContain("Sanitised Job.description on row j2 (removed characters Postgres cannot store)"); + }); + + it("inserts nothing new when run a second time", async () => { + const source = sourceData(); + const target = memoryClient({}); + await copyAll(source.client, target.client, plan, () => {}); + const result = await copyAll(source.client, target.client, plan, () => {}); + expect(result.tables.every((t) => t.source === t.target)).toBe(true); + expect(target.tables.job).toHaveLength(3); + expect(target.joins._JobToTag).toHaveLength(3); + }); + + it("aborts on a count mismatch, naming the table", async () => { + const source = sourceData(); + const target = memoryClient({ tag: [{ id: "stray", label: "Left over" }] }); + await expect(copyAll(source.client, target.client, plan, () => {})).rejects.toThrow( + /Row counts differ after the copy: Tag \(1 in SQLite, 2 in Postgres\)/, + ); + }); + + it("names the model when an insert fails", async () => { + const source = sourceData(); + const target = memoryClient({}); + target.client.job.createMany = async () => { + throw new Error("Foreign key constraint violated: Job_companyId_fkey"); + }; + await expect(copyAll(source.client, target.client, plan, () => {})).rejects.toThrow( + "Import failed while copying Job: Foreign key constraint violated: Job_companyId_fkey", + ); + }); +}); diff --git a/__tests__/importDecide.spec.ts b/__tests__/importDecide.spec.ts new file mode 100644 index 000000000..026b33656 --- /dev/null +++ b/__tests__/importDecide.spec.ts @@ -0,0 +1,128 @@ +import fs from "fs"; +import path from "path"; +import { + LAST_SQLITE_MIGRATION, + checkLegacySchema, + decideImport, + sentinelPath, + type ImportFacts, +} from "../scripts/import/decide"; + +const facts = (over: Partial = {}): ImportFacts => ({ + legacyPath: "/data/dev.db", + legacyFileExists: true, + skipImport: false, + postgresHasUsers: false, + legacyHasUsers: true, + sentinelExists: false, + sentinelText: "", + ...over, +}); + +describe("decideImport", () => { + it("skips a fresh install with no legacy file", () => { + const d = decideImport(facts({ legacyFileExists: false })); + expect(d).toEqual({ + action: "skip", + message: "No legacy SQLite database found at /data/dev.db, skipping import.", + }); + }); + + it("imports when Postgres is empty and there is no sentinel", () => { + expect(decideImport(facts())).toEqual({ action: "import" }); + }); + + it("skips on a normal boot: data and sentinel both present", () => { + const d = decideImport(facts({ postgresHasUsers: true, sentinelExists: true })); + expect(d.action).toBe("skip"); + }); + + it("stops when the volume was lost after an import", () => { + const d = decideImport( + facts({ sentinelExists: true, sentinelText: "2026-11-02T14:02:11.000Z\n" }), + ); + expect(d.action).toBe("stop"); + const message = (d as { message: string }).message; + expect(message).toContain("dev.db was already imported on 2026-11-02 14:02"); + expect(message).toContain("./restore-db.sh "); + expect(message).toContain("delete jobsyncdb/data/dev.db.imported-at"); + expect(message).toContain("Nothing has been modified. Exiting."); + }); + + it("skips instead of stopping when the imported dev.db had no users", () => { + const d = decideImport(facts({ sentinelExists: true, legacyHasUsers: false })); + expect(d).toEqual({ + action: "skip", + message: "/data/dev.db was already imported and has no users, skipping import.", + }); + }); + + it.each(["", " \n", "not a date"])("counts sentinel text %j as present", (sentinelText) => { + const d = decideImport(facts({ sentinelExists: true, sentinelText })); + expect(d.action).toBe("stop"); + expect((d as { message: string }).message).toContain("already imported on an earlier date"); + }); + + it("stops when Postgres has data but nothing was confirmed imported", () => { + const d = decideImport(facts({ postgresHasUsers: true })); + expect(d.action).toBe("stop"); + const message = (d as { message: string }).message; + expect(message).toContain("create an empty file named"); + expect(message).toContain("jobsyncdb/data/dev.db.imported-at"); + expect(message).toContain("empty the Postgres database, restart."); + expect(message).toContain("docker volume rm jobsync_pgdata"); + expect(message).not.toContain("SKIP_IMPORT"); + }); + + it("names the stranded state when SKIP_IMPORT hides an empty database", () => { + const d = decideImport(facts({ skipImport: true })); + expect(d.action).toBe("skip"); + const message = (d as { message: string }).message; + expect(message).toContain("SKIP_IMPORT=1 is set, so the SQLite import was NOT attempted."); + expect(message).toContain("Postgres is empty and /data/dev.db exists"); + expect(message).toContain("Remove SKIP_IMPORT from .env"); + }); + + it("lets SKIP_IMPORT pass both stop states, still loudly", () => { + for (const over of [{ sentinelExists: true }, { postgresHasUsers: true }]) { + const d = decideImport(facts({ skipImport: true, ...over })); + expect(d.action).toBe("skip"); + expect((d as { message: string }).message).toContain("SKIP_IMPORT=1 is set"); + } + }); + + it("shows a custom legacy path as given", () => { + const d = decideImport( + facts({ legacyPath: "/mnt/old/app.db", sentinelExists: true }), + ); + expect((d as { message: string }).message).toContain("delete /mnt/old/app.db.imported-at"); + }); + + it("derives the sentinel beside the legacy file", () => { + expect(sentinelPath("/data/dev.db")).toBe("/data/dev.db.imported-at"); + }); +}); + +describe("checkLegacySchema", () => { + it("accepts a database that has the last SQLite migration", () => { + expect(checkLegacySchema(["20240524022540_init", LAST_SQLITE_MIGRATION])).toBeNull(); + }); + + it("refuses a database from an older release", () => { + const message = checkLegacySchema(["20240524022540_init"]); + expect(message).toContain("older JobSync release"); + expect(message).toContain(LAST_SQLITE_MIGRATION); + expect(message).toContain( + 'docker run --rm -v "$(pwd)/jobsyncdb/data:/data" -e DATABASE_URL=file:/data/dev.db ghcr.io/gsync/jobsync:1.1.22', + ); + expect(message).toContain("Nothing has been modified. Exiting."); + }); + + it("names the newest migration in the retired history", () => { + const dirs = fs + .readdirSync(path.resolve(__dirname, "../prisma/migrations-sqlite-legacy")) + .filter((name) => /^\d/.test(name)) + .sort(); + expect(LAST_SQLITE_MIGRATION).toBe(dirs[dirs.length - 1]); + }); +}); diff --git a/__tests__/importOrder.spec.ts b/__tests__/importOrder.spec.ts new file mode 100644 index 000000000..3e7c274ea --- /dev/null +++ b/__tests__/importOrder.spec.ts @@ -0,0 +1,100 @@ +import { Prisma } from "@prisma/client"; +import { + buildImportPlan, + delegateName, + type DmmfModel, +} from "../scripts/import/order"; + +const models = Prisma.dmmf.datamodel.models as unknown as DmmfModel[]; + +const model = (name: string, fields: DmmfModel["fields"] = []): DmmfModel => ({ + name, + fields: [{ name: "id", kind: "scalar", type: "String", isList: false, isId: true }, ...fields], +}); +const fk = (name: string, type: string, from: string) => ({ + name, + kind: "object", + type, + isList: false, + relationName: `${name}Rel`, + relationFromFields: [from], +}); + +describe("buildImportPlan on the real schema", () => { + const plan = buildImportPlan(models); + + it("covers every model exactly once", () => { + expect([...plan.order].sort()).toEqual(models.map((m) => m.name).sort()); + }); + + it("places every foreign-key target before its holder", () => { + for (const m of models) { + for (const field of m.fields) { + const from = field.relationFromFields ?? []; + if (field.kind !== "object" || from.length === 0) continue; + if (m.name === "User" && from.includes("defaultResumeId")) continue; + expect(plan.order.indexOf(field.type)).toBeLessThan(plan.order.indexOf(m.name)); + } + } + }); + + it("defers only User.defaultResumeId", () => { + expect(plan.deferred).toEqual([{ model: "User", field: "defaultResumeId" }]); + expect(plan.order.indexOf("User")).toBeLessThan(plan.order.indexOf("Resume")); + }); + + it("finds both implicit join tables with Prisma's A/B order", () => { + expect(plan.joinTables).toEqual( + expect.arrayContaining([ + { table: "_JobToTag", a: "Job", b: "Tag" }, + { table: "_QuestionToTag", a: "Question", b: "Tag" }, + ]), + ); + expect(plan.joinTables).toHaveLength(2); + }); + + it("does not treat Tag.skills as a join table", () => { + expect(plan.joinTables.map((j) => j.table)).not.toContain("_SkillToTag"); + }); + + it("can cursor every model on a string id", () => { + for (const m of models) { + const id = m.fields.find((f) => f.isId); + expect(id, m.name).toMatchObject({ name: "id", type: "String" }); + } + }); + + it("maps model names to client delegates", () => { + expect(delegateName("JobStageType")).toBe("jobStageType"); + expect(delegateName("AiCall")).toBe("aiCall"); + }); +}); + +describe("buildImportPlan guards", () => { + it("throws on a second cycle instead of guessing", () => { + const cyclic = [model("A", [fk("b", "B", "bId")]), model("B", [fk("a", "A", "aId")])]; + expect(() => buildImportPlan(cyclic)).toThrow(/cycle among: A, B/); + }); + + it("throws on a self-referencing foreign key", () => { + expect(() => buildImportPlan([model("Node", [fk("parent", "Node", "parentId")])])).toThrow( + /Self-referencing foreign key on Node\.parent/, + ); + }); + + it("picks up a third implicit many-to-many", () => { + const list = (name: string, type: string) => ({ + name, + kind: "object", + type, + isList: true, + relationName: "ContactToTag", + relationFromFields: [], + }); + const plan = buildImportPlan([ + model("Tag", [list("contacts", "Contact")]), + model("Contact", [list("tags", "Tag")]), + ]); + expect(plan.joinTables).toEqual([{ table: "_ContactToTag", a: "Contact", b: "Tag" }]); + }); +}); diff --git a/__tests__/importRun.spec.ts b/__tests__/importRun.spec.ts new file mode 100644 index 000000000..bee51a9e1 --- /dev/null +++ b/__tests__/importRun.spec.ts @@ -0,0 +1,172 @@ +import { LAST_SQLITE_MIGRATION } from "../scripts/import/decide"; +import { runImport, type ImportDeps } from "../scripts/import/run"; + +const MODELS = [ + { name: "User", fields: [{ name: "id", kind: "scalar", type: "String", isList: false, isId: true }] }, +]; + +function harness(over: { + env?: Record; + files?: Record; + targetUsers?: number; + sourceUsers?: { id: string }[]; + migrations?: string[]; + failCopy?: boolean; + failWrite?: boolean; +} = {}) { + const files: Record = { "/data/dev.db": "sqlite", ...over.files }; + const log: string[] = []; + const targetRows: { id: string }[] = Array.from({ length: over.targetUsers ?? 0 }, (_, i) => ({ id: `t${i}` })); + const sourceRows = over.sourceUsers ?? [{ id: "u1" }, { id: "u2" }]; + let committed = false; + let opened = 0; + + const userDelegate = (rows: { id: string }[]) => ({ + count: async () => rows.length, + findMany: async ({ cursor }: { cursor?: { id: string } }) => (cursor ? [] : rows), + createMany: async ({ data }: { data: { id: string }[] }) => { + if (over.failCopy) throw new Error("Foreign key constraint violated"); + rows.push(...data); + }, + }); + + const target: any = { + user: userDelegate(targetRows), + $transaction: async (fn: (tx: any) => Promise, options: unknown) => { + expect(options).toEqual({ timeout: 300_000, maxWait: 300_000 }); + const before = [...targetRows]; + try { + const result = await fn(target); + committed = true; + return result; + } catch (error) { + targetRows.length = 0; + targetRows.push(...before); + throw error; + } + }, + }; + + const deps: ImportDeps = { + env: over.env ?? {}, + fileExists: (p) => p in files, + readText: (p) => files[p], + writeText: (p, text) => { + if (over.failWrite) throw new Error("EACCES: permission denied"); + files[p] = text; + }, + openSource: (url) => { + opened++; + expect(url).toBe(`file:${over.env?.SQLITE_LEGACY_PATH ?? "/data/dev.db"}`); + return { + user: userDelegate(sourceRows), + $queryRawUnsafe: async () => + (over.migrations ?? [LAST_SQLITE_MIGRATION]).map((migration_name) => ({ migration_name })), + $disconnect: async () => {}, + }; + }, + target, + models: MODELS, + log: (line) => log.push(line), + now: () => new Date("2026-11-02T14:02:11.000Z"), + }; + + return { deps, files, log, targetRows, opened: () => opened, committed: () => committed }; +} + +describe("runImport", () => { + it("imports, reports per table and writes the sentinel", async () => { + const h = harness(); + expect(await runImport(h.deps)).toBe(0); + expect(h.targetRows).toHaveLength(2); + expect(h.log).toContain("User: 2 → 2 ✓"); + expect(h.log.some((l) => l.startsWith("Import complete: 2 rows across 1 tables"))).toBe(true); + expect(h.files["/data/dev.db.imported-at"]).toBe("2026-11-02T14:02:11.000Z\n"); + expect(h.files["/data/dev.db"]).toBe("sqlite"); + }); + + it("skips a fresh install without opening anything", async () => { + const h = harness({ files: {} }); + delete h.files["/data/dev.db"]; + expect(await runImport(h.deps)).toBe(0); + expect(h.opened()).toBe(0); + }); + + it("is a no-op on a normal boot", async () => { + const h = harness({ targetUsers: 3, files: { "/data/dev.db.imported-at": "2026-11-02T14:02:11.000Z\n" } }); + expect(await runImport(h.deps)).toBe(0); + expect(h.opened()).toBe(0); + expect(h.targetRows).toHaveLength(3); + }); + + it("stops on a lost volume", async () => { + const h = harness({ files: { "/data/dev.db.imported-at": "2026-11-02T14:02:11.000Z\n" } }); + expect(await runImport(h.deps)).toBe(1); + expect(h.log.join("\n")).toContain("The database it was imported into appears to be missing."); + expect(h.opened()).toBe(1); + }); + + it("boots when the imported dev.db had no users", async () => { + const h = harness({ sourceUsers: [], files: { "/data/dev.db.imported-at": "2026-11-02T14:02:11.000Z\n" } }); + expect(await runImport(h.deps)).toBe(0); + expect(h.log.join("\n")).toContain("has no users, skipping import."); + expect(h.targetRows).toHaveLength(0); + }); + + it("stops on data with no sentinel", async () => { + const h = harness({ targetUsers: 1 }); + expect(await runImport(h.deps)).toBe(1); + expect(h.log.join("\n")).toContain("never confirmed imported"); + }); + + it("boots past everything with SKIP_IMPORT, loudly", async () => { + const h = harness({ env: { SKIP_IMPORT: "1" } }); + expect(await runImport(h.deps)).toBe(0); + expect(h.log.join("\n")).toContain("has not been migrated"); + expect(h.targetRows).toHaveLength(0); + }); + + it("honours SQLITE_LEGACY_PATH for the file and the sentinel", async () => { + const h = harness({ env: { SQLITE_LEGACY_PATH: "/mnt/old.db" }, files: { "/mnt/old.db": "sqlite" } }); + expect(await runImport(h.deps)).toBe(0); + expect(h.files["/mnt/old.db.imported-at"]).toBeDefined(); + }); + + it("stops on an older legacy schema", async () => { + const h = harness({ migrations: ["20240524022540_init"] }); + expect(await runImport(h.deps)).toBe(1); + expect(h.log.join("\n")).toContain("older JobSync release"); + expect(h.targetRows).toHaveLength(0); + expect(h.files["/data/dev.db.imported-at"]).toBeUndefined(); + }); + + it("writes no sentinel when the copy fails", async () => { + const h = harness({ failCopy: true }); + expect(await runImport(h.deps)).toBe(1); + expect(h.targetRows).toHaveLength(0); + expect(h.files["/data/dev.db.imported-at"]).toBeUndefined(); + const text = h.log.join("\n"); + expect(text).toContain("Import failed while copying User"); + expect(text).toContain("Postgres was left empty and dev.db was not modified."); + }); + + it("dry run rolls back and exits 2", async () => { + const h = harness({ env: { IMPORT_DRY_RUN: "1" } }); + expect(await runImport(h.deps)).toBe(2); + expect(h.targetRows).toHaveLength(0); + expect(h.committed()).toBe(false); + expect(h.files["/data/dev.db.imported-at"]).toBeUndefined(); + const text = h.log.join("\n"); + expect(text).toContain("User: 2 → 2 ✓"); + expect(text).toContain("Dry run complete — nothing was written."); + }); + + it("reports an unwritable sentinel", async () => { + const h = harness({ failWrite: true }); + expect(await runImport(h.deps)).toBe(1); + expect(h.targetRows).toHaveLength(2); + const text = h.log.join("\n"); + expect(text).toContain("The import succeeded, but its record could not be written"); + expect(text).toContain("jobsyncdb/data/dev.db.imported-at"); + }); +}); diff --git a/__tests__/importSanitize.spec.ts b/__tests__/importSanitize.spec.ts new file mode 100644 index 000000000..98b79501d --- /dev/null +++ b/__tests__/importSanitize.spec.ts @@ -0,0 +1,35 @@ +import { sanitizeRow, sanitizeString } from "../scripts/import/sanitize"; + +describe("sanitizeString", () => { + it("strips NUL", () => { + expect(sanitizeString("a\u0000b")).toBe("ab"); + }); + + it("strips unpaired surrogates on either side", () => { + expect(sanitizeString("a\uD83Db")).toBe("ab"); + expect(sanitizeString("a\uDE00b")).toBe("ab"); + }); + + it("keeps a valid surrogate pair", () => { + expect(sanitizeString("ok 😀")).toBe("ok 😀"); + }); +}); + +describe("sanitizeRow", () => { + it("returns the same object and reports nothing when clean", () => { + const row = { id: "1", title: "Engineer", applied: true, createdAt: new Date(0) }; + const touched: string[] = []; + expect(sanitizeRow(row, (c) => touched.push(c))).toBe(row); + expect(touched).toEqual([]); + }); + + it("cleans string columns only and names each one", () => { + const createdAt = new Date(0); + const row = { id: "1", description: "pdf\u0000text", notes: "x\uD83D", salary: null, createdAt }; + const touched: string[] = []; + const clean = sanitizeRow(row, (c) => touched.push(c)); + expect(clean).toEqual({ id: "1", description: "pdftext", notes: "x", salary: null, createdAt }); + expect(touched).toEqual(["description", "notes"]); + expect(row.description).toBe("pdf\u0000text"); + }); +}); diff --git a/__tests__/interview.actions.spec.ts b/__tests__/interview.actions.spec.ts index e9bc8354b..a3f75b132 100644 --- a/__tests__/interview.actions.spec.ts +++ b/__tests__/interview.actions.spec.ts @@ -90,10 +90,10 @@ describe("getInterviewList", () => { { Job: { companyId: "co-1" } }, { OR: [ - { Job: { JobTitle: { label: { contains: "acme" } } } }, - { Job: { Company: { label: { contains: "acme" } } } }, - { StageType: { label: { contains: "acme" } } }, - { interviewers: { some: { Contact: { name: { contains: "acme" } } } } }, + { Job: { JobTitle: { label: { contains: "acme", mode: "insensitive" } } } }, + { Job: { Company: { label: { contains: "acme", mode: "insensitive" } } } }, + { StageType: { label: { contains: "acme", mode: "insensitive" } } }, + { interviewers: { some: { Contact: { name: { contains: "acme", mode: "insensitive" } } } } }, ], }, ]); diff --git a/__tests__/job.actions.spec.ts b/__tests__/job.actions.spec.ts index fa0fb5bc3..2f11333c1 100644 --- a/__tests__/job.actions.spec.ts +++ b/__tests__/job.actions.spec.ts @@ -389,7 +389,7 @@ describe("jobActions", () => { it("keeps the search OR alongside the status OR", async () => { await getJobsList(1, 10, { ...EMPTY_JOB_FACETS, statuses: ["applied"] }, "Developer"); const where = whereOf(); - expect(where.OR).toContainEqual({ description: { contains: "Developer" } }); + expect(where.OR).toContainEqual({ description: { contains: "Developer", mode: "insensitive" } }); expect(where.AND).toContainEqual({ OR: [{ Status: { value: { in: ["applied"] } } }], }); @@ -424,11 +424,11 @@ describe("jobActions", () => { where: expect.objectContaining({ userId: mockUser.id, OR: [ - { JobTitle: { label: { contains: "Amazon" } } }, - { Company: { label: { contains: "Amazon" } } }, - { Location: { label: { contains: "Amazon" } } }, - { JobSource: { label: { contains: "Amazon" } } }, - { description: { contains: "Amazon" } }, + { JobTitle: { label: { contains: "Amazon", mode: "insensitive" } } }, + { Company: { label: { contains: "Amazon", mode: "insensitive" } } }, + { Location: { label: { contains: "Amazon", mode: "insensitive" } } }, + { JobSource: { label: { contains: "Amazon", mode: "insensitive" } } }, + { description: { contains: "Amazon", mode: "insensitive" } }, ], }), }), @@ -437,11 +437,11 @@ describe("jobActions", () => { expect.objectContaining({ where: expect.objectContaining({ OR: [ - { JobTitle: { label: { contains: "Amazon" } } }, - { Company: { label: { contains: "Amazon" } } }, - { Location: { label: { contains: "Amazon" } } }, - { JobSource: { label: { contains: "Amazon" } } }, - { description: { contains: "Amazon" } }, + { JobTitle: { label: { contains: "Amazon", mode: "insensitive" } } }, + { Company: { label: { contains: "Amazon", mode: "insensitive" } } }, + { Location: { label: { contains: "Amazon", mode: "insensitive" } } }, + { JobSource: { label: { contains: "Amazon", mode: "insensitive" } } }, + { description: { contains: "Amazon", mode: "insensitive" } }, ], }), }), @@ -457,7 +457,7 @@ describe("jobActions", () => { const findManyCall = (prisma.job.findMany as any).mock.calls[0][0]; expect(findManyCall.where.OR).toContainEqual({ - JobTitle: { label: { contains: "Developer" } }, + JobTitle: { label: { contains: "Developer", mode: "insensitive" } }, }); }); @@ -470,7 +470,7 @@ describe("jobActions", () => { const findManyCall = (prisma.job.findMany as any).mock.calls[0][0]; expect(findManyCall.where.OR).toContainEqual({ - Company: { label: { contains: "Google" } }, + Company: { label: { contains: "Google", mode: "insensitive" } }, }); }); @@ -483,7 +483,7 @@ describe("jobActions", () => { const findManyCall = (prisma.job.findMany as any).mock.calls[0][0]; expect(findManyCall.where.OR).toContainEqual({ - Location: { label: { contains: "Remote" } }, + Location: { label: { contains: "Remote", mode: "insensitive" } }, }); }); @@ -496,7 +496,7 @@ describe("jobActions", () => { const findManyCall = (prisma.job.findMany as any).mock.calls[0][0]; expect(findManyCall.where.OR).toContainEqual({ - JobSource: { label: { contains: "LinkedIn" } }, + JobSource: { label: { contains: "LinkedIn", mode: "insensitive" } }, }); }); @@ -519,7 +519,7 @@ describe("jobActions", () => { const findManyCall = (prisma.job.findMany as any).mock.calls[0][0]; expect(findManyCall.where.OR).not.toContainEqual({ - JobSource: { label: { contains: "LinkedIn" } }, + JobSource: { label: { contains: "LinkedIn", mode: "insensitive" } }, }); }); @@ -532,7 +532,7 @@ describe("jobActions", () => { const findManyCall = (prisma.job.findMany as any).mock.calls[0][0]; expect(findManyCall.where.OR).toContainEqual({ - description: { contains: "React" }, + description: { contains: "React", mode: "insensitive" }, }); }); @@ -663,13 +663,13 @@ describe("jobActions", () => { const findManyCall = (prisma.job.findMany as any).mock.calls[0][0]; expect(findManyCall.where.OR).toEqual([ - { JobTitle: { label: { contains: "Developer" } } }, - { Location: { label: { contains: "Developer" } } }, - { JobSource: { label: { contains: "Developer" } } }, - { description: { contains: "Developer" } }, + { JobTitle: { label: { contains: "Developer", mode: "insensitive" } } }, + { Location: { label: { contains: "Developer", mode: "insensitive" } } }, + { JobSource: { label: { contains: "Developer", mode: "insensitive" } } }, + { description: { contains: "Developer", mode: "insensitive" } }, ]); expect(findManyCall.where.OR).not.toContainEqual({ - Company: { label: { contains: "Developer" } }, + Company: { label: { contains: "Developer", mode: "insensitive" } }, }); }); @@ -682,7 +682,7 @@ describe("jobActions", () => { const findManyCall = (prisma.job.findMany as any).mock.calls[0][0]; expect(findManyCall.where.OR).toContainEqual({ - Company: { label: { contains: "Developer" } }, + Company: { label: { contains: "Developer", mode: "insensitive" } }, }); }); @@ -787,7 +787,7 @@ describe("jobActions", () => { const findManyCall = (prisma.job.findMany as any).mock.calls[0][0]; expect(findManyCall.where.OR).not.toContainEqual({ - JobTitle: { label: { contains: "React" } }, + JobTitle: { label: { contains: "React", mode: "insensitive" } }, }); }); @@ -800,7 +800,7 @@ describe("jobActions", () => { const findManyCall = (prisma.job.findMany as any).mock.calls[0][0]; expect(findManyCall.where.OR).toContainEqual({ - JobTitle: { label: { contains: "React" } }, + JobTitle: { label: { contains: "React", mode: "insensitive" } }, }); }); @@ -882,7 +882,7 @@ describe("jobActions", () => { const findManyCall = (prisma.job.findMany as any).mock.calls[0][0]; expect(findManyCall.where.OR).not.toContainEqual({ - Location: { label: { contains: "React" } }, + Location: { label: { contains: "React", mode: "insensitive" } }, }); }); @@ -895,7 +895,7 @@ describe("jobActions", () => { const findManyCall = (prisma.job.findMany as any).mock.calls[0][0]; expect(findManyCall.where.OR).toContainEqual({ - Location: { label: { contains: "React" } }, + Location: { label: { contains: "React", mode: "insensitive" } }, }); }); @@ -1030,7 +1030,7 @@ describe("jobActions", () => { Company: { value: "google" }, applied: true, }); - expect(totalWhere.OR).toContainEqual({ description: { contains: "Dev" } }); + expect(totalWhere.OR).toContainEqual({ description: { contains: "Dev", mode: "insensitive" } }); const acceptedWhere = (prisma.job.count as any).mock.calls[1][0].where; expect(acceptedWhere.AND[1]).toEqual({ discoveryStatus: "accepted" }); expect(acceptedWhere.AND[0].userId).toBe(mockUser.id); @@ -1085,7 +1085,7 @@ describe("jobActions", () => { File: true, }, }, - tags: true, + tags: { orderBy: [{ label: "asc" }, { id: "asc" }] }, contactLinks: { include: { Role: true, diff --git a/__tests__/jobLocation.actions.spec.ts b/__tests__/jobLocation.actions.spec.ts index 5838076ef..4a0558846 100644 --- a/__tests__/jobLocation.actions.spec.ts +++ b/__tests__/jobLocation.actions.spec.ts @@ -61,6 +61,7 @@ describe("Job Location Actions", () => { expect(result).toEqual(mockLocations); expect(prisma.location.findMany).toHaveBeenCalledWith({ where: { createdBy: mockUser.id }, + orderBy: [{ label: "asc" }, { id: "asc" }], }); }); @@ -103,7 +104,7 @@ describe("Job Location Actions", () => { where: { createdBy: mockUser.id }, skip: 0, take: 10, - orderBy: [{ jobsApplied: { _count: "desc" } }, { label: "asc" }], + orderBy: [{ jobsApplied: { _count: "desc" } }, { label: "asc" }, { id: "desc" }], }); expect(prisma.location.count).toHaveBeenCalledWith({ where: { createdBy: mockUser.id }, @@ -142,7 +143,7 @@ describe("Job Location Actions", () => { }, }, }, - orderBy: [{ jobsApplied: { _count: "desc" } }, { label: "asc" }], + orderBy: [{ jobsApplied: { _count: "desc" } }, { label: "asc" }, { id: "desc" }], }); }); @@ -187,13 +188,13 @@ describe("Job Location Actions", () => { expect(result).toEqual({ data: mockData, total: 1 }); expect(prisma.location.findMany).toHaveBeenCalledWith({ - where: { createdBy: mockUser.id, OR: [{ label: { contains: "New" } }] }, + where: { createdBy: mockUser.id, OR: [{ label: { contains: "New", mode: "insensitive" } }] }, skip: 0, take: 10, - orderBy: [{ jobsApplied: { _count: "desc" } }, { label: "asc" }], + orderBy: [{ jobsApplied: { _count: "desc" } }, { label: "asc" }, { id: "desc" }], }); expect(prisma.location.count).toHaveBeenCalledWith({ - where: { createdBy: mockUser.id, OR: [{ label: { contains: "New" } }] }, + where: { createdBy: mockUser.id, OR: [{ label: { contains: "New", mode: "insensitive" } }] }, }); }); @@ -208,7 +209,7 @@ describe("Job Location Actions", () => { where: { createdBy: mockUser.id }, skip: 0, take: 10, - orderBy: [{ jobsApplied: { _count: "desc" } }, { label: "asc" }], + orderBy: [{ jobsApplied: { _count: "desc" } }, { label: "asc" }, { id: "desc" }], }); }); }); diff --git a/__tests__/jobSource.actions.spec.ts b/__tests__/jobSource.actions.spec.ts index d525dd00b..7b7f5622e 100644 --- a/__tests__/jobSource.actions.spec.ts +++ b/__tests__/jobSource.actions.spec.ts @@ -57,7 +57,7 @@ describe("Job Source Actions", () => { where: { createdBy: mockUser.id }, skip: 0, take: 10, - orderBy: [{ jobsApplied: { _count: "desc" } }, { label: "asc" }], + orderBy: [{ jobsApplied: { _count: "desc" } }, { label: "asc" }, { id: "desc" }], }); expect(prisma.jobSource.count).toHaveBeenCalledWith({ where: { createdBy: mockUser.id }, @@ -95,7 +95,7 @@ describe("Job Source Actions", () => { }, }, }, - orderBy: [{ jobsApplied: { _count: "desc" } }, { label: "asc" }], + orderBy: [{ jobsApplied: { _count: "desc" } }, { label: "asc" }, { id: "desc" }], }); expect(prisma.job.groupBy).toHaveBeenCalledWith({ by: ["jobSourceId"], @@ -143,13 +143,13 @@ describe("Job Source Actions", () => { expect(result).toEqual({ data: mockData, total: 1 }); expect(prisma.jobSource.findMany).toHaveBeenCalledWith({ - where: { createdBy: mockUser.id, OR: [{ label: { contains: "Link" } }] }, + where: { createdBy: mockUser.id, OR: [{ label: { contains: "Link", mode: "insensitive" } }] }, skip: 0, take: 10, - orderBy: [{ jobsApplied: { _count: "desc" } }, { label: "asc" }], + orderBy: [{ jobsApplied: { _count: "desc" } }, { label: "asc" }, { id: "desc" }], }); expect(prisma.jobSource.count).toHaveBeenCalledWith({ - where: { createdBy: mockUser.id, OR: [{ label: { contains: "Link" } }] }, + where: { createdBy: mockUser.id, OR: [{ label: { contains: "Link", mode: "insensitive" } }] }, }); }); @@ -164,7 +164,7 @@ describe("Job Source Actions", () => { where: { createdBy: mockUser.id }, skip: 0, take: 10, - orderBy: [{ jobsApplied: { _count: "desc" } }, { label: "asc" }], + orderBy: [{ jobsApplied: { _count: "desc" } }, { label: "asc" }, { id: "desc" }], }); }); }); diff --git a/__tests__/jobtitle.actions.spec.ts b/__tests__/jobtitle.actions.spec.ts index 66a3572d9..50b3a2e5b 100644 --- a/__tests__/jobtitle.actions.spec.ts +++ b/__tests__/jobtitle.actions.spec.ts @@ -57,6 +57,7 @@ describe("Job Title Actions", () => { expect(result).toEqual(mockTitles); expect(prisma.jobTitle.findMany).toHaveBeenCalledWith({ where: { createdBy: mockUser.id }, + orderBy: [{ label: "asc" }, { id: "asc" }], }); }); @@ -99,7 +100,7 @@ describe("Job Title Actions", () => { where: { createdBy: mockUser.id }, skip: 0, take: 10, - orderBy: [{ jobs: { _count: "desc" } }, { label: "asc" }], + orderBy: [{ jobs: { _count: "desc" } }, { label: "asc" }, { id: "desc" }], }); expect(prisma.jobTitle.count).toHaveBeenCalledWith({ where: { createdBy: mockUser.id }, @@ -138,7 +139,7 @@ describe("Job Title Actions", () => { }, }, }, - orderBy: [{ jobs: { _count: "desc" } }, { label: "asc" }], + orderBy: [{ jobs: { _count: "desc" } }, { label: "asc" }, { id: "desc" }], }); }); @@ -185,13 +186,13 @@ describe("Job Title Actions", () => { expect(result).toEqual({ data: mockData, total: 1 }); expect(prisma.jobTitle.findMany).toHaveBeenCalledWith({ - where: { createdBy: mockUser.id, OR: [{ label: { contains: "Dev" } }] }, + where: { createdBy: mockUser.id, OR: [{ label: { contains: "Dev", mode: "insensitive" } }] }, skip: 0, take: 10, - orderBy: [{ jobs: { _count: "desc" } }, { label: "asc" }], + orderBy: [{ jobs: { _count: "desc" } }, { label: "asc" }, { id: "desc" }], }); expect(prisma.jobTitle.count).toHaveBeenCalledWith({ - where: { createdBy: mockUser.id, OR: [{ label: { contains: "Dev" } }] }, + where: { createdBy: mockUser.id, OR: [{ label: { contains: "Dev", mode: "insensitive" } }] }, }); }); @@ -206,7 +207,7 @@ describe("Job Title Actions", () => { where: { createdBy: mockUser.id }, skip: 0, take: 10, - orderBy: [{ jobs: { _count: "desc" } }, { label: "asc" }], + orderBy: [{ jobs: { _count: "desc" } }, { label: "asc" }, { id: "desc" }], }); }); }); diff --git a/__tests__/postgresQueryParity.spec.ts b/__tests__/postgresQueryParity.spec.ts new file mode 100644 index 000000000..88e623bc4 --- /dev/null +++ b/__tests__/postgresQueryParity.spec.ts @@ -0,0 +1,273 @@ +import { UNANALYZED_MATCH_MARKER } from "@/actions/job/shared"; +import { getTagList } from "@/actions/tag.actions"; +import { getDiscoveredJobs } from "@/actions/automation/discoveredJobs"; +import { getAutomationRuns } from "@/actions/automation/runs"; +import { getAutomationsList } from "@/actions/automation/queries"; +import { getQuestionsList } from "@/actions/question.actions"; +import { getContactList } from "@/actions/contact/queries"; +import { getResumeList, getResumeById } from "@/actions/profile/resume"; +import { getTasksList } from "@/actions/task/queries"; +import { getReferenceEntityList } from "@/actions/referenceList"; +import { getInterviewList } from "@/actions/interview.actions"; +import { getActivitiesList } from "@/actions/activity/activities"; +import { getCoverLetterList } from "@/actions/coverLetter.actions"; +import { getContactRoleList } from "@/actions/contactRole.actions"; +import { getJobStageTypeList } from "@/actions/jobStageType.actions"; +import { getJobsList, getJobsIterator } from "@/actions/job/queries"; +import { getRecentJobs } from "@/actions/dashboard/recent"; +import { getCompanyList } from "@/actions/company/queries"; +import prisma from "@/lib/db"; + +type Call = { model: string; method: string; args: any }; + +const h = vi.hoisted(() => ({ + calls: [] as { model: string; method: string; args: any }[], + returns: {} as Record, +})); + +// Every delegate method records its call and returns an empty result, so a +// site can be driven without hand-listing the methods it happens to use. +vi.mock("@/lib/db", () => { + const empty = (method: string) => + method === "count" ? 0 : method === "findMany" || method === "groupBy" ? [] : null; + const delegates: Record = {}; + const delegate = (model: string) => + new Proxy( + {}, + { + get: (_target, method: string) => async (args: any) => { + h.calls.push({ model, method, args }); + const key = `${model}.${method}`; + if (!(key in h.returns)) return empty(method); + const value = h.returns[key]; + return typeof value === "function" ? value(args) : value; + }, + }, + ); + return { + default: new Proxy( + {}, + { get: (_target, model: string) => (delegates[model] ??= delegate(model)) }, + ), + }; +}); + +vi.mock("@/utils/user.utils", () => ({ + getCurrentUser: async () => ({ id: "user-1" }), +})); +vi.mock("next/cache", () => ({ revalidatePath: vi.fn(), revalidateTag: vi.fn() })); + +const findManyCalls = (model: string): Call[] => + h.calls.filter((c) => c.model === model && c.method === "findMany"); + +// Every { contains } object anywhere inside a where clause +const containsFilters = (node: unknown, found: any[] = []): any[] => { + if (Array.isArray(node)) node.forEach((n) => containsFilters(n, found)); + else if (node && typeof node === "object") { + if ("contains" in node) found.push(node); + Object.values(node).forEach((n) => containsFilters(n, found)); + } + return found; +}; + +const lastKey = (orderBy: unknown) => { + const list = Array.isArray(orderBy) ? orderBy : [orderBy]; + return list[list.length - 1]; +}; + +beforeEach(() => { + h.calls.length = 0; + for (const key of Object.keys(h.returns)) delete h.returns[key]; +}); + +describe("search parity: user search is case-insensitive", () => { + const sites: [string, string, () => Promise][] = [ + ["tags", "tag", () => getTagList(1, 10, "Goo")], + ["questions", "question", () => getQuestionsList(1, 10, undefined, "Goo")], + ["contacts", "contact", () => getContactList(1, 10, "Goo")], + ["tasks", "task", () => getTasksList(1, 10, undefined, undefined, "Goo")], + ["activities", "activity", () => getActivitiesList(1, 10, "Goo")], + ["contact roles", "contactRole", () => getContactRoleList(1, 10, "Goo")], + ["stage types", "jobStageType", () => getJobStageTypeList(1, 10, "Goo")], + ["jobs", "job", () => getJobsList(1, 10, undefined, "Goo")], + [ + "reference lists", + "company", + () => + getReferenceEntityList({ + model: (prisma as any).company, + userId: "user-1", + fkField: "companyId", + appliedRelation: "jobsApplied", + page: 1, + limit: 10, + search: "Goo", + }), + ], + ]; + + it.each(sites)("%s", async (_name, model, run) => { + await run(); + const filters = findManyCalls(model).flatMap((c) => containsFilters(c.args.where)); + expect(filters.length).toBeGreaterThan(0); + for (const filter of filters) { + expect(filter).toEqual({ contains: "Goo", mode: "insensitive" }); + } + }); + + it("interviews", async () => { + h.returns["jobStage.count"] = 1; + await getInterviewList(1, 10, "Goo"); + const filters = findManyCalls("jobStage").flatMap((c) => containsFilters(c.args.where)); + expect(filters).toHaveLength(4); + for (const filter of filters) { + expect(filter).toEqual({ contains: "Goo", mode: "insensitive" }); + } + }); + + it("leaves the app-written unanalyzed marker case-sensitive", async () => { + h.returns["job.count"] = 1; + await getJobsList(1, 10, undefined, undefined, undefined, undefined, undefined, undefined, undefined, { + field: "matchScore", + dir: "desc", + }); + const markers = h.calls + .filter((c) => c.model === "job") + .flatMap((c) => containsFilters(c.args?.where)) + .filter((f) => f.contains === UNANALYZED_MATCH_MARKER); + expect(markers.length).toBeGreaterThan(0); + for (const marker of markers) expect(marker).not.toHaveProperty("mode"); + }); +}); + +describe("NULL ordering parity", () => { + it("keeps undated tasks first in the due-date grouping", async () => { + await getTasksList(1, 10, undefined, undefined, undefined, "dueDate"); + const [call] = findManyCalls("task"); + expect(call.args.orderBy[0]).toEqual({ dueDate: { sort: "asc", nulls: "first" } }); + }); + + it("keeps unscored discovered jobs last", async () => { + await getDiscoveredJobs(); + const [call] = findManyCalls("job"); + expect(call.args.orderBy[0]).toEqual({ matchScore: { sort: "desc", nulls: "last" } }); + }); + + it("keeps undated applied jobs last on the Recent Jobs card", async () => { + await getRecentJobs(); + const [call] = findManyCalls("job"); + expect(call.args.orderBy[0]).toEqual({ appliedDate: { sort: "desc", nulls: "last" } }); + expect(lastKey(call.args.orderBy)).toEqual({ id: "desc" }); + }); + + it("keeps undated watched companies last", async () => { + await getCompanyList(1, 10, undefined, undefined, "watchlist"); + const [call] = findManyCalls("company"); + expect(call.args.orderBy[0]).toEqual({ watchedAt: { sort: "desc", nulls: "last" } }); + }); +}); + +describe("pagination tiebreaker: every paged sort ends on id", () => { + const sites: [string, string, () => Promise][] = [ + ["tags", "tag", () => getTagList()], + ["discovered jobs", "job", () => getDiscoveredJobs()], + ["questions", "question", () => getQuestionsList()], + ["contacts", "contact", () => getContactList()], + ["automations", "automation", () => getAutomationsList()], + ["resumes", "resume", () => getResumeList()], + ["activities", "activity", () => getActivitiesList()], + ["cover letters", "coverLetter", () => getCoverLetterList()], + ["contact roles", "contactRole", () => getContactRoleList()], + ["stage types", "jobStageType", () => getJobStageTypeList()], + [ + "reference lists", + "company", + () => + getReferenceEntityList({ + model: (prisma as any).company, + userId: "user-1", + fkField: "companyId", + appliedRelation: "jobsApplied", + page: 1, + limit: 10, + }), + ], + [ + "watchlist order passed in by the caller", + "company", + () => + getReferenceEntityList({ + model: (prisma as any).company, + userId: "user-1", + fkField: "companyId", + appliedRelation: "jobsApplied", + orderBy: [{ watchedAt: { sort: "desc", nulls: "last" } }, { label: "asc" }], + page: 1, + limit: 10, + }), + ], + ]; + + it.each(sites)("%s", async (_name, model, run) => { + await run(); + const calls = findManyCalls(model).filter((c) => "take" in c.args); + expect(calls.length).toBeGreaterThan(0); + for (const call of calls) expect(lastKey(call.args.orderBy)).toEqual({ id: "desc" }); + }); + + it.each(["dueDate", "createdDate", "updatedDate", "activityType", undefined] as const)( + "tasks grouped by %s", + async (groupBy) => { + await getTasksList(1, 10, undefined, undefined, undefined, groupBy as any); + const [call] = findManyCalls("task"); + expect(lastKey(call.args.orderBy)).toEqual({ id: "desc" }); + }, + ); + + it("automation runs", async () => { + h.returns["automation.findFirst"] = { id: "auto-1" }; + await getAutomationRuns("auto-1"); + const [call] = findManyCalls("automationRun"); + expect(lastKey(call.args.orderBy)).toEqual({ id: "desc" }); + }); + + it("resumes with a pinned default, on both pages", async () => { + h.returns["user.findUnique"] = { defaultResumeId: "resume-1" }; + await getResumeList(1, 10); + await getResumeList(2, 10); + const calls = findManyCalls("resume"); + expect(calls).toHaveLength(2); + for (const call of calls) expect(lastKey(call.args.orderBy)).toEqual({ id: "desc" }); + }); + + it("interviews, dated and undated halves", async () => { + h.returns["jobStage.count"] = (args: any) => + args.where.AND?.some((c: any) => c.occurredAt?.not === null) ? 1 : 3; + await getInterviewList(1, 10); + const calls = findManyCalls("jobStage"); + expect(calls).toHaveLength(2); + for (const call of calls) expect(lastKey(call.args.orderBy)).toEqual({ id: "desc" }); + }); + + it("the job export iterator", async () => { + for await (const _chunk of getJobsIterator()) break; + const [call] = findManyCalls("job"); + expect(lastKey(call.args.orderBy)).toEqual({ id: "desc" }); + }); +}); + +describe("unordered reads: resume entries carry an explicit order", () => { + it("orders every entry relation on the resume detail read", async () => { + await getResumeById("resume-1"); + const call = h.calls.find((c) => c.model === "resume" && c.method === "findUnique")!; + const sections = call.args.include.ResumeSections; + expect(sections.orderBy).toEqual({ id: "asc" }); + expect(sections.include.workExperiences.orderBy).toEqual([{ createdAt: "asc" }, { id: "asc" }]); + expect(sections.include.educations.orderBy).toEqual([{ createdAt: "asc" }, { id: "asc" }]); + expect(sections.include.licenseOrCertifications.orderBy).toEqual([ + { issueDate: { sort: "desc", nulls: "last" } }, + { title: "asc" }, + { id: "asc" }, + ]); + }); +}); diff --git a/__tests__/postgresText.spec.ts b/__tests__/postgresText.spec.ts new file mode 100644 index 000000000..7d163e49b --- /dev/null +++ b/__tests__/postgresText.spec.ts @@ -0,0 +1,11 @@ +import { sanitizeString } from "@/lib/postgresText"; + +describe("sanitizeString", () => { + it("removes NUL and lone surrogates Postgres cannot store", () => { + expect(sanitizeString("a\u0000b\uD800c\uDC00d")).toBe("abcd"); + }); + + it("keeps valid surrogate pairs and ordinary text", () => { + expect(sanitizeString("Résumé 😀\n")).toBe("Résumé 😀\n"); + }); +}); diff --git a/__tests__/question.actions.spec.ts b/__tests__/question.actions.spec.ts index 5c4ee8eed..050abc4b6 100644 --- a/__tests__/question.actions.spec.ts +++ b/__tests__/question.actions.spec.ts @@ -68,8 +68,10 @@ describe("Question Actions", () => { where: { createdBy: mockUser.id }, skip: 0, take: 10, - orderBy: [{ createdAt: "desc" }], - include: { tags: true }, + orderBy: [{ createdAt: "desc" }, { id: "desc" }], + include: { + tags: { orderBy: [{ label: "asc" }, { id: "asc" }] }, + }, }), ); }); @@ -134,8 +136,8 @@ describe("Question Actions", () => { where: { createdBy: mockUser.id, OR: [ - { question: { contains: "react" } }, - { answer: { contains: "react" } }, + { question: { contains: "react", mode: "insensitive" } }, + { answer: { contains: "react", mode: "insensitive" } }, ], }, }), @@ -155,8 +157,8 @@ describe("Question Actions", () => { createdBy: mockUser.id, tags: { some: { id: "tag-1" } }, OR: [ - { question: { contains: "react" } }, - { answer: { contains: "react" } }, + { question: { contains: "react", mode: "insensitive" } }, + { answer: { contains: "react", mode: "insensitive" } }, ], }, }), diff --git a/__tests__/referenceList.spec.ts b/__tests__/referenceList.spec.ts index b61b17234..016c54825 100644 --- a/__tests__/referenceList.spec.ts +++ b/__tests__/referenceList.spec.ts @@ -46,7 +46,7 @@ describe("getReferenceEntityList", () => { where: { createdBy: userId }, skip: 0, take: 10, - orderBy: [{ jobsApplied: { _count: "desc" } }, { label: "asc" }], + orderBy: [{ jobsApplied: { _count: "desc" } }, { label: "asc" }, { id: "desc" }], }); expect(model.count).toHaveBeenCalledWith({ where: { createdBy: userId } }); expect(prisma.job.groupBy).not.toHaveBeenCalled(); @@ -80,7 +80,7 @@ describe("getReferenceEntityList", () => { }, }, }, - orderBy: [{ jobs: { _count: "desc" } }, { label: "asc" }], + orderBy: [{ jobs: { _count: "desc" } }, { label: "asc" }, { id: "desc" }], }); }); @@ -206,11 +206,11 @@ describe("getReferenceEntityList", () => { expect(model.findMany).toHaveBeenCalledWith( expect.objectContaining({ - where: { createdBy: userId, OR: [{ label: { contains: "Ama" } }] }, + where: { createdBy: userId, OR: [{ label: { contains: "Ama", mode: "insensitive" } }] }, }) ); expect(model.count).toHaveBeenCalledWith({ - where: { createdBy: userId, OR: [{ label: { contains: "Ama" } }] }, + where: { createdBy: userId, OR: [{ label: { contains: "Ama", mode: "insensitive" } }] }, }); }); @@ -320,8 +320,8 @@ describe("getReferenceEntityList", () => { where: { createdBy: userId, OR: [ - { label: { contains: "acme" } }, - { atsToken: { contains: "acme" } }, + { label: { contains: "acme", mode: "insensitive" } }, + { atsToken: { contains: "acme", mode: "insensitive" } }, ], }, }) @@ -343,7 +343,7 @@ describe("getReferenceEntityList", () => { }); expect(model.findMany).toHaveBeenCalledWith( - expect.objectContaining({ orderBy }) + expect.objectContaining({ orderBy: [...orderBy, { id: "desc" }] }) ); }); describe("relationCounts", () => { diff --git a/__tests__/resolveDbUrl.spec.ts b/__tests__/resolveDbUrl.spec.ts new file mode 100644 index 000000000..21ce41372 --- /dev/null +++ b/__tests__/resolveDbUrl.spec.ts @@ -0,0 +1,49 @@ +import { resolveDatabaseUrl } from "../scripts/wait/resolve"; + +const BUNDLED = "postgres://jobsync:b3und!ed@db:5432/jobsync"; +const EXTERNAL = "postgres://me:pw@192.168.0.162:5432/jobsync"; + +describe("resolveDatabaseUrl", () => { + it("keeps the url as found when BUNDLED_DATABASE_URL is absent", () => { + expect(resolveDatabaseUrl({ DATABASE_URL: "file:/data/dev.db" })).toEqual({ + url: "file:/data/dev.db", + warning: null, + }); + }); + + it.each([undefined, "", " "])("uses the bundled url when DATABASE_URL is %j", (raw) => { + expect(resolveDatabaseUrl({ DATABASE_URL: raw, BUNDLED_DATABASE_URL: BUNDLED })).toEqual({ + url: BUNDLED, + warning: null, + }); + }); + + it.each([undefined, "1"])("swaps a file: url for the bundled one when BUNDLED_DB is %j", (flag) => { + const result = resolveDatabaseUrl({ + DATABASE_URL: "file:./dev.db", + BUNDLED_DATABASE_URL: BUNDLED, + BUNDLED_DB: flag, + }); + expect(result.url).toBe(BUNDLED); + expect(result.warning).toContain("Ignoring DATABASE_URL=file:./dev.db"); + expect(result.warning).toContain("You can delete that line from .env."); + expect(result.warning).not.toContain("b3und!ed"); + }); + + it("keeps a file: url when the bundled database is switched off", () => { + expect( + resolveDatabaseUrl({ + DATABASE_URL: "file:/data/dev.db", + BUNDLED_DATABASE_URL: BUNDLED, + BUNDLED_DB: "0", + }), + ).toEqual({ url: "file:/data/dev.db", warning: null }); + }); + + it("keeps an external url", () => { + expect(resolveDatabaseUrl({ DATABASE_URL: EXTERNAL, BUNDLED_DATABASE_URL: BUNDLED })).toEqual({ + url: EXTERNAL, + warning: null, + }); + }); +}); diff --git a/__tests__/resumeCopy.actions.spec.ts b/__tests__/resumeCopy.actions.spec.ts index 0475ae4c6..bfa5387a8 100644 --- a/__tests__/resumeCopy.actions.spec.ts +++ b/__tests__/resumeCopy.actions.spec.ts @@ -131,6 +131,7 @@ describe("copyResume", () => { startDate: new Date("2020-01-01"), endDate: null, description: "

Work

", + createdAt: new Date("2024-03-01T10:00:00Z"), }, ], educations: [], @@ -209,6 +210,8 @@ describe("copyResume", () => { startDate: new Date("2020-01-01"), endDate: null, description: "

Work

", + // Kept so the copy lists entries in the source's order + createdAt: new Date("2024-03-01T10:00:00Z"), }, ]); }); diff --git a/__tests__/tag.actions.spec.ts b/__tests__/tag.actions.spec.ts index 2961888f3..076bdb487 100644 --- a/__tests__/tag.actions.spec.ts +++ b/__tests__/tag.actions.spec.ts @@ -105,7 +105,7 @@ describe("Tag Actions", () => { where: { createdBy: mockUser.id }, skip: 0, take: 10, - orderBy: { jobs: { _count: "desc" } }, + orderBy: [{ jobs: { _count: "desc" } }, { id: "desc" }], }), ); }); @@ -153,11 +153,11 @@ describe("Tag Actions", () => { expect(result).toEqual({ data: mockData, total: 1 }); expect(prisma.tag.findMany).toHaveBeenCalledWith( expect.objectContaining({ - where: { createdBy: mockUser.id, label: { contains: "Rea" } }, + where: { createdBy: mockUser.id, label: { contains: "Rea", mode: "insensitive" } }, }), ); expect(prisma.tag.count).toHaveBeenCalledWith({ - where: { createdBy: mockUser.id, label: { contains: "Rea" } }, + where: { createdBy: mockUser.id, label: { contains: "Rea", mode: "insensitive" } }, }); }); diff --git a/__tests__/task.actions.spec.ts b/__tests__/task.actions.spec.ts index 5b8216e28..36a3e7f1f 100644 --- a/__tests__/task.actions.spec.ts +++ b/__tests__/task.actions.spec.ts @@ -88,7 +88,12 @@ describe("taskActions", () => { select: { id: true }, }, }, - orderBy: [{ priority: "desc" }, { createdAt: "desc" }, { updatedAt: "desc" }], + orderBy: [ + { priority: "desc" }, + { createdAt: "desc" }, + { updatedAt: "desc" }, + { id: "desc" }, + ], skip: 0, take: 25, }); @@ -119,7 +124,12 @@ describe("taskActions", () => { select: { id: true }, }, }, - orderBy: [{ priority: "desc" }, { createdAt: "desc" }, { updatedAt: "desc" }], + orderBy: [ + { priority: "desc" }, + { createdAt: "desc" }, + { updatedAt: "desc" }, + { id: "desc" }, + ], skip: 10, take: 10, }); @@ -150,7 +160,12 @@ describe("taskActions", () => { select: { id: true }, }, }, - orderBy: [{ priority: "desc" }, { createdAt: "desc" }, { updatedAt: "desc" }], + orderBy: [ + { priority: "desc" }, + { createdAt: "desc" }, + { updatedAt: "desc" }, + { id: "desc" }, + ], skip: 0, take: 10, }); @@ -184,7 +199,12 @@ describe("taskActions", () => { select: { id: true }, }, }, - orderBy: [{ priority: "desc" }, { createdAt: "desc" }, { updatedAt: "desc" }], + orderBy: [ + { priority: "desc" }, + { createdAt: "desc" }, + { updatedAt: "desc" }, + { id: "desc" }, + ], skip: 0, take: 10, }); @@ -208,9 +228,10 @@ describe("taskActions", () => { }, }, orderBy: [ - { dueDate: "asc" }, + { dueDate: { sort: "asc", nulls: "first" } }, { priority: "desc" }, { createdAt: "desc" }, + { id: "desc" }, ], skip: 0, take: 25, @@ -245,6 +266,7 @@ describe("taskActions", () => { { activityType: { label: "asc" } }, { priority: "desc" }, { createdAt: "desc" }, + { id: "desc" }, ], skip: 0, take: 25, @@ -852,6 +874,7 @@ describe("taskActions", () => { where: { createdBy: mockUser.id, }, + orderBy: [{ createdAt: "asc" }, { id: "asc" }], include: { _count: { select: { diff --git a/__tests__/waitForDb.spec.ts b/__tests__/waitForDb.spec.ts new file mode 100644 index 000000000..d66e7ec9a --- /dev/null +++ b/__tests__/waitForDb.spec.ts @@ -0,0 +1,226 @@ +import { checkUrl, classify, maintenanceUrl, quoteIdent, redact, UPGRADE_GUIDE_URL } from "../scripts/wait/classify"; +import { waitForDb, type WaitDeps } from "../scripts/wait/run"; + +const SECRET = "s3cr@t:pa/ss#word"; +const URL_OK = `postgres://jobsync:${encodeURIComponent(SECRET)}@db.example.internal:5433/jobsync`; + +const failure = (errorCode?: string, message = "boom") => Object.assign(new Error(message), { errorCode }); + +function harness(over: { + url?: string; + env?: Record; + attempts?: (Error | null)[]; + createFails?: Error; + dbResolves?: boolean; +} = {}) { + const log: string[] = []; + const attempts = [...(over.attempts ?? [null])]; + let clock = 0; + const created: { url: string; name: string }[] = []; + let connects = 0; + const deps: WaitDeps = { + env: { DATABASE_URL: "url" in over ? over.url : URL_OK, ...over.env }, + connect: async () => { + connects++; + const next = attempts.length > 1 ? attempts.shift() : attempts[0]; + if (next) throw next; + }, + createDatabase: async (url, name) => { + if (over.createFails) throw over.createFails; + created.push({ url, name }); + }, + resolves: async () => over.dbResolves ?? true, + sleep: async (ms) => { + clock += ms; + }, + now: () => clock, + log: (line) => log.push(line), + }; + return { deps, log, created, connects: () => connects, text: () => log.join("\n") }; +} + +describe("checkUrl", () => { + it("parses host, port and database", () => { + expect(checkUrl(URL_OK)).toEqual({ + ok: true, + target: { host: "db.example.internal", port: "5433", database: "jobsync" }, + }); + }); + + it("defaults the port and accepts postgresql://", () => { + expect(checkUrl("postgresql://u:p@host/app")).toEqual({ + ok: true, + target: { host: "host", port: "5432", database: "app" }, + }); + }); + + it.each([undefined, "", " "])("reports %j as not set", (raw) => { + expect(checkUrl(raw)).toEqual({ ok: false, message: "DATABASE_URL is not set." }); + }); + + it.each(["file:/data/dev.db", "file:./dev.db"])("names the SQLite url %s as found", (raw) => { + const result = checkUrl(raw) as { ok: false; message: string }; + expect(result.ok).toBe(false); + expect(result.message).toContain(`DATABASE_URL still points at SQLite (${raw}).`); + expect(result.message).toContain(UPGRADE_GUIDE_URL); + // Reached only with BUNDLED_DB=0: a plain leftover falls back to bundled + expect(result.message).toContain("Set DATABASE_URL to your own Postgres, or remove BUNDLED_DB=0"); + expect(result.message).not.toContain("Delete the DATABASE_URL line"); + expect(result.message).toContain("Your existing database has NOT been modified."); + }); + + it.each(["not a url", "mysql://u:p@h/db", "postgres://u:p@host", "postgres://u:p@/db"])( + "reports %s as malformed", + (raw) => { + const result = checkUrl(raw) as { ok: false; message: string }; + expect(result.message).toContain("DATABASE_URL is malformed."); + }, + ); +}); + +describe("classify", () => { + it.each([ + ["P1000", "auth"], + ["P1003", "missing-database"], + ["P1012", "bad-url"], + ["P1001", "retry"], + ["P1002", "retry"], + ["P1017", "retry"], + ["P9999", "retry"], + [undefined, "retry"], + ])("%s -> %s", (code, verdict) => { + expect(classify(code)).toBe(verdict); + }); +}); + +describe("helpers", () => { + it("points the maintenance url at postgres, keeping credentials and params", () => { + expect(maintenanceUrl("postgres://u:p@host:5433/app?sslmode=require")).toBe( + "postgres://u:p@host:5433/postgres?sslmode=require", + ); + }); + + it("quotes an identifier", () => { + expect(quoteIdent('job"sync')).toBe('"job""sync"'); + }); + + it("redacts a password whose % is not a valid escape", () => { + expect(redact("auth failed for postgres://u:p%zz@h/db", "postgres://u:p%zz@h/db")).toBe( + "auth failed for postgres://u:***@h/db", + ); + }); + + it("keeps a default password readable where it is the user or db name", () => { + const raw = "postgres://jobsync:jobsync@db:5432/jobsync"; + expect(redact(`User \`jobsync\` denied on \`jobsync.public\` via ${raw}`, raw)).toBe( + "User `jobsync` denied on `jobsync.public` via postgres://jobsync:***@db:5432/jobsync", + ); + }); +}); + +describe("waitForDb", () => { + it("returns 0 on a first-try connection", async () => { + const h = harness(); + expect(await waitForDb(h.deps)).toBe(0); + expect(h.text()).toContain("Database reachable at db.example.internal:5433/jobsync."); + }); + + it("retries a starting database and then succeeds", async () => { + const h = harness({ attempts: [failure("P1001"), failure("P1002"), null] }); + expect(await waitForDb(h.deps)).toBe(0); + expect(h.connects()).toBe(3); + expect(h.text()).toContain("waiting for database at db.example.internal:5433"); + }); + + it("retries an unknown error code within the window, then gives up", async () => { + const h = harness({ attempts: [failure("P7777")] }); + expect(await waitForDb(h.deps)).toBe(1); + expect(h.connects()).toBeGreaterThan(5); + expect(h.text()).toContain("Could not reach the database at db.example.internal:5433 within 60 seconds."); + }); + + it("exits at once on an auth failure, with the external causes", async () => { + const h = harness({ attempts: [failure("P1000")] }); + expect(await waitForDb(h.deps)).toBe(1); + expect(h.connects()).toBe(1); + expect(h.text()).toContain("percent-encoded"); + expect(h.text()).toContain("pg_hba.conf"); + }); + + it("names password rotation for the bundled database", async () => { + const h = harness({ url: "postgres://jobsync:jobsync@db:5432/jobsync", attempts: [failure("P1000")] }); + expect(await waitForDb(h.deps)).toBe(1); + expect(h.text()).toContain("POSTGRES_PASSWORD was changed after the database volume was created"); + }); + + it("exits at once when the url is not set, is SQLite or is malformed", async () => { + for (const url of [undefined, "file:/data/dev.db", "nonsense"]) { + const h = harness({ url }); + expect(await waitForDb(h.deps)).toBe(1); + expect(h.connects()).toBe(0); + } + }); + + it("creates a missing database, says so, and continues", async () => { + const h = harness({ attempts: [failure("P1003"), null] }); + expect(await waitForDb(h.deps)).toBe(0); + expect(h.created).toEqual([ + { url: `postgres://jobsync:${encodeURIComponent(SECRET)}@db.example.internal:5433/postgres`, name: '"jobsync"' }, + ]); + expect(h.text()).toContain('Database "jobsync" did not exist on db.example.internal:5433 — created it.'); + expect(h.text()).toContain("a typo in the database"); + }); + + it("falls back to the manual SQL when creation is refused", async () => { + const h = harness({ + attempts: [failure("P1003")], + createFails: new Error("permission denied to create database"), + }); + expect(await waitForDb(h.deps)).toBe(1); + const text = h.text(); + expect(text).toContain("could not create it: permission denied to create database"); + expect(text).toContain("CREATE DATABASE jobsync OWNER ;"); + expect(text).toContain("GRANT ALL ON SCHEMA public TO ;"); + }); + + it("does not create when CREATE_DATABASE_IF_MISSING=0", async () => { + const h = harness({ attempts: [failure("P1003")], env: { CREATE_DATABASE_IF_MISSING: "0" } }); + expect(await waitForDb(h.deps)).toBe(1); + expect(h.created).toEqual([]); + expect(h.text()).toContain("CREATE DATABASE jobsync OWNER ;"); + }); + + it("does not loop when the database is still missing after creating it", async () => { + const h = harness({ attempts: [failure("P1003")] }); + expect(await waitForDb(h.deps)).toBe(1); + expect(h.created).toHaveLength(1); + }); + + it("explains an unresolvable bundled host", async () => { + const h = harness({ + url: "postgres://jobsync:jobsync@db:5432/jobsync", + attempts: [failure("P1001")], + dbResolves: false, + }); + expect(await waitForDb(h.deps)).toBe(1); + expect(h.text()).toContain('DATABASE_URL points at host "db", but no such host exists'); + expect(h.text()).toContain("BUNDLED_DB=0"); + }); + + it("never logs the password", async () => { + const cases = [ + { attempts: [failure("P1000")] }, + { attempts: [failure("P1012")] }, + { attempts: [failure("P1001", `Can't reach ${URL_OK}`)] }, + { attempts: [failure("P1003")], createFails: new Error(`denied for ${URL_OK}`) }, + { attempts: [failure("P1003"), null] }, + { url: `postgres://jobsync:${SECRET}@host/db`, attempts: [failure("P1000")] }, + ]; + for (const c of cases) { + const h = harness(c); + await waitForDb(h.deps); + expect(h.text()).not.toContain(SECRET); + expect(h.text()).not.toContain(encodeURIComponent(SECRET)); + } + }); +}); diff --git a/backup-db.sh b/backup-db.sh new file mode 100755 index 000000000..54ee9921a --- /dev/null +++ b/backup-db.sh @@ -0,0 +1,96 @@ +#!/bin/bash + +# JobSync database backup +# Usage: ./backup-db.sh +# Writes jobsyncdb/data/backups/jobsync-YYYY-MM-DD-HHMM.sql.gz + +# pipefail matters: without it a failed pg_dump still lets gzip exit 0 and +# leaves a dated, valid, empty archive that looks like a backup. +set -euo pipefail + +SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" +cd "$SCRIPT_DIR" + +# Last uncommented NAME=value line in .env, with surrounding quotes removed +env_value() { + [ -f .env ] || return 0 + { grep -E "^[[:space:]]*$1=" .env || true; } | tail -n 1 | cut -d= -f2- \ + | sed -e 's/^["'\'']//' -e 's/["'\'']$//' +} + +if docker compose version >/dev/null 2>&1; then + COMPOSE_CMD="docker compose" +elif command -v docker-compose >/dev/null 2>&1; then + COMPOSE_CMD="docker-compose" +else + echo "Error: Docker Compose not found" + exit 1 +fi + +BUNDLED_DB="$(env_value BUNDLED_DB)" +DATABASE_URL="$(env_value DATABASE_URL)" +PG_DUMP_IMAGE="$(env_value PG_DUMP_IMAGE)" +PG_DUMP_IMAGE="${PG_DUMP_IMAGE:-pgvector/pgvector:pg18}" +export DATABASE_URL + +BACKUP_DIR="jobsyncdb/data/backups" +if ! mkdir -p "$BACKUP_DIR" 2>/dev/null || [ ! -w "$BACKUP_DIR" ]; then + echo "Cannot write to $BACKUP_DIR. The app container owns jobsyncdb/data; re-run with sudo." + exit 1 +fi + +# Dump to a temporary name; only a complete dump gets a dated file name +TMP="$(mktemp "$BACKUP_DIR/.jobsync-dump.XXXXXX")" +ERR="$(mktemp)" +trap 'rm -f "$TMP" "$ERR"' EXIT + +# The app's rule: unset, a leftover file: or host db all mean bundled +db_mode() { + case "$DATABASE_URL" in + postgres://*|postgresql://*) ;; + *) echo bundled; return ;; + esac + local host="${DATABASE_URL#*://}" + host="${host%%/*}" + host="${host##*@}" + host="${host%%:*}" + if [ "$host" = "db" ]; then echo bundled; else echo external; fi +} + +# pg_dump and psql reject Prisma-only URL options such as ?schema=public; +# real libpq options like sslmode are kept +libpq_url() { + printf '%s' "$DATABASE_URL" | sed -E 's/([?&])(schema|connection_limit|pool_timeout|pgbouncer|statement_cache_size|socket_timeout)=[^&]*/\1/g; s/\?&+/?/; s/&&+/\&/g; s/[?&]$//' +} + +if [ "$(db_mode)" = "external" ]; then + DATABASE_URL="$(libpq_url)" + echo "Backing up the external database with $PG_DUMP_IMAGE..." + # pg_dump comes from a container: a Docker-only host has no client tools + dump() { + docker run --rm -e DATABASE_URL --add-host=host.docker.internal:host-gateway \ + "$PG_DUMP_IMAGE" sh -c 'pg_dump "$DATABASE_URL"' + } +else + if [ "$BUNDLED_DB" = "0" ]; then + echo "BUNDLED_DB=0 is set in .env but DATABASE_URL does not point at your own Postgres: there is no database to back up." + exit 1 + fi + echo "Backing up the bundled database..." + dump() { $COMPOSE_CMD exec -T db pg_dump -U jobsync jobsync; } +fi + +if ! dump 2>"$ERR" | gzip > "$TMP"; then + echo "Backup FAILED. No backup file was written." + cat "$ERR" + if grep -qi "server version mismatch" "$ERR"; then + echo "" + echo "Your Postgres server is newer than the dump tool. Set PG_DUMP_IMAGE in .env" + echo "to an image of the server's major version, for example PG_DUMP_IMAGE=postgres:19." + fi + exit 1 +fi + +OUT="$BACKUP_DIR/jobsync-$(date +%F-%H%M).sql.gz" +mv "$TMP" "$OUT" +echo "Backup written to $OUT ($(du -h "$OUT" | cut -f1))" diff --git a/deploy.ps1 b/deploy.ps1 index ded66a5de..525ec9717 100644 --- a/deploy.ps1 +++ b/deploy.ps1 @@ -52,6 +52,20 @@ if (-not (Test-Path ".git")) { exit 1 } +# A local edit to docker-compose.yml blocks the pull, and discarding it to +# get unstuck would throw away an inline ENCRYPTION_KEY. Stop and explain. +git diff --quiet HEAD -- docker-compose.yml +if ($LASTEXITCODE -ne 0) { + Write-Host "docker-compose.yml has local edits, which would block this update." -ForegroundColor Red + Write-Host "" + git --no-pager diff HEAD -- docker-compose.yml + Write-Host "" + Write-Host "Copy each value you changed into .env (for example ENCRYPTION_KEY=...)," -ForegroundColor Yellow + Write-Host "then discard the edit with: git checkout -- docker-compose.yml" -ForegroundColor Yellow + Write-Host "Do not discard it first: without your ENCRYPTION_KEY in .env, saved API keys stop decrypting." -ForegroundColor Red + exit 1 +} + Write-Host "[2/5] Fetching latest changes from GitHub..." -ForegroundColor Green git fetch origin Assert-LastExit "git fetch failed" diff --git a/deploy.sh b/deploy.sh index a9977d99b..35df284f4 100755 --- a/deploy.sh +++ b/deploy.sh @@ -29,11 +29,11 @@ cd "$SCRIPT_DIR" echo -e "${GREEN}[1/5]${NC} Current directory: $SCRIPT_DIR" echo "" -# Detect Docker Compose (standalone v2 or plugin) -if command -v docker-compose >/dev/null 2>&1; then - COMPOSE_CMD="docker-compose" -elif docker compose version >/dev/null 2>&1; then +# Detect Docker Compose (v2 plugin first, then the standalone binary) +if docker compose version >/dev/null 2>&1; then COMPOSE_CMD="docker compose" +elif command -v docker-compose >/dev/null 2>&1; then + COMPOSE_CMD="docker-compose" else echo -e "${RED}Error: Docker Compose not found${NC}" exit 1 @@ -48,6 +48,19 @@ if [ ! -d ".git" ]; then exit 1 fi +# A local edit to docker-compose.yml blocks the pull, and discarding it to +# get unstuck would throw away an inline ENCRYPTION_KEY. Stop and explain. +if ! git diff --quiet HEAD -- docker-compose.yml; then + echo -e "${RED}docker-compose.yml has local edits, which would block this update.${NC}" + echo "" + git --no-pager diff HEAD -- docker-compose.yml + echo "" + echo -e "Copy each value you changed into ${YELLOW}.env${NC} (for example ENCRYPTION_KEY=...)," + echo -e "then discard the edit with: ${YELLOW}git checkout -- docker-compose.yml${NC}" + echo -e "${RED}Do not discard it first: without your ENCRYPTION_KEY in .env, saved API keys stop decrypting.${NC}" + exit 1 +fi + # Fetch latest changes echo -e "${GREEN}[2/5]${NC} Fetching latest changes from GitHub..." git fetch origin diff --git a/docker-compose.lan-db.yml b/docker-compose.lan-db.yml new file mode 100644 index 000000000..889e1bcfc --- /dev/null +++ b/docker-compose.lan-db.yml @@ -0,0 +1,8 @@ +# Opt-in: publishes the bundled database on the LAN. Never on by default — +# the bundled password is printed in this public repository. +# Enable from .env: +# COMPOSE_FILE=docker-compose.yml:docker-compose.lan-db.yml +services: + db: + ports: + - "5432:5432" diff --git a/docker-compose.yml b/docker-compose.yml index 503f8819f..b0cfa1be2 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -1,4 +1,31 @@ services: + db: + # Stock Postgres 18 plus the vector extension, which nothing enables yet + image: pgvector/pgvector:pg18 + container_name: jobsync_db + # Optional BUNDLED_DB=0 in .env stops it when using your own Postgres + scale: ${BUNDLED_DB:-1} + # No ports published. docker-compose.lan-db.yml opens 5432 on request. + environment: + - POSTGRES_USER=jobsync + - POSTGRES_PASSWORD=${POSTGRES_PASSWORD:-jobsync} + - POSTGRES_DB=jobsync + # Do NOT add TZ here. DB-side timestamp defaults would then be written + # hours off from the ones Prisma writes. The app's TZ governs display. + volumes: + # 18+ mounts the parent directory, NOT /var/lib/postgresql/data: on + # that path the data lands in an anonymous volume a prune can remove. + - pgdata:/var/lib/postgresql + healthcheck: + # For `docker compose ps` only. Startup ordering belongs to the app's + # wait-for-db, which also works when this service is scaled to 0, so + # there is deliberately no depends_on. + test: ["CMD-SHELL", "pg_isready -U jobsync -d jobsync"] + interval: 5s + timeout: 5s + retries: 10 + restart: unless-stopped + app: image: ghcr.io/gsync/jobsync:latest container_name: jobsync_app @@ -7,7 +34,13 @@ services: environment: - NODE_ENV=production - MCP_ENABLED=${MCP_ENABLED:-true} - - DATABASE_URL=file:/data/dev.db + # As found; the entrypoint swaps unset or a leftover file: for bundled + - DATABASE_URL=${DATABASE_URL:-} + - BUNDLED_DATABASE_URL=postgres://jobsync:${POSTGRES_PASSWORD:-jobsync}@db:5432/jobsync + - BUNDLED_DB=${BUNDLED_DB:-1} + - CREATE_DATABASE_IF_MISSING=${CREATE_DATABASE_IF_MISSING:-} + - SKIP_IMPORT=${SKIP_IMPORT:-} + - IMPORT_DRY_RUN=${IMPORT_DRY_RUN:-} - AUTH_SECRET=${AUTH_SECRET:-your-auth-secret} - ENCRYPTION_KEY=${ENCRYPTION_KEY:-you-encryption-key-here} - NEXTAUTH_URL=${NEXTAUTH_URL:-http://localhost:3737} @@ -20,6 +53,8 @@ services: - OPENROUTER_API_KEY=${OPENROUTER_API_KEY:-} - RAPIDAPI_KEY=${RAPIDAPI_KEY:-} volumes: + # Permanent: uploaded resumes live here, and the one-time import reads + # the legacy dev.db from the same mount. - ./jobsyncdb/data:/data extra_hosts: - "host.docker.internal:host-gateway" @@ -29,4 +64,10 @@ services: interval: 30s timeout: 10s retries: 3 - start_period: 30s + # The first boot after upgrading also runs the one-time import + start_period: 120s + +volumes: + pgdata: + # Fixed, so renaming the project folder cannot orphan the data + name: jobsync_pgdata diff --git a/docker-entrypoint.sh b/docker-entrypoint.sh index 647112fa7..44ccb4e7c 100755 --- a/docker-entrypoint.sh +++ b/docker-entrypoint.sh @@ -7,9 +7,31 @@ if [ -z "$AUTH_SECRET" ]; then echo "AUTH_SECRET was not set — generated a temporary secret for this container." fi -# Run migrations as root (before switching users) +# Resolve DATABASE_URL once for every step below. A separate assignment, so +# set -e sees a failure that `export VAR="$(...)"` would hide. +RESOLVED_DATABASE_URL="$(node scripts/dist/resolve-db-url.js)" +export DATABASE_URL="$RESOLVED_DATABASE_URL" + +# Wait for Postgres; exits non-zero at once on a permanent misconfiguration +node scripts/dist/wait-for-db.js + +# Apply the schema as root (before switching users) npx -y prisma@6.19.0 migrate deploy +# One-time SQLite -> Postgres import. A no-op without a legacy dev.db, and it +# must finish before the server (and its scheduler) starts. +IMPORT_STATUS=0 +node scripts/dist/import-sqlite.js || IMPORT_STATUS=$? +# Exit 2 is a finished dry run and 1 a failed or refused import: idle, or the +# restart policy would repeat it until someone stops the container +if [ "$IMPORT_STATUS" -eq 2 ]; then + echo "Dry run finished. Remove IMPORT_DRY_RUN from .env and restart." +fi +if [ "$IMPORT_STATUS" -ne 0 ]; then + trap 'exit 0' TERM INT + while :; do sleep 3600 & wait $!; done +fi + # Fix /data permissions and run app as nextjs user chown -R nextjs:nodejs /data export HOME=/home/nextjs diff --git a/eslint.config.mjs b/eslint.config.mjs index 946bba57e..f10e92171 100644 --- a/eslint.config.mjs +++ b/eslint.config.mjs @@ -8,7 +8,7 @@ const __dirname = dirname(__filename); const compat = new FlatCompat({ baseDirectory: __dirname }); const eslintConfig = [ - { ignores: [".next/", "coverage/", "__mocks__/"] }, + { ignores: [".next/", "coverage/", "__mocks__/", "src/generated/", "scripts/dist/"] }, ...compat.extends("next/core-web-vitals"), { // Playwright's fixture `use` callback is not a React hook. diff --git a/playwright.config.ts b/playwright.config.ts index 732a62f17..e8ddd5d45 100644 --- a/playwright.config.ts +++ b/playwright.config.ts @@ -19,8 +19,8 @@ export default defineConfig({ /* Retry on CI only */ retries: process.env.CI ? 2 : 0, /* Cap local concurrency: too many parallel contexts (workers x 3 browsers) - starve the single dev server + SQLite, making combobox "Create" server - actions slow enough to time out. */ + starve the single dev server and its database, making combobox + "Create" server actions slow enough to time out. */ workers: process.env.CI ? 1 : 4, /* Heavier flows (e.g. adding a resume experience makes several sequential server-action "Create" calls) need headroom under that load. */ diff --git a/prisma/migrations/20240524022540_init/migration.sql b/prisma/migrations-sqlite-legacy/20240524022540_init/migration.sql similarity index 100% rename from prisma/migrations/20240524022540_init/migration.sql rename to prisma/migrations-sqlite-legacy/20240524022540_init/migration.sql diff --git a/prisma/migrations/20240528002543_add_string_id_to_user/migration.sql b/prisma/migrations-sqlite-legacy/20240528002543_add_string_id_to_user/migration.sql similarity index 100% rename from prisma/migrations/20240528002543_add_string_id_to_user/migration.sql rename to prisma/migrations-sqlite-legacy/20240528002543_add_string_id_to_user/migration.sql diff --git a/prisma/migrations/20240604110546_add_label_and_value/migration.sql b/prisma/migrations-sqlite-legacy/20240604110546_add_label_and_value/migration.sql similarity index 100% rename from prisma/migrations/20240604110546_add_label_and_value/migration.sql rename to prisma/migrations-sqlite-legacy/20240604110546_add_label_and_value/migration.sql diff --git a/prisma/migrations/20240612084946_add_joburl_and_applied_fields/migration.sql b/prisma/migrations-sqlite-legacy/20240612084946_add_joburl_and_applied_fields/migration.sql similarity index 100% rename from prisma/migrations/20240612084946_add_joburl_and_applied_fields/migration.sql rename to prisma/migrations-sqlite-legacy/20240612084946_add_joburl_and_applied_fields/migration.sql diff --git a/prisma/migrations/20240624230528_add_profile_resume_models/migration.sql b/prisma/migrations-sqlite-legacy/20240624230528_add_profile_resume_models/migration.sql similarity index 100% rename from prisma/migrations/20240624230528_add_profile_resume_models/migration.sql rename to prisma/migrations-sqlite-legacy/20240624230528_add_profile_resume_models/migration.sql diff --git a/prisma/migrations/20240627061730_update_resume_section_and_summary_relationship/migration.sql b/prisma/migrations-sqlite-legacy/20240627061730_update_resume_section_and_summary_relationship/migration.sql similarity index 100% rename from prisma/migrations/20240627061730_update_resume_section_and_summary_relationship/migration.sql rename to prisma/migrations-sqlite-legacy/20240627061730_update_resume_section_and_summary_relationship/migration.sql diff --git a/prisma/migrations/20241103213832_resume_upload/migration.sql b/prisma/migrations-sqlite-legacy/20241103213832_resume_upload/migration.sql similarity index 100% rename from prisma/migrations/20241103213832_resume_upload/migration.sql rename to prisma/migrations-sqlite-legacy/20241103213832_resume_upload/migration.sql diff --git a/prisma/migrations/20241210004820_add_activity_and_activty_type_table/migration.sql b/prisma/migrations-sqlite-legacy/20241210004820_add_activity_and_activty_type_table/migration.sql similarity index 100% rename from prisma/migrations/20241210004820_add_activity_and_activty_type_table/migration.sql rename to prisma/migrations-sqlite-legacy/20241210004820_add_activity_and_activty_type_table/migration.sql diff --git a/prisma/migrations/20260113163354_add_task_model/migration.sql b/prisma/migrations-sqlite-legacy/20260113163354_add_task_model/migration.sql similarity index 100% rename from prisma/migrations/20260113163354_add_task_model/migration.sql rename to prisma/migrations-sqlite-legacy/20260113163354_add_task_model/migration.sql diff --git a/prisma/migrations/20260130042742_add_automations/migration.sql b/prisma/migrations-sqlite-legacy/20260130042742_add_automations/migration.sql similarity index 100% rename from prisma/migrations/20260130042742_add_automations/migration.sql rename to prisma/migrations-sqlite-legacy/20260130042742_add_automations/migration.sql diff --git a/prisma/migrations/20260131063928_add_user_settings/migration.sql b/prisma/migrations-sqlite-legacy/20260131063928_add_user_settings/migration.sql similarity index 100% rename from prisma/migrations/20260131063928_add_user_settings/migration.sql rename to prisma/migrations-sqlite-legacy/20260131063928_add_user_settings/migration.sql diff --git a/prisma/migrations/20260212202940_add_createdby_to_jobsource/migration.sql b/prisma/migrations-sqlite-legacy/20260212202940_add_createdby_to_jobsource/migration.sql similarity index 100% rename from prisma/migrations/20260212202940_add_createdby_to_jobsource/migration.sql rename to prisma/migrations-sqlite-legacy/20260212202940_add_createdby_to_jobsource/migration.sql diff --git a/prisma/migrations/20260216_multi_user_unique_constraints/migration.sql b/prisma/migrations-sqlite-legacy/20260216_multi_user_unique_constraints/migration.sql similarity index 100% rename from prisma/migrations/20260216_multi_user_unique_constraints/migration.sql rename to prisma/migrations-sqlite-legacy/20260216_multi_user_unique_constraints/migration.sql diff --git a/prisma/migrations/20260217_add_location_unique_constraint/migration.sql b/prisma/migrations-sqlite-legacy/20260217_add_location_unique_constraint/migration.sql similarity index 100% rename from prisma/migrations/20260217_add_location_unique_constraint/migration.sql rename to prisma/migrations-sqlite-legacy/20260217_add_location_unique_constraint/migration.sql diff --git a/prisma/migrations/20260219201822_apply/migration.sql b/prisma/migrations-sqlite-legacy/20260219201822_apply/migration.sql similarity index 100% rename from prisma/migrations/20260219201822_apply/migration.sql rename to prisma/migrations-sqlite-legacy/20260219201822_apply/migration.sql diff --git a/prisma/migrations/20260302223041_add_note_model/migration.sql b/prisma/migrations-sqlite-legacy/20260302223041_add_note_model/migration.sql similarity index 100% rename from prisma/migrations/20260302223041_add_note_model/migration.sql rename to prisma/migrations-sqlite-legacy/20260302223041_add_note_model/migration.sql diff --git a/prisma/migrations/20260303014226_add_tag_model/migration.sql b/prisma/migrations-sqlite-legacy/20260303014226_add_tag_model/migration.sql similarity index 100% rename from prisma/migrations/20260303014226_add_tag_model/migration.sql rename to prisma/migrations-sqlite-legacy/20260303014226_add_tag_model/migration.sql diff --git a/prisma/migrations/20260305175020_add_question_model/migration.sql b/prisma/migrations-sqlite-legacy/20260305175020_add_question_model/migration.sql similarity index 100% rename from prisma/migrations/20260305175020_add_question_model/migration.sql rename to prisma/migrations-sqlite-legacy/20260305175020_add_question_model/migration.sql diff --git a/prisma/migrations/20260326034736_add_cover_letter/migration.sql b/prisma/migrations-sqlite-legacy/20260326034736_add_cover_letter/migration.sql similarity index 100% rename from prisma/migrations/20260326034736_add_cover_letter/migration.sql rename to prisma/migrations-sqlite-legacy/20260326034736_add_cover_letter/migration.sql diff --git a/prisma/migrations/20260626144202_add_mcp_access_token/migration.sql b/prisma/migrations-sqlite-legacy/20260626144202_add_mcp_access_token/migration.sql similarity index 100% rename from prisma/migrations/20260626144202_add_mcp_access_token/migration.sql rename to prisma/migrations-sqlite-legacy/20260626144202_add_mcp_access_token/migration.sql diff --git a/prisma/migrations/20260627071346_add_skill_section/migration.sql b/prisma/migrations-sqlite-legacy/20260627071346_add_skill_section/migration.sql similarity index 100% rename from prisma/migrations/20260627071346_add_skill_section/migration.sql rename to prisma/migrations-sqlite-legacy/20260627071346_add_skill_section/migration.sql diff --git a/prisma/migrations/20260628080148_add_automation_source_config/migration.sql b/prisma/migrations-sqlite-legacy/20260628080148_add_automation_source_config/migration.sql similarity index 100% rename from prisma/migrations/20260628080148_add_automation_source_config/migration.sql rename to prisma/migrations-sqlite-legacy/20260628080148_add_automation_source_config/migration.sql diff --git a/prisma/migrations/20260628161416_add_contact_info_urls/migration.sql b/prisma/migrations-sqlite-legacy/20260628161416_add_contact_info_urls/migration.sql similarity index 100% rename from prisma/migrations/20260628161416_add_contact_info_urls/migration.sql rename to prisma/migrations-sqlite-legacy/20260628161416_add_contact_info_urls/migration.sql diff --git a/prisma/migrations/20260628161718_add_contact_info_url_labels/migration.sql b/prisma/migrations-sqlite-legacy/20260628161718_add_contact_info_url_labels/migration.sql similarity index 100% rename from prisma/migrations/20260628161718_add_contact_info_url_labels/migration.sql rename to prisma/migrations-sqlite-legacy/20260628161718_add_contact_info_url_labels/migration.sql diff --git a/prisma/migrations/20260701071535_add_question_created_via/migration.sql b/prisma/migrations-sqlite-legacy/20260701071535_add_question_created_via/migration.sql similarity index 100% rename from prisma/migrations/20260701071535_add_question_created_via/migration.sql rename to prisma/migrations-sqlite-legacy/20260701071535_add_question_created_via/migration.sql diff --git a/prisma/migrations/20260702153647_add_user_default_resume/migration.sql b/prisma/migrations-sqlite-legacy/20260702153647_add_user_default_resume/migration.sql similarity index 100% rename from prisma/migrations/20260702153647_add_user_default_resume/migration.sql rename to prisma/migrations-sqlite-legacy/20260702153647_add_user_default_resume/migration.sql diff --git a/prisma/migrations/20260707151433_add_workplace_type_to_job/migration.sql b/prisma/migrations-sqlite-legacy/20260707151433_add_workplace_type_to_job/migration.sql similarity index 100% rename from prisma/migrations/20260707151433_add_workplace_type_to_job/migration.sql rename to prisma/migrations-sqlite-legacy/20260707151433_add_workplace_type_to_job/migration.sql diff --git a/prisma/migrations/20260710000001_automation_run_single_active/migration.sql b/prisma/migrations-sqlite-legacy/20260710000001_automation_run_single_active/migration.sql similarity index 100% rename from prisma/migrations/20260710000001_automation_run_single_active/migration.sql rename to prisma/migrations-sqlite-legacy/20260710000001_automation_run_single_active/migration.sql diff --git a/prisma/migrations/20260710000002_job_automation_url_unique/migration.sql b/prisma/migrations-sqlite-legacy/20260710000002_job_automation_url_unique/migration.sql similarity index 100% rename from prisma/migrations/20260710000002_job_automation_url_unique/migration.sql rename to prisma/migrations-sqlite-legacy/20260710000002_job_automation_url_unique/migration.sql diff --git a/prisma/migrations/20260711003153_activity_task_one_to_many/migration.sql b/prisma/migrations-sqlite-legacy/20260711003153_activity_task_one_to_many/migration.sql similarity index 100% rename from prisma/migrations/20260711003153_activity_task_one_to_many/migration.sql rename to prisma/migrations-sqlite-legacy/20260711003153_activity_task_one_to_many/migration.sql diff --git a/prisma/migrations/20260712184644_add_resume_review_data/migration.sql b/prisma/migrations-sqlite-legacy/20260712184644_add_resume_review_data/migration.sql similarity index 100% rename from prisma/migrations/20260712184644_add_resume_review_data/migration.sql rename to prisma/migrations-sqlite-legacy/20260712184644_add_resume_review_data/migration.sql diff --git a/prisma/migrations/20260722065252_job_description_completeness/migration.sql b/prisma/migrations-sqlite-legacy/20260722065252_job_description_completeness/migration.sql similarity index 100% rename from prisma/migrations/20260722065252_job_description_completeness/migration.sql rename to prisma/migrations-sqlite-legacy/20260722065252_job_description_completeness/migration.sql diff --git a/prisma/migrations/20260802221904_chat_conversation/migration.sql b/prisma/migrations-sqlite-legacy/20260802221904_chat_conversation/migration.sql similarity index 100% rename from prisma/migrations/20260802221904_chat_conversation/migration.sql rename to prisma/migrations-sqlite-legacy/20260802221904_chat_conversation/migration.sql diff --git a/prisma/migrations/20260813075758_add_activity_breaks/migration.sql b/prisma/migrations-sqlite-legacy/20260813075758_add_activity_breaks/migration.sql similarity index 100% rename from prisma/migrations/20260813075758_add_activity_breaks/migration.sql rename to prisma/migrations-sqlite-legacy/20260813075758_add_activity_breaks/migration.sql diff --git a/prisma/migrations/20260904000000_job_status_offer_outcomes/migration.sql b/prisma/migrations-sqlite-legacy/20260904000000_job_status_offer_outcomes/migration.sql similarity index 100% rename from prisma/migrations/20260904000000_job_status_offer_outcomes/migration.sql rename to prisma/migrations-sqlite-legacy/20260904000000_job_status_offer_outcomes/migration.sql diff --git a/prisma/migrations/20260904120000_remove_jsearch_rapidapi/migration.sql b/prisma/migrations-sqlite-legacy/20260904120000_remove_jsearch_rapidapi/migration.sql similarity index 100% rename from prisma/migrations/20260904120000_remove_jsearch_rapidapi/migration.sql rename to prisma/migrations-sqlite-legacy/20260904120000_remove_jsearch_rapidapi/migration.sql diff --git a/prisma/migrations/20260907153300_company_watchlist/migration.sql b/prisma/migrations-sqlite-legacy/20260907153300_company_watchlist/migration.sql similarity index 100% rename from prisma/migrations/20260907153300_company_watchlist/migration.sql rename to prisma/migrations-sqlite-legacy/20260907153300_company_watchlist/migration.sql diff --git a/prisma/migrations/20260912131248_contacts/migration.sql b/prisma/migrations-sqlite-legacy/20260912131248_contacts/migration.sql similarity index 100% rename from prisma/migrations/20260912131248_contacts/migration.sql rename to prisma/migrations-sqlite-legacy/20260912131248_contacts/migration.sql diff --git a/prisma/migrations/20260912170952_contact_default_role/migration.sql b/prisma/migrations-sqlite-legacy/20260912170952_contact_default_role/migration.sql similarity index 100% rename from prisma/migrations/20260912170952_contact_default_role/migration.sql rename to prisma/migrations-sqlite-legacy/20260912170952_contact_default_role/migration.sql diff --git a/prisma/migrations/20260919144828_job_stages_timeline/migration.sql b/prisma/migrations-sqlite-legacy/20260919144828_job_stages_timeline/migration.sql similarity index 100% rename from prisma/migrations/20260919144828_job_stages_timeline/migration.sql rename to prisma/migrations-sqlite-legacy/20260919144828_job_stages_timeline/migration.sql diff --git a/prisma/migrations/20260926120000_backfill_job_applied/migration.sql b/prisma/migrations-sqlite-legacy/20260926120000_backfill_job_applied/migration.sql similarity index 100% rename from prisma/migrations/20260926120000_backfill_job_applied/migration.sql rename to prisma/migrations-sqlite-legacy/20260926120000_backfill_job_applied/migration.sql diff --git a/prisma/migrations/20260927052959_automation_notifications/migration.sql b/prisma/migrations-sqlite-legacy/20260927052959_automation_notifications/migration.sql similarity index 100% rename from prisma/migrations/20260927052959_automation_notifications/migration.sql rename to prisma/migrations-sqlite-legacy/20260927052959_automation_notifications/migration.sql diff --git a/prisma/migrations/20260929012120_ai_usage/migration.sql b/prisma/migrations-sqlite-legacy/20260929012120_ai_usage/migration.sql similarity index 100% rename from prisma/migrations/20260929012120_ai_usage/migration.sql rename to prisma/migrations-sqlite-legacy/20260929012120_ai_usage/migration.sql diff --git a/prisma/migrations/20261002141429_company_notes_size_linkedin/migration.sql b/prisma/migrations-sqlite-legacy/20261002141429_company_notes_size_linkedin/migration.sql similarity index 100% rename from prisma/migrations/20261002141429_company_notes_size_linkedin/migration.sql rename to prisma/migrations-sqlite-legacy/20261002141429_company_notes_size_linkedin/migration.sql diff --git a/prisma/migrations-sqlite-legacy/migration_lock.toml b/prisma/migrations-sqlite-legacy/migration_lock.toml new file mode 100644 index 000000000..2a5a44419 --- /dev/null +++ b/prisma/migrations-sqlite-legacy/migration_lock.toml @@ -0,0 +1,3 @@ +# Please do not edit this file manually +# It should be added in your version-control system (e.g., Git) +provider = "sqlite" diff --git a/prisma/migrations/00000000000000_init_postgres/migration.sql b/prisma/migrations/00000000000000_init_postgres/migration.sql new file mode 100644 index 000000000..12baeaf33 --- /dev/null +++ b/prisma/migrations/00000000000000_init_postgres/migration.sql @@ -0,0 +1,988 @@ +-- CreateTable +CREATE TABLE "User" ( + "id" TEXT NOT NULL, + "name" TEXT NOT NULL, + "email" TEXT NOT NULL, + "password" TEXT NOT NULL, + "createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP, + "defaultResumeId" TEXT, + + CONSTRAINT "User_pkey" PRIMARY KEY ("id") +); + +-- CreateTable +CREATE TABLE "ApiKey" ( + "id" TEXT NOT NULL, + "userId" TEXT NOT NULL, + "provider" TEXT NOT NULL, + "encryptedKey" TEXT NOT NULL, + "iv" TEXT NOT NULL, + "last4" TEXT NOT NULL, + "label" TEXT, + "createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP, + "updatedAt" TIMESTAMP(3) NOT NULL, + "lastUsedAt" TIMESTAMP(3), + + CONSTRAINT "ApiKey_pkey" PRIMARY KEY ("id") +); + +-- CreateTable +CREATE TABLE "UserSettings" ( + "id" TEXT NOT NULL, + "userId" TEXT NOT NULL, + "settings" TEXT NOT NULL DEFAULT '{}', + "createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP, + "updatedAt" TIMESTAMP(3) NOT NULL, + + CONSTRAINT "UserSettings_pkey" PRIMARY KEY ("id") +); + +-- CreateTable +CREATE TABLE "Profile" ( + "id" TEXT NOT NULL, + "userId" TEXT NOT NULL, + + CONSTRAINT "Profile_pkey" PRIMARY KEY ("id") +); + +-- CreateTable +CREATE TABLE "Resume" ( + "id" TEXT NOT NULL, + "profileId" TEXT NOT NULL, + "title" TEXT NOT NULL, + "createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP, + "updatedAt" TIMESTAMP(3) NOT NULL, + "FileId" TEXT, + "reviewData" TEXT, + + CONSTRAINT "Resume_pkey" PRIMARY KEY ("id") +); + +-- CreateTable +CREATE TABLE "CoverLetter" ( + "id" TEXT NOT NULL, + "profileId" TEXT NOT NULL, + "title" TEXT NOT NULL, + "content" TEXT NOT NULL, + "createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP, + "updatedAt" TIMESTAMP(3) NOT NULL, + + CONSTRAINT "CoverLetter_pkey" PRIMARY KEY ("id") +); + +-- CreateTable +CREATE TABLE "File" ( + "id" TEXT NOT NULL, + "fileName" TEXT NOT NULL, + "filePath" TEXT NOT NULL, + "fileType" TEXT NOT NULL, + "uploadedAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP, + + CONSTRAINT "File_pkey" PRIMARY KEY ("id") +); + +-- CreateTable +CREATE TABLE "ResumeSection" ( + "id" TEXT NOT NULL, + "resumeId" TEXT NOT NULL, + "sectionTitle" TEXT NOT NULL, + "sectionType" TEXT NOT NULL, + "summaryId" TEXT, + + CONSTRAINT "ResumeSection_pkey" PRIMARY KEY ("id") +); + +-- CreateTable +CREATE TABLE "ContactInfo" ( + "id" TEXT NOT NULL, + "createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP, + "updatedAt" TIMESTAMP(3) NOT NULL, + "resumeId" TEXT NOT NULL, + "firstName" TEXT NOT NULL, + "lastName" TEXT NOT NULL, + "headline" TEXT NOT NULL, + "email" TEXT NOT NULL, + "phone" TEXT NOT NULL, + "address" TEXT, + "url1" TEXT, + "url1Label" TEXT, + "url2" TEXT, + "url2Label" TEXT, + + CONSTRAINT "ContactInfo_pkey" PRIMARY KEY ("id") +); + +-- CreateTable +CREATE TABLE "Summary" ( + "id" TEXT NOT NULL, + "createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP, + "updatedAt" TIMESTAMP(3) NOT NULL, + "content" TEXT NOT NULL, + + CONSTRAINT "Summary_pkey" PRIMARY KEY ("id") +); + +-- CreateTable +CREATE TABLE "WorkExperience" ( + "id" TEXT NOT NULL, + "createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP, + "updatedAt" TIMESTAMP(3) NOT NULL, + "companyId" TEXT NOT NULL, + "jobTitleId" TEXT NOT NULL, + "startDate" TIMESTAMP(3) NOT NULL, + "endDate" TIMESTAMP(3), + "description" TEXT NOT NULL, + "locationId" TEXT NOT NULL, + "resumeSectionId" TEXT, + + CONSTRAINT "WorkExperience_pkey" PRIMARY KEY ("id") +); + +-- CreateTable +CREATE TABLE "Education" ( + "id" TEXT NOT NULL, + "createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP, + "updatedAt" TIMESTAMP(3) NOT NULL, + "institution" TEXT NOT NULL, + "degree" TEXT NOT NULL, + "fieldOfStudy" TEXT NOT NULL, + "startDate" TIMESTAMP(3) NOT NULL, + "endDate" TIMESTAMP(3), + "description" TEXT, + "locationId" TEXT NOT NULL, + "resumeSectionId" TEXT, + + CONSTRAINT "Education_pkey" PRIMARY KEY ("id") +); + +-- CreateTable +CREATE TABLE "LicenseOrCertification" ( + "id" TEXT NOT NULL, + "title" TEXT NOT NULL, + "organization" TEXT NOT NULL, + "issueDate" TIMESTAMP(3), + "expirationDate" TIMESTAMP(3), + "credentialUrl" TEXT, + "resumeSectionId" TEXT, + + CONSTRAINT "LicenseOrCertification_pkey" PRIMARY KEY ("id") +); + +-- CreateTable +CREATE TABLE "OtherSection" ( + "id" TEXT NOT NULL, + "createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP, + "updatedAt" TIMESTAMP(3) NOT NULL, + "title" TEXT NOT NULL, + "content" TEXT NOT NULL, + "resumeSectionId" TEXT, + + CONSTRAINT "OtherSection_pkey" PRIMARY KEY ("id") +); + +-- CreateTable +CREATE TABLE "Contact" ( + "id" TEXT NOT NULL, + "name" TEXT NOT NULL, + "title" TEXT, + "email" TEXT, + "phone" TEXT, + "linkedinUrl" TEXT, + "companyId" TEXT, + "locationId" TEXT, + "relationship" TEXT, + "workedAtCompanyId" TEXT, + "workedFrom" TIMESTAMP(3), + "workedTo" TIMESTAMP(3), + "roleId" TEXT, + "notes" TEXT, + "lastContactedAt" TIMESTAMP(3), + "createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP, + "updatedAt" TIMESTAMP(3) NOT NULL, + "createdBy" TEXT NOT NULL, + + CONSTRAINT "Contact_pkey" PRIMARY KEY ("id") +); + +-- CreateTable +CREATE TABLE "ContactRole" ( + "id" TEXT NOT NULL, + "label" TEXT NOT NULL, + "value" TEXT NOT NULL, + "createdBy" TEXT NOT NULL, + + CONSTRAINT "ContactRole_pkey" PRIMARY KEY ("id") +); + +-- CreateTable +CREATE TABLE "JobContact" ( + "id" TEXT NOT NULL, + "jobId" TEXT NOT NULL, + "contactId" TEXT NOT NULL, + "roleId" TEXT NOT NULL, + "createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP, + + CONSTRAINT "JobContact_pkey" PRIMARY KEY ("id") +); + +-- CreateTable +CREATE TABLE "JobTitle" ( + "id" TEXT NOT NULL, + "label" TEXT NOT NULL, + "value" TEXT NOT NULL, + "createdBy" TEXT NOT NULL, + + CONSTRAINT "JobTitle_pkey" PRIMARY KEY ("id") +); + +-- CreateTable +CREATE TABLE "JobStatus" ( + "id" TEXT NOT NULL, + "label" TEXT NOT NULL, + "value" TEXT NOT NULL, + + CONSTRAINT "JobStatus_pkey" PRIMARY KEY ("id") +); + +-- CreateTable +CREATE TABLE "Location" ( + "id" TEXT NOT NULL, + "label" TEXT NOT NULL, + "value" TEXT NOT NULL, + "stateProv" TEXT, + "country" TEXT, + "createdBy" TEXT NOT NULL, + + CONSTRAINT "Location_pkey" PRIMARY KEY ("id") +); + +-- CreateTable +CREATE TABLE "Company" ( + "id" TEXT NOT NULL, + "label" TEXT NOT NULL, + "value" TEXT NOT NULL, + "logoUrl" TEXT, + "createdBy" TEXT NOT NULL, + "watched" BOOLEAN NOT NULL DEFAULT false, + "watchedAt" TIMESTAMP(3), + "atsProvider" TEXT, + "atsToken" TEXT, + "atsHost" TEXT, + "websiteUrl" TEXT, + "careersUrl" TEXT, + "industry" TEXT, + "linkedinUrl" TEXT, + "size" TEXT, + "notes" TEXT, + + CONSTRAINT "Company_pkey" PRIMARY KEY ("id") +); + +-- CreateTable +CREATE TABLE "JobSource" ( + "id" TEXT NOT NULL, + "label" TEXT NOT NULL, + "value" TEXT NOT NULL, + "createdBy" TEXT NOT NULL, + + CONSTRAINT "JobSource_pkey" PRIMARY KEY ("id") +); + +-- CreateTable +CREATE TABLE "JobStageType" ( + "id" TEXT NOT NULL, + "label" TEXT NOT NULL, + "value" TEXT NOT NULL, + "statusId" TEXT NOT NULL, + "sortOrder" INTEGER NOT NULL DEFAULT 0, + "createdBy" TEXT NOT NULL, + + CONSTRAINT "JobStageType_pkey" PRIMARY KEY ("id") +); + +-- CreateTable +CREATE TABLE "JobStage" ( + "id" TEXT NOT NULL, + "jobId" TEXT NOT NULL, + "stageTypeId" TEXT NOT NULL, + "occurredAt" TIMESTAMP(3), + "isCurrent" BOOLEAN NOT NULL DEFAULT false, + "outcome" TEXT, + "notes" TEXT, + "durationMins" INTEGER, + "format" TEXT, + "location" TEXT, + "createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP, + "updatedAt" TIMESTAMP(3) NOT NULL, + + CONSTRAINT "JobStage_pkey" PRIMARY KEY ("id") +); + +-- CreateTable +CREATE TABLE "JobStageInterviewer" ( + "id" TEXT NOT NULL, + "stageId" TEXT NOT NULL, + "contactId" TEXT NOT NULL, + "createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP, + + CONSTRAINT "JobStageInterviewer_pkey" PRIMARY KEY ("id") +); + +-- CreateTable +CREATE TABLE "JobStagePrepQuestion" ( + "id" TEXT NOT NULL, + "stageId" TEXT NOT NULL, + "questionId" TEXT NOT NULL, + "asked" BOOLEAN NOT NULL DEFAULT false, + "askedAt" TIMESTAMP(3), + "createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP, + + CONSTRAINT "JobStagePrepQuestion_pkey" PRIMARY KEY ("id") +); + +-- CreateTable +CREATE TABLE "Job" ( + "id" TEXT NOT NULL, + "userId" TEXT NOT NULL, + "jobUrl" TEXT, + "description" TEXT NOT NULL, + "jobType" TEXT NOT NULL, + "workplaceType" TEXT, + "createdAt" TIMESTAMP(3) NOT NULL, + "applied" BOOLEAN NOT NULL DEFAULT false, + "appliedDate" TIMESTAMP(3), + "dueDate" TIMESTAMP(3), + "statusId" TEXT NOT NULL, + "jobTitleId" TEXT NOT NULL, + "companyId" TEXT NOT NULL, + "jobSourceId" TEXT, + "salaryRange" TEXT, + "locationId" TEXT, + "resumeId" TEXT, + "coverLetterId" TEXT, + "automationId" TEXT, + "matchScore" INTEGER, + "matchData" TEXT, + "discoveryStatus" TEXT, + "discoveredAt" TIMESTAMP(3), + "createdVia" TEXT, + "descriptionCompleteness" TEXT, + + CONSTRAINT "Job_pkey" PRIMARY KEY ("id") +); + +-- CreateTable +CREATE TABLE "ActivityType" ( + "id" TEXT NOT NULL, + "label" TEXT NOT NULL, + "value" TEXT NOT NULL, + "description" TEXT, + "createdBy" TEXT NOT NULL, + "createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP, + "updatedAt" TIMESTAMP(3) NOT NULL, + + CONSTRAINT "ActivityType_pkey" PRIMARY KEY ("id") +); + +-- CreateTable +CREATE TABLE "Activity" ( + "id" TEXT NOT NULL, + "userId" TEXT NOT NULL, + "activityName" TEXT NOT NULL, + "startTime" TIMESTAMP(3) NOT NULL, + "endTime" TIMESTAMP(3), + "duration" INTEGER, + "breakMinutes" INTEGER NOT NULL DEFAULT 0, + "breakStartedAt" TIMESTAMP(3), + "breakPlannedMins" INTEGER, + "description" TEXT, + "createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP, + "updatedAt" TIMESTAMP(3) NOT NULL, + "activityTypeId" TEXT NOT NULL, + "taskId" TEXT, + + CONSTRAINT "Activity_pkey" PRIMARY KEY ("id") +); + +-- CreateTable +CREATE TABLE "Task" ( + "id" TEXT NOT NULL, + "userId" TEXT NOT NULL, + "title" TEXT NOT NULL, + "description" TEXT, + "status" TEXT NOT NULL DEFAULT 'in-progress', + "priority" INTEGER NOT NULL DEFAULT 5, + "percentComplete" INTEGER NOT NULL DEFAULT 0, + "dueDate" TIMESTAMP(3), + "activityTypeId" TEXT, + "createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP, + "updatedAt" TIMESTAMP(3) NOT NULL, + + CONSTRAINT "Task_pkey" PRIMARY KEY ("id") +); + +-- CreateTable +CREATE TABLE "Automation" ( + "id" TEXT NOT NULL, + "userId" TEXT NOT NULL, + "name" TEXT NOT NULL, + "jobBoard" TEXT NOT NULL, + "keywords" TEXT NOT NULL, + "location" TEXT NOT NULL, + "sourceConfig" TEXT, + "resumeId" TEXT NOT NULL, + "matchThreshold" INTEGER NOT NULL DEFAULT 80, + "scheduleHour" INTEGER NOT NULL, + "nextRunAt" TIMESTAMP(3), + "lastRunAt" TIMESTAMP(3), + "status" TEXT NOT NULL DEFAULT 'active', + "createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP, + "updatedAt" TIMESTAMP(3) NOT NULL, + + CONSTRAINT "Automation_pkey" PRIMARY KEY ("id") +); + +-- CreateTable +CREATE TABLE "AutomationRun" ( + "id" TEXT NOT NULL, + "automationId" TEXT NOT NULL, + "jobsSearched" INTEGER NOT NULL DEFAULT 0, + "jobsDeduplicated" INTEGER NOT NULL DEFAULT 0, + "jobsProcessed" INTEGER NOT NULL DEFAULT 0, + "jobsMatched" INTEGER NOT NULL DEFAULT 0, + "jobsSaved" INTEGER NOT NULL DEFAULT 0, + "status" TEXT NOT NULL DEFAULT 'running', + "errorMessage" TEXT, + "blockedReason" TEXT, + "funnelStats" TEXT, + "startedAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP, + "completedAt" TIMESTAMP(3), + + CONSTRAINT "AutomationRun_pkey" PRIMARY KEY ("id") +); + +-- CreateTable +CREATE TABLE "Notification" ( + "id" TEXT NOT NULL, + "userId" TEXT NOT NULL, + "automationId" TEXT NOT NULL, + "runId" TEXT, + "kind" TEXT NOT NULL, + "boardToken" TEXT, + "payload" TEXT NOT NULL, + "occurrences" INTEGER NOT NULL DEFAULT 1, + "occurredAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP, + "readAt" TIMESTAMP(3), + "dismissedAt" TIMESTAMP(3), + "createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP, + + CONSTRAINT "Notification_pkey" PRIMARY KEY ("id") +); + +-- CreateTable +CREATE TABLE "AiCall" ( + "id" TEXT NOT NULL, + "userId" TEXT NOT NULL, + "feature" TEXT NOT NULL, + "provider" TEXT NOT NULL, + "model" TEXT NOT NULL, + "status" TEXT NOT NULL, + "inputTokens" INTEGER, + "outputTokens" INTEGER, + "durationMs" INTEGER NOT NULL, + "firstTokenMs" INTEGER, + "numCtx" INTEGER, + "startedAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP, + + CONSTRAINT "AiCall_pkey" PRIMARY KEY ("id") +); + +-- CreateTable +CREATE TABLE "Note" ( + "id" TEXT NOT NULL, + "jobId" TEXT NOT NULL, + "userId" TEXT NOT NULL, + "content" TEXT NOT NULL, + "createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP, + "updatedAt" TIMESTAMP(3) NOT NULL, + + CONSTRAINT "Note_pkey" PRIMARY KEY ("id") +); + +-- CreateTable +CREATE TABLE "Tag" ( + "id" TEXT NOT NULL, + "label" TEXT NOT NULL, + "value" TEXT NOT NULL, + "createdBy" TEXT NOT NULL, + + CONSTRAINT "Tag_pkey" PRIMARY KEY ("id") +); + +-- CreateTable +CREATE TABLE "Skill" ( + "id" TEXT NOT NULL, + "createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP, + "updatedAt" TIMESTAMP(3) NOT NULL, + "category" TEXT, + "order" INTEGER NOT NULL DEFAULT 0, + "tagId" TEXT NOT NULL, + "resumeSectionId" TEXT NOT NULL, + + CONSTRAINT "Skill_pkey" PRIMARY KEY ("id") +); + +-- CreateTable +CREATE TABLE "Question" ( + "id" TEXT NOT NULL, + "question" TEXT NOT NULL, + "answer" TEXT, + "createdBy" TEXT NOT NULL, + "createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP, + "updatedAt" TIMESTAMP(3) NOT NULL, + "createdVia" TEXT, + + CONSTRAINT "Question_pkey" PRIMARY KEY ("id") +); + +-- CreateTable +CREATE TABLE "McpAccessToken" ( + "id" TEXT NOT NULL, + "userId" TEXT NOT NULL, + "name" TEXT NOT NULL, + "tokenHash" TEXT NOT NULL, + "tokenPrefix" TEXT NOT NULL, + "scopes" TEXT NOT NULL, + "expiresAt" TIMESTAMP(3) NOT NULL, + "lastUsedAt" TIMESTAMP(3), + "createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP, + + CONSTRAINT "McpAccessToken_pkey" PRIMARY KEY ("id") +); + +-- CreateTable +CREATE TABLE "ChatConversation" ( + "id" TEXT NOT NULL, + "userId" TEXT NOT NULL, + "messages" TEXT NOT NULL, + "createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP, + "updatedAt" TIMESTAMP(3) NOT NULL, + + CONSTRAINT "ChatConversation_pkey" PRIMARY KEY ("id") +); + +-- CreateTable +CREATE TABLE "_JobToTag" ( + "A" TEXT NOT NULL, + "B" TEXT NOT NULL, + + CONSTRAINT "_JobToTag_AB_pkey" PRIMARY KEY ("A","B") +); + +-- CreateTable +CREATE TABLE "_QuestionToTag" ( + "A" TEXT NOT NULL, + "B" TEXT NOT NULL, + + CONSTRAINT "_QuestionToTag_AB_pkey" PRIMARY KEY ("A","B") +); + +-- CreateIndex +CREATE UNIQUE INDEX "User_email_key" ON "User"("email"); + +-- CreateIndex +CREATE INDEX "ApiKey_userId_idx" ON "ApiKey"("userId"); + +-- CreateIndex +CREATE UNIQUE INDEX "ApiKey_userId_provider_key" ON "ApiKey"("userId", "provider"); + +-- CreateIndex +CREATE UNIQUE INDEX "UserSettings_userId_key" ON "UserSettings"("userId"); + +-- CreateIndex +CREATE UNIQUE INDEX "Resume_FileId_key" ON "Resume"("FileId"); + +-- CreateIndex +CREATE UNIQUE INDEX "ResumeSection_summaryId_key" ON "ResumeSection"("summaryId"); + +-- CreateIndex +CREATE UNIQUE INDEX "ContactInfo_resumeId_key" ON "ContactInfo"("resumeId"); + +-- CreateIndex +CREATE INDEX "Contact_createdBy_idx" ON "Contact"("createdBy"); + +-- CreateIndex +CREATE INDEX "Contact_companyId_idx" ON "Contact"("companyId"); + +-- CreateIndex +CREATE INDEX "Contact_workedAtCompanyId_idx" ON "Contact"("workedAtCompanyId"); + +-- CreateIndex +CREATE INDEX "Contact_roleId_idx" ON "Contact"("roleId"); + +-- CreateIndex +CREATE UNIQUE INDEX "ContactRole_value_createdBy_key" ON "ContactRole"("value", "createdBy"); + +-- CreateIndex +CREATE INDEX "JobContact_contactId_idx" ON "JobContact"("contactId"); + +-- CreateIndex +CREATE UNIQUE INDEX "JobContact_jobId_contactId_roleId_key" ON "JobContact"("jobId", "contactId", "roleId"); + +-- CreateIndex +CREATE UNIQUE INDEX "JobTitle_value_createdBy_key" ON "JobTitle"("value", "createdBy"); + +-- CreateIndex +CREATE UNIQUE INDEX "JobStatus_value_key" ON "JobStatus"("value"); + +-- CreateIndex +CREATE UNIQUE INDEX "Location_value_createdBy_key" ON "Location"("value", "createdBy"); + +-- CreateIndex +CREATE INDEX "Company_createdBy_watched_idx" ON "Company"("createdBy", "watched"); + +-- CreateIndex +CREATE UNIQUE INDEX "Company_value_createdBy_key" ON "Company"("value", "createdBy"); + +-- CreateIndex +CREATE UNIQUE INDEX "Company_createdBy_atsProvider_atsToken_key" ON "Company"("createdBy", "atsProvider", "atsToken"); + +-- CreateIndex +CREATE UNIQUE INDEX "JobSource_value_createdBy_key" ON "JobSource"("value", "createdBy"); + +-- CreateIndex +CREATE INDEX "JobStageType_createdBy_idx" ON "JobStageType"("createdBy"); + +-- CreateIndex +CREATE UNIQUE INDEX "JobStageType_value_createdBy_key" ON "JobStageType"("value", "createdBy"); + +-- CreateIndex +CREATE INDEX "JobStage_jobId_idx" ON "JobStage"("jobId"); + +-- CreateIndex +CREATE INDEX "JobStage_stageTypeId_idx" ON "JobStage"("stageTypeId"); + +-- CreateIndex +CREATE INDEX "JobStageInterviewer_contactId_idx" ON "JobStageInterviewer"("contactId"); + +-- CreateIndex +CREATE UNIQUE INDEX "JobStageInterviewer_stageId_contactId_key" ON "JobStageInterviewer"("stageId", "contactId"); + +-- CreateIndex +CREATE INDEX "JobStagePrepQuestion_questionId_idx" ON "JobStagePrepQuestion"("questionId"); + +-- CreateIndex +CREATE UNIQUE INDEX "JobStagePrepQuestion_stageId_questionId_key" ON "JobStagePrepQuestion"("stageId", "questionId"); + +-- CreateIndex +CREATE INDEX "Job_userId_automationId_idx" ON "Job"("userId", "automationId"); + +-- CreateIndex +CREATE INDEX "Job_userId_discoveryStatus_idx" ON "Job"("userId", "discoveryStatus"); + +-- CreateIndex +CREATE UNIQUE INDEX "ActivityType_value_createdBy_key" ON "ActivityType"("value", "createdBy"); + +-- CreateIndex +CREATE INDEX "Task_userId_idx" ON "Task"("userId"); + +-- CreateIndex +CREATE INDEX "Task_userId_status_idx" ON "Task"("userId", "status"); + +-- CreateIndex +CREATE INDEX "Task_userId_activityTypeId_idx" ON "Task"("userId", "activityTypeId"); + +-- CreateIndex +CREATE INDEX "Task_userId_dueDate_idx" ON "Task"("userId", "dueDate"); + +-- CreateIndex +CREATE INDEX "Automation_userId_idx" ON "Automation"("userId"); + +-- CreateIndex +CREATE INDEX "Automation_status_nextRunAt_idx" ON "Automation"("status", "nextRunAt"); + +-- CreateIndex +CREATE INDEX "AutomationRun_automationId_idx" ON "AutomationRun"("automationId"); + +-- CreateIndex +CREATE INDEX "AutomationRun_startedAt_idx" ON "AutomationRun"("startedAt"); + +-- CreateIndex +CREATE INDEX "Notification_userId_occurredAt_idx" ON "Notification"("userId", "occurredAt"); + +-- CreateIndex +CREATE INDEX "Notification_userId_readAt_idx" ON "Notification"("userId", "readAt"); + +-- CreateIndex +CREATE INDEX "Notification_automationId_kind_boardToken_idx" ON "Notification"("automationId", "kind", "boardToken"); + +-- CreateIndex +CREATE INDEX "AiCall_userId_startedAt_idx" ON "AiCall"("userId", "startedAt"); + +-- CreateIndex +CREATE INDEX "Note_jobId_idx" ON "Note"("jobId"); + +-- CreateIndex +CREATE INDEX "Note_userId_idx" ON "Note"("userId"); + +-- CreateIndex +CREATE UNIQUE INDEX "Tag_value_createdBy_key" ON "Tag"("value", "createdBy"); + +-- CreateIndex +CREATE INDEX "Skill_resumeSectionId_idx" ON "Skill"("resumeSectionId"); + +-- CreateIndex +CREATE INDEX "Question_createdBy_idx" ON "Question"("createdBy"); + +-- CreateIndex +CREATE UNIQUE INDEX "McpAccessToken_tokenHash_key" ON "McpAccessToken"("tokenHash"); + +-- CreateIndex +CREATE INDEX "McpAccessToken_userId_idx" ON "McpAccessToken"("userId"); + +-- CreateIndex +CREATE UNIQUE INDEX "McpAccessToken_userId_name_key" ON "McpAccessToken"("userId", "name"); + +-- CreateIndex +CREATE UNIQUE INDEX "ChatConversation_userId_key" ON "ChatConversation"("userId"); + +-- CreateIndex +CREATE INDEX "_JobToTag_B_index" ON "_JobToTag"("B"); + +-- CreateIndex +CREATE INDEX "_QuestionToTag_B_index" ON "_QuestionToTag"("B"); + +-- AddForeignKey +ALTER TABLE "User" ADD CONSTRAINT "User_defaultResumeId_fkey" FOREIGN KEY ("defaultResumeId") REFERENCES "Resume"("id") ON DELETE SET NULL ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "ApiKey" ADD CONSTRAINT "ApiKey_userId_fkey" FOREIGN KEY ("userId") REFERENCES "User"("id") ON DELETE RESTRICT ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "UserSettings" ADD CONSTRAINT "UserSettings_userId_fkey" FOREIGN KEY ("userId") REFERENCES "User"("id") ON DELETE RESTRICT ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "Profile" ADD CONSTRAINT "Profile_userId_fkey" FOREIGN KEY ("userId") REFERENCES "User"("id") ON DELETE RESTRICT ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "Resume" ADD CONSTRAINT "Resume_profileId_fkey" FOREIGN KEY ("profileId") REFERENCES "Profile"("id") ON DELETE RESTRICT ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "Resume" ADD CONSTRAINT "Resume_FileId_fkey" FOREIGN KEY ("FileId") REFERENCES "File"("id") ON DELETE SET NULL ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "CoverLetter" ADD CONSTRAINT "CoverLetter_profileId_fkey" FOREIGN KEY ("profileId") REFERENCES "Profile"("id") ON DELETE RESTRICT ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "ResumeSection" ADD CONSTRAINT "ResumeSection_summaryId_fkey" FOREIGN KEY ("summaryId") REFERENCES "Summary"("id") ON DELETE SET NULL ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "ResumeSection" ADD CONSTRAINT "ResumeSection_resumeId_fkey" FOREIGN KEY ("resumeId") REFERENCES "Resume"("id") ON DELETE RESTRICT ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "ContactInfo" ADD CONSTRAINT "ContactInfo_resumeId_fkey" FOREIGN KEY ("resumeId") REFERENCES "Resume"("id") ON DELETE RESTRICT ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "WorkExperience" ADD CONSTRAINT "WorkExperience_companyId_fkey" FOREIGN KEY ("companyId") REFERENCES "Company"("id") ON DELETE RESTRICT ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "WorkExperience" ADD CONSTRAINT "WorkExperience_jobTitleId_fkey" FOREIGN KEY ("jobTitleId") REFERENCES "JobTitle"("id") ON DELETE RESTRICT ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "WorkExperience" ADD CONSTRAINT "WorkExperience_locationId_fkey" FOREIGN KEY ("locationId") REFERENCES "Location"("id") ON DELETE RESTRICT ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "WorkExperience" ADD CONSTRAINT "WorkExperience_resumeSectionId_fkey" FOREIGN KEY ("resumeSectionId") REFERENCES "ResumeSection"("id") ON DELETE SET NULL ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "Education" ADD CONSTRAINT "Education_locationId_fkey" FOREIGN KEY ("locationId") REFERENCES "Location"("id") ON DELETE RESTRICT ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "Education" ADD CONSTRAINT "Education_resumeSectionId_fkey" FOREIGN KEY ("resumeSectionId") REFERENCES "ResumeSection"("id") ON DELETE SET NULL ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "LicenseOrCertification" ADD CONSTRAINT "LicenseOrCertification_resumeSectionId_fkey" FOREIGN KEY ("resumeSectionId") REFERENCES "ResumeSection"("id") ON DELETE SET NULL ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "OtherSection" ADD CONSTRAINT "OtherSection_resumeSectionId_fkey" FOREIGN KEY ("resumeSectionId") REFERENCES "ResumeSection"("id") ON DELETE SET NULL ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "Contact" ADD CONSTRAINT "Contact_companyId_fkey" FOREIGN KEY ("companyId") REFERENCES "Company"("id") ON DELETE SET NULL ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "Contact" ADD CONSTRAINT "Contact_locationId_fkey" FOREIGN KEY ("locationId") REFERENCES "Location"("id") ON DELETE SET NULL ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "Contact" ADD CONSTRAINT "Contact_workedAtCompanyId_fkey" FOREIGN KEY ("workedAtCompanyId") REFERENCES "Company"("id") ON DELETE SET NULL ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "Contact" ADD CONSTRAINT "Contact_roleId_fkey" FOREIGN KEY ("roleId") REFERENCES "ContactRole"("id") ON DELETE SET NULL ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "Contact" ADD CONSTRAINT "Contact_createdBy_fkey" FOREIGN KEY ("createdBy") REFERENCES "User"("id") ON DELETE RESTRICT ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "ContactRole" ADD CONSTRAINT "ContactRole_createdBy_fkey" FOREIGN KEY ("createdBy") REFERENCES "User"("id") ON DELETE RESTRICT ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "JobContact" ADD CONSTRAINT "JobContact_jobId_fkey" FOREIGN KEY ("jobId") REFERENCES "Job"("id") ON DELETE CASCADE ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "JobContact" ADD CONSTRAINT "JobContact_contactId_fkey" FOREIGN KEY ("contactId") REFERENCES "Contact"("id") ON DELETE CASCADE ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "JobContact" ADD CONSTRAINT "JobContact_roleId_fkey" FOREIGN KEY ("roleId") REFERENCES "ContactRole"("id") ON DELETE RESTRICT ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "JobTitle" ADD CONSTRAINT "JobTitle_createdBy_fkey" FOREIGN KEY ("createdBy") REFERENCES "User"("id") ON DELETE RESTRICT ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "Location" ADD CONSTRAINT "Location_createdBy_fkey" FOREIGN KEY ("createdBy") REFERENCES "User"("id") ON DELETE RESTRICT ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "Company" ADD CONSTRAINT "Company_createdBy_fkey" FOREIGN KEY ("createdBy") REFERENCES "User"("id") ON DELETE RESTRICT ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "JobSource" ADD CONSTRAINT "JobSource_createdBy_fkey" FOREIGN KEY ("createdBy") REFERENCES "User"("id") ON DELETE RESTRICT ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "JobStageType" ADD CONSTRAINT "JobStageType_statusId_fkey" FOREIGN KEY ("statusId") REFERENCES "JobStatus"("id") ON DELETE RESTRICT ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "JobStageType" ADD CONSTRAINT "JobStageType_createdBy_fkey" FOREIGN KEY ("createdBy") REFERENCES "User"("id") ON DELETE RESTRICT ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "JobStage" ADD CONSTRAINT "JobStage_jobId_fkey" FOREIGN KEY ("jobId") REFERENCES "Job"("id") ON DELETE CASCADE ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "JobStage" ADD CONSTRAINT "JobStage_stageTypeId_fkey" FOREIGN KEY ("stageTypeId") REFERENCES "JobStageType"("id") ON DELETE RESTRICT ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "JobStageInterviewer" ADD CONSTRAINT "JobStageInterviewer_stageId_fkey" FOREIGN KEY ("stageId") REFERENCES "JobStage"("id") ON DELETE CASCADE ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "JobStageInterviewer" ADD CONSTRAINT "JobStageInterviewer_contactId_fkey" FOREIGN KEY ("contactId") REFERENCES "Contact"("id") ON DELETE CASCADE ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "JobStagePrepQuestion" ADD CONSTRAINT "JobStagePrepQuestion_stageId_fkey" FOREIGN KEY ("stageId") REFERENCES "JobStage"("id") ON DELETE CASCADE ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "JobStagePrepQuestion" ADD CONSTRAINT "JobStagePrepQuestion_questionId_fkey" FOREIGN KEY ("questionId") REFERENCES "Question"("id") ON DELETE RESTRICT ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "Job" ADD CONSTRAINT "Job_userId_fkey" FOREIGN KEY ("userId") REFERENCES "User"("id") ON DELETE RESTRICT ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "Job" ADD CONSTRAINT "Job_statusId_fkey" FOREIGN KEY ("statusId") REFERENCES "JobStatus"("id") ON DELETE RESTRICT ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "Job" ADD CONSTRAINT "Job_jobTitleId_fkey" FOREIGN KEY ("jobTitleId") REFERENCES "JobTitle"("id") ON DELETE RESTRICT ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "Job" ADD CONSTRAINT "Job_companyId_fkey" FOREIGN KEY ("companyId") REFERENCES "Company"("id") ON DELETE RESTRICT ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "Job" ADD CONSTRAINT "Job_jobSourceId_fkey" FOREIGN KEY ("jobSourceId") REFERENCES "JobSource"("id") ON DELETE SET NULL ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "Job" ADD CONSTRAINT "Job_locationId_fkey" FOREIGN KEY ("locationId") REFERENCES "Location"("id") ON DELETE SET NULL ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "Job" ADD CONSTRAINT "Job_resumeId_fkey" FOREIGN KEY ("resumeId") REFERENCES "Resume"("id") ON DELETE SET NULL ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "Job" ADD CONSTRAINT "Job_coverLetterId_fkey" FOREIGN KEY ("coverLetterId") REFERENCES "CoverLetter"("id") ON DELETE SET NULL ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "Job" ADD CONSTRAINT "Job_automationId_fkey" FOREIGN KEY ("automationId") REFERENCES "Automation"("id") ON DELETE SET NULL ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "ActivityType" ADD CONSTRAINT "ActivityType_createdBy_fkey" FOREIGN KEY ("createdBy") REFERENCES "User"("id") ON DELETE RESTRICT ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "Activity" ADD CONSTRAINT "Activity_userId_fkey" FOREIGN KEY ("userId") REFERENCES "User"("id") ON DELETE RESTRICT ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "Activity" ADD CONSTRAINT "Activity_activityTypeId_fkey" FOREIGN KEY ("activityTypeId") REFERENCES "ActivityType"("id") ON DELETE RESTRICT ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "Activity" ADD CONSTRAINT "Activity_taskId_fkey" FOREIGN KEY ("taskId") REFERENCES "Task"("id") ON DELETE SET NULL ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "Task" ADD CONSTRAINT "Task_userId_fkey" FOREIGN KEY ("userId") REFERENCES "User"("id") ON DELETE RESTRICT ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "Task" ADD CONSTRAINT "Task_activityTypeId_fkey" FOREIGN KEY ("activityTypeId") REFERENCES "ActivityType"("id") ON DELETE SET NULL ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "Automation" ADD CONSTRAINT "Automation_userId_fkey" FOREIGN KEY ("userId") REFERENCES "User"("id") ON DELETE RESTRICT ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "Automation" ADD CONSTRAINT "Automation_resumeId_fkey" FOREIGN KEY ("resumeId") REFERENCES "Resume"("id") ON DELETE RESTRICT ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "AutomationRun" ADD CONSTRAINT "AutomationRun_automationId_fkey" FOREIGN KEY ("automationId") REFERENCES "Automation"("id") ON DELETE CASCADE ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "Notification" ADD CONSTRAINT "Notification_userId_fkey" FOREIGN KEY ("userId") REFERENCES "User"("id") ON DELETE RESTRICT ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "Notification" ADD CONSTRAINT "Notification_automationId_fkey" FOREIGN KEY ("automationId") REFERENCES "Automation"("id") ON DELETE CASCADE ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "AiCall" ADD CONSTRAINT "AiCall_userId_fkey" FOREIGN KEY ("userId") REFERENCES "User"("id") ON DELETE RESTRICT ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "Note" ADD CONSTRAINT "Note_jobId_fkey" FOREIGN KEY ("jobId") REFERENCES "Job"("id") ON DELETE CASCADE ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "Note" ADD CONSTRAINT "Note_userId_fkey" FOREIGN KEY ("userId") REFERENCES "User"("id") ON DELETE RESTRICT ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "Tag" ADD CONSTRAINT "Tag_createdBy_fkey" FOREIGN KEY ("createdBy") REFERENCES "User"("id") ON DELETE RESTRICT ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "Skill" ADD CONSTRAINT "Skill_tagId_fkey" FOREIGN KEY ("tagId") REFERENCES "Tag"("id") ON DELETE RESTRICT ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "Skill" ADD CONSTRAINT "Skill_resumeSectionId_fkey" FOREIGN KEY ("resumeSectionId") REFERENCES "ResumeSection"("id") ON DELETE RESTRICT ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "Question" ADD CONSTRAINT "Question_createdBy_fkey" FOREIGN KEY ("createdBy") REFERENCES "User"("id") ON DELETE RESTRICT ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "McpAccessToken" ADD CONSTRAINT "McpAccessToken_userId_fkey" FOREIGN KEY ("userId") REFERENCES "User"("id") ON DELETE CASCADE ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "ChatConversation" ADD CONSTRAINT "ChatConversation_userId_fkey" FOREIGN KEY ("userId") REFERENCES "User"("id") ON DELETE CASCADE ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "_JobToTag" ADD CONSTRAINT "_JobToTag_A_fkey" FOREIGN KEY ("A") REFERENCES "Job"("id") ON DELETE CASCADE ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "_JobToTag" ADD CONSTRAINT "_JobToTag_B_fkey" FOREIGN KEY ("B") REFERENCES "Tag"("id") ON DELETE CASCADE ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "_QuestionToTag" ADD CONSTRAINT "_QuestionToTag_A_fkey" FOREIGN KEY ("A") REFERENCES "Question"("id") ON DELETE CASCADE ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "_QuestionToTag" ADD CONSTRAINT "_QuestionToTag_B_fkey" FOREIGN KEY ("B") REFERENCES "Tag"("id") ON DELETE CASCADE ON UPDATE CASCADE; + + +-- Partial unique indexes. Prisma's schema cannot express these, so +-- `migrate diff` neither emits nor checks them: keep them by hand. + +-- One running/cancelling run per automation (guards the dedup race) +CREATE UNIQUE INDEX "AutomationRun_automationId_active_key" +ON "AutomationRun"("automationId") +WHERE "status" IN ('running', 'cancelling'); + +-- One discovered job per user and URL; manual jobs may reuse a URL +CREATE UNIQUE INDEX "Job_userId_jobUrl_automation_key" +ON "Job"("userId", "jobUrl") +WHERE "automationId" IS NOT NULL; + +-- One current stage per job; Job.statusId is derived from it +CREATE UNIQUE INDEX "JobStage_jobId_current_key" +ON "JobStage"("jobId") +WHERE "isCurrent" = true; diff --git a/prisma/migrations/migration_lock.toml b/prisma/migrations/migration_lock.toml index 2a5a44419..044d57cdb 100644 --- a/prisma/migrations/migration_lock.toml +++ b/prisma/migrations/migration_lock.toml @@ -1,3 +1,3 @@ # Please do not edit this file manually # It should be added in your version-control system (e.g., Git) -provider = "sqlite" +provider = "postgresql" diff --git a/prisma/schema.prisma b/prisma/schema.prisma index 222d36032..e8b681e7f 100644 --- a/prisma/schema.prisma +++ b/prisma/schema.prisma @@ -8,7 +8,7 @@ generator client { } datasource db { - provider = "sqlite" + provider = "postgresql" url = env("DATABASE_URL") } diff --git a/prisma/schema.sqlite-snapshot.prisma b/prisma/schema.sqlite-snapshot.prisma new file mode 100644 index 000000000..e44ca6574 --- /dev/null +++ b/prisma/schema.sqlite-snapshot.prisma @@ -0,0 +1,735 @@ +// FROZEN. The last SQLite-release schema, read only by scripts/import-sqlite.ts +// to copy a legacy dev.db into Postgres. Never edit the models below; delete +// this file when the import shim is retired (spec section 15). + +generator client { + provider = "prisma-client-js" + output = "../src/generated/sqlite-snapshot" +} + +datasource db { + provider = "sqlite" + url = env("SQLITE_SNAPSHOT_URL") +} + +model User { + id String @id @default(uuid()) + name String + email String @unique + password String + createdAt DateTime @default(now()) + jobsApplied Job[] + JobTitle JobTitle[] + Location Location[] + Company Company[] + JobSource JobSource[] + Contact Contact[] + ContactRole ContactRole[] + JobStageType JobStageType[] + Profile Profile[] + Activity Activity[] + ActivityType ActivityType[] + Task Task[] + Automation Automation[] + Settings UserSettings? + ApiKey ApiKey[] + Note Note[] + Tag Tag[] + Question Question[] + McpAccessToken McpAccessToken[] + Notification Notification[] + aiCalls AiCall[] + ChatConversation ChatConversation? + defaultResumeId String? + defaultResume Resume? @relation("UserDefaultResume", fields: [defaultResumeId], references: [id], onDelete: SetNull) +} + +model ApiKey { + id String @id @default(uuid()) + userId String + user User @relation(fields: [userId], references: [id]) + provider String + encryptedKey String + iv String + last4 String + label String? + createdAt DateTime @default(now()) + updatedAt DateTime @updatedAt + lastUsedAt DateTime? + + @@unique([userId, provider]) + @@index([userId]) +} + +model UserSettings { + id String @id @default(uuid()) + userId String @unique + user User @relation(fields: [userId], references: [id]) + settings String @default("{}") + createdAt DateTime @default(now()) + updatedAt DateTime @updatedAt +} + +model Profile { + id String @id @default(uuid()) + userId String + user User @relation(fields: [userId], references: [id]) + resumes Resume[] + coverLetters CoverLetter[] +} + +model Resume { + id String @id @default(uuid()) + profileId String + title String + createdAt DateTime @default(now()) + updatedAt DateTime @updatedAt + ContactInfo ContactInfo? + ResumeSections ResumeSection[] + profile Profile @relation(fields: [profileId], references: [id]) + File File? @relation(fields: [FileId], references: [id]) + FileId String? @unique + Job Job[] + Automation Automation[] + defaultForUser User[] @relation("UserDefaultResume") + reviewData String? +} + +model CoverLetter { + id String @id @default(uuid()) + profileId String + title String + content String + createdAt DateTime @default(now()) + updatedAt DateTime @updatedAt + profile Profile @relation(fields: [profileId], references: [id]) + Job Job[] +} + +model File { + id String @id @default(uuid()) + fileName String + filePath String + fileType String + uploadedAt DateTime @default(now()) + Resume Resume? +} + +model ResumeSection { + id String @id @default(uuid()) + resumeId String + sectionTitle String + sectionType String + summaryId String? @unique + summary Summary? @relation(fields: [summaryId], references: [id]) + workExperiences WorkExperience[] + educations Education[] + licenseOrCertifications LicenseOrCertification[] + others OtherSection[] + skills Skill[] + Resume Resume @relation(fields: [resumeId], references: [id]) +} + +model ContactInfo { + id String @id @default(uuid()) + createdAt DateTime @default(now()) + updatedAt DateTime @updatedAt + resumeId String @unique + firstName String + lastName String + headline String + email String + phone String + address String? + url1 String? + url1Label String? + url2 String? + url2Label String? + resume Resume @relation(fields: [resumeId], references: [id]) +} + +model Summary { + id String @id @default(uuid()) + createdAt DateTime @default(now()) + updatedAt DateTime @updatedAt + content String + ResumeSection ResumeSection? @relation +} + +model WorkExperience { + id String @id @default(uuid()) + createdAt DateTime @default(now()) + updatedAt DateTime @updatedAt + companyId String + Company Company @relation(fields: [companyId], references: [id]) + jobTitleId String + jobTitle JobTitle @relation(fields: [jobTitleId], references: [id]) + startDate DateTime + endDate DateTime? + description String + locationId String + location Location @relation(fields: [locationId], references: [id]) + resumeSectionId String? + ResumeSection ResumeSection? @relation(fields: [resumeSectionId], references: [id]) +} + +model Education { + id String @id @default(uuid()) + createdAt DateTime @default(now()) + updatedAt DateTime @updatedAt + institution String + degree String + fieldOfStudy String + startDate DateTime + endDate DateTime? + description String? + locationId String + location Location @relation(fields: [locationId], references: [id]) + resumeSectionId String? + ResumeSection ResumeSection? @relation(fields: [resumeSectionId], references: [id]) +} + +model LicenseOrCertification { + id String @id @default(uuid()) + title String + organization String + issueDate DateTime? + expirationDate DateTime? + credentialUrl String? + ResumeSection ResumeSection? @relation(fields: [resumeSectionId], references: [id]) + resumeSectionId String? +} + +model OtherSection { + id String @id @default(uuid()) + createdAt DateTime @default(now()) + updatedAt DateTime @updatedAt + title String + content String + ResumeSection ResumeSection? @relation(fields: [resumeSectionId], references: [id]) + resumeSectionId String? +} + +model Contact { + id String @id @default(uuid()) + name String + title String? + email String? + phone String? + linkedinUrl String? + + // Where they work now + companyId String? + Company Company? @relation("ContactCompany", fields: [companyId], references: [id]) + locationId String? + Location Location? @relation(fields: [locationId], references: [id]) + + // Shared history — how you know them + relationship String? + workedAtCompanyId String? + WorkedAtCompany Company? @relation("ContactWorkedAt", fields: [workedAtCompanyId], references: [id]) + workedFrom DateTime? + workedTo DateTime? + + // What this person generally is to you. The per-job role on JobContact wins + // for a specific job; this is the standing answer and the Phase 5 prefill. + roleId String? + Role ContactRole? @relation(fields: [roleId], references: [id]) + + notes String? + lastContactedAt DateTime? + + createdAt DateTime @default(now()) + updatedAt DateTime @updatedAt + createdBy String // userId + user User @relation(fields: [createdBy], references: [id]) + + jobLinks JobContact[] + stageLinks JobStageInterviewer[] + + @@index([createdBy]) + @@index([companyId]) + @@index([workedAtCompanyId]) + @@index([roleId]) +} + +model ContactRole { + id String @id @default(uuid()) + label String + value String + createdBy String // userId + user User @relation(fields: [createdBy], references: [id]) + jobContacts JobContact[] + contacts Contact[] + + @@unique([value, createdBy]) +} + +model JobContact { + id String @id @default(uuid()) + jobId String + Job Job @relation(fields: [jobId], references: [id], onDelete: Cascade) + contactId String + Contact Contact @relation(fields: [contactId], references: [id], onDelete: Cascade) + roleId String + Role ContactRole @relation(fields: [roleId], references: [id]) + createdAt DateTime @default(now()) + + @@unique([jobId, contactId, roleId]) + @@index([contactId]) +} + +model JobTitle { + id String @id @default(uuid()) + label String + value String + jobs Job[] + createdBy String // userId + user User @relation(fields: [createdBy], references: [id]) + workExperiences WorkExperience[] + + @@unique([value, createdBy]) +} + +model JobStatus { + id String @id @default(uuid()) + label String + value String @unique + jobs Job[] + stageTypes JobStageType[] +} + +model Location { + id String @id @default(uuid()) + label String + value String + stateProv String? + country String? + jobsApplied Job[] + createdBy String // userId + user User @relation(fields: [createdBy], references: [id]) + educations Education[] + workExperience WorkExperience[] + contacts Contact[] + + @@unique([value, createdBy]) +} + +model Company { + id String @id @default(uuid()) + label String + value String + jobsApplied Job[] + logoUrl String? // field is optional + createdBy String // userId + user User @relation(fields: [createdBy], references: [id]) + workExperiences WorkExperience[] + + contacts Contact[] @relation("ContactCompany") + formerColleagues Contact[] @relation("ContactWorkedAt") + + // Watch intent — independent of whether a board is linked + watched Boolean @default(false) + watchedAt DateTime? + + // Board coordinates; atsHost is Lever-only + atsProvider String? + atsToken String? + atsHost String? + + // Company attributes, unrelated to boards + websiteUrl String? + careersUrl String? + industry String? + linkedinUrl String? + size String? + notes String? + + @@unique([value, createdBy]) + @@unique([createdBy, atsProvider, atsToken]) + @@index([createdBy, watched]) +} + +model JobSource { + id String @id @default(uuid()) + label String + value String + jobsApplied Job[] + createdBy String // userId + user User @relation(fields: [createdBy], references: [id]) + + @@unique([value, createdBy]) +} + +// The kind of stage. Mirrors ContactRole in ownership and resolution, and +// JobSource in being user-extensible. statusId is both the derived-status +// source and the interview-kind discriminator (spec decisions 3 and 10). +model JobStageType { + id String @id @default(uuid()) + label String + value String + statusId String + Status JobStatus @relation(fields: [statusId], references: [id]) + sortOrder Int @default(0) + createdBy String // userId + user User @relation(fields: [createdBy], references: [id]) + stages JobStage[] + + @@unique([value, createdBy]) + @@index([createdBy]) +} + +// One entry in a job's timeline. No ownership column: a stage has no meaning +// apart from its job, so ownership traverses Job: { userId }. +model JobStage { + id String @id @default(uuid()) + jobId String + Job Job @relation(fields: [jobId], references: [id], onDelete: Cascade) + stageTypeId String + StageType JobStageType @relation(fields: [stageTypeId], references: [id]) + occurredAt DateTime? + isCurrent Boolean @default(false) + outcome String? + notes String? + durationMins Int? + format String? + location String? + createdAt DateTime @default(now()) + updatedAt DateTime @updatedAt + interviewers JobStageInterviewer[] + prepQuestions JobStagePrepQuestion[] + + @@index([jobId]) + @@index([stageTypeId]) + // Also has a hand-written partial unique index (not expressible here): + // UNIQUE(jobId) WHERE isCurrent = 1 + // See migrations/20260919000000_job_stages_timeline +} + +model JobStageInterviewer { + id String @id @default(uuid()) + stageId String + Stage JobStage @relation(fields: [stageId], references: [id], onDelete: Cascade) + contactId String + Contact Contact @relation(fields: [contactId], references: [id], onDelete: Cascade) + createdAt DateTime @default(now()) + + @@unique([stageId, contactId]) + @@index([contactId]) +} + +// questionId is deliberately NOT cascading: erasing the record that a question +// was asked in a real interview is worse than a delete that fails loudly. +model JobStagePrepQuestion { + id String @id @default(uuid()) + stageId String + Stage JobStage @relation(fields: [stageId], references: [id], onDelete: Cascade) + questionId String + Question Question @relation(fields: [questionId], references: [id]) + asked Boolean @default(false) + askedAt DateTime? + createdAt DateTime @default(now()) + + @@unique([stageId, questionId]) + @@index([questionId]) +} + +model Job { + id String @id @default(uuid()) + userId String + User User @relation(fields: [userId], references: [id]) + jobUrl String? + description String + jobType String + workplaceType String? + createdAt DateTime + applied Boolean @default(false) + appliedDate DateTime? + dueDate DateTime? + statusId String + Status JobStatus @relation(fields: [statusId], references: [id]) + JobTitle JobTitle @relation(fields: [jobTitleId], references: [id]) + jobTitleId String + Company Company @relation(fields: [companyId], references: [id]) + companyId String + JobSource JobSource? @relation(fields: [jobSourceId], references: [id]) + jobSourceId String? + salaryRange String? + Location Location? @relation(fields: [locationId], references: [id]) + locationId String? + Resume Resume? @relation(fields: [resumeId], references: [id]) + resumeId String? + CoverLetter CoverLetter? @relation(fields: [coverLetterId], references: [id]) + coverLetterId String? + Notes Note[] + tags Tag[] + contactLinks JobContact[] + stages JobStage[] + + // Automation discovery fields + automationId String? + automation Automation? @relation(fields: [automationId], references: [id], onDelete: SetNull) + matchScore Int? + matchData String? + discoveryStatus String? + discoveredAt DateTime? + + // MCP provenance: which agent/token created this job (null = UI or automation) + createdVia String? + // Word-count classification of `description` at write time: + // "title-only" | "partial" | "full". Null for pre-existing/UI jobs. + descriptionCompleteness String? + + @@index([userId, automationId]) + @@index([userId, discoveryStatus]) + // Also has a hand-written partial unique index (not expressible here): + // UNIQUE(userId, jobUrl) WHERE automationId IS NOT NULL + // See migrations/20260710000002_job_automation_url_unique +} + +model ActivityType { + id String @id @default(uuid()) + label String + value String + description String? + createdBy String // userId + user User @relation(fields: [createdBy], references: [id]) + createdAt DateTime @default(now()) + updatedAt DateTime @updatedAt + + Activities Activity[] + Tasks Task[] + + @@unique([value, createdBy]) +} + +model Activity { + id String @id @default(uuid()) + userId String + User User @relation(fields: [userId], references: [id]) + activityName String + startTime DateTime + endTime DateTime? + duration Int? + // Completed break time in minutes. Existing rows default to 0, so + // duration keeps meaning (endTime - startTime) for everything already saved. + breakMinutes Int @default(0) + // Set only while a break is open — this is the on-break flag. + breakStartedAt DateTime? + // Drives the countdown ring only; never the arithmetic. + breakPlannedMins Int? + description String? + createdAt DateTime @default(now()) + updatedAt DateTime @updatedAt + + activityType ActivityType @relation(fields: [activityTypeId], references: [id]) + activityTypeId String + + taskId String? + task Task? @relation(fields: [taskId], references: [id]) +} + +model Task { + id String @id @default(uuid()) + userId String + user User @relation(fields: [userId], references: [id]) + + title String + description String? + status String @default("in-progress") + priority Int @default(5) + percentComplete Int @default(0) + dueDate DateTime? + + activityTypeId String? + activityType ActivityType? @relation(fields: [activityTypeId], references: [id]) + + activities Activity[] + + createdAt DateTime @default(now()) + updatedAt DateTime @updatedAt + + @@index([userId]) + @@index([userId, status]) + @@index([userId, activityTypeId]) + @@index([userId, dueDate]) +} + +model Automation { + id String @id @default(uuid()) + userId String + user User @relation(fields: [userId], references: [id]) + + name String + jobBoard String + keywords String + location String + sourceConfig String? // JSON; per-source params (Greenhouse watchlist, titles) + resumeId String + resume Resume @relation(fields: [resumeId], references: [id]) + matchThreshold Int @default(80) + + scheduleHour Int + nextRunAt DateTime? + lastRunAt DateTime? + + status String @default("active") + + createdAt DateTime @default(now()) + updatedAt DateTime @updatedAt + + runs AutomationRun[] + discoveredJobs Job[] + notifications Notification[] + + @@index([userId]) + @@index([status, nextRunAt]) +} + +model AutomationRun { + id String @id @default(uuid()) + automationId String + automation Automation @relation(fields: [automationId], references: [id], onDelete: Cascade) + + jobsSearched Int @default(0) + jobsDeduplicated Int @default(0) + jobsProcessed Int @default(0) + jobsMatched Int @default(0) + jobsSaved Int @default(0) + + status String @default("running") + errorMessage String? + blockedReason String? + funnelStats String? // JSON: ordered per-source funnel for the run summary + + startedAt DateTime @default(now()) + completedAt DateTime? + + @@index([automationId]) + @@index([startedAt]) + // Also has a hand-written partial unique index (not expressible here): + // UNIQUE(automationId) WHERE status IN ('running', 'cancelling') + // See migrations/20260710000001_automation_run_single_active +} + +model Notification { + id String @id @default(uuid()) + userId String + user User @relation(fields: [userId], references: [id]) + automationId String + automation Automation @relation(fields: [automationId], references: [id], onDelete: Cascade) + runId String? // last run that produced/refreshed it; plain id, no FK + + kind String // "run" | "error" | "board" + boardToken String? // set only for kind "board"; the D2 refresh key + payload String // JSON, shape per kind in notification.model.ts + occurrences Int @default(1) + + occurredAt DateTime @default(now()) + readAt DateTime? + dismissedAt DateTime? + createdAt DateTime @default(now()) + + @@index([userId, occurredAt]) + @@index([userId, readAt]) + @@index([automationId, kind, boardToken]) +} + +model AiCall { + id String @id @default(uuid()) + userId String + user User @relation(fields: [userId], references: [id]) + feature String // AiUsageFeature in aiUsage.model.ts + provider String + model String + status String // AiCallStatus in aiUsage.model.ts + inputTokens Int? + outputTokens Int? + durationMs Int + firstTokenMs Int? + numCtx Int? // Ollama only, and only where the app passes num_ctx + startedAt DateTime @default(now()) + + @@index([userId, startedAt]) +} + +model Note { + id String @id @default(uuid()) + jobId String + job Job @relation(fields: [jobId], references: [id], onDelete: Cascade) + userId String + user User @relation(fields: [userId], references: [id]) + content String + createdAt DateTime @default(now()) + updatedAt DateTime @updatedAt + + @@index([jobId]) + @@index([userId]) +} + +model Tag { + id String @id @default(uuid()) + label String + value String + createdBy String + user User @relation(fields: [createdBy], references: [id]) + jobs Job[] + questions Question[] + skills Skill[] + + @@unique([value, createdBy]) +} + +model Skill { + id String @id @default(uuid()) + createdAt DateTime @default(now()) + updatedAt DateTime @updatedAt + category String? + order Int @default(0) + tagId String + Tag Tag @relation(fields: [tagId], references: [id]) + resumeSectionId String + ResumeSection ResumeSection @relation(fields: [resumeSectionId], references: [id]) + + @@index([resumeSectionId]) +} + +model Question { + id String @id @default(uuid()) + question String + answer String? + createdBy String + user User @relation(fields: [createdBy], references: [id]) + tags Tag[] + prepLinks JobStagePrepQuestion[] + createdAt DateTime @default(now()) + updatedAt DateTime @updatedAt + createdVia String? // MCP provenance: which agent/token created this, mirrors Job.createdVia + + @@index([createdBy]) +} + +model McpAccessToken { + id String @id @default(uuid()) + userId String + User User @relation(fields: [userId], references: [id], onDelete: Cascade) + name String + tokenHash String @unique + tokenPrefix String + scopes String // JSON array string; new tokens are always ["jobs:write","questions:write","resume:write"] + expiresAt DateTime + lastUsedAt DateTime? + createdAt DateTime @default(now()) + + @@unique([userId, name]) + @@index([userId]) +} + +model ChatConversation { + id String @id @default(uuid()) + userId String @unique + messages String // JSON-serialized UIMessage[] + createdAt DateTime @default(now()) + updatedAt DateTime @updatedAt + user User @relation(fields: [userId], references: [id], onDelete: Cascade) +} diff --git a/restore-db.sh b/restore-db.sh new file mode 100755 index 000000000..608af22e8 --- /dev/null +++ b/restore-db.sh @@ -0,0 +1,127 @@ +#!/bin/bash + +# JobSync database restore +# Usage: ./restore-db.sh +# Loads a backup-db.sh dump into an EMPTY database. Refuses one with data. + +set -euo pipefail + +SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" + +FILE="${1:-}" +if [ -z "$FILE" ] || [ ! -f "$FILE" ]; then + echo "Usage: ./restore-db.sh " + echo "Backups are in jobsyncdb/data/backups/" + exit 1 +fi +FILE="$( cd "$( dirname "$FILE" )" && pwd )/$( basename "$FILE" )" +cd "$SCRIPT_DIR" + +env_value() { + [ -f .env ] || return 0 + { grep -E "^[[:space:]]*$1=" .env || true; } | tail -n 1 | cut -d= -f2- \ + | sed -e 's/^["'\'']//' -e 's/["'\'']$//' +} + +if docker compose version >/dev/null 2>&1; then + COMPOSE_CMD="docker compose" +elif command -v docker-compose >/dev/null 2>&1; then + COMPOSE_CMD="docker-compose" +else + echo "Error: Docker Compose not found" + exit 1 +fi + +BUNDLED_DB="$(env_value BUNDLED_DB)" +DATABASE_URL="$(env_value DATABASE_URL)" +PG_DUMP_IMAGE="$(env_value PG_DUMP_IMAGE)" +PG_DUMP_IMAGE="${PG_DUMP_IMAGE:-pgvector/pgvector:pg18}" +export DATABASE_URL + +# A corrupt archive would otherwise feed psql empty input that "succeeds" +if ! gunzip -t "$FILE" 2>/dev/null; then + echo "$FILE is not a readable gzip archive. Nothing was changed." + exit 1 +fi + +# psql on stdin, in whichever mode .env selects. ON_ERROR_STOP is required: +# without it psql carries on past errors and still exits 0. +# The app's rule: unset, a leftover file: or host db all mean bundled +db_mode() { + case "$DATABASE_URL" in + postgres://*|postgresql://*) ;; + *) echo bundled; return ;; + esac + local host="${DATABASE_URL#*://}" + host="${host%%/*}" + host="${host##*@}" + host="${host%%:*}" + if [ "$host" = "db" ]; then echo bundled; else echo external; fi +} + +# pg_dump and psql reject Prisma-only URL options such as ?schema=public; +# real libpq options like sslmode are kept +libpq_url() { + printf '%s' "$DATABASE_URL" | sed -E 's/([?&])(schema|connection_limit|pool_timeout|pgbouncer|statement_cache_size|socket_timeout)=[^&]*/\1/g; s/\?&+/?/; s/&&+/\&/g; s/[?&]$//' +} + +# The reset below drops public, which would hit another app's tables when +# JobSync lives in its own schema of a shared database +SCHEMA="$(printf '%s' "$DATABASE_URL" | sed -nE 's/.*[?&]schema=([^&]*).*/\1/p')" +if [ -n "$SCHEMA" ] && [ "$SCHEMA" != "public" ]; then + echo "DATABASE_URL sets ?schema=$SCHEMA. Restore supports only the public schema." + echo "Nothing was changed." + exit 1 +fi + +if [ "$(db_mode)" = "external" ]; then + DATABASE_URL="$(libpq_url)" + run_psql() { + docker run --rm -i -e DATABASE_URL --add-host=host.docker.internal:host-gateway \ + "$PG_DUMP_IMAGE" sh -c 'psql -v ON_ERROR_STOP=1 "$@" "$DATABASE_URL"' sh "$@" + } +else + if [ "$BUNDLED_DB" = "0" ]; then + echo "BUNDLED_DB=0 is set in .env but DATABASE_URL does not point at your own Postgres: there is no database to restore into." + exit 1 + fi + run_psql() { $COMPOSE_CMD exec -T db psql -v ON_ERROR_STOP=1 -U jobsync -d jobsync "$@"; } +fi + +if ! HAS_TABLE="$(echo "SELECT count(*) FROM pg_tables WHERE schemaname = 'public' AND tablename = 'User';" | run_psql -tA)"; then + echo "Could not reach the database. Is it running? Nothing was changed." + exit 1 +fi +if [ "$HAS_TABLE" = "1" ]; then + USERS="$(echo 'SELECT count(*) FROM "User";' | run_psql -tA)" + if [ "$USERS" != "0" ]; then + echo "This database already contains data ($USERS account(s)). Restore only loads into an empty database." + echo "Nothing was changed. To replace live data on purpose, empty the database first." + exit 1 + fi +fi + +# The app is stopped so a restarting container cannot run migrations mid-load +echo "Stopping the app..." +$COMPOSE_CMD stop app + +# A fresh volume already has the tables from migrate deploy, and a plain dump +# has no DROP statements, so the schema is reset before loading. +echo "Resetting the schema..." +if ! echo "DROP SCHEMA public CASCADE; CREATE SCHEMA public;" | run_psql -q; then + echo "Could not reset the public schema. On an external server the connecting role must own it." + echo "The app is stopped. Fix the cause and re-run, or start it with: $COMPOSE_CMD start app" + exit 1 +fi + +echo "Loading $FILE..." +# One transaction: a failed load rolls back to an empty schema, never half +if ! gunzip -c "$FILE" | run_psql -q -1 >/dev/null; then + echo "Restore FAILED and was rolled back. The database is empty and the app is stopped." + echo "Fix the cause and re-run this script." + exit 1 +fi + +echo "Starting the app..." +$COMPOSE_CMD start app +echo "Restore complete." diff --git a/scripts/import-sqlite.ts b/scripts/import-sqlite.ts new file mode 100644 index 000000000..ed663912b --- /dev/null +++ b/scripts/import-sqlite.ts @@ -0,0 +1,40 @@ +import fs from "node:fs"; +import path from "node:path"; +import { PrismaClient } from "@prisma/client"; +import type * as Snapshot from "../src/generated/sqlite-snapshot"; +import { runImport } from "./import/run"; + +// Resolved from the working directory, not this file: the compiled copy in +// scripts/dist sits one level deeper than this source does. +const snapshot: typeof Snapshot = require( + path.resolve(process.cwd(), "src/generated/sqlite-snapshot"), +); + +async function main(): Promise { + const target = new PrismaClient(); + try { + return await runImport({ + env: process.env, + fileExists: (file) => fs.existsSync(file), + readText: (file) => fs.readFileSync(file, "utf8"), + writeText: (file, text) => fs.writeFileSync(file, text), + openSource: (url) => new snapshot.PrismaClient({ datasources: { db: { url } } }), + target, + // The snapshot's model list, so the copy never asks the old database + // for a model that only the newer Postgres schema has. + models: snapshot.Prisma.dmmf.datamodel.models, + log: (line) => console.log(line), + now: () => new Date(), + }); + } finally { + await target.$disconnect(); + } +} + +main().then( + (code) => process.exit(code), + (error) => { + console.error(error instanceof Error ? error.message : error); + process.exit(1); + }, +); diff --git a/scripts/import/copy.ts b/scripts/import/copy.ts new file mode 100644 index 000000000..13de05075 --- /dev/null +++ b/scripts/import/copy.ts @@ -0,0 +1,128 @@ +import { delegateName, type ImportPlan } from "./order"; +import { sanitizeRow } from "./sanitize"; + +// Postgres caps one statement at 32767 bind parameters; Job has ~35 columns, +// so 500 rows stays well under it. Do not raise this to 1000. +export const IMPORT_BATCH_SIZE = 500; + +// Structural on purpose: the real argument is a Prisma client or transaction +// client, and this module must not import either generated client. +export type CopyClient = Record; + +export interface CopyResult { + tables: { name: string; source: number; target: number }[]; + sanitized: number; +} + +type Row = Record & { id: string }; + +const reason = (error: unknown) => + error instanceof Error ? error.message : String(error); + +const joinCount = async (client: CopyClient, table: string) => { + const rows = await client.$queryRawUnsafe(`SELECT COUNT(*) AS n FROM "${table}"`); + return Number(rows[0].n); +}; + +export async function copyAll( + source: CopyClient, + target: CopyClient, + plan: ImportPlan, + log: (line: string) => void, + batchSize: number = IMPORT_BATCH_SIZE, +): Promise { + let sanitized = 0; + const pending: { model: string; id: string; field: string; value: unknown }[] = []; + + for (const model of plan.order) { + const key = delegateName(model); + const deferred = plan.deferred.filter((d) => d.model === model).map((d) => d.field); + let cursor: string | undefined; + + for (;;) { + const rows: Row[] = await source[key].findMany({ + take: batchSize, + orderBy: { id: "asc" }, + ...(cursor ? { cursor: { id: cursor }, skip: 1 } : {}), + }); + if (rows.length === 0) break; + cursor = rows[rows.length - 1].id; + + const data = rows.map((row) => { + const clean = sanitizeRow(row, (column) => { + sanitized++; + log(`Sanitised ${model}.${column} on row ${row.id} (removed characters Postgres cannot store)`); + }); + if (deferred.length === 0) return clean; + const copy: Row = { ...clean }; + for (const field of deferred) { + if (copy[field] !== null && copy[field] !== undefined) pending.push({ model, id: row.id, field, value: copy[field] }); + copy[field] = null; + } + return copy; + }); + + try { + await target[key].createMany({ data, skipDuplicates: true }); + } catch (error) { + throw new Error(`Import failed while copying ${model}: ${reason(error)}`); + } + if (rows.length < batchSize) break; + } + } + + for (const edge of pending) { + await target[delegateName(edge.model)].update({ + where: { id: edge.id }, + data: { [edge.field]: edge.value }, + }); + } + + // Implicit many-to-many tables have no delegate, so they go through raw SQL + // and must carry ON CONFLICT themselves to stay re-runnable. + for (const join of plan.joinTables) { + const pairs: { A: string; B: string }[] = await source.$queryRawUnsafe( + `SELECT "A", "B" FROM "${join.table}"`, + ); + for (let i = 0; i < pairs.length; i += batchSize) { + const chunk = pairs.slice(i, i + batchSize); + const values = chunk.map((_, n) => `($${n * 2 + 1}, $${n * 2 + 2})`).join(", "); + try { + await target.$executeRawUnsafe( + `INSERT INTO "${join.table}" ("A", "B") VALUES ${values} ON CONFLICT DO NOTHING`, + ...chunk.flatMap((pair) => [pair.A, pair.B]), + ); + } catch (error) { + throw new Error(`Import failed while copying ${join.table}: ${reason(error)}`); + } + } + } + + // ON CONFLICT can only hide a conflict, so the counts are what make it safe + const tables: CopyResult["tables"] = []; + for (const model of plan.order) { + const key = delegateName(model); + tables.push({ + name: model, + source: await source[key].count(), + target: await target[key].count(), + }); + } + for (const join of plan.joinTables) { + tables.push({ + name: join.table, + source: await joinCount(source, join.table), + target: await joinCount(target, join.table), + }); + } + + const wrong = tables.filter((t) => t.source !== t.target); + if (wrong.length > 0) { + const detail = wrong + .map((t) => `${t.name} (${t.source} in SQLite, ${t.target} in Postgres)`) + .join(", "); + throw new Error(`Row counts differ after the copy: ${detail}`); + } + + return { tables, sanitized }; +} diff --git a/scripts/import/decide.ts b/scripts/import/decide.ts new file mode 100644 index 000000000..bc4a40118 --- /dev/null +++ b/scripts/import/decide.ts @@ -0,0 +1,130 @@ +// The whole import decision is two facts: is Postgres empty, and was this +// dev.db already imported. No mtime comparison (spec section 4). + +// Newest directory in prisma/migrations-sqlite-legacy; a spec pins it there +export const LAST_SQLITE_MIGRATION = "20261002141429_company_notes_size_linkedin"; +// The image tag of the last release that still ran on SQLite +const LAST_SQLITE_IMAGE = "ghcr.io/gsync/jobsync:1.1.22"; + +export interface ImportFacts { + legacyPath: string; + legacyFileExists: boolean; + skipImport: boolean; + postgresHasUsers: boolean; + legacyHasUsers: boolean; + sentinelExists: boolean; + sentinelText: string; +} + +export type ImportDecision = + | { action: "import" } + | { action: "skip"; message: string } + | { action: "stop"; message: string }; + +export const sentinelPath = (legacyPath: string) => `${legacyPath}.imported-at`; + +// Operators see the bind mount, not the container path +export const hostPath = (containerPath: string) => + containerPath.startsWith("/data/") + ? `jobsyncdb/data/${containerPath.slice("/data/".length)}` + : containerPath; + +const importedOn = (sentinelText: string) => { + const text = sentinelText.trim(); + if (!text || Number.isNaN(Date.parse(text))) return "an earlier date"; + return new Date(text).toISOString().slice(0, 16).replace("T", " "); +}; + +const NOTHING_MODIFIED = "Nothing has been modified. Exiting."; + +export function decideImport(facts: ImportFacts): ImportDecision { + const { legacyPath } = facts; + const sentinel = hostPath(sentinelPath(legacyPath)); + + if (!facts.legacyFileExists) { + return { + action: "skip", + message: `No legacy SQLite database found at ${legacyPath}, skipping import.`, + }; + } + + if (facts.skipImport) { + const lines = ["SKIP_IMPORT=1 is set, so the SQLite import was NOT attempted."]; + if (!facts.postgresHasUsers) { + lines.push( + `Postgres is empty and ${legacyPath} exists — your existing data`, + "has not been migrated and the app is starting with no data.", + ); + } + lines.push("", "Remove SKIP_IMPORT from .env and restart once the cause is fixed."); + return { action: "skip", message: lines.join("\n") }; + } + + if (facts.postgresHasUsers && facts.sentinelExists) { + return { action: "skip", message: "SQLite data was already imported, skipping import." }; + } + + if (!facts.postgresHasUsers && facts.sentinelExists) { + // An empty dev.db has nothing a lost volume could have taken + if (!facts.legacyHasUsers) { + return { + action: "skip", + message: `${legacyPath} was already imported and has no users, skipping import.`, + }; + } + return { + action: "stop", + message: [ + `Postgres is empty, but dev.db was already imported on ${importedOn(facts.sentinelText)}.`, + "The database it was imported into appears to be missing.", + "", + "To restore a backup: ./restore-db.sh ", + " (dumps are in jobsyncdb/data/backups/)", + "To re-import the old SQLite data as it was on that date:", + ` delete ${sentinel}, restart.`, + "", + NOTHING_MODIFIED, + ].join("\n"), + }; + } + + if (facts.postgresHasUsers) { + return { + action: "stop", + message: [ + "Postgres already contains data, but dev.db has no import record —", + "its contents were never confirmed imported.", + "", + "To keep the Postgres data: create an empty file named", + ` ${sentinel}, restart.`, + "To use the SQLite data: empty the Postgres database, restart. For the", + " bundled database: docker compose down, then", + " docker volume rm jobsync_pgdata.", + "", + NOTHING_MODIFIED, + ].join("\n"), + }; + } + + return { action: "import" }; +} + +// The snapshot client describes the last SQLite schema only, so a dev.db +// from an older release would fail mid-copy with an opaque column error. +export function checkLegacySchema(appliedMigrations: readonly string[]): string | null { + if (appliedMigrations.includes(LAST_SQLITE_MIGRATION)) return null; + return [ + "This dev.db comes from an older JobSync release and cannot be imported directly.", + `It is missing the migration ${LAST_SQLITE_MIGRATION}.`, + "", + "Bring it up to date by starting the last SQLite release once. From the", + "folder that holds docker-compose.yml, with JobSync stopped, run:", + "", + ` docker run --rm -v "$(pwd)/jobsyncdb/data:/data" -e DATABASE_URL=file:/data/dev.db ${LAST_SQLITE_IMAGE}`, + "", + "Wait for its server to report it is ready, press Ctrl+C, then start JobSync", + "again. That command lets the old release migrate dev.db; this one never does.", + "", + NOTHING_MODIFIED, + ].join("\n"); +} diff --git a/scripts/import/order.ts b/scripts/import/order.ts new file mode 100644 index 000000000..850b9d27e --- /dev/null +++ b/scripts/import/order.ts @@ -0,0 +1,105 @@ +// Table order for the one-time SQLite -> Postgres copy, derived from the +// Prisma model list so a new model cannot be silently left out. + +export interface DmmfField { + name: string; + kind: string; + type: string; + isList: boolean; + isId?: boolean; + relationName?: string | null; + relationFromFields?: readonly string[] | null; +} + +export interface DmmfModel { + name: string; + fields: readonly DmmfField[]; +} + +export interface DeferredEdge { + model: string; + field: string; +} + +export interface JoinTable { + table: string; + a: string; + b: string; +} + +export interface ImportPlan { + order: string[]; + joinTables: JoinTable[]; + deferred: DeferredEdge[]; +} + +// User -> Resume -> Profile -> User is the schema's one cycle. This edge is +// nullable, so it is copied as null and set once Resume has landed. +export const DEFERRED_EDGES: readonly DeferredEdge[] = [ + { model: "User", field: "defaultResumeId" }, +]; + +export const delegateName = (model: string) => + model[0].toLowerCase() + model.slice(1); + +const isDeferred = (model: string, from: readonly string[]) => + DEFERRED_EDGES.some((e) => e.model === model && from.includes(e.field)); + +export function buildImportPlan(models: readonly DmmfModel[]): ImportPlan { + const byName = new Map(models.map((m) => [m.name, m])); + const needs = new Map>(); + const joins = new Map(); + + for (const model of models) { + const targets = new Set(); + for (const field of model.fields) { + if (field.kind !== "object") continue; + const from = field.relationFromFields ?? []; + + if (from.length > 0) { + if (isDeferred(model.name, from)) continue; + if (field.type === model.name) { + throw new Error( + `Self-referencing foreign key on ${model.name}.${field.name} — the import cannot order it.`, + ); + } + targets.add(field.type); + continue; + } + + // A list with no FK on either side is an implicit many-to-many, kept + // by Prisma in a hidden _ table that has no client delegate. + if (!field.isList || !field.relationName) continue; + const other = byName + .get(field.type) + ?.fields.find( + (f) => + f.kind === "object" && + f.relationName === field.relationName && + f.name !== field.name, + ); + if (!other?.isList || (other.relationFromFields ?? []).length > 0) continue; + const [a, b] = [model.name, field.type].sort(); + joins.set(field.relationName, { table: `_${field.relationName}`, a, b }); + } + needs.set(model.name, targets); + } + + const order: string[] = []; + while (needs.size > 0) { + const ready = [...needs.keys()].filter((name) => + [...needs.get(name)!].every((target) => order.includes(target)), + ); + if (ready.length === 0) { + throw new Error( + `Foreign-key cycle among: ${[...needs.keys()].join(", ")}. Only User.defaultResumeId is deferred; a second cycle is a schema decision, not something the import guesses.`, + ); + } + for (const name of ready) { + order.push(name); + needs.delete(name); + } + } + + return { order, joinTables: [...joins.values()], deferred: [...DEFERRED_EDGES] }; +} diff --git a/scripts/import/run.ts b/scripts/import/run.ts new file mode 100644 index 000000000..97bd4950e --- /dev/null +++ b/scripts/import/run.ts @@ -0,0 +1,126 @@ +import { copyAll, type CopyClient, type CopyResult } from "./copy"; +import { checkLegacySchema, decideImport, hostPath, sentinelPath } from "./decide"; +import { buildImportPlan, type DmmfModel } from "./order"; + +export interface ImportDeps { + env: Record; + fileExists(path: string): boolean; + readText(path: string): string; + writeText(path: string, text: string): void; + openSource(url: string): CopyClient; + target: CopyClient; + models: readonly DmmfModel[]; + log(line: string): void; + now(): Date; +} + +const DEFAULT_LEGACY_PATH = "/data/dev.db"; +// One transaction holds the whole copy, well past Prisma's 5s default +const TRANSACTION_OPTIONS = { timeout: 300_000, maxWait: 300_000 }; + +class DryRunRollback extends Error { + constructor(readonly result: CopyResult) { + super("dry run"); + } +} + +const reason = (error: unknown) => + error instanceof Error ? error.message : String(error); + +function report(result: CopyResult, log: (line: string) => void) { + for (const table of result.tables) { + log(`${table.name}: ${table.source} → ${table.target} ✓`); + } + const rows = result.tables.reduce((sum, t) => sum + t.target, 0); + log( + `Import complete: ${rows} rows across ${result.tables.length} tables, ${result.sanitized} values sanitised.`, + ); +} + +async function countLegacyUsers(deps: ImportDeps, legacyPath: string) { + const source = deps.openSource(`file:${legacyPath}`); + try { + return await source.user.count(); + } finally { + await source.$disconnect?.(); + } +} + +// Returns the process exit code: 0 continue to boot, 1 stop, 2 dry run done +export async function runImport(deps: ImportDeps): Promise { + const { env, log } = deps; + const legacyPath = env.SQLITE_LEGACY_PATH || DEFAULT_LEGACY_PATH; + const sentinel = sentinelPath(legacyPath); + const legacyFileExists = deps.fileExists(legacyPath); + const sentinelExists = legacyFileExists && deps.fileExists(sentinel); + const postgresHasUsers = legacyFileExists + ? (await deps.target.user.count()) > 0 + : false; + // Only the lost-volume case reads dev.db before deciding + const legacyHasUsers = + sentinelExists && !postgresHasUsers + ? (await countLegacyUsers(deps, legacyPath)) > 0 + : true; + + const decision = decideImport({ + legacyPath, + legacyFileExists, + skipImport: env.SKIP_IMPORT === "1", + postgresHasUsers, + legacyHasUsers, + sentinelExists, + sentinelText: sentinelExists ? deps.readText(sentinel) : "", + }); + + if (decision.action !== "import") { + log(decision.message); + return decision.action === "skip" ? 0 : 1; + } + + const source = deps.openSource(`file:${legacyPath}`); + try { + const applied: { migration_name: string }[] = await source.$queryRawUnsafe( + 'SELECT migration_name FROM "_prisma_migrations" WHERE finished_at IS NOT NULL', + ); + const refusal = checkLegacySchema(applied.map((row) => row.migration_name)); + if (refusal) { + log(refusal); + return 1; + } + + const dryRun = env.IMPORT_DRY_RUN === "1"; + const plan = buildImportPlan(deps.models); + log(`Importing ${legacyPath} into Postgres${dryRun ? " (dry run)" : ""}…`); + + let result: CopyResult; + try { + result = await deps.target.$transaction(async (tx: CopyClient) => { + const copied = await copyAll(source, tx, plan, log); + if (dryRun) throw new DryRunRollback(copied); + return copied; + }, TRANSACTION_OPTIONS); + } catch (error) { + if (error instanceof DryRunRollback) { + report(error.result, log); + log("Dry run complete — nothing was written. Remove IMPORT_DRY_RUN from .env and restart to import for real."); + return 2; + } + log(reason(error)); + log("Postgres was left empty and dev.db was not modified. Fix the cause and restart to retry, or set SKIP_IMPORT=1 once to start the app without importing."); + return 1; + } + + report(result, log); + + try { + deps.writeText(sentinel, `${deps.now().toISOString()}\n`); + } catch (error) { + log(`The import succeeded, but its record could not be written (${reason(error)}).`); + log(`Create an empty file named ${hostPath(sentinel)} and restart. Without it the next start cannot tell that dev.db was already imported.`); + return 1; + } + return 0; + } finally { + await source.$disconnect?.(); + } +} diff --git a/scripts/import/sanitize.ts b/scripts/import/sanitize.ts new file mode 100644 index 000000000..2b490c60e --- /dev/null +++ b/scripts/import/sanitize.ts @@ -0,0 +1,24 @@ +// Postgres text rejects NUL and lone surrogates; SQLite TEXT and JS strings +// accept both, and PDF extraction and scraped postings produce them. +const UNSTORABLE = + /\u0000|[\uD800-\uDBFF](?![\uDC00-\uDFFF])|(?>( + row: T, + onChange: (column: string) => void, +): T { + let clean: Record | null = null; + for (const [column, value] of Object.entries(row)) { + if (typeof value !== "string") continue; + const next = sanitizeString(value); + if (next === value) continue; + clean ??= { ...row }; + clean[column] = next; + onChange(column); + } + return (clean ?? row) as T; +} diff --git a/scripts/resolve-db-url.ts b/scripts/resolve-db-url.ts new file mode 100644 index 000000000..b3526e83f --- /dev/null +++ b/scripts/resolve-db-url.ts @@ -0,0 +1,6 @@ +import { resolveDatabaseUrl } from "./wait/resolve"; + +// stdout is captured by the entrypoint; only the warning reaches the log +const { url, warning } = resolveDatabaseUrl(process.env); +if (warning) console.error(warning); +process.stdout.write(url); diff --git a/scripts/tsconfig.json b/scripts/tsconfig.json new file mode 100644 index 000000000..e7f011d07 --- /dev/null +++ b/scripts/tsconfig.json @@ -0,0 +1,15 @@ +{ + "compilerOptions": { + "target": "ES2022", + "module": "node16", + "moduleResolution": "node16", + "lib": ["ES2022"], + "types": ["node"], + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "rootDir": ".", + "outDir": "dist" + }, + "files": ["resolve-db-url.ts", "wait-for-db.ts", "import-sqlite.ts"] +} diff --git a/scripts/wait-for-db.ts b/scripts/wait-for-db.ts new file mode 100644 index 000000000..632d00b92 --- /dev/null +++ b/scripts/wait-for-db.ts @@ -0,0 +1,41 @@ +import { lookup } from "node:dns/promises"; +import { PrismaClient } from "@prisma/client"; +import { waitForDb } from "./wait/run"; + +const client = (url: string) => new PrismaClient({ datasources: { db: { url } } }); + +waitForDb({ + env: process.env, + // $connect(), never a throwaway query: a lazy first query loses errorCode + connect: async (url) => { + const prisma = client(url); + try { + await prisma.$connect(); + } finally { + await prisma.$disconnect().catch(() => {}); + } + }, + // One standalone statement: CREATE DATABASE cannot run in a transaction + createDatabase: async (url, quotedName) => { + const prisma = client(url); + try { + await prisma.$executeRawUnsafe(`CREATE DATABASE ${quotedName}`); + } finally { + await prisma.$disconnect().catch(() => {}); + } + }, + resolves: (host) => + lookup(host).then( + () => true, + () => false, + ), + sleep: (ms) => new Promise((resolve) => setTimeout(resolve, ms)), + now: () => Date.now(), + log: (line) => console.log(line), +}).then( + (code) => process.exit(code), + (error) => { + console.error(error instanceof Error ? error.message : error); + process.exit(1); + }, +); diff --git a/scripts/wait/classify.ts b/scripts/wait/classify.ts new file mode 100644 index 000000000..a483753fa --- /dev/null +++ b/scripts/wait/classify.ts @@ -0,0 +1,140 @@ +// toWikiPageName capitalises every word of the file name, "To" included +export const UPGRADE_GUIDE_URL = "https://github.com/Gsync/jobsync/wiki/Upgrading-To-Postgres"; + +export interface Target { + host: string; + port: string; + database: string; +} + +export type UrlCheck = { ok: true; target: Target } | { ok: false; message: string }; + +const MALFORMED = + "DATABASE_URL is malformed. Expected postgres://user:password@host:5432/database — a password containing @ : / or # must be percent-encoded."; + +export function checkUrl(raw: string | undefined): UrlCheck { + if (!raw || raw.trim() === "") return { ok: false, message: "DATABASE_URL is not set." }; + + // Checked before Prisma sees it: P1012 covers this and every other bad + // string. Reached only with BUNDLED_DB=0 or an older compose file. + if (raw.startsWith("file:")) { + return { + ok: false, + message: [ + `DATABASE_URL still points at SQLite (${raw}).`, + `This release requires Postgres — see ${UPGRADE_GUIDE_URL}.`, + "Set DATABASE_URL to your own Postgres, or remove BUNDLED_DB=0", + "to use the bundled database.", + "Your existing database has NOT been modified.", + ].join("\n"), + }; + } + + let url: URL; + try { + url = new URL(raw); + } catch { + return { ok: false, message: MALFORMED }; + } + const database = decodeURIComponent(url.pathname.replace(/^\//, "")); + if (!/^postgres(ql)?:$/.test(url.protocol) || !url.hostname || !database) { + return { ok: false, message: MALFORMED }; + } + return { ok: true, target: { host: url.hostname, port: url.port || "5432", database } }; +} + +export type Verdict = "retry" | "auth" | "missing-database" | "bad-url"; + +// Anything unlisted retries: the window is bounded, so an unknown code still +// ends in a non-zero exit, only with a coherent log. +export function classify(errorCode: string | undefined): Verdict { + switch (errorCode) { + case "P1000": + return "auth"; + case "P1003": + return "missing-database"; + case "P1012": + return "bad-url"; + default: + return "retry"; + } +} + +export function maintenanceUrl(raw: string): string { + const url = new URL(raw); + url.pathname = "/postgres"; + return url.toString(); +} + +export const quoteIdent = (name: string) => `"${name.replace(/"/g, '""')}"`; + +// Driver messages can echo the connection string. Only the :password@ form +// is masked, since the default password "jobsync" is also the user and db name +export function redact(message: string, raw: string): string { + let password = ""; + try { + password = new URL(raw).password; + } catch { + return message; + } + if (!password) return message; + const masked = message.split(`:${password}@`).join(":***@"); + // A stray % (p%zz) is a valid URL but not a valid escape + try { + return masked.split(`:${decodeURIComponent(password)}@`).join(":***@"); + } catch { + return masked; + } +} + +export const MALFORMED_MESSAGE = MALFORMED; + +export function authMessage(target: Target): string { + const head = `Authentication failed for the database at ${target.host}:${target.port}.`; + if (target.host === "db") { + return [ + head, + "", + "The most likely cause is that POSTGRES_PASSWORD was changed after the database volume was created.", + "That variable only applies the first time the volume is initialised. Either change it back, or run", + "ALTER USER jobsync PASSWORD '' inside the jobsync_db container.", + ].join("\n"); + } + return [ + head, + "", + "Check DATABASE_URL: a password containing @ : / or # must be percent-encoded.", + "On a self-managed server, also check pg_hba.conf has an entry for this host's address.", + ].join("\n"); +} + +export function createdMessage(target: Target): string { + return [ + `Database "${target.database}" did not exist on ${target.host}:${target.port} — created it.`, + "If that name looks wrong, check DATABASE_URL: a typo in the database", + "name will silently produce a new, empty database rather than an error.", + ].join("\n"); +} + +export function missingDatabaseMessage(target: Target, why: string | null): string { + const head = why + ? [`Database "${target.database}" does not exist on ${target.host}:${target.port}, and JobSync`, `could not create it: ${why}`] + : [`Database "${target.database}" does not exist on ${target.host}:${target.port}.`]; + return [ + ...head, + "", + "Create it manually with:", + ` CREATE DATABASE ${target.database} OWNER ;`, + "", + "On Postgres 15+, if the role does not own the database, also:", + " GRANT ALL ON SCHEMA public TO ;", + ].join("\n"); +} + +export const BUNDLED_HOST_MESSAGE = [ + 'DATABASE_URL points at host "db", but no such host exists on the', + "Docker network — the bundled database container is not running.", + "", + "If BUNDLED_DB=0 is set in your .env, either remove it (to use the", + "bundled database) or set DATABASE_URL to your own Postgres.", +].join("\n"); diff --git a/scripts/wait/resolve.ts b/scripts/wait/resolve.ts new file mode 100644 index 000000000..95cfe3272 --- /dev/null +++ b/scripts/wait/resolve.ts @@ -0,0 +1,23 @@ +export interface Resolved { + url: string; + warning: string | null; +} + +// A file: url can never be valid on Postgres, so a leftover one means "unset" +export function resolveDatabaseUrl(env: Record): Resolved { + const raw = env.DATABASE_URL ?? ""; + const bundled = env.BUNDLED_DATABASE_URL; + // An older compose file passes no bundled address: leave it to checkUrl + if (!bundled) return { url: raw, warning: null }; + if (raw.trim() === "") return { url: bundled, warning: null }; + if (raw.startsWith("file:") && env.BUNDLED_DB !== "0") { + return { + url: bundled, + warning: [ + `Ignoring DATABASE_URL=${raw} — SQLite is no longer supported.`, + "Using the bundled Postgres database. You can delete that line from .env.", + ].join("\n"), + }; + } + return { url: raw, warning: null }; +} diff --git a/scripts/wait/run.ts b/scripts/wait/run.ts new file mode 100644 index 000000000..ecd02ea3d --- /dev/null +++ b/scripts/wait/run.ts @@ -0,0 +1,92 @@ +import { + BUNDLED_HOST_MESSAGE, + MALFORMED_MESSAGE, + authMessage, + checkUrl, + classify, + createdMessage, + maintenanceUrl, + missingDatabaseMessage, + quoteIdent, + redact, +} from "./classify"; + +export interface WaitDeps { + env: Record; + // Must reject with the driver error untouched: errorCode is only present + // when the failure came from $connect(), never from a lazy first query. + connect(url: string): Promise; + createDatabase(maintenanceUrl: string, quotedName: string): Promise; + resolves(host: string): Promise; + sleep(ms: number): Promise; + now(): number; + log(line: string): void; +} + +const WAIT_WINDOW_MS = 60_000; +const MAX_BACKOFF_MS = 5_000; + +const reason = (error: unknown, raw: string) => + redact(error instanceof Error ? error.message : String(error), raw); + +// Returns the process exit code +export async function waitForDb(deps: WaitDeps): Promise { + const raw = deps.env.DATABASE_URL; + const checked = checkUrl(raw); + if (!checked.ok) { + deps.log(checked.message); + return 1; + } + const url = raw as string; + const { target } = checked; + const where = `${target.host}:${target.port}`; + const deadline = deps.now() + WAIT_WINDOW_MS; + let attempt = 0; + let created = false; + + for (;;) { + try { + await deps.connect(url); + deps.log(`Database reachable at ${where}/${target.database}.`); + return 0; + } catch (error) { + const verdict = classify((error as { errorCode?: string }).errorCode); + + if (verdict === "auth") { + deps.log(authMessage(target)); + return 1; + } + if (verdict === "bad-url") { + deps.log(MALFORMED_MESSAGE); + return 1; + } + if (verdict === "missing-database") { + if (created || deps.env.CREATE_DATABASE_IF_MISSING === "0") { + deps.log(missingDatabaseMessage(target, null)); + return 1; + } + try { + await deps.createDatabase(maintenanceUrl(url), quoteIdent(target.database)); + } catch (createError) { + deps.log(missingDatabaseMessage(target, reason(createError, url))); + return 1; + } + created = true; + deps.log(createdMessage(target)); + continue; + } + + if (deps.now() >= deadline) { + if (target.host === "db" && !(await deps.resolves("db"))) { + deps.log(BUNDLED_HOST_MESSAGE); + } else { + deps.log(`Could not reach the database at ${where} within 60 seconds. Last error: ${reason(error, url)}`); + } + return 1; + } + attempt++; + deps.log(`waiting for database at ${where}… (attempt ${attempt})`); + await deps.sleep(Math.min(attempt * 1000, MAX_BACKOFF_MS)); + } + } +} diff --git a/src/actions/activity/activities.ts b/src/actions/activity/activities.ts index c9036fcf9..92ecab85f 100644 --- a/src/actions/activity/activities.ts +++ b/src/actions/activity/activities.ts @@ -37,9 +37,9 @@ export const getActivitiesList = async ( if (search) { whereClause.OR = [ - { activityName: { contains: search } }, - { description: { contains: search } }, - { activityType: { label: { contains: search } } }, + { activityName: { contains: search, mode: "insensitive" } }, + { description: { contains: search, mode: "insensitive" } }, + { activityType: { label: { contains: search, mode: "insensitive" } } }, ]; } @@ -47,9 +47,7 @@ export const getActivitiesList = async ( prisma.activity.findMany({ where: whereClause, select: ACTIVITY_LIST_SELECT, - orderBy: { - createdAt: "desc", - }, + orderBy: [{ createdAt: "desc" }, { id: "desc" }], skip: offset, take: limit, }), diff --git a/src/actions/activity/activityTypes.ts b/src/actions/activity/activityTypes.ts index 23eb8327c..add1104ea 100644 --- a/src/actions/activity/activityTypes.ts +++ b/src/actions/activity/activityTypes.ts @@ -12,6 +12,7 @@ export const getAllActivityTypes = async (): Promise => { where: { createdBy: user.id, }, + orderBy: [{ createdAt: "asc" }, { id: "asc" }], }); return activityTypes; } catch (error) { @@ -68,6 +69,7 @@ export const getActivityTypeList = async ( // Fetch all activity types with counts, then sort by total duration const allTypes = await prisma.activityType.findMany({ where: whereClause, + orderBy: [{ createdAt: "asc" }, { id: "asc" }], select: { id: true, label: true, diff --git a/src/actions/apiKey.actions.ts b/src/actions/apiKey.actions.ts index d65444357..0e3ffb9bc 100644 --- a/src/actions/apiKey.actions.ts +++ b/src/actions/apiKey.actions.ts @@ -21,6 +21,7 @@ export async function getUserApiKeys(): Promise<{ const keys = await db.apiKey.findMany({ where: { userId: user.id }, + orderBy: [{ createdAt: "asc" }, { id: "asc" }], select: { id: true, provider: true, diff --git a/src/actions/automation/discoveredJobs.ts b/src/actions/automation/discoveredJobs.ts index a1106e4a6..69b534a65 100644 --- a/src/actions/automation/discoveredJobs.ts +++ b/src/actions/automation/discoveredJobs.ts @@ -61,7 +61,10 @@ export async function getDiscoveredJobs(options?: { where, skip, take: limit, - orderBy: { [sortBy]: sortOrder }, + orderBy: [ + { [sortBy]: { sort: sortOrder, nulls: "last" } }, + { id: "desc" }, + ], include: { automation: { select: { id: true, name: true }, diff --git a/src/actions/automation/queries.ts b/src/actions/automation/queries.ts index 03a3882a5..57ab004cc 100644 --- a/src/actions/automation/queries.ts +++ b/src/actions/automation/queries.ts @@ -25,7 +25,7 @@ export async function getAutomationsList( where: { userId: user.id }, skip, take: limit, - orderBy: { createdAt: "desc" }, + orderBy: [{ createdAt: "desc" }, { id: "desc" }], include: { resume: { select: { id: true, title: true }, diff --git a/src/actions/automation/runs.ts b/src/actions/automation/runs.ts index 86118850c..a65412f69 100644 --- a/src/actions/automation/runs.ts +++ b/src/actions/automation/runs.ts @@ -37,7 +37,7 @@ export async function getAutomationRuns( where: { automationId }, skip, take: limit, - orderBy: { startedAt: "desc" }, + orderBy: [{ startedAt: "desc" }, { id: "desc" }], }), db.automationRun.count({ where: { automationId } }), ]); diff --git a/src/actions/company/queries.ts b/src/actions/company/queries.ts index b71c65201..1064113fb 100644 --- a/src/actions/company/queries.ts +++ b/src/actions/company/queries.ts @@ -43,7 +43,10 @@ export const getCompanyList = async ( ...(watchlist ? { extraWhere: { watched: true }, - orderBy: [{ watchedAt: "desc" }, { label: "asc" }], + orderBy: [ + { watchedAt: { sort: "desc", nulls: "last" } }, + { label: "asc" }, + ], } : {}), searchFields: ["label", "atsToken"], @@ -66,6 +69,7 @@ export const getAllCompanies = async (): Promise => { where: { createdBy: user.id, }, + orderBy: [{ label: "asc" }, { id: "asc" }], }); return companies; } catch (error) { diff --git a/src/actions/contact/queries.ts b/src/actions/contact/queries.ts index 0c4af44e5..0723f1841 100644 --- a/src/actions/contact/queries.ts +++ b/src/actions/contact/queries.ts @@ -20,9 +20,9 @@ export const getContactList = async ( const where: any = { createdBy: user.id }; if (search) { where.OR = [ - { name: { contains: search } }, - { email: { contains: search } }, - { title: { contains: search } }, + { name: { contains: search, mode: "insensitive" } }, + { email: { contains: search, mode: "insensitive" } }, + { title: { contains: search, mode: "insensitive" } }, ]; } // Either the standing role on the person or a role they hold on some job, @@ -41,7 +41,7 @@ export const getContactList = async ( // Name only. The spec's cold-contact use for lastContactedAt implies a // sort, but a sortable header means an orderBy threaded through the // action and the list hook plus a nulls-last rule — deferred to v2. - orderBy: [{ name: "asc" }], + orderBy: [{ name: "asc" }, { id: "desc" }], }), prisma.contact.count({ where }), ]); diff --git a/src/actions/contactRole.actions.ts b/src/actions/contactRole.actions.ts index 73b722eb5..23e4dc39b 100644 --- a/src/actions/contactRole.actions.ts +++ b/src/actions/contactRole.actions.ts @@ -28,7 +28,7 @@ export const getContactRoleList = async ( const user = await requireUser(); const whereClause: any = { createdBy: user.id }; if (search) { - whereClause.label = { contains: search }; + whereClause.label = { contains: search, mode: "insensitive" }; } const [data, total] = await Promise.all([ @@ -37,7 +37,7 @@ export const getContactRoleList = async ( skip: (page - 1) * limit, take: limit, include: { _count: { select: { jobContacts: true, contacts: true } } }, - orderBy: [{ label: "asc" }], + orderBy: [{ label: "asc" }, { id: "desc" }], }), prisma.contactRole.count({ where: whereClause }), ]); diff --git a/src/actions/coverLetter.actions.ts b/src/actions/coverLetter.actions.ts index e6a1c937b..97498c51c 100644 --- a/src/actions/coverLetter.actions.ts +++ b/src/actions/coverLetter.actions.ts @@ -40,9 +40,7 @@ export const getCoverLetterList = async ( }, }, }, - orderBy: { - createdAt: "desc", - }, + orderBy: [{ createdAt: "desc" }, { id: "desc" }], }), prisma.coverLetter.count({ where: { diff --git a/src/actions/dashboard/recent.ts b/src/actions/dashboard/recent.ts index 1f4dcecdc..09ee27c4f 100644 --- a/src/actions/dashboard/recent.ts +++ b/src/actions/dashboard/recent.ts @@ -17,9 +17,10 @@ export const getRecentJobs = async (): Promise => { Status: true, Location: true, }, - orderBy: { - appliedDate: "desc", - }, + orderBy: [ + { appliedDate: { sort: "desc", nulls: "last" } }, + { id: "desc" }, + ], take: APP_CONSTANTS.RECENT_NUM_JOBS_ACTIVITIES, }); return list; diff --git a/src/actions/interview.actions.ts b/src/actions/interview.actions.ts index f8a62ab65..058b1f9f1 100644 --- a/src/actions/interview.actions.ts +++ b/src/actions/interview.actions.ts @@ -35,10 +35,10 @@ const buildInterviewWhere = ( if (search) { and.push({ OR: [ - { Job: { JobTitle: { label: { contains: search } } } }, - { Job: { Company: { label: { contains: search } } } }, - { StageType: { label: { contains: search } } }, - { interviewers: { some: { Contact: { name: { contains: search } } } } }, + { Job: { JobTitle: { label: { contains: search, mode: "insensitive" } } } }, + { Job: { Company: { label: { contains: search, mode: "insensitive" } } } }, + { StageType: { label: { contains: search, mode: "insensitive" } } }, + { interviewers: { some: { Contact: { name: { contains: search, mode: "insensitive" } } } } }, ], }); } @@ -59,9 +59,8 @@ export const getInterviewList = async ( const where = buildInterviewWhere(user.id, search, stageTypeId, companyId); // Ordering stays in the query because the list is server-paged. - // Undated rounds sort last. SQLite has no NULLS LAST and - // Prisma's `nulls` option is unsupported there, so the dated rows are - // paged first and the undated ones continue the list once they run out. + // Undated rounds sort last: the dated rows are paged first and the undated + // ones continue the list. Kept from SQLite, which had no NULLS LAST. const and = where.AND ?? []; const datedWhere = { ...where, AND: [...and, { occurredAt: { not: null } }] }; const undatedWhere = { ...where, AND: [...and, { occurredAt: null }] }; @@ -80,7 +79,7 @@ export const getInterviewList = async ( skip, take: datedTake, include: INTERVIEW_LIST_INCLUDE, - orderBy: [{ occurredAt: "desc" }, { createdAt: "desc" }], + orderBy: [{ occurredAt: "desc" }, { createdAt: "desc" }, { id: "desc" }], }) : []; @@ -92,7 +91,7 @@ export const getInterviewList = async ( skip: Math.max(0, skip - datedTotal), take: undatedTake, include: INTERVIEW_LIST_INCLUDE, - orderBy: { createdAt: "desc" }, + orderBy: [{ createdAt: "desc" }, { id: "desc" }], }) : []; diff --git a/src/actions/job/queries.ts b/src/actions/job/queries.ts index 1b9921b1e..5b049fc2c 100644 --- a/src/actions/job/queries.ts +++ b/src/actions/job/queries.ts @@ -64,7 +64,9 @@ const JOB_DETAILS_INCLUDE = { }, }, CoverLetter: true, - tags: true, + tags: { + orderBy: [{ label: "asc" as const }, { id: "asc" as const }], + }, contactLinks: { include: { Role: true, @@ -155,19 +157,19 @@ const buildJobsWhereClause = (userId: string, filters: JobsListFilters) => { if (search) { const searchConditions: Record[] = []; if (!titleValue) { - searchConditions.push({ JobTitle: { label: { contains: search } } }); + searchConditions.push({ JobTitle: { label: { contains: search, mode: "insensitive" } } }); } if (!companyValue) { - searchConditions.push({ Company: { label: { contains: search } } }); + searchConditions.push({ Company: { label: { contains: search, mode: "insensitive" } } }); } if (!locationValue) { - searchConditions.push({ Location: { label: { contains: search } } }); + searchConditions.push({ Location: { label: { contains: search, mode: "insensitive" } } }); } if (!sourceValue) { - searchConditions.push({ JobSource: { label: { contains: search } } }); + searchConditions.push({ JobSource: { label: { contains: search, mode: "insensitive" } } }); } searchConditions.push( - { description: { contains: search } }, + { description: { contains: search, mode: "insensitive" } }, ); whereClause.OR = searchConditions; } @@ -322,7 +324,7 @@ export async function* getJobsIterator(filter?: string, pageSize = 200) { select: JOB_EXPORT_SELECT, skip, take: pageSize, - orderBy: { createdAt: "desc" }, + orderBy: [{ createdAt: "desc" }, { id: "desc" }], }); if (!chunk.length) { diff --git a/src/actions/job/references.ts b/src/actions/job/references.ts index af6cff257..bddd8d92c 100644 --- a/src/actions/job/references.ts +++ b/src/actions/job/references.ts @@ -31,6 +31,7 @@ export const getJobSourceList = async (): Promise => { where: { createdBy: user.id, }, + orderBy: [{ label: "asc" }, { id: "asc" }], }); return list; } catch (error) { diff --git a/src/actions/job/shared.ts b/src/actions/job/shared.ts index 3625fe5eb..1d6f0fc87 100644 --- a/src/actions/job/shared.ts +++ b/src/actions/job/shared.ts @@ -29,7 +29,8 @@ type StatusStageTx = Parameters[0] & { // // stageTypeId is an argument, never resolved here: resolveStageTypeForStatusId // writes through the base prisma client, and calling it inside the caller's -// transaction contends for SQLite's write lock (D4). +// transaction would write outside that transaction, so a rollback would not +// undo it (D4). export async function appendStatusStage( tx: StatusStageTx, jobId: string, diff --git a/src/actions/jobLocation.actions.ts b/src/actions/jobLocation.actions.ts index 3b276edc4..a04ec876a 100644 --- a/src/actions/jobLocation.actions.ts +++ b/src/actions/jobLocation.actions.ts @@ -12,6 +12,7 @@ export const getAllJobLocations = async (): Promise => { where: { createdBy: user.id, }, + orderBy: [{ label: "asc" }, { id: "asc" }], }); return list; } catch (error) { diff --git a/src/actions/jobStage/prep.ts b/src/actions/jobStage/prep.ts index 626bf6ed4..ae6dda00d 100644 --- a/src/actions/jobStage/prep.ts +++ b/src/actions/jobStage/prep.ts @@ -28,8 +28,8 @@ export const addStagePrepQuestions = async ( assertInterviewStage(stage); if (owned !== ids.length) throw new Error("Question not found"); - // SQLite has no skipDuplicates, so the already-linked ids are subtracted - // here rather than left to the unique index. + // The already-linked ids are subtracted here rather than left to the + // unique index. skipDuplicates is available now; switching is later work. const linked = await prisma.jobStagePrepQuestion.findMany({ where: { stageId, questionId: { in: ids } }, select: { questionId: true }, diff --git a/src/actions/jobStageType.actions.ts b/src/actions/jobStageType.actions.ts index 7718be6e1..cb0a5243b 100644 --- a/src/actions/jobStageType.actions.ts +++ b/src/actions/jobStageType.actions.ts @@ -31,7 +31,7 @@ export const getJobStageTypeList = async ( const user = await requireUser(); const whereClause: any = { createdBy: user.id }; if (search) { - whereClause.label = { contains: search }; + whereClause.label = { contains: search, mode: "insensitive" }; } const [data, total] = await Promise.all([ @@ -40,7 +40,7 @@ export const getJobStageTypeList = async ( skip: (page - 1) * limit, take: limit, include: { Status: true, _count: { select: { stages: true } } }, - orderBy: [{ sortOrder: "asc" }, { label: "asc" }], + orderBy: [{ sortOrder: "asc" }, { label: "asc" }, { id: "desc" }], }), prisma.jobStageType.count({ where: whereClause }), ]); diff --git a/src/actions/jobtitle.actions.ts b/src/actions/jobtitle.actions.ts index 90aabacd1..f314b41d6 100644 --- a/src/actions/jobtitle.actions.ts +++ b/src/actions/jobtitle.actions.ts @@ -13,6 +13,7 @@ export const getAllJobTitles = async (): Promise => { where: { createdBy: user?.id, }, + orderBy: [{ label: "asc" }, { id: "asc" }], }); return list; } catch (error) { diff --git a/src/actions/profile/resume.ts b/src/actions/profile/resume.ts index a779d1149..a92a2ecd5 100644 --- a/src/actions/profile/resume.ts +++ b/src/actions/profile/resume.ts @@ -38,7 +38,7 @@ export const getResumeList = async ( prisma.resume.findMany({ where: restWhere, select: resumeListSelect, - orderBy: { createdAt: "desc" }, + orderBy: [{ createdAt: "desc" }, { id: "desc" }], take: limit - 1, }), ]); @@ -47,7 +47,7 @@ export const getResumeList = async ( rawData = await prisma.resume.findMany({ where: restWhere, select: resumeListSelect, - orderBy: { createdAt: "desc" }, + orderBy: [{ createdAt: "desc" }, { id: "desc" }], skip: (page - 1) * limit - 1, take: limit, }); @@ -56,7 +56,7 @@ export const getResumeList = async ( rawData = await prisma.resume.findMany({ where, select: resumeListSelect, - orderBy: { createdAt: "desc" }, + orderBy: [{ createdAt: "desc" }, { id: "desc" }], skip: (page - 1) * limit, take: limit, }); diff --git a/src/actions/profile/resumeCopy.ts b/src/actions/profile/resumeCopy.ts index f10ce360a..d15beb8a3 100644 --- a/src/actions/profile/resumeCopy.ts +++ b/src/actions/profile/resumeCopy.ts @@ -48,7 +48,7 @@ export const copyResume = async ( try { const user = await requireUser(); - // Read outside the transaction so the SQLite write lock is held briefly + // Read outside the transaction so it stays short const source = await prisma.resume.findUnique({ where: { id: resumeId, profile: { userId: user.id } }, select: resumeCopySelect, @@ -122,6 +122,7 @@ export const copyResume = async ( startDate: w.startDate, endDate: w.endDate, description: w.description, + createdAt: w.createdAt, })), }); } @@ -137,6 +138,7 @@ export const copyResume = async ( startDate: e.startDate, endDate: e.endDate, description: e.description, + createdAt: e.createdAt, })), }); } @@ -160,6 +162,7 @@ export const copyResume = async ( resumeSectionId: newSection.id, title: o.title, content: o.content, + createdAt: o.createdAt, })), }); } diff --git a/src/actions/profile/shared.ts b/src/actions/profile/shared.ts index bfe9dfac5..13d7c0393 100644 --- a/src/actions/profile/shared.ts +++ b/src/actions/profile/shared.ts @@ -1,5 +1,6 @@ import prisma from "@/lib/db"; import { isResumeFilePath } from "@/lib/resumeFiles"; +import { ENTRY_ORDER, CERTIFICATION_ORDER } from "@/lib/jobs/resumeDetailInclude"; // Not a "use server" module: it exports selects, so it stays internal to // src/actions/profile/ and is never imported by a client component. @@ -63,12 +64,13 @@ export const resumeCopySelect = { title: true, ContactInfo: true, ResumeSections: { + orderBy: { id: "asc" }, include: { summary: true, - workExperiences: true, - educations: true, - licenseOrCertifications: true, - others: true, + workExperiences: { orderBy: ENTRY_ORDER }, + educations: { orderBy: ENTRY_ORDER }, + licenseOrCertifications: { orderBy: CERTIFICATION_ORDER }, + others: { orderBy: ENTRY_ORDER }, skills: true, }, }, diff --git a/src/actions/question.actions.ts b/src/actions/question.actions.ts index 3d75ceb3d..ec7086187 100644 --- a/src/actions/question.actions.ts +++ b/src/actions/question.actions.ts @@ -37,8 +37,8 @@ export const getQuestionsList = async ( if (search) { whereClause.OR = [ - { question: { contains: search } }, - { answer: { contains: search } }, + { question: { contains: search, mode: "insensitive" } }, + { answer: { contains: search, mode: "insensitive" } }, ]; } @@ -46,9 +46,9 @@ export const getQuestionsList = async ( prisma.question.findMany({ where: whereClause, include: { - tags: true, + tags: { orderBy: [{ label: "asc" }, { id: "asc" }] }, }, - orderBy: [{ createdAt: "desc" }], + orderBy: [{ createdAt: "desc" }, { id: "desc" }], skip: offset, take: limit, }), diff --git a/src/actions/referenceList.ts b/src/actions/referenceList.ts index 8cee6ce18..ac0c94674 100644 --- a/src/actions/referenceList.ts +++ b/src/actions/referenceList.ts @@ -64,7 +64,7 @@ export const getReferenceEntityList = async ({ if (search) { const fields = searchFields ?? ["label"]; whereClause.OR = fields.map((field) => ({ - [field]: { contains: search }, + [field]: { contains: search, mode: "insensitive" }, })); } @@ -111,10 +111,14 @@ export const getReferenceEntityList = async ({ } : {}), // Two keys, not one: the applied-count sort alone has no tiebreak, so - // equal-count rows page unstably under infinite scroll. - orderBy: orderBy ?? [ - { [appliedRelation]: { _count: "desc" } }, - { label: "asc" }, + // equal-count rows page unstably under infinite scroll. The trailing id + // makes the order total, which Postgres needs across OFFSET pages. + orderBy: [ + ...(orderBy ?? [ + { [appliedRelation]: { _count: "desc" } }, + { label: "asc" }, + ]), + { id: "desc" }, ], }), model.count({ diff --git a/src/actions/tag.actions.ts b/src/actions/tag.actions.ts index 7964121d2..1c2e673ad 100644 --- a/src/actions/tag.actions.ts +++ b/src/actions/tag.actions.ts @@ -30,7 +30,7 @@ export const getTagList = async ( const whereClause: any = { createdBy: user.id }; if (search) { - whereClause.label = { contains: search }; + whereClause.label = { contains: search, mode: "insensitive" }; } const [data, total] = await Promise.all([ @@ -44,7 +44,7 @@ export const getTagList = async ( value: true, _count: { select: { jobs: true, questions: true, skills: true } }, }, - orderBy: { jobs: { _count: "desc" } }, + orderBy: [{ jobs: { _count: "desc" } }, { id: "desc" }], }), prisma.tag.count({ where: whereClause }), ]); diff --git a/src/actions/task/queries.ts b/src/actions/task/queries.ts index 778e32eba..8c4ca63b2 100644 --- a/src/actions/task/queries.ts +++ b/src/actions/task/queries.ts @@ -16,29 +16,38 @@ function getTasksOrderBy(groupBy?: TaskGroupBy) { switch (groupBy) { case "dueDate": return [ - { dueDate: "asc" as const }, + // Undated tasks first, as SQLite sorted them; Postgres defaults last + { dueDate: { sort: "asc" as const, nulls: "first" as const } }, { priority: "desc" as const }, { createdAt: "desc" as const }, + { id: "desc" as const }, ]; case "createdDate": - return [{ createdAt: "desc" as const }, { priority: "desc" as const }]; + return [ + { createdAt: "desc" as const }, + { priority: "desc" as const }, + { id: "desc" as const }, + ]; case "updatedDate": return [ { updatedAt: "desc" as const }, { priority: "desc" as const }, { createdAt: "desc" as const }, + { id: "desc" as const }, ]; case "activityType": return [ { activityType: { label: "asc" as const } }, { priority: "desc" as const }, { createdAt: "desc" as const }, + { id: "desc" as const }, ]; default: return [ { priority: "desc" as const }, { createdAt: "desc" as const }, { updatedAt: "desc" as const }, + { id: "desc" as const }, ]; } } @@ -70,9 +79,9 @@ export const getTasksList = async ( if (search) { whereClause.OR = [ - { title: { contains: search } }, - { description: { contains: search } }, - { activityType: { label: { contains: search } } }, + { title: { contains: search, mode: "insensitive" } }, + { description: { contains: search, mode: "insensitive" } }, + { activityType: { label: { contains: search, mode: "insensitive" } } }, ]; } @@ -140,6 +149,7 @@ export const getActivityTypesWithTaskCounts = async (): Promise< where: { createdBy: user.id, }, + orderBy: [{ createdAt: "asc" }, { id: "asc" }], include: { _count: { select: { diff --git a/src/lib/agent/resumeLookup.ts b/src/lib/agent/resumeLookup.ts index 8588357ed..eee512978 100644 --- a/src/lib/agent/resumeLookup.ts +++ b/src/lib/agent/resumeLookup.ts @@ -8,8 +8,8 @@ export type ResumeLookup = | { status: "needs_selection"; resumes: { id: string; title: string }[] } | { status: "no_resumes" }; -// SQLite has no `mode: "insensitive"` in Prisma, so title matching happens -// in JS over this list. It is also exactly what the picker renders, so the +// Title matching happens in JS over this list (it predates Postgres and +// mode: "insensitive"). It is also exactly what the picker renders, so the // query is not extra work. async function listTitles(userId: string) { return prisma.resume.findMany({ diff --git a/src/lib/agent/tools/addJob.ts b/src/lib/agent/tools/addJob.ts index 57469542b..17d763432 100644 --- a/src/lib/agent/tools/addJob.ts +++ b/src/lib/agent/tools/addJob.ts @@ -5,6 +5,7 @@ import { AgentAddJobParseSchema, AgentAddJobSchema } from "@/models/agent.schema import { createJobFromNames } from "@/lib/jobs/createJobFromNames"; import { JobResolutionError } from "@/lib/jobs/resolve"; import { log } from "@/lib/telemetry"; +import { sanitizeString } from "@/lib/postgresText"; import type { AgentAddJobResult } from "@/models/agent.model"; // The model usually reaches here having omitted jobDescription because the @@ -53,7 +54,9 @@ export function buildAddJobTool(userId: string, pastedText?: string) { } const input = parsed.data; - const jobDescription = pastedText ?? input.jobDescription; + const jobDescription = sanitizeString( + pastedText ?? input.jobDescription ?? "", + ); if (!jobDescription) return failed(NO_DESCRIPTION, NO_DESCRIPTION_DISPLAY); try { diff --git a/src/lib/ai/import/extract-text.ts b/src/lib/ai/import/extract-text.ts index ba33bfd65..b6c6d9b62 100644 --- a/src/lib/ai/import/extract-text.ts +++ b/src/lib/ai/import/extract-text.ts @@ -1,6 +1,7 @@ // Text extraction from PDF and DOCX resume uploads import { APP_CONSTANTS } from "@/lib/constants"; +import { sanitizeString } from "@/lib/postgresText"; // Magic bytes for file-type sniffing (Buffer — server-only, not in constants.ts) export const PDF_MAGIC = Buffer.from([0x25, 0x50, 0x44, 0x46]); // %PDF @@ -86,7 +87,7 @@ async function extractPdf(buf: Buffer): Promise { const { text: pages } = await pdfExtractText(pdf, { mergePages: false }); const pageArray = Array.isArray(pages) ? pages : [pages as string]; - const text = pageArray.slice(0, pagesToRead).join("\n"); + const text = sanitizeString(pageArray.slice(0, pagesToRead).join("\n")); if (!text.trim()) { throw Object.assign(new Error("NO_TEXT"), { code: "NO_TEXT" }); @@ -128,18 +129,19 @@ async function extractDocx(buf: Buffer): Promise { const mammoth = await import("mammoth"); const result = await mammoth.extractRawText({ buffer: buf }); - if (!result.value.trim()) { + const value = sanitizeString(result.value); + if (!value.trim()) { throw Object.assign(new Error("NO_TEXT"), { code: "NO_TEXT" }); } const text = - result.value.length > APP_CONSTANTS.RESUME_IMPORT_MAX_EXTRACTED_CHARS - ? result.value.slice(0, APP_CONSTANTS.RESUME_IMPORT_MAX_EXTRACTED_CHARS) - : result.value; + value.length > APP_CONSTANTS.RESUME_IMPORT_MAX_EXTRACTED_CHARS + ? value.slice(0, APP_CONSTANTS.RESUME_IMPORT_MAX_EXTRACTED_CHARS) + : value; return { text, - truncated: result.value.length > APP_CONSTANTS.RESUME_IMPORT_MAX_EXTRACTED_CHARS, + truncated: value.length > APP_CONSTANTS.RESUME_IMPORT_MAX_EXTRACTED_CHARS, }; } diff --git a/src/lib/aiUsage/summarize.ts b/src/lib/aiUsage/summarize.ts index c8e9f2875..0a0e9d4fa 100644 --- a/src/lib/aiUsage/summarize.ts +++ b/src/lib/aiUsage/summarize.ts @@ -23,7 +23,7 @@ export type AiCallRecord = { const DAY_MS = 24 * 60 * 60 * 1000; -// Nearest rank, in JS: SQLite has no percentile function. +// Nearest rank, in JS: kept engine-independent rather than percentile_cont. export function percentile(values: number[], p: number): number | null { if (values.length === 0) return null; const sorted = [...values].sort((a, b) => a - b); diff --git a/src/lib/backup/import.ts b/src/lib/backup/import.ts index 1749ef37f..f11e4b068 100644 --- a/src/lib/backup/import.ts +++ b/src/lib/backup/import.ts @@ -22,6 +22,7 @@ import { } from "./ordering"; import { writeSnapshot } from "./snapshot"; import { log } from "@/lib/telemetry"; +import { sanitizeString } from "@/lib/postgresText"; import { isResumeFilePath } from "@/lib/resumeFiles"; import { BackupDataSchema, type BackupData, type BackupManifest } from "./schema"; @@ -87,7 +88,11 @@ async function readData(zip: JSZip): Promise { if (!entry) throw new BackupError("Backup is missing data.json."); let raw: unknown; try { - raw = JSON.parse(await entry.async("string")); + // A backup taken on SQLite can hold NULs, and one would fail the whole + // restore transaction on Postgres + raw = JSON.parse(await entry.async("string"), (_key, value) => + typeof value === "string" ? sanitizeString(value) : value, + ); } catch { throw new BackupError("data.json is not valid JSON."); } @@ -429,6 +434,7 @@ export async function importBackup( ); if (error instanceof BackupError) throw error; const message = error instanceof Error ? error.message : ""; + // Dead on Postgres (a SQLite lock message); removed with the import shim. if (/database is locked|SQLITE_BUSY/i.test(message)) { throw new BackupError("The database is busy. Please try the import again."); } diff --git a/src/lib/backup/snapshot.ts b/src/lib/backup/snapshot.ts index 32092b1be..4a669a2d6 100644 --- a/src/lib/backup/snapshot.ts +++ b/src/lib/backup/snapshot.ts @@ -86,10 +86,11 @@ export async function readSnapshot( } // Prunes on count and on total bytes. The count alone is not a disk bound: -// nothing caps how large one snapshot is, these sit on the same volume as the -// SQLite database, and an import/rollback loop writes one every time. The -// newest is always kept, even if it alone exceeds the byte budget — dropping -// the only record of the state a user just left is worse than overshooting. +// nothing caps how large one snapshot is, these sit on the /data bind mount +// alongside uploaded resumes, and an import/rollback loop writes one every +// time. The newest is always kept, even if it alone exceeds the byte budget — +// dropping the only record of the state a user just left is worse than +// overshooting. export async function pruneSnapshots( userId: string, keep: number, diff --git a/src/lib/constants.ts b/src/lib/constants.ts index 15d29dba3..08650cb19 100644 --- a/src/lib/constants.ts +++ b/src/lib/constants.ts @@ -245,8 +245,8 @@ export const APP_CONSTANTS = { BACKUP_MAX_UNCOMPRESSED_BYTES: 100 * 1024 * 1024, BACKUP_MAX_ENTRIES: 5000, // Pre-import snapshots retained per user. The count alone does not bound - // disk — nothing caps one snapshot's size — and these live on the same - // volume as the SQLite database, so prune on total bytes as well. + // disk — nothing caps one snapshot's size — and these live on the /data + // bind mount, alongside uploaded resumes, so prune on total bytes as well. BACKUP_SNAPSHOT_KEEP: 5, BACKUP_SNAPSHOT_MAX_TOTAL_BYTES: 250 * 1024 * 1024, diff --git a/src/lib/jobs/resumeDetailInclude.ts b/src/lib/jobs/resumeDetailInclude.ts index 11050d677..814b32249 100644 --- a/src/lib/jobs/resumeDetailInclude.ts +++ b/src/lib/jobs/resumeDetailInclude.ts @@ -4,13 +4,28 @@ // getDefaultResumeForUser (MCP path) stay in sync. profile.actions.ts has // "use server" at the top, which only allows async-function exports, so this // object can't live there directly. +// Postgres returns rows in storage order and an UPDATE moves the row, so an +// unordered relation would reshuffle a resume after any edit. +export const ENTRY_ORDER = [ + { createdAt: "asc" as const }, + { id: "asc" as const }, +]; +// No createdAt on this model: newest credential first, undated ones last +export const CERTIFICATION_ORDER = [ + { issueDate: { sort: "desc" as const, nulls: "last" as const } }, + { title: "asc" as const }, + { id: "asc" as const }, +]; + export const resumeDetailInclude = { ContactInfo: true, File: true, ResumeSections: { + orderBy: { id: "asc" }, include: { summary: true, workExperiences: { + orderBy: ENTRY_ORDER, include: { jobTitle: true, Company: true, @@ -18,11 +33,12 @@ export const resumeDetailInclude = { }, }, educations: { + orderBy: ENTRY_ORDER, include: { location: true, }, }, - licenseOrCertifications: true, + licenseOrCertifications: { orderBy: CERTIFICATION_ORDER }, skills: { include: { Tag: true } }, }, }, diff --git a/src/lib/jobs/sortStages.ts b/src/lib/jobs/sortStages.ts index 1c4a64117..902bfd325 100644 --- a/src/lib/jobs/sortStages.ts +++ b/src/lib/jobs/sortStages.ts @@ -2,8 +2,8 @@ // chronology: a job is routinely added days after it was applied to, so the // New stage's date (when the job entered JobSync) postdates the Applied // stage's. Dates break ties within one sortOrder. Undated stages sort last -// among their peers — SQLite sorts NULL first on ASC and Prisma's -// nulls:"last" is unsupported there, so none of this can live in the query. +// among their peers. This ordering predates Postgres and stays in JavaScript +// so the Timeline tab can share it. // Both the server actions and the Timeline tab sort through this one // comparator: the job-details page loads stages with no orderBy at all. export function sortStages< diff --git a/src/lib/jobs/updateJobFromNames.ts b/src/lib/jobs/updateJobFromNames.ts index 3896fda67..212507164 100644 --- a/src/lib/jobs/updateJobFromNames.ts +++ b/src/lib/jobs/updateJobFromNames.ts @@ -175,7 +175,7 @@ export async function updateJobFromNames( } // D4: this resolve can create a row and writes through the base client, so - // it must not run while the transaction below holds SQLite's write lock. + // it stays outside the transaction below, which could not roll it back. const stageTypeId = statusId ? await resolveStageTypeForStatusId(statusId, userId) : null; diff --git a/src/lib/listSort.ts b/src/lib/listSort.ts index acb29aba7..fa2e3f6ef 100644 --- a/src/lib/listSort.ts +++ b/src/lib/listSort.ts @@ -51,8 +51,8 @@ const withCondition = (where: Where, extra: Where): Where => { return { ...where, AND: [...and, extra] }; }; -// SQLite has no NULLS LAST and Prisma's `nulls` option is unsupported there, -// so valued rows page first and the list continues into the blank ones. +// Valued rows page first and the list continues into the blank ones. Kept +// from SQLite, which had no NULLS LAST; one nulls:"last" query is later work. export async function findManySorted( model: ListSortDelegate, query: { where: Where; skip: number; take: number; select?: unknown }, diff --git a/src/lib/mcp/tools/addJobsBatch.ts b/src/lib/mcp/tools/addJobsBatch.ts index 8df22cc5c..8bc9afdf7 100644 --- a/src/lib/mcp/tools/addJobsBatch.ts +++ b/src/lib/mcp/tools/addJobsBatch.ts @@ -2,8 +2,8 @@ import { z } from "zod"; import { McpAddJobSchema } from "@/models/mcp.schema"; import { handleAddJob } from "@/lib/mcp/tools/addJob"; -// Sequential, not Promise.all: entity resolution is check-then-act, and -// SQLite serializes writes anyway — parallelism here only buys races. +// Sequential, not Promise.all: entity resolution is check-then-act, so +// parallel calls naming the same new entity race to create it twice. export async function handleAddJobsBatch( input: { jobs: Array> }, userId: string, diff --git a/src/lib/postgresText.ts b/src/lib/postgresText.ts new file mode 100644 index 000000000..03ae22f8b --- /dev/null +++ b/src/lib/postgresText.ts @@ -0,0 +1,9 @@ +// Postgres text rejects NUL and lone surrogates; PDF extraction, scraped +// postings and pasted text can all carry them. Same rule as the one-time +// import's scripts/import/sanitize.ts, which scripts/ cannot import from here. +const UNSTORABLE = + /\u0000|[\uD800-\uDBFF](?![\uDC00-\uDFFF])|(?`, or, to deliberately start again from the old SQLite data, delete `jobsyncdb/data/dev.db.imported-at` and restart. + +## How do I back up JobSync now? + +Run `./backup-db.sh`, which writes a dated dump into `jobsyncdb/data/backups/`. On Windows run it from WSL or Git Bash; there is no PowerShell version yet. The database no longer lives in the `jobsyncdb` folder, so a folder-level backup on its own no longer covers it; the dump puts a copy back inside that folder. On Linux you may need `sudo`, and the script says so when it does. Restore with `./restore-db.sh `, which only loads into an empty database. + +Copying Postgres's data directory while it is running is not a valid backup. The in-app Data Backup is still the way to move one account's data, but it leaves out accounts, API keys and MCP tokens, so it does not replace a dump. Take a dump before you ever delete `dev.db`. + +## Where is my data stored now? + +In a Docker volume named `jobsync_pgdata`, under Docker's own storage, not in `jobsyncdb/data/`. If you placed `jobsyncdb` on a particular disk on purpose, the database is no longer on it; to keep it there, declare the `pgdata` volume as a bind mount. Uploaded resume files stay in `jobsyncdb/data/` as before. The volume holds Postgres 18 data, so a future move to Postgres 19 is a dump and restore, not an image tag change. + +## How do I use my own Postgres instead of the bundled one? + +Set `DATABASE_URL` in `.env` to your server's address; never edit `docker-compose.yml`. That alone switches the app and the backup scripts to your server. The bundled container then sits idle; add `BUNDLED_DB=0` to `.env` if you want it stopped. Point `DATABASE_URL` at a dedicated, empty database. JobSync creates the database if it is missing and the role is allowed to; check the "created it" line on first start, because a typo in the name creates a new empty database rather than an error. Set `CREATE_DATABASE_IF_MISSING=0` to make a missing database an error instead. When a start-up message tells you to empty the Postgres database, drop and recreate that database on your server; the `docker volume rm` command it shows applies to the bundled database only. + +Checklist for your server: + +- Postgres 17 or newer. +- A role that can log in and either owns the database or has `GRANT ALL ON SCHEMA public`. +- Managed providers usually need `?sslmode=require`, and the direct endpoint rather than a pooled one. +- From inside the container use `host.docker.internal`, not `localhost`, for a Postgres on the same machine. +- Percent-encode a password containing `@`, `:`, `/` or `#`. +- If your server is newer than Postgres 18, set `PG_DUMP_IMAGE` in `.env` to its version, for example `postgres:19`, so backups work. +- Upcoming: a future release uses the `pgvector` extension for resume embeddings. If you run your own Postgres, plan to have it available; the bundled database already includes it.