diff --git a/.gitignore b/.gitignore index 9af8448..b4da330 100644 --- a/.gitignore +++ b/.gitignore @@ -34,3 +34,5 @@ decisions/ # Personal Claude Code hook config (machine-specific absolute paths) .claude/settings.local.json + +.env \ No newline at end of file diff --git a/CLAUDE.md b/CLAUDE.md index 0b6b59a..42a5926 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -229,7 +229,7 @@ Each page/component is organized by: ### API Features (Complete + Audited May 2026) -- ✅ Meter Groups (CRUD, batch; dynamic sorting; `POST /:id/reset` and its `current_version`/`versions` fields are **@deprecated** — version tracking now lives per-property on `Property.meter_groups[entry]`, see `decisions/`) +- ✅ Meter Groups (CRUD, batch; dynamic sorting; **submeter** version tracking lives per-property on `Property.meter_groups[entry]`, but `POST /:id/reset` and `MeterGroup.current_version`/`versions` are **still the live source of truth for main meters** — the "migrate main meters too" backfill was never done and is not pending, so do NOT treat those fields as fully deprecated or delete them, see `decisions/20260608_meter-group-version-tracking-moved-to-property.md` Amendment 2026-07-19) - ✅ Properties (CRUD, batch; dynamic sorting; optimized duplicate detection) - ✅ Tenants (CRUD, batch; dynamic sorting) - ✅ Readings (CRUD, batch; auto-billing on single create; anomaly guard; meter rollback prevention; utility extraction) @@ -239,10 +239,9 @@ Each page/component is organized by: - ✅ Image Extraction (`POST /image-extraction/readings` + `POST /image-extraction/billings` — vision OCR via the user's configured `llm-config` vision provider, Groq or Ollama Cloud only; no Gemini) - ✅ Reports (`GET /reports/summary`, `/consumption`, `/billing-trends`, `/collection-status`) - ✅ Bills (`POST /bills/ocr` — functional 3-step UI wizard; overlaps with image-extraction) -- ⚠️ Users (`POST /users` — partial stub for user role management) +- ✅ Users (`POST /users` — creates both the Firebase Auth account and Firestore profile server-side via the Admin SDK in one call; the client never touches Firebase Auth for this flow, so the acting admin's own session is never hijacked by the newly created account) - ✅ LLM Config (`GET`/`PATCH /llm-config` for the chatbot provider/model/API key + `PATCH /llm-config/vision` for an **independent** vision provider/model/API key used by OCR — separate because not every provider has a usable free vision model; reuses the chatbot's API key when both configs use the same provider; API keys AES-256-GCM encrypted at rest via `lib/crypto.lib.ts`) - ✅ Chatbot (`POST /chatbot` — insight assistant scoped to the authenticated user's data, tool-calling via `lib/llm.lib.ts` against Groq/Ollama Cloud; regex jailbreak guard in `chatbot.guard.ts`) -- ✅ Photo Settings (`GET`/`PATCH /photo-settings` — per-user `savePhotos` preference, defaults to `false`; web and mobile check it before attaching a meter-reading `image_url` on create. OCR suggest always works regardless; billing-cycle/bill photos are never persisted either way) **Audit Highlights (25 fixes)**: @@ -265,7 +264,7 @@ Each page/component is organized by: - ✅ Readings (filterable list; True Total column; batch form with decoupled OCR suggest; archive page) - ✅ Billings (cycle-centric: expandable cycles with nested billings; bill photo OCR autofill; cycle edit modal for rate/consumption/date corrections; archive page) - ✅ Bills / OCR upload (3-step wizard: upload → review/map → submit, creates a billing cycle) -- 🚧 Reports (stub — API module ready, UI not built) +- ✅ Reports (analytics dashboard: filters, summary cards, consumption/billing-trends charts, collection-status cards, per-property table) - 🚧 Settings (partial — payment + user management tabs scaffolded; LLM Provider tab added) - ✅ Insight Chatbot (`ChatWidget` floating widget, mounted globally on all protected routes) @@ -276,7 +275,7 @@ Each page/component is organized by: - ✅ CaptureReadings (3-step wizard: meter group select → per-property readings + camera with auto OCR suggest → review & batch submit) - ✅ ReadingHistory (filterable by utility type + property) - ✅ Billings (grouped by status: overdue/pending/paid; mark-as-paid action) -- ✅ Settings (account info + sign out; save-photos preference toggle) +- ✅ Settings (account info + sign out) --- diff --git a/PRIVACY.md b/PRIVACY.md index 891fb77..050f17e 100644 --- a/PRIVACY.md +++ b/PRIVACY.md @@ -11,12 +11,13 @@ consumption data is processed. - **Property/tenant records**: room/unit names, tenant names, occupancy dates — entered by staff, stored in Firestore. -- **Meter readings**: numeric consumption values, timestamps, and (optionally) a photo of - the meter face, if the per-user photo-settings preference (`GET/PATCH /photo-settings`, - default **off**) is enabled. See `api/CLAUDE.md` → "Photo Settings". +- **Meter readings**: numeric consumption values and timestamps. A photo of the meter face + may be captured to drive the OCR "suggest" feature, but it is never persisted — it exists + only in-memory for that one request and is discarded immediately after (readings have no + `image_url` field). - **Billing records and billing cycles**: consumption, rates, and computed charges derived from readings — no bill/cycle photos are ever persisted (no `image_url` field exists on - the billing-cycle model). + the billing-cycle model either). - **Account data**: email and role, via Firebase Authentication. ## Third-Party AI Processors (OCR / Vision) @@ -127,8 +128,8 @@ data exposure beyond what the requesting admin's role already permits. purges cascade to their already-archived readings/billings, mirroring the existing archive cascade, so a purge cannot leave orphaned records behind. See `api/CLAUDE.md` → "Archive-Then-Purge Lifecycle". -- Meter-reading photos are only stored if the photo-settings preference is explicitly - enabled per user; bill/billing-cycle photos are never stored. +- Meter-reading, bill, and billing-cycle photos are never stored — every photo is used + transiently, in-memory, for the OCR "suggest" call only, then discarded. - This document does not set a fixed data-retention period — retention is governed by operational need and, if the organization is subject to a specific data protection regime (e.g. the Philippine Data Privacy Act), by that regime's requirements. The diff --git a/api/functions/.eslintrc.js b/api/functions/.eslintrc.js index 4c34c83..c8505ce 100644 --- a/api/functions/.eslintrc.js +++ b/api/functions/.eslintrc.js @@ -27,6 +27,7 @@ "jest.config.ts", "jest.setup.ts", "seed.js", + "scripts/**/*", // One-off admin scripts, outside the main tsconfig "src" scope. ], plugins: [ "@typescript-eslint", diff --git a/api/functions/.gitignore b/api/functions/.gitignore index 2e00864..d47a097 100644 --- a/api/functions/.gitignore +++ b/api/functions/.gitignore @@ -13,4 +13,7 @@ secrets/ .env logs/ -src/migrations/ \ No newline at end of file +src/migrations/ + +# Generated report from the one-off backfill script (runtime artifact) +scripts/*.report.json \ No newline at end of file diff --git a/api/functions/CLAUDE.md b/api/functions/CLAUDE.md index e3be546..f992276 100644 --- a/api/functions/CLAUDE.md +++ b/api/functions/CLAUDE.md @@ -258,7 +258,7 @@ Represents snapshots of meter consumption. **Single create has a critical side e - reading_amount must be non-negative - reading_date cannot be in the future - No duplicate readings per meter group per month (enforced at write time) -- `image_url` is optional +- Readings have no `image_url` field — meter photos are only ever used transiently for OCR suggest (`POST /readings/ocr`), never persisted - `meter_version` is server-set from the meter group's `current_version` at creation time (not provided by client) - Anomaly guard: if the reading delta exceeds 5× the rolling average for that meter group, returns 422 with a descriptive message — record a meter group reset first if the meter was physically replaced - **Cascade delete**: `DELETE /:id` soft-deletes the reading + all billings that reference it (as previous or current reading) (atomic transaction) @@ -343,7 +343,7 @@ Represents billing periods with validation and rate calculation. 5. Reject if cycle-level total is outside ±3% of `billing_consumption` **OCR endpoint** (`POST /billing-cycles/ocr`): -- Accepts `{ image_url: string }` (data URL or HTTPS URL of a utility bill photo) +- Accepts `{ image_url: string }` — a base64 `data:image/*;base64,...` URL of a utility bill photo (no fetchable URLs — see SSRF note below) - Returns `{ billing_start_date, billing_end_date, billing_consumption, billing_rate, raw_amount }` - Returns 404 if the user has no `vision_model` configured in `llm-config` (no fallback), 422 if the vision model cannot extract the data or any numeric field is invalid - Requires `admin` or `landlord` role @@ -364,7 +364,7 @@ no Gemini, no fallback. | POST | `/image-extraction/billings` | OCR utility bill photo → `{ billing_start_date, billing_end_date, billing_consumption, billing_rate, raw_amount }` | **Business rules**: -- Accepts `{ image_url: string }` — data URL or HTTPS URL +- Accepts `{ image_url: string }` — base64 `data:image/*;base64,...` only (no fetchable URLs — see SSRF note below) - Returns 404 if the user has no `vision_model` configured in `llm-config`, 422 if extraction fails, 400 if the image URL is invalid - Backed by `src/lib/vision-ocr.lib.ts` (via `src/lib/llm.lib.ts`'s `LlmClient`, same Groq/Ollama Cloud providers as the chatbot) - Requires authentication (BearerAuth) @@ -415,22 +415,6 @@ Restricted to the `admin` role (`requireRole("admin")` in `chatbot.route.ts`) |--------|------|---------| | POST | `/chatbot` | Send a message (+ optional up-to-20-message history); returns `{ reply }`. Admin-only. | -### Photo Settings (`/photo-settings` — protected) - -Located: `src/features/photo-settings/` - -Per-user preference for whether meter-reading photos get persisted (`image_url`) when a reading is created — separate collection, same one-doc-per-user pattern as `llm-config`. Unlike `llm-config`, `GET` never 404s: no stored doc just means the default applies. - -| Method | Path | Purpose | -|--------|------|---------| -| GET | `/photo-settings` | Fetch `{savePhotos: boolean}` — defaults to `false` if never configured | -| PATCH | `/photo-settings` | Set `{savePhotos: boolean}` | - -**Business rules**: -- Default is `false` (disabled) — a captured/uploaded photo is still used transiently for OCR suggest (`POST /readings/ocr`, `POST /image-extraction/readings`), but web and mobile clients strip `image_url` from the create payload unless this is `true`. -- Utility-bill / billing-cycle photos are **never** persisted regardless of this setting — there's no `image_url` field on the billing-cycle model to begin with, so this setting only governs meter-reading photos. -- Enforcement is client-side (web `readings` page, mobile `CaptureReadings`) — the backend doesn't inspect this setting itself; `POST /readings`/`POST /readings/batch` still accept an optional `image_url` as before. - ### Stub & Incomplete Features The following feature folders exist but are **not fully implemented**: @@ -628,18 +612,6 @@ api/functions/src/features/chatbot/ └── chatbot.swagger.ts ``` -### Photo Settings -``` -api/functions/src/features/photo-settings/ -├── photo-settings.model.ts → PhotoSettings (save_photos: boolean) -├── photo-settings.dto.ts -├── photo-settings.repository.ts → One doc per user, same pattern as llm-config.repository.ts -├── photo-settings.service.ts → get() defaults to {savePhotos: false} when no doc exists (never 404s) -├── photo-settings.controller.ts -├── photo-settings.route.ts -└── photo-settings.swagger.ts -``` - ### Authentication (special case — public routes) ``` api/functions/src/features/auth/ @@ -669,7 +641,7 @@ api/functions/src/ │ ├── auth.lib.ts → JWT generation (jsonwebtoken) │ ├── llm.lib.ts → LlmClient — OpenAI-compatible chat-completions client (Groq, Ollama Cloud); serializes text+image content per-provider │ ├── vision-ocr.lib.ts → Vision OCR (readings, bills) via LlmClient — no Gemini, requires `vision_model` -│ ├── image-fetch.util.ts → SSRF-guarded image fetch/decode shared by vision-ocr.lib.ts +│ ├── image-fetch.util.ts → Decodes base64 `data:image/*;base64,...` payloads shared by vision-ocr.lib.ts — no server-side URL fetch exists, eliminating SSRF entirely (all OCR endpoints reject non-`data:` image_url via the shared `ImageUrlSchema`) │ ├── ocr-parsing.util.ts → OCR prompts + response parsing shared by vision-ocr.lib.ts │ ├── crypto.lib.ts → AES-256-GCM encrypt/decryptSecret() for LLM API keys │ └── ... (Realtime DB, storage stubs) diff --git a/api/functions/package-lock.json b/api/functions/package-lock.json index 6a1911c..38acddb 100644 --- a/api/functions/package-lock.json +++ b/api/functions/package-lock.json @@ -5890,9 +5890,9 @@ } }, "node_modules/firebase-functions": { - "version": "7.2.5", - "resolved": "https://registry.npmjs.org/firebase-functions/-/firebase-functions-7.2.5.tgz", - "integrity": "sha512-K+pP0AknluAguLRbD96hibyXbnOgwnvd4hkExWdGrxnNCLoj8LBFj08uvJYxyvhsCgYzQumrUaHBW4lsXKSiRg==", + "version": "7.3.0", + "resolved": "https://registry.npmjs.org/firebase-functions/-/firebase-functions-7.3.0.tgz", + "integrity": "sha512-S3JhjESOWMq13iDodwgUPWNQ3gLAu7oTLDwF7hTtisyjVanEeQzYezgW+JTfd8I0kCJQzjGPC/wHQ19IL7CgaA==", "license": "MIT", "dependencies": { "@types/cors": "^2.8.5", @@ -5910,7 +5910,7 @@ "peerDependencies": { "@apollo/server": "^5.2.0", "@as-integrations/express4": "^1.1.2", - "firebase-admin": "^11.10.0 || ^12.0.0 || ^13.0.0", + "firebase-admin": "^11.10.0 || ^12.0.0 || ^13.0.0 || ^14.0.0", "graphql": "^16.12.0" }, "peerDependenciesMeta": { @@ -11891,9 +11891,9 @@ "optional": true }, "node_modules/websocket-driver": { - "version": "0.7.4", - "resolved": "https://registry.npmjs.org/websocket-driver/-/websocket-driver-0.7.4.tgz", - "integrity": "sha512-b17KeDIQVjvb0ssuSDF2cYXSg2iztliJ4B9WdsuB6J952qCPKmnVq4DyW5motImXHDC1cBT/1UezrJVsKw5zjg==", + "version": "0.7.5", + "resolved": "https://registry.npmjs.org/websocket-driver/-/websocket-driver-0.7.5.tgz", + "integrity": "sha512-ZL2+3c7kMBdIRCMz6l8jQMHyGVxj+UL+xVk74Ombiciboca8rHa15L86B19E5oh1pL9Ii/uj54gtsIrZGMo6zA==", "license": "Apache-2.0", "dependencies": { "http-parser-js": ">=0.5.1", diff --git a/api/functions/package.json b/api/functions/package.json index 2388eed..567b344 100644 --- a/api/functions/package.json +++ b/api/functions/package.json @@ -15,7 +15,8 @@ "deploy": "firebase deploy --only functions", "deploy:staging": "cross-env APP_ENV=staging firebase deploy --project staging --only functions", "deploy:prod": "cross-env APP_ENV=production firebase deploy --project production --only functions", - "logs": "firebase functions:log" + "logs": "firebase functions:log", + "backfill:billing-meter-group": "ts-node scripts/backfill-billing-meter-group.ts" }, "engines": { "node": "24" diff --git a/api/functions/scripts/backfill-billing-meter-group.ts b/api/functions/scripts/backfill-billing-meter-group.ts new file mode 100644 index 0000000..bd5b97a --- /dev/null +++ b/api/functions/scripts/backfill-billing-meter-group.ts @@ -0,0 +1,129 @@ +/** + * One-time backfill: denormalizes meter_group_id + billing_period_date onto existing + * Billing documents, resolved from each billing's current_reading_id. Idempotent — already + * backfilled billings are skipped, so it's safe to re-run. + * + * Usage (from api/functions/): + * APP_ENV=staging GOOGLE_APPLICATION_CREDENTIALS=... npx ts-node scripts/backfill-billing-meter-group.ts --dry-run + * APP_ENV=staging GOOGLE_APPLICATION_CREDENTIALS=... npx ts-node scripts/backfill-billing-meter-group.ts --apply + * + * --dry-run (default if neither flag is passed) writes a report to + * scripts/backfill-billing-meter-group.report.json and performs no writes. + * --apply performs the writes, in batches of 500 (Firestore batch limit). + */ +import * as fs from "node:fs"; +import * as path from "node:path"; +import {firestore} from "../src/config/firebase.config"; +import {COLLECTIONS} from "../src/constants/collection.constants"; + +const BATCH_LIMIT = 500; +const REPORT_PATH = path.join(__dirname, "backfill-billing-meter-group.report.json"); + +interface ReportEntry { + billingId: string; + currentReadingId: string; + status: "resolved" | "already-backfilled" | "reading-not-found"; + meterGroupId?: string; + billingPeriodDate?: string; +} + +async function main() { + const apply = process.argv.includes("--apply"); + const mode = apply ? "apply" : "dry-run"; + console.log(`Running backfill in ${mode} mode...`); + + const billingsSnap = await firestore.collection(COLLECTIONS.BILLINGS).get(); + console.log(`Found ${billingsSnap.size} billing documents.`); + + const report: ReportEntry[] = []; + const toWrite: { ref: FirebaseFirestore.DocumentReference; meterGroupId: string; billingPeriodDate: FirebaseFirestore.Timestamp }[] = []; + + // Firestore getAll() accepts many refs in one round trip; chunk to stay well under + // request-size limits. + const readingIds = Array.from( + new Set(billingsSnap.docs.map((d) => d.data().current_reading_id as string).filter(Boolean)) + ); + const readingById = new Map(); + const CHUNK = 300; + for (let i = 0; i < readingIds.length; i += CHUNK) { + const chunk = readingIds.slice(i, i + CHUNK); + const refs = chunk.map((id) => firestore.collection(COLLECTIONS.READINGS).doc(id)); + const snaps = await firestore.getAll(...refs); + snaps.forEach((snap, idx) => { + if (snap.exists) readingById.set(chunk[idx], snap.data()!); + }); + } + + for (const doc of billingsSnap.docs) { + const data = doc.data(); + const currentReadingId = data.current_reading_id as string; + + if (data.meter_group_id && data.billing_period_date) { + report.push({ + billingId: doc.id, + currentReadingId, + status: "already-backfilled", + meterGroupId: data.meter_group_id, + }); + continue; + } + + const reading = readingById.get(currentReadingId); + if (!reading) { + report.push({billingId: doc.id, currentReadingId, status: "reading-not-found"}); + continue; + } + + report.push({ + billingId: doc.id, + currentReadingId, + status: "resolved", + meterGroupId: reading.meter_group_id, + billingPeriodDate: reading.reading_date?.toDate?.().toISOString(), + }); + toWrite.push({ + ref: doc.ref, + meterGroupId: reading.meter_group_id, + billingPeriodDate: reading.reading_date, + }); + } + + const resolved = report.filter((r) => r.status === "resolved").length; + const alreadyDone = report.filter((r) => r.status === "already-backfilled").length; + const failed = report.filter((r) => r.status === "reading-not-found").length; + console.log(`Resolved: ${resolved}, already backfilled: ${alreadyDone}, failed: ${failed}`); + + fs.writeFileSync(REPORT_PATH, JSON.stringify({mode, resolved, alreadyDone, failed, entries: report}, null, 2)); + console.log(`Report written to ${REPORT_PATH}`); + + if (!apply) { + console.log("Dry-run complete. No writes performed. Re-run with --apply to write changes."); + return; + } + + if (failed > 0) { + console.warn(`WARNING: ${failed} billing(s) could not be resolved (see report) — these will remain un-backfilled.`); + } + + for (let i = 0; i < toWrite.length; i += BATCH_LIMIT) { + const chunk = toWrite.slice(i, i + BATCH_LIMIT); + const batch = firestore.batch(); + for (const item of chunk) { + batch.update(item.ref, { + meter_group_id: item.meterGroupId, + billing_period_date: item.billingPeriodDate, + }); + } + await batch.commit(); + console.log(`Wrote batch ${i / BATCH_LIMIT + 1} (${chunk.length} documents).`); + } + + console.log(`Apply complete. ${toWrite.length} billing document(s) updated.`); +} + +main() + .then(() => process.exit(0)) + .catch((err) => { + console.error(err); + process.exit(1); + }); diff --git a/api/functions/src/config/rate-limit.config.ts b/api/functions/src/config/rate-limit.config.ts index 2f6bb81..ab6a4d5 100644 --- a/api/functions/src/config/rate-limit.config.ts +++ b/api/functions/src/config/rate-limit.config.ts @@ -52,3 +52,21 @@ export const chatbotRateLimiter = rateLimit({ store: buildStore("rl:chatbot:"), keyGenerator: (req: AuthenticatedRequest) => req.user?.userId ?? ipKeyGenerator(req.ip ?? ""), }); + +/** + * Vision-OCR endpoints (/image-extraction/*, /readings/ocr, /billing-cycles/ocr, + * /bills/ocr) are cost-bearing like the chatbot (each call hits the tenant's own + * configured vision provider), but are also a normal, repeated part of the reading + * and bill capture workflow rather than incidental chat — so the cap sits between + * the tight chatbot limit (30/hr) and the generic per-IP limit (1000/hr), and is + * keyed per-user (not per-IP) for the same shared-office-IP reason as chatbotRateLimiter. + */ +export const ocrRateLimiter = rateLimit({ + windowMs: 60 * 60 * 1000, + limit: 150, + standardHeaders: "draft-8", + legacyHeaders: false, + message: {error: TOO_MANY_REQUESTS_MESSAGE}, + store: buildStore("rl:ocr:"), + keyGenerator: (req: AuthenticatedRequest) => req.user?.userId ?? ipKeyGenerator(req.ip ?? ""), +}); diff --git a/api/functions/src/config/swagger.config.ts b/api/functions/src/config/swagger.config.ts index 5ac5e20..c7b74c0 100644 --- a/api/functions/src/config/swagger.config.ts +++ b/api/functions/src/config/swagger.config.ts @@ -12,7 +12,6 @@ import {billsPaths} from "../features/bills/bills.swagger"; import {paths as imageExtractionPaths} from "../features/image-extraction/image-extraction.swagger"; import {reportsPaths} from "../features/reports/reports.swagger"; import {llmConfigPaths} from "../features/llm-config/llm-config.swagger"; -import {photoSettingsPaths} from "../features/photo-settings/photo-settings.swagger"; import {chatbotPaths} from "../features/chatbot/chatbot.swagger"; const swaggerSpec = { @@ -471,11 +470,6 @@ const swaggerSpec = { reading_date: { $ref: "#/components/schemas/Timestamp", }, - image_url: { - type: "string", - format: "uri", - description: "Optional photo of the meter (requires Firebase Storage)", - }, meter_version: { type: "integer", minimum: 1, @@ -504,11 +498,6 @@ const swaggerSpec = { reading_date: { $ref: "#/components/schemas/Timestamp", }, - image_url: { - type: "string", - format: "uri", - description: "Optional photo URL (requires Firebase Storage to be configured)", - }, }, required: ["meter_group_id", "property_id", "reading_amount", "reading_date"], }, @@ -530,10 +519,6 @@ const swaggerSpec = { reading_date: { $ref: "#/components/schemas/Timestamp", }, - image_url: { - type: "string", - format: "uri", - }, }, }, PaginatedReadings: { @@ -935,9 +920,8 @@ const swaggerSpec = { properties: { image_url: { type: "string", - format: "uri", - description: "URL of the bill image to process", - example: "https://storage.googleapis.com/bucket/bills/bill.jpg", + description: "Base64 data URL of the bill image to process (data:image/*;base64,...)", + example: "data:image/jpeg;base64,/9j/4AAQSkZJRg...", }, }, }, @@ -1013,25 +997,6 @@ const swaggerSpec = { }, required: ["provider", "model"], }, - PhotoSettingsResponse: { - type: "object", - properties: { - savePhotos: { - type: "boolean", - description: "Whether meter-reading photos are persisted (image_url) on create. Defaults to false. Bill/billing-cycle photos are never persisted regardless.", - }, - }, - required: ["savePhotos"], - }, - UpsertPhotoSettingsRequest: { - type: "object", - properties: { - savePhotos: { - type: "boolean", - }, - }, - required: ["savePhotos"], - }, ChatHistoryMessage: { type: "object", properties: { @@ -1128,7 +1093,6 @@ const swaggerSpec = { ...reportsPaths, ...llmConfigPaths, ...chatbotPaths, - ...photoSettingsPaths, }, }; diff --git a/api/functions/src/constants/collection.constants.ts b/api/functions/src/constants/collection.constants.ts index 854fb22..08fc7c8 100644 --- a/api/functions/src/constants/collection.constants.ts +++ b/api/functions/src/constants/collection.constants.ts @@ -9,5 +9,4 @@ USERS: "users", READING_LOCKS: "reading_locks", LLM_CONFIG: "llm_config", - PHOTO_SETTINGS: "photo_settings", }; diff --git a/api/functions/src/features/billing-cycle/billing-cycle.controller.ts b/api/functions/src/features/billing-cycle/billing-cycle.controller.ts index b851bf8..30c0e26 100644 --- a/api/functions/src/features/billing-cycle/billing-cycle.controller.ts +++ b/api/functions/src/features/billing-cycle/billing-cycle.controller.ts @@ -11,15 +11,14 @@ import { } from "./billing-cycle.dto"; import {AppError} from "../../utils/error.util"; import {ImageExtractionService} from "../image-extraction/image-extraction.service"; -import {cacheDelPattern} from "../../utils/cache.util"; +import {makeClearCacheHandler} from "../../utils/clear-cache-handler.util"; export const createBillingCycle = async ( req: AuthenticatedRequest, res: Response ): Promise => { const data = req.body as CreateBillingCycleDTO; - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; const result = await billingCycleService.create(userId, data); res.status(201).json(result); }; @@ -29,8 +28,7 @@ export const createBatchBillingCycles = async ( res: Response ): Promise => { const data = req.body as CreateBillingCycleDTO[]; - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; const result = await billingCycleService.createBatch(userId, data); res.status(201).json(result); }; @@ -40,8 +38,7 @@ export const getBillingCycleById = async ( res: Response ): Promise => { const {id} = req.params as unknown as BillingCycleByIdParamsDTO; - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; const billingCycle = await billingCycleService.getById(userId, id); if (!billingCycle) { @@ -56,10 +53,10 @@ export const getBillingCycles = async ( res: Response ): Promise => { const query = req.query as unknown as GetBillingCyclesQueryDTO; - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; const result = await billingCycleService.search(userId, { + meterGroupId: query.meterGroupId, billingStartDate: query.billingStartDate, billingEndDate: query.billingEndDate, sortBy: query.sortBy, @@ -77,8 +74,7 @@ export const updateBillingCycle = async ( ): Promise => { const {id} = req.params as unknown as BillingCycleByIdParamsDTO; const data = req.body as Partial; - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; const result = await billingCycleService.update(userId, id, data); res.status(200).json(result); }; @@ -88,8 +84,7 @@ export const updateBatchBillingCycles = async ( res: Response ): Promise => { const updates = req.body as { id: string; data: Partial }[]; - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; const result = await billingCycleService.updateBatch(userId, updates); res.status(200).json(result); }; @@ -99,8 +94,7 @@ export const deleteBillingCycle = async ( res: Response ): Promise => { const {id} = req.params as unknown as BillingCycleByIdParamsDTO; - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; await billingCycleService.delete(userId, id); res.status(204).send(); }; @@ -110,8 +104,7 @@ export const softDeleteBillingCycle = async ( res: Response ): Promise => { const {id} = req.params as unknown as BillingCycleByIdParamsDTO; - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; const result = await billingCycleService.softDelete(userId, id); res.status(200).json(result); }; @@ -121,8 +114,7 @@ export const restoreBillingCycle = async ( res: Response ): Promise => { const {id} = req.params as unknown as BillingCycleByIdParamsDTO; - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; const result = await billingCycleService.restore(userId, id); res.status(200).json(result); }; @@ -132,8 +124,7 @@ export const purgeBillingCycle = async ( res: Response ): Promise => { const {id} = req.params as unknown as BillingCycleByIdParamsDTO; - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; await billingCycleService.purge(userId, id); res.status(204).send(); }; @@ -143,8 +134,7 @@ export const ocrBillingCycle = async ( res: Response ): Promise => { const {image_url} = req.body as OcrBillingCycleDTO; - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; const extracted = await ImageExtractionService.extractBillingFromImage(image_url, userId); const validated = OcrBillingCycleResponseSchema.parse({ billing_start_date: extracted.billing_start_date, @@ -156,10 +146,4 @@ export const ocrBillingCycle = async ( res.status(200).json(validated); }; -export const clearCache = async ( - _req: AuthenticatedRequest, - res: Response -): Promise => { - const deletedCount = await cacheDelPattern("utilitool:billing-cycles:*"); - res.status(200).json({message: `Cleared ${deletedCount} cache entries for billing cycles`}); -}; +export const clearCache = makeClearCacheHandler("billing-cycles", "billing cycles"); diff --git a/api/functions/src/features/billing-cycle/billing-cycle.dto.ts b/api/functions/src/features/billing-cycle/billing-cycle.dto.ts index ab287dd..da1444c 100644 --- a/api/functions/src/features/billing-cycle/billing-cycle.dto.ts +++ b/api/functions/src/features/billing-cycle/billing-cycle.dto.ts @@ -80,6 +80,7 @@ export type BillingCycleByIdParamsDTO = z.infer; diff --git a/api/functions/src/features/billing-cycle/billing-cycle.route.ts b/api/functions/src/features/billing-cycle/billing-cycle.route.ts index 0a4439f..68606e3 100644 --- a/api/functions/src/features/billing-cycle/billing-cycle.route.ts +++ b/api/functions/src/features/billing-cycle/billing-cycle.route.ts @@ -23,6 +23,7 @@ import { } from "./billing-cycle.dto"; import {validateRequest} from "../../middlewares/validate-request.middleware"; import {requireRole} from "../../middlewares/require-role.middleware"; +import {ocrRateLimiter} from "../../config/rate-limit.config"; const router = Router(); @@ -48,6 +49,7 @@ router.patch( router.post( "/ocr", + ocrRateLimiter, validateRequest({body: OcrBillingCycleDTOSchema}), requireRole("admin", "landlord"), ocrBillingCycle diff --git a/api/functions/src/features/billing-cycle/billing-cycle.service.ts b/api/functions/src/features/billing-cycle/billing-cycle.service.ts index 98c7962..cc65bd3 100644 --- a/api/functions/src/features/billing-cycle/billing-cycle.service.ts +++ b/api/functions/src/features/billing-cycle/billing-cycle.service.ts @@ -13,10 +13,15 @@ import {firestore} from "../../config/firebase.config"; import {COLLECTIONS} from "../../constants/collection.constants"; import {snapshotToModel} from "../../utils/firestore.util"; import {BatchCreateResult} from "../../utils/batch-result.util"; +import {applyDateRangeFilter} from "../../utils/date-range-filter.util"; const validator = new BillingCycleValidator(); const CACHE_TTL = 15 * 60; // 15 minutes +function repoFor(userId: string): CachedRepository { + return new CachedRepository(billingCycleRepository, userId, "billing-cycles", CACHE_TTL); +} + async function injectMainMeterBilling( userId: string, data: CreateBillingCycleDTO @@ -112,6 +117,7 @@ async function injectMainMeterBilling( } type BillingCycleSearchOptions = { + meterGroupId?: string; billingStartDate?: string; billingEndDate?: string; sortBy?: string; @@ -129,7 +135,7 @@ export const billingCycleService = { await validator.validateSubmeterConsumption(data); const enrichedData = await injectMainMeterBilling(userId, data); await validator.validateCreate(enrichedData); - const cachedRepo = new CachedRepository(billingCycleRepository, userId, "billing-cycles", CACHE_TTL); + const cachedRepo = repoFor(userId); return cachedRepo.create(enrichedData); }, @@ -181,7 +187,7 @@ export const billingCycleService = { let created: BillingCycle[] = []; if (toCreate.length > 0) { - const cachedRepo = new CachedRepository(billingCycleRepository, userId, "billing-cycles", CACHE_TTL); + const cachedRepo = repoFor(userId); created = await cachedRepo.createBatch(toCreate); } @@ -193,11 +199,13 @@ export const billingCycleService = { userId: string, options: BillingCycleSearchOptions ): Promise> { - const cachedRepo = new CachedRepository(billingCycleRepository, userId, "billing-cycles", CACHE_TTL); + const cachedRepo = repoFor(userId); // For archived queries, we need custom date filtering if (options.archived) { - const filters: Record = {}; + const filters: Record = { + ...(options.meterGroupId ? {meter_group_id: options.meterGroupId} : {}), + }; if (options.billingStartDate) { filters.billing_start_date = {gte: new Date(options.billingStartDate)}; } @@ -221,32 +229,24 @@ export const billingCycleService = { orderDirection: options.sortOrder ?? "desc", cursor: options.cursor, archived: false, - filters: {}, + filters: { + ...(options.meterGroupId ? {meter_group_id: options.meterGroupId} : {}), + }, }); // Post-filter for date ranges (can't query dates directly in Firestore query) - if (options.billingStartDate || options.billingEndDate) { - if (options.billingStartDate) { - const startDate = new Date(options.billingStartDate); - result.data = result.data.filter((bc) => { - const bcDate = bc.billing_start_date instanceof Date ? bc.billing_start_date : new Date(bc.billing_start_date as any); - return bcDate >= startDate; - }); - } - if (options.billingEndDate) { - const endDate = new Date(options.billingEndDate); - result.data = result.data.filter((bc) => { - const bcDate = bc.billing_end_date instanceof Date ? bc.billing_end_date : new Date(bc.billing_end_date as any); - return bcDate <= endDate; - }); - } - } + result.data = applyDateRangeFilter(result.data, { + startDate: options.billingStartDate, + endDate: options.billingEndDate, + startField: "billing_start_date", + endField: "billing_end_date", + }); return result; }, async getById(userId: string, id: string): Promise { - const cachedRepo = new CachedRepository(billingCycleRepository, userId, "billing-cycles", CACHE_TTL); + const cachedRepo = repoFor(userId); return cachedRepo.getById(id); }, @@ -255,24 +255,24 @@ export const billingCycleService = { id: string, data: Partial ): Promise { - await validator.validateUpdate(data); - const cachedRepo = new CachedRepository(billingCycleRepository, userId, "billing-cycles", CACHE_TTL); + await validator.validateUpdate(id, data); + const cachedRepo = repoFor(userId); return cachedRepo.update(id, data); }, async updateBatch(userId: string, updates: {id: string, data: Partial}[]): Promise { await validator.validateUpdateBatch(updates); - const cachedRepo = new CachedRepository(billingCycleRepository, userId, "billing-cycles", CACHE_TTL); + const cachedRepo = repoFor(userId); return cachedRepo.updateBatch(updates); }, async delete(userId: string, id: string): Promise { - const cachedRepo = new CachedRepository(billingCycleRepository, userId, "billing-cycles", CACHE_TTL); + const cachedRepo = repoFor(userId); await cachedRepo.delete(id); }, async softDelete(userId: string, id: string): Promise { - const cachedRepo = new CachedRepository(billingCycleRepository, userId, "billing-cycles", CACHE_TTL); + const cachedRepo = repoFor(userId); return cachedRepo.softDelete(id); }, @@ -281,7 +281,7 @@ export const billingCycleService = { if (!billingCycle) { throw new AppError(404, "Billing cycle not found"); } - const cachedRepo = new CachedRepository(billingCycleRepository, userId, "billing-cycles", CACHE_TTL); + const cachedRepo = repoFor(userId); return cachedRepo.restore(id); }, @@ -290,7 +290,7 @@ export const billingCycleService = { * archive-then-purge lifecycle — throws 409 if the cycle is still active. */ async purge(userId: string, id: string): Promise { - const cachedRepo = new CachedRepository(billingCycleRepository, userId, "billing-cycles", CACHE_TTL); + const cachedRepo = repoFor(userId); await cachedRepo.purge(id); }, }; diff --git a/api/functions/src/features/billing-cycle/billing-cycle.swagger.ts b/api/functions/src/features/billing-cycle/billing-cycle.swagger.ts index c2e50da..9a663a2 100644 --- a/api/functions/src/features/billing-cycle/billing-cycle.swagger.ts +++ b/api/functions/src/features/billing-cycle/billing-cycle.swagger.ts @@ -77,6 +77,15 @@ export const billingCyclePaths = { "Retrieve paginated list of billing cycles. Can filter by start and end dates. Note: cursor and date filters are mutually exclusive.", security: [{BearerAuth: []}], parameters: [ + { + name: "meterGroupId", + in: "query", + description: "Filter by meter group ID", + schema: { + type: "string", + minLength: 1, + }, + }, { name: "billingStartDate", in: "query", @@ -202,7 +211,7 @@ export const billingCyclePaths = { properties: { image_url: { type: "string", - description: "Data URL (data:image/...) or public HTTPS URL of the utility bill photo", + description: "Base64 data URL of the utility bill photo (data:image/*;base64,...)", }, }, }, diff --git a/api/functions/src/features/billing-cycle/billing-cycle.validator.test.ts b/api/functions/src/features/billing-cycle/billing-cycle.validator.test.ts index 8460f22..b14607c 100644 --- a/api/functions/src/features/billing-cycle/billing-cycle.validator.test.ts +++ b/api/functions/src/features/billing-cycle/billing-cycle.validator.test.ts @@ -171,6 +171,7 @@ describe('BillingCycleValidator', () => { it('accepts the correct true-reading delta across the reset (5.5)', async () => { await expect( validator.validateSubmeterConsumption({ + meter_group_id: 'mg-1', billing_ids: { 'b-1': 5.5 }, } as any) ).resolves.toBeUndefined(); @@ -179,6 +180,7 @@ describe('BillingCycleValidator', () => { it('rejects an overstated consumption that used the full true reading (639.5) instead of the delta', async () => { await expect( validator.validateSubmeterConsumption({ + meter_group_id: 'mg-1', billing_ids: { 'b-1': 639.5 }, } as any) ).rejects.toMatchObject({ @@ -187,4 +189,80 @@ describe('BillingCycleValidator', () => { }); }); }); + + describe('meter_group_id cross-check against billings\' readings', () => { + const prevReading = { + id: 'r-1', + meter_group_id: 'mg-1', + property_id: 'prop-1', + reading_amount: 100, + reading_date: startTs, + meter_version: 1, + }; + const currReading = { + id: 'r-2', + meter_group_id: 'mg-1', + property_id: 'prop-1', + reading_amount: 200, + reading_date: endTs, + meter_version: 1, + }; + const property = { id: 'prop-1', meter_groups: {} }; + const meterGroup = { id: 'mg-1', current_version: 1, versions: {} }; + + beforeEach(() => { + jest.mocked(readingRepository.getByIds).mockResolvedValue([prevReading, currReading] as any); + jest.mocked(propertyRepository.getByIds).mockResolvedValue([property] as any); + jest.mocked(meterGroupRepository.getByIds).mockResolvedValue([meterGroup] as any); + }); + + it('rejects create when the cycle meter_group_id does not match its billings\' readings', async () => { + await expect( + validator.validateCreate({ + meter_group_id: 'mg-OTHER', + billing_ids: { 'b-1': 100 }, + billing_rate: 5, + billing_consumption: 100, + billing_start_date: startTs, + billing_end_date: endTs, + } as any) + ).rejects.toMatchObject({ + statusCode: 400, + message: expect.stringContaining('meter group'), + }); + }); + + it('accepts create when the cycle meter_group_id matches its billings\' readings', async () => { + await expect( + validator.validateCreate({ + meter_group_id: 'mg-1', + billing_ids: { 'b-1': 100 }, + billing_rate: 5, + billing_consumption: 100, + billing_start_date: startTs, + billing_end_date: endTs, + } as any) + ).resolves.toBeUndefined(); + }); + + it('rejects a PATCH changing only meter_group_id to one that does not match the stored billings', async () => { + // Stored cycle is for mg-1 with b-1; PATCH switches only meter_group_id to + // mg-OTHER, which no longer matches b-1's reading (mg-1). + jest.mocked(billingCycleRepository.getById).mockResolvedValue( + { id: 'bc-1', meter_group_id: 'mg-1', billing_ids: { 'b-1': 100 } } as any + ); + await expect( + validator.validateUpdate('bc-1', { meter_group_id: 'mg-OTHER' }) + ).rejects.toMatchObject({ statusCode: 400 }); + }); + + it('accepts a PATCH changing only meter_group_id to one that matches the stored billings', async () => { + jest.mocked(billingCycleRepository.getById).mockResolvedValue( + { id: 'bc-1', meter_group_id: 'mg-OTHER', billing_ids: { 'b-1': 100 } } as any + ); + await expect( + validator.validateUpdate('bc-1', { meter_group_id: 'mg-1' }) + ).resolves.toBeUndefined(); + }); + }); }); diff --git a/api/functions/src/features/billing-cycle/billing-cycle.validator.ts b/api/functions/src/features/billing-cycle/billing-cycle.validator.ts index 3179bc6..955d3d1 100644 --- a/api/functions/src/features/billing-cycle/billing-cycle.validator.ts +++ b/api/functions/src/features/billing-cycle/billing-cycle.validator.ts @@ -27,7 +27,8 @@ export class BillingCycleValidator { } private async validateBillingConsumptionAmounts( - billingIds: Record + billingIds: Record, + expectedMeterGroupId: string ): Promise { const PER_BILLING_TOLERANCE = 0.05; const billingIdList = Object.keys(billingIds); @@ -76,6 +77,19 @@ export class BillingCycleValidator { if (!prevReading || !currReading) continue; + // Cross-check the cycle's declared meter_group_id against the meter group + // of the reading each billing actually summarizes — otherwise a cycle can + // claim a meter_group_id that doesn't match its billings' readings, and + // the denormalized field silently lies (nothing else validates it). + if (currReading.meter_group_id !== expectedMeterGroupId) { + throw new AppError( + 400, + `Billing ${billingId}: its reading belongs to meter group ` + + `${currReading.meter_group_id}, but the billing cycle is for meter group ` + + `${expectedMeterGroupId}` + ); + } + const meterGroup = meterGroupMap.get(currReading.meter_group_id); const property = propertyMap.get(billing.property_id); @@ -181,7 +195,7 @@ export class BillingCycleValidator { } await this.validateBillingIdsExist(billingIds); - await this.validateBillingConsumptionAmounts(data.billing_ids); + await this.validateBillingConsumptionAmounts(data.billing_ids, data.meter_group_id); } async validateCreate(data: CreateBillingCycleDTO): Promise { @@ -192,7 +206,7 @@ export class BillingCycleValidator { } await this.validateBillingIdsExist(billingIds); - await this.validateBillingConsumptionAmounts(data.billing_ids); + await this.validateBillingConsumptionAmounts(data.billing_ids, data.meter_group_id); await this.ensureBillingCycleNotDuplicate(data.meter_group_id, data.billing_start_date); this.validateBillingDates( @@ -239,6 +253,7 @@ export class BillingCycleValidator { } async validateUpdate( + id: string, data: Partial ): Promise { if (data.billing_ids !== undefined) { @@ -249,7 +264,24 @@ export class BillingCycleValidator { } await this.validateBillingIdsExist(billingIds); - await this.validateBillingConsumptionAmounts(data.billing_ids); + } + + // Re-run the meter-group cross-check whenever billing_ids OR meter_group_id + // changes — a PATCH touching only meter_group_id (no billing_ids in the + // payload) would otherwise bypass validation entirely. Resolve whichever + // side isn't in this PATCH from the stored cycle. + if (data.billing_ids !== undefined || data.meter_group_id !== undefined) { + let effectiveBillingIds = data.billing_ids; + let effectiveMeterGroupId = data.meter_group_id; + if (!effectiveBillingIds || !effectiveMeterGroupId) { + const existing = await billingCycleRepository.getById(id); + if (!existing) { + throw new AppError(404, "Billing cycle not found"); + } + effectiveBillingIds = effectiveBillingIds ?? existing.billing_ids; + effectiveMeterGroupId = effectiveMeterGroupId ?? existing.meter_group_id; + } + await this.validateBillingConsumptionAmounts(effectiveBillingIds, effectiveMeterGroupId); } if (data.billing_start_date && data.billing_end_date) { @@ -279,8 +311,8 @@ export class BillingCycleValidator { async validateUpdateBatch( updates: {id: string; data: Partial}[] ): Promise { - for (const {data} of updates) { - await this.validateUpdate(data); + for (const {id, data} of updates) { + await this.validateUpdate(id, data); } } } diff --git a/api/functions/src/features/billing/billing.controller.ts b/api/functions/src/features/billing/billing.controller.ts index 39b0821..7debaa9 100644 --- a/api/functions/src/features/billing/billing.controller.ts +++ b/api/functions/src/features/billing/billing.controller.ts @@ -8,15 +8,15 @@ import { UpdateBillingDTO, } from "./billing.dto"; import {AppError} from "../../utils/error.util"; -import {cacheDelPattern} from "../../utils/cache.util"; +import {makeClearCacheHandler} from "../../utils/clear-cache-handler.util"; +import type {BatchGetQueryDTO} from "../../utils/batch-get.dto"; export const createBilling = async ( req: AuthenticatedRequest, res: Response ): Promise => { const data = req.body as CreateBillingDTO; - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; const result = await billingService.create(userId, data); res.status(201).json(result); }; @@ -26,8 +26,7 @@ export const createBatchBillings = async ( res: Response ): Promise => { const data = req.body as CreateBillingDTO[]; - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; const result = await billingService.createBatch(userId, data); res.status(201).json(result); }; @@ -37,8 +36,7 @@ export const getBillingById = async ( res: Response ): Promise => { const {id} = req.params as unknown as BillingByIdParamsDTO; - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; const billing = await billingService.getById(userId, id); if (!billing) { @@ -48,16 +46,27 @@ export const getBillingById = async ( res.status(200).json(billing); }; +export const getBillingsByIds = async ( + req: AuthenticatedRequest, + res: Response +): Promise => { + const {ids} = req.query as unknown as BatchGetQueryDTO; + const billings = await billingService.getByIds(ids); + res.status(200).json(billings); +}; + export const getBillings = async ( req: AuthenticatedRequest, res: Response ): Promise => { const query = req.query as unknown as GetBillingsQueryDTO; - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; const result = await billingService.search(userId, { propertyId: query.propertyId, + meterGroupId: query.meterGroupId, + startDate: query.startDate, + endDate: query.endDate, sortBy: query.sortBy, sortOrder: query.sortOrder, limit: query.limit, @@ -73,8 +82,7 @@ export const updateBilling = async ( ): Promise => { const {id} = req.params as unknown as BillingByIdParamsDTO; const data = req.body as Partial; - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; const result = await billingService.update(userId, id, data); res.status(200).json(result); }; @@ -84,8 +92,7 @@ export const updateBatchBillings = async ( res: Response ): Promise => { const updates = req.body as { id: string; data: Partial }[]; - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; const result = await billingService.updateBatch(userId, updates); res.status(200).json(result); }; @@ -95,8 +102,7 @@ export const deleteBilling = async ( res: Response ): Promise => { const {id} = req.params as unknown as BillingByIdParamsDTO; - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; await billingService.delete(userId, id); res.status(204).send(); }; @@ -106,8 +112,7 @@ export const softDeleteBilling = async ( res: Response ): Promise => { const {id} = req.params as unknown as BillingByIdParamsDTO; - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; const result = await billingService.softDelete(userId, id); res.status(200).json(result); }; @@ -117,8 +122,7 @@ export const restoreBilling = async ( res: Response ): Promise => { const {id} = req.params as unknown as BillingByIdParamsDTO; - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; const result = await billingService.restore(userId, id); res.status(200).json(result); }; @@ -128,16 +132,9 @@ export const purgeBilling = async ( res: Response ): Promise => { const {id} = req.params as unknown as BillingByIdParamsDTO; - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; await billingService.purge(userId, id); res.status(204).send(); }; -export const clearCache = async ( - _req: AuthenticatedRequest, - res: Response -): Promise => { - const deletedCount = await cacheDelPattern("utilitool:billings:*"); - res.status(200).json({message: `Cleared ${deletedCount} cache entries for billings`}); -}; +export const clearCache = makeClearCacheHandler("billings", "billings"); diff --git a/api/functions/src/features/billing/billing.dto.ts b/api/functions/src/features/billing/billing.dto.ts index ca5e050..02a4eda 100644 --- a/api/functions/src/features/billing/billing.dto.ts +++ b/api/functions/src/features/billing/billing.dto.ts @@ -37,6 +37,9 @@ export type BillingByIdParamsDTO = z.infer; export const GetBillingsQueryDTOSchema = z .object({ propertyId: z.string().trim().min(1).optional(), + meterGroupId: z.string().trim().min(1).optional(), + startDate: z.union([z.string().datetime(), z.string().date()]).optional(), + endDate: z.union([z.string().datetime(), z.string().date()]).optional(), sortBy: z.enum(["created_at", "payment_status"]).optional(), sortOrder: z.enum(["asc", "desc"]).optional(), limit: z.coerce.number().int().min(1).max(100).default(20), @@ -53,5 +56,19 @@ export const GetBillingsQueryDTOSchema = z path: ["cursor"], }); } + if (value.meterGroupId && value.cursor) { + context.addIssue({ + code: "custom", + message: "cursor cannot be combined with meterGroupId", + path: ["cursor"], + }); + } + if ((value.startDate || value.endDate) && value.cursor) { + context.addIssue({ + code: "custom", + message: "cursor cannot be combined with startDate or endDate", + path: ["cursor"], + }); + } }); export type GetBillingsQueryDTO = z.infer; diff --git a/api/functions/src/features/billing/billing.model.ts b/api/functions/src/features/billing/billing.model.ts index 160c3de..1ef00c0 100644 --- a/api/functions/src/features/billing/billing.model.ts +++ b/api/functions/src/features/billing/billing.model.ts @@ -1,9 +1,12 @@ -import {BaseModel} from "../../utils/model.util"; +import {Timestamp} from "firebase-admin/firestore"; +import {BaseModel} from "../../utils/model.util"; export interface Billing extends BaseModel { property_id: string; previous_reading_id: string; current_reading_id: string; + meter_group_id: string; + billing_period_date: Timestamp; payment_status: "pending" | "paid"; paid_at?: string; } diff --git a/api/functions/src/features/billing/billing.route.ts b/api/functions/src/features/billing/billing.route.ts index 7388cdd..eb07762 100644 --- a/api/functions/src/features/billing/billing.route.ts +++ b/api/functions/src/features/billing/billing.route.ts @@ -2,6 +2,7 @@ import {Router} from "express"; import { createBilling, getBillingById, + getBillingsByIds, getBillings, updateBilling, softDeleteBilling, @@ -19,6 +20,7 @@ import { UpdateBillingBatchDTOSchema, UpdateBillingDTOSchema, } from "./billing.dto"; +import {BatchGetQueryDTOSchema} from "../../utils/batch-get.dto"; import {validateRequest} from "../../middlewares/validate-request.middleware"; import {requireRole} from "../../middlewares/require-role.middleware"; @@ -50,6 +52,12 @@ router.post( createBilling ); +router.get( + "/batch-get", + validateRequest({query: BatchGetQueryDTOSchema}), + getBillingsByIds +); + router.get( "/", validateRequest({query: GetBillingsQueryDTOSchema}), diff --git a/api/functions/src/features/billing/billing.service.ts b/api/functions/src/features/billing/billing.service.ts index 154848b..657a66d 100644 --- a/api/functions/src/features/billing/billing.service.ts +++ b/api/functions/src/features/billing/billing.service.ts @@ -1,4 +1,4 @@ -import {FieldValue, Transaction, DocumentData} from "firebase-admin/firestore"; +import {FieldValue, Transaction, DocumentData, Timestamp} from "firebase-admin/firestore"; import {firestore} from "../../config/firebase.config"; import {billingRepository} from "./billing.repository"; import {Billing} from "./billing.model"; @@ -8,15 +8,41 @@ import {BillingValidator} from "./billing.validator"; import {AppError} from "../../utils/error.util"; import {COLLECTIONS} from "../../constants/collection.constants"; import {snapshotToModel} from "../../utils/firestore.util"; +import {applyDateRangeFilter} from "../../utils/date-range-filter.util"; import {validateMeterRollback} from "../reading/reading.util"; import {cacheSet} from "../../utils/cache.util"; +import {listAppend} from "../../utils/list-cache.util"; import {CachedRepository} from "../../lib/cached-repository.lib"; +import {readingRepository} from "../reading/reading.repository"; const validator = new BillingValidator(); const CACHE_TTL = 10 * 60; // 10 minutes +function repoFor(userId: string): CachedRepository { + return new CachedRepository(billingRepository, userId, "billings", CACHE_TTL); +} + +/** + * Derives the denormalized fields Billing copies off its current reading + * (meter_group_id + billing_period_date). Single source of the copy logic so + * create/createBatch/createFromReadings and update() can't drift apart. + * Accepts any reading-shaped value (DocumentData from a txn snapshot or a + * resolved Reading model) — both carry meter_group_id/reading_date. + */ +function deriveBillingDenormalizedFields( + currReading: { meter_group_id: string; reading_date: Timestamp } +): { meter_group_id: string; billing_period_date: Timestamp } { + return { + meter_group_id: currReading.meter_group_id, + billing_period_date: currReading.reading_date, + }; +} + type BillingSearchOptions = { propertyId?: string; + meterGroupId?: string; + startDate?: string; + endDate?: string; sortBy?: string; sortOrder?: "asc" | "desc"; limit: number; @@ -71,6 +97,7 @@ export const billingService = { newBillingId = newRef.id; txn.set(newRef, { ...data, + ...deriveBillingDenormalizedFields(currReading as any), payment_status: "pending" as const, created_at: FieldValue.serverTimestamp(), is_deleted: false, @@ -82,65 +109,162 @@ export const billingService = { const snap = await firestore.collection(COLLECTIONS.BILLINGS).doc(newBillingId!).get(); const billing = snapshotToModel(snap); await cacheSet(`utilitool:billings:id:${billing.id}`, billing, CACHE_TTL); + await listAppend(`utilitool:billings:all:${userId}`, billing, CACHE_TTL); return billing; }, async createBatch(userId: string, data: CreateBillingDTO[]): Promise { await validator.validateBatch(data); - const cachedRepo = new CachedRepository(billingRepository, userId, "billings", CACHE_TTL); + + // Batch is capped at 10 items (CreateBillingBatchDTOSchema), so this is a small, + // one-time lookup — resolves meter_group_id/reading_date for the denormalized fields + // below, mirroring what create()/createFromReadings() already get for free from their + // transaction reads. + const currentReadingIds = Array.from(new Set(data.map((item) => item.current_reading_id))); + const currentReadings = await readingRepository.getByIds(currentReadingIds); + const readingById = new Map( + currentReadings.filter((r): r is NonNullable => r !== null).map((r) => [r.id, r]) + ); + + const cachedRepo = repoFor(userId); const created = await cachedRepo.createBatch( - data.map((item) => ({ - ...item, - payment_status: "pending" as const, - })) + data.map((item) => { + const currReading = readingById.get(item.current_reading_id)!; + return { + ...item, + ...deriveBillingDenormalizedFields(currReading), + payment_status: "pending" as const, + }; + }) ); return created; }, async search(userId: string, options: BillingSearchOptions): Promise> { - const cachedRepo = new CachedRepository(billingRepository, userId, "billings", CACHE_TTL); - return cachedRepo.search({ + const cachedRepo = repoFor(userId); + + // Archived queries go direct to Firestore, same as billing-cycle.service.ts — + // range filters can be pushed down to the query since there's no list cache involved. + if (options.archived) { + const filters: Record = { + ...(options.propertyId ? {property_id: options.propertyId} : {}), + ...(options.meterGroupId ? {meter_group_id: options.meterGroupId} : {}), + }; + if (options.startDate) { + filters.billing_period_date = {...filters.billing_period_date, gte: new Date(options.startDate)}; + } + if (options.endDate) { + filters.billing_period_date = {...filters.billing_period_date, lte: new Date(options.endDate)}; + } + return billingRepository.search({ + limit: options.limit, + orderBy: (options.sortBy ?? "created_at") as any, + orderDirection: options.sortOrder ?? "desc", + cursor: options.cursor, + archived: true, + filters, + }); + } + + // Active items: load the cached list, apply equality filters via CachedRepository, + // then post-filter date range in memory — CachedRepository.applyFilters rejects range + // filters outright, so date filtering can't be pushed into the same call. Mirrors the + // same two-step pattern billing-cycle.service.ts uses for its date filters. + const result = await cachedRepo.search({ limit: options.limit, orderBy: (options.sortBy ?? "created_at") as any, orderDirection: options.sortOrder ?? "desc", cursor: options.cursor, - archived: options.archived, + archived: false, filters: { ...(options.propertyId ? {property_id: options.propertyId} : {}), + ...(options.meterGroupId ? {meter_group_id: options.meterGroupId} : {}), }, }); + + result.data = applyDateRangeFilter(result.data, { + startDate: options.startDate, + endDate: options.endDate, + startField: "billing_period_date", + endField: "billing_period_date", + }); + + return result; }, async getById(userId: string, id: string): Promise { - const cachedRepo = new CachedRepository(billingRepository, userId, "billings", CACHE_TTL); + const cachedRepo = repoFor(userId); return cachedRepo.getById(id); }, + /** + * Batch ID lookup for clients that otherwise resolve IDs one at a time + * (e.g. the UI's entity-lookup-cache.ts) — one round-trip instead of N. + * Goes through the raw repository (not the per-user list cache), same as + * every other getByIds caller in this codebase (reports.service.ts, etc.). + */ + async getByIds(ids: string[]): Promise { + const results = await billingRepository.getByIds(ids); + return results.filter((b): b is Billing => b !== null); + }, + async update(userId: string, id: string, data: Partial & { payment_status?: "pending" | "paid"; paid_at?: string }): Promise { - await validator.validateUpdate(data); + await validator.validateUpdate(id, data); - const updateData = {...data}; + const updateData: Record = {...data}; if (data.payment_status === "paid" && !data.paid_at) { updateData.paid_at = new Date().toISOString(); } - const cachedRepo = new CachedRepository(billingRepository, userId, "billings", CACHE_TTL); + // Keep the denormalized fields in sync when the current reading changes via + // the PATCH correction escape hatch — otherwise meter_group_id / + // billing_period_date would silently point at the old reading. + if (data.current_reading_id) { + const currReading = await readingRepository.getById(data.current_reading_id); + if (!currReading) { + throw new AppError(404, "Current reading not found"); + } + Object.assign(updateData, deriveBillingDenormalizedFields(currReading)); + } + + const cachedRepo = repoFor(userId); return cachedRepo.update(id, updateData); }, async updateBatch(userId: string, updates: {id: string, data: Partial}[]): Promise { await validator.validateUpdateBatch(updates); - const cachedRepo = new CachedRepository(billingRepository, userId, "billings", CACHE_TTL); - return cachedRepo.updateBatch(updates); + + // Re-derive denormalized fields for any item whose current reading changes, + // mirroring update()'s single-item sync. + const changedReadingIds = Array.from( + new Set(updates.map((u) => u.data.current_reading_id).filter((x): x is string => !!x)) + ); + const readingById = new Map>>(); + if (changedReadingIds.length > 0) { + const readings = await readingRepository.getByIds(changedReadingIds); + readings.forEach((r) => r && readingById.set(r.id, r)); + } + const enriched = updates.map((u) => { + if (u.data.current_reading_id) { + const currReading = readingById.get(u.data.current_reading_id); + if (currReading) { + return {id: u.id, data: {...u.data, ...deriveBillingDenormalizedFields(currReading)}}; + } + } + return u; + }); + + const cachedRepo = repoFor(userId); + return cachedRepo.updateBatch(enriched); }, async delete(userId: string, id: string): Promise { - const cachedRepo = new CachedRepository(billingRepository, userId, "billings", CACHE_TTL); + const cachedRepo = repoFor(userId); await cachedRepo.delete(id); }, async softDelete(userId: string, id: string): Promise { - const cachedRepo = new CachedRepository(billingRepository, userId, "billings", CACHE_TTL); + const cachedRepo = repoFor(userId); return cachedRepo.softDelete(id); }, @@ -149,7 +273,7 @@ export const billingService = { if (!billing) { throw new AppError(404, "Billing not found"); } - const cachedRepo = new CachedRepository(billingRepository, userId, "billings", CACHE_TTL); + const cachedRepo = repoFor(userId); return cachedRepo.restore(id); }, @@ -158,7 +282,7 @@ export const billingService = { * archive-then-purge lifecycle — throws 409 if the billing is still active. */ async purge(userId: string, id: string): Promise { - const cachedRepo = new CachedRepository(billingRepository, userId, "billings", CACHE_TTL); + const cachedRepo = repoFor(userId); await cachedRepo.purge(id); }, @@ -198,6 +322,7 @@ export const billingService = { property_id: propertyId, previous_reading_id: prevReadingId, current_reading_id: currReadingId, + ...deriveBillingDenormalizedFields(currReading as any), payment_status: "pending" as const, created_at: FieldValue.serverTimestamp(), is_deleted: false, diff --git a/api/functions/src/features/billing/billing.swagger.ts b/api/functions/src/features/billing/billing.swagger.ts index bfd2490..80762b7 100644 --- a/api/functions/src/features/billing/billing.swagger.ts +++ b/api/functions/src/features/billing/billing.swagger.ts @@ -86,6 +86,31 @@ export const billingPaths = { minLength: 1, }, }, + { + name: "meterGroupId", + in: "query", + description: "Filter by meter group ID", + schema: { + type: "string", + minLength: 1, + }, + }, + { + name: "startDate", + in: "query", + description: "Filter by billing_period_date >= startDate (ISO 8601)", + schema: { + type: "string", + }, + }, + { + name: "endDate", + in: "query", + description: "Filter by billing_period_date <= endDate (ISO 8601)", + schema: { + type: "string", + }, + }, { name: "limit", in: "query", diff --git a/api/functions/src/features/billing/billing.test.ts b/api/functions/src/features/billing/billing.test.ts index 1e7afc5..6b2a625 100644 --- a/api/functions/src/features/billing/billing.test.ts +++ b/api/functions/src/features/billing/billing.test.ts @@ -1,5 +1,6 @@ jest.mock('./billing.repository'); jest.mock('./billing.validator'); +jest.mock('../reading/reading.repository'); // Mock Firestore transaction used in billingService.create jest.mock('../../config/firebase.config', () => { @@ -31,6 +32,7 @@ jest.mock('../../config/firebase.config', () => { import { describe, it, expect, jest, beforeEach } from '@jest/globals'; import { billingService } from './billing.service'; import { billingRepository } from './billing.repository'; +import { readingRepository } from '../reading/reading.repository'; import { BillingValidator } from './billing.validator'; import { CreateBillingDTOSchema, UpdateBillingDTOSchema, BillingByIdParamsDTOSchema, CreateBillingBatchDTOSchema, UpdateBillingBatchDTOSchema } from './billing.dto'; import { AppError } from '../../utils/error.util'; @@ -80,6 +82,10 @@ describe('billingService', () => { const mocks = [mockBilling({ id: 'billing-1' }), mockBilling({ id: 'billing-2' })]; jest.mocked(BillingValidator.prototype.validateBatch).mockResolvedValue(undefined); jest.mocked(billingRepository.createBatch).mockResolvedValue(mocks); + jest.mocked(readingRepository.getByIds).mockResolvedValue([ + { id: 'reading-2', meter_group_id: 'mg-1', reading_date: now } as any, + { id: 'reading-3', meter_group_id: 'mg-1', reading_date: now } as any, + ]); const input = [ { property_id: 'prop-1', previous_reading_id: 'reading-1', current_reading_id: 'reading-2' }, diff --git a/api/functions/src/features/billing/billing.validator.test.ts b/api/functions/src/features/billing/billing.validator.test.ts index 39a6d6d..2e5e73e 100644 --- a/api/functions/src/features/billing/billing.validator.test.ts +++ b/api/functions/src/features/billing/billing.validator.test.ts @@ -115,4 +115,53 @@ describe('BillingValidator', () => { })).rejects.toMatchObject({ statusCode: 404 }); }); }); + + describe('validateUpdate — correction escape hatch', () => { + const storedBilling = { + id: 'billing-1', + property_id: 'prop-1', + previous_reading_id: 'r-prev', + current_reading_id: 'r-old', + }; + + it('skips cross-entity validation when no reading id changes', async () => { + await expect(validator.validateUpdate('billing-1', {})).resolves.toBeUndefined(); + }); + + it('rejects a lone current_reading_id PATCH pointing at a different meter group', async () => { + jest.mocked(billingRepository.getById).mockResolvedValue(storedBilling as any); + jest.mocked(propertyRepository.getById).mockResolvedValue(mockProperty() as any); + jest.mocked(readingRepository.getById).mockImplementation(async (id: string) => { + if (id === 'r-prev') return mockReading('r-prev', 'mg-1', 100) as any; + if (id === 'r-new') return mockReading('r-new', 'mg-2', 200) as any; + return null; + }); + + await expect( + validator.validateUpdate('billing-1', { current_reading_id: 'r-new' }) + ).rejects.toMatchObject({ statusCode: 400 }); + }); + + it('resolves a lone current_reading_id PATCH when same meter group and greater', async () => { + jest.mocked(billingRepository.getById).mockResolvedValue(storedBilling as any); + jest.mocked(propertyRepository.getById).mockResolvedValue(mockProperty() as any); + jest.mocked(readingRepository.getById).mockImplementation(async (id: string) => { + if (id === 'r-prev') return mockReading('r-prev', 'mg-1', 100) as any; + if (id === 'r-new') return mockReading('r-new', 'mg-1', 300) as any; + return null; + }); + + await expect( + validator.validateUpdate('billing-1', { current_reading_id: 'r-new' }) + ).resolves.toBeUndefined(); + }); + + it('rejects when the billing to update no longer exists', async () => { + jest.mocked(billingRepository.getById).mockResolvedValue(null); + + await expect( + validator.validateUpdate('billing-1', { current_reading_id: 'r-new' }) + ).rejects.toMatchObject({ statusCode: 404 }); + }); + }); }); diff --git a/api/functions/src/features/billing/billing.validator.ts b/api/functions/src/features/billing/billing.validator.ts index 001ed0d..3c701b4 100644 --- a/api/functions/src/features/billing/billing.validator.ts +++ b/api/functions/src/features/billing/billing.validator.ts @@ -158,42 +158,58 @@ export class BillingValidator { } async validateUpdate( + id: string, data: Partial ): Promise { - if ( - data.property_id || - data.previous_reading_id || - data.current_reading_id - ) { - const propertyId = data.property_id; - const previousReadingId = data.previous_reading_id; - const currentReadingId = data.current_reading_id; - - if (propertyId && previousReadingId && currentReadingId) { - const [property, previousReading, currentReading] = await Promise.all([ - propertyRepository.getById(propertyId), - readingRepository.getById(previousReadingId), - readingRepository.getById(currentReadingId), - ]); - - if (!property) { - throw new AppError(404, "Property not found"); - } - if (!previousReading || !currentReading) { - throw new AppError(404, "Reading not found"); - } - - await this.validateReadingsBelongToProperty(property, previousReading, currentReading); - this.validatePropertyIsNotMainMeter(property, currentReading.meter_group_id); + // Only a change to one of the readings needs cross-entity re-validation. + const changesReadingPair = + data.previous_reading_id !== undefined || data.current_reading_id !== undefined; + + if (!changesReadingPair) { + return; + } + + // The correction escape hatch: a PATCH may send only current_reading_id (or + // only previous_reading_id). Resolve the full triple, falling back to the + // stored billing for whatever isn't in this PATCH, so a lone reading change + // still passes the same-meter-group / rollback / main-meter checks instead + // of silently skipping them. + let propertyId = data.property_id; + let previousReadingId = data.previous_reading_id; + let currentReadingId = data.current_reading_id; + + if (!propertyId || !previousReadingId || !currentReadingId) { + const existing = await billingRepository.getById(id); + if (!existing) { + throw new AppError(404, "Billing not found"); } + propertyId = propertyId ?? existing.property_id; + previousReadingId = previousReadingId ?? existing.previous_reading_id; + currentReadingId = currentReadingId ?? existing.current_reading_id; + } + + const [property, previousReading, currentReading] = await Promise.all([ + propertyRepository.getById(propertyId), + readingRepository.getById(previousReadingId), + readingRepository.getById(currentReadingId), + ]); + + if (!property) { + throw new AppError(404, "Property not found"); + } + if (!previousReading || !currentReading) { + throw new AppError(404, "Reading not found"); } + + await this.validateReadingsBelongToProperty(property, previousReading, currentReading); + this.validatePropertyIsNotMainMeter(property, currentReading.meter_group_id); } async validateUpdateBatch( updates: {id: string; data: Partial}[] ): Promise { - for (const {data} of updates) { - await this.validateUpdate(data); + for (const {id, data} of updates) { + await this.validateUpdate(id, data); } } } diff --git a/api/functions/src/features/bills/bills.controller.ts b/api/functions/src/features/bills/bills.controller.ts index e15ddfb..9a7d0c9 100644 --- a/api/functions/src/features/bills/bills.controller.ts +++ b/api/functions/src/features/bills/bills.controller.ts @@ -2,7 +2,6 @@ import {Response} from "express"; import type {AuthenticatedRequest} from "../../utils/auth.util"; import {ImageExtractionService} from "../image-extraction/image-extraction.service"; import {OcrBillDTO, OcrBillResponse} from "./bills.dto"; -import {AppError} from "../../utils/error.util"; /** * Thin wrapper around the shared image-extraction service — this endpoint and @@ -13,8 +12,7 @@ import {AppError} from "../../utils/error.util"; */ export const ocrBill = async (req: AuthenticatedRequest, res: Response): Promise => { const data = req.body as OcrBillDTO; - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; const extracted = await ImageExtractionService.extractBillingFromImage(data.image_url, userId); diff --git a/api/functions/src/features/bills/bills.route.ts b/api/functions/src/features/bills/bills.route.ts index 0bd22b8..c7cf7fb 100644 --- a/api/functions/src/features/bills/bills.route.ts +++ b/api/functions/src/features/bills/bills.route.ts @@ -2,12 +2,14 @@ import {Router} from "express"; import {ocrBill} from "./bills.controller"; import {validateRequest} from "../../middlewares/validate-request.middleware"; import {requireRole} from "../../middlewares/require-role.middleware"; +import {ocrRateLimiter} from "../../config/rate-limit.config"; import {OcrBillDTOSchema} from "./bills.dto"; const router = Router(); router.post( "/ocr", + ocrRateLimiter, validateRequest({body: OcrBillDTOSchema}), requireRole("admin", "landlord", "assistant"), ocrBill diff --git a/api/functions/src/features/bills/bills.swagger.ts b/api/functions/src/features/bills/bills.swagger.ts index 288e408..31ee306 100644 --- a/api/functions/src/features/bills/bills.swagger.ts +++ b/api/functions/src/features/bills/bills.swagger.ts @@ -44,9 +44,8 @@ export const schemas = { properties: { image_url: { type: "string", - format: "uri", - description: "URL of the bill image to process", - example: "https://storage.googleapis.com/bucket/bills/bill.jpg", + description: "Base64 data URL of the bill image to process (data:image/*;base64,...)", + example: "data:image/jpeg;base64,/9j/4AAQSkZJRg...", }, }, }, diff --git a/api/functions/src/features/chatbot/chatbot.controller.ts b/api/functions/src/features/chatbot/chatbot.controller.ts index d6021ea..fc85961 100644 --- a/api/functions/src/features/chatbot/chatbot.controller.ts +++ b/api/functions/src/features/chatbot/chatbot.controller.ts @@ -2,11 +2,9 @@ import type {AuthenticatedRequest} from "../../utils/auth.util"; import {Response} from "express"; import {chatbotService} from "./chatbot.service"; import {ChatRequestDTO} from "./chatbot.dto"; -import {AppError} from "../../utils/error.util"; export const postChatMessage = async (req: AuthenticatedRequest, res: Response): Promise => { - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; const {message, history} = req.body as ChatRequestDTO; const reply = await chatbotService.chat(userId, message, history); res.status(200).json({reply}); diff --git a/api/functions/src/features/chatbot/chatbot.tools.ts b/api/functions/src/features/chatbot/chatbot.tools.ts index 64d2efa..140a7fd 100644 --- a/api/functions/src/features/chatbot/chatbot.tools.ts +++ b/api/functions/src/features/chatbot/chatbot.tools.ts @@ -439,25 +439,25 @@ export async function getBillingReports(args: GetBillingReportsArgs) { const userId = "chatbot"; // unused by reportsService — every report is scoped by query filters, not caller switch (args.reportType) { - case "summary": - return reportsService.getSummary(userId, query); - case "consumption": { - const report = await reportsService.getConsumption(userId, query); - return { - by_month: report.by_month, - by_property: report.by_property.map((p) => ({ - propertyName: p.room_name, - electricity: p.electricity, - water: p.water, - })), - }; - } - case "billing_trends": - return reportsService.getBillingTrends(userId, query); - case "collection_status": - return reportsService.getCollectionStatus(userId, query); - default: - throw new AppError(400, `Unknown reportType: ${args.reportType}`); + case "summary": + return reportsService.getSummary(userId, query); + case "consumption": { + const report = await reportsService.getConsumption(userId, query); + return { + by_month: report.by_month, + by_property: report.by_property.map((p) => ({ + propertyName: p.room_name, + electricity: p.electricity, + water: p.water, + })), + }; + } + case "billing_trends": + return reportsService.getBillingTrends(userId, query); + case "collection_status": + return reportsService.getCollectionStatus(userId, query); + default: + throw new AppError(400, `Unknown reportType: ${args.reportType}`); } } diff --git a/api/functions/src/features/image-extraction/image-extraction.controller.ts b/api/functions/src/features/image-extraction/image-extraction.controller.ts index a9e6f5d..7731157 100644 --- a/api/functions/src/features/image-extraction/image-extraction.controller.ts +++ b/api/functions/src/features/image-extraction/image-extraction.controller.ts @@ -3,7 +3,6 @@ import type {AuthenticatedRequest} from "../../utils/auth.util"; import {ImageExtractionService} from "./image-extraction.service"; import {ImageExtractionValidator} from "./image-extraction.validator"; import type {ExtractReadingRequest, ExtractBillingRequest} from "./image-extraction.dto"; -import {AppError} from "../../utils/error.util"; const validator = new ImageExtractionValidator(); @@ -11,8 +10,7 @@ export async function extractReadingFromImage( req: AuthenticatedRequest, res: Response ) { - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; const body = req.body as ExtractReadingRequest; validator.validateExtractReading(body); const result = await ImageExtractionService.extractReadingFromImage(body.image_url, userId); @@ -23,8 +21,7 @@ export async function extractBillingFromImage( req: AuthenticatedRequest, res: Response ) { - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; const body = req.body as ExtractBillingRequest; validator.validateExtractBilling(body); const result = await ImageExtractionService.extractBillingFromImage(body.image_url, userId); diff --git a/api/functions/src/features/image-extraction/image-extraction.route.ts b/api/functions/src/features/image-extraction/image-extraction.route.ts index 6995890..d2f2f61 100644 --- a/api/functions/src/features/image-extraction/image-extraction.route.ts +++ b/api/functions/src/features/image-extraction/image-extraction.route.ts @@ -2,17 +2,20 @@ import {Router} from "express"; import {validateRequest} from "../../middlewares/validate-request.middleware"; import {ExtractReadingFromImageSchema, ExtractBillingFromImageSchema} from "./image-extraction.dto"; import * as imageExtractionController from "./image-extraction.controller"; +import {ocrRateLimiter} from "../../config/rate-limit.config"; export const imageExtractionRouter = Router(); imageExtractionRouter.post( "/readings", + ocrRateLimiter, validateRequest({body: ExtractReadingFromImageSchema}), imageExtractionController.extractReadingFromImage ); imageExtractionRouter.post( "/billings", + ocrRateLimiter, validateRequest({body: ExtractBillingFromImageSchema}), imageExtractionController.extractBillingFromImage ); diff --git a/api/functions/src/features/image-extraction/image-extraction.swagger.ts b/api/functions/src/features/image-extraction/image-extraction.swagger.ts index 3bf41dc..89f5f2d 100644 --- a/api/functions/src/features/image-extraction/image-extraction.swagger.ts +++ b/api/functions/src/features/image-extraction/image-extraction.swagger.ts @@ -13,8 +13,7 @@ export const paths = { properties: { image_url: { type: "string", - format: "uri", - description: "URL of the meter photo (data URL or HTTPS)", + description: "Base64 data URL of the meter photo (data:image/*;base64,...)", }, }, required: ["image_url"], @@ -61,8 +60,7 @@ export const paths = { properties: { image_url: { type: "string", - format: "uri", - description: "URL of the utility bill photo (data URL or HTTPS)", + description: "Base64 data URL of the utility bill photo (data:image/*;base64,...)", }, }, required: ["image_url"], diff --git a/api/functions/src/features/llm-config/llm-config.controller.ts b/api/functions/src/features/llm-config/llm-config.controller.ts index bdf1b87..b2d338e 100644 --- a/api/functions/src/features/llm-config/llm-config.controller.ts +++ b/api/functions/src/features/llm-config/llm-config.controller.ts @@ -2,11 +2,9 @@ import type {AuthenticatedRequest} from "../../utils/auth.util"; import {Response} from "express"; import {llmConfigService} from "./llm-config.service"; import {UpsertLlmConfigDTO, UpsertVisionLlmConfigDTO} from "./llm-config.dto"; -import {AppError} from "../../utils/error.util"; export const getLlmConfig = async (req: AuthenticatedRequest, res: Response): Promise => { - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; const result = await llmConfigService.get(userId); if (!result) { res.status(200).json({ @@ -23,16 +21,14 @@ export const getLlmConfig = async (req: AuthenticatedRequest, res: Response): Pr }; export const upsertLlmConfig = async (req: AuthenticatedRequest, res: Response): Promise => { - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; const data = req.body as UpsertLlmConfigDTO; const result = await llmConfigService.upsert(userId, data); res.status(200).json(result); }; export const upsertVisionLlmConfig = async (req: AuthenticatedRequest, res: Response): Promise => { - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; const data = req.body as UpsertVisionLlmConfigDTO; const result = await llmConfigService.upsertVision(userId, data); res.status(200).json(result); diff --git a/api/functions/src/features/meter-group/meter-group.controller.ts b/api/functions/src/features/meter-group/meter-group.controller.ts index 13b9083..5ba83b9 100644 --- a/api/functions/src/features/meter-group/meter-group.controller.ts +++ b/api/functions/src/features/meter-group/meter-group.controller.ts @@ -8,15 +8,14 @@ import { UpdateMeterGroupDTO, } from "./meter-group.dto"; import {AppError} from "../../utils/error.util"; -import {cacheDelPattern} from "../../utils/cache.util"; +import {makeClearCacheHandler} from "../../utils/clear-cache-handler.util"; export const createMeterGroup = async ( req: AuthenticatedRequest, res: Response ): Promise => { const data = req.body as CreateMeterGroupDTO; - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; const result = await meterGroupService.create(userId, data); res.status(201).json(result); }; @@ -26,8 +25,7 @@ export const createBatchMeterGroups = async ( res: Response ): Promise => { const data = req.body as CreateMeterGroupDTO[]; - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; const result = await meterGroupService.createBatch(userId, data); res.status(201).json(result); }; @@ -37,8 +35,7 @@ export const getMeterGroupById = async ( res: Response ): Promise => { const {id} = req.params as unknown as MeterGroupByIdParamsDTO; - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; const meterGroup = await meterGroupService.getById(userId, id); if (!meterGroup) { @@ -53,8 +50,7 @@ export const getMeterGroups = async ( res: Response ): Promise => { const query = req.query as unknown as GetMeterGroupsQueryDTO; - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; const result = await meterGroupService.search(userId, { meterName: query.meterName, @@ -75,8 +71,7 @@ export const updateMeterGroup = async ( ): Promise => { const {id} = req.params as unknown as MeterGroupByIdParamsDTO; const data = req.body as Partial; - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; const result = await meterGroupService.update(userId, id, data); res.status(200).json(result); }; @@ -86,8 +81,7 @@ export const updateBatchMeterGroups = async ( res: Response ): Promise => { const updates = req.body as { id: string; data: Partial }[]; - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; const result = await meterGroupService.updateBatch(userId, updates); res.status(200).json(result); }; @@ -97,8 +91,7 @@ export const deleteMeterGroup = async ( res: Response ): Promise => { const {id} = req.params as unknown as MeterGroupByIdParamsDTO; - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; await meterGroupService.delete(userId, id); res.status(204).send(); }; @@ -108,8 +101,7 @@ export const softDeleteMeterGroup = async ( res: Response ): Promise => { const {id} = req.params as unknown as MeterGroupByIdParamsDTO; - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; const result = await meterGroupService.softDelete(userId, id); res.status(200).json(result); }; @@ -119,8 +111,7 @@ export const restoreMeterGroup = async ( res: Response ): Promise => { const {id} = req.params as unknown as MeterGroupByIdParamsDTO; - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; const result = await meterGroupService.restore(userId, id); res.status(200).json(result); }; @@ -130,8 +121,6 @@ export const purgeMeterGroup = async ( res: Response ): Promise => { const {id} = req.params as unknown as MeterGroupByIdParamsDTO; - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); await meterGroupService.purge(id); res.status(204).send(); }; @@ -141,16 +130,9 @@ export const recordMeterGroupReset = async ( res: Response ): Promise => { const {id} = req.params as unknown as MeterGroupByIdParamsDTO; - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; const result = await meterGroupService.recordReset(userId, id); res.status(200).json(result); }; -export const clearCache = async ( - _req: AuthenticatedRequest, - res: Response -): Promise => { - const deletedCount = await cacheDelPattern("utilitool:meter-groups:*"); - res.status(200).json({message: `Cleared ${deletedCount} cache entries for meter groups`}); -}; +export const clearCache = makeClearCacheHandler("meter-groups", "meter groups"); diff --git a/api/functions/src/features/meter-group/meter-group.model.ts b/api/functions/src/features/meter-group/meter-group.model.ts index 8851af0..48b201c 100644 --- a/api/functions/src/features/meter-group/meter-group.model.ts +++ b/api/functions/src/features/meter-group/meter-group.model.ts @@ -11,15 +11,18 @@ export interface MeterGroup extends BaseModel { meter_name: string; utility_type: UtilityType; /** - * @deprecated Version tracking has moved to `Property.meter_groups[entry].current_version`. - * Submeter entries already track their own versions independently; main-meter entries are - * pending backfill (see `migrations/backfill-property-meter-versions.ts`). Do not write new - * data here — read paths should resolve from the property entry once that backfill lands. + * Deprecated FOR SUBMETERS ONLY — submeter version tracking moved to + * `Property.meter_groups[entry].current_version`. For **main meters this is still the live, + * required source of truth** for `Reading.meter_version`: `propertyService.recordMeterGroupReset` + * rejects main-meter resets and routes them here, and `resolveVersionsSource`/`resolveMeterVersion` + * read this field for main meters. The planned "migrate main meters too" backfill was never done + * and is not pending — do NOT delete this field. See + * `decisions/20260608_meter-group-version-tracking-moved-to-property.md` (Amendment 2026-07-19). */ current_version: number; /** - * @deprecated Version history has moved to `Property.meter_groups[entry].versions`. - * See `current_version` deprecation note for migration status. + * Deprecated FOR SUBMETERS ONLY — see `current_version`. Still the live source of version history + * for **main meters**; do not delete. */ versions: Record; } diff --git a/api/functions/src/features/meter-group/meter-group.service.ts b/api/functions/src/features/meter-group/meter-group.service.ts index 2694d74..28005ec 100644 --- a/api/functions/src/features/meter-group/meter-group.service.ts +++ b/api/functions/src/features/meter-group/meter-group.service.ts @@ -5,18 +5,21 @@ import {CreateMeterGroupDTO} from "./meter-group.dto"; import {readingRepository} from "../reading/reading.repository"; import {PaginatedResult} from "../../utils/pagination.util"; import {MeterGroupValidator} from "./meter-group.validator"; -import {AppError} from "../../utils/error.util"; +import {AppError, getOrThrow} from "../../utils/error.util"; import {collectionRef} from "../../lib/firestore.lib"; import {COLLECTIONS} from "../../constants/collection.constants"; import {listRemove} from "../../utils/list-cache.util"; import {cacheDel, cacheSet} from "../../utils/cache.util"; import {cascadeDeleteMeterGroup, cascadeRestoreMeterGroup, cascadePurgeMeterGroup} from "../../utils/cascade-delete.util"; import {CachedRepository} from "../../lib/cached-repository.lib"; -import {logger} from "../../utils/logger.util"; const validator = new MeterGroupValidator(); const CACHE_TTL = 30 * 60; // 30 minutes +function repoFor(userId: string): CachedRepository { + return new CachedRepository(meterGroupRepository, userId, "meter-groups", CACHE_TTL); +} + type MeterGroupSearchOptions = { meterName?: string; utilityType?: MeterGroup["utility_type"]; @@ -66,8 +69,7 @@ function applyProjections( export const meterGroupService = { async create(userId: string, data: CreateMeterGroupDTO): Promise { await validator.validateCreate(data); - const cachedRepo = new CachedRepository(meterGroupRepository, userId, "meter-groups", CACHE_TTL); - return cachedRepo.create({ + return repoFor(userId).create({ ...data, current_version: 1, versions: {}, @@ -76,8 +78,7 @@ export const meterGroupService = { async createBatch(userId: string, data: CreateMeterGroupDTO[]): Promise { await validator.validateBatch(data); - const cachedRepo = new CachedRepository(meterGroupRepository, userId, "meter-groups", CACHE_TTL); - return cachedRepo.createBatch( + return repoFor(userId).createBatch( data.map((item) => ({ ...item, current_version: 1, @@ -90,8 +91,7 @@ export const meterGroupService = { userId: string, options: MeterGroupSearchOptions ): Promise> { - const cachedRepo = new CachedRepository(meterGroupRepository, userId, "meter-groups", CACHE_TTL); - const result = await cachedRepo.search({ + const result = await repoFor(userId).search({ limit: options.limit, orderBy: (options.sortBy ?? "created_at") as any, orderDirection: options.sortOrder ?? "desc", @@ -106,22 +106,16 @@ export const meterGroupService = { }, async getById(userId: string, id: string): Promise { - const cachedRepo = new CachedRepository(meterGroupRepository, userId, "meter-groups", CACHE_TTL); - return cachedRepo.getById(id); + return repoFor(userId).getById(id); }, async update(userId: string, id: string, data: Partial): Promise { - const meterGroup = await meterGroupRepository.getById(id); - if (!meterGroup) { - throw new AppError(404, "Meter group not found"); - } - const cachedRepo = new CachedRepository(meterGroupRepository, userId, "meter-groups", CACHE_TTL); - return cachedRepo.update(id, data); + await getOrThrow(meterGroupRepository.getById.bind(meterGroupRepository), id, "Meter group"); + return repoFor(userId).update(id, data); }, async updateBatch(userId: string, updates: {id: string, data: Partial}[]): Promise { - const cachedRepo = new CachedRepository(meterGroupRepository, userId, "meter-groups", CACHE_TTL); - return cachedRepo.updateBatch(updates); + return repoFor(userId).updateBatch(updates); }, async delete(userId: string, id: string): Promise { @@ -136,15 +130,11 @@ export const meterGroupService = { throw new AppError(409, "Cannot delete meter group: it has active readings. Archive readings first."); } - const cachedRepo = new CachedRepository(meterGroupRepository, userId, "meter-groups", CACHE_TTL); - await cachedRepo.delete(id); + await repoFor(userId).delete(id); }, async softDelete(userId: string, id: string): Promise { - const meterGroup = await meterGroupRepository.getById(id); - if (!meterGroup) { - throw new AppError(404, "Meter group not found"); - } + await getOrThrow(meterGroupRepository.getById.bind(meterGroupRepository), id, "Meter group"); await cascadeDeleteMeterGroup(id); const deleted = await meterGroupRepository.getById(id); @@ -154,10 +144,7 @@ export const meterGroupService = { }, async restore(userId: string, id: string): Promise { - const meterGroup = await meterGroupRepository.getById(id); - if (!meterGroup) { - throw new AppError(404, "Meter group not found"); - } + await getOrThrow(meterGroupRepository.getById.bind(meterGroupRepository), id, "Meter group"); // cascadeRestoreMeterGroup already refreshes id caches and invalidates list // caches (wholesale, for all users) for meter-groups/readings/billings — // appending here would race with that invalidation and risk duplicates. @@ -168,21 +155,14 @@ export const meterGroupService = { }, /** - * @deprecated Meter-group-level version tracking is being replaced by per-property version - * tracking on `Property.meter_groups[entry]` (submeters already migrated; main meters pending - * backfill). This endpoint remains functional for main meters in the interim, but new - * integrations should not depend on `MeterGroup.current_version`/`versions`. + * Submeter version tracking has moved to `Property.meter_groups[entry]` + * (see `propertyService.recordMeterGroupReset`) — main-meter resets still + * route through this endpoint and remain the active source of truth for + * `Reading.meter_version` (see `reading.service.ts`), so it is not + * deprecated despite the field-level `@deprecated` notes on the model. */ async recordReset(userId: string, id: string): Promise { - logger.warn( - {userId, meterGroupId: id}, - "[Deprecated] meterGroupService.recordReset: meter-group-level version tracking is being replaced by per-property tracking on Property.meter_groups[entry]" - ); - - const meterGroup = await meterGroupRepository.getById(id); - if (!meterGroup) { - throw new AppError(404, "Meter group not found"); - } + const meterGroup = await getOrThrow(meterGroupRepository.getById.bind(meterGroupRepository), id, "Meter group"); const latestReadingsResult = await readingRepository.search({ limit: 1, @@ -211,8 +191,7 @@ export const meterGroupService = { current_version: currentVersion + 1, versions: updatedVersions, }; - const cachedRepo = new CachedRepository(meterGroupRepository, userId, "meter-groups", CACHE_TTL); - return cachedRepo.update(id, updatePayload); + return repoFor(userId).update(id, updatePayload); }, /** diff --git a/api/functions/src/features/photo-settings/photo-settings.controller.ts b/api/functions/src/features/photo-settings/photo-settings.controller.ts deleted file mode 100644 index 0f744e6..0000000 --- a/api/functions/src/features/photo-settings/photo-settings.controller.ts +++ /dev/null @@ -1,20 +0,0 @@ -import type {AuthenticatedRequest} from "../../utils/auth.util"; -import {Response} from "express"; -import {photoSettingsService} from "./photo-settings.service"; -import {UpsertPhotoSettingsDTO} from "./photo-settings.dto"; -import {AppError} from "../../utils/error.util"; - -export const getPhotoSettings = async (req: AuthenticatedRequest, res: Response): Promise => { - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); - const result = await photoSettingsService.get(userId); - res.status(200).json(result); -}; - -export const upsertPhotoSettings = async (req: AuthenticatedRequest, res: Response): Promise => { - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); - const data = req.body as UpsertPhotoSettingsDTO; - const result = await photoSettingsService.upsert(userId, data); - res.status(200).json(result); -}; diff --git a/api/functions/src/features/photo-settings/photo-settings.dto.ts b/api/functions/src/features/photo-settings/photo-settings.dto.ts deleted file mode 100644 index e2957a8..0000000 --- a/api/functions/src/features/photo-settings/photo-settings.dto.ts +++ /dev/null @@ -1,14 +0,0 @@ -import {z} from "zod"; - -export const UpsertPhotoSettingsDTOSchema = z.object({ - savePhotos: z.boolean(), -}); -export type UpsertPhotoSettingsDTO = z.infer; - -export interface PhotoSettingsResponseDTO { - // Whether to persist meter-reading photos (image_url) when creating readings. - // Defaults to false — OCR suggest still works either way, but the photo - // itself is discarded before submission unless explicitly enabled. Bill / - // billing-cycle photos are never persisted regardless of this setting. - savePhotos: boolean; -} diff --git a/api/functions/src/features/photo-settings/photo-settings.model.ts b/api/functions/src/features/photo-settings/photo-settings.model.ts deleted file mode 100644 index 3baa4d7..0000000 --- a/api/functions/src/features/photo-settings/photo-settings.model.ts +++ /dev/null @@ -1,5 +0,0 @@ -import {BaseModel} from "../../utils/model.util"; - -export interface PhotoSettings extends BaseModel { - save_photos: boolean; -} diff --git a/api/functions/src/features/photo-settings/photo-settings.repository.ts b/api/functions/src/features/photo-settings/photo-settings.repository.ts deleted file mode 100644 index 2eeb269..0000000 --- a/api/functions/src/features/photo-settings/photo-settings.repository.ts +++ /dev/null @@ -1,19 +0,0 @@ -import {PhotoSettings} from "./photo-settings.model"; -import {COLLECTIONS} from "../../constants/collection.constants"; -import {getDocument, setDocument, updateDocument} from "../../lib/firestore.lib"; -import {WithoutBaseModel} from "../../utils/model.util"; - -// One settings document per user, keyed by userId (same pattern as llm-config). -export const photoSettingsRepository = { - async getByUserId(userId: string): Promise { - return getDocument(COLLECTIONS.PHOTO_SETTINGS, userId); - }, - - async create(userId: string, data: WithoutBaseModel): Promise { - return setDocument(COLLECTIONS.PHOTO_SETTINGS, userId, data); - }, - - async update(userId: string, data: Partial>): Promise { - return updateDocument(COLLECTIONS.PHOTO_SETTINGS, userId, data); - }, -}; diff --git a/api/functions/src/features/photo-settings/photo-settings.route.ts b/api/functions/src/features/photo-settings/photo-settings.route.ts deleted file mode 100644 index 0fdc9ad..0000000 --- a/api/functions/src/features/photo-settings/photo-settings.route.ts +++ /dev/null @@ -1,16 +0,0 @@ -import {Router} from "express"; -import {getPhotoSettings, upsertPhotoSettings} from "./photo-settings.controller"; -import {UpsertPhotoSettingsDTOSchema} from "./photo-settings.dto"; -import {validateRequest} from "../../middlewares/validate-request.middleware"; - -const router = Router(); - -router.get("/", getPhotoSettings); - -router.patch( - "/", - validateRequest({body: UpsertPhotoSettingsDTOSchema}), - upsertPhotoSettings -); - -export const photoSettingsRouter = router; diff --git a/api/functions/src/features/photo-settings/photo-settings.service.ts b/api/functions/src/features/photo-settings/photo-settings.service.ts deleted file mode 100644 index 102bbe3..0000000 --- a/api/functions/src/features/photo-settings/photo-settings.service.ts +++ /dev/null @@ -1,29 +0,0 @@ -import {photoSettingsRepository} from "./photo-settings.repository"; -import {UpsertPhotoSettingsDTO, PhotoSettingsResponseDTO} from "./photo-settings.dto"; -import {PhotoSettings} from "./photo-settings.model"; - -const DEFAULT_SAVE_PHOTOS = false; - -function toResponseDTO(config: PhotoSettings): PhotoSettingsResponseDTO { - return {savePhotos: config.save_photos}; -} - -export const photoSettingsService = { - // Unlike llm-config, this always resolves — no config yet just means the - // (disabled-by-default) default applies, not a 404. - async get(userId: string): Promise { - const config = await photoSettingsRepository.getByUserId(userId); - return config ? toResponseDTO(config) : {savePhotos: DEFAULT_SAVE_PHOTOS}; - }, - - async upsert(userId: string, data: UpsertPhotoSettingsDTO): Promise { - const existing = await photoSettingsRepository.getByUserId(userId); - const payload = {save_photos: data.savePhotos}; - - const result = existing ? - await photoSettingsRepository.update(userId, payload) : - await photoSettingsRepository.create(userId, payload as Omit); - - return toResponseDTO(result); - }, -}; diff --git a/api/functions/src/features/photo-settings/photo-settings.swagger.ts b/api/functions/src/features/photo-settings/photo-settings.swagger.ts deleted file mode 100644 index cb23874..0000000 --- a/api/functions/src/features/photo-settings/photo-settings.swagger.ts +++ /dev/null @@ -1,54 +0,0 @@ -export const photoSettingsPaths = { - "/photo-settings": { - get: { - tags: ["Photo Settings"], - summary: "Get the current user's photo-saving preference", - description: "Whether meter-reading photos should be persisted (image_url) when creating readings. Defaults to false (disabled) if never configured — OCR suggest still works, the photo is just discarded before submission. Bill / billing-cycle photos are never persisted regardless of this setting.", - security: [{BearerAuth: []}], - responses: { - "200": { - description: "Photo-saving preference", - content: { - "application/json": { - schema: {$ref: "#/components/schemas/PhotoSettingsResponse"}, - }, - }, - }, - "401": { - description: "Unauthorized", - content: {"application/json": {schema: {$ref: "#/components/schemas/ErrorResponse"}}}, - }, - }, - }, - patch: { - tags: ["Photo Settings"], - summary: "Set the current user's photo-saving preference", - security: [{BearerAuth: []}], - requestBody: { - content: { - "application/json": { - schema: {$ref: "#/components/schemas/UpsertPhotoSettingsRequest"}, - }, - }, - }, - responses: { - "200": { - description: "Updated photo-saving preference", - content: { - "application/json": { - schema: {$ref: "#/components/schemas/PhotoSettingsResponse"}, - }, - }, - }, - "400": { - description: "Validation error", - content: {"application/json": {schema: {$ref: "#/components/schemas/ValidationErrorResponse"}}}, - }, - "401": { - description: "Unauthorized", - content: {"application/json": {schema: {$ref: "#/components/schemas/ErrorResponse"}}}, - }, - }, - }, - }, -}; diff --git a/api/functions/src/features/property/property.controller.ts b/api/functions/src/features/property/property.controller.ts index a6486a1..2d58606 100644 --- a/api/functions/src/features/property/property.controller.ts +++ b/api/functions/src/features/property/property.controller.ts @@ -11,15 +11,14 @@ import { UpdatePropertyDTO, } from "./property.dto"; import {AppError} from "../../utils/error.util"; -import {cacheDelPattern} from "../../utils/cache.util"; +import {makeClearCacheHandler} from "../../utils/clear-cache-handler.util"; export const createProperty = async ( req: AuthenticatedRequest, res: Response ): Promise => { const data = req.body as CreatePropertyDTO; - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; const result = await propertyService.create(userId, data); res.status(201).json(result); }; @@ -29,8 +28,7 @@ export const createBatchProperties = async ( res: Response ): Promise => { const data = req.body as CreatePropertyBatchDTO; - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; const result = await propertyService.createBatch(userId, data); res.status(201).json(result); }; @@ -40,8 +38,7 @@ export const getPropertyById = async ( res: Response ): Promise => { const {id} = req.params as unknown as PropertyByIdParamsDTO; - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; const property = await propertyService.getById(userId, id); if (!property) { @@ -56,8 +53,7 @@ export const getProperties = async ( res: Response ): Promise => { const query = req.query as unknown as GetPropertiesQueryDTO; - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; const result = await propertyService.search(userId, { roomName: query.roomName, @@ -78,8 +74,7 @@ export const updateProperty = async ( ): Promise => { const {id} = req.params as unknown as PropertyByIdParamsDTO; const data = req.body as UpdatePropertyDTO; - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; const result = await propertyService.update(userId, id, data); res.status(200).json(result); }; @@ -89,8 +84,7 @@ export const updateBatchProperties = async ( res: Response ): Promise => { const updates = req.body as UpdatePropertyBatchDTO; - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; const result = await propertyService.updateBatch(userId, updates); res.status(200).json(result); }; @@ -100,8 +94,7 @@ export const deleteProperty = async ( res: Response ): Promise => { const {id} = req.params as unknown as PropertyByIdParamsDTO; - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; await propertyService.delete(userId, id); res.status(204).send(); }; @@ -111,8 +104,7 @@ export const softDeleteProperty = async ( res: Response ): Promise => { const {id} = req.params as unknown as PropertyByIdParamsDTO; - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; const result = await propertyService.softDelete(userId, id); res.status(200).json(result); }; @@ -122,8 +114,7 @@ export const restoreProperty = async ( res: Response ): Promise => { const {id} = req.params as unknown as PropertyByIdParamsDTO; - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; const result = await propertyService.restore(userId, id); res.status(200).json(result); }; @@ -133,8 +124,6 @@ export const purgeProperty = async ( res: Response ): Promise => { const {id} = req.params as unknown as PropertyByIdParamsDTO; - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); await propertyService.purge(id); res.status(204).send(); }; @@ -144,16 +133,9 @@ export const recordPropertyMeterGroupReset = async ( res: Response ): Promise => { const {id, meterGroupId} = req.params as unknown as PropertyMeterGroupResetParamsDTO; - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; const result = await propertyService.recordMeterGroupReset(userId, id, meterGroupId); res.status(200).json(result); }; -export const clearCache = async ( - _req: AuthenticatedRequest, - res: Response -): Promise => { - const deletedCount = await cacheDelPattern("utilitool:properties:*"); - res.status(200).json({message: `Cleared ${deletedCount} cache entries for properties`}); -}; +export const clearCache = makeClearCacheHandler("properties", "properties"); diff --git a/api/functions/src/features/property/property.service.ts b/api/functions/src/features/property/property.service.ts index ff3438c..56c1660 100644 --- a/api/functions/src/features/property/property.service.ts +++ b/api/functions/src/features/property/property.service.ts @@ -1,4 +1,4 @@ -import {AppError} from "../../utils/error.util"; +import {AppError, getOrThrow} from "../../utils/error.util"; import {PaginatedResult} from "../../utils/pagination.util"; import {propertyRepository} from "./property.repository"; import {CreatePropertyDTO, UpdatePropertyDTO} from "./property.dto"; @@ -15,6 +15,10 @@ import {Timestamp} from "firebase-admin/firestore"; const validator = new PropertyValidator(); const CACHE_TTL = 20 * 60; // 20 minutes +function repoFor(userId: string): CachedRepository { + return new CachedRepository(propertyRepository, userId, "properties", CACHE_TTL); +} + type PropertySearchOptions = { roomName?: string; meterGroupId?: string; @@ -35,29 +39,24 @@ function cleanMeterGroups(meterGroups: Record { await validator.validateCreate(data); - const cachedRepo = new CachedRepository(propertyRepository, userId, "properties", CACHE_TTL); - return cachedRepo.create({...data, meter_groups: cleanMeterGroups(data.meter_groups)}); + return repoFor(userId).create({...data, meter_groups: cleanMeterGroups(data.meter_groups)}); }, async createBatch(userId: string, data: CreatePropertyDTO[]): Promise { await validator.validateBatchCreate(data); - const cachedRepo = new CachedRepository(propertyRepository, userId, "properties", CACHE_TTL); - return cachedRepo.createBatch(data.map((d) => ({...d, meter_groups: cleanMeterGroups(d.meter_groups)}))); + return repoFor(userId).createBatch(data.map((d) => ({...d, meter_groups: cleanMeterGroups(d.meter_groups)}))); }, async getById(userId: string, id: string): Promise { - const cachedRepo = new CachedRepository(propertyRepository, userId, "properties", CACHE_TTL); - return cachedRepo.getById(id); + return repoFor(userId).getById(id); }, async search( userId: string, options: PropertySearchOptions ): Promise> { - const cachedRepo = new CachedRepository(propertyRepository, userId, "properties", CACHE_TTL); - // Need custom filtering for meter_group_id since it's in a nested map - const result = await cachedRepo.search({ + const result = await repoFor(userId).search({ limit: options.limit, orderBy: (options.sortBy ?? "created_at") as any, orderDirection: options.sortOrder ?? "desc", @@ -84,43 +83,29 @@ export const propertyService = { }, async update(userId: string, id: string, data: UpdatePropertyDTO): Promise { - const property = await propertyRepository.getById(id); - - if (!property) { - throw new AppError(404, "Property not found"); - } + const property = await getOrThrow(propertyRepository.getById.bind(propertyRepository), id, "Property"); await validator.validateUpdate(property, data); - const cachedRepo = new CachedRepository(propertyRepository, userId, "properties", CACHE_TTL); const cleanData = data.meter_groups ? {...data, meter_groups: cleanMeterGroups(data.meter_groups)} : (data as any); - return cachedRepo.update(id, cleanData); + return repoFor(userId).update(id, cleanData); }, async updateBatch(userId: string, updates: { id: string; data: UpdatePropertyDTO }[]): Promise { await validator.validateBatchUpdate(updates); - const cachedRepo = new CachedRepository(propertyRepository, userId, "properties", CACHE_TTL); const cleanUpdates = updates.map((u) => ({ id: u.id, data: u.data.meter_groups ? {...u.data, meter_groups: cleanMeterGroups(u.data.meter_groups)} : (u.data as any), })); - return cachedRepo.updateBatch(cleanUpdates); + return repoFor(userId).updateBatch(cleanUpdates); }, async delete(userId: string, id: string): Promise { - const property = await propertyRepository.getById(id); - if (!property) { - throw new AppError(404, "Property not found"); - } - - const cachedRepo = new CachedRepository(propertyRepository, userId, "properties", CACHE_TTL); - await cachedRepo.delete(id); + await getOrThrow(propertyRepository.getById.bind(propertyRepository), id, "Property"); + await repoFor(userId).delete(id); }, async softDelete(userId: string, id: string): Promise { - const property = await propertyRepository.getById(id); - if (!property) { - throw new AppError(404, "Property not found"); - } + await getOrThrow(propertyRepository.getById.bind(propertyRepository), id, "Property"); await cascadeDeleteProperty(id); const deleted = await propertyRepository.getById(id); @@ -130,10 +115,7 @@ export const propertyService = { }, async restore(userId: string, id: string): Promise { - const property = await propertyRepository.getById(id); - if (!property) { - throw new AppError(404, "Property not found"); - } + await getOrThrow(propertyRepository.getById.bind(propertyRepository), id, "Property"); // cascadeRestoreProperty already refreshes id caches and invalidates list // caches (wholesale, for all users) for properties/readings/billings — @@ -159,10 +141,7 @@ export const propertyService = { * own MeterGroupEntry version tracking (main meters stay on the meter group). */ async recordMeterGroupReset(userId: string, propertyId: string, meterGroupId: string): Promise { - const property = await propertyRepository.getById(propertyId); - if (!property) { - throw new AppError(404, "Property not found"); - } + const property = await getOrThrow(propertyRepository.getById.bind(propertyRepository), propertyId, "Property"); const entryKey = Object.keys(property.meter_groups).find( (key) => property.meter_groups[key].meter_group_id === meterGroupId @@ -207,8 +186,7 @@ export const propertyService = { versions: updatedVersions, }; - const cachedRepo = new CachedRepository(propertyRepository, userId, "properties", CACHE_TTL); - return cachedRepo.update(propertyId, { + return repoFor(userId).update(propertyId, { meter_groups: { ...property.meter_groups, [entryKey]: updatedEntry, diff --git a/api/functions/src/features/reading/reading.controller.ts b/api/functions/src/features/reading/reading.controller.ts index 464d7db..4fa510b 100644 --- a/api/functions/src/features/reading/reading.controller.ts +++ b/api/functions/src/features/reading/reading.controller.ts @@ -11,15 +11,15 @@ import { } from "./reading.dto"; import {AppError} from "../../utils/error.util"; import {ImageExtractionService} from "../image-extraction/image-extraction.service"; -import {cacheDelPattern} from "../../utils/cache.util"; +import {makeClearCacheHandler} from "../../utils/clear-cache-handler.util"; +import type {BatchGetQueryDTO} from "../../utils/batch-get.dto"; export const createReading = async ( req: AuthenticatedRequest, res: Response ): Promise => { const data = req.body as CreateReadingDTO; - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; const result = await readingService.create(userId, data); res.status(201).json(result); }; @@ -29,8 +29,7 @@ export const createSeedReading = async ( res: Response ): Promise => { const data = req.body as CreateSeedReadingDTO; - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; const result = await readingService.createSeed(userId, data); res.status(201).json(result); }; @@ -40,8 +39,7 @@ export const createBatchReadings = async ( res: Response ): Promise => { const data = req.body as CreateReadingDTO[]; - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; const result = await readingService.createBatch(userId, data); res.status(201).json(result); }; @@ -51,8 +49,7 @@ export const getReadingById = async ( res: Response ): Promise => { const {id} = req.params as unknown as ReadingByIdParamsDTO; - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; const reading = await readingService.getById(userId, id); if (!reading) { @@ -62,13 +59,21 @@ export const getReadingById = async ( res.status(200).json(reading); }; +export const getReadingsByIds = async ( + req: AuthenticatedRequest, + res: Response +): Promise => { + const {ids} = req.query as unknown as BatchGetQueryDTO; + const readings = await readingService.getByIds(ids); + res.status(200).json(readings); +}; + export const getReadings = async ( req: AuthenticatedRequest, res: Response ): Promise => { const query = req.query as unknown as GetReadingsQueryDTO; - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; const result = await readingService.search(userId, { meterGroupId: query.meterGroupId, @@ -90,8 +95,7 @@ export const updateReading = async ( ): Promise => { const {id} = req.params as unknown as ReadingByIdParamsDTO; const data = req.body as Partial; - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; const result = await readingService.update(userId, id, data); res.status(200).json(result); }; @@ -101,8 +105,7 @@ export const updateBatchReadings = async ( res: Response ): Promise => { const updates = req.body as { id: string; data: Partial }[]; - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; const result = await readingService.updateBatch(userId, updates); res.status(200).json(result); }; @@ -112,8 +115,7 @@ export const deleteReading = async ( res: Response ): Promise => { const {id} = req.params as unknown as ReadingByIdParamsDTO; - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; await readingService.delete(userId, id); res.status(204).send(); }; @@ -123,8 +125,7 @@ export const softDeleteReading = async ( res: Response ): Promise => { const {id} = req.params as unknown as ReadingByIdParamsDTO; - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; const result = await readingService.softDelete(userId, id); res.status(200).json(result); }; @@ -134,8 +135,7 @@ export const restoreReading = async ( res: Response ): Promise => { const {id} = req.params as unknown as ReadingByIdParamsDTO; - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; const result = await readingService.restore(userId, id); res.status(200).json(result); }; @@ -145,8 +145,6 @@ export const purgeReading = async ( res: Response ): Promise => { const {id} = req.params as unknown as ReadingByIdParamsDTO; - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); await readingService.purge(id); res.status(204).send(); }; @@ -156,16 +154,9 @@ export const ocrReading = async ( res: Response ): Promise => { const data = req.body as OcrReadingDTO; - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; const extracted = await ImageExtractionService.extractReadingFromImage(data.image_url, userId); res.status(200).json({suggested_reading_amount: extracted.reading_amount}); }; -export const clearCache = async ( - _req: AuthenticatedRequest, - res: Response -): Promise => { - const deletedCount = await cacheDelPattern("utilitool:readings:*"); - res.status(200).json({message: `Cleared ${deletedCount} cache entries for readings`}); -}; +export const clearCache = makeClearCacheHandler("readings", "readings"); diff --git a/api/functions/src/features/reading/reading.dto.ts b/api/functions/src/features/reading/reading.dto.ts index f58d595..d55c0d7 100644 --- a/api/functions/src/features/reading/reading.dto.ts +++ b/api/functions/src/features/reading/reading.dto.ts @@ -1,5 +1,6 @@ import {z} from "zod"; import {parseTimestamp} from "../../utils/firestore.util"; +import {ImageUrlSchema} from "../../utils/image-url.util"; // Create DTOS export const CreateReadingDTOSchema = z.object({ @@ -16,7 +17,6 @@ export const CreateReadingDTOSchema = z.object({ {message: "reading_date must be a valid date"} ) .transform((val) => parseTimestamp(val)), - image_url: z.url().optional(), }); export type CreateReadingDTO = z.infer; @@ -25,7 +25,7 @@ export type CreateSeedReadingDTO = z.infer; // OCR DTOS export const OcrReadingDTOSchema = z.object({ - image_url: z.url(), + image_url: ImageUrlSchema, }); export type OcrReadingDTO = z.infer; diff --git a/api/functions/src/features/reading/reading.model.ts b/api/functions/src/features/reading/reading.model.ts index b211574..d6e07c8 100644 --- a/api/functions/src/features/reading/reading.model.ts +++ b/api/functions/src/features/reading/reading.model.ts @@ -6,6 +6,5 @@ export interface Reading extends BaseModel { property_id: string; reading_amount: number; reading_date: Timestamp; - image_url?: string; meter_version: number; } diff --git a/api/functions/src/features/reading/reading.route.ts b/api/functions/src/features/reading/reading.route.ts index 748cfd2..a039c28 100644 --- a/api/functions/src/features/reading/reading.route.ts +++ b/api/functions/src/features/reading/reading.route.ts @@ -3,6 +3,7 @@ import { createReading, createSeedReading, getReadingById, + getReadingsByIds, getReadings, updateReading, softDeleteReading, @@ -23,8 +24,10 @@ import { UpdateReadingDTOSchema, OcrReadingDTOSchema, } from "./reading.dto"; +import {BatchGetQueryDTOSchema} from "../../utils/batch-get.dto"; import {validateRequest} from "../../middlewares/validate-request.middleware"; import {requireRole} from "../../middlewares/require-role.middleware"; +import {ocrRateLimiter} from "../../config/rate-limit.config"; const router = Router(); @@ -36,6 +39,7 @@ router.post( router.post( "/ocr", + ocrRateLimiter, validateRequest({body: OcrReadingDTOSchema}), requireRole("admin", "landlord", "assistant"), ocrReading @@ -69,6 +73,12 @@ router.post( createReading ); +router.get( + "/batch-get", + validateRequest({query: BatchGetQueryDTOSchema}), + getReadingsByIds +); + router.get( "/", validateRequest({query: GetReadingsQueryDTOSchema}), diff --git a/api/functions/src/features/reading/reading.service.ts b/api/functions/src/features/reading/reading.service.ts index 98d099a..4603825 100644 --- a/api/functions/src/features/reading/reading.service.ts +++ b/api/functions/src/features/reading/reading.service.ts @@ -21,6 +21,10 @@ import {BatchCreateResult} from "../../utils/batch-result.util"; const validator = new ReadingValidator(); const CACHE_TTL = 10 * 60; // 10 minutes +function repoFor(userId: string): CachedRepository { + return new CachedRepository(readingRepository, userId, "readings", CACHE_TTL); +} + type ReadingCreatePayload = CreateReadingDTO & { meter_version: number }; @@ -147,7 +151,7 @@ export const readingService = { const meterGroup = await meterGroupRepository.getById(data.meter_group_id); const meter_version = meterGroup!.current_version ?? 1; const payload: ReadingCreatePayload = {...data, meter_version}; - const cachedRepo = new CachedRepository(readingRepository, userId, "readings", CACHE_TTL); + const cachedRepo = repoFor(userId); return cachedRepo.create(payload); }, @@ -187,7 +191,7 @@ export const readingService = { }; } - const cachedRepo = new CachedRepository(readingRepository, userId, "readings", CACHE_TTL); + const cachedRepo = repoFor(userId); return cachedRepo.search({ limit: options.limit, orderBy: (options.sortBy ?? "created_at") as any, @@ -202,24 +206,35 @@ export const readingService = { }, async getById(userId: string, id: string): Promise { - const cachedRepo = new CachedRepository(readingRepository, userId, "readings", CACHE_TTL); + const cachedRepo = repoFor(userId); return cachedRepo.getById(id); }, + /** + * Batch ID lookup for clients that otherwise resolve IDs one at a time + * (e.g. the UI's entity-lookup-cache.ts) — one round-trip instead of N. + * Goes through the raw repository (not the per-user list cache), same as + * every other getByIds caller in this codebase (reports.service.ts, etc.). + */ + async getByIds(ids: string[]): Promise { + const results = await readingRepository.getByIds(ids); + return results.filter((r): r is Reading => r !== null); + }, + async update(userId: string, id: string, data: Partial): Promise { await validator.validateUpdate(data); - const cachedRepo = new CachedRepository(readingRepository, userId, "readings", CACHE_TTL); + const cachedRepo = repoFor(userId); return cachedRepo.update(id, data); }, async updateBatch(userId: string, updates: {id: string, data: Partial}[]): Promise { await validator.validateUpdateBatch(updates); - const cachedRepo = new CachedRepository(readingRepository, userId, "readings", CACHE_TTL); + const cachedRepo = repoFor(userId); return cachedRepo.updateBatch(updates); }, async delete(userId: string, id: string): Promise { - const cachedRepo = new CachedRepository(readingRepository, userId, "readings", CACHE_TTL); + const cachedRepo = repoFor(userId); await cachedRepo.delete(id); }, diff --git a/api/functions/src/features/reading/reading.util.ts b/api/functions/src/features/reading/reading.util.ts index b19ad01..e58827c 100644 --- a/api/functions/src/features/reading/reading.util.ts +++ b/api/functions/src/features/reading/reading.util.ts @@ -340,7 +340,7 @@ export async function createReadingWithAutoBilling( const snap = await readingRef.get(); const reading = snapshotToModel(snap); await cacheSet(`utilitool:readings:id:${reading.id}`, reading, CACHE_TTL); - await listAppend(`utilitool:readings:all:${userId}`, reading); + await listAppend(`utilitool:readings:all:${userId}`, reading, CACHE_TTL); if (billingId) { const billing = await billingRepository.getById(billingId); @@ -385,13 +385,13 @@ export async function createBatchReadingsWithAutoBilling( const snap = await readingRef.get(); const reading = snapshotToModel(snap); await cacheSet(`utilitool:readings:id:${reading.id}`, reading, CACHE_TTL); - await listAppend(`utilitool:readings:all:${userId}`, reading); + await listAppend(`utilitool:readings:all:${userId}`, reading, CACHE_TTL); if (billingId) { const billing = await billingRepository.getById(billingId); if (billing) { await cacheSet(`utilitool:billings:id:${billing.id}`, billing, 10 * 60); - await listAppend(`utilitool:billings:all:${userId}`, billing); + await listAppend(`utilitool:billings:all:${userId}`, billing, 10 * 60); } } diff --git a/api/functions/src/features/reports/reports.controller.ts b/api/functions/src/features/reports/reports.controller.ts index 395be5c..9bf7761 100644 --- a/api/functions/src/features/reports/reports.controller.ts +++ b/api/functions/src/features/reports/reports.controller.ts @@ -1,12 +1,10 @@ import {Response} from "express"; -import {AppError} from "../../utils/error.util"; -import {getSummary, getConsumption, getBillingTrends, getCollectionStatus} from "./reports.service"; +import {getSummary, getConsumption, getBillingTrends, getCollectionStatus, getAllReports} from "./reports.service"; import type {ReportQueryDTO} from "./reports.dto"; import type {AuthenticatedRequest} from "../../utils/auth.util"; export async function getSummaryReport(req: AuthenticatedRequest, res: Response) { - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; const query = req.query as ReportQueryDTO; const summary = await getSummary(userId, query); res.set("Cache-Control", "no-store"); @@ -14,8 +12,7 @@ export async function getSummaryReport(req: AuthenticatedRequest, res: Response) } export async function getConsumptionReport(req: AuthenticatedRequest, res: Response) { - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; const query = req.query as ReportQueryDTO; const consumption = await getConsumption(userId, query); res.set("Cache-Control", "no-store"); @@ -23,8 +20,7 @@ export async function getConsumptionReport(req: AuthenticatedRequest, res: Respo } export async function getBillingTrendsReport(req: AuthenticatedRequest, res: Response) { - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; const query = req.query as ReportQueryDTO; const trends = await getBillingTrends(userId, query); res.set("Cache-Control", "no-store"); @@ -32,10 +28,17 @@ export async function getBillingTrendsReport(req: AuthenticatedRequest, res: Res } export async function getCollectionStatusReport(req: AuthenticatedRequest, res: Response) { - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; const query = req.query as ReportQueryDTO; const status = await getCollectionStatus(userId, query); res.set("Cache-Control", "no-store"); res.json(status); } + +export async function getAllReportsHandler(req: AuthenticatedRequest, res: Response) { + const userId = req.user!.userId; + const query = req.query as ReportQueryDTO; + const reports = await getAllReports(userId, query); + res.set("Cache-Control", "no-store"); + res.json(reports); +} diff --git a/api/functions/src/features/reports/reports.model.ts b/api/functions/src/features/reports/reports.model.ts index 6d45fb8..b75beba 100644 --- a/api/functions/src/features/reports/reports.model.ts +++ b/api/functions/src/features/reports/reports.model.ts @@ -50,6 +50,13 @@ export interface CollectionStatusReport { overdue: CollectionStatusItem; } +export interface CombinedReports { + summary: ReportSummary; + consumption: ConsumptionReport; + billingTrends: BillingTrendsReport; + collectionStatus: CollectionStatusReport; +} + export interface JoinedBilling { billingId: string; amount: number; diff --git a/api/functions/src/features/reports/reports.route.ts b/api/functions/src/features/reports/reports.route.ts index e7893af..0f59045 100644 --- a/api/functions/src/features/reports/reports.route.ts +++ b/api/functions/src/features/reports/reports.route.ts @@ -1,18 +1,46 @@ import {Router} from "express"; import {validateRequest} from "../../middlewares/validate-request.middleware"; +import {requireRole} from "../../middlewares/require-role.middleware"; import {ReportQueryDTOSchema} from "./reports.dto"; import { getSummaryReport, getConsumptionReport, getBillingTrendsReport, getCollectionStatusReport, + getAllReportsHandler, } from "./reports.controller"; const router = Router(); -router.get("/summary", validateRequest({query: ReportQueryDTOSchema}), getSummaryReport); -router.get("/consumption", validateRequest({query: ReportQueryDTOSchema}), getConsumptionReport); -router.get("/billing-trends", validateRequest({query: ReportQueryDTOSchema}), getBillingTrendsReport); -router.get("/collection-status", validateRequest({query: ReportQueryDTOSchema}), getCollectionStatusReport); +router.get( + "/", + requireRole("admin", "landlord"), + validateRequest({query: ReportQueryDTOSchema}), + getAllReportsHandler +); +router.get( + "/summary", + requireRole("admin", "landlord"), + validateRequest({query: ReportQueryDTOSchema}), + getSummaryReport +); +router.get( + "/consumption", + requireRole("admin", "landlord"), + validateRequest({query: ReportQueryDTOSchema}), + getConsumptionReport +); +router.get( + "/billing-trends", + requireRole("admin", "landlord"), + validateRequest({query: ReportQueryDTOSchema}), + getBillingTrendsReport +); +router.get( + "/collection-status", + requireRole("admin", "landlord"), + validateRequest({query: ReportQueryDTOSchema}), + getCollectionStatusReport +); export const reportsRouter = router; diff --git a/api/functions/src/features/reports/reports.service.test.ts b/api/functions/src/features/reports/reports.service.test.ts index a1a62d9..83cafed 100644 --- a/api/functions/src/features/reports/reports.service.test.ts +++ b/api/functions/src/features/reports/reports.service.test.ts @@ -5,7 +5,13 @@ jest.mock('../property/property.repository'); jest.mock('../reading/reading.repository'); import {describe, it, expect, jest, beforeEach} from '@jest/globals'; -import {getSummary} from './reports.service'; +import { + getSummary, + getConsumption, + getBillingTrends, + getCollectionStatus, + getAllReports, +} from './reports.service'; import {billingCycleRepository} from '../billing-cycle/billing-cycle.repository'; import {billingRepository} from '../billing/billing.repository'; import {meterGroupRepository} from '../meter-group/meter-group.repository'; @@ -85,3 +91,78 @@ describe('getSummary - centavo-exact totals', () => { expect(result.total_revenue).toBe(0); }); }); + +describe('getAllReports - parity with the four granular endpoints', () => { + beforeEach(() => { + jest.clearAllMocks(); + + const future = new Date(Date.now() + 7 * 24 * 60 * 60 * 1000); + const past = new Date(Date.now() - 7 * 24 * 60 * 60 * 1000); + + (billingCycleRepository.search as jest.Mock).mockResolvedValue({ + data: [ + { + id: 'cycle1', + billing_rate: 10.125, + billing_start_date: toDate(past), + billing_end_date: toDate(future), + overdue_date: toDate(future), + billing_ids: {b1: 3, b2: 7}, + }, + ], + hasMore: false, + nextCursor: null, + } as never); + + (billingRepository.getByIds as jest.Mock).mockResolvedValue([ + { + id: 'b1', + property_id: 'p1', + previous_reading_id: 'r1-prev', + current_reading_id: 'r1-curr', + payment_status: 'paid', + }, + { + id: 'b2', + property_id: 'p1', + previous_reading_id: 'r2-prev', + current_reading_id: 'r2-curr', + payment_status: 'pending', + }, + ] as never); + + (propertyRepository.getByIds as jest.Mock).mockResolvedValue([ + { + id: 'p1', + room_name: 'Room 1', + meter_groups: {electricity: {meter_group_id: 'mg1', is_main_meter: true}}, + }, + ] as never); + + (meterGroupRepository.getByIds as jest.Mock).mockResolvedValue([ + {id: 'mg1', utility_type: 'electricity', versions: undefined}, + ] as never); + + (readingRepository.getByIds as jest.Mock).mockResolvedValue([ + {id: 'r1-prev', meter_group_id: 'mg1', reading_amount: 100, meter_version: 1}, + {id: 'r1-curr', meter_group_id: 'mg1', reading_amount: 103, meter_version: 1}, + {id: 'r2-prev', meter_group_id: 'mg1', reading_amount: 200, meter_version: 1}, + {id: 'r2-curr', meter_group_id: 'mg1', reading_amount: 207, meter_version: 1}, + ] as never); + }); + + it('returns the same shapes as calling the four granular functions independently', async () => { + const query = {}; + + const [summary, consumption, billingTrends, collectionStatus] = await Promise.all([ + getSummary('user1', query), + getConsumption('user1', query), + getBillingTrends('user1', query), + getCollectionStatus('user1', query), + ]); + + const combined = await getAllReports('user1', query); + + expect(combined).toEqual({summary, consumption, billingTrends, collectionStatus}); + }); +}); diff --git a/api/functions/src/features/reports/reports.service.ts b/api/functions/src/features/reports/reports.service.ts index 042746b..7cf2d91 100644 --- a/api/functions/src/features/reports/reports.service.ts +++ b/api/functions/src/features/reports/reports.service.ts @@ -5,6 +5,8 @@ import {propertyRepository} from "../property/property.repository"; import {readingRepository} from "../reading/reading.repository"; import {calculateTrueReading, resolveVersionsSource} from "../reading/reading.util"; import {billAmount, sumMoney} from "../../utils/money.util"; +import {cacheGet, cacheSet} from "../../utils/cache.util"; +import {fetchAllPages} from "../../utils/list-cache.util"; import type {SearchFilter, RangeFilter} from "../../lib/repository.lib"; import type {BillingCycle} from "../billing-cycle/billing-cycle.model"; import type {WithoutBaseModel} from "../../utils/model.util"; @@ -13,6 +15,7 @@ import type { ConsumptionReport, BillingTrendsReport, CollectionStatusReport, + CombinedReports, JoinedBilling, } from "./reports.model"; import type {ReportQueryDTO} from "./reports.dto"; @@ -44,15 +47,17 @@ function buildDateRangeFilters(query: ReportQueryDTO): SearchFilter1000 non-deleted cycles doesn't get reports silently computed from only the + // newest 1000 by created_at desc. const filters = buildDateRangeFilters(query); - const cyclesResult = await billingCycleRepository.search({ + let cycles = await fetchAllPages((cursor) => billingCycleRepository.search({ limit: 1000, orderBy: "created_at", + cursor, filters, - }); - - let cycles = cyclesResult.data; + })); // 2. Additional in-memory filter for endDate range to ensure both bounds are respected // (runs whenever endDate is supplied, not only alongside startDate — an endDate-only @@ -201,9 +206,40 @@ async function buildJoinedData(userId: string, query: ReportQueryDTO): Promise { +// Short-TTL cache over the shared join (cycles → billings → properties → meter groups → +// readings). There is no write-path invalidation hook for this cache — billing/cycle/ +// reading mutations elsewhere don't know it exists — so the TTL is kept short (60s) to +// bound staleness on a financial page rather than relying on invalidation. This is an +// accepted tradeoff (see decisions/20260719_landing2-deviations-and-reports-finding.md +// Part 3): it only helps repeat opens/re-applied filters within the window, not the very +// first parallel burst of requests for a given query. +const JOINED_DATA_CACHE_TTL_SECONDS = 60; + +function joinedDataCacheKey(userId: string, query: ReportQueryDTO): string { + const sortedEntries = Object.entries(query) + .filter(([, value]) => value !== undefined && value !== "") + .sort(([a], [b]) => a.localeCompare(b)); + return `utilitool:reports:joined:${userId}:${JSON.stringify(sortedEntries)}`; +} + +async function buildJoinedDataCached(userId: string, query: ReportQueryDTO): Promise { + const key = joinedDataCacheKey(userId, query); + const cached = await cacheGet(key); + if (cached) { + // Dates round-trip through JSON as ISO strings — revive them before returning. + return cached.map((j) => ({ + ...j, + cycleStartDate: new Date(j.cycleStartDate), + cycleEndDate: new Date(j.cycleEndDate), + })); + } + const joinedData = await buildJoinedData(userId, query); + await cacheSet(key, joinedData, JOINED_DATA_CACHE_TTL_SECONDS); + return joinedData; +} +function summarizeJoined(joinedData: JoinedBilling[]): ReportSummary { const totalRevenue = sumMoney( joinedData.filter((j) => j.paymentStatus === "paid").map((j) => j.amount) ); @@ -227,15 +263,7 @@ export async function getSummary(userId: string, query: ReportQueryDTO): Promise }; } -export async function getConsumption(userId: string, query: ReportQueryDTO): Promise { - // Reuses buildJoinedData so consumption here is computed the same version-aware way - // (calculateTrueReading against Property.meter_groups[entry].versions) as every other - // report — previously this endpoint read the raw, non-version-aware number frozen on - // cycle.billing_ids at cycle-creation time, so a cycle whose meter reset after creation - // (or was backfilled with the old naive number) showed different consumption here than - // on the Summary/Billing-Trends/Collection-Status reports for the same underlying data. - const joinedData = await buildJoinedData(userId, query); - +function aggregateConsumption(joinedData: JoinedBilling[]): ConsumptionReport { const monthMap = new Map>(); for (const j of joinedData) { const month = j.cycleStartDate.toISOString().slice(0, 7); @@ -258,10 +286,10 @@ export async function getConsumption(userId: string, query: ReportQueryDTO): Pro })) .sort((a, b) => a.period.localeCompare(b.period)); - const propertyConsumption = new Map>(); + const propertyConsumption = new Map(); for (const j of joinedData) { if (!propertyConsumption.has(j.propertyId)) { - propertyConsumption.set(j.propertyId, {electricity: 0, water: 0}); + propertyConsumption.set(j.propertyId, {roomName: j.roomName, electricity: 0, water: 0}); } const propData = propertyConsumption.get(j.propertyId)!; if (j.utilityType === "electricity") { @@ -272,28 +300,23 @@ export async function getConsumption(userId: string, query: ReportQueryDTO): Pro } const by_property = Array.from(propertyConsumption.entries()) - .map(([propId, data]) => { - const j = joinedData.find((jb) => jb.propertyId === propId); - return { - property_id: propId, - room_name: j?.roomName || "Unknown", - electricity: Math.round(data.electricity * 100) / 100, - water: Math.round(data.water * 100) / 100, - }; - }) + .map(([propId, data]) => ({ + property_id: propId, + room_name: data.roomName || "Unknown", + electricity: Math.round(data.electricity * 100) / 100, + water: Math.round(data.water * 100) / 100, + })) .sort((a, b) => a.room_name.localeCompare(b.room_name)); return {by_month, by_property}; } -export async function getBillingTrends(userId: string, query: ReportQueryDTO): Promise { - const joinedData = await buildJoinedData(userId, query); - +function aggregateBillingTrends(joinedData: JoinedBilling[]): BillingTrendsReport { // Group by month const monthMap = new Map(); for (const billing of joinedData) { - // Bucketed by cycle start date to match getConsumption's grouping (both use + // Bucketed by cycle start date to match aggregateConsumption's grouping (both use // billing_start_date) — bucketing by end date instead put the same cycle in a // different month on this chart vs. the Consumption chart whenever a cycle spans // a month boundary. @@ -328,12 +351,7 @@ export async function getBillingTrends(userId: string, query: ReportQueryDTO): P return {by_month}; } -export async function getCollectionStatus( - userId: string, - query: ReportQueryDTO -): Promise { - const joinedData = await buildJoinedData(userId, query); - +function aggregateCollectionStatus(joinedData: JoinedBilling[]): CollectionStatusReport { const paid = joinedData.filter((j) => j.paymentStatus === "paid"); const pending = joinedData.filter((j) => j.paymentStatus === "pending" && !j.isOverdue); const overdue = joinedData.filter((j) => j.isOverdue); @@ -353,3 +371,47 @@ export async function getCollectionStatus( }, }; } + +export async function getSummary(userId: string, query: ReportQueryDTO): Promise { + const joinedData = await buildJoinedDataCached(userId, query); + return summarizeJoined(joinedData); +} + +export async function getConsumption(userId: string, query: ReportQueryDTO): Promise { + // Reuses buildJoinedData so consumption here is computed the same version-aware way + // (calculateTrueReading against Property.meter_groups[entry].versions) as every other + // report — previously this endpoint read the raw, non-version-aware number frozen on + // cycle.billing_ids at cycle-creation time, so a cycle whose meter reset after creation + // (or was backfilled with the old naive number) showed different consumption here than + // on the Summary/Billing-Trends/Collection-Status reports for the same underlying data. + const joinedData = await buildJoinedDataCached(userId, query); + return aggregateConsumption(joinedData); +} + +export async function getBillingTrends(userId: string, query: ReportQueryDTO): Promise { + const joinedData = await buildJoinedDataCached(userId, query); + return aggregateBillingTrends(joinedData); +} + +export async function getCollectionStatus( + userId: string, + query: ReportQueryDTO +): Promise { + const joinedData = await buildJoinedDataCached(userId, query); + return aggregateCollectionStatus(joinedData); +} + +/** + * Computes all four report shapes from a single shared join, instead of the four HTTP + * endpoints above each independently re-fetching and re-joining the same underlying data. + */ +export async function getAllReports(userId: string, query: ReportQueryDTO): Promise { + const joinedData = await buildJoinedDataCached(userId, query); + + return { + summary: summarizeJoined(joinedData), + consumption: aggregateConsumption(joinedData), + billingTrends: aggregateBillingTrends(joinedData), + collectionStatus: aggregateCollectionStatus(joinedData), + }; +} diff --git a/api/functions/src/features/reports/reports.swagger.ts b/api/functions/src/features/reports/reports.swagger.ts index 04d5e49..b8dc159 100644 --- a/api/functions/src/features/reports/reports.swagger.ts +++ b/api/functions/src/features/reports/reports.swagger.ts @@ -1,4 +1,97 @@ export const reportsPaths = { + "/reports": { + get: { + tags: ["Reports"], + summary: "Get all reports in one call", + description: + "Retrieve summary, consumption, billing-trends, and collection-status reports from a single shared join, instead of calling the four individual endpoints separately.", + security: [{BearerAuth: []}], + parameters: [ + { + name: "startDate", + in: "query", + description: "Filter cycles by start date (ISO 8601)", + schema: { + type: "string", + format: "date-time", + }, + }, + { + name: "endDate", + in: "query", + description: "Filter cycles by end date (ISO 8601)", + schema: { + type: "string", + format: "date-time", + }, + }, + { + name: "meterGroupId", + in: "query", + description: "Filter by specific meter group", + schema: { + type: "string", + }, + }, + { + name: "propertyId", + in: "query", + description: "Filter by specific property", + schema: { + type: "string", + }, + }, + ], + responses: { + "200": { + description: "Combined reports data", + content: { + "application/json": { + schema: { + type: "object", + properties: { + summary: {$ref: "#/components/schemas/ReportSummary"}, + consumption: {$ref: "#/components/schemas/ConsumptionReport"}, + billingTrends: {$ref: "#/components/schemas/BillingTrendsReport"}, + collectionStatus: {$ref: "#/components/schemas/CollectionStatusReport"}, + }, + }, + }, + }, + }, + "400": { + description: "Validation error", + content: { + "application/json": { + schema: { + $ref: "#/components/schemas/ValidationErrorResponse", + }, + }, + }, + }, + "401": { + description: "Unauthorized", + content: { + "application/json": { + schema: { + $ref: "#/components/schemas/ErrorResponse", + }, + }, + }, + }, + "500": { + description: "Internal server error", + content: { + "application/json": { + schema: { + $ref: "#/components/schemas/ErrorResponse", + }, + }, + }, + }, + }, + }, + }, "/reports/summary": { get: { tags: ["Reports"], diff --git a/api/functions/src/features/tenant/tenant.controller.ts b/api/functions/src/features/tenant/tenant.controller.ts index a5593bc..98feefa 100644 --- a/api/functions/src/features/tenant/tenant.controller.ts +++ b/api/functions/src/features/tenant/tenant.controller.ts @@ -10,15 +10,14 @@ import { UpdateTenantDTO, } from "./tenant.dto"; import {AppError} from "../../utils/error.util"; -import {cacheDelPattern} from "../../utils/cache.util"; +import {makeClearCacheHandler} from "../../utils/clear-cache-handler.util"; export const createTenant = async ( req: AuthenticatedRequest, res: Response ): Promise => { const data = req.body as CreateTenantDTO; - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; const result = await tenantService.create(userId, data); res.status(201).json(result); }; @@ -28,8 +27,7 @@ export const createBatchTenants = async ( res: Response ): Promise => { const data = req.body as CreateTenantBatchDTO; - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; const result = await tenantService.createBatch(userId, data); res.status(201).json(result); }; @@ -39,8 +37,7 @@ export const getTenantById = async ( res: Response ): Promise => { const {id} = req.params as unknown as TenantByIdParamsDTO; - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; const tenant = await tenantService.getById(userId, id); if (!tenant) { @@ -55,8 +52,7 @@ export const getTenants = async ( res: Response ): Promise => { const query = req.query as unknown as GetTenantsQueryDTO; - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; const result = await tenantService.search(userId, { tenantName: query.tenantName, @@ -77,8 +73,7 @@ export const updateTenant = async ( ): Promise => { const {id} = req.params as unknown as TenantByIdParamsDTO; const data = req.body as UpdateTenantDTO; - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; const result = await tenantService.update(userId, id, data); res.status(200).json(result); }; @@ -88,8 +83,7 @@ export const updateBatchTenants = async ( res: Response ): Promise => { const updates = req.body as UpdateTenantBatchDTO; - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; const result = await tenantService.updateBatch(userId, updates); res.status(200).json(result); }; @@ -99,8 +93,7 @@ export const deleteTenant = async ( res: Response ): Promise => { const {id} = req.params as unknown as TenantByIdParamsDTO; - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; await tenantService.delete(userId, id); res.status(204).send(); }; @@ -110,8 +103,7 @@ export const softDeleteTenant = async ( res: Response ): Promise => { const {id} = req.params as unknown as TenantByIdParamsDTO; - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; const result = await tenantService.softDelete(userId, id); res.status(200).json(result); }; @@ -121,8 +113,7 @@ export const restoreTenant = async ( res: Response ): Promise => { const {id} = req.params as unknown as TenantByIdParamsDTO; - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; const result = await tenantService.restore(userId, id); res.status(200).json(result); }; @@ -132,16 +123,9 @@ export const purgeTenant = async ( res: Response ): Promise => { const {id} = req.params as unknown as TenantByIdParamsDTO; - const userId = req.user?.userId; - if (!userId) throw new AppError(401, "User not authenticated"); + const userId = req.user!.userId; await tenantService.purge(userId, id); res.status(204).send(); }; -export const clearCache = async ( - _req: AuthenticatedRequest, - res: Response -): Promise => { - const deletedCount = await cacheDelPattern("utilitool:tenants:*"); - res.status(200).json({message: `Cleared ${deletedCount} cache entries for tenants`}); -}; +export const clearCache = makeClearCacheHandler("tenants", "tenants"); diff --git a/api/functions/src/features/tenant/tenant.service.ts b/api/functions/src/features/tenant/tenant.service.ts index bb50562..8b4dd9e 100644 --- a/api/functions/src/features/tenant/tenant.service.ts +++ b/api/functions/src/features/tenant/tenant.service.ts @@ -1,5 +1,5 @@ import {Timestamp} from "firebase-admin/firestore"; -import {AppError} from "../../utils/error.util"; +import {AppError, getOrThrow} from "../../utils/error.util"; import {PaginatedResult} from "../../utils/pagination.util"; import {tenantRepository} from "./tenant.repository"; import {CreateTenantDTO, UpdateTenantDTO} from "./tenant.dto"; @@ -14,6 +14,10 @@ import {Property} from "../property/property.model"; const validator = new TenantValidator(); const CACHE_TTL = 20 * 60; // 20 minutes +function repoFor(userId: string): CachedRepository { + return new CachedRepository(tenantRepository, userId, "tenants", CACHE_TTL); +} + /** * Atomically re-checks the tenant-count cap and creates the tenant doc inside * one Firestore transaction. This closes the TOCTOU race where two concurrent @@ -97,15 +101,14 @@ export const tenantService = { const property = await validator.validateCreate(data); const tenant = await createTenantWithCapacityCheck(data.property_id, property.tenant_amount, data); - const cachedRepo = new CachedRepository(tenantRepository, userId, "tenants", CACHE_TTL); - await cachedRepo.cacheCreatedItem(tenant); + await repoFor(userId).cacheCreatedItem(tenant); return tenant; }, async createBatch(userId: string, data: CreateTenantDTO[]): Promise { const propertyMap = await validator.validateBatchCreate(data); - const cachedRepo = new CachedRepository(tenantRepository, userId, "tenants", CACHE_TTL); + const cachedRepo = repoFor(userId); // Sequential, one transaction per item (not a single batch write): each item // re-checks the tenant-count cap for its property inside its own transaction, @@ -125,13 +128,11 @@ export const tenantService = { }, async getById(userId: string, id: string): Promise { - const cachedRepo = new CachedRepository(tenantRepository, userId, "tenants", CACHE_TTL); - return cachedRepo.getById(id); + return repoFor(userId).getById(id); }, async search(userId: string, options: TenantSearchOptions): Promise> { - const cachedRepo = new CachedRepository(tenantRepository, userId, "tenants", CACHE_TTL); - return cachedRepo.search({ + return repoFor(userId).search({ limit: options.limit, orderBy: (options.sortBy ?? "created_at") as any, orderDirection: options.sortOrder ?? "desc", @@ -145,13 +146,10 @@ export const tenantService = { }, async update(userId: string, id: string, data: UpdateTenantDTO): Promise { - const tenant = await tenantRepository.getById(id); - if (!tenant) { - throw new AppError(404, "Tenant not found"); - } + const tenant = await getOrThrow(tenantRepository.getById.bind(tenantRepository), id, "Tenant"); const newProperty = await validator.validateUpdate(tenant, data); - const cachedRepo = new CachedRepository(tenantRepository, userId, "tenants", CACHE_TTL); + const cachedRepo = repoFor(userId); if (newProperty) { const updated = await updateTenantWithCapacityCheck(tenant, newProperty, data); @@ -164,7 +162,7 @@ export const tenantService = { async updateBatch(userId: string, updates: { id: string; data: UpdateTenantDTO }[]): Promise { const transferMap = await validator.validateBatchUpdate(updates); - const cachedRepo = new CachedRepository(tenantRepository, userId, "tenants", CACHE_TTL); + const cachedRepo = repoFor(userId); const results: Tenant[] = new Array(updates.length); const nonTransferUpdates: { id: string; data: UpdateTenantDTO }[] = []; @@ -195,33 +193,18 @@ export const tenantService = { }, async delete(userId: string, id: string): Promise { - const tenant = await tenantRepository.getById(id); - if (!tenant) { - throw new AppError(404, "Tenant not found"); - } - - const cachedRepo = new CachedRepository(tenantRepository, userId, "tenants", CACHE_TTL); - await cachedRepo.delete(id); + await getOrThrow(tenantRepository.getById.bind(tenantRepository), id, "Tenant"); + await repoFor(userId).delete(id); }, async softDelete(userId: string, id: string): Promise { - const tenant = await tenantRepository.getById(id); - if (!tenant) { - throw new AppError(404, "Tenant not found"); - } - - const cachedRepo = new CachedRepository(tenantRepository, userId, "tenants", CACHE_TTL); - return cachedRepo.softDelete(id); + await getOrThrow(tenantRepository.getById.bind(tenantRepository), id, "Tenant"); + return repoFor(userId).softDelete(id); }, async restore(userId: string, id: string): Promise { - const tenant = await tenantRepository.getById(id); - if (!tenant) { - throw new AppError(404, "Tenant not found"); - } - - const cachedRepo = new CachedRepository(tenantRepository, userId, "tenants", CACHE_TTL); - return cachedRepo.restore(id); + await getOrThrow(tenantRepository.getById.bind(tenantRepository), id, "Tenant"); + return repoFor(userId).restore(id); }, /** @@ -229,7 +212,6 @@ export const tenantService = { * archive-then-purge lifecycle — throws 409 if the tenant is still active. */ async purge(userId: string, id: string): Promise { - const cachedRepo = new CachedRepository(tenantRepository, userId, "tenants", CACHE_TTL); - await cachedRepo.purge(id); + await repoFor(userId).purge(id); }, }; diff --git a/api/functions/src/features/user/user.controller.ts b/api/functions/src/features/user/user.controller.ts index 895e61b..8975d89 100644 --- a/api/functions/src/features/user/user.controller.ts +++ b/api/functions/src/features/user/user.controller.ts @@ -1,11 +1,42 @@ import {Request, Response} from "express"; +import * as admin from "firebase-admin"; import {CreateUserDTO} from "./user.dto"; import {userRepository} from "../auth/auth.repository"; +import {setDocument} from "../../lib/firestore.lib"; +import {COLLECTIONS} from "../../constants/collection.constants"; import {AppError} from "../../utils/error.util"; import type {User} from "../auth/auth.model"; +// Maps Firebase Admin SDK auth error codes to messages matching what the UI +// previously mapped from the client SDK, since account creation now happens +// entirely server-side (the client no longer calls createUserWithEmailAndPassword, +// which used to hijack the acting admin's own session by signing the client SDK +// into the newly created account). +const ADMIN_AUTH_ERROR_MESSAGES: Record = { + "auth/email-already-exists": {status: 409, message: "An account with this email already exists."}, + "auth/invalid-password": {status: 400, message: "Password is too weak. Use at least 8 characters."}, + "auth/invalid-email": {status: 400, message: "Invalid email address."}, +}; + export async function createUser(req: Request, unknown, CreateUserDTO>, res: Response) { - const {uid, role} = req.body; + const {email, password, displayName, role} = req.body; + + let uid: string; + try { + const userRecord = await admin.auth().createUser({ + email, + password, + ...(displayName ? {displayName} : {}), + }); + uid = userRecord.uid; + } catch (err: unknown) { + const code = (err as {code?: string}).code ?? ""; + const mapped = ADMIN_AUTH_ERROR_MESSAGES[code]; + if (mapped) { + throw new AppError(mapped.status, mapped.message); + } + throw new AppError(400, "Failed to create user account"); + } // Check if user profile already exists (non-deleted) const existing = await userRepository.getById(uid); @@ -23,16 +54,17 @@ export async function createUser(req: Request, unknown, Cr user = await userRepository.update(uid, {role}); } } else { - // Create new user + // Create new user, keyed by the Firebase Auth uid — required so that + // requireRole's getById(req.user.userId) (userId being this same uid, from + // the verified ID token) actually finds this profile and its chosen role + // on the new user's first request, instead of GET /auth/me's auto-create + // silently defaulting them to "landlord" because no doc existed at that ID. // Note: email and display_name are empty strings as per audit plan - user = await userRepository.create({ + user = await setDocument(COLLECTIONS.USERS, uid, { email: "", display_name: "", role, - is_deleted: false, - created_at: new Date(), - updated_at: new Date(), - } as unknown as User); + }); } res.status(201).json(user); diff --git a/api/functions/src/features/user/user.dto.ts b/api/functions/src/features/user/user.dto.ts index 73e2883..3f0a48f 100644 --- a/api/functions/src/features/user/user.dto.ts +++ b/api/functions/src/features/user/user.dto.ts @@ -1,7 +1,9 @@ import {z} from "zod"; export const CreateUserDTOSchema = z.object({ - uid: z.string().min(1, "uid is required"), + email: z.email("Invalid email address"), + password: z.string().min(8, "Password must be at least 8 characters"), + displayName: z.string().trim().optional(), role: z.enum(["admin", "landlord", "assistant"], { message: "role must be admin, landlord, or assistant", }), diff --git a/api/functions/src/features/user/user.route.ts b/api/functions/src/features/user/user.route.ts index 79300a3..15828c5 100644 --- a/api/functions/src/features/user/user.route.ts +++ b/api/functions/src/features/user/user.route.ts @@ -3,11 +3,21 @@ import {validateRequest} from "../../middlewares/validate-request.middleware"; import {requireRole} from "../../middlewares/require-role.middleware"; import {CreateUserDTOSchema} from "./user.dto"; import {createUser} from "./user.controller"; +import {AppError} from "../../utils/error.util"; const router = Router(); +// Account creation is temporarily disabled — the app is currently single-tenant with +// one active user, so onboarding more accounts isn't needed right now. Re-enable by +// removing this guard (paired with the matching UI guard in +// ui/src/routes/(app)/settings/users/+page.svelte). +function accountCreationDisabled(): never { + throw new AppError(403, "Account creation is currently disabled."); +} + router.post( "/", + accountCreationDisabled, validateRequest({body: CreateUserDTOSchema}), requireRole("admin"), createUser diff --git a/api/functions/src/features/user/user.test.ts b/api/functions/src/features/user/user.test.ts index 2db4623..b622fea 100644 --- a/api/functions/src/features/user/user.test.ts +++ b/api/functions/src/features/user/user.test.ts @@ -1,6 +1,7 @@ jest.mock('firebase-admin', () => ({ auth: jest.fn().mockReturnValue({ verifyIdToken: jest.fn(), + createUser: jest.fn(), }), firestore: jest.fn().mockReturnValue({ settings: jest.fn(), @@ -18,6 +19,9 @@ jest.mock('../../config/firebase.config', () => ({ firebaseApp: {}, })); jest.mock('../auth/auth.repository'); +jest.mock('../../lib/firestore.lib', () => ({ + setDocument: jest.fn(), +})); import { describe, it, expect, jest, beforeEach } from '@jest/globals'; import request from 'supertest'; @@ -27,6 +31,7 @@ import 'express-async-errors'; import { userRouter } from './user.route'; import { authMiddleware } from '../../middlewares/auth.middleware'; import { userRepository } from '../auth/auth.repository'; +import { setDocument } from '../../lib/firestore.lib'; import { errorHandler } from '../../middlewares/error-handler.middleware'; import * as admin from 'firebase-admin'; @@ -52,17 +57,45 @@ describe('POST /users', () => { jest.clearAllMocks(); }); - it('should create a user with the specified role', async () => { + it('should return 403 "Account creation is currently disabled" regardless of role or payload validity', async () => { + // Account creation is temporarily disabled (see user.route.ts's accountCreationDisabled + // guard) — the app is single-tenant with one active user, so onboarding is paused. This + // guard runs before validation/role-check, so even an invalid payload or a non-admin + // caller gets the same 403 rather than reaching those checks. + jest.mocked(admin.auth().verifyIdToken).mockResolvedValue({ uid: 'admin-uid' } as any); + + const response = await request(app) + .post('/users') + .set('Authorization', 'Bearer valid-token') + .send({ email: 'newuser@test.com', password: 'password123', role: 'assistant' }); + + expect(response.status).toBe(403); + expect(response.body.error ?? response.body.message).toMatch(/currently disabled/i); + expect(admin.auth().createUser).not.toHaveBeenCalled(); + }); +}); + +// The following describe block tests the actual create-user behavior (validation, role +// gating, restore-on-recreate) and is preserved so it can be re-enabled with a one-line +// `.skip` removal once account creation is turned back on — see the guard's own comment in +// user.route.ts for the paired re-enable step. +describe.skip('POST /users (disabled while account creation is paused — re-enable together with user.route.ts)', () => { + beforeEach(() => { + jest.clearAllMocks(); + }); + + it('should create the Firebase Auth account server-side and the profile with the specified role', async () => { const newUid = 'new-meter-reader-1'; jest.mocked(admin.auth().verifyIdToken).mockResolvedValue({ uid: 'admin-uid' } as any); + jest.mocked(admin.auth().createUser).mockResolvedValue({ uid: newUid } as any); // Keyed on id (not call order) — the requireRole role-cache is a module-level // singleton that persists across tests, so it may skip calling getById for // 'admin-uid' on later tests, leaving mockResolvedValueOnce queues misaligned. jest.mocked(userRepository.getById).mockImplementation(async (id: string) => { if (id === 'admin-uid') return mockAdminUser as any; // requireRole: fetch caller's role - return null; // createUser: target does not exist + return null; // createUser: target does not exist yet }); - jest.mocked(userRepository.create).mockResolvedValue({ + jest.mocked(setDocument).mockResolvedValue({ id: newUid, email: '', display_name: '', @@ -75,38 +108,58 @@ describe('POST /users', () => { const response = await request(app) .post('/users') .set('Authorization', 'Bearer valid-token') - .send({ uid: newUid, role: 'assistant' }); + .send({ email: 'newuser@test.com', password: 'password123', role: 'assistant' }); expect(response.status).toBe(201); + expect(admin.auth().createUser).toHaveBeenCalledWith( + expect.objectContaining({ email: 'newuser@test.com', password: 'password123' }) + ); + expect(setDocument).toHaveBeenCalledWith( + expect.any(String), + newUid, + expect.objectContaining({ role: 'assistant' }) + ); expect(response.body.id).toBe(newUid); expect(response.body.role).toBe('assistant'); }); - it('should return 409 if user profile already exists', async () => { + it('should return 409 if the email already has a Firebase Auth account', async () => { jest.mocked(admin.auth().verifyIdToken).mockResolvedValue({ uid: 'admin-uid' } as any); - jest.mocked(userRepository.getById).mockImplementation(async (id: string) => { - if (id === 'admin-uid') return mockAdminUser as any; // requireRole - return { ...mockAdminUser, id: 'existing-user', is_deleted: false } as any; // createUser: target exists - }); + jest.mocked(userRepository.getById).mockResolvedValue(mockAdminUser as any); + jest.mocked(admin.auth().createUser).mockRejectedValue({ code: 'auth/email-already-exists' }); const response = await request(app) .post('/users') .set('Authorization', 'Bearer valid-token') - .send({ uid: 'existing-user', role: 'landlord' }); + .send({ email: 'existing@test.com', password: 'password123', role: 'landlord' }); expect(response.status).toBe(409); }); it('should return 400 if role is invalid', async () => { jest.mocked(admin.auth().verifyIdToken).mockResolvedValue({ uid: 'admin-uid' } as any); - jest.mocked(userRepository.getById).mockResolvedValue(mockAdminUser); + jest.mocked(userRepository.getById).mockResolvedValue(mockAdminUser as any); + + const response = await request(app) + .post('/users') + .set('Authorization', 'Bearer valid-token') + .send({ email: 'newuser@test.com', password: 'password123', role: 'invalid-role' }); + + expect(response.status).toBe(400); + expect(admin.auth().createUser).not.toHaveBeenCalled(); + }); + + it('should return 400 if password is too short', async () => { + jest.mocked(admin.auth().verifyIdToken).mockResolvedValue({ uid: 'admin-uid' } as any); + jest.mocked(userRepository.getById).mockResolvedValue(mockAdminUser as any); const response = await request(app) .post('/users') .set('Authorization', 'Bearer valid-token') - .send({ uid: 'some-uid', role: 'invalid-role' }); + .send({ email: 'newuser@test.com', password: 'short', role: 'assistant' }); expect(response.status).toBe(400); + expect(admin.auth().createUser).not.toHaveBeenCalled(); }); it('should return 403 if non-admin tries to create user', async () => { @@ -120,8 +173,9 @@ describe('POST /users', () => { const response = await request(app) .post('/users') .set('Authorization', 'Bearer valid-token') - .send({ uid: 'new-uid', role: 'assistant' }); + .send({ email: 'newuser@test.com', password: 'password123', role: 'assistant' }); expect(response.status).toBe(403); + expect(admin.auth().createUser).not.toHaveBeenCalled(); }); }); diff --git a/api/functions/src/index.ts b/api/functions/src/index.ts index ef14a91..031615c 100644 --- a/api/functions/src/index.ts +++ b/api/functions/src/index.ts @@ -28,7 +28,6 @@ import {imageExtractionRouter} from "./features/image-extraction/image-extractio import {reportsRouter} from "./features/reports/reports.route"; import {llmConfigRouter} from "./features/llm-config/llm-config.route"; import {chatbotRouter} from "./features/chatbot/chatbot.route"; -import {photoSettingsRouter} from "./features/photo-settings/photo-settings.route"; const app = express(); @@ -97,7 +96,6 @@ app.use("/image-extraction", imageExtractionRouter); app.use("/reports", reportsRouter); app.use("/llm-config", llmConfigRouter); app.use("/chatbot", chatbotRouter); -app.use("/photo-settings", photoSettingsRouter); // Error handling app.use(errorHandler); diff --git a/api/functions/src/lib/cached-repository.lib.ts b/api/functions/src/lib/cached-repository.lib.ts index 836a9be..48b10b1 100644 --- a/api/functions/src/lib/cached-repository.lib.ts +++ b/api/functions/src/lib/cached-repository.lib.ts @@ -157,7 +157,7 @@ export class CachedRepository { */ async cacheCreatedItem(item: T): Promise { await cacheSet(this.idCacheKey(item.id), item, this.cacheTTL); - await listAppend(this.listCacheKey(), item); + await listAppend(this.listCacheKey(), item, this.cacheTTL); } /** @@ -178,7 +178,7 @@ export class CachedRepository { // Update both cache tiers for each item for (const item of created) { await cacheSet(this.idCacheKey(item.id), item, this.cacheTTL); - await listAppend(this.listCacheKey(), item); + await listAppend(this.listCacheKey(), item, this.cacheTTL); } return created; diff --git a/api/functions/src/lib/image-fetch.util.ts b/api/functions/src/lib/image-fetch.util.ts index 69de663..c7bbe46 100644 --- a/api/functions/src/lib/image-fetch.util.ts +++ b/api/functions/src/lib/image-fetch.util.ts @@ -10,68 +10,25 @@ const ALLOWED_IMAGE_MIME_TYPES = new Set([ "image/heif", ]); -function validateImageUrl(imageUrl: string): void { - let parsed: URL; - try { - parsed = new URL(imageUrl); - } catch { - throw new Error("Invalid image URL"); - } - - if (!["http:", "https:"].includes(parsed.protocol)) { - throw new Error("Invalid image URL: only http and https are allowed"); - } - - // Block RFC-1918, loopback, link-local, and GCP metadata service - const blockedPattern = - /^(10\.\d+\.\d+\.\d+|172\.(1[6-9]|2\d|3[01])\.\d+\.\d+|192\.168\.\d+\.\d+|127\.\d+\.\d+\.\d+|169\.254\.\d+\.\d+|::1|localhost|metadata\.google\.internal)/i; - - if (blockedPattern.test(parsed.hostname)) { - throw new Error("Invalid image URL: private or reserved addresses are not allowed"); - } -} - +// Images are only ever provided as inline `data:image/*;base64,...` payloads +// (enforced upstream by ImageUrlSchema) — the server never fetches a +// client-supplied URL, so there is no SSRF surface here to guard against. async function resolveImage(imageUrl: string): Promise<{ buffer: Buffer; mimeType: string }> { - // Handle data URLs (e.g., data:image/jpeg;base64,...) - if (imageUrl.startsWith("data:")) { - const [header, base64Data] = imageUrl.split(","); - if (!base64Data) throw new Error("Invalid data URL format"); - // Extract MIME type from "data:image/png;base64" prefix - const mimeType = header.split(":")[1]?.split(";")[0] ?? "image/jpeg"; - if (!ALLOWED_IMAGE_MIME_TYPES.has(mimeType)) { - throw new Error(`Unsupported image type: ${mimeType}`); - } - const buffer = Buffer.from(base64Data, "base64"); - if (buffer.byteLength > MAX_IMAGE_BYTES) { - throw new Error(`Image exceeds maximum allowed size of ${MAX_IMAGE_BYTES} bytes`); - } - return {buffer, mimeType}; + if (!imageUrl.startsWith("data:")) { + throw new Error("Invalid image URL: only data:image/*;base64,... URLs are allowed"); } - validateImageUrl(imageUrl); - - // Handle regular URLs - const response = await fetch(imageUrl); - if (!response.ok) { - throw new Error(`Failed to fetch image: ${response.statusText}`); - } - - const contentLength = response.headers.get("content-length"); - if (contentLength && Number(contentLength) > MAX_IMAGE_BYTES) { - throw new Error(`Image exceeds maximum allowed size of ${MAX_IMAGE_BYTES} bytes`); - } - - const contentType = response.headers.get("content-type") ?? "image/jpeg"; - const mimeType = contentType.split(";")[0].trim(); + const [header, base64Data] = imageUrl.split(","); + if (!base64Data) throw new Error("Invalid data URL format"); + // Extract MIME type from "data:image/png;base64" prefix + const mimeType = header.split(":")[1]?.split(";")[0] ?? "image/jpeg"; if (!ALLOWED_IMAGE_MIME_TYPES.has(mimeType)) { throw new Error(`Unsupported image type: ${mimeType}`); } - - const buffer = Buffer.from(await response.arrayBuffer()); + const buffer = Buffer.from(base64Data, "base64"); if (buffer.byteLength > MAX_IMAGE_BYTES) { throw new Error(`Image exceeds maximum allowed size of ${MAX_IMAGE_BYTES} bytes`); } - return {buffer, mimeType}; } diff --git a/api/functions/src/utils/batch-get.dto.ts b/api/functions/src/utils/batch-get.dto.ts new file mode 100644 index 0000000..2db5d72 --- /dev/null +++ b/api/functions/src/utils/batch-get.dto.ts @@ -0,0 +1,15 @@ +import {z} from "zod"; + +/** + * Shared `GET /:feature/batch-get?ids=a,b,c` query schema — comma-separated IDs, + * capped to match the entity-lookup-cache.ts client's realistic per-page batch size. + */ +export const BatchGetQueryDTOSchema = z.object({ + ids: z + .string() + .trim() + .min(1) + .transform((val) => Array.from(new Set(val.split(",").map((s) => s.trim()).filter(Boolean)))) + .pipe(z.array(z.string()).min(1).max(100)), +}); +export type BatchGetQueryDTO = z.infer; diff --git a/api/functions/src/utils/cascade-delete.util.ts b/api/functions/src/utils/cascade-delete.util.ts index fda861d..d5d4f04 100644 --- a/api/functions/src/utils/cascade-delete.util.ts +++ b/api/functions/src/utils/cascade-delete.util.ts @@ -6,6 +6,8 @@ import {cacheDel, cacheDelPattern} from "./cache.util"; import {AppError} from "./error.util"; import {readingLockId, isAlreadyExistsError} from "../features/reading/reading.util"; +type CascadeMode = "soft-delete" | "restore" | "purge"; + /** * Release the "one reading per meter_group+property+month" lock doc (see * reading.util.ts) for a reading being soft-deleted, so a legitimate @@ -75,6 +77,103 @@ async function invalidateListCaches(featureNames: string[]): Promise { await Promise.all(featureNames.map((featureName) => cacheDelPattern(`utilitool:${featureName}:all:*`))); } +/** Soft-delete/restore/hard-delete a single doc ref per cascade mode. */ +function mutateDoc(txn: FirebaseFirestore.Transaction, ref: FirebaseFirestore.DocumentReference, mode: CascadeMode): void { + if (mode === "purge") txn.delete(ref); + else if (mode === "soft-delete") txn.update(ref, {is_deleted: true, deleted_at: new Date()}); + else txn.update(ref, {is_deleted: false, deleted_at: null}); +} + +function assertExists(snap: FirebaseFirestore.DocumentSnapshot, label: string): void { + if (!snap.exists) throw new AppError(404, `${label} not found`); +} + +function assertArchivedForPurge(snap: FirebaseFirestore.DocumentSnapshot, label: string): void { + assertExists(snap, label); + if (snap.data()?.is_deleted !== true) { + throw new AppError(409, `Cannot permanently delete an active ${label.toLowerCase()}. Archive it first.`); + } +} + +/** + * Query readings by `field == value` (scoped to the appropriate is_deleted + * state for the given mode) and mutate each one in the transaction, + * releasing/reclaiming its month lock along the way. Returns the matched IDs. + */ +async function cascadeReadingsBy( + txn: FirebaseFirestore.Transaction, + field: "property_id" | "meter_group_id", + value: string, + mode: CascadeMode +): Promise { + const snap = await collectionRef(COLLECTIONS.READINGS) + .where(field, "==", value) + .where("is_deleted", "==", mode !== "soft-delete") + .get(); + + for (const doc of snap.docs) { + mutateDoc(txn, doc.ref, mode); + if (mode === "purge") continue; + const data = doc.data(); + if (mode === "soft-delete") releaseReadingLock(txn, data.meter_group_id, data.property_id, data.reading_date); + else reclaimReadingLock(txn, data.meter_group_id, data.property_id, data.reading_date); + } + + return snap.docs.map((doc) => doc.id); +} + +/** Query billings directly by property_id and mutate each in the transaction. */ +async function cascadeBillingsByPropertyId( + txn: FirebaseFirestore.Transaction, + propertyId: string, + mode: CascadeMode +): Promise { + const snap = await collectionRef(COLLECTIONS.BILLINGS) + .where("property_id", "==", propertyId) + .where("is_deleted", "==", mode !== "soft-delete") + .get(); + + snap.docs.forEach((doc) => mutateDoc(txn, doc.ref, mode)); + return snap.docs.map((doc) => doc.id); +} + +/** + * Find billings referencing any of the given reading IDs (as previous or + * current reading) and mutate each in the transaction. Skips the query + * entirely when there are no reading IDs to match, avoiding a wasted scan. + */ +async function cascadeBillingsByReadingIds( + txn: FirebaseFirestore.Transaction, + readingIds: string[], + mode: CascadeMode +): Promise { + if (readingIds.length === 0) return []; + + const snap = await collectionRef(COLLECTIONS.BILLINGS).where("is_deleted", "==", mode !== "soft-delete").get(); + + const idSet = new Set(readingIds); + const matched = snap.docs.filter((doc) => { + const billing = doc.data(); + return idSet.has(billing.previous_reading_id) || idSet.has(billing.current_reading_id); + }); + + matched.forEach((doc) => mutateDoc(txn, doc.ref, mode)); + return matched.map((doc) => doc.id); +} + +async function refreshCaches( + entityCollectionCacheName: string, + entityId: string, + readingIds: string[], + billingIds: string[], + listCacheNames: string[] +): Promise { + await cacheDel(`utilitool:${entityCollectionCacheName}:id:${entityId}`); + await Promise.all(readingIds.map((id) => cacheDel(`utilitool:readings:id:${id}`))); + await Promise.all(billingIds.map((id) => cacheDel(`utilitool:billings:id:${id}`))); + await invalidateListCaches(listCacheNames); +} + export interface CascadeDeleteSummary { primary: number; // The entity being deleted (always 1) readings?: number; @@ -86,63 +185,21 @@ export interface CascadeDeleteSummary { * Uses a transaction for atomicity. */ export async function cascadeDeleteProperty(propertyId: string): Promise { - const summary: CascadeDeleteSummary = {primary: 1, readings: 0, billings: 0}; - const readingIds: string[] = []; - const billingIds: string[] = []; + let readingIds: string[] = []; + let billingIds: string[] = []; await firestore.runTransaction(async (txn) => { const propertyRef = firestore.collection(COLLECTIONS.PROPERTIES).doc(propertyId); - const propertySnap = await txn.get(propertyRef); - - if (!propertySnap.exists) { - throw new Error("Property not found"); - } - - // Soft-delete property + assertExists(await txn.get(propertyRef), "Property"); txn.update(propertyRef, {is_deleted: true, deleted_at: new Date()}); - // Find all readings for this property - const readingsSnap = await collectionRef(COLLECTIONS.READINGS) - .where("property_id", "==", propertyId) - .where("is_deleted", "==", false) - .get(); - - const foundReadingIds = readingsSnap.docs.map((doc) => doc.id); - readingIds.push(...foundReadingIds); - summary.readings = foundReadingIds.length; - - // Soft-delete all readings + release their month locks - for (const readingDoc of readingsSnap.docs) { - txn.update(readingDoc.ref, {is_deleted: true, deleted_at: new Date()}); - const readingData = readingDoc.data(); - releaseReadingLock(txn, readingData.meter_group_id, propertyId, readingData.reading_date); - } - - // Find all billings for this property - const billingsSnap = await collectionRef(COLLECTIONS.BILLINGS) - .where("property_id", "==", propertyId) - .where("is_deleted", "==", false) - .get(); - - const foundBillingIds = billingsSnap.docs.map((doc) => doc.id); - billingIds.push(...foundBillingIds); - summary.billings = foundBillingIds.length; - - // Soft-delete all billings - for (const billingDoc of billingsSnap.docs) { - txn.update(billingDoc.ref, {is_deleted: true, deleted_at: new Date()}); - } + readingIds = await cascadeReadingsBy(txn, "property_id", propertyId, "soft-delete"); + billingIds = await cascadeBillingsByPropertyId(txn, propertyId, "soft-delete"); }); - // Clear id caches after txn commits - await cacheDel(`utilitool:properties:id:${propertyId}`); - await Promise.all(readingIds.map((readingId) => cacheDel(`utilitool:readings:id:${readingId}`))); - await Promise.all(billingIds.map((billingId) => cacheDel(`utilitool:billings:id:${billingId}`))); - - // Clear list caches so cascaded readings/billings stop appearing in active lists - await invalidateListCaches(["properties", "readings", "billings"]); + await refreshCaches("properties", propertyId, readingIds, billingIds, ["properties", "readings", "billings"]); - return summary; + return {primary: 1, readings: readingIds.length, billings: billingIds.length}; } /** @@ -151,73 +208,21 @@ export async function cascadeDeleteProperty(propertyId: string): Promise { - const summary: CascadeDeleteSummary = {primary: 1, readings: 0, billings: 0}; - const readingIds: string[] = []; - const billingIds: string[] = []; + let readingIds: string[] = []; + let billingIds: string[] = []; await firestore.runTransaction(async (txn) => { const meterGroupRef = firestore.collection(COLLECTIONS.METER_GROUPS).doc(meterGroupId); - const meterGroupSnap = await txn.get(meterGroupRef); - - if (!meterGroupSnap.exists) { - throw new Error("Meter group not found"); - } - - // Soft-delete meter group + assertExists(await txn.get(meterGroupRef), "Meter group"); txn.update(meterGroupRef, {is_deleted: true, deleted_at: new Date()}); - // Find all readings for this meter group - const readingsSnap = await collectionRef(COLLECTIONS.READINGS) - .where("meter_group_id", "==", meterGroupId) - .where("is_deleted", "==", false) - .get(); - - const foundReadingIds = readingsSnap.docs.map((doc) => doc.id); - readingIds.push(...foundReadingIds); - summary.readings = foundReadingIds.length; - - // Soft-delete all readings + release their month locks - for (const readingDoc of readingsSnap.docs) { - txn.update(readingDoc.ref, {is_deleted: true, deleted_at: new Date()}); - const readingData = readingDoc.data(); - releaseReadingLock(txn, meterGroupId, readingData.property_id, readingData.reading_date); - } - - // Find all billings that reference these readings - if (foundReadingIds.length > 0) { - const billingsSnap = await collectionRef(COLLECTIONS.BILLINGS) - .where("is_deleted", "==", false) - .get(); - - const foundReadingIdSet = new Set(foundReadingIds); - const billingsToDelete = billingsSnap.docs.filter((doc) => { - const billing = doc.data(); - return ( - foundReadingIdSet.has(billing.previous_reading_id) || - foundReadingIdSet.has(billing.current_reading_id) - ); - }); - - const foundBillingIds = billingsToDelete.map((doc) => doc.id); - billingIds.push(...foundBillingIds); - summary.billings = billingsToDelete.length; - - // Soft-delete all related billings - for (const billingDoc of billingsToDelete) { - txn.update(billingDoc.ref, {is_deleted: true, deleted_at: new Date()}); - } - } + readingIds = await cascadeReadingsBy(txn, "meter_group_id", meterGroupId, "soft-delete"); + billingIds = await cascadeBillingsByReadingIds(txn, readingIds, "soft-delete"); }); - // Clear id caches after txn commits - await cacheDel(`utilitool:meter-groups:id:${meterGroupId}`); - await Promise.all(readingIds.map((readingId) => cacheDel(`utilitool:readings:id:${readingId}`))); - await Promise.all(billingIds.map((billingId) => cacheDel(`utilitool:billings:id:${billingId}`))); - - // Clear list caches so cascaded readings/billings stop appearing in active lists - await invalidateListCaches(["meter-groups", "readings", "billings"]); + await refreshCaches("meter-groups", meterGroupId, readingIds, billingIds, ["meter-groups", "readings", "billings"]); - return summary; + return {primary: 1, readings: readingIds.length, billings: billingIds.length}; } /** @@ -226,50 +231,23 @@ export async function cascadeDeleteMeterGroup(meterGroupId: string): Promise { - const summary: CascadeDeleteSummary = {primary: 1, billings: 0}; - const billingIds: string[] = []; + let billingIds: string[] = []; await firestore.runTransaction(async (txn) => { const readingRef = firestore.collection(COLLECTIONS.READINGS).doc(readingId); const readingSnap = await txn.get(readingRef); - - if (!readingSnap.exists) { - throw new Error("Reading not found"); - } - - // Soft-delete reading + assertExists(readingSnap, "Reading"); txn.update(readingRef, {is_deleted: true, deleted_at: new Date()}); const readingData = readingSnap.data()!; releaseReadingLock(txn, readingData.meter_group_id, readingData.property_id, readingData.reading_date); - // Find all billings that reference this reading - const billingsSnap = await collectionRef(COLLECTIONS.BILLINGS) - .where("is_deleted", "==", false) - .get(); - - const billingsToDelete = billingsSnap.docs.filter((doc) => { - const billing = doc.data(); - return billing.previous_reading_id === readingId || billing.current_reading_id === readingId; - }); - - billingIds.push(...billingsToDelete.map((doc) => doc.id)); - summary.billings = billingsToDelete.length; - - // Soft-delete all related billings - for (const billingDoc of billingsToDelete) { - txn.update(billingDoc.ref, {is_deleted: true, deleted_at: new Date()}); - } + billingIds = await cascadeBillingsByReadingIds(txn, [readingId], "soft-delete"); }); - // Clear id caches after txn commits - await cacheDel(`utilitool:readings:id:${readingId}`); - await Promise.all(billingIds.map((billingId) => cacheDel(`utilitool:billings:id:${billingId}`))); - - // Clear list caches so cascaded billings stop appearing in active lists - await invalidateListCaches(["readings", "billings"]); + await refreshCaches("readings", readingId, [], billingIds, ["readings", "billings"]); - return summary; + return {primary: 1, billings: billingIds.length}; } /** @@ -280,49 +258,23 @@ export async function cascadeDeleteReading(readingId: string): Promise { - const summary: CascadeDeleteSummary = {primary: 1, readings: 0, billings: 0}; - const readingIds: string[] = []; - const billingIds: string[] = []; + let readingIds: string[] = []; + let billingIds: string[] = []; await firestore.runTransaction(async (txn) => { const propertyRef = firestore.collection(COLLECTIONS.PROPERTIES).doc(propertyId); - const propertySnap = await txn.get(propertyRef); - - if (!propertySnap.exists) { - throw new AppError(404, "Property not found"); - } - if (propertySnap.data()?.is_deleted !== true) { - throw new AppError(409, "Cannot permanently delete an active property. Archive it first."); - } - + assertArchivedForPurge(await txn.get(propertyRef), "Property"); txn.delete(propertyRef); - const readingsSnap = await collectionRef(COLLECTIONS.READINGS) - .where("property_id", "==", propertyId) - .where("is_deleted", "==", true) - .get(); - - const foundReadingIds = readingsSnap.docs.map((doc) => doc.id); - readingIds.push(...foundReadingIds); - summary.readings = foundReadingIds.length; - readingsSnap.docs.forEach((doc) => txn.delete(doc.ref)); - - const billingsSnap = await collectionRef(COLLECTIONS.BILLINGS) - .where("property_id", "==", propertyId) - .where("is_deleted", "==", true) - .get(); - - const foundBillingIds = billingsSnap.docs.map((doc) => doc.id); - billingIds.push(...foundBillingIds); - summary.billings = foundBillingIds.length; - billingsSnap.docs.forEach((doc) => txn.delete(doc.ref)); + readingIds = await cascadeReadingsBy(txn, "property_id", propertyId, "purge"); + billingIds = await cascadeBillingsByPropertyId(txn, propertyId, "purge"); }); await cacheDel(`utilitool:properties:id:${propertyId}`); - await Promise.all(readingIds.map((readingId) => cacheDel(`utilitool:readings:id:${readingId}`))); - await Promise.all(billingIds.map((billingId) => cacheDel(`utilitool:billings:id:${billingId}`))); + await Promise.all(readingIds.map((id) => cacheDel(`utilitool:readings:id:${id}`))); + await Promise.all(billingIds.map((id) => cacheDel(`utilitool:billings:id:${id}`))); - return summary; + return {primary: 1, readings: readingIds.length, billings: billingIds.length}; } /** @@ -331,59 +283,23 @@ export async function cascadePurgeProperty(propertyId: string): Promise { - const summary: CascadeDeleteSummary = {primary: 1, readings: 0, billings: 0}; - const readingIds: string[] = []; - const billingIds: string[] = []; + let readingIds: string[] = []; + let billingIds: string[] = []; await firestore.runTransaction(async (txn) => { const meterGroupRef = firestore.collection(COLLECTIONS.METER_GROUPS).doc(meterGroupId); - const meterGroupSnap = await txn.get(meterGroupRef); - - if (!meterGroupSnap.exists) { - throw new AppError(404, "Meter group not found"); - } - if (meterGroupSnap.data()?.is_deleted !== true) { - throw new AppError(409, "Cannot permanently delete an active meter group. Archive it first."); - } - + assertArchivedForPurge(await txn.get(meterGroupRef), "Meter group"); txn.delete(meterGroupRef); - const readingsSnap = await collectionRef(COLLECTIONS.READINGS) - .where("meter_group_id", "==", meterGroupId) - .where("is_deleted", "==", true) - .get(); - - const foundReadingIds = readingsSnap.docs.map((doc) => doc.id); - readingIds.push(...foundReadingIds); - summary.readings = foundReadingIds.length; - readingsSnap.docs.forEach((doc) => txn.delete(doc.ref)); - - if (foundReadingIds.length > 0) { - const billingsSnap = await collectionRef(COLLECTIONS.BILLINGS) - .where("is_deleted", "==", true) - .get(); - - const foundReadingIdSet = new Set(foundReadingIds); - const billingsToPurge = billingsSnap.docs.filter((doc) => { - const billing = doc.data(); - return ( - foundReadingIdSet.has(billing.previous_reading_id) || - foundReadingIdSet.has(billing.current_reading_id) - ); - }); - - const foundBillingIds = billingsToPurge.map((doc) => doc.id); - billingIds.push(...foundBillingIds); - summary.billings = billingsToPurge.length; - billingsToPurge.forEach((doc) => txn.delete(doc.ref)); - } + readingIds = await cascadeReadingsBy(txn, "meter_group_id", meterGroupId, "purge"); + billingIds = await cascadeBillingsByReadingIds(txn, readingIds, "purge"); }); await cacheDel(`utilitool:meter-groups:id:${meterGroupId}`); - await Promise.all(readingIds.map((readingId) => cacheDel(`utilitool:readings:id:${readingId}`))); - await Promise.all(billingIds.map((billingId) => cacheDel(`utilitool:billings:id:${billingId}`))); + await Promise.all(readingIds.map((id) => cacheDel(`utilitool:readings:id:${id}`))); + await Promise.all(billingIds.map((id) => cacheDel(`utilitool:billings:id:${id}`))); - return summary; + return {primary: 1, readings: readingIds.length, billings: billingIds.length}; } /** @@ -391,40 +307,20 @@ export async function cascadePurgeMeterGroup(meterGroupId: string): Promise { - const summary: CascadeDeleteSummary = {primary: 1, billings: 0}; - const billingIds: string[] = []; + let billingIds: string[] = []; await firestore.runTransaction(async (txn) => { const readingRef = firestore.collection(COLLECTIONS.READINGS).doc(readingId); - const readingSnap = await txn.get(readingRef); - - if (!readingSnap.exists) { - throw new AppError(404, "Reading not found"); - } - if (readingSnap.data()?.is_deleted !== true) { - throw new AppError(409, "Cannot permanently delete an active reading. Archive it first."); - } - + assertArchivedForPurge(await txn.get(readingRef), "Reading"); txn.delete(readingRef); - const billingsSnap = await collectionRef(COLLECTIONS.BILLINGS) - .where("is_deleted", "==", true) - .get(); - - const billingsToPurge = billingsSnap.docs.filter((doc) => { - const billing = doc.data(); - return billing.previous_reading_id === readingId || billing.current_reading_id === readingId; - }); - - billingIds.push(...billingsToPurge.map((doc) => doc.id)); - summary.billings = billingsToPurge.length; - billingsToPurge.forEach((doc) => txn.delete(doc.ref)); + billingIds = await cascadeBillingsByReadingIds(txn, [readingId], "purge"); }); await cacheDel(`utilitool:readings:id:${readingId}`); - await Promise.all(billingIds.map((billingId) => cacheDel(`utilitool:billings:id:${billingId}`))); + await Promise.all(billingIds.map((id) => cacheDel(`utilitool:billings:id:${id}`))); - return summary; + return {primary: 1, billings: billingIds.length}; } /** @@ -432,62 +328,21 @@ export async function cascadePurgeReading(readingId: string): Promise { - const summary: CascadeDeleteSummary = {primary: 1, readings: 0, billings: 0}; - const readingIds: string[] = []; - const billingIds: string[] = []; + let readingIds: string[] = []; + let billingIds: string[] = []; await runCascadeTransaction(async (txn) => { const propertyRef = firestore.collection(COLLECTIONS.PROPERTIES).doc(propertyId); - const propertySnap = await txn.get(propertyRef); - - if (!propertySnap.exists) { - throw new AppError(404, "Property not found"); - } - - // Restore property + assertExists(await txn.get(propertyRef), "Property"); txn.update(propertyRef, {is_deleted: false, deleted_at: null}); - // Find all soft-deleted readings for this property - const readingsSnap = await collectionRef(COLLECTIONS.READINGS) - .where("property_id", "==", propertyId) - .where("is_deleted", "==", true) - .get(); - - const foundReadingIds = readingsSnap.docs.map((doc) => doc.id); - readingIds.push(...foundReadingIds); - summary.readings = foundReadingIds.length; - - // Restore all readings + reclaim their month locks - for (const readingDoc of readingsSnap.docs) { - txn.update(readingDoc.ref, {is_deleted: false, deleted_at: null}); - const readingData = readingDoc.data(); - reclaimReadingLock(txn, readingData.meter_group_id, propertyId, readingData.reading_date); - } - - // Find all soft-deleted billings for this property - const billingsSnap = await collectionRef(COLLECTIONS.BILLINGS) - .where("property_id", "==", propertyId) - .where("is_deleted", "==", true) - .get(); - - billingIds.push(...billingsSnap.docs.map((doc) => doc.id)); - summary.billings = billingsSnap.size; - - // Restore all billings - for (const billingDoc of billingsSnap.docs) { - txn.update(billingDoc.ref, {is_deleted: false, deleted_at: null}); - } + readingIds = await cascadeReadingsBy(txn, "property_id", propertyId, "restore"); + billingIds = await cascadeBillingsByPropertyId(txn, propertyId, "restore"); }); - // Refresh id caches so restored entities aren't served stale (archived) from cache - await cacheDel(`utilitool:properties:id:${propertyId}`); - await Promise.all(readingIds.map((readingId) => cacheDel(`utilitool:readings:id:${readingId}`))); - await Promise.all(billingIds.map((billingId) => cacheDel(`utilitool:billings:id:${billingId}`))); + await refreshCaches("properties", propertyId, readingIds, billingIds, ["properties", "readings", "billings"]); - // Clear list caches so cascaded readings/billings reappear in active lists - await invalidateListCaches(["properties", "readings", "billings"]); - - return summary; + return {primary: 1, readings: readingIds.length, billings: billingIds.length}; } /** @@ -495,72 +350,21 @@ export async function cascadeRestoreProperty(propertyId: string): Promise { - const summary: CascadeDeleteSummary = {primary: 1, readings: 0, billings: 0}; - const allReadingIds: string[] = []; - const allBillingIds: string[] = []; + let readingIds: string[] = []; + let billingIds: string[] = []; await runCascadeTransaction(async (txn) => { const meterGroupRef = firestore.collection(COLLECTIONS.METER_GROUPS).doc(meterGroupId); - const meterGroupSnap = await txn.get(meterGroupRef); - - if (!meterGroupSnap.exists) { - throw new AppError(404, "Meter group not found"); - } - - // Restore meter group + assertExists(await txn.get(meterGroupRef), "Meter group"); txn.update(meterGroupRef, {is_deleted: false, deleted_at: null}); - // Find all soft-deleted readings for this meter group - const readingsSnap = await collectionRef(COLLECTIONS.READINGS) - .where("meter_group_id", "==", meterGroupId) - .where("is_deleted", "==", true) - .get(); - - const readingIds = readingsSnap.docs.map((doc) => doc.id); - allReadingIds.push(...readingIds); - summary.readings = readingIds.length; - - // Restore all readings + reclaim their month locks - for (const readingDoc of readingsSnap.docs) { - txn.update(readingDoc.ref, {is_deleted: false, deleted_at: null}); - const readingData = readingDoc.data(); - reclaimReadingLock(txn, meterGroupId, readingData.property_id, readingData.reading_date); - } - - // Find all soft-deleted billings that reference these readings - if (readingIds.length > 0) { - const billingsSnap = await collectionRef(COLLECTIONS.BILLINGS) - .where("is_deleted", "==", true) - .get(); - - const readingIdSet = new Set(readingIds); - const billingsToRestore = billingsSnap.docs.filter((doc) => { - const billing = doc.data(); - return ( - readingIdSet.has(billing.previous_reading_id) || - readingIdSet.has(billing.current_reading_id) - ); - }); - - allBillingIds.push(...billingsToRestore.map((doc) => doc.id)); - summary.billings = billingsToRestore.length; - - // Restore all related billings - for (const billingDoc of billingsToRestore) { - txn.update(billingDoc.ref, {is_deleted: false, deleted_at: null}); - } - } + readingIds = await cascadeReadingsBy(txn, "meter_group_id", meterGroupId, "restore"); + billingIds = await cascadeBillingsByReadingIds(txn, readingIds, "restore"); }); - // Refresh id caches so restored entities aren't served stale (archived) from cache - await cacheDel(`utilitool:meter-groups:id:${meterGroupId}`); - await Promise.all(allReadingIds.map((readingId) => cacheDel(`utilitool:readings:id:${readingId}`))); - await Promise.all(allBillingIds.map((billingId) => cacheDel(`utilitool:billings:id:${billingId}`))); - - // Clear list caches so cascaded readings/billings reappear in active lists - await invalidateListCaches(["meter-groups", "readings", "billings"]); + await refreshCaches("meter-groups", meterGroupId, readingIds, billingIds, ["meter-groups", "readings", "billings"]); - return summary; + return {primary: 1, readings: readingIds.length, billings: billingIds.length}; } /** @@ -568,48 +372,21 @@ export async function cascadeRestoreMeterGroup(meterGroupId: string): Promise { - const summary: CascadeDeleteSummary = {primary: 1, billings: 0}; - const billingIds: string[] = []; + let billingIds: string[] = []; await runCascadeTransaction(async (txn) => { const readingRef = firestore.collection(COLLECTIONS.READINGS).doc(readingId); const readingSnap = await txn.get(readingRef); - - if (!readingSnap.exists) { - throw new AppError(404, "Reading not found"); - } - - // Restore reading + assertExists(readingSnap, "Reading"); txn.update(readingRef, {is_deleted: false, deleted_at: null}); const readingData = readingSnap.data()!; reclaimReadingLock(txn, readingData.meter_group_id, readingData.property_id, readingData.reading_date); - // Find all soft-deleted billings that reference this reading - const billingsSnap = await collectionRef(COLLECTIONS.BILLINGS) - .where("is_deleted", "==", true) - .get(); - - const billingsToRestore = billingsSnap.docs.filter((doc) => { - const billing = doc.data(); - return billing.previous_reading_id === readingId || billing.current_reading_id === readingId; - }); - - billingIds.push(...billingsToRestore.map((doc) => doc.id)); - summary.billings = billingsToRestore.length; - - // Restore all related billings - for (const billingDoc of billingsToRestore) { - txn.update(billingDoc.ref, {is_deleted: false, deleted_at: null}); - } + billingIds = await cascadeBillingsByReadingIds(txn, [readingId], "restore"); }); - // Refresh id caches so restored entities aren't served stale (archived) from cache - await cacheDel(`utilitool:readings:id:${readingId}`); - await Promise.all(billingIds.map((billingId) => cacheDel(`utilitool:billings:id:${billingId}`))); - - // Clear list caches so cascaded billings reappear in active lists - await invalidateListCaches(["readings", "billings"]); + await refreshCaches("readings", readingId, [], billingIds, ["readings", "billings"]); - return summary; + return {primary: 1, billings: billingIds.length}; } diff --git a/api/functions/src/utils/clear-cache-handler.util.ts b/api/functions/src/utils/clear-cache-handler.util.ts new file mode 100644 index 0000000..b5d1bd8 --- /dev/null +++ b/api/functions/src/utils/clear-cache-handler.util.ts @@ -0,0 +1,15 @@ +import type {Response} from "express"; +import type {AuthenticatedRequest} from "./auth.util"; +import {cacheDelPattern} from "./cache.util"; + +/** + * Every feature's `POST /cache/clear` handler was byte-identical except for the + * cache-key prefix and the human-readable label in the response message — + * this factory is the single copy. + */ +export function makeClearCacheHandler(cacheKeyPrefix: string, label: string) { + return async (_req: AuthenticatedRequest, res: Response): Promise => { + const deletedCount = await cacheDelPattern(`utilitool:${cacheKeyPrefix}:*`); + res.status(200).json({message: `Cleared ${deletedCount} cache entries for ${label}`}); + }; +} diff --git a/api/functions/src/utils/date-range-filter.util.ts b/api/functions/src/utils/date-range-filter.util.ts new file mode 100644 index 0000000..b9d534a --- /dev/null +++ b/api/functions/src/utils/date-range-filter.util.ts @@ -0,0 +1,41 @@ +import {parseTimestamp} from "./firestore.util"; +import type {BaseModel} from "./model.util"; +import type {Timestamp} from "firebase-admin/firestore"; + +/** + * Post-filters an in-memory list by a date range. Used for the cached (active-item) + * search path in billing.service.ts / billing-cycle.service.ts, where dates can't be + * pushed down into the Firestore query (CachedRepository.applyFilters only supports + * equality filters — see its doc comment). + * + * startField/endField may be the same field (e.g. billing.billing_period_date) or + * different fields (e.g. billing-cycle's billing_start_date / billing_end_date) — + * the two comparisons are independent. + */ +export function applyDateRangeFilter( + items: T[], + options: { + startDate?: string; + endDate?: string; + startField: keyof T & string; + endField: keyof T & string; + } +): T[] { + let result = items; + const {startDate, endDate, startField, endField} = options; + + if (startDate) { + const start = new Date(startDate); + result = result.filter( + (item) => parseTimestamp(item[startField] as unknown as Timestamp).toDate() >= start + ); + } + if (endDate) { + const end = new Date(endDate); + result = result.filter( + (item) => parseTimestamp(item[endField] as unknown as Timestamp).toDate() <= end + ); + } + + return result; +} diff --git a/api/functions/src/utils/error.util.ts b/api/functions/src/utils/error.util.ts index 714ea3b..ec08d68 100644 --- a/api/functions/src/utils/error.util.ts +++ b/api/functions/src/utils/error.util.ts @@ -12,3 +12,16 @@ export class AppError extends Error { this.name = "AppError"; } } + +/** Fetch by id via the given lookup, or throw a 404 AppError with a consistent message. */ +export async function getOrThrow( + getById: (id: string) => Promise, + id: string, + label: string +): Promise { + const entity = await getById(id); + if (!entity) { + throw new AppError(404, `${label} not found`); + } + return entity; +} diff --git a/api/functions/src/utils/image-url.util.test.ts b/api/functions/src/utils/image-url.util.test.ts index 00c571e..2752043 100644 --- a/api/functions/src/utils/image-url.util.test.ts +++ b/api/functions/src/utils/image-url.util.test.ts @@ -2,39 +2,21 @@ import { describe, it, expect } from '@jest/globals'; import { isSafeImageUrl, ImageUrlSchema } from './image-url.util'; describe('isSafeImageUrl', () => { - it('allows https URLs to public hosts', () => { - expect(isSafeImageUrl('https://storage.googleapis.com/bucket/photo.jpg')).toBe(true); - }); - it('allows data:image/* URLs', () => { expect(isSafeImageUrl('data:image/png;base64,iVBORw0KGgo=')).toBe(true); + expect(isSafeImageUrl('data:image/jpeg;base64,/9j/4AAQ')).toBe(true); }); - it('rejects http (non-https) URLs', () => { - expect(isSafeImageUrl('http://example.com/photo.jpg')).toBe(false); - }); - - it('rejects the cloud metadata endpoint', () => { + it('rejects https URLs (no server-side fetch is ever performed)', () => { + expect(isSafeImageUrl('https://storage.googleapis.com/bucket/photo.jpg')).toBe(false); expect(isSafeImageUrl('https://169.254.169.254/latest/meta-data/')).toBe(false); }); - it('rejects loopback hosts', () => { - expect(isSafeImageUrl('https://127.0.0.1/photo.jpg')).toBe(false); - expect(isSafeImageUrl('https://localhost/photo.jpg')).toBe(false); - }); - - it('rejects private RFC1918 ranges', () => { - expect(isSafeImageUrl('https://10.0.0.5/photo.jpg')).toBe(false); - expect(isSafeImageUrl('https://172.16.0.1/photo.jpg')).toBe(false); - expect(isSafeImageUrl('https://172.31.255.255/photo.jpg')).toBe(false); - expect(isSafeImageUrl('https://192.168.1.1/photo.jpg')).toBe(false); - }); - - it('allows a public host that merely resembles a private one lexically', () => { - expect(isSafeImageUrl('https://172.32.0.1/photo.jpg')).toBe(true); + it('rejects http URLs', () => { + expect(isSafeImageUrl('http://example.com/photo.jpg')).toBe(false); }); - it('rejects malformed URLs', () => { + it('rejects malformed or non-data strings', () => { expect(isSafeImageUrl('not-a-url')).toBe(false); }); @@ -46,17 +28,15 @@ describe('isSafeImageUrl', () => { }); describe('ImageUrlSchema', () => { - it('parses a valid https URL', () => { - expect(ImageUrlSchema.parse('https://example.com/photo.jpg')).toBe( - 'https://example.com/photo.jpg' - ); - }); - it('parses a valid data URL', () => { const dataUrl = 'data:image/jpeg;base64,/9j/4AAQ'; expect(ImageUrlSchema.parse(dataUrl)).toBe(dataUrl); }); + it('throws on an https URL, even to a public host', () => { + expect(() => ImageUrlSchema.parse('https://example.com/photo.jpg')).toThrow(); + }); + it('throws on an SSRF-prone internal URL', () => { expect(() => ImageUrlSchema.parse('https://169.254.169.254/latest/meta-data/')).toThrow(); }); diff --git a/api/functions/src/utils/image-url.util.ts b/api/functions/src/utils/image-url.util.ts index bb55ceb..ead10c5 100644 --- a/api/functions/src/utils/image-url.util.ts +++ b/api/functions/src/utils/image-url.util.ts @@ -1,58 +1,16 @@ import {z} from "zod"; -// Blocks SSRF: only allow `data:image/*` URLs or `https://` URLs to a public host. -// Rejects internal/private/loopback/link-local hosts so a server-side fetch of a -// user-supplied image_url can't be used to probe internal services or the cloud -// metadata endpoint (169.254.169.254). -const PRIVATE_HOSTNAME_PATTERNS = [ - /^localhost$/i, - /^127\./, - /^0\.0\.0\.0$/, - /^10\./, - /^172\.(1[6-9]|2\d|3[0-1])\./, - /^192\.168\./, - /^169\.254\./, - /^::1$/, - /^\[?::1\]?$/, - /^fe80:/i, - /^fc00:/i, - /^fd[0-9a-f]{2}:/i, -]; - -function isPrivateHostname(hostname: string): boolean { - return PRIVATE_HOSTNAME_PATTERNS.some((pattern) => pattern.test(hostname)); -} +// Eliminates SSRF entirely: the server never fetches a client-supplied URL. +// Only inline `data:image/*;base64,...` payloads are accepted, so there is no +// code path where the server issues an outbound request to an attacker-chosen +// host (previously guarded by a hostname/IP blocklist, which is inherently +// incomplete against DNS-controlled hosts). +const DATA_IMAGE_URL_PATTERN = /^data:image\/[a-zA-Z0-9.+-]+;base64,/; export function isSafeImageUrl(value: string): boolean { - if (value.startsWith("data:image/")) { - return true; - } - - let url: URL; - try { - url = new URL(value); - } catch { - return false; - } - - if (url.protocol !== "https:") { - return false; - } - - return !isPrivateHostname(url.hostname); + return DATA_IMAGE_URL_PATTERN.test(value); } export const ImageUrlSchema = z .string() - .refine((value) => { - try { - // Reuse url() validation for non-data URLs; data URLs skip it. - if (!value.startsWith("data:")) { - z.string().url().parse(value); - } - return true; - } catch { - return false; - } - }, "Invalid image URL") - .refine(isSafeImageUrl, "image_url must be an https:// URL to a public host, or a data:image/* URL"); + .refine(isSafeImageUrl, "image_url must be a data:image/*;base64,... URL"); diff --git a/api/functions/src/utils/list-cache.util.ts b/api/functions/src/utils/list-cache.util.ts index 474c273..87738bd 100644 --- a/api/functions/src/utils/list-cache.util.ts +++ b/api/functions/src/utils/list-cache.util.ts @@ -125,15 +125,14 @@ export function paginate( export async function listAppend( cacheKey: string, - item: T + item: T, + ttlSeconds: number = 30 * 60 ): Promise { const cached = await cacheGet(cacheKey); if (!cached) return; // Cache miss, let next GET populate it cached.push(item); - // Refresh TTL - maintain original 20-30 min TTL - const ttl = 30 * 60; // Default to 30 min - await cacheSet(cacheKey, cached, ttl); + await cacheSet(cacheKey, cached, ttlSeconds); } export async function listUpdate( diff --git a/mobile/BUILD.md b/mobile/BUILD.md index c27e913..ad79be3 100644 --- a/mobile/BUILD.md +++ b/mobile/BUILD.md @@ -23,24 +23,29 @@ npx cap open android # Open Android Studio - Enable USB debugging on physical device - Connect via `adb devices` -## Security: Network Security Config (required after a fresh `npx cap add android`) - -The `android/` directory is git-ignored and regenerated locally, so this step must be repeated -whenever the Android project is recreated from scratch: - -1. Create `android/app/src/main/res/xml/network_security_config.xml`: - ```xml - - - - - - - - - ``` -2. Add `android:networkSecurityConfig="@xml/network_security_config"` to the `` tag - in `android/app/src/main/AndroidManifest.xml`. +## Security: Network Security Config (enforced automatically) + +The `android/` directory is git-ignored and regenerated locally via `npx cap add android`, so this +config previously had to be manually recreated every time — an easy step to forget on a fresh +checkout, a new machine, or after deleting `android/` to fix a sync issue. It's now applied +automatically by `scripts/apply-network-security-config.mjs`, wired into both +`npm run cap:add:android` and `npm run cap:sync` (see `package.json`). Run +`npm run verify:network-security-config` any time to check/re-apply it manually. + +The script creates `android/app/src/main/res/xml/network_security_config.xml`: +```xml + + + + + + + + +``` +and adds `android:networkSecurityConfig="@xml/network_security_config"` to the `` tag +in `android/app/src/main/AndroidManifest.xml` if it's missing. It's idempotent — safe to run +repeatedly, and a no-op once both are already in place. This explicitly denies cleartext (`http://`) traffic app-wide, rather than relying on the Android API-level default. Pairs with the build-time HTTPS guard in `src/lib/api/client.ts`. diff --git a/mobile/CLAUDE.md b/mobile/CLAUDE.md index 27091d8..91e44af 100644 --- a/mobile/CLAUDE.md +++ b/mobile/CLAUDE.md @@ -46,17 +46,16 @@ mobile/ │ └── lib/ │ ├── api/ │ │ ├── client.ts → Base fetch: Bearer token + 401 retry -│ │ ├── meter-groups.ts → listMeterGroups, getMeterGroup -│ │ ├── readings.ts → listReadings, getReading, createReading, createReadingsBatch, ocrReadingImage -│ │ ├── properties.ts → listProperties, getProperty -│ │ ├── billings.ts → listBillings, getBilling, updateBillingStatus -│ │ ├── billing-cycles.ts → listBillingCycles, getBillingCycle -│ │ └── photo-settings.ts → getPhotoSettings, upsertPhotoSettings (GET/PATCH /photo-settings) +│ │ ├── meter-groups.ts → listMeterGroups +│ │ ├── readings.ts → listReadings, getReading, createReadingsBatch, ocrReadingImage +│ │ ├── properties.ts → listProperties +│ │ ├── billings.ts → listBillings, updateBillingStatus +│ │ └── billing-cycles.ts → listBillingCycles │ ├── utils/ │ │ ├── auth-errors.ts → getReadableAuthError │ │ ├── billing-cycle.util.ts → getStatusSummary, getCyclePaidAmount, getCycleOutstandingAmount │ │ ├── format.ts → getReadingUnit, formatReading -│ │ ├── timestamp.ts → toDate, formatTimestampDate, formatTimestampDateTime (Firestore timestamp handling) +│ │ ├── timestamp.ts → toDate, formatTimestampDate (Firestore timestamp handling) │ │ └── utility-colors.ts → getUtilityTypeBadgeClasses │ └── stores/ │ └── session.ts → sessionCache: module-level cache for meterGroups + properties (survives screen nav, cleared on sign-out) @@ -97,12 +96,11 @@ All API calls go through `src/lib/api/client.ts`: | Module | Functions | API Endpoints | |--------|-----------|---------------| -| `meter-groups.ts` | `listMeterGroups`, `getMeterGroup` | `GET /meter-groups?summary=true`, `GET /meter-groups/:id` | -| `readings.ts` | `listReadings`, `getReading`, `createReading`, `createReadingsBatch` | `GET /readings`, `GET /readings/:id`, `POST /readings`, `POST /readings/batch` | -| `properties.ts` | `listProperties`, `getProperty` | `GET /properties`, `GET /properties/:id` | -| `billings.ts` | `listBillings`, `getBilling`, `updateBillingStatus` | `GET /billings`, `GET /billings/:id`, `PATCH /billings/:id` | -| `billing-cycles.ts` | `listBillingCycles`, `getBillingCycle` | `GET /billing-cycles`, `GET /billing-cycles/:id` | -| `photo-settings.ts` | `getPhotoSettings`, `upsertPhotoSettings` | `GET /photo-settings`, `PATCH /photo-settings` | +| `meter-groups.ts` | `listMeterGroups` | `GET /meter-groups?summary=true` | +| `readings.ts` | `listReadings`, `getReading`, `createReadingsBatch` | `GET /readings`, `GET /readings/:id`, `POST /readings/batch` | +| `properties.ts` | `listProperties` | `GET /properties` | +| `billings.ts` | `listBillings`, `updateBillingStatus` | `GET /billings`, `PATCH /billings/:id` | +| `billing-cycles.ts` | `listBillingCycles` | `GET /billing-cycles` | --- @@ -117,7 +115,7 @@ All API calls go through `src/lib/api/client.ts`: Properties are filtered client-side: only properties whose `meter_groups` values include the selected meter group ID. -**Photo persistence**: gated by the `savePhotos` preference from `GET /photo-settings` (defaults to `false`, configurable in Settings). The captured photo is always used in-memory for the OCR suggest call; `image_url` is only included in the create/batch payload when `savePhotos` is `true` — otherwise the photo never leaves the device beyond that one OCR request. +**Photo handling**: a captured photo is only ever used in-memory for the OCR suggest call (`POST /readings/ocr`) — it is never included in the create/batch payload and is never persisted anywhere. Photos are sent to the API as base64 `data:` URIs, not fetchable URLs. ### ReadingHistory — Filterable List `src/screens/ReadingHistory.svelte` diff --git a/mobile/package.json b/mobile/package.json index 780c479..6ac38b1 100644 --- a/mobile/package.json +++ b/mobile/package.json @@ -6,9 +6,10 @@ "dev": "vite", "build": "vite build", "preview": "vite preview", - "cap:add:android": "npx cap add android", + "cap:add:android": "npx cap add android && node scripts/apply-network-security-config.mjs", "cap:open:android": "npx cap open android", - "cap:sync": "npx cap sync" + "cap:sync": "npx cap sync && node scripts/apply-network-security-config.mjs", + "verify:network-security-config": "node scripts/apply-network-security-config.mjs" }, "devDependencies": { "@capacitor/cli": "^6.1.0", diff --git a/mobile/scripts/apply-network-security-config.mjs b/mobile/scripts/apply-network-security-config.mjs new file mode 100644 index 0000000..2bc0432 --- /dev/null +++ b/mobile/scripts/apply-network-security-config.mjs @@ -0,0 +1,87 @@ +#!/usr/bin/env node +// Enforces the Android cleartext-traffic guard documented in BUILD.md. `android/` +// is git-ignored and regenerated locally via `npx cap add android`, so this step +// was previously a manual, unenforced instruction — easy to forget on a fresh +// checkout, a new machine, or after deleting `android/` to fix a sync issue. Wired +// into `cap:add:android` and `cap:sync` (see package.json) so it runs every time +// those commands do, instead of relying on a human to remember BUILD.md. +// +// Safe to run repeatedly: no-ops if the config already exists and is wired up. +// No-ops (with a warning, not a failure) if `android/` hasn't been generated yet, +// since `cap:add:android` runs this only after `cap add android` succeeds, but a +// bare `npm install` or first clone shouldn't fail over a platform that hasn't +// been added. + +import { existsSync, mkdirSync, readFileSync, writeFileSync } from "node:fs"; +import { dirname, join } from "node:path"; +import { fileURLToPath } from "node:url"; + +const __dirname = dirname(fileURLToPath(import.meta.url)); +const mobileRoot = join(__dirname, ".."); +const androidRoot = join(mobileRoot, "android"); +const xmlDir = join(androidRoot, "app", "src", "main", "res", "xml"); +const configPath = join(xmlDir, "network_security_config.xml"); +const manifestPath = join(androidRoot, "app", "src", "main", "AndroidManifest.xml"); + +const NETWORK_SECURITY_CONFIG = ` + + + + + + + +`; + +function main() { + if (!existsSync(androidRoot)) { + console.warn( + "[network-security-config] android/ not found yet — skipping " + + "(run `npm run cap:add:android` first)." + ); + return; + } + + let changed = false; + + if (!existsSync(configPath)) { + mkdirSync(xmlDir, { recursive: true }); + writeFileSync(configPath, NETWORK_SECURITY_CONFIG, "utf8"); + console.log(`[network-security-config] created ${configPath}`); + changed = true; + } + + if (!existsSync(manifestPath)) { + throw new Error( + `[network-security-config] android/ exists but AndroidManifest.xml is missing at ${manifestPath} — android/ may be partially generated; try removing it and re-running \`npm run cap:add:android\`.` + ); + } + + const manifest = readFileSync(manifestPath, "utf8"); + const applicationTagMatch = manifest.match(/]*>/s); + + if (!applicationTagMatch) { + throw new Error( + `[network-security-config] could not find an tag in ${manifestPath} — apply android:networkSecurityConfig="@xml/network_security_config" manually.` + ); + } + + const applicationTag = applicationTagMatch[0]; + if (!applicationTag.includes("android:networkSecurityConfig")) { + const patchedTag = applicationTag.replace( + />$/, + '\n android:networkSecurityConfig="@xml/network_security_config">' + ); + writeFileSync(manifestPath, manifest.replace(applicationTag, patchedTag), "utf8"); + console.log(`[network-security-config] wired networkSecurityConfig into ${manifestPath}`); + changed = true; + } + + console.log( + changed + ? "[network-security-config] cleartext traffic protection applied." + : "[network-security-config] already up to date." + ); +} + +main(); diff --git a/mobile/src/lib/api/billing-cycles.ts b/mobile/src/lib/api/billing-cycles.ts index 4fbdfb7..f5be6f6 100644 --- a/mobile/src/lib/api/billing-cycles.ts +++ b/mobile/src/lib/api/billing-cycles.ts @@ -31,7 +31,3 @@ export async function listBillingCycles(params?: { const path = query.toString() ? `/billing-cycles?${query}` : '/billing-cycles'; return apiGet(path); } - -export async function getBillingCycle(id: string): Promise { - return apiGet(`/billing-cycles/${id}`); -} diff --git a/mobile/src/lib/api/billings.ts b/mobile/src/lib/api/billings.ts index 61d4b54..5c981f2 100644 --- a/mobile/src/lib/api/billings.ts +++ b/mobile/src/lib/api/billings.ts @@ -18,10 +18,6 @@ export async function listBillings(propertyId?: string) { return apiGet(`/billings${params.toString() ? '?' + params.toString() : ''}`); } -export async function getBilling(id: string) { - return apiGet(`/billings/${id}`); -} - export async function updateBillingStatus(id: string, paymentStatus: string) { return apiPatch(`/billings/${id}`, { payment_status: paymentStatus }); } diff --git a/mobile/src/lib/api/meter-groups.ts b/mobile/src/lib/api/meter-groups.ts index 6f9c30b..6ccc394 100644 --- a/mobile/src/lib/api/meter-groups.ts +++ b/mobile/src/lib/api/meter-groups.ts @@ -14,7 +14,3 @@ export interface MeterGroup { export async function listMeterGroups() { return apiGet('/meter-groups?summary=true'); } - -export async function getMeterGroup(id: string) { - return apiGet(`/meter-groups/${id}`); -} diff --git a/mobile/src/lib/api/photo-settings.ts b/mobile/src/lib/api/photo-settings.ts deleted file mode 100644 index 28a0346..0000000 --- a/mobile/src/lib/api/photo-settings.ts +++ /dev/null @@ -1,13 +0,0 @@ -import { apiGet, apiPatch } from './client'; - -export interface PhotoSettings { - savePhotos: boolean; -} - -export async function getPhotoSettings(): Promise { - return apiGet('/photo-settings'); -} - -export async function upsertPhotoSettings(data: PhotoSettings): Promise { - return apiPatch('/photo-settings', data); -} diff --git a/mobile/src/lib/api/properties.ts b/mobile/src/lib/api/properties.ts index 70516c6..7fabc38 100644 --- a/mobile/src/lib/api/properties.ts +++ b/mobile/src/lib/api/properties.ts @@ -18,7 +18,3 @@ export interface Property { export async function listProperties() { return apiGet('/properties'); } - -export async function getProperty(id: string) { - return apiGet(`/properties/${id}`); -} diff --git a/mobile/src/lib/api/readings.ts b/mobile/src/lib/api/readings.ts index 32d52f6..a4c9830 100644 --- a/mobile/src/lib/api/readings.ts +++ b/mobile/src/lib/api/readings.ts @@ -6,7 +6,6 @@ export interface Reading { property_id: string; reading_amount: number; reading_date: string; - image_url?: string; meter_version: number; created_at: string; updated_at: string; @@ -18,7 +17,6 @@ export interface CreateReadingRequest { property_id: string; reading_amount: number; reading_date: string; - image_url?: string; } export interface BatchReadingRequest { @@ -46,10 +44,6 @@ export async function getReading(id: string): Promise { return apiGet(`/readings/${id}`); } -export async function createReading(data: CreateReadingRequest) { - return apiPost('/readings', data); -} - export async function createReadingsBatch(data: BatchReadingRequest): Promise> { if (!data.readings || data.readings.length === 0) { throw new Error('Cannot submit an empty batch — add at least one reading.'); diff --git a/mobile/src/lib/stores/session.ts b/mobile/src/lib/stores/session.ts index c74564b..251e393 100644 --- a/mobile/src/lib/stores/session.ts +++ b/mobile/src/lib/stores/session.ts @@ -1,5 +1,5 @@ -import type { MeterGroup } from '../api/meter-groups'; -import type { Property } from '../api/properties'; +import { listMeterGroups, type MeterGroup } from '../api/meter-groups'; +import { listProperties, type Property } from '../api/properties'; // Module-level session cache — survives screen navigation, cleared on sign-out let _meterGroups: MeterGroup[] | null = null; @@ -16,6 +16,23 @@ export const sessionCache = { _properties = d; }, + // Returns the cached list, fetching and caching it on first call. + async getOrFetchMeterGroups(): Promise { + if (!_meterGroups) { + const res = await listMeterGroups(); + _meterGroups = res.data || []; + } + return _meterGroups!; + }, + + async getOrFetchProperties(): Promise { + if (!_properties) { + const res = await listProperties(); + _properties = res.data || []; + } + return _properties!; + }, + clear: () => { _meterGroups = null; _properties = null; diff --git a/mobile/src/lib/utils/auth-errors.ts b/mobile/src/lib/utils/auth-errors.ts index d80b820..b674684 100644 --- a/mobile/src/lib/utils/auth-errors.ts +++ b/mobile/src/lib/utils/auth-errors.ts @@ -1,30 +1,18 @@ +// Maps Firebase auth error codes to user-friendly messages. +const AUTH_ERROR_MESSAGES: Record = { + 'auth/invalid-email': 'Invalid email address', + 'auth/user-disabled': 'This account has been disabled', + 'auth/user-not-found': 'Invalid email or password', + 'auth/invalid-login-credentials': 'Invalid email or password', + 'auth/invalid-password': 'Invalid email or password', + 'auth/too-many-requests': 'Too many failed attempts. Please try again later', + 'auth/network-request-failed': 'Network error. Please check your connection', + 'auth/operation-not-allowed': 'This authentication method is not available' +}; + export function getReadableAuthError(error: any): string { if (typeof error === 'string') return error; if (!error) return 'Authentication failed'; - const code = error.code || ''; - const message = error.message || ''; - - // Map Firebase error codes to user-friendly messages - if (code === 'auth/invalid-email') { - return 'Invalid email address'; - } - if (code === 'auth/user-disabled') { - return 'This account has been disabled'; - } - if (code === 'auth/user-not-found' || code === 'auth/invalid-login-credentials' || code === 'auth/invalid-password') { - return 'Invalid email or password'; - } - if (code === 'auth/too-many-requests') { - return 'Too many failed attempts. Please try again later'; - } - if (code === 'auth/network-request-failed') { - return 'Network error. Please check your connection'; - } - if (code === 'auth/operation-not-allowed') { - return 'This authentication method is not available'; - } - - // Fallback to generic message for unknown errors - return 'Unable to sign in. Please try again'; + return AUTH_ERROR_MESSAGES[error.code] || 'Unable to sign in. Please try again'; } diff --git a/mobile/src/lib/utils/readings-wizard.util.ts b/mobile/src/lib/utils/readings-wizard.util.ts new file mode 100644 index 0000000..f62500a --- /dev/null +++ b/mobile/src/lib/utils/readings-wizard.util.ts @@ -0,0 +1,11 @@ +import type { Property, MeterGroupEntry } from '../api/properties'; + +export function findMeterGroupEntry(property: Property, meterGroupId: string): MeterGroupEntry | undefined { + return Object.values(property.meter_groups).find((entry) => entry.meter_group_id === meterGroupId); +} + +// Main-meter properties need a one-time baseline "seed" reading (POST /readings/seed) +// before they can take regular batch readings. +export function needsSeedReading(property: Property, meterGroupId: string): boolean { + return findMeterGroupEntry(property, meterGroupId)?.is_main_meter === true; +} diff --git a/mobile/src/lib/utils/timestamp.ts b/mobile/src/lib/utils/timestamp.ts index 2e2f9ef..67c624f 100644 --- a/mobile/src/lib/utils/timestamp.ts +++ b/mobile/src/lib/utils/timestamp.ts @@ -34,13 +34,3 @@ export function formatTimestampDate(timestamp: FirestoreTimestamp | string | any }); } -export function formatTimestampDateTime(timestamp: FirestoreTimestamp | string | any): string { - const date = toDate(timestamp); - return date.toLocaleDateString('en-US', { - month: 'short', - day: 'numeric', - year: 'numeric', - hour: '2-digit', - minute: '2-digit' - }); -} diff --git a/mobile/src/screens/Billings.svelte b/mobile/src/screens/Billings.svelte index ab631a4..d1bf5da 100644 --- a/mobile/src/screens/Billings.svelte +++ b/mobile/src/screens/Billings.svelte @@ -2,7 +2,6 @@ import { listBillingCycles, type BillingCycle } from '../lib/api/billing-cycles'; import { listBillings, updateBillingStatus, type Billing } from '../lib/api/billings'; import { listMeterGroups, type MeterGroup } from '../lib/api/meter-groups'; - import { listProperties } from '../lib/api/properties'; import { formatTimestampDate } from '../lib/utils/timestamp'; import { getReadingUnit } from '../lib/utils/format'; import { getStatusSummary } from '../lib/utils/billing-cycle.util'; @@ -35,13 +34,7 @@ meterGroups = meterGroupsRes.data; } - // Fetch property names from cache or API - let properties = sessionCache.getProperties(); - if (!properties) { - const propsRes = await listProperties(); - properties = propsRes.data || []; - sessionCache.setProperties(properties); - } + const properties = await sessionCache.getOrFetchProperties(); const names: Record = {}; const propertyMap = new Map(properties.map(p => [p.id, p.room_name])); diff --git a/mobile/src/screens/CaptureReadings.svelte b/mobile/src/screens/CaptureReadings.svelte index dab1adb..dcb1d39 100644 --- a/mobile/src/screens/CaptureReadings.svelte +++ b/mobile/src/screens/CaptureReadings.svelte @@ -1,11 +1,11 @@ + +
+ +
diff --git a/ui/src/lib/stores/archive-page.svelte.ts b/ui/src/lib/stores/archive-page.svelte.ts new file mode 100644 index 0000000..6c650ff --- /dev/null +++ b/ui/src/lib/stores/archive-page.svelte.ts @@ -0,0 +1,79 @@ +import type { PaginatedResult } from '$lib/types/api.types'; + +interface ArchivePageConfig { + listFn: (params: { limit: number; archived: true }) => Promise>; + restoreFn: (id: string) => Promise; + clearCacheFn: () => Promise<{ message: string }>; + entityLabel: string; + /** Optional side-fetch run alongside the main list (e.g. property/meter-group name lookups). */ + loadExtra?: () => Promise; +} + +export function createArchivePageState(config: ArchivePageConfig) { + let data = $state>({ data: [], nextCursor: null, hasMore: false }); + let isLoading = $state(false); + let error = $state(''); + let restoringId = $state(null); + let isClearing = $state(false); + + async function loadData() { + isLoading = true; + error = ''; + try { + const [result] = await Promise.all([ + config.listFn({ limit: 100, archived: true }), + config.loadExtra?.() ?? Promise.resolve() + ]); + data = result; + } catch (err) { + error = err instanceof Error ? err.message : `Failed to load archived ${config.entityLabel}s`; + } finally { + isLoading = false; + } + } + + async function handleRestore(id: string) { + restoringId = id; + try { + await config.restoreFn(id); + await loadData(); + } catch (err) { + error = err instanceof Error ? err.message : `Failed to restore ${config.entityLabel}`; + } finally { + restoringId = null; + } + } + + async function handleClearCache() { + isClearing = true; + try { + const result = await config.clearCacheFn(); + error = result.message; + } catch (err) { + error = err instanceof Error ? err.message : 'Failed to clear cache'; + } finally { + isClearing = false; + } + } + + return { + get data() { + return data; + }, + get isLoading() { + return isLoading; + }, + get error() { + return error; + }, + get restoringId() { + return restoringId; + }, + get isClearing() { + return isClearing; + }, + loadData, + handleRestore, + handleClearCache + }; +} diff --git a/ui/src/lib/stores/entity-lookup-cache.ts b/ui/src/lib/stores/entity-lookup-cache.ts new file mode 100644 index 0000000..58ae9f5 --- /dev/null +++ b/ui/src/lib/stores/entity-lookup-cache.ts @@ -0,0 +1,79 @@ +import { getBillingsByIds as fetchBillingsByIds } from '$lib/api/billings'; +import { getReadingsByIds as fetchReadingsByIds } from '$lib/api/readings'; +import type { Billing } from '$lib/types/billing.types'; +import type { Reading } from '$lib/types/reading.types'; + +/** + * Plain (non-Svelte-state) in-memory ID → entity caches for billings and readings. + * + * The denormalized meter_group_id/billing_period_date fields (Landing 1) let the pages + * fetch scoped *lists* directly, so this only handles ID-based lookups: resolving the + * billing/reading IDs a set of visible billing cycles references, without pulling whole + * collections. Per-ID reads hit the API's CachedRepository.getById cache, which is far + * less prone to invalidation churn than the per-collection list cache (a single mark-paid + * invalidates the whole list, but only one ID entry). + * + * Only uncached IDs are fetched; results are merged into a persistent module-level cache. + * Call invalidateEntityLookupCache() after mutations, alongside the existing reload paths. + */ + +const billingCache = new Map(); +const readingCache = new Map(); + +// Server caps a single batch-get call at 100 IDs (batch-get.dto.ts) — chunk larger +// requests instead of failing validation. +const BATCH_GET_CHUNK_SIZE = 100; + +async function resolveByIds( + ids: string[], + cache: Map, + fetchByIds: (ids: string[]) => Promise +): Promise> { + const uniqueIds = Array.from(new Set(ids.filter((id) => id && id.trim()))); + const missing = uniqueIds.filter((id) => !cache.has(id)); + + if (missing.length > 0) { + const chunks: string[][] = []; + for (let i = 0; i < missing.length; i += BATCH_GET_CHUNK_SIZE) { + chunks.push(missing.slice(i, i + BATCH_GET_CHUNK_SIZE)); + } + + try { + const fetchedChunks = await Promise.all(chunks.map((chunk) => fetchByIds(chunk))); + for (const entity of fetchedChunks.flat()) { + cache.set(entity.id, entity); + } + } catch { + // A batch call failing (e.g. all-missing/soft-deleted IDs) resolves to + // "nothing newly cached" rather than aborting — callers already treat + // absent IDs as "unknown". + } + } + + // Return only the requested subset, in a fresh map the caller owns. + const result = new Map(); + for (const id of uniqueIds) { + const entity = cache.get(id); + if (entity) result.set(id, entity); + } + return result; +} + +export async function getBillingsByIds(ids: string[]): Promise> { + return resolveByIds(ids, billingCache, fetchBillingsByIds); +} + +export async function getReadingsByIds(ids: string[]): Promise> { + return resolveByIds(ids, readingCache, fetchReadingsByIds); +} + +/** Update a single cached billing in place (e.g. after a local mark-as-paid) without a refetch. */ +export function setCachedBilling(billing: Billing): void { + billingCache.set(billing.id, billing); +} + +/** Clear both caches. Call after mutations that may change billing/reading data. */ +export function invalidateEntityLookupCache(): void { + billingCache.clear(); + readingCache.clear(); +} diff --git a/ui/src/lib/stores/list-store.svelte.ts b/ui/src/lib/stores/list-store.svelte.ts new file mode 100644 index 0000000..209e736 --- /dev/null +++ b/ui/src/lib/stores/list-store.svelte.ts @@ -0,0 +1,66 @@ +interface PaginatedResult { + data: T[]; + hasMore?: boolean; + nextCursor?: string | null; +} + +const STALE_MS = 5 * 60 * 1000; // 5 minutes +const PAGE_SIZE = 100; + +/** Shared cache-with-staleness pattern behind meterGroupsStore/propertiesStore/tenantsStore. */ +export function createListStore( + fetchFn: (params: { limit: number; cursor?: string }) => Promise> +) { + let _data = $state([]); + let _lastFetched = $state(null); + let _isLoading = $state(false); + + return { + get data() { + return _data; + }, + get isLoading() { + return _isLoading; + }, + get isStale() { + return _lastFetched === null || Date.now() - _lastFetched > STALE_MS; + }, + + async fetch(force = false) { + if (!force && !this.isStale && _data.length > 0) { + return _data; + } + + if (_isLoading) { + return _data; + } + + _isLoading = true; + try { + // Page through with cursors at a uniform limit instead of one oversized request, + // matching the pattern used everywhere else. Consumers still get the full list in + // _data — these collections are small, so assembling all pages is cheap. + const all: T[] = []; + let cursor: string | undefined; + do { + const result = await fetchFn({ limit: PAGE_SIZE, cursor }); + all.push(...result.data); + cursor = result.hasMore ? (result.nextCursor ?? undefined) : undefined; + } while (cursor); + _data = all; + _lastFetched = Date.now(); + return _data; + } finally { + _isLoading = false; + } + }, + + invalidate() { + _lastFetched = null; + }, + + async refresh() { + return this.fetch(true); + } + }; +} diff --git a/ui/src/lib/stores/meter-groups.svelte.ts b/ui/src/lib/stores/meter-groups.svelte.ts index 1fddc81..3a93b22 100644 --- a/ui/src/lib/stores/meter-groups.svelte.ts +++ b/ui/src/lib/stores/meter-groups.svelte.ts @@ -1,47 +1,4 @@ import { getMeterGroups } from '$lib/api/meter-groups'; -import type { MeterGroup } from '$lib/types/meter-group.types'; +import { createListStore } from '$lib/stores/list-store.svelte'; -let _data = $state([]); -let _lastFetched = $state(null); -let _isLoading = $state(false); -const STALE_MS = 5 * 60 * 1000; // 5 minutes - -export const meterGroupsStore = { - get data() { - return _data; - }, - get isLoading() { - return _isLoading; - }, - get isStale() { - return _lastFetched === null || Date.now() - _lastFetched > STALE_MS; - }, - - async fetch(force = false) { - if (!force && !this.isStale && _data.length > 0) { - return _data; - } - - if (_isLoading) { - return _data; - } - - _isLoading = true; - try { - const result = await getMeterGroups({ limit: 1000 }); - _data = result.data; - _lastFetched = Date.now(); - return _data; - } finally { - _isLoading = false; - } - }, - - invalidate() { - _lastFetched = null; - }, - - async refresh() { - return this.fetch(true); - } -}; +export const meterGroupsStore = createListStore(getMeterGroups); diff --git a/ui/src/lib/stores/properties.svelte.ts b/ui/src/lib/stores/properties.svelte.ts index 49f6c2a..88a45f4 100644 --- a/ui/src/lib/stores/properties.svelte.ts +++ b/ui/src/lib/stores/properties.svelte.ts @@ -1,47 +1,4 @@ import { getProperties } from '$lib/api/properties'; -import type { Property } from '$lib/types/property.types'; +import { createListStore } from '$lib/stores/list-store.svelte'; -let _data = $state([]); -let _lastFetched = $state(null); -let _isLoading = $state(false); -const STALE_MS = 5 * 60 * 1000; // 5 minutes - -export const propertiesStore = { - get data() { - return _data; - }, - get isLoading() { - return _isLoading; - }, - get isStale() { - return _lastFetched === null || Date.now() - _lastFetched > STALE_MS; - }, - - async fetch(force = false) { - if (!force && !this.isStale && _data.length > 0) { - return _data; - } - - if (_isLoading) { - return _data; - } - - _isLoading = true; - try { - const result = await getProperties({ limit: 1000 }); - _data = result.data; - _lastFetched = Date.now(); - return _data; - } finally { - _isLoading = false; - } - }, - - invalidate() { - _lastFetched = null; - }, - - async refresh() { - return this.fetch(true); - } -}; +export const propertiesStore = createListStore(getProperties); diff --git a/ui/src/lib/stores/tenants.svelte.ts b/ui/src/lib/stores/tenants.svelte.ts index 50e4bc8..d584b0f 100644 --- a/ui/src/lib/stores/tenants.svelte.ts +++ b/ui/src/lib/stores/tenants.svelte.ts @@ -1,47 +1,4 @@ import { getTenants } from '$lib/api/tenants'; -import type { Tenant } from '$lib/types/tenant.types'; +import { createListStore } from '$lib/stores/list-store.svelte'; -let _data = $state([]); -let _lastFetched = $state(null); -let _isLoading = $state(false); -const STALE_MS = 5 * 60 * 1000; // 5 minutes - -export const tenantsStore = { - get data() { - return _data; - }, - get isLoading() { - return _isLoading; - }, - get isStale() { - return _lastFetched === null || Date.now() - _lastFetched > STALE_MS; - }, - - async fetch(force = false) { - if (!force && !this.isStale && _data.length > 0) { - return _data; - } - - if (_isLoading) { - return _data; - } - - _isLoading = true; - try { - const result = await getTenants({ limit: 1000 }); - _data = result.data; - _lastFetched = Date.now(); - return _data; - } finally { - _isLoading = false; - } - }, - - invalidate() { - _lastFetched = null; - }, - - async refresh() { - return this.fetch(true); - } -}; +export const tenantsStore = createListStore(getTenants); diff --git a/ui/src/lib/types/billing.types.ts b/ui/src/lib/types/billing.types.ts index b776d03..17558ef 100644 --- a/ui/src/lib/types/billing.types.ts +++ b/ui/src/lib/types/billing.types.ts @@ -1,9 +1,11 @@ -import type { BaseModel } from './api.types'; +import type { BaseModel, FirestoreTimestamp } from './api.types'; export interface Billing extends BaseModel { property_id: string; previous_reading_id: string; current_reading_id: string; + meter_group_id: string; + billing_period_date: FirestoreTimestamp; payment_status: 'pending' | 'paid'; paid_at?: string; } diff --git a/ui/src/lib/types/photo-settings.types.ts b/ui/src/lib/types/photo-settings.types.ts deleted file mode 100644 index 19a6a35..0000000 --- a/ui/src/lib/types/photo-settings.types.ts +++ /dev/null @@ -1,7 +0,0 @@ -export interface PhotoSettingsResponse { - savePhotos: boolean; -} - -export interface UpsertPhotoSettingsRequest { - savePhotos: boolean; -} diff --git a/ui/src/lib/types/reading.types.ts b/ui/src/lib/types/reading.types.ts index 7a18a03..9d2dd9e 100644 --- a/ui/src/lib/types/reading.types.ts +++ b/ui/src/lib/types/reading.types.ts @@ -5,7 +5,6 @@ export interface Reading extends BaseModel { property_id: string; reading_amount: number; reading_date: FirestoreTimestamp; - image_url?: string; meter_version: number; } @@ -14,7 +13,6 @@ export interface CreateReadingRequest { property_id: string; reading_amount: number; reading_date: FirestoreTimestamp | string; - image_url?: string; } export interface CreateSeedReadingRequest { @@ -22,12 +20,10 @@ export interface CreateSeedReadingRequest { property_id: string; reading_amount: number; reading_date: FirestoreTimestamp | string; - image_url?: string; } export interface UpdateReadingRequest { meter_group_id?: string; reading_amount?: number; reading_date?: FirestoreTimestamp | string; - image_url?: string; } diff --git a/ui/src/lib/types/reports.types.ts b/ui/src/lib/types/reports.types.ts index 1d3e484..b2399f8 100644 --- a/ui/src/lib/types/reports.types.ts +++ b/ui/src/lib/types/reports.types.ts @@ -56,3 +56,10 @@ export interface ReportQueryParams { meterGroupId?: string; propertyId?: string; } + +export interface CombinedReportsResponse { + summary: ReportSummary; + consumption: ConsumptionReport; + billingTrends: BillingTrendsReport; + collectionStatus: CollectionStatusReport; +} diff --git a/ui/src/lib/utils/billing-cycle.util.ts b/ui/src/lib/utils/billing-cycle.util.ts index c8d7b21..2143835 100644 --- a/ui/src/lib/utils/billing-cycle.util.ts +++ b/ui/src/lib/utils/billing-cycle.util.ts @@ -1,27 +1,28 @@ import type { BillingCycle } from '$lib/types/billing-cycle.types'; import type { Billing } from '$lib/types/billing.types'; +import { billAmount, sumMoney } from './money'; export function getCyclePaidAmount(cycle: BillingCycle, billings: Map): number { - let total = 0; + const amounts: number[] = []; for (const [billingId, consumption] of Object.entries(cycle.billing_ids)) { const billing = billings.get(billingId); if (billing?.payment_status === 'paid') { - total += consumption * cycle.billing_rate; + amounts.push(billAmount(consumption, cycle.billing_rate)); } } - return total; + return sumMoney(amounts); } export function getCycleOutstandingAmount( cycle: BillingCycle, billings: Map ): number { - let total = 0; + const amounts: number[] = []; for (const [billingId, consumption] of Object.entries(cycle.billing_ids)) { const billing = billings.get(billingId); if (billing?.payment_status !== 'paid') { - total += consumption * cycle.billing_rate; + amounts.push(billAmount(consumption, cycle.billing_rate)); } } - return total; + return sumMoney(amounts); } diff --git a/ui/src/lib/utils/format.ts b/ui/src/lib/utils/format.ts index f483708..8c70493 100644 --- a/ui/src/lib/utils/format.ts +++ b/ui/src/lib/utils/format.ts @@ -1,4 +1,6 @@ import { format, parse } from 'date-fns'; +import type { FirestoreTimestamp } from '$lib/types/api.types'; +import { toDate } from '$lib/utils/timestamp'; export function formatCurrency(amount: number): string { return new Intl.NumberFormat('en-PH', { @@ -25,6 +27,10 @@ export function formatDate(date: Date): string { return format(date, 'MMM d, yyyy'); } +export function formatFirestoreDate(timestamp: FirestoreTimestamp | string): string { + return formatDate(toDate(timestamp)); +} + export function formatLongDate(date: Date): string { return format(date, 'MMMM d, yyyy'); } diff --git a/ui/src/lib/utils/true-reading.test.ts b/ui/src/lib/utils/true-reading.test.ts index 9ca0c1b..54e2dc4 100644 --- a/ui/src/lib/utils/true-reading.test.ts +++ b/ui/src/lib/utils/true-reading.test.ts @@ -56,7 +56,6 @@ describe('true-reading (submeter, post-reset cumulative offset)', () => { reading_amount: 5.5, reading_date: '' as any, meter_version: 2, - image_url: null as any, created_at: '' as any, updated_at: '' as any, is_deleted: false, diff --git a/ui/src/routes/(app)/billings/+page.svelte b/ui/src/routes/(app)/billings/+page.svelte index cb0a653..df4de75 100644 --- a/ui/src/routes/(app)/billings/+page.svelte +++ b/ui/src/routes/(app)/billings/+page.svelte @@ -11,17 +11,23 @@ } from '$lib/api/billing-cycles'; import { getBillings, createBilling, updateBilling, softDeleteBilling } from '$lib/api/billings'; import { getReadings } from '$lib/api/readings'; + import { + getReadingsByIds, + invalidateEntityLookupCache, + setCachedBilling + } from '$lib/stores/entity-lookup-cache'; import { getProperties } from '$lib/api/properties'; import { getMeterGroups } from '$lib/api/meter-groups'; import type { BillingCycle, UpdateBillingCycleRequest } from '$lib/types/billing-cycle.types'; import type { Billing, UpdateBillingRequest } from '$lib/types/billing.types'; import type { Reading } from '$lib/types/reading.types'; import type { Property, MeterGroupEntry } from '$lib/types/property.types'; - import type { MeterGroup, MeterGroupVersionEntry } from '$lib/types/meter-group.types'; + import type { MeterGroup } from '$lib/types/meter-group.types'; import type { PaginatedResult } from '$lib/types/api.types'; import { formatDate, formatCurrency, formatReading, getReadingUnit } from '$lib/utils/format'; import { billAmount, sumMoney } from '$lib/utils/money'; import { toDate } from '$lib/utils/timestamp'; + import { trueReading } from '$lib/utils/true-reading'; import { getUtilityTypeBadgeClasses } from '$lib/utils/utility-colors'; import EmptyState from '$lib/components/shared/EmptyState.svelte'; import TableSkeleton from '$lib/components/shared/TableSkeleton.svelte'; @@ -34,8 +40,20 @@ let cycles = $state([]); let billings: SvelteMap = new SvelteMap(); + // allBillings stays a full load: the payment-status filter (getCyclePaymentStatus) filters + // ALL cycles by whether every billing under them is paid, which needs every billing's status + // — there's no scoped server query for that. Readings, by contrast, are fetched scoped below. let allBillings = $state([]); - let readings = $state([]); + // Readings are no longer bulk-loaded. Scoped, purpose-specific reading sets replace the old + // full `readings` array: + // - cycleFormReadings: the selected meter group's readings (discovery / gap-fill / overrides) + // - editReadings: the edited billing's property's readings (edit modal selects) + // - stragglerReadings: the straggler property's readings on the cycle's meter group + // - readingMap: prev/current readings for the billings of expanded / printed cycles + let cycleFormReadings = $state([]); + let editReadings = $state([]); + let stragglerReadings = $state([]); + let readingMap: SvelteMap = new SvelteMap(); let properties = $state([]); let meterGroups = $state([]); let isLoading = $state(false); @@ -112,47 +130,8 @@ const round = (value: number, decimals = 2) => Math.round(value * Math.pow(10, decimals)) / Math.pow(10, decimals); - // Version-aware "true reading" — mirrors the API's billing-cycle validator so consumption - // previews stay correct across meter resets (cumulative offset of prior versions + raw amount). - function getCumulativeOffset( - versions: Record | undefined, - version: number - ): number { - if (!versions) return 0; - let offset = 0; - for (let v = 1; v < version; v++) { - const versionData = versions[String(v)]; - if (versionData) offset += versionData.last_reading; - } - return offset; - } - - function getVersionsSource( - meterGroup: MeterGroup | undefined, - property: Property | undefined, - meterGroupId: string - ): Record | undefined { - let versionsSource = meterGroup?.versions; - if (property) { - const entry = Object.values(property.meter_groups).find( - (e): e is MeterGroupEntry => typeof e === 'object' && e?.meter_group_id === meterGroupId - ); - if (entry && !entry.is_main_meter) { - versionsSource = entry.versions; - } - } - return versionsSource; - } - - function trueReading( - reading: Reading, - meterGroup: MeterGroup | undefined, - property: Property | undefined - ): number { - const versionsSource = getVersionsSource(meterGroup, property, reading.meter_group_id); - return getCumulativeOffset(versionsSource, reading.meter_version ?? 1) + reading.reading_amount; - } - + // Version-aware "true reading" (imported from $lib/utils/true-reading) mirrors the + // API's billing-cycle validator so consumption previews stay correct across meter resets. function readingConsumption( currReading: Reading, prevReading: Reading | undefined, @@ -165,9 +144,9 @@ } const billingCycleReadings = $derived.by(() => { - if (!readings || !cycleFormMeterGroup || !cycleFormEndDate) return []; + if (!cycleFormReadings.length || !cycleFormMeterGroup || !cycleFormEndDate) return []; const endDate = new Date(cycleFormEndDate); - return readings.filter((reading) => { + return cycleFormReadings.filter((reading) => { if (reading.meter_group_id !== cycleFormMeterGroup) return false; const readingDate = toDate(reading.reading_date); return ( @@ -177,18 +156,25 @@ }); }); - // Filter readings for edit modal based on selected property's meter groups - const editModalReadings = $derived.by(() => { - if (!readings || !editData.property_id) return readings || []; - const selectedProp = properties.find((p) => p.id === editData.property_id); - if (!selectedProp) return readings; - - const meterGroupIds = Object.values(selectedProp.meter_groups || {}) - .filter((e): e is any => e !== undefined && e !== null) - .map((e) => e.meter_group_id); - return readings.filter( - (r) => meterGroupIds.includes(r.meter_group_id) && r.property_id === editData.property_id - ); + // Edit-modal reading options — fetched scoped to the edited billing's property (a property's + // readings are all for its own meter groups, so propertyId scoping is sufficient). + $effect(() => { + const propertyId = editData.property_id; + if (!propertyId) { + editReadings = []; + return; + } + let cancelled = false; + getReadings({ propertyId, limit: 100 }) + .then((res) => { + if (!cancelled) editReadings = res.data; + }) + .catch(() => { + if (!cancelled) editReadings = []; + }); + return () => { + cancelled = true; + }; }); // Get utility type for cycle form meter group @@ -233,20 +219,21 @@ isLoading = true; error = ''; try { - const [cyclesData, billingsData, readingsData, propertiesResult, meterGroupsResult] = - await Promise.all([ - fetchAllPages((cursor) => getBillingCycles({ limit: 100, cursor })), - fetchAllPages((cursor) => getBillings({ limit: 100, cursor })), - fetchAllPages((cursor) => getReadings({ limit: 100, cursor })), - getProperties({ limit: 100 }), - getMeterGroups({ limit: 100 }) - ]); + // Readings are no longer bulk-loaded here — resolved scoped/on-demand instead. + // Clear the ID lookup cache so a reload after a mutation can't serve stale readings. + invalidateEntityLookupCache(); + const [cyclesData, billingsData, propertiesResult, meterGroupsResult] = await Promise.all([ + fetchAllPages((cursor) => getBillingCycles({ limit: 100, cursor })), + fetchAllPages((cursor) => getBillings({ limit: 100, cursor })), + getProperties({ limit: 100 }), + getMeterGroups({ limit: 100 }) + ]); cycles = cyclesData; allBillings = billingsData; - readings = readingsData; properties = propertiesResult.data; meterGroups = meterGroupsResult.data; billings.clear(); + readingMap.clear(); currentPage = 1; } catch (err) { error = err instanceof Error ? err.message : 'Failed to load data'; @@ -271,6 +258,18 @@ if (cycle) { const cycleBillings = allBillings.filter((b) => b.id in cycle.billing_ids); billings.set(cycleId, cycleBillings); + await resolveReadingsFor(cycleBillings); + } + } + + // Resolve (and cache) the previous/current readings for a set of billings into readingMap, + // so the expanded billing table and receipt printing can show reading amounts without the + // old full readings load. + async function resolveReadingsFor(billingList: Billing[]) { + const ids = billingList.flatMap((b) => [b.previous_reading_id, b.current_reading_id]); + const resolved = await getReadingsByIds(ids); + for (const [id, reading] of resolved) { + readingMap.set(id, reading); } } @@ -318,6 +317,10 @@ bills[idx] = billing; } } + + // Keep the cross-page entity-lookup cache (Dashboard) in sync — otherwise + // it still holds the pre-mark-as-paid copy until its TTL/invalidation clears. + setCachedBilling(billing); } } catch (err) { error = err instanceof Error ? err.message : 'Failed to mark as paid'; @@ -332,7 +335,7 @@ const endDate = new Date(cycleFormEndDate); const base = allBillings .filter((billing) => { - const currReading = readingMap.get(billing.current_reading_id); + const currReading = cycleFormReadingMap.get(billing.current_reading_id); if (!currReading) return false; if (currReading.meter_group_id !== cycleFormMeterGroup) return false; const readingDate = toDate(currReading.reading_date); @@ -342,8 +345,8 @@ ); }) .map((billing) => { - const currReading = readingMap.get(billing.current_reading_id)!; - const prevReading = readingMap.get(billing.previous_reading_id); + const currReading = cycleFormReadingMap.get(billing.current_reading_id)!; + const prevReading = cycleFormReadingMap.get(billing.previous_reading_id); const property = properties.find((p) => p.id === billing.property_id); // Version-aware true-reading diff — matches the API's billing-cycle validator so the @@ -364,8 +367,8 @@ return base.map((entry) => { const override = cycleFormOverrideSelections.get(entry.propertyId); if (!override) return entry; - const prevReading = readingMap.get(override.prev_reading_id); - const currReading = readingMap.get(override.curr_reading_id); + const prevReading = cycleFormReadingMap.get(override.prev_reading_id); + const currReading = cycleFormReadingMap.get(override.curr_reading_id); if (!prevReading || !currReading) return entry; const property = properties.find((p) => p.id === entry.propertyId); const consumption = readingConsumption(currReading, prevReading, property); @@ -395,7 +398,7 @@ .map((property) => ({ propertyId: property.id, propertyName: property.room_name, - availableReadings: readings.filter( + availableReadings: cycleFormReadings.filter( (r) => r.meter_group_id === cycleFormMeterGroup && r.property_id === property.id && @@ -407,8 +410,24 @@ .filter((g) => g.availableReadings.length >= 2); } - function runDiscovery() { + // Discovery needs the selected meter group's readings — fetch them scoped (one meter group, + // small) before computing, replacing the old full readings load. + async function runDiscovery() { error = ''; + if (!cycleFormMeterGroup || !cycleFormEndDate) { + cycleFormDiscoveredBillings = []; + cycleFormGapProperties = []; + return; + } + try { + cycleFormReadings = await fetchAllPages((cursor) => + getReadings({ meterGroupId: cycleFormMeterGroup, limit: 100, cursor }) + ); + } catch (err) { + error = err instanceof Error ? err.message : 'Failed to load readings for this meter group'; + cycleFormReadings = []; + return; + } cycleFormDiscoveredBillings = discoverBillings(); cycleFormGapProperties = discoverGapProperties(); } @@ -419,6 +438,7 @@ cycleFormEndDate = ''; cycleFormDueDate = ''; cycleFormRate = 0; + cycleFormReadings = []; cycleFormDiscoveredBillings = []; cycleFormGapProperties = []; cycleFormTotalConsumption = 0; @@ -431,24 +451,24 @@ billOcrRawAmount = null; } - function autoCalculateCycleDates() { + async function autoCalculateCycleDates() { if (!cycleFormMeterGroup) return; - // Find the last billing cycle for this meter group - const meterGroupCycles = cycles - .filter((c) => { - const cycleBillingIds = Object.keys(c.billing_ids); - const cycleBillings = allBillings.filter((b) => cycleBillingIds.includes(b.id)); - if (cycleBillings.length === 0) return false; - const firstBilling = cycleBillings[0]; - const firstReading = readingMap.get(firstBilling.current_reading_id); - return firstReading?.meter_group_id === cycleFormMeterGroup; - }) - .sort( - (a, b) => - new Date(toDate(b.billing_end_date)).getTime() - - new Date(toDate(a.billing_end_date)).getTime() - ); + // Find the last billing cycle for this meter group via a scoped query — the cycle's own + // denormalized meter_group_id makes this a precise lookup, correct regardless of how many + // total cycles exist (the old "scan the loaded cycles" approach silently reset dates to + // the current month once cycle volume exceeded a page). + let meterGroupCycles: BillingCycle[] = []; + try { + const res = await getBillingCycles({ meterGroupId: cycleFormMeterGroup, limit: 100 }); + meterGroupCycles = res.data + .slice() + .sort( + (a, b) => toDate(b.billing_end_date).getTime() - toDate(a.billing_end_date).getTime() + ); + } catch { + // Leave meterGroupCycles empty → falls back to the current-month default below. + } if (meterGroupCycles.length === 0) { // No previous cycle, use current month @@ -545,8 +565,8 @@ // Gap-fill billings don't exist yet — create them first, then fold their // consumption into the same billing_ids map the cycle expects. for (const gap of gapsToFill) { - const prevReading = readings.find((r) => r.id === gap.selectedPrevReadingId); - const currReading = readings.find((r) => r.id === gap.selectedCurrReadingId); + const prevReading = cycleFormReadings.find((r) => r.id === gap.selectedPrevReadingId); + const currReading = cycleFormReadings.find((r) => r.id === gap.selectedCurrReadingId); if (!prevReading || !currReading) continue; const newBilling = await createBilling({ @@ -680,20 +700,40 @@ ); }); + // Fetch the straggler property's readings on this cycle's meter group, scoped, when the + // selected property changes — replaces filtering the old full readings array. + $effect(() => { + const propertyId = stragglerPropertyId; + const meterGroupId = stragglerCycle ? getCycleMeterGroupId(stragglerCycle) : null; + if (!propertyId || !meterGroupId) { + stragglerReadings = []; + return; + } + let cancelled = false; + getReadings({ propertyId, meterGroupId, limit: 100 }) + .then((res) => { + if (!cancelled) stragglerReadings = res.data; + }) + .catch(() => { + if (!cancelled) stragglerReadings = []; + }); + return () => { + cancelled = true; + }; + }); + // Readings for the selected straggler property on this cycle's meter group, oldest first. const stragglerAvailableReadings = $derived.by(() => { if (!stragglerCycle || !stragglerPropertyId) return []; - const meterGroupId = getCycleMeterGroupId(stragglerCycle); - if (!meterGroupId) return []; - return readings - .filter((r) => r.property_id === stragglerPropertyId && r.meter_group_id === meterGroupId) + return stragglerReadings + .slice() .sort((a, b) => toDate(a.reading_date).getTime() - toDate(b.reading_date).getTime()); }); const stragglerConsumptionPreview = $derived.by(() => { - const curr = readings.find((r) => r.id === stragglerCurrReadingId); + const curr = stragglerReadings.find((r) => r.id === stragglerCurrReadingId); if (!curr) return 0; - const prev = readings.find((r) => r.id === stragglerPrevReadingId); + const prev = stragglerReadings.find((r) => r.id === stragglerPrevReadingId); const property = properties.find((p) => p.id === stragglerPropertyId); return readingConsumption(curr, prev, property); }); @@ -710,8 +750,8 @@ } isAddingStraggler = true; try { - const currReading = readings.find((r) => r.id === stragglerCurrReadingId); - const prevReading = readings.find((r) => r.id === stragglerPrevReadingId); + const currReading = stragglerReadings.find((r) => r.id === stragglerCurrReadingId); + const prevReading = stragglerReadings.find((r) => r.id === stragglerPrevReadingId); const property = properties.find((p) => p.id === stragglerPropertyId); if (!currReading) throw new Error('Selected current reading not found'); @@ -755,7 +795,9 @@ } const billingMap = $derived.by(() => new Map(allBillings.map((b) => [b.id, b]))); - const readingMap = $derived.by(() => new Map(readings.map((r) => [r.id, r]))); + // Lookup over the meter-group-scoped reading set used by cycle discovery/gap-fill. + // (readingMap above is the on-demand cache for expanded/printed cycles.) + const cycleFormReadingMap = $derived.by(() => new Map(cycleFormReadings.map((r) => [r.id, r]))); const meterGroupMap = $derived.by(() => new Map(meterGroups.map((m) => [m.id, m]))); function getCycleUtilityType(cycle: BillingCycle): string { @@ -823,19 +865,27 @@ if (!dateValue) return false; const cycleDate = toDate(dateValue); if (filterDateFrom && cycleDate < new SvelteDate(filterDateFrom)) return false; - if (filterDateTo && cycleDate > new SvelteDate(filterDateTo)) return false; + if (filterDateTo) { + // filterDateTo parses as UTC midnight — compare against the end of that + // day (23:59:59.999 UTC), not its start, so a cycle dated exactly on the + // selected end date isn't excluded by its own time-of-day component. + const toCutoff = new SvelteDate(filterDateTo); + toCutoff.setUTCHours(23, 59, 59, 999); + if (cycleDate > toCutoff) return false; + } } return true; }) .sort( - (a, b) => - toDate(b.billing_start_date).getTime() - toDate(a.billing_start_date).getTime() + (a, b) => toDate(b.billing_start_date).getTime() - toDate(a.billing_start_date).getTime() ); }); const cyclesPageSize = 20; let currentPage = $state(1); - const totalPages = $derived.by(() => Math.max(1, Math.ceil(filteredCycles.length / cyclesPageSize))); + const totalPages = $derived.by(() => + Math.max(1, Math.ceil(filteredCycles.length / cyclesPageSize)) + ); const pagedCycles = $derived.by(() => { const start = (currentPage - 1) * cyclesPageSize; return filteredCycles.slice(start, start + cyclesPageSize); @@ -868,7 +918,7 @@ const cycleBillings = billings.get(firstCycle.id) || []; if (cycleBillings.length > 0) { const firstBilling = cycleBillings[0]; - const firstReading = readings.find((r) => r.id === firstBilling.current_reading_id); + const firstReading = readingMap.get(firstBilling.current_reading_id); if (firstReading) { const meterGroup = meterGroups.find((m) => m.id === firstReading.meter_group_id); utilityType = meterGroup?.utility_type || 'electricity'; @@ -888,8 +938,8 @@ const cycleBillings = billings.get(cycle.id) || []; for (const billing of cycleBillings) { const property = properties.find((p) => p.id === billing.property_id); - const currReading = readings.find((r) => r.id === billing.current_reading_id); - const prevReading = readings.find((r) => r.id === billing.previous_reading_id); + const currReading = readingMap.get(billing.current_reading_id); + const prevReading = readingMap.get(billing.previous_reading_id); const consumption = cycle.billing_ids[billing.id] ?? 0; const billRate = cycle.billing_rate; @@ -1078,9 +1128,9 @@ - {getCycleMeterGroupName(cycle)} + {getCycleMeterGroupName(cycle)} {cycleUtilityType}{cycleUtilityType}
@@ -1619,266 +1670,270 @@
-
-
-
Consumption
-
- {cycle.billing_consumption.toLocaleString()} - {getReadingUnit(getCycleUtilityType(cycle))} -
+
+
+
Consumption
+
+ {cycle.billing_consumption.toLocaleString()} + {getReadingUnit(getCycleUtilityType(cycle))}
-
-
Rate
-
- ₱{cycle.billing_rate.toFixed(2)}/{getReadingUnit( - getCycleUtilityType(cycle) - )} -
+
+
+
Rate
+
+ ₱{cycle.billing_rate.toFixed(2)}/{getReadingUnit( + getCycleUtilityType(cycle) + )}
-
-
Total Amount
-
- {formatCurrency( - billAmount(cycle.billing_consumption, cycle.billing_rate) - )} -
+
+
+
Total Amount
+
+ {formatCurrency( + billAmount(cycle.billing_consumption, cycle.billing_rate) + )}
-
-
Currently Paid
-
- {formatCurrency(getCyclePaidAmount(cycle))} -
+
+
+
Currently Paid
+
+ {formatCurrency(getCyclePaidAmount(cycle))}
- -
-
- - - - -
+
+ +
+
+ + + +
+
- - {#if expandedCycleId === cycle.id} -
- {#if cycleBillings?.length === 0} -
- -
- {:else if cycleBillings} -
- - - - - - - - - - - - - - {#each cycleBillings || [] as billing (billing.id)} - {@const billingProperty = properties.find( - (p) => p.id === billing.property_id - )} - {@const cycleMeterGroup = cycle.meter_group_id - ? meterGroupMap.get(cycle.meter_group_id) - : undefined} - {@const meterEntry = cycle.meter_group_id - ? cycleMeterGroup?.utility_type === 'water' - ? billingProperty?.meter_groups.water - : billingProperty?.meter_groups.electricity - : null} - {@const isMainMeter = - typeof meterEntry === 'string' - ? false - : (meterEntry?.is_main_meter ?? false)} - {@const previousReading = readingMap.get(billing.previous_reading_id)} - {@const currentReading = readingMap.get(billing.current_reading_id)} - {@const previousMeterGroup = previousReading - ? meterGroupMap.get(previousReading.meter_group_id) - : undefined} - {@const currentMeterGroup = currentReading - ? meterGroupMap.get(currentReading.meter_group_id) - : undefined} - - - - - - + + {/each} + +
PropertyPrevious ReadingCurrent Reading - Consumption - AmountStatusActions
-
- {billingProperty?.room_name ?? 'Unknown Property'} -
- 		- {#if previousReading} - {formatReading( - trueReading( - previousReading, - previousMeterGroup, - billingProperty - ), - previousMeterGroup?.utility_type || 'electricity' - )} - {:else} - N/A - {/if} - - {#if currentReading} - {formatReading( - trueReading(currentReading, currentMeterGroup, billingProperty), - currentMeterGroup?.utility_type || 'electricity' - )} - {:else} - N/A - {/if} - - {#if currentReading} - {(cycle.billing_ids[billing.id] ?? 0).toLocaleString()} - {getReadingUnit(currentMeterGroup?.utility_type || 'electricity')} - {:else} - N/A - {/if} - - {formatCurrency( - (cycle.billing_ids[billing.id] ?? 0) * cycle.billing_rate + + {#if expandedCycleId === cycle.id} +
+ {#if cycleBillings?.length === 0} +
+ +
+ {:else if cycleBillings} +
+ + + + + + + + + + + + + + {#each cycleBillings || [] as billing (billing.id)} + {@const billingProperty = properties.find( + (p) => p.id === billing.property_id + )} + {@const cycleMeterGroup = cycle.meter_group_id + ? meterGroupMap.get(cycle.meter_group_id) + : undefined} + {@const meterEntry = cycle.meter_group_id + ? cycleMeterGroup?.utility_type === 'water' + ? billingProperty?.meter_groups.water + : billingProperty?.meter_groups.electricity + : null} + {@const isMainMeter = + typeof meterEntry === 'string' + ? false + : (meterEntry?.is_main_meter ?? false)} + {@const previousReading = readingMap.get(billing.previous_reading_id)} + {@const currentReading = readingMap.get(billing.current_reading_id)} + {@const previousMeterGroup = previousReading + ? meterGroupMap.get(previousReading.meter_group_id) + : undefined} + {@const currentMeterGroup = currentReading + ? meterGroupMap.get(currentReading.meter_group_id) + : undefined} + {@const meterWasReset = + previousReading && + currentReading && + (previousReading.meter_version ?? 1) !== + (currentReading.meter_version ?? 1)} + + + - - + + + + + - - {/each} - -
PropertyPrevious ReadingCurrent Reading + Consumption + AmountStatusActions
+
+ {billingProperty?.room_name ?? 'Unknown Property'} +
+ 		+ {#if previousReading} + {formatReading( + previousReading.reading_amount, + previousMeterGroup?.utility_type || 'electricity' )} - - - -
- {#if billing.payment_status === 'pending'} - - {/if} -
+ {#if currentReading} + {formatReading( + currentReading.reading_amount, + currentMeterGroup?.utility_type || 'electricity' + )} + {#if meterWasReset} + meter reset - - + {/if} + {:else} + N/A + {/if} + + {#if currentReading} + {(cycle.billing_ids[billing.id] ?? 0).toLocaleString()} + {getReadingUnit(currentMeterGroup?.utility_type || 'electricity')} + {:else} + N/A + {/if} + + {formatCurrency( + (cycle.billing_ids[billing.id] ?? 0) * cycle.billing_rate + )} + + + +
+ {#if billing.payment_status === 'pending'} -
-
-
- {:else} - - {/if} -
- {/if} - - {/each} + {/if} + + + +
+
+ {:else} + + {/if} +
+ {/if} +
+ {/each} {#if totalPages > 1}
@@ -1939,7 +1994,7 @@ bind:value={editData.previous_reading_id} class="mt-1 w-full rounded border border-gray-300 px-3 py-2" > - {#each editModalReadings as reading (reading.id)} + {#each editReadings as reading (reading.id)}
@@ -784,7 +784,7 @@ > - {formatDate(toDate(tenant.tenant_start_date))} + {formatFirestoreDate(tenant.tenant_start_date)} {/each} @@ -851,7 +851,7 @@ )} - {formatDate(toDate(reading.reading_date))} + {formatFirestoreDate(reading.reading_date)} {/each} @@ -940,9 +940,9 @@ {prevReading && currReading - ? formatDate(toDate(prevReading.reading_date)) + + ? formatFirestoreDate(prevReading.reading_date) + ' - ' + - formatDate(toDate(currReading.reading_date)) + formatFirestoreDate(currReading.reading_date) : 'N/A'} diff --git a/ui/src/routes/(app)/properties/archive/+page.svelte b/ui/src/routes/(app)/properties/archive/+page.svelte index b010d12..38dbfe8 100644 --- a/ui/src/routes/(app)/properties/archive/+page.svelte +++ b/ui/src/routes/(app)/properties/archive/+page.svelte @@ -1,22 +1,17 @@
-
- -
+
diff --git a/ui/src/routes/(app)/readings/+page.svelte b/ui/src/routes/(app)/readings/+page.svelte index ca5318a..3878ffb 100644 --- a/ui/src/routes/(app)/readings/+page.svelte +++ b/ui/src/routes/(app)/readings/+page.svelte @@ -16,17 +16,10 @@ import type { MeterGroup } from '$lib/types/meter-group.types'; import type { Property } from '$lib/types/property.types'; import type { PaginatedResult } from '$lib/types/api.types'; - import { formatDate, formatLongDate, formatReading, getReadingUnit } from '$lib/utils/format'; + import { formatFirestoreDate, formatLongDate, formatReading } from '$lib/utils/format'; import { toDate } from '$lib/utils/timestamp'; - import { uploadToStorage } from '$lib/utils/firebase-storage'; import { compressImage } from '$lib/utils/image-compression'; - import { getPhotoSettings } from '$lib/api/photo-settings'; - import { - trueReading, - resolveCurrentVersion, - getVersionsSource, - getCumulativeOffset - } from '$lib/utils/true-reading'; + import { resolveCurrentVersion, getVersionsSource } from '$lib/utils/true-reading'; import EmptyState from '$lib/components/shared/EmptyState.svelte'; import TableSkeleton from '$lib/components/shared/TableSkeleton.svelte'; import EditModal from '$lib/components/shared/EditModal.svelte'; @@ -112,18 +105,8 @@ let isUpdating = $state(false); - // Defaults to false (don't persist photos to Storage) until loaded — matches the - // backend default so there's no window where an unloaded setting reads as "save". - let savePhotos = $state(false); - onMount(async () => { await loadData(); - try { - const settings = await getPhotoSettings(); - savePhotos = settings.savePhotos; - } catch { - // Keep the safe default (don't persist) if the setting fails to load. - } }); async function applyFilters() { @@ -304,9 +287,7 @@ reading_date: { _seconds: Math.floor(new Date(manualReadingForm.reading_date).getTime() / 1000), _nanoseconds: 0 - }, - image_url: - savePhotos && manualReadingForm.image_url ? manualReadingForm.image_url : undefined + } } as any; const isSeed = await shouldSeedReading( @@ -344,7 +325,7 @@ // Compress image to avoid "request entity too large" errors const compressedDataUrl = await compressImage(file, 800, 0.7); - // Show preview and run Suggest immediately — don't wait for Storage + // Photo is only ever used transiently for OCR suggest — never persisted. row.data_url = compressedDataUrl; row.image_url = compressedDataUrl; } catch (err) { @@ -356,19 +337,6 @@ // Auto-suggest a reading value from the photo — no separate Suggest button. await handleSuggestReading(rowIndex); - - // Silently upgrade to a persistent Storage URL in the background — only when the - // user has opted in via Photo Settings. Otherwise the data URL stays in-memory - // only, long enough to have been OCR'd above, and is never persisted. - if (row.data_url && savePhotos) { - uploadToStorage(file, `readings/${Date.now()}_${file.name}`) - .then((url) => { - row.image_url = url; - }) - .catch(() => { - /* keep data URL */ - }); - } } async function handleManualImageUpload(file: File | null) { @@ -395,18 +363,6 @@ } catch (err) { error = err instanceof Error ? err.message : 'Failed to suggest reading'; } - - // Silently upgrade to a persistent Storage URL in the background — only when the - // user has opted in via Photo Settings. - if (savePhotos) { - uploadToStorage(file, `readings/${Date.now()}_${file.name}`) - .then((url) => { - manualReadingForm.image_url = url; - }) - .catch(() => { - /* keep data URL */ - }); - } } async function handleCreateBatch() { @@ -435,10 +391,7 @@ reading_date: { _seconds: Math.floor(dateObj.getTime() / 1000), _nanoseconds: 0 - }, - // Only attach the photo when the user has opted into saving it — - // otherwise it was only ever used in-memory for the Suggest button. - image_url: savePhotos && row.image_url ? row.image_url : undefined + } })); const result = await createReadingsBatch(readingsData); @@ -667,11 +620,13 @@ row.property, selectedMeterGroup )} - {@const offset = getCumulativeOffset(versionsSource, version)} - {@const unit = getReadingUnit(selectedMg?.utility_type || 'electricity')} + {@const resetInfo = + version > 1 ? versionsSource?.[String(version - 1)] : undefined}

- True total: {(offset + row.reading_amount).toLocaleString()} - {unit} + Meter v{version} + {#if resetInfo} + (reset {formatFirestoreDate(resetInfo.reset_at)} from {resetInfo.last_reading.toLocaleString()}) + {/if}

{/if} @@ -884,7 +839,7 @@
{#if isLoading} - + {:else if readings.data.length === 0}
@@ -916,8 +871,7 @@ Property Meter Group Reading - True Total - Photo + Meter Cycle Date Created Actions @@ -927,6 +881,14 @@ {#each readings.data as item (item.id)} {@const itemProperty = propertyMap.get(item.property_id)} {@const itemMeterGroup = meterGroupMap.get(item.meter_group_id)} + {@const itemMeterVersion = item.meter_version ?? 1} + {@const itemVersionsSource = getVersionsSource( + itemMeterGroup, + itemProperty, + item.meter_group_id + )} + {@const itemResetInfo = + itemMeterVersion > 1 ? itemVersionsSource?.[String(itemMeterVersion - 1)] : undefined} {formatReading(item.reading_amount, itemMeterGroup?.utility_type || 'electricity')} - - {trueReading(item, itemMeterGroup, itemProperty).toLocaleString()} - {getReadingUnit(itemMeterGroup?.utility_type || 'electricity')} - - - {#if item.image_url} - - - Meter reading - - {:else} - No image + + v{itemMeterVersion} + {#if itemResetInfo} + reset {formatFirestoreDate(itemResetInfo.reset_at)}, prior meter ended at {itemResetInfo.last_reading.toLocaleString()} {/if} - {formatDate(toDate(item.reading_date))} - {formatDate(toDate(item.created_at))} + {formatFirestoreDate(item.reading_date)} + {formatFirestoreDate(item.created_at)} { diff --git a/ui/src/routes/(app)/readings/archive/+page.svelte b/ui/src/routes/(app)/readings/archive/+page.svelte index a836a74..472791d 100644 --- a/ui/src/routes/(app)/readings/archive/+page.svelte +++ b/ui/src/routes/(app)/readings/archive/+page.svelte @@ -2,24 +2,24 @@ import { onMount } from 'svelte'; import { getReadings, restoreReading, clearCache } from '$lib/api/readings'; import { getMeterGroups } from '$lib/api/meter-groups'; - import type { Reading } from '$lib/types/reading.types'; import type { MeterGroup } from '$lib/types/meter-group.types'; - import type { PaginatedResult } from '$lib/types/api.types'; - import { formatDate } from '$lib/utils/format'; - import { toDate } from '$lib/utils/timestamp'; + import { formatFirestoreDate } from '$lib/utils/format'; import ArchivePageTemplate from '$lib/components/shared/ArchivePageTemplate.svelte'; - import { Trash2 } from 'lucide-svelte'; + import ClearCacheButton from '$lib/components/shared/ClearCacheButton.svelte'; + import { createArchivePageState } from '$lib/stores/archive-page.svelte'; - let data = $state>({ - data: [], - nextCursor: null, - hasMore: false - }); let meterGroups = $state>(new Map()); - let isLoading = $state(false); - let error = $state(''); - let restoringId = $state(null); - let isClearing = $state(false); + + const page = createArchivePageState({ + listFn: getReadings, + restoreFn: restoreReading, + clearCacheFn: clearCache, + entityLabel: 'reading', + loadExtra: async () => { + const result = await getMeterGroups({ limit: 100 }); + meterGroups = new Map(result.data.map((mg: MeterGroup) => [mg.id, mg.meter_name])); + } + }); const columns = [ { @@ -35,79 +35,26 @@ { key: 'reading_date', label: 'Reading Date', - format: (v: any) => formatDate(toDate(v)) + format: (v: any) => formatFirestoreDate(v) } ]; onMount(async () => { - await loadData(); + await page.loadData(); }); - - async function loadData() { - isLoading = true; - error = ''; - try { - const [readingsResult, meterGroupsResult] = await Promise.all([ - getReadings({ limit: 100, archived: true }), - getMeterGroups({ limit: 100 }) - ]); - - // Build meter group name map - meterGroups = new Map(meterGroupsResult.data.map((mg: MeterGroup) => [mg.id, mg.meter_name])); - - data = readingsResult; - } catch (err) { - error = err instanceof Error ? err.message : 'Failed to load archived readings'; - } finally { - isLoading = false; - } - } - - async function handleRestore(id: string) { - restoringId = id; - try { - await restoreReading(id); - await loadData(); - } catch (err) { - error = err instanceof Error ? err.message : 'Failed to restore reading'; - } finally { - restoringId = null; - } - } - - async function handleClearCache() { - isClearing = true; - try { - const result = await clearCache(); - error = result.message; - } catch (err) { - error = err instanceof Error ? err.message : 'Failed to clear cache'; - } finally { - isClearing = false; - } - }
-
- -
+
diff --git a/ui/src/routes/(app)/reports/+page.svelte b/ui/src/routes/(app)/reports/+page.svelte index 3d01d79..548b0da 100644 --- a/ui/src/routes/(app)/reports/+page.svelte +++ b/ui/src/routes/(app)/reports/+page.svelte @@ -2,12 +2,7 @@ import { onMount } from 'svelte'; let Bar = $state(); let Line = $state(); - import { - getSummaryReport, - getConsumptionReport, - getBillingTrendsReport, - getCollectionStatusReport - } from '$lib/api/reports'; + import { getAllReports } from '$lib/api/reports'; import { getMeterGroups } from '$lib/api/meter-groups'; import { getProperties } from '$lib/api/properties'; import { formatCurrency } from '$lib/utils/format'; @@ -152,22 +147,26 @@ error = ''; const params: Record = {}; - if (startDate) params.startDate = formatDateToISO(startDate); + if (startDate) { + params.startDate = formatDateToISO(startDate); + } else { + // Default to a 12-month bound instead of an unbounded "To Present" scan, so the + // Firestore range filter actually bounds the cycle scan on first load — same + // window the Dashboard bounds its fetch to (dashboard/+page.svelte). + const now = new Date(); + const windowStart = new Date(now.getFullYear(), now.getMonth() - 12, now.getDate()); + params.startDate = windowStart.toISOString(); + } if (!usePresent && endDate) params.endDate = formatDateToISO(endDate); if (selectedMeterGroupId) params.meterGroupId = selectedMeterGroupId; if (selectedPropertyId) params.propertyId = selectedPropertyId; - const [summaryData, consumptionData, trendsData, collectionData] = await Promise.all([ - getSummaryReport(params), - getConsumptionReport(params), - getBillingTrendsReport(params), - getCollectionStatusReport(params) - ]); + const combined = await getAllReports(params); - summary = summaryData; - consumption = consumptionData; - billingTrends = trendsData; - collectionStatus = collectionData; + summary = combined.summary; + consumption = combined.consumption; + billingTrends = combined.billingTrends; + collectionStatus = combined.collectionStatus; } catch (err) { error = err instanceof Error ? err.message : 'Failed to load reports'; } finally { diff --git a/ui/src/routes/(app)/settings/+page.svelte b/ui/src/routes/(app)/settings/+page.svelte index b0faad9..b7d9f35 100644 --- a/ui/src/routes/(app)/settings/+page.svelte +++ b/ui/src/routes/(app)/settings/+page.svelte @@ -130,20 +130,6 @@ Go to settings →
- - - -

Photo Settings

-

- Control whether meter-reading photos are saved after OCR, on web and mobile -

-
- Go to settings → -
-
diff --git a/ui/src/routes/(app)/settings/photos/+page.svelte b/ui/src/routes/(app)/settings/photos/+page.svelte deleted file mode 100644 index 9a03d84..0000000 --- a/ui/src/routes/(app)/settings/photos/+page.svelte +++ /dev/null @@ -1,100 +0,0 @@ - - -
-
-

Photo Settings

-

- Control whether meter-reading photos are kept after being used for OCR suggestions. -

-
- -
- {#if error} -
- {error} -
- {/if} - - {#if success} -
- {success} -
- {/if} - - {#if isLoading} -

Loading...

- {:else} -
-
-

Save meter-reading photos

-

- When off (default), a captured photo is only used in-memory to suggest a reading value — - it's discarded before the reading is submitted, never stored. When on, the photo is kept - and attached to the reading. -

-

- Applies to meter-reading photos on web and mobile. Utility-bill photos used for - billing-cycle OCR are never saved either way. -

-
- -
- {/if} -
-
diff --git a/ui/src/routes/(app)/settings/users/+page.svelte b/ui/src/routes/(app)/settings/users/+page.svelte index fcab5d6..b353561 100644 --- a/ui/src/routes/(app)/settings/users/+page.svelte +++ b/ui/src/routes/(app)/settings/users/+page.svelte @@ -1,7 +1,11 @@
-
- -
+