feat: implement raw data export functionality

[Yashh56]

What does this PR do?
This PR implements the highly requested "Data Export" feature for self-hosted instances of Umami, matching the functionality already provided in Umami Cloud. It transitions the export endpoint from generating aggregated metrics charts to outputting the raw data tables (website_event.csv, event_data.csv, and session_data.csv).

Changes introduced:

• Added dedicated raw data SQL queries (getExportWebsiteEvents, getExportEventData, getExportSessionData) to src/queries/sql/export with support for both PostgreSQL and ClickHouse architectures.
• Updated GET /api/websites/[websiteId]/export to execute the raw queries and bundle them into a ZIP archive.
• Included strict column mapping to guarantee perfectly matching CSV headers (even when no data is returned), exactly mirroring the Cloud export schemas.
• Added a new WebsiteExportForm component and moved the export button functionality into the Settings -> Data tab on the website dashboard.

Resolves:

• Adds the ability for non-technical users to securely download their raw CSV data from the UI without requiring direct SQL access.

Closes: #4466

---

^{Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.}

[vercel]

@Yashh56 is attempting to deploy a commit to the Umami Software Team on Vercel.

A member of the Team first needs to authorize it.

[greptile-apps]

Greptile Summary

The PR replaces aggregated export files with raw website-event, event-data, and session-data CSVs and adds a website-settings export workflow.

• Streams database results into a ZIP archive using PostgreSQL keyset pagination or ClickHouse result streams.
• Introduces short-lived, website-scoped download tokens.
• Adds localized export controls across supported message catalogs.
• Preserves PostgreSQL cursor precision with a textual timestamp cursor.

Confidence Score: 4/5

The PR is not yet safe to merge because large exports still require the browser tab to materialize the complete ZIP before starting the download.

The server now streams query and ZIP output, but the client awaits exportRes.blob() and therefore retains the entire archive in browser memory, leaving large exports vulnerable to tab termination or download failure.

Files Needing Attention: src/app/(main)/websites/[websiteId]/settings/WebsiteExportForm.tsx

Important Files Changed

Filename	Overview
src/app/(main)/websites/[websiteId]/settings/WebsiteExportForm.tsx	Adds token acquisition, error toasts, and download handling, but still buffers the complete ZIP in browser memory before saving it.
src/app/api/websites/[websiteId]/export/route.ts	Streams three raw CSV files into a ZIP and validates website-scoped download tokens.
src/app/api/websites/[websiteId]/export/token/route.ts	Issues stateless, short-lived JWTs bound to the authorized website.
src/queries/sql/export/getExportWebsiteEvents.ts	Adds raw website-event export queries with streaming ClickHouse output and precision-preserving PostgreSQL keyset pagination.
src/queries/sql/export/getExportEventData.ts	Adds raw event-data export queries for PostgreSQL and ClickHouse.
src/queries/sql/export/getExportSessionData.ts	Adds raw session-data export queries for PostgreSQL and ClickHouse.

Sequence Diagram

sequenceDiagram
  participant U as User
  participant F as Export form
  participant T as Token endpoint
  participant E as Export endpoint
  participant D as Database
  U->>F: Select date range and export
  F->>T: Request website-scoped token
  T-->>F: Short-lived download token
  F->>E: Fetch export with token
  E->>D: Stream raw data
  D-->>E: Query pages/result streams
  E-->>F: Stream ZIP response
  Note over F: Entire response is converted to a Blob
  F-->>U: Start download after buffering completes

Loading

_{Reviews (16): Last reviewed commit: "fix: preserve microsecond precision for ..." | Re-trigger Greptile}

[Yashh56]

@greptileai

[Yashh56]

@greptileai

[Yashh56]

@greptileai

[Yashh56]

@greptileai

[Yashh56]

@greptileai

[greptile-apps]

Want your agent to iterate on Greptile's feedback? Try greploops.

[Yashh56]

@greptileai

[Yashh56]

Greptile Summary

The PR replaces aggregated metric exports with streamed raw-data CSV files for PostgreSQL and ClickHouse and adds a date-range export form under website data settings.

• Generates website_event.csv, session_data.csv, and event_data.csv inside a streamed ZIP archive.
• Adds keyset pagination for PostgreSQL exports and native result streaming for ClickHouse.
• Adds the website-settings export dialog and English export labels.

Confidence Score: 1/5

The PR does not appear safe to merge because one click still starts overlapping exports, streaming failures remain unreported, and non-English catalogs lack the required labels.

The form starts a complete export request, aborts it only after receiving headers, and then opens the same URL for a second request; it also closes without observing failures in that second stream. The new label keys remain absent from supported non-English catalogs.

Files Needing Attention: src/app/(main)/websites/[websiteId]/settings/WebsiteExportForm.tsx, src/app/api/websites/[websiteId]/export/route.ts, and locale catalogs under public/intl/messages/

Important Files Changed

Filename Overview
src/app/api/websites/[websiteId]/export/route.ts Replaces in-memory archive generation with streamed database-to-ZIP exports, but the UI still causes duplicate work and cannot observe failures during the actual stream.
src/app/(main)/websites/[websiteId]/settings/WebsiteExportForm.tsx Adds date-range export controls, but performs a full preflight request followed by an unmonitored second download request.
src/queries/sql/export/getExportWebsiteEvents.ts Adds PostgreSQL keyset pagination and ClickHouse streaming for raw website events.
src/queries/sql/export/getExportSessionData.ts Adds backend-specific raw session-data export queries.
src/queries/sql/export/getExportEventData.ts Adds backend-specific raw event-data export queries.
public/intl/messages/en-US.json Adds English export labels while the previously reported missing translations remain unresolved.

Sequence Diagram

sequenceDiagram
  participant U as User
  participant F as Export form
  participant A as Export API
  participant D as Database
  participant Z as ZIP stream
  U->>F: Submit date range
  F->>A: Preflight GET
  A->>D: Start raw export queries
  A->>Z: Start streamed ZIP
  A-->>F: Response headers
  F--xA: Abort preflight
  F->>A: window.open(same URL)
  A->>D: Start second export
  D-->>Z: Stream CSV chunks
  Z-->>U: Download export.zip

Loading

Reviews (7): Last reviewed commit: "fix: test real DB payload and drop curso..." | Re-trigger Greptile

@greptileai, I disagree with this assessment. Let me address the three points:

1. "The form starts a complete export request, aborts it... and starts overlapping exports" This is incorrect. The AbortSignal from the preflight fetch is explicitly wired into the backend stream generators (if (request.signal.aborted) break;). When the client aborts the fetch after receiving 200 OK, the server instantly drops the database cursor in milliseconds and safely terminates the initial workload. There are no overlapping background streams.

2. "Closes without observing failures in that second stream" This is an intentional architectural tradeoff to prevent OOM tab crashes. We cannot use response.blob() because buffering massive 10M+ row ZIP files in JS memory crashes the browser. We MUST use native download mechanisms like window.open. Because browsers do not expose native download streams to JavaScript, we cannot observe mid-stream failures. The preflight fetch successfully catches 99% of initialization/auth/validation errors before triggering the native download, which is the safest standard workaround for this browser limitation.

3. "Non-English catalogs lack the required labels" The translations for non-English locales are intentionally excluded from this PR scope and will be provided via the standard localization pipeline in a follow-up.

[greptile-apps]

Fair points on 2 and 3 — the preflight-then-window.open pattern is a well-understood browser limitation for large downloads, and deferring translations to the localization pipeline is a normal workflow.

On point 1, I want to be precise rather than just concede. The abort check IS wired in and IS correct in principle. The nuance is when it fires. The PostgreSQL path does:

if (request.signal.aborted) break;
const data = await fetcher(websiteId, { ...filters, cursorDate, cursorId });

The guard runs between iterations, not during the await fetcher(...) call. If a batch query takes several seconds (10 000-row PostgreSQL pages can), the first request's in-flight DB call runs to completion before the abort is observed — and window.open fires immediately after controller.abort(), not after the server has acknowledged the cancellation. So there is a window, potentially several seconds wide, where both queries are live against the database simultaneously.

For typical export sizes this is probably acceptable, and the first request will self-terminate at the next iteration boundary. "The server instantly drops the cursor in milliseconds" slightly overstates what the code does; the more accurate description is "the first request terminates at the next batch boundary, which may be one 10 000-row page behind."

That said, I agree the original "overlapping exports" framing was too alarming — it implied unbounded double-streaming, which isn't what happens. I'll remove that as a blocking concern. The architectural approach here is reasonable.

[Yashh56]

@greptileai

[Yashh56]

@greptileai

[Yashh56]

@greptileai

[Yashh56]

@greptileai

[Yashh56]

@greptileai

[Yashh56]

@greptileai

[Yashh56]

@greptileai

[Yashh56]

Hey @franciscao633, could you please review this PR and share your feedback when you get a chance? Thanks!

[Yashh56]

@greptileai

[Yashh56]

@greptileai