Skip to content

perf(web): optimize document upload validation with static sets, MIME checks, and 50MB size guard - #1604

Open
muskanai0409-cell wants to merge 1 commit into
supermemoryai:mainfrom
muskanai0409-cell:perf/upload-validation-static-sets
Open

perf(web): optimize document upload validation with static sets, MIME checks, and 50MB size guard#1604
muskanai0409-cell wants to merge 1 commit into
supermemoryai:mainfrom
muskanai0409-cell:perf/upload-validation-static-sets

Conversation

@muskanai0409-cell

Copy link
Copy Markdown

Fixes #1559

What changed

Refactors isAcceptedFile() in apps/web/components/add-document/file.tsx per the issue:

  • Static sets hoisted to module scopeALLOWED_EXTENSIONS and ALLOWED_MIME_TYPES are now created once at module load instead of re-instantiating a Set on every call, eliminating redundant heap allocations during batch drag-and-drop.
  • 50MB fail-fast size guardisAcceptedFile() now rejects files over MAX_FILE_SIZE_BYTES (50MB), matching the limit already enforced by chat attachments (CHAT_ATTACHMENT_MAX_BYTES in apps/web/components/chat/attachments.ts) and the backend workers. Oversized files never start uploading, and addFiles() shows a dedicated toast ("One file exceeds the 50 MB limit") separate from the unsupported-type toast, so users get accurate feedback.
  • Direct O(1) MIME verification — the file's MIME type is checked against ALLOWED_MIME_TYPES first (application/pdf, Word/Excel Office types, text/csv, text/plain, text/markdown, application/json, text/html) with image/* prefix support, so extensionless mobile uploads (e.g. a blob with type: "application/pdf") are accepted. MIME parameters such as ;charset=utf-8 are stripped before the lookup. The extension check is kept as a fallback for files with no/unknown MIME type.
  • application/pdf added to FILE_ACCEPT, aligning it with CHAT_ATTACHMENT_ACCEPT.

Behavior notes

  • Previously accepted files with a supported extension are still accepted; the only newly rejected inputs are files >50MB, which the gateway rejected anyway.
  • Newly accepted: extensionless files whose MIME type is a supported document type (previously only markdown/JSON/HTML/image MIME types passed).

Verification

  • Unit-tested the validator with bun test (extension matrix, case-insensitivity, extensionless MIME acceptance, charset parameters, image/*, exact-50MB boundary, >50MB rejection, unsupported types) — 12 tests, all passing locally.
  • bunx biome check on the changed file: clean.
  • tsc --noEmit in apps/web: no new errors introduced (the 84 pre-existing errors on main are unchanged).

🤖 Generated with Claude Code

… checks, and 50MB size guard

- Hoist ALLOWED_EXTENSIONS and ALLOWED_MIME_TYPES Sets to module scope so
  batch drag-and-drop no longer re-allocates them per file check
- Fail fast on files larger than 50MB (matching the chat attachment and
  backend worker limit) with a dedicated toast instead of uploading them
  only to be rejected at the gateway
- Verify MIME types directly (application/pdf, Office types, text/csv,
  text/plain, ...) so extensionless uploads such as mobile blobs are
  accepted, stripping parameters like ";charset=utf-8" before lookup,
  with the extension check kept as fallback
- Add application/pdf to FILE_ACCEPT, matching CHAT_ATTACHMENT_ACCEPT

Fixes supermemoryai#1559

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

perf(web): optimize document upload validation with static sets, MIME checks, and 50MB size guard

1 participant