Skip to content

feat(api): add multipart upload for large photos - #53

Open
MehrshadFb wants to merge 1 commit into
mainfrom
feat/photo-multipart-upload
Open

MehrshadFb wants to merge 1 commit into
mainfrom
feat/photo-multipart-upload

Conversation

@MehrshadFb

Copy link
Copy Markdown
Collaborator

Stacked on #52; the diff here is only the multipart change once that merges.

Summary

  • Multipart upload for photos of 5 MiB and more, alongside the single-PUT flow:
    • POST /events/:eventId/photos/multipart-uploads { contentType, sizeBytes } reserves quota like a single slot, opens an S3 multipart upload, stores its id and part size on the row, and returns the layout: { photoId, sizeBytes, partSizeBytes, expiresAt, parts: [{ partNumber, sizeBytes, uploadUrl, uploaded }] }. Parts are fixed at 5 MiB (S3's minimum), so a 25 MB file is five; each uploadUrl is a presigned UploadPart bound to that part's exact Content-Length.
    • GET /photos/:photoId/multipart-upload returns the same layout with fresh URLs and uploaded: true on the parts S3 already holds, for resuming after a drop, an app restart, or expired URLs.
    • POST /photos/:photoId/multipart-upload/complete (no body) lists the parts on S3, checks every planned part is there at its planned size, calls CompleteMultipartUpload with the ETags S3 reported, then verifies the assembled object exactly like confirm and flips the row to READY. Returns the same { photoId, status } as confirm; a second call returns READY again.
    • DELETE /photos/:photoId aborts an open upload before deleting; so do the stale-PENDING sweeper and every slot release (rejected confirm, expired upload, failed initiate).
  • S3Service gains createMultipartUpload, getPresignedUploadPartUrl, listMultipartParts, completeMultipartUpload (client-side S3 refusals come back as { completed: false, code }), and abortMultipartUpload (an unknown upload counts as aborted).
  • PhotosService exposes assertCanUploadToEvent, verifyUploads, and releaseUploadSlots so the new PhotoMultipartService reuses the same authorization, verification, and release paths; the READY flip clears the multipart columns.
  • Migration 20260906135921_add_photo_multipart_upload: nullable Photo.multipartUploadId (varchar 2048) and Photo.multipartPartSizeBytes.
  • Terraform: the API user gains s3:AbortMultipartUpload and s3:ListMultipartUploadParts on bucket objects.
  • Tests: part planner, S3Service multipart methods (including the real signer's X-Amz-SignedHeaders for a part URL), PhotoMultipartService, the abort paths in PhotosService and the sweeper, and e2e for the three endpoints and delete.
  • Docs: §1 gains "Multipart upload", plus §4, §5, §6, §7, §8, §10 updates.

Why

A single PUT lands whole or not at all, so a 20 MB photo that drops at 95% on cellular starts over from zero. With parts uploaded, retried, and resumed independently, a drop costs one part, and an upload survives an app restart because the API, not the client, remembers what S3 already holds.

API

  • New endpoints above; InitiateMultipartUploadDto, MultipartUploadResponseDto, MultipartPartResponseDto added to the spec. Existing endpoints and DTOs are unchanged.
  • Failure modes: parts missing at completion → 400 naming them; S3 refuses the assembly (InvalidPart, EntityTooSmall, …) → 400 with the code, upload still open; S3 no longer knows the upload → slot released, 410 Gone; another uploader's photo, a single-PUT slot, or a non-PENDING row → 404; lost event access → 403.
  • Client contract: PUT each part with exactly its sizeBytes as Content-Length (no Content-Type needed on parts); nothing to persist per part. Use complete, not confirm, to finish a multipart upload.
  • Mobile: only lib/api/generated regenerated (four files), no hand-written mobile change.

Notes

  • Deploy order: terraform apply in api/infra before deploying this. Without s3:ListMultipartUploadParts every complete and GET returns 500, and without s3:AbortMultipartUpload deletes of pending multipart rows fail.
  • Parts of an upload that never completes are billed but invisible to ListObjectsV2, so the orphan reconciler never sees them. The existing lifecycle rule aborts incomplete uploads after one day, the sweeper aborts before deleting the row, and every release path aborts first.
  • The part size is stored per row so an in-flight upload keeps its layout if the constant changes.
  • Files under 5 MiB are rejected with 400; they gain nothing from multipart and keep using upload-urls.

Test plan

  • npx jest --watchman=false (453 tests)
  • npx jest --config ./test/jest-e2e.json --watchman=false (129 tests)
  • npm run typecheck
  • npm run format:check and eslint on the changed files
  • npm run openapi:generate, then pnpm run openapi:generate, tsc --noEmit, lint, and format:check in mobile/
  • terraform fmt -check
  • Migration applied with prisma migrate dev against a scratch Postgres 18
  • terraform apply (needs the admin profile)
  • CI passes on PR

@github-actions

github-actions Bot commented Sep 6, 2026

Copy link
Copy Markdown

API unit-test coverage

Metric Coverage
Statements 74.94% (975/1301)
Branches 57.5% (410/713)
Functions 79.74% (185/232)
Lines 73.4% (853/1162)

Unit suite only; controllers are exercised by the e2e suite.

@MehrshadFb
MehrshadFb force-pushed the feat/event-delete-s3-purge branch from 721fe28 to e7f4296 Compare September 9, 2026 02:46
@MehrshadFb
MehrshadFb force-pushed the feat/photo-multipart-upload branch 2 times, most recently from bb85125 to bf9ceef Compare September 9, 2026 02:52
@MehrshadFb
MehrshadFb force-pushed the feat/event-delete-s3-purge branch from e7f4296 to 4ac0e76 Compare September 9, 2026 02:52
@MehrshadFb
MehrshadFb changed the base branch from feat/event-delete-s3-purge to main September 9, 2026 03:01
@MehrshadFb
MehrshadFb force-pushed the feat/photo-multipart-upload branch from bf9ceef to 3ac57c0 Compare September 9, 2026 03:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant