Skip to content

feat(backlog): presign resumable fallback, login rate-limit and e2e private HLS - #42

Merged
SlavaKuntsov merged 5 commits into
mainfrom
feat/backlog-presign-rate-limit-e2e
Sep 4, 2026
Merged

feat(backlog): presign resumable fallback, login rate-limit and e2e private HLS#42
SlavaKuntsov merged 5 commits into
mainfrom
feat/backlog-presign-rate-limit-e2e

Conversation

@SlavaKuntsov

Copy link
Copy Markdown
Owner

Closes #19
Closes #20
Closes #21

Summary

Backlog phase — 3 issues: brute-force protection for login, Content-Range resumable fallback for presigned upload, and e2e coverage for presign + private HLS.

  • auth services/auth/src/core/limiter.py:1 — slowapi 5/min/IP on POST /api/v1/auth/login with redis (redis:7-alpine in compose) fallback to memory; 429 after limit (services/auth/src/routers/auth.py:60, services/auth/src/main.py:11).
  • upload services/upload/internal/handler/resumable.go:1GET /api/v1/videos/{id}/resumable offset + PUT /resumable with Content-Range: bytes start-end/total (308 until complete, 200 when done, 416 on mismatch) via StatObjectSize/GetObject/PutObject; gateway proxies /resumable (services/gateway/cmd/server/main.go:121); frontend frontend/src/lib/api.ts:158 adds getResumableOffset/uploadResumable with fallback in uploadVideo.
  • infra/e2e scripts/e2e.sh:262 — adds steps 8 presign POST /presign→PUT presigned→POST /complete→poll ready→HLS, 9 private PATCH visibility private→403 without token→200 with hls-token + Cache-Control private, 10 resumable 308→200 incl. 50% split resume.
  • fix services/upload/internal/storage/minio.go:60 — generate presigned URL for public host (MINIO_PUBLIC_ENDPOINT) without breaking SigV4 (tmp client Region us-east-1, BucketLookupPath); services/gateway/cmd/server/main.go:121 — remove invalid wildcard */resumable causing chi panic.

Acceptance criteria

Verification

  • make lint-go — 0 issues
  • make lint-py — black/flake8/mypy pass
  • make lint-front — next lint 0 warnings
  • make test-go — upload handler 13 passed (incl. 4 resumable)
  • make test-py — auth 12 passed (incl. rate-limit), transcoder 20 passed
  • make up --build -d && make e2e — PASS (presign 200, private 403→200, resumable 308→200)

Remaining risks

  • resumable append is Get+append+Put (O(n) re-upload per chunk) — ok for e2e 32KB sample, not for 5GB multi-chunk (accepted for backlog; MinIO multipart would be next step)
  • limiter falls back to memory when redis unreachable locally (probe in limiter.py) — in compose redis is reachable, limits distributed

Issue #19: redis in compose, slowapi limiter on POST /api/v1/auth/login returns 429 after 5 req/min per IP. Falls back to memory when redis unreachable (local pytest). Adds REDIS_URL to .env.example and compose.
Issue #20: GET/PUT /api/v1/videos/{id}/resumable handles Content-Range bytes start-end/total, validates offset via StatObjectSize and appends via Get+Put. Gateway proxies resumable endpoints, frontend adds getResumableOffset/uploadResumable with automatic fallback in uploadVideo on presigned PUT failure (50% resume test).
Issues #19,#21: redis:7-alpine service with healthcheck and redisdata volume, auth REDIS_URL env; e2e now tests presign PUT→complete→HLS, private HLS 403/200 with hls-token and Cache-Control private, and Content-Range resumable 308→200 including 50% split resume.
…panic

chi wildcard * must be last segment; keep only {id} param route. Fixes panic at main.go:123.
…SigV4

Previously rewrote host after signing which broke X-Amz-Signature (host is signed header). Now creates tmp minio client with public host (localhost:9000) and Region us-east-1 to generate correct signature. Also stores accessKey/secretKey in MinioClient for reuse. Fixes e2e presign 403 SignatureDoesNotMatch and 500 dial tcp connection refused (tmp client now uses BucketLookupPath and Region to avoid bucket location fetch).
@SlavaKuntsov
SlavaKuntsov merged commit 91efb5b into main Sep 4, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

P1 infra: extend e2e for presign and private HLS P1 upload: add Content-Range resumable fallback P1 auth: add login rate-limit against brute-force

1 participant