Repository navigation
feat(uploads): bind presigned PUT URLs to the file's SHA-256 checksum - #1981
Conversation
generatePresignedPutUrl signs x-amz-checksum-sha256 (base64 of the raw
digest of the client's contentHash) into the URL as a header, so the object
store rejects a body that does not hash to it and a PUT that omits it.
headObject asks for ChecksumMode ENABLED and returns checksumSha256.
Clients send the header: putToPresignedUrl accepts { signal, contentHash }
(and hashes an in-memory body itself when no hash is given), uploadFile,
the storage transport and site-deploy pass the hash they already compute,
and bucket CORS allows x-amz-checksum-sha256.
|
I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".
|
|
Review complete. 🟡 1 medium 💬 Inline comments (1)
🧹 Nitpicks (1) — 🟢 1 low
The change makes upload integrity checksum-bound end to end: the upload client computes the SHA-256 of the body and sends it as a signed header, the GraphQL upload surface requires and forwards the hash, and the presigned-URL plugin signs it into the PUT so tampered payloads fail signature validation. Bucket CORS rules now allow the checksum header, and site-deploy CAS uploads pass the hash through the retry wrapper. Tests cover the new signer, checksum helpers, and integration flows.
No critical or high-severity issues were found; the main medium-severity concern is that Reviewed commit: 8df8ea6 |
There was a problem hiding this comment.
This PR binds every presigned S3 PUT to a client-supplied SHA-256 checksum via an unhoistable x-amz-checksum-sha256 header, threading the hash from the upload client through GraphQL to the signer and updating CORS, CAS upload, and tests accordingly.
Key findings
- 🟡 Optional
contentHashlets custom transports break uploads — transport.ts:31
| putObject( | ||
| url: string, | ||
| body: ArrayBuffer, | ||
| contentType: string, | ||
| signal?: AbortSignal, | ||
| contentHash?: string, | ||
| ): Promise<void>; |
There was a problem hiding this comment.
🟡 bug · medium
Optional contentHash lets custom transports break uploads
StorageTransport.putObject accepts contentHash as an optional trailing parameter (graphql/query/src/storage/transport.ts:31), but the server now makes contentHash a required upload input and signs x-amz-checksum-sha256 into every presigned PUT as an unhoistable header (graphile/graphile-presigned-url-plugin/src/s3-signer.ts:71). A pre-existing custom transport that ignores the fifth argument still type-checks but silently sends no header, so its PUTs fail S3 signature validation with an opaque 403. The same optional signature appears in packages/site-deploy/src/types.ts at line 102.
📋 Prompt for AI Agents
In graphql/query/src/storage/transport.ts lines 31-37, change putObject's trailing contentHash?: string to a required contentHash: string, and update the call site in graphql/query/src/storage/upload.ts:117, the default transport in packages/upload-client/src/storage-transport.ts, the PutObject type in packages/site-deploy/src/types.ts:102, and the test fakes in graphql/query/tests/storage.node.test.ts:119. The server signs x-amz-checksum-sha256 into every presigned PUT URL, so a transport that silently drops the hash produces opaque 403 signature errors; requiring the parameter forces third-party implementations to handle it at compile time.
Summary
A presigned PUT URL was signed over
content-length;hostonly, so any body of the declared size could be written behind a content-addressed key — including a second PUT through the same URL after the first one was confirmed. The URL is now signed over the file's SHA-256 as well, so the object store itself refuses bytes that don't hash to thecontentHashthe upload was requested with.graphile-presigned-url-plugin(s3-signer.ts):checksumSha256Base64is base64 of the raw digest (Buffer.from(hex, 'hex')), the S3 checksum encoding — not base64 of the hex string.unhoistableHeadersmatters: the presigner otherwise moves the checksum into the query string, where the store ignores it. As a signed header a wrong body gets400 BadDigestand a PUT without the header gets403 SignatureDoesNotMatch(verified against RustFS ins3-signer.integration.test.ts).requestUploadUrlpasses the already-validatedcontentHash; custom-key replace/versioning paths are unchanged (no conditional write — the key can still be replaced, only by bytes that hash to the new request'scontentHash).Clients:
@constructive-io/upload-client:putToPresignedUrl(url, body, contentType, signalOrOptions?)takes{ signal?, contentHash? }(anAbortSignalstill works). When nocontentHashis given and the body is in memory (string / ArrayBuffer / typed array / Blob) it hashes it itself (sha256HexOfBody), so existing callers keep working.uploadFile, the XHR progress path andstorageTransport.putObjectpass the hash they already compute;presignedPutHeaders/sha256HexToBase64are exported for raw-fetchcallers.@constructive-io/graphql-queryStorageTransport.putObject(..., signal?, contentHash?);@constructive-io/site-deployPutObjectgets the same trailingcontentHashand the default put passesfile.hash.@constructive-io/bucket-provisioner: CORS allowsx-amz-checksum-sha256.Rollout: a store that receives the header on a URL that wasn't signed over it still accepts the PUT, so clients can send it before the server signs it; once a server signs it, clients that don't send it get 403, so update raw-
fetchclients first.Link to Devin session: https://app.devin.ai/sessions/136c0078e2fa4f39a88a32ffd4480ed8
Open in Devin Desktop: https://app.devin.ai/desktop/session/136c0078e2fa4f39a88a32ffd4480ed8?variant=devin
Requested by: @pyramation