feat: let the S3 client address a bucket path-style - #378
nGervasyuk wants to merge 1 commit into
Conversation
S3-compatible storage — MinIO, Ceph, Garage — is normally reached path-style (host/bucket/key). AWS uses virtual-host style (bucket.host/key) and the SDK assumes that, with no environment variable for the switch, so a deployment pointing VRT at its own storage got requests for a hostname that does not resolve. The endpoint itself the SDK already reads from AWS_ENDPOINT_URL; only the addressing style was unreachable. AWS_S3_FORCE_PATH_STYLE=true turns it on. Unset, the SDK keeps deciding for itself, so nothing changes for an AWS deployment.
|
Warning Review limit reachedNext included review available in 11 minutes. View limit detailsLimit details: You’ve used the included review currently available. You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Review configuration: ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Team Run ID: 📒 Files selected for processing (2)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Closing — not wanted upstream for now. The change came out of standing up a production-shaped stack locally against MinIO, where the SDK's virtual-host addressing makes every upload fail; it is not needed by the deployment that prompted it, which is on AWS proper. Leaving it here rather than deleting the reasoning: |
S3-compatible storage — MinIO, Ceph, Garage — is normally reached path-style (
host/bucket/key). AWS uses virtual-host style (bucket.host/key), the SDK assumes that, and it has no environment variable for the switch.new S3Client()is constructed with no configuration, so there was no way to change it.The result is that pointing VRT at anything other than AWS fails at the first upload:
The endpoint itself the SDK already reads from
AWS_ENDPOINT_URL, so that half worked; only the addressing style was unreachable.AWS_S3_FORCE_PATH_STYLE=truenow turns it on. Unset, the SDK keeps deciding for itself, so nothing changes for an AWS deployment — the client is constructed with an empty config exactly as before.Why it is worth having
Beyond self-hosters using their own object storage, this is what makes it possible to stand up a stack that behaves like production locally: images in S3-compatible storage, reached through the API's signed-URL redirect. That path — the redirect, the pre-signed URL, its caching — simply does not exist on the HDD backend, so a local HDD instance cannot reproduce or reveal anything about it. Verified here against MinIO end to end: uploads land in the bucket,
/images/:nameredirects to a path-style signed URL, and the browser follows it.Tests
Two, watched failing first: the client is constructed with
forcePathStylewhen the variable is set, and is left to the SDK's own choice when it is not.All backend spec files pass.