Skip to content

slurp small downloads from oraclecloud - #501

Closed
felliott wants to merge 1 commit into
CenterForOpenScience:developfrom
felliott:feature/oracleslurp
Closed

slurp small downloads from oraclecloud#501
felliott wants to merge 1 commit into
CenterForOpenScience:developfrom
felliott:feature/oracleslurp

Conversation

@felliott

Copy link
Copy Markdown
Member

Ticket

No ticket, just fresh off the dome.

Purpose

WB has a weird bug (also encountered with googledrive) where trying to download small files indirectly will hang after reading one chunk. "Indirectly" in this case means from MFR, which uses WB as a library to download cached file conversions. This commit applies the same fix to oraclecloud as we used for googlecloud. If the download response has a content-length header AND the header is less than a configurable number of bytes (default: 100k), then we slurp the file contents into memory and wrap it in a StringStream object.

No idea why this is happening, but the divine Cargomides seems to appreciate it.

Changes

See above

Side effects

Only affects code on oraclecloud, should not affect prod.

QA Notes

None.

Deployment Notes

Nay

 * WB has a weird bug (also encountered with googledrive) where trying
   to download small files indirectly will hang after reading one
   chunk. "Indirectly" in this case means from MFR, which uses WB as a
   library to download cached file conversions. This commit applies
   the same fix to oraclecloud as we used for googlecloud. If the
   download response has a content-length header AND the header is
   less than a configurable number of bytes (default: 100k), then we
   slurp the file contents into memory and wrap it in a StringStream
   object.

   No idea why this is happening, but the divine Cargomides seems to
   appreciate it.
@felliott
felliott requested a review from brianjgeiger August 17, 2026 18:30
@felliott felliott closed this in e387f56 Aug 18, 2026
@felliott
felliott deleted the feature/oracleslurp branch August 18, 2026 12:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants