Repository navigation
feat(preheat): size the snapshot inside the job, not the create request - #129
Closed
mangoknight wants to merge 1 commit into
Closed
mangoknight wants to merge 1 commit into
mangoknight wants to merge 1 commit into
Conversation
Creating a preheat job measured every file of the snapshot before answering, inside the scheduler's 10s request. Batching paths-info (dingodb#127) fixed repositories with a few hundred files, but uonlp/CulturaX has 9059: 71 batches at 1-4s each through the regional exit, so create still failed with "context canceled". - Create only resolves the commit and file list (one request) and registers the job with the upstream usedStorage as an estimate. - The task measures the snapshot first (Prepare), records the exact size locally and reports it to the scheduler, then transfers. A sizing failure now fails the job with its reason instead of the create call. - Sizes come from one revision?blobs=true request (9059 files: 1.7s direct), falling back to batched paths-info when sizes are missing. - Resume reports "running" with the size once preparation is done, so the scheduler no longer shows a resumed job as interrupted. - Completed-job reuse checks against the size that job recorded, since a create request now only carries the estimate. - proto: UpdateCacheJobStatusReq.usedStorage (field 8). The running report is best effort and never queued for replay, so it cannot land after a terminal status. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Superseded by #128.
Moves exact snapshot sizing out of the create request into the job's prepare phase (one
revision?blobs=truerequest, falling back to batched paths-info), reports measured size and running status, and addsUpdateCacheJobStatusReq.usedStorage(field 8).🤖 Generated with Claude Code