feat: add the ModelArtifact expectedDigest anchor - #702
Merged
Merged
Conversation
- spec.expectedDigest asserts the manifest digest a hub source must resolve to; admission accepts it on hub sources only, refusing claim and image sources each with the reason their identity rules it out, and the whole-spec immutability ratchet covers it - status.resolved.digestSource records where a resolved digest came from: Hub, the hub's own listing, or Expected, the spec's anchor - the stale modelscope member comment, left reserved-shaped by #699, now describes the opened source - generated deepcopy, crd, protobuf, openapi and applyconfiguration artifacts Task 1 of model-artifact-expected-digest. Signed-off-by: thxCode <thxcode0824@gmail.com>
- a resolution whose manifest digest differs from the anchor is refused as DigestMismatch with both digests in the message, and DigestMismatch joins the revoking reasons - the anchored revalidation re-lists the tree at the resolved commit and re-compares the digest instead of probing one file; the unanchored probe is unchanged - a SourceUnavailable confirmed on the refusal staircase writes the anchor as the identity (digestSource Expected, no revision or counts) and the artifact never contacts the hub again - the hub interface grows the manifest-returning ListManifest both clients already serve - mutations prove the comparison and the anchored revalidation tests go red Task 2 of model-artifact-expected-digest. Signed-off-by: thxCode <thxcode0824@gmail.com>
…ly chain - an anchored artifact under Engine delivery waits with AnchorNeedsNodeDelivery, the message naming why: an engine downloads by repository and revision and cannot anchor-verify what it fetched, and an Expected identity has no commit - an Expected identity's node chain is peers only: the manifest comes from a peer's published listing bound to the digest (Puller.FetchManifest), and with no peer holding the tree the mount fails naming the digest nothing holds - the hub-identity chain is unchanged, still listing anchor-checked before any byte moves Task 3 of model-artifact-expected-digest. Signed-off-by: thxCode <thxcode0824@gmail.com>
- artifact.md: the expectedDigest field and its hub-only admission, the anchored resolution and revalidation states, digestSource, the confirmed SourceUnavailable fallback and what an Expected identity delivers, the shipped access model for Expected identities, the AnchorNeedsNodeDelivery row, the migration contract, and the downgrade-window note - node-store.md: the Expected identity's peers-only manifest and the SourceUnavailable row's anchored case - README index: the artifact page's description carries the anchor Task 4 of model-artifact-expected-digest. Signed-off-by: thxCode <thxcode0824@gmail.com>
- a match resolves with digestSource Hub; a foreign anchor is refused DigestMismatch with both digests in the message - a twice-unreachable hub writes the anchor as the identity with the hub's request log flat, and the Expected identity mounts from the peer with the hub log still flat - Engine delivery holds both anchored artifacts with AnchorNeedsNodeDelivery and no Pod - the peer leg skips on one worker or with E2E_C114_OFFLINE=1; the run restores both Settings it flips Task 5 of model-artifact-expected-digest. Signed-off-by: thxCode <thxcode0824@gmail.com>
…V pin - fetchTree built every file's URL by calling the hub, which is nil on the peers-only path and would panic before a peer byte was pulled; the URL is now empty when there is no hub, and the per-file loop's no-hub branch was already the loud no-source failure - the KV identity pin the spec's acceptance names: the digest's leading digits for a hub artifact and for an Expected identity alike, the UID's hash only for a claim Task 5 of model-artifact-expected-digest. Signed-off-by: thxCode <thxcode0824@gmail.com>
The artifact helper appends its last argument under spec:, six spaces landed the field inside the huggingFace member and strict decoding refused every anchored artifact the case created. Task 5 of model-artifact-expected-digest. Signed-off-by: thxCode <thxcode0824@gmail.com>
The replica render synthesizes the runner image from the instance type's observed hardware; the cluster's generic type has none, so the render failed before WeightsReady could carry the anchor block. Name the stock python image on the role — the block creates no Pod whatever the image is. Task 5 of model-artifact-expected-digest. Signed-off-by: thxCode <thxcode0824@gmail.com>
T1-T6 delivered: the anchor's admission, assertion at resolution and revalidation, the confirmed-outage Expected identity, the peers-only chain, the Engine block, the documentation, and the e2e case. The plan-gate adjudications are recorded in the spec's Open Questions and Alternatives. Task 6 of model-artifact-expected-digest. Signed-off-by: thxCode <thxcode0824@gmail.com>
thxCode
added
kind/enhancement
|
✅ OpenCodeReview: Review complete: 0 finding(s) across 1 selected item(s). Reviewed |
![gpustack-code-review[bot]](https://avatars.githubusercontent.com/in/4788230?s=60&v=4){kind=small}
…nd class fromPeersOnly replaced FetchManifest's error wholesale: a cancellation under the listing — the waiter window firing — was misclassified as SourceUnavailable and consumed the digest's backoff, and every other underlying cause never reached the plugin's log. A cancellation now keeps its canceled class (no backoff, the shape fetchTree's wait already uses), and any other failure logs its cause beside the loud no-source. The ledger's tenant-free message is unchanged. Task 5 of model-artifact-expected-digest. Signed-off-by: thxCode <thxcode0824@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What type of PR is this?
/kind enhancement
/kind api-change
/area worker
/area testing
What this PR does / why we need it:
Adds an optional
expectedDigestto theModelArtifactspec: the manifest digest the user asserts this artifact resolves to. The manifest format v1 was published precisely so such an assertion could later be compared with it; this PR opens that comparison and what it unlocks.expectedDigestvalidates assha256:+ 64 lowercase hex and is accepted on hub sources only — a claim's content is whatever the volume holds at mount time (dynamically provisioned claims differ per provisioning) and its identity is the claim itself, which the user confirms; an image's identity is its reference's digest. Immutable with the whole spec.DigestMismatchwith both digests in the message, andDigestMismatchjoins the revoking reasons. The periodic revalidation of an anchored artifact re-lists the tree at the resolved commit and re-canonicalizes instead of probing one file, so a hub or mirror that stops serving the pinned content fails the same staircase; unanchored artifacts keep the two-request check.SourceUnavailableconfirmed on the refusal staircase (one blip is not a verdict) writes the anchor as the resolution:status.resolved.manifestDigest= the anchor,digestSource: Expected, no revision or counts — and the hub is never contacted again: no revalidation, no token probe. A hub that answered is never anchored around.AnchorNeedsNodeDelivery: the engine downloads by repository and revision and cannot anchor-verify what it fetched; an Expected identity has no commit to pin). An Expected identity's node chain is peers only: the manifest comes from a peer's published listing bound to the digest (Puller.FetchManifest), and with no peer holding the tree the mount fails loudly naming the digest nothing holds.NodeModelStorestatus (adjudicated and documented indocs/model-store/artifact.md); hub-verified identities keep resolving with the namespace's own credential. enhancement: migrate pre-existing GPUStack model files into the node cache #693 (legacy-cache migration) is deferred:expectedDigestis recorded in the docs as the acceptance contract that future import must satisfy — verify against the anchor, publish only through the store's pipeline, lazy, one-shot.E2E evidence:
case-1and the newcase-114(eight checks: match resolvesdigestSource: Hub; a foreign anchor refusedDigestMismatchnaming both digests; a twice-unreachable hub writing the Expected identity with the hub's request log flat 9 → 9;AnchorNeedsNodeDeliveryon both anchored kinds with no Pod created; the Expected identity mounting with the hub log still flat) ran on kind v1.35.5 against images11-e2e, built from this branch's head42c34e1f. The current head355129adadds only the spec document and the #701 rebase, so the tested tree is equivalent for every code path.test chart(chart-testing install of the full stack) ran green on the same head family. Per-task commits carry the unit suites; AC2/AC3 are mutation-verified red (21fcec69records the KV pin and the nil-hub fix the end-of-build review caught).Which issue(s) this PR links to:
NONE
Special notes for your reviewer:
ModelScopemember comment, left reserved-shaped by feat(worker): open the ModelScope ModelArtifact source end to end #699, is corrected here (pure comment).case-114flips two Settings (hub endpoint, delivery mode) and restores both in its trap; its peer leg auto-skips on one worker or withE2E_C114_OFFLINE=1.Verifiedcondition stays deferred: its only novel scenario was a claim anchor, which this PR deliberately does not open.Does this PR introduce a user-facing change?