Skip to content

CCL-2423 | Name the cause behind a node's not-ready condition - #476

Merged
fabribet merged 4 commits into
mainfrom
bug/fabribet/CCL-2423-refactor-node-downtime-schema
Sep 11, 2026
Merged

fabribet merged 4 commits into
mainfrom
bug/fabribet/CCL-2423-refactor-node-downtime-schema

Conversation

@fabribet

@fabribet fabribet commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

What changed

ClusterNodeNotReadyInfo now explains a not-ready node in two tiers. condition still says what is wrong with the node; a new ClusterNodeNotReadyReason enum says why, with reason_message carrying the sentence to render. ClusterNodeActionKind also gains VIEW_LOGS, for causes that only show up in the database's own output.

event, event_message and ClusterNodeTerminationEvent are deprecated rather than removed, so this release is additive and buf breaking reports nothing. They are superseded by the reason tier and come out in a follow-up, once nothing writes them.

What the old schema could not do, for a node whose image cannot be pulled:

Before

condition: CLUSTER_NODE_NOT_READY_CONDITION_CONTAINER_START_BLOCKED
event:     unset, because the field only ever described what terminated a container

After

condition: CLUSTER_NODE_NOT_READY_CONDITION_CONTAINER_START_BLOCKED
reason:    CLUSTER_NODE_NOT_READY_REASON_IMAGE_UNAVAILABLE

The mapping is the part worth checking. VOLUME_UNAVAILABLE is the one reason reported under two conditions, and which one you get is decided by the storage class's binding mode rather than by the underlying problem:

Reason (CLUSTER_NODE_NOT_READY_REASON_) Reported under (CLUSTER_NODE_NOT_READY_CONDITION_)
INSUFFICIENT_RESOURCES POD_SCHEDULING_ERROR
NODE_CONSTRAINTS_UNSATISFIED POD_SCHEDULING_ERROR
VOLUME_UNAVAILABLE POD_SCHEDULING_ERROR, CONTAINER_START_BLOCKED
IMAGE_UNAVAILABLE CONTAINER_START_BLOCKED
CONFIGURATION_MISSING CONTAINER_START_BLOCKED
OUT_OF_MEMORY CONTAINER_RESTARTING
PROCESS_EXITED CONTAINER_RESTARTING

Why

The condition/event pair from #442 could name only two causes, both of them terminations, and only under CONTAINER_RESTARTING. Building the producer side found the gap. A pod that will not schedule, an image that will not pull, a missing secret: the user got a condition and no cause at all. Splitting the explanation gives the causes a tier of their own, one that grows as more become recognizable without disturbing the conditions. It also lets a single cause be reported under whichever condition it surfaces in.

Notes

  • Why the old fields are deprecated and not deleted. Not the external consumers, though they matter too: it is the window between this release and cluster-api adopting it. cluster-api on main imports ClusterNodeTerminationEvent at module scope in grpc_api/cluster/v1/not_ready_mapper.py, so a release without that enum turns any bump of the dependency into an ImportError in a module on the GetCluster and ListClusters path. Bumps there are routine and unrelated to this work, so whoever hit it would be someone who never touched node status, in a file they had no reason to be reading. Keeping the fields for one release decouples the proto release from the cluster-api adoption.
  • Removal is a follow-up, once cluster-api writes reason and nothing reads event. That one is breaking, will need the buf skip breaking label, and should say so in its release notes.
  • buf breaking reports nothing against main: the change is additive, so no label is needed here.
  • The branch takes a detour. The middle commit removes the deprecated fields and the third reverts that, so the net diff against main is the deprecation. Reviewing the diff rather than the commits is the shorter path.
  • No wire numbers move. reason takes 6 and reason_message 7 rather than reusing 3 and 4, which also lets the eventual removal reserve those numbers without disturbing anything.
  • CLUSTER_NODE_NOT_READY_CONDITION_UNKNOWN is documented as not emitted. Where the node status does not say what is wrong, not_ready_info stays unset rather than carrying a placeholder.
  • A DOCUMENTATION action was never guaranteed and now says so; it is attached only where a page exists for the specific cause.
  • The generated Go, Python, TypeScript and OpenAPI clients are buf generate output. Review the proto.
  • Populating reason happens in cluster-api, which needs this merged and released first.

@fabribet fabribet self-assigned this Sep 1, 2026
@fabribet fabribet added bug Something isn't working fix labels Sep 1, 2026
@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown

PR Packages Published

Python Package:

  • Version: 0.184.0.dev476+3277587
  • Package: qdrant-cloud-public-api
  • Registry: https://us-python.pkg.dev/qdrant-cloud/python/
  • To update run: uv add qdrant-cloud-public-api==0.184.0.dev476+3277587

NPM Package:

  • Version: 0.184.0-dev476.3277587
  • Package: @qdrant/qdrant-cloud-public-api
  • Registry: https://us-npm.pkg.dev/qdrant-cloud/npm/
  • To update run: npm install @qdrant/qdrant-cloud-public-api@0.184.0-dev476.3277587

@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown

The latest Buf updates on your PR. Results from workflow Pull Request / linting (pull_request).

BuildFormatLintBreakingUpdated (UTC)
✅ passed✅ passed✅ passed✅ passedSep 10, 2026, 12:05 PM

@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown

The latest Buf updates on your PR. Results from workflow Pull Request / linting (pull_request).

BuildFormatLintBreakingUpdated (UTC)
✅ passed✅ passed✅ passed✅ passedSep 1, 2026, 3:40 PM

@fabribet fabribet changed the title CCL-2423 | Report why a node is not ready, not just what is wrong CCL-2423 | Name the cause behind a node's not-ready condition Sep 1, 2026
@fabribet
fabribet marked this pull request as ready for review September 3, 2026 15:24
@fabribet
fabribet requested a review from a team as a code owner September 3, 2026 15:24

@sonasingh46 sonasingh46 left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

…-schema

Resolve conflicts in cluster.proto: keep the node-downtime refactor (the
reason/reason_message tier, the deprecation of event/event_message and the
CLUSTER_NODE_ACTION_KIND_VIEW_LOGS action) and layer on the OUTPUT_ONLY field
behaviors main added in #487, including on the two new fields.

Generated bindings regenerated with `make generate`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@fabribet
fabribet merged commit 333a993 into main Sep 11, 2026
11 checks passed
@fabribet
fabribet deleted the bug/fabribet/CCL-2423-refactor-node-downtime-schema branch September 11, 2026 15:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working fix

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants