chore(deps): Update kindest/node Docker tag to v1.37.0 - #2583
Merged
Conversation
github-actions
Bot
added
the
dependencies
github-actions
Bot
added
renovate
dependencies
mchmarny
approved these changes
Sep 4, 2026
Contributor
Author
Coverage Report ✅
Coverage BadgeNo Go source files changed in this PR. |
mchmarny
added a commit
that referenced
this pull request
Sep 4, 2026
tools/setup-tools guards installs on presence (command_exists), not version, so the E2E lane kept whatever binary was already on the box. Stale binaries arrived from the runner image and from the cache step's restore-keys prefix fallback, which restores an older tool cache whenever .settings.yaml changes the exact key. That pinned E2E to kind v0.31.0 against the v0.33.0 pin. v0.31.0 emits a kubeadm.k8s.io/v1beta3 ClusterConfiguration, which kubeadm 1.37 in kindest/node:v1.37.0 rejects, so cluster-create has failed on main since #2583 bumped the node image. Six other pins were ignored the same way: ko, ctlptl, tilt, aws, hauler, and helm (v3.20.0 against the v4.2.4 pin). Pass --upgrade so CI installs the .settings.yaml versions. Signed-off-by: Mark Chmarny <mark@chmarny.com>
This was referenced Sep 4, 2026
yuanchen8911
added a commit
to yuanchen8911/aicr
that referenced
this pull request
Sep 4, 2026
…bility agentgateway-crds v1.3.1 declares 90 CEL validation rules of the form 'rule: matches(self, ...)' on duration strings with no maxLength. With the string length unbounded, the apiserver's estimated cost for those rules exceeds its budget - by 16.1x for the worst one. Kubernetes 1.37 enforces that budget where 1.36 did not, so installing the CRDs fails: CustomResourceDefinition "agentgatewaypolicies.agentgateway.dev" is invalid: ... estimated rule cost exceeds budget by factor of 16.1x This surfaced when NVIDIA#2583 bumped kindest/node to v1.37.0: every inference lane fails at agentgateway CRD install, while training lanes are unaffected because agentgateway is inference-only. It is not a defect introduced by that bump - the rules were always over budget, 1.36 simply did not enforce it. agentgateway v1.5.0 replaces all 90 of those CEL rules with plain 'maxLength: 32' plus 'pattern:' OpenAPI validation, which carries no CEL cost - exactly the remedy the apiserver error recommends. Both components are pinned only by registry defaultVersion and no overlay overrides them, so this reaches every inference recipe through the platform-inference mixin: 15 recipes across eks/gke/oke/bcm and h100/gb200/gb300/b200. Signed-off-by: Yuan Chen <yuanchen97@gmail.com>
mchmarny
added a commit
that referenced
this pull request
Sep 4, 2026
The kindest/node bump to v1.37.0 (#2583) put KWOK on a Kubernetes 1.37 apiserver, which enforces a stricter CEL cost budget on CRD validation rules. The pinned agentgateway/agentgateway-crds v1.3.1 charts declare 90 unbounded 'type: string' fields carrying x-kubernetes-validations rules, so 1.37 rejects the CRDs outright with 'estimated rule cost exceeds budget by factor of 16.1x'. Every inference recipe has been red on main since. Chart v1.4.0 added the maxLength bounds that make the cost estimable and v1.5.0 replaced those CEL rules with plain 'pattern:'. Verified against real apiservers via envtest: v1.3.1 is rejected on 1.37 and accepted on 1.36, while v1.5.0 is accepted on both, so the pin moves without gating and kind stays on 1.37. Also scope the controller's new write RBAC. v1.5.0 splits RBAC into a cluster-wide read role plus a write-capable '-deployer' ClusterRole covering Deployments, DaemonSets, Secrets, and ServiceAccounts, and binds it with a ClusterRoleBinding when rbac.gatewayNamespaces is empty. v1.3.1 granted no DaemonSet write at all, so adopting the chart unchanged would hand a network-facing controller a privileged-DaemonSet-on-every-node primitive. AICR provisions exactly one Gateway, in agentgateway-system, so pin rbac.gatewayNamespaces to that namespace: the write role is then bound by a namespaced RoleBinding and cluster-wide read is unchanged. Signed-off-by: Mark Chmarny <mark@chmarny.com>
mchmarny
added a commit
that referenced
this pull request
Sep 4, 2026
The kindest/node bump to v1.37.0 (#2583) put KWOK on a Kubernetes 1.37 apiserver, which enforces a stricter CEL cost budget on CRD validation rules. The pinned agentgateway/agentgateway-crds v1.3.1 charts declare 90 unbounded 'type: string' fields carrying x-kubernetes-validations rules, so 1.37 rejects the CRDs outright with 'estimated rule cost exceeds budget by factor of 16.1x'. Every inference recipe has been red on main since. Chart v1.4.0 added the maxLength bounds that make the cost estimable and v1.5.0 replaced those CEL rules with plain 'pattern:'. Verified against real apiservers via envtest: v1.3.1 is rejected on 1.37 and accepted on 1.36, while v1.5.0 is accepted on both, so the pin moves without gating and kind stays on 1.37. Also scope the controller's new write RBAC. v1.5.0 splits RBAC into a cluster-wide read role plus a write-capable '-deployer' ClusterRole covering Deployments, DaemonSets, Secrets, and ServiceAccounts, and binds it with a ClusterRoleBinding when rbac.gatewayNamespaces is empty. v1.3.1 granted no DaemonSet write at all, so adopting the chart unchanged would hand a network-facing controller a privileged-DaemonSet-on-every-node primitive. AICR provisions exactly one Gateway, in agentgateway-system, so pin rbac.gatewayNamespaces to that namespace: the write role is then bound by a namespaced RoleBinding and cluster-wide read is unchanged. Signed-off-by: Mark Chmarny <mark@chmarny.com>
10 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR contains the following updates:
v1.36.1→v1.37.0Warning
Some dependencies could not be looked up. Check the warning logs for more information.
Configuration
📅 Schedule: (UTC)
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR is behind base branch, or you tick the rebase/retry checkbox.
🔕 Ignore: Close this PR and you won't be reminded about this update again.
This PR has been generated by Mend Renovate.