Skip to content

Impersonate only ACE extras#437

Open
ArnobKumarSaha wants to merge 2 commits into
masterfrom
arnob-extra-allow
Open

Impersonate only ACE extras#437
ArnobKumarSaha wants to merge 2 commits into
masterfrom
arnob-extra-allow

Conversation

@ArnobKumarSaha

Copy link
Copy Markdown
Member

Problem

An EditorModel request from a spoke cluster imported into ACE through Rancher fails with:

appreleases.drivers.x-helm.dev "myc" is forbidden: User "system:serviceaccount:kubeops:kube-ui-server"
cannot impersonate-on:user-info:get resource "appreleases" in API group "drivers.x-helm.dev"
in the namespace "demo"

kube-ui-server reads the AppRelease as the calling user via impersonation, so the read is authorized against the caller's own RBAC. impersonableExtra dropped only the reserved authentication.kubernetes.io/* extras and forwarded everything else.

Kubernetes authorizes impersonated extras per key, as a subresource: extra key k needs impersonate on userextras/k in authentication.k8s.io. The installer chart grants exactly one — userextras/ace.appscode.com/org-id. Rancher authenticates its own user and forwards principalid, username, requesthost, requesttokenid; none is an ACE key, so the request was denied in impersonation authorization (round 1), before the impersonated user's RBAC was ever consulted.

Reproduced against the apiserver with the server's own SA token, which takes b3, Rancher and the console out of the picture:

$ curl -sk -H "Authorization: Bearer $TOKEN" \
       -H 'Impersonate-User: user-5klht' -H 'Impersonate-Extra-principalid: local://user-5klht' \
       https://<apiserver>/apis/drivers.x-helm.dev/v1alpha1/namespaces/demo/appreleases/myc

userextras.authentication.k8s.io "local://user-5klht" is forbidden: User
"system:serviceaccount:kubeops:kube-ui-server" cannot impersonate resource
"userextras/principalid" in API group "authentication.k8s.io" at the cluster scope

The confusing impersonate-on:user-info:get wording is Kubernetes 1.36's constrained impersonation (KEP-5284, beta/on by default) reporting the same denial through a different mode. Granting the constrained verbs would not help: constrained impersonation rejects extra keys that are not domain-prefixed, so bare keys like principalid can only ever go through the legacy path. Running a 1.35 spoke would not help either — it only changes the message.

Fix

Allowlist instead of denylist: forward only ace.appscode.com/* extras.

Every platform invents its own extras — Rancher's principalid, EKS' arn/sessionName, OpenShift's scopes.authorization.openshift.io, plus the apiserver's own authentication.kubernetes.io/*. Chasing each with wider chart RBAC never converges; an allowlist is correct on any provider and needs no installer release. RBAC ignores extras entirely, so this changes no authorization decision on an ordinary cluster.

The second commit bumps kmodules.xyz/client-go for the same fix in DelegatingClient.Impersonate (kmodules/client-go#645), which is the impersonation path used by pkg/graph.

Impact on pkg/graph

graph.NewClient (pkg/graph/setup.go:335) is the only route into DelegatingClient.Impersonate from this repo. Its callers: ExecRawQuery / ExecQuery (setup.go:253,289), _renderPageBlock (renderer.go:231), renderDashboard (dashboard.go:51) and renderdashboard/storage.go:135.

It bails out to the un-impersonated client unless the caller carries exactly one ace.appscode.com/org-id extra, which bounds the change:

  • Rancher / EKS / OpenShift callers — no change; these paths never impersonated them.
  • ACE callers carrying only org-id — no change.
  • ACE callers carrying org-id plus another extra (e.g. authentication.kubernetes.io/credential-id, injected by the apiserver for token auth on k8s >= 1.30) — previously a hard denial, now works.

It cannot break an impersonation that currently works: under the chart's RBAC any extra outside the single grant already caused a full denial, so the allowlist only turns denials into successes. The exception is a cluster whose ClusterRole was manually widened to userextras/* — there, extras no longer reach the apiserver, which matters only to a custom authorization webhook, since RBAC ignores extras.

Caveats for the reviewer

  • Merge order. go.mod pins kmodules.xyz/client-go v0.34.6-0.20260726164328-778b1ba80e3a, a pseudo-version of the branch behind Impersonate only ACE extras kmodules/client-go#645. That PR needs to merge and be tagged, and this one re-pinned to the tag, before merging.
  • kmapi.ClientKeyPrefix is client.ace.appscode.com/, which does not match the ace.appscode.com/ prefix. Nothing in this repo or its vendor tree uses it as an extra key, but if b3 or another producer sends one it will be dropped — that key would have been denied under the current chart RBAC anyway.

Refs

Picks up the ACE-extras allowlist in DelegatingClient.Impersonate
(kmodules/client-go#645).

Signed-off-by: Arnob Kumar Saha <arnob@appscode.com>
impersonableExtra dropped just the reserved authentication.kubernetes.io/*
keys and forwarded every other extra from the caller's identity. Kubernetes
authorizes impersonated extras per key, as userextras/<key>, and the chart
grants kube-ui-server exactly one of those -- ace.appscode.com/org-id. So a
caller carrying provider-injected extras had the whole EditorModel request
denied during impersonation authorization, before its own RBAC was consulted.

This is what a spoke imported through Rancher hits: Rancher forwards
principalid, username, requesthost and requesttokenid, none of them ACE keys.

Turn the denylist into an allowlist keeping only ace.appscode.com/* extras.
Every platform invents its own -- Rancher's principalid, EKS' arn, OpenShift's
scopes.authorization.openshift.io -- so widening the chart RBAC never
converges, while an allowlist is correct on any provider. RBAC ignores extras,
so this changes no authorization decision on an ordinary cluster.

Signed-off-by: Arnob Kumar Saha <arnob@appscode.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant