A user changes an Unleash instance's release channel in Console. The displayed version stays on the old value with a green checkmark, nothing indicates work is happening, and then at some point it flips. A failed upgrade looks the same as a slow one for five minutes.
This traces the signal from the CRD to the browser and identifies where it is lost. Not where it was assumed to be.
The coordination needed is two repos, not three
The natural assumption is Console + nais-api + bifrost. bifrost is not involved.
- bifrost is not in the read path at all. nais-api runs a Kubernetes informer directly against the
unleashes CRD in the management cluster — internal/unleash/dataloader.go:45-49, queries.go:39-41. Console's instance data never passes through bifrost.
- bifrost also does not drop anything. Its handlers serialize the raw CRD (
pkg/api/http/v1/handlers/unleash.go:70, :99, :371), so conditions, resolvedReleaseChannelImage and metadata.generation all go over the wire.
So: nais-api and console-frontend. unleasherator is optional for the base fix but needed for a decent failure story.
Where it dies
nais-api/internal/unleash/models.go:33-66 — toUnleashInstance, run inside the watcher's converter, so the cache never holds the dropped fields:
instance := &UnleashInstance{
Name: u.Name,
Version: u.Status.Version,
Ready: u.Status.Reconciled && u.Status.Connected,
...
}
Three of six status fields are read. Dropped by name:
| Field |
Why it matters |
Status.ResolvedReleaseChannelImage |
The target image. This is the field that makes "upgrading" renderable, and it is discarded here. |
Status.Conditions |
Every reason, message and transition time — including the failure message quoted below. |
Status.ReleaseChannelName |
The observed channel. nais-api reads the spec side instead (models.go:62-64), so Console shows desired state that flips instantly. |
The CRD's own comment at unleash_types.go:286-288 says "Rather than relying on this value, check the conditions instead." nais-api relies on the booleans and drops the conditions.
The GraphQL schema then has nowhere to put progress: internal/graph/schema/unleash.graphqls:112-142 compresses the entire rollout lifecycle into ready: Boolean!.
A second, smaller drop: bifrost returns the channel's full Image; models.go:208-223 discards it.
Why the version "flips suddenly"
In unleasherator, Status.Version is only written after a successful live connection to the server (unleash_controller.go:1069-1075) and is never cleared or marked stale. So it holds the old value from the moment the spec changes until the rollout completes, then jumps.
During the entire window the CRD reads reconciled: true, connected: true, version: <old> — byte-identical to steady state except for resolvedReleaseChannelImage. waitForDeployment blocks for up to 5 minutes (:58, :929-943) writing no status at all.
So there is no intermediate state to show — but there is a usable target-vs-actual gap, and it is exactly the field nais-api drops.
Smallest coherent fix
nais-api — roughly three struct fields, three schema fields, three assignments:
resolvedReleaseChannelImage: String on UnleashInstance (already on the watched CRD)
statusMessage: String from conditions[Reconciled].message
image: String on UnleashReleaseChannel
console-frontend — compare target tag to version; when they differ show "Upgrading to vX…" and poll while they differ. Both pieces already exist there: an extractVersion() tag parser (+page.svelte:594-607), and a conditional-polling pattern with a document.hidden guard in WorkloadHealth.svelte:152-168.
Today Console fires the mutation and does one refetch (+page.svelte:150-170) — which, given the above, is guaranteed to read the old version. A polling helper exists at :188-193 but is only wired to instance creation, and stops when ready is true — which it already is during an upgrade.
Two traps for whoever picks this up
Do not use releaseChannel.currentVersion as the target. It is derived from instances whose resolved image matches (releasechannel_controller.go:1693-1711) — which precedes the rollout — while the version is harvested from such an instance. It can advertise the old version as current. Use the image tag.
Do not build on the mutation response. The generated bifrost client types status as {connected, version} only (bifrostclient/client.gen.go:108-115), so Reconciled is unreachable and ready is always false on mutation responses (models.go:226-288). Rely on the watcher-backed query.
What this does not fix
- A failing upgrade still looks like a slow one for five minutes, then becomes a bare X with the old version. The useful text —
"Deployment rollout timed out after 5m0s" — is already sitting in conditions[Reconciled].message; exposing statusMessage turns an unexplained X into a sentence, but does not make it appear sooner.
Degraded is never set for upgrade failure — only on the deletion path (unleash_controller.go:149,172). And the Unleash controller emits no Kubernetes Events at all.
- No granularity. A badge and a target version; no pod counts, no ETA.
Related findings worth their own issues
Filed separately, but they bear on any progress UI built here:
- Unleash conditions never carry
ObservedGeneration, so nothing downstream can tell whether Reconciled=True refers to the new spec or the old one. RemoteUnleash already does this correctly (remoteunleash_controller.go:401) — the fix is one line, copied.
- A per-instance progress schema already exists and is dead.
ReleaseChannelStatus.InstanceStatus with Phase/StartTime/EndTime/Message/Ready (releasechannel_types.go:133-134,184-202) is never written by any controller. The domain model for this feature was designed and never shipped.
- Switching into a channel that is mid-rollout silently gives you the channel's old image (
internal/resources/unleash.go:567-588), with nothing anywhere saying so. A user could pick a channel and correctly conclude nothing happened — because nothing did.
A user changes an Unleash instance's release channel in Console. The displayed version stays on the old value with a green checkmark, nothing indicates work is happening, and then at some point it flips. A failed upgrade looks the same as a slow one for five minutes.
This traces the signal from the CRD to the browser and identifies where it is lost. Not where it was assumed to be.
The coordination needed is two repos, not three
The natural assumption is Console + nais-api + bifrost. bifrost is not involved.
unleashesCRD in the management cluster —internal/unleash/dataloader.go:45-49,queries.go:39-41. Console's instance data never passes through bifrost.pkg/api/http/v1/handlers/unleash.go:70,:99,:371), so conditions,resolvedReleaseChannelImageandmetadata.generationall go over the wire.So: nais-api and console-frontend. unleasherator is optional for the base fix but needed for a decent failure story.
Where it dies
nais-api/internal/unleash/models.go:33-66—toUnleashInstance, run inside the watcher's converter, so the cache never holds the dropped fields:Three of six status fields are read. Dropped by name:
Status.ResolvedReleaseChannelImageStatus.ConditionsStatus.ReleaseChannelNamemodels.go:62-64), so Console shows desired state that flips instantly.The CRD's own comment at
unleash_types.go:286-288says "Rather than relying on this value, check the conditions instead." nais-api relies on the booleans and drops the conditions.The GraphQL schema then has nowhere to put progress:
internal/graph/schema/unleash.graphqls:112-142compresses the entire rollout lifecycle intoready: Boolean!.A second, smaller drop: bifrost returns the channel's full
Image;models.go:208-223discards it.Why the version "flips suddenly"
In unleasherator,
Status.Versionis only written after a successful live connection to the server (unleash_controller.go:1069-1075) and is never cleared or marked stale. So it holds the old value from the moment the spec changes until the rollout completes, then jumps.During the entire window the CRD reads
reconciled: true, connected: true, version: <old>— byte-identical to steady state except forresolvedReleaseChannelImage.waitForDeploymentblocks for up to 5 minutes (:58,:929-943) writing no status at all.So there is no intermediate state to show — but there is a usable target-vs-actual gap, and it is exactly the field nais-api drops.
Smallest coherent fix
nais-api — roughly three struct fields, three schema fields, three assignments:
resolvedReleaseChannelImage: StringonUnleashInstance(already on the watched CRD)statusMessage: Stringfromconditions[Reconciled].messageimage: StringonUnleashReleaseChannelconsole-frontend — compare target tag to
version; when they differ show "Upgrading to vX…" and poll while they differ. Both pieces already exist there: anextractVersion()tag parser (+page.svelte:594-607), and a conditional-polling pattern with adocument.hiddenguard inWorkloadHealth.svelte:152-168.Today Console fires the mutation and does one refetch (
+page.svelte:150-170) — which, given the above, is guaranteed to read the old version. A polling helper exists at:188-193but is only wired to instance creation, and stops whenreadyis true — which it already is during an upgrade.Two traps for whoever picks this up
Do not use
releaseChannel.currentVersionas the target. It is derived from instances whose resolved image matches (releasechannel_controller.go:1693-1711) — which precedes the rollout — while the version is harvested from such an instance. It can advertise the old version as current. Use the image tag.Do not build on the mutation response. The generated bifrost client types
statusas{connected, version}only (bifrostclient/client.gen.go:108-115), soReconciledis unreachable andreadyis always false on mutation responses (models.go:226-288). Rely on the watcher-backed query.What this does not fix
"Deployment rollout timed out after 5m0s"— is already sitting inconditions[Reconciled].message; exposingstatusMessageturns an unexplained X into a sentence, but does not make it appear sooner.Degradedis never set for upgrade failure — only on the deletion path (unleash_controller.go:149,172). And the Unleash controller emits no Kubernetes Events at all.Related findings worth their own issues
Filed separately, but they bear on any progress UI built here:
ObservedGeneration, so nothing downstream can tell whetherReconciled=Truerefers to the new spec or the old one.RemoteUnleashalready does this correctly (remoteunleash_controller.go:401) — the fix is one line, copied.ReleaseChannelStatus.InstanceStatuswithPhase/StartTime/EndTime/Message/Ready(releasechannel_types.go:133-134,184-202) is never written by any controller. The domain model for this feature was designed and never shipped.internal/resources/unleash.go:567-588), with nothing anywhere saying so. A user could pick a channel and correctly conclude nothing happened — because nothing did.