Skip to content

fix(lb): act only on balancers labelled for the service - #62

Open
lexfrei wants to merge 1 commit into
fix/cleanup-single-deletefrom
fix/balancer-ownership
Open

lexfrei wants to merge 1 commit into
fix/cleanup-single-deletefrom
fix/balancer-ownership

Conversation

@lexfrei

@lexfrei lexfrei commented Sep 29, 2026

Copy link
Copy Markdown
Collaborator

robotlb now tells its own balancers apart from everything else in the Hetzner project, and a Service can no longer land on a balancer another Service uses.

New balancers get the namespace in their default name and an ownership label. Without the robotlb/balancer annotation the name is <service>.<namespace>. Both parts are DNS labels without dots, so the name is at most 127 characters (Hetzner allows 128) and can't collide with a name from an earlier release. The create request sets the robotlb/service-uid label. hcloud-cloud-controller-manager uses the same scheme with hcloud-ccm/service-uid.

robotlb finds a Service's balancer by that label and changes or deletes only a balancer labelled for that Service. A release never looks a balancer up by name. So a Service released again can't delete a balancer another Service has since created under its old name.

The label also replaces the balancer ID annotation from #47. It identifies the balancer the same way, can't go stale between creating the balancer and recording its ID, and saves a patch of the Service. robotlb/balancer-id was never released, so it is removed.

Balancers from earlier releases have no label, so a reconcile adopts them when they look like the Service's own. The check applies to an unlabelled balancer under the old name or the name the Service asks for: every target must be an IP, and at least one of them a node of the Service. robotlb adds the label and keeps the name and any labels the balancer already has. Adopting only balancers with no labels at all would be stricter, but a balancer someone labelled by hand would then be replaced by a new one with a new IP. While the Service has no target nodes, the check can't be made, and the Service waits with a warning event instead of replacing the old balancer with a new one and a new IP.

Any other balancer is never touched. Under the old name robotlb skips it, and the Service gets a new balancer under <service>.<namespace> without an event. Under the name the Service asks for, the Service gets a warning event instead:

  • labelled for another Service: the event names that Service's UID;
  • unlabelled: the event gives the hcloud load-balancer add-label command to hand it over.

Two balancers matching one Service now also produce an event instead of a silent skip.

Some behaviour changes, and the README upgrade notes cover each of them:

  • Services with the same name in different namespaces stop sharing a balancer. One keeps it, the others get a new balancer and a new IP. If they reconcile at the same time, the others may configure the shared balancer once more before they move.
  • Two Services can't share a balancer through the same annotation anymore. The second one gets the warning.
  • Changing robotlb/balancer after creation doesn't rename the balancer or create a new one.
  • A balancer is deleted only when it carries the label. The balancer of a Service deleted or changed from LoadBalancer before its first successful reconcile on this release stays in the project and keeps being billed. That includes the Services fix(service): release the balancer when a service stops needing it #47 releases on the first start. hcloud load-balancer list --selector '!robotlb/service-uid' lists them.
  • An old balancer without targets, for example because Hetzner refused every node, doesn't pass the check. Under the old name it's replaced by a new balancer with a new IP and stays behind unlabelled.
  • A Service restored with a new UID doesn't get its old balancer back automatically. If the old balancer has the name the Service asks for, the event names the old UID. If it has the name from an earlier release, the Service gets a new balancer and a new IP without a warning, and the old one stays behind. The README has the commands for both cases.

Some risk remains. Someone who can edit a Service in the cluster can still take over an unlabelled balancer that already targets the cluster's nodes. That is either another robotlb balancer that hasn't been migrated yet, until its own Service reconciles, or a balancer someone made by hand for this cluster. Two clusters in one project with overlapping private subnets can also match each other's unlabelled balancers. Before this change, any of those was taken by name alone.

Stacked on #57.

Closes #41
Closes #43

A balancer was found only by its name, and without the robotlb/balancer
annotation that name was the service name without the namespace. Services
with the same name in two namespaces shared one balancer, overwrote each
other's ports and targets, and releasing one deleted the other's. Any
balancer whose name a service resolved to, including one named in the
annotation, could be reconfigured or deleted, even one robotlb never
created.

New balancers are named <service>.<namespace> and carry the
robotlb/service-uid label from the create request on. The balancer of a
service is found by that label, and only a labelled balancer is changed
or deleted. A release never goes by name, so a service released again
cannot hit a balancer another service created under the old name since.

The label replaces the robotlb/balancer-id annotation, which recorded
the balancer ID for the release: the label identifies the balancer the
same way, cannot go stale between creating a balancer and recording it,
and saves a patch of the service. The annotation was never released.

Balancers from earlier releases have no label. A reconcile adopts an
unlabelled balancer under the old name or the name the service asks for
only when every target is an IP and one of them is a node of the
service, so a balancer pointing at other hosts is never taken over.
While the service has no target nodes, an unlabelled balancer with IP
targets cannot be judged, and the service waits with a warning event
instead of replacing it. Otherwise, under the old name a balancer that
is not adopted is skipped, and the service gets a new balancer under
<service>.<namespace> without an event. Under the name the service asks
for it is left alone with a warning event: one labelled for another
service names that service's UID, an unlabelled one names the command
to hand it over.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant