Skip to content

chore: backport 20250602 (v0.18.7) #1318

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
michaelawyu merged 5 commits into from
Jun 3, 2026
Merged

chore: backport 20250602 (v0.18.7) #1318

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
678eb6a
chore: bump distroless/base from `fb282f8` to `a557d78` in /docker (#…
dependabot[bot] May 29, 2026
97118e7
fix: fix subsequent failure metrics and revert some changes (#717)
britaniar May 29, 2026
660c868
feat: migrating to validating admission policies for our validating l…
michaelawyu Jun 1, 2026
2b79699
chore: Setup squad as a repo assistant (#724)
serbrech Jun 1, 2026
7e69fd7
Merge remote-tracking branch 'cncf/main' into release/vap-special
michaelawyu Jun 2, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
70 changes: 37 additions & 33 deletions charts/hub-agent/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -89,39 +89,43 @@ _See [helm install](https://helm.sh/docs/helm/helm_install/) for command documen

## Parameters

| Parameter | Description | Default |
|:------------------------------------------|:------------------------------------------------------------------------------------------|:-------------------------------------------------|
| `replicaCount` | Number of hub-agent replicas to deploy | `1` |
| `image.repository` | Image repository | `ghcr.io/kubefleet-dev/kubefleet/hub-agent` |
| `image.pullPolicy` | Image pull policy | `Always` |
| `image.tag` | Image release tag (empty uses chart `appVersion`) | `""` |
| `namespace` | Namespace where this chart is installed | `fleet-system` |
| `resources` | Resource requests/limits for the container | limits: 500m CPU, 1Gi; requests: 100m CPU, 128Mi |
| `affinity` | Node affinity for hub-agent pods | `{}` |
| `tolerations` | Tolerations for hub-agent pods | `[]` |
| `logVerbosity` | Log level (klog V logs) | `5` |
| `enableWebhook` | Enable webhook server | `true` |
| `webhookServiceName` | Webhook service name | `fleetwebhook` |
| `enableGuardRail` | Enable guard rail webhook configurations | `true` |
| `webhookClientConnectionType` | Connection type for webhook client (service or url) | `service` |
| `useCertManager` | Use cert-manager for webhook certificate management (requires `enableWorkload=true`) | `false` |
| `webhookCertSecretName` | Name of the Secret where cert-manager stores the certificate (required when enabled) | `unset` |
| `enableClusterInventoryAPI` | Enable cluster inventory APIs | `true` |
| `enableStagedUpdateRunAPIs` | Enable staged update run APIs | `true` |
| `enableEvictionAPIs` | Enable eviction APIs | `true` |
| `enablePprof` | Enable pprof endpoint | `true` |
| `pprofPort` | pprof server port | `6065` |
| `hubAPIQPS` | QPS for fleet-apiserver (not including events/node heartbeat) | `250` |
| `hubAPIBurst` | Burst for fleet-apiserver (not including events/node heartbeat) | `1000` |
| `MaxConcurrentClusterPlacement` | Max concurrent ClusterResourcePlacement operations | `100` |
| `ConcurrentResourceChangeSyncs` | Max concurrent resourceChange reconcilers | `20` |
| `logFileMaxSize` | Max log file size before rotation (optional) | `unset` |
| `MaxFleetSizeSupported` | Max number of member clusters supported | `100` |
| `forceDeleteWaitTime` | Grace period before force-deleting resources | `15m0s` |
| `clusterUnhealthyThreshold` | Threshold duration for marking a cluster unhealthy | `3m0s` |
| `resourceSnapshotCreationMinimumInterval` | The minimum interval at which resource snapshots could be created. | `30s` |
| `resourceChangesCollectionDuration` | The duration for collecting resource changes into one snapshot. | `15s` |
| `enableWorkload` | Enable kubernetes builtin workload to run in hub cluster. | `false` |
| Parameter | Description | Default |
|:----------|:------------|:--------|
| `replicaCount` | Number of hub-agent replicas to deploy | `1` |
| `image.repository` | Image repository | `ghcr.io/kubefleet-dev/kubefleet/hub-agent` |
| `image.pullPolicy` | Image pull policy | `Always` |
| `image.tag` | Image release tag (empty uses chart `appVersion`) | `""` |
| `namespace` | Namespace where this chart is installed | `fleet-system` |
| `resources` | Resource requests/limits for the container | limits: 500m CPU, 1Gi; requests: 100m CPU, 128Mi |
| `affinity` | Node affinity for hub-agent pods | `{}` |
| `tolerations` | Tolerations for hub-agent pods | `[]` |
| `logVerbosity` | Log level (klog V logs) | `5` |
| `enableWebhook` | Enable webhook server | `true` |
| `webhookServiceName` | Webhook service name | `fleetwebhook` |
| `enableGuardRail` | Enable guard rail webhook configurations | `true` |
| `webhookClientConnectionType` | Connection type for webhook client (service or url) | `service` |
| `useCertManager` | Use cert-manager for webhook certificate management (requires `enableWorkload=true`) | `false` |
| `webhookCertSecretName` | Name of the Secret where cert-manager stores the certificate (required when enabled) | `unset` |
| `enableClusterInventoryAPI` | Enable cluster inventory APIs | `true` |
| `enableStagedUpdateRunAPIs` | Enable staged update run APIs | `true` |
| `enableEvictionAPIs` | Enable eviction APIs | `true` |
| `enablePprof` | Enable pprof endpoint | `true` |
| `pprofPort` | pprof server port | `6065` |
| `hubAPIQPS` | QPS for fleet-apiserver (not including events/node heartbeat) | `250` |
| `hubAPIBurst` | Burst for fleet-apiserver (not including events/node heartbeat) | `1000` |
| `MaxConcurrentClusterPlacement` | Max concurrent ClusterResourcePlacement operations | `100` |
| `ConcurrentResourceChangeSyncs` | Max concurrent resourceChange reconcilers | `20` |
| `logFileMaxSize` | Max log file size before rotation (optional) | `unset` |
| `MaxFleetSizeSupported` | Max number of member clusters supported | `100` |
| `forceDeleteWaitTime` | Grace period before force-deleting resources | `15m0s` |
| `clusterUnhealthyThreshold` | Threshold duration for marking a cluster unhealthy | `3m0s` |
| `resourceSnapshotCreationMinimumInterval` | The minimum interval at which resource snapshots could be created | `30s` |
| `resourceChangesCollectionDuration` | The duration for collecting resource changes into one snapshot | `15s` |
| `enableWorkload` | Enable kubernetes builtin workload to run in hub cluster | `false` |
| `additionalConfigData` | Additional key-value data to include in the hub agent config map | `{}` |
| `additionalConfigDataMountPath` | Mount path for the additional config data volume | `/etc/kubefleet/additional-config` |
| `enableAdmissionPolicyManager` | Enable the admission policy manager to enforce VAP-based policies on the hub cluster | `false` |
| `admissionPolicyManagerConfigName` | Name of the key that contains the admission policy manager configuration in the hub agent config map | `""` |

## Certificate Management

Expand Down
11 changes: 11 additions & 0 deletions charts/hub-agent/templates/config.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
{{- if .Values.additionalConfigData }}
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ include "hub-agent.fullname" . }}-config
namespace: {{ .Values.namespace }}
labels:
{{- include "hub-agent.labels" . | nindent 4 }}
data:
{{- .Values.additionalConfigData | toYaml | nindent 2 }}
{{- end }}
22 changes: 22 additions & 0 deletions charts/hub-agent/templates/deployment.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,13 @@ spec:
- --cluster-unhealthy-threshold={{ .Values.clusterUnhealthyThreshold }}
- --resource-snapshot-creation-minimum-interval={{ .Values.resourceSnapshotCreationMinimumInterval }}
- --resource-changes-collection-duration={{ .Values.resourceChangesCollectionDuration }}
- --enable-admission-policy-manager={{ .Values.enableAdmissionPolicyManager }}
{{- if and .Values.admissionPolicyManagerConfigName (not .Values.additionalConfigData) }}
{{- fail "ERROR: admissionPolicyManagerConfigName is set but additionalConfigData is empty; must provide admission policy manager configuration data" }}
{{- end }}
{{- if and .Values.additionalConfigData .Values.admissionPolicyManagerConfigName }}
- --admission-policy-manager-config={{ .Values.additionalConfigDataMountPath }}/{{ .Values.admissionPolicyManagerConfigName }}
{{- end }}
ports:
- name: metrics
containerPort: 8080
Expand Down Expand Up @@ -105,6 +112,11 @@ spec:
# This path must match FleetWebhookCertDir in pkg/webhook/webhook.go
mountPath: /tmp/k8s-webhook-server/serving-certs
readOnly: true
{{- if .Values.additionalConfigData }}
- name: additional-config
mountPath: {{ .Values.additionalConfigDataMountPath }}
readOnly: true
{{- end }}
{{- else }}
volumeMounts:
- name: webhook-cert
Expand All @@ -113,6 +125,11 @@ spec:
# the read only root filesystem setup would block the agent from attempting to
# clear the directory.
mountPath: /tmp/k8s-webhook-server/
{{- if .Values.additionalConfigData }}
- name: additional-config
mountPath: {{ .Values.additionalConfigDataMountPath }}
readOnly: true
{{- end }}
{{- end }}
volumes:
- name: webhook-cert
Expand All @@ -125,6 +142,11 @@ spec:
{{- else }}
emptyDir: {}
{{- end }}
{{- if .Values.additionalConfigData }}
- name: additional-config
configMap:
name: {{ include "hub-agent.fullname" . }}-config
{{- end }}
{{- with .Values.affinity }}
affinity:
{{- toYaml . | nindent 8 }}
Expand Down
13 changes: 13 additions & 0 deletions charts/hub-agent/values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,3 +66,16 @@ hubAPIBurst: 1000
MaxConcurrentClusterPlacement: 100
ConcurrentResourceChangeSyncs: 20
MaxFleetSizeSupported: 100

additionalConfigData: {}
additionalConfigDataMountPath: /etc/kubefleet/additional-config

# Local chart specific changes:
# Admission policy manager is enabled by default in the managed solution; the
# chart default value is overridden here to verify this behavior.
enableAdmissionPolicyManager: true

# Local chart specific changes:
# Due to release complications, the admission policy manager in the managed solution
# always use the default configuration as defined in the code.
admissionPolicyManagerConfigName: ""
50 changes: 46 additions & 4 deletions cmd/hubagent/main.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,30 +27,33 @@ import (
apiextensionsv1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1"
"k8s.io/apimachinery/pkg/runtime"
utilruntime "k8s.io/apimachinery/pkg/util/runtime"
"k8s.io/apimachinery/pkg/util/yaml"
clientgoscheme "k8s.io/client-go/kubernetes/scheme"
"k8s.io/client-go/rest"
"k8s.io/klog/v2"
clusterinventory "sigs.k8s.io/cluster-inventory-api/apis/v1alpha1"
ctrl "sigs.k8s.io/controller-runtime"
"sigs.k8s.io/controller-runtime/pkg/cache"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/healthz"
"sigs.k8s.io/controller-runtime/pkg/log/zap"
"sigs.k8s.io/controller-runtime/pkg/manager"
metricsserver "sigs.k8s.io/controller-runtime/pkg/metrics/server"
ctrlwebhook "sigs.k8s.io/controller-runtime/pkg/webhook"

fleetnetworkingv1alpha1 "go.goms.io/fleet-networking/api/v1alpha1"
"go.goms.io/fleet/pkg/webhook/managedresource"

clusterv1beta1 "go.goms.io/fleet/apis/cluster/v1beta1"
placementv1alpha1 "go.goms.io/fleet/apis/placement/v1alpha1"
placementv1beta1 "go.goms.io/fleet/apis/placement/v1beta1"
"go.goms.io/fleet/cmd/hubagent/options"
"go.goms.io/fleet/cmd/hubagent/workload"
"go.goms.io/fleet/pkg/admissionpolicymanager"
mcv1beta1 "go.goms.io/fleet/pkg/controllers/membercluster/v1beta1"
readiness "go.goms.io/fleet/pkg/utils/informer/readiness"
"go.goms.io/fleet/pkg/utils/validator"
"go.goms.io/fleet/pkg/webhook"
"go.goms.io/fleet/pkg/webhook/managedresource"
// +kubebuilder:scaffold:imports
)

Expand Down Expand Up @@ -161,7 +164,7 @@ func main() {
exitWithErrorFunc()
}

klog.V(2).InfoS("starting hubagent")
klog.V(2).InfoS("starting hub agent")
if opts.FeatureFlags.EnableV1Beta1APIs {
klog.Info("Setting up memberCluster v1beta1 controller")
if err = (&mcv1beta1.Reconciler{
Expand All @@ -184,7 +187,7 @@ func main() {
exitWithErrorFunc()
}

if opts.WebhookOpts.EnableWebhooks {
if opts.WebhookAndAdmissionPolicyOpts.EnableWebhooks {
// Generate webhook configuration with certificates
webhookConfig, err := webhook.NewWebhookConfigFromOptions(mgr, opts, FleetWebhookPort)
if err != nil {
Expand All @@ -209,7 +212,7 @@ func main() {
// When using cert-manager, add a readiness check to ensure CA bundles are injected before marking ready.
// This prevents the pod from accepting traffic before cert-manager has populated the webhook CA bundles,
// which would cause webhook calls to fail.
if opts.WebhookOpts.UseCertManager {
if opts.WebhookAndAdmissionPolicyOpts.UseCertManager {
if err := mgr.AddReadyzCheck("cert-manager-ca-injection", func(req *http.Request) error {
return webhookConfig.CheckCAInjection(req.Context())
}); err != nil {
Expand All @@ -221,6 +224,45 @@ func main() {
}

ctx := ctrl.SetupSignalHandler()

if opts.WebhookAndAdmissionPolicyOpts.EnableAdmissionPolicyManager {
policyManagerCfgs := admissionpolicymanager.DefaultPolicyGeneratorConfigs
if len(opts.WebhookAndAdmissionPolicyOpts.AdmissionPolicyManagerConfig) != 0 {
// Read user-provided admission policy manager config from given path.
cfgData, err := os.ReadFile(opts.WebhookAndAdmissionPolicyOpts.AdmissionPolicyManagerConfig)
if err != nil {
klog.ErrorS(err, "failed to read the admission policy manager config file")
exitWithErrorFunc()
}

policyManagerCfgs = &admissionpolicymanager.PolicyGeneratorConfigs{}
if err := yaml.Unmarshal(cfgData, policyManagerCfgs); err != nil {
klog.ErrorS(err, "failed to unmarshal the admission policy manager config file")
exitWithErrorFunc()
}

// Note that validation has been performed when the flags are parsed.
}

// Create a separate client for the admission policy manager to use, as the cached client from
// the controller manager side is not initialized yet at this point.
hubUncachedClient, err := client.New(defaultCfg, client.Options{Scheme: scheme})
if err != nil {
klog.ErrorS(err, "failed to create uncached client for the admission policy manager")
exitWithErrorFunc()
}
policyMgr, err := admissionpolicymanager.New(hubUncachedClient, policyManagerCfgs)
if err != nil {
klog.ErrorS(err, "failed to create the admission policy manager")
exitWithErrorFunc()
}

if err := policyMgr.Start(ctx); err != nil {
klog.ErrorS(err, "failed to start the admission policy manager")
exitWithErrorFunc()
}
}

if err := workload.SetupControllers(ctx, &wg, mgr, defaultCfg, opts); err != nil {
klog.ErrorS(err, "unable to set up controllers")
exitWithErrorFunc()
Expand Down
6 changes: 3 additions & 3 deletions cmd/hubagent/options/options.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,8 +28,8 @@ type Options struct {
// Options that concern the setup of the controller manager instance in use by the KubeFleet hub agent.
CtrlMgrOpts ControllerManagerOptions

// KubeFleet webhook related options.
WebhookOpts WebhookOptions
// KubeFleet webhook and admission policy related options.
WebhookAndAdmissionPolicyOpts WebhookAndAdmissionPolicyOptions

// Feature flags that control the enabling of certain features in the hub agent.
FeatureFlags FeatureFlags
Expand All @@ -51,7 +51,7 @@ func NewOptions() *Options {
func (o *Options) AddFlags(flags *flag.FlagSet) {
o.LeaderElectionOpts.AddFlags(flags)
o.CtrlMgrOpts.AddFlags(flags)
o.WebhookOpts.AddFlags(flags)
o.WebhookAndAdmissionPolicyOpts.AddFlags(flags)
o.FeatureFlags.AddFlags(flags)
o.ClusterMgmtOpts.AddFlags(flags)
o.PlacementMgmtOpts.AddFlags(flags)
Expand Down
Loading
Loading