Implmemnted Feature Windows Gameserver Move Alpha to beta#4560
Implmemnted Feature Windows Gameserver Move Alpha to beta#4560Sivasankaran25 wants to merge 20 commits into
Conversation
|
Build Failed 😭 Build Id: 1d89798a-9fcf-422e-a2a2-c65b60892f56 Status: FAILURE To get permission to view the Cloud Build view, join the agones-discuss Google Group. |
|
Build Failed 😭 Build Id: e35ec88a-3848-4a70-8f87-3fe1b11a06e5 Status: FAILURE To get permission to view the Cloud Build view, join the agones-discuss Google Group. |
|
/gcbrun |
|
Build Failed 😭 Build Id: 0894d8f3-6aa3-484e-8c51-b7d089f35370 Status: FAILURE To get permission to view the Cloud Build view, join the agones-discuss Google Group. |
|
Build Failed 😭 Build Id: f680848d-d334-4731-a973-ddd02c1184e4 Status: FAILURE To get permission to view the Cloud Build view, join the agones-discuss Google Group. |
|
Build Failed 😭 Build Id: 441eecfa-987c-407a-9c93-4dfaa26a41b6 Status: FAILURE To get permission to view the Cloud Build view, join the agones-discuss Google Group. |
|
Build Failed 😭 Build Id: c486f2ce-f194-47ec-a6cf-7a9112176a76 Status: FAILURE To get permission to view the Cloud Build view, join the agones-discuss Google Group. |
|
/gcbrun |
|
Build Failed 😭 Build Id: 03abe090-4b08-4114-85dc-109f562c5d16 Status: FAILURE To get permission to view the Cloud Build view, join the agones-discuss Google Group. |
igooch
left a comment
There was a problem hiding this comment.
GKE Autopilot does not support Windows Nodes https://docs.cloud.google.com/kubernetes-engine/docs/concepts/windows-server-gke#limitations. Try running tests on Standard only.
|
Build Failed 😭 Build Id: d9561616-5ef7-4585-9398-81a4d7bce436 Status: FAILURE To get permission to view the Cloud Build view, join the agones-discuss Google Group. |
|
Build Failed 😭 Build Id: d899272a-a35a-4aed-86fa-6eb15696a675 Status: FAILURE To get permission to view the Cloud Build view, join the agones-discuss Google Group. |
|
Build Failed 😭 Build Id: c4b35f9c-0c5d-421a-a377-e514d5730e33 Status: FAILURE To get permission to view the Cloud Build view, join the agones-discuss Google Group. |
|
Curious to hear how the work here is going? I can't see the clusters, but what is failing here? |
The |
|
Could we use something like what we did here to skip autopilot? agones/test/e2e/gameserver_test.go Line 263 in 00f9151 |
Yes Sure I will try |
|
Build Failed 😭 Build Id: 3ab22cd7-7fe0-41b2-b442-25b7e0366410 Status: FAILURE To get permission to view the Cloud Build view, join the agones-discuss Google Group. |
|
Build Failed 😭 Build Id: d9c984fe-6414-419b-97de-45d737e5cc2c Status: FAILURE To get permission to view the Cloud Build view, join the agones-discuss Google Group. |
| containers: | ||
| - name: simple-game-server | ||
| image: us-docker.pkg.dev/agones-images/examples/simple-game-server:0.43 | ||
| image: us-docker.pkg.dev/agones-images/examples/simple-game-server:0.42-windows_amd64-ltsc2019 |
There was a problem hiding this comment.
This shouldn't be necessary -- the manifest should load the windows image.
| {{% alert title="Warning" color="warning" %}} | ||
| Running `GameServers` on Windows nodes is currently Alpha, and any feedback | ||
| would be appreciated. | ||
| {{% alert title="Note" color="info" %}} |
There was a problem hiding this comment.
All doc changes need to be feature shortcoded - https://agones.dev/site/docs/contribute/documentation-editing-contribution/
| | linux | `amd64` | **Stable** | | ||
| | linux | `arm64` | Alpha | | ||
| | [windows] | `amd64` | Alpha | | ||
| | [windows] | `amd64` | Beta | |
There was a problem hiding this comment.
Same here - feature shortcoded.
|
|
||
| // windowsGameServerImage is the published windows/amd64 simple-game-server. | ||
| // Uses ltsc2019 — the currently available tag in agones-images registry. | ||
| const windowsGameServerImage = "us-docker.pkg.dev/agones-images/examples/simple-game-server:0.42-windows_amd64-ltsc2019" |
There was a problem hiding this comment.
I'd double check the manifest for this image - but pretty sure it's a multi-arch and should select the correct image by detecting the host arch
| # The test itself calls t.Skip() when CloudProduct == "gke-autopilot". | ||
| # Runs parallel with e2e-feature-gates to avoid adding wall-clock time. | ||
| # | ||
| - name: e2e-runner |
There was a problem hiding this comment.
🤔 if the test windows_test.go is in the e2e folder - why do we need this extra entry in this config? Shouldn't it just run in the standard e2e test suite?
There was a problem hiding this comment.
I'll reiterate this - we have tests that only run on autopilot, why do we need this extra build step? This should just be a single extra test that runs on standard GKE clusters, that tests the windows nodes.
|
Build Failed 😭 Build Id: e02d0b91-aa8d-4994-80d2-0e2797f14165 Status: FAILURE To get permission to view the Cloud Build view, join the agones-discuss Google Group. |
|
Build Failed 😭 Build Id: 67fb4e1b-8d4d-4cae-b37c-d4abd7c00ea8 Status: FAILURE To get permission to view the Cloud Build view, join the agones-discuss Google Group. |
|
Build Failed 😭 Build Id: 197cd300-465a-45f5-a92b-182bb44f0c56 Status: FAILURE To get permission to view the Cloud Build view, join the agones-discuss Google Group. |
| # The test itself calls t.Skip() when CloudProduct == "gke-autopilot". | ||
| # Runs parallel with e2e-feature-gates to avoid adding wall-clock time. | ||
| # | ||
| - name: e2e-runner |
There was a problem hiding this comment.
I'll reiterate this - we have tests that only run on autopilot, why do we need this extra build step? This should just be a single extra test that runs on standard GKE clusters, that tests the windows nodes.
|
|
||
| gs := framework.DefaultGameServer(framework.Namespace) | ||
| gs.Spec.Template.Spec.NodeSelector = map[string]string{ | ||
| "kubernetes.io/os": "windows", |
| - started-by-${_PARENT_BUILD_ID} | ||
| timeout: 5400s # 1.5h | ||
| queueTtl: 7200s # 2h // only one set of e2es should be running at once | ||
| logsBucket: gs://agones-build-logs |
There was a problem hiding this comment.
Don't remove the log bucket - that's why I can't see the logs.
| # | ||
| # Run Windows GameServer smoke test. | ||
| # Windows nodes are NOT supported on GKE Autopilot: | ||
| # https://cloud.google.com/kubernetes-engine/docs/concepts/windows-server-gke#limitations | ||
| # The test itself calls t.Skip() when CloudProduct == "gke-autopilot". | ||
| # Runs parallel with e2e-feature-gates to avoid adding wall-clock time. | ||
| # | ||
| - name: e2e-runner | ||
| args: | ||
| - ${_FEATURE_WITHOUT_GATE} | ||
| - ${_CLOUD_PRODUCT} | ||
| - ${_TEST_CLUSTER_NAME} | ||
| - ${_TEST_CLUSTER_LOCATION} | ||
| - ${_REGISTRY} | ||
| - ${_GS_TEST_IMAGE} | ||
| id: e2e-windows-smoke | ||
| waitFor: | ||
| - build-e2e | ||
| timeout: 900s |
There was a problem hiding this comment.
| # | |
| # Run Windows GameServer smoke test. | |
| # Windows nodes are NOT supported on GKE Autopilot: | |
| # https://cloud.google.com/kubernetes-engine/docs/concepts/windows-server-gke#limitations | |
| # The test itself calls t.Skip() when CloudProduct == "gke-autopilot". | |
| # Runs parallel with e2e-feature-gates to avoid adding wall-clock time. | |
| # | |
| - name: e2e-runner | |
| args: | |
| - ${_FEATURE_WITHOUT_GATE} | |
| - ${_CLOUD_PRODUCT} | |
| - ${_TEST_CLUSTER_NAME} | |
| - ${_TEST_CLUSTER_LOCATION} | |
| - ${_REGISTRY} | |
| - ${_GS_TEST_IMAGE} | |
| id: e2e-windows-smoke | |
| waitFor: | |
| - build-e2e | |
| timeout: 900s | |
| timeout: 900s |
| func TestWindowsCreateConnect(t *testing.T) { | ||
| t.Parallel() | ||
|
|
||
| if framework.CloudProduct == "gke-autopilot" { |
There was a problem hiding this comment.
Since you have this, it should run as part of the standard set of e2e tests -- no need for the extra build step.
|
Build Failed 😭 Build Id: 2d8351fd-4531-43fd-8c89-bfbe5c8192dd Status: FAILURE To get permission to view the Cloud Build view, join the agones-discuss Google Group. |
|
/gcbrun |
|
Build Failed 😭 Build Id: 1a5bebee-8115-4f71-a368-fb94b3d61bac Status: FAILURE To get permission to view the Cloud Build view, join the agones-discuss Google Group. |
There was a problem hiding this comment.
Thanks for moving Windows GameServers to Beta! The docs and support-tier changes look right, and the Autopilot handling is correct — the test is skipped on Autopilot and the node pool is only added to the standard cluster.
However, TestWindowsCreateConnect will fail consistently on the standard e2e cluster (where it actually runs) for two reasons. Fixes are suggested inline, plus one blocker that lives outside this diff is that no Windows image is built for e2e:
cloudbuild.yaml:188 builds and pushes the e2e simple-game-server image with WITH_WINDOWS=0, so its manifest is linux/amd64-only (examples/simple-game-server/Makefile only adds the windows_amd64 layers when WITH_WINDOWS=1). The test comment claims framework.GameServerImage is multi-arch with a windows/amd64 variant — that isn't true in CI, so a Windows-pinned pod has no runnable image and never reaches Ready. This needs WITH_WINDOWS=1 for the e2e image build step:
- 'WITH_WINDOWS=1'Inline suggestions
- Add the Windows node toleration and prefer the
SkipOnCloudProducthelper (test file). - Set the Windows taint on the node pool and restore the missing trailing newline (
module.tf).
Nice work overall — once the e2e image includes a Windows variant and the pod can tolerate/schedule onto the Windows node, this should go green.
| if framework.CloudProduct == "gke-autopilot" { | ||
| t.Skip("Windows nodes are not supported on GKE Autopilot") | ||
| } | ||
|
|
||
| gs := framework.DefaultGameServer(framework.Namespace) | ||
| gs.Spec.Template.Spec.NodeSelector = map[string]string{ | ||
| "kubernetes.io/os": "windows", | ||
| } | ||
| // framework.GameServerImage is multi-arch (linux/amd64 + windows/amd64), | ||
| // so no separate Windows-tagged image is required - the correct | ||
| // platform variant is selected automatically based on the node. | ||
| gs.Spec.Template.Spec.Containers[0].Image = framework.GameServerImage |
There was a problem hiding this comment.
🔴 GKE taints Windows nodes node.kubernetes.io/os=windows:NoSchedule, but DefaultGameServer sets no toleration, so the pod can't schedule onto the Windows node. Also prefer the existing SkipOnCloudProduct helper, and drop the multi-arch comment — the e2e image is built with WITH_WINDOWS=0 (see summary), so GameServerImage has no windows/amd64 variant today.
| if framework.CloudProduct == "gke-autopilot" { | |
| t.Skip("Windows nodes are not supported on GKE Autopilot") | |
| } | |
| gs := framework.DefaultGameServer(framework.Namespace) | |
| gs.Spec.Template.Spec.NodeSelector = map[string]string{ | |
| "kubernetes.io/os": "windows", | |
| } | |
| // framework.GameServerImage is multi-arch (linux/amd64 + windows/amd64), | |
| // so no separate Windows-tagged image is required - the correct | |
| // platform variant is selected automatically based on the node. | |
| gs.Spec.Template.Spec.Containers[0].Image = framework.GameServerImage | |
| framework.SkipOnCloudProduct(t, "gke-autopilot", "Windows nodes are not supported on GKE Autopilot") | |
| gs := framework.DefaultGameServer(framework.Namespace) | |
| gs.Spec.Template.Spec.NodeSelector = map[string]string{ | |
| "kubernetes.io/os": "windows", | |
| } | |
| gs.Spec.Template.Spec.Tolerations = []corev1.Toleration{{ | |
| Key: "node.kubernetes.io/os", | |
| Operator: corev1.TolerationOpEqual, | |
| Value: "windows", | |
| Effect: corev1.TaintEffectNoSchedule, | |
| }} | |
| gs.Spec.Template.Spec.Containers[0].Image = framework.GameServerImage |
|
Build Failed 😭 Build Id: f6275ab8-4205-4849-a395-7f559c635004 Status: FAILURE To get permission to view the Cloud Build view, join the agones-discuss Google Group. |
|
Build Failed 😭 Build Id: 5915080e-be9a-40a2-bc48-888ba68fb9ab Status: FAILURE To get permission to view the Cloud Build view, join the agones-discuss Google Group. |
What type of PR is this?
/kind feature
What this PR does / Why we need it:
Which issue(s) this PR fixes:
Closes #4523
Special notes for your reviewer: