Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 0 additions & 19 deletions .yamato/project.metafile
Original file line number Diff line number Diff line change
Expand Up @@ -219,25 +219,6 @@ unified_test_platforms:
image: package-ci/ubuntu-22.04:v4.87.0
flavor: b1.large

# Restricts the unified job to unified-only tests so nothing else runs on the alpha editor.
#
# This is a REGULAR EXPRESSION, not a glob. UTR passes it to the editor as -testFilter, which ends up
# in UnityEngine.TestRunner's FullNameFilter -> NUnit ValueMatchFilter -> new Regex(pattern).IsMatch().
# A glob-style "*Unified*" throws "Quantifier {x,y} following nothing" and fails the whole run.
# The other jobs' "Unity.Netcode.RuntimeTests.*" works because it is also a valid regex - it just
# happens to read like a glob.
#
# The match is against the NUnit *full* test name, which includes both fixture and method arguments.
#
# DO NOT widen this to ".*Unified.*". "Unified" appearing in a test name does NOT mean that test was
# deliberately converted to run against hybrid prefabs. HostOrServer gained UnifiedServer/UnifiedHost
# members under UNIFIED_NETCODE, and NUnit expands a bare [Values] on an enum parameter (and
# enum-typed fixture constructors) to every member. So with N4E installed, much of the existing suite
# silently grows unified cases - e.g. NetworkVariableTests(Default).AllNetworkVariableTypes(UnifiedHost),
# which nobody wrote. ".*Unified.*" selected 138 tests, 84 of which failed, against the 1 test this
# job exists to validate. Deciding which of those should pass, and how they opt in, is Goal-2.
unified_test_filter: ".*UnifiedNetworkTransformTest.*"


# Scripting backends used by Standalone RunTimeTests---------------------------------------------------

Expand Down
52 changes: 16 additions & 36 deletions .yamato/unified-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,47 +2,17 @@
---

# DESCRIPTION--------------------------------------------------------------------------
# This job validates NGO running against Netcode for Entities (N4E) through the unified API.
# A "hybrid prefab" is an NGO prefab (NetworkObject at the root) that also carries a GhostObject.
# When one or more hybrid prefabs are in the NetworkManager's prefab list, NGO hands transform
# synchronization to N4E's snapshot system and tunnels its own batched messages over
# UnifiedNetcodeTransport (N4E's out-of-band RPC) instead of using a NetworkTransport such as UTP.
# UnifiedNetworkTransformTest is the validation test that this path works end to end.

# WHY THIS JOB IS SEPARATE FROM EVERY OTHER JOB------------------------------------------
# 1. It needs an editor that bundles a com.unity.netcode with the unified API. That editor
# (unified_editors.default in project.metafile) is not one of the validation_editors, and NGO
# still has to keep building and testing against editors that have no unified API at all.
# 2. It needs com.unity.netcode in the testproject so that the UNIFIED_NETCODE define is set
# (see the versionDefines in Unity.Netcode.Runtime.asmdef). The committed
# testproject/Packages/manifest.json deliberately does NOT reference it, so this job swaps in
# testproject/Packages/manifest-unified.json instead.
# 3. It is wired into pr_code_changes_checks and develop_nightly the same way the CMB service tests
# are, so it runs automatically on PRs targeting develop / develop-3.x.x / release. Be aware that
# this makes an unsupported alpha editor part of the PR gate: when N4E lands breaking changes in
# trunk this job goes red and the pin has to be bumped to unblock PRs.
# _triggers.yml also has unified_pr_checks, so it can be kicked off with "/ci unified" on PRs
# that pr_code_changes_checks does not cover.
# The NGO job that runs unified netcode hybrid prefab based integration tests.

# CONFIGURATION STRUCTURE--------------------------------------------------------------
# Jobs are generated using nested loops:
# 1. For all unified test platforms (currently Ubuntu only, see project.metafile)
# 2. For all unified editors (currently a single pinned alpha, see project.metafile)

# TECHNICAL CONSIDERATIONS---------------------------------------------------------------
# This job runs in Editor context only (no player builds required), like project-tests.yml.
# Only playmode is run: every unified test is an integration test and there are no unified EditMode tests.
# The run is restricted to unified tests via unified_test_filter so that nothing else is exercised
# on the alpha editor. Non-unified tests are covered by the regular jobs on the supported editors.
# packages-lock.json is removed because the committed lock was resolved against a much older editor
# and its builtin package versions do not exist in the unified editor.

# QUALITY CONSIDERATIONS--------------------------------------------------------------------
# TODO: the manifest swap means testproject/Packages/manifest-unified.json has to be kept in sync
# with manifest.json by hand. Revisit once N4E is a hard dependency of NGO and one manifest
# can cover both cases.
# TODO: unified_test_filter is a test-name pattern. Replacing it with an NUnit category (for
# example [Category("Unified")]) would be less fragile once more fixtures gain unified variants.

#------------------------------------------------------------------------------------

Expand All @@ -58,18 +28,28 @@ unified_test_{{ project.name }}_{{ platform.name }}_{{ editor }}:
{% if platform.model %}
model: {{ platform.model }} # This is set only in platforms where we want non-default model to use (more information in project.metafile)
{% endif %}

variables:
# When UNIFIED_TESTS is set, only integration tests that opt in via overriding NetcodeIntegrationTest.UseUnifiedTests and returning true will run during this job.
# NetcodeIntegrationTest.UseUnifiedTests defaults to returning false (for now) making this an opt-in mode.
# Note: If USE_CMB_SERVICE is set along with UNIFIED_TESTS, then UNIFIED_TESTS is ignored until distributed authority is supported.
UNIFIED_TESTS: "true"

commands:
# Swap in the manifest that pulls in com.unity.netcode (N4E). This is what causes UNIFIED_NETCODE
# to be defined and therefore what makes the unified tests compile at all.
# Swap in the manifest that has the unified packages and versions required.
# Without the N4E package, UNIFIED_NETCODE is undefined and the unified API excluded.
- cp {{ project.path }}/Packages/manifest-unified.json {{ project.path }}/Packages/manifest.json
- unity-downloader-cli --fast --wait -u {{ editor }} -c Editor # Installing basic editor for tests execution
- UnifiedTestRunner --testproject={{ project.path }} --suite=playmode --artifacts-path=test-results --editor-location=.Editor --testfilter="{{ unified_test_filter }}" --rerun-strategy=Test --retry={{ num_test_retries }} --clean-library-on-rerun --timeout={{ test_timeout }}
# Remove the committed lock as it is configured for 6000.6 which does not contain the unified API required.
- rm -f {{ project.path }}/Packages/packages-lock.json

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Didn't the 3.x branch delete all the packages-lock files?


- unity-downloader-cli --fast --wait -u {{ editor }} -c Editor
- UnifiedTestRunner --testproject={{ project.path }} --suite=playmode --artifacts-path=test-results --editor-location=.Editor --rerun-strategy=Test --retry={{ num_test_retries }} --clean-library-on-rerun --timeout={{ test_timeout }}
artifacts:
logs:
paths:
- "test-results/**/*"
dependencies:
- .yamato/_run-all.yml#run_quick_checks # initial checks to perform fast validation of common errors
- .yamato/_run-all.yml#run_quick_checks
{% endfor -%}
{% endfor -%}
{% endfor -%}
Original file line number Diff line number Diff line change
Expand Up @@ -1412,12 +1412,12 @@ private bool UnifiedIsConfiguredCorrectly()
{
if (NetCodeConfig.Global == null)
{
Log.Error(new Context(LogLevel.Error, "You must create a {nameof(NetCodeConfig)} and set it to a single world in order to run in hybrid mode!").AddTag("Unified"));
Log.Error(new Context(LogLevel.Error, $"You must create a {nameof(NetCodeConfig)} and set it to a single world in order to run in hybrid mode!").AddTag("Unified"));
return false;
}
if (NetCodeConfig.Global.HostWorldModeSelection != NetCodeConfig.HostWorldMode.SingleWorld)
{
Log.Error(new Context(LogLevel.Error, "You must configure {nameof(NetCodeConfig)} to only use a single world in order to run in hybrid mode!").AddTag("Unified"));
Log.Error(new Context(LogLevel.Error, $"You must configure {nameof(NetCodeConfig)} to only use a single world in order to run in hybrid mode!").AddTag("Unified"));
return false;
}
return true;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@ public void OneTimeSetup()
{
// TODO: [CmbServiceTests] if this test is deemed needed to test against the CMB server then update this test.
NetcodeIntegrationTestHelpers.IgnoreIfServiceEnviromentVariableSet();
// Excluding from unified tests. If deemed needed, update test, then remove.
NetcodeIntegrationTestHelpers.IgnoreIfUnifiedTestsEnvironmentVariableSet();
}

[SetUp]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,8 @@ public void OneTimeSetup()
{
// TODO: [CmbServiceTests] if this test is deemed needed to test against the CMB server then update this test.
NetcodeIntegrationTestHelpers.IgnoreIfServiceEnviromentVariableSet();
// Excluding from unified tests. If deemed needed, update test, then remove.
NetcodeIntegrationTestHelpers.IgnoreIfUnifiedTestsEnvironmentVariableSet();
}

[Test]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,8 @@ public void OneTimeSetup()
{
// This test does not need to run against the Rust server.
NetcodeIntegrationTestHelpers.IgnoreIfServiceEnviromentVariableSet();
// Excluding from unified tests. If deemed needed, update test, then remove.
NetcodeIntegrationTestHelpers.IgnoreIfUnifiedTestsEnvironmentVariableSet();
}

[Test]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,8 @@ public void OneTimeSetup()
{
// TODO: [CmbServiceTests] if this test is deemed needed to test against the CMB server then update this test.
NetcodeIntegrationTestHelpers.IgnoreIfServiceEnviromentVariableSet();
// Excluding from unified tests. If deemed needed, update test, then remove.
NetcodeIntegrationTestHelpers.IgnoreIfUnifiedTestsEnvironmentVariableSet();
}

/// <summary>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,8 @@ public void OneTimeSetup()
{
// This test does not need to run against the Rust server.
NetcodeIntegrationTestHelpers.IgnoreIfServiceEnviromentVariableSet();
// Excluding from unified tests. If deemed needed, update test, then remove.
NetcodeIntegrationTestHelpers.IgnoreIfUnifiedTestsEnvironmentVariableSet();
}

[Test]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,8 @@ public void OneTimeSetup()
{
// This test does not need to run against the Rust server.
NetcodeIntegrationTestHelpers.IgnoreIfServiceEnviromentVariableSet();
// Excluding from unified tests. If deemed needed, update test, then remove.
NetcodeIntegrationTestHelpers.IgnoreIfUnifiedTestsEnvironmentVariableSet();
}

[Test]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@ namespace Unity.Netcode.RuntimeTests
{
// These tests do not need to run against the Rust server.
[IgnoreIfServiceEnvironmentVariableSet]
// These tests do not exercise hybrid prefabs.
[IgnoreIfUnifiedTestsEnvironmentVariableSet]
internal class NetworkTransformStateTests
{
[Test]
Expand Down Expand Up @@ -199,6 +201,8 @@ private void AssertTransformStateEquals(bool[] expected, NetworkTransformState a

// These tests do not need to run against the Rust server.
[IgnoreIfServiceEnvironmentVariableSet]
// These tests do not exercise hybrid prefabs.
[IgnoreIfUnifiedTestsEnvironmentVariableSet]
[TestFixture(TransformSpace.World, Precision.Full, Rotation.Euler)]
[TestFixture(TransformSpace.World, Precision.Half, Rotation.Euler)]
[TestFixture(TransformSpace.Local, Precision.Full, Rotation.Euler)]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,12 @@ public UnifiedNetworkTransformTest(HostOrServer hostOrServer) : base(hostOrServe
{
}

// This is the validation test for the unified API, so it always runs during a unified test pass.
protected override bool UseUnifiedTests()
{
return true;
}

protected override bool OnSetVerboseDebug()
{
return false;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@ public void OneTimeSetup()
{
// This test does not need to run against the Rust server.
NetcodeIntegrationTestHelpers.IgnoreIfServiceEnviromentVariableSet();
// Excluding from unified tests. If deemed needed, update test, then remove.
NetcodeIntegrationTestHelpers.IgnoreIfUnifiedTestsEnvironmentVariableSet();
}

[Test]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -282,6 +282,11 @@ protected override void OnOneTimeTearDown()
/// </summary>
private void InitializeServerAndClients(HostOrServer useHost)
{
#if UNIFIED_NETCODE
// HostOrServer arrives as a test method parameter here, so the base class cannot filter the
// hybrid prefab cases NUnit generates from the enum. Do it before anything is started.
ApplyUnifiedTestFilter(useHost);
#endif
s_ClientNetworkVariableTestInstances.Clear();
m_PlayerPrefab.AddComponent<NetworkVariableTest>();

Expand Down Expand Up @@ -351,6 +356,10 @@ private void InitializeServerAndClients(HostOrServer useHost)
[Test]
public void AllNetworkVariableTypes([Values] HostOrServer useHost)
{
#if UNIFIED_NETCODE
// Filter before CreateNetworkObjectPrefab, which routes to CreateHybridPrefab for hybrid cases.
ApplyUnifiedTestFilter(useHost);
#endif
var prefabToSpawn = CreateNetworkObjectPrefab("NetVarTest");
prefabToSpawn.AddComponent<NetworkVariableTestComponent>();

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@ protected override void OnOneTimeSetup()
{
// TODO: [CmbServiceTests] if this test is deemed needed to test against the CMB server then update this test.
NetcodeIntegrationTestHelpers.IgnoreIfServiceEnviromentVariableSet();
// Excluding from unified tests. If deemed needed, update test, then remove.
NetcodeIntegrationTestHelpers.IgnoreIfUnifiedTestsEnvironmentVariableSet();
base.OnOneTimeSetup();
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,8 @@ protected override void OnOneTimeSetup()
{
// TODO: [CmbServiceTests] if this test is deemed needed to test against the CMB server then update this test.
NetcodeIntegrationTestHelpers.IgnoreIfServiceEnviromentVariableSet();
// Excluding from unified tests. If deemed needed, update test, then remove.
NetcodeIntegrationTestHelpers.IgnoreIfUnifiedTestsEnvironmentVariableSet();
base.OnOneTimeSetup();
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,8 @@ protected override void OnOneTimeSetup()
{
// TODO: [CmbServiceTests] if this test is deemed needed to test against the CMB server then update this test.
NetcodeIntegrationTestHelpers.IgnoreIfServiceEnviromentVariableSet();
// Excluding from unified tests. If deemed needed, update test, then remove.
NetcodeIntegrationTestHelpers.IgnoreIfUnifiedTestsEnvironmentVariableSet();
base.OnOneTimeSetup();
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,8 @@ public void OneTimeSetup()
{
// This test does not need to run against the Rust server.
NetcodeIntegrationTestHelpers.IgnoreIfServiceEnviromentVariableSet();
// Excluding from unified tests. If deemed needed, update test, then remove.
NetcodeIntegrationTestHelpers.IgnoreIfUnifiedTestsEnvironmentVariableSet();
}

[SetUp]
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
using System;
using NUnit.Framework;
using NUnit.Framework.Interfaces;
using NUnit.Framework.Internal;

namespace Unity.Netcode.TestHelpers.Runtime
{
[AttributeUsage(AttributeTargets.Class, AllowMultiple = false, Inherited = true)]
internal class IgnoreIfUnifiedTestsEnvironmentVariableSetAttribute : NUnitAttribute, IApplyToTest
{
public void ApplyToTest(Test test)
{
// NotRunnable is the more weighty status, always respect it first
if (test.RunState == RunState.NotRunnable)
{
return;
}

if (NetcodeIntegrationTestHelpers.UnifiedTestRun())
{
test.RunState = RunState.Ignored;
test.Properties.Set("_SKIPREASON", NetcodeIntegrationTestHelpers.IgnoredForUnifiedTestsReason);
}
}
}
}

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading