test(neutron-understack): ML2 mechanism scenario tests + traceability - #2241
Draft
cardoe wants to merge 21 commits into
Draft
test(neutron-understack): ML2 mechanism scenario tests + traceability#2241cardoe wants to merge 21 commits into
cardoe wants to merge 21 commits into
Conversation
Reuse neutron's Ml2PluginV2TestCase to stand up a real Ml2Plugin on
in-memory SQLite with our mechanism drivers loaded, so tests drive real
create/update/delete through the ML2 manager and assert which calls
reach our drivers and with what data.
SCENARIOS.md is the test-plan catalog; each scenario test is tagged
@pytest.mark.scenario("<ID>"). test_scenario_coverage.py enforces
catalog<->marker equivalence and conftest.py fails collection if a
scenario test lacks its marker.
Pin neutron/neutron-lib to the understack/2026.1 fork (matching the
container) via [tool.uv.sources], and add neutron's test-only deps.
Exercise the Ironic vif-attach lifecycle through the ML2 manager: hierarchical VXLAN->dynamic-VLAN binding across the understack and undersync drivers, refusal without physical_network, refusal of unsupported vnic_types, vif-detach reconcile, and port-delete segment release. Catalogued as BM-BIND-01..05 in SCENARIOS.md. BM-BIND-04 asserts vif-detach releases the dynamic VLAN segment, which the driver does not do today (it leaks, releasing only on delete). It is marked xfail(strict=True) and tracked by #2239, so it flips to a failure once fixed, prompting removal of the marker.
Tag each scenario test's JUnit case with a `scenario` property, and for neutron-understack publish a scenario ID -> status traceability matrix to the CI job summary plus upload the JUnit as an artifact. scripts/scenario-report.py builds the matrix from the catalog + JUnit and labels xfailed scenarios as known bugs. SCENARIOS.md stays in-repo next to the tests as the source of truth; it is developer-facing and not published to the operator docs site.
Add UnderstackMl2RouterScenarioBase (real L3 router + flavors plugin via ML2TestFramework) and a VRF-RTR-01 scenario: a VXLAN network with baremetal ports bound to two physnets, then a VRF router attached on the internal side. The test asserts undersync syncs both physnets so the switches carrying those ports are reconciled for the new router. It does not today, so the test is xfail(strict=True), tracked by #2240. The VRF flavor is simulated by patching routers._router_has_flavor, as the existing router unit tests do.
cardoe
force-pushed
the
neutron-ml2-scenario-tests
branch
from
August 24, 2026 22:31
1c50b5f to
58ce188
Compare
Extend the router scenarios to the SVI flavor. SVI-RTR-01 mirrors VRF-RTR-01 but attaches an SVI router to an address-scoped IPv4 subnet (SVI routers require a scope), and asserts undersync syncs the physnets of the network's bound baremetal ports. It does not today -- the same gap as VRF, tracked by #2240 -- so the test is xfail(strict=True). The shared _bind_baremetal_port helper moves onto UnderstackMl2RouterScenarioBase. The SVI flavor is simulated by patching _router_has_flavor and svi._is_svi_router, mirroring the unit tests.
Upload each project's JUnit and add a publish-test-results job that runs EnricoMi/publish-unit-test-result-action once to post a pass/fail/skip summary as a PR comment and check. Pinned by SHA; needs checks and pull-requests write. Fork PRs would need a workflow_run-triggered variant, noted inline.
Python test results188 tests 188 ✅ 1s ⏱️ Results for commit 39f1959. ♻️ This comment has been updated with latest results. |
Add trunk scenarios on a real neutron trunk plugin (UnderstackMl2TrunkScenarioBase). TRUNK-SUB-ADD and TRUNK-SUB-DEL assert undersync syncs the parent baremetal port's physnet when a subport is added or removed; TRUNK-PARENT-NOIP asserts the sync fires even when the parent has no IP. BM-BIND-06 asserts a bound baremetal port with no IP still emits the physnet sync. Move _bind_baremetal_port to the shared mixin with a fixed_ips option. utils.fetch_network_node_trunk_id (live OVN/Ironic lookup) is stubbed.
Replace the vague "planned/future" note with a structured backlog of not-yet-covered scenarios grouped by area (port bind/create/delete, trunk, SVI validation, router, VNI/providers, cross-cutting), each with a proposed ID to promote to a catalogued heading when its test lands. Kept as plain bullets so the coverage check ignores them. Cisco ASA is explicitly out of scope.
BM-BIND-REUSE-01 asserts a second baremetal port on the same network+physnet reuses the existing dynamic VLAN segment. PROV-BIND-01 binds a port on the provisioning network. PROV-DEL-01 asserts deleting a provisioning-network port syncs but retains the VLAN segment (the clean/provision cycle reuses it), unlike BM-BIND-05's tenant delete.
Add SVI negative scenarios exercising the create_port_precommit scope validator: SVI-VAL-NOSCOPE-01 (subnet with no address scope), SVI-VAL-IPV6-01 (IPv6 subnet), and SVI-VAL-CONFLICT-01 (conflicting per-IP-version scopes). Each attach is rejected; the validator's BadRequest is surfaced by the ML2 manager as MechanismDriverError. Factor the SVI-flavor patching into an _as_svi_router() helper and make _scoped_ipv4_subnet parameterizable for multiple scopes.
TRUNK-DEL-01 asserts deleting a trunk cleans the parent switchport and syncs. TRUNK-MULTI-01 adds two subports in one operation. TRUNK-PARENT- UNBOUND-01 asserts an unbound parent yields no switchport config and no sync. TRUNK-SEGID-RANGE-01 asserts a subport seg_id outside [1, 3799] is rejected (SubportSegmentationIDError surfaced as CallbackFailure).
Add a FakeOvnClient (records localnet LSP create/delete, short-circuits the vxlan HCG workaround) and UnderstackMl2RouterOvnScenarioBase (L3 + flavors + trunk plugin, network-node physnet + OVS opts). RTR-ATTACH-01 asserts a non-flavored router attach builds the uplink (dynamic segment, shared port, network-node trunk subport, OVN localnet). RTR-SECOND-01 asserts a second router on the network is a no-op. RTR-DETACH-01 asserts remove_router_interface tears the uplink down.
Wire a real L3 service provider + flavor + service profile for the Palo Alto router flavor, with IronicClient faked (single-node netdev pool). PALO-ADOPT-01 asserts creating a router of that flavor adopts the available node; PALO-RELEASE-01 asserts deleting it returns the node. Adds FakeIronicClient to the scenario fakes.
VRF-DETACH-01 is the teardown counterpart of VRF-RTR-01: removing a VRF router interface should sync the physnets of baremetal ports still bound on the network, but does not today (understack#2240). Marked xfail(strict=True). Note DRYRUN-01 in the backlog: undersync_dry_run lives inside the (mocked) Undersync client, so it has no scenario-observable effect.
Act on the PR code review (test/CI hygiene; no product code changed): - Gate the EnricoMi publish job to same-repo PRs (fork tokens are read-only and would fail the check). - Add a test asserting the neutron/neutron-lib revs in [tool.uv.sources] match the container Dockerfile ARGs, so they cannot silently drift. - Share catalog-heading parsing (catalog.py) between the coverage test and the report script, removing the divergent regexes. - Render a strict-xfail XPASS distinctly, not a plain fail. - Scope the SVI IPv6 subnet so SVI-VAL-IPV6-01 pins the IPv6 guard.
Round of review follow-ups on the scenario suite: - Enforce catalog<->test parity from pytest's collected items (in the conftest hook) instead of regex-scanning sources, which could count a marker in a comment or dead code. Validate marker shape and share the scenario-ID pattern via catalog.py; unit-test the validator. - Strengthen the trunk and router scenarios to assert the resulting DB state -- subport binding levels, dynamic VLAN segments (allocated, released, distinct), trunk membership, and the OVN localnet port -- not just that undersync.sync was called. - Update SCENARIOS.md descriptions to match.
Both the trunk and router-OVN scenarios defined their own _trunk_subports helper (one via get_trunk, one via SubPort.get_objects). Move a single get_trunk-based helper onto the shared mixin and drop the duplicates (and the now-unused SubPort import).
Prefix the non-flavored router scenarios with OVN (they exercise the
default OVN L3 provider) and standardize the router family as
<TYPE>-ROUTER-<OP>:
- RTR-{ATTACH,SECOND,DETACH}-01 -> OVN-ROUTER-{ATTACH,SECOND,DETACH}-01
- VRF-RTR-01 -> VRF-ROUTER-ATTACH-01
- VRF-DETACH-01 -> VRF-ROUTER-DETACH-01
- SVI-RTR-01 -> SVI-ROUTER-ATTACH-01
- PALO-{ADOPT,RELEASE}-01 -> PALO-ROUTER-{ADOPT,RELEASE}-01
SVI-VAL-* and the backlog OVN-ROUTER-* entries updated to match.
The "Router interface (VRF flavor)" catalog section holds both the VRF and SVI flavored router scenarios; retitle it and note the SVI detection patch so the heading matches its contents.
Both flavors realize the router as an SVI on the switching fabric; the only difference is which VRF the SVI lands in. The VRF flavor creates a dedicated VRF and places the SVI into it; the SVI flavor drops the SVI into a well-known VRF from the base switch config. Note this in the catalog so the two near-mirror scenarios read as intended.
Document the router-uplink segment leak (understack#2245) as an xfail on this test branch instead of fixing it here: OVN-ROUTER-DETACH-01 asserts the dynamic uplink segment is released on teardown and is marked xfail(strict=True). OVN-ROUTER-ATTACH-01 also now asserts the uplink segment is dynamic. The driver fix lives on a separate branch.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds an in-process ML2 scenario test layer for the
neutron-understackmechanism drivers, plus a catalog + CI traceability report. Scenarios drive real
create_network/create_port/ router operations through a realMl2Plugin(in-memory SQLite) with our
understack+undersyncdrivers loaded, and assertwhich calls reach our drivers and with what data — the same style upstream neutron
uses to test its own ML2 drivers.
Why
Existing tests use
Ml2PluginNoInit, which bypasses the DB and the ML2 manager andtests one method against a hand-built context. That can't exercise the manager, the
two-driver hierarchical binding chain, or real dynamic-segment allocation — exactly
what a vif-attach or router-attach scenario needs.
What's here
tests/scenarios/base.py): reuses neutron'sMl2PluginV2TestCase(and
ML2TestFrameworkfor router scenarios); mocks only the Undersync HTTPclient. Test-neutron is pinned to the
understack/2026.1fork the containerships (via
[tool.uv.sources], matchingcontainers/neutron/Dockerfile).BM-BIND-01..05): hierarchical VXLAN→dynamic-VLANbinding across both drivers, refusal without
physical_network, unsupportedvnic_type, vif-detach, and port-delete segment release.
VRF-RTR-01): a VXLAN network with baremetal ports on twophysnets, then a VRF router attached on the internal side.
tests/scenarios/SCENARIOS.md) + coverage enforcement:every scenario is tagged
@pytest.mark.scenario("<ID>");test_scenario_coverage.pyenforces catalog↔marker equivalence both ways andconftest.pyfails collection on an untagged scenario test.scenarioproperty;code-test.yamlemits JUnit and (for neutron-understack)publishes a scenario ID → status matrix to the job summary + uploads the JUnit.
Two confirmed bugs, pinned as strict-xfail
These scenarios assert the desired behavior and are
xfail(strict=True), so eachflips to a failure (prompting removal of the marker) once fixed:
BM-BIND-04→ neutron-understack leaks dynamic VLAN segments on baremetal vif-detach #2239: dynamic VLAN segment leaks on vif-detach.VRF-RTR-01→ VRF router interface attach does not sync bound baremetal port physnets #2240: VRF router interface attach does not sync the physnets ofbaremetal ports already bound on the network.
Try it
Draft: sharing for early feedback on the approach before extending to more scenarios
(trunks, non-flavored routers — the latter needs the OVN IDL faked).