Skip to content

bug(bmc-mock): GB300-family host mocks collide on MachineId (hardcoded serial numbers) #3274

Description

@RajMandaliya

Version

main

Describe the bug.

Environment

Built from source, main branch, commit 69c69bf (2026-07-08).

Summary

All four GB300-family host hardware mocks in bmc-mock hardcode fixed
serial number literals instead of generating unique values per instance.
Since machine_id_serial_number() checks the system serial first (and
only falls back to chassis serial for DPUs, not regular hosts), multiple
mock instances of the same hardware type end up with the same
MachineId
.

Two severities exist:

  • Full collision (both system and chassis serials hardcoded to the
    exact same literal for every instance): NvidiaDgxGb300,
    SupermicroGb300Nvl
  • Partial collision (system serial hardcoded/shared, chassis serial
    correctly varies per instance): LenovoGB300Nvl, NvidiaDgxVr

Since MachineId derivation only reads the system serial (with no
non-DPU fallback), all four collide the same way regardless of severity
category.

Steps to reproduce

Spin up two separate mock instances of the same hardware type and
generate a MachineId for each:

let h1 = test_support::lenovo_gb300_bmc().await; // or dgx_gb300_bmc(), etc.
let h2 = test_support::lenovo_gb300_bmc().await;
// ... explore both, call generate_machine_id(true) on each ...
assert_ne!(id1, id2); // fails for all four hardware types

Expected behavior

Two distinct mock machine instances of the same hardware type should
produce different MachineIds.

Actual behavior

They produce the identical MachineId. Verified failing for all four:
LenovoGB300Nvl, NvidiaDgxGb300, SupermicroGb300Nvl, and
NvidiaDgxVr.

Root cause

In crates/bmc-mock/src/machine_info.rs:

// dgx_gb300_nvl() - full collision
system_0_serial_number: "1332425360072".into(),
chassis_0_serial_number: "1332425360072".into(),

// supermicro_gb300_nvl() - full collision
system_0_serial_number: "A978250X6404492".into(),
chassis_0_serial_number: "HA261S056572".into(),

// lenovo_gb300_nvl() / dgx_vr_nvl() - partial collision
system_0_serial_number: "012345678901234567890123".into(),
chassis_0_serial_number: Cow::Borrowed(&self.serial), // unique per machine

self.serial (used correctly in the last two) is derived from a
freshly-allocated MAC address in HostMachineInfo::new, so it's always
unique. None of the hardcoded literals are.

Why this wasn't caught

The existing exploration tests for these hardware types (e.g.
explore_lenovo_gb300) each only spin up a single mock instance, so
there was never a test comparing two instances of the same type against
each other. This looks like a genuine coverage gap rather than an
intentional scope decision — the chassis-serial fallback in
machine_id_serial_number() was deliberately added for BF4 DPUs
specifically (#2903, #2906), and was never extended to regular hosts,
presumably because real host hardware has genuinely unique system
serials. This only surfaces as a problem in mock data.

Impact

Any local dev/test environment spinning up multiple mock machines of the
same GB300-family hardware type (via machine-a-tron or direct bmc-mock
test helpers) will have those machines collide on MachineId in
site-explorer.

Suggested fix

Derive system_0_serial_number (and, where hardcoded,
chassis_0_serial_number too) from a per-instance unique value for all
four hardware types, matching the existing pattern used for
self.serial.

Happy to open a PR for this.

Minimum reproducible example

// Minimal reproducible example — add as a test in crates/bmc-explorer/tests/
// and run with: cargo test -p bmc-explorer <test_name> -- --nocapture

use bmc_explorer::nv_generate_exploration_report;
use bmc_mock::test_support;

#[tokio::test]
async fn machine_id_collision_repro() {
    let h1 = test_support::lenovo_gb300_bmc().await;
    let h2 = test_support::lenovo_gb300_bmc().await;
    let config = /* explorer config, see crates/bmc-explorer/tests/common.rs */;

    let mut report1 = nv_generate_exploration_report(h1.service_root, &config)
        .await
        .unwrap();
    let mut report2 = nv_generate_exploration_report(h2.service_root, &config)
        .await
        .unwrap();

    let id1 = report1.generate_machine_id(true).unwrap().unwrap();
    let id2 = report2.generate_machine_id(true).unwrap().unwrap();

    assert_ne!(id1, id2); // FAILS: both produce the same MachineId
}

Relevant log output

running 1 test
thread 'two_lenovo_gb300_hosts_should_not_collide' panicked at crates/bmc-explorer/tests/lenovo_gb300_explore.rs:113:5:
assertion `left != right` failed: two distinct LenovoGB300Nvl hosts must not produce the same MachineId
  left: fm100pskibdln6n1j098gft4fld8sundldonmi04saohs49ek7aou91puf0
 right: fm100pskibdln6n1j098gft4fld8sundldonmi04saohs49ek7aou91puf0
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
test two_lenovo_gb300_hosts_should_not_collide ... FAILED

thread 'two_nvidia_dgx_gb300_hosts_should_not_collide' panicked at crates/bmc-explorer/tests/lenovo_gb300_explore.rs:141:5:
assertion `left != right` failed: two distinct NvidiaDgxGb300 hosts must not produce the same MachineId
  left: fm100psc4jsf77bf5v25riuclq1u3a6705unk7ilvqcut1r9b2hsp4lnt80
 right: fm100psc4jsf77bf5v25riuclq1u3a6705unk7ilvqcut1r9b2hsp4lnt80
test two_nvidia_dgx_gb300_hosts_should_not_collide ... FAILED

thread 'two_supermicro_gb300_hosts_should_not_collide' panicked at crates/bmc-explorer/tests/lenovo_gb300_explore.rs:169:5:
assertion `left != right` failed: two distinct SupermicroGb300Nvl hosts must not produce the same MachineId
  left: fm100psbd4ptl5ogmof9rpo3k4inpmm37d8og9em2ivbs3vp2okhq50g9c0
 right: fm100psbd4ptl5ogmof9rpo3k4inpmm37d8og9em2ivbs3vp2okhq50g9c0
test two_supermicro_gb300_hosts_should_not_collide ... FAILED

thread 'two_nvidia_dgx_vr_hosts_should_not_collide' panicked at crates/bmc-explorer/tests/lenovo_gb300_explore.rs:197:5:
assertion `left != right` failed: two distinct NvidiaDgxVr hosts must not produce the same MachineId
  left: fm100pskibdln6n1j098gft4fld8sundldonmi04saohs49ek7aou91puf0
 right: fm100pskibdln6n1j098gft4fld8sundldonmi04saohs49ek7aou91puf0
test two_nvidia_dgx_vr_hosts_should_not_collide ... FAILED

failures:
    two_lenovo_gb300_hosts_should_not_collide
    two_nvidia_dgx_gb300_hosts_should_not_collide
    two_supermicro_gb300_hosts_should_not_collide
    two_nvidia_dgx_vr_hosts_should_not_collide

test result: FAILED. 0 passed; 4 failed; 0 ignored; 0 measured; 0 filtered out

Other/Misc.

No response

Code of Conduct

  • I agree to follow NVIDIA Infra Controller's Code of Conduct
  • I have searched the open bugs and have found no duplicates for this bug report

Metadata

Metadata

Assignees

Labels

bugA defect in existing software (deprecated - use issue type, but it's needed for reporting now)

Type

Projects

Status
Closed

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions