Problem
Several validations accept provider_hidden=true evidence and pass without verifying anything. The AWS reference passes ~15 observability checks and the security BMC checks this way. As written, a provider can be certified on NVSwitch/NVLink/BMC telemetry by asserting the plane is provider-owned — the check verifies nothing.
Found during the per-check requires: inventory (2026-07-21) tracing every module-suite check to its stub scripts and validation contract.
Affected checks
Observability — always/AWS provider-hidden:
- OBS07-01
EastWestNetworkTelemetryCheck, OBS08-01 ManagementNetworkTelemetryCheck, OBS09-01 NvswitchFabricTelemetryCheck
- OBS17-01
BmcSelLogsCheck, TELEM04-01 BmcGpuTelemetryCheck, TELEM05-01 StorageCapacityTelemetryCheck
- TELEM07-01
GpuNvlinkTelemetryCheck, TELEM08-01 SwitchNvlinkTelemetryCheck (hidden even in NICo)
- OBS14-01 / OBS15-01 / OBS16-01 switch logs (hidden in both AWS and NICo)
- OBS11/12/13 fabric-manager/SM/UFM logs pass hidden on AWS (real NICo UFM implementations exist)
Security:
- CNP10-01
BmcProtocolSecurityCheck emits a fixed "no customer BMC surface" pass
- SEC12-01/02/03 BMC checks pass provider-hidden on non-BM providers
Decision needed
Is "plane is provider-owned/hidden" a legitimate certification pass? Options:
- Legitimate, but visible — keep the escape, record
provider_hidden as a distinct result status ("attested hidden" vs "verified") in upload/reporting so reviewers can distinguish.
- Hole — require real evidence; these checks become bare_metal/GPU-fabric-shaped (
requires) and the escape is removed.
- Per-plane hybrid — allowed for planes genuinely provider-owned in a cloud model (switch logs), disallowed where the declared capability implies tenant visibility (e.g. BMC/SEL and NVLink telemetry must be verified when
bare_metal is declared).
Interplay with capability declarations: whatever the answer, it should be consistent with the declared-capability obligation model — a bare_metal-declaring provider passing BMC checks via "hidden" is the case that most looks like a hole.
🤖 Generated with Claude Code
Problem
Several validations accept
provider_hidden=trueevidence and pass without verifying anything. The AWS reference passes ~15 observability checks and the security BMC checks this way. As written, a provider can be certified on NVSwitch/NVLink/BMC telemetry by asserting the plane is provider-owned — the check verifies nothing.Found during the per-check
requires:inventory (2026-07-21) tracing every module-suite check to its stub scripts and validation contract.Affected checks
Observability — always/AWS provider-hidden:
EastWestNetworkTelemetryCheck, OBS08-01ManagementNetworkTelemetryCheck, OBS09-01NvswitchFabricTelemetryCheckBmcSelLogsCheck, TELEM04-01BmcGpuTelemetryCheck, TELEM05-01StorageCapacityTelemetryCheckGpuNvlinkTelemetryCheck, TELEM08-01SwitchNvlinkTelemetryCheck(hidden even in NICo)Security:
BmcProtocolSecurityCheckemits a fixed "no customer BMC surface" passDecision needed
Is "plane is provider-owned/hidden" a legitimate certification pass? Options:
provider_hiddenas a distinct result status ("attested hidden" vs "verified") in upload/reporting so reviewers can distinguish.requires) and the escape is removed.bare_metalis declared).Interplay with capability declarations: whatever the answer, it should be consistent with the declared-capability obligation model — a bare_metal-declaring provider passing BMC checks via "hidden" is the case that most looks like a hole.
🤖 Generated with Claude Code