Skip to content

bug: route server health checking is too stringent causing unnecessary drop in allocatable capacity #4113

Description

@fengzhou-nv

Version

0.8.0

Describe the bug.

Users can provide a list of IPs as route_server in NICo config. NICo sends these to DPUs where DPU agent renders every route server as a BGP neighbor in the routeserver peer group.

However, if any of the route server is having issues, NICo creates BgpPeeringRouteServer health alert with PreventAllocations and PreventHostStateChanges classification. If route servers are redundant and carry the same EVPN routes (which should be in production settings), the classifications would unnecessarily reduce available capacity of the site or blocking of host operations.

NICo should be smarter in handling partial availability of route servers. If there is no redundancy or no route server, then NICo should really alert/mark the machines as unhealthy. Otherwise, it would serve better with some warning or metric that operators can alert on.

Minimum reproducible example

Relevant log output

Other/Misc.

No response

Code of Conduct

  • I agree to follow NVIDIA Infra Controller's Code of Conduct
  • I have searched the open bugs and have found no duplicates for this bug report

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugA defect in existing software (deprecated - use issue type, but it's needed for reporting now)

    Type

    Projects

    Status
    Triage

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions