Dynamically calculate the required free space in switch `/bootflash`

[takishida]

Please use 👍 to indicate interest in this enhancement.

Describe the enhancement

For some upgrades, having 50% free space in a switch’s /bootflash is not sufficient to complete the upgrade.

Specifically, when an upgrade crosses the ACI 6.0(2) 32-bit/64-bit boundary, some switches reboot twice:

1. The first reboot upgrades the switch to the 32-bit target version.
2. The second reboot upgrades the switch to the corresponding 64-bit version.

To complete both upgrades, the switch may need approximately 7 GB of free /bootflash space before the first upgrade begins. This is about 58% of the 12 GB /bootflash capacity used by ACI switches as of ACI 6.2(2).

Current behavior

The Switch Node /bootflash usage rule always uses 50% as its threshold.

Suggested behavior

The Switch Node /bootflash usage rule should dynamically calculate the required free space based on the applicable upgrade scenario.

Scenario 1: Crossing the ACI 6.0(2) 32-bit/64-bit boundary

Use the following calculation when:

• The upgrade crosses the ACI 6.0(2) 32-bit/64-bit boundary.
• The 32-bit target image is larger than the current image.

2 × (64-bit target image size + 32-bit target image size − current image size)

Scenario 2: All other upgrades

2 × target image size

Note

• In Scenario 1, the second reboot uses the 32-bit target image as its base instead of the original image. Therefore, the size difference between the 32-bit target image and the current image must be included when calculating the free space required before the first reboot.
• Each image size is doubled because the image must be extracted.

How to reproduce

A typical case occurs when an ACI switch has an NX-OS image stored in /bootflash.

An internal cleanup process normally deletes unnecessary files to keep /bootflash usage below 50%. However, NX-OS images—and certain other files, such as the image for the current ACI version—are excluded from cleanup because users may have placed an NX-OS image on the switch in preparation for converting it from an ACI switch to an NX-OS switch.

To reproduce the issue:

1. Copy an NX-OS image to the ACI switch’s /bootflash.
2. Ensure that /bootflash usage is just below 50%.
3. Upgrade the ACI switch across the 32-bit/64-bit boundary—for example, from 5.3(2f) to 6.1(6f).

Note

When the current ACI version has the fix from CSCwt53667, the NX-OS image in /bootflash is deleted upon an upgrade (See issue #375). However, it typically affects the upgrade when you cross the 32-bit/64-bit boundary where the source version is older than 6.0(2) without the fix from CSCwt53667. Also, the issue could technically happen without an NX-OS image when another file excluded from cleanup is present especially when the size of the new switch image is getting bigger.

Detailed reproduction example

Step 1: Check /bootflash before the upgrade

f2-leaf1# ls -l /bootflash/
total 5729316
-rw-rw-rw- 1 root  root      441789 Aug 17 23:53 CpuUsage.Log
-rw-rw-rw- 1 root  admin 2090298245 Aug 17 18:07 aci-n9000-dk9.15.3.2f.bin
-rw-r--r-- 1 root  root  1800344453 Aug 17 18:23 auto-s
-rwxrwxrwx 1 root  root         268 Apr 15 13:06 clean-config-log
-rw-rw-rw- 1 root  root           2 Aug 17 22:28 diag_bootup
drwx------ 2 root  root       49152 Aug 17 18:12 lost+found
drwxr-xr-x 4 root  root        4096 Aug 19  2025 lxc
-rw-r--r-- 1 root  root    18634672 Aug 18 12:00 mem_log.txt
-rw-r--r-- 1 root  root      135501 Aug 18 12:00 mts_buffer_log.log
-rwx------ 1 admin admin 1956867584 Aug 18 12:03 nxos.9.3.8.bin
f2-leaf1#
f2-leaf1#
f2-leaf1# df -h /bootflash/
Filesystem      Size  Used Avail Use% Mounted on
/dev/sda4        12G  5.5G  5.6G  50% /bootflash
f2-leaf1#
f2-leaf1# moquery -d sys/eqptcapacity/fspartition-bootflash
# eqptcapacity.FSPartition
name         : bootflash
avail        : 5775032
childAction  :
dn           : sys/eqptcapacity/fspartition-bootflash
memAlert     : normal
modTs        : never
monPolDn     : uni/fabric/monfab-default
path         : /bootflash
rn           : fspartition-bootflash
status       :
used         : 5760072

The exact utilization is below the current 50% threshold:

used / (used + avail) = 0.4993515446414701
                      = 49.994%

Step 2: Pre-download the 32-bit target image

f2-leaf1# ls -l /bootflash/
total 8352908
-rw-rw-rw- 1 root  root      441789 Aug 17 23:53 CpuUsage.Log
-rw-rw-rw- 1 root  admin 2090298245 Aug 17 18:07 aci-n9000-dk9.15.3.2f.bin
-rw-rw-rw- 1 root  root  2685181327 Aug 18 13:02 aci-n9000-dk9.16.1.6f.bin
-rw-r--r-- 1 root  root  1800344453 Aug 17 18:23 auto-s
-rwxrwxrwx 1 root  root         268 Apr 15 13:06 clean-config-log
-rw-rw-rw- 1 root  root           2 Aug 17 22:28 diag_bootup
-rw-r--r-- 1 root  root          56 Aug 18 13:05 disk_log.txt
-rw-rw-rw- 1 root  root           7 Aug 18 13:02 imgDnldStatus
drwx------ 2 root  root       49152 Aug 17 18:12 lost+found
drwxr-xr-x 4 root  root        4096 Aug 19  2025 lxc
-rw-r--r-- 1 root  root    20000654 Aug 18 13:00 mem_log.txt
-rw-r--r-- 1 root  root      137217 Aug 18 13:00 mts_buffer_log.log
-rwx------ 1 admin admin 1956867584 Aug 18 12:03 nxos.9.3.8.bin
f2-leaf1#
f2-leaf1# df -h /bootflash/
Filesystem      Size  Used Avail Use% Mounted on
/dev/sda4        12G  8.0G  3.1G  73% /bootflash
f2-leaf1#

Step 3: Trigger the upgrade and extract the 32-bit image

f2-leaf1# ls -l /bootflash/
total 10852332
-rw-rw-rw- 1 root  root     1149537 Aug 18 13:20 CpuUsage.Log
-rw-rw-rw- 1 root  admin 2090298245 Aug 17 18:07 aci-n9000-dk9.15.3.2f.bin
-rw-rw-rw- 1 root  root  2685181327 Aug 18 13:02 aci-n9000-dk9.16.1.6f.bin
-rw-rw-rw- 1 root  root  2330228111 Aug 18 13:15 aci-n9000-dk9.16.1.6f.bin-isan
-rw-r--r-- 1 root  root  1800344453 Aug 17 18:23 auto-s
-rwxrwxrwx 1 root  root         268 Apr 15 13:06 clean-config-log
-rw-rw-rw- 1 root  root           2 Aug 17 22:28 diag_bootup
-rw-r--r-- 1 root  root          56 Aug 18 13:20 disk_log.txt
-rwxr-xr-x 1 root  root   228007805 Aug 11 20:20 fpga_install.bin
-rw-rw-rw- 1 root  root           7 Aug 18 13:02 imgDnldStatus
drwx------ 2 root  root       49152 Aug 18 13:15 lost+found
drwxr-xr-x 4 root  root        4096 Aug 19  2025 lxc
-rw-r--r-- 1 root  root    20457447 Aug 18 13:20 mem_log.txt
-rw-r--r-- 1 root  root      138026 Aug 18 13:20 mts_buffer_log.log
-rwx------ 1 admin admin 1956867584 Aug 18 12:03 nxos.9.3.8.bin
f2-leaf1#
f2-leaf1# df -h /bootflash/
Filesystem      Size  Used Avail Use% Mounted on
/dev/sda4        12G   11G  637M  95% /bootflash

Step 4: After the first reboot, the 64-bit image is downloaded automatically

f2-leaf1# ls -l /bootflash
total 9571684
-rw-rw-rw- 1 root  root     1734084 Aug 18 20:33 CpuUsage.Log
-rw-rw-rw- 1 root  admin 2825636804 Aug 18 20:50 aci-n9000-dk9.16.1.6f-cs_64.bin
-rw-rw-rw- 1 root  root  2685181327 Aug 18 20:02 aci-n9000-dk9.16.1.6f.bin
-rw-r--r-- 1 root  root  2330228111 Aug 18 20:26 auto-s
-rwxrwxrwx 1 root  root         268 Apr 15 20:06 clean-config-log
-rw-rw-rw- 1 root  root           2 Aug 18 20:33 diag_bootup
-rw-r--r-- 1 root  root          56 Aug 18 21:20 disk_log.txt
drwx------ 2 root  root       49152 Aug 18 20:15 lost+found
drwxr-xr-x 4 root  root        4096 Aug 20  2025 lxc
-rw-r--r-- 1 root  root     1503735 Aug 18 21:20 mem_log.txt
-rw-r--r-- 1 root  root      139914 Aug 18 21:20 mts_buffer_log.log
-rwx------ 1 admin admin 1956867584 Aug 18 19:03 nxos.9.3.8.bin
-rw-r--r-- 1 root  admin        156 Aug 18 20:51 upgTimeline.txt
f2-leaf1#
f2-leaf1# ls -hl /bootflash | grep bin
-rw-rw-rw- 1 root  admin 2.7G Aug 18 20:50 aci-n9000-dk9.16.1.6f-cs_64.bin
-rw-rw-rw- 1 root  root  2.6G Aug 18 20:02 aci-n9000-dk9.16.1.6f.bin
-rwx------ 1 admin admin 1.9G Aug 18 19:03 nxos.9.3.8.bin
f2-leaf1#
f2-leaf1# df -h /bootflash
Filesystem      Size  Used Avail Use% Mounted on
/dev/sda4        12G  9.2G  1.9G  84% /bootflash
f2-leaf1#
f2-leaf1# moquery -c pconsBootStrapTracking | grep debug
debugMessage                     : Auto Upgrade in progress

Step 5: Extraction of the 64-bit image fails

f2-leaf1# tail -n 20 /mnt/pss/installer_frontend.log | grep installer_download_image
<Tue Aug 18 20:46:16 2026> installer_download_image: url=http://10.21.0.1:7777/fwrepo/aci-n9000-dk9.16.1.6f-cs_64.bin mmode=0 ks=aci-n9000-dk9.16.1.6f-cs_64.bin, sys=aci-n9000-dk9.16.1.6f-cs_64.bin-isan
<Tue Aug 18 20:46:16 2026> installer_download_image: Image available. Skipping download
<Tue Aug 18 20:46:16 2026> installer_download_image: Not enough space in /bootflash to extract isan image.

Although the initial /bootflash utilization passed the current rule at 49.994%, the switch did not have enough space to extract the 64-bit image and complete the second stage of the upgrade.