Following the feedback on #863 and #872 — opening this to discuss before sending code.
Three separate problems in the integration suite. Each is small and independent of the others.
1. The suite tests a different kernel than the tool ships
container installs kata 3.28.0 and its 6.18.15 kernel. The integration suite fetches 3.17.0, so it exercises a guest with a different feature set to the one users actually run.
The visible case is nested runtimes: 3.17.0 was built without nf_tables, so a docker daemon inside a container fails under test and works on what ships. A test failure there tells you nothing about the shipped guest.
Kata also moved from xz to zstd between those releases, so the archive is no longer named for its compression.
2. The reaper deletes the rootfs it means to preserve
The reaper deletes everything in the test directory except the unpacked rootfs. It holds that path as FileManager reports it, under /var, and compares it against contentsOfDirectory entries, which come back under /private/var. The two spellings never match, so the rootfs is removed along with the per-test files.
The unpack coordinator still records it as unpacked, so the next test opens a rootfs that is no longer on disk. The failure surfaces in whichever test runs next rather than in the cleanup that caused it, which is what makes it awkward to read.
I can post exact reproduction steps if that would help.
3. make test reports success for tests it did not run
Tests guarded for Linux are compiled out on macOS, so the target passes without running them. The target that does run them is not mentioned anywhere outside the makefile.
I have fixes for all three — around 20 lines across 3 files, no user-visible behaviour change. Happy to send them as one small PR or split them up, whichever you prefer, and equally happy to leave any of them if they are not worth your time.
Separately, there is a larger change sitting behind these: replacing the positional correlation between a machine's storage configuration and its reported attachments with one shape that names the roles. That is a substantial change and I have not opened an issue for it. Let me know if you would like one and I will write it up on its own; otherwise I will leave it.
Following the feedback on #863 and #872 — opening this to discuss before sending code.
Three separate problems in the integration suite. Each is small and independent of the others.
1. The suite tests a different kernel than the tool ships
containerinstalls kata 3.28.0 and its 6.18.15 kernel. The integration suite fetches 3.17.0, so it exercises a guest with a different feature set to the one users actually run.The visible case is nested runtimes: 3.17.0 was built without
nf_tables, so a docker daemon inside a container fails under test and works on what ships. A test failure there tells you nothing about the shipped guest.Kata also moved from xz to zstd between those releases, so the archive is no longer named for its compression.
2. The reaper deletes the rootfs it means to preserve
The reaper deletes everything in the test directory except the unpacked rootfs. It holds that path as
FileManagerreports it, under/var, and compares it againstcontentsOfDirectoryentries, which come back under/private/var. The two spellings never match, so the rootfs is removed along with the per-test files.The unpack coordinator still records it as unpacked, so the next test opens a rootfs that is no longer on disk. The failure surfaces in whichever test runs next rather than in the cleanup that caused it, which is what makes it awkward to read.
I can post exact reproduction steps if that would help.
3.
make testreports success for tests it did not runTests guarded for Linux are compiled out on macOS, so the target passes without running them. The target that does run them is not mentioned anywhere outside the makefile.
I have fixes for all three — around 20 lines across 3 files, no user-visible behaviour change. Happy to send them as one small PR or split them up, whichever you prefer, and equally happy to leave any of them if they are not worth your time.
Separately, there is a larger change sitting behind these: replacing the positional correlation between a machine's storage configuration and its reported attachments with one shape that names the roles. That is a substantial change and I have not opened an issue for it. Let me know if you would like one and I will write it up on its own; otherwise I will leave it.