Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions docs/npu.md
Original file line number Diff line number Diff line change
Expand Up @@ -936,6 +936,23 @@ Two facts about what the container holds, both reported, not hidden:
scale 0, zero 0). That is 12.5 % more bytes than the pack; the true K is in the container's
manifest and in `config.json`, and nothing reads a padding column as a weight.

### Where a 1.9 pack's rewritten `.fconst` goes

A Ryzen AI 1.9 pack's Token `.fconst` is rewritten into its 1.8 form before it is read. The copy is as large as the source
(3.6 GB for an h4096 pack), so where it lands matters: on strixhalo `/tmp` is tmpfs, which means RAM. The importer tries these
directories in order:

1. `$ONEBIT_FCONST_SCRATCH`, if set;
2. `$XDG_CACHE_HOME/1bit/fconst19`, or `~/.cache/1bit/fconst19` (on disk);
3. `$TMPDIR`, then `/tmp`, then the current directory.

The copy is a cache: it is reused while the source's size and mtime and the rules version match. On each write the importer
sweeps the scratch directories. It removes other identities of the same pack and geometry, the pre-version name, and any
`1bit-fconst19*` copy nobody has written for 6 hours. Without an override, **every** candidate directory is swept, not only the
one being written, so copies older binaries left in `/tmp` get reclaimed (engine#1907). With `ONEBIT_FCONST_SCRATCH` set, only
that directory is swept, and a run with an override never removes another lane's files. The test suite sets one per run and
removes it at exit.

### Linking the kernel set the container will be served with

A model directory carries its own NPU kernels in `npu/` (see "A model directory"), and for an imported
Expand Down
25 changes: 24 additions & 1 deletion npu/pack_import.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -752,7 +752,17 @@ std::string materialise_token_fconst(const std::string& pack_dir, const Dims& di
// GiB of tmpfs, had been resident all day).
const std::string legacy_name = std::string(kScratchStem) + "-" + tag + "-" + std::to_string(dims.hidden) + "-" +
std::to_string(dims.heads) + "-" + std::to_string(dims.layers) + ".fconst";
evict_stale_canonical_copies(d.string(), name, ident_prefix, legacy_name, kScratchMaxAgeSeconds);
// Sweep EVERY candidate directory, not only the one written (engine#1907). Since #1801 moved the cache to disk, a
// write lands in ~/.cache, so the sweep never visited /tmp again, and the copies older binaries had left there
// (17.0 GiB of tmpfs, i.e. RAM, measured 2026-11-10) stayed until reboot. The other directories are only swept,
// never created, and the rule is the same one as here: the exact name is kept wherever it is.
// An explicit ONEBIT_FCONST_SCRATCH is a lane saying "my scratch is here": then only that directory is swept, so a
// run with an override (the test suite sets one) never removes another lane's files from the shared /tmp or cache.
std::vector<std::string> dir_names;
const char* override_dir = std::getenv("ONEBIT_FCONST_SCRATCH");
if (override_dir && *override_dir) dir_names.push_back(d.string());
else for (const auto& x : dirs) dir_names.push_back(x.string());
sweep_fconst_scratch_dirs(dir_names, d.string(), name, ident_prefix, legacy_name, kScratchMaxAgeSeconds);
// Say where it went whenever that is not the ordinary place: the copy is a few hundred MB and it is scratch.
std::error_code ec2;
const fs::path tmpdir = [&] { try { return fs::temp_directory_path(); } catch (...) { return fs::path(); } }();
Expand Down Expand Up @@ -4908,4 +4918,17 @@ void evict_stale_canonical_copies(const std::string& dir, const std::string& kee
}
}

void sweep_fconst_scratch_dirs(const std::vector<std::string>& dirs, const std::string& written_dir,
const std::string& keep_name, const std::string& prefix, const std::string& legacy_name,
int64_t max_age_seconds) {
// engine#1907: every candidate is swept, not only the one being written. A directory that does not exist is skipped,
// never created: only the written one is made ready, by its caller.
for (const auto& dir : dirs) {
std::error_code ec;
if (dir == written_dir || fs::is_directory(dir, ec))
evict_stale_canonical_copies(dir, keep_name, prefix, legacy_name, max_age_seconds);
}
}


} // namespace onebit::npu::pack_import
8 changes: 8 additions & 0 deletions npu/pack_import.h
Original file line number Diff line number Diff line change
Expand Up @@ -207,6 +207,14 @@ void evict_stale_canonical_copies(const std::string& dir, const std::string& kee
const std::string& legacy_name = std::string(),
int64_t max_age_seconds = 0);

// engine#1907: apply evict_stale_canonical_copies() to EVERY scratch candidate, not only the one being written. Since #1801 the
// write lands in the on-disk cache, so a sweep of that directory alone never reached /tmp again, and the copies older binaries
// had left in tmpfs (17.0 GiB of RAM, measured 2026-11-10) stayed until reboot. `written_dir` is swept even if it is not yet a
// directory as seen here; every other entry is swept only when it already exists, and is never created.
void sweep_fconst_scratch_dirs(const std::vector<std::string>& dirs, const std::string& written_dir,
const std::string& keep_name, const std::string& prefix, const std::string& legacy_name,
int64_t max_age_seconds);

// The text artifacts a runnable container needs beside model.q4nx, and which of them the pack
// actually carries. A pack fetched weight by weight can be weights-only (engine#1024; AMD's repos
// do publish the text, and how much of it varies per pack), and an importer that says nothing about
Expand Down
61 changes: 60 additions & 1 deletion tests/npu_pack_import_test.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -731,6 +731,44 @@ void test_evict_stale_canonical_copies() {
" geometry/age rules + 3 degenerate)\n");
}

// engine#1907: the sweep ran only in the directory being written. Since #1801 that is the on-disk cache, so /tmp was never swept
// again, and 17.0 GiB of copies left there by older binaries stayed in tmpfs (RAM). Every candidate is swept now; one that does
// not exist is not created.
void test_sweep_fconst_scratch_dirs() {
const fs::path root = fs::temp_directory_path() / ("1bit-sweep-" + std::to_string(::getpid()));
std::error_code ec;
fs::remove_all(root, ec);
const fs::path written = root / "cache", other = root / "tmp", absent = root / "never-made";
fs::create_directories(written);
fs::create_directories(other);
auto touch = [&](const fs::path& d, const std::string& base) {
std::ofstream o(d / base, std::ios::binary | std::ios::trunc);
o << 'x';
};
const std::string prefix = "1bit-fconst19v1-12345-896-14-24-";
const std::string keep = prefix + "291653952-76263210.fconst";
const std::string stale = prefix + "291653952-99999.fconst";
const std::string legacy = "1bit-fconst19-12345-896-14-24.fconst";
const std::string old_other = "1bit-fconst19v1-77777-1536-12-28-995712640-73345656.fconst";
const std::string fresh_other = "1bit-fconst19v1-88888-1536-12-28-995712640-73345656.fconst";
for (const auto& f : {keep, stale, legacy, old_other, fresh_other}) touch(other, f);
touch(written, keep);
const auto seven_hours_ago = fs::file_time_type::clock::now() - std::chrono::hours(7);
fs::last_write_time(other / old_other, seven_hours_ago, ec);
pi::sweep_fconst_scratch_dirs({written.string(), other.string(), absent.string()}, written.string(), keep, prefix, legacy,
6 * 3600);
const auto in_other = [&](const std::string& b) { return fs::exists(other / b); };
check(!in_other(old_other), "sweep: an abandoned copy in a directory that was NOT written goes (the /tmp left behind by #1801)");
check(!in_other(stale) && !in_other(legacy),
"sweep: a stale identity and the legacy name of this pack go from the other directory too");
check(in_other(keep), "sweep: the exact name is kept in every directory (another environment may reuse it there)");
check(in_other(fresh_other), "sweep: a fresh copy of another pack stays -- another run may be writing it");
check(fs::exists(written / keep), "sweep: the copy being written is kept");
check(!fs::exists(absent), "sweep: a candidate that does not exist is not created");
fs::remove_all(root, ec);
std::printf(" scratch sweep: every candidate directory, 6 arms\n");
}

void test_modelbin_genre_refusal(const std::string& root) {
const fs::path dir = fs::path(root) / "crafted-modelbin";
fs::remove_all(dir);
Expand Down Expand Up @@ -2891,6 +2929,19 @@ void test_fconst_block_headers() {
// cache whose size is the pack's own .fconst, and on this box /tmp is tmpfs, so where it lands is a RAM decision. The bug was a
// cache directory that was never created -- every pack then fell through to /tmp and nothing noticed.
void test_fconst_scratch_order() {
// engine#1907: this arm rewrites four variables, and it used to leave HOME=/home/x with TMPDIR and the override unset, so
// every crafted import after it fell through to the literal /tmp (179 of the 188 files found there were this suite's).
// Save and restore them, so the rest of the run keeps the per-run scratch directory main() set.
struct Saved { const char* name; bool had; std::string value; };
std::vector<Saved> saved;
for (const char* n : {"ONEBIT_FCONST_SCRATCH", "XDG_CACHE_HOME", "HOME", "TMPDIR"}) {
const char* v = std::getenv(n);
saved.push_back({n, v != nullptr, v ? v : ""});
}
struct Restore {
std::vector<Saved>& s;
~Restore() { for (const auto& x : s) { if (x.had) setenv(x.name, x.value.c_str(), 1); else unsetenv(x.name); } }
} restore{saved};
const auto first_with = [](const char* scratch, const char* xdg, const char* home) {
if (scratch) setenv("ONEBIT_FCONST_SCRATCH", scratch, 1); else unsetenv("ONEBIT_FCONST_SCRATCH");
if (xdg) setenv("XDG_CACHE_HOME", xdg, 1); else unsetenv("XDG_CACHE_HOME");
Expand Down Expand Up @@ -2945,7 +2996,6 @@ void test_fconst_scratch_order() {
check(std::count(no_tmpdir.begin(), no_tmpdir.end(), std::string("/tmp")) == 1,
"fconst scratch: with TMPDIR unset, /tmp appears exactly once (no duplicate from the resolver)");
std::system(("rm -rf " + tdir).c_str());
setenv("HOME", "/home/x", 1); // leave the environment as the arms above expect it
}
std::printf("fconst scratch: order is override -> disk cache -> temp/cwd, pinned\n");
}
Expand Down Expand Up @@ -3662,6 +3712,14 @@ void test_qk_norm_refusal() {
}

int main(int argc, char** argv) {
// engine#1907: every crafted 1.9 import writes a canonical copy into the scratch cache. Without this the suite left 0-byte
// copies in the user's real ~/.cache/1bit/fconst19 and in /tmp on every run. Give the run its own directory, removed at exit.
const fs::path run_scratch = fs::temp_directory_path() / ("1bit-pack-test-scratch-" + std::to_string(::getpid()));
if (!std::getenv("ONEBIT_FCONST_SCRATCH")) setenv("ONEBIT_FCONST_SCRATCH", run_scratch.c_str(), 1);
struct ScratchCleanup {
fs::path p;
~ScratchCleanup() { std::error_code ec; fs::remove_all(p, ec); }
} scratch_cleanup{run_scratch};
test_qk_norm_refusal();
test_tail_head_placement();
const std::string golden_dir = argc > 1 ? argv[1] : "";
Expand All @@ -3683,6 +3741,7 @@ int main(int argc, char** argv) {
test_external_container_names();
test_external_packed_genre(golden_dir.empty() ? "." : golden_dir);
test_canonical_copy_reusable(); test_evict_stale_canonical_copies();
test_sweep_fconst_scratch_dirs();
test_crafted_refusals(golden_dir.empty() ? "." : golden_dir);
test_modelbin_genre_refusal(golden_dir.empty() ? "." : golden_dir);
test_1671_plan_head_arm(golden_dir.empty() ? "." : golden_dir);
Expand Down
Loading