Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions docs/npu.md
Original file line number Diff line number Diff line change
Expand Up @@ -1030,6 +1030,26 @@ Two facts about what the container holds, both reported, not hidden:
scale 0, zero 0). That is 12.5 % more bytes than the pack; the true K is in the container's
manifest and in `config.json`, and nothing reads a padding column as a weight.

### Where a 1.9 pack's rewritten `.fconst` goes

A Ryzen AI 1.9 pack's Token `.fconst` is rewritten into its 1.8 form before it is read. The copy is as large as the source
(3.6 GB for an h4096 pack), so where it lands matters: on strixhalo `/tmp` is tmpfs, which means RAM. The importer tries these
directories in order:

1. `$ONEBIT_FCONST_SCRATCH`, if set;
2. `$XDG_CACHE_HOME/1bit/fconst19`, or `~/.cache/1bit/fconst19` (on disk);
3. `$TMPDIR`, then `/tmp`, then the current directory.

The copy is a cache: it is reused while the source's size and mtime and the rules version match. On each write the importer
sweeps the scratch directories. It removes other identities of the same pack and geometry, the pre-version name, and any
`1bit-fconst19*` copy nobody has written for 6 hours. Without an override, **every** candidate directory is swept, not only the
one being written, so copies older binaries left in `/tmp` get reclaimed (engine#1907). With `ONEBIT_FCONST_SCRATCH` set, only
that directory is swept, and a run with an override never removes another lane's files. The current directory is the
one candidate that is never swept unless the copy is being written there: it is only the last write fallback, and a
working directory's files are not the importer's to clean. The 6 hours are what keep a concurrent run's copy alive in a
shared directory, so the number stays above the longest import; it is not the knob for disk or tmpfs pressure. The test suite sets one per run and
removes it at exit.

### Linking the kernel set the container will be served with

A model directory carries its own NPU kernels in `npu/` (see "A model directory"), and for an imported
Expand Down
43 changes: 42 additions & 1 deletion npu/pack_import.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -763,6 +763,9 @@ std::string materialise_token_fconst(const std::string& pack_dir, const Dims& di
constexpr const char* kScratchStem = "1bit-fconst19";
// A scratch copy older than this is a pack no run is importing: no import holds one for hours, so an mtime this far back
// means the pack was abandoned, not that somebody is using it. Kept well above the longest import measured (~minutes).
// This number is what protects a CONCURRENT run's copy in a shared directory (the sweep reaches every candidate
// directory now, engine#1907), so it has to stay above the longest import: do not lower it to relieve disk or tmpfs
// pressure -- remove the files, or point ONEBIT_FCONST_SCRATCH at disk, instead.
constexpr int64_t kScratchMaxAgeSeconds = 6 * 3600;
// Everything but the source identity: the group this run evicts its stale siblings from (engine#1770).
const std::string ident_prefix = "1bit-fconst19v" + std::to_string(kCanonicalFormVersion) + "-" + tag + "-" +
Expand Down Expand Up @@ -798,7 +801,17 @@ std::string materialise_token_fconst(const std::string& pack_dir, const Dims& di
// GiB of tmpfs, had been resident all day).
const std::string legacy_name = std::string(kScratchStem) + "-" + tag + "-" + std::to_string(dims.hidden) + "-" +
std::to_string(dims.heads) + "-" + std::to_string(dims.layers) + ".fconst";
evict_stale_canonical_copies(d.string(), name, ident_prefix, legacy_name, kScratchMaxAgeSeconds);
// Sweep EVERY candidate directory, not only the one written (engine#1907). Since #1801 moved the cache to disk, a
// write lands in ~/.cache, so the sweep never visited /tmp again, and the copies older binaries had left there
// (17.0 GiB of tmpfs, i.e. RAM, measured 2026-11-10) stayed until reboot. The other directories are only swept,
// never created, and the rule is the same one as here: the exact name is kept wherever it is.
// An explicit ONEBIT_FCONST_SCRATCH is a lane saying "my scratch is here": then only that directory is swept, so a
// run with an override (the test suite sets one) never removes another lane's files from the shared /tmp or cache.
std::vector<std::string> dir_names;
const char* override_dir = std::getenv("ONEBIT_FCONST_SCRATCH");
if (override_dir && *override_dir) dir_names.push_back(d.string());
else for (const auto& x : dirs) dir_names.push_back(x.string());
sweep_fconst_scratch_dirs(dir_names, d.string(), name, ident_prefix, legacy_name, kScratchMaxAgeSeconds);
// Say where it went whenever that is not the ordinary place: the copy is a few hundred MB and it is scratch.
std::error_code ec2;
const fs::path tmpdir = [&] { try { return fs::temp_directory_path(); } catch (...) { return fs::path(); } }();
Expand Down Expand Up @@ -5623,4 +5636,32 @@ void evict_stale_canonical_copies(const std::string& dir, const std::string& kee
}
}

void sweep_fconst_scratch_dirs(const std::vector<std::string>& dirs, const std::string& written_dir,
const std::string& keep_name, const std::string& prefix, const std::string& legacy_name,
int64_t max_age_seconds) {
// engine#1907: every candidate is swept, not only the one being written. A directory that does not exist is skipped,
// never created: only the written one is made ready, by its caller.
//
// The CURRENT DIRECTORY is the one candidate that is not this importer's to clean (#1915 review): it is in the list
// only as the last write fallback, and a lane that runs `npu-pack` from inside its own scratch would otherwise have
// that scratch's old `1bit-fconst19*` files removed. So it is swept only when it is the directory being written --
// which is what happened before #1907 and is the evict-on-write rule of engine#1770, unchanged. The comparison is
// on canonical paths: `.`, a relative spelling and the absolute path are the same directory. A directory listed
// twice is swept once.
std::error_code cwd_ec;
const fs::path cwd = fs::weakly_canonical(fs::current_path(cwd_ec), cwd_ec);
std::set<std::string> seen;
for (const auto& dir : dirs) {
if (dir.empty()) continue;
std::error_code ec, cec;
const bool written = dir == written_dir;
const fs::path canon = fs::weakly_canonical(dir, cec);
if (!written && (cwd_ec || cec || canon == cwd)) continue;
if (!cec && !seen.insert(canon.string()).second) continue;
if (written || fs::is_directory(dir, ec))
evict_stale_canonical_copies(dir, keep_name, prefix, legacy_name, max_age_seconds);
}
}


} // namespace onebit::npu::pack_import
8 changes: 8 additions & 0 deletions npu/pack_import.h
Original file line number Diff line number Diff line change
Expand Up @@ -307,6 +307,14 @@ void evict_stale_canonical_copies(const std::string& dir, const std::string& kee
const std::string& legacy_name = std::string(),
int64_t max_age_seconds = 0);

// engine#1907: apply evict_stale_canonical_copies() to EVERY scratch candidate, not only the one being written. Since #1801 the
// write lands in the on-disk cache, so a sweep of that directory alone never reached /tmp again, and the copies older binaries
// had left in tmpfs (17.0 GiB of RAM, measured 2026-11-10) stayed until reboot. `written_dir` is swept even if it is not yet a
// directory as seen here; every other entry is swept only when it already exists, and is never created.
void sweep_fconst_scratch_dirs(const std::vector<std::string>& dirs, const std::string& written_dir,
const std::string& keep_name, const std::string& prefix, const std::string& legacy_name,
int64_t max_age_seconds);

// The text artifacts a runnable container needs beside model.q4nx, and which of them the pack
// actually carries. A pack fetched weight by weight can be weights-only (engine#1024; AMD's repos
// do publish the text, and how much of it varies per pack), and an importer that says nothing about
Expand Down
Loading
Loading