Skip to content

q4nx cache: the merged sweep has no invoker, so a box that never serves never evicts (59 G in 12 h) #2263

Closed

Description

@bong-water-water-bong

What

1bit q4nx-cache evict (engine#1788, merged as #2251) is correct, tested and merged — and nothing invokes it. Verified on this box:

$ git grep -ln "q4nx-cache" origin/main -- .
app/main.cpp          # the dispatch
app/q4nx_cache.{h,cpp} # the implementation
tests/q4nx_cache_test.cpp
$ systemctl list-timers --all | grep -i 'q4nx\|1bit'   # nothing
$ crontab -l | grep -i q4nx                            # nothing
$ grep -rl "q4nx-cache" ~/.config/systemd              # nothing

scripts/weekly.sh is the Sunday release script, not a housekeeping hook, so there is no in-repo periodic invoker either.

Why the code cannot fix this by itself

The cache has two eviction triggers: serve start (evict-on-write) and the manual command. Both require running the binary. A box that reviews PRs all night and never starts a serve — which is what a review/CI lane is — runs neither, so the cache grows and the box falls under the build gate.

Evidence

On this box, with the command present and merged for 9 hours:

cache entries disk free
after the last hand-prune 26 G 10 —
12 h later, #2251 merged 59 G 12 172 G (gate 175 G)
after a manual 24 h age cut 24 G 5 211 G

The 59 G→24 G reclaim needed nothing but the command's own --min-age policy. The manual prune is a patch over the gap, not a fix: it will regrow the same way.

What a fix looks like

A scheduler — systemd --user timer or equivalent — running 1bit q4nx-cache evict --max-gib 40 --min-age 86400 hourly. That is a box-configuration change rather than a code change, so it needs an owner decision rather than a PR.

If instead you prefer it in-repo, the honest options are thin:

  • evict on serve exit as well as start — helps a box that serves once, not one that never serves;
  • a scripts/ hook that some external orchestrator calls — still an invoker, just moved.

Neither removes the need for something outside the repo to call 1bit.

Why it matters now

Disk free decides whether any lane may build (the 175 G gate). A 59 G cache against a 931 G volume took the box to 3 G below the gate, which is a fleet-wide build freeze for a cache that is regenerated by re-importing a pack.

Activity

  1. bong-water-water-bong commented on Nov 12, 2026

    @bong-water-water-bong
    CollaboratorAuthor

    Call-site census: three writers, one evictor, and the evictor is behind serve

    Root-cause detail that settles how this should be fixed rather than leaving it a choice:

    Writers of ~/.cache/1bit/q4nx (all via the one naming scheme tag + "-q4nx1-" + hex16(fnv1a(source)), npu/q4nx_device.cpp:181):

    • npu/q4nx_device.cpp — the conversion itself; its reuse path already touches the entry's mtime as "a use", in those words, with a comment pointing at this cache's sweep. So the producer already cooperates with a sweep that nothing runs.
    • npu/forward/src/q4nx_forward.cpp:889 — the per-op forward's conversion.
    • app/serve.cpp — the restore/load path.

    Evictor, the whole of it:

    $ git grep -n "q4nx_cache_evict" origin/main -- app npu tools
    origin/main:app/serve.cpp:2977:  q4nx_cache_evict(cache, keep, q4nx_cache_bound_bytes(...));
    

    One call site, in serve start.

    Why this changes the fix

    An evict-on-conversion anchored where entries are created would be the natural code fix — but npu/q4nx_device.cpp cannot call app/q4nx_cache.h without inverting the app/ → npu/ layering, and its only non-serve caller is also in npu/. So the two code options are:

    1. Move the sweep down into npu/ (e.g. npu/q4nx_cache.h) and have app/ include it. A real change, and the owner's call on layering.
    2. Evict at the app layer wherever a command converts — but the only app paths that convert are serve and forward-serve, which already evict or could.

    And neither covers the case that produced this issue: a box that runs neither. On a review/CI box nothing converts, nothing serves, and no eviction trigger exists by design — so the code is behaving as written and the gap is that no outside actor calls 1bit.

    Therefore

    The hourly timer remains the only fix that covers a box that never runs the binary. The in-repo work worth doing is (1) — a small layering change that bounds the cache for every box that does convert — but it should be filed as its own change, not sold as a fix for this one.

    I am not treating "add an evict call" as the fix here, because it would look like one and leave this box exactly where it was.

  2. bong-water-water-bong commented on Nov 12, 2026

    @bong-water-water-bong
    CollaboratorAuthor

    Correction to my own census: the obstacle is a static-library cycle, not a layering rule

    My previous comment said an evict-on-conversion "cannot call app/q4nx_cache.h without inverting the app/ → npu/ layering". That was too vague to act on, and partly wrong. The verified facts:

    CMakeLists.txt:98    add_library(onebit ... app/embed_serve.cpp app/embed_model.cpp app/q4nx_cache.cpp)
    CMakeLists.txt:848   add_library(onebit_npu STATIC ... npu/q4nx_device.cpp ...)
    CMakeLists.txt:849   target_include_directories(onebit_npu PUBLIC npu)
                         onebit links: onebit_ep onebit_jinja onebit_laya onebit_moe onebit_npu onebit_npu_dx onebit_npu_forward
    
    • Include paths are not the obstacle. Several test targets already carry ${CMAKE_SOURCE_DIR}/app and ${CMAKE_SOURCE_DIR}/npu together (systemone_route_test, npu_chatml_test, npu_template_test), so mixing the two headers is routine here.
    • The obstacle is that app/q4nx_cache.cpp is compiled into onebit, and onebit links onebit_npu. Calling q4nx_cache_evict from npu/q4nx_device.cpp would make onebit_npu need onebit — a static-library cycle, which is a link error rather than a style objection.

    So the fix shape is small and now known: extract the sweep into its own target, since app/q4nx_cache.cpp depends on nothing but the filesystem —

    add_library(onebit_q4nx_cache STATIC app/q4nx_cache.cpp)
    target_link_libraries(onebit     PRIVATE onebit_q4nx_cache)
    target_link_libraries(onebit_npu PRIVATE onebit_q4nx_cache)
    

    after which npu/q4nx_device.cpp can evict at the conversion site and every box that converts is bounded — import, container, the device tests, benches — instead of only a box that starts a serve. q4nx_cache_test (CMakeLists.txt:1265, which already builds app/q4nx_cache.cpp directly) should link the new target rather than compile the file a second time.

    What this does and does not fix: it removes the unbounded-growth property for the boxes that actually produced 59 G. It still does not cover a box that converts nothing, which is why the hourly timer in the original report stands as the only fix for a review/CI box.

    I am not opening this PR myself: the eviction lands on the serve/import path, and with the NPU off (owner, 11-12 01:05Z) I can neither build onebit_npu nor exercise a conversion end to end — so I would be shipping a CMake refactor I could not run. It is a good first PR for whoever has a build window, and the three facts above are what they need.

  3. bong-water-water-bong commented on Nov 12, 2026

    @bong-water-water-bong
    CollaboratorAuthor

    The invoker is merged (#2269), so this closes — with the one boundary the fix has, stated plainly.

    scripts/install-q4nx-cache-timer.sh is on main:

    • what it does: installs a systemd --user timer that runs 1bit q4nx-cache evict --max-gib 40 --min-age 86400 hourly — the invoker this issue is about, since the two existing triggers (the write-time sweep inside serve, and the manual command) both need a binary to run, which a review/CI/build box never starts.
    • it is opt-in and inert until run: it writes only into ~/.config/systemd/user/, needs no root, and changes nothing in the repo. --dry-run prints the units and verifies them with systemd-analyze, --uninstall removes them and leaves the cache alone.
    • it refuses three silent-no-op installations: a binary older than the command (1bit q4nx-cache printing unknown command — this box's deployed rc-2026-11-03-c22 is one), --max-gib 0 (which the CLI documents as sweep disabled, so the timer would fire hourly and do nothing), and an empty --interval (an OnCalendar= that never fires).

    What remains, and it is a box action rather than a code one:

    scripts/install-q4nx-cache-timer.sh --dry-run     # read the units
    scripts/install-q4nx-cache-timer.sh               # install, hourly, 40 GiB bound, 24 h age floor
    

    Until that runs on a box, that box still never sweeps — the repository ships the invoker, it does not install itself. Measured on this box while it was open: 23 G in the cache with head-images/ alone 15.4 G at 4 h old, and #2263's own case was 59 G against a 931 G volume. The bound is the defence; --min-age 3600 --interval '*:0/30' is the supported faster pair if a box needs one sooner than the 24 h floor allows.

    Closing: the missing invoker was the defect, and it is no longer missing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions