Skip to content

Show versions and labelled sections on the Statistics page - #301

Merged
danielplohmann merged 1 commit into
masterfrom
feat/statistics-page-versions
Oct 29, 2026
Merged

danielplohmann merged 1 commit into
masterfrom
feat/statistics-page-versions

Conversation

@r0ny123

@r0ny123 r0ny123 commented Oct 20, 2026 •

Copy link
Copy Markdown
Collaborator

The Statistics page printed the backend's status as raw key names, and only the first twelve of them ({% if loop.index <= 12 %} in statistics.html:9 on master 6564602). mcrit 1.13 reports 15 keys and 1.16 reports 17, so the page cut off the MinHash compatibility threshold and every stale-hash count, which are the rows that say a maintenance job is due. Neither MCRITweb's version nor the backend's was on the page. This shows the versions, puts the rest into a few labelled sections, and drops nothing the backend sends.

Change.

  • statistics_page() (mcritweb/statistics.py:136) turns getStatus()'s answer into what the template renders:
    • Versions: MCRITweb (MCRITWEB_VERSION), the MCRIT backend (backend_version()), the mcrit client library installed next to MCRITweb (mcrit_client_version(), statistics.py:72), and the backend's SMDA. The client library is there because requirements.txt only sets a floor, so it can drift from the backend's version.
    • Corpus: families, samples, functions, unique PicHashes, with thousands separators (CORPUS_FIELDS, statistics.py:17).
    • Index: storage, MinHash bands, the MinHash compatibility threshold, the database revision and the time of its last change, in UTC, the clock MongoDbStorage._getCurrentTimestamp uses on every release from 1.12 on (INDEX_FIELDS, statistics.py:23).
    • Maintenance: one note per count above zero, saying what it means and which job redoes it (MAINTENANCE_COUNTS, statistics.py:35). The meanings are taken from the comments next to each count in mcrit's MinHashIndex.getStatus: stale MinHashes are redone by Repair stale MinHashes, stale block hashes and stale PicHashes by Recalculate PicHashes, and unrehashable samples cannot be. inline_xcfg_remaining points at mcrit's migrate_xcfg_split, which runs on the backend host. When nothing is due, the section says so.
    • Escaper fingerprints per architecture, or Intel's alone from a backend older than 1.13.
    • Other: any key the page has no label for (statistics.py:160), so a newer backend is never trimmed again.
  • explore.statistics (mcritweb/views/explore.py:873) passes the status and both versions. It keeps asking for the distinct PicHash count, the client's default: the count is one of the corpus figures, and Count distinct PicHashes for /status from the count index mcrit#282, which answers it from pichash_counts, makes it cheap.
  • backend_version() moves from views/administration.py to views/client.py:108, unchanged, so the admin server page and this page share it.
  • A backend that gives no status still gets a page with the versions and a sentence saying the status is missing.
  • The manual's Statistics section describes the sections.

The page is visitor-level, as before. It shows versions and counts only: no paths, host names, URLs or tokens.

Verification. On the testbed, master (6564602) and this branch as side instances against the same backend, logged in as a visitor, GET only, first with mcrit 1.13.0 and smda 4.9.0, then after upgrading the same corpus to mcrit 1.16.0 and smda 4.11.0:

  • master: 12 rows with key names, ending at inline_xcfg_remaining. minhash_compatibility_threshold and every stale count are not on the page, on both releases.
  • this branch on 1.13.0: MCRITweb 1.6.0, MCRIT backend 1.13.0, mcrit client library 1.13.0, SMDA 4.9.0; Functions 26,436, Unique PicHashes 10,426; threshold 4.4.5; last change 2026-09-23 20:30:12; "The backend reports nothing that needs a maintenance job"; five escaper fingerprints.
  • this branch on 1.16.0: MCRIT backend 1.16.0, mcrit client library 1.16.0, SMDA 4.11.0, and one maintenance note: "4 samples have PicHashes from an smda older than the compatibility threshold. An admin can run Recalculate PicHashes under Server Management." Those are the corpus's four samples from smda 1.5.12. On master that count is the 16th key and is not shown. Running Recalculate PicHashes afterwards cleared the count, and the note went away.
  • No script errors on either page. The page makes three backend requests where master made two: the reachability probe, /status?with_pichash=True, and /version.

check_corpus.py: as in the snapshot.

Tests. New tests/testStatisticsPage.py, with the testbed's status answers under mcrit 1.13.0 and 1.16.0 as captured, and one with 1.16's keys and every maintenance job due:

  • the three versions are shown; a backend that gives no version or no status says "unknown" or "not reported", and the page still renders;
  • sections carry labels rather than key names on both releases, and the timestamp reads as UTC;
  • a healthy backend gets no maintenance note, the upgraded testbed gets the one for its four samples, and each due job gets one;
  • a key the page does not know is listed under Other;
  • whatever the backend sends is escaped.

tests/testServerStatus.py: the test that pinned the 12-row cap now checks that every key is shown, and the captured-status test checks the labels.

18 of the 23 new or changed tests fail on master. test_what_the_backend_sends_is_escaped is a guard and passes there too, since master printed the values through autoescaping as well. Full suite with Chromium on e7f4ff6: 2090 passed, 11 skipped; ruff check . clean.

Taking out the escaping, the Other list, the above-zero check, the xcfg note, the number formatting, the backend version or the MCRITweb fallback each fails at least one test.

Relation to open PRs. A trial merge (git merge-tree) with every open PR:

The page printed the first twelve keys of the backend's status as they
came, raw names and all. On mcrit 1.13 that cut off the stale MinHash
and block hash counts, the rows that say a maintenance job is due, and
the versions of MCRITweb and of the backend were nowhere on it.

statistics_page() in mcritweb/statistics.py turns the status answer
into a Versions, a Corpus and an Index section with labels and
thousands separators, a Maintenance list that names a count only when
it is above zero together with the job that redoes it, and the escaper
fingerprints per architecture. A key it has no label for goes under
Other, so a newer backend is never trimmed. The versions are
MCRITweb's (MCRITWEB_VERSION), the backend's (backend_version, moved
from the admin view to views/client.py so both pages share it), the
installed mcrit client library's and the backend's SMDA.

The database's last change is shown as a UTC date and time, which is
the clock mcrit stamps it with on every release from 1.12 on.

The page keeps asking for the distinct PicHash count, which is one of
the corpus figures. A backend without a status answer still gets the
versions and a sentence saying so.
@danielplohmann
danielplohmann merged commit f8bb82f into master Oct 29, 2026
8 checks passed
danielplohmann added a commit that referenced this pull request Oct 29, 2026
r0ny123 added a commit that referenced this pull request Oct 29, 2026
Master's templates (#253's layout pass, #254's token notices, #301's
Statistics page and others) use classes the cut did not keep, among them
.table-light, .breadcrumb, .btn-outline-secondary and -danger and a few
spacing utilities. scripts/subset_bootstrap.py adds their rules: 54,417
bytes now.
r0ny123 added a commit that referenced this pull request Oct 30, 2026
Master's legacy token notices and statistics page (#254, #301) draw
fa-key and fa-triangle-exclamation; scripts/subset_fontawesome.py adds
their two glyph rules to fontawesome-subset.css.
@r0ny123
r0ny123 deleted the feat/statistics-page-versions branch October 30, 2026 06:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants