Skip to content

Stop the remaining head scripts and help screenshots from delaying first paint (#63, stacked on #126) - #245

Open
r0ny123 wants to merge 27 commits into
familiary:masterfrom
r0ny123:fix/lighthouse-performance
Open

r0ny123 wants to merge 27 commits into
familiary:masterfrom
r0ny123:fix/lighthouse-performance

Conversation

@r0ny123

@r0ny123 r0ny123 commented Sep 27, 2026 •

Copy link
Copy Markdown
Collaborator

Stacked on #126. Merge that first; until then this diff also shows #126's commits. The three commits of this PR are the last three.

Part of #63. With #126 in place, a Lighthouse 13.5 desktop run of a local build scores Performance 100 on the index and /explore/samples, against 83-87 on master. These commits close the remaining render-blocking and LCP findings on other pages:

  • autocomplete.js and page_loading.js are deferred. Nothing calls them while the page parses: page_loading.js only registers listeners, and every new Autocomplete already waits for DOMContentLoaded. jQuery stays a plain script because inline blocks call $ while parsing.
  • On the single function and function comparison pages, d3, graphlib, graphlib-dot, dagre-d3 and convexhull move from <head> to immediately before main.js/main_duo.js, in the same order. That is about 120 kB gzipped that no longer holds back first paint. Function comparison goes from 99 to 100.
  • /help lazy-loads its screenshots after the first. There are about 2 MB of them, and all were competing with the page's largest paint. /help goes from 99 to 100.

Each follows Chrome's performance insight docs for render-blocking requests and LCP discovery.

Tests

tests/testPageAssets.py pins the defers and the CFG library order, and tests/testManual.py pins the lazy screenshots. ruff check . is clean. pytest passes except the environment-only testPythonFloor case that #242 addresses.

Measured and inferred

Measured, locally: Performance 100 on every page type with the desktop preset, on a build of master + #126 + this PR + the accessibility PR, offline against tests/fixtures. The mobile preset scores 86-100. Recovering the rest there would mean deferring jQuery, which about 20 templates' inline scripts call while the page parses. #304 does that, and measures what it buys: jquery.js leaves the render-blocking list, but the mobile score stays within the spread of its runs.

Inferred: a deployment reaches these numbers only once the assets are compressed (#126 does it in the app for stylesheets, scripts and JSON; HTML pages stay uncompressed because of the CSRF token they carry) and the backend round trip is short (#114). Production currently serves uncompressed assets.

The accessibility PR (#244) is merged now. Master (6564602, batch C) and the current #126 are merged in, without conflicts.

8cba370 merges #126's 4c80af2, which renames #126's second ADR to 0017; nothing in this PR's own files changes.

Issue #63, first bullet: "remove data-tables js/css once it is not needed
anymore". base.html loaded 623 KB of render-blocking JavaScript and 41 KB of
CSS on every page in the app, for two calls.

DataTables is removed outright, not moved. Its one initialiser selected
`#job-table`, and the jobs page renders its table with `table_id=active` -
the job method - so the selector matched nothing on every request. Confirmed
against the running app: `id="job-table"` appears in zero of /data/jobs,
?active=..., ?state=... and /. It was also configured with paging, searching
and info all off, so even where it had matched it would only have restyled
the table.

jQuery UI moves to result_cross.html, the only page that calls `.sortable()`.
{% block style %} runs after base.html's scripts, so jQuery is in place first.

Measured with Chromium, total decoded bytes per page:

    /                     1667 KB -> 1017 KB
    /explore/samples      1667 KB -> 1017 KB
    /data/jobs            1667 KB -> 1017 KB
    /data/result/<cross>  1691 KB -> 1595 KB   (keeps jQuery UI)

Behaviour is unchanged: `.sortable()` still applies on the cross compare, the
clipboard tooltips still get Bootstrap instances rather than jQuery UI's -
both libraries define `.tooltip()`, and removing jQuery UI makes which one
wins deterministic rather than a load-order race - and no page logs a console
error.

Deliberately not done: `defer` on the remaining scripts, and removing jQuery.
Both need an audit of 23 inline blocks and 36 onclick handlers, several of
which call functions declared at the top level of those blocks; that is a
separate change. The remaining majority of the render-blocking weight is CSS,
not JS - bootstrap.css alone is 196 KB and is the unminified build.
Every unit job on every Python version fails with:

    /opt/hostedtoolcache/Python/3.11.16/x64/bin/python: No module named pytest

The Makefile said "pytest and pytest-cov arrive with mcrit, so requirements.txt
covers both". mcrit declares them under its `dev` extra:

    $ python -c "import importlib.metadata as m; print(m.requires('mcrit'))"
    ... 'pytest; extra == "dev"', 'pytest-cov; extra == "dev"' ...

so a plain `pip install -r requirements.txt` does not bring them in, and the
suite has not actually run in CI since mcrit moved them there. Ruff is green
throughout, which is why this looked like a working pipeline.

Installed in the workflow rather than added to requirements.txt: neither is a
runtime dependency and a deployment should not get them. Pinned, for the same
reason the lint job pins ruff. The two stale comments that said otherwise are
corrected.
`init` still ran `pip install -r requirements.txt` alone, so a fresh checkout
that follows the README lands without pytest and `make test` fails with
"No module named pytest" - the same failure the CI change on this branch fixes
for the runner, left unfixed for a contributor.

pytest and pytest-cov are not runtime dependencies and are not in
requirements.txt; mcrit declares them under its `dev` extra, which nothing
here requests, so they do not arrive transitively either. Install them from
`init`, pinned to the versions CI uses, and say so once above the target
rather than in a comment further down that describes a step that never ran.

Reported by Codex review.
…rst one was safe

The previous commit did #63's first bullet. The other four are "consider", "enable
text compression" and "what to inline"; a recorded, measured decision is the
resolution for those, so ADR-0015 makes them. Numbers first, all from Chromium
driving the app over real HTTP against the captured corpus.

  * text compression is the largest win left, and it is not ours. Nothing is
    compressed anywhere: Flask's static handler does not, and neither does the
    reference deployment - nginx.conf, mcritweb_plain.conf and mcritweb_ssl.conf in
    danielplohmann/docker-mcrit at 7e9501a carry no gzip directive, and location /
    proxies /static/ straight through. Simulating `gzip on;` in front of this branch,
    cold Fast-3G load of /, median of 7: first paint 3872 ms -> 1024 ms, wire
    1,069,548 B -> 354,002 B. That is more than the whole of the first bullet, and it
    belongs in docker-mcrit.

  * lazy dropzone: right target, wrong shape today. dropzone.js is 353,924 B, 38% of
    every page, but it loads at the end of <body>, so blocking it outright - an upper
    bound - moves first paint by 8 ms. It buys DOMContentLoaded (1585 -> 1009 ms) and
    23% of the transfer. The cost is Flask-Dropzone: dropzone.config() emits
    Dropzone.options.myDropzone *after* the library and only works because discovery
    waits for DOMContentLoaded. Injected late, contentLoaded() sees readyState
    "complete" and binds the element with default options - no CSRF header, no
    autoProcessQueue: false. Doing it right means replacing create()/config() by hand
    in the one place static/dropzone.js's local patch has to keep working.
    "autocomplete on mouseover" is 4,621 B, 1,507 gzipped: no.

  * removing jQuery: no. 15 of 26 inline blocks call $, 36 inline event attributes
    hang off them, trace_CFG/main.js and main_duo.js expect it from base.html, jQuery
    UI cannot load without it, and .tooltip()/.modal() go through Bootstrap's jQuery
    bridge. 89,503 B, 30,882 gzipped - under 9% of a compressed page.

  * inline vs reference is already right: 7,141 B of inline JS against 902,940 B of
    referenced library. The remaining CSS weight (bootstrap.css is the unminified
    build) compresses 8x, so gzip answers that too.

ADR-0004 asked #63 to assert rather than assume which library owns $.fn.tooltip, and
called the current arrangement a load-order race. Measured, it is not one. Bootstrap
5.0.2 defines its jQuery plugins on DOMContentLoaded, after every plain <script> in
the document, so it wins the name in both arrangements - jQuery UI before it, as
base.html used to, and after it, as result_cross.html now does. Checked on every page
that renders a [data-toggle="tooltip"] target: $.fn.tooltip.Constructor.NAME is
"tooltip", bootstrap.Tooltip.getInstance(el) is set, $(el).data("ui-tooltip") is not.
What does flip it is loading jQuery UI *after* DOMContentLoaded, which is exactly what
a lazy loader would do - so the hazard is lazy loading, not ordering.
test_bootstrap_is_loaded_synchronously pins that.

The one factual correction: the previous commit said DataTables' initialiser "selected
an element the app never renders". The app does render id="job-table" - job_table
still defaults to it and job_overview and single_sample take the default, confirmed on
/explore/samples/1. What made DataTables dead is narrower: the initialiser lived on
the jobs page, which passes table_id=active. The removal stands; the reason in the
comments and the test docstring did not, and now says what was measured.

New tests, each mutation-checked by breaking the code it guards:

  TestTheJobsPage             - none of the three jobs views renders id="job-table"
  TestTheCrossComparePage     - the rendered page ships jquery-ui, jquery.js reaches
                                it first, and no other page ships it at all
  test_only_base_html_loads_a_jquery      - both CFG templates' second jQuery stays
                                            commented out
  test_bootstrap_is_loaded_synchronously  - no defer/async on the bundle

code_of() now strips HTML comments as well as Jinja ones, which is what makes the
second-jQuery check see the commented-out includes as absent rather than present.
…zone, gzip

Font Awesome is reduced to the icons the templates use, Bootstrap ships
minified and deferred, the Dropzone script and stylesheet load on the first
drag or on opening the modal, and responses over 1 KiB of compressible types
leave gzipped (MCRITWEB_COMPRESS_RESPONSES turns it off behind a proxy that
compresses itself).
ADR-0015 had left compression to the proxy and the other bullets to a later
audit. Both happened; 0015 is marked superseded and 0016 holds the
measurements and the rules that follow from them.
The type-ahead conflicts are the ones that matter. This branch's side of
js/ac_family_names.html and the submit dropzone still carried the old
family-name code - names shipped with |tojson and pushed unescaped into a
widget that renders suggestions through innerHTML, which is the injection
master closed in familiary#168 (and familiary#146 moved to a JSON endpoint). Master's
escaped code is kept in both places; what this branch contributes there
is only its reason to change them, that Bootstrap is now deferred, so the
widgets are built once the deferred scripts have run. In
ac_family_names.html the DOMContentLoaded listener is also the scope,
replacing master's IIFE, so this branch's timing test can see it. The
Playwright tests that drive the real widget pass with Bootstrap deferred.

The rest:
- base.html: the minified Bootstrap and the Font Awesome subset, with
  master's signed-in font preload (#62) - the subset loads the same
  ../webfonts/fa-solid-900.woff2 the preload names; Bootstrap and
  post_action.js deferred, page_loading.js kept a plain script as master
  has it; clearDropzone keeps the lazy-load guard with master's const.
- the dropzone keeps mcritInitDropzone(), with master's const myDropzone
  (#61) - nothing outside that function uses it.
- import.html takes master's import status script, with dropzone.min.js.
- README keeps the compression paragraph and master's reverse-proxy
  section.
- the Font Awesome subset is regenerated for three icons master's
  templates added (arrows-rotate, code, diagram-project), nothing else
  changes in it; the subset script lists the webfont file stems as
  non-glyphs, since master's preload and its comment name
  fa-solid-900.woff2 and the icon scan read that as an icon.

1026 passed, ruff clean.
autocomplete.js and page_loading.js were plain scripts in <head>, so they
blocked first paint although nothing calls them while the page parses:
page_loading.js only registers event listeners, and every `new Autocomplete`
already waits for DOMContentLoaded, which fires after all deferred scripts
have run. Both are now deferred alongside Bootstrap and post_action.js.
jQuery stays a plain script because inline blocks call `$` while parsing.
The single function and function comparison pages loaded d3, graphlib,
graphlib-dot, dagre-d3 and convexhull.js as plain scripts in <head>, about
120 kB gzipped (390 kB raw) that held back the first paint although
nothing uses them before the CFG scripts at the end of the page and
window.onload. They now load immediately before main.js / main_duo.js, in
the same order, so every script that uses them still runs after them.
Lighthouse's desktop run of the comparison page no longer lists them as
render-blocking.
/help shows 21 screenshots, about 2 MB, and the browser fetched all of
them on load, so the first one - the page's largest contentful paint -
shared the connection with twenty images below the fold. Every screenshot
after the first now carries loading="lazy". The first stays eager:
lazy-loading the largest paint element would delay it.
With loading="lazy" and no width or height, a screenshot is a 2px placeholder
until it arrives. The browser scrolls to /help#jobs-queue, the screenshots now
in view load and grow, and the heading ends up 4,782px below the window (1400x900)
and stays there. Every section link past the first few screenshots missed.

manual.py reads each PNG's size from its header and writes it on the <img>.
With height: auto in style.css the space is reserved before the image loads,
and the image still scales down with the column. Every anchor now lands at the
top of the window, at 1400x900 and 800x700, and only the screenshots in view
download.
… into fix/63-page-specific-libraries

Master replaced the family names shipped with each page by a type-ahead that
fetches them as they are typed (familiary#192), and the submit dropzone now includes
js/ac_family_names.html instead of building its own list. Kept master's
widget and this branch's reason to touch it: Bootstrap is deferred, so the
widgets are built in a DOMContentLoaded listener rather than master's IIFE.
The dropzone macro takes master's signature without `families` and keeps
`lazy`, which base.html's overlay still passes.
The lazy loader, the query and submit pages and the import page loaded
static/dropzone.min.js, which is stock Dropzone 5.9.2. Only dropzone.js
carries the patch that sends the head of a dropped file, so a drop posted
no file_header and the submit form was no longer pre-filled. All three
load dropzone.js again; dropzone.min.js is the retired file now.

A failed fetch of the script left mcritDropzoneLoading set, so the overlay
never opened again on that page. It is cleared on error, and the next drag
tries again without adding a second stylesheet. A dragover while the
script is on its way no longer queues the same callback again. The two
URLs go into the script through |tojson.
send_file answers If-None-Match against the plain ETag before the
after_request hook renames it, so a client revalidating its gzipped copy
never matched and downloaded every asset again. The conditional check is
made again against the renamed ETag.

HTML is no longer compressed. A page carries the session's CSRF token, the
same in every response, next to input it reflects, which is what a BREACH
attack needs. Downloads, the /api passthrough, bodies of unknown length
and bodies over 1 MiB are left alone too: the exports and the cached
result download are streamed so that they never sit in memory whole, and
reading them in to gzip them undid that.

gzip;q=0 is a refusal; request.accept_encodings reads it as one. The
README said docker-mcrit's NGINX compresses; its configuration has no
gzip directive (ADR 0015).
@r0ny123

r0ny123 commented Oct 19, 2026 •

Copy link
Copy Markdown
Collaborator Author

CI is red only in tests/testSubmitMetadata.py: 8 failed, 1411 passed on b570111, in a file this PR doesn't touch. CI now installs mcrit 1.16.0, whose McritClient uses its own requests.Session subclass, and the test stub doesn't provide it. #297 fixes that. I left the fix out of this PR so the two don't overlap. Once #297 is merged, a rerun here should be green.

…irect in its test

Master's templates now use fa-pen, which the Font Awesome subset did not carry, so
scripts/subset_fontawesome.py is rerun. /data/jobs now redirects to the URL that
names its first tab, so the DataTables id check follows the redirect to the page
that is actually rendered.
Batch C brought master docs/adr/0016-slow-function-search-is-mcrits.md,
so this branch's 0016 shared the number. The record moves to 0017 and
ADR-0015's status line points at it; nothing else names the file.
Each screenshot carries the pixel size of its file as width and height, but the
page is box-sizing: border-box and a screenshot has a 1px border, so the
picture inside was 2px narrower than its file (1265px for a 1267px file) and
was resampled. The border now goes around the size, and the largest width the
column allows loses the 2px, so a screenshot that does not fit still shrinks to
the column. The room kept for it before it loads is unchanged.
r0ny123 added a commit that referenced this pull request Oct 30, 2026
r0ny123 added a commit to r0ny123/mcritweb that referenced this pull request Oct 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant