Skip to content

Extract the result-page match tables into one set of macros, and sort them over the whole list (#50) - #152

Open
r0ny123 wants to merge 3 commits into
familiary:masterfrom
r0ny123:fix/50-deduplicate-result-tables
Open

r0ny123 wants to merge 3 commits into
familiary:masterfrom
r0ny123:fix/50-deduplicate-result-tables

Conversation

@r0ny123

@r0ny123 r0ny123 commented Aug 30, 2026 •

Copy link
Copy Markdown
Collaborator

Update (2026-10-25), after the batch E review. Beyond what the old title said, this adds server-side column sorting to every paginated match table on the result pages: a header click sorts the whole list, then slices the page (see "Sorting" below). It lands after #275, which is on master now: the sortable headers put an onclick on every result page, which is only safe with #275's escaping. It conflicts with #253 and #302 (and #160, if that goes first) in the markup it moves into table/match_row.html, not mechanically, so it is rebuilt on master once those have landed, as the last review asked.

Closes #50. The result pages each carried their own copy of every match table: nine header/body pairs in four templates, 244 column_type == branches between them. They now share six macros in table/match_row.html, and the paginated tables sort by clicking a header, over the whole list rather than the page on screen.

This is the branch rebuilt on master after batch C (6564602), as asked in the last review. The macros were written again from master's markup, so they carry everything master changed in those tables: #194's function_rows, #148's rounded score cells and divisor tooltips, #244's aria labels, and the closed icon tags. What #122, #142 and #145 changed (job names in the headings, matching_statistics, match_diagram) sits outside the tables and stays in the pages as master has it. The old history (up to 7db5fe5) no longer applied and was replaced by three commits.

Change.

  1. Extraction (5ac8494). table/match_row.html holds three header/row pairs:

    macros table callers
    famlib_header / famlib_row family, library and sample matches all (x2), family, sample, vs
    aggregated_function_header / aggregated_function_row aggregated functions all, family
    matched_function_header / matched_function_row function to function sample, vs
    • The macros emit cells only; the <tr> stays at the call site, because its colour is per table and because the library table of result_compare_all.html opens its row with <tr ...></tr>. That quirk is kept, so the pages render as before.
    • A macro does not see the caller's context, so matching_result, scp, funp and Round the score columns instead of truncating them (part of #7) #148's divisors are parameters (famlib_row, mcritweb/templates/table/match_row.html:87).
    • The 1-vs-1 page has no is_unique_match cell: against one other sample a function's uniqueness across the corpus is not known. It is unique_match_known=False on its calls (match_row.html:215, :241).
    • result_compare_function.html keeps its own table; its family B and sample B columns are in none of the others.
    • column_type == branches: 244 on master, 92 here (70 in the macro file, 22 left in result_compare_function.html).
  2. Tests (cd18e16), in a new tests/testResultTables.py rather than in testResultPages.py, which other PRs also change.

  3. Sorting (64631b0).

    • sorted_page() (mcritweb/views/result_sorting.py:90) sorts the whole list a table is drawn from and then slices the page. Without a sort, or with a column it has no key for, it builds the page the way master does. The aggregated function table therefore keeps Result views rebuild function aggregations and sample-match passes repeatedly #194's page-only aggregation, and only a sorted page aggregates the whole report (aggregated_function_rows, mcritweb/views/data.py:1141). No template slices any more; the view hands each table its rows.
    • Pagination answers sort_by, is_ascending and get_sort_link as CursorPagination does (mcritweb/views/pagination.py:73, :141), so sortable_header_col works unchanged. The parameters come from the table's page parameter (famp -> fampsort, fampasc), so the three tables of the overview sort independently. A sort link goes back to page 1 and keeps the other parameters.
    • sortable_header_col gains a colspan (mcritweb/templates/table/pagination_widget.html:196) for the Direct and Frequency headers, which cover a score and its library-excluded twin. The twins have no header of their own and do not sort.
    • The two tables with nothing to page through, the one sample of the sample-filtered page and the 1-vs-1 similarity scores, keep plain headers (header_cell, match_row.html:45).
    • The family and sample header icons now close their <i>, on the function page too.

From the review:

  • The aggregated is_library_match / is_unique_match cells use master's closed form (match_row.html:181, :183), and a test now fails on an <i> left open in any match cell or header.
  • Function rows are sorted over the full aggregate and then sliced, and an unknown funpsort falls back to aggregate_function_matches_page. sorted_page never sees a sliced page: it takes the full list and the unsorted page as two callables.
  • famlib_row takes divisors.

Verification. On the captured reports (tests/fixtures/, mcrit 1.13.0 data), every result page variant was rendered with master (6564602) and with each commit: every report, page two, eight filter shapes, and every ?famid=, ?samid= and ?funid= the overview links to, with page two and a score filter on each. 960 pages, all answering 200. Whitespace and the session's CSRF token normalised:

master vs this
after the extraction (5ac8494) 960 of 960 pages identical
after the sorting (64631b0), outside <thead> 960 of 960 identical
after the sorting, header cells (text, icons, colspan, alignment) 960 of 960 identical

What the sorting changes in the headers is the sort link and icon on each sortable column, and the closing </i> of the family and sample icons. The comparison was checked against a change it must see: dropping unique_match_known=False from the 1-vs-1 page made all 9 of its renders differ.

On the testbed (mcrit 1.13.0), master (6564602) and this branch as side instances against the live backend, GET only: 41 pages of five finished jobs (two 1-vs-N, two 1-vs-1, one SMDA query), each with page two, a page of 25, and two of each ?famid=, ?samid= and ?funid= it links to. All 41 answered 200 on both, none rendered as corrupted, and the same comparison as above found them identical outside <thead> and in every header cell. The aggregated table of 6abcb034595a8fbde6630826 (242 functions), sorted by size descending and read over three pages of 100, is the master list in descending size order. A sorted page of 100 took 0.16 s against 0.10 s unsorted (median of 3). check_corpus.py: as in the snapshot.

Tests. tests/testResultTables.py, on the captured corpus with every matched function resolvable (as testResultPages.py widens it):

  • every match table on the overview and the famid, samid and funid views has as many body cells as its header has columns, times its rows;
  • no match cell or header leaves an <i> open;
  • the 1-vs-1 page renders the stored sample columns minus Uniq, compared with the sample-filtered page that gets the same list;
  • the score tooltips divide by the view's divisors, with sentinel values in place of recover_score_divisors();
  • a table sorted by size, read over four pages of 250, holds every row once and in order; a descending second page of 10 is the 11th to 20th largest;
  • an unknown sort column leaves the page as it is;
  • an unsorted page aggregates only its own functions, a sorted one the whole report;
  • sorting the families leaves the function table's order alone;
  • a sort link goes to page 1 and keeps the page size, the filters and the anchor.

Five pass on master too and guard the refactor: the first four, and the unknown sort column, which master ignores as well. Each test was checked by taking its guard out: the flag dropped from both vs calls or the header only, the open-<i> form in a cell or a header, binweight in place of the divisors, sorting the sliced page, no fallback, the direction ignored, the page kept in the sort link, and one set of sort parameters for all tables. Each fails at least one test.

Full suite with Chromium: 2090 passed, 11 skipped. testBrowserScoreTooltips.py, which drives the result pages' score tooltips, and testBrowser.py are among the skips there, because they launch Playwright's own Chromium rather than MCRITWEB_CHROMIUM; run with /opt/pw-browsers/chromium as the executable, their 5 tests pass. ruff check . clean.

Relation to open PRs. A trial merge (git merge-tree) with each open PR that touches the same files:

@danielplohmann

Copy link
Copy Markdown
Collaborator

Held out of the batch C integration (#298, #299), and here is why. This PR replaces the inline match tables of result_compare_all/family/sample/vs.html with macros in table/match_row.html, and those inline tables have moved since it branched:

Merging means re-applying all of that inside the macros, so it's a rebase onto master once batch C lands. Three things for the rebase:

  1. match_row.html:180,182 (the aggregated is_library_match / is_unique_match cells) bring back the class="fa-solid fa-times-circle"> {% endif %}</i> form that master fixed. When the value is true, the <i> is never closed.
  2. For server-side sorting of function rows, either sort the full aggregate and slice it, or fall back to aggregate_function_matches_page when funp.sort_by isn't a known key. Don't feed sorted_page a page that is already sliced. Sort then slice is correct today, and the count stays right.
  3. The macros need a divisors parameter for Round the score columns instead of truncating them (part of #7) #148's tooltips.

The sort/pagination interplay itself looked right: the reset to page 1, the sort params carried through original_args, and a stable sort that keeps the score order.

🤖 Generated with Claude Code

https://claude.ai/code/session_01UCjRSPdSunUm9rFRrQqGsG

Issue #50, the deduplication half. The four result pages each carried their
own copy of every column_type dispatch chain: nine header/body pairs in
result_compare_all, _family, _sample and _vs, differing in indentation and,
on the 1-vs-1 page, in one column. They now call six macros in the new
table/match_row.html.

The macros emit cells only. The <tr> stays at the call site, because its
background colour is per table and because the library table on
result_compare_all.html opens its row differently from the other copies.
That quirk is kept, not fixed, so the pages render as before.

A macro does not see the caller's context, so matching_result, the score
colour provider, the function pagination and the score divisors of familiary#148 are
parameters now. url_for, request and join_hint_strings are globals.

result_compare_vs.html had no branch for is_unique_match: against a single
other sample a function's uniqueness across the corpus is not known (see the
note in data.result()). That stays, as unique_match_known=False on the vs
calls, so the reason is visible at the call site.

result_compare_function.html keeps its own table. Its columns (family B,
sample B) are in none of the others.

Rendered on the captured reports, 960 pages (every report, page two, eight
filter shapes and every famid, samid and funid the pages link to): the same
HTML as master once whitespace and the session's CSRF token are normalised.
Jinja renders a macro called with the wrong arity as nothing, so a slip in a
match_row.html call leaves a page that answers 200 with a table that lost or
gained a column. tests/testResultTables.py looks at the tables themselves, on
the overview and the famid, samid and funid views of the captured reports:

- every match table has as many body cells as its header has columns (by
  colspan) times its rows;
- no body cell leaves an <i> open, the shape the aggregated table's library
  and unique cells had before master fixed them;
- the 1-vs-1 page renders the stored sample columns minus Uniq, compared
  against the sample-filtered page that gets the same list. Dropping
  unique_match_known=False from both vs calls keeps the table rectangular, so
  only this sees it;
- the score tooltips divide by the divisors the view computes, checked with
  sentinel values in place of recover_score_divisors().

All four pass on master as well; they guard the refactor. Each fails when its
guard is taken out: the flag dropped from both vs calls or from the header
only, the old open-<i> form, binweight in place of the divisors.
The second box of issue #50. Every paginated match table on the result pages
now sorts by clicking its header: the family, library and function tables of
the overview, the sample and function tables of a family, and the function
tables of the sample-filtered, 1-vs-1 and function pages.

The sort runs in the view. These tables are server-paginated, so a sort in
the browser would order one page and leave the list as it was. sorted_page()
in views/result_sorting.py sorts the whole list a table is drawn from and then
slices the page, so page two starts where page one ended. Without a sort, or
with a sort column it has no key for, it builds the page as before: the
aggregated function table keeps the page-only aggregation of familiary#194, and only a
sorted page aggregates the whole report. The view hands each template its
rows; the templates no longer slice.

Pagination answers sort_by, is_ascending and get_sort_link the way
CursorPagination does, so sortable_header_col works unchanged. The parameters
come from each table's page parameter (famp -> fampsort, fampasc), which lets
the three tables of result_compare_all.html sort independently. A sort link
goes back to page 1 and keeps the other parameters.

sortable_header_col gains a colspan for the Direct and Frequency headers,
which cover a score and its library-excluded twin. The twins have no header of
their own and so do not sort; splitting the header would change what the page
shows. The two tables with nothing to page through, the sample-filtered
page's one sample and the 1-vs-1 similarity scores, keep plain headers.

The family and sample headers opened their icon's <i> and never closed it;
they close it now, on every result page including the function page.

Rendered on the captured reports, the same 960 pages as for the extraction:
everything outside <thead> is the same HTML as master, and every header cell
has the same text, icons, span and alignment. What differs is the sort link
and icon on each sortable header.
@r0ny123
r0ny123 force-pushed the fix/50-deduplicate-result-tables branch from 7db5fe5 to 64631b0 Compare October 20, 2026 00:57
@r0ny123

r0ny123 commented Oct 20, 2026 •

Copy link
Copy Markdown
Collaborator Author

Thanks. Rebuilt on master after batch C (6564602) rather than rebased: the old history no longer applied, so the macros were written again from master's markup, as three commits (5ac8494, cd18e16, 64631b0). They carry #194's function_rows, #148's rounded cells and divisor tooltips, and #244's aria labels; #122, #142 and #145 sit outside the tables and stay as master has them. On the captured reports, 960 page variants render the same HTML as master after the extraction, and after the sorting everything outside <thead> is the same and every header cell has the same text, icons, span and alignment. The description has the details and a live check.

  1. Open <i>. The aggregated library and unique cells use master's closed form (table/match_row.html:181, :183). testResultTables.py now fails on an <i> left open in any match cell or header; the family and sample header icons had that too, and close now.
  2. Sorting. sorted_page() (views/result_sorting.py:90) takes the full list and the unsorted page as two callables. With a known sort key it sorts the full aggregate and slices it; without one, or with an unknown funpsort, it calls aggregate_function_matches_page as master does (aggregated_function_rows, views/data.py:1141). So it never sees a sliced page, and an unsorted page still aggregates only its own functions; a test counts the rows getAggregatedFunctionMatches builds in both cases.
  3. Divisors. famlib_row takes divisors (match_row.html:87). A test replaces recover_score_divisors with sentinel values and checks every score tooltip divides by them.

#253 and #160 now conflict with this in the moved markup (header icons, thead classes, status colours); whichever merges later applies its change inside match_row.html.

The branch is force-pushed over 7db5fe5.

@danielplohmann

Copy link
Copy Markdown
Collaborator

The three earlier points are fixed, thanks. It now conflicts non-mechanically with #253 and #302 (and #160 if that
goes first), so please rebuild it once those land. It must land after #275, which is now on master: it puts the
sortable-header onclick on every result page, which is only safe with #275's escaping. Also worth a line in the
description: it adds server-side column sorting to every result table, beyond what the title says.

@r0ny123 r0ny123 changed the title Extract the result-page match tables into one set of macros (part of #50) Extract the result-page match tables into one set of macros, and sort them over the whole list (#50) Oct 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

table widgets for tables within result pages

2 participants