Skip to content

Keep the Function Overview's rows in a table model - #38

Merged
danielplohmann merged 10 commits into
familiary:mainfrom
r0ny123:binja-overview-model
Oct 15, 2026
Merged

danielplohmann merged 10 commits into
familiary:mainfrom
r0ny123:binja-overview-model

Conversation

@r0ny123

@r0ny123 r0ny123 commented Oct 1, 2026 •

Copy link
Copy Markdown
Collaborator

This builds on #37, which builds on #35, #33 and #31; the diff shows their commits until they merge. Its own commits are 63273af, 44f7d19 and 30dd580, rebased unchanged onto the rebased #37.

flowchart LR
  main --> 31["#31"] --> 33["#33"] --> 35["#35"] --> 37["#37 chunked label fetch"] --> this["this PR: table model"]
Loading

The Function Overview was a QTableWidget with an item per cell and, once labels exist, a persistent QComboBox editor in every row. Every render rebuilt all of it, about 705 combo boxes for the 1,109 rows of matching job 11353. This PR changes three things:

  • The overview becomes a QTableView over a QAbstractTableModel (OverviewTable.py). The label column is painted by a delegate that draws a combo box, and a real QComboBox exists only for the cell being edited.
  • Matches are grouped by local function once per result.
  • The aggregates for a threshold and filter are cached until the result or the labels change.

Selection tracking, "(de)select all", the label import, right-click resolve, double-click navigation, sorting and the criticality colours keep their behaviour. function_name_mapping is gone; the rows of the model replace it.

The table below measures matching job 11353 of sample e0b6f853… (221,398 matches, 13,880 labelled remote functions) in Binary Ninja 6.0 against mcrit.malpedia.io. Times include painting, measured by a synchronous repaint of the viewport, with the median of three:

action #37 this PR
render the current result again 0.88–0.93 s 0.03 s
filter none → labels 0.79–0.82 s 0.03–0.04 s
filter labels → none 0.93–0.94 s 0.03–0.04 s
filter none → conflicted 0.59–0.61 s 0.03 s
longest UI stall while labels arrive 1.1–5.6 s 0.12–0.18 s
longest UI stall while the result loads and renders 0.52 s 0.27 s

Offscreen, without painting, a new result took 0.86–0.97 s before and 0.29–0.32 s after. A click on "no filter" took 0.61–0.71 s before and 0.11–0.15 s after, and a switch from none to labels took 1.24–1.42 s before and 0.10–0.12 s after.

A review found two regressions, fixed in 99d33b6. First, the delegate tinted label cells through the palette's button brush, which macOS's native style ignores, so cells there lost their criticality colour. Tinted cells are now drawn with a Fusion style. Second, without any labels the label cells showed the opt-out entry -|- instead of -.

tests/ui_qt/test_overview_table.py covers the model and delegate:

  • offsets and counts sort by value;
  • a sorted row still finds the row it was filled into;
  • picking a label changes only that row;
  • without labels the column is plain;
  • one click opens one combo box and a pick commits it;
  • thousands of rows fill without a widget per row;
  • label cells are painted in their criticality colour. This test now scales its probe by the device pixel ratio, and fails under QT_QPA_PLATFORM=cocoa without the fix.

tests/ui_qt/test_function_overview.py checks that the grouped aggregation matches the old per-match loop, that new labels bypass the cache, and that a result without labels shows -. pytest gives 227 passed, tests/ui_qt passes under both the offscreen and cocoa platforms, and the Binary Ninja GUI test passes all 176 checks against a local MCRIT 1.12.0. The IDA GUI test was updated for the removed function_name_mapping but not run, because no licensed IDA was available.

The overview created a combo box for every row and kept it open, so
filling or filtering a result cost a widget per row: on a synthetic
result of 20,000 functions a filter click took 120 s, and on 705
functions 1.2 s.

The table is now a QTableView over a model that holds the rows and sorts
them by value. The label column is painted like a combo box by a
delegate, and a real one exists only for the cell being edited, opened
by a click. Right click and double click work from the model's rows.

Filter click: 120.6 s to 0.95 s at 20,000 functions, 1.17 s to 0.25 s at
705.
With a result of 221,370 matches over 705 functions the overview took
0.7 to 0.9 s per click even with the table in a model: nearly all of it
was the pass over every match that builds each function's families,
samples, functions and labels, run again on each filter click, and the
per-match search of a list for the matched function ids.

The matches are now grouped by local function when a result arrives, the
aggregates are kept per threshold and filter until the result or the
fetched labels change, labels are sorted once per aggregate, and the
drop-down entries are built only for the row being edited.

On that result, in Qt offscreen mode: new result 2.9 s before the model,
0.8 s now; a filter click 3.4 to 3.8 s before, 0.5 s the first time and
0.1 to 0.15 s once aggregated.
The label delegate tinted a cell through the palette's Button brush,
which macOS's native style ignores, so under that style every label
cell was drawn grey instead of in its criticality colour. Tinted cells
are now drawn with a Fusion style. The painting test sampled logical
coordinates on a grabbed image, which on a Retina screen is twice as
large; it now scales by the device pixel ratio, and fails without the
fix under QT_QPA_PLATFORM=cocoa.

Without labels for any match the label column has no drop-downs, and
its cells showed "-|-", the opt-out entry. They show "-" again, which
the label import passes over without counting the row as skipped.
@r0ny123
r0ny123 force-pushed the binja-overview-model branch from 99d33b6 to 30dd580 Compare October 1, 2026 19:46

@danielplohmann danielplohmann left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, this is a big improvement. I checked the model and delegate against what the old QTableWidget did per cell: label choices and picks, criticality colours, sorting, source rows after a sort, right-click, double-click, filters and the threshold. Offscreen, with a fake session, it behaves as before. Sorting now runs in Python, so nothing can raise into Qt's sort. Lint, the sync checks and the suite (249 tests) pass on the tree merged with main. One thing before merge.

_aggregates grows without limit. It is keyed by (threshold_value, filtered) (FunctionOverviewWidget.py:452, :485) and cleared only when a new result is grouped (:438). Every threshold the user steps through keeps a full set of per-function aggregates. On a synthetic 221,000-match result, stepping the spin box from 50 to 100 kept about 780 MB, inside IDA or Binary Ninja. Could it keep only the current threshold? Clicking through the filters, which is what the cache is for, still hits it.

Smaller points, not blocking:

  • editorEvent passes a plain QModelIndex to the 0 ms timer (OverviewTable.py:278). If the model resets first, createEditor raises IndexError (reproduced offscreen) or opens on another row. A QPersistentModelIndex with a validity check fixes it.
  • The label column sorts by its text (OverviewTable.py:139), so "100|…" sorts among the 10s. Sorting by (score, label) would match the docstring.
  • The drop-down cannot be opened from the keyboard. EditKeyPressed (F2) for the label column would help.
  • The second line of the changelog entry is over 100 characters.

Merging main in conflicts in CHANGELOG.md (keep all bullets, main's first) and tests/binja/gui_integration.py. The second is only adjacency: keep your select_cell and main's double_click_graph, which already goes through double_click. I could not run Binary Ninja or IDA here.

# Conflicts:
#	CHANGELOG.md
#	tests/binja/gui_integration.py
_aggregates was keyed by threshold and filter and cleared only when a
new result was grouped, so every threshold the spin box stepped through
kept a full set of per-function aggregates. On a synthetic 221,000-match
result, stepping from 50 to 100 kept about 780 MB. A new threshold now
drops the others; clicking through the filters, which the cache is for,
still hits it.
- A click opened its drop-down from a 0 ms timer with a plain
  QModelIndex. If the model was reset or sorted first, createEditor
  raised IndexError or opened on another row. The timer now holds a
  QPersistentModelIndex and edits only while it is valid.
- The label column sorted by its text, so "100|..." sorted among the
  10s. It now sorts by score, then label, with the opt-out first.
- A drop-down can be opened from the keyboard with the edit key: F2,
  or Return on macOS. Only the label cells are editable, so the key
  does nothing elsewhere.
- The changelog entry is rewrapped and says what is cached per
  threshold.
@r0ny123

r0ny123 commented Oct 12, 2026

Copy link
Copy Markdown
Collaborator Author

Thanks. I merged main in, keeping all the changelog bullets with main's first, plus both select_cell and double_click_graph in the GUI test. Two commits address the review:

  • fbf1b51: _aggregates keeps only the current threshold. A new threshold drops the others, so stepping the spin box keeps at most the two filter variants of one threshold, and clicking through the filters still hits the cache. A test steps from 50 to 100 and checks what is left; it fails without the change.
  • 283af9f, the smaller points:
    • The click's timer holds a QPersistentModelIndex and edits only while it is valid. A test resets the model before the timer fires and checks that nothing is edited.
    • The label column sorts by (score, label), with the opt-out first.
    • The edit key opens the drop-down. Qt's edit key is F2, but Return on macOS, so the test sends the platform's key. Only label cells are editable.
    • The changelog entry is rewrapped and says the aggregates are kept for the current threshold.

The three new delegate and sort tests fail on the old code. Lint, the sync checks and the suite (253 tests) pass.

@danielplohmann danielplohmann left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, both commits address the review. fbf1b51 keeps only the current threshold's aggregates, and 283af9f covers the persistent index, the score-then-label sort, the edit key and the changelog wrap. The new tests fail on the old code. Lint, the sync checks and the suite (253 tests) pass, and the branch is current with main. Merged together with #36 there are no conflicts and 262 tests pass.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants