Repository navigation
Keep the Function Overview's rows in a table model - #38
Conversation
The overview created a combo box for every row and kept it open, so filling or filtering a result cost a widget per row: on a synthetic result of 20,000 functions a filter click took 120 s, and on 705 functions 1.2 s. The table is now a QTableView over a model that holds the rows and sorts them by value. The label column is painted like a combo box by a delegate, and a real one exists only for the cell being edited, opened by a click. Right click and double click work from the model's rows. Filter click: 120.6 s to 0.95 s at 20,000 functions, 1.17 s to 0.25 s at 705.
With a result of 221,370 matches over 705 functions the overview took 0.7 to 0.9 s per click even with the table in a model: nearly all of it was the pass over every match that builds each function's families, samples, functions and labels, run again on each filter click, and the per-match search of a list for the matched function ids. The matches are now grouped by local function when a result arrives, the aggregates are kept per threshold and filter until the result or the fetched labels change, labels are sorted once per aggregate, and the drop-down entries are built only for the row being edited. On that result, in Qt offscreen mode: new result 2.9 s before the model, 0.8 s now; a filter click 3.4 to 3.8 s before, 0.5 s the first time and 0.1 to 0.15 s once aggregated.
The label delegate tinted a cell through the palette's Button brush, which macOS's native style ignores, so under that style every label cell was drawn grey instead of in its criticality colour. Tinted cells are now drawn with a Fusion style. The painting test sampled logical coordinates on a grabbed image, which on a Retina screen is twice as large; it now scales by the device pixel ratio, and fails without the fix under QT_QPA_PLATFORM=cocoa. Without labels for any match the label column has no drop-downs, and its cells showed "-|-", the opt-out entry. They show "-" again, which the label import passes over without counting the row as skipped.
99d33b6 to
30dd580
Compare
# Conflicts: # CHANGELOG.md # tests/ui_qt/test_function_overview.py
danielplohmann
left a comment
There was a problem hiding this comment.
Thanks, this is a big improvement. I checked the model and delegate against what the old QTableWidget did per cell: label choices and picks, criticality colours, sorting, source rows after a sort, right-click, double-click, filters and the threshold. Offscreen, with a fake session, it behaves as before. Sorting now runs in Python, so nothing can raise into Qt's sort. Lint, the sync checks and the suite (249 tests) pass on the tree merged with main. One thing before merge.
_aggregates grows without limit. It is keyed by (threshold_value, filtered) (FunctionOverviewWidget.py:452, :485) and cleared only when a new result is grouped (:438). Every threshold the user steps through keeps a full set of per-function aggregates. On a synthetic 221,000-match result, stepping the spin box from 50 to 100 kept about 780 MB, inside IDA or Binary Ninja. Could it keep only the current threshold? Clicking through the filters, which is what the cache is for, still hits it.
Smaller points, not blocking:
editorEventpasses a plainQModelIndexto the 0 ms timer (OverviewTable.py:278). If the model resets first,createEditorraisesIndexError(reproduced offscreen) or opens on another row. AQPersistentModelIndexwith a validity check fixes it.- The label column sorts by its text (
OverviewTable.py:139), so "100|…" sorts among the 10s. Sorting by (score, label) would match the docstring. - The drop-down cannot be opened from the keyboard.
EditKeyPressed(F2) for the label column would help. - The second line of the changelog entry is over 100 characters.
Merging main in conflicts in CHANGELOG.md (keep all bullets, main's first) and tests/binja/gui_integration.py. The second is only adjacency: keep your select_cell and main's double_click_graph, which already goes through double_click. I could not run Binary Ninja or IDA here.
# Conflicts: # CHANGELOG.md # tests/binja/gui_integration.py
_aggregates was keyed by threshold and filter and cleared only when a new result was grouped, so every threshold the spin box stepped through kept a full set of per-function aggregates. On a synthetic 221,000-match result, stepping from 50 to 100 kept about 780 MB. A new threshold now drops the others; clicking through the filters, which the cache is for, still hits it.
- A click opened its drop-down from a 0 ms timer with a plain QModelIndex. If the model was reset or sorted first, createEditor raised IndexError or opened on another row. The timer now holds a QPersistentModelIndex and edits only while it is valid. - The label column sorted by its text, so "100|..." sorted among the 10s. It now sorts by score, then label, with the opt-out first. - A drop-down can be opened from the keyboard with the edit key: F2, or Return on macOS. Only the label cells are editable, so the key does nothing elsewhere. - The changelog entry is rewrapped and says what is cached per threshold.
|
Thanks. I merged
The three new delegate and sort tests fail on the old code. Lint, the sync checks and the suite (253 tests) pass. |
danielplohmann
left a comment
There was a problem hiding this comment.
Thanks, both commits address the review. fbf1b51 keeps only the current threshold's aggregates, and 283af9f covers the persistent index, the score-then-label sort, the edit key and the changelog wrap. The new tests fail on the old code. Lint, the sync checks and the suite (253 tests) pass, and the branch is current with main. Merged together with #36 there are no conflicts and 262 tests pass.
This builds on #37, which builds on #35, #33 and #31; the diff shows their commits until they merge. Its own commits are 63273af, 44f7d19 and 30dd580, rebased unchanged onto the rebased #37.
The Function Overview was a
QTableWidgetwith an item per cell and, once labels exist, a persistentQComboBoxeditor in every row. Every render rebuilt all of it, about 705 combo boxes for the 1,109 rows of matching job 11353. This PR changes three things:QTableViewover aQAbstractTableModel(OverviewTable.py). The label column is painted by a delegate that draws a combo box, and a realQComboBoxexists only for the cell being edited.Selection tracking, "(de)select all", the label import, right-click resolve, double-click navigation, sorting and the criticality colours keep their behaviour.
function_name_mappingis gone; the rows of the model replace it.The table below measures matching job 11353 of sample
e0b6f853…(221,398 matches, 13,880 labelled remote functions) in Binary Ninja 6.0 against mcrit.malpedia.io. Times include painting, measured by a synchronous repaint of the viewport, with the median of three:Offscreen, without painting, a new result took 0.86–0.97 s before and 0.29–0.32 s after. A click on "no filter" took 0.61–0.71 s before and 0.11–0.15 s after, and a switch from none to labels took 1.24–1.42 s before and 0.10–0.12 s after.
A review found two regressions, fixed in 99d33b6. First, the delegate tinted label cells through the palette's button brush, which macOS's native style ignores, so cells there lost their criticality colour. Tinted cells are now drawn with a Fusion style. Second, without any labels the label cells showed the opt-out entry
-|-instead of-.tests/ui_qt/test_overview_table.pycovers the model and delegate:QT_QPA_PLATFORM=cocoawithout the fix.tests/ui_qt/test_function_overview.pychecks that the grouped aggregation matches the old per-match loop, that new labels bypass the cache, and that a result without labels shows-.pytestgives 227 passed,tests/ui_qtpasses under both the offscreen and cocoa platforms, and the Binary Ninja GUI test passes all 176 checks against a local MCRIT 1.12.0. The IDA GUI test was updated for the removedfunction_name_mappingbut not run, because no licensed IDA was available.