Repository navigation
Fetch labels off the UI thread and render the Function Overview once - #33
Conversation
"Fetch labels for matches" asked the server for every matched function on the UI thread. On a result with 221,398 matches that is one request for 151,306 ids, which took 58 seconds against mcrit.malpedia.io, and Binary Ninja froze for all of it. It now goes through Backend.run_request like the other requests, with the button disabled until the answer arrives. IDA runs it synchronously as before. queryFunctionEntriesById swaps a merged copy into matched_function_entries instead of updating the dict in place, since requests now write it from worker threads while the overview iterates it on the UI thread. Rendering that result in the Function Overview took 10 seconds, 8 of them in resizeRowToContents called once per row. The rows are sized once from the first row, which gives the same height. A new result also rendered the table three times, because setting the score range emitted valueChanged, and a filter click rendered it twice, because the radio button losing its check emits toggled too. Both now render once. Per filter click on that result: 8.9 s to 0.95 s with the labels filter, 22.2 s to 1.41 s with no filter.
danielplohmann
left a comment
There was a problem hiding this comment.
Thanks, this reads well and the numbers make the case. Merged with current main (now that #31 is in), lint, the sync checks and the suite (216 tests) pass here. The three new overview tests fail against main's widget and pass with this PR.
I checked the parts that could go wrong:
blockSignalsaround the spin box hides nothing:handleSpinThresholdChangeis its only listener, andpopulateFunctionTablereads the value right afterensureSpinBoxRange.queryFunctionEntriesByIdcatches its own errors and returnsNone, sodonealways runs and the button comes back.- A second fetch started while one runs (e.g. auto-fetch after loading another result) sends an overlapping request, and the first answer re-enables the button early. That's harmless here, and the generation counter in #37 covers it, so no change is needed in this PR.
About the IDA workflow: ida-tests.yml needs the licence secrets, which a dispatch can only use on a branch in this repository, so I'll run it from a branch here or on main after the merge.
The merge-base changed after approval.
danielplohmann
left a comment
There was a problem hiding this comment.
Re-approving: the earlier approval was dismissed because merging #31 moved the merge-base. The review above stands.
The merge-base changed after approval.
Keeps all four Unreleased/Changed bullets: main's off-UI-thread requests and client alignment first, then this branch's label fetch and overview rendering. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
danielplohmann
left a comment
There was a problem hiding this comment.
Merged main into the branch (d90e925); the conflict was only CHANGELOG.md, and all four Unreleased/Changed bullets are kept. Lint, the sync checks and the suite (226 tests) pass on the merged branch. The review above stands.
This builds on #31, which is its first commit; only 9b24453 is new. #35 builds on this PR, and #34 is independent of it.
Two parts of the Function Overview stall the UI on a large matching result. Sample
e0b6f853…(2,329 functions) has a result on mcrit.malpedia.io with 221,398 matches that point at 151,306 remote functions.The first is "Fetch labels for matches", which #31 did not cover. It requested all of those ids on the UI thread in one request that took 58 seconds, and Binary Ninja froze for all of it. It now goes through
Backend.run_requestlike the other requests, and its button stays disabled until the answer arrives. IDA and headless run it synchronously as before. Because requests now writematched_function_entriesfrom worker threads while the overview iterates it on the UI thread,queryFunctionEntriesByIdno longer updates that dict in place. It builds a merged copy under a lock and swaps it in.The second is rendering. Drawing the result took 10 seconds, 8 of them in
resizeRowToContents, which ran once per row. The overview now sizes the first row and uses that height for every row, which gives the same 19 px rows. A new result was also rendered three times:ensureSpinBoxRangeset the score range and value, and each emittedvalueChanged. It now blocks those signals, since its caller renders with the new value right after. A filter click was rendered twice, because the radio button losing its check also emitstoggled. Only the newly checked button renders now.These times are per filter click, with the overview offscreen on the real result (13,880 labelled remote functions), median of three:
The next table comes from a read-only Binary Ninja 6.0 GUI session against mcrit.malpedia.io with the same sample. That session loaded the existing matching job and uploaded or created nothing. "Before" is #31, and "after" is this PR merged with #34, #35 and the SMDA change in danielplohmann/smda#378. The rows this PR accounts for:
The label fetch still takes a minute on the server, but Binary Ninja stays usable during it. The remaining 0.92 s stall is the overview rendering the labels.
tests/ui_qt/test_function_overview.pyruns offscreen and checks three things: a new result renders the table once, a filter click renders it once, and the label fetch goes throughrun_requestwith the button disabled until the answer arrives. All three fail against #31's widget. The Binary Ninja GUI test now waits for the label fetch and checks that it ran off the UI thread with the button disabled, passing 176 checks.pytestgives 214 passed, and the settings and metadata sync checks andrun_quality_checks.pypass. The IDA GUI and IDALib tests were not run locally, because no licensed IDA was available. The rendering changes are shared with IDA, so please trigger the IDA workflow on this branch.