Skip to content

Fetch labels off the UI thread and render the Function Overview once - #33

Merged
danielplohmann merged 3 commits into
familiary:mainfrom
r0ny123:binja-label-fetch
Oct 12, 2026
Merged

danielplohmann merged 3 commits into
familiary:mainfrom
r0ny123:binja-label-fetch

Conversation

@r0ny123

@r0ny123 r0ny123 commented Sep 30, 2026 •

Copy link
Copy Markdown
Collaborator

This builds on #31, which is its first commit; only 9b24453 is new. #35 builds on this PR, and #34 is independent of it.

flowchart LR
  main --> 31["#31 requests off the UI thread"]
  31 --> 33["#33 label fetch, overview rendering"]
  33 --> 35["#35 status bar tasks, per-file log"]
  31 --> 34["#34 connection reuse, parallel block lookups"]
Loading

Two parts of the Function Overview stall the UI on a large matching result. Sample e0b6f853… (2,329 functions) has a result on mcrit.malpedia.io with 221,398 matches that point at 151,306 remote functions.

The first is "Fetch labels for matches", which #31 did not cover. It requested all of those ids on the UI thread in one request that took 58 seconds, and Binary Ninja froze for all of it. It now goes through Backend.run_request like the other requests, and its button stays disabled until the answer arrives. IDA and headless run it synchronously as before. Because requests now write matched_function_entries from worker threads while the overview iterates it on the UI thread, queryFunctionEntriesById no longer updates that dict in place. It builds a merged copy under a lock and swaps it in.

The second is rendering. Drawing the result took 10 seconds, 8 of them in resizeRowToContents, which ran once per row. The overview now sizes the first row and uses that height for every row, which gives the same 19 px rows. A new result was also rendered three times: ensureSpinBoxRange set the score range and value, and each emitted valueChanged. It now blocks those signals, since its caller renders with the new value right after. A filter click was rendered twice, because the radio button losing its check also emits toggled. Only the newly checked button renders now.

These times are per filter click, with the overview offscreen on the real result (13,880 labelled remote functions), median of three:

filter rows before after
labels 705 8.93 s 0.95 s
none 1,109 22.24 s 1.41 s

The next table comes from a read-only Binary Ninja 6.0 GUI session against mcrit.malpedia.io with the same sample. That session loaded the existing matching job and uploaded or created nothing. "Before" is #31, and "after" is this PR merged with #34, #35 and the SMDA change in danielplohmann/smda#378. The rows this PR accounts for:

step before: duration / longest UI stall after: duration / longest UI stall
fetch the job list and result, render the overview 14.72 s / 9.38 s 4.14 s / 0.53 s
filter click (labels, then none) 18.41 s / 18.41 s 0.56 s / 0.56 s
fetch labels for matches 67.97 s / 67.97 s 59.96 s / 0.92 s

The label fetch still takes a minute on the server, but Binary Ninja stays usable during it. The remaining 0.92 s stall is the overview rendering the labels.

tests/ui_qt/test_function_overview.py runs offscreen and checks three things: a new result renders the table once, a filter click renders it once, and the label fetch goes through run_request with the button disabled until the answer arrives. All three fail against #31's widget. The Binary Ninja GUI test now waits for the label fetch and checks that it ran off the UI thread with the button disabled, passing 176 checks. pytest gives 214 passed, and the settings and metadata sync checks and run_quality_checks.py pass. The IDA GUI and IDALib tests were not run locally, because no licensed IDA was available. The rendering changes are shared with IDA, so please trigger the IDA workflow on this branch.

"Fetch labels for matches" asked the server for every matched function
on the UI thread. On a result with 221,398 matches that is one request
for 151,306 ids, which took 58 seconds against mcrit.malpedia.io, and
Binary Ninja froze for all of it. It now goes through
Backend.run_request like the other requests, with the button disabled
until the answer arrives. IDA runs it synchronously as before.

queryFunctionEntriesById swaps a merged copy into
matched_function_entries instead of updating the dict in place, since
requests now write it from worker threads while the overview iterates
it on the UI thread.

Rendering that result in the Function Overview took 10 seconds, 8 of
them in resizeRowToContents called once per row. The rows are sized
once from the first row, which gives the same height. A new result also
rendered the table three times, because setting the score range emitted
valueChanged, and a filter click rendered it twice, because the radio
button losing its check emits toggled too. Both now render once.

Per filter click on that result: 8.9 s to 0.95 s with the labels
filter, 22.2 s to 1.41 s with no filter.
danielplohmann
danielplohmann previously approved these changes Oct 12, 2026

@danielplohmann danielplohmann left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, this reads well and the numbers make the case. Merged with current main (now that #31 is in), lint, the sync checks and the suite (216 tests) pass here. The three new overview tests fail against main's widget and pass with this PR.

I checked the parts that could go wrong:

  • blockSignals around the spin box hides nothing: handleSpinThresholdChange is its only listener, and populateFunctionTable reads the value right after ensureSpinBoxRange.
  • queryFunctionEntriesById catches its own errors and returns None, so done always runs and the button comes back.
  • A second fetch started while one runs (e.g. auto-fetch after loading another result) sends an overlapping request, and the first answer re-enables the button early. That's harmless here, and the generation counter in #37 covers it, so no change is needed in this PR.

About the IDA workflow: ida-tests.yml needs the licence secrets, which a dispatch can only use on a branch in this repository, so I'll run it from a branch here or on main after the merge.

@r0ny123
r0ny123 dismissed danielplohmann’s stale review October 12, 2026 07:25

The merge-base changed after approval.

danielplohmann
danielplohmann previously approved these changes Oct 12, 2026

@danielplohmann danielplohmann left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-approving: the earlier approval was dismissed because merging #31 moved the merge-base. The review above stands.

@r0ny123
r0ny123 dismissed danielplohmann’s stale review October 12, 2026 07:30

The merge-base changed after approval.

Keeps all four Unreleased/Changed bullets: main's off-UI-thread requests
and client alignment first, then this branch's label fetch and overview
rendering.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@danielplohmann danielplohmann left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Merged main into the branch (d90e925); the conflict was only CHANGELOG.md, and all four Unreleased/Changed bullets are kept. Lint, the sync checks and the suite (226 tests) pass on the merged branch. The review above stands.

@danielplohmann
danielplohmann merged commit 65382db into familiary:main Oct 12, 2026
10 checks passed
@r0ny123
r0ny123 deleted the binja-label-fetch branch October 15, 2026 19:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants