Repository navigation
Run Binary Ninja's MCRIT requests off the UI thread - #31
Conversation
Uploading the report, Function Scope and Block Scope queries, and fetching a matching result ran on the UI thread, so a slow or unreachable server froze Binary Ninja for up to the request timeout, and with live queries on that happened on every cursor move. Backend.run_request(work, on_done) runs work, the requests only, and then on_done on the UI thread. Binary Ninja runs work on a plain thread, without the analysis wait an export needs. The default, and so IDA and the headless backend, runs it synchronously as before. McritInterface's upload no longer refreshes the local info widget itself; the upload button's completion does. The activity line was already safe to update from any thread. The widgets read what the request needs (the query report, the score threshold, the block hashes) on the UI thread, and render only when the answer arrives. An answer for a function the cursor has left is cached but not shown. When everything a render needs is cached, as after a filter change, it renders at once without a request. The Binary Ninja GUI test waits for the job request and the result fetch. It checks that the job request and a Function Scope query run off the UI thread, and that a late answer for a function the cursor left is not rendered. That last check fails without the guard. Closes familiary#25
danielplohmann
left a comment
There was a problem hiding this comment.
Thanks, moving the requests off the UI thread is the right direction, and the cached-path shortcut keeps filter changes snappy. The suite passes locally (211 tests). We have no Binary Ninja here, so I checked the threading by reading the code paths and ran what could run in isolation.
Four things to fix before merge, all from the UI thread now being free while a request runs:
- A label can be applied to the wrong function (inline on
FunctionMatchWidget.py). - Block Scope can render a stale answer for an older block-size threshold (inline on
BlockMatchWidget.py). matched_function_entriesis now mutated from a worker while the Function Overview may iterate it (inline onFunctionMatchWidget.py).- A raising request leaves the widget on "Querying…" with no warning (inline on
BinjaBackend.py).
Smaller, fine as a follow-up: Upload and Get Match Result stay enabled while their request runs, so a second click starts a second request (a second ResultChooserDialog, possibly a duplicate matching job). And Function Scope does not reset last_viewed when the remote cache is unavailable, while Block Scope resets last_viewed_function, so only Block Scope retries after the server comes back.
Function Scope's query now runs off the UI thread, and last_viewed moves to the new function as soon as the query starts. Until its answer arrives, the names table still lists the previous function's labels and stays clickable, so a double-click applied an old function's label to the new one. The label now goes to current_function_offset, which is set when the tables are filled.
Block Scope dropped an answer only when it was for another function. Two requests for the same function with different block-size thresholds (4, then 8 while the first was still running) both passed, and whichever answered last was shown, so the table could list the blocks for 4 while the spin box said 8. Each view now takes a number, and an answer is shown only for the latest one. The function check stays, since with live tracking off the cursor moves on without asking for a view.
queryFunctionEntriesById now runs on a worker thread for Function Scope,
and updated matched_function_entries in place while the Function
Overview may iterate the same dict on the UI thread, which raises
"dictionary changed size during iteration". Its None-to-{} check was
also a check-then-set between two workers. It now merges into a copy
under a lock and swaps the copy in, so a loop over the old dict is not
disturbed and no worker's entries are lost.
When a request raised, run_request only logged it and never called on_done, so the widget stayed on "Querying matches for function 0x..." with nothing on screen saying why. It now shows a warning like run_background does.
When the family and sample lists could not be fetched, Block Scope cleared last_viewed_function so the next cursor move in the function tried again, but Function Scope kept last_viewed and skipped that function until the cursor left it. It now clears last_viewed too.
Both now run off the UI thread and their actions stayed enabled, so a second click started a second request: another upload, or a second result chooser dialog after the first, each able to request a new matching job. Upload is disabled until the upload answers, and Get Match Result until the chosen result is shown or the job query fails.
|
Thanks for checking the threading so carefully. All four points are fixed, plus both follow-ups; details are in the inline replies.
New tests cover the Block Scope and cache fixes, and each fails without its fix. The suite passes (213 tests), and the Binary Ninja 6 GUI integration test passes against a local MCRIT 1.12.0 (173 checks, 0 failures). The later PRs in the stack (#33–#41) have these commits merged in. |
danielplohmann
left a comment
There was a problem hiding this comment.
Thanks, all four fixes and both follow-ups hold up. I checked each commit against its inline thread. updateActivityInfo goes through a signal, so the request methods can call it from the worker. Lint, the sync checks and the suite (213 tests) pass here, and merged with #30 the suite passes too (223 tests).
Closes #25.
On Binary Ninja, uploading the report, Function Scope and Block Scope queries, and fetching a matching result ran on the UI thread. A slow or unreachable MCRIT server froze Binary Ninja for up to the request timeout, and with live queries on, that happened on every cursor move.
How
Backend.run_request(work, on_done)runswork, the requests only, and thenon_doneon the UI thread.workon a plain thread, without the analysis wait thatrun_backgroundhas for exports.McritInterface.uploadReportno longer refreshes the local info widget. The upload button's completion does that on the UI thread. The activity line was already signal-based and safe from any thread.Checks
Local, against MCRIT 1.12.0 and smda 4.9.0:
-A(145 checks), the IDALib test and the headless test pass unchanged.