Repository navigation
Tint matched blocks in Binary Ninja's own graph and linear views - #36
Conversation
A new "MCRIT Matches" render layer colours the blocks of the local function that MCRIT matched when a function or block match is opened. Backend gets no-op show/clear hooks; a menu action clears the tint. Render layers run when a view regenerates its lines, so after the colouring changes the backend calls refreshCurrentViewContents on every UIContext, as Binary Ninja's follow_reg_render_layer example does. mcrit_plugin.binja imports its modules inside register_plugin, so that importing one of them, as the stubbed unit tests do, does not import the others. On a 3,955-instruction function (1,361 blocks) in Binary Ninja 6.0, the layer was applied to the visible lines immediately after the double click in linear and graph view. Calling Function.reanalyze() instead repainted only after about 1.6 s, or not within 1.5 s, and made the graph view lay the function out again.
The GUI test checks every sidebar action's validity and effect from a table of expected values, which had no entry for "MCRIT\Clear Match Coloring", so the test stopped with a KeyError. It now expects the action to be valid once a report exists, checks that double clicking a function match colours the local function's blocks, and that the action removes that colouring.
80a0f3b to
f580273
Compare
danielplohmann
left a comment
There was a problem hiding this comment.
Thanks, this is a nice addition. A render layer means nothing is written to the user's database or undo history, the tint is set on the main thread, and IDA is unchanged. It merges into main without conflicts. Lint, the sync checks and the suite (244 tests) pass on the merged tree. One thing before merge.
The wrong local block can get the "current" tint. _onTableBlockMatchesDoubleClicked reads self.parent.current_block and current_function at click time (BlockMatchWidget.py:565-566). Both follow the cursor (updateCurrentBlock), but the block tables do not: hook_refresh returns early while the cursor stays in the same function. So if I query block B1, click into block B2 of the same function and double-click a match, B2 gets the current colour instead of B1. With live tracking on, while a block query for another function is still running, the tint comes out empty and the previous one is cleared. Please remember the block and function the tables show (for example in populateBlockMatchTable and next to _last_block_matches) and use those. A small tests/ui_qt test with a fake backend would cover it; the render-layer tests do not reach the widget.
Should be fixed too, though I could not check them in Binary Ninja:
- HLIL and Pseudo-C linear view get no tint. The base
RenderLayer.apply_to_linear_view_objectsends HLIL and language-representation bodies toapply_to_high_level_il_body, notapply_to_block. Either tint those lines byline.address, or narrow the changelog's "linear views". apply_to_blockreplaces every line's highlight. On a function match the whole local function is tinted, so the analyst's own instruction highlights disappear while the layer is on. Could it leave lines that already have a highlight alone?
Smaller points:
### Addedshould come before### Changed, with a blank line above it.show_local_match_coloringrefreshes the views twice:clear_local_match_coloring()already refreshes. Setting the new ranges and refreshing once is enough.color_atscans every range for every rendered block; abisecton the sorted starts keeps large functions cheap._forgetinMcritSidebar.pycould callMatchRenderLayer.clear(session_id), so the ranges do not outlive the session.HIGHLIGHT_ALPHAcould reuseGRAPH_HIGHLIGHT_ALPHA.
The approval will also need main merged in once more, since #40's merge moved the merge-base.
Double clicking a block match read current_block and current_function at click time, but those follow the cursor while the block tables do not: hook_refresh keeps the tables while the cursor stays in the same function. After querying block B1 and clicking into block B2, B2 got the current colour. With live tracking on, while a query for another function was still running, the tint came out empty and replaced the previous one. The widget now remembers the function and block its tables show and tints those.
- Linear view hands HLIL and Pseudo-C function bodies to apply_to_high_level_il_body, not apply_to_block, so they got no tint. The layer now tints those lines by their address. - apply_to_block replaced every line's highlight, so on a function match the analyst's own instruction highlights disappeared while the layer was on. Lines that already have a highlight keep it. - A block's range is found with a binary search over the sorted range starts instead of a scan, which kept large functions slow to render. - Showing a new tint refreshes the views once instead of twice. - Closing a file's MCRIT sidebar drops its tint ranges. - The graph viewer and the layer share one highlight alpha. - The changelog lists Added before Changed and names HLIL and Pseudo-C. The Binary Ninja GUI test now runs the layer over the disassembly, HLIL and Pseudo-C linear views of the tinted function; the HLIL check fails with the previous layer.
|
Thanks. I merged
I checked the linear-view part in Binary Ninja 6. The GUI test now runs the layer over |
danielplohmann
left a comment
There was a problem hiding this comment.
Thanks, both commits address the review. d223900 tints the block and function the tables show, and the new tests fail without it. e6dcd00 tints HLIL and Pseudo-C lines, keeps the analyst's highlights, and covers the smaller points. Lint, the sync checks and the suite (249 tests) pass, and the branch is current with main. Merged together with #38 there are no conflicts and 262 tests pass.
One optional nit: _forget (McritSidebar.py:77) now reads bv.file.session_id before the widget leaves _SIDEBAR_WIDGETS. If that read ever raised on a closing file, the widget would stay in the list. Clearing the ranges after the removal avoids that. It does not block the merge.
Keeps familiary#36's _shown_function and _shown_block next to _graph_fetches, and extends the match graph tests' fakes for the local tint step familiary#36 added to the graph-opening path: show_local_match_coloring on the backend, an "a" side in the matcher's colouring and a local SMDA report for Block Scope.
This builds on #40, which builds on #35, #33 and #31; the diff shows their commits until they merge. Its own commits are cab8b30 and f580273. #40 fetches the remote function off the UI thread, so the local tint is applied in the callback that runs when the remote function arrives, for the function and block that were clicked even if the cursor moved meanwhile.
When a function or block match is opened, the plugin colours the remote function's blocks in a separate graph report, but the local function's own blocks stay plain in Binary Ninja's views. This adds an "MCRIT Matches" render layer that tints the matched local blocks, at half opacity, in Binary Ninja's own graph and linear views, including IL views, where each IL block takes the colour of the disassembly block it was lifted from. The layer is on by default and appears in the view's render layer menu. A new action, "MCRIT\Clear Match Coloring", removes the tint. The backend gains two no-op hooks,
show_local_match_coloringandclear_local_match_coloring, so IDA's behaviour does not change.Render layers only run when a view regenerates its lines, so after the colouring changes the backend calls
refreshCurrentViewContentson everyUIContext. Binary Ninja's ownfollow_reg_render_layer.pyexample does the same. CallingFunction.reanalyze()repaints too, but late and unreliably. The comparison below is from Binary Ninja 6.0 on the largest function of samplee0b6f853…(3,955 instructions, 1,361 blocks), double-clicking its match against mcrit.malpedia.io:Function.reanalyze()refreshCurrentViewContents()Screenshots of the real views confirmed the tint in both linear and graph view after the double click, with the function untinted before it. The layer's menu entry was not checked in the GUI. That menu is built when it opens, so the run could not reach it. The layer is registered as enabled by default, and the tint only appears when it is enabled.
The double click itself still blocks the UI for about 3.7 s on that function while it downloads the remote function and runs the matcher. That was already the case before this change and is not addressed here.
tests/binja/test_match_render_layer.pycovers four cases with a stubbedbinaryninja:mcrit_plugin.binjanow imports its modules insideregister_plugin, so that importing one of them, as the stubbed tests do, does not import the others. Before that change, the new test andtests/binja/test_secrets.pyfailed with seven errors when run together. The Binary Ninja GUI test now expects the new action, and checks that double clicking a function match colours the local blocks and that "MCRIT\Clear Match Coloring" removes the colouring. Against a local MCRIT 1.12.0 it passes all 174 checks.pytestgives 214 passed, andrun_quality_checks.pyand the settings and metadata sync checks pass. The IDA tests were not run.