Repository navigation
Export Binary Ninja reports through smda.binja - #22
Conversation
41f2a4a to
991b33e
Compare
991b33e to
91d7e44
Compare
danielplohmann
left a comment
There was a problem hiding this comment.
Reviewed the last commit (91d7e44) against the released smda 4.9.0. It's good to go once #17 and #23 are in.
- It matches the
v4.9.0API:smda.binja.BinjaExporter.exportBinaryView(bv),smda.binja.BinjaInterface.BinjaInterface(bv), and the(j_)?sub_default-name regex (BinjaInterface.py:9). No reference toBinjaSmdaInterfaceis left in the tree. - The
pythonsinput foroffline-dependencies.ymland the RELEASING instructions for a manual run are correct. - The CI pins: I've asked for them to move to 4.9.0 in #17, so the licensed IDA job runs 2.0.0 against the smda users will actually install. This PR then only inherits them, and step 2 of "Do not merge before" is done.
A side note that is older than this PR: according to PyPI, smda dropped Python 3.10 at 4.4.1 (4.3.10 is the last release with >=3.10). So the IDA offline bundles built for 3.10 have always resolved to smda 4.3.10, and so has any IDA install running Python 3.10. Nothing breaks, since the calls the plugin makes exist from 4.3.0 on, but those users miss every SMDA fix since. A line in README or RELEASING would be enough.
91d7e44 to
77b87f4
Compare
|
Thanks. The 4.9.0 CI pins landed in #17 (7aa4f95), and this PR is rebased onto it. With smda 4.9.0 from PyPI, the Binary Ninja GUI test (167 checks), the headless test, the IDA 9.3 GUI test (145 checks) and the IDALib test pass locally. The Python 3.10 note went into #17's README, because it concerns the IDA install: SMDA 4.3.10 is the last release for 3.10, so an IDA on 3.10 and the 3.10 wheelhouse bundle get that version. |
All of these behaved this way in 1.1.10. - Both the Function Overview and the result chooser allow sorting, which moves rows, while the Overview's label mappings and dropdown editors and the chooser's job_infos and status colours stayed keyed by the row each entry was filled into. After a header click, Import Labels applied each label to another function and the chooser loaded another job than the selected one. Each cell now carries the row it was filled into, or its job's index, and every lookup goes through that. Both GUI tests import labels from a table sorted by descending offset and check each renamed function against its own matches' labels; without the fix that check names three functions. - Function Scope's refresh returned before reading the cursor while no report existed, so Query Current Function and Query Current Block right after Convert found no current function. The cursor is followed first, and Query Current Block takes the block under the cursor. - MCRIT MinHashes a function only above 10 instructions and answers a query for one of exactly 10 with a 500; Function Scope now requires more than 10. - A failed PicBlockHash query was cached as None and read as "no matches"; only answers are cached now. - Function and Block Scope tested sample_infos against None, which it never is, so a failed sample list showed "unknown" hashes instead of "Remote family/sample info unavailable". - A failed job query opened the result chooser, which then claimed no matching results existed; the handler stops after the failure message. - Sample Info kept a family picked for an earlier result even when the next result has no match in it; it falls back to the best family. - Block Scope said "Live Function Queries are deactivated", and the IDA graph view's hover hint showed a placeholder; it shows the block's offset. SmdaGraphViewer's header no longer calls it a Hex-Rays example, and draw() no longer calls OnGetText only to discard it.
Neither did in 1.1.10. - MCRIT hands the query parameters of POST /query/function to MinHashIndex.getMatchesForSmdaFunction, and those of a single-sample matching job to Worker.getMatchesForSample. Neither takes sample_group_only, so with the setting on every Function Scope query answered 500 and every matching job failed. MCRIT's own client sends it only for cross compares, which the plugin does not request, and MCRITweb forwards neither request's parameters. The plugin no longer sends it; the setting stays declared, titled and documented as deprecated with no effect, and the vendored client's docstring says what MCRIT does with it. - config_override.json was merged into the defaults, which McritConfig reads only when the settings store raises. ida-settings answers with the declared default for any declared key, and Binary Ninja's Settings has every key once they are registered, so an override never applied. The overrides are kept apart and read first. README says the file's keys take precedence, so it should hold only the keys to force, and where it goes for Binary Ninja. - The config tests get an empty plugin root, so a developer's own config_override.json cannot change the defaults they pin. - An override in the plugin under test would now beat the settings the integration runners use, so all four refuse to start while one is there, before they install or write anything.
The plugin's first commit brought code over from IDAscope, written for IDA 6: attributes that kept json and os reachable on IDA 6.x for Windows, a datetime alias for the same reason, comments about shared MCRIT4IDA modules and IDA < 6.9, and in QtShim a PySide 1 branch for IDA 6.8 and below in each of the 50 accessors. The plugin requires IDA 9, nothing read the attributes, and the branches were unreachable. Every QtShim accessor still resolves under PySide6, and IDA 9.0/9.1 keep PyQt5. A comment typo is fixed on the way.
sample_group_only is removed. MCRIT fails every matching job and function query that carries it, so no working setup depends on it. With hcli 0.26 and ida-settings 3.5.1, a value that an earlier version stored stays in ida-config.json and is ignored: installing a version that no longer declares the key succeeds, and the remaining settings read, set and list as before. The vendored McritClient keeps its parameters, because MCRIT's cross compare takes them. config_override.json now takes precedence, so a full copy of the old template would pin every setting without anyone noticing. The plugin prints the keys an override forces when it loads. The template shows two keys instead of all eighteen, and an override whose top level is not a JSON object is ignored instead of failing the import. NumberQTableWidgetItem compared its text with float(), so sorting Function Scope's match table by its hex offsets raised inside Qt's sort, which crashes the process under PySide6 6.11. Hex text now compares by value. Scope queries: - An empty family or sample list from a reachable server counts as fetched; only a failed request (None) is fetched again. - A failed PicBlockHash lookup skips the rest of that visit's lookups instead of each waiting out the same timeout. - Query Current Block takes the function from the block under the cursor, since the Function Scope hook can lag behind on Binary Ninja. An offscreen PySide6 test covers the sorted Function Overview, the result chooser and hex sorting, and the pytest job installs PySide6 to run it. The IDA entry-point test hides Qt modules that other tests loaded, so it checks only the entry point's own imports.
SMDA's next release, 4.9.0, ships the Binary Ninja exporter as smda.binja, so the copy carried here goes away and the backend uses SMDA's. The exporter is then versioned with the report format it produces. Binary Ninja's requirement rises to smda>=4.9.0; the IDA plugin's requirement is unchanged. The import stays inside the method that needs it, as the other smda imports in the backend do, so a missing or old smda fails the export rather than the plugin load. ELF and Mach-O reports record the entry point relative to the base address, as native SMDA reports and IDA exports do; the copy passed base 0 and recorded an absolute address. PE reports are unchanged, since their entry point was already an RVA. smda.binja treats Binary Ninja's j_sub_ thunk names as default names, like sub_. The backend's default-name check follows it, so an auto-named j_sub_ thunk counts as unnamed both in the report and for label import. IDA and SMDA's IDA interface keep j_ thunk names, as before. That SMDA needs Python 3.11 or newer, so the Python 3.10 job of the Binary Ninja offline bundle could no longer resolve requirements.txt. offline-dependencies.yml takes the Python versions as an input, the Binary Ninja release asks for 3.11 to 3.13, and RELEASING.md gives the inputs for a manual run; the IDA bundles keep 3.10 to 3.13. AGENTS.md and docs/development.md name smda.binja and the smda>=4.9.0 requirement, README names the bundle's Python versions, and the changelog records each change for Binary Ninja users.
77b87f4 to
0dc36e3
Compare
danielplohmann
left a comment
There was a problem hiding this comment.
Approving at 0dc36e3, the same patch as the reviewed 91d7e44, rebased onto #23. It matches the released smda 4.9.0 smda.binja API, and CI is green, including the headless MCRIT job on 4.9.0. No licensed IDA rerun, since this touches only the Binary Ninja path.
The merge-base changed after approval.
Stacked on #23; the merge order is under "Do not merge before". Until those merge, their commits appear here too; the change itself is the last commit.
What changes
BinjaBackendexports throughsmda.binja.BinjaExporter.exportBinaryViewand reads the view throughsmda.binja.BinjaInterface. The copy of the exporter carried here,mcrit_plugin/binja/BinjaSmdaInterface.py, is deleted. The exporter is then versioned with the report format it produces (Adopt the Binary Ninja exporter from mcrit-plugin as smda.binja danielplohmann/smda#356, shipped by feat(binja): add Binary Ninja exporter as smda.binja danielplohmann/smda#357).smda.binjaimport stays inside the method that needs it, as the other smda imports in the backend do, so a missing or old smda fails the export rather than the plugin load.smda>=4.9.0inrequirements.txtandplugin.json. The IDA plugin's requirement is unchanged.smda.binjatreats Binary Ninja'sj_sub_thunk names as default names, likesub_(IDA and SMDA's IDA interface keepj_thunk names).BinjaBackend.has_default_function_namefollows it, so an auto-namedj_sub_thunk counts as unnamed both in the report and for label import.offline-dependencies.ymltakes the Python versions as an input, andRELEASING.mdgives the inputs for a manual run. The IDA bundles keep 3.10 to 3.13.docs/development.mdupdated.Behaviour
ELF and Mach-O reports record the entry point relative to the base address, as native SMDA reports and IDA exports do; the copy passed base 0 and recorded an absolute address. PE reports are unchanged, since their entry point was already an RVA.
j_sub_thunks no longer carry their name into the report. Everything else is identical, verified headlessly with Binary Ninja 6.0 against SMDA master: an x64 PE DLL (24 functions, 9 named) gives the same report on both branches, and a Mach-O x86-64 binary (135 functions) differs only inoep.Checks
Local at the head, against MCRIT 1.12.0 and SMDA 4.9.0 from PyPI: ruff clean, 214 tests pass, the metadata and settings sync checks pass, the Binary Ninja 6.0 GUI test (167 checks) and the headless test pass, and so do the IDA 9.3 GUI test in autonomous mode,
-A(145 checks) and the IDALib test.Do not merge before
refactor: move the export engine out of smda.ida danielplohmann/smda#360, ci: accept binja as a PR title scope danielplohmann/smda#358, feat(binja): add Binary Ninja exporter as smda.binja danielplohmann/smda#357 merge.Done 2026-09-28.SMDA 4.9.0 is released.Done; Multi-backend plugin: shared core, IDA and Binary Ninja frontends #17 moves CI's pins to 4.9.0 and this PR inherits them.Multi-backend plugin: shared core, IDA and Binary Ninja frontends #17 merges.Done. Fix label import, scope queries and settings that shipped wrong in 1.1.10 #23 merges.