Skip to content

Show where a sample's sha256 comes from on the sample page - #321

Open
r0ny123 wants to merge 3 commits into
masterfrom
feat/270-sha256-provenance
Open

r0ny123 wants to merge 3 commits into
masterfrom
feat/270-sha256-provenance

Conversation

@r0ny123

@r0ny123 r0ny123 commented Oct 25, 2026 •

Copy link
Copy Markdown
Collaborator

The MCRITweb side of familiary/mcrit#270, option A, which familiary/mcrit#297 implements in mcrit, and of familiary/mcrit#306 stacked on it. mcrit now records SampleEntry.sha256_verified:

  • true when MCRIT disassembled the sample from an uploaded binary;
  • false when it was indexed from a submitted report (the IDA plugin, POST /samples) or an import, so the sha256 is only what that report states;
  • null for samples stored before, and for query samples.

What the sample page shows. In its SHA256 row:

  • true: a "verified against the uploaded binary" badge;
  • false: a note that the sha256 was taken from a submitted report or an import and was not checked against a binary;
  • null, or an entry without the attribute: nothing, since "not verified" would claim more than is known.

The value is read with the existing getattr template filter. sample_column_table is shared with the result and link hunt pages, whose samples come from stored job results, so it shows the provenance only when asked with show_sha256_provenance, and only the sample page asks.

Until mcrit is released. MCRITweb reads samples through the installed mcrit package's SampleEntry.fromDict. 1.16.0 drops the key even when the backend sends it, so with it nothing shows and nothing breaks; the marker appears once the package has the field. The README entry says so. No version floor is raised.

Uploading the binary of an unverified sample (535fbe6). With #306, POST /samples/binary with the bytes of a sample whose sha256_verified is not true disassembles them, replaces the stored analysis under the same sample_id and marks it verified; only a verified sample is still refused with 409. data.submit and promote_query looked the sha256 up first and stopped at "Sample was already in database" for any known hash, so the binary never reached mcrit and a reported sample could not be verified from MCRITweb.

A replacement whose sample is gone (7f5f575). #306 can finish the job with "sample_info": None when the sample is deleted during the replacement. The result page read result_json['sample_info']['sample_id'] directly and failed. It now goes to the sample when the result names one, and otherwise says "The sample this job added is no longer in the collection." and opens the samples list, as a deleteSample result does.

Tests:

  • tests/testSha256Provenance.py, six tests: each of the three states; a real SampleEntry with the attribute removed; only a table that asks shows the provenance; each column of a comparison table shows its own sample's. The tests that expect nothing first check that the same page shows the note for false. All six fail with the templates reverted.
  • tests/testUpload.py and tests/testPromoteQuery.py: a binary of an unverified sample (false, null, absent) is sent and lands on the job; a typed family and version travel with it; a backend that refuses lands on the sample as before; a verified sample is not sent; a report of a known sample verifies nothing; a mapped query is sent as the dump it was. The 10 cases for the new behaviour fail without 535fbe6.
  • tests/testResultPages.py: the result of a binary whose sample is gone (sample_info None, missing, empty) does not fail; each failed before 7f5f575 with a TypeError or KeyError. The normal redirect is checked too.
  • Full suite with Chromium: 2191 passed. ruff check . is clean.

mcrit now records whether it disassembled a sample from an uploaded
binary (SampleEntry.sha256_verified, familiary/mcrit#270). A sample
indexed from a submitted report or an import carries only the sha256
that report states, and nothing on the page told the two apart.

The SHA256 row of the sample page now says "verified against the
uploaded binary" when the value is True, and carries a note that the
sha256 was taken from a submitted report or an import and not checked
against a binary when it is False. None shows nothing, and so does an
entry without the attribute: the mcrit package installed with MCRITweb
builds its entries with SampleEntry.fromDict, which drops the key in
releases that predate the field, and "not verified" would claim more
than is known. The attribute is read with the existing getattr filter.

sample_column_table is shared with the result and link hunt pages, whose
samples come out of stored job results, so it shows the provenance only
when asked with show_sha256_provenance, and only the sample page asks.

The conflict answer for a binary whose sha256 is already stored is left
alone. McritClient.addBinarySample returns None for a 409, or raises
McritConflict with just the message when raise_client_errors is set, and
has no raw mode, so neither the status nor the answer's sample_id and
sha256_verified reach MCRITweb.
Submitting a binary whose sha256 is already a sample stopped at
"Sample was already in database" without asking MCRIT, and promoting a
binary query did the same. With familiary/mcrit#306, MCRIT replaces the
analysis of a sample whose sha256_verified is false or null with its
own disassembly of the uploaded bytes, so a user holding the real
binary of a report-submitted sample could not verify it from here.

Both routes now send the binary when the known sample is not verified
(False, None, or no attribute on an older mcrit package) and go to the
job. Blank family and version are sent as None, since mcrit applies
what the upload names to the replaced sample. An mcrit without #306
answers 409, which addBinarySample returns as None, and the routes then
fall back to the old message and the sample page. A verified sample
and an .smda report of a known sha256 are unchanged.
The result page redirected an addBinarySample job to
result_json['sample_info']['sample_id'] unguarded. familiary/mcrit#306
finishes the job with "sample_info": None when the sample is deleted
while its uploaded binary replaces its analysis, and an older result may
lack the key, so the page failed with a 500.

Without a sample id the page now says the sample is no longer in the
collection and goes to the samples list, as the deleteSample result
does. A result with a sample still goes to that sample.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant