Repository navigation
Conversation
mcrit now records whether it disassembled a sample from an uploaded binary (SampleEntry.sha256_verified, familiary/mcrit#270). A sample indexed from a submitted report or an import carries only the sha256 that report states, and nothing on the page told the two apart. The SHA256 row of the sample page now says "verified against the uploaded binary" when the value is True, and carries a note that the sha256 was taken from a submitted report or an import and not checked against a binary when it is False. None shows nothing, and so does an entry without the attribute: the mcrit package installed with MCRITweb builds its entries with SampleEntry.fromDict, which drops the key in releases that predate the field, and "not verified" would claim more than is known. The attribute is read with the existing getattr filter. sample_column_table is shared with the result and link hunt pages, whose samples come out of stored job results, so it shows the provenance only when asked with show_sha256_provenance, and only the sample page asks. The conflict answer for a binary whose sha256 is already stored is left alone. McritClient.addBinarySample returns None for a 409, or raises McritConflict with just the message when raise_client_errors is set, and has no raw mode, so neither the status nor the answer's sample_id and sha256_verified reach MCRITweb.
Submitting a binary whose sha256 is already a sample stopped at "Sample was already in database" without asking MCRIT, and promoting a binary query did the same. With familiary/mcrit#306, MCRIT replaces the analysis of a sample whose sha256_verified is false or null with its own disassembly of the uploaded bytes, so a user holding the real binary of a report-submitted sample could not verify it from here. Both routes now send the binary when the known sample is not verified (False, None, or no attribute on an older mcrit package) and go to the job. Blank family and version are sent as None, since mcrit applies what the upload names to the replaced sample. An mcrit without #306 answers 409, which addBinarySample returns as None, and the routes then fall back to the old message and the sample page. A verified sample and an .smda report of a known sha256 are unchanged.
The result page redirected an addBinarySample job to result_json['sample_info']['sample_id'] unguarded. familiary/mcrit#306 finishes the job with "sample_info": None when the sample is deleted while its uploaded binary replaces its analysis, and an older result may lack the key, so the page failed with a 500. Without a sample id the page now says the sample is no longer in the collection and goes to the samples list, as the deleteSample result does. A result with a sample still goes to that sample.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The MCRITweb side of familiary/mcrit#270, option A, which familiary/mcrit#297 implements in mcrit, and of familiary/mcrit#306 stacked on it. mcrit now records
SampleEntry.sha256_verified:truewhen MCRIT disassembled the sample from an uploaded binary;falsewhen it was indexed from a submitted report (the IDA plugin,POST /samples) or an import, so the sha256 is only what that report states;nullfor samples stored before, and for query samples.What the sample page shows. In its SHA256 row:
true: a "verified against the uploaded binary" badge;false: a note that the sha256 was taken from a submitted report or an import and was not checked against a binary;null, or an entry without the attribute: nothing, since "not verified" would claim more than is known.The value is read with the existing
getattrtemplate filter.sample_column_tableis shared with the result and link hunt pages, whose samples come from stored job results, so it shows the provenance only when asked withshow_sha256_provenance, and only the sample page asks.Until mcrit is released. MCRITweb reads samples through the installed mcrit package's
SampleEntry.fromDict. 1.16.0 drops the key even when the backend sends it, so with it nothing shows and nothing breaks; the marker appears once the package has the field. The README entry says so. No version floor is raised.Uploading the binary of an unverified sample (535fbe6). With #306,
POST /samples/binarywith the bytes of a sample whosesha256_verifiedis not true disassembles them, replaces the stored analysis under the samesample_idand marks it verified; only a verified sample is still refused with 409.data.submitandpromote_querylooked the sha256 up first and stopped at "Sample was already in database" for any known hash, so the binary never reached mcrit and a reported sample could not be verified from MCRITweb.sha256_verifiedis false, null or absent now sends the file, and the user lands on the job with: "Sample already has this sha256, but MCRIT has not verified it against a binary. MCRIT is replacing its stored analysis with its own disassembly of this file."addBinarySamplereturns asNone; the user then gets today's "Sample was already in database" and the sample page..smdareport of a known sha256, behave as before, without a request: a report cannot verify anything.A replacement whose sample is gone (7f5f575). #306 can finish the job with
"sample_info": Nonewhen the sample is deleted during the replacement. The result page readresult_json['sample_info']['sample_id']directly and failed. It now goes to the sample when the result names one, and otherwise says "The sample this job added is no longer in the collection." and opens the samples list, as adeleteSampleresult does.Tests:
tests/testSha256Provenance.py, six tests: each of the three states; a realSampleEntrywith the attribute removed; only a table that asks shows the provenance; each column of a comparison table shows its own sample's. The tests that expect nothing first check that the same page shows the note forfalse. All six fail with the templates reverted.tests/testUpload.pyandtests/testPromoteQuery.py: a binary of an unverified sample (false, null, absent) is sent and lands on the job; a typed family and version travel with it; a backend that refuses lands on the sample as before; a verified sample is not sent; a report of a known sample verifies nothing; a mapped query is sent as the dump it was. The 10 cases for the new behaviour fail without 535fbe6.tests/testResultPages.py: the result of a binary whose sample is gone (sample_infoNone, missing, empty) does not fail; each failed before 7f5f575 with a TypeError or KeyError. The normal redirect is checked too.ruff check .is clean.