Skip to content

Answer input the UI never sends with 400s instead of 500s - #283

Merged
danielplohmann merged 4 commits into
masterfrom
fix/bad-input-answers-not-500
Oct 16, 2026
Merged

danielplohmann merged 4 commits into
masterfrom
fix/bad-input-answers-not-500

Conversation

@r0ny123

@r0ny123 r0ny123 commented Oct 9, 2026 •

Copy link
Copy Markdown
Collaborator

A security review of master listed four robustness findings: requests the UI never sends that end in an unhandled exception, a 500 and a traceback in the log. None of them reads or writes anything it should not; each is a parse of request input with no answer for a bad value. This fixes them, except the upload parsing in analyze.query and submit, which #141 already reworks (see the last section).

Change.

request on master now
/analyze/cross_compare?samples=abc or ?cache=1,x ValueError from int() page renders, flash "not a list of sample ids" (mcritweb/views/analyze.py:251), as start_cross_compare already says
/data/import with JSON nested too deep RecursionError (the handler caught KeyError and ValueError) refused like any other non-MCRIT file (mcritweb/views/data.py:217)
/data/request_filename_info with a filename that is not a string AttributeError on .endswith treated as no filename (data.py:1417)
/data/request_filename_info with an .smda header whose base_addr has 5000 digits ValueError: Python refuses to convert an int of more than 4300 digits not read as a base address: at most 20 digits, a 64-bit one (data.py:1441)
/data/submit_or_query with an unknown form_type TypeError: the view returned nothing 400 (data.py:1466)
/data/jobs/<id>?refresh=abc or ?forward=x ValueError (the code caught only the TypeError of a missing value) read as absent, through the existing parse_integer_query_param (data.py:1297)
POST /api/query/function and POST /api/samples with a body that is not a report ValueError from SmdaReport.fromDict, or RecursionError from get_json for deep nesting 400 (mcritweb/views/api.py:116, :174)
the function comparison of two functions with more than 223 distinct unmatched instructions Exception("Too many distinct instructions ...") out of get_function_diff (data.py:372) compared

The two API routes now read the body with read_smda_report (mcritweb/views/utility.py:25). It catches ValueError and RecursionError from json.loads, and what SmdaReport.fromDict raises for something that is not a report. With smda 4.9.0 that was ValueError for every malformed shape tried; smda 4.2.13, the oldest mcrit allows, raises TypeError and KeyError for some of the same input, so those are caught too. Valid reports read as before: a UTF-8 BOM, UTF-16 and an empty body parse as get_json(force=True) parsed them. On /api/query/function the body is read after #273's cap, which reads no further than it needs to.

The function comparison's edit distance maps each distinct instruction to one character. The characters ran from chr(0x20) and stopped at chr(0xff), so the 224th distinct instruction raised. _symbol (mcritweb/views/functiondiff.py:164) now continues up the code points and skips the surrogates, which are not characters. rapidfuzz compares any code points; on 1000 x 1000 random blocks of 3 to 12 instructions it took 0.18 s with characters past 0x10000 against 0.21 s with the current ones (integer lists, the other option, took 0.37 s). A pair past 223 now costs what any pair of that size costs in the block-pair loop (0.4 s for 1000 x 1000 blocks and 2.5 s for 3000 x 3000, as measured in the review), instead of failing early; a finished comparison is memoized as before.

Verification. On the live testbed (mcrit 1.13.0), master (1cdc195) on :5000 against a side instance of this branch, same backend and account, requests that are refused before anything reaches the backend:

request master this branch
GET /analyze/cross_compare?samples=abc 500 200
GET /analyze/cross_compare?cache=1,x 500 200
GET /data/jobs/<id>?refresh=abc 500 200
GET /data/jobs/<id>?forward=x 500 200

The branch's log has no traceback for them, and the corpus check afterwards reports it unchanged. The side instance ran an earlier version of this commit that also changed analyze.query; that part is gone, the four routes above are unchanged since. The function comparison limit did not show on the testbed's corpus: comparing the largest function of each of eight samples against each other (28 pairs) stayed below 223 distinct unmatched instructions, since operands are escaped. Like the review, I could show it only at the function level; a crafted sample, whose instructions its author chooses, reaches it at will.

Tests. In tests/testBadRequests.py:

  • test_input_the_ui_never_sends_is_answered_not_raised sends each request in the first table but the comparison (11 cases) and asserts an answer below 500. 10 fail without this commit; a null filename with a null header already fell into master's own except.
  • test_the_api_answers_a_body_that_is_not_a_report_with_400 checks the 400 on both API routes, and test_a_report_that_older_smda_cannot_read_is_none pins the exceptions older smda raises.
  • TestJobPageParameters runs the job page on a captured job made to look unfinished, so the auto-refresh tag can appear: a number still sets it, a non-number leaves it out. test_a_hand_edited_sample_list_is_reported covers cross_compare.

In tests/testFunctionPages.py, test_more_than_223_distinct_instructions_are_compared pairs 300 one-instruction blocks with their twins at distance 0, and test_the_symbols_skip_the_surrogates pins the jump over them.

22 of the 24 new tests fail with the views before this commit (#273's head); the null case above and the guard TestJobPageParameters::test_a_number_still_sets_the_refresh pass there. tests/testCsrf.py::test_the_api_blueprint_is_exempt took any 400 from POST /api/samples for the CSRF refusal, and that route now answers its {} with its own 400, so the test looks for the CSRF message instead; it still fails when the exemption is removed.

Relation to open PRs.

analyze.query refuses a query upload larger than QUERY_UPLOAD_LIMITS
allows for the account's role (1 MiB for a visitor by default), but the
/api routes that queue the same kind of query, query/binary and
query/function, never looked at it. A visitor's token could queue a
query of up to MAX_CONTENT_LENGTH that the same account's browser would
have been refused.

Both routes now apply the same mapping and answer 403, as the form does.
The declared Content-Length is compared first, so an oversized body is
turned away without being read. A body with no declared length (chunked)
is read with one byte more than the cap as the request's limit, so it is
measured without reading more than that. The other /api routes that take a body
are id lists, not queries, and are not capped here.
A body without a Content-Length was measured by setting
request.max_content_length to one byte over the cap and reading it.
That assignment needs Flask 3.1, which made it per-request; on Flask
3.0, which requirements.txt still allows, the property has no setter
and the request ended in AttributeError, a 500. Checked with Flask
3.0.3.

A capped role's chunked body is now refused without being read, as
analyze.query will do for its uploads. McritClient sends bytes through
requests, which always declares a length, so no client of the API is
turned away by this. Uncapped roles are not affected.
@r0ny123
r0ny123 force-pushed the fix/bad-input-answers-not-500 branch from c48e441 to 939f086 Compare October 13, 2026 09:26
mcrit 1.15.0, which CI now installs, makes McritClient send through a
requests.Session() it opens in its constructor. The test swaps the
client's requests module for a stub with get and post only, so creating
the client raised AttributeError and all eight tests in the file
errored. The stub now hands itself back as the session. Same change as
in #285; it works with 1.13 and 1.15.
Several routes parsed request input without handling a bad value, so a
hand-made request ended in an unhandled exception:
- cross_compare: a non-numeric samples or cache list (int())
- import: JSON nested deeper than json.load follows (RecursionError)
- request_filename_info: a filename that is not a string, and a
  base_addr in the file header longer than int() converts
- submit_or_query: an unknown form_type, for which the view returned
  nothing
- the job page: a refresh or forward value that is not a number
- /api/query/function and POST /api/samples: a body that is not a
  report, which SmdaReport.fromDict raised on, or nested too deep for
  get_json
Each now answers 400, or reads the value as absent where it is only a
display option. The same kind of parse in analyze.query and submit is
left to #141, which already reworks both.

The function comparison's edit distance mapped instructions to the
characters chr(0x20) to chr(0xff) and raised past 223 distinct ones, so
two large or unusual functions failed the comparison page. The symbols
now run up the code points, skipping the surrogates; rapidfuzz compares
them as fast. Such a pair now costs what any pair of that size costs
in the block-pair loop, instead of failing early.

test_the_api_blueprint_is_exempt took any 400 from POST /api/samples
for the CSRF refusal; that route now answers {} with its own 400, so
the test looks for the CSRF message instead.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants