From 889aabf6c2371ebb3268554427e8c14b6bf4d462 Mon Sep 17 00:00:00 2001 From: scuttletoby <123843277+scuttletoby@users.noreply.github.com> Date: Tue, 17 Nov 2026 23:18:05 +0100 Subject: [PATCH] head-check: config file mode + #403 follow-ups (#413) The merged-config check asked git only for the config's object, so a symlink, or a submodule entry naming the config's own blob, passed: the gate reads the blob, but the queue opens the file from a checkout and stops. It now reads the tree entry with git ls-tree, accepts only modes 100644 and 100755, and the unchanged skip compares the whole entry. A merge-tree that takes over 300 seconds is refused. The release notes and READMEs say the two-PR race is only narrowed (up-to-date branches close it) and that checkout mode needs a new event, not a re-run. New tests pin the caps at their edges, a merge-tree exit of 2 or more and merge.directoryRenames. Co-Authored-By: Claude Opus 5.5 --- fork-ci/RELEASE-NOTES.md | 19 +- fork-ci/actions/head-check/README.md | 16 +- fork-ci/actions/head-check/head_check.py | 67 ++++-- fork-ci/actions/head-check/head_check.test.py | 199 +++++++++++++++--- fork-ci/actions/pr-queue/README.md | 13 +- 5 files changed, 252 insertions(+), 62 deletions(-) diff --git a/fork-ci/RELEASE-NOTES.md b/fork-ci/RELEASE-NOTES.md index 89d5869ff9119..946d9bf853bf7 100644 --- a/fork-ci/RELEASE-NOTES.md +++ b/fork-ci/RELEASE-NOTES.md @@ -21,7 +21,9 @@ reason) when that config: - does not parse (`config.parse`: not JSON, empty, an unknown or bad field, a key written twice, a bad `review` block in any mode), or is in `enforce` with a rule order `enforce` refuses; -- is deleted, or is no longer a file; +- is deleted, or is no longer a regular file (git mode 100644 or 100755): a folder, a symlink or a submodule entry + is refused, even one whose object is the config's own (the queue opens the config from a checkout, where none of + them is a file); - is over the caps on a config a pull request writes: 32 KiB, 128 characters per glob in `review.rules[].paths`, and 2,048 glob characters in all; - cannot be read or checked at all (any error is a refusal that names it, never a crash or a pass). @@ -29,15 +31,24 @@ reason) when that config: Merged, any of these would leave a base config that refuses every pull request after it, the fix too. A pull request that leaves the config exactly as the base has it is not read again, so the caps never apply to a base config that is already bigger. In bare mode, a head that does not merge cleanly into the base is refused too (GitHub cannot merge it -either). +either), and so is a `git merge-tree` that takes over 300 seconds. + +This narrows the race of two pull requests that each leave a good config but merge into a bad one; it does not close +it. The check judges a head on the base as of its run, and no trigger runs it again when new commits land on the base +(`edited` fires on a retarget or a title or body edit), so a pass from before the first merge still stands for the +second. Only the ruleset's "Require branches to be up to date before merging" closes it: the second pull request is +then behind, and the update that brings it up to date is a new head, judged on the merged config. **Upgrade:** - To delete the queue config, first take its checks out of the branch ruleset; a pull request that deletes it is refused. - A config change that needs more than the caps has to be pushed by an owner. -- After an owner fixes a refused base config by hand, open pull requests no longer need to merge the base first: they - are judged on what they would merge. +- After an owner fixes a refused base config by hand, open pull requests no longer need to merge the base first: a + new run judges them on what they would merge. In checkout mode that takes a new event (a push, a reopen, a title or + body edit): a re-run of the old `pull_request` job checks out the old merge commit again, which holds the old base. +- A base whose config is a symlink or a submodule entry (only an owner's push can leave one now) refuses every pull + request until one makes it a regular file again; that one is judged as a new config. ### pr-queue and head-check: stricter rule order in `review` (#378) diff --git a/fork-ci/actions/head-check/README.md b/fork-ci/actions/head-check/README.md index 1d45c47369161..20b41e4f1769f 100644 --- a/fork-ci/actions/head-check/README.md +++ b/fork-ci/actions/head-check/README.md @@ -90,7 +90,7 @@ make that job a required check (the config's `head_check`): ```yaml on: pull_request: - # edited: a base change (a stacked pull request retargeted) runs it again + # edited: a retarget to another base branch runs it again (new commits on the base do not) types: [opened, synchronize, reopened, ready_for_review, edited] permissions: {} jobs: @@ -114,9 +114,17 @@ jobs: The check reads the comments when it runs. After a lens or an override is posted for the head, run it again (re-run the job, or let a tool the project uses for its ready command do that); a new push runs it anyway. The `edited` -trigger runs it again when the base branch changes: the answer depends on the -base (its config, and which merged-in commits are the base's). A title or body -edit also runs it; it only reads, so that costs one short job. +trigger runs it again when the pull request is retargeted to another base +branch: the answer depends on the base (its config, and which merged-in +commits are the base's). A title or body edit also runs it; it only reads, so +that costs one short job. Nothing runs it when new commits land on the base, +so a pass can be older than the base: two pull requests that each leave a good +config can still merge into one that does not parse. Turn on "Require branches +to be up to date before merging" in the ruleset (the queue needs it anyway): +the second pull request is then behind, and its update is a new head, judged +on the merged config. A re-run of an old job checks out the old merge commit +again; after the base changed, a new event (a push, a reopen, a title edit) +judges the head on the new base. **Pin it by the release sha, always.** Use the full sha a `ci/vN` release names (the release sha of ci/vN), never a branch, a tag alone or a local path diff --git a/fork-ci/actions/head-check/head_check.py b/fork-ci/actions/head-check/head_check.py index 3c35d98e06f44..f92330211e9de 100644 --- a/fork-ci/actions/head-check/head_check.py +++ b/fork-ci/actions/head-check/head_check.py @@ -73,14 +73,16 @@ def __init__(self, path, run=None, bare=None, remote='origin'): # Bare, git logs in the way the queue does (gh's login, GH_TOKEN), never with a helper of the machine's config. self.auth = pr_queue.GIT_AUTH if self.bare else () - def git(self, *a, errors='strict'): - """errors: how git's output is decoded; 'surrogateescape' for file names, which need not be UTF-8.""" + def git(self, *a, errors='strict', timeout=None): + """errors: how git's output is decoded; 'surrogateescape' for file names, which need not be UTF-8. timeout: + seconds, after which git is killed and subprocess.TimeoutExpired raised.""" if self.run: return self.run(*a) with tempfile.TemporaryDirectory(prefix='head-check-') as empty: return subprocess.run(['git', *GIT_SAFE, *self.auth, '--git-dir', self.git_dir, *a], cwd=empty, text=True, - errors=errors, capture_output=True, env={**os.environ, 'GIT_TERMINAL_PROMPT': '0', - 'GIT_ATTR_NOSYSTEM': '1', **pr_queue.GIT_ENV}) + errors=errors, capture_output=True, timeout=timeout, + env={**os.environ, 'GIT_TERMINAL_PROMPT': '0', 'GIT_ATTR_NOSYSTEM': '1', + **pr_queue.GIT_ENV}) def problem(self): """Why the repository cannot be read as git made it, or None: not a git folder, shallow (the base's history is @@ -398,6 +400,10 @@ def cli(argv=None, env=None, gh=gh_run, sleep=time.sleep, git=None, out=print): MAX_CONFIG_BYTES = 32 * 1024 MAX_GLOB_LENGTH = 128 # characters of one glob in review.rules[].paths MAX_GLOB_TOTAL = 2048 # characters of all of them together +# The queue opens the config from a checkout (config.load): only a regular file is one there. git keeps a symlink +# (120000) as a blob of its target text, and a submodule (160000) can name any object, so the object alone says nothing. +FILE_MODES = ('100644', '100755') +MERGE_TREE_SECONDS = 300 # a merge that takes longer is refused; the gate's job has 15 minutes in all LOCKS = ('the base config would refuse every pull request after it, the one that fixes it too, and the queue would ' 'stop') @@ -432,9 +438,10 @@ def merged_config_problem(checkout, path): """Why the config this pull request would merge must not merge, or None. The config that judges a head is the base's, so a head is judged by the old config and could merge a new one that does not parse (or is in enforce with a rule order enforce refuses, or is deleted); after it, the base config refuses every head, the one that fixes it - too, and the queue stops. So the config the merge leaves must parse (config.parse), under the caps above. It is - the merge's, never the head's own copy: two pull requests that each leave a good config can merge into a bad one, - and an older head's stale copy says nothing once an owner fixed the base. Checkout mode reads GitHub's merge commit + too, and the queue stops. So the config the merge leaves must parse (config.parse), under the caps above, and stay + a regular file (FILE_MODES: the queue opens it from a checkout). It is the merge's, never the head's own copy: two + pull requests that each leave a good config can merge into a bad one, and an older head's stale copy says nothing + once an owner fixed the base. Checkout mode reads GitHub's merge commit (HEAD), bare mode `git merge-tree` of the base and the head (a head that does not merge cleanly is refused: the merged config is unknown, and GitHub cannot merge it either). The merged config is never used for anything else: the rules that judge the head stay the base's. Any exception is a refusal that says why, never a crash or a pass.""" @@ -448,12 +455,32 @@ def merged_config_problem(checkout, path): f'{type(e).__name__}: {review_judge.shown(str(e))[:300]}') +def tree_entry(checkout, tree, rel): + """(mode, type, oid) of the entry `rel` in `tree` as git ls-tree lists it, or None when there is none. Raises + ValueError when git fails or lists anything but that one entry (the caller refuses).""" + r = checkout.git('ls-tree', '-z', tree, '--', rel, errors='surrogateescape') + records = [x for x in (r.stdout or '').split('\0') if x] if r.returncode == 0 else None + if records == []: + return None + meta, _, name = records[0].partition('\t') if records else ('', '', '') + fields = meta.split(' ') + if len(records or ()) != 1 or name != rel or len(fields) != 3 or not re.fullmatch(r'[0-7]{6}', fields[0]) \ + or not SHA.fullmatch(fields[2]): + said = review_judge.shown((r.stderr if r.returncode else r.stdout or '').strip()[:200]) + raise ValueError(f'git ls-tree {tree[:12]} -- {rel} did not list one entry: {said}') + return tuple(fields) + + def _merged_config_problem(checkout, path, base, head): rel = repo_path(path) merged = 'HEAD' if checkout.bare: - m = checkout.git('-c', 'merge.directoryRenames=true', 'merge-tree', '--write-tree', base, head, - errors='surrogateescape') + try: + m = checkout.git('-c', 'merge.directoryRenames=true', 'merge-tree', '--write-tree', base, head, + errors='surrogateescape', timeout=MERGE_TREE_SECONDS) + except subprocess.TimeoutExpired: + return (f'git merge-tree of the base {base[:12]} and the head {head[:12]} took over {MERGE_TREE_SECONDS} ' + 'seconds, so the config it would merge is unknown') tree = (m.stdout or '').split()[:1] if m.returncode == 1: return (f'the head {head[:12]} does not merge cleanly into the base {base[:12]}, so the config it would ' @@ -462,18 +489,20 @@ def _merged_config_problem(checkout, path, base, head): said = review_judge.shown((m.stderr or m.stdout or '').strip()[:200]) return f'git merge-tree of the base {base[:12]} and the head {head[:12]} failed: {said}' merged = tree[0] - new = checkout.git('rev-parse', '--verify', '-q', f'{merged}:{rel}') - if new.returncode != 0: + new = tree_entry(checkout, merged, rel) + if new is None: return (f'merged, this pull request deletes {path}: {LOCKS}. To stop using the queue, take its checks out of ' 'the branch ruleset first') - old = checkout.git('rev-parse', '--verify', '-q', f'{base}:{rel}') - if old.returncode == 0 and old.stdout.strip() == new.stdout.strip(): - return None # the merge leaves the base's own config, which base_config parsed - oid = new.stdout.strip() - kind = checkout.git('cat-file', '-t', oid).stdout.strip() - if kind != 'blob': - what = review_judge.shown(kind[:20]) or 'thing' - return f'merged, this pull request leaves {path} as a {what}, not a file: {LOCKS}' + mode, kind, oid = new + if kind == 'tree': + return f'merged, this pull request leaves {path} as a tree, not a file: {LOCKS}' + if mode not in FILE_MODES or kind != 'blob': # even when the base holds the same: unknown never passes + what = {'120000': 'symlink', '160000': 'submodule'}.get(mode) or review_judge.shown(kind[:20]) + return (f'merged, this pull request leaves {path} as a {what} (mode {mode}), not a file (mode 100644 or ' + '100755): the queue reads the config from a checkout, where it is no file, and would stop. Make it a ' + 'file again') + if tree_entry(checkout, base, rel) == new: + return None # the merge leaves the base's own config, mode and text, which base_config parsed size = checkout.git('cat-file', '-s', oid).stdout.strip() if not size.isdigit() or int(size) > MAX_CONFIG_BYTES: return config_size_problem(path, size) diff --git a/fork-ci/actions/head-check/head_check.test.py b/fork-ci/actions/head-check/head_check.test.py index c2fe512ab0d04..329be38b4b1b0 100644 --- a/fork-ci/actions/head-check/head_check.test.py +++ b/fork-ci/actions/head-check/head_check.test.py @@ -972,6 +972,83 @@ def test_a_base_config_over_the_caps_still_judges_heads_that_leave_it_alone(self self.assertEqual(code, H.REFUSED, out) # a change to it is read under the caps self.assertIn('bytes the check reads', out) + def entry(self, commit, mode, oid, parent): + """`commit`'s tree with the config's entry made (mode, oid), committed on `parent`: a symlink (120000) or a + submodule (160000), which World.commit (it writes files) cannot make.""" + env = {**os.environ, **GIT_ENV, 'GIT_INDEX_FILE': os.path.join(self.w.dir, 'entry.index')} + + def git(*a): + return subprocess.run(['git', *a], cwd=self.w.work, env=env, text=True, capture_output=True, + check=True).stdout.strip() + git('read-tree', commit) + git('update-index', '--add', '--cacheinfo', f'{mode},{oid},{CONFIG_PATH}') + sha = git('commit-tree', git('write-tree'), '-p', parent, '-m', 'entry') + self.w.n += 1 + git('push', '-q', 'origin', f'{sha}:refs/keep/{self.w.n}') + return sha + + def blob(self, text): + return subprocess.run(['git', 'hash-object', '-w', '--stdin'], cwd=self.w.work, input=text, text=True, + capture_output=True, check=True).stdout.strip() + + def test_a_config_that_is_no_regular_file_is_refused(self): # #413 item 1: the queue open()s it and would stop + base = self.base(mode=None) + same = self.w.git(self.w.work, 'rev-parse', f'{base}:{CONFIG_PATH}') + edited = self.blob(json.dumps({**CFG_RAW, 'merge_method': 'merge'})) + docs = self.head(base, mode=None, **{'docs/a.md': 'a\n'}) + for name, mode, oid, what in (('a symlink, the same text', '120000', same, 'symlink'), + ('a symlink whose text is edited JSON that parses', '120000', edited, 'symlink'), + ("a submodule with the config's own id", '160000', same, 'submodule')): + with self.subTest(name): + code, out = self.check(base, self.entry(docs, mode, oid, base), EVERY_LENS) + self.assertEqual(code, H.REFUSED, out) + self.assertIn(f'merged, this pull request leaves {CONFIG_PATH} as a {what} (mode {mode}), not a file', + out) + for name, oid in (('the same text', same), ('edited text', edited)): + with self.subTest('an executable file, ' + name): # 100755 is a file: the queue reads it + code, out = self.check(base, self.entry(docs, '100755', oid, base), EVERY_LENS) + self.assertEqual(code, 0, out) + + def test_a_base_that_holds_a_symlink_config_refuses_heads_until_one_makes_it_a_file(self): # old state: no lock + m = self.base(mode=None) + oid = self.w.git(self.w.work, 'rev-parse', f'{m}:{CONFIG_PATH}') + base = self.entry(m, '120000', oid, m) # the gate reads the link's text; the queue's open() cannot + code, out = self.check(base, self.entry(self.head(base, mode=None, **{'docs/a.md': 'a\n'}), '120000', oid, + base), EVERY_LENS) + self.assertEqual(code, H.REFUSED, out) # it leaves the symlink: never passed + self.assertIn('as a symlink (mode 120000), not a file', out) + for name, cfg in (('the same text', CFG_RAW), ('edited text', {**CFG_RAW, 'merge_method': 'merge'})): + with self.subTest('a head that makes it a file again, ' + name): + fix = self.w.commit(base_files(cfg_raw=cfg, **PROMPTS, **self.STORE), [base]) + code, out = self.check(base, fix, EVERY_LENS) + self.assertEqual(code, 0, out) + big = json.dumps({**CFG_RAW, 'lens_format_doc': 'd' * H.MAX_CONFIG_BYTES}) # the gate reads it, uncapped + base = self.entry(m, '120000', self.blob(big), m) + fix = self.w.commit(base_files(cfg_raw=json.loads(big), **PROMPTS, **self.STORE), [base]) + self.assertEqual(self.w.git(self.w.work, 'rev-parse', f'{fix}:{CONFIG_PATH}'), self.blob(big)) + code, out = self.check(base, fix, EVERY_LENS) # the same text, now a file: a new entry, read under the caps + self.assertEqual(code, H.REFUSED, out) + self.assertIn('bytes the check reads', out) + + def test_the_caps_hold_at_their_edge(self): # #413 item 4: the documented values, exactly at and one over + base = self.base('shadow') + at = 'd/' * 62 + '*.py' # 128 characters + rules = [dict(REVIEW_BLOCK['rules'][0], paths=[at]), *REVIEW_BLOCK['rules'][1:]] + cfg = review_cfg('shadow', rules=rules) + text = json.dumps(cfg) + cfg['lens_format_doc'] += 'd' * (32 * 1024 - len(text)) # 32 KiB exactly + self.assertEqual((len(at), len(json.dumps(cfg).encode())), (128, 32 * 1024)) + code, out = self.check(base, self.w.commit(self.tree(json.dumps(cfg)), [base]), EVERY_LENS) + self.assertEqual(code, 0, out) + over_size = dict(cfg, lens_format_doc=cfg['lens_format_doc'] + 'd') + over_glob = review_cfg('shadow', rules=[dict(rules[0], paths=['x' + at]), *rules[1:]]) + for name, over, why in (('32 KiB + 1', json.dumps(over_size), 'at 32769 bytes, over the 32768 bytes'), + ('a glob of 129', json.dumps(over_glob), 'a glob of 129 characters, over the 128')): + with self.subTest(name): + code, out = self.check(base, self.w.commit(self.tree(over), [base]), EVERY_LENS) + self.assertEqual(code, H.REFUSED, out) + self.assertIn(why, out) + def test_a_config_path_with_a_leading_dot_slash_is_the_same_file(self): # #388 item 5: git show rev:./x differs base = self.w.commit(self.tree(review_cfg('shadow', rules=BAD_ORDER)), [self.m0]) dot = './' + CONFIG_PATH @@ -1023,6 +1100,28 @@ def test_a_head_that_does_not_merge_cleanly_is_refused(self): # #388 item 1: t self.assertIn(f'the head {head[:12]} does not merge cleanly into the base {b1[:12]}', out) self.assertEqual(self.check(b0, head, EVERY_LENS)[0], 0) # the same head on the base it merges into + def test_the_merge_follows_a_directory_rename_as_the_queue_does(self): # #413 item 4: merge.directoryRenames + # pr_queue.holds merges with merge.directoryRenames=true, so the check does too: the base adds the config into a + # folder the head renamed, the merge moves it along, and the config's path is gone (git's default, conflict, + # would refuse it as a conflict instead). + path = 'E/q.json' + m = self.w.commit(base_files(**{**PROMPTS, 'E/a.txt': lines(40), 'E/b.txt': lines(40, ['b'], 3)}), [self.m0]) + base = self.w.commit(base_files(**{**PROMPTS, 'E/a.txt': lines(40), 'E/b.txt': lines(40, ['b'], 3), + path: json.dumps(CFG_RAW)}), [m]) + head = self.w.commit(base_files(**{**PROMPTS, 'F/a.txt': lines(40), 'F/b.txt': lines(40, ['b'], 3)}), [m]) + code, out = self.check(base, head, EVERY_LENS, config_path=path) + self.assertEqual(code, H.REFUSED, out) + self.assertIn(f'merged, this pull request deletes {path}', out) + + def test_a_merge_tree_that_takes_too_long_is_refused(self): # #413 item 5: a timeout refuses, never passes + base = self.base('shadow') + head = self.head(base, 'shadow', **{'docs/a.md': 'a\n'}) + with unittest.mock.patch.object(H, 'MERGE_TREE_SECONDS', 1e-6): + code, out = self.check(base, head, EVERY_LENS) + self.assertEqual(code, H.REFUSED, out) + self.assertIn(f'git merge-tree of the base {base[:12]} and the head {head[:12]} took over', out) + self.assertEqual(self.check(base, head, EVERY_LENS)[0], 0) # the same merge, given its time + class ChangedPaths(unittest.TestCase): """changed_paths and path_judgement against git answers a real repository rarely gives.""" @@ -1093,22 +1192,13 @@ def git(self, *a, **kw): 'passed: OSError: git vanished', why) def test_a_config_over_the_size_cap_is_never_read(self): # its size comes first: a 1 GB blob never reaches memory - for size in ('999999999', 'nonsense'): + for size in ('999999999', 'nonsense', str(32 * 1024 + 1)): with self.subTest(size): - class Checkout: - bare = None - - def merge_parents(self): - return (C, A) - - def git(self, *a, **kw): - answer = {('rev-parse', '--verify'): ('1' if a[-1].startswith(C) else '2') * 40, - ('cat-file', '-t'): 'blob\n', ('cat-file', '-s'): size + '\n'}.get(a[:2]) - if answer is None: - raise AssertionError(f'read: {a}') - return types.SimpleNamespace(returncode=0, stdout=answer, stderr='') - why = H.merged_config_problem(Checkout(), CONFIG_PATH) + why = H.merged_config_problem(reading({('cat-file', '-s'): (0, size + '\n')}), CONFIG_PATH) self.assertIn(f'leaves {CONFIG_PATH} at {size} bytes, over the {H.MAX_CONFIG_BYTES} bytes', why) + text = json.dumps(CFG_RAW) # exactly at the cap: read and parsed + self.assertIsNone(H.merged_config_problem( + reading({('cat-file', '-s'): (0, f'{32 * 1024}\n'), ('cat-file', 'blob'): (0, text)}), CONFIG_PATH)) def test_text_that_breaks_the_parser_is_a_refusal(self): for name, text, why in (('deep nesting', '[' * 20000, 'RecursionError'), @@ -1117,24 +1207,73 @@ def test_text_that_breaks_the_parser_is_a_refusal(self): self.assertIn(why, H.config_text_problem(text, CONFIG_PATH)) def test_a_config_that_is_not_utf8_is_a_refusal(self): # strict: a config is text, and base_config reads it so - class Checkout: - bare = None - - def merge_parents(self): - return (C, A) - - def git(self, *a, **kw): - if a[:2] == ('rev-parse', '--verify'): - return types.SimpleNamespace(returncode=0, stdout=('1' if a[-1].startswith(C) else '2') * 40, - stderr='') - if a[:2] == ('cat-file', '-t'): - return types.SimpleNamespace(returncode=0, stdout='blob\n', stderr='') - if a[:2] == ('cat-file', '-s'): - return types.SimpleNamespace(returncode=0, stdout='9\n', stderr='') - b'{"x": "\xff"}'.decode() - why = H.merged_config_problem(Checkout(), CONFIG_PATH) + def blob(*a): + b'{"x": "\xff"}'.decode() + why = H.merged_config_problem(reading({('cat-file', '-s'): (0, '9\n'), ('cat-file', 'blob'): blob}), + CONFIG_PATH) self.assertIn('could not be checked, so it is not passed: UnicodeDecodeError', why) + def test_the_caps_are_the_documented_values_exactly_at_and_one_over(self): # #413 item 4: the docs name them + self.assertEqual((H.MAX_CONFIG_BYTES, H.MAX_GLOB_LENGTH, H.MAX_GLOB_TOTAL), (32 * 1024, 128, 2048)) + cfg = dict(CFG_RAW, lens_format_doc='') + cfg['lens_format_doc'] = 'd' * (32 * 1024 - len(json.dumps(cfg))) + self.assertIsNone(H.config_text_problem(json.dumps(cfg), CONFIG_PATH)) # 32 KiB exactly + cfg['lens_format_doc'] += 'd' + self.assertIn('at 32769 bytes, over the 32768 bytes', H.config_text_problem(json.dumps(cfg), CONFIG_PATH) or '') + + def rules(*globs): + return {'review': {'rules': [{'paths': list(globs)}]}} + self.assertIsNone(H.glob_cap_problem(rules('g' * 128))) + self.assertIn('a glob of 129 characters, over the 128', H.glob_cap_problem(rules('g' * 129)) or '') + self.assertIsNone(H.glob_cap_problem(rules(*['g' * 128] * 16))) # 2,048 in all + self.assertIn('globs of 2049 characters in all, over the 2048', + H.glob_cap_problem(rules(*['g' * 128] * 16, 'g')) or '') + + def test_a_merge_tree_that_fails_is_refused_whatever_it_printed(self): # #413 item 4: exit 2 or more + text = json.dumps(CFG_RAW) + for code in (2, 128): + with self.subTest(code): + checkout = reading({'merge-tree': (code, 'f' * 40 + '\n'), ('cat-file', '-s'): (0, f'{len(text)}\n'), + ('cat-file', 'blob'): (0, text)}, bare=(C, A)) + why = H.merged_config_problem(checkout, CONFIG_PATH) + self.assertIn(f'git merge-tree of the base {C[:12]} and the head {A[:12]} failed', why or '') + clean = reading({**checkout.answers, 'merge-tree': (0, 'f' * 40 + '\n')}, bare=(C, A)) + self.assertIsNone(H.merged_config_problem(clean, CONFIG_PATH)) # the same answers with exit 0 pass + + def test_a_tree_entry_git_lists_oddly_is_a_refusal(self): # #413 item 1: only one record, of the config's path + for name, out in (('another path', f'100644 blob {B}\tother.json\0'), + ('two records', f'100644 blob {B}\t{CONFIG_PATH}\x00100644 blob {B}\t{CONFIG_PATH}\0'), + ('no sha', f'100644 blob xyz\t{CONFIG_PATH}\0'), ('no mode', f'blob {B}\t{CONFIG_PATH}\0'), + ('a mode that is none', f'10o644 blob {B}\t{CONFIG_PATH}\0')): + with self.subTest(name): + why = H.merged_config_problem(reading({'ls-tree': (0, out)}), CONFIG_PATH) + self.assertIn('could not be checked, so it is not passed', why or '') + why = H.merged_config_problem(reading({'ls-tree': (128, '')}), CONFIG_PATH) + self.assertIn('could not be checked, so it is not passed', why or '') + + +def reading(answers, bare=None): + """A checkout of the base C and the head A whose git answers each call by its words (`merge-tree`, `ls-tree`, or + the first two): (code, stdout), or a function that raises. Without an `ls-tree` answer the config is a file, the + base's blob 1... at the base and 2... in the merge. A call it has no answer for fails the test.""" + class Checkout: + def merge_parents(self): + return bare or (C, A) + + def git(self, *a, **kw): + key = 'merge-tree' if 'merge-tree' in a else a[0] if a[0] == 'ls-tree' else a[:2] + answer = self.answers.get(key) + if answer is None and key == 'ls-tree': + answer = (0, f'100644 blob {("1" if a[2] == C else "2") * 40}\t{a[-1]}\0') + if answer is None: + raise AssertionError(f'read: {a}') + if callable(answer): + return answer(*a) + return types.SimpleNamespace(returncode=answer[0], stdout=answer[1], stderr='boom' if answer[0] else '') + checkout = Checkout() + checkout.bare, checkout.answers = bare, answers + return checkout + # The shapes the #378 security lens measured, and the worst ones under the caps: each in a child process with a time and # a memory limit, so a regression stalls or fails this test, never the run. diff --git a/fork-ci/actions/pr-queue/README.md b/fork-ci/actions/pr-queue/README.md index 5769f849dd1b2..3907933611296 100644 --- a/fork-ci/actions/pr-queue/README.md +++ b/fork-ci/actions/pr-queue/README.md @@ -160,11 +160,14 @@ config from the base tip, so a base config that does not parse refuses every head, the one that fixes it too, and the queue stops (an owner then has to push the fix by hand). The gate therefore refuses a pull request whose merged config (GitHub's merge commit, or `git merge-tree` of base and head in the gate's bare -mode) does not parse, is deleted, or is in `enforce` with a rule order `enforce` -refuses (the order checks above), and caps what it reads of a config a pull -request changes (32 KiB, 128 characters per glob, 2,048 in all). That one check -reads the merged config; everything else, the rules that judge the head too, the -base's. +mode) does not parse, is deleted, is no regular file (a folder, a symlink or a +submodule entry), or is in `enforce` with a rule order `enforce` refuses (the +order checks above), and caps what it reads of a config a pull request changes +(32 KiB, 128 characters per glob, 2,048 in all). That one check reads the +merged config; everything else, the rules that judge the head too, the base's. +It judges the merge with the base of its run: two pull requests that each pass +alone can still merge into a bad config unless the ruleset requires branches to +be up to date, which the queue needs anyway. **The empty input.** The gate refuses an empty `required-lenses` input (a head with no lens comment would pass), except when the config at the base tip has a