Conversation
Rewrite negative controls that passed even with the guarded behavior broken, add @enum.unique to ErrorCode, and move the offline check_error_codes.py self-tests out of the `integration` marker so CI actually runs them.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What / why
Part of Batch 1 (correctness first) from #790. Each rewritten test was proven to FAIL by temporarily breaking the guarded production code locally, running the test, and restoring.
test_file_locking.py::TestTryFlock::test_flock_skipped_when_no_fcntl- patchesconfig_store.fcntlwith a mock and assertsflock.assert_not_called(). Proof: disabling theif not _HAS_FCNTL: returnguard now fails the test (before, the real flock on fd 42 raised OSError, which was suppressed, so it passed).test_errors.py::test_no_duplicate_values->test_no_aliased_members, plus@enum.uniqueonErrorCode. The old test iterated the enum, which skips aliases, so it could never fail. The new one checks__members__for aliases. Proof: removing@uniqueand adding a duplicate value fails it. With@uniquein place, a duplicate fails at import.test_server_permissions.py::...::test_app_without_an_engine_fails_closed- nowpytest.raises(PermissionDeniedError, match="not built by create_app")instead ofstatus_code != 200. Proof: removing the fail-closed branch inget_permission_enginenow fails withAttributeError(before, the 500 from that crash also passed).test_permissions_cli.py::...::test_set_rejected_without_confirmation- drops the mock ofrequire_random_code_confirmationso the real non-TTY refusal runs, and assertsstore.load().permissions is None. Proof: moving the save before the confirmation fails the test.test_billing_cli.py::...::test_per_project_errors_surface_as_warnings_exit_0- uses multi-letter aliases and asserts the full rendered warning line (Warning: Project 'beta': ...) plus the healthy row. Proof: skipping_emit_errorsfails it (before, it only asserted that the letter "b" appeared in the output).test_auto_update.py::...::test_sentinel_is_set_even_when_body_raises- asserts_AUTO_UPDATE_RAN is Trueafter the crash and that a second call never reaches the body again. Proof: not setting the sentinel fails it.test_search_service.py::TestResolveApiTypes::test_multiple_types_deduped- input now contains a repeat and two spellings of the same API type; it asserts the exact ordered result. Proof: removing theseencheck fails it. (The issue body files this under "semantic layer", but the test lives in the search service.)test_integration.pyerror-code guard self-tests (TestCheckErrorCodesGuard,TestErrorCodesDocCompleteness) - removed theintegrationmarker. They are offline, and CI's-m "not integration"was skipping them. Paths are now anchored to the repo root. The planted-literal and enum-usage tests now call the script's real_collect_violationsinstead of a copy of its logic. Proof: breaking the scanner inscripts/check_error_codes.pynow failstest_guard_catches_planted_literal(the old copy-of-logic version would still pass).Not in this PR
test_e2e_lineage_deep.py::test_12is untouched: it is an E2E test that needs live tokens.How tested
make checkis fully green: lint, format, ty, changelog check, error-code guard, 6902 passed / 15 skipped. No version bump and no changelog entry.Refs #790