fix(tests): keep Dependencies/.env out of the master suite (and fix the loader guard) - #182
Merged
Merged
Conversation
…not later One of the guards added in #181 fails on the operator's machine and nowhere else. test_the_flag_is_set_for_the_load_only asserted that, by the time it ran, os.environ held the same SL_HUNTING_ENABLED as before the master was loaded. On the operator's box it found 'true' where it had been unset. The loader is not the cause. It sets the flag inside patch.dict, which restores the environment when the load ends - verified. The value comes back later, from the NEXT import in the test module: flattrade_execution.py calls load_dotenv() at import time, outside any patch, and re-reads Dependencies/.env. Proved in isolation: importing that one module moves SL_HUNTING_ENABLED from unset to 'true'. CI and every worktree have no .env, which is why the guard passed there and on #181's CI. The fix records the value the moment the master's load block ends, before that import, and the guard (renamed test_the_loader_puts_back_the_flag_it_set) compares THAT against the pre-load value. It now tests exactly what it claims - that this loader puts back what it found - and is indifferent to .env loading elsewhere. Reproduced without copying any credentials: a throwaway Dependencies/.env containing only "SL_HUNTING_ENABLED=true" makes the old guard fail (617 run, 1 failure) and the new one pass; without it both pass. The throwaway file was removed. Negative-tested with AND without that .env: setting the flag for the whole process before the load, and setting it after the load instead of inside patch.dict, are both caught by the rewritten guard; a control rewording passes. The flattrade .env leak itself is pre-existing and left alone here - it loads the operator's whole .env into the test process, which makes the master suite less hermetic on that machine than in CI. Flagged separately. Gates: master 617 OK both with the one-line .env and without; 28 market-data-health; 1594 pytest; ruff; mypy (80 files); compileall. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The previous commit on this PR made one guard read its value at the right
moment. This removes the cause it worked around: on the operator's machine the
master suite ran against the operator's .env, and CI - which has no .env - did
not.
TWO LEAKS, NOT ONE. The master, dhan_execution and flattrade_execution call
load_dotenv() on Dependencies/.env at import time.
1. flattrade_execution and its diagnostic were imported OUTSIDE any patch,
so their load copied the whole .env - flags and broker credentials alike
- into os.environ for every test that followed.
2. The master's own load was already inside patch.dict, which restored
os.environ afterwards - but its load_dotenv still ran, and still let .env
values decide the master's import-time CONSTANTS.
MEASURED, without touching a real .env: a throwaway Dependencies/.env holding
the 616 key NAMES from env.example, each set to a placeholder.
leaked into os.environ tests that changed
before (this PR's parent) 616 21
Flattrade imports wrapped only 0 12
all three loads isolated 0 0
Of the 21, nine read the environment at run time (broker routing x4, the
live/paper gates, the per-strategy virtual-trading toggle) and the other twelve
depended on master constants set at import (the websocket feed, the strangle
re-entry, order correlation). None needs a .env - all pass without one - so
none was relying on it; they were exposed to it. With the operator's real
.env they passed only because those values happen to be sane.
THE FIX, TEST-SIDE ONLY. Every module-level repository load in the suite now
runs inside _isolated_from_dotenv: patch.dict restores os.environ exactly (and
carries the values a load needs - the master's DHAN_* stubs and
SL_HUNTING_ENABLED), and dotenv.load_dotenv is stubbed for the duration so the
file is not read at all. Production's import-time load_dotenv is unchanged, and
no production default moves. Follows the precedent in
Tests/Data Extractors/test_index_fetch_construction.py.
GUARDS, one of which holds in CI (where the others would be blind):
* behavioural - snapshots os.environ before the first repository import and
after the last, and fails naming any variable added, removed or changed
(names only, never values: on that machine a value can be a credential);
* probe - writes a throwaway module and .env to a temp dir and loads it with
the REAL load_dotenv: bare, it must pick the value up (the control);
isolated, it must not reach a constant or os.environ;
* structural - the master and both Flattrade loads must go through the
isolation, so a bare exec_module fails here instead of passing in CI and
misbehaving on the trading machine.
Negative-tested in BOTH conditions (no .env, and the 616-key throwaway .env):
reverting the master load to a bare patch.dict, reverting the Flattrade load to
a bare exec_module, dropping the load_dotenv stub, and dropping the environment
restore - all four caught, each by the guard meant for it; with the throwaway
.env the reverted fixes also bring back the original test failures. A control
rewording passes. The throwaway .env was removed every time.
Gates: master 620 OK, identical with and without a .env; 28
market-data-health; 1594 pytest; coverage report exits 0 under fail_under 73.0;
check_coverage_thresholds.py passed; ruff; mypy (80 files); compileall; bandit.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A guard from #181 fails on your machine — and only there
Running the gates on merged
mainin your checkout: 617 run, 1 failure —test_the_flag_is_set_for_the_load_only, one of the guards #181 added. It foundSL_HUNTING_ENABLED='true'where it had been unset before the master loaded.The loader isn't the cause. It sets the flag inside
patch.dict, which restores the environment when the load ends. The value comes back from the next import in the test module:flattrade_execution.pycallsload_dotenv()at import time, outside any patch, and re-readsDependencies/.env. Proven in isolation — importing that one module movesSL_HUNTING_ENABLEDfrom unset to'true'.CI and every worktree have no
.env, which is why the guard passed on #181's CI.The fix
Record the value the moment the master's load block ends — before that import — and have the guard (renamed
test_the_loader_puts_back_the_flag_it_set) compare that to the pre-load value. It now tests exactly what it claims, and doesn't care about.envloading elsewhere.Proof
Reproduced without copying any credentials — a throwaway
Dependencies/.envcontaining onlySL_HUNTING_ENABLED=true:.env(your condition).env(CI's condition)Negative-tested in both conditions: setting the flag for the whole process before the load, and setting it after the load instead of inside
patch.dict, are both caught; a control rewording passes. The throwaway.envwas removed.Gates: master 617 OK with and without the one-line
.env· 28 market-data-health · 1594 pytest · ruff · mypy (80 files) · compileall.Second commit: keep
Dependencies/.envout of the master suite entirelyThe first commit worked around the symptom. This removes the cause: on your machine the master suite ran against your
.env; CI, which has none, didn't.Two leaks, not one. The master,
dhan_executionandflattrade_executionall callload_dotenv()at import..env— flags and broker credentials — intoos.environfor every later test.patch.dict, which restoresos.environ— but itsload_dotenvstill ran, and still let.envvalues set the master's import-time constants.Measured without touching your real
.env— a throwaway.envholding the 616 key names fromenv.example, each set to a placeholder:os.environOf the 21: nine read the environment at run time (broker routing ×4, the live/paper gates, the per-strategy toggle), twelve depended on master constants set at import (websocket feed, strangle re-entry, order correlation). None needs a
.env— all pass without one — so none was relying on it; they were exposed to it. They passed on your box only because your values are sane.The fix is test-side only. Every module-level repository load goes through
_isolated_from_dotenv:patch.dictrestoresos.environexactly (carrying the values a load needs), anddotenv.load_dotenvis stubbed so the file isn't read at all. Production'sload_dotenvis unchanged; no default moves. Follows the precedent intest_index_fetch_construction.py.Guards — one of which holds in CI, where the others would be blind:
os.environsnapshotted before the first repo import and after the last; fails naming any variable added/removed/changed — names only, never values, since on your machine a value can be a credential;.envin a temp dir, loaded with the realload_dotenv— bare, it must pick the value up (the control); isolated, it must not;exec_modulefails here instead of passing CI and misbehaving on the trading box.Negative-tested in both conditions (no
.env, and the 616-key throwaway): reverting the master load, reverting the Flattrade load, dropping the stub, and dropping the restore — all four caught, each by the guard meant for it; with the throwaway.envthe reverted fixes also bring back the original failures. Control passes. The throwaway.envwas removed every time.Gates (this commit): master 620 OK, identical with and without a
.env· 28 market-data-health · 1594 pytest ·coverage reportexits 0 under the 73% floor · coverage policy passed · ruff · mypy · compileall · bandit.🤖 Generated with Claude Code