Skip to content

fix(gpu): forward -e env and --project on governed Forge/Modal runs - #652

Open
jqueguiner wants to merge 1 commit into
mainfrom
fix/governed-engine-env
Open

fix(gpu): forward -e env and --project on governed Forge/Modal runs#652
jqueguiner wants to merge 1 commit into
mainfrom
fix/governed-engine-env

Conversation

@jqueguiner

Copy link
Copy Markdown
Owner

Bug

openrunner gpu run --provider forge -e KEY=VALUE app.py silently drops the environment.

run_engine_serverside_sync() takes no env parameter, so nothing the caller passes with -e reaches the engine client. An App whose local_entrypoint reads the environment dies immediately:

$ openrunner gpu run --provider forge train.py -p gladia/openpii-medical \
    -e OPENRUNNER_API_KEY=or_... -e OPENMED_DATASET=gladia/openmed-ner-1m
[openrunner] launching forge server-side (token stays in OpenRunner; nothing runs on this machine) ...
KeyError: 'OPENRUNNER_API_KEY'

The same command with a client-side FORGE_TOKEN works — run_forge_sync() does forward env=env_dict. So the governed path, which the docs recommend, is the one that breaks, and it fails with no hint that -e was ignored.

--project had the same shape of bug: accepted by the function signature, never used.

Fix

Plumb env end-to-end: CLI → run_engine_serverside_syncengine_run_streamPOST /gpu/engine/run → engine-executor → the engine-client subprocess. --project is passed along as OPENRUNNER_PROJECT (an explicit -e OPENRUNNER_PROJECT=... still wins).

Security

Caller env is untrusted input that ends up next to a decrypted org credential, so it is constrained on both sides:

  • API validates it — POSIX name syntax, 64 vars max, 8 KiB per value, no NUL — and rejects reserved names with 422: engine tokens (FORGE_TOKEN, FORGE_API_URL, MODAL_TOKEN_*), EXECUTOR_SECRET, and loader/interpreter hooks (LD_PRELOAD, LD_LIBRARY_PATH, LD_AUDIT, PATH, PYTHONPATH, PYTHONSTARTUP, PYTHONHOME, BASH_ENV, ENV, IFS). Denial is case-insensitive.
  • Executor merges caller env under spec["env"](creds), so credentials always win even if validation is ever bypassed, and drops EXECUTOR_SECRET from the child environment.

FORGE_API_URL matters most here: without the deny + ordering, a caller could point the forge client at their own control plane and capture the org's token — the exact thing the governed path exists to prevent.

Tests

  • sdk/tests/test_gpu/test_engine_serverside.py — env + project forwarded, explicit OPENRUNNER_PROJECT wins, empty env when none given.
  • src/api/tests/test_gpu_engine_env.py (new, 21 cases) — happy path, every reserved name, bad names, limits, NUL.
sdk:      263 passed
src/api:  67 passed (existing gpu suites) + 21 new

`openrunner gpu run --provider forge -e KEY=VALUE app.py` silently dropped
the environment: run_engine_serverside_sync() took no env at all, so an App
whose local_entrypoint read os.environ["OPENRUNNER_API_KEY"] died with
KeyError on the governed path while the same command with a client-side
FORGE_TOKEN worked. --project was accepted and ignored the same way.

Plumb env end-to-end (CLI -> SDK runner -> API /gpu/engine/run -> executor)
and pass --project along as OPENRUNNER_PROJECT for run attribution.

Caller env is untrusted, so:
- the API validates it (POSIX name syntax, 64 vars, 8 KiB per value, no NUL)
  and rejects reserved names (engine tokens, EXECUTOR_SECRET, loader and
  interpreter hooks such as LD_PRELOAD/PYTHONPATH/PATH/BASH_ENV);
- the executor merges it *under* the org's credentials, so it can never swap
  the engine token or aim the engine client at another control plane, and it
  drops EXECUTOR_SECRET from the child environment.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant