Add model discovery integration tests with managed config - #655
Closed
andy-xu-db wants to merge 6 commits into
Closed
andy-xu-db wants to merge 6 commits into
andy-xu-db wants to merge 6 commits into
Conversation
andy-xu-db
changed the base branch from
main
to
codex/model-discovery-test-stack
September 16, 2026 08:44
andy-xu-db
force-pushed
the
andy/model-discovery-integration-tests
branch
from
September 16, 2026 15:03
b9006d2 to
1fda02d
Compare
andy-xu-db
changed the base branch from
codex/model-discovery-test-stack
to
andy/model-discovery-integration-unmanaged
September 16, 2026 15:03
andy-xu-db
force-pushed
the
andy/model-discovery-integration-unmanaged
branch
from
September 16, 2026 18:32
9751ad8 to
88b4d6f
Compare
andy-xu-db
force-pushed
the
andy/model-discovery-integration-tests
branch
from
September 16, 2026 18:32
1fda02d to
d895788
Compare
andy-xu-db
changed the base branch from
andy/model-discovery-integration-unmanaged
to
codex/model-discovery-test-stack
September 16, 2026 20:06
andy-xu-db
force-pushed
the
andy/model-discovery-integration-tests
branch
from
September 16, 2026 20:06
d895788 to
b7837b7
Compare
andy-xu-db
force-pushed
the
andy/model-discovery-integration-tests
branch
from
September 17, 2026 21:37
8cc5eb8 to
ecd7c40
Compare
andy-xu-db
force-pushed
the
codex/model-discovery-test-stack
branch
from
September 17, 2026 21:37
2c0b3df to
a3ee267
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
CodingAgentConfigonce per agent module, derive one MPS-backed test variant, and reuse that mode-0600 stub across every isolated test session.Managed workspace
The managed jobs use the workspace configured by the GitHub Actions secret
E2E_ADMIN_WORKSPACE. The runner mints a short-lived bearer withE2E_ADMIN_SP_CLIENT_IDandE2E_ADMIN_SP_CLIENT_SECRET; it does not select a developer Databricks profile or use a PAT.Each module calls
GET /api/ai-gateway/v2/coding-agent-configsonce and keeps the first config, matching production selection. The test variant changes only the selected agent's model source and removes its incompatible static defaults:main.default.ci_e2e_anthropic_mpsmain.default.ci_e2e_openai_mpsThe stub is outside the uploaded artifact root and reused through
UCODE_MANAGED_CONFIG_STUB.test_ug_configure_managed.pyremains un-stubbed so the real fetch/cache/wire contract is still covered.Cases
Each test function includes its Tests-table case number (
test_case_01_...throughtest_case_12_...).Stack
codex/model-discovery-test-stack(rebased on currentmain).Verification
76d42794a3061efe256f32dc3f5a6b5d7a644168.tests/test_integration_contract.py: 4 passed.The run's separate Claude agent-launch shard fails because
system.ai.claude-opus-5rejects Claude Code'sadvisor_20260301tool type. That path is outside this PR's managed-discovery files; aggregate checks inherit the same failure.