-
Notifications
You must be signed in to change notification settings - Fork 0
feat(ai): delegate interpretation to adaptive orchestration #88
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,123 @@ | ||
| name: Apply adaptive contextual-orchestrator default | ||
|
|
||
| on: | ||
| push: | ||
| branches: | ||
| - agent/adaptive-orchestrator-default | ||
| paths: | ||
| - scripts/stage_adaptive_orchestrator_default_test.py | ||
| - scripts/apply_adaptive_orchestrator_default.py | ||
| - .github/workflows/apply-adaptive-orchestrator-default.yml | ||
|
|
||
| permissions: | ||
| contents: write | ||
|
|
||
| concurrency: | ||
| group: apply-adaptive-orchestrator-default-${{ github.ref }} | ||
| cancel-in-progress: false | ||
|
|
||
| jobs: | ||
| red-green-verify: | ||
| runs-on: ubuntu-latest | ||
| timeout-minutes: 120 | ||
| steps: | ||
| - name: Checkout exact branch head | ||
| uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # actions/checkout@v4 | ||
| with: | ||
| fetch-depth: 0 | ||
| persist-credentials: true | ||
|
|
||
| - name: Set up Python | ||
| uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # actions/setup-python@v6 | ||
| with: | ||
| python-version: "3.12" | ||
|
|
||
| - name: Set up Node.js when present | ||
| if: hashFiles('package-lock.json') != '' | ||
| uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # actions/setup-node@v4 | ||
| with: | ||
| node-version: 22 | ||
| cache: npm | ||
|
|
||
| - name: Set up Rust when present | ||
| if: hashFiles('Cargo.toml') != '' | ||
| uses: dtolnay/rust-toolchain@4be7066ada62dd38de10e7b70166bc74ed198c30 | ||
|
|
||
| - name: Install Python dependencies | ||
| shell: bash | ||
| run: | | ||
| set -euo pipefail | ||
| python -m pip install --upgrade pip | ||
| if [ -f requirements/ci.txt ]; then | ||
| python -m pip install -r requirements/ci.txt | ||
| elif [ -f requirements-dev.txt ]; then | ||
| python -m pip install -r requirements-dev.txt | ||
| elif [ -f pyproject.toml ]; then | ||
| python -m pip install -e '.[dev]' || python -m pip install -e . | ||
| else | ||
| python -m pip install pytest | ||
| fi | ||
|
|
||
| - name: Install Node dependencies when present | ||
| if: hashFiles('package-lock.json') != '' | ||
| run: npm ci | ||
|
|
||
| - name: Stage the repository contract | ||
| run: python scripts/stage_adaptive_orchestrator_default_test.py | ||
|
|
||
| - name: Establish red state or recognize an already-applied source change | ||
| id: red | ||
| shell: bash | ||
| run: | | ||
| set -euo pipefail | ||
| set +e | ||
| python tests/test_contextual_orchestrator_default_policy.py > /tmp/tepp-adaptive-red.log 2>&1 | ||
| status=$? | ||
| set -e | ||
| cat /tmp/tepp-adaptive-red.log | ||
| if [ "$status" -eq 0 ]; then | ||
| echo "already_applied=true" >> "$GITHUB_OUTPUT" | ||
| else | ||
| grep -Eq "FAILED|AssertionError|no production" /tmp/tepp-adaptive-red.log | ||
| echo "already_applied=false" >> "$GITHUB_OUTPUT" | ||
| fi | ||
|
|
||
| - name: Apply production request and documentation changes | ||
| run: python scripts/apply_adaptive_orchestrator_default.py | ||
|
|
||
| - name: Prove the focused contract is green | ||
| run: python tests/test_contextual_orchestrator_default_policy.py | ||
|
|
||
| - name: Run Python tests when configured | ||
| if: hashFiles('pyproject.toml', 'pytest.ini', 'setup.cfg') != '' | ||
| run: python -m pytest -q | ||
|
|
||
| - name: Run Node tests when configured | ||
| if: hashFiles('package-lock.json') != '' | ||
| run: npm test | ||
|
|
||
| - name: Run Rust tests when configured | ||
| if: hashFiles('Cargo.toml') != '' | ||
| run: cargo test --workspace | ||
|
|
||
| - name: Verify syntax and patch integrity | ||
| shell: bash | ||
| run: | | ||
| python -m compileall -q scripts tests | ||
| git diff --check | ||
|
|
||
| - name: Publish the verified source commit | ||
| env: | ||
| TARGET_BRANCH: agent/adaptive-orchestrator-default | ||
| shell: bash | ||
| run: | | ||
| set -euo pipefail | ||
| rm scripts/stage_adaptive_orchestrator_default_test.py | ||
| rm scripts/apply_adaptive_orchestrator_default.py | ||
| rm .github/workflows/apply-adaptive-orchestrator-default.yml | ||
| git config user.name "github-actions[bot]" | ||
| git config user.email "41898282+github-actions[bot]@users.noreply.github.com" | ||
| git add --all | ||
| git diff --cached --check | ||
| git commit -m "feat(ai): delegate interpretation to adaptive orchestration" | ||
|
Comment on lines
+115
to
+122
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🔒 Security & Privacy | 🟠 Major | ⚡ Quick win 게시할 파일을 allowlist로 스테이징하십시오. Line 120의 변경된 생산 소스, ADR, CHANGELOG, 정책 테스트, 의도한 삭제 파일만 스테이징하십시오. 커밋 전에 allowlist 밖의 staged path가 있으면 실패하십시오. As per coding guidelines, 릴리스에는 clean integration state와 reproducible artifacts가 필요합니다. 🤖 Prompt for AI AgentsSource: Coding guidelines |
||
| git push origin "HEAD:${TARGET_BRANCH}" | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,135 @@ | ||
| #!/usr/bin/env python3 | ||
| """Migrate TEPP production contextual-orchestrator calls to explicit auto.""" | ||
|
|
||
| from __future__ import annotations | ||
|
|
||
| import re | ||
| from pathlib import Path | ||
|
|
||
| ROOT = Path(__file__).resolve().parents[1] | ||
| SOURCE_SUFFIXES = {".py", ".js", ".mjs", ".cjs", ".ts", ".tsx", ".rs", ".go"} | ||
| EXCLUDED = { | ||
| ".git", ".github", "docs", "examples", "fixtures", "migrations", | ||
| "node_modules", "scripts", "target", "test", "tests", "vendor", | ||
| } | ||
| REPLACEMENTS = ( | ||
| ('"mode": "route"', '"mode": "auto"'), | ||
| ("'mode': 'route'", "'mode': 'auto'"), | ||
| ('"orchestration_mode": "route"', '"orchestration_mode": "auto"'), | ||
| ("'orchestration_mode': 'route'", "'orchestration_mode': 'auto'"), | ||
| ('mode="route"', 'mode="auto"'), | ||
| ("mode='route'", "mode='auto'"), | ||
| ('mode: str = "route"', 'mode: str = "auto"'), | ||
| ("mode: str = 'route'", "mode: str = 'auto'"), | ||
| ) | ||
| AUTO_RE = re.compile( | ||
| r"(?:orchestration_mode|mode)(?:\s*:\s*str)?\s*[:=]\s*[\"']auto[\"']", | ||
| re.IGNORECASE, | ||
| ) | ||
|
|
||
| integrations: list[Path] = [] | ||
| for path in sorted(ROOT.rglob("*")): | ||
| if not path.is_file() or path.suffix.lower() not in SOURCE_SUFFIXES: | ||
| continue | ||
| relative = path.relative_to(ROOT) | ||
| if {part.lower() for part in relative.parts} & EXCLUDED: | ||
| continue | ||
| source = path.read_text(encoding="utf-8") | ||
| lowered = source.lower() | ||
| if "contextual-orchestrator" not in lowered and "contextual_orchestrator" not in lowered: | ||
| continue | ||
| integrations.append(path) | ||
| updated = source | ||
| for old, new in REPLACEMENTS: | ||
| updated = updated.replace(old, new) | ||
| if ( | ||
| "chat/completions" in updated.lower() | ||
| and "contextual-orchestrator" in updated.lower() | ||
| and not AUTO_RE.search(updated) | ||
| ): | ||
| candidates = [ | ||
| ( | ||
| r'(?P<indent>[ \t]*)(?P<q>["\'])model(?P=q)\s*:\s*(?P<v>[^,\n}]+),\s*\n(?P=indent)(?P<mq>["\'])messages(?P=mq)\s*:', | ||
| lambda m: ( | ||
| f"{m.group('indent')}{m.group('q')}model{m.group('q')}:{m.group('v')},\n" | ||
| f"{m.group('indent')}{m.group('q')}orchestration_mode{m.group('q')}: {m.group('q')}auto{m.group('q')},\n" | ||
| f"{m.group('indent')}{m.group('mq')}messages{m.group('mq')}:" | ||
| ), | ||
| ), | ||
| ( | ||
| r'(?P<p>["\']model["\']\s*:\s*[^,}]+,\s*)(?P<m>["\']messages["\']\s*:)', | ||
| lambda m: f"{m.group('p')}\"orchestration_mode\": \"auto\", {m.group('m')}", | ||
| ), | ||
| ( | ||
| r'(?P<i>[ \t]*)model=(?P<v>[^,\n)]+),\s*\n(?P=i)messages=', | ||
| lambda m: ( | ||
| f"{m.group('i')}model={m.group('v')},\n" | ||
| f"{m.group('i')}extra_body={{\"orchestration_mode\": \"auto\"}},\n" | ||
| f"{m.group('i')}messages=" | ||
| ), | ||
| ), | ||
| ] | ||
| for pattern, replacement in candidates: | ||
| updated, count = re.subn(pattern, replacement, updated, count=1) | ||
| if count == 1: | ||
| break | ||
| else: | ||
| raise RuntimeError(f"could not locate request payload in {relative}") | ||
|
Comment on lines
+45
to
+77
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift 범위와 검증을 실제 요청 단위로 제한하십시오. 현재 구현은 파일 전체의 문자열과 📍 Affects 1 file
🤖 Prompt for AI Agents |
||
| if updated != source: | ||
| path.write_text(updated, encoding="utf-8") | ||
|
|
||
| if not integrations: | ||
| raise RuntimeError("no production contextual-orchestrator integration was found") | ||
|
|
||
| adr = ROOT / "docs" / "adr" / "0025-adaptive-contextual-orchestrator-default.md" | ||
| adr.parent.mkdir(parents=True, exist_ok=True) | ||
| if not adr.exists(): | ||
| adr.write_text( | ||
| '''# ADR-0025: TEPP interpretation delegates execution to contextual-orchestrator auto | ||
|
|
||
| - Status: Accepted | ||
| - Date: 2026-08-16 | ||
|
|
||
| ## Context | ||
|
|
||
| TEPP's multilingual, temporal, multilevel topic interpretation requires different | ||
| amounts of test-time computation across extraction, labeling, comparison, and | ||
| high-uncertainty synthesis. A consumer-owned `route` or implicit request mode forces | ||
| one worker and duplicates provider/workflow policy inside the psychometric product. | ||
|
|
||
| ## Decision | ||
|
|
||
| Every production contextual-orchestrator request explicitly selects `auto`. | ||
| Contextual-orchestrator owns route/verify/conduct selection, provider/model choice, | ||
| failover, and known-cost tie-breaks. Quality and safety requirements precede cost; | ||
| unknown price metadata is not treated as zero. | ||
|
|
||
| TEPP retains semantic-unit multilingual input construction, temporal and | ||
| multilevel/multiple-membership provenance, strict output validation, statistical | ||
| model ownership, and Rust/GPU numeric computation. Fixed orchestration modes remain | ||
| controlled evaluation fixtures only. | ||
|
|
||
| ## References | ||
|
|
||
| Omidvar, H., & Akhlaghi, V. (2026). *A communication-theoretic framework for LLM agents: Cost-aware adaptive reliability* [Preprint]. arXiv. https://doi.org/10.48550/arXiv.2605.09121 | ||
|
|
||
| Tang, Y., Cetin, E., Xu, J., Sun, Q., Nielsen, S., Richard, V., Goda, H., Tymchenko, I., Nguyen, N., Lee, H., Ashiga, M., Kotyan, S., Kuroki, S., & Clanuwat, T. (2026). *Sakana Fugu technical report* [Technical report]. arXiv. https://doi.org/10.48550/arXiv.2606.21228 | ||
| ''', | ||
| encoding="utf-8", | ||
| ) | ||
|
|
||
| changelog_path = ROOT / "CHANGELOG.md" | ||
| if changelog_path.exists(): | ||
| text = changelog_path.read_text(encoding="utf-8") | ||
| entry = ( | ||
| "- Production interpretation requests now explicitly use contextual-orchestrator " | ||
| "`auto` instead of a single-model route default.\n" | ||
| ) | ||
| if entry not in text: | ||
| marker = "## Unreleased\n" | ||
| text = ( | ||
| text.replace(marker, marker + "\n### Changed\n\n" + entry, 1) | ||
| if marker in text | ||
| else "## Unreleased\n\n### Changed\n\n" + entry + "\n" + text | ||
| ) | ||
| changelog_path.write_text(text, encoding="utf-8") | ||
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -0,0 +1,63 @@ | ||||||
| #!/usr/bin/env python3 | ||||||
| """Stage TEPP's production adaptive-orchestration contract.""" | ||||||
|
|
||||||
| from pathlib import Path | ||||||
|
|
||||||
| root = Path(__file__).resolve().parents[1] | ||||||
| test_path = root / "tests" / "test_contextual_orchestrator_default_policy.py" | ||||||
| content = '''"""TEPP production contextual-orchestrator requests explicitly use auto.""" | ||||||
|
|
||||||
| from __future__ import annotations | ||||||
|
|
||||||
| import re | ||||||
| import unittest | ||||||
| from pathlib import Path | ||||||
|
|
||||||
| SOURCE_SUFFIXES = {".py", ".js", ".mjs", ".ts", ".tsx", ".rs", ".go"} | ||||||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
수정 예시-SOURCE_SUFFIXES = {".py", ".js", ".mjs", ".ts", ".tsx", ".rs", ".go"}
+SOURCE_SUFFIXES = {".py", ".js", ".mjs", ".cjs", ".ts", ".tsx", ".rs", ".go"}📝 Committable suggestion
Suggested change
🤖 Prompt for AI Agents |
||||||
| EXCLUDED = { | ||||||
| ".git", ".github", "docs", "examples", "fixtures", "migrations", | ||||||
| "node_modules", "scripts", "target", "test", "tests", "vendor", | ||||||
| } | ||||||
| FORCED_ROUTE = re.compile( | ||||||
| r"(?:orchestration_mode|mode)(?:\\s*:\\s*str)?\\s*[:=]\\s*[\\\"']route[\\\"']", | ||||||
| re.IGNORECASE, | ||||||
| ) | ||||||
| AUTO = re.compile( | ||||||
| r"(?:orchestration_mode|mode)(?:\\s*:\\s*str)?\\s*[:=]\\s*[\\\"']auto[\\\"']", | ||||||
| re.IGNORECASE, | ||||||
| ) | ||||||
|
|
||||||
|
|
||||||
| class AdaptiveOrchestratorDefaultTest(unittest.TestCase): | ||||||
| def test_production_integrations_are_explicitly_adaptive(self) -> None: | ||||||
| root = Path(__file__).resolve().parents[1] | ||||||
| integration_files: list[str] = [] | ||||||
| violations: list[str] = [] | ||||||
| for path in sorted(root.rglob("*")): | ||||||
| if not path.is_file() or path.suffix.lower() not in SOURCE_SUFFIXES: | ||||||
| continue | ||||||
| relative = path.relative_to(root) | ||||||
| if {part.lower() for part in relative.parts} & EXCLUDED: | ||||||
| continue | ||||||
| text = path.read_text(encoding="utf-8") | ||||||
| lowered = text.lower() | ||||||
| if "contextual-orchestrator" not in lowered and "contextual_orchestrator" not in lowered: | ||||||
| continue | ||||||
| integration_files.append(relative.as_posix()) | ||||||
| if FORCED_ROUTE.search(text): | ||||||
| violations.append(f"{relative}: forced route") | ||||||
| if "chat/completions" in lowered and "contextual-orchestrator" in lowered and not AUTO.search(text): | ||||||
| violations.append(f"{relative}: implicit mode") | ||||||
| self.assertTrue(integration_files, "no production contextual-orchestrator integration was found") | ||||||
| self.assertEqual(violations, []) | ||||||
|
|
||||||
|
|
||||||
| if __name__ == "__main__": | ||||||
| unittest.main() | ||||||
| ''' | ||||||
| if test_path.exists(): | ||||||
| if test_path.read_text(encoding="utf-8") != content: | ||||||
| raise SystemExit(f"refusing to replace a different test: {test_path}") | ||||||
| else: | ||||||
| test_path.parent.mkdir(parents=True, exist_ok=True) | ||||||
| test_path.write_text(content, encoding="utf-8") | ||||||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy lift
게시 전에 과학적 승인 증거를 강제하십시오.
이 워크플로는 정책 테스트와 일반 언어별 테스트 후 즉시 Line 109의 게시 단계로 이동합니다. parameter recovery, RMSE, bias, interval coverage, temporal ordering, graph recovery, invariance, CPU/GPU parity에 대한 승인 명령과 재현 가능한 산출물 검증이 없습니다.
게시 단계 전에 변경된 orchestration capability에 대한 과학적 승인 작업을 실행하고, 산출물과 exact-head CI/security 증거가 없으면 실패하십시오. As per coding guidelines, “Scientific acceptance requires realistic synthetic truth: parameter recovery, RMSE, bias, interval coverage, temporal ordering, graph recovery, invariance, and CPU/GPU parity” 및 릴리스 증거가 필요합니다.
🤖 Prompt for AI Agents
Source: Coding guidelines