Summary
CI never runs the tests/dincli/ full-GI integration suite. The Python job runs pytest -m "not integration" -q (.github/workflows/ci.yml:136-137), and tests/test_integration_marker.py checks that exactly that directory is excluded. As a result the suite has drifted from the foundry contracts without anyone noticing. On develop @ 740a613 it can't pass:
- Phase 1: it asserts a
proxyAdmin key, but both deploy scripts now write per-contract proxyAdmin* keys (tests/dincli/test_01_platform.py:135-137,164).
- Task contracts: they are deployed from hardhat artifacts (
tests/dincli/constants.py:50). Those are the pre-foundry contracts, with no commit-reveal, rewards, seed lock or encrypted test data.
- The GI phase: it never funds the reward pool, registers auditor keys, or runs the three reveal phases. It also asserts
GIended as "23" (it's 25).
task-plan-051026-2 (PR #229, pending review) fixes the suite so it runs a complete GI on the foundry contracts. This issue tracks the follow-up: once the suite passes, run it in CI so it can't drift again.
Proposal
A separate integration job in ci.yml, kept apart from the fast Python job:
- Triggers: PRs and pushes to
develop that touch foundry/src/**, foundry/script/**, dincli/** or tests/dincli/**.
- Setup:
- Foundry, with
forge build plus npm ci in foundry/.
- Anvil via
foundry/anvil.sh. Its --block-time 2 is what the seed-lock polling relies on.
- An IPFS (kubo) daemon.
- Docker, for the sandboxed
din-worker containers that train-lms, evaluate and aggregate-* spawn.
- Python 3.12 with
pip install -e ..
- Run:
pytest tests/dincli/ -v -x -m integration --tb=short. Upload the dincli logs and the anvil output as artifacts when it fails.
- Rollout: non-blocking (
continue-on-error) at first. Make it a required check once it has been stable for a couple of weeks.
- Cost: a full GI trains 9 clients and runs auditor and aggregator jobs in containers. Measure the runtime once. If it's too slow per PR, run it on
push to develop and nightly instead, and run it on PRs only when a label is set.
Acceptance
- The
integration job runs the full-GI suite on anvil against the foundry contracts, through GIended.
- Failures upload logs.
Developer/CONTRIBUTING.md and Documentation/technical/testing/dincli-testing-guide.md describe the job and how to reproduce it locally.
Depends on
- task-plan-051026-2 (PR #229) TP-4: the suite has to pass a complete GI first.
Summary
CI never runs the
tests/dincli/full-GI integration suite. The Python job runspytest -m "not integration" -q(.github/workflows/ci.yml:136-137), andtests/test_integration_marker.pychecks that exactly that directory is excluded. As a result the suite has drifted from the foundry contracts without anyone noticing. Ondevelop@740a613it can't pass:proxyAdminkey, but both deploy scripts now write per-contractproxyAdmin*keys (tests/dincli/test_01_platform.py:135-137,164).tests/dincli/constants.py:50). Those are the pre-foundry contracts, with no commit-reveal, rewards, seed lock or encrypted test data.GIendedas "23" (it's 25).task-plan-051026-2 (PR #229, pending review) fixes the suite so it runs a complete GI on the foundry contracts. This issue tracks the follow-up: once the suite passes, run it in CI so it can't drift again.
Proposal
A separate
integrationjob inci.yml, kept apart from the fast Python job:developthat touchfoundry/src/**,foundry/script/**,dincli/**ortests/dincli/**.forge buildplusnpm ciinfoundry/.foundry/anvil.sh. Its--block-time 2is what the seed-lock polling relies on.din-workercontainers thattrain-lms,evaluateandaggregate-*spawn.pip install -e ..pytest tests/dincli/ -v -x -m integration --tb=short. Upload the dincli logs and the anvil output as artifacts when it fails.continue-on-error) at first. Make it a required check once it has been stable for a couple of weeks.pushtodevelopand nightly instead, and run it on PRs only when a label is set.Acceptance
integrationjob runs the full-GI suite on anvil against the foundry contracts, throughGIended.Developer/CONTRIBUTING.mdandDocumentation/technical/testing/dincli-testing-guide.mddescribe the job and how to reproduce it locally.Depends on