Skip to content

ci: run the tests/dincli full-GI integration suite in a separate CI job #228

Description

@umermjd11

Summary

CI never runs the tests/dincli/ full-GI integration suite. The Python job runs pytest -m "not integration" -q (.github/workflows/ci.yml:136-137), and tests/test_integration_marker.py checks that exactly that directory is excluded. As a result the suite has drifted from the foundry contracts without anyone noticing. On develop @ 740a613 it can't pass:

  • Phase 1: it asserts a proxyAdmin key, but both deploy scripts now write per-contract proxyAdmin* keys (tests/dincli/test_01_platform.py:135-137,164).
  • Task contracts: they are deployed from hardhat artifacts (tests/dincli/constants.py:50). Those are the pre-foundry contracts, with no commit-reveal, rewards, seed lock or encrypted test data.
  • The GI phase: it never funds the reward pool, registers auditor keys, or runs the three reveal phases. It also asserts GIended as "23" (it's 25).

task-plan-051026-2 (PR #229, pending review) fixes the suite so it runs a complete GI on the foundry contracts. This issue tracks the follow-up: once the suite passes, run it in CI so it can't drift again.

Proposal

A separate integration job in ci.yml, kept apart from the fast Python job:

  • Triggers: PRs and pushes to develop that touch foundry/src/**, foundry/script/**, dincli/** or tests/dincli/**.
  • Setup:
    • Foundry, with forge build plus npm ci in foundry/.
    • Anvil via foundry/anvil.sh. Its --block-time 2 is what the seed-lock polling relies on.
    • An IPFS (kubo) daemon.
    • Docker, for the sandboxed din-worker containers that train-lms, evaluate and aggregate-* spawn.
    • Python 3.12 with pip install -e ..
  • Run: pytest tests/dincli/ -v -x -m integration --tb=short. Upload the dincli logs and the anvil output as artifacts when it fails.
  • Rollout: non-blocking (continue-on-error) at first. Make it a required check once it has been stable for a couple of weeks.
  • Cost: a full GI trains 9 clients and runs auditor and aggregator jobs in containers. Measure the runtime once. If it's too slow per PR, run it on push to develop and nightly instead, and run it on PRs only when a label is set.

Acceptance

  • The integration job runs the full-GI suite on anvil against the foundry contracts, through GIended.
  • Failures upload logs.
  • Developer/CONTRIBUTING.md and Documentation/technical/testing/dincli-testing-guide.md describe the job and how to reproduce it locally.

Depends on

  • task-plan-051026-2 (PR #229) TP-4: the suite has to pass a complete GI first.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions