Skip to content

feat(indexer): implement DIN Protocol subgraph — platform + task-level contracts - #29

Open
robertocarlous wants to merge 18 commits into
InfiniteZeroFoundation:feat/din-indexerfrom
robertocarlous:feat/din-indexer
Open

robertocarlous wants to merge 18 commits into
InfiniteZeroFoundation:feat/din-indexerfrom
robertocarlous:feat/din-indexer

Conversation

@robertocarlous

@robertocarlous robertocarlous commented Jul 12, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Implements the full DIN Protocol subgraph covering both platform-level and
task-level contracts, along with GraphQL integration into the CLI.


P4-IDX1 — Design

  • Event coverage audit (subgraph/docs/event-coverage-audit.md) — per-contract
    event table, gap analysis, and three proposed contract-side additions
  • GraphQL entity schema (subgraph/schema.graphql) — 17 entities covering model
    registration, manifest updates, validators, stake/slash/withdrawal, slasher
    registrations, DIN mint events, and token transfers
  • Subgraph manifest (subgraph/subgraph.yaml) — 4 data sources, 31 handlers,
    network din-local (chainId 1337)
  • Daemon event schema (subgraph/docs/daemon-event-schema.md) — subscription
    priority table for dind, coverage gaps, WebSocket recommendation for P4-7.1

P4-IDX2 — Implementation

  • AssemblyScript mapping handlers (subgraph/src/) — one file per contract:
    registry.ts (15 handlers), staking.ts, coordinator.ts, token.ts,
    shared utils.ts. Storage call bridge used for pending event additions
  • Local Graph node stack (subgraph/docker-compose.yml) — graph-node v0.34.1
    • IPFS Kubo + Postgres 16 against local anvil; quickstart in compose header
  • Example queries (subgraph/docs/example-queries.md) — 14 queries across
    validator registry, model registry, slash history, and slasher audit trail

P4-IDX3 — CLI Integration

  • Replaced both RPC enumeration loops in dincli/cli/dindao.py with GraphQL
    queries; silent fallback to RPC on connection error, HTTP error, or GraphQL
    error body
  • Unit tests (tests/test_list_pending_requests.py) — 6 tests, no live node
    required
  • Handoff notes (subgraph/docs/handoff.md) — DIN_SUBGRAPH_URL config,
    remaining RPC loop candidates in lms.py and aggregation.py
  • Doc-correction pass (Developer/issues/indexer.md) — resolved all 5 open
    questions, confirmed no stale references, added artefacts table

Task-level contracts (DINTaskCoordinator / DINTaskAuditor)

Implemented following the contract event audit and Umer's design confirmation.

  • Contract event audit (Documentation/technical/audits/task-contract-event-audit.md)
    — existing events, 23-site silent GI state machine, missing submission and
    finalization events, and six proposed additions with exact signatures and emit sites
  • 13 new schema entities covering the full GI lifecycle: GlobalIteration,
    aggregator/auditor registrations, T1/T2 batches, audit batches, local model
    submissions, scoring, eligibility voting/finalization, slash events, pass score
    governance, and GI state transition history
  • Dynamic data source templates in subgraph.yaml — one
    DINTaskCoordinator + DINTaskAuditor pair instantiated per model when
    ModelApproved fires on DINModelRegistry
  • registry.ts updated to call createWithContext() with shared context
    (modelId, paired contract addresses) so handlers on either side resolve the
    model back-reference and paired address without extra storage calls
  • taskCoordinator.ts — 4 handlers: aggregator registration, T1/T2 batch
    creation, aggregator slashing
  • taskAuditor.ts — 8 handlers: auditor registration, audit batch creation,
    scoring, eligibility voting/finalization, pass score governance, auditor
    slashing. Includes loadOrCreateLMS() stub so required relationship fields
    are never null before LocalModelSubmitted event lands

Pending — blocked on contract event additions

Five handlers are commented out in subgraph.yaml pending the additions
proposed in the audit (§4). All schema fields and handler stubs are in place.

Event Contract Unblocks
GIStateChanged(uint GI, uint8 newState) DINTaskCoordinator Full GI state history
LocalModelSubmitted(uint GI, uint modelIndex, address client, bytes32 cid) DINTaskAuditor Client submission tracking
T1AggregationSubmitted / T2AggregationSubmitted DINTaskCoordinator Aggregation submission indexing
T1BatchFinalized / T2Finalized DINTaskCoordinator Winning CID and global model CID

Uncommenting the entries in subgraph.yaml and regenerating the ABI is all
that is needed once these events land in the contracts.

Audits all events emitted by DINModelRegistry, DinValidatorStake,
DinCoordinator, and DinToken against indexer reconstruction requirements.

Findings:
- ModelRegistrationRequested: missing isOpenSource and fee — cannot filter
  pending requests by type or surface fee amounts without a storage call
- ManifestUpdateRequested: missing requester address — cannot attribute
  pending updates to their submitter
- DinCoordinator.withdraw(): no event emitted — ETH treasury outflows are
  invisible to the indexer
- SlasherContractAdded/Removed: identical signatures on DinCoordinator and
  DinValidatorStake — resolved at schema level via sourceContract field,
  no contract change needed
- ValidatorJailed path is dead code (jailedUntil is never set); flagged
  for the P3-4.x slashing spec to ensure a jailing event lands when that
  mechanism is added

Proposed additions (confirmed with Umer, 2026-07-09 — land as one PR after
audit review):
  1. ModelRegistrationRequested: + bool isOpenSource, uint256 fee
  2. ManifestUpdateRequested: + address indexed requester
  3. DinCoordinator: new ETHTreasuryWithdrawn(address indexed to, uint256 amount)

Includes daemon event schema table (P4-7.1 feed) mapping platform events to
dind subscription requirements and coverage gaps.
…ph (P4-IDX1)

Defines all queryable entities for the DIN platform subgraph covering
DINModelRegistry, DinValidatorStake, DinCoordinator, and DinToken.

Entities:
  Registry:  ModelRegistrationRequest, Model, ManifestUpdateRequest,
             ModelKillSwitchEvent, FeeSnapshot, RegistryFeeWithdrawal,
             DAOAdminTransfer
  Staking:   Validator (aggregate), StakeEvent, SlashEvent, UnstakeRequest,
             WithdrawalClaim, SlasherRegistration
  Coordinator: DINMintEvent, TreasuryWithdrawal, ExchangeRateSnapshot
  Token:     TokenMintEvent, TokenTransferEvent

Design notes:
- SlasherRegistration carries sourceContract to keep DinCoordinator and
  DinValidatorStake event streams distinguishable (identical event signatures)
- ModelRegistrationRequest.isOpenSource / .feePaid and ManifestUpdateRequest
  .requester are marked pending the §6.1/6.2 event additions; mapping handlers
  will use storage calls as a bridge until those events land
- TreasuryWithdrawal entity is defined but will not be populated until the
  ETHTreasuryWithdrawn event addition (audit §6.3) lands
- Task-level contracts (DINTaskCoordinator/DINTaskAuditor) are out of scope;
  deferred to P5+ dynamic data sources
Defines the Graph Protocol subgraph manifest covering all four platform
contracts on the local development chain (chainId 1337, network: din-local).

Data sources:
  - DINModelRegistry  → src/registry.ts    (15 event handlers)
  - DinValidatorStake → src/staking.ts     (8 event handlers)
  - DinCoordinator    → src/coordinator.ts (5 event handlers + 1 commented-out
                                            pending ETHTreasuryWithdrawn addition)
  - DinToken          → src/token.ts       (2 event handlers)

ABIs copied from dincli/abis/ into subgraph/abis/ — single source of truth
until a build step syncs them automatically.

Contract addresses are placeholders (0x000...001–004); replace with actual
deployed addresses after running the platform contract deployment script.

SlasherContractAdded/Removed duplicate-signature handling: DinValidatorStake
handlers produce the canonical slasher-set records; DinCoordinator handlers
produce audit-trail records with sourceContract set to the coordinator address.
Defines which platform-contract events the dind daemon must subscribe to,
the reaction each event triggers in the daemon job queue, and all current
coverage gaps. Handoff document from P4-IDX1 design to Santiago's P4-7.1
on-chain event listening engine.

Contents:
  §1 Subscription priority — immediate (blacklist/kill-switch), high-priority
     (slash/approval/manifest), and informational buckets
  §2 Full event payload reference for all daemon-consumed events, including
     PENDING field annotations for the two confirmed additions (audit §6.1/6.2)
  §3 Coverage gaps:
     - Task-level GI events (T1AggregationStarted etc.) not in platform
       contracts — listed as P5+ task-level indexing requirement with the
       exact event schema needed for daemon filtering
     - ValidatorJailed dead code path — required event shape documented for
       when P3-4.x slashing spec adds the jailing mechanism
     - ETHTreasuryWithdrawn missing from DinCoordinator.withdraw()
  §4 Subscription mechanics recommendation for Santiago (WebSocket
     eth_subscribe, lastProcessedBlock per-contract, replay on restart)
  §5 Integration order — dincli → SDK extraction → dind, per Umer 2026-07-09
…ontracts

Five source files covering all 30 event handlers declared in subgraph.yaml.

src/utils.ts
  - eventId(): stable txHash-logIndex entity IDs for event-based records
  - loadOrCreateValidator(): initialises Validator aggregate on first encounter
  - syncValidatorStatus(): mirrors DinValidatorStake._syncValidatorStatus()
    to keep the Validator.status field in sync after every mutation

src/registry.ts  (DINModelRegistry — 15 handlers)
  - Registration request lifecycle: Requested → Approved / Rejected
  - Bridge for missing event fields: modelRequests(id) storage call populates
    isOpenSource, feePaid, manifestCID, taskCoordinator, taskAuditor until the
    audit §6.1/6.2 event additions land
  - Manifest update lifecycle: Requested → Updated / Rejected; requester
    derived from Model.owner as bridge until §6.2 addition lands
  - Kill switch: ModelDisabled / ModelEnabled update Model.disabled and emit
    ModelKillSwitchEvent
  - Fee snapshots: individual setters read full fee schedule from storage so
    every FeeSnapshot carries all four values; FeesUpdated reads from event
    params directly (atomic — no storage call needed)
  - FeesWithdrawn → RegistryFeeWithdrawal
  - DAOAdminUpdated → DAOAdminTransfer

src/staking.ts  (DinValidatorStake — 8 handlers)
  - ValidatorStaked / Slashed / UnstakeRequested / WithdrawalClaimed update the
    Validator aggregate and create the corresponding history entities
  - Slash handler mirrors contract's active-stake-first, then pending-withdrawal
    deduction logic for accurate aggregate tracking
  - Blacklisted / Unblacklisted flip Validator.blacklisted and re-sync status
  - SlasherContractAdded/Removed: canonical slasher-set records with
    sourceContract = DinValidatorStake address

src/coordinator.ts  (DinCoordinator — 5 handlers)
  - EthDepositAndDINminted → DINMintEvent
  - DinPerEthUpdated → ExchangeRateSnapshot
  - ValidatorStakeContractUpdated: intentional no-op (one-time deploy wiring)
  - SlasherContractAdded/Removed: audit-trail SlasherRegistration records with
    sourceContract = DinCoordinator address (distinct from staking.ts records)
  - ETHTreasuryWithdrawn handler stubbed and commented pending audit §6.3

src/token.ts  (DinToken — 2 handlers)
  - TokensMinted → TokenMintEvent
  - Transfer → TokenTransferEvent (covers mints from 0x0 and normal transfers)
Runs graph-node, IPFS (kubo), and Postgres against the local anvil chain
(chainId 1153 1337, network: din-local) for subgraph development and testing.

Services:
  graph-node  graphprotocol/graph-node:v0.34.1
    - ports 8000 (HTTP), 8001 (WS), 8020 (deploy), 8030 (status), 8040 (metrics)
    - ethereum env var maps din-local → host.docker.internal:8545
    - GRAPH_ETHEREUM_TARGET_TRIGGERS_PER_BLOCK_RANGE=1 for on-demand anvil blocks
  ipfs        ipfs/kubo:v0.27.0
    - port 5001 (API); used by graph-node and graph deploy
  postgres    postgres:16-alpine
    - POSTGRES_INITDB_ARGS sets UTF8/C locale required by graph-node TOAST

Both ipfs and postgres have healthchecks; graph-node depends_on postgres
with condition: service_healthy so it waits for the DB to be ready before
attempting to connect.

.env.example documents the ETHEREUM_RPC and Postgres credential overrides.
Linux users must replace host.docker.internal with 172.17.0.1.

Startup sequence is documented in the docker-compose.yml header comment.
Replace the two for-idx-in-range enumeration loops in
list-pending-requests with GraphQL queries to the local subgraph,
with a silent fallback to the original RPC loops when the
graph-node endpoint is unavailable.

Add unit tests covering the GraphQL happy path and all three
fallback triggers (connection error, GraphQL error body, HTTP 5xx).
Documents the integration point, GraphQL queries used, fallback
behaviour, DIN_SUBGRAPH_URL config, and follow-up RPC loop
candidates in lms.py and aggregation.py.
@umeradl umeradl added this to the Devnet 3.0 milestone Jul 12, 2026
@umeradl

umeradl commented Jul 13, 2026

Copy link
Copy Markdown
Member

@robertocarlous One correction and a note on what to work on next.

Correction — there is no Phase 5. The "deferred to P5+" framing in this PR description and in the earlier issue discussion was a mistake on my part: Developer/ROADMAP.md has no P5 — the roadmap currently runs through P3 (cryptoeconomic layer) and P4 (daemon + indexer). Please disregard the P5+ references.

Next step: until your next weekly task is assigned, continue working on the subgraph — extend it to the task-level contracts (DINTaskCoordinator / DINTaskAuditor), i.e. the dynamic data sources work previously labelled out of scope. Since task-level events follow the federated-learning lifecycle (Global Iterations, LMS, auditor scoring, two-tier aggregation, slashing), read Documentation/public/workflows/model-workflow.md first to build that context before designing the task-level entities. The lms.py and aggregation.py RPC-loop call sites listed in your handoff notes are the natural integration targets for this follow-up.

@umeradl umeradl mentioned this pull request Jul 13, 2026
13 tasks
… DINTaskAuditor templates

taskCoordinator.ts — four handlers covering aggregator registration,
T1/T2 batch creation, and aggregator slashing. GlobalIteration is
lazily instantiated on the first event per GI.

taskAuditor.ts — eight handlers covering auditor registration, audit
batch creation, scoring, eligibility voting/finalization, pass score
governance, and auditor slashing. Introduces loadOrCreateLMS() which
pre-creates a stub LocalModelSubmission (zero client/modelCID) on first
reference so the required relationship field on AuditScoreSubmission,
EligibilityVote, and EligibilityResult is never null at query time;
stubs are upgraded in-place once the LocalModelSubmitted event lands.
@robertocarlous robertocarlous changed the title Feat/din indexer feat(indexer): implement DIN Protocol subgraph — platform + task-level contracts Jul 14, 2026
@robertocarlous

Copy link
Copy Markdown
Collaborator Author

@umermjd11 Pls can you take a look at this, thank you

@umeradl umeradl added the Draft label Jul 14, 2026
@robertocarlous

robertocarlous commented Jul 30, 2026 •

Copy link
Copy Markdown
Collaborator Author

Hi @umeradl

I Reviewed the branch before it is merged. Three things worth resolving:

1. P3-staking gap (most impactful)

feat/p3-staking (PR #65) is in flight and adds six new events to DinValidatorStake:
ValidatorJailed, ValidatorReactivated, MinStakeUpdated, UnbondingPeriodUpdated, ModelStakeBoundsUpdated, MaxConcurrentRegistrationsPerStakeUnitUpdated. It also converts MIN_STAKE from a Solidity constant into a governable storage variable.

The indexer branch predates all of this. Concretely:

  • abis/DinValidatorStake.json is stale — none of the new functions or events are in the ABI
  • subgraph.yaml has no event handlers registered for any of the six new events
  • staking.ts doesn't import or process ValidatorJailed / ValidatorReactivated
  • syncValidatorStatus in utils.ts hardcodes MIN_STAKE = BigInt.fromString("10000000000000000000") — now that the value is governable, the indexed Validator.status will drift from on-chain truth whenever the DAO adjusts the floor
  • syncValidatorStatus has no Jailed branch — a jailed validator will be incorrectly computed as None or Active after reactivation
  • The schema has no JailEvent entity and Validator has no jailedUntil field

The event-coverage-audit (§2.2) itself flags this: "when a jailing mechanism is added, it must emit a ValidatorJailed event with the jailedUntil timestamp for the indexer to reconstruct status correctly." That moment has arrived with PR #65.

Suggested options:

  • If this PR is merged first, i can open a follow-up ticket to update the staking handler, ABI, schema, and syncValidatorStatus once PR Feat/p3 staking #65 lands — clear, sequential
  • Or coordinate a combined update so both land together — avoids a known-broken window in the indexer

Either way, the merge order needs to be explicit and the follow-up work tracked.

2. first: 1000 query limit

Both GraphQL queries in dincli/cli/dindao.py use first: 1000:

"{ modelRegistrationRequests(where: { processed: false }, first: 1000) ... }"
"{ manifestUpdateRequests(where: { processed: false }, first: 1000) ... }"

This silently truncates results beyond 1000 pending requests. For devnet scale this is fine, but the fallback to RPC enumeration would still return the full set — so the two paths diverge at scale. Either add pagination or add a comment documenting the known limit.

3. Missing update-subgraph-addresses.sh

subgraph.yaml references a helper script:

"A helper script (scripts/update-subgraph-addresses.sh) will patch this file automatically once P4-IDX2 setup tooling is in place."

The script doesn't exist on the branch. All four contract addresses in subgraph.yaml are still placeholders (0x000...0001 through 0x000...0004). A first-time developer following the README won't know what to do here. Either add the script now or replace the comment with a manual step in the setup docs.

@umeradl

umeradl commented Aug 3, 2026

Copy link
Copy Markdown
Member

Hey @robertocarlous — taking these in order:

1. P3-staking gap (PR #65) — merge order

Agreed this needs fixing before we merge as final, rather than shipping known-stale. Proposal: rather than either landing this as-is with a follow-up ticket, or blocking on #65 merging into develop first — can you branch off feat/din-indexer, merge #65's feat/p3-staking into it locally, and open a new PR with the follow-up (updated ABI, subgraph.yaml handlers for the six new events, staking.ts handling ValidatorJailed/ValidatorReactivated, syncValidatorStatus reading MIN_STAKE dynamically instead of hardcoded, and the missing JailEvent entity/jailedUntil field)? That gets this PR closed out cleanly without a stale window, and doesn't block on #65's own review/merge timeline into develop first. Naming's up to you — something like feat/din-indexer-65 works.

2. first: 1000 query limit

Agreed this is fine for devnet scale — go with your second suggestion: add a comment in dincli/cli/dindao.py next to both queries documenting the known limit (silently truncates beyond 1000 pending requests; RPC-fallback path doesn't share this cap) rather than building pagination now. Open a backlog item if/when devnet scale actually approaches it.

3. Missing update-subgraph-addresses.sh

Went ahead and wrote it rather than deferring — drop this in as subgraph/scripts/update-subgraph-addresses.sh:

#!/usr/bin/env bash
# update-subgraph-addresses.sh
# Patches subgraph/subgraph.yaml's placeholder platform-contract addresses
# with the real addresses from a deployments JSON (foundry or hardhat schema).
#
# Usage:
#   ./scripts/update-subgraph-addresses.sh [path/to/deployments.json]
#
# Defaults to foundry/deployments/localhost.json, falling back to
# hardhat/deployments/localhost.json if the first doesn't exist.

set -euo pipefail

SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
SUBGRAPH_YAML="$SCRIPT_DIR/../subgraph.yaml"
REPO_ROOT="$SCRIPT_DIR/../.."

DEPLOYMENTS_JSON="${1:-}"
if [[ -z "$DEPLOYMENTS_JSON" ]]; then
  if [[ -f "$REPO_ROOT/foundry/deployments/localhost.json" ]]; then
    DEPLOYMENTS_JSON="$REPO_ROOT/foundry/deployments/localhost.json"
  elif [[ -f "$REPO_ROOT/hardhat/deployments/localhost.json" ]]; then
    DEPLOYMENTS_JSON="$REPO_ROOT/hardhat/deployments/localhost.json"
  else
    echo "error: no deployments JSON found. Deploy the platform contracts first" >&2
    echo "  (foundry: forge script script/DeployPlatform.s.sol ...," >&2
    echo "   hardhat: npx hardhat run scripts/deploy.ts --network localhost)" >&2
    echo "or pass an explicit path: $0 <path/to/deployments.json>" >&2
    exit 1
  fi
fi

if ! command -v jq >/dev/null 2>&1; then
  echo "error: jq is required (https://jqlang.github.io/jq/)" >&2
  exit 1
fi

echo "Reading addresses from $DEPLOYMENTS_JSON"

# yaml "name:" value -> key in the deployments JSON
declare -A CONTRACT_TO_KEY=(
  [DINModelRegistry]=dinModelRegistry
  [DinValidatorStake]=dinValidatorStake
  [DinCoordinator]=dinCoordinator
  [DinToken]=dinToken
)

TMP="$(mktemp)"
cp "$SUBGRAPH_YAML" "$TMP"

for contract in "${!CONTRACT_TO_KEY[@]}"; do
  key="${CONTRACT_TO_KEY[$contract]}"
  address="$(jq -r --arg k "$key" '.[$k] // empty' "$DEPLOYMENTS_JSON")"
  if [[ -z "$address" ]]; then
    echo "error: key '$key' not found in $DEPLOYMENTS_JSON" >&2
    rm -f "$TMP"
    exit 1
  fi

  # Scope the replacement to this data source's block: find "name: <contract>",
  # then the next "address:" line after it, and patch only that one line.
  name_line="$(grep -n "name: $contract\$" "$TMP" | head -1 | cut -d: -f1)"
  if [[ -z "$name_line" ]]; then
    echo "warning: data source '$contract' not found in subgraph.yaml, skipping" >&2
    continue
  fi
  offset="$(tail -n "+$name_line" "$TMP" | grep -n "address:" | head -1 | cut -d: -f1)"
  address_line=$((name_line + offset - 1))

  sed -i.bak "${address_line}s|address: \".*\"|address: \"$address\"|" "$TMP"
  rm -f "$TMP.bak"
  echo "  $contract -> $address"
done

mv "$TMP" "$SUBGRAPH_YAML"
echo "Updated $SUBGRAPH_YAML"

Matches the existing dincli import-deployments/DeployPlatform.s.sol schema (dinCoordinator, dinModelRegistry, dinToken, dinValidatorStake, proxyAdmin keys — verified against an actual foundry/deployments/localhost.json on my end) and scopes each address replacement to its own name: block rather than a blind global find/replace, so ordering changes in subgraph.yaml don't break it. Only dependency is jq. Test it against a real local deploy before merging — I haven't run it end-to-end myself, only checked the JSON schema and the sed addressing logic by hand.

Thanks for catching all three — #1 is the one that actually needs a decision before this merges; #2 is fine landing as you suggested; #3 has a ready-to-drop-in script above.

@umermjd11

Copy link
Copy Markdown
Collaborator

Reviewed against feat/din-indexer (base, not develop) in an isolated worktree, head 866521a — 18 commits, GitHub agrees mergeable: MERGEABLE, mergeStateStatus: CLEAN. Ran the actual codegen pipeline, the new test file, and cross-checked entity/handler/event claims against the real schema, manifest, mappings, and the target Solidity contracts, rather than just reading the PR description.

Claimed: "4 data sources, 31 handlers" (P4-IDX1, platform-contract manifest).

Verified — exact match: subgraph/subgraph.yaml's dataSources: block has exactly 4 kind: ethereum sources (DINModelRegistry, DinValidatorStake, DinCoordinator, DinToken) and exactly 31 handler: entries. The two DINTaskCoordinator/DINTaskAuditor sources live separately under templates: (2 sources, 18 handlers) for the task-level dynamic-data-source addition — correctly kept out of the "4/31" count.

Claimed: "registry.ts (15 handlers)".

Verified — exact match: grep -c '^export function handle' on subgraph/src/registry.ts returns 15.

Claimed: "17 entities" (platform, P4-IDX1) / "13 new schema entities" (task-level, P4-IDX2).

Verified — off by a small margin, not a defect: schema.graphql actually defines 18 platform-level type ... @entity blocks (ModelRegistrationRequest → TokenTransferEvent) and 16 task-level ones (GlobalIteration → GIStateTransition) — 34 total vs. the 17+13=30 claimed in the PR body. Every entity referenced by the mapping code exists; the discrepancy looks like the PR description undercounting rather than missing/extra entities, worth a one-line fix to the description but not a merge blocker.

Claimed: ModelApproved instantiates the two task-level dynamic data source templates via a shared context.

Verified — real: registry.ts:108-115, handleModelApproved calls DINTaskCoordinatorTemplate.createWithContext(...) and DINTaskAuditorTemplate.createWithContext(...) with a shared DataSourceContext built from the resolved modelId, matching the PR description exactly.

Claimed: dincli/cli/dindao.py's RPC loops were replaced with GraphQL queries, "silent fallback to RPC on connection error, HTTP error, or GraphQL error body."

Verified — real, correctly implemented: _query_subgraph() (dindao.py:17-31) wraps the POST + raise_for_status() + explicit "errors" in body check in one broad try/except Exception: return None, and both call sites in list_pending_requests (model + manifest branches) check if gql_data is not None before falling back to the original on-chain RPC loop. This covers all three fallback triggers described.

Claimed: "Unit tests (tests/test_list_pending_requests.py) — 6 tests, no live node required."

Verified — false as written, all 6 fail: ran pytest tests/test_list_pending_requests.py -v against the repo's own pyproject.toml typer>=0.9.0 pin (tested with typer 0.20.0). All 6 tests raise AttributeError: 'function' object has no attribute 'callback' at the line that invokes dindao_module.list_pending_requests.callback(ctx, req_type="model"). This isn't a version-skew artifact — I confirmed directly that @app.command()-registered Typer functions are never given a .callback attribute (tested live: hasattr(foo, 'callback') → False on a minimal @app.command() function, independent of Typer version). Every other test file in this suite invokes commands through CliRunner (per CLAUDE.md's documented convention); this is the only file using the .callback pattern, and it doesn't exist. The underlying fallback logic itself is real and correct (see above) — only this test file's invocation mechanism is broken.

Claimed (implicit, P4-IDX1/IDX2): the subgraph manifest/schema/mappings are internally consistent and buildable.

Verified — false, codegen fails outright: npm install (no package-lock.json is committed, so npm ci isn't possible — worth noting separately) then npx graph codegen fails immediately with Invalid GraphQL schema: Syntax Error: Unterminated string at schema.graphql:412. Root cause: the GlobalIteration.currentState field's description uses a single-double-quoted GraphQL string ("...") spanning three lines (412-414) — GraphQL single-line description strings can't contain a literal newline; a multi-line description needs a triple-quoted block string ("""..."""), like every other multi-line description elsewhere in this same file. Confirmed this is the only such occurrence via a full-file scan. This is a one-character-class fix (swap "/" for """/""" around lines 412 and 414) but as committed, the subgraph does not build.

Not independently re-verified: the AssemblyScript mapping logic in staking.ts, coordinator.ts, token.ts, taskCoordinator.ts, taskAuditor.ts beyond the event-name cross-check below (no live graph-node instance to actually index against); the docker-compose stack (subgraph/docker-compose.yml) wasn't spun up. Event names used across all six mapping files (ModelApproved, ValidatorStaked, TokensMinted, Transfer, etc.) were cross-checked against event X( declarations in foundry/src/*.sol — all resolved except Transfer (inherited from ERC20Upgradeable, submodule not checked out in this worktree — standard OZ event, not a concern) and ETHTreasuryWithdrawn, which is explicitly commented out in coordinator.ts as a stub for a not-yet-added contract event (matches the PR's own event-coverage-audit.md §6.3 proposal) — not a real gap.


Every structural claim about the manifest/schema/mapping shape checked out exactly. Two real problems surfaced by actually running things: the schema has a GraphQL syntax error that breaks graph codegen/graph build outright (a real blocker, small fix), and the new test file's 6 tests all fail on a nonexistent .callback attribute (the code it's testing is fine; the test itself needs to switch to the CliRunner pattern the rest of the suite uses). Recommend fixing both before this lands on feat/din-indexer — everything else held up under direct verification.

This PR targets feat/din-indexer, not develop — that base branch is currently frozen at 805ce9d, 187 commits behind develop. PR #72 is stacked directly on top of this PR's head (72's head is 25 commits ahead of this PR's head, 0 behind — it already contains every commit here), so #29 needs to land on feat/din-indexer first, before #72.

@umermjd11

Copy link
Copy Markdown
Collaborator

Files changed (26) — as of 866521a (PR head)

Diffed against merge-base 805ce9d (feat/din-indexer's current tip — this PR is the base/first PR on this integration branch, so the merge-base is the branch tip itself, not develop; develop has moved 187 commits past 805ce9d in the meantime, but none of that drift touches these 26 files — all are net-new except dincli/cli/dindao.py). GitHub agrees: mergeable: MERGEABLE, mergeStateStatus: CLEAN. Confirmed with a local git diff/worktree checkout — clean, no conflicts.

(Table below is per-file, key/value, following the format from PR #63's review.)

subgraph/schema.graphql

Field Value
Change New
Lines +750/-0
Diff (what exactly is in this PR) 34 GraphQL @entity types: 18 platform-level (model registry, validator stake/slash/withdrawal, DIN mint/transfer, slasher registrations) + 16 task-level GI-lifecycle entities (GlobalIteration, aggregator/auditor registrations, T1/T2 batches, audit batches, LMS, scoring, eligibility, slash events, state transitions).
Functionality — how & why How: defines the indexable data model every mapping handler writes into. Why: the single source of truth for what the subgraph exposes over GraphQL to dincli and external consumers.
Diff vs current feat/din-indexer tip None — net-new file.
Recommended merge proposal Needs a fix before merge, not merged as-is: GlobalIteration.currentState's description (lines 412-414) is a single-double-quoted GraphQL string spanning 3 physical lines, which is invalid GraphQL syntax (multi-line descriptions require """ block strings). Confirmed by running npx graph codegen, which fails immediately with Invalid GraphQL schema: Syntax Error: Unterminated string at line 412. Full-file scan found no other occurrences.
Actual merge proposal Soon
Pending proposal Swap the offending "/" pair for """/""" around lines 412 and 414.
Local merge conflict No
GitHub merge conflict No

subgraph/subgraph.yaml

Field Value
Change New
Lines +341/-0
Diff (what exactly is in this PR) Manifest: 4 static dataSources (DINModelRegistry, DinValidatorStake, DinCoordinator, DinToken, 31 handlers total) + 2 templates (DINTaskCoordinator, DINTaskAuditor, 18 handlers total) for per-model dynamic instantiation, network din-local (chainId 1337).
Functionality — how & why How: wires each contract ABI + handler function to the events it listens for; the templates section lets ModelApproved spin up a fresh indexing context per model. Why: this is the actual indexing configuration graph-node consumes — everything else in subgraph/ is inert without it.
Diff vs current feat/din-indexer tip None — net-new file.
Recommended merge proposal Merged as-is — data source/handler counts verified exact against the PR's own claim (4 sources/31 handlers static, matches grep count exactly). Downstream of the schema.graphql fix above to actually build.
Actual merge proposal Soon
Pending proposal None.
Local merge conflict No
GitHub merge conflict No

subgraph/src/registry.ts

Field Value
Change New
Lines +306/-0
Diff (what exactly is in this PR) 15 exported handler functions for DINModelRegistry events (registration/approval/rejection, manifest updates, enable/disable, fee updates, DAO admin transfer); handleModelApproved additionally instantiates the two task-contract templates via createWithContext().
Functionality — how & why How: handleModelApproved loads the pending ModelRegistrationRequest, marks it processed, reads the full Model struct back from storage via getModel() (avoiding a second event for full field coverage), then creates a DataSourceContext carrying modelId and spins up DINTaskCoordinatorTemplate/DINTaskAuditorTemplate so the two per-model contracts start being indexed immediately. Why: task-level contracts are deployed per-model at approval time — the subgraph has no static address for them and must react by creating dynamic data sources at the moment ModelApproved fires.
Diff vs current feat/din-indexer tip None — net-new file.
Recommended merge proposal Merged as-is — createWithContext() call sites and event-name cross-check against foundry/src/DINModelRegistry.sol both verified directly.
Actual merge proposal Soon
Pending proposal None.
Local merge conflict No
GitHub merge conflict No

subgraph/src/staking.ts, subgraph/src/coordinator.ts, subgraph/src/token.ts, subgraph/src/taskCoordinator.ts, subgraph/src/taskAuditor.ts, subgraph/src/utils.ts

Field Value
Change New (6 files)
Lines +169/-0, +108/-0, +26/-0, +115/-0, +243/-0, +53/-0
Diff (what exactly is in this PR) Remaining per-contract mapping handlers (DinValidatorStake, DinCoordinator, DinToken) and the two task-level template handlers (DINTaskCoordinator, DINTaskAuditor), plus shared helpers in utils.ts.
Functionality — how & why How & why: same pattern as registry.ts above — one handler per contract event, writing/updating the corresponding schema entities. coordinator.ts includes a commented-out stub for ETHTreasuryWithdrawn, explicitly deferred because that event doesn't exist on-chain yet (tracked in this PR's own event-coverage-audit.md §6.3) — not a gap, an intentional placeholder.
Diff vs current feat/din-indexer tip None — all net-new files.
Recommended merge proposal Merged as-is — every event name used across these files (ValidatorStaked, TokensMinted, Transfer, etc.) resolved against real event X(...) declarations in foundry/src/*.sol (Transfer is the inherited standard ERC20Upgradeable event; OZ submodule wasn't checked out in this worktree so not independently re-confirmed there, but this is standard and not a concern).
Actual merge proposal Soon
Pending proposal None.
Local merge conflict No
GitHub merge conflict No

subgraph/abis/DINModelRegistry.json, DINTaskAuditor.json, DINTaskCoordinator.json, DinCoordinator.json, DinToken.json, DinValidatorStake.json

Field Value
Change New (6 files)
Lines +934/-0, +1221/-0, +1339/-0, +290/-0, +383/-0, +639/-0
Diff (what exactly is in this PR) Compiled ABI JSON for all 6 contracts the subgraph indexes, referenced by subgraph.yaml's dataSources/templates.
Functionality — how & why How: graph codegen reads these to generate the typed AssemblyScript bindings mapping files import from ../generated/.... Why: the subgraph can't decode event logs or make eth-call bindings without them.
Diff vs current feat/din-indexer tip None — net-new files.
Recommended merge proposal Merged as-is — not independently re-verified against the actual compiled foundry/out/*.json artifacts byte-for-byte (would require a full forge build, out of scope here), but codegen consumed them without complaint once the schema syntax error was worked around locally.
Actual merge proposal Soon
Pending proposal None.
Local merge conflict No
GitHub merge conflict No

subgraph/package.json

Field Value
Change New
Lines +16/-0
Diff (what exactly is in this PR) @graphprotocol/graph-cli@^0.71.0, @graphprotocol/graph-ts@^0.35.1, codegen/build/create:local/deploy:local/remove:local scripts.
Functionality — how & why How: defines the subgraph's own Node toolchain, separate from the repo root's Python tooling and from foundry//hardhat/'s own package.jsons. Why: graph-cli is what actually runs codegen/build/deploy against a local or hosted graph-node.
Diff vs current feat/din-indexer tip None — net-new file.
Recommended merge proposal Needs a fix, not merged as-is: no package-lock.json is committed, so npm ci (the command this repo's own CLAUDE.md/foundry convention relies on for reproducible installs) fails with EUSAGE. Had to fall back to plain npm install to verify anything downstream — that resolved 536 packages with 33 audited vulnerabilities (9 low/9 moderate/13 high/2 critical) against the unpinned ^0.71.0/^0.35.1 ranges.
Actual merge proposal Soon
Pending proposal Commit subgraph/package-lock.json from a real npm install run so builds are reproducible and npm ci works, matching the convention foundry/ already established.
Local merge conflict No
GitHub merge conflict No

subgraph/docker-compose.yml

Field Value
Change New
Lines +97/-0
Diff (what exactly is in this PR) Local graph-node v0.34.1 + IPFS Kubo + Postgres 16 stack targeting local anvil, with a quickstart comment block.
Functionality — how & why How: brings up the full local indexing stack the mapping handlers above get deployed into. Why: lets a developer test the subgraph end-to-end against a local chain without a hosted Graph node.
Diff vs current feat/din-indexer tip None — net-new file.
Recommended merge proposal Merged as-is — not independently spun up in this review (would require a running anvil instance + real block production; out of scope for a static-verification pass).
Actual merge proposal Soon
Pending proposal None.
Local merge conflict No
GitHub merge conflict No

subgraph/.env.example

Field Value
Change New
Lines +11/-0
Diff (what exactly is in this PR) Template env vars for the compose stack / CLI's DIN_SUBGRAPH_URL.
Functionality — how & why How & why: documents the env surface so a developer can copy to .env and fill in real values — same convention as the repo root's .env.example.
Diff vs current feat/din-indexer tip None — net-new file.
Recommended merge proposal Merged as-is.
Actual merge proposal Soon
Pending proposal None.
Local merge conflict No
GitHub merge conflict No

dincli/cli/dindao.py

Field Value
Change Modified
Lines +64/-12
Diff (what exactly is in this PR) Adds _query_subgraph() helper + _SUBGRAPH_URL; replaces the two on-chain enumeration loops in list_pending_requests (model + manifest branches) with a GraphQL query first, falling back to the original RPC loop only when the query returns None.
Functionality — how & why How: _query_subgraph() POSTs the query, calls raise_for_status(), explicitly checks for a GraphQL "errors" body, and catches everything in one broad except Exception: return None — so connection errors, HTTP errors, and GraphQL error bodies all degrade to the RPC path identically. Why: avoids an O(n) chain of eth_calls against totalModelRequests()/modelRequests(i) for every pending-request listing once a subgraph is available, while staying correct (same output) when one isn't.
Diff vs current feat/din-indexer tip None — the modified lines are the only change to this file since 805ce9d.
Recommended merge proposal Merged as-is — read the actual diff (not the PR's paraphrase); the fallback trigger set matches the PR's claim exactly.
Actual merge proposal Soon
Pending proposal None.
Local merge conflict No
GitHub merge conflict No

tests/test_list_pending_requests.py

Field Value
Change New
Lines +165/-0
Diff (what exactly is in this PR) 6 tests covering the GraphQL model/manifest paths and the 3 RPC-fallback triggers (connection error, GraphQL error body, HTTP error).
Functionality — how & why How: each test patches dindao_module._requests.post and invokes dindao_module.list_pending_requests.callback(ctx, req_type=...) directly. Why it fails: @registry_app.command(...)-decorated Typer functions are never given a .callback attribute — confirmed live against the repo's own typer>=0.9.0 pin (tested at 0.20.0) that a minimal @app.command() function has hasattr(fn, "callback") == False. Every other test file in this suite goes through CliRunner instead (the documented convention per CLAUDE.md); this file is the outlier.
Diff vs current feat/din-indexer tip None — net-new file.
Recommended merge proposal Needs a fix, not merged as-is: ran pytest tests/test_list_pending_requests.py -v — all 6 tests fail with AttributeError: 'function' object has no attribute 'callback'. The logic under test (verified above, in dindao.py) is correct; only the test's invocation mechanism is broken.
Actual merge proposal Soon
Pending proposal Rewrite the 6 tests to invoke via typer.testing.CliRunner (matching tests/test_dintoken.py/tests/test_ipfs_config.py) instead of a nonexistent .callback attribute.
Local merge conflict No
GitHub merge conflict No

Developer/issues/indexer.md

Field Value
Change Modified (doc-correction pass)
Lines +72/-0
Diff (what exactly is in this PR) Resolves all 5 open questions the doc previously flagged, per the PR description's P4-IDX3 doc-correction pass.
Functionality — how & why How & why: tracking doc, no runtime effect — closes out backlog questions now that the implementation answers them.
Diff vs current feat/din-indexer tip None — net-new content in an existing tracked doc.
Recommended merge proposal Merged as-is.
Actual merge proposal Soon
Pending proposal None.
Local merge conflict No
GitHub merge conflict No

Documentation/technical/audits/task-contract-event-audit.md

Field Value
Change New
Lines +232/-0
Diff (what exactly is in this PR) Event audit for DINTaskCoordinator/DINTaskAuditor: existing events, the 23-site silent GI state machine, missing submission/finalization events, six proposed additions with exact signatures/emit sites.
Functionality — how & why How & why: design/reference doc justifying the task-level schema entities and the [PENDING — audit §N.N] placeholders scattered through schema.graphql (e.g. the currentState field this review flags for its syntax bug references this same audit). No runtime effect.
Diff vs current feat/din-indexer tip None — net-new file.
Recommended merge proposal Merged as-is.
Actual merge proposal Soon
Pending proposal None.
Local merge conflict No
GitHub merge conflict No

subgraph/docs/daemon-event-schema.md, subgraph/docs/event-coverage-audit.md, subgraph/docs/example-queries.md, subgraph/docs/handoff.md

Field Value
Change New (4 files)
Lines +244/-0, +239/-0, +265/-0, +96/-0
Diff (what exactly is in this PR) Supporting docs: dind subscription-priority schema for P4-7.1, the platform-contract event-coverage audit (source of the ETHTreasuryWithdrawn stub noted above), 14 example GraphQL queries, and handoff notes (DIN_SUBGRAPH_URL config, remaining RPC-loop candidates in lms.py/aggregation.py).
Functionality — how & why How & why: reference/handoff material, no runtime effect. example-queries.md's 14 queries weren't independently executed against a live graph-node in this review (no running stack).
Diff vs current feat/din-indexer tip None — net-new files.
Recommended merge proposal Merged as-is.
Actual merge proposal Soon
Pending proposal None.
Local merge conflict No
GitHub merge conflict No

Verification

npm install (subgraph/, no lockfile — see package.json row) then npx graph codegen → fails on schema.graphql's GraphQL syntax error (see that file's row). pytest tests/test_list_pending_requests.py -v (repo's pyDIN venv, typer 0.20.0 against the typer>=0.9.0 pin) → 6 failed, 0 passed. Entity/handler/data-source counts cross-checked with grep/awk against the PR's own claims. Event names cross-checked against foundry/src/*.sol event declarations (OZ-submodule-inherited Transfer excepted — submodules weren't initialized in this worktree). Full detail in the deep-verification comment above.

Local vs. GitHub agree: GitHub's mergeable: MERGEABLE/mergeStateStatus: CLEAN is about git-level conflict-freedom against feat/din-indexer, which local git diff/checkout confirms — that's orthogonal to the two functional bugs above (GraphQL syntax error, broken test file), neither of which git conflict detection would ever catch.

umeradl added a commit that referenced this pull request Sep 24, 2026
…indexer events for Robbert

Three parts, three PRs into develop: Part A (#151, BL-11) future-block
dispute-subgroup seed; Part B (#152) platform-resolved treasury
forwarding through one burn/forward helper; Part C (#153) six
task-level lifecycle events, with the subgraph half landing on PR #29
after PR #72 is folded into it (Discussion #162).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Abidoyesimze

Copy link
Copy Markdown
Collaborator

Heads up for the uint8 → GIstates subgraph mapping: GIstates gained two more members (issue #156 M-1, task_240926_18 Part C — commit-then-reveal on T1/T2 aggregation submissions, mirroring LMSevaluationRevealStarted's insert-in-lifecycle-position precedent). Full updated ordinal table:

Value  State Name
─────  ──────────────────────────────────
  0    AwaitingDINTaskAuditorToBeSet
  1    AwaitingDINTaskCoordinatorAsSlasher
  2    AwaitingDINTaskAuditorAsSlasher
  3    AwaitingGenesisModel
  4    GenesisModelCreated
  5    GIstarted
  6    DINaggregatorsRegistrationStarted
  7    DINaggregatorsRegistrationClosed
  8    DINauditorsRegistrationStarted
  9    DINauditorsRegistrationClosed
 10    LMSstarted
 11    LMSclosed
 12    AuditorsBatchesCreated
 13    LMSevaluationStarted
 14    LMSevaluationRevealStarted
 15    LMSevaluationClosed
 16    T1nT2Bcreated
 17    T1AggregationStarted
 18    T1AggregationRevealStarted   ← new
 19    T1AggregationDone             (was 18)
 20    T2AggregationStarted          (was 19)
 21    T2AggregationRevealStarted   ← new
 22    T2AggregationDone             (was 20)
 23    AuditorsSlashed               (was 21)
 24    AggregatorsSlashed            (was 22)
 25    GIended                       (was 23)

DINTaskCoordinator.submitT1Aggregation/submitT2Aggregation are also gone — replaced by commitT1Aggregation/revealT1Aggregation and commitT2Aggregation/revealT2Aggregation. T1AggregationSubmitted/T2AggregationSubmitted now fire from the reveal call, not commit, with unchanged signatures. Two new events if useful for the subgraph: T1AggregationCommitted/T2AggregationCommitted(GI, batchId, aggregator, commitHash) — added speculatively; happy to drop them if the subgraph doesn't need commit-time visibility, just say so and I'll remove them in a follow-up.

PR incoming shortly (issue #156, task_240926_18 Part C) — will link it here once open. Not pushing to your branch, just flagging ahead of the regen per the task's instructions.

@umermjd11

Copy link
Copy Markdown
Collaborator

Heads-up: DINTaskCoordinator view-ABI change in PR #211

PR #211 (task_021026_19 Parts A+B, issues #201 Part A and #206) brings DINTaskCoordinator back under EIP-170. It removes 12 public getters and adds one new view:

  • Removed: t1SubmissionCID, t1Submitted, t1Votes, t1CommitHash, t1Committed, the same five for t2, and the tier1Batches/tier2Batches auto-getters.
  • Added: getAggregatorSubmission(gi, TierKind tier, batchId, aggregator) → (committed, commitHash, submitted, cid, votes). votes is the count for that aggregator's own revealed CID, and 0 before the reveal. getTier1Batch/getTier2Batch are unchanged.

No state-changing function, event, or storage slot changes. As far as I can see, this PR's mappings only .bind() DINModelRegistry and never call these getters, so no handler should break. Only the bundled DINTaskCoordinator ABI JSON goes stale. Once #211 lands, regenerate it from foundry/out/DINTaskCoordinator.sol/DINTaskCoordinator.json or copy dincli/abis/DINTaskCoordinator.json.

I haven't pushed to this branch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants