IPFS Layer

IPFS Layer

🚧 This wiki describes DevNet 2.0, soon to be launched. It follows the contracts and dincli on the develop branch; the currently running DevNet may still use older contracts and commands. Pre-launch discussion: #102.

The blockchain coordinates the network, but it never stores the heavy artifacts. Everything content-shaped in DIN lives on IPFS, and the contracts store and vote on CIDs (content identifiers) only:

• model weights — genesis model, clients' local models, T1/T2 aggregated models
• service files — the model owner's Python logic (model.py, client.py, auditor.py, aggregator.py, modelowner.py)
• manifests — each model's manifest.json
• test datasets — per-audit-batch evaluation data, encrypted so only that batch's auditors can read it
• contract ABIs — custom task-contract ABIs referenced from a manifest

Because a CID is a cryptographic hash of the content, on-chain CID votes double as integrity checks: two aggregators produced the same model if and only if they submitted the same CID.

One interface, three backends

All upload/retrieve traffic in dincli goes through a single abstraction (upload_to_ipfs / retrieve_from_ipfs in dincli/services/ipfs.py) rather than scattered HTTP calls. The backend is chosen by configuration — participants can switch providers without touching any workflow:

Provider	When to use	Setup
env (default)	you run your own IPFS node (kubo RPC) or any IPFS-compatible HTTP API	IPFS_API_URL_ADD / IPFS_API_URL_RETRIEVE in the project .env
filebase (recommended)	you want a managed, pinned backend without running a node	dincli system configure-ipfs --provider filebase --api-key <filebase_rpc_token>
custom	full control — any storage you can wrap in Python	--provider custom --service-path /abs/path/to/custom_ipfs.py

Check or change the active provider anytime:

dincli system configure-ipfs                     # show current config
dincli system configure-ipfs --provider env      # switch explicitly

Notes per provider

• env accepts an API root (http://127.0.0.1:5001/api/v0), full add/cat endpoints, or a retrieve URL template containing {cid}. If you run a local node, make sure artifacts stay pinned — a garbage-collected model breaks the round for everyone who needs it.
• filebase uploads through Filebase's IPFS RPC API and issues a pin request after every upload; the token is stored in the user-level dincli config (per-provider, ipfs_api_key_<provider>).
• custom modules must export two functions — upload_to_ipfs(file_path, msg=None) -> str (returns a non-empty CID) and retrieve_from_ipfs(cid, file_path) -> int | None (writes the artifact to file_path). That's the whole contract; anything satisfying it can back the network's storage.

Reading without a provider: public gateway fallback

Participants who only need to read artifacts can opt into a public gateway: set IPFS_PUBLIC_GATEWAY=1 (uses https://ipfs.io/ipfs) or to any http(s):// gateway URL. A configured IPFS_API_URL_RETRIEVE still takes precedence. Uploads always need a real provider — and dincli's own guidance steers uploaders to Filebase, since a casually-run local node is rarely reachable or pinned well enough for others to fetch from.

Integrity: CIDs are verified on download

For the env and filebase providers, every download is checked against the CID that was requested before it is used: dincli recomputes the CID locally (ipfs add -n, which needs the kubo binary and an initialized repo but no running daemon) and writes the file atomically only if it matches. If kubo isn't installed, the check is skipped with a warning. custom providers are responsible for their own integrity.

CID-based caching

Content addressing makes caching trivial: same CID, same bytes. dincli caches every fetched artifact in its cache directory keyed by CID, so services, models, and manifests are downloaded only when their CID actually changes — repeated GI phases don't re-fetch unchanged files.

Further reading

• IPFS Configuration Guide — full provider reference, env URL shapes (kubo RPC vs gateway), gateway fallback, CID verification, and migration notes (the legacy "ipfs node" value maps to env)
• Setup Guide — IPFS as part of first-time setup
• DIN CLI — the component all this traffic flows through