Filecoin storage support: which provider, given Lighthouse retrieval is payment-gated?

[umeradl]

Background

DIN currently uses Filebase as its default IPFS pinning provider. All model artifacts (client updates, aggregated models, manifests, service files) flow through dincli/services/ipfs.py and are referenced on-chain by CID only — the contract layer is fully provider-agnostic. Adding a Filecoin-backed provider therefore requires only an adapter behind the existing custom provider path: no contract changes, no CID format changes.

We want Filecoin-backed storage because:

• Decentralisation: a trust-minimised ML coordination network shouldn't depend on a single centralised pinning vendor.
• Workload fit: model weight updates are write-once, content-addressed, and referenced across training rounds — permanency and auditability matter more than ultra-low read latency.
• Cost: at Filecoin rates, a typical round (100 validators × 10 MB updates ≈ 1 GB) costs a fraction of a cent — effectively a rounding error in any fee structure.

The internal design doc (Developer/discussion/add-filecoin-support.md) recommended Lighthouse as the first integration, and PR #16 implemented a Lighthouse provider adapter.

What we found: Lighthouse retrieval is payment-gated ⚠️

While testing PR #16 with a real Lighthouse account (not just mocks):

• ✅ Upload works exactly as documented and returns a valid CID.
• ❌ Retrieval does not. GET https://gateway.lighthouse.storage/ipfs/<cid> returns 402 Payment Required — including for the file's own uploader, with and without the uploader's own API key attached (identical response either way, so this is not an auth-header issue).
• ❌ The content is not discoverable on the public IPFS network at all. A generic public gateway (ipfs.io) fails for the same CID with "no providers found for the CID".

This breaks DIN's core assumption: an artifact's CID must be retrievable by other participants (aggregators fetching client updates, auditors fetching models) — not just uploadable. A provider where even the uploader can't read their own file back without a paid retrieval plan is not viable as-is.

The question

How should we add Filecoin-backed storage, and which provider should we choose?

Candidates we're aware of:

Option	Pros	Concerns
Storacha (Web3.Storage)	UCAN delegated-upload capabilities are a first-class primitive — exactly the scoped credential model we need for sponsored uploads in P3; multi-provider Filecoin deals	Need to verify retrieval is genuinely open (public gateway + IPFS DHT announcement), not gated like Lighthouse
Lighthouse + paid retrieval plan	Adapter already written in PR #16	Retrieval gated behind payment; unclear if a plan makes CIDs publicly resolvable or only via their gateway
Filebase Filecoin bucket	Zero migration friction — same API and credential model we use today	Filebase remains the centralised intermediary; least decentralisation gain
Direct Filecoin deals / FVM	Maximum trustlessness; smart-contract-native storage deals (escrow, SLA verification, auto-renewal)	Deal lifecycle management is significant operational complexity; retrieval latency
Something else?	—	—

Specific questions:

1. Is Storacha's retrieval path actually open? Before we swap adapters, has anyone verified that content uploaded to Storacha resolves via public IPFS gateways / is announced to the DHT?
2. Is there any Filecoin-backed provider where uploaded CIDs are reliably retrievable by third parties for free (or where retrieval cost is bounded and payable by the model owner, not each reader)?
3. Should we reconsider an FVM-native approach — storage deals managed by contract (escrow, provider SLA verification, automated renewal) — even if it's more work, given it aligns with the P3 sponsored-upload/storage-budget design?
4. Is a hybrid sensible? Filecoin for durable archival + a hot IPFS pin (even Filebase) for the active round's retrieval path?

Any hands-on experience with these providers' retrieval behaviour — especially failure modes like the one we hit with Lighthouse — would be extremely valuable.

cc @Abidoyesimze — you've built on FVM (storage-deal escrow with SLA verification and auto-renewal), which is directly relevant here; would love your take on the provider choice and whether an FVM-native path is worth the complexity.
cc @Santiagocetran (PR #16 author)

[Abidoyesimze]

Hey, dug into this a bit.

1. Yeah, Storacha retrieval is just standard IPFS gateway stuff — storacha.link or any public gateway, no payment step. Content's actually discoverable on the wider IPFS network too, not just their infra. Only catch is their own gateway rate-limits at 200 req/min per IP, shouldn't matter for us.
2. Storacha Forge (their FOC-backed tier) is the one that actually works for us here — storage cost is on the uploader ($5.99/TB/month, PDP-backed), retrieval through open IPFS gateway is free for anyone. That's what we want: model owner pays, readers don't.
   Lighthouse's a dead end btw — their docs now openly describe it as "x402 pay-per-use retrieval," every fetch returns 402, you pay in USDC on Base, no exception even for the uploader. So what we hit in PR feat(validator-readiness): keystore security hardening + Filecoin (Lighthouse) provider adapter #16 wasn't a bug, that's just how it works now.
3. Bigger thing — Filecoin launched "Filecoin Onchain Cloud" in Nov 2025, and it's basically already built most of what we'd have hand-rolled: PDP proofs check storage every ~24h with a real penalty system (10%/25%/50% on repeated failures, cuts off after 10), Filecoin Pay streams payment and pauses automatically if proofs fail, and there's no fixed-term deal to renew — rails just keep going as long as they're funded.
   The part that actually matters for P3: Filecoin Pay has an "operator" concept built in — a sponsor deposits funds and approves a contract to manage rails on their behalf, capped by a rate + lockup allowance. That's basically a native storage-budget primitive already.
   So I don't think we need custom FVM escrow logic. But I also wouldn't just default to the hosted Synapse SDK flow if we want sponsor-controlled budgets — we'd want a small DIN contract that gets approved as the Filecoin Pay operator, so we keep our own sponsorship rules but inherit their proof/payment mechanics. Still real contract work, just way less than building escrow + SLA + renewal from scratch.
4. Given answer 2 and 3, I don't think we need the hybrid anymore — Storacha Forge already is that hybrid (UCAN uploads + open retrieval + Filecoin-backed durability in one product).

[Abidoyesimze]

Correction to my recommendation above: Storacha's infrastructure is down as of today, not just its marketing site.

• storacha.network / docs.storacha.network — Cloudflare Error 1000 ("DNS points to prohibited IP")
• console.storacha.network and up.storacha.network (the likely API host) — don't resolve at all (NXDOMAIN)
• storacha.link and w3s.link — the actual retrieval gateways — now redirect into the same broken storacha.network root

That's upload, auth, and retrieval, not a cosmetic frontend problem. Not building a recommendation on infra that's down.

Pivoting the recommendation to Filecoin Onchain Cloud — specifically Filecoin Pin (filecoin-pin CLI/library, built on FOC's Synapse SDK). Full writeup + integration guide: #28.

Short version:

• Retrieval: unauthenticated, same requirement that ruled out Lighthouse. CDN-accelerated through FilBeam by default, with egress cost drawn from the uploader's lockup — "model owner pays, readers don't" out of the box, no separate paid tier needed for it (unlike Storacha, where that required Forge specifically).
• Auth: no UCANs, but Session Keys turn out to be a close functional parallel — scoped permissions (storage ops only, not fund transfer), expiring, revocable, with a two-party session generate (CI side) → session authorize (owner side) flow that mirrors the Storacha CI delegation pattern almost exactly.
• For dincli: filecoin-pin implements the standard IPFS Pinning Service API directly (confirmed from the repo's own description) — the same shape the existing Filebase adapter likely targets, worth checking before assuming a rewrite. The server/daemon mode exposing that API is explicitly Beta/not-production per their own docs though; CLI/Action/library are the production-ready surfaces.

PR: #28

[umeradl]

Hey @Abidoyesimze ! Please look into how we can integrate Storacha (this is the rebranded version of web3.storage).

Specifically, I need you to figure out and document:

Web UI: How to set up accounts and manage data.

CLI: The workflow for uploading and retrieving data.

Authentication: How they handle "API keys"—heads up, they use decentralized UCANs instead of traditional static keys, so pay close attention to how we handle delegations for our backend/CI environment.

Let's get a quick markdown guide or snippet ready on this. Thanks!

[Abidoyesimze]

Okay will do that and get back.

[Abidoyesimze]

Hey @umeradl

Short version on the three things you asked about:

• Web UI: console.storacha.network — email → confirm link → plan selection. Payment method required even for free ("Mild", $0/mo, 5GB storage+egress) — it's in their ToS, not optional, so worth knowing before setup.
• CLI: storacha login → space create → up → retrieve via any public IPFS gateway (no credentials needed to read, matches what I found on the retrieval question). --json output for scripting.
• Auth: no static API keys at all — UCAN delegation instead. For our case (backend/CI, no interactive email step), the pattern is: generate a scoped Agent keypair (storacha key create), delegate it just the 4 upload capabilities it needs (storacha delegation create ... --base64), then drive it via STORACHA_PRINCIPAL + the proof. Full detail + the JS client equivalent in the guide.

One design note worth flagging early: auditors/aggregators reading artifacts back don't need any delegation at all, since retrieval is unauthenticated — only the upload path needs credentials. Minting one shared principal for every role would be the easy path but means a compromised reader-only node could also upload/overwrite; worth scoping per-role from the start.

PR: #26

[umeradl]

Storacha Web Access Down — Switching to CLI
Both Storacha endpoints are unreachable right now:
console.storacha.network — DNS NXDOMAIN (domain doesn't resolve)
storacha.network — Cloudflare Error 1000 ("DNS points to prohibited IP")
This is a DNS/Cloudflare configuration issue on their end, not a local problem. Since the web console and landing page are both broken, the only viable path forward is the CLI (w3).
Storacha (formerly web3.storage) is an active Protocol Labs project — their GitHub repos show recent commits through May 2026, so the service itself isn't dead, just the web frontends. The CLI uses UCAN-based auth and operates independently of these DNS records.

[umeradl]

I tried

storacha login you@example.com

but CLI is failing because it's trying to reach up.storacha.network — and that subdomain also doesn't resolve (ENOTFOUND). This confirms the DNS issue isn't limited to the web console and landing page; the entire storacha.network zone appears to be down or misconfigured right now. Please investigate the storacha network seems stale and not working

[Abidoyesimze]

Yeah will check with their team on discord

[Abidoyesimze]

Hi

So I have reach out to them.
Storacha is no longer operating it's now deprecated and merged into filecoin onchain cloud

[umeradl]

Lets explore filecoin onchain cloud web and cli

[Abidoyesimze]

Hey @umeradl

You can check my research on #28

[umermjd11]

Landing on Filecoin Onchain Cloud as the way forward for DIN's storage layer.

The case from the research above: PDP proofs give us real, ongoing storage verification (checked ~every 24h, escalating penalties on repeated failure, cutoff after 10) and Filecoin Pay's operator/allowance model gives us the sponsor-controlled storage budget primitive DIN needs — a sponsor deposits funds and approves a contract to manage rails on their behalf, capped by rate + lockup. That's the piece we'd otherwise have had to hand-roll as custom FVM escrow + SLA verification + renewal logic. No need to build that ourselves.

Storacha Forge is FOC-backed (PDP-backed storage, open IPFS gateway retrieval, uploader pays/readers free — exactly the cost split we want for model artifacts), so it's the practical integration path into FOC unless the DNS/console outage turns out to be more than transient. @Abidoyesimze, keep going on the CLI-based integration guide — flag here if the outage doesn't clear up and we need to evaluate going directly against the Synapse SDK / Filecoin Pay contracts instead of through Storacha.