Skip to content

docs: add wallet roles, upload config, and startup errors to migration guide #115

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
vilenarios merged 1 commit into from
Jun 8, 2026
Merged

docs: add wallet roles, upload config, and startup errors to migration guide #115

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 19 additions & 2 deletions content/build/run-a-gateway/manage/environment-variables.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -179,7 +179,9 @@ The gateway talks to four Solana programs that together implement the ar.io prot
| -------------------------- | ------ | ------------------------------------ | -------------------------------------------------------------------------------------------------------------------- |
| `AR_IO_WALLET` | string | - | Operator Solana public key (base58). Display label surfaced on `/ar-io/info` |
| `SOLANA_RPC_URL` | string | `https://api.mainnet-beta.solana.com`| Solana JSON-RPC endpoint. Public defaults throttle hard — use a premium provider (QuickNode, Helius, Triton) in production |
| `SOLANA_KEYPAIR_PATH` | string | - | Path to the operator's 64-byte Solana keypair JSON file (signs cranker instructions when `ENABLE_EPOCH_CRANKING=true`) |
| `SOLANA_KEYPAIR_PATH` | string | - | Path to the operator's 64-byte Solana keypair JSON file. Signs `join_network`, `update_gateway_settings`, and cranker instructions. Inside the container the path must start with `/app/wallets/` |
| `SOLANA_PRIVATE_KEY` | string | - | Alternative to `SOLANA_KEYPAIR_PATH`: base58-encoded 64-byte secret (Phantom export format). Mutually exclusive with the file form |
| `ENABLE_EPOCH_CRANKING` | boolean | unset (= off) | When `true`, the observer runs permissionless epoch instructions (`close_observation`, `tick_epoch`, etc.). "When unset, observer skips cranking." Set `false` to make the off-state explicit |
| `ARIO_CORE_PROGRAM_ID` | string | - | `ario-core` program ID (token, staking, epoch state) |
| `ARIO_GAR_PROGRAM_ID` | string | - | `ario-gar` program ID (Gateway Registry; joins, observations, distributions) |
| `ARIO_ARNS_PROGRAM_ID` | string | - | `ario-arns` program ID (ArNS name registry) |
Expand Down Expand Up @@ -374,7 +376,22 @@ The default public Solana RPC is rate-limited and may block `getProgramAccounts`

| Variable | Type | Default | Description |
| ------------------------------ | ------- | ------- | ------------------------------------- |
| `SUBMIT_CONTRACT_INTERACTIONS` | boolean | `true` | Submit observations to Solana programs |
| `SUBMIT_CONTRACT_INTERACTIONS` | boolean | `true` | Submit observations to Solana programs. Pre-flight no-ops unless your pubkey is in `epoch.prescribed_observers` — harmless to leave at default before `join_network` |

### Upload Wallet Identities

The observer uploads report bundles to Turbo. The upload signer is resolved from the first matching env in the [precedence chain](/build/run-a-gateway/manage/solana-migration#upload-signing-precedence). Setting envs from more than one chain group at once is rejected at startup.

| Variable | Type | Default | Description |
| --------------------------------- | ------ | ------- | --------------------------------------------------------------------------- |
| `ARWEAVE_UPLOAD_KEY_FILE` | string | - | Path to an Arweave JWK file. Highest priority for upload signing |
| `ARWEAVE_UPLOAD_JWK` | string | - | Inline Arweave JWK JSON. Lower priority than the file form |
| `ETHEREUM_UPLOAD_PRIVATE_KEY_FILE`| string | - | Path to a 32-byte hex private key (with or without `0x` prefix) |
| `ETHEREUM_UPLOAD_PRIVATE_KEY` | string | - | Inline hex private key. Lower priority than the file form |
| `SOLANA_UPLOAD_KEYPAIR_PATH` | string | - | Path to a separate Solana keypair JSON for uploads. Ignored when any `ARWEAVE_UPLOAD_*` or `ETHEREUM_UPLOAD_*` is set |
| `SOLANA_UPLOAD_PRIVATE_KEY` | string | - | Alternative to above: base58 secret. Mutually exclusive with the file form |

When none of the above are set, uploads fall back to the observer key, then the operator key.

### Offset Observation

Expand Down
100 changes: 100 additions & 0 deletions content/build/run-a-gateway/manage/solana-migration.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -135,6 +135,14 @@ Complete these steps before the cutover date to ensure uninterrupted reward elig
chmod 600 wallets/*.json
```

<Callout type="warn" title="Use container paths, not host paths">
Env vars must use the **in-container** path (`/app/wallets/...`), not the host path (`./wallets/...`). Docker Compose bind-mounts `${WALLETS_PATH:-./wallets}` to `/app/wallets` inside the container.

| Wrong (host path) | Right (container path) |
|---|---|
| `SOLANA_KEYPAIR_PATH=./wallets/operator-keypair.json` | `SOLANA_KEYPAIR_PATH=/app/wallets/operator-keypair.json` |
</Callout>

Skip this step entirely if you set `OBSERVER_PRIVATE_KEY` / `SOLANA_PRIVATE_KEY` env vars (base58 strings) instead.
</Step>

Expand Down Expand Up @@ -207,6 +215,85 @@ Complete these steps before the cutover date to ensure uninterrupted reward elig
</Step>
</Steps>

## Wallet Roles and Configuration Patterns

The gateway uses up to four distinct wallet roles. Understanding these helps you pick the right configuration for your setup.

| Role | What it signs | Env vars | Fallback |
|---|---|---|---|
| **Operator** (+ cranker) | `join_network`, `update_gateway_settings`, permissionless cranker instructions | `SOLANA_KEYPAIR_PATH` or `SOLANA_PRIVATE_KEY` | — (required) |
| **Observer** | `save_observations` transactions | `OBSERVER_KEYPAIR_PATH` or `OBSERVER_PRIVATE_KEY` | Falls back to operator key |
| **Upload** | Observer report bundles sent to Turbo | See [upload precedence](#upload-signing-precedence) below | Falls back to observer → operator Solana key |
| **HTTPSIG signer** | RFC 9421 response headers | Uses observer Solana key when set | Auto-generated standalone Ed25519 key |

<Callout type="info">
Setting both the file-path and inline forms for the same role (e.g. `SOLANA_KEYPAIR_PATH` **and** `SOLANA_PRIVATE_KEY`) is rejected at startup as ambiguous. Pick one.
</Callout>

### Supported Configurations

These are the five supported wallet setups. **Pattern 1 is the recommended default** — one key does everything. Pattern 2 is the most common migration path for operators who already have an Arweave JWK.

| # | Operator | Observer | Upload | Required envs |
|---|---|---|---|---|
| **1** | Solana | = operator | = operator (Solana) | `SOLANA_KEYPAIR_PATH` |
| **2** | Solana | = operator | Arweave JWK | `SOLANA_KEYPAIR_PATH` + `ARWEAVE_UPLOAD_KEY_FILE` |
| **3** | Solana A | Solana B | Solana C | `SOLANA_KEYPAIR_PATH` + `OBSERVER_KEYPAIR_PATH` + `SOLANA_UPLOAD_KEYPAIR_PATH` |
| **4** | Solana A | Solana B | Arweave JWK | `SOLANA_KEYPAIR_PATH` + `OBSERVER_KEYPAIR_PATH` + `ARWEAVE_UPLOAD_KEY_FILE` |
| **5** | Solana A | Solana B | Ethereum | `SOLANA_KEYPAIR_PATH` + `OBSERVER_KEYPAIR_PATH` + `ETHEREUM_UPLOAD_PRIVATE_KEY_FILE` |

#### Pattern 1 — Single Solana keypair (recommended)

```bash
# One key for operator + observer + uploads
SOLANA_KEYPAIR_PATH=/app/wallets/operator-keypair.json
SOLANA_RPC_URL=<your dedicated RPC endpoint>
AR_IO_WALLET=<your Solana pubkey>
OBSERVER_WALLET=<your Solana pubkey>
ENABLE_EPOCH_CRANKING=false # flip to true when ready
```

#### Pattern 2 — Keep existing Arweave JWK for uploads

The most common path for operators migrating from a pre-Solana setup. Your existing Arweave JWK continues signing report bundles while the Solana keypair handles protocol interactions.

```bash
SOLANA_KEYPAIR_PATH=/app/wallets/operator-keypair.json
ARWEAVE_UPLOAD_KEY_FILE=/app/wallets/<your-arweave-jwk>.json
SOLANA_RPC_URL=<your dedicated RPC endpoint>
AR_IO_WALLET=<your Solana pubkey>
OBSERVER_WALLET=<your Solana pubkey>
ENABLE_EPOCH_CRANKING=false
```

### Upload Signing Precedence

The gateway picks the first matching upload signer from this list:

```
1. ARWEAVE_UPLOAD_KEY_FILE (file) → ArweaveSigner
2. ARWEAVE_UPLOAD_JWK (inline) → ArweaveSigner
3. ETHEREUM_UPLOAD_PRIVATE_KEY_FILE (file) → EthereumSigner
4. ETHEREUM_UPLOAD_PRIVATE_KEY (inline) → EthereumSigner
5. SOLANA_UPLOAD_KEYPAIR_PATH (explicit) → SolanaSigner
6. Fallback: OBSERVER_KEYPAIR_PATH ?? SOLANA_KEYPAIR_PATH → SolanaSigner
```

<Callout type="warn">
Setting upload envs from more than one chain at once (e.g. `ARWEAVE_UPLOAD_KEY_FILE` **plus** `ETHEREUM_UPLOAD_PRIVATE_KEY`) raises a startup error listing every conflicting env. Pick exactly one upload chain.
</Callout>

### Key Formats

Solana keypairs come in two common formats. Both encode the same 64-byte secret (`seed(32) || pubkey(32)`):

| Format | Example | Source |
|---|---|---|
| **JSON array** (Solana CLI standard) | `[12,34,56,...]` — 64 uint8 integers | `solana-keygen new --outfile keypair.json` |
| **base58 secret** | 87–88 character base58 string | Phantom "export private key", browser wallets |

Use the JSON file with `*_KEYPAIR_PATH` env vars, or the base58 string with `*_PRIVATE_KEY` env vars — never both for the same role.

## Troubleshooting

### "Observer is restart-looping with 'Epoch 0 PDA not found'"
Expand Down Expand Up @@ -248,6 +335,19 @@ The gateway hydrates an ArNS names cache at boot, paginating through the on-chai

Fix: bump `@ar.io/sdk` in your gateway's image (`package.json`) to the latest `^4.0.0-solana.*` and rebuild.

### Wallet Configuration Startup Errors

The gateway validates wallet configuration at startup. Errors are loud and name the offending env:

| Error pattern | Cause | Fix |
|---|---|---|
| `multiple chain groups configured for upload role` | Upload envs from more than one chain are set (e.g. `ARWEAVE_UPLOAD_*` and `ETHEREUM_UPLOAD_*`) | Pick one upload chain and remove the others |
| `ambiguous: both ... set for role` | File-path and inline forms for the same role are both set (e.g. `SOLANA_KEYPAIR_PATH` + `SOLANA_PRIVATE_KEY`) | Use one form per role |
| `material at SOLANA_KEYPAIR_PATH does not look like a Solana keypair` | An Arweave JWK or other JSON was placed at the Solana keypair path | Check you copied the right file — Solana keypairs are a JSON array of 64 integers, not a JWK object |
| `material at ARWEAVE_UPLOAD_KEY_FILE does not look like an Arweave JWK` | A Solana keypair (JSON array) was placed at the Arweave upload slot | Swap the file for your Arweave JWK |
| `SOLANA_KEYPAIR_PATH not set in Solana mode` | The operator key is missing entirely | Set `SOLANA_KEYPAIR_PATH` or `SOLANA_PRIVATE_KEY` |
| `OBSERVER_KEYPAIR_PATH does not match on-chain Gateway.observer_address` | The observer key doesn't match what was registered at `join_network` | Update the key to match, or call `update_observer_address` on-chain |

## New Risks to Be Aware Of

### Gateway Pruning
Expand Down
35 changes: 12 additions & 23 deletions content/build/run-a-gateway/manage/verification-headers.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -113,36 +113,25 @@ Add the following to your gateway's `.env` file:
HTTPSIG_ENABLED=true
```

On startup, the gateway automatically generates an Ed25519 key at `data/keys/httpsig.pem` if one doesn't exist. The key also functions as a valid Solana address.
#### Signing Key Selection

#### Optional: Attestation
When `OBSERVER_KEYPAIR_PATH` or `OBSERVER_PRIVATE_KEY` is set, the gateway uses the observer's Solana keypair directly as the HTTPSIG signing key. Verifiers derive the Solana address from the public key in the `keyId` and look it up in the on-chain Gateway Registry — no separate attestation is needed.

To link your signing key to your on-chain identity, configure an observer wallet. The gateway will create an RSA-signed attestation linking the Ed25519 key to your Arweave wallet and upload it to Arweave:
When neither observer env is set, the gateway auto-generates a standalone Ed25519 key at `data/keys/httpsig.pem`. Responses are still signed, but the signer can't be tied back to the on-chain registry.

```bash
# Observer wallet for attestation (Arweave address)
OBSERVER_WALLET=your-arweave-address
WALLETS_PATH=wallets

# Automatically upload attestation to Arweave (default: true)
HTTPSIG_UPLOAD_ATTESTATION=true

# Optional: include staked gateway address in attestation
AR_IO_WALLET=your-gateway-address
```

Place your Arweave wallet JWK file in the `WALLETS_PATH` directory. The attestation is cached to `data/keys/httpsig-attestation.json` and only regenerated if the key or wallet changes.
<Callout type="info">
If you use a single Solana key for both operator and observer (Pattern 1 in the [migration guide](/build/run-a-gateway/manage/solana-migration#supported-configurations)), HTTPSIG falls back to the auto-generated key because the gateway only piggybacks on `OBSERVER_KEYPAIR_PATH`/`OBSERVER_PRIVATE_KEY` when they're **explicitly set**. To get on-chain-verifiable HTTPSIG with a single key, explicitly set `OBSERVER_KEYPAIR_PATH` (or `OBSERVER_PRIVATE_KEY`) to the same value as your operator key.
</Callout>

### Configuration Reference

| Variable | Default | Description |
|----------|---------|-------------|
| `HTTPSIG_ENABLED` | `false` | Enable RFC 9421 response signing |
| `HTTPSIG_KEY_FILE` | `data/keys/httpsig.pem` | Path to Ed25519 private key (auto-generated if missing) |
| `HTTPSIG_KEY_FILE` | `data/keys/httpsig.pem` | Path to standalone Ed25519 private key (auto-generated if missing). Ignored when `OBSERVER_KEYPAIR_PATH` or `OBSERVER_PRIVATE_KEY` is set |
| `HTTPSIG_BIND_REQUEST` | `true` | Include request method and path in signature (prevents replay) |
| `OBSERVER_WALLET` | - | Arweave wallet address for attestation signing |
| `WALLETS_PATH` | `wallets` | Directory containing Arweave wallet JWK files |
| `HTTPSIG_UPLOAD_ATTESTATION` | `true` | Upload attestation to Arweave automatically |
| `OBSERVER_KEYPAIR_PATH` | - | Path to a 64-byte Solana keypair JSON. When set, used as the HTTPSIG signing key — verifiable against the on-chain GAR |
| `OBSERVER_PRIVATE_KEY` | - | Alternative: base58-encoded 64-byte secret. Mutually exclusive with the file form |

### Verifying It's Working

Expand Down Expand Up @@ -173,9 +162,9 @@ Only responses containing at least one trust-relevant header are signed. Non-dat

Clients can verify signed responses through the following chain:

1. **Signature verification** - The public key is embedded in the `Signature-Input` header's `keyid` parameter and is verifiable via the Web Crypto API in modern browsers.
2. **Identity verification** - If attestation is configured, follow the attestation chain from the Ed25519 signing key to the operator's Arweave RSA wallet to confirm the signer is a registered gateway. The attestation is available at `/ar-io/info` and on Arweave.
3. **Body integrity** - Compare the `Content-Digest` header against a locally computed hash, or walk the Arweave signature chain from the signed `X-AR-IO-Data-Id`.
1. **Signature verification** — The public key is embedded in the `Signature-Input` header's `keyid` parameter and is verifiable via the Web Crypto API in modern browsers.
2. **Identity verification** — When the observer Solana key is used for signing, derive the Solana address from the public key and look it up in the on-chain Gateway Registry (GAR). A match confirms the signer is a registered gateway operator.
3. **Body integrity** — Compare the `Content-Digest` header against a locally computed hash, or walk the Arweave signature chain from the signed `X-AR-IO-Data-Id`.

## Related

Expand Down
Loading