diff --git a/.claude/skills/update-stale-refs/SKILL.md b/.claude/skills/update-stale-refs/SKILL.md new file mode 100644 index 000000000..f2d2fe0c7 --- /dev/null +++ b/.claude/skills/update-stale-refs/SKILL.md @@ -0,0 +1,144 @@ +--- +name: update-stale-refs +description: Use when adjudicating flagged GitHub links from scripts/versioning/check-github-refs.js, or when a docs page links to upstream code that has since changed. Decides whether a page's prose is still accurate after upstream moved, and proposes the smallest edit that makes it correct. Not for bumping refs the checker already auto-fixed. +--- + +# Adjudicating stale upstream references + +`scripts/versioning/check-github-refs.js` bumps the links it can prove are safe. It flags the rest. This skill decides what to do with a flag. + +A flag is not a broken link to repair. It is a signal that a page may describe code that no longer exists or no longer behaves as written. The link is the symptom. Read for the defect. + +## Correct the prose when it is wrong. Change nothing else. + +The target is a page that is accurate against the code at the shipping ref. Not a page that explains how it got there. + +If a sentence is false, change it. Change the minimum needed to make it true, and stop. Do not introduce information the page did not already carry, do not expand a correction into an explanation, and do not add examples or caveats that were not there. + +Never write version-comparative prose. No "as of v0.55", "previously", "this changed in", "no longer", "has been removed", "in older versions". A reader arrives at this page long after the release and needs to know how the system works, not what it used to do. `docs-house-style` requires timeless writing; this is that rule applied to a ref sweep. Version history belongs in the changelog and the upgrade guide, which already have it. + +So, in order of preference: + +1. Sentence still true, link drifted: fix the link. Touch no prose. +2. Sentence false because the thing it describes is gone: delete the sentence, and any list item, heading, table row, diagram label or table-of-contents entry that only existed to support it. +3. Sentence false because the behaviour differs: change the words that are wrong. Keep the sentence's shape, length and level of detail. + +Write the corrected page as though the code had always been this way. The reference fix for this skill renamed a heading, deleted a bullet and a dead link, and edited two strings; net negative on prose. That is the shape to aim for. + +## Scope: decide whether a link should be touched at all + +Work this out before adjudicating anything. Getting it wrong here produces confident, wrong edits. + +**Architecture decision records and RFCs are liveness-only.** For any page under `reference/architecture/` or `reference/rfc/`, the only question is whether the link 404s. If it resolves, leave it exactly as it is: do not bump the ref, do not adjust the line range, do not touch the prose. An ADR records a decision as of a moment, so it legitimately cites code that has since changed or been deleted. If it does 404, propose the most recent ref at which the path still exists, and an older release branch or tag is a perfectly good answer. + +**Version-specific pages keep their own version's refs.** Anything under `//upgrade/` or `//changelog/` is about one release. A v0.54 upgrade guide citing the v0.54 changelog is correct, and bumping it to the newest branch makes the page contradict itself. This was a real bug: "breaking changes in v0.54.0, see the Changelog" ended up pointing at v0.55's changelog, and "the v0.53.x to v0.54.x upgrade reference" at v0.55's `UPGRADING.md`. More generally, if a sentence names a version, its link must match the version the sentence names, not the newest one. + +**A cross-product link takes the dependency's version, not the host page's.** A CometBFT URL on an SDK page does not go to `release/v0.55.x`, which does not exist in the CometBFT repo. Read the host product's `go.mod` at its shipping ref to find which version it actually depends on, and use that. SDK v0.55 pins `cometbft v0.40.0`, so CometBFT links on SDK pages belong on `v0.40.x`. + +**Never run a global find-and-replace across the changed set.** Every ref decision is per-link and depends on the surrounding sentence. A blanket replace produced both of the version-mismatch bugs above, once via tooling and once by hand. + +## Verdicts + +Every flag resolves to exactly one. `no-change` is a real answer and often the right one. + +| Verdict | When | Action | +| ------- | ---- | ------ | +| `no-change` | Upstream changed in a way the prose does not depend on | Bump the ref or relocate the anchor. Touch nothing else. | +| `minimal-edit` | The page states something false | Smallest edit that makes it true. Delete rather than explain. No new information. | +| `needs-decision` | The correct fix depends on intent only a maintainer knows | Do not edit. Write up the options with evidence. | + +A `no-change` verdict must mean you read the prose and confirmed it still holds, not that you only looked at the link. If you could not confirm it, say so rather than defaulting to `no-change`. + +Never invent a replacement target. If a linked file was deleted with no successor, say so rather than pointing at the closest-looking file. Basename matching is not rename detection; `store.go` has 23 candidates in the SDK tree. + +## Procedure + +For each flagged link: + +### 1. Read the upstream change + +The flag carries the diff hunk in `evidence`. If it is absent or truncated, fetch the file at both refs before deciding. Do not infer from the path alone. + +### 2. Read the entire page + +Not the flagged line, not the section. The whole file. + +This is not optional, and it is the step most likely to be skipped. A concept usually appears in more than one place on a page. On `sdk/latest/learn/concepts/store.mdx`, store tracing appeared three times: in its own section, in the page's table-of-contents list, and inside an ASCII diagram of the storage stack. Fixing only the flagged section would have left two stale mentions and an in-page anchor pointing at a heading that no longer existed. + +Before proposing an edit, search the page for every mention of the affected concept, including: + +- table-of-contents or "on this page" lists +- headings, and any anchor that targets them +- code blocks, ASCII diagrams, and tables +- "Next steps" and cross-reference links + +### 3. Decide whether the prose is actually wrong + +Upstream refactors constantly. A function moving between files does not make a conceptual page inaccurate. Ask what the page claims, and whether that claim is still true. Only a false claim justifies an edit. + +### 4. Propose the minimal edit + +Follow `docs-house-style` for all prose and formatting rules. This skill does not restate them. Beyond that: + +- If a heading changes, update every anchor that targets it. `npx mint broken-links` validates page paths only. It does not check anchors, so a stale anchor ships silently. +- Anchor slugs: `.` becomes `-`, `/` is kept and percent-encoded as `%2F`, and smart punctuation rewrites heading text before slugging. See the anchor rules in the root `CLAUDE.md`. +- Internal links are absolute Mintlify paths with no extension, for example `/sdk/latest/learn/concepts/store`. +- Do not add a callout to announce the change. + +### 5. Check which directory to edit + +- Before a freeze, edit `next/`. The promotion copies it to `latest/`, so one pass covers both. +- After a freeze, edit `latest/`, then run `node scripts/sync-latest-to-next.js `. It preserves the destination's front matter, so `noindex` and `canonical` on the `next/` copy survive. +- Never edit an archived version directory such as `v0.54/` or `v0.38/`. Those are frozen snapshots and their links are correct for the version they document. + +## Output + +Propose everything as a batch. Do not apply edits until a maintainer approves the batch. + +Per flag: + +``` +: [verdict] + upstream: what changed, one or two lines + claim: what the page currently asserts + accurate: yes | no + edit: the diff, or "none" + also: other places on the page touched by the same concept, or "none" +``` + +Then a summary: counts per verdict, and the `needs-decision` items with their options. + +After approval, apply the edits, sync if `latest/` was edited, then verify with `npx mint broken-links` plus a manual check of any anchor whose heading changed. + +## Review pass, mandatory before handing back + +Any sweep that touches more than a handful of pages gets an independent review pass. Do not skip it and do not review your own work in the same context that produced it. + +Build the list of every changed page from `git status --porcelain`, split it into batches, and dispatch one review agent per batch on a small model. Each agent runs `git diff -- ` for its pages and reports problems only. Reviewers must not edit anything. + +This exists because a bulk sweep fails in ways the author cannot see. Real defects that only a review pass caught: + +- A find-and-replace bumped a v0.54 upgrade guide to v0.55, leaving "breaking changes in v0.54.0, see the Changelog" pointing at the v0.55 changelog, and "the v0.53.x to v0.54.x upgrade reference" pointing at v0.55's `UPGRADING.md`. +- An automated line-mapper rewrote an anchor to a line that existed but held unrelated code, so the link still returned 200. + +Give reviewers these flags: + +1. Version mismatch. If the prose names a version, the link must match that version, not the newest one. +2. Display text carrying a line number that disagrees with the URL's `#L` anchor. +3. A changed line that now contradicts something else on the same page. +4. Prose that no longer parses, or a count that no longer matches the list beneath it. +5. Version-comparative phrasing introduced. +6. An em dash introduced. +7. Mechanical damage: a stray backtick or punctuation inside a URL, a half-replaced string, a broken markdown link, duplicated words. + +And tell them explicitly what is deliberate, or they will report it: + +- An older ref on an ADR or RFC page, which cites code as it stood at decision time. +- A dead URL shown as plain text or inline code rather than a link. +- A version-specific page under `/upgrade/` or `/changelog/` keeping its own version's refs. + +Expect false positives on that last group; an unlinked dead URL looks exactly like corruption. Triage every flag against the page before acting, and treat a reviewer's report as a list of questions rather than a list of defects. + +## Recording the work + +Log to the branch's file in `work-log/` per `work-log/CLAUDE.md`: what changed and why, not how. Group a page's edits into one entry. A flag that resolved to `no-change` or `needs-decision` is worth a line too, so the next person does not re-investigate it. diff --git a/CLAUDE.md b/CLAUDE.md index 664539444..4a225552e 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -97,11 +97,15 @@ url: "https://example.com" Mintlify preserves special characters in anchor IDs. Rules: - Spaces become `-` +- `.` becomes `-` (e.g. `v0.53.x` becomes `#v0-53-x`) - `&`, `+`, `=`, `@`, `#`, `$`, `%` are kept with surrounding hyphens (e.g. `Gas & Fees` becomes `#gas-&-fees`) -- `/` is kept as-is, no surrounding hyphens (e.g. `x/gov` becomes `#x/gov`) -- `?`, `!`, `(`, `)`, `:`, `` ` ``, `—`, `*`, `.` are dropped (surrounding spaces still become `-`) +- `/` is kept, percent-encoded (e.g. `x/params` becomes `#x%2Fparams`) +- `?`, `!`, `(`, `)`, `:`, `` ` ``, `—`, `*` are dropped (surrounding spaces still become `-`) - `-` in a heading stays as `-`, spaces around it collapse (e.g. `A - B` becomes `#a-b`) - All characters lowercased +- Adjacent replacements collapse to one `-` + +Smart punctuation rewrites headings before slugging; copy anchors from a render. ## snippets/ @@ -138,7 +142,33 @@ cd scripts/versioning && npm run changelogs -- --product --target next cd scripts/versioning && npm run changelogs -- --product --target next --source --unreleased-as --current-only ``` -### 2. Freeze the Version +### 2. Update Version-Pinned Content in `next/` + +Do this before freezing, not after. The freeze copies `next/` to `latest/`, so anything fixed in `next/` first lands in both directories in one pass. Fixing it afterwards means editing `latest/` and then syncing every file back to `next/`. + +Two things are version-pinned and do not follow the freeze on their own: + +**Version label in front matter (SDK only).** Five pages render the version under the page title via their `description`: + +```bash +grep -rn 'description: "Version: v' sdk/next --include='*.mdx' +``` + +**GitHub links pinned to the previous release branch.** Pages link into the product repo at `release/v0..x` (SDK) or `v0..x` (CometBFT), and those refs keep pointing at the old version. Use the checker rather than a find-and-replace: + +```bash +node scripts/versioning/check-github-refs.js --product --targets next --json /tmp/flags.json +``` + +Review the report, then apply the safe rewrites with `--fix`. It bumps only what it can prove is safe and flags the rest. + +Do not blind-replace these by hand. Pinned tags and commit SHAs are deliberate historical citations, and bumping a ref under a `#L` line anchor can leave the link working while pointing at unrelated code. See the GitHub link section in [`scripts/versioning/CLAUDE.md`](scripts/versioning/CLAUDE.md) for the rules and the measured failure rates. + +Hand the `--json` output to the [`update-stale-refs`](.claude/skills/update-stale-refs/SKILL.md) skill, which decides whether a flagged link means the page's prose needs a correction. + +A stale ref is often a symptom rather than the problem. A link that 404s at its current ref usually means the prose describes something upstream deleted, so check what the page claims before repointing the URL. + +### 3. Freeze the Version Run the freeze script from `scripts/versioning/`. This promotes `next/` to `latest/`, rewrites all internal links, injects `noindex` into `next/` pages, and updates `versions.json`. @@ -160,13 +190,13 @@ If the product has pre-existing archived version directories (e.g. `v0.53/`, `v1 node tag-archived.js --product --all ``` -### 3. Check for Broken Links +### 4. Check for Broken Links ```bash npx mint broken-links ``` -Fix any broken links before committing. +Fix any broken links before committing. Note that this checks internal page paths only. It does not validate heading anchors and it does not check external URLs, so nothing here catches a dead or misdirected GitHub link. ## Scripts diff --git a/cometbft/latest/changelog/release-notes.mdx b/cometbft/latest/changelog/release-notes.mdx index 843673ed8..c18127d4b 100644 --- a/cometbft/latest/changelog/release-notes.mdx +++ b/cometbft/latest/changelog/release-notes.mdx @@ -5,87 +5,125 @@ mode: "wide" --- - This page tracks releases and changes for v0.39.0. For the full release history, see the [CHANGELOG](https://github.com/cometbft/cometbft/blob/main/CHANGELOG.md) on GitHub. + This page tracks releases and changes for v0.40.0. For the full release history, see the [CHANGELOG](https://github.com/cometbft/cometbft/blob/main/CHANGELOG.md) on GitHub. - + ## BUG FIXES -- `[evidence]` Add validation for Light Client Attack evidence ByzantineValidators -- `[types]` Fix buffer offset bug in `ProposerPriorityHash` that caused hash collisions when validator priorities differed -- `[p2p]` fix(privval): Ephemeral Port Exhaustion -- `[blocksync]` fix(blocksync): `ExtendedCommit` verification via next blocks `LastCommit` -- [p2p] fix(lp2p): enforce stream max size ([\#5647](https://github.com/cometbft/cometbft/pull/5647)) -- `[metrics]` fix(metrics)!: peer_send_queue_size -- `[statesync]` fix adaptive_sync and streamline stateSync logic -- `[blocksync]` Modify blocksync to use full commit verification instead of light -- `[adaptivesync]` Simplify loop, reuse blockExec.ValidateBlock +- `[blocksync]` tolerate late BlockResponse from honest peers after switching to consensus + ([\#5959](https://github.com/cometbft/cometbft/pull/5959)) +- `[mempool]` include proto framing overhead in AppReactor batch size to prevent peer teardown + ([\#5956](https://github.com/cometbft/cometbft/pull/5956)) +- `[blocksync]` document `adaptive_sync` equivocation risk for validator nodes + ([\#5953](https://github.com/cometbft/cometbft/pull/5953)) +- `[abci]` fix socket transport missing `InsertTx` and `ReapTxs` cases in + `handleRequest` and `resMatchesReq`, causing `ErrUnexpectedResponse` and + node self-kill when `mempool.type = "app"` with the default socket transport + ([\#5958](https://github.com/cometbft/cometbft/pull/5958)) +- `[flowrate]` fix flaky `TestWriter` by comparing `Idle` with a duration + tolerance instead of exact equality + ([\#5929](https://github.com/cometbft/cometbft/pull/5929)) +- `[rpc]` escape the request `Host` in the endpoints listing page so it cannot + break out of the generated HTML + ([\#5921](https://github.com/cometbft/cometbft/pull/5921)) +- `[consensus]` Fix `double_sign_check_height = 1` performing no double-sign + checks due to off-by-one error in loop condition (`i < N` should be + `i <= N`). The value `1` now correctly checks the previous block as intended. + ([\#5668](https://github.com/cometbft/cometbft/pull/5668)) +- `[rpc/jsonrpc]` reject non-finite, fractional, and out-of-int64-range + numeric IDs in request decoding instead of silently saturating to + `math.MinInt`, which previously made distinct large IDs collide. + ([\#5861](https://github.com/cometbft/cometbft/pull/5861)) +- `[consensus]` a proposer now self-verifies its own vote extension before + broadcasting its precommit, so an application whose `ExtendVote` and + `VerifyVoteExtension` handlers are inconsistent halts the node with a clear + `CONSENSUS FAILURE` instead of stalling the whole network + ([\#5204](https://github.com/cometbft/cometbft/issues/5204)) +- `[blocksync]` fix deadlock in `AddBlock` caused by holding `pool.mtx` during + `sendError` + ([\#5931](https://github.com/cometbft/cometbft/pull/5931)) +- `[blocksync]` hold `pool.mtx` and recompute `maxPeerHeight` in `Enable()` + ([\#5888](https://github.com/cometbft/cometbft/pull/5888)) +- `[inspect]` fix flaky `TestInspectRun` and consolidate start/stop handshake + ([\#5891](https://github.com/cometbft/cometbft/pull/5891)) +- `[p2p]` fix flaky switch tests by replacing fixed sleeps with deterministic peer-wait polling + ([\#5918](https://github.com/cometbft/cometbft/pull/5918)) +- `[p2p]` fix race and goroutine leak in `TestTransportMultiplexAcceptNonBlocking` test + ([\#5878](https://github.com/cometbft/cometbft/pull/5878)) +- `[evidence]` fix flaky `TestReactorsGossipNoCommittedEvidence` test + ([\#5870](https://github.com/cometbft/cometbft/pull/5870)) +- `[blocksync]` fix flaky `TestBlockPoolBasic` deadlock under `-race` + ([\#5867](https://github.com/cometbft/cometbft/pull/5867)) +- `[blocksync]` fix removeTimedoutPeers deadlock found via Byzantine prevote gossip race + ([\#5839](https://github.com/cometbft/cometbft/pull/5839)) +- `[mempool]` fix setRecheckFull/setDone race causing spurious ErrRecheckFull. + ([\#5837](https://github.com/cometbft/cometbft/pull/5837)) +- `[abci]` fix deadlock when response callback re-enters the client. + ([\#5850](https://github.com/cometbft/cometbft/pull/5850)) +- `[node]` use kernel-assigned ephemeral ports and fix `OnStart` cleanup + ([\#5868](https://github.com/cometbft/cometbft/pull/5868)) +- `[node]` close partial listeners on startRPC failure + ([\#5869](https://github.com/cometbft/cometbft/pull/5869)) +- `[lp2p]` remove `MaxStreamSize` clamp in `StreamReadSized` + ([\#5954](https://github.com/cometbft/cometbft/pull/5954)) +- `[lp2p]` fallback to conn remote addr when resolving inbound peer + ([\#5879](https://github.com/cometbft/cometbft/pull/5879)) +- `[consensus]` release cs.mtx before sending to statsMsgQueue + ([\#5813](https://github.com/cometbft/cometbft/pull/5813)) +- `[mempool]` truncate proto field number to int32 in filter's ReadTag + ([\#5948](https://github.com/cometbft/cometbft/pull/5948)) +- `[privval]` preempt sleep retries in privval signer client + ([\#5934](https://github.com/cometbft/cometbft/pull/5934)) ## IMPROVEMENTS -- `[ci]`: add lp2p testnet ([\#5643](https://github.com/cometbft/cometbft/pull/5643)) -- `[mempool]` feat!(p2p): introduce follower-mode. Improve lib-p2p integraap access -- `[types]` Add validation for `AuthorityParams.Authority` field in consensus params, enforcing a maximum length of 256 characters ([#5511](https://github.com/cometbft/cometbft/pull/5511)) -- `[mempool]` perf(mempool/cache): Optimize LRUTxCache.Remove to reduce lock contention and map access -- `[e2e]` add support for testing different keytypes, including BLS -- `[crypto]` Reduce BLS signature size to 48 bytes by increasing pubkey size to -- `[statesync]` Add configurable `max-snapshot-chunks` parameter to validate max amount of chunks in a `SnapshotResponse`. -- `[p2p]` feat(lp2p): make reactor queue configurable -- `[cli]` print lib-p2p peer id -- `[p2p]` Add warning when go-libp2p transport is enabled, conveying that the setting -- `[p2p]` feat(p2p): add adaptive sync for comet-p2p +- `[blocksync]` replace `numPending int32` with `atomic.Int32` and document `BlockPool` field ownership + ([\#5889](https://github.com/cometbft/cometbft/pull/5889)) +- `[execution]` cache validator set within a block cycle. + ([\#5834](https://github.com/cometbft/cometbft/pull/5834)) +- `[state]` skip the proposer-priority advance when loading validators for the + block-replay commit-info path (`LoadValidatorsFast`); up to ~900x faster at + the largest checkpoint offsets. + ([\#5204](https://github.com/cometbft/cometbft/issues/5204)) +- `[consensus]` reuse encode/decode buffers in WALEncoder and WALDecoder. + ([\#5865](https://github.com/cometbft/cometbft/pull/5865)) +- `[blocksync]` validate blocksync response sender and signature count + ([\#5860](https://github.com/cometbft/cometbft/pull/5860)) +- `[autofile]` skip fsync in `FlushAndSync` when no new data was written + ([\#5866](https://github.com/cometbft/cometbft/pull/5866)) +- `[mempool]` Implement `MsgBytesFilter` in Reactor to prevent heap amplification attack + ([\#5946](https://github.com/cometbft/cometbft/pull/5946)) +- `[privval]` Dynamically calculate privval maxRemoteSignerMsgSize. + ([\#5985](https://github.com/cometbft/cometbft/pull/5985)) +- `[types]` Update default max block bytes param to account for increased signature size of mldsa65. + ([\#5987](https://github.com/cometbft/cometbft/pull/5987)) +- `[config]` Update the default max_tx_bytes to account for increased signature size of mlsdsa65. + ([\#5989](https://github.com/cometbft/cometbft/pull/5989)) +- `[crypto]` Add UnmarshalJSON to secp256k1eth key type. + ([\#5990](https://github.com/cometbft/cometbft/pull/5990)) ## FEATURES -- `[p2p]` feat(lp2p): implemented resource limiter ([\#5671](https://github.com/cometbft/cometbft/pull/5671)) -- `[p2p]` feat(consensus): add adaptive sync blocksync-to-consensus ingestion ([\#5633](https://github.com/cometbft/cometbft/pull/5633)) -- `[p2p]` feat(lp2p): implement Peer info methods (`NodeInfo`, `RemoteIP`, `RemoteAddr`, `IsOutbound`) -- `[p2p]` feat(lp2p): stop/reconnect peers that failed ([\#5618](https://github.com/cometbft/cometbft/pull/5618)) -- `[p2p]` Add experimental support for lib-p2p networking ([\#5463](https://github.com/cometbft/cometbft/pull/5463)) -- `[crypto]` Add support for BLS12-381 keys. Since the implementation needs -- `[mempool]` Add a metric (a counter) to measure whether a tx was received more than once. -- `[p2p]` Rename `IPeerSet#List` to `Copy`, add `Random`, `ForEach` methods. -- `[mempool]` When the node is performing block sync or state sync, the mempool -- Optimized the PSQL indexer -- `[p2p]` make `PeerSet.Remove` more efficient (Author: @odeke-em) -- `[light]` Remove duplicated signature checks in `light.VerifyNonAdjacent` -- `[state/indexer]` Lower the heap allocation of transaction searches -- `[libs/json]` Lower the memory overhead of JSON encoding by using JSON encoders internally -- `[log]` allow strip out all debug-level code from the binary at compile time using build flags -- `[types]` Small reduction in memory allocation via swapping Key with Equals in VoteSet -- `[event-bus]` Remove the debug logs in PublishEventTx, which were noticed production slowdowns. -- `[state/execution]` Cache the block hash computation inside of the Block Type, so we only compute it once. -- `[consensus/state]` Remove a redundant `VerifyBlock` call in `FinalizeCommit` -- `[p2p/channel]` Speedup `ProtoIO` writer creation time, and thereby speedup channel writing by 5%. -- `[p2p/conn]` Minor speedup (3%) to connection.WritePacketMsgTo, by removing MinInt calls. -- `[blockstore]` Remove a redundant `Header.ValidateBasic` call in `LoadBlockMeta`, 75% reducing this time. -- `[p2p]` Lower `flush_throttle_timeout` to 10ms -- `[types]` Significantly speedup types.MakePartSet and types.AddPart, which are used in creating a block proposal -- `[types] Make a new method`GetByAddressMut` for `ValSet`, which does not copy the returned validator. -- `[consensus]` Make Vote messages only take one peerstate mutex -- `[consensus]` Make the consensus reactor no longer have packets on receive take the consensus lock. Consensus will now update the reactor's view after every relevant change through the existing synchronous event bus subscription. -- `[p2p/conn]` Speedup secret connection large writes, by buffering the write to the underlying connection. -- `[consensus]` Make broadcasting `HasVote` and `HasProposalBlockPart` control messages use `TrySend` instead of `Send`. This saves notable amounts of performance, while at the same time those messages are for preventing redundancy, not critical, and may be dropped without risks for the protocol. -- `[p2p/conn]` Removes several heap allocations per packet send, stemming from how we double-wrap packets prior to proto marshalling them in the connection layer. This change reduces the memory overhead and speeds up the code. -- `[p2p/conn]` Speedup secret connection large packet reads, by buffering the read to the underlying connection. -- `[mempool]` In the broadcast routine, get the pointer to the peer's state once, before starting to iterate through the list of transactions. -- `[consensus]` Make mempool updates asynchronous from consensus Commit's, -- [consensus] Add peer height metric publication to the consensus reactor's peer state. +- `[config]` Add EventBusBufferCapacity setting. + ([\#5849](https://github.com/cometbft/cometbft/pull/5849)) +- `[abci/server]` Accept pre-bound listener in socket and gRPC servers. + ([\#5904](https://github.com/cometbft/cometbft/pull/5904)) +- `[crypto]` Add ml-dsa-65 keytype. + ([\#5875](https://github.com/cometbft/cometbft/pull/5875)) +- `[crypto]` Add `secp256k1eth` keytype: go-ethereum-compatible secp256k1 signing + (legacy Keccak-256, 65-byte `[R||S||V]` signatures, 20-byte Ethereum addresses). + ([\#5907](https://github.com/cometbft/cometbft/pull/5907)) -## BUG-FIXES +## STATE-BREAKING -- `[evidence]` Use structured logging for consensus buffer flush error -- `[consensus]` Reject oversized proposals -- `[store]` Prune extended commits properly -- `[mempool]` Fix mutex in `CListMempool.Flush` method, by changing it from read-lock to write-lock -- `[crypto/bls12381]` Fix JSON marshal of private key -- `[crypto/bls12381]` Modify `Sign`, `Verify` to use `dstMinPk` -- `[bits]` Validate BitArray mismatched Bits and Elems length -- `[cli]` Prevent inadvertent rollover of IPs in `cometbft testnet` config generator +- `[crypto]` `secp256k1eth` verification now requires exact 65-byte recoverable + `[R||S||V]` signatures with canonical `V` in `{0,1}`. +- `[state]` `MedianTime` skips `Nil` and `Absent` precommits, aligning with `VerifyCommit`'s commit tally. + ([\#5901](https://github.com/cometbft/cometbft/pull/5901)) ## API-BREAKING -- `[p2p]` Rename `IPeerSet#List` to `Copy`, add `Random`, `ForEach` methods. -- `[crypto]` Remove Sr25519 curve -- `[rpc]` The endpoints `broadcast_tx_*` now return an error when the node is +- `[crypto]` Add ml-dsa-65 keytype. + ([\#5875](https://github.com/cometbft/cometbft/pull/5875)) diff --git a/cometbft/latest/docs/app-dev/Using-ABCI-CLI.mdx b/cometbft/latest/docs/app-dev/Using-ABCI-CLI.mdx index 568d77685..2619e01c2 100644 --- a/cometbft/latest/docs/app-dev/Using-ABCI-CLI.mdx +++ b/cometbft/latest/docs/app-dev/Using-ABCI-CLI.mdx @@ -64,7 +64,7 @@ purposes. We'll start a kvstore application, which was installed at the same time as `abci-cli` above. The kvstore just stores transactions in a Merkle tree. Its code can be found -[here](https://github.com/cometbft/cometbft/blob/v0.38.x/abci/example/kvstore/kvstore.go). +[here](https://github.com/cometbft/cometbft/blob/v0.40.x/abci/example/kvstore/kvstore.go). Start the application by running: @@ -104,7 +104,7 @@ response. The server may be generic for a particular language, and we provide a [reference implementation in -Golang](https://github.com/cometbft/cometbft/tree/v0.38.x/abci/server). See the +Golang](https://github.com/cometbft/cometbft/tree/v0.40.x/abci/server). See the [list of other ABCI implementations](https://github.com/tendermint/awesome#ecosystem) for servers in other languages. diff --git a/cometbft/latest/docs/core/Running-in-production.mdx b/cometbft/latest/docs/core/Running-in-production.mdx index 15ae1aa0f..7f927b1d3 100644 --- a/cometbft/latest/docs/core/Running-in-production.mdx +++ b/cometbft/latest/docs/core/Running-in-production.mdx @@ -369,7 +369,7 @@ proposing the next block). By default, CometBFT checks whether a peer's address is routable before saving it to the address book. The address is considered as routable if the IP -is [valid and within allowed ranges](https://github.com/cometbft/cometbft/blob/v0.38.x/p2p/netaddress.go#L258). +is [valid and within allowed ranges](https://github.com/cometbft/cometbft/blob/v0.40.x/p2p/netaddress.go#L259). This may not be the case for private or local networks, where your IP range is usually strictly limited and private. In that case, you need to set `addr_book_strict` diff --git a/cometbft/latest/docs/core/Subscribing-to-events-via-Websocket.mdx b/cometbft/latest/docs/core/Subscribing-to-events-via-Websocket.mdx index 70102289f..dffd217ef 100644 --- a/cometbft/latest/docs/core/Subscribing-to-events-via-Websocket.mdx +++ b/cometbft/latest/docs/core/Subscribing-to-events-via-Websocket.mdx @@ -63,7 +63,7 @@ Prior to version `v0.38.x`, floats were not supported as query parameters. When the validator set changes, the ValidatorSetUpdates event is published. The event carries a list of pubkey/power pairs. The list is the same as what CometBFT receives from the ABCI application (see the [EndBlock -section](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/abci/abci++_methods.md#endblock) in +section](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/abci/abci++_methods.md#endblock) in the ABCI spec). Response: diff --git a/cometbft/latest/docs/core/Using-CometBFT.mdx b/cometbft/latest/docs/core/Using-CometBFT.mdx index 657a2b0bb..c1a3864b1 100644 --- a/cometbft/latest/docs/core/Using-CometBFT.mdx +++ b/cometbft/latest/docs/core/Using-CometBFT.mdx @@ -39,7 +39,7 @@ cometbft testnet --help The `genesis.json` file in `$CMTHOME/config/` defines the initial CometBFT state upon genesis of the blockchain ([see -definition](https://github.com/cometbft/cometbft/blob/v0.38.x/types/genesis.go)). +definition](https://github.com/cometbft/cometbft/blob/v0.40.x/types/genesis.go)). #### Fields @@ -49,7 +49,7 @@ definition](https://github.com/cometbft/cometbft/blob/v0.38.x/types/genesis.go)) chain IDs, you will have a bad time. The ChainID must be less than 50 symbols. - `initial_height`: Height at which CometBFT should begin. If a blockchain is conducting a network upgrade, starting from the stopped height brings uniqueness to previous heights. -- `consensus_params` ([see spec](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/core/data_structures.md#consensusparams)) +- `consensus_params` ([see spec](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/core/data_structures.md#consensusparams)) - `block` - `max_bytes`: Max block size, in bytes. - `max_gas`: Max gas per block. @@ -71,7 +71,7 @@ definition](https://github.com/cometbft/cometbft/blob/v0.38.x/types/genesis.go)) application will initialize the validator set upon `InitChain`. - `pub_key`: The first element specifies the key type, using the declared `PubKeyName` for the adopted - [key type](https://github.com/cometbft/cometbft/blob/v0.38.x/crypto/ed25519/ed25519.go#L36). + [key type](https://github.com/cometbft/cometbft/blob/v0.40.x/crypto/ed25519/ed25519.go#L36). The second element are the pubkey bytes. - `power`: The validator's voting power. - `name`: Name of the validator (optional). @@ -565,7 +565,7 @@ library will deny making connections to peers with the same IP address. ### Upgrading See the -[UPGRADING.md](https://github.com/cometbft/cometbft/blob/v0.38.x/UPGRADING.md) +[UPGRADING.md](https://github.com/cometbft/cometbft/blob/v0.40.x/UPGRADING.md) guide. You may need to reset your chain between major breaking releases. Although, we expect CometBFT to have fewer breaking releases in the future (especially after the 1.0 release). diff --git a/cometbft/latest/docs/core/block-structure.mdx b/cometbft/latest/docs/core/block-structure.mdx index 0639a0da3..8e9d8a33f 100644 --- a/cometbft/latest/docs/core/block-structure.mdx +++ b/cometbft/latest/docs/core/block-structure.mdx @@ -14,5 +14,5 @@ component—that's the best place to get started. To dig deeper, check out the [types package documentation][types]. -[data_structures]: https://github.com/cometbft/cometbft/blob/v0.38.x/spec/core/data_structures.md +[data_structures]: https://github.com/cometbft/cometbft/blob/v0.40.x/spec/core/data_structures.md [types]: https://pkg.go.dev/github.com/cometbft/cometbft/types diff --git a/cometbft/latest/docs/core/block-sync.mdx b/cometbft/latest/docs/core/block-sync.mdx index ae64a95a7..82e8be9b3 100644 --- a/cometbft/latest/docs/core/block-sync.mdx +++ b/cometbft/latest/docs/core/block-sync.mdx @@ -24,7 +24,7 @@ process. Once caught up, the daemon will switch out of Block Sync and into normal consensus mode. After running for some time, the node is considered `caught up` if it has at least one peer and its height is at least as high as the max reported peer height. See [the IsCaughtUp -method](https://github.com/cometbft/cometbft/blob/v0.38.x/blocksync/pool.go#L168). +method](https://github.com/cometbft/cometbft/blob/v0.40.x/blocksync/pool.go#L232). Note: While there have historically been multiple versions of blocksync (v0, v1, and v2), all versions other than v0 have been deprecated in favor of the simplest and most well-understood algorithm. diff --git a/cometbft/latest/docs/core/configuration.mdx b/cometbft/latest/docs/core/configuration.mdx index 1f249f6fa..4d1c76f7d 100644 --- a/cometbft/latest/docs/core/configuration.mdx +++ b/cometbft/latest/docs/core/configuration.mdx @@ -26,7 +26,7 @@ like the file below; however, double-check by inspecting the # The version of the CometBFT binary that created or # last modified the config file. Do not modify this. -version = "0.38.0" +version = "0.40.0" ####################################################################### ### Main Base Config Options ### @@ -39,20 +39,14 @@ proxy_app = "tcp://127.0.0.1:26658" # A custom human readable name for this node moniker = "anonymous" -# Database backend: goleveldb | cleveldb | boltdb | rocksdb | badgerdb -# * goleveldb (github.com/syndtr/goleveldb) -# - UNMAINTAINED -# - stable +# Database backend: goleveldb | cleveldb | rocksdb | badgerdb +# * goleveldb (github.com/syndtr/goleveldb - most popular implementation) # - pure go # - stable # * cleveldb (uses levigo wrapper) # - fast # - requires gcc # - use cleveldb build tag (go build -tags cleveldb) -# * boltdb (uses etcd's fork of bolt - github.com/etcd-io/bbolt) -# - EXPERIMENTAL -# - may be faster in some use-cases (random reads - indexer) -# - use boltdb build tag (go build -tags boltdb) # * rocksdb (uses github.com/tecbot/gorocksdb) # - EXPERIMENTAL # - requires gcc @@ -96,6 +90,11 @@ abci = "socket" # so the app can decide if we should keep the connection or not filter_peers = false +# Buffer capacity for the internal EventBus. A value of 0 means unbuffered +# (publishers block until subscribers receive). Higher values reduce back-pressure +# at the cost of memory. +event_bus_buffer_capacity = 0 + ####################################################################### ### Advanced Configuration Options ### @@ -188,14 +187,9 @@ experimental_close_on_slow_client = false # See https://github.com/tendermint/tendermint/issues/3435 timeout_broadcast_tx_commit = "10s" -# Maximum number of requests that can be sent in a JSON-RPC batch request. -# Possible values: number greater than 0. -# If the number of requests sent in a JSON-RPC batch exceed the maximum batch -# size configured, an error will be returned. -# The default value is set to `10`, which will limit the number of requests -# to 10 requests per JSON-RPC batch request. -# If you don't want to enforce a maximum number of requests for a batch -# request, set this value to `0`. +# Maximum number of requests that can be sent in a batch +# If the value is set to '0' (zero-value), then no maximum batch size will be +# enforced for a JSON-RPC batch request. max_request_batch_size = 10 # Maximum size of request body, in bytes @@ -261,7 +255,7 @@ unconditional_peer_ids = "" persistent_peers_max_dial_period = "0s" # Time to wait before flushing messages out on the connection -flush_throttle_timeout = "100ms" +flush_throttle_timeout = "10ms" # Maximum size of a message packet payload, in bytes max_packet_msg_payload_size = 1024 @@ -291,6 +285,64 @@ allow_duplicate_ip = false handshake_timeout = "20s" dial_timeout = "3s" +# Experimental: configuration for go-libp2p +[p2p.libp2p] + +# Enabled set true to use go-libp2p for networking instead of CometBFT's p2p. +enabled = false + +# Bootstrap peers to connect to +# format: { host, id, private (opt), persistent (opt), unconditional (opt) } +bootstrap_peers = [] + + +# Options for scaling concurrent p2p message queues. +# Tune workers to keep the system near the ideal operating point: +# enough concurrency for throughput while keeping processing latency low. +[p2p.libp2p.scaler] + +# Min and max concurrent worker range. +min_workers = 4 +max_workers = 32 + +# Target latency threshold: +# scale up when observed latency is below this value, scale down when above it. +threshold_latency = "100ms" + +# Override a specific reactor (case-insensitive), for example: +# [[p2p.libp2p.scaler.overrides]] +# reactor = "BLOCKSYNC" +# min_workers = 2 +# max_workers = 16 +# threshold_latency = "250ms" +# +# By default, MEMPOOL reactor is overridden to have increased throughput +# If you want to disable this, explicitly set override to an empty list: +# overrides = [] +[[p2p.libp2p.scaler.overrides]] +reactor = "MEMPOOL" +min_workers = 8 +max_workers = 512 +threshold_latency = "500ms" + +# Configuration for resource limits +[p2p.libp2p.limits] + +# Resource management modes: +# - disabled: no resource limits. Use only in trusted environments (e.g. local dev, testing). +# Disabling limits can expose the node to resource exhaustion from malicious peers. +# - default: libp2p's built-in limits. Memory is 1/8th of total system RAM, capped at 128MB min +# and 1GB max. Suitable for most production deployments. +# - custom: disable limits for app protocols but enforce max_peers and max_peer_streams. +# Use when you need tighter control over peer count and stream concurrency. +mode = "default" + +# Maximum number of peers (custom mode only) +max_peers = 0 + +# Maximum number of concurrent streams per peer (custom mode only) +max_peer_streams = 0 + ####################################################### ### Mempool Configuration Option ### ####################################################### @@ -304,6 +356,7 @@ dial_timeout = "3s" # - "nop" : nop-mempool (short for no operation; the ABCI app is responsible # for storing, disseminating and proposing txs). "create_empty_blocks=false" is # not supported. +# - "app" : app-side mempool (the ABCI app is responsible for mempool, comet only broadcasts txs). type = "flood" # Recheck (default: true) defines whether CometBFT should recheck the @@ -313,6 +366,17 @@ type = "flood" # you can disable rechecking. recheck = true +# recheck_timeout is the time the application has during the rechecking process +# to return CheckTx responses, once all requests have been sent. Responses that +# arrive after the timeout expires are discarded. It only applies to +# non-local ABCI clients and when recheck is enabled. +# +# The ideal value will strongly depend on the application. It could roughly be estimated as the +# average size of the mempool multiplied by the average time it takes the application to validate one +# transaction. We consider that the ABCI application runs in the same location as the CometBFT binary +# so that the recheck duration is not affected by network delays when making requests and receiving responses. +recheck_timeout = "1s" + # Broadcast (default: true) defines whether the mempool should relay # transactions to other peers. Setting this to false will stop the mempool # from relaying transactions to other peers until they are included in a @@ -322,7 +386,7 @@ broadcast = true # WalPath (default: "") configures the location of the Write Ahead Log # (WAL) for the mempool. The WAL is disabled by default. To enable, set -# wal_dir to where you want the WAL to be written (e.g. +# WalPath to where you want the WAL to be written (e.g. # "data/mempool.wal"). wal_dir = "" @@ -344,13 +408,39 @@ keep-invalid-txs-in-cache = false # Maximum size of a single transaction. # NOTE: the max size of a tx transmitted over the network is {max_tx_bytes}. -max_tx_bytes = 1048576 +max_tx_bytes = 4194304 # Maximum size of a batch of transactions to send to a peer # Including space needed by encoding (one varint per transaction). # XXX: Unused due to https://github.com/tendermint/tendermint/issues/5796 max_batch_bytes = 0 +# Experimental parameters to limit gossiping txs to up to the specified number of peers. +# We use two independent upper values for persistent and non-persistent peers. +# Unconditional peers are not affected by this feature. +# If we are connected to more than the specified number of persistent peers, only send txs to +# ExperimentalMaxGossipConnectionsToPersistentPeers of them. If one of those +# persistent peers disconnects, activate another persistent peer. +# Similarly for non-persistent peers, with an upper limit of +# ExperimentalMaxGossipConnectionsToNonPersistentPeers. +# If set to 0, the feature is disabled for the corresponding group of peers, that is, the +# number of active connections to that group of peers is not bounded. +# For non-persistent peers, if enabled, a value of 10 is recommended based on experimental +# performance results using the default P2P configuration. +experimental_max_gossip_connections_to_persistent_peers = 0 +experimental_max_gossip_connections_to_non_persistent_peers = 0 + +# App mempool only: size of LRU cache for seen transactions (deduplication). +seen_cache_size = 100000 +# App mempool only: max bytes passed to ReapTxs (0 = no limit). +reap_max_bytes = 0 +# App mempool only: max gas passed to ReapTxs (0 = no limit). +reap_max_gas = 0 +# App mempool only: interval between ReapTxs calls when streaming txs from app. +reap_interval = "500ms" +# App mempool only: delay after which a tx is forgotten for ABCI.CheckTx +check_tx_retry_delay = "5s" + ####################################################### ### State Sync Configuration Options ### ####################################################### @@ -387,6 +477,9 @@ chunk_request_timeout = "10s" # The number of concurrent chunk fetchers to run (default: 1). chunk_fetchers = "4" +# Maximum number of chunks allowed in a snapshot (default: 100000). +max_snapshot_chunks = 100000 + ####################################################### ### Block Sync Configuration Options ### ####################################################### @@ -400,6 +493,15 @@ chunk_fetchers = "4" # 1) "v0" - the default block sync implementation version = "v0" +# Experimental Adaptive sync (bool): +# +# Run both BLOCKSYNC and CONSENSUS for improved liveness, connectivity, and performance. +# NOTE: On validator nodes, running consensus concurrently with blocksync while catching up +# risks equivocation — consensus can sign votes for heights where the ingestor has not yet +# committed the already-decided block. The HRS file and KMS are not sufficient backstops +# for this scenario. Only enable on validators if you understand and accept this risk. +adaptive_sync = false + ####################################################### ### Consensus Configuration Options ### ####################################################### @@ -411,11 +513,11 @@ wal_file = "data/cs.wal/wal" timeout_propose = "3s" # How much timeout_propose increases with each round timeout_propose_delta = "500ms" -# How long we wait after receiving +2/3 prevotes for "anything" (ie. not a single block or nil) +# How long we wait after receiving +2/3 prevotes for “anything” (ie. not a single block or nil) timeout_prevote = "1s" # How much the timeout_prevote increases with each round timeout_prevote_delta = "500ms" -# How long we wait after receiving +2/3 precommits for "anything" (ie. not a single block or nil) +# How long we wait after receiving +2/3 precommits for “anything” (ie. not a single block or nil) timeout_precommit = "1s" # How much the timeout_precommit increases with each round timeout_precommit_delta = "500ms" @@ -441,6 +543,9 @@ create_empty_blocks_interval = "0s" peer_gossip_sleep_duration = "100ms" peer_query_maj23_sleep_duration = "2s" +# Maximum allowed difference between proposed block time and wall-clock time. +block_time_tolerance = "1m0s" + ####################################################### ### Storage Configuration Options ### ####################################################### @@ -495,8 +600,7 @@ max_open_connections = 3 # Instrumentation namespace namespace = "cometbft" - - ``` +``` ## Empty blocks vs. no empty blocks diff --git a/cometbft/latest/docs/core/how-to-read-logs.mdx b/cometbft/latest/docs/core/how-to-read-logs.mdx index e9060a3e2..018b33a25 100644 --- a/cometbft/latest/docs/core/how-to-read-logs.mdx +++ b/cometbft/latest/docs/core/how-to-read-logs.mdx @@ -62,7 +62,7 @@ I[10-04|13:54:30.392] Started node module=main n Next follows a standard block creation cycle, where we enter a new round, propose a block, receive more than 2/3 of prevotes, then precommits, and finally have a chance to commit a block. For details, -please refer to [Byzantine Consensus Algorithm](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/consensus/consensus.md). +please refer to [Byzantine Consensus Algorithm](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/consensus/consensus.md). ```sh I[10-04|13:54:30.393] enterNewRound(91/0). Current: 91/0/RoundStepNewHeight module=consensus @@ -113,7 +113,7 @@ brief overview of what they do. - `abci-client` As mentioned in [Application Development Guide](/cometbft/latest/docs/app-dev/Using-ABCI-CLI), CometBFT acts as an ABCI client with respect to the application and maintains 3 connections: mempool, consensus, and query. The code used by CometBFT can - be found [here](https://github.com/cometbft/cometbft/blob/v0.38.x/abci/client). + be found [here](https://github.com/cometbft/cometbft/blob/v0.40.x/abci/client). - `blockchain` Provides storage, pool (a group of peers), and reactor for both storing and exchanging blocks between peers. - `consensus` The heart of CometBFT, which is the @@ -123,17 +123,17 @@ brief overview of what they do. from a crash. - `events` Simple event notification system. The list of events can be found - [here](https://github.com/cometbft/cometbft/blob/v0.38.x/types/events.go). + [here](https://github.com/cometbft/cometbft/blob/v0.40.x/types/events.go). You can subscribe to them by calling `subscribe` RPC method. Refer to [RPC docs](/cometbft/latest/api-reference/rpc/index) for additional information. - `mempool` Mempool module handles all incoming transactions, whenever they are coming from peers or the application. - `p2p` Provides an abstraction around peer-to-peer communication. For more details, please check out the - [README](https://github.com/cometbft/cometbft/blob/v0.38.x/p2p/README.md). + [README](https://github.com/cometbft/cometbft/blob/v0.40.x/p2p/README.md). - `rpc` [CometBFT's RPC](/cometbft/latest/api-reference/rpc/index). - `rpc-server` RPC server. For implementation details, please read the - [doc.go](https://github.com/cometbft/cometbft/blob/v0.38.x/rpc/jsonrpc/doc.go). + [doc.go](https://github.com/cometbft/cometbft/blob/v0.40.x/rpc/jsonrpc/doc.go). - `state` Represents the latest state and execution submodule, which executes blocks against the application. - `types` A collection of the publicly exposed types and methods to diff --git a/cometbft/latest/docs/core/light-client.mdx b/cometbft/latest/docs/core/light-client.mdx index 0c984c272..a0a6dd586 100644 --- a/cometbft/latest/docs/core/light-client.mdx +++ b/cometbft/latest/docs/core/light-client.mdx @@ -15,7 +15,7 @@ package](https://pkg.go.dev/github.com/cometbft/cometbft/light?tab=doc). The objective of the light client protocol is to get a commit for a recent block hash where the commit includes a majority of signatures from the last known validator set. From there, all the application state is verifiable with -[Merkle proofs](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/core/encoding.md#iavl-tree). +[Merkle proofs](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/core/encoding.md#iavl-tree). ## Properties diff --git a/cometbft/latest/docs/experimental/lib-p2p.mdx b/cometbft/latest/docs/experimental/lib-p2p.mdx index a44846785..7562d4722 100644 --- a/cometbft/latest/docs/experimental/lib-p2p.mdx +++ b/cometbft/latest/docs/experimental/lib-p2p.mdx @@ -27,8 +27,8 @@ and implementations across many languages and transport protocols (TCP, QUIC, We You can refer to the implementation in the CometBFT codebase here: -- [lp2p](https://github.com/cometbft/cometbft/tree/main/lp2p) -- [internal/autopool](https://github.com/cometbft/cometbft/tree/main/internal/autopool) +- [lp2p](https://github.com/cometbft/cometbft/tree/v0.40.x/lp2p) +- [internal/autopool](https://github.com/cometbft/cometbft/tree/v0.40.x/internal/autopool) ## Performance and Liveness diff --git a/cometbft/latest/docs/guides/Creating-a-built-in-application-in-Go.mdx b/cometbft/latest/docs/guides/Creating-a-built-in-application-in-Go.mdx index 9c37e8288..64395e70d 100644 --- a/cometbft/latest/docs/guides/Creating-a-built-in-application-in-Go.mdx +++ b/cometbft/latest/docs/guides/Creating-a-built-in-application-in-Go.mdx @@ -121,7 +121,7 @@ go build CometBFT communicates with the application through the Application BlockChain Interface (ABCI). The messages exchanged through the interface are defined in the ABCI [protobuf -file](https://github.com/cometbft/cometbft/blob/v0.38.x/proto/tendermint/abci/types.proto). +file](https://github.com/cometbft/cometbft/blob/v0.40.x/proto/tendermint/abci/types.proto). We begin by creating the basic scaffolding for an ABCI application by creating a new type, `KVStoreApplication`, which implements the @@ -701,7 +701,7 @@ signal.Notify(c, os.Interrupt, syscall.SIGTERM) Our application is almost ready to run, but first we'll need to populate the CometBFT configuration files. The following command will create a `cometbft-home` directory in your project and add a basic set of configuration files in `cometbft-home/config/`. -For more information on what these files contain, see [the configuration documentation](https://github.com/cometbft/cometbft/blob/v0.38.x/docs/core/configuration.md). +For more information on what these files contain, see [the configuration documentation](https://github.com/cometbft/cometbft/blob/v0.40.x/docs/core/configuration.md). From the root of your project, run: diff --git a/cometbft/latest/docs/guides/Creating-an-application-in-Go.mdx b/cometbft/latest/docs/guides/Creating-an-application-in-Go.mdx index eebb8fc9a..09b40d3ce 100644 --- a/cometbft/latest/docs/guides/Creating-an-application-in-Go.mdx +++ b/cometbft/latest/docs/guides/Creating-an-application-in-Go.mdx @@ -121,7 +121,7 @@ go build CometBFT communicates with the application through the Application BlockChain Interface (ABCI). The messages exchanged through the interface are defined in the ABCI [protobuf -file](https://github.com/cometbft/cometbft/blob/v0.38.x/proto/tendermint/abci/types.proto). +file](https://github.com/cometbft/cometbft/blob/v0.40.x/proto/tendermint/abci/types.proto). We begin by creating the basic scaffolding for an ABCI application by creating a new type, `KVStoreApplication`, which implements the @@ -594,7 +594,7 @@ signal.Notify(c, os.Interrupt, syscall.SIGTERM) Our application is almost ready to run, but first we'll need to populate the CometBFT configuration files. The following command will create a `cometbft-home` directory in your project and add a basic set of configuration files in `cometbft-home/config/`. -For more information on what these files contain, see [the configuration documentation](https://github.com/cometbft/cometbft/blob/v0.38.x/docs/core/configuration.md). +For more information on what these files contain, see [the configuration documentation](https://github.com/cometbft/cometbft/blob/v0.40.x/docs/core/configuration.md). From the root of your project, run: diff --git a/cometbft/latest/docs/introduction/intro.mdx b/cometbft/latest/docs/introduction/intro.mdx index 8c8c49151..b9d9bc022 100644 --- a/cometbft/latest/docs/introduction/intro.mdx +++ b/cometbft/latest/docs/introduction/intro.mdx @@ -129,7 +129,7 @@ consensus engine and provides a particular application state. ## ABCI Overview The [Application BlockChain Interface -(ABCI)](https://github.com/cometbft/cometbft/tree/v0.38.x/abci) +(ABCI)](https://github.com/cometbft/cometbft/tree/v0.40.x/abci) allows for Byzantine Fault Tolerant replication of applications written in any programming language. @@ -194,7 +194,7 @@ core to the application. The application replies with corresponding response messages. The messages are specified here: [ABCI Message -Types](https://github.com/cometbft/cometbft/blob/v0.38.x/proto/tendermint/abci/types.proto). +Types](https://github.com/cometbft/cometbft/blob/v0.40.x/proto/tendermint/abci/types.proto). The **FinalizeBlock** message is the workhorse of the application. Each transaction in the blockchain is finalized within this message. The diff --git a/cometbft/latest/docs/networks/Docker-Compose.mdx b/cometbft/latest/docs/networks/Docker-Compose.mdx index 40fed8a35..6f43e50d9 100644 --- a/cometbft/latest/docs/networks/Docker-Compose.mdx +++ b/cometbft/latest/docs/networks/Docker-Compose.mdx @@ -95,7 +95,7 @@ rm -rf ./build/node* ## Configuring ABCI containers -To use your own ABCI applications with the 4-node setup, edit the [docker-compose.yaml](https://github.com/cometbft/cometbft/blob/v0.38.x/docker-compose.yml) file and add images for your ABCI application. +To use your own ABCI applications with the 4-node setup, edit the [docker-compose.yaml](https://github.com/cometbft/cometbft/blob/v0.40.x/docker-compose.yml) file and add images for your ABCI application. ```yml abci0: @@ -144,7 +144,7 @@ To use your own ABCI applications with the 4-node setup, edit the [docker-compos ``` -Override the [command](https://github.com/cometbft/cometbft/blob/v0.38.x/networks/local/localnode/Dockerfile#L11) in each node to connect to its ABCI. +Override the [command](https://github.com/cometbft/cometbft/blob/v0.40.x/networks/local/localnode/Dockerfile#L11) in each node to connect to its ABCI. ```yml node0: diff --git a/cometbft/latest/docs/qa/CometBFT-QA-38.mdx b/cometbft/latest/docs/qa/CometBFT-QA-38.mdx index ced165cde..cdd7231d8 100644 --- a/cometbft/latest/docs/qa/CometBFT-QA-38.mdx +++ b/cometbft/latest/docs/qa/CometBFT-QA-38.mdx @@ -531,4 +531,4 @@ Observe that in all runs, the average number of transactions in the mempool quic [\#539]: https://github.com/cometbft/cometbft/issues/539 [\#546]: https://github.com/cometbft/cometbft/issues/546 [\#562]: https://github.com/cometbft/cometbft/issues/562 -[end-to-end]: https://github.com/cometbft/cometbft/tree/main/test/e2e +[end-to-end]: https://github.com/cometbft/cometbft/tree/v0.38.0-alpha.2/test/e2e diff --git a/cometbft/latest/docs/qa/Method.mdx b/cometbft/latest/docs/qa/Method.mdx index b57545425..38c9ceaaa 100644 --- a/cometbft/latest/docs/qa/Method.mdx +++ b/cometbft/latest/docs/qa/Method.mdx @@ -13,7 +13,7 @@ This baseline is then compared with results obtained in later versions. Out of the testnet-based test cases described in [the releases document][releases], we focused on two of them: _200 Node Test_ and _Rotating Nodes Test_. -[releases]: https://github.com/cometbft/cometbft/blob/v0.38.x/RELEASES.md#large-scale-testnets +[releases]: https://github.com/cometbft/cometbft/blob/v0.40.x/RELEASES.md#large-scale-testnets ## Software Dependencies @@ -152,8 +152,8 @@ The CometBFT team should improve it at every iteration to increase the amount of This script generates a series of plots per experiment and configuration that may help with visualizing latency vs throughput variation. -[`latency_throughput.py`]: https://github.com/cometbft/cometbft/tree/v0.38.x/scripts/qa/reporting#latency-vs-throughput-plotting -[`latency_plotter.py`]: https://github.com/cometbft/cometbft/tree/v0.38.x/scripts/qa/reporting#latency-vs-throughput-plotting-version-2 +[`latency_throughput.py`]: https://github.com/cometbft/cometbft/tree/v0.40.x/scripts/qa/reporting#latency-vs-throughput-plotting +[`latency_plotter.py`]: https://github.com/cometbft/cometbft/tree/v0.40.x/scripts/qa/reporting#latency-vs-throughput-plotting-version-2 #### Extracting Prometheus Metrics @@ -164,7 +164,7 @@ The CometBFT team should improve it at every iteration to increase the amount of 4. Identify the time window you want to plot in your graphs. 5. Execute the [`prometheus_plotter.py`] script for the time window. -[`prometheus_plotter.py`]: https://github.com/cometbft/cometbft/tree/v0.38.x/scripts/qa/reporting#prometheus-metrics +[`prometheus_plotter.py`]: https://github.com/cometbft/cometbft/tree/v0.40.x/scripts/qa/reporting#prometheus-metrics ## Rotating Node Testnet diff --git a/cometbft/latest/spec/abci/Client-and-server.mdx b/cometbft/latest/spec/abci/Client-and-server.mdx index 6efa095f7..759042f7c 100644 --- a/cometbft/latest/spec/abci/Client-and-server.mdx +++ b/cometbft/latest/spec/abci/Client-and-server.mdx @@ -15,7 +15,7 @@ You are expected to have read all previous sections of ABCI++ specification, nam ## Message Protocol and Synchrony The message protocol consists of pairs of requests and responses defined in the -[protobuf file](https://github.com/cometbft/cometbft/blob/v0.38.x/proto/tendermint/abci/types.proto). +[protobuf file](https://github.com/cometbft/cometbft/blob/v0.40.x/proto/tendermint/abci/types.proto). Some messages have no fields, while others may include byte-arrays, strings, integers, or custom protobuf types. @@ -44,7 +44,7 @@ The implementations in CometBFT's repository can be tested using `abci-cli` by s the `--abci` flag appropriately. See examples, in various stages of maintenance, in -[Go](https://github.com/cometbft/cometbft/tree/master/abci/server), +[Go](https://github.com/cometbft/cometbft/tree/v0.40.x/abci/server), [JavaScript](https://github.com/tendermint/js-abci), and [Java](https://github.com/jTendermint/jabci). diff --git a/cometbft/latest/spec/abci/CometBFTs-expected-behavior.mdx b/cometbft/latest/spec/abci/CometBFTs-expected-behavior.mdx index d445073f6..16d2d7d22 100644 --- a/cometbft/latest/spec/abci/CometBFTs-expected-behavior.mdx +++ b/cometbft/latest/spec/abci/CometBFTs-expected-behavior.mdx @@ -118,7 +118,7 @@ Let us now examine the grammar line by line, providing further details. At the end of a successful attempt, CometBFT calls `Info` to make sure the reconstructed state's _AppHash_ matches the one in the block header at the corresponding height. Note that the state of the application does not contain vote extensions itself. The application can rely on - [CometBFT to ensure](https://github.com/cometbft/cometbft/blob/main/docs/references/rfc/rfc-100-abci-vote-extension-propag.md#base-implementation-persist-and-propagate-extended-commit-history) + [CometBFT to ensure](https://github.com/cometbft/cometbft/blob/v0.40.x/docs/references/rfc/rfc-100-abci-vote-extension-propag.md#base-implementation-persist-and-propagate-extended-commit-history) the node has all the relevant data to proceed with the execution beyond this point. >```abnf @@ -259,7 +259,7 @@ However, the application can use the existing `retain_height` parameter to decid history it wants to keep, just as is done with the block history. The network-wide implications of the usage of `retain_height` stay the same. The decision to store -historical commits and potential optimizations, are discussed in detail in [RFC-100](https://github.com/cometbft/cometbft/blob/main/docs/references/rfc/rfc-100-abci-vote-extension-propag.md#current-limitations-and-possible-implementations) +historical commits and potential optimizations, are discussed in detail in [RFC-100](https://github.com/cometbft/cometbft/blob/v0.40.x/docs/references/rfc/rfc-100-abci-vote-extension-propag.md#current-limitations-and-possible-implementations) ## Handling upgrades to ABCI 2.0 diff --git a/cometbft/latest/spec/abci/Overview.mdx b/cometbft/latest/spec/abci/Overview.mdx index 8d1650202..1d0229d61 100644 --- a/cometbft/latest/spec/abci/Overview.mdx +++ b/cometbft/latest/spec/abci/Overview.mdx @@ -21,7 +21,7 @@ for handling all ABCI++ methods. Thus, CometBFT always sends the `Request*` messages and receives the `Response*` messages in return. -All ABCI++ messages and methods are defined in [protocol buffers](https://github.com/cometbft/cometbft/blob/v0.38.x/proto/tendermint/abci/types.proto). +All ABCI++ messages and methods are defined in [protocol buffers](https://github.com/cometbft/cometbft/blob/v0.40.x/proto/tendermint/abci/types.proto). This allows CometBFT to run with applications written in many programming languages. This specification is split as follows: diff --git a/cometbft/latest/spec/abci/Requirements-for-the-Application.mdx b/cometbft/latest/spec/abci/Requirements-for-the-Application.mdx index 643071d10..cabe0d788 100644 --- a/cometbft/latest/spec/abci/Requirements-for-the-Application.mdx +++ b/cometbft/latest/spec/abci/Requirements-for-the-Application.mdx @@ -263,9 +263,9 @@ the state for each connection, which are synchronized upon `Commit` calls. In principle, each of the four ABCI++ connections operates concurrently with one another. This means applications need to ensure access to state is thread safe. Both the -[default in-process ABCI client](https://github.com/cometbft/cometbft/blob/v0.38.x/abci/client/local_client.go#L13) +[default in-process ABCI client](https://github.com/cometbft/cometbft/blob/v0.40.x/abci/client/local_client.go#L13) and the -[default Go ABCI server](https://github.com/cometbft/cometbft/blob/v0.38.x/abci/server/socket_server.go#L20) +[default Go ABCI server](https://github.com/cometbft/cometbft/blob/v0.40.x/abci/server/socket_server.go#L20) use a global lock to guard the handling of events across all connections, so they are not concurrent at all. This means whether your app is compiled in-process with CometBFT using the `NewLocalClient`, or run out-of-process using the `SocketServer`, @@ -543,13 +543,15 @@ a given public key can only appear once within a given update. If an update incl duplicates, the block execution will fail irrecoverably. Structure `ValidatorUpdate` contains a public key, which is used to identify the validator: -The public key currently supports three types: +The public key currently supports the following types: - `ed25519` - `secp256k1` -- `bls12381` +- `secp256k1eth` +- `bls12_381` +- `ml_dsa_65` -Structure `ValidatorUpdate` also contains an `ìnt64` field denoting the validator's new power. +Structure `ValidatorUpdate` also contains an `int64` field denoting the validator's new power. Applications must ensure that `ValidatorUpdate` structures abide by the following rules: @@ -578,18 +580,19 @@ all full nodes have the same value at a given height. #### List of Parameters -These are the current consensus parameters (as of v0.38.x): +These are the current consensus parameters: 1. [ABCIParams.VoteExtensionsEnableHeight](#abciparamsvoteextensionsenableheight) -2. [BlockParams.MaxBytes](#blockparamsmaxbytes) -3. [BlockParams.MaxGas](#blockparamsmaxgas) -4. [EvidenceParams.MaxAgeDuration](#evidenceparamsmaxageduration) -5. [EvidenceParams.MaxAgeNumBlocks](#evidenceparamsmaxagenumblocks) -6. [EvidenceParams.MaxBytes](#evidenceparamsmaxbytes) -7. [ValidatorParams.PubKeyTypes](#validatorparamspubkeytypes) -8. [VersionParams.App](#versionparamsapp) +2. [AuthorityParams.Authority](#authorityparamsauthority) +3. [BlockParams.MaxBytes](#blockparamsmaxbytes) +4. [BlockParams.MaxGas](#blockparamsmaxgas) +5. [EvidenceParams.MaxAgeDuration](#evidenceparamsmaxageduration) +6. [EvidenceParams.MaxAgeNumBlocks](#evidenceparamsmaxagenumblocks) +7. [EvidenceParams.MaxBytes](#evidenceparamsmaxbytes) +8. [ValidatorParams.PubKeyTypes](#validatorparamspubkeytypes) +9. [VersionParams.App](#versionparamsapp) -#### ABCIParams.VoteExtensionsEnableHeight +##### ABCIParams.VoteExtensionsEnableHeight This parameter is either 0 or a positive height at which vote extensions become mandatory. If the value is zero (which is the default), vote @@ -608,6 +611,11 @@ include the vote extensions from height `H`. For all heights after `H` Must always be set to a future height, 0, or the same height that was previously set. Once the chain's height reaches the value set, it cannot be changed to a different value. +##### AuthorityParams.Authority + +An opaque, application-defined authority string. CometBFT does not interpret it and +only enforces a maximum length. The default is the empty string. + ##### BlockParams.MaxBytes The maximum size of a complete Protobuf encoded block. @@ -638,7 +646,10 @@ If the Application sets value -1, consensus will: Must have `MaxBytes == -1` OR `0 < MaxBytes <= 100 MB`. > Bear in mind that the default value for the `BlockParams.MaxBytes` consensus -> parameter accepts as valid blocks with size up to 21 MB. +> parameter accepts as valid blocks with size up to roughly 53 MiB: a 21 MiB +> budget for block data, plus a worst-case commit reserve sized for the maximum +> validator set at the maximum signature size. Enabling a large-signature key +> type such as `ml_dsa_65` is what makes that reserve large. > If the Application's use case does not need blocks of that size, > or if the impact (specially on bandwidth consumption and block latency) > of propagating blocks of that size was not evaluated, @@ -1022,7 +1033,7 @@ from the genesis file and light client RPC servers. It also calls `Info` to veri Once the state machine has been restored and CometBFT has gathered this additional information, it transitions to consensus. As of ABCI 2.0, CometBFT ensures the necessary conditions -to switch are met [RFC-100](https://github.com/cometbft/cometbft/blob/main/docs/references/rfc/rfc-100-abci-vote-extension-propag.md#base-implementation-persist-and-propagate-extended-commit-history). +to switch are met [RFC-100](https://github.com/cometbft/cometbft/blob/v0.40.x/docs/references/rfc/rfc-100-abci-vote-extension-propag.md#base-implementation-persist-and-propagate-extended-commit-history). From the application's point of view, these operations are transparent, unless the application has just upgraded to ABCI 2.0. In that case, the application needs to be properly configured and aware of certain constraints in terms of when to provide vote extensions. More details can be found in the section below. @@ -1035,7 +1046,7 @@ Introducing vote extensions requires changes to the configuration of the applica First of all, switching to a version of CometBFT with vote extensions, requires a coordinated upgrade. For a detailed description on the upgrade path, please refer to the corresponding -[section](https://github.com/cometbft/cometbft/blob/main/docs/references/rfc/rfc-100-abci-vote-extension-propag.md#upgrade-path) in RFC-100. +[section](https://github.com/cometbft/cometbft/blob/v0.40.x/docs/references/rfc/rfc-100-abci-vote-extension-propag.md#upgrade-path) in RFC-100. There is a newly introduced [**consensus parameter**](/cometbft/latest/spec/abci/Requirements-for-the-Application#consensus-parameters): `VoteExtensionsEnableHeight`. This parameter represents the height at which vote extensions are diff --git a/cometbft/latest/spec/consensus/Byzantine-Consensus-Algorithm.mdx b/cometbft/latest/spec/consensus/Byzantine-Consensus-Algorithm.mdx index cc6d643cb..5054bfebc 100644 --- a/cometbft/latest/spec/consensus/Byzantine-Consensus-Algorithm.mdx +++ b/cometbft/latest/spec/consensus/Byzantine-Consensus-Algorithm.mdx @@ -15,7 +15,7 @@ order: 1 - A node is said to be _at_ a given height, round, and step, or at `(H,R,S)`, or at `(H,R)` in short to omit the step. - To _prevote_ or _precommit_ something means to broadcast a prevote - or precommit [vote](https://github.com/cometbft/cometbft/blob/af3bc47df982e271d4d340a3c5e0d773e440466d/types/vote.go#L50) + or precommit [vote](https://github.com/cometbft/cometbft/blob/v0.40.x/types/vote.go#L64-L75) for something. - A vote _at_ `(H,R)` is a vote signed with the bytes for `H` and `R` included in its [sign-bytes](/cometbft/latest/spec/core/Data_structures#vote). @@ -106,7 +106,7 @@ example, - Nodes gossip prevotes for the proposed PoLC (proof-of-lock-change) round if one is proposed. - Nodes gossip to nodes lagging in blockchain height with block - [commits](https://github.com/cometbft/cometbft/blob/af3bc47df982e271d4d340a3c5e0d773e440466d/types/block.go#L738) + [commits](https://github.com/cometbft/cometbft/blob/v0.40.x/types/block.go#L852-L866) for older blocks. - Nodes opportunistically gossip `ReceivedVote` messages to hint peers what votes it already has. @@ -121,7 +121,7 @@ A proposal is signed and published by the designated proposer at each round. The proposer is chosen by a deterministic and non-choking round robin selection algorithm that selects proposers in proportion to their voting power (see -[implementation](https://github.com/cometbft/cometbft/blob/af3bc47df982e271d4d340a3c5e0d773e440466d/types/validator_set.go#L51)). +[implementation](https://github.com/cometbft/cometbft/blob/v0.40.x/types/validator_set.go#L56-L65)). A proposal at `(H,R)` is composed of a block and an optional latest `PoLC-Round < R` which is included iff the proposer knows of one. This @@ -293,7 +293,7 @@ may make JSet verification/gossip logic easier to implement. ### Censorship Attacks Due to the definition of a block -[commit](https://github.com/cometbft/cometbft/blob/v0.38.x/docs/core/validators.md), any 1/3+ coalition of +[commit](https://github.com/cometbft/cometbft/blob/v0.40.x/docs/core/validators.md), any 1/3+ coalition of validators can halt the blockchain by not broadcasting their votes. Such a coalition can also censor particular transactions by rejecting blocks that include these transactions, though this would result in a diff --git a/cometbft/latest/spec/consensus/Evidence.mdx b/cometbft/latest/spec/consensus/Evidence.mdx index 6dd552356..4ba4fac1f 100644 --- a/cometbft/latest/spec/consensus/Evidence.mdx +++ b/cometbft/latest/spec/consensus/Evidence.mdx @@ -52,11 +52,11 @@ different, more lightweight verification method they are subject to a different kind of 1/3+ attack whereby the byzantine validators could sign an alternative light block that the light client will think is valid. Detection, explained in greater detail -[here](https://github.com/cometbft/cometbft/blob/main/spec/light-client/detection/detection_003_reviewed.md), involves comparison +[here](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/light-client/detection/detection_003_reviewed.md), involves comparison with multiple other nodes in the hope that at least one is "honest". An "honest" node will return a challenging light block for the light client to validate. If this challenging light block also meets the -[validation criteria](https://github.com/cometbft/cometbft/blob/main/spec/light-client/verification/verification_001_published.md) +[validation criteria](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/light-client/verification/verification_001_published.md) then the light client sends the "forged" light block to the node. [Verification](#lightclientattackevidence) is addressed further down. diff --git a/cometbft/latest/spec/consensus/WAL.mdx b/cometbft/latest/spec/consensus/WAL.mdx index d984bfeaf..40356052f 100644 --- a/cometbft/latest/spec/consensus/WAL.mdx +++ b/cometbft/latest/spec/consensus/WAL.mdx @@ -10,7 +10,7 @@ It also issues fsync syscall through node (to prevent double signing). Under the hood, it uses -[autofile.Group](https://github.com/cometbft/cometbft/blob/af3bc47df982e271d4d340a3c5e0d773e440466d/libs/autofile/group.go#L54), +[autofile.Group](https://github.com/cometbft/cometbft/blob/v0.40.x/libs/autofile/group.go#L56-L79), which rotates files when those get too big (> 10MB). The total maximum size is 1GB. We only need the latest block and the block before it, @@ -31,5 +31,5 @@ WAL. Then it will go to precommit, and that time it will work because the private validator contains the `LastSignBytes` and then we’ll replay the precommit from the WAL. -Make sure to read about [WAL corruption](https://github.com/cometbft/cometbft/blob/v0.38.x/docs/core/running-in-production.md#wal-corruption) +Make sure to read about [WAL corruption](https://github.com/cometbft/cometbft/blob/v0.40.x/docs/core/running-in-production.md#wal-corruption) and recovery strategies. diff --git a/cometbft/latest/spec/consensus/light-client/accountability.md b/cometbft/latest/spec/consensus/light-client/accountability.md index 3907e8d47..d6e49444f 100644 --- a/cometbft/latest/spec/consensus/light-client/accountability.md +++ b/cometbft/latest/spec/consensus/light-client/accountability.md @@ -1,3 +1,3 @@ # Fork accountability -Deprecated, please see [light-client/accountability](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/accountability). +Deprecated, please see [light-client/accountability](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/light-client/accountability). diff --git a/cometbft/latest/spec/consensus/light-client/detection.md b/cometbft/latest/spec/consensus/light-client/detection.md index 9e70726c7..d9c54e042 100644 --- a/cometbft/latest/spec/consensus/light-client/detection.md +++ b/cometbft/latest/spec/consensus/light-client/detection.md @@ -1,3 +1,3 @@ # Detection -Deprecated, please see [light-client/detection](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/detection). +Deprecated, please see [light-client/detection](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/light-client/detection). diff --git a/cometbft/latest/spec/consensus/light-client/verification.md b/cometbft/latest/spec/consensus/light-client/verification.md index d0e2bf1e5..c6bd06f7f 100644 --- a/cometbft/latest/spec/consensus/light-client/verification.md +++ b/cometbft/latest/spec/consensus/light-client/verification.md @@ -1,3 +1,3 @@ # Core Verification -Deprecated, please see [light-client/verification](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/verification). +Deprecated, please see [light-client/verification](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/light-client/verification). diff --git a/cometbft/latest/spec/consensus/proposer-based-timestamp/pbts-sysmodel_001_draft.md b/cometbft/latest/spec/consensus/proposer-based-timestamp/pbts-sysmodel_001_draft.md index 06f9e8ea5..ac30156ed 100644 --- a/cometbft/latest/spec/consensus/proposer-based-timestamp/pbts-sysmodel_001_draft.md +++ b/cometbft/latest/spec/consensus/proposer-based-timestamp/pbts-sysmodel_001_draft.md @@ -188,4 +188,4 @@ Back to [main document][main]. [arXiv]: https://arxiv.org/abs/1807.04938 -[CMBC-FM-2THIRDS-link]: https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/verification/verification_002_draft.md#cmbc-fm-2thirds1 +[CMBC-FM-2THIRDS-link]: https://github.com/cometbft/cometbft/blob/v0.40.x/spec/light-client/verification/verification_002_draft.md#cmbc-fm-2thirds1 diff --git a/cometbft/latest/spec/core/Data_structures.mdx b/cometbft/latest/spec/core/Data_structures.mdx index 5d0dbbff3..7e8b740d3 100644 --- a/cometbft/latest/spec/core/Data_structures.mdx +++ b/cometbft/latest/spec/core/Data_structures.mdx @@ -50,7 +50,7 @@ and a list of evidence of malfeasance (ie. signing conflicting votes). | Name | Type | Description | Validation | |--------|-------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------| | Header | [Header](#header) | Header corresponding to the block. This field contains information used throughout consensus and other areas of the protocol. To find out what it contains, visit [header](#header) | Must adhere to the validation rules of [header](#header) | -| Data | [Data](#data) | Data contains a list of transactions. The contents of the transaction is unknown to CometBFT. | This field can be empty or populated, but no validation is performed. Applications can perform validation on individual transactions prior to block creation using [checkTx](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/abci/abci%2B%2B_methods.md#checktx). +| Data | [Data](#data) | Data contains a list of transactions. The contents of the transaction is unknown to CometBFT. | This field can be empty or populated, but no validation is performed. Applications can perform validation on individual transactions prior to block creation using [checkTx](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/abci/abci%2B%2B_methods.md#checktx). | Evidence | [EvidenceList](#evidencelist) | Evidence contains a list of infractions committed by validators. | Can be empty, but when populated the validations rules from [evidenceList](#evidencelist) apply | | LastCommit | [Commit](#commit) | `LastCommit` includes one vote for every validator. All votes must either be for the previous block, nil or absent. If a vote is for the previous block it must have a valid signature from the corresponding validator. The sum of the voting power of the validators that voted must be greater than 2/3 of the total voting power of the complete validator set. The number of votes in a commit is limited to 10000 (see `types.MaxVotesCount`). | Must be empty for the initial height and must adhere to the validation rules of [commit](#commit). | @@ -144,7 +144,7 @@ versioning that this can refer to) | Name | type | Description | Validation | |-------|--------|---------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------| | Block | uint64 | This number represents the block version and must be the same throughout an operational network | Must be equal to block version being used in a network (`block.Version.Block == state.Version.Consensus.Block`) | -| App | uint64 | App version is decided on by the application. Read [here](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/abci/abci++_app_requirements.md) | `block.Version.App == state.Version.Consensus.App` | +| App | uint64 | App version is decided on by the application. Read [here](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/abci/abci++_app_requirements.md) | `block.Version.App == state.Version.Consensus.App` | ## BlockID @@ -224,7 +224,7 @@ to reconstruct the vote set given the validator set. | Signature | [Signature](#signature) | Signature corresponding to the validators participation in consensus. | The length of the signature must be > 0 and < than 64 | NOTE: `ValidatorAddress` and `Timestamp` fields may be removed in the future -(see [ADR-25](https://github.com/cometbft/cometbft/blob/main/docs/references/architecture/tendermint-core/adr-025-commit.md)). +(see [ADR-25](https://github.com/cometbft/cometbft/blob/v0.40.x/docs/references/architecture/tendermint-core/adr-025-commit.md)). ## ExtendedCommitSig diff --git a/cometbft/latest/spec/core/encoding.mdx b/cometbft/latest/spec/core/encoding.mdx index 9402bf79a..5c17a0f2c 100644 --- a/cometbft/latest/spec/core/encoding.mdx +++ b/cometbft/latest/spec/core/encoding.mdx @@ -18,7 +18,7 @@ For details on varints, see the [protobuf spec](https://developers.google.com/protocol-buffers/docs/encoding#varints). For example, the byte-array `[0xA, 0xB]` would be encoded as `0x020A0B`, -while a byte-array containing 300 entires beginning with `[0xA, 0xB, ...]` would +while a byte-array containing 300 entries beginning with `[0xA, 0xB, ...]` would be encoded as `0xAC020A0B...` where `0xAC02` is the UVarint encoding of 300. ## Hashing @@ -40,7 +40,7 @@ include details of the private keys beyond their type and name. ### Key Types -Each type specifies it's own pubkey, address, and signature format. +Each type specifies its own pubkey, address, and signature format. #### Ed25519 @@ -58,7 +58,31 @@ CometBFT adopts [zip215](https://zips.z.cash/zip-0215) for verification of ed255 #### Secp256k1 -The address is the first 20-bytes of the SHA256 hash of the raw 32-byte public key: +The address is the RIPEMD160 hash of the SHA256 hash of the raw 33-byte compressed public key. RIPEMD160 produces 20 bytes, so there is no truncation step: + +```go +address = RIPEMD160(SHA256(pubkey)) +``` + +#### Secp256k1Eth + +This key type is compatible with go-ethereum. The public key is a 33-byte compressed SEC1 key. + +The address is the last 20 bytes of the legacy Keccak-256 hash of the uncompressed public key: + +```go +address = Keccak256(uncompressedPubKey[1:])[12:] +``` + +The signature is a 65-byte go-ethereum signature in `[R || S || V]` form. Verification requires exactly 65 bytes, a canonical lower-S value, and a recovery byte `V` of `0` or `1`. + +#### ML-DSA-65 + +This key type is the NIST ML-DSA-65 post-quantum signature scheme (FIPS 204). The public key is 1952 bytes, and the signature is 3309 bytes. + +For creating, enabling, and rotating ML-DSA-65 validator consensus keys, see [the post-quantum key guides](/sdk/next/keys/post-quantum-keys). + +The address is the first 20 bytes of the SHA256 hash of the public key, matching the Ed25519 convention: ```go address = SHA256(pubkey)[:20] diff --git a/cometbft/latest/spec/core/genesis.mdx b/cometbft/latest/spec/core/genesis.mdx index cfec3119a..8b125188f 100644 --- a/cometbft/latest/spec/core/genesis.mdx +++ b/cometbft/latest/spec/core/genesis.mdx @@ -24,7 +24,7 @@ The genesis file is the starting point of a chain. An application will populate > Note: For evidence to be considered invalid, evidence must be older than both `max_age_num_blocks` and `max_age_duration` - `validator` - - `pub_key_types`: Defines which curves are to be accepted as a valid validator consensus key. CometBFT supports ed25519, secp256k1, and bls12381. + - `pub_key_types`: Defines which curves are to be accepted as a valid validator consensus key. CometBFT supports ed25519, secp256k1, secp256k1eth, bls12_381, and ml_dsa_65. The default is ed25519 alone, and bls12_381 requires a binary built with the `bls12381` build tag. - `version` - `app_version`: The version of the application. This is set by the application and is used to identify which version of the app a user should be using in order to operate a node. diff --git a/cometbft/latest/spec/light-client/Fork-Detection.mdx b/cometbft/latest/spec/light-client/Fork-Detection.mdx index 9f5b3a4c8..eba15b9c9 100644 --- a/cometbft/latest/spec/light-client/Fork-Detection.mdx +++ b/cometbft/latest/spec/light-client/Fork-Detection.mdx @@ -14,13 +14,13 @@ This directory captures the ongoing work and discussion on fork detection both in the context of a Cosmos light node and in the context of IBC. It contains the following files -### [detection.md](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/detection/detection_003_reviewed.md) +### [detection.md](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/light-client/detection/detection_003_reviewed.md) a draft of the light node fork detection including "proof of fork" definition, that is, the data structure to submit evidence to full nodes. -### [discussions.md](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/detection/discussions.md) +### [discussions.md](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/light-client/detection/discussions.md) A collection of ideas and intuitions from recent discussions @@ -29,14 +29,14 @@ A collection of ideas and intuitions from recent discussions which fork detection happens - a discussion about lightstore semantics -### [req-ibc-detection.md](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/detection/req-ibc-detection.md) +### [req-ibc-detection.md](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/light-client/detection/req-ibc-detection.md) - a collection of requirements for fork detection in the IBC context. In particular it contains a section "Required Changes in ICS 007" with necessary updates to ICS 007 to support Cosmos fork detection -### [draft-functions.md](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/detection/draft-functions.md) +### [draft-functions.md](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/light-client/detection/draft-functions.md) In order to address the collected requirements, we started to sketch some functions that we will need in the future when we specify in more diff --git a/cometbft/latest/spec/light-client/Light-Client-Specification.mdx b/cometbft/latest/spec/light-client/Light-Client-Specification.mdx index bc9363cc6..1b53daf79 100644 --- a/cometbft/latest/spec/light-client/Light-Client-Specification.mdx +++ b/cometbft/latest/spec/light-client/Light-Client-Specification.mdx @@ -6,7 +6,7 @@ order: 1 This directory contains work-in-progress English and TLA+ specifications for the Light Client protocol. Implementations of the light client can be found in [Rust](https://github.com/informalsystems/tendermint-rs/tree/master/light-client) and -[Go](https://github.com/cometbft/cometbft/tree/v0.38.x/light). +[Go](https://github.com/cometbft/cometbft/tree/v0.40.x/light). Light clients are assumed to be initialized once from a trusted source with a trusted header and validator set. The light client @@ -25,10 +25,10 @@ In case a lightclient attack is detected, the lightclient submits evidence to a ## Commit Verification -The [English specification](https://github.com/cometbft/cometbft/blob/main/spec/light-client/verification/verification_001_published.md) describes the light client +The [English specification](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/light-client/verification/verification_001_published.md) describes the light client commit verification problem in terms of the temporal properties -[LCV-DIST-SAFE.1](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/verification/verification_001_published.md#lcv-dist-safe1) and -[LCV-DIST-LIVE.1](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/verification/verification_001_published.md#lcv-dist-live1). +[LCV-DIST-SAFE.1](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/light-client/verification/verification_001_published.md#lcv-dist-safe1) and +[LCV-DIST-LIVE.1](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/light-client/verification/verification_001_published.md#lcv-dist-live1). Commit verification is assumed to operate within the Cosmos Failure Model, where +2/3 of validators are correct for some time period and validator sets can change arbitrarily at each height. @@ -40,18 +40,18 @@ many intermediate headers by exploiting overlap in trusted and untrusted validat When there is not enough overlap, a bisection routine can be used to find a minimal set of headers that do provide the required overlap. -The [TLA+ specification ver. 001](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/verification/Lightclient_A_1.tla) +The [TLA+ specification ver. 001](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/light-client/verification/Lightclient_A_1.tla) is a formal description of the commit verification protocol executed by a client, including the safety and termination, which can be model checked with Apalache. A more detailed TLA+ specification of -[Light client verification ver. 003](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/verification/Lightclient_003_draft.tla) +[Light client verification ver. 003](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/light-client/verification/Lightclient_003_draft.tla) is currently under peer review. The `MC*.tla` files contain concrete parameters for the -[TLA+ specification](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/verification/Lightclient_A_1.tla), in order to do model checking. -For instance, [MC4_3_faulty.tla](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/verification/MC4_3_faulty.tla) contains the following parameters +[TLA+ specification](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/light-client/verification/Lightclient_A_1.tla), in order to do model checking. +For instance, [MC4_3_faulty.tla](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/light-client/verification/MC4_3_faulty.tla) contains the following parameters for the nodes, heights, the trusting period, the clock drifts, correctness of the primary node, and the ratio of the faulty processes: @@ -83,8 +83,8 @@ $DIR/apalache-tests/scripts/parse-logs.py --human . All lines in `results.csv` should report `Deadlock`, which means that the algorithm has terminated and no invariant violation was found. -Similar to [002bmc-apalache-ok.csv](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/verification/002bmc-apalache-ok.csv), -file [003bmc-apalache-error.csv](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/verification/003bmc-apalache-error.csv) specifies +Similar to [002bmc-apalache-ok.csv](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/light-client/verification/002bmc-apalache-ok.csv), +file [003bmc-apalache-error.csv](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/light-client/verification/003bmc-apalache-error.csv) specifies the set of experiments that should result in counterexamples: ```sh @@ -96,7 +96,7 @@ All lines in `results.csv` should report `Error`. The following table summarizes the experimental results for Light client verification version 001. The TLA+ properties can be found in the -[TLA+ specification](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/verification/Lightclient_A_1.tla). +[TLA+ specification](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/light-client/verification/Lightclient_A_1.tla). The experiments were run in an AWS instance equipped with 32GB RAM and a 4-core Intel® Xeon® CPU E5-2686 v4 @ 2.30GHz CPU. We write “`✗=k`” when a bug is reported at depth k, and “`✓<=k`” when @@ -108,7 +108,7 @@ The experimental results for version 003 are to be added. ## Attack Detection -The [English specification](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/detection/detection_003_reviewed.md) +The [English specification](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/light-client/detection/detection_003_reviewed.md) defines light client attacks (and how they differ from blockchain forks), and describes the problem of a light client detecting these attacks by communicating with a network of full nodes, @@ -120,19 +120,19 @@ protocol matches corresponding headers provided by the secondaries. If this is not the case, the protocol analyses the verification traces of the involved full nodes and generates -[evidence](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/detection/detection_003_reviewed.md#cmbc-lc-evidence-data1) +[evidence](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/light-client/detection/detection_003_reviewed.md#cmbc-lc-evidence-data1) of misbehavior that can be submitted to a full node so that the faulty validators can be punished. -The [TLA+ specification](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/detection/LCDetector_003_draft.tla) +The [TLA+ specification](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/light-client/detection/LCDetector_003_draft.tla) is a formal description of the detection protocol for two peers, including the safety and termination, which can be model checked with Apalache. The `LCD_MC*.tla` files contain concrete parameters for the -[TLA+ specification](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/detection/LCDetector_003_draft.tla), +[TLA+ specification](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/light-client/detection/LCDetector_003_draft.tla), in order to run the model checker. -For instance, [LCD_MC4_4_faulty.tla](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/verification/MC4_4_faulty.tla) +For instance, [LCD_MC4_4_faulty.tla](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/light-client/verification/MC4_4_faulty.tla) contains the following parameters for the nodes, heights, the trusting period, the clock drifts, correctness of the nodes, and the ratio of the faulty processes: @@ -166,8 +166,8 @@ $DIR/apalache-tests/scripts/parse-logs.py --human . All lines in `results.csv` should report `Deadlock`, which means that the algorithm has terminated and no invariant violation was found. -Similar to [004bmc-apalache-ok.csv](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/verification/004bmc-apalache-ok.csv), -file [005bmc-apalache-error.csv](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/verification/005bmc-apalache-error.csv) specifies +Similar to [004bmc-apalache-ok.csv](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/light-client/verification/004bmc-apalache-ok.csv), +file [005bmc-apalache-error.csv](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/light-client/verification/005bmc-apalache-error.csv) specifies the set of experiments that should result in counterexamples: ```sh @@ -181,21 +181,21 @@ The detailed experimental results are to be added soon. ## Accountability -The [English specification](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/attacks/isolate-attackers_002_reviewed.md) -defines the protocol that is executed on a full node upon receiving attack [evidence](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/attacks/isolate-attackers_002_reviewed.md#cmbc-lc-evidence-data1) from a lightclient. In particular, the protocol handles three types of attacks +The [English specification](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/light-client/attacks/isolate-attackers_002_reviewed.md) +defines the protocol that is executed on a full node upon receiving attack [evidence](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/light-client/attacks/isolate-attackers_002_reviewed.md#cmbc-lc-evidence-data1) from a lightclient. In particular, the protocol handles three types of attacks - lunatic - equivocation - amnesia -We discussed in the [last part](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/attacks/isolate-attackers_002_reviewed.md#Part-III---Completeness) of the English specification +We discussed in the [last part](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/light-client/attacks/isolate-attackers_002_reviewed.md#Part-III---Completeness) of the English specification that the non-lunatic cases are defined by having the same validator set in the conflicting blocks. For these cases, computer-aided analysis of [Tendermint Consensus in TLA+](/cometbft/latest/spec/light-client/Accountability) shows that equivocation and amnesia capture all non-lunatic attacks. -The [TLA+ specification](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/attacks/Isolation_001_draft.tla) +The [TLA+ specification](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/light-client/attacks/Isolation_001_draft.tla) is a formal description of the protocol, including the safety property, which can be model checked with Apalache. -Similar to the other specifications, [MC_5_3.tla](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/attacks/MC_5_3.tla) contains concrete parameters to run the model checker. The specification can be checked within seconds. +Similar to the other specifications, [MC_5_3.tla](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/light-client/attacks/MC_5_3.tla) contains concrete parameters to run the model checker. The specification can be checked within seconds. [tendermint-accountability](/cometbft/latest/spec/light-client/Accountability) diff --git a/cometbft/latest/spec/p2p/Implementation-of-the-p2p-layer.mdx b/cometbft/latest/spec/p2p/Implementation-of-the-p2p-layer.mdx index 8f0288787..8d10a7035 100644 --- a/cometbft/latest/spec/p2p/Implementation-of-the-p2p-layer.mdx +++ b/cometbft/latest/spec/p2p/Implementation-of-the-p2p-layer.mdx @@ -23,21 +23,21 @@ documentation also applies to the releases `v0.37.*` and `v0.38.*` [^v35]. ## Contents The documentation follows the organization of the -[`p2p` package](https://github.com/cometbft/cometbft/tree/v0.34.x/p2p), +[`p2p` package](https://github.com/cometbft/cometbft/tree/v0.40.x/p2p), which implements the following abstractions: -- [Transport](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/p2p/implementation/transport.md): establishes secure and authenticated +- [Transport](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/p2p/implementation/transport.md): establishes secure and authenticated connections with peers; -- [Switch](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/p2p/implementation/switch.md): responsible for dialing peers and accepting +- [Switch](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/p2p/implementation/switch.md): responsible for dialing peers and accepting connections from peers, for managing established connections, and for routing messages between the reactors and peers, that is, between local and remote instances of the CometBFT protocols; -- [PEX Reactor](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/p2p/implementation/pex.md): due to the several roles of this component, the +- [PEX Reactor](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/p2p/implementation/pex.md): due to the several roles of this component, the documentation is split in several parts: - - [Peer Exchange protocol](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/p2p/implementation/pex-protocol.md): enables nodes to exchange peer addresses, thus implementing a peer discovery service; - - [Address Book](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/p2p/implementation/addressbook.md): stores discovered peer addresses and + - [Peer Exchange protocol](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/p2p/implementation/pex-protocol.md): enables nodes to exchange peer addresses, thus implementing a peer discovery service; + - [Address Book](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/p2p/implementation/addressbook.md): stores discovered peer addresses and quality metrics associated to peers with which the node has interacted; - - [Peer Manager](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/p2p/implementation/peer_manager.md): defines when and to which peers a node + - [Peer Manager](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/p2p/implementation/peer_manager.md): defines when and to which peers a node should dial, in order to establish outbound connections; -- [Types](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/p2p/implementation/types.md) and [Configuration](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/p2p/implementation/configuration.md) provide a list of +- [Types](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/p2p/implementation/types.md) and [Configuration](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/p2p/implementation/configuration.md) provide a list of existing types and configuration parameters used by the p2p package. diff --git a/cometbft/latest/spec/p2p/implementation/switch.md b/cometbft/latest/spec/p2p/implementation/switch.md index e87985336..4056b7dfc 100644 --- a/cometbft/latest/spec/p2p/implementation/switch.md +++ b/cometbft/latest/spec/p2p/implementation/switch.md @@ -50,11 +50,11 @@ The `DialPeersAsync` method receives a list of peer addresses (strings) and dials all of them in parallel. It is invoked in two situations: -- In the [setup](https://github.com/cometbft/cometbft/blob/v0.34.x/node/node.go#L987) +- In the [setup](https://github.com/cometbft/cometbft/blob/v0.40.x/node/node.go#L712) of a node, to establish connections with every configured persistent peer - In the RPC package, to implement two unsafe RPC commands, not used in production: - [`DialSeeds`](https://github.com/cometbft/cometbft/blob/v0.34.x/rpc/core/net.go#L47) and - [`DialPeers`](https://github.com/cometbft/cometbft/blob/v0.34.x/rpc/core/net.go#L87) + [`DialSeeds`](https://github.com/cometbft/cometbft/blob/v0.40.x/rpc/core/net.go#L51) and + [`DialPeers`](https://github.com/cometbft/cometbft/blob/v0.40.x/rpc/core/net.go#L94) The received list of peer addresses to dial is parsed into `NetAddress` instances. In case of parsing errors, the method returns. An exception is made for diff --git a/cometbft/latest/spec/p2p/implementation/transport.md b/cometbft/latest/spec/p2p/implementation/transport.md index 121e48467..c9b89b225 100644 --- a/cometbft/latest/spec/p2p/implementation/transport.md +++ b/cometbft/latest/spec/p2p/implementation/transport.md @@ -43,9 +43,9 @@ The `NetAddress` method exports the listen address configured for the transport. The maximum number of simultaneous incoming connections accepted by the listener is bound to `MaxNumInboundPeer` plus the configured number of unconditional peers, using the `MultiplexTransportMaxIncomingConnections` option, -in the node [initialization](https://github.com/cometbft/cometbft/blob/v0.34.x/node/node.go#L563). +in the node [initialization](https://github.com/cometbft/cometbft/blob/v0.40.x/node/setup.go#L459). -This method is called when a node is [started](https://github.com/cometbft/cometbft/blob/v0.34.x/node/node.go#L974). +This method is called when a node is [started](https://github.com/cometbft/cometbft/blob/v0.40.x/node/node.go#L699). In case of errors, the `acceptPeers` routine is not started and the error is returned. ## Accept @@ -190,7 +190,7 @@ an `ErrRejected` error with reason `isIncompatible` is returned. The `Close` method closes the TCP listener created by the `Listen` method, and sends a signal for interrupting the `acceptPeers` routine. -This method is called when a node is [stopped](https://github.com/cometbft/cometbft/blob/v0.34.x/node/node.go#L1023). +This method is called when a node is [stopped](https://github.com/cometbft/cometbft/blob/v0.40.x/node/node.go#L764). ## Cleanup diff --git a/cometbft/latest/spec/p2p/reactor-api/API-for-Reactors.mdx b/cometbft/latest/spec/p2p/reactor-api/API-for-Reactors.mdx index 770ca75ce..5e720b7ac 100644 --- a/cometbft/latest/spec/p2p/reactor-api/API-for-Reactors.mdx +++ b/cometbft/latest/spec/p2p/reactor-api/API-for-Reactors.mdx @@ -323,11 +323,11 @@ could not be enqueued, because the channel's send queue is still full, after a The `TrySend()` method is a _non-blocking_ method, it _immediately_ returns `false` when the channel's send queue is full. -[peer-interface]: https://github.com/cometbft/cometbft/blob/v0.38.x/p2p/peer.go -[service-interface]: https://github.com/cometbft/cometbft/blob/v0.38.x/libs/service/service.go -[switch-type]: https://github.com/cometbft/cometbft/blob/v0.38.x/p2p/switch.go +[peer-interface]: https://github.com/cometbft/cometbft/blob/v0.40.x/p2p/peer.go +[service-interface]: https://github.com/cometbft/cometbft/blob/v0.40.x/libs/service/service.go +[switch-type]: https://github.com/cometbft/cometbft/blob/v0.40.x/p2p/switch.go -[reactor-interface]: https://github.com/cometbft/cometbft/blob/v0.38.x/p2p/base_reactor.go +[reactor-interface]: https://github.com/cometbft/cometbft/blob/v0.40.x/p2p/base_reactor.go [reactor-registration]: /cometbft/latest/spec/p2p/reactor-api/Reactor-Api#registration [reactor-channels]: /cometbft/latest/spec/p2p/reactor-api/Reactor-Api#registration [reactor-addpeer]: /cometbft/latest/spec/p2p/reactor-api/Reactor-Api#peer-management diff --git a/cometbft/latest/spec/p2p/reactor-api/Reactor-Api.mdx b/cometbft/latest/spec/p2p/reactor-api/Reactor-Api.mdx index fb45e390e..6a310f94b 100644 --- a/cometbft/latest/spec/p2p/reactor-api/Reactor-Api.mdx +++ b/cometbft/latest/spec/p2p/reactor-api/Reactor-Api.mdx @@ -14,7 +14,7 @@ invoked and determines what the p2p layer expects from a reactor, this documentation focuses on the **temporal behaviour** that a reactor implementation should expect from the p2p layer. (That is, in which orders the functions may be called) -This specification is accompanied by the [`reactor.qnt`](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/p2p/reactor-api/reactor.qnt) file, +This specification is accompanied by the [`reactor.qnt`](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/p2p/reactor-api/reactor.qnt) file, a more comprehensive model of the reactor's operation written in [Quint][quint-repo], an executable specification language. The methods declared in the [`Reactor`][reactor-interface] interface are @@ -105,7 +105,7 @@ documented in the companion [API for Reactors](/cometbft/latest/spec/p2p/reactor ## Service interface -A reactor must implement the [`Service`](https://github.com/cometbft/cometbft/blob/v0.38.x/libs/service/service.go) interface, +A reactor must implement the [`Service`](https://github.com/cometbft/cometbft/blob/v0.40.x/libs/service/service.go) interface, in particular, a startup `OnStart()` and a shutdown `OnStop()` methods: ```abnf @@ -229,5 +229,5 @@ Two important observations regarding the implementation of the `Receive` method: In other words, while `Receive` does not return, other messages from the same sender are not delivered to any reactor. -[reactor-interface]: https://github.com/cometbft/cometbft/blob/v0.38.x/p2p/base_reactor.go +[reactor-interface]: https://github.com/cometbft/cometbft/blob/v0.40.x/p2p/base_reactor.go [quint-repo]: https://github.com/informalsystems/quint diff --git a/cometbft/latest/spec/p2p/reactor-api/Reactors.mdx b/cometbft/latest/spec/p2p/reactor-api/Reactors.mdx index 7d95141dc..b0eff051f 100644 --- a/cometbft/latest/spec/p2p/reactor-api/Reactors.mdx +++ b/cometbft/latest/spec/p2p/reactor-api/Reactors.mdx @@ -43,4 +43,4 @@ The remaining of the documentation is organized as follows: layer to the reactors, through the `Switch` and `Peer` abstractions. In other words, the interaction of the protocol layer with the p2p layer (top-down). -[reactor-interface]: https://github.com/cometbft/cometbft/blob/v0.38.x/p2p/base_reactor.go +[reactor-interface]: https://github.com/cometbft/cometbft/blob/v0.40.x/p2p/base_reactor.go diff --git a/cometbft/next/changelog/release-notes.mdx b/cometbft/next/changelog/release-notes.mdx index 843673ed8..631debf4f 100644 --- a/cometbft/next/changelog/release-notes.mdx +++ b/cometbft/next/changelog/release-notes.mdx @@ -1,91 +1,131 @@ --- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/changelog/release-notes' title: "Changelog" description: "Release history and changelog for Cosmos COMETBFT" mode: "wide" --- - This page tracks releases and changes for v0.39.0. For the full release history, see the [CHANGELOG](https://github.com/cometbft/cometbft/blob/main/CHANGELOG.md) on GitHub. + This page tracks releases and changes for v0.40.0. For the full release history, see the [CHANGELOG](https://github.com/cometbft/cometbft/blob/main/CHANGELOG.md) on GitHub. - + ## BUG FIXES -- `[evidence]` Add validation for Light Client Attack evidence ByzantineValidators -- `[types]` Fix buffer offset bug in `ProposerPriorityHash` that caused hash collisions when validator priorities differed -- `[p2p]` fix(privval): Ephemeral Port Exhaustion -- `[blocksync]` fix(blocksync): `ExtendedCommit` verification via next blocks `LastCommit` -- [p2p] fix(lp2p): enforce stream max size ([\#5647](https://github.com/cometbft/cometbft/pull/5647)) -- `[metrics]` fix(metrics)!: peer_send_queue_size -- `[statesync]` fix adaptive_sync and streamline stateSync logic -- `[blocksync]` Modify blocksync to use full commit verification instead of light -- `[adaptivesync]` Simplify loop, reuse blockExec.ValidateBlock +- `[blocksync]` tolerate late BlockResponse from honest peers after switching to consensus + ([\#5959](https://github.com/cometbft/cometbft/pull/5959)) +- `[mempool]` include proto framing overhead in AppReactor batch size to prevent peer teardown + ([\#5956](https://github.com/cometbft/cometbft/pull/5956)) +- `[blocksync]` document `adaptive_sync` equivocation risk for validator nodes + ([\#5953](https://github.com/cometbft/cometbft/pull/5953)) +- `[abci]` fix socket transport missing `InsertTx` and `ReapTxs` cases in + `handleRequest` and `resMatchesReq`, causing `ErrUnexpectedResponse` and + node self-kill when `mempool.type = "app"` with the default socket transport + ([\#5958](https://github.com/cometbft/cometbft/pull/5958)) +- `[flowrate]` fix flaky `TestWriter` by comparing `Idle` with a duration + tolerance instead of exact equality + ([\#5929](https://github.com/cometbft/cometbft/pull/5929)) +- `[rpc]` escape the request `Host` in the endpoints listing page so it cannot + break out of the generated HTML + ([\#5921](https://github.com/cometbft/cometbft/pull/5921)) +- `[consensus]` Fix `double_sign_check_height = 1` performing no double-sign + checks due to off-by-one error in loop condition (`i < N` should be + `i <= N`). The value `1` now correctly checks the previous block as intended. + ([\#5668](https://github.com/cometbft/cometbft/pull/5668)) +- `[rpc/jsonrpc]` reject non-finite, fractional, and out-of-int64-range + numeric IDs in request decoding instead of silently saturating to + `math.MinInt`, which previously made distinct large IDs collide. + ([\#5861](https://github.com/cometbft/cometbft/pull/5861)) +- `[consensus]` a proposer now self-verifies its own vote extension before + broadcasting its precommit, so an application whose `ExtendVote` and + `VerifyVoteExtension` handlers are inconsistent halts the node with a clear + `CONSENSUS FAILURE` instead of stalling the whole network + ([\#5204](https://github.com/cometbft/cometbft/issues/5204)) +- `[blocksync]` fix deadlock in `AddBlock` caused by holding `pool.mtx` during + `sendError` + ([\#5931](https://github.com/cometbft/cometbft/pull/5931)) +- `[blocksync]` hold `pool.mtx` and recompute `maxPeerHeight` in `Enable()` + ([\#5888](https://github.com/cometbft/cometbft/pull/5888)) +- `[inspect]` fix flaky `TestInspectRun` and consolidate start/stop handshake + ([\#5891](https://github.com/cometbft/cometbft/pull/5891)) +- `[p2p]` fix flaky switch tests by replacing fixed sleeps with deterministic peer-wait polling + ([\#5918](https://github.com/cometbft/cometbft/pull/5918)) +- `[p2p]` fix race and goroutine leak in `TestTransportMultiplexAcceptNonBlocking` test + ([\#5878](https://github.com/cometbft/cometbft/pull/5878)) +- `[evidence]` fix flaky `TestReactorsGossipNoCommittedEvidence` test + ([\#5870](https://github.com/cometbft/cometbft/pull/5870)) +- `[blocksync]` fix flaky `TestBlockPoolBasic` deadlock under `-race` + ([\#5867](https://github.com/cometbft/cometbft/pull/5867)) +- `[blocksync]` fix removeTimedoutPeers deadlock found via Byzantine prevote gossip race + ([\#5839](https://github.com/cometbft/cometbft/pull/5839)) +- `[mempool]` fix setRecheckFull/setDone race causing spurious ErrRecheckFull. + ([\#5837](https://github.com/cometbft/cometbft/pull/5837)) +- `[abci]` fix deadlock when response callback re-enters the client. + ([\#5850](https://github.com/cometbft/cometbft/pull/5850)) +- `[node]` use kernel-assigned ephemeral ports and fix `OnStart` cleanup + ([\#5868](https://github.com/cometbft/cometbft/pull/5868)) +- `[node]` close partial listeners on startRPC failure + ([\#5869](https://github.com/cometbft/cometbft/pull/5869)) +- `[lp2p]` remove `MaxStreamSize` clamp in `StreamReadSized` + ([\#5954](https://github.com/cometbft/cometbft/pull/5954)) +- `[lp2p]` fallback to conn remote addr when resolving inbound peer + ([\#5879](https://github.com/cometbft/cometbft/pull/5879)) +- `[consensus]` release cs.mtx before sending to statsMsgQueue + ([\#5813](https://github.com/cometbft/cometbft/pull/5813)) +- `[mempool]` truncate proto field number to int32 in filter's ReadTag + ([\#5948](https://github.com/cometbft/cometbft/pull/5948)) +- `[privval]` preempt sleep retries in privval signer client + ([\#5934](https://github.com/cometbft/cometbft/pull/5934)) ## IMPROVEMENTS -- `[ci]`: add lp2p testnet ([\#5643](https://github.com/cometbft/cometbft/pull/5643)) -- `[mempool]` feat!(p2p): introduce follower-mode. Improve lib-p2p integraap access -- `[types]` Add validation for `AuthorityParams.Authority` field in consensus params, enforcing a maximum length of 256 characters ([#5511](https://github.com/cometbft/cometbft/pull/5511)) -- `[mempool]` perf(mempool/cache): Optimize LRUTxCache.Remove to reduce lock contention and map access -- `[e2e]` add support for testing different keytypes, including BLS -- `[crypto]` Reduce BLS signature size to 48 bytes by increasing pubkey size to -- `[statesync]` Add configurable `max-snapshot-chunks` parameter to validate max amount of chunks in a `SnapshotResponse`. -- `[p2p]` feat(lp2p): make reactor queue configurable -- `[cli]` print lib-p2p peer id -- `[p2p]` Add warning when go-libp2p transport is enabled, conveying that the setting -- `[p2p]` feat(p2p): add adaptive sync for comet-p2p +- `[blocksync]` replace `numPending int32` with `atomic.Int32` and document `BlockPool` field ownership + ([\#5889](https://github.com/cometbft/cometbft/pull/5889)) +- `[execution]` cache validator set within a block cycle. + ([\#5834](https://github.com/cometbft/cometbft/pull/5834)) +- `[state]` skip the proposer-priority advance when loading validators for the + block-replay commit-info path (`LoadValidatorsFast`); up to ~900x faster at + the largest checkpoint offsets. + ([\#5204](https://github.com/cometbft/cometbft/issues/5204)) +- `[consensus]` reuse encode/decode buffers in WALEncoder and WALDecoder. + ([\#5865](https://github.com/cometbft/cometbft/pull/5865)) +- `[blocksync]` validate blocksync response sender and signature count + ([\#5860](https://github.com/cometbft/cometbft/pull/5860)) +- `[autofile]` skip fsync in `FlushAndSync` when no new data was written + ([\#5866](https://github.com/cometbft/cometbft/pull/5866)) +- `[mempool]` Implement `MsgBytesFilter` in Reactor to prevent heap amplification attack + ([\#5946](https://github.com/cometbft/cometbft/pull/5946)) +- `[privval]` Dynamically calculate privval maxRemoteSignerMsgSize. + ([\#5985](https://github.com/cometbft/cometbft/pull/5985)) +- `[types]` Update default max block bytes param to account for increased signature size of mldsa65. + ([\#5987](https://github.com/cometbft/cometbft/pull/5987)) +- `[config]` Update the default max_tx_bytes to account for increased signature size of mlsdsa65. + ([\#5989](https://github.com/cometbft/cometbft/pull/5989)) +- `[crypto]` Add UnmarshalJSON to secp256k1eth key type. + ([\#5990](https://github.com/cometbft/cometbft/pull/5990)) ## FEATURES -- `[p2p]` feat(lp2p): implemented resource limiter ([\#5671](https://github.com/cometbft/cometbft/pull/5671)) -- `[p2p]` feat(consensus): add adaptive sync blocksync-to-consensus ingestion ([\#5633](https://github.com/cometbft/cometbft/pull/5633)) -- `[p2p]` feat(lp2p): implement Peer info methods (`NodeInfo`, `RemoteIP`, `RemoteAddr`, `IsOutbound`) -- `[p2p]` feat(lp2p): stop/reconnect peers that failed ([\#5618](https://github.com/cometbft/cometbft/pull/5618)) -- `[p2p]` Add experimental support for lib-p2p networking ([\#5463](https://github.com/cometbft/cometbft/pull/5463)) -- `[crypto]` Add support for BLS12-381 keys. Since the implementation needs -- `[mempool]` Add a metric (a counter) to measure whether a tx was received more than once. -- `[p2p]` Rename `IPeerSet#List` to `Copy`, add `Random`, `ForEach` methods. -- `[mempool]` When the node is performing block sync or state sync, the mempool -- Optimized the PSQL indexer -- `[p2p]` make `PeerSet.Remove` more efficient (Author: @odeke-em) -- `[light]` Remove duplicated signature checks in `light.VerifyNonAdjacent` -- `[state/indexer]` Lower the heap allocation of transaction searches -- `[libs/json]` Lower the memory overhead of JSON encoding by using JSON encoders internally -- `[log]` allow strip out all debug-level code from the binary at compile time using build flags -- `[types]` Small reduction in memory allocation via swapping Key with Equals in VoteSet -- `[event-bus]` Remove the debug logs in PublishEventTx, which were noticed production slowdowns. -- `[state/execution]` Cache the block hash computation inside of the Block Type, so we only compute it once. -- `[consensus/state]` Remove a redundant `VerifyBlock` call in `FinalizeCommit` -- `[p2p/channel]` Speedup `ProtoIO` writer creation time, and thereby speedup channel writing by 5%. -- `[p2p/conn]` Minor speedup (3%) to connection.WritePacketMsgTo, by removing MinInt calls. -- `[blockstore]` Remove a redundant `Header.ValidateBasic` call in `LoadBlockMeta`, 75% reducing this time. -- `[p2p]` Lower `flush_throttle_timeout` to 10ms -- `[types]` Significantly speedup types.MakePartSet and types.AddPart, which are used in creating a block proposal -- `[types] Make a new method`GetByAddressMut` for `ValSet`, which does not copy the returned validator. -- `[consensus]` Make Vote messages only take one peerstate mutex -- `[consensus]` Make the consensus reactor no longer have packets on receive take the consensus lock. Consensus will now update the reactor's view after every relevant change through the existing synchronous event bus subscription. -- `[p2p/conn]` Speedup secret connection large writes, by buffering the write to the underlying connection. -- `[consensus]` Make broadcasting `HasVote` and `HasProposalBlockPart` control messages use `TrySend` instead of `Send`. This saves notable amounts of performance, while at the same time those messages are for preventing redundancy, not critical, and may be dropped without risks for the protocol. -- `[p2p/conn]` Removes several heap allocations per packet send, stemming from how we double-wrap packets prior to proto marshalling them in the connection layer. This change reduces the memory overhead and speeds up the code. -- `[p2p/conn]` Speedup secret connection large packet reads, by buffering the read to the underlying connection. -- `[mempool]` In the broadcast routine, get the pointer to the peer's state once, before starting to iterate through the list of transactions. -- `[consensus]` Make mempool updates asynchronous from consensus Commit's, -- [consensus] Add peer height metric publication to the consensus reactor's peer state. +- `[config]` Add EventBusBufferCapacity setting. + ([\#5849](https://github.com/cometbft/cometbft/pull/5849)) +- `[abci/server]` Accept pre-bound listener in socket and gRPC servers. + ([\#5904](https://github.com/cometbft/cometbft/pull/5904)) +- `[crypto]` Add ml-dsa-65 keytype. + ([\#5875](https://github.com/cometbft/cometbft/pull/5875)) +- `[crypto]` Add `secp256k1eth` keytype: go-ethereum-compatible secp256k1 signing + (legacy Keccak-256, 65-byte `[R||S||V]` signatures, 20-byte Ethereum addresses). + ([\#5907](https://github.com/cometbft/cometbft/pull/5907)) -## BUG-FIXES +## STATE-BREAKING -- `[evidence]` Use structured logging for consensus buffer flush error -- `[consensus]` Reject oversized proposals -- `[store]` Prune extended commits properly -- `[mempool]` Fix mutex in `CListMempool.Flush` method, by changing it from read-lock to write-lock -- `[crypto/bls12381]` Fix JSON marshal of private key -- `[crypto/bls12381]` Modify `Sign`, `Verify` to use `dstMinPk` -- `[bits]` Validate BitArray mismatched Bits and Elems length -- `[cli]` Prevent inadvertent rollover of IPs in `cometbft testnet` config generator +- `[crypto]` `secp256k1eth` verification now requires exact 65-byte recoverable + `[R||S||V]` signatures with canonical `V` in `{0,1}`. +- `[state]` `MedianTime` skips `Nil` and `Absent` precommits, aligning with `VerifyCommit`'s commit tally. + ([\#5901](https://github.com/cometbft/cometbft/pull/5901)) ## API-BREAKING -- `[p2p]` Rename `IPeerSet#List` to `Copy`, add `Random`, `ForEach` methods. -- `[crypto]` Remove Sr25519 curve -- `[rpc]` The endpoints `broadcast_tx_*` now return an error when the node is +- `[crypto]` Add ml-dsa-65 keytype. + ([\#5875](https://github.com/cometbft/cometbft/pull/5875)) diff --git a/cometbft/next/docs/app-dev/Using-ABCI-CLI.mdx b/cometbft/next/docs/app-dev/Using-ABCI-CLI.mdx index 01ef20e42..ec4e38fbd 100644 --- a/cometbft/next/docs/app-dev/Using-ABCI-CLI.mdx +++ b/cometbft/next/docs/app-dev/Using-ABCI-CLI.mdx @@ -65,7 +65,7 @@ purposes. We'll start a kvstore application, which was installed at the same time as `abci-cli` above. The kvstore just stores transactions in a Merkle tree. Its code can be found -[here](https://github.com/cometbft/cometbft/blob/v0.38.x/abci/example/kvstore/kvstore.go). +[here](https://github.com/cometbft/cometbft/blob/v0.40.x/abci/example/kvstore/kvstore.go). Start the application by running: @@ -105,7 +105,7 @@ response. The server may be generic for a particular language, and we provide a [reference implementation in -Golang](https://github.com/cometbft/cometbft/tree/v0.38.x/abci/server). See the +Golang](https://github.com/cometbft/cometbft/tree/v0.40.x/abci/server). See the [list of other ABCI implementations](https://github.com/tendermint/awesome#ecosystem) for servers in other languages. diff --git a/cometbft/next/docs/core/Running-in-production.mdx b/cometbft/next/docs/core/Running-in-production.mdx index ca3609e6d..8d337a506 100644 --- a/cometbft/next/docs/core/Running-in-production.mdx +++ b/cometbft/next/docs/core/Running-in-production.mdx @@ -370,7 +370,7 @@ proposing the next block). By default, CometBFT checks whether a peer's address is routable before saving it to the address book. The address is considered as routable if the IP -is [valid and within allowed ranges](https://github.com/cometbft/cometbft/blob/v0.38.x/p2p/netaddress.go#L258). +is [valid and within allowed ranges](https://github.com/cometbft/cometbft/blob/v0.40.x/p2p/netaddress.go#L259). This may not be the case for private or local networks, where your IP range is usually strictly limited and private. In that case, you need to set `addr_book_strict` diff --git a/cometbft/next/docs/core/Subscribing-to-events-via-Websocket.mdx b/cometbft/next/docs/core/Subscribing-to-events-via-Websocket.mdx index 82dff7dbd..798d4972f 100644 --- a/cometbft/next/docs/core/Subscribing-to-events-via-Websocket.mdx +++ b/cometbft/next/docs/core/Subscribing-to-events-via-Websocket.mdx @@ -64,7 +64,7 @@ Prior to version `v0.38.x`, floats were not supported as query parameters. When the validator set changes, the ValidatorSetUpdates event is published. The event carries a list of pubkey/power pairs. The list is the same as what CometBFT receives from the ABCI application (see the [EndBlock -section](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/abci/abci++_methods.md#endblock) in +section](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/abci/abci++_methods.md#endblock) in the ABCI spec). Response: diff --git a/cometbft/next/docs/core/Using-CometBFT.mdx b/cometbft/next/docs/core/Using-CometBFT.mdx index 290c7f9c8..86dd5689e 100644 --- a/cometbft/next/docs/core/Using-CometBFT.mdx +++ b/cometbft/next/docs/core/Using-CometBFT.mdx @@ -40,7 +40,7 @@ cometbft testnet --help The `genesis.json` file in `$CMTHOME/config/` defines the initial CometBFT state upon genesis of the blockchain ([see -definition](https://github.com/cometbft/cometbft/blob/v0.38.x/types/genesis.go)). +definition](https://github.com/cometbft/cometbft/blob/v0.40.x/types/genesis.go)). #### Fields @@ -50,7 +50,7 @@ definition](https://github.com/cometbft/cometbft/blob/v0.38.x/types/genesis.go)) chain IDs, you will have a bad time. The ChainID must be less than 50 symbols. - `initial_height`: Height at which CometBFT should begin. If a blockchain is conducting a network upgrade, starting from the stopped height brings uniqueness to previous heights. -- `consensus_params` ([see spec](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/core/data_structures.md#consensusparams)) +- `consensus_params` ([see spec](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/core/data_structures.md#consensusparams)) - `block` - `max_bytes`: Max block size, in bytes. - `max_gas`: Max gas per block. @@ -72,7 +72,7 @@ definition](https://github.com/cometbft/cometbft/blob/v0.38.x/types/genesis.go)) application will initialize the validator set upon `InitChain`. - `pub_key`: The first element specifies the key type, using the declared `PubKeyName` for the adopted - [key type](https://github.com/cometbft/cometbft/blob/v0.38.x/crypto/ed25519/ed25519.go#L36). + [key type](https://github.com/cometbft/cometbft/blob/v0.40.x/crypto/ed25519/ed25519.go#L36). The second element are the pubkey bytes. - `power`: The validator's voting power. - `name`: Name of the validator (optional). @@ -566,7 +566,7 @@ library will deny making connections to peers with the same IP address. ### Upgrading See the -[UPGRADING.md](https://github.com/cometbft/cometbft/blob/v0.38.x/UPGRADING.md) +[UPGRADING.md](https://github.com/cometbft/cometbft/blob/v0.40.x/UPGRADING.md) guide. You may need to reset your chain between major breaking releases. Although, we expect CometBFT to have fewer breaking releases in the future (especially after the 1.0 release). diff --git a/cometbft/next/docs/core/block-structure.mdx b/cometbft/next/docs/core/block-structure.mdx index fb53c83de..58938013f 100644 --- a/cometbft/next/docs/core/block-structure.mdx +++ b/cometbft/next/docs/core/block-structure.mdx @@ -15,5 +15,5 @@ component—that's the best place to get started. To dig deeper, check out the [types package documentation][types]. -[data_structures]: https://github.com/cometbft/cometbft/blob/v0.38.x/spec/core/data_structures.md +[data_structures]: https://github.com/cometbft/cometbft/blob/v0.40.x/spec/core/data_structures.md [types]: https://pkg.go.dev/github.com/cometbft/cometbft/types diff --git a/cometbft/next/docs/core/block-sync.mdx b/cometbft/next/docs/core/block-sync.mdx index 4aa4e8550..5a106c73d 100644 --- a/cometbft/next/docs/core/block-sync.mdx +++ b/cometbft/next/docs/core/block-sync.mdx @@ -25,7 +25,7 @@ process. Once caught up, the daemon will switch out of Block Sync and into normal consensus mode. After running for some time, the node is considered `caught up` if it has at least one peer and its height is at least as high as the max reported peer height. See [the IsCaughtUp -method](https://github.com/cometbft/cometbft/blob/v0.38.x/blocksync/pool.go#L168). +method](https://github.com/cometbft/cometbft/blob/v0.40.x/blocksync/pool.go#L232). Note: While there have historically been multiple versions of blocksync (v0, v1, and v2), all versions other than v0 have been deprecated in favor of the simplest and most well-understood algorithm. diff --git a/cometbft/next/docs/core/configuration.mdx b/cometbft/next/docs/core/configuration.mdx index 13237aaf4..59160d782 100644 --- a/cometbft/next/docs/core/configuration.mdx +++ b/cometbft/next/docs/core/configuration.mdx @@ -27,7 +27,7 @@ like the file below; however, double-check by inspecting the # The version of the CometBFT binary that created or # last modified the config file. Do not modify this. -version = "0.38.0" +version = "0.40.0" ####################################################################### ### Main Base Config Options ### @@ -40,20 +40,14 @@ proxy_app = "tcp://127.0.0.1:26658" # A custom human readable name for this node moniker = "anonymous" -# Database backend: goleveldb | cleveldb | boltdb | rocksdb | badgerdb -# * goleveldb (github.com/syndtr/goleveldb) -# - UNMAINTAINED -# - stable +# Database backend: goleveldb | cleveldb | rocksdb | badgerdb +# * goleveldb (github.com/syndtr/goleveldb - most popular implementation) # - pure go # - stable # * cleveldb (uses levigo wrapper) # - fast # - requires gcc # - use cleveldb build tag (go build -tags cleveldb) -# * boltdb (uses etcd's fork of bolt - github.com/etcd-io/bbolt) -# - EXPERIMENTAL -# - may be faster in some use-cases (random reads - indexer) -# - use boltdb build tag (go build -tags boltdb) # * rocksdb (uses github.com/tecbot/gorocksdb) # - EXPERIMENTAL # - requires gcc @@ -97,6 +91,11 @@ abci = "socket" # so the app can decide if we should keep the connection or not filter_peers = false +# Buffer capacity for the internal EventBus. A value of 0 means unbuffered +# (publishers block until subscribers receive). Higher values reduce back-pressure +# at the cost of memory. +event_bus_buffer_capacity = 0 + ####################################################################### ### Advanced Configuration Options ### @@ -189,14 +188,9 @@ experimental_close_on_slow_client = false # See https://github.com/tendermint/tendermint/issues/3435 timeout_broadcast_tx_commit = "10s" -# Maximum number of requests that can be sent in a JSON-RPC batch request. -# Possible values: number greater than 0. -# If the number of requests sent in a JSON-RPC batch exceed the maximum batch -# size configured, an error will be returned. -# The default value is set to `10`, which will limit the number of requests -# to 10 requests per JSON-RPC batch request. -# If you don't want to enforce a maximum number of requests for a batch -# request, set this value to `0`. +# Maximum number of requests that can be sent in a batch +# If the value is set to '0' (zero-value), then no maximum batch size will be +# enforced for a JSON-RPC batch request. max_request_batch_size = 10 # Maximum size of request body, in bytes @@ -262,7 +256,7 @@ unconditional_peer_ids = "" persistent_peers_max_dial_period = "0s" # Time to wait before flushing messages out on the connection -flush_throttle_timeout = "100ms" +flush_throttle_timeout = "10ms" # Maximum size of a message packet payload, in bytes max_packet_msg_payload_size = 1024 @@ -292,6 +286,64 @@ allow_duplicate_ip = false handshake_timeout = "20s" dial_timeout = "3s" +# Experimental: configuration for go-libp2p +[p2p.libp2p] + +# Enabled set true to use go-libp2p for networking instead of CometBFT's p2p. +enabled = false + +# Bootstrap peers to connect to +# format: { host, id, private (opt), persistent (opt), unconditional (opt) } +bootstrap_peers = [] + + +# Options for scaling concurrent p2p message queues. +# Tune workers to keep the system near the ideal operating point: +# enough concurrency for throughput while keeping processing latency low. +[p2p.libp2p.scaler] + +# Min and max concurrent worker range. +min_workers = 4 +max_workers = 32 + +# Target latency threshold: +# scale up when observed latency is below this value, scale down when above it. +threshold_latency = "100ms" + +# Override a specific reactor (case-insensitive), for example: +# [[p2p.libp2p.scaler.overrides]] +# reactor = "BLOCKSYNC" +# min_workers = 2 +# max_workers = 16 +# threshold_latency = "250ms" +# +# By default, MEMPOOL reactor is overridden to have increased throughput +# If you want to disable this, explicitly set override to an empty list: +# overrides = [] +[[p2p.libp2p.scaler.overrides]] +reactor = "MEMPOOL" +min_workers = 8 +max_workers = 512 +threshold_latency = "500ms" + +# Configuration for resource limits +[p2p.libp2p.limits] + +# Resource management modes: +# - disabled: no resource limits. Use only in trusted environments (e.g. local dev, testing). +# Disabling limits can expose the node to resource exhaustion from malicious peers. +# - default: libp2p's built-in limits. Memory is 1/8th of total system RAM, capped at 128MB min +# and 1GB max. Suitable for most production deployments. +# - custom: disable limits for app protocols but enforce max_peers and max_peer_streams. +# Use when you need tighter control over peer count and stream concurrency. +mode = "default" + +# Maximum number of peers (custom mode only) +max_peers = 0 + +# Maximum number of concurrent streams per peer (custom mode only) +max_peer_streams = 0 + ####################################################### ### Mempool Configuration Option ### ####################################################### @@ -305,6 +357,7 @@ dial_timeout = "3s" # - "nop" : nop-mempool (short for no operation; the ABCI app is responsible # for storing, disseminating and proposing txs). "create_empty_blocks=false" is # not supported. +# - "app" : app-side mempool (the ABCI app is responsible for mempool, comet only broadcasts txs). type = "flood" # Recheck (default: true) defines whether CometBFT should recheck the @@ -314,6 +367,17 @@ type = "flood" # you can disable rechecking. recheck = true +# recheck_timeout is the time the application has during the rechecking process +# to return CheckTx responses, once all requests have been sent. Responses that +# arrive after the timeout expires are discarded. It only applies to +# non-local ABCI clients and when recheck is enabled. +# +# The ideal value will strongly depend on the application. It could roughly be estimated as the +# average size of the mempool multiplied by the average time it takes the application to validate one +# transaction. We consider that the ABCI application runs in the same location as the CometBFT binary +# so that the recheck duration is not affected by network delays when making requests and receiving responses. +recheck_timeout = "1s" + # Broadcast (default: true) defines whether the mempool should relay # transactions to other peers. Setting this to false will stop the mempool # from relaying transactions to other peers until they are included in a @@ -323,7 +387,7 @@ broadcast = true # WalPath (default: "") configures the location of the Write Ahead Log # (WAL) for the mempool. The WAL is disabled by default. To enable, set -# wal_dir to where you want the WAL to be written (e.g. +# WalPath to where you want the WAL to be written (e.g. # "data/mempool.wal"). wal_dir = "" @@ -345,13 +409,39 @@ keep-invalid-txs-in-cache = false # Maximum size of a single transaction. # NOTE: the max size of a tx transmitted over the network is {max_tx_bytes}. -max_tx_bytes = 1048576 +max_tx_bytes = 4194304 # Maximum size of a batch of transactions to send to a peer # Including space needed by encoding (one varint per transaction). # XXX: Unused due to https://github.com/tendermint/tendermint/issues/5796 max_batch_bytes = 0 +# Experimental parameters to limit gossiping txs to up to the specified number of peers. +# We use two independent upper values for persistent and non-persistent peers. +# Unconditional peers are not affected by this feature. +# If we are connected to more than the specified number of persistent peers, only send txs to +# ExperimentalMaxGossipConnectionsToPersistentPeers of them. If one of those +# persistent peers disconnects, activate another persistent peer. +# Similarly for non-persistent peers, with an upper limit of +# ExperimentalMaxGossipConnectionsToNonPersistentPeers. +# If set to 0, the feature is disabled for the corresponding group of peers, that is, the +# number of active connections to that group of peers is not bounded. +# For non-persistent peers, if enabled, a value of 10 is recommended based on experimental +# performance results using the default P2P configuration. +experimental_max_gossip_connections_to_persistent_peers = 0 +experimental_max_gossip_connections_to_non_persistent_peers = 0 + +# App mempool only: size of LRU cache for seen transactions (deduplication). +seen_cache_size = 100000 +# App mempool only: max bytes passed to ReapTxs (0 = no limit). +reap_max_bytes = 0 +# App mempool only: max gas passed to ReapTxs (0 = no limit). +reap_max_gas = 0 +# App mempool only: interval between ReapTxs calls when streaming txs from app. +reap_interval = "500ms" +# App mempool only: delay after which a tx is forgotten for ABCI.CheckTx +check_tx_retry_delay = "5s" + ####################################################### ### State Sync Configuration Options ### ####################################################### @@ -388,6 +478,9 @@ chunk_request_timeout = "10s" # The number of concurrent chunk fetchers to run (default: 1). chunk_fetchers = "4" +# Maximum number of chunks allowed in a snapshot (default: 100000). +max_snapshot_chunks = 100000 + ####################################################### ### Block Sync Configuration Options ### ####################################################### @@ -401,6 +494,15 @@ chunk_fetchers = "4" # 1) "v0" - the default block sync implementation version = "v0" +# Experimental Adaptive sync (bool): +# +# Run both BLOCKSYNC and CONSENSUS for improved liveness, connectivity, and performance. +# NOTE: On validator nodes, running consensus concurrently with blocksync while catching up +# risks equivocation — consensus can sign votes for heights where the ingestor has not yet +# committed the already-decided block. The HRS file and KMS are not sufficient backstops +# for this scenario. Only enable on validators if you understand and accept this risk. +adaptive_sync = false + ####################################################### ### Consensus Configuration Options ### ####################################################### @@ -412,11 +514,11 @@ wal_file = "data/cs.wal/wal" timeout_propose = "3s" # How much timeout_propose increases with each round timeout_propose_delta = "500ms" -# How long we wait after receiving +2/3 prevotes for "anything" (ie. not a single block or nil) +# How long we wait after receiving +2/3 prevotes for “anything” (ie. not a single block or nil) timeout_prevote = "1s" # How much the timeout_prevote increases with each round timeout_prevote_delta = "500ms" -# How long we wait after receiving +2/3 precommits for "anything" (ie. not a single block or nil) +# How long we wait after receiving +2/3 precommits for “anything” (ie. not a single block or nil) timeout_precommit = "1s" # How much the timeout_precommit increases with each round timeout_precommit_delta = "500ms" @@ -442,6 +544,9 @@ create_empty_blocks_interval = "0s" peer_gossip_sleep_duration = "100ms" peer_query_maj23_sleep_duration = "2s" +# Maximum allowed difference between proposed block time and wall-clock time. +block_time_tolerance = "1m0s" + ####################################################### ### Storage Configuration Options ### ####################################################### @@ -496,8 +601,7 @@ max_open_connections = 3 # Instrumentation namespace namespace = "cometbft" - - ``` +``` ## Empty blocks vs. no empty blocks diff --git a/cometbft/next/docs/core/how-to-read-logs.mdx b/cometbft/next/docs/core/how-to-read-logs.mdx index ae0dfc917..71d64c2e0 100644 --- a/cometbft/next/docs/core/how-to-read-logs.mdx +++ b/cometbft/next/docs/core/how-to-read-logs.mdx @@ -63,7 +63,7 @@ I[10-04|13:54:30.392] Started node module=main n Next follows a standard block creation cycle, where we enter a new round, propose a block, receive more than 2/3 of prevotes, then precommits, and finally have a chance to commit a block. For details, -please refer to [Byzantine Consensus Algorithm](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/consensus/consensus.md). +please refer to [Byzantine Consensus Algorithm](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/consensus/consensus.md). ```sh I[10-04|13:54:30.393] enterNewRound(91/0). Current: 91/0/RoundStepNewHeight module=consensus @@ -114,7 +114,7 @@ brief overview of what they do. - `abci-client` As mentioned in [Application Development Guide](/cometbft/next/docs/app-dev/Using-ABCI-CLI), CometBFT acts as an ABCI client with respect to the application and maintains 3 connections: mempool, consensus, and query. The code used by CometBFT can - be found [here](https://github.com/cometbft/cometbft/blob/v0.38.x/abci/client). + be found [here](https://github.com/cometbft/cometbft/blob/v0.40.x/abci/client). - `blockchain` Provides storage, pool (a group of peers), and reactor for both storing and exchanging blocks between peers. - `consensus` The heart of CometBFT, which is the @@ -124,17 +124,17 @@ brief overview of what they do. from a crash. - `events` Simple event notification system. The list of events can be found - [here](https://github.com/cometbft/cometbft/blob/v0.38.x/types/events.go). + [here](https://github.com/cometbft/cometbft/blob/v0.40.x/types/events.go). You can subscribe to them by calling `subscribe` RPC method. Refer to [RPC docs](/cometbft/next/api-reference/rpc/index) for additional information. - `mempool` Mempool module handles all incoming transactions, whenever they are coming from peers or the application. - `p2p` Provides an abstraction around peer-to-peer communication. For more details, please check out the - [README](https://github.com/cometbft/cometbft/blob/v0.38.x/p2p/README.md). + [README](https://github.com/cometbft/cometbft/blob/v0.40.x/p2p/README.md). - `rpc` [CometBFT's RPC](/cometbft/next/api-reference/rpc/index). - `rpc-server` RPC server. For implementation details, please read the - [doc.go](https://github.com/cometbft/cometbft/blob/v0.38.x/rpc/jsonrpc/doc.go). + [doc.go](https://github.com/cometbft/cometbft/blob/v0.40.x/rpc/jsonrpc/doc.go). - `state` Represents the latest state and execution submodule, which executes blocks against the application. - `types` A collection of the publicly exposed types and methods to diff --git a/cometbft/next/docs/core/light-client.mdx b/cometbft/next/docs/core/light-client.mdx index 7de279794..b0aaf49f0 100644 --- a/cometbft/next/docs/core/light-client.mdx +++ b/cometbft/next/docs/core/light-client.mdx @@ -16,7 +16,7 @@ package](https://pkg.go.dev/github.com/cometbft/cometbft/light?tab=doc). The objective of the light client protocol is to get a commit for a recent block hash where the commit includes a majority of signatures from the last known validator set. From there, all the application state is verifiable with -[Merkle proofs](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/core/encoding.md#iavl-tree). +[Merkle proofs](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/core/encoding.md#iavl-tree). ## Properties diff --git a/cometbft/next/docs/experimental/lib-p2p.mdx b/cometbft/next/docs/experimental/lib-p2p.mdx index 748c19579..6e07d39f1 100644 --- a/cometbft/next/docs/experimental/lib-p2p.mdx +++ b/cometbft/next/docs/experimental/lib-p2p.mdx @@ -29,8 +29,8 @@ and implementations across many languages and transport protocols (TCP, QUIC, We You can refer to the implementation in the CometBFT codebase here: -- [lp2p](https://github.com/cometbft/cometbft/tree/main/lp2p) -- [internal/autopool](https://github.com/cometbft/cometbft/tree/main/internal/autopool) +- [lp2p](https://github.com/cometbft/cometbft/tree/v0.40.x/lp2p) +- [internal/autopool](https://github.com/cometbft/cometbft/tree/v0.40.x/internal/autopool) ## Performance and Liveness diff --git a/cometbft/next/docs/guides/Creating-a-built-in-application-in-Go.mdx b/cometbft/next/docs/guides/Creating-a-built-in-application-in-Go.mdx index 44af44ecc..61e10d6a5 100644 --- a/cometbft/next/docs/guides/Creating-a-built-in-application-in-Go.mdx +++ b/cometbft/next/docs/guides/Creating-a-built-in-application-in-Go.mdx @@ -122,7 +122,7 @@ go build CometBFT communicates with the application through the Application BlockChain Interface (ABCI). The messages exchanged through the interface are defined in the ABCI [protobuf -file](https://github.com/cometbft/cometbft/blob/v0.38.x/proto/tendermint/abci/types.proto). +file](https://github.com/cometbft/cometbft/blob/v0.40.x/proto/tendermint/abci/types.proto). We begin by creating the basic scaffolding for an ABCI application by creating a new type, `KVStoreApplication`, which implements the @@ -702,7 +702,7 @@ signal.Notify(c, os.Interrupt, syscall.SIGTERM) Our application is almost ready to run, but first we'll need to populate the CometBFT configuration files. The following command will create a `cometbft-home` directory in your project and add a basic set of configuration files in `cometbft-home/config/`. -For more information on what these files contain, see [the configuration documentation](https://github.com/cometbft/cometbft/blob/v0.38.x/docs/core/configuration.md). +For more information on what these files contain, see [the configuration documentation](https://github.com/cometbft/cometbft/blob/v0.40.x/docs/core/configuration.md). From the root of your project, run: diff --git a/cometbft/next/docs/guides/Creating-an-application-in-Go.mdx b/cometbft/next/docs/guides/Creating-an-application-in-Go.mdx index 21a876d06..d9c6d313b 100644 --- a/cometbft/next/docs/guides/Creating-an-application-in-Go.mdx +++ b/cometbft/next/docs/guides/Creating-an-application-in-Go.mdx @@ -122,7 +122,7 @@ go build CometBFT communicates with the application through the Application BlockChain Interface (ABCI). The messages exchanged through the interface are defined in the ABCI [protobuf -file](https://github.com/cometbft/cometbft/blob/v0.38.x/proto/tendermint/abci/types.proto). +file](https://github.com/cometbft/cometbft/blob/v0.40.x/proto/tendermint/abci/types.proto). We begin by creating the basic scaffolding for an ABCI application by creating a new type, `KVStoreApplication`, which implements the @@ -595,7 +595,7 @@ signal.Notify(c, os.Interrupt, syscall.SIGTERM) Our application is almost ready to run, but first we'll need to populate the CometBFT configuration files. The following command will create a `cometbft-home` directory in your project and add a basic set of configuration files in `cometbft-home/config/`. -For more information on what these files contain, see [the configuration documentation](https://github.com/cometbft/cometbft/blob/v0.38.x/docs/core/configuration.md). +For more information on what these files contain, see [the configuration documentation](https://github.com/cometbft/cometbft/blob/v0.40.x/docs/core/configuration.md). From the root of your project, run: diff --git a/cometbft/next/docs/introduction/intro.mdx b/cometbft/next/docs/introduction/intro.mdx index 26e87dca0..05bd3aede 100644 --- a/cometbft/next/docs/introduction/intro.mdx +++ b/cometbft/next/docs/introduction/intro.mdx @@ -130,7 +130,7 @@ consensus engine and provides a particular application state. ## ABCI Overview The [Application BlockChain Interface -(ABCI)](https://github.com/cometbft/cometbft/tree/v0.38.x/abci) +(ABCI)](https://github.com/cometbft/cometbft/tree/v0.40.x/abci) allows for Byzantine Fault Tolerant replication of applications written in any programming language. @@ -195,7 +195,7 @@ core to the application. The application replies with corresponding response messages. The messages are specified here: [ABCI Message -Types](https://github.com/cometbft/cometbft/blob/v0.38.x/proto/tendermint/abci/types.proto). +Types](https://github.com/cometbft/cometbft/blob/v0.40.x/proto/tendermint/abci/types.proto). The **FinalizeBlock** message is the workhorse of the application. Each transaction in the blockchain is finalized within this message. The diff --git a/cometbft/next/docs/networks/Docker-Compose.mdx b/cometbft/next/docs/networks/Docker-Compose.mdx index 7b53ac82c..3154a02e6 100644 --- a/cometbft/next/docs/networks/Docker-Compose.mdx +++ b/cometbft/next/docs/networks/Docker-Compose.mdx @@ -96,7 +96,7 @@ rm -rf ./build/node* ## Configuring ABCI containers -To use your own ABCI applications with the 4-node setup, edit the [docker-compose.yaml](https://github.com/cometbft/cometbft/blob/v0.38.x/docker-compose.yml) file and add images for your ABCI application. +To use your own ABCI applications with the 4-node setup, edit the [docker-compose.yaml](https://github.com/cometbft/cometbft/blob/v0.40.x/docker-compose.yml) file and add images for your ABCI application. ```yml abci0: @@ -145,7 +145,7 @@ To use your own ABCI applications with the 4-node setup, edit the [docker-compos ``` -Override the [command](https://github.com/cometbft/cometbft/blob/v0.38.x/networks/local/localnode/Dockerfile#L11) in each node to connect to its ABCI. +Override the [command](https://github.com/cometbft/cometbft/blob/v0.40.x/networks/local/localnode/Dockerfile#L11) in each node to connect to its ABCI. ```yml node0: diff --git a/cometbft/next/docs/qa/CometBFT-QA-38.mdx b/cometbft/next/docs/qa/CometBFT-QA-38.mdx index 4eb685d90..53011fc0f 100644 --- a/cometbft/next/docs/qa/CometBFT-QA-38.mdx +++ b/cometbft/next/docs/qa/CometBFT-QA-38.mdx @@ -532,4 +532,4 @@ Observe that in all runs, the average number of transactions in the mempool quic [\#539]: https://github.com/cometbft/cometbft/issues/539 [\#546]: https://github.com/cometbft/cometbft/issues/546 [\#562]: https://github.com/cometbft/cometbft/issues/562 -[end-to-end]: https://github.com/cometbft/cometbft/tree/main/test/e2e +[end-to-end]: https://github.com/cometbft/cometbft/tree/v0.38.0-alpha.2/test/e2e diff --git a/cometbft/next/docs/qa/Method.mdx b/cometbft/next/docs/qa/Method.mdx index 224d76fa4..24ec01fe9 100644 --- a/cometbft/next/docs/qa/Method.mdx +++ b/cometbft/next/docs/qa/Method.mdx @@ -14,7 +14,7 @@ This baseline is then compared with results obtained in later versions. Out of the testnet-based test cases described in [the releases document][releases], we focused on two of them: _200 Node Test_ and _Rotating Nodes Test_. -[releases]: https://github.com/cometbft/cometbft/blob/v0.38.x/RELEASES.md#large-scale-testnets +[releases]: https://github.com/cometbft/cometbft/blob/v0.40.x/RELEASES.md#large-scale-testnets ## Software Dependencies @@ -153,8 +153,8 @@ The CometBFT team should improve it at every iteration to increase the amount of This script generates a series of plots per experiment and configuration that may help with visualizing latency vs throughput variation. -[`latency_throughput.py`]: https://github.com/cometbft/cometbft/tree/v0.38.x/scripts/qa/reporting#latency-vs-throughput-plotting -[`latency_plotter.py`]: https://github.com/cometbft/cometbft/tree/v0.38.x/scripts/qa/reporting#latency-vs-throughput-plotting-version-2 +[`latency_throughput.py`]: https://github.com/cometbft/cometbft/tree/v0.40.x/scripts/qa/reporting#latency-vs-throughput-plotting +[`latency_plotter.py`]: https://github.com/cometbft/cometbft/tree/v0.40.x/scripts/qa/reporting#latency-vs-throughput-plotting-version-2 #### Extracting Prometheus Metrics @@ -165,7 +165,7 @@ The CometBFT team should improve it at every iteration to increase the amount of 4. Identify the time window you want to plot in your graphs. 5. Execute the [`prometheus_plotter.py`] script for the time window. -[`prometheus_plotter.py`]: https://github.com/cometbft/cometbft/tree/v0.38.x/scripts/qa/reporting#prometheus-metrics +[`prometheus_plotter.py`]: https://github.com/cometbft/cometbft/tree/v0.40.x/scripts/qa/reporting#prometheus-metrics ## Rotating Node Testnet diff --git a/cometbft/next/spec/abci/Client-and-server.mdx b/cometbft/next/spec/abci/Client-and-server.mdx index 60da93e18..c3823c055 100644 --- a/cometbft/next/spec/abci/Client-and-server.mdx +++ b/cometbft/next/spec/abci/Client-and-server.mdx @@ -16,7 +16,7 @@ You are expected to have read all previous sections of ABCI++ specification, nam ## Message Protocol and Synchrony The message protocol consists of pairs of requests and responses defined in the -[protobuf file](https://github.com/cometbft/cometbft/blob/v0.38.x/proto/tendermint/abci/types.proto). +[protobuf file](https://github.com/cometbft/cometbft/blob/v0.40.x/proto/tendermint/abci/types.proto). Some messages have no fields, while others may include byte-arrays, strings, integers, or custom protobuf types. @@ -45,7 +45,7 @@ The implementations in CometBFT's repository can be tested using `abci-cli` by s the `--abci` flag appropriately. See examples, in various stages of maintenance, in -[Go](https://github.com/cometbft/cometbft/tree/master/abci/server), +[Go](https://github.com/cometbft/cometbft/tree/v0.40.x/abci/server), [JavaScript](https://github.com/tendermint/js-abci), and [Java](https://github.com/jTendermint/jabci). diff --git a/cometbft/next/spec/abci/CometBFTs-expected-behavior.mdx b/cometbft/next/spec/abci/CometBFTs-expected-behavior.mdx index d592fd7ee..fa3fb9fdc 100644 --- a/cometbft/next/spec/abci/CometBFTs-expected-behavior.mdx +++ b/cometbft/next/spec/abci/CometBFTs-expected-behavior.mdx @@ -119,7 +119,7 @@ Let us now examine the grammar line by line, providing further details. At the end of a successful attempt, CometBFT calls `Info` to make sure the reconstructed state's _AppHash_ matches the one in the block header at the corresponding height. Note that the state of the application does not contain vote extensions itself. The application can rely on - [CometBFT to ensure](https://github.com/cometbft/cometbft/blob/main/docs/references/rfc/rfc-100-abci-vote-extension-propag.md#base-implementation-persist-and-propagate-extended-commit-history) + [CometBFT to ensure](https://github.com/cometbft/cometbft/blob/v0.40.x/docs/references/rfc/rfc-100-abci-vote-extension-propag.md#base-implementation-persist-and-propagate-extended-commit-history) the node has all the relevant data to proceed with the execution beyond this point. >```abnf @@ -260,7 +260,7 @@ However, the application can use the existing `retain_height` parameter to decid history it wants to keep, just as is done with the block history. The network-wide implications of the usage of `retain_height` stay the same. The decision to store -historical commits and potential optimizations, are discussed in detail in [RFC-100](https://github.com/cometbft/cometbft/blob/main/docs/references/rfc/rfc-100-abci-vote-extension-propag.md#current-limitations-and-possible-implementations) +historical commits and potential optimizations, are discussed in detail in [RFC-100](https://github.com/cometbft/cometbft/blob/v0.40.x/docs/references/rfc/rfc-100-abci-vote-extension-propag.md#current-limitations-and-possible-implementations) ## Handling upgrades to ABCI 2.0 diff --git a/cometbft/next/spec/abci/Overview.mdx b/cometbft/next/spec/abci/Overview.mdx index 8783b350d..2b113abbb 100644 --- a/cometbft/next/spec/abci/Overview.mdx +++ b/cometbft/next/spec/abci/Overview.mdx @@ -22,7 +22,7 @@ for handling all ABCI++ methods. Thus, CometBFT always sends the `Request*` messages and receives the `Response*` messages in return. -All ABCI++ messages and methods are defined in [protocol buffers](https://github.com/cometbft/cometbft/blob/v0.38.x/proto/tendermint/abci/types.proto). +All ABCI++ messages and methods are defined in [protocol buffers](https://github.com/cometbft/cometbft/blob/v0.40.x/proto/tendermint/abci/types.proto). This allows CometBFT to run with applications written in many programming languages. This specification is split as follows: diff --git a/cometbft/next/spec/abci/Requirements-for-the-Application.mdx b/cometbft/next/spec/abci/Requirements-for-the-Application.mdx index 88ea52285..88f25ced5 100644 --- a/cometbft/next/spec/abci/Requirements-for-the-Application.mdx +++ b/cometbft/next/spec/abci/Requirements-for-the-Application.mdx @@ -264,9 +264,9 @@ the state for each connection, which are synchronized upon `Commit` calls. In principle, each of the four ABCI++ connections operates concurrently with one another. This means applications need to ensure access to state is thread safe. Both the -[default in-process ABCI client](https://github.com/cometbft/cometbft/blob/v0.38.x/abci/client/local_client.go#L13) +[default in-process ABCI client](https://github.com/cometbft/cometbft/blob/v0.40.x/abci/client/local_client.go#L13) and the -[default Go ABCI server](https://github.com/cometbft/cometbft/blob/v0.38.x/abci/server/socket_server.go#L20) +[default Go ABCI server](https://github.com/cometbft/cometbft/blob/v0.40.x/abci/server/socket_server.go#L20) use a global lock to guard the handling of events across all connections, so they are not concurrent at all. This means whether your app is compiled in-process with CometBFT using the `NewLocalClient`, or run out-of-process using the `SocketServer`, @@ -544,13 +544,15 @@ a given public key can only appear once within a given update. If an update incl duplicates, the block execution will fail irrecoverably. Structure `ValidatorUpdate` contains a public key, which is used to identify the validator: -The public key currently supports three types: +The public key currently supports the following types: - `ed25519` - `secp256k1` -- `bls12381` +- `secp256k1eth` +- `bls12_381` +- `ml_dsa_65` -Structure `ValidatorUpdate` also contains an `ìnt64` field denoting the validator's new power. +Structure `ValidatorUpdate` also contains an `int64` field denoting the validator's new power. Applications must ensure that `ValidatorUpdate` structures abide by the following rules: @@ -579,18 +581,19 @@ all full nodes have the same value at a given height. #### List of Parameters -These are the current consensus parameters (as of v0.38.x): +These are the current consensus parameters: 1. [ABCIParams.VoteExtensionsEnableHeight](#abciparamsvoteextensionsenableheight) -2. [BlockParams.MaxBytes](#blockparamsmaxbytes) -3. [BlockParams.MaxGas](#blockparamsmaxgas) -4. [EvidenceParams.MaxAgeDuration](#evidenceparamsmaxageduration) -5. [EvidenceParams.MaxAgeNumBlocks](#evidenceparamsmaxagenumblocks) -6. [EvidenceParams.MaxBytes](#evidenceparamsmaxbytes) -7. [ValidatorParams.PubKeyTypes](#validatorparamspubkeytypes) -8. [VersionParams.App](#versionparamsapp) +2. [AuthorityParams.Authority](#authorityparamsauthority) +3. [BlockParams.MaxBytes](#blockparamsmaxbytes) +4. [BlockParams.MaxGas](#blockparamsmaxgas) +5. [EvidenceParams.MaxAgeDuration](#evidenceparamsmaxageduration) +6. [EvidenceParams.MaxAgeNumBlocks](#evidenceparamsmaxagenumblocks) +7. [EvidenceParams.MaxBytes](#evidenceparamsmaxbytes) +8. [ValidatorParams.PubKeyTypes](#validatorparamspubkeytypes) +9. [VersionParams.App](#versionparamsapp) -#### ABCIParams.VoteExtensionsEnableHeight +##### ABCIParams.VoteExtensionsEnableHeight This parameter is either 0 or a positive height at which vote extensions become mandatory. If the value is zero (which is the default), vote @@ -609,6 +612,11 @@ include the vote extensions from height `H`. For all heights after `H` Must always be set to a future height, 0, or the same height that was previously set. Once the chain's height reaches the value set, it cannot be changed to a different value. +##### AuthorityParams.Authority + +An opaque, application-defined authority string. CometBFT does not interpret it and +only enforces a maximum length. The default is the empty string. + ##### BlockParams.MaxBytes The maximum size of a complete Protobuf encoded block. @@ -639,7 +647,10 @@ If the Application sets value -1, consensus will: Must have `MaxBytes == -1` OR `0 < MaxBytes <= 100 MB`. > Bear in mind that the default value for the `BlockParams.MaxBytes` consensus -> parameter accepts as valid blocks with size up to 21 MB. +> parameter accepts as valid blocks with size up to roughly 53 MiB: a 21 MiB +> budget for block data, plus a worst-case commit reserve sized for the maximum +> validator set at the maximum signature size. Enabling a large-signature key +> type such as `ml_dsa_65` is what makes that reserve large. > If the Application's use case does not need blocks of that size, > or if the impact (specially on bandwidth consumption and block latency) > of propagating blocks of that size was not evaluated, @@ -1023,7 +1034,7 @@ from the genesis file and light client RPC servers. It also calls `Info` to veri Once the state machine has been restored and CometBFT has gathered this additional information, it transitions to consensus. As of ABCI 2.0, CometBFT ensures the necessary conditions -to switch are met [RFC-100](https://github.com/cometbft/cometbft/blob/main/docs/references/rfc/rfc-100-abci-vote-extension-propag.md#base-implementation-persist-and-propagate-extended-commit-history). +to switch are met [RFC-100](https://github.com/cometbft/cometbft/blob/v0.40.x/docs/references/rfc/rfc-100-abci-vote-extension-propag.md#base-implementation-persist-and-propagate-extended-commit-history). From the application's point of view, these operations are transparent, unless the application has just upgraded to ABCI 2.0. In that case, the application needs to be properly configured and aware of certain constraints in terms of when to provide vote extensions. More details can be found in the section below. @@ -1036,7 +1047,7 @@ Introducing vote extensions requires changes to the configuration of the applica First of all, switching to a version of CometBFT with vote extensions, requires a coordinated upgrade. For a detailed description on the upgrade path, please refer to the corresponding -[section](https://github.com/cometbft/cometbft/blob/main/docs/references/rfc/rfc-100-abci-vote-extension-propag.md#upgrade-path) in RFC-100. +[section](https://github.com/cometbft/cometbft/blob/v0.40.x/docs/references/rfc/rfc-100-abci-vote-extension-propag.md#upgrade-path) in RFC-100. There is a newly introduced [**consensus parameter**](/cometbft/next/spec/abci/Requirements-for-the-Application#consensus-parameters): `VoteExtensionsEnableHeight`. This parameter represents the height at which vote extensions are diff --git a/cometbft/next/spec/consensus/Byzantine-Consensus-Algorithm.mdx b/cometbft/next/spec/consensus/Byzantine-Consensus-Algorithm.mdx index 1ee50a857..72043031b 100644 --- a/cometbft/next/spec/consensus/Byzantine-Consensus-Algorithm.mdx +++ b/cometbft/next/spec/consensus/Byzantine-Consensus-Algorithm.mdx @@ -16,7 +16,7 @@ order: 1 - A node is said to be _at_ a given height, round, and step, or at `(H,R,S)`, or at `(H,R)` in short to omit the step. - To _prevote_ or _precommit_ something means to broadcast a prevote - or precommit [vote](https://github.com/cometbft/cometbft/blob/af3bc47df982e271d4d340a3c5e0d773e440466d/types/vote.go#L50) + or precommit [vote](https://github.com/cometbft/cometbft/blob/v0.40.x/types/vote.go#L64-L75) for something. - A vote _at_ `(H,R)` is a vote signed with the bytes for `H` and `R` included in its [sign-bytes](/cometbft/next/spec/core/Data_structures#vote). @@ -107,7 +107,7 @@ example, - Nodes gossip prevotes for the proposed PoLC (proof-of-lock-change) round if one is proposed. - Nodes gossip to nodes lagging in blockchain height with block - [commits](https://github.com/cometbft/cometbft/blob/af3bc47df982e271d4d340a3c5e0d773e440466d/types/block.go#L738) + [commits](https://github.com/cometbft/cometbft/blob/v0.40.x/types/block.go#L852-L866) for older blocks. - Nodes opportunistically gossip `ReceivedVote` messages to hint peers what votes it already has. @@ -122,7 +122,7 @@ A proposal is signed and published by the designated proposer at each round. The proposer is chosen by a deterministic and non-choking round robin selection algorithm that selects proposers in proportion to their voting power (see -[implementation](https://github.com/cometbft/cometbft/blob/af3bc47df982e271d4d340a3c5e0d773e440466d/types/validator_set.go#L51)). +[implementation](https://github.com/cometbft/cometbft/blob/v0.40.x/types/validator_set.go#L56-L65)). A proposal at `(H,R)` is composed of a block and an optional latest `PoLC-Round < R` which is included iff the proposer knows of one. This @@ -294,7 +294,7 @@ may make JSet verification/gossip logic easier to implement. ### Censorship Attacks Due to the definition of a block -[commit](https://github.com/cometbft/cometbft/blob/v0.38.x/docs/core/validators.md), any 1/3+ coalition of +[commit](https://github.com/cometbft/cometbft/blob/v0.40.x/docs/core/validators.md), any 1/3+ coalition of validators can halt the blockchain by not broadcasting their votes. Such a coalition can also censor particular transactions by rejecting blocks that include these transactions, though this would result in a diff --git a/cometbft/next/spec/consensus/Evidence.mdx b/cometbft/next/spec/consensus/Evidence.mdx index 3206fd054..9cf3a0492 100644 --- a/cometbft/next/spec/consensus/Evidence.mdx +++ b/cometbft/next/spec/consensus/Evidence.mdx @@ -53,11 +53,11 @@ different, more lightweight verification method they are subject to a different kind of 1/3+ attack whereby the byzantine validators could sign an alternative light block that the light client will think is valid. Detection, explained in greater detail -[here](https://github.com/cometbft/cometbft/blob/main/spec/light-client/detection/detection_003_reviewed.md), involves comparison +[here](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/light-client/detection/detection_003_reviewed.md), involves comparison with multiple other nodes in the hope that at least one is "honest". An "honest" node will return a challenging light block for the light client to validate. If this challenging light block also meets the -[validation criteria](https://github.com/cometbft/cometbft/blob/main/spec/light-client/verification/verification_001_published.md) +[validation criteria](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/light-client/verification/verification_001_published.md) then the light client sends the "forged" light block to the node. [Verification](#lightclientattackevidence) is addressed further down. diff --git a/cometbft/next/spec/consensus/WAL.mdx b/cometbft/next/spec/consensus/WAL.mdx index eb7c758bc..00ad59bcc 100644 --- a/cometbft/next/spec/consensus/WAL.mdx +++ b/cometbft/next/spec/consensus/WAL.mdx @@ -11,7 +11,7 @@ It also issues fsync syscall through node (to prevent double signing). Under the hood, it uses -[autofile.Group](https://github.com/cometbft/cometbft/blob/af3bc47df982e271d4d340a3c5e0d773e440466d/libs/autofile/group.go#L54), +[autofile.Group](https://github.com/cometbft/cometbft/blob/v0.40.x/libs/autofile/group.go#L56-L79), which rotates files when those get too big (> 10MB). The total maximum size is 1GB. We only need the latest block and the block before it, @@ -32,5 +32,5 @@ WAL. Then it will go to precommit, and that time it will work because the private validator contains the `LastSignBytes` and then we’ll replay the precommit from the WAL. -Make sure to read about [WAL corruption](https://github.com/cometbft/cometbft/blob/v0.38.x/docs/core/running-in-production.md#wal-corruption) +Make sure to read about [WAL corruption](https://github.com/cometbft/cometbft/blob/v0.40.x/docs/core/running-in-production.md#wal-corruption) and recovery strategies. diff --git a/cometbft/next/spec/consensus/light-client/accountability.md b/cometbft/next/spec/consensus/light-client/accountability.md index 3907e8d47..d6e49444f 100644 --- a/cometbft/next/spec/consensus/light-client/accountability.md +++ b/cometbft/next/spec/consensus/light-client/accountability.md @@ -1,3 +1,3 @@ # Fork accountability -Deprecated, please see [light-client/accountability](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/accountability). +Deprecated, please see [light-client/accountability](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/light-client/accountability). diff --git a/cometbft/next/spec/consensus/light-client/detection.md b/cometbft/next/spec/consensus/light-client/detection.md index 9e70726c7..d9c54e042 100644 --- a/cometbft/next/spec/consensus/light-client/detection.md +++ b/cometbft/next/spec/consensus/light-client/detection.md @@ -1,3 +1,3 @@ # Detection -Deprecated, please see [light-client/detection](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/detection). +Deprecated, please see [light-client/detection](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/light-client/detection). diff --git a/cometbft/next/spec/consensus/light-client/verification.md b/cometbft/next/spec/consensus/light-client/verification.md index d0e2bf1e5..c6bd06f7f 100644 --- a/cometbft/next/spec/consensus/light-client/verification.md +++ b/cometbft/next/spec/consensus/light-client/verification.md @@ -1,3 +1,3 @@ # Core Verification -Deprecated, please see [light-client/verification](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/verification). +Deprecated, please see [light-client/verification](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/light-client/verification). diff --git a/cometbft/next/spec/consensus/proposer-based-timestamp/pbts-sysmodel_001_draft.md b/cometbft/next/spec/consensus/proposer-based-timestamp/pbts-sysmodel_001_draft.md index 06f9e8ea5..ac30156ed 100644 --- a/cometbft/next/spec/consensus/proposer-based-timestamp/pbts-sysmodel_001_draft.md +++ b/cometbft/next/spec/consensus/proposer-based-timestamp/pbts-sysmodel_001_draft.md @@ -188,4 +188,4 @@ Back to [main document][main]. [arXiv]: https://arxiv.org/abs/1807.04938 -[CMBC-FM-2THIRDS-link]: https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/verification/verification_002_draft.md#cmbc-fm-2thirds1 +[CMBC-FM-2THIRDS-link]: https://github.com/cometbft/cometbft/blob/v0.40.x/spec/light-client/verification/verification_002_draft.md#cmbc-fm-2thirds1 diff --git a/cometbft/next/spec/core/Data_structures.mdx b/cometbft/next/spec/core/Data_structures.mdx index 1272a2bb3..100a0aa74 100644 --- a/cometbft/next/spec/core/Data_structures.mdx +++ b/cometbft/next/spec/core/Data_structures.mdx @@ -51,7 +51,7 @@ and a list of evidence of malfeasance (ie. signing conflicting votes). | Name | Type | Description | Validation | |--------|-------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------| | Header | [Header](#header) | Header corresponding to the block. This field contains information used throughout consensus and other areas of the protocol. To find out what it contains, visit [header](#header) | Must adhere to the validation rules of [header](#header) | -| Data | [Data](#data) | Data contains a list of transactions. The contents of the transaction is unknown to CometBFT. | This field can be empty or populated, but no validation is performed. Applications can perform validation on individual transactions prior to block creation using [checkTx](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/abci/abci%2B%2B_methods.md#checktx). +| Data | [Data](#data) | Data contains a list of transactions. The contents of the transaction is unknown to CometBFT. | This field can be empty or populated, but no validation is performed. Applications can perform validation on individual transactions prior to block creation using [checkTx](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/abci/abci%2B%2B_methods.md#checktx). | Evidence | [EvidenceList](#evidencelist) | Evidence contains a list of infractions committed by validators. | Can be empty, but when populated the validations rules from [evidenceList](#evidencelist) apply | | LastCommit | [Commit](#commit) | `LastCommit` includes one vote for every validator. All votes must either be for the previous block, nil or absent. If a vote is for the previous block it must have a valid signature from the corresponding validator. The sum of the voting power of the validators that voted must be greater than 2/3 of the total voting power of the complete validator set. The number of votes in a commit is limited to 10000 (see `types.MaxVotesCount`). | Must be empty for the initial height and must adhere to the validation rules of [commit](#commit). | @@ -145,7 +145,7 @@ versioning that this can refer to) | Name | type | Description | Validation | |-------|--------|---------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------| | Block | uint64 | This number represents the block version and must be the same throughout an operational network | Must be equal to block version being used in a network (`block.Version.Block == state.Version.Consensus.Block`) | -| App | uint64 | App version is decided on by the application. Read [here](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/abci/abci++_app_requirements.md) | `block.Version.App == state.Version.Consensus.App` | +| App | uint64 | App version is decided on by the application. Read [here](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/abci/abci++_app_requirements.md) | `block.Version.App == state.Version.Consensus.App` | ## BlockID @@ -225,7 +225,7 @@ to reconstruct the vote set given the validator set. | Signature | [Signature](#signature) | Signature corresponding to the validators participation in consensus. | The length of the signature must be > 0 and < than 64 | NOTE: `ValidatorAddress` and `Timestamp` fields may be removed in the future -(see [ADR-25](https://github.com/cometbft/cometbft/blob/main/docs/references/architecture/tendermint-core/adr-025-commit.md)). +(see [ADR-25](https://github.com/cometbft/cometbft/blob/v0.40.x/docs/references/architecture/tendermint-core/adr-025-commit.md)). ## ExtendedCommitSig diff --git a/cometbft/next/spec/core/encoding.mdx b/cometbft/next/spec/core/encoding.mdx index 581d40ed8..378bb35c3 100644 --- a/cometbft/next/spec/core/encoding.mdx +++ b/cometbft/next/spec/core/encoding.mdx @@ -19,7 +19,7 @@ For details on varints, see the [protobuf spec](https://developers.google.com/protocol-buffers/docs/encoding#varints). For example, the byte-array `[0xA, 0xB]` would be encoded as `0x020A0B`, -while a byte-array containing 300 entires beginning with `[0xA, 0xB, ...]` would +while a byte-array containing 300 entries beginning with `[0xA, 0xB, ...]` would be encoded as `0xAC020A0B...` where `0xAC02` is the UVarint encoding of 300. ## Hashing @@ -41,7 +41,7 @@ include details of the private keys beyond their type and name. ### Key Types -Each type specifies it's own pubkey, address, and signature format. +Each type specifies its own pubkey, address, and signature format. #### Ed25519 @@ -59,7 +59,31 @@ CometBFT adopts [zip215](https://zips.z.cash/zip-0215) for verification of ed255 #### Secp256k1 -The address is the first 20-bytes of the SHA256 hash of the raw 32-byte public key: +The address is the RIPEMD160 hash of the SHA256 hash of the raw 33-byte compressed public key. RIPEMD160 produces 20 bytes, so there is no truncation step: + +```go +address = RIPEMD160(SHA256(pubkey)) +``` + +#### Secp256k1Eth + +This key type is compatible with go-ethereum. The public key is a 33-byte compressed SEC1 key. + +The address is the last 20 bytes of the legacy Keccak-256 hash of the uncompressed public key: + +```go +address = Keccak256(uncompressedPubKey[1:])[12:] +``` + +The signature is a 65-byte go-ethereum signature in `[R || S || V]` form. Verification requires exactly 65 bytes, a canonical lower-S value, and a recovery byte `V` of `0` or `1`. + +#### ML-DSA-65 + +This key type is the NIST ML-DSA-65 post-quantum signature scheme (FIPS 204). The public key is 1952 bytes, and the signature is 3309 bytes. + +For creating, enabling, and rotating ML-DSA-65 validator consensus keys, see [the post-quantum key guides](/sdk/next/keys/post-quantum-keys). + +The address is the first 20 bytes of the SHA256 hash of the public key, matching the Ed25519 convention: ```go address = SHA256(pubkey)[:20] diff --git a/cometbft/next/spec/core/genesis.mdx b/cometbft/next/spec/core/genesis.mdx index 649418959..c2cc877f0 100644 --- a/cometbft/next/spec/core/genesis.mdx +++ b/cometbft/next/spec/core/genesis.mdx @@ -25,7 +25,7 @@ The genesis file is the starting point of a chain. An application will populate > Note: For evidence to be considered invalid, evidence must be older than both `max_age_num_blocks` and `max_age_duration` - `validator` - - `pub_key_types`: Defines which curves are to be accepted as a valid validator consensus key. CometBFT supports ed25519, secp256k1, and bls12381. + - `pub_key_types`: Defines which curves are to be accepted as a valid validator consensus key. CometBFT supports ed25519, secp256k1, secp256k1eth, bls12_381, and ml_dsa_65. The default is ed25519 alone, and bls12_381 requires a binary built with the `bls12381` build tag. - `version` - `app_version`: The version of the application. This is set by the application and is used to identify which version of the app a user should be using in order to operate a node. diff --git a/cometbft/next/spec/light-client/Fork-Detection.mdx b/cometbft/next/spec/light-client/Fork-Detection.mdx index c38c50f27..2edaff3eb 100644 --- a/cometbft/next/spec/light-client/Fork-Detection.mdx +++ b/cometbft/next/spec/light-client/Fork-Detection.mdx @@ -15,13 +15,13 @@ This directory captures the ongoing work and discussion on fork detection both in the context of a Cosmos light node and in the context of IBC. It contains the following files -### [detection.md](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/detection/detection_003_reviewed.md) +### [detection.md](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/light-client/detection/detection_003_reviewed.md) a draft of the light node fork detection including "proof of fork" definition, that is, the data structure to submit evidence to full nodes. -### [discussions.md](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/detection/discussions.md) +### [discussions.md](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/light-client/detection/discussions.md) A collection of ideas and intuitions from recent discussions @@ -30,14 +30,14 @@ A collection of ideas and intuitions from recent discussions which fork detection happens - a discussion about lightstore semantics -### [req-ibc-detection.md](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/detection/req-ibc-detection.md) +### [req-ibc-detection.md](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/light-client/detection/req-ibc-detection.md) - a collection of requirements for fork detection in the IBC context. In particular it contains a section "Required Changes in ICS 007" with necessary updates to ICS 007 to support Cosmos fork detection -### [draft-functions.md](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/detection/draft-functions.md) +### [draft-functions.md](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/light-client/detection/draft-functions.md) In order to address the collected requirements, we started to sketch some functions that we will need in the future when we specify in more diff --git a/cometbft/next/spec/light-client/Light-Client-Specification.mdx b/cometbft/next/spec/light-client/Light-Client-Specification.mdx index f61346dc0..8a10b9249 100644 --- a/cometbft/next/spec/light-client/Light-Client-Specification.mdx +++ b/cometbft/next/spec/light-client/Light-Client-Specification.mdx @@ -7,7 +7,7 @@ order: 1 This directory contains work-in-progress English and TLA+ specifications for the Light Client protocol. Implementations of the light client can be found in [Rust](https://github.com/informalsystems/tendermint-rs/tree/master/light-client) and -[Go](https://github.com/cometbft/cometbft/tree/v0.38.x/light). +[Go](https://github.com/cometbft/cometbft/tree/v0.40.x/light). Light clients are assumed to be initialized once from a trusted source with a trusted header and validator set. The light client @@ -26,10 +26,10 @@ In case a lightclient attack is detected, the lightclient submits evidence to a ## Commit Verification -The [English specification](https://github.com/cometbft/cometbft/blob/main/spec/light-client/verification/verification_001_published.md) describes the light client +The [English specification](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/light-client/verification/verification_001_published.md) describes the light client commit verification problem in terms of the temporal properties -[LCV-DIST-SAFE.1](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/verification/verification_001_published.md#lcv-dist-safe1) and -[LCV-DIST-LIVE.1](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/verification/verification_001_published.md#lcv-dist-live1). +[LCV-DIST-SAFE.1](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/light-client/verification/verification_001_published.md#lcv-dist-safe1) and +[LCV-DIST-LIVE.1](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/light-client/verification/verification_001_published.md#lcv-dist-live1). Commit verification is assumed to operate within the Cosmos Failure Model, where +2/3 of validators are correct for some time period and validator sets can change arbitrarily at each height. @@ -41,18 +41,18 @@ many intermediate headers by exploiting overlap in trusted and untrusted validat When there is not enough overlap, a bisection routine can be used to find a minimal set of headers that do provide the required overlap. -The [TLA+ specification ver. 001](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/verification/Lightclient_A_1.tla) +The [TLA+ specification ver. 001](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/light-client/verification/Lightclient_A_1.tla) is a formal description of the commit verification protocol executed by a client, including the safety and termination, which can be model checked with Apalache. A more detailed TLA+ specification of -[Light client verification ver. 003](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/verification/Lightclient_003_draft.tla) +[Light client verification ver. 003](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/light-client/verification/Lightclient_003_draft.tla) is currently under peer review. The `MC*.tla` files contain concrete parameters for the -[TLA+ specification](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/verification/Lightclient_A_1.tla), in order to do model checking. -For instance, [MC4_3_faulty.tla](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/verification/MC4_3_faulty.tla) contains the following parameters +[TLA+ specification](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/light-client/verification/Lightclient_A_1.tla), in order to do model checking. +For instance, [MC4_3_faulty.tla](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/light-client/verification/MC4_3_faulty.tla) contains the following parameters for the nodes, heights, the trusting period, the clock drifts, correctness of the primary node, and the ratio of the faulty processes: @@ -84,8 +84,8 @@ $DIR/apalache-tests/scripts/parse-logs.py --human . All lines in `results.csv` should report `Deadlock`, which means that the algorithm has terminated and no invariant violation was found. -Similar to [002bmc-apalache-ok.csv](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/verification/002bmc-apalache-ok.csv), -file [003bmc-apalache-error.csv](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/verification/003bmc-apalache-error.csv) specifies +Similar to [002bmc-apalache-ok.csv](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/light-client/verification/002bmc-apalache-ok.csv), +file [003bmc-apalache-error.csv](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/light-client/verification/003bmc-apalache-error.csv) specifies the set of experiments that should result in counterexamples: ```sh @@ -97,7 +97,7 @@ All lines in `results.csv` should report `Error`. The following table summarizes the experimental results for Light client verification version 001. The TLA+ properties can be found in the -[TLA+ specification](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/verification/Lightclient_A_1.tla). +[TLA+ specification](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/light-client/verification/Lightclient_A_1.tla). The experiments were run in an AWS instance equipped with 32GB RAM and a 4-core Intel® Xeon® CPU E5-2686 v4 @ 2.30GHz CPU. We write “`✗=k`” when a bug is reported at depth k, and “`✓<=k`” when @@ -109,7 +109,7 @@ The experimental results for version 003 are to be added. ## Attack Detection -The [English specification](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/detection/detection_003_reviewed.md) +The [English specification](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/light-client/detection/detection_003_reviewed.md) defines light client attacks (and how they differ from blockchain forks), and describes the problem of a light client detecting these attacks by communicating with a network of full nodes, @@ -121,19 +121,19 @@ protocol matches corresponding headers provided by the secondaries. If this is not the case, the protocol analyses the verification traces of the involved full nodes and generates -[evidence](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/detection/detection_003_reviewed.md#cmbc-lc-evidence-data1) +[evidence](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/light-client/detection/detection_003_reviewed.md#cmbc-lc-evidence-data1) of misbehavior that can be submitted to a full node so that the faulty validators can be punished. -The [TLA+ specification](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/detection/LCDetector_003_draft.tla) +The [TLA+ specification](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/light-client/detection/LCDetector_003_draft.tla) is a formal description of the detection protocol for two peers, including the safety and termination, which can be model checked with Apalache. The `LCD_MC*.tla` files contain concrete parameters for the -[TLA+ specification](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/detection/LCDetector_003_draft.tla), +[TLA+ specification](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/light-client/detection/LCDetector_003_draft.tla), in order to run the model checker. -For instance, [LCD_MC4_4_faulty.tla](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/verification/MC4_4_faulty.tla) +For instance, [LCD_MC4_4_faulty.tla](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/light-client/verification/MC4_4_faulty.tla) contains the following parameters for the nodes, heights, the trusting period, the clock drifts, correctness of the nodes, and the ratio of the faulty processes: @@ -167,8 +167,8 @@ $DIR/apalache-tests/scripts/parse-logs.py --human . All lines in `results.csv` should report `Deadlock`, which means that the algorithm has terminated and no invariant violation was found. -Similar to [004bmc-apalache-ok.csv](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/verification/004bmc-apalache-ok.csv), -file [005bmc-apalache-error.csv](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/verification/005bmc-apalache-error.csv) specifies +Similar to [004bmc-apalache-ok.csv](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/light-client/verification/004bmc-apalache-ok.csv), +file [005bmc-apalache-error.csv](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/light-client/verification/005bmc-apalache-error.csv) specifies the set of experiments that should result in counterexamples: ```sh @@ -182,21 +182,21 @@ The detailed experimental results are to be added soon. ## Accountability -The [English specification](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/attacks/isolate-attackers_002_reviewed.md) -defines the protocol that is executed on a full node upon receiving attack [evidence](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/attacks/isolate-attackers_002_reviewed.md#cmbc-lc-evidence-data1) from a lightclient. In particular, the protocol handles three types of attacks +The [English specification](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/light-client/attacks/isolate-attackers_002_reviewed.md) +defines the protocol that is executed on a full node upon receiving attack [evidence](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/light-client/attacks/isolate-attackers_002_reviewed.md#cmbc-lc-evidence-data1) from a lightclient. In particular, the protocol handles three types of attacks - lunatic - equivocation - amnesia -We discussed in the [last part](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/attacks/isolate-attackers_002_reviewed.md#Part-III---Completeness) of the English specification +We discussed in the [last part](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/light-client/attacks/isolate-attackers_002_reviewed.md#Part-III---Completeness) of the English specification that the non-lunatic cases are defined by having the same validator set in the conflicting blocks. For these cases, computer-aided analysis of [Tendermint Consensus in TLA+](/cometbft/next/spec/light-client/Accountability) shows that equivocation and amnesia capture all non-lunatic attacks. -The [TLA+ specification](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/attacks/Isolation_001_draft.tla) +The [TLA+ specification](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/light-client/attacks/Isolation_001_draft.tla) is a formal description of the protocol, including the safety property, which can be model checked with Apalache. -Similar to the other specifications, [MC_5_3.tla](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/attacks/MC_5_3.tla) contains concrete parameters to run the model checker. The specification can be checked within seconds. +Similar to the other specifications, [MC_5_3.tla](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/light-client/attacks/MC_5_3.tla) contains concrete parameters to run the model checker. The specification can be checked within seconds. [tendermint-accountability](/cometbft/next/spec/light-client/Accountability) diff --git a/cometbft/next/spec/p2p/Implementation-of-the-p2p-layer.mdx b/cometbft/next/spec/p2p/Implementation-of-the-p2p-layer.mdx index 96db49a0f..d040624f8 100644 --- a/cometbft/next/spec/p2p/Implementation-of-the-p2p-layer.mdx +++ b/cometbft/next/spec/p2p/Implementation-of-the-p2p-layer.mdx @@ -24,21 +24,21 @@ documentation also applies to the releases `v0.37.*` and `v0.38.*` [^v35]. ## Contents The documentation follows the organization of the -[`p2p` package](https://github.com/cometbft/cometbft/tree/v0.34.x/p2p), +[`p2p` package](https://github.com/cometbft/cometbft/tree/v0.40.x/p2p), which implements the following abstractions: -- [Transport](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/p2p/implementation/transport.md): establishes secure and authenticated +- [Transport](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/p2p/implementation/transport.md): establishes secure and authenticated connections with peers; -- [Switch](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/p2p/implementation/switch.md): responsible for dialing peers and accepting +- [Switch](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/p2p/implementation/switch.md): responsible for dialing peers and accepting connections from peers, for managing established connections, and for routing messages between the reactors and peers, that is, between local and remote instances of the CometBFT protocols; -- [PEX Reactor](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/p2p/implementation/pex.md): due to the several roles of this component, the +- [PEX Reactor](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/p2p/implementation/pex.md): due to the several roles of this component, the documentation is split in several parts: - - [Peer Exchange protocol](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/p2p/implementation/pex-protocol.md): enables nodes to exchange peer addresses, thus implementing a peer discovery service; - - [Address Book](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/p2p/implementation/addressbook.md): stores discovered peer addresses and + - [Peer Exchange protocol](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/p2p/implementation/pex-protocol.md): enables nodes to exchange peer addresses, thus implementing a peer discovery service; + - [Address Book](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/p2p/implementation/addressbook.md): stores discovered peer addresses and quality metrics associated to peers with which the node has interacted; - - [Peer Manager](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/p2p/implementation/peer_manager.md): defines when and to which peers a node + - [Peer Manager](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/p2p/implementation/peer_manager.md): defines when and to which peers a node should dial, in order to establish outbound connections; -- [Types](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/p2p/implementation/types.md) and [Configuration](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/p2p/implementation/configuration.md) provide a list of +- [Types](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/p2p/implementation/types.md) and [Configuration](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/p2p/implementation/configuration.md) provide a list of existing types and configuration parameters used by the p2p package. diff --git a/cometbft/next/spec/p2p/implementation/switch.md b/cometbft/next/spec/p2p/implementation/switch.md index e87985336..4056b7dfc 100644 --- a/cometbft/next/spec/p2p/implementation/switch.md +++ b/cometbft/next/spec/p2p/implementation/switch.md @@ -50,11 +50,11 @@ The `DialPeersAsync` method receives a list of peer addresses (strings) and dials all of them in parallel. It is invoked in two situations: -- In the [setup](https://github.com/cometbft/cometbft/blob/v0.34.x/node/node.go#L987) +- In the [setup](https://github.com/cometbft/cometbft/blob/v0.40.x/node/node.go#L712) of a node, to establish connections with every configured persistent peer - In the RPC package, to implement two unsafe RPC commands, not used in production: - [`DialSeeds`](https://github.com/cometbft/cometbft/blob/v0.34.x/rpc/core/net.go#L47) and - [`DialPeers`](https://github.com/cometbft/cometbft/blob/v0.34.x/rpc/core/net.go#L87) + [`DialSeeds`](https://github.com/cometbft/cometbft/blob/v0.40.x/rpc/core/net.go#L51) and + [`DialPeers`](https://github.com/cometbft/cometbft/blob/v0.40.x/rpc/core/net.go#L94) The received list of peer addresses to dial is parsed into `NetAddress` instances. In case of parsing errors, the method returns. An exception is made for diff --git a/cometbft/next/spec/p2p/implementation/transport.md b/cometbft/next/spec/p2p/implementation/transport.md index 121e48467..c9b89b225 100644 --- a/cometbft/next/spec/p2p/implementation/transport.md +++ b/cometbft/next/spec/p2p/implementation/transport.md @@ -43,9 +43,9 @@ The `NetAddress` method exports the listen address configured for the transport. The maximum number of simultaneous incoming connections accepted by the listener is bound to `MaxNumInboundPeer` plus the configured number of unconditional peers, using the `MultiplexTransportMaxIncomingConnections` option, -in the node [initialization](https://github.com/cometbft/cometbft/blob/v0.34.x/node/node.go#L563). +in the node [initialization](https://github.com/cometbft/cometbft/blob/v0.40.x/node/setup.go#L459). -This method is called when a node is [started](https://github.com/cometbft/cometbft/blob/v0.34.x/node/node.go#L974). +This method is called when a node is [started](https://github.com/cometbft/cometbft/blob/v0.40.x/node/node.go#L699). In case of errors, the `acceptPeers` routine is not started and the error is returned. ## Accept @@ -190,7 +190,7 @@ an `ErrRejected` error with reason `isIncompatible` is returned. The `Close` method closes the TCP listener created by the `Listen` method, and sends a signal for interrupting the `acceptPeers` routine. -This method is called when a node is [stopped](https://github.com/cometbft/cometbft/blob/v0.34.x/node/node.go#L1023). +This method is called when a node is [stopped](https://github.com/cometbft/cometbft/blob/v0.40.x/node/node.go#L764). ## Cleanup diff --git a/cometbft/next/spec/p2p/reactor-api/API-for-Reactors.mdx b/cometbft/next/spec/p2p/reactor-api/API-for-Reactors.mdx index 458f4c2fc..c7ba959f6 100644 --- a/cometbft/next/spec/p2p/reactor-api/API-for-Reactors.mdx +++ b/cometbft/next/spec/p2p/reactor-api/API-for-Reactors.mdx @@ -324,11 +324,11 @@ could not be enqueued, because the channel's send queue is still full, after a The `TrySend()` method is a _non-blocking_ method, it _immediately_ returns `false` when the channel's send queue is full. -[peer-interface]: https://github.com/cometbft/cometbft/blob/v0.38.x/p2p/peer.go -[service-interface]: https://github.com/cometbft/cometbft/blob/v0.38.x/libs/service/service.go -[switch-type]: https://github.com/cometbft/cometbft/blob/v0.38.x/p2p/switch.go +[peer-interface]: https://github.com/cometbft/cometbft/blob/v0.40.x/p2p/peer.go +[service-interface]: https://github.com/cometbft/cometbft/blob/v0.40.x/libs/service/service.go +[switch-type]: https://github.com/cometbft/cometbft/blob/v0.40.x/p2p/switch.go -[reactor-interface]: https://github.com/cometbft/cometbft/blob/v0.38.x/p2p/base_reactor.go +[reactor-interface]: https://github.com/cometbft/cometbft/blob/v0.40.x/p2p/base_reactor.go [reactor-registration]: /cometbft/next/spec/p2p/reactor-api/Reactor-Api#registration [reactor-channels]: /cometbft/next/spec/p2p/reactor-api/Reactor-Api#registration [reactor-addpeer]: /cometbft/next/spec/p2p/reactor-api/Reactor-Api#peer-management diff --git a/cometbft/next/spec/p2p/reactor-api/Reactor-Api.mdx b/cometbft/next/spec/p2p/reactor-api/Reactor-Api.mdx index 0dca9a0af..d4a61aeea 100644 --- a/cometbft/next/spec/p2p/reactor-api/Reactor-Api.mdx +++ b/cometbft/next/spec/p2p/reactor-api/Reactor-Api.mdx @@ -15,7 +15,7 @@ invoked and determines what the p2p layer expects from a reactor, this documentation focuses on the **temporal behaviour** that a reactor implementation should expect from the p2p layer. (That is, in which orders the functions may be called) -This specification is accompanied by the [`reactor.qnt`](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/p2p/reactor-api/reactor.qnt) file, +This specification is accompanied by the [`reactor.qnt`](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/p2p/reactor-api/reactor.qnt) file, a more comprehensive model of the reactor's operation written in [Quint][quint-repo], an executable specification language. The methods declared in the [`Reactor`][reactor-interface] interface are @@ -106,7 +106,7 @@ documented in the companion [API for Reactors](/cometbft/next/spec/p2p/reactor-a ## Service interface -A reactor must implement the [`Service`](https://github.com/cometbft/cometbft/blob/v0.38.x/libs/service/service.go) interface, +A reactor must implement the [`Service`](https://github.com/cometbft/cometbft/blob/v0.40.x/libs/service/service.go) interface, in particular, a startup `OnStart()` and a shutdown `OnStop()` methods: ```abnf @@ -230,5 +230,5 @@ Two important observations regarding the implementation of the `Receive` method: In other words, while `Receive` does not return, other messages from the same sender are not delivered to any reactor. -[reactor-interface]: https://github.com/cometbft/cometbft/blob/v0.38.x/p2p/base_reactor.go +[reactor-interface]: https://github.com/cometbft/cometbft/blob/v0.40.x/p2p/base_reactor.go [quint-repo]: https://github.com/informalsystems/quint diff --git a/cometbft/next/spec/p2p/reactor-api/Reactors.mdx b/cometbft/next/spec/p2p/reactor-api/Reactors.mdx index 904d3c9ad..4127151cc 100644 --- a/cometbft/next/spec/p2p/reactor-api/Reactors.mdx +++ b/cometbft/next/spec/p2p/reactor-api/Reactors.mdx @@ -44,4 +44,4 @@ The remaining of the documentation is organized as follows: layer to the reactors, through the `Switch` and `Peer` abstractions. In other words, the interaction of the protocol layer with the p2p layer (top-down). -[reactor-interface]: https://github.com/cometbft/cometbft/blob/v0.38.x/p2p/base_reactor.go +[reactor-interface]: https://github.com/cometbft/cometbft/blob/v0.40.x/p2p/base_reactor.go diff --git a/cometbft/v0.39/api-reference/rpc/index.mdx b/cometbft/v0.39/api-reference/rpc/index.mdx new file mode 100644 index 000000000..583f9a05d --- /dev/null +++ b/cometbft/v0.39/api-reference/rpc/index.mdx @@ -0,0 +1,220 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/api-reference/rpc/index' +title: "CometBFT RPC" +description: "The RPC server provides an API for interacting with a CometBFT node, including blockchain data, transaction broadcasting, and node information." +--- + +CometBFT RPC is a remote procedure call protocol that provides access to blockchain data, transaction broadcasting, validator information, and node status. The RPC server supports multiple transport protocols to accommodate different use cases. + +CometBFT supports the following RPC protocols: + +* **URI over HTTP** - REST-like interface for simple queries +* **JSONRPC over HTTP** - Standard JSON-RPC 2.0 protocol +* **JSONRPC over WebSockets** - Persistent connection with subscription support + +## Configuration + +RPC can be configured by tuning parameters under the `[rpc]` section in the `$CMTHOME/config/config.toml` file or by using the `--rpc.X` command-line flags. + +### Default Settings + +The default RPC listen address is `tcp://127.0.0.1:26657`. To set another address, update the `laddr` config parameter: + +```toml +[rpc] + +# TCP or UNIX socket address for the RPC server to listen on +laddr = "tcp://127.0.0.1:26657" + +# A list of origins a cross-domain request can be executed from +# Default value '[]' disables cors support +# Use '["*"]' to allow any origin +cors_allowed_origins = [] + +# A list of methods the client is allowed to use with cross-domain requests +cors_allowed_methods = ["HEAD", "GET", "POST"] + +# A list of non simple headers the client is allowed to use with cross-domain requests +cors_allowed_headers = ["Origin", "Accept", "Content-Type", "X-Requested-With", "X-Server-Time"] +``` + +### CORS Configuration + +CORS (Cross-Origin Resource Sharing) can be enabled by setting the following config parameters: +- `cors_allowed_origins` - List of allowed origin domains +- `cors_allowed_methods` - HTTP methods allowed for CORS requests +- `cors_allowed_headers` - Headers allowed in CORS requests + +## Protocol Examples + +### URI over HTTP + +A REST-like interface for simple queries: + +```bash +# Get block at height 5 +curl http://localhost:26657/block?height=5 + +# Get node status +curl http://localhost:26657/status + +# Get validators at height 1 +curl http://localhost:26657/validators?height=1 +``` + +### JSONRPC over HTTP + +JSONRPC requests can be POST'd to the root RPC endpoint: + +```bash +# Get block at height 5 +curl --header "Content-Type: application/json" \ + --request POST \ + --data '{"method": "block", "params": ["5"], "id": 1}' \ + http://localhost:26657 + +# Broadcast a transaction +curl --header "Content-Type: application/json" \ + --request POST \ + --data '{"method": "broadcast_tx_sync", "params": [""], "id": 1}' \ + http://localhost:26657 +``` + +Response format: +```json +{ + "jsonrpc": "2.0", + "id": 1, + "result": { + "height": "5", + "hash": "...", + "time": "..." + } +} +``` + +### JSONRPC over WebSockets + +JSONRPC requests can also be made via WebSocket for real-time updates. The WebSocket endpoint is at `/websocket`, e.g. `localhost:26657/websocket`. + + +Asynchronous RPC functions like event `subscribe` and `unsubscribe` are **only available via WebSockets**. + + +#### Subscribing to Events + +Using the [websocat](https://github.com/vi/websocat) tool, you can subscribe to 'NewBlock' events: + +```bash +echo '{ + "jsonrpc": "2.0", + "method": "subscribe", + "id": 0, + "params": { + "query": "tm.event='\''NewBlock'\''" + } +}' | websocat -n -t ws://127.0.0.1:26657/websocket +``` + +You'll receive notifications as new events occur: + +```json +{ + "jsonrpc": "2.0", + "id": 0, + "result": { + "query": "tm.event='NewBlock'", + "data": { + "type": "tendermint/event/NewBlock", + "value": { + "block": { ... }, + "result_begin_block": { ... }, + "result_end_block": { ... } + } + } + } +} +``` + +#### Available Event Types + +You can subscribe to the following event types: +- `NewBlock` - Emitted when a new block is committed +- `NewBlockHeader` - Emitted for new block headers +- `Tx` - Emitted for transactions +- `ValidatorSetUpdates` - Emitted when the validator set changes + +## Endpoint Categories + +The CometBFT RPC API is organized into several categories: + +| Category | Description | Example Methods | +|----------|-------------|-----------------| +| **Info** | Node information and blockchain data | `status`, `health`, `net_info`, `blockchain`, `block` | +| **Tx** | Transaction broadcasting and queries | `broadcast_tx_sync`, `broadcast_tx_async`, `tx`, `tx_search` | +| **ABCI** | Application Blockchain Interface queries | `abci_info`, `abci_query` | +| **Evidence** | Evidence of misbehavior | `broadcast_evidence` | +| **Unsafe** | Administrative operations (requires manual enablement) | `dial_seeds`, `dial_peers`, `unsafe_flush_mempool` | + + +**Unsafe Methods**: Methods in the "Unsafe" category can affect node operation and must be manually enabled in the configuration. They should only be used by node operators who understand the implications. + + +## Arguments + +Arguments which expect strings or byte arrays may be passed as: +- Quoted strings: `"abc"` +- `0x`-prefixed hex strings: `0x616263` + +## Common Use Cases + +### Querying Blockchain Data + +```bash +# Get the latest block +curl http://localhost:26657/block + +# Get block at specific height +curl http://localhost:26657/block?height=100 + +# Search for transactions +curl 'http://localhost:26657/tx_search?query="tx.height>100"&prove=false' +``` + +### Broadcasting Transactions + +```bash +# Synchronous broadcast (waits for CheckTx) +curl http://localhost:26657/broadcast_tx_sync?tx=0x01234567 + +# Asynchronous broadcast (returns immediately) +curl http://localhost:26657/broadcast_tx_async?tx=0x01234567 + +# Commit broadcast (waits for block inclusion) +curl http://localhost:26657/broadcast_tx_commit?tx=0x01234567 +``` + +### Node Status and Health + +```bash +# Check node health +curl http://localhost:26657/health + +# Get comprehensive node status +curl http://localhost:26657/status + +# Get network information +curl http://localhost:26657/net_info +``` + +## Next Steps + + + + Browse the complete API reference in the sidebar - all ~30 RPC methods with interactive examples and detailed parameters + + + Learn more about subscribing to real-time events + + diff --git a/cometbft/v0.39/api-reference/rpc/openapi.yaml b/cometbft/v0.39/api-reference/rpc/openapi.yaml new file mode 100644 index 000000000..c6fc6c8df --- /dev/null +++ b/cometbft/v0.39/api-reference/rpc/openapi.yaml @@ -0,0 +1,2909 @@ +openapi: 3.0.0 +info: + title: CometBFT RPC + contact: + name: CometBFT + url: https://docs.cosmos.network/cometbft + description: | + CometBFT supports the following RPC protocols: + + * URI over HTTP + * JSONRPC over HTTP + * JSONRPC over websockets + + ## Configuration + + RPC can be configured by tuning parameters under `[rpc]` table in the + `$CMTHOME/config/config.toml` file or by using the `--rpc.X` command-line + flags. + + The default RPC listen address is `tcp://127.0.0.1:26657`. + To set another address, set the `laddr` config parameter to desired value. + CORS (Cross-Origin Resource Sharing) can be enabled by setting + `cors_allowed_origins`, `cors_allowed_methods`, `cors_allowed_headers` + config parameters. + + If testing using a local RPC node, under the `[rpc]` + section change the `cors_allowed_origins` property, please add the URL of + the site where this OpenAPI document is running, for example: + + `cors_allowed_origins = ["http://localhost:8088"]` + + or if testing from the official documentation site: + + `cors_allowed_origins = ["https://docs.cosmos.network"]` + + ## Arguments + + Arguments which expect strings or byte arrays may be passed as quoted + strings, like `"abc"` or as `0x`-prefixed strings, like `0x616263`. + + ## URI/HTTP + + A REST like interface. + + curl localhost:26657/block?height=5 + + ## JSONRPC/HTTP + + JSONRPC requests can be POST'd to the root RPC endpoint via HTTP. + + curl --header "Content-Type: application/json" --request POST --data '{"method": "block", "params": ["5"], "id": 1}' localhost:26657 + + ## JSONRPC/websockets + + JSONRPC requests can be also made via websocket. + The websocket endpoint is at `/websocket`, e.g. `localhost:26657/websocket`. + Asynchronous RPC functions like event `subscribe` and `unsubscribe` are + only available via websockets. + + For example using the [websocat](https://github.com/vi/websocat) tool, you can subscribe for 'NewBlock` events + with the following command: + + echo '{ "jsonrpc": "2.0","method": "subscribe","id": 0,"params": {"query": "tm.event='"'NewBlock'"'"} }' | websocat -n -t ws://127.0.0.1:26657/websocket + version: v0.38.x + license: + name: Apache 2.0 + url: https://github.com/cometbft/cometbft/blob/v0.38.x/LICENSE +servers: + - url: https://rpc.cosmos.directory/cosmoshub + description: Interact with the CometBFT RPC from a public node in the Cosmos registry + - url: http://localhost:26657 + description: Interact with CometBFT RPC node running locally +tags: + - name: Info + description: Informations about the node APIs + - name: Tx + description: Transactions broadcast APIs + - name: ABCI + description: ABCI APIs + - name: Evidence + description: Evidence APIs + - name: Unsafe + description: Unsafe APIs +paths: + /broadcast_tx_sync: + get: + summary: broadcast_tx_sync + tags: + - Tx + operationId: broadcast_tx_sync + description: |- + Returns with the response from CheckTx. Does not wait for DeliverTx result. + + If you want to be sure that the transaction is included in a block, you can + subscribe for the result using JSONRPC via a websocket. See + https://docs.cosmos.network/cometbft/v0.38/docs/core/Subscribing-to-events-via-Websocket + If you haven't received anything after a couple of blocks, resend it. If the + same happens again, send it to some other node. A few reasons why it could + happen: + + 1. malicious node can drop or pretend it had committed your tx + 2. malicious proposer (not necessary the one you're communicating with) can + drop transactions, which might become valid in the future + (https://github.com/tendermint/tendermint/issues/3322) + + + Please refer to [formatting/encoding rules](https://docs.cosmos.network/cometbft/v0.38/docs/core/Using-CometBFT#formatting) + for additional details + parameters: + - in: query + name: tx + required: true + schema: + type: string + example: '456' + description: The transaction + responses: + '200': + description: Empty + content: + application/json: + schema: + $ref: '#/components/schemas/BroadcastTxResponse' + '500': + description: Error + content: + application/json: + schema: + $ref: '#/components/schemas/ErrorResponse' + /broadcast_tx_async: + get: + summary: broadcast_tx_async + tags: + - Tx + operationId: broadcast_tx_async + description: |- + Returns right away, with no response. Does not wait for CheckTx nor DeliverTx results. + + If you want to be sure that the transaction is included in a block, you can + subscribe for the result using JSONRPC via a websocket. See + https://docs.cosmos.network/cometbft/v0.38/docs/core/Subscribing-to-events-via-Websocket + If you haven't received anything after a couple of blocks, resend it. If the + same happens again, send it to some other node. A few reasons why it could + happen: + + 1. malicious node can drop or pretend it had committed your tx + 2. malicious proposer (not necessary the one you're communicating with) can + drop transactions, which might become valid in the future + (https://github.com/tendermint/tendermint/issues/3322) + 3. node can be offline + + Please refer to [formatting/encoding rules](https://docs.cosmos.network/cometbft/v0.38/docs/core/Using-CometBFT#formatting) + for additional details + parameters: + - in: query + name: tx + required: true + schema: + type: string + example: '123' + description: The transaction + responses: + '200': + description: empty answer + content: + application/json: + schema: + $ref: '#/components/schemas/BroadcastTxResponse' + '500': + description: empty error + content: + application/json: + schema: + $ref: '#/components/schemas/ErrorResponse' + /broadcast_tx_commit: + get: + summary: broadcast_tx_commit + tags: + - Tx + operationId: broadcast_tx_commit + description: |- + Returns with the responses from CheckTx and DeliverTx. + + IMPORTANT: use only for testing and development. In production, use + BroadcastTxSync or BroadcastTxAsync. You can subscribe for the transaction + result using JSONRPC via a websocket. See + https://docs.cosmos.network/cometbft/v0.38/docs/core/Subscribing-to-events-via-Websocket + + CONTRACT: only returns error if mempool.CheckTx() errs or if we timeout + waiting for tx to commit. + + If CheckTx or DeliverTx fail, no error will be returned, but the returned result + will contain a non-OK ABCI code. + + Please refer to [formatting/encoding rules](https://docs.cosmos.network/cometbft/v0.38/docs/core/Using-CometBFT#formatting) + for additional details + parameters: + - in: query + name: tx + required: true + schema: + type: string + example: '785' + description: The transaction + responses: + '200': + description: empty answer + content: + application/json: + schema: + $ref: '#/components/schemas/BroadcastTxCommitResponse' + '500': + description: empty error + content: + application/json: + schema: + $ref: '#/components/schemas/ErrorResponse' + /check_tx: + get: + summary: check_tx + tags: + - Tx + operationId: check_tx + description: |- + Checks the transaction without executing it. + + The transaction won't be added to the mempool. + + Please refer to [formatting/encoding rules](https://docs.cosmos.network/cometbft/v0.38/docs/core/Using-CometBFT#formatting) + for additional details + + Upon success, the `Cache-Control` header will be set with the default + maximum age. + parameters: + - in: query + name: tx + required: true + schema: + type: string + example: '785' + description: The transaction + responses: + '200': + description: ABCI application's CheckTx response + content: + application/json: + schema: + $ref: '#/components/schemas/CheckTxResponse' + '500': + description: empty error + content: + application/json: + schema: + $ref: '#/components/schemas/ErrorResponse' + /health: + get: + summary: health + tags: + - Info + operationId: health + description: |- + Node heartbeat + + Get node health. Returns empty result (200 OK) on success, no response - in case of an error. + responses: + '200': + description: Gets Node Health + content: + application/json: + schema: + $ref: '#/components/schemas/EmptyResponse' + '500': + description: empty error + content: + application/json: + schema: + $ref: '#/components/schemas/ErrorResponse' + /status: + get: + summary: status + operationId: status + tags: + - Info + description: |- + Node Status + + Get CometBFT status including node info, pubkey, latest block hash, app hash, block height and time. + responses: + '200': + description: Status of the node + content: + application/json: + schema: + $ref: '#/components/schemas/StatusResponse' + '500': + description: empty error + content: + application/json: + schema: + $ref: '#/components/schemas/ErrorResponse' + /net_info: + get: + summary: net_info + operationId: net_info + tags: + - Info + description: |- + Network information + + Get network info. + responses: + '200': + description: empty answer + content: + application/json: + schema: + $ref: '#/components/schemas/NetInfoResponse' + '500': + description: empty error + content: + application/json: + schema: + $ref: '#/components/schemas/ErrorResponse' + /dial_seeds: + get: + summary: dial_seeds + operationId: dial_seeds + tags: + - Unsafe + description: |- + Dial Seeds (Unsafe) + + Dial a peer, this route in under unsafe, and has to manually enabled to use + + **Example:** curl 'localhost:26657/dial_seeds?seeds=\["f9baeaa15fedf5e1ef7448dd60f46c01f1a9e9c4@1.2.3.4:26656","0491d373a8e0fcf1023aaf18c51d6a1d0d4f31bd@5.6.7.8:26656"\]' + parameters: + - in: query + name: peers + description: list of seed nodes to dial + schema: + type: array + items: + type: string + example: f9baeaa15fedf5e1ef7448dd60f46c01f1a9e9c4@1.2.3.4:26656 + responses: + '200': + description: Dialing seeds in progress. See /net_info for details + content: + application/json: + schema: + $ref: '#/components/schemas/dialResp' + '500': + description: empty error + content: + application/json: + schema: + $ref: '#/components/schemas/ErrorResponse' + /dial_peers: + get: + summary: dial_peers + operationId: dial_peers + tags: + - Unsafe + description: |- + Add Peers/Persistent Peers (unsafe) + + Set a persistent peer, this route in under unsafe, and has to manually enabled to use. + + **Example:** curl 'localhost:26657/dial_peers?peers=\["f9baeaa15fedf5e1ef7448dd60f46c01f1a9e9c4@1.2.3.4:26656","0491d373a8e0fcf1023aaf18c51d6a1d0d4f31bd@5.6.7.8:26656"\]&persistent=false' + parameters: + - in: query + name: persistent + description: Have the peers you are dialing be persistent + schema: + type: boolean + example: true + - in: query + name: unconditional + description: Have the peers you are dialing be unconditional + schema: + type: boolean + example: true + - in: query + name: private + description: Have the peers you are dialing be private + schema: + type: boolean + example: true + - in: query + name: peers + description: array of peers to dial + schema: + type: array + items: + type: string + example: f9baeaa15fedf5e1ef7448dd60f46c01f1a9e9c4@1.2.3.4:26656 + responses: + '200': + description: Dialing seeds in progress. See /net_info for details + content: + application/json: + schema: + $ref: '#/components/schemas/dialResp' + '500': + description: empty error + content: + application/json: + schema: + $ref: '#/components/schemas/ErrorResponse' + /blockchain: + get: + summary: blockchain + operationId: blockchain + parameters: + - in: query + name: minHeight + description: Minimum block height to return + schema: + type: integer + example: 1 + - in: query + name: maxHeight + description: Maximum block height to return + schema: + type: integer + example: 2 + tags: + - Info + description: |- + Get block headers (max: 20) for minHeight <= height <= maxHeight. + + Get block headers for minHeight <= height <= maxHeight. + + At most 20 items will be returned. + + Upon success, the `Cache-Control` header will be set with the default + maximum age. + responses: + '200': + description: Block headers, returned in descending order (highest first). + content: + application/json: + schema: + $ref: '#/components/schemas/BlockchainResponse' + '500': + description: Error + content: + application/json: + schema: + $ref: '#/components/schemas/ErrorResponse' + /header: + get: + summary: header + operationId: header + parameters: + - in: query + name: height + schema: + type: integer + default: 0 + example: 1 + description: height to return. If no height is provided, it will fetch the latest header. + tags: + - Info + description: |- + Get header at a specified height + + Get Header. + + If the `height` field is set to a non-default value, upon success, the + `Cache-Control` header will be set with the default maximum age. + responses: + '200': + description: Header informations. + content: + application/json: + schema: + $ref: '#/components/schemas/BlockHeader' + '500': + description: Error + content: + application/json: + schema: + $ref: '#/components/schemas/ErrorResponse' + /header_by_hash: + get: + summary: header_by_hash + operationId: header_by_hash + parameters: + - in: query + name: hash + description: header hash + required: true + schema: + type: string + example: '0xD70952032620CC4E2737EB8AC379806359D8E0B17B0488F627997A0B043ABDED' + tags: + - Info + description: |- + Get header by hash + + Get Header By Hash. + + Upon success, the `Cache-Control` header will be set with the default + maximum age. + responses: + '200': + description: Header informations. + content: + application/json: + schema: + $ref: '#/components/schemas/BlockHeader' + '500': + description: Error + content: + application/json: + schema: + $ref: '#/components/schemas/ErrorResponse' + /block: + get: + summary: block + operationId: block + parameters: + - in: query + name: height + schema: + type: integer + default: 0 + example: 1 + description: height to return. If no height is provided, it will fetch the latest block. + tags: + - Info + description: |- + Get block at a specified height + + Get Block. + + If the `height` field is set to a non-default value, upon success, the + `Cache-Control` header will be set with the default maximum age. + responses: + '200': + description: Block informations. + content: + application/json: + schema: + $ref: '#/components/schemas/BlockResponse' + '500': + description: Error + content: + application/json: + schema: + $ref: '#/components/schemas/ErrorResponse' + /block_by_hash: + get: + summary: block_by_hash + operationId: block_by_hash + parameters: + - in: query + name: hash + description: block hash + required: true + schema: + type: string + example: '0xD70952032620CC4E2737EB8AC379806359D8E0B17B0488F627997A0B043ABDED' + tags: + - Info + description: |- + Get block by hash + + Get Block By Hash. + + Upon success, the `Cache-Control` header will be set with the default + maximum age. + responses: + '200': + description: Block informations. + content: + application/json: + schema: + $ref: '#/components/schemas/BlockResponse' + '500': + description: Error + content: + application/json: + schema: + $ref: '#/components/schemas/ErrorResponse' + /block_results: + get: + summary: block_results + operationId: block_results + parameters: + - in: query + name: height + description: height to return. If no height is provided, it will fetch information regarding the latest block. + schema: + type: integer + default: 0 + example: 1 + tags: + - Info + description: |- + Get block results at a specified height + + Get block_results. + + If the `height` field is set to a non-default value, upon success, the + `Cache-Control` header will be set with the default maximum age. + responses: + '200': + description: Block results. + content: + application/json: + schema: + $ref: '#/components/schemas/BlockResultsResponse' + '500': + description: Error + content: + application/json: + schema: + $ref: '#/components/schemas/ErrorResponse' + /commit: + get: + summary: commit + operationId: commit + parameters: + - in: query + name: height + description: height to return. If no height is provided, it will fetch commit informations regarding the latest block. + schema: + type: integer + default: 0 + example: 1 + tags: + - Info + description: |- + Get commit results at a specified height + + Get Commit. + + If the `height` field is set to a non-default value, upon success, the + `Cache-Control` header will be set with the default maximum age. + responses: + '200': + description: | + Commit results. + + canonical switches from false to true for block H once block H+1 has been committed. Until then it's subjective and only reflects what this node has seen so far. + content: + application/json: + schema: + $ref: '#/components/schemas/CommitResponse' + '500': + description: Error + content: + application/json: + schema: + $ref: '#/components/schemas/ErrorResponse' + /validators: + get: + summary: validators + operationId: validators + parameters: + - in: query + name: height + description: height to return. If no height is provided, it will fetch validator set which corresponds to the latest block. + schema: + type: integer + default: 0 + example: 1 + - in: query + name: page + description: Page number (1-based) + required: false + schema: + type: integer + default: 1 + example: 1 + - in: query + name: per_page + description: 'Number of entries per page (max: 100)' + required: false + schema: + type: integer + example: 30 + default: 30 + tags: + - Info + description: |- + Get validator set at a specified height + + Get Validators. Validators are sorted first by voting power + (descending), then by address (ascending). + + If the `height` field is set to a non-default value, upon success, the + `Cache-Control` header will be set with the default maximum age. + responses: + '200': + description: Commit results. + content: + application/json: + schema: + $ref: '#/components/schemas/ValidatorsResponse' + '500': + description: Error + content: + application/json: + schema: + $ref: '#/components/schemas/ErrorResponse' + /genesis: + get: + summary: genesis + operationId: genesis + tags: + - Info + description: |- + Get Genesis + + Get genesis. + + Upon success, the `Cache-Control` header will be set with the default + maximum age. + responses: + '200': + description: Genesis results. + content: + application/json: + schema: + $ref: '#/components/schemas/GenesisResponse' + '500': + description: Error + content: + application/json: + schema: + $ref: '#/components/schemas/ErrorResponse' + /genesis_chunked: + get: + summary: genesis_chunked + operationId: genesis_chunked + tags: + - Info + description: |- + Get Genesis in multiple chunks + + Get genesis document in multiple chunks to make it easier to iterate + through larger genesis structures. Each chunk is produced by converting + the genesis document to JSON and then splitting the resulting payload + into 16MB blocks, and then Base64-encoding each block. + + Upon success, the `Cache-Control` header will be set with the default + maximum age. + parameters: + - in: query + name: chunk + description: Sequence number of the chunk to download. + schema: + type: integer + default: 0 + example: 1 + responses: + '200': + description: Genesis chunk response. + content: + application/json: + schema: + $ref: '#/components/schemas/GenesisChunkedResponse' + '500': + description: Error + content: + application/json: + schema: + $ref: '#/components/schemas/ErrorResponse' + /dump_consensus_state: + get: + summary: dump_consensus_state + operationId: dump_consensus_state + tags: + - Info + description: |- + Get consensus state + + Get consensus state. + + Not safe to call from inside the ABCI application during a block execution. + responses: + '200': + description: | + Complete consensus state. + + See https://pkg.go.dev/github.com/cometbft/cometbft/types?tab=doc#Vote.String for Vote string description. + content: + application/json: + schema: + $ref: '#/components/schemas/DumpConsensusResponse' + '500': + description: Error + content: + application/json: + schema: + $ref: '#/components/schemas/ErrorResponse' + /consensus_state: + get: + summary: consensus_state + operationId: consensus_state + tags: + - Info + description: |- + Get consensus state + + Get consensus state. + + Not safe to call from inside the ABCI application during a block execution. + responses: + '200': + description: consensus state results. + content: + application/json: + schema: + $ref: '#/components/schemas/ConsensusStateResponse' + '500': + description: Error + content: + application/json: + schema: + $ref: '#/components/schemas/ErrorResponse' + /consensus_params: + get: + summary: consensus_params + operationId: consensus_params + parameters: + - in: query + name: height + description: height to return. If no height is provided, it will fetch commit informations regarding the latest block. + schema: + type: integer + default: 0 + example: 1 + tags: + - Info + description: |- + Get consensus parameters + + Get consensus parameters. + + If the `height` field is set to a non-default value, upon success, the + `Cache-Control` header will be set with the default maximum age. + responses: + '200': + description: consensus parameters results. + content: + application/json: + schema: + $ref: '#/components/schemas/ConsensusParamsResponse' + '500': + description: Error + content: + application/json: + schema: + $ref: '#/components/schemas/ErrorResponse' + /unconfirmed_txs: + get: + summary: unconfirmed_txs + operationId: unconfirmed_txs + parameters: + - in: query + name: limit + description: Maximum number of unconfirmed transactions to return (max 100) + required: false + schema: + type: integer + default: 30 + example: 1 + tags: + - Info + description: |- + Get the list of unconfirmed transactions + + Get list of unconfirmed transactions + responses: + '200': + description: List of unconfirmed transactions + content: + application/json: + schema: + $ref: '#/components/schemas/UnconfirmedTransactionsResponse' + '500': + description: Error + content: + application/json: + schema: + $ref: '#/components/schemas/ErrorResponse' + /num_unconfirmed_txs: + get: + summary: num_unconfirmed_txs + operationId: num_unconfirmed_txs + tags: + - Info + description: |- + Get data about unconfirmed transactions + + Get data about unconfirmed transactions + responses: + '200': + description: status about unconfirmed transactions + content: + application/json: + schema: + $ref: '#/components/schemas/NumUnconfirmedTransactionsResponse' + '500': + description: Error + content: + application/json: + schema: + $ref: '#/components/schemas/ErrorResponse' + /tx_search: + get: + summary: tx_search + description: |- + Search for transactions + + Search for transactions w/ their results. + + See /subscribe for the query syntax. + operationId: tx_search + parameters: + - in: query + name: query + description: Query + required: true + schema: + type: string + example: '"tx.height=1000"' + - in: query + name: prove + description: Include proofs of the transactions inclusion in the block + required: false + schema: + type: boolean + default: false + example: true + - in: query + name: page + description: Page number (1-based) + required: false + schema: + type: integer + default: 1 + example: 1 + - in: query + name: per_page + description: 'Number of entries per page (max: 100)' + required: false + schema: + type: integer + default: 30 + example: 30 + - in: query + name: order_by + description: Order in which transactions are sorted ("asc" or "desc"), by height & index. If empty, default sorting will be still applied. + required: false + schema: + type: string + default: asc + example: asc + tags: + - Info + responses: + '200': + description: List of unconfirmed transactions + content: + application/json: + schema: + $ref: '#/components/schemas/TxSearchResponse' + '500': + description: Error + content: + application/json: + schema: + $ref: '#/components/schemas/ErrorResponse' + /block_search: + get: + summary: block_search + description: |- + Search for blocks by FinalizeBlock events + + Search for blocks by FinalizeBlock events. + + See /subscribe for the query syntax. + operationId: block_search + parameters: + - in: query + name: query + description: Query + required: true + schema: + type: string + example: '"block.height > 1000"' + - in: query + name: page + description: Page number (1-based) + required: false + schema: + type: integer + default: 1 + example: 1 + - in: query + name: per_page + description: 'Number of entries per page (max: 100)' + required: false + schema: + type: integer + default: 30 + example: 30 + - in: query + name: order_by + description: Order in which blocks are sorted ("asc" or "desc"), by height. If empty, default sorting will be still applied. + required: false + schema: + type: string + default: desc + example: asc + tags: + - Info + responses: + '200': + description: List of paginated blocks matching the search criteria. + content: + application/json: + schema: + $ref: '#/components/schemas/BlockSearchResponse' + '500': + description: Error + content: + application/json: + schema: + $ref: '#/components/schemas/ErrorResponse' + /tx: + get: + summary: tx + operationId: tx + parameters: + - in: query + name: hash + description: hash of transaction to retrieve + required: true + schema: + type: string + example: '0xD70952032620CC4E2737EB8AC379806359D8E0B17B0488F627997A0B043ABDED' + - in: query + name: prove + description: Include proofs of the transaction's inclusion in the block + required: false + schema: + type: boolean + example: true + default: false + tags: + - Info + description: |- + Get transactions by hash + + Get a transaction + + Upon success, the `Cache-Control` header will be set with the default + maximum age. + responses: + '200': + description: Get a transaction` + content: + application/json: + schema: + $ref: '#/components/schemas/TxResponse' + '500': + description: Error + content: + application/json: + schema: + $ref: '#/components/schemas/ErrorResponse' + /abci_info: + get: + summary: abci_info + operationId: abci_info + tags: + - ABCI + description: |- + Get info about the application. + + Get info about the application. + + Upon success, the `Cache-Control` header will be set with the default + maximum age. + responses: + '200': + description: Get some info about the application. + content: + application/json: + schema: + $ref: '#/components/schemas/ABCIInfoResponse' + '500': + description: Error + content: + application/json: + schema: + $ref: '#/components/schemas/ErrorResponse' + /abci_query: + get: + summary: abci_query + operationId: abci_query + parameters: + - in: query + name: path + description: Path to the data ("/a/b/c") + required: true + schema: + type: string + example: '"/a/b/c"' + - in: query + name: data + description: Data + required: true + schema: + type: string + example: IHAVENOIDEA + - in: query + name: height + description: Height (0 means latest) + required: false + schema: + type: integer + example: 1 + default: 0 + - in: query + name: prove + description: Include proofs of the transactions inclusion in the block + required: false + schema: + type: boolean + example: true + default: false + tags: + - ABCI + description: |- + Query the application for some information. + + Query the application for some information. + responses: + '200': + description: Response of the submitted query + content: + application/json: + schema: + $ref: '#/components/schemas/ABCIQueryResponse' + '500': + description: Error + content: + application/json: + schema: + $ref: '#/components/schemas/ErrorResponse' + /broadcast_evidence: + get: + summary: broadcast_evidence + operationId: broadcast_evidence + parameters: + - in: query + name: evidence + description: JSON evidence + required: true + schema: + type: string + example: JSON_EVIDENCE_encoded + tags: + - Info + description: |- + Broadcast evidence of the misbehavior. + + Broadcast evidence of the misbehavior. + responses: + '200': + description: Broadcast evidence of the misbehavior. + content: + application/json: + schema: + $ref: '#/components/schemas/BroadcastEvidenceResponse' + '500': + description: Error + content: + application/json: + schema: + $ref: '#/components/schemas/ErrorResponse' +components: + schemas: + JSONRPC: + type: object + properties: + id: + type: integer + example: 0 + jsonrpc: + type: string + example: '2.0' + EmptyResponse: + description: Empty Response + allOf: + - $ref: '#/components/schemas/JSONRPC' + - type: object + properties: + result: + type: object + additionalProperties: {} + ErrorResponse: + description: Error Response + allOf: + - $ref: '#/components/schemas/JSONRPC' + - type: object + properties: + error: + type: string + example: Description of failure + ProtocolVersion: + type: object + properties: + p2p: + type: string + example: '7' + block: + type: string + example: '10' + app: + type: string + example: '0' + PubKey: + type: object + properties: + type: + type: string + example: tendermint/PubKeyEd25519 + value: + type: string + example: A6DoBUypNtUAyEHWtQ9bFjfNg8Bo9CrnkUGl6k6OHN4= + NodeInfo: + type: object + properties: + protocol_version: + $ref: '#/components/schemas/ProtocolVersion' + id: + type: string + example: 5576458aef205977e18fd50b274e9b5d9014525a + listen_addr: + type: string + example: tcp:0.0.0.0:26656 + network: + type: string + example: cosmoshub-2 + version: + type: string + example: 0.32.1 + channels: + type: string + example: '4020212223303800' + moniker: + type: string + example: moniker-node + other: + type: object + properties: + tx_index: + type: string + example: 'on' + rpc_address: + type: string + example: tcp:0.0.0.0:26657 + SyncInfo: + type: object + properties: + latest_block_hash: + type: string + example: 790BA84C3545FCCC49A5C629CEE6EA58A6E875C3862175BDC11EE7AF54703501 + latest_app_hash: + type: string + example: C9AEBB441B787D9F1D846DE51F3826F4FD386108B59B08239653ABF59455C3F8 + latest_block_height: + type: string + example: '1262196' + latest_block_time: + type: string + example: '2019-08-01T11:52:22.818762194Z' + earliest_block_hash: + type: string + example: 790BA84C3545FCCC49A5C629CEE6EA58A6E875C3862175BDC11EE7AF54703501 + earliest_app_hash: + type: string + example: C9AEBB441B787D9F1D846DE51F3826F4FD386108B59B08239653ABF59455C3F8 + earliest_block_height: + type: string + example: '1262196' + earliest_block_time: + type: string + example: '2019-08-01T11:52:22.818762194Z' + catching_up: + type: boolean + example: false + ValidatorInfo: + type: object + properties: + address: + type: string + example: 5D6A51A8E9899C44079C6AF90618BA0369070E6E + pub_key: + $ref: '#/components/schemas/PubKey' + voting_power: + type: string + example: '0' + Status: + description: Status Response + type: object + properties: + node_info: + $ref: '#/components/schemas/NodeInfo' + sync_info: + $ref: '#/components/schemas/SyncInfo' + validator_info: + $ref: '#/components/schemas/ValidatorInfo' + StatusResponse: + description: Status Response + allOf: + - $ref: '#/components/schemas/JSONRPC' + - type: object + properties: + result: + $ref: '#/components/schemas/Status' + Monitor: + type: object + properties: + Active: + type: boolean + example: true + Start: + type: string + example: '2019-07-31T14:31:28.66Z' + Duration: + type: string + example: '168901060000000' + Idle: + type: string + example: '168901040000000' + Bytes: + type: string + example: '5' + Samples: + type: string + example: '1' + InstRate: + type: string + example: '0' + CurRate: + type: string + example: '0' + AvgRate: + type: string + example: '0' + PeakRate: + type: string + example: '0' + BytesRem: + type: string + example: '0' + TimeRem: + type: string + example: '0' + Progress: + type: integer + example: 0 + Channel: + type: object + properties: + ID: + type: integer + example: 48 + SendQueueCapacity: + type: string + example: '1' + SendQueueSize: + type: string + example: '0' + Priority: + type: string + example: '5' + RecentlySent: + type: string + example: '0' + ConnectionStatus: + type: object + properties: + Duration: + type: string + example: '168901057956119' + SendMonitor: + $ref: '#/components/schemas/Monitor' + RecvMonitor: + $ref: '#/components/schemas/Monitor' + Channels: + type: array + items: + $ref: '#/components/schemas/Channel' + Peer: + type: object + properties: + node_info: + $ref: '#/components/schemas/NodeInfo' + is_outbound: + type: boolean + example: true + connection_status: + $ref: '#/components/schemas/ConnectionStatus' + remote_ip: + type: string + example: 95.179.155.35 + NetInfo: + type: object + properties: + listening: + type: boolean + example: true + listeners: + type: array + items: + type: string + example: Listener(@) + n_peers: + type: string + example: '1' + peers: + type: array + items: + $ref: '#/components/schemas/Peer' + NetInfoResponse: + description: NetInfo Response + allOf: + - $ref: '#/components/schemas/JSONRPC' + - type: object + properties: + result: + $ref: '#/components/schemas/NetInfo' + BlockMeta: + type: object + properties: + block_id: + $ref: '#/components/schemas/BlockID' + block_size: + type: integer + example: 1000000 + header: + $ref: '#/components/schemas/BlockHeader' + num_txs: + type: string + example: '54' + Blockchain: + type: object + required: + - last_height + - block_metas + properties: + last_height: + type: string + example: '1276718' + block_metas: + type: array + items: + $ref: '#/components/schemas/BlockMeta' + BlockchainResponse: + description: Blockchain info + allOf: + - $ref: '#/components/schemas/JSONRPC' + - type: object + properties: + result: + $ref: '#/components/schemas/Blockchain' + Commit: + required: + - type + - height + - round + - block_id + - timestamp + - validator_address + - validator_index + - signature + properties: + type: + type: integer + example: 2 + height: + type: string + example: '1262085' + round: + type: integer + example: 0 + block_id: + $ref: '#/components/schemas/BlockID' + timestamp: + type: string + example: '2019-08-01T11:39:38.867269833Z' + validator_address: + type: string + example: 000001E443FD237E4B616E2FA69DF4EE3D49A94F + validator_index: + type: integer + example: 0 + signature: + type: string + example: DBchvucTzAUEJnGYpNvMdqLhBAHG4Px8BsOBB3J3mAFCLGeuG7uJqy+nVngKzZdPhPi8RhmE/xcw/M9DOJjEDg== + Block: + type: object + properties: + header: + $ref: '#/components/schemas/BlockHeader' + data: + type: array + items: + type: string + example: yQHwYl3uCkKoo2GaChRnd+THLQ2RM87nEZrE19910Z28ABIUWW/t8AtIMwcyU0sT32RcMDI9GF0aEAoFdWF0b20SBzEwMDAwMDASEwoNCgV1YXRvbRIEMzEwMRCd8gEaagom61rphyEDoJPxlcjRoNDtZ9xMdvs+lRzFaHe2dl2P5R2yVCWrsHISQKkqX5H1zXAIJuC57yw0Yb03Fwy75VRip0ZBtLiYsUqkOsPUoQZAhDNP+6LY+RUwz/nVzedkF0S29NZ32QXdGv0= + evidence: + type: array + items: + $ref: '#/components/schemas/Evidence' + last_commit: + type: object + properties: + height: + type: integer + round: + type: integer + block_id: + $ref: '#/components/schemas/BlockID' + signatures: + type: array + items: + $ref: '#/components/schemas/Commit' + Evidence: + type: object + properties: + type: + type: string + height: + type: integer + time: + type: integer + total_voting_power: + type: integer + validator: + $ref: '#/components/schemas/Validator' + BlockComplete: + type: object + properties: + block_id: + $ref: '#/components/schemas/BlockID' + block: + $ref: '#/components/schemas/Block' + BlockResponse: + description: Blockc info + allOf: + - $ref: '#/components/schemas/JSONRPC' + - type: object + properties: + result: + $ref: '#/components/schemas/BlockComplete' + BlockResultsResponse: + type: object + required: + - jsonrpc + - id + - result + properties: + jsonrpc: + type: string + example: '2.0' + id: + type: integer + example: 0 + result: + type: object + required: + - height + properties: + height: + type: string + example: '12' + txs_results: + type: array + nullable: true + items: + type: object + properties: + code: + type: string + example: '0' + data: + type: string + example: '' + log: + type: string + example: not enough gas + info: + type: string + example: '' + gas_wanted: + type: string + example: '100' + gas_used: + type: string + example: '100' + events: + type: array + nullable: true + items: + type: object + properties: + type: + type: string + example: app + attributes: + type: array + nullable: false + items: + $ref: '#/components/schemas/Event' + codespace: + type: string + example: ibc + finalize_block_events: + type: array + nullable: true + items: + type: object + properties: + type: + type: string + example: app + attributes: + type: array + nullable: false + items: + $ref: '#/components/schemas/Event' + validator_updates: + type: array + nullable: true + items: + type: object + properties: + pub_key: + type: object + required: + - type + - value + properties: + type: + type: string + example: tendermint/PubKeyEd25519 + value: + type: string + example: 9tK9IT+FPdf2qm+5c2qaxi10sWP+3erWTKgftn2PaQM= + power: + type: string + example: '300' + consensus_param_updates: + $ref: '#/components/schemas/ConsensusParams' + CommitResponse: + type: object + required: + - jsonrpc + - id + - result + properties: + jsonrpc: + type: string + example: '2.0' + id: + type: integer + example: 0 + result: + required: + - signed_header + - canonical + properties: + signed_header: + required: + - header + - commit + properties: + header: + $ref: '#/components/schemas/BlockHeader' + commit: + required: + - height + - round + - block_id + - signatures + properties: + height: + type: string + example: '1311801' + round: + type: integer + example: 0 + block_id: + $ref: '#/components/schemas/BlockID' + signatures: + type: array + items: + type: object + properties: + block_id_flag: + type: integer + example: 2 + validator_address: + type: string + example: 000001E443FD237E4B616E2FA69DF4EE3D49A94F + timestamp: + type: string + example: '2019-04-22T17:01:58.376629719Z' + signature: + type: string + example: 14jaTQXYRt8kbLKEhdHq7AXycrFImiLuZx50uOjs2+Zv+2i7RTG/jnObD07Jo2ubZ8xd7bNBJMqkgtkd0oQHAw== + type: object + type: object + canonical: + type: boolean + example: true + type: object + ValidatorsResponse: + type: object + required: + - jsonrpc + - id + - result + properties: + jsonrpc: + type: string + example: '2.0' + id: + type: integer + example: 0 + result: + required: + - block_height + - validators + properties: + block_height: + type: string + example: '55' + validators: + type: array + items: + $ref: '#/components/schemas/ValidatorPriority' + count: + type: string + example: '1' + total: + type: string + example: '25' + type: object + GenesisResponse: + type: object + required: + - jsonrpc + - id + - result + properties: + jsonrpc: + type: string + example: '2.0' + id: + type: integer + example: 0 + result: + type: object + required: + - genesis + properties: + genesis: + type: object + required: + - genesis_time + - chain_id + - initial_height + - consensus_params + - validators + - app_hash + properties: + genesis_time: + type: string + example: '2019-04-22T17:00:00Z' + chain_id: + type: string + example: cosmoshub-2 + initial_height: + type: string + example: '2' + consensus_params: + $ref: '#/components/schemas/ConsensusParams' + validators: + type: array + items: + type: object + properties: + address: + type: string + example: B00A6323737F321EB0B8D59C6FD497A14B60938A + pub_key: + required: + - type + - value + properties: + type: + type: string + example: tendermint/PubKeyEd25519 + value: + type: string + example: cOQZvh/h9ZioSeUMZB/1Vy1Xo5x2sjrVjlE/qHnYifM= + type: object + power: + type: string + example: '9328525' + name: + type: string + example: Certus One + app_hash: + type: string + example: '' + app_state: + properties: {} + type: object + GenesisChunkedResponse: + type: object + required: + - jsonrpc + - id + - result + properties: + jsonrpc: + type: string + example: '2.0' + id: + type: integer + example: 0 + result: + required: + - chunk + - total + - data + properties: + chunk: + type: integer + example: 0 + total: + type: integer + example: 1 + data: + type: string + example: Z2VuZXNpcwo= + DumpConsensusResponse: + type: object + required: + - jsonrpc + - id + - result + properties: + jsonrpc: + type: string + example: '2.0' + id: + type: integer + example: 0 + result: + required: + - round_state + - peers + properties: + round_state: + required: + - height + - round + - step + - start_time + - commit_time + - validators + - proposal + - proposal_block + - proposal_block_parts + - locked_round + - locked_block + - locked_block_parts + - valid_round + - valid_block + - valid_block_parts + - votes + - commit_round + - last_commit + - last_validators + - triggered_timeout_precommit + properties: + height: + type: string + example: '1311801' + round: + type: integer + example: 0 + step: + type: integer + example: 3 + start_time: + type: string + example: '2019-08-05T11:28:49.064658805Z' + commit_time: + type: string + example: '2019-08-05T11:28:44.064658805Z' + validators: + required: + - validators + - proposer + properties: + validators: + type: array + items: + $ref: '#/components/schemas/ValidatorPriority' + proposer: + $ref: '#/components/schemas/ValidatorPriority' + type: object + locked_round: + type: integer + example: -1 + valid_round: + type: string + example: '-1' + votes: + type: array + items: + type: object + properties: + round: + type: string + example: '0' + prevotes: + type: array + nullable: true + items: + type: string + example: + - nil-Vote + - Vote{19:46A3F8B8393B 1311801/00/1(Prevote) 000000000000 64CE682305CB @ 2019-08-05T11:28:47.374703444Z} + prevotes_bit_array: + type: string + example: BA{100:___________________x________________________________________________________________________________} 209706/170220253 = 0.00 + precommits: + type: array + nullable: true + items: + type: string + example: + - nil-Vote + precommits_bit_array: + type: string + example: BA{100:____________________________________________________________________________________________________} 0/170220253 = 0.00 + commit_round: + type: integer + example: -1 + last_commit: + nullable: true + required: + - votes + - votes_bit_array + - peer_maj_23s + properties: + votes: + type: array + items: + type: string + example: + - Vote{0:000001E443FD 1311800/00/2(Precommit) 3071ADB27D1A 77EE1B6B6847 @ 2019-08-05T11:28:43.810128139Z} + votes_bit_array: + type: string + example: BA{100:xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx} 170220253/170220253 = 1.00 + peer_maj_23s: + properties: {} + type: object + type: object + last_validators: + required: + - validators + - proposer + properties: + validators: + type: array + items: + $ref: '#/components/schemas/ValidatorPriority' + proposer: + $ref: '#/components/schemas/ValidatorPriority' + type: object + triggered_timeout_precommit: + type: boolean + example: false + type: object + peers: + type: array + items: + type: object + properties: + node_address: + type: string + example: 357f6a6c1d27414579a8185060aa8adf9815c43c@68.183.41.207:26656 + peer_state: + required: + - round_state + - stats + properties: + round_state: + required: + - height + - round + - step + - start_time + - proposal + - proposal_block_parts_header + - proposal_block_parts + - proposal_pol_round + - proposal_pol + - prevotes + - precommits + - last_commit_round + - last_commit + - catchup_commit_round + - catchup_commit + properties: + height: + type: string + example: '1311801' + round: + type: string + example: '0' + step: + type: integer + example: 3 + start_time: + type: string + example: '2019-08-05T11:28:49.21730864Z' + proposal: + type: boolean + example: false + proposal_block_parts_header: + required: + - total + - hash + properties: + total: + type: integer + example: 0 + hash: + type: string + example: '' + type: object + proposal_pol_round: + nullable: true + type: integer + example: -1 + proposal_pol: + nullable: true + type: string + example: ____________________________________________________________________________________________________ + prevotes: + nullable: true + type: string + example: ___________________x________________________________________________________________________________ + precommits: + nullable: true + type: string + example: ____________________________________________________________________________________________________ + last_commit_round: + nullable: true + type: integer + example: 0 + last_commit: + nullable: true + type: string + example: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx + catchup_commit_round: + type: integer + nullable: true + example: -1 + catchup_commit: + nullable: true + type: string + example: ____________________________________________________________________________________________________ + type: object + stats: + required: + - votes + - block_parts + properties: + votes: + type: string + example: '1159558' + block_parts: + type: string + example: '4786' + type: object + type: object + type: object + ConsensusStateResponse: + type: object + required: + - jsonrpc + - id + - result + properties: + jsonrpc: + type: string + example: '2.0' + id: + type: integer + example: 0 + result: + required: + - round_state + properties: + round_state: + required: + - height/round/step + - start_time + - proposal_block_hash + - locked_block_hash + - valid_block_hash + - height_vote_set + - proposer + properties: + height/round/step: + type: string + example: 1262197/0/8 + start_time: + type: string + example: '2019-08-01T11:52:38.962730289Z' + proposal_block_hash: + type: string + example: 634ADAF1F402663BEC2ABC340ECE8B4B45AA906FA603272ACC5F5EED3097E009 + locked_block_hash: + type: string + example: 634ADAF1F402663BEC2ABC340ECE8B4B45AA906FA603272ACC5F5EED3097E009 + valid_block_hash: + type: string + example: 634ADAF1F402663BEC2ABC340ECE8B4B45AA906FA603272ACC5F5EED3097E009 + height_vote_set: + type: array + items: + type: object + properties: + round: + type: integer + example: 0 + prevotes: + type: array + items: + type: string + example: + - Vote{0:000001E443FD 1262197/00/1(Prevote) 634ADAF1F402 7BB974E1BA40 @ 2019-08-01T11:52:35.513572509Z} + - nil-Vote + prevotes_bit_array: + type: string + example: BA{100:xxxxxxxxxxxxxxxxx_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx} 169753436/170151262 = 1.00 + precommits: + type: array + items: + type: string + example: + - Vote{5:18C78D135C9D 1262197/00/2(Precommit) 634ADAF1F402 8B5EFFFEABCD @ 2019-08-01T11:52:36.25600005Z} + - nil-Vote + precommits_bit_array: + type: string + example: BA{100:xxxxxx_xxxxx_xxxx_x_xxx_xx_xx_xx__x_x_x__xxxxxxxxxxxxxx_xxxx_xx_xxxxxx_xxxxxxxx_xxxx_xxx_x_xxxx__xxx} 118726247/170151262 = 0.70 + proposer: + type: object + properties: + address: + type: string + example: D540AB022088612AC74B287D076DBFBC4A377A2E + index: + type: integer + example: 0 + type: object + type: object + ConsensusParamsResponse: + type: object + required: + - jsonrpc + - id + - result + properties: + jsonrpc: + type: string + example: '2.0' + id: + type: integer + example: 0 + result: + type: object + required: + - block_height + - consensus_params + properties: + block_height: + type: string + example: '1' + consensus_params: + $ref: '#/components/schemas/ConsensusParams' + NumUnconfirmedTransactionsResponse: + type: object + required: + - jsonrpc + - id + - result + properties: + jsonrpc: + type: string + example: '2.0' + id: + type: integer + example: 0 + result: + required: + - n_txs + - total + - total_bytes + properties: + n_txs: + type: string + example: '31' + total: + type: string + example: '82' + total_bytes: + type: string + example: '19974' + type: object + UnconfirmedTransactionsResponse: + type: object + required: + - jsonrpc + - id + - result + properties: + jsonrpc: + type: string + example: '2.0' + id: + type: integer + example: 0 + result: + required: + - n_txs + - total + - total_bytes + - txs + properties: + n_txs: + type: string + example: '82' + total: + type: string + example: '82' + total_bytes: + type: string + example: '19974' + txs: + type: array + nullable: true + items: + type: string + nullable: true + example: + - gAPwYl3uCjCMTXENChSMnIkb5ZpYHBKIZqecFEV2tuZr7xIUA75/FmYq9WymsOBJ0XSJ8yV8zmQKMIxNcQ0KFIyciRvlmlgcEohmp5wURXa25mvvEhQbrvwbvlNiT+Yjr86G+YQNx7kRVgowjE1xDQoUjJyJG+WaWBwSiGannBRFdrbma+8SFK2m+1oxgILuQLO55n8mWfnbIzyPCjCMTXENChSMnIkb5ZpYHBKIZqecFEV2tuZr7xIUQNGfkmhTNMis4j+dyMDIWXdIPiYKMIxNcQ0KFIyciRvlmlgcEohmp5wURXa25mvvEhS8sL0D0wwgGCItQwVowak5YB38KRIUCg4KBXVhdG9tEgUxMDA1NBDoxRgaagom61rphyECn8x7emhhKdRCB2io7aS/6Cpuq5NbVqbODmqOT3jWw6kSQKUresk+d+Gw0BhjiggTsu8+1voW+VlDCQ1GRYnMaFOHXhyFv7BCLhFWxLxHSAYT8a5XqoMayosZf9mANKdXArA= + type: object + TxSearchResponse: + type: object + required: + - jsonrpc + - id + - result + properties: + jsonrpc: + type: string + example: '2.0' + id: + type: integer + example: 0 + result: + required: + - txs + - total_count + properties: + txs: + type: array + items: + type: object + properties: + hash: + type: string + example: D70952032620CC4E2737EB8AC379806359D8E0B17B0488F627997A0B043ABDED + height: + type: string + example: '1000' + index: + type: integer + example: 0 + tx_result: + required: + - log + - gas_wanted + - gas_used + - tags + properties: + log: + type: string + example: '[{"msg_index":"0","success":true,"log":""}]' + gas_wanted: + type: string + example: '200000' + gas_used: + type: string + example: '28596' + tags: + $ref: '#/components/schemas/Event' + type: object + tx: + type: string + example: 5wHwYl3uCkaoo2GaChQmSIu8hxpJxLcCuIi8fiHN4TMwrRIU/Af1cEG7Rcs/6LjTl7YjRSymJfYaFAoFdWF0b20SCzE0OTk5OTk1MDAwEhMKDQoFdWF0b20SBDUwMDAQwJoMGmoKJuta6YchAwswBShaB1wkZBctLIhYqBC3JrAI28XGzxP+rVEticGEEkAc+khTkKL9CDE47aDvjEHvUNt+izJfT4KVF2v2JkC+bmlH9K08q3PqHeMI9Z5up+XMusnTqlP985KF+SI5J3ZOIhhNYWRlIGJ5IENpcmNsZSB3aXRoIGxvdmU= + proof: + required: + - RootHash + - Data + - Proof + properties: + RootHash: + type: string + example: 72FE6BF6D4109105357AECE0A82E99D0F6288854D16D8767C5E72C57F876A14D + Data: + type: string + example: 5wHwYl3uCkaoo2GaChQmSIu8hxpJxLcCuIi8fiHN4TMwrRIU/Af1cEG7Rcs/6LjTl7YjRSymJfYaFAoFdWF0b20SCzE0OTk5OTk1MDAwEhMKDQoFdWF0b20SBDUwMDAQwJoMGmoKJuta6YchAwswBShaB1wkZBctLIhYqBC3JrAI28XGzxP+rVEticGEEkAc+khTkKL9CDE47aDvjEHvUNt+izJfT4KVF2v2JkC+bmlH9K08q3PqHeMI9Z5up+XMusnTqlP985KF+SI5J3ZOIhhNYWRlIGJ5IENpcmNsZSB3aXRoIGxvdmU= + Proof: + required: + - total + - index + - leaf_hash + - aunts + properties: + total: + type: string + example: '2' + index: + type: string + example: '0' + leaf_hash: + type: string + example: eoJxKCzF3m72Xiwb/Q43vJ37/2Sx8sfNS9JKJohlsYI= + aunts: + type: array + items: + type: string + example: + - eWb+HG/eMmukrQj4vNGyFYb3nKQncAWacq4HF5eFzDY= + type: object + type: object + total_count: + type: string + example: '2' + type: object + TxResponse: + type: object + required: + - jsonrpc + - id + - result + properties: + jsonrpc: + type: string + example: '2.0' + id: + type: integer + example: 0 + result: + required: + - hash + - height + - index + - tx_result + - tx + properties: + hash: + type: string + example: D70952032620CC4E2737EB8AC379806359D8E0B17B0488F627997A0B043ABDED + height: + type: string + example: '1000' + index: + type: integer + example: 0 + tx_result: + required: + - log + - gas_wanted + - gas_used + - tags + properties: + log: + type: string + example: '[{"msg_index":"0","success":true,"log":""}]' + gas_wanted: + type: string + example: '200000' + gas_used: + type: string + example: '28596' + tags: + type: array + items: + $ref: '#/components/schemas/Event' + type: object + tx: + type: string + example: 5wHwYl3uCkaoo2GaChQmSIu8hxpJxLcCuIi8fiHN4TMwrRIU/Af1cEG7Rcs/6LjTl7YjRSymJfYaFAoFdWF0b20SCzE0OTk5OTk1MDAwEhMKDQoFdWF0b20SBDUwMDAQwJoMGmoKJuta6YchAwswBShaB1wkZBctLIhYqBC3JrAI28XGzxP+rVEticGEEkAc+khTkKL9CDE47aDvjEHvUNt+izJfT4KVF2v2JkC+bmlH9K08q3PqHeMI9Z5up+XMusnTqlP985KF+SI5J3ZOIhhNYWRlIGJ5IENpcmNsZSB3aXRoIGxvdmU= + type: object + ABCIInfoResponse: + type: object + required: + - jsonrpc + - id + properties: + jsonrpc: + type: string + example: '2.0' + id: + type: integer + example: 0 + result: + required: + - response + properties: + response: + required: + - data + - version + - app_version + - last_block_height + - last_block_app_hash + properties: + data: + type: string + example: '{"size":0}' + version: + type: string + example: 0.16.1 + app_version: + type: string + example: '1' + last_block_height: + type: string + example: '1314126' + last_block_app_hash: + type: string + example: C9AEBB441B787D9F1D846DE51F3826F4FD386108B59B08239653ABF59455C3F8 + type: object + type: object + ABCIQueryResponse: + type: object + required: + - error + - result + - id + - jsonrpc + properties: + error: + type: string + example: '' + result: + required: + - response + properties: + response: + required: + - log + - height + - proof + - value + - key + - index + - code + properties: + log: + type: string + example: exists + height: + type: string + example: '0' + proof: + type: string + example: 010114FED0DAD959F36091AD761C922ABA3CBF1D8349990101020103011406AA2262E2F448242DF2C2607C3CDC705313EE3B0001149D16177BC71E445476174622EA559715C293740C + value: + type: string + example: '61626364' + key: + type: string + example: '61626364' + index: + type: string + example: '-1' + code: + type: string + example: '0' + type: object + type: object + id: + type: integer + example: 0 + jsonrpc: + type: string + example: '2.0' + BroadcastEvidenceResponse: + type: object + required: + - id + - jsonrpc + properties: + error: + type: string + example: '' + result: + type: string + example: '' + id: + type: integer + example: 0 + jsonrpc: + type: string + example: '2.0' + BroadcastTxCommitResponse: + type: object + required: + - error + - result + - id + - jsonrpc + properties: + error: + type: string + example: '' + result: + required: + - height + - hash + - deliver_tx + - check_tx + properties: + height: + type: string + example: '26682' + hash: + type: string + example: 75CA0F856A4DA078FC4911580360E70CEFB2EBEE + deliver_tx: + required: + - log + - data + - code + properties: + log: + type: string + example: '' + data: + type: string + example: '' + code: + type: string + example: '0' + type: object + check_tx: + required: + - log + - data + - code + properties: + log: + type: string + example: '' + data: + type: string + example: '' + code: + type: string + example: '0' + type: object + type: object + id: + type: integer + example: 0 + jsonrpc: + type: string + example: '2.0' + CheckTxResponse: + type: object + required: + - error + - result + - id + - jsonrpc + properties: + error: + type: string + example: '' + result: + required: + - log + - data + - code + properties: + code: + type: string + example: '0' + data: + type: string + example: '' + log: + type: string + example: '' + info: + type: string + example: '' + gas_wanted: + type: string + example: '1' + gas_used: + type: string + example: '0' + events: + type: array + nullable: true + items: + type: object + properties: + type: + type: string + example: app + attributes: + type: array + nullable: false + items: + $ref: '#/components/schemas/Event' + codespace: + type: string + example: bank + type: object + id: + type: integer + example: 0 + jsonrpc: + type: string + example: '2.0' + BroadcastTxResponse: + type: object + required: + - jsonrpc + - id + - result + - error + properties: + jsonrpc: + type: string + example: '2.0' + id: + type: integer + example: 0 + result: + required: + - code + - data + - log + - hash + properties: + code: + type: string + example: '0' + data: + type: string + example: '' + log: + type: string + example: '' + codespace: + type: string + example: ibc + hash: + type: string + example: 0D33F2F03A5234F38706E43004489E061AC40A2E + type: object + error: + type: string + example: '' + dialResp: + type: object + properties: + Log: + type: string + example: Dialing seeds in progress. See /net_info for details + BlockSearchResponse: + type: object + required: + - jsonrpc + - id + - result + properties: + jsonrpc: + type: string + example: '2.0' + id: + type: integer + example: 0 + result: + required: + - blocks + - total_count + properties: + blocks: + type: array + items: + $ref: '#/components/schemas/BlockComplete' + total_count: + type: integer + example: 2 + type: object + ValidatorPriority: + type: object + properties: + address: + type: string + example: 000001E443FD237E4B616E2FA69DF4EE3D49A94F + pub_key: + required: + - type + - value + properties: + type: + type: string + example: tendermint/PubKeyEd25519 + value: + type: string + example: 9tK9IT+FPdf2qm+5c2qaxi10sWP+3erWTKgftn2PaQM= + type: object + voting_power: + type: string + example: '239727' + proposer_priority: + type: string + example: '-11896414' + Validator: + type: object + properties: + pub_key: + $ref: '#/components/schemas/PubKey' + voting_power: + type: integer + address: + type: string + ConsensusParams: + type: object + nullable: true + required: + - block + - evidence + - validator + properties: + block: + type: object + required: + - max_bytes + - max_gas + - time_iota_ms + properties: + max_bytes: + type: string + example: '22020096' + max_gas: + type: string + example: '1000' + time_iota_ms: + type: string + example: '1000' + evidence: + type: object + required: + - max_age + properties: + max_age: + type: string + example: '100000' + validator: + type: object + required: + - pub_key_types + properties: + pub_key_types: + type: array + items: + type: string + example: + - ed25519 + Event: + type: object + properties: + key: + type: string + example: action + value: + type: string + example: send + index: + type: boolean + example: false + BlockHeader: + required: + - version + - chain_id + - height + - time + - last_block_id + - last_commit_hash + - data_hash + - validators_hash + - next_validators_hash + - consensus_hash + - app_hash + - last_results_hash + - evidence_hash + - proposer_address + properties: + version: + required: + - block + - app + properties: + block: + type: string + example: '10' + app: + type: string + example: '0' + type: object + chain_id: + type: string + example: cosmoshub-2 + height: + type: string + example: '12' + time: + type: string + example: '2019-04-22T17:01:51.701356223Z' + last_block_id: + $ref: '#/components/schemas/BlockID' + last_commit_hash: + type: string + example: 21B9BC845AD2CB2C4193CDD17BFC506F1EBE5A7402E84AD96E64171287A34812 + data_hash: + type: string + example: 970886F99E77ED0D60DA8FCE0447C2676E59F2F77302B0C4AA10E1D02F18EF73 + validators_hash: + type: string + example: D658BFD100CA8025CFD3BECFE86194322731D387286FBD26E059115FD5F2BCA0 + next_validators_hash: + type: string + example: D658BFD100CA8025CFD3BECFE86194322731D387286FBD26E059115FD5F2BCA0 + consensus_hash: + type: string + example: 0F2908883A105C793B74495EB7D6DF2EEA479ED7FC9349206A65CB0F9987A0B8 + app_hash: + type: string + example: 223BF64D4A01074DC523A80E76B9BBC786C791FB0A1893AC5B14866356FCFD6C + last_results_hash: + type: string + example: '' + evidence_hash: + type: string + example: '' + proposer_address: + type: string + example: D540AB022088612AC74B287D076DBFBC4A377A2E + type: object + BlockID: + required: + - hash + - parts + properties: + hash: + type: string + example: 112BC173FD838FB68EB43476816CD7B4C6661B6884A9E357B417EE957E1CF8F7 + parts: + required: + - total + - hash + properties: + total: + type: integer + example: 1 + hash: + type: string + example: 38D4B26B5B725C4F13571EFE022C030390E4C33C8CF6F88EDD142EA769642DBD + type: object diff --git a/cometbft/v0.39/changelog/release-notes.mdx b/cometbft/v0.39/changelog/release-notes.mdx new file mode 100644 index 000000000..3c8b79e88 --- /dev/null +++ b/cometbft/v0.39/changelog/release-notes.mdx @@ -0,0 +1,213 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/changelog/release-notes' +title: "Changelog" +description: "Release history and changelog for Cosmos COMETBFT" +mode: "wide" +--- + + + This page tracks releases and changes for v0.39.3. For the full release history, see the [CHANGELOG](https://github.com/cometbft/cometbft/blob/main/CHANGELOG.md) on GitHub. + + + +## DEPENDENCIES + +- Bump pq version to 1.12.0 + ([\#5713](https://github.com/cometbft/cometbft/pull/5713)) + +## FEATURES + +- `[mempool]` Allow CheckTx retries for BroadcastTxSync + ([\#5802](https://github.com/cometbft/cometbft/pull/5802)) + + + +## DEPENDENCIES + +- `[build]` Bump `github.com/Masterminds/semver/v3` from `3.4.0` to `3.5.0` + ([\#5823](https://github.com/cometbft/cometbft/pull/5823)) + +## BUG FIXES + +- `[mempool]` App mempool waits before broadcasting. + ([\#5800](https://github.com/cometbft/cometbft/pull/5800)) +- `[blocksync]` Prevent maxPeerHeight poisoning + ([\#5803](https://github.com/cometbft/cometbft/pull/5803)) +- `[p2p]` Add lp2p reactor panic recovery + ([\#5816](https://github.com/cometbft/cometbft/pull/5816)) +- `[light]` Stop witness comparison after divergence checks + ([\#5820](https://github.com/cometbft/cometbft/pull/5820)) +- `[abci]` fix(abci): prevent panic on unlock in socket server panic recovery + ([\#5593](https://github.com/cometbft/cometbft/pull/5593)) + +## IMPROVEMENTS + +- `[abci,mempool]` Add Krakatoa app-mempool flow, including ABCI app-connection methods and app mempool/reactor wiring. + ([`f4a9ba936`](https://github.com/cometbft/cometbft/commit/f4a9ba936), [\#5791](https://github.com/cometbft/cometbft/pull/5791)) +- `[e2e]` Introduce app-mempool e2e network fixtures for simple, perturbed, and libp2p scenarios. + ([`f4a9ba936`](https://github.com/cometbft/cometbft/commit/f4a9ba936)) +- `[execution,state]` Add height validation in state execution and consensus paths + ([\#5804](https://github.com/cometbft/cometbft/pull/5804)) +- `[consensus]` perf(consensus): skip fsync for unsigned internal messages (block parts) + ([\#5695](https://github.com/cometbft/cometbft/pull/5695)) + + + +## BUG FIXES + +- `[autopool]` Fix autopool worker message handling recovery + ([\#5775](https://github.com/cometbft/cometbft/pull/5775)) +- `[types]` Fix nil vote handling + ([\#5777](https://github.com/cometbft/cometbft/pull/5777)) + +## IMPROVEMENTS + +- `[node]` do not log experimental warning for libp2p if starting node at height 0 + ([\#5776](https://github.com/cometbft/cometbft/pull/5776)) + + + +## BUG FIXES + +- `[evidence]` Add validation for Light Client Attack evidence ByzantineValidators + ([\#5638](https://github.com/cometbft/cometbft/pull/5638)) +- `[types]` Fix buffer offset bug in `ProposerPriorityHash` that caused hash collisions when validator priorities differed + ([\#5613](https://github.com/cometbft/cometbft/pull/5613)) +- `[p2p]` fix(privval): Ephemeral Port Exhaustion + ([\#5433](https://github.com/cometbft/cometbft/pull/5433)) +- `[blocksync]` fix(blocksync): `ExtendedCommit` verification via next blocks `LastCommit` + ([\#5629](https://github.com/cometbft/cometbft/pull/5629)) +- [p2p] fix(lp2p): enforce stream max size ([\#5647](https://github.com/cometbft/cometbft/pull/5647)) +- `[metrics]` fix(metrics)!: peer_send_queue_size + ([\#5648](https://github.com/cometbft/cometbft/pull/5648)) +- `[statesync]` fix adaptive_sync and streamline stateSync logic + ([\#5663](https://github.com/cometbft/cometbft/pull/5663)) +- `[blocksync]` Modify blocksync to use full commit verification instead of light + ([\#5663](https://github.com/cometbft/cometbft/pull/5663)) +- `[adaptivesync]` Simplify loop, reuse blockExec.ValidateBlock + ([\#5717](https://github.com/cometbft/cometbft/pull/5717)) + +## IMPROVEMENTS + +- `[consensus]` perf(consensus): skip fsync for unsigned internal messages (block parts) ([\#5695](https://github.com/cometbft/cometbft/pull/5695)) +- `[ci]`: add lp2p testnet ([\#5643](https://github.com/cometbft/cometbft/pull/5643)) +- `[mempool]` feat!(p2p): introduce follower-mode. Improve lib-p2p integraap access +- `[types]` Add validation for `AuthorityParams.Authority` field in consensus params, enforcing a maximum length of 256 characters ([#5511](https://github.com/cometbft/cometbft/pull/5511)) +- `[mempool]` perf(mempool/cache): Optimize LRUTxCache.Remove to reduce lock contention and map access + ([\#5244](https://github.com/cometbft/cometbft/pull/5244)) +- `[e2e]` add support for testing different keytypes, including BLS + ([\#3513](https://github.com/cometbft/cometbft/pull/3513)) +- `[crypto]` Reduce BLS signature size to 48 bytes by increasing pubkey size to + 192 bytes ([\#3624](https://github.com/cometbft/cometbft/issues/3624)) +- `[p2p]` feat(lp2p): make reactor queue configurable + ([\#5662](https://github.com/cometbft/cometbft/pull/5662)) +- `[cli]` print lib-p2p peer id + ([\#5667](https://github.com/cometbft/cometbft/pull/5667)) +- `[p2p]` Add warning when go-libp2p transport is enabled, conveying that the setting + should only be activated if it can be enabled simultaneously for all validators + and peer IDs have been predetermined and exchanged + ([\#5692](https://github.com/cometbft/cometbft/pull/5692)) +- `[p2p]` feat(p2p): add adaptive sync for comet-p2p + ([\#5705](https://github.com/cometbft/cometbft/pull/5705)) +- `[blocksync]` fix redo event loss, stale event cancellation, and optimize retry timer + ([\#5592](https://github.com/cometbft/cometbft/pull/5592)) + +## FEATURES + +- `[p2p]` feat(lp2p): implemented resource limiter ([\#5671](https://github.com/cometbft/cometbft/pull/5671)) +- `[p2p]` feat(consensus): add adaptive sync blocksync-to-consensus ingestion ([\#5633](https://github.com/cometbft/cometbft/pull/5633)) +- `[p2p]` feat(lp2p): implement Peer info methods (`NodeInfo`, `RemoteIP`, `RemoteAddr`, `IsOutbound`) + for `/net_info` RPC compatibility with libp2p transport ([\#5619](https://github.com/cometbft/cometbft/pull/5619)) +- `[p2p]` feat(lp2p): stop/reconnect peers that failed ([\#5618](https://github.com/cometbft/cometbft/pull/5618)) +- `[p2p]` Add experimental support for lib-p2p networking ([\#5463](https://github.com/cometbft/cometbft/pull/5463)) +- `[crypto]` Add support for BLS12-381 keys. Since the implementation needs + `cgo` and brings in new dependencies, we use the `bls12381` build flag to + enable it ([\#2765](https://github.com/cometbft/cometbft/pull/2765)) +- `[mempool]` Add a metric (a counter) to measure whether a tx was received more than once. + ([\#634](https://github.com/cometbft/cometbft/pull/634)) +- `[p2p]` Rename `IPeerSet#List` to `Copy`, add `Random`, `ForEach` methods. + Rename `PeerSet#List` to `Copy`, add `Random`, `ForEach` methods. + ([\#2246](https://github.com/cometbft/cometbft/pull/2246)) +- `[mempool]` When the node is performing block sync or state sync, the mempool + reactor now discards incoming transactions from peers, and does not propagate + transactions to peers. + ([\#785](https://github.com/cometbft/cometbft/issues/785)) +- Optimized the PSQL indexer + ([\#2142](https://github.com/cometbft/cometbft/pull/2142)) thanks to external contributor @k0marov ! +- `[p2p]` make `PeerSet.Remove` more efficient (Author: @odeke-em) + ([\#2246](https://github.com/cometbft/cometbft/pull/2246)) +- `[light]` Remove duplicated signature checks in `light.VerifyNonAdjacent` + ([\#2365](https://github.com/cometbft/cometbft/issues/2365)) +- `[state/indexer]` Lower the heap allocation of transaction searches + ([\#2839](https://github.com/cometbft/cometbft/pull/2839)) +- `[libs/json]` Lower the memory overhead of JSON encoding by using JSON encoders internally + ([\#2846](https://github.com/cometbft/cometbft/pull/2846)). +- `[log]` allow strip out all debug-level code from the binary at compile time using build flags + ([\#2847](https://github.com/cometbft/cometbft/issues/2847)) +- `[types]` Small reduction in memory allocation via swapping Key with Equals in VoteSet + ([\#1112](https://github.com/cometbft/cometbft/issues/1112)) +- `[event-bus]` Remove the debug logs in PublishEventTx, which were noticed production slowdowns. + ([\#2911](https://github.com/cometbft/cometbft/pull/2911)) +- `[state/execution]` Cache the block hash computation inside of the Block Type, so we only compute it once. + ([\#2924](https://github.com/cometbft/cometbft/pull/2924)) +- `[consensus/state]` Remove a redundant `VerifyBlock` call in `FinalizeCommit` + ([\#2928](https://github.com/cometbft/cometbft/pull/2928)) +- `[p2p/channel]` Speedup `ProtoIO` writer creation time, and thereby speedup channel writing by 5%. + ([\#2949](https://github.com/cometbft/cometbft/pull/2949)) +- `[p2p/conn]` Minor speedup (3%) to connection.WritePacketMsgTo, by removing MinInt calls. + ([\#2952](https://github.com/cometbft/cometbft/pull/2952)) +- `[blockstore]` Remove a redundant `Header.ValidateBasic` call in `LoadBlockMeta`, 75% reducing this time. + ([\#2964](https://github.com/cometbft/cometbft/pull/2964)) +- `[p2p]` Lower `flush_throttle_timeout` to 10ms + ([\#2988](https://github.com/cometbft/cometbft/issues/2988)) +- `[types]` Significantly speedup types.MakePartSet and types.AddPart, which are used in creating a block proposal + ([\#3117](https://github.com/cometbft/cometbft/issues/3117)) +- `[types] Make a new method`GetByAddressMut` for `ValSet`, which does not copy the returned validator. + ([\#3119](https://github.com/cometbft/cometbft/issues/3119)) +- `[consensus]` Make Vote messages only take one peerstate mutex + ([\#3156](https://github.com/cometbft/cometbft/issues/3156)) +- `[consensus]` Make the consensus reactor no longer have packets on receive take the consensus lock. Consensus will now update the reactor's view after every relevant change through the existing synchronous event bus subscription. + ([\#3211](https://github.com/cometbft/cometbft/pull/3211)) +- `[p2p/conn]` Speedup secret connection large writes, by buffering the write to the underlying connection. + ([\#3346](https://github.com/cometbft/cometbft/pull/3346)) +- `[consensus]` Make broadcasting `HasVote` and `HasProposalBlockPart` control messages use `TrySend` instead of `Send`. This saves notable amounts of performance, while at the same time those messages are for preventing redundancy, not critical, and may be dropped without risks for the protocol. + ([\#3151](https://github.com/cometbft/cometbft/issues/3151)) +- `[p2p/conn]` Removes several heap allocations per packet send, stemming from how we double-wrap packets prior to proto marshalling them in the connection layer. This change reduces the memory overhead and speeds up the code. + ([\#3423](https://github.com/cometbft/cometbft/issues/3423)) +- `[p2p/conn]` Speedup secret connection large packet reads, by buffering the read to the underlying connection. + ([\#3419](https://github.com/cometbft/cometbft/pull/3419)) +- `[mempool]` In the broadcast routine, get the pointer to the peer's state once, before starting to iterate through the list of transactions. + ([\#3430](https://github.com/cometbft/cometbft/pull/3430)) +- `[consensus]` Make mempool updates asynchronous from consensus Commit's, + reducing latency for reaching consensus timeouts. + ([#3008](https://github.com/cometbft/cometbft/pull/3008)) +- [consensus] Add peer height metric publication to the consensus reactor's peer state. + ([#5517](https://github.com/cometbft/cometbft/pull/5517)) + +## BUG-FIXES + +- `[evidence]` Use structured logging for consensus buffer flush error + ([\#5465](https://github.com/cometbft/cometbft/pull/5465)) +- `[mempool]` Fix mutex in `CListMempool.Flush` method, by changing it from read-lock to write-lock + ([\#2443](https://github.com/cometbft/cometbft/issues/2443)). +- `[crypto/bls12381]` Fix JSON marshal of private key + ([\#4772](https://github.com/cometbft/cometbft/pull/4772)) +- `[crypto/bls12381]` Modify `Sign`, `Verify` to use `dstMinPk` + ([\#4783](https://github.com/cometbft/cometbft/issues/4783)) +- `[cli]` Prevent inadvertent rollover of IPs in `cometbft testnet` config generator + ([\#5541](https://github.com/cometbft/cometbft/pull/5541)) +- `[abci]` fix(abci): prevent panic on unlock in socket server panic recovery + ([\#5593](https://github.com/cometbft/cometbft/pull/5593)) + +## API-BREAKING + +- `[p2p]` Rename `IPeerSet#List` to `Copy`, add `Random`, `ForEach` methods. + Rename `PeerSet#List` to `Copy`, add `Random`, `ForEach` methods. + ([\#2246](https://github.com/cometbft/cometbft/pull/2246)) +- `[crypto]` Remove Sr25519 curve + ([\#3646](https://github.com/cometbft/cometbft/pull/3646)) +- `[rpc]` The endpoints `broadcast_tx_*` now return an error when the node is + performing block sync or state sync. + ([\#785](https://github.com/cometbft/cometbft/issues/785)) + diff --git a/cometbft/v0.39/docs/README.mdx b/cometbft/v0.39/docs/README.mdx new file mode 100644 index 000000000..33cd79e6e --- /dev/null +++ b/cometbft/v0.39/docs/README.mdx @@ -0,0 +1,45 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/docs/README' +title: CometBFT Documentation +description: CometBFT is a blockchain application platform. +footer: + newsletter: false +--- +{/* trigger rebuild */} + +# CometBFT + +Welcome to the CometBFT documentation! + +CometBFT is a blockchain application platform; it provides the equivalent +of a web server, database, and supporting libraries for blockchain applications +written in any programming language. Like a web server serving web applications, +CometBFT serves blockchain applications. + +More formally, CometBFT performs Byzantine Fault Tolerant (BFT) +State Machine Replication (SMR) for arbitrary deterministic, finite state machines. +For more background, see [What is CometBFT?](/cometbft/v0.39/docs/introduction/intro#what-is-cometbft). + +To get started quickly with an example application, see the [quick start guide](/cometbft/v0.39/docs/guides/Quick-Start). + +To learn about application development on CometBFT, see the [Application Blockchain Interface](/cometbft/v0.39/spec/abci/Overview). + +For more details on using CometBFT, see the respective documentation for +[CometBFT internals](/cometbft/v0.39/docs/core/), [benchmarking and monitoring](/cometbft/v0.39/docs/tools/), and [network deployments](/cometbft/v0.39/docs/networks/). + +## Contribute + +To recommend a change to the documentation, please submit a PR. Each major +release's documentation is housed on the corresponding release branch, e.g., for +the v0.34 release series, the documentation is housed on the `v0.34.x` branch. + +When submitting changes that affect all releases, please start by submitting a +PR to the docs on `main`—this will be backported to the relevant release +branches. If a change is exclusively relevant to a specific release, please +target that release branch with your PR. + +Changes to the documentation will be reviewed by the team and, if accepted and +merged, published to (/cometbft) for the respective version(s). + +The build process for the documentation is housed in the [Cosmos docs repository](https://github.com/cosmos/docs/tree/main/cometbft). diff --git a/cometbft/v0.39/docs/app-dev/Application-Architecture-Guide.mdx b/cometbft/v0.39/docs/app-dev/Application-Architecture-Guide.mdx new file mode 100644 index 000000000..31d3572ed --- /dev/null +++ b/cometbft/v0.39/docs/app-dev/Application-Architecture-Guide.mdx @@ -0,0 +1,56 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/docs/app-dev/Application-Architecture-Guide' +title: Application Architecture Guide +order: 4 +--- + +Here we provide a brief guide on the recommended architecture of a +CometBFT blockchain application. + +We distinguish here between two forms of "application". The first is the +end-user application, like a desktop-based wallet app that a user downloads, +which is where the user actually interacts with the system. The other is the +ABCI application, which is the logic that actually runs on the blockchain. +Transactions sent by an end-user application are ultimately processed by the ABCI +application after being committed by CometBFT. + +The end-user application communicates with a REST API exposed by the application. +The application runs CometBFT nodes and verifies CometBFT light-client proofs +through the CometBFT RPC. The CometBFT process communicates with +a local ABCI application, where the user query or transaction is actually +processed. + +The ABCI application must be a deterministic result of the CometBFT +consensus - any external influence on the application state that didn't +come through CometBFT could cause a consensus failure. Thus _nothing_ +should communicate with the ABCI application except CometBFT via ABCI. + +If the ABCI application is written in Go, it can be compiled into the +CometBFT binary. Otherwise, it should use a Unix socket to communicate +with CometBFT. If it's necessary to use TCP, extra care must be taken +to encrypt and authenticate the connection. + +All reads from the ABCI application happen through the CometBFT `/abci_query` +endpoint. All writes to the ABCI application happen through the CometBFT +`/broadcast_tx_*` endpoints. + +The Light-Client Daemon is what provides light clients (end users) with +nearly all the security of a full node. It formats and broadcasts +transactions, and verifies proofs of queries and transaction results. +Note that it need not be a daemon - the Light-Client logic could instead +be implemented in the same process as the end-user application. + +Note for those ABCI applications with weaker security requirements, the +functionality of the Light-Client Daemon can be moved into the ABCI +application process itself. That said, exposing the ABCI application process +to anything besides CometBFT over ABCI requires extreme caution, as +all transactions, and possibly all queries, should still pass through +CometBFT. + +See the following for more extensive documentation: + +- [Interchain Standard for the Light-Client REST API](https://github.com/cosmos/cosmos-sdk/pull/1617) (legacy/deprecated) +- [CometBFT RPC Docs](/cometbft/v0.39/api-reference/rpc/index) +- [CometBFT in Production](/cometbft/v0.39/docs/core/Running-in-production) +- [ABCI spec](/cometbft/v0.39/spec/abci/Overview) diff --git a/cometbft/v0.39/docs/app-dev/Getting-Started.mdx b/cometbft/v0.39/docs/app-dev/Getting-Started.mdx new file mode 100644 index 000000000..1d6c6bf02 --- /dev/null +++ b/cometbft/v0.39/docs/app-dev/Getting-Started.mdx @@ -0,0 +1,200 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/docs/app-dev/Getting-Started' +title: Getting Started +order: 2 +--- + +## First CometBFT App + +As a general-purpose blockchain engine, CometBFT is agnostic to the +application you want to run. So, to run a complete blockchain that does +something useful, you must start two programs: one is CometBFT, +the other is your application, which can be written in any programming +language. + +CometBFT handles all the p2p and consensus logic, and just forwards transactions to the +application when they need to be validated, or when they're ready to be +executed and committed. + +In this guide, we show you some examples of how to run an application +using CometBFT. + +### Install + +The first apps we will work with are written in Go. To install them, you +need to [install Go](https://golang.org/doc/install), put +`$GOPATH/bin` in your `$PATH`, and enable Go modules. If you use `bash`, +follow these instructions: + +```bash +echo export GOPATH=\"\$HOME/go\" >> ~/.bash_profile +echo export PATH=\"\$PATH:\$GOPATH/bin\" >> ~/.bash_profile +``` + +Then run + +```bash +go get github.com/cometbft/cometbft +cd $GOPATH/src/github.com/cometbft/cometbft +make install_abci +``` + +Now you should have the `abci-cli` installed; run `abci-cli` to see the list of commands: + +``` +Usage: + abci-cli [command] + +Available Commands: + batch run a batch of abci commands against an application + check_tx validate a transaction + commit commit the application state and return the Merkle root hash + completion Generate the autocompletion script for the specified shell + console start an interactive ABCI console for multiple commands + echo have the application echo a message + finalize_block deliver a block of transactions to the application + help Help about any command + info get some info about the application + kvstore ABCI demo example + prepare_proposal prepare proposal + process_proposal process proposal + query query the application state + test run integration tests + version print ABCI console version + +Flags: + --abci string either socket or grpc (default "socket") + --address string address of application socket (default "tcp://0.0.0.0:26658") + -h, --help help for abci-cli + --log_level string set the logger level (default "debug") + -v, --verbose print the command and results as if it were a console session + +Use "abci-cli [command] --help" for more information about a command. +``` + +You'll notice the `kvstore` command, an example application written in Go. + +Now, let's run an app! + +## KVStore - A First Example + +The kvstore app is a [Merkle +tree](https://en.wikipedia.org/wiki/Merkle_tree) that just stores all +transactions. If the transaction contains an `=`, e.g., `key=value`, then +the `value` is stored under the `key` in the Merkle tree. Otherwise, the +full transaction bytes are stored as the key and the value. + +Let's start a kvstore application. + +```sh +abci-cli kvstore +``` + +In another terminal, we can start CometBFT. You should already have the +CometBFT binary installed. If not, follow the steps from +[here](/cometbft/v0.39/docs/guides/Install-CometBFT). If you have never run CometBFT +before, use: + +```sh +cometbft init +cometbft node +``` + +If you have used CometBFT, you may want to reset the data for a new +blockchain by running `cometbft unsafe-reset-all`. Then you can run +`cometbft node` to start CometBFT and connect to the app. For more +details, see [the guide on using CometBFT](/cometbft/v0.39/docs/core/Using-CometBFT). + +You should see CometBFT making blocks! We can get the status of our +CometBFT node as follows: + +```sh +curl -s localhost:26657/status +``` + +The `-s` just silences `curl`. For nicer output, pipe the result into a +tool like [jq](https://stedolan.github.io/jq/) or `json_pp`. + +Now let's send some transactions to the kvstore. + +```sh +curl -s 'localhost:26657/broadcast_tx_commit?tx="abcd"' +``` + +Note the single quote (`'`) around the URL, which ensures that the +double quotes (`"`) are not escaped by bash. This command sent a +transaction with bytes `abcd`, so `abcd` will be stored as both the key +and the value in the Merkle tree. The response should look something +like: + +```json +{ + "jsonrpc": "2.0", + "id": "", + "result": { + "check_tx": {}, + "deliver_tx": { + "tags": [ + { + "key": "YXBwLmNyZWF0b3I=", + "value": "amFl" + }, + { + "key": "YXBwLmtleQ==", + "value": "YWJjZA==" + } + ] + }, + "hash": "9DF66553F98DE3C26E3C3317A3E4CED54F714E39", + "height": 14 + } +} +``` + +We can confirm that our transaction worked and the value got stored by +querying the app: + +```sh +curl -s 'localhost:26657/abci_query?data="abcd"' +``` + +The result should look like: + +```json +{ + "jsonrpc": "2.0", + "id": "", + "result": { + "response": { + "log": "exists", + "index": "-1", + "key": "YWJjZA==", + "value": "YWJjZA==" + } + } +} +``` + +Note the `value` in the result (`YWJjZA==`); this is the base64 encoding +of the ASCII of `abcd`. You can verify this in a Python 2 shell by +running `"YWJjZA==".decode('base64')` or in a Python 3 shell by running +`import codecs; codecs.decode(b"YWJjZA==", 'base64').decode('ascii')`. +Stay tuned for a future release that [makes this output more +human-readable](https://github.com/tendermint/tendermint/issues/1794). + +Now let's try setting a different key and value: + +```sh +curl -s 'localhost:26657/broadcast_tx_commit?tx="name=satoshi"' +``` + +Now if we query for `name`, we should get `satoshi`, or `c2F0b3NoaQ==` +in base64: + +```sh +curl -s 'localhost:26657/abci_query?data="name"' +``` + +Try some other transactions and queries to make sure everything is +working! diff --git a/cometbft/v0.39/docs/app-dev/Indexing-Transactions.mdx b/cometbft/v0.39/docs/app-dev/Indexing-Transactions.mdx new file mode 100644 index 000000000..7dde249f8 --- /dev/null +++ b/cometbft/v0.39/docs/app-dev/Indexing-Transactions.mdx @@ -0,0 +1,306 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/docs/app-dev/Indexing-Transactions' +title: Indexing Transactions +order: 5 +--- + +CometBFT allows you to index transactions and blocks and later query or +subscribe to their results. Transactions are indexed by `ResponseFinalizeBlock.tx_results.events` and +blocks are indexed by `ResponseFinalizeBlock.events`. However, transactions +are also indexed by a primary key which includes the transaction hash and maps +to and stores the corresponding transaction results. Blocks are indexed by a primary key +which includes the block height and maps to and stores the block height, i.e., +the block itself is never stored. + +Each event contains a type and a list of attributes, which are key-value pairs +denoting something about what happened during the method's execution. For more +details on `Events`, see the [ABCI][/cometbft/v0.39/spec/abci/Outline#events] documentation. + +An `Event` has a composite key associated with it. A `compositeKey` is +constructed by its type and key separated by a dot. + +For example: + +```json +"jack": [ + "account.number": 100 +] +``` + +would be equal to the composite key of `jack.account.number`. + +By default, CometBFT will index all transactions by their respective hashes +and height and blocks by their height. + +CometBFT allows for different events within the same height to have +equal attributes. + +## Configuration + +Operators can configure indexing via the `[tx_index]` section. The `indexer` +field takes a series of supported indexers. If `null` is included, indexing will +be turned off regardless of other values provided. + +```toml +[tx-index] + +# The backend database to back the indexer. +# If indexer is "null", no indexer service will be used. +# +# The application will set which txs to index. In some cases a node operator will be able +# to decide which txs to index based on configuration set in the application. +# +# Options: +# 1) "null" +# 2) "kv" (default) - the simplest possible indexer, backed by key-value storage (defaults to levelDB; see DBBackend). +# - When "kv" is chosen "tx.height" and "tx.hash" will always be indexed. +# 3) "psql" - the indexer services backed by PostgreSQL. +# indexer = "kv" +``` + +### Supported Indexers + +#### KV + +The `kv` indexer type is an embedded key-value store supported by the main +underlying CometBFT database. Using the `kv` indexer type allows you to query +for block and transaction events directly against CometBFT's RPC. However, the +query syntax is limited, and so this indexer type might be deprecated or removed +entirely in the future. + +**Implementation and data layout** + +The kv indexer stores each attribute of an event individually by creating a composite key +with: + +- event type, +- attribute key, +- attribute value, +- event generator (e.g., `FinalizeBlock`), +- the height, and +- event counter. + +For example, the following events: + +``` +Type: "transfer", + Attributes: []abci.EventAttribute{ + {Key: "sender", Value: "Bob", Index: true}, + {Key: "recipient", Value: "Alice", Index: true}, + {Key: "balance", Value: "100", Index: true}, + {Key: "note", Value: "nothing", Index: true}, + }, + +``` + +``` +Type: "transfer", + Attributes: []abci.EventAttribute{ + {Key: "sender", Value: "Tom", Index: true}, + {Key: "recipient", Value: "Alice", Index: true}, + {Key: "balance", Value: "200", Index: true}, + {Key: "note", Value: "nothing", Index: true}, + }, +``` + +will be represented as follows in the store, assuming these events result from the `FinalizeBlock` call for height 1: + +``` +Key value +---- event1 ------ +transferSenderBobFinalizeBlock11 1 +transferRecipientAliceFinalizeBlock11 1 +transferBalance100FinalizeBlock11 1 +transferNoteNothingFinalizeBlock11 1 +---- event2 ------ +transferSenderTomFinalizeBlock12 1 +transferRecipientAliceFinalizeBlock12 1 +transferBalance200FinalizeBlock12 1 +transferNoteNothingFinalizeBlock12 1 + +``` + +The event number is a local variable kept by the indexer and incremented when a new event is processed. +It is an `int64` variable and has no other semantics besides being used to associate attributes belonging to the same events within a height. +This variable is not atomically incremented as event indexing is deterministic. **Should this ever change**, the event ID generation +will be broken. + +#### PostgreSQL + +The `psql` indexer type allows an operator to enable block and transaction event +indexing by proxying it to an external PostgreSQL instance, allowing for the events +to be stored in relational models. Since the events are stored in an RDBMS, operators +can leverage SQL to perform a series of rich and complex queries that are not +supported by the `kv` indexer type. Since operators can leverage SQL directly, +searching is not enabled for the `psql` indexer type via CometBFT's RPC—any +such query will fail. + +Note that the SQL schema is stored in `state/indexer/sink/psql/schema.sql`, and operators +must explicitly create the relations prior to starting CometBFT and enabling +the `psql` indexer type. + +Example: + +```shell +psql ... -f state/indexer/sink/psql/schema.sql +``` + +## Default Indexes + +The CometBFT transaction and block event indexer indexes a few select reserved events +by default. + +### Transactions + +The following indexes are indexed by default: + +- `tx.height` +- `tx.hash` + +### Blocks + +The following indexes are indexed by default: + +- `block.height` + +## Adding Events + +Applications are free to define which events to index. CometBFT does not +expose functionality to define which events to index and which to ignore. In +your application's `FinalizeBlock` method, add the `Events` field with pairs of +UTF-8 encoded strings (e.g., "transfer.sender": "Bob", "transfer.recipient": +"Alice", "transfer.balance": "100"). + +Example: + +```go +func (app *Application) FinalizeBlock(_ context.Context, req *types.RequestFinalizeBlock) (*types.ResponseFinalizeBlock, error) { + + //... + tx_results[0] := &types.ExecTxResult{ + Code: CodeTypeOK, + // With every transaction we can emit a series of events. To make it simple, we just emit the same events. + Events: []types.Event{ + { + Type: "app", + Attributes: []types.EventAttribute{ + {Key: "creator", Value: "Cosmoshi Netowoko", Index: true}, + {Key: "key", Value: key, Index: true}, + {Key: "index_key", Value: "index is working", Index: true}, + {Key: "noindex_key", Value: "index is working", Index: false}, + }, + }, + { + Type: "app", + Attributes: []types.EventAttribute{ + {Key: "creator", Value: "Cosmoshi", Index: true}, + {Key: "key", Value: value, Index: true}, + {Key: "index_key", Value: "index is working", Index: true}, + {Key: "noindex_key", Value: "index is working", Index: false}, + }, + }, + }, + } + + block_events = []types.Event{ + { + Type: "loan", + Attributes: []types.EventAttribute{ + { Key: "account_no", Value: "1", Index: true}, + { Key: "amount", Value: "200", Index: true }, + }, + }, + { + Type: "loan", + Attributes: []types.EventAttribute{ + { Key: "account_no", Value: "2", Index: true }, + { Key: "amount", Value: "300", Index: true}, + }, + }, + } + return &types.ResponseFinalizeBlock{TxResults: tx_results, Events: block_events} +} +``` + +If the indexer is not `null`, the transaction will be indexed. Each event is +indexed using a composite key in the form of `{eventType}.{eventAttribute}={eventValue}`, +e.g., `transfer.sender=bob`. + +## Querying Transaction Events + +You can query for a paginated set of transactions by their events by calling the +`/tx_search` RPC endpoint: + +```bash +curl "localhost:26657/tx_search?query=\"message.sender='cosmos1...'\"&prove=true" +``` + +Check out the [RPC API reference](/cometbft/v0.39/api-reference/rpc/index) for the `tx_search` endpoint +for more information on query syntax and other options. + +## Subscribing to Transactions + +Clients can subscribe to transactions with the given tags via WebSocket by providing +a query to the `/subscribe` RPC endpoint. + +```json +{ + "jsonrpc": "2.0", + "method": "subscribe", + "id": "0", + "params": { + "query": "message.sender='cosmos1...'" + } +} +``` + +Check out the [RPC API documentation](/cometbft/v0.39/api-reference/rpc/index) for more information +on query syntax and other options. + +## Querying Block Events + +You can query for a paginated set of blocks by their events by calling the +`/block_search` RPC endpoint: + +```bash +curl "localhost:26657/block_search?query=\"block.height > 10\"" +``` + +Storing the event sequence was introduced in CometBFT 0.34.26. Before that, up +until Tendermint Core 0.34.26, the event sequence was not stored in the kvstore, +and events were stored only by height. That means that queries returned blocks +and transactions whose event attributes matched within the height but could match +across different events at that height. + +This behavior was fixed with CometBFT 0.34.26+. However, if the data was +indexed with earlier versions of Tendermint Core and not re-indexed, that data +will be queried as if all the attributes within a height occurred within the +same event. + +## Event Attribute Value Types + +Users can use anything as an event value. However, if the event attribute value +is a number, the following needs to be taken into account: + +- Negative numbers will not be properly retrieved when querying the indexer. +- Event values are converted to big floats (from the `big/math` package). The + precision of the floating-point number is set to the bit length of the + integer it is supposed to represent, so that there is no loss of information + due to insufficient precision. This was not present before CometBFT v0.38.x, + and all float values were ignored. +- As of CometBFT v0.38.x, queries can contain floating-point numbers as well. +- Note that comparing to floats can be imprecise with a high number of decimals. + +## Event Type and Attribute Key Format + +An event type/attribute key is a string that can contain any Unicode letter or +digit, as well as the following characters: `.` (dot), `-` (dash), `_` +(underscore). The event type/attribute key must not start with `-` (dash) or +`.` (dot). + +``` +^[\w]+[\.-\w]?$ +``` + +[abci-events]: /cometbft/v0.39/spec/abci/abci++_basic_concepts#events diff --git a/cometbft/v0.39/docs/app-dev/Using-ABCI-CLI.mdx b/cometbft/v0.39/docs/app-dev/Using-ABCI-CLI.mdx new file mode 100644 index 000000000..15e961cc9 --- /dev/null +++ b/cometbft/v0.39/docs/app-dev/Using-ABCI-CLI.mdx @@ -0,0 +1,213 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/docs/app-dev/Using-ABCI-CLI' +title: Using ABCI-CLI +order: 3 +--- + +To facilitate testing and debugging of ABCI servers and simple apps, we +built a CLI, the `abci-cli`, for sending ABCI messages from the command +line. + +## Install + +Make sure you [have Go installed](https://golang.org/doc/install). + +Next, install the `abci-cli` tool and example applications: + +```sh +git clone https://github.com/cometbft/cometbft.git +cd cometbft +make install_abci +``` + +Now run `abci-cli` to see the list of commands: + +```sh +Usage: + abci-cli [command] + +Available Commands: + batch run a batch of abci commands against an application + check_tx validate a transaction + commit commit the application state and return the Merkle root hash + completion Generate the autocompletion script for the specified shell + console start an interactive ABCI console for multiple commands + echo have the application echo a message + finalize_block deliver a block of transactions to the application + help Help about any command + info get some info about the application + kvstore ABCI demo example + prepare_proposal prepare proposal + process_proposal process proposal + query query the application state + test run integration tests + version print ABCI console version + +Flags: + --abci string either socket or grpc (default "socket") + --address string address of application socket (default "tcp://0.0.0.0:26658") + -h, --help help for abci-cli + --log_level string set the logger level (default "debug") + -v, --verbose print the command and results as if it were a console session + +Use "abci-cli [command] --help" for more information about a command. +``` + +## KVStore - First Example + +The `abci-cli` tool lets us send ABCI messages to our application to +help build and debug them. + +The most important messages are `deliver_tx`, `check_tx`, and `commit`, +but there are others for convenience, configuration, and information +purposes. + +We'll start a kvstore application, which was installed at the same time as +`abci-cli` above. The kvstore just stores transactions in a Merkle tree. Its +code can be found +[here](https://github.com/cometbft/cometbft/blob/v0.38.x/abci/example/kvstore/kvstore.go). + +Start the application by running: + +```sh +abci-cli kvstore +``` + +And in another terminal, run + +```sh +abci-cli echo hello +abci-cli info +``` + +You'll see something like: + +```sh +-> data: hello +-> data.hex: 68656C6C6F +``` + +and: + +```sh +-> data: {"size":0} +-> data.hex: 7B2273697A65223A307D +``` + +An ABCI application must provide two things: + +- a socket server +- a handler for ABCI messages + +When we run the `abci-cli` tool, we open a new connection to the +application's socket server, send the given ABCI message, and wait for a +response. + +The server may be generic for a particular language, and we provide a +[reference implementation in +Golang](https://github.com/cometbft/cometbft/tree/v0.38.x/abci/server). See the +[list of other ABCI implementations](https://github.com/tendermint/awesome#ecosystem) for servers in +other languages. + +The handler is specific to the application, and may be arbitrary, so +long as it is deterministic and conforms to the ABCI interface +specification. + +So when we run `abci-cli info`, we open a new connection to the ABCI +server, which calls the `Info()` method on the application, which tells +us the number of transactions in our Merkle tree. + +Now, since every command opens a new connection, we provide the +`abci-cli console` and `abci-cli batch` commands to allow multiple ABCI +messages to be sent over a single connection. + +Running `abci-cli console` should drop you into an interactive console for +speaking ABCI messages to your application. + +Try running these commands: + +```sh +> echo hello +-> code: OK +-> data: hello +-> data.hex: 0x68656C6C6F + +> info +-> code: OK +-> data: {"size":0} +-> data.hex: 0x7B2273697A65223A307D + +> prepare_proposal "abc=123" +-> code: OK +-> log: Succeeded. Tx: abc=123 + +> process_proposal "abc==456" +-> code: OK +-> status: REJECT + +> process_proposal "abc=123" +-> code: OK +-> status: ACCEPT + +> finalize_block "abc=123" +-> code: OK +-> code: OK +-> data.hex: 0x0200000000000000 + +> commit +-> code: OK + +> info +-> code: OK +-> data: {"size":1} +-> data.hex: 0x7B2273697A65223A317D + +> query "abc" +-> code: OK +-> log: exists +-> height: 0 +-> key: abc +-> key.hex: 616263 +-> value: 123 +-> value.hex: 313233 + +> finalize_block "def=xyz" "ghi=123" +-> code: OK +-> code: OK +-> code: OK +-> data.hex: 0x0600000000000000 + +> commit +-> code: OK + +> query "def" +-> code: OK +-> log: exists +-> height: 0 +-> key: def +-> key.hex: 646566 +-> value: xyz +-> value.hex: 78797A +``` + +Note that if we do `finalize_block "abc" ...` it will store `(abc, abc)`, but if +we do `finalize_block "abc=efg" ...` it will store `(abc, efg)`. + +You could put the commands in a file and run +`abci-cli --verbose batch < myfile`. + +Note that the `abci-cli` is designed strictly for testing and debugging. In a real +deployment, the role of sending messages is taken by CometBFT, which +connects to the app using four separate connections, each with its own +pattern of messages. + +For examples of running an ABCI app with CometBFT, see the +[getting started guide](/cometbft/v0.39/docs/app-dev/Getting-Started). + +## Bounties + +Want to write an app in your favorite language?! We'd be happy +to add you to our [ecosystem](https://github.com/tendermint/awesome#ecosystem)! +See [funding](https://github.com/interchainio/funding) opportunities from the +[Interchain Foundation](https://interchain.io) for implementations in new languages and more. diff --git a/cometbft/v0.39/docs/core/RPC.mdx b/cometbft/v0.39/docs/core/RPC.mdx new file mode 100644 index 000000000..e3248456a --- /dev/null +++ b/cometbft/v0.39/docs/core/RPC.mdx @@ -0,0 +1,52 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/docs/core/RPC' +title: RPC +order: 9 +--- + +CometBFT provides a comprehensive RPC API for interacting with nodes, querying blockchain data, broadcasting transactions, and subscribing to real-time events. + + + Visit the API Reference page for a list of all CometBFT RPC methods + + +The RPC server supports multiple protocols: +- **URI over HTTP** - REST-like interface for simple queries +- **JSONRPC over HTTP** - Standard JSON-RPC 2.0 protocol +- **JSONRPC over WebSockets** - Persistent connection with subscription support + + +## Quick Start + +By default, the RPC server listens on `tcp://127.0.0.1:26657`. You can query it using curl: + +```bash +# Get node status +curl http://localhost:26657/status + +# Get block at height 5 +curl http://localhost:26657/block?height=5 + +# Health check +curl http://localhost:26657/health +``` + +## Configuration + +The RPC server can be configured in your `config.toml` file under the `[rpc]` section: + +```toml +[rpc] + +# TCP or UNIX socket address for the RPC server to listen on +laddr = "tcp://127.0.0.1:26657" + +# A list of origins a cross-domain request can be executed from +cors_allowed_origins = [] + +# Maximum number of simultaneous connections +max_open_connections = 900 +``` + +See the [Configuration](/cometbft/v0.39/docs/core/configuration) page for more details on RPC configuration options. diff --git a/cometbft/v0.39/docs/core/Running-in-production.mdx b/cometbft/v0.39/docs/core/Running-in-production.mdx new file mode 100644 index 000000000..d2897af93 --- /dev/null +++ b/cometbft/v0.39/docs/core/Running-in-production.mdx @@ -0,0 +1,414 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/docs/core/Running-in-production' +title: Running in production +order: 4 +--- + +## Database + +By default, CometBFT uses the `syndtr/goleveldb` package for its in-process +key-value database. If you want maximal performance, it may be best to install +the real C implementation of LevelDB and compile CometBFT to use that using +`make build COMETBFT_BUILD_OPTIONS=cleveldb`. See the [install +instructions](/cometbft/v0.39/docs/guides/Install-CometBFT) for details. + +CometBFT keeps multiple distinct databases in the `$CMTHOME/data`: + +- `blockstore.db`: Keeps the entire blockchain - stores blocks, + block commits, and block metadata, each indexed by height. Used to sync new + peers. +- `evidence.db`: Stores all verified evidence of misbehavior. +- `state.db`: Stores the current blockchain state (i.e. height, validators, + consensus params). Only grows if consensus params or validators change. Also + used to temporarily store intermediate results during block processing. +- `tx_index.db`: Indexes transactions by tx hash and height. The tx results are indexed if they are added to the `FinalizeBlock` response in the application. + +By default, CometBFT will only index transactions by their hash and height. If +you want the result events to be indexed, see [indexing +transactions](/cometbft/v0.39/docs/app-dev/Indexing-Transactions) for details. + +Applications can expose block pruning strategies to the node operator. +Please read the documentation of your application to find out more details. + +Applications can use [state sync](/cometbft/v0.39/docs/core/state-sync) to help nodes bootstrap quickly. + +## Logging + +Default logging level (`log_level = "main:info,state:info,statesync:info,*:error"`) should suffice for +normal operation mode. Read [this +post](https://blog.cosmos.network/one-of-the-exciting-new-features-in-0-10-0-release-is-smart-log-level-flag-e2506b4ab756) +for details on how to configure the `log_level` config variable. Some of the +modules can be found [here](/cometbft/v0.39/docs/core/how-to-read-logs#list-of-modules). If +you're trying to debug CometBFT or asked to provide logs with debug +logging level, you can do so by running CometBFT with +`--log_level="*:debug"`. + +## Write Ahead Logs (WAL) + +CometBFT uses write ahead logs for the consensus (`cs.wal`) and the mempool +(`mempool.wal`). Both WALs have a max size of 1GB and are automatically rotated. + +### Consensus WAL + +The `consensus.wal` is used to ensure we can recover from a crash at any point +in the consensus state machine. +It writes all consensus messages (timeouts, proposals, block parts, or votes) +to a single file, flushing to disk before processing messages from its own +validator. Since CometBFT validators are expected to never sign a conflicting vote, the +WAL ensures we can always recover deterministically to the latest state of the consensus without +using the network or re-signing any consensus messages. + +If your `consensus.wal` is corrupted, see [below](#wal-corruption). + +### Mempool WAL + +The `mempool.wal` logs all incoming transactions before running CheckTx, but is +otherwise not used in any programmatic way. It's just a kind of manual +safeguard. Note the mempool provides no durability guarantees - a tx sent to one or many nodes +may never make it into the blockchain if those nodes crash before being able to +propose it. Clients must monitor their transactions by subscribing over websockets, +polling for them, or using `/broadcast_tx_commit`. In the worst case, transactions can be +resent from the mempool WAL manually. + +For the above reasons, the `mempool.wal` is disabled by default. To enable, set +`mempool.wal_dir` to where you want the WAL to be located (e.g. +`data/mempool.wal`). + +## DoS Exposure and Mitigation + +Validators are supposed to set up [Sentry Node Architecture](/cometbft/v0.39/docs/core/Validators) +to prevent Denial-of-Service attacks. + +### P2P + +The core of the CometBFT peer-to-peer system is `MConnection`. Each +connection has `MaxPacketMsgPayloadSize`, which is the maximum packet +size and bounded send & receive queues. One can impose restrictions on +send & receive rate per connection (`SendRate`, `RecvRate`). + +The number of open P2P connections can become quite large and hit the operating system's open +file limit (since TCP connections are considered files on UNIX-based systems). Nodes should be +given a sizable open file limit, e.g. 8192, via `ulimit -n 8192` or other deployment-specific +mechanisms. + +### RPC + +#### Attack Exposure and Mitigation + +**It is generally not recommended for RPC endpoints to be exposed publicly, and +especially so if the node in question is a validator**, as the CometBFT RPC does +not currently provide advanced security features. Public exposure of RPC +endpoints without appropriate protection can make the associated node vulnerable +to a variety of attacks. + +It is entirely up to operators to ensure, if nodes' RPC endpoints have to be +exposed publicly, that appropriate measures have been taken to mitigate against +attacks. Some examples of mitigation measures include, but are not limited to: + +- Never publicly exposing the RPC endpoints of validators (i.e. if the RPC + endpoints absolutely have to be exposed, ensure you do so only on full nodes + and with appropriate protection) +- Correct usage of rate-limiting, authentication, and caching (e.g. as provided + by reverse proxies like [nginx](https://nginx.org/) and/or DDoS protection + services like [Cloudflare](https://www.cloudflare.com)) +- Only exposing the specific endpoints absolutely necessary for the relevant use + cases (configurable via nginx/Cloudflare/etc.) + +If no expertise is available to the operator to assist with securing nodes' RPC +endpoints, it is strongly recommended to never expose those endpoints publicly. + +**Under no condition should any of the [unsafe RPC endpoints](/cometbft/v0.39/api-reference/rpc/index#endpoint-categories) +ever be exposed publicly.** + +#### Endpoints Returning Multiple Entries + +Endpoints returning multiple entries are limited by default to return 30 +elements (100 max). See the [RPC Documentation](/cometbft/v0.39/api-reference/rpc/index) +for more information. + +## Debugging CometBFT + +If you ever have to debug CometBFT, the first thing you should probably do is +check out the logs. See [How to read logs](/cometbft/v0.39/docs/core/how-to-read-logs), where we +explain what certain log statements mean. + +If, after skimming through the logs, things are still not clear, the next thing +to try is querying the `/status` RPC endpoint. It provides the necessary info: +whether the node is syncing or not, what height it is on, etc. + +```bash +curl http(s)://{ip}:{rpcPort}/status +``` + +`/dump_consensus_state` will give you a detailed overview of the consensus +state (proposer, latest validators, peer states). From it, you should be able +to figure out why, for example, the network had halted. + +```bash +curl http(s)://{ip}:{rpcPort}/dump_consensus_state +``` + +There is a reduced version of this endpoint - `/consensus_state`, which returns +just the votes seen at the current height. + +If, after consulting with the logs and the above endpoints, you still have no idea +what's happening, consider using the `cometbft debug kill` subcommand. This +command will scrape all the available info and kill the process. See +[Debugging](/cometbft/v0.39/docs/tools/debugging) for the exact format. + +You can inspect the resulting archive yourself or create an issue on +[Github](https://github.com/cometbft/cometbft). Before opening an issue, +however, be sure to check if there's [no existing +issue](https://github.com/cometbft/cometbft/issues) already. + +## Monitoring CometBFT + +Each CometBFT instance has a standard `/health` RPC endpoint, which responds +with 200 (OK) if everything is fine and 500 (or no response) if something is +wrong. + +Other useful endpoints include the previously mentioned `/status`, `/net_info`, and +`/validators`. + +CometBFT can also report and serve Prometheus metrics. See +[Metrics](/cometbft/v0.39/docs/core/metrics). + +The `cometbft debug dump` subcommand can be used to periodically dump useful +information into an archive. See [Debugging](/cometbft/v0.39/docs/tools/debugging) for more +information. + +## What happens when my app dies + +You are supposed to run CometBFT under a [process +supervisor](https://en.wikipedia.org/wiki/Process_supervision) (like +systemd or runit). It will ensure CometBFT is always running (despite +possible errors). + +Getting back to the original question, if your application dies, +CometBFT will panic. After a process supervisor restarts your +application, CometBFT should be able to reconnect successfully. The +order of restart does not matter for it. + +## Signal handling + +We catch SIGINT and SIGTERM and try to clean up nicely. For other +signals we use the default behavior in Go: +[Default behavior of signals in Go programs](https://golang.org/pkg/os/signal/#hdr-Default_behavior_of_signals_in_Go_programs). + +## Corruption + +**NOTE:** Make sure you have a backup of the CometBFT data directory. + +### Possible causes + +Remember that most corruption is caused by hardware issues: + +- RAID controllers with faulty/worn out battery backup, and an unexpected power loss +- Hard disk drives with write-back cache enabled, and an unexpected power loss +- Cheap SSDs with insufficient power-loss protection, and an unexpected power loss +- Defective RAM +- Defective or overheating CPU(s) + +Other causes can be: + +- Database systems configured with fsync=off and an OS crash or power loss +- Filesystems configured to use write barriers plus a storage layer that ignores write barriers. LVM is a particular culprit. +- CometBFT bugs +- Operating system bugs +- Admin error (e.g., directly modifying CometBFT data-directory contents) + +(Source: [https://wiki.postgresql.org/wiki/Corruption](https://wiki.postgresql.org/wiki/Corruption)) + +### WAL Corruption + +If consensus WAL is corrupted at the latest height and you are trying to start +CometBFT, replay will fail with panic. + +Recovering from data corruption can be hard and time-consuming. Here are two approaches you can take: + +1. Delete the WAL file and restart CometBFT. It will attempt to sync with other peers. +2. Try to repair the WAL file manually: + +1) Create a backup of the corrupted WAL file: + + ```sh + cp "$CMTHOME/data/cs.wal/wal" > /tmp/corrupted_wal_backup + ``` + +2) Use `./scripts/wal2json` to create a human-readable version: + + ```sh + ./scripts/wal2json/wal2json "$CMTHOME/data/cs.wal/wal" > /tmp/corrupted_wal + ``` + +3) Search for a "CORRUPTED MESSAGE" line. +4) By looking at the previous message and the message after the corrupted one + and looking at the logs, try to rebuild the message. If the subsequent + messages are marked as corrupted too (this may happen if the length header + got corrupted or some writes did not make it to the WAL ~ truncation), + then remove all the lines starting from the corrupted one and restart + CometBFT. + + ```sh + $EDITOR /tmp/corrupted_wal + ``` + +5) After editing, convert this file back into binary form by running: + + ```sh + ./scripts/json2wal/json2wal /tmp/corrupted_wal $CMTHOME/data/cs.wal/wal + ``` + +## Hardware + +### Processor and Memory + +While actual specs vary depending on the load and validator count, minimal +requirements are: + +- 1GB RAM +- 25GB of disk space +- 1.4 GHz CPU + +SSD disks are preferable for applications with high transaction throughput. + +Recommended: + +- 2GB RAM +- 100GB SSD +- x64 2.0 GHz 2v CPU + +While for now, CometBFT stores all the history and it may require significant +disk space over time, we are planning to implement state syncing (See [this +issue](https://github.com/tendermint/tendermint/issues/828)). So, storing all +the past blocks will not be necessary. + +### Validator signing on 32 bit architectures (or ARM) + +Both our `ed25519` and `secp256k1` implementations require constant time +`uint64` multiplication. Non-constant time crypto can (and has) leaked +private keys on both `ed25519` and `secp256k1`. This doesn't exist in hardware +on 32 bit x86 platforms ([source](https://bearssl.org/ctmul.html)), and it +depends on the compiler to enforce that it is constant time. It's unclear at +this point whether the Golang compiler does this correctly for all +implementations. + +**We do not support nor recommend running a validator on 32 bit architectures OR +the "VIA Nano 2000 Series", and the architectures in the ARM section rated +"S-".** + +### Operating Systems + +CometBFT can be compiled for a wide range of operating systems thanks to the Go +language (the list of \$OS/\$ARCH pairs can be found +[here](https://golang.org/doc/install/source#environment)). + +While we do not favor any operating system, more secure and stable Linux server +distributions (like CentOS) should be preferred over desktop operating systems +(like Mac OS). + +### Miscellaneous + +NOTE: If you are going to use CometBFT in a public domain, make sure +you read [hardware recommendations](https://cosmos.network/validators) for a validator in the +Cosmos network. + +## Configuration parameters + +- `p2p.flush_throttle_timeout` +- `p2p.max_packet_msg_payload_size` +- `p2p.send_rate` +- `p2p.recv_rate` + +If you are going to use CometBFT in a private domain and you have a +private high-speed network among your peers, it makes sense to lower +flush throttle timeout and increase other params. + +```toml +[p2p] + +send_rate=20000000 # 2MB/s +recv_rate=20000000 # 2MB/s +flush_throttle_timeout=10 +max_packet_msg_payload_size=10240 # 10KB +``` + +- `mempool.recheck` + +After every block, CometBFT rechecks every transaction left in the +mempool to see if transactions committed in that block affected the +application state, so some of the transactions left may become invalid. +If that does not apply to your application, you can disable it by +setting `mempool.recheck=false`. + +- `mempool.broadcast` + +Setting this to false will stop the mempool from relaying transactions +to other peers until they are included in a block. It means only the +peer you send the tx to will see it until it is included in a block. + +- `consensus.skip_timeout_commit` + +We want `skip_timeout_commit=false` when there is economics on the line +because proposers should wait to hear for more votes. But if you don't +care about that and want the fastest consensus, you can skip it. It will +be kept false by default for public deployments (e.g. [Cosmos +Hub](https://hub.cosmos.network/)) while for enterprise +applications, setting it to true is not a problem. + +- `consensus.peer_gossip_sleep_duration` + +You can try to reduce the time your node sleeps before checking if +there's something to send its peers. + +- `consensus.timeout_commit` + +You can also try lowering `timeout_commit` (time we sleep before +proposing the next block). + +- `p2p.addr_book_strict` + +By default, CometBFT checks whether a peer's address is routable before +saving it to the address book. The address is considered as routable if the IP +is [valid and within allowed ranges](https://github.com/cometbft/cometbft/blob/v0.38.x/p2p/netaddress.go#L258). + +This may not be the case for private or local networks, where your IP range is usually +strictly limited and private. In that case, you need to set `addr_book_strict` +to `false` (turn it off). + +- `rpc.max_open_connections` + +By default, the number of simultaneous connections is limited because most OSes +give you a limited number of file descriptors. + +If you want to accept a greater number of connections, you will need to increase +these limits. + +[Sysctls to tune the system to be able to open more connections](https://github.com/launchdarkly/tcpkali/blob/master/doc/tcpkali.man.md#sysctls-to-tune-the-system-to-be-able-to-open-more-connections) + +The process file limits must also be increased, e.g. via `ulimit -n 8192`. + +...for N connections, such as 50k: + +```md +kern.maxfiles=10000+2*N # BSD +kern.maxfilesperproc=100+2*N # BSD +kern.ipc.maxsockets=10000+2*N # BSD +fs.file-max=10000+2*N # Linux +net.ipv4.tcp_max_orphans=N # Linux + +# For load-generating clients. +net.ipv4.ip_local_port_range="10000 65535" # Linux. +net.inet.ip.portrange.first=10000 # BSD/Mac. +net.inet.ip.portrange.last=65535 # (Enough for N < 55535) +net.ipv4.tcp_tw_reuse=1 # Linux +net.inet.tcp.maxtcptw=2*N # BSD + +# If using netfilter on Linux: +net.netfilter.nf_conntrack_max=N +echo $((N/8)) > /sys/module/nf_conntrack/parameters/hashsize +``` + +A similar option exists for limiting the number of gRPC connections - +`rpc.grpc_max_open_connections`. diff --git a/cometbft/v0.39/docs/core/Subscribing-to-events-via-Websocket.mdx b/cometbft/v0.39/docs/core/Subscribing-to-events-via-Websocket.mdx new file mode 100644 index 000000000..eb5fe4c5b --- /dev/null +++ b/cometbft/v0.39/docs/core/Subscribing-to-events-via-Websocket.mdx @@ -0,0 +1,97 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/docs/core/Subscribing-to-events-via-Websocket' +title: Subscribing to events via Websocket +order: 7 +--- + +CometBFT emits different events, which you can subscribe to via +[Websocket](https://en.wikipedia.org/wiki/WebSocket). This can be useful +for third-party applications (for analysis) or for inspecting state. + +[List of events](https://godoc.org/github.com/cometbft/cometbft/types#pkg-constants) + +To connect to a node via websocket from the CLI, you can use a tool such as +[wscat](https://github.com/websockets/wscat) and run: + +```sh +wscat -c ws://127.0.0.1:26657/websocket +``` + +NOTE: If your node's RPC endpoint is TLS-enabled, use the scheme `wss` instead of `ws`. + +You can subscribe to any of the events above by calling the `subscribe` RPC +method via Websocket along with a valid query. + +```json +{ + "jsonrpc": "2.0", + "method": "subscribe", + "id": 0, + "params": { + "query": "tm.event='NewBlock'" + } +} +``` + +Check out [the API docs](/cometbft/v0.39/api-reference/rpc/index) for +more information on query syntax and other options. + +You can also use tags, given you have included them in the FinalizeBlock +response, to query transaction results. See [Indexing +transactions](/cometbft/v0.39/docs/app-dev/Indexing-Transactions) for details. + +## Query parameter and event type restrictions + +While CometBFT imposes no restrictions on the application with regard to the type of +the event output, there are several considerations that need to be taken into account +when querying events with numeric values. + +- Queries convert all numeric event values to `big.Float`, provided by `math/big`. Integers +are converted into a float with a precision equal to the number of bits needed +to represent this integer. This is done to avoid precision loss for big integers when they +are converted with the default precision (`64`). +- When comparing two values, if either one of them is a float, the other one will be represented +as a big float. Integers are again parsed as big floats with a precision equal to the number +of bits required to represent them. +- As with all floating point comparisons, comparing floats with decimal values can lead to imprecise +results. +- Queries cannot include negative numbers. + +Prior to version `v0.38.x`, floats were not supported as query parameters. + +## ValidatorSetUpdates + +When the validator set changes, the ValidatorSetUpdates event is published. The +event carries a list of pubkey/power pairs. The list is the same as what +CometBFT receives from the ABCI application (see the [EndBlock +section](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/abci/abci++_methods.md#endblock) in +the ABCI spec). + +Response: + +```json +{ + "jsonrpc": "2.0", + "id": 0, + "result": { + "query": "tm.event='ValidatorSetUpdates'", + "data": { + "type": "tendermint/event/ValidatorSetUpdates", + "value": { + "validator_updates": [ + { + "address": "09EAD022FD25DE3A02E64B0FE9610B1417183EE4", + "pub_key": { + "type": "tendermint/PubKeyEd25519", + "value": "ww0z4WaZ0Xg+YI10w43wTWbBmM3dpVza4mmSQYsd0ck=" + }, + "voting_power": "10", + "proposer_priority": "0" + } + ] + } + } + } +} +``` diff --git a/cometbft/v0.39/docs/core/Using-CometBFT.mdx b/cometbft/v0.39/docs/core/Using-CometBFT.mdx new file mode 100644 index 000000000..6571168bb --- /dev/null +++ b/cometbft/v0.39/docs/core/Using-CometBFT.mdx @@ -0,0 +1,573 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/docs/core/Using-CometBFT' +title: Using CometBFT +order: 2 +--- +{/* trigger rebuild */} + +This is a guide to using the `cometbft` program from the command line. +It assumes only that you have the `cometbft` binary installed and have +some rudimentary idea of what CometBFT and ABCI are. + +You can see the help menu with `cometbft --help`, and the version +number with `cometbft version`. + +## Directory Root + +The default directory for blockchain data is `~/.cometbft`. Override +this by setting the `CMTHOME` environment variable. + +## Initialize + +Initialize the root directory by running: + +```sh +cometbft init +``` + +This will create a new private key (`priv_validator_key.json`) and a +genesis file (`genesis.json`) containing the associated public key in +`$CMTHOME/config`. This is all that's necessary to run a local testnet +with one validator. + +For more elaborate initialization, see the testnet command: + +```sh +cometbft testnet --help +``` + +### Genesis + +The `genesis.json` file in `$CMTHOME/config/` defines the initial +CometBFT state upon genesis of the blockchain ([see +definition](https://github.com/cometbft/cometbft/blob/v0.38.x/types/genesis.go)). + +#### Fields + +- `genesis_time`: Official time of blockchain start. +- `chain_id`: ID of the blockchain. **This must be unique for + every blockchain.** If your testnet blockchains do not have unique + chain IDs, you will have a bad time. The ChainID must be less than 50 symbols. +- `initial_height`: Height at which CometBFT should begin. If a blockchain is conducting a network upgrade, + starting from the stopped height brings uniqueness to previous heights. +- `consensus_params` ([see spec](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/core/data_structures.md#consensusparams)) + - `block` + - `max_bytes`: Max block size, in bytes. + - `max_gas`: Max gas per block. + - `evidence` + - `max_age_num_blocks`: Max age of evidence, in blocks. The basic formula + for calculating this is: MaxAgeDuration / (average block time). + - `max_age_duration`: Max age of evidence, in time. It should correspond + with an app's "unbonding period" or other similar mechanism for handling + [Nothing-At-Stake + attacks](https://vitalik.eth.limo/general/2017/12/31/pos_faq.html#what-is-the-nothing-at-stake-problem-and-how-can-it-be-fixed). + - `max_bytes`: This sets the maximum size in bytes of evidence that can be committed + in a single block and should fall comfortably under the max block bytes. + - `validator` + - `pub_key_types`: Public key types validators can use. + - `version` + - `app_version`: ABCI application version. +- `validators`: List of initial validators. Note this may be overridden entirely by the + application, and may be left empty to make explicit that the + application will initialize the validator set upon `InitChain`. + - `pub_key`: The first element specifies the key type, + using the declared `PubKeyName` for the adopted + [key type](https://github.com/cometbft/cometbft/blob/v0.38.x/crypto/ed25519/ed25519.go#L36). + The second element are the pubkey bytes. + - `power`: The validator's voting power. + - `name`: Name of the validator (optional). +- `app_hash`: The expected application hash (as returned by the + `ResponseInfo` ABCI message) upon genesis. If the app's hash does + not match, CometBFT will panic. +- `app_state`: The application state (e.g., initial distribution + of tokens). + +> :warning: **ChainID must be unique to every blockchain. Reusing old chainID can cause issues** + +#### Sample genesis.json + +```json +{ + "genesis_time": "2023-01-21T11:17:42.341227868Z", + "chain_id": "test-chain-ROp9KF", + "initial_height": "0", + "consensus_params": { + "block": { + "max_bytes": "22020096", + "max_gas": "-1", + }, + "evidence": { + "max_age_num_blocks": "100000", + "max_age_duration": "172800000000000", + "max_bytes": 51200, + }, + "validator": { + "pub_key_types": [ + "ed25519" + ] + } + }, + "validators": [ + { + "address": "B547AB87E79F75A4A3198C57A8C2FDAF8628CB47", + "pub_key": { + "type": "tendermint/PubKeyEd25519", + "value": "P/V6GHuZrb8rs/k1oBorxc6vyXMlnzhJmv7LmjELDys=" + }, + "power": "10", + "name": "" + } + ], + "app_hash": "" +} +``` + +## Run + +To run a CometBFT node, use: + +```bash +cometbft node +``` + +By default, CometBFT will try to connect to an ABCI application on +`tcp://127.0.0.1:26658`. If you have the `kvstore` ABCI app installed, run it in +another window. If you don't, kill CometBFT and run an in-process version of +the `kvstore` app: + +```bash +cometbft node --proxy_app=kvstore +``` + +After a few seconds, you should see blocks start streaming in. Note that blocks +are produced regularly, even if there are no transactions. See [No Empty +Blocks](#no-empty-blocks) below to modify this setting. + +CometBFT supports in-process versions of the `counter`, `kvstore`, and `noop` +apps that ship as examples with `abci-cli`. It's easy to compile your app +in-process with CometBFT if it's written in Go. If your app is not written in +Go, run it in another process, and use the `--proxy_app` flag to specify the +address of the socket it is listening on, for instance: + +```bash +cometbft node --proxy_app=/var/run/abci.sock +``` + +You can find out what flags are supported by running `cometbft node --help`. + +## Transactions + +To send a transaction, use `curl` to make requests to the CometBFT RPC +server, for example: + +```sh +curl http://localhost:26657/broadcast_tx_commit?tx=\"abcd\" +``` + +We can see the chain's status at the `/status` endpoint: + +```sh +curl http://localhost:26657/status | json_pp +``` + +and the `latest_app_hash` in particular: + +```sh +curl http://localhost:26657/status | json_pp | grep latest_app_hash +``` + +Visit `http://localhost:26657` in your browser to see the list of other +endpoints. Some take no arguments (like `/status`), while others specify +the argument name and use `_` as a placeholder. + + +> TIP: Find the RPC Documentation [here](/cometbft/v0.39/api-reference/rpc/index) + +### Formatting + +The following nuances when sending/formatting transactions should be +taken into account: + +With `GET`: + +To send a UTF8 string byte array, quote the value of the tx parameter: + +```sh +curl 'http://localhost:26657/broadcast_tx_commit?tx="hello"' +``` + +which sends a 5 byte transaction: "h e l l o" \[68 65 6c 6c 6f\]. + +Note the URL must be wrapped with single quotes, else bash will ignore +the double quotes. To avoid the single quotes, escape the double quotes: + +```sh +curl http://localhost:26657/broadcast_tx_commit?tx=\"hello\" +``` + +Using a special character: + +```sh +curl 'http://localhost:26657/broadcast_tx_commit?tx="€5"' +``` + +sends a 4 byte transaction: "€5" (UTF8) \[e2 82 ac 35\]. + +To send as raw hex, omit quotes AND prefix the hex string with `0x`: + +```sh +curl http://localhost:26657/broadcast_tx_commit?tx=0x01020304 +``` + +which sends a 4 byte transaction: \[01 02 03 04\]. + +With `POST` (using `json`), the raw hex must be `base64` encoded: + +```sh +curl --data-binary '{"jsonrpc":"2.0","id":"anything","method":"broadcast_tx_commit","params": {"tx": "AQIDBA=="}}' -H 'content-type:text/plain;' http://localhost:26657 +``` + +which sends the same 4 byte transaction: \[01 02 03 04\]. + +Note that raw hex cannot be used in `POST` transactions. + +## Reset + +> :warning: **UNSAFE** Only do this in development and only if you can +afford to lose all blockchain data! + + +To reset a blockchain, stop the node and run: + +```sh +cometbft unsafe_reset_all +``` + +This command will remove the data directory and reset private validator and +address book files. + +## Configuration + +CometBFT uses a `config.toml` for configuration. For details, see [the +config specification](/cometbft/v0.39/docs/core/configuration). + +Notable options include the socket address of the application +(`proxy_app`), the listening address of the CometBFT peer +(`p2p.laddr`), and the listening address of the RPC server +(`rpc.laddr`). + +Some fields from the config file can be overwritten with flags. + +## No Empty Blocks + +While the default behavior of `cometbft` is still to create blocks +approximately once per second, it is possible to disable empty blocks or +set a block creation interval. In the former case, blocks will be +created when there are new transactions or when the AppHash changes. + +To configure CometBFT to not produce empty blocks unless there are +transactions or the app hash changes, run CometBFT with this +additional flag: + +```sh +cometbft node --consensus.create_empty_blocks=false +``` + +or set the configuration via the `config.toml` file: + +```toml +[consensus] +create_empty_blocks = false +``` + +Remember: because the default is to _create empty blocks_, avoiding +empty blocks requires the config option to be set to `false`. + +The block interval setting allows for a delay (in time.Duration format [ParseDuration](https://golang.org/pkg/time/#ParseDuration)) between the +creation of each new empty block. It can be set with this additional flag: + +```sh +--consensus.create_empty_blocks_interval="5s" +``` + +or set the configuration via the `config.toml` file: + +```toml +[consensus] +create_empty_blocks_interval = "5s" +``` + +With this setting, empty blocks will be produced every 5s if no block +has been produced otherwise, regardless of the value of +`create_empty_blocks`. + +## Broadcast API + +Earlier, we used the `broadcast_tx_commit` endpoint to send a +transaction. When a transaction is sent to a CometBFT node, it will +run via `CheckTx` against the application. If it passes `CheckTx`, it +will be included in the mempool, broadcast to other peers, and +eventually included in a block. + +Since there are multiple phases to processing a transaction, we offer +multiple endpoints to broadcast a transaction: + +```md +/broadcast_tx_async +/broadcast_tx_sync +/broadcast_tx_commit +``` + +These correspond to no-processing, processing through the mempool, and +processing through a block, respectively. That is, `broadcast_tx_async` +will return right away without waiting to hear if the transaction is +even valid, while `broadcast_tx_sync` will return with the result of +running the transaction through `CheckTx`. Using `broadcast_tx_commit` +will wait until the transaction is committed in a block or until some +timeout is reached, but will return right away if the transaction does +not pass `CheckTx`. The return value for `broadcast_tx_commit` includes +two fields, `check_tx` and `deliver_tx`, pertaining to the result of +running the transaction through those ABCI messages. + +The benefit of using `broadcast_tx_commit` is that the request returns +after the transaction is committed (i.e., included in a block), but that +can take on the order of a second. For a quick result, use +`broadcast_tx_sync`, but the transaction will not be committed until +later, and by that point its effect on the state may change. + +Note the mempool does not provide strong guarantees - just because a tx passed +CheckTx (i.e., was accepted into the mempool) doesn't mean it will be committed, +as nodes with the tx in their mempool may crash before they get to propose. +For more information, see the [mempool +write-ahead-log](/cometbft/v0.39/docs/core/Running-in-production#mempool-wal). + +## CometBFT Networks + +When `cometbft init` is run, both a `genesis.json` and +`priv_validator_key.json` are created in `~/.cometbft/config`. The +`genesis.json` might look like: + +```json +{ + "validators" : [ + { + "pub_key" : { + "value" : "h3hk+QE8c6QLTySp8TcfzclJw/BG79ziGB/pIA+DfPE=", + "type" : "tendermint/PubKeyEd25519" + }, + "power" : 10, + "name" : "" + } + ], + "app_hash" : "", + "chain_id" : "test-chain-rDlYSN", + "genesis_time" : "0001-01-01T00:00:00Z" +} +``` + +And the `priv_validator_key.json`: + +```json +{ + "last_step" : 0, + "last_round" : "0", + "address" : "B788DEDE4F50AD8BC9462DE76741CCAFF87D51E2", + "pub_key" : { + "value" : "h3hk+QE8c6QLTySp8TcfzclJw/BG79ziGB/pIA+DfPE=", + "type" : "tendermint/PubKeyEd25519" + }, + "last_height" : "0", + "priv_key" : { + "value" : "JPivl82x+LfVkp8i3ztoTjY6c6GJ4pBxQexErOCyhwqHeGT5ATxzpAtPJKnxNx/NyUnD8Ebv3OIYH+kgD4N88Q==", + "type" : "tendermint/PrivKeyEd25519" + } +} +``` + +The `priv_validator_key.json` actually contains a private key, and should +thus be kept absolutely secret; for now we work with the plain text. +Note the `last_` fields, which are used to prevent us from signing +conflicting messages. + +Note also that the `pub_key` (the public key) in the +`priv_validator_key.json` is also present in the `genesis.json`. + +The genesis file contains the list of public keys which may participate +in the consensus, and their corresponding voting power. Greater than 2/3 +of the voting power must be active (i.e., the corresponding private keys +must be producing signatures) for the consensus to make progress. In our +case, the genesis file contains the public key of our +`priv_validator_key.json`, so a CometBFT node started with the default +root directory will be able to make progress. Voting power uses an int64 +but must be positive, thus the range is 0 through 9223372036854775807. +Because of how the current proposer selection algorithm works, we do not +recommend having voting powers greater than 10\^12 (i.e., 1 trillion). + +If we want to add more nodes to the network, we have two choices: we can +add a new validator node, who will also participate in the consensus by +proposing blocks and voting on them, or we can add a new non-validator +node, who will not participate directly, but will verify and keep up +with the consensus protocol. + +### Peers + +#### Seed + +A seed node is a node who relays the addresses of other peers which they know +of. These nodes constantly crawl the network to try to get more peers. The +addresses which the seed node relays get saved into a local address book. Once +these are in the address book, you will connect to those addresses directly. +Basically the seed node's job is just to relay everyone's addresses. You won't +connect to seed nodes once you have received enough addresses, so typically you +only need them on the first start. The seed node will immediately disconnect +from you after sending you some addresses. + +#### Persistent Peer + +Persistent peers are people you want to be constantly connected with. If you +disconnect you will try to connect directly back to them as opposed to using +another address from the address book. On restarts you will always try to +connect to these peers regardless of the size of your address book. + +All peers relay peers they know of by default. This is called the peer exchange +protocol (PEX). With PEX, peers will be gossiping about known peers and forming +a network, storing peer addresses in the addrbook. Because of this, you don't +have to use a seed node if you have a live persistent peer. + +#### Connecting to Peers + +To connect to peers on start-up, specify them in the +`$CMTHOME/config/config.toml` or on the command line. Use `seeds` to +specify seed nodes, and +`persistent_peers` to specify peers that your node will maintain +persistent connections with. + +For example, + +```sh +cometbft node --p2p.seeds "f9baeaa15fedf5e1ef7448dd60f46c01f1a9e9c4@1.2.3.4:26656,0491d373a8e0fcf1023aaf18c51d6a1d0d4f31bd@5.6.7.8:26656" +``` + +Alternatively, you can use the `/dial_seeds` endpoint of the RPC to +specify seeds for a running node to connect to: + +```sh +curl 'localhost:26657/dial_seeds?seeds=\["f9baeaa15fedf5e1ef7448dd60f46c01f1a9e9c4@1.2.3.4:26656","0491d373a8e0fcf1023aaf18c51d6a1d0d4f31bd@5.6.7.8:26656"\]' +``` + +Note, with PEX enabled, you +should not need seeds after the first start. + +If you want CometBFT to connect to a specific set of addresses and +maintain a persistent connection with each, you can use the +`--p2p.persistent_peers` flag or the corresponding setting in the +`config.toml` or the `/dial_peers` RPC endpoint to do it without +stopping the CometBFT instance. + +```sh +cometbft node --p2p.persistent_peers "429fcf25974313b95673f58d77eacdd434402665@10.11.12.13:26656,96663a3dd0d7b9d17d4c8211b191af259621c693@10.11.12.14:26656" + +curl 'localhost:26657/dial_peers?persistent=true&peers=\["429fcf25974313b95673f58d77eacdd434402665@10.11.12.13:26656","96663a3dd0d7b9d17d4c8211b191af259621c693@10.11.12.14:26656"\]' +``` + +### Adding a Non-Validator + +Adding a non-validator is simple. Just copy the original `genesis.json` +to `~/.cometbft/config` on the new machine and start the node, +specifying seeds or persistent peers as necessary. If no seeds or +persistent peers are specified, the node won't make any blocks, because +it's not a validator, and it won't hear about any blocks, because it's +not connected to the other peer. + +### Adding a Validator + +The easiest way to add new validators is to do it in the `genesis.json` +before starting the network. For instance, we could make a new +`priv_validator_key.json` and copy its `pub_key` into the above genesis. + +We can generate a new `priv_validator_key.json` with the command: + +```sh +cometbft gen_validator +``` + +Now we can update our genesis file. For instance, if the new +`priv_validator_key.json` looks like: + +```json +{ + "address" : "5AF49D2A2D4F5AD4C7C8C4CC2FB020131E9C4902", + "pub_key" : { + "value" : "l9X9+fjkeBzDfPGbUM7AMIRE6uJN78zN5+lk5OYotek=", + "type" : "tendermint/PubKeyEd25519" + }, + "priv_key" : { + "value" : "EDJY9W6zlAw+su6ITgTKg2nTZcHAH1NMTW5iwlgmNDuX1f35+OR4HMN88ZtQzsAwhETq4k3vzM3n6WTk5ii16Q==", + "type" : "tendermint/PrivKeyEd25519" + }, + "last_step" : 0, + "last_round" : "0", + "last_height" : "0" +} +``` + +then the new `genesis.json` will be: + +```json +{ + "validators" : [ + { + "pub_key" : { + "value" : "h3hk+QE8c6QLTySp8TcfzclJw/BG79ziGB/pIA+DfPE=", + "type" : "tendermint/PubKeyEd25519" + }, + "power" : 10, + "name" : "" + }, + { + "pub_key" : { + "value" : "l9X9+fjkeBzDfPGbUM7AMIRE6uJN78zN5+lk5OYotek=", + "type" : "cometbft/PubKeyEd25519" + }, + "power" : 10, + "name" : "" + } + ], + "app_hash" : "", + "chain_id" : "test-chain-rDlYSN", + "genesis_time" : "0001-01-01T00:00:00Z" +} +``` + +Update the `genesis.json` in `~/.cometbft/config`. Copy the genesis +file and the new `priv_validator_key.json` to the `~/.cometbft/config` on +a new machine. + +Now run `cometbft node` on both machines, and use either +`--p2p.persistent_peers` or the `/dial_peers` to get them to peer up. +They should start making blocks, and will only continue to do so as long +as both of them are online. + +To make a CometBFT network that can tolerate one of the validators +failing, you need at least four validator nodes (e.g., 2/3). + +Updating validators in a live network is supported but must be +explicitly programmed by the application developer. See the [application +developers guide](/cometbft/v0.39/docs/app-dev/Using-ABCI-CLI) for more details. + +### Local Network + +To run a network locally, say on a single machine, you must change the `_laddr` +fields in the `config.toml` (or using the flags) so that the listening +addresses of the various sockets don't conflict. Additionally, you must set +`addr_book_strict=false` in the `config.toml`, otherwise CometBFT's p2p +library will deny making connections to peers with the same IP address. + +### Upgrading + +See the +[UPGRADING.md](https://github.com/cometbft/cometbft/blob/v0.38.x/UPGRADING.md) +guide. You may need to reset your chain between major breaking releases. +Although, we expect CometBFT to have fewer breaking releases in the future +(especially after the 1.0 release). diff --git a/cometbft/v0.39/docs/core/Validators.mdx b/cometbft/v0.39/docs/core/Validators.mdx new file mode 100644 index 000000000..ab92f15f9 --- /dev/null +++ b/cometbft/v0.39/docs/core/Validators.mdx @@ -0,0 +1,101 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/docs/core/Validators' +title: Validators +order: 6 +--- + +Validators are responsible for committing new blocks in the blockchain. +These validators participate in the consensus protocol by broadcasting +_votes_ which contain cryptographic signatures signed by each +validator's private key. + +Some Proof-of-Stake consensus algorithms aim to create a "completely" +decentralized system where all stakeholders (even those who are not always +available online) participate in the committing of blocks. CometBFT has a +different approach to block creation. Validators are expected to be online, and +the set of validators is permissioned/curated by the ABCI application. +Proof-of-stake is not required, but can be implemented on top of CometBFT +consensus. That is, validators may be required to post collateral on-chain, +off-chain, or may not be required to post any collateral at all. + +Validators have a cryptographic key-pair and an associated amount of +"voting power". Voting power need not be the same. + +## Becoming a Validator + +There are two ways to become a validator. + +1. They can be pre-established in the [genesis state](/cometbft/v0.39/docs/core/Using-CometBFT#genesis) +2. The ABCI app responds to the FinalizeBlock message with changes to the + existing validator set. + +## Setting up a Validator + +When setting up a validator there are countless ways to configure your setup. This guide is aimed at showing one of them, the sentry node design. This design is mainly for DDoS prevention. + +### Network Layout + +![ALT Network Layout](../imgs/sentry_layout.png) + +The diagram is based on AWS; other cloud providers will have similar solutions to design a network. Running nodes is not limited to cloud providers; you can run nodes on bare metal systems as well. The architecture will be the same no matter which setup you decide to go with. + +The proposed network diagram is similar to the classical backend/frontend separation of services in a corporate environment. The "backend" in this case is the private network of the validator in the data center. The data center network might involve multiple subnets, firewalls, and redundancy devices, which are not detailed in this diagram. The important point is that the data center allows direct connectivity to the chosen cloud environment. Amazon AWS has "Direct Connect", while Google Cloud has "Partner Interconnect". This is a dedicated connection to the cloud provider (usually directly to your virtual private cloud instance in one of the regions). + +All sentry nodes (the "frontend") connect to the validator using this private connection. The validator does not have a public IP address to provide its services. + +Amazon has multiple availability zones within a region. One can install sentry nodes in other regions too. In this case, the second, third, and further regions need to have a private connection to the validator node. This can be achieved by VPC Peering ("VPC Network Peering" in Google Cloud). In this case, the second, third, and further region sentry nodes will be directed to the first region and through the direct connect to the data center, arriving at the validator. + +A more persistent solution (not detailed in the diagram) is to have multiple direct connections to different regions from the data center. This way VPC Peering is not mandatory, although still beneficial for the sentry nodes. This overcomes the risk of depending on one region. It is more costly. + +### Local Configuration + +![ALT Local Configuration](../imgs/sentry_local_config.png) + +The validator will only talk to the sentries that are provided, the sentry nodes will communicate to the validator via a secret connection and the rest of the network through a normal connection. The sentry nodes do have the option of communicating with each other as well. + +When initializing nodes there are five parameters in the `config.toml` that may need to be altered. + +- `pex:` boolean. This turns the peer exchange reactor on or off for a node. When `pex=false`, only the `persistent_peers` list is available for connection. +- `persistent_peers:` a comma-separated list of `nodeID@ip:port` values that define a list of peers that are expected to be online at all times. This is necessary at first startup because by setting `pex=false` the node will not be able to join the network. +- `unconditional_peer_ids:` comma-separated list of nodeID's. These nodes will be connected to no matter the limits of inbound and outbound peers. This is useful when sentry nodes have full address books. +- `private_peer_ids:` comma-separated list of nodeID's. These nodes will not be gossiped to the network. This is an important field as you do not want your validator IP gossiped to the network. +- `addr_book_strict:` boolean. By default, nodes with a routable address will be considered for connection. If this setting is turned off (false), non-routable IP addresses, like addresses in a private network, can be added to the address book. +- `double_sign_check_height` int64 height. How many blocks to look back to check existence of the node's consensus votes before joining consensus. When non-zero, the node will panic upon restart if the same consensus key was used to sign `double_sign_check_height` last blocks. So, validators should stop the state machine, wait for some blocks, and then restart the state machine to avoid panic. + +#### Validator Node Configuration + +| Config Option | Setting | +| ------------------------ | -------------------------- | +| pex | false | +| persistent_peers | list of sentry nodes | +| private_peer_ids | none | +| unconditional_peer_ids | optionally sentry node IDs | +| addr_book_strict | false | +| double_sign_check_height | 10 | + +The validator node should have `pex=false` so it does not gossip to the entire network. The persistent peers will be your sentry nodes. Private peers can be left empty as the validator is not trying to hide who it is communicating with. Setting unconditional peers is optional for a validator because they will not have full address books. + +#### Sentry Node Configuration + +| Config Option | Setting | +| ---------------------- | --------------------------------------------- | +| pex | true | +| persistent_peers | validator node, optionally other sentry nodes | +| private_peer_ids | validator node ID | +| unconditional_peer_ids | validator node ID, optionally sentry node IDs | +| addr_book_strict | false | + +The sentry nodes should be able to talk to the entire network, hence why `pex=true`. The persistent peers of a sentry node will be the validator, and optionally other sentry nodes. The sentry nodes should make sure that they do not gossip the validator's IP; to do this you must put the validator's nodeID as a private peer. The unconditional peer IDs will be the validator ID and optionally other sentry nodes. + +> Note: Do not forget to secure your node's firewalls when setting them up. + +More information can be found at this link: + +- [https://forum.cosmos.network/t/sentry-node-architecture-overview/454](https://forum.cosmos.network/t/sentry-node-architecture-overview/454) + +### Validator keys + +Protecting a validator's consensus key is the most important factor to consider when designing your setup. The key that a validator is given upon creation of the node is called a consensus key; it has to be online at all times in order to vote on blocks. It is **not recommended** to merely hold your private key in the default JSON file (`priv_validator_key.json`). Fortunately, the [Interchain Foundation](https://interchain.io) has worked with a team to build a key management server for validators. You can find documentation on how to use it [here](https://github.com/iqlusioninc/tmkms); it is used extensively in production. You are not limited to using this tool; there are also [HSMs](https://safenet.gemalto.com/data-encryption/hardware-security-modules-hsms/). There is no single recommended HSM. + +Currently CometBFT uses [Ed25519](https://ed25519.cr.yp.to/) keys which are widely supported across the security sector and HSMs. diff --git a/cometbft/v0.39/docs/core/block-structure.mdx b/cometbft/v0.39/docs/core/block-structure.mdx new file mode 100644 index 000000000..78687584d --- /dev/null +++ b/cometbft/v0.39/docs/core/block-structure.mdx @@ -0,0 +1,20 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/docs/core/block-structure' +title: Block Structure +order: 8 +--- + +The CometBFT consensus engine records all agreements by 2/3+ of nodes +into a blockchain, which is replicated among all nodes. This blockchain is +accessible via various RPC endpoints, mainly `/block?height=` to get the full +block, as well as `/blockchain?minHeight=_&maxHeight=_` to get a list of +headers. But what exactly is stored in these blocks? + +The [specification][data_structures] contains a detailed description of each +component—that's the best place to get started. + +To dig deeper, check out the [types package documentation][types]. + +[data_structures]: https://github.com/cometbft/cometbft/blob/v0.38.x/spec/core/data_structures.md +[types]: https://pkg.go.dev/github.com/cometbft/cometbft/types diff --git a/cometbft/v0.39/docs/core/block-sync.mdx b/cometbft/v0.39/docs/core/block-sync.mdx new file mode 100644 index 000000000..a1e30294b --- /dev/null +++ b/cometbft/v0.39/docs/core/block-sync.mdx @@ -0,0 +1,123 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/docs/core/block-sync' +title: Block Sync +order: 10 +--- + +*Formerly known as Fast Sync* + +In a proof-of-work blockchain, syncing with the chain is the same +process as staying up-to-date with the consensus: download blocks, and +look for the one with the most total work. In proof-of-stake, the +consensus process is more complex, as it involves rounds of +communication between the nodes to determine what block should be +committed next. Using this process to sync up with the blockchain from +scratch can take a very long time. It's much faster to just download +blocks and check the Merkle tree of validators than to run the real-time +consensus gossip protocol. + +## Using Block Sync + +When starting from scratch, nodes will use Block Sync mode. +In this mode, the CometBFT daemon +will sync hundreds of times faster than if it used the real-time consensus +process. Once caught up, the daemon will switch out of Block Sync and into +normal consensus mode. After running for some time, the node is considered +`caught up` if it has at least one peer and its height is at least as high as +the max reported peer height. See [the IsCaughtUp +method](https://github.com/cometbft/cometbft/blob/v0.38.x/blocksync/pool.go#L168). + +Note: While there have historically been multiple versions of blocksync (v0, v1, and v2), all versions +other than v0 have been deprecated in favor of the simplest and most well-understood algorithm. + +```toml +####################################################### +### Block Sync Configuration Options ### +####################################################### +[blocksync] + +# Block Sync version to use: +# +# In v0.37, v1 and v2 of the block sync protocols were deprecated. +# Please use v0 instead. +# +# 1) "v0" - the default block sync implementation +version = "v0" +``` + +## AdaptiveSync + + + This is an experimental feature. + It has been tested under a range of network conditions and perturbations, but you should validate it + for your specific workload before enabling it on a production mainnet chain. + + +AdaptiveSync allows a node to run blocksync and consensus at the same time. + +In the default flow, a node starts in blocksync, catches up, then switches to consensus. +Under sustained load (for example, busy RPC nodes), a node can remain behind and struggle to catch up. +With short block times, this can hurt network liveness. + +With `adaptive_sync` enabled, consensus still works normally, but it can also ingest already available +blocks from blocksync. This acts as a fallback path when a node is behind, allowing it to recover +more quickly during traffic spikes and continue progressing with the network. + +AdaptiveSync does not change consensus safety or finality rules. It changes catch-up behavior, not block validity rules. + +### Scope and compatibility + +- AdaptiveSync is intended for nodes that can temporarily lag, especially RPC-heavy or high-throughput deployments. +- It is experimental; enable it gradually and validate in your own network conditions before broad rollout. +- If you run mixed node roles (validators, sentries, RPC nodes), test each role separately. + +### How it works + +1. The node continues running consensus as usual. +2. If the node is behind, blocks obtained by blocksync can be handed to consensus ingestion. +3. If a candidate block is already included by consensus, it is skipped. +4. If not already included, it is ingested and applied through normal validation paths. +5. The node converges faster during transient load spikes while preserving normal consensus behavior. + +### When to use it + +Enable `adaptive_sync` if your nodes can temporarily fall behind and need better recovery behavior: + +- **High-throughput or bursty traffic** where load spikes can delay vote processing. +- **Short block times** where slow catch-up can impact liveness sooner. +- **RPC-heavy nodes** that may lag during periods of high request volume. + +If your network is stable and nodes consistently keep up, the default mode may already be sufficient. + +**Notes**: + +- Running this fallback path may slightly increase CPU and I/O during catch-up windows +- Constant blocksync message processing may increase network traffic. +- Gains are most visible during temporary overload. + +### Configuration + +AdaptiveSync is disabled by default. +To enable it, set `adaptive_sync = true` in the `[blocksync]` section of `config.toml`: + +```toml +[blocksync] +version = "v0" +adaptive_sync = true +``` + +### Metrics + +```text +# counter: blocksync block was already included by consensus --> skip +cometbft_blocksync_already_included_blocks + +# counter: blocksync block was ingested by consensus --> ingest +cometbft_blocksync_ingested_blocks + +# histogram: duration of ingesting a non-skipped block +cometbft_blocksync_ingested_block_duration_bucket +cometbft_blocksync_ingested_block_duration_count +cometbft_blocksync_ingested_block_duration_sum +``` diff --git a/cometbft/v0.39/docs/core/configuration.mdx b/cometbft/v0.39/docs/core/configuration.mdx new file mode 100644 index 000000000..0a96259fe --- /dev/null +++ b/cometbft/v0.39/docs/core/configuration.mdx @@ -0,0 +1,622 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/docs/core/configuration' +title: Configuration +order: 3 +--- + +CometBFT can be configured via a TOML file in +`$CMTHOME/config/config.toml`. Some of these parameters can be overridden by +command-line flags. For most users, the options in the `##### main base configuration options #####` section are intended to be modified, while config options +further below are intended for advanced power users. + +## Options + +The default configuration file created by `cometbft init` has all +the parameters set with their default values. It will look something +like the file below; however, double-check by inspecting the +`config.toml` created with your version of `cometbft` installed: + +```toml +# This is a TOML config file. +# For more information, see https://github.com/toml-lang/toml + +# NOTE: Any path below can be absolute (e.g. "/var/myawesomeapp/data") or +# relative to the home directory (e.g. "data"). The home directory is +# "$HOME/.cometbft" by default, but could be changed via $CMTHOME env variable +# or --home cmd flag. + +# The version of the CometBFT binary that created or +# last modified the config file. Do not modify this. +version = "0.38.0" + +####################################################################### +### Main Base Config Options ### +####################################################################### + +# TCP or UNIX socket address of the ABCI application, +# or the name of an ABCI application compiled in with the CometBFT binary +proxy_app = "tcp://127.0.0.1:26658" + +# A custom human readable name for this node +moniker = "anonymous" + +# Database backend: goleveldb | cleveldb | boltdb | rocksdb | badgerdb +# * goleveldb (github.com/syndtr/goleveldb) +# - UNMAINTAINED +# - stable +# - pure go +# - stable +# * cleveldb (uses levigo wrapper) +# - fast +# - requires gcc +# - use cleveldb build tag (go build -tags cleveldb) +# * boltdb (uses etcd's fork of bolt - github.com/etcd-io/bbolt) +# - EXPERIMENTAL +# - may be faster in some use-cases (random reads - indexer) +# - use boltdb build tag (go build -tags boltdb) +# * rocksdb (uses github.com/tecbot/gorocksdb) +# - EXPERIMENTAL +# - requires gcc +# - use rocksdb build tag (go build -tags rocksdb) +# * badgerdb (uses github.com/dgraph-io/badger) +# - EXPERIMENTAL +# - use badgerdb build tag (go build -tags badgerdb) +db_backend = "goleveldb" + +# Database directory +db_dir = "data" + +# Output level for logging, including package level options +log_level = "info" + +# Output format: 'plain' (colored text) or 'json' +log_format = "plain" + +##### additional base config options ##### + +# Path to the JSON file containing the initial validator set and other meta data +genesis_file = "config/genesis.json" + +# Path to the JSON file containing the private key to use as a validator in the consensus protocol +priv_validator_key_file = "config/priv_validator_key.json" + +# Path to the JSON file containing the last sign state of a validator +priv_validator_state_file = "data/priv_validator_state.json" + +# TCP or UNIX socket address for CometBFT to listen on for +# connections from an external PrivValidator process +priv_validator_laddr = "" + +# Path to the JSON file containing the private key to use for node authentication in the p2p protocol +node_key_file = "config/node_key.json" + +# Mechanism to connect to the ABCI application: socket | grpc +abci = "socket" + +# If true, query the ABCI app on connecting to a new peer +# so the app can decide if we should keep the connection or not +filter_peers = false + + +####################################################################### +### Advanced Configuration Options ### +####################################################################### + +####################################################### +### RPC Server Configuration Options ### +####################################################### +[rpc] + +# TCP or UNIX socket address for the RPC server to listen on +laddr = "tcp://127.0.0.1:26657" + +# A list of origins a cross-domain request can be executed from +# Default value '[]' disables cors support +# Use '["*"]' to allow any origin +cors_allowed_origins = [] + +# A list of methods the client is allowed to use with cross-domain requests +cors_allowed_methods = ["HEAD", "GET", "POST", ] + +# A list of non simple headers the client is allowed to use with cross-domain requests +cors_allowed_headers = ["Origin", "Accept", "Content-Type", "X-Requested-With", "X-Server-Time", ] + +# TCP or UNIX socket address for the gRPC server to listen on +# NOTE: This server only supports /broadcast_tx_commit +grpc_laddr = "" + +# Maximum number of simultaneous connections. +# Does not include RPC (HTTP&WebSocket) connections. See max_open_connections +# If you want to accept a larger number than the default, make sure +# you increase your OS limits. +# 0 - unlimited. +# Should be < {ulimit -Sn} - {MaxNumInboundPeers} - {MaxNumOutboundPeers} - {N of wal, db and other open files} +# 1024 - 40 - 10 - 50 = 924 = ~900 +grpc_max_open_connections = 900 + +# Activate unsafe RPC commands like /dial_seeds and /unsafe_flush_mempool +unsafe = false + +# Maximum number of simultaneous connections (including WebSocket). +# Does not include gRPC connections. See grpc_max_open_connections +# If you want to accept a larger number than the default, make sure +# you increase your OS limits. +# 0 - unlimited. +# Should be < {ulimit -Sn} - {MaxNumInboundPeers} - {MaxNumOutboundPeers} - {N of wal, db and other open files} +# 1024 - 40 - 10 - 50 = 924 = ~900 +max_open_connections = 900 + +# Maximum number of unique clientIDs that can /subscribe +# If you're using /broadcast_tx_commit, set to the estimated maximum number +# of broadcast_tx_commit calls per block. +max_subscription_clients = 100 + +# Maximum number of unique queries a given client can /subscribe to +# If you're using GRPC (or Local RPC client) and /broadcast_tx_commit, set to +# the estimated maximum number of broadcast_tx_commit calls per block. +max_subscriptions_per_client = 5 + +# Experimental parameter to specify the maximum number of events a node will +# buffer, per subscription, before returning an error and closing the +# subscription. Must be set to at least 100, but higher values will accommodate +# higher event throughput rates (and will use more memory). +experimental_subscription_buffer_size = 200 + +# Experimental parameter to specify the maximum number of RPC responses that +# can be buffered per WebSocket client. If clients cannot read from the +# WebSocket endpoint fast enough, they will be disconnected, so increasing this +# parameter may reduce the chances of them being disconnected (but will cause +# the node to use more memory). +# +# Must be at least the same as "experimental_subscription_buffer_size", +# otherwise connections could be dropped unnecessarily. This value should +# ideally be somewhat higher than "experimental_subscription_buffer_size" to +# accommodate non-subscription-related RPC responses. +experimental_websocket_write_buffer_size = 200 + +# If a WebSocket client cannot read fast enough, at present we may +# silently drop events instead of generating an error or disconnecting the +# client. +# +# Enabling this experimental parameter will cause the WebSocket connection to +# be closed instead if it cannot read fast enough, allowing for greater +# predictability in subscription behavior. +experimental_close_on_slow_client = false + +# How long to wait for a tx to be committed during /broadcast_tx_commit. +# WARNING: Using a value larger than 10s will result in increasing the +# global HTTP write timeout, which applies to all connections and endpoints. +# See https://github.com/tendermint/tendermint/issues/3435 +timeout_broadcast_tx_commit = "10s" + +# Maximum number of requests that can be sent in a JSON-RPC batch request. +# Possible values: number greater than 0. +# If the number of requests sent in a JSON-RPC batch exceed the maximum batch +# size configured, an error will be returned. +# The default value is set to `10`, which will limit the number of requests +# to 10 requests per JSON-RPC batch request. +# If you don't want to enforce a maximum number of requests for a batch +# request, set this value to `0`. +max_request_batch_size = 10 + +# Maximum size of request body, in bytes +max_body_bytes = 1000000 + +# Maximum size of request header, in bytes +max_header_bytes = 1048576 + +# The path to a file containing certificate that is used to create the HTTPS server. +# Might be either absolute path or path related to CometBFT's config directory. +# If the certificate is signed by a certificate authority, +# the certFile should be the concatenation of the server's certificate, any intermediates, +# and the CA's certificate. +# NOTE: both tls_cert_file and tls_key_file must be present for CometBFT to create HTTPS server. +# Otherwise, HTTP server is run. +tls_cert_file = "" + +# The path to a file containing matching private key that is used to create the HTTPS server. +# Might be either absolute path or path related to CometBFT's config directory. +# NOTE: both tls_cert_file and tls_key_file must be present for CometBFT to create HTTPS server. +# Otherwise, HTTP server is run. +tls_key_file = "" + +# pprof listen address (https://golang.org/pkg/net/http/pprof) +pprof_laddr = "" + +####################################################### +### P2P Configuration Options ### +####################################################### +[p2p] + +# Address to listen for incoming connections +laddr = "tcp://0.0.0.0:26656" + +# Address to advertise to peers for them to dial. If empty, will use the same +# port as the laddr, and will introspect on the listener to figure out the +# address. IP and port are required. Example: 159.89.10.97:26656 +external_address = "" + +# Comma separated list of seed nodes to connect to +seeds = "" + +# Comma separated list of nodes to keep persistent connections to +persistent_peers = "" + +# Path to address book +addr_book_file = "config/addrbook.json" + +# Set true for strict address routability rules +# Set false for private or local networks +addr_book_strict = true + +# Maximum number of inbound peers +max_num_inbound_peers = 40 + +# Maximum number of outbound peers to connect to, excluding persistent peers +max_num_outbound_peers = 10 + +# List of node IDs, to which a connection will be (re)established ignoring any existing limits +unconditional_peer_ids = "" + +# Maximum pause when redialing a persistent peer (if zero, exponential backoff is used) +persistent_peers_max_dial_period = "0s" + +# Time to wait before flushing messages out on the connection +flush_throttle_timeout = "100ms" + +# Maximum size of a message packet payload, in bytes +max_packet_msg_payload_size = 1024 + +# Rate at which packets can be sent, in bytes/second +send_rate = 5120000 + +# Rate at which packets can be received, in bytes/second +recv_rate = 5120000 + +# Set true to enable the peer-exchange reactor +pex = true + +# Seed mode, in which node constantly crawls the network and looks for +# peers. If another node asks it for addresses, it responds and disconnects. +# +# Does not work if the peer-exchange reactor is disabled. +seed_mode = false + +# Comma separated list of peer IDs to keep private (will not be gossiped to other peers) +private_peer_ids = "" + +# Toggle to disable guard against peers connecting from the same ip. +allow_duplicate_ip = false + +# Peer connection configuration. +handshake_timeout = "20s" +dial_timeout = "3s" + +####################################################### +### Mempool Configuration Option ### +####################################################### +[mempool] + +# The type of mempool for this node to use. +# +# Possible types: +# - "flood" : concurrent linked list mempool with flooding gossip protocol +# (default) +# - "nop" : nop-mempool (short for no operation; the ABCI app is responsible +# for storing, disseminating and proposing txs). "create_empty_blocks=false" is +# not supported. +type = "flood" + +# Recheck (default: true) defines whether CometBFT should recheck the +# validity for all remaining transaction in the mempool after a block. +# Since a block affects the application state, some transactions in the +# mempool may become invalid. If this does not apply to your application, +# you can disable rechecking. +recheck = true + +# Broadcast (default: true) defines whether the mempool should relay +# transactions to other peers. Setting this to false will stop the mempool +# from relaying transactions to other peers until they are included in a +# block. In other words, if Broadcast is disabled, only the peer you send +# the tx to will see it until it is included in a block. +broadcast = true + +# WalPath (default: "") configures the location of the Write Ahead Log +# (WAL) for the mempool. The WAL is disabled by default. To enable, set +# wal_dir to where you want the WAL to be written (e.g. +# "data/mempool.wal"). +wal_dir = "" + +# Maximum number of transactions in the mempool +size = 5000 + +# Limit the total size of all txs in the mempool. +# This only accounts for raw transactions (e.g. given 1MB transactions and +# max_txs_bytes=5MB, mempool will only accept 5 transactions). +max_txs_bytes = 1073741824 + +# Size of the cache (used to filter transactions we saw earlier) in transactions +cache_size = 10000 + +# Do not remove invalid transactions from the cache (default: false) +# Set to true if it's not possible for any invalid transaction to become valid +# again in the future. +keep-invalid-txs-in-cache = false + +# Maximum size of a single transaction. +# NOTE: the max size of a tx transmitted over the network is {max_tx_bytes}. +max_tx_bytes = 1048576 + +# Maximum size of a batch of transactions to send to a peer +# Including space needed by encoding (one varint per transaction). +# XXX: Unused due to https://github.com/tendermint/tendermint/issues/5796 +max_batch_bytes = 0 + +####################################################### +### State Sync Configuration Options ### +####################################################### +[statesync] +# State sync rapidly bootstraps a new node by discovering, fetching, and restoring a state machine +# snapshot from peers instead of fetching and replaying historical blocks. Requires some peers in +# the network to take and serve state machine snapshots. State sync is not attempted if the node +# has any local state (LastBlockHeight > 0). The node will have a truncated block history, +# starting from the height of the snapshot. +enable = false + +# RPC servers (comma-separated) for light client verification of the synced state machine and +# retrieval of state data for node bootstrapping. Also needs a trusted height and corresponding +# header hash obtained from a trusted source, and a period during which validators can be trusted. +# +# For Cosmos SDK-based chains, trust_period should usually be about 2/3 of the unbonding time (~2 +# weeks) during which they can be financially punished (slashed) for misbehavior. +rpc_servers = "" +trust_height = 0 +trust_hash = "" +trust_period = "168h0m0s" + +# Time to spend discovering snapshots before initiating a restore. +discovery_time = "15s" + +# Temporary directory for state sync snapshot chunks, defaults to the OS tempdir (typically /tmp). +# Will create a new, randomly named directory within, and remove it when done. +temp_dir = "" + +# The timeout duration before re-requesting a chunk, possibly from a different +# peer (default: 1 minute). +chunk_request_timeout = "10s" + +# The number of concurrent chunk fetchers to run (default: 1). +chunk_fetchers = "4" + +####################################################### +### Block Sync Configuration Options ### +####################################################### +[blocksync] + +# Block Sync version to use: +# +# In v0.37, v1 and v2 of the block sync protocols were deprecated. +# Please use v0 instead. +# +# 1) "v0" - the default block sync implementation +version = "v0" + +####################################################### +### Consensus Configuration Options ### +####################################################### +[consensus] + +wal_file = "data/cs.wal/wal" + +# How long we wait for a proposal block before prevoting nil +timeout_propose = "3s" +# How much timeout_propose increases with each round +timeout_propose_delta = "500ms" +# How long we wait after receiving +2/3 prevotes for "anything" (ie. not a single block or nil) +timeout_prevote = "1s" +# How much the timeout_prevote increases with each round +timeout_prevote_delta = "500ms" +# How long we wait after receiving +2/3 precommits for "anything" (ie. not a single block or nil) +timeout_precommit = "1s" +# How much the timeout_precommit increases with each round +timeout_precommit_delta = "500ms" +# How long we wait after committing a block, before starting on the new +# height (this gives us a chance to receive some more precommits, even +# though we already have +2/3). +timeout_commit = "1s" + +# How many blocks to look back to check existence of the node's consensus votes before joining consensus +# When non-zero, the node will panic upon restart +# if the same consensus key was used to sign {double_sign_check_height} last blocks. +# So, validators should stop the state machine, wait for some blocks, and then restart the state machine to avoid panic. +double_sign_check_height = 0 + +# Make progress as soon as we have all the precommits (as if TimeoutCommit = 0) +skip_timeout_commit = false + +# EmptyBlocks mode and possible interval between empty blocks +create_empty_blocks = true +create_empty_blocks_interval = "0s" + +# Reactor sleep duration parameters +peer_gossip_sleep_duration = "100ms" +peer_query_maj23_sleep_duration = "2s" + +####################################################### +### Storage Configuration Options ### +####################################################### +[storage] + +# Set to true to discard ABCI responses from the state store, which can save a +# considerable amount of disk space. Set to false to ensure ABCI responses are +# persisted. ABCI responses are required for /block_results RPC queries, and to +# reindex events in the command-line tool. +discard_abci_responses = false + +####################################################### +### Transaction Indexer Configuration Options ### +####################################################### +[tx_index] + +# What indexer to use for transactions +# +# The application will set which txs to index. In some cases a node operator will be able +# to decide which txs to index based on configuration set in the application. +# +# Options: +# 1) "null" +# 2) "kv" (default) - the simplest possible indexer, backed by key-value storage (defaults to levelDB; see DBBackend). +# - When "kv" is chosen "tx.height" and "tx.hash" will always be indexed. +# 3) "psql" - the indexer services backed by PostgreSQL. +# When "kv" or "psql" is chosen "tx.height" and "tx.hash" will always be indexed. +indexer = "kv" + +# The PostgreSQL connection configuration, the connection format: +# postgresql://:@:/? +psql-conn = "" + +####################################################### +### Instrumentation Configuration Options ### +####################################################### +[instrumentation] + +# When true, Prometheus metrics are served under /metrics on +# PrometheusListenAddr. +# Check out the documentation for the list of available metrics. +prometheus = false + +# Address to listen for Prometheus collector(s) connections +prometheus_listen_addr = ":26660" + +# Maximum number of simultaneous connections. +# If you want to accept a larger number than the default, make sure +# you increase your OS limits. +# 0 - unlimited. +max_open_connections = 3 + +# Instrumentation namespace +namespace = "cometbft" + + ``` + +## Empty blocks vs. no empty blocks + +### create_empty_blocks = true + +If `create_empty_blocks` is set to `true` in your config, blocks will be created approximately every second (with default consensus parameters). You can regulate the delay between blocks by changing `timeout_commit`. For example, `timeout_commit = "10s"` should result in approximately 10-second blocks. + +### create_empty_blocks = false + +In this setting, blocks are created when transactions are received. + +Note that after block H, CometBFT creates something we call a "proof block" (only if the application hash changed) H+1. The reason for this is to support proofs. If you have a transaction in block H that changes the state to X, the new application hash will only be included in block H+1. If after your transaction is committed, you want to get a light-client proof for the new state (X), you need the new block to be committed in order to do that because the new block has the new application hash for state X. That's why we create a new (empty) block if the application hash changes. Otherwise, you won't be able to make a proof for the new state. + +Additionally, if you set `create_empty_blocks_interval` to something other than the default (`0`), CometBFT will create empty blocks even in the absence of transactions every `create_empty_blocks_interval`. For instance, with `create_empty_blocks = false` and `create_empty_blocks_interval = "30s"`, CometBFT will only create blocks if there are transactions, or after waiting 30 seconds without receiving any transactions. + +## Consensus timeouts explained + +There's a variety of information about timeouts in [Running in +production](/cometbft/v0.39/docs/core/Running-in-production#configuration-parameters). +You can also find a more detailed explanation in the paper describing +the Tendermint consensus algorithm, adopted by CometBFT: [The latest +gossip on BFT consensus](https://arxiv.org/abs/1807.04938). + +```toml +[consensus] +... + +timeout_propose = "3s" +timeout_propose_delta = "500ms" +timeout_prevote = "1s" +timeout_prevote_delta = "500ms" +timeout_precommit = "1s" +timeout_precommit_delta = "500ms" +timeout_commit = "1s" +``` + +Note that in a successful round, the only timeout that we absolutely wait +no matter what is `timeout_commit`. + +Here's a brief summary of the timeouts: + +- `timeout_propose` = how long a validator should wait for a proposal block before prevoting nil +- `timeout_propose_delta` = how much `timeout_propose` increases with each round +- `timeout_prevote` = how long a validator should wait after receiving +2/3 prevotes for + anything (i.e., not a single block or nil) +- `timeout_prevote_delta` = how much `timeout_prevote` increases with each round +- `timeout_precommit` = how long a validator should wait after receiving +2/3 precommits for + anything (i.e., not a single block or nil) +- `timeout_precommit_delta` = how much `timeout_precommit` increases with each round +- `timeout_commit` = how long a validator should wait after committing a block before starting + on the new height (this gives us a chance to receive some more precommits, + even though we already have +2/3) + +### The adverse effect of using inconsistent `timeout_propose` in a network + +Here's an interesting question: What happens if a particular validator sets a +very small `timeout_propose`, as compared to the rest of the network? + +Imagine there are only two validators in your network: Alice and Bob. Bob sets +`timeout_propose` to 0s. Alice uses the default value of 3s. Let's say they +both have equal voting power. Given the proposer selection algorithm is a +weighted round-robin, you may expect Alice and Bob to take turns proposing +blocks, with a result like: + +``` +#1 block - Alice +#2 block - Bob +#3 block - Alice +#4 block - Bob +... +``` + +What happens in reality is, however, a little bit different: + +``` +#1 block - Bob +#2 block - Bob +#3 block - Bob +#4 block - Bob +``` + +That's because Bob doesn't wait for a proposal from Alice (prevotes `nil`). +This leaves Alice no chance to commit a block. Note that every block Bob +creates needs a vote from Alice to constitute 2/3+. Bob always gets one because +Alice has `timeout_propose` set to 3s. Alice never gets one because Bob has it +set to 0s. + +Imagine now there are ten geographically distributed validators. One of them +(Bob) sets `timeout_propose` to 0s. Others have it set to 3s. Now, Bob won't be +able to move at his own speed because it still needs 2/3 votes of the other +validators, and it takes time to propagate those. In other words, the network moves at +the speed of time to accumulate 2/3+ of votes (prevotes & precommits), not at +the speed of the fastest proposer. + +> Isn't block production determined by voting power? + +If it were determined solely by voting power, it wouldn't be possible to ensure +liveness. Timeouts exist because the network can't rely on a single proposer +being available and must move on if such proposer is not responding. + +> How can we address situations where someone arbitrarily adjusts their block +> production time to gain an advantage? + +The impact shown above is negligible in a decentralized network with sufficient +decentralization. + +### The adverse effect of using inconsistent `timeout_commit` in a network + +Let's look at the same scenario as before. There are ten geographically +distributed validators. One of them (Bob) sets `timeout_commit` to 0s. Others +have it set to 1s (the default value). Now, Bob will be the fastest producer +because he doesn't wait for additional precommits after creating a block. If +waiting for precommits (`timeout_commit`) is not incentivized, Bob will accrue +more rewards compared to the other 9 validators. + +This is because Bob has the advantage of broadcasting his proposal early (1 +second earlier than the others). But it also makes it possible for Bob to miss +a proposal from another validator and prevote `nil` due to him starting +`timeout_propose` earlier. In other words, if Bob's `timeout_commit` is too low compared +to other validators, then he might miss some proposals and get slashed for +inactivity. diff --git a/cometbft/v0.39/docs/core/how-to-read-logs.mdx b/cometbft/v0.39/docs/core/how-to-read-logs.mdx new file mode 100644 index 000000000..087cebb2f --- /dev/null +++ b/cometbft/v0.39/docs/core/how-to-read-logs.mdx @@ -0,0 +1,142 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/docs/core/how-to-read-logs' +title: How to read logs +order: 7 +--- + +## Walkthrough example + +We first create three connections (mempool, consensus, and query) to the +application (running `kvstore` locally in this case). + +```sh +I[10-04|13:54:27.364] Starting multiAppConn module=proxy impl=multiAppConn +I[10-04|13:54:27.366] Starting localClient module=abci-client connection=query impl=localClient +I[10-04|13:54:27.366] Starting localClient module=abci-client connection=mempool impl=localClient +I[10-04|13:54:27.367] Starting localClient module=abci-client connection=consensus impl=localClient +``` + +Then CometBFT and the application perform a handshake. + +```sh +I[10-04|13:54:27.367] ABCI Handshake module=consensus appHeight=90 appHash=E0FBAFBF6FCED8B9786DDFEB1A0D4FA2501BADAD +I[10-04|13:54:27.368] ABCI Replay Blocks module=consensus appHeight=90 storeHeight=90 stateHeight=90 +I[10-04|13:54:27.368] Completed ABCI Handshake - CometBFT and App are synced module=consensus appHeight=90 appHash=E0FBAFBF6FCED8B9786DDFEB1A0D4FA2501BADAD +``` + +After that, we start a few more things like the event switch and reactors. + +```sh +I[10-04|13:54:27.374] Starting EventSwitch module=types impl=EventSwitch +I[10-04|13:54:27.375] This node is a validator module=consensus +I[10-04|13:54:27.379] Starting Node module=main impl=Node +I[10-04|13:54:27.381] Local listener module=p2p ip=:: port=26656 +I[10-04|13:54:30.386] Starting DefaultListener module=p2p impl=Listener(@10.0.2.15:26656) +I[10-04|13:54:30.387] Starting P2P Switch module=p2p impl="P2P Switch" +I[10-04|13:54:30.387] Starting MempoolReactor module=mempool impl=MempoolReactor +I[10-04|13:54:30.387] Starting BlockchainReactor module=blockchain impl=BlockchainReactor +I[10-04|13:54:30.387] Starting ConsensusReactor module=consensus impl=ConsensusReactor +I[10-04|13:54:30.387] ConsensusReactor module=consensus fastSync=false +I[10-04|13:54:30.387] Starting ConsensusState module=consensus impl=ConsensusState +I[10-04|13:54:30.387] Starting WAL module=consensus wal=/home/vagrant/.cometbft/data/cs.wal/wal impl=WAL +I[10-04|13:54:30.388] Starting TimeoutTicker module=consensus impl=TimeoutTicker +``` + +Notice the second row where CometBFT reports that "This node is a +validator". It could also be just an observer (regular node). + +Next, we replay all the messages from the WAL. + +```sh +I[10-04|13:54:30.390] Catchup by replaying consensus messages module=consensus height=91 +I[10-04|13:54:30.390] Replay: New Step module=consensus height=91 round=0 step=RoundStepNewHeight +I[10-04|13:54:30.390] Replay: Done module=consensus +``` + +The "Started node" message signals that everything is ready for work. + +```sh +I[10-04|13:54:30.391] Starting RPC HTTP server on tcp socket 0.0.0.0:26657 module=rpc-server +I[10-04|13:54:30.392] Started node module=main nodeInfo="NodeInfo{id: DF22D7C92C91082324A1312F092AA1DA197FA598DBBFB6526E, moniker: anonymous, network: test-chain-3MNw2N [remote , listen 10.0.2.15:26656], version: 0.11.0-10f361fc ([wire_version=0.6.2 p2p_version=0.5.0 consensus_version=v1/0.2.2 rpc_version=0.7.0/3 tx_index=on rpc_addr=tcp://0.0.0.0:26657])}" +``` + +Next follows a standard block creation cycle, where we enter a new +round, propose a block, receive more than 2/3 of prevotes, then +precommits, and finally have a chance to commit a block. For details, +please refer to [Byzantine Consensus Algorithm](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/consensus/consensus.md). + +```sh +I[10-04|13:54:30.393] enterNewRound(91/0). Current: 91/0/RoundStepNewHeight module=consensus +I[10-04|13:54:30.393] enterPropose(91/0). Current: 91/0/RoundStepNewRound module=consensus +I[10-04|13:54:30.393] enterPropose: Our turn to propose module=consensus proposer=125B0E3C5512F5C2B0E1109E31885C4511570C42 privValidator="PrivValidator{125B0E3C5512F5C2B0E1109E31885C4511570C42 LH:90, LR:0, LS:3}" +I[10-04|13:54:30.394] Signed proposal module=consensus height=91 round=0 proposal="Proposal{91/0 1:21B79872514F (-1,:0:000000000000) {/10EDEDD7C84E.../}}" +I[10-04|13:54:30.397] Received complete proposal block module=consensus height=91 hash=F671D562C7B9242900A286E1882EE64E5556FE9E +I[10-04|13:54:30.397] enterPrevote(91/0). Current: 91/0/RoundStepPropose module=consensus +I[10-04|13:54:30.397] enterPrevote: ProposalBlock is valid module=consensus height=91 round=0 +I[10-04|13:54:30.398] Signed and pushed vote module=consensus height=91 round=0 vote="Vote{0:125B0E3C5512 91/00/1(Prevote) F671D562C7B9 {/89047FFC21D8.../}}" err=null +I[10-04|13:54:30.401] Added to prevote module=consensus vote="Vote{0:125B0E3C5512 91/00/1(Prevote) F671D562C7B9 {/89047FFC21D8.../}}" prevotes="VoteSet{H:91 R:0 T:1 +2/3:F671D562C7B9242900A286E1882EE64E5556FE9E:1:21B79872514F BA{1:X} map[]}" +I[10-04|13:54:30.401] enterPrecommit(91/0). Current: 91/0/RoundStepPrevote module=consensus +I[10-04|13:54:30.401] enterPrecommit: +2/3 prevoted proposal block. Locking module=consensus hash=F671D562C7B9242900A286E1882EE64E5556FE9E +I[10-04|13:54:30.402] Signed and pushed vote module=consensus height=91 round=0 vote="Vote{0:125B0E3C5512 91/00/2(Precommit) F671D562C7B9 {/80533478E41A.../}}" err=null +I[10-04|13:54:30.404] Added to precommit module=consensus vote="Vote{0:125B0E3C5512 91/00/2(Precommit) F671D562C7B9 {/80533478E41A.../}}" precommits="VoteSet{H:91 R:0 T:2 +2/3:F671D562C7B9242900A286E1882EE64E5556FE9E:1:21B79872514F BA{1:X} map[]}" +I[10-04|13:54:30.404] enterCommit(91/0). Current: 91/0/RoundStepPrecommit module=consensus +I[10-04|13:54:30.405] Finalizing commit of block with 0 txs module=consensus height=91 hash=F671D562C7B9242900A286E1882EE64E5556FE9E root=E0FBAFBF6FCED8B9786DDFEB1A0D4FA2501BADAD +I[10-04|13:54:30.405] Block{ + Header{ + ChainID: test-chain-3MNw2N + Height: 91 + Time: 2017-10-04 13:54:30.393 +0000 UTC + NumTxs: 0 + LastBlockID: F15AB8BEF9A6AAB07E457A6E16BC410546AA4DC6:1:D505DA273544 + LastCommit: 56FEF2EFDB8B37E9C6E6D635749DF3169D5F005D + Data: + Validators: CE25FBFF2E10C0D51AA1A07C064A96931BC8B297 + App: E0FBAFBF6FCED8B9786DDFEB1A0D4FA2501BADAD + }#F671D562C7B9242900A286E1882EE64E5556FE9E + Data{ + + }# + Commit{ + BlockID: F15AB8BEF9A6AAB07E457A6E16BC410546AA4DC6:1:D505DA273544 + Precommits: Vote{0:125B0E3C5512 90/00/2(Precommit) F15AB8BEF9A6 {/FE98E2B956F0.../}} + }#56FEF2EFDB8B37E9C6E6D635749DF3169D5F005D +}#F671D562C7B9242900A286E1882EE64E5556FE9E module=consensus +I[10-04|13:54:30.408] Executed block module=state height=91 validTxs=0 invalidTxs=0 +I[10-04|13:54:30.410] Committed state module=state height=91 txs=0 hash=E0FBAFBF6FCED8B9786DDFEB1A0D4FA2501BADAD +I[10-04|13:54:30.410] Recheck txs module=mempool numtxs=0 height=91 +``` + +## List of modules + +Here is the list of modules you may encounter in CometBFT's logs and a +brief overview of what they do. + +- `abci-client` As mentioned in [Application Development Guide](/cometbft/v0.39/docs/app-dev/Using-ABCI-CLI), CometBFT acts as an ABCI + client with respect to the application and maintains 3 connections: + mempool, consensus, and query. The code used by CometBFT can + be found [here](https://github.com/cometbft/cometbft/blob/v0.38.x/abci/client). +- `blockchain` Provides storage, pool (a group of peers), and reactor + for both storing and exchanging blocks between peers. +- `consensus` The heart of CometBFT, which is the + implementation of the consensus algorithm. Includes two + "submodules": `wal` (write-ahead logging) for ensuring data + integrity and `replay` to replay blocks and messages on recovery + from a crash. +- `events` Simple event notification system. The list of events can be + found + [here](https://github.com/cometbft/cometbft/blob/v0.38.x/types/events.go). + You can subscribe to them by calling `subscribe` RPC method. Refer + to [RPC docs](/cometbft/v0.39/api-reference/rpc/index) for additional information. +- `mempool` Mempool module handles all incoming transactions, whenever + they are coming from peers or the application. +- `p2p` Provides an abstraction around peer-to-peer communication. For + more details, please check out the + [README](https://github.com/cometbft/cometbft/blob/v0.38.x/p2p/README.md). +- `rpc` [CometBFT's RPC](/cometbft/v0.39/api-reference/rpc/index). +- `rpc-server` RPC server. For implementation details, please read the + [doc.go](https://github.com/cometbft/cometbft/blob/v0.38.x/rpc/jsonrpc/doc.go). +- `state` Represents the latest state and execution submodule, which + executes blocks against the application. +- `types` A collection of the publicly exposed types and methods to + work with them. diff --git a/cometbft/v0.39/docs/core/light-client.mdx b/cometbft/v0.39/docs/core/light-client.mdx new file mode 100644 index 000000000..32d8b4785 --- /dev/null +++ b/cometbft/v0.39/docs/core/light-client.mdx @@ -0,0 +1,70 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/docs/core/light-client' +title: Light Client +order: 13 +--- + +Light clients are an important part of the complete blockchain system for most +applications. CometBFT provides unique speed and security properties for +light client applications. + +See our [light +package](https://pkg.go.dev/github.com/cometbft/cometbft/light?tab=doc). + +## Overview + +The objective of the light client protocol is to get a commit for a recent +block hash where the commit includes a majority of signatures from the last +known validator set. From there, all the application state is verifiable with +[Merkle proofs](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/core/encoding.md#iavl-tree). + +## Properties + +- You get the full collateralized security benefits of CometBFT; no + need to wait for confirmations. +- You get the full speed benefits of CometBFT; transactions + commit instantly. +- You can get the most recent version of the application state + non-interactively (without committing anything to the blockchain). For + example, this means that you can get the most recent value of a name from the + name registry without worrying about fork censorship attacks, without posting + a commit and waiting for confirmations. It's fast, secure, and free! + +## Where to obtain trusted height & hash + +[Trust Options](https://pkg.go.dev/github.com/cometbft/cometbft/light?tab=doc#TrustOptions) + +One way to obtain a semi-trusted hash & height is to query multiple full nodes +and compare their hashes: + +```bash +$ curl -s https://233.123.0.140:26657:26657/commit | jq "{height: .result.signed_header.header.height, hash: .result.signed_header.commit.block_id.hash}" +{ + "height": "273", + "hash": "188F4F36CBCD2C91B57509BBF231C777E79B52EE3E0D90D06B1A25EB16E6E23D" +} +``` + +## Running a light client as an HTTP proxy server + +CometBFT comes with a built-in `cometbft light` command, which can be used +to run a light client proxy server, verifying CometBFT RPC. All calls that +can be tracked back to a block header by a proof will be verified before +passing them back to the caller. Other than that, it will present the same +interface as a full CometBFT node. + +You can start the light client proxy server by running `cometbft light `, +with a variety of flags to specify the primary node, the witness nodes (which cross-check +the information provided by the primary), the hash and height of the trusted header, +and more. + +For example: + +```bash +$ cometbft light supernova -p tcp://233.123.0.140:26657 \ + -w tcp://179.63.29.15:26657,tcp://144.165.223.135:26657 \ + --height=10 --hash=37E9A6DD3FA25E83B22C18835401E8E56088D0D7ABC6FD99FCDC920DD76C1C57 +``` + +For additional options, run `cometbft light --help`. diff --git a/cometbft/v0.39/docs/core/mempool.mdx b/cometbft/v0.39/docs/core/mempool.mdx new file mode 100644 index 000000000..e728995fe --- /dev/null +++ b/cometbft/v0.39/docs/core/mempool.mdx @@ -0,0 +1,445 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/docs/core/mempool' +title: Mempool +order: 12 +--- + +A mempool (a contraction of memory and pool) is a node's data structure for +storing information on uncommitted transactions. It acts as a sort of waiting +room for transactions that have not yet been committed. + +CometBFT currently supports three types of mempools: [`flood`](#1-flood), [`nop`](#2-nop), and [`app`](#3-app). + +## 1. Flood + +The `flood` mempool stores transactions in a concurrent linked list. When a new +transaction is received, it first checks if there's space for it (`size` and +`max_txs_bytes` config options) and that it's not too big (`max_tx_bytes` config +option). Then, it checks if this transaction has already been seen before by using +an LRU cache (`cache_size` regulates the cache's size). If all checks pass and +the transaction is not in the cache (meaning it's new), the ABCI +[`CheckTxAsync`][1] method is called. The ABCI application validates the +transaction using its own rules. + +If the transaction is deemed valid by the ABCI application, it's added to the linked list. + +The mempool's name (`flood`) comes from the dissemination mechanism. When a new +transaction is added to the linked list, the mempool sends it to all connected +peers. Peers themselves gossip this transaction to their peers and so on. One +can say that each transaction "floods" the network, hence the name `flood`. + +Note there are experimental config options +`experimental_max_gossip_connections_to_persistent_peers` and +`experimental_max_gossip_connections_to_non_persistent_peers` to limit the +number of peers a transaction is broadcast to. Also, you can turn off +broadcasting with the `broadcast` config option. + +After each committed block, CometBFT rechecks all uncommitted transactions (can +be disabled with the `recheck` config option) by repeatedly calling the ABCI +`CheckTxAsync`. + +### Transaction ordering + +Currently, there's no ordering of transactions other than the order they've +arrived (via RPC or from other nodes). + +So the only way to specify the order is to send them to a single node. + +valA: + +- `tx1` +- `tx2` +- `tx3` + +If the transactions are split up across different nodes, there's no way to +ensure they are processed in the expected order. + +valA: + +- `tx1` +- `tx2` + +valB: + +- `tx3` + +If valB is the proposer, the order might be: + +- `tx3` +- `tx1` +- `tx2` + +If valA is the proposer, the order might be: + +- `tx1` +- `tx2` +- `tx3` + +That said, if the transactions contain some internal value, like an +order/nonce/sequence number, the application can reject transactions that are +out of order. So if a node receives `tx3`, then `tx1`, it can reject `tx3` and then +accept `tx1`. The sender can then retry sending `tx3`, which should probably be +rejected until the node has seen `tx2`. + +## 2. Nop + +The `nop` (short for no operation) mempool is used when the ABCI application developer wants to +build their own mempool. When `type = "nop"`, transactions are not stored anywhere +and are not gossiped to other peers using the P2P network. + +Submitting a transaction via the existing RPC methods (`BroadcastTxSync`, +`BroadcastTxAsync`, and `BroadcastTxCommit`) will always result in an error. + +Because there's no way for the consensus to know if transactions are available +to be committed, the node will always create blocks, which can be empty +sometimes. Using `consensus.create_empty_blocks=false` is prohibited in such +cases. + +The ABCI application becomes responsible for storing, disseminating, and +proposing transactions using [`PrepareProposal`][2]. The concrete design is up +to the ABCI application developers. + +[1]: /cometbft/v0.39/spec/abci/Methods#checktx +[2]: /cometbft/v0.39/spec/abci/Methods#prepareproposal + +## 3. App + + +The CometBFT `app` mempool is distinct from the [Cosmos SDK's application mempool](/sdk/latest/guides/abci/app-mempool). The SDK's application mempool controls transaction ordering at block proposal time. The CometBFT `app` mempool delegates the entire transaction lifecycle (storage, gossip, and rechecking) from CometBFT to the application. The CometBFT `app` mempool is currently implemented in [Cosmos EVM](/evm/latest/documentation/concepts/mempool). + + +The `app` mempool (also known as the Krakatoa mempool) is used when the ABCI application wants a middle ground +between the `flood` and `nop` mempool. + +The `app` mempool delegates transaction storage, validation, and rechecking +entirely to the ABCI application. CometBFT acts as a thin proxy — receiving +transactions from RPC and P2P, forwarding them to the application via ABCI, and +broadcasting application reaped transactions to peers. + +### Motivation + +The traditional flood mempool architecture has several limitations: + +**ABCI lock contention**: In the `flood` mempool, `CheckTx` calls hold the ABCI +connection lock. This lock is shared with consensus-critical operations like +`PrepareProposal` and `FinalizeBlock`. Since `CheckTx` volume is directly +proportional to network load and fully driven by external actors submitting +transactions, this means an externally influenced workload can hold up block +building and finalization. During rechecking after a committed block, the +problem compounds — all incoming transactions and consensus operations must +wait for the full recheck pass to complete. + +**Limited application control**: The application has no control over when +rechecking occurs, how transactions are prioritized during recheck, or how the +mempool interacts with block building. CometBFT drives the entire lifecycle. + +**Redundant state management**: CometBFT maintains its own transaction storage +(the concurrent linked list) even though the application often needs its own +mempool for ordering and prioritization. This leads to duplicated state and +synchronization overhead. + +The `app` mempool eliminates these issues by making the application the single +source of truth for mempool state. CometBFT no longer holds the ABCI lock for +mempool operations: `InsertTx` and `ReapTxs` are called concurrently and the +application is responsible for its own synchronization. + +### Quick Start + +To enable the Krakatoa `app` mempool, set the mempool type in your CometBFT +`config.toml`: + +```toml +[mempool] +type = "app" +``` + +This switches CometBFT from the default `flood` mempool to the application +delegated model. CometBFT will forward transactions to your application via +`InsertTx` and pull validated transactions back via `ReapTxs`, rather than +managing mempool state itself. + +Your application must implement the `InsertTx` and `ReapTxs` ABCI handlers. + +### New ABCI Methods + +Two new methods are added to the ABCI `Application` interface as part of the +mempool connection: + +#### InsertTx + +```protobuf +service ABCIApplication { + rpc InsertTx(RequestInsertTx) returns (ResponseInsertTx); +} + +message RequestInsertTx { + bytes tx = 1; +} + +message ResponseInsertTx { + uint32 code = 1; +} +``` + +`InsertTx` is called when CometBFT receives a transaction, either from an RPC +client (`BroadcastTxSync`, `BroadcastTxAsync`) or from a peer via P2P gossip. +The application is expected to validate and store the transaction in its own +mempool. + +**Response codes**: + +| Code | Meaning | CometBFT Behavior | +|------|---------|-------------------| +| `0` (OK) | Transaction accepted | Transaction is marked as seen and will not be re-inserted | +| `1` - `31,999` | Transaction rejected | Transaction is marked as seen and will not be retried | +| `>= 32,000` (Retry) | Temporary rejection | Transaction is removed from the seen cache so it can be retried later | + +The retry mechanism is useful when the application's mempool is temporarily at +capacity. By returning a retry code, the application signals that the +transaction is not inherently invalid — it simply cannot be accepted right now. +When the transaction is received again (from a peer or resubmitted via RPC), it +will be forwarded to the application again. + +**Concurrency guarantee**: `InsertTx` calls are thread-safe from CometBFT's +perspective. Multiple goroutines may call `InsertTx` concurrently (e.g., +transactions arriving from different peers simultaneously). The application is +responsible for its own internal synchronization. + +**No ABCI lock**: Unlike `CheckTx` in the `flood` mempool, `InsertTx` does not +hold the ABCI connection lock. This means `InsertTx` calls do not block +consensus operations, and consensus operations do not block `InsertTx`. + +#### ReapTxs + +```protobuf +service ABCIApplication { + rpc ReapTxs(RequestReapTxs) returns (ResponseReapTxs); +} + +message RequestReapTxs { + uint64 max_bytes = 1; + uint64 max_gas = 2; +} + +message ResponseReapTxs { + repeated bytes txs = 1; +} +``` + +`ReapTxs` is called periodically by the `AppReactor` to retrieve new, validated +transactions from the application for p2p broadcast. The application should +return transactions that are ready for gossip — typically transactions that +have been validated and are eligible for block inclusion. + +When `max_bytes` and `max_gas` are both zero, the application should return all +available transactions without limits. + +### AppMempool + +The `AppMempool` is the CometBFT side implementation that fulfills the +`Mempool` interface while delegating all real work to the application. + +#### What AppMempool does + +- Proxies incoming transactions to the application via `InsertTx` +- Maintains a seen cache (LRU, 100k entries) to avoid re-inserting duplicate + transactions +- Validates transaction size against `max_tx_bytes` before forwarding +- Handles retry semantics by removing retryable transactions from the seen cache + +#### What AppMempool does NOT do + +- Store transactions — the application owns all mempool state +- Call `Update` after blocks — rechecking is the application's responsibility +- Provide transactions for `ReapMaxBytesMaxGas` — always returns nil, since the + application builds blocks via `PrepareProposal` + +### AppReactor + +The `AppReactor` replaces the traditional mempool `Reactor` for P2P +transaction gossip. + +#### Broadcasting + +The reactor runs a background loop that: + +1. Calls `ReapTxs` on the application every `reap_interval` duration (default 500ms). +2. Chunks the returned transactions into batches (up to `MaxBatchBytes`) +3. Broadcasts each batch to all connected peers + +#### Receiving + +When a peer sends transactions, the reactor: + +1. Deserializes the transaction batch from the P2P envelope +2. Calls `InsertTx` on the `AppMempool` for each transaction +3. Logs and discards transactions that fail insertion (already seen, too large, + or rejected by the application) + +#### Supporting `BroadcastTx...` methods + +In an effort to support existing chains and CometBFT tx broadcast RPC methods, +compatibility with `BroadcastTxSync`, `BroadcastTxAsync`, and +`BroadcastTxCommit` has been maintained when using the `app` mempool. + +Transactions ingested via these RPC call `CheckTx` in the hot path without the +ABCI connection lock. If the ABCI application would like to support these +methods, they must wire a `CheckTxHandler` into their application and manage +the locking relative to other ABCI state changes themselves. + +It is highly recommended for applications to not use these methods when using +an `app` mempool. Applications should implement application side RPC methods +for tx ingestion and insert these txs directly into their `app` mempool +implementation (or other comparable data structure), only relying on CometBFT +to inform them of transactions that are received over the p2p network. + +### Transaction Lifecycle + +With the `app` mempool, the transaction lifecycle changes significantly: + +#### Previous Lifecycle (flood mempool) + +1. Transaction arrives via RPC or P2P +2. CometBFT validates size and checks the seen cache +3. CometBFT calls `CheckTx` on the application (holds ABCI lock) +4. If valid, CometBFT stores the transaction in its linked list +5. CometBFT broadcasts the transaction to all peers +6. At block proposal time, CometBFT calls `ReapMaxBytesMaxGas` and passes + transactions to `PrepareProposal` +7. After block commit, CometBFT rechecks all remaining transactions via + `CheckTx` (holds ABCI lock for the entire recheck) + +#### Updated P2P Lifecycle (app mempool) + +1. Transaction arrives via P2P +2. CometBFT validates size and checks the seen cache +3. CometBFT calls `InsertTx` on the application (no ABCI lock) +4. The application validates and stores the transaction in its own mempool +5. The `AppReactor` periodically calls `ReapTxs` and broadcasts returned + transactions to peers +6. At block proposal time, `PrepareProposal` receives no transactions from + CometBFT — the application builds the block from its own mempool +7. After block commit, the application runs its own recheck logic on its own + schedule + +#### Updated application RPC Lifecycle (app mempool) + +1. Transaction arrives to application via application side RPC +2. Application validates and inserts the tx into its `app` mempool + implementation. Note that validating can happen after insert, it's up to the + application! +3. The application provides the tx to CometBFT once validated by returning its + bytes via the `ReapTxs` ABCI method. +4. CometBFT gossips the validated transaction to peers. +5. At block proposal time, `PrepareProposal` receives no transactions from + CometBFT: the application builds the block from its own mempool +6. After block commit, the application runs its own recheck logic on its own + schedule + +#### Updated (`broadcast_tx_...`) RPC Lifecycle (app mempool) + +1. Transaction arrives via RPC +2. CometBFT validates size and checks the seen cache +3. CometBFT calls `CheckTx` on the application (no ABCI lock, it is up to the + application to perform any necessary locking here). `CheckTx` is used here to + maintain API compatibility with existing clients. Applications implementing + their own application side mempool should strongly consider implementing their + own application side RPC methods to directly handle transaction ingestion, + rather than relying on CometBFT's. +4. The application validates and stores the transaction in its own mempool +5. The `AppReactor` periodically calls `ReapTxs` and broadcasts returned + transactions to peers +6. At block proposal time, `PrepareProposal` receives no transactions from + CometBFT — the application builds the block from its own mempool +7. After block commit, the application runs its own recheck logic on its own + schedule + +### Block Building + +Since the `AppMempool` returns nil from `ReapMaxBytesMaxGas`, the block +executor passes no mempool transactions to `PrepareProposal`. The application's +`PrepareProposalHandler` is expected to select transactions directly from its +own mempool. + +This gives the application full control over transaction ordering, +prioritization, and inclusion. + +### Application Guarantees and Responsibilities + +When implementing `InsertTx` and `ReapTxs`, applications should be aware of the +following: + +#### CometBFT guarantees to the application + +- `InsertTx` will not be called with empty transactions +- `InsertTx` will not be called with transactions exceeding `max_tx_bytes` +- Transactions returning a retry code will be removed from the seen cache and + may be re-submitted +- `ReapTxs` will be called periodically (every 500ms) regardless of whether new + transactions have arrived +- `InsertTx` and `ReapTxs` will not hold the ABCI connection lock + +#### Application responsibilities + +- **Concurrency**: The application must handle concurrent `InsertTx` calls safely +- **Rechecking**: The application must (optionally) implement its own + transaction revalidation after blocks are committed +- **Block building**: The application must select transactions for blocks in its + `PrepareProposalHandler` — CometBFT will not provide mempool transactions +- **Storage**: The application must manage its own transaction storage and eviction + +### Configuration + +To enable the `app` mempool, set the mempool type in `config.toml`: + +```toml +[mempool] +type = "app" +``` + +The following options apply to the `app` mempool: + +#### Shared options + +These options are shared with other mempool types: + +| Option | Description | Default | +|--------|-------------|---------| +| `max_tx_bytes` | Maximum size of a single transaction (checked before `InsertTx`) | `1048576` (1 MB) | +| `max_batch_bytes` | Maximum size of a broadcast batch | `0` (no limit) | +| `broadcast` | Enable or disable P2P transaction broadcasting | `true` | + +#### App mempool options + +These options only apply when `type = "app"`: + +| Option | Description | Default | +|--------|-------------|---------| +| `seen_cache_size` | Size of the LRU cache for deduplicating seen transactions. Prevents re-inserting transactions already forwarded to the application. | `100000` | +| `check_tx_retry_delay` | Delay after which a tx is removed from the seen cache after forwarding to the application via `CheckTx`. If a non-retryable error code is returned, the full delay is used before removing from the cache (allowing a retry). If a retryable error code is returned, 1/10th of the delay is used. | `"500ms"` | +| `reap_max_bytes` | Informs the application the maximum amount of bytes it should return from each call to `ReapTxs`. `0` means no limit. | `0` | +| `reap_max_gas` | Informs the application the maximum amount of gas it should return from each call to `ReapTxs`. `0` means no limit. | `0` | +| `reap_interval` | Interval between `ReapTxs` calls. | `"500ms"` | + +#### Example + +```toml +[mempool] +type = "app" +max_tx_bytes = 1048576 + +# App mempool options +seen_cache_size = 100000 +reap_max_bytes = 0 +reap_max_gas = 0 +reap_interval = "500ms" +check_tx_retry_delay = "500ms" +``` + +Options specific to the `flood` mempool (`size`, `max_txs_bytes`, `cache_size`, +`recheck`, etc.) have no effect when using the `app` mempool. + +### Related Documentation +- [Cosmos EVM Krakatoa Mempool](/evm/latest/documentation/concepts/mempool) - Cosmos EVM's mempool is an implementation of an `app` mempool +- [ABCI Methods](/cometbft/v0.39/spec/abci/Methods) - ABCI method specification diff --git a/cometbft/v0.39/docs/core/metrics.mdx b/cometbft/v0.39/docs/core/metrics.mdx new file mode 100644 index 000000000..d3ad146d5 --- /dev/null +++ b/cometbft/v0.39/docs/core/metrics.mdx @@ -0,0 +1,76 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/docs/core/metrics' +title: Metrics +order: 5 +--- + +CometBFT can report and serve Prometheus metrics, which in turn can +be consumed by Prometheus collector(s). + +This functionality is disabled by default. + +To enable Prometheus metrics, set `instrumentation.prometheus=true` in your +config file. Metrics will be served under `/metrics` on port 26660 by default. +The listen address can be changed in the config file (see +`instrumentation.prometheus\_listen\_addr`). + +## List of available metrics + +The following metrics are available: + +| **Name** | **Type** | **Tags** | **Description** | +|--------------------------------------------|-----------|------------------|--------------------------------------------------------------------------------------------------------------------------------------------| +| abci\_connection\_method\_timing\_seconds | Histogram | method, type | Timings for each of the ABCI methods | +| blocksync\_syncing | Gauge | | Either 0 (not block syncing) or 1 (syncing) | +| consensus\_height | Gauge | | Height of the chain | +| consensus\_validators | Gauge | | Number of validators | +| consensus\_validators\_power | Gauge | | Total voting power of all validators | +| consensus\_validator\_power | Gauge | | Voting power of the node if in the validator set | +| consensus\_validator\_last\_signed\_height | Gauge | | Last height the node signed a block, if the node is a validator | +| consensus\_validator\_missed\_blocks | Gauge | | Total number of blocks missed for the node, if the node is a validator | +| consensus\_missing\_validators | Gauge | | Number of validators who did not sign | +| consensus\_missing\_validators\_power | Gauge | | Total voting power of the missing validators | +| consensus\_byzantine\_validators | Gauge | | Number of validators who tried to double sign | +| consensus\_byzantine\_validators\_power | Gauge | | Total voting power of the byzantine validators | +| consensus\_block\_interval\_seconds | Histogram | | Time between this and the last block (Block.Header.Time) in seconds | +| consensus\_rounds | Gauge | | Number of rounds | +| consensus\_num\_txs | Gauge | | Number of transactions | +| consensus\_total\_txs | Gauge | | Total number of transactions committed | +| consensus\_block\_parts | Counter | peer\_id | Number of block parts transmitted by peer | +| consensus\_latest\_block\_height | Gauge | | /status sync\_info number | +| consensus\_block\_size\_bytes | Gauge | | Block size in bytes | +| consensus\_step\_duration | Histogram | step | Histogram of durations for each step in the consensus protocol | +| consensus\_round\_duration | Histogram | | Histogram of durations for all the rounds that have occurred since the process started | +| consensus\_block\_gossip\_parts\_received | Counter | matches\_current | Number of block parts received by the node | +| consensus\_quorum\_prevote\_delay | Gauge | | Interval in seconds between the proposal timestamp and the timestamp of the earliest prevote that achieved a quorum | +| consensus\_full\_prevote\_delay | Gauge | | Interval in seconds between the proposal timestamp and the timestamp of the latest prevote in a round where all validators voted | +| consensus\_vote\_extension\_receive\_count | Counter | status | Number of vote extensions received | +| consensus\_proposal\_receive\_count | Counter | status | Total number of proposals received by the node since process start | +| consensus\_proposal\_create\_count | Counter | | Total number of proposals created by the node since process start | +| consensus\_round\_voting\_power\_percent | Gauge | vote\_type | A value between 0 and 1.0 representing the percentage of the total voting power per vote type received within a round | +| consensus\_late\_votes | Counter | vote\_type | Number of votes received by the node since process start that correspond to earlier heights and rounds than this node is currently in | +| p2p\_message\_send\_bytes\_total | Counter | message\_type | Number of bytes sent to all peers per message type | +| p2p\_message\_receive\_bytes\_total | Counter | message\_type | Number of bytes received from all peers per message type | +| p2p\_peers | Gauge | | Number of peers the node is connected to | +| p2p\_peer\_receive\_bytes\_total | Counter | peer\_id, chID | Number of bytes per channel received from a given peer | +| p2p\_peer\_send\_bytes\_total | Counter | peer\_id, chID | Number of bytes per channel sent to a given peer | +| p2p\_peer\_pending\_send\_bytes | Gauge | peer\_id | Number of pending bytes to be sent to a given peer | +| p2p\_num\_txs | Gauge | peer\_id | Number of transactions submitted by each peer\_id | +| p2p\_pending\_send\_bytes | Gauge | peer\_id | Amount of data pending to be sent to peer | +| mempool\_size | Gauge | | Number of uncommitted transactions | +| mempool\_tx\_size\_bytes | Histogram | | Transaction sizes in bytes | +| mempool\_failed\_txs | Counter | | Number of failed transactions | +| mempool\_recheck\_times | Counter | | Number of transactions rechecked in the mempool | +| state\_block\_processing\_time | Histogram | | Time spent processing FinalizeBlock in ms | +| state\_consensus\_param\_updates | Counter | | Number of consensus parameter updates returned by the application since process start | +| state\_validator\_set\_updates | Counter | | Number of validator set updates returned by the application since process start | +| statesync\_syncing | Gauge | | Either 0 (not state syncing) or 1 (syncing) | + +## Useful queries + +Percentage of missing + byzantine validators: + +```md +((consensus\_byzantine\_validators\_power + consensus\_missing\_validators\_power) / consensus\_validators\_power) * 100 +``` diff --git a/cometbft/v0.39/docs/core/state-sync.mdx b/cometbft/v0.39/docs/core/state-sync.mdx new file mode 100644 index 000000000..967391f9e --- /dev/null +++ b/cometbft/v0.39/docs/core/state-sync.mdx @@ -0,0 +1,51 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/docs/core/state-sync' +title: State Sync +order: 11 +--- + +With block sync, a node downloads all of the application's data from genesis and verifies it. +With state sync, your node will download data related to the head or near the head of the chain and verify the data. +This leads to drastically shorter times for joining a network. + +## Using State Sync + +State sync will continuously work in the background to supply nodes with chunked data when bootstrapping. + +> NOTE: Before trying to use state sync, see if the application you are operating a node for supports it. + +Under the state sync section in `config.toml`, you will find multiple settings that need to be configured in order for your node to use state sync. + +Let's break down the settings: + +- `enable`: Enable is to inform the node that you will be using state sync to bootstrap your node. +- `rpc_servers`: RPC servers are needed because state sync utilizes the light client for verification. + - 2 servers are required, more is always helpful. +- `temp_dir`: Temporary directory to store the chunks in the machine's local storage. If nothing is set, it will create a directory in `/tmp`. + +The next information you will need to acquire through publicly exposed RPCs or a block explorer which you trust. + +- `trust_height`: Trusted height defines at which height your node should trust the chain. +- `trust_hash`: Trusted hash is the hash in the `BlockID` corresponding to the trusted height. +- `trust_period`: Trust period is the period in which headers can be verified. + > :warning: This value should be significantly smaller than the unbonding period. + +If you are relying on publicly exposed RPCs to get the needed information, you can use `curl` and [`jq`][jq]. + +Example: + +```bash +curl -s https://233.123.0.140:26657/commit | jq "{height: .result.signed_header.header.height, hash: .result.signed_header.commit.block_id.hash}" +``` + +The response will be: + +```json +{ + "height": "273", + "hash": "188F4F36CBCD2C91B57509BBF231C777E79B52EE3E0D90D06B1A25EB16E6E23D" +} +``` + +[jq]: https://jqlang.github.io/jq/ diff --git a/cometbft/v0.39/docs/experimental/lib-p2p.mdx b/cometbft/v0.39/docs/experimental/lib-p2p.mdx new file mode 100644 index 000000000..785b57baa --- /dev/null +++ b/cometbft/v0.39/docs/experimental/lib-p2p.mdx @@ -0,0 +1,228 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/docs/experimental/lib-p2p' +order: 1 +title: "Lib-P2P Networking" +description: "Performant and scalable peer-to-peer networking for CometBFT" +icon: "waypoints" +--- + + + This is an experimental feature. + It has been tested under a range of network conditions and perturbations, but you should validate it + for your specific workload before enabling it on a production mainnet chain. + + + + Use only alongside [AdaptiveSync](/cometbft/v0.39/docs/core/block-sync#adaptivesync) + + +This is an experimental networking layer based on [go-libp2p](https://libp2p.io/). +It adds a new transport and connection-management layer for peer-to-peer communication, +while keeping the reactor-facing CometBFT API unchanged. + +Actors still use the same core p2p concepts (`Switch`, `Peer`, `PeerSet`, `Reactor`, `Envelope`, ...). +The transport implementation under those abstractions is `lib-p2p` instead of `comet-p2p`. + +lib-p2p is a widely used networking stack with production-ready peer-to-peer features, +and implementations across many languages and transport protocols (TCP, QUIC, WebSockets, and more). + +You can refer to the implementation in the CometBFT codebase here: + +- [lp2p](https://github.com/cometbft/cometbft/tree/main/lp2p) +- [internal/autopool](https://github.com/cometbft/cometbft/tree/main/internal/autopool) + +## Performance and Liveness + +In high-load conditions, legacy `comet-p2p` can become a networking bottleneck for modern blockchain workloads: + +- It is more prone to congestion under concurrent message pressure. +- Stream/message handling is less effective at scaling with traffic spikes. +- This limits end-to-end throughput when the rest of the stack is optimized. + +The `lib-p2p` integration addresses this with native stream-oriented transport, concurrent receive pipelines, +and autoscaled worker pools per reactor, which helps reduce queue pressure and improve message flow under load. +Beyond raw throughput, this also improves network liveness by making peer communication and block propagation +more resilient under sustained congestion and sudden load spikes. + +In our benchmarks, together with additional performance improvements across the stack, we reached over 2000 TPS, +and `lib-p2p` has been one of the key unblockers enabling that result. + +## Differences in Transport and Peer IDs + +`lib-p2p` uses its own [peer ID format](https://github.com/libp2p/specs/blob/master/peer-ids/peer-ids.md), +which is different from `comet-p2p`. The two formats are not compatible. + +```text +# comet-p2p peer ID format +539ffc12a0ac78970dab31ae8cdcfdd4285b2162@10.186.73.3:26656 + +# lib-p2p peer ID format +{ host = "10.186.73.3:26656", id = "12D3KooWRuTppVZGE7qhanfsHfmzkWUZnnRbTxbgWYvKibij9niy" } +``` + +To print your node's lib-p2p peer ID from the CLI: + +```bash +cometbft show-node-id --libp2p +# e.g. 12D3KooWJwoqHMXukQGFg425582Jr2Cq9VLE6MtbRt21hRrudjqM +``` + + + lib-p2p uses QUIC instead of TCP, ensure UDP communication is allowed in your firewall. + + +## Configuration + +Configure `lib-p2p` in the `p2p.libp2p` section of `config.toml`. +All other p2p settings are ignored except `external_address` and `laddr`. + +By default, the node listens on UDP port `26656`. + + + To validate successful configuration, check logs for:
+ **"EXPERIMENTAL: go-libp2p transport is enabled."** + + + +```toml +[p2p.libp2p] + +# Enabled set true to use go-libp2p for networking instead of CometBFT's p2p. +enabled = true + +# Bootstrap peers to connect to +# format: { host, id, private (opt), persistent (opt), unconditional (opt) } +# DNS resolution is also supported (e.g. "example.com:26656") +bootstrap_peers = [ + { host = "10.186.73.3:26656", id = "12D3KooWRuTppVZGE7qhanfsHfmzkWUZnnRbTxbgWYvKibij9niy", persistent = true }, + { host = "10.186.73.5:26656", id = "12D3KooWHjC8SJFVpAvY3qpM5PPeXSpQZvLxcZb7Tjr1kHMLEFtS", persistent = true }, + { host = "10.186.73.6:26656", id = "12D3KooWJFbLcqdPpNP7E1EXC4tDiPtxEGDM6K7RXpDsdWmVnjSu", persistent = true }, +] + +# Options for scaling concurrent p2p message queues. +[p2p.libp2p.scaler] +min_workers = 4 +max_workers = 32 + +# downscale concurrency if P90 latency of message processing +# is longer than the threshold +threshold_latency = "100ms" + +## Optional per-reactor override. +# [[p2p.libp2p.scaler.overrides]] +# reactor = "MEMPOOL" +# min_workers = 8 +# max_workers = 512 +# threshold_latency = "500ms" + +# Resource limits mode: +# - disabled: no limits (unsafe on untrusted/public networks) +# - default: lib-p2p defaults +# - custom: enforce explicit peer/stream caps +[p2p.libp2p.limits] +mode = "default" + +# Used only in custom mode. +# max_peers = 200 +# max_peer_streams = 16 +``` + +Each bootstrap peer supports these options: + +- `persistent`: ensures the peer is always (re)connected. +- `unconditional`: not affected by the max number of peers limit. +- `private`: peer is not gossiped to other peers. + +### Queue Scaler + +The queue scaler controls reactor receive concurrency using a throughput/latency feedback loop. + +- `min_workers`, `max_workers`: lower/upper worker bounds per reactor. +- `threshold_latency`: target processing-latency threshold. +- `overrides`: per-reactor values (case-insensitive reactor name). + +In most deployments, the defaults are enough and should be used first. +Tune only when metrics show persistent queue growth, elevated receive latency, or poor throughput. + +### Resource Manager + +Resource manager mode determines connection and stream limits: + +- `default`: uses libp2p autoscaled limits and sane built-in protocol caps. +- `custom`: disables most cometbft p2p limits but enforces explicit `max_peers` and `max_peer_streams` caps. +- `disabled`: no limits, useful for controlled benchmarking and local testing. + +Recommended tuning flow if defaults are not enough: + +1. Start with `mode = "default"` and observe metrics under representative load. +2. If limits are still unclear, run short, controlled tests with `mode = "disabled"` to discover required headroom. +3. Move to `mode = "custom"` and set conservative `max_peers` / `max_peer_streams` caps based on measurements. +4. Re-test and keep safety margin; avoid running public networks long-term in `disabled` mode. + +You can find more details about lib-p2p resource manager here: +- [libp2p/go-libp2p: p2p/host/resource-manager/README.md](https://github.com/libp2p/go-libp2p/blob/master/p2p/host/resource-manager/README.md) +- [libp2p/go-libp2p: p2p/host/resource-manager/limit_defaults.go](https://github.com/libp2p/go-libp2p/blob/062200be7aa1d18a0f54eefb17b0dbe2e96f0a79/p2p/host/resource-manager/limit_defaults.go#L663) + +## Implementation Details + +From the CometBFT actor-model perspective, the API stays the same: +`Reactor`, `Peer`, `PeerSet`, `Switch`, and envelope flow remain compatible, +so existing reactors can run without protocol-level rewrites. + +At the connection layer, lib-p2p replaces CometBFT secret connection with lib-p2p native +identity and [secure handshake mechanisms](https://github.com/libp2p/specs/blob/master/tls/tls.md). +This means peer session establishment, encryption negotiation, and remote identification are handled +by the lib-p2p transport stack. + +CometBFT channel traffic is mapped to lib-p2p protocol handlers: + +- Each CometBFT channel is exposed under a lib-p2p `protocol.ID` namespace (for example `/p2p/cometbft/1.0.0/...`). +- Messages are exchanged over lib-p2p streams bound to those protocol handlers. +- Inbound handling is concurrent, with a priority FIFO queue and worker pool to process + reactor traffic in parallel under load. + +The worker pool is autoscaled by `autopool`: + +- It tracks per-message processing durations and computes decisions from throughput EWMA, + queue pressure, and latency percentile (P90). +- High throughput growth or queue pressure scales workers up; high P90 latency above the + configured threshold triggers shrink to avoid overload. +- Default limits are 4-32 workers per reactor; mempool uses a wider range (8-512) to + absorb bursty transaction traffic. +- Priority ordering is preserved before dispatch (`Receive()` pushes by priority, then workers consume in parallel). + +## Comparison and Limitations + + +The current release does **not** include peer exchange (PEX). +Nodes must use explicit bootstrap peers/static topology; PEX will be added in a future release. + + + +| Area | comet-p2p | lib-p2p | +| -------------------- | ---------------------------------- | ------------------------------------- | +| Transport | TCP | QUIC | +| Peer identity | Comet peer IDs (`@host:port`) | lib-p2p peer IDs (`12D3Koo...`) | +| Connection handshake | Comet secret connection | lib-p2p identity and secure handshake | +| Peer exchange | PEX + address book flow | **No PEX in this release** | + +Because peer identification formats differ, you cannot run a mixed `comet-p2p` / `lib-p2p` network. + +## Metrics + +Key metrics for `lib-p2p` queue-scaler and resource tuning: + +```text +cometbft_p2p_message_reactor_queue_concurrency +cometbft_p2p_messages_reactor_in_flight +cometbft_p2p_messages_reactor_pending_duration +cometbft_p2p_message_reactor_receive_duration +cometbft_p2p_messages_received +cometbft_p2p_peer_receive_bytes_total +cometbft_p2p_peer_send_bytes_total +cometbft_p2p_message_receive_bytes_total +cometbft_p2p_message_send_bytes_total +cometbft_p2p_peer_send_queue_size +cometbft_p2p_peers +``` diff --git a/cometbft/v0.39/docs/explanation/core/metrics.md b/cometbft/v0.39/docs/explanation/core/metrics.md new file mode 100644 index 000000000..90bc8a867 --- /dev/null +++ b/cometbft/v0.39/docs/explanation/core/metrics.md @@ -0,0 +1,92 @@ +--- +order: 5 +--- + +# Metrics + +CometBFT can report and serve the Prometheus metrics, which in their turn can +be consumed by Prometheus collector(s). + +This functionality is disabled by default. + +To enable the Prometheus metrics, set `instrumentation.prometheus=true` in your +config file. Metrics will be served under `/metrics` on 26660 port by default. +Listen address can be changed in the config file (see +`instrumentation.prometheus\_listen\_addr`). + +## List of available metrics + +The following metrics are available: + +| **Name** | **Type** | **Tags** | **Description** | +| ------------------------------------------------------- | --------- | ------------------ | -------------------------------------------------------------------------------------------------------------------------------------- | +| abci\_connection\_method\_timing\_seconds | Histogram | method, type | Timings for each of the ABCI methods | +| blocksync\_syncing | Gauge | | Either 0 (not block syncing) or 1 (syncing) | +| consensus\_height | Gauge | | Height of the chain | +| consensus\_validators | Gauge | | Number of validators | +| consensus\_validators\_power | Gauge | validator\_address | Total voting power of all validators | +| consensus\_validator\_power | Gauge | validator\_address | Voting power of the node if in the validator set | +| consensus\_validator\_last\_signed\_height | Gauge | validator\_address | Last height the node signed a block, if the node is a validator | +| consensus\_validator\_missed\_blocks | Gauge | | Total amount of blocks missed for the node, if the node is a validator | +| consensus\_missing\_validators | Gauge | | Number of validators who did not sign | +| consensus\_missing\_validators\_power | Gauge | | Total voting power of the missing validators | +| consensus\_byzantine\_validators | Gauge | | Number of validators who tried to double sign | +| consensus\_byzantine\_validators\_power | Gauge | | Total voting power of the byzantine validators | +| consensus\_block\_interval\_seconds | Histogram | | Time between this and last block (Block.Header.Time) in seconds | +| consensus\_rounds | Gauge | | Number of rounds | +| consensus\_num\_txs | Gauge | | Number of transactions | +| consensus\_total\_txs | Gauge | | Total number of transactions committed | +| consensus\_block\_parts | Counter | peer\_id | Number of blockparts transmitted by peer | +| consensus\_latest\_block\_height | Gauge | | /status sync\_info number | +| consensus\_block\_size\_bytes | Gauge | | Block size in bytes | +| consensus\_step\_duration\_seconds | Histogram | step | Histogram of durations for each step in the consensus protocol | +| consensus\_round\_duration\_seconds | Histogram | | Histogram of durations for all the rounds that have occurred since the process started | +| consensus\_block\_gossip\_parts\_received | Counter | matches\_current | Number of block parts received by the node | +| consensus\_quorum\_prevote\_delay | Gauge | proposer\_address | Interval in seconds between the proposal timestamp and the timestamp of the earliest prevote that achieved a quorum | +| consensus\_full\_prevote\_delay | Gauge | proposer\_address | Interval in seconds between the proposal timestamp and the timestamp of the latest prevote in a round where all validators voted | +| consensus\_vote\_extension\_receive\_count | Counter | status | Number of vote extensions received | +| consensus\_proposal\_receive\_count | Counter | status | Total number of proposals received by the node since process start | +| consensus\_proposal\_create\_count | Counter | | Total number of proposals created by the node since process start | +| consensus\_round\_voting\_power\_percent | Gauge | vote\_type | A value between 0 and 1.0 representing the percentage of the total voting power per vote type received within a round | +| consensus\_late\_votes | Counter | vote\_type | Number of votes received by the node since process start that correspond to earlier heights and rounds than this node is currently in. | +| consensus\_duplicate\_vote | Counter | | Number of times we received a duplicate vote. | +| consensus\_duplicate\_block\_part | Counter | | Number of times we received a duplicate block part. | +| consensus\_proposal\_timestamp\_difference | Histogram | is\_timely | Difference between the timestamp in the proposal message and the local time of the validator at the time it received the message. | +| p2p\_message\_send\_bytes\_total | Counter | message\_type | Number of bytes sent to all peers per message type | +| p2p\_message\_receive\_bytes\_total | Counter | message\_type | Number of bytes received from all peers per message type | +| p2p\_peers | Gauge | | Number of peers node's connected to | +| p2p\_peer\_pending\_send\_bytes | Gauge | peer\_id | Number of pending bytes to be sent to a given peer | +| p2p\_recv\_rate\_limiter\_delay | Counter | peer\_id | Time in seconds spent sleeping by the receive rate limiter, in seconds. | +| p2p\_send\_rate\_limiter\_delay | Counter | peer\_id | Time in seconds spent sleeping by the send rate limiter, in seconds. | +| mempool\_size | Gauge | | Number of uncommitted transactions in the mempool | +| mempool\_size\_bytes | Gauge | | Total size of the mempool in bytes | +| mempool\_tx\_size\_bytes | Histogram | | Histogram of transaction sizes in bytes | +| mempool\_evicted\_txs | Counter | | Number of transactions that make it into the mempool and were later evicted for being invalid | +| mempool\_failed\_txs | Counter | | Number of transactions that failed to make it into the mempool for being invalid | +| mempool\_rejected\_txs | Counter | | Number of transactions that failed to make it into the mempool due to resource limits | +| mempool\_recheck\_times | Counter | | Number of times transactions are rechecked in the mempool | +| mempool\_already\_received\_txs | Counter | | Number of times transactions were received more than once | +| mempool\_active\_outbound\_connections | Gauge | | Number of connections being actively used for gossiping transaction (experimental) | +| mempool\_recheck\_duration\_seconds | Gauge | | Cumulative time spent rechecking transactions | +| state\_consensus\_param\_updates | Counter | | Number of consensus parameter updates returned by the application since process start | +| state\_validator\_set\_updates | Counter | | Number of validator set updates returned by the application since process start | +| state\_pruning\_service\_block\_retain\_height | Gauge | | Accepted block retain height set by the data companion | +| state\_pruning\_service\_block\_results\_retain\_height | Gauge | | Accepted block results retain height set by the data companion | +| state\_pruning\_service\_tx\_indexer\_retain\_height | Gauge | | Accepted transactions indices retain height set by the data companion | +| state\_pruning\_service\_block\_indexer\_retain\_height | Gauge | | Accepted blocks indices retain height set by the data companion | +| state\_application\_block\_retain\_height | Gauge | | Accepted block retain height set by the application | +| state\_block\_store\_base\_height | Gauge | | First height at which a block is available | +| state\_abciresults\_base\_height | Gauge | | First height at which ABCI results are available | +| state\_tx\_indexer\_base\_height | Gauge | | First height at which tx indices are available | +| state\_block\_indexer\_base\_height | Gauge | | First height at which block indices are available | +| state\_store\_access\_duration\_seconds | Histogram | method | Duration of accesses to the state store labeled by which method was called on the store | +| state\_fire\_block\_events\_delay\_seconds | Gauge | | Duration of event firing related to a new block | +| statesync\_syncing | Gauge | | Either 0 (not state syncing) or 1 (syncing) | + +## Useful queries + +Percentage of missing + byzantine validators: + +```md +((consensus\_byzantine\_validators\_power + consensus\_missing\_validators\_power) / consensus\_validators\_power) * 100 +``` diff --git a/cometbft/v0.39/docs/guides/Creating-a-built-in-application-in-Go.mdx b/cometbft/v0.39/docs/guides/Creating-a-built-in-application-in-Go.mdx new file mode 100644 index 000000000..82fc79718 --- /dev/null +++ b/cometbft/v0.39/docs/guides/Creating-a-built-in-application-in-Go.mdx @@ -0,0 +1,799 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/docs/guides/Creating-a-built-in-application-in-Go' +title: Creating a built-in application in Go +order: 2 +--- + +## Guide Assumptions + +This guide is designed for beginners who want to get started with a CometBFT +application from scratch. It does not assume that you have any prior +experience with CometBFT. + +CometBFT is a service that provides a Byzantine Fault Tolerant consensus engine +for state-machine replication. The replicated state machine, or "application", can be written +in any language that can send and receive protocol buffer messages in a client-server model. +Applications written in Go can also use CometBFT as a library and run the service in the same +process as the application. + +By following along with this tutorial, you will create a CometBFT application called kvstore, +a (very) simple distributed BFT key-value store. +The application will be written in Go, and +some understanding of the Go programming language is expected. +If you have never written Go, you may want to go through [Learn X in Y minutes +Where X=Go](https://learnxinyminutes.com/docs/go/) first to familiarize +yourself with the syntax. + +Note: Please use the latest released version of this guide and of CometBFT. +We strongly advise against using unreleased commits for your development. + +### Built-in app vs external app + +On the one hand, to get maximum performance, you can run your application in +the same process as CometBFT, as long as your application is written in Go. +[Cosmos SDK](https://github.com/cosmos/cosmos-sdk) is written +this way. +This is the approach followed in this tutorial. + +On the other hand, having a separate application might give you better security +guarantees, as two processes would be communicating via an established binary protocol. +CometBFT will not have access to the application's state. +If that is the way you wish to proceed, use the [Creating an application in Go](/cometbft/v0.39/docs/guides/Creating-an-application-in-Go) guide instead of this one. + +## 1.1 Installing Go + +Verify that you have the latest version of Go installed (refer to the [official guide for installing Go](https://golang.org/doc/install)): + +```bash +$ go version +go version go1.22.11 darwin/amd64 +``` + +## 1.2 Creating a new Go project + +We'll start by creating a new Go project. + +```bash +mkdir kvstore +``` + +Inside the example directory, create a `main.go` file with the following content: + +```go +package main + +import ( + "fmt" +) + +func main() { + fmt.Println("Hello, CometBFT") +} +``` + +When run, this should print "Hello, CometBFT" to the standard output. + +```bash +cd kvstore +$ go run main.go +Hello, CometBFT +``` + +We are going to use [Go modules](https://github.com/golang/go/wiki/Modules) for +dependency management, so let's start by including a dependency on the latest version of +CometBFT, `v0.38.0` in this example. + +```bash +go mod init kvstore +go get github.com/cometbft/cometbft@v0.38.0 +``` + +After running the above commands, you will see two generated files, `go.mod` and `go.sum`. +The `go.mod` file should look similar to: + +```go +module kvstore + +go 1.22 + +require ( +github.com/cometbft/cometbft v0.38.0 +) +``` + +XXX: CometBFT `v0.38.0` uses a slightly outdated `gogoproto` library, which +may fail to compile with newer Go versions. To avoid any compilation errors, +upgrade `gogoproto` manually: + +```bash +go get github.com/cosmos/gogoproto@v1.4.11 +``` + +As you write the kvstore application, you can rebuild the binary by +pulling any new dependencies and recompiling it. + +```bash +go get +go build +``` + +## 1.3 Writing a CometBFT application + +CometBFT communicates with the application through the Application +BlockChain Interface (ABCI). The messages exchanged through the interface are +defined in the ABCI [protobuf +file](https://github.com/cometbft/cometbft/blob/v0.38.x/proto/tendermint/abci/types.proto). + +We begin by creating the basic scaffolding for an ABCI application by +creating a new type, `KVStoreApplication`, which implements the +methods defined by the `abcitypes.Application` interface. + +Create a file called `app.go` with the following contents: + +```go +package main + +import ( + abcitypes "github.com/cometbft/cometbft/abci/types" + "context" +) + +type KVStoreApplication struct{} + +var _ abcitypes.Application = (*KVStoreApplication)(nil) + +func NewKVStoreApplication() *KVStoreApplication { + return &KVStoreApplication{} +} + +func (app *KVStoreApplication) Info(_ context.Context, info *abcitypes.RequestInfo) (*abcitypes.ResponseInfo, error) { + return &abcitypes.ResponseInfo{}, nil +} + +func (app *KVStoreApplication) Query(_ context.Context, req *abcitypes.RequestQuery) (*abcitypes.ResponseQuery, error) { + return &abcitypes.ResponseQuery{}, nil +} + +func (app *KVStoreApplication) CheckTx(_ context.Context, check *abcitypes.RequestCheckTx) (*abcitypes.ResponseCheckTx, error) { + return &abcitypes.ResponseCheckTx{}, nil +} + +func (app *KVStoreApplication) InitChain(_ context.Context, chain *abcitypes.RequestInitChain) (*abcitypes.ResponseInitChain, error) { + return &abcitypes.ResponseInitChain{}, nil +} + +func (app *KVStoreApplication) PrepareProposal(_ context.Context, proposal *abcitypes.RequestPrepareProposal) (*abcitypes.ResponsePrepareProposal, error) { + return &abcitypes.ResponsePrepareProposal{}, nil +} + +func (app *KVStoreApplication) ProcessProposal(_ context.Context, proposal *abcitypes.RequestProcessProposal) (*abcitypes.ResponseProcessProposal, error) { + return &abcitypes.ResponseProcessProposal{}, nil +} + +func (app *KVStoreApplication) FinalizeBlock(_ context.Context, req *abcitypes.RequestFinalizeBlock) (*abcitypes.ResponseFinalizeBlock, error) { + return &abcitypes.ResponseFinalizeBlock{}, nil +} + +func (app KVStoreApplication) Commit(_ context.Context, commit *abcitypes.RequestCommit) (*abcitypes.ResponseCommit, error) { + return &abcitypes.ResponseCommit{}, nil +} + +func (app *KVStoreApplication) ListSnapshots(_ context.Context, snapshots *abcitypes.RequestListSnapshots) (*abcitypes.ResponseListSnapshots, error) { + return &abcitypes.ResponseListSnapshots{}, nil +} + +func (app *KVStoreApplication) OfferSnapshot(_ context.Context, snapshot *abcitypes.RequestOfferSnapshot) (*abcitypes.ResponseOfferSnapshot, error) { + return &abcitypes.ResponseOfferSnapshot{}, nil +} + +func (app *KVStoreApplication) LoadSnapshotChunk(_ context.Context, chunk *abcitypes.RequestLoadSnapshotChunk) (*abcitypes.ResponseLoadSnapshotChunk, error) { + return &abcitypes.ResponseLoadSnapshotChunk{}, nil +} + +func (app *KVStoreApplication) ApplySnapshotChunk(_ context.Context, chunk *abcitypes.RequestApplySnapshotChunk) (*abcitypes.ResponseApplySnapshotChunk, error) { + return &abcitypes.ResponseApplySnapshotChunk{Result: abcitypes.ResponseApplySnapshotChunk_ACCEPT}, nil +} + +func (app KVStoreApplication) ExtendVote(_ context.Context, extend *abcitypes.RequestExtendVote) (*abcitypes.ResponseExtendVote, error) { + return &abcitypes.ResponseExtendVote{}, nil +} + +func (app *KVStoreApplication) VerifyVoteExtension(_ context.Context, verify *abcitypes.RequestVerifyVoteExtension) (*abcitypes.ResponseVerifyVoteExtension, error) { + return &abcitypes.ResponseVerifyVoteExtension{}, nil +} +``` + +The types used here are defined in the CometBFT library and were added as a dependency +to the project when you ran `go get`. If your IDE is not recognizing the types, go ahead and run the command again. + +```bash +go get github.com/cometbft/cometbft@v0.38.0 +``` + +Now go back to `main.go` and modify the `main` function so it matches the following, +where an instance of the `KVStoreApplication` type is created. + +```go +func main() { + fmt.Println("Hello, CometBFT") + + _ = NewKVStoreApplication() +} +``` + +You can recompile and run the application now by running `go get` and `go build`, but it does +not do anything. +So let's revisit the code, adding the logic needed to implement our minimal key-value store +and to start it along with the CometBFT service. + +### 1.3.1 Add a persistent data store + +Our application will need to write its state out to persistent storage so that it +can stop and start without losing all of its data. + +For this tutorial, we will use [BadgerDB](https://github.com/dgraph-io/badger), a +fast embedded key-value store. + +First, add Badger as a dependency of your go module using the `go get` command: + +`go get github.com/dgraph-io/badger/v3` + +Next, let's update the application and its constructor to receive a handle to the database, as follows: + +```go +type KVStoreApplication struct { + db *badger.DB + onGoingBlock *badger.Txn +} + +var _ abcitypes.Application = (*KVStoreApplication)(nil) + +func NewKVStoreApplication(db *badger.DB) *KVStoreApplication { + return &KVStoreApplication{db: db} +} +``` + +The `onGoingBlock` keeps track of the Badger transaction that will update the application's state when a block +is completed. Don't worry about it for now; we'll get to that later. + +Next, update the `import` stanza at the top to include the Badger library: + +```go +import( + "github.com/dgraph-io/badger/v3" + abcitypes "github.com/cometbft/cometbft/abci/types" +) +``` + +Finally, update the `main.go` file to invoke the updated constructor: + +```go + _ = NewKVStoreApplication(nil) +``` + +### 1.3.2 CheckTx + +When CometBFT receives a new transaction from a client or from another full node, +CometBFT asks the application if the transaction is acceptable using the `CheckTx` method. +Invalid transactions will not be shared with other nodes and will not become part of any blocks and, therefore, will not be executed by the application. + +In our application, a transaction is a string with the form `key=value`, indicating a key and value to write to the store. + +The most basic validation check we can perform is to check if the transaction conforms to the `key=value` pattern. +For that, let's add the following helper method to `app.go`: + +```go +func (app *KVStoreApplication) isValid(tx []byte) uint32 { + // check format + parts := bytes.Split(tx, []byte("=")) + if len(parts) != 2 { + return 1 + } + + return 0 +} +``` + +Now you can rewrite the `CheckTx` method to use the helper function: + +```go +func (app *KVStoreApplication) CheckTx(_ context.Context, check *abcitypes.RequestCheckTx) (*abcitypes.ResponseCheckTx, error) { + code := app.isValid(check.Tx) + return &abcitypes.ResponseCheckTx{Code: code}, nil +} +``` + +While this `CheckTx` is simple and only validates that the transaction is well-formed, +it is very common for `CheckTx` to make more complex use of the state of an application. +For example, you may refuse to overwrite an existing value, or you can associate +versions to the key-value pairs and allow the caller to specify a version to +perform a conditional update. + +Depending on the checks and on the conditions violated, the function may return +different values, but any response with a non-zero code will be considered invalid +by CometBFT. Our `CheckTx` logic returns 0 to CometBFT when a transaction passes +its validation checks. The specific value of the code is meaningless to CometBFT. +Non-zero codes are logged by CometBFT, so applications can provide more specific +information on why the transaction was rejected. + +Note that `CheckTx` does not execute the transaction; it only verifies that the transaction could be executed. We do not know yet if the rest of the network has agreed to accept this transaction into a block. + +Finally, make sure to add the `bytes` package to the `import` stanza at the top of `app.go`: + +```go +import( + "bytes" + + "github.com/dgraph-io/badger/v3" + abcitypes "github.com/cometbft/cometbft/abci/types" +) +``` + +### 1.3.3 FinalizeBlock + +When the CometBFT consensus engine has decided on the block, the block is transferred to the +application via `FinalizeBlock`. +`FinalizeBlock` is an ABCI method introduced in CometBFT `v0.38.0`. This replaces the functionality provided previously (pre-`v0.38.0`) by the combination of ABCI methods `BeginBlock`, `DeliverTx`, and `EndBlock`. `FinalizeBlock`'s parameters are an aggregation of those in `BeginBlock`, `DeliverTx`, and `EndBlock`. + +This method is responsible for executing the block and returning a response to the consensus engine. +Providing a single `FinalizeBlock` method to signal the finalization of a block simplifies the ABCI interface and increases flexibility in the execution pipeline. + +The `FinalizeBlock` method executes the block, including any necessary transaction processing and state updates, and returns a `ResponseFinalizeBlock` object, which contains any necessary information about the executed block. + +**Note:** `FinalizeBlock` only prepares the update to be made and does not change the state of the application. The state change is actually committed in a later stage, i.e., in the `commit` phase. + +Note that to implement these calls in our application, we're going to make use of Badger's transaction mechanism. We will always refer to these as Badger transactions, not to confuse them with the transactions included in the blocks delivered by CometBFT, the _application transactions_. + +First, let's create a new Badger transaction during `FinalizeBlock`. All application transactions in the current block will be executed within this Badger transaction. +Next, let's modify `FinalizeBlock` to add the `key` and `value` to the Badger transaction every time our application processes a new application transaction from the list received through `RequestFinalizeBlock`. + +Note that we check the validity of the transaction _again_ during `FinalizeBlock`. + +```go +func (app *KVStoreApplication) FinalizeBlock(_ context.Context, req *abcitypes.RequestFinalizeBlock) (*abcitypes.ResponseFinalizeBlock, error) { + var txs = make([]*abcitypes.ExecTxResult, len(req.Txs)) + + app.onGoingBlock = app.db.NewTransaction(true) + for i, tx := range req.Txs { + if code := app.isValid(tx); code != 0 { + log.Printf("Error: invalid transaction index %v", i) + txs[i] = &abcitypes.ExecTxResult{Code: code} + } else { + parts := bytes.SplitN(tx, []byte("="), 2) + key, value := parts[0], parts[1] + log.Printf("Adding key %s with value %s", key, value) + + if err := app.onGoingBlock.Set(key, value); err != nil { + log.Panicf("Error writing to database, unable to execute tx: %v", err) + } + + log.Printf("Successfully added key %s with value %s", key, value) + + txs[i] = &abcitypes.ExecTxResult{} + } + } + + return &abcitypes.ResponseFinalizeBlock{ + TxResults: txs, + }, nil +} +``` + +Transactions are not guaranteed to be valid when they are delivered to an application, even if they were valid when they were proposed. + +This can happen if the application state is used to determine transaction validity. +The application state may have changed between the initial execution of `CheckTx` and the transaction delivery in `FinalizeBlock` in a way that rendered the transaction no longer valid. + +**Note** that `FinalizeBlock` cannot yet commit the Badger transaction we were building during the block execution. + +Other methods, such as `Query`, rely on a consistent view of the application's state; the application should only update its state by committing the Badger transactions when the full block has been delivered and the `Commit` method is invoked. + +The `Commit` method tells the application to make permanent the effects of +the application transactions. +Let's update the method to terminate the pending Badger transaction and +persist the resulting state: + +```go +func (app KVStoreApplication) Commit(_ context.Context, commit *abcitypes.RequestCommit) (*abcitypes.ResponseCommit, error) { + return &abcitypes.ResponseCommit{}, app.onGoingBlock.Commit() +} +``` + +Finally, make sure to add the log library to the `import` stanza as well: + +```go +import ( + "bytes" + "log" + + "github.com/dgraph-io/badger/v3" + abcitypes "github.com/cometbft/cometbft/abci/types" +) +``` + +You may have noticed that the application we are writing will crash if it receives +an unexpected error from the Badger database during the `FinalizeBlock` or `Commit` methods. +This is not an accident. If the application received an error from the database, there +is no deterministic way for it to make progress, so the only safe option is to terminate. +Once the application is restarted, the transactions in the block that failed execution will +be re-executed and should succeed if the Badger error was transient. + +### 1.3.4 Query + +When a client tries to read some information from the `kvstore`, the request will be +handled in the `Query` method. To do this, let's rewrite the `Query` method in `app.go`: + +```go +func (app *KVStoreApplication) Query(_ context.Context, req *abcitypes.RequestQuery) (*abcitypes.ResponseQuery, error) { + resp := abcitypes.ResponseQuery{Key: req.Data} + + dbErr := app.db.View(func(txn *badger.Txn) error { + item, err := txn.Get(req.Data) + if err != nil { + if err != badger.ErrKeyNotFound { + return err + } + resp.Log = "key does not exist" + return nil + } + + return item.Value(func(val []byte) error { + resp.Log = "exists" + resp.Value = val + return nil + }) + }) + if dbErr != nil { + log.Panicf("Error reading database, unable to execute query: %v", dbErr) + } + return &resp, nil +} +``` + +Since it reads only committed data from the store, transactions that are part of a block +that is being processed are not reflected in the query result. + +### 1.3.5 PrepareProposal and ProcessProposal + +`PrepareProposal` and `ProcessProposal` are methods introduced in CometBFT v0.37.0 +to give the application more control over the construction and processing of transaction blocks. + +When CometBFT sees that valid transactions (validated through `CheckTx`) are available to be +included in blocks, it groups some of these transactions and then gives the application a chance +to modify the group by invoking `PrepareProposal`. + +The application is free to modify the group before returning from the call, as long as the resulting set +does not use more bytes than `RequestPrepareProposal.max_tx_bytes`. +For example, the application may reorder, add, or even remove transactions from the group to improve the +execution of the block once accepted. + +In the following code, the application simply returns the unmodified group of transactions: + +```go +func (app *KVStoreApplication) PrepareProposal(_ context.Context, proposal *abcitypes.RequestPrepareProposal) (*abcitypes.ResponsePrepareProposal, error) { + return &abcitypes.ResponsePrepareProposal{Txs: proposal.Txs}, nil +} +``` + +Once a proposed block is received by a node, the proposal is passed to the application to give +its blessing before voting to accept the proposal. + +This mechanism may be used for different reasons, for example, to deal with blocks manipulated +by malicious nodes, in which case the block should not be considered valid. + +The following code simply accepts all proposals: + +```go +func (app *KVStoreApplication) ProcessProposal(_ context.Context, proposal *abcitypes.RequestProcessProposal) (*abcitypes.ResponseProcessProposal, error) { + return &abcitypes.ResponseProcessProposal{Status: abcitypes.ResponseProcessProposal_ACCEPT}, nil +} +``` + +## 1.4 Starting an application and a CometBFT instance in the same process + +Now that we have the basic functionality of our application in place, let's put +it all together inside of our `main.go` file. + +Change the contents of your `main.go` file to the following. + +```go +package main + +import ( + "flag" + "fmt" + "github.com/cometbft/cometbft/p2p" + "github.com/cometbft/cometbft/privval" + "github.com/cometbft/cometbft/proxy" + "log" + "os" + "os/signal" + "path/filepath" + "syscall" + + "github.com/dgraph-io/badger/v3" + "github.com/spf13/viper" + cfg "github.com/cometbft/cometbft/config" + cmtflags "github.com/cometbft/cometbft/libs/cli/flags" + cmtlog "github.com/cometbft/cometbft/libs/log" + nm "github.com/cometbft/cometbft/node" +) + +var homeDir string + +func init() { + flag.StringVar(&homeDir, "cmt-home", "", "Path to the CometBFT config directory (if empty, uses $HOME/.cometbft)") +} + +func main() { + flag.Parse() + if homeDir == "" { + homeDir = os.ExpandEnv("$HOME/.cometbft") + } + + config := cfg.DefaultConfig() + config.SetRoot(homeDir) + viper.SetConfigFile(fmt.Sprintf("%s/%s", homeDir, "config/config.toml")) + + if err := viper.ReadInConfig(); err != nil { + log.Fatalf("Reading config: %v", err) + } + if err := viper.Unmarshal(config); err != nil { + log.Fatalf("Decoding config: %v", err) + } + if err := config.ValidateBasic(); err != nil { + log.Fatalf("Invalid configuration data: %v", err) + } + dbPath := filepath.Join(homeDir, "badger") + db, err := badger.Open(badger.DefaultOptions(dbPath)) + + if err != nil { + log.Fatalf("Opening database: %v", err) + } + defer func() { + if err := db.Close(); err != nil { + log.Printf("Closing database: %v", err) + } + }() + + app := NewKVStoreApplication(db) + + pv := privval.LoadFilePV( + config.PrivValidatorKeyFile(), + config.PrivValidatorStateFile(), + ) + + nodeKey, err := p2p.LoadNodeKey(config.NodeKeyFile()) + if err != nil { + log.Fatalf("failed to load node's key: %v", err) + } + + logger := cmtlog.NewTMLogger(cmtlog.NewSyncWriter(os.Stdout)) + logger, err = cmtflags.ParseLogLevel(config.LogLevel, logger, cfg.DefaultLogLevel) + + if err != nil { + log.Fatalf("failed to parse log level: %v", err) + } + + node, err := nm.NewNode( + config, + pv, + nodeKey, + proxy.NewLocalClientCreator(app), + nm.DefaultGenesisDocProviderFunc(config), + cfg.DefaultDBProvider, + nm.DefaultMetricsProvider(config.Instrumentation), + logger, + ) + + if err != nil { + log.Fatalf("Creating node: %v", err) + } + + node.Start() + defer func() { + node.Stop() + node.Wait() + }() + + c := make(chan os.Signal, 1) + signal.Notify(c, os.Interrupt, syscall.SIGTERM) + <-c +} +``` + +This is a huge blob of code, so let's break it down into pieces. + +First, we use [viper](https://github.com/spf13/viper) to load the CometBFT configuration files, which we will generate later: + +```go +config := cfg.DefaultConfig() + +config.SetRoot(homeDir) + +viper.SetConfigFile(fmt.Sprintf("%s/%s", homeDir, "config/config.toml")) +if err := viper.ReadInConfig(); err != nil { + log.Fatalf("Reading config: %v", err) +} +if err := viper.Unmarshal(config); err != nil { + log.Fatalf("Decoding config: %v", err) +} +if err := config.ValidateBasic(); err != nil { + log.Fatalf("Invalid configuration data: %v", err) +} +``` + +Next, we initialize the Badger database and create an app instance. + +```go +dbPath := filepath.Join(homeDir, "badger") +db, err := badger.Open(badger.DefaultOptions(dbPath)) +if err != nil { + log.Fatalf("Opening database: %v", err) +} +defer func() { + if err := db.Close(); err != nil { + log.Fatalf("Closing database: %v", err) + } +}() + +app := NewKVStoreApplication(db) +``` + +We use `FilePV`, which is a private validator (i.e., a thing which signs consensus +messages). Normally, you would use `SignerRemote` to connect to an external +[HSM](https://pkg.go.dev/github.com/certusone/yubihsm-go). + +```go +pv := privval.LoadFilePV( + config.PrivValidatorKeyFile(), + config.PrivValidatorStateFile(), +) +``` + +`nodeKey` is needed to identify the node in a p2p network. + +```go +nodeKey, err := p2p.LoadNodeKey(config.NodeKeyFile()) +if err != nil { + return nil, fmt.Errorf("failed to load node's key: %w", err) +} +``` + +Now we have everything set up to run the CometBFT node. We construct +a node by passing it the configuration, the logger, a handle to our application, and +the genesis information: + +```go +node, err := nm.NewNode( + config, + pv, + nodeKey, + proxy.NewLocalClientCreator(app), + nm.DefaultGenesisDocProviderFunc(config), + cfg.DefaultDBProvider, + nm.DefaultMetricsProvider(config.Instrumentation), + logger) + +if err != nil { + log.Fatalf("Creating node: %v", err) +} +``` + +Finally, we start the node, i.e., the CometBFT service inside our application: + +```go +node.Start() +defer func() { + node.Stop() + node.Wait() +}() +``` + +The additional logic at the end of the file allows the program to catch SIGTERM. This means that the node can shut down gracefully when an operator tries to kill the program: + +```go +c := make(chan os.Signal, 1) +signal.Notify(c, os.Interrupt, syscall.SIGTERM) +<-c +``` + +## 1.5 Initializing and Running + +Our application is almost ready to run, but first we'll need to populate the CometBFT configuration files. +The following command will create a `cometbft-home` directory in your project and add a basic set of configuration files in `cometbft-home/config/`. +For more information on what these files contain, see [the configuration documentation](https://github.com/cometbft/cometbft/blob/v0.38.x/docs/core/configuration.md). + +From the root of your project, run: + +```bash +go run github.com/cometbft/cometbft/cmd/cometbft@v0.38.0 init --home /tmp/cometbft-home +``` + +You should see an output similar to the following: + +```bash +I[2023-25-04|09:06:34.444] Generated private validator module=main keyFile=/tmp/cometbft-home/config/priv_validator_key.json stateFile=/tmp/cometbft-home/data/priv_validator_state.json +I[2023-25-04|09:06:34.444] Generated node key module=main path=/tmp/cometbft-home/config/node_key.json +I[2023-25-04|09:06:34.444] Generated genesis file module=main path=/tmp/cometbft-home/config/genesis.json +``` + +Now rebuild the app: + +```bash +go build -mod=mod # use -mod=mod to automatically refresh the dependencies +``` + +Everything is now in place to run your application. Run: + +```bash +./kvstore -cmt-home /tmp/cometbft-home +``` + +The application will start, and you should see a continuous output starting with: + +```bash +badger 2023-04-25 09:08:50 INFO: All 0 tables opened in 0s +badger 2023-04-25 09:08:50 INFO: Discard stats nextEmptySlot: 0 +badger 2023-04-25 09:08:50 INFO: Set nextTxnTs to 0 +I[2023-04-25|09:08:50.085] service start module=proxy msg="Starting multiAppConn service" impl=multiAppConn +I[2023-04-25|09:08:50.085] service start module=abci-client connection=query msg="Starting localClient service" impl=localClient +I[2023-04-25|09:08:50.085] service start module=abci-client connection=snapshot msg="Starting localClient service" impl=localClient +... +``` + +More importantly, the application using CometBFT is producing blocks 🎉🎉 and you can see this reflected in the log output in lines like this: + +```bash +I[2023-04-25|09:08:52.147] received proposal module=consensus proposal="Proposal{2/0 (F518444C0E348270436A73FD0F0B9DFEA758286BEB29482F1E3BEA75330E825C:1:C73D3D1273F2, -1) AD19AE292A45 @ 2023-04-25T12:08:52.143393Z}" +I[2023-04-25|09:08:52.152] received complete proposal block module=consensus height=2 hash=F518444C0E348270436A73FD0F0B9DFEA758286BEB29482F1E3BEA75330E825C +I[2023-04-25|09:08:52.160] finalizing commit of block module=consensus height=2 hash=F518444C0E348270436A73FD0F0B9DFEA758286BEB29482F1E3BEA75330E825C root= num_txs=0 +I[2023-04-25|09:08:52.167] executed block module=state height=2 num_valid_txs=0 num_invalid_txs=0 +I[2023-04-25|09:08:52.171] committed state module=state height=2 num_txs=0 app_hash= +``` + +The blocks, as you can see from the `num_valid_txs=0` part, are empty, but let's remedy that next. + +## 1.6 Using the application + +Let's try submitting a transaction to our new application. +Open another terminal window and run the following curl command: + +```bash +curl -s 'localhost:26657/broadcast_tx_commit?tx="cometbft=rocks"' +``` + +If everything went well, you should see a response indicating which height the +transaction was included in the blockchain. + +Finally, let's make sure that the transaction really was persisted by the application. +Run the following command: + +```bash +curl -s 'localhost:26657/abci_query?data="cometbft"' +``` + +Let's examine the response object that this request returns. +The request returns a `json` object with a `key` and `value` field set. + +```json +... + "key": "dGVuZGVybWludA==", + "value": "cm9ja3M=", +... +``` + +Those values don't look like the `key` and `value` we sent to CometBFT. +What's going on here? + +The response contains a `base64` encoded representation of the data we submitted. +To get the original value out of this data, we can use the `base64` command line utility: + +```bash +echo "cm9ja3M=" | base64 -d +``` + +## Outro + +Hope you could run everything smoothly. If you have any difficulties running through this tutorial, reach out to us via [Discord](https://discord.com/invite/interchain) or open a new [issue](https://github.com/cometbft/cometbft/issues/new/choose) on GitHub. diff --git a/cometbft/v0.39/docs/guides/Creating-an-application-in-Go.mdx b/cometbft/v0.39/docs/guides/Creating-an-application-in-Go.mdx new file mode 100644 index 000000000..c175610ab --- /dev/null +++ b/cometbft/v0.39/docs/guides/Creating-an-application-in-Go.mdx @@ -0,0 +1,711 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/docs/guides/Creating-an-application-in-Go' +title: Creating an application in Go +order: 1 +--- + +## Guide Assumptions + +This guide is designed for beginners who want to get started with a CometBFT +application from scratch. It does not assume that you have any prior +experience with CometBFT. + +CometBFT is a service that provides a Byzantine Fault Tolerant consensus engine +for state-machine replication. The replicated state-machine, or "application", can be written +in any language that can send and receive protocol buffer messages in a client-server model. +Applications written in Go can also use CometBFT as a library and run the service in the same +process as the application. + +By following along with this tutorial, you will create a CometBFT application called kvstore, +a (very) simple distributed BFT key-value store. +The application will be written in Go, and +some understanding of the Go programming language is expected. +If you have never written Go, you may want to go through [Learn X in Y minutes +Where X=Go](https://learnxinyminutes.com/docs/go/) first to familiarize +yourself with the syntax. + +Note: Please use the latest released version of this guide and of CometBFT. +We strongly advise against using unreleased commits for your development. + +### Built-in app vs external app + +On the one hand, to get maximum performance you can run your application in +the same process as CometBFT, as long as your application is written in Go. +[Cosmos SDK](https://github.com/cosmos/cosmos-sdk) is written +this way. +If that is the way you wish to proceed, use the [Creating a built-in application in Go](/cometbft/v0.39/docs/guides/Creating-a-built-in-application-in-Go) guide instead of this one. + +On the other hand, having a separate application might give you better security +guarantees as two processes would be communicating via an established binary protocol. +CometBFT will not have access to the application's state. +This is the approach followed in this tutorial. + +## 1.1 Installing Go + +Verify that you have the latest version of Go installed (refer to the [official guide for installing Go](https://golang.org/doc/install)): + +```bash +$ go version +go version go1.22.11 darwin/amd64 +``` + +## 1.2 Creating a new Go project + +We'll start by creating a new Go project. + +```bash +mkdir kvstore +``` + +Inside the example directory, create a `main.go` file with the following content: + +```go +package main + +import ( + "fmt" +) + +func main() { + fmt.Println("Hello, CometBFT") +} +``` + +When run, this should print "Hello, CometBFT" to the standard output. + +```bash +cd kvstore +$ go run main.go +Hello, CometBFT +``` + +We are going to use [Go modules](https://github.com/golang/go/wiki/Modules) for +dependency management, so let's start by including a dependency on the latest version of +CometBFT, `v0.38.0` in this example. + +```bash +go mod init kvstore +go get github.com/cometbft/cometbft@v0.38.0 +``` + +After running the above commands, you will see two generated files, `go.mod` and `go.sum`. +The go.mod file should look similar to: + +```go +module kvstore + +go 1.22 + +require ( +github.com/cometbft/cometbft v0.38.0 +) +``` + +XXX: CometBFT `v0.38.0` uses a slightly outdated `gogoproto` library, which +may fail to compile with newer Go versions. To avoid any compilation errors, +upgrade `gogoproto` manually: + +```bash +go get github.com/cosmos/gogoproto@v1.4.11 +``` + +As you write the kvstore application, you can rebuild the binary by +pulling any new dependencies and recompiling it. + +```bash +go get +go build +``` + +## 1.3 Writing a CometBFT application + +CometBFT communicates with the application through the Application +BlockChain Interface (ABCI). The messages exchanged through the interface are +defined in the ABCI [protobuf +file](https://github.com/cometbft/cometbft/blob/v0.38.x/proto/tendermint/abci/types.proto). + +We begin by creating the basic scaffolding for an ABCI application by +creating a new type, `KVStoreApplication`, which implements the +methods defined by the `abcitypes.Application` interface. + +Create a file called `app.go` with the following contents: + +```go +package main + +import ( + abcitypes "github.com/cometbft/cometbft/abci/types" + "context" +) + +type KVStoreApplication struct{} + +var _ abcitypes.Application = (*KVStoreApplication)(nil) + +func NewKVStoreApplication() *KVStoreApplication { + return &KVStoreApplication{} +} + +func (app *KVStoreApplication) Info(_ context.Context, info *abcitypes.RequestInfo) (*abcitypes.ResponseInfo, error) { + return &abcitypes.ResponseInfo{}, nil +} + +func (app *KVStoreApplication) Query(_ context.Context, req *abcitypes.RequestQuery) (*abcitypes.ResponseQuery, error) { + return &abcitypes.ResponseQuery{}, nil +} + +func (app *KVStoreApplication) CheckTx(_ context.Context, check *abcitypes.RequestCheckTx) (*abcitypes.ResponseCheckTx, error) { + return &abcitypes.ResponseCheckTx{Code: code}, nil +} + +func (app *KVStoreApplication) InitChain(_ context.Context, chain *abcitypes.RequestInitChain) (*abcitypes.ResponseInitChain, error) { + return &abcitypes.ResponseInitChain{}, nil +} + +func (app *KVStoreApplication) PrepareProposal(_ context.Context, proposal *abcitypes.RequestPrepareProposal) (*abcitypes.ResponsePrepareProposal, error) { + return &abcitypes.ResponsePrepareProposal{}, nil +} + +func (app *KVStoreApplication) ProcessProposal(_ context.Context, proposal *abcitypes.RequestProcessProposal) (*abcitypes.ResponseProcessProposal, error) { + return &abcitypes.ResponseProcessProposal{}, nil +} + +func (app *KVStoreApplication) FinalizeBlock(_ context.Context, req *abcitypes.RequestFinalizeBlock) (*abcitypes.ResponseFinalizeBlock, error) { + return &abcitypes.ResponseFinalizeBlock{}, nil +} + +func (app KVStoreApplication) Commit(_ context.Context, commit *abcitypes.RequestCommit) (*abcitypes.ResponseCommit, error) { + return &abcitypes.ResponseCommit{}, nil +} + +func (app *KVStoreApplication) ListSnapshots(_ context.Context, snapshots *abcitypes.RequestListSnapshots) (*abcitypes.ResponseListSnapshots, error) { + return &abcitypes.ResponseListSnapshots{}, nil +} + +func (app *KVStoreApplication) OfferSnapshot(_ context.Context, snapshot *abcitypes.RequestOfferSnapshot) (*abcitypes.ResponseOfferSnapshot, error) { + return &abcitypes.ResponseOfferSnapshot{}, nil +} + +func (app *KVStoreApplication) LoadSnapshotChunk(_ context.Context, chunk *abcitypes.RequestLoadSnapshotChunk) (*abcitypes.ResponseLoadSnapshotChunk, error) { + return &abcitypes.ResponseLoadSnapshotChunk{}, nil +} + +func (app *KVStoreApplication) ApplySnapshotChunk(_ context.Context, chunk *abcitypes.RequestApplySnapshotChunk) (*abcitypes.ResponseApplySnapshotChunk, error) { + + return &abcitypes.ResponseApplySnapshotChunk{Result: abcitypes.ResponseApplySnapshotChunk_ACCEPT}, nil +} + +func (app KVStoreApplication) ExtendVote(_ context.Context, extend *abcitypes.RequestExtendVote) (*abcitypes.ResponseExtendVote, error) { + return &abcitypes.ResponseExtendVote{}, nil +} + +func (app *KVStoreApplication) VerifyVoteExtension(_ context.Context, verify *abcitypes.RequestVerifyVoteExtension) (*abcitypes.ResponseVerifyVoteExtension, error) { + return &abcitypes.ResponseVerifyVoteExtension{}, nil +} +``` + +The types used here are defined in the CometBFT library and were added as a dependency +to the project when you ran `go get`. If your IDE is not recognizing the types, go ahead and run the command again. + +```bash +go get github.com/cometbft/cometbft@v0.38.0 +``` + +Now go back to `main.go` and modify the `main` function so it matches the following, +where an instance of the `KVStoreApplication` type is created. + +```go +func main() { + fmt.Println("Hello, CometBFT") + + _ = NewKVStoreApplication() +} +``` + +You can recompile and run the application now by running `go get` and `go build`, but it does +not do anything. +So let's revisit the code, adding the logic needed to implement our minimal key-value store +and to start it along with the CometBFT Service. + +### 1.3.1 Add a persistent data store + +Our application will need to write its state out to persistent storage so that it +can stop and start without losing all of its data. + +For this tutorial, we will use [BadgerDB](https://github.com/dgraph-io/badger), a +fast embedded key-value store. + +First, add Badger as a dependency of your go module using the `go get` command: + +`go get github.com/dgraph-io/badger/v3` + +Next, let's update the application and its constructor to receive a handle to the database, as follows: + +```go +type KVStoreApplication struct { + db *badger.DB + onGoingBlock *badger.Txn +} + +var _ abcitypes.Application = (*KVStoreApplication)(nil) + +func NewKVStoreApplication(db *badger.DB) *KVStoreApplication { + return &KVStoreApplication{db: db} +} +``` + +The `onGoingBlock` keeps track of the Badger transaction that will update the application's state when a block +is completed. Don't worry about it for now; we'll get to that later. + +Next, update the `import` stanza at the top to include the Badger library: + +```go +import( + "github.com/dgraph-io/badger/v3" + abcitypes "github.com/cometbft/cometbft/abci/types" +) +``` + +Finally, update the `main.go` file to invoke the updated constructor: + +```go +_ = NewKVStoreApplication(nil) +``` + +### 1.3.2 CheckTx + +When CometBFT receives a new transaction from a client, or from another full node, +CometBFT asks the application if the transaction is acceptable, using the `CheckTx` method. +Invalid transactions will not be shared with other nodes and will not become part of any blocks and, therefore, will not be executed by the application. + +In our application, a transaction is a string with the form `key=value`, indicating a key and value to write to the store. + +The most basic validation check we can perform is to check if the transaction conforms to the `key=value` pattern. +For that, let's add the following helper method to app.go: + +```go +func (app *KVStoreApplication) isValid(tx []byte) uint32 { + // check format + parts := bytes.Split(tx, []byte("=")) + if len(parts) != 2 { + return 1 + } + return 0 +} +``` + +Now you can rewrite the `CheckTx` method to use the helper function: + +```go +func (app *KVStoreApplication) CheckTx(_ context.Context, check *abcitypes.RequestCheckTx) (*abcitypes.ResponseCheckTx, error) { + code := app.isValid(check.Tx) + return &abcitypes.ResponseCheckTx{Code: code}, nil +} +``` + +While this `CheckTx` is simple and only validates that the transaction is well-formed, +it is very common for `CheckTx` to make more complex use of the state of an application. +For example, you may refuse to overwrite an existing value, or you can associate +versions with the key-value pairs and allow the caller to specify a version to +perform a conditional update. + +Depending on the checks and the conditions violated, the function may return +different values, but any response with a non-zero code will be considered invalid +by CometBFT. Our `CheckTx` logic returns 0 to CometBFT when a transaction passes +its validation checks. The specific value of the code is meaningless to CometBFT. +Non-zero codes are logged by CometBFT so applications can provide more specific +information on why the transaction was rejected. + +Note that `CheckTx` does not execute the transaction; it only verifies that the transaction could be executed. We do not know yet if the rest of the network has agreed to accept this transaction into a block. + +Finally, make sure to add the bytes package to the `import` stanza at the top of `app.go`: + +```go +import( + "bytes" + + "github.com/dgraph-io/badger/v3" + abcitypes "github.com/cometbft/cometbft/abci/types" +) +``` + +### 1.3.3 FinalizeBlock + +When the CometBFT consensus engine has decided on the block, the block is transferred to the +application via the `FinalizeBlock` method. +`FinalizeBlock` is an ABCI method introduced in CometBFT `v0.38.0`. This replaces the functionality provided previously (pre-`v0.38.0`) by the combination of ABCI methods `BeginBlock`, `DeliverTx`, and `EndBlock`. +`FinalizeBlock`'s parameters are an aggregation of those in `BeginBlock`, `DeliverTx`, and `EndBlock`. + +This method is responsible for executing the block and returning a response to the consensus engine. +Providing a single `FinalizeBlock` method to signal the finalization of a block simplifies the ABCI interface and increases flexibility in the execution pipeline. + +The `FinalizeBlock` method executes the block, including any necessary transaction processing and state updates, and returns a `ResponseFinalizeBlock` object which contains any necessary information about the executed block. + +**Note:** `FinalizeBlock` only prepares the update to be made and does not change the state of the application. The state change is actually committed at a later stage, in the `commit` phase. + +Note that to implement these calls in our application, we're going to make use of Badger's transaction mechanism. We will always refer to these as Badger transactions, not to be confused with the transactions included in the blocks delivered by CometBFT, the _application transactions_. + +First, let's create a new Badger transaction during `FinalizeBlock`. All application transactions in the current block will be executed within this Badger transaction. +Next, let's modify `FinalizeBlock` to add the `key` and `value` to the database transaction every time our application processes a new application transaction from the list received through `RequestFinalizeBlock`. + +Note that we check the validity of the transaction _again_ during `FinalizeBlock`. + +```go +func (app *KVStoreApplication) FinalizeBlock(_ context.Context, req *abcitypes.RequestFinalizeBlock) (*abcitypes.ResponseFinalizeBlock, error) { + var txs = make([]*abcitypes.ExecTxResult, len(req.Txs)) + + app.onGoingBlock = app.db.NewTransaction(true) + for i, tx := range req.Txs { + if code := app.isValid(tx); code != 0 { + log.Printf("Error in tx in if") + txs[i] = &abcitypes.ExecTxResult{Code: code} + } else { + parts := bytes.SplitN(tx, []byte("="), 2) + key, value := parts[0], parts[1] + log.Printf("Adding key %s with value %s", key, value) + + if err := app.onGoingBlock.Set(key, value); err != nil { + log.Panicf("Error writing to database, unable to execute tx: %v", err) + } + log.Printf("Successfully added key %s with value %s", key, value) + + txs[i] = &abcitypes.ExecTxResult{} + } + } + + return &abcitypes.ResponseFinalizeBlock{ + TxResults: txs, + }, nil +} +``` + +Transactions are not guaranteed to be valid when they are delivered to an application, even if they were valid when they were proposed. + +This can happen if the application state is used to determine transaction validity. The application state may have changed between the initial execution of `CheckTx` and the transaction delivery in `FinalizeBlock` in a way that rendered the transaction no longer valid. + +**Note** that `FinalizeBlock` cannot yet commit the Badger transaction we were building during the block execution. + +Other methods, such as `Query`, rely on a consistent view of the application's state; the application should only update its state by committing the Badger transactions when the full block has been delivered and the `Commit` method is invoked. + +The `Commit` method tells the application to make permanent the effects of the application transactions. +Let's update the method to terminate the pending Badger transaction and persist the resulting state: + +```go +func (app KVStoreApplication) Commit(_ context.Context, commit *abcitypes.RequestCommit) (*abcitypes.ResponseCommit, error) { + return &abcitypes.ResponseCommit{}, app.onGoingBlock.Commit() +} +``` + +Finally, make sure to add the log library to the `import` stanza as well: + +```go +import ( + "bytes" + "log" + + "github.com/dgraph-io/badger/v3" + abcitypes "github.com/cometbft/cometbft/abci/types" +) +``` + +You may have noticed that the application we are writing will crash if it receives +an unexpected error from the Badger database during the `FinalizeBlock` or `Commit` methods. +This is not an accident. If the application received an error from the database, there +is no deterministic way for it to make progress, so the only safe option is to terminate. + +### 1.3.4 Query + +When a client tries to read some information from the `kvstore`, the request will be +handled in the `Query` method. To do this, let's rewrite the `Query` method in `app.go`: + +```go +func (app *KVStoreApplication) Query(_ context.Context, req *abcitypes.RequestQuery) (*abcitypes.ResponseQuery, error) { + resp := abcitypes.ResponseQuery{Key: req.Data} + + dbErr := app.db.View(func(txn *badger.Txn) error { + item, err := txn.Get(req.Data) + if err != nil { + if err != badger.ErrKeyNotFound { + return err + } + resp.Log = "key does not exist" + return nil + } + + return item.Value(func(val []byte) error { + resp.Log = "exists" + resp.Value = val + return nil + }) + }) + if dbErr != nil { + log.Panicf("Error reading database, unable to execute query: %v", dbErr) + } + return &resp, nil +} +``` + +Since it reads only committed data from the store, transactions that are part of a block +that is being processed are not reflected in the query result. + +### 1.3.5 PrepareProposal and ProcessProposal + +`PrepareProposal` and `ProcessProposal` are methods introduced in CometBFT v0.37.0 +to give the application more control over the construction and processing of transaction blocks. + +When CometBFT sees that valid transactions (validated through `CheckTx`) are available to be +included in blocks, it groups some of these transactions and then gives the application a chance +to modify the group by invoking `PrepareProposal`. + +The application is free to modify the group before returning from the call, as long as the resulting set +does not use more bytes than `RequestPrepareProposal.max_tx_bytes`. +For example, the application may reorder, add, or even remove transactions from the group to improve the +execution of the block once accepted. + +In the following code, the application simply returns the unmodified group of transactions: + +```go +func (app *KVStoreApplication) PrepareProposal(_ context.Context, proposal *abcitypes.RequestPrepareProposal) (*abcitypes.ResponsePrepareProposal, error) { + return &abcitypes.ResponsePrepareProposal{Txs: proposal.Txs}, nil +} +``` + +Once a proposed block is received by a node, the proposal is passed to the +application to determine its validity before voting to accept the proposal. + +This mechanism may be used for different reasons, for example to deal with blocks manipulated +by malicious nodes, in which case the block should not be considered valid. + +The following code simply accepts all proposals: + +```go +func (app *KVStoreApplication) ProcessProposal(_ context.Context, proposal *abcitypes.RequestProcessProposal) (*abcitypes.ResponseProcessProposal, error) { + return &abcitypes.ResponseProcessProposal{Status: abcitypes.ResponseProcessProposal_ACCEPT}, nil +} +``` + +## 1.4 Starting an application and a CometBFT instance + +Now that we have the basic functionality of our application in place, let's put +it all together inside of our `main.go` file. + +Change the contents of your `main.go` file to the following. + +```go +package main + +import ( + "flag" + "fmt" + abciserver "github.com/cometbft/cometbft/abci/server" + "log" + "os" + "os/signal" + "path/filepath" + "syscall" + + "github.com/dgraph-io/badger/v3" + cmtlog "github.com/cometbft/cometbft/libs/log" +) + +var homeDir string +var socketAddr string + +func init() { + flag.StringVar(&homeDir, "kv-home", "", "Path to the kvstore directory (if empty, uses $HOME/.kvstore)") + flag.StringVar(&socketAddr, "socket-addr", "unix://example.sock", "Unix domain socket address (if empty, uses \"unix://example.sock\"") +} + +func main() { + flag.Parse() + if homeDir == "" { + homeDir = os.ExpandEnv("$HOME/.kvstore") + } + + dbPath := filepath.Join(homeDir, "badger") + db, err := badger.Open(badger.DefaultOptions(dbPath)) + if err != nil { + log.Fatalf("Opening database: %v", err) + } + + defer func() { + if err := db.Close(); err != nil { + log.Fatalf("Closing database: %v", err) + } + }() + + app := NewKVStoreApplication(db) + logger := cmtlog.NewTMLogger(cmtlog.NewSyncWriter(os.Stdout)) + + server := abciserver.NewSocketServer(socketAddr, app) + server.SetLogger(logger) + + if err := server.Start(); err != nil { + fmt.Fprintf(os.Stderr, "error starting socket server: %v", err) + + os.Exit(1) + } + defer server.Stop() + + c := make(chan os.Signal, 1) + signal.Notify(c, os.Interrupt, syscall.SIGTERM) + <-c +} +``` + +This is a large block of code, so let's break it down into pieces. + +First, we initialize the Badger database and create an app instance: + +```go +dbPath := filepath.Join(homeDir, "badger") +db, err := badger.Open(badger.DefaultOptions(dbPath)) +if err != nil { + log.Fatalf("Opening database: %v", err) +} +defer func() { + if err := db.Close(); err != nil { + log.Fatalf("Closing database: %v", err) + } +}() + +app := NewKVStoreApplication(db) +``` + +Then we start the ABCI server and add some signal handling to gracefully stop +it upon receiving SIGTERM or Ctrl-C. CometBFT will act as a client, +which connects to our server and sends us transactions and other messages. + +```go +server := abciserver.NewSocketServer(socketAddr, app) +server.SetLogger(logger) + +if err := server.Start(); err != nil { + fmt.Fprintf(os.Stderr, "error starting socket server: %v", err) + os.Exit(1) +} +defer server.Stop() + +c := make(chan os.Signal, 1) +signal.Notify(c, os.Interrupt, syscall.SIGTERM) +<-c +``` + +## 1.5 Initializing and Running + +Our application is almost ready to run, but first we'll need to populate the CometBFT configuration files. +The following command will create a `cometbft-home` directory in your project and add a basic set of configuration files in `cometbft-home/config/`. +For more information on what these files contain, see [the configuration documentation](https://github.com/cometbft/cometbft/blob/v0.38.x/docs/core/configuration.md). + +From the root of your project, run: + +```bash +go run github.com/cometbft/cometbft/cmd/cometbft@v0.38.0 init --home /tmp/cometbft-home +``` + +You should see an output similar to the following: + +```bash +I[2023-04-25|09:06:34.444] Generated private validator module=main keyFile=/tmp/cometbft-home/config/priv_validator_key.json stateFile=/tmp/cometbft-home/data/priv_validator_state.json +I[2023-04-25|09:06:34.444] Generated node key module=main path=/tmp/cometbft-home/config/node_key.json +I[2023-04-25|09:06:34.444] Generated genesis file module=main path=/tmp/cometbft-home/config/genesis.json +``` + +Now rebuild the app: + +```bash +go build -mod=mod # use -mod=mod to automatically refresh the dependencies +``` + +Everything is now in place to run your application. Run: + +```bash +./kvstore -kv-home /tmp/badger-home +``` + +The application will start, and you should see an output similar to the following: + +```bash +badger 2023-04-25 17:01:28 INFO: All 0 tables opened in 0s +badger 2023-04-25 17:01:28 INFO: Discard stats nextEmptySlot: 0 +badger 2023-04-25 17:01:28 INFO: Set nextTxnTs to 0 +I[2023-04-25|17:01:28.726] service start msg="Starting ABCIServer service" impl=ABCIServer +I[2023-04-25|17:01:28.726] Waiting for new connection... +``` + +Then we need to start the CometBFT service and point it to our application. +Open a new terminal window and cd to the same folder where the app is running. +Then execute the following command: + +```bash +go run github.com/cometbft/cometbft/cmd/cometbft@v0.38.0 node --home /tmp/cometbft-home --proxy_app=unix://example.sock +``` + +This should start the full node and connect to our ABCI application, which will be +reflected in the application output. + +```sh +I[2023-04-25|17:07:08.124] service start msg="Starting ABCIServer service" impl=ABCIServer +I[2023-04-25|17:07:08.124] Waiting for new connection... +I[2023-04-25|17:08:12.702] Accepted a new connection +I[2023-04-25|17:08:12.703] Waiting for new connection... +I[2023-04-25|17:08:12.703] Accepted a new connection +I[2023-04-25|17:08:12.703] Waiting for new connection... +``` + +Also, the application using CometBFT Core is producing blocks 🎉🎉 and you can see this reflected in the log output of the service in lines like this: + +```bash +I[2023-04-25|09:08:52.147] received proposal module=consensus proposal="Proposal{2/0 (F518444C0E348270436A73FD0F0B9DFEA758286BEB29482F1E3BEA75330E825C:1:C73D3D1273F2, -1) AD19AE292A45 @ 2023-04-25T12:08:52.143393Z}" +I[2023-04-25|09:08:52.147] received proposal module=consensus proposal="Proposal{2/0 (F518444C0E348270436A73FD0F0B9DFEA758286BEB29482F1E3BEA75330E825C:1:C73D3D1273F2, -1) AD19AE292A45 @ 2023-04-25T12:08:52.143393Z}" +I[2023-04-25|09:08:52.152] received complete proposal block module=consensus height=2 hash=F518444C0E348270436A73FD0F0B9DFEA758286BEB29482F1E3BEA75330E825C +I[2023-04-25|09:08:52.160] finalizing commit of block module=consensus height=2 hash=F518444C0E348270436A73FD0F0B9DFEA758286BEB29482F1E3BEA75330E825C root= num_txs=0 +I[2023-04-25|09:08:52.167] executed block module=state height=2 num_valid_txs=0 num_invalid_txs=0 +I[2023-04-25|09:08:52.171] committed state module=state height=2 num_txs=0 app_hash= +``` + +The blocks, as you can see from the `num_valid_txs=0` part, are empty, but let's remedy that next. + +## 1.6 Using the application + +Let's try submitting a transaction to our new application. +Open another terminal window and run the following curl command: + +```bash +curl -s 'localhost:26657/broadcast_tx_commit?tx="cometbft=rocks"' +``` + +If everything went well, you should see a response indicating which height the +transaction was included in the blockchain. + +Finally, let's make sure that transaction really was persisted by the application. +Run the following command: + +```bash +curl -s 'localhost:26657/abci_query?data="cometbft"' +``` + +Let's examine the response object that this request returns. +The request returns a `json` object with a `key` and `value` field set. + +```json +... + "key": "dGVuZGVybWludA==", + "value": "cm9ja3M=", +... +``` + +Those values don't look like the `key` and `value` we sent to CometBFT. +What's going on here? + +The response contains a `base64` encoded representation of the data we submitted. +To get the original value out of this data, we can use the `base64` command line utility: + +```bash +echo "cm9ja3M=" | base64 -d +``` + +## Outro + +We hope you were able to run everything smoothly. If you have any difficulties running through this tutorial, reach out to us via [discord](https://discord.com/invite/interchain) or open a new [issue](https://github.com/cometbft/cometbft/issues/new/choose) on Github. diff --git a/cometbft/v0.39/docs/guides/Install-CometBFT.mdx b/cometbft/v0.39/docs/guides/Install-CometBFT.mdx new file mode 100644 index 000000000..a21d30013 --- /dev/null +++ b/cometbft/v0.39/docs/guides/Install-CometBFT.mdx @@ -0,0 +1,125 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/docs/guides/Install-CometBFT' +title: Install CometBFT +order: 3 +--- + +## From Go Package + +Install the latest version of CometBFT's Go package: + +```sh +go install github.com/cometbft/cometbft/cmd/cometbft@latest +``` + +Install a specific version of CometBFT's Go package: + +```sh +go install github.com/cometbft/cometbft/cmd/cometbft@v0.38 +``` + +## From Binary + +To download pre-built binaries, see the [releases page](https://github.com/cometbft/cometbft/releases). + +## From Source + +You'll need `go` [installed](https://golang.org/doc/install) and the required +environment variables set, which can be done with the following commands: + +```sh +echo export GOPATH=\"\$HOME/go\" >> ~/.bash_profile +echo export PATH=\"\$PATH:\$GOPATH/bin\" >> ~/.bash_profile +``` + +### Get Source Code + +```sh +git clone https://github.com/cometbft/cometbft.git +cd cometbft +``` + +### Compile + +```sh +make install +``` + +to put the binary in `$GOPATH/bin`, or use: + +```sh +make build +``` + +to put the binary in `./build`. + +**DISCLAIMER:** The binary of CometBFT is built/installed without the DWARF +symbol table. If you would like to build/install CometBFT with the DWARF +symbol and debug information, remove `-s -w` from `BUILD_FLAGS` in the makefile. + +The latest CometBFT is now installed. You can verify the installation by +running: + +```sh +cometbft version +``` + +## Reinstall + +If you already have CometBFT installed and you make updates, simply run: + +```sh +make install +``` + +To upgrade, run: + +```sh +git pull origin main +make install +``` + +## Compile with CLevelDB Support + +Install [LevelDB](https://github.com/google/leveldb) (minimum version is 1.7). + +Install LevelDB with snappy (optional). Below are commands for Ubuntu: + +```sh +sudo apt-get update +sudo apt install build-essential + +sudo apt-get install libsnappy-dev + +wget https://github.com/google/leveldb/archive/v1.23.tar.gz && \ + tar -zxvf v1.23.tar.gz && \ + cd leveldb-1.23/ && \ + make && \ + sudo cp -r out-static/lib* out-shared/lib* /usr/local/lib/ && \ + cd include/ && \ + sudo cp -r leveldb /usr/local/include/ && \ + sudo ldconfig && \ + rm -f v1.23.tar.gz +``` + +Set a database backend to `cleveldb`: + +```toml +# config/config.toml +db_backend = "cleveldb" +``` + +To install CometBFT, run: + +```sh +CGO_LDFLAGS="-lsnappy" make install COMETBFT_BUILD_OPTIONS=cleveldb +``` + +or run: + +```sh +CGO_LDFLAGS="-lsnappy" make build COMETBFT_BUILD_OPTIONS=cleveldb +``` + +which puts the binary in `./build`. diff --git a/cometbft/v0.39/docs/guides/Quick-Start.mdx b/cometbft/v0.39/docs/guides/Quick-Start.mdx new file mode 100644 index 000000000..dee7a584c --- /dev/null +++ b/cometbft/v0.39/docs/guides/Quick-Start.mdx @@ -0,0 +1,165 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/docs/guides/Quick-Start' +title: Quick Start +order: 2 +--- + +## Overview + +This is a quick start guide. If you have a general idea about how CometBFT +works and want to get started right away, continue. + +## Install + +See the [install guide](/cometbft/v0.39/docs/guides/Install-CometBFT). + +## Initialization + +Running: + +```sh +cometbft init +``` + +will create the required files for a single, local node. + +These files are found in `$HOME/.cometbft`: + +```sh +$ ls $HOME/.cometbft + +config data + +$ ls $HOME/.cometbft/config/ + +config.toml genesis.json node_key.json priv_validator.json +``` + +For a single, local node, no further configuration is required. +Configuring a cluster is covered further below. + +## Local Node + +Start CometBFT with a simple in-process application: + +```sh +cometbft node --proxy_app=kvstore +``` + +> Note: `kvstore` is a non-persistent app. If you would like to run an application with persistence, run `--proxy_app=persistent_kvstore`. + +and blocks will start to stream in: + +```sh +I[01-06|01:45:15.592] Executed block module=state height=1 validTxs=0 invalidTxs=0 +I[01-06|01:45:15.624] Committed state module=state height=1 txs=0 appHash= +``` + +Check the status with: + +```sh +curl -s localhost:26657/status +``` + +### Sending Transactions + +With the KVstore app running, we can send transactions: + +```sh +curl -s 'localhost:26657/broadcast_tx_commit?tx="abcd"' +``` + +and check that it worked with: + +```sh +curl -s 'localhost:26657/abci_query?data="abcd"' +``` + +We can send transactions with a key and value too: + +```sh +curl -s 'localhost:26657/broadcast_tx_commit?tx="name=satoshi"' +``` + +and query the key: + +```sh +curl -s 'localhost:26657/abci_query?data="name"' +``` + +where the value is returned in hex. + +## Cluster of Nodes + +First, create four Ubuntu cloud machines. The following was tested on Digital +Ocean Ubuntu 16.04 x64 (3GB/1CPU, 20GB SSD). We'll refer to their respective IP +addresses below as IP1, IP2, IP3, IP4. + +Then, `ssh` into each machine and install CometBFT following the [instructions](/cometbft/v0.39/docs/guides/Install-CometBFT). + +Next, use the `cometbft testnet` command to create four directories of config files (found in `./mytestnet`) and copy each directory to the relevant machine in the cloud, so that each machine has a `$HOME/mytestnet/node[0-3]` directory. + +Before you can start the network, you'll need peer identifiers (IPs are not enough and can change). We'll refer to them as ID1, ID2, ID3, ID4. + +```sh +cometbft show_node_id --home ./mytestnet/node0 +cometbft show_node_id --home ./mytestnet/node1 +cometbft show_node_id --home ./mytestnet/node2 +cometbft show_node_id --home ./mytestnet/node3 +``` + +Here's a handy Bash script to compile the persistent peers string, which will +be needed for our next step: + +```bash +#!/bin/bash + +# Check if the required argument is provided +if [ $# -eq 0 ]; then + echo "Usage: $0 ..." + exit 1 +fi + +# Command to run on each IP +BASE_COMMAND="cometbft show_node_id --home ./mytestnet/node" + +# Initialize an array to store results +PERSISTENT_PEERS="" + +# Iterate through provided IPs +for i in "${!@}"; do + IP="${!i}" + NODE_IDX=$((i - 1)) # Adjust for zero-based indexing + + echo "Getting ID of $IP (node $NODE_IDX)..." + + # Run the command on the current IP and capture the result + ID=$($BASE_COMMAND$NODE_IDX) + + # Store the result in the array + PERSISTENT_PEERS+="$ID@$IP:26656" + + # Add a comma if not the last IP + if [ $i -lt $# ]; then + PERSISTENT_PEERS+="," + fi +done + +echo "$PERSISTENT_PEERS" +``` + +Finally, from each machine, run: + +```sh +cometbft node --home ./mytestnet/node0 --proxy_app=kvstore --p2p.persistent_peers="ID1@IP1:26656,ID2@IP2:26656,ID3@IP3:26656,ID4@IP4:26656" +cometbft node --home ./mytestnet/node1 --proxy_app=kvstore --p2p.persistent_peers="ID1@IP1:26656,ID2@IP2:26656,ID3@IP3:26656,ID4@IP4:26656" +cometbft node --home ./mytestnet/node2 --proxy_app=kvstore --p2p.persistent_peers="ID1@IP1:26656,ID2@IP2:26656,ID3@IP3:26656,ID4@IP4:26656" +cometbft node --home ./mytestnet/node3 --proxy_app=kvstore --p2p.persistent_peers="ID1@IP1:26656,ID2@IP2:26656,ID3@IP3:26656,ID4@IP4:26656" +``` + +Note that after the third node is started, blocks will start to stream in +because >2/3 of validators (defined in the `genesis.json`) have come online. +Persistent peers can also be specified in the `config.toml`. See [here](/cometbft/v0.39/docs/core/configuration) for more information about configuration options. + +Transactions can then be sent as covered in the single, local node example above. diff --git a/cometbft/v0.39/docs/guides/README.md b/cometbft/v0.39/docs/guides/README.md new file mode 100644 index 000000000..2336ec9d6 --- /dev/null +++ b/cometbft/v0.39/docs/guides/README.md @@ -0,0 +1,12 @@ +--- +order: false +parent: + order: 2 +--- + +# Guides + +- [Installing CometBFT](./Install-CometBFT.mdx) +- [Quick-start using CometBFT](./Quick-Start.mdx) +- [Creating a built-in application in Go](./Creating-a-built-in-application-in-Go.mdx) +- [Creating an external application in Go](./Creating-an-application-in-Go.mdx) diff --git a/cometbft/v0.39/docs/imgs/abci.png b/cometbft/v0.39/docs/imgs/abci.png new file mode 100644 index 000000000..73111cafd Binary files /dev/null and b/cometbft/v0.39/docs/imgs/abci.png differ diff --git a/cometbft/v0.39/docs/imgs/consensus_logic.png b/cometbft/v0.39/docs/imgs/consensus_logic.png new file mode 100644 index 000000000..22b70b265 Binary files /dev/null and b/cometbft/v0.39/docs/imgs/consensus_logic.png differ diff --git a/cometbft/v0.39/docs/imgs/contributing.png b/cometbft/v0.39/docs/imgs/contributing.png new file mode 100644 index 000000000..bb4bc6b5f Binary files /dev/null and b/cometbft/v0.39/docs/imgs/contributing.png differ diff --git a/cometbft/v0.39/docs/imgs/light_client_bisection_alg.png b/cometbft/v0.39/docs/imgs/light_client_bisection_alg.png new file mode 100644 index 000000000..a960ee69f Binary files /dev/null and b/cometbft/v0.39/docs/imgs/light_client_bisection_alg.png differ diff --git a/cometbft/v0.39/docs/imgs/sentry_layout.png b/cometbft/v0.39/docs/imgs/sentry_layout.png new file mode 100644 index 000000000..7d7dff44d Binary files /dev/null and b/cometbft/v0.39/docs/imgs/sentry_layout.png differ diff --git a/cometbft/v0.39/docs/imgs/sentry_local_config.png b/cometbft/v0.39/docs/imgs/sentry_local_config.png new file mode 100644 index 000000000..4fdb2fe58 Binary files /dev/null and b/cometbft/v0.39/docs/imgs/sentry_local_config.png differ diff --git a/cometbft/v0.39/docs/introduction/intro.mdx b/cometbft/v0.39/docs/introduction/intro.mdx new file mode 100644 index 000000000..22a3da7b9 --- /dev/null +++ b/cometbft/v0.39/docs/introduction/intro.mdx @@ -0,0 +1,335 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/docs/introduction/intro' +order: 1 +parent: + title: Introduction + order: 1 +--- +{/* trigger rebuild */} +{/* trigger rebuild */} + + +# What is CometBFT + +CometBFT is software for securely and consistently replicating an +application on many machines. By securely, we mean that CometBFT works +as long as fewer than 1/3 of machines fail in arbitrary ways. By consistently, +we mean that every non-faulty machine sees the same transaction log and +computes the same state. Secure and consistent replication is a +fundamental problem in distributed systems; it plays a critical role in +the fault tolerance of a broad range of applications, from currencies +to elections to infrastructure orchestration and beyond. + +The ability to tolerate machines failing in arbitrary ways, including +becoming malicious, is known as Byzantine fault tolerance (BFT). The +theory of BFT is decades old, but software implementations have only +become popular recently, due largely to the success of "blockchain +technology" like Bitcoin and Ethereum. Blockchain technology is just a +reformalization of BFT in a more modern setting, with emphasis on +peer-to-peer networking and cryptographic authentication. The name +derives from the way transactions are batched in blocks, where each +block contains a cryptographic hash of the previous one, forming a +chain. + +CometBFT consists of two chief technical components: a blockchain +consensus engine and a generic application interface. +The consensus engine, +which is based on the [Tendermint consensus algorithm][tendermint-paper], +ensures that the same transactions are +recorded on every machine in the same order. The application interface, +called the Application BlockChain Interface (ABCI), delivers the transactions +to applications for processing. Unlike other +blockchain and consensus solutions, which come pre-packaged with built-in +state machines (like a fancy key-value store or a quirky scripting +language), developers can use CometBFT for BFT state machine +replication of applications written in whatever programming language and +development environment is right for them. + +CometBFT is designed to be easy to use, simple to understand, highly +performant, and useful for a wide variety of distributed applications. + +## CometBFT vs. X + +CometBFT is broadly similar to two classes of software. The first +class consists of distributed key-value stores, like Zookeeper, etcd, +and Consul, which use non-BFT consensus. The second class is known as +"blockchain technology" and consists of both cryptocurrencies like +Bitcoin and Ethereum, and alternative distributed ledger designs like +Hyperledger's Burrow. + +### Zookeeper, etcd, Consul + +Zookeeper, etcd, and Consul are all implementations of key-value stores +atop a classical, non-BFT consensus algorithm. Zookeeper uses an +algorithm called Zookeeper Atomic Broadcast, while etcd and Consul use +the Raft log replication algorithm. A +typical cluster contains 3-5 machines and can tolerate crash failures +in fewer than 1/2 of the machines (e.g., 1 out of 3 or 2 out of 5), +but even a single Byzantine fault can jeopardize the whole system. + +Each offering provides a slightly different implementation of a +feature-rich key-value store, but all are generally focused on +providing basic services to distributed systems, such as dynamic +configuration, service discovery, locking, leader election, and so on. + +CometBFT is in essence similar software, but with two key differences: + +- It is Byzantine Fault Tolerant, meaning it can only tolerate fewer than 1/3 + of machines failing, but those failures can include arbitrary behavior— + including hacking and malicious attacks. +- It does not specify a + particular application, like a fancy key-value store. Instead, it + focuses on arbitrary state machine replication, so developers can build + the application logic that's right for them, from key-value stores to + cryptocurrency to e-voting platforms and beyond. + +### Bitcoin, Ethereum, etc. + +The [Tendermint consensus algorithm][tendermint-paper], adopted by CometBFT, +emerged in the tradition of cryptocurrencies like Bitcoin, +Ethereum, etc., with the goal of providing a more efficient and secure +consensus algorithm than Bitcoin's Proof of Work. In the early days, +Tendermint consensus-based blockchains had a simple currency built in, and to participate in +consensus, users had to "bond" units of the currency into a security +deposit which could be revoked if they misbehaved—this is what made +Tendermint consensus a Proof-of-Stake algorithm. + +Since then, CometBFT has evolved to be a general-purpose blockchain +consensus engine that can host arbitrary application states. That means +it can be used as a plug-and-play replacement for the consensus engines +of other blockchain software. So one can take the current Ethereum code +base, whether in Rust, Go, or Haskell, and run it as an ABCI +application using CometBFT. Indeed, [we did that with +Ethereum](https://github.com/cosmos/ethermint). And we plan to do +the same for Bitcoin, ZCash, and various other deterministic +applications as well. + +Another example of a cryptocurrency application built on CometBFT is +[the Cosmos network](http://cosmos.network). + +### Other Blockchain Projects + +[Fabric](https://github.com/hyperledger/fabric) takes a similar approach +to CometBFT, but is more opinionated about how the state is managed +and requires that all application behavior runs in potentially many +Docker containers, modules it calls "chaincode". It uses an +implementation of [PBFT](http://pmg.csail.mit.edu/papers/osdi99.pdf) +from a team at IBM that is [augmented to handle potentially +non-deterministic +chaincode](https://drops.dagstuhl.de/opus/volltexte/2017/7093/pdf/LIPIcs-OPODIS-2016-24.pdf). +It is possible to implement this Docker-based behavior as an ABCI app in +CometBFT, though extending CometBFT to handle non-determinism +remains for future work. + +[Burrow](https://github.com/hyperledger/burrow) is an implementation of +the Ethereum Virtual Machine and Ethereum transaction mechanics, with +additional features for a name registry, permissions, and native +contracts, and an alternative blockchain API. It uses CometBFT as its +consensus engine and provides a particular application state. + +## ABCI Overview + +The [Application BlockChain Interface +(ABCI)](https://github.com/cometbft/cometbft/tree/v0.38.x/abci) +allows for Byzantine Fault Tolerant replication of applications +written in any programming language. + +### Motivation + +Thus far, all blockchain "stacks" (such as +[Bitcoin](https://github.com/bitcoin/bitcoin)) have had a monolithic +design. That is, each blockchain stack is a single program that handles +all the concerns of a decentralized ledger; this includes P2P +connectivity, the "mempool" broadcasting of transactions, consensus on +the most recent block, account balances, Turing-complete contracts, +user-level permissions, etc. + +Using a monolithic architecture is typically bad practice in computer +science. It makes it difficult to reuse components of the code, and +attempts to do so result in complex maintenance procedures for forks of +the codebase. This is especially true when the codebase is not modular +in design and suffers from "spaghetti code". + +Another problem with monolithic design is that it limits you to the +language of the blockchain stack (or vice versa). In the case of +Ethereum, which supports a Turing-complete bytecode virtual machine, it +limits you to languages that compile down to that bytecode; while the +[list](https://github.com/pirapira/awesome-ethereum-virtual-machine#programming-languages-that-compile-into-evm) +is growing, it is still very limited. + +In contrast, our approach is to decouple the consensus engine and P2P +layers from the details of the state of the particular +blockchain application. We do this by abstracting away the details of +the application to an interface, which is implemented as a socket +protocol. + +### Intro to ABCI + +[CometBFT](https://github.com/cometbft/cometbft), the +"consensus engine", communicates with the application via a socket +protocol that satisfies the ABCI, the CometBFT Socket Protocol. + +To draw an analogy, let's talk about a well-known cryptocurrency, +Bitcoin. Bitcoin is a cryptocurrency blockchain where each node +maintains a fully audited Unspent Transaction Output (UTXO) database. If +one wanted to create a Bitcoin-like system on top of ABCI, CometBFT +would be responsible for + +- Sharing blocks and transactions between nodes +- Establishing a canonical/immutable order of transactions + (the blockchain) + +The application will be responsible for + +- Maintaining the UTXO database +- Validating cryptographic signatures of transactions +- Preventing transactions from spending non-existent transactions +- Allowing clients to query the UTXO database + +CometBFT is able to decompose the blockchain design by offering a very +simple API (i.e., the ABCI) between the application process and consensus +process. + +The ABCI consists of 3 primary message types that get delivered from the +core to the application. The application replies with corresponding +response messages. + +The messages are specified here: [ABCI Message +Types](https://github.com/cometbft/cometbft/blob/v0.38.x/proto/tendermint/abci/types.proto). + +The **FinalizeBlock** message is the workhorse of the application. Each +transaction in the blockchain is finalized within this message. The +application needs to validate each transaction received with the +**FinalizeBlock** message against the current state, application protocol, +and the cryptographic credentials of the transaction. FinalizeBlock only +prepares the update to be made and does not change the state of the application. +The state change is actually committed at a later stage, i.e., in the commit phase. + +The **CheckTx** message is used for validating transactions. +CometBFT's mempool first checks the +validity of a transaction with **CheckTx** and only relays valid +transactions to its peers. For instance, an application may check an +incrementing sequence number in the transaction and return an error upon +**CheckTx** if the sequence number is old. Alternatively, they might use +a capabilities-based system that requires capabilities to be renewed +with every transaction. + +The **Commit** message is used to compute a cryptographic commitment to +the current application state, to be placed into the next block header. +This has some handy properties. Inconsistencies in updating that state +will now appear as blockchain forks, which catches a whole class of +programming errors. This also simplifies the development of secure +lightweight clients, as Merkle-hash proofs can be verified by checking +against the block hash, and the block hash is signed by a quorum. + +There can be multiple ABCI socket connections to an application. +CometBFT creates four ABCI connections to the application: one +for the validation of transactions when broadcasting in the mempool, one for +the consensus engine to run block proposals, one for creating snapshots of the +application state, and one more for querying the application state. + +It's probably evident that application designers need to very carefully +design their message handlers to create a blockchain that does anything +useful, but this architecture provides a place to start. The diagram +below illustrates the flow of messages via ABCI. + +![abci](../imgs/abci.png) + +## A Note on Determinism + +The logic for blockchain transaction processing must be deterministic. +If the application logic weren't deterministic, consensus would not be +reached among the CometBFT replica nodes. + +Solidity on Ethereum is a great language of choice for blockchain +applications because, among other reasons, it is a completely +deterministic programming language. However, it's also possible to +create deterministic applications using existing popular languages like +Java, C++, Python, or Go by avoiding +sources of non-determinism such as: + +- random number generators (without deterministic seeding) +- race conditions on threads (or avoiding threads altogether) +- the system clock +- uninitialized memory (in unsafe programming languages like C + or C++) +- [floating point + arithmetic](http://gafferongames.com/networking-for-game-programmers/floating-point-determinism/) +- language features that are random (e.g., map iteration in Go) + +While programmers can avoid non-determinism by being careful, it is also +possible to create a special linter or static analyzer for each language +to check for determinism. In the future, we may work with partners to +create such tools. + +## Consensus Overview + +CometBFT adopts the [Tendermint consensus][tendermint-paper], +an easy-to-understand, mostly asynchronous, BFT consensus algorithm. +The algorithm follows a simple state machine that looks like this: + +![consensus-logic](../imgs/consensus_logic.png) + +Participants in the algorithm are called **validators**; they take turns +proposing blocks of transactions and voting on them. Blocks are +committed in a chain, with one block at each **height**. A block may +fail to be committed, in which case the algorithm moves to the next +**round**, and a new validator gets to propose a block for that height. +Two stages of voting are required to successfully commit a block; we +call them **pre-vote** and **pre-commit**. + +There is a picture of a couple doing the polka because validators are +doing something like a polka dance. When more than two-thirds of the +validators pre-vote for the same block, we call that a **polka**. Every +pre-commit must be justified by a polka in the same round. +A block is committed when +more than 2/3 of validators pre-commit for the same block in the same +round. + +Validators may fail to commit a block for a number of reasons; the +current proposer may be offline, or the network may be slow. Tendermint consensus +allows them to establish that a validator should be skipped. Validators +wait a small amount of time to receive a complete proposal block from +the proposer before voting to move to the next round. This reliance on a +timeout is what makes Tendermint consensus a weakly synchronous algorithm, rather +than an asynchronous one. However, the rest of the algorithm is +asynchronous, and validators only make progress after hearing from more +than two-thirds of the validator set. A simplifying element of +Tendermint consensus is that it uses the same mechanism to commit a block as it +does to skip to the next round. + +Assuming fewer than one-third of the validators are Byzantine, the Tendermint consensus algorithm +guarantees that safety will never be violated—that is, validators will +never commit conflicting blocks at the same height. To do this, it +introduces a few **locking** rules which modulate which paths can be +followed in the flow diagram. Once a validator precommits a block, it is +locked on that block. Then, + +1. it must prevote for the block it is locked on +2. it can only unlock and precommit for a new block if there is a + polka for that block in a later round + +## Stake + +In many systems, not all validators will have the same "weight" in the +consensus protocol. Thus, we are not so much interested in one-third or +two-thirds of the validators, but in those proportions of the total +voting power, which may not be uniformly distributed across individual +validators. + +Since CometBFT can replicate arbitrary applications, it is possible to +define a currency and denominate the voting power in that currency. +When voting power is denominated in a native currency, the system is +often referred to as Proof-of-Stake. Validators can be forced, by logic +in the application, to "bond" their currency holdings in a security +deposit that can be destroyed if they're found to misbehave in the +consensus protocol. This adds an economic element to the security of the +protocol, allowing one to quantify the cost of violating the assumption +that fewer than one-third of voting power is Byzantine. + +The [Cosmos Network](https://cosmos.network) is designed to use this +Proof-of-Stake mechanism across an array of cryptocurrencies implemented +as ABCI applications. + +[tendermint-paper]: https://arxiv.org/abs/1807.04938 diff --git a/cometbft/v0.39/docs/networks/Docker-Compose.mdx b/cometbft/v0.39/docs/networks/Docker-Compose.mdx new file mode 100644 index 000000000..6149a38dc --- /dev/null +++ b/cometbft/v0.39/docs/networks/Docker-Compose.mdx @@ -0,0 +1,180 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/docs/networks/Docker-Compose' +title: Docker Compose +order: 2 +--- + +With Docker Compose, you can spin up local testnets with a single command. + +## Requirements + +1. [Install CometBFT](/cometbft/v0.39/docs/guides/Install-CometBFT) +2. [Install docker](https://docs.docker.com/engine/installation/) +3. [Install docker-compose](https://docs.docker.com/compose/install/) + +## Build + +Build the `cometbft` binary and, optionally, the `cometbft/localnode` +docker image. + +Note the binary will be mounted into the container so it can be updated without +rebuilding the image. + +```sh +# Build the linux binary in ./build +make build-linux + +# (optionally) Build cometbft/localnode image +make build-docker-localnode +``` + +## Run a testnet + +To start a 4-node testnet, run: + +```sh +make localnet-start +``` + +The nodes bind their RPC servers to ports 26657, 26660, 26662, and 26664 on the +host. + +This command creates a 4-node network using the localnode image. + +The nodes of the network expose their P2P and RPC endpoints to the host machine +on ports 26656-26657, 26659-26660, 26661-26662, and 26663-26664 respectively. + +To update the binary, just rebuild it and restart the nodes: + +```sh +make build-linux +make localnet-start +``` + +## Configuration + +The `make localnet-start` command creates files for a 4-node testnet in `./build` by +calling the `cometbft testnet` command. + +The `./build` directory is mounted to the `/cometbft` mount point to attach +the binary and config files to the container. + +To change the number of validators / non-validators, change the `localnet-start` Makefile target [here](https://github.com/cometbft/cometbft-docs/blob/main/Makefile): + +```makefile +localnet-start: localnet-stop + @if ! [ -f build/node0/config/genesis.json ]; then docker run --rm -v $(CURDIR)/build:/cometbft:Z cometbft/localnode testnet --v 5 --n 3 --o . --populate-persistent-peers --starting-ip-address 192.167.10.2 ; fi + docker compose up -d +``` + +The command will now generate config files for 5 validators and 3 +non-validators. Along with generating new config files, the docker-compose file needs to be edited. +Adding 4 more nodes is required in order to fully utilize the config files that were generated. + +```yml + node3: # bump by 1 for every node + container_name: node3 # bump by 1 for every node + image: "cometbft/localnode" + environment: + - ID=3 + - LOG=${LOG:-cometbft.log} + ports: + - "26663-26664:26656-26657" # Bump 26663-26664 by one for every node + volumes: + - ./build:/cometbft:Z + networks: + localnet: + ipv4_address: 192.167.10.5 # bump the final digit by 1 for every node +``` + +Before running it, don't forget to clean up the old files: + +```sh +# Clear the build folder +rm -rf ./build/node* +``` + +## Configuring ABCI containers + +To use your own ABCI applications with the 4-node setup, edit the [docker-compose.yaml](https://github.com/cometbft/cometbft/blob/v0.38.x/docker-compose.yml) file and add images for your ABCI application. + +```yml + abci0: + container_name: abci0 + image: "abci-image" + build: + context: . + dockerfile: abci.Dockerfile + command: + networks: + localnet: + ipv4_address: 192.167.10.6 + + abci1: + container_name: abci1 + image: "abci-image" + build: + context: . + dockerfile: abci.Dockerfile + command: + networks: + localnet: + ipv4_address: 192.167.10.7 + + abci2: + container_name: abci2 + image: "abci-image" + build: + context: . + dockerfile: abci.Dockerfile + command: + networks: + localnet: + ipv4_address: 192.167.10.8 + + abci3: + container_name: abci3 + image: "abci-image" + build: + context: . + dockerfile: abci.Dockerfile + command: + networks: + localnet: + ipv4_address: 192.167.10.9 + +``` + +Override the [command](https://github.com/cometbft/cometbft/blob/v0.38.x/networks/local/localnode/Dockerfile#L11) in each node to connect to its ABCI. + +```yml + node0: + container_name: node0 + image: "cometbft/localnode" + ports: + - "26656-26657:26656-26657" + environment: + - ID=0 + - LOG=$${LOG:-cometbft.log} + volumes: + - ./build:/cometbft:Z + command: node --proxy_app=tcp://abci0:26658 + networks: + localnet: + ipv4_address: 192.167.10.2 +``` + +Similarly configure node1, node2, and node3, then [run the testnet](#run-a-testnet). + +## Logging + +The log is saved under the attached volume, in the `cometbft.log` file. If the +`LOG` environment variable is set to `stdout` at start, the log is not saved, +but printed to the screen. + +## Special binaries + +If you have multiple binaries with different names, you can specify which one +to run with the `BINARY` environment variable. The path of the binary is relative +to the attached volume. diff --git a/cometbft/v0.39/docs/presubmit.sh b/cometbft/v0.39/docs/presubmit.sh new file mode 100755 index 000000000..0dec7d32a --- /dev/null +++ b/cometbft/v0.39/docs/presubmit.sh @@ -0,0 +1,39 @@ +#!/bin/bash +# +# This script verifies that each document in the docs and architecture +# directory has a corresponding table-of-contents entry in its README file. +# +# This can be run manually from the command line. +# It is also run in CI via the docs-toc.yml workflow. +# +set -euo pipefail + +readonly base="$(dirname $0)" +cd "$base" + +readonly workdir="$(mktemp -d)" +trap "rm -fr -- '$workdir'" EXIT + +checktoc() { + local dir="$1" + local tag="$2"'-*-*' + local out="$workdir/${dir}.out.txt" + ( + cd "$dir" >/dev/null + find . -maxdepth 1 -type f -name "$tag" -not -exec grep -q "({})" README.md ';' -print + ) > "$out" + if [[ -s "$out" ]] ; then + echo "-- The following files in $dir lack a ToC entry: +" + cat "$out" + return 1 + fi +} + +err=0 + +# Verify that each RFC and ADR has a ToC entry in its README file. +checktoc architecture adr || ((err++)) +checktoc rfc rfc || ((err++)) + +exit $err diff --git a/cometbft/v0.39/docs/qa/CometBFT-QA-34.mdx b/cometbft/v0.39/docs/qa/CometBFT-QA-34.mdx new file mode 100644 index 000000000..38d583a9a --- /dev/null +++ b/cometbft/v0.39/docs/qa/CometBFT-QA-34.mdx @@ -0,0 +1,372 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/docs/qa/CometBFT-QA-34' +order: 1 +parent: + title: CometBFT QA Results v0.34.x + description: This is a report on the results obtained when running v0.34.x on testnets + order: 3 +--- + +# CometBFT QA Results v0.34.x + +## v0.34.x - From Tendermint Core to CometBFT + +This section reports on the QA process we followed before releasing the first `v0.34.x` version +from our CometBFT repository. + +The changes with respect to the last version of `v0.34.x` +(namely `v0.34.26`, released from the Informal Systems' Tendermint Core fork) +are minimal, and focus on rebranding our fork of Tendermint Core to CometBFT in places +where there is no substantial risk of breaking compatibility +with earlier Tendermint Core versions of `v0.34.x`. + +Indeed, CometBFT versions of `v0.34.x` (`v0.34.27` and subsequent) should fulfill +the following compatibility-related requirements: + +* Operators can easily upgrade a `v0.34.x` version of Tendermint Core to CometBFT. +* Upgrades from Tendermint Core to CometBFT can be uncoordinated for versions of the `v0.34.x` branch. +* Nodes running CometBFT must be interoperable with those running Tendermint Core in the same chain, + as long as all are running a `v0.34.x` version. + +These QA tests focus on the third bullet, whereas the first two bullets are tested using our _e2e tests_. + +It would be prohibitively time-consuming to test mixed networks of all combinations of existing `v0.34.x` +versions, combined with the CometBFT release candidate under test. +Therefore, our testing focuses on the last Tendermint Core version (`v0.34.26`) and the CometBFT release +candidate under test. + +We run the _200 node test_, but not the _rotating node test_. The effort of running the latter +is not justified given the amount and nature of the changes we are testing with respect to the +full QA cycle run previously on `v0.34.x`. +Since the changes to the system's logic are minimal, we are interested in these performance requirements: + +* The CometBFT release candidate under test performs similarly to Tendermint Core (i.e., the baseline) + * when used at scale (i.e., in a large network of CometBFT nodes) + * when used at scale in a mixed network (i.e., some nodes are running CometBFT + and others are running an older Tendermint Core version) + +Therefore, we carry out a complete run of the _200-node test_ on the following networks: + +* A homogeneous 200-node testnet, where all nodes are running the CometBFT release candidate under test. +* A mixed network where 1/2 (99 out of 200) of the nodes are running the CometBFT release candidate under test, + and the rest (101 out of 200) are running Tendermint Core `v0.34.26`. +* A mixed network where 1/3 (66 out of 200) of the nodes are running the CometBFT release candidate under test, + and the rest (134 out of 200) are running Tendermint Core `v0.34.26`. +* A mixed network where 2/3 (133 out of 200) of the nodes are running the CometBFT release candidate under test, + and the rest (67 out of 200) are running Tendermint Core `v0.34.26`. + +## Configuration and Results +In the following sections we provide the results of the _200 node test_. +Each section reports the baseline results (for reference), the homogeneous network scenario (all CometBFT nodes), +and the mixed networks with 1/2, 1/3, and 2/3 of Tendermint Core nodes. + +### Saturation Point + +As the CometBFT release candidate under test has minimal changes +with respect to Tendermint Core `v0.34.26`, other than the rebranding changes, +we can confidently reuse the results from the `v0.34.x` baseline test regarding +the [saturation point](/cometbft/v0.39/docs/qa/TMCore-QA-34#finding-the-saturation-point). + +Therefore, we will simply use a load of (`r=200,c=2`) +(see the explanation [here](/cometbft/v0.39/docs/qa/TMCore-QA-34#finding-the-saturation-point)) on all experiments. + +We also include the baseline results for quick reference and comparison. + +### Experiments + +On each of the three networks, the test consists of 4 experiments, with the goal of +ensuring the data obtained is consistent across experiments. + +On each of the networks, we pick only one representative run to present and discuss the +results. + + +## Examining Latencies +For each network, the figures plot the four experiments carried out with the network. +We can see that the latencies follow comparable patterns across all experiments. + +Unique identifiers (UUIDs) for each execution are presented on top of each graph. +We refer to these UUIDs to indicate the representative runs. + +### CometBFT Homogeneous Network + +![latencies](/cometbft/v0.39/docs/qa/img34/homogeneous/all_experiments.png) + +### 1/2 Tendermint Core - 1/2 CometBFT + +![latencies](/cometbft/v0.39/docs/qa/img34/cmt1tm1/all_experiments.png) + +### 1/3 Tendermint Core - 2/3 CometBFT + +![latencies](/cometbft/v0.39/docs/qa/img34/cmt2tm1/all_experiments.png) + +### 2/3 Tendermint Core - 1/3 CometBFT + +![latencies_all_tm2_3_cmt1_3](/cometbft/v0.39/docs/qa/img34/v034_200node_tm2cmt1/all_experiments.png) + + +## Prometheus Metrics + +This section reports on the key Prometheus metrics extracted from the following experiments: + +* Baseline results: `v0.34.x`, obtained in October 2022 and reported [here](/cometbft/v0.39/docs/qa/TMCore-QA-34). +* CometBFT homogeneous network: experiment with UUID starting with `be8c`. +* Mixed network, 1/2 Tendermint Core `v0.34.26` and 1/2 running CometBFT: experiment with UUID starting with `04ee`. +* Mixed network, 1/3 Tendermint Core `v0.34.26` and 2/3 running CometBFT: experiment with UUID starting with `fc5e`. +* Mixed network, 2/3 Tendermint Core `v0.34.26` and 1/3 running CometBFT: experiment with UUID starting with `4759`. + +We make explicit comparisons between the baseline and the homogeneous setups, but refrain from +commenting on the mixed network experiments unless they show some exceptional results. + +### Mempool Size + +For each reported experiment, we show two graphs. +The first shows the evolution over time of the cumulative number of transactions +inside all full nodes' mempools at a given time. + +The second one shows the evolution of the average over all full nodes. + +#### Baseline + +![mempool-cumulative](/cometbft/v0.39/docs/qa/img34/baseline/mempool_size.png) + +![mempool-avg](/cometbft/v0.39/docs/qa/img34/baseline/avg_mempool_size.png) + +#### CometBFT Homogeneous Network + +The results for the homogeneous network and the baseline are similar in terms of outstanding transactions. + +![mempool-cumulative-homogeneous](/cometbft/v0.39/docs/qa/img34/homogeneous/mempool_size.png) + +![mempool-avg-homogeneous](/cometbft/v0.39/docs/qa/img34/homogeneous/avg_mempool_size.png) + +#### 1/2 Tendermint Core - 1/2 CometBFT + +![mempool size](/cometbft/v0.39/docs/qa/img34/cmt1tm1/mempool_size.png) + +![average mempool size](/cometbft/v0.39/docs/qa/img34/cmt1tm1/avg_mempool_size.png) + +#### 1/3 Tendermint Core - 2/3 CometBFT + +![mempool size](/cometbft/v0.39/docs/qa/img34/cmt2tm1/mempool_size.png) + +![average mempool size](/cometbft/v0.39/docs/qa/img34/cmt2tm1/avg_mempool_size.png) + +#### 2/3 Tendermint Core - 1/3 CometBFT + +![mempool_tm2_3_cmt_1_3](/cometbft/v0.39/docs/qa/img34/v034_200node_tm2cmt1/mempool_size.png) + +![mempool-avg_tm2_3_cmt_1_3](/cometbft/v0.39/docs/qa/img34/v034_200node_tm2cmt1/avg_mempool_size.png) + +### Consensus Rounds per Height + +The following graphs show the rounds needed to complete each height and agree on a block. + +A value of `0` shows that only one round was required (with ID `0`), and a value of `1` shows that two rounds were required. + +#### Baseline +We can see that round 1 is reached with a certain frequency. + +![rounds](/cometbft/v0.39/docs/qa/img34/baseline/rounds.png) + +#### CometBFT Homogeneous Network + +Most heights finished in round 0. Some nodes needed to advance to round 1 at various moments, +and a few nodes even needed to advance to round 2 at one point. +This coincides with the time at which we observed the biggest peak in mempool size +on the corresponding plot shown above. + +![rounds-homogeneous](/cometbft/v0.39/docs/qa/img34/homogeneous/rounds.png) + +#### 1/2 Tendermint Core - 1/2 CometBFT + +![peers](/cometbft/v0.39/docs/qa/img34/cmt1tm1/rounds.png) + +#### 1/3 Tendermint Core - 2/3 CometBFT + +![peers](/cometbft/v0.39/docs/qa/img34/cmt2tm1/rounds.png) + +#### 2/3 Tendermint Core - 1/3 CometBFT + +![rounds-tm2_3_cmt1_3](/cometbft/v0.39/docs/qa/img34/v034_200node_tm2cmt1/rounds.png) + +### Peers + +The following plots show how many peers a node had throughout the experiment. + +The thick red dashed line represents the moving average over a sliding window of 20 seconds. + +#### Baseline + +The following graph shows that the number of peers was stable throughout the experiment. +Seed nodes typically have a higher number of peers. +The fact that non-seed nodes reach more than 50 peers is due to +[#9548](https://github.com/tendermint/tendermint/issues/9548). + +![peers](/cometbft/v0.39/docs/qa/img34/baseline/peers.png) + +#### CometBFT Homogeneous Network + +The results for the homogeneous network are very similar to the baseline. +The only difference is that the seed nodes seem to lose peers in the middle of the experiment. +However, this cannot be attributed to the differences in the code, which are mainly rebranding. + +![peers-homogeneous](/cometbft/v0.39/docs/qa/img34/homogeneous/peers.png) + +#### 1/2 Tendermint Core - 1/2 CometBFT + +![peers](/cometbft/v0.39/docs/qa/img34/cmt1tm1/peers.png) + +#### 1/3 Tendermint Core - 2/3 CometBFT + +![peers](/cometbft/v0.39/docs/qa/img34/cmt2tm1/peers.png) + +#### 2/3 Tendermint Core - 1/3 CometBFT + +As in the homogeneous case, there is some variation in the number of peers for some nodes. +These, however, do not affect the average. + +![peers-tm2_3_cmt1_3](/cometbft/v0.39/docs/qa/img34/v034_200node_tm2cmt1/peers.png) + +### Blocks Produced per Minute, Transactions Processed per Minute + +The following plots show the rate of block production and the rate of transactions delivered throughout the experiments. + +In both graphs, rates are calculated over a sliding window of 20 seconds. +The thick red dashed line shows the rates' moving averages. + +#### Baseline + +The average number of blocks per minute oscillates between 10 and 40. + +![heights](/cometbft/v0.39/docs/qa/img34/baseline/block_rate_regular.png) + +The number of transactions per minute tops around 30k. + +![total-txs](/cometbft/v0.39/docs/qa/img34/baseline/total_txs_rate_regular.png) + + +#### CometBFT Homogeneous Network + +The plot showing the block production rate shows that the rate oscillates around 20 blocks per minute, +mostly within the same range as the baseline. + +![heights-homogeneous-rate](/cometbft/v0.39/docs/qa/img34/homogeneous/block_rate_regular.png) + +The plot showing the transaction rate shows the rate stays around 20,000 transactions per minute, +also topping around 30k. + +![txs-homogeneous-rate](/cometbft/v0.39/docs/qa/img34/homogeneous/total_txs_rate_regular.png) + +#### 1/2 Tendermint Core - 1/2 CometBFT + +![height rate](/cometbft/v0.39/docs/qa/img34/cmt1tm1/block_rate_regular.png) + +![transaction rate](/cometbft/v0.39/docs/qa/img34/cmt1tm1/total_txs_rate_regular.png) + +#### 1/3 Tendermint Core - 2/3 CometBFT + +![height rate](/cometbft/v0.39/docs/qa/img34/cmt2tm1/block_rate_regular.png) + +![transaction rate](/cometbft/v0.39/docs/qa/img34/cmt2tm1/total_txs_rate_regular.png) + +#### 2/3 Tendermint Core - 1/3 CometBFT + +![height rate](/cometbft/v0.39/docs/qa/img34/v034_200node_tm2cmt1/block_rate_regular.png) + +![transaction rate](/cometbft/v0.39/docs/qa/img34/v034_200node_tm2cmt1/total_txs_rate_regular.png) + +### Memory Resident Set Size + +The following graphs show the Resident Set Size (RSS) of all monitored processes and the average value. + +#### Baseline + +![rss](/cometbft/v0.39/docs/qa/img34/baseline/memory.png) + +![rss-avg](/cometbft/v0.39/docs/qa/img34/baseline/avg_memory.png) + +#### CometBFT Homogeneous Network + +This is the plot for the homogeneous network, which is slightly more stable than the baseline over +the time of the experiment. + +![rss-homogeneous](/cometbft/v0.39/docs/qa/img34/homogeneous/memory.png) + +And this is the average plot. It oscillates around 560 MiB, which is noticeably lower than the baseline. + +![rss-avg-homogeneous](/cometbft/v0.39/docs/qa/img34/homogeneous/avg_memory.png) + +#### 1/2 Tendermint Core - 1/2 CometBFT + +![rss](/cometbft/v0.39/docs/qa/img34/cmt1tm1/memory.png) + +![rss average](/cometbft/v0.39/docs/qa/img34/cmt1tm1/avg_memory.png) + +#### 1/3 Tendermint Core - 2/3 CometBFT + +![rss](/cometbft/v0.39/docs/qa/img34/cmt2tm1/memory.png) + +![rss average](/cometbft/v0.39/docs/qa/img34/cmt2tm1/avg_memory.png) + +#### 2/3 Tendermint Core - 1/3 CometBFT + +![rss](/cometbft/v0.39/docs/qa/img34/v034_200node_tm2cmt1/memory.png) + +![rss average](/cometbft/v0.39/docs/qa/img34/v034_200node_tm2cmt1/avg_memory.png) + +### CPU Utilization + +The following graphs show the `load1` of nodes, as typically shown in the first line of the Unix `top` +command, and their average value. + +#### Baseline + +![load1](/cometbft/v0.39/docs/qa/img34/baseline/cpu.png) + +![load1-avg](/cometbft/v0.39/docs/qa/img34/baseline/avg_cpu.png) + +#### CometBFT Homogeneous Network + +The load in the homogeneous network is, similarly to the baseline case, below 5 and, therefore, normal. + +![load1-homogeneous](/cometbft/v0.39/docs/qa/img34/homogeneous/cpu.png) + +As expected, the average plot also looks similar. + +![load1-homogeneous-avg](/cometbft/v0.39/docs/qa/img34/homogeneous/avg_cpu.png) + +#### 1/2 Tendermint Core - 1/2 CometBFT + +![load1](/cometbft/v0.39/docs/qa/img34/cmt1tm1/cpu.png) + +![average load1](/cometbft/v0.39/docs/qa/img34/cmt1tm1/avg_cpu.png) + +#### 1/3 Tendermint Core - 2/3 CometBFT + +![load1](/cometbft/v0.39/docs/qa/img34/cmt2tm1/cpu.png) + +![average load1](/cometbft/v0.39/docs/qa/img34/cmt2tm1/avg_cpu.png) + +#### 2/3 Tendermint Core - 1/3 CometBFT + +![load1](/cometbft/v0.39/docs/qa/img34/v034_200node_tm2cmt1/cpu.png) + +![average load1](/cometbft/v0.39/docs/qa/img34/v034_200node_tm2cmt1/avg_cpu.png) + +## Test Results + +The comparison of the baseline results and the homogeneous case shows that both scenarios had similar numbers and are therefore equivalent. + +The mixed node cases show that networks operate normally with a mix of compatible Tendermint Core and CometBFT versions. +Although not the main goal, a comparison of metric numbers with the homogeneous case and the baseline scenarios shows similar results, and therefore we can conclude that mixing compatible Tendermint Core and CometBFT introduces no performance degradation. + +A conclusion of these tests is shown in the following table, along with the commit versions used in the experiments. + +| Scenario | Date | Version | Result | +|--|--|--|--| +| CometBFT Homogeneous Network | 2023-02-08 | 3b783434f26b0e87994e6a77c5411927aad9ce3f | Pass | +| 1/2 Tendermint Core
1/2 CometBFT | 2023-02-14 | CometBFT: 3b783434f26b0e87994e6a77c5411927aad9ce3f
Tendermint Core: 66c2cb63416e66bff08e11f9088e21a0ed142790 | Pass | +| 1/3 Tendermint Core
2/3 CometBFT | 2023-02-08 | CometBFT: 3b783434f26b0e87994e6a77c5411927aad9ce3f
Tendermint Core: 66c2cb63416e66bff08e11f9088e21a0ed142790 | Pass | +| 2/3 Tendermint Core
1/3 CometBFT | 2023-02-08 | CometBFT: 3b783434f26b0e87994e6a77c5411927aad9ce3f
Tendermint Core: 66c2cb63416e66bff08e11f9088e21a0ed142790 | Pass | diff --git a/cometbft/v0.39/docs/qa/CometBFT-QA-37.mdx b/cometbft/v0.39/docs/qa/CometBFT-QA-37.mdx new file mode 100644 index 000000000..690b9da0f --- /dev/null +++ b/cometbft/v0.39/docs/qa/CometBFT-QA-37.mdx @@ -0,0 +1,153 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/docs/qa/CometBFT-QA-37' +order: 1 +parent: + title: CometBFT QA Results v0.37.x + description: This is a report on the results obtained when running CometBFT v0.37.x on testnets + order: 5 +--- + +# CometBFT QA Results v0.37.x + +This iteration of the QA was run on CometBFT `v0.37.0-alpha3`, the first `v0.37.x` version from the CometBFT repository. + +The changes with respect to the baseline, `TM v0.37.x` as of Oct 12, 2022 (Commit: 1cf9d8e276afe8595cba960b51cd056514965fd1), include the rebranding of our fork of Tendermint Core to CometBFT and several improvements, described in the CometBFT [CHANGELOG](https://github.com/cometbft/cometbft/blob/v0.37.0-alpha.3/CHANGELOG.md). + +## Testbed + +As in other iterations of our QA process, we have used a 200-node network as a testbed, plus nodes to introduce load and collect metrics. + +### Saturation Point + +As in previous iterations, in our QA experiments, the system is subjected to a load slightly under a saturation point. +The method to identify the saturation point is explained [here](/cometbft/v0.39/docs/qa/TMCore-QA-34#finding-the-saturation-point) and its application to the baseline is described [here](/cometbft/v0.39/docs/qa/TMCore-QA-37#finding-the-saturation-point). +We use the same saturation point, that is, `c`, the number of connections created by the load runner process to the target node, is 2 and `r`, the rate or number of transactions issued per second, is 200. + +## Examining Latencies + +The following figure plots six experiments carried out with the network. +Unique identifiers (UUIDs) for each execution are presented on top of each graph. + +![latencies](/cometbft/v0.39/docs/qa/img37/200nodes_cmt037/all_experiments.png) + +We can see that the latencies follow comparable patterns across all experiments. +Therefore, in the following sections we will only present the results for one representative run, chosen randomly, with UUID starting with `75cb89a8`. + +![latencies](/cometbft/v0.39/docs/qa/img37/200nodes_cmt037/e_75cb89a8-f876-4698-82f3-8aaab0b361af.png) + +For reference, the following figure shows the latencies of different configurations of the baseline. +`c=02 r=200` corresponds to the same configuration as in this experiment. + +![all-latencies](/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_200node_latencies.png) + +As can be seen, latencies are similar. + +## Prometheus Metrics on the Chosen Experiment + +This section further examines key metrics for this experiment extracted from Prometheus data regarding the chosen experiment. + +### Mempool Size + +The mempool size, a count of the number of transactions in the mempool, was shown to be stable and homogeneous at all full nodes. +It did not exhibit any unconstrained growth. +The plot below shows the evolution over time of the cumulative number of transactions inside all full nodes' mempools at a given time. + +![mempoool-cumulative](/cometbft/v0.39/docs/qa/img37/200nodes_cmt037/mempool_size.png) + +The following picture shows the evolution of the average mempool size over all full nodes, which mostly oscillates between 1500 and 2000 outstanding transactions. + +![mempool-avg](/cometbft/v0.39/docs/qa/img37/200nodes_cmt037/avg_mempool_size.png) + +The peaks observed coincide with the moments when some nodes reached round 1 of consensus (see below). + +The behavior is similar to that observed in the baseline, presented next. + +![mempool-cumulative-baseline](/cometbft/v0.39/docs/qa/img37/200nodes_tm037/mempool_size.png) + +![mempool-avg-baseline](/cometbft/v0.39/docs/qa/img37/200nodes_tm037/avg_mempool_size.png) + +### Peers + +The number of peers was stable at all nodes. +It was higher for the seed nodes (around 140) than for the rest (between 16 and 78). +The red dashed line denotes the average value. + +![peers](/cometbft/v0.39/docs/qa/img37/200nodes_cmt037/peers.png) + +Just as in the baseline, shown next, the fact that non-seed nodes reach more than 50 peers is due to [\#9548]. + +![peers](/cometbft/v0.39/docs/qa/img37/200nodes_tm037/peers.png) + +### Consensus Rounds per Height + +Most heights took just one round, that is, round 0, but some nodes needed to advance to round 1 and eventually round 2. + +![rounds](/cometbft/v0.39/docs/qa/img37/200nodes_cmt037/rounds.png) + +The following specific run of the baseline presented better results, only requiring up to round 1, but reaching higher rounds is not uncommon in the corresponding software version. + +![rounds](/cometbft/v0.39/docs/qa/img37/200nodes_tm037/rounds.png) + +### Blocks Produced per Minute, Transactions Processed per Minute + +The following plot shows the rate at which blocks were created, from the point of view of each node. +That is, it shows when each node learned that a new block had been agreed upon. + +![heights](/cometbft/v0.39/docs/qa/img37/200nodes_cmt037/block_rate.png) + +For most of the time when load was being applied to the system, most of the nodes stayed around 20 to 25 blocks/minute. + +The spike to more than 175 blocks/minute is due to a slow node catching up. + +The collective spike on the right of the graph marks the end of the load injection, when blocks become smaller (empty) and impose less strain on the network. +This behavior is reflected in the following graph, which shows the number of transactions processed per minute. + +![total-txs](/cometbft/v0.39/docs/qa/img37/200nodes_cmt037/total_txs_rate.png) + +The baseline experienced a similar behavior, shown in the following two graphs. +The first depicts the block rate. + +![heights-baseline](/cometbft/v0.39/docs/qa/img37/200nodes_tm037/block_rate_regular.png) + +The second plots the transaction rate. + +![total-txs-baseline](/cometbft/v0.39/docs/qa/img37/200nodes_tm037/total_txs_rate_regular.png) + +### Memory Resident Set Size + +The Resident Set Size of all monitored processes is plotted below, with maximum memory usage of 2GB. + +![rss](/cometbft/v0.39/docs/qa/img37/200nodes_cmt037/memory.png) + +A similar behavior was shown in the baseline, presented next. + +![rss](/cometbft/v0.39/docs/qa/img37/200nodes_tm037/memory.png) + +The memory of all processes went down as the load was removed, showing no signs of unconstrained growth. + +#### CPU Utilization + +The best metric from Prometheus to gauge CPU utilization in a Unix machine is `load1`, +as it usually appears in the +[output of `top`](https://www.digitalocean.com/community/tutorials/load-average-in-linux). + +It is contained below 5 on most nodes, as seen in the following graph. + +![load1](/cometbft/v0.39/docs/qa/img37/200nodes_cmt037/cpu.png) + +A similar behavior was seen in the baseline. + +![load1-baseline](/cometbft/v0.39/docs/qa/img37/200nodes_tm037/cpu.png) + +## Test Results + +The comparison against the baseline results shows that both scenarios had similar numbers and are therefore equivalent. + +A conclusion of these tests is shown in the following table, along with the commit versions used in the experiments. + +| Scenario | Date | Version | Result | +|--|--|--|--| +|CometBFT | 2023-02-14 | v0.37.0-alpha3 (bef9a830e7ea7da30fa48f2cc236b1f465cc5833) | Pass + +[\#9548]: https://github.com/tendermint/tendermint/issues/9548 diff --git a/cometbft/v0.39/docs/qa/CometBFT-QA-38.mdx b/cometbft/v0.39/docs/qa/CometBFT-QA-38.mdx new file mode 100644 index 000000000..fde2873d0 --- /dev/null +++ b/cometbft/v0.39/docs/qa/CometBFT-QA-38.mdx @@ -0,0 +1,536 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/docs/qa/CometBFT-QA-38' +order: 1 +parent: + title: CometBFT QA Results v0.38.x + description: This is a report on the results obtained when running CometBFT v0.38.x on testnets + order: 5 +--- + +# CometBFT QA Results v0.38.x + +This iteration of the QA was run on CometBFT `v0.38.0-alpha.2`, the second +`v0.38.x` version from the CometBFT repository. + +The changes with respect to the baseline, `v0.37.0-alpha.3` from Feb 21, 2023, +include the introduction of the `FinalizeBlock` method to complete the full +range of ABCI++ functionality (ABCI 2.0), and several other improvements +described in the +[CHANGELOG](https://github.com/cometbft/cometbft/blob/v0.38.0-alpha.2/CHANGELOG.md). + +## Issues Discovered + +* (critical, fixed) [\#539] and [\#546] - This bug causes the proposer to crash in + `PrepareProposal` because it does not have extensions when it should. + This happens mainly when the proposer was catching up. +* (critical, fixed) [\#562] - There were several bugs in the metrics-related + logic that were causing panics when the testnets were started. + +## 200 Node Testnet + +As in other iterations of our QA process, we have used a 200-node network as a +testbed, plus nodes to introduce load and collect metrics. + +### Saturation Point + +As in previous iterations of our QA experiments, we first find the transaction +load at which the system begins to show degraded performance. Then we run the +experiments with the system subjected to a load slightly under the saturation +point. The method to identify the saturation point is explained +[here](/cometbft/v0.39/docs/qa/CometBFT-QA-34#saturation-point) and its application to the baseline +is described [here](/cometbft/v0.39/docs/qa/TMCore-QA-37#finding-the-saturation-point). + +The following table summarizes the results for the different experiments +(extracted from +[`v038_report_tabbed.txt`](https://raw.githubusercontent.com/cometbft/cometbft/v0.38.x/docs/qa/img38/200nodes/v038_report_tabbed.txt)). The X axis +(`c`) is the number of connections created by the load runner process to the +target node. The Y axis (`r`) is the rate or number of transactions issued per +second. + +| | c=1 | c=2 | c=4 | +| ------ | --------: | --------: | ----: | +| r=200 | 17800 | **33259** | 33259 | +| r=400 | **35600** | 41565 | 41384 | +| r=800 | 36831 | 38686 | 40816 | +| r=1600 | 40600 | 45034 | 39830 | + +We can observe in the table that the system is saturated beyond the diagonal +defined by the entries `c=1,r=400` and `c=2,r=200`. Entries in the diagonal have +the same amount of transaction load, so we can consider them equivalent. For the +chosen diagonal, the expected number of processed transactions is `1 * 400 tx/s * 89 s = 35600`. +(Note that we use 89 out of 90 seconds of the experiment because the last transaction batch +coincides with the end of the experiment and is thus not sent.) The experiments in the diagonal +below expect double that number, that is, `1 * 800 tx/s * 89 s = 71200`, but the +system is not able to process such a load, thus it is saturated. + +Therefore, for the rest of these experiments, we chose `c=1,r=400` as the +configuration. We could have chosen the equivalent `c=2,r=200`, which is the same +as used in our baseline version, but for simplicity we decided to use the one with +only one connection. + +Also note that, compared to the previous QA tests, we have tried to find the +saturation point within a higher range of load values for the rate `r`. In +particular, we ran tests with `r` equal to or above `200`, while in the previous +tests `r` was `200` or lower. In particular, for our baseline version we didn't +run the experiment on the configuration `c=1,r=400`. + +For comparison, this is the table for the baseline version, where the +saturation point is beyond the diagonal defined by `r=200,c=2` and `r=100,c=4`. + +| | c=1 | c=2 | c=4 | +| ----- | ----: | --------: | --------: | +| r=25 | 2225 | 4450 | 8900 | +| r=50 | 4450 | 8900 | 17800 | +| r=100 | 8900 | 17800 | **35600** | +| r=200 | 17800 | **35600** | 38660 | + +### Latencies + +The following figure plots the latencies of the experiment carried out with the +configuration `c=1,r=400`. + +![latency-1-400](/cometbft/v0.39/docs/qa/img38/200nodes/e_de676ecf-038e-443f-a26a-27915f29e312.png) + +For reference, the following figure shows the latencies of one of the +experiments for `c=2,r=200` in the baseline. + +![latency-2-200-37](/cometbft/v0.39/docs/qa/img37/200nodes_cmt037/e_75cb89a8-f876-4698-82f3-8aaab0b361af.png) + +As can be seen, in most cases the latencies are very similar, and in some cases, +the baseline has slightly higher latencies than the version under test. Thus, +from this small experiment, we can say that the latencies measured for the two +versions are equivalent, or at least that the version under test is not worse +than the baseline. + +### Prometheus Metrics on the Chosen Experiment + +This section further examines key metrics for this experiment extracted from +Prometheus data regarding the chosen experiment with configuration `c=1,r=400`. + +#### Mempool Size + +The mempool size, a count of the number of transactions in the mempool, was +shown to be stable and homogeneous at all full nodes. It did not exhibit any +unconstrained growth. The plot below shows the evolution over time of the +cumulative number of transactions inside all full nodes' mempools at a given +time. + +![mempool-cumulative](/cometbft/v0.39/docs/qa/img38/200nodes/mempool_size.png) + +The following picture shows the evolution of the average mempool size over all +full nodes, which mostly oscillates between 1000 and 2500 outstanding +transactions. + +![mempool-avg](/cometbft/v0.39/docs/qa/img38/200nodes/avg_mempool_size.png) + +The peaks observed coincide with the moments when some nodes reached round 1 of +consensus (see below). + +The behavior is similar to that observed in the baseline, presented next. + +![mempool-cumulative-baseline](/cometbft/v0.39/docs/qa/img37/200nodes_cmt037/mempool_size.png) + +![mempool-avg-baseline](/cometbft/v0.39/docs/qa/img37/200nodes_cmt037/avg_mempool_size.png) + +#### Peers + +The number of peers was stable at all nodes. It was higher for the seed nodes +(around 140) than for the rest (between 20 and 70 for most nodes). The red +dashed line denotes the average value. + +![peers](/cometbft/v0.39/docs/qa/img38/200nodes/peers.png) + +Just as in the baseline, shown next, the fact that non-seed nodes reach more +than 50 peers is due to [\#9548]. + +![peers](/cometbft/v0.39/docs/qa/img37/200nodes_cmt037/peers.png) + +#### Consensus Rounds per Height + +Most heights took just one round, that is, round 0, but some nodes needed to +advance to round 1. + +![rounds](/cometbft/v0.39/docs/qa/img38/200nodes/rounds.png) + +The following specific run of the baseline required some nodes to reach round 1. + +![rounds](/cometbft/v0.39/docs/qa/img37/200nodes_cmt037/rounds.png) + +#### Blocks Produced per Minute, Transactions Processed per Minute + +The following plot shows the rate at which blocks were created, from the point +of view of each node. That is, it shows when each node learned that a new block +had been agreed upon. + +![heights](/cometbft/v0.39/docs/qa/img38/200nodes/block_rate.png) + +For most of the time when load was being applied to the system, most of the +nodes stayed around 20 blocks/minute. + +The spike to more than 100 blocks/minute is due to a slow node catching up. + +The baseline experienced similar behavior. + +![heights-baseline](/cometbft/v0.39/docs/qa/img37/200nodes_cmt037/block_rate.png) + +The collective spike on the right of the graph marks the end of the load +injection, when blocks become smaller (empty) and impose less strain on the +network. This behavior is reflected in the following graph, which shows the +number of transactions processed per minute. + +![total-txs](/cometbft/v0.39/docs/qa/img38/200nodes/total_txs_rate.png) + +The following is the transaction processing rate of the baseline, which is +similar to the above. + +![total-txs-baseline](/cometbft/v0.39/docs/qa/img37/200nodes_cmt037/total_txs_rate.png) + +#### Memory Resident Set Size + +The following graph shows the Resident Set Size of all monitored processes, with +maximum memory usage of 1.6GB, slightly lower than the baseline shown after. + +![rss](/cometbft/v0.39/docs/qa/img38/200nodes/memory.png) + +Similar behavior was shown in the baseline, with even slightly higher memory +usage. + +![rss](/cometbft/v0.39/docs/qa/img37/200nodes_cmt037/memory.png) + +The memory of all processes went down as the load was removed, showing no signs +of unconstrained growth. + +#### CPU Utilization + +##### Comparison to Baseline + +The best metric from Prometheus to gauge CPU utilization on a Unix machine is +`load1`, as it usually appears in the [output of +`top`](https://www.digitalocean.com/community/tutorials/load-average-in-linux). + +The load is contained below 5 on most nodes, as seen in the following graph. + +![load1](/cometbft/v0.39/docs/qa/img38/200nodes/cpu.png) + +The baseline had similar behavior. + +![load1-baseline](/cometbft/v0.39/docs/qa/img37/200nodes_cmt037/cpu.png) + +##### Impact of Vote Extension Signature Verification + +It is important to note that the baseline (`v0.37.x`) does not implement vote extensions, +whereas the version under test (`v0.38.0-alpha.2`) _does_ implement them, and they are +configured to be activated since height 1. +The e2e application used in these tests verifies all received vote extension signatures (up to 175) +twice per height: upon `PrepareProposal` (for sanity) and upon `ProcessProposal` (to demonstrate how +real applications can do it). + +The fact that there is no noticeable difference in the CPU utilization plots of +the baseline and `v0.38.0-alpha.2` means that re-verifying up to 175 vote extension signatures twice +(besides the initial verification done by CometBFT when receiving them from the network) +has no performance impact in the current version of the system: the bottlenecks are elsewhere. +Thus, we should focus on optimizing other parts of the system: the ones that cause the current +bottlenecks (mempool gossip duplication, leaner proposal structure, optimized consensus gossip). + +### Test Results + +The comparison against the baseline results shows that both scenarios had similar +numbers and are therefore equivalent. + +A summary of these tests is shown in the following table, along with the +commit versions used in the experiments. + +| Scenario | Date | Version | Result | +| -------- | ---------- | ---------------------------------------------------------- | ------ | +| 200-node | 2023-05-21 | v0.38.0-alpha.2 (1f524d12996204f8fd9d41aa5aca215f80f06f5e) | Pass | + +## Rotating Node Testnet + +We use `c=1,r=400` as load, which can be considered a safe workload, as it was close to (but below) +the saturation point in the 200 node testnet. This testnet has fewer nodes (10 validators and 25 full nodes). + +Importantly, the baseline considered in this section is `v0.37.0-alpha.2` (Tendermint Core), +which is **different** from the one used in the [previous section](/cometbft/v0.39/docs/qa/Method#200-node-testnet). +The reason is that this testnet was not re-tested for `v0.37.0-alpha.3` (CometBFT), +since it was not deemed necessary. + +Unlike in the baseline tests, the version of CometBFT used for these tests is _not_ affected by [\#9539], +which was fixed right after having run the rotating testnet for `v0.37`. +As a result, the load introduced in this iteration of the test is higher as transactions do not get rejected. + +### Latencies + +The plot of all latencies can be seen here. + +![rotating-all-latencies](/cometbft/v0.39/docs/qa/img38/rotating/rotating_latencies.png) + +This is similar to the baseline. + +![rotating-all-latencies](/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_rotating_latencies.png) + +The average increase of about 1 second with respect to the baseline is due to the higher +transaction load produced (remember the baseline was affected by [\#9539], whereby most transactions +produced were rejected by `CheckTx`). + +### Prometheus Metrics + +The set of metrics shown here roughly matches those shown for the baseline (`v0.37`) for the same experiment. +We also show the baseline results for comparison. + +#### Blocks and Transactions per Minute + +The following plot shows the blocks produced per minute. + +![rotating-heights](/cometbft/v0.39/docs/qa/img38/rotating/rotating_block_rate.png) + +This is similar to the baseline, shown below. + +![rotating-heights-bl](/cometbft/v0.39/docs/qa/img37/rotating/rotating_block_rate.png) + +The following plot shows only the heights reported by ephemeral nodes, both when they were blocksyncing +and when they were running consensus. +The second plot is the baseline plot for comparison. The baseline lacks the heights when the nodes were +blocksyncing as that metric was implemented afterwards. + +![rotating-heights-ephe](/cometbft/v0.39/docs/qa/img38/rotating/rotating_eph_heights.png) + +![rotating-heights-ephe-bl](/cometbft/v0.39/docs/qa/img37/rotating/rotating_eph_heights.png) + +We see that heights follow a similar pattern in both plots: they grow in length as the experiment advances. + +The following plot shows the transactions processed per minute. + +![rotating-total-txs](/cometbft/v0.39/docs/qa/img38/rotating/rotating_txs_rate.png) + +For comparison, this is the baseline plot. + +![rotating-total-txs-bl](/cometbft/v0.39/docs/qa/img37/rotating/rotating_txs_rate.png) + +We can see the rate is much lower in the baseline plot. +The reason is that the baseline was affected by [\#9539], whereby `CheckTx` rejected most transactions +produced by the load runner. + +#### Peers + +The plot below shows the evolution of the number of peers throughout the experiment. + +![rotating-peers](/cometbft/v0.39/docs/qa/img38/rotating/rotating_peers.png) + +This is the baseline plot, for comparison. + +![rotating-peers-bl](/cometbft/v0.39/docs/qa/img37/rotating/rotating_peers.png) + +The plotted values and their evolution are comparable in both plots. + +For further details on these plots, see [this section](/cometbft/v0.39/docs/qa/TMCore-QA-34#peers-2). + +#### Memory Resident Set Size + +The average Resident Set Size (RSS) over all processes is notably larger on `v0.38.0-alpha.2` than on the baseline. +The reason for this is, again, the fact that `CheckTx` was rejecting most transactions submitted on the baseline +and therefore the overall transaction load was lower on the baseline. +This is consistent with the difference seen in the transaction rate plots +in the [previous section](#blocks-and-transactions-per-minute). + +![rotating-rss-avg](/cometbft/v0.39/docs/qa/img38/rotating/rotating_avg_memory.png) + +![rotating-rss-avg-bl](/cometbft/v0.39/docs/qa/img37/rotating/rotating_avg_memory.png) + +#### CPU Utilization + +The plots show metric `load1` for all nodes for `v0.38.0-alpha.2` and for the baseline. + +![rotating-load1](/cometbft/v0.39/docs/qa/img38/rotating/rotating_cpu.png) + +![rotating-load1-bl](/cometbft/v0.39/docs/qa/img37/rotating/rotating_cpu.png) + +In both cases, it is contained under 5 most of the time, which is considered normal load. +The load seems to be more significant on `v0.38.0-alpha.2` on average because of the larger +number of transactions processed per minute as compared to the baseline. + +### Test Result + +| Scenario | Date | Version | Result | +| -------- | ---------- | ---------------------------------------------------------- | ------ | +| Rotating | 2023-05-23 | v0.38.0-alpha.2 (e9abb116e29beb830cf111b824c8e2174d538838) | Pass | + +## Vote Extensions Testbed + +In this testnet we evaluate the effect of varying the sizes of vote extensions added to pre-commit votes on the performance of CometBFT. +The test uses the Key/Value store in our [end-to-end] test framework, which has the following simplified flow: + +1. When validators send their pre-commit votes for a block at height $i$, they first extend the vote as they see fit in `ExtendVote`. +2. When a proposer for height $i+1$ creates a block to propose, in `PrepareProposal`, it prepends the transactions with a special transaction, which modifies a reserved key. The transaction value is derived from the extensions from height $i$; in this example, the value is derived from the vote extensions and includes the set itself, hex encoded as a string. +3. When a validator sends their pre-vote for the block proposed in $i+1$, they first double check in `ProcessProposal` that the special transaction in the block was properly built by the proposer. +4. When validators send their pre-commit for the block proposed in $i+1$, they first extend the vote, and the steps repeat for heights $i+2$ and so on. + +For this test, extensions are random sequences of bytes with a predefined `vote_extension_size`. +Hence, two effects are seen on the network. +First, pre-commit vote message sizes will increase by the specified `vote_extension_size` and, second, block messages will increase by twice `vote_extension_size`, given the hex encoding of extensions, times the number of extensions received, i.e., at least 2/3 of 175. + +All tests were performed on commit d5baba237ab3a04c1fd4a7b10927ba2e6a2aab27, which corresponds to v0.38.0-alpha.2 plus commits to add the ability to vary the vote extension sizes to the test application. +Although the same commit is used for the baseline, in this configuration the behavior observed is the same as in the "vanilla" v0.38.0-alpha.2 test application, that is, vote extensions are 8-byte integers, compressed as variable-size integers instead of a random sequence of size `vote_extension_size`. + +The following table summarizes the test cases. + +| Name | Extension Size (bytes) | Date | +| -------- | ---------------------- | ---------- | +| baseline | 8 (varint) | 2023-05-26 | +| 2k | 2048 | 2023-05-29 | +| 4k | 4094 | 2023-05-29 | +| 8k | 8192 | 2023-05-26 | +| 16k | 16384 | 2023-05-26 | +| 32k | 32768 | 2023-05-26 | + +### Latency + +The following figures show the latencies observed in each of the 5 runs of each experiment; +the red line shows the average of each run. +It can be easily seen from these graphs that the larger the vote extension size, the more latency varies and the more common higher latencies become. +Even in the case of extensions of size 2k, the mean latency goes from below 5s to nearly 10s. + +**Baseline** + +![](/cometbft/v0.39/docs/qa/img38/voteExtensions/all_experiments_baseline.png) + +**2k** + +![](/cometbft/v0.39/docs/qa/img38/voteExtensions/all_experiments_2k.png) + +**4k** + +![](/cometbft/v0.39/docs/qa/img38/voteExtensions/all_experiments_4k.png) + +**8k** + +![](/cometbft/v0.39/docs/qa/img38/voteExtensions/all_experiments_8k.png) + +**16k** + +![](/cometbft/v0.39/docs/qa/img38/voteExtensions/all_experiments_16k.png) + +**32k** + +![](/cometbft/v0.39/docs/qa/img38/voteExtensions/all_experiments_32k.png) + +The following graphs combine all the runs of the same experiment. +They show that latency variation greatly increases with the increase of vote extensions. +In particular, for the 16k and 32k cases, the system goes through large gaps without transaction delivery. +As discussed later, this is the result of heights taking multiple rounds to finish and new transactions being held until the next block is agreed upon. + +| | | +| ---------------------------------------------------------- | ------------------------------------------------ | +| baseline ![](/cometbft/v0.39/docs/qa/img38/voteExtensions/all_c1r400_baseline.png) | 2k ![](/cometbft/v0.39/docs/qa/img38/voteExtensions/all_c1r400_2k.png) | +| 4k ![](/cometbft/v0.39/docs/qa/img38/voteExtensions/all_c1r400_4k.png) | 8k ![](/cometbft/v0.39/docs/qa/img38/voteExtensions/all_c1r400_8k.png) | +| 16k ![](/cometbft/v0.39/docs/qa/img38/voteExtensions/all_c1r400_16k.png) | 32k ![](/cometbft/v0.39/docs/qa/img38/voteExtensions/all_c1r400_32k.png) | + +### Blocks and Transactions per Minute + +The following plots show the blocks produced per minute and transactions processed per minute. +We have divided the presentation into an overview section, which shows the metrics for the whole experiment (five runs) and a detailed sample, which shows the metrics for the first of the five runs. +We repeat the approach for the other metrics as well. +The dashed red line shows the moving average over a 20s window. + +#### Overview + +It is clear from the overview plots that as the vote extension sizes increase, the rate of block creation decreases. +Although the rate of transaction processing also decreases, it does not seem to decrease as fast. + +| Experiment | Block creation rate | Transaction rate | +| ------------ | ----------------------------------------------------------- | ------------------------------------------------------------- | +| **baseline** | ![block rate](/cometbft/v0.39/docs/qa/img38/voteExtensions/baseline_block_rate.png) | ![txs rate](/cometbft/v0.39/docs/qa/img38/voteExtensions/baseline_total_txs_rate.png) | +| **2k** | ![block rate](/cometbft/v0.39/docs/qa/img38/voteExtensions/02k_block_rate.png) | ![txs rate](/cometbft/v0.39/docs/qa/img38/voteExtensions/02k_total_txs_rate.png) | +| **4k** | ![block rate](/cometbft/v0.39/docs/qa/img38/voteExtensions/04k_block_rate.png) | ![txs rate](/cometbft/v0.39/docs/qa/img38/voteExtensions/04k_total_txs_rate.png) | +| **8k** | ![block rate](/cometbft/v0.39/docs/qa/img38/voteExtensions/8k_block_rate.png) | ![txs rate](/cometbft/v0.39/docs/qa/img38/voteExtensions/08k_total_txs_rate.png) | +| **16k** | ![block rate](/cometbft/v0.39/docs/qa/img38/voteExtensions/16k_block_rate.png) | ![txs rate](/cometbft/v0.39/docs/qa/img38/voteExtensions/16k_total_txs_rate.png) | +| **32k** | ![block rate](/cometbft/v0.39/docs/qa/img38/voteExtensions/32k_block_rate.png) | ![txs rate](/cometbft/v0.39/docs/qa/img38/voteExtensions/32k_total_txs_rate.png) | + +#### First Run + +| Experiment | Block creation rate | Transaction rate | +| ------------ | ------------------------------------------------------------- | --------------------------------------------------------------- | +| **baseline** | ![block rate](/cometbft/v0.39/docs/qa/img38/voteExtensions/baseline_1_block_rate.png) | ![txs rate](/cometbft/v0.39/docs/qa/img38/voteExtensions/baseline_1_total_txs_rate.png) | +| **2k** | ![block rate](/cometbft/v0.39/docs/qa/img38/voteExtensions/02k_1_block_rate.png) | ![txs rate](/cometbft/v0.39/docs/qa/img38/voteExtensions/02k_1_total_txs_rate.png) | +| **4k** | ![block rate](/cometbft/v0.39/docs/qa/img38/voteExtensions/04k_1_block_rate.png) | ![txs rate](/cometbft/v0.39/docs/qa/img38/voteExtensions/04k_1_total_txs_rate.png) | +| **8k** | ![block rate](/cometbft/v0.39/docs/qa/img38/voteExtensions/08k_1_block_rate.png) | ![txs rate](/cometbft/v0.39/docs/qa/img38/voteExtensions/08k_1_total_txs_rate.png) | +| **16k** | ![block rate](/cometbft/v0.39/docs/qa/img38/voteExtensions/16k_1_block_rate.png) | ![txs rate](/cometbft/v0.39/docs/qa/img38/voteExtensions/16k_1_total_txs_rate.png) | +| **32k** | ![block rate](/cometbft/v0.39/docs/qa/img38/voteExtensions/32k_1_block_rate.png) | ![txs rate](/cometbft/v0.39/docs/qa/img38/voteExtensions/32k_1_total_txs_rate.png) | + +### Number of Rounds + +The effect of vote extensions is also felt in the number of rounds needed to reach consensus. +The following graphs show the number of the highest round required to reach consensus during the whole experiment. + +In the baseline and low vote extension lengths, most blocks were agreed upon during round 0. +As the load increases, more and more rounds were required. +In the 32k case we see round 5 being reached frequently. + +| Experiment | Number of Rounds per Block | +| ------------ | ------------------------------------------------------------- | +| **baseline** | ![number of rounds](/cometbft/v0.39/docs/qa/img38/voteExtensions/baseline_rounds.png) | +| **2k** | ![number of rounds](/cometbft/v0.39/docs/qa/img38/voteExtensions/02k_rounds.png) | +| **4k** | ![number of rounds](/cometbft/v0.39/docs/qa/img38/voteExtensions/04k_rounds.png) | +| **8k** | ![number of rounds](/cometbft/v0.39/docs/qa/img38/voteExtensions/08k_rounds.png) | +| **16k** | ![number of rounds](/cometbft/v0.39/docs/qa/img38/voteExtensions/16k_rounds.png) | +| **32k** | ![number of rounds](/cometbft/v0.39/docs/qa/img38/voteExtensions/32k_rounds.png) | + +We conjecture that the reason is that the timeouts used are inadequate for the extra traffic in the network. + +### CPU + +The CPU usage reached the same peaks in all tests, but the following graphs show that with larger vote extensions, nodes take longer to reduce the CPU usage. +This could mean that a backlog of processing is forming during the execution of the tests with larger extensions. + +| Experiment | CPU | +| ------------ | ----------------------------------------------------- | +| **baseline** | ![cpu-avg](/cometbft/v0.39/docs/qa/img38/voteExtensions/baseline_avg_cpu.png) | +| **2k** | ![cpu-avg](/cometbft/v0.39/docs/qa/img38/voteExtensions/02k_avg_cpu.png) | +| **4k** | ![cpu-avg](/cometbft/v0.39/docs/qa/img38/voteExtensions/04k_avg_cpu.png) | +| **8k** | ![cpu-avg](/cometbft/v0.39/docs/qa/img38/voteExtensions/08k_avg_cpu.png) | +| **16k** | ![cpu-avg](/cometbft/v0.39/docs/qa/img38/voteExtensions/16k_avg_cpu.png) | +| **32k** | ![cpu-avg](/cometbft/v0.39/docs/qa/img38/voteExtensions/32k_avg_cpu.png) | + +### Resident Memory + +The same conclusion reached for CPU usage may be drawn for the memory. +That is, a backlog of work is formed during the tests and catching up (freeing of memory) happens after the test is done. + +A more worrying trend is that the bottom of the memory usage seems to increase between runs. +We have investigated this in longer runs and confirmed that there is no such trend. + +| Experiment | Resident Set Size | +| ------------ | -------------------------------------------------------- | +| **baseline** | ![rss-avg](/cometbft/v0.39/docs/qa/img38/voteExtensions/baseline_avg_memory.png) | +| **2k** | ![rss-avg](/cometbft/v0.39/docs/qa/img38/voteExtensions/02k_avg_memory.png) | +| **4k** | ![rss-avg](/cometbft/v0.39/docs/qa/img38/voteExtensions/04k_avg_memory.png) | +| **8k** | ![rss-avg](/cometbft/v0.39/docs/qa/img38/voteExtensions/08k_avg_memory.png) | +| **16k** | ![rss-avg](/cometbft/v0.39/docs/qa/img38/voteExtensions/16k_avg_memory.png) | +| **32k** | ![rss-avg](/cometbft/v0.39/docs/qa/img38/voteExtensions/32k_avg_memory.png) | + +### Mempool Size + +This metric shows how many transactions are outstanding in the nodes' mempools. +Observe that in all runs, the average number of transactions in the mempool quickly drops to near zero between runs. + +| Experiment | Mempool Size | +| ------------ | ------------------------------------------------------------------ | +| **baseline** | ![mempool-avg](/cometbft/v0.39/docs/qa/img38/voteExtensions/baseline_avg_mempool_size.png) | +| **2k** | ![mempool-avg](/cometbft/v0.39/docs/qa/img38/voteExtensions/02k_avg_mempool_size.png) | +| **4k** | ![mempool-avg](/cometbft/v0.39/docs/qa/img38/voteExtensions/04k_avg_mempool_size.png) | +| **8k** | ![mempool-avg](/cometbft/v0.39/docs/qa/img38/voteExtensions/08k_avg_mempool_size.png) | +| **16k** | ![mempool-avg](/cometbft/v0.39/docs/qa/img38/voteExtensions/16k_avg_mempool_size.png) | +| **32k** | ![mempool-avg](/cometbft/v0.39/docs/qa/img38/voteExtensions/32k_avg_mempool_size.png) | + +### Results + +| Scenario | Date | Version | Result | +| -------- | ---------- | ------------------------------------------------------------------------------------- | ------ | +| VESize | 2023-05-23 | v0.38.0-alpha.2 + varying vote extensions (9fc711b6514f99b2dc0864fc703cb81214f01783) | N/A | + +[\#9539]: https://github.com/tendermint/tendermint/issues/9539 +[\#9548]: https://github.com/tendermint/tendermint/issues/9548 +[\#539]: https://github.com/cometbft/cometbft/issues/539 +[\#546]: https://github.com/cometbft/cometbft/issues/546 +[\#562]: https://github.com/cometbft/cometbft/issues/562 +[end-to-end]: https://github.com/cometbft/cometbft/tree/main/test/e2e diff --git a/cometbft/v0.39/docs/qa/CometBFT-qa.mdx b/cometbft/v0.39/docs/qa/CometBFT-qa.mdx new file mode 100644 index 000000000..d4dc93979 --- /dev/null +++ b/cometbft/v0.39/docs/qa/CometBFT-qa.mdx @@ -0,0 +1,23 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/docs/qa/CometBFT-qa' +title: CometBFT Quality Assurance +order: 1 +--- + +This directory keeps track of the process followed by the CometBFT team +for Quality Assurance before cutting a release. +This directory is to live in multiple branches. On each release branch, +the contents of this directory reflect the status of the process +at the time the Quality Assurance process was applied for that release. + +File [method](/cometbft/v0.39/docs/qa/Method) keeps track of the process followed to obtain the results +used to decide if a release is passing the Quality Assurance process. +The results obtained in each release are stored in their own directory. +The following releases have undergone the Quality Assurance process, and the corresponding reports include detailed information on tests and comparisons with the baseline. + +* [TM v0.34.x](/cometbft/v0.39/docs/qa/TMCore-QA-34) - Tested prior to releasing Tendermint Core v0.34.22. +* [v0.34.x](/cometbft/v0.39/docs/qa/CometBFT-QA-34) - Tested prior to releasing v0.34.27, using TM v0.34.x results as baseline. +* [TM v0.37.x](/cometbft/v0.39/docs/qa/TMCore-QA-37) - Tested prior to releasing TM v0.37.x, using TM v0.34.x results as baseline. +* [v0.37.x](/cometbft/v0.39/docs/qa/CometBFT-QA-37) - Tested on CometBFT v0.37.0-alpha3, using TM v0.37.x results as baseline. +* [v0.38.x](/cometbft/v0.39/docs/qa/CometBFT-QA-38) - Tested on v0.38.0-alpha.2, using v0.37.x results as baseline. diff --git a/cometbft/v0.39/docs/qa/Method.mdx b/cometbft/v0.39/docs/qa/Method.mdx new file mode 100644 index 000000000..424ec26e4 --- /dev/null +++ b/cometbft/v0.39/docs/qa/Method.mdx @@ -0,0 +1,260 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/docs/qa/Method' +title: Method +order: 1 +--- + +This document provides a detailed description of the QA process. +It is intended to be used by engineers reproducing the experimental setup for future tests of CometBFT. + +The (first iteration of the) QA process as described [in the RELEASES.md document][releases] +was applied to version v0.34.x in order to have a set of results acting as a benchmarking baseline. +This baseline is then compared with results obtained in later versions. + +Out of the testnet-based test cases described in [the releases document][releases], we focused on two of them: +_200 Node Test_ and _Rotating Nodes Test_. + +[releases]: https://github.com/cometbft/cometbft/blob/v0.38.x/RELEASES.md#large-scale-testnets + +## Software Dependencies + +### Infrastructure Requirements to Run the Tests + +* An account at Digital Ocean (DO), with a high droplet limit (>202) +* The machine to orchestrate the tests should have the following installed: + * A clone of the [testnet repository][testnet-repo] + * This repository contains all the scripts mentioned in the remainder of this section + * [Digital Ocean CLI][doctl] + * [Terraform CLI][Terraform] + * [Ansible CLI][Ansible] + +[testnet-repo]: https://github.com/cometbft/qa-infra +[Ansible]: https://docs.ansible.com/ansible/latest/index.html +[Terraform]: https://www.terraform.io/docs +[doctl]: https://docs.digitalocean.com/reference/doctl/how-to/install/ + +### Requirements for Result Extraction + +* [Prometheus DB][prometheus] to collect metrics from nodes +* Prometheus DB to process queries (may be a different node from the previous one) +* blockstore DB of one of the full nodes in the testnet + + +[prometheus]: https://prometheus.io/ + +## 200 Node Testnet + +### Running the test + +This section explains how the tests were carried out for reproducibility purposes. + +1. [If you haven't done it before] + Follow steps 1-4 of the `README.md` at the top of the testnet repository to configure Terraform and `doctl`. +2. Copy file `testnets/testnet200.toml` onto `testnet.toml` (do NOT commit this change). +3. Set the variable `VERSION_TAG` in the `Makefile` to the git hash that is to be tested. + * If you are running the base test, which implies a homogeneous network (all nodes are running the same version), + then make sure makefile variable `VERSION2_WEIGHT` is set to 0. + * If you are running a mixed network, set the variable `VERSION2_TAG` to the other version you want deployed + in the network. + Then adjust the weight variables `VERSION_WEIGHT` and `VERSION2_WEIGHT` to configure the + desired proportion of nodes running each of the two configured versions. +4. Follow steps 5-10 of the `README.md` to configure and start the 200 node testnet. + * WARNING: Do NOT forget to run `make terraform-destroy` as soon as you are done with the tests (see step 9). +5. As a sanity check, connect to the Prometheus node's web interface (port 9090) + and check the graph for the `cometbft_consensus_height` metric. All nodes + should be increasing their heights. + + * You can find the Prometheus node's IP address in `ansible/hosts` under section `[prometheus]`. + * The following URL will display the metrics `cometbft_consensus_height` and `cometbft_mempool_size`: + + ``` + http://:9090/classic/graph?g0.range_input=1h&g0.expr=cometbft_consensus_height&g0.tab=0&g1.range_input=1h&g1.expr=cometbft_mempool_size&g1.tab=0 + ``` + +6. You now need to start the load runner that will produce transaction load. + * If you don't know the saturation load of the version you are testing, you need to discover it. + * Run `make loadrunners-init`. This will copy the loader scripts to the + `testnet-load-runner` node and install the load tool. + * Find the IP address of the `testnet-load-runner` node in + `ansible/hosts` under section `[loadrunners]`. + * `ssh` into `testnet-load-runner`. + * Edit the script `/root/200-node-loadscript.sh` in the load runner + node to provide the IP address of a full node (for example, + `validator000`). This node will receive all transactions from the + load runner node. + * Run `/root/200-node-loadscript.sh` from the load runner node. + * This script will take about 40 minutes to run, so it is suggested to + first run `tmux` in case the ssh session breaks. + * It is running 90-second-long experiments in a loop with different + loads. + * If you already know the saturation load, you can simply run the test (several times) for 90 seconds with a load somewhat + below saturation: + * Set makefile variables `LOAD_CONNECTIONS`, `LOAD_TX_RATE` to values that will produce the desired transaction load. + * Set `LOAD_TOTAL_TIME` to 90 (seconds). + * Run `make runload` and wait for it to complete. You may want to run this several times so the data from different runs can be compared. +7. Run `make retrieve-data` to gather all relevant data from the testnet into the orchestrating machine. + * Alternatively, you may want to run `make retrieve-prometheus-data` and `make retrieve-blockstore` separately. + The end result will be the same. + * `make retrieve-blockstore` accepts the following values in makefile variable `RETRIEVE_TARGET_HOST`: + * `any`: (which is the default) picks up a full node and retrieves the blockstore from that node only. + * `all`: retrieves the blockstore from all full nodes; this is extremely slow and consumes plenty of bandwidth, + so use it with care. + * the name of a particular full node (e.g., `validator01`): retrieves the blockstore from that node only. +8. Verify that the data was collected without errors: + * at least one blockstore DB for a CometBFT validator + * the Prometheus database from the Prometheus node + * for extra care, you can run `zip -T` on the `prometheus.zip` file and (one of) the `blockstore.db.zip` file(s) +9. **Run `make terraform-destroy`** + * Don't forget to type `yes`! Otherwise you're in trouble. + +### Result Extraction + +The method for extracting the results described here is highly manual (and exploratory) at this stage. +The CometBFT team should improve it at every iteration to increase the amount of automation. + +#### Steps + +1. Unzip the blockstore into a directory. +2. To identify saturation points: + 1. Extract the latency report for all the experiments. + * Run these commands from the directory containing the `blockstore.db` folder. + * It is advisable to adjust the hash in the `go run` command to the latest possible. + * ```bash + mkdir results + go run github.com/cometbft/cometbft/test/loadtime/cmd/report@3003ef7 --database-type goleveldb --data-dir ./ > results/report.txt + ``` + 2. File `report.txt` contains an unordered list of experiments with varying concurrent connections and transaction rate. + You will need to separate data per experiment. + + * Create files `report01.txt`, `report02.txt`, `report04.txt`, and for each experiment in file `report.txt`, + copy its related lines to the filename that matches the number of connections, for example: + + ```bash + for cnum in 1 2 4; do echo "$cnum"; grep "Connections: $cnum" results/report.txt -B 2 -A 10 > results/report$cnum.txt; done + ``` + + * Sort the experiments in `report01.txt` in ascending tx rate order. Likewise for `report02.txt` and `report04.txt`. + * Otherwise just keep `report.txt` and skip to the next step. + 3. Generate file `report_tabbed.txt` by showing the contents of `report01.txt`, `report02.txt`, `report04.txt` side by side. + * This effectively creates a table where rows are a particular tx rate and columns are a particular number of websocket connections. + * Combine the column files into a single table file: + * Replace tabs by spaces in all column files. For example, + `sed -i.bak 's/\t/ /g' results/report1.txt`. + * Merge the new column files into one: + `paste results/report1.txt results/report2.txt results/report4.txt | column -s $'\t' -t > report_tabbed.txt` + +3. To generate a latency vs throughput plot, extract the data as a CSV: + * ```bash + go run github.com/cometbft/cometbft/test/loadtime/cmd/report@3003ef7 --database-type goleveldb --data-dir ./ --csv results/raw.csv + ``` + * Follow the instructions for the [`latency_throughput.py`] script. + This plot is useful to visualize the saturation point. + * Alternatively, follow the instructions for the [`latency_plotter.py`] script. + This script generates a series of plots per experiment and configuration that may + help with visualizing latency vs throughput variation. + +[`latency_throughput.py`]: https://github.com/cometbft/cometbft/tree/v0.38.x/scripts/qa/reporting#latency-vs-throughput-plotting +[`latency_plotter.py`]: https://github.com/cometbft/cometbft/tree/v0.38.x/scripts/qa/reporting#latency-vs-throughput-plotting-version-2 + +#### Extracting Prometheus Metrics + +1. Stop the prometheus server if it is running as a service (e.g., a `systemd` unit). +2. Unzip the prometheus database retrieved from the testnet, and move it to replace the + local prometheus database. +3. Start the prometheus server and make sure no error logs appear at startup. +4. Identify the time window you want to plot in your graphs. +5. Execute the [`prometheus_plotter.py`] script for the time window. + +[`prometheus_plotter.py`]: https://github.com/cometbft/cometbft/tree/v0.38.x/scripts/qa/reporting#prometheus-metrics + +## Rotating Node Testnet + +### Running the test + +This section explains how the tests were carried out for reproducibility purposes. + +1. [If you haven't done it before] + Follow steps 1-4 of the `README.md` at the top of the testnet repository to configure Terraform and `doctl`. +2. Copy file `testnet_rotating.toml` onto `testnet.toml` (do NOT commit this change). +3. Set variable `VERSION_TAG` to the git hash that is to be tested. +4. Run `make terraform-apply EPHEMERAL_SIZE=25`. + * WARNING: Do NOT forget to run `make terraform-destroy` as soon as you are done with the tests. +5. Follow steps 6-10 of the `README.md` to configure and start the "stable" part of the rotating node testnet. +6. As a sanity check, connect to the Prometheus node's web interface and check the graph for the `tendermint_consensus_height` metric. + All nodes should be increasing their heights. +7. On a different shell: + * Run `make runload LOAD_CONNECTIONS=X LOAD_TX_RATE=Y LOAD_TOTAL_TIME=Z`. + * `X` and `Y` should reflect a load below the saturation point (see, e.g., + [this paragraph](/cometbft/v0.39/docs/qa/TMCore-QA-34#finding-the-saturation-point) for further info). + * `Z` (in seconds) should be big enough to keep running throughout the test, until we manually stop it in step 9. + In principle, a good value for `Z` is `7200` (2 hours). +8. Run `make rotate` to start the script that creates the ephemeral nodes and kills them when they are caught up. + * WARNING: If you run this command from your laptop, the laptop needs to be up and connected for the full length + of the experiment. + * `http://:9090/classic/graph?g0.range_input=100m&g0.expr=cometbft_consensus_height%7Bjob%3D~%22ephemeral.*%22%7D%20or%20cometbft_blocksync_latest_block_height%7Bjob%3D~%22ephemeral.*%22%7D&g0.tab=0&g1.range_input=100m&g1.expr=cometbft_mempool_size%7Bjob!~%22ephemeral.*%22%7D&g1.tab=0&g2.range_input=100m&g2.expr=cometbft_consensus_num_txs%7Bjob!~%22ephemeral.*%22%7D&g2.tab=0` + is an example Prometheus URL you can use to monitor the test case's progress. +9. When the height of the chain reaches 3000, stop the `make runload` script. +10. When the rotate script has made two iterations (i.e., all ephemeral nodes have caught up twice) + after height 3000 was reached, stop `make rotate`. +11. Run `make stop-network`. +12. Run `make retrieve-data` to gather all relevant data from the testnet into the orchestrating machine. +13. Verify that the data was collected without errors: + * at least one blockstore DB for a CometBFT validator + * the Prometheus database from the Prometheus node + * for extra care, you can run `zip -T` on the `prometheus.zip` file and (one of) the `blockstore.db.zip` file(s) +14. **Run `make terraform-destroy`** + +Steps 8 to 10 are highly manual at the moment and will be improved in next iterations. + +### Result Extraction + +In order to obtain a latency plot, follow the instructions above for the 200 node experiment, +but the `results.txt` file contains only one experiment. + +As for Prometheus, the same method as for the 200 node experiment can be applied. + +## Vote Extensions Testnet + +### Running the test + +This section explains how the tests were carried out for reproducibility purposes. + +1. [If you haven't done it before] + Follow steps 1-4 of the `README.md` at the top of the testnet repository to configure Terraform and `doctl`. +2. Copy file `varyVESize.toml` onto `testnet.toml` (do NOT commit this change). +3. Set variable `VERSION_TAG` in the `Makefile` to the git hash that is to be tested. +4. Follow steps 5-10 of the `README.md` to configure and start the testnet. + * WARNING: Do NOT forget to run `make terraform-destroy` as soon as you are done with the tests. +5. Configure the load runner to produce the desired transaction load. + * Set makefile variables `ROTATE_CONNECTIONS`, `ROTATE_TX_RATE` to values that will produce the desired transaction load. + * Set `ROTATE_TOTAL_TIME` to 150 (seconds). + * Set `ITERATIONS` to the number of iterations that each configuration should run for. +6. Execute steps 5-10 of the `README.md` file at the testnet repository. + +7. Repeat the following steps for each desired `vote_extension_size`: + 1. Update the configuration (you can skip this step if you didn't change the `vote_extension_size`). + * Update the `vote_extensions_size` in the `testnet.toml` to the desired value. + * `make configgen` + * `ANSIBLE_SSH_RETRIES=10 ansible-playbook ./ansible/re-init-testapp.yaml -u root -i ./ansible/hosts --limit=validators -e "testnet_dir=testnet" -f 20` + * `make restart` + 2. Run the test. + * `make runload` + This will repeat the tests `ITERATIONS` times every time it is invoked. + 3. Collect your data. + * `make retrieve-data` + Gathers all relevant data from the testnet into the orchestrating machine, inside folder `experiments`. + Two subfolders are created: one blockstore DB for a CometBFT validator and one for the Prometheus DB data. + * Verify that the data was collected without errors with `zip -T` on the `prometheus.zip` file and (one of) the `blockstore.db.zip` file(s). +8. Clean up your setup. + * `make terraform-destroy`; don't forget that you need to type **yes** for it to complete. + + +### Result Extraction + +In order to obtain a latency plot, follow the instructions above for the 200 node experiment, but: + +* The `results.txt` file contains only one experiment. +* Therefore, no need for any `for` loops. + +As for Prometheus, the same method as for the 200 node experiment can be applied. diff --git a/cometbft/v0.39/docs/qa/TMCore-QA-34.mdx b/cometbft/v0.39/docs/qa/TMCore-QA-34.mdx new file mode 100644 index 000000000..5d54fcb79 --- /dev/null +++ b/cometbft/v0.39/docs/qa/TMCore-QA-34.mdx @@ -0,0 +1,278 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/docs/qa/TMCore-QA-34' +order: 1 +parent: + title: Tendermint Core QA Results v0.34.x + description: This is a report on the results obtained when running v0.34.x on testnets + order: 2 +--- + +# Tendermint Core QA Results v0.34.x + +## 200 Node Testnet + +### Finding the Saturation Point + +The first goal when examining the results of the tests is identifying the saturation point. +The saturation point is a setup with a transaction load large enough to prevent the testnet +from being stable: the load runner tries to produce slightly more transactions than can +be processed by the testnet. + +The following table summarizes the results for v0.34.x for the different experiments +(extracted from file [`v034_report_tabbed.txt`](https://raw.githubusercontent.com/cometbft/cometbft/v0.38.x/docs/qa/img34/v034_report_tabbed.txt)). + +The X axis of this table is `c`, the number of connections created by the load runner process to the target node. +The Y axis of this table is `r`, the rate or number of transactions issued per second. + +| | c=1 | c=2 | c=4 | +| :--- | ----: | ----: | ----: | +| r=25 | 2225 | 4450 | 8900 | +| r=50 | 4450 | 8900 | 17800 | +| r=100 | 8900 | 17800 | 35600 | +| r=200 | 17800 | 35600 | 38660 | + +The table shows the number of 1024-byte-long transactions that were produced by the load runner +and processed by Tendermint Core during the 90 seconds of the experiment's duration. +Each cell in the table refers to an experiment with a particular number of websocket connections (`c`) +to a chosen validator and the number of transactions per second that the load runner +tries to produce (`r`). Note that the overall load the tool attempts to generate is $c \cdot r$. + +We can see that the saturation point is beyond the diagonal that spans cells + +* `r=200,c=2` +* `r=100,c=4` + +given that the total number of transactions should be close to the product rate × the number of connections × experiment time. + +All experiments below the saturation diagonal (`r=200,c=4`) have in common that the total +number of transactions processed is noticeably less than the product $c \cdot r \cdot 89$ (89 seconds, since the last batch never gets sent), +which is the expected number of transactions when the system is able to handle the load well. +With (`r=200,c=4`), we obtained 38660, whereas the theoretical number of transactions should +have been $200 \cdot 4 \cdot 89 = 71200$. + +At this point, we chose an experiment at the limit of the saturation diagonal +in order to further study the performance of this release. +**The chosen experiment is (`r=200,c=2`)**. + +This is a plot of the CPU load (average over 1 minute, as output by `top`) of the load runner for (`r=200,c=2`), +where we can see that the load stays close to 0 most of the time. + +![load-load-runner](/cometbft/v0.39/docs/qa/img34/v034_r200c2_load-runner.png) + +### Examining Latencies + +The method described [here](/cometbft/v0.39/docs/qa/Method) allows us to plot the latencies of transactions +for all experiments. + +![all-latencies](/cometbft/v0.39/docs/qa/img34/v034_200node_latencies.png) + +As we can see, even the experiments beyond the saturation diagonal managed to keep +transaction latency stable (i.e., not constantly increasing). +Our interpretation is that contention within Tendermint Core was propagated +via the websockets to the load runner; +hence, the load runner could not produce the target load but a fraction of it. + +Further examination of the Prometheus data (see below) showed that the mempool contained many transactions +at steady state but did not grow much without quickly returning to this steady state. This demonstrates +that the Tendermint Core network was able to process transactions at least as quickly as they +were submitted to the mempool. Finally, the test script ensured that at the end of an experiment, the +mempool was empty so that all transactions submitted to the chain were processed. + +Finally, the number of points present in the plot appears to be much less than expected given the +number of transactions in each experiment, particularly close to or above the saturation diagonal. +This is a visual effect of the plot; what appear to be points in the plot are actually potentially huge +clusters of points. To corroborate this, we have zoomed in the plot above by setting (carefully chosen) +tiny axis intervals. The cluster shown below looks like a single point in the plot above. + +![all-latencies-zoomed](/cometbft/v0.39/docs/qa/img34/v034_200node_latencies_zoomed.png) + +The plot of latencies can be used as a baseline to compare with other releases. + +The following plot summarizes average latencies versus overall throughput +across different numbers of WebSocket connections to the node into which +transactions are being loaded. + +![latency-vs-throughput](/cometbft/v0.39/docs/qa/img34/v034_latency_throughput.png) + +### Prometheus Metrics on the Chosen Experiment + +As mentioned [above](#finding-the-saturation-point), the chosen experiment is `r=200,c=2`. +This section further examines key metrics for this experiment extracted from Prometheus data. + +#### Mempool Size + +The mempool size, a count of the number of transactions in the mempool, was shown to be stable and homogeneous +at all full nodes. It did not exhibit any unconstrained growth. +The plot below shows the evolution over time of the cumulative number of transactions inside all full nodes' mempools +at a given time. +The two spikes that can be observed correspond to a period where consensus instances proceeded beyond the initial round +at some nodes. + +![mempool-cumulative](/cometbft/v0.39/docs/qa/img34/v034_r200c2_mempool_size.png) + +The plot below shows the evolution of the average over all full nodes, which oscillates between 1500 and 2000 +outstanding transactions. + +![mempool-avg](/cometbft/v0.39/docs/qa/img34/v034_r200c2_mempool_size_avg.png) + +The peaks observed coincide with the moments when some nodes proceeded beyond the initial round of consensus (see below). + +#### Peers + +The number of peers was stable at all nodes. +It was higher for the seed nodes (around 140) than for the rest (between 21 and 74). +The fact that non-seed nodes reach more than 50 peers is due to #9548. + +![peers](/cometbft/v0.39/docs/qa/img34/v034_r200c2_peers.png) + +#### Consensus Rounds per Height + +Most nodes used only round 0 for most heights, but some nodes needed to advance to round 1 for some heights. + +![rounds](/cometbft/v0.39/docs/qa/img34/v034_r200c2_rounds.png) + +#### Blocks Produced per Minute, Transactions Processed per Minute + +The blocks produced per minute are the slope of this plot. + +![heights](/cometbft/v0.39/docs/qa/img34/v034_r200c2_heights.png) + +Over a period of 2 minutes, the height goes from 530 to 569. +This results in an average of 19.5 blocks produced per minute. + +The transactions processed per minute are the slope of this plot. + +![total-txs](/cometbft/v0.39/docs/qa/img34/v034_r200c2_total-txs.png) + +Over a period of 2 minutes, the total goes from 64525 to 100125 transactions, +resulting in 17800 transactions per minute. However, we can see in the plot that +all transactions in the load are processed long before the two minutes. +If we adjust the time window for when transactions are processed (approx. 105 seconds), +we obtain 20343 transactions per minute. + +#### Memory Resident Set Size + +Resident Set Size of all monitored processes is plotted below. + +![rss](/cometbft/v0.39/docs/qa/img34/v034_r200c2_rss.png) + +The average over all processes oscillates around 1.2 GiB and does not demonstrate unconstrained growth. + +![rss-avg](/cometbft/v0.39/docs/qa/img34/v034_r200c2_rss_avg.png) + +#### CPU Utilization + +The best metric from Prometheus to gauge CPU utilization on a Unix machine is `load1`, +as it usually appears in the +[output of `top`](https://www.digitalocean.com/community/tutorials/load-average-in-linux). + +![load1](/cometbft/v0.39/docs/qa/img34/v034_r200c2_load1.png) + +It is contained in most cases below 5, which is generally considered acceptable load. + +### Test Result + +**Result: N/A** (v0.34.x is the baseline) + +Date: 2022-10-14 + +Version: 3ec6e424d6ae4c96867c2dcf8310572156068bb6 + +## Rotating Node Testnet + +For this testnet, we will use a load that can safely be considered below the saturation +point for the size of this testnet (between 13 and 38 full nodes): `c=4,r=800`. + +N.B.: The version of CometBFT used for these tests is affected by #9539. +However, the reduced load that reaches the mempools is orthogonal to the functionality +we are focusing on here. + +### Latencies + +The plot of all latencies can be seen in the following plot. + +![rotating-all-latencies](/cometbft/v0.39/docs/qa/img34/v034_rotating_latencies.png) + +We can observe there are some very high latencies toward the end of the test. +Upon suspicion that they are duplicate transactions, we examined the latencies +raw file and discovered there are more than 100K duplicate transactions. + +The following plot shows the latencies file where all duplicate transactions have +been removed, i.e., only the first occurrence of a duplicate transaction is kept. + +![rotating-all-latencies-uniq](/cometbft/v0.39/docs/qa/img34/v034_rotating_latencies_uniq.png) + +This problem, existing in `v0.34.x`, will need to be addressed, perhaps in the same way +we addressed it when running the 200 node test with high loads: increasing the `cache_size` +configuration parameter. + +### Prometheus Metrics + +The set of metrics shown here are fewer than for the 200 node experiment. +We are only interested in those for which the catch-up process (blocksync) may have an impact. + +#### Blocks and Transactions per Minute + +Just as shown for the 200 node test, the blocks produced per minute are the gradient of this plot. + +![rotating-heights](/cometbft/v0.39/docs/qa/img34/v034_rotating_heights.png) + +Over a period of 5229 seconds, the height goes from 2 to 3638. +This results in an average of 41 blocks produced per minute. + +The following plot shows only the heights reported by ephemeral nodes +(which are also included in the plot above). Note that the _height_ metric +is only shown _once the node has switched to consensus_, hence the gaps +when nodes are killed, wiped out, started from scratch, and catching up. + +![rotating-heights-ephe](/cometbft/v0.39/docs/qa/img34/v034_rotating_heights_ephe.png) + +The transactions processed per minute are the gradient of this plot. + +![rotating-total-txs](/cometbft/v0.39/docs/qa/img34/v034_rotating_total-txs.png) + +The small lines we see periodically close to `y=0` are the transactions that +ephemeral nodes start processing when they are caught up. + +Over a period of 5229 seconds, the total goes from 0 to 387697 transactions, +resulting in 4449 transactions per minute. We can see some abrupt changes in +the plot's gradient. This will need to be investigated. + +#### Peers + +The plot below shows the evolution in peers throughout the experiment. +The periodic changes observed are due to the ephemeral nodes being stopped, +wiped out, and recreated. + +![rotating-peers](/cometbft/v0.39/docs/qa/img34/v034_rotating_peers.png) + +The validators' plots are concentrated at the higher part of the graph, whereas the ephemeral nodes +are mostly at the lower part. + +#### Memory Resident Set Size + +The average Resident Set Size (RSS) over all processes seems stable and slightly growing toward the end. +This might be related to the increase in transaction load observed above. + +![rotating-rss-avg](/cometbft/v0.39/docs/qa/img34/v034_rotating_rss_avg.png) + +The memory taken by the validators and the ephemeral nodes (when they are up) is comparable. + +#### CPU Utilization + +The plot shows metric `load1` for all nodes. + +![rotating-load1](/cometbft/v0.39/docs/qa/img34/v034_rotating_load1.png) + +It is contained under 5 most of the time, which is considered normal load. +The purple line, which follows a different pattern, is the validator receiving all +transactions via RPC from the load runner process. + +### Test Result + +**Result: N/A** + +Date: 2022-10-10 + +Version: a28c987f5a604ff66b515dd415270063e6fb069d diff --git a/cometbft/v0.39/docs/qa/TMCore-QA-37.mdx b/cometbft/v0.39/docs/qa/TMCore-QA-37.mdx new file mode 100644 index 000000000..c4bdfb360 --- /dev/null +++ b/cometbft/v0.39/docs/qa/TMCore-QA-37.mdx @@ -0,0 +1,326 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/docs/qa/TMCore-QA-37' +order: 1 +parent: + title: Tendermint Core QA Results v0.37.x + description: This is a report on the results obtained when running TM v0.37.x on testnets + order: 4 +--- + +# Tendermint Core QA Results v0.37.x + +## Issues Discovered + +During this iteration of the QA process, the following issues were found: + +* (critical, fixed) [\#9533] - This bug caused full nodes to sometimes get stuck + when blocksyncing, requiring a manual restart to unblock them. Importantly, + this bug was also present in v0.34.x and the fix was also backported in + [\#9534]. +* (critical, fixed) [\#9539] - `loadtime` is very likely to include more than + one "=" character in transactions, which is rejected by the e2e application. +* (critical, fixed) [\#9581] - Absent prometheus label makes CometBFT crash + when enabling Prometheus metric collection. +* (non-critical, not fixed) [\#9548] - Full nodes can go over 50 connected + peers, which is not intended by the default configuration. +* (non-critical, not fixed) [\#9537] - With the default mempool cache setting, + duplicated transactions are not rejected when gossiped and eventually flood + all mempools. The 200 node testnets were thus run with a value of 200000 (as + opposed to the default 10000). + +## 200 Node Testnet + +### Finding the Saturation Point + +The first goal is to identify the saturation point and compare it with the baseline (v0.34.x). +For further details, see [this paragraph](/cometbft/v0.39/docs/qa/TMCore-QA-34#finding-the-saturation-point) +in the baseline version. + +The following table summarizes the results for v0.37.x for the different experiments +(extracted from file [`v037_report_tabbed.txt`](https://raw.githubusercontent.com/cometbft/cometbft/v0.38.x/docs/qa/img37/200nodes_tm037/v037_report_tabbed.txt)). + +The X axis of this table is `c`, the number of connections created by the load runner process to the target node. +The Y axis of this table is `r`, the rate or number of transactions issued per second. + +| | c=1 | c=2 | c=4 | +| :--- | ----: | ----: | ----: | +| r=25 | 2225 | 4450 | 8900 | +| r=50 | 4450 | 8900 | 17800 | +| r=100 | 8900 | 17800 | 35600 | +| r=200 | 17800 | 35600 | 38660 | + +For comparison, this is the table with the baseline version. + +| | c=1 | c=2 | c=4 | +| :--- | ----: | ----: | ----: | +| r=25 | 2225 | 4450 | 8900 | +| r=50 | 4450 | 8900 | 17800 | +| r=100 | 8900 | 17800 | 35400 | +| r=200 | 17800 | 35600 | 37358 | + +The saturation point is beyond the diagonal: + +* `r=200,c=2` +* `r=100,c=4` + +which is at the same place as the baseline. For more details on the saturation point, see +[this paragraph](/cometbft/v0.39/docs/qa/TMCore-QA-34#finding-the-saturation-point) in the baseline version. + +The experiment chosen to examine Prometheus metrics is the same as in the baseline: +**`r=200,c=2`**. + +The load runner's CPU load was negligible (near 0) when running `r=200,c=2`. + +### Examining Latencies + +The method described [here](/cometbft/v0.39/docs/qa/Method) allows us to plot the latencies of transactions +for all experiments. + +![all-latencies](/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_200node_latencies.png) + +The data seen in the plot is similar to that of the baseline. + +![all-latencies-bl](/cometbft/v0.39/docs/qa/img34/v034_200node_latencies.png) + +Therefore, for further details on these plots, +see [this paragraph](/cometbft/v0.39/docs/qa/CometBFT-QA-34#examining-latencies) in the baseline version. + +The following plot summarizes average latencies versus overall throughputs +across different numbers of WebSocket connections to the node into which +transactions are being loaded. + +![latency-vs-throughput](/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_latency_throughput.png) + +This is similar to the baseline plot: + +![latency-vs-throughput-bl](/cometbft/v0.39/docs/qa/img34/v034_latency_throughput.png) + +### Prometheus Metrics on the Chosen Experiment + +As mentioned [above](#finding-the-saturation-point), the chosen experiment is `r=200,c=2`. +This section further examines key metrics for this experiment extracted from Prometheus data. + +#### Mempool Size + +The mempool size, a count of the number of transactions in the mempool, was shown to be stable and homogeneous +at all full nodes. It did not exhibit any unconstrained growth. +The plot below shows the evolution over time of the cumulative number of transactions inside all full nodes' mempools +at a given time. + +![mempool-cumulative](/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_r200c2_mempool_size.png) + +The plot below shows the evolution of the average over all full nodes, which oscillates between 1500 and 2000 outstanding transactions. + +![mempool-avg](/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_r200c2_mempool_size_avg.png) + +The peaks observed coincide with the moments when some nodes reached round 1 of consensus (see below). + +**These plots yield similar results to the baseline**: + +![mempool-cumulative-bl](/cometbft/v0.39/docs/qa/img34/v034_r200c2_mempool_size.png) + +![mempool-avg-bl](/cometbft/v0.39/docs/qa/img34/v034_r200c2_mempool_size_avg.png) + +#### Peers + +The number of peers was stable at all nodes. +It was higher for the seed nodes (around 140) than for the rest (between 16 and 78). + +![peers](/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_r200c2_peers.png) + +Just as in the baseline, the fact that non-seed nodes reach more than 50 peers is due to #9548. + +**This plot yields similar results to the baseline**: + +![peers-bl](/cometbft/v0.39/docs/qa/img34/v034_r200c2_peers.png) + +#### Consensus Rounds per Height + +Most heights took just one round, but some nodes needed to advance to round 1 at some point. + +![rounds](/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_r200c2_rounds.png) + +**This plot yields slightly better results than the baseline**: + +![rounds-bl](/cometbft/v0.39/docs/qa/img34/v034_r200c2_rounds.png) + +#### Blocks Produced per Minute, Transactions Processed per Minute + +The blocks produced per minute are the gradient of this plot. + +![heights](/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_r200c2_heights.png) + +Over a period of 2 minutes, the height goes from 477 to 524. +This results in an average of 23.5 blocks produced per minute. + +The transactions processed per minute are the gradient of this plot. + +![total-txs](/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_r200c2_total-txs.png) + +Over a period of 2 minutes, the total goes from 64525 to 100125 transactions, +resulting in 17800 transactions per minute. However, we can see in the plot that +all transactions in the load are processed long before the two minutes. +If we adjust the time window when transactions are processed (approximately 90 seconds), +we obtain 23733 transactions per minute. + +**These plots yield similar results to the baseline**: + +![heights-bl](/cometbft/v0.39/docs/qa/img34/v034_r200c2_heights.png) + +![total-txs](/cometbft/v0.39/docs/qa/img34/v034_r200c2_total-txs.png) + +#### Memory Resident Set Size + +Resident Set Size of all monitored processes is plotted below. + +![rss](/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_r200c2_rss.png) + +The average over all processes oscillates around 380 MiB and does not demonstrate unconstrained growth. + +![rss-avg](/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_r200c2_rss_avg.png) + +**These plots yield similar results to the baseline**: + +![rss-bl](/cometbft/v0.39/docs/qa/img34/v034_r200c2_rss.png) + +![rss-avg-bl](/cometbft/v0.39/docs/qa/img34/v034_r200c2_rss_avg.png) + +#### CPU Utilization + +The best metric from Prometheus to gauge CPU utilization in a Unix machine is `load1`, +as it usually appears in the +[output of `top`](https://www.digitalocean.com/community/tutorials/load-average-in-linux). + +![load1](/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_r200c2_load1.png) + +It is contained below 5 on most nodes. + +**This plot yields similar results to the baseline**: + +![load1](/cometbft/v0.39/docs/qa/img34/v034_r200c2_load1.png) + +### Test Result + +**Result: PASS** + +Date: 2022-10-14 + +Version: 1cf9d8e276afe8595cba960b51cd056514965fd1 + +## Rotating Node Testnet + +We use the same load as in the baseline: `c=4,r=800`. + +Just as in the baseline tests, the version of CometBFT used for these tests is affected by #9539. +See this paragraph in the [baseline report](/cometbft/v0.39/docs/qa/Method#rotating-node-testnet) for further details. +Finally, note that this setup allows for a fairer comparison between this version and the baseline. + +### Latencies + +The plot of all latencies can be seen here. + +![rotating-all-latencies](/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_rotating_latencies.png) + +This is similar to the baseline. + +![rotating-all-latencies-bl](/cometbft/v0.39/docs/qa/img34/v034_rotating_latencies_uniq.png) + +Note that we are comparing against the baseline plot with _unique_ +transactions. This is because the problem with duplicate transactions +detected during the baseline experiment did not show up for `v0.37`, +which is _not_ proof that the problem is not present in `v0.37`. + +### Prometheus Metrics + +The set of metrics shown here match those shown on the baseline (`v0.34`) for the same experiment. +We also show the baseline results for comparison. + +#### Blocks and Transactions per Minute + +The blocks produced per minute are the gradient of this plot. + +![rotating-heights](/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_rotating_heights.png) + +Over a period of 4446 seconds, the height goes from 5 to 3323. +This results in an average of 45 blocks produced per minute, +which is similar to the baseline, shown below. + +![rotating-heights-bl](/cometbft/v0.39/docs/qa/img34/v034_rotating_heights.png) + +The following two plots show only the heights reported by ephemeral nodes. +The second plot is the baseline plot for comparison. + +![rotating-heights-ephe](/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_rotating_heights_ephe.png) + +![rotating-heights-ephe-bl](/cometbft/v0.39/docs/qa/img34/v034_rotating_heights_ephe.png) + +By the length of the segments, we can see that ephemeral nodes in `v0.37` +catch up slightly faster. + +The transactions processed per minute are the gradient of this plot. + +![rotating-total-txs](/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_rotating_total-txs.png) + +Over a period of 3852 seconds, the total goes from 597 to 267298 transactions in one of the validators, +resulting in 4154 transactions per minute, which is slightly lower than the baseline, +although the baseline had to deal with duplicate transactions. + +For comparison, this is the baseline plot. + +![rotating-total-txs-bl](/cometbft/v0.39/docs/qa/img34/v034_rotating_total-txs.png) + +#### Peers + +The plot below shows the evolution of the number of peers throughout the experiment. + +![rotating-peers](/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_rotating_peers.png) + +This is the baseline plot, for comparison. + +![rotating-peers-bl](/cometbft/v0.39/docs/qa/img34/v034_rotating_peers.png) + +The plotted values and their evolution are comparable in both plots. + +For further details on these plots, see the baseline report. + +#### Memory Resident Set Size + +The average Resident Set Size (RSS) over all processes looks slightly more stable +on `v0.37` (first plot) than on the baseline (second plot). + +![rotating-rss-avg](/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_rotating_rss_avg.png) + +![rotating-rss-avg-bl](/cometbft/v0.39/docs/qa/img34/v034_rotating_rss_avg.png) + +The memory taken by the validators and the ephemeral nodes when they are up is comparable (not shown in the plots), +just as observed in the baseline. + +#### CPU Utilization + +The plot shows metric `load1` for all nodes. + +![rotating-load1](/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_rotating_load1.png) + +![rotating-load1-bl](/cometbft/v0.39/docs/qa/img34/v034_rotating_load1.png) + +In both cases, it is contained under 5 most of the time, which is considered normal load. +The green line in the `v0.37` plot and the purple line in the baseline plot (`v0.34`) +correspond to the validators receiving all transactions, via RPC, from the load runner process. +In both cases, they oscillate around 5 (normal load). The main difference is that other +nodes are generally less loaded in `v0.37`. + +### Test Result + +**Result: PASS** + +Date: 2022-10-10 + +Version: 155110007b9d8b83997a799016c1d0844c8efbaf + +[\#9533]: https://github.com/tendermint/tendermint/pull/9533 +[\#9534]: https://github.com/tendermint/tendermint/pull/9534 +[\#9539]: https://github.com/tendermint/tendermint/issues/9539 +[\#9548]: https://github.com/tendermint/tendermint/issues/9548 +[\#9537]: https://github.com/tendermint/tendermint/issues/9537 +[\#9581]: https://github.com/tendermint/tendermint/issues/9581 diff --git a/cometbft/v0.39/docs/qa/img34/baseline/avg_cpu.png b/cometbft/v0.39/docs/qa/img34/baseline/avg_cpu.png new file mode 100644 index 000000000..622456df6 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/baseline/avg_cpu.png differ diff --git a/cometbft/v0.39/docs/qa/img34/baseline/avg_memory.png b/cometbft/v0.39/docs/qa/img34/baseline/avg_memory.png new file mode 100644 index 000000000..55f213f5e Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/baseline/avg_memory.png differ diff --git a/cometbft/v0.39/docs/qa/img34/baseline/avg_mempool_size.png b/cometbft/v0.39/docs/qa/img34/baseline/avg_mempool_size.png new file mode 100644 index 000000000..ec7407295 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/baseline/avg_mempool_size.png differ diff --git a/cometbft/v0.39/docs/qa/img34/baseline/block_rate_regular.png b/cometbft/v0.39/docs/qa/img34/baseline/block_rate_regular.png new file mode 100644 index 000000000..bdc7aa28d Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/baseline/block_rate_regular.png differ diff --git a/cometbft/v0.39/docs/qa/img34/baseline/cpu.png b/cometbft/v0.39/docs/qa/img34/baseline/cpu.png new file mode 100644 index 000000000..ac4fc2695 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/baseline/cpu.png differ diff --git a/cometbft/v0.39/docs/qa/img34/baseline/memory.png b/cometbft/v0.39/docs/qa/img34/baseline/memory.png new file mode 100644 index 000000000..17336bd1b Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/baseline/memory.png differ diff --git a/cometbft/v0.39/docs/qa/img34/baseline/mempool_size.png b/cometbft/v0.39/docs/qa/img34/baseline/mempool_size.png new file mode 100644 index 000000000..fafba68c1 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/baseline/mempool_size.png differ diff --git a/cometbft/v0.39/docs/qa/img34/baseline/peers.png b/cometbft/v0.39/docs/qa/img34/baseline/peers.png new file mode 100644 index 000000000..05a288a35 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/baseline/peers.png differ diff --git a/cometbft/v0.39/docs/qa/img34/baseline/rounds.png b/cometbft/v0.39/docs/qa/img34/baseline/rounds.png new file mode 100644 index 000000000..79f3348a2 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/baseline/rounds.png differ diff --git a/cometbft/v0.39/docs/qa/img34/baseline/total_txs_rate_regular.png b/cometbft/v0.39/docs/qa/img34/baseline/total_txs_rate_regular.png new file mode 100644 index 000000000..d80bef12c Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/baseline/total_txs_rate_regular.png differ diff --git a/cometbft/v0.39/docs/qa/img34/cmt1tm1/all_experiments.png b/cometbft/v0.39/docs/qa/img34/cmt1tm1/all_experiments.png new file mode 100644 index 000000000..4dc857edc Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/cmt1tm1/all_experiments.png differ diff --git a/cometbft/v0.39/docs/qa/img34/cmt1tm1/avg_cpu.png b/cometbft/v0.39/docs/qa/img34/cmt1tm1/avg_cpu.png new file mode 100644 index 000000000..cabd273a5 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/cmt1tm1/avg_cpu.png differ diff --git a/cometbft/v0.39/docs/qa/img34/cmt1tm1/avg_memory.png b/cometbft/v0.39/docs/qa/img34/cmt1tm1/avg_memory.png new file mode 100644 index 000000000..c8e576177 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/cmt1tm1/avg_memory.png differ diff --git a/cometbft/v0.39/docs/qa/img34/cmt1tm1/avg_mempool_size.png b/cometbft/v0.39/docs/qa/img34/cmt1tm1/avg_mempool_size.png new file mode 100644 index 000000000..b41199dc0 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/cmt1tm1/avg_mempool_size.png differ diff --git a/cometbft/v0.39/docs/qa/img34/cmt1tm1/block_rate_regular.png b/cometbft/v0.39/docs/qa/img34/cmt1tm1/block_rate_regular.png new file mode 100644 index 000000000..9b3a0b827 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/cmt1tm1/block_rate_regular.png differ diff --git a/cometbft/v0.39/docs/qa/img34/cmt1tm1/cpu.png b/cometbft/v0.39/docs/qa/img34/cmt1tm1/cpu.png new file mode 100644 index 000000000..cd5acdeb2 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/cmt1tm1/cpu.png differ diff --git a/cometbft/v0.39/docs/qa/img34/cmt1tm1/memory.png b/cometbft/v0.39/docs/qa/img34/cmt1tm1/memory.png new file mode 100644 index 000000000..6f56b3ccf Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/cmt1tm1/memory.png differ diff --git a/cometbft/v0.39/docs/qa/img34/cmt1tm1/mempool_size.png b/cometbft/v0.39/docs/qa/img34/cmt1tm1/mempool_size.png new file mode 100644 index 000000000..862a0bdd4 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/cmt1tm1/mempool_size.png differ diff --git a/cometbft/v0.39/docs/qa/img34/cmt1tm1/peers.png b/cometbft/v0.39/docs/qa/img34/cmt1tm1/peers.png new file mode 100644 index 000000000..737cf3dff Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/cmt1tm1/peers.png differ diff --git a/cometbft/v0.39/docs/qa/img34/cmt1tm1/rounds.png b/cometbft/v0.39/docs/qa/img34/cmt1tm1/rounds.png new file mode 100644 index 000000000..17884813a Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/cmt1tm1/rounds.png differ diff --git a/cometbft/v0.39/docs/qa/img34/cmt1tm1/total_txs_rate_regular.png b/cometbft/v0.39/docs/qa/img34/cmt1tm1/total_txs_rate_regular.png new file mode 100644 index 000000000..8b0cc0d42 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/cmt1tm1/total_txs_rate_regular.png differ diff --git a/cometbft/v0.39/docs/qa/img34/cmt2tm1/all_experiments.png b/cometbft/v0.39/docs/qa/img34/cmt2tm1/all_experiments.png new file mode 100644 index 000000000..4e6f73d35 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/cmt2tm1/all_experiments.png differ diff --git a/cometbft/v0.39/docs/qa/img34/cmt2tm1/avg_cpu.png b/cometbft/v0.39/docs/qa/img34/cmt2tm1/avg_cpu.png new file mode 100644 index 000000000..92fea31bd Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/cmt2tm1/avg_cpu.png differ diff --git a/cometbft/v0.39/docs/qa/img34/cmt2tm1/avg_memory.png b/cometbft/v0.39/docs/qa/img34/cmt2tm1/avg_memory.png new file mode 100644 index 000000000..f362798d8 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/cmt2tm1/avg_memory.png differ diff --git a/cometbft/v0.39/docs/qa/img34/cmt2tm1/avg_mempool_size.png b/cometbft/v0.39/docs/qa/img34/cmt2tm1/avg_mempool_size.png new file mode 100644 index 000000000..b73e577b7 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/cmt2tm1/avg_mempool_size.png differ diff --git a/cometbft/v0.39/docs/qa/img34/cmt2tm1/block_rate_regular.png b/cometbft/v0.39/docs/qa/img34/cmt2tm1/block_rate_regular.png new file mode 100644 index 000000000..5fc7a5560 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/cmt2tm1/block_rate_regular.png differ diff --git a/cometbft/v0.39/docs/qa/img34/cmt2tm1/cpu.png b/cometbft/v0.39/docs/qa/img34/cmt2tm1/cpu.png new file mode 100644 index 000000000..15df58abb Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/cmt2tm1/cpu.png differ diff --git a/cometbft/v0.39/docs/qa/img34/cmt2tm1/memory.png b/cometbft/v0.39/docs/qa/img34/cmt2tm1/memory.png new file mode 100644 index 000000000..b0feab107 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/cmt2tm1/memory.png differ diff --git a/cometbft/v0.39/docs/qa/img34/cmt2tm1/mempool_size.png b/cometbft/v0.39/docs/qa/img34/cmt2tm1/mempool_size.png new file mode 100644 index 000000000..b3a1514f9 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/cmt2tm1/mempool_size.png differ diff --git a/cometbft/v0.39/docs/qa/img34/cmt2tm1/peers.png b/cometbft/v0.39/docs/qa/img34/cmt2tm1/peers.png new file mode 100644 index 000000000..558d4c129 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/cmt2tm1/peers.png differ diff --git a/cometbft/v0.39/docs/qa/img34/cmt2tm1/rounds.png b/cometbft/v0.39/docs/qa/img34/cmt2tm1/rounds.png new file mode 100644 index 000000000..3c22a5cf3 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/cmt2tm1/rounds.png differ diff --git a/cometbft/v0.39/docs/qa/img34/cmt2tm1/total_txs_rate_regular.png b/cometbft/v0.39/docs/qa/img34/cmt2tm1/total_txs_rate_regular.png new file mode 100644 index 000000000..ae98df217 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/cmt2tm1/total_txs_rate_regular.png differ diff --git a/cometbft/v0.39/docs/qa/img34/homogeneous/all_experiments.png b/cometbft/v0.39/docs/qa/img34/homogeneous/all_experiments.png new file mode 100644 index 000000000..d8768f6a5 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/homogeneous/all_experiments.png differ diff --git a/cometbft/v0.39/docs/qa/img34/homogeneous/avg_cpu.png b/cometbft/v0.39/docs/qa/img34/homogeneous/avg_cpu.png new file mode 100644 index 000000000..7df188951 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/homogeneous/avg_cpu.png differ diff --git a/cometbft/v0.39/docs/qa/img34/homogeneous/avg_memory.png b/cometbft/v0.39/docs/qa/img34/homogeneous/avg_memory.png new file mode 100644 index 000000000..e800cbce2 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/homogeneous/avg_memory.png differ diff --git a/cometbft/v0.39/docs/qa/img34/homogeneous/avg_mempool_size.png b/cometbft/v0.39/docs/qa/img34/homogeneous/avg_mempool_size.png new file mode 100644 index 000000000..beb323e64 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/homogeneous/avg_mempool_size.png differ diff --git a/cometbft/v0.39/docs/qa/img34/homogeneous/block_rate_regular.png b/cometbft/v0.39/docs/qa/img34/homogeneous/block_rate_regular.png new file mode 100644 index 000000000..2a71ab70d Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/homogeneous/block_rate_regular.png differ diff --git a/cometbft/v0.39/docs/qa/img34/homogeneous/cpu.png b/cometbft/v0.39/docs/qa/img34/homogeneous/cpu.png new file mode 100644 index 000000000..8e8c9227a Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/homogeneous/cpu.png differ diff --git a/cometbft/v0.39/docs/qa/img34/homogeneous/memory.png b/cometbft/v0.39/docs/qa/img34/homogeneous/memory.png new file mode 100644 index 000000000..190c622a3 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/homogeneous/memory.png differ diff --git a/cometbft/v0.39/docs/qa/img34/homogeneous/mempool_size.png b/cometbft/v0.39/docs/qa/img34/homogeneous/mempool_size.png new file mode 100644 index 000000000..ec1c79a24 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/homogeneous/mempool_size.png differ diff --git a/cometbft/v0.39/docs/qa/img34/homogeneous/peers.png b/cometbft/v0.39/docs/qa/img34/homogeneous/peers.png new file mode 100644 index 000000000..3c8b0a2e0 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/homogeneous/peers.png differ diff --git a/cometbft/v0.39/docs/qa/img34/homogeneous/rounds.png b/cometbft/v0.39/docs/qa/img34/homogeneous/rounds.png new file mode 100644 index 000000000..660f31d93 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/homogeneous/rounds.png differ diff --git a/cometbft/v0.39/docs/qa/img34/homogeneous/total_txs_rate_regular.png b/cometbft/v0.39/docs/qa/img34/homogeneous/total_txs_rate_regular.png new file mode 100644 index 000000000..a9025b666 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/homogeneous/total_txs_rate_regular.png differ diff --git a/cometbft/v0.39/docs/qa/img34/v034_200node_latencies.png b/cometbft/v0.39/docs/qa/img34/v034_200node_latencies.png new file mode 100644 index 000000000..afd1060ca Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/v034_200node_latencies.png differ diff --git a/cometbft/v0.39/docs/qa/img34/v034_200node_latencies_zoomed.png b/cometbft/v0.39/docs/qa/img34/v034_200node_latencies_zoomed.png new file mode 100644 index 000000000..1ff936442 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/v034_200node_latencies_zoomed.png differ diff --git a/cometbft/v0.39/docs/qa/img34/v034_200node_tm2cmt1/all_experiments.png b/cometbft/v0.39/docs/qa/img34/v034_200node_tm2cmt1/all_experiments.png new file mode 100644 index 000000000..e91a87eff Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/v034_200node_tm2cmt1/all_experiments.png differ diff --git a/cometbft/v0.39/docs/qa/img34/v034_200node_tm2cmt1/avg_cpu.png b/cometbft/v0.39/docs/qa/img34/v034_200node_tm2cmt1/avg_cpu.png new file mode 100644 index 000000000..a1b0ef79e Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/v034_200node_tm2cmt1/avg_cpu.png differ diff --git a/cometbft/v0.39/docs/qa/img34/v034_200node_tm2cmt1/avg_memory.png b/cometbft/v0.39/docs/qa/img34/v034_200node_tm2cmt1/avg_memory.png new file mode 100644 index 000000000..f9d9b9933 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/v034_200node_tm2cmt1/avg_memory.png differ diff --git a/cometbft/v0.39/docs/qa/img34/v034_200node_tm2cmt1/avg_mempool_size.png b/cometbft/v0.39/docs/qa/img34/v034_200node_tm2cmt1/avg_mempool_size.png new file mode 100644 index 000000000..c2b896060 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/v034_200node_tm2cmt1/avg_mempool_size.png differ diff --git a/cometbft/v0.39/docs/qa/img34/v034_200node_tm2cmt1/block_rate_regular.png b/cometbft/v0.39/docs/qa/img34/v034_200node_tm2cmt1/block_rate_regular.png new file mode 100644 index 000000000..5a5417bdf Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/v034_200node_tm2cmt1/block_rate_regular.png differ diff --git a/cometbft/v0.39/docs/qa/img34/v034_200node_tm2cmt1/c2r200_merged.png b/cometbft/v0.39/docs/qa/img34/v034_200node_tm2cmt1/c2r200_merged.png new file mode 100644 index 000000000..45de9ce72 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/v034_200node_tm2cmt1/c2r200_merged.png differ diff --git a/cometbft/v0.39/docs/qa/img34/v034_200node_tm2cmt1/cpu.png b/cometbft/v0.39/docs/qa/img34/v034_200node_tm2cmt1/cpu.png new file mode 100644 index 000000000..eabfa9661 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/v034_200node_tm2cmt1/cpu.png differ diff --git a/cometbft/v0.39/docs/qa/img34/v034_200node_tm2cmt1/memory.png b/cometbft/v0.39/docs/qa/img34/v034_200node_tm2cmt1/memory.png new file mode 100644 index 000000000..70014c1f9 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/v034_200node_tm2cmt1/memory.png differ diff --git a/cometbft/v0.39/docs/qa/img34/v034_200node_tm2cmt1/mempool_size.png b/cometbft/v0.39/docs/qa/img34/v034_200node_tm2cmt1/mempool_size.png new file mode 100644 index 000000000..5f4c44b2a Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/v034_200node_tm2cmt1/mempool_size.png differ diff --git a/cometbft/v0.39/docs/qa/img34/v034_200node_tm2cmt1/peers.png b/cometbft/v0.39/docs/qa/img34/v034_200node_tm2cmt1/peers.png new file mode 100644 index 000000000..c35c84675 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/v034_200node_tm2cmt1/peers.png differ diff --git a/cometbft/v0.39/docs/qa/img34/v034_200node_tm2cmt1/rounds.png b/cometbft/v0.39/docs/qa/img34/v034_200node_tm2cmt1/rounds.png new file mode 100644 index 000000000..7d1034bcb Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/v034_200node_tm2cmt1/rounds.png differ diff --git a/cometbft/v0.39/docs/qa/img34/v034_200node_tm2cmt1/total_txs_rate_regular.png b/cometbft/v0.39/docs/qa/img34/v034_200node_tm2cmt1/total_txs_rate_regular.png new file mode 100644 index 000000000..2e8a40af6 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/v034_200node_tm2cmt1/total_txs_rate_regular.png differ diff --git a/cometbft/v0.39/docs/qa/img34/v034_latency_throughput.png b/cometbft/v0.39/docs/qa/img34/v034_latency_throughput.png new file mode 100644 index 000000000..3674fe47b Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/v034_latency_throughput.png differ diff --git a/cometbft/v0.39/docs/qa/img34/v034_r200c2_heights.png b/cometbft/v0.39/docs/qa/img34/v034_r200c2_heights.png new file mode 100644 index 000000000..11f3bba43 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/v034_r200c2_heights.png differ diff --git a/cometbft/v0.39/docs/qa/img34/v034_r200c2_load-runner.png b/cometbft/v0.39/docs/qa/img34/v034_r200c2_load-runner.png new file mode 100644 index 000000000..70211b0d2 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/v034_r200c2_load-runner.png differ diff --git a/cometbft/v0.39/docs/qa/img34/v034_r200c2_load1.png b/cometbft/v0.39/docs/qa/img34/v034_r200c2_load1.png new file mode 100644 index 000000000..11012844d Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/v034_r200c2_load1.png differ diff --git a/cometbft/v0.39/docs/qa/img34/v034_r200c2_mempool_size.png b/cometbft/v0.39/docs/qa/img34/v034_r200c2_mempool_size.png new file mode 100644 index 000000000..c5d690200 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/v034_r200c2_mempool_size.png differ diff --git a/cometbft/v0.39/docs/qa/img34/v034_r200c2_mempool_size_avg.png b/cometbft/v0.39/docs/qa/img34/v034_r200c2_mempool_size_avg.png new file mode 100644 index 000000000..bda399fe5 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/v034_r200c2_mempool_size_avg.png differ diff --git a/cometbft/v0.39/docs/qa/img34/v034_r200c2_peers.png b/cometbft/v0.39/docs/qa/img34/v034_r200c2_peers.png new file mode 100644 index 000000000..a0aea7ada Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/v034_r200c2_peers.png differ diff --git a/cometbft/v0.39/docs/qa/img34/v034_r200c2_rounds.png b/cometbft/v0.39/docs/qa/img34/v034_r200c2_rounds.png new file mode 100644 index 000000000..215be100d Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/v034_r200c2_rounds.png differ diff --git a/cometbft/v0.39/docs/qa/img34/v034_r200c2_rss.png b/cometbft/v0.39/docs/qa/img34/v034_r200c2_rss.png new file mode 100644 index 000000000..6d14dced0 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/v034_r200c2_rss.png differ diff --git a/cometbft/v0.39/docs/qa/img34/v034_r200c2_rss_avg.png b/cometbft/v0.39/docs/qa/img34/v034_r200c2_rss_avg.png new file mode 100644 index 000000000..8dec67da2 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/v034_r200c2_rss_avg.png differ diff --git a/cometbft/v0.39/docs/qa/img34/v034_r200c2_total-txs.png b/cometbft/v0.39/docs/qa/img34/v034_r200c2_total-txs.png new file mode 100644 index 000000000..177d5f1c3 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/v034_r200c2_total-txs.png differ diff --git a/cometbft/v0.39/docs/qa/img34/v034_report_tabbed.txt b/cometbft/v0.39/docs/qa/img34/v034_report_tabbed.txt new file mode 100644 index 000000000..251495474 --- /dev/null +++ b/cometbft/v0.39/docs/qa/img34/v034_report_tabbed.txt @@ -0,0 +1,52 @@ +Experiment ID: 3d5cf4ef-1a1a-4b46-aa2d-da5643d2e81e │Experiment ID: 80e472ec-13a1-4772-a827-3b0c907fb51d │Experiment ID: 07aca6cf-c5a4-4696-988f-e3270fc6333b + │ │ + Connections: 1 │ Connections: 2 │ Connections: 4 + Rate: 25 │ Rate: 25 │ Rate: 25 + Size: 1024 │ Size: 1024 │ Size: 1024 + │ │ + Total Valid Tx: 2225 │ Total Valid Tx: 4450 │ Total Valid Tx: 8900 + Total Negative Latencies: 0 │ Total Negative Latencies: 0 │ Total Negative Latencies: 0 + Minimum Latency: 599.404362ms │ Minimum Latency: 448.145181ms │ Minimum Latency: 412.485729ms + Maximum Latency: 3.539686885s │ Maximum Latency: 3.237392049s │ Maximum Latency: 12.026665368s + Average Latency: 1.441485349s │ Average Latency: 1.441267946s │ Average Latency: 2.150192457s + Standard Deviation: 541.049869ms │ Standard Deviation: 525.040007ms │ Standard Deviation: 2.233852478s + │ │ +Experiment ID: 953dc544-dd40-40e8-8712-20c34c3ce45e │Experiment ID: d31fc258-16e7-45cd-9dc8-13ab87bc0b0a │Experiment ID: 15d90a7e-b941-42f4-b411-2f15f857739e + │ │ + Connections: 1 │ Connections: 2 │ Connections: 4 + Rate: 50 │ Rate: 50 │ Rate: 50 + Size: 1024 │ Size: 1024 │ Size: 1024 + │ │ + Total Valid Tx: 4450 │ Total Valid Tx: 8900 │ Total Valid Tx: 17800 + Total Negative Latencies: 0 │ Total Negative Latencies: 0 │ Total Negative Latencies: 0 + Minimum Latency: 482.046942ms │ Minimum Latency: 435.458913ms │ Minimum Latency: 510.746448ms + Maximum Latency: 3.761483455s │ Maximum Latency: 7.175583584s │ Maximum Latency: 6.551497882s + Average Latency: 1.450408183s │ Average Latency: 1.681673116s │ Average Latency: 1.738083875s + Standard Deviation: 587.560056ms │ Standard Deviation: 1.147902047s │ Standard Deviation: 943.46522ms + │ │ +Experiment ID: 9a0b9980-9ce6-4db5-a80a-65ca70294b87 │Experiment ID: df8fa4f4-80af-4ded-8a28-356d15018b43 │Experiment ID: d0e41c2c-89c0-4f38-8e34-ca07adae593a + │ │ + Connections: 1 │ Connections: 2 │ Connections: 4 + Rate: 100 │ Rate: 100 │ Rate: 100 + Size: 1024 │ Size: 1024 │ Size: 1024 + │ │ + Total Valid Tx: 8900 │ Total Valid Tx: 17800 │ Total Valid Tx: 35600 + Total Negative Latencies: 0 │ Total Negative Latencies: 0 │ Total Negative Latencies: 0 + Minimum Latency: 477.417219ms │ Minimum Latency: 564.29247ms │ Minimum Latency: 840.71089ms + Maximum Latency: 6.63744785s │ Maximum Latency: 6.988553219s │ Maximum Latency: 9.555312398s + Average Latency: 1.561216103s │ Average Latency: 1.76419063s │ Average Latency: 3.200941683s + Standard Deviation: 1.011333552s │ Standard Deviation: 1.068459423s │ Standard Deviation: 1.732346601s + │ │ +Experiment ID: 493df3ee-4a36-4bce-80f8-6d65da66beda │Experiment ID: 13060525-f04f-46f6-8ade-286684b2fe50 │Experiment ID: 1777cbd2-8c96-42e4-9ec7-9b21f2225e4d + │ │ + Connections: 1 │ Connections: 2 │ Connections: 4 + Rate: 200 │ Rate: 200 │ Rate: 200 + Size: 1024 │ Size: 1024 │ Size: 1024 + │ │ + Total Valid Tx: 17800 │ Total Valid Tx: 35600 │ Total Valid Tx: 38660 + Total Negative Latencies: 0 │ Total Negative Latencies: 0 │ Total Negative Latencies: 0 + Minimum Latency: 493.705261ms │ Minimum Latency: 955.090573ms │ Minimum Latency: 1.9485821s + Maximum Latency: 7.440921872s │ Maximum Latency: 10.086673491s │ Maximum Latency: 17.73103976s + Average Latency: 1.875510582s │ Average Latency: 3.438130099s │ Average Latency: 8.143862237s + Standard Deviation: 1.304336995s │ Standard Deviation: 1.966391574s │ Standard Deviation: 3.943140002s + diff --git a/cometbft/v0.39/docs/qa/img34/v034_rotating_heights.png b/cometbft/v0.39/docs/qa/img34/v034_rotating_heights.png new file mode 100644 index 000000000..47913c282 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/v034_rotating_heights.png differ diff --git a/cometbft/v0.39/docs/qa/img34/v034_rotating_heights_ephe.png b/cometbft/v0.39/docs/qa/img34/v034_rotating_heights_ephe.png new file mode 100644 index 000000000..981b93d6c Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/v034_rotating_heights_ephe.png differ diff --git a/cometbft/v0.39/docs/qa/img34/v034_rotating_latencies.png b/cometbft/v0.39/docs/qa/img34/v034_rotating_latencies.png new file mode 100644 index 000000000..f0a54ed5b Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/v034_rotating_latencies.png differ diff --git a/cometbft/v0.39/docs/qa/img34/v034_rotating_latencies_uniq.png b/cometbft/v0.39/docs/qa/img34/v034_rotating_latencies_uniq.png new file mode 100644 index 000000000..e5d694a16 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/v034_rotating_latencies_uniq.png differ diff --git a/cometbft/v0.39/docs/qa/img34/v034_rotating_load1.png b/cometbft/v0.39/docs/qa/img34/v034_rotating_load1.png new file mode 100644 index 000000000..e9c385b85 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/v034_rotating_load1.png differ diff --git a/cometbft/v0.39/docs/qa/img34/v034_rotating_peers.png b/cometbft/v0.39/docs/qa/img34/v034_rotating_peers.png new file mode 100644 index 000000000..ab5c8732d Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/v034_rotating_peers.png differ diff --git a/cometbft/v0.39/docs/qa/img34/v034_rotating_rss_avg.png b/cometbft/v0.39/docs/qa/img34/v034_rotating_rss_avg.png new file mode 100644 index 000000000..9a4167320 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/v034_rotating_rss_avg.png differ diff --git a/cometbft/v0.39/docs/qa/img34/v034_rotating_total-txs.png b/cometbft/v0.39/docs/qa/img34/v034_rotating_total-txs.png new file mode 100644 index 000000000..1ce5f47e9 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img34/v034_rotating_total-txs.png differ diff --git a/cometbft/v0.39/docs/qa/img37/200nodes_cmt037/all_experiments.png b/cometbft/v0.39/docs/qa/img37/200nodes_cmt037/all_experiments.png new file mode 100644 index 000000000..a61896101 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img37/200nodes_cmt037/all_experiments.png differ diff --git a/cometbft/v0.39/docs/qa/img37/200nodes_cmt037/avg_mempool_size.png b/cometbft/v0.39/docs/qa/img37/200nodes_cmt037/avg_mempool_size.png new file mode 100644 index 000000000..5a0f1a7b4 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img37/200nodes_cmt037/avg_mempool_size.png differ diff --git a/cometbft/v0.39/docs/qa/img37/200nodes_cmt037/block_rate.png b/cometbft/v0.39/docs/qa/img37/200nodes_cmt037/block_rate.png new file mode 100644 index 000000000..13ddf74be Binary files /dev/null and b/cometbft/v0.39/docs/qa/img37/200nodes_cmt037/block_rate.png differ diff --git a/cometbft/v0.39/docs/qa/img37/200nodes_cmt037/cpu.png b/cometbft/v0.39/docs/qa/img37/200nodes_cmt037/cpu.png new file mode 100644 index 000000000..518bc96b1 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img37/200nodes_cmt037/cpu.png differ diff --git a/cometbft/v0.39/docs/qa/img37/200nodes_cmt037/e_75cb89a8-f876-4698-82f3-8aaab0b361af.png b/cometbft/v0.39/docs/qa/img37/200nodes_cmt037/e_75cb89a8-f876-4698-82f3-8aaab0b361af.png new file mode 100644 index 000000000..0909fb0ec Binary files /dev/null and b/cometbft/v0.39/docs/qa/img37/200nodes_cmt037/e_75cb89a8-f876-4698-82f3-8aaab0b361af.png differ diff --git a/cometbft/v0.39/docs/qa/img37/200nodes_cmt037/memory.png b/cometbft/v0.39/docs/qa/img37/200nodes_cmt037/memory.png new file mode 100644 index 000000000..c31affad6 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img37/200nodes_cmt037/memory.png differ diff --git a/cometbft/v0.39/docs/qa/img37/200nodes_cmt037/mempool_size.png b/cometbft/v0.39/docs/qa/img37/200nodes_cmt037/mempool_size.png new file mode 100644 index 000000000..92f620df7 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img37/200nodes_cmt037/mempool_size.png differ diff --git a/cometbft/v0.39/docs/qa/img37/200nodes_cmt037/peers.png b/cometbft/v0.39/docs/qa/img37/200nodes_cmt037/peers.png new file mode 100644 index 000000000..61d3e76cf Binary files /dev/null and b/cometbft/v0.39/docs/qa/img37/200nodes_cmt037/peers.png differ diff --git a/cometbft/v0.39/docs/qa/img37/200nodes_cmt037/rounds.png b/cometbft/v0.39/docs/qa/img37/200nodes_cmt037/rounds.png new file mode 100644 index 000000000..f6bc51446 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img37/200nodes_cmt037/rounds.png differ diff --git a/cometbft/v0.39/docs/qa/img37/200nodes_cmt037/total_txs_rate.png b/cometbft/v0.39/docs/qa/img37/200nodes_cmt037/total_txs_rate.png new file mode 100644 index 000000000..d3c0dbef9 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img37/200nodes_cmt037/total_txs_rate.png differ diff --git a/cometbft/v0.39/docs/qa/img37/200nodes_tm037/avg_mempool_size.png b/cometbft/v0.39/docs/qa/img37/200nodes_tm037/avg_mempool_size.png new file mode 100644 index 000000000..9d95b25d7 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img37/200nodes_tm037/avg_mempool_size.png differ diff --git a/cometbft/v0.39/docs/qa/img37/200nodes_tm037/block_rate_regular.png b/cometbft/v0.39/docs/qa/img37/200nodes_tm037/block_rate_regular.png new file mode 100644 index 000000000..999288e36 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img37/200nodes_tm037/block_rate_regular.png differ diff --git a/cometbft/v0.39/docs/qa/img37/200nodes_tm037/cpu.png b/cometbft/v0.39/docs/qa/img37/200nodes_tm037/cpu.png new file mode 100644 index 000000000..6a11aeede Binary files /dev/null and b/cometbft/v0.39/docs/qa/img37/200nodes_tm037/cpu.png differ diff --git a/cometbft/v0.39/docs/qa/img37/200nodes_tm037/memory.png b/cometbft/v0.39/docs/qa/img37/200nodes_tm037/memory.png new file mode 100644 index 000000000..98f5893de Binary files /dev/null and b/cometbft/v0.39/docs/qa/img37/200nodes_tm037/memory.png differ diff --git a/cometbft/v0.39/docs/qa/img37/200nodes_tm037/mempool_size.png b/cometbft/v0.39/docs/qa/img37/200nodes_tm037/mempool_size.png new file mode 100644 index 000000000..7b0d9e298 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img37/200nodes_tm037/mempool_size.png differ diff --git a/cometbft/v0.39/docs/qa/img37/200nodes_tm037/peers.png b/cometbft/v0.39/docs/qa/img37/200nodes_tm037/peers.png new file mode 100644 index 000000000..ff566edf7 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img37/200nodes_tm037/peers.png differ diff --git a/cometbft/v0.39/docs/qa/img37/200nodes_tm037/rounds.png b/cometbft/v0.39/docs/qa/img37/200nodes_tm037/rounds.png new file mode 100644 index 000000000..deb201f76 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img37/200nodes_tm037/rounds.png differ diff --git a/cometbft/v0.39/docs/qa/img37/200nodes_tm037/total_txs_rate_regular.png b/cometbft/v0.39/docs/qa/img37/200nodes_tm037/total_txs_rate_regular.png new file mode 100644 index 000000000..08316dadb Binary files /dev/null and b/cometbft/v0.39/docs/qa/img37/200nodes_tm037/total_txs_rate_regular.png differ diff --git a/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_200node_latencies.png b/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_200node_latencies.png new file mode 100644 index 000000000..ad469bb29 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_200node_latencies.png differ diff --git a/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_latency_throughput.png b/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_latency_throughput.png new file mode 100644 index 000000000..baf34b2c7 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_latency_throughput.png differ diff --git a/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_r200c2_heights.png b/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_r200c2_heights.png new file mode 100644 index 000000000..360283f14 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_r200c2_heights.png differ diff --git a/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_r200c2_load1.png b/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_r200c2_load1.png new file mode 100644 index 000000000..11d6dfcf7 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_r200c2_load1.png differ diff --git a/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_r200c2_mempool_size.png b/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_r200c2_mempool_size.png new file mode 100644 index 000000000..a2f3bd401 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_r200c2_mempool_size.png differ diff --git a/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_r200c2_mempool_size_avg.png b/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_r200c2_mempool_size_avg.png new file mode 100644 index 000000000..480d4aebc Binary files /dev/null and b/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_r200c2_mempool_size_avg.png differ diff --git a/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_r200c2_peers.png b/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_r200c2_peers.png new file mode 100644 index 000000000..222da73f6 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_r200c2_peers.png differ diff --git a/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_r200c2_rounds.png b/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_r200c2_rounds.png new file mode 100644 index 000000000..7afaaac57 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_r200c2_rounds.png differ diff --git a/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_r200c2_rss.png b/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_r200c2_rss.png new file mode 100644 index 000000000..730a1bc49 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_r200c2_rss.png differ diff --git a/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_r200c2_rss_avg.png b/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_r200c2_rss_avg.png new file mode 100644 index 000000000..3f6cf9f6d Binary files /dev/null and b/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_r200c2_rss_avg.png differ diff --git a/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_r200c2_total-txs.png b/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_r200c2_total-txs.png new file mode 100644 index 000000000..62dced2c8 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_r200c2_total-txs.png differ diff --git a/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_report_tabbed.txt b/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_report_tabbed.txt new file mode 100644 index 000000000..aa4aa4e60 --- /dev/null +++ b/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_report_tabbed.txt @@ -0,0 +1,52 @@ +Experiment ID: af129eae-7039-4c76-8c37-cff9ac636a84 │Experiment ID: 0f88bd33-9bf0-4197-8d1d-9a737c301ec6 │Experiment ID: 88227cad-2ba8-4eb6-b493-041d8120b46f + │ │ + Connections: 1 │ Connections: 2 │ Connections: 4 + Rate: 25 │ Rate: 25 │ Rate: 25 + Size: 1024 │ Size: 1024 │ Size: 1024 + │ │ + Total Valid Tx: 2225 │ Total Valid Tx: 4450 │ Total Valid Tx: 8900 + Total Negative Latencies: 0 │ Total Negative Latencies: 0 │ Total Negative Latencies: 0 + Minimum Latency: 506.248587ms │ Minimum Latency: 469.53452ms │ Minimum Latency: 588.900721ms + Maximum Latency: 3.032125789s │ Maximum Latency: 6.548830955s │ Maximum Latency: 6.533739843s + Average Latency: 1.427767726s │ Average Latency: 1.448582257s │ Average Latency: 1.717432341s + Standard Deviation: 524.11782ms │ Standard Deviation: 768.684133ms │ Standard Deviation: 1.000015768s + │ │ +Experiment ID: f03d39bd-0233-4b3c-b461-543445ae1d4b │Experiment ID: 46674f1c-e591-4e36-bb9b-f375c19fc475 │Experiment ID: 5385c159-8d4d-455b-bced-dcd4a3209988 + │ │ + Connections: 1 │ Connections: 2 │ Connections: 4 + Rate: 50 │ Rate: 50 │ Rate: 50 + Size: 1024 │ Size: 1024 │ Size: 1024 + │ │ + Total Valid Tx: 4450 │ Total Valid Tx: 8900 │ Total Valid Tx: 17800 + Total Negative Latencies: 0 │ Total Negative Latencies: 0 │ Total Negative Latencies: 0 + Minimum Latency: 477.46027ms │ Minimum Latency: 455.757111ms │ Minimum Latency: 594.749081ms + Maximum Latency: 2.483895394s │ Maximum Latency: 2.904715695s │ Maximum Latency: 9.294950389s + Average Latency: 1.407374662s │ Average Latency: 1.397385779s │ Average Latency: 2.621122536s + Standard Deviation: 505.150067ms │ Standard Deviation: 551.67603ms │ Standard Deviation: 1.772725794s + │ │ +Experiment ID: 9161b4a7-d75c-455f-b82d-2b5235d533cf │Experiment ID: 993a13a8-9db1-4b2b-9c20-71a5b85e4bbf │Experiment ID: ad1eb9e1-f4d6-41fd-9ba7-0f1f7dde1e3e + │ │ + Connections: 1 │ Connections: 2 │ Connections: 4 + Rate: 100 │ Rate: 100 │ Rate: 100 + Size: 1024 │ Size: 1024 │ Size: 1024 + │ │ + Total Valid Tx: 8900 │ Total Valid Tx: 17800 │ Total Valid Tx: 35400 + Total Negative Latencies: 0 │ Total Negative Latencies: 0 │ Total Negative Latencies: 0 + Minimum Latency: 448.050467ms │ Minimum Latency: 605.436195ms │ Minimum Latency: 1.16816912s + Maximum Latency: 3.789711139s │ Maximum Latency: 7.292770222s │ Maximum Latency: 11.378681842s + Average Latency: 1.451342158s │ Average Latency: 2.07457999s │ Average Latency: 3.918384209s + Standard Deviation: 644.075973ms │ Standard Deviation: 1.230204022s │ Standard Deviation: 2.172400458s + │ │ +Experiment ID: 3cbe9c3d-9c43-4c9f-b5ca-b567d20bbd57 │Experiment ID: af836c5e-d9b6-4d5d-971c-2fc7f07aa2a0 │Experiment ID: 77606397-4989-41d4-b13b-f1f4d1af063f + │ │ + Connections: 1 │ Connections: 2 │ Connections: 4 + Rate: 200 │ Rate: 200 │ Rate: 200 + Size: 1024 │ Size: 1024 │ Size: 1024 + │ │ + Total Valid Tx: 17800 │ Total Valid Tx: 35600 │ Total Valid Tx: 37358 + Total Negative Latencies: 0 │ Total Negative Latencies: 0 │ Total Negative Latencies: 0 + Minimum Latency: 519.984701ms │ Minimum Latency: 820.755087ms │ Minimum Latency: 1.712574804s + Maximum Latency: 12.609056712s │ Maximum Latency: 9.260798095s │ Maximum Latency: 25.739223696s + Average Latency: 2.717853101s │ Average Latency: 3.477731881s │ Average Latency: 8.547725264s + Standard Deviation: 2.390778155s │ Standard Deviation: 1.675000913s │ Standard Deviation: 4.76961569s + diff --git a/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_rotating_heights.png b/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_rotating_heights.png new file mode 100644 index 000000000..882de51e4 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_rotating_heights.png differ diff --git a/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_rotating_heights_ephe.png b/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_rotating_heights_ephe.png new file mode 100644 index 000000000..1ab2521e8 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_rotating_heights_ephe.png differ diff --git a/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_rotating_latencies.png b/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_rotating_latencies.png new file mode 100644 index 000000000..94548c8b9 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_rotating_latencies.png differ diff --git a/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_rotating_load1.png b/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_rotating_load1.png new file mode 100644 index 000000000..03b7412da Binary files /dev/null and b/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_rotating_load1.png differ diff --git a/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_rotating_peers.png b/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_rotating_peers.png new file mode 100644 index 000000000..86304760b Binary files /dev/null and b/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_rotating_peers.png differ diff --git a/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_rotating_rss_avg.png b/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_rotating_rss_avg.png new file mode 100644 index 000000000..d45c045b7 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_rotating_rss_avg.png differ diff --git a/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_rotating_total-txs.png b/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_rotating_total-txs.png new file mode 100644 index 000000000..50b4c2e3f Binary files /dev/null and b/cometbft/v0.39/docs/qa/img37/200nodes_tm037/v037_rotating_total-txs.png differ diff --git a/cometbft/v0.39/docs/qa/img37/rotating/rotating_avg_memory.png b/cometbft/v0.39/docs/qa/img37/rotating/rotating_avg_memory.png new file mode 100644 index 000000000..7feb2e812 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img37/rotating/rotating_avg_memory.png differ diff --git a/cometbft/v0.39/docs/qa/img37/rotating/rotating_block_rate.png b/cometbft/v0.39/docs/qa/img37/rotating/rotating_block_rate.png new file mode 100644 index 000000000..4bbc3c999 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img37/rotating/rotating_block_rate.png differ diff --git a/cometbft/v0.39/docs/qa/img37/rotating/rotating_cpu.png b/cometbft/v0.39/docs/qa/img37/rotating/rotating_cpu.png new file mode 100644 index 000000000..ef4c7d30d Binary files /dev/null and b/cometbft/v0.39/docs/qa/img37/rotating/rotating_cpu.png differ diff --git a/cometbft/v0.39/docs/qa/img37/rotating/rotating_eph_heights.png b/cometbft/v0.39/docs/qa/img37/rotating/rotating_eph_heights.png new file mode 100644 index 000000000..36850fb52 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img37/rotating/rotating_eph_heights.png differ diff --git a/cometbft/v0.39/docs/qa/img37/rotating/rotating_peers.png b/cometbft/v0.39/docs/qa/img37/rotating/rotating_peers.png new file mode 100644 index 000000000..c45a4d4d7 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img37/rotating/rotating_peers.png differ diff --git a/cometbft/v0.39/docs/qa/img37/rotating/rotating_txs_rate.png b/cometbft/v0.39/docs/qa/img37/rotating/rotating_txs_rate.png new file mode 100644 index 000000000..5462738e5 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img37/rotating/rotating_txs_rate.png differ diff --git a/cometbft/v0.39/docs/qa/img38/200nodes/avg_mempool_size.png b/cometbft/v0.39/docs/qa/img38/200nodes/avg_mempool_size.png new file mode 100644 index 000000000..36cd5f6c7 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/200nodes/avg_mempool_size.png differ diff --git a/cometbft/v0.39/docs/qa/img38/200nodes/block_rate.png b/cometbft/v0.39/docs/qa/img38/200nodes/block_rate.png new file mode 100644 index 000000000..b2042865d Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/200nodes/block_rate.png differ diff --git a/cometbft/v0.39/docs/qa/img38/200nodes/c1r400.png b/cometbft/v0.39/docs/qa/img38/200nodes/c1r400.png new file mode 100644 index 000000000..0c27c144f Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/200nodes/c1r400.png differ diff --git a/cometbft/v0.39/docs/qa/img38/200nodes/cpu.png b/cometbft/v0.39/docs/qa/img38/200nodes/cpu.png new file mode 100644 index 000000000..15f74aeb0 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/200nodes/cpu.png differ diff --git a/cometbft/v0.39/docs/qa/img38/200nodes/e_de676ecf-038e-443f-a26a-27915f29e312.png b/cometbft/v0.39/docs/qa/img38/200nodes/e_de676ecf-038e-443f-a26a-27915f29e312.png new file mode 100644 index 000000000..21f5cab6e Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/200nodes/e_de676ecf-038e-443f-a26a-27915f29e312.png differ diff --git a/cometbft/v0.39/docs/qa/img38/200nodes/memory.png b/cometbft/v0.39/docs/qa/img38/200nodes/memory.png new file mode 100644 index 000000000..3e01c4ccd Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/200nodes/memory.png differ diff --git a/cometbft/v0.39/docs/qa/img38/200nodes/mempool_size.png b/cometbft/v0.39/docs/qa/img38/200nodes/mempool_size.png new file mode 100644 index 000000000..6a897a84a Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/200nodes/mempool_size.png differ diff --git a/cometbft/v0.39/docs/qa/img38/200nodes/peers.png b/cometbft/v0.39/docs/qa/img38/200nodes/peers.png new file mode 100644 index 000000000..874700519 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/200nodes/peers.png differ diff --git a/cometbft/v0.39/docs/qa/img38/200nodes/rounds.png b/cometbft/v0.39/docs/qa/img38/200nodes/rounds.png new file mode 100644 index 000000000..5ac53cad8 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/200nodes/rounds.png differ diff --git a/cometbft/v0.39/docs/qa/img38/200nodes/total_txs_rate.png b/cometbft/v0.39/docs/qa/img38/200nodes/total_txs_rate.png new file mode 100644 index 000000000..ac7f5e686 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/200nodes/total_txs_rate.png differ diff --git a/cometbft/v0.39/docs/qa/img38/200nodes/v038_report_tabbed.txt b/cometbft/v0.39/docs/qa/img38/200nodes/v038_report_tabbed.txt new file mode 100644 index 000000000..c482aeac8 --- /dev/null +++ b/cometbft/v0.39/docs/qa/img38/200nodes/v038_report_tabbed.txt @@ -0,0 +1,40 @@ +Experiment ID: 93024f38-a008-443d-9aa7-9ac44c9fe15b Experiment ID: d65a486e-4712-41b5-9f41-97e491895d2e Experiment ID: 9c39184b-b8c7-46a2-bacb-40f9961fb7a1 + Connections: 1 Connections: 2 Connections: 4 + Rate: 200 Rate: 200 Rate: 200 + Size: 1024 Size: 1024 Size: 1024 + Total Valid Tx: 17800 Total Valid Tx: 33259 Total Valid Tx: 33259 + Total Negative Latencies: 0 Total Negative Latencies: 0 Total Negative Latencies: 0 + Minimum Latency: 562.805076ms Minimum Latency: 894.026089ms Minimum Latency: 2.166875257s + Maximum Latency: 7.623963559s Maximum Latency: 16.941216187s Maximum Latency: 15.701598288s + Average Latency: 1.860012628s Average Latency: 4.033134276s Average Latency: 7.592412668s + Standard Deviation: 1.169158915s Standard Deviation: 3.427243686s Standard Deviation: 2.951797195s +Experiment ID: de676ecf-038e-443f-a26a-27915f29e312 Experiment ID: 39d571b8-f39b-4aec-bd6a-e94f28a42a63 Experiment ID: 5b855105-60b5-4c2d-ba5c-fdad0213765c + Connections: 1 Connections: 2 Connections: 4 + Rate: 400 Rate: 400 Rate: 400 + Size: 1024 Size: 1024 Size: 1024 + Total Valid Tx: 35600 Total Valid Tx: 41565 Total Valid Tx: 41384 + Total Negative Latencies: 0 Total Negative Latencies: 0 Total Negative Latencies: 0 + Minimum Latency: 565.640641ms Minimum Latency: 1.650712046s Minimum Latency: 2.796290248s + Maximum Latency: 10.051316705s Maximum Latency: 15.897581951s Maximum Latency: 20.124431723s + Average Latency: 3.499369173s Average Latency: 8.635543807s Average Latency: 10.596146863s + Standard Deviation: 1.926805844s Standard Deviation: 2.535678364s Standard Deviation: 3.193742233s +Experiment ID: db10ca9e-6cf8-4dc9-9284-6e767e4b4346 Experiment ID: f57af87d-d342-41f7-a0eb-baa87a4b2257 Experiment ID: 32819ea0-1a59-41de-8aa6-b70f68697520 + Connections: 1 Connections: 2 Connections: 4 + Rate: 800 Rate: 800 Rate: 800 + Size: 1024 Size: 1024 Size: 1024 + Total Valid Tx: 36831 Total Valid Tx: 38686 Total Valid Tx: 40816 + Total Negative Latencies: 0 Total Negative Latencies: 0 Total Negative Latencies: 0 + Minimum Latency: 1.203966853s Minimum Latency: 728.863446ms Minimum Latency: 1.559342549s + Maximum Latency: 21.411365818s Maximum Latency: 24.349050642s Maximum Latency: 25.791215028s + Average Latency: 9.213156739s Average Latency: 11.194994374s Average Latency: 11.950851892s + Standard Deviation: 4.909584729s Standard Deviation: 5.199186587s Standard Deviation: 4.315394253s +Experiment ID: 587762c4-3fd4-4799-9f3b-9e6971b353ba Experiment ID: 489b2623-a3e4-453f-a771-5d05e7de4a1f Experiment ID: 98605df2-3b16-46db-8675-2980bc84ea2b + Connections: 1 Connections: 2 Connections: 4 + Rate: 1600 Rate: 1600 Rate: 1600 + Size: 1024 Size: 1024 Size: 1024 + Total Valid Tx: 40600 Total Valid Tx: 45034 Total Valid Tx: 39830 + Total Negative Latencies: 0 Total Negative Latencies: 0 Total Negative Latencies: 0 + Minimum Latency: 998.07523ms Minimum Latency: 1.43819209s Minimum Latency: 1.50664776s + Maximum Latency: 18.565312759s Maximum Latency: 17.098811297s Maximum Latency: 20.346885373s + Average Latency: 8.78128586s Average Latency: 8.957419021s Average Latency: 12.113245591s + Standard Deviation: 3.305897473s Standard Deviation: 2.734640455s Standard Deviation: 4.029854219s diff --git a/cometbft/v0.39/docs/qa/img38/rotating/rotating_avg_memory.png b/cometbft/v0.39/docs/qa/img38/rotating/rotating_avg_memory.png new file mode 100644 index 000000000..43dadcabe Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/rotating/rotating_avg_memory.png differ diff --git a/cometbft/v0.39/docs/qa/img38/rotating/rotating_block_rate.png b/cometbft/v0.39/docs/qa/img38/rotating/rotating_block_rate.png new file mode 100644 index 000000000..e627064a3 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/rotating/rotating_block_rate.png differ diff --git a/cometbft/v0.39/docs/qa/img38/rotating/rotating_cpu.png b/cometbft/v0.39/docs/qa/img38/rotating/rotating_cpu.png new file mode 100644 index 000000000..f51403d40 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/rotating/rotating_cpu.png differ diff --git a/cometbft/v0.39/docs/qa/img38/rotating/rotating_eph_heights.png b/cometbft/v0.39/docs/qa/img38/rotating/rotating_eph_heights.png new file mode 100644 index 000000000..6c8e08eee Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/rotating/rotating_eph_heights.png differ diff --git a/cometbft/v0.39/docs/qa/img38/rotating/rotating_latencies.png b/cometbft/v0.39/docs/qa/img38/rotating/rotating_latencies.png new file mode 100644 index 000000000..8031d7cda Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/rotating/rotating_latencies.png differ diff --git a/cometbft/v0.39/docs/qa/img38/rotating/rotating_peers.png b/cometbft/v0.39/docs/qa/img38/rotating/rotating_peers.png new file mode 100644 index 000000000..b0ac6a2b0 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/rotating/rotating_peers.png differ diff --git a/cometbft/v0.39/docs/qa/img38/rotating/rotating_txs_rate.png b/cometbft/v0.39/docs/qa/img38/rotating/rotating_txs_rate.png new file mode 100644 index 000000000..d3ae71413 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/rotating/rotating_txs_rate.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/02k_1_block_rate.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/02k_1_block_rate.png new file mode 100644 index 000000000..ff314bd7b Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/02k_1_block_rate.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/02k_1_total_txs_rate.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/02k_1_total_txs_rate.png new file mode 100644 index 000000000..67a0cbb58 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/02k_1_total_txs_rate.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/02k_avg_cpu.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/02k_avg_cpu.png new file mode 100644 index 000000000..2eb9683ec Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/02k_avg_cpu.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/02k_avg_memory.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/02k_avg_memory.png new file mode 100644 index 000000000..955d11362 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/02k_avg_memory.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/02k_avg_mempool_size.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/02k_avg_mempool_size.png new file mode 100644 index 000000000..426867db6 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/02k_avg_mempool_size.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/02k_block_rate.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/02k_block_rate.png new file mode 100644 index 000000000..0a53ccf78 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/02k_block_rate.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/02k_rounds.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/02k_rounds.png new file mode 100644 index 000000000..b932e80d4 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/02k_rounds.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/02k_total_txs_rate.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/02k_total_txs_rate.png new file mode 100644 index 000000000..c7040fb86 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/02k_total_txs_rate.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/04k_1_block_rate.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/04k_1_block_rate.png new file mode 100644 index 000000000..9f9564a6f Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/04k_1_block_rate.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/04k_1_total_txs_rate.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/04k_1_total_txs_rate.png new file mode 100644 index 000000000..e69096d7e Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/04k_1_total_txs_rate.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/04k_avg_cpu.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/04k_avg_cpu.png new file mode 100644 index 000000000..be2519292 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/04k_avg_cpu.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/04k_avg_memory.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/04k_avg_memory.png new file mode 100644 index 000000000..50503a3ab Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/04k_avg_memory.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/04k_avg_mempool_size.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/04k_avg_mempool_size.png new file mode 100644 index 000000000..3e3eea8ed Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/04k_avg_mempool_size.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/04k_block_rate.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/04k_block_rate.png new file mode 100644 index 000000000..f0bd5c2a1 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/04k_block_rate.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/04k_rounds.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/04k_rounds.png new file mode 100644 index 000000000..64bb878f7 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/04k_rounds.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/04k_total_txs_rate.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/04k_total_txs_rate.png new file mode 100644 index 000000000..be5ab70ae Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/04k_total_txs_rate.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/08k_1_avg_mempool_size.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/08k_1_avg_mempool_size.png new file mode 100644 index 000000000..00cc236c3 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/08k_1_avg_mempool_size.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/08k_1_block_rate.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/08k_1_block_rate.png new file mode 100644 index 000000000..9caa120f7 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/08k_1_block_rate.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/08k_1_rounds.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/08k_1_rounds.png new file mode 100644 index 000000000..809a97eed Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/08k_1_rounds.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/08k_1_total_txs_rate.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/08k_1_total_txs_rate.png new file mode 100644 index 000000000..ce1c5c7d8 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/08k_1_total_txs_rate.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/08k_avg_cpu.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/08k_avg_cpu.png new file mode 100644 index 000000000..c78af4f29 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/08k_avg_cpu.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/08k_avg_memory.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/08k_avg_memory.png new file mode 100644 index 000000000..cd36d0562 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/08k_avg_memory.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/08k_avg_mempool_size.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/08k_avg_mempool_size.png new file mode 100644 index 000000000..dd852e9bb Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/08k_avg_mempool_size.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/08k_rounds.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/08k_rounds.png new file mode 100644 index 000000000..0bd983039 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/08k_rounds.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/08k_total_txs_rate.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/08k_total_txs_rate.png new file mode 100644 index 000000000..87cb6e4ba Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/08k_total_txs_rate.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/16k_1_avg_mempool_size.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/16k_1_avg_mempool_size.png new file mode 100644 index 000000000..3eb5b73d3 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/16k_1_avg_mempool_size.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/16k_1_block_rate.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/16k_1_block_rate.png new file mode 100644 index 000000000..12025af8c Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/16k_1_block_rate.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/16k_1_rounds.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/16k_1_rounds.png new file mode 100644 index 000000000..15d6feb22 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/16k_1_rounds.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/16k_1_total_txs_rate.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/16k_1_total_txs_rate.png new file mode 100644 index 000000000..65cb0c115 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/16k_1_total_txs_rate.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/16k_avg_cpu.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/16k_avg_cpu.png new file mode 100644 index 000000000..6fff44dab Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/16k_avg_cpu.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/16k_avg_memory.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/16k_avg_memory.png new file mode 100644 index 000000000..218ef0bd6 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/16k_avg_memory.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/16k_avg_mempool_size.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/16k_avg_mempool_size.png new file mode 100644 index 000000000..73881a153 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/16k_avg_mempool_size.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/16k_block_rate.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/16k_block_rate.png new file mode 100644 index 000000000..73cbba282 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/16k_block_rate.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/16k_rounds.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/16k_rounds.png new file mode 100644 index 000000000..7458188b7 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/16k_rounds.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/16k_total_txs_rate.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/16k_total_txs_rate.png new file mode 100644 index 000000000..5d44a422d Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/16k_total_txs_rate.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/32k_1_avg_mempool_size.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/32k_1_avg_mempool_size.png new file mode 100644 index 000000000..273f1b8b8 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/32k_1_avg_mempool_size.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/32k_1_block_rate.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/32k_1_block_rate.png new file mode 100644 index 000000000..d469e9475 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/32k_1_block_rate.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/32k_1_rounds.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/32k_1_rounds.png new file mode 100644 index 000000000..347263dd8 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/32k_1_rounds.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/32k_1_total_txs_rate.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/32k_1_total_txs_rate.png new file mode 100644 index 000000000..0d0451acd Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/32k_1_total_txs_rate.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/32k_avg_cpu.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/32k_avg_cpu.png new file mode 100644 index 000000000..5464681c2 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/32k_avg_cpu.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/32k_avg_memory.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/32k_avg_memory.png new file mode 100644 index 000000000..4cea5df70 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/32k_avg_memory.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/32k_avg_mempool_size.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/32k_avg_mempool_size.png new file mode 100644 index 000000000..e573eca2a Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/32k_avg_mempool_size.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/32k_block_rate.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/32k_block_rate.png new file mode 100644 index 000000000..f3ebf6255 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/32k_block_rate.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/32k_rounds.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/32k_rounds.png new file mode 100644 index 000000000..a7a0597c2 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/32k_rounds.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/32k_total_txs_rate.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/32k_total_txs_rate.png new file mode 100644 index 000000000..fdd93e252 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/32k_total_txs_rate.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/8k_block_rate.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/8k_block_rate.png new file mode 100644 index 000000000..da0a378ad Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/8k_block_rate.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/all_c1r400_16k.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/all_c1r400_16k.png new file mode 100644 index 000000000..d7e18134f Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/all_c1r400_16k.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/all_c1r400_2k.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/all_c1r400_2k.png new file mode 100644 index 000000000..9682d84a2 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/all_c1r400_2k.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/all_c1r400_32k.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/all_c1r400_32k.png new file mode 100644 index 000000000..5179fd212 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/all_c1r400_32k.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/all_c1r400_4k.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/all_c1r400_4k.png new file mode 100644 index 000000000..46d67719c Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/all_c1r400_4k.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/all_c1r400_64k.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/all_c1r400_64k.png new file mode 100644 index 000000000..04ff7e760 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/all_c1r400_64k.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/all_c1r400_8k.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/all_c1r400_8k.png new file mode 100644 index 000000000..b54ed7d89 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/all_c1r400_8k.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/all_c1r400_baseline.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/all_c1r400_baseline.png new file mode 100644 index 000000000..2c094f58f Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/all_c1r400_baseline.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/all_experiments_16k.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/all_experiments_16k.png new file mode 100644 index 000000000..c7a1f6da2 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/all_experiments_16k.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/all_experiments_2k.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/all_experiments_2k.png new file mode 100644 index 000000000..ea717d559 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/all_experiments_2k.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/all_experiments_32k.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/all_experiments_32k.png new file mode 100644 index 000000000..ee4cf339d Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/all_experiments_32k.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/all_experiments_4k.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/all_experiments_4k.png new file mode 100644 index 000000000..eb4c569cb Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/all_experiments_4k.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/all_experiments_64k.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/all_experiments_64k.png new file mode 100644 index 000000000..f7abbae58 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/all_experiments_64k.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/all_experiments_8k.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/all_experiments_8k.png new file mode 100644 index 000000000..cbaaf5c9e Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/all_experiments_8k.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/all_experiments_baseline.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/all_experiments_baseline.png new file mode 100644 index 000000000..b27ec5d62 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/all_experiments_baseline.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/baseline_1_avg_mempool_size.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/baseline_1_avg_mempool_size.png new file mode 100644 index 000000000..63c86687b Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/baseline_1_avg_mempool_size.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/baseline_1_block_rate.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/baseline_1_block_rate.png new file mode 100644 index 000000000..46f0a4ee8 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/baseline_1_block_rate.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/baseline_1_rounds.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/baseline_1_rounds.png new file mode 100644 index 000000000..1e6db5e38 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/baseline_1_rounds.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/baseline_1_total_txs_rate.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/baseline_1_total_txs_rate.png new file mode 100644 index 000000000..75f9ab435 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/baseline_1_total_txs_rate.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/baseline_avg_cpu.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/baseline_avg_cpu.png new file mode 100644 index 000000000..2c1bca8bf Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/baseline_avg_cpu.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/baseline_avg_memory.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/baseline_avg_memory.png new file mode 100644 index 000000000..f0529880b Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/baseline_avg_memory.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/baseline_avg_mempool_size.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/baseline_avg_mempool_size.png new file mode 100644 index 000000000..179693cc6 Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/baseline_avg_mempool_size.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/baseline_block_rate.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/baseline_block_rate.png new file mode 100644 index 000000000..20073522c Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/baseline_block_rate.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/baseline_rounds.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/baseline_rounds.png new file mode 100644 index 000000000..468d4e2ff Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/baseline_rounds.png differ diff --git a/cometbft/v0.39/docs/qa/img38/voteExtensions/baseline_total_txs_rate.png b/cometbft/v0.39/docs/qa/img38/voteExtensions/baseline_total_txs_rate.png new file mode 100644 index 000000000..306793d5d Binary files /dev/null and b/cometbft/v0.39/docs/qa/img38/voteExtensions/baseline_total_txs_rate.png differ diff --git a/cometbft/v0.39/docs/tools/Overview.mdx b/cometbft/v0.39/docs/tools/Overview.mdx new file mode 100644 index 000000000..88a0a1c28 --- /dev/null +++ b/cometbft/v0.39/docs/tools/Overview.mdx @@ -0,0 +1,17 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/docs/tools/Overview' +title: Overview +order: 1 +--- + +CometBFT has some tools that are associated with it for: + +- [Debugging](/cometbft/v0.39/docs/tools/debugging) +- [Benchmarking](#benchmarking) + +## Benchmarking + +- [https://github.com/informalsystems/tm-load-test](https://github.com/informalsystems/tm-load-test) + +`tm-load-test` is a distributed load testing tool (and framework) for load testing CometBFT networks. diff --git a/cometbft/v0.39/docs/tools/debugging.mdx b/cometbft/v0.39/docs/tools/debugging.mdx new file mode 100644 index 000000000..3aec2d27e --- /dev/null +++ b/cometbft/v0.39/docs/tools/debugging.mdx @@ -0,0 +1,106 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/docs/tools/debugging' +title: Debugging +order: 1 +--- + +## CometBFT debug kill + +CometBFT comes with a `debug` sub-command that allows you to kill a live +CometBFT process while collecting useful information in a compressed archive. +The information includes the configuration used, consensus state, network +state, the node's status, the WAL, and even the stack trace of the process +before exit. These files can be useful to examine when debugging a faulty +CometBFT process. + +```bash +cometbft debug kill --home= +``` + +will write debug info into a compressed archive. The archive will contain the +following: + +```sh +├── config.toml +├── consensus_state.json +├── net_info.json +├── stacktrace.out +├── status.json +└── wal +``` + +Under the hood, `debug kill` fetches info from `/status`, `/net_info`, and +`/dump_consensus_state` HTTP endpoints, and kills the process with `-6`, which +captures the goroutine dump. + +## CometBFT debug dump + +The `debug dump` sub-command allows you to dump debugging data into +compressed archives at a regular interval. These archives contain the goroutine +and heap profiles in addition to the consensus state, network info, node +status, and the WAL. + +```bash +cometbft debug dump --home= +``` + +will perform similarly to `kill` except it only polls the node and +dumps debugging data every frequency seconds to a compressed archive under a +given destination directory. Each archive will contain: + +```sh +├── consensus_state.json +├── goroutine.out +├── heap.out +├── net_info.json +├── status.json +└── wal +``` + +Note: goroutine.out and heap.out will only be written if a profile address is +provided and is operational. This command is blocking and will log any error. + +## CometBFT Inspect + +CometBFT includes an `inspect` command for querying CometBFT's state store and block +store over CometBFT RPC. + +When the CometBFT consensus engine detects inconsistent state, it will crash the +entire CometBFT process. +While in this inconsistent state, a node running CometBFT will not start up. +The `inspect` command runs only a subset of CometBFT's RPC endpoints for querying the block store +and state store. +`inspect` allows operators to query a read-only view of the state. +`inspect` does not run the consensus engine at all and can therefore be used to debug +processes that have crashed due to inconsistent state. + +### Running inspect + +Start up the `inspect` tool on the machine where CometBFT crashed using: +```bash +cometbft inspect --home= +``` + +`inspect` will use the data directory specified in your CometBFT configuration file. +`inspect` will also run the RPC server at the address specified in your CometBFT configuration file. + +### Using inspect + +With the `inspect` server running, you can access RPC endpoints that are critically important +for debugging. +Calling the `/status`, `/consensus_state`, and `/dump_consensus_state` RPC endpoints +will return useful information about the CometBFT consensus state. + +To start the `inspect` process, run +```bash +cometbft inspect +``` + +### RPC endpoints + +The list of available RPC endpoints can be found by making a request to the RPC port. +For an `inspect` process running on `127.0.0.1:26657`, navigate your browser to +`http://127.0.0.1:26657/` to retrieve the list of enabled RPC endpoints. + +Additional information on the CometBFT RPC endpoints can be found in the [RPC documentation](/cometbft/v0.39/api-reference/rpc/index). diff --git a/cometbft/v0.39/spec/CometBFT-Spec.mdx b/cometbft/v0.39/spec/CometBFT-Spec.mdx new file mode 100644 index 000000000..336746d16 --- /dev/null +++ b/cometbft/v0.39/spec/CometBFT-Spec.mdx @@ -0,0 +1,96 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/spec/CometBFT-Spec' +order: 1 +title: Overview +parent: + title: Spec + order: 7 +--- +{/* trigger rebuild */} + +# CometBFT Spec + +This is a markdown specification of CometBFT. +It defines the base data structures, how they are validated, +and how they are communicated over the network. + +If you find discrepancies between the spec and the code that +do not have an associated issue or pull request on github, +please submit them to our [bug bounty](https://github.com/cometbft/cometbft#security)! + +## Contents + +- [Overview](#overview) + +### Data Structures + +- [Encoding and Digests](/cometbft/v0.39/spec/core/encoding) +- [Blockchain](/cometbft/v0.39/spec/core/Data_structures) +- [State](/cometbft/v0.39/spec/core/state) + +### Consensus Protocol + +- [Consensus Algorithm](/cometbft/v0.39/spec/consensus/Byzantine-Consensus-Algorithm) +- [Creating a proposal](/cometbft/v0.39/spec/consensus/Creating-Proposal) +- [Time](/cometbft/v0.39/spec/consensus/BFT-Time) +- [Light-Client](/cometbft/v0.39/spec/consensus/Light-Client) + +### P2P and Network Protocols + +- [The Base P2P Layer](/cometbft/v0.39/spec/p2p/legacy-docs/Peer-Discovery): multiplex the protocols ("reactors") on authenticated and encrypted TCP connections +- [Peer Exchange (PEX)](/cometbft/v0.39/spec/p2p/legacy-docs/messages/Peer-Exchange): gossip known peer addresses so peers can find each other +- [Block Sync](/cometbft/v0.39/spec/p2p/legacy-docs/messages/block-sync): gossip blocks so peers can catch up quickly +- [Consensus](/cometbft/v0.39/spec/p2p/legacy-docs/messages/consensus): gossip votes and block parts so new blocks can be committed +- [Mempool](/cometbft/v0.39/spec/p2p/legacy-docs/messages/mempool): gossip transactions so they get included in blocks +- [Evidence](/cometbft/v0.39/spec/p2p/legacy-docs/messages/evidence): sending invalid evidence will stop the peer + +### RPC + +- [RPC SPEC](/cometbft/v0.39/spec/rpc/Rpc-Spe): Specification of the CometBFT remote procedure call interface. + +### Software + +- [ABCI](/cometbft/v0.39/spec/abci/Overview): Details about interactions between the + application and consensus engine over ABCI +- [Write-Ahead Log](/cometbft/v0.39/spec/consensus/WAL): Details about how the consensus + engine preserves data and recovers from crash failures + +## Overview + +CometBFT provides Byzantine Fault Tolerant State Machine Replication using +hash-linked batches of transactions. Such transaction batches are called "blocks". +Hence, CometBFT defines a "blockchain". + +Each block in CometBFT has a unique index - its Height. +Heights in the blockchain are monotonic. +Each block is committed by a known set of weighted Validators. +Membership and weighting within this validator set may change over time. +CometBFT guarantees the safety and liveness of the blockchain +as long as less than 1/3 of the total weight of the Validator set +is malicious or faulty. + +A commit in CometBFT is a set of signed messages from more than 2/3 of +the total weight of the current Validator set. Validators take turns proposing +blocks and voting on them. Once enough votes are received, the block is considered +committed. These votes are included in the _next_ block as proof that the previous block +was committed - they cannot be included in the current block, as that block has already been +created. + +Once a block is committed, it can be executed against an application. +The application returns results for each of the transactions in the block. +The application can also return changes to be made to the validator set, +as well as a cryptographic digest of its latest state. + +CometBFT is designed to enable efficient verification and authentication +of the latest state of the blockchain. To achieve this, it embeds +cryptographic commitments to certain information in the block "header". +This information includes the contents of the block (eg. the transactions), +the validator set committing the block, as well as the various results returned by the application. +Note, however, that block execution only occurs _after_ a block is committed. +Thus, application results can only be included in the _next_ block. + +Also note that information like the transaction results and the validator set are never +directly included in the block - only their cryptographic digests (Merkle roots) are. +Hence, verification of a block requires a separate data structure to store this information. +We call this the `State`. Block verification also requires access to the previous block. diff --git a/cometbft/v0.39/spec/abci/Client-and-server.mdx b/cometbft/v0.39/spec/abci/Client-and-server.mdx new file mode 100644 index 000000000..ff041dd24 --- /dev/null +++ b/cometbft/v0.39/spec/abci/Client-and-server.mdx @@ -0,0 +1,93 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/spec/abci/Client-and-server' +order: 5 +title: Client and Server +--- + +This section is for those looking to implement their own ABCI Server, perhaps in +a new programming language. + +You are expected to have read all previous sections of ABCI++ specification, namely +[Basic Concepts](/cometbft/v0.39/spec/abci/Outline), +[Methods](/cometbft/v0.39/spec/abci/Methods), +[Application Requirements](/cometbft/v0.39/spec/abci/Requirements-for-the-Application), and +[Expected Behavior](/cometbft/v0.39/spec/abci/CometBFTs-expected-behavior). + +## Message Protocol and Synchrony + +The message protocol consists of pairs of requests and responses defined in the +[protobuf file](https://github.com/cometbft/cometbft/blob/v0.38.x/proto/tendermint/abci/types.proto). + +Some messages have no fields, while others may include byte-arrays, strings, integers, +or custom protobuf types. + +For more details on protobuf, see the [documentation](https://developers.google.com/protocol-buffers/docs/overview). + +{/* +As of v0.36 requests are synchronous. For each of ABCI++'s four connections (see +[Connections](/cometbft/v0.39/spec/abci/abci++_app_requirements)), when CometBFT issues a request to the +Application, it will wait for the response before continuing execution. As a side effect, +requests and responses are ordered for each connection, but not necessarily across connections. +*/} +## Server Implementations + +To use ABCI in your programming language of choice, there must be an ABCI +server in that language. CometBFT supports four implementations of the ABCI server: + +- in CometBFT's repository: + - In-process + - ABCI-socket + - GRPC +- [tendermint-rs](https://github.com/informalsystems/tendermint-rs) +- [tower-abci](https://github.com/penumbra-zone/tower-abci) + +The implementations in CometBFT's repository can be tested using `abci-cli` by setting +the `--abci` flag appropriately. + +See examples, in various stages of maintenance, in +[Go](https://github.com/cometbft/cometbft/tree/master/abci/server), +[JavaScript](https://github.com/tendermint/js-abci), and +[Java](https://github.com/jTendermint/jabci). + +### In Process + +The simplest implementation uses function calls in Golang. +This means ABCI applications written in Golang can be linked with CometBFT and run as a single binary. + +### GRPC + +If you are not using Golang, +but [GRPC](https://grpc.io/) is available in your language, this is the easiest approach, +though it will have significant performance overhead. + +Please check GRPC's documentation to know to set up the Application as an +ABCI GRPC server. + +### Socket + +The CometBFT Socket Protocol is an asynchronous, raw socket server protocol which provides ordered +message passing over Unix or TCP sockets. Messages are serialized using Protobuf3 and length-prefixed +with an [unsigned varint](https://developers.google.com/protocol-buffers/docs/encoding?csw=1#varints) + +If gRPC is not available in your language, or you require higher performance, or +otherwise enjoy programming, you may implement your own ABCI server using the +CometBFT Socket Protocol. The first step is still to auto-generate the +relevant data types and codec in your language using `protoc`, and then you need to +ensure you handle the unsigned `varint`-based message length encoding scheme +when reading and writing messages to the socket. + +Note that our length prefixing scheme does not apply to gRPC. + +Also note that your ABCI server must be able to handle multiple connections, +as CometBFT uses four connections. + +## Client + +There are currently two use-cases for an ABCI client. One is testing +tools that allow ABCI requests to be sent to the actual application via +command line. An example of this is `abci-cli`, which accepts CLI commands +to send corresponding ABCI requests. +The other is a consensus engine, such as CometBFT, +which makes ABCI requests to the application as prescribed by the consensus +algorithm used. diff --git a/cometbft/v0.39/spec/abci/CometBFTs-expected-behavior.mdx b/cometbft/v0.39/spec/abci/CometBFTs-expected-behavior.mdx new file mode 100644 index 000000000..6fbda5db4 --- /dev/null +++ b/cometbft/v0.39/spec/abci/CometBFTs-expected-behavior.mdx @@ -0,0 +1,282 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/spec/abci/CometBFTs-expected-behavior' +order: 4 +title: CometBFT's expected behavior +--- + +## Valid method call sequences + +This section describes what the Application can expect from CometBFT. + +The Tendermint consensus algorithm, currently adopted in CometBFT, is designed to protect safety under any network conditions, as long as +less than 1/3 of validators' voting power is byzantine. Most of the time, though, the network will behave +synchronously, no process will fall behind, and there will be no byzantine process. The following describes +what will happen during a block height _h_ in these frequent, benign conditions: + +* Consensus will decide in round 0, for height _h_; +* `PrepareProposal` will be called exactly once at the proposer process of round 0, height _h_; +* `ProcessProposal` will be called exactly once at all processes, and + will return _accept_ in its `Response*`; +* `ExtendVote` will be called exactly once at all processes; +* `VerifyVoteExtension` will be called exactly _n-1_ times at each validator process, where _n_ is + the number of validators, and will always return _accept_ in its `Response*`; +* `FinalizeBlock` will be called exactly once at all processes, conveying the same prepared + block that all calls to `PrepareProposal` and `ProcessProposal` had previously reported for + height _h_; and +* `Commit` will finally be called exactly once at all processes at the end of height _h_. + +However, the Application logic must be ready to cope with any possible run of the consensus algorithm for a given +height, including bad periods (byzantine proposers, network being asynchronous). +In these cases, the sequence of calls to ABCI++ methods may not be so straightforward, but +the Application should still be able to handle them, e.g., without crashing. +The purpose of this section is to define what these sequences look like in a precise way. + +As mentioned in the [Basic Concepts](/cometbft/v0.39/spec/abci/Outline) section, CometBFT +acts as a client of ABCI++ and the Application acts as a server. Thus, it is up to CometBFT to +determine when and in which order the different ABCI++ methods will be called. A well-written +Application design should consider _any_ of these possible sequences. + +The following grammar, written in case-sensitive Augmented Backus–Naur form (ABNF, specified +in [IETF rfc7405](https://datatracker.ietf.org/doc/html/rfc7405)), specifies all possible +sequences of calls to ABCI++, taken by a **correct process**, across all heights from the genesis block, +including recovery runs, from the point of view of the Application. + +```abnf +start = clean-start / recovery + +clean-start = init-chain [state-sync] consensus-exec +state-sync = *state-sync-attempt success-sync info +state-sync-attempt = offer-snapshot *apply-chunk +success-sync = offer-snapshot 1*apply-chunk + +recovery = info consensus-exec + +consensus-exec = (inf)consensus-height +consensus-height = *consensus-round decide commit +consensus-round = proposer / non-proposer + +proposer = *got-vote [prepare-proposal [process-proposal]] [extend] +extend = *got-vote extend-vote *got-vote +non-proposer = *got-vote [process-proposal] [extend] + +init-chain = %s"" +offer-snapshot = %s"" +apply-chunk = %s"" +info = %s"" +prepare-proposal = %s"" +process-proposal = %s"" +extend-vote = %s"" +got-vote = %s"" +decide = %s"" +commit = %s"" +``` + +We have kept some ABCI methods out of the grammar, in order to keep it as clear and concise as possible. +A common reason for keeping all these methods out is that they all can be called at any point in a sequence defined +by the grammar above. Other reasons depend on the method in question: + +* `Echo` and `Flush` are only used for debugging purposes. Further, their handling by the Application should be trivial. +* `CheckTx` is detached from the main method call sequence that drives block execution. +* `Query` provides read-only access to the current Application state, so handling it should also be independent from + block execution. +* Similarly, `ListSnapshots` and `LoadSnapshotChunk` provide read-only access to the Application's previously created + snapshots (if any), and help populate the parameters of `OfferSnapshot` and `ApplySnapshotChunk` at a process performing + state-sync while bootstrapping. Unlike `ListSnapshots` and `LoadSnapshotChunk`, both `OfferSnapshot` + and `ApplySnapshotChunk` _are_ included in the grammar. + +Finally, method `Info` is a special case. The method's purpose is three-fold, it can be used + +1. as part of handling an RPC call from an external client, +2. as a handshake between CometBFT and the Application upon recovery to check whether any blocks need + to be replayed, and +3. at the end of _state-sync_ to verify that the correct state has been reached. + +We have left `Info`'s first purpose out of the grammar for the same reasons as all the others: it can happen +at any time, and has nothing to do with the block execution sequence. The second and third purposes, on the other +hand, are present in the grammar. + +Let us now examine the grammar line by line, providing further details. + +* When a process starts, it may do so for the first time or after a crash (it is recovering). + +>```abnf +>start = clean-start / recovery +>``` + +* If the process is starting from scratch, CometBFT first calls `InitChain`, then it may optionally + start a _state-sync_ mechanism to catch up with other processes. Finally, it enters normal + consensus execution. + +>```abnf +>clean-start = init-chain [state-sync] consensus-exec +>``` + +* In _state-sync_ mode, CometBFT makes one or more attempts at synchronizing the Application's state. + At the beginning of each attempt, it offers the Application a snapshot found at another process. + If the Application accepts the snapshot, a sequence of calls to `ApplySnapshotChunk` method follow + to provide the Application with all the snapshots needed, in order to reconstruct the state locally. + A successful attempt must provide at least one chunk via `ApplySnapshotChunk`. + At the end of a successful attempt, CometBFT calls `Info` to make sure the reconstructed state's + _AppHash_ matches the one in the block header at the corresponding height. Note that the state + of the application does not contain vote extensions itself. The application can rely on + [CometBFT to ensure](https://github.com/cometbft/cometbft/blob/main/docs/references/rfc/rfc-100-abci-vote-extension-propag.md#base-implementation-persist-and-propagate-extended-commit-history) + the node has all the relevant data to proceed with the execution beyond this point. + +>```abnf +>state-sync = *state-sync-attempt success-sync info +>state-sync-attempt = offer-snapshot *apply-chunk +>success-sync = offer-snapshot 1*apply-chunk +>``` + +* In recovery mode, CometBFT first calls `Info` to know from which height it needs to replay decisions + to the Application. After this, CometBFT enters consensus execution, first in replay mode and then + in normal mode. + +>```abnf +>recovery = info consensus-exec +>``` + +* The non-terminal `consensus-exec` is a key point in this grammar. It is an infinite sequence of + consensus heights. The grammar is thus an + [omega-grammar](https://dl.acm.org/doi/10.5555/2361476.2361481), since it produces infinite + sequences of terminals (i.e., the API calls). + +>```abnf +>consensus-exec = (inf)consensus-height +>``` + +* A consensus height consists of zero or more rounds before deciding and executing via a call to + `FinalizeBlock`, followed by a call to `Commit`. In each round, the sequence of method calls + depends on whether the local process is the proposer or not. Note that, if a height contains zero + rounds, this means the process is replaying an already decided value (catch-up mode). + When calling `FinalizeBlock` with a block, the consensus algorithm run by CometBFT guarantees + that at least one non-byzantine validator has run `ProcessProposal` on that block. + + +>```abnf +>consensus-height = *consensus-round decide commit +>consensus-round = proposer / non-proposer +>``` + +* For every round, if the local process is the proposer of the current round, CometBFT calls `PrepareProposal`. + A successful execution of `PrepareProposal` results in a proposal block being (i) signed and (ii) stored + (e.g., in stable storage). + + A crash during this step will direct how the node proceeds the next time it is executed, for the same round, after restarted. + If it crashed before (i), then, during the recovery, `PrepareProposal` will execute as if for the first time. + Following a crash between (i) and (ii) and in (the likely) case `PrepareProposal` produces a different block, + the signing of this block will fail, which means that the new block will not be stored or broadcast. + If the crash happened after (ii), then signing fails but nothing happens to the stored block. + + If a block was stored, it is sent to all validators, including the proposer. + Receiving a proposal block triggers `ProcessProposal` with such a block. + + Then, optionally, the Application is + asked to extend its vote for that round. Calls to `VerifyVoteExtension` can come at any time: the + local process may be slightly late in the current round, or votes may come from a future round + of this height. + +>```abnf +>proposer = *got-vote [prepare-proposal [process-proposal]] [extend] +>extend = *got-vote extend-vote *got-vote +>``` + +* Also for every round, if the local process is _not_ the proposer of the current round, CometBFT + will call `ProcessProposal` at most once. + Under certain conditions, CometBFT may not call `ProcessProposal` in a round; + see [this section](/cometbft/v0.39/spec/abci/Introduction#scenario-3) for an example. + At most one call to `ExtendVote` may occur only after + `ProcessProposal` is called. A number of calls to `VerifyVoteExtension` can occur in any order + with respect to `ProcessProposal` and `ExtendVote` throughout the round. The reasons are the same + as above, namely, the process running slightly late in the current round, or votes from future + rounds of this height received. + +>```abnf +>non-proposer = *got-vote [process-proposal] [extend] +>``` + +* Finally, the grammar describes all its terminal symbols, which denote the different ABCI++ method calls that + may appear in a sequence. + +>```abnf +>init-chain = %s"" +>offer-snapshot = %s"" +>apply-chunk = %s"" +>info = %s"" +>prepare-proposal = %s"" +>process-proposal = %s"" +>extend-vote = %s"" +>got-vote = %s"" +>decide = %s"" +>commit = %s"" +>``` + +## Adapting existing Applications that use ABCI + +In some cases, an existing Application using the legacy ABCI may need to be adapted to work with ABCI++ +with as minimal changes as possible. In this case, of course, ABCI++ will not provide any advantage with respect +to the existing implementation, but will keep the same guarantees already provided by ABCI. +Here is how ABCI++ methods should be implemented. + +First of all, all the methods that did not change from ABCI 0.17.0 to ABCI 2.0, namely `Echo`, `Flush`, `Info`, `InitChain`, +`Query`, `CheckTx`, `ListSnapshots`, `LoadSnapshotChunk`, `OfferSnapshot`, and `ApplySnapshotChunk`, do not need +to undergo any changes in their implementation. + +As for the new methods: + +* `PrepareProposal` must create a list of [transactions](/cometbft/v0.39/spec/abci/Methods#prepareproposal) + by copying over the transaction list passed in `RequestPrepareProposal.txs`, in the same order. + + The Application must check whether the size of all transactions exceeds the byte limit + (`RequestPrepareProposal.max_tx_bytes`). If so, the Application must remove transactions at the + end of the list until the total byte size is at or below the limit. +* `ProcessProposal` must set `ResponseProcessProposal.status` to _accept_ and return. +* `ExtendVote` is to set `ResponseExtendVote.extension` to an empty byte array and return. +* `VerifyVoteExtension` must set `ResponseVerifyVoteExtension.accept` to _true_ if the extension is + an empty byte array and _false_ otherwise, then return. +* `FinalizeBlock` is to coalesce the implementation of methods `BeginBlock`, `DeliverTx`, and + `EndBlock`. Legacy applications looking to reuse old code that implemented `DeliverTx` should + wrap the legacy `DeliverTx` logic in a loop that executes one transaction iteration per + transaction in `RequestFinalizeBlock.tx`. + +Finally, `Commit`, which is kept in ABCI++, no longer returns the `AppHash`. It is now up to +`FinalizeBlock` to do so. Thus, a slight refactoring of the old `Commit` implementation will be +needed to move the return of `AppHash` to `FinalizeBlock`. + +## Accommodating for vote extensions + +In a manner transparent to the application, CometBFT ensures the node is provided with all +the data it needs to participate in consensus. + +In the case of recovering from a crash, or joining the network via state sync, CometBFT will make +sure the node acquires the necessary vote extensions before switching to consensus. + +If a node is already in consensus but falls behind, during catch-up, CometBFT will provide the node with +vote extensions from past heights by retrieving the extensions within `ExtendedCommit` for old heights that it had previously stored. + +We realize this is sub-optimal due to the increase in storage needed to store the extensions, we are +working on an optimization of this implementation which should alleviate this concern. +However, the application can use the existing `retain_height` parameter to decide how much +history it wants to keep, just as is done with the block history. The network-wide implications +of the usage of `retain_height` stay the same. +The decision to store +historical commits and potential optimizations, are discussed in detail in [RFC-100](https://github.com/cometbft/cometbft/blob/main/docs/references/rfc/rfc-100-abci-vote-extension-propag.md#current-limitations-and-possible-implementations) + +## Handling upgrades to ABCI 2.0 + +If applications upgrade to ABCI 2.0, CometBFT internally ensures that the [application setup](/cometbft/v0.39/spec/abci/Requirements-for-the-Application#application-configuration-required-to-switch-to-abci-2-0) is reflected in its operation. +CometBFT retrieves from the application configuration the value of `VoteExtensionsEnableHeight`( _he_,), +the height at which vote extensions are required for consensus to proceed, and uses it to determine the data it stores and data it sends to a peer that is catching up. + +Namely, upon saving the block for a given height _h_ in the block store at decision time + +* if _h ≥ he_, the corresponding extended commit that was used to decide locally is saved as well +* if _h < he_, there are no changes to the data saved + +In the catch-up mechanism, when a node _f_ realizes that another peer is at height _hp_, which is more than 2 heights behind height _hf_, + +* if _hp ≥ he_, _f_ uses the extended commit to + reconstruct the precommit votes with their corresponding extensions +* if _hp < he_, _f_ uses the canonical commit to reconstruct the precommit votes, + as done for ABCI 1.0 and earlier. diff --git a/cometbft/v0.39/spec/abci/Introduction.mdx b/cometbft/v0.39/spec/abci/Introduction.mdx new file mode 100644 index 000000000..e1aeb76dd --- /dev/null +++ b/cometbft/v0.39/spec/abci/Introduction.mdx @@ -0,0 +1,167 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/spec/abci/Introduction' +order: 6 +title: ABCI++ extra +--- + +# Introduction + +In the section [CometBFT's expected behaviour](/cometbft/v0.39/spec/abci/CometBFTs-expected-behavior#valid-method-call-sequences), +we presented the most common behaviour, usually referred to as the good case. +However, the grammar specified in the same section is more general and covers more scenarios +that an Application designer needs to account for. + +In this section, we give more information about these possible scenarios. We focus on methods +introduced by ABCI++: `PrepareProposal` and `ProcessProposal`. Specifically, we concentrate +on the part of the grammar presented below. + +```abnf +consensus-height = *consensus-round decide commit +consensus-round = proposer / non-proposer + +proposer = [prepare-proposal process-proposal] +non-proposer = [process-proposal] +``` + +We can see from the grammar that we can have several rounds before deciding a block. The reasons +why one round may not be enough are: + +* network asynchrony, and +* a Byzantine process being the proposer. + +If we assume that the consensus algorithm decides on block $X$ in round $r$, in the rounds +$r' <= r$, CometBFT can exhibit any of the following behaviours: + +1. Call `PrepareProposal` and/or `ProcessProposal` for block $X$. +1. Call `PrepareProposal` and/or `ProcessProposal` for block $Y \neq X$. +1. Does not call `PrepareProposal` and/or `ProcessProposal`. + +In the rounds in which the process is the proposer, CometBFT's `PrepareProposal` call is always followed by the +`ProcessProposal` call. The reason is that the process also broadcasts the proposal to itself, which is locally delivered and triggers the `ProcessProposal` call. +The proposal processed by `ProcessProposal` is the same as what was returned by any of the preceding `PrepareProposal` invoked for the same height and round. +While in the absence of restarts there is only one such preceding invocations, if the proposer restarts there could have been one extra invocation to `PrepareProposal` for each restart. + +As the number of rounds the consensus algorithm needs to decide in a given run is a priori unknown, the +application needs to account for any number of rounds, where each round can exhibit any of these three +behaviours. Recall that the application is unaware of the internals of consensus and thus of the rounds. + +# Possible scenarios + +The unknown number of rounds we can have when following the consensus algorithm yields a vast number of +scenarios we can expect. Listing them all is unfeasible. However, here we give several of them and draw the +main conclusions. Specifically, we will show that before block $X$ is decided: + +1. On a correct node, `PrepareProposal` may be called multiple times and for different blocks ([**Scenario 1**](#scenario-1)). +1. On a correct node, `ProcessProposal` may be called multiple times and for different blocks ([**Scenario 2**](#scenario-2)). +1. On a correct node, `PrepareProposal` and `ProcessProposal` for block $X$ may not be called ([**Scenario 3**](#scenario-3)). +1. On a correct node, `PrepareProposal` and `ProcessProposal` may not be called at all ([**Scenario 4**](#scenario-4)). + + +## Basic information + +Each scenario is presented from the perspective of a process $p$. More precisely, we show what happens in +each round's $step$ of the [Tendermint consensus algorithm](https://arxiv.org/pdf/1807.04938.pdf). While in +practice the consensus algorithm works with respect to voting power of the validators, in this document +we refer to number of processes (e.g., $n$, $f+1$, $2f+1$) for simplicity. The legend is below: + +### Round X + +1. **Propose:** Describes what happens while $step_p = propose$. +1. **Prevote:** Describes what happens while $step_p = prevote$. +1. **Precommit:** Describes what happens while $step_p = precommit$. + +## Scenario 1 + +$p$ calls `ProcessProposal` many times with different values. + +### Round 0 + +1. **Propose:** The proposer of this round is a Byzantine process, and it chooses not to send the proposal +message. Therefore, $p$'s $timeoutPropose$ expires, it sends $Prevote$ for $nil$, and it does not call +`ProcessProposal`. All correct processes do the same. +1. **Prevote:** $p$ eventually receives $2f+1$ $Prevote$ messages for $nil$ and starts $timeoutPrevote$. +When $timeoutPrevote$ expires it sends $Precommit$ for $nil$. +1. **Precommit:** $p$ eventually receives $2f+1$ $Precommit$ messages for $nil$ and starts $timeoutPrecommit$. +When it expires, it moves to the next round. + +### Round 1 + +1. **Propose:** A correct process is the proposer in this round. Its $validValue$ is $nil$, and it is free +to generate and propose a new block $Y$. Process $p$ receives this proposal in time, calls `ProcessProposal` +for block $Y$, and broadcasts a $Prevote$ message for it. +1. **Prevote:** Due to network asynchrony less than $2f+1$ processes send $Prevote$ for this block. +Therefore, $p$ does not update $validValue$ in this round. +1. **Precommit:** Since less than $2f+1$ processes send $Prevote$, no correct process will lock on this +block and send $Precommit$ message. As a consequence, $p$ does not decide on $Y$. + +### Round 2 + +1. **Propose:** Same as in [**Round 1**](#round-1), just another correct process is the proposer, and it +proposes another value $Z$. Process $p$ receives the proposal on time, calls `ProcessProposal` for new block +$Z$, and broadcasts a $Prevote$ message for it. +1. **Prevote:** Same as in [**Round 1**](#round-1). +1. **Precommit:** Same as in [**Round 1**](#round-1). + + +Rounds like these can continue until we have a round in which process $p$ updates its $validValue$ or until +we reach round $r$ where process $p$ decides on a block. After that, it will not call `ProcessProposal` +anymore for this height. + +## Scenario 2 + +$p$ calls `PrepareProposal` many times with different values. + +### Round 0 + +1. **Propose:** Process $p$ is the proposer in this round. Its $validValue$ is $nil$, and it is free to +generate and propose new block $Y$. Before proposing, it calls `PrepareProposal` for $Y$. After that, it +broadcasts the proposal, delivers it to itself, calls `ProcessProposal` and broadcasts $Prevote$ for it. +1. **Prevote:** Due to network asynchrony less than $2f+1$ processes receive the proposal on time and send +$Prevote$ for it. Therefore, $p$ does not update $validValue$ in this round. +1. **Precommit:** Since less than $2f+1$ processes send $Prevote$, no correct process will lock on this +block and send non-$nil$ $Precommit$ message. As a consequence, $p$ does not decide on $Y$. + +After this round, we can have multiple rounds like those in [Scenario 1](#scenario-1). The important thing +is that process $p$ should not update its $validValue$. Consequently, when process $p$ reaches the round +when it is again the proposer, it will ask the mempool for the new block again, and the mempool may return a +different block $Z$, and we can have the same round as [Round 0](#round-0-1) just for a different block. As +a result, process $p$ calls `PrepareProposal` again but for a different value. When it reaches round $r$ +some process will propose block $X$ and if $p$ receives $2f+1$ $Precommit$ messages, it will decide on this +value. + + +## Scenario 3 + +$p$ calls `PrepareProposal` and `ProcessProposal` for many values, but decides on a value for which it did +not call `PrepareProposal` or `ProcessProposal`. + +In this scenario, in all rounds before $r$ we can have any round presented in [Scenario 1](#scenario-1) or +[Scenario 2](#scenario-2). What is important is that: + +* no proposer proposed block $X$ or if it did, process $p$, due to asynchrony, did not receive it in time, +so it did not call `ProcessProposal`, and + +* if $p$ was the proposer it proposed some other value $\neq X$. + +### Round $r$ + +1. **Propose:** A correct process is the proposer in this round, and it proposes block $X$. +Due to asynchrony, the proposal message arrives to process $p$ after its $timeoutPropose$ +expires and it sends $Prevote$ for $nil$. Consequently, process $p$ does not call +`ProcessProposal` for block $X$. However, the same proposal arrives at other processes +before their $timeoutPropose$ expires, and they send $Prevote$ for this proposal. +1. **Prevote:** Process $p$ receives $2f+1$ $Prevote$ messages for proposal $X$, updates correspondingly its +$validValue$ and $lockedValue$ and sends $Precommit$ message. All correct processes do the same. +1. **Precommit:** Finally, process $p$ receives $2f+1$ $Precommit$ messages, and decides on block $X$. + + + +## Scenario 4 + +[Scenario 3](#scenario-3) can be translated into a scenario where $p$ does not call `PrepareProposal` and +`ProcessProposal` at all. For this, it is necessary that process $p$ is not the proposer in any of the +rounds $0 <= r' <= r$ and that due to network asynchrony or Byzantine proposer, it does not receive the +proposal before $timeoutPropose$ expires. As a result, it will enter round $r$ without calling +`PrepareProposal` and `ProcessProposal` before it, and as shown in Round $r$ of [Scenario 3](#scenario-3) it +will decide in this round. Again without calling any of these two calls. diff --git a/cometbft/v0.39/spec/abci/Methods.mdx b/cometbft/v0.39/spec/abci/Methods.mdx new file mode 100644 index 000000000..c0bb0f44a --- /dev/null +++ b/cometbft/v0.39/spec/abci/Methods.mdx @@ -0,0 +1,911 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/spec/abci/Methods' +order: 2 +title: Methods +--- + +## Methods existing in ABCI + +### Echo + +* **Request**: + * `Message (string)`: A string to echo back +* **Response**: + * `Message (string)`: The input string +* **Usage**: + * Echo a string to test an ABCI client/server implementation + +### Flush + +* **Usage**: + * Signals that messages queued on the client should be flushed to + the server. It is called periodically by the client + implementation to ensure asynchronous requests are actually + sent, and is called immediately to make a synchronous request, + which returns when the Flush response comes back. + +### Info + +* **Request**: + + | Name | Type | Description | Field Number | + |---------------|--------|----------------------------------------|--------------| + | version | string | The CometBFT software semantic version | 1 | + | block_version | uint64 | The CometBFT Block version | 2 | + | p2p_version | uint64 | The CometBFT P2P version | 3 | + | abci_version | string | The CometBFT ABCI semantic version | 4 | + +* **Response**: + + | Name | Type | Description | Field Number | Deterministic | + |---------------------|--------|-----------------------------------------------------|--------------|---------------| + | data | string | Some arbitrary information | 1 | N/A | + | version | string | The application software semantic version | 2 | N/A | + | app_version | uint64 | The application version | 3 | N/A | + | last_block_height | int64 | Latest height for which the app persisted its state | 4 | N/A | + | last_block_app_hash | bytes | Latest AppHash returned by `FinalizeBlock` | 5 | N/A | + +* **Usage**: + * Return information about the application state. + * Used to sync CometBFT with the application during a handshake + that happens on startup or on recovery. + * The returned `app_version` will be included in the Header of every block. + * CometBFT expects `last_block_app_hash` and `last_block_height` to + be updated and persisted during `Commit`. + +> Note: Semantic version is a reference to [semantic versioning](https://semver.org/). Semantic versions in info will be displayed as X.X.x. + +### InitChain + +* **Request**: + + | Name | Type | Description | Field Number | + |------------------|-------------------------------------------------|-----------------------------------------------------|--------------| + | time | [google.protobuf.Timestamp][protobuf-timestamp] | Genesis time | 1 | + | chain_id | string | ID of the blockchain. | 2 | + | consensus_params | [ConsensusParams](#consensusparams) | Initial consensus-critical parameters. | 3 | + | validators | repeated [ValidatorUpdate](#validatorupdate) | Initial genesis validators, sorted by voting power. | 4 | + | app_state_bytes | bytes | Serialized initial application state. JSON bytes. | 5 | + | initial_height | int64 | Height of the initial block (typically `1`). | 6 | + +* **Response**: + + | Name | Type | Description | Field Number | Deterministic | + |------------------|----------------------------------------------|--------------------------------------------------|--------------|---------------| + | consensus_params | [ConsensusParams](#consensusparams) | Initial consensus-critical parameters (optional) | 1 | Yes | + | validators | repeated [ValidatorUpdate](#validatorupdate) | Initial validator set (optional). | 2 | Yes | + | app_hash | bytes | Initial application hash. | 3 | Yes | + +* **Usage**: + * Called once upon genesis. + * If `ResponseInitChain.Validators` is empty, the initial validator set will be the `RequestInitChain.Validators` + * If `ResponseInitChain.Validators` is not empty, it will be the initial + validator set (regardless of what is in `RequestInitChain.Validators`). + * This allows the app to decide if it wants to accept the initial validator + set proposed by CometBFT (ie. in the genesis file), or if it wants to use + a different one (perhaps computed based on some application specific + information in the genesis file). + * Both `RequestInitChain.Validators` and `ResponseInitChain.Validators` are [ValidatorUpdate](#validatorupdate) structs. + So, technically, they both are _updating_ the set of validators from the empty set. + +### Query + +* **Request**: + + | Name | Type | Description | Field Number | + |--------|--------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------| + | data | bytes | Request parameters for the application to interpret analogously to a [URI query component](https://www.rfc-editor.org/rfc/rfc3986#section-3.4). Can be used with or in lieu of `path`. | 1 | + | path | string | A request path for the application to interpret analogously to a [URI path component](https://www.rfc-editor.org/rfc/rfc3986#section-3.3) in e.g. routing. Can be used with or in lieu of `data`. Applications MUST interpret "/store" or any path starting with "/store/" as a query by key on the underlying store, in which case a key SHOULD be specified in `data`. Applications SHOULD allow queries over specific types like `/accounts/...` or `/votes/...`. | 2 | + | height | int64 | The block height against which to query (default=0 returns data for the latest committed block). Note that this is the height of the block containing the application's Merkle root hash, which represents the state as it was after committing the block at Height-1. | 3 | + | prove | bool | Return Merkle proof with response if possible. | 4 | + +* **Response**: + + | Name | Type | Description | Field Number | Deterministic | + |-----------|-----------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------|---------------| + | code | uint32 | Response code. | 1 | N/A | + | log | string | The output of the application's logger. | 3 | N/A | + | info | string | Additional information. | 4 | N/A | + | index | int64 | The index of the key in the tree. | 5 | N/A | + | key | bytes | The key of the matching data. | 6 | N/A | + | value | bytes | The value of the matching data. | 7 | N/A | + | proof_ops | [ProofOps](#proofops) | Serialized proof for the value data, if requested, to be verified against the `app_hash` for the given Height. | 8 | N/A | + | height | int64 | The block height from which data was derived. Note that this is the height of the block containing the application's Merkle root hash, which represents the state as it was after committing the block at Height-1 | 9 | N/A | + | codespace | string | Namespace for the `code`. | 10 | N/A | + +* **Usage**: + * Query for data from the application at current or past height. + * Optionally return Merkle proof. + * Merkle proof includes self-describing `type` field to support many types + of Merkle trees and encoding formats. + +### CheckTx + +* **Request**: + + | Name | Type | Description | Field Number | + |------|-------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------| + | tx | bytes | The request transaction bytes | 1 | + | type | CheckTxType | One of `CheckTx_New` or `CheckTx_Recheck`. `CheckTx_New` is the default and means that a full check of the tranasaction is required. `CheckTx_Recheck` types are used when the mempool is initiating a normal recheck of a transaction. | 2 | + +* **Response**: + + | Name | Type | Description | Field Number | Deterministic | + |------------|---------------------------------------------------|----------------------------------------------------------------------|--------------|---------------| + | code | uint32 | Response code. | 1 | N/A | + | data | bytes | Result bytes, if any. | 2 | N/A | + | log | string | The output of the application's logger. | 3 | N/A | + | info | string | Additional information. | 4 | N/A | + | gas_wanted | int64 | Amount of gas requested for transaction. | 5 | N/A | + | gas_used | int64 | Amount of gas consumed by transaction. | 6 | N/A | + | events | repeated [Event](/cometbft/v0.39/spec/abci/Outline#events) | Type & Key-Value events for indexing transactions (e.g. by account). | 7 | N/A | + | codespace | string | Namespace for the `code`. | 8 | N/A | + +* **Usage**: + + * Technically optional - not involved in processing blocks. + * Guardian of the mempool: every node runs `CheckTx` before letting a + transaction into its local mempool. + * The transaction may come from an external user or another node + * `CheckTx` validates the transaction against the current state of the application, + for example, checking signatures and account balances, but does not apply any + of the state changes described in the transaction. + * Transactions where `ResponseCheckTx.Code != 0` will be rejected - they will not be broadcast + to other nodes or included in a proposal block. + CometBFT attributes no other value to the response code. + +### Commit + +#### Parameters and Types + +* **Request**: + + Commit signals the application to persist application state. It takes no parameters. + +* **Response**: + + | Name | Type | Description | Field Number | Deterministic | + |---------------|-------|------------------------------------------------------------------------|--------------|---------------| + | retain_height | int64 | Blocks below this height may be removed. Defaults to `0` (retain all). | 3 | No | + +* **Usage**: + + * Signal the Application to persist the application state. + Application is expected to persist its state at the end of this call, before calling `ResponseCommit`. + * Use `ResponseCommit.retain_height` with caution! If all nodes in the network remove historical + blocks then this data is permanently lost, and no new nodes will be able to join the network and + bootstrap, unless state sync is enabled on the chain. Historical blocks may also be required for other purposes, e.g. auditing, replay of + non-persisted heights, light client verification, and so on. + +### ListSnapshots + +* **Request**: + + Empty request asking the application for a list of snapshots. + +* **Response**: + + | Name | Type | Description | Field Number | Deterministic | + |-----------|--------------------------------|--------------------------------|--------------|---------------| + | snapshots | repeated [Snapshot](#snapshot) | List of local state snapshots. | 1 | N/A | + +* **Usage**: + * Used during state sync to discover available snapshots on peers. + * See `Snapshot` data type for details. + +### LoadSnapshotChunk + +* **Request**: + + | Name | Type | Description | Field Number | + |--------|--------|-----------------------------------------------------------------------|--------------| + | height | uint64 | The height of the snapshot the chunk belongs to. | 1 | + | format | uint32 | The application-specific format of the snapshot the chunk belongs to. | 2 | + | chunk | uint32 | The chunk index, starting from `0` for the initial chunk. | 3 | + +* **Response**: + + | Name | Type | Description | Field Number | Deterministic | + |-------|-------|--------------------------------------------------------------------------------------------------------------------------------------------------------|--------------|---------------| + | chunk | bytes | The binary chunk contents, in an arbitrary format. Chunk messages cannot be larger than 16 MB _including metadata_, so 10 MB is a good starting point. | 1 | N/A | + +* **Usage**: + * Used during state sync to retrieve snapshot chunks from peers. + +### OfferSnapshot + +* **Request**: + + | Name | Type | Description | Field Number | + |----------|-----------------------|--------------------------------------------------------------------------|--------------| + | snapshot | [Snapshot](#snapshot) | The snapshot offered for restoration. | 1 | + | app_hash | bytes | The light client-verified app hash for this height, from the blockchain. | 2 | + +* **Response**: + + | Name | Type | Description | Field Number | Deterministic | + |--------|-------------------|-----------------------------------|--------------|---------------| + | result | [Result](#result) | The result of the snapshot offer. | 1 | N/A | + +#### Result + +```protobuf + enum Result { + UNKNOWN = 0; // Unknown result, abort all snapshot restoration + ACCEPT = 1; // Snapshot is accepted, start applying chunks. + ABORT = 2; // Abort snapshot restoration, and don't try any other snapshots. + REJECT = 3; // Reject this specific snapshot, try others. + REJECT_FORMAT = 4; // Reject all snapshots with this `format`, try others. + REJECT_SENDER = 5; // Reject all snapshots from all senders of this snapshot, try others. + } +``` + +* **Usage**: + * `OfferSnapshot` is called when bootstrapping a node using state sync. The application may + accept or reject snapshots as appropriate. Upon accepting, CometBFT will retrieve and + apply snapshot chunks via `ApplySnapshotChunk`. The application may also choose to reject a + snapshot in the chunk response, in which case it should be prepared to accept further + `OfferSnapshot` calls. + * Only `AppHash` can be trusted, as it has been verified by the light client. Any other data + can be spoofed by adversaries, so applications should employ additional verification schemes + to avoid denial-of-service attacks. The verified `AppHash` is automatically checked against + the restored application at the end of snapshot restoration. + * For more information, see the `Snapshot` data type or the [state sync section](/cometbft/v0.39/spec/p2p/legacy-docs/messages/state-sync). + +### ApplySnapshotChunk + +* **Request**: + + | Name | Type | Description | Field Number | + |--------|--------|---------------------------------------------------------------------------|--------------| + | index | uint32 | The chunk index, starting from `0`. CometBFT applies chunks sequentially. | 1 | + | chunk | bytes | The binary chunk contents, as returned by `LoadSnapshotChunk`. | 2 | + | sender | string | The P2P ID of the node who sent this chunk. | 3 | + +* **Response**: + + | Name | Type | Description | Field Number | Deterministic | + |----------------|---------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------|---------------| + | result | Result (see below) | The result of applying this chunk. | 1 | N/A | + | refetch_chunks | repeated uint32 | Refetch and reapply the given chunks, regardless of `result`. Only the listed chunks will be refetched, and reapplied in sequential order. | 2 | N/A | + | reject_senders | repeated string | Reject the given P2P senders, regardless of `Result`. Any chunks already applied will not be refetched unless explicitly requested, but queued chunks from these senders will be discarded, and new chunks or other snapshots rejected. | 3 | N/A | + +```proto + enum Result { + UNKNOWN = 0; // Unknown result, abort all snapshot restoration + ACCEPT = 1; // The chunk was accepted. + ABORT = 2; // Abort snapshot restoration, and don't try any other snapshots. + RETRY = 3; // Reapply this chunk, combine with `RefetchChunks` and `RejectSenders` as appropriate. + RETRY_SNAPSHOT = 4; // Restart this snapshot from `OfferSnapshot`, reusing chunks unless instructed otherwise. + REJECT_SNAPSHOT = 5; // Reject this snapshot, try a different one. + } +``` + +* **Usage**: + * The application can choose to refetch chunks and/or ban P2P peers as appropriate. CometBFT + will not do this unless instructed by the application. + * The application may want to verify each chunk, e.g. by attaching chunk hashes in + `Snapshot.Metadata` and/or incrementally verifying contents against `AppHash`. + * When all chunks have been accepted, CometBFT will make an ABCI `Info` call to verify that + `LastBlockAppHash` and `LastBlockHeight` matches the expected values, and record the + `AppVersion` in the node state. It then switches to block sync or consensus and joins the + network. + * If CometBFT is unable to retrieve the next chunk after some time (e.g. because no suitable + peers are available), it will reject the snapshot and try a different one via `OfferSnapshot`. + The application should be prepared to reset and accept it or abort as appropriate. + +## New methods introduced in ABCI 2.0 + +### PrepareProposal + +#### Parameters and Types + +* **Request**: + + | Name | Type | Description | Field Number | + |----------------------|-------------------------------------------------|-----------------------------------------------------------------------------------------------|--------------| + | max_tx_bytes | int64 | Currently configured maximum size in bytes taken by the modified transactions. | 1 | + | txs | repeated bytes | Preliminary list of transactions that have been picked as part of the block to propose. | 2 | + | local_last_commit | [ExtendedCommitInfo](#extendedcommitinfo) | Info about the last commit, obtained locally from CometBFT's data structures. | 3 | + | misbehavior | repeated [Misbehavior](#misbehavior) | List of information about validators that misbehaved. | 4 | + | height | int64 | The height of the block that will be proposed. | 5 | + | time | [google.protobuf.Timestamp][protobuf-timestamp] | Timestamp of the block that that will be proposed. | 6 | + | next_validators_hash | bytes | Merkle root of the next validator set. | 7 | + | proposer_address | bytes | [Address](/cometbft/v0.39/spec/core/Data_structures#address) of the validator that is creating the proposal. | 8 | + +* **Response**: + + | Name | Type | Description | Field Number | Deterministic | + |------|----------------|---------------------------------------------------------------------------------------------|--------------|---------------| + | txs | repeated bytes | Possibly modified list of transactions that have been picked as part of the proposed block. | 2 | No | + +* **Usage**: + * `RequestPrepareProposal`'s parameters `txs`, `misbehavior`, `height`, `time`, + `next_validators_hash`, and `proposer_address` are the same as in `RequestProcessProposal` + and `RequestFinalizeBlock`. + * `RequestPrepareProposal.local_last_commit` is a set of the precommit votes for the previous + height, including the ones that led to the decision of the previous block, + together with their corresponding vote extensions. + * The `height`, `time`, and `proposer_address` values match the values from the header of the + proposed block. + * `RequestPrepareProposal` contains a preliminary set of transactions `txs` that CometBFT + retrieved from the mempool, called _raw proposal_. The Application can modify this + set and return a modified set of transactions via `ResponsePrepareProposal.txs` . + * The Application _can_ modify the raw proposal: it can reorder, remove or add transactions. + Let `tx` be a transaction in `txs` (set of transactions within `RequestPrepareProposal`): + * If the Application considers that `tx` should not be proposed in this block, e.g., + there are other transactions with higher priority, then it should not include it in + `ResponsePrepareProposal.txs`. However, this will not remove `tx` from the mempool. + * If the Application wants to add a new transaction to the proposed block, then the + Application includes it in `ResponsePrepareProposal.txs`. CometBFT will not add + the transaction to the mempool. + * The Application should be aware that removing and adding transactions may compromise + _traceability_. + > Consider the following example: the Application transforms a client-submitted + transaction `t1` into a second transaction `t2`, i.e., the Application asks CometBFT + to remove `t1` from the block and add `t2` to the block. If a client wants to eventually check what + happened to `t1`, it will discover that `t1` is not in a + committed block (assuming a _re-CheckTx_ evicted it from the mempool), getting the wrong idea that `t1` did not make it into a block. Note + that `t2` _will be_ in a committed block, but unless the Application tracks this + information, no component will be aware of it. Thus, if the Application wants + traceability, it is its responsibility's to support it. For instance, the Application + could attach to a transformed transaction a list with the hashes of the transactions it + derives from. + * The Application MAY configure CometBFT to include a list of transactions in `RequestPrepareProposal.txs` + whose total size in bytes exceeds `RequestPrepareProposal.max_tx_bytes`. + If the Application sets `ConsensusParams.Block.MaxBytes` to -1, CometBFT + will include _all_ transactions currently in the mempool in `RequestPrepareProposal.txs`, + which may not fit in `RequestPrepareProposal.max_tx_bytes`. + Therefore, if the size of `RequestPrepareProposal.txs` is greater than + `RequestPrepareProposal.max_tx_bytes`, the Application MUST remove transactions to ensure + that the `RequestPrepareProposal.max_tx_bytes` limit is respected by those transactions + returned in `ResponsePrepareProposal.txs`. + This is specified in [Requirement 2](/cometbft/v0.39/spec/abci/Requirements-for-the-Application). + * As a result of executing the prepared proposal, the Application may produce block events or transaction events. + The Application must keep those events until a block is decided and then pass them on to CometBFT via + `ResponseFinalizeBlock`. + * CometBFT does NOT provide any additional validity checks (such as checking for duplicate + transactions). + {/* + As a sanity check, CometBFT will check the returned parameters for validity if the Application modified them. + In particular, `ResponsePrepareProposal.txs` will be deemed invalid if there are duplicate transactions in the list. + */} + * If CometBFT fails to validate the `ResponsePrepareProposal`, CometBFT will assume the + Application is faulty and crash. + * The implementation of `PrepareProposal` MAY be non-deterministic. + + +#### When does CometBFT call "PrepareProposal" ? + +When a validator _p_ enters consensus round _r_, height _h_, in which _p_ is the proposer, +and _p_'s _validValue_ is `nil`: + +1. CometBFT collects outstanding transactions from _p_'s mempool + * the transactions will be collected in order of priority + * _p_'s CometBFT creates a block header. +2. _p_'s CometBFT calls `RequestPrepareProposal` with the newly generated block, the local + commit of the previous height (with vote extensions), and any outstanding evidence of + misbehavior. The call is synchronous: CometBFT's execution will block until the Application + returns from the call. +3. The Application uses the information received (transactions, commit info, misbehavior, time) to + (potentially) modify the proposal. + * the Application MAY fully execute the block and produce a candidate state (immediate execution) + * the Application can manipulate transactions: + * leave transactions untouched + * add new transactions (not present initially) to the proposal + * remove transactions from the proposal (but not from the mempool thus effectively _delaying_ them) - the + Application does not include the transaction in `ResponsePrepareProposal.txs`. + * modify transactions (e.g. aggregate them). As explained above, this compromises client traceability, unless + it is implemented at the Application level. + * reorder transactions - the Application reorders transactions in the list + * the Application MAY use the vote extensions in the commit info to modify the proposal, in which case it is suggested + that extensions be validated in the same maner as done in `VerifyVoteExtension`, since extensions of votes included + in the commit info after the minimum of +2/3 had been reached are not verified. +4. The Application includes the transaction list (whether modified or not) in the return parameters + (see the rules in section _Usage_), and returns from the call. +5. _p_ uses the (possibly) modified block as _p_'s proposal in round _r_, height _h_. + +Note that, if _p_ has a non-`nil` _validValue_ in round _r_, height _h_, +the consensus algorithm will use it as proposal and will not call `RequestPrepareProposal`. + +### ProcessProposal + +#### Parameters and Types + +* **Request**: + + | Name | Type | Description | Field Number | + |----------------------|-------------------------------------------------|-------------------------------------------------------------------------------------------|--------------| + | txs | repeated bytes | List of transactions of the proposed block. | 1 | + | proposed_last_commit | [CommitInfo](#commitinfo) | Info about the last commit, obtained from the information in the proposed block. | 2 | + | misbehavior | repeated [Misbehavior](#misbehavior) | List of information about validators that misbehaved. | 3 | + | hash | bytes | The hash of the proposed block. | 4 | + | height | int64 | The height of the proposed block. | 5 | + | time | [google.protobuf.Timestamp][protobuf-timestamp] | Timestamp of the proposed block. | 6 | + | next_validators_hash | bytes | Merkle root of the next validator set. | 7 | + | proposer_address | bytes | [Address](/cometbft/v0.39/spec/core/Data_structures#address) of the validator that created the proposal. | 8 | + +* **Response**: + + | Name | Type | Description | Field Number | Deterministic | + |--------|-----------------------------------|------------------------------------------------------------------|--------------|---------------| + | status | [ProposalStatus](#proposalstatus) | `enum` that signals if the application finds the proposal valid. | 1 | Yes | + +* **Usage**: + * Contains all information on the proposed block needed to fully execute it. + * The Application may fully execute the block as though it was handling + `RequestFinalizeBlock`. + * However, any resulting state changes must be kept as _candidate state_, + and the Application should be ready to discard it in case another block is decided. + * `RequestProcessProposal` is also called at the proposer of a round. + Normally the call to `RequestProcessProposal` occurs right after the call to `RequestPrepareProposal` and + `RequestProcessProposal` matches the block produced based on `ResponsePrepareProposal` (i.e., + `RequestPrepareProposal.txs` equals `RequestProcessProposal.txs`). + However, no such guarantee is made since, in the presence of failures, `RequestProcessProposal` may match + `ResponsePrepareProposal` from an earlier invocation or `ProcessProposal` may not be invoked at all. + * The height and time values match the values from the header of the proposed block. + * If `ResponseProcessProposal.status` is `REJECT`, consensus assumes the proposal received + is not valid. + * The Application MAY fully execute the block (immediate execution) + * The implementation of `ProcessProposal` MUST be deterministic. Moreover, the value of + `ResponseProcessProposal.status` MUST **exclusively** depend on the parameters passed in + the call to `RequestProcessProposal`, and the last committed Application state + (see [Requirements](/cometbft/v0.39/spec/abci/Requirements-for-the-Application) section). + * Moreover, application implementors SHOULD always set `ResponseProcessProposal.status` to `ACCEPT`, + unless they _really_ know what the potential liveness implications of returning `REJECT` are. + +#### When does CometBFT call "ProcessProposal" ? + +When a node _p_ enters consensus round _r_, height _h_, in which _q_ is the proposer (possibly _p_ = _q_): + +1. _p_ sets up timer `ProposeTimeout`. +2. If _p_ is the proposer, _p_ executes steps 1-6 in [PrepareProposal](#prepareproposal). +3. Upon reception of Proposal message (which contains the header) for round _r_, height _h_ from + _q_, _p_ verifies the block header. +4. Upon reception of Proposal message, along with all the block parts, for round _r_, height _h_ + from _q_, _p_ follows the validators' algorithm to check whether it should prevote for the + proposed block, or `nil`. +5. If the validators' consensus algorithm indicates _p_ should prevote non-nil: + 1. CometBFT calls `RequestProcessProposal` with the block. The call is synchronous. + 2. The Application checks/processes the proposed block, which is read-only, and returns + `ACCEPT` or `REJECT` in the `ResponseProcessProposal.status` field. + * The Application, depending on its needs, may call `ResponseProcessProposal` + * either after it has completely processed the block (immediate execution), + * or after doing some basic checks, and process the block asynchronously. In this case the + Application will not be able to reject the block, or force prevote/precommit `nil` + afterwards. + * or immediately, returning `ACCEPT`, if _p_ is not a validator + and the Application does not want non-validating nodes to handle `ProcessProposal` + 3. If _p_ is a validator and the returned value is + * `ACCEPT`: _p_ prevotes on this proposal for round _r_, height _h_. + * `REJECT`: _p_ prevotes `nil`. + * + +### ExtendVote + +#### Parameters and Types + +* **Request**: + + | Name | Type | Description | Field Number | + |----------------------|-------------------------------------------------|-------------------------------------------------------------------------------------------|--------------| + | hash | bytes | The header hash of the proposed block that the vote extension is to refer to. | 1 | + | height | int64 | Height of the proposed block (for sanity check). | 2 | + | time | [google.protobuf.Timestamp][protobuf-timestamp] | Timestamp of the proposed block (that the extension is to refer to). | 3 | + | txs | repeated bytes | List of transactions of the block that the extension is to refer to. | 4 | + | proposed_last_commit | [CommitInfo](#commitinfo) | Info about the last proposed block's last commit. | 5 | + | misbehavior | repeated [Misbehavior](#misbehavior) | List of information about validators that misbehaved contained in the proposed block. | 6 | + | next_validators_hash | bytes | Merkle root of the next validator set contained in the proposed block. | 7 | + | proposer_address | bytes | [Address](/cometbft/v0.39/spec/core/Data_structures#address) of the validator that created the proposal. | 8 | + +* **Response**: + + | Name | Type | Description | Field Number | Deterministic | + |----------------|-------|-------------------------------------------------------|--------------|---------------| + | vote_extension | bytes | Information signed by by CometBFT. Can have 0 length. | 1 | No | + +* **Usage**: + * `ResponseExtendVote.vote_extension` is application-generated information that will be signed + by CometBFT and attached to the Precommit message. + * The Application may choose to use an empty vote extension (0 length). + * The contents of `RequestExtendVote` correspond to the proposed block on which the consensus algorithm + will send the Precommit message. + * `ResponseExtendVote.vote_extension` will only be attached to a non-`nil` Precommit message. If the consensus algorithm is to + precommit `nil`, it will not call `RequestExtendVote`. + * The Application logic that creates the extension can be non-deterministic. + +#### When does CometBFT call `ExtendVote`? + +When a validator _p_ is in consensus state _prevote_ of round _r_, height _h_, in which _q_ is the proposer; and _p_ has received + +* the Proposal message _v_ for round _r_, height _h_, along with all the block parts, from _q_, +* `Prevote` messages from _2f + 1_ validators' voting power for round _r_, height _h_, prevoting for the same block _id(v)_, + +then _p_ locks _v_ and sends a Precommit message in the following way + +1. _p_ sets _lockedValue_ and _validValue_ to _v_, and sets _lockedRound_ and _validRound_ to _r_ +2. _p_'s CometBFT calls `RequestExtendVote` with _v_ (`RequestExtendVote`). The call is synchronous. +3. The Application returns an array of bytes, `ResponseExtendVote.extension`, which is not interpreted by the consensus algorithm. +4. _p_ sets `ResponseExtendVote.extension` as the value of the `extension` field of type + [CanonicalVoteExtension](/cometbft/v0.39/spec/core/Data_structures#canonicalvoteextension), + populates the other fields in [CanonicalVoteExtension](/cometbft/v0.39/spec/core/Data_structures#canonicalvoteextension), + and signs the populated data structure. +5. _p_ constructs and signs the [CanonicalVote](/cometbft/v0.39/spec/core/Data_structures#canonicalvote) structure. +6. _p_ constructs the Precommit message (i.e. [Vote](/cometbft/v0.39/spec/core/Data_structures#vote) structure) + using [CanonicalVoteExtension](/cometbft/v0.39/spec/core/Data_structures#canonicalvoteextension) + and [CanonicalVote](/cometbft/v0.39/spec/core/Data_structures#canonicalvoteextension). +7. _p_ broadcasts the Precommit message. + +In the cases when _p_ is to broadcast `precommit nil` messages (either _2f+1_ `prevote nil` messages received, +or _timeoutPrevote_ triggered), _p_'s CometBFT does **not** call `RequestExtendVote` and will not include +a [CanonicalVoteExtension](/cometbft/v0.39/spec/core/Data_structures#canonicalvoteextension) field in the `precommit nil` message. + +### VerifyVoteExtension + +#### Parameters and Types + +* **Request**: + + | Name | Type | Description | Field Number | + |-------------------|-------|-------------------------------------------------------------------------------------------|--------------| + | hash | bytes | The hash of the proposed block that the vote extension refers to. | 1 | + | validator_address | bytes | [Address](/cometbft/v0.39/spec/core/Data_structures#address) of the validator that signed the extension. | 2 | + | height | int64 | Height of the block (for sanity check). | 3 | + | vote_extension | bytes | Application-specific information signed by CometBFT. Can have 0 length. | 4 | + +* **Response**: + + | Name | Type | Description | Field Number | Deterministic | + |--------|-------------------------------|----------------------------------------------------------------|--------------|---------------| + | status | [VerifyStatus](#verifystatus) | `enum` signaling if the application accepts the vote extension | 1 | Yes | + +* **Usage**: + * `RequestVerifyVoteExtension.vote_extension` can be an empty byte array. The Application's + interpretation of it should be + that the Application running at the process that sent the vote chose not to extend it. + CometBFT will always call `RequestVerifyVoteExtension`, even for 0 length vote extensions. + * `RequestVerifyVoteExtension` is not called for precommit votes sent by the local process. + * `RequestVerifyVoteExtension.hash` refers to a proposed block. There is not guarantee that + this proposed block has previously been exposed to the Application via `ProcessProposal`. + * If `ResponseVerifyVoteExtension.status` is `REJECT`, the consensus algorithm will reject the whole received vote. + See the [Requirements](/cometbft/v0.39/spec/abci/Requirements-for-the-Application) section to understand the potential + liveness implications of this. + * The implementation of `VerifyVoteExtension` MUST be deterministic. Moreover, the value of + `ResponseVerifyVoteExtension.status` MUST **exclusively** depend on the parameters passed in + the call to `RequestVerifyVoteExtension`, and the last committed Application state + (see [Requirements](/cometbft/v0.39/spec/abci/Requirements-for-the-Application) section). + * Moreover, application implementers SHOULD always set `ResponseVerifyVoteExtension.status` to `ACCEPT`, + unless they _really_ know what the potential liveness implications of returning `REJECT` are. + +#### When does CometBFT call `VerifyVoteExtension`? + +When a node _p_ is in consensus round _r_, height _h_, and _p_ receives a Precommit +message for round _r_, height _h_ from validator _q_ (_q_ ≠ _p_): + +1. If the Precommit message does not contain a vote extension with a valid signature, _p_ + discards the Precommit message as invalid. + * a 0-length vote extension is valid as long as its accompanying signature is also valid. +2. Else, _p_'s CometBFT calls `RequestVerifyVoteExtension`. +3. The Application returns `ACCEPT` or `REJECT` via `ResponseVerifyVoteExtension.status`. +4. If the Application returns + * `ACCEPT`, _p_ will keep the received vote, together with its corresponding + vote extension in its internal data structures. It will be used to populate the [ExtendedCommitInfo](#extendedcommitinfo) + structure in calls to `RequestPrepareProposal`, in rounds of height _h + 1_ where _p_ is the proposer. + * `REJECT`, _p_ will deem the Precommit message invalid and discard it. + +When a node _p_ is in consensus round _0_, height _h_, and _p_ receives a Precommit +message for CommitRound _r_, height _h-1_ from validator _q_ (_q_ ≠ _p_), _p_ +MAY add the Precommit message and associated extension to [ExtendedCommitInfo](#extendedcommitinfo) +without calling `RequestVerifyVoteExtension` to verify it. + + +### FinalizeBlock + +#### Parameters and Types + +* **Request**: + + | Name | Type | Description | Field Number | + |----------------------|-------------------------------------------------|-------------------------------------------------------------------------------------------|--------------| + | txs | repeated bytes | List of transactions committed as part of the block. | 1 | + | decided_last_commit | [CommitInfo](#commitinfo) | Info about the last commit, obtained from the block that was just decided. | 2 | + | misbehavior | repeated [Misbehavior](#misbehavior) | List of information about validators that misbehaved. | 3 | + | hash | bytes | The block's hash. | 4 | + | height | int64 | The height of the finalized block. | 5 | + | time | [google.protobuf.Timestamp][protobuf-timestamp] | Timestamp of the finalized block. | 6 | + | next_validators_hash | bytes | Merkle root of the next validator set. | 7 | + | proposer_address | bytes | [Address](/cometbft/v0.39/spec/core/Data_structures#address) of the validator that created the proposal. | 8 | + +* **Response**: + + | Name | Type | Description | Field Number | Deterministic | + |-------------------------|---------------------------------------------------|----------------------------------------------------------------------------------|--------------|---------------| + | events | repeated [Event](/cometbft/v0.39/spec/abci/Outline#events) | Type & Key-Value events for indexing | 1 | No | + | tx_results | repeated [ExecTxResult](#exectxresult) | List of structures containing the data resulting from executing the transactions | 2 | Yes | + | validator_updates | repeated [ValidatorUpdate](#validatorupdate) | Changes to validator set (set voting power to 0 to remove). | 3 | Yes | + | consensus_param_updates | [ConsensusParams](#consensusparams) | Changes to gas, size, and other consensus-related parameters. | 4 | Yes | + | app_hash | bytes | The Merkle root hash of the application state. | 5 | Yes | + +* **Usage**: + * Contains the fields of the newly decided block. + * This method is equivalent to the call sequence `BeginBlock`, [`DeliverTx`], + and `EndBlock` in ABCI 1.0. + * The height and time values match the values from the header of the proposed block. + * The Application can use `RequestFinalizeBlock.decided_last_commit` and `RequestFinalizeBlock.misbehavior` + to determine rewards and punishments for the validators. + * The Application executes the transactions in `RequestFinalizeBlock.txs` deterministically, + according to the rules set up by the Application, before returning control to CometBFT. + Alternatively, it can apply the candidate state corresponding to the same block previously + executed via `PrepareProposal` or `ProcessProposal`. + * `ResponseFinalizeBlock.tx_results[i].Code == 0` only if the _i_-th transaction is fully valid. + * The Application must provide values for `ResponseFinalizeBlock.app_hash`, + `ResponseFinalizeBlock.tx_results`, `ResponseFinalizeBlock.validator_updates`, and + `ResponseFinalizeBlock.consensus_param_updates` as a result of executing the block. + * The values for `ResponseFinalizeBlock.validator_updates`, or + `ResponseFinalizeBlock.consensus_param_updates` may be empty. In this case, CometBFT will keep + the current values. + * `ResponseFinalizeBlock.validator_updates`, triggered by block `H`, affect validation + for blocks `H+1`, `H+2`, and `H+3`. Heights following a validator update are affected in the following way: + * Height `H+1`: `NextValidatorsHash` includes the new `validator_updates` value. + * Height `H+2`: The validator set change takes effect and `ValidatorsHash` is updated. + * Height `H+3`: `*_last_commit` fields in `PrepareProposal`, `ProcessProposal`, and + `FinalizeBlock` now include the altered validator set. + * `ResponseFinalizeBlock.consensus_param_updates` returned for block `H` apply to the consensus + params for block `H+1`. For more information on the consensus parameters, + see the [consensus parameters](/cometbft/v0.39/spec/abci/Requirements-for-the-Application#consensus-parameters) + section. + * `ResponseFinalizeBlock.app_hash` contains an (optional) Merkle root hash of the application state. + * `ResponseFinalizeBlock.app_hash` is included as the `Header.AppHash` in the next block. + * `ResponseFinalizeBlock.app_hash` may also be empty or hard-coded, but MUST be + **deterministic** - it must not be a function of anything that did not come from the parameters + of `RequestFinalizeBlock` and the previous committed state. + * Later calls to `Query` can return proofs about the application state anchored + in this Merkle root hash. + * The implementation of `FinalizeBlock` MUST be deterministic, since it is + making the Application's state evolve in the context of state machine replication. + * Currently, CometBFT will fill up all fields in `RequestFinalizeBlock`, even if they were + already passed on to the Application via `RequestPrepareProposal` or `RequestProcessProposal`. + * When calling `FinalizeBlock` with a block, the consensus algorithm run by CometBFT guarantees + that at least one non-byzantine validator has run `ProcessProposal` on that block. + +#### When does CometBFT call `FinalizeBlock`? + +When a node _p_ is in consensus height _h_, and _p_ receives + +* the Proposal message with block _v_ for a round _r_, along with all its block parts, from _q_, + which is the proposer of round _r_, height _h_, +* `Precommit` messages from _2f + 1_ validators' voting power for round _r_, height _h_, + precommitting the same block _id(v)_, + +then _p_ decides block _v_ and finalizes consensus for height _h_ in the following way + +1. _p_ persists _v_ as the decision for height _h_. +2. _p_'s CometBFT calls `RequestFinalizeBlock` with _v_'s data. The call is synchronous. +3. _p_'s Application executes block _v_. +4. _p_'s Application calculates and returns the _AppHash_, along with a list containing + the outputs of each of the transactions executed. +5. _p_'s CometBFT hashes all the transaction outputs and stores it in _ResultHash_. +6. _p_'s CometBFT persists the transaction outputs, _AppHash_, and _ResultsHash_. +7. _p_'s CometBFT locks the mempool — no calls to `CheckTx` on new transactions. +8. _p_'s CometBFT calls `RequestCommit` to instruct the Application to persist its state. +9. _p_'s CometBFT, optionally, re-checks all outstanding transactions in the mempool + against the newly persisted Application state. +10. _p_'s CometBFT unlocks the mempool — newly received transactions can now be checked. +11. _p_ starts consensus for height _h+1_, round 0 + +## Data Types existing in ABCI + +Most of the data structures used in ABCI are shared [common data structures](/cometbft/v0.39/spec/core/Data_structures). In certain cases, ABCI uses different data structures which are documented here: + +### Validator + +* **Fields**: + + | Name | Type | Description | Field Number | + |---------|-------|------------------------------------------------------------|--------------| + | address | bytes | [Address](/cometbft/v0.39/spec/core/Data_structures#address) of validator | 1 | + | power | int64 | Voting power of the validator | 3 | + +* **Usage**: + * Validator identified by address + * Used as part of `VoteInfo` within `CommitInfo` (used in `ProcessProposal` + and `FinalizeBlock`), and `ExtendedCommitInfo` (used in `PrepareProposal`). + * Does not include PubKey to avoid sending potentially large quantum pubkeys + over the ABCI + +### ValidatorUpdate + +* **Fields**: + + | Name | Type | Description | Field Number | Deterministic | + |---------|--------------------------------------------------|-------------------------------|--------------|---------------| + | pub_key | [Public Key](/cometbft/v0.39/spec/core/Data_structures) | Public key of the validator | 1 | Yes | + | power | int64 | Voting power of the validator | 2 | Yes | + +* **Usage**: + * Validator identified by PubKey + * Used to tell CometBFT to update the validator set + +### Misbehavior + +* **Fields**: + + | Name | Type | Description | Field Number | + |--------------------|-------------------------------------------------|--------------------------------------------------------------|--------------| + | type | [MisbehaviorType](#misbehaviortype) | Type of the misbehavior. An enum of possible misbehaviors. | 1 | + | validator | [Validator](#validator) | The offending validator | 2 | + | height | int64 | Height when the offense occurred | 3 | + | time | [google.protobuf.Timestamp][protobuf-timestamp] | Timestamp of the block that was committed at height `height` | 4 | + | total_voting_power | int64 | Total voting power of the validator set at height `height` | 5 | + +#### MisbehaviorType + +* **Fields** + + MisbehaviorType is an enum with the listed fields: + + | Name | Field Number | + |---------------------|--------------| + | UNKNOWN | 0 | + | DUPLICATE_VOTE | 1 | + | LIGHT_CLIENT_ATTACK | 2 | + +### ConsensusParams + +* **Fields**: + + | Name | Type | Description | Field Number | Deterministic | + |-----------|---------------------------------------------------------------|------------------------------------------------------------------------------|--------------|---------------| + | block | [BlockParams](/cometbft/v0.39/spec/core/Data_structures#blockparams) | Parameters limiting the size of a block and time between consecutive blocks. | 1 | Yes | + | evidence | [EvidenceParams](/cometbft/v0.39/spec/core/Data_structures#evidenceparams) | Parameters limiting the validity of evidence of byzantine behaviour. | 2 | Yes | + | validator | [ValidatorParams](/cometbft/v0.39/spec/core/Data_structures#validatorparams) | Parameters limiting the types of public keys validators can use. | 3 | Yes | + | version | [VersionsParams](/cometbft/v0.39/spec/core/Data_structures#versionparams) | The ABCI application version. | 4 | Yes | + +### ProofOps + +* **Fields**: + + | Name | Type | Description | Field Number | Deterministic | + |------|------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------|---------------| + | ops | repeated [ProofOp](#proofop) | List of chained Merkle proofs, of possibly different types. The Merkle root of one op is the value being proven in the next op. The Merkle root of the final op should equal the ultimate root hash being verified against.. | 1 | N/A | + +### ProofOp + +* **Fields**: + + | Name | Type | Description | Field Number | Deterministic | + |------|--------|------------------------------------------------|--------------|---------------| + | type | string | Type of Merkle proof and how it's encoded. | 1 | N/A | + | key | bytes | Key in the Merkle tree that this proof is for. | 2 | N/A | + | data | bytes | Encoded Merkle proof for the key. | 3 | N/A | + +### Snapshot + +* **Fields**: + + | Name | Type | Description | Field Number | Deterministic | + |----------|--------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------|---------------| + | height | uint64 | The height at which the snapshot was taken (after commit). | 1 | N/A | + | format | uint32 | An application-specific snapshot format, allowing applications to version their snapshot data format and make backwards-incompatible changes. CometBFT does not interpret this. | 2 | N/A | + | chunks | uint32 | The number of chunks in the snapshot. Must be at least 1 (even if empty). | 3 | N/A | + | hash | bytes | An arbitrary snapshot hash. Must be equal only for identical snapshots across nodes. CometBFT does not interpret the hash, it only compares them. | 4 | N/A | + | metadata | bytes | Arbitrary application metadata, for example chunk hashes or other verification data. | 5 | N/A | + +* **Usage**: + * Used for state sync snapshots, see the [state sync section](/cometbft/v0.39/spec/p2p/legacy-docs/messages/state-sync) for details. + * A snapshot is considered identical across nodes only if _all_ fields are equal (including + `Metadata`). Chunks may be retrieved from all nodes that have the same snapshot. + * When sent across the network, a snapshot message can be at most 4 MB. + +## Data types introduced or modified in ABCI++ + +### VoteInfo + +* **Fields**: + + | Name | Type | Description | Field Number | + |---------------|-------------------------------------------------------|------------------------------------------------------------------------------------------|--------------| + | validator | [Validator](#validator) | The validator that sent the vote. | 1 | + | block_id_flag | [BlockIDFlag](/cometbft/v0.39/spec/core/Data_structures#blockidflag) | Indicates whether the validator voted the last block, nil, or its vote was not received. | 3 | + +* **Usage**: + * Indicates whether a validator signed the last block, allowing for rewards based on validator availability. + * This information is typically extracted from a proposed or decided block. + +### ExtendedVoteInfo + +* **Fields**: + + | Name | Type | Description | Field Number | + |---------------------|-------------------------------------------------------|---------------------------------------------------------------------------------------------|--------------| + | validator | [Validator](#validator) | The validator that sent the vote. | 1 | + | vote_extension | bytes | Non-deterministic extension provided by the sending validator's Application. | 3 | + | extension_signature | bytes | Signature of the vote extension produced by the sending validator and verified by CometBFT. | 4 | + | block_id_flag | [BlockIDFlag](/cometbft/v0.39/spec/core/Data_structures#blockidflag) | Indicates whether the validator voted the last block, nil, or its vote was not received. | 5 | + +* **Usage**: + * Indicates whether a validator signed the last block, allowing for rewards based on validator availability. + * This information is extracted from CometBFT's data structures in the local process. + * `vote_extension` contains the sending validator's vote extension, whose signature was verified by CometBFT. It can be empty. + * `extension_signature` is the signature of the vote extension, which was verified verified by CometBFT. This way, we expose the signature to the application for further processing or verification. + +### CommitInfo + +* **Fields**: + + | Name | Type | Description | Field Number | + |-------|--------------------------------|----------------------------------------------------------------------------------------------|--------------| + | round | int32 | Commit round. Reflects the round at which the block proposer decided in the previous height. | 1 | + | votes | repeated [VoteInfo](#voteinfo) | List of validators' addresses in the last validator set with their voting information. | 2 | + +* **Notes** + * The `VoteInfo` in `votes` are ordered by the voting power of the validators (descending order, highest to lowest voting power). + * CometBFT guarantees the `votes` ordering through its logic to update the validator set in which, in the end, the validators are sorted (descending) by their voting power. + * The ordering is also persisted when a validator set is saved in the store. + * The validator set is loaded from the store when building the `CommitInfo`, ensuring order is maintained from the persisted validator set. + +### ExtendedCommitInfo + +* **Fields**: + + | Name | Type | Description | Field Number | + |-------|------------------------------------------------|-------------------------------------------------------------------------------------------------------------------|--------------| + | round | int32 | Commit round. Reflects the round at which the block proposer decided in the previous height. | 1 | + | votes | repeated [ExtendedVoteInfo](#extendedvoteinfo) | List of validators' addresses in the last validator set with their voting information, including vote extensions. | 2 | + +* **Notes** + * The `ExtendedVoteInfo` in `votes` are ordered by the voting power of the validators (descending order, highest to lowest voting power). + * CometBFT guarantees the `votes` ordering through its logic to update the validator set in which, in the end, the validators are sorted (descending) by their voting power. + * The ordering is also persisted when a validator set is saved in the store. + * The validator set is loaded from the store when building the `ExtendedCommitInfo`, ensuring order is maintained from the persisted validator set. + +### ExecTxResult + +* **Fields**: + + | Name | Type | Description | Field Number | Deterministic | + |------------|---------------------------------------------------|----------------------------------------------------------------------|--------------|---------------| + | code | uint32 | Response code. | 1 | Yes | + | data | bytes | Result bytes, if any. | 2 | Yes | + | log | string | The output of the application's logger. | 3 | No | + | info | string | Additional information. | 4 | No | + | gas_wanted | int64 | Amount of gas requested for transaction. | 5 | Yes | + | gas_used | int64 | Amount of gas consumed by transaction. | 6 | Yes | + | events | repeated [Event](/cometbft/v0.39/spec/abci/Outline#events) | Type & Key-Value events for indexing transactions (e.g. by account). | 7 | No | + | codespace | string | Namespace for the `code`. | 8 | Yes | + +### ProposalStatus + +```proto +enum ProposalStatus { + UNKNOWN = 0; // Unknown status. Returning this from the application is always an error. + ACCEPT = 1; // Status that signals that the application finds the proposal valid. + REJECT = 2; // Status that signals that the application finds the proposal invalid. +} +``` + +* **Usage**: + * Used within the [ProcessProposal](#processproposal) response. + * If `Status` is `UNKNOWN`, a problem happened in the Application. CometBFT will assume the application is faulty and crash. + * If `Status` is `ACCEPT`, the consensus algorithm accepts the proposal and will issue a Prevote message for it. + * If `Status` is `REJECT`, the consensus algorithm rejects the proposal and will issue a Prevote for `nil` instead. + + +### VerifyStatus + +```proto +enum VerifyStatus { + UNKNOWN = 0; // Unknown status. Returning this from the application is always an error. + ACCEPT = 1; // Status that signals that the application finds the vote extension valid. + REJECT = 2; // Status that signals that the application finds the vote extension invalid. +} +``` + +* **Usage**: + * Used within the [VerifyVoteExtension](#verifyvoteextension) response. + * If `Status` is `UNKNOWN`, a problem happened in the Application. CometBFT will assume the application is faulty and crash. + * If `Status` is `ACCEPT`, the consensus algorithm will accept the vote as valid. + * If `Status` is `REJECT`, the consensus algorithm will reject the vote as invalid. + +[protobuf-timestamp]: https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#google.protobuf.Timestamp diff --git a/cometbft/v0.39/spec/abci/Outline.mdx b/cometbft/v0.39/spec/abci/Outline.mdx new file mode 100644 index 000000000..3a20da7b4 --- /dev/null +++ b/cometbft/v0.39/spec/abci/Outline.mdx @@ -0,0 +1,439 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/spec/abci/Outline' +order: 1 +title: Overview and basic concepts +--- +{/* trigger rebuild */} + +## Outline + +- [Overview and basic concepts](#overview-and-basic-concepts) + - [ABCI++ vs. ABCI](#abci-vs-abci) + - [Methods overview](#methods-overview) + - [Consensus/block execution methods](#consensusblock-execution-methods) + - [Mempool methods](#mempool-methods) + - [Info methods](#info-methods) + - [State-sync methods](#state-sync-methods) + - [Other methods](#other-methods) + - [Proposal timeout](#proposal-timeout) + - [Deterministic State-Machine Replication](#deterministic-state-machine-replication) + - [Events](#events) + - [Evidence](#evidence) + - [Errors](#errors) + - [`CheckTx`](#checktx) + - [`ExecTxResult` (as part of `FinalizeBlock`)](#exectxresult-as-part-of-finalizeblock) + - [`Query`](#query) + +# Overview and basic concepts + +## ABCI 2.0 vs. ABCI + +[↑ Back to Outline](#outline) + +The Application's main role is to execute blocks decided (a.k.a. finalized) by consensus. The +decided blocks are the consensus's main output to the (replicated) Application. With ABCI, the +application only interacts with consensus at *decision* time. This restricted mode of interaction +prevents numerous features for the Application, including many scalability improvements that are +now better understood than when ABCI was first written. For example, many ideas proposed to improve +scalability can be boiled down to "make the block proposers do work, so the network does not have +to". This includes optimizations such as transaction level signature aggregation, state transition +proofs, etc. Furthermore, many new security properties cannot be achieved in the current paradigm, +as the Application cannot require validators to do more than executing the transactions contained in +finalized blocks. This includes features such as threshold cryptography, and guaranteed IBC +connection attempts. + +ABCI 2.0 addresses these limitations by allowing the application to intervene at three key places of +consensus execution: (a) at the moment a new proposal is to be created, (b) at the moment a +proposal is to be validated, and (c) at the moment a (precommit) vote is sent/received. +The new interface allows block proposers to perform application-dependent +work in a block through the `PrepareProposal` method (a); and validators to perform application-dependent work +and checks in a proposed block through the `ProcessProposal` method (b); and applications to require their validators +to do more than just validate blocks through the `ExtendVote` and `VerifyVoteExtensions` methods (c). + +Furthermore, ABCI 2.0 coalesces {`BeginBlock`, [`DeliverTx`], `EndBlock`} into `FinalizeBlock`, as a +simplified, efficient way to deliver a decided block to the Application. + +## Methods overview + +[↑ Back to Outline](#outline) + +Methods can be classified into four categories: *consensus*, *mempool*, *info*, and *state-sync*. + +### Consensus/block execution methods + +The first time a new blockchain is started, CometBFT calls `InitChain`. From then on, method +`FinalizeBlock` is executed upon the decision of each block, resulting in an updated Application +state. During the execution of an instance of consensus, which decides the block for a given +height, and before method `FinalizeBlock` is called, methods `PrepareProposal`, `ProcessProposal`, +`ExtendVote`, and `VerifyVoteExtension` may be called several times. See +[CometBFT's expected behavior](/cometbft/v0.39/spec/abci/CometBFTs-expected-behavior) for details on the possible +call sequences of these methods. + +- [**InitChain:**](/cometbft/v0.39/spec/abci/Methods#initchain) This method initializes the blockchain. + CometBFT calls it once upon genesis. + +- [**PrepareProposal:**](/cometbft/v0.39/spec/abci/Methods#prepareproposal) It allows the block + proposer to perform application-dependent work in a block before proposing it. + This enables, for instance, batch optimizations to a block, which has been empirically + demonstrated to be a key component for improved performance. Method `PrepareProposal` is called + every time CometBFT is about to broadcast a Proposal message and *validValue* is `nil`. + CometBFT gathers outstanding transactions from the + mempool, generates a block header, and uses them to create a block to propose. Then, it calls + `RequestPrepareProposal` with the newly created proposal, called *raw proposal*. The Application + can make changes to the raw proposal, such as reordering, adding and removing transactions, before returning the + (potentially) modified proposal, called *prepared proposal* in the `ResponsePrepareProposal`. + The logic modifying the raw proposal MAY be non-deterministic. + +- [**ProcessProposal:**](/cometbft/v0.39/spec/abci/Methods#processproposal) It allows a validator to + perform application-dependent work in a proposed block. This enables features such as immediate + block execution, and allows the Application to reject invalid blocks. + + CometBFT calls it when it receives a proposal and *validValue* is `nil`. + The Application cannot modify the proposal at this point but can reject it if + invalid. If that is the case, the consensus algorithm will prevote `nil` on the proposal, which has + strong liveness implications for CometBFT. As a general rule, the Application + SHOULD accept a prepared proposal passed via `ProcessProposal`, even if a part of + the proposal is invalid (e.g., an invalid transaction); the Application can + ignore the invalid part of the prepared proposal at block execution time. + The logic in `ProcessProposal` MUST be deterministic. + +- [**ExtendVote:**](/cometbft/v0.39/spec/abci/Methods#extendvote) It allows applications to let their + validators do more than just validate within consensus. `ExtendVote` allows applications to + include non-deterministic data, opaque to the consensus algorithm, to precommit messages (the final round of + voting). The data, called *vote extension*, will be broadcast and received together with the + vote it is extending, and will be made available to the Application in the next height, + in the rounds where the local process is the proposer. + CometBFT calls `ExtendVote` when the consensus algorithm is about to send a non-`nil` precommit message. + If the Application does not have vote extension information to provide at that time, it returns + a 0-length byte array as its vote extension. + The logic in `ExtendVote` MAY be non-deterministic. + +- [**VerifyVoteExtension:**](/cometbft/v0.39/spec/abci/Methods#verifyvoteextension) It allows + validators to validate the vote extension data attached to a precommit message. If the validation + fails, the whole precommit message will be deemed invalid and ignored by consensus algorithm. + This has a negative impact on liveness, i.e., if vote extensions repeatedly cannot be + verified by correct validators, the consensus algorithm may not be able to finalize a block even if sufficiently + many (+2/3) validators send precommit votes for that block. Thus, `VerifyVoteExtension` + should be implemented with special care. + As a general rule, an Application that detects an invalid vote extension SHOULD + accept it in `ResponseVerifyVoteExtension` and ignore it in its own logic. CometBFT calls it when + a process receives a precommit message with a (possibly empty) vote extension, for the current height. It is not called for precommit votes received after the height is concluded but while waiting to accumulate more precommit votes. + The logic in `VerifyVoteExtension` MUST be deterministic. + +- [**FinalizeBlock:**](/cometbft/v0.39/spec/abci/Methods#finalizeblock) It delivers a decided block to the + Application. The Application must execute the transactions in the block deterministically and + update its state accordingly. Cryptographic commitments to the block and transaction results, + returned via the corresponding parameters in `ResponseFinalizeBlock`, are included in the header + of the next block. CometBFT calls it when a new block is decided. + When calling `FinalizeBlock` with a block, the consensus algorithm run by CometBFT guarantees + that at least one non-byzantine validator has run `ProcessProposal` on that block. + +- [**Commit:**](/cometbft/v0.39/spec/abci/Methods#commit) Instructs the Application to persist its + state. It is a fundamental part of CometBFT's crash-recovery mechanism that ensures the + synchronization between CometBFT and the Application upon recovery. CometBFT calls it just after + having persisted the data returned by calls to `ResponseFinalizeBlock`. The Application can now discard + any state or data except the one resulting from executing the transactions in the decided block. + +### Mempool methods + +- [**CheckTx:**](/cometbft/v0.39/spec/abci/Methods#checktx) This method allows the Application to validate + transactions. Validation can be stateless (e.g., checking signatures ) or stateful + (e.g., account balances). The type of validation performed is up to the application. If a + transaction passes the validation, then CometBFT adds it to the mempool; otherwise the + transaction is discarded. + CometBFT calls it when it receives a new transaction either coming from an external + user (e.g., a client) or another node. Furthermore, CometBFT can be configured to call + re-`CheckTx` on all outstanding transactions in the mempool after calling `Commit` for a block. + +### Info methods + +- [**Info:**](/cometbft/v0.39/spec/abci/Methods#info) Used to sync CometBFT with the Application during a + handshake that happens upon recovery, or on startup when state-sync is used. + +- [**Query:**](/cometbft/v0.39/spec/abci/Methods#query) This method can be used to query the Application for + information about the application state. + +### State-sync methods + +State sync allows new nodes to rapidly bootstrap by discovering, fetching, and applying +state machine (application) snapshots instead of replaying historical blocks. For more details, see the +[state sync documentation](/cometbft/v0.39/spec/p2p/legacy-docs/messages/state-sync). + +New nodes discover and request snapshots from other nodes in the P2P network. +A CometBFT node that receives a request for snapshots from a peer will call +`ListSnapshots` on its Application. The Application returns the list of locally available +snapshots. +Note that the list does not contain the actual snapshots but metadata about them: height at which +the snapshot was taken, application-specific verification data and more (see +[snapshot data type](/cometbft/v0.39/spec/abci/Methods#snapshot) for more details). After receiving a +list of available snapshots from a peer, the new node can offer any of the snapshots in the list to +its local Application via the `OfferSnapshot` method. The Application can check at this point the +validity of the snapshot metadata. + +Snapshots may be quite large and are thus broken into smaller "chunks" that can be +assembled into the whole snapshot. Once the Application accepts a snapshot and +begins restoring it, CometBFT will fetch snapshot "chunks" from existing nodes. +The node providing "chunks" will fetch them from its local Application using +the `LoadSnapshotChunk` method. + +As the new node receives "chunks" it will apply them sequentially to the local +application with `ApplySnapshotChunk`. When all chunks have been applied, the +Application's `AppHash` is retrieved via an `Info` query. +To ensure that the sync proceeded correctly, CometBFT compares the local Application's `AppHash` +to the `AppHash` stored on the blockchain (verified via +[light client verification](/cometbft/v0.39/spec/light-client/verification)). + +In summary: + +- [**ListSnapshots:**](/cometbft/v0.39/spec/abci/Methods#listsnapshots) Used by nodes to discover available + snapshots on peers. + +- [**OfferSnapshot:**](/cometbft/v0.39/spec/abci/Methods#offersnapshot) When a node receives a snapshot from a + peer, CometBFT uses this method to offer the snapshot to the Application. + +- [**LoadSnapshotChunk:**](/cometbft/v0.39/spec/abci/Methods#loadsnapshotchunk) Used by CometBFT to retrieve + snapshot chunks from the Application to send to peers. + +- [**ApplySnapshotChunk:**](/cometbft/v0.39/spec/abci/Methods#applysnapshotchunk) Used by CometBFT to hand + snapshot chunks to the Application. + +### Other methods + +Additionally, there is a [**Flush**](/cometbft/v0.39/spec/abci/Methods#flush) method that is called on every connection, +and an [**Echo**](/cometbft/v0.39/spec/abci/Methods#echo) method that is used for debugging. + +More details on managing state across connections can be found in the section on +[Managing Application State](/cometbft/v0.39/spec/abci/Requirements-for-the-Application#managing-the-application-state-and-related-topics). + +## Proposal timeout + +`PrepareProposal` stands on the consensus algorithm critical path, +i.e., CometBFT cannot make progress while this method is being executed. +Hence, if the Application takes a long time preparing a proposal, +the default value of *TimeoutPropose* might not be sufficient +to accommodate the method's execution and validator nodes might time out and prevote `nil`. +The proposal, in this case, will probably be rejected and a new round will be necessary. + +Timeouts are automatically increased for each new round of a height and, if the execution of `PrepareProposal` is bound, eventually *TimeoutPropose* will be long enough to accommodate the execution of `PrepareProposal`. +However, relying on this self adaptation could lead to performance degradation and, therefore, +operators are suggested to adjust the initial value of *TimeoutPropose* in CometBFT's configuration file, +in order to suit the needs of the particular application being deployed. + +This is particularly important if applications implement *immediate execution*. +To implement this technique, proposers need to execute the block being proposed within `PrepareProposal`, which could take longer than *TimeoutPropose*. + +## Deterministic State-Machine Replication + +[↑ Back to Outline](#outline) + +ABCI applications must implement deterministic finite-state machines to be +securely replicated by the CometBFT consensus engine. This means block execution +must be strictly deterministic: given the same +ordered set of transactions, all nodes will compute identical responses, for all +successive `FinalizeBlock` calls. This is critical because the +responses are included in the header of the next block, either via a Merkle root +or directly, so all nodes must agree on exactly what they are. + +For this reason, it is recommended that application state is not exposed to any +external user or process except via the ABCI connections to a consensus engine +like CometBFT. The Application must only change its state based on input +from block execution (`FinalizeBlock` calls), and not through +any other kind of request. This is the only way to ensure all nodes see the same +transactions and compute the same results. + +Applications that implement immediate execution (execute the blocks +that are about to be proposed, in `PrepareProposal`, or that require validation, in `ProcessProposal`) produce a new candidate state before a block is decided. +The state changes caused by processing those +proposed blocks must never replace the previous state until `FinalizeBlock` confirms +that the proposed block was decided and `Commit` is invoked for it. + +The same is true to Applications that quickly accept blocks and execute the +blocks optimistically in parallel with the remaining consensus steps to save +time during `FinalizeBlock`; they must only apply state changes in `Commit`. + +Additionally, vote extensions or the validation thereof (via `ExtendVote` or +`VerifyVoteExtension`) must *never* have side effects on the current state. +They can only be used when their data is provided in a `RequestPrepareProposal` call but, again, +without side effects to the app state. + +If there is some non-determinism in the state machine, consensus will eventually +fail as nodes disagree over the correct values for the block header. The +non-determinism must be fixed and the nodes restarted. + +Sources of non-determinism in applications may include: + +- Hardware failures + - Cosmic rays, overheating, etc. +- Node-dependent state + - Random numbers + - Time +- Underspecification + - Library version changes + - Race conditions + - Floating point numbers + - JSON or protobuf serialization + - Iterating through hash-tables/maps/dictionaries +- External Sources + - Filesystem + - Network calls (eg. some external REST API service) + +See [#56](https://github.com/tendermint/abci/issues/56) for the original discussion. + +Note that some methods (e.g., `Query` and `FinalizeBlock`) may return +non-deterministic data in the form of `Info`, `Log` and/or `Events` fields. The +`Log` is intended for the literal output from the Application's logger, while +the `Info` is any additional info that should be returned. These fields are not +included in block header computations, so we don't need agreement on them. See +each field's description on whether it must be deterministic or not. + +## Events + +[↑ Back to Outline](#outline) + +Method `FinalizeBlock` includes an `events` field at the top level in its +`Response*`, and one `events` field per transaction included in the block. +Applications may respond to this ABCI 2.0 method with an event list for each executed +transaction, and a general event list for the block itself. +Events allow applications to associate metadata with transactions and blocks. +Events returned via `FinalizeBlock` do not impact the consensus algorithm in any way +and instead exist to power subscriptions and queries of CometBFT state. + +An `Event` contains a `type` and a list of `EventAttributes`, which are key-value +string pairs denoting metadata about what happened during the method's (or transaction's) +execution. `Event` values can be used to index transactions and blocks according to what +happened during their execution. + +Each event has a `type` which is meant to categorize the event for a particular +`Response*` or `Tx`. A `Response*` or `Tx` may contain multiple events with duplicate +`type` values, where each distinct entry is meant to categorize attributes for a +particular event. Every key and value in an event's attributes must be UTF-8 +encoded strings along with the event type itself. + +```protobuf +message Event { + string type = 1; + repeated EventAttribute attributes = 2; +} +``` + +The attributes of an `Event` consist of a `key`, a `value`, and an `index` +flag. The index flag notifies the CometBFT indexer to index the attribute. + +The `type` and `attributes` fields are non-deterministic and may vary across +different nodes in the network. + +```protobuf +message EventAttribute { + string key = 1; + string value = 2; + bool index = 3; // nondeterministic +} +``` + +Example: + +```go + abci.ResponseFinalizeBlock{ + // ... + Events: []abci.Event{ + { + Type: "validator.provisions", + Attributes: []abci.EventAttribute{ + abci.EventAttribute{Key: "address", Value: "...", Index: true}, + abci.EventAttribute{Key: "amount", Value: "...", Index: true}, + abci.EventAttribute{Key: "balance", Value: "...", Index: true}, + }, + }, + { + Type: "validator.provisions", + Attributes: []abci.EventAttribute{ + abci.EventAttribute{Key: "address", Value: "...", Index: true}, + abci.EventAttribute{Key: "amount", Value: "...", Index: false}, + abci.EventAttribute{Key: "balance", Value: "...", Index: false}, + }, + }, + { + Type: "validator.slashed", + Attributes: []abci.EventAttribute{ + abci.EventAttribute{Key: "address", Value: "...", Index: false}, + abci.EventAttribute{Key: "amount", Value: "...", Index: true}, + abci.EventAttribute{Key: "reason", Value: "...", Index: true}, + }, + }, + // ... + }, +} +``` + +## Evidence + +[↑ Back to Outline](#outline) + +CometBFT's security model relies on the use of evidences of misbehavior. An evidence is an +irrefutable proof of malicious behavior by a network participant. It is the responsibility of +CometBFT to detect such malicious behavior. When malicious behavior is detected, CometBFT +will gossip evidences of misbehavior to other nodes and commit the evidences to +the chain once they are verified by a subset of validators. These evidences will then be +passed on to the Application through ABCI++. It is the responsibility of the +Application to handle evidence of misbehavior and exercise punishment. + +There are two forms of evidence: Duplicate Vote and Light Client Attack. More +information can be found in either [data structures](/cometbft/v0.39/spec/core/Data_structures) +or [accountability](/cometbft/v0.39/spec/light-client/Accountability). + +EvidenceType has the following protobuf format: + +```protobuf +enum EvidenceType { + UNKNOWN = 0; + DUPLICATE_VOTE = 1; + LIGHT_CLIENT_ATTACK = 2; +} +``` + +## Errors + +[↑ Back to Outline](#outline) + +The `Query` and `CheckTx` methods include a `Code` field in their `Response*`. +Field `Code` is meant to contain an application-specific response code. +A response code of `0` indicates no error. Any other response code +indicates to CometBFT that an error occurred. + +These methods also return a `Codespace` string to CometBFT. This field is +used to disambiguate `Code` values returned by different domains of the +Application. The `Codespace` is a namespace for the `Code`. + +Methods `Echo`, `Info`, `Commit` and `InitChain` do not return errors. +An error in any of these methods represents a critical issue that CometBFT +has no reasonable way to handle. If there is an error in one +of these methods, the Application must crash to ensure that the error is safely +handled by an operator. + +Method `FinalizeBlock` is a special case. It contains a number of +`Code` and `Codespace` fields as part of type `ExecTxResult`. Each of +these codes reports errors related to the transaction it is attached to. +However, `FinalizeBlock` does not return errors at the top level, so the +same considerations on critical issues made for `Echo`, `Info`, and +`InitChain` also apply here. + +The handling of non-zero response codes by CometBFT is described below. + +### `CheckTx` + +When CometBFT receives a `ResponseCheckTx` with a non-zero `Code`, the associated +transaction will not be added to CometBFT's mempool or it will be removed if +it is already included. + +### `ExecTxResult` (as part of `FinalizeBlock`) + +The `ExecTxResult` type delivers transaction results from the Application to CometBFT. When +CometBFT receives a `ResponseFinalizeBlock` containing an `ExecTxResult` with a non-zero `Code`, +the response code is logged. Past `Code` values can be queried by clients. As the transaction was +part of a decided block, the `Code` does not influence consensus. + +### `Query` + +When CometBFT receives a `ResponseQuery` with a non-zero `Code`, this code is +returned directly to the client that initiated the query. diff --git a/cometbft/v0.39/spec/abci/Overview.mdx b/cometbft/v0.39/spec/abci/Overview.mdx new file mode 100644 index 000000000..72ae7879d --- /dev/null +++ b/cometbft/v0.39/spec/abci/Overview.mdx @@ -0,0 +1,44 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/spec/abci/Overview' +order: 1 +parent: + title: ABCI++ + order: 3 +--- +{/* trigger rebuild */} + +# ABCI++ + +## Introduction + +ABCI++ is a major evolution of ABCI (**A**pplication **B**lock**c**hain **I**nterface). +Like its predecessor, ABCI++ is the interface between CometBFT (a state-machine +replication engine) and the actual state machine being replicated (i.e., the Application). +The API consists of a set of _methods_, each with a corresponding `Request` and `Response` +message type. + +The methods are always initiated by CometBFT. The Application implements its logic +for handling all ABCI++ methods. +Thus, CometBFT always sends the `Request*` messages and receives the `Response*` messages +in return. + +All ABCI++ messages and methods are defined in [protocol buffers](https://github.com/cometbft/cometbft/blob/v0.38.x/proto/tendermint/abci/types.proto). +This allows CometBFT to run with applications written in many programming languages. + +This specification is split as follows: + +- [Overview and basic concepts](/cometbft/v0.39/spec/abci/Outline) - interface's overview and concepts + needed to understand other parts of this specification. +- [Methods](/cometbft/v0.39/spec/abci/Methods) - complete details on all ABCI++ methods + and message types. +- [Requirements for the Application](/cometbft/v0.39/spec/abci/Requirements-for-the-Application) - formal requirements + on the Application's logic to ensure CometBFT properties such as liveness. These requirements define what + CometBFT expects from the Application; second part on managing ABCI application state and related topics. +- [CometBFT's expected behavior](/cometbft/v0.39/spec/abci/CometBFTs-expected-behavior) - specification of + how the different ABCI++ methods may be called by CometBFT. This explains what the Application + is to expect from CometBFT. +- [Example scenarios](/cometbft/v0.39/spec/abci/Introduction) - specific scenarios showing why the Application needs to account +for any CometBFT's behaviour prescribed by the specification. +- [Client and Server](/cometbft/v0.39/spec/abci/Client-and-server) - for those looking to implement their + own ABCI application servers. diff --git a/cometbft/v0.39/spec/abci/Requirements-for-the-Application.mdx b/cometbft/v0.39/spec/abci/Requirements-for-the-Application.mdx new file mode 100644 index 000000000..213b4964c --- /dev/null +++ b/cometbft/v0.39/spec/abci/Requirements-for-the-Application.mdx @@ -0,0 +1,1070 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/spec/abci/Requirements-for-the-Application' +order: 3 +title: Requirements for the Application +--- + +- [Requirements for the Application](#requirements-for-the-application) + - [Formal Requirements](#formal-requirements) + - [Consensus Connection Requirements](#consensus-connection-requirements) + - [Mempool Connection Requirements](#mempool-connection-requirements) + - [Managing the Application state and related topics](#managing-the-application-state-and-related-topics) + - [Connection State](#connection-state) + - [Concurrency](#concurrency) + - [FinalizeBlock](#finalizeblock) + - [Commit](#commit) + - [Candidate States](#candidate-states) + - [States and ABCI++ Connections](#states-and-abci-connections) + - [Consensus Connection](#consensus-connection) + - [Mempool Connection](#mempool-connection) + - [Replay Protection](#replay-protection) + - [Info/Query Connection](#infoquery-connection) + - [Snapshot Connection](#snapshot-connection) + - [Transaction Results](#transaction-results) + - [Gas](#gas) + - [Specifics of `ResponseCheckTx`](#specifics-of-responsechecktx) + - [Specifics of `ExecTxResult`](#specifics-of-exectxresult) + - [Updating the Validator Set](#updating-the-validator-set) + - [Consensus Parameters](#consensus-parameters) + - [List of Parameters](#list-of-parameters) + - [ABCIParams.VoteExtensionsEnableHeight](#abciparamsvoteextensionsenableheight) + - [BlockParams.MaxBytes](#blockparamsmaxbytes) + - [BlockParams.MaxGas](#blockparamsmaxgas) + - [EvidenceParams.MaxAgeDuration](#evidenceparamsmaxageduration) + - [EvidenceParams.MaxAgeNumBlocks](#evidenceparamsmaxagenumblocks) + - [EvidenceParams.MaxBytes](#evidenceparamsmaxbytes) + - [ValidatorParams.PubKeyTypes](#validatorparamspubkeytypes) + - [VersionParams.App](#versionparamsapp) + - [Updating Consensus Parameters](#updating-consensus-parameters) + - [`InitChain`](#initchain) + - [`FinalizeBlock`, `PrepareProposal`/`ProcessProposal`](#finalizeblock-prepareproposalprocessproposal) + - [`Query`](#query) + - [Query Proofs](#query-proofs) + - [Peer Filtering](#peer-filtering) + - [Paths](#paths) + - [Crash Recovery](#crash-recovery) + - [State Sync](#state-sync) + - [Taking Snapshots](#taking-snapshots) + - [Bootstrapping a Node](#bootstrapping-a-node) + - [Snapshot Discovery](#snapshot-discovery) + - [Snapshot Restoration](#snapshot-restoration) + - [Snapshot Verification](#snapshot-verification) + - [Transition to Consensus](#transition-to-consensus) + - [Application configuration required to switch to ABCI 2.0](#application-configuration-required-to-switch-to-abci-20) + + +## Formal Requirements + +### Consensus Connection Requirements + +This section specifies what CometBFT expects from the Application. It is structured as a set +of formal requirements that can be used for testing and verification of the Application's logic. + +Let *p* and *q* be two correct processes. +Let *rp* (resp. *rq*) be a round of height *h* where *p* (resp. *q*) is the +proposer. +Let *sp,h-1* be *p*'s Application's state committed for height *h-1*. +Let *vp* (resp. *vq*) be the block that *p*'s (resp. *q*'s) CometBFT passes +on to the Application +via `RequestPrepareProposal` as proposer of round *rp* (resp *rq*), height *h*, +also known as the raw proposal. +Let *up* (resp. *uq*) the possibly modified block *p*'s (resp. *q*'s) Application +returns via `ResponsePrepareProposal` to CometBFT, also known as the prepared proposal. + +Process *p*'s prepared proposal can differ in two different rounds where *p* is the proposer. + +- Requirement 1 [`PrepareProposal`, timeliness]: If *p*'s Application fully executes prepared blocks in + `PrepareProposal` and the network is in a synchronous period while processes *p* and *q* are in *rp*, + then the value of *TimeoutPropose* at *q* must be such that *q*'s propose timer does not time out + (which would result in *q* prevoting `nil` in *rp*). + +Full execution of blocks at `PrepareProposal` time stands on CometBFT's critical path. Thus, +Requirement 1 ensures the Application or operator will set a value for `TimeoutPropose` such that the time it takes +to fully execute blocks in `PrepareProposal` does not interfere with CometBFT's propose timer. +Note that the violation of Requirement 1 may lead to further rounds, but will not +compromise liveness because even though `TimeoutPropose` is used as the initial +value for proposal timeouts, CometBFT will be dynamically adjust these timeouts +such that they will eventually be enough for completing `PrepareProposal`. + +- Requirement 2 [`PrepareProposal`, tx-size]: When *p*'s Application calls `ResponsePrepareProposal`, the + total size in bytes of the transactions returned does not exceed `RequestPrepareProposal.max_tx_bytes`. + +Busy blockchains might seek to gain full visibility into transactions in CometBFT's mempool, +rather than having visibility only on *a* subset of those transactions that fit in a block. +The application can do so by setting `ConsensusParams.Block.MaxBytes` to -1. +This instructs CometBFT (a) to enforce the maximum possible value for `MaxBytes` (100 MB) at CometBFT level, +and (b) to provide *all* transactions in the mempool when calling `RequestPrepareProposal`. +Under these settings, the aggregated size of all transactions may exceed `RequestPrepareProposal.max_tx_bytes`. +Hence, Requirement 2 ensures that the size in bytes of the transaction list returned by the application will never +cause the resulting block to go beyond its byte size limit. + +- Requirement 3 [`PrepareProposal`, `ProcessProposal`, coherence]: For any two correct processes *p* and *q*, + if *q*'s CometBFT calls `RequestProcessProposal` on *up*, + *q*'s Application returns Accept in `ResponseProcessProposal`. + +Requirement 3 makes sure that blocks proposed by correct processes *always* pass the correct receiving process's +`ProcessProposal` check. +On the other hand, if there is a deterministic bug in `PrepareProposal` or `ProcessProposal` (or in both), +strictly speaking, this makes all processes that hit the bug byzantine. This is a problem in practice, +as very often validators are running the Application from the same codebase, so potentially *all* would +likely hit the bug at the same time. This would result in most (or all) processes prevoting `nil`, with the +serious consequences on CometBFT's liveness that this entails. Due to its criticality, Requirement 3 is a +target for extensive testing and automated verification. + +- Requirement 4 [`ProcessProposal`, determinism-1]: `ProcessProposal` is a (deterministic) function of the current + state and the block that is about to be applied. In other words, for any correct process *p*, and any arbitrary block *u*, + if *p*'s CometBFT calls `RequestProcessProposal` on *u* at height *h*, + then *p*'s Application's acceptance or rejection **exclusively** depends on *u* and *sp,h-1*. + +- Requirement 5 [`ProcessProposal`, determinism-2]: For any two correct processes *p* and *q*, and any arbitrary + block *u*, + if *p*'s (resp. *q*'s) CometBFT calls `RequestProcessProposal` on *u* at height *h*, + then *p*'s Application accepts *u* if and only if *q*'s Application accepts *u*. + Note that this requirement follows from Requirement 4 and the Agreement property of consensus. + +Requirements 4 and 5 ensure that all correct processes will react in the same way to a proposed block, even +if the proposer is Byzantine. However, `ProcessProposal` may contain a bug that renders the +acceptance or rejection of the block non-deterministic, and therefore prevents processes hitting +the bug from fulfilling Requirements 4 or 5 (effectively making those processes Byzantine). +In such a scenario, CometBFT's liveness cannot be guaranteed. +Again, this is a problem in practice if most validators are running the same software, as they are likely +to hit the bug at the same point. There is currently no clear solution to help with this situation, so +the Application designers/implementors must proceed very carefully with the logic/implementation +of `ProcessProposal`. As a general rule `ProcessProposal` SHOULD always accept the block. + +According to the Tendermint consensus algorithm, currently adopted in CometBFT, +a correct process can broadcast at most one precommit +message in round *r*, height *h*. +Since, as stated in the [Methods](/cometbft/v0.39/spec/abci/Methods#extendvote) section, `ResponseExtendVote` +is only called when the consensus algorithm +is about to broadcast a non-`nil` precommit message, a correct process can only produce one vote extension +in round *r*, height *h*. +Let *erp* be the vote extension that the Application of a correct process *p* returns via +`ResponseExtendVote` in round *r*, height *h*. +Let *wrp* be the proposed block that *p*'s CometBFT passes to the Application via `RequestExtendVote` +in round *r*, height *h*. + +- Requirement 6 [`ExtendVote`, `VerifyVoteExtension`, coherence]: For any two different correct + processes *p* and *q*, if *q* receives *erp* from *p* in height *h*, *q*'s + Application returns Accept in `ResponseVerifyVoteExtension`. + +Requirement 6 constrains the creation and handling of vote extensions in a similar way as Requirement 3 +constrains the creation and handling of proposed blocks. +Requirement 6 ensures that extensions created by correct processes *always* pass the `VerifyVoteExtension` +checks performed by correct processes receiving those extensions. +However, if there is a (deterministic) bug in `ExtendVote` or `VerifyVoteExtension` (or in both), +we will face the same liveness issues as described for Requirement 5, as Precommit messages with invalid vote +extensions will be discarded. + +- Requirement 7 [`VerifyVoteExtension`, determinism-1]: `VerifyVoteExtension` is a (deterministic) function of + the current state, the vote extension received, and the prepared proposal that the extension refers to. + In other words, for any correct process *p*, and any arbitrary vote extension *e*, and any arbitrary + block *w*, if *p*'s (resp. *q*'s) CometBFT calls `RequestVerifyVoteExtension` on *e* and *w* at height *h*, + then *p*'s Application's acceptance or rejection **exclusively** depends on *e*, *w* and *sp,h-1*. + +- Requirement 8 [`VerifyVoteExtension`, determinism-2]: For any two correct processes *p* and *q*, + and any arbitrary vote extension *e*, and any arbitrary block *w*, + if *p*'s (resp. *q*'s) CometBFT calls `RequestVerifyVoteExtension` on *e* and *w* at height *h*, + then *p*'s Application accepts *e* if and only if *q*'s Application accepts *e*. + Note that this requirement follows from Requirement 7 and the Agreement property of consensus. + +Requirements 7 and 8 ensure that the validation of vote extensions will be deterministic at all +correct processes. +Requirements 7 and 8 protect against arbitrary vote extension data from Byzantine processes, +in a similar way as Requirements 4 and 5 protect against arbitrary proposed blocks. +Requirements 7 and 8 can be violated by a bug inducing non-determinism in +`VerifyVoteExtension`. In this case liveness can be compromised. +Extra care should be put in the implementation of `ExtendVote` and `VerifyVoteExtension`. +As a general rule, `VerifyVoteExtension` SHOULD always accept the vote extension. + +- Requirement 9 [*all*, no-side-effects]: *p*'s calls to `RequestPrepareProposal`, + `RequestProcessProposal`, `RequestExtendVote`, and `RequestVerifyVoteExtension` at height *h* do + not modify *sp,h-1*. + + +- Requirement 10 [`ExtendVote`, `FinalizeBlock`, non-dependency]: for any correct process *p*, +and any vote extension *e* that *p* received at height *h*, the computation of +*sp,h* does not depend on *e*. + +The call to correct process *p*'s `RequestFinalizeBlock` at height *h*, with block *vp,h* +passed as parameter, creates state *sp,h*. +Additionally, *p*'s `FinalizeBlock` creates a set of transaction results *Tp,h*. + +- Requirement 11 [`FinalizeBlock`, determinism-1]: For any correct process *p*, + *sp,h* exclusively depends on *sp,h-1* and *vp,h*. + +- Requirement 12 [`FinalizeBlock`, determinism-2]: For any correct process *p*, + the contents of *Tp,h* exclusively depend on *sp,h-1* and *vp,h*. + +Note that Requirements 11 and 12, combined with the Agreement property of consensus ensure +state machine replication, i.e., the Application state evolves consistently at all correct processes. + +Also, notice that neither `PrepareProposal` nor `ExtendVote` have determinism-related +requirements associated. +Indeed, `PrepareProposal` is not required to be deterministic: + +- *up* may depend on *vp* and *sp,h-1*, but may also depend on other values or operations. +- *vp = vq ⇏ up = uq*. + +Likewise, `ExtendVote` can also be non-deterministic: + +- *erp* may depend on *wrp* and *sp,h-1*, + but may also depend on other values or operations. +- *wrp = wrq ⇏ + erp = erq* + +### Mempool Connection Requirements + +Let *CheckTxCodestx,p,h* denote the set of result codes returned by *p*'s Application, +via `ResponseCheckTx`, +to successive calls to `RequestCheckTx` occurring while the Application is at height *h* +and having transaction *tx* as parameter. +*CheckTxCodestx,p,h* is a set since *p*'s Application may +return different result codes during height *h*. +If *CheckTxCodestx,p,h* is a singleton set, i.e. the Application always returned +the same result code in `ResponseCheckTx` while at height *h*, +we define *CheckTxCodetx,p,h* as the singleton value of *CheckTxCodestx,p,h*. +If *CheckTxCodestx,p,h* is not a singleton set, *CheckTxCodetx,p,h* is undefined. +Let predicate *OK(CheckTxCodetx,p,h)* denote whether *CheckTxCodetx,p,h* is `SUCCESS`. + +- Requirement 13 [`CheckTx`, eventual non-oscillation]: For any transaction *tx*, + there exists a boolean value *b*, + and a height *hstable* such that, + for any correct process *p*, + *CheckTxCodetx,p,h* is defined, and + *OK(CheckTxCodetx,p,h) = b* + for any height *h ≥ hstable*. + +Requirement 13 ensures that +a transaction will eventually stop oscillating between `CheckTx` success and failure +if it stays in *p's* mempool for long enough. +This condition on the Application's behavior allows the mempool to ensure that +a transaction will leave the mempool of all full nodes, +either because it is expunged everywhere due to failing `CheckTx` calls, +or because it stays valid long enough to be gossipped, proposed and decided. +Although Requirement 13 defines a global *hstable*, application developers +can consider such stabilization height as local to process *p* (*hp,stable*), +without loss for generality. +In contrast, the value of *b* MUST be the same across all processes. + +## Managing the Application state and related topics + +### Connection State + +CometBFT maintains four concurrent ABCI++ connections, namely +[Consensus Connection](#consensus-connection), +[Mempool Connection](#mempool-connection), +[Info/Query Connection](#infoquery-connection), and +[Snapshot Connection](#snapshot-connection). +It is common for an application to maintain a distinct copy of +the state for each connection, which are synchronized upon `Commit` calls. + +#### Concurrency + +In principle, each of the four ABCI++ connections operates concurrently with one +another. This means applications need to ensure access to state is +thread safe. Both the +[default in-process ABCI client](https://github.com/cometbft/cometbft/blob/v0.38.x/abci/client/local_client.go#L13) +and the +[default Go ABCI server](https://github.com/cometbft/cometbft/blob/v0.38.x/abci/server/socket_server.go#L20) +use a global lock to guard the handling of events across all connections, so they are not +concurrent at all. This means whether your app is compiled in-process with +CometBFT using the `NewLocalClient`, or run out-of-process using the `SocketServer`, +ABCI messages from all connections are received in sequence, one at a +time. + +The existence of this global mutex means Go application developers can get thread safety for application state by routing all reads and writes through the ABCI system. Thus it may be unsafe to expose application state directly to an RPC interface, and unless explicit measures are taken, all queries should be routed through the ABCI Query method. + +#### FinalizeBlock + +When the consensus algorithm decides on a block, CometBFT uses `FinalizeBlock` to send the +decided block's data to the Application, which uses it to transition its state, but MUST NOT persist it; +persisting MUST be done during `Commit`. + +The Application must remember the latest height from which it +has run a successful `Commit` so that it can tell CometBFT where to +pick up from when it recovers from a crash. See information on the Handshake +[here](#crash-recovery). + +#### Commit + +The Application should persist its state during `Commit`, before returning from it. + +Before invoking `Commit`, CometBFT locks the mempool and flushes the mempool connection. This ensures that +no new messages +will be received on the mempool connection during this processing step, providing an opportunity to safely +update all four +connection states to the latest committed state at the same time. + +When `Commit` returns, CometBFT unlocks the mempool. + +WARNING: if the ABCI app logic processing the `Commit` message sends a +`/broadcast_tx_sync` or `/broadcast_tx` and waits for the response +before proceeding, it will deadlock. Executing `broadcast_tx` calls +involves acquiring the mempool lock that CometBFT holds during the `Commit` call. +Synchronous mempool-related calls must be avoided as part of the sequential logic of the +`Commit` function. + +#### Candidate States + +CometBFT calls `PrepareProposal` when it is about to send a proposed block to the network. +Likewise, CometBFT calls `ProcessProposal` upon reception of a proposed block from the +network. The proposed block's data +that is disclosed to the Application by these two methods is the following: + +- the transaction list +- the `LastCommit` referring to the previous block +- the block header's hash (except in `PrepareProposal`, where it is not known yet) +- list of validators that misbehaved +- the block's timestamp +- `NextValidatorsHash` +- Proposer address + +The Application may decide to *immediately* execute the given block (i.e., upon `PrepareProposal` +or `ProcessProposal`). There are two main reasons why the Application may want to do this: + +- *Avoiding invalid transactions in blocks*. + In order to be sure that the block does not contain *any* invalid transaction, there may be + no way other than fully executing the transactions in the block as though it was the *decided* + block. +- *Quick `FinalizeBlock` execution*. + Upon reception of the decided block via `FinalizeBlock`, if that same block was executed + upon `PrepareProposal` or `ProcessProposal` and the resulting state was kept in memory, the + Application can simply apply that state (faster) to the main state, rather than reexecuting + the decided block (slower). + +`PrepareProposal`/`ProcessProposal` can be called many times for a given height. Moreover, +it is not possible to accurately predict which of the blocks proposed in a height will be decided, +being delivered to the Application in that height's `FinalizeBlock`. +Therefore, the state resulting from executing a proposed block, denoted a *candidate state*, should +be kept in memory as a possible final state for that height. When `FinalizeBlock` is called, the Application should +check if the decided block corresponds to one of its candidate states; if so, it will apply it as +its *ExecuteTxState* (see [Consensus Connection](#consensus-connection) below), +which will be persisted during the upcoming `Commit` call. + +Under adverse conditions (e.g., network instability), the consensus algorithm might take many rounds. +In this case, potentially many proposed blocks will be disclosed to the Application for a given height. +By the nature of Tendermint consensus algorithm, currently adopted in CometBFT, the number of proposed blocks received by the Application +for a particular height cannot be bound, so Application developers must act with care and use mechanisms +to bound memory usage. As a general rule, the Application should be ready to discard candidate states +before `FinalizeBlock`, even if one of them might end up corresponding to the +decided block and thus have to be reexecuted upon `FinalizeBlock`. + +### [States and ABCI++ Connections](#states-and-abci-connections) + +#### Consensus Connection + +The Consensus Connection should maintain an *ExecuteTxState* — the working state +for block execution. It should be updated by the call to `FinalizeBlock` +during block execution and committed to disk as the "latest +committed state" during `Commit`. Execution of a proposed block (via `PrepareProposal`/`ProcessProposal`) +**must not** update the *ExecuteTxState*, but rather be kept as a separate candidate state until `FinalizeBlock` +confirms which of the candidate states (if any) can be used to update *ExecuteTxState*. + +#### Mempool Connection + +The mempool Connection maintains *CheckTxState*. CometBFT sequentially processes an incoming +transaction (via RPC from client or P2P from the gossip layer) against *CheckTxState*. +If the processing does not return any error, the transaction is accepted into the mempool +and CometBFT starts gossipping it. +*CheckTxState* should be reset to the latest committed state +at the end of every `Commit`. + +During the execution of a consensus instance, the *CheckTxState* may be updated concurrently with the +*ExecuteTxState*, as messages may be sent concurrently on the Consensus and Mempool connections. +At the end of the consensus instance, as described above, CometBFT locks the mempool and flushes +the mempool connection before calling `Commit`. This ensures that all pending `CheckTx` calls are +responded to and no new ones can begin. + +After the `Commit` call returns, while still holding the mempool lock, `CheckTx` is run again on all +transactions that remain in the node's local mempool after filtering those included in the block. +Parameter `Type` in `RequestCheckTx` +indicates whether an incoming transaction is new (`CheckTxType_New`), or a +recheck (`CheckTxType_Recheck`). + +Finally, after re-checking transactions in the mempool, CometBFT will unlock +the mempool connection. New transactions are once again able to be processed through `CheckTx`. + +Note that `CheckTx` is just a weak filter to keep invalid transactions out of the mempool and, +ultimately, ouf of the blockchain. +Since the transaction cannot be guaranteed to be checked against the exact same state as it +will be executed as part of a (potential) decided block, `CheckTx` shouldn't check *everything* +that affects the transaction's validity, in particular those checks whose validity may depend on +transaction ordering. `CheckTx` is weak because a Byzantine node need not care about `CheckTx`; +it can propose a block full of invalid transactions if it wants. The mechanism ABCI++ has +in place for dealing with such behavior is `ProcessProposal`. + +##### Replay Protection + +It is possible for old transactions to be sent again to the Application. This is typically +undesirable for all transactions, except for a generally small subset of them which are idempotent. + +The mempool has a mechanism to prevent duplicated transactions from being processed. +This mechanism is nevertheless best-effort (currently based on the indexer) +and does not provide any guarantee of non duplication. +It is thus up to the Application to implement an application-specific +replay protection mechanism with strong guarantees as part of the logic in `CheckTx`. + +#### Info/Query Connection + +The Info (or Query) Connection should maintain a `QueryState`. This connection has two +purposes: 1) having the application answer the queries CometBFT receives from users +(see section [Query](#query)), +and 2) synchronizing CometBFT and the Application at start up time (see +[Crash Recovery](#crash-recovery)) +or after state sync (see [State Sync](#state-sync)). + +`QueryState` is a read-only copy of *ExecuteTxState* as it was after the last +`Commit`, i.e. +after the full block has been processed and the state committed to disk. + +#### Snapshot Connection + +The Snapshot Connection is used to serve state sync snapshots for other nodes +and/or restore state sync snapshots to a local node being bootstrapped. +Snapshot management is optional: an Application may choose not to implement it. + +For more information, see Section [State Sync](#state-sync). + +### Transaction Results + +The Application is expected to return a list of +[`ExecTxResult`](/cometbft/v0.39/spec/abci/Methods#exectxresult) in +[`ResponseFinalizeBlock`](/cometbft/v0.39/spec/abci/Methods#finalizeblock). The list of transaction +results MUST respect the same order as the list of transactions delivered via +[`RequestFinalizeBlock`](/cometbft/v0.39/spec/abci/Methods#finalizeblock). +This section discusses the fields inside this structure, along with the fields in +[`ResponseCheckTx`](/cometbft/v0.39/spec/abci/Methods#checktx), +whose semantics are similar. + +The `Info` and `Log` fields are +non-deterministic values for debugging/convenience purposes. CometBFT logs them but they +are otherwise ignored. + +#### Gas + +Ethereum introduced the notion of *gas* as an abstract representation of the +cost of the resources consumed by nodes when processing a transaction. Every operation in the +Ethereum Virtual Machine uses some amount of gas. +Gas has a market-variable price based on which miners can accept or reject to execute a +particular operation. + +Users propose a maximum amount of gas for their transaction; if the transaction uses less, they get +the difference credited back. CometBFT adopts a similar abstraction, +though uses it only optionally and weakly, allowing applications to define +their own sense of the cost of execution. + +In CometBFT, the [ConsensusParams.Block.MaxGas](#consensus-parameters) limits the amount of +total gas that can be used by all transactions in a block. +The default value is `-1`, which means the block gas limit is not enforced, or that the concept of +gas is meaningless. + +Responses contain a `GasWanted` and `GasUsed` field. The former is the maximum +amount of gas the sender of a transaction is willing to use, and the latter is how much it actually +used. Applications should enforce that `GasUsed <= GasWanted` — i.e. transaction execution +or validation should fail before it can use more resources than it requested. + +When `MaxGas > -1`, CometBFT enforces the following rules: + +- `GasWanted <= MaxGas` for every transaction in the mempool +- `(sum of GasWanted in a block) <= MaxGas` when proposing a block + +If `MaxGas == -1`, no rules about gas are enforced. + +In v0.34.x and earlier versions, CometBFT does not enforce anything about Gas in consensus, +only in the mempool. +This means it does not guarantee that committed blocks satisfy these rules. +It is the application's responsibility to return non-zero response codes when gas limits are exceeded +when executing the transactions of a block. +Since the introduction of `PrepareProposal` and `ProcessProposal` in v.0.37.x, it is now possible +for the Application to enforce that all blocks proposed (and voted for) in consensus — and thus all +blocks decided — respect the `MaxGas` limits described above. + +Since the Application should enforce that `GasUsed <= GasWanted` when executing a transaction, and +it can use `PrepareProposal` and `ProcessProposal` to enforce that `(sum of GasWanted in a block) <= MaxGas` +in all proposed or prevoted blocks, +we have: + +- `(sum of GasUsed in a block) <= MaxGas` for every block + +The `GasUsed` field is ignored by CometBFT. + +#### Specifics of `ResponseCheckTx` + +If `Code != 0`, it will be rejected from the mempool and hence +not broadcasted to other peers and not included in a proposal block. + +`Data` contains the result of the `CheckTx` transaction execution, if any. It does not need to be +deterministic since, given a transaction, nodes' Applications +might have a different *CheckTxState* values when they receive it and check their validity +via `CheckTx`. +CometBFT ignores this value in `ResponseCheckTx`. + +From v0.34.x on, there is a `Priority` field in `ResponseCheckTx` that can be +used to explicitly prioritize transactions in the mempool for inclusion in a block +proposal. + +#### Specifics of `ExecTxResult` + +`FinalizeBlock` is the workhorse of the blockchain. CometBFT delivers the decided block, +including the list of all its transactions synchronously to the Application. +The block delivered (and thus the transaction order) is the same at all correct nodes as guaranteed +by the Agreement property of consensus. + +The `Data` field in `ExecTxResult` contains an array of bytes with the transaction result. +It must be deterministic (i.e., the same value must be returned at all nodes), but it can contain arbitrary +data. Likewise, the value of `Code` must be deterministic. +If `Code != 0`, the transaction will be marked invalid, +though it is still included in the block. Invalid transactions are not indexed, as they are +considered analogous to those that failed `CheckTx`. + +Both the `Code` and `Data` are included in a structure that is hashed into the +`LastResultsHash` of the block header in the next height. + +`Events` include any events for the execution, which CometBFT will use to index +the transaction by. This allows transactions to be queried according to what +events took place during their execution. + +### Updating the Validator Set + +The application may set the validator set during +[`InitChain`](/cometbft/v0.39/spec/abci/Methods#initchain), and may update it during +[`FinalizeBlock`](/cometbft/v0.39/spec/abci/Methods#finalizeblock). In both cases, a structure of type +[`ValidatorUpdate`](/cometbft/v0.39/spec/abci/Methods#validatorupdate) is returned. + +The `InitChain` method, used to initialize the Application, can return a list of validators. +If the list is empty, CometBFT will use the validators loaded from the genesis +file. +If the list returned by `InitChain` is not empty, CometBFT will use its contents as the validator set. +This way the application can set the initial validator set for the +blockchain. + +Applications must ensure that a single set of validator updates does not contain duplicates, i.e. +a given public key can only appear once within a given update. If an update includes +duplicates, the block execution will fail irrecoverably. + +Structure `ValidatorUpdate` contains a public key, which is used to identify the validator: +The public key currently supports three types: + +- `ed25519` +- `secp256k1` +- `bls12381` + +Structure `ValidatorUpdate` also contains an `ìnt64` field denoting the validator's new power. +Applications must ensure that +`ValidatorUpdate` structures abide by the following rules: + +- power must be non-negative +- if power is set to 0, the validator must be in the validator set; it will be removed from the set +- if power is greater than 0: + - if the validator is not in the validator set, it will be added to the + set with the given power + - if the validator is in the validator set, its power will be adjusted to the given power +- the total power of the new validator set must not exceed `MaxTotalVotingPower`, where + `MaxTotalVotingPower = MaxInt64 / 8` + +Note the updates returned after processing the block at height `H` will only take effect +at block `H+2` (see Section [Methods](/cometbft/v0.39/spec/abci/Methods)). + +### Consensus Parameters + +`ConsensusParams` are global parameters that apply to all validators in a blockchain. +They enforce certain limits in the blockchain, like the maximum size +of blocks, amount of gas used in a block, and the maximum acceptable age of +evidence. They can be set in +[`InitChain`](/cometbft/v0.39/spec/abci/Methods#initchain), and updated in +[`FinalizeBlock`](/cometbft/v0.39/spec/abci/Methods#finalizeblock). +These parameters are deterministically set and/or updated by the Application, so +all full nodes have the same value at a given height. + +#### List of Parameters + +These are the current consensus parameters (as of v0.38.x): + +1. [ABCIParams.VoteExtensionsEnableHeight](#abciparamsvoteextensionsenableheight) +2. [BlockParams.MaxBytes](#blockparamsmaxbytes) +3. [BlockParams.MaxGas](#blockparamsmaxgas) +4. [EvidenceParams.MaxAgeDuration](#evidenceparamsmaxageduration) +5. [EvidenceParams.MaxAgeNumBlocks](#evidenceparamsmaxagenumblocks) +6. [EvidenceParams.MaxBytes](#evidenceparamsmaxbytes) +7. [ValidatorParams.PubKeyTypes](#validatorparamspubkeytypes) +8. [VersionParams.App](#versionparamsapp) + +#### ABCIParams.VoteExtensionsEnableHeight + +This parameter is either 0 or a positive height at which vote extensions +become mandatory. If the value is zero (which is the default), vote +extensions are not expected. Otherwise, at all heights greater than the +configured height `H` vote extensions must be present (even if empty). +When the configured height `H` is reached, `PrepareProposal` will not +include vote extensions yet, but `ExtendVote` and `VerifyVoteExtension` will +be called. Then, when reaching height `H+1`, `PrepareProposal` will +include the vote extensions from height `H`. For all heights after `H` + +- vote extensions cannot be disabled, +- they are mandatory: all precommit messages sent MUST have an extension + attached. Nevertheless, the application MAY provide 0-length + extensions. + +Must always be set to a future height, 0, or the same height that was previously set. +Once the chain's height reaches the value set, it cannot be changed to a different value. + +##### BlockParams.MaxBytes + +The maximum size of a complete Protobuf encoded block. +This is enforced by the consensus algorithm. + +This implies a maximum transaction size that is `MaxBytes`, less the expected size of +the header, the validator set, and any included evidence in the block. + +The Application should be aware that honest validators *may* produce and +broadcast blocks with up to the configured `MaxBytes` size. +As a result, the consensus +[timeout parameters](/cometbft/v0.39/docs/core/configuration#consensus-timeouts-explained) +adopted by nodes should be configured so as to account for the worst-case +latency for the delivery of a full block with `MaxBytes` size to all validators. + +If the Application wants full control over the size of blocks, +it can do so by enforcing a byte limit set up at the Application level. +This Application-internal limit is used by `PrepareProposal` to bound the total size +of transactions it returns, and by `ProcessProposal` to reject any received block +whose total transaction size is bigger than the enforced limit. +In such case, the Application MAY set `MaxBytes` to -1. + +If the Application sets value -1, consensus will: + +- consider that the actual value to enforce is 100 MB +- will provide *all* transactions in the mempool in calls to `PrepareProposal` + +Must have `MaxBytes == -1` OR `0 < MaxBytes <= 100 MB`. + +> Bear in mind that the default value for the `BlockParams.MaxBytes` consensus +> parameter accepts as valid blocks with size up to 21 MB. +> If the Application's use case does not need blocks of that size, +> or if the impact (specially on bandwidth consumption and block latency) +> of propagating blocks of that size was not evaluated, +> it is strongly recommended to wind down this default value. + +##### BlockParams.MaxGas + +The maximum of the sum of `GasWanted` that will be allowed in a proposed block. +This is *not* enforced by the consensus algorithm. +It is left to the Application to enforce (ie. if transactions are included past the +limit, they should return non-zero codes). It is used by CometBFT to limit the +transactions included in a proposed block. + +Must have `MaxGas >= -1`. +If `MaxGas == -1`, no limit is enforced. + +##### EvidenceParams.MaxAgeDuration + +This is the maximum age of evidence in time units. +This is enforced by the consensus algorithm. + +If a block includes evidence older than this (AND the evidence was created more +than `MaxAgeNumBlocks` ago), the block will be rejected (validators won't vote +for it). + +Must have `MaxAgeDuration > 0`. + +##### EvidenceParams.MaxAgeNumBlocks + +This is the maximum age of evidence in blocks. +This is enforced by the consensus algorithm. + +If a block includes evidence older than this (AND the evidence was created more +than `MaxAgeDuration` ago), the block will be rejected (validators won't vote +for it). + +Must have `MaxAgeNumBlocks > 0`. + +##### EvidenceParams.MaxBytes + +This is the maximum size of total evidence in bytes that can be committed to a +single block. It should fall comfortably under the max block bytes. + +Its value must not exceed the size of +a block minus its overhead ( ~ `BlockParams.MaxBytes`). + +Must have `MaxBytes > 0`. + +##### ValidatorParams.PubKeyTypes + +The parameter restricts the type of keys validators can use. The parameter uses ABCI pubkey naming, not Amino names. + +##### VersionParams.App + +This is the version of the ABCI application. + +#### Updating Consensus Parameters + +The application may set the `ConsensusParams` during +[`InitChain`](/cometbft/v0.39/spec/abci/Methods#initchain), +and update them during +[`FinalizeBlock`](/cometbft/v0.39/spec/abci/Methods#finalizeblock). +If the `ConsensusParams` is empty, it will be ignored. Each field +that is not empty will be applied in full. For instance, if updating the +`Block.MaxBytes`, applications must also set the other `Block` fields (like +`Block.MaxGas`), even if they are unchanged, as they will otherwise cause the +value to be updated to the default. + +##### `InitChain` + +`ResponseInitChain` includes a `ConsensusParams` parameter. +If `ConsensusParams` is `nil`, CometBFT will use the params loaded in the genesis +file. If `ConsensusParams` is not `nil`, CometBFT will use it. +This way the application can determine the initial consensus parameters for the +blockchain. + +##### `FinalizeBlock`, `PrepareProposal`/`ProcessProposal` + +`ResponseFinalizeBlock` accepts a `ConsensusParams` parameter. +If `ConsensusParams` is `nil`, CometBFT will do nothing. +If `ConsensusParams` is not `nil`, CometBFT will use it. +This way the application can update the consensus parameters over time. + +The updates returned in block `H` will take effect right away for block +`H+1`. + +### `Query` + +`Query` is a generic method with lots of flexibility to enable diverse sets +of queries on application state. CometBFT makes use of `Query` to filter new peers +based on ID and IP, and exposes `Query` to the user over RPC. + +Note that calls to `Query` are not replicated across nodes, but rather query the +local node's state - hence they may return stale reads. For reads that require +consensus, use a transaction. + +The most important use of `Query` is to return Merkle proofs of the application state at some height +that can be used for efficient application-specific light-clients. + +Note CometBFT has technically no requirements from the `Query` +message for normal operation - that is, the ABCI app developer need not implement +Query functionality if they do not wish to. + +#### Query Proofs + +The CometBFT block header includes a number of hashes, each providing an +anchor for some type of proof about the blockchain. The `ValidatorsHash` enables +quick verification of the validator set, the `DataHash` gives quick +verification of the transactions included in the block. + +The `AppHash` is unique in that it is application specific, and allows for +application-specific Merkle proofs about the state of the application. +While some applications keep all relevant state in the transactions themselves +(like Bitcoin and its UTXOs), others maintain a separated state that is +computed deterministically *from* transactions, but is not contained directly in +the transactions themselves (like Ethereum contracts and accounts). +For such applications, the `AppHash` provides a much more efficient way to verify light-client proofs. + +ABCI applications can take advantage of more efficient light-client proofs for +their state as follows: + +- return the Merkle root of the deterministic application state in + `ResponseFinalizeBlock.Data`. This Merkle root will be included as the `AppHash` in the next block. +- return efficient Merkle proofs about that application state in `ResponseQuery.Proof` + that can be verified using the `AppHash` of the corresponding block. + +For instance, this allows an application's light-client to verify proofs of +absence in the application state, something which is much less efficient to do using the block hash. + +Some applications (eg. Ethereum, Cosmos-SDK) have multiple "levels" of Merkle trees, +where the leaves of one tree are the root hashes of others. To support this, and +the general variability in Merkle proofs, the `ResponseQuery.Proof` has some minimal structure: + +```protobuf +message ProofOps { + repeated ProofOp ops = 1 +} + +message ProofOp { + string type = 1; + bytes key = 2; + bytes data = 3; +} +``` + +Each `ProofOp` contains a proof for a single key in a single Merkle tree, of the specified `type`. +This allows ABCI to support many different kinds of Merkle trees, encoding +formats, and proofs (eg. of presence and absence) just by varying the `type`. +The `data` contains the actual encoded proof, encoded according to the `type`. +When verifying the full proof, the root hash for one ProofOp is the value being +verified for the next ProofOp in the list. The root hash of the final ProofOp in +the list should match the `AppHash` being verified against. + +#### Peer Filtering + +When CometBFT connects to a peer, it sends two queries to the ABCI application +using the following paths, with no additional data: + +- `/p2p/filter/addr/`, where `` denote the IP address and + the port of the connection +- `p2p/filter/id/`, where `` is the peer node ID (ie. the + pubkey.Address() for the peer's PubKey) + +If either of these queries return a non-zero ABCI code, CometBFT will refuse +to connect to the peer. + +#### Paths + +Queries are directed at paths, and may optionally include additional data. + +The expectation is for there to be some number of high level paths +differentiating concerns, like `/p2p`, `/store`, and `/app`. Currently, +CometBFT only uses `/p2p`, for filtering peers. For more advanced use, see the +implementation of +[Query in the Cosmos-SDK](https://github.com/cosmos/cosmos-sdk/blob/e2037f7696fed4fdd4bc076f9e7053fe8178a881/baseapp/abci.go#L557-L565). + +### Crash Recovery + +CometBFT and the application are expected to crash together and there should not +exist a scenario where the application has persisted state of a height greater than the +latest height persisted by CometBFT. + +In practice, persisting the state of a height consists of three steps, the last of which +is the call to the application's `Commit` method, the only place where the application is expected to +persist/commit its state. +On startup (upon recovery), CometBFT calls the `Info` method on the Info Connection to get the latest +committed state of the app. The app MUST return information consistent with the +last block for which it successfully completed `Commit`. + +The three steps performed before the state of a height is considered persisted are: + +- The block is stored by CometBFT in the blockstore +- CometBFT has stored the state returned by the application through `FinalizeBlockResponse` +- The application has committed its state within `Commit`. + +The following diagram depicts the order in which these events happen, and the corresponding +ABCI functions that are called and executed by CometBFT and the application: + + +``` +APP: Execute block Persist application state + / return ResultFinalizeBlock / + / / +Event: ------------- block_stored ------------ / ------------ state_stored --------------- / ----- app_persisted_state + | / | / | +CometBFT: Decide --- Persist block -- Call FinalizeBlock - Persist results ---------- Call Commit -- + on in the (txResults, validator + Block block store updates...) + +``` + +As these three steps are not atomic, we observe different cases based on which steps have been executed +before the crash occurred +(we assume that at least `block_stored` has been executed, otherwise, there is no state persisted, +and the operations for this height are repeated entirely): + +- `block_stored`: we replay `FinalizeBlock` and the steps afterwards. +- `block_stored` and `state_stored`: As the app did not persist its state within `Commit`, we need to re-execute + `FinalizeBlock` to retrieve the results and compare them to the state stored by CometBFT within `state_stored`. + The expected case is that the states will match, otherwise CometBFT panics. +- `block_stored`, `state_stored`, `app_persisted_state`: we move on to the next height. + +Based on the sequence of these events, CometBFT will panic if any of the steps in the sequence happen out of order, +that is if: + +- The application has persisted a block at a height higher than the blocked saved during `state_stored`. +- The `block_stored` step persisted a block at a height smaller than the `state_stored` +- And the difference between the heights of the blocks persisted by `state_stored` and `block_stored` is more +than 1 (this corresponds to a scenario where we stored two blocks in the block store but never persisted the state of the first +block, which should never happen). + +A special case is when a crash happens before the first block is committed - that is, after calling +`InitChain`. In that case, the application's state should still be at height 0 and thus `InitChain` +will be called again. + + +### State Sync + +A new node joining the network can simply join consensus at the genesis height and replay all +historical blocks until it is caught up. However, for large chains this can take a significant +amount of time, often on the order of days or weeks. + +State sync is an alternative mechanism for bootstrapping a new node, where it fetches a snapshot +of the state machine at a given height and restores it. Depending on the application, this can +be several orders of magnitude faster than replaying blocks. + +Note that state sync does not currently backfill historical blocks, so the node will have a +truncated block history - users are advised to consider the broader network implications of this in +terms of block availability and auditability. This functionality may be added in the future. + +For details on the specific ABCI calls and types, see the +[methods](/cometbft/v0.39/spec/abci/Methods) section. + +#### Taking Snapshots + +Applications that want to support state syncing must take state snapshots at regular intervals. How +this is accomplished is entirely up to the application. A snapshot consists of some metadata and +a set of binary chunks in an arbitrary format: + +- `Height (uint64)`: The height at which the snapshot is taken. It must be taken after the given + height has been committed, and must not contain data from any later heights. + +- `Format (uint32)`: An arbitrary snapshot format identifier. This can be used to version snapshot + formats, e.g. to switch from Protobuf to MessagePack for serialization. The application can use + this when restoring to choose whether to accept or reject a snapshot. + +- `Chunks (uint32)`: The number of chunks in the snapshot. Each chunk contains arbitrary binary + data, and should be less than 16 MB; 10 MB is a good starting point. + +- `Hash ([]byte)`: An arbitrary hash of the snapshot. This is used to check whether a snapshot is + the same across nodes when downloading chunks. + +- `Metadata ([]byte)`: Arbitrary snapshot metadata, e.g. chunk hashes for verification or any other + necessary info. + +For a snapshot to be considered the same across nodes, all of these fields must be identical. When +sent across the network, snapshot metadata messages are limited to 4 MB. + +When a new node is running state sync and discovering snapshots, CometBFT will query an existing +application via the ABCI `ListSnapshots` method to discover available snapshots, and load binary +snapshot chunks via `LoadSnapshotChunk`. The application is free to choose how to implement this +and which formats to use, but must provide the following guarantees: + +- **Consistent:** A snapshot must be taken at a single isolated height, unaffected by + concurrent writes. This can be accomplished by using a data store that supports ACID + transactions with snapshot isolation. + +- **Asynchronous:** Taking a snapshot can be time-consuming, so it must not halt chain progress, + for example by running in a separate thread. + +- **Deterministic:** A snapshot taken at the same height in the same format must be identical + (at the byte level) across nodes, including all metadata. This ensures good availability of + chunks, and that they fit together across nodes. + +A very basic approach might be to use a datastore with MVCC transactions (such as RocksDB), +start a transaction immediately after block commit, and spawn a new thread which is passed the +transaction handle. This thread can then export all data items, serialize them using e.g. +Protobuf, hash the byte stream, split it into chunks, and store the chunks in the file system +along with some metadata - all while the blockchain is applying new blocks in parallel. + +A more advanced approach might include incremental verification of individual chunks against the +chain app hash, parallel or batched exports, compression, and so on. + +Old snapshots should be removed after some time - generally only the last two snapshots are needed +(to prevent the last one from being removed while a node is restoring it). + +#### Bootstrapping a Node + +An empty node can be state synced by setting the configuration option `statesync.enabled = +true`. The node also needs the chain genesis file for basic chain info, and configuration for +light client verification of the restored snapshot: a set of CometBFT RPC servers, and a +trusted header hash and corresponding height from a trusted source, via the `statesync` +configuration section. + +Once started, the node will connect to the P2P network and begin discovering snapshots. These +will be offered to the local application via the `OfferSnapshot` ABCI method. Once a snapshot +is accepted CometBFT will fetch and apply the snapshot chunks. After all chunks have been +successfully applied, CometBFT verifies the app's `AppHash` against the chain using the light +client, then switches the node to normal consensus operation. + +#### Snapshot Discovery + +When the empty node joins the P2P network, it asks all peers to report snapshots via the +`ListSnapshots` ABCI call (limited to 10 per node). After some time, the node picks the most +suitable snapshot (generally prioritized by height, format, and number of peers), and offers it +to the application via `OfferSnapshot`. The application can choose a number of responses, +including accepting or rejecting it, rejecting the offered format, rejecting the peer who sent +it, and so on. CometBFT will keep discovering and offering snapshots until one is accepted or +the application aborts. + +#### Snapshot Restoration + +Once a snapshot has been accepted via `OfferSnapshot`, CometBFT begins downloading chunks from +any peers that have the same snapshot (i.e. that have identical metadata fields). Chunks are +spooled in a temporary directory, and then given to the application in sequential order via +`ApplySnapshotChunk` until all chunks have been accepted. + +The method for restoring snapshot chunks is entirely up to the application. + +During restoration, the application can respond to `ApplySnapshotChunk` with instructions for how +to continue. This will typically be to accept the chunk and await the next one, but it can also +ask for chunks to be refetched (either the current one or any number of previous ones), P2P peers +to be banned, snapshots to be rejected or retried, and a number of other responses - see the ABCI +reference for details. + +If CometBFT fails to fetch a chunk after some time, it will reject the snapshot and try a +different one via `OfferSnapshot` - the application can choose whether it wants to support +restarting restoration, or simply abort with an error. + +#### Snapshot Verification + +Once all chunks have been accepted, CometBFT issues an `Info` ABCI call to retrieve the +`LastBlockAppHash`. This is compared with the trusted app hash from the chain, retrieved and +verified using the light client. CometBFT also checks that `LastBlockHeight` corresponds to the +height of the snapshot. + +This verification ensures that an application is valid before joining the network. However, the +snapshot restoration may take a long time to complete, so applications may want to employ additional +verification during the restore to detect failures early. This might e.g. include incremental +verification of each chunk against the app hash (using bundled Merkle proofs), checksums to +protect against data corruption by the disk or network, and so on. However, it is important to +note that the only trusted information available is the app hash, and all other snapshot metadata +can be spoofed by adversaries. + +Apps may also want to consider state sync denial-of-service vectors, where adversaries provide +invalid or harmful snapshots to prevent nodes from joining the network. The application can +counteract this by asking CometBFT to ban peers. As a last resort, node operators can use +P2P configuration options to whitelist a set of trusted peers that can provide valid snapshots. + +#### Transition to Consensus + +Once the snapshots have all been restored, CometBFT gathers additional information necessary for +bootstrapping the node (e.g. chain ID, consensus parameters, validator sets, and block headers) +from the genesis file and light client RPC servers. It also calls `Info` to verify the following: + +- that the app hash from the snapshot it has delivered to the Application matches the apphash + stored in the next height's block + +- that the version that the Application returns in `ResponseInfo` matches the version in the + current height's block header + +Once the state machine has been restored and CometBFT has gathered this additional +information, it transitions to consensus. As of ABCI 2.0, CometBFT ensures the necessary conditions +to switch are met [RFC-100](https://github.com/cometbft/cometbft/blob/main/docs/references/rfc/rfc-100-abci-vote-extension-propag.md#base-implementation-persist-and-propagate-extended-commit-history). +From the application's point of view, these operations are transparent, unless the application has just upgraded to ABCI 2.0. +In that case, the application needs to be properly configured and aware of certain constraints in terms of when +to provide vote extensions. More details can be found in the section below. + +Once a node switches to consensus, it operates like any other node, apart from having a truncated block history at the height of the restored snapshot. + +## Application configuration required to switch to ABCI 2.0 + +Introducing vote extensions requires changes to the configuration of the application. + +First of all, switching to a version of CometBFT with vote extensions, requires a coordinated upgrade. +For a detailed description on the upgrade path, please refer to the corresponding +[section](https://github.com/cometbft/cometbft/blob/main/docs/references/rfc/rfc-100-abci-vote-extension-propag.md#upgrade-path) in RFC-100. + +There is a newly introduced [**consensus parameter**](/cometbft/v0.39/spec/abci/Requirements-for-the-Application#consensus-parameters): `VoteExtensionsEnableHeight`. +This parameter represents the height at which vote extensions are +required for consensus to proceed, with 0 being the default value (no vote extensions). +A chain can enable vote extensions either: + +- at genesis by setting `VoteExtensionsEnableHeight` to be equal, e.g., to the `InitialHeight` +- or via the application logic by changing the `ConsensusParam` to configure the +`VoteExtensionsEnableHeight`. + +Once the (coordinated) upgrade to ABCI 2.0 has taken place, at height *hu*, +the value of `VoteExtensionsEnableHeight` MAY be set to some height, *he*, +which MUST be higher than the current height of the chain. Thus the earliest value for + *he* is *hu* + 1. + +Once a node reaches the configured height, +for all heights *h ≥ he*, the consensus algorithm will +reject as invalid any precommit messages that do not have signed vote extension data. +If the application requires it, a 0-length vote extension is allowed, but it MUST be signed +and present in the precommit message. +Likewise, for all heights *h < he*, any precommit messages that *do* have vote extensions +will also be rejected as malformed. +Height *he* is somewhat special, as calls to `PrepareProposal` MUST NOT +have vote extension data, but all precommit votes in that height MUST carry a vote extension, +even if the extension is `nil`. +Height *he + 1* is the first height for which `PrepareProposal` MUST have vote +extension data and all precommit votes in that height MUST have a vote extension. + +Corollary, [CometBFT will decide](/cometbft/v0.39/spec/abci/CometBFTs-expected-behavior#handling-upgrades-to-abci-2-0) which data to store, and require for successful operations, based on the current height +of the chain. diff --git a/cometbft/v0.39/spec/blockchain/Blockchain.mdx b/cometbft/v0.39/spec/blockchain/Blockchain.mdx new file mode 100644 index 000000000..9208c54b3 --- /dev/null +++ b/cometbft/v0.39/spec/blockchain/Blockchain.mdx @@ -0,0 +1,12 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/spec/blockchain/Blockchain' +title: Blockchain +order: 1 +--- + +This section describes the core types and functionality of the CometBFT protocol implementation. + +[Core Data Structures](/cometbft/v0.39/spec/core/Data_structures) +[Encoding](/cometbft/v0.39/spec/core/encoding) +[State](/cometbft/v0.39/spec/core/state) diff --git a/cometbft/v0.39/spec/blockchain/blockchain.md b/cometbft/v0.39/spec/blockchain/blockchain.md new file mode 100644 index 000000000..9415f836e --- /dev/null +++ b/cometbft/v0.39/spec/blockchain/blockchain.md @@ -0,0 +1,3 @@ +# Blockchain + +Deprecated see [core/data_structures.mdx](../core/Data_structures.mdx) diff --git a/cometbft/v0.39/spec/blockchain/encoding.md b/cometbft/v0.39/spec/blockchain/encoding.md new file mode 100644 index 000000000..aa2c9ab3f --- /dev/null +++ b/cometbft/v0.39/spec/blockchain/encoding.md @@ -0,0 +1,3 @@ +# Encoding + +Deprecated see [core/data_structures.md](../core/encoding.md) diff --git a/cometbft/v0.39/spec/blockchain/state.md b/cometbft/v0.39/spec/blockchain/state.md new file mode 100644 index 000000000..f4f1d9525 --- /dev/null +++ b/cometbft/v0.39/spec/blockchain/state.md @@ -0,0 +1,3 @@ +# State + +Deprecated see [core/state.md](../core/state.md) diff --git a/cometbft/v0.39/spec/consensus/BFT-Time.mdx b/cometbft/v0.39/spec/consensus/BFT-Time.mdx new file mode 100644 index 000000000..e18d9afac --- /dev/null +++ b/cometbft/v0.39/spec/consensus/BFT-Time.mdx @@ -0,0 +1,58 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/spec/consensus/BFT-Time' +title: BFT Time +order: 2 +--- + +CometBFT provides a deterministic, Byzantine fault-tolerant, source of time. +Time in CometBFT is defined with the Time field of the block header. + +It satisfies the following properties: + +- Time Monotonicity: Time is monotonically increasing, i.e., given +a header H1 for height h1 and a header H2 for height `h2 = h1 + 1`, `H1.Time < H2.Time`. +- Time Validity: Given a set of Commit votes that forms the `block.LastCommit` field, a range of +valid values for the Time field of the block header is defined only by +Precommit messages (from the LastCommit field) sent by correct processes, i.e., +a faulty process cannot arbitrarily increase the Time value. + +In the context of CometBFT, time is of type int64 and denotes UNIX time in milliseconds, i.e., +corresponds to the number of milliseconds since January 1, 1970. +Before defining rules that need to be enforced by Tendermint, the consensus algorithm adopted in CometBFT, +so the properties above holds, we introduce the following definition: + +- median of a Commit is equal to the median of `Vote.Time` fields of the `Vote` messages, +where the value of `Vote.Time` is counted number of times proportional to the process voting power. As +the voting power is not uniform (one process one vote), a vote message is actually an aggregator of the same votes whose +number is equal to the voting power of the process that has casted the corresponding votes message. + +Let's consider the following example: + +- we have four processes p1, p2, p3 and p4, with the following voting power distribution (p1, 23), (p2, 27), (p3, 10) +and (p4, 10). The total voting power is 70 (`N = 3f+1`, where `N` is the total voting power, and `f` is the maximum voting +power of the faulty processes), so we assume that the faulty processes have at most 23 of voting power. +Furthermore, we have the following vote messages in some LastCommit field (we ignore all fields except Time field): + - (p1, 100), (p2, 98), (p3, 1000), (p4, 500). We assume that p3 and p4 are faulty processes. Let's assume that the + `block.LastCommit` message contains votes of processes p2, p3 and p4. Median is then chosen the following way: + the value 98 is counted 27 times, the value 1000 is counted 10 times and the value 500 is counted also 10 times. + So the median value will be the value 98. No matter what set of messages with at least `2f+1` voting power we + choose, the median value will always be between the values sent by correct processes. + +We ensure Time Monotonicity and Time Validity properties by the following rules: + +- let rs denotes `RoundState` (consensus internal state) of some process. Then +`rs.ProposalBlock.Header.Time == median(rs.LastCommit) && +rs.Proposal.Timestamp == rs.ProposalBlock.Header.Time`. + +- Furthermore, when creating the `vote` message, the following rules for determining `vote.Time` field should hold: + + - if `rs.LockedBlock` is defined then + `vote.Time = max(rs.LockedBlock.Timestamp + time.Millisecond, time.Now())`, where `time.Now()` + denotes local Unix time in milliseconds + + - else if `rs.Proposal` is defined then + `vote.Time = max(rs.Proposal.Timestamp + time.Millisecond,, time.Now())`, + + - otherwise, `vote.Time = time.Now())`. In this case vote is for `nil` so it is not taken into account for + the timestamp of the next block. diff --git a/cometbft/v0.39/spec/consensus/Byzantine-Consensus-Algorithm.mdx b/cometbft/v0.39/spec/consensus/Byzantine-Consensus-Algorithm.mdx new file mode 100644 index 000000000..83ed32ee3 --- /dev/null +++ b/cometbft/v0.39/spec/consensus/Byzantine-Consensus-Algorithm.mdx @@ -0,0 +1,352 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/spec/consensus/Byzantine-Consensus-Algorithm' +title: Byzantine Consensus Algorithm +order: 1 +--- + +## Terms + +- The network is composed of optionally connected _nodes_. Nodes + directly connected to a particular node are called _peers_. +- The consensus process in deciding the next block (at some _height_ + `H`) is composed of one or many _rounds_. +- `NewHeight`, `Propose`, `Prevote`, `Precommit`, and `Commit` + represent state machine states of a round. (aka `RoundStep` or + just "step"). +- A node is said to be _at_ a given height, round, and step, or at + `(H,R,S)`, or at `(H,R)` in short to omit the step. +- To _prevote_ or _precommit_ something means to broadcast a prevote + or precommit [vote](https://github.com/cometbft/cometbft/blob/af3bc47df982e271d4d340a3c5e0d773e440466d/types/vote.go#L50) + for something. +- A vote _at_ `(H,R)` is a vote signed with the bytes for `H` and `R` + included in its [sign-bytes](/cometbft/v0.39/spec/core/Data_structures#vote). +- _+2/3_ is short for "more than 2/3" +- _1/3+_ is short for "1/3 or more" +- A set of +2/3 of prevotes for a particular block or `` at + `(H,R)` is called a _proof-of-lock-change_ or _PoLC_ for short. + +## State Machine Overview + +At each height of the blockchain a round-based protocol is run to +determine the next block. Each round is composed of three _steps_ +(`Propose`, `Prevote`, and `Precommit`), along with two special steps +`Commit` and `NewHeight`. + +In the optimal scenario, the order of steps is: + +```md +NewHeight -> (Propose -> Prevote -> Precommit)+ -> Commit -> NewHeight ->... +``` + +The sequence `(Propose -> Prevote -> Precommit)` is called a _round_. +There may be more than one round required to commit a block at a given +height. Examples for why more rounds may be required include: + +- The designated proposer was not online. +- The block proposed by the designated proposer was not valid. +- The block proposed by the designated proposer did not propagate + in time. +- The block proposed was valid, but +2/3 of prevotes for the proposed + block were not received in time for enough validator nodes by the + time they reached the `Precommit` step. Even though +2/3 of prevotes + are necessary to progress to the next step, at least one validator + may have voted `` or maliciously voted for something else. +- The block proposed was valid, and +2/3 of prevotes were received for + enough nodes, but +2/3 of precommits for the proposed block were not + received for enough validator nodes. + +Some of these problems are resolved by moving onto the next round & +proposer. Others are resolved by increasing certain round timeout +parameters over each successive round. + +## State Machine Diagram + +```md + +-------------------------------------+ + v |(Wait til `CommmitTime+timeoutCommit`) + +-----------+ +-----+-----+ + +----------> | Propose +--------------+ | NewHeight | + | +-----------+ | +-----------+ + | | ^ + |(Else, after timeoutPrecommit) v | ++-----+-----+ +-----------+ | +| Precommit | <------------------------+ Prevote | | ++-----+-----+ +-----------+ | + |(When +2/3 Precommits for block found) | + v | ++--------------------------------------------------------------------+ +| Commit | +| | +| * Set CommitTime = now; | +| * Wait for block, then stage/save/commit block; | ++--------------------------------------------------------------------+ +``` + +# Background Gossip + +A node may not have a corresponding validator private key, but it +nevertheless plays an active role in the consensus process by relaying +relevant meta-data, proposals, blocks, and votes to its peers. A node +that has the private keys of an active validator and is engaged in +signing votes is called a _validator-node_. All nodes (not just +validator-nodes) have an associated state (the current height, round, +and step) and work to make progress. + +Between two nodes there exists a `Connection`, and multiplexed on top of +this connection are fairly throttled `Channel`s of information. An +epidemic gossip protocol is implemented among some of these channels to +bring peers up to speed on the most recent state of consensus. For +example, + +- Nodes gossip `PartSet` parts of the current round's proposer's + proposed block. A LibSwift inspired algorithm is used to quickly + broadcast blocks across the gossip network. +- Nodes gossip prevote/precommit votes. A node `NODE_A` that is ahead + of `NODE_B` can send `NODE_B` prevotes or precommits for `NODE_B`'s + current (or future) round to enable it to progress forward. +- Nodes gossip prevotes for the proposed PoLC (proof-of-lock-change) + round if one is proposed. +- Nodes gossip to nodes lagging in blockchain height with block + [commits](https://github.com/cometbft/cometbft/blob/af3bc47df982e271d4d340a3c5e0d773e440466d/types/block.go#L738) + for older blocks. +- Nodes opportunistically gossip `ReceivedVote` messages to hint peers what + votes it already has. +- Nodes broadcast their current state to all neighboring peers. (but + is not gossiped further) + +There's more, but let's not get ahead of ourselves here. + +## Proposals + +A proposal is signed and published by the designated proposer at each +round. The proposer is chosen by a deterministic and non-choking round +robin selection algorithm that selects proposers in proportion to their +voting power (see +[implementation](https://github.com/cometbft/cometbft/blob/af3bc47df982e271d4d340a3c5e0d773e440466d/types/validator_set.go#L51)). + +A proposal at `(H,R)` is composed of a block and an optional latest +`PoLC-Round < R` which is included iff the proposer knows of one. This +hints the network to allow nodes to unlock (when safe) to ensure the +liveness property. + +## State Machine Spec + +### Propose Step (height:H,round:R) + +Upon entering `Propose`: + +- The designated proposer proposes a block at `(H,R)`. + +The `Propose` step ends: + +- After `timeoutProposeR` after entering `Propose`. --> goto + `Prevote(H,R)` +- After receiving proposal block and all prevotes at `PoLC-Round`. --> + goto `Prevote(H,R)` +- After [common exit conditions](#common-exit-conditions) + +### Prevote Step (height:H,round:R) + +Upon entering `Prevote`, each validator broadcasts its prevote vote. + +- First, if the validator is locked on a block since `LastLockRound` + but now has a PoLC for something else at round `PoLC-Round` where + `LastLockRound < PoLC-Round < R`, then it unlocks. +- If the validator is still locked on a block, it prevotes that. +- Else, if the proposed block from `Propose(H,R)` is good, it + prevotes that. +- Else, if the proposal is invalid or wasn't received on time, it + prevotes ``. + +The `Prevote` step ends: + +- After +2/3 prevotes for a particular block or ``. -->; goto + `Precommit(H,R)` +- After `timeoutPrevote` after receiving any +2/3 prevotes. --> goto + `Precommit(H,R)` +- After [common exit conditions](#common-exit-conditions) + +### Precommit Step (height:H,round:R) + +Upon entering `Precommit`, each validator broadcasts its precommit vote. + +- If the validator has a PoLC at `(H,R)` for a particular block `B`, it + (re)locks (or changes lock to) and precommits `B` and sets + `LastLockRound = R`. +- Else, if the validator has a PoLC at `(H,R)` for ``, it unlocks + and precommits ``. +- Else, it keeps the lock unchanged and precommits ``. + +A precommit for `` means "I didn’t see a PoLC for this round, but I +did get +2/3 prevotes and waited a bit". + +The Precommit step ends: + +- After +2/3 precommits for ``. --> goto `Propose(H,R+1)` +- After `timeoutPrecommit` after receiving any +2/3 precommits. --> goto + `Propose(H,R+1)` +- After [common exit conditions](#common-exit-conditions) + +### Common exit conditions + +- After +2/3 precommits for a particular block. --> goto + `Commit(H)` +- After any +2/3 prevotes received at `(H,R+x)`. --> goto + `Prevote(H,R+x)` +- After any +2/3 precommits received at `(H,R+x)`. --> goto + `Precommit(H,R+x)` + +### Commit Step (height:H) + +- Set `CommitTime = now()` +- Wait until block is received. --> goto `NewHeight(H+1)` + +### NewHeight Step (height:H) + +- Move `Precommits` to `LastCommit` and increment height. +- Set `StartTime = CommitTime+timeoutCommit` +- Wait until `StartTime` to receive straggler commits. --> goto + `Propose(H,0)` + +## Proofs + +### Proof of Safety + +Assume that at most -1/3 of the voting power of validators is byzantine. +If a validator commits block `B` at round `R`, it's because it saw +2/3 +of precommits at round `R`. This implies that 1/3+ of honest nodes are +still locked at round `R' > R`. These locked validators will remain +locked until they see a PoLC at `R' > R`, but this won't happen because +1/3+ are locked and honest, so at most -2/3 are available to vote for +anything other than `B`. + +### Proof of Liveness + +If 1/3+ honest validators are locked on two different blocks from +different rounds, a proposers' `PoLC-Round` will eventually cause nodes +locked from the earlier round to unlock. Eventually, the designated +proposer will be one that is aware of a PoLC at the later round. Also, +`timeoutProposalR` increments with round `R`, while the size of a +proposal are capped, so eventually the network is able to "fully gossip" +the whole proposal (e.g. the block & PoLC). + +### Proof of Fork Accountability + +Define the JSet (justification-vote-set) at height `H` of a validator +`V1` to be all the votes signed by the validator at `H` along with +justification PoLC prevotes for each lock change. For example, if `V1` +signed the following precommits: `Precommit(B1 @ round 0)`, +`Precommit( @ round 1)`, `Precommit(B2 @ round 4)` (note that no +precommits were signed for rounds 2 and 3, and that's ok), +`Precommit(B1 @ round 0)` must be justified by a PoLC at round 0, and +`Precommit(B2 @ round 4)` must be justified by a PoLC at round 4; but +the precommit for `` at round 1 is not a lock-change by definition +so the JSet for `V1` need not include any prevotes at round 1, 2, or 3 +(unless `V1` happened to have prevoted for those rounds). + +Further, define the JSet at height `H` of a set of validators `VSet` to +be the union of the JSets for each validator in `VSet`. For a given +commit by honest validators at round `R` for block `B` we can construct +a JSet to justify the commit for `B` at `R`. We say that a JSet +_justifies_ a commit at `(H,R)` if all the committers (validators in the +commit-set) are each justified in the JSet with no duplicitous vote +signatures (by the committers). + +- **Lemma**: When a fork is detected by the existence of two + conflicting [commits](/cometbft/v0.39/spec/core/Data_structures#commit), the + union of the JSets for both commits (if they can be compiled) must + include double-signing by at least 1/3+ of the validator set. + **Proof**: The commit cannot be at the same round, because that + would immediately imply double-signing by 1/3+. Take the union of + the JSets of both commits. If there is no double-signing by at least + 1/3+ of the validator set in the union, then no honest validator + could have precommitted any different block after the first commit. + Yet, +2/3 did. Reductio ad absurdum. + +As a corollary, when there is a fork, an external process can determine +the blame by requiring each validator to justify all of its round votes. +Either we will find 1/3+ who cannot justify at least one of their votes, +and/or, we will find 1/3+ who had double-signed. + +### Alternative algorithm + +Alternatively, we can take the JSet of a commit to be the "full commit". +That is, if light clients and validators do not consider a block to be +committed unless the JSet of the commit is also known, then we get the +desirable property that if there ever is a fork (e.g. there are two +conflicting "full commits"), then 1/3+ of the validators are immediately +punishable for double-signing. + +There are many ways to ensure that the gossip network efficiently share +the JSet of a commit. One solution is to add a new message type that +tells peers that this node has (or does not have) a +2/3 majority for B +(or) at (H,R), and a bitarray of which votes contributed towards that +majority. Peers can react by responding with appropriate votes. + +We will implement such an algorithm for the next iteration of the +consensus protocol. + +Other potential improvements include adding more data in votes such as +the last known PoLC round that caused a lock change, and the last voted +round/step (or, we may require that validators not skip any votes). This +may make JSet verification/gossip logic easier to implement. + +### Censorship Attacks + +Due to the definition of a block +[commit](https://github.com/cometbft/cometbft/blob/v0.38.x/docs/core/validators.md), any 1/3+ coalition of +validators can halt the blockchain by not broadcasting their votes. Such +a coalition can also censor particular transactions by rejecting blocks +that include these transactions, though this would result in a +significant proportion of block proposals to be rejected, which would +slow down the rate of block commits of the blockchain, reducing its +utility and value. The malicious coalition might also broadcast votes in +a trickle so as to grind blockchain block commits to a near halt, or +engage in any combination of these attacks. + +If a global active adversary were also involved, it can partition the +network in such a way that it may appear that the wrong subset of +validators were responsible for the slowdown. This is not just a +limitation of Tendermint, but rather a limitation of all consensus +protocols whose network is potentially controlled by an active +adversary. + +### Overcoming Forks and Censorship Attacks + +For these types of attacks, a subset of the validators through external +means should coordinate to sign a reorg-proposal that chooses a fork +(and any evidence thereof) and the initial subset of validators with +their signatures. Validators who sign such a reorg-proposal forego its +collateral on all other forks. Clients should verify the signatures on +the reorg-proposal, verify any evidence, and make a judgement or prompt +the end-user for a decision. For example, a phone wallet app may prompt +the user with a security warning, while a refrigerator may accept any +reorg-proposal signed by +1/2 of the original validators. + +No non-synchronous Byzantine fault-tolerant algorithm can come to +consensus when 1/3+ of validators are dishonest, yet a fork assumes that +1/3+ of validators have already been dishonest by double-signing or +lock-changing without justification. So, signing the reorg-proposal is a +coordination problem that cannot be solved by any non-synchronous +protocol (i.e. automatically, and without making assumptions about the +reliability of the underlying network). It must be provided by means +external to the weakly-synchronous Tendermint consensus algorithm. For +now, we leave the problem of reorg-proposal coordination to human +coordination via internet media. Validators must take care to ensure +that there are no significant network partitions, to avoid situations +where two conflicting reorg-proposals are signed. + +Assuming that the external coordination medium and protocol is robust, +it follows that forks are less of a concern than [censorship +attacks](#censorship-attacks). + +### Canonical vs subjective commit + +We distinguish between "canonical" and "subjective" commits. A subjective commit is what +each validator sees locally when they decide to commit a block. The canonical commit is +what is included by the proposer of the next block in the `LastCommit` field of +the block. This is what makes it canonical and ensures every validator agrees on the canonical commit, +even if it is different from the +2/3 votes a validator has seen, which caused the validator to +commit the respective block. Each block contains a canonical +2/3 commit for the previous +block. diff --git a/cometbft/v0.39/spec/consensus/Consensus-Paper.mdx b/cometbft/v0.39/spec/consensus/Consensus-Paper.mdx new file mode 100644 index 000000000..a1dd05327 --- /dev/null +++ b/cometbft/v0.39/spec/consensus/Consensus-Paper.mdx @@ -0,0 +1,29 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/spec/consensus/Consensus-Paper' +title: Consensus Paper +order: 1 +--- + +The repository contains the specification (and the proofs) of the Tendermint +consensus protocol, adopted in CometBFT. + +## How to install Latex on MacOS + +MacTex is Latex distribution for MacOS. You can download it [here](http://www.tug.org/mactex/mactex-download.html). + +Popular IDE for Latex-based projects is TexStudio. It can be downloaded +[here](https://www.texstudio.org/). + +## How to build project + +In order to compile the latex files (and write bibliography), execute + +`$ pdflatex paper`
+`$ bibtex paper`
+`$ pdflatex paper`
+`$ pdflatex paper`
+ +The generated file is paper.pdf. You can open it with + +`$ open paper.pdf` diff --git a/cometbft/v0.39/spec/consensus/Creating-Proposal.mdx b/cometbft/v0.39/spec/consensus/Creating-Proposal.mdx new file mode 100644 index 000000000..d17ee5cf0 --- /dev/null +++ b/cometbft/v0.39/spec/consensus/Creating-Proposal.mdx @@ -0,0 +1,63 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/spec/consensus/Creating-Proposal' +title: Creating a proposal +order: 2 +--- + +A block consists of a header, transactions, votes (the commit), +and a list of evidence of malfeasance (eg. signing conflicting votes). + +Outstanding evidence items get priority over outstanding transactions in the mempool. +All in all, the block MUST NOT exceed `ConsensusParams.Block.MaxBytes`, +or 100MB if `ConsensusParams.Block.MaxBytes == -1`. + +## Reaping transactions from the mempool + +When we reap transactions from the mempool, we calculate maximum data +size by subtracting maximum header size (`MaxHeaderBytes`), the maximum +protobuf overhead for a block (`MaxOverheadForBlock`), the size of +the last commit (if present) and evidence (if present). While reaping +we account for protobuf overhead for each transaction. + +```go +func MaxDataBytes(maxBytes, evidenceBytes int64, valsCount int) int64 { + return maxBytes - + MaxOverheadForBlock - + MaxHeaderBytes - + MaxCommitBytes(valsCount) - + evidenceBytes +} +``` + +If `ConsensusParams.Block.MaxBytes == -1`, we reap *all* outstanding transactions from the mempool + +## Preparing the proposal + +Once the transactions have been reaped from the mempool according to the rules described above, +CometBFT calls `PrepareProposal` to the application with the transaction list that has just been reaped. +As part of this call the application can remove, add, or reorder transactions in the transaction list. + +The `RequestPrepareProposal` contains two important fields: + +* `MaxTxBytes`, which contains the value returned by `MaxDataBytes` described above. + The application MUST NOT return a list of transactions whose size exceeds this number. +* `Txs`, which contains the list of reaped transactions. + +For more details on `PrepareProposal`, please see the +[relevant part of the spec](/cometbft/v0.39/spec/abci/Methods#prepareproposal) + +## Validating transactions in the mempool + +Before we accept a transaction in the mempool, we check if its size is no more +than {MaxDataSize}. {MaxDataSize} is calculated using the same formula as +above, except we assume there is no evidence. + +```go +func MaxDataBytesNoEvidence(maxBytes int64, valsCount int) int64 { + return maxBytes - + MaxOverheadForBlock - + MaxHeaderBytes - + MaxCommitBytes(valsCount) +} +``` diff --git a/cometbft/v0.39/spec/consensus/Evidence.mdx b/cometbft/v0.39/spec/consensus/Evidence.mdx new file mode 100644 index 000000000..03693c2d6 --- /dev/null +++ b/cometbft/v0.39/spec/consensus/Evidence.mdx @@ -0,0 +1,204 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/spec/consensus/Evidence' +title: Evidence +order: 4 +--- + +Evidence is an important component of CometBFT's security model. Whilst the core +consensus protocol provides correctness gaurantees for state machine replication +that can tolerate less than 1/3 failures, the evidence system looks to detect and +gossip byzantine faults whose combined power is greater than or equal to 1/3. It is worth noting that +the evidence system is designed purely to detect possible attacks, gossip them, +commit them on chain and inform the application running on top of CometBFT. +Evidence in itself does not punish "bad actors", this is left to the discretion +of the application. A common form of punishment is slashing where the validators +that were caught violating the protocol have all or a portion of their voting +power removed. Evidence, given the assumption that 1/3+ of the network is still +byzantine, is susceptible to censorship and should therefore be considered added +security on a "best effort" basis. + +This document walks through the various forms of evidence, how they are detected, +gossiped, verified and committed. + +> NOTE: Evidence here is internal to CometBFT and should not be confused with +> application evidence + +## Detection + +### Equivocation + +Equivocation is the most fundamental of byzantine faults. Simply put, to prevent +replication of state across all nodes, a validator tries to convince some subset +of nodes to commit one block whilst convincing another subset to commit a +different block. This is achieved by double voting (hence +`DuplicateVoteEvidence`). A successful duplicate vote attack requires greater +than 1/3 voting power and a (temporary) network partition between the aforementioned +subsets. This is because in consensus, votes are gossiped around. When a node +observes two conflicting votes from the same peer, it will use the two votes of +evidence and begin gossiping this evidence to other nodes. [Verification](#duplicatevoteevidence) is addressed further down. + +```go +type DuplicateVoteEvidence struct { + VoteA Vote + VoteB Vote + + // and abci specific fields +} +``` + +### Light Client Attacks + +Light clients also comply with the 1/3+ security model, however, by using a +different, more lightweight verification method they are subject to a +different kind of 1/3+ attack whereby the byzantine validators could sign an +alternative light block that the light client will think is valid. Detection, +explained in greater detail +[here](https://github.com/cometbft/cometbft/blob/main/spec/light-client/detection/detection_003_reviewed.md), involves comparison +with multiple other nodes in the hope that at least one is "honest". An "honest" +node will return a challenging light block for the light client to validate. If +this challenging light block also meets the +[validation criteria](https://github.com/cometbft/cometbft/blob/main/spec/light-client/verification/verification_001_published.md) +then the light client sends the "forged" light block to the node. +[Verification](#lightclientattackevidence) is addressed further down. + +```go +type LightClientAttackEvidence struct { + ConflictingBlock LightBlock + CommonHeight int64 + + // and abci specific fields +} +``` + +## Verification + +If a node receives evidence, it will first try to verify it, then persist it. +Evidence of byzantine behavior should only be committed once (uniqueness) and +should be committed within a certain period from the point that it occurred +(timely). Timelines is defined by the `EvidenceParams`: `MaxAgeNumBlocks` and +`MaxAgeDuration`. In Proof of Stake chains where validators are bonded, evidence +age should be less than the unbonding period so validators still can be +punished. Given these two propoerties the following initial checks are made. + +1. Has the evidence expired? This is done by taking the height of the `Vote` + within `DuplicateVoteEvidence` or `CommonHeight` within + `LightClientAttakEvidence`. The evidence height is then used to retrieve the + header and thus the time of the block that corresponds to the evidence. If + `CurrentHeight - MaxAgeNumBlocks > EvidenceHeight` && `CurrentTime - + MaxAgeDuration > EvidenceTime`, the evidence is considered expired and + ignored. + +2. Has the evidence already been committed? The evidence pool tracks the hash of + all committed evidence and uses this to determine uniqueness. If a new + evidence has the same hash as a committed one, the new evidence will be + ignored. + +### DuplicateVoteEvidence + +Valid `DuplicateVoteEvidence` must adhere to the following rules: + +- Validator Address, Height, Round and Type must be the same for both votes + +- BlockID must be different for both votes (BlockID can be for a nil block) + +- Validator must have been in the validator set at that height + +- Vote signature must be correctly signed. This also uses `ChainID` so we know + that the fault occurred on this chain + +### LightClientAttackEvidence + +Valid Light Client Attack Evidence must adhere to the following rules: + +- If the header of the light block is invalid, thus indicating a lunatic attack, + the node must check that they can use `verifySkipping` from their header at + the common height to the conflicting header + +- If the header is valid, then the validator sets are the same and this is + either a form of equivocation or amnesia. We therefore check that 2/3 of the + validator set also signed the conflicting header. + +- The nodes own header at the same height as the conflicting header must have a + different hash to the conflicting header. + +- If the nodes latest header is less in height to the conflicting header, then + the node must check that the conflicting block has a time that is less than + this latest header (This is a forward lunatic attack). + +## Gossiping + +If a node verifies evidence it then broadcasts it to all peers, continously sending +the same evidence once every 10 seconds until the evidence is seen on chain or +expires. + +## Commiting on Chain + +Evidence takes strict priority over regular transactions, thus a block is filled +with evidence first and transactions take up the remainder of the space. To +mitigate the threat of an already punished node from spamming the network with +more evidence, the size of the evidence in a block can be capped by +`EvidenceParams.MaxBytes`. Nodes receiving blocks with evidence will validate +the evidence before sending `Prevote` and `Precommit` votes. The evidence pool +will usually cache verifications so that this process is much quicker. + +## Sending Evidence to the Application + +After evidence is committed, the block is then processed by the block executor +which delivers the evidence to the application via `EndBlock`. Evidence is +stripped of the actual proof, split up per faulty validator and only the +validator, height, time and evidence type is sent. + +```proto +enum EvidenceType { + UNKNOWN = 0; + DUPLICATE_VOTE = 1; + LIGHT_CLIENT_ATTACK = 2; +} + +message Evidence { + EvidenceType type = 1; + // The offending validator + Validator validator = 2 [(gogoproto.nullable) = false]; + // The height when the offense occurred + int64 height = 3; + // The corresponding time where the offense occurred + google.protobuf.Timestamp time = 4 [ + (gogoproto.nullable) = false, (gogoproto.stdtime) = true]; + // Total voting power of the validator set in case the ABCI application does + // not store historical validators. + // https://github.com/tendermint/tendermint/issues/4581 + int64 total_voting_power = 5; +} +``` + +`DuplicateVoteEvidence` and `LightClientAttackEvidence` are self-contained in +the sense that the evidence can be used to derive the `abci.Evidence` that is +sent to the application. Because of this, extra fields are necessary: + +```go +type DuplicateVoteEvidence struct { + VoteA *Vote + VoteB *Vote + + // abci specific information + TotalVotingPower int64 + ValidatorPower int64 + Timestamp time.Time +} + +type LightClientAttackEvidence struct { + ConflictingBlock *LightBlock + CommonHeight int64 + + // abci specific information + ByzantineValidators []*Validator + TotalVotingPower int64 + Timestamp time.Time +} +``` + +These ABCI specific fields don't affect validity of the evidence itself but must +be consistent amongst nodes and agreed upon on chain. If evidence with the +incorrect abci information is sent, a node will create new evidence from it and +replace the ABCI fields with the correct information. diff --git a/cometbft/v0.39/spec/consensus/Light-Client.mdx b/cometbft/v0.39/spec/consensus/Light-Client.mdx new file mode 100644 index 000000000..841a961c2 --- /dev/null +++ b/cometbft/v0.39/spec/consensus/Light-Client.mdx @@ -0,0 +1,12 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/spec/consensus/Light-Client' +order: 1 +parent: + title: Light Client + order: false +--- + +# Light Client Protocol + +Deprecated, please see [light-client](/cometbft/v0.39/spec/light-client/Light-Client-Specification). diff --git a/cometbft/v0.39/spec/consensus/Overview.mdx b/cometbft/v0.39/spec/consensus/Overview.mdx new file mode 100644 index 000000000..7f93eb0a2 --- /dev/null +++ b/cometbft/v0.39/spec/consensus/Overview.mdx @@ -0,0 +1,31 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/spec/consensus/Overview' +order: 1 +parent: + title: Consensus + order: 4 +--- + +# Consensus + +Specification of the consensus protocol implemented in CometBFT. + +## Contents + +- [Consensus Paper](/cometbft/v0.39/spec/consensus/Consensus-Paper) - Latex paper on + [arxiv](https://arxiv.org/abs/1807.04938) describing the + Tendermint consensus algorithm, adopted in CometBFT, with proofs of safety and termination. +- [BFT Time](/cometbft/v0.39/spec/consensus/BFT-Time) - How the timestamp in a CometBFT + block header is computed in a Byzantine Fault Tolerant manner +- [Creating Proposal](/cometbft/v0.39/spec/consensus/Creating-Proposal) - How a proposer + creates a block proposal for consensus +- [Light Client Protocol](/cometbft/v0.39/spec/consensus/Light-Client) - A protocol for light weight consensus + verification and syncing to the latest state +- [Validator Signing](/cometbft/v0.39/spec/consensus/Validator-Signing) - Rules for cryptographic signatures + produced by validators. +- [Write Ahead Log](/cometbft/v0.39/spec/consensus/WAL) - Write ahead log used by the + consensus state machine to recover from crashes. + +There is also a [stale markdown description](/cometbft/v0.39/spec/consensus/Byzantine-Consensus-Algorithm) of the consensus state machine +(TODO update this). diff --git a/cometbft/v0.39/spec/consensus/Proposer-Selection.mdx b/cometbft/v0.39/spec/consensus/Proposer-Selection.mdx new file mode 100644 index 000000000..c745e2618 --- /dev/null +++ b/cometbft/v0.39/spec/consensus/Proposer-Selection.mdx @@ -0,0 +1,325 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/spec/consensus/Proposer-Selection' +order: 3 +--- + +# Proposer Selection Procedure + +This document specifies the Proposer Selection Procedure that is used in Tendermint, the consensus algorithm adopted in CometBFT, to choose a round proposer. +As Tendermint is “leader-based consensus protocol”, the proposer selection is critical for its correct functioning. + +At a given block height, the proposer selection algorithm runs with the same validator set at each round . +Between heights, an updated validator set may be specified by the application as part of the ABCIResponses' EndBlock. + +## Requirements for Proposer Selection + +This sections covers the requirements with Rx being mandatory and Ox optional requirements. +The following requirements must be met by the Proposer Selection procedure: + +### R1: Determinism + +Given a validator set `V`, and two honest validators `p` and `q`, for each height `h` and each round `r` the following must hold: + + `proposer_p(h,r) = proposer_q(h,r)` + +where `proposer_p(h,r)` is the proposer returned by the Proposer Selection Procedure at process `p`, at height `h` and round `r`. + +### R2: Fairness + +Given a validator set with total voting power P and a sequence S of elections. In any sub-sequence of S with length C*P, a validator v must be elected as proposer P/VP(v) times, i.e. with frequency: + + f(v) ~ VP(v) / P + +where C is a tolerance factor for validator set changes with following values: + +- C == 1 if there are no validator set changes +- C ~ k when there are validator changes + +*[this needs more work]* + +## Basic Algorithm + +At its core, the proposer selection procedure uses a weighted round-robin algorithm. + +A model that gives a good intuition on how/ why the selection algorithm works and it is fair is that of a priority queue. The validators move ahead in this queue according to their voting power (the higher the voting power the faster a validator moves towards the head of the queue). When the algorithm runs the following happens: + +- all validators move "ahead" according to their powers: for each validator, increase the priority by the voting power +- first in the queue becomes the proposer: select the validator with highest priority +- move the proposer back in the queue: decrease the proposer's priority by the total voting power + +Notation: + +- vset - the validator set +- n - the number of validators +- VP(i) - voting power of validator i +- A(i) - accumulated priority for validator i +- P - total voting power of set +- avg - average of all validator priorities +- prop - proposer + +Simple view at the Selection Algorithm: + +```md + def ProposerSelection (vset): + + // compute priorities and elect proposer + for each validator i in vset: + A(i) += VP(i) + prop = max(A) + A(prop) -= P +``` + +## Stable Set + +Consider the validator set: + +Validator | p1 | p2 +----------|----|--- +VP | 1 | 3 + +Assuming no validator changes, the following table shows the proposer priority computation over a few runs. Four runs of the selection procedure are shown, starting with the 5th the same values are computed. +Each row shows the priority queue and the process place in it. The proposer is the closest to the head, the rightmost validator. As priorities are updated, the validators move right in the queue. The proposer moves left as its priority is reduced after election. + +| Priority Run | -2 | -1 | 0 | 1 | 2 | 3 | 4 | 5 | Alg step | +|----------------|----|----|-------|----|-------|----|----|----|------------------| +| | | | p1,p2 | | | | | | Initialized to 0 | +| run 1 | | | | p1 | | p2 | | | A(i)+=VP(i) | +| | | p2 | | p1 | | | | | A(p2)-= P | +| run 2 | | | | | p1,p2 | | | | A(i)+=VP(i) | +| | p1 | | | | p2 | | | | A(p1)-= P | +| run 3 | | p1 | | | | | | p2 | A(i)+=VP(i) | +| | | p1 | | p2 | | | | | A(p2)-= P | +| run 4 | | | p1 | | | | p2 | | A(i)+=VP(i) | +| | | | p1,p2 | | | | | | A(p2)-= P | + +It can be shown that: + +- At the end of each run k+1 the sum of the priorities is the same as at end of run k. If a new set's priorities are initialized to 0 then the sum of priorities will be 0 at each run while there are no changes. +- The max distance between priorites is (n-1) *P.*[formal proof not finished]* + +## Validator Set Changes + +Between proposer selection runs the validator set may change. Some changes have implications on the proposer election. + +### Voting Power Change + +Consider again the earlier example and assume that the voting power of p1 is changed to 4: + +Validator | p1 | p2 +----------|----|--- +VP | 4 | 3 + +Let's also assume that before this change the proposer priorites were as shown in first row (last run). As it can be seen, the selection could run again, without changes, as before. + +| Priority Run | -2 | -1 | 0 | 1 | 2 | Comment | +|----------------|----|----|---|----|----|-------------------| +| last run | | p2 | | p1 | | __update VP(p1)__ | +| next run | | | | | p2 | A(i)+=VP(i) | +| | p1 | | | | p2 | A(p1)-= P | + +However, when a validator changes power from a high to a low value, some other validator remain far back in the queue for a long time. This scenario is considered again in the Proposer Priority Range section. + +As before: + +- At the end of each run k+1 the sum of the priorities is the same as at run k. +- The max distance between priorites is (n-1) * P. + +### Validator Removal + +Consider a new example with set: + +Validator | p1 | p2 | p3 +----------|----|----|--- +VP | 1 | 2 | 3 + +Let's assume that after the last run the proposer priorities were as shown in first row with their sum being 0. After p2 is removed, at the end of next proposer selection run (penultimate row) the sum of priorities is -2 (minus the priority of the removed process). + +The procedure could continue without modifications. However, after a sufficiently large number of modifications in validator set, the priority values would migrate towards maximum or minimum allowed values causing truncations due to overflow detection. +For this reason, the selection procedure adds another __new step__ that centers the current priority values such that the priority sum remains close to 0. + +| Priority Run | -3 | -2 | -1 | 0 | 1 | 2 | 3 | Comment | +|----------------|----|----|----|---|----|----|----|-----------------------| +| last run | p3 | | | | p1 | p2 | | __remove p2__ | +| nextrun | | | | | | | | | +| __new step__ | | p3 | | | | p1 | | A(i) -= avg, avg = -1 | +| | | | | | p3 | | p1 | A(i)+=VP(i) | +| | | | p1 | | p3 | | | A(p1)-= P | + +The modified selection algorithm is: + +```md + def ProposerSelection (vset): + + // center priorities around zero + avg = sum(A(i) for i in vset)/len(vset) + for each validator i in vset: + A(i) -= avg + + // compute priorities and elect proposer + for each validator i in vset: + A(i) += VP(i) + prop = max(A) + A(prop) -= P +``` + +Observations: + +- The sum of priorities is now close to 0. Due to integer division the sum is an integer in (-n, n), where n is the number of validators. + +### New Validator + +When a new validator is added, same problem as the one described for removal appears, the sum of priorities in the new set is not zero. This is fixed with the centering step introduced above. + +One other issue that needs to be addressed is the following. A validator V that has just been elected is moved to the end of the queue. If the validator set is large and/ or other validators have significantly higher power, V will have to wait many runs to be elected. If V removes and re-adds itself to the set, it would make a significant (albeit unfair) "jump" ahead in the queue. + +In order to prevent this, when a new validator is added, its initial priority is set to: + +```md + A(V) = -1.125 * P +``` + +where P is the total voting power of the set including V. + +Current implementation uses the penalty factor of 1.125 because it provides a small punishment that is efficient to calculate. See [here](https://github.com/tendermint/tendermint/pull/2785#discussion_r235038971) for more details. + +If we consider the validator set where p3 has just been added: + +Validator | p1 | p2 | p3 +----------|----|----|--- +VP | 1 | 3 | 8 + +then p3 will start with proposer priority: + +```md + A(p3) = -1.125 * (1 + 3 + 8) ~ -13 +``` + +Note that since current computation uses integer division there is penalty loss when sum of the voting power is less than 8. + +In the next run, p3 will still be ahead in the queue, elected as proposer and moved back in the queue. + +| Priority Run | -13 | -9 | -5 | -2 | -1 | 0 | 1 | 2 | 5 | 6 | 7 | Alg step | +|----------------|-----|----|----|----|----|---|---|----|----|----|----|-----------------------| +| last run | | | | p2 | | | | p1 | | | | __add p3__ | +| | p3 | | | p2 | | | | p1 | | | | A(p3) = -13 | +| next run | | p3 | | | | | | p2 | | p1 | | A(i) -= avg, avg = -4 | +| | | | | | p3 | | | | p2 | | p1 | A(i)+=VP(i) | +| | | | p1 | | p3 | | | | p2 | | | A(p1)-=P | + +## Proposer Priority Range + +With the introduction of centering, some interesting cases occur. Low power validators that bind early in a set that includes high power validator(s) benefit from subsequent additions to the set. This is because these early validators run through more right shift operations during centering, operations that increase their priority. + +As an example, consider the set where p2 is added after p1, with priority -1.125 * 80k = -90k. After the selection procedure runs once: + +Validator | p1 | p2 | Comment +----------|------|------|------------------ +VP | 80k | 10 | +A | 0 | -90k | __added p2__ +A | 45k | -45k | __run selection__ + +Then execute the following steps: + +1. Add a new validator p3: + + Validator | p1 | p2 | p3 + ----------|-----|----|--- + VP | 80k | 10 | 10 + +2. Run selection once. The notation '..p'/'p..' means very small deviations compared to column priority. + + | Priority Run | -90k.. | -60k | -45k | -15k | 0 | 45k | 75k | 155k | Comment | + |---------------|--------|------|------|------|---|-----|-----|------|--------------| + | last run | p3 | | p2 | | | p1 | | | __added p3__ | + | next run + | *right_shift*| | p3 | | p2 | | | p1 | | A(i) -= avg,avg=-30k + | | | ..p3| | ..p2| | | | p1 | A(i)+=VP(i) + | | | ..p3| | ..p2| | | p1.. | | A(p1)-=P, P=80k+20 + +3. Remove p1 and run selection once: + + Validator | p3 | p2 | Comment + ----------|--------|-------|------------------ + VP | 10 | 10 | + A | -60k | -15k | + A | -22.5k | 22.5k | __run selection__ + +At this point, while the total voting power is 20, the distance between priorities is 45k. It will take 4500 runs for p3 to catch up with p2. + +In order to prevent these types of scenarios, the selection algorithm performs scaling of priorities such that the difference between min and max values is smaller than two times the total voting power. + +The modified selection algorithm is: + +```md + def ProposerSelection (vset): + + // scale the priority values + diff = max(A)-min(A) + threshold = 2 * P + if diff > threshold: + scale = diff/threshold + for each validator i in vset: + A(i) = A(i)/scale + + // center priorities around zero + avg = sum(A(i) for i in vset)/len(vset) + for each validator i in vset: + A(i) -= avg + + // compute priorities and elect proposer + for each validator i in vset: + A(i) += VP(i) + prop = max(A) + A(prop) -= P +``` + +Observations: + +- With this modification, the maximum distance between priorites becomes 2 * P. + +Note also that even during steady state the priority range may increase beyond 2 * P. The scaling introduced here helps to keep the range bounded. + +## Wrinkles + +### Validator Power Overflow Conditions + +The validator voting power is a positive number stored as an int64. When a validator is added the `1.125 * P` computation must not overflow. As a consequence the code handling validator updates (add and update) checks for overflow conditions making sure the total voting power is never larger than the largest int64 `MAX`, with the property that `1.125 * MAX` is still in the bounds of int64. Fatal error is return when overflow condition is detected. + +### Proposer Priority Overflow/ Underflow Handling + +The proposer priority is stored as an int64. The selection algorithm performs additions and subtractions to these values and in the case of overflows and underflows it limits the values to: + +```go + MaxInt64 = 1 << 63 - 1 + MinInt64 = -1 << 63 +``` + +## Requirement Fulfillment Claims + +__[R1]__ + +The proposer algorithm is deterministic giving consistent results across executions with same transactions and validator set modifications. +[WIP - needs more detail] + +__[R2]__ + +Given a set of processes with the total voting power P, during a sequence of elections of length P, the number of times any process is selected as proposer is equal to its voting power. The sequence of the P proposers then repeats. If we consider the validator set: + +Validator | p1 | p2 +----------|----|--- +VP | 1 | 3 + +With no other changes to the validator set, the current implementation of proposer selection generates the sequence: +`p2, p1, p2, p2, p2, p1, p2, p2,...` or [`p2, p1, p2, p2`]* +A sequence that starts with any circular permutation of the [`p2, p1, p2, p2`] sub-sequence would also provide the same degree of fairness. In fact these circular permutations show in the sliding window (over the generated sequence) of size equal to the length of the sub-sequence. + +Assigning priorities to each validator based on the voting power and updating them at each run ensures the fairness of the proposer selection. In addition, every time a validator is elected as proposer its priority is decreased with the total voting power. + +Intuitively, a process v jumps ahead in the queue at most (max(A) - min(A))/VP(v) times until it reaches the head and is elected. The frequency is then: + +```md + f(v) ~ VP(v)/(max(A)-min(A)) = 1/k * VP(v)/P +``` + +For current implementation, this means v should be proposer at least VP(v) times out of k * P runs, with scaling factor k=2. diff --git a/cometbft/v0.39/spec/consensus/Validator-Signing.mdx b/cometbft/v0.39/spec/consensus/Validator-Signing.mdx new file mode 100644 index 000000000..6ba839507 --- /dev/null +++ b/cometbft/v0.39/spec/consensus/Validator-Signing.mdx @@ -0,0 +1,234 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/spec/consensus/Validator-Signing' +title: Validator Signing +order: 5 +--- + +Here we specify the rules for validating a proposal and vote before signing. +First we include some general notes on validating data structures common to both types. +We then provide specific validation rules for each. Finally, we include validation rules to prevent double-sigining. + +## SignedMsgType + +The `SignedMsgType` is a single byte that refers to the type of the message +being signed. It is defined in Go as follows: + +```go +// SignedMsgType is a type of signed message in the consensus. +type SignedMsgType byte + +const ( + // Votes + PrevoteType SignedMsgType = 0x01 + PrecommitType SignedMsgType = 0x02 + + // Proposals + ProposalType SignedMsgType = 0x20 +) +``` + +All signed messages must correspond to one of these types. + +## Timestamp + +Timestamp validation is subtle and there are currently no bounds placed on the +timestamp included in a proposal or vote. It is expected that validators will honestly +report their local clock time. The median of all timestamps +included in a commit is used as the timestamp for the next block height. + +Timestamps are expected to be strictly monotonic for a given validator, though +this is not currently enforced. + +## ChainID + +ChainID is an unstructured string with a max length of 50-bytes. +In the future, the ChainID may become structured, and may take on longer lengths. +For now, it is recommended that signers be configured for a particular ChainID, +and to only sign votes and proposals corresponding to that ChainID. + +## BlockID + +BlockID is the structure used to represent the block: + +```go +type BlockID struct { + Hash []byte + PartsHeader PartSetHeader +} + +type PartSetHeader struct { + Hash []byte + Total int +} +``` + +To be included in a valid vote or proposal, BlockID must either represent a `nil` block, or a complete one. +We introduce two methods, `BlockID.IsZero()` and `BlockID.IsComplete()` for these cases, respectively. + +`BlockID.IsZero()` returns true for BlockID `b` if each of the following +are true: + +```go +b.Hash == nil +b.PartsHeader.Total == 0 +b.PartsHeader.Hash == nil +``` + +`BlockID.IsComplete()` returns true for BlockID `b` if each of the following +are true: + +```go +len(b.Hash) == 32 +b.PartsHeader.Total > 0 +len(b.PartsHeader.Hash) == 32 +``` + +## Proposals + +The structure of a proposal for signing looks like: + +```go +type CanonicalProposal struct { + Type SignedMsgType // type alias for byte + Height int64 `binary:"fixed64"` + Round int64 `binary:"fixed64"` + POLRound int64 `binary:"fixed64"` + BlockID BlockID + Timestamp time.Time + ChainID string +} +``` + +A proposal is valid if each of the following lines evaluates to true for proposal `p`: + +```go +p.Type == 0x20 +p.Height > 0 +p.Round >= 0 +p.POLRound >= -1 +p.BlockID.IsComplete() +``` + +In other words, a proposal is valid for signing if it contains the type of a Proposal +(0x20), has a positive, non-zero height, a +non-negative round, a POLRound not less than -1, and a complete BlockID. + +## Votes + +The structure of a vote for signing looks like: + +```go +type CanonicalVote struct { + Type SignedMsgType // type alias for byte + Height int64 `binary:"fixed64"` + Round int64 `binary:"fixed64"` + BlockID BlockID + Timestamp time.Time + ChainID string +} +``` + +A vote is valid if each of the following lines evaluates to true for vote `v`: + +```go +v.Type == 0x1 || v.Type == 0x2 +v.Height > 0 +v.Round >= 0 +v.BlockID.IsZero() || v.BlockID.IsComplete() +``` + +In other words, a vote is valid for signing if it contains the type of a Prevote +or Precommit (0x1 or 0x2, respectively), has a positive, non-zero height, a +non-negative round, and an empty or valid BlockID. + +## Invalid Votes and Proposals + +Votes and proposals which do not satisfy the above rules are considered invalid. +Peers gossipping invalid votes and proposals may be disconnected from other peers on the network. +Note, however, that there is not currently any explicit mechanism to punish validators signing votes or proposals that fail +these basic validation rules. + +## Double Signing + +Signers must be careful not to sign conflicting messages, also known as "double signing" or "equivocating". +CometBFT has mechanisms to publish evidence of validators that signed conflicting votes, so they can be punished +by the application. Note CometBFT does not currently handle evidence of conflciting proposals, though it may in the future. + +### State + +To prevent such double signing, signers must track the height, round, and type of the last message signed. +Assume the signer keeps the following state, `s`: + +```go +type LastSigned struct { + Height int64 + Round int64 + Type SignedMsgType // byte +} +``` + +After signing a vote or proposal `m`, the signer sets: + +```go +s.Height = m.Height +s.Round = m.Round +s.Type = m.Type +``` + +### Proposals + +A signer should only sign a proposal `p` if any of the following lines are true: + +```go +p.Height > s.Height +p.Height == s.Height && p.Round > s.Round +``` + +In other words, a proposal should only be signed if it's at a higher height, or a higher round for the same height. +Once a proposal or vote has been signed for a given height and round, a proposal should never be signed for the same height and round. + +### Votes + +A signer should only sign a vote `v` if any of the following lines are true: + +```go +v.Height > s.Height +v.Height == s.Height && v.Round > s.Round +v.Height == s.Height && v.Round == s.Round && v.Step == 0x1 && s.Step == 0x20 +v.Height == s.Height && v.Round == s.Round && v.Step == 0x2 && s.Step != 0x2 +``` + +In other words, a vote should only be signed if it's: + +- at a higher height +- at a higher round for the same height +- a prevote for the same height and round where we haven't signed a prevote or precommit (but have signed a proposal) +- a precommit for the same height and round where we haven't signed a precommit (but have signed a proposal and/or a prevote) + +This means that once a validator signs a prevote for a given height and round, the only other message it can sign for that height and round is a precommit. +And once a validator signs a precommit for a given height and round, it must not sign any other message for that same height and round. + +Note this includes votes for `nil`, ie. where `BlockID.IsZero()` is true. If a +signer has already signed a vote where `BlockID.IsZero()` is true, it cannot +sign another vote with the same type for the same height and round where +`BlockID.IsComplete()` is true. Thus only a single vote of a particular type +(ie. 0x01 or 0x02) can be signed for the same height and round. + +### Other Rules + +According to the rules of Tendermint consensus algorithm, adopted in CometBFT, once a validator precommits for +a block, they become "locked" on that block, which means they can't prevote for +another block unless they see sufficient justification (ie. a polka from a +higher round). For more details, see the [consensus +spec](https://arxiv.org/abs/1807.04938). + +Violating this rule is known as "amnesia". In contrast to equivocation, +which is easy to detect, amnesia is difficult to detect without access to votes +from all the validators, as this is what constitutes the justification for +"unlocking". Hence, amnesia is not punished within the protocol, and cannot +easily be prevented by a signer. If enough validators simultaneously commit an +amnesia attack, they may cause a fork of the blockchain, at which point an +off-chain protocol must be engaged to collect votes from all the validators and +determine who misbehaved. For more details, see [fork +detection](https://github.com/tendermint/tendermint/pull/3978). diff --git a/cometbft/v0.39/spec/consensus/WAL.mdx b/cometbft/v0.39/spec/consensus/WAL.mdx new file mode 100644 index 000000000..33558ce2d --- /dev/null +++ b/cometbft/v0.39/spec/consensus/WAL.mdx @@ -0,0 +1,37 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/spec/consensus/WAL' +title: WAL +order: 6 +--- + +Consensus module writes every message to the WAL (write-ahead log). + +It also issues fsync syscall through +[File#Sync](https://golang.org/pkg/os/#File.Sync) for messages signed by this +node (to prevent double signing). + +Under the hood, it uses +[autofile.Group](https://github.com/cometbft/cometbft/blob/af3bc47df982e271d4d340a3c5e0d773e440466d/libs/autofile/group.go#L54), +which rotates files when those get too big (> 10MB). + +The total maximum size is 1GB. We only need the latest block and the block before it, +but if the former is dragging on across many rounds, we want all those rounds. + +## Replay + +Consensus module will replay all the messages of the last height written to WAL +before a crash (if such occurs). + +The private validator may try to sign messages during replay because it runs +somewhat autonomously and does not know about replay process. + +For example, if we got all the way to precommit in the WAL and then crash, +after we replay the proposal message, the private validator will try to sign a +prevote. But it will fail. That's ok because we’ll see the prevote later in the +WAL. Then it will go to precommit, and that time it will work because the +private validator contains the `LastSignBytes` and then we’ll replay the +precommit from the WAL. + +Make sure to read about [WAL corruption](https://github.com/cometbft/cometbft/blob/v0.38.x/docs/core/running-in-production.md#wal-corruption) +and recovery strategies. diff --git a/cometbft/v0.39/spec/consensus/consensus-paper/IEEEtran.bst b/cometbft/v0.39/spec/consensus/consensus-paper/IEEEtran.bst new file mode 100644 index 000000000..53fbc030a --- /dev/null +++ b/cometbft/v0.39/spec/consensus/consensus-paper/IEEEtran.bst @@ -0,0 +1,2417 @@ +%% +%% IEEEtran.bst +%% BibTeX Bibliography Style file for IEEE Journals and Conferences (unsorted) +%% Version 1.12 (2007/01/11) +%% +%% Copyright (c) 2003-2007 Michael Shell +%% +%% Original starting code base and algorithms obtained from the output of +%% Patrick W. Daly's makebst package as well as from prior versions of +%% IEEE BibTeX styles: +%% +%% 1. Howard Trickey and Oren Patashnik's ieeetr.bst (1985/1988) +%% 2. Silvano Balemi and Richard H. Roy's IEEEbib.bst (1993) +%% +%% Support sites: +%% http://www.michaelshell.org/tex/ieeetran/ +%% http://www.ctan.org/tex-archive/macros/latex/contrib/IEEEtran/ +%% and/or +%% http://www.ieee.org/ +%% +%% For use with BibTeX version 0.99a or later +%% +%% This is a numerical citation style. +%% +%%************************************************************************* +%% Legal Notice: +%% This code is offered as-is without any warranty either expressed or +%% implied; without even the implied warranty of MERCHANTABILITY or +%% FITNESS FOR A PARTICULAR PURPOSE! +%% User assumes all risk. +%% In no event shall IEEE or any contributor to this code be liable for +%% any damages or losses, including, but not limited to, incidental, +%% consequential, or any other damages, resulting from the use or misuse +%% of any information contained here. +%% +%% All comments are the opinions of their respective authors and are not +%% necessarily endorsed by the IEEE. +%% +%% This work is distributed under the LaTeX Project Public License (LPPL) +%% ( http://www.latex-project.org/ ) version 1.3, and may be freely used, +%% distributed and modified. A copy of the LPPL, version 1.3, is included +%% in the base LaTeX documentation of all distributions of LaTeX released +%% 2003/12/01 or later. +%% Retain all contribution notices and credits. +%% ** Modified files should be clearly indicated as such, including ** +%% ** renaming them and changing author support contact information. ** +%% +%% File list of work: IEEEabrv.bib, IEEEfull.bib, IEEEexample.bib, +%% IEEEtran.bst, IEEEtranS.bst, IEEEtranSA.bst, +%% IEEEtranN.bst, IEEEtranSN.bst, IEEEtran_bst_HOWTO.pdf +%%************************************************************************* +% +% +% Changelog: +% +% 1.00 (2002/08/13) Initial release +% +% 1.10 (2002/09/27) +% 1. Corrected minor bug for improperly formed warning message when a +% book was not given a title. Thanks to Ming Kin Lai for reporting this. +% 2. Added support for CTLname_format_string and CTLname_latex_cmd fields +% in the BST control entry type. +% +% 1.11 (2003/04/02) +% 1. Fixed bug with URLs containing underscores when using url.sty. Thanks +% to Ming Kin Lai for reporting this. +% +% 1.12 (2007/01/11) +% 1. Fixed bug with unwanted comma before "et al." when an entry contained +% more than two author names. Thanks to Pallav Gupta for reporting this. +% 2. Fixed bug with anomalous closing quote in tech reports that have a +% type, but without a number or address. Thanks to Mehrdad Mirreza for +% reporting this. +% 3. Use braces in \providecommand in begin.bib to better support +% latex2html. TeX style length assignments OK with recent versions +% of latex2html - 1.71 (2002/2/1) or later is strongly recommended. +% Use of the language field still causes trouble with latex2html. +% Thanks to Federico Beffa for reporting this. +% 4. Added IEEEtran.bst ID and version comment string to .bbl output. +% 5. Provide a \BIBdecl hook that allows the user to execute commands +% just prior to the first entry. +% 6. Use default urlstyle (is using url.sty) of "same" rather than rm to +% better work with a wider variety of bibliography styles. +% 7. Changed month abbreviations from Sept., July and June to Sep., Jul., +% and Jun., respectively, as IEEE now does. Thanks to Moritz Borgmann +% for reporting this. +% 8. Control entry types should not be considered when calculating longest +% label width. +% 9. Added alias www for electronic/online. +% 10. Added CTLname_url_prefix control entry type. + + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +%% DEFAULTS FOR THE CONTROLS OF THE BST STYLE %% +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +% These are the defaults for the user adjustable controls. The values used +% here can be overridden by the user via IEEEtranBSTCTL entry type. + +% NOTE: The recommended LaTeX command to invoke a control entry type is: +% +%\makeatletter +%\def\bstctlcite{\@ifnextchar[{\@bstctlcite}{\@bstctlcite[@auxout]}} +%\def\@bstctlcite[#1]#2{\@bsphack +% \@for\@citeb:=#2\do{% +% \edef\@citeb{\expandafter\@firstofone\@citeb}% +% \if@filesw\immediate\write\csname #1\endcsname{\string\citation{\@citeb}}\fi}% +% \@esphack} +%\makeatother +% +% It is called at the start of the document, before the first \cite, like: +% \bstctlcite{IEEEexample:BSTcontrol} +% +% IEEEtran.cls V1.6 and later does provide this command. + + + +% #0 turns off the display of the number for articles. +% #1 enables +FUNCTION {default.is.use.number.for.article} { #1 } + + +% #0 turns off the display of the paper and type fields in @inproceedings. +% #1 enables +FUNCTION {default.is.use.paper} { #1 } + + +% #0 turns off the forced use of "et al." +% #1 enables +FUNCTION {default.is.forced.et.al} { #0 } + +% The maximum number of names that can be present beyond which an "et al." +% usage is forced. Be sure that num.names.shown.with.forced.et.al (below) +% is not greater than this value! +% Note: There are many instances of references in IEEE journals which have +% a very large number of authors as well as instances in which "et al." is +% used profusely. +FUNCTION {default.max.num.names.before.forced.et.al} { #10 } + +% The number of names that will be shown with a forced "et al.". +% Must be less than or equal to max.num.names.before.forced.et.al +FUNCTION {default.num.names.shown.with.forced.et.al} { #1 } + + +% #0 turns off the alternate interword spacing for entries with URLs. +% #1 enables +FUNCTION {default.is.use.alt.interword.spacing} { #1 } + +% If alternate interword spacing for entries with URLs is enabled, this is +% the interword spacing stretch factor that will be used. For example, the +% default "4" here means that the interword spacing in entries with URLs can +% stretch to four times normal. Does not have to be an integer. Note that +% the value specified here can be overridden by the user in their LaTeX +% code via a command such as: +% "\providecommand\BIBentryALTinterwordstretchfactor{1.5}" in addition to +% that via the IEEEtranBSTCTL entry type. +FUNCTION {default.ALTinterwordstretchfactor} { "4" } + + +% #0 turns off the "dashification" of repeated (i.e., identical to those +% of the previous entry) names. IEEE normally does this. +% #1 enables +FUNCTION {default.is.dash.repeated.names} { #1 } + + +% The default name format control string. +FUNCTION {default.name.format.string}{ "{f.~}{vv~}{ll}{, jj}" } + + +% The default LaTeX font command for the names. +FUNCTION {default.name.latex.cmd}{ "" } + + +% The default URL prefix. +FUNCTION {default.name.url.prefix}{ "[Online]. Available:" } + + +% Other controls that cannot be accessed via IEEEtranBSTCTL entry type. + +% #0 turns off the terminal startup banner/completed message so as to +% operate more quietly. +% #1 enables +FUNCTION {is.print.banners.to.terminal} { #1 } + + + + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +%% FILE VERSION AND BANNER %% +%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +FUNCTION{bst.file.version} { "1.12" } +FUNCTION{bst.file.date} { "2007/01/11" } +FUNCTION{bst.file.website} { "http://www.michaelshell.org/tex/ieeetran/bibtex/" } + +FUNCTION {banner.message} +{ is.print.banners.to.terminal + { "-- IEEEtran.bst version" " " * bst.file.version * + " (" * bst.file.date * ") " * "by Michael Shell." * + top$ + "-- " bst.file.website * + top$ + "-- See the " quote$ * "IEEEtran_bst_HOWTO.pdf" * quote$ * " manual for usage information." * + top$ + } + { skip$ } + if$ +} + +FUNCTION {completed.message} +{ is.print.banners.to.terminal + { "" + top$ + "Done." + top$ + } + { skip$ } + if$ +} + + + + +%%%%%%%%%%%%%%%%%%%%%% +%% STRING CONSTANTS %% +%%%%%%%%%%%%%%%%%%%%%% + +FUNCTION {bbl.and}{ "and" } +FUNCTION {bbl.etal}{ "et~al." } +FUNCTION {bbl.editors}{ "eds." } +FUNCTION {bbl.editor}{ "ed." } +FUNCTION {bbl.edition}{ "ed." } +FUNCTION {bbl.volume}{ "vol." } +FUNCTION {bbl.of}{ "of" } +FUNCTION {bbl.number}{ "no." } +FUNCTION {bbl.in}{ "in" } +FUNCTION {bbl.pages}{ "pp." } +FUNCTION {bbl.page}{ "p." } +FUNCTION {bbl.chapter}{ "ch." } +FUNCTION {bbl.paper}{ "paper" } +FUNCTION {bbl.part}{ "pt." } +FUNCTION {bbl.patent}{ "Patent" } +FUNCTION {bbl.patentUS}{ "U.S." } +FUNCTION {bbl.revision}{ "Rev." } +FUNCTION {bbl.series}{ "ser." } +FUNCTION {bbl.standard}{ "Std." } +FUNCTION {bbl.techrep}{ "Tech. Rep." } +FUNCTION {bbl.mthesis}{ "Master's thesis" } +FUNCTION {bbl.phdthesis}{ "Ph.D. dissertation" } +FUNCTION {bbl.st}{ "st" } +FUNCTION {bbl.nd}{ "nd" } +FUNCTION {bbl.rd}{ "rd" } +FUNCTION {bbl.th}{ "th" } + + +% This is the LaTeX spacer that is used when a larger than normal space +% is called for (such as just before the address:publisher). +FUNCTION {large.space} { "\hskip 1em plus 0.5em minus 0.4em\relax " } + +% The LaTeX code for dashes that are used to represent repeated names. +% Note: Some older IEEE journals used something like +% "\rule{0.275in}{0.5pt}\," which is fairly thick and runs right along +% the baseline. However, IEEE now uses a thinner, above baseline, +% six dash long sequence. +FUNCTION {repeated.name.dashes} { "------" } + + + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +%% PREDEFINED STRING MACROS %% +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +MACRO {jan} {"Jan."} +MACRO {feb} {"Feb."} +MACRO {mar} {"Mar."} +MACRO {apr} {"Apr."} +MACRO {may} {"May"} +MACRO {jun} {"Jun."} +MACRO {jul} {"Jul."} +MACRO {aug} {"Aug."} +MACRO {sep} {"Sep."} +MACRO {oct} {"Oct."} +MACRO {nov} {"Nov."} +MACRO {dec} {"Dec."} + + + +%%%%%%%%%%%%%%%%%% +%% ENTRY FIELDS %% +%%%%%%%%%%%%%%%%%% + +ENTRY + { address + assignee + author + booktitle + chapter + day + dayfiled + edition + editor + howpublished + institution + intype + journal + key + language + month + monthfiled + nationality + note + number + organization + pages + paper + publisher + school + series + revision + title + type + url + volume + year + yearfiled + CTLuse_article_number + CTLuse_paper + CTLuse_forced_etal + CTLmax_names_forced_etal + CTLnames_show_etal + CTLuse_alt_spacing + CTLalt_stretch_factor + CTLdash_repeated_names + CTLname_format_string + CTLname_latex_cmd + CTLname_url_prefix + } + {} + { label } + + + + +%%%%%%%%%%%%%%%%%%%%%%% +%% INTEGER VARIABLES %% +%%%%%%%%%%%%%%%%%%%%%%% + +INTEGERS { prev.status.punct this.status.punct punct.std + punct.no punct.comma punct.period + prev.status.space this.status.space space.std + space.no space.normal space.large + prev.status.quote this.status.quote quote.std + quote.no quote.close + prev.status.nline this.status.nline nline.std + nline.no nline.newblock + status.cap cap.std + cap.no cap.yes} + +INTEGERS { longest.label.width multiresult nameptr namesleft number.label numnames } + +INTEGERS { is.use.number.for.article + is.use.paper + is.forced.et.al + max.num.names.before.forced.et.al + num.names.shown.with.forced.et.al + is.use.alt.interword.spacing + is.dash.repeated.names} + + +%%%%%%%%%%%%%%%%%%%%%% +%% STRING VARIABLES %% +%%%%%%%%%%%%%%%%%%%%%% + +STRINGS { bibinfo + longest.label + oldname + s + t + ALTinterwordstretchfactor + name.format.string + name.latex.cmd + name.url.prefix} + + + + +%%%%%%%%%%%%%%%%%%%%%%%%% +%% LOW LEVEL FUNCTIONS %% +%%%%%%%%%%%%%%%%%%%%%%%%% + +FUNCTION {initialize.controls} +{ default.is.use.number.for.article 'is.use.number.for.article := + default.is.use.paper 'is.use.paper := + default.is.forced.et.al 'is.forced.et.al := + default.max.num.names.before.forced.et.al 'max.num.names.before.forced.et.al := + default.num.names.shown.with.forced.et.al 'num.names.shown.with.forced.et.al := + default.is.use.alt.interword.spacing 'is.use.alt.interword.spacing := + default.is.dash.repeated.names 'is.dash.repeated.names := + default.ALTinterwordstretchfactor 'ALTinterwordstretchfactor := + default.name.format.string 'name.format.string := + default.name.latex.cmd 'name.latex.cmd := + default.name.url.prefix 'name.url.prefix := +} + + +% This IEEEtran.bst features a very powerful and flexible mechanism for +% controlling the capitalization, punctuation, spacing, quotation, and +% newlines of the formatted entry fields. (Note: IEEEtran.bst does not need +% or use the newline/newblock feature, but it has been implemented for +% possible future use.) The output states of IEEEtran.bst consist of +% multiple independent attributes and, as such, can be thought of as being +% vectors, rather than the simple scalar values ("before.all", +% "mid.sentence", etc.) used in most other .bst files. +% +% The more flexible and complex design used here was motivated in part by +% IEEE's rather unusual bibliography style. For example, IEEE ends the +% previous field item with a period and large space prior to the publisher +% address; the @electronic entry types use periods as inter-item punctuation +% rather than the commas used by the other entry types; and URLs are never +% followed by periods even though they are the last item in the entry. +% Although it is possible to accommodate these features with the conventional +% output state system, the seemingly endless exceptions make for convoluted, +% unreliable and difficult to maintain code. +% +% IEEEtran.bst's output state system can be easily understood via a simple +% illustration of two most recently formatted entry fields (on the stack): +% +% CURRENT_ITEM +% "PREVIOUS_ITEM +% +% which, in this example, is to eventually appear in the bibliography as: +% +% "PREVIOUS_ITEM," CURRENT_ITEM +% +% It is the job of the output routine to take the previous item off of the +% stack (while leaving the current item at the top of the stack), apply its +% trailing punctuation (including closing quote marks) and spacing, and then +% to write the result to BibTeX's output buffer: +% +% "PREVIOUS_ITEM," +% +% Punctuation (and spacing) between items is often determined by both of the +% items rather than just the first one. The presence of quotation marks +% further complicates the situation because, in standard English, trailing +% punctuation marks are supposed to be contained within the quotes. +% +% IEEEtran.bst maintains two output state (aka "status") vectors which +% correspond to the previous and current (aka "this") items. Each vector +% consists of several independent attributes which track punctuation, +% spacing, quotation, and newlines. Capitalization status is handled by a +% separate scalar because the format routines, not the output routine, +% handle capitalization and, therefore, there is no need to maintain the +% capitalization attribute for both the "previous" and "this" items. +% +% When a format routine adds a new item, it copies the current output status +% vector to the previous output status vector and (usually) resets the +% current (this) output status vector to a "standard status" vector. Using a +% "standard status" vector in this way allows us to redefine what we mean by +% "standard status" at the start of each entry handler and reuse the same +% format routines under the various inter-item separation schemes. For +% example, the standard status vector for the @book entry type may use +% commas for item separators, while the @electronic type may use periods, +% yet both entry handlers exploit many of the exact same format routines. +% +% Because format routines have write access to the output status vector of +% the previous item, they can override the punctuation choices of the +% previous format routine! Therefore, it becomes trivial to implement rules +% such as "Always use a period and a large space before the publisher." By +% pushing the generation of the closing quote mark to the output routine, we +% avoid all the problems caused by having to close a quote before having all +% the information required to determine what the punctuation should be. +% +% The IEEEtran.bst output state system can easily be expanded if needed. +% For instance, it is easy to add a "space.tie" attribute value if the +% bibliography rules mandate that two items have to be joined with an +% unbreakable space. + +FUNCTION {initialize.status.constants} +{ #0 'punct.no := + #1 'punct.comma := + #2 'punct.period := + #0 'space.no := + #1 'space.normal := + #2 'space.large := + #0 'quote.no := + #1 'quote.close := + #0 'cap.no := + #1 'cap.yes := + #0 'nline.no := + #1 'nline.newblock := +} + +FUNCTION {std.status.using.comma} +{ punct.comma 'punct.std := + space.normal 'space.std := + quote.no 'quote.std := + nline.no 'nline.std := + cap.no 'cap.std := +} + +FUNCTION {std.status.using.period} +{ punct.period 'punct.std := + space.normal 'space.std := + quote.no 'quote.std := + nline.no 'nline.std := + cap.yes 'cap.std := +} + +FUNCTION {initialize.prev.this.status} +{ punct.no 'prev.status.punct := + space.no 'prev.status.space := + quote.no 'prev.status.quote := + nline.no 'prev.status.nline := + punct.no 'this.status.punct := + space.no 'this.status.space := + quote.no 'this.status.quote := + nline.no 'this.status.nline := + cap.yes 'status.cap := +} + +FUNCTION {this.status.std} +{ punct.std 'this.status.punct := + space.std 'this.status.space := + quote.std 'this.status.quote := + nline.std 'this.status.nline := +} + +FUNCTION {cap.status.std}{ cap.std 'status.cap := } + +FUNCTION {this.to.prev.status} +{ this.status.punct 'prev.status.punct := + this.status.space 'prev.status.space := + this.status.quote 'prev.status.quote := + this.status.nline 'prev.status.nline := +} + + +FUNCTION {not} +{ { #0 } + { #1 } + if$ +} + +FUNCTION {and} +{ { skip$ } + { pop$ #0 } + if$ +} + +FUNCTION {or} +{ { pop$ #1 } + { skip$ } + if$ +} + + +% convert the strings "yes" or "no" to #1 or #0 respectively +FUNCTION {yes.no.to.int} +{ "l" change.case$ duplicate$ + "yes" = + { pop$ #1 } + { duplicate$ "no" = + { pop$ #0 } + { "unknown boolean " quote$ * swap$ * quote$ * + " in " * cite$ * warning$ + #0 + } + if$ + } + if$ +} + + +% pushes true if the single char string on the stack is in the +% range of "0" to "9" +FUNCTION {is.num} +{ chr.to.int$ + duplicate$ "0" chr.to.int$ < not + swap$ "9" chr.to.int$ > not and +} + +% multiplies the integer on the stack by a factor of 10 +FUNCTION {bump.int.mag} +{ #0 'multiresult := + { duplicate$ #0 > } + { #1 - + multiresult #10 + + 'multiresult := + } + while$ +pop$ +multiresult +} + +% converts a single character string on the stack to an integer +FUNCTION {char.to.integer} +{ duplicate$ + is.num + { chr.to.int$ "0" chr.to.int$ - } + {"noninteger character " quote$ * swap$ * quote$ * + " in integer field of " * cite$ * warning$ + #0 + } + if$ +} + +% converts a string on the stack to an integer +FUNCTION {string.to.integer} +{ duplicate$ text.length$ 'namesleft := + #1 'nameptr := + #0 'numnames := + { nameptr namesleft > not } + { duplicate$ nameptr #1 substring$ + char.to.integer numnames bump.int.mag + + 'numnames := + nameptr #1 + + 'nameptr := + } + while$ +pop$ +numnames +} + + + + +% The output routines write out the *next* to the top (previous) item on the +% stack, adding punctuation and such as needed. Since IEEEtran.bst maintains +% the output status for the top two items on the stack, these output +% routines have to consider the previous output status (which corresponds to +% the item that is being output). Full independent control of punctuation, +% closing quote marks, spacing, and newblock is provided. +% +% "output.nonnull" does not check for the presence of a previous empty +% item. +% +% "output" does check for the presence of a previous empty item and will +% remove an empty item rather than outputing it. +% +% "output.warn" is like "output", but will issue a warning if it detects +% an empty item. + +FUNCTION {output.nonnull} +{ swap$ + prev.status.punct punct.comma = + { "," * } + { skip$ } + if$ + prev.status.punct punct.period = + { add.period$ } + { skip$ } + if$ + prev.status.quote quote.close = + { "''" * } + { skip$ } + if$ + prev.status.space space.normal = + { " " * } + { skip$ } + if$ + prev.status.space space.large = + { large.space * } + { skip$ } + if$ + write$ + prev.status.nline nline.newblock = + { newline$ "\newblock " write$ } + { skip$ } + if$ +} + +FUNCTION {output} +{ duplicate$ empty$ + 'pop$ + 'output.nonnull + if$ +} + +FUNCTION {output.warn} +{ 't := + duplicate$ empty$ + { pop$ "empty " t * " in " * cite$ * warning$ } + 'output.nonnull + if$ +} + +% "fin.entry" is the output routine that handles the last item of the entry +% (which will be on the top of the stack when "fin.entry" is called). + +FUNCTION {fin.entry} +{ this.status.punct punct.no = + { skip$ } + { add.period$ } + if$ + this.status.quote quote.close = + { "''" * } + { skip$ } + if$ +write$ +newline$ +} + + +FUNCTION {is.last.char.not.punct} +{ duplicate$ + "}" * add.period$ + #-1 #1 substring$ "." = +} + +FUNCTION {is.multiple.pages} +{ 't := + #0 'multiresult := + { multiresult not + t empty$ not + and + } + { t #1 #1 substring$ + duplicate$ "-" = + swap$ duplicate$ "," = + swap$ "+" = + or or + { #1 'multiresult := } + { t #2 global.max$ substring$ 't := } + if$ + } + while$ + multiresult +} + +FUNCTION {capitalize}{ "u" change.case$ "t" change.case$ } + +FUNCTION {emphasize} +{ duplicate$ empty$ + { pop$ "" } + { "\emph{" swap$ * "}" * } + if$ +} + +FUNCTION {do.name.latex.cmd} +{ name.latex.cmd + empty$ + { skip$ } + { name.latex.cmd "{" * swap$ * "}" * } + if$ +} + +% IEEEtran.bst uses its own \BIBforeignlanguage command which directly +% invokes the TeX hyphenation patterns without the need of the Babel +% package. Babel does a lot more than switch hyphenation patterns and +% its loading can cause unintended effects in many class files (such as +% IEEEtran.cls). +FUNCTION {select.language} +{ duplicate$ empty$ 'pop$ + { language empty$ 'skip$ + { "\BIBforeignlanguage{" language * "}{" * swap$ * "}" * } + if$ + } + if$ +} + +FUNCTION {tie.or.space.prefix} +{ duplicate$ text.length$ #3 < + { "~" } + { " " } + if$ + swap$ +} + +FUNCTION {get.bbl.editor} +{ editor num.names$ #1 > 'bbl.editors 'bbl.editor if$ } + +FUNCTION {space.word}{ " " swap$ * " " * } + + +% Field Conditioners, Converters, Checkers and External Interfaces + +FUNCTION {empty.field.to.null.string} +{ duplicate$ empty$ + { pop$ "" } + { skip$ } + if$ +} + +FUNCTION {either.or.check} +{ empty$ + { pop$ } + { "can't use both " swap$ * " fields in " * cite$ * warning$ } + if$ +} + +FUNCTION {empty.entry.warn} +{ author empty$ title empty$ howpublished empty$ + month empty$ year empty$ note empty$ url empty$ + and and and and and and + { "all relevant fields are empty in " cite$ * warning$ } + 'skip$ + if$ +} + + +% The bibinfo system provides a way for the electronic parsing/acquisition +% of a bibliography's contents as is done by ReVTeX. For example, a field +% could be entered into the bibliography as: +% \bibinfo{volume}{2} +% Only the "2" would show up in the document, but the LaTeX \bibinfo command +% could do additional things with the information. IEEEtran.bst does provide +% a \bibinfo command via "\providecommand{\bibinfo}[2]{#2}". However, it is +% currently not used as the bogus bibinfo functions defined here output the +% entry values directly without the \bibinfo wrapper. The bibinfo functions +% themselves (and the calls to them) are retained for possible future use. +% +% bibinfo.check avoids acting on missing fields while bibinfo.warn will +% issue a warning message if a missing field is detected. Prior to calling +% the bibinfo functions, the user should push the field value and then its +% name string, in that order. + +FUNCTION {bibinfo.check} +{ swap$ duplicate$ missing$ + { pop$ pop$ "" } + { duplicate$ empty$ + { swap$ pop$ } + { swap$ pop$ } + if$ + } + if$ +} + +FUNCTION {bibinfo.warn} +{ swap$ duplicate$ missing$ + { swap$ "missing " swap$ * " in " * cite$ * warning$ pop$ "" } + { duplicate$ empty$ + { swap$ "empty " swap$ * " in " * cite$ * warning$ } + { swap$ pop$ } + if$ + } + if$ +} + + +% IEEE separates large numbers with more than 4 digits into groups of +% three. IEEE uses a small space to separate these number groups. +% Typical applications include patent and page numbers. + +% number of consecutive digits required to trigger the group separation. +FUNCTION {large.number.trigger}{ #5 } + +% For numbers longer than the trigger, this is the blocksize of the groups. +% The blocksize must be less than the trigger threshold, and 2 * blocksize +% must be greater than the trigger threshold (can't do more than one +% separation on the initial trigger). +FUNCTION {large.number.blocksize}{ #3 } + +% What is actually inserted between the number groups. +FUNCTION {large.number.separator}{ "\," } + +% So as to save on integer variables by reusing existing ones, numnames +% holds the current number of consecutive digits read and nameptr holds +% the number that will trigger an inserted space. +FUNCTION {large.number.separate} +{ 't := + "" + #0 'numnames := + large.number.trigger 'nameptr := + { t empty$ not } + { t #-1 #1 substring$ is.num + { numnames #1 + 'numnames := } + { #0 'numnames := + large.number.trigger 'nameptr := + } + if$ + t #-1 #1 substring$ swap$ * + t #-2 global.max$ substring$ 't := + numnames nameptr = + { duplicate$ #1 nameptr large.number.blocksize - substring$ swap$ + nameptr large.number.blocksize - #1 + global.max$ substring$ + large.number.separator swap$ * * + nameptr large.number.blocksize - 'numnames := + large.number.blocksize #1 + 'nameptr := + } + { skip$ } + if$ + } + while$ +} + +% Converts all single dashes "-" to double dashes "--". +FUNCTION {n.dashify} +{ large.number.separate + 't := + "" + { t empty$ not } + { t #1 #1 substring$ "-" = + { t #1 #2 substring$ "--" = not + { "--" * + t #2 global.max$ substring$ 't := + } + { { t #1 #1 substring$ "-" = } + { "-" * + t #2 global.max$ substring$ 't := + } + while$ + } + if$ + } + { t #1 #1 substring$ * + t #2 global.max$ substring$ 't := + } + if$ + } + while$ +} + + +% This function detects entries with names that are identical to that of +% the previous entry and replaces the repeated names with dashes (if the +% "is.dash.repeated.names" user control is nonzero). +FUNCTION {name.or.dash} +{ 's := + oldname empty$ + { s 'oldname := s } + { s oldname = + { is.dash.repeated.names + { repeated.name.dashes } + { s 'oldname := s } + if$ + } + { s 'oldname := s } + if$ + } + if$ +} + +% Converts the number string on the top of the stack to +% "numerical ordinal form" (e.g., "7" to "7th"). There is +% no artificial limit to the upper bound of the numbers as the +% least significant digit always determines the ordinal form. +FUNCTION {num.to.ordinal} +{ duplicate$ #-1 #1 substring$ "1" = + { bbl.st * } + { duplicate$ #-1 #1 substring$ "2" = + { bbl.nd * } + { duplicate$ #-1 #1 substring$ "3" = + { bbl.rd * } + { bbl.th * } + if$ + } + if$ + } + if$ +} + +% If the string on the top of the stack begins with a number, +% (e.g., 11th) then replace the string with the leading number +% it contains. Otherwise retain the string as-is. s holds the +% extracted number, t holds the part of the string that remains +% to be scanned. +FUNCTION {extract.num} +{ duplicate$ 't := + "" 's := + { t empty$ not } + { t #1 #1 substring$ + t #2 global.max$ substring$ 't := + duplicate$ is.num + { s swap$ * 's := } + { pop$ "" 't := } + if$ + } + while$ + s empty$ + 'skip$ + { pop$ s } + if$ +} + +% Converts the word number string on the top of the stack to +% Arabic string form. Will be successful up to "tenth". +FUNCTION {word.to.num} +{ duplicate$ "l" change.case$ 's := + s "first" = + { pop$ "1" } + { skip$ } + if$ + s "second" = + { pop$ "2" } + { skip$ } + if$ + s "third" = + { pop$ "3" } + { skip$ } + if$ + s "fourth" = + { pop$ "4" } + { skip$ } + if$ + s "fifth" = + { pop$ "5" } + { skip$ } + if$ + s "sixth" = + { pop$ "6" } + { skip$ } + if$ + s "seventh" = + { pop$ "7" } + { skip$ } + if$ + s "eighth" = + { pop$ "8" } + { skip$ } + if$ + s "ninth" = + { pop$ "9" } + { skip$ } + if$ + s "tenth" = + { pop$ "10" } + { skip$ } + if$ +} + + +% Converts the string on the top of the stack to numerical +% ordinal (e.g., "11th") form. +FUNCTION {convert.edition} +{ duplicate$ empty$ 'skip$ + { duplicate$ #1 #1 substring$ is.num + { extract.num + num.to.ordinal + } + { word.to.num + duplicate$ #1 #1 substring$ is.num + { num.to.ordinal } + { "edition ordinal word " quote$ * edition * quote$ * + " may be too high (or improper) for conversion" * " in " * cite$ * warning$ + } + if$ + } + if$ + } + if$ +} + + + + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +%% LATEX BIBLIOGRAPHY CODE %% +%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +FUNCTION {start.entry} +{ newline$ + "\bibitem{" write$ + cite$ write$ + "}" write$ + newline$ + "" + initialize.prev.this.status +} + +% Here we write out all the LaTeX code that we will need. The most involved +% code sequences are those that control the alternate interword spacing and +% foreign language hyphenation patterns. The heavy use of \providecommand +% gives users a way to override the defaults. Special thanks to Javier Bezos, +% Johannes Braams, Robin Fairbairns, Heiko Oberdiek, Donald Arseneau and all +% the other gurus on comp.text.tex for their help and advice on the topic of +% \selectlanguage, Babel and BibTeX. +FUNCTION {begin.bib} +{ "% Generated by IEEEtran.bst, version: " bst.file.version * " (" * bst.file.date * ")" * + write$ newline$ + preamble$ empty$ 'skip$ + { preamble$ write$ newline$ } + if$ + "\begin{thebibliography}{" longest.label * "}" * + write$ newline$ + "\providecommand{\url}[1]{#1}" + write$ newline$ + "\csname url@samestyle\endcsname" + write$ newline$ + "\providecommand{\newblock}{\relax}" + write$ newline$ + "\providecommand{\bibinfo}[2]{#2}" + write$ newline$ + "\providecommand{\BIBentrySTDinterwordspacing}{\spaceskip=0pt\relax}" + write$ newline$ + "\providecommand{\BIBentryALTinterwordstretchfactor}{" + ALTinterwordstretchfactor * "}" * + write$ newline$ + "\providecommand{\BIBentryALTinterwordspacing}{\spaceskip=\fontdimen2\font plus " + write$ newline$ + "\BIBentryALTinterwordstretchfactor\fontdimen3\font minus \fontdimen4\font\relax}" + write$ newline$ + "\providecommand{\BIBforeignlanguage}[2]{{%" + write$ newline$ + "\expandafter\ifx\csname l@#1\endcsname\relax" + write$ newline$ + "\typeout{** WARNING: IEEEtran.bst: No hyphenation pattern has been}%" + write$ newline$ + "\typeout{** loaded for the language `#1'. Using the pattern for}%" + write$ newline$ + "\typeout{** the default language instead.}%" + write$ newline$ + "\else" + write$ newline$ + "\language=\csname l@#1\endcsname" + write$ newline$ + "\fi" + write$ newline$ + "#2}}" + write$ newline$ + "\providecommand{\BIBdecl}{\relax}" + write$ newline$ + "\BIBdecl" + write$ newline$ +} + +FUNCTION {end.bib} +{ newline$ "\end{thebibliography}" write$ newline$ } + +FUNCTION {if.url.alt.interword.spacing} +{ is.use.alt.interword.spacing + {url empty$ 'skip$ {"\BIBentryALTinterwordspacing" write$ newline$} if$} + { skip$ } + if$ +} + +FUNCTION {if.url.std.interword.spacing} +{ is.use.alt.interword.spacing + {url empty$ 'skip$ {"\BIBentrySTDinterwordspacing" write$ newline$} if$} + { skip$ } + if$ +} + + + + +%%%%%%%%%%%%%%%%%%%%%%%% +%% LONGEST LABEL PASS %% +%%%%%%%%%%%%%%%%%%%%%%%% + +FUNCTION {initialize.longest.label} +{ "" 'longest.label := + #1 'number.label := + #0 'longest.label.width := +} + +FUNCTION {longest.label.pass} +{ type$ "ieeetranbstctl" = + { skip$ } + { number.label int.to.str$ 'label := + number.label #1 + 'number.label := + label width$ longest.label.width > + { label 'longest.label := + label width$ 'longest.label.width := + } + { skip$ } + if$ + } + if$ +} + + + + +%%%%%%%%%%%%%%%%%%%%% +%% FORMAT HANDLERS %% +%%%%%%%%%%%%%%%%%%%%% + +%% Lower Level Formats (used by higher level formats) + +FUNCTION {format.address.org.or.pub.date} +{ 't := + "" + year empty$ + { "empty year in " cite$ * warning$ } + { skip$ } + if$ + address empty$ t empty$ and + year empty$ and month empty$ and + { skip$ } + { this.to.prev.status + this.status.std + cap.status.std + address "address" bibinfo.check * + t empty$ + { skip$ } + { punct.period 'prev.status.punct := + space.large 'prev.status.space := + address empty$ + { skip$ } + { ": " * } + if$ + t * + } + if$ + year empty$ month empty$ and + { skip$ } + { t empty$ address empty$ and + { skip$ } + { ", " * } + if$ + month empty$ + { year empty$ + { skip$ } + { year "year" bibinfo.check * } + if$ + } + { month "month" bibinfo.check * + year empty$ + { skip$ } + { " " * year "year" bibinfo.check * } + if$ + } + if$ + } + if$ + } + if$ +} + + +FUNCTION {format.names} +{ 'bibinfo := + duplicate$ empty$ 'skip$ { + this.to.prev.status + this.status.std + 's := + "" 't := + #1 'nameptr := + s num.names$ 'numnames := + numnames 'namesleft := + { namesleft #0 > } + { s nameptr + name.format.string + format.name$ + bibinfo bibinfo.check + 't := + nameptr #1 > + { nameptr num.names.shown.with.forced.et.al #1 + = + numnames max.num.names.before.forced.et.al > + is.forced.et.al and and + { "others" 't := + #1 'namesleft := + } + { skip$ } + if$ + namesleft #1 > + { ", " * t do.name.latex.cmd * } + { s nameptr "{ll}" format.name$ duplicate$ "others" = + { 't := } + { pop$ } + if$ + t "others" = + { " " * bbl.etal emphasize * } + { numnames #2 > + { "," * } + { skip$ } + if$ + bbl.and + space.word * t do.name.latex.cmd * + } + if$ + } + if$ + } + { t do.name.latex.cmd } + if$ + nameptr #1 + 'nameptr := + namesleft #1 - 'namesleft := + } + while$ + cap.status.std + } if$ +} + + + + +%% Higher Level Formats + +%% addresses/locations + +FUNCTION {format.address} +{ address duplicate$ empty$ 'skip$ + { this.to.prev.status + this.status.std + cap.status.std + } + if$ +} + + + +%% author/editor names + +FUNCTION {format.authors}{ author "author" format.names } + +FUNCTION {format.editors} +{ editor "editor" format.names duplicate$ empty$ 'skip$ + { ", " * + get.bbl.editor + capitalize + * + } + if$ +} + + + +%% date + +FUNCTION {format.date} +{ + month "month" bibinfo.check duplicate$ empty$ + year "year" bibinfo.check duplicate$ empty$ + { swap$ 'skip$ + { this.to.prev.status + this.status.std + cap.status.std + "there's a month but no year in " cite$ * warning$ } + if$ + * + } + { this.to.prev.status + this.status.std + cap.status.std + swap$ 'skip$ + { + swap$ + " " * swap$ + } + if$ + * + } + if$ +} + +FUNCTION {format.date.electronic} +{ month "month" bibinfo.check duplicate$ empty$ + year "year" bibinfo.check duplicate$ empty$ + { swap$ + { pop$ } + { "there's a month but no year in " cite$ * warning$ + pop$ ")" * "(" swap$ * + this.to.prev.status + punct.no 'this.status.punct := + space.normal 'this.status.space := + quote.no 'this.status.quote := + cap.yes 'status.cap := + } + if$ + } + { swap$ + { swap$ pop$ ")" * "(" swap$ * } + { "(" swap$ * ", " * swap$ * ")" * } + if$ + this.to.prev.status + punct.no 'this.status.punct := + space.normal 'this.status.space := + quote.no 'this.status.quote := + cap.yes 'status.cap := + } + if$ +} + + + +%% edition/title + +% Note: IEEE considers the edition to be closely associated with +% the title of a book. So, in IEEEtran.bst the edition is normally handled +% within the formatting of the title. The format.edition function is +% retained here for possible future use. +FUNCTION {format.edition} +{ edition duplicate$ empty$ 'skip$ + { this.to.prev.status + this.status.std + convert.edition + status.cap + { "t" } + { "l" } + if$ change.case$ + "edition" bibinfo.check + "~" * bbl.edition * + cap.status.std + } + if$ +} + +% This is used to format the booktitle of a conference proceedings. +% Here we use the "intype" field to provide the user a way to +% override the word "in" (e.g., with things like "presented at") +% Use of intype stops the emphasis of the booktitle to indicate that +% we no longer mean the written conference proceedings, but the +% conference itself. +FUNCTION {format.in.booktitle} +{ booktitle "booktitle" bibinfo.check duplicate$ empty$ 'skip$ + { this.to.prev.status + this.status.std + select.language + intype missing$ + { emphasize + bbl.in " " * + } + { intype " " * } + if$ + swap$ * + cap.status.std + } + if$ +} + +% This is used to format the booktitle of collection. +% Here the "intype" field is not supported, but "edition" is. +FUNCTION {format.in.booktitle.edition} +{ booktitle "booktitle" bibinfo.check duplicate$ empty$ 'skip$ + { this.to.prev.status + this.status.std + select.language + emphasize + edition empty$ 'skip$ + { ", " * + edition + convert.edition + "l" change.case$ + * "~" * bbl.edition * + } + if$ + bbl.in " " * swap$ * + cap.status.std + } + if$ +} + +FUNCTION {format.article.title} +{ title duplicate$ empty$ 'skip$ + { this.to.prev.status + this.status.std + "t" change.case$ + } + if$ + "title" bibinfo.check + duplicate$ empty$ 'skip$ + { quote.close 'this.status.quote := + is.last.char.not.punct + { punct.std 'this.status.punct := } + { punct.no 'this.status.punct := } + if$ + select.language + "``" swap$ * + cap.status.std + } + if$ +} + +FUNCTION {format.article.title.electronic} +{ title duplicate$ empty$ 'skip$ + { this.to.prev.status + this.status.std + cap.status.std + "t" change.case$ + } + if$ + "title" bibinfo.check + duplicate$ empty$ + { skip$ } + { select.language } + if$ +} + +FUNCTION {format.book.title.edition} +{ title "title" bibinfo.check + duplicate$ empty$ + { "empty title in " cite$ * warning$ } + { this.to.prev.status + this.status.std + select.language + emphasize + edition empty$ 'skip$ + { ", " * + edition + convert.edition + status.cap + { "t" } + { "l" } + if$ + change.case$ + * "~" * bbl.edition * + } + if$ + cap.status.std + } + if$ +} + +FUNCTION {format.book.title} +{ title "title" bibinfo.check + duplicate$ empty$ 'skip$ + { this.to.prev.status + this.status.std + cap.status.std + select.language + emphasize + } + if$ +} + + + +%% journal + +FUNCTION {format.journal} +{ journal duplicate$ empty$ 'skip$ + { this.to.prev.status + this.status.std + cap.status.std + select.language + emphasize + } + if$ +} + + + +%% how published + +FUNCTION {format.howpublished} +{ howpublished duplicate$ empty$ 'skip$ + { this.to.prev.status + this.status.std + cap.status.std + } + if$ +} + + + +%% institutions/organization/publishers/school + +FUNCTION {format.institution} +{ institution duplicate$ empty$ 'skip$ + { this.to.prev.status + this.status.std + cap.status.std + } + if$ +} + +FUNCTION {format.organization} +{ organization duplicate$ empty$ 'skip$ + { this.to.prev.status + this.status.std + cap.status.std + } + if$ +} + +FUNCTION {format.address.publisher.date} +{ publisher "publisher" bibinfo.warn format.address.org.or.pub.date } + +FUNCTION {format.address.publisher.date.nowarn} +{ publisher "publisher" bibinfo.check format.address.org.or.pub.date } + +FUNCTION {format.address.organization.date} +{ organization "organization" bibinfo.check format.address.org.or.pub.date } + +FUNCTION {format.school} +{ school duplicate$ empty$ 'skip$ + { this.to.prev.status + this.status.std + cap.status.std + } + if$ +} + + + +%% volume/number/series/chapter/pages + +FUNCTION {format.volume} +{ volume empty.field.to.null.string + duplicate$ empty$ 'skip$ + { this.to.prev.status + this.status.std + bbl.volume + status.cap + { capitalize } + { skip$ } + if$ + swap$ tie.or.space.prefix + "volume" bibinfo.check + * * + cap.status.std + } + if$ +} + +FUNCTION {format.number} +{ number empty.field.to.null.string + duplicate$ empty$ 'skip$ + { this.to.prev.status + this.status.std + status.cap + { bbl.number capitalize } + { bbl.number } + if$ + swap$ tie.or.space.prefix + "number" bibinfo.check + * * + cap.status.std + } + if$ +} + +FUNCTION {format.number.if.use.for.article} +{ is.use.number.for.article + { format.number } + { "" } + if$ +} + +% IEEE does not seem to tie the series so closely with the volume +% and number as is done in other bibliography styles. Instead the +% series is treated somewhat like an extension of the title. +FUNCTION {format.series} +{ series empty$ + { "" } + { this.to.prev.status + this.status.std + bbl.series " " * + series "series" bibinfo.check * + cap.status.std + } + if$ +} + + +FUNCTION {format.chapter} +{ chapter empty$ + { "" } + { this.to.prev.status + this.status.std + type empty$ + { bbl.chapter } + { type "l" change.case$ + "type" bibinfo.check + } + if$ + chapter tie.or.space.prefix + "chapter" bibinfo.check + * * + cap.status.std + } + if$ +} + + +% The intended use of format.paper is for paper numbers of inproceedings. +% The paper type can be overridden via the type field. +% We allow the type to be displayed even if the paper number is absent +% for things like "postdeadline paper" +FUNCTION {format.paper} +{ is.use.paper + { paper empty$ + { type empty$ + { "" } + { this.to.prev.status + this.status.std + type "type" bibinfo.check + cap.status.std + } + if$ + } + { this.to.prev.status + this.status.std + type empty$ + { bbl.paper } + { type "type" bibinfo.check } + if$ + " " * paper + "paper" bibinfo.check + * + cap.status.std + } + if$ + } + { "" } + if$ +} + + +FUNCTION {format.pages} +{ pages duplicate$ empty$ 'skip$ + { this.to.prev.status + this.status.std + duplicate$ is.multiple.pages + { + bbl.pages swap$ + n.dashify + } + { + bbl.page swap$ + } + if$ + tie.or.space.prefix + "pages" bibinfo.check + * * + cap.status.std + } + if$ +} + + + +%% technical report number + +FUNCTION {format.tech.report.number} +{ number "number" bibinfo.check + this.to.prev.status + this.status.std + cap.status.std + type duplicate$ empty$ + { pop$ + bbl.techrep + } + { skip$ } + if$ + "type" bibinfo.check + swap$ duplicate$ empty$ + { pop$ } + { tie.or.space.prefix * * } + if$ +} + + + +%% note + +FUNCTION {format.note} +{ note empty$ + { "" } + { this.to.prev.status + this.status.std + punct.period 'this.status.punct := + note #1 #1 substring$ + duplicate$ "{" = + { skip$ } + { status.cap + { "u" } + { "l" } + if$ + change.case$ + } + if$ + note #2 global.max$ substring$ * "note" bibinfo.check + cap.yes 'status.cap := + } + if$ +} + + + +%% patent + +FUNCTION {format.patent.date} +{ this.to.prev.status + this.status.std + year empty$ + { monthfiled duplicate$ empty$ + { "monthfiled" bibinfo.check pop$ "" } + { "monthfiled" bibinfo.check } + if$ + dayfiled duplicate$ empty$ + { "dayfiled" bibinfo.check pop$ "" * } + { "dayfiled" bibinfo.check + monthfiled empty$ + { "dayfiled without a monthfiled in " cite$ * warning$ + * + } + { " " swap$ * * } + if$ + } + if$ + yearfiled empty$ + { "no year or yearfiled in " cite$ * warning$ } + { yearfiled "yearfiled" bibinfo.check + swap$ + duplicate$ empty$ + { pop$ } + { ", " * swap$ * } + if$ + } + if$ + } + { month duplicate$ empty$ + { "month" bibinfo.check pop$ "" } + { "month" bibinfo.check } + if$ + day duplicate$ empty$ + { "day" bibinfo.check pop$ "" * } + { "day" bibinfo.check + month empty$ + { "day without a month in " cite$ * warning$ + * + } + { " " swap$ * * } + if$ + } + if$ + year "year" bibinfo.check + swap$ + duplicate$ empty$ + { pop$ } + { ", " * swap$ * } + if$ + } + if$ + cap.status.std +} + +FUNCTION {format.patent.nationality.type.number} +{ this.to.prev.status + this.status.std + nationality duplicate$ empty$ + { "nationality" bibinfo.warn pop$ "" } + { "nationality" bibinfo.check + duplicate$ "l" change.case$ "united states" = + { pop$ bbl.patentUS } + { skip$ } + if$ + " " * + } + if$ + type empty$ + { bbl.patent "type" bibinfo.check } + { type "type" bibinfo.check } + if$ + * + number duplicate$ empty$ + { "number" bibinfo.warn pop$ } + { "number" bibinfo.check + large.number.separate + swap$ " " * swap$ * + } + if$ + cap.status.std +} + + + +%% standard + +FUNCTION {format.organization.institution.standard.type.number} +{ this.to.prev.status + this.status.std + organization duplicate$ empty$ + { pop$ + institution duplicate$ empty$ + { "institution" bibinfo.warn } + { "institution" bibinfo.warn " " * } + if$ + } + { "organization" bibinfo.warn " " * } + if$ + type empty$ + { bbl.standard "type" bibinfo.check } + { type "type" bibinfo.check } + if$ + * + number duplicate$ empty$ + { "number" bibinfo.check pop$ } + { "number" bibinfo.check + large.number.separate + swap$ " " * swap$ * + } + if$ + cap.status.std +} + +FUNCTION {format.revision} +{ revision empty$ + { "" } + { this.to.prev.status + this.status.std + bbl.revision + revision tie.or.space.prefix + "revision" bibinfo.check + * * + cap.status.std + } + if$ +} + + +%% thesis + +FUNCTION {format.master.thesis.type} +{ this.to.prev.status + this.status.std + type empty$ + { + bbl.mthesis + } + { + type "type" bibinfo.check + } + if$ +cap.status.std +} + +FUNCTION {format.phd.thesis.type} +{ this.to.prev.status + this.status.std + type empty$ + { + bbl.phdthesis + } + { + type "type" bibinfo.check + } + if$ +cap.status.std +} + + + +%% URL + +FUNCTION {format.url} +{ url empty$ + { "" } + { this.to.prev.status + this.status.std + cap.yes 'status.cap := + name.url.prefix " " * + "\url{" * url * "}" * + punct.no 'this.status.punct := + punct.period 'prev.status.punct := + space.normal 'this.status.space := + space.normal 'prev.status.space := + quote.no 'this.status.quote := + } + if$ +} + + + + +%%%%%%%%%%%%%%%%%%%% +%% ENTRY HANDLERS %% +%%%%%%%%%%%%%%%%%%%% + + +% Note: In many journals, IEEE (or the authors) tend not to show the number +% for articles, so the display of the number is controlled here by the +% switch "is.use.number.for.article" +FUNCTION {article} +{ std.status.using.comma + start.entry + if.url.alt.interword.spacing + format.authors "author" output.warn + name.or.dash + format.article.title "title" output.warn + format.journal "journal" bibinfo.check "journal" output.warn + format.volume output + format.number.if.use.for.article output + format.pages output + format.date "year" output.warn + format.note output + format.url output + fin.entry + if.url.std.interword.spacing +} + +FUNCTION {book} +{ std.status.using.comma + start.entry + if.url.alt.interword.spacing + author empty$ + { format.editors "author and editor" output.warn } + { format.authors output.nonnull } + if$ + name.or.dash + format.book.title.edition output + format.series output + author empty$ + { skip$ } + { format.editors output } + if$ + format.address.publisher.date output + format.volume output + format.number output + format.note output + format.url output + fin.entry + if.url.std.interword.spacing +} + +FUNCTION {booklet} +{ std.status.using.comma + start.entry + if.url.alt.interword.spacing + format.authors output + name.or.dash + format.article.title "title" output.warn + format.howpublished "howpublished" bibinfo.check output + format.organization "organization" bibinfo.check output + format.address "address" bibinfo.check output + format.date output + format.note output + format.url output + fin.entry + if.url.std.interword.spacing +} + +FUNCTION {electronic} +{ std.status.using.period + start.entry + if.url.alt.interword.spacing + format.authors output + name.or.dash + format.date.electronic output + format.article.title.electronic output + format.howpublished "howpublished" bibinfo.check output + format.organization "organization" bibinfo.check output + format.address "address" bibinfo.check output + format.note output + format.url output + fin.entry + empty.entry.warn + if.url.std.interword.spacing +} + +FUNCTION {inbook} +{ std.status.using.comma + start.entry + if.url.alt.interword.spacing + author empty$ + { format.editors "author and editor" output.warn } + { format.authors output.nonnull } + if$ + name.or.dash + format.book.title.edition output + format.series output + format.address.publisher.date output + format.volume output + format.number output + format.chapter output + format.pages output + format.note output + format.url output + fin.entry + if.url.std.interword.spacing +} + +FUNCTION {incollection} +{ std.status.using.comma + start.entry + if.url.alt.interword.spacing + format.authors "author" output.warn + name.or.dash + format.article.title "title" output.warn + format.in.booktitle.edition "booktitle" output.warn + format.series output + format.editors output + format.address.publisher.date.nowarn output + format.volume output + format.number output + format.chapter output + format.pages output + format.note output + format.url output + fin.entry + if.url.std.interword.spacing +} + +FUNCTION {inproceedings} +{ std.status.using.comma + start.entry + if.url.alt.interword.spacing + format.authors "author" output.warn + name.or.dash + format.article.title "title" output.warn + format.in.booktitle "booktitle" output.warn + format.series output + format.editors output + format.volume output + format.number output + publisher empty$ + { format.address.organization.date output } + { format.organization "organization" bibinfo.check output + format.address.publisher.date output + } + if$ + format.paper output + format.pages output + format.note output + format.url output + fin.entry + if.url.std.interword.spacing +} + +FUNCTION {manual} +{ std.status.using.comma + start.entry + if.url.alt.interword.spacing + format.authors output + name.or.dash + format.book.title.edition "title" output.warn + format.howpublished "howpublished" bibinfo.check output + format.organization "organization" bibinfo.check output + format.address "address" bibinfo.check output + format.date output + format.note output + format.url output + fin.entry + if.url.std.interword.spacing +} + +FUNCTION {mastersthesis} +{ std.status.using.comma + start.entry + if.url.alt.interword.spacing + format.authors "author" output.warn + name.or.dash + format.article.title "title" output.warn + format.master.thesis.type output.nonnull + format.school "school" bibinfo.warn output + format.address "address" bibinfo.check output + format.date "year" output.warn + format.note output + format.url output + fin.entry + if.url.std.interword.spacing +} + +FUNCTION {misc} +{ std.status.using.comma + start.entry + if.url.alt.interword.spacing + format.authors output + name.or.dash + format.article.title output + format.howpublished "howpublished" bibinfo.check output + format.organization "organization" bibinfo.check output + format.address "address" bibinfo.check output + format.pages output + format.date output + format.note output + format.url output + fin.entry + empty.entry.warn + if.url.std.interword.spacing +} + +FUNCTION {patent} +{ std.status.using.comma + start.entry + if.url.alt.interword.spacing + format.authors output + name.or.dash + format.article.title output + format.patent.nationality.type.number output + format.patent.date output + format.note output + format.url output + fin.entry + empty.entry.warn + if.url.std.interword.spacing +} + +FUNCTION {periodical} +{ std.status.using.comma + start.entry + if.url.alt.interword.spacing + format.editors output + name.or.dash + format.book.title "title" output.warn + format.series output + format.volume output + format.number output + format.organization "organization" bibinfo.check output + format.date "year" output.warn + format.note output + format.url output + fin.entry + if.url.std.interword.spacing +} + +FUNCTION {phdthesis} +{ std.status.using.comma + start.entry + if.url.alt.interword.spacing + format.authors "author" output.warn + name.or.dash + format.article.title "title" output.warn + format.phd.thesis.type output.nonnull + format.school "school" bibinfo.warn output + format.address "address" bibinfo.check output + format.date "year" output.warn + format.note output + format.url output + fin.entry + if.url.std.interword.spacing +} + +FUNCTION {proceedings} +{ std.status.using.comma + start.entry + if.url.alt.interword.spacing + format.editors output + name.or.dash + format.book.title "title" output.warn + format.series output + format.volume output + format.number output + publisher empty$ + { format.address.organization.date output } + { format.organization "organization" bibinfo.check output + format.address.publisher.date output + } + if$ + format.note output + format.url output + fin.entry + if.url.std.interword.spacing +} + +FUNCTION {standard} +{ std.status.using.comma + start.entry + if.url.alt.interword.spacing + format.authors output + name.or.dash + format.book.title "title" output.warn + format.howpublished "howpublished" bibinfo.check output + format.organization.institution.standard.type.number output + format.revision output + format.date output + format.note output + format.url output + fin.entry + if.url.std.interword.spacing +} + +FUNCTION {techreport} +{ std.status.using.comma + start.entry + if.url.alt.interword.spacing + format.authors "author" output.warn + name.or.dash + format.article.title "title" output.warn + format.howpublished "howpublished" bibinfo.check output + format.institution "institution" bibinfo.warn output + format.address "address" bibinfo.check output + format.tech.report.number output.nonnull + format.date "year" output.warn + format.note output + format.url output + fin.entry + if.url.std.interword.spacing +} + +FUNCTION {unpublished} +{ std.status.using.comma + start.entry + if.url.alt.interword.spacing + format.authors "author" output.warn + name.or.dash + format.article.title "title" output.warn + format.date output + format.note "note" output.warn + format.url output + fin.entry + if.url.std.interword.spacing +} + + +% The special entry type which provides the user interface to the +% BST controls +FUNCTION {IEEEtranBSTCTL} +{ is.print.banners.to.terminal + { "** IEEEtran BST control entry " quote$ * cite$ * quote$ * " detected." * + top$ + } + { skip$ } + if$ + CTLuse_article_number + empty$ + { skip$ } + { CTLuse_article_number + yes.no.to.int + 'is.use.number.for.article := + } + if$ + CTLuse_paper + empty$ + { skip$ } + { CTLuse_paper + yes.no.to.int + 'is.use.paper := + } + if$ + CTLuse_forced_etal + empty$ + { skip$ } + { CTLuse_forced_etal + yes.no.to.int + 'is.forced.et.al := + } + if$ + CTLmax_names_forced_etal + empty$ + { skip$ } + { CTLmax_names_forced_etal + string.to.integer + 'max.num.names.before.forced.et.al := + } + if$ + CTLnames_show_etal + empty$ + { skip$ } + { CTLnames_show_etal + string.to.integer + 'num.names.shown.with.forced.et.al := + } + if$ + CTLuse_alt_spacing + empty$ + { skip$ } + { CTLuse_alt_spacing + yes.no.to.int + 'is.use.alt.interword.spacing := + } + if$ + CTLalt_stretch_factor + empty$ + { skip$ } + { CTLalt_stretch_factor + 'ALTinterwordstretchfactor := + "\renewcommand{\BIBentryALTinterwordstretchfactor}{" + ALTinterwordstretchfactor * "}" * + write$ newline$ + } + if$ + CTLdash_repeated_names + empty$ + { skip$ } + { CTLdash_repeated_names + yes.no.to.int + 'is.dash.repeated.names := + } + if$ + CTLname_format_string + empty$ + { skip$ } + { CTLname_format_string + 'name.format.string := + } + if$ + CTLname_latex_cmd + empty$ + { skip$ } + { CTLname_latex_cmd + 'name.latex.cmd := + } + if$ + CTLname_url_prefix + missing$ + { skip$ } + { CTLname_url_prefix + 'name.url.prefix := + } + if$ + + + num.names.shown.with.forced.et.al max.num.names.before.forced.et.al > + { "CTLnames_show_etal cannot be greater than CTLmax_names_forced_etal in " cite$ * warning$ + max.num.names.before.forced.et.al 'num.names.shown.with.forced.et.al := + } + { skip$ } + if$ +} + + +%%%%%%%%%%%%%%%%%%% +%% ENTRY ALIASES %% +%%%%%%%%%%%%%%%%%%% +FUNCTION {conference}{inproceedings} +FUNCTION {online}{electronic} +FUNCTION {internet}{electronic} +FUNCTION {webpage}{electronic} +FUNCTION {www}{electronic} +FUNCTION {default.type}{misc} + + + +%%%%%%%%%%%%%%%%%% +%% MAIN PROGRAM %% +%%%%%%%%%%%%%%%%%% + +READ + +EXECUTE {initialize.controls} +EXECUTE {initialize.status.constants} +EXECUTE {banner.message} + +EXECUTE {initialize.longest.label} +ITERATE {longest.label.pass} + +EXECUTE {begin.bib} +ITERATE {call.type$} +EXECUTE {end.bib} + +EXECUTE{completed.message} + + +%% That's all folks, mds. diff --git a/cometbft/v0.39/spec/consensus/consensus-paper/IEEEtran.cls b/cometbft/v0.39/spec/consensus/consensus-paper/IEEEtran.cls new file mode 100644 index 000000000..9c967d555 --- /dev/null +++ b/cometbft/v0.39/spec/consensus/consensus-paper/IEEEtran.cls @@ -0,0 +1,4733 @@ +%% +%% IEEEtran.cls 2011/11/03 version V1.8 based on +%% IEEEtran.cls 2007/03/05 version V1.7a +%% The changes in V1.8 are made with a single goal in mind: +%% to change the look of the output using the [conference] option +%% and the default font size (10pt) to match the Word template more closely. +%% These changes may well have undesired side effects when other options +%% are in force! +%% +%% +%% This is the official IEEE LaTeX class for authors of the Institute of +%% Electrical and Electronics Engineers (IEEE) Transactions journals and +%% conferences. +%% +%% Support sites: +%% http://www.michaelshell.org/tex/ieeetran/ +%% http://www.ctan.org/tex-archive/macros/latex/contrib/IEEEtran/ +%% and +%% http://www.ieee.org/ +%% +%% Based on the original 1993 IEEEtran.cls, but with many bug fixes +%% and enhancements (from both JVH and MDS) over the 1996/7 version. +%% +%% +%% Contributors: +%% Gerry Murray (1993), Silvano Balemi (1993), +%% Jon Dixon (1996), Peter N"uchter (1996), +%% Juergen von Hagen (2000), and Michael Shell (2001-2007) +%% +%% +%% Copyright (c) 1993-2000 by Gerry Murray, Silvano Balemi, +%% Jon Dixon, Peter N"uchter, +%% Juergen von Hagen +%% and +%% Copyright (c) 2001-2007 by Michael Shell +%% +%% Current maintainer (V1.3 to V1.7): Michael Shell +%% See: +%% http://www.michaelshell.org/ +%% for current contact information. +%% +%% Special thanks to Peter Wilson (CUA) and Donald Arseneau +%% for allowing the inclusion of the \@ifmtarg command +%% from their ifmtarg LaTeX package. +%% +%%************************************************************************* +%% Legal Notice: +%% This code is offered as-is without any warranty either expressed or +%% implied; without even the implied warranty of MERCHANTABILITY or +%% FITNESS FOR A PARTICULAR PURPOSE! +%% User assumes all risk. +%% In no event shall IEEE or any contributor to this code be liable for +%% any damages or losses, including, but not limited to, incidental, +%% consequential, or any other damages, resulting from the use or misuse +%% of any information contained here. +%% +%% All comments are the opinions of their respective authors and are not +%% necessarily endorsed by the IEEE. +%% +%% This work is distributed under the LaTeX Project Public License (LPPL) +%% ( http://www.latex-project.org/ ) version 1.3, and may be freely used, +%% distributed and modified. A copy of the LPPL, version 1.3, is included +%% in the base LaTeX documentation of all distributions of LaTeX released +%% 2003/12/01 or later. +%% Retain all contribution notices and credits. +%% ** Modified files should be clearly indicated as such, including ** +%% ** renaming them and changing author support contact information. ** +%% +%% File list of work: IEEEtran.cls, IEEEtran_HOWTO.pdf, bare_adv.tex, +%% bare_conf.tex, bare_jrnl.tex, bare_jrnl_compsoc.tex +%% +%% Major changes to the user interface should be indicated by an +%% increase in the version numbers. If a version is a beta, it will +%% be indicated with a BETA suffix, i.e., 1.4 BETA. +%% Small changes can be indicated by appending letters to the version +%% such as "IEEEtran_v14a.cls". +%% In all cases, \Providesclass, any \typeout messages to the user, +%% \IEEEtransversionmajor and \IEEEtransversionminor must reflect the +%% correct version information. +%% The changes should also be documented via source comments. +%%************************************************************************* +%% +% +% Available class options +% e.g., \documentclass[10pt,conference]{IEEEtran} +% +% *** choose only one from each category *** +% +% 9pt, 10pt, 11pt, 12pt +% Sets normal font size. The default is 10pt. +% +% conference, journal, technote, peerreview, peerreviewca +% determines format mode - conference papers, journal papers, +% correspondence papers (technotes), or peer review papers. The user +% should also select 9pt when using technote. peerreview is like +% journal mode, but provides for a single-column "cover" title page for +% anonymous peer review. The paper title (without the author names) is +% repeated at the top of the page after the cover page. For peer review +% papers, the \IEEEpeerreviewmaketitle command must be executed (will +% automatically be ignored for non-peerreview modes) at the place the +% cover page is to end, usually just after the abstract (keywords are +% not normally used with peer review papers). peerreviewca is like +% peerreview, but allows the author names to be entered and formatted +% as with conference mode so that author affiliation and contact +% information can be easily seen on the cover page. +% The default is journal. +% +% draft, draftcls, draftclsnofoot, final +% determines if paper is formatted as a widely spaced draft (for +% handwritten editor comments) or as a properly typeset final version. +% draftcls restricts draft mode to the class file while all other LaTeX +% packages (i.e., \usepackage{graphicx}) will behave as final - allows +% for a draft paper with visible figures, etc. draftclsnofoot is like +% draftcls, but does not display the date and the word "DRAFT" at the foot +% of the pages. If using one of the draft modes, the user will probably +% also want to select onecolumn. +% The default is final. +% +% letterpaper, a4paper +% determines paper size: 8.5in X 11in or 210mm X 297mm. CHANGING THE PAPER +% SIZE WILL NOT ALTER THE TYPESETTING OF THE DOCUMENT - ONLY THE MARGINS +% WILL BE AFFECTED. In particular, documents using the a4paper option will +% have reduced side margins (A4 is narrower than US letter) and a longer +% bottom margin (A4 is longer than US letter). For both cases, the top +% margins will be the same and the text will be horizontally centered. +% For final submission to IEEE, authors should use US letter (8.5 X 11in) +% paper. Note that authors should ensure that all post-processing +% (ps, pdf, etc.) uses the same paper specificiation as the .tex document. +% Problems here are by far the number one reason for incorrect margins. +% IEEEtran will automatically set the default paper size under pdflatex +% (without requiring a change to pdftex.cfg), so this issue is more +% important to dvips users. Fix config.ps, config.pdf, or ~/.dvipsrc for +% dvips, or use the dvips -t papersize option instead as needed. See the +% testflow documentation +% http://www.ctan.org/tex-archive/macros/latex/contrib/IEEEtran/testflow +% for more details on dvips paper size configuration. +% The default is letterpaper. +% +% oneside, twoside +% determines if layout follows single sided or two sided (duplex) +% printing. The only notable change is with the headings at the top of +% the pages. +% The default is oneside. +% +% onecolumn, twocolumn +% determines if text is organized into one or two columns per page. One +% column mode is usually used only with draft papers. +% The default is twocolumn. +% +% compsoc +% Use the format of the IEEE Computer Society. +% +% romanappendices +% Use the "Appendix I" convention when numbering appendices. IEEEtran.cls +% now defaults to Alpha "Appendix A" convention - the opposite of what +% v1.6b and earlier did. +% +% captionsoff +% disables the display of the figure/table captions. Some IEEE journals +% request that captions be removed and figures/tables be put on pages +% of their own at the end of an initial paper submission. The endfloat +% package can be used with this class option to achieve this format. +% +% nofonttune +% turns off tuning of the font interword spacing. Maybe useful to those +% not using the standard Times fonts or for those who have already "tuned" +% their fonts. +% The default is to enable IEEEtran to tune font parameters. +% +% +%---------- +% Available CLASSINPUTs provided (all are macros unless otherwise noted): +% \CLASSINPUTbaselinestretch +% \CLASSINPUTinnersidemargin +% \CLASSINPUToutersidemargin +% \CLASSINPUTtoptextmargin +% \CLASSINPUTbottomtextmargin +% +% Available CLASSINFOs provided: +% \ifCLASSINFOpdf (TeX if conditional) +% \CLASSINFOpaperwidth (macro) +% \CLASSINFOpaperheight (macro) +% \CLASSINFOnormalsizebaselineskip (length) +% \CLASSINFOnormalsizeunitybaselineskip (length) +% +% Available CLASSOPTIONs provided: +% all class option flags (TeX if conditionals) unless otherwise noted, +% e.g., \ifCLASSOPTIONcaptionsoff +% point size options provided as a single macro: +% \CLASSOPTIONpt +% which will be defined as 9, 10, 11, or 12 depending on the document's +% normalsize point size. +% also, class option peerreviewca implies the use of class option peerreview +% and classoption draft implies the use of class option draftcls + + + + + +\ProvidesClass{IEEEtran}[2012/11/21 V1.8c by Harald Hanche-Olsen and Anders Christensen] +\typeout{-- Based on V1.7a by Michael Shell} +\typeout{-- See the "IEEEtran_HOWTO" manual for usage information.} +\typeout{-- http://www.michaelshell.org/tex/ieeetran/} +\NeedsTeXFormat{LaTeX2e} + +% IEEEtran.cls version numbers, provided as of V1.3 +% These values serve as a way a .tex file can +% determine if the new features are provided. +% The version number of this IEEEtrans.cls can be obtained from +% these values. i.e., V1.4 +% KEEP THESE AS INTEGERS! i.e., NO {4a} or anything like that- +% (no need to enumerate "a" minor changes here) +\def\IEEEtransversionmajor{1} +\def\IEEEtransversionminor{7} + +% These do nothing, but provide them like in article.cls +\newif\if@restonecol +\newif\if@titlepage + + +% class option conditionals +\newif\ifCLASSOPTIONonecolumn \CLASSOPTIONonecolumnfalse +\newif\ifCLASSOPTIONtwocolumn \CLASSOPTIONtwocolumntrue + +\newif\ifCLASSOPTIONoneside \CLASSOPTIONonesidetrue +\newif\ifCLASSOPTIONtwoside \CLASSOPTIONtwosidefalse + +\newif\ifCLASSOPTIONfinal \CLASSOPTIONfinaltrue +\newif\ifCLASSOPTIONdraft \CLASSOPTIONdraftfalse +\newif\ifCLASSOPTIONdraftcls \CLASSOPTIONdraftclsfalse +\newif\ifCLASSOPTIONdraftclsnofoot \CLASSOPTIONdraftclsnofootfalse + +\newif\ifCLASSOPTIONpeerreview \CLASSOPTIONpeerreviewfalse +\newif\ifCLASSOPTIONpeerreviewca \CLASSOPTIONpeerreviewcafalse + +\newif\ifCLASSOPTIONjournal \CLASSOPTIONjournaltrue +\newif\ifCLASSOPTIONconference \CLASSOPTIONconferencefalse +\newif\ifCLASSOPTIONtechnote \CLASSOPTIONtechnotefalse + +\newif\ifCLASSOPTIONnofonttune \CLASSOPTIONnofonttunefalse + +\newif\ifCLASSOPTIONcaptionsoff \CLASSOPTIONcaptionsofffalse + +\newif\ifCLASSOPTIONcompsoc \CLASSOPTIONcompsocfalse + +\newif\ifCLASSOPTIONromanappendices \CLASSOPTIONromanappendicesfalse + + +% class info conditionals + +% indicates if pdf (via pdflatex) output +\newif\ifCLASSINFOpdf \CLASSINFOpdffalse + + +% V1.6b internal flag to show if using a4paper +\newif\if@IEEEusingAfourpaper \@IEEEusingAfourpaperfalse + + + +% IEEEtran class scratch pad registers +% dimen +\newdimen\@IEEEtrantmpdimenA +\newdimen\@IEEEtrantmpdimenB +% count +\newcount\@IEEEtrantmpcountA +\newcount\@IEEEtrantmpcountB +% token list +\newtoks\@IEEEtrantmptoksA + +% we use \CLASSOPTIONpt so that we can ID the point size (even for 9pt docs) +% as well as LaTeX's \@ptsize to retain some compatability with some +% external packages +\def\@ptsize{0} +% LaTeX does not support 9pt, so we set \@ptsize to 0 - same as that of 10pt +\DeclareOption{9pt}{\def\CLASSOPTIONpt{9}\def\@ptsize{0}} +\DeclareOption{10pt}{\def\CLASSOPTIONpt{10}\def\@ptsize{0}} +\DeclareOption{11pt}{\def\CLASSOPTIONpt{11}\def\@ptsize{1}} +\DeclareOption{12pt}{\def\CLASSOPTIONpt{12}\def\@ptsize{2}} + + + +\DeclareOption{letterpaper}{\setlength{\paperheight}{11in}% + \setlength{\paperwidth}{8.5in}% + \@IEEEusingAfourpaperfalse + \def\CLASSOPTIONpaper{letter}% + \def\CLASSINFOpaperwidth{8.5in}% + \def\CLASSINFOpaperheight{11in}} + + +\DeclareOption{a4paper}{\setlength{\paperheight}{297mm}% + \setlength{\paperwidth}{210mm}% + \@IEEEusingAfourpapertrue + \def\CLASSOPTIONpaper{a4}% + \def\CLASSINFOpaperwidth{210mm}% + \def\CLASSINFOpaperheight{297mm}} + +\DeclareOption{oneside}{\@twosidefalse\@mparswitchfalse + \CLASSOPTIONonesidetrue\CLASSOPTIONtwosidefalse} +\DeclareOption{twoside}{\@twosidetrue\@mparswitchtrue + \CLASSOPTIONtwosidetrue\CLASSOPTIONonesidefalse} + +\DeclareOption{onecolumn}{\CLASSOPTIONonecolumntrue\CLASSOPTIONtwocolumnfalse} +\DeclareOption{twocolumn}{\CLASSOPTIONtwocolumntrue\CLASSOPTIONonecolumnfalse} + +% If the user selects draft, then this class AND any packages +% will go into draft mode. +\DeclareOption{draft}{\CLASSOPTIONdrafttrue\CLASSOPTIONdraftclstrue + \CLASSOPTIONdraftclsnofootfalse} +% draftcls is for a draft mode which will not affect any packages +% used by the document. +\DeclareOption{draftcls}{\CLASSOPTIONdraftfalse\CLASSOPTIONdraftclstrue + \CLASSOPTIONdraftclsnofootfalse} +% draftclsnofoot is like draftcls, but without the footer. +\DeclareOption{draftclsnofoot}{\CLASSOPTIONdraftfalse\CLASSOPTIONdraftclstrue + \CLASSOPTIONdraftclsnofoottrue} +\DeclareOption{final}{\CLASSOPTIONdraftfalse\CLASSOPTIONdraftclsfalse + \CLASSOPTIONdraftclsnofootfalse} + +\DeclareOption{journal}{\CLASSOPTIONpeerreviewfalse\CLASSOPTIONpeerreviewcafalse + \CLASSOPTIONjournaltrue\CLASSOPTIONconferencefalse\CLASSOPTIONtechnotefalse} + +\DeclareOption{conference}{\CLASSOPTIONpeerreviewfalse\CLASSOPTIONpeerreviewcafalse + \CLASSOPTIONjournalfalse\CLASSOPTIONconferencetrue\CLASSOPTIONtechnotefalse} + +\DeclareOption{technote}{\CLASSOPTIONpeerreviewfalse\CLASSOPTIONpeerreviewcafalse + \CLASSOPTIONjournalfalse\CLASSOPTIONconferencefalse\CLASSOPTIONtechnotetrue} + +\DeclareOption{peerreview}{\CLASSOPTIONpeerreviewtrue\CLASSOPTIONpeerreviewcafalse + \CLASSOPTIONjournalfalse\CLASSOPTIONconferencefalse\CLASSOPTIONtechnotefalse} + +\DeclareOption{peerreviewca}{\CLASSOPTIONpeerreviewtrue\CLASSOPTIONpeerreviewcatrue + \CLASSOPTIONjournalfalse\CLASSOPTIONconferencefalse\CLASSOPTIONtechnotefalse} + +\DeclareOption{nofonttune}{\CLASSOPTIONnofonttunetrue} + +\DeclareOption{captionsoff}{\CLASSOPTIONcaptionsofftrue} + +\DeclareOption{compsoc}{\CLASSOPTIONcompsoctrue} + +\DeclareOption{romanappendices}{\CLASSOPTIONromanappendicestrue} + + +% default to US letter paper, 10pt, twocolumn, one sided, final, journal +\ExecuteOptions{letterpaper,10pt,twocolumn,oneside,final,journal} +% overrride these defaults per user requests +\ProcessOptions + + + +% Computer Society conditional execution command +\long\def\@IEEEcompsoconly#1{\relax\ifCLASSOPTIONcompsoc\relax#1\relax\fi\relax} +% inverse +\long\def\@IEEEnotcompsoconly#1{\relax\ifCLASSOPTIONcompsoc\else\relax#1\relax\fi\relax} +% compsoc conference +\long\def\@IEEEcompsocconfonly#1{\relax\ifCLASSOPTIONcompsoc\ifCLASSOPTIONconference\relax#1\relax\fi\fi\relax} +% compsoc not conference +\long\def\@IEEEcompsocnotconfonly#1{\relax\ifCLASSOPTIONcompsoc\ifCLASSOPTIONconference\else\relax#1\relax\fi\fi\relax} + + +% IEEE uses Times Roman font, so we'll default to Times. +% These three commands make up the entire times.sty package. +\renewcommand{\sfdefault}{phv} +\renewcommand{\rmdefault}{ptm} +\renewcommand{\ttdefault}{pcr} + +\@IEEEcompsoconly{\typeout{-- Using IEEE Computer Society mode.}} + +% V1.7 compsoc nonconference papers, use Palatino/Palladio as the main text font, +% not Times Roman. +\@IEEEcompsocnotconfonly{\renewcommand{\rmdefault}{ppl}} + +% enable Times/Palatino main text font +\normalfont\selectfont + + + + + +% V1.7 conference notice message hook +\def\@IEEEconsolenoticeconference{\typeout{}% +\typeout{** Conference Paper **}% +\typeout{Before submitting the final camera ready copy, remember to:}% +\typeout{}% +\typeout{ 1. Manually equalize the lengths of two columns on the last page}% +\typeout{ of your paper;}% +\typeout{}% +\typeout{ 2. Ensure that any PostScript and/or PDF output post-processing}% +\typeout{ uses only Type 1 fonts and that every step in the generation}% +\typeout{ process uses the appropriate paper size.}% +\typeout{}} + + +% we can send console reminder messages to the user here +\AtEndDocument{\ifCLASSOPTIONconference\@IEEEconsolenoticeconference\fi} + + +% warn about the use of single column other than for draft mode +\ifCLASSOPTIONtwocolumn\else% + \ifCLASSOPTIONdraftcls\else% + \typeout{** ATTENTION: Single column mode is not typically used with IEEE publications.}% + \fi% +\fi + + +% V1.7 improved paper size setting code. +% Set pdfpage and dvips paper sizes. Conditional tests are similar to that +% of ifpdf.sty. Retain within {} to ensure tested macros are never altered, +% even if only effect is to set them to \relax. +% if \pdfoutput is undefined or equal to relax, output a dvips special +{\@ifundefined{pdfoutput}{\AtBeginDvi{\special{papersize=\CLASSINFOpaperwidth,\CLASSINFOpaperheight}}}{% +% pdfoutput is defined and not equal to \relax +% check for pdfpageheight existence just in case someone sets pdfoutput +% under non-pdflatex. If exists, set them regardless of value of \pdfoutput. +\@ifundefined{pdfpageheight}{\relax}{\global\pdfpagewidth\paperwidth +\global\pdfpageheight\paperheight}% +% if using \pdfoutput=0 under pdflatex, send dvips papersize special +\ifcase\pdfoutput +\AtBeginDvi{\special{papersize=\CLASSINFOpaperwidth,\CLASSINFOpaperheight}}% +\else +% we are using pdf output, set CLASSINFOpdf flag +\global\CLASSINFOpdftrue +\fi}} + +% let the user know the selected papersize +\typeout{-- Using \CLASSINFOpaperwidth\space x \CLASSINFOpaperheight\space +(\CLASSOPTIONpaper)\space paper.} + +\ifCLASSINFOpdf +\typeout{-- Using PDF output.} +\else +\typeout{-- Using DVI output.} +\fi + + +% The idea hinted here is for LaTeX to generate markleft{} and markright{} +% automatically for you after you enter \author{}, \journal{}, +% \journaldate{}, journalvol{}, \journalnum{}, etc. +% However, there may be some backward compatibility issues here as +% well as some special applications for IEEEtran.cls and special issues +% that may require the flexible \markleft{}, \markright{} and/or \markboth{}. +% We'll leave this as an open future suggestion. +%\newcommand{\journal}[1]{\def\@journal{#1}} +%\def\@journal{} + + + +% pointsize values +% used with ifx to determine the document's normal size +\def\@IEEEptsizenine{9} +\def\@IEEEptsizeten{10} +\def\@IEEEptsizeeleven{11} +\def\@IEEEptsizetwelve{12} + + + +% FONT DEFINITIONS (No sizexx.clo file needed) +% V1.6 revised font sizes, displayskip values and +% revised normalsize baselineskip to reduce underfull vbox problems +% on the 58pc = 696pt = 9.5in text height we want +% normalsize #lines/column baselineskip (aka leading) +% 9pt 63 11.0476pt (truncated down) +% 10pt 58 12pt (exact) +% 11pt 52 13.3846pt (truncated down) +% 12pt 50 13.92pt (exact) +% + +% we need to store the nominal baselineskip for the given font size +% in case baselinestretch ever changes. +% this is a dimen, so it will not hold stretch or shrink +\newdimen\@IEEEnormalsizeunitybaselineskip +\@IEEEnormalsizeunitybaselineskip\baselineskip + +\ifx\CLASSOPTIONpt\@IEEEptsizenine +\typeout{-- This is a 9 point document.} +\def\normalsize{\@setfontsize{\normalsize}{9}{11.0476pt}}% +\setlength{\@IEEEnormalsizeunitybaselineskip}{11.0476pt}% +\normalsize +\abovedisplayskip 1.5ex plus3pt minus1pt% +\belowdisplayskip \abovedisplayskip% +\abovedisplayshortskip 0pt plus3pt% +\belowdisplayshortskip 1.5ex plus3pt minus1pt +\def\small{\@setfontsize{\small}{8.5}{10pt}} +\def\footnotesize{\@setfontsize{\footnotesize}{8}{9pt}} +\def\scriptsize{\@setfontsize{\scriptsize}{7}{8pt}} +\def\tiny{\@setfontsize{\tiny}{5}{6pt}} +% sublargesize is the same as large - 10pt +\def\sublargesize{\@setfontsize{\sublargesize}{10}{12pt}} +\def\large{\@setfontsize{\large}{10}{12pt}} +\def\Large{\@setfontsize{\Large}{12}{14pt}} +\def\LARGE{\@setfontsize{\LARGE}{14}{17pt}} +\def\huge{\@setfontsize{\huge}{17}{20pt}} +\def\Huge{\@setfontsize{\Huge}{20}{24pt}} +\fi + + +% Check if we have selected 10 points +\ifx\CLASSOPTIONpt\@IEEEptsizeten +\typeout{-- This is a 10 point document.} +\def\normalsize{\@setfontsize{\normalsize}{10}{11}}% +\setlength{\@IEEEnormalsizeunitybaselineskip}{11pt}% +\normalsize +\abovedisplayskip 1.5ex plus4pt minus2pt% +\belowdisplayskip \abovedisplayskip% +\abovedisplayshortskip 0pt plus4pt% +\belowdisplayshortskip 1.5ex plus4pt minus2pt +\def\small{\@setfontsize{\small}{9}{10pt}} +\def\footnotesize{\@setfontsize{\footnotesize}{8}{9pt}} +\def\scriptsize{\@setfontsize{\scriptsize}{7}{8pt}} +\def\tiny{\@setfontsize{\tiny}{5}{6pt}} +% sublargesize is a tad smaller than large - 11pt +\def\sublargesize{\@setfontsize{\sublargesize}{11}{13.4pt}} +\def\large{\@setfontsize{\large}{12}{14pt}} +\def\Large{\@setfontsize{\Large}{14}{17pt}} +\def\LARGE{\@setfontsize{\LARGE}{17}{20pt}} +\def\huge{\@setfontsize{\huge}{20}{24pt}} +\def\Huge{\@setfontsize{\Huge}{24}{28pt}} +\fi + + +% Check if we have selected 11 points +\ifx\CLASSOPTIONpt\@IEEEptsizeeleven +\typeout{-- This is an 11 point document.} +\def\normalsize{\@setfontsize{\normalsize}{11}{13.3846pt}}% +\setlength{\@IEEEnormalsizeunitybaselineskip}{13.3846pt}% +\normalsize +\abovedisplayskip 1.5ex plus5pt minus3pt% +\belowdisplayskip \abovedisplayskip% +\abovedisplayshortskip 0pt plus5pt% +\belowdisplayshortskip 1.5ex plus5pt minus3pt +\def\small{\@setfontsize{\small}{10}{12pt}} +\def\footnotesize{\@setfontsize{\footnotesize}{9}{10.5pt}} +\def\scriptsize{\@setfontsize{\scriptsize}{8}{9pt}} +\def\tiny{\@setfontsize{\tiny}{6}{7pt}} +% sublargesize is the same as large - 12pt +\def\sublargesize{\@setfontsize{\sublargesize}{12}{14pt}} +\def\large{\@setfontsize{\large}{12}{14pt}} +\def\Large{\@setfontsize{\Large}{14}{17pt}} +\def\LARGE{\@setfontsize{\LARGE}{17}{20pt}} +\def\huge{\@setfontsize{\huge}{20}{24pt}} +\def\Huge{\@setfontsize{\Huge}{24}{28pt}} +\fi + + +% Check if we have selected 12 points +\ifx\CLASSOPTIONpt\@IEEEptsizetwelve +\typeout{-- This is a 12 point document.} +\def\normalsize{\@setfontsize{\normalsize}{12}{13.92pt}}% +\setlength{\@IEEEnormalsizeunitybaselineskip}{13.92pt}% +\normalsize +\abovedisplayskip 1.5ex plus6pt minus4pt% +\belowdisplayskip \abovedisplayskip% +\abovedisplayshortskip 0pt plus6pt% +\belowdisplayshortskip 1.5ex plus6pt minus4pt +\def\small{\@setfontsize{\small}{10}{12pt}} +\def\footnotesize{\@setfontsize{\footnotesize}{9}{10.5pt}} +\def\scriptsize{\@setfontsize{\scriptsize}{8}{9pt}} +\def\tiny{\@setfontsize{\tiny}{6}{7pt}} +% sublargesize is the same as large - 14pt +\def\sublargesize{\@setfontsize{\sublargesize}{14}{17pt}} +\def\large{\@setfontsize{\large}{14}{17pt}} +\def\Large{\@setfontsize{\Large}{17}{20pt}} +\def\LARGE{\@setfontsize{\LARGE}{20}{24pt}} +\def\huge{\@setfontsize{\huge}{22}{26pt}} +\def\Huge{\@setfontsize{\Huge}{24}{28pt}} +\fi + + +% V1.6 The Computer Modern Fonts will issue a substitution warning for +% 24pt titles (24.88pt is used instead) increase the substitution +% tolerance to turn off this warning +\def\fontsubfuzz{.9pt} +% However, the default (and correct) Times font will scale exactly as needed. + + +% warn the user in case they forget to use the 9pt option with +% technote +\ifCLASSOPTIONtechnote% + \ifx\CLASSOPTIONpt\@IEEEptsizenine\else% + \typeout{** ATTENTION: Technotes are normally 9pt documents.}% + \fi% +\fi + + +% V1.7 +% Improved \textunderscore to provide a much better fake _ when used with +% OT1 encoding. Under OT1, detect use of pcr or cmtt \ttfamily and use +% available true _ glyph for those two typewriter fonts. +\def\@IEEEstringptm{ptm} % Times Roman family +\def\@IEEEstringppl{ppl} % Palatino Roman family +\def\@IEEEstringphv{phv} % Helvetica Sans Serif family +\def\@IEEEstringpcr{pcr} % Courier typewriter family +\def\@IEEEstringcmtt{cmtt} % Computer Modern typewriter family +\DeclareTextCommandDefault{\textunderscore}{\leavevmode +\ifx\f@family\@IEEEstringpcr\string_\else +\ifx\f@family\@IEEEstringcmtt\string_\else +\ifx\f@family\@IEEEstringptm\kern 0em\vbox{\hrule\@width 0.5em\@height 0.5pt\kern -0.3ex}\else +\ifx\f@family\@IEEEstringppl\kern 0em\vbox{\hrule\@width 0.5em\@height 0.5pt\kern -0.3ex}\else +\ifx\f@family\@IEEEstringphv\kern -0.03em\vbox{\hrule\@width 0.62em\@height 0.52pt\kern -0.33ex}\kern -0.03em\else +\kern 0.09em\vbox{\hrule\@width 0.6em\@height 0.44pt\kern -0.63pt\kern -0.42ex}\kern 0.09em\fi\fi\fi\fi\fi\relax} + + + + +% set the default \baselinestretch +\def\baselinestretch{1} +\ifCLASSOPTIONdraftcls + \def\baselinestretch{1.5}% default baselinestretch for draft modes +\fi + + +% process CLASSINPUT baselinestretch +\ifx\CLASSINPUTbaselinestretch\@IEEEundefined +\else + \edef\baselinestretch{\CLASSINPUTbaselinestretch} % user CLASSINPUT override + \typeout{** ATTENTION: Overriding \string\baselinestretch\space to + \baselinestretch\space via \string\CLASSINPUT.} +\fi + +\normalsize % make \baselinestretch take affect + + + + +% store the normalsize baselineskip +\newdimen\CLASSINFOnormalsizebaselineskip +\CLASSINFOnormalsizebaselineskip=\baselineskip\relax +% and the normalsize unity (baselinestretch=1) baselineskip +% we could save a register by giving the user access to +% \@IEEEnormalsizeunitybaselineskip. However, let's protect +% its read only internal status +\newdimen\CLASSINFOnormalsizeunitybaselineskip +\CLASSINFOnormalsizeunitybaselineskip=\@IEEEnormalsizeunitybaselineskip\relax +% store the nominal value of jot +\newdimen\IEEEnormaljot +\IEEEnormaljot=0.25\baselineskip\relax + +% set \jot +\jot=\IEEEnormaljot\relax + + + + +% V1.6, we are now going to fine tune the interword spacing +% The default interword glue for Times under TeX appears to use a +% nominal interword spacing of 25% (relative to the font size, i.e., 1em) +% a maximum of 40% and a minimum of 19%. +% For example, 10pt text uses an interword glue of: +% +% 2.5pt plus 1.49998pt minus 0.59998pt +% +% However, IEEE allows for a more generous range which reduces the need +% for hyphenation, especially for two column text. Furthermore, IEEE +% tends to use a little bit more nominal space between the words. +% IEEE's interword spacing percentages appear to be: +% 35% nominal +% 23% minimum +% 50% maximum +% (They may even be using a tad more for the largest fonts such as 24pt.) +% +% for bold text, IEEE increases the spacing a little more: +% 37.5% nominal +% 23% minimum +% 55% maximum + +% here are the interword spacing ratios we'll use +% for medium (normal weight) +\def\@IEEEinterspaceratioM{0.35} +\def\@IEEEinterspaceMINratioM{0.23} +\def\@IEEEinterspaceMAXratioM{0.50} + +% for bold +\def\@IEEEinterspaceratioB{0.375} +\def\@IEEEinterspaceMINratioB{0.23} +\def\@IEEEinterspaceMAXratioB{0.55} + + +% command to revise the interword spacing for the current font under TeX: +% \fontdimen2 = nominal interword space +% \fontdimen3 = interword stretch +% \fontdimen4 = interword shrink +% since all changes to the \fontdimen are global, we can enclose these commands +% in braces to confine any font attribute or length changes +\def\@@@IEEEsetfontdimens#1#2#3{{% +\setlength{\@IEEEtrantmpdimenB}{\f@size pt}% grab the font size in pt, could use 1em instead. +\setlength{\@IEEEtrantmpdimenA}{#1\@IEEEtrantmpdimenB}% +\fontdimen2\font=\@IEEEtrantmpdimenA\relax +\addtolength{\@IEEEtrantmpdimenA}{-#2\@IEEEtrantmpdimenB}% +\fontdimen3\font=-\@IEEEtrantmpdimenA\relax +\setlength{\@IEEEtrantmpdimenA}{#1\@IEEEtrantmpdimenB}% +\addtolength{\@IEEEtrantmpdimenA}{-#3\@IEEEtrantmpdimenB}% +\fontdimen4\font=\@IEEEtrantmpdimenA\relax}} + +% revise the interword spacing for each font weight +\def\@@IEEEsetfontdimens{{% +\mdseries +\@@@IEEEsetfontdimens{\@IEEEinterspaceratioM}{\@IEEEinterspaceMAXratioM}{\@IEEEinterspaceMINratioM}% +\bfseries +\@@@IEEEsetfontdimens{\@IEEEinterspaceratioB}{\@IEEEinterspaceMAXratioB}{\@IEEEinterspaceMINratioB}% +}} + +% revise the interword spacing for each font shape +% \slshape is not often used for IEEE work and is not altered here. The \scshape caps are +% already a tad too large in the free LaTeX fonts (as compared to what IEEE uses) so we +% won't alter these either. +\def\@IEEEsetfontdimens{{% +\normalfont +\@@IEEEsetfontdimens +\normalfont\itshape +\@@IEEEsetfontdimens +}} + +% command to revise the interword spacing for each font size (and shape +% and weight). Only the \rmfamily is done here as \ttfamily uses a +% fixed spacing and \sffamily is not used as the main text of IEEE papers. +\def\@IEEEtunefonts{{\selectfont\rmfamily +\tiny\@IEEEsetfontdimens +\scriptsize\@IEEEsetfontdimens +\footnotesize\@IEEEsetfontdimens +\small\@IEEEsetfontdimens +\normalsize\@IEEEsetfontdimens +\sublargesize\@IEEEsetfontdimens +\large\@IEEEsetfontdimens +\LARGE\@IEEEsetfontdimens +\huge\@IEEEsetfontdimens +\Huge\@IEEEsetfontdimens}} + +% if the nofonttune class option is not given, revise the interword spacing +% now - in case IEEEtran makes any default length measurements, and make +% sure all the default fonts are loaded +\ifCLASSOPTIONnofonttune\else +\@IEEEtunefonts +\fi + +% and again at the start of the document in case the user loaded different fonts +\AtBeginDocument{\ifCLASSOPTIONnofonttune\else\@IEEEtunefonts\fi} + + + +% V1.6 +% LaTeX is a little to quick to use hyphenations +% So, we increase the penalty for their use and raise +% the badness level that triggers an underfull hbox +% warning. The author may still have to tweak things, +% but the appearance will be much better "right out +% of the box" than that under V1.5 and prior. +% TeX default is 50 +\hyphenpenalty=750 +% If we didn't adjust the interword spacing, 2200 might be better. +% The TeX default is 1000 +\hbadness=1350 +% IEEE does not use extra spacing after punctuation +\frenchspacing + +% V1.7 increase this a tad to discourage equation breaks +\binoppenalty=1000 % default 700 +\relpenalty=800 % default 500 + + +% margin note stuff +\marginparsep 10pt +\marginparwidth 20pt +\marginparpush 25pt + + +% if things get too close, go ahead and let them touch +\lineskip 0pt +\normallineskip 0pt +\lineskiplimit 0pt +\normallineskiplimit 0pt + +% The distance from the lower edge of the text body to the +% footline +\footskip 0.4in + +% normally zero, should be relative to font height. +% put in a little rubber to help stop some bad breaks (underfull vboxes) +\parskip 0ex plus 0.2ex minus 0.1ex +\ifCLASSOPTIONconference +\parskip 6pt plus 2pt minus 1pt +\fi + +\parindent 1.0em +\ifCLASSOPTIONconference +\parindent 14.45pt +\fi + +\topmargin -49.0pt +\headheight 12pt +\headsep 0.25in + +% use the normal font baselineskip +% so that \topskip is unaffected by changes in \baselinestretch +\topskip=\@IEEEnormalsizeunitybaselineskip +\textheight 58pc % 9.63in, 696pt +% Tweak textheight to a perfect integer number of lines/page. +% The normal baselineskip for each document point size is used +% to determine these values. +\ifx\CLASSOPTIONpt\@IEEEptsizenine\textheight=63\@IEEEnormalsizeunitybaselineskip\fi % 63 lines/page +\ifx\CLASSOPTIONpt\@IEEEptsizeten\textheight=58\@IEEEnormalsizeunitybaselineskip\fi % 58 lines/page +\ifx\CLASSOPTIONpt\@IEEEptsizeeleven\textheight=52\@IEEEnormalsizeunitybaselineskip\fi % 52 lines/page +\ifx\CLASSOPTIONpt\@IEEEptsizetwelve\textheight=50\@IEEEnormalsizeunitybaselineskip\fi % 50 lines/page + + +\columnsep 1.5pc +\textwidth 184.2mm + + +% the default side margins are equal +\if@IEEEusingAfourpaper +\oddsidemargin 14.32mm +\evensidemargin 14.32mm +\else +\oddsidemargin 0.680in +\evensidemargin 0.680in +\fi +% compensate for LaTeX's 1in offset +\addtolength{\oddsidemargin}{-1in} +\addtolength{\evensidemargin}{-1in} + + + +% adjust margins for conference mode +\ifCLASSOPTIONconference + \topmargin -0.25in + % we retain the reserved, but unused space for headers + \addtolength{\topmargin}{-\headheight} + \addtolength{\topmargin}{-\headsep} + \textheight 9.25in % The standard for conferences (668.4975pt) + % Tweak textheight to a perfect integer number of lines/page. + \ifx\CLASSOPTIONpt\@IEEEptsizenine\textheight=61\@IEEEnormalsizeunitybaselineskip\fi % 61 lines/page + \ifx\CLASSOPTIONpt\@IEEEptsizeten\textheight=62\@IEEEnormalsizeunitybaselineskip\fi % 62 lines/page + \ifx\CLASSOPTIONpt\@IEEEptsizeeleven\textheight=50\@IEEEnormalsizeunitybaselineskip\fi % 50 lines/page + \ifx\CLASSOPTIONpt\@IEEEptsizetwelve\textheight=48\@IEEEnormalsizeunitybaselineskip\fi % 48 lines/page +\fi + + +% compsoc conference +\ifCLASSOPTIONcompsoc +\ifCLASSOPTIONconference + % compsoc conference use a larger value for columnsep + \columnsep 0.375in + % compsoc conferences want 1in top margin, 1.125in bottom margin + \topmargin 0in + \addtolength{\topmargin}{-6pt}% we tweak this a tad to better comply with top of line stuff + % we retain the reserved, but unused space for headers + \addtolength{\topmargin}{-\headheight} + \addtolength{\topmargin}{-\headsep} + \textheight 8.875in % (641.39625pt) + % Tweak textheight to a perfect integer number of lines/page. + \ifx\CLASSOPTIONpt\@IEEEptsizenine\textheight=58\@IEEEnormalsizeunitybaselineskip\fi % 58 lines/page + \ifx\CLASSOPTIONpt\@IEEEptsizeten\textheight=53\@IEEEnormalsizeunitybaselineskip\fi % 53 lines/page + \ifx\CLASSOPTIONpt\@IEEEptsizeeleven\textheight=48\@IEEEnormalsizeunitybaselineskip\fi % 48 lines/page + \ifx\CLASSOPTIONpt\@IEEEptsizetwelve\textheight=46\@IEEEnormalsizeunitybaselineskip\fi % 46 lines/page + \textwidth 6.5in + % the default side margins are equal + \if@IEEEusingAfourpaper + \oddsidemargin 22.45mm + \evensidemargin 22.45mm + \else + \oddsidemargin 1in + \evensidemargin 1in + \fi + % compensate for LaTeX's 1in offset + \addtolength{\oddsidemargin}{-1in} + \addtolength{\evensidemargin}{-1in} +\fi\fi + + + +% draft mode settings override that of all other modes +% provides a nice 1in margin all around the paper and extra +% space between the lines for editor's comments +\ifCLASSOPTIONdraftcls + % want 1in from top of paper to text + \setlength{\topmargin}{-\headsep}% + \addtolength{\topmargin}{-\headheight}% + % we want 1in side margins regardless of paper type + \oddsidemargin 0in + \evensidemargin 0in + % set the text width + \setlength{\textwidth}{\paperwidth}% + \addtolength{\textwidth}{-2.0in}% + \setlength{\textheight}{\paperheight}% + \addtolength{\textheight}{-2.0in}% + % digitize textheight to be an integer number of lines. + % this may cause the bottom margin to be off a tad + \addtolength{\textheight}{-1\topskip}% + \divide\textheight by \baselineskip% + \multiply\textheight by \baselineskip% + \addtolength{\textheight}{\topskip}% +\fi + + + +% process CLASSINPUT inner/outer margin +% if inner margin defined, but outer margin not, set outer to inner. +\ifx\CLASSINPUTinnersidemargin\@IEEEundefined +\else + \ifx\CLASSINPUToutersidemargin\@IEEEundefined + \edef\CLASSINPUToutersidemargin{\CLASSINPUTinnersidemargin} + \fi +\fi + +\ifx\CLASSINPUToutersidemargin\@IEEEundefined +\else + % if outer margin defined, but inner margin not, set inner to outer. + \ifx\CLASSINPUTinnersidemargin\@IEEEundefined + \edef\CLASSINPUTinnersidemargin{\CLASSINPUToutersidemargin} + \fi + \setlength{\oddsidemargin}{\CLASSINPUTinnersidemargin} + \ifCLASSOPTIONtwoside + \setlength{\evensidemargin}{\CLASSINPUToutersidemargin} + \else + \setlength{\evensidemargin}{\CLASSINPUTinnersidemargin} + \fi + \addtolength{\oddsidemargin}{-1in} + \addtolength{\evensidemargin}{-1in} + \setlength{\textwidth}{\paperwidth} + \addtolength{\textwidth}{-\CLASSINPUTinnersidemargin} + \addtolength{\textwidth}{-\CLASSINPUToutersidemargin} + \typeout{** ATTENTION: Overriding inner side margin to \CLASSINPUTinnersidemargin\space and + outer side margin to \CLASSINPUToutersidemargin\space via \string\CLASSINPUT.} +\fi + + + +% process CLASSINPUT top/bottom text margin +% if toptext margin defined, but bottomtext margin not, set bottomtext to toptext margin +\ifx\CLASSINPUTtoptextmargin\@IEEEundefined +\else + \ifx\CLASSINPUTbottomtextmargin\@IEEEundefined + \edef\CLASSINPUTbottomtextmargin{\CLASSINPUTtoptextmargin} + \fi +\fi + +\ifx\CLASSINPUTbottomtextmargin\@IEEEundefined +\else + % if bottomtext margin defined, but toptext margin not, set toptext to bottomtext margin + \ifx\CLASSINPUTtoptextmargin\@IEEEundefined + \edef\CLASSINPUTtoptextmargin{\CLASSINPUTbottomtextmargin} + \fi + \setlength{\topmargin}{\CLASSINPUTtoptextmargin} + \addtolength{\topmargin}{-1in} + \addtolength{\topmargin}{-\headheight} + \addtolength{\topmargin}{-\headsep} + \setlength{\textheight}{\paperheight} + \addtolength{\textheight}{-\CLASSINPUTtoptextmargin} + \addtolength{\textheight}{-\CLASSINPUTbottomtextmargin} + % in the default format we use the normal baselineskip as topskip + % we only need 0.7 of this to clear typical top text and we need + % an extra 0.3 spacing at the bottom for descenders. This will + % correct for both. + \addtolength{\topmargin}{-0.3\@IEEEnormalsizeunitybaselineskip} + \typeout{** ATTENTION: Overriding top text margin to \CLASSINPUTtoptextmargin\space and + bottom text margin to \CLASSINPUTbottomtextmargin\space via \string\CLASSINPUT.} +\fi + + + + + + + +% LIST SPACING CONTROLS + +% Controls the amount of EXTRA spacing +% above and below \trivlist +% Both \list and IED lists override this. +% However, \trivlist will use this as will most +% things built from \trivlist like the \center +% environment. +\topsep 0.5\baselineskip + +% Controls the additional spacing around lists preceded +% or followed by blank lines. IEEE does not increase +% spacing before or after paragraphs so it is set to zero. +% \z@ is the same as zero, but faster. +\partopsep \z@ + +% Controls the spacing between paragraphs in lists. +% IEEE does not increase spacing before or after paragraphs +% so this is also zero. +% With IEEEtran.cls, global changes to +% this value DO affect lists (but not IED lists). +\parsep \z@ + +% Controls the extra spacing between list items. +% IEEE does not put extra spacing between items. +% With IEEEtran.cls, global changes to this value DO affect +% lists (but not IED lists). +\itemsep \z@ + +% \itemindent is the amount to indent the FIRST line of a list +% item. It is auto set to zero within the \list environment. To alter +% it, you have to do so when you call the \list. +% However, IEEE uses this for the theorem environment +% There is an alternative value for this near \leftmargini below +\itemindent -1em + +% \leftmargin, the spacing from the left margin of the main text to +% the left of the main body of a list item is set by \list. +% Hence this statement does nothing for lists. +% But, quote and verse do use it for indention. +\leftmargin 2em + +% we retain this stuff from the older IEEEtran.cls so that \list +% will work the same way as before. However, itemize, enumerate and +% description (IED) could care less about what these are as they +% all are overridden. +\leftmargini 2em +%\itemindent 2em % Alternative values: sometimes used. +%\leftmargini 0em +\leftmarginii 1em +\leftmarginiii 1.5em +\leftmarginiv 1.5em +\leftmarginv 1.0em +\leftmarginvi 1.0em +\labelsep 0.5em +\labelwidth \z@ + + +% The old IEEEtran.cls behavior of \list is retained. +% However, the new V1.3 IED list environments override all the +% @list stuff (\@listX is called within \list for the +% appropriate level just before the user's list_decl is called). +% \topsep is now 2pt as IEEE puts a little extra space around +% lists - used by those non-IED macros that depend on \list. +% Note that \parsep and \itemsep are not redefined as in +% the sizexx.clo \@listX (which article.cls uses) so global changes +% of these values DO affect \list +% +\def\@listi{\leftmargin\leftmargini \topsep 2pt plus 1pt minus 1pt} +\let\@listI\@listi +\def\@listii{\leftmargin\leftmarginii\labelwidth\leftmarginii% + \advance\labelwidth-\labelsep \topsep 2pt} +\def\@listiii{\leftmargin\leftmarginiii\labelwidth\leftmarginiii% + \advance\labelwidth-\labelsep \topsep 2pt} +\def\@listiv{\leftmargin\leftmarginiv\labelwidth\leftmarginiv% + \advance\labelwidth-\labelsep \topsep 2pt} +\def\@listv{\leftmargin\leftmarginv\labelwidth\leftmarginv% + \advance\labelwidth-\labelsep \topsep 2pt} +\def\@listvi{\leftmargin\leftmarginvi\labelwidth\leftmarginvi% + \advance\labelwidth-\labelsep \topsep 2pt} + + +% IEEE uses 5) not 5. +\def\labelenumi{\theenumi)} \def\theenumi{\arabic{enumi}} + +% IEEE uses a) not (a) +\def\labelenumii{\theenumii)} \def\theenumii{\alph{enumii}} + +% IEEE uses iii) not iii. +\def\labelenumiii{\theenumiii)} \def\theenumiii{\roman{enumiii}} + +% IEEE uses A) not A. +\def\labelenumiv{\theenumiv)} \def\theenumiv{\Alph{enumiv}} + +% exactly the same as in article.cls +\def\p@enumii{\theenumi} +\def\p@enumiii{\theenumi(\theenumii)} +\def\p@enumiv{\p@enumiii\theenumiii} + +% itemized list label styles +\def\labelitemi{$\bullet$} +\def\labelitemii{$\circ$} +\def\labelitemiii{\vrule height 0.8ex depth -0.2ex width 0.6ex} +\def\labelitemiv{$\ast$} + + + +% **** V1.3 ENHANCEMENTS **** +% Itemize, Enumerate and Description (IED) List Controls +% *************************** +% +% +% IEEE seems to use at least two different values by +% which ITEMIZED list labels are indented to the right +% For The Journal of Lightwave Technology (JLT) and The Journal +% on Selected Areas in Communications (JSAC), they tend to use +% an indention equal to \parindent. For Transactions on Communications +% they tend to indent ITEMIZED lists a little more--- 1.3\parindent. +% We'll provide both values here for you so that you can choose +% which one you like in your document using a command such as: +% setlength{\IEEEilabelindent}{\IEEEilabelindentB} +\newdimen\IEEEilabelindentA +\IEEEilabelindentA \parindent + +\newdimen\IEEEilabelindentB +\IEEEilabelindentB 1.3\parindent +% However, we'll default to using \parindent +% which makes more sense to me +\newdimen\IEEEilabelindent +\IEEEilabelindent \IEEEilabelindentA + + +% This controls the default amount the enumerated list labels +% are indented to the right. +% Normally, this is the same as the paragraph indention +\newdimen\IEEEelabelindent +\IEEEelabelindent \parindent + +% This controls the default amount the description list labels +% are indented to the right. +% Normally, this is the same as the paragraph indention +\newdimen\IEEEdlabelindent +\IEEEdlabelindent \parindent + +% This is the value actually used within the IED lists. +% The IED environments automatically set its value to +% one of the three values above, so global changes do +% not have any effect +\newdimen\IEEElabelindent +\IEEElabelindent \parindent + +% The actual amount labels will be indented is +% \IEEElabelindent multiplied by the factor below +% corresponding to the level of nesting depth +% This provides a means by which the user can +% alter the effective \IEEElabelindent for deeper +% levels +% There may not be such a thing as correct "standard IEEE" +% values. What IEEE actually does may depend on the specific +% circumstances. +% The first list level almost always has full indention. +% The second levels I've seen have only 75% of the normal indentation +% Three level or greater nestings are very rare. I am guessing +% that they don't use any indentation. +\def\IEEElabelindentfactori{1.0} % almost always one +\def\IEEElabelindentfactorii{0.75} % 0.0 or 1.0 may be used in some cases +\def\IEEElabelindentfactoriii{0.0} % 0.75? 0.5? 0.0? +\def\IEEElabelindentfactoriv{0.0} +\def\IEEElabelindentfactorv{0.0} +\def\IEEElabelindentfactorvi{0.0} + +% value actually used within IED lists, it is auto +% set to one of the 6 values above +% global changes here have no effect +\def\IEEElabelindentfactor{1.0} + +% This controls the default spacing between the end of the IED +% list labels and the list text, when normal text is used for +% the labels. +\newdimen\IEEEiednormlabelsep +\IEEEiednormlabelsep \parindent + +% This controls the default spacing between the end of the IED +% list labels and the list text, when math symbols are used for +% the labels (nomenclature lists). IEEE usually increases the +% spacing in these cases +\newdimen\IEEEiedmathlabelsep +\IEEEiedmathlabelsep 1.2em + +% This controls the extra vertical separation put above and +% below each IED list. IEEE usually puts a little extra spacing +% around each list. However, this spacing is barely noticeable. +\newskip\IEEEiedtopsep +\IEEEiedtopsep 2pt plus 1pt minus 1pt + + +% This command is executed within each IED list environment +% at the beginning of the list. You can use this to set the +% parameters for some/all your IED list(s) without disturbing +% global parameters that affect things other than lists. +% i.e., renewcommand{\IEEEiedlistdecl}{\setlength{\labelsep}{5em}} +% will alter the \labelsep for the next list(s) until +% \IEEEiedlistdecl is redefined. +\def\IEEEiedlistdecl{\relax} + +% This command provides an easy way to set \leftmargin based +% on the \labelwidth, \labelsep and the argument \IEEElabelindent +% Usage: \IEEEcalcleftmargin{width-to-indent-the-label} +% output is in the \leftmargin variable, i.e., effectively: +% \leftmargin = argument + \labelwidth + \labelsep +% Note controlled spacing here, shield end of lines with % +\def\IEEEcalcleftmargin#1{\setlength{\leftmargin}{#1}% +\addtolength{\leftmargin}{\labelwidth}% +\addtolength{\leftmargin}{\labelsep}} + +% This command provides an easy way to set \labelwidth to the +% width of the given text. It is the same as +% \settowidth{\labelwidth}{label-text} +% and useful as a shorter alternative. +% Typically used to set \labelwidth to be the width +% of the longest label in the list +\def\IEEEsetlabelwidth#1{\settowidth{\labelwidth}{#1}} + +% When this command is executed, IED lists will use the +% IEEEiedmathlabelsep label separation rather than the normal +% spacing. To have an effect, this command must be executed via +% the \IEEEiedlistdecl or within the option of the IED list +% environments. +\def\IEEEusemathlabelsep{\setlength{\labelsep}{\IEEEiedmathlabelsep}} + +% A flag which controls whether the IED lists automatically +% calculate \leftmargin from \IEEElabelindent, \labelwidth and \labelsep +% Useful if you want to specify your own \leftmargin +% This flag must be set (\IEEEnocalcleftmargintrue or \IEEEnocalcleftmarginfalse) +% via the \IEEEiedlistdecl or within the option of the IED list +% environments to have an effect. +\newif\ifIEEEnocalcleftmargin +\IEEEnocalcleftmarginfalse + +% A flag which controls whether \IEEElabelindent is multiplied by +% the \IEEElabelindentfactor for each list level. +% This flag must be set via the \IEEEiedlistdecl or within the option +% of the IED list environments to have an effect. +\newif\ifIEEEnolabelindentfactor +\IEEEnolabelindentfactorfalse + + +% internal variable to indicate type of IED label +% justification +% 0 - left; 1 - center; 2 - right +\def\@IEEEiedjustify{0} + + +% commands to allow the user to control IED +% label justifications. Use these commands within +% the IED environment option or in the \IEEEiedlistdecl +% Note that changing the normal list justifications +% is nonstandard and IEEE may not like it if you do so! +% I include these commands as they may be helpful to +% those who are using these enhanced list controls for +% other non-IEEE related LaTeX work. +% itemize and enumerate automatically default to right +% justification, description defaults to left. +\def\IEEEiedlabeljustifyl{\def\@IEEEiedjustify{0}}%left +\def\IEEEiedlabeljustifyc{\def\@IEEEiedjustify{1}}%center +\def\IEEEiedlabeljustifyr{\def\@IEEEiedjustify{2}}%right + + + + +% commands to save to and restore from the list parameter copies +% this allows us to set all the list parameters within +% the list_decl and prevent \list (and its \@list) +% from overriding any of our parameters +% V1.6 use \edefs instead of dimen's to conserve dimen registers +% Note controlled spacing here, shield end of lines with % +\def\@IEEEsavelistparams{\edef\@IEEEiedtopsep{\the\topsep}% +\edef\@IEEEiedlabelwidth{\the\labelwidth}% +\edef\@IEEEiedlabelsep{\the\labelsep}% +\edef\@IEEEiedleftmargin{\the\leftmargin}% +\edef\@IEEEiedpartopsep{\the\partopsep}% +\edef\@IEEEiedparsep{\the\parsep}% +\edef\@IEEEieditemsep{\the\itemsep}% +\edef\@IEEEiedrightmargin{\the\rightmargin}% +\edef\@IEEEiedlistparindent{\the\listparindent}% +\edef\@IEEEieditemindent{\the\itemindent}} + +% Note controlled spacing here +\def\@IEEErestorelistparams{\topsep\@IEEEiedtopsep\relax% +\labelwidth\@IEEEiedlabelwidth\relax% +\labelsep\@IEEEiedlabelsep\relax% +\leftmargin\@IEEEiedleftmargin\relax% +\partopsep\@IEEEiedpartopsep\relax% +\parsep\@IEEEiedparsep\relax% +\itemsep\@IEEEieditemsep\relax% +\rightmargin\@IEEEiedrightmargin\relax% +\listparindent\@IEEEiedlistparindent\relax% +\itemindent\@IEEEieditemindent\relax} + + +% v1.6b provide original LaTeX IED list environments +% note that latex.ltx defines \itemize and \enumerate, but not \description +% which must be created by the base classes +% save original LaTeX itemize and enumerate +\let\LaTeXitemize\itemize +\let\endLaTeXitemize\enditemize +\let\LaTeXenumerate\enumerate +\let\endLaTeXenumerate\endenumerate + +% provide original LaTeX description environment from article.cls +\newenvironment{LaTeXdescription} + {\list{}{\labelwidth\z@ \itemindent-\leftmargin + \let\makelabel\descriptionlabel}} + {\endlist} +\newcommand*\descriptionlabel[1]{\hspace\labelsep + \normalfont\bfseries #1} + + +% override LaTeX's default IED lists +\def\itemize{\@IEEEitemize} +\def\enditemize{\@endIEEEitemize} +\def\enumerate{\@IEEEenumerate} +\def\endenumerate{\@endIEEEenumerate} +\def\description{\@IEEEdescription} +\def\enddescription{\@endIEEEdescription} + +% provide the user with aliases - may help those using packages that +% override itemize, enumerate, or description +\def\IEEEitemize{\@IEEEitemize} +\def\endIEEEitemize{\@endIEEEitemize} +\def\IEEEenumerate{\@IEEEenumerate} +\def\endIEEEenumerate{\@endIEEEenumerate} +\def\IEEEdescription{\@IEEEdescription} +\def\endIEEEdescription{\@endIEEEdescription} + + +% V1.6 we want to keep the IEEEtran IED list definitions as our own internal +% commands so they are protected against redefinition +\def\@IEEEitemize{\@ifnextchar[{\@@IEEEitemize}{\@@IEEEitemize[\relax]}} +\def\@IEEEenumerate{\@ifnextchar[{\@@IEEEenumerate}{\@@IEEEenumerate[\relax]}} +\def\@IEEEdescription{\@ifnextchar[{\@@IEEEdescription}{\@@IEEEdescription[\relax]}} +\def\@endIEEEitemize{\endlist} +\def\@endIEEEenumerate{\endlist} +\def\@endIEEEdescription{\endlist} + + +% DO NOT ALLOW BLANK LINES TO BE IN THESE IED ENVIRONMENTS +% AS THIS WILL FORCE NEW PARAGRAPHS AFTER THE IED LISTS +% IEEEtran itemized list MDS 1/2001 +% Note controlled spacing here, shield end of lines with % +\def\@@IEEEitemize[#1]{% + \ifnum\@itemdepth>3\relax\@toodeep\else% + \ifnum\@listdepth>5\relax\@toodeep\else% + \advance\@itemdepth\@ne% + \edef\@itemitem{labelitem\romannumeral\the\@itemdepth}% + % get the labelindentfactor for this level + \advance\@listdepth\@ne% we need to know what the level WILL be + \edef\IEEElabelindentfactor{\csname IEEElabelindentfactor\romannumeral\the\@listdepth\endcsname}% + \advance\@listdepth-\@ne% undo our increment + \def\@IEEEiedjustify{2}% right justified labels are default + % set other defaults + \IEEEnocalcleftmarginfalse% + \IEEEnolabelindentfactorfalse% + \topsep\IEEEiedtopsep% + \IEEElabelindent\IEEEilabelindent% + \labelsep\IEEEiednormlabelsep% + \partopsep 0ex% + \parsep 0ex% + \itemsep \parskip% + \rightmargin 0em% + \listparindent 0em% + \itemindent 0em% + % calculate the label width + % the user can override this later if + % they specified a \labelwidth + \settowidth{\labelwidth}{\csname labelitem\romannumeral\the\@itemdepth\endcsname}% + \@IEEEsavelistparams% save our list parameters + \list{\csname\@itemitem\endcsname}{% + \@IEEErestorelistparams% override any list{} changes + % to our globals + \let\makelabel\@IEEEiedmakelabel% v1.6b setup \makelabel + \IEEEiedlistdecl% let user alter parameters + #1\relax% + % If the user has requested not to use the + % labelindent factor, don't revise \labelindent + \ifIEEEnolabelindentfactor\relax% + \else\IEEElabelindent=\IEEElabelindentfactor\labelindent% + \fi% + % Unless the user has requested otherwise, + % calculate our left margin based + % on \IEEElabelindent, \labelwidth and + % \labelsep + \ifIEEEnocalcleftmargin\relax% + \else\IEEEcalcleftmargin{\IEEElabelindent}% + \fi}\fi\fi}% + + +% DO NOT ALLOW BLANK LINES TO BE IN THESE IED ENVIRONMENTS +% AS THIS WILL FORCE NEW PARAGRAPHS AFTER THE IED LISTS +% IEEEtran enumerate list MDS 1/2001 +% Note controlled spacing here, shield end of lines with % +\def\@@IEEEenumerate[#1]{% + \ifnum\@enumdepth>3\relax\@toodeep\else% + \ifnum\@listdepth>5\relax\@toodeep\else% + \advance\@enumdepth\@ne% + \edef\@enumctr{enum\romannumeral\the\@enumdepth}% + % get the labelindentfactor for this level + \advance\@listdepth\@ne% we need to know what the level WILL be + \edef\IEEElabelindentfactor{\csname IEEElabelindentfactor\romannumeral\the\@listdepth\endcsname}% + \advance\@listdepth-\@ne% undo our increment + \def\@IEEEiedjustify{2}% right justified labels are default + % set other defaults + \IEEEnocalcleftmarginfalse% + \IEEEnolabelindentfactorfalse% + \topsep\IEEEiedtopsep% + \IEEElabelindent\IEEEelabelindent% + \labelsep\IEEEiednormlabelsep% + \partopsep 0ex% + \parsep 0ex% + \itemsep 0ex% + \rightmargin 0em% + \listparindent 0em% + \itemindent 0em% + % calculate the label width + % We'll set it to the width suitable for all labels using + % normalfont 1) to 9) + % The user can override this later + \settowidth{\labelwidth}{9)}% + \@IEEEsavelistparams% save our list parameters + \list{\csname label\@enumctr\endcsname}{\usecounter{\@enumctr}% + \@IEEErestorelistparams% override any list{} changes + % to our globals + \let\makelabel\@IEEEiedmakelabel% v1.6b setup \makelabel + \IEEEiedlistdecl% let user alter parameters + #1\relax% + % If the user has requested not to use the + % IEEElabelindent factor, don't revise \IEEElabelindent + \ifIEEEnolabelindentfactor\relax% + \else\IEEElabelindent=\IEEElabelindentfactor\IEEElabelindent% + \fi% + % Unless the user has requested otherwise, + % calculate our left margin based + % on \IEEElabelindent, \labelwidth and + % \labelsep + \ifIEEEnocalcleftmargin\relax% + \else\IEEEcalcleftmargin{\IEEElabelindent}% + \fi}\fi\fi}% + + +% DO NOT ALLOW BLANK LINES TO BE IN THESE IED ENVIRONMENTS +% AS THIS WILL FORCE NEW PARAGRAPHS AFTER THE IED LISTS +% IEEEtran description list MDS 1/2001 +% Note controlled spacing here, shield end of lines with % +\def\@@IEEEdescription[#1]{% + \ifnum\@listdepth>5\relax\@toodeep\else% + % get the labelindentfactor for this level + \advance\@listdepth\@ne% we need to know what the level WILL be + \edef\IEEElabelindentfactor{\csname IEEElabelindentfactor\romannumeral\the\@listdepth\endcsname}% + \advance\@listdepth-\@ne% undo our increment + \def\@IEEEiedjustify{0}% left justified labels are default + % set other defaults + \IEEEnocalcleftmarginfalse% + \IEEEnolabelindentfactorfalse% + \topsep\IEEEiedtopsep% + \IEEElabelindent\IEEEdlabelindent% + % assume normal labelsep + \labelsep\IEEEiednormlabelsep% + \partopsep 0ex% + \parsep 0ex% + \itemsep 0ex% + \rightmargin 0em% + \listparindent 0em% + \itemindent 0em% + % Bogus label width in case the user forgets + % to set it. + % TIP: If you want to see what a variable's width is you + % can use the TeX command \showthe\width-variable to + % display it on the screen during compilation + % (This might be helpful to know when you need to find out + % which label is the widest) + \settowidth{\labelwidth}{Hello}% + \@IEEEsavelistparams% save our list parameters + \list{}{\@IEEErestorelistparams% override any list{} changes + % to our globals + \let\makelabel\@IEEEiedmakelabel% v1.6b setup \makelabel + \IEEEiedlistdecl% let user alter parameters + #1\relax% + % If the user has requested not to use the + % labelindent factor, don't revise \IEEElabelindent + \ifIEEEnolabelindentfactor\relax% + \else\IEEElabelindent=\IEEElabelindentfactor\IEEElabelindent% + \fi% + % Unless the user has requested otherwise, + % calculate our left margin based + % on \IEEElabelindent, \labelwidth and + % \labelsep + \ifIEEEnocalcleftmargin\relax% + \else\IEEEcalcleftmargin{\IEEElabelindent}\relax% + \fi}\fi} + +% v1.6b we use one makelabel that does justification as needed. +\def\@IEEEiedmakelabel#1{\relax\if\@IEEEiedjustify 0\relax +\makebox[\labelwidth][l]{\normalfont #1}\else +\if\@IEEEiedjustify 1\relax +\makebox[\labelwidth][c]{\normalfont #1}\else +\makebox[\labelwidth][r]{\normalfont #1}\fi\fi} + + +% VERSE and QUOTE +% V1.7 define environments with newenvironment +\newenvironment{verse}{\let\\=\@centercr + \list{}{\itemsep\z@ \itemindent -1.5em \listparindent \itemindent + \rightmargin\leftmargin\advance\leftmargin 1.5em}\item\relax} + {\endlist} +\newenvironment{quotation}{\list{}{\listparindent 1.5em \itemindent\listparindent + \rightmargin\leftmargin \parsep 0pt plus 1pt}\item\relax} + {\endlist} +\newenvironment{quote}{\list{}{\rightmargin\leftmargin}\item\relax} + {\endlist} + + +% \titlepage +% provided only for backward compatibility. \maketitle is the correct +% way to create the title page. +\newif\if@restonecol +\def\titlepage{\@restonecolfalse\if@twocolumn\@restonecoltrue\onecolumn + \else \newpage \fi \thispagestyle{empty}\c@page\z@} +\def\endtitlepage{\if@restonecol\twocolumn \else \newpage \fi} + +% standard values from article.cls +\arraycolsep 5pt +\arrayrulewidth .4pt +\doublerulesep 2pt + +\tabcolsep 6pt +\tabbingsep 0.5em + + +%% FOOTNOTES +% +%\skip\footins 10pt plus 4pt minus 2pt +% V1.6 respond to changes in font size +% space added above the footnotes (if present) +\skip\footins 0.9\baselineskip plus 0.4\baselineskip minus 0.2\baselineskip + +% V1.6, we need to make \footnotesep responsive to changes +% in \baselineskip or strange spacings will result when in +% draft mode. Here is a little LaTeX secret - \footnotesep +% determines the height of an invisible strut that is placed +% *above* the baseline of footnotes after the first. Since +% LaTeX considers the space for characters to be 0.7/baselineskip +% above the baseline and 0.3/baselineskip below it, we need to +% use 0.7/baselineskip as a \footnotesep to maintain equal spacing +% between all the lines of the footnotes. IEEE often uses a tad +% more, so use 0.8\baselineskip. This slightly larger value also helps +% the text to clear the footnote marks. Note that \thanks in IEEEtran +% uses its own value of \footnotesep which is set in \maketitle. +{\footnotesize +\global\footnotesep 0.8\baselineskip} + +\def\unnumberedfootnote{\gdef\@thefnmark{\quad}\@footnotetext} + +\skip\@mpfootins 0.3\baselineskip +\fboxsep = 3pt +\fboxrule = .4pt +% V1.6 use 1em, then use LaTeX2e's \@makefnmark +% Note that IEEE normally *left* aligns the footnote marks, so we don't need +% box resizing tricks here. +%\long\def\@makefnmark{\scriptsize\normalfont\@thefnmark} +\long\def\@makefntext#1{\parindent 1em\indent\hbox{\@makefnmark}#1}% V1.6 use 1em +\long\def\@maketablefntext#1{\raggedleft\leavevmode\hbox{\@makefnmark}#1} +% V1.7 compsoc does not use superscipts for footnote marks +\ifCLASSOPTIONcompsoc +\def\@IEEEcompsocmakefnmark{\hbox{\normalfont\@thefnmark.\ }} +\long\def\@makefntext#1{\parindent 1em\indent\hbox{\@IEEEcompsocmakefnmark}#1} +\fi + +% IEEE does not use footnote rules. Or do they? +\def\footnoterule{\vskip-2pt \hrule height 0.6pt depth \z@ \vskip1.6pt\relax} +\toks@\expandafter{\@setminipage\let\footnoterule\relax\footnotesep\z@} +\edef\@setminipage{\the\toks@} + +% V1.7 for compsoc, IEEE uses a footnote rule only for \thanks. We devise a "one-shot" +% system to implement this. +\newif\if@IEEEenableoneshotfootnoterule +\@IEEEenableoneshotfootnoterulefalse +\ifCLASSOPTIONcompsoc +\def\footnoterule{\relax\if@IEEEenableoneshotfootnoterule +\kern-5pt +\hbox to \columnwidth{\hfill\vrule width 0.5\columnwidth height 0.4pt\hfill} +\kern4.6pt +\global\@IEEEenableoneshotfootnoterulefalse +\else +\relax +\fi} +\fi + +% V1.6 do not allow LaTeX to break a footnote across multiple pages +\interfootnotelinepenalty=10000 + +% V1.6 discourage breaks within equations +% Note that amsmath normally sets this to 10000, +% but LaTeX2e normally uses 100. +\interdisplaylinepenalty=2500 + +% default allows section depth up to /paragraph +\setcounter{secnumdepth}{4} + +% technotes do not allow /paragraph +\ifCLASSOPTIONtechnote + \setcounter{secnumdepth}{3} +\fi +% neither do compsoc conferences +\@IEEEcompsocconfonly{\setcounter{secnumdepth}{3}} + + +\newcounter{section} +\newcounter{subsection}[section] +\newcounter{subsubsection}[subsection] +\newcounter{paragraph}[subsubsection] + +% used only by IEEEtran's IEEEeqnarray as other packages may +% have their own, different, implementations +\newcounter{IEEEsubequation}[equation] + +% as shown when called by user from \ref, \label and in table of contents +\def\theequation{\arabic{equation}} % 1 +\def\theIEEEsubequation{\theequation\alph{IEEEsubequation}} % 1a (used only by IEEEtran's IEEEeqnarray) +\ifCLASSOPTIONcompsoc +% compsoc is all arabic +\def\thesection{\arabic{section}} +\def\thesubsection{\thesection.\arabic{subsection}} +\def\thesubsubsection{\thesubsection.\arabic{subsubsection}} +\def\theparagraph{\thesubsubsection.\arabic{paragraph}} +\else +\def\thesection{\Roman{section}} % I +% V1.7, \mbox prevents breaks around - +\def\thesubsection{\mbox{\thesection-\Alph{subsection}}} % I-A +% V1.7 use I-A1 format used by IEEE rather than I-A.1 +\def\thesubsubsection{\thesubsection\arabic{subsubsection}} % I-A1 +\def\theparagraph{\thesubsubsection\alph{paragraph}} % I-A1a +\fi + +% From Heiko Oberdiek. Because of the \mbox in \thesubsection, we need to +% tell hyperref to disable the \mbox command when making PDF bookmarks. +% This done already with hyperref.sty version 6.74o and later, but +% it will not hurt to do it here again for users of older versions. +\@ifundefined{pdfstringdefPreHook}{\let\pdfstringdefPreHook\@empty}{}% +\g@addto@macro\pdfstringdefPreHook{\let\mbox\relax} + + +% Main text forms (how shown in main text headings) +% V1.6, using \thesection in \thesectiondis allows changes +% in the former to automatically appear in the latter +\ifCLASSOPTIONcompsoc + \ifCLASSOPTIONconference% compsoc conference + \def\thesectiondis{\thesection.} + \def\thesubsectiondis{\thesectiondis\arabic{subsection}.} + \def\thesubsubsectiondis{\thesubsectiondis\arabic{subsubsection}.} + \def\theparagraphdis{\thesubsubsectiondis\arabic{paragraph}.} + \else% compsoc not conferencs + \def\thesectiondis{\thesection} + \def\thesubsectiondis{\thesectiondis.\arabic{subsection}} + \def\thesubsubsectiondis{\thesubsectiondis.\arabic{subsubsection}} + \def\theparagraphdis{\thesubsubsectiondis.\arabic{paragraph}} + \fi +\else% not compsoc + \def\thesectiondis{\thesection.} % I. + \def\thesubsectiondis{\Alph{subsection}.} % B. + \def\thesubsubsectiondis{\arabic{subsubsection})} % 3) + \def\theparagraphdis{\alph{paragraph})} % d) +\fi + +% just like LaTeX2e's \@eqnnum +\def\theequationdis{{\normalfont \normalcolor (\theequation)}}% (1) +% IEEEsubequation used only by IEEEtran's IEEEeqnarray +\def\theIEEEsubequationdis{{\normalfont \normalcolor (\theIEEEsubequation)}}% (1a) +% redirect LaTeX2e's equation number display and all that depend on +% it, through IEEEtran's \theequationdis +\def\@eqnnum{\theequationdis} + + + +% V1.7 provide string macros as article.cls does +\def\contentsname{Contents} +\def\listfigurename{List of Figures} +\def\listtablename{List of Tables} +\def\refname{References} +\def\indexname{Index} +\def\figurename{Fig.} +\def\tablename{TABLE} +\@IEEEcompsocconfonly{\def\figurename{Figure}\def\tablename{Table}} +\def\partname{Part} +\def\appendixname{Appendix} +\def\abstractname{Abstract} +% IEEE specific names +\def\IEEEkeywordsname{Keywords} +\def\IEEEproofname{Proof} + + +% LIST OF FIGURES AND TABLES AND TABLE OF CONTENTS +% +\def\@pnumwidth{1.55em} +\def\@tocrmarg{2.55em} +\def\@dotsep{4.5} +\setcounter{tocdepth}{3} + +% adjusted some spacings here so that section numbers will not easily +% collide with the section titles. +% VIII; VIII-A; and VIII-A.1 are usually the worst offenders. +% MDS 1/2001 +\def\tableofcontents{\section*{\contentsname}\@starttoc{toc}} +\def\l@section#1#2{\addpenalty{\@secpenalty}\addvspace{1.0em plus 1pt}% + \@tempdima 2.75em \begingroup \parindent \z@ \rightskip \@pnumwidth% + \parfillskip-\@pnumwidth {\bfseries\leavevmode #1}\hfil\hbox to\@pnumwidth{\hss #2}\par% + \endgroup} +% argument format #1:level, #2:labelindent,#3:labelsep +\def\l@subsection{\@dottedtocline{2}{2.75em}{3.75em}} +\def\l@subsubsection{\@dottedtocline{3}{6.5em}{4.5em}} +% must provide \l@ defs for ALL sublevels EVEN if tocdepth +% is such as they will not appear in the table of contents +% these defs are how TOC knows what level these things are! +\def\l@paragraph{\@dottedtocline{4}{6.5em}{5.5em}} +\def\l@subparagraph{\@dottedtocline{5}{6.5em}{6.5em}} +\def\listoffigures{\section*{\listfigurename}\@starttoc{lof}} +\def\l@figure{\@dottedtocline{1}{0em}{2.75em}} +\def\listoftables{\section*{\listtablename}\@starttoc{lot}} +\let\l@table\l@figure + + +%% Definitions for floats +%% +%% Normal Floats +\floatsep 1\baselineskip plus 0.2\baselineskip minus 0.2\baselineskip +\textfloatsep 1.7\baselineskip plus 0.2\baselineskip minus 0.4\baselineskip +\@fptop 0pt plus 1fil +\@fpsep 0.75\baselineskip plus 2fil +\@fpbot 0pt plus 1fil +\def\topfraction{0.9} +\def\bottomfraction{0.4} +\def\floatpagefraction{0.8} +% V1.7, let top floats approach 90% of page +\def\textfraction{0.1} + +%% Double Column Floats +\dblfloatsep 1\baselineskip plus 0.2\baselineskip minus 0.2\baselineskip + +\dbltextfloatsep 1.7\baselineskip plus 0.2\baselineskip minus 0.4\baselineskip +% Note that it would be nice if the rubber here actually worked in LaTeX2e. +% There is a long standing limitation in LaTeX, first discovered (to the best +% of my knowledge) by Alan Jeffrey in 1992. LaTeX ignores the stretchable +% portion of \dbltextfloatsep, and as a result, double column figures can and +% do result in an non-integer number of lines in the main text columns with +% underfull vbox errors as a consequence. A post to comp.text.tex +% by Donald Arseneau confirms that this had not yet been fixed in 1998. +% IEEEtran V1.6 will fix this problem for you in the titles, but it doesn't +% protect you from other double floats. Happy vspace'ing. + +\@dblfptop 0pt plus 1fil +\@dblfpsep 0.75\baselineskip plus 2fil +\@dblfpbot 0pt plus 1fil +\def\dbltopfraction{0.8} +\def\dblfloatpagefraction{0.8} +\setcounter{dbltopnumber}{4} + +\intextsep 1\baselineskip plus 0.2\baselineskip minus 0.2\baselineskip +\setcounter{topnumber}{2} +\setcounter{bottomnumber}{2} +\setcounter{totalnumber}{4} + + + +% article class provides these, we should too. +\newlength\abovecaptionskip +\newlength\belowcaptionskip +% but only \abovecaptionskip is used above figure captions and *below* table +% captions +\setlength\abovecaptionskip{0.65\baselineskip} +\setlength\belowcaptionskip{0.75\baselineskip} +% V1.6 create hooks in case the caption spacing ever needs to be +% overridden by a user +\def\@IEEEfigurecaptionsepspace{\vskip\abovecaptionskip\relax}% +\def\@IEEEtablecaptionsepspace{\vskip\belowcaptionskip\relax}% + + +% 1.6b revise caption system so that \@makecaption uses two arguments +% as with LaTeX2e. Otherwise, there will be problems when using hyperref. +\def\@IEEEtablestring{table} + +\ifCLASSOPTIONcompsoc +% V1.7 compsoc \@makecaption +\ifCLASSOPTIONconference% compsoc conference +\long\def\@makecaption#1#2{% +% test if is a for a figure or table +\ifx\@captype\@IEEEtablestring% +% if a table, do table caption +\normalsize\begin{center}{\normalfont\sffamily\normalsize {#1.}~ #2}\end{center}% +\@IEEEtablecaptionsepspace +% if not a table, format it as a figure +\else +\@IEEEfigurecaptionsepspace +\setbox\@tempboxa\hbox{\normalfont\sffamily\normalsize {#1.}~ #2}% +\ifdim \wd\@tempboxa >\hsize% +% if caption is longer than a line, let it wrap around +\setbox\@tempboxa\hbox{\normalfont\sffamily\normalsize {#1.}~ }% +\parbox[t]{\hsize}{\normalfont\sffamily\normalsize \noindent\unhbox\@tempboxa#2}% +% if caption is shorter than a line, center +\else% +\hbox to\hsize{\normalfont\sffamily\normalsize\hfil\box\@tempboxa\hfil}% +\fi\fi} +\else% nonconference compsoc +\long\def\@makecaption#1#2{% +% test if is a for a figure or table +\ifx\@captype\@IEEEtablestring% +% if a table, do table caption +\normalsize\begin{center}{\normalfont\sffamily\normalsize #1}\\{\normalfont\sffamily\normalsize #2}\end{center}% +\@IEEEtablecaptionsepspace +% if not a table, format it as a figure +\else +\@IEEEfigurecaptionsepspace +\setbox\@tempboxa\hbox{\normalfont\sffamily\normalsize {#1.}~ #2}% +\ifdim \wd\@tempboxa >\hsize% +% if caption is longer than a line, let it wrap around +\setbox\@tempboxa\hbox{\normalfont\sffamily\normalsize {#1.}~ }% +\parbox[t]{\hsize}{\normalfont\sffamily\normalsize \noindent\unhbox\@tempboxa#2}% +% if caption is shorter than a line, left justify +\else% +\hbox to\hsize{\normalfont\sffamily\normalsize\box\@tempboxa\hfil}% +\fi\fi} +\fi + +\else% traditional noncompsoc \@makecaption +\long\def\@makecaption#1#2{% +% test if is a for a figure or table +\ifx\@captype\@IEEEtablestring% +% if a table, do table caption +\footnotesize{\centering\normalfont\footnotesize#1.\qquad\scshape #2\par}% +\@IEEEtablecaptionsepspace +% if not a table, format it as a figure +\else +\@IEEEfigurecaptionsepspace +% 3/2001 use footnotesize, not small; use two nonbreaking spaces, not one +\setbox\@tempboxa\hbox{\normalfont\footnotesize {#1.}~~ #2}% +\ifdim \wd\@tempboxa >\hsize% +% if caption is longer than a line, let it wrap around +\setbox\@tempboxa\hbox{\normalfont\footnotesize {#1.}~~ }% +\parbox[t]{\hsize}{\normalfont\footnotesize\noindent\unhbox\@tempboxa#2}% +% if caption is shorter than a line, center if conference, left justify otherwise +\else% +\ifCLASSOPTIONconference \hbox to\hsize{\normalfont\footnotesize\box\@tempboxa\hfil}% +\else \hbox to\hsize{\normalfont\footnotesize\box\@tempboxa\hfil}% +\fi\fi\fi} +\fi + + + +% V1.7 disable captions class option, do so in a way that retains operation of \label +% within \caption +\ifCLASSOPTIONcaptionsoff +\long\def\@makecaption#1#2{\vspace*{2em}\footnotesize\begin{center}{\footnotesize #1}\end{center}% +\let\@IEEEtemporiglabeldefsave\label +\let\@IEEEtemplabelargsave\relax +\def\label##1{\gdef\@IEEEtemplabelargsave{##1}}% +\setbox\@tempboxa\hbox{#2}% +\let\label\@IEEEtemporiglabeldefsave +\ifx\@IEEEtemplabelargsave\relax\else\label{\@IEEEtemplabelargsave}\fi} +\fi + + +% V1.7 define end environments with \def not \let so as to work OK with +% preview-latex +\newcounter{figure} +\def\thefigure{\@arabic\c@figure} +\def\fps@figure{tbp} +\def\ftype@figure{1} +\def\ext@figure{lof} +\def\fnum@figure{\figurename~\thefigure} +\def\figure{\@float{figure}} +\def\endfigure{\end@float} +\@namedef{figure*}{\@dblfloat{figure}} +\@namedef{endfigure*}{\end@dblfloat} +\newcounter{table} +\ifCLASSOPTIONcompsoc +\def\thetable{\arabic{table}} +\else +\def\thetable{\@Roman\c@table} +\fi +\def\fps@table{tbp} +\def\ftype@table{2} +\def\ext@table{lot} +\def\fnum@table{\tablename~\thetable} +% V1.6 IEEE uses 8pt text for tables +% to default to footnotesize, we hack into LaTeX2e's \@floatboxreset and pray +\def\table{\def\@floatboxreset{\reset@font\scriptsize\@setminipage}% + \let\@makefntext\@maketablefntext + \@float{table}} +\def\endtable{\end@float} +% v1.6b double column tables need to default to footnotesize as well. +\@namedef{table*}{\def\@floatboxreset{\reset@font\scriptsize\@setminipage}\@dblfloat{table}} +\@namedef{endtable*}{\end@dblfloat} + + + + +%% +%% START OF IEEEeqnarry DEFINITIONS +%% +%% Inspired by the concepts, examples, and previous works of LaTeX +%% coders and developers such as Donald Arseneau, Fred Bartlett, +%% David Carlisle, Tony Liu, Frank Mittelbach, Piet van Oostrum, +%% Roland Winkler and Mark Wooding. +%% I don't make the claim that my work here is even near their calibre. ;) + + +% hook to allow easy changeover to IEEEtran.cls/tools.sty error reporting +\def\@IEEEclspkgerror{\ClassError{IEEEtran}} + +\newif\if@IEEEeqnarraystarform% flag to indicate if the environment was called as the star form +\@IEEEeqnarraystarformfalse + +\newif\if@advanceIEEEeqncolcnt% tracks if the environment should advance the col counter +% allows a way to make an \IEEEeqnarraybox that can be used within an \IEEEeqnarray +% used by IEEEeqnarraymulticol so that it can work properly in both +\@advanceIEEEeqncolcnttrue + +\newcount\@IEEEeqnnumcols % tracks how many IEEEeqnarray cols are defined +\newcount\@IEEEeqncolcnt % tracks how many IEEEeqnarray cols the user actually used + + +% The default math style used by the columns +\def\IEEEeqnarraymathstyle{\displaystyle} +% The default text style used by the columns +% default to using the current font +\def\IEEEeqnarraytextstyle{\relax} + +% like the iedlistdecl but for \IEEEeqnarray +\def\IEEEeqnarraydecl{\relax} +\def\IEEEeqnarrayboxdecl{\relax} + +% \yesnumber is the opposite of \nonumber +% a novel concept with the same def as the equationarray package +% However, we give IEEE versions too since some LaTeX packages such as +% the MDWtools mathenv.sty redefine \nonumber to something else. +\providecommand{\yesnumber}{\global\@eqnswtrue} +\def\IEEEyesnumber{\global\@eqnswtrue} +\def\IEEEnonumber{\global\@eqnswfalse} + + +\def\IEEEyessubnumber{\global\@IEEEissubequationtrue\global\@eqnswtrue% +\if@IEEEeqnarrayISinner% only do something inside an IEEEeqnarray +\if@IEEElastlinewassubequation\addtocounter{equation}{-1}\else\setcounter{IEEEsubequation}{1}\fi% +\def\@currentlabel{\p@IEEEsubequation\theIEEEsubequation}\fi} + +% flag to indicate that an equation is a sub equation +\newif\if@IEEEissubequation% +\@IEEEissubequationfalse + +% allows users to "push away" equations that get too close to the equation numbers +\def\IEEEeqnarraynumspace{\hphantom{\if@IEEEissubequation\theIEEEsubequationdis\else\theequationdis\fi}} + +% provides a way to span multiple columns within IEEEeqnarray environments +% will consider \if@advanceIEEEeqncolcnt before globally advancing the +% column counter - so as to work within \IEEEeqnarraybox +% usage: \IEEEeqnarraymulticol{number cols. to span}{col type}{cell text} +\long\def\IEEEeqnarraymulticol#1#2#3{\multispan{#1}% +% check if column is defined +\relax\expandafter\ifx\csname @IEEEeqnarraycolDEF#2\endcsname\@IEEEeqnarraycolisdefined% +\csname @IEEEeqnarraycolPRE#2\endcsname#3\relax\relax\relax\relax\relax% +\relax\relax\relax\relax\relax\csname @IEEEeqnarraycolPOST#2\endcsname% +\else% if not, error and use default type +\@IEEEclspkgerror{Invalid column type "#2" in \string\IEEEeqnarraymulticol.\MessageBreak +Using a default centering column instead}% +{You must define IEEEeqnarray column types before use.}% +\csname @IEEEeqnarraycolPRE@IEEEdefault\endcsname#3\relax\relax\relax\relax\relax% +\relax\relax\relax\relax\relax\csname @IEEEeqnarraycolPOST@IEEEdefault\endcsname% +\fi% +% advance column counter only if the IEEEeqnarray environment wants it +\if@advanceIEEEeqncolcnt\global\advance\@IEEEeqncolcnt by #1\relax\fi} + +% like \omit, but maintains track of the column counter for \IEEEeqnarray +\def\IEEEeqnarrayomit{\omit\if@advanceIEEEeqncolcnt\global\advance\@IEEEeqncolcnt by 1\relax\fi} + + +% provides a way to define a letter referenced column type +% usage: \IEEEeqnarraydefcol{col. type letter/name}{pre insertion text}{post insertion text} +\def\IEEEeqnarraydefcol#1#2#3{\expandafter\def\csname @IEEEeqnarraycolPRE#1\endcsname{#2}% +\expandafter\def\csname @IEEEeqnarraycolPOST#1\endcsname{#3}% +\expandafter\def\csname @IEEEeqnarraycolDEF#1\endcsname{1}} + + +% provides a way to define a numerically referenced inter-column glue types +% usage: \IEEEeqnarraydefcolsep{col. glue number}{glue definition} +\def\IEEEeqnarraydefcolsep#1#2{\expandafter\def\csname @IEEEeqnarraycolSEP\romannumeral #1\endcsname{#2}% +\expandafter\def\csname @IEEEeqnarraycolSEPDEF\romannumeral #1\endcsname{1}} + + +\def\@IEEEeqnarraycolisdefined{1}% just a macro for 1, used for checking undefined column types + + +% expands and appends the given argument to the \@IEEEtrantmptoksA token list +% used to build up the \halign preamble +\def\@IEEEappendtoksA#1{\edef\@@IEEEappendtoksA{\@IEEEtrantmptoksA={\the\@IEEEtrantmptoksA #1}}% +\@@IEEEappendtoksA} + +% also appends to \@IEEEtrantmptoksA, but does not expand the argument +% uses \toks8 as a scratchpad register +\def\@IEEEappendNOEXPANDtoksA#1{\toks8={#1}% +\edef\@@IEEEappendNOEXPANDtoksA{\@IEEEtrantmptoksA={\the\@IEEEtrantmptoksA\the\toks8}}% +\@@IEEEappendNOEXPANDtoksA} + +% define some common column types for the user +% math +\IEEEeqnarraydefcol{l}{$\IEEEeqnarraymathstyle}{$\hfil} +\IEEEeqnarraydefcol{c}{\hfil$\IEEEeqnarraymathstyle}{$\hfil} +\IEEEeqnarraydefcol{r}{\hfil$\IEEEeqnarraymathstyle}{$} +\IEEEeqnarraydefcol{L}{$\IEEEeqnarraymathstyle{}}{{}$\hfil} +\IEEEeqnarraydefcol{C}{\hfil$\IEEEeqnarraymathstyle{}}{{}$\hfil} +\IEEEeqnarraydefcol{R}{\hfil$\IEEEeqnarraymathstyle{}}{{}$} +% text +\IEEEeqnarraydefcol{s}{\IEEEeqnarraytextstyle}{\hfil} +\IEEEeqnarraydefcol{t}{\hfil\IEEEeqnarraytextstyle}{\hfil} +\IEEEeqnarraydefcol{u}{\hfil\IEEEeqnarraytextstyle}{} + +% vertical rules +\IEEEeqnarraydefcol{v}{}{\vrule width\arrayrulewidth} +\IEEEeqnarraydefcol{vv}{\vrule width\arrayrulewidth\hfil}{\hfil\vrule width\arrayrulewidth} +\IEEEeqnarraydefcol{V}{}{\vrule width\arrayrulewidth\hskip\doublerulesep\vrule width\arrayrulewidth} +\IEEEeqnarraydefcol{VV}{\vrule width\arrayrulewidth\hskip\doublerulesep\vrule width\arrayrulewidth\hfil}% +{\hfil\vrule width\arrayrulewidth\hskip\doublerulesep\vrule width\arrayrulewidth} + +% horizontal rules +\IEEEeqnarraydefcol{h}{}{\leaders\hrule height\arrayrulewidth\hfil} +\IEEEeqnarraydefcol{H}{}{\leaders\vbox{\hrule width\arrayrulewidth\vskip\doublerulesep\hrule width\arrayrulewidth}\hfil} + +% plain +\IEEEeqnarraydefcol{x}{}{} +\IEEEeqnarraydefcol{X}{$}{$} + +% the default column type to use in the event a column type is not defined +\IEEEeqnarraydefcol{@IEEEdefault}{\hfil$\IEEEeqnarraymathstyle}{$\hfil} + + +% a zero tabskip (used for "-" col types) +\def\@IEEEeqnarraycolSEPzero{0pt plus 0pt minus 0pt} +% a centering tabskip (used for "+" col types) +\def\@IEEEeqnarraycolSEPcenter{1000pt plus 0pt minus 1000pt} + +% top level default tabskip glues for the start, end, and inter-column +% may be reset within environments not always at the top level, e.g., \IEEEeqnarraybox +\edef\@IEEEeqnarraycolSEPdefaultstart{\@IEEEeqnarraycolSEPcenter}% default start glue +\edef\@IEEEeqnarraycolSEPdefaultend{\@IEEEeqnarraycolSEPcenter}% default end glue +\edef\@IEEEeqnarraycolSEPdefaultmid{\@IEEEeqnarraycolSEPzero}% default inter-column glue + + + +% creates a vertical rule that extends from the bottom to the top a a cell +% Provided in case other packages redefine \vline some other way. +% usage: \IEEEeqnarrayvrule[rule thickness] +% If no argument is provided, \arrayrulewidth will be used for the rule thickness. +\newcommand\IEEEeqnarrayvrule[1][\arrayrulewidth]{\vrule\@width#1\relax} + +% creates a blank separator row +% usage: \IEEEeqnarrayseprow[separation length][font size commands] +% default is \IEEEeqnarrayseprow[0.25\normalbaselineskip][\relax] +% blank arguments inherit the default values +% uses \skip5 as a scratch register - calls \@IEEEeqnarraystrutsize which uses more scratch registers +\def\IEEEeqnarrayseprow{\relax\@ifnextchar[{\@IEEEeqnarrayseprow}{\@IEEEeqnarrayseprow[0.25\normalbaselineskip]}} +\def\@IEEEeqnarrayseprow[#1]{\relax\@ifnextchar[{\@@IEEEeqnarrayseprow[#1]}{\@@IEEEeqnarrayseprow[#1][\relax]}} +\def\@@IEEEeqnarrayseprow[#1][#2]{\def\@IEEEeqnarrayseprowARGONE{#1}% +\ifx\@IEEEeqnarrayseprowARGONE\@empty% +% get the skip value, based on the font commands +% use skip5 because \IEEEeqnarraystrutsize uses \skip0, \skip2, \skip3 +% assign within a bogus box to confine the font changes +{\setbox0=\hbox{#2\relax\global\skip5=0.25\normalbaselineskip}}% +\else% +{\setbox0=\hbox{#2\relax\global\skip5=#1}}% +\fi% +\@IEEEeqnarrayhoptolastcolumn\IEEEeqnarraystrutsize{\skip5}{0pt}[\relax]\relax} + +% creates a blank separator row, but omits all the column templates +% usage: \IEEEeqnarrayseprowcut[separation length][font size commands] +% default is \IEEEeqnarrayseprowcut[0.25\normalbaselineskip][\relax] +% blank arguments inherit the default values +% uses \skip5 as a scratch register - calls \@IEEEeqnarraystrutsize which uses more scratch registers +\def\IEEEeqnarrayseprowcut{\multispan{\@IEEEeqnnumcols}\relax% span all the cols +% advance column counter only if the IEEEeqnarray environment wants it +\if@advanceIEEEeqncolcnt\global\advance\@IEEEeqncolcnt by \@IEEEeqnnumcols\relax\fi% +\@ifnextchar[{\@IEEEeqnarrayseprowcut}{\@IEEEeqnarrayseprowcut[0.25\normalbaselineskip]}} +\def\@IEEEeqnarrayseprowcut[#1]{\relax\@ifnextchar[{\@@IEEEeqnarrayseprowcut[#1]}{\@@IEEEeqnarrayseprowcut[#1][\relax]}} +\def\@@IEEEeqnarrayseprowcut[#1][#2]{\def\@IEEEeqnarrayseprowARGONE{#1}% +\ifx\@IEEEeqnarrayseprowARGONE\@empty% +% get the skip value, based on the font commands +% use skip5 because \IEEEeqnarraystrutsize uses \skip0, \skip2, \skip3 +% assign within a bogus box to confine the font changes +{\setbox0=\hbox{#2\relax\global\skip5=0.25\normalbaselineskip}}% +\else% +{\setbox0=\hbox{#2\relax\global\skip5=#1}}% +\fi% +\IEEEeqnarraystrutsize{\skip5}{0pt}[\relax]\relax} + + + +% draws a single rule across all the columns optional +% argument determines the rule width, \arrayrulewidth is the default +% updates column counter as needed and turns off struts +% usage: \IEEEeqnarrayrulerow[rule line thickness] +\def\IEEEeqnarrayrulerow{\multispan{\@IEEEeqnnumcols}\relax% span all the cols +% advance column counter only if the IEEEeqnarray environment wants it +\if@advanceIEEEeqncolcnt\global\advance\@IEEEeqncolcnt by \@IEEEeqnnumcols\relax\fi% +\@ifnextchar[{\@IEEEeqnarrayrulerow}{\@IEEEeqnarrayrulerow[\arrayrulewidth]}} +\def\@IEEEeqnarrayrulerow[#1]{\leaders\hrule height#1\hfil\relax% put in our rule +% turn off any struts +\IEEEeqnarraystrutsize{0pt}{0pt}[\relax]\relax} + + +% draws a double rule by using a single rule row, a separator row, and then +% another single rule row +% first optional argument determines the rule thicknesses, \arrayrulewidth is the default +% second optional argument determines the rule spacing, \doublerulesep is the default +% usage: \IEEEeqnarraydblrulerow[rule line thickness][rule spacing] +\def\IEEEeqnarraydblrulerow{\multispan{\@IEEEeqnnumcols}\relax% span all the cols +% advance column counter only if the IEEEeqnarray environment wants it +\if@advanceIEEEeqncolcnt\global\advance\@IEEEeqncolcnt by \@IEEEeqnnumcols\relax\fi% +\@ifnextchar[{\@IEEEeqnarraydblrulerow}{\@IEEEeqnarraydblrulerow[\arrayrulewidth]}} +\def\@IEEEeqnarraydblrulerow[#1]{\relax\@ifnextchar[{\@@IEEEeqnarraydblrulerow[#1]}% +{\@@IEEEeqnarraydblrulerow[#1][\doublerulesep]}} +\def\@@IEEEeqnarraydblrulerow[#1][#2]{\def\@IEEEeqnarraydblrulerowARG{#1}% +% we allow the user to say \IEEEeqnarraydblrulerow[][] +\ifx\@IEEEeqnarraydblrulerowARG\@empty% +\@IEEEeqnarrayrulerow[\arrayrulewidth]% +\else% +\@IEEEeqnarrayrulerow[#1]\relax% +\fi% +\def\@IEEEeqnarraydblrulerowARG{#2}% +\ifx\@IEEEeqnarraydblrulerowARG\@empty% +\\\IEEEeqnarrayseprow[\doublerulesep][\relax]% +\else% +\\\IEEEeqnarrayseprow[#2][\relax]% +\fi% +\\\multispan{\@IEEEeqnnumcols}% +% advance column counter only if the IEEEeqnarray environment wants it +\if@advanceIEEEeqncolcnt\global\advance\@IEEEeqncolcnt by \@IEEEeqnnumcols\relax\fi% +\def\@IEEEeqnarraydblrulerowARG{#1}% +\ifx\@IEEEeqnarraydblrulerowARG\@empty% +\@IEEEeqnarrayrulerow[\arrayrulewidth]% +\else% +\@IEEEeqnarrayrulerow[#1]% +\fi% +} + +% draws a double rule by using a single rule row, a separator (cutting) row, and then +% another single rule row +% first optional argument determines the rule thicknesses, \arrayrulewidth is the default +% second optional argument determines the rule spacing, \doublerulesep is the default +% usage: \IEEEeqnarraydblrulerow[rule line thickness][rule spacing] +\def\IEEEeqnarraydblrulerowcut{\multispan{\@IEEEeqnnumcols}\relax% span all the cols +% advance column counter only if the IEEEeqnarray environment wants it +\if@advanceIEEEeqncolcnt\global\advance\@IEEEeqncolcnt by \@IEEEeqnnumcols\relax\fi% +\@ifnextchar[{\@IEEEeqnarraydblrulerowcut}{\@IEEEeqnarraydblrulerowcut[\arrayrulewidth]}} +\def\@IEEEeqnarraydblrulerowcut[#1]{\relax\@ifnextchar[{\@@IEEEeqnarraydblrulerowcut[#1]}% +{\@@IEEEeqnarraydblrulerowcut[#1][\doublerulesep]}} +\def\@@IEEEeqnarraydblrulerowcut[#1][#2]{\def\@IEEEeqnarraydblrulerowARG{#1}% +% we allow the user to say \IEEEeqnarraydblrulerow[][] +\ifx\@IEEEeqnarraydblrulerowARG\@empty% +\@IEEEeqnarrayrulerow[\arrayrulewidth]% +\else% +\@IEEEeqnarrayrulerow[#1]% +\fi% +\def\@IEEEeqnarraydblrulerowARG{#2}% +\ifx\@IEEEeqnarraydblrulerowARG\@empty% +\\\IEEEeqnarrayseprowcut[\doublerulesep][\relax]% +\else% +\\\IEEEeqnarrayseprowcut[#2][\relax]% +\fi% +\\\multispan{\@IEEEeqnnumcols}% +% advance column counter only if the IEEEeqnarray environment wants it +\if@advanceIEEEeqncolcnt\global\advance\@IEEEeqncolcnt by \@IEEEeqnnumcols\relax\fi% +\def\@IEEEeqnarraydblrulerowARG{#1}% +\ifx\@IEEEeqnarraydblrulerowARG\@empty% +\@IEEEeqnarrayrulerow[\arrayrulewidth]% +\else% +\@IEEEeqnarrayrulerow[#1]% +\fi% +} + + + +% inserts a full row's worth of &'s +% relies on \@IEEEeqnnumcols to provide the correct number of columns +% uses \@IEEEtrantmptoksA, \count0 as scratch registers +\def\@IEEEeqnarrayhoptolastcolumn{\@IEEEtrantmptoksA={}\count0=1\relax% +\loop% add cols if the user did not use them all +\ifnum\count0<\@IEEEeqnnumcols\relax% +\@IEEEappendtoksA{&}% +\advance\count0 by 1\relax% update the col count +\repeat% +\the\@IEEEtrantmptoksA%execute the &'s +} + + + +\newif\if@IEEEeqnarrayISinner % flag to indicate if we are within the lines +\@IEEEeqnarrayISinnerfalse % of an IEEEeqnarray - after the IEEEeqnarraydecl + +\edef\@IEEEeqnarrayTHEstrutheight{0pt} % height and depth of IEEEeqnarray struts +\edef\@IEEEeqnarrayTHEstrutdepth{0pt} + +\edef\@IEEEeqnarrayTHEmasterstrutheight{0pt} % default height and depth of +\edef\@IEEEeqnarrayTHEmasterstrutdepth{0pt} % struts within an IEEEeqnarray + +\edef\@IEEEeqnarrayTHEmasterstrutHSAVE{0pt} % saved master strut height +\edef\@IEEEeqnarrayTHEmasterstrutDSAVE{0pt} % and depth + +\newif\if@IEEEeqnarrayusemasterstrut % flag to indicate that the master strut value +\@IEEEeqnarrayusemasterstruttrue % is to be used + + + +% saves the strut height and depth of the master strut +\def\@IEEEeqnarraymasterstrutsave{\relax% +\expandafter\skip0=\@IEEEeqnarrayTHEmasterstrutheight\relax% +\expandafter\skip2=\@IEEEeqnarrayTHEmasterstrutdepth\relax% +% remove stretchability +\dimen0\skip0\relax% +\dimen2\skip2\relax% +% save values +\edef\@IEEEeqnarrayTHEmasterstrutHSAVE{\the\dimen0}% +\edef\@IEEEeqnarrayTHEmasterstrutDSAVE{\the\dimen2}} + +% restores the strut height and depth of the master strut +\def\@IEEEeqnarraymasterstrutrestore{\relax% +\expandafter\skip0=\@IEEEeqnarrayTHEmasterstrutHSAVE\relax% +\expandafter\skip2=\@IEEEeqnarrayTHEmasterstrutDSAVE\relax% +% remove stretchability +\dimen0\skip0\relax% +\dimen2\skip2\relax% +% restore values +\edef\@IEEEeqnarrayTHEmasterstrutheight{\the\dimen0}% +\edef\@IEEEeqnarrayTHEmasterstrutdepth{\the\dimen2}} + + +% globally restores the strut height and depth to the +% master values and sets the master strut flag to true +\def\@IEEEeqnarraystrutreset{\relax% +\expandafter\skip0=\@IEEEeqnarrayTHEmasterstrutheight\relax% +\expandafter\skip2=\@IEEEeqnarrayTHEmasterstrutdepth\relax% +% remove stretchability +\dimen0\skip0\relax% +\dimen2\skip2\relax% +% restore values +\xdef\@IEEEeqnarrayTHEstrutheight{\the\dimen0}% +\xdef\@IEEEeqnarrayTHEstrutdepth{\the\dimen2}% +\global\@IEEEeqnarrayusemasterstruttrue} + + +% if the master strut is not to be used, make the current +% values of \@IEEEeqnarrayTHEstrutheight, \@IEEEeqnarrayTHEstrutdepth +% and the use master strut flag, global +% this allows user strut commands issued in the last column to be carried +% into the isolation/strut column +\def\@IEEEeqnarrayglobalizestrutstatus{\relax% +\if@IEEEeqnarrayusemasterstrut\else% +\xdef\@IEEEeqnarrayTHEstrutheight{\@IEEEeqnarrayTHEstrutheight}% +\xdef\@IEEEeqnarrayTHEstrutdepth{\@IEEEeqnarrayTHEstrutdepth}% +\global\@IEEEeqnarrayusemasterstrutfalse% +\fi} + + + +% usage: \IEEEeqnarraystrutsize{height}{depth}[font size commands] +% If called outside the lines of an IEEEeqnarray, sets the height +% and depth of both the master and local struts. If called inside +% an IEEEeqnarray line, sets the height and depth of the local strut +% only and sets the flag to indicate the use of the local strut +% values. If the height or depth is left blank, 0.7\normalbaselineskip +% and 0.3\normalbaselineskip will be used, respectively. +% The optional argument can be used to evaluate the lengths under +% a different font size and styles. If none is specified, the current +% font is used. +% uses scratch registers \skip0, \skip2, \skip3, \dimen0, \dimen2 +\def\IEEEeqnarraystrutsize#1#2{\relax\@ifnextchar[{\@IEEEeqnarraystrutsize{#1}{#2}}{\@IEEEeqnarraystrutsize{#1}{#2}[\relax]}} +\def\@IEEEeqnarraystrutsize#1#2[#3]{\def\@IEEEeqnarraystrutsizeARG{#1}% +\ifx\@IEEEeqnarraystrutsizeARG\@empty% +{\setbox0=\hbox{#3\relax\global\skip3=0.7\normalbaselineskip}}% +\skip0=\skip3\relax% +\else% arg one present +{\setbox0=\hbox{#3\relax\global\skip3=#1\relax}}% +\skip0=\skip3\relax% +\fi% if null arg +\def\@IEEEeqnarraystrutsizeARG{#2}% +\ifx\@IEEEeqnarraystrutsizeARG\@empty% +{\setbox0=\hbox{#3\relax\global\skip3=0.3\normalbaselineskip}}% +\skip2=\skip3\relax% +\else% arg two present +{\setbox0=\hbox{#3\relax\global\skip3=#2\relax}}% +\skip2=\skip3\relax% +\fi% if null arg +% remove stretchability, just to be safe +\dimen0\skip0\relax% +\dimen2\skip2\relax% +% dimen0 = height, dimen2 = depth +\if@IEEEeqnarrayISinner% inner does not touch master strut size +\edef\@IEEEeqnarrayTHEstrutheight{\the\dimen0}% +\edef\@IEEEeqnarrayTHEstrutdepth{\the\dimen2}% +\@IEEEeqnarrayusemasterstrutfalse% do not use master +\else% outer, have to set master strut too +\edef\@IEEEeqnarrayTHEmasterstrutheight{\the\dimen0}% +\edef\@IEEEeqnarrayTHEmasterstrutdepth{\the\dimen2}% +\edef\@IEEEeqnarrayTHEstrutheight{\the\dimen0}% +\edef\@IEEEeqnarrayTHEstrutdepth{\the\dimen2}% +\@IEEEeqnarrayusemasterstruttrue% use master strut +\fi} + + +% usage: \IEEEeqnarraystrutsizeadd{added height}{added depth}[font size commands] +% If called outside the lines of an IEEEeqnarray, adds the given height +% and depth to both the master and local struts. +% If called inside an IEEEeqnarray line, adds the given height and depth +% to the local strut only and sets the flag to indicate the use +% of the local strut values. +% In both cases, if a height or depth is left blank, 0pt is used instead. +% The optional argument can be used to evaluate the lengths under +% a different font size and styles. If none is specified, the current +% font is used. +% uses scratch registers \skip0, \skip2, \skip3, \dimen0, \dimen2 +\def\IEEEeqnarraystrutsizeadd#1#2{\relax\@ifnextchar[{\@IEEEeqnarraystrutsizeadd{#1}{#2}}{\@IEEEeqnarraystrutsizeadd{#1}{#2}[\relax]}} +\def\@IEEEeqnarraystrutsizeadd#1#2[#3]{\def\@IEEEeqnarraystrutsizearg{#1}% +\ifx\@IEEEeqnarraystrutsizearg\@empty% +\skip0=0pt\relax% +\else% arg one present +{\setbox0=\hbox{#3\relax\global\skip3=#1}}% +\skip0=\skip3\relax% +\fi% if null arg +\def\@IEEEeqnarraystrutsizearg{#2}% +\ifx\@IEEEeqnarraystrutsizearg\@empty% +\skip2=0pt\relax% +\else% arg two present +{\setbox0=\hbox{#3\relax\global\skip3=#2}}% +\skip2=\skip3\relax% +\fi% if null arg +% remove stretchability, just to be safe +\dimen0\skip0\relax% +\dimen2\skip2\relax% +% dimen0 = height, dimen2 = depth +\if@IEEEeqnarrayISinner% inner does not touch master strut size +% get local strut size +\expandafter\skip0=\@IEEEeqnarrayTHEstrutheight\relax% +\expandafter\skip2=\@IEEEeqnarrayTHEstrutdepth\relax% +% add it to the user supplied values +\advance\dimen0 by \skip0\relax% +\advance\dimen2 by \skip2\relax% +% update the local strut size +\edef\@IEEEeqnarrayTHEstrutheight{\the\dimen0}% +\edef\@IEEEeqnarrayTHEstrutdepth{\the\dimen2}% +\@IEEEeqnarrayusemasterstrutfalse% do not use master +\else% outer, have to set master strut too +% get master strut size +\expandafter\skip0=\@IEEEeqnarrayTHEmasterstrutheight\relax% +\expandafter\skip2=\@IEEEeqnarrayTHEmasterstrutdepth\relax% +% add it to the user supplied values +\advance\dimen0 by \skip0\relax% +\advance\dimen2 by \skip2\relax% +% update the local and master strut sizes +\edef\@IEEEeqnarrayTHEmasterstrutheight{\the\dimen0}% +\edef\@IEEEeqnarrayTHEmasterstrutdepth{\the\dimen2}% +\edef\@IEEEeqnarrayTHEstrutheight{\the\dimen0}% +\edef\@IEEEeqnarrayTHEstrutdepth{\the\dimen2}% +\@IEEEeqnarrayusemasterstruttrue% use master strut +\fi} + + +% allow user a way to see the struts +\newif\ifIEEEvisiblestruts +\IEEEvisiblestrutsfalse + +% inserts an invisible strut using the master or local strut values +% uses scratch registers \skip0, \skip2, \dimen0, \dimen2 +\def\@IEEEeqnarrayinsertstrut{\relax% +\if@IEEEeqnarrayusemasterstrut +% get master strut size +\expandafter\skip0=\@IEEEeqnarrayTHEmasterstrutheight\relax% +\expandafter\skip2=\@IEEEeqnarrayTHEmasterstrutdepth\relax% +\else% +% get local strut size +\expandafter\skip0=\@IEEEeqnarrayTHEstrutheight\relax% +\expandafter\skip2=\@IEEEeqnarrayTHEstrutdepth\relax% +\fi% +% remove stretchability, probably not needed +\dimen0\skip0\relax% +\dimen2\skip2\relax% +% dimen0 = height, dimen2 = depth +% allow user to see struts if desired +\ifIEEEvisiblestruts% +\vrule width0.2pt height\dimen0 depth\dimen2\relax% +\else% +\vrule width0pt height\dimen0 depth\dimen2\relax\fi} + + +% creates an invisible strut, useable even outside \IEEEeqnarray +% if \IEEEvisiblestrutstrue, the strut will be visible and 0.2pt wide. +% usage: \IEEEstrut[height][depth][font size commands] +% default is \IEEEstrut[0.7\normalbaselineskip][0.3\normalbaselineskip][\relax] +% blank arguments inherit the default values +% uses \dimen0, \dimen2, \skip0, \skip2 +\def\IEEEstrut{\relax\@ifnextchar[{\@IEEEstrut}{\@IEEEstrut[0.7\normalbaselineskip]}} +\def\@IEEEstrut[#1]{\relax\@ifnextchar[{\@@IEEEstrut[#1]}{\@@IEEEstrut[#1][0.3\normalbaselineskip]}} +\def\@@IEEEstrut[#1][#2]{\relax\@ifnextchar[{\@@@IEEEstrut[#1][#2]}{\@@@IEEEstrut[#1][#2][\relax]}} +\def\@@@IEEEstrut[#1][#2][#3]{\mbox{#3\relax% +\def\@IEEEstrutARG{#1}% +\ifx\@IEEEstrutARG\@empty% +\skip0=0.7\normalbaselineskip\relax% +\else% +\skip0=#1\relax% +\fi% +\def\@IEEEstrutARG{#2}% +\ifx\@IEEEstrutARG\@empty% +\skip2=0.3\normalbaselineskip\relax% +\else% +\skip2=#2\relax% +\fi% +% remove stretchability, probably not needed +\dimen0\skip0\relax% +\dimen2\skip2\relax% +\ifIEEEvisiblestruts% +\vrule width0.2pt height\dimen0 depth\dimen2\relax% +\else% +\vrule width0.0pt height\dimen0 depth\dimen2\relax\fi}} + + +% enables strut mode by setting a default strut size and then zeroing the +% \baselineskip, \lineskip, \lineskiplimit and \jot +\def\IEEEeqnarraystrutmode{\IEEEeqnarraystrutsize{0.7\normalbaselineskip}{0.3\normalbaselineskip}[\relax]% +\baselineskip=0pt\lineskip=0pt\lineskiplimit=0pt\jot=0pt} + + + +\def\IEEEeqnarray{\@IEEEeqnarraystarformfalse\@IEEEeqnarray} +\def\endIEEEeqnarray{\end@IEEEeqnarray} + +\@namedef{IEEEeqnarray*}{\@IEEEeqnarraystarformtrue\@IEEEeqnarray} +\@namedef{endIEEEeqnarray*}{\end@IEEEeqnarray} + + +% \IEEEeqnarray is an enhanced \eqnarray. +% The star form defaults to not putting equation numbers at the end of each row. +% usage: \IEEEeqnarray[decl]{cols} +\def\@IEEEeqnarray{\relax\@ifnextchar[{\@@IEEEeqnarray}{\@@IEEEeqnarray[\relax]}} +\def\@@IEEEeqnarray[#1]#2{% + % default to showing the equation number or not based on whether or not + % the star form was involked + \if@IEEEeqnarraystarform\global\@eqnswfalse + \else% not the star form + \global\@eqnswtrue + \fi% if star form + \@IEEEissubequationfalse% default to no subequations + \@IEEElastlinewassubequationfalse% assume last line is not a sub equation + \@IEEEeqnarrayISinnerfalse% not yet within the lines of the halign + \@IEEEeqnarraystrutsize{0pt}{0pt}[\relax]% turn off struts by default + \@IEEEeqnarrayusemasterstruttrue% use master strut till user asks otherwise + \IEEEvisiblestrutsfalse% diagnostic mode defaults to off + % no extra space unless the user specifically requests it + \lineskip=0pt\relax + \lineskiplimit=0pt\relax + \baselineskip=\normalbaselineskip\relax% + \jot=\IEEEnormaljot\relax% + \mathsurround\z@\relax% no extra spacing around math + \@advanceIEEEeqncolcnttrue% advance the col counter for each col the user uses, + % used in \IEEEeqnarraymulticol and in the preamble build + \stepcounter{equation}% advance equation counter before first line + \setcounter{IEEEsubequation}{0}% no subequation yet + \def\@currentlabel{\p@equation\theequation}% redefine the ref label + \IEEEeqnarraydecl\relax% allow a way for the user to make global overrides + #1\relax% allow user to override defaults + \let\\\@IEEEeqnarraycr% replace newline with one that can put in eqn. numbers + \global\@IEEEeqncolcnt\z@% col. count = 0 for first line + \@IEEEbuildpreamble #2\end\relax% build the preamble and put it into \@IEEEtrantmptoksA + % put in the column for the equation number + \ifnum\@IEEEeqnnumcols>0\relax\@IEEEappendtoksA{&}\fi% col separator for those after the first + \toks0={##}% + % advance the \@IEEEeqncolcnt for the isolation col, this helps with error checking + \@IEEEappendtoksA{\global\advance\@IEEEeqncolcnt by 1\relax}% + % add the isolation column + \@IEEEappendtoksA{\tabskip\z@skip\bgroup\the\toks0\egroup}% + % advance the \@IEEEeqncolcnt for the equation number col, this helps with error checking + \@IEEEappendtoksA{&\global\advance\@IEEEeqncolcnt by 1\relax}% + % add the equation number col to the preamble + \@IEEEappendtoksA{\tabskip\z@skip\hb@xt@\z@\bgroup\hss\the\toks0\egroup}% + % note \@IEEEeqnnumcols does not count the equation col or isolation col + % set the starting tabskip glue as determined by the preamble build + \tabskip=\@IEEEBPstartglue\relax + % begin the display alignment + \@IEEEeqnarrayISinnertrue% commands are now within the lines + $$\everycr{}\halign to\displaywidth\bgroup + % "exspand" the preamble + \span\the\@IEEEtrantmptoksA\cr} + +% enter isolation/strut column (or the next column if the user did not use +% every column), record the strut status, complete the columns, do the strut if needed, +% restore counters to correct values and exit +\def\end@IEEEeqnarray{\@IEEEeqnarrayglobalizestrutstatus&\@@IEEEeqnarraycr\egroup% +\if@IEEElastlinewassubequation\global\advance\c@IEEEsubequation\m@ne\fi% +\global\advance\c@equation\m@ne% +$$\@ignoretrue} + +% need a way to remember if last line is a subequation +\newif\if@IEEElastlinewassubequation% +\@IEEElastlinewassubequationfalse + +% IEEEeqnarray uses a modifed \\ instead of the plain \cr to +% end rows. This allows for things like \\*[vskip amount] +% This "cr" macros are modified versions those for LaTeX2e's eqnarray +% the {\ifnum0=`} braces must be kept away from the last column to avoid +% altering spacing of its math, so we use & to advance to the next column +% as there is an isolation/strut column after the user's columns +\def\@IEEEeqnarraycr{\@IEEEeqnarrayglobalizestrutstatus&% save strut status and advance to next column + {\ifnum0=`}\fi + \@ifstar{% + \global\@eqpen\@M\@IEEEeqnarrayYCR + }{% + \global\@eqpen\interdisplaylinepenalty \@IEEEeqnarrayYCR + }% +} + +\def\@IEEEeqnarrayYCR{\@testopt\@IEEEeqnarrayXCR\z@skip} + +\def\@IEEEeqnarrayXCR[#1]{% + \ifnum0=`{\fi}% + \@@IEEEeqnarraycr + \noalign{\penalty\@eqpen\vskip\jot\vskip #1\relax}}% + +\def\@@IEEEeqnarraycr{\@IEEEtrantmptoksA={}% clear token register + \advance\@IEEEeqncolcnt by -1\relax% adjust col count because of the isolation column + \ifnum\@IEEEeqncolcnt>\@IEEEeqnnumcols\relax + \@IEEEclspkgerror{Too many columns within the IEEEeqnarray\MessageBreak + environment}% + {Use fewer \string &'s or put more columns in the IEEEeqnarry column\MessageBreak + specifications.}\relax% + \else + \loop% add cols if the user did not use them all + \ifnum\@IEEEeqncolcnt<\@IEEEeqnnumcols\relax + \@IEEEappendtoksA{&}% + \advance\@IEEEeqncolcnt by 1\relax% update the col count + \repeat + % this number of &'s will take us the the isolation column + \fi + % execute the &'s + \the\@IEEEtrantmptoksA% + % handle the strut/isolation column + \@IEEEeqnarrayinsertstrut% do the strut if needed + \@IEEEeqnarraystrutreset% reset the strut system for next line or IEEEeqnarray + &% and enter the equation number column + % is this line needs an equation number, display it and advance the + % (sub)equation counters, record what type this line was + \if@eqnsw% + \if@IEEEissubequation\theIEEEsubequationdis\addtocounter{equation}{1}\stepcounter{IEEEsubequation}% + \global\@IEEElastlinewassubequationtrue% + \else% display a standard equation number, initialize the IEEEsubequation counter + \theequationdis\stepcounter{equation}\setcounter{IEEEsubequation}{0}% + \global\@IEEElastlinewassubequationfalse\fi% + \fi% + % reset the eqnsw flag to indicate default preference of the display of equation numbers + \if@IEEEeqnarraystarform\global\@eqnswfalse\else\global\@eqnswtrue\fi + \global\@IEEEissubequationfalse% reset the subequation flag + % reset the number of columns the user actually used + \global\@IEEEeqncolcnt\z@\relax + % the real end of the line + \cr} + + + + + +% \IEEEeqnarraybox is like \IEEEeqnarray except the box form puts everything +% inside a vtop, vbox, or vcenter box depending on the letter in the second +% optional argument (t,b,c). Vbox is the default. Unlike \IEEEeqnarray, +% equation numbers are not displayed and \IEEEeqnarraybox can be nested. +% \IEEEeqnarrayboxm is for math mode (like \array) and does not put the vbox +% within an hbox. +% \IEEEeqnarrayboxt is for text mode (like \tabular) and puts the vbox within +% a \hbox{$ $} construct. +% \IEEEeqnarraybox will auto detect whether to use \IEEEeqnarrayboxm or +% \IEEEeqnarrayboxt depending on the math mode. +% The third optional argument specifies the width this box is to be set to - +% natural width is the default. +% The * forms do not add \jot line spacing +% usage: \IEEEeqnarraybox[decl][pos][width]{cols} +\def\IEEEeqnarrayboxm{\@IEEEeqnarraystarformfalse\@IEEEeqnarrayboxHBOXSWfalse\@IEEEeqnarraybox} +\def\endIEEEeqnarrayboxm{\end@IEEEeqnarraybox} +\@namedef{IEEEeqnarrayboxm*}{\@IEEEeqnarraystarformtrue\@IEEEeqnarrayboxHBOXSWfalse\@IEEEeqnarraybox} +\@namedef{endIEEEeqnarrayboxm*}{\end@IEEEeqnarraybox} + +\def\IEEEeqnarrayboxt{\@IEEEeqnarraystarformfalse\@IEEEeqnarrayboxHBOXSWtrue\@IEEEeqnarraybox} +\def\endIEEEeqnarrayboxt{\end@IEEEeqnarraybox} +\@namedef{IEEEeqnarrayboxt*}{\@IEEEeqnarraystarformtrue\@IEEEeqnarrayboxHBOXSWtrue\@IEEEeqnarraybox} +\@namedef{endIEEEeqnarrayboxt*}{\end@IEEEeqnarraybox} + +\def\IEEEeqnarraybox{\@IEEEeqnarraystarformfalse\ifmmode\@IEEEeqnarrayboxHBOXSWfalse\else\@IEEEeqnarrayboxHBOXSWtrue\fi% +\@IEEEeqnarraybox} +\def\endIEEEeqnarraybox{\end@IEEEeqnarraybox} + +\@namedef{IEEEeqnarraybox*}{\@IEEEeqnarraystarformtrue\ifmmode\@IEEEeqnarrayboxHBOXSWfalse\else\@IEEEeqnarrayboxHBOXSWtrue\fi% +\@IEEEeqnarraybox} +\@namedef{endIEEEeqnarraybox*}{\end@IEEEeqnarraybox} + +% flag to indicate if the \IEEEeqnarraybox needs to put things into an hbox{$ $} +% for \vcenter in non-math mode +\newif\if@IEEEeqnarrayboxHBOXSW% +\@IEEEeqnarrayboxHBOXSWfalse + +\def\@IEEEeqnarraybox{\relax\@ifnextchar[{\@@IEEEeqnarraybox}{\@@IEEEeqnarraybox[\relax]}} +\def\@@IEEEeqnarraybox[#1]{\relax\@ifnextchar[{\@@@IEEEeqnarraybox[#1]}{\@@@IEEEeqnarraybox[#1][b]}} +\def\@@@IEEEeqnarraybox[#1][#2]{\relax\@ifnextchar[{\@@@@IEEEeqnarraybox[#1][#2]}{\@@@@IEEEeqnarraybox[#1][#2][\relax]}} + +% #1 = decl; #2 = t,b,c; #3 = width, #4 = col specs +\def\@@@@IEEEeqnarraybox[#1][#2][#3]#4{\@IEEEeqnarrayISinnerfalse % not yet within the lines of the halign + \@IEEEeqnarraymasterstrutsave% save current master strut values + \@IEEEeqnarraystrutsize{0pt}{0pt}[\relax]% turn off struts by default + \@IEEEeqnarrayusemasterstruttrue% use master strut till user asks otherwise + \IEEEvisiblestrutsfalse% diagnostic mode defaults to off + % no extra space unless the user specifically requests it + \lineskip=0pt\relax% + \lineskiplimit=0pt\relax% + \baselineskip=\normalbaselineskip\relax% + \jot=\IEEEnormaljot\relax% + \mathsurround\z@\relax% no extra spacing around math + % the default end glues are zero for an \IEEEeqnarraybox + \edef\@IEEEeqnarraycolSEPdefaultstart{\@IEEEeqnarraycolSEPzero}% default start glue + \edef\@IEEEeqnarraycolSEPdefaultend{\@IEEEeqnarraycolSEPzero}% default end glue + \edef\@IEEEeqnarraycolSEPdefaultmid{\@IEEEeqnarraycolSEPzero}% default inter-column glue + \@advanceIEEEeqncolcntfalse% do not advance the col counter for each col the user uses, + % used in \IEEEeqnarraymulticol and in the preamble build + \IEEEeqnarrayboxdecl\relax% allow a way for the user to make global overrides + #1\relax% allow user to override defaults + \let\\\@IEEEeqnarrayboxcr% replace newline with one that allows optional spacing + \@IEEEbuildpreamble #4\end\relax% build the preamble and put it into \@IEEEtrantmptoksA + % add an isolation column to the preamble to stop \\'s {} from getting into the last col + \ifnum\@IEEEeqnnumcols>0\relax\@IEEEappendtoksA{&}\fi% col separator for those after the first + \toks0={##}% + % add the isolation column to the preamble + \@IEEEappendtoksA{\tabskip\z@skip\bgroup\the\toks0\egroup}% + % set the starting tabskip glue as determined by the preamble build + \tabskip=\@IEEEBPstartglue\relax + % begin the alignment + \everycr{}% + % use only the very first token to determine the positioning + % this stops some problems when the user uses more than one letter, + % but is probably not worth the effort + % \noindent is used as a delimiter + \def\@IEEEgrabfirstoken##1##2\noindent{\let\@IEEEgrabbedfirstoken=##1}% + \@IEEEgrabfirstoken#2\relax\relax\noindent + % \@IEEEgrabbedfirstoken has the first token, the rest are discarded + % if we need to put things into and hbox and go into math mode, do so now + \if@IEEEeqnarrayboxHBOXSW \leavevmode \hbox \bgroup $\fi% + % use the appropriate vbox type + \if\@IEEEgrabbedfirstoken t\relax\vtop\else\if\@IEEEgrabbedfirstoken c\relax% + \vcenter\else\vbox\fi\fi\bgroup% + \@IEEEeqnarrayISinnertrue% commands are now within the lines + \ifx#3\relax\halign\else\halign to #3\relax\fi% + \bgroup + % "exspand" the preamble + \span\the\@IEEEtrantmptoksA\cr} + +% carry strut status and enter the isolation/strut column, +% exit from math mode if needed, and exit +\def\end@IEEEeqnarraybox{\@IEEEeqnarrayglobalizestrutstatus% carry strut status +&% enter isolation/strut column +\@IEEEeqnarrayinsertstrut% do strut if needed +\@IEEEeqnarraymasterstrutrestore% restore the previous master strut values +% reset the strut system for next IEEEeqnarray +% (sets local strut values back to previous master strut values) +\@IEEEeqnarraystrutreset% +% ensure last line, exit from halign, close vbox +\crcr\egroup\egroup% +% exit from math mode and close hbox if needed +\if@IEEEeqnarrayboxHBOXSW $\egroup\fi} + + + +% IEEEeqnarraybox uses a modifed \\ instead of the plain \cr to +% end rows. This allows for things like \\[vskip amount] +% This "cr" macros are modified versions those for LaTeX2e's eqnarray +% For IEEEeqnarraybox, \\* is the same as \\ +% the {\ifnum0=`} braces must be kept away from the last column to avoid +% altering spacing of its math, so we use & to advance to the isolation/strut column +% carry strut status into isolation/strut column +\def\@IEEEeqnarrayboxcr{\@IEEEeqnarrayglobalizestrutstatus% carry strut status +&% enter isolation/strut column +\@IEEEeqnarrayinsertstrut% do strut if needed +% reset the strut system for next line or IEEEeqnarray +\@IEEEeqnarraystrutreset% +{\ifnum0=`}\fi% +\@ifstar{\@IEEEeqnarrayboxYCR}{\@IEEEeqnarrayboxYCR}} + +% test and setup the optional argument to \\[] +\def\@IEEEeqnarrayboxYCR{\@testopt\@IEEEeqnarrayboxXCR\z@skip} + +% IEEEeqnarraybox does not automatically increase line spacing by \jot +\def\@IEEEeqnarrayboxXCR[#1]{\ifnum0=`{\fi}% +\cr\noalign{\if@IEEEeqnarraystarform\else\vskip\jot\fi\vskip#1\relax}} + + + +% starts the halign preamble build +\def\@IEEEbuildpreamble{\@IEEEtrantmptoksA={}% clear token register +\let\@IEEEBPcurtype=u%current column type is not yet known +\let\@IEEEBPprevtype=s%the previous column type was the start +\let\@IEEEBPnexttype=u%next column type is not yet known +% ensure these are valid +\def\@IEEEBPcurglue={0pt plus 0pt minus 0pt}% +\def\@IEEEBPcurcolname{@IEEEdefault}% name of current column definition +% currently acquired numerically referenced glue +% use a name that is easier to remember +\let\@IEEEBPcurnum=\@IEEEtrantmpcountA% +\@IEEEBPcurnum=0% +% tracks number of columns in the preamble +\@IEEEeqnnumcols=0% +% record the default end glues +\edef\@IEEEBPstartglue{\@IEEEeqnarraycolSEPdefaultstart}% +\edef\@IEEEBPendglue{\@IEEEeqnarraycolSEPdefaultend}% +% now parse the user's column specifications +\@@IEEEbuildpreamble} + + +% parses and builds the halign preamble +\def\@@IEEEbuildpreamble#1#2{\let\@@nextIEEEbuildpreamble=\@@IEEEbuildpreamble% +% use only the very first token to check the end +% \noindent is used as a delimiter as \end can be present here +\def\@IEEEgrabfirstoken##1##2\noindent{\let\@IEEEgrabbedfirstoken=##1}% +\@IEEEgrabfirstoken#1\relax\relax\noindent +\ifx\@IEEEgrabbedfirstoken\end\let\@@nextIEEEbuildpreamble=\@@IEEEfinishpreamble\else% +% identify current and next token type +\@IEEEgetcoltype{#1}{\@IEEEBPcurtype}{1}% current, error on invalid +\@IEEEgetcoltype{#2}{\@IEEEBPnexttype}{0}% next, no error on invalid next +% if curtype is a glue, get the glue def +\if\@IEEEBPcurtype g\@IEEEgetcurglue{#1}{\@IEEEBPcurglue}\fi% +% if curtype is a column, get the column def and set the current column name +\if\@IEEEBPcurtype c\@IEEEgetcurcol{#1}\fi% +% if curtype is a numeral, acquire the user defined glue +\if\@IEEEBPcurtype n\@IEEEprocessNcol{#1}\fi% +% process the acquired glue +\if\@IEEEBPcurtype g\@IEEEprocessGcol\fi% +% process the acquired col +\if\@IEEEBPcurtype c\@IEEEprocessCcol\fi% +% ready prevtype for next col spec. +\let\@IEEEBPprevtype=\@IEEEBPcurtype% +% be sure and put back the future token(s) as a group +\fi\@@nextIEEEbuildpreamble{#2}} + + +% executed just after preamble build is completed +% warn about zero cols, and if prevtype type = u, put in end tabskip glue +\def\@@IEEEfinishpreamble#1{\ifnum\@IEEEeqnnumcols<1\relax +\@IEEEclspkgerror{No column specifiers declared for IEEEeqnarray}% +{At least one column type must be declared for each IEEEeqnarray.}% +\fi%num cols less than 1 +%if last type undefined, set default end tabskip glue +\if\@IEEEBPprevtype u\@IEEEappendtoksA{\tabskip=\@IEEEBPendglue}\fi} + + +% Identify and return the column specifier's type code +\def\@IEEEgetcoltype#1#2#3{% +% use only the very first token to determine the type +% \noindent is used as a delimiter as \end can be present here +\def\@IEEEgrabfirstoken##1##2\noindent{\let\@IEEEgrabbedfirstoken=##1}% +\@IEEEgrabfirstoken#1\relax\relax\noindent +% \@IEEEgrabfirstoken has the first token, the rest are discarded +% n = number +% g = glue (any other char in catagory 12) +% c = letter +% e = \end +% u = undefined +% third argument: 0 = no error message, 1 = error on invalid char +\let#2=u\relax% assume invalid until know otherwise +\ifx\@IEEEgrabbedfirstoken\end\let#2=e\else +\ifcat\@IEEEgrabbedfirstoken\relax\else% screen out control sequences +\if0\@IEEEgrabbedfirstoken\let#2=n\else +\if1\@IEEEgrabbedfirstoken\let#2=n\else +\if2\@IEEEgrabbedfirstoken\let#2=n\else +\if3\@IEEEgrabbedfirstoken\let#2=n\else +\if4\@IEEEgrabbedfirstoken\let#2=n\else +\if5\@IEEEgrabbedfirstoken\let#2=n\else +\if6\@IEEEgrabbedfirstoken\let#2=n\else +\if7\@IEEEgrabbedfirstoken\let#2=n\else +\if8\@IEEEgrabbedfirstoken\let#2=n\else +\if9\@IEEEgrabbedfirstoken\let#2=n\else +\ifcat,\@IEEEgrabbedfirstoken\let#2=g\relax +\else\ifcat a\@IEEEgrabbedfirstoken\let#2=c\relax\fi\fi\fi\fi\fi\fi\fi\fi\fi\fi\fi\fi\fi\fi +\if#2u\relax +\if0\noexpand#3\relax\else\@IEEEclspkgerror{Invalid character in column specifications}% +{Only letters, numerals and certain other symbols are allowed \MessageBreak +as IEEEeqnarray column specifiers.}\fi\fi} + + +% identify the current letter referenced column +% if invalid, use a default column +\def\@IEEEgetcurcol#1{\expandafter\ifx\csname @IEEEeqnarraycolDEF#1\endcsname\@IEEEeqnarraycolisdefined% +\def\@IEEEBPcurcolname{#1}\else% invalid column name +\@IEEEclspkgerror{Invalid column type "#1" in column specifications.\MessageBreak +Using a default centering column instead}% +{You must define IEEEeqnarray column types before use.}% +\def\@IEEEBPcurcolname{@IEEEdefault}\fi} + + +% identify and return the predefined (punctuation) glue value +\def\@IEEEgetcurglue#1#2{% +% ! = \! (neg small) -0.16667em (-3/18 em) +% , = \, (small) 0.16667em ( 3/18 em) +% : = \: (med) 0.22222em ( 4/18 em) +% ; = \; (large) 0.27778em ( 5/18 em) +% ' = \quad 1em +% " = \qquad 2em +% . = 0.5\arraycolsep +% / = \arraycolsep +% ? = 2\arraycolsep +% * = 1fil +% + = \@IEEEeqnarraycolSEPcenter +% - = \@IEEEeqnarraycolSEPzero +% Note that all em values are referenced to the math font (textfont2) fontdimen6 +% value for 1em. +% +% use only the very first token to determine the type +% this prevents errant tokens from getting in the main text +% \noindent is used as a delimiter here +\def\@IEEEgrabfirstoken##1##2\noindent{\let\@IEEEgrabbedfirstoken=##1}% +\@IEEEgrabfirstoken#1\relax\relax\noindent +% get the math font 1em value +% LaTeX2e's NFSS2 does not preload the fonts, but \IEEEeqnarray needs +% to gain access to the math (\textfont2) font's spacing parameters. +% So we create a bogus box here that uses the math font to ensure +% that \textfont2 is loaded and ready. If this is not done, +% the \textfont2 stuff here may not work. +% Thanks to Bernd Raichle for his 1997 post on this topic. +{\setbox0=\hbox{$\displaystyle\relax$}}% +% fontdimen6 has the width of 1em (a quad). +\@IEEEtrantmpdimenA=\fontdimen6\textfont2\relax% +% identify the glue value based on the first token +% we discard anything after the first +\if!\@IEEEgrabbedfirstoken\@IEEEtrantmpdimenA=-0.16667\@IEEEtrantmpdimenA\edef#2{\the\@IEEEtrantmpdimenA}\else +\if,\@IEEEgrabbedfirstoken\@IEEEtrantmpdimenA=0.16667\@IEEEtrantmpdimenA\edef#2{\the\@IEEEtrantmpdimenA}\else +\if:\@IEEEgrabbedfirstoken\@IEEEtrantmpdimenA=0.22222\@IEEEtrantmpdimenA\edef#2{\the\@IEEEtrantmpdimenA}\else +\if;\@IEEEgrabbedfirstoken\@IEEEtrantmpdimenA=0.27778\@IEEEtrantmpdimenA\edef#2{\the\@IEEEtrantmpdimenA}\else +\if'\@IEEEgrabbedfirstoken\@IEEEtrantmpdimenA=1\@IEEEtrantmpdimenA\edef#2{\the\@IEEEtrantmpdimenA}\else +\if"\@IEEEgrabbedfirstoken\@IEEEtrantmpdimenA=2\@IEEEtrantmpdimenA\edef#2{\the\@IEEEtrantmpdimenA}\else +\if.\@IEEEgrabbedfirstoken\@IEEEtrantmpdimenA=0.5\arraycolsep\edef#2{\the\@IEEEtrantmpdimenA}\else +\if/\@IEEEgrabbedfirstoken\edef#2{\the\arraycolsep}\else +\if?\@IEEEgrabbedfirstoken\@IEEEtrantmpdimenA=2\arraycolsep\edef#2{\the\@IEEEtrantmpdimenA}\else +\if *\@IEEEgrabbedfirstoken\edef#2{0pt plus 1fil minus 0pt}\else +\if+\@IEEEgrabbedfirstoken\edef#2{\@IEEEeqnarraycolSEPcenter}\else +\if-\@IEEEgrabbedfirstoken\edef#2{\@IEEEeqnarraycolSEPzero}\else +\edef#2{\@IEEEeqnarraycolSEPzero}% +\@IEEEclspkgerror{Invalid predefined inter-column glue type "#1" in\MessageBreak +column specifications. Using a default value of\MessageBreak +0pt instead}% +{Only !,:;'"./?*+ and - are valid predefined glue types in the\MessageBreak +IEEEeqnarray column specifications.}\fi\fi\fi\fi\fi\fi\fi\fi\fi\fi\fi\fi} + + + +% process a numerical digit from the column specification +% and look up the corresponding user defined glue value +% can transform current type from n to g or a as the user defined glue is acquired +\def\@IEEEprocessNcol#1{\if\@IEEEBPprevtype g% +\@IEEEclspkgerror{Back-to-back inter-column glue specifiers in column\MessageBreak +specifications. Ignoring consecutive glue specifiers\MessageBreak +after the first}% +{You cannot have two or more glue types next to each other\MessageBreak +in the IEEEeqnarray column specifications.}% +\let\@IEEEBPcurtype=a% abort this glue, future digits will be discarded +\@IEEEBPcurnum=0\relax% +\else% if we previously aborted a glue +\if\@IEEEBPprevtype a\@IEEEBPcurnum=0\let\@IEEEBPcurtype=a%maintain digit abortion +\else%acquire this number +% save the previous type before the numerical digits started +\if\@IEEEBPprevtype n\else\let\@IEEEBPprevsavedtype=\@IEEEBPprevtype\fi% +\multiply\@IEEEBPcurnum by 10\relax% +\advance\@IEEEBPcurnum by #1\relax% add in number, \relax is needed to stop TeX's number scan +\if\@IEEEBPnexttype n\else%close acquisition +\expandafter\ifx\csname @IEEEeqnarraycolSEPDEF\expandafter\romannumeral\number\@IEEEBPcurnum\endcsname\@IEEEeqnarraycolisdefined% +\edef\@IEEEBPcurglue{\csname @IEEEeqnarraycolSEP\expandafter\romannumeral\number\@IEEEBPcurnum\endcsname}% +\else%user glue not defined +\@IEEEclspkgerror{Invalid user defined inter-column glue type "\number\@IEEEBPcurnum" in\MessageBreak +column specifications. Using a default value of\MessageBreak +0pt instead}% +{You must define all IEEEeqnarray numerical inter-column glue types via\MessageBreak +\string\IEEEeqnarraydefcolsep \space before they are used in column specifications.}% +\edef\@IEEEBPcurglue{\@IEEEeqnarraycolSEPzero}% +\fi% glue defined or not +\let\@IEEEBPcurtype=g% change the type to reflect the acquired glue +\let\@IEEEBPprevtype=\@IEEEBPprevsavedtype% restore the prev type before this number glue +\@IEEEBPcurnum=0\relax%ready for next acquisition +\fi%close acquisition, get glue +\fi%discard or acquire number +\fi%prevtype glue or not +} + + +% process an acquired glue +% add any acquired column/glue pair to the preamble +\def\@IEEEprocessGcol{\if\@IEEEBPprevtype a\let\@IEEEBPcurtype=a%maintain previous glue abortions +\else +% if this is the start glue, save it, but do nothing else +% as this is not used in the preamble, but before +\if\@IEEEBPprevtype s\edef\@IEEEBPstartglue{\@IEEEBPcurglue}% +\else%not the start glue +\if\@IEEEBPprevtype g%ignore if back to back glues +\@IEEEclspkgerror{Back-to-back inter-column glue specifiers in column\MessageBreak +specifications. Ignoring consecutive glue specifiers\MessageBreak +after the first}% +{You cannot have two or more glue types next to each other\MessageBreak +in the IEEEeqnarray column specifications.}% +\let\@IEEEBPcurtype=a% abort this glue +\else% not a back to back glue +\if\@IEEEBPprevtype c\relax% if the previoustype was a col, add column/glue pair to preamble +\ifnum\@IEEEeqnnumcols>0\relax\@IEEEappendtoksA{&}\fi +\toks0={##}% +% make preamble advance col counter if this environment needs this +\if@advanceIEEEeqncolcnt\@IEEEappendtoksA{\global\advance\@IEEEeqncolcnt by 1\relax}\fi +% insert the column defintion into the preamble, being careful not to expand +% the column definition +\@IEEEappendtoksA{\tabskip=\@IEEEBPcurglue}% +\@IEEEappendNOEXPANDtoksA{\begingroup\csname @IEEEeqnarraycolPRE}% +\@IEEEappendtoksA{\@IEEEBPcurcolname}% +\@IEEEappendNOEXPANDtoksA{\endcsname}% +\@IEEEappendtoksA{\the\toks0}% +\@IEEEappendNOEXPANDtoksA{\relax\relax\relax\relax\relax% +\relax\relax\relax\relax\relax\csname @IEEEeqnarraycolPOST}% +\@IEEEappendtoksA{\@IEEEBPcurcolname}% +\@IEEEappendNOEXPANDtoksA{\endcsname\relax\relax\relax\relax\relax% +\relax\relax\relax\relax\relax\endgroup}% +\advance\@IEEEeqnnumcols by 1\relax%one more column in the preamble +\else% error: non-start glue with no pending column +\@IEEEclspkgerror{Inter-column glue specifier without a prior column\MessageBreak +type in the column specifications. Ignoring this glue\MessageBreak +specifier}% +{Except for the first and last positions, glue can be placed only\MessageBreak +between column types.}% +\let\@IEEEBPcurtype=a% abort this glue +\fi% previous was a column +\fi% back-to-back glues +\fi% is start column glue +\fi% prev type not a +} + + +% process an acquired letter referenced column and, if necessary, add it to the preamble +\def\@IEEEprocessCcol{\if\@IEEEBPnexttype g\else +\if\@IEEEBPnexttype n\else +% we have a column followed by something other than a glue (or numeral glue) +% so we must add this column to the preamble now +\ifnum\@IEEEeqnnumcols>0\relax\@IEEEappendtoksA{&}\fi%col separator for those after the first +\if\@IEEEBPnexttype e\@IEEEappendtoksA{\tabskip=\@IEEEBPendglue\relax}\else%put in end glue +\@IEEEappendtoksA{\tabskip=\@IEEEeqnarraycolSEPdefaultmid\relax}\fi% or default mid glue +\toks0={##}% +% make preamble advance col counter if this environment needs this +\if@advanceIEEEeqncolcnt\@IEEEappendtoksA{\global\advance\@IEEEeqncolcnt by 1\relax}\fi +% insert the column definition into the preamble, being careful not to expand +% the column definition +\@IEEEappendNOEXPANDtoksA{\begingroup\csname @IEEEeqnarraycolPRE}% +\@IEEEappendtoksA{\@IEEEBPcurcolname}% +\@IEEEappendNOEXPANDtoksA{\endcsname}% +\@IEEEappendtoksA{\the\toks0}% +\@IEEEappendNOEXPANDtoksA{\relax\relax\relax\relax\relax% +\relax\relax\relax\relax\relax\csname @IEEEeqnarraycolPOST}% +\@IEEEappendtoksA{\@IEEEBPcurcolname}% +\@IEEEappendNOEXPANDtoksA{\endcsname\relax\relax\relax\relax\relax% +\relax\relax\relax\relax\relax\endgroup}% +\advance\@IEEEeqnnumcols by 1\relax%one more column in the preamble +\fi%next type not numeral +\fi%next type not glue +} + + +%% +%% END OF IEEEeqnarry DEFINITIONS +%% + + + + +% set up the running headings, this complex because of all the different +% modes IEEEtran supports +\if@twoside + \ifCLASSOPTIONtechnote + \def\ps@headings{% + \def\@oddhead{\hbox{}\scriptsize\leftmark \hfil \thepage} + \def\@evenhead{\scriptsize\thepage \hfil \leftmark\hbox{}} + \ifCLASSOPTIONdraftcls + \ifCLASSOPTIONdraftclsnofoot + \def\@oddfoot{}\def\@evenfoot{}% + \else + \def\@oddfoot{\scriptsize\@date\hfil DRAFT} + \def\@evenfoot{\scriptsize DRAFT\hfil\@date} + \fi + \else + \def\@oddfoot{}\def\@evenfoot{} + \fi} + \else % not a technote + \def\ps@headings{% + \ifCLASSOPTIONconference + \def\@oddhead{} + \def\@evenhead{} + \else + \def\@oddhead{\hbox{}\scriptsize\rightmark \hfil \thepage} + \def\@evenhead{\scriptsize\thepage \hfil \leftmark\hbox{}} + \fi + \ifCLASSOPTIONdraftcls + \def\@oddhead{\hbox{}\scriptsize\rightmark \hfil \thepage} + \def\@evenhead{\scriptsize\thepage \hfil \leftmark\hbox{}} + \ifCLASSOPTIONdraftclsnofoot + \def\@oddfoot{}\def\@evenfoot{}% + \else + \def\@oddfoot{\scriptsize\@date\hfil DRAFT} + \def\@evenfoot{\scriptsize DRAFT\hfil\@date} + \fi + \else + \def\@oddfoot{}\def\@evenfoot{}% + \fi} + \fi +\else % single side +\def\ps@headings{% + \ifCLASSOPTIONconference + \def\@oddhead{} + \def\@evenhead{} + \else + \def\@oddhead{\hbox{}\scriptsize\leftmark \hfil \thepage} + \def\@evenhead{} + \fi + \ifCLASSOPTIONdraftcls + \def\@oddhead{\hbox{}\scriptsize\leftmark \hfil \thepage} + \def\@evenhead{} + \ifCLASSOPTIONdraftclsnofoot + \def\@oddfoot{} + \else + \def\@oddfoot{\scriptsize \@date \hfil DRAFT} + \fi + \else + \def\@oddfoot{} + \fi + \def\@evenfoot{}} +\fi + + +% title page style +\def\ps@IEEEtitlepagestyle{\def\@oddfoot{}\def\@evenfoot{}% +\ifCLASSOPTIONconference + \def\@oddhead{}% + \def\@evenhead{}% +\else + \def\@oddhead{\hbox{}\scriptsize\leftmark \hfil \thepage}% + \def\@evenhead{\scriptsize\thepage \hfil \leftmark\hbox{}}% +\fi +\ifCLASSOPTIONdraftcls + \def\@oddhead{\hbox{}\scriptsize\leftmark \hfil \thepage}% + \def\@evenhead{\scriptsize\thepage \hfil \leftmark\hbox{}}% + \ifCLASSOPTIONdraftclsnofoot\else + \def\@oddfoot{\scriptsize \@date\hfil DRAFT}% + \def\@evenfoot{\scriptsize DRAFT\hfil \@date}% + \fi +\else + % all non-draft mode footers + \if@IEEEusingpubid + % for title pages that are using a pubid + % do not repeat pubid if using peer review option + \ifCLASSOPTIONpeerreview + \else + \footskip 0pt% + \ifCLASSOPTIONcompsoc + \def\@oddfoot{\hss\normalfont\scriptsize\raisebox{-1.5\@IEEEnormalsizeunitybaselineskip}[0ex][0ex]{\@IEEEpubid}\hss}% + \def\@evenfoot{\hss\normalfont\scriptsize\raisebox{-1.5\@IEEEnormalsizeunitybaselineskip}[0ex][0ex]{\@IEEEpubid}\hss}% + \else + \def\@oddfoot{\hss\normalfont\footnotesize\raisebox{1.5ex}[1.5ex]{\@IEEEpubid}\hss}% + \def\@evenfoot{\hss\normalfont\footnotesize\raisebox{1.5ex}[1.5ex]{\@IEEEpubid}\hss}% + \fi + \fi + \fi +\fi} + + +% peer review cover page style +\def\ps@IEEEpeerreviewcoverpagestyle{% +\def\@oddhead{}\def\@evenhead{}% +\def\@oddfoot{}\def\@evenfoot{}% +\ifCLASSOPTIONdraftcls + \ifCLASSOPTIONdraftclsnofoot\else + \def\@oddfoot{\scriptsize \@date\hfil DRAFT}% + \def\@evenfoot{\scriptsize DRAFT\hfil \@date}% + \fi +\else + % non-draft mode footers + \if@IEEEusingpubid + \footskip 0pt% + \ifCLASSOPTIONcompsoc + \def\@oddfoot{\hss\normalfont\scriptsize\raisebox{-1.5\@IEEEnormalsizeunitybaselineskip}[0ex][0ex]{\@IEEEpubid}\hss}% + \def\@evenfoot{\hss\normalfont\scriptsize\raisebox{-1.5\@IEEEnormalsizeunitybaselineskip}[0ex][0ex]{\@IEEEpubid}\hss}% + \else + \def\@oddfoot{\hss\normalfont\footnotesize\raisebox{1.5ex}[1.5ex]{\@IEEEpubid}\hss}% + \def\@evenfoot{\hss\normalfont\footnotesize\raisebox{1.5ex}[1.5ex]{\@IEEEpubid}\hss}% + \fi + \fi +\fi} + + +% start with empty headings +\def\rightmark{}\def\leftmark{} + + +%% Defines the command for putting the header. \footernote{TEXT} is the same +%% as \markboth{TEXT}{TEXT}. +%% Note that all the text is forced into uppercase, if you have some text +%% that needs to be in lower case, for instance et. al., then either manually +%% set \leftmark and \rightmark or use \MakeLowercase{et. al.} within the +%% arguments to \markboth. +\def\markboth#1#2{\def\leftmark{\@IEEEcompsoconly{\sffamily}\MakeUppercase{#1}}% +\def\rightmark{\@IEEEcompsoconly{\sffamily}\MakeUppercase{#2}}} +\def\footernote#1{\markboth{#1}{#1}} + +\def\today{\ifcase\month\or + January\or February\or March\or April\or May\or June\or + July\or August\or September\or October\or November\or December\fi + \space\number\day, \number\year} + + + + +%% CITATION AND BIBLIOGRAPHY COMMANDS +%% +%% V1.6 no longer supports the older, nonstandard \shortcite and \citename setup stuff +% +% +% Modify Latex2e \@citex to separate citations with "], [" +\def\@citex[#1]#2{% + \let\@citea\@empty + \@cite{\@for\@citeb:=#2\do + {\@citea\def\@citea{], [}% + \edef\@citeb{\expandafter\@firstofone\@citeb\@empty}% + \if@filesw\immediate\write\@auxout{\string\citation{\@citeb}}\fi + \@ifundefined{b@\@citeb}{\mbox{\reset@font\bfseries ?}% + \G@refundefinedtrue + \@latex@warning + {Citation `\@citeb' on page \thepage \space undefined}}% + {\hbox{\csname b@\@citeb\endcsname}}}}{#1}} + +% V1.6 we create hooks for the optional use of Donald Arseneau's +% cite.sty package. cite.sty is "smart" and will notice that the +% following format controls are already defined and will not +% redefine them. The result will be the proper sorting of the +% citation numbers and auto detection of 3 or more entry "ranges" - +% all in IEEE style: [1], [2], [5]--[7], [12] +% This also allows for an optional note, i.e., \cite[mynote]{..}. +% If the \cite with note has more than one reference, the note will +% be applied to the last of the listed references. It is generally +% desired that if a note is given, only one reference is listed in +% that \cite. +% Thanks to Mr. Arseneau for providing the required format arguments +% to produce the IEEE style. +\def\citepunct{], [} +\def\citedash{]--[} + +% V1.7 default to using same font for urls made by url.sty +\AtBeginDocument{\csname url@samestyle\endcsname} + +% V1.6 class files should always provide these +\def\newblock{\hskip .11em\@plus.33em\@minus.07em} +\let\@openbib@code\@empty + + +% Provide support for the control entries of IEEEtran.bst V1.00 and later. +% V1.7 optional argument allows for a different aux file to be specified in +% order to handle multiple bibliographies. For example, with multibib.sty: +% \newcites{sec}{Secondary Literature} +% \bstctlcite[@auxoutsec]{BSTcontrolhak} +\def\bstctlcite{\@ifnextchar[{\@bstctlcite}{\@bstctlcite[@auxout]}} +\def\@bstctlcite[#1]#2{\@bsphack + \@for\@citeb:=#2\do{% + \edef\@citeb{\expandafter\@firstofone\@citeb}% + \if@filesw\immediate\write\csname #1\endcsname{\string\citation{\@citeb}}\fi}% + \@esphack} + +% V1.6 provide a way for a user to execute a command just before +% a given reference number - used to insert a \newpage to balance +% the columns on the last page +\edef\@IEEEtriggerrefnum{0} % the default of zero means that + % the command is not executed +\def\@IEEEtriggercmd{\newpage} + +% allow the user to alter the triggered command +\long\def\IEEEtriggercmd#1{\long\def\@IEEEtriggercmd{#1}} + +% allow user a way to specify the reference number just before the +% command is executed +\def\IEEEtriggeratref#1{\@IEEEtrantmpcountA=#1% +\edef\@IEEEtriggerrefnum{\the\@IEEEtrantmpcountA}}% + +% trigger command at the given reference +\def\@IEEEbibitemprefix{\@IEEEtrantmpcountA=\@IEEEtriggerrefnum\relax% +\advance\@IEEEtrantmpcountA by -1\relax% +\ifnum\c@enumiv=\@IEEEtrantmpcountA\relax\@IEEEtriggercmd\relax\fi} + + +\def\@biblabel#1{[#1]} + +% compsoc journals left align the reference numbers +\@IEEEcompsocnotconfonly{\def\@biblabel#1{[#1]\hfill}} + +% controls bib item spacing +\def\IEEEbibitemsep{2.5pt plus .5pt} + +\@IEEEcompsocconfonly{\def\IEEEbibitemsep{1\baselineskip plus 0.25\baselineskip minus 0.25\baselineskip}} + + +\def\thebibliography#1{\section*{\refname}% + \addcontentsline{toc}{section}{\refname}% + % V1.6 add some rubber space here and provide a command trigger + \footnotesize\@IEEEcompsocconfonly{\small}\vskip 0.3\baselineskip plus 0.1\baselineskip minus 0.1\baselineskip% + \list{\@biblabel{\@arabic\c@enumiv}}% + {\settowidth\labelwidth{\@biblabel{#1}}% + \leftmargin\labelwidth + \labelsep 1em + \advance\leftmargin\labelsep\relax + \itemsep \IEEEbibitemsep\relax + \usecounter{enumiv}% + \let\p@enumiv\@empty + \renewcommand\theenumiv{\@arabic\c@enumiv}}% + \let\@IEEElatexbibitem\bibitem% + \def\bibitem{\@IEEEbibitemprefix\@IEEElatexbibitem}% +\def\newblock{\hskip .11em plus .33em minus .07em}% +% originally: +% \sloppy\clubpenalty4000\widowpenalty4000% +% by adding the \interlinepenalty here, we make it more +% difficult, but not impossible, for LaTeX to break within a reference. +% IEEE almost never breaks a reference (but they do it more often with +% technotes). You may get an underfull vbox warning around the bibliography, +% but the final result will be much more like what IEEE will publish. +% MDS 11/2000 +\ifCLASSOPTIONtechnote\sloppy\clubpenalty4000\widowpenalty4000\interlinepenalty100% +\else\sloppy\clubpenalty4000\widowpenalty4000\interlinepenalty500\fi% + \sfcode`\.=1000\relax} +\let\endthebibliography=\endlist + + + + +% TITLE PAGE COMMANDS +% +% +% \IEEEmembership is used to produce the sublargesize italic font used to indicate author +% IEEE membership. compsoc uses a large size sans slant font +\def\IEEEmembership#1{{\@IEEEnotcompsoconly{\sublargesize}\normalfont\@IEEEcompsoconly{\sffamily}\textit{#1}}} + + +% \IEEEauthorrefmark{} produces a footnote type symbol to indicate author affiliation. +% When given an argument of 1 to 9, \IEEEauthorrefmark{} follows the standard LaTeX footnote +% symbol sequence convention. However, for arguments 10 and above, \IEEEauthorrefmark{} +% reverts to using lower case roman numerals, so it cannot overflow. Do note that you +% cannot use \footnotemark[] in place of \IEEEauthorrefmark{} within \author as the footnote +% symbols will have been turned off to prevent \thanks from creating footnote marks. +% \IEEEauthorrefmark{} produces a symbol that appears to LaTeX as having zero vertical +% height - this allows for a more compact line packing, but the user must ensure that +% the interline spacing is large enough to prevent \IEEEauthorrefmark{} from colliding +% with the text above. +% V1.7 make this a robust command +\DeclareRobustCommand*{\IEEEauthorrefmark}[1]{\raisebox{0pt}[0pt][0pt]{\textsuperscript{\footnotesize\ensuremath{\ifcase#1\or *\or \dagger\or \ddagger\or% + \mathsection\or \mathparagraph\or \|\or **\or \dagger\dagger% + \or \ddagger\ddagger \else\textsuperscript{\expandafter\romannumeral#1}\fi}}}} + + +% FONT CONTROLS AND SPACINGS FOR CONFERENCE MODE AUTHOR NAME AND AFFILIATION BLOCKS +% +% The default font styles for the author name and affiliation blocks (confmode) +\def\@IEEEauthorblockNstyle{\normalfont\@IEEEcompsocnotconfonly{\sffamily}\sublargesize\@IEEEcompsocconfonly{\large}} +\def\@IEEEauthorblockAstyle{\normalfont\@IEEEcompsocnotconfonly{\sffamily}\@IEEEcompsocconfonly{\itshape}\normalsize\@IEEEcompsocconfonly{\large}} +% The default if the user does not use an author block +\def\@IEEEauthordefaulttextstyle{\normalfont\@IEEEcompsocnotconfonly{\sffamily}\sublargesize} + +% spacing from title (or special paper notice) to author name blocks (confmode) +% can be negative +\def\@IEEEauthorblockconfadjspace{-0.25em} +% compsoc conferences need more space here +\@IEEEcompsocconfonly{\def\@IEEEauthorblockconfadjspace{0.75\@IEEEnormalsizeunitybaselineskip}} +\ifCLASSOPTIONconference\def\@IEEEauthorblockconfadjspace{20pt}\fi + +% spacing between name and affiliation blocks (confmode) +% This can be negative. +% IEEE doesn't want any added spacing here, but I will leave these +% controls in place in case they ever change their mind. +% Personally, I like 0.75ex. +%\def\@IEEEauthorblockNtopspace{0.75ex} +%\def\@IEEEauthorblockAtopspace{0.75ex} +\def\@IEEEauthorblockNtopspace{0.0ex} +\def\@IEEEauthorblockAtopspace{0.0ex} +% baseline spacing within name and affiliation blocks (confmode) +% must be positive, spacings below certain values will make +% the position of line of text sensitive to the contents of the +% line above it i.e., whether or not the prior line has descenders, +% subscripts, etc. For this reason it is a good idea to keep +% these above 2.6ex +\def\@IEEEauthorblockNinterlinespace{2.6ex} +\def\@IEEEauthorblockAinterlinespace{2.75ex} + +% This tracks the required strut size. +% See the \@IEEEauthorhalign command for the actual default value used. +\def\@IEEEauthorblockXinterlinespace{2.7ex} + +% variables to retain font size and style across groups +% values given here have no effect as they will be overwritten later +\gdef\@IEEESAVESTATEfontsize{10} +\gdef\@IEEESAVESTATEfontbaselineskip{12} +\gdef\@IEEESAVESTATEfontencoding{OT1} +\gdef\@IEEESAVESTATEfontfamily{ptm} +\gdef\@IEEESAVESTATEfontseries{m} +\gdef\@IEEESAVESTATEfontshape{n} + +% saves the current font attributes +\def\@IEEEcurfontSAVE{\global\let\@IEEESAVESTATEfontsize\f@size% +\global\let\@IEEESAVESTATEfontbaselineskip\f@baselineskip% +\global\let\@IEEESAVESTATEfontencoding\f@encoding% +\global\let\@IEEESAVESTATEfontfamily\f@family% +\global\let\@IEEESAVESTATEfontseries\f@series% +\global\let\@IEEESAVESTATEfontshape\f@shape} + +% restores the saved font attributes +\def\@IEEEcurfontRESTORE{\fontsize{\@IEEESAVESTATEfontsize}{\@IEEESAVESTATEfontbaselineskip}% +\fontencoding{\@IEEESAVESTATEfontencoding}% +\fontfamily{\@IEEESAVESTATEfontfamily}% +\fontseries{\@IEEESAVESTATEfontseries}% +\fontshape{\@IEEESAVESTATEfontshape}% +\selectfont} + + +% variable to indicate if the current block is the first block in the column +\newif\if@IEEEprevauthorblockincol \@IEEEprevauthorblockincolfalse + + +% the command places a strut with height and depth = \@IEEEauthorblockXinterlinespace +% we use this technique to have complete manual control over the spacing of the lines +% within the halign environment. +% We set the below baseline portion at 30%, the above +% baseline portion at 70% of the total length. +% Responds to changes in the document's \baselinestretch +\def\@IEEEauthorstrutrule{\@IEEEtrantmpdimenA\@IEEEauthorblockXinterlinespace% +\@IEEEtrantmpdimenA=\baselinestretch\@IEEEtrantmpdimenA% +\rule[-0.3\@IEEEtrantmpdimenA]{0pt}{\@IEEEtrantmpdimenA}} + + +% blocks to hold the authors' names and affilations. +% Makes formatting easy for conferences +% +% use real definitions in conference mode +% name block +\def\IEEEauthorblockN#1{\relax\@IEEEauthorblockNstyle% set the default text style +\gdef\@IEEEauthorblockXinterlinespace{0pt}% disable strut for spacer row +% the \expandafter hides the \cr in conditional tex, see the array.sty docs +% for details, probably not needed here as the \cr is in a macro +% do a spacer row if needed +\if@IEEEprevauthorblockincol\expandafter\@IEEEauthorblockNtopspaceline\fi +\global\@IEEEprevauthorblockincoltrue% we now have a block in this column +%restore the correct strut value +\gdef\@IEEEauthorblockXinterlinespace{\@IEEEauthorblockNinterlinespace}% +% input the author names +#1% +% end the row if the user did not already +\crcr} +% spacer row for names +\def\@IEEEauthorblockNtopspaceline{\cr\noalign{\vskip\@IEEEauthorblockNtopspace}} +% +% affiliation block +\def\IEEEauthorblockA#1{\relax\@IEEEauthorblockAstyle% set the default text style +\gdef\@IEEEauthorblockXinterlinespace{0pt}%disable strut for spacer row +% the \expandafter hides the \cr in conditional tex, see the array.sty docs +% for details, probably not needed here as the \cr is in a macro +% do a spacer row if needed +\if@IEEEprevauthorblockincol\expandafter\@IEEEauthorblockAtopspaceline\fi +\global\@IEEEprevauthorblockincoltrue% we now have a block in this column +%restore the correct strut value +\gdef\@IEEEauthorblockXinterlinespace{\@IEEEauthorblockAinterlinespace}% +% input the author affiliations +#1% +% end the row if the user did not already +\crcr} +% spacer row for affiliations +\def\@IEEEauthorblockAtopspaceline{\cr\noalign{\vskip\@IEEEauthorblockAtopspace}} + + +% allow papers to compile even if author blocks are used in modes other +% than conference or peerreviewca. For such cases, we provide dummy blocks. +\ifCLASSOPTIONconference +\else + \ifCLASSOPTIONpeerreviewca\else + % not conference or peerreviewca mode + \def\IEEEauthorblockN#1{#1}% + \def\IEEEauthorblockA#1{#1}% + \fi +\fi + + + +% we provide our own halign so as not to have to depend on tabular +\def\@IEEEauthorhalign{\@IEEEauthordefaulttextstyle% default text style + \lineskip=0pt\relax% disable line spacing + \lineskiplimit=0pt\relax% + \baselineskip=0pt\relax% + \@IEEEcurfontSAVE% save the current font + \mathsurround\z@\relax% no extra spacing around math + \let\\\@IEEEauthorhaligncr% replace newline with halign friendly one + \tabskip=0pt\relax% no column spacing + \everycr{}% ensure no problems here + \@IEEEprevauthorblockincolfalse% no author blocks yet + \def\@IEEEauthorblockXinterlinespace{2.7ex}% default interline space + \vtop\bgroup%vtop box + \halign\bgroup&\relax\hfil\@IEEEcurfontRESTORE\relax ##\relax + \hfil\@IEEEcurfontSAVE\@IEEEauthorstrutrule\cr} + +% ensure last line, exit from halign, close vbox +\def\end@IEEEauthorhalign{\crcr\egroup\egroup} + +% handle bogus star form +\def\@IEEEauthorhaligncr{{\ifnum0=`}\fi\@ifstar{\@@IEEEauthorhaligncr}{\@@IEEEauthorhaligncr}} + +% test and setup the optional argument to \\[] +\def\@@IEEEauthorhaligncr{\@testopt\@@@IEEEauthorhaligncr\z@skip} + +% end the line and do the optional spacer +\def\@@@IEEEauthorhaligncr[#1]{\ifnum0=`{\fi}\cr\noalign{\vskip#1\relax}} + + + +% flag to prevent multiple \and warning messages +\newif\if@IEEEWARNand +\@IEEEWARNandtrue + +% if in conference or peerreviewca modes, we support the use of \and as \author is a +% tabular environment, otherwise we warn the user that \and is invalid +% outside of conference or peerreviewca modes. +\def\and{\relax} % provide a bogus \and that we will then override + +\renewcommand{\and}[1][\relax]{\if@IEEEWARNand\typeout{** WARNING: \noexpand\and is valid only + when in conference or peerreviewca}\typeout{modes (line \the\inputlineno).}\fi\global\@IEEEWARNandfalse} + +\ifCLASSOPTIONconference% +\renewcommand{\and}[1][\hfill]{\end{@IEEEauthorhalign}#1\begin{@IEEEauthorhalign}}% +\fi +\ifCLASSOPTIONpeerreviewca +\renewcommand{\and}[1][\hfill]{\end{@IEEEauthorhalign}#1\begin{@IEEEauthorhalign}}% +\fi + + +% page clearing command +% based on LaTeX2e's \cleardoublepage, but allows different page styles +% for the inserted blank pages +\def\@IEEEcleardoublepage#1{\clearpage\if@twoside\ifodd\c@page\else +\hbox{}\thispagestyle{#1}\newpage\if@twocolumn\hbox{}\thispagestyle{#1}\newpage\fi\fi\fi} + + +% user command to invoke the title page +\def\maketitle{\par% + \begingroup% + \normalfont% + \def\thefootnote{}% the \thanks{} mark type is empty + \def\footnotemark{}% and kill space from \thanks within author + \let\@makefnmark\relax% V1.7, must *really* kill footnotemark to remove all \textsuperscript spacing as well. + \footnotesize% equal spacing between thanks lines + \footnotesep 0.7\baselineskip%see global setting of \footnotesep for more info + % V1.7 disable \thanks note indention for compsoc + \@IEEEcompsoconly{\long\def\@makefntext##1{\parindent 1em\noindent\hbox{\@makefnmark}##1}}% + \normalsize% + \ifCLASSOPTIONpeerreview + \newpage\global\@topnum\z@ \@maketitle\@IEEEstatictitlevskip\@IEEEaftertitletext% + \thispagestyle{IEEEpeerreviewcoverpagestyle}\@thanks% + \else + \if@twocolumn% + \ifCLASSOPTIONtechnote% + \newpage\global\@topnum\z@ \@maketitle\@IEEEstatictitlevskip\@IEEEaftertitletext% + \else + \twocolumn[\@maketitle\@IEEEstatictitlevskip\@IEEEaftertitletext]% + \fi + \else + \newpage\global\@topnum\z@ \@maketitle\@IEEEstatictitlevskip\@IEEEaftertitletext% + \fi + \thispagestyle{IEEEtitlepagestyle}\@thanks% + \fi + % pullup page for pubid if used. + \if@IEEEusingpubid + \enlargethispage{-\@IEEEpubidpullup}% + \fi + \endgroup + \setcounter{footnote}{0}\let\maketitle\relax\let\@maketitle\relax + \gdef\@thanks{}% + % v1.6b do not clear these as we will need the title again for peer review papers + % \gdef\@author{}\gdef\@title{}% + \let\thanks\relax} + + + +% V1.7 parbox to format \@IEEEcompsoctitleabstractindextext +\long\def\@IEEEcompsoctitleabstractindextextbox#1{\parbox{0.915\textwidth}{#1}} + +% formats the Title, authors names, affiliations and special paper notice +% THIS IS A CONTROLLED SPACING COMMAND! Do not allow blank lines or unintentional +% spaces to enter the definition - use % at the end of each line +\def\@maketitle{\newpage +\begingroup\centering +\ifCLASSOPTIONtechnote% technotes + {\bfseries\large\@IEEEcompsoconly{\sffamily}\@title\par}\vskip 1.3em{\lineskip .5em\@IEEEcompsoconly{\sffamily}\@author + \@IEEEspecialpapernotice\par{\@IEEEcompsoconly{\vskip 1.5em\relax + \@IEEEcompsoctitleabstractindextextbox{\@IEEEcompsoctitleabstractindextext}\par + \hfill\@IEEEcompsocdiamondline\hfill\hbox{}\par}}}\relax +\else% not a technote + \vskip0.2em{\Huge\@IEEEcompsoconly{\sffamily}\@IEEEcompsocconfonly{\normalfont\normalsize\vskip 2\@IEEEnormalsizeunitybaselineskip + \bfseries\Large}\@title\par}\vskip1.0em\par% + % V1.6 handle \author differently if in conference mode + \ifCLASSOPTIONconference% + {\@IEEEspecialpapernotice\mbox{}\vskip\@IEEEauthorblockconfadjspace% + \mbox{}\hfill\begin{@IEEEauthorhalign}\@author\end{@IEEEauthorhalign}\hfill\mbox{}\par}\relax + \else% peerreviewca, peerreview or journal + \ifCLASSOPTIONpeerreviewca + % peerreviewca handles author names just like conference mode + {\@IEEEcompsoconly{\sffamily}\@IEEEspecialpapernotice\mbox{}\vskip\@IEEEauthorblockconfadjspace% + \mbox{}\hfill\begin{@IEEEauthorhalign}\@author\end{@IEEEauthorhalign}\hfill\mbox{}\par + {\@IEEEcompsoconly{\vskip 1.5em\relax + \@IEEEcompsoctitleabstractindextextbox{\@IEEEcompsoctitleabstractindextext}\par\hfill + \@IEEEcompsocdiamondline\hfill\hbox{}\par}}}\relax + \else% journal or peerreview + {\lineskip.5em\@IEEEcompsoconly{\sffamily}\sublargesize\@author\@IEEEspecialpapernotice\par + {\@IEEEcompsoconly{\vskip 1.5em\relax + \@IEEEcompsoctitleabstractindextextbox{\@IEEEcompsoctitleabstractindextext}\par\hfill + \@IEEEcompsocdiamondline\hfill\hbox{}\par}}}\relax + \fi + \fi +\fi\par\endgroup} + + + +% V1.7 Computer Society "diamond line" which follows index terms for nonconference papers +\def\@IEEEcompsocdiamondline{\vrule depth 0pt height 0.5pt width 4cm\hspace{7.5pt}% +\raisebox{-3.5pt}{\fontfamily{pzd}\fontencoding{U}\fontseries{m}\fontshape{n}\fontsize{11}{12}\selectfont\char70}% +\hspace{7.5pt}\vrule depth 0pt height 0.5pt width 4cm\relax} + +% V1.7 standard LateX2e \thanks, but with \itshape under compsoc. Also make it a \long\def +% We also need to trigger the one-shot footnote rule +\def\@IEEEtriggeroneshotfootnoterule{\global\@IEEEenableoneshotfootnoteruletrue} + + +\long\def\thanks#1{\footnotemark + \protected@xdef\@thanks{\@thanks + \protect\footnotetext[\the\c@footnote]{\@IEEEcompsoconly{\itshape + \protect\@IEEEtriggeroneshotfootnoterule\relax}\ignorespaces#1}}} +\let\@thanks\@empty + +% V1.7 allow \author to contain \par's. This is needed to allow \thanks to contain \par. +\long\def\author#1{\gdef\@author{#1}} + + +% in addition to setting up IEEEitemize, we need to remove a baselineskip space above and +% below it because \list's \pars introduce blank lines because of the footnote struts. +\def\@IEEEsetupcompsocitemizelist{\def\labelitemi{$\bullet$}% +\setlength{\IEEElabelindent}{0pt}\setlength{\parskip}{0pt}% +\setlength{\partopsep}{0pt}\setlength{\topsep}{0.5\baselineskip}\vspace{-1\baselineskip}\relax} + + +% flag for fake non-compsoc \IEEEcompsocthanksitem - prevents line break on very first item +\newif\if@IEEEbreakcompsocthanksitem \@IEEEbreakcompsocthanksitemfalse + +\ifCLASSOPTIONcompsoc +% V1.7 compsoc bullet item \thanks +% also, we need to redefine this to destroy the argument in \@IEEEdynamictitlevspace +\long\def\IEEEcompsocitemizethanks#1{\relax\@IEEEbreakcompsocthanksitemfalse\footnotemark + \protected@xdef\@thanks{\@thanks + \protect\footnotetext[\the\c@footnote]{\itshape\protect\@IEEEtriggeroneshotfootnoterule + {\let\IEEEiedlistdecl\relax\protect\begin{IEEEitemize}[\protect\@IEEEsetupcompsocitemizelist]\ignorespaces#1\relax + \protect\end{IEEEitemize}}\protect\vspace{-1\baselineskip}}}} +\DeclareRobustCommand*{\IEEEcompsocthanksitem}{\item} +\else +% non-compsoc, allow for dual compilation via rerouting to normal \thanks +\long\def\IEEEcompsocitemizethanks#1{\thanks{#1}} +% redirect to "pseudo-par" \hfil\break\indent after swallowing [] from \IEEEcompsocthanksitem[] +\DeclareRobustCommand{\IEEEcompsocthanksitem}{\@ifnextchar [{\@IEEEthanksswallowoptionalarg}% +{\@IEEEthanksswallowoptionalarg[\relax]}} +% be sure and break only after first item, be sure and ignore spaces after optional argument +\def\@IEEEthanksswallowoptionalarg[#1]{\relax\if@IEEEbreakcompsocthanksitem\hfil\break +\indent\fi\@IEEEbreakcompsocthanksitemtrue\ignorespaces} +\fi + + +% V1.6b define the \IEEEpeerreviewmaketitle as needed +\ifCLASSOPTIONpeerreview +\def\IEEEpeerreviewmaketitle{\@IEEEcleardoublepage{empty}% +\ifCLASSOPTIONtwocolumn +\twocolumn[\@IEEEpeerreviewmaketitle\@IEEEdynamictitlevspace] +\else +\newpage\@IEEEpeerreviewmaketitle\@IEEEstatictitlevskip +\fi +\thispagestyle{IEEEtitlepagestyle}} +\else +% \IEEEpeerreviewmaketitle does nothing if peer review option has not been selected +\def\IEEEpeerreviewmaketitle{\relax} +\fi + +% peerreview formats the repeated title like the title in journal papers. +\def\@IEEEpeerreviewmaketitle{\begin{center}\@IEEEcompsoconly{\sffamily}% +\normalfont\normalsize\vskip0.2em{\Huge\@title\par}\vskip1.0em\par +\end{center}} + + + +% V1.6 +% this is a static rubber spacer between the title/authors and the main text +% used for single column text, or when the title appears in the first column +% of two column text (technotes). +\def\@IEEEstatictitlevskip{{\normalfont\normalsize +% adjust spacing to next text +% v1.6b handle peer review papers +\ifCLASSOPTIONpeerreview +% for peer review papers, the same value is used for both title pages +% regardless of the other paper modes + \vskip 1\baselineskip plus 0.375\baselineskip minus 0.1875\baselineskip +\else + \ifCLASSOPTIONconference% conference + \vskip 0.6\baselineskip + \else% + \ifCLASSOPTIONtechnote% technote + \vskip 1\baselineskip plus 0.375\baselineskip minus 0.1875\baselineskip% + \else% journal uses more space + \vskip 2.5\baselineskip plus 0.75\baselineskip minus 0.375\baselineskip% + \fi + \fi +\fi}} + + +% V1.6 +% This is a dynamically determined rigid spacer between the title/authors +% and the main text. This is used only for single column titles over two +% column text (most common) +% This is bit tricky because we have to ensure that the textheight of the +% main text is an integer multiple of \baselineskip +% otherwise underfull vbox problems may develop in the second column of the +% text on the titlepage +% The possible use of \IEEEpubid must also be taken into account. +\def\@IEEEdynamictitlevspace{{% + % we run within a group so that all the macros can be forgotten when we are done + \long\def\thanks##1{\relax}%don't allow \thanks to run when we evaluate the vbox height + \long\def\IEEEcompsocitemizethanks##1{\relax}%don't allow \IEEEcompsocitemizethanks to run when we evaluate the vbox height + \normalfont\normalsize% we declare more descriptive variable names + \let\@IEEEmaintextheight=\@IEEEtrantmpdimenA%height of the main text columns + \let\@IEEEINTmaintextheight=\@IEEEtrantmpdimenB%height of the main text columns with integer # lines + % set the nominal and minimum values for the title spacer + % the dynamic algorithm will not allow the spacer size to + % become less than \@IEEEMINtitlevspace - instead it will be + % lengthened + % default to journal values + \def\@IEEENORMtitlevspace{2.5\baselineskip}% + \def\@IEEEMINtitlevspace{2\baselineskip}% + % conferences and technotes need tighter spacing + \ifCLASSOPTIONconference%conference + \def\@IEEENORMtitlevspace{1\baselineskip}% + \def\@IEEEMINtitlevspace{0.75\baselineskip}% + \fi + \ifCLASSOPTIONtechnote%technote + \def\@IEEENORMtitlevspace{1\baselineskip}% + \def\@IEEEMINtitlevspace{0.75\baselineskip}% + \fi% + % get the height that the title will take up + \ifCLASSOPTIONpeerreview + \settoheight{\@IEEEmaintextheight}{\vbox{\hsize\textwidth \@IEEEpeerreviewmaketitle}}% + \else + \settoheight{\@IEEEmaintextheight}{\vbox{\hsize\textwidth \@maketitle}}% + \fi + \@IEEEmaintextheight=-\@IEEEmaintextheight% title takes away from maintext, so reverse sign + % add the height of the page textheight + \advance\@IEEEmaintextheight by \textheight% + % correct for title pages using pubid + \ifCLASSOPTIONpeerreview\else + % peerreview papers use the pubid on the cover page only. + % And the cover page uses a static spacer. + \if@IEEEusingpubid\advance\@IEEEmaintextheight by -\@IEEEpubidpullup\fi + \fi% + % subtract off the nominal value of the title bottom spacer + \advance\@IEEEmaintextheight by -\@IEEENORMtitlevspace% + % \topskip takes away some too + \advance\@IEEEmaintextheight by -\topskip% + % calculate the column height of the main text for lines + % now we calculate the main text height as if holding + % an integer number of \normalsize lines after the first + % and discard any excess fractional remainder + % we subtracted the first line, because the first line + % is placed \topskip into the maintext, not \baselineskip like the + % rest of the lines. + \@IEEEINTmaintextheight=\@IEEEmaintextheight% + \divide\@IEEEINTmaintextheight by \baselineskip% + \multiply\@IEEEINTmaintextheight by \baselineskip% + % now we calculate how much the title spacer height will + % have to be reduced from nominal (\@IEEEREDUCEmaintextheight is always + % a positive value) so that the maintext area will contain an integer + % number of normal size lines + % we change variable names here (to avoid confusion) as we no longer + % need \@IEEEINTmaintextheight and can reuse its dimen register + \let\@IEEEREDUCEmaintextheight=\@IEEEINTmaintextheight% + \advance\@IEEEREDUCEmaintextheight by -\@IEEEmaintextheight% + \advance\@IEEEREDUCEmaintextheight by \baselineskip% + % this is the calculated height of the spacer + % we change variable names here (to avoid confusion) as we no longer + % need \@IEEEmaintextheight and can reuse its dimen register + \let\@IEEECOMPENSATElen=\@IEEEmaintextheight% + \@IEEECOMPENSATElen=\@IEEENORMtitlevspace% set the nominal value + % we go with the reduced length if it is smaller than an increase + \ifdim\@IEEEREDUCEmaintextheight < 0.5\baselineskip\relax% + \advance\@IEEECOMPENSATElen by -\@IEEEREDUCEmaintextheight% + % if the resulting spacer is too small back out and go with an increase instead + \ifdim\@IEEECOMPENSATElen<\@IEEEMINtitlevspace\relax% + \advance\@IEEECOMPENSATElen by \baselineskip% + \fi% + \else% + % go with an increase because it is closer to the nominal than a decrease + \advance\@IEEECOMPENSATElen by -\@IEEEREDUCEmaintextheight% + \advance\@IEEECOMPENSATElen by \baselineskip% + \fi% + % set the calculated rigid spacer + \vspace{\@IEEECOMPENSATElen}}} + + + +% V1.6 +% we allow the user access to the last part of the title area +% useful in emergencies such as when a different spacing is needed +% This text is NOT compensated for in the dynamic sizer. +\let\@IEEEaftertitletext=\relax +\long\def\IEEEaftertitletext#1{\def\@IEEEaftertitletext{#1}} + +% V1.7 provide a way for users to enter abstract and keywords +% into the onecolumn title are. This text is compensated for +% in the dynamic sizer. +\let\@IEEEcompsoctitleabstractindextext=\relax +\long\def\IEEEcompsoctitleabstractindextext#1{\def\@IEEEcompsoctitleabstractindextext{#1}} +% V1.7 provide a way for users to get the \@IEEEcompsoctitleabstractindextext if +% not in compsoc journal mode - this way abstract and keywords can be placed +% in their conventional position if not in compsoc mode. +\def\IEEEdisplaynotcompsoctitleabstractindextext{% +\ifCLASSOPTIONcompsoc% display if compsoc conf +\ifCLASSOPTIONconference\@IEEEcompsoctitleabstractindextext\fi +\else% or if not compsoc +\@IEEEcompsoctitleabstractindextext\fi} + + +% command to allow alteration of baselinestretch, but only if the current +% baselineskip is unity. Used to tweak the compsoc abstract and keywords line spacing. +\def\@IEEEtweakunitybaselinestretch#1{{\def\baselinestretch{1}\selectfont +\global\@tempskipa\baselineskip}\ifnum\@tempskipa=\baselineskip% +\def\baselinestretch{#1}\selectfont\fi\relax} + + +% abstract and keywords are in \small, except +% for 9pt docs in which they are in \footnotesize +% Because 9pt docs use an 8pt footnotesize, \small +% becomes a rather awkward 8.5pt +\def\@IEEEabskeysecsize{\small} +\ifx\CLASSOPTIONpt\@IEEEptsizenine + \def\@IEEEabskeysecsize{\footnotesize} +\fi + +% compsoc journals use \footnotesize, compsoc conferences use normalsize +\@IEEEcompsoconly{\def\@IEEEabskeysecsize{\footnotesize}} +\@IEEEcompsocconfonly{\def\@IEEEabskeysecsize{\normalsize}} + + + + +% V1.6 have abstract and keywords strip leading spaces, pars and newlines +% so that spacing is more tightly controlled. +\def\abstract{\normalfont + \if@twocolumn + \par\@IEEEabskeysecsize\bfseries\leavevmode\kern-1pt\textit{\abstractname}---\relax + \else + \begin{center}\vspace{-1.78ex}\@IEEEabskeysecsize\textbf{\abstractname}\end{center}\quotation\@IEEEabskeysecsize + \fi\@IEEEgobbleleadPARNLSP} +% V1.6 IEEE wants only 1 pica from end of abstract to introduction heading when in +% conference mode (the heading already has this much above it) +\def\endabstract{\relax\ifCLASSOPTIONconference\vspace{0ex}\else\vspace{1.34ex}\fi\par\if@twocolumn\else\endquotation\fi + \normalfont\normalsize} + +\def\IEEEkeywords{\normalfont + \if@twocolumn + \@IEEEabskeysecsize\bfseries\leavevmode\kern-1pt\textit{\IEEEkeywordsname}---\relax + \else + \begin{center}\@IEEEabskeysecsize\textbf{\IEEEkeywordsname}\end{center}\quotation\@IEEEabskeysecsize + \fi\itshape\@IEEEgobbleleadPARNLSP} +\def\endIEEEkeywords{\relax\ifCLASSOPTIONtechnote\vspace{1.34ex}\else\vspace{0.5ex}\fi + \par\if@twocolumn\else\endquotation\fi% + \normalfont\normalsize} + +% V1.7 compsoc keywords index terms +\ifCLASSOPTIONcompsoc + \ifCLASSOPTIONconference% compsoc conference +\def\abstract{\normalfont + \begin{center}\@IEEEabskeysecsize\textbf{\large\abstractname}\end{center}\vskip 0.5\baselineskip plus 0.1\baselineskip minus 0.1\baselineskip + \if@twocolumn\else\quotation\fi\itshape\@IEEEabskeysecsize% + \par\@IEEEgobbleleadPARNLSP} +\def\IEEEkeywords{\normalfont\vskip 1.5\baselineskip plus 0.25\baselineskip minus 0.25\baselineskip + \begin{center}\@IEEEabskeysecsize\textbf{\large\IEEEkeywordsname}\end{center}\vskip 0.5\baselineskip plus 0.1\baselineskip minus 0.1\baselineskip + \if@twocolumn\else\quotation\fi\itshape\@IEEEabskeysecsize% + \par\@IEEEgobbleleadPARNLSP} + \else% compsoc not conference +\def\abstract{\normalfont\@IEEEtweakunitybaselinestretch{1.15}\sffamily + \if@twocolumn + \@IEEEabskeysecsize\noindent\textbf{\abstractname}---\relax + \else + \begin{center}\vspace{-1.78ex}\@IEEEabskeysecsize\textbf{\abstractname}\end{center}\quotation\@IEEEabskeysecsize% + \fi\@IEEEgobbleleadPARNLSP} +\def\IEEEkeywords{\normalfont\@IEEEtweakunitybaselinestretch{1.15}\sffamily + \if@twocolumn + \@IEEEabskeysecsize\vskip 0.5\baselineskip plus 0.25\baselineskip minus 0.25\baselineskip\noindent + \textbf{\IEEEkeywordsname}---\relax + \else + \begin{center}\@IEEEabskeysecsize\textbf{\IEEEkeywordsname}\end{center}\quotation\@IEEEabskeysecsize% + \fi\@IEEEgobbleleadPARNLSP} + \fi +\fi + + + +% gobbles all leading \, \\ and \par, upon finding first token that +% is not a \ , \\ or a \par, it ceases and returns that token +% +% used to strip leading \, \\ and \par from the input +% so that such things in the beginning of an environment will not +% affect the formatting of the text +\long\def\@IEEEgobbleleadPARNLSP#1{\let\@IEEEswallowthistoken=0% +\let\@IEEEgobbleleadPARNLSPtoken#1% +\let\@IEEEgobbleleadPARtoken=\par% +\let\@IEEEgobbleleadNLtoken=\\% +\let\@IEEEgobbleleadSPtoken=\ % +\def\@IEEEgobbleleadSPMACRO{\ }% +\ifx\@IEEEgobbleleadPARNLSPtoken\@IEEEgobbleleadPARtoken% +\let\@IEEEswallowthistoken=1% +\fi% +\ifx\@IEEEgobbleleadPARNLSPtoken\@IEEEgobbleleadNLtoken% +\let\@IEEEswallowthistoken=1% +\fi% +\ifx\@IEEEgobbleleadPARNLSPtoken\@IEEEgobbleleadSPtoken% +\let\@IEEEswallowthistoken=1% +\fi% +% a control space will come in as a macro +% when it is the last one on a line +\ifx\@IEEEgobbleleadPARNLSPtoken\@IEEEgobbleleadSPMACRO% +\let\@IEEEswallowthistoken=1% +\fi% +% if we have to swallow this token, do so and taste the next one +% else spit it out and stop gobbling +\ifx\@IEEEswallowthistoken 1\let\@IEEEnextgobbleleadPARNLSP=\@IEEEgobbleleadPARNLSP\else% +\let\@IEEEnextgobbleleadPARNLSP=#1\fi% +\@IEEEnextgobbleleadPARNLSP}% + + + + +% TITLING OF SECTIONS +\def\@IEEEsectpunct{:\ \,} % Punctuation after run-in section heading (headings which are + % part of the paragraphs), need little bit more than a single space + % spacing from section number to title +% compsoc conferences use regular period/space punctuation +\ifCLASSOPTIONcompsoc +\ifCLASSOPTIONconference +\def\@IEEEsectpunct{.\ } +\fi\fi + +\def\@seccntformat#1{\hb@xt@ 1.4em{\csname the#1dis\endcsname\hss\relax}} +\def\@seccntformatinl#1{\hb@xt@ 1.1em{\csname the#1dis\endcsname\hss\relax}} +\def\@seccntformatch#1{\csname the#1dis\endcsname\hskip 1em\relax} + +\ifCLASSOPTIONcompsoc +% compsoc journals need extra spacing +\ifCLASSOPTIONconference\else +\def\@seccntformat#1{\csname the#1dis\endcsname\hskip 1em\relax} +\fi\fi + +%v1.7 put {} after #6 to allow for some types of user font control +%and use \@@par rather than \par +\def\@sect#1#2#3#4#5#6[#7]#8{% + \ifnum #2>\c@secnumdepth + \let\@svsec\@empty + \else + \refstepcounter{#1}% + % load section label and spacer into \@svsec + \ifnum #2=1 + \protected@edef\@svsec{\@seccntformatch{#1}\relax}% + \else + \ifnum #2>2 + \protected@edef\@svsec{\@seccntformatinl{#1}\relax}% + \else + \protected@edef\@svsec{\@seccntformat{#1}\relax}% + \fi + \fi + \fi% + \@tempskipa #5\relax + \ifdim \@tempskipa>\z@% tempskipa determines whether is treated as a high + \begingroup #6{\relax% or low level heading + \noindent % subsections are NOT indented + % print top level headings. \@svsec is label, #8 is heading title + % IEEE does not block indent the section title text, it flows like normal + {\hskip #3\relax\@svsec}{\interlinepenalty \@M #8\@@par}}% + \endgroup + \addcontentsline{toc}{#1}{\ifnum #2>\c@secnumdepth\relax\else + \protect\numberline{\csname the#1\endcsname}\fi#7}% + \else % printout low level headings + % svsechd seems to swallow the trailing space, protect it with \mbox{} + % got rid of sectionmark stuff + \def\@svsechd{#6{\hskip #3\relax\@svsec #8\@IEEEsectpunct\mbox{}}% + \addcontentsline{toc}{#1}{\ifnum #2>\c@secnumdepth\relax\else + \protect\numberline{\csname the#1\endcsname}\fi#7}}% + \fi%skip down + \@xsect{#5}} + + +% section* handler +%v1.7 put {} after #4 to allow for some types of user font control +%and use \@@par rather than \par +\def\@ssect#1#2#3#4#5{\@tempskipa #3\relax + \ifdim \@tempskipa>\z@ + %\begingroup #4\@hangfrom{\hskip #1}{\interlinepenalty \@M #5\par}\endgroup + % IEEE does not block indent the section title text, it flows like normal + \begingroup \noindent #4{\relax{\hskip #1}{\interlinepenalty \@M #5\@@par}}\endgroup + % svsechd swallows the trailing space, protect it with \mbox{} + \else \def\@svsechd{#4{\hskip #1\relax #5\@IEEEsectpunct\mbox{}}}\fi + \@xsect{#3}} + + +%% SECTION heading spacing and font +%% +% arguments are: #1 - sectiontype name +% (for \@sect) #2 - section level +% #3 - section heading indent +% #4 - top separation (absolute value used, neg indicates not to indent main text) +% If negative, make stretch parts negative too! +% #5 - (absolute value used) positive: bottom separation after heading, +% negative: amount to indent main text after heading +% Both #4 and #5 negative means to indent main text and use negative top separation +% #6 - font control +% You've got to have \normalfont\normalsize in the font specs below to prevent +% trouble when you do something like: +% \section{Note}{\ttfamily TT-TEXT} is known to ... +% IEEE sometimes REALLY stretches the area before a section +% heading by up to about 0.5in. However, it may not be a good +% idea to let LaTeX have quite this much rubber. +\ifCLASSOPTIONconference% +% IEEE wants section heading spacing to decrease for conference mode +\def\section{\@startsection{section}{1}{\z@}{1.5ex plus 1.5ex minus 0.5ex}% +{1sp}{\normalfont\normalsize\centering\scshape}}% +\def\subsection{\@startsection{subsection}{2}{\z@}{1.5ex plus 1.5ex minus 0.5ex}% +{1sp}{\normalfont\normalsize\itshape}}% +\else % for journals +\def\section{\@startsection{section}{1}{\z@}{3.0ex plus 1.5ex minus 1.5ex}% V1.6 3.0ex from 3.5ex +{0.7ex plus 1ex minus 0ex}{\normalfont\normalsize\centering\scshape}}% +\def\subsection{\@startsection{subsection}{2}{\z@}{3.5ex plus 1.5ex minus 1.5ex}% +{0.7ex plus .5ex minus 0ex}{\normalfont\normalsize\itshape}}% +\fi + +% for both journals and conferences +% decided to put in a little rubber above the section, might help somebody +\def\subsubsection{\@startsection{subsubsection}{3}{\parindent}{0ex plus 0.1ex minus 0.1ex}% +{0ex}{\normalfont\normalsize\itshape}}% +\def\paragraph{\@startsection{paragraph}{4}{2\parindent}{0ex plus 0.1ex minus 0.1ex}% +{0ex}{\normalfont\normalsize\itshape}}% + + +% compsoc +\ifCLASSOPTIONcompsoc +\ifCLASSOPTIONconference +% compsoc conference +\def\section{\@startsection{section}{1}{\z@}{1\baselineskip plus 0.25\baselineskip minus 0.25\baselineskip}% +{1\baselineskip plus 0.25\baselineskip minus 0.25\baselineskip}{\normalfont\large\bfseries}}% +\def\subsection{\@startsection{subsection}{2}{\z@}{1\baselineskip plus 0.25\baselineskip minus 0.25\baselineskip}% +{1\baselineskip plus 0.25\baselineskip minus 0.25\baselineskip}{\normalfont\sublargesize\bfseries}}% +\def\subsubsection{\@startsection{subsubsection}{3}{\z@}{1\baselineskip plus 0.25\baselineskip minus 0.25\baselineskip}% +{0ex}{\normalfont\normalsize\bfseries}}% +\def\paragraph{\@startsection{paragraph}{4}{2\parindent}{0ex plus 0.1ex minus 0.1ex}% +{0ex}{\normalfont\normalsize}}% +\else% compsoc journals +% use negative top separation as compsoc journals do not indent paragraphs after section titles +\def\section{\@startsection{section}{1}{\z@}{-3ex plus -2ex minus -1.5ex}% +{0.7ex plus 1ex minus 0ex}{\normalfont\large\sffamily\bfseries\scshape}}% +% Note that subsection and smaller may not be correct for the Computer Society, +% I have to look up an example. +\def\subsection{\@startsection{subsection}{2}{\z@}{-3.5ex plus -1.5ex minus -1.5ex}% +{0.7ex plus .5ex minus 0ex}{\normalfont\normalsize\sffamily\bfseries}}% +\def\subsubsection{\@startsection{subsubsection}{3}{\z@}{-2.5ex plus -1ex minus -1ex}% +{0.5ex plus 0.5ex minus 0ex}{\normalfont\normalsize\sffamily\itshape}}% +\def\paragraph{\@startsection{paragraph}{4}{2\parindent}{-0ex plus -0.1ex minus -0.1ex}% +{0ex}{\normalfont\normalsize}}% +\fi\fi + + + + +%% ENVIRONMENTS +% "box" symbols at end of proofs +\def\IEEEQEDclosed{\mbox{\rule[0pt]{1.3ex}{1.3ex}}} % for a filled box +% V1.6 some journals use an open box instead that will just fit around a closed one +\def\IEEEQEDopen{{\setlength{\fboxsep}{0pt}\setlength{\fboxrule}{0.2pt}\fbox{\rule[0pt]{0pt}{1.3ex}\rule[0pt]{1.3ex}{0pt}}}} +\ifCLASSOPTIONcompsoc +\def\IEEEQED{\IEEEQEDopen} % default to open for compsoc +\else +\def\IEEEQED{\IEEEQEDclosed} % otherwise default to closed +\fi + +% v1.7 name change to avoid namespace collision with amsthm. Also add support +% for an optional argument. +\def\IEEEproof{\@ifnextchar[{\@IEEEproof}{\@IEEEproof[\IEEEproofname]}} +\def\@IEEEproof[#1]{\par\noindent\hspace{2em}{\itshape #1: }} +\def\endIEEEproof{\hspace*{\fill}~\IEEEQED\par} + + +%\itemindent is set to \z@ by list, so define new temporary variable +\newdimen\@IEEEtmpitemindent +\def\@begintheorem#1#2{\@IEEEtmpitemindent\itemindent\topsep 0pt\rmfamily\trivlist% + \item[\hskip \labelsep{\indent\itshape #1\ #2:}]\itemindent\@IEEEtmpitemindent} +\def\@opargbegintheorem#1#2#3{\@IEEEtmpitemindent\itemindent\topsep 0pt\rmfamily \trivlist% +% V1.6 IEEE is back to using () around theorem names which are also in italics +% Thanks to Christian Peel for reporting this. + \item[\hskip\labelsep{\indent\itshape #1\ #2\ (#3):}]\itemindent\@IEEEtmpitemindent} +% V1.7 remove bogus \unskip that caused equations in theorems to collide with +% lines below. +\def\@endtheorem{\endtrivlist} + +% V1.6 +% display command for the section the theorem is in - so that \thesection +% is not used as this will be in Roman numerals when we want arabic. +% LaTeX2e uses \def\@thmcounter#1{\noexpand\arabic{#1}} for the theorem number +% (second part) display and \def\@thmcountersep{.} as a separator. +% V1.7 intercept calls to the section counter and reroute to \@IEEEthmcounterinsection +% to allow \appendix(ices} to override as needed. +% +% special handler for sections, allows appendix(ices) to override +\gdef\@IEEEthmcounterinsection#1{\arabic{#1}} +% string macro +\edef\@IEEEstringsection{section} + +% redefine the #1#2[#3] form of newtheorem to use a hook to \@IEEEthmcounterinsection +% if section in_counter is used +\def\@xnthm#1#2[#3]{% + \expandafter\@ifdefinable\csname #1\endcsname + {\@definecounter{#1}\@newctr{#1}[#3]% + \edef\@IEEEstringtmp{#3} + \ifx\@IEEEstringtmp\@IEEEstringsection + \expandafter\xdef\csname the#1\endcsname{% + \noexpand\@IEEEthmcounterinsection{#3}\@thmcountersep + \@thmcounter{#1}}% + \else + \expandafter\xdef\csname the#1\endcsname{% + \expandafter\noexpand\csname the#3\endcsname \@thmcountersep + \@thmcounter{#1}}% + \fi + \global\@namedef{#1}{\@thm{#1}{#2}}% + \global\@namedef{end#1}{\@endtheorem}}} + + + +%% SET UP THE DEFAULT PAGESTYLE +\ps@headings +\pagenumbering{arabic} + +% normally the page counter starts at 1 +\setcounter{page}{1} +% however, for peerreview the cover sheet is page 0 or page -1 +% (for duplex printing) +\ifCLASSOPTIONpeerreview + \if@twoside + \setcounter{page}{-1} + \else + \setcounter{page}{0} + \fi +\fi + +% standard book class behavior - let bottom line float up and down as +% needed when single sided +\ifCLASSOPTIONtwoside\else\raggedbottom\fi +% if two column - turn on twocolumn, allow word spacings to stretch more and +% enforce a rigid position for the last lines +\ifCLASSOPTIONtwocolumn +% the peer review option delays invoking twocolumn + \ifCLASSOPTIONpeerreview\else + \twocolumn + \fi +\sloppy +\flushbottom +\fi + + + + +% \APPENDIX and \APPENDICES definitions + +% This is the \@ifmtarg command from the LaTeX ifmtarg package +% by Peter Wilson (CUA) and Donald Arseneau +% \@ifmtarg is used to determine if an argument to a command +% is present or not. +% For instance: +% \@ifmtarg{#1}{\typeout{empty}}{\typeout{has something}} +% \@ifmtarg is used with our redefined \section command if +% \appendices is invoked. +% The command \section will behave slightly differently depending +% on whether the user specifies a title: +% \section{My appendix title} +% or not: +% \section{} +% This way, we can eliminate the blank lines where the title +% would be, and the unneeded : after Appendix in the table of +% contents +\begingroup +\catcode`\Q=3 +\long\gdef\@ifmtarg#1{\@xifmtarg#1QQ\@secondoftwo\@firstoftwo\@nil} +\long\gdef\@xifmtarg#1#2Q#3#4#5\@nil{#4} +\endgroup +% end of \@ifmtarg defs + + +% V1.7 +% command that allows the one time saving of the original definition +% of section to \@IEEEappendixsavesection for \appendix or \appendices +% we don't save \section here as it may be redefined later by other +% packages (hyperref.sty, etc.) +\def\@IEEEsaveoriginalsectiononce{\let\@IEEEappendixsavesection\section +\let\@IEEEsaveoriginalsectiononce\relax} + +% neat trick to grab and process the argument from \section{argument} +% we process differently if the user invoked \section{} with no +% argument (title) +% note we reroute the call to the old \section* +\def\@IEEEprocessthesectionargument#1{% +\@ifmtarg{#1}{% +\@IEEEappendixsavesection*{\appendixname~\thesectiondis}% +\addcontentsline{toc}{section}{\appendixname~\thesection}}{% +\@IEEEappendixsavesection*{\appendixname~\thesectiondis \\* #1}% +\addcontentsline{toc}{section}{\appendixname~\thesection: #1}}} + +% we use this if the user calls \section{} after +% \appendix-- which has no meaning. So, we ignore the +% command and its argument. Then, warn the user. +\def\@IEEEdestroythesectionargument#1{\typeout{** WARNING: Ignoring useless +\protect\section\space in Appendix (line \the\inputlineno).}} + + +% remember \thesection forms will be displayed in \ref calls +% and in the Table of Contents. +% The \sectiondis form is used in the actual heading itself + +% appendix command for one single appendix +% normally has no heading. However, if you want a +% heading, you can do so via the optional argument: +% \appendix[Optional Heading] +\def\appendix{\relax} +\renewcommand{\appendix}[1][]{\@IEEEsaveoriginalsectiononce\par + % v1.6 keep hyperref's identifiers unique + \gdef\theHsection{Appendix.A}% + % v1.6 adjust hyperref's string name for the section + \xdef\Hy@chapapp{appendix}% + \setcounter{section}{0}% + \setcounter{subsection}{0}% + \setcounter{subsubsection}{0}% + \setcounter{paragraph}{0}% + \gdef\thesection{A}% + \gdef\thesectiondis{}% + \gdef\thesubsection{\Alph{subsection}}% + \gdef\@IEEEthmcounterinsection##1{A} + \refstepcounter{section}% update the \ref counter + \@ifmtarg{#1}{\@IEEEappendixsavesection*{\appendixname}% + \addcontentsline{toc}{section}{\appendixname}}{% + \@IEEEappendixsavesection*{\appendixname~\\* #1}% + \addcontentsline{toc}{section}{\appendixname: #1}}% + % redefine \section command for appendix + % leave \section* as is + \def\section{\@ifstar{\@IEEEappendixsavesection*}{% + \@IEEEdestroythesectionargument}}% throw out the argument + % of the normal form +} + + + +% appendices command for multiple appendices +% user then calls \section with an argument (possibly empty) to +% declare the individual appendices +\def\appendices{\@IEEEsaveoriginalsectiononce\par + % v1.6 keep hyperref's identifiers unique + \gdef\theHsection{Appendix.\Alph{section}}% + % v1.6 adjust hyperref's string name for the section + \xdef\Hy@chapapp{appendix}% + \setcounter{section}{-1}% we want \refstepcounter to use section 0 + \setcounter{subsection}{0}% + \setcounter{subsubsection}{0}% + \setcounter{paragraph}{0}% + \ifCLASSOPTIONromanappendices% + \gdef\thesection{\Roman{section}}% + \gdef\thesectiondis{\Roman{section}}% + \@IEEEcompsocconfonly{\gdef\thesectiondis{\Roman{section}.}}% + \gdef\@IEEEthmcounterinsection##1{A\arabic{##1}} + \else% + \gdef\thesection{\Alph{section}}% + \gdef\thesectiondis{\Alph{section}}% + \@IEEEcompsocconfonly{\gdef\thesectiondis{\Alph{section}.}}% + \gdef\@IEEEthmcounterinsection##1{\Alph{##1}} + \fi% + \refstepcounter{section}% update the \ref counter + \setcounter{section}{0}% NEXT \section will be the FIRST appendix + % redefine \section command for appendices + % leave \section* as is + \def\section{\@ifstar{\@IEEEappendixsavesection*}{% process the *-form + \refstepcounter{section}% or is a new section so, + \@IEEEprocessthesectionargument}}% process the argument + % of the normal form +} + + + +% \IEEEPARstart +% Definition for the big two line drop cap letter at the beginning of the +% first paragraph of journal papers. The first argument is the first letter +% of the first word, the second argument is the remaining letters of the +% first word which will be rendered in upper case. +% In V1.6 this has been completely rewritten to: +% +% 1. no longer have problems when the user begins an environment +% within the paragraph that uses \IEEEPARstart. +% 2. auto-detect and use the current font family +% 3. revise handling of the space at the end of the first word so that +% interword glue will now work as normal. +% 4. produce correctly aligned edges for the (two) indented lines. +% +% We generalize things via control macros - playing with these is fun too. +% +% V1.7 added more control macros to make it easy for IEEEtrantools.sty users +% to change the font style. +% +% the number of lines that are indented to clear it +% may need to increase if using decenders +\def\@IEEEPARstartDROPLINES{2} +% minimum number of lines left on a page to allow a \@IEEEPARstart +% Does not take into consideration rubber shrink, so it tends to +% be overly cautious +\def\@IEEEPARstartMINPAGELINES{2} +% V1.7 the height of the drop cap is adjusted to match the height of this text +% in the current font (when \IEEEPARstart is called). +\def\@IEEEPARstartHEIGHTTEXT{T} +% the depth the letter is lowered below the baseline +% the height (and size) of the letter is determined by the sum +% of this value and the height of the \@IEEEPARstartHEIGHTTEXT in the current +% font. It is a good idea to set this value in terms of the baselineskip +% so that it can respond to changes therein. +\def\@IEEEPARstartDROPDEPTH{1.1\baselineskip} +% V1.7 the font the drop cap will be rendered in, +% can take zero or one argument. +\def\@IEEEPARstartFONTSTYLE{\bfseries} +% V1.7 any additional, non-font related commands needed to modify +% the drop cap letter, can take zero or one argument. +\def\@IEEEPARstartCAPSTYLE{\MakeUppercase} +% V1.7 the font that will be used to render the rest of the word, +% can take zero or one argument. +\def\@IEEEPARstartWORDFONTSTYLE{\relax} +% V1.7 any additional, non-font related commands needed to modify +% the rest of the word, can take zero or one argument. +\def\@IEEEPARstartWORDCAPSTYLE{\MakeUppercase} +% This is the horizontal separation distance from the drop letter to the main text. +% Lengths that depend on the font (e.g., ex, em, etc.) will be referenced +% to the font that is active when \IEEEPARstart is called. +\def\@IEEEPARstartSEP{0.15em} +% V1.7 horizontal offset applied to the left of the drop cap. +\def\@IEEEPARstartHOFFSET{0em} +% V1.7 Italic correction command applied at the end of the drop cap. +\def\@IEEEPARstartITLCORRECT{\/} + +% V1.7 compoc uses nonbold drop cap and small caps word style +\ifCLASSOPTIONcompsoc +\def\@IEEEPARstartFONTSTYLE{\mdseries} +\def\@IEEEPARstartWORDFONTSTYLE{\scshape} +\def\@IEEEPARstartWORDCAPSTYLE{\relax} +\fi + +% definition of \IEEEPARstart +% THIS IS A CONTROLLED SPACING AREA, DO NOT ALLOW SPACES WITHIN THESE LINES +% +% The token \@IEEEPARstartfont will be globally defined after the first use +% of \IEEEPARstart and will be a font command which creates the big letter +% The first argument is the first letter of the first word and the second +% argument is the rest of the first word(s). +\def\IEEEPARstart#1#2{\par{% +% if this page does not have enough space, break it and lets start +% on a new one +\@IEEEtranneedspace{\@IEEEPARstartMINPAGELINES\baselineskip}{\relax}% +% V1.7 move this up here in case user uses \textbf for \@IEEEPARstartFONTSTYLE +% which uses command \leavevmode which causes an unwanted \indent to be issued +\noindent +% calculate the desired height of the big letter +% it extends from the top of \@IEEEPARstartHEIGHTTEXT in the current font +% down to \@IEEEPARstartDROPDEPTH below the current baseline +\settoheight{\@IEEEtrantmpdimenA}{\@IEEEPARstartHEIGHTTEXT}% +\addtolength{\@IEEEtrantmpdimenA}{\@IEEEPARstartDROPDEPTH}% +% extract the name of the current font in bold +% and place it in \@IEEEPARstartFONTNAME +\def\@IEEEPARstartGETFIRSTWORD##1 ##2\relax{##1}% +{\@IEEEPARstartFONTSTYLE{\selectfont\edef\@IEEEPARstartFONTNAMESPACE{\fontname\font\space}% +\xdef\@IEEEPARstartFONTNAME{\expandafter\@IEEEPARstartGETFIRSTWORD\@IEEEPARstartFONTNAMESPACE\relax}}}% +% define a font based on this name with a point size equal to the desired +% height of the drop letter +\font\@IEEEPARstartsubfont\@IEEEPARstartFONTNAME\space at \@IEEEtrantmpdimenA\relax% +% save this value as a counter (integer) value (sp points) +\@IEEEtrantmpcountA=\@IEEEtrantmpdimenA% +% now get the height of the actual letter produced by this font size +\settoheight{\@IEEEtrantmpdimenB}{\@IEEEPARstartsubfont\@IEEEPARstartCAPSTYLE{#1}}% +% If something bogus happens like the first argument is empty or the +% current font is strange, do not allow a zero height. +\ifdim\@IEEEtrantmpdimenB=0pt\relax% +\typeout{** WARNING: IEEEPARstart drop letter has zero height! (line \the\inputlineno)}% +\typeout{ Forcing the drop letter font size to 10pt.}% +\@IEEEtrantmpdimenB=10pt% +\fi% +% and store it as a counter +\@IEEEtrantmpcountB=\@IEEEtrantmpdimenB% +% Since a font size doesn't exactly correspond to the height of the capital +% letters in that font, the actual height of the letter, \@IEEEtrantmpcountB, +% will be less than that desired, \@IEEEtrantmpcountA +% we need to raise the font size, \@IEEEtrantmpdimenA +% by \@IEEEtrantmpcountA / \@IEEEtrantmpcountB +% But, TeX doesn't have floating point division, so we have to use integer +% division. Hence the use of the counters. +% We need to reduce the denominator so that the loss of the remainder will +% have minimal affect on the accuracy of the result +\divide\@IEEEtrantmpcountB by 200% +\divide\@IEEEtrantmpcountA by \@IEEEtrantmpcountB% +% Then reequalize things when we use TeX's ability to multiply by +% floating point values +\@IEEEtrantmpdimenB=0.005\@IEEEtrantmpdimenA% +\multiply\@IEEEtrantmpdimenB by \@IEEEtrantmpcountA% +% \@IEEEPARstartfont is globaly set to the calculated font of the big letter +% We need to carry this out of the local calculation area to to create the +% big letter. +\global\font\@IEEEPARstartfont\@IEEEPARstartFONTNAME\space at \@IEEEtrantmpdimenB% +% Now set \@IEEEtrantmpdimenA to the width of the big letter +% We need to carry this out of the local calculation area to set the +% hanging indent +\settowidth{\global\@IEEEtrantmpdimenA}{\@IEEEPARstartfont +\@IEEEPARstartCAPSTYLE{#1\@IEEEPARstartITLCORRECT}}}% +% end of the isolated calculation environment +% add in the extra clearance we want +\advance\@IEEEtrantmpdimenA by \@IEEEPARstartSEP\relax% +% add in the optional offset +\advance\@IEEEtrantmpdimenA by \@IEEEPARstartHOFFSET\relax% +% V1.7 don't allow negative offsets to produce negative hanging indents +\@IEEEtrantmpdimenB\@IEEEtrantmpdimenA +\ifnum\@IEEEtrantmpdimenB < 0 \@IEEEtrantmpdimenB 0pt\fi +% \@IEEEtrantmpdimenA has the width of the big letter plus the +% separation space and \@IEEEPARstartfont is the font we need to use +% Now, we make the letter and issue the hanging indent command +% The letter is placed in a box of zero width and height so that other +% text won't be displaced by it. +\hangindent\@IEEEtrantmpdimenB\hangafter=-\@IEEEPARstartDROPLINES% +\makebox[0pt][l]{\hspace{-\@IEEEtrantmpdimenA}% +\raisebox{-\@IEEEPARstartDROPDEPTH}[0pt][0pt]{\hspace{\@IEEEPARstartHOFFSET}% +\@IEEEPARstartfont\@IEEEPARstartCAPSTYLE{#1\@IEEEPARstartITLCORRECT}% +\hspace{\@IEEEPARstartSEP}}}% +{\@IEEEPARstartWORDFONTSTYLE{\@IEEEPARstartWORDCAPSTYLE{\selectfont#2}}}} + + + + + + +% determines if the space remaining on a given page is equal to or greater +% than the specified space of argument one +% if not, execute argument two (only if the remaining space is greater than zero) +% and issue a \newpage +% +% example: \@IEEEtranneedspace{2in}{\vfill} +% +% Does not take into consideration rubber shrinkage, so it tends to +% be overly cautious +% Based on an example posted by Donald Arseneau +% Note this macro uses \@IEEEtrantmpdimenB internally for calculations, +% so DO NOT PASS \@IEEEtrantmpdimenB to this routine +% if you need a dimen register, import with \@IEEEtrantmpdimenA instead +\def\@IEEEtranneedspace#1#2{\penalty-100\begingroup%shield temp variable +\@IEEEtrantmpdimenB\pagegoal\advance\@IEEEtrantmpdimenB-\pagetotal% space left +\ifdim #1>\@IEEEtrantmpdimenB\relax% not enough space left +\ifdim\@IEEEtrantmpdimenB>\z@\relax #2\fi% +\newpage% +\fi\endgroup} + + + +% IEEEbiography ENVIRONMENT +% Allows user to enter biography leaving place for picture (adapts to font size) +% As of V1.5, a new optional argument allows you to have a real graphic! +% V1.5 and later also fixes the "colliding biographies" which could happen when a +% biography's text was shorter than the space for the photo. +% MDS 7/2001 +% V1.6 prevent multiple biographies from making multiple TOC entries +\newif\if@IEEEbiographyTOCentrynotmade +\global\@IEEEbiographyTOCentrynotmadetrue + +% biography counter so hyperref can jump directly to the biographies +% and not just the previous section +\newcounter{IEEEbiography} +\setcounter{IEEEbiography}{0} + +% photo area size +\def\@IEEEBIOphotowidth{1.0in} % width of the biography photo area +\def\@IEEEBIOphotodepth{1.25in} % depth (height) of the biography photo area +% area cleared for photo +\def\@IEEEBIOhangwidth{1.14in} % width cleared for the biography photo area +\def\@IEEEBIOhangdepth{1.25in} % depth cleared for the biography photo area + % actual depth will be a multiple of + % \baselineskip, rounded up +\def\@IEEEBIOskipN{4\baselineskip}% nominal value of the vskip above the biography + +\newenvironment{IEEEbiography}[2][]{\normalfont\@IEEEcompsoconly{\sffamily}\footnotesize% +\unitlength 1in\parskip=0pt\par\parindent 1em\interlinepenalty500% +% we need enough space to support the hanging indent +% the nominal value of the spacer +% and one extra line for good measure +\@IEEEtrantmpdimenA=\@IEEEBIOhangdepth% +\advance\@IEEEtrantmpdimenA by \@IEEEBIOskipN% +\advance\@IEEEtrantmpdimenA by 1\baselineskip% +% if this page does not have enough space, break it and lets start +% with a new one +\@IEEEtranneedspace{\@IEEEtrantmpdimenA}{\relax}% +% nominal spacer can strech, not shrink use 1fil so user can out stretch with \vfill +\vskip \@IEEEBIOskipN plus 1fil minus 0\baselineskip% +% the default box for where the photo goes +\def\@IEEEtempbiographybox{{\setlength{\fboxsep}{0pt}\framebox{% +\begin{minipage}[b][\@IEEEBIOphotodepth][c]{\@IEEEBIOphotowidth}\centering PLACE\\ PHOTO\\ HERE \end{minipage}}}}% +% +% detect if the optional argument was supplied, this requires the +% \@ifmtarg command as defined in the appendix section above +% and if so, override the default box with what they want +\@ifmtarg{#1}{\relax}{\def\@IEEEtempbiographybox{\mbox{\begin{minipage}[b][\@IEEEBIOphotodepth][c]{\@IEEEBIOphotowidth}% +\centering% +#1% +\end{minipage}}}}% end if optional argument supplied +% Make an entry into the table of contents only if we have not done so before +\if@IEEEbiographyTOCentrynotmade% +% link labels to the biography counter so hyperref will jump +% to the biography, not the previous section +\setcounter{IEEEbiography}{-1}% +\refstepcounter{IEEEbiography}% +\addcontentsline{toc}{section}{Biographies}% +\global\@IEEEbiographyTOCentrynotmadefalse% +\fi% +% one more biography +\refstepcounter{IEEEbiography}% +% Make an entry for this name into the table of contents +\addcontentsline{toc}{subsection}{#2}% +% V1.6 properly handle if a new paragraph should occur while the +% hanging indent is still active. Do this by redefining \par so +% that it will not start a new paragraph. (But it will appear to the +% user as if it did.) Also, strip any leading pars, newlines, or spaces. +\let\@IEEEBIOORGparCMD=\par% save the original \par command +\edef\par{\hfil\break\indent}% the new \par will not be a "real" \par +\settoheight{\@IEEEtrantmpdimenA}{\@IEEEtempbiographybox}% get height of biography box +\@IEEEtrantmpdimenB=\@IEEEBIOhangdepth% +\@IEEEtrantmpcountA=\@IEEEtrantmpdimenB% countA has the hang depth +\divide\@IEEEtrantmpcountA by \baselineskip% calculates lines needed to produce the hang depth +\advance\@IEEEtrantmpcountA by 1% ensure we overestimate +% set the hanging indent +\hangindent\@IEEEBIOhangwidth% +\hangafter-\@IEEEtrantmpcountA% +% reference the top of the photo area to the top of a capital T +\settoheight{\@IEEEtrantmpdimenB}{\mbox{T}}% +% set the photo box, give it zero width and height so as not to disturb anything +\noindent\makebox[0pt][l]{\hspace{-\@IEEEBIOhangwidth}\raisebox{\@IEEEtrantmpdimenB}[0pt][0pt]{% +\raisebox{-\@IEEEBIOphotodepth}[0pt][0pt]{\@IEEEtempbiographybox}}}% +% now place the author name and begin the bio text +\noindent\textbf{#2\ }\@IEEEgobbleleadPARNLSP}{\relax\let\par=\@IEEEBIOORGparCMD\par% +% 7/2001 V1.5 detect when the biography text is shorter than the photo area +% and pad the unused area - preventing a collision from the next biography entry +% MDS +\ifnum \prevgraf <\@IEEEtrantmpcountA\relax% detect when the biography text is shorter than the photo + \advance\@IEEEtrantmpcountA by -\prevgraf% calculate how many lines we need to pad + \advance\@IEEEtrantmpcountA by -1\relax% we compensate for the fact that we indented an extra line + \@IEEEtrantmpdimenA=\baselineskip% calculate the length of the padding + \multiply\@IEEEtrantmpdimenA by \@IEEEtrantmpcountA% + \noindent\rule{0pt}{\@IEEEtrantmpdimenA}% insert an invisible support strut +\fi% +\par\normalfont} + + + +% V1.6 +% added biography without a photo environment +\newenvironment{IEEEbiographynophoto}[1]{% +% Make an entry into the table of contents only if we have not done so before +\if@IEEEbiographyTOCentrynotmade% +% link labels to the biography counter so hyperref will jump +% to the biography, not the previous section +\setcounter{IEEEbiography}{-1}% +\refstepcounter{IEEEbiography}% +\addcontentsline{toc}{section}{Biographies}% +\global\@IEEEbiographyTOCentrynotmadefalse% +\fi% +% one more biography +\refstepcounter{IEEEbiography}% +% Make an entry for this name into the table of contents +\addcontentsline{toc}{subsection}{#1}% +\normalfont\@IEEEcompsoconly{\sffamily}\footnotesize\interlinepenalty500% +\vskip 4\baselineskip plus 1fil minus 0\baselineskip% +\parskip=0pt\par% +\noindent\textbf{#1\ }\@IEEEgobbleleadPARNLSP}{\relax\par\normalfont} + + +% provide the user with some old font commands +% got this from article.cls +\DeclareOldFontCommand{\rm}{\normalfont\rmfamily}{\mathrm} +\DeclareOldFontCommand{\sf}{\normalfont\sffamily}{\mathsf} +\DeclareOldFontCommand{\tt}{\normalfont\ttfamily}{\mathtt} +\DeclareOldFontCommand{\bf}{\normalfont\bfseries}{\mathbf} +\DeclareOldFontCommand{\it}{\normalfont\itshape}{\mathit} +\DeclareOldFontCommand{\sl}{\normalfont\slshape}{\@nomath\sl} +\DeclareOldFontCommand{\sc}{\normalfont\scshape}{\@nomath\sc} +\DeclareRobustCommand*\cal{\@fontswitch\relax\mathcal} +\DeclareRobustCommand*\mit{\@fontswitch\relax\mathnormal} + + +% SPECIAL PAPER NOTICE COMMANDS +% +% holds the special notice text +\def\@IEEEspecialpapernotice{\relax} + +% for special papers, like invited papers, the user can do: +% \IEEEspecialpapernotice{(Invited Paper)} before \maketitle +\def\IEEEspecialpapernotice#1{\ifCLASSOPTIONconference% +\def\@IEEEspecialpapernotice{{\Large#1\vspace*{1em}}}% +\else% +\def\@IEEEspecialpapernotice{{\\*[1.5ex]\sublargesize\textit{#1}}\vspace*{-2ex}}% +\fi} + + + + +% PUBLISHER ID COMMANDS +% to insert a publisher's ID footer +% V1.6 \IEEEpubid has been changed so that the change in page size and style +% occurs in \maketitle. \IEEEpubid must now be issued prior to \maketitle +% use \IEEEpubidadjcol as before - in the second column of the title page +% These changes allow \maketitle to take the reduced page height into +% consideration when dynamically setting the space between the author +% names and the maintext. +% +% the amount the main text is pulled up to make room for the +% publisher's ID footer +% IEEE uses about 1.3\baselineskip for journals, +% dynamic title spacing will clean up the fraction +\def\@IEEEpubidpullup{1.3\baselineskip} +\ifCLASSOPTIONtechnote +% for technotes it must be an integer of baselineskip as there can be no +% dynamic title spacing for two column mode technotes (the title is in the +% in first column) and we should maintain an integer number of lines in the +% second column +% There are some examples (such as older issues of "Transactions on +% Information Theory") in which IEEE really pulls the text off the ID for +% technotes - about 0.55in (or 4\baselineskip). We'll use 2\baselineskip +% and call it even. +\def\@IEEEpubidpullup{2\baselineskip} +\fi + +% V1.7 compsoc does not use a pullup +\ifCLASSOPTIONcompsoc +\def\@IEEEpubidpullup{0pt} +\fi + +% holds the ID text +\def\@IEEEpubid{\relax} + +% flag so \maketitle can tell if \IEEEpubid was called +\newif\if@IEEEusingpubid +\global\@IEEEusingpubidfalse +% issue this command in the page to have the ID at the bottom +% V1.6 use before \maketitle +\def\IEEEpubid#1{\def\@IEEEpubid{#1}\global\@IEEEusingpubidtrue} + + +% command which will pull up (shorten) the column it is executed in +% to make room for the publisher ID. Place in the second column of +% the title page when using \IEEEpubid +% Is smart enough not to do anything when in single column text or +% if the user hasn't called \IEEEpubid +% currently needed in for the second column of a page with the +% publisher ID. If not needed in future releases, please provide this +% command and define it as \relax for backward compatibility +% v1.6b do not allow command to operate if the peer review option has been +% selected because \IEEEpubidadjcol will not be on the cover page. +% V1.7 do nothing if compsoc +\def\IEEEpubidadjcol{\ifCLASSOPTIONcompsoc\else\ifCLASSOPTIONpeerreview\else +\if@twocolumn\if@IEEEusingpubid\enlargethispage{-\@IEEEpubidpullup}\fi\fi\fi\fi} + +% Special thanks to Peter Wilson, Daniel Luecking, and the other +% gurus at comp.text.tex, for helping me to understand how best to +% implement the IEEEpubid command in LaTeX. + + + +%% Lockout some commands under various conditions + +% general purpose bit bucket +\newsavebox{\@IEEEtranrubishbin} + +% flags to prevent multiple warning messages +\newif\if@IEEEWARNthanks +\newif\if@IEEEWARNIEEEPARstart +\newif\if@IEEEWARNIEEEbiography +\newif\if@IEEEWARNIEEEbiographynophoto +\newif\if@IEEEWARNIEEEpubid +\newif\if@IEEEWARNIEEEpubidadjcol +\newif\if@IEEEWARNIEEEmembership +\newif\if@IEEEWARNIEEEaftertitletext +\@IEEEWARNthankstrue +\@IEEEWARNIEEEPARstarttrue +\@IEEEWARNIEEEbiographytrue +\@IEEEWARNIEEEbiographynophototrue +\@IEEEWARNIEEEpubidtrue +\@IEEEWARNIEEEpubidadjcoltrue +\@IEEEWARNIEEEmembershiptrue +\@IEEEWARNIEEEaftertitletexttrue + + +%% Lockout some commands when in various modes, but allow them to be restored if needed +%% +% save commands which might be locked out +% so that the user can later restore them if needed +\let\@IEEESAVECMDthanks\thanks +\let\@IEEESAVECMDIEEEPARstart\IEEEPARstart +\let\@IEEESAVECMDIEEEbiography\IEEEbiography +\let\@IEEESAVECMDendIEEEbiography\endIEEEbiography +\let\@IEEESAVECMDIEEEbiographynophoto\IEEEbiographynophoto +\let\@IEEESAVECMDendIEEEbiographynophoto\endIEEEbiographynophoto +\let\@IEEESAVECMDIEEEpubid\IEEEpubid +\let\@IEEESAVECMDIEEEpubidadjcol\IEEEpubidadjcol +\let\@IEEESAVECMDIEEEmembership\IEEEmembership +\let\@IEEESAVECMDIEEEaftertitletext\IEEEaftertitletext + + +% disable \IEEEPARstart when in draft mode +% This may have originally been done because the pre-V1.6 drop letter +% algorithm had problems with a non-unity baselinestretch +% At any rate, it seems too formal to have a drop letter in a draft +% paper. +\ifCLASSOPTIONdraftcls +\def\IEEEPARstart#1#2{#1#2\if@IEEEWARNIEEEPARstart\typeout{** ATTENTION: \noexpand\IEEEPARstart + is disabled in draft mode (line \the\inputlineno).}\fi\global\@IEEEWARNIEEEPARstartfalse} +\fi +% and for technotes +\ifCLASSOPTIONtechnote +\def\IEEEPARstart#1#2{#1#2\if@IEEEWARNIEEEPARstart\typeout{** WARNING: \noexpand\IEEEPARstart + is locked out for technotes (line \the\inputlineno).}\fi\global\@IEEEWARNIEEEPARstartfalse} +\fi + + +% lockout unneeded commands when in conference mode +\ifCLASSOPTIONconference +% when locked out, \thanks, \IEEEbiography, \IEEEbiographynophoto, \IEEEpubid, +% \IEEEmembership and \IEEEaftertitletext will all swallow their given text. +% \IEEEPARstart will output a normal character instead +% warn the user about these commands only once to prevent the console screen +% from filling up with redundant messages +\def\thanks#1{\if@IEEEWARNthanks\typeout{** WARNING: \noexpand\thanks + is locked out when in conference mode (line \the\inputlineno).}\fi\global\@IEEEWARNthanksfalse} +\def\IEEEPARstart#1#2{#1#2\if@IEEEWARNIEEEPARstart\typeout{** WARNING: \noexpand\IEEEPARstart + is locked out when in conference mode (line \the\inputlineno).}\fi\global\@IEEEWARNIEEEPARstartfalse} + + +% LaTeX treats environments and commands with optional arguments differently. +% the actual ("internal") command is stored as \\commandname +% (accessed via \csname\string\commandname\endcsname ) +% the "external" command \commandname is a macro with code to determine +% whether or not the optional argument is presented and to provide the +% default if it is absent. So, in order to save and restore such a command +% we would have to save and restore \\commandname as well. But, if LaTeX +% ever changes the way it names the internal names, the trick would break. +% Instead let us just define a new environment so that the internal +% name can be left undisturbed. +\newenvironment{@IEEEbogusbiography}[2][]{\if@IEEEWARNIEEEbiography\typeout{** WARNING: \noexpand\IEEEbiography + is locked out when in conference mode (line \the\inputlineno).}\fi\global\@IEEEWARNIEEEbiographyfalse% +\setbox\@IEEEtranrubishbin\vbox\bgroup}{\egroup\relax} +% and make biography point to our bogus biography +\let\IEEEbiography=\@IEEEbogusbiography +\let\endIEEEbiography=\end@IEEEbogusbiography + +\renewenvironment{IEEEbiographynophoto}[1]{\if@IEEEWARNIEEEbiographynophoto\typeout{** WARNING: \noexpand\IEEEbiographynophoto + is locked out when in conference mode (line \the\inputlineno).}\fi\global\@IEEEWARNIEEEbiographynophotofalse% +\setbox\@IEEEtranrubishbin\vbox\bgroup}{\egroup\relax} + +\def\IEEEpubid#1{\if@IEEEWARNIEEEpubid\typeout{** WARNING: \noexpand\IEEEpubid + is locked out when in conference mode (line \the\inputlineno).}\fi\global\@IEEEWARNIEEEpubidfalse} +\def\IEEEpubidadjcol{\if@IEEEWARNIEEEpubidadjcol\typeout{** WARNING: \noexpand\IEEEpubidadjcol + is locked out when in conference mode (line \the\inputlineno).}\fi\global\@IEEEWARNIEEEpubidadjcolfalse} +\def\IEEEmembership#1{\if@IEEEWARNIEEEmembership\typeout{** WARNING: \noexpand\IEEEmembership + is locked out when in conference mode (line \the\inputlineno).}\fi\global\@IEEEWARNIEEEmembershipfalse} +\def\IEEEaftertitletext#1{\if@IEEEWARNIEEEaftertitletext\typeout{** WARNING: \noexpand\IEEEaftertitletext + is locked out when in conference mode (line \the\inputlineno).}\fi\global\@IEEEWARNIEEEaftertitletextfalse} +\fi + + +% provide a way to restore the commands that are locked out +\def\IEEEoverridecommandlockouts{% +\typeout{** ATTENTION: Overriding command lockouts (line \the\inputlineno).}% +\let\thanks\@IEEESAVECMDthanks% +\let\IEEEPARstart\@IEEESAVECMDIEEEPARstart% +\let\IEEEbiography\@IEEESAVECMDIEEEbiography% +\let\endIEEEbiography\@IEEESAVECMDendIEEEbiography% +\let\IEEEbiographynophoto\@IEEESAVECMDIEEEbiographynophoto% +\let\endIEEEbiographynophoto\@IEEESAVECMDendIEEEbiographynophoto% +\let\IEEEpubid\@IEEESAVECMDIEEEpubid% +\let\IEEEpubidadjcol\@IEEESAVECMDIEEEpubidadjcol% +\let\IEEEmembership\@IEEESAVECMDIEEEmembership% +\let\IEEEaftertitletext\@IEEESAVECMDIEEEaftertitletext} + + + +% need a backslash character for typeout output +{\catcode`\|=0 \catcode`\\=12 +|xdef|@IEEEbackslash{\}} + + +% hook to allow easy disabling of all legacy warnings +\def\@IEEElegacywarn#1#2{\typeout{** ATTENTION: \@IEEEbackslash #1 is deprecated (line \the\inputlineno). +Use \@IEEEbackslash #2 instead.}} + + +% provide for legacy commands +\def\authorblockA{\@IEEElegacywarn{authorblockA}{IEEEauthorblockA}\IEEEauthorblockA} +\def\authorblockN{\@IEEElegacywarn{authorblockN}{IEEEauthorblockN}\IEEEauthorblockN} +\def\authorrefmark{\@IEEElegacywarn{authorrefmark}{IEEEauthorrefmark}\IEEEauthorrefmark} +\def\PARstart{\@IEEElegacywarn{PARstart}{IEEEPARstart}\IEEEPARstart} +\def\pubid{\@IEEElegacywarn{pubid}{IEEEpubid}\IEEEpubid} +\def\pubidadjcol{\@IEEElegacywarn{pubidadjcol}{IEEEpubidadjcol}\IEEEpubidadjcol} +\def\QED{\@IEEElegacywarn{QED}{IEEEQED}\IEEEQED} +\def\QEDclosed{\@IEEElegacywarn{QEDclosed}{IEEEQEDclosed}\IEEEQEDclosed} +\def\QEDopen{\@IEEElegacywarn{QEDopen}{IEEEQEDopen}\IEEEQEDopen} +\def\specialpapernotice{\@IEEElegacywarn{specialpapernotice}{IEEEspecialpapernotice}\IEEEspecialpapernotice} + + + +% provide for legacy environments +\def\biography{\@IEEElegacywarn{biography}{IEEEbiography}\IEEEbiography} +\def\biographynophoto{\@IEEElegacywarn{biographynophoto}{IEEEbiographynophoto}\IEEEbiographynophoto} +\def\keywords{\@IEEElegacywarn{keywords}{IEEEkeywords}\IEEEkeywords} +\def\endbiography{\endIEEEbiography} +\def\endbiographynophoto{\endIEEEbiographynophoto} +\def\endkeywords{\endIEEEkeywords} + + +% provide for legacy IED commands/lengths when possible +\let\labelindent\IEEElabelindent +\def\calcleftmargin{\@IEEElegacywarn{calcleftmargin}{IEEEcalcleftmargin}\IEEEcalcleftmargin} +\def\setlabelwidth{\@IEEElegacywarn{setlabelwidth}{IEEEsetlabelwidth}\IEEEsetlabelwidth} +\def\usemathlabelsep{\@IEEElegacywarn{usemathlabelsep}{IEEEusemathlabelsep}\IEEEusemathlabelsep} +\def\iedlabeljustifyc{\@IEEElegacywarn{iedlabeljustifyc}{IEEEiedlabeljustifyc}\IEEEiedlabeljustifyc} +\def\iedlabeljustifyl{\@IEEElegacywarn{iedlabeljustifyl}{IEEEiedlabeljustifyl}\IEEEiedlabeljustifyl} +\def\iedlabeljustifyr{\@IEEElegacywarn{iedlabeljustifyr}{IEEEiedlabeljustifyr}\IEEEiedlabeljustifyr} + + + +% let \proof use the IEEEtran version even after amsthm is loaded +% \proof is now deprecated in favor of \IEEEproof +\AtBeginDocument{\def\proof{\@IEEElegacywarn{proof}{IEEEproof}\IEEEproof}\def\endproof{\endIEEEproof}} + +% V1.7 \overrideIEEEmargins is no longer supported. +\def\overrideIEEEmargins{% +\typeout{** WARNING: \string\overrideIEEEmargins \space no longer supported (line \the\inputlineno).}% +\typeout{** Use the \string\CLASSINPUTinnersidemargin, \string\CLASSINPUToutersidemargin \space controls instead.}} + + +\endinput + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%% End of IEEEtran.cls %%%%%%%%%%%%%%%%%%%%%%%%%%%% +% That's all folks! + diff --git a/cometbft/v0.39/spec/consensus/consensus-paper/algorithmicplus.sty b/cometbft/v0.39/spec/consensus/consensus-paper/algorithmicplus.sty new file mode 100644 index 000000000..de7ca01ea --- /dev/null +++ b/cometbft/v0.39/spec/consensus/consensus-paper/algorithmicplus.sty @@ -0,0 +1,195 @@ +% ALGORITHMICPLUS STYLE +% for LaTeX version 2e +% Original ``algorithmic.sty'' by -- 1994 Peter Williams +% Bug fix (13 July 2004) by Arnaud Giersch +% Includes ideas from 'algorithmicext' by Martin Biely +% and 'distribalgo' by Xavier Defago +% Modifications: Martin Hutle +% +% This style file is free software; you can redistribute it and/or +% modify it under the terms of the GNU Lesser General Public +% License as published by the Free Software Foundation; either +% version 2 of the License, or (at your option) any later version. +% +% This style file is distributed in the hope that it will be useful, +% but WITHOUT ANY WARRANTY; without even the implied warranty of +% MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +% Lesser General Public License for more details. +% +% You should have received a copy of the GNU Lesser General Public +% License along with this style file; if not, write to the +% Free Software Foundation, Inc., 59 Temple Place - Suite 330, +% Boston, MA 02111-1307, USA. +% +\NeedsTeXFormat{LaTeX2e} +\ProvidesPackage{algorithmicplus} +\typeout{Document Style `algorithmicplus' - environment, replaces `algorithmic'} +% +\RequirePackage{ifthen} +\RequirePackage{calc} +\newboolean{ALC@noend} +\setboolean{ALC@noend}{false} +\newcounter{ALC@line} +\newcounter{ALC@rem} +\newcounter{ALC@depth} +\newcounter{ALCPLUS@lastline} +\newlength{\ALC@tlm} +% +\DeclareOption{noend}{\setboolean{ALC@noend}{true}} +% +\ProcessOptions +% +% ALGORITHMIC +\newcommand{\algorithmiclnosize}{\small} +\newcommand{\algorithmiclnofont}{\tt} +\newcommand{\algorithmiclnodelimiter}{:} +% +\newcommand{\algorithmicrequire}{\textbf{Require:}} +\newcommand{\algorithmicensure}{\textbf{Ensure:}} +\newcommand{\algorithmiccomment}[1]{\{#1\}} +\newcommand{\algorithmicend}{\textbf{end}} +\newcommand{\algorithmicif}{\textbf{if}} +\newcommand{\algorithmicthen}{\textbf{then}} +\newcommand{\algorithmicelse}{\textbf{else}} +\newcommand{\algorithmicelsif}{\algorithmicelse\ \algorithmicif} +\newcommand{\algorithmicendif}{\algorithmicend\ \algorithmicif} +\newcommand{\algorithmicfor}{\textbf{for}} +\newcommand{\algorithmicforall}{\textbf{for all}} +\newcommand{\algorithmicdo}{\textbf{do}} +\newcommand{\algorithmicendfor}{\algorithmicend\ \algorithmicfor} +\newcommand{\algorithmicwhile}{\textbf{while}} +\newcommand{\algorithmicendwhile}{\algorithmicend\ \algorithmicwhile} +\newcommand{\algorithmicloop}{\textbf{loop}} +\newcommand{\algorithmicendloop}{\algorithmicend\ \algorithmicloop} +\newcommand{\algorithmicrepeat}{\textbf{repeat}} +\newcommand{\algorithmicuntil}{\textbf{until}} +\def\ALC@item[#1]{% +\if@noparitem \@donoparitem + \else \if@inlabel \indent \par \fi + \ifhmode \unskip\unskip \par \fi + \if@newlist \if@nobreak \@nbitem \else + \addpenalty\@beginparpenalty + \addvspace\@topsep \addvspace{-\parskip}\fi + \else \addpenalty\@itempenalty \addvspace\itemsep + \fi + \global\@inlabeltrue +\fi +\everypar{\global\@minipagefalse\global\@newlistfalse + \if@inlabel\global\@inlabelfalse \hskip -\parindent \box\@labels + \penalty\z@ \fi + \everypar{}}\global\@nobreakfalse +\if@noitemarg \@noitemargfalse \if@nmbrlist \refstepcounter{\@listctr}\fi \fi +\sbox\@tempboxa{\makelabel{#1}}% +\global\setbox\@labels + \hbox{\unhbox\@labels \hskip \itemindent + \hskip -\labelwidth \hskip -\ALC@tlm + \ifdim \wd\@tempboxa >\labelwidth + \box\@tempboxa + \else \hbox to\labelwidth {\unhbox\@tempboxa}\fi + \hskip \ALC@tlm}\ignorespaces} +% +\newenvironment{algorithmic}[1][0]{ +\setcounter{ALC@depth}{\@listdepth}% +\let\@listdepth\c@ALC@depth% +\let\@item\ALC@item + \newcommand{\ALC@lno}{% +\ifthenelse{\equal{\arabic{ALC@rem}}{0}} +{{\algorithmiclnosize\algorithmiclnofont \arabic{ALC@line}\algorithmiclnodelimiter}}{}% +} +\let\@listii\@listi +\let\@listiii\@listi +\let\@listiv\@listi +\let\@listv\@listi +\let\@listvi\@listi +\let\@listvii\@listi + \newenvironment{ALC@g}{ + \begin{list}{\ALC@lno}{ \itemsep\z@ \itemindent\z@ + \listparindent\z@ \rightmargin\z@ + \topsep\z@ \partopsep\z@ \parskip\z@\parsep\z@ + \leftmargin 1em + \addtolength{\ALC@tlm}{\leftmargin} + } + } + {\end{list}} + \newcommand{\ALC@it}{\refstepcounter{ALC@line}\addtocounter{ALC@rem}{1}\ifthenelse{\equal{\arabic{ALC@rem}}{#1}}{\setcounter{ALC@rem}{0}}{}\item} + \newcommand{\ALC@com}[1]{\ifthenelse{\equal{##1}{default}}% +{}{\ \algorithmiccomment{##1}}} + \newcommand{\REQUIRE}{\item[\algorithmicrequire]} + \newcommand{\ENSURE}{\item[\algorithmicensure]} + \newcommand{\STATE}{\ALC@it} + \newcommand{\COMMENT}[1]{\algorithmiccomment{##1}} + \newenvironment{ALC@if}{\begin{ALC@g}}{\end{ALC@g}} + \newenvironment{ALC@for}{\begin{ALC@g}}{\end{ALC@g}} + \newenvironment{ALC@whl}{\begin{ALC@g}}{\end{ALC@g}} + \newenvironment{ALC@loop}{\begin{ALC@g}}{\end{ALC@g}} + \newenvironment{ALC@rpt}{\begin{ALC@g}}{\end{ALC@g}} + \renewcommand{\\}{\@centercr} + \newcommand{\IF}[2][default]{\ALC@it\algorithmicif\ ##2\ \algorithmicthen% +\ALC@com{##1}\begin{ALC@if}} + \newcommand{\ELSE}[1][default]{\end{ALC@if}\ALC@it\algorithmicelse% +\ALC@com{##1}\begin{ALC@if}} + \newcommand{\ELSIF}[2][default]% +{\end{ALC@if}\ALC@it\algorithmicelsif\ ##2\ \algorithmicthen% +\ALC@com{##1}\begin{ALC@if}} + \newcommand{\FOR}[2][default]{\ALC@it\algorithmicfor\ ##2\ \algorithmicdo% +\ALC@com{##1}\begin{ALC@for}} + \newcommand{\FORALL}[2][default]{\ALC@it\algorithmicforall\ ##2\ % +\algorithmicdo% +\ALC@com{##1}\begin{ALC@for}} + \newcommand{\WHILE}[2][default]{\ALC@it\algorithmicwhile\ ##2\ % +\algorithmicdo% +\ALC@com{##1}\begin{ALC@whl}} + \newcommand{\LOOP}[1][default]{\ALC@it\algorithmicloop% +\ALC@com{##1}\begin{ALC@loop}} + \newcommand{\REPEAT}[1][default]{\ALC@it\algorithmicrepeat% +\ALC@com{##1}\begin{ALC@rpt}} + \newcommand{\UNTIL}[1]{\end{ALC@rpt}\ALC@it\algorithmicuntil\ ##1} + \ifthenelse{\boolean{ALC@noend}}{ + \newcommand{\ENDIF}{\end{ALC@if}} + \newcommand{\ENDFOR}{\end{ALC@for}} + \newcommand{\ENDWHILE}{\end{ALC@whl}} + \newcommand{\ENDLOOP}{\end{ALC@loop}} + }{ + \newcommand{\ENDIF}{\end{ALC@if}\ALC@it\algorithmicendif} + \newcommand{\ENDFOR}{\end{ALC@for}\ALC@it\algorithmicendfor} + \newcommand{\ENDWHILE}{\end{ALC@whl}\ALC@it\algorithmicendwhile} + \newcommand{\ENDLOOP}{\end{ALC@loop}\ALC@it\algorithmicendloop} + } + \renewcommand{\@toodeep}{} + \begin{list}{\ALC@lno}{\setcounter{ALC@line}{0}\setcounter{ALC@rem}{0}% + \itemsep\z@ \itemindent\z@ \listparindent\z@% + \partopsep\z@ \parskip\z@ \parsep\z@% + \labelsep 0.5em \topsep 0.2em% +\ifthenelse{\equal{#1}{0}} + {\labelwidth 0.5em } + {\labelwidth 1.2em } +\leftmargin\labelwidth \addtolength{\leftmargin}{\labelsep} + \ALC@tlm\labelsep + } +} +{% +\setcounter{ALCPLUS@lastline}{\value{ALC@line}}% +\end{list}} + +\newcommand{\continuecounting}{\setcounter{ALC@line}{\value{ALCPLUS@lastline}}} +\newcommand{\startcounting}[1]{\setcounter{ALC@line}{#1}\addtocounter{ALC@line}{-1}} + +\newcommand{\EMPTY}{\item[]} +\newcommand{\SPACE}{\vspace{3mm}} +\newcommand{\SHORTSPACE}{\vspace{1mm}} +\newcommand{\newlinetag}[3]{\newcommand{#1}[#2]{\item[#3]}} +\newcommand{\newconstruct}[5]{% + \newenvironment{ALC@\string#1}{\begin{ALC@g}}{\end{ALC@g}} + \newcommand{#1}[2][default]{\ALC@it#2\ ##2\ #3% + \ALC@com{##1}\begin{ALC@\string#1}} + \ifthenelse{\boolean{ALC@noend}}{ + \newcommand{#4}{\end{ALC@\string#1}} + }{ + \newcommand{#4}{\end{ALC@\string#1}\ALC@it#5} + } +} + +\newconstruct{\INDENT}{}{}{\ENDINDENT}{} + +\newcommand{\setlinenosize}[1]{\renewcommand{\algorithmiclnosize}{#1}} +\newcommand{\setlinenofont}[1]{\renewcommand{\algorithmiclnofont}{#1}} diff --git a/cometbft/v0.39/spec/consensus/consensus-paper/conclusion.tex b/cometbft/v0.39/spec/consensus/consensus-paper/conclusion.tex new file mode 100644 index 000000000..dd17ccf44 --- /dev/null +++ b/cometbft/v0.39/spec/consensus/consensus-paper/conclusion.tex @@ -0,0 +1,16 @@ +\section{Conclusion} \label{sec:conclusion} + +We have proposed a new Byzantine-fault tolerant consensus algorithm that is the +core of the Tendermint BFT SMR platform. The algorithm is designed for the wide +area network with high number of mutually distrusted nodes that communicate +over gossip based peer-to-peer network. It has only a single mode of execution +and the communication pattern is very similar to the "normal" case of the +state-of-the art PBFT algorithm. The algorithm ensures termination with a novel +mechanism that takes advantage of the gossip based communication between nodes. +The proposed algorithm and the proofs are simple and elegant, and we believe +that this makes it easier to understand and implement correctly. + +\section*{Acknowledgment} + +We would like to thank Anton Kaliaev, Ismail Khoffi and Dahlia Malkhi for comments on an earlier version of the paper. We also want to thank Marko Vukolic, Ming Chuan Lin, Maria Potop-Butucaru, Sara Tucci, Antonella Del Pozzo and Yackolley Amoussou-Guenou for pointing out the liveness issues +in the previous version of the algorithm. Finally, we want to thank the Tendermint team members and all project contributors for making Tendermint such a great platform. diff --git a/cometbft/v0.39/spec/consensus/consensus-paper/consensus.tex b/cometbft/v0.39/spec/consensus/consensus-paper/consensus.tex new file mode 100644 index 000000000..3265b61c7 --- /dev/null +++ b/cometbft/v0.39/spec/consensus/consensus-paper/consensus.tex @@ -0,0 +1,397 @@ + +\section{Tendermint consensus algorithm} \label{sec:tendermint} + +\newcommand\Disseminate{\textbf{Disseminate}} + +\newcommand\Proposal{\mathsf{PROPOSAL}} +\newcommand\ProposalPart{\mathsf{PROPOSAL\mbox{-}PART}} +\newcommand\PrePrepare{\mathsf{INIT}} \newcommand\Prevote{\mathsf{PREVOTE}} +\newcommand\Precommit{\mathsf{PRECOMMIT}} +\newcommand\Decision{\mathsf{DECISION}} + +\newcommand\ViewChange{\mathsf{VC}} +\newcommand\ViewChangeAck{\mathsf{VC\mbox{-}ACK}} +\newcommand\NewPrePrepare{\mathsf{VC\mbox{-}INIT}} +\newcommand\coord{\mathsf{proposer}} + +\newcommand\newHeight{newHeight} \newcommand\newRound{newRound} +\newcommand\nil{nil} \newcommand\id{id} \newcommand{\propose}{propose} +\newcommand\prevote{prevote} \newcommand\prevoteWait{prevoteWait} +\newcommand\precommit{precommit} \newcommand\precommitWait{precommitWait} +\newcommand\commit{commit} + +\newcommand\timeoutPropose{timeoutPropose} +\newcommand\timeoutPrevote{timeoutPrevote} +\newcommand\timeoutPrecommit{timeoutPrecommit} +\newcommand\proofOfLocking{proof\mbox{-}of\mbox{-}locking} + +\begin{algorithm}[htb!] \def\baselinestretch{1} \scriptsize\raggedright + \begin{algorithmic}[1] + \SHORTSPACE + \INIT{} + \STATE $h_p := 0$ + \COMMENT{current height, or consensus instance we are currently executing} + \STATE $round_p := 0$ \COMMENT{current round number} + \STATE $step_p \in \set{\propose, \prevote, \precommit}$ + \STATE $decision_p[] := nil$ + \STATE $lockedValue_p := nil$ + \STATE $lockedRound_p := -1$ + \STATE $validValue_p := nil$ + \STATE $validRound_p := -1$ + \ENDINIT + \SHORTSPACE + \STATE \textbf{upon} start \textbf{do} $StartRound(0)$ + \SHORTSPACE + \FUNCTION{$StartRound(round)$} \label{line:tab:startRound} + \STATE $round_p \assign round$ + \STATE $step_p \assign \propose$ + \IF{$\coord(h_p, round_p) = p$} + \IF{$validValue_p \neq \nil$} \label{line:tab:isThereLockedValue} + \STATE $proposal \assign validValue_p$ \ELSE \STATE $proposal \assign + getValue()$ + \label{line:tab:getValidValue} + \ENDIF + \STATE \Broadcast\ $\li{\Proposal,h_p, round_p, proposal, validRound_p}$ + \label{line:tab:send-proposal} + \ELSE + \STATE \textbf{schedule} $OnTimeoutPropose(h_p, + round_p)$ to be executed \textbf{after} $\timeoutPropose(round_p)$ + \ENDIF + \ENDFUNCTION + + \SPACE + \UPON{$\li{\Proposal,h_p,round_p, v, -1}$ \From\ $\coord(h_p,round_p)$ + \With\ $step_p = \propose$} \label{line:tab:recvProposal} + \IF{$valid(v) \wedge (lockedRound_p = -1 \vee lockedValue_p = v$)} + \label{line:tab:accept-proposal-2} + \STATE \Broadcast \ $\li{\Prevote,h_p,round_p,id(v)}$ + \label{line:tab:prevote-proposal} + \ELSE + \label{line:tab:acceptProposal1} + \STATE \Broadcast \ $\li{\Prevote,h_p,round_p,\nil}$ + \label{line:tab:prevote-nil} + \ENDIF + \STATE $step_p \assign \prevote$ \label{line:tab:setStateToPrevote1} + \ENDUPON + + \SPACE + \UPON{$\li{\Proposal,h_p,round_p, v, vr}$ \From\ $\coord(h_p,round_p)$ + \textbf{AND} $2f+1$ $\li{\Prevote,h_p, vr,id(v)}$ \With\ $step_p = \propose \wedge (vr \ge 0 \wedge vr < round_p)$} + \label{line:tab:acceptProposal} + \IF{$valid(v) \wedge (lockedRound_p \le vr + \vee lockedValue_p = v)$} \label{line:tab:cond-prevote-higher-proposal} + \STATE \Broadcast \ $\li{\Prevote,h_p,round_p,id(v)}$ + \label{line:tab:prevote-higher-proposal} + \ELSE + \label{line:tab:acceptProposal2} + \STATE \Broadcast \ $\li{\Prevote,h_p,round_p,\nil}$ + \label{line:tab:prevote-nil2} + \ENDIF + \STATE $step_p \assign \prevote$ \label{line:tab:setStateToPrevote3} + \ENDUPON + + \SPACE + \UPON{$2f+1$ $\li{\Prevote,h_p, round_p,*}$ \With\ $step_p = \prevote$ for the first time} + \label{line:tab:recvAny2/3Prevote} + \STATE \textbf{schedule} $OnTimeoutPrevote(h_p, round_p)$ to be executed \textbf{after} $\timeoutPrevote(round_p)$ \label{line:tab:timeoutPrevote} + \ENDUPON + + \SPACE + \UPON{$\li{\Proposal,h_p,round_p, v, *}$ \From\ $\coord(h_p,round_p)$ + \textbf{AND} $2f+1$ $\li{\Prevote,h_p, round_p,id(v)}$ \With\ $valid(v) \wedge step_p \ge \prevote$ for the first time} + \label{line:tab:recvPrevote} + \IF{$step_p = \prevote$} + \STATE $lockedValue_p \assign v$ \label{line:tab:setLockedValue} + \STATE $lockedRound_p \assign round_p$ \label{line:tab:setLockedRound} + \STATE \Broadcast \ $\li{\Precommit,h_p,round_p,id(v))}$ + \label{line:tab:precommit-v} + \STATE $step_p \assign \precommit$ \label{line:tab:setStateToCommit} + \ENDIF + \STATE $validValue_p \assign v$ \label{line:tab:setValidRound} + \STATE $validRound_p \assign round_p$ \label{line:tab:setValidValue} + \ENDUPON + + \SHORTSPACE + \UPON{$2f+1$ $\li{\Prevote,h_p,round_p, \nil}$ + \With\ $step_p = \prevote$} + \STATE \Broadcast \ $\li{\Precommit,h_p,round_p, \nil}$ + \label{line:tab:precommit-v-1} + \STATE $step_p \assign \precommit$ + \ENDUPON + + \SPACE + \UPON{$2f+1$ $\li{\Precommit,h_p,round_p,*}$ for the first time} + \label{line:tab:startTimeoutPrecommit} + \STATE \textbf{schedule} $OnTimeoutPrecommit(h_p, round_p)$ to be executed \textbf{after} $\timeoutPrecommit(round_p)$ + + \ENDUPON + + \SPACE + \UPON{$\li{\Proposal,h_p,r, v, *}$ \From\ $\coord(h_p,r)$ \textbf{AND} + $2f+1$ $\li{\Precommit,h_p,r,id(v)}$ \With\ $decision_p[h_p] = \nil$} + \label{line:tab:onDecideRule} + \IF{$valid(v)$} \label{line:tab:validDecisionValue} + \STATE $decision_p[h_p] = v$ \label{line:tab:decide} + \STATE$h_p \assign h_p + 1$ \label{line:tab:increaseHeight} + \STATE reset $lockedRound_p$, $lockedValue_p$, $validRound_p$ and $validValue_p$ to initial values + and empty message log + \STATE $StartRound(0)$ + \ENDIF + \ENDUPON + + \SHORTSPACE + \UPON{$f+1$ $\li{*,h_p,round, *, *}$ \textbf{with} $round > round_p$} + \label{line:tab:skipRounds} + \STATE $StartRound(round)$ \label{line:tab:nextRound2} + \ENDUPON + + \SHORTSPACE + \FUNCTION{$OnTimeoutPropose(height,round)$} \label{line:tab:onTimeoutPropose} + \IF{$height = h_p \wedge round = round_p \wedge step_p = \propose$} + \STATE \Broadcast \ $\li{\Prevote,h_p,round_p, \nil}$ + \label{line:tab:prevote-nil-on-timeout} + \STATE $step_p \assign \prevote$ + \ENDIF + \ENDFUNCTION + + \SHORTSPACE + \FUNCTION{$OnTimeoutPrevote(height,round)$} \label{line:tab:onTimeoutPrevote} + \IF{$height = h_p \wedge round = round_p \wedge step_p = \prevote$} + \STATE \Broadcast \ $\li{\Precommit,h_p,round_p,\nil}$ + \label{line:tab:precommit-nil-onTimeout} + \STATE $step_p \assign \precommit$ + \ENDIF + \ENDFUNCTION + + \SHORTSPACE + \FUNCTION{$OnTimeoutPrecommit(height,round)$} \label{line:tab:onTimeoutPrecommit} + \IF{$height = h_p \wedge round = round_p$} + \STATE $StartRound(round_p + 1)$ \label{line:tab:nextRound} + \ENDIF + \ENDFUNCTION + \end{algorithmic} \caption{Tendermint consensus algorithm} + \label{alg:tendermint} +\end{algorithm} + +In this section we present the Tendermint Byzantine fault-tolerant consensus +algorithm. The algorithm is specified by the pseudo-code shown in +Algorithm~\ref{alg:tendermint}. We present the algorithm as a set of \emph{upon +rules} that are executed atomically\footnote{In case several rules are active +at the same time, the first rule to be executed is picked randomly. The +correctness of the algorithm does not depend on the order in which rules are +executed.}. We assume that processes exchange protocol messages using a gossip +protocol and that both sent and received messages are stored in a local message +log for every process. An upon rule is triggered once the message log contains +messages such that the corresponding condition evaluates to $\tt{true}$. The +condition that assumes reception of $X$ messages of a particular type and +content denotes reception of messages whose senders have aggregate voting power at +least equal to $X$. For example, the condition $2f+1$ $\li{\Precommit,h_p,r,id(v)}$, +evaluates to true upon reception of $\Precommit$ messages for height $h_p$, +a round $r$ and with value equal to $id(v)$ whose senders have aggregate voting +power at least equal to $2f+1$. Some of the rules ends with "for the first time" constraint +to denote that it is triggered only the first time a corresponding condition evaluates +to $\tt{true}$. This is because those rules do not always change the state of algorithm +variables so without this constraint, the algorithm could keep +executing those rules forever. The variables with index $p$ are process local state +variables, while variables without index $p$ are value placeholders. The sign +$*$ denotes any value. + +We denote with $n$ the total voting power of processes in the system, and we +assume that the total voting power of faulty processes in the system is bounded +with a system parameter $f$. The algorithm assumes that $n > 3f$, i.e., it +requires that the total voting power of faulty processes is smaller than one +third of the total voting power. For simplicity we present the algorithm for +the case $n = 3f + 1$. + +The algorithm proceeds in rounds, where each round has a dedicated +\emph{proposer}. The mapping of rounds to proposers is known to all processes +and is given as a function $\coord(h, round)$, returning the proposer for +the round $round$ in the consensus instance $h$. We +assume that the proposer selection function is weighted round-robin, where +processes are rotated proportional to their voting power\footnote{A validator +with more voting power is selected more frequently, proportional to its power. +More precisely, during a sequence of rounds of size $n$, every process is +proposer in a number of rounds equal to its voting power.}. +The internal protocol state transitions are triggered by message reception and +by expiration of timeouts. There are three timeouts in Algorithm \ref{alg:tendermint}: +$\timeoutPropose$, $\timeoutPrevote$ and $\timeoutPrecommit$. +The timeouts prevent the algorithm from blocking and +waiting forever for some condition to be true, ensure that processes continuously +transition between rounds, and guarantee that eventually (after GST) communication +between correct processes is timely and reliable so they can decide. +The last role is achieved by increasing the timeouts with every new round $r$, +i.e, $timeoutX(r) = initTimeoutX + r*timeoutDelta$; +they are reset for every new height (consensus +instance). + +Processes exchange the following messages in Tendermint: $\Proposal$, +$\Prevote$ and $\Precommit$. The $\Proposal$ message is used by the proposer of +the current round to suggest a potential decision value, while $\Prevote$ and +$\Precommit$ are votes for a proposed value. According to the classification of +consensus algorithms from \cite{RMS10:dsn}, Tendermint, like PBFT +\cite{CL02:tcs} and DLS \cite{DLS88:jacm}, belongs to class 3, so it requires +two voting steps (three communication exchanges in total) to decide a value. +The Tendermint consensus algorithm is designed for the blockchain context where +the value to decide is a block of transactions (ie. it is potentially quite +large, consisting of many transactions). Therefore, in the Algorithm +\ref{alg:tendermint} (similar as in \cite{CL02:tcs}) we are explicit about +sending a value (block of transactions) and a small, constant size value id (a +unique value identifier, normally a hash of the value, i.e., if $\id(v) = +\id(v')$, then $v=v'$). The $\Proposal$ message is the only one carrying the +value; $\Prevote$ and $\Precommit$ messages carry the value id. A correct +process decides on a value $v$ in Tendermint upon receiving the $\Proposal$ for +$v$ and $2f+1$ voting-power equivalent $\Precommit$ messages for $\id(v)$ in +some round $r$. In order to send $\Precommit$ message for $v$ in a round $r$, a +correct process waits to receive the $\Proposal$ and $2f+1$ of the +corresponding $\Prevote$ messages in the round $r$. Otherwise, +it sends $\Precommit$ message with a special $\nil$ value. +This ensures that correct processes can $\Precommit$ only a +single value (or $\nil$) in a round. As +proposers may be faulty, the proposed value is treated by correct processes as +a suggestion (it is not blindly accepted), and a correct process tells others +if it accepted the $\Proposal$ for value $v$ by sending $\Prevote$ message for +$\id(v)$; otherwise it sends $\Prevote$ message with the special $\nil$ value. + +Every process maintains the following variables in the Algorithm +\ref{alg:tendermint}: $step$, $lockedValue$, $lockedRound$, $validValue$ and +$validRound$. The $step$ denotes the current state of the internal Tendermint +state machine, i.e., it reflects the stage of the algorithm execution in the +current round. The $lockedValue$ stores the most recent value (with respect to +a round number) for which a $\Precommit$ message has been sent. The +$lockedRound$ is the last round in which the process sent a $\Precommit$ +message that is not $\nil$. We also say that a correct process locks a value +$v$ in a round $r$ by setting $lockedValue = v$ and $lockedRound = r$ before +sending $\Precommit$ message for $\id(v)$. As a correct process can decide a +value $v$ only if $2f+1$ $\Precommit$ messages for $\id(v)$ are received, this +implies that a possible decision value is a value that is locked by at least +$f+1$ voting power equivalent of correct processes. Therefore, any value $v$ +for which $\Proposal$ and $2f+1$ of the corresponding $\Prevote$ messages are +received in some round $r$ is a \emph{possible decision} value. The role of the +$validValue$ variable is to store the most recent possible decision value; the +$validRound$ is the last round in which $validValue$ is updated. Apart from +those variables, a process also stores the current consensus instance ($h_p$, +called \emph{height} in Tendermint), and the current round number ($round_p$) +and attaches them to every message. Finally, a process also stores an array of +decisions, $decision_p$ (Tendermint assumes a sequence of consensus instances, +one for each height). + +Every round starts by a proposer suggesting a value with the $\Proposal$ +message (see line \ref{line:tab:send-proposal}). In the initial round of each +height, the proposer is free to chose the value to suggest. In the +Algorithm~\ref{alg:tendermint}, a correct process obtains a value to propose +using an external function $getValue()$ that returns a valid value to +propose. In the following rounds, a correct proposer will suggest a new value +only if $validValue = \nil$; otherwise $validValue$ is proposed (see +lines~\ref{line:tab:isThereLockedValue}-\ref{line:tab:getValidValue}). +In addition to the value proposed, the $\Proposal$ message also +contains the $validRound$ so other processes are informed about the last round +in which the proposer observed $validValue$ as a possible decision value. +Note that if a correct proposer $p$ sends $validValue$ with the $validRound$ in the +$\Proposal$, this implies that the process $p$ received $\Proposal$ and the +corresponding $2f+1$ $\Prevote$ messages for $validValue$ in the round +$validRound$. +If a correct process sends $\Proposal$ message with $validValue$ ($validRound > -1$) +at time $t > GST$, by the \emph{Gossip communication} property, the +corresponding $\Proposal$ and the $\Prevote$ messages will be received by all +correct processes before time $t+\Delta$. Therefore, all correct processes will +be able to verify the correctness of the suggested value as it is supported by +the $\Proposal$ and the corresponding $2f+1$ voting power equivalent $\Prevote$ +messages. + +A correct process $p$ accepts the proposal for a value $v$ (send $\Prevote$ +for $id(v)$) if an external \emph{valid} function returns $true$ for the value +$v$, and if $p$ hasn't locked any value ($lockedRound = -1$) or $p$ has locked +the value $v$ ($lockedValue = v$); see the line +\ref{line:tab:accept-proposal-2}. In case the proposed pair is $(v,vr \ge 0)$ and a +correct process $p$ has locked some value, it will accept +$v$ if it is a more recent possible decision value\footnote{As +explained above, the possible decision value in a round $r$ is the one for +which $\Proposal$ and the corresponding $2f+1$ $\Prevote$ messages are received +for the round $r$.}, $vr > lockedRound_p$, or if $lockedValue = v$ +(see line~\ref{line:tab:cond-prevote-higher-proposal}). Otherwise, a correct +process will reject the proposal by sending $\Prevote$ message with $\nil$ +value. A correct process will send $\Prevote$ message with $\nil$ value also in +case $\timeoutPropose$ expired (it is triggered when a correct process starts a +new round) and a process has not sent $\Prevote$ message in the current round +yet (see the line \ref{line:tab:onTimeoutPropose}). + +If a correct process receives $\Proposal$ message for some value $v$ and $2f+1$ +$\Prevote$ messages for $\id(v)$, then it sends $\Precommit$ message with +$\id(v)$. Otherwise, it sends $\Precommit$ $\nil$. A correct process will send +$\Precommit$ message with $\nil$ value also in case $\timeoutPrevote$ expired +(it is started when a correct process sent $\Prevote$ message and received any +$2f+1$ $\Prevote$ messages) and a process has not sent $\Precommit$ message in +the current round yet (see the line \ref{line:tab:onTimeoutPrecommit}). A +correct process decides on some value $v$ if it receives in some round $r$ +$\Proposal$ message for $v$ and $2f+1$ $\Precommit$ messages with $\id(v)$ (see +the line \ref{line:tab:decide}). To prevent the algorithm from blocking and +waiting forever for this condition to be true, the Algorithm +\ref{alg:tendermint} relies on $\timeoutPrecommit$. It is triggered after a +process receives any set of $2f+1$ $\Precommit$ messages for the current round. +If the $\timeoutPrecommit$ expires and a process has not decided yet, the +process starts the next round (see the line \ref{line:tab:onTimeoutPrecommit}). +When a correct process $p$ decides, it starts the next consensus instance +(for the next height). The \emph{Gossip communication} property ensures +that $\Proposal$ and $2f+1$ $\Prevote$ messages that led $p$ to decide +are eventually received by all correct processes, so they will also decide. + +\subsection{Termination mechanism} + +Tendermint ensures termination by a novel mechanism that benefits from the +gossip based nature of communication (see \emph{Gossip communication} +property). It requires managing two additional variables, $validValue$ and +$validRound$ that are then used by the proposer during the propose step as +explained above. The $validValue$ and $validRound$ are updated to $v$ and $r$ +by a correct process in a round $r$ when the process receives valid $\Proposal$ +message for the value $v$ and the corresponding $2f+1$ $\Prevote$ messages for +$id(v)$ in the round $r$ (see the rule at line~\ref{line:tab:recvPrevote}). + +We now give briefly the intuition how managing and proposing $validValue$ +and $validRound$ ensures termination. Formal treatment is left for +Section~\ref{sec:proof}. + +The first thing to note is that during good period, because of the +\emph{Gossip communication} property, if a correct process $p$ locks a value +$v$ in some round $r$, all correct processes will update $validValue$ to $v$ +and $validRound$ to $r$ before the end of the round $r$ (we prove this formally +in the Section~\ref{sec:proof}). The intuition is that messages that led to $p$ +locking a value $v$ in the round $r$ will be gossiped to all correct processes +before the end of the round $r$, so it will update $validValue$ and +$validRound$ (the line~\ref{line:tab:recvPrevote}). Therefore, if a correct +process locks some value during good period, $validValue$ and $validRound$ are +updated by all correct processes so that the value proposed in the following +rounds will be acceptable by all correct processes. Note +that it could happen that during good period, no correct process locks a value, +but some correct process $q$ updates $validValue$ and $validRound$ during some +round. As no correct process locks a value in this case, $validValue_q$ and +$validRound_q$ will also be acceptable by all correct processes as +$validRound_q > lockedRound_c$ for every correct process $c$ and as the +\emph{Gossip communication} property ensures that the corresponding $\Prevote$ +messages that $q$ received in the round $validRound_q$ are received by all +correct processes $\Delta$ time later. + +Finally, it could happen that after GST, there is a long sequence of rounds in which +no correct process neither locks a value nor update $validValue$ and $validRound$. +In this case, during this sequence of rounds, the proposed value suggested by correct +processes was not accepted by all correct processes. Note that this sequence of rounds +is always finite as at the beginning of every +round there is at least a single correct process $c$ such that $validValue_c$ +and $validRound_c$ are acceptable by every correct process. This is true as +there exists a correct process $c$ such that for every other correct process +$p$, $validRound_c > lockedRound_p$ or $validValue_c = lockedValue_p$. This is +true as $c$ is the process that has locked a value in the most recent round +among all correct processes (or no correct process locked any value). Therefore, +eventually $c$ will be the proper in some round and the proposed value will be accepted +by all correct processes, terminating therefore this sequence of +rounds. + +Therefore, updating $validValue$ and $validRound$ variables, and the +\emph{Gossip communication} property, together ensures that eventually, during +the good period, there exists a round with a correct proposer whose proposed +value will be accepted by all correct processes, and all correct processes will +terminate in that round. Note that this mechanism, contrary to the common +termination mechanism illustrated in the +Figure~\ref{ch3:fig:coordinator-change}, does not require exchanging any +additional information in addition to messages already sent as part of what is +normally being called "normal" case. + diff --git a/cometbft/v0.39/spec/consensus/consensus-paper/definitions.tex b/cometbft/v0.39/spec/consensus/consensus-paper/definitions.tex new file mode 100644 index 000000000..454dd445d --- /dev/null +++ b/cometbft/v0.39/spec/consensus/consensus-paper/definitions.tex @@ -0,0 +1,126 @@ +\section{Definitions} \label{sec:definitions} + +\subsection{Model} + +We consider a system of processes that communicate by exchanging messages. +Processes can be correct or faulty, where a faulty process can behave in an +arbitrary way, i.e., we consider Byzantine faults. We assume that each process +has some amount of voting power (voting power of a process can be $0$). +Processes in our model are not part of a single administrative domain; +therefore we cannot enforce a direct network connectivity between all +processes. Instead, we assume that each process is connected to a subset of +processes called peers, such that there is an indirect communication channel +between all correct processes. Communication between processes is established +using a gossip protocol \cite{Dem1987:gossip}. + +Formally, we model the network communication using a variant of the \emph{partially +synchronous system model}~\cite{DLS88:jacm}: in all executions of the system +there is a bound $\Delta$ and an instant GST (Global Stabilization Time) such +that all communication among correct processes after GST is reliable and +$\Delta$-timely, i.e., if a correct process $p$ sends message $m$ at time $t +\ge GST$ to a correct process $q$, then $q$ will receive $m$ before $t + +\Delta$\footnote{Note that as we do not assume direct communication channels + among all correct processes, this implies that before the message $m$ + reaches $q$, it might pass through a number of correct processes that will +forward the message $m$ using gossip protocol towards $q$.}. +In addition to the standard \emph{partially + synchronous system model}~\cite{DLS88:jacm}, we assume an auxiliary property +that captures gossip-based nature of communication\footnote{The details of the Tendermint gossip protocol will be discussed in a separate + technical report. }: + + +\begin{itemize} \item \emph{Gossip communication:} If a correct process $p$ + sends some message $m$ at time $t$, all correct processes will receive + $m$ before $max\{t, GST\} + \Delta$. Furthermore, if a correct process $p$ + receives some message $m$ at time $t$, all correct processes will receive + $m$ before $max\{t, GST\} + \Delta$. \end{itemize} + + +The bound $\Delta$ and GST are system +parameters whose values are not required to be known for the safety of our +algorithm. Termination of the algorithm is guaranteed within a bounded duration +after GST. In practice, the algorithm will work correctly in the slightly +weaker variant of the model where the system alternates between (long enough) +good periods (corresponds to the \emph{after} GST period where system is +reliable and $\Delta$-timely) and bad periods (corresponds to the period +\emph{before} GST during which the system is asynchronous and messages can be +lost), but consideration of the GST model simplifies the discussion. + +We assume that process steps (which might include sending and receiving +messages) take zero time. Processes are equipped with clocks so they can +measure local timeouts. +Spoofing/impersonation attacks are assumed to be impossible at all times due to +the use of public-key cryptography, i.e., we assume that all protocol messages contains a digital signature. +Therefore, when a correct +process $q$ receives a signed message $m$ from its peer, the process $q$ can +verify who was the original sender of the message $m$ and if the message signature is valid. +We do not explicitly state a signature verification step in the pseudo-code of the algorithm to improve readability; +we assume that only messages with the valid signature are considered at that level (and messages with invalid signatures +are dropped). + + + +%Messages that are being gossiped are created by the consensus layer. We can + %think about consensus protocol as a content creator, which %defines what + %messages should be disseminated using the gossip protocol. A correct + %process creates the message for dissemination either i) %explicitly, by + %invoking \emph{send} function as part of the consensus protocol or ii) + %implicitly, by receiving a message from some other %process. Note that in + %the case ii) gossiping of messages is implicit, i.e., it happens without + %explicit send clause in the consensus algorithm %whenever a correct + %process receives some messages in the consensus algorithm\footnote{If a + %message is received by a correct process at %the consensus level then it + %is considered valid from the protocol point of view, i.e., it has a + %correct signature, a proper message structure %and a valid height and + %round number.}. + +%\item Processes keep resending messages (in case of failures or message loss) + %until all its peers get them. This ensures that every message %sent or + %received by a correct process is eventually received by all correct + %processes. + +\subsection{State Machine Replication} + +State machine replication (SMR) is a general approach for replicating services +modeled as a deterministic state machine~\cite{Lam78:cacm,Sch90:survey}. The +key idea of this approach is to guarantee that all replicas start in the same +state and then apply requests from clients in the same order, thereby +guaranteeing that the replicas' states will not diverge. Following +Schneider~\cite{Sch90:survey}, we note that the following is key for +implementing a replicated state machine tolerant to (Byzantine) faults: + +\begin{itemize} \item \emph{Replica Coordination.} All [non-faulty] replicas + receive and process the same sequence of requests. \end{itemize} + +Moreover, as Schneider also notes, this property can be decomposed into two +parts, \emph{Agreement} and \emph{Order}: Agreement requires all (non-faulty) +replicas to receive all requests, and Order requires that the order of received +requests is the same at all replicas. + +There is an additional requirement that needs to be ensured by Byzantine +tolerant state machine replication: only requests (called transactions in the +Tendermint terminology) proposed by clients are executed. In Tendermint, +transaction verification is the responsibility of the service that is being +replicated; upon receiving a transaction from the client, the Tendermint +process will ask the service if the request is valid, and only valid requests +will be processed. + + \subsection{Consensus} \label{sec:consensus} + +Tendermint solves state machine replication by sequentially executing consensus +instances to agree on each block of transactions that are +then executed by the service being replicated. We consider a variant of the +Byzantine consensus problem called Validity Predicate-based Byzantine consensus +that is motivated by blockchain systems~\cite{GLR17:red-belly-bc}. The problem +is defined by an agreement, a termination, and a validity property. + + \begin{itemize} \item \emph{Agreement:} No two correct processes decide on + different values. \item \emph{Termination:} All correct processes + eventually decide on a value. \item \emph{Validity:} A decided value + is valid, i.e., it satisfies the predefined predicate denoted + \emph{valid()}. \end{itemize} + + This variant of the Byzantine consensus problem has an application-specific + \emph{valid()} predicate to indicate whether a value is valid. In the context + of blockchain systems, for example, a value is not valid if it does not + contain an appropriate hash of the last value (block) added to the blockchain. diff --git a/cometbft/v0.39/spec/consensus/consensus-paper/homodel.sty b/cometbft/v0.39/spec/consensus/consensus-paper/homodel.sty new file mode 100644 index 000000000..19f83e926 --- /dev/null +++ b/cometbft/v0.39/spec/consensus/consensus-paper/homodel.sty @@ -0,0 +1,32 @@ +\newcommand{\NC}{\mbox{\it NC}} +\newcommand{\HO}{\mbox{\it HO}} +\newcommand{\AS}{\mbox{\it AS}} +\newcommand{\SK}{\mbox{\it SK}} +\newcommand{\SHO}{\mbox{\it SHO}} +\newcommand{\AHO}{\mbox{\it AHO}} +\newcommand{\CONS}{\mbox{\it CONS}} +\newcommand{\K}{\mbox{\it K}} + +\newcommand{\Alg}{\mathcal{A}} +\newcommand{\Pred}{\mathcal{P}} +\newcommand{\Spr}{S_p^r} +\newcommand{\Tpr}{T_p^r} +\newcommand{\mupr}{\vec{\mu}_p^{\,r}} + +\newcommand{\MSpr}{S_p^{\rho}} +\newcommand{\MTpr}{T_p^{\rho}} + + + +\newconstruct{\SEND}{$\Spr$:}{}{\ENDSEND}{} +\newconstruct{\TRAN}{$\Tpr$:}{}{\ENDTRAN}{} +\newconstruct{\ROUND}{\textbf{Round}}{\!\textbf{:}}{\ENDROUND}{} +\newconstruct{\VARIABLES}{\textbf{Variables:}}{}{\ENDVARIABLES}{} +\newconstruct{\INIT}{\textbf{Initialization:}}{}{\ENDINIT}{} + +\newconstruct{\MSEND}{$\MSpr$:}{}{\ENDMSEND}{} +\newconstruct{\MTRAN}{$\MTpr$:}{}{\ENDMTRAN}{} + +\newconstruct{\SROUND}{\textbf{Selection Round}}{\!\textbf{:}}{\ENDSROUND}{} +\newconstruct{\VROUND}{\textbf{Validation Round}}{\!\textbf{:}}{\ENDVROUND}{} +\newconstruct{\DROUND}{\textbf{Decision Round}}{\!\textbf{:}}{\ENDDROUND}{} diff --git a/cometbft/v0.39/spec/consensus/consensus-paper/intro.tex b/cometbft/v0.39/spec/consensus/consensus-paper/intro.tex new file mode 100644 index 000000000..493b509e9 --- /dev/null +++ b/cometbft/v0.39/spec/consensus/consensus-paper/intro.tex @@ -0,0 +1,138 @@ +\section{Introduction} \label{sec:tendermint} + +Consensus is a fundamental problem in distributed computing. It +is important because of it's role in State Machine Replication (SMR), a generic +approach for replicating services that can be modeled as a deterministic state +machine~\cite{Lam78:cacm, Sch90:survey}. The key idea of this approach is that +service replicas start in the same initial state, and then execute requests +(also called transactions) in the same order; thereby guaranteeing that +replicas stay in sync with each other. The role of consensus in the SMR +approach is ensuring that all replicas receive transactions in the same order. +Traditionally, deployments of SMR based systems are in data-center settings +(local area network), have a small number of replicas (three to seven) and are +typically part of a single administration domain (e.g., Chubby +\cite{Bur:osdi06}); therefore they handle benign (crash) failures only, as more +general forms of failure (in particular, malicious or Byzantine faults) are +considered to occur with only negligible probability. + +The success of cryptocurrencies and blockchain systems in recent years (e.g., +\cite{Nak2012:bitcoin, But2014:ethereum}) pose a whole new set of challenges on +the design and deployment of SMR based systems: reaching agreement over wide +area network, among large number of nodes (hundreds or thousands) that are not +part of the same administrative domain, and where a subset of nodes can behave +maliciously (Byzantine faults). Furthermore, contrary to the previous +data-center deployments where nodes are fully connected to each other, in +blockchain systems, a node is only connected to a subset of other nodes, so +communication is achieved by gossip-based peer-to-peer protocols. +The new requirements demand designs and algorithms that are not necessarily +present in the classical academic literature on Byzantine fault tolerant +consensus (or SMR) systems (e.g., \cite{DLS88:jacm, CL02:tcs}) as the primary +focus was different setup. + +In this paper we describe a novel Byzantine-fault tolerant consensus algorithm +that is the core of the BFT SMR platform called Tendermint\footnote{The + Tendermint platform is available open source at + https://github.com/tendermint/tendermint.}. The Tendermint platform consists of +a high-performance BFT SMR implementation written in Go, a flexible interface +for +building arbitrary deterministic applications above the consensus, and a suite +of tools for deployment and management. + +The Tendermint consensus algorithm is inspired by the PBFT SMR +algorithm~\cite{CL99:osdi} and the DLS algorithm for authenticated faults (the +Algorithm 2 from \cite{DLS88:jacm}). Similar to DLS algorithm, Tendermint +proceeds in +rounds\footnote{Tendermint is not presented in the basic round model of + \cite{DLS88:jacm}. Furthermore, we use the term round differently than in + \cite{DLS88:jacm}; in Tendermint a round denotes a sequence of communication + steps instead of a single communication step in \cite{DLS88:jacm}.}, where each +round has a dedicated proposer (also called coordinator or +leader) and a process proceeds to a new round as part of normal +processing (not only in case the proposer is faulty or suspected as being faulty +by enough processes as in PBFT). +The communication pattern of each round is very similar to the "normal" case +of PBFT. Therefore, in preferable conditions (correct proposer, timely and +reliable communication between correct processes), Tendermint decides in three +communication steps (the same as PBFT). + +The major novelty and contribution of the Tendermint consensus algorithm is a +new termination mechanism. As explained in \cite{MHS09:opodis, RMS10:dsn}, the +existing BFT consensus (and SMR) algorithms for the partially synchronous +system model (for example PBFT~\cite{CL99:osdi}, \cite{DLS88:jacm}, +\cite{MA06:tdsc}) typically relies on the communication pattern illustrated in +Figure~\ref{ch3:fig:coordinator-change} for termination. The +Figure~\ref{ch3:fig:coordinator-change} illustrates messages exchanged during +the proposer change when processes start a new round\footnote{There is no + consistent terminology in the distributed computing terminology on naming + sequence of communication steps that corresponds to a logical unit. It is + sometimes called a round, phase or a view.}. It guarantees that eventually (ie. +after some Global Stabilization Time, GST), there exists a round with a correct +proposer that will bring the system into a univalent configuration. +Intuitively, in a round in which the proposed value is accepted +by all correct processes, and communication between correct processes is +timely and reliable, all correct processes decide. + + +\begin{figure}[tbh!] \def\rdstretch{5} \def\ystretch{3} \centering + \begin{rounddiag}{4}{2} \round{1}{~} \rdmessage{1}{1}{$v_1$} + \rdmessage{2}{1}{$v_2$} \rdmessage{3}{1}{$v_3$} \rdmessage{4}{1}{$v_4$} + \round{2}{~} \rdmessage{1}{1}{$x, [v_{1..4}]$} + \rdmessage{1}{2}{$~~~~~~x, [v_{1..4}]$} \rdmessage{1}{3}{$~~~~~~~~x, + [v_{1..4}]$} \rdmessage{1}{4}{$~~~~~~~x, [v_{1..4}]$} \end{rounddiag} + \vspace{-5mm} \caption{\boldmath Proposer (coordinator) change: $p_1$ is the + new proposer.} \label{ch3:fig:coordinator-change} \end{figure} + +To ensure that a proposed value is accepted by all correct +processes\footnote{The proposed value is not blindly accepted by correct + processes in BFT algorithms. A correct process always verifies if the proposed + value is safe to be accepted so that safety properties of consensus are not + violated.} +a proposer will 1) build the global state by receiving messages from other +processes, 2) select the safe value to propose and 3) send the selected value +together with the signed messages +received in the first step to support it. The +value $v_i$ that a correct process sends to the next proposer normally +corresponds to a value the process considers as acceptable for a decision: + +\begin{itemize} \item in PBFT~\cite{CL99:osdi} and DLS~\cite{DLS88:jacm} it is + not the value itself but a set of $2f+1$ signed messages with the same + value id, \item in Fast Byzantine Paxos~\cite{MA06:tdsc} the value + itself is being sent. \end{itemize} + +In both cases, using this mechanism in our system model (ie. high +number of nodes over gossip based network) would have high communication +complexity that increases with the number of processes: in the first case as +the message sent depends on the total number of processes, and in the second +case as the value (block of transactions) is sent by each process. The set of +messages received in the first step are normally piggybacked on the proposal +message (in the Figure~\ref{ch3:fig:coordinator-change} denoted with +$[v_{1..4}]$) to justify the choice of the selected value $x$. Note that +sending this message also does not scale with the number of processes in the +system. + +We designed a novel termination mechanism for Tendermint that better suits the +system model we consider. It does not require additional communication (neither +sending new messages nor piggybacking information on the existing messages) and +it is fully based on the communication pattern that is very similar to the +normal case in PBFT \cite{CL99:osdi}. Therefore, there is only a single mode of +execution in Tendermint, i.e., there is no separation between the normal and +the recovery mode, which is the case in other PBFT-like protocols (e.g., +\cite{CL99:osdi}, \cite{Ver09:spinning} or \cite{Cle09:aardvark}). We believe +this makes Tendermint simpler to understand and implement correctly. + +Note that the orthogonal approach for reducing message complexity in order to +improve +scalability and decentralization (number of processes) of BFT consensus +algorithms is using advanced cryptography (for example Boneh-Lynn-Shacham (BLS) +signatures \cite{BLS2001:crypto}) as done for example in SBFT +\cite{Gue2018:sbft}. + +The remainder of the paper is as follows: Section~\ref{sec:definitions} defines +the system model and gives the problem definitions. Tendermint +consensus algorithm is presented in Section~\ref{sec:tendermint} and the +proofs are given in Section~\ref{sec:proof}. We conclude in +Section~\ref{sec:conclusion}. + + + + diff --git a/cometbft/v0.39/spec/consensus/consensus-paper/latex8.bst b/cometbft/v0.39/spec/consensus/consensus-paper/latex8.bst new file mode 100644 index 000000000..2c7af5647 --- /dev/null +++ b/cometbft/v0.39/spec/consensus/consensus-paper/latex8.bst @@ -0,0 +1,1124 @@ + +% --------------------------------------------------------------- +% +% $Id: latex8.bst,v 1.1 1995/09/15 15:13:49 ienne Exp $ +% +% by Paolo.Ienne@di.epfl.ch +% + +% --------------------------------------------------------------- +% +% no guarantee is given that the format corresponds perfectly to +% IEEE 8.5" x 11" Proceedings, but most features should be ok. +% +% --------------------------------------------------------------- +% +% `latex8' from BibTeX standard bibliography style `abbrv' +% version 0.99a for BibTeX versions 0.99a or later, LaTeX version 2.09. +% Copyright (C) 1985, all rights reserved. +% Copying of this file is authorized only if either +% (1) you make absolutely no changes to your copy, including name, or +% (2) if you do make changes, you name it something other than +% btxbst.doc, plain.bst, unsrt.bst, alpha.bst, and abbrv.bst. +% This restriction helps ensure that all standard styles are identical. +% The file btxbst.doc has the documentation for this style. + +ENTRY + { address + author + booktitle + chapter + edition + editor + howpublished + institution + journal + key + month + note + number + organization + pages + publisher + school + series + title + type + volume + year + } + {} + { label } + +INTEGERS { output.state before.all mid.sentence after.sentence after.block } + +FUNCTION {init.state.consts} +{ #0 'before.all := + #1 'mid.sentence := + #2 'after.sentence := + #3 'after.block := +} + +STRINGS { s t } + +FUNCTION {output.nonnull} +{ 's := + output.state mid.sentence = + { ", " * write$ } + { output.state after.block = + { add.period$ write$ + newline$ + "\newblock " write$ + } + { output.state before.all = + 'write$ + { add.period$ " " * write$ } + if$ + } + if$ + mid.sentence 'output.state := + } + if$ + s +} + +FUNCTION {output} +{ duplicate$ empty$ + 'pop$ + 'output.nonnull + if$ +} + +FUNCTION {output.check} +{ 't := + duplicate$ empty$ + { pop$ "empty " t * " in " * cite$ * warning$ } + 'output.nonnull + if$ +} + +FUNCTION {output.bibitem} +{ newline$ + "\bibitem{" write$ + cite$ write$ + "}" write$ + newline$ + "" + before.all 'output.state := +} + +FUNCTION {fin.entry} +{ add.period$ + write$ + newline$ +} + +FUNCTION {new.block} +{ output.state before.all = + 'skip$ + { after.block 'output.state := } + if$ +} + +FUNCTION {new.sentence} +{ output.state after.block = + 'skip$ + { output.state before.all = + 'skip$ + { after.sentence 'output.state := } + if$ + } + if$ +} + +FUNCTION {not} +{ { #0 } + { #1 } + if$ +} + +FUNCTION {and} +{ 'skip$ + { pop$ #0 } + if$ +} + +FUNCTION {or} +{ { pop$ #1 } + 'skip$ + if$ +} + +FUNCTION {new.block.checka} +{ empty$ + 'skip$ + 'new.block + if$ +} + +FUNCTION {new.block.checkb} +{ empty$ + swap$ empty$ + and + 'skip$ + 'new.block + if$ +} + +FUNCTION {new.sentence.checka} +{ empty$ + 'skip$ + 'new.sentence + if$ +} + +FUNCTION {new.sentence.checkb} +{ empty$ + swap$ empty$ + and + 'skip$ + 'new.sentence + if$ +} + +FUNCTION {field.or.null} +{ duplicate$ empty$ + { pop$ "" } + 'skip$ + if$ +} + +FUNCTION {emphasize} +{ duplicate$ empty$ + { pop$ "" } + { "{\em " swap$ * "}" * } + if$ +} + +INTEGERS { nameptr namesleft numnames } + +FUNCTION {format.names} +{ 's := + #1 'nameptr := + s num.names$ 'numnames := + numnames 'namesleft := + { namesleft #0 > } + { s nameptr "{f.~}{vv~}{ll}{, jj}" format.name$ 't := + nameptr #1 > + { namesleft #1 > + { ", " * t * } + { numnames #2 > + { "," * } + 'skip$ + if$ + t "others" = + { " et~al." * } + { " and " * t * } + if$ + } + if$ + } + 't + if$ + nameptr #1 + 'nameptr := + + namesleft #1 - 'namesleft := + } + while$ +} + +FUNCTION {format.authors} +{ author empty$ + { "" } + { author format.names } + if$ +} + +FUNCTION {format.editors} +{ editor empty$ + { "" } + { editor format.names + editor num.names$ #1 > + { ", editors" * } + { ", editor" * } + if$ + } + if$ +} + +FUNCTION {format.title} +{ title empty$ + { "" } + { title "t" change.case$ } + if$ +} + +FUNCTION {n.dashify} +{ 't := + "" + { t empty$ not } + { t #1 #1 substring$ "-" = + { t #1 #2 substring$ "--" = not + { "--" * + t #2 global.max$ substring$ 't := + } + { { t #1 #1 substring$ "-" = } + { "-" * + t #2 global.max$ substring$ 't := + } + while$ + } + if$ + } + { t #1 #1 substring$ * + t #2 global.max$ substring$ 't := + } + if$ + } + while$ +} + +FUNCTION {format.date} +{ year empty$ + { month empty$ + { "" } + { "there's a month but no year in " cite$ * warning$ + month + } + if$ + } + { month empty$ + 'year + { month " " * year * } + if$ + } + if$ +} + +FUNCTION {format.btitle} +{ title emphasize +} + +FUNCTION {tie.or.space.connect} +{ duplicate$ text.length$ #3 < + { "~" } + { " " } + if$ + swap$ * * +} + +FUNCTION {either.or.check} +{ empty$ + 'pop$ + { "can't use both " swap$ * " fields in " * cite$ * warning$ } + if$ +} + +FUNCTION {format.bvolume} +{ volume empty$ + { "" } + { "volume" volume tie.or.space.connect + series empty$ + 'skip$ + { " of " * series emphasize * } + if$ + "volume and number" number either.or.check + } + if$ +} + +FUNCTION {format.number.series} +{ volume empty$ + { number empty$ + { series field.or.null } + { output.state mid.sentence = + { "number" } + { "Number" } + if$ + number tie.or.space.connect + series empty$ + { "there's a number but no series in " cite$ * warning$ } + { " in " * series * } + if$ + } + if$ + } + { "" } + if$ +} + +FUNCTION {format.edition} +{ edition empty$ + { "" } + { output.state mid.sentence = + { edition "l" change.case$ " edition" * } + { edition "t" change.case$ " edition" * } + if$ + } + if$ +} + +INTEGERS { multiresult } + +FUNCTION {multi.page.check} +{ 't := + #0 'multiresult := + { multiresult not + t empty$ not + and + } + { t #1 #1 substring$ + duplicate$ "-" = + swap$ duplicate$ "," = + swap$ "+" = + or or + { #1 'multiresult := } + { t #2 global.max$ substring$ 't := } + if$ + } + while$ + multiresult +} + +FUNCTION {format.pages} +{ pages empty$ + { "" } + { pages multi.page.check + { "pages" pages n.dashify tie.or.space.connect } + { "page" pages tie.or.space.connect } + if$ + } + if$ +} + +FUNCTION {format.vol.num.pages} +{ volume field.or.null + number empty$ + 'skip$ + { "(" number * ")" * * + volume empty$ + { "there's a number but no volume in " cite$ * warning$ } + 'skip$ + if$ + } + if$ + pages empty$ + 'skip$ + { duplicate$ empty$ + { pop$ format.pages } + { ":" * pages n.dashify * } + if$ + } + if$ +} + +FUNCTION {format.chapter.pages} +{ chapter empty$ + 'format.pages + { type empty$ + { "chapter" } + { type "l" change.case$ } + if$ + chapter tie.or.space.connect + pages empty$ + 'skip$ + { ", " * format.pages * } + if$ + } + if$ +} + +FUNCTION {format.in.ed.booktitle} +{ booktitle empty$ + { "" } + { editor empty$ + { "In " booktitle emphasize * } + { "In " format.editors * ", " * booktitle emphasize * } + if$ + } + if$ +} + +FUNCTION {empty.misc.check} + +{ author empty$ title empty$ howpublished empty$ + month empty$ year empty$ note empty$ + and and and and and + key empty$ not and + { "all relevant fields are empty in " cite$ * warning$ } + 'skip$ + if$ +} + +FUNCTION {format.thesis.type} +{ type empty$ + 'skip$ + { pop$ + type "t" change.case$ + } + if$ +} + +FUNCTION {format.tr.number} +{ type empty$ + { "Technical Report" } + 'type + if$ + number empty$ + { "t" change.case$ } + { number tie.or.space.connect } + if$ +} + +FUNCTION {format.article.crossref} +{ key empty$ + { journal empty$ + { "need key or journal for " cite$ * " to crossref " * crossref * + warning$ + "" + } + { "In {\em " journal * "\/}" * } + if$ + } + { "In " key * } + if$ + " \cite{" * crossref * "}" * +} + +FUNCTION {format.crossref.editor} +{ editor #1 "{vv~}{ll}" format.name$ + editor num.names$ duplicate$ + #2 > + { pop$ " et~al." * } + { #2 < + 'skip$ + { editor #2 "{ff }{vv }{ll}{ jj}" format.name$ "others" = + { " et~al." * } + { " and " * editor #2 "{vv~}{ll}" format.name$ * } + if$ + } + if$ + } + if$ +} + +FUNCTION {format.book.crossref} +{ volume empty$ + { "empty volume in " cite$ * "'s crossref of " * crossref * warning$ + "In " + } + { "Volume" volume tie.or.space.connect + " of " * + } + if$ + editor empty$ + editor field.or.null author field.or.null = + or + { key empty$ + { series empty$ + { "need editor, key, or series for " cite$ * " to crossref " * + crossref * warning$ + "" * + } + { "{\em " * series * "\/}" * } + if$ + } + { key * } + if$ + } + { format.crossref.editor * } + if$ + " \cite{" * crossref * "}" * +} + +FUNCTION {format.incoll.inproc.crossref} +{ editor empty$ + editor field.or.null author field.or.null = + or + { key empty$ + { booktitle empty$ + { "need editor, key, or booktitle for " cite$ * " to crossref " * + crossref * warning$ + "" + } + { "In {\em " booktitle * "\/}" * } + if$ + } + { "In " key * } + if$ + } + { "In " format.crossref.editor * } + if$ + " \cite{" * crossref * "}" * +} + +FUNCTION {article} +{ output.bibitem + format.authors "author" output.check + new.block + format.title "title" output.check + new.block + crossref missing$ + { journal emphasize "journal" output.check + format.vol.num.pages output + format.date "year" output.check + } + { format.article.crossref output.nonnull + format.pages output + } + if$ + new.block + note output + fin.entry +} + +FUNCTION {book} +{ output.bibitem + author empty$ + { format.editors "author and editor" output.check } + { format.authors output.nonnull + crossref missing$ + { "author and editor" editor either.or.check } + 'skip$ + if$ + } + if$ + new.block + format.btitle "title" output.check + crossref missing$ + { format.bvolume output + new.block + format.number.series output + new.sentence + publisher "publisher" output.check + address output + } + { new.block + format.book.crossref output.nonnull + } + if$ + format.edition output + format.date "year" output.check + new.block + note output + fin.entry +} + +FUNCTION {booklet} +{ output.bibitem + format.authors output + new.block + format.title "title" output.check + howpublished address new.block.checkb + howpublished output + address output + format.date output + new.block + note output + fin.entry +} + +FUNCTION {inbook} +{ output.bibitem + author empty$ + { format.editors "author and editor" output.check } + { format.authors output.nonnull + + crossref missing$ + { "author and editor" editor either.or.check } + 'skip$ + if$ + } + if$ + new.block + format.btitle "title" output.check + crossref missing$ + { format.bvolume output + format.chapter.pages "chapter and pages" output.check + new.block + format.number.series output + new.sentence + publisher "publisher" output.check + address output + } + { format.chapter.pages "chapter and pages" output.check + new.block + format.book.crossref output.nonnull + } + if$ + format.edition output + format.date "year" output.check + new.block + note output + fin.entry +} + +FUNCTION {incollection} +{ output.bibitem + format.authors "author" output.check + new.block + format.title "title" output.check + new.block + crossref missing$ + { format.in.ed.booktitle "booktitle" output.check + format.bvolume output + format.number.series output + format.chapter.pages output + new.sentence + publisher "publisher" output.check + address output + format.edition output + format.date "year" output.check + } + { format.incoll.inproc.crossref output.nonnull + format.chapter.pages output + } + if$ + new.block + note output + fin.entry +} + +FUNCTION {inproceedings} +{ output.bibitem + format.authors "author" output.check + new.block + format.title "title" output.check + new.block + crossref missing$ + { format.in.ed.booktitle "booktitle" output.check + format.bvolume output + format.number.series output + format.pages output + address empty$ + { organization publisher new.sentence.checkb + organization output + publisher output + format.date "year" output.check + } + { address output.nonnull + format.date "year" output.check + new.sentence + organization output + publisher output + } + if$ + } + { format.incoll.inproc.crossref output.nonnull + format.pages output + } + if$ + new.block + note output + fin.entry +} + +FUNCTION {conference} { inproceedings } + +FUNCTION {manual} +{ output.bibitem + author empty$ + { organization empty$ + 'skip$ + { organization output.nonnull + address output + } + if$ + } + { format.authors output.nonnull } + if$ + new.block + format.btitle "title" output.check + author empty$ + { organization empty$ + { address new.block.checka + address output + } + 'skip$ + if$ + } + { organization address new.block.checkb + organization output + address output + } + if$ + format.edition output + format.date output + new.block + note output + fin.entry +} + +FUNCTION {mastersthesis} +{ output.bibitem + format.authors "author" output.check + new.block + format.title "title" output.check + new.block + "Master's thesis" format.thesis.type output.nonnull + school "school" output.check + address output + format.date "year" output.check + new.block + note output + fin.entry +} + +FUNCTION {misc} +{ output.bibitem + format.authors output + title howpublished new.block.checkb + format.title output + howpublished new.block.checka + howpublished output + format.date output + new.block + note output + fin.entry + empty.misc.check +} + +FUNCTION {phdthesis} +{ output.bibitem + format.authors "author" output.check + new.block + format.btitle "title" output.check + new.block + "PhD thesis" format.thesis.type output.nonnull + school "school" output.check + address output + format.date "year" output.check + new.block + note output + fin.entry +} + +FUNCTION {proceedings} +{ output.bibitem + editor empty$ + { organization output } + { format.editors output.nonnull } + + if$ + new.block + format.btitle "title" output.check + format.bvolume output + format.number.series output + address empty$ + { editor empty$ + { publisher new.sentence.checka } + { organization publisher new.sentence.checkb + organization output + } + if$ + publisher output + format.date "year" output.check + } + { address output.nonnull + format.date "year" output.check + new.sentence + editor empty$ + 'skip$ + { organization output } + if$ + publisher output + } + if$ + new.block + note output + fin.entry +} + +FUNCTION {techreport} +{ output.bibitem + format.authors "author" output.check + new.block + format.title "title" output.check + new.block + format.tr.number output.nonnull + institution "institution" output.check + address output + format.date "year" output.check + new.block + note output + fin.entry +} + +FUNCTION {unpublished} +{ output.bibitem + format.authors "author" output.check + new.block + format.title "title" output.check + new.block + note "note" output.check + format.date output + fin.entry +} + +FUNCTION {default.type} { misc } + +MACRO {jan} {"Jan."} + +MACRO {feb} {"Feb."} + +MACRO {mar} {"Mar."} + +MACRO {apr} {"Apr."} + +MACRO {may} {"May"} + +MACRO {jun} {"June"} + +MACRO {jul} {"July"} + +MACRO {aug} {"Aug."} + +MACRO {sep} {"Sept."} + +MACRO {oct} {"Oct."} + +MACRO {nov} {"Nov."} + +MACRO {dec} {"Dec."} + +MACRO {acmcs} {"ACM Comput. Surv."} + +MACRO {acta} {"Acta Inf."} + +MACRO {cacm} {"Commun. ACM"} + +MACRO {ibmjrd} {"IBM J. Res. Dev."} + +MACRO {ibmsj} {"IBM Syst.~J."} + +MACRO {ieeese} {"IEEE Trans. Softw. Eng."} + +MACRO {ieeetc} {"IEEE Trans. Comput."} + +MACRO {ieeetcad} + {"IEEE Trans. Comput.-Aided Design Integrated Circuits"} + +MACRO {ipl} {"Inf. Process. Lett."} + +MACRO {jacm} {"J.~ACM"} + +MACRO {jcss} {"J.~Comput. Syst. Sci."} + +MACRO {scp} {"Sci. Comput. Programming"} + +MACRO {sicomp} {"SIAM J. Comput."} + +MACRO {tocs} {"ACM Trans. Comput. Syst."} + +MACRO {tods} {"ACM Trans. Database Syst."} + +MACRO {tog} {"ACM Trans. Gr."} + +MACRO {toms} {"ACM Trans. Math. Softw."} + +MACRO {toois} {"ACM Trans. Office Inf. Syst."} + +MACRO {toplas} {"ACM Trans. Prog. Lang. Syst."} + +MACRO {tcs} {"Theoretical Comput. Sci."} + +READ + +FUNCTION {sortify} +{ purify$ + "l" change.case$ +} + +INTEGERS { len } + +FUNCTION {chop.word} +{ 's := + 'len := + s #1 len substring$ = + { s len #1 + global.max$ substring$ } + 's + if$ +} + +FUNCTION {sort.format.names} +{ 's := + #1 'nameptr := + "" + s num.names$ 'numnames := + numnames 'namesleft := + { namesleft #0 > } + { nameptr #1 > + { " " * } + 'skip$ + if$ + s nameptr "{vv{ } }{ll{ }}{ f{ }}{ jj{ }}" format.name$ 't := + nameptr numnames = t "others" = and + { "et al" * } + { t sortify * } + if$ + nameptr #1 + 'nameptr := + namesleft #1 - 'namesleft := + } + while$ +} + +FUNCTION {sort.format.title} +{ 't := + "A " #2 + "An " #3 + "The " #4 t chop.word + chop.word + chop.word + sortify + #1 global.max$ substring$ +} + +FUNCTION {author.sort} +{ author empty$ + { key empty$ + { "to sort, need author or key in " cite$ * warning$ + "" + } + { key sortify } + if$ + } + { author sort.format.names } + if$ +} + +FUNCTION {author.editor.sort} +{ author empty$ + { editor empty$ + { key empty$ + { "to sort, need author, editor, or key in " cite$ * warning$ + "" + } + { key sortify } + if$ + } + { editor sort.format.names } + if$ + } + { author sort.format.names } + if$ +} + +FUNCTION {author.organization.sort} +{ author empty$ + + { organization empty$ + { key empty$ + { "to sort, need author, organization, or key in " cite$ * warning$ + "" + } + { key sortify } + if$ + } + { "The " #4 organization chop.word sortify } + if$ + } + { author sort.format.names } + if$ +} + +FUNCTION {editor.organization.sort} +{ editor empty$ + { organization empty$ + { key empty$ + { "to sort, need editor, organization, or key in " cite$ * warning$ + "" + } + { key sortify } + if$ + } + { "The " #4 organization chop.word sortify } + if$ + } + { editor sort.format.names } + if$ +} + +FUNCTION {presort} +{ type$ "book" = + type$ "inbook" = + or + 'author.editor.sort + { type$ "proceedings" = + 'editor.organization.sort + { type$ "manual" = + 'author.organization.sort + 'author.sort + if$ + } + if$ + } + if$ + " " + * + year field.or.null sortify + * + " " + * + title field.or.null + sort.format.title + * + #1 entry.max$ substring$ + 'sort.key$ := +} + +ITERATE {presort} + +SORT + +STRINGS { longest.label } + +INTEGERS { number.label longest.label.width } + +FUNCTION {initialize.longest.label} +{ "" 'longest.label := + #1 'number.label := + #0 'longest.label.width := +} + +FUNCTION {longest.label.pass} +{ number.label int.to.str$ 'label := + number.label #1 + 'number.label := + label width$ longest.label.width > + { label 'longest.label := + label width$ 'longest.label.width := + } + 'skip$ + if$ +} + +EXECUTE {initialize.longest.label} + +ITERATE {longest.label.pass} + +FUNCTION {begin.bib} +{ preamble$ empty$ + 'skip$ + { preamble$ write$ newline$ } + if$ + "\begin{thebibliography}{" longest.label * + "}\setlength{\itemsep}{-1ex}\small" * write$ newline$ +} + +EXECUTE {begin.bib} + +EXECUTE {init.state.consts} + +ITERATE {call.type$} + +FUNCTION {end.bib} +{ newline$ + "\end{thebibliography}" write$ newline$ +} + +EXECUTE {end.bib} + +% end of file latex8.bst +% --------------------------------------------------------------- + + + diff --git a/cometbft/v0.39/spec/consensus/consensus-paper/latex8.sty b/cometbft/v0.39/spec/consensus/consensus-paper/latex8.sty new file mode 100644 index 000000000..1e6b0dc7e --- /dev/null +++ b/cometbft/v0.39/spec/consensus/consensus-paper/latex8.sty @@ -0,0 +1,168 @@ +% --------------------------------------------------------------- +% +% $Id: latex8.sty,v 1.2 1995/09/15 15:31:13 ienne Exp $ +% +% by Paolo.Ienne@di.epfl.ch +% +% --------------------------------------------------------------- +% +% no guarantee is given that the format corresponds perfectly to +% IEEE 8.5" x 11" Proceedings, but most features should be ok. +% +% --------------------------------------------------------------- +% with LaTeX2e: +% ============= +% +% use as +% \documentclass[times,10pt,twocolumn]{article} +% \usepackage{latex8} +% \usepackage{times} +% +% --------------------------------------------------------------- + +% with LaTeX 2.09: +% ================ +% +% use as +% \documentstyle[times,art10,twocolumn,latex8]{article} +% +% --------------------------------------------------------------- +% with both versions: +% =================== +% +% specify \pagestyle{empty} to omit page numbers in the final +% version +% +% specify references as +% \bibliographystyle{latex8} +% \bibliography{...your files...} +% +% use Section{} and SubSection{} instead of standard section{} +% and subsection{} to obtain headings in the form +% "1.3. My heading" +% +% --------------------------------------------------------------- + +\typeout{IEEE 8.5 x 11-Inch Proceedings Style `latex8.sty'.} + +% ten point helvetica bold required for captions +% in some sites the name of the helvetica bold font may differ, +% change the name here: +\font\tenhv = phvb at 10pt +%\font\tenhv = phvb7t at 10pt + +% eleven point times bold required for second-order headings +% \font\elvbf = cmbx10 scaled 1100 +\font\elvbf = ptmb scaled 1100 + +% set dimensions of columns, gap between columns, and paragraph indent +\setlength{\textheight}{8.875in} +\setlength{\textwidth}{6.875in} +\setlength{\columnsep}{0.3125in} +\setlength{\topmargin}{0in} +\setlength{\headheight}{0in} +\setlength{\headsep}{0in} +\setlength{\parindent}{1pc} +\setlength{\oddsidemargin}{-.304in} +\setlength{\evensidemargin}{-.304in} + +% memento from size10.clo +% \normalsize{\@setfontsize\normalsize\@xpt\@xiipt} +% \small{\@setfontsize\small\@ixpt{11}} +% \footnotesize{\@setfontsize\footnotesize\@viiipt{9.5}} +% \scriptsize{\@setfontsize\scriptsize\@viipt\@viiipt} +% \tiny{\@setfontsize\tiny\@vpt\@vipt} +% \large{\@setfontsize\large\@xiipt{14}} +% \Large{\@setfontsize\Large\@xivpt{18}} +% \LARGE{\@setfontsize\LARGE\@xviipt{22}} +% \huge{\@setfontsize\huge\@xxpt{25}} +% \Huge{\@setfontsize\Huge\@xxvpt{30}} + +\def\@maketitle + { + \newpage + \null + \vskip .375in + \begin{center} + {\Large \bf \@title \par} + % additional two empty lines at the end of the title + \vspace*{24pt} + { + \large + \lineskip .5em + \begin{tabular}[t]{c} + \@author + \end{tabular} + \par + } + % additional small space at the end of the author name + \vskip .5em + { + \large + \begin{tabular}[t]{c} + \@affiliation + \end{tabular} + \par + \ifx \@empty \@email + \else + \begin{tabular}{r@{~}l} + E-mail: & {\tt \@email} + \end{tabular} + \par + \fi + } + % additional empty line at the end of the title block + \vspace*{12pt} + \end{center} + } + +\def\abstract + {% + \centerline{\large\bf Abstract}% + \vspace*{12pt}% + \it% + } + +\def\endabstract + { + % additional empty line at the end of the abstract + \vspace*{12pt} + } + +\def\affiliation#1{\gdef\@affiliation{#1}} \gdef\@affiliation{} + +\def\email#1{\gdef\@email{#1}} +\gdef\@email{} + +\newlength{\@ctmp} +\newlength{\@figindent} +\setlength{\@figindent}{1pc} + +\long\def\@makecaption#1#2{ + \vskip 10pt + \setbox\@tempboxa\hbox{\tenhv\noindent #1.~#2} + \setlength{\@ctmp}{\hsize} + \addtolength{\@ctmp}{-\@figindent}\addtolength{\@ctmp}{-\@figindent} + % IF longer than one indented paragraph line + \ifdim \wd\@tempboxa >\@ctmp + % THEN set as an indented paragraph + \begin{list}{}{\leftmargin\@figindent \rightmargin\leftmargin} + \item[]\tenhv #1.~#2\par + \end{list} + \else + % ELSE center + \hbox to\hsize{\hfil\box\@tempboxa\hfil} + \fi} + +% correct heading spacing and type +\def\section{\@startsection {section}{1}{\z@} + {14pt plus 2pt minus 2pt}{14pt plus 2pt minus 2pt} {\large\bf}} +\def\subsection{\@startsection {subsection}{2}{\z@} + {13pt plus 2pt minus 2pt}{13pt plus 2pt minus 2pt} {\elvbf}} + +% add the period after section numbers +\newcommand{\Section}[1]{\section{\hskip -1em.~#1}} +\newcommand{\SubSection}[1]{\subsection{\hskip -1em.~#1}} + +% end of file latex8.sty +% --------------------------------------------------------------- diff --git a/cometbft/v0.39/spec/consensus/consensus-paper/lit.bib b/cometbft/v0.39/spec/consensus/consensus-paper/lit.bib new file mode 100644 index 000000000..4abc83e70 --- /dev/null +++ b/cometbft/v0.39/spec/consensus/consensus-paper/lit.bib @@ -0,0 +1,1659 @@ +%--- conferences -------------------------------------------------- +@STRING{WDAG96 = "Proceedings of the 10th International Workshop + on Distributed Algorithms (WDAG'96)"} +@STRING{WDAG97 = "Proceedings of the 11th International Workshop + on Distributed Algorithms (WDAG'97)"} +@STRING{DISC98 = "Proceedings of the 12th International Conference + on Distributed Computing ({DISC}'98)"} +@STRING{DISC99 = "Proceedings of the 13th International Conference + on Distributed Computing ({DISC}'99)"} +@STRING{DISC98 = "Proceedings of the 13th International Conference + on Distributed Computing ({DISC}'98)"} +@STRING{DISC99 = "Proceedings of the 13th International Conference + on Distributed Computing ({DISC}'99)"} +@STRING{DISC00 = "Proceedings of the 14th International Conference + on Distributed Computing ({DISC}'00)"} +@STRING{DISC01 = "Proceedings of the 15th International Conference + on Distributed Computing ({DISC}'01)"} +@STRING{DISC02 = "Proceedings of the 16th International Conference + on Distributed Computing ({DISC}'02)"} +@STRING{DISC03 = "Proceedings of the 17th International Conference + on Distributed Computing ({DISC}'03)"} +@STRING{DISC04 = "Proceedings of the 18th International Conference + on Distributed Computing ({DISC}'04)"} +@STRING{DISC05 = "Proceedings of the 19th International Conference + on Distributed Computing ({DISC}'05)"} +@STRING{PODC83 = "Proceeding of the 1st Annual {ACM} Symposium on + Principles of Distributed Computing ({PODC}'83)"} +@STRING{PODC91 = "Proceeding of the 9th Annual {ACM} Symposium on + Principles of Distributed Computing ({PODC}'91)"} +@STRING{PODC94 = "Proceeding of the 12th Annual {ACM} Symposium on + Principles of Distributed Computing ({PODC}'94)"} +@STRING{PODC95 = "Proceeding of the 13th Annual {ACM} Symposium on + Principles of Distributed Computing ({PODC}'95)"} +@STRING{PODC96 = "Proceeding of the 14th Annual {ACM} Symposium on + Principles of Distributed Computing ({PODC}'96)"} +@STRING{PODC97 = "Proceeding of the 15th Annual {ACM} Symposium on + Principles of Distributed Computing ({PODC}'97)"} +@STRING{PODC98 = "Proceeding of the 16th Annual {ACM} Symposium on + Principles of Distributed Computing ({PODC}'98)"} +@STRING{PODC99 = "Proceeding of the 17th Annual {ACM} Symposium on + Principles of Distributed Computing ({PODC}'99)"} +@STRING{PODC00 = "Proceeding of the 18th Annual {ACM} Symposium on + Principles of Distributed Computing ({PODC}'00)"} +@STRING{PODC01 = "Proceeding of the 19th Annual {ACM} Symposium on + Principles of Distributed Computing ({PODC}'01)"} +@STRING{PODC02 = "Proceeding of the 20th Annual {ACM} Symposium on + Principles of Distributed Computing ({PODC}'02)"} +@STRING{PODC03 = "Proceeding of the 21st Annual {ACM} Symposium on + Principles of Distributed Computing ({PODC}'03)"} +@STRING{PODC03 = "Proceeding of the 22nd Annual {ACM} Symposium on + Principles of Distributed Computing ({PODC}'03)"} +@STRING{PODC04 = "Proceeding of the 23rd Annual {ACM} Symposium on + Principles of Distributed Computing ({PODC}'04)"} +@STRING{PODC05 = "Proceeding of the 24th Annual {ACM} Symposium on + Principles of Distributed Computing ({PODC}'05)"} +@STRING{PODC06 = "Proceedings of the 25th Annual {ACM} Symposium on + Principles of Distributed Computing ({PODC}'06)"} +@STRING{PODC07 = "Proceedings of the 26th Annual {ACM} Symposium on + Principles of Distributed Computing ({PODC}'07)"} +@STRING{STOC91 = "Proceedings of the 23rd Annual {ACM} Symposium on + Theory of Computing ({STOC}'91)"} +@STRING{WSS01 = "Proceedings of the 5th International Workshop on + Self-Stabilizing Systems ({WSS} '01)"} +@STRING{SSS06 = "Proceedings of the 8th International Symposium on + Stabilization, Safety, and Security of Distributed + Systems ({SSS} '06)"} +@STRING{DSN00 = "Dependable Systems and Networks ({DSN} 2000)"} +@STRING{DSN05 = "Dependable Systems and Networks ({DSN} 2005)"} +@STRING{DSN06 = "Dependable Systems and Networks ({DSN} 2006)"} +@STRING{DSN07 = "Dependable Systems and Networks ({DSN} 2007)"} + +%--- journals ----------------------------------------------------- +@STRING{PPL = "Parallel Processing Letters"} +@STRING{IPL = "Information Processing Letters"} +@STRING{DC = "Distributed Computing"} +@STRING{JACM = "Journal of the ACM"} +@STRING{IC = "Information and Control"} +@STRING{TCS = "Theoretical Computer Science"} +@STRING{ACMTCS = "ACM Transactions on Computer Systems"} +@STRING{TDSC = "Transactions on Dependable and Secure Computing"} +@STRING{TPLS = "ACM Trans. Program. Lang. Syst."} + +%--- publisher ---------------------------------------------------- +@STRING{ACM = "ACM Press"} +@STRING{IEEE = "IEEE"} +@STRING{SPR = "Springer-Verlag"} + +%--- institution -------------------------------------------------- +@STRING{TUAuto = {Technische Universit\"at Wien, Department of + Automation}} +@STRING{TUECS = {Technische Universit\"at Wien, Embedded Computing + Systems Group}} + + +%------------------------------------------------------------------ +@article{ABND+90:jacm, + author = {Hagit Attiya and Amotz Bar-Noy and Danny Dolev and + David Peleg and R{\"u}diger Reischuk}, + title = {Renaming in an asynchronous environment}, + journal = JACM, + volume = {37}, + number = {3}, + year = {1990}, + pages = {524--548}, + publisher = ACM, + address = {New York, NY, USA}, +} + +@article{ABND95:jacm, + author = {Hagit Attiya and Amotz Bar-Noy and Danny Dolev}, + title = {Sharing memory robustly in message-passing systems}, + journal = JACM, + volume = {42}, + number = {1}, + year = {1995}, + pages = {124--142}, + publisher = ACM, + address = {New York, NY, USA}, +} + +@inproceedings{ACKM04:podc, + author = {Ittai Abraham and Gregory Chockler and Idit Keidar + and Dahlia Malkhi}, + title = {Byzantine disk paxos: optimal resilience with + byzantine shared memory.}, + booktitle = PODC04, + year = {2004}, + pages = {226-235} +} + +@article{ACKM05:dc, + author = {Ittai Abraham and Gregory Chockler and Idit Keidar + and Dahlia Malkhi}, + title = {Byzantine disk paxos: optimal resilience with + byzantine shared memory.}, + journal = DC, + volume = {18}, + number = {5}, + year = {2006}, + pages = {387-408} +} + +@article{ACT00:dc, + author = "Marcos Kawazoe Aguilera and Wei Chen and Sam Toueg", + title = "Failure Detection and Consensus in the + Crash-Recovery Model", + journal = DC, + year = 2000, + month = apr, + volume = 13, + number = 2, + pages = "99--125", + url = + "http://www.cs.cornell.edu/home/sam/FDpapers/crash-recovery-finaldcversion.ps" +} + +@article{ACT00:siam, + author = "Marcos Kawazoe Aguilera and Wei Chen and Sam Toueg", + title = "On quiescent reliable communication", + journal = "SIAM Journal of Computing", + year = 2000, + volume = 29, + number = 6, + pages = "2040--2073", + month = apr +} + +@inproceedings{ACT97:wdag, + author = "Marcos Kawazoe Aguilera and Wei Chen and Sam Toueg", + title = "Heartbeat: A Timeout-Free Failure Detector for + Quiescent Reliable Communication", + booktitle = WDAG97, + year = 1997, + pages = "126--140", + url = + "http://simon.cs.cornell.edu/Info/People/weichen/research/mypapers/wdag97final.ps" +} + +@article{ACT98:disc, + author = "Marcos Kawazoe Aguilera and Wei Chen and Sam Toueg", + title = "Failure Detection and Consensus in the + Crash-Recovery Model", + journal = DISC98, + year = 1998, + pages = "231--245", + publisher = SPR +} + +@article{ACT99:tcs, + author = "Marcos Kawazoe Aguilera and Wei Chen and Sam Toueg", + title = "Using the Heartbeat Failure Detector for Quiescent + Reliable Communication and Consensus in + Partitionable Networks", + journal = "Theoretical Computer Science", + year = 1999, + month = jun, + volume = 220, + number = 1, + pages = "3--30", + url = + "http://www.cs.cornell.edu/home/sam/FDpapers/TCS98final.ps" +} + +@inproceedings{ADGF+04:ispdc, + author = {Anceaume, Emmanuelle and Delporte-Gallet, Carole and + Fauconnier, Hugues and Hurfin, Michel and Le Lann, + G{\'e}rard }, + title = {Designing Modular Services in the Scattered + Byzantine Failure Model.}, + booktitle = {ISPDC/HeteroPar}, + year = {2004}, + pages = {262-269} +} + +@inproceedings{ADGF+06:dsn, + author = {Marcos Kawazoe Aguilera and Carole Delporte-Gallet + and Hugues Fauconnier and Sam Toueg}, + title = {Consensus with Byzantine Failures and Little System + Synchrony.}, + booktitle = DSN06, + year = {2006}, + pages = {147-155} +} + +@inproceedings{ADGFT01:disc, + author = "Marcos Kawazoe Aguilera and Carole Delporte-Gallet + and Hugues Fauconnier and Sam Toueg", + title = "Stable Leader Election", + booktitle = DISC01, + year = 2001, + pages = "108--122", + publisher = SPR +} + +@inproceedings{ADGFT03:podc, + author = "Marcos K. Aguilera and Carole Delporte-Gallet and + Hugues Fauconnier and Sam Toueg", + title = "On implementing {O}mega with weak reliability and + synchrony assumptions", + booktitle = PODC03, + year = 2003, + publisher = ACM +} + +@inproceedings{ADGFT04:podc, + author = {Marcos K. Aguilera and Carole Delporte-Gallet and + Hugues Fauconnier and Sam Toueg}, + title = {Communication-efficient leader election and + consensus with limited link synchrony}, + booktitle = PODC04, + year = 2004, + pages = {328--337}, + address = {St. John's, Newfoundland, Canada}, + publisher = ACM +} + +@inproceedings{ADGFT06:dsn, + author = {Marcos Kawazoe Aguilera and Carole Delporte-Gallet + and Hugues Fauconnier and Sam Toueg}, + title = {Consensus with Byzantine Failures and Little System + Synchrony.}, + booktitle = DSN06, + year = 2006, + pages = {147-155}, + ee = + {http://doi.ieeecomputersociety.org/10.1109/DSN.2006.22}, + bibsource = {DBLP, http://dblp.uni-trier.de} +} + +@inproceedings{ADLS91:stoc, + author = "Hagit Attiya and Cynthia Dwork and Nancy A. Lynch + and Larry J. Stockmeyer", + title = "Bounds on the Time to Reach Agreement in the + Presence of Timing Uncertainty", + booktitle = STOC91, + year = 1991, + pages = "359--369", +} + +@article{AT99:ipl, + author = "Marcos Kawazoe Aguilera and Sam Toueg", + title = "A Simple Bivalency Proof that t -Resilient Consensus + Requires t + 1 Rounds", + journal = IPL, + volume = "71", + number = "3-4", + pages = "155--158", + year = "1999" +} + +@Book{AW04:book, + author = {Attiya, Hagit and Welch, Jennifer}, + title = {Distributed Computing}, + publisher = {John Wiley {\&} Sons}, + edition = {2nd}, + year = {2004} +} + +@Book{AW98:book, + author = {Hagit Attiya and Jennifer Welch}, + title = {Distributed Computing}, + publisher = {McGraw-Hill Publishing Company}, + year = {1998} +} + +@InBook{AW98:book:chap12, + author = {Hagit Attiya and Jennifer Welch}, + title = {Distributed Computing}, + publisher = {McGraw-Hill Publishing Company}, + year = {1998}, + chapter = {12, "Improving the fault-tolerance of algorithms"} +} + +@inproceedings{ABHMS11:disc, + author = {Hagit Attiya and + Fatemeh Borran and + Martin Hutle and + Zarko Milosevic and + Andr{\'e} Schiper}, + title = {Structured Derivation of Semi-Synchronous Algorithms}, + booktitle = {DISC}, + year = {2011}, + pages = {374-388} +} + +@inproceedings{BCBG+07:podc, + author = {Martin Biely and Bernadette Charron-Bost and Antoine + Gaillard and Martin Hutle and Andr{\'e} Schiper and + Josef Widder}, + title = {Tolerating Corrupted Communication}, + publisher = ACM, + booktitle = PODC07, + year = {2007} +} + +@InProceedings{BCBT96:wdag, + author = {Anindya Basu and Bernadette Charron-Bost and Sam + Toueg}, + title = {Simulating Reliable Links with Unreliable Links in + the Presence of Process Crashes}, + pages = {105--122}, + booktitle = {WDAG 1996}, + editor = {Babao{\u g}lu, {\"O}zalp}, + year = {1996}, + month = {Oct}, + volume = {1151}, + ISBN = {3-540-61769-8}, + pubisher = {Springer}, + series = {Lecture Notes in Computer Science}, +} + +@article{BDFG03:sigact, + author = "R. Boichat and P. Dutta and S. Frolund and + R. Guerraoui", + title = "Reconstructing {P}axos", + journal = "ACM SIGACT News", + year = "2003", + volume = "34", + number = "1", + pages = "47-67" +} + +@unpublished{BHR+06:note, + author = "Martin Biely and Martin Hutle and Sergio Rajsbaum + and Ulrich Schmid and Corentin Travers and Josef + Widder", + title = "Discussion note on moving timely links", + note = "Unpublished", + month = apr, + year = 2006 +} + +@article{BHRT03:jda, + author = {Roberto Baldoni and Jean-Michel H{\'e}lary and + Michel Raynal and L{\'e}naick Tanguy}, + title = {Consensus in Byzantine asynchronous systems.}, + journal = {J. Discrete Algorithms}, + volume = {1}, + number = {2}, + year = {2003}, + pages = {185-210}, + ee = {http://dx.doi.org/10.1016/S1570-8667(03)00025-X}, + bibsource = {DBLP, http://dblp.uni-trier.de} +} + +@unpublished{BHSS08:tdsc, + author = {Fatemeh Borran and Martin Hutle and Nuno Santos and + Andr{\'e} Schiper}, + title = {Solving Consensus with Communication Predicates: + A~Quantitative Approach}, + note = {Under submission}, + year = {2008} +} + +@inproceedings{Ben83:podc, + author = {Michael Ben-Or}, + title = {Another Advantage of Free Choice: Completely + Asynchronous Agreement Protocols}, + booktitle = {PODC}, + year = {1983}, +} + +@inproceedings{Bra04:podc, + author = {Bracha, Gabriel}, + title = {An asynchronous [(n - 1)/3]-resilient consensus protocol}, + booktitle = {PODC '84: Proceedings of the third annual ACM symposium on Principles of distributed computing}, + year = {1984}, + isbn = {0-89791-143-1}, + pages = {154--162}, + location = {Vancouver, British Columbia, Canada}, + doi = {http://doi.acm.org/10.1145/800222.806743}, + publisher = {ACM}, + address = {New York, NY, USA}, + } + + +@inproceedings{CBGS00:dsn, + author = "Bernadette Charron-Bost and Rachid Guerraoui and + Andr{\'{e}} Schiper", + title = "Synchronous System and Perfect Failure Detector: + {S}olvability and efficiency issues", + booktitle = DSN00, + publisher = "{IEEE} Computer Society", + address = "New York, {USA}", + pages = "523--532", + year = "2000" +} + +@inproceedings{CBS06:prdc, + author = {Bernadette Charron-Bost and Andr{\'e} Schiper}, + title = {Improving Fast Paxos: being optimistic with no + overhead}, + booktitle = {Pacific Rim Dependable Computing, Proceedings}, + year = {2006} +} + +@article{CBS09, + author = {B. Charron-Bost and A. Schiper}, + title = {The {H}eard-{O}f model: computing in distributed systems with benign failures}, + journal ={Distributed Computing}, + number = {1}, + volume = {22}, + pages = {49-71}, + year ={2009} + } + + +@article{CBS07:sigact, + author = {Bernadette Charron-Bost and Andr\'{e} Schiper}, + title = {Harmful dogmas in fault tolerant distributed + computing}, + journal = {SIGACT News}, + volume = {38}, + number = {1}, + year = {2007}, + pages = {53--61}, +} + +@techreport{CBS07:tr, + author = {Charron-Bost, Bernadette and Schiper, Andr{\'{e}}}, + title = {The Heard-Of Model: Unifying all Benign Failures}, + institution = {EPFL}, + year = 2007, + OPTnumber = {LSR-REPORT-2006-004} +} + +@article{CELT00:jacm, + author = {Soma Chaudhuri and Maurice Erlihy and Nancy A. Lynch + and Mark R. Tuttle}, + title = {Tight bounds for k-set agreement}, + journal = JACM, + volume = {47}, + number = {5}, + year = {2000}, + pages = {912--943}, + publisher = ACM, + address = {New York, NY, USA}, +} + +@article{CF99:tpds, + author = "Flaviu Cristian and Christof Fetzer", + title = "The Timed Asynchronous Distributed System Model", + journal = "IEEE Transactions on Parallel and Distributed + Systems", + volume = "10", + number = "6", + pages = "642--657", + year = "1999" +} + +@article{CHT96:jacm, + author = "Tushar Deepak Chandra and Vassos Hadzilacos and Sam + Toueg", + title = "The Weakest Failure Detector for Solving Consensus", + journal = {JACM}, + year = {1996}, +} + +@article{CL02:tcs, + author = {Miguel Castro and Barbara Liskov}, + title = {Practical byzantine fault tolerance and proactive + recovery}, + journal = {ACMTCS}, + year = {2002}, +} + +@inproceedings{CL99:osdi, + author = {Miguel Castro and Barbara Liskov}, + title = {Practical byzantine fault tolerance and proactive + recovery}, + booktitle = {Proceedings of the 3rd Symposium on Operating + Systems Design and Implementation}, + year = {1999}, + month = feb +} + +@inproceedings{CT91:podc, + author = {Tushar Deepak Chandra and Sam Toueg}, + title = {Unreliable Failure Detectors for Asynchronous + Systems (Preliminary Version)}, + booktitle = PODC91, + year = {1991}, + pages = {325-340} +} + +@article{CT96:jacm1, + author = "Tushar Deepak Chandra and Sam Toueg", + title = "Unreliable Failure Detectors for Reliable + Distributed Systems", + journal = {JACM}, + year = {1996}, +} + +@inproceedings{CTA00:dsn, + author = "Wei Chen and Sam Toueg and Marcos Kawazoe Aguilera", + title = "On the Quality of Service of Failure Detectors", + booktitle = "Proceedings IEEE International Conference on + Dependable Systems and Networks (DSN / FTCS'30)", + address = "New York City, USA", + year = 2000 +} + +@TechReport{DFKM96:tr, + author = {Danny Dolev and Roy Friedman and Idit Keidar and + Dahlia Malkhi}, + title = {Failure detectors in omission failure environments}, + institution = {Department of Computer Science, Cornell University}, + year = {1996}, + type = {Technical Report}, + number = {96-1608} +} + +@inproceedings{DG02:podc, + author = {Partha Dutta and Rachid Guerraoui}, + title = {The inherent price of indulgence}, + booktitle = PODC02, + year = 2002, + pages = {88--97}, + location = {Monterey, California}, + publisher = ACM, + address = {New York, NY, USA}, +} + +@inproceedings{DGFG+04:podc, + author = {Carole Delporte-Gallet and Hugues Fauconnier and + Rachid Guerraoui and Vassos Hadzilacos and Petr + Kouznetsov and Sam Toueg}, + title = {The weakest failure detectors to solve certain + fundamental problems in distributed computing}, + booktitle = PODC04, + year = 2004, + pages = {338--346}, + location = {St. John's, Newfoundland, Canada}, + publisher = ACM, + address = {New York, NY, USA} +} + +@inproceedings{DGL05:dsn, + author = {Partha Dutta and Rachid Guerraoui and Leslie + Lamport}, + title = {How Fast Can Eventual Synchrony Lead to Consensus?}, + booktitle = {Proceedings of the 2005 International Conference on + Dependable Systems and Networks (DSN'05)}, + pages = {22--27}, + year = {2005}, + address = {Los Alamitos, CA, USA} +} + +@article{DLS88:jacm, + author = "Cynthia Dwork and Nancy Lynch and Larry Stockmeyer", + title = "Consensus in the Presence of Partial Synchrony", + journal = {JACM}, + year = {1988}, +} + +@article{DPLL00:tcs, + author = "De Prisco, Roberto and Butler Lampson and Nancy + Lynch", + title = "Revisiting the {PAXOS} algorithm", + journal = TCS, + volume = "243", + number = "1--2", + pages = "35--91", + year = "2000" +} + +@techreport{DS97:tr, + author = {A. Doudou and A. Schiper}, + title = {Muteness Failure Detectors for Consensus with + {B}yzantine Processes}, + institution = {EPFL, Dept d'Informatique}, + year = {1997}, + type = {TR}, + month = {October}, + number = {97/230}, +} + +@inproceedings{DS98:podc, + author = {A. Doudou and A. Schiper}, + title = {Muteness Detectors for Consensus with {B}yzantine + Processes ({B}rief {A}nnouncement)}, + booktitle = {PODC}, + month = jul, + year = {1998} +} + +@article{DSU04:survey, + author = {D{\'e}fago, Xavier and Schiper, Andr{\'e} and Urb\'{a}n, P{\'e}ter}, + title = {Total order broadcast and multicast algorithms: Taxonomy and survey}, + journal = {ACM Comput. Surv.}, + issue_date = {December 2004}, + volume = {36}, + number = {4}, + month = dec, + year = {2004}, + issn = {0360-0300}, + pages = {372--421}, + numpages = {50}, + publisher = {ACM}, + address = {New York, NY, USA}, + keywords = {Distributed systems, agreement problems, atomic broadcast, atomic multicast, classification, distributed algorithms, fault-tolerance, global ordering, group communication, message passing, survey, taxonomy, total ordering}, +} + +@article{DeCandia07:dynamo, + author = {DeCandia, Giuseppe and Hastorun, Deniz and Jampani, Madan and Kakulapati, Gunavardhan and Lakshman, Avinash and Pilchin, Alex and Sivasubramanian, Swaminathan and Vosshall, Peter and Vogels, Werner}, + title = {Dynamo: amazon's highly available key-value store}, + journal = {SIGOPS Oper. Syst. Rev.}, + issue_date = {December 2007}, + volume = {41}, + number = {6}, + month = oct, + year = {2007}, + issn = {0163-5980}, + pages = {205--220}, + numpages = {16}, + publisher = {ACM}, + address = {New York, NY, USA}, + keywords = {performance, reliability, scalability}, +} + + +@book{Dol00:book, + author = {Shlomi Dolev}, + title = {Self-Stabilization}, + publisher = {The MIT Press}, + year = {2000} +} + +@inproceedings{FC95:podc, + author = "Christof Fetzer and Flaviu Cristian", + title = "Lower Bounds for Convergence Function Based Clock + Synchronization", + booktitle = PODC95, + year = 1995, + pages = "137--143" +} + +@article{FLP85:jacm, + author = "Michael J. Fischer and Nancy A. Lynch and + M. S. Paterson", + title = "Impossibility of Distributed Consensus with one + Faulty Process", + journal = {JACM}, + year = {1985}, +} + +@article{FMR05:tdsc, + author = {Roy Friedman and Achour Most{\'e}faoui and Michel + Raynal}, + title = {Simple and Efficient Oracle-Based Consensus + Protocols for Asynchronous Byzantine Systems.}, + journal = TDSC, + volume = {2}, + number = {1}, + year = {2005}, + pages = {46-56}, + ee = {http://dx.doi.org/10.1109/TDSC.2005.13}, + bibsource = {DBLP, http://dblp.uni-trier.de} +} + +@inproceedings{FS04:podc, + author = "Christof Fetzer and Ulrich Schmid", + title = "Brief announcement: on the possibility of consensus + in asynchronous systems with finite average response + times.", + booktitle = PODC04, + year = 2004, + pages = 402 +} + +@InProceedings{GL00:disc, + author = {Eli Gafni and Lesli Lamport}, + title = {Disk Paxos}, + booktitle = DISC00, + pages = {330--344}, + year = {2000}, +} + +@Article{GL03:dc, + author = {Eli Gafni and Lesli Lamport}, + title = {Disk Paxos}, + journal = DC, + year = 2003, + volume = {16}, + number = {1}, + pages = {1--20} +} + +@inproceedings{GP01:wss, + author = "Felix C. G{\"a}rtner and Stefan Pleisch", + title = "({I}m)Possibilities of Predicate Detection in + Crash-Affected Systems", + booktitle = WSS01, + year = 2001, + pages = "98--113" +} + +@inproceedings{GP02:disc, + author = "Felix C. G{\"a}rtner and Stefan Pleisch", + title = "Failure Detection Sequencers: Necessary and + Sufficient Information about Failures to Solve + Predicate Detection", + booktitle = DISC02, + year = 2002, + pages = "280--294" +} + +@inproceedings{GS96:wdag, + author = {Rachid Guerraoui and Andr{\'e} Schiper}, + title = {{``Gamma-Accurate''} Failure Detectors}, + booktitle = WDAG96, + year = {1996}, + pages = {269--286}, + publisher = SPR, + address = {London, UK} +} + +@inproceedings{Gaf98:podc, + author = {Eli Gafni}, + title = {Round-by-round fault detectors (extended abstract): + unifying synchrony and asynchrony}, + booktitle = PODC98, + year = {1998}, + pages = {143--152}, + address = {Puerto Vallarta, Mexico}, + publisher = ACM +} + +@incollection{Gra78:book, + author = {Jim N. Gray}, + title = {Notes on data base operating systems}, + booktitle = {Operating Systems: An Advanced Course}, + chapter = {3.F}, + publisher = {Springer}, + year = {1978}, + editor = {R. Bayer, R.M. Graham, G. Seegm\"uller}, + volume = {60}, + series = {Lecture Notes in Computer Science}, + address = {New York}, + pages = {465}, +} + +@InProceedings{HMR98:srds, + author = {Hurfin, M. and Mostefaoui, A. and Raynal, M.}, + title = {Consensus in asynchronous systems where processes + can crash and recover}, + booktitle = {Seventeenth IEEE Symposium on Reliable Distributed + Systems, Proceedings. }, + pages = { 280--286}, + year = {1998}, + address = {West Lafayette, IN}, + month = oct, + organization = {IEEE} +} + +@inproceedings{HMSZ06:sss, + author = "Martin Hutle and Dahlia Malkhi and Ulrich Schmid and + Lidong Zhou", + title = "Brief Announcement: Chasing the Weakest System Model + for Implementing {$\Omega$} and Consensus", + booktitle = SSS06, + year = 2006 +} + +@incollection{HT93:ds, + author = {Hadzilacos, Vassos and Toueg, Sam}, + title = {Fault-tolerant broadcasts and related problems}, + booktitle = {Distributed systems (2nd Ed.)}, + editor = {Mullender, Sape}, + year = {1993}, + isbn = {0-201-62427-3}, + pages = {97--145}, + numpages = {49} +} + + +@inproceedings{HS06:opodis, + author = {Heinrich Moser and Ulrich Schmid}, + title = {Optimal Clock Synchronization Revisited: Upper and + Lower Bounds in Real-Time Systems}, + booktitle = { Principles of Distributed Systems}, + pages = {94--109}, + year = {2006}, + volume = {4305}, + series = {Lecture Notes in Computer Science}, + publisher = SPR +} + +@techreport{HS06:tr, + author = {Martin Hutle and Andr{\'e} Schiper}, + title = { Communication predicates: A high-level abstraction + for coping with transient and dynamic faults}, + institution = {EPFL}, + number = { LSR-REPORT-2006-006 }, + year = {2006} +} + +@inproceedings{HS07:dsn, + author = {Martin Hutle and Andr{\'e} Schiper}, + title = { Communication predicates: A high-level abstraction + for coping with transient and dynamic faults}, + year = 2007, + booktitle = DSN07, + publisher = IEEE, + location = {Edinburgh,UK}, + pages = {92--10}, + month = jun +} + +@article{Her91:tpls, + author = {Maurice Herlihy}, + title = {Wait-free synchronization}, + journal = TPLS, + volume = {13}, + number = {1}, + year = {1991}, + pages = {124--149}, + publisher = ACM, + address = {New York, NY, USA}, +} + +@article{Kot09:zyzzyva, + author = {Kotla, Ramakrishna and Alvisi, Lorenzo and Dahlin, Mike and Clement, Allen and Wong, Edmund}, + title = {Zyzzyva: Speculative Byzantine fault tolerance}, + journal = {ACM Trans. Comput. Syst.}, + issue_date = {December 2009}, + volume = {27}, + number = {4}, + month = jan, + year = {2010}, + issn = {0734-2071}, + pages = {7:1--7:39}, + articleno = {7}, + numpages = {39}, + publisher = {ACM}, + address = {New York, NY, USA}, + keywords = {Byzantine fault tolerance, output commit, replication, speculative execution}, +} + + +@inproceedings{KMMS97:opodis, + author = "Kim Potter Kihlstrom and Louise E. Moser and + P. M. Melliar-Smith", + title = "Solving Consensus in a Byzantine Environment Using + an Unreliable Fault Detector", + booktitle = "Proceedings of the International Conference on + Principles of Distributed Systems (OPODIS)", + year = 1997, + month = dec, + address = "Chantilly, France", + pages = "61--75" +} + +@inproceedings{KS06:podc, + author = {Idit Keidar and Alexander Shraer}, + title = {Timeliness, failure-detectors, and consensus + performance}, + booktitle = PODC06, + year = {2006}, + pages = {169--178}, + location = {Denver, Colorado, USA}, + publisher = {ACM Press}, + address = {New York, NY, USA}, +} + +@InProceedings{LFA99:disc, + author = {Mikel Larrea and Antonio Fern\'andez and Sergio + Ar\'evalo}, + title = {Efficient algorithms to implement unreliable failure + detectors in partially synchronous systems}, + year = 1999, + month = sep, + pages = {34-48}, + series = "LNCS 1693", + booktitle = DISC99, + publisher = SPR, + address = {Bratislava, Slovaquia}, +} + +@article{LL84:ic, + author = "Jennifer Lundelius and Nancy A. Lynch", + title = "An Upper and Lower Bound for Clock Synchronization", + journal = IC, + volume = 62, + number = {2/3}, + year = 1984, + pages = {190--204} +} + +@techreport{LLS03:tr, + title = {How to Implement a Timer-free Perfect Failure + Detector in Partially Synchronous Systems}, + author = {Le Lann, G\'erard and Schmid, Ulrich}, + institution = TUAuto, + number = "183/1-127", + month = jan, + year = 2003 +} + +@article{LSP82:tpls, + author = {Leslie Lamport and Robert Shostak and Marshall + Pease}, + title = {The {B}yzantine Generals Problem}, + journal = {ACM Trans. Program. Lang. Syst.}, + year = {1982}, +} + +@inproceedings{Lam01:podc, + author = {Butler Lampson}, + title = {The ABCD's of Paxos}, + booktitle = {PODC}, + year = {2001}, + +} + +@inproceedings{Lam03:fddc, + author = {Leslie Lamport}, + title = {Lower Bounds for Asynchronous Consensus}, + booktitle = {Future Directions in Distributed Computing}, + pages = {22--23}, + year = {2003}, + editor = {Andr{\'e} Schiper and Alex A. Shvartsman and Hakim + Weatherspoon and Ben Y. Zhao}, + number = {2584}, + series = {Lecture Notes in Computer Science}, + publisher = SPR +} + +@techreport{Lam04:tr, + author = {Leslie Lamport}, + title = {Lower Bounds for Asynchronous Consensus}, + institution = {Microsoft Research}, + year = {2004}, + number = {MSR-TR-2004-72} +} + +@techreport{Lam05:tr, + author = {Leslie Lamport}, + title = {Fast Paxos}, + institution = {Microsoft Research}, + year = {2005}, + number = {MSR-TR-2005-12} +} + +@techreport{Lam05:tr-33, + author = {Leslie Lamport}, + title = {Generalized Consensus and Paxos}, + institution = {Microsoft Research}, + year = {2005}, + number = {MSR-TR-2005-33} +} + +@Misc{Lam06:slides, + author = {Leslie Lamport}, + title = {Byzantine Paxos}, + howpublished = {Unpublished slides}, + year = {2006} +} + +@Article{Lam86:dc, + author = {Lesli Lamport}, + title = {On Interprocess Communication--Part I: Basic + Formalism, Part II: Algorithms}, + journal = DC, + year = 1986, + volume = 1, + number = 2, + pages = {77--101} +} + +@Article {Lam98:tcs, + author = {Leslie Lamport}, + title = {The part-time parliament}, + journal = ACMTCS, + year = 1998, + volume = 16, + number = 2, + month = may, + pages = {133-169}, +} + +@book{Lyn96:book, + author = {Nancy Lynch}, + title = {Distributed Algorithms}, + publisher = {Morgan Kaufman}, + year = {1996}, +} + +@inproceedings{MA05:dsn, + author = {Martin, J.-P. and Alvisi, L. }, + title = {Fast Byzantine consensus}, + booktitle = DSN05, + pages = {402--411}, + year = {2005}, + month = jun, + organization = {IEEE}, +} + +@article{MA06:tdsc, + author = {Martin, J.-P. and Alvisi, L. }, + title = {Fast {B}yzantine Consensus}, + journal = {TDSC}, + year = {2006}, +} + +@InProceedings{MOZ05:dsn, + author = {Dahlia Malkhi and Florin Oprea and Lidong Zhou}, + title = {{$\Omega$} Meets Paxos: Leader Election and + Stability without Eventual Timely Links}, + booktitle = DSN05, + year = {2005} +} + +@inproceedings{MR00:podc, + author = "Achour Most{\'e}faoui and Michel Raynal", + title = "k-set agreement with limited accuracy failure + detectors", + booktitle = PODC00, + year = 2000, + pages = {143--152}, + location = {Portland, Oregon, United States}, + publisher = ACM +} + +@article{MR01:ppl, + author = "Achour Most{\'e}faoui and Michel Raynal", + title = "Leader-Based Consensus", + journal = PPL, + volume = 11, + number = 1, + year = 2001, + pages = {95--107} +} + +@techreport{OGS97:tr, + author = "Rui Oliveira and Rachid Guerraoui and {Andr\'e} + Schiper", + title = "Consensus in the crash-recover model", + number = "TR-97/239", + year = "1997" +} + +@article{PSL80:jacm, + author = {M. Pease and R. Shostak and L. Lamport}, + title = {Reaching Agreement in the Presence of Faults}, + journal = JACM, + volume = {27}, + number = {2}, + year = {1980}, + pages = {228--234}, + publisher = ACM, + address = ACMADDR, +} + +@article{ST87:jacm, + author = "T. K. Srikanth and Sam Toueg", + title = "Optimal clock synchronization", + journal = JACM, + volume = 34, + number = 3, + year = 1987, + pages = "626--645" +} + +@article{ST87:dc, + author = {T. K. Srikanth and Sam Toueg,}, + title = {Simulating authenticated broadcasts to derive simple fault-tolerant algorithms}, + journal = DC, + volume = {2}, + number = {2}, + year = {1987}, + pages = {80-94} +} + + +@inproceedings{SW89:stacs, + author = {Santoro, Nicola and Widmayer, Peter}, + title = {Time is not a healer}, + booktitle = {Proc.\ 6th Annual Symposium on Theor.\ Aspects of + Computer Science (STACS'89)}, + publisher = "Springer-Verlag", + series = {LNCS}, + volume = "349", + address = "Paderborn, Germany", + pages = "304-313", + year = "1989", + month = feb, +} + +@inproceedings{SW90:sigal, + author = {Nicola Santoro and Peter Widmayer}, + title = {Distributed Function Evaluation in the Presence of + Transmission Faults.}, + booktitle = {SIGAL International Symposium on Algorithms}, + year = {1990}, + pages = {358-367} +} + +@inproceedings{SWR02:icdcs, + author = {Ulrich Schmid and Bettina Weiss and John Rushby}, + title = {Formally Verified Byzantine Agreement in Presence of + Link Faults}, + booktitle = "22nd International Conference on Distributed + Computing Systems (ICDCS'02)", + year = 2002, + month = jul # " 2-5, ", + pages = "608--616", + address = "Vienna, Austria", +} + +@incollection{Sch93a:mullender, + Author = {F. B. Schneider}, + Title = {What Good are Models and What Models are Good}, + BookTitle = {Distributed Systems}, + Year = {1993}, + Editor = {Sape Mullender}, + Publisher = {ACM Press}, + Pages = {169-197}, +} + +@article{VL96:ic, + author = {George Varghese and Nancy A. Lynch}, + title = {A Tradeoff Between Safety and Liveness for + Randomized Coordinated Attack.}, + journal = {Inf. Comput.}, + volume = {128}, + number = {1}, + year = 1996, + pages = {57--71} +} + +@inproceedings{WGWB07:dsn, + title = {Synchronous Consensus with Mortal Byzantines}, + author = {Josef Widder and Günther Gridling and Bettina Weiss + and Jean-Paul Blanquart}, + year = {2007}, + booktitle = DSN07, + publisher = IEEE +} + +@inproceedings{Wid03:disc, + author = {Josef Widder}, + title = {Booting clock Synchronization in Partially + Synchronous Systems}, + booktitle = DISC03, + year = {2003}, + pages = {121--135} +} + +@techreport{Zie04:tr, + author = {Piotr Zieli{\'n}ski}, + title = {Paxos at War}, + institution = {University of Cambridge}, + year = {2004}, + number = {UCAM-CL-TR-593}, +} + +@article{Lam78:cacm, + author = {Leslie Lamport}, + title = {Time, clocks, and the ordering of events in a + distributed system}, + journal = {Commun. ACM}, + year = {1978}, +} + +@Article{Gue06:cj, + author = {Guerraoui, R. and Raynal, M.}, + journal = {The {C}omputer {J}ournal}, + title = {The {A}lpha of {I}ndulgent {C}onsensus}, + year = {2006} +} + +@Article{Gue03:toc, + affiliation = {EPFL}, + author = {Guerraoui, Rachid and Raynal, Michel}, + journal = {{IEEE} {T}rans. on {C}omputers}, + title = {The {I}nformation {S}tructure of {I}ndulgent {C}onsensus}, + year = {2004}, +} + +@techreport{Cas00, + author = {Castro, Miguel}, + title = {Practical {B}yzantine Fault-Tolerance. {PhD} thesis}, + institution = {MIT}, + year = 2000, +} + +@inproceedings{SongRSD08:icdcn, + author = {Yee Jiun Song and + Robbert van Renesse and + Fred B. Schneider and + Danny Dolev}, + title = {The Building Blocks of Consensus}, + booktitle = {ICDCN}, + year = {2008}, +} + + +@inproceedings{BS09:icdcn, + author = {Borran, Fatemeh and Schiper, Andr{\'e}}, + + title = {A {L}eader-free {B}yzantine {C}onsensus {A}lgorithm}, + note = {To appear in ICDCN, 2010}, +} + + +@inproceedings{MHS09:opodis, + author = {Zarko Milosevic and Martin Hutle and Andr{\'e} + Schiper}, + title = {Unifying {B}yzantine Consensus Algorithms with {W}eak + {I}nteractive {C}onsistency}, + note = {To appear in OPODIS 2009}, +} + +@inproceedings{MRR:dsn02, + author = {Most\'{e}faoui, Achour and Rajsbaum, Sergio and Raynal, Michel}, + title = {A Versatile and Modular Consensus Protocol}, + booktitle = {DSN}, + year = {2002}, + } + +@article{MR98:dc, + author = {Dahlia Malkhi and + Michael K. Reiter}, + title = {Byzantine Quorum Systems}, + journal = {Distributed Computing}, + year = {1998}, +} + +@inproceedings{Rei:ccs94, + author = {Reiter, Michael K.}, + title = {Secure agreement protocols: reliable and atomic group multicast in rampart}, + booktitle = {CCS}, + year = {1994}, + pages = {68--80}, + numpages = {13} +} + + +@techreport{RMS09-tr, + author = {Olivier R\"utti and Zarko Milosevic and Andr\'e Schiper}, + title = {{G}eneric construction of consensus algorithm for benign and {B}yzantine faults}, + institution = {EPFL-IC}, + number = {LSR-REPORT-2009-005}, + year = 2009, +} + +@inproceedings{Li:srds07, + author = {Li, Harry C. and Clement, Allen and Aiyer, Amitanand S. and Alvisi, Lorenzo}, + title = {The Paxos Register}, + booktitle = {SRDS}, + year = {2007}, + } + + @article{Amir11:prime, + author = {Amir, Yair and Coan, Brian and Kirsch, Jonathan and Lane, John}, + title = {Prime: Byzantine Replication under Attack}, + journal = {IEEE Trans. Dependable Secur. Comput.}, + issue_date = {July 2011}, + volume = {8}, + number = {4}, + month = jul, + year = {2011}, + issn = {1545-5971}, + pages = {564--577}, + numpages = {14}, + publisher = {IEEE Computer Society Press}, + address = {Los Alamitos, CA, USA}, + keywords = {Performance under attack, Byzantine fault tolerance, replicated state machines, distributed systems.}, +} + +@inproceedings{Mao08:mencius, + author = {Mao, Yanhua and Junqueira, Flavio P. and Marzullo, Keith}, + title = {Mencius: building efficient replicated state machines for WANs}, + booktitle = {OSDI}, + year = {2008}, + pages = {369--384}, + numpages = {16} +} + +@article{Sch90:survey, + author = {Schneider, Fred B.}, + title = {Implementing fault-tolerant services using the state machine approach: a tutorial}, + journal = {ACM Comput. Surv.}, + volume = {22}, + number = {4}, + month = dec, + year = {1990} +} + + +@techreport{HT94:TR, + author = {Hadzilacos, Vassos and Toueg, Sam}, + title = {A Modular Approach to Fault-Tolerant Broadcasts and Related Problems}, + year = {1994}, + source = {http://www.ncstrl.org:8900/ncstrl/servlet/search?formname=detail\&id=oai%3Ancstrlh%3Acornellcs%3ACORNELLCS%3ATR94-1425}, + publisher = {Cornell University}, + address = {Ithaca, NY, USA}, +} + +@inproceedings{Ver09:spinning, + author = {Veronese, Giuliana Santos and Correia, Miguel and Bessani, Alysson Neves and Lung, Lau Cheuk}, + title = {Spin One's Wheels? Byzantine Fault Tolerance with a Spinning Primary}, + booktitle = {SRDS}, + year = {2009}, + numpages = {10} +} + +@inproceedings{Cle09:aardvark, + author = {Clement, Allen and Wong, Edmund and Alvisi, Lorenzo and Dahlin, Mike and Marchetti, Mirco}, + title = {Making Byzantine fault tolerant systems tolerate Byzantine faults}, + booktitle = {NSDI}, + year = {2009}, + pages = {153--168}, + numpages = {16} +} + +@inproceedings{Aiyer05:barB, + author = {Aiyer, Amitanand S. and Alvisi, Lorenzo and Clement, Allen and Dahlin, Mike and Martin, Jean-Philippe and Porth, Carl}, + title = {BAR fault tolerance for cooperative services}, + booktitle = {SOSP}, + year = {2005}, + pages = {45--58}, + numpages = {14} +} + +@inproceedings{Cach01:crypto, + author = {Cachin, Christian and Kursawe, Klaus and Petzold, Frank and Shoup, Victor}, + title = {Secure and Efficient Asynchronous Broadcast Protocols}, + booktitle = {CRYPTO}, + year = {2001}, + pages = {524--541}, + numpages = {18} +} + +@article{Moniz11:ritas, + author = {Moniz, Henrique and Neves, Nuno Ferreria and Correia, Miguel and Verissimo, Paulo}, + title = {RITAS: Services for Randomized Intrusion Tolerance}, + journal = {IEEE Trans. Dependable Secur. Comput.}, + volume = {8}, + number = {1}, + month = jan, + year = {2011}, + pages = {122--136}, + numpages = {15} +} + +@inproceedings{MHS11:jabc, + author = {Milosevic, Zarko and Hutle, Martin and Schiper, Andre}, + title = {On the Reduction of Atomic Broadcast to Consensus with Byzantine Faults}, + booktitle = {SRDS}, + year = {2011}, + pages = {235--244}, + numpages = {10} +} + +@incollection{DHSZ03, + author={Driscoll, Kevin and Hall, Brendan and Sivencrona, Håkan and Zumsteg, Phil}, + title={Byzantine Fault Tolerance, from Theory to Reality}, + year={2003}, + booktitle={Computer Safety, Reliability, and Security}, + volume={2788}, + pages={235--248} +} + +@inproceedings{RMES:dsn07, + author = {Olivier R{\"u}tti and + Sergio Mena and + Richard Ekwall and + Andr{\'e} Schiper}, + title = {On the Cost of Modularity in Atomic Broadcast}, + booktitle = {DSN}, + year = {2007}, + pages = {635-644} +} + +@article{Ben:jc92, + author = {Charles H. Bennett and + Fran\c{c}ois Bessette and + Gilles Brassard and + Louis Salvail and + John A. Smolin}, + title = {Experimental Quantum Cryptography}, + journal = {J. Cryptology}, + volume = {5}, + number = {1}, + year = {1992}, + pages = {3-28} +} + +@inproceedings{Aiyer:disc08, + author = {Aiyer, Amitanand S. and Alvisi, Lorenzo and Bazzi, Rida A. and Clement, Allen}, + title = {Matrix Signatures: From MACs to Digital Signatures in Distributed Systems}, + booktitle = {DISC}, + year = {2008}, + pages = {16--31}, + numpages = {16} +} + +@inproceedings{Biel13:dsn, + author = {Biely, Martin and Delgado, Pamela and Milosevic, Zarko and Schiper, Andr{\'e}}, + title = {Distal: A Framework for Implementing Fault-tolerant Distributed Algorithms}, + note = {To appear in DSN, 2013}, + year = 2013 +} + +@inproceedings{BS10:icdcn, + author = {Borran, Fatemeh and Schiper, Andr{\'e}}, + title = {A leader-free Byzantine consensus algorithm}, + booktitle = {ICDCN}, + year = {2010}, + pages = {67--78}, + numpages = {12} +} + +@article{Cor06:cj, + author = {Correia, Miguel and Neves, Nuno Ferreira and Ver\'{\i}ssimo, Paulo}, + title = {From Consensus to Atomic Broadcast: Time-Free Byzantine-Resistant Protocols without Signatures}, + journal = {Comput. J.}, + volume = {49}, + number = {1}, + year = {2006}, + pages = {82--96}, + numpages = {15} +} + +@inproceedings{RMS10:dsn, + author = {Olivier R{\"u}tti and + Zarko Milosevic and + Andr{\'e} Schiper}, + title = {Generic construction of consensus algorithms for benign + and Byzantine faults}, + booktitle = {DSN}, + year = {2010}, + pages = {343-352} +} + + + +@inproceedings{HKJR:usenix10, + author = {Hunt, Patrick and Konar, Mahadev and Junqueira, Flavio P. and Reed, Benjamin}, + title = {ZooKeeper: wait-free coordination for internet-scale systems}, + OPTbooktitle = {Proceedings of the 2010 USENIX conference on USENIX annual technical conference}, + booktitle = {USENIXATC}, + year = {2010}, + OPTlocation = {Boston, MA}, + pages = {11}, + numpages = {1}, + OPTurl = {http://dl.acm.org/citation.cfm?id=1855840.1855851}, + acmid = {1855851}, + OPTpublisher = {USENIX Association}, + OPTaddress = {Berkeley, CA, USA}, +} + +@inproceedings{Bur:osdi06, + author = {Burrows, Mike}, + title = {The Chubby lock service for loosely-coupled distributed systems}, + booktitle = {OSDI}, + year = {2006}, + pages = {335--350}, + numpages = {16}, +} + +@INPROCEEDINGS{Mao09:hotdep, + author = {Yanhua Mao and Flavio P. Junqueira and Keith Marzullo}, + title = {Towards low latency state machine replication for uncivil wide-area networks}, + booktitle = {HotDep}, + year = {2009} +} + +@inproceedings{Chun07:a2m, + author = {Chun, Byung-Gon and Maniatis, Petros and Shenker, Scott and Kubiatowicz, John}, + title = {Attested append-only memory: making adversaries stick to their word}, + booktitle = {SOSP}, + year = {2007}, + pages = {189--204}, + numpages = {16} +} + +@TECHREPORT{MBS:epfltr, + author = {Zarko Milosevic and Martin Biely and Andr\'e Schiper}, + title = {Bounded {D}elay in {B}yzantine {T}olerant {S}tate {M}achine {R}eplication}, + year = 2013, + month = april, + institution = {EPFL}, + number = {185962}, +} + +@book{BH09:datacenter, + author = {Barroso, Luiz Andre and Hoelzle, Urs}, + title = {The Datacenter as a Computer: An Introduction to the Design of Warehouse-Scale Machines}, + year = {2009}, + isbn = {159829556X, 9781598295566}, + edition = {1st}, + publisher = {Morgan and Claypool Publishers}, +} + +@inproceedings{Kir11:csiirw, + author = {Kirsch, Jonathan and Goose, Stuart and Amir, Yair and Skare, Paul}, + title = {Toward survivable SCADA}, + booktitle = {CSIIRW}, + year = {2011}, + pages = {21:1--21:1}, + articleno = {21}, + numpages = {1} +} + +@inproceedings{Ongaro14:raft, + author = {Ongaro, Diego and Ousterhout, John}, + title = {In Search of an Understandable Consensus Algorithm}, + booktitle = {Proceedings of the 2014 USENIX Conference on USENIX Annual Technical Conference}, + series = {USENIX ATC'14}, + year = {2014}, + isbn = {978-1-931971-10-2}, + location = {Philadelphia, PA}, + pages = {305--320}, + numpages = {16}, + url = {http://dl.acm.org/citation.cfm?id=2643634.2643666}, + acmid = {2643666}, + publisher = {USENIX Association}, + address = {Berkeley, CA, USA}, +} + +@article{GLR17:red-belly-bc, + author = {Tyler Crain and + Vincent Gramoli and + Mikel Larrea and + Michel Raynal}, + title = {Leader/Randomization/Signature-free Byzantine Consensus for Consortium + Blockchains}, + journal = {CoRR}, + volume = {abs/1702.03068}, + year = {2017}, + url = {http://arxiv.org/abs/1702.03068}, + archivePrefix = {arXiv}, + eprint = {1702.03068}, + timestamp = {Wed, 07 Jun 2017 14:41:08 +0200}, + biburl = {http://dblp.org/rec/bib/journals/corr/CrainGLR17}, + bibsource = {dblp computer science bibliography, http://dblp.org} +} + + +@misc{Nak2012:bitcoin, + added-at = {2014-04-17T08:33:06.000+0200}, + author = {Nakamoto, Satoshi}, + biburl = {https://www.bibsonomy.org/bibtex/23db66df0fc9fa2b5033f096a901f1c36/ngnn}, + interhash = {423c2cdff70ba0cd0bca55ebb164d770}, + intrahash = {3db66df0fc9fa2b5033f096a901f1c36}, + keywords = {imported}, + timestamp = {2014-04-17T08:33:06.000+0200}, + title = {Bitcoin: A peer-to-peer electronic cash system}, + url = {http://www.bitcoin.org/bitcoin.pdf}, + year = 2009 +} + +@misc{But2014:ethereum, + author = {Vitalik Buterin}, + title = {Ethereum: A next-generation smart contract and decentralized application platform}, + year = {2014}, + howpublished = {\url{https://github.com/ethereum/wiki/wiki/White-Paper}}, + note = {Accessed: 2018-07-11}, + url = {https://github.com/ethereum/wiki/wiki/White-Paper}, +} + +@inproceedings{Dem1987:gossip, + author = {Demers, Alan and Greene, Dan and Hauser, Carl and Irish, Wes and Larson, John and Shenker, Scott and Sturgis, Howard and Swinehart, Dan and Terry, Doug}, + title = {Epidemic Algorithms for Replicated Database Maintenance}, + booktitle = {Proceedings of the Sixth Annual ACM Symposium on Principles of Distributed Computing}, + series = {PODC '87}, + year = {1987}, + isbn = {0-89791-239-X}, + location = {Vancouver, British Columbia, Canada}, + pages = {1--12}, + numpages = {12}, + url = {http://doi.acm.org/10.1145/41840.41841}, + doi = {10.1145/41840.41841}, + acmid = {41841}, + publisher = {ACM}, + address = {New York, NY, USA}, +} + +@article{Gue2018:sbft, + author = {Guy Golan{-}Gueta and + Ittai Abraham and + Shelly Grossman and + Dahlia Malkhi and + Benny Pinkas and + Michael K. Reiter and + Dragos{-}Adrian Seredinschi and + Orr Tamir and + Alin Tomescu}, + title = {{SBFT:} a Scalable Decentralized Trust Infrastructure for Blockchains}, + journal = {CoRR}, + volume = {abs/1804.01626}, + year = {2018}, + url = {http://arxiv.org/abs/1804.01626}, + archivePrefix = {arXiv}, + eprint = {1804.01626}, + timestamp = {Tue, 01 May 2018 19:46:29 +0200}, + biburl = {https://dblp.org/rec/bib/journals/corr/abs-1804-01626}, + bibsource = {dblp computer science bibliography, https://dblp.org} +} + +@inproceedings{BLS2001:crypto, + author = {Boneh, Dan and Lynn, Ben and Shacham, Hovav}, + title = {Short Signatures from the Weil Pairing}, + booktitle = {Proceedings of the 7th International Conference on the Theory and Application of Cryptology and Information Security: Advances in Cryptology}, + series = {ASIACRYPT '01}, + year = {2001}, + isbn = {3-540-42987-5}, + pages = {514--532}, + numpages = {19}, + url = {http://dl.acm.org/citation.cfm?id=647097.717005}, + acmid = {717005}, + publisher = {Springer-Verlag}, + address = {Berlin, Heidelberg}, +} + + diff --git a/cometbft/v0.39/spec/consensus/consensus-paper/paper.tex b/cometbft/v0.39/spec/consensus/consensus-paper/paper.tex new file mode 100644 index 000000000..22f8b405f --- /dev/null +++ b/cometbft/v0.39/spec/consensus/consensus-paper/paper.tex @@ -0,0 +1,153 @@ +%\documentclass[conference]{IEEEtran} +\documentclass[conference,onecolumn,draft,a4paper]{IEEEtran} +% Add the compsoc option for Computer Society conferences. +% +% If IEEEtran.cls has not been installed into the LaTeX system files, +% manually specify the path to it like: +% \documentclass[conference]{../sty/IEEEtran} + + + +% *** GRAPHICS RELATED PACKAGES *** +% +\ifCLASSINFOpdf +\else +\fi + +% correct bad hyphenation here +\hyphenation{op-tical net-works semi-conduc-tor} + +%\usepackage[caption=false,font=footnotesize]{subfig} +\usepackage{tikz} +\usetikzlibrary{decorations,shapes,backgrounds,calc} +\tikzstyle{msg}=[->,black,>=latex] +\tikzstyle{rubber}=[|<->|] +\tikzstyle{announce}=[draw=blue,fill=blue,shape=diamond,right,minimum + height=2mm,minimum width=1.6667mm,inner sep=0pt] +\tikzstyle{decide}=[draw=red,fill=red,shape=isosceles triangle,right,minimum + height=2mm,minimum width=1.6667mm,inner sep=0pt,shape border rotate=90] +\tikzstyle{cast}=[draw=green!50!black,fill=green!50!black,shape=circle,left,minimum + height=2mm,minimum width=1.6667mm,inner sep=0pt] + + +\usepackage{multirow} +\usepackage{graphicx} +\usepackage{epstopdf} +\usepackage{amssymb} +\usepackage{rounddiag} +\graphicspath{{../}} + +\usepackage{technote} +\usepackage{homodel} +\usepackage{enumerate} +%%\usepackage{ulem}\normalem + +% to center caption +\usepackage{caption} + +\newcommand{\textstretch}{1.4} +\newcommand{\algostretch}{1} +\newcommand{\eqnstretch}{0.5} + +\newconstruct{\FOREACH}{\textbf{for each}}{\textbf{do}}{\ENDFOREACH}{} + +%\newconstruct{\ON}{\textbf{on}}{\textbf{do}}{\ENDON}{\textbf{end on}} +\newcommand\With{\textbf{while}} +\newcommand\From{\textbf{from}} +\newcommand\Broadcast{\textbf{broadcast}} +\newcommand\PBroadcast{send} +\newcommand\UpCall{\textbf{UpCall}} +\newcommand\DownCall{\textbf{DownCall}} +\newcommand \Call{\textbf{Call}} +\newident{noop} +\newconstruct{\UPON}{\textbf{upon}}{\textbf{do}}{\ENDUPON}{} + + + +\newcommand{\abcast}{\mathsf{to\mbox{\sf-}broadcast}} +\newcommand{\adeliver}{\mathsf{to\mbox{\sf-}deliver}} + +\newcommand{\ABCAgreement}{\emph{TO-Agreement}} +\newcommand{\ABCIntegrity}{\emph{TO-Integrity}} +\newcommand{\ABCValidity}{\emph{TO-Validity}} +\newcommand{\ABCTotalOrder}{\emph{TO-Order}} +\newcommand{\ABCBoundedDelivery}{\emph{TO-Bounded Delivery}} + + +\newcommand{\tabc}{\mathit{atab\mbox{\sf-}cast}} +\newcommand{\anno}{\mathit{atab\mbox{\sf-}announce}} +\newcommand{\abort}{\mathit{atab\mbox{\sf-}abort}} +\newcommand{\tadel}{\mathit{atab\mbox{\sf-}deliver}} + +\newcommand{\ATABAgreement}{\emph{ATAB-Agreement}} +\newcommand{\ATABAbort}{\emph{ATAB-Abort}} +\newcommand{\ATABIntegrity}{\emph{ATAB-Integrity}} +\newcommand{\ATABValidity}{\emph{ATAB-Validity}} +\newcommand{\ATABAnnounce}{\emph{ATAB-Announcement}} +\newcommand{\ATABTermination}{\emph{ATAB-Termination}} +%\newcommand{\ATABFastAnnounce}{\emph{ATAB-Fast-Announcement}} + +%% Command for observations. +\newtheorem{observation}{Observation} + + +%% HO ALGORITHM DEFINITIONS +\newconstruct{\FUNCTION}{\textbf{Function}}{\textbf{:}}{\ENDFUNCTION}{} + +%% Uncomment the following four lines to remove remarks and visible traces of +%% modifications in the document +%%\renewcommand{\sout}[1]{\relaxx} +%%\renewcommand{\uline}[1]{#1} +%% \renewcommand{\uwave}[1]{#1} + \renewcommand{\note}[2][default]{\relax} + + +%% The following commands can be used to generate TR or Conference version of the paper +\newcommand{\tr}[1]{} +\renewcommand{\tr}[1]{#1} +\newcommand{\onlypaper}[1]{#1} +%\renewcommand{\onlypaper}[1]{} +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +%\pagestyle{plain} +%\pagestyle{empty} + +%% IEEE tweaks +%\setlength{\IEEEilabelindent}{.5\parindent} +%\setlength{\IEEEiednormlabelsep}{.5\parindent} + +\begin{document} +% +% paper title +% can use linebreaks \\ within to get better formatting as desired +\title{The latest gossip on BFT consensus\vspace{-0.7\baselineskip}} + + + +\author{\IEEEauthorblockN{\large Ethan Buchman, Jae Kwon and Zarko Milosevic\\} + \IEEEauthorblockN{\large Tendermint}\\ + %\\\vspace{-0.5\baselineskip} + \IEEEauthorblockN{September 24, 2018} +} + +% make the title area +\maketitle +\vspace*{0.5em} + +\begin{abstract} +This paper presents Tendermint, a new protocol for ordering events in a distributed network under adversarial conditions. More commonly known as Byzantine Fault Tolerant (BFT) consensus or atomic broadcast, the problem has attracted significant attention in recent years due to the widespread success of blockchain-based digital currencies, such as Bitcoin and Ethereum, which successfully solved the problem in a public setting without a central authority. Tendermint modernizes classic academic work on the subject and simplifies the design of the BFT algorithm by relying on a peer-to-peer gossip protocol among nodes. +\end{abstract} + +%\noindent \textbf{Keywords:} Blockchain, Byzantine Fault Tolerance, State Machine %Replication + +\input{intro} +\input{definitions} +\input{consensus} +\input{proof} +\input{conclusion} + +\bibliographystyle{IEEEtran} +\bibliography{lit} + +%\appendix + +\end{document} diff --git a/cometbft/v0.39/spec/consensus/consensus-paper/proof.tex b/cometbft/v0.39/spec/consensus/consensus-paper/proof.tex new file mode 100644 index 000000000..1c84d9b11 --- /dev/null +++ b/cometbft/v0.39/spec/consensus/consensus-paper/proof.tex @@ -0,0 +1,280 @@ +\section{Proof of Tendermint consensus algorithm} \label{sec:proof} + +\begin{lemma} \label{lemma:majority-intersection} For all $f\geq 0$, any two +sets of processes with voting power at least equal to $2f+1$ have at least one +correct process in common. \end{lemma} + +\begin{proof} As the total voting power is equal to $n=3f+1$, we have $2(2f+1) + = n+f+1$. This means that the intersection of two sets with the voting + power equal to $2f+1$ contains at least $f+1$ voting power in common, \ie, + at least one correct process (as the total voting power of faulty processes + is $f$). The result follows directly from this. \end{proof} + +\begin{lemma} \label{lemma:locked-decision_value-prevote-v} If $f+1$ correct +processes lock value $v$ in round $r_0$ ($lockedValue = v$ and $lockedRound = +r_0$), then in all rounds $r > r_0$, they send $\Prevote$ for $id(v)$ or +$\nil$. \end{lemma} + +\begin{proof} We prove the result by induction on $r$. + +\emph{Base step $r = r_0 + 1:$} Let's denote with $C$ the set of correct +processes with voting power equal to $f+1$. By the rules at +line~\ref{line:tab:recvProposal} and line~\ref{line:tab:acceptProposal}, the +processes from the set $C$ can't accept $\Proposal$ for any value different +from $v$ in round $r$, and therefore can't send a $\li{\Prevote,height_p, +r,id(v')}$ message, if $v' \neq v$. Therefore, the Lemma holds for the base +step. + +\emph{Induction step from $r_1$ to $r_1+1$:} We assume that no process from the +set $C$ has sent $\Prevote$ for values different than $id(v)$ or $\nil$ until +round $r_1 + 1$. We now prove that the Lemma also holds for round $r_1 + 1$. As +processes from the set $C$ send $\Prevote$ for $id(v)$ or $\nil$ in rounds $r_0 +\le r \le r_1$, by Lemma~\ref{lemma:majority-intersection} there is no value +$v' \neq v$ for which it is possible to receive $2f+1$ $\Prevote$ messages in +those rounds (i). Therefore, we have for all processes from the set $C$, +$lockedValue = v$ and $lockedRound \ge r_0$. Let's assume by a contradiction +that a process $q$ from the set $C$ sends $\Prevote$ in round $r_1 + 1$ for +value $id(v')$, where $v' \neq v$. This is possible only by +line~\ref{line:tab:prevote-higher-proposal}. Note that this implies that $q$ +received $2f+1$ $\li{\Prevote,h_q, r,id(v')}$ messages, where $r > r_0$ and $r +< r_1 +1$ (see line~\ref{line:tab:cond-prevote-higher-proposal}). A +contradiction with (i) and Lemma~\ref{lemma:majority-intersection}. +\end{proof} + +\begin{lemma} \label{lemma:agreement} Algorithm~\ref{alg:tendermint} satisfies +Agreement. \end{lemma} + +\begin{proof} Let round $r_0$ be the first round of height $h$ such that some + correct process $p$ decides $v$. We now prove that if some correct process + $q$ decides $v'$ in some round $r \ge r_0$, then $v = v'$. + +In case $r = r_0$, $q$ has received at least $2f+1$ +$\li{\Precommit,h_p,r_0,id(v')}$ messages at line~\ref{line:tab:onDecideRule}, +while $p$ has received at least $2f+1$ $\li{\Precommit,h_p,r_0,id(v)}$ +messages. By Lemma~\ref{lemma:majority-intersection} two sets of messages of +voting power $2f+1$ intersect in at least one correct process. As a correct +process sends a single $\Precommit$ message in a round, then $v=v'$. + +We prove the case $r > r_0$ by contradiction. By the +rule~\ref{line:tab:onDecideRule}, $p$ has received at least $2f+1$ voting-power +equivalent of $\li{\Precommit,h_p,r_0,id(v)}$ messages, i.e., at least $f+1$ +voting-power equivalent correct processes have locked value $v$ in round $r_0$ and have +sent those messages (i). Let denote this set of messages with $C$. On the +other side, $q$ has received at least $2f+1$ voting power equivalent of +$\li{\Precommit,h_q, r,id(v')}$ messages. As the voting power of all faulty +processes is at most $f$, some correct process $c$ has sent one of those +messages. By the rule at line~\ref{line:tab:recvPrevote}, $c$ has locked value +$v'$ in round $r$ before sending $\li{\Precommit,h_q, r,id(v')}$. Therefore $c$ +has received $2f+1$ $\Prevote$ messages for $id(v')$ in round $r > r_0$ (see +line~\ref{line:tab:recvPrevote}). By Lemma~\ref{lemma:majority-intersection}, a +process from the set $C$ has sent $\Prevote$ message for $id(v')$ in round $r$. +A contradiction with (i) and Lemma~\ref{lemma:locked-decision_value-prevote-v}. +\end{proof} + +\begin{lemma} \label{lemma:agreement} Algorithm~\ref{alg:tendermint} satisfies +Validity. \end{lemma} + +\begin{proof} Trivially follows from the rule at line +\ref{line:tab:validDecisionValue} which ensures that only valid values can be +decided. \end{proof} + +\begin{lemma} \label{lemma:round-synchronisation} If we assume that: +\begin{enumerate} + \item a correct process $p$ is the first correct process to + enter a round $r>0$ at time $t > GST$ (for every correct process + $c$, $round_c \le r$ at time $t$) + \item the proposer of round $r$ is + a correct process $q$ + \item for every correct process $c$, + $lockedRound_c \le validRound_q$ at time $t$ + \item $\timeoutPropose(r) + > 2\Delta + \timeoutPrecommit(r-1)$, $\timeoutPrevote(r) > 2\Delta$ and + $\timeoutPrecommit(r) > 2\Delta$, +\end{enumerate} +then all correct processes decide in round $r$ before $t + 4\Delta + + \timeoutPrecommit(r-1)$. +\end{lemma} + +\begin{proof} As $p$ is the first correct process to enter round $r$, it + executed the line~\ref{line:tab:nextRound} after $\timeoutPrecommit(r-1)$ + expired. Therefore, $p$ received $2f+1$ $\Precommit$ messages in the round + $r-1$ before time $t$. By the \emph{Gossip communication} property, all + correct processes will receive those messages the latest at time $t + + \Delta$. Correct processes that are in rounds $< r-1$ at time $t$ will + enter round $r-1$ (see the rule at line~\ref{line:tab:nextRound2}) and + trigger $\timeoutPrecommit(r-1)$ (see rule~\ref{line:tab:startTimeoutPrecommit}) + by time $t+\Delta$. Therefore, all correct processes will start round $r$ + by time $t+\Delta+\timeoutPrecommit(r-1)$ (i). + +In the worst case, the process $q$ is the last correct process to enter round +$r$, so $q$ starts round $r$ and sends $\Proposal$ message for some value $v$ +at time $t + \Delta + \timeoutPrecommit(r-1)$. Therefore, all correct processes +receive the $\Proposal$ message from $q$ the latest by time $t + 2\Delta + +\timeoutPrecommit(r-1)$. Therefore, if $\timeoutPropose(r) > 2\Delta + +\timeoutPrecommit(r-1)$, all correct processes will receive $\Proposal$ message +before $\timeoutPropose(r)$ expires. + +By (3) and the rules at line~\ref{line:tab:recvProposal} and +\ref{line:tab:acceptProposal}, all correct processes will accept the +$\Proposal$ message for value $v$ and will send a $\Prevote$ message for +$id(v)$ by time $t + 2\Delta + \timeoutPrecommit(r-1)$. Note that by the +\emph{Gossip communication} property, the $\Prevote$ messages needed to trigger +the rule at line~\ref{line:tab:acceptProposal} are received before time $t + +\Delta$. + +By time $t + 3\Delta + \timeoutPrecommit(r-1)$, all correct processes will receive +$\Proposal$ for $v$ and $2f+1$ corresponding $\Prevote$ messages for $id(v)$. +By the rule at line~\ref{line:tab:recvPrevote}, all correct processes will send +a $\Precommit$ message (see line~\ref{line:tab:precommit-v}) for $id(v)$ by +time $t + 3\Delta + \timeoutPrecommit(r-1)$. Therefore, by time $t + 4\Delta + +\timeoutPrecommit(r-1)$, all correct processes will have received the $\Proposal$ +for $v$ and $2f+1$ $\Precommit$ messages for $id(v)$, so they decide at +line~\ref{line:tab:decide} on $v$. + +This scenario holds if every correct process $q$ sends a $\Precommit$ message +before $\timeoutPrevote(r)$ expires, and if $\timeoutPrecommit(r)$ does not expire +before $t + 4\Delta + \timeoutPrecommit(r-1)$. Let's assume that a correct process +$c_1$ is the first correct process to trigger $\timeoutPrevote(r)$ (see the rule +at line~\ref{line:tab:recvAny2/3Prevote}) at time $t_1 > t$. This implies that +before time $t_1$, $c_1$ received a $\Proposal$ ($step_{c_1}$ must be +$\prevote$ by the rule at line~\ref{line:tab:recvAny2/3Prevote}) and a set of +$2f+1$ $\Prevote$ messages. By time $t_1 + \Delta$, all correct processes will +receive those messages. Note that even if some correct process was in the +smaller round before time $t_1$, at time $t_1 + \Delta$ it will start round $r$ +after receiving those messages (see the rule at +line~\ref{line:tab:skipRounds}). Therefore, all correct processes will send +their $\Prevote$ message for $id(v)$ by time $t_1 + \Delta$, and all correct +processes will receive those messages the by time $t_1 + 2\Delta$. Therefore, +as $\timeoutPrevote(r) > 2\Delta$, this ensures that all correct processes receive +$\Prevote$ messages from all correct processes before their respective local +$\timeoutPrevote(r)$ expire. + +On the other hand, $\timeoutPrecommit(r)$ is triggered in a correct process $c_2$ +after it receives any set of $2f+1$ $\Precommit$ messages for the first time. +Let's denote with $t_2 > t$ the earliest point in time $\timeoutPrecommit(r)$ is +triggered in some correct process $c_2$. This implies that $c_2$ has received +at least $f+1$ $\Precommit$ messages for $id(v)$ from correct processes, i.e., +those processes have received $\Proposal$ for $v$ and $2f+1$ $\Prevote$ +messages for $id(v)$ before time $t_2$. By the \emph{Gossip communication} +property, all correct processes will receive those messages by time $t_2 + +\Delta$, and will send $\Precommit$ messages for $id(v)$. Note that even if +some correct processes were at time $t_2$ in a round smaller than $r$, by the +rule at line~\ref{line:tab:skipRounds} they will enter round $r$ by time $t_2 + +\Delta$. Therefore, by time $t_2 + 2\Delta$, all correct processes will +receive $\Proposal$ for $v$ and $2f+1$ $\Precommit$ messages for $id(v)$. So if +$\timeoutPrecommit(r) > 2\Delta$, all correct processes will decide before the +timeout expires. \end{proof} + + +\begin{lemma} \label{lemma:validValue} If a correct process $p$ locks a value + $v$ at time $t_0 > GST$ in some round $r$ ($lockedValue = v$ and + $lockedRound = r$) and $\timeoutPrecommit(r) > 2\Delta$, then all correct + processes set $validValue$ to $v$ and $validRound$ to $r$ before starting + round $r+1$. \end{lemma} + +\begin{proof} In order to prove this Lemma, we need to prove that if the + process $p$ locks a value $v$ at time $t_0$, then no correct process will + leave round $r$ before time $t_0 + \Delta$ (unless it has already set + $validValue$ to $v$ and $validRound$ to $r$). It is sufficient to prove + this, since by the \emph{Gossip communication} property the messages that + $p$ received at time $t_0$ and that triggered rule at + line~\ref{line:tab:recvPrevote} will be received by time $t_0 + \Delta$ by + all correct processes, so all correct processes that are still in round $r$ + will set $validValue$ to $v$ and $validRound$ to $r$ (by the rule at + line~\ref{line:tab:recvPrevote}). To prove this, we need to compute the + earliest point in time a correct process could leave round $r$ without + updating $validValue$ to $v$ and $validRound$ to $r$ (we denote this time + with $t_1$). The Lemma is correct if $t_0 + \Delta < t_1$. + +If the process $p$ locks a value $v$ at time $t_0$, this implies that $p$ +received the valid $\Proposal$ message for $v$ and $2f+1$ +$\li{\Prevote,h,r,id(v)}$ at time $t_0$. At least $f+1$ of those messages are +sent by correct processes. Let's denote this set of correct processes as $C$. By +Lemma~\ref{lemma:majority-intersection} any set of $2f+1$ $\Prevote$ messages +in round $r$ contains at least a single message from the set $C$. + +Let's denote as time $t$ the earliest point in time a correct process, $c_1$, triggered +$\timeoutPrevote(r)$. This implies that $c_1$ received $2f+1$ $\Prevote$ messages +(see the rule at line \ref{line:tab:recvAny2/3Prevote}), where at least one of +those messages was sent by a process $c_2$ from the set $C$. Therefore, process +$c_2$ had received $\Proposal$ message before time $t$. By the \emph{Gossip +communication} property, all correct processes will receive $\Proposal$ and +$2f+1$ $\Prevote$ messages for round $r$ by time $t+\Delta$. The latest point +in time $p$ will trigger $\timeoutPrevote(r)$ is $t+\Delta$\footnote{Note that +even if $p$ was in smaller round at time $t$ it will start round $r$ by time +$t+\Delta$.}. So the latest point in time $p$ can lock the value $v$ in +round $r$ is $t_0 = t+\Delta+\timeoutPrevote(r)$ (as at this point +$\timeoutPrevote(r)$ expires, so a process sends $\Precommit$ $\nil$ and updates +$step$ to $\precommit$, see line \ref{line:tab:onTimeoutPrevote}). + +Note that according to the Algorithm \ref{alg:tendermint}, a correct process +can not send a $\Precommit$ message before receiving $2f+1$ $\Prevote$ +messages. Therefore, no correct process can send a $\Precommit$ message in +round $r$ before time $t$. If a correct process sends a $\Precommit$ message +for $\nil$, it implies that it has waited for the full duration of +$\timeoutPrevote(r)$ (see line +\ref{line:tab:precommit-nil-onTimeout})\footnote{The other case in which a +correct process $\Precommit$ for $\nil$ is after receiving $2f+1$ $Prevote$ for +$\nil$ messages, see the line \ref{line:tab:precommit-v-1}. By +Lemma~\ref{lemma:majority-intersection}, this is not possible in round $r$.}. +Therefore, no correct process can send $\Precommit$ for $\nil$ before time $t + +\timeoutPrevote(r)$ (*). + +A correct process $q$ that enters round $r+1$ must wait (i) $\timeoutPrecommit(r)$ +(see line \ref{line:tab:nextRound}) or (ii) receiving $f+1$ messages from the +round $r+1$ (see the line \ref{line:tab:skipRounds}). In the former case, $q$ +receives $2f+1$ $\Precommit$ messages before starting $\timeoutPrecommit(r)$. If +at least a single $\Precommit$ message from a correct process (at least $f+1$ +voting power equivalent of those messages is sent by correct processes) is for +$\nil$, then $q$ cannot start round $r+1$ before time $t_1 = t + +\timeoutPrevote(r) + \timeoutPrecommit(r)$ (see (*)). Therefore in this case we have: +$t_0 + \Delta < t_1$, i.e., $t+2\Delta+\timeoutPrevote(r) < t + \timeoutPrevote(r) + +\timeoutPrecommit(r)$, and this is true whenever $\timeoutPrecommit(r) > 2\Delta$, so +Lemma holds in this case. + +If in the set of $2f+1$ $\Precommit$ messages $q$ receives, there is at least a +single $\Precommit$ for $id(v)$ message from a correct process $c$, then $q$ +can start the round $r+1$ the earliest at time $t_1 = t+\timeoutPrecommit(r)$. In +this case, by the \emph{Gossip communication} property, all correct processes +will receive $\Proposal$ and $2f+1$ $\Prevote$ messages (that $c$ received +before time $t$) the latest at time $t+\Delta$. Therefore, $q$ will set +$validValue$ to $v$ and $validRound$ to $r$ the latest at time $t+\Delta$. As +$t+\Delta < t+\timeoutPrecommit(r)$, whenever $\timeoutPrecommit(r) > \Delta$, the +Lemma holds also in this case. + +In case (ii), $q$ received at least a single message from a correct process $c$ +from the round $r+1$. The earliest point in time $c$ could have started round +$r+1$ is $t+\timeoutPrecommit(r)$ in case it received a $\Precommit$ message for +$v$ from some correct process in the set of $2f+1$ $\Precommit$ messages it +received. The same reasoning as above holds also in this case, so $q$ set +$validValue$ to $v$ and $validRound$ to $r$ the latest by time $t+\Delta$. As +$t+\Delta < t+\timeoutPrecommit(r)$, whenever $\timeoutPrecommit(r) > \Delta$, the +Lemma holds also in this case. \end{proof} + +\begin{lemma} \label{lemma:agreement} Algorithm~\ref{alg:tendermint} satisfies +Termination. \end{lemma} + +\begin{proof} Lemma~\ref{lemma:round-synchronisation} defines a scenario in + which all correct processes decide. We now prove that within a bounded + duration after GST such a scenario will unfold. Let's assume that at time + $GST$ the highest round started by a correct process is $r_0$, and that + there exists a correct process $p$ such that the following holds: for every + correct process $c$, $lockedRound_c \le validRound_p$. Furthermore, we + assume that $p$ will be the proposer in some round $r_1 > r$ (this is + ensured by the $\coord$ function). + +We have two cases to consider. In the first case, for all rounds $r \ge r_0$ +and $r < r_1$, no correct process locks a value (set $lockedRound$ to $r$). So +in round $r_1$ we have the scenario from the +Lemma~\ref{lemma:round-synchronisation}, so all correct processes decides in +round $r_1$. + +In the second case, a correct process locks a value $v$ in round $r_2$, where +$r_2 \ge r_0$ and $r_2 < r_1$. Let's assume that $r_2$ is the highest round +before $r_1$ in which some correct process $q$ locks a value. By Lemma +\ref{lemma:validValue} at the end of round $r_2$ the following holds for all +correct processes $c$: $validValue_c = lockedValue_q$ and $validRound_c = r_2$. +Then in round $r_1$, the conditions for the +Lemma~\ref{lemma:round-synchronisation} holds, so all correct processes decide. +\end{proof} + diff --git a/cometbft/v0.39/spec/consensus/consensus-paper/rounddiag.sty b/cometbft/v0.39/spec/consensus/consensus-paper/rounddiag.sty new file mode 100644 index 000000000..a6ca5d883 --- /dev/null +++ b/cometbft/v0.39/spec/consensus/consensus-paper/rounddiag.sty @@ -0,0 +1,62 @@ +% ROUNDDIAG STYLE +% for LaTeX version 2e +% by -- 2008 Martin Hutle +% +% This style file is free software; you can redistribute it and/or +% modify it under the terms of the GNU Lesser General Public +% License as published by the Free Software Foundation; either +% version 2 of the License, or (at your option) any later version. +% +% This style file is distributed in the hope that it will be useful, +% but WITHOUT ANY WARRANTY; without even the implied warranty of +% MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +% Lesser General Public License for more details. +% +% You should have received a copy of the GNU Lesser General Public +% License along with this style file; if not, write to the +% Free Software Foundation, Inc., 59 Temple Place - Suite 330, +% Boston, MA 02111-1307, USA. +% +\NeedsTeXFormat{LaTeX2e} +\ProvidesPackage{rounddiag} +\typeout{Document Style `rounddiag' - provides simple round diagrams} +% +\RequirePackage{ifthen} +\RequirePackage{calc} +\RequirePackage{tikz} + +\def\rdstretch{3} + +\tikzstyle{msg}=[->,thick,>=latex] +\tikzstyle{rndline}=[dotted] +\tikzstyle{procline}=[dotted] + +\newenvironment{rounddiag}[2]{ +\begin{center} +\begin{tikzpicture} +\foreach \i in {1,...,#1}{ + \draw[procline] (0,#1-\i) node[xshift=-1em]{$p_{\i}$} -- (#2*\rdstretch+1,#1-\i); +} +\foreach \i in {0,...,#2}{ + \draw[rndline] (\i*\rdstretch+0.5,0) -- (\i*\rdstretch+0.5,#1-1); +} +\newcommand{\rdat}[2]{ + (##2*\rdstretch+0.5,#1-##1) +}% +\newcommand{\round}[2]{% + \def\rdround{##1} + \ifthenelse{\equal{##2}{}}{}{ + \node[yshift=-1em] at ({##1*\rdstretch+0.5-0.5*\rdstretch},0) {##2}; + } +}% +\newcommand{\rdmessage}[3]{\draw[msg] + (\rdround*\rdstretch-\rdstretch+0.5,#1-##1) -- node[yshift=1.2ex]{##3} + (\rdround*\rdstretch+0.5,#1-##2);}% +\newcommand{\rdalltoall}{% + \foreach \i in {1,...,#1}{ + \foreach \j in {1,...,#1}{ + { \rdmessage{\i}{\j}{}}}}}% +}{% +\end{tikzpicture} +\end{center} +} diff --git a/cometbft/v0.39/spec/consensus/consensus-paper/technote.sty b/cometbft/v0.39/spec/consensus/consensus-paper/technote.sty new file mode 100644 index 000000000..5353f13cd --- /dev/null +++ b/cometbft/v0.39/spec/consensus/consensus-paper/technote.sty @@ -0,0 +1,118 @@ +\NeedsTeXFormat{LaTeX2e} +\ProvidesPackage{technote}[2007/11/09] +\typeout{Template for quick notes with some useful definitions} + +\RequirePackage{ifthen} +\RequirePackage{calc} +\RequirePackage{amsmath,amssymb,amsthm} +\RequirePackage{epsfig} +\RequirePackage{algorithm} +\RequirePackage[noend]{algorithmicplus} + +\newboolean{technote@noedit} +\setboolean{technote@noedit}{false} +\DeclareOption{noedit}{\setboolean{technote@noedit}{true}} + +\newcounter{technote@lang} +\setcounter{technote@lang}{0} +\DeclareOption{german}{\setcounter{technote@lang}{1}} +\DeclareOption{french}{\setcounter{technote@lang}{2}} + +\DeclareOption{fullpage}{ +\oddsidemargin -10mm % Margin on odd side pages (default=0mm) +\evensidemargin -10mm % Margin on even side pages (default=0mm) +\topmargin -10mm % Top margin space (default=16mm) +\headheight \baselineskip % Height of headers (default=0mm) +\headsep \baselineskip % Separation spc btw header and text (d=0mm) +\footskip 30pt % Separation spc btw text and footer (d=30pt) +\textheight 230mm % Total text height (default=200mm) +\textwidth 180mm % Total text width (default=160mm) +} + +\renewcommand{\algorithmiccomment}[1]{\hfill/* #1 */} +\renewcommand{\algorithmiclnosize}{\scriptsize} + +\newboolean{technote@truenumbers} +\setboolean{technote@truenumbers}{false} +\DeclareOption{truenumbers}{\setboolean{technote@truenumbers}{true}} + +\ProcessOptions + +\newcommand{\N}{\ifthenelse{\boolean{technote@truenumbers}}% + {\mbox{\rm I\hspace{-.5em}N}}% + {\mathbb{N}}} + +\newcommand{\R}{\ifthenelse{\boolean{technote@truenumbers}}% + {\mbox{\rm I\hspace{-.2em}R}}% + {\mathbb{R}}} + +\newcommand{\Z}{\mathbb{Z}} + +\newcommand{\set}[1]{\left\{#1\right\}} +\newcommand{\mathsc}[1]{\mbox{\sc #1}} +\newcommand{\li}[1]{\langle#1\rangle} +\newcommand{\st}{\;s.t.\;} +\newcommand{\Real}{\R} +\newcommand{\Natural}{\N} +\newcommand{\Integer}{\Z} + +% edit commands +\newcommand{\newedit}[2]{ + \newcommand{#1}[2][default]{% + \ifthenelse{\boolean{technote@noedit}}{}{ + \par\vspace{2mm} + \noindent + \begin{tabular}{|l|}\hline + \parbox{\linewidth-\tabcolsep*2}{{\bf #2:}\hfill\ifthenelse{\equal{##1}{default}}{}{##1}}\\\hline + \parbox{\linewidth-\tabcolsep*2}{\rule{0pt}{5mm}##2\rule[-2mm]{0pt}{2mm}}\\\hline + \end{tabular} + \par\vspace{2mm} + } + } +} + +\newedit{\note}{Note} +\newedit{\comment}{Comment} +\newedit{\question}{Question} +\newedit{\content}{Content} +\newedit{\problem}{Problem} + +\newcommand{\mnote}[1]{\marginpar{\scriptsize\it + \begin{minipage}[t]{0.8 in} + \raggedright #1 + \end{minipage}}} + +\newcommand{\Insert}[1]{\underline{#1}\marginpar{$|$}} + +\newcommand{\Delete}[1]{\marginpar{$|$} +} + +% lemma, theorem, etc. +\newtheorem{lemma}{Lemma} +\newtheorem{proposition}{Proposition} +\newtheorem{theorem}{Theorem} +\newtheorem{corollary}{Corollary} +\newtheorem{assumption}{Assumption} +\newtheorem{definition}{Definition} + +\gdef\op|{\,|\;} +\gdef\op:{\,:\;} +\newcommand{\assign}{\leftarrow} +\newcommand{\inc}[1]{#1 \assign #1 + 1} +\newcommand{\isdef}{:=} + +\newcommand{\ident}[1]{\mathit{#1}} +\def\newident#1{\expandafter\def\csname #1\endcsname{\ident{#1}}} + +\newcommand{\eg}{{\it e.g.}} +\newcommand{\ie}{{\it i.e.}} +\newcommand{\apriori}{{\it apriori}} +\newcommand{\etal}{{\it et al.}} + +\newcommand\ps@technote{% + \renewcommand\@oddhead{\theheader}% + \let\@evenhead\@oddhead + \renewcommand\@evenfoot + {\hfil\normalfont\textrm{\thepage}\hfil}% + \let\@oddfoot\@evenfoot +} diff --git a/cometbft/v0.39/spec/consensus/light-client/accountability.md b/cometbft/v0.39/spec/consensus/light-client/accountability.md new file mode 100644 index 000000000..3907e8d47 --- /dev/null +++ b/cometbft/v0.39/spec/consensus/light-client/accountability.md @@ -0,0 +1,3 @@ +# Fork accountability + +Deprecated, please see [light-client/accountability](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/accountability). diff --git a/cometbft/v0.39/spec/consensus/light-client/assets/light-node-image.png b/cometbft/v0.39/spec/consensus/light-client/assets/light-node-image.png new file mode 100644 index 000000000..f0b93c6e4 Binary files /dev/null and b/cometbft/v0.39/spec/consensus/light-client/assets/light-node-image.png differ diff --git a/cometbft/v0.39/spec/consensus/light-client/detection.md b/cometbft/v0.39/spec/consensus/light-client/detection.md new file mode 100644 index 000000000..9e70726c7 --- /dev/null +++ b/cometbft/v0.39/spec/consensus/light-client/detection.md @@ -0,0 +1,3 @@ +# Detection + +Deprecated, please see [light-client/detection](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/detection). diff --git a/cometbft/v0.39/spec/consensus/light-client/verification.md b/cometbft/v0.39/spec/consensus/light-client/verification.md new file mode 100644 index 000000000..d0e2bf1e5 --- /dev/null +++ b/cometbft/v0.39/spec/consensus/light-client/verification.md @@ -0,0 +1,3 @@ +# Core Verification + +Deprecated, please see [light-client/verification](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/verification). diff --git a/cometbft/v0.39/spec/consensus/proposer-based-timestamp/README.md b/cometbft/v0.39/spec/consensus/proposer-based-timestamp/README.md new file mode 100644 index 000000000..2972d8765 --- /dev/null +++ b/cometbft/v0.39/spec/consensus/proposer-based-timestamp/README.md @@ -0,0 +1,20 @@ +# Proposer-Based Timestamps + +This section describes a version of the Tendermint consensus algorithm, adopted in CometBFT, +which uses proposer-based timestamps. + +## Contents + +- [Proposer-Based Time][main] (entry point) +- [Part I - System Model and Properties][sysmodel] +- [Part II - Protocol Specification][algorithm] +- [TLA+ Specification][proposertla] + + +[algorithm]: ./pbts-algorithm_001_draft.md + +[sysmodel]: ./pbts-sysmodel_001_draft.md + +[main]: ./pbts_001_draft.md + +[proposertla]: ./tla/TendermintPBT_001_draft.tla diff --git a/cometbft/v0.39/spec/consensus/proposer-based-timestamp/pbts-algorithm_001_draft.md b/cometbft/v0.39/spec/consensus/proposer-based-timestamp/pbts-algorithm_001_draft.md new file mode 100644 index 000000000..ee8ca693d --- /dev/null +++ b/cometbft/v0.39/spec/consensus/proposer-based-timestamp/pbts-algorithm_001_draft.md @@ -0,0 +1,160 @@ +# Proposer-Based Time - Part II + +## Updated Consensus Algorithm + +### Outline + +The algorithm in the [arXiv paper][arXiv] evaluates rules of the received messages without making explicit how these messages are received. In our solution, we will make some message filtering explicit. We will assume that there are message reception steps (where messages are received and possibly stored locally for later evaluation of rules) and processing steps (the latter roughly as described in a way similar to the pseudo code of the arXiv paper). + +In contrast to the original algorithm the field `proposal` in the `PROPOSE` message is a pair `(v, time)`, of the proposed consensus value `v` and the proposed time `time`. + +#### **[PBTS-RECEPTION-STEP.0]** + +In the reception step at process `p` at local time `now_p`, upon receiving a message `m`: + +- if the message `m` is of type `PROPOSE` and satisfies `now_p - PRECISION < m.time < now_p + PRECISION + MSGDELAY`, then mark the message as `timely` + +> if `m` does not satisfy the constraint consider it `untimely` + + +#### **[PBTS-PROCESSING-STEP.0]** + +In the processing step, based on the messages stored, the rules of the algorithms are +executed. Note that the processing step only operates on messages +for the current height. The consensus algorithm rules are defined by the following updates to arXiv paper. + +#### New `StartRound` + +There are two additions + +- in case the proposer's local time is smaller than the time of the previous block, the proposer waits until this is not the case anymore (to ensure the block time is monotonically increasing) +- the proposer sends its time `now_p` as part of its proposal + +We update the timeout for the `PROPOSE` step according to the following reasoning: + +- If a correct proposer needs to wait to make sure its proposed time is larger than the `blockTime` of the previous block, then it sends by realtime `blockTime + ACCURACY` (By this time, its local clock must exceed `blockTime`) +- the receiver will receive a `PROPOSE` message by `blockTime + ACCURACY + MSGDELAY` +- the receiver's local clock will be `<= blockTime + 2 * ACCURACY + MSGDELAY` +- thus when the receiver `p` enters this round it can set its timeout to a value `waitingTime => blockTime + 2 * ACCURACY + MSGDELAY - now_p` + +So we should set the timeout to `max(timeoutPropose(round_p), waitingTime)`. + +> If, in the future, a block delay parameter `BLOCKDELAY` is introduced, this means +that the proposer should wait for `now_p > blockTime + BLOCKDELAY` before sending a `PROPOSE` message. +Also, `BLOCKDELAY` needs to be added to `waitingTime`. + +#### **[PBTS-ALG-STARTROUND.0]** + +```go +function StartRound(round) { + blockTime ← block time of block h_p - 1 + waitingTime ← blockTime + 2 * ACCURACY + MSGDELAY - now_p + round_p ← round + step_p ← propose + if proposer(h_p, round_p) = p { + wait until now_p > blockTime // new wait condition + if validValue_p != nil { + proposal ← (validValue_p, now_p) // added "now_p" + } + else { + proposal ← (getValue(), now_p) // added "now_p" + } + broadcast ⟨PROPOSAL, h_p, round_p, proposal, validRound_p⟩ + } + else { + schedule OnTimeoutPropose(h_p,round_p) to be executed after max(timeoutPropose(round_p), waitingTime) + } +} +``` + +#### New Rule Replacing Lines 22 - 27 + +- a validator prevotes for the consensus value `v` **and** the time `t` +- the code changes as the `PROPOSAL` message carries time (while `lockedValue` does not) + +#### **[PBTS-ALG-UPON-PROP.0]** + +```go +upon timely(⟨PROPOSAL, h_p, round_p, (v,t), −1⟩) from proposer(h_p, round_p) while step_p = propose do { + if valid(v) ∧ (lockedRound_p = −1 ∨ lockedValue_p = v) { + broadcast ⟨PREVOTE, h_p, round_p, id(v,t)⟩ + } + else { + broadcast ⟨PREVOTE, h_p, round_p, nil⟩ + } + step_p ← prevote +} +``` + +#### New Rule Replacing Lines 28 - 33 + +In case consensus is not reached in round 1, in `StartRound` the proposer of future rounds may propose the same value but with a different time. +Thus, the time `tprop` in the `PROPOSAL` message need not match the time `tvote` in the (old) `PREVOTE` messages. +A validator may send `PREVOTE` for the current round as long as the value `v` matches. +This gives the following rule: + +#### **[PBTS-ALG-OLD-PREVOTE.0]** + +```go +upon timely(⟨PROPOSAL, h_p, round_p, (v, tprop), vr⟩) from proposer(h_p, round_p) AND 2f + 1 ⟨PREVOTE, h_p, vr, id((v, tvote)⟩ +while step_p = propose ∧ (vr ≥ 0 ∧ vr < round_p) do { + if valid(v) ∧ (lockedRound_p ≤ vr ∨ lockedValue_p = v) { + broadcast ⟨PREVOTE, h_p, roundp, id(v, tprop)⟩ + } + else { + broadcast ⟨PREVOTE, hp, roundp, nil⟩ + } + step_p ← prevote +} +``` + +#### New Rule Replacing Lines 36 - 43 + +- As above, in the following `(v,t)` is part of the message rather than `v` +- the stored values (i.e., `lockedValue`, `validValue`) do not contain the time + +#### **[PBTS-ALG-NEW-PREVOTE.0]** + +```go +upon timely(⟨PROPOSAL, h_p, round_p, (v,t), ∗⟩) from proposer(h_p, round_p) AND 2f + 1 ⟨PREVOTE, h_p, round_p, id(v,t)⟩ while valid(v) ∧ step_p ≥ prevote for the first time do { + if step_p = prevote { + lockedValue_p ← v + lockedRound_p ← round_p + broadcast ⟨PRECOMMIT, h_p, round_p, id(v,t))⟩ + step_p ← precommit + } + validValue_p ← v + validRound_p ← round_p +} +``` + +#### New Rule Replacing Lines 49 - 54 + +- we decide on `v` as well as on the time from the proposal message +- here we do not care whether the proposal was received timely. + +> In particular we need to take care of the case where the proposer is untimely to one correct validator only. We need to ensure that this validator decides if all decide. + +#### **[PBTS-ALG-DECIDE.0]** + +```go +upon ⟨PROPOSAL, h_p, r, (v,t), ∗⟩ from proposer(h_p, r) AND 2f + 1 ⟨PRECOMMIT, h_p, r, id(v,t)⟩ while decisionp[h_p] = nil do { + if valid(v) { + decision_p [h_p] = (v,t) // decide on time too + h_p ← h_p + 1 + reset lockedRound_p , lockedValue_p, validRound_p and validValue_p to initial values and empty message log + StartRound(0) + } +} +``` + +**All other rules remains unchanged.** + +Back to [main document][main]. + +[main]: ./pbts_001_draft.md + +[arXiv]: https://arxiv.org/abs/1807.04938 + + + diff --git a/cometbft/v0.39/spec/consensus/proposer-based-timestamp/pbts-sysmodel_001_draft.md b/cometbft/v0.39/spec/consensus/proposer-based-timestamp/pbts-sysmodel_001_draft.md new file mode 100644 index 000000000..06f9e8ea5 --- /dev/null +++ b/cometbft/v0.39/spec/consensus/proposer-based-timestamp/pbts-sysmodel_001_draft.md @@ -0,0 +1,191 @@ +# Proposer-Based Time - Part I + +## System Model + +### Time and Clocks + +#### **[PBTS-CLOCK-NEWTON.0]** + +There is a reference Newtonian real-time `t` (UTC). + +Every correct validator `V` maintains a synchronized clock `C_V` that ensures: + +#### **[PBTS-CLOCK-PRECISION.0]** + +There exists a system parameter `PRECISION` such that for any two correct validators `V` and `W`, and at any real-time `t`, +`|C_V(t) - C_W(t)| < PRECISION` + + +### Message Delays + +We do not want to interfere with the timing assumptions of Tendermint consensus algorithm. +We will postulate a timing restriction, which, if satisfied, ensures that liveness is preserved. + +In general the local clock may drift from the global time. (It may progress faster, e.g., one second of clock time might take 1.005 seconds of real-time). As a result the local clock and the global clock may be measured in different time units. Usually, the message delay is measured in global clock time units. To estimate the correct local timeout precisely, we would need to estimate the clock time duration of a message delay taking into account the clock drift. For simplicity we ignore this, and directly postulate the message delay assumption in terms of local time. + + +#### **[PBTS-MSG-D.0]** + +There exists a system parameter `MSGDELAY` for message end-to-end delays **counted in clock-time**. + +> Observe that [PBTS-MSG-D.0] imposes constraints on message delays as well as on the clock. + +#### **[PBTS-MSG-FAIR.0]** + +The message end-to-end delay between a correct proposer and a correct validator (for `PROPOSE` messages) is less than `MSGDELAY`. + + +## Problem Statement + +In this section we define the properties of Tendermint consensus algorithm (cf. the [arXiv paper][arXiv]) in this new system model. + +#### **[PBTS-PROPOSE.0]** + +A proposer proposes a pair `(v,t)` of consensus value `v` and time `t`. + +> We then restrict the allowed decisions along the following lines: + +#### **[PBTS-INV-AGREEMENT.0]** + +[Agreement] No two correct validators decide on different values `v`. + +#### **[PBTS-INV-TIME-VAL.0]** + +[Time-Validity] If a correct validator decides on `t` then `t` is "OK" (we will formalize this below), even if up to `2f` validators are faulty. + +However, the properties of Tendermint consensus algorithm are of more interest with respect to the blocks, that is, what is written into a block and when. We therefore, in the following, will give the safety and liveness properties from this block-centric viewpoint. +For this, observe that the time `t` decided at consensus height `k` will be written in the block of height `k+1`, and will be supported by `2f + 1` `PRECOMMIT` messages of the same consensus round `r`. The time written in the block, we will denote by `b.time` (to distinguish it from the term `bfttime` used for median-based time). For this, it is important to have the following consensus algorithm property: + +#### **[PBTS-INV-TIME-AGR.0]** + +[Time-Agreement] If two correct validators decide in the same round, then they decide on the same `t`. + +#### **[PBTS-DECISION-ROUND.0]** + +Note that the relation between consensus decisions, on the one hand, and blocks, on the other hand, is not immediate; in particular if we consider time: In the proposed solution, +as validators may decide in different rounds, they may decide on different times. +The proposer of the next block, may pick a commit (at least `2f + 1` `PRECOMMIT` messages from one round), and thus it picks a decision round that is going to become "canonic". +As a result, the proposer implicitly has a choice of one of the times that belong to rounds in which validators decided. Observe that this choice was implicitly the case already in the median-based `bfttime`. +However, as most consensus instances terminate within one round on the Cosmos hub, this is hardly ever observed in practice. + + + +Finally, observe that the agreement ([Agreement] and [Time-Agreement]) properties are based on the Cosmos security model [CMBC-FM-2THIRDS.0][CMBC-FM-2THIRDS-link] of more than 2/3 correct validators, while [Time-Validity] is based on more than 1/3 correct validators. + +### SAFETY + +Here we will provide specifications that relate local time to block time. However, since we do not assume (by now) that local time is linked to real-time, these specifications also do not provide a relation between block time and real-time. Such properties are given [later](#real-time-safety). + +For a correct validator `V`, let `beginConsensus(V,k)` be the local time when it sets its height to `k`, and let `endConsensus(V,k)` be the time when it sets its height to `k + 1`. + +Let + +- `beginConsensus(k)` be the minimum over `beginConsensus(V,k)`, and +- `last-beginConsensus(k)` be the maximum over `beginConsensus(V,k)`, and +- `endConsensus(k)` the maximum over `endConsensus(V,k)` + +for all correct validators `V`. + +> Observe that `beginConsensus(k) <= last-beginConsensus(k)` and if local clocks are monotonic, then `last-beginConsensus(k) <= endConsensus(k)`. + +#### **[PBTS-CLOCK-GROW.0]** + +We assume that during one consensus instance, local clocks are not set back, in particular for each correct validator `V` and each height `k`, we have `beginConsensus(V,k) < endConsensus(V,k)`. + + +#### **[PBTS-CONSENSUS-TIME-VALID.0]** + +If + +- there is a valid commit `c` for height `k`, and +- `c` contains a `PRECOMMIT` message by at least one correct validator, + +then the time `b.time` in the block `b` that is signed by `c` satisfies + +- `beginConsensus(k) - PRECISION <= b.time < endConsensus(k) + PRECISION + MSGDELAY`. + + +> [PBTS-CONSENSUS-TIME-VALID.0] is based on an analysis where the proposer is faulty (and does does not count towards `beginConsensus(k)` and `endConsensus(k)`), and we estimate the times at which correct validators receive and `accept` the `propose` message. If the proposer is correct we obtain + +#### **[PBTS-CONSENSUS-LIVE-VALID-CORR-PROP.0]** + +If the proposer of round 1 is correct, and + +- [CMBC-FM-2THIRDS.0] holds for a block of height `k - 1`, and +- [PBTS-MSG-FAIR.0], and +- [PBTS-CLOCK-PRECISION.0], and +- [PBTS-CLOCK-GROW.0] (**TODO:** is that enough?) + +then eventually (within bounded time) every correct validator decides in round 1. + +#### **[PBTS-CONSENSUS-SAFE-VALID-CORR-PROP.0]** + +If the proposer of round 1 is correct, and + +- [CMBC-FM-2THIRDS.0] holds for a block of height `k - 1`, and +- [PBTS-MSG-FAIR.0], and +- [PBTS-CLOCK-PRECISION.0], and +- [PBTS-CLOCK-GROW.0] (**TODO:** is that enough?) + +then `beginConsensus_k <= b.time <= last-beginConsensus_k`. + + +> For the above two properties we will assume that a correct proposer `v` sends its `PROPOSAL` at its local time `beginConsensus(v,k)`. + +### LIVENESS + +If + +- [CMBC-FM-2THIRDS.0] holds for a block of height `k - 1`, and +- [PBTS-MSG-FAIR.0], +- [PBTS-CLOCK.0], and +- [PBTS-CLOCK-GROW.0] (**TODO:** is that enough?) + +then eventually there is a valid commit `c` for height `k`. + + +### REAL-TIME SAFETY + +> We want to give a property that can be exploited from the outside, that is, given a block with some time stored in it, what is the estimate at which real-time the block was generated. To do so, we need to link clock-time to real-time; which is not the case with [PBTS-CLOCK.0]. For this, we introduce the following assumption on the clocks: + +#### **[PBTS-CLOCKSYNC-EXTERNAL.0]** + +There is a system parameter `ACCURACY`, such that for all real-times `t` and all correct validators `V`, + +- `| C_V(t) - t | < ACCURACY`. + +> `ACCURACY` is not necessarily visible at the code level. The properties below just show that the smaller +its value, the closer the block time will be to real-time + +#### **[PBTS-CONSENSUS-PTIME.0]** + +LET `m` be a propose message. We consider the following two real-times `proposalTime(m)` and `propRecvTime(m)`: + +- if the proposer is correct and sends `m` at time `t`, we write `proposalTime(m)` for real-time `t`. +- if first correct validator receives `m` at time `t`, we write `propRecvTime(m)` for real-time `t`. + + +#### **[PBTS-CONSENSUS-REALTIME-VALID.0]** + +Let `b` be a block with a valid commit that contains at least one `precommit` message by a correct validator (and `proposalTime` is the time for the height/round `propose` message `m` that triggered the `precommit`). Then: + +`propRecvTime(m) - ACCURACY - PRECISION < b.time < propRecvTime(m) + ACCURACY + PRECISION + MSGDELAY` + + +#### **[PBTS-CONSENSUS-REALTIME-VALID-CORR.0]** + +Let `b` be a block with a valid commit that contains at least one `precommit` message by a correct validator (and `proposalTime` is the time for the height/round `propose` message `m` that triggered the `precommit`). Then, if the proposer is correct: + +`proposalTime(m) - ACCURACY < b.time < proposalTime(m) + ACCURACY` + +> by the algorithm at time `proposalTime(m)` the proposer fixes `m.time <- now_p(proposalTime(m))` + +> "triggered the `PRECOMMIT`" implies that the data in `m` and `b` are "matching", that is, `m` proposed the values that are actually stored in `b`. + +Back to [main document][main]. + +[main]: ./pbts_001_draft.md + +[arXiv]: https://arxiv.org/abs/1807.04938 + +[CMBC-FM-2THIRDS-link]: https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/verification/verification_002_draft.md#cmbc-fm-2thirds1 diff --git a/cometbft/v0.39/spec/consensus/proposer-based-timestamp/pbts_001_draft.md b/cometbft/v0.39/spec/consensus/proposer-based-timestamp/pbts_001_draft.md new file mode 100644 index 000000000..f71d7ab80 --- /dev/null +++ b/cometbft/v0.39/spec/consensus/proposer-based-timestamp/pbts_001_draft.md @@ -0,0 +1,269 @@ +# Proposer-Based Time + +## Current BFTTime + +### Description + +In CometBFT, the first version of how time is computed and stored in a block works as follows: + +- validators send their current local time as part of `precommit` messages +- upon collecting the `precommit` messages that the proposer uses to build a commit to be put in the next block, the proposer computes the `time` of the next block as the median (weighted over voting power) of the times in the `precommit` messages. + +### Analysis + +1. **Fault tolerance.** The computed median time is called [`bfttime`][bfttime] as it is indeed fault-tolerant: if **less than a third** of the validators is faulty (counted in voting power), it is guaranteed that the computed time lies between the minimum and the maximum times sent by correct validators. +1. **Effect of faulty validators.** If more than `1/2` of the voting power (which is in fact more than one third and less than two thirds of the voting power) is held by faulty validators, then the time is under total control of the faulty validators. (This is particularly challenging in the context of [lightclient][lcspec] security.) +1. **Proposer influence on block time.** The proposer of the next block has a degree of freedom in choosing the `bfttime`, since it computes the median time based on the timestamps from `precommit` messages sent by + `2f + 1` correct validators. + 1. If there are `n` different timestamps in the `precommit` messages, the proposer can use any subset of timestamps that add up to `2f + 1` + of the voting power in order to compute the median. + 1. If the validators decide in different rounds, the proposer can decide on which round the median computation is based. +1. **Liveness.** The liveness of the protocol: + 1. does not depend on clock synchronization, + 1. depends on bounded message delays. +1. **Relation to real time.** There is no clock synchronizaton, which implies that there is **no relation** between the computed block `time` and real time. +1. **Aggregate signatures.** As the `precommit` messages contain the local times, all these `precommit` messages typically differ in the time field, which **prevents** the use of aggregate signatures. + +## Suggested Proposer-Based Time + +### Outline + +An alternative approach to time has been discussed: Rather than having the validators send the time in the `precommit` messages, the proposer in the consensus algorithm sends its time in the `propose` message, and the validators locally check whether the time is OK (by comparing to their local clock). + +This proposed solution adds the requirement of having synchronized clocks, and other implicit assumptions. + +### Comparison of the Suggested Method to the Old One + +1. **Fault tolerance.** Maintained in the suggested protocol. +1. **Effect of faulty validators.** Eliminated in the suggested protocol, + that is, the block `time` can be corrupted only in the extreme case when + `>2/3` of the validators are faulty. +1. **Proposer influence on block time.** The proposer of the next block + has less freedom when choosing the block time. + 1. This scenario is eliminated in the suggested protocol, provided that there are `<1/3` faulty validators. + 1. This scenario is still there. +1. **Liveness.** The liveness of the suggested protocol: + 1. depends on the introduced assumptions on synchronized clocks (see below), + 1. still depends on the message delays (unavoidable). +1. **Relation to real time.** We formalize clock synchronization, and obtain a **well-defined relation** between the block `time` and real time. +1. **Aggregate signatures.** The `precommit` messages free of time, which **allows** for aggregate signatures. + +### Protocol Overview + +#### Proposed Time + +We assume that the field `proposal` in the `PROPOSE` message is a pair `(v, time)`, of the proposed consensus value `v` and the proposed time `time`. + +#### Reception Step + +In the reception step at node `p` at local time `now_p`, upon receiving a message `m`: + +- **if** the message `m` is of type `PROPOSE` and satisfies `now_p - PRECISION < m.time < now_p + PRECISION + MSGDELAY`, then mark the message as `timely`. +(`PRECISION` and `MSGDELAY` being system parameters, see [below](#safety-and-liveness)) + +> after the presentation in the dev session, we realized that different semantics for the reception step is closer aligned to the implementation. Instead of dropping propose messages, we keep all of them, and mark timely ones. + +#### Processing Step + +- Start round + + + + + + + + + + + + +
arXiv paperProposer-based time
+ +```go +function StartRound(round) { + round_p ← round + step_p ← propose + if proposer(h_p, round_p) = p { + + + if validValue_p != nil { + + proposal ← validValue_p + } else { + + proposal ← getValue() + } + broadcast ⟨PROPOSAL, h_p, round_p, proposal, validRound_p⟩ + } else { + schedule OnTimeoutPropose(h_p,round_p) to + be executed after timeoutPropose(round_p) + } +} +``` + + + +```go +function StartRound(round) { + round_p ← round + step_p ← propose + if proposer(h_p, round_p) = p { + // new wait condition + wait until now_p > block time of block h_p - 1 + if validValue_p != nil { + // add "now_p" + proposal ← (validValue_p, now_p) + } else { + // add "now_p" + proposal ← (getValue(), now_p) + } + broadcast ⟨PROPOSAL, h_p, round_p, proposal, validRound_p⟩ + } else { + schedule OnTimeoutPropose(h_p,round_p) to + be executed after timeoutPropose(round_p) + } +} +``` + +
+ +- Rule on lines 28-35 + + + + + + + + + + + + +
arXiv paperProposer-based time
+ +```go +upon timely(⟨PROPOSAL, h_p, round_p, v, vr⟩) + from proposer(h_p, round_p) + AND 2f + 1 ⟨PREVOTE, h_p, vr, id(v)⟩ +while step_p = propose ∧ (vr ≥ 0 ∧ vr < round_p) do { + if valid(v) ∧ (lockedRound_p ≤ vr ∨ lockedValue_p = v) { + + broadcast ⟨PREVOTE, h_p, round_p, id(v)⟩ + } else { + broadcast ⟨PREVOTE, hp, round_p, nil⟩ + } +} +``` + + + +```go +upon timely(⟨PROPOSAL, h_p, round_p, (v, tprop), vr⟩) + from proposer(h_p, round_p) + AND 2f + 1 ⟨PREVOTE, h_p, vr, id(v, tvote)⟩ + while step_p = propose ∧ (vr ≥ 0 ∧ vr < round_p) do { + if valid(v) ∧ (lockedRound_p ≤ vr ∨ lockedValue_p = v) { + // send hash of v and tprop in PREVOTE message + broadcast ⟨PREVOTE, h_p, round_p, id(v, tprop)⟩ + } else { + broadcast ⟨PREVOTE, hp, round_p, nil⟩ + } + } +``` + +
+ +- Rule on lines 49-54 + + + + + + + + + + + + +
arXiv paperProposer-based time
+ +```go +upon ⟨PROPOSAL, h_p, r, v, ∗⟩ from proposer(h_p, r) + AND 2f + 1 ⟨PRECOMMIT, h_p, r, id(v)⟩ + while decisionp[h_p] = nil do { + if valid(v) { + + decision_p [h_p] = v + h_p ← h_p + 1 + reset lockedRound_p , lockedValue_p, validRound_p and + validValue_p to initial values and empty message log + StartRound(0) + } + } +``` + + + +```go +upon ⟨PROPOSAL, h_p, r, (v,t), ∗⟩ from proposer(h_p, r) + AND 2f + 1 ⟨PRECOMMIT, h_p, r, id(v,t)⟩ + while decisionp[h_p] = nil do { + if valid(v) { + // decide on time too + decision_p [h_p] = (v,t) + h_p ← h_p + 1 + reset lockedRound_p , lockedValue_p, validRound_p and + validValue_p to initial values and empty message log + StartRound(0) + } + } +``` + +
+ +- Other rules are extended in a similar way, or remain unchanged + +### Property Overview + +#### Safety and Liveness + +For safety (Point 1, Point 2, Point 3i) and liveness (Point 4) we need +the following assumptions: + +- There exists a system parameter `PRECISION` such that for any two correct validators `V` and `W`, and at any real-time `t`, their local times `C_V(t)` and `C_W(t)` differ by less than `PRECISION` time units, +i.e., `|C_V(t) - C_W(t)| < PRECISION` +- The message end-to-end delay between a correct proposer and a correct validator (for `PROPOSE` messages) is less than `MSGDELAY`. + +#### Relation to Real-Time + +For analyzing real-time safety (Point 5), we use a system parameter `ACCURACY`, such that for all real-times `t` and all correct validators `V`, we have `| C_V(t) - t | < ACCURACY`. + +> `ACCURACY` is not necessarily visible at the code level. We might even view `ACCURACY` as variable over time. The smaller it is during a consensus instance, the closer the block time will be to real-time. +> +> Note that `PRECISION` and `MSGDELAY` show up in the code. + +### Detailed Specification + +This specification describes the changes needed to be done to the Tendermint consensus algorithm as described in the [arXiv paper][arXiv] and the simplified specification in [TLA+][tlatender], and makes precise the underlying assumptions and the required properties. + +- [Part I - System Model and Properties][sysmodel] +- [Part II - Protocol specification][algorithm] +- [TLA+ Specification][proposertla] + +[arXiv]: https://arxiv.org/abs/1807.04938 + +[tlatender]: ../../light-client/accountability/README.md + +[bfttime]: ../bft-time.md + +[lcspec]: ../../light-client/README.md + +[algorithm]: ./pbts-algorithm_001_draft.md + +[sysmodel]: ./pbts-sysmodel_001_draft.md + + +[proposertla]: ./tla/TendermintPBT_001_draft.tla diff --git a/cometbft/v0.39/spec/consensus/proposer-based-timestamp/tla/TendermintPBT_001_draft.tla b/cometbft/v0.39/spec/consensus/proposer-based-timestamp/tla/TendermintPBT_001_draft.tla new file mode 100644 index 000000000..d8524540f --- /dev/null +++ b/cometbft/v0.39/spec/consensus/proposer-based-timestamp/tla/TendermintPBT_001_draft.tla @@ -0,0 +1,597 @@ +-------------------- MODULE TendermintPBT_001_draft --------------------------- +(* + A TLA+ specification of a simplified Tendermint consensus algorithm, with added clocks + and proposer-based timestamps. This TLA+ specification extends and modifies + the Tendermint TLA+ specification for fork accountability: + https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/accountability/TendermintAcc_004_draft.tla + + * Version 1. A preliminary specification. + + Zarko Milosevic, Igor Konnov, Informal Systems, 2019-2020. + Ilina Stoilkovska, Josef Widder, Informal Systems, 2021. + *) + +EXTENDS Integers, FiniteSets + +(********************* PROTOCOL PARAMETERS **********************************) +CONSTANTS + Corr, \* the set of correct processes + Faulty, \* the set of Byzantine processes, may be empty + N, \* the total number of processes: correct, defective, and Byzantine + T, \* an upper bound on the number of Byzantine processes + ValidValues, \* the set of valid values, proposed both by correct and faulty + InvalidValues, \* the set of invalid values, never proposed by the correct ones + MaxRound, \* the maximal round number + MaxTimestamp, \* the maximal value of the clock tick + Delay, \* message delay + Precision, \* clock precision: the maximal difference between two local clocks + Accuracy, \* clock accuracy: the maximal difference between a local clock and the real time + Proposer, \* the proposer function from 0..NRounds to 1..N + ClockDrift \* is there clock drift between the local clocks and the global clock + +ASSUME(N = Cardinality(Corr \union Faulty)) + +(*************************** DEFINITIONS ************************************) +AllProcs == Corr \union Faulty \* the set of all processes +Rounds == 0..MaxRound \* the set of potential rounds +Timestamps == 0..MaxTimestamp \* the set of clock ticks +NilRound == -1 \* a special value to denote a nil round, outside of Rounds +NilTimestamp == -1 \* a special value to denote a nil timestamp, outside of Ticks +RoundsOrNil == Rounds \union {NilRound} +Values == ValidValues \union InvalidValues \* the set of all values +NilValue == "None" \* a special value for a nil round, outside of Values +Proposals == Values \X Timestamps +NilProposal == <> +ValuesOrNil == Values \union {NilValue} +Decisions == Values \X Timestamps \X Rounds +NilDecision == <> + + +\* a value hash is modeled as identity +Id(v) == v + +\* The validity predicate +IsValid(v) == v \in ValidValues + +\* the two thresholds that are used in the algorithm +THRESHOLD1 == T + 1 \* at least one process is not faulty +THRESHOLD2 == 2 * T + 1 \* a quorum when having N > 3 * T + +Min(S) == CHOOSE x \in S : \A y \in S : x <= y + +Max(S) == CHOOSE x \in S : \A y \in S : y <= x + +(********************* TYPE ANNOTATIONS FOR APALACHE **************************) +\* the operator for type annotations +a <: b == a + +\* the type of message records +MT == [type |-> STRING, src |-> STRING, round |-> Int, + proposal |-> <>, validRound |-> Int, id |-> <>] + +RP == <> + +\* a type annotation for a message +AsMsg(m) == m <: MT +\* a type annotation for a set of messages +SetOfMsgs(S) == S <: {MT} +\* a type annotation for an empty set of messages +EmptyMsgSet == SetOfMsgs({}) + +SetOfRcvProp(S) == S <: {RP} +EmptyRcvProp == SetOfRcvProp({}) + +SetOfProc(S) == S <: {STRING} +EmptyProcSet == SetOfProc({}) + +(********************* PROTOCOL STATE VARIABLES ******************************) +VARIABLES + round, \* a process round number: Corr -> Rounds + localClock, \* a process local clock: Corr -> Ticks + realTime, \* a reference Newtonian real time + step, \* a process step: Corr -> { "PROPOSE", "PREVOTE", "PRECOMMIT", "DECIDED" } + decision, \* process decision: Corr -> ValuesOrNil + lockedValue, \* a locked value: Corr -> ValuesOrNil + lockedRound, \* a locked round: Corr -> RoundsOrNil + validValue, \* a valid value: Corr -> ValuesOrNil + validRound \* a valid round: Corr -> RoundsOrNil + +\* book-keeping variables +VARIABLES + msgsPropose, \* PROPOSE messages broadcast in the system, Rounds -> Messages + msgsPrevote, \* PREVOTE messages broadcast in the system, Rounds -> Messages + msgsPrecommit, \* PRECOMMIT messages broadcast in the system, Rounds -> Messages + receivedTimelyProposal, \* used to keep track when a process receives a timely PROPOSAL message, {<>} + inspectedProposal, \* used to keep track when a process tries to receive a message, [Rounds -> <>] + evidence, \* the messages that were used by the correct processes to make transitions + action, \* we use this variable to see which action was taken + beginConsensus, \* the minimum of the local clocks in the initial state, Int + endConsensus, \* the local time when a decision is made, [Corr -> Int] + lastBeginConsensus, \* the maximum of the local clocks in the initial state, Int + proposalTime, \* the real time when a proposer proposes in a round, [Rounds -> Int] + proposalReceivedTime \* the real time when a correct process first receives a proposal message in a round, [Rounds -> Int] + +(* to see a type invariant, check TendermintAccInv3 *) + +\* a handy definition used in UNCHANGED +vars == <> + +(********************* PROTOCOL INITIALIZATION ******************************) +FaultyProposals(r) == + SetOfMsgs([type: {"PROPOSAL"}, src: Faulty, + round: {r}, proposal: Proposals, validRound: RoundsOrNil]) + +AllFaultyProposals == + SetOfMsgs([type: {"PROPOSAL"}, src: Faulty, + round: Rounds, proposal: Proposals, validRound: RoundsOrNil]) + +FaultyPrevotes(r) == + SetOfMsgs([type: {"PREVOTE"}, src: Faulty, round: {r}, id: Proposals]) + +AllFaultyPrevotes == + SetOfMsgs([type: {"PREVOTE"}, src: Faulty, round: Rounds, id: Proposals]) + +FaultyPrecommits(r) == + SetOfMsgs([type: {"PRECOMMIT"}, src: Faulty, round: {r}, id: Proposals]) + +AllFaultyPrecommits == + SetOfMsgs([type: {"PRECOMMIT"}, src: Faulty, round: Rounds, id: Proposals]) + +AllProposals == + SetOfMsgs([type: {"PROPOSAL"}, src: AllProcs, + round: Rounds, proposal: Proposals, validRound: RoundsOrNil]) + +RoundProposals(r) == + SetOfMsgs([type: {"PROPOSAL"}, src: AllProcs, + round: {r}, proposal: Proposals, validRound: RoundsOrNil]) + +BenignRoundsInMessages(msgfun) == + \* the message function never contains a message for a wrong round + \A r \in Rounds: + \A m \in msgfun[r]: + r = m.round + +\* The initial states of the protocol. Some faults can be in the system already. +Init == + /\ round = [p \in Corr |-> 0] + /\ \/ /\ ~ClockDrift + /\ localClock \in [Corr -> 0..Accuracy] + \/ /\ ClockDrift + /\ localClock = [p \in Corr |-> 0] + /\ realTime = 0 + /\ step = [p \in Corr |-> "PROPOSE"] + /\ decision = [p \in Corr |-> NilDecision] + /\ lockedValue = [p \in Corr |-> NilValue] + /\ lockedRound = [p \in Corr |-> NilRound] + /\ validValue = [p \in Corr |-> NilValue] + /\ validRound = [p \in Corr |-> NilRound] + /\ msgsPropose \in [Rounds -> SUBSET AllFaultyProposals] + /\ msgsPrevote \in [Rounds -> SUBSET AllFaultyPrevotes] + /\ msgsPrecommit \in [Rounds -> SUBSET AllFaultyPrecommits] + /\ receivedTimelyProposal = EmptyRcvProp + /\ inspectedProposal = [r \in Rounds |-> EmptyProcSet] + /\ BenignRoundsInMessages(msgsPropose) + /\ BenignRoundsInMessages(msgsPrevote) + /\ BenignRoundsInMessages(msgsPrecommit) + /\ evidence = EmptyMsgSet + /\ action' = "Init" + /\ beginConsensus = Min({localClock[p] : p \in Corr}) + /\ endConsensus = [p \in Corr |-> NilTimestamp] + /\ lastBeginConsensus = Max({localClock[p] : p \in Corr}) + /\ proposalTime = [r \in Rounds |-> NilTimestamp] + /\ proposalReceivedTime = [r \in Rounds |-> NilTimestamp] + +(************************ MESSAGE PASSING ********************************) +BroadcastProposal(pSrc, pRound, pProposal, pValidRound) == + LET newMsg == + AsMsg([type |-> "PROPOSAL", src |-> pSrc, round |-> pRound, + proposal |-> pProposal, validRound |-> pValidRound]) + IN + msgsPropose' = [msgsPropose EXCEPT ![pRound] = msgsPropose[pRound] \union {newMsg}] + +BroadcastPrevote(pSrc, pRound, pId) == + LET newMsg == AsMsg([type |-> "PREVOTE", + src |-> pSrc, round |-> pRound, id |-> pId]) + IN + msgsPrevote' = [msgsPrevote EXCEPT ![pRound] = msgsPrevote[pRound] \union {newMsg}] + +BroadcastPrecommit(pSrc, pRound, pId) == + LET newMsg == AsMsg([type |-> "PRECOMMIT", + src |-> pSrc, round |-> pRound, id |-> pId]) + IN + msgsPrecommit' = [msgsPrecommit EXCEPT ![pRound] = msgsPrecommit[pRound] \union {newMsg}] + + +(***************************** TIME **************************************) + +\* [PBTS-CLOCK-PRECISION.0] +SynchronizedLocalClocks == + \A p \in Corr : \A q \in Corr : + p /= q => + \/ /\ localClock[p] >= localClock[q] + /\ localClock[p] - localClock[q] < Precision + \/ /\ localClock[p] < localClock[q] + /\ localClock[q] - localClock[p] < Precision + +\* [PBTS-PROPOSE.0] +Proposal(v, t) == + <> + +\* [PBTS-DECISION-ROUND.0] +Decision(v, t, r) == + <> + +(**************** MESSAGE PROCESSING TRANSITIONS *************************) +\* lines 12-13 +StartRound(p, r) == + /\ step[p] /= "DECIDED" \* a decided process does not participate in consensus + /\ round' = [round EXCEPT ![p] = r] + /\ step' = [step EXCEPT ![p] = "PROPOSE"] + +\* lines 14-19, a proposal may be sent later +InsertProposal(p) == + LET r == round[p] IN + /\ p = Proposer[r] + /\ step[p] = "PROPOSE" + \* if the proposer is sending a proposal, then there are no other proposals + \* by the correct processes for the same round + /\ \A m \in msgsPropose[r]: m.src /= p + /\ \E v \in ValidValues: + LET proposal == IF validValue[p] /= NilValue + THEN Proposal(validValue[p], localClock[p]) + ELSE Proposal(v, localClock[p]) IN + + /\ BroadcastProposal(p, round[p], proposal, validRound[p]) + /\ proposalTime' = [proposalTime EXCEPT ![r] = realTime] + /\ UNCHANGED <> + /\ action' = "InsertProposal" + +\* a new action used to filter messages that are not on time +\* [PBTS-RECEPTION-STEP.0] +ReceiveProposal(p) == + \E v \in Values, t \in Timestamps: + /\ LET r == round[p] IN + LET msg == + AsMsg([type |-> "PROPOSAL", src |-> Proposer[round[p]], + round |-> round[p], proposal |-> Proposal(v, t), validRound |-> NilRound]) IN + /\ msg \in msgsPropose[round[p]] + /\ p \notin inspectedProposal[r] + /\ <> \notin receivedTimelyProposal + /\ inspectedProposal' = [inspectedProposal EXCEPT ![r] = @ \union {p}] + /\ \/ /\ localClock[p] - Precision < t + /\ t < localClock[p] + Precision + Delay + /\ receivedTimelyProposal' = receivedTimelyProposal \union {<>} + /\ \/ /\ proposalReceivedTime[r] = NilTimestamp + /\ proposalReceivedTime' = [proposalReceivedTime EXCEPT ![r] = realTime] + \/ /\ proposalReceivedTime[r] /= NilTimestamp + /\ UNCHANGED proposalReceivedTime + \/ /\ \/ localClock[p] - Precision >= t + \/ t >= localClock[p] + Precision + Delay + /\ UNCHANGED <> + /\ UNCHANGED <> + /\ action' = "ReceiveProposal" + +\* lines 22-27 +UponProposalInPropose(p) == + \E v \in Values, t \in Timestamps: + /\ step[p] = "PROPOSE" (* line 22 *) + /\ LET msg == + AsMsg([type |-> "PROPOSAL", src |-> Proposer[round[p]], + round |-> round[p], proposal |-> Proposal(v, t), validRound |-> NilRound]) IN + /\ <> \in receivedTimelyProposal \* updated line 22 + /\ evidence' = {msg} \union evidence + /\ LET mid == (* line 23 *) + IF IsValid(v) /\ (lockedRound[p] = NilRound \/ lockedValue[p] = v) + THEN Id(Proposal(v, t)) + ELSE NilProposal + IN + BroadcastPrevote(p, round[p], mid) \* lines 24-26 + /\ step' = [step EXCEPT ![p] = "PREVOTE"] + /\ UNCHANGED <> + /\ action' = "UponProposalInPropose" + +\* lines 28-33 +\* [PBTS-ALG-OLD-PREVOTE.0] +UponProposalInProposeAndPrevote(p) == + \E v \in Values, t1 \in Timestamps, t2 \in Timestamps, vr \in Rounds: + /\ step[p] = "PROPOSE" /\ 0 <= vr /\ vr < round[p] \* line 28, the while part + /\ LET msg == + AsMsg([type |-> "PROPOSAL", src |-> Proposer[round[p]], + round |-> round[p], proposal |-> Proposal(v, t1), validRound |-> vr]) + IN + /\ <> \in receivedTimelyProposal \* updated line 28 + /\ LET PV == { m \in msgsPrevote[vr]: m.id = Id(Proposal(v, t2)) } IN + /\ Cardinality(PV) >= THRESHOLD2 \* line 28 + /\ evidence' = PV \union {msg} \union evidence + /\ LET mid == (* line 29 *) + IF IsValid(v) /\ (lockedRound[p] <= vr \/ lockedValue[p] = v) + THEN Id(Proposal(v, t1)) + ELSE NilProposal + IN + BroadcastPrevote(p, round[p], mid) \* lines 24-26 + /\ step' = [step EXCEPT ![p] = "PREVOTE"] + /\ UNCHANGED <> + /\ action' = "UponProposalInProposeAndPrevote" + + \* lines 34-35 + lines 61-64 (onTimeoutPrevote) +UponQuorumOfPrevotesAny(p) == + /\ step[p] = "PREVOTE" \* line 34 and 61 + /\ \E MyEvidence \in SUBSET msgsPrevote[round[p]]: + \* find the unique voters in the evidence + LET Voters == { m.src: m \in MyEvidence } IN + \* compare the number of the unique voters against the threshold + /\ Cardinality(Voters) >= THRESHOLD2 \* line 34 + /\ evidence' = MyEvidence \union evidence + /\ BroadcastPrecommit(p, round[p], NilProposal) + /\ step' = [step EXCEPT ![p] = "PRECOMMIT"] + /\ UNCHANGED <> + /\ action' = "UponQuorumOfPrevotesAny" + +\* lines 36-46 +\* [PBTS-ALG-NEW-PREVOTE.0] +UponProposalInPrevoteOrCommitAndPrevote(p) == + \E v \in ValidValues, t \in Timestamps, vr \in RoundsOrNil: + /\ step[p] \in {"PREVOTE", "PRECOMMIT"} \* line 36 + /\ LET msg == + AsMsg([type |-> "PROPOSAL", src |-> Proposer[round[p]], + round |-> round[p], proposal |-> Proposal(v, t), validRound |-> vr]) IN + /\ <> \in receivedTimelyProposal \* updated line 36 + /\ LET PV == { m \in msgsPrevote[round[p]]: m.id = Id(Proposal(v, t)) } IN + /\ Cardinality(PV) >= THRESHOLD2 \* line 36 + /\ evidence' = PV \union {msg} \union evidence + /\ IF step[p] = "PREVOTE" + THEN \* lines 38-41: + /\ lockedValue' = [lockedValue EXCEPT ![p] = v] + /\ lockedRound' = [lockedRound EXCEPT ![p] = round[p]] + /\ BroadcastPrecommit(p, round[p], Id(Proposal(v, t))) + /\ step' = [step EXCEPT ![p] = "PRECOMMIT"] + ELSE + UNCHANGED <> + \* lines 42-43 + /\ validValue' = [validValue EXCEPT ![p] = v] + /\ validRound' = [validRound EXCEPT ![p] = round[p]] + /\ UNCHANGED <> + /\ action' = "UponProposalInPrevoteOrCommitAndPrevote" + +\* lines 47-48 + 65-67 (onTimeoutPrecommit) +UponQuorumOfPrecommitsAny(p) == + /\ \E MyEvidence \in SUBSET msgsPrecommit[round[p]]: + \* find the unique committers in the evidence + LET Committers == { m.src: m \in MyEvidence } IN + \* compare the number of the unique committers against the threshold + /\ Cardinality(Committers) >= THRESHOLD2 \* line 47 + /\ evidence' = MyEvidence \union evidence + /\ round[p] + 1 \in Rounds + /\ StartRound(p, round[p] + 1) + /\ UNCHANGED <> + /\ action' = "UponQuorumOfPrecommitsAny" + +\* lines 49-54 +\* [PBTS-ALG-DECIDE.0] +UponProposalInPrecommitNoDecision(p) == + /\ decision[p] = NilDecision \* line 49 + /\ \E v \in ValidValues, t \in Timestamps (* line 50*) , r \in Rounds, vr \in RoundsOrNil: + /\ LET msg == AsMsg([type |-> "PROPOSAL", src |-> Proposer[r], + round |-> r, proposal |-> Proposal(v, t), validRound |-> vr]) IN + /\ msg \in msgsPropose[r] \* line 49 + /\ p \in inspectedProposal[r] + /\ LET PV == { m \in msgsPrecommit[r]: m.id = Id(Proposal(v, t)) } IN + /\ Cardinality(PV) >= THRESHOLD2 \* line 49 + /\ evidence' = PV \union {msg} \union evidence + /\ decision' = [decision EXCEPT ![p] = Decision(v, t, round[p])] \* update the decision, line 51 + \* The original algorithm does not have 'DECIDED', but it increments the height. + \* We introduced 'DECIDED' here to prevent the process from changing its decision. + /\ endConsensus' = [endConsensus EXCEPT ![p] = localClock[p]] + /\ step' = [step EXCEPT ![p] = "DECIDED"] + /\ UNCHANGED <> + /\ action' = "UponProposalInPrecommitNoDecision" + +\* the actions below are not essential for safety, but added for completeness + +\* lines 20-21 + 57-60 +OnTimeoutPropose(p) == + /\ step[p] = "PROPOSE" + /\ p /= Proposer[round[p]] + /\ BroadcastPrevote(p, round[p], NilProposal) + /\ step' = [step EXCEPT ![p] = "PREVOTE"] + /\ UNCHANGED <> + /\ action' = "OnTimeoutPropose" + +\* lines 44-46 +OnQuorumOfNilPrevotes(p) == + /\ step[p] = "PREVOTE" + /\ LET PV == { m \in msgsPrevote[round[p]]: m.id = Id(NilProposal) } IN + /\ Cardinality(PV) >= THRESHOLD2 \* line 36 + /\ evidence' = PV \union evidence + /\ BroadcastPrecommit(p, round[p], Id(NilProposal)) + /\ step' = [step EXCEPT ![p] = "PRECOMMIT"] + /\ UNCHANGED <> + /\ action' = "OnQuorumOfNilPrevotes" + +\* lines 55-56 +OnRoundCatchup(p) == + \E r \in {rr \in Rounds: rr > round[p]}: + LET RoundMsgs == msgsPropose[r] \union msgsPrevote[r] \union msgsPrecommit[r] IN + \E MyEvidence \in SUBSET RoundMsgs: + LET Faster == { m.src: m \in MyEvidence } IN + /\ Cardinality(Faster) >= THRESHOLD1 + /\ evidence' = MyEvidence \union evidence + /\ StartRound(p, r) + /\ UNCHANGED <> + /\ action' = "OnRoundCatchup" + + +(********************* PROTOCOL TRANSITIONS ******************************) +\* advance the global clock +AdvanceRealTime == + /\ realTime < MaxTimestamp + /\ realTime' = realTime + 1 + /\ \/ /\ ~ClockDrift + /\ localClock' = [p \in Corr |-> localClock[p] + 1] + \/ /\ ClockDrift + /\ UNCHANGED localClock + /\ UNCHANGED <> + /\ action' = "AdvanceRealTime" + +\* advance the local clock of node p +AdvanceLocalClock(p) == + /\ localClock[p] < MaxTimestamp + /\ localClock' = [localClock EXCEPT ![p] = @ + 1] + /\ UNCHANGED <> + /\ action' = "AdvanceLocalClock" + +\* process timely messages +MessageProcessing(p) == + \* start round + \/ InsertProposal(p) + \* reception step + \/ ReceiveProposal(p) + \* processing step + \/ UponProposalInPropose(p) + \/ UponProposalInProposeAndPrevote(p) + \/ UponQuorumOfPrevotesAny(p) + \/ UponProposalInPrevoteOrCommitAndPrevote(p) + \/ UponQuorumOfPrecommitsAny(p) + \/ UponProposalInPrecommitNoDecision(p) + \* the actions below are not essential for safety, but added for completeness + \/ OnTimeoutPropose(p) + \/ OnQuorumOfNilPrevotes(p) + \/ OnRoundCatchup(p) + +(* + * A system transition. In this specificatiom, the system may eventually deadlock, + * e.g., when all processes decide. This is expected behavior, as we focus on safety. + *) +Next == + \/ AdvanceRealTime + \/ /\ ClockDrift + /\ \E p \in Corr: AdvanceLocalClock(p) + \/ /\ SynchronizedLocalClocks + /\ \E p \in Corr: MessageProcessing(p) + +----------------------------------------------------------------------------- + +(*************************** INVARIANTS *************************************) + +\* [PBTS-INV-AGREEMENT.0] +AgreementOnValue == + \A p, q \in Corr: + /\ decision[p] /= NilDecision + /\ decision[q] /= NilDecision + => \E v \in ValidValues, t1 \in Timestamps, t2 \in Timestamps, r1 \in Rounds, r2 \in Rounds : + /\ decision[p] = Decision(v, t1, r1) + /\ decision[q] = Decision(v, t2, r2) + +\* [PBTS-INV-TIME-AGR.0] +AgreementOnTime == + \A p, q \in Corr: + \A v1 \in ValidValues, v2 \in ValidValues, t1 \in Timestamps, t2 \in Timestamps, r \in Rounds : + /\ decision[p] = Decision(v1, t1, r) + /\ decision[q] = Decision(v2, t2, r) + => t1 = t2 + +\* [PBTS-CONSENSUS-TIME-VALID.0] +ConsensusTimeValid == + \A p \in Corr, t \in Timestamps : + \* if a process decides on v and t + (\E v \in ValidValues, r \in Rounds : decision[p] = Decision(v, t, r)) + \* then + => /\ beginConsensus - Precision <= t + /\ t < endConsensus[p] + Precision + Delay + +\* [PBTS-CONSENSUS-SAFE-VALID-CORR-PROP.0] +ConsensusSafeValidCorrProp == + \A v \in ValidValues, t \in Timestamps : + \* if the proposer in the first round is correct + (/\ Proposer[0] \in Corr + \* and there exists a process that decided on v, t + /\ \E p \in Corr, r \in Rounds : decision[p] = Decision(v, t, r)) + \* then t is between the minimal and maximal initial local time + => /\ beginConsensus <= t + /\ t <= lastBeginConsensus + +\* [PBTS-CONSENSUS-REALTIME-VALID-CORR.0] +ConsensusRealTimeValidCorr == + \A t \in Timestamps, r \in Rounds : + (/\ \E p \in Corr, v \in ValidValues : decision[p] = Decision(v, t, r) + /\ proposalTime[r] /= NilTimestamp) + => /\ proposalTime[r] - Accuracy < t + /\ t < proposalTime[r] + Accuracy + +\* [PBTS-CONSENSUS-REALTIME-VALID.0] +ConsensusRealTimeValid == + \A t \in Timestamps, r \in Rounds : + (\E p \in Corr, v \in ValidValues : decision[p] = Decision(v, t, r)) + => /\ proposalReceivedTime[r] - Accuracy - Precision < t + /\ t < proposalReceivedTime[r] + Accuracy + Precision + Delay + +\* [PBTS-MSG-FAIR.0] +BoundedDelay == + \A r \in Rounds : + (/\ proposalTime[r] /= NilTimestamp + /\ proposalTime[r] + Delay < realTime) + => inspectedProposal[r] = Corr + +\* [PBTS-CONSENSUS-TIME-LIVE.0] +ConsensusTimeLive == + \A r \in Rounds, p \in Corr : + (/\ proposalTime[r] /= NilTimestamp + /\ proposalTime[r] + Delay < realTime + /\ Proposer[r] \in Corr + /\ round[p] <= r) + => \E msg \in RoundProposals(r) : <> \in receivedTimelyProposal + +\* a conjunction of all invariants +Inv == + /\ AgreementOnValue + /\ AgreementOnTime + /\ ConsensusTimeValid + /\ ConsensusSafeValidCorrProp + /\ ConsensusRealTimeValid + /\ ConsensusRealTimeValidCorr + /\ BoundedDelay + +Liveness == + ConsensusTimeLive + +============================================================================= diff --git a/cometbft/v0.39/spec/core/Data_structures.mdx b/cometbft/v0.39/spec/core/Data_structures.mdx new file mode 100644 index 000000000..abe24d4a0 --- /dev/null +++ b/cometbft/v0.39/spec/core/Data_structures.mdx @@ -0,0 +1,502 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/spec/core/Data_structures' +title: Data Structures +order: 1 +--- + +Here we describe the data structures in the CometBFT blockchain and the rules for validating them. + +The CometBFT blockchain consists of a short list of data types: + +- [Data Structures](#data-structures) + - [Block](#block) + - [Execution](#execution) + - [Header](#header) + - [Version](#version) + - [BlockID](#blockid) + - [PartSetHeader](#partsetheader) + - [Part](#part) + - [Time](#time) + - [Data](#data) + - [Commit](#commit) + - [CommitSig](#commitsig) + - [BlockIDFlag](#blockidflag) + - [Vote](#vote) + - [CanonicalVote](#canonicalvote) + - [Proposal](#proposal) + - [SignedMsgType](#signedmsgtype) + - [Signature](#signature) + - [EvidenceList](#evidencelist) + - [Evidence](#evidence) + - [DuplicateVoteEvidence](#duplicatevoteevidence) + - [LightClientAttackEvidence](#lightclientattackevidence) + - [LightBlock](#lightblock) + - [SignedHeader](#signedheader) + - [ValidatorSet](#validatorset) + - [Validator](#validator) + - [Address](#address) + - [ConsensusParams](#consensusparams) + - [BlockParams](#blockparams) + - [EvidenceParams](#evidenceparams) + - [ValidatorParams](#validatorparams) + - [VersionParams](#versionparams) + - [Proof](#proof) + + +## Block + +A block consists of a header, transactions, votes (the commit), +and a list of evidence of malfeasance (ie. signing conflicting votes). + +| Name | Type | Description | Validation | +|--------|-------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------| +| Header | [Header](#header) | Header corresponding to the block. This field contains information used throughout consensus and other areas of the protocol. To find out what it contains, visit [header](#header) | Must adhere to the validation rules of [header](#header) | +| Data | [Data](#data) | Data contains a list of transactions. The contents of the transaction is unknown to CometBFT. | This field can be empty or populated, but no validation is performed. Applications can perform validation on individual transactions prior to block creation using [checkTx](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/abci/abci%2B%2B_methods.md#checktx). +| Evidence | [EvidenceList](#evidencelist) | Evidence contains a list of infractions committed by validators. | Can be empty, but when populated the validations rules from [evidenceList](#evidencelist) apply | +| LastCommit | [Commit](#commit) | `LastCommit` includes one vote for every validator. All votes must either be for the previous block, nil or absent. If a vote is for the previous block it must have a valid signature from the corresponding validator. The sum of the voting power of the validators that voted must be greater than 2/3 of the total voting power of the complete validator set. The number of votes in a commit is limited to 10000 (see `types.MaxVotesCount`). | Must be empty for the initial height and must adhere to the validation rules of [commit](#commit). | + +## Execution + +Once a block is validated, it can be executed against the state. + +The state follows this recursive equation: + +```go +state(initialHeight) = InitialState +state(h+1) <- Execute(state(h), ABCIApp, block(h)) +``` + +where `InitialState` includes the initial consensus parameters and validator set, +and `ABCIApp` is an ABCI application that can return results and changes to the validator +set (TODO). Execute is defined as: + +```go +func Execute(state State, app ABCIApp, block Block) State { + // Fuction ApplyBlock executes block of transactions against the app and returns the new root hash of the app state, + // modifications to the validator set and the changes of the consensus parameters. + AppHash, ValidatorChanges, ConsensusParamChanges := app.ApplyBlock(block) + + nextConsensusParams := UpdateConsensusParams(state.ConsensusParams, ConsensusParamChanges) + return State{ + ChainID: state.ChainID, + InitialHeight: state.InitialHeight, + LastResults: abciResponses.DeliverTxResults, + AppHash: AppHash, + LastValidators: state.Validators, + Validators: state.NextValidators, + NextValidators: UpdateValidators(state.NextValidators, ValidatorChanges), + ConsensusParams: nextConsensusParams, + Version: { + Consensus: { + AppVersion: nextConsensusParams.Version.AppVersion, + }, + }, + } +} +``` + +Validating a new block is first done prior to the `prevote`, `precommit` & `finalizeCommit` stages. + +The steps to validate a new block are: + +- Check the validity rules of the block and its fields. +- Check the versions (Block & App) are the same as in local state. +- Check the chainID's match. +- Check the height is correct. +- Check the `LastBlockID` corresponds to BlockID currently in state. +- Check the hashes in the header match those in state. +- Verify the LastCommit against state, this step is skipped for the initial height. + - This is where checking the signatures correspond to the correct block will be made. +- Make sure the proposer is part of the validator set. +- Validate bock time. + - Make sure the new blocks time is after the previous blocks time. + - Calculate the medianTime and check it against the blocks time. + - If the blocks height is the initial height then check if it matches the genesis time. +- Validate the evidence in the block. Note: Evidence can be empty + +## Header + +A block header contains metadata about the block and about the consensus, as well as commitments to +the data in the current block, the previous block, and the results returned by the application: + +| Name | Type | Description | Validation | +|-------------------|---------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| Version | [Version](#version) | Version defines the application and block versions being used. | Must adhere to the validation rules of [Version](#version) | +| ChainID | String | ChainID is the ID of the chain. This must be unique to your chain. | ChainID must be less than 50 bytes. | +| Height | uint64 | Height is the height for this header. | Must be > 0, >= initialHeight, and == previous Height+1 | +| Time | [Time](#time) | The timestamp is equal to the weighted median of validators present in the last commit. Read more on time in the [BFT-time section](/cometbft/v0.39/spec/consensus/BFT-Time). Note: the timestamp of a vote must be greater by at least one millisecond than that of the block being voted on. | Time must be >= previous header timestamp + consensus parameters TimeIotaMs. The timestamp of the first block must be equal to the genesis time (since there's no votes to compute the median). | +| LastBlockID | [BlockID](#blockid) | BlockID of the previous block. | Must adhere to the validation rules of [blockID](#blockid). The first block has `block.Header.LastBlockID == BlockID{}`. | +| LastCommitHash | slice of bytes (`[]byte`) | MerkleRoot of the lastCommit's signatures. The signatures represent the validators that committed to the last block. The first block has an empty slices of bytes for the hash. | Must be of length 32 | +| DataHash | slice of bytes (`[]byte`) | MerkleRoot of the hash of transactions. **Note**: The transactions are hashed before being included in the merkle tree, the leaves of the Merkle tree are the hashes, not the transactions themselves. | Must be of length 32 | +| ValidatorHash | slice of bytes (`[]byte`) | MerkleRoot of the current validator set. The validators are first sorted by voting power (descending), then by address (ascending) prior to computing the MerkleRoot. | Must be of length 32 | +| NextValidatorHash | slice of bytes (`[]byte`) | MerkleRoot of the next validator set. The validators are first sorted by voting power (descending), then by address (ascending) prior to computing the MerkleRoot. | Must be of length 32 | +| ConsensusHash | slice of bytes (`[]byte`) | Hash of the protobuf encoded consensus parameters. | Must be of length 32 | +| AppHash | slice of bytes (`[]byte`) | Arbitrary byte array returned by the application after executing and commiting the previous block. It serves as the basis for validating any merkle proofs that comes from the ABCI application and represents the state of the actual application rather than the state of the blockchain itself. The first block's `block.Header.AppHash` is given by `ResponseInitChain.app_hash`. | This hash is determined by the application, CometBFT can not perform validation on it. | +| LastResultHash | slice of bytes (`[]byte`) | `LastResultsHash` is the root hash of a Merkle tree built from `ResponseDeliverTx` responses (`Log`,`Info`, `Codespace` and `Events` fields are ignored). | Must be of length 32. The first block has `block.Header.ResultsHash == MerkleRoot(nil)`, i.e. the hash of an empty input, for RFC-6962 conformance. | +| EvidenceHash | slice of bytes (`[]byte`) | MerkleRoot of the evidence of Byzantine behavior included in this block. | Must be of length 32 | +| ProposerAddress | slice of bytes (`[]byte`) | Address of the original proposer of the block. Validator must be in the current validatorSet. | Must be of length 20 | + +## Version + +NOTE: that this is more specifically the consensus version and doesn't include information like the +P2P Version. (TODO: we should write a comprehensive document about +versioning that this can refer to) + +| Name | type | Description | Validation | +|-------|--------|---------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------| +| Block | uint64 | This number represents the block version and must be the same throughout an operational network | Must be equal to block version being used in a network (`block.Version.Block == state.Version.Consensus.Block`) | +| App | uint64 | App version is decided on by the application. Read [here](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/abci/abci++_app_requirements.md) | `block.Version.App == state.Version.Consensus.App` | + +## BlockID + +The `BlockID` contains two distinct Merkle roots of the block. The `BlockID` includes these two hashes, as well as the number of parts (ie. `len(MakeParts(block))`) + +| Name | Type | Description | Validation | +|---------------|---------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------| +| Hash | slice of bytes (`[]byte`) | MerkleRoot of all the fields in the header (ie. `MerkleRoot(header)`. | hash must be of length 32 | +| PartSetHeader | [PartSetHeader](#partsetheader) | Used for secure gossiping of the block during consensus, is the MerkleRoot of the complete serialized block cut into parts (ie. `MerkleRoot(MakeParts(block))`). | Must adhere to the validation rules of [PartSetHeader](#partsetheader) | + +See [MerkleRoot](/cometbft/v0.39/spec/core/encoding#merkleroot) for details. + +## PartSetHeader + +| Name | Type | Description | Validation | +|-------|---------------------------|-----------------------------------|----------------------| +| Total | int32 | Total amount of parts for a block | Must be > 0 | +| Hash | slice of bytes (`[]byte`) | MerkleRoot of a serialized block | Must be of length 32 | + +## Part + +Part defines a part of a block. In CometBFT blocks are broken into `parts` for gossip. + +| Name | Type | Description | Validation | +|-------|-----------------|-----------------------------------|----------------------| +| index | int32 | Total amount of parts for a block | Must be > 0 | +| bytes | bytes | MerkleRoot of a serialized block | Must be of length 32 | +| proof | [Proof](#proof) | MerkleRoot of a serialized block | Must be of length 32 | + +## Time + +CometBFT uses the [Google.Protobuf.Timestamp](https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#google.protobuf.Timestamp) +format, which uses two integers, one 64 bit integer for Seconds and a 32 bit integer for Nanoseconds. + +## Data + +Data is just a wrapper for a list of transactions, where transactions are arbitrary byte arrays: + +| Name | Type | Description | Validation | +|------|----------------------------|------------------------|-----------------------------------------------------------------------------| +| Txs | Matrix of bytes ([][]byte) | Slice of transactions. | Validation does not occur on this field, this data is unknown to CometBFT | + +## Commit + +Commit is a simple wrapper for a list of signatures, with one for each validator. It also contains the relevant BlockID, height and round: + +| Name | Type | Description | Validation | +|------------|----------------------------------|----------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------| +| Height | int64 | Height at which this commit was created. | Must be > 0 | +| Round | int32 | Round that the commit corresponds to. | Must be > 0 | +| BlockID | [BlockID](#blockid) | The blockID of the corresponding block. | Must adhere to the validation rules of [BlockID](#blockid). | +| Signatures | Array of [CommitSig](#commitsig) | Array of commit signatures that correspond to current validator set. | Length of signatures must be > 0 and adhere to the validation of each individual [Commitsig](#commitsig) | + +## ExtendedCommit + +`ExtendedCommit`, similarly to Commit, wraps a list of votes with signatures together with other data needed to verify them. +In addition, it contains the verified vote extensions, one for each non-`nil` vote, along with the extension signatures. + +| Name | Type | Description | Validation | +|--------------------|------------------------------------------|-------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------| +| Height | int64 | Height at which this commit was created. | Must be > 0 | +| Round | int32 | Round that the commit corresponds to. | Must be > 0 | +| BlockID | [BlockID](#blockid) | The blockID of the corresponding block. | Must adhere to the validation rules of [BlockID](#blockid). | +| ExtendedSignatures | Array of [ExtendedCommitSig](#commitsig) | The current validator set's commit signatures, extension, and extension signatures. | Length of signatures must be > 0 and adhere to the validation of each individual [ExtendedCommitSig](#extendedcommitsig) | + +## CommitSig + +`CommitSig` represents a signature of a validator, who has voted either for nil, +a particular `BlockID` or was absent. It's a part of the `Commit` and can be used +to reconstruct the vote set given the validator set. + +| Name | Type | Description | Validation | +|------------------|-----------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------| +| BlockIDFlag | [BlockIDFlag](#blockidflag) | Represents the validators participation in consensus: its vote was not received, voted for the block that received the majority, or voted for nil | Must be one of the fields in the [BlockIDFlag](#blockidflag) enum | +| ValidatorAddress | [Address](#address) | Address of the validator | Must be of length 20 | +| Timestamp | [Time](#time) | This field will vary from `CommitSig` to `CommitSig`. It represents the timestamp of the validator. | [Time](#time) | +| Signature | [Signature](#signature) | Signature corresponding to the validators participation in consensus. | The length of the signature must be > 0 and < than 64 | + +NOTE: `ValidatorAddress` and `Timestamp` fields may be removed in the future +(see [ADR-25](https://github.com/cometbft/cometbft/blob/main/docs/references/architecture/tendermint-core/adr-025-commit.md)). + +## ExtendedCommitSig + +`ExtendedCommitSig` represents a signature of a validator that has voted either for `nil`, +a particular `BlockID` or was absent. It is part of the `ExtendedCommit` and can be used +to reconstruct the vote set given the validator set. +Additionally it contains the vote extensions that were attached to each non-`nil` precommit vote. +All these extensions have been verified by the application operating at the signing validator's node. + +| Name | Type | Description | Validation | +|--------------------|-----------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------| +| BlockIDFlag | [BlockIDFlag](#blockidflag) | Represents the validators participation in consensus: its vote was not received, voted for the block that received the majority, or voted for nil | Must be one of the fields in the [BlockIDFlag](#blockidflag) enum | +| ValidatorAddress | [Address](#address) | Address of the validator | Must be of length 20 | +| Timestamp | [Time](#time) | This field will vary from `CommitSig` to `CommitSig`. It represents the timestamp of the validator. | | +| Signature | [Signature](#signature) | Signature corresponding to the validators participation in consensus. | Length must be > 0 and < 64 | +| Extension | bytes | Vote extension provided by the Application running on the sender of the precommit vote, and verified by the local application. | Length must be zero if BlockIDFlag is not `Commit` | +| ExtensionSignature | [Signature](#signature) | Signature of the vote extension. | Length must be > 0 and < than 64 if BlockIDFlag is `Commit`, else 0 | + +## BlockIDFlag + +BlockIDFlag represents which BlockID the [signature](#commitsig) is for. + +```go +enum BlockIDFlag { + BLOCK_ID_FLAG_UNKNOWN = 0; // indicates an error condition + BLOCK_ID_FLAG_ABSENT = 1; // the vote was not received + BLOCK_ID_FLAG_COMMIT = 2; // voted for the block that received the majority + BLOCK_ID_FLAG_NIL = 3; // voted for nil +} +``` + +## Vote + +A vote is a signed message from a validator for a particular block. +The vote includes information about the validator signing it. When stored in the blockchain or propagated over the network, votes are encoded in Protobuf. + +| Name | Type | Description | Validation | +|--------------------|---------------------------------|------------------------------------------------------------------------------------------|------------------------------------------| +| Type | [SignedMsgType](#signedmsgtype) | The type of message the vote refers to | Must be `PrevoteType` or `PrecommitType` | +| Height | int64 | Height for which this vote was created for | Must be > 0 | +| Round | int32 | Round that the commit corresponds to. | Must be > 0 | +| BlockID | [BlockID](#blockid) | The blockID of the corresponding block. | | +| Timestamp | [Time](#time) | Timestamp represents the time at which a validator signed. | | +| ValidatorAddress | bytes | Address of the validator | Length must be equal to 20 | +| ValidatorIndex | int32 | Index at a specific block height corresponding to the Index of the validator in the set. | Must be > 0 | +| Signature | bytes | Signature by the validator if they participated in consensus for the associated block. | Length must be > 0 and < 64 | +| Extension | bytes | Vote extension provided by the Application running at the validator's node. | Length can be 0 | +| ExtensionSignature | bytes | Signature for the extension | Length must be > 0 and < 64 | + +## CanonicalVote + +CanonicalVote is for validator signing. This type will not be present in a block. +Votes are represented via `CanonicalVote` and also encoded using protobuf via `type.SignBytes` which includes the `ChainID`, +and uses a different ordering of the fields. + +| Name | Type | Description | Validation | +|-----------|---------------------------------|-----------------------------------------|------------------------------------------| +| Type | [SignedMsgType](#signedmsgtype) | The type of message the vote refers to | Must be `PrevoteType` or `PrecommitType` | +| Height | int64 | Height in which the vote was provided. | Must be > 0 | +| Round | int64 | Round in which the vote was provided. | Must be > 0 | +| BlockID | string | ID of the block the vote refers to. | | +| Timestamp | string | Time of the vote. | | +| ChainID | string | ID of the blockchain running consensus. | | + +For signing, votes are represented via [`CanonicalVote`](#canonicalvote) and also encoded using protobuf via +`type.SignBytes` which includes the `ChainID`, and uses a different ordering of +the fields. + +We define a method `Verify` that returns `true` if the signature verifies against the pubkey for the `SignBytes` +using the given ChainID: + +```go +func (vote *Vote) Verify(chainID string, pubKey crypto.PubKey) error { + if !bytes.Equal(pubKey.Address(), vote.ValidatorAddress) { + return ErrVoteInvalidValidatorAddress + } + v := vote.ToProto() + if !pubKey.VerifyBytes(types.VoteSignBytes(chainID, v), vote.Signature) { + return ErrVoteInvalidSignature + } + return nil +} +``` + +### CanonicalVoteExtension + +Vote extensions are signed using a representation similar to votes. +This is the structure to marshall in order to obtain the bytes to sign or verify the signature. + +| Name | Type | Description | Validation | +|-----------|--------|---------------------------------------------|----------------------| +| Extension | bytes | Vote extension provided by the Application. | Can have zero length | +| Height | int64 | Height in which the extension was provided. | Must be > 0 | +| Round | int64 | Round in which the extension was provided. | Must be > 0 | +| ChainID | string | ID of the blockchain running consensus. | | + +## Proposal + +Proposal contains height and round for which this proposal is made, BlockID as a unique identifier +of proposed block, timestamp, and POLRound (a so-called Proof-of-Lock (POL) round) that is needed for +termination of the consensus. If POLRound >= 0, then BlockID corresponds to the block that +is locked in POLRound. The message is signed by the validator private key. + +| Name | Type | Description | Validation | +|-----------|---------------------------------|---------------------------------------------------------------------------------------|---------------------------------------------------------| +| Type | [SignedMsgType](#signedmsgtype) | Represents a Proposal [SignedMsgType](#signedmsgtype) | Must be `ProposalType` [signedMsgType](#signedmsgtype) | +| Height | uint64 | Height for which this vote was created for | Must be > 0 | +| Round | int32 | Round that the commit corresponds to. | Must be > 0 | +| POLRound | int64 | Proof of lock | Must be > 0 | +| BlockID | [BlockID](#blockid) | The blockID of the corresponding block. | [BlockID](#blockid) | +| Timestamp | [Time](#time) | Timestamp represents the time at which a validator signed. | [Time](#time) | +| Signature | slice of bytes (`[]byte`) | Signature by the validator if they participated in consensus for the associated bock. | Length of signature must be > 0 and < 64 | + +## SignedMsgType + +Signed message type represents a signed messages in consensus. + +```proto +enum SignedMsgType { + + SIGNED_MSG_TYPE_UNKNOWN = 0; + // Votes + SIGNED_MSG_TYPE_PREVOTE = 1; + SIGNED_MSG_TYPE_PRECOMMIT = 2; + + // Proposal + SIGNED_MSG_TYPE_PROPOSAL = 32; +} +``` + +## Signature + +Signatures in CometBFT are raw bytes representing the underlying signature. + +See the [signature spec](/cometbft/v0.39/spec/core/encoding#key-types) for more. + +## EvidenceList + +EvidenceList is a simple wrapper for a list of evidence: + +| Name | Type | Description | Validation | +|----------|--------------------------------|----------------------------------------|-----------------------------------------------------------------| +| Evidence | Array of [Evidence](#evidence) | List of verified [evidence](#evidence) | Validation adheres to individual types of [Evidence](#evidence) | + +## Evidence + +Evidence in CometBFT is used to indicate breaches in the consensus by a validator. + +More information on how evidence works in CometBFT can be found [here](/cometbft/v0.39/spec/consensus/Evidence) + +### DuplicateVoteEvidence + +`DuplicateVoteEvidence` represents a validator that has voted for two different blocks +in the same round of the same height. Votes are lexicographically sorted on `BlockID`. + +| Name | Type | Description | Validation | +|------------------|---------------|--------------------------------------------------------------------|-----------------------------------------------------| +| VoteA | [Vote](#vote) | One of the votes submitted by a validator when they equivocated | VoteA must adhere to [Vote](#vote) validation rules | +| VoteB | [Vote](#vote) | The second vote submitted by a validator when they equivocated | VoteB must adhere to [Vote](#vote) validation rules | +| TotalVotingPower | int64 | The total power of the validator set at the height of equivocation | Must be equal to nodes own copy of the data | +| ValidatorPower | int64 | Power of the equivocating validator at the height | Must be equal to the nodes own copy of the data | +| Timestamp | [Time](#time) | Time of the block where the equivocation occurred | Must be equal to the nodes own copy of the data | + +### LightClientAttackEvidence + +`LightClientAttackEvidence` is a generalized evidence that captures all forms of known attacks on +a light client such that a full node can verify, propose and commit the evidence on-chain for +punishment of the malicious validators. There are three forms of attacks: Lunatic, Equivocation +and Amnesia. These attacks are exhaustive. You can find a more detailed overview of this [here](/cometbft/v0.39/spec/light-client/Accountability#the-misbehavior-of-faulty-validators) + +| Name | Type | Description | Validation | +|----------------------|------------------------------------|----------------------------------------------------------------------|------------------------------------------------------------------| +| ConflictingBlock | [LightBlock](#lightblock) | Read Below | Must adhere to the validation rules of [lightBlock](#lightblock) | +| CommonHeight | int64 | Read Below | must be > 0 | +| Byzantine Validators | Array of [Validators](#validator) | validators that acted maliciously | Read Below | +| TotalVotingPower | int64 | The total power of the validator set at the height of the infraction | Must be equal to the nodes own copy of the data | +| Timestamp | [Time](#time) | Time of the block where the infraction occurred | Must be equal to the nodes own copy of the data | + +## LightBlock + +LightBlock is the core data structure of the [light client](/cometbft/v0.39/spec/light-client/Light-Client-Specification). It combines two data structures needed for verification ([signedHeader](#signedheader) & [validatorSet](#validatorset)). + +| Name | Type | Description | Validation | +|--------------|-------------------------------|----------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------| +| SignedHeader | [SignedHeader](#signedheader) | The header and commit, these are used for verification purposes. To find out more visit [light client docs](/cometbft/v0.39/spec/light-client/Light-Client-Specification) | Must not be nil and adhere to the validation rules of [signedHeader](#signedheader) | +| ValidatorSet | [ValidatorSet](#validatorset) | The validatorSet is used to help with verify that the validators in that committed the infraction were truly in the validator set. | Must not be nil and adhere to the validation rules of [validatorSet](#validatorset) | + +The `SignedHeader` and `ValidatorSet` are linked by the hash of the validator set(`SignedHeader.ValidatorsHash == ValidatorSet.Hash()`. + +## SignedHeader + +The SignedhHeader is the [header](#header) accompanied by the commit to prove it. + +| Name | Type | Description | Validation | +|--------|-------------------|-------------------|-----------------------------------------------------------------------------------| +| Header | [Header](#header) | [Header](#header) | Header cannot be nil and must adhere to the [Header](#header) validation criteria | +| Commit | [Commit](#commit) | [Commit](#commit) | Commit cannot be nil and must adhere to the [Commit](#commit) criteria | + +## ValidatorSet + +| Name | Type | Description | Validation | +|------------|----------------------------------|----------------------------------------------------|-------------------------------------------------------------------------------------------------------------------| +| Validators | Array of [validator](#validator) | List of the active validators at a specific height | The list of validators can not be empty or nil and must adhere to the validation rules of [validator](#validator) | +| Proposer | [validator](#validator) | The block proposer for the corresponding block | The proposer cannot be nil and must adhere to the validation rules of [validator](#validator) | + +## Validator + +| Name | Type | Description | Validation | +|------------------|---------------------------|---------------------------------------------------------------------------------------------------|---------------------------------------------------| +| Address | [Address](#address) | Validators Address | Length must be of size 20 | +| Pubkey | slice of bytes (`[]byte`) | Validators Public Key | must be a length greater than 0 | +| VotingPower | int64 | Validators voting power | cannot be < 0 | +| ProposerPriority | int64 | Validators proposer priority. This is used to gauge when a validator is up next to propose blocks | No validation, value can be negative and positive | + +## Address + +Address is a type alias of a slice of bytes. The address is calculated by hashing the public key using sha256 and truncating it to only use the first 20 bytes of the slice. + +```go +const ( + TruncatedSize = 20 +) + +func SumTruncated(bz []byte) []byte { + hash := sha256.Sum256(bz) + return hash[:TruncatedSize] +} +``` + +## ConsensusParams + +| Name | Type | Description | Field Number | +|-----------|-------------------------------------|------------------------------------------------------------------------------|--------------| +| block | [BlockParams](#blockparams) | Parameters limiting the size of a block and time between consecutive blocks. | 1 | +| evidence | [EvidenceParams](#evidenceparams) | Parameters limiting the validity of evidence of byzantine behavior. | 2 | +| validator | [ValidatorParams](#validatorparams) | Parameters limiting the types of public keys validators can use. | 3 | +| version | [BlockParams](#blockparams) | The ABCI application version. | 4 | + +### BlockParams + +| Name | Type | Description | Field Number | +|--------------|-------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------| +| max_bytes | int64 | Max size of a block, in bytes. | 1 | +| max_gas | int64 | Max sum of `GasWanted` in a proposed block. NOTE: blocks that violate this may be committed if there are Byzantine proposers. It's the application's responsibility to handle this when processing a block! | 2 | + +### EvidenceParams + +| Name | Type | Description | Field Number | +|--------------------|------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------| +| max_age_num_blocks | int64 | Max age of evidence, in blocks. | 1 | +| max_age_duration | [google.protobuf.Duration](https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#google.protobuf.Duration) | Max age of evidence, in time. It should correspond with an app's "unbonding period" or other similar mechanism for handling [Nothing-At-Stake attacks](https://vitalik.eth.limo/general/2017/12/31/pos_faq.html#what-is-the-nothing-at-stake-problem-and-how-can-it-be-fixed). | 2 | +| max_bytes | int64 | maximum size in bytes of total evidence allowed to be entered into a block | 3 | + +### ValidatorParams + +| Name | Type | Description | Field Number | +|---------------|-----------------|-----------------------------------------------------------------------|--------------| +| pub_key_types | repeated string | List of accepted public key types. Uses same naming as `PubKey.Type`. | 1 | + +### VersionParams + +| Name | Type | Description | Field Number | +|-------------|--------|-------------------------------|--------------| +| app_version | uint64 | The ABCI application version. | 1 | + +## Proof + +| Name | Type | Description | Field Number | +|-----------|----------------|-----------------------------------------------|--------------| +| total | int64 | Total number of items. | 1 | +| index | int64 | Index item to prove. | 2 | +| leaf_hash | bytes | Hash of item value. | 3 | +| aunts | repeated bytes | Hashes from leaf's sibling to a root's child. | 4 | diff --git a/cometbft/v0.39/spec/core/Overview.mdx b/cometbft/v0.39/spec/core/Overview.mdx new file mode 100644 index 000000000..dd8052cc7 --- /dev/null +++ b/cometbft/v0.39/spec/core/Overview.mdx @@ -0,0 +1,15 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/spec/core/Overview' +order: 1 +parent: + title: Core + order: 3 +--- + +This section describes the core types and functionality of the CometBFT protocol implementation. + +- [Core Data Structures](/cometbft/v0.39/spec/core/Data_structures) +- [Encoding](/cometbft/v0.39/spec/core/encoding) +- [Genesis](/cometbft/v0.39/spec/core/genesis) +- [State](/cometbft/v0.39/spec/core/state) \ No newline at end of file diff --git a/cometbft/v0.39/spec/core/encoding.mdx b/cometbft/v0.39/spec/core/encoding.mdx new file mode 100644 index 000000000..72b63918c --- /dev/null +++ b/cometbft/v0.39/spec/core/encoding.mdx @@ -0,0 +1,305 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/spec/core/encoding' +title: Encoding +order: 2 +--- + +## Protocol Buffers + +CometBFT uses [Protocol Buffers](https://developers.google.com/protocol-buffers), specifically proto3, for all data structures. + +Please see the [Proto3 language guide](https://developers.google.com/protocol-buffers/docs/proto3) for more details. + +## Byte Arrays + +The encoding of a byte array is simply the raw-bytes prefixed with the length of +the array as a `UVarint` (what proto calls a `Varint`). + +For details on varints, see the [protobuf +spec](https://developers.google.com/protocol-buffers/docs/encoding#varints). + +For example, the byte-array `[0xA, 0xB]` would be encoded as `0x020A0B`, +while a byte-array containing 300 entires beginning with `[0xA, 0xB, ...]` would +be encoded as `0xAC020A0B...` where `0xAC02` is the UVarint encoding of 300. + +## Hashing + +CometBFT uses `SHA256` as its hash function. +Objects are always serialized before being hashed. +So `SHA256(obj)` is short for `SHA256(ProtoEncoding(obj))`. + +## Public Key Cryptography + +CometBFT uses Protobuf [Oneof](https://developers.google.com/protocol-buffers/docs/proto3#oneof) +to distinguish between different types public keys, and signatures. +Additionally, for each public key, CometBFT +defines an Address function that can be used as a more compact identifier in +place of the public key. Here we list the concrete types, their names, +and prefix bytes for public keys and signatures, as well as the address schemes +for each PubKey. Note for brevity we don't +include details of the private keys beyond their type and name. + +### Key Types + +Each type specifies it's own pubkey, address, and signature format. + +#### Ed25519 + +The address is the first 20-bytes of the SHA256 hash of the raw 32-byte public key: + +```go +address = SHA256(pubkey)[:20] +``` + +The signature is the raw 64-byte ED25519 signature. + +CometBFT adopts [zip215](https://zips.z.cash/zip-0215) for verification of ed25519 signatures. + +> Note: This change will be released in the next major release of CometBFT. + +#### Secp256k1 + +The address is the first 20-bytes of the SHA256 hash of the raw 32-byte public key: + +```go +address = SHA256(pubkey)[:20] +``` + +## Other Common Types + +### BitArray + +The BitArray is used in some consensus messages to represent votes received from +validators, or parts received in a block. It is represented +with a struct containing the number of bits (`Bits`) and the bit-array itself +encoded in base64 (`Elems`). + +| Name | Type | +|-------|----------------------------| +| bits | int64 | +| elems | slice of int64 (`[]int64`) | + +Note BitArray receives a special JSON encoding in the form of `x` and `_` +representing `1` and `0`. Ie. the BitArray `10110` would be JSON encoded as +`"x_xx_"` + +### Part + +Part is used to break up blocks into pieces that can be gossiped in parallel +and securely verified using a Merkle tree of the parts. + +Part contains the index of the part (`Index`), the actual +underlying data of the part (`Bytes`), and a Merkle proof that the part is contained in +the set (`Proof`). + +| Name | Type | +|-------|---------------------------| +| index | uint32 | +| bytes | slice of bytes (`[]byte`) | +| proof | [proof](#merkle-proof) | + +See details of SimpleProof, below. + +### MakeParts + +Encode an object using Protobuf and slice it into parts. +CometBFT uses a part size of 65536 bytes, and allows a maximum of 1601 parts +(see `types.MaxBlockPartsCount`). This corresponds to the hard-coded block size +limit of 100MB. + +```go +func MakeParts(block Block) []Part +``` + +## Merkle Trees + +For an overview of Merkle trees, see +[wikipedia](https://en.wikipedia.org/wiki/Merkle_tree) + +We use the RFC 6962 specification of a merkle tree, with sha256 as the hash function. +Merkle trees are used throughout CometBFT to compute a cryptographic digest of a data structure. +The differences between RFC 6962 and the simplest form a merkle tree are that: + +1. leaf nodes and inner nodes have different hashes. + This is for "second pre-image resistance", to prevent the proof to an inner node being valid as the proof of a leaf. + The leaf nodes are `SHA256(0x00 || leaf_data)`, and inner nodes are `SHA256(0x01 || left_hash || right_hash)`. + +2. When the number of items isn't a power of two, the left half of the tree is as big as it could be. + (The largest power of two less than the number of items) This allows new leaves to be added with less + recomputation. For example: + +```md + Simple Tree with 6 items Simple Tree with 7 items + + * * + / \ / \ + / \ / \ + / \ / \ + / \ / \ + * * * * + / \ / \ / \ / \ + / \ / \ / \ / \ + / \ / \ / \ / \ + * * h4 h5 * * * h6 + / \ / \ / \ / \ / \ +h0 h1 h2 h3 h0 h1 h2 h3 h4 h5 +``` + +### MerkleRoot + +The function `MerkleRoot` is a simple recursive function defined as follows: + +```go +// SHA256([]byte{}) +func emptyHash() []byte { + return tmhash.Sum([]byte{}) +} + +// SHA256(0x00 || leaf) +func leafHash(leaf []byte) []byte { + return tmhash.Sum(append(0x00, leaf...)) +} + +// SHA256(0x01 || left || right) +func innerHash(left []byte, right []byte) []byte { + return tmhash.Sum(append(0x01, append(left, right...)...)) +} + +// largest power of 2 less than k +func getSplitPoint(k int) { ... } + +func MerkleRoot(items [][]byte) []byte{ + switch len(items) { + case 0: + return empthHash() + case 1: + return leafHash(items[0]) + default: + k := getSplitPoint(len(items)) + left := MerkleRoot(items[:k]) + right := MerkleRoot(items[k:]) + return innerHash(left, right) + } +} +``` + +Note: `MerkleRoot` operates on items which are arbitrary byte arrays, not +necessarily hashes. For items which need to be hashed first, we introduce the +`Hashes` function: + +```go +func Hashes(items [][]byte) [][]byte { + return SHA256 of each item +} +``` + +Note: we will abuse notion and invoke `MerkleRoot` with arguments of type `struct` or type `[]struct`. +For `struct` arguments, we compute a `[][]byte` containing the protobuf encoding of each +field in the struct, in the same order the fields appear in the struct. +For `[]struct` arguments, we compute a `[][]byte` by protobuf encoding the individual `struct` elements. + +### Merkle Proof + +Proof that a leaf is in a Merkle tree is composed as follows: + +| Name | Type | +|----------|----------------------------| +| total | int64 | +| index | int64 | +| leafHash | slice of bytes (`[]byte`) | +| aunts | Matrix of bytes ([][]byte) | + +Which is verified as follows: + +```golang +func (proof Proof) Verify(rootHash []byte, leaf []byte) bool { + assert(proof.LeafHash, leafHash(leaf) + + computedHash := computeHashFromAunts(proof.Index, proof.Total, proof.LeafHash, proof.Aunts) + return computedHash == rootHash +} + +func computeHashFromAunts(index, total int, leafHash []byte, innerHashes [][]byte) []byte{ + assert(index < total && index >= 0 && total > 0) + + if total == 1{ + assert(len(proof.Aunts) == 0) + return leafHash + } + + assert(len(innerHashes) > 0) + + numLeft := getSplitPoint(total) // largest power of 2 less than total + if index < numLeft { + leftHash := computeHashFromAunts(index, numLeft, leafHash, innerHashes[:len(innerHashes)-1]) + assert(leftHash != nil) + return innerHash(leftHash, innerHashes[len(innerHashes)-1]) + } + rightHash := computeHashFromAunts(index-numLeft, total-numLeft, leafHash, innerHashes[:len(innerHashes)-1]) + assert(rightHash != nil) + return innerHash(innerHashes[len(innerHashes)-1], rightHash) +} +``` + +The number of aunts is limited to 100 (`MaxAunts`) to protect the node against DOS attacks. +This limits the tree size to 2^100 leaves, which should be sufficient for any +conceivable purpose. + +### IAVL+ Tree + +Because CometBFT only uses a Simple Merkle Tree, application developers are expected to use their own Merkle tree in their applications. For example, the IAVL+ Tree - an immutable self-balancing binary tree for persisting application state is used by the [Cosmos SDK](https://github.com/cosmos/cosmos-sdk/blob/ae77f0080a724b159233bd9b289b2e91c0de21b5/docs/interfaces/lite/specification.md) + +## JSON + +CometBFT has its own JSON encoding in order to keep backwards compatibility with the previous RPC layer. + +Registered types are encoded as: + +```json +{ + "type": "", + "value": +} +``` + +For instance, an ED25519 PubKey would look like: + +```json +{ + "type": "tendermint/PubKeyEd25519", + "value": "uZ4h63OFWuQ36ZZ4Bd6NF+/w9fWUwrOncrQsackrsTk=" +} +``` + +Where the `"value"` is the base64 encoding of the raw pubkey bytes, and the +`"type"` is the type name for Ed25519 pubkeys. + +### Signed Messages + +Signed messages (eg. votes, proposals) in the consensus are encoded using protobuf. + +When signing, the elements of a message are re-ordered so the fixed-length fields +are first, making it easy to quickly check the type, height, and round. +The `ChainID` is also appended to the end. +We call this encoding the SignBytes. For instance, SignBytes for a vote is the protobuf encoding of the following struct: + +```protobuf +message CanonicalVote { + SignedMsgType type = 1; + sfixed64 height = 2; // canonicalization requires fixed size encoding here + sfixed64 round = 3; // canonicalization requires fixed size encoding here + CanonicalBlockID block_id = 4; + google.protobuf.Timestamp timestamp = 5; + string chain_id = 6; +} +``` + +The field ordering and the fixed sized encoding for the first three fields is optimized to ease parsing of SignBytes +in HSMs. It creates fixed offsets for relevant fields that need to be read in this context. + +> Note: All canonical messages are length prefixed. + +For more details, see the [signing spec](/cometbft/v0.39/spec/consensus/Validator-Signing). +Also, see the motivating discussion in +[#1622](https://github.com/tendermint/tendermint/issues/1622). diff --git a/cometbft/v0.39/spec/core/genesis.mdx b/cometbft/v0.39/spec/core/genesis.mdx new file mode 100644 index 000000000..5837b38ef --- /dev/null +++ b/cometbft/v0.39/spec/core/genesis.mdx @@ -0,0 +1,39 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/spec/core/genesis' +title: Genesis +order: 3 +--- + +The genesis file is the starting point of a chain. An application will populate the `app_state` field in the genesis with their required fields. CometBFT is not able to validate this section because it is unaware what application state consists of. + +## Genesis Fields + +- `genesis_time`: The genesis time is the time the blockchain started or will start. If nodes are started before this time they will sit idle until the time specified. +- `chain_id`: The chainid is the chain identifier. Every chain should have a unique identifier. When conducting a fork based upgrade, we recommend changing the chainid to avoid network or consensus errors. +- `initial_height`: This field is the starting height of the blockchain. When conducting a chain restart to avoid restarting at height 1, the network is able to start at a specified height. +- `consensus_params` + - `block` + - `max_bytes`: The max amount of bytes a block can be. + - `max_gas`: The maximum amount of gas that a block can have. + - `time_iota_ms`: This parameter has no value anymore in CometBFT. + +- `evidence` + - `max_age_num_blocks`: After this preset amount of blocks has passed a single piece of evidence is considered invalid + - `max_age_duration`: After this preset amount of time has passed a single piece of evidence is considered invalid. + - `max_bytes`: The max amount of bytes of all evidence included in a block. + +> Note: For evidence to be considered invalid, evidence must be older than both `max_age_num_blocks` and `max_age_duration` + +- `validator` + - `pub_key_types`: Defines which curves are to be accepted as a valid validator consensus key. CometBFT supports ed25519, secp256k1, and bls12381. + +- `version` + - `app_version`: The version of the application. This is set by the application and is used to identify which version of the app a user should be using in order to operate a node. + +- `validators` + - This is an array of validators. This validator set is used as the starting validator set of the chain. This field can be empty, if the application sets the validator set in `InitChain`. + +- `app_hash`: The applications state root hash. This field does not need to be populated at the start of the chain, the application may provide the needed information via `Initchain`. + +- `app_state`: This section is filled in by the application and is unknown to CometBFT. diff --git a/cometbft/v0.39/spec/core/state.mdx b/cometbft/v0.39/spec/core/state.mdx new file mode 100644 index 000000000..1ba29d3b0 --- /dev/null +++ b/cometbft/v0.39/spec/core/state.mdx @@ -0,0 +1,132 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/spec/core/state' +title: State +order: 4 +--- + +The state contains information whose cryptographic digest is included in block headers, and thus is +necessary for validating new blocks. For instance, the validators set and the results of +transactions are never included in blocks, but their Merkle roots are: +the state keeps track of them. + +The `State` object itself is an implementation detail, since it is never +included in a block or gossiped over the network, and we never compute +its hash. The persistence or query interface of the `State` object +is an implementation detail and not included in the specification. +However, the types in the `State` object are part of the specification, since +the Merkle roots of the `State` objects are included in blocks and values are used during +validation. + +```go +type State struct { + ChainID string + InitialHeight int64 + + LastBlockHeight int64 + LastBlockID types.BlockID + LastBlockTime time.Time + + Version Version + LastResults []Result + AppHash []byte + + LastValidators ValidatorSet + Validators ValidatorSet + NextValidators ValidatorSet + + ConsensusParams ConsensusParams +} +``` + +The chain ID and initial height are taken from the genesis file, and not changed again. The +initial height will be `1` in the typical case, `0` is an invalid value. + +Note there is a hard-coded limit of 10000 validators. This is inherited from the +limit on the number of votes in a commit. + +Further information on [`Validator`'s](/cometbft/v0.39/spec/core/Data_structures#validator), +[`ValidatorSet`'s](/cometbft/v0.39/spec/core/Data_structures#validatorset) and +[`ConsensusParams`'s](/cometbft/v0.39/spec/core/Data_structures#consensusparams) can +be found in [data structures](/cometbft/v0.39/spec/core/Data_structures) + +## Execution + +State gets updated at the end of executing a block. Of specific interest is `ResponseEndBlock` and +`ResponseCommit` + +```go +type ResponseEndBlock struct { + ValidatorUpdates []ValidatorUpdate `protobuf:"bytes,1,rep,name=validator_updates,json=validatorUpdates,proto3" json:"validator_updates"` + ConsensusParamUpdates *types1.ConsensusParams `protobuf:"bytes,2,opt,name=consensus_param_updates,json=consensusParamUpdates,proto3" json:"consensus_param_updates,omitempty"` + Events []Event `protobuf:"bytes,3,rep,name=events,proto3" json:"events,omitempty"` +} +``` + +where + +```go +type ValidatorUpdate struct { + PubKey crypto.PublicKey `protobuf:"bytes,1,opt,name=pub_key,json=pubKey,proto3" json:"pub_key"` + Power int64 `protobuf:"varint,2,opt,name=power,proto3" json:"power,omitempty"` +} +``` + +and + +```go +type ResponseCommit struct { + // reserve 1 + Data []byte `protobuf:"bytes,2,opt,name=data,proto3" json:"data,omitempty"` + RetainHeight int64 `protobuf:"varint,3,opt,name=retain_height,json=retainHeight,proto3" json:"retain_height,omitempty"` +} +``` + +`ValidatorUpdates` are used to add and remove validators to the current set as well as update +validator power. Setting validator power to 0 in `ValidatorUpdate` will cause the validator to be +removed. `ConsensusParams` are safely copied across (i.e. if a field is nil it gets ignored) and the +`Data` from the `ResponseCommit` is used as the `AppHash` + +## Version + +```go +type Version struct { + consensus Consensus + software string +} +``` + +[`Consensus`](/cometbft/v0.39/spec/core/Data_structures#version) contains the protocol version for the blockchain and the +application. + +## Block + +The total size of a block is limited in bytes by the `ConsensusParams.Block.MaxBytes`. +Proposed blocks must be less than this size, and will be considered invalid +otherwise. + +The Application may set `ConsensusParams.Block.MaxBytes` to -1. +In that case, the actual block limit is set to 100 MB, +and CometBFT will provide all transactions in the mempool as part of `PrepareProposal`. +The application has to be careful to return a list of transactions in `ResponsePrepareProposal` +whose size is less than or equal to `RequestPrepareProposal.MaxTxBytes`. + +Blocks should additionally be limited by the amount of "gas" consumed by the +transactions in the block, though this is not yet implemented. + +## Evidence + +For evidence in a block to be valid, it must satisfy: + +```go +block.Header.Time-evidence.Time < ConsensusParams.Evidence.MaxAgeDuration && + block.Header.Height-evidence.Height < ConsensusParams.Evidence.MaxAgeNumBlocks +``` + +A block must not contain more than `ConsensusParams.Evidence.MaxBytes` of evidence. This is +implemented to mitigate spam attacks. + +## Validator + +Validators from genesis file and `ResponseEndBlock` must have pubkeys of type ∈ +`ConsensusParams.Validator.PubKeyTypes`. diff --git a/cometbft/v0.39/spec/ivy-proofs/Dockerfile b/cometbft/v0.39/spec/ivy-proofs/Dockerfile new file mode 100644 index 000000000..be60151fd --- /dev/null +++ b/cometbft/v0.39/spec/ivy-proofs/Dockerfile @@ -0,0 +1,37 @@ +# we need python2 support, which was dropped after buster: +FROM debian:buster + +RUN echo 'debconf debconf/frontend select Noninteractive' | debconf-set-selections +RUN apt-get update +RUN apt-get install -y apt-utils + +# Install and configure locale `en_US.UTF-8` +RUN apt-get install -y locales && \ + sed -i -e "s/# $en_US.*/en_US.UTF-8 UTF-8/" /etc/locale.gen && \ + dpkg-reconfigure --frontend=noninteractive locales && \ + update-locale LANG=en_US.UTF-8 +ENV LANG=en_US.UTF-8 + +RUN apt-get update +RUN apt-get install -y git python2 python-pip g++ cmake python-ply python-tk tix pkg-config libssl-dev python-setuptools + +# create a user: +RUN useradd -ms /bin/bash user +USER user +WORKDIR /home/user + +RUN git clone --recurse-submodules https://github.com/kenmcmil/ivy.git +WORKDIR /home/user/ivy/ +RUN git checkout 271ee38980699115508eb90a0dd01deeb750a94b + +RUN python2.7 build_submodules.py +RUN mkdir -p "/home/user/python/lib/python2.7/site-packages" +ENV PYTHONPATH="/home/user/python/lib/python2.7/site-packages" +# need to install pyparsing manually because otherwise wrong version found +RUN pip install pyparsing +RUN python2.7 setup.py install --prefix="/home/user/python/" +ENV PATH=$PATH:"/home/user/python/bin/" +WORKDIR /home/user/tendermint-proof/ + +ENTRYPOINT ["/home/user/tendermint-proof/check_proofs.sh"] + diff --git a/cometbft/v0.39/spec/ivy-proofs/README.md b/cometbft/v0.39/spec/ivy-proofs/README.md new file mode 100644 index 000000000..00a4bed25 --- /dev/null +++ b/cometbft/v0.39/spec/ivy-proofs/README.md @@ -0,0 +1,33 @@ +# Ivy Proofs + +```copyright +Copyright (c) 2020 Galois, Inc. +SPDX-License-Identifier: Apache-2.0 +``` + +## Contents + +This folder contains: + +* `tendermint.ivy`, a specification of Tendermint algorithm as described in *The latest gossip on BFT consensus* by E. Buchman, J. Kwon, Z. Milosevic. +* `abstract_tendermint.ivy`, a more abstract specification of Tendermint that is more verification-friendly. +* `classic_safety.ivy`, a proof that Tendermint satisfies the classic safety property of BFT consensus: if every two quorums have a well-behaved node in common, then no two well-behaved nodes ever disagree. +* `accountable_safety_1.ivy`, a proof that, assuming every quorum contains at least one well-behaved node, if two well-behaved nodes disagree, then there is evidence demonstrating at least f+1 nodes misbehaved. +* `accountable_safety_2.ivy`, a proof that, regardless of any assumption about quorums, well-behaved nodes cannot be framed by malicious nodes. In other words, malicious nodes can never construct evidence that incriminates a well-behaved node. +* `network_shim.ivy`, the network model and a convenience `shim` object to interface with the Tendermint specification. +* `domain_model.ivy`, a specification of the domain model underlying the Tendermint specification, i.e. rounds, value, quorums, etc. + +All specifications and proofs are written in [Ivy](https://github.com/kenmcmil/ivy). + +The license above applies to all files in this folder. + + +## Building and running + +The easiest way to check the proofs is to use [Docker](https://www.docker.com/). + +1. Install [Docker](https://docs.docker.com/get-docker/) and [Docker Compose](https://docs.docker.com/compose/install/). +2. Build a Docker image: `docker-compose build` +3. Run the proofs inside the Docker container: `docker-compose run +tendermint-proof`. This will check all the proofs with the `ivy_check` +command and write the output of `ivy_check` to a subdirectory of `./output/' diff --git a/cometbft/v0.39/spec/ivy-proofs/abstract_tendermint.ivy b/cometbft/v0.39/spec/ivy-proofs/abstract_tendermint.ivy new file mode 100644 index 000000000..4a160be2a --- /dev/null +++ b/cometbft/v0.39/spec/ivy-proofs/abstract_tendermint.ivy @@ -0,0 +1,178 @@ +#lang ivy1.7 +# --- +# layout: page +# title: Abstract specification of Tendermint in Ivy +# --- + +# Here we define an abstract version of the Tendermint specification. We use +# two main forms of abstraction: a) We abstract over how information is +# transmitted (there is no network). b) We abstract functions using relations. +# For example, we abstract over a node's current round, instead only tracking +# with a relation which rounds the node has left. We do something similar for +# the `lockedRound` variable. This is in order to avoid using a function from +# node to round, and it allows us to emit verification conditions that are +# efficiently solvable by Z3. + +# This specification also defines the observations that are used to adjudicate +# misbehavior. Well-behaved nodes faithfully observe every message that they +# use to take a step, while Byzantine nodes can fake observations about +# themselves (including withholding observations). Misbehavior is defined using +# the collection of all observations made (in reality, those observations must +# be collected first, but we do not model this process). + +include domain_model + +module abstract_tendermint = { + +# Protocol state +# ############## + + relation left_round(N:node, R:round) + relation prevoted(N:node, R:round, V:value) + relation precommitted(N:node, R:round, V:value) + relation decided(N:node, R:round, V:value) + relation locked(N:node, R:round, V:value) + +# Accountability relations +# ######################## + + relation observed_prevoted(N:node, R:round, V:value) + relation observed_precommitted(N:node, R:round, V:value) + +# relations that are defined in terms of the previous two: + relation observed_equivocation(N:node) + relation observed_unlawful_prevote(N:node) + relation agreement + relation accountability_violation + + object defs = { # we hide those definitions and use them only when needed + private { + definition [observed_equivocation_def] observed_equivocation(N) = exists V1,V2,R . + V1 ~= V2 & (observed_precommitted(N,R,V1) & observed_precommitted(N,R,V2) | observed_prevoted(N,R,V1) & observed_prevoted(N,R,V2)) + + definition [observed_unlawful_prevote_def] observed_unlawful_prevote(N) = exists V1,V2,R1,R2 . + V1 ~= value.nil & V2 ~= value.nil & V1 ~= V2 & R1 < R2 & observed_precommitted(N,R1,V1) & observed_prevoted(N,R2,V2) + & forall Q,R . R1 <= R & R < R2 & nset.is_quorum(Q) -> exists N2 . nset.member(N2,Q) & ~observed_prevoted(N2,R,V2) + + definition [agreement_def] agreement = forall N1,N2,R1,R2,V1,V2 . well_behaved(N1) & well_behaved(N2) & decided(N1,R1,V1) & decided(N2,R2,V2) -> V1 = V2 + + definition [accountability_violation_def] accountability_violation = exists Q1,Q2 . nset.is_quorum(Q1) & nset.is_quorum(Q2) & (forall N . nset.member(N,Q1) & nset.member(N,Q2) -> observed_equivocation(N) | observed_unlawful_prevote(N)) + } + } + +# Protocol transitions +# #################### + + after init { + left_round(N,R) := R < 0; + prevoted(N,R,V) := false; + precommitted(N,R,V) := false; + decided(N,R,V) := false; + locked(N,R,V) := false; + + observed_prevoted(N,R,V) := false; + observed_precommitted(N,R,V) := false; + } + +# Actions are named after the corresponding line numbers in the Tendermint +# arXiv paper. + + action l_11(n:node, r:round) = { # start round r + require ~left_round(n,r); + left_round(n,R) := R < r; + } + + action l_22(n:node, rp:round, v:value) = { + require ~left_round(n,rp); + require ~prevoted(n,rp,V) & ~precommitted(n,rp,V); + require (forall R,V . locked(n,R,V) -> V = v) | v = value.nil; + prevoted(n, rp, v) := true; + left_round(n, R) := R < rp; # leave all lower rounds. + + observed_prevoted(n, rp, v) := observed_prevoted(n, rp, v) | well_behaved(n); # the node observes itself + } + + action l_28(n:node, rp:round, v:value, vr:round, q:nset) = { + require ~left_round(n,rp) & ~prevoted(n,rp,V); + require ~prevoted(n,rp,V) & ~precommitted(n,rp,V); + require vr < rp; + require nset.is_quorum(q) & (forall N . nset.member(N,q) -> (prevoted(N,vr,v) | ~well_behaved(N))); + var proposal:value; + if value.valid(v) & ((forall R0,V0 . locked(n,R0,V0) -> R0 <= vr) | (forall R,V . locked(n,R,V) -> V = v)) { + proposal := v; + } + else { + proposal := value.nil; + }; + prevoted(n, rp, proposal) := true; + left_round(n, R) := R < rp; # leave all lower rounds + + observed_prevoted(N, vr, v) := observed_prevoted(N, vr, v) | (well_behaved(n) & nset.member(N,q)); # the node observes the prevotes of quorum q + observed_prevoted(n, rp, proposal) := observed_prevoted(n, rp, proposal) | well_behaved(n); # the node observes itself + } + + action l_36(n:node, rp:round, v:value, q:nset) = { + require v ~= value.nil; + require ~left_round(n,rp); + require exists V . prevoted(n,rp,V); + require ~precommitted(n,rp,V); + require nset.is_quorum(q) & (forall N . nset.member(N,q) -> (prevoted(N,rp,v) | ~well_behaved(N))); + precommitted(n, rp, v) := true; + left_round(n, R) := R < rp; # leave all lower rounds + locked(n,R,V) := R <= rp & V = v; + + observed_prevoted(N, rp, v) := observed_prevoted(N, rp, v) | (well_behaved(n) & nset.member(N,q)); # the node observes the prevotes of quorum q + observed_precommitted(n, rp, v) := observed_precommitted(n, rp, v) | well_behaved(n); # the node observes itself + } + + action l_44(n:node, rp:round, q:nset) = { + require ~left_round(n,rp); + require ~precommitted(n,rp,V); + require nset.is_quorum(q) & (forall N .nset.member(N,q) -> (prevoted(N,rp,value.nil) | ~well_behaved(N))); + precommitted(n, rp, value.nil) := true; + left_round(n, R) := R < rp; # leave all lower rounds + + observed_prevoted(N, rp, value.nil) := observed_prevoted(N, rp, value.nil) | (well_behaved(n) & nset.member(N,q)); # the node observes the prevotes of quorum q + observed_precommitted(n, rp, value.nil) := observed_precommitted(n, rp, value.nil) | well_behaved(n); # the node observes itself + } + + action l_57(n:node, rp:round) = { + require ~left_round(n,rp); + require ~prevoted(n,rp,V); + prevoted(n, rp, value.nil) := true; + left_round(n, R) := R < rp; # leave all lower rounds + + observed_prevoted(n, rp, value.nil) := observed_prevoted(n, rp, value.nil) | well_behaved(n); # the node observes itself + } + + action l_61(n:node, rp:round) = { + require ~left_round(n,rp); + require ~precommitted(n,rp,V); + precommitted(n, rp, value.nil) := true; + left_round(n, R) := R < rp; # leave all lower rounds + + observed_precommitted(n, rp, value.nil) := observed_precommitted(n, rp, value.nil) | well_behaved(n); # the node observes itself + } + + action decide(n:node, r:round, v:value, q:nset) = { + require v ~= value.nil; + require nset.is_quorum(q) & (forall N . nset.member(N, q) -> (precommitted(N, r, v) | ~well_behaved(N))); + decided(n, r, v) := true; + + observed_precommitted(N, r, v) := observed_precommitted(N, r, v) | (well_behaved(n) & nset.member(N,q)); # the node observes the precommits of quorum q + + } + + action misbehave = { +# Byzantine nodes can claim they observed whatever they want about themselves, +# but they cannot remove observations. Note that we use assume because we don't +# want those to be checked; we just want them to be true (that's the model of +# Byzantine behavior). + observed_prevoted(N,R,V) := *; + assume (old observed_prevoted(N,R,V)) -> observed_prevoted(N,R,V); + assume well_behaved(N) -> old observed_prevoted(N,R,V) = observed_prevoted(N,R,V); + observed_precommitted(N,R,V) := *; + assume (old observed_precommitted(N,R,V)) -> observed_precommitted(N,R,V); + assume well_behaved(N) -> old observed_precommitted(N,R,V) = observed_precommitted(N,R,V); + } +} diff --git a/cometbft/v0.39/spec/ivy-proofs/accountable_safety_1.ivy b/cometbft/v0.39/spec/ivy-proofs/accountable_safety_1.ivy new file mode 100644 index 000000000..02bdf1add --- /dev/null +++ b/cometbft/v0.39/spec/ivy-proofs/accountable_safety_1.ivy @@ -0,0 +1,143 @@ +#lang ivy1.7 +# --- +# layout: page +# title: Proof of Classic Safety +# --- + +include tendermint +include abstract_tendermint + +# Here we prove the first accountability property: if two well-behaved nodes +# disagree, then there are two quorums Q1 and Q2 such that all members of the +# intersection of Q1 and Q2 have violated the accountability properties. + +# The proof is done in two steps: first we prove the abstract specification +# satisfies the property, and then we show by refinement that this property +# also holds in the concrete specification. + +# To see what is checked in the refinement proof, use `ivy_show isolate=accountable_safety_1 accountable_safety_1.ivy` +# To see what is checked in the abstract correctness proof, use `ivy_show isolate=abstract_accountable_safety_1 accountable_safety_1.ivy` +# To check the whole proof, use `ivy_check accountable_safety_1.ivy`. + + +# Proof of the accountability property in the abstract specification +# ================================================================== + +# We prove with tactics (see `lemma_1` and `lemma_2`) that, if some basic +# invariants hold (see `invs` below), then the accountability property holds. + +isolate abstract_accountable_safety = { + + instantiate abstract_tendermint + +# The main property +# ----------------- + +# If there is disagreement, then there is evidence that a third of the nodes +# have violated the protocol: + invariant [accountability] agreement | accountability_violation + proof { + apply lemma_1.thm # this reduces to goal to three subgoals: p1, p2, and p3 (see their definition below) + proof [p1] { + assume invs.inv1 + } + proof [p2] { + assume invs.inv2 + } + proof [p3] { + assume invs.inv3 + } + } + +# The invariants +# -------------- + + isolate invs = { + + # well-behaved nodes observe their own actions faithfully: + invariant [inv1] well_behaved(N) -> (observed_precommitted(N,R,V) = precommitted(N,R,V)) + # if a value is precommitted by a well-behaved node, then a quorum is observed to prevote it: + invariant [inv2] (exists N . well_behaved(N) & precommitted(N,R,V)) & V ~= value.nil -> exists Q . nset.is_quorum(Q) & forall N2 . nset.member(N2,Q) -> observed_prevoted(N2,R,V) + # if a value is decided by a well-behaved node, then a quorum is observed to precommit it: + invariant [inv3] (exists N . well_behaved(N) & decided(N,R,V)) -> 0 <= R & V ~= value.nil & exists Q . nset.is_quorum(Q) & forall N2 . nset.member(N2,Q) -> observed_precommitted(N2,R,V) + private { + invariant (precommitted(N,R,V) | prevoted(N,R,V)) -> 0 <= R + invariant R < 0 -> left_round(N,R) + } + + } with this, nset, round, accountable_bft.max_2f_byzantine + +# The theorems proved with tactics +# -------------------------------- + +# Using complete induction on rounds, we prove that, assuming that the +# invariants inv1, inv2, and inv3 hold, the accountability property holds. + +# For technical reasons, we separate the proof in two steps + isolate lemma_1 = { + + specification { + theorem [thm] { + property [p1] forall N,R,V . well_behaved(N) -> (observed_precommitted(N,R,V) = precommitted(N,R,V)) + property [p2] forall R,V . (exists N . well_behaved(N) & precommitted(N,R,V)) & V ~= value.nil -> exists Q . nset.is_quorum(Q) & forall N2 . nset.member(N2,Q) -> observed_prevoted(N2,R,V) + property [p3] forall R,V. (exists N . well_behaved(N) & decided(N,R,V)) -> 0 <= R & V ~= value.nil & exists Q . nset.is_quorum(Q) & forall N2 . nset.member(N2,Q) -> observed_precommitted(N2,R,V) + #------------------------------------------------------------------------------------------------------------------------------------------- + property agreement | accountability_violation + } + proof { + assume inductive_property # the theorem follows from what we prove by induction below + } + } + + implementation { + # complete induction is not built-in, so we introduce it with an axiom. Note that this only holds for a type where 0 is the smallest element + axiom [complete_induction] { + relation p(X:round) + { # base case + property p(0) + } + { # inductive step: show that if the property is true for all X lower or equal to x and y=x+1, then the property is true of y + individual a:round + individual b:round + property (forall X. 0 <= X & X <= a -> p(X)) & round.succ(a,b) -> p(b) + } + #-------------------------- + property forall X . 0 <= X -> p(X) + } + + # The main lemma: if inv1 and inv2 below hold and a quorum is observed to + # precommit V1 at R1 and another quorum is observed to precommit V2~=V1 at + # R2>=R1, then the intersection of two quorums (i.e. f+1 nodes) is observed to + # violate the protocol. We prove this by complete induction on R2. + theorem [inductive_property] { + property [p1] forall N,R,V . well_behaved(N) -> (observed_precommitted(N,R,V) = precommitted(N,R,V)) + property [p2] forall R,V . (exists N . well_behaved(N) & precommitted(N,R,V)) -> V = value.nil | exists Q . nset.is_quorum(Q) & forall N2 . nset.member(N2,Q) -> observed_prevoted(N2,R,V) + #----------------------------------------------------------------------------------------------------------------------- + property forall R2. 0 <= R2 -> ((exists V2,Q1,R1,V1,Q1 . V1 ~= value.nil & V2 ~= value.nil & V1 ~= V2 & 0 <= R1 & R1 <= R2 & nset.is_quorum(Q1) & (forall N . nset.member(N,Q1) -> observed_precommitted(N,R1,V1)) & (exists Q2 . nset.is_quorum(Q2) & forall N . nset.member(N,Q2) -> observed_prevoted(N,R2,V2))) -> accountability_violation) + } + proof { + apply complete_induction # the two subgoals (base case and inductive case) are then discharged automatically + # NOTE: this can take a long time depending on the SMT random seed (to try a different seed, use `ivy_check seed=$RANDOM` + } + } + } with this, round, nset, accountable_bft.max_2f_byzantine, defs.observed_equivocation_def, defs.observed_unlawful_prevote_def, defs.accountability_violation_def, defs.agreement_def + +} with round + +# The final proof +# =============== + +isolate accountable_safety_1 = { + +# First we instantiate the concrete protocol: + instantiate tendermint(abstract_accountable_safety) + +# We then define what we mean by agreement + relation agreement + definition [agreement_def] agreement = forall N1,N2. well_behaved(N1) & well_behaved(N2) & server.decision(N1) ~= value.nil & server.decision(N2) ~= value.nil -> server.decision(N1) = server.decision(N2) + + invariant abstract_accountable_safety.agreement -> agreement + + invariant [accountability] agreement | abstract_accountable_safety.accountability_violation + +} with value, round, proposers, shim, abstract_accountable_safety, abstract_accountable_safety.defs.agreement_def, accountable_safety_1.agreement_def diff --git a/cometbft/v0.39/spec/ivy-proofs/accountable_safety_2.ivy b/cometbft/v0.39/spec/ivy-proofs/accountable_safety_2.ivy new file mode 100644 index 000000000..7fb928909 --- /dev/null +++ b/cometbft/v0.39/spec/ivy-proofs/accountable_safety_2.ivy @@ -0,0 +1,52 @@ +#lang ivy1.7 + +include tendermint +include abstract_tendermint + +# Here we prove the second accountability property: no well-behaved node is +# ever observed to violate the accountability properties. + +# The proof is done in two steps: first we prove the the abstract specification +# satisfies the property, and then we show by refinement that this property +# also holds in the concrete specification. + +# To see what is checked in the refinement proof, use `ivy_show isolate=accountable_safety_2 accountable_safety_2.ivy` +# To see what is checked in the abstract correctness proof, use `ivy_show isolate=abstract_accountable_safety_2 accountable_safety_2.ivy` +# To check the whole proof, use `ivy_check complete=fo accountable_safety_2.ivy`. + +# Proof that the property holds in the abstract specification +# ============================================================ + +isolate abstract_accountable_safety_2 = { + + instantiate abstract_tendermint + +# the main property: + invariant [wb_never_punished] well_behaved(N) -> ~(observed_equivocation(N) | observed_unlawful_prevote(N)) + +# the main invariant for proving wb_not_punished: + invariant well_behaved(N) & precommitted(N,R,V) & ~locked(N,R,V) & V ~= value.nil -> exists R2,V2 . V2 ~= value.nil & R < R2 & precommitted(N,R2,V2) & locked(N,R2,V2) + + invariant (exists N . well_behaved(N) & precommitted(N,R,V) & V ~= value.nil) -> exists Q . nset.is_quorum(Q) & forall N . nset.member(N,Q) -> observed_prevoted(N,R,V) + + invariant well_behaved(N) -> (observed_prevoted(N,R,V) <-> prevoted(N,R,V)) + invariant well_behaved(N) -> (observed_precommitted(N,R,V) <-> precommitted(N,R,V)) + +# nodes stop prevoting or precommitting in lower rounds when doing so in a higher round: + invariant well_behaved(N) & prevoted(N,R2,V2) & R1 < R2 -> left_round(N,R1) + invariant well_behaved(N) & locked(N,R2,V2) & R1 < R2 -> left_round(N,R1) + + invariant [precommit_unique_per_round] well_behaved(N) & precommitted(N,R,V1) & precommitted(N,R,V2) -> V1 = V2 + +} with nset, round, abstract_accountable_safety_2.defs.observed_equivocation_def, abstract_accountable_safety_2.defs.observed_unlawful_prevote_def + +# Proof that the property holds in the concrete specification +# =========================================================== + +isolate accountable_safety_2 = { + + instantiate tendermint(abstract_accountable_safety_2) + + invariant well_behaved(N) -> ~(abstract_accountable_safety_2.observed_equivocation(N) | abstract_accountable_safety_2.observed_unlawful_prevote(N)) + +} with round, value, shim, abstract_accountable_safety_2, abstract_accountable_safety_2.defs.observed_equivocation_def, abstract_accountable_safety_2.defs.observed_unlawful_prevote_def diff --git a/cometbft/v0.39/spec/ivy-proofs/check_proofs.sh b/cometbft/v0.39/spec/ivy-proofs/check_proofs.sh new file mode 100755 index 000000000..6afd1a962 --- /dev/null +++ b/cometbft/v0.39/spec/ivy-proofs/check_proofs.sh @@ -0,0 +1,39 @@ +#!/bin/bash + +# returns non-zero error code if any proof fails + +success=0 +log_dir=$(cat /dev/urandom | tr -cd 'a-f0-9' | head -c 6) +cmd="ivy_check seed=$RANDOM" +mkdir -p output/$log_dir + +echo "Checking classic safety:" +res=$($cmd classic_safety.ivy | tee "output/$log_dir/classic_safety.txt" | tail -n 1) +if [ "$res" = "OK" ]; then + echo "OK" +else + echo "FAILED" + success=1 +fi + +echo "Checking accountable safety 1:" +res=$($cmd accountable_safety_1.ivy | tee "output/$log_dir/accountable_safety_1.txt" | tail -n 1) +if [ "$res" = "OK" ]; then + echo "OK" +else + echo "FAILED" + success=1 +fi + +echo "Checking accountable safety 2:" +res=$($cmd complete=fo accountable_safety_2.ivy | tee "output/$log_dir/accountable_safety_2.txt" | tail -n 1) +if [ "$res" = "OK" ]; then + echo "OK" +else + echo "FAILED" + success=1 +fi + +echo +echo "See ivy_check output in the output/ folder" +exit $success diff --git a/cometbft/v0.39/spec/ivy-proofs/classic_safety.ivy b/cometbft/v0.39/spec/ivy-proofs/classic_safety.ivy new file mode 100644 index 000000000..b422a2c17 --- /dev/null +++ b/cometbft/v0.39/spec/ivy-proofs/classic_safety.ivy @@ -0,0 +1,85 @@ +#lang ivy1.7 +# --- +# layout: page +# title: Proof of Classic Safety +# --- + +include tendermint +include abstract_tendermint + +# Here we prove the classic safety property: assuming that every two quorums +# have a well-behaved node in common, no two well-behaved nodes ever disagree. + +# The proof is done in two steps: first we prove the the abstract specification +# satisfies the property, and then we show by refinement that this property +# also holds in the concrete specification. + +# To see what is checked in the refinement proof, use `ivy_show isolate=classic_safety classic_safety.ivy` +# To see what is checked in the abstract correctness proof, use `ivy_show isolate=abstract_classic_safety classic_safety.ivy` + +# To check the whole proof, use `ivy_check classic_safety.ivy`. + +# Note that all the verification conditions sent to Z3 for this proof are in +# EPR. + +# Classic safety in the abstract model +# ==================================== + +# We start by proving that classic safety holds in the abstract model. + +isolate abstract_classic_safety = { + + instantiate abstract_tendermint + + invariant [classic_safety] classic_bft.quorum_intersection & decided(N1,R1,V1) & decided(N2,R2,V2) -> V1 = V2 + +# The notion of choosable value +# ----------------------------- + + relation choosable(R:round, V:value) + definition choosable(R,V) = exists Q . nset.is_quorum(Q) & forall N . well_behaved(N) & nset.member(N,Q) -> ~left_round(N,R) | precommitted(N,R,V) + +# Main invariants +# --------------- + +# `classic_safety` is inductive relative to those invariants + + invariant [decision_is_quorum_precommit] (exists N1 . decided(N1,R,V)) -> exists Q. nset.is_quorum(Q) & forall N2. well_behaved(N2) & nset.member(N2, Q) -> precommitted(N2,R,V) + + invariant [precommitted_is_quorum_prevote] V ~= value.nil & (exists N1 . precommitted(N1,R,V)) -> exists Q. nset.is_quorum(Q) & forall N2. well_behaved(N2) & nset.member(N2, Q) -> prevoted(N2,R,V) + + invariant [prevote_unique_per_round] prevoted(N,R,V1) & prevoted(N,R,V2) -> V1 = V2 + +# This is the core invariant: as long as a precommitted value is still choosable, it remains protected by a lock and prevents any new value from being prevoted: + invariant [locks] classic_bft.quorum_intersection & V ~= value.nil & precommitted(N,R,V) & choosable(R,V) -> locked(N,R,V) & forall R2,V2 . R < R2 & prevoted(N,R2,V2) -> V2 = V | V2 = value.nil + +# Supporting invariants +# --------------------- + +# The main invariants are inductive relative to those + + invariant decided(N,R,V) -> V ~= value.nil + + invariant left_round(N,R2) & R1 < R2 -> left_round(N,R1) # if a node left round R2>R1, then it also left R1: + + invariant prevoted(N,R2,V2) & R1 < R2 -> left_round(N,R1) + invariant precommitted(N,R2,V2) & R1 < R2 -> left_round(N,R1) + +} with round, nset, classic_bft.quorum_intersection_def + +# The refinement proof +# ==================== + +# Now, thanks to the refinement relation that we establish in +# `concrete_tendermint.ivy`, we prove that classic safety transfers to the +# concrete specification: +isolate classic_safety = { + + # We instantiate the `tendermint` module providing `abstract_classic_safety` as abstract model. + instantiate tendermint(abstract_classic_safety) + + # We prove that if every two quorums have a well-behaved node in common, + # then well-behaved nodes never disagree: + invariant [classic_safety] classic_bft.quorum_intersection & server.decision(N1) ~= value.nil & server.decision(N2) ~= value.nil -> server.decision(N1) = server.decision(N2) + +} with value, round, proposers, shim, abstract_classic_safety # here we list all the specifications that we rely on for this proof diff --git a/cometbft/v0.39/spec/ivy-proofs/count_lines.sh b/cometbft/v0.39/spec/ivy-proofs/count_lines.sh new file mode 100755 index 000000000..b2c457e21 --- /dev/null +++ b/cometbft/v0.39/spec/ivy-proofs/count_lines.sh @@ -0,0 +1,13 @@ +#!/bin/bash + +r='^\s*$\|^\s*\#\|^\s*\}\s*$\|^\s*{\s*$' # removes comments and blank lines and lines that contain only { or } +N1=`cat tendermint.ivy domain_model.ivy network_shim.ivy | grep -v $r'\|.*invariant.*' | wc -l` +N2=`cat abstract_tendermint.ivy | grep "observed_" | wc -l` # the observed_* variables specify the observations of the nodes +SPEC_LINES=`expr $N1 + $N2` +echo "spec lines: $SPEC_LINES" +N3=`cat abstract_tendermint.ivy | grep -v $r'\|.*observed_.*' | wc -l` +N4=`cat accountable_safety_1.ivy | grep -v $r | wc -l` +PROOF_LINES=`expr $N3 + $N4` +echo "proof lines: $PROOF_LINES" +RATIO=`bc <<< "scale=2;$PROOF_LINES / $SPEC_LINES"` +echo "proof-to-code ratio for the accountable-safety property: $RATIO" diff --git a/cometbft/v0.39/spec/ivy-proofs/docker-compose.yml b/cometbft/v0.39/spec/ivy-proofs/docker-compose.yml new file mode 100644 index 000000000..e0612d4b1 --- /dev/null +++ b/cometbft/v0.39/spec/ivy-proofs/docker-compose.yml @@ -0,0 +1,7 @@ +version: '3' +services: + tendermint-proof: + build: . + volumes: + - ./:/home/user/tendermint-proof:ro + - ./output:/home/user/tendermint-proof/output:rw diff --git a/cometbft/v0.39/spec/ivy-proofs/domain_model.ivy b/cometbft/v0.39/spec/ivy-proofs/domain_model.ivy new file mode 100644 index 000000000..1fd3cc99e --- /dev/null +++ b/cometbft/v0.39/spec/ivy-proofs/domain_model.ivy @@ -0,0 +1,144 @@ +#lang ivy1.7 + +include order # this is a file from the standard library (`ivy/ivy/include/1.7/order.ivy`) + +isolate round = { + type this + individual minus_one:this + relation succ(R1:round, R2:round) + action incr(i:this) returns (j:this) + specification { +# to simplify verification, we treat rounds as an abstract totally ordered set with a successor relation. + instantiate totally_ordered(this) + property minus_one < 0 + property succ(X,Z) -> (X < Z & ~(X < Y & Y < Z)) + after incr { + ensure succ(i,j) + } + } + implementation { +# here we prove that the abstraction is sound. + interpret this -> int # rounds are integers in the Tendermint specification. + definition minus_one = 0-1 + definition succ(R1,R2) = R2 = R1 + 1 + implement incr { + j := i+1; + } + } +} + +instance node : iterable # nodes are a set with an order, that can be iterated over (see order.ivy in the standard library) + +relation well_behaved(N:node) # whether a node is well-behaved or not. NOTE: Used only in the proof and the Byzantine model; Nodes do know know who is well-behaved and who is not. + +isolate proposers = { + # each round has a unique proposer in Tendermint. In order to avoid a + # function from round to node (which makes verification more difficult), we + # abstract over this function using a relation. + relation is_proposer(N:node, R:round) + export action get_proposer(r:round) returns (n:node) + specification { + property is_proposer(N1,R) & is_proposer(N2,R) -> N1 = N2 + after get_proposer { + ensure is_proposer(n,r); + } + } + implementation { + function f(R:round):node + definition f(r:round) = <<>> + definition is_proposer(N,R) = N = f(R) + implement get_proposer { + n := f(r); + } + } +} + +isolate value = { # the type of values + type this + relation valid(V:value) + individual nil:value + specification { + property ~valid(nil) + } + implementation { + interpret value -> bv[2] + definition nil = <<< -1 >>> # let's say nil is -1 + definition valid(V) = V ~= nil + } +} + +object nset = { # the type of node sets + type this # a set of N=3f+i nodes for 0 + #include + namespace hash_space { + template + class hash > { + public: + size_t operator()(const std::set &s) const { + hash h; + size_t res = 0; + for (const T &e : s) + res += h(e); + return res; + } + }; + } + >>> + interpret nset -> <<< std::set<`node`> >>> + definition member(n:node, s:nset) = <<< `s`.find(`n`) != `s`.end() >>> + definition is_quorum(s:nset) = <<< 3*`s`.size() > 2*`node.size` >>> + definition is_blocking(s:nset) = <<< 3*`s`.size() > `node.size` >>> + implement empty { + <<< + >>> + } + implement insert { + <<< + `t` = `s`; + `t`.insert(`n`); + >>> + } + <<< encode `nset` + + std::ostream &operator <<(std::ostream &s, const `nset` &a) { + s << "{"; + for (auto iter = a.begin(); iter != a.end(); iter++) { + if (iter != a.begin()) s << ", "; + s << *iter; + } + s << "}"; + return s; + } + + template <> + `nset` _arg<`nset`>(std::vector &args, unsigned idx, long long bound) { + throw std::invalid_argument("Not implemented"); // no syntax for nset values in the REPL + } + + >>> + } +} + +object classic_bft = { + relation quorum_intersection + private { + definition [quorum_intersection_def] quorum_intersection = forall Q1,Q2. nset.is_quorum(Q1) & nset.is_quorum(Q2) + -> exists N. well_behaved(N) & nset.member(N, Q1) & nset.member(N, Q2) # every two quorums have a well-behaved node in common + } +} + +trusted isolate accountable_bft = { + # this is our baseline assumption about quorums: + private { + property [max_2f_byzantine] nset.is_quorum(Q) -> exists N . well_behaved(N) & nset.member(N,Q) # every quorum has a well-behaved member + } +} diff --git a/cometbft/v0.39/spec/ivy-proofs/network_shim.ivy b/cometbft/v0.39/spec/ivy-proofs/network_shim.ivy new file mode 100644 index 000000000..ebc3a04fc --- /dev/null +++ b/cometbft/v0.39/spec/ivy-proofs/network_shim.ivy @@ -0,0 +1,133 @@ +#lang ivy1.7 +# --- +# layout: page +# title: Network model and network shim +# --- + +# Here we define a network module, which is our model of the network, and a +# shim module that sits on top of the network and which, upon receiving a +# message, calls the appropriate protocol handler. + +include domain_model + +# Here we define an enumeration type for identifying the 3 different types of +# messages that nodes send. +object msg_kind = { # TODO: merge with step_t + type this = {proposal, prevote, precommit} +} + +# Here we define the type of messages `msg`. Its members are structs with the fields described below. +object msg = { + type this = struct { + m_kind : msg_kind, + m_src : node, + m_round : round, + m_value : value, + m_vround : round + } +} + +# This is our model of the network: +isolate net = { + + export action recv(dst:node,v:msg) + action send(src:node,dst:node,v:msg) + # Note that the `recv` action is exported, meaning that it can be called + # non-deterministically by the environment any time it is enabled. In other + # words, a packet that is in flight can be received at any time. In this + # sense, the network is fully asynchronous. Moreover, there is no + # requirement that a given message will be received at all. + + # The state of the network consists of all the packets that have been + # sent so far, along with their destination. + relation sent(V:msg, N:node) + + after init { + sent(V, N) := false + } + + before send { + sent(v,dst) := true + } + + before recv { + require sent(v,dst) # only sent messages can be received. + } +} + +# The network shim sits on top of the network and, upon receiving a message, +# calls the appropriate protocol handler. It also exposes a `broadcast` action +# that sends to all nodes. + +isolate shim = { + + # In order not repeat the same code for each handler, we use a handler + # module parameterized by the type of message it will handle. Below we + # instantiate this module for the 3 types of messages of Tendermint + module handler(p_kind) = { + action handle(dst:node,m:msg) + object spec = { + before handle { + assert sent(m,dst) & m.m_kind = p_kind + } + } + } + + instance proposal_handler : handler(msg_kind.proposal) + instance prevote_handler : handler(msg_kind.prevote) + instance precommit_handler : handler(msg_kind.precommit) + + relation sent(M:msg,N:node) + + action broadcast(src:node,m:msg) + action send(src:node,dst:node,m:msg) + + specification { + after init { + sent(M,D) := false; + } + before broadcast { + sent(m,D) := true + } + before send { + sent(m,dst) := true + } + } + + # Here we give an implementation of it that satisfies its specification: + implementation { + + implement net.recv(dst:node,m:msg) { + + if m.m_kind = msg_kind.proposal { + call proposal_handler.handle(dst,m) + } + else if m.m_kind = msg_kind.prevote { + call prevote_handler.handle(dst,m) + } + else if m.m_kind = msg_kind.precommit { + call precommit_handler.handle(dst,m) + } + } + + implement broadcast { # broadcast sends to all nodes, including the sender. + var iter := node.iter.create(0); + while ~iter.is_end + invariant net.sent(M,D) -> sent(M,D) + { + var n := iter.val; + call net.send(src,n,m); + iter := iter.next; + } + } + + implement send { + call net.send(src,dst,m) + } + + private { + invariant net.sent(M,D) -> sent(M,D) + } + } + +} with net, node # to prove that the shim implementation satisfies the shim specification, we rely on the specification of net and node. diff --git a/cometbft/v0.39/spec/ivy-proofs/output/.gitignore b/cometbft/v0.39/spec/ivy-proofs/output/.gitignore new file mode 100644 index 000000000..5e7d2734c --- /dev/null +++ b/cometbft/v0.39/spec/ivy-proofs/output/.gitignore @@ -0,0 +1,4 @@ +# Ignore everything in this directory +* +# Except this file +!.gitignore diff --git a/cometbft/v0.39/spec/ivy-proofs/tendermint.ivy b/cometbft/v0.39/spec/ivy-proofs/tendermint.ivy new file mode 100644 index 000000000..b7678bef9 --- /dev/null +++ b/cometbft/v0.39/spec/ivy-proofs/tendermint.ivy @@ -0,0 +1,420 @@ +#lang ivy1.7 +# --- +# layout: page +# title: Specification of Tendermint in Ivy +# --- + +# This specification closely follows the pseudo-code given in "The latest +# gossip on BFT consensus" by E. Buchman, J. Kwon, Z. Milosevic +# + +include domain_model +include network_shim + +# We model the Tendermint protocol as an Ivy object. Like in Object-Oriented +# Programming, the basic structuring unit in Ivy is the object. Objects have +# internal state and actions (i.e. methods in OO parlance) that modify their +# state. We model Tendermint as an object whose actions represent steps taken +# by individual nodes in the protocol. Actions in Ivy can have preconditions, +# and a valid execution is a sequence of actions whose preconditions are all +# satisfied in the state in which they are called. + +# For technical reasons, we define below a `tendermint` module instead of an +# object. Ivy modules are a little bit like classes in OO programs, and like +# classes they can be instantiated to obtain objects. To instantiate the +# `tendermint` module, we must provide an abstract-protocol object. This allows +# us to use different abstract-protocol objects for different parts of the +# proof, and to do so without too much notational burden (we could have used +# Ivy monitors, but then we would need to prefix every variable name by the +# name of the object containing it, which clutters things a bit compared to the +# approach we took). + +# The abstract-protocol object is called by the resulting tendermint object so +# as to run the abstract protocol alongside the concrete protocol. This allows +# us to transfer properties proved of the abstract protocol to the concrete +# protocol, as follows. First, we prove that running the abstract protocol in +# this way results in a valid execution of the abstract protocol. This is done +# by checking that all preconditions of the abstract actions are satisfied at +# their call sites. Second, we establish a relation between abstract state and +# concrete state (in the form of invariants of the resulting, two-object +# transition system) that allow us to transfer properties proved in the +# abstract protocol to the concrete protocol (for example, we prove that any +# decision made in the Tendermint protocol is also made in the abstract +# protocol; if the abstract protocol satisfies the agreement property, this +# allows us to conclude that the Tendermint protocol also does). + +# The abstract protocol object that we will use is always the same, and only +# the abstract properties that we prove about it change in the different +# instantiations of the `tendermint` module. Thus we provide common invariants +# that a) allow to prove that the abstract preconditions are met, and b) +# provide a refinement relation (see end of the module) relating the state of +# Tendermint to the state of the abstract protocol. + +# In the model, Byzantine nodes can send whatever messages they want, except +# that they cannot forge sender identities. This reflects the fact that, in +# practice, nodes use public key cryptography to sign their messages. + +# Finally, note that the observations that serve to adjudicate misbehavior are +# defined only in the abstract protocol (they happen in the abstract actions). + +module tendermint(abstract_protocol) = { + + # the initial value of a node: + function init_val(N:node): value + + # the three type of steps + object step_t = { + type this = {propose, prevote, precommit} + } # refer to those e.g. as step_t.propose + + object server(n:node) = { + + # the current round of a node + individual round_p: round + + individual step: step_t + + individual decision: value + + individual lockedValue: value + individual lockedRound: round + + individual validValue: value + individual validRound: round + + + relation done_l34(R:round) + relation done_l36(R:round, V:value) + relation done_l47(R:round) + + # variables for scheduling request + relation propose_timer_scheduled(R:round) + relation prevote_timer_scheduled(R:round) + relation precommit_timer_scheduled(R:round) + + relation _recved_proposal(Sender:node, R:round, V:value, VR:round) + relation _recved_prevote(Sender:node, R:round, V:value) + relation _recved_precommit(Sender:node, R:round, V:value) + + relation _has_started + + after init { + round_p := 0; + step := step_t.propose; + decision := value.nil; + + lockedValue := value.nil; + lockedRound := round.minus_one; + + validValue := value.nil; + validRound := round.minus_one; + + done_l34(R) := false; + done_l36(R, V) := false; + done_l47(R) := false; + + propose_timer_scheduled(R) := false; + prevote_timer_scheduled(R) := false; + precommit_timer_scheduled(R) := false; + + _recved_proposal(Sender, R, V, VR) := false; + _recved_prevote(Sender, R, V) := false; + _recved_precommit(Sender, R, V) := false; + + _has_started := false; + } + + action getValue returns (v:value) = { + v := init_val(n) + } + + export action start = { + require ~_has_started; + _has_started := true; + # line 10 + call startRound(0); + } + + # line 11-21 + action startRound(r:round) = { + # line 12 + round_p := r; + + # line 13 + step := step_t.propose; + + var proposal : value; + + # line 14 + if (proposers.get_proposer(r) = n) { + if validValue ~= value.nil { # line 15 + proposal := validValue; # line 16 + } else { + proposal := getValue(); # line 18 + }; + call broadcast_proposal(r, proposal, validRound); # line 19 + } else { + propose_timer_scheduled(r) := true; # line 21 + }; + + call abstract_protocol.l_11(n, r); + } + + # This action, as not exported, can only be called at specific call sites. + action broadcast_proposal(r:round, v:value, vr:round) = { + var m: msg; + m.m_kind := msg_kind.proposal; + m.m_src := n; + m.m_round := r; + m.m_value := v; + m.m_vround := vr; + call shim.broadcast(n,m); + } + + implement shim.proposal_handler.handle(msg:msg) { + _recved_proposal(msg.m_src, msg.m_round, msg.m_value, msg.m_vround) := true; + } + + # line 22-27 + export action l_22(v:value) = { + require _has_started; + require _recved_proposal(proposers.get_proposer(round_p), round_p, v, round.minus_one); + require step = step_t.propose; + + if (value.valid(v) & (lockedRound = round.minus_one | lockedValue = v)) { + call broadcast_prevote(round_p, v); # line 24 + call abstract_protocol.l_22(n, round_p, v); + } else { + call broadcast_prevote(round_p, value.nil); # line 26 + call abstract_protocol.l_22(n, round_p, value.nil); + }; + + # line 27 + step := step_t.prevote; + } + + # line 28-33 + export action l_28(r:round, v:value, vr:round, q:nset) = { + require _has_started; + require r = round_p; + require _recved_proposal(proposers.get_proposer(r), r, v, vr); + require nset.is_quorum(q); + require nset.member(N,q) -> _recved_prevote(N,vr,v); + require step = step_t.propose; + require vr >= 0 & vr < r; + + # line 29 + if (value.valid(v) & (lockedRound <= vr | lockedValue = v)) { + call broadcast_prevote(r, v); + } else { + call broadcast_prevote(r, value.nil); + }; + + call abstract_protocol.l_28(n,r,v,vr,q); + step := step_t.prevote; + } + + action broadcast_prevote(r:round, v:value) = { + var m: msg; + m.m_kind := msg_kind.prevote; + m.m_src := n; + m.m_round := r; + m.m_value := v; + call shim.broadcast(n,m); + } + + implement shim.prevote_handler.handle(msg:msg) { + _recved_prevote(msg.m_src, msg.m_round, msg.m_value) := true; + } + + # line 34-35 + export action l_34(r:round, q:nset) = { + require _has_started; + require round_p = r; + require nset.is_quorum(q); + require exists V . nset.member(N,q) -> _recved_prevote(N,r,V); + require step = step_t.prevote; + require ~done_l34(r); + done_l34(r) := true; + + prevote_timer_scheduled(r) := true; + } + + + # line 36-43 + export action l_36(r:round, v:value, q:nset) = { + require _has_started; + require r = round_p; + require exists VR . round.minus_one <= VR & VR < r & _recved_proposal(proposers.get_proposer(r), r, v, VR); + require nset.is_quorum(q); + require nset.member(N,q) -> _recved_prevote(N,r,v); + require value.valid(v); + require step = step_t.prevote | step = step_t.precommit; + + require ~done_l36(r,v); + done_l36(r, v) := true; + + if step = step_t.prevote { + lockedValue := v; # line 38 + lockedRound := r; # line 39 + call broadcast_precommit(r, v); # line 40 + step := step_t.precommit; # line 41 + call abstract_protocol.l_36(n, r, v, q); + }; + + validValue := v; # line 42 + validRound := r; # line 43 + } + + # line 44-46 + export action l_44(r:round, q:nset) = { + require _has_started; + require r = round_p; + require nset.is_quorum(q); + require nset.member(N,q) -> _recved_prevote(N,r,value.nil); + require step = step_t.prevote; + + call broadcast_precommit(r, value.nil); # line 45 + step := step_t.precommit; # line 46 + + call abstract_protocol.l_44(n, r, q); + } + + action broadcast_precommit(r:round, v:value) = { + var m: msg; + m.m_kind := msg_kind.precommit; + m.m_src := n; + m.m_round := r; + m.m_value := v; + call shim.broadcast(n,m); + } + + implement shim.precommit_handler.handle(msg:msg) { + _recved_precommit(msg.m_src, msg.m_round, msg.m_value) := true; + } + + + # line 47-48 + export action l_47(r:round, q:nset) = { + require _has_started; + require round_p = r; + require nset.is_quorum(q); + require nset.member(N,q) -> exists V . _recved_precommit(N,r,V); + require ~done_l47(r); + done_l47(r) := true; + + precommit_timer_scheduled(r) := true; + } + + + # line 49-54 + export action l_49_decide(r:round, v:value, q:nset) = { + require _has_started; + require exists VR . round.minus_one <= VR & VR < r & _recved_proposal(proposers.get_proposer(r), r, v, VR); + require nset.is_quorum(q); + require nset.member(N,q) -> _recved_precommit(N,r,v); + require decision = value.nil; + + if value.valid(v) { + decision := v; + # MORE for next height + call abstract_protocol.decide(n, r, v, q); + } + } + + # line 55-56 + export action l_55(r:round, b:nset) = { + require _has_started; + require nset.is_blocking(b); + require nset.member(N,b) -> exists VR . round.minus_one <= VR & VR < r & exists V . _recved_proposal(N,r,V,VR) | _recved_prevote(N,r,V) | _recved_precommit(N,r,V); + require r > round_p; + call startRound(r); # line 56 + } + + # line 57-60 + export action onTimeoutPropose(r:round) = { + require _has_started; + require propose_timer_scheduled(r); + require r = round_p; + require step = step_t.propose; + call broadcast_prevote(r,value.nil); + step := step_t.prevote; + + call abstract_protocol.l_57(n,r); + + propose_timer_scheduled(r) := false; + } + + # line 61-64 + export action onTimeoutPrevote(r:round) = { + require _has_started; + require prevote_timer_scheduled(r); + require r = round_p; + require step = step_t.prevote; + call broadcast_precommit(r,value.nil); + step := step_t.precommit; + + call abstract_protocol.l_61(n,r); + + prevote_timer_scheduled(r) := false; + } + + # line 65-67 + export action onTimeoutPrecommit(r:round) = { + require _has_started; + require precommit_timer_scheduled(r); + require r = round_p; + call startRound(round.incr(r)); + + precommit_timer_scheduled(r) := false; + } + +# The Byzantine actions +# --------------------- + +# Byzantine nodes can send whatever they want, but they cannot send +# messages on behalf of well-behaved nodes. In practice this is implemented +# using cryptography (e.g. public-key cryptography). + + export action byzantine_send(m:msg, dst:node) = { + require ~well_behaved(n); + require ~well_behaved(m.m_src); # cannot forge the identity of well-behaved nodes + call shim.send(n,dst,m); + } + +# Byzantine nodes can also report fake observations, as defined in the abstract protocol. + export action fake_observations = { + call abstract_protocol.misbehave + } + +# Invariants +# ---------- + +# We provide common invariants that a) allow to prove that the abstract +# preconditions are met, and b) provide a refinement relation. + + + specification { + + invariant 0 <= round_p + invariant abstract_protocol.left_round(n,R) <-> R < round_p + + invariant lockedRound ~= round.minus_one -> forall R,V . abstract_protocol.locked(n,R,V) <-> R <= lockedRound & lockedValue = V + invariant lockedRound = round.minus_one -> forall R,V . ~abstract_protocol.locked(n,R,V) + + invariant forall M:msg . well_behaved(M.m_src) & M.m_kind = msg_kind.prevote & shim.sent(M,N) -> abstract_protocol.prevoted(M.m_src,M.m_round,M.m_value) + invariant well_behaved(N) & _recved_prevote(N,R,V) -> abstract_protocol.prevoted(N,R,V) + invariant forall M:msg . well_behaved(M.m_src) & M.m_kind = msg_kind.precommit & shim.sent(M,N) -> abstract_protocol.precommitted(M.m_src,M.m_round,M.m_value) + invariant well_behaved(N) & _recved_precommit(N,R,V) -> abstract_protocol.precommitted(N,R,V) + + invariant (step = step_t.prevote | step = step_t.propose) -> ~abstract_protocol.precommitted(n,round_p,V) + invariant step = step_t.propose -> ~abstract_protocol.prevoted(n,round_p,V) + invariant step = step_t.prevote -> exists V . abstract_protocol.prevoted(n,round_p,V) + + invariant round_p < R -> ~(abstract_protocol.prevoted(n,R,V) | abstract_protocol.precommitted(n,R,V)) + invariant ~_has_started -> step = step_t.propose & ~(abstract_protocol.prevoted(n,R,V) | abstract_protocol.precommitted(n,R,V)) & round_p = 0 + + invariant decision ~= value.nil -> exists R . abstract_protocol.decided(n,R,decision) + } + } +} diff --git a/cometbft/v0.39/spec/ivy-proofs/tendermint_test.ivy b/cometbft/v0.39/spec/ivy-proofs/tendermint_test.ivy new file mode 100644 index 000000000..1299fc086 --- /dev/null +++ b/cometbft/v0.39/spec/ivy-proofs/tendermint_test.ivy @@ -0,0 +1,127 @@ +#lang ivy1.7 + +include tendermint +include abstract_tendermint + +isolate ghost_ = { + instantiate abstract_tendermint +} + +isolate protocol = { + instantiate tendermint(ghost_) # here we instantiate the parameter of the tendermint module with `ghost_`; however note that we don't extract any code for `ghost_` (it's not in the list of object in the extract, and it's thus sliced away). + implementation { + definition init_val(n:node) = <<< `n`%2 >>> + } + # attribute test = impl +} with ghost_, shim, value, round, proposers + +# Here we run a simple scenario that exhibits an execution in which nodes make +# a decision. We do this to rule out trivial modeling errors. + +# One option to check that this scenario is valid is to run it in Ivy's REPL. +# For this, first compile the scenario: +#```ivyc target=repl isolate=code trace=true tendermint_test.ivy +# Then, run the produced binary (e.g. for 4 nodes): +#``` ./tendermint_test 4 +# Finally, call the action: +#``` scenarios.scenario_1 +# Note that Ivy will check at runtime that all action preconditions are +# satisfied. For example, runing the scenario twice will cause a violation of +# the precondition of the `start` action, because a node cannot start twice +# (see `require ~_has_started` in action `start`). + +# Another possibility would be to run `ivy_check` on the scenario, but that +# does not seem to work at the moment. + +isolate scenarios = { + individual all:nset # will be used as parameter to actions requiring a quorum + + after init { + var iter := node.iter.create(0); + while ~iter.is_end + { + all := all.insert(iter.val); + iter := iter.next; + }; + assert nset.is_quorum(all); # we can also use asserts to make sure we are getting what we expect + } + + export action scenario_1 = { + # all nodes start: + var iter := node.iter.create(0); + while ~iter.is_end + { + call protocol.server.start(iter.val); + iter := iter.next; + }; + # all nodes receive the leader's proposal: + var m:msg; + m.m_kind := msg_kind.proposal; + m.m_src := 0; + m.m_round := 0; + m.m_value := 0; + m.m_vround := round.minus_one; + iter := node.iter.create(0); + while ~iter.is_end + { + call net.recv(iter.val,m); + iter := iter.next; + }; + # all nodes prevote: + iter := node.iter.create(0); + while ~iter.is_end + { + call protocol.server.l_22(iter.val,0); + iter := iter.next; + }; + # all nodes receive each other's prevote messages; + m.m_kind := msg_kind.prevote; + m.m_vround := 0; + iter := node.iter.create(0); + while ~iter.is_end + { + var iter2 := node.iter.create(0); # the sender + while ~iter2.is_end + { + m.m_src := iter2.val; + call net.recv(iter.val,m); + iter2 := iter2.next; + }; + iter := iter.next; + }; + # all nodes precommit: + iter := node.iter.create(0); + while ~iter.is_end + { + call protocol.server.l_36(iter.val,0,0,all); + iter := iter.next; + }; + # all nodes receive each other's pre-commits + m.m_kind := msg_kind.precommit; + iter := node.iter.create(0); + while ~iter.is_end + { + var iter2 := node.iter.create(0); # the sender + while ~iter2.is_end + { + m.m_src := iter2.val; + call net.recv(iter.val,m); + iter2 := iter2.next; + }; + iter := iter.next; + }; + # now all nodes can decide: + iter := node.iter.create(0); + while ~iter.is_end + { + call protocol.server.l_49_decide(iter.val,0,0,all); + iter := iter.next; + }; + } + + # TODO: add more scenarios + +} with round, node, proposers, value, nset, protocol, shim, net + +# extract code = protocol, shim, round, node +extract code = round, node, proposers, value, nset, protocol, shim, net, scenarios diff --git a/cometbft/v0.39/spec/light-client/Accountability.mdx b/cometbft/v0.39/spec/light-client/Accountability.mdx new file mode 100644 index 000000000..ffb880596 --- /dev/null +++ b/cometbft/v0.39/spec/light-client/Accountability.mdx @@ -0,0 +1,307 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/spec/light-client/Accountability' +order: 1 +parent: + title: Accountability + order: 4 +--- + +# Fork accountability + +## Problem Statement + +Tendermint consensus algorithm guarantees the following specifications for all heights: + +* agreement -- no two correct full nodes decide differently. +* validity -- the decided block satisfies the predefined predicate *valid()*. +* termination -- all correct full nodes eventually decide, + +If the faulty validators have less than 1/3 of voting power in the current validator set. In the case where this assumption +does not hold, each of the specification may be violated. + +The agreement property says that for a given height, any two correct validators that decide on a block for that height decide on the same block. That the block was indeed generated by the blockchain, can be verified starting from a trusted (genesis) block, and checking that all subsequent blocks are properly signed. + +However, faulty nodes may forge blocks and try to convince users (light clients) that the blocks had been correctly generated. In addition, Tendermint agreement might be violated in the case where 1/3 or more of the voting power belongs to faulty validators: Two correct validators decide on different blocks. The latter case motivates the term "fork": as Tendermint consensus also agrees on the next validator set, correct validators may have decided on disjoint next validator sets, and the chain branches into two or more partitions (possibly having faulty validators in common) and each branch continues to generate blocks independently of the other. + +We say that a fork is a case in which there are two commits for different blocks at the same height of the blockchain. The problem is to ensure that in those cases we are able to detect faulty validators (and not mistakenly accuse correct validators), and incentivize therefore validators to behave according to the protocol specification. + +**Conceptual Limit.** In order to prove misbehavior of a node, we have to show that the behavior deviates from correct behavior with respect to a given algorithm. Thus, an algorithm that detects misbehavior of nodes executing some algorithm *A* must be defined with respect to algorithm *A*. In our case, *A* is Tendermint consensus (+ other protocols in the infrastructure; e.g., Cosmos full nodes and the Light Client). If the consensus algorithm is changed/updated/optimized in the future, we have to check whether changes to the accountability algorithm are also required. All the discussions in this document are thus inherently specific to Tendermint consensus and the Light Client specification. + +**Q:** Should we distinguish agreement for validators and full nodes for agreement? The case where all correct validators agree on a block, but a correct full node decides on a different block seems to be slightly less severe that the case where two correct validators decide on different blocks. Still, if a contaminated full node becomes validator that may be problematic later on. Also it is not clear how gossiping is impaired if a contaminated full node is on a different branch. + +*Remark.* In the case 1/3 or more of the voting power belongs to faulty validators, also validity and termination can be broken. Termination can be broken if faulty processes just do not send the messages that are needed to make progress. Due to asynchrony, this is not punishable, because faulty validators can always claim they never received the messages that would have forced them to send messages. + +## The Misbehavior of Faulty Validators + +Forks are the result of faulty validators deviating from the protocol. In principle several such deviations can be detected without a fork actually occurring: + +1. double proposal: A faulty proposer proposes two different values (blocks) for the same height and the same round in Tendermint consensus. + +2. double signing: Tendermint consensus forces correct validators to prevote and precommit for at most one value per round. In case a faulty validator sends multiple prevote and/or precommit messages for different values for the same height/round, this is a misbehavior. + +3. lunatic validator: Tendermint consensus forces correct validators to prevote and precommit only for values *v* that satisfy *valid(v)*. If faulty validators prevote and precommit for *v* although *valid(v)=false* this is misbehavior. + +*Remark.* In isolation, Point 3 is an attack on validity (rather than agreement). However, the prevotes and precommits can then also be used to forge blocks. + +1. amnesia: Tendermint consensus has a locking mechanism. If a validator has some value v locked, then it can only prevote/precommit for v or nil. Sending prevote/precomit message for a different value v' (that is not nil) while holding lock on value v is misbehavior. + +2. spurious messages: In Tendermint consensus most of the message send instructions are guarded by threshold guards, e.g., one needs to receive *2f + 1* prevote messages to send precommit. Faulty validators may send precommit without having received the prevote messages. + +Independently of a fork happening, punishing this behavior might be important to prevent forks altogether. This should keep attackers from misbehaving: if less than 1/3 of the voting power is faulty, this misbehavior is detectable but will not lead to a safety violation. Thus, unless they have 1/3 or more (or in some cases more than 2/3) of the voting power attackers have the incentive to not misbehave. If attackers control too much voting power, we have to deal with forks, as discussed in this document. + +## Two types of forks + +* Fork-Full. Two correct validators decide on different blocks for the same height. Since also the next validator sets are decided upon, the correct validators may be partitioned to participate in two distinct branches of the forked chain. + +As in this case we have two different blocks (both having the same right/no right to exist), a central system invariant (one block per height decided by correct validators) is violated. As full nodes are contaminated in this case, the contamination can spread also to light clients. However, even without breaking this system invariant, light clients can be subject to a fork: + +* Fork-Light. All correct validators decide on the same block for height *h*, but faulty processes (validators or not), forge a different block for that height, in order to fool users (who use the light client). + +# Attack scenarios + +## On-chain attacks + +### Equivocation (one round) + +There are several scenarios in which forks might happen. The first is double signing within a round. + +* F1. Equivocation: faulty validators sign multiple vote messages (prevote and/or precommit) for different values *during the same round r* at a given height h. + +### Flip-flopping + +Tendermint consensus implements a locking mechanism: If a correct validator *p* receives proposal for value v and *2f + 1* prevotes for a value *id(v)* in round *r*, it locks *v* and remembers *r*. In this case, *p* also sends a precommit message for *id(v)*, which later may serve as proof that *p* locked *v*. +In subsequent rounds, *p* only sends prevote messages for a value it had previously locked. However, it is possible to change the locked value if in a future round *r' > r*, if the process receives proposal and *2f + 1* prevotes for a different value *v'*. In this case, *p* could send a prevote/precommit for *id(v')*. This algorithmic feature can be exploited in two ways: + +* F2. Faulty Flip-flopping (Amnesia): faulty validators precommit some value *id(v)* in round *r* (value *v* is locked in round *r*) and then prevote for different value *id(v')* in higher round *r' > r* without previously correctly unlocking value *v*. In this case faulty processes "forget" that they have locked value *v* and prevote some other value in the following rounds. +Some correct validators might have decided on *v* in *r*, and other correct validators decide on *v'* in *r'*. Here we can have branching on the main chain (Fork-Full). + +* F3. Correct Flip-flopping (Back to the past): There are some precommit messages signed by (correct) validators for value *id(v)* in round *r*. Still, *v* is not decided upon, and all processes move on to the next round. Then correct validators (correctly) lock and decide a different value *v'* in some round *r' > r*. And the correct validators continue; there is no branching on the main chain. +However, faulty validators may use the correct precommit messages from round *r* together with a posteriori generated faulty precommit messages for round *r* to forge a block for a value that was not decided on the main chain (Fork-Light). + +## Off-chain attacks + +F1-F3 may contaminate the state of full nodes (and even validators). Contaminated (but otherwise correct) full nodes may thus communicate faulty blocks to light clients. +Similarly, without actually interfering with the main chain, we can have the following: + +* F4. Phantom validators: faulty validators vote (sign prevote and precommit messages) in heights in which they are not part of the validator sets (at the main chain). + +* F5. Lunatic validator: faulty validator that sign vote messages to support (arbitrary) application state that is different from the application state that resulted from valid state transitions. + +## Types of victims + +We consider three types of potential attack victims: + +* FN: full node +* LCS: light client with sequential header verification +* LCB: light client with bisection based header verification + +F1 and F2 can be used by faulty validators to actually create multiple branches on the blockchain. That means that correctly operating full nodes decide on different blocks for the same height. Until a fork is detected locally by a full node (by receiving evidence from others or by some other local check that fails), the full node can spread corrupted blocks to light clients. + +*Remark.* If full nodes take a branch different from the one taken by the validators, it may be that the liveness of the gossip protocol may be affected. We should eventually look at this more closely. However, as it does not influence safety it is not a primary concern. + +F3 is similar to F1, except that no two correct validators decide on different blocks. It may still be the case that full nodes become affected. + +In addition, without creating a fork on the main chain, light clients can be contaminated by more than a third of validators that are faulty and sign a forged header +F4 cannot fool correct full nodes as they know the current validator set. Similarly, LCS know who the validators are. Hence, F4 is an attack against LCB that do not necessarily know the complete prefix of headers (Fork-Light), as they trust a header that is signed by at least one correct validator (trusting period method). + +The following table gives an overview of how the different attacks may affect different nodes. F1-F3 are *on-chain* attacks so they can corrupt the state of full nodes. Then if a light client (LCS or LCB) contacts a full node to obtain headers (or blocks), the corrupted state may propagate to the light client. + +F4 and F5 are *off-chain*, that is, these attacks cannot be used to corrupt the state of full nodes (which have sufficient knowledge on the state of the chain to not be fooled). + +| Attack | FN | LCS | LCB | +|:------:|:------:|:------:|:------:| +| F1 | direct | FN | FN | +| F2 | direct | FN | FN | +| F3 | direct | FN | FN | +| F4 | | | direct | +| F5 | | | direct | + +**Q:** Light clients are more vulnerable than full nodes, because the former do only verify headers but do not execute transactions. What kind of certainty is gained by a full node that executes a transaction? + +As a full node verifies all transactions, it can only be +contaminated by an attack if the blockchain itself violates its invariant (one block per height), that is, in case of a fork that leads to branching. + +## Detailed Attack Scenarios + +### Equivocation based attacks + +In case of equivocation based attacks, faulty validators sign multiple votes (prevote and/or precommit) in the same +round of some height. This attack can be executed on both full nodes and light clients. It requires 1/3 or more of voting power to be executed. + +#### Scenario 1: Equivocation on the main chain + +Validators: + +* CA - a set of correct validators with less than 1/3 of the voting power +* CB - a set of correct validators with less than 1/3 of the voting power +* CA and CB are disjoint +* F - a set of faulty validators with 1/3 or more voting power + +Observe that this setting violates the Cosmos failure model. + +Execution: + +* A faulty proposer proposes block A to CA +* A faulty proposer proposes block B to CB +* Validators from the set CA and CB prevote for A and B, respectively. +* Faulty validators from the set F prevote both for A and B. +* The faulty prevote messages + * for A arrive at CA long before the B messages + * for B arrive at CB long before the A messages +* Therefore correct validators from set CA and CB will observe +more than 2/3 of prevotes for A and B and precommit for A and B, respectively. +* Faulty validators from the set F precommit both values A and B. +* Thus, we have more than 2/3 commits for both A and B. + +Consequences: + +* Creating evidence of misbehavior is simple in this case as we have multiple messages signed by the same faulty processes for different values in the same round. + +* We have to ensure that these different messages reach a correct process (full node, monitor?), which can submit evidence. + +* This is an attack on the full node level (Fork-Full). +* It extends also to the light clients, +* For both we need a detection and recovery mechanism. + +#### Scenario 2: Equivocation to a light client (LCS) + +Validators: + +* a set F of faulty validators with more than 2/3 of the voting power. + +Execution: + +* for the main chain F behaves nicely +* F coordinates to sign a block B that is different from the one on the main chain. +* the light clients obtains B and trusts at as it is signed by more than 2/3 of the voting power. + +Consequences: + +Once equivocation is used to attack light client it opens space +for different kind of attacks as application state can be diverged in any direction. For example, it can modify validator set such that it contains only validators that do not have any stake bonded. Note that after a light client is fooled by a fork, that means that an attacker can change application state and validator set arbitrarily. + +In order to detect such (equivocation-based attack), the light client would need to cross check its state with some correct validator (or to obtain a hash of the state from the main chain using out of band channels). + +*Remark.* The light client would be able to create evidence of misbehavior, but this would require to pull potentially a lot of data from correct full nodes. Maybe we need to figure out different architecture where a light client that is attacked will push all its data for the current unbonding period to a correct node that will inspect this data and submit corresponding evidence. There are also architectures that assumes a special role (sometimes called fisherman) whose goal is to collect as much as possible useful data from the network, to do analysis and create evidence transactions. That functionality is outside the scope of this document. + +*Remark.* The difference between LCS and LCB might only be in the amount of voting power needed to convince light client about arbitrary state. In case of LCB where security threshold is at minimum, an attacker can arbitrarily modify application state with 1/3 or more of voting power, while in case of LCS it requires more than 2/3 of the voting power. + +### Flip-flopping: Amnesia based attacks + +In case of amnesia, faulty validators lock some value *v* in some round *r*, and then vote for different value *v'* in higher rounds without correctly unlocking value *v*. This attack can be used both on full nodes and light clients. + +#### Scenario 3: At most 2/3 of faults + +Validators: + +* a set F of faulty validators with 1/3 or more but at most 2/3 of the voting power +* a set C of correct validators + +Execution: + +* Faulty validators commit (without exposing it on the main chain) a block A in round *r* by collecting more than 2/3 of the + voting power (containing correct and faulty validators). +* All validators (correct and faulty) reach a round *r' > r*. +* Some correct validators in C do not lock any value before round *r'*. +* The faulty validators in F deviate from Tendermint consensus by ignoring that they locked A in *r*, and propose a different block B in *r'*. +* As the validators in C that have not locked any value find B acceptable, they accept the proposal for B and commit a block B. + +*Remark.* In this case, the more than 1/3 of faulty validators do not need to commit an equivocation (F1) as they only vote once per round in the execution. + +If a light client is attacked using this attack with 1/3 or more of voting power (and less than 2/3), the attacker cannot change the application state arbitrarily. Rather, the attacker is limited to a state a correct validator finds acceptable: In the execution above, correct validators still find the value acceptable, however, the block the light client trusts deviates from the one on the main chain. + +#### Scenario 4: More than 2/3 of faults + +In case there is an attack with more than 2/3 of the voting power, an attacker can arbitrarily change application state. + +Validators: + +* a set F1 of faulty validators with 1/3 or more of the voting power +* a set F2 of faulty validators with less than 1/3 of the voting power + +Execution + +* Similar to Scenario 3 (however, messages by correct validators are not needed) +* The faulty validators in F1 lock value A in round *r* +* They sign a different value in follow-up rounds +* F2 does not lock A in round *r* + +Consequences: + +* The validators in F1 will be detectable by the fork accountability mechanisms. +* The validators in F2 cannot be detected using this mechanism. +Only in case they signed something which conflicts with the application this can be used against them. Otherwise, they do not do anything incorrect. + +**Q:** do we need to define a special kind of attack for the case where a validator sign arbitrarily state? It seems that detecting such attack requires a different mechanism that would require as an evidence a sequence of blocks that led to that state. This might be very tricky to implement. + +### Back to the past + +In this kind of attack, faulty validators take advantage of the fact that they did not sign messages in some of the past rounds. Due to the asynchronous network in which Tendermint operates, we cannot easily differentiate between such an attack and delayed message. This kind of attack can be used at both full nodes and light clients. + +#### Scenario 5 + +Validators: + +* C1 - a set of correct validators with over 1/3 of the voting power +* C2 - a set of correct validators with 1/3 of the voting power +* C1 and C2 are disjoint +* F - a set of faulty validators with less than 1/3 voting power +* one additional faulty process *q* +* F and *q* violate the Cosmos failure model. + +Execution: + +* in a round *r* of height *h* we have C1 precommitting a value A, +* C2 precommits nil, +* F does not send any message +* *q* precommits nil. +* In some round *r' > r*, F and *q* and C2 commit some other value B different from A. +* F and *fp* "go back to the past" and sign precommit message for value A in round *r*. +* Together with precomit messages of C1 this is sufficient for a commit for value A. + +Consequences: + +* Only a single faulty validator that previously precommited nil did equivocation, while the other 1/3 of faulty validators actually executed an attack that has exactly the same sequence of messages as part of amnesia attack. Detecting this kind of attack boil down to mechanisms for equivocation and amnesia. + +**Q:** should we keep this as a separate kind of attack? It seems that equivocation, amnesia and phantom validators are the only kind of attack we need to support and this gives us security also in other cases. This would not be surprising as equivocation and amnesia are attacks that followed from the protocol and phantom attack is not really an attack to Tendermint but more to the Cosmos Proof of Stake module. + +### Phantom validators + +In case of phantom validators, processes that are not part of the current validator set but are still bonded (as attack happen during their unbonding period) can be part of the attack by signing vote messages. This attack can be executed against both full nodes and light clients. + +#### Scenario 6 + +Validators: + +* F -- a set of faulty validators that are not part of the validator set on the main chain at height *h + k* + +Execution: + +* There is a fork, and there exist two different headers for height *h + k*, with different validator sets: + * VS2 on the main chain + * forged header VS2', signed by F (and others) + +* a light client has a trust in a header for height *h* (and the corresponding validator set VS1). +* As part of bisection header verification, it verifies the header at height *h + k* with new validator set VS2'. + +Consequences: + +* To detect this, a node needs to see both, the forged header and the canonical header from the chain. +* If this is the case, detecting these kind of attacks is easy as it just requires verifying if processes are signing messages in heights in which they are not part of the validator set. + +**Remark.** We can have phantom-validator-based attacks as a follow up of equivocation or amnesia based attack where forked state contains validators that are not part of the validator set at the main chain. In this case, they keep signing messages contributed to a forked chain (the wrong branch) although they are not part of the validator set on the main chain. This attack can also be used to attack full node during a period of time it is eclipsed. + +**Remark.** Phantom validator evidence has been removed from implementation as it was deemed, although possibly a plausible form of evidence, not relevant. Any attack on +the light client involving a phantom validator will have needed to be initiated by 1/3+ lunatic +validators that can forge a new validator set that includes the phantom validator. Only in +that case will the light client accept the phantom validators vote. We need only worry about +punishing the 1/3+ lunatic cabal, that is the root cause of the attack. + +### Lunatic validator + +Lunatic validator agrees to sign commit messages for arbitrary application state. It is used to attack light clients. +Note that detecting this behavior require application knowledge. Detecting this behavior can probably be done by +referring to the block before the one in which height happen. + +**Q:** can we say that in this case a validator declines to check if a proposed value is valid before voting for it? diff --git a/cometbft/v0.39/spec/light-client/Fork-Detection.mdx b/cometbft/v0.39/spec/light-client/Fork-Detection.mdx new file mode 100644 index 000000000..88f2a2bc2 --- /dev/null +++ b/cometbft/v0.39/spec/light-client/Fork-Detection.mdx @@ -0,0 +1,77 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/spec/light-client/Fork-Detection' +order: 1 +parent: + title: Fork Detection + order: 2 +--- + +# Cosmos fork detection and IBC fork detection + +## Status + +This is a work in progress. +This directory captures the ongoing work and discussion on fork +detection both in the context of a Cosmos light node and in the +context of IBC. It contains the following files + +### [detection.md](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/detection/detection_003_reviewed.md) + +a draft of the light node fork detection including "proof of fork" + definition, that is, the data structure to submit evidence to full + nodes. + +### [discussions.md](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/detection/discussions.md) + +A collection of ideas and intuitions from recent discussions + +- the outcome of recent discussion +- a sketch of the light client supervisor to provide the context in + which fork detection happens +- a discussion about lightstore semantics + +### [req-ibc-detection.md](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/detection/req-ibc-detection.md) + +- a collection of requirements for fork detection in the IBC + context. In particular it contains a section "Required Changes in + ICS 007" with necessary updates to ICS 007 to support Cosmos + fork detection + +### [draft-functions.md](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/detection/draft-functions.md) + +In order to address the collected requirements, we started to sketch +some functions that we will need in the future when we specify in more +detail the + +- fork detections +- proof of fork generation +- proof of fork verification + +on the following components. + +- IBC on-chain components +- Relayer + +### TODOs + +We decided to merge the files while there are still open points to +address to record the current state an move forward. In particular, +the following points need to be addressed: + +- [https://github.com/informalsystems/tendermint-rs/pull/479#discussion_r466504876](https://github.com/informalsystems/tendermint-rs/pull/479#discussion_r466504876) + +- [https://github.com/informalsystems/tendermint-rs/pull/479#discussion_r466493900](https://github.com/informalsystems/tendermint-rs/pull/479#discussion_r466493900) + +- [https://github.com/informalsystems/tendermint-rs/pull/479#discussion_r466489045](https://github.com/informalsystems/tendermint-rs/pull/479#discussion_r466489045) + +- [https://github.com/informalsystems/tendermint-rs/pull/479#discussion_r466491471](https://github.com/informalsystems/tendermint-rs/pull/479#discussion_r466491471) + +Most likely we will write a specification on the light client +supervisor along the outcomes of + +- [https://github.com/informalsystems/tendermint-rs/pull/509](https://github.com/informalsystems/tendermint-rs/pull/509) + +that also addresses initialization + +- [https://github.com/tendermint/spec/issues/131](https://github.com/tendermint/spec/issues/131) diff --git a/cometbft/v0.39/spec/light-client/Light-Client-Specification.mdx b/cometbft/v0.39/spec/light-client/Light-Client-Specification.mdx new file mode 100644 index 000000000..7ef96ebb3 --- /dev/null +++ b/cometbft/v0.39/spec/light-client/Light-Client-Specification.mdx @@ -0,0 +1,203 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/spec/light-client/Light-Client-Specification' +title: Light Client Specification +order: 1 +--- + +This directory contains work-in-progress English and TLA+ specifications for the Light Client +protocol. Implementations of the light client can be found in +[Rust](https://github.com/informalsystems/tendermint-rs/tree/master/light-client) and +[Go](https://github.com/cometbft/cometbft/tree/v0.38.x/light). + +Light clients are assumed to be initialized once from a trusted source +with a trusted header and validator set. The light client +protocol allows a client to then securely update its trusted state by requesting and +verifying a minimal set of data from a network of full nodes (at least one of which is correct). + +The light client is decomposed into two main components: + +- [Commit Verification](#commit-verification) - verify signed headers and associated validator + set changes from a single full node, called primary +- [Attack Detection](#attack-detection) - verify commits across multiple full nodes (called secondaries) and detect conflicts (ie. the existence of a lightclient attack) + +In case a lightclient attack is detected, the lightclient submits evidence to a full node which is responsible for "accountability", that is, punishing attackers: + +- [Accountability](#accountability) - given evidence for an attack, compute a set of validators that are responsible for it. + +## Commit Verification + +The [English specification](https://github.com/cometbft/cometbft/blob/main/spec/light-client/verification/verification_001_published.md) describes the light client +commit verification problem in terms of the temporal properties +[LCV-DIST-SAFE.1](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/verification/verification_001_published.md#lcv-dist-safe1) and +[LCV-DIST-LIVE.1](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/verification/verification_001_published.md#lcv-dist-live1). +Commit verification is assumed to operate within the Cosmos Failure Model, where +2/3 of validators are correct for some time period and +validator sets can change arbitrarily at each height. + +A light client protocol is also provided, including all checks that +need to be performed on headers, commits, and validator sets +to satisfy the temporal properties - so a light client can continuously +synchronize with a blockchain. Clients can skip possibly +many intermediate headers by exploiting overlap in trusted and untrusted validator sets. +When there is not enough overlap, a bisection routine can be used to find a +minimal set of headers that do provide the required overlap. + +The [TLA+ specification ver. 001](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/verification/Lightclient_A_1.tla) +is a formal description of the +commit verification protocol executed by a client, including the safety and +termination, which can be model checked with Apalache. + +A more detailed TLA+ specification of +[Light client verification ver. 003](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/verification/Lightclient_003_draft.tla) +is currently under peer review. + +The `MC*.tla` files contain concrete parameters for the +[TLA+ specification](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/verification/Lightclient_A_1.tla), in order to do model checking. +For instance, [MC4_3_faulty.tla](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/verification/MC4_3_faulty.tla) contains the following parameters +for the nodes, heights, the trusting period, the clock drifts, +correctness of the primary node, and the ratio of the faulty processes: + +```tla +AllNodes == {"n1", "n2", "n3", "n4"} +TRUSTED_HEIGHT == 1 +TARGET_HEIGHT == 3 +TRUSTING_PERIOD == 1400 \* the trusting period in some time units +CLOCK_DRIFT = 10 \* how much we assume the local clock is drifting +REAL_CLOCK_DRIFT = 3 \* how much the local clock is actually drifting +IS_PRIMARY_CORRECT == FALSE +FAULTY_RATIO == <<1, 3>> \* < 1 / 3 faulty validators +``` + +To run a complete set of experiments, clone [apalache](https://github.com/informalsystems/apalache) and [apalache-tests](https://github.com/informalsystems/apalache-tests) into a directory `$DIR` and run the following commands: + +```sh +$DIR/apalache-tests/scripts/mk-run.py --memlimit 28 002bmc-apalache-ok.csv $DIR/apalache . out +./out/run-all.sh +``` + +After the experiments have finished, you can collect the logs by executing the following command: + +```sh +cd ./out +$DIR/apalache-tests/scripts/parse-logs.py --human . +``` + +All lines in `results.csv` should report `Deadlock`, which means that the algorithm +has terminated and no invariant violation was found. + +Similar to [002bmc-apalache-ok.csv](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/verification/002bmc-apalache-ok.csv), +file [003bmc-apalache-error.csv](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/verification/003bmc-apalache-error.csv) specifies +the set of experiments that should result in counterexamples: + +```sh +$DIR/apalache-tests/scripts/mk-run.py --memlimit 28 003bmc-apalache-error.csv $DIR/apalache . out +./out/run-all.sh +``` + +All lines in `results.csv` should report `Error`. + +The following table summarizes the experimental results for Light client verification +version 001. The TLA+ properties can be found in the +[TLA+ specification](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/verification/Lightclient_A_1.tla). + The experiments were run in an AWS instance equipped with 32GB +RAM and a 4-core Intel® Xeon® CPU E5-2686 v4 @ 2.30GHz CPU. +We write “`✗=k`” when a bug is reported at depth k, and “`✓<=k`” when +no bug is reported up to depth k. + +![Experimental results](/cometbft/v0.39/spec/light-client/experiments.png) + +The experimental results for version 003 are to be added. + +## Attack Detection + +The [English specification](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/detection/detection_003_reviewed.md) +defines light client attacks (and how they differ from blockchain +forks), and describes the problem of a light client detecting +these attacks by communicating with a network of full nodes, +where at least one is correct. + +The specification also contains a detection protocol that checks +whether the header obtained from the primary via the verification +protocol matches corresponding headers provided by the secondaries. +If this is not the case, the protocol analyses the verification traces +of the involved full nodes +and generates +[evidence](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/detection/detection_003_reviewed.md#cmbc-lc-evidence-data1) +of misbehavior that can be submitted to a full node so that +the faulty validators can be punished. + +The [TLA+ specification](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/detection/LCDetector_003_draft.tla) +is a formal description of the +detection protocol for two peers, including the safety and +termination, which can be model checked with Apalache. + +The `LCD_MC*.tla` files contain concrete parameters for the +[TLA+ specification](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/detection/LCDetector_003_draft.tla), +in order to run the model checker. +For instance, [LCD_MC4_4_faulty.tla](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/verification/MC4_4_faulty.tla) +contains the following parameters +for the nodes, heights, the trusting period, the clock drifts, +correctness of the nodes, and the ratio of the faulty processes: + +```tla +AllNodes == {"n1", "n2", "n3", "n4"} +TRUSTED_HEIGHT == 1 +TARGET_HEIGHT == 3 +TRUSTING_PERIOD == 1400 \* the trusting period in some time units +CLOCK_DRIFT = 10 \* how much we assume the local clock is drifting +REAL_CLOCK_DRIFT = 3 \* how much the local clock is actually drifting +IS_PRIMARY_CORRECT == FALSE +IS_SECONDARY_CORRECT == FALSE +FAULTY_RATIO == <<1, 3>> \* < 1 / 3 faulty validators +``` + +To run a complete set of experiments, clone [apalache](https://github.com/informalsystems/apalache) and [apalache-tests](https://github.com/informalsystems/apalache-tests) into a directory `$DIR` and run the following commands: + +```sh +$DIR/apalache-tests/scripts/mk-run.py --memlimit 28 004bmc-apalache-ok.csv $DIR/apalache . out +./out/run-all.sh +``` + +After the experiments have finished, you can collect the logs by executing the following command: + +```sh +cd ./out +$DIR/apalache-tests/scripts/parse-logs.py --human . +``` + +All lines in `results.csv` should report `Deadlock`, which means that the algorithm +has terminated and no invariant violation was found. + +Similar to [004bmc-apalache-ok.csv](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/verification/004bmc-apalache-ok.csv), +file [005bmc-apalache-error.csv](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/verification/005bmc-apalache-error.csv) specifies +the set of experiments that should result in counterexamples: + +```sh +$DIR/apalache-tests/scripts/mk-run.py --memlimit 28 005bmc-apalache-error.csv $DIR/apalache . out +./out/run-all.sh +``` + +All lines in `results.csv` should report `Error`. + +The detailed experimental results are to be added soon. + +## Accountability + +The [English specification](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/attacks/isolate-attackers_002_reviewed.md) +defines the protocol that is executed on a full node upon receiving attack [evidence](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/attacks/isolate-attackers_002_reviewed.md#cmbc-lc-evidence-data1) from a lightclient. In particular, the protocol handles three types of attacks + +- lunatic +- equivocation +- amnesia + +We discussed in the [last part](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/attacks/isolate-attackers_002_reviewed.md#Part-III---Completeness) of the English specification +that the non-lunatic cases are defined by having the same validator set in the conflicting blocks. For these cases, +computer-aided analysis of [Tendermint Consensus in TLA+](/cometbft/v0.39/spec/light-client/Accountability) shows that equivocation and amnesia capture all non-lunatic attacks. + +The [TLA+ specification](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/attacks/Isolation_001_draft.tla) +is a formal description of the +protocol, including the safety property, which can be model checked with Apalache. + +Similar to the other specifications, [MC_5_3.tla](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/light-client/attacks/MC_5_3.tla) contains concrete parameters to run the model checker. The specification can be checked within seconds. + +[tendermint-accountability](/cometbft/v0.39/spec/light-client/Accountability) diff --git a/cometbft/v0.39/spec/light-client/assets/light-node-image.png b/cometbft/v0.39/spec/light-client/assets/light-node-image.png new file mode 100644 index 000000000..f0b93c6e4 Binary files /dev/null and b/cometbft/v0.39/spec/light-client/assets/light-node-image.png differ diff --git a/cometbft/v0.39/spec/light-client/experiments.png b/cometbft/v0.39/spec/light-client/experiments.png new file mode 100644 index 000000000..94166ffa3 Binary files /dev/null and b/cometbft/v0.39/spec/light-client/experiments.png differ diff --git a/cometbft/v0.39/spec/light-client/verification.mdx b/cometbft/v0.39/spec/light-client/verification.mdx new file mode 100644 index 000000000..8d92e3597 --- /dev/null +++ b/cometbft/v0.39/spec/light-client/verification.mdx @@ -0,0 +1,586 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/spec/light-client/verification' +order: 1 +parent: + title: Verification + order: 2 +--- +# Core Verification + +## Problem statement + +We assume that the light client knows a (base) header `inithead` it trusts (by social consensus or because +the light client has decided to trust the header before). The goal is to check whether another header +`newhead` can be trusted based on the data in `inithead`. + +The correctness of the protocol is based on the assumption that `inithead` was generated by an instance of +Tendermint consensus. + +### Failure Model + +For the purpose of the following definitions we assume that there exists a function +`validators` that returns the corresponding validator set for the given hash. + +The light client protocol is defined with respect to the following failure model: + +Given a known bound `TRUSTED_PERIOD`, and a block `b` with header `h` generated at time `Time` +(i.e. `h.Time = Time`), a set of validators that hold more than 2/3 of the voting power +in `validators(b.Header.NextValidatorsHash)` is correct until time `b.Header.Time + TRUSTED_PERIOD`. + +*Assumption*: "correct" is defined w.r.t. realtime (some Newtonian global notion of time, i.e., wall time), +while `Header.Time` corresponds to the [BFT time](/cometbft/v0.39/spec/consensus/BFT-Time). In this note, we assume that clocks of correct processes +are synchronized (for example using NTP), and therefore there is bounded clock drift (`CLOCK_DRIFT`) between local clocks and +BFT time. More precisely, for every correct light client process and every `header.Time` (i.e. BFT Time, for a header correctly +generated by the Tendermint consensus), the following inequality holds: `Header.Time < now + CLOCK_DRIFT`, +where `now` corresponds to the system clock at the light client process. + +Furthermore, we assume that `TRUSTED_PERIOD` is (several) order of magnitude bigger than `CLOCK_DRIFT` (`TRUSTED_PERIOD >> CLOCK_DRIFT`), +as `CLOCK_DRIFT` (using NTP) is in the order of milliseconds and `TRUSTED_PERIOD` is in the order of weeks. + +We expect a light client process defined in this document to be used in the context in which there is some +larger period during which misbehaving validators can be detected and punished (we normally refer to it as `UNBONDING_PERIOD` +due to the "bonding" mechanism in modern proof of stake systems). Furthermore, we assume that +`TRUSTED_PERIOD < UNBONDING_PERIOD` and that they are normally of the same order of magnitude, for example +`TRUSTED_PERIOD = UNBONDING_PERIOD / 2`. + +The specification in this document considers an implementation of the light client under the Failure Model defined above. +Mechanisms like `fork accountability` and `evidence submission` are defined in the context of `UNBONDING_PERIOD` and +they incentivize validators to follow the protocol specification defined in this document. If they don't, +and we have 1/3 (or more) faulty validators, safety may be violated. Our approach then is +to *detect* these cases (after the fact), and take suitable repair actions (automatic and social). +This is discussed in document on [Fork accountability](/cometbft/v0.39/spec/light-client/Accountability). + +The term "trusted" above indicates that the correctness of the protocol depends on +this assumption. It is in the responsibility of the user that runs the light client to make sure that the risk +of trusting a corrupted/forged `inithead` is negligible. + +*Remark*: This failure model might change to a hybrid version that takes heights into account in the future. + +### High Level Solution + +Upon initialization, the light client is given a header `inithead` it trusts (by +social consensus). When a light clients sees a new signed header `snh`, it has to decide whether to trust the new +header. Trust can be obtained by (possibly) the combination of three methods. + +1. **Uninterrupted sequence of headers.** Given a trusted header `h` and an untrusted header `h1`, +the light client trusts a header `h1` if it trusts all headers in between `h` and `h1`. + +2. **Trusted period.** Given a trusted header `h`, an untrusted header `h1 > h` and `TRUSTED_PERIOD` during which +the failure model holds, we can check whether at least one validator, that has been continuously correct +from `h.Time` until now, has signed `h1`. If this is the case, we can trust `h1`. + +3. **Bisection.** If a check according to 2. (trusted period) fails, the light client can try to +obtain a header `hp` whose height lies between `h` and `h1` in order to check whether `h` can be used to +get trust for `hp`, and `hp` can be used to get trust for `snh`. If this is the case we can trust `h1`; +if not, we continue recursively until either we found set of headers that can build (transitively) trust relation +between `h` and `h1`, or we failed as two consecutive headers don't verify against each other. + +## Definitions + +### Data structures + +In the following, only the details of the data structures needed for this specification are given. + + ```go + type Header struct { + Height int64 + Time Time // the chain time when the header (block) was generated + + LastBlockID BlockID // prev block info + ValidatorsHash []byte // hash of the validators for the current block + NextValidatorsHash []byte // hash of the validators for the next block + } + + type SignedHeader struct { + Header Header + Commit Commit // commit for the given header + } + + type ValidatorSet struct { + Validators []Validator + TotalVotingPower int64 + } + + type Validator struct { + Address Address // validator address (we assume validator's addresses are unique) + VotingPower int64 // validator's voting power + } + + type TrustedState { + SignedHeader SignedHeader + ValidatorSet ValidatorSet + } + ``` + +### Functions + +For the purpose of this light client specification, we assume that the Cosmos Full Node +exposes the following functions over RPC: + +```go + // returns signed header: Header with Commit, for the given height + func Commit(height int64) (SignedHeader, error) + + // returns validator set for the given height + func Validators(height int64) (ValidatorSet, error) +``` + +Furthermore, we assume the following auxiliary functions: + +```go + // returns true if the commit is for the header, ie. if it contains + // the correct hash of the header; otherwise false + func matchingCommit(header Header, commit Commit) bool + + // returns the set of validators from the given validator set that + // committed the block (that correctly signed the block) + // it assumes signature verification so it can be computationally expensive + func signers(commit Commit, validatorSet ValidatorSet) []Validator + + // returns the voting power the validators in v1 have according to their voting power in set v2 + // it does not assume signature verification + func votingPowerIn(v1 []Validator, v2 ValidatorSet) int64 + + // returns hash of the given validator set + func hash(v2 ValidatorSet) []byte +``` + +In the functions below we will be using `trustThreshold` as a parameter. For simplicity +we assume that `trustThreshold` is a float between `1/3` and `2/3` and we will not be checking it +in the pseudo-code. + +**VerifySingle.** The function `VerifySingle` attempts to validate given untrusted header and the corresponding validator sets +based on a given trusted state. It ensures that the trusted state is still within its trusted period, +and that the untrusted header is within assumed `clockDrift` bound of the passed time `now`. +Note that this function is not making external (RPC) calls to the full node; the whole logic is +based on the local (given) state. This function is supposed to be used by the IBC handlers. + +```go +func VerifySingle(untrustedSh SignedHeader, + untrustedVs ValidatorSet, + untrustedNextVs ValidatorSet, + trustedState TrustedState, + trustThreshold float, + trustingPeriod Duration, + clockDrift Duration, + now Time) (TrustedState, error) { + + if untrustedSh.Header.Time > now + clockDrift { + return (trustedState, ErrInvalidHeaderTime) + } + + trustedHeader = trustedState.SignedHeader.Header + if !isWithinTrustedPeriod(trustedHeader, trustingPeriod, now) { + return (state, ErrHeaderNotWithinTrustedPeriod) + } + + // we assume that time it takes to execute verifySingle function + // is several order of magnitudes smaller than trustingPeriod + error = verifySingle( + trustedState, + untrustedSh, + untrustedVs, + untrustedNextVs, + trustThreshold) + + if error != nil return (state, error) + + // the untrusted header is now trusted + newTrustedState = TrustedState(untrustedSh, untrustedNextVs) + return (newTrustedState, nil) +} + +// return true if header is within its light client trusted period; otherwise returns false +func isWithinTrustedPeriod(header Header, + trustingPeriod Duration, + now Time) bool { + + return header.Time + trustedPeriod > now +} +``` + +Note that in case `VerifySingle` returns without an error (untrusted header +is successfully verified) then we have a guarantee that the transition of the trust +from `trustedState` to `newTrustedState` happened during the trusted period of +`trustedState.SignedHeader.Header`. + +TODO: Explain what happens in case `VerifySingle` returns with an error. + +**verifySingle.** The function `verifySingle` verifies a single untrusted header +against a given trusted state. It includes all validations and signature verification. +It is not publicly exposed since it does not check for header expiry (time constraints) +and hence it's possible to use it incorrectly. + +```go +func verifySingle(trustedState TrustedState, + untrustedSh SignedHeader, + untrustedVs ValidatorSet, + untrustedNextVs ValidatorSet, + trustThreshold float) error { + + untrustedHeader = untrustedSh.Header + untrustedCommit = untrustedSh.Commit + + trustedHeader = trustedState.SignedHeader.Header + trustedVs = trustedState.ValidatorSet + + if trustedHeader.Height >= untrustedHeader.Height return ErrNonIncreasingHeight + if trustedHeader.Time >= untrustedHeader.Time return ErrNonIncreasingTime + + // validate the untrusted header against its commit, vals, and next_vals + error = validateSignedHeaderAndVals(untrustedSh, untrustedVs, untrustedNextVs) + if error != nil return error + + // check for adjacent headers + if untrustedHeader.Height == trustedHeader.Height + 1 { + if trustedHeader.NextValidatorsHash != untrustedHeader.ValidatorsHash { + return ErrInvalidAdjacentHeaders + } + } else { + error = verifyCommitTrusting(trustedVs, untrustedCommit, untrustedVs, trustThreshold) + if error != nil return error + } + + // verify the untrusted commit + return verifyCommitFull(untrustedVs, untrustedCommit) +} + +// returns nil if header and validator sets are consistent; otherwise returns error +func validateSignedHeaderAndVals(signedHeader SignedHeader, vs ValidatorSet, nextVs ValidatorSet) error { + header = signedHeader.Header + if hash(vs) != header.ValidatorsHash return ErrInvalidValidatorSet + if hash(nextVs) != header.NextValidatorsHash return ErrInvalidNextValidatorSet + if !matchingCommit(header, signedHeader.Commit) return ErrInvalidCommitValue + return nil +} + +// returns nil if at least single correst signer signed the commit; otherwise returns error +func verifyCommitTrusting(trustedVs ValidatorSet, + commit Commit, + untrustedVs ValidatorSet, + trustLevel float) error { + + totalPower := trustedVs.TotalVotingPower + signedPower := votingPowerIn(signers(commit, untrustedVs), trustedVs) + + // check that the signers account for more than max(1/3, trustLevel) of the voting power + // this ensures that there is at least single correct validator in the set of signers + if signedPower < max(1/3, trustLevel) * totalPower return ErrInsufficientVotingPower + return nil +} + +// returns nil if commit is signed by more than 2/3 of voting power of the given validator set +// return error otherwise +func verifyCommitFull(vs ValidatorSet, commit Commit) error { + totalPower := vs.TotalVotingPower; + signedPower := votingPowerIn(signers(commit, vs), vs) + + // check the signers account for +2/3 of the voting power + if signedPower * 3 <= totalPower * 2 return ErrInvalidCommit + return nil +} +``` + +**VerifyHeaderAtHeight.** The function `VerifyHeaderAtHeight` captures high level +logic, i.e., application call to the light client module to download and verify header +for some height. + +```go +func VerifyHeaderAtHeight(untrustedHeight int64, + trustedState TrustedState, + trustThreshold float, + trustingPeriod Duration, + clockDrift Duration) (TrustedState, error)) { + + trustedHeader := trustedState.SignedHeader.Header + + now := System.Time() + if !isWithinTrustedPeriod(trustedHeader, trustingPeriod, now) { + return (trustedState, ErrHeaderNotWithinTrustedPeriod) + } + + newTrustedState, err := VerifyBisection(untrustedHeight, + trustedState, + trustThreshold, + trustingPeriod, + clockDrift, + now) + + if err != nil return (trustedState, err) + + now = System.Time() + if !isWithinTrustedPeriod(trustedHeader, trustingPeriod, now) { + return (trustedState, ErrHeaderNotWithinTrustedPeriod) + } + + return (newTrustedState, err) +} +``` + +Note that in case `VerifyHeaderAtHeight` returns without an error (untrusted header +is successfully verified) then we have a guarantee that the transition of the trust +from `trustedState` to `newTrustedState` happened during the trusted period of +`trustedState.SignedHeader.Header`. + +In case `VerifyHeaderAtHeight` returns with an error, then either (i) the full node we are talking to is faulty +or (ii) the trusted header has expired (it is outside its trusted period). In case (i) the full node is faulty so +light client should disconnect and reinitialize with new peer. In the case (ii) as the trusted header has expired, +we need to reinitialize light client with a new trusted header (that is within its trusted period), +but we don't necessarily need to disconnect from the full node we are talking to (as we haven't observed full node misbehavior in this case). + +**VerifyBisection.** The function `VerifyBisection` implements +recursive logic for checking if it is possible building trust +relationship between `trustedState` and untrusted header at the given height over +finite set of (downloaded and verified) headers. + +```go +func VerifyBisection(untrustedHeight int64, + trustedState TrustedState, + trustThreshold float, + trustingPeriod Duration, + clockDrift Duration, + now Time) (TrustedState, error) { + + untrustedSh, error := Commit(untrustedHeight) + if error != nil return (trustedState, ErrRequestFailed) + + untrustedHeader = untrustedSh.Header + + // note that we pass now during the recursive calls. This is fine as + // all other untrusted headers we download during recursion will be + // for a smaller heights, and therefore should happen before. + if untrustedHeader.Time > now + clockDrift { + return (trustedState, ErrInvalidHeaderTime) + } + + untrustedVs, error := Validators(untrustedHeight) + if error != nil return (trustedState, ErrRequestFailed) + + untrustedNextVs, error := Validators(untrustedHeight + 1) + if error != nil return (trustedState, ErrRequestFailed) + + error = verifySingle( + trustedState, + untrustedSh, + untrustedVs, + untrustedNextVs, + trustThreshold) + + if fatalError(error) return (trustedState, error) + + if error == nil { + // the untrusted header is now trusted. + newTrustedState = TrustedState(untrustedSh, untrustedNextVs) + return (newTrustedState, nil) + } + + // at this point in time we need to do bisection + pivotHeight := ceil((trustedHeader.Height + untrustedHeight) / 2) + + error, newTrustedState = VerifyBisection(pivotHeight, + trustedState, + trustThreshold, + trustingPeriod, + clockDrift, + now) + if error != nil return (newTrustedState, error) + + return VerifyBisection(untrustedHeight, + newTrustedState, + trustThreshold, + trustingPeriod, + clockDrift, + now) +} + +func fatalError(err) bool { + return err == ErrHeaderNotWithinTrustedPeriod OR + err == ErrInvalidAdjacentHeaders OR + err == ErrNonIncreasingHeight OR + err == ErrNonIncreasingTime OR + err == ErrInvalidValidatorSet OR + err == ErrInvalidNextValidatorSet OR + err == ErrInvalidCommitValue OR + err == ErrInvalidCommit +} +``` + +### The case `untrustedHeader.Height < trustedHeader.Height` + +In the use case where someone tells the light client that application data that is relevant for it +can be read in the block of height `k` and the light client trusts a more recent header, we can use the +hashes to verify headers "down the chain." That is, we iterate down the heights and check the hashes in each step. + +*Remark.* For the case were the light client trusts two headers `i` and `j` with `i < k < j`, we should +discuss/experiment whether the forward or the backward method is more effective. + +```go +func VerifyHeaderBackwards(trustedHeader Header, + untrustedHeader Header, + trustingPeriod Duration, + clockDrift Duration) error { + + if untrustedHeader.Height >= trustedHeader.Height return ErrErrNonDecreasingHeight + if untrustedHeader.Time >= trustedHeader.Time return ErrNonDecreasingTime + + now := System.Time() + if !isWithinTrustedPeriod(trustedHeader, trustingPeriod, now) { + return ErrHeaderNotWithinTrustedPeriod + } + + old := trustedHeader + for i := trustedHeader.Height - 1; i > untrustedHeader.Height; i-- { + untrustedSh, error := Commit(i) + if error != nil return ErrRequestFailed + + if (hash(untrustedSh.Header) != old.LastBlockID.Hash) { + return ErrInvalidAdjacentHeaders + } + + old := untrustedSh.Header + } + + if hash(untrustedHeader) != old.LastBlockID.Hash { + return ErrInvalidAdjacentHeaders + } + + now := System.Time() + if !isWithinTrustedPeriod(trustedHeader, trustingPeriod, now) { + return ErrHeaderNotWithinTrustedPeriod + } + + return nil + } +``` + +*Assumption*: In the following, we assume that *untrusted_h.Header.height > trusted_h.Header.height*. We will quickly discuss the other case in the next section. + +We consider the following set-up: + +- the light client communicates with one full node +- the light client locally stores all the headers that has passed basic verification and that are within light client trust period. In the pseudo code below we +write *Store.Add(header)* for this. If a header failed to verify, then +the full node we are talking to is faulty and we should disconnect from it and reinitialize with new peer. +- If `CanTrust` returns *error*, then the light client has seen a forged header or the trusted header has expired (it is outside its trusted period). + - In case of forged header, the full node is faulty so light client should disconnect and reinitialize with new peer. If the trusted header has expired, + we need to reinitialize light client with new trusted header (that is within its trusted period), but we don't necessarily need to disconnect from the full node + we are talking to (as we haven't observed full node misbehavior in this case). + +## Correctness of the Light Client Protocols + +### Definitions + +- `TRUSTED_PERIOD`: trusted period +- for realtime `t`, the predicate `correct(v,t)` is true if the validator `v` + follows the protocol until time `t` (we will see about recovery later). +- Validator fields. We will write a validator as a tuple `(v,p)` such that + - `v` is the identifier (i.e., validator address; we assume identifiers are unique in each validator set) + - `p` is its voting power +- For each header `h`, we write `trust(h) = true` if the light client trusts `h`. + +### Failure Model + +If a block `b` with a header `h` is generated at time `Time` (i.e. `h.Time = Time`), then a set of validators that +hold more than `2/3` of the voting power in `validators(h.NextValidatorsHash)` is correct until time +`h.Time + TRUSTED_PERIOD`. + +Formally, + +```latex +[ +\sum_{(v,p) \in validators(h.NextValidatorsHash) \wedge correct(v,h.Time + TRUSTED_PERIOD)} p > +\frac{2}{3} \sum_{(v,p) \in validators(h.NextValidatorsHash)} p +] +``` + + +The light client communicates with a full node and learns new headers. The goal is to locally decide whether to trust a header. Our implementation needs to ensure the following two properties: + +- *Light Client Completeness*: If a header `h` was correctly generated by an instance of Tendermint consensus (and its age is less than the trusted period), +then the light client should eventually set `trust(h)` to `true`. + +- *Light Client Accuracy*: If a header `h` was *not generated* by an instance of Tendermint consensus, then the light client should never set `trust(h)` to true. + +*Remark*: If in the course of the computation, the light client obtains certainty that some headers were forged by adversaries +(that is were not generated by an instance of Tendermint consensus), it may submit (a subset of) the headers it has seen as evidence of misbehavior. + +*Remark*: In Completeness we use "eventually", while in practice `trust(h)` should be set to true before `h.Time + TRUSTED_PERIOD`. If not, the header +cannot be trusted because it is too old. + +*Remark*: If a header `h` is marked with `trust(h)`, but it is too old at some point in time we denote with `now` (`h.Time + TRUSTED_PERIOD < now`), +then the light client should set `trust(h)` to `false` again at time `now`. + +*Assumption*: Initially, the light client has a header `inithead` that it trusts, that is, `inithead` was correctly generated by the Tendermint consensus. + +To reason about the correctness, we may prove the following invariant. + +*Verification Condition: light Client Invariant.* + For each light client `l` and each header `h`: +if `l` has set `trust(h) = true`, + then validators that are correct until time `h.Time + TRUSTED_PERIOD` have more than two thirds of the voting power in `validators(h.NextValidatorsHash)`. + +Formally, + +```latex +[ +\sum_{(v,p) \in validators(h.NextValidatorsHash) \wedge correct(v,h.Time + TRUSTED_PERIOD)} p > +\frac{2}{3} \sum_{(v,p) \in validators(h.NextValidatorsHash)} p +] +``` + +*Remark.* To prove the invariant, we will have to prove that the light client only trusts headers that were correctly generated by Tendermint consensus. +Then the formula above follows from the failure model. + +## Details + +**Observation 1.** If `h.Time + TRUSTED_PERIOD > now`, we trust the validator set `validators(h.NextValidatorsHash)`. + +When we say we trust `validators(h.NextValidatorsHash)` we do `not` trust that each individual validator in `validators(h.NextValidatorsHash)` +is correct, but we only trust the fact that less than `1/3` of them are faulty (more precisely, the faulty ones have less than `1/3` of the total voting power). + +*`VerifySingle` correctness arguments* + +Light Client Accuracy: + +- Assume by contradiction that `untrustedHeader` was not generated correctly and the light client sets trust to true because `verifySingle` returns without error. +- `trustedState` is trusted and sufficiently new +- by the Failure Model, less than `1/3` of the voting power held by faulty validators => at least one correct validator `v` has signed `untrustedHeader`. +- as `v` is correct up to now, it followed the Tendermint consensus protocol at least up to signing `untrustedHeader` => `untrustedHeader` was correctly generated. +We arrive at the required contradiction. + +Light Client Completeness: + +- The check is successful if sufficiently many validators of `trustedState` are still validators in the height `untrustedHeader.Height` and signed `untrustedHeader`. +- If `untrustedHeader.Height = trustedHeader.Height + 1`, and both headers were generated correctly, the test passes. + +*Verification Condition:* We may need an invariant stating that if `untrustedSignedHeader.Header.Height = trustedHeader.Height + 1` then +`signers(untrustedSignedHeader.Commit) \subseteq validators(trustedHeader.NextValidatorsHash)`. + +*Remark*: The variable `trustThreshold` can be used if the user believes that relying on one correct validator is not sufficient. +However, in case of (frequent) changes in the validator set, the higher the `trustThreshold` is chosen, the more unlikely it becomes that +`verifySingle` returns with an error for non-adjacent headers. + +- `VerifyBisection` correctness arguments (sketch)* + +Light Client Accuracy: + +- Assume by contradiction that the header at `untrustedHeight` obtained from the full node was not generated correctly and +the light client sets trust to true because `VerifyBisection` returns without an error. +- `VerifyBisection` returns without error only if all calls to `verifySingle` in the recursion return without error (return `nil`). +- Thus we have a sequence of headers that all satisfied the `verifySingle` +- again a contradiction + +light Client Completeness: + +This is only ensured if upon `Commit(pivot)` the light client is always provided with a correctly generated header. + +*Stalling* + +With `VerifyBisection`, a faulty full node could stall a light client by creating a long sequence of headers that are queried one-by-one by the light client and look OK, +before the light client eventually detects a problem. There are several ways to address this: + +- Each call to `Commit` could be issued to a different full node +- Instead of querying header by header, the light client tells a full node which header it trusts, and the height of the header it needs. The full node responds with +the header along with a proof consisting of intermediate headers that the light client can use to verify. Roughly, `VerifyBisection` would then be executed at the full node. +- We may set a timeout how long `VerifyBisection` may take. diff --git a/cometbft/v0.39/spec/p2p/Implementation-of-the-p2p-layer.mdx b/cometbft/v0.39/spec/p2p/Implementation-of-the-p2p-layer.mdx new file mode 100644 index 000000000..631e257cc --- /dev/null +++ b/cometbft/v0.39/spec/p2p/Implementation-of-the-p2p-layer.mdx @@ -0,0 +1,45 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/spec/p2p/Implementation-of-the-p2p-layer' +order: 1 +title: Implementation +--- + +# Implementation of the p2p layer + +This section documents the implementation of the peer-to-peer (p2p) +communication layer in CometBFT. + +The documentation was [produced](https://github.com/tendermint/tendermint/pull/9348) +using the `v0.34.*` releases +and the branch [`v0.34.x`](https://github.com/cometbft/cometbft/tree/v0.34.x) +of this repository as reference. +As there were no substancial changes in the p2p implementation, the +documentation also applies to the releases `v0.37.*` and `v0.38.*` [^v35]. + +[^v35]: The releases `v0.35.*` and `v0.36.*`, which included a major + refactoring of the p2p layer implementation, were [discontinued][v35postmorten]. + +[v35postmorten]: https://interchain-io.medium.com/discontinuing-tendermint-v0-35-a-postmortem-on-the-new-networking-layer-3696c811dabc + +## Contents + +The documentation follows the organization of the +[`p2p` package](https://github.com/cometbft/cometbft/tree/v0.34.x/p2p), +which implements the following abstractions: + +- [Transport](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/p2p/implementation/transport.md): establishes secure and authenticated + connections with peers; +- [Switch](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/p2p/implementation/switch.md): responsible for dialing peers and accepting + connections from peers, for managing established connections, and for + routing messages between the reactors and peers, + that is, between local and remote instances of the CometBFT protocols; +- [PEX Reactor](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/p2p/implementation/pex.md): due to the several roles of this component, the + documentation is split in several parts: + - [Peer Exchange protocol](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/p2p/implementation/pex-protocol.md): enables nodes to exchange peer addresses, thus implementing a peer discovery service; + - [Address Book](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/p2p/implementation/addressbook.md): stores discovered peer addresses and + quality metrics associated to peers with which the node has interacted; + - [Peer Manager](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/p2p/implementation/peer_manager.md): defines when and to which peers a node + should dial, in order to establish outbound connections; +- [Types](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/p2p/implementation/types.md) and [Configuration](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/p2p/implementation/configuration.md) provide a list of + existing types and configuration parameters used by the p2p package. diff --git a/cometbft/v0.39/spec/p2p/Peer-to-Peer.mdx b/cometbft/v0.39/spec/p2p/Peer-to-Peer.mdx new file mode 100644 index 000000000..56974e58d --- /dev/null +++ b/cometbft/v0.39/spec/p2p/Peer-to-Peer.mdx @@ -0,0 +1,48 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/spec/p2p/Peer-to-Peer' +order: 1 +parent: + title: P2P + order: 6 +--- + +# Peer-to-Peer Communication + +A CometBFT network is composed of multiple CometBFT instances, hereafter called +`nodes`, that interact by exchanging messages. + +The CometBFT protocols are designed under the assumption of a partially-connected network model. +This means that a node is not assumed to be directly connected to every other +node in the network. +Instead, each node is directly connected to only a subset of other nodes, +hereafter called its `peers`. + +The peer-to-peer (p2p) communication layer is then the component of CometBFT that: + +1. establishes connections between nodes in a CometBFT network +2. manages the communication between a node and the connected peers +3. intermediates the exchange of messages between peers in CometBFT protocols + +The specification the p2p layer is a work in progress, +tracked by [issue #19](https://github.com/cometbft/cometbft/issues/19). +The current content is organized as follows: + +- [`implementation`](/cometbft/v0.39/spec/p2p/Implementation-of-the-p2p-layer): documents the current state + of the implementation of the p2p layer, covering the main components of the + `p2p` package. The documentation covers, in a fairly comprehensive way, + the items 1. and 2. from the list above. +- [`reactor-api`](/cometbft/v0.39/spec/p2p/reactor-api/Reactor-Api): specifies the API offered by the + p2p layer to the protocol layer, through the `Reactor` abstraction. + This is a high-level specification (i.e., it should not be implementation-specific) + of the p2p layer API, covering item 3. from the list above. +- [`legacy-docs`](/cometbft/v0.39/spec/p2p/legacy-docs/Overview): We keep older documentation in + the `legacy-docs` directory, as overall, it contains useful information. + However, part of this content is redundant, + being more comprehensively covered in more recent documents, + and some implementation details might be outdated + (see [issue #981](https://github.com/cometbft/cometbft/issues/981)). + +In addition to this content, some unfinished, work in progress, and auxiliary +material can be found in the +[knowledge-base](https://github.com/cometbft/knowledge-base/tree/main/p2p) repository. diff --git a/cometbft/v0.39/spec/p2p/images/p2p-reactors.png b/cometbft/v0.39/spec/p2p/images/p2p-reactors.png new file mode 100644 index 000000000..5515976c1 Binary files /dev/null and b/cometbft/v0.39/spec/p2p/images/p2p-reactors.png differ diff --git a/cometbft/v0.39/spec/p2p/images/p2p_state.png b/cometbft/v0.39/spec/p2p/images/p2p_state.png new file mode 100644 index 000000000..c86d01682 Binary files /dev/null and b/cometbft/v0.39/spec/p2p/images/p2p_state.png differ diff --git a/cometbft/v0.39/spec/p2p/implementation/addressbook.md b/cometbft/v0.39/spec/p2p/implementation/addressbook.md new file mode 100644 index 000000000..26b950421 --- /dev/null +++ b/cometbft/v0.39/spec/p2p/implementation/addressbook.md @@ -0,0 +1,368 @@ +# Address Book + +The address book tracks information about peers, i.e., about other nodes in the network. + +The primary information stored in the address book are peer addresses. +A peer address is composed by a node ID and a network address; a network +address is composed by an IP address or a DNS name plus a port number. +The same node ID can be associated to multiple network addresses. + +There are two sources for the addresses stored in the address book. +The [Peer Exchange protocol](./pex-protocol.md) stores in the address book +the peer addresses it discovers, i.e., it learns from connected peers. +And the [Switch](./switch.md) registers the addresses of peers with which it +has interacted: to which it has dialed or from which it has accepted a +connection. + +The address book also records additional information about peers with which the +node has interacted, from which is possible to rank peers. +The Switch reports [connection attempts](#dial-attempts) to a peer address; too +much failed attempts indicate that a peer address is invalid. +Reactors, in they turn, report a peer as [good](#good-peers) when it behaves as +expected, or as a [bad peer](#bad-peers), when it misbehaves. + +There are two entities that retrieve peer addresses from the address book. +The [Peer Manager](./peer_manager.md) retrieves peer addresses to dial, so to +establish outbound connections. +This selection is random, but has a configurable bias towards peers that have +been marked as good peers. +The [Peer Exchange protocol](./pex-protocol.md) retrieves random samples of +addresses to offer (send) to peers. +This selection is also random but it includes, in particular for nodes that +operate in seed mode, some bias toward peers marked as good ones. + +## Buckets + +Peer addresses are stored in buckets. +There are buckets for new addresses and buckets for old addresses. +The buckets for new addresses store addresses of peers about which the node +does not have much information; the first address registered for a peer ID is +always stored in a bucket for new addresses. +The buckets for old addresses store addresses of peers with which the node has +interacted and that were reported as [good peers](#good-peers) by a reactor. +An old address therefore can be seen as an alias for a good address. + +> Note that new addresses does not mean bad addresses. +> The addresses of peers marked as [bad peers](#bad-peers) are removed from the +> buckets where they are stored, and temporarily kept in a table of banned peers. + +The number of buckets is fixed and there are more buckets for new addresses +(`256`) than buckets for old addresses (`64`), a ratio of 4:1. +Each bucket can store up to `64` addresses. +When a bucket becomes full, the peer address with the lowest ranking is removed +from the bucket. +The first choice is to remove bad addresses, with multiple failed attempts +associated. +In the absence of those, the *oldest* address in the bucket is removed, i.e., +the address with the oldest last attempt to dial. + +When a bucket for old addresses becomes full, the lowest-ranked peer address in +the bucket is moved to a bucket of new addresses. +When a bucket for new addresses becomes full, the lowest-ranked peer address in +the bucket is removed from the address book. +In other words, exceeding old or good addresses are downgraded to new +addresses, while exceeding new addresses are dropped. + +The bucket that stores an `address` is defined by the following two methods, +for new and old addresses: + +- `calcNewBucket(address, source) = hash(key + groupKey(source) + hash(key + groupKey(address) + groupKey(source)) % newBucketsPerGroup) % newBucketCount` +- `calcOldBucket(address) = hash(key + groupKey(address) + hash(key + address) % oldBucketsPerGroup) % oldBucketCount` + +The `key` is a fixed random 96-bit (8-byte) string. +The `groupKey` for an address is a string representing its network group. +The `source` of an address is the address of the peer from which we learn the +address.. +The first (internal) hash is reduced to an integer up to `newBucketsPerGroup = +32`, for new addresses, and `oldBucketsPerGroup = 4`, for old addresses. +The second (external) hash is reduced to bucket indexes, in the interval from 0 +to the number of new (`newBucketCount = 256`) or old (`oldBucketCount = 64`) buckets. + +Notice that new addresses with sources from the same network group are more +likely to end up in the same bucket, therefore to competing for it. +For old address, instead, two addresses are more likely to end up in the same +bucket when they belong to the same network group. + +## Adding addresses + +The `AddAddress` method adds the address of a peer to the address book. + +The added address is associated to a *source* address, which identifies the +node from which the peer address was learned. + +Addresses are added to the address book in the following situations: + +1. When a peer address is learned via PEX protocol, having the sender + of the PEX message as its source +2. When an inbound peer is added, in this case the peer itself is set as the + source of its own address +3. When the switch is instructed to dial addresses via the `DialPeersAsync` + method, in this case the node itself is set as the source + +If the added address contains a node ID that is not registered in the address +book, the address is added to a [bucket](#buckets) of new addresses. +Otherwise, the additional address for an existing node ID is **not added** to +the address book when: + +- The last address added with the same node ID is stored in an old bucket, so + it is considered a "good" address +- There are addresses associated to the same node ID stored in + `maxNewBucketsPerAddress = 4` distinct buckets +- Randomly, with a probability that increases exponentially with the number of + buckets in which there is an address with the same node ID. + So, a new address for a node ID which is already present in one bucket is + added with 50% of probability; if the node ID is present in two buckets, the + probability decreases to 25%; and if it is present in three buckets, the + probability is 12.5%. + +The new address is also added to the `addrLookup` table, which stores +`knownAddress` entries indexed by their node IDs. +If the new address is from an unknown peer, a new entry is added to the +`addrLookup` table; otherwise, the existing entry is updated with the new +address. +Entries of this table contain, among other fields, the list of buckets where +addresses of a peer are stored. +The `addrLookup` table is used by most of the address book methods (e.g., +`HasAddress`, `IsGood`, `MarkGood`, `MarkAttempt`), as it provides fast access +to addresses. + +### Errors + +- if the added address or the associated source address are nil +- if the added address is invalid +- if the added address is the local node's address +- if the added address ID is of a [banned](#bad-peers) peer +- if either the added address or the associated source address IDs are configured as private IDs +- if `routabilityStrict` is set and the address is not routable +- in case of failures computing the bucket for the new address (`calcNewBucket` method) +- if the added address instance, which is a new address, is configured as an + old address (sanity check of `addToNewBucket` method) + +## Need for Addresses + +The `NeedMoreAddrs` method verifies whether the address book needs more addresses. + +It is invoked by the PEX reactor to define whether to request peer addresses +to a new outbound peer or to a randomly selected connected peer. + +The address book needs more addresses when it has less than `1000` addresses +registered, counting all buckets for new and old addresses. + +## Pick address + +The `PickAddress` method returns an address stored in the address book, chosen +at random with a configurable bias towards new addresses. + +It is invoked by the Peer Manager to obtain a peer address to dial, as part of +its `ensurePeers` routine. +The bias starts from 10%, when the peer has no outbound peers, increasing by +10% for each outbound peer the node has, up to 90%, when the node has at least +8 outbound peers. + +The configured bias is a parameter that influences the probability of choosing +an address from a bucket of new addresses or from a bucket of old addresses. +A second parameter influencing this choice is the number of new and old +addresses stored in the address book. +In the absence of bias (i.e., if the configured bias is 50%), the probability +of picking a new address is given by the square root of the number of new +addresses divided by the sum of the square roots of the numbers of new and old +addresses. +By adding a bias toward new addresses (i.e., configured bias larger than 50%), +the portion on the sample occupied by the square root of the number of new +addresses increases, while the corresponding portion for old addresses decreases. +As a result, it becomes more likely to pick a new address at random from this sample. + +> The use of the square roots softens the impact of disproportional numbers of +> new and old addresses in the address book. This is actually the expected +> scenario, as there are 4 times more buckets for new addresses than buckets +> for old addresses. + +Once the type of address, new or old, is defined, a non-empty bucket of this +type is selected at random. +From the selected bucket, an address is chosen at random and returned. +If all buckets of the selected type are empty, no address is returned. + +## Random selection + +The `GetSelection` method returns a selection of addresses stored in the +address book, with no bias toward new or old addresses. + +It is invoked by the PEX protocol to obtain a list of peer addresses with two +purposes: + +- To send to a peer in a PEX response, in the case of outbound peers or of + nodes not operating in seed mode +- To crawl, in the case of nodes operating in seed mode, as part of every + interaction of the `crawlPeersRoutine` + +The selection is a random subset of the peer addresses stored in the +`addrLookup` table, which stores the last address added for each peer ID. +The target size of the selection is `23%` (`getSelectionPercent`) of the +number of addresses stored in the address book, but it should not be lower than +`32` (`minGetSelection`) --- if it is, all addresses in the book are returned +--- nor greater than `250` (`maxGetSelection`). + +> The random selection is produced by: +> +> - Retrieving all entries of the `addrLookup` map, which by definition are +> returned in random order. +> - Randomly shuffling the retrieved list, using the Fisher-Yates algorithm + +## Random selection with bias + +The `GetSelectionWithBias` method returns a selection of addresses stored in +the address book, with bias toward new addresses. + +It is invoked by the PEX protocol to obtain a list of peer addresses to be sent +to a peer in a PEX response. +This method is only invoked by seed nodes, when replying to a PEX request +received from an inbound peer (i.e., a peer that dialed the seed node). +The bias used in this scenario is hard-coded to 30%, meaning that 70% of +the returned addresses are expected to be old addresses. + +The number of addresses that compose the selection is computed in the same way +as for the non-biased random selection. +The bias toward new addresses is implemented by requiring that the configured +bias, interpreted as a percentage, of the select addresses come from buckets of +new addresses, while the remaining come from buckets of old addresses. +Since the number of old addresses is typically lower than the number of new +addresses, it is possible that the address book does not have enough old +addresses to include in the selection. +In this case, additional new addresses are included in the selection. +Thus, the configured bias, in practice, is towards old addresses, not towards +new addresses. + +To randomly select addresses of a type, the address book considers all +addresses present in every bucket of that type. +This list of all addresses of a type is randomly shuffled, and the requested +number of addresses are retrieved from the tail of this list. +The returned selection contains, at its beginning, a random selection of new +addresses in random order, followed by a random selection of old addresses, in +random order. + +## Dial Attempts + +The `MarkAttempt` method records a failed attempt to connect to an address. + +It is invoked by the Peer Manager when it fails dialing a peer, but the failure +is not in the authentication step (`ErrSwitchAuthenticationFailure` error). +In case of authentication errors, the peer is instead marked as a [bad peer](#bad-peers). + +The failed connection attempt is recorded in the address registered for the +peer's ID in the `addrLookup` table, which is the last address added with that ID. +The known address' counter of failed `Attempts` is increased and the failure +time is registered in `LastAttempt`. + +The possible effect of recording multiple failed connect attempts to a peer is +to turn its address into a *bad* address (do not confuse with banned addresses). +A known address becomes bad if it is stored in buckets of new addresses, and +when connection attempts: + +- Have not been made over a week, i.e., `LastAttempt` is older than a week +- Have failed 3 times and never succeeded, i.e., `LastSucess` field is unset +- Have failed 10 times in the last week, i.e., `LastSucess` is older than a week + +Addresses marked as *bad* are the first candidates to be removed from a bucket of +new addresses when the bucket becomes full. + +> Note that failed connection attempts are reported for a peer address, but in +> fact the address book records them for a peer. +> +> More precisely, failed connection attempts are recorded in the entry of the +> `addrLookup` table with reported peer ID, which contains the last address +> added for that node ID, which is not necessarily the reported peer address. + +## Good peers + +The `MarkGood` method marks a peer ID as good. + +It is invoked by the consensus reactor, via switch, when the number of useful +messages received from a peer is a multiple of `10000`. +Vote and block part messages are considered for this number, they must be valid +and not be duplicated messages to be considered useful. + +> The `SwitchReporter` type of `behaviour` package also invokes the `MarkGood` +> method when a "reason" associated with consensus votes and block parts is +> reported. +> No reactor, however, currently provides these "reasons" to the `SwitchReporter`. + +The effect of this action is that the address registered for the peer's ID in the +`addrLookup` table, which is the last address added with that ID, is marked as +good and moved to a bucket of old addresses. +An address marked as good has its failed to connect counter and timestamp reset. +If the destination bucket of old addresses is full, the oldest address in the +bucket is moved (downgraded) to a bucket of new addresses. + +Moving the peer address to a bucket of old addresses has the effect of +upgrading, or increasing the ranking of a peer in the address book. + +## Bad peers + +The `MarkBad` method marks a peer as bad and bans it for a period of time. + +This method is only invoked within the PEX reactor, with a banning time of 24 +hours, for the following reasons: + +- A peer misbehaves in the [PEX protocol](./pex-protocol.md#misbehavior) +- When the `maxAttemptsToDial` limit (`16`) is reached for a peer +- If an `ErrSwitchAuthenticationFailure` error is returned when dialing a peer + +The effect of this action is that the address registered for the peer's ID in the +`addrLookup` table, which is the last address added with that ID, is banned for +a period of time. +The banned peer is removed from the `addrLookup` table and from all buckets +where its addresses are stored. + +The information about banned peers, however, is not discarded. +It is maintained in the `badPeers` map, indexed by peer ID. +This allows, in particular, addresses of banned peers to be +[reinstated](#reinstating-addresses), i.e., to be added +back to the address book, when their ban period expires. + +## Reinstating addresses + +The `ReinstateBadPeers` method attempts to re-add banned addresses to the address book. + +It is invoked by the PEX reactor when dialing new peers. +This action is taken before requesting additional addresses to peers, +in the case that the node needs more peer addresses. + +The set of banned peer addresses is retrieved from the `badPeers` map. +Addresses that are not any longer banned, i.e., whose banned period has expired, +are added back to the address book as new addresses, while the corresponding +node IDs are removed from the `badPeers` map. + +## Removing addresses + +The `RemoveAddress` method removes an address from the address book. + +It is invoked by the switch when it dials a peer or accepts a connection from a +peer that ends up being the node itself (`IsSelf` error). +In both cases, the address dialed or accepted is also added to the address book +as a local address, via the `AddOurAddress` method. + +The same logic is also internally used by the address book for removing +addresses of a peer that is [marked as a bad peer](#bad-peers). + +The entry registered with the peer ID of the address in the `addrLookup` table, +which is the last address added with that ID, is removed from all buckets where +it is stored and from the `addrLookup` table. + +> FIXME: is it possible that addresses with the same ID as the removed address, +> but with distinct network addresses, are kept in buckets of the address book? +> While they will not be accessible anymore, as there is no reference to them +> in the `addrLookup`, they will still be there. + +## Persistence + +The `loadFromFile` method, called when the address book is started, reads +address book entries from a file, passed to the address book constructor. +The file, at this point, does not need to exist. + +The `saveRoutine` is started when the address book is started. +It saves the address book to the configured file every `dumpAddressInterval`, +hard-coded to 2 minutes. +It is also possible to save the content of the address book using the `Save` +method. +Saving the address book content to a file acquires the address book lock, also +employed by all other public methods. diff --git a/cometbft/v0.39/spec/p2p/implementation/configuration.md b/cometbft/v0.39/spec/p2p/implementation/configuration.md new file mode 100644 index 000000000..9f172c22c --- /dev/null +++ b/cometbft/v0.39/spec/p2p/implementation/configuration.md @@ -0,0 +1,49 @@ +# CometBFT p2p configuration + +This document contains configurable parameters a node operator can use to tune the p2p behaviour. + +| Parameter| Default| Description | +| --- | --- | ---| +| ListenAddress | "tcp://0.0.0.0:26656" | Address to listen for incoming connections (0.0.0.0:0 means any interface, any port) | +| ExternalAddress | "" | Address to advertise to peers for them to dial | +| [Seeds](./pex-protocol.md#seed-nodes) | empty | Comma separated list of seed nodes to connect to (ID@host:port )| +| [Persistent peers](./peer_manager.md#persistent-peers) | empty | Comma separated list of nodes to keep persistent connections to (ID@host:port ) | +| [AddrBook](./addressbook.md) | defaultAddrBookPath | Path do address book | +| AddrBookStrict | true | Set true for strict address routability rules and false for private or local networks | +| [MaxNumInboundPeers](./switch.md#accepting-peers) | 40 | Maximum number of inbound peers | +| [MaxNumOutboundPeers](./peer_manager.md#ensure-peers) | 10 | Maximum number of outbound peers to connect to, excluding persistent peers | +| [UnconditionalPeers](./switch.md#accepting-peers) | empty | These are IDs of the peers which are allowed to be (re)connected as both inbound or outbound regardless of whether the node reached `max_num_inbound_peers` or `max_num_outbound_peers` or not. | +| PersistentPeersMaxDialPeriod| 0 * time.Second | Maximum pause when redialing a persistent peer (if zero, exponential backoff is used) | +| FlushThrottleTimeout |100 * time.Millisecond| Time to wait before flushing messages out on the connection | +| MaxPacketMsgPayloadSize | 1024 | Maximum size of a message packet payload, in bytes | +| SendRate | 5120000 (5 mB/s) | Rate at which packets can be sent, in bytes/second | +| RecvRate | 5120000 (5 mB/s) | Rate at which packets can be received, in bytes/second| +| [PexReactor](./pex.md) | true | Set true to enable the peer-exchange reactor | +| SeedMode | false | Seed mode, in which node constantly crawls the network and looks for. Does not work if the peer-exchange reactor is disabled. | +| PrivatePeerIDs | empty | Comma separated list of peer IDsthat we do not add to the address book or gossip to other peers. They stay private to us. | +| AllowDuplicateIP | false | Toggle to disable guard against peers connecting from the same ip.| +| [HandshakeTimeout](./transport.md#connection-upgrade) | 20 * time.Second | Timeout for handshake completion between peers | +| [DialTimeout](./switch.md#dialing-peers) | 3 * time.Second | Timeout for dialing a peer | + + +These parameters can be set using the `$CMTHOME/config/config.toml` file. A subset of them can also be changed via command line using the following command line flags: + +| Parameter | Flag | Example | +| --- | --- | --- | +| Listen address| `p2p.laddr` | "tcp://0.0.0.0:26656" | +| Seed nodes | `p2p.seeds` | `--p2p.seeds “id100000000000000000000000000000000@1.2.3.4:26656,id200000000000000000000000000000000@2.3.4.5:4444”` | +| Persistent peers | `p2p.persistent_peers` | `--p2p.persistent_peers “id100000000000000000000000000000000@1.2.3.4:26656,id200000000000000000000000000000000@2.3.4.5:26656”` | +| Unconditional peers | `p2p.unconditional_peer_ids` | `--p2p.unconditional_peer_ids “id100000000000000000000000000000000,id200000000000000000000000000000000”` | +| PexReactor | `p2p.pex` | `--p2p.pex` | +| Seed mode | `p2p.seed_mode` | `--p2p.seed_mode` | +| Private peer ids | `p2p.private_peer_ids` | `--p2p.private_peer_ids “id100000000000000000000000000000000,id200000000000000000000000000000000”` | + + **Note on persistent peers** + + If `persistent_peers_max_dial_period` is set greater than zero, the +pause between each dial to each persistent peer will not exceed `persistent_peers_max_dial_period` +during exponential backoff and we keep trying again without giving up. + +If `seeds` and `persistent_peers` intersect, +the user will be warned that seeds may auto-close connections +and that the node may not be able to keep the connection persistent. diff --git a/cometbft/v0.39/spec/p2p/implementation/peer_manager.md b/cometbft/v0.39/spec/p2p/implementation/peer_manager.md new file mode 100644 index 000000000..3184fa03e --- /dev/null +++ b/cometbft/v0.39/spec/p2p/implementation/peer_manager.md @@ -0,0 +1,140 @@ +# Peer Manager + +The peer manager is responsible for establishing connections with peers. +It defines when a node should dial peers and which peers it should dial. +The peer manager is not an implementation abstraction of the p2p layer, +but a role that is played by the [PEX reactor](./pex.md). + +## Outbound peers + +The `ensurePeersRoutine` is a persistent routine intended to ensure that a node +is connected to `MaxNumOutboundPeers` outbound peers. +This routine is continuously executed by regular nodes, i.e. nodes not +operating in seed mode, as part of the PEX reactor implementation. + +The logic defining when the node should dial peers, for selecting peers to dial +and for actually dialing them is implemented in the `ensurePeers` method. +This method is periodically invoked -- every `ensurePeersPeriod`, with default +value to 30 seconds -- by the `ensurePeersRoutine`. + +A node is expected to dial peers whenever the number of outbound peers is lower +than the configured `MaxNumOutboundPeers` parameter. +The current number of outbound peers is retrieved from the switch, using the +`NumPeers` method, which also reports the number of nodes to which the switch +is currently dialing. +If the number of outbound peers plus the number of dialing routines equals to +`MaxNumOutboundPeers`, nothing is done. +Otherwise, the `ensurePeers` method will attempt to dial node addresses in +order to reach the target number of outbound peers. + +Once defined that the node needs additional outbound peers, the node queries +the address book for candidate addresses. +This is done using the [`PickAddress`](./addressbook.md#pick-address) method, +which returns an address selected at random on the address book, with some bias +towards new or old addresses. +When the node has up to 3 outbound peers, the adopted bias is towards old +addresses, i.e., addresses of peers that are believed to be "good". +When the node has from 5 outbound peers, the adopted bias is towards new +addresses, i.e., addresses of peers about which the node has not yet collected +much information. +So, the more outbound peers a node has, the less conservative it will be when +selecting new peers. + +The selected peer addresses are then dialed in parallel, by starting a dialing +routine per peer address. +Dialing a peer address can fail for multiple reasons. +The node might have attempted to dial the peer too many times. +In this case, the peer address is marked as bad and removed from the address book. +The node might have attempted and failed to dial the peer recently +and the exponential `backoffDuration` has not yet passed. +Or the current connection attempt might fail, which is registered in the address book. +None of these errors are explicitly handled by the `ensurePeers` method, which +also does not wait until the connections are established. + +The third step of the `ensurePeers` method is to ensure that the address book +has enough addresses. +This is done, first, by [reinstating banned peers](./addressbook.md#Reinstating-addresses) +whose ban period has expired. +Then, the node randomly selects a connected peer, which can be either an +inbound or outbound peer, to [requests addresses](./pex-protocol.md#Requesting-Addresses) +using the PEX protocol. +Last, and this action is only performed if the node could not retrieve any new +address to dial from the address book, the node dials the configured seed nodes +in order to establish a connection to at least one of them. + +### Fast dialing + +As above described, seed nodes are actually the last source of peer addresses +for regular nodes. +They are contacted by a node when, after an invocation of the `ensurePeers` +method, no suitable peer address to dial is retrieved from the address book +(e.g., because it is empty). + +Once a connection with a seed node is established, the node immediately +[sends a PEX request](./pex-protocol.md#Requesting-Addresses) to it, as it is +added as an outbound peer. +When the corresponding PEX response is received, the addresses provided by the +seed node are added to the address book. +As a result, in the next invocation of the `ensurePeers` method, the node +should be able to dial some of the peer addresses provided by the seed node. + +However, as observed in this [issue](https://github.com/tendermint/tendermint/issues/2093), +it can take some time, up to `ensurePeersPeriod` or 30 seconds, from when the +node receives new peer addresses and when it dials the received addresses. +To avoid this delay, which can be particularly relevant when the node has no +peers, a node immediately attempts to dial peer addresses when they are +received from a peer that is locally configured as a seed node. + +> This was implemented in a rough way, leading to inconsistencies described in +> this [issue](https://github.com/cometbft/cometbft/issues/486), +> fixed by this [PR](https://github.com/cometbft/cometbft/pull/3360). + +### First round + +When the PEX reactor is started, the `ensurePeersRoutine` is created and it +runs thorough the operation of a node, periodically invoking the `ensurePeers` +method. +However, if when the persistent routine is started the node already has some +peers, either inbound or outbound peers, or is dialing some addresses, the +first invocation of `ensurePeers` is delayed by a random amount of time from 0 +to `ensurePeersPeriod`. + +### Persistent peers + +The node configuration can contain a list of *persistent peers*. +Those peers have preferential treatment compared to regular peers and the node +is always trying to connect to them. +Moreover, these peers are not removed from the address book in the case of +multiple failed dial attempts. + +On startup, the node immediately tries to dial the configured persistent peers +by calling the switch's [`DialPeersAsync`](./switch.md#manual-operation) method. +This is not done in the p2p package, but it is part of the procedure to set up a node. + +> TODO: the handling of persistent peers should be described in more detail. + +### Life cycle + +The picture below is a first attempt of illustrating the life cycle of an outbound peer: + + + +A peer can be in the following states: + +- Candidate peers: peer addresses stored in the address boook, that can be + retrieved via the [`PickAddress`](./addressbook.md#pick-address) method +- [Dialing](./switch.md#dialing-peers): peer addresses that are currently being + dialed. This state exists to ensure that a single dialing routine exist per peer. +- [Reconnecting](./switch.md#reconnect-to-peer): persistent peers to which a node + is currently reconnecting, as a previous connection attempt has failed. +- Connected peers: peers that a node has successfully dialed, added as outbound peers. +- [Bad peers](./addressbook.md#bad-peers): peers marked as bad in the address + book due to exhibited [misbehavior](./pex-protocol.md#misbehavior). + Peers can be reinstated after being marked as bad. + +## Pending of documentation + +The `dialSeeds` method of the PEX reactor. + +The `dialPeer` method of the PEX reactor. +This includes `dialAttemptsInfo`, `maxBackoffDurationForPeer` methods. diff --git a/cometbft/v0.39/spec/p2p/implementation/pex-protocol.md b/cometbft/v0.39/spec/p2p/implementation/pex-protocol.md new file mode 100644 index 000000000..760a56bd9 --- /dev/null +++ b/cometbft/v0.39/spec/p2p/implementation/pex-protocol.md @@ -0,0 +1,240 @@ +# Peer Exchange Protocol + +The Peer Exchange (PEX) protocol enables nodes to exchange peer addresses, thus +implementing a peer discovery mechanism. + +The PEX protocol uses two messages: + +- `PexRequest`: sent by a node to [request](#requesting-addresses) peer + addresses to a peer +- `PexAddrs`: a list of peer addresses [provided](#providing-addresses) to a + peer as response to a `PexRequest` message + +While all nodes, with few exceptions, participate on the PEX protocol, +a subset of nodes, configured as [seed nodes](#seed-nodes) have a particular +role in the protocol. +They crawl the network, connecting to random peers, in order to learn as many +peer addresses as possible to provide to other nodes. + +## Requesting Addresses + +A node requests peer addresses by sending a `PexRequest` message to a peer. + +For regular nodes, not operating in seed mode, a PEX request is sent when +the node *needs* peers addresses, a condition checked: + +1. When an *outbound* peer is added, causing the node to request addresses from + the new peer +2. Periodically, by the `ensurePeersRoutine`, causing the node to request peer + addresses to a randomly selected peer + +A node needs more peer addresses when its addresses book has +[less than 1000 records](./addressbook.md#need-for-addresses). +It is thus reasonable to assume that the common case is that a peer needs more +peer addresses, so that PEX requests are sent whenever the above two situations happen. + +A PEX request is sent when a new *outbound* peer is added. +The same does not happen with new inbound peers because the implementation +considers outbound peers, that the node has chosen for dialing, more +trustworthy than inbound peers, that the node has accepted. +Moreover, when a node is short of peer addresses, it dials the configured seed nodes; +since they are added as outbound peers, the node can immediately request peer addresses. + +The `ensurePeersRoutine` periodically checks, by default every 30 seconds (`ensurePeersPeriod`), +whether the node has enough outbound peers. +If it does not have, the node tries dialing some peer addresses stored in the address book. +As part of this procedure, the node selects a peer at random, +from the set of connected peers retrieved from the switch, +and sends a PEX request to the selected peer. + +Sending a PEX request to a peer is implemented by the `RequestAddrs` method of +the PEX reactor. + +### Responses + +After a PEX request is sent to a peer, the node expects to receive, +as a response, a `PexAddrs` message from the peer. +This message encodes a list of peer addresses that are +[added to address book](./addressbook.md#adding-addresses), +having the peer from which the PEX response was received as their source. + +Received PEX responses are handled by the `ReceiveAddrs` method of the PEX reactor. +In the case of a PEX response received from a peer which is configured as +a seed node, the PEX reactor attempts immediately to dial the provided peer +addresses, as detailed [here](./peer_manager.md#fast-dialing). + +### Misbehavior + +Sending multiple PEX requests to a peer, before receiving a reply from it, +is considered a misbehavior. +To prevent it, the node maintains a `requestsSent` set of outstanding +requests, indexed by destination peers. +While a peer ID is present in the `requestsSent` set, the node does not send +further PEX requests to that peer. +A peer ID is removed from the `requestsSent` set when a PEX response is +received from it. + +Sending a PEX response to a peer that has not requested peer addresses +is also considered a misbehavior. +So, if a PEX response is received from a peer that is not registered in +the `requestsSent` set, a `ErrUnsolicitedList` error is produced. +This leads the peer to be disconnected and [marked as a bad peer](./addressbook.md#bad-peers). + +## Providing Addresses + +When a node receives a `PexRequest` message from a peer, +it replies with a `PexAddrs` message. + +This message encodes a [random selection of peer addresses](./addressbook.md#random-selection) +retrieved from the address book. + +Sending a PEX response to a peer is implemented by the `SendAddrs` method of +the PEX reactor. + +### Misbehavior + +Requesting peer addresses too often is considered a misbehavior. +Since node are expected to send PEX requests every `ensurePeersPeriod`, +the minimum accepted interval between requests from the same peer is set +to `ensurePeersPeriod / 3`, 10 seconds by default. + +The `receiveRequest` method is responsible for verifying this condition. +The node keeps a `lastReceivedRequests` map with the time of the last PEX +request received from every peer. +If the interval between successive requests is less than the minimum accepted +one, the peer is disconnected and [marked as a bad peer](./addressbook.md#bad-peers). +An exception is made for the first two PEX requests received from a peer. + +> The probably reason is that, when a new peer is added, the two conditions for +> a node to request peer addresses can be triggered with an interval lower than +> the minimum accepted interval. +> Since this is a legit behavior, it should not be punished. + +## Seed nodes + +A seed node is a node configured to operate in `SeedMode`. + +### Crawling peers + +Seed nodes crawl the network, connecting to random peers and sending PEX +requests to them, in order to learn as many peer addresses as possible. +More specifically, a node operating in seed mode sends PEX requests in two cases: + +1. When an outbound peer is added, and the seed node needs more peer addresses, + it requests peer addresses to the new peer +2. Periodically, the `crawlPeersRoutine` sends PEX requests to a random set of + peers, whose addresses are registered in the Address Book + +The first case also applies for nodes not operating in seed mode. +The second case replaces the second for regular nodes, as seed nodes do not +run the `ensurePeersRoutine`, as regular nodes, +but run the `crawlPeersRoutine`, which is not run by regular nodes. + +The `crawlPeersRoutine` periodically, every 30 seconds (`crawlPeerPeriod`), +starts a new peer discovery round. +First, the seed node retrieves a random selection of peer addresses from its +Address Book. +This selection is produced in the same way as in the random selection of peer +addresses that are [provided](#providing-addresses) to a requesting peer. +Peers that the seed node has crawled recently, +less than 2 minutes ago (`minTimeBetweenCrawls`), are removed from this selection. +The remaining peer addresses are registered in the `crawlPeerInfos` table. + +The seed node is not necessarily connected to the peer whose address is +selected for each round of crawling. +So, the seed node dials the selected peer addresses. +This is performed in foreground, one peer at a time. +As a result, a round of crawling can take a substantial amount of time. +For each selected peer it succeeds dialing to, this include already connected +peers, the seed node sends a PEX request. + +Dialing a selected peer address can fail for multiple reasons. +The seed node might have attempted to dial the peer too many times. +In this case, the peer address is marked as [bad in the address book](./addressbook.md#bad-peers). +The seed node might have attempted to dial the peer recently, without success, +and the exponential `backoffDuration` has not yet passed. +Or the current connection attempt might fail, which is registered in the address book. + +Failures to dial to a peer address produce an information that is important for +a seed node. +They indicate that a peer is unreachable, or is not operating correctly, and +therefore its address should not be provided to other nodes. +This occurs when, due to multiple failed connection attempts or authentication +failures, the peer address ends up being removed from the address book. +As a result, the periodically crawling of selected peers not only enables the +discovery of new peers, but also allows the seed node to stop providing +addresses of bad peers. + +### Offering addresses + +Nodes operating in seed mode handle PEX requests differently than regular +nodes, whose operation is described [here](#providing-addresses). + +This distinction exists because nodes dial a seed node with the main, if not +exclusive goal of retrieving peer addresses. +In other words, nodes do not dial a seed node because they intend to have it as +a peer in the multiple CometBFT protocols, but because they believe that a +seed node is a good source of addresses of nodes to which they can establish +connections and interact in the multiple CometBFT protocols. + +So, when a seed node receives a `PexRequest` message from an inbound peer, +it sends a `PexAddrs` message, containing a selection of peer +addresses, back to the peer and *disconnects* from it. +Seed nodes therefore treat inbound connections from peers as a short-term +connections, exclusively intended to retrieve peer addresses. +Once the requested peer addresses are sent, the connection with the peer is closed. + +Moreover, the selection of peer addresses provided to inbound peers by a seed +node, although still essentially random, has a [bias toward old +addresses](./addressbook.md#random-selection-with-bias). +The selection bias is defined by `biasToSelectNewPeers`, hard-coded to `30%`, +meaning that `70%` of the peer addresses provided by a seed node are expected +to be old addresses. +Although this nomenclature is not clear, *old* addresses are the addresses that +survived the most in the address book, that is, are addresses that the seed +node believes being from *good* peers (more details [here](./addressbook.md#good-peers)). + +Another distinction is on the handling of potential [misbehavior](#misbehavior-1) +of peers requesting addresses. +A seed node does not enforce, a priori, a minimal interval between PEX requests +from inbound peers. +Instead, it does not reply to more than one PEX request per peer inbound +connection, and, as above mentioned, it disconnects from incoming peers after +responding to them. +If the same peer dials again to the seed node and requests peer addresses, the +seed node will reply to this peer like it was the first time it has requested +peer addresses. + +> This is more an implementation restriction than a desired behavior. +> The `lastReceivedRequests` map stores the last time a PEX request was +> received from a peer, and the entry relative to a peer is removed from this +> map when the peer is disconnected. +> +> It is debatable whether this approach indeed prevents abuse against seed nodes. + +### Disconnecting from peers + +Seed nodes treat connections with peers as short-term connections, which are +mainly, if not exclusively, intended to exchange peer addresses. + +In the case of inbound peers, that have dialed the seed node, the intent of the +connection is achieved once a PEX response is sent to the peer. +The seed node thus disconnects from an inbound peer after sending a `PexAddrs` +message to it. + +In the case of outbound peers, which the seed node has dialed for crawling peer +addresses, the intent of the connection is essentially achieved when a PEX +response is received from the peer. +The seed node, however, does not disconnect from a peer after receiving a +selection of peer addresses from it. +As a result, after some rounds of crawling, a seed node will have established +connections to a substantial amount of peers. + +To couple with the existence of multiple connections with peers that have no +longer purpose for the seed node, the `crawlPeersRoutine` also invokes, after +each round of crawling, the `attemptDisconnects` method. +This method retrieves the list of connected peers from the switch, and +disconnects from peers that are not persistent peers, and with which a +connection is established for more than `SeedDisconnectWaitPeriod`. +This period is a configuration parameter, set to 28 hours when the PEX reactor +is created by the default node constructor. diff --git a/cometbft/v0.39/spec/p2p/implementation/pex.md b/cometbft/v0.39/spec/p2p/implementation/pex.md new file mode 100644 index 000000000..8243eaa55 --- /dev/null +++ b/cometbft/v0.39/spec/p2p/implementation/pex.md @@ -0,0 +1,111 @@ +# PEX Reactor + +The PEX reactor is one of the reactors running in a CometBFT node. + +Its implementation is located in the `p2p/pex` package, and it is considered +part of the implementation of the p2p layer. + +This document overviews the implementation of the PEX reactor, describing how +the methods from the `Reactor` interface are implemented. + +The actual operation of the PEX reactor is presented in documents describing +the roles played by the PEX reactor in the p2p layer: + +- [Address Book](./addressbook.md): stores known peer addresses and information + about peers to which the node is connected or has attempted to connect +- [Peer Manager](./peer_manager.md): manages connections established with peers, + defining when a node should dial peers and which peers it should dial +- [Peer Exchange protocol](./pex-protocol.md): enables nodes to exchange peer + addresses, thus implementing a peer discovery service + +## OnStart + +The `OnStart` method implements `BaseService` and starts the PEX reactor. + +The [address book](./addressbook.md), which is a `Service` is started. +This loads the address book content from disk, +and starts a routine that periodically persists the address book content to disk. + +The PEX reactor is configured with the addresses of a number of seed nodes, +the `Seeds` parameter of the `ReactorConfig`. +The addresses of seed nodes are parsed into `NetAddress` instances and resolved +into IP addresses, which is implemented by the `checkSeeds` method. +Valid seed node addresses are stored in the `seedAddrs` field, +and are used by the `dialSeeds` method to contact the configured seed nodes. + +The last action is to start one of the following persistent routines, based on +the `SeedMode` configuration parameter: + +- Regular nodes run the `ensurePeersRoutine` to check whether the node has + enough outbound peers, dialing peers when necessary +- Seed nodes run the `crawlPeersRoutine` to periodically start a new round + of [crawling](./pex-protocol.md#Crawling-peers) to discover as many peer + addresses as possible + +### Errors + +Errors encountered when loading the address book from disk are returned, +and prevent the reactor from being started. +An exception is made for the `service.ErrAlreadyStarted` error, which is ignored. + +Errors encountered when parsing the configured addresses of seed nodes +are returned and cause the reactor startup to fail. +An exception is made for DNS resolution `ErrNetAddressLookup` errors, +which are not deemed fatal and are only logged as invalid addresses. + +If none of the configured seed node addresses is valid, and the loaded address +book is empty, the reactor is not started and an error is returned. + +## OnStop + +The `OnStop` method implements `BaseService` and stops the PEX reactor. + +The address book routine that periodically saves its content to disk is stopped. + +## GetChannels + +The `GetChannels` method, from the `Reactor` interface, returns the descriptor +of the channel used by the PEX protocol. + +The channel ID is `PexChannel` (0), with priority `1`, send queue capacity of +`10`, and maximum message size of `64000` bytes. + +## AddPeer + +The `AddPeer` method, from the `Reactor` interface, +adds a new peer to the PEX protocol. + +If the new peer is an **inbound peer**, i.e., if the peer has dialed the node, +the peer's address is [added to the address book](./addressbook.md#adding-addresses). +Since the peer was authenticated when establishing a secret connection with it, +the source of the peer address is trusted, and its source is set by the peer itself. +In the case of an outbound peer, the node should already have its address in +the address book, as the switch has dialed the peer. + +If the peer is an **outbound peer**, i.e., if the node has dialed the peer, +and the PEX protocol needs more addresses, +the node [sends a PEX request](./pex-protocol.md#Requesting-Addresses) to the peer. +The same is not done when inbound peers are added because they are deemed least +trustworthy than outbound peers. + +## RemovePeer + +The `RemovePeer` method, from the `Reactor` interface, +removes a peer from the PEX protocol. + +The peer's ID is removed from the tables tracking PEX requests +[sent](./pex-protocol.md#misbehavior) but not yet replied +and PEX requests [received](./pex-protocol.md#misbehavior-1). + +## Receive + +The `Receive` method, from the `Reactor` interface, +handles a message received by the PEX protocol. + +A node receives two type of messages as part of the PEX protocol: + +- `PexRequest`: a request for addresses received from a peer, handled as + described [here](./pex-protocol.md#providing-addresses) +- `PexAddrs`: a list of addresses received from a peer, as a reponse to a PEX + request sent by the node, as described [here](./pex-protocol.md#responses) + diff --git a/cometbft/v0.39/spec/p2p/implementation/switch.md b/cometbft/v0.39/spec/p2p/implementation/switch.md new file mode 100644 index 000000000..e87985336 --- /dev/null +++ b/cometbft/v0.39/spec/p2p/implementation/switch.md @@ -0,0 +1,238 @@ +# Switch + +The switch is a core component of the p2p layer. +It manages the procedures for [dialing peers](#dialing-peers) and +[accepting](#accepting-peers) connections from peers, which are actually +implemented by the [transport](./transport.md). +It also manages the reactors, i.e., protocols implemented by the node that +interact with its peers. +Once a connection with a peer is established, the peer is [added](#add-peer) to +the switch and all registered reactors. +Reactors may also instruct the switch to [stop a peer](#stop-peer), namely +disconnect from it. +The switch, in this case, makes sure that the peer is removed from all +registered reactors. + +## Dialing peers + +Dialing a peer is implemented by the `DialPeerWithAddress` method. + +This method is invoked by the [peer manager](./peer_manager.md#ensure-peers) +to dial a peer address and establish a connection with an outbound peer. + +The switch keeps a single dialing routine per peer ID. +This is ensured by keeping a synchronized map `dialing` with the IDs of peers +to which the peer is dialing. +A peer ID is added to `dialing` when the `DialPeerWithAddress` method is called +for that peer, and it is removed when the method returns for whatever reason. +The method returns immediately when invoked for a peer which ID is already in +the `dialing` structure. + +The actual dialing is implemented by the [`Dial`](./transport.md#dial) method +of the transport configured for the switch, in the `addOutboundPeerWithConfig` +method. +If the transport succeeds establishing a connection, the returned `Peer` is +added to the switch using the [`addPeer`](#add-peer) method. +This operation can fail, returning an error. In this case, the switch invokes +the transport's [`Cleanup`](./transport.md#cleanup) method to clean any resources +associated with the peer. + +If the transport fails to establish a connection with the peer that is configured +as a persistent peer, the switch spawns a routine to [reconnect to the peer](#reconnect-to-peer). +If the peer is already in the `reconnecting` state, the spawned routine has no +effect and returns immediately. +This is in fact a likely scenario, as the `reconnectToPeer` routine relies on +this same `DialPeerWithAddress` method for dialing peers. + +### Manual operation + +The `DialPeersAsync` method receives a list of peer addresses (strings) +and dials all of them in parallel. +It is invoked in two situations: + +- In the [setup](https://github.com/cometbft/cometbft/blob/v0.34.x/node/node.go#L987) +of a node, to establish connections with every configured persistent peer +- In the RPC package, to implement two unsafe RPC commands, not used in production: + [`DialSeeds`](https://github.com/cometbft/cometbft/blob/v0.34.x/rpc/core/net.go#L47) and + [`DialPeers`](https://github.com/cometbft/cometbft/blob/v0.34.x/rpc/core/net.go#L87) + +The received list of peer addresses to dial is parsed into `NetAddress` instances. +In case of parsing errors, the method returns. An exception is made for +DNS resolution `ErrNetAddressLookup` errors, which do not interrupt the procedure. + +As the peer addresses provided to this method are typically not known by the node, +contrarily to the addressed dialed using the `DialPeerWithAddress` method, +they are added to the node's address book, which is persisted to disk. + +The switch dials the provided peers in parallel. +The list of peer addresses is randomly shuffled, and for each peer a routine is +spawned. +Each routine sleeps for a random interval, up to 3 seconds, then invokes the +`DialPeerWithAddress` method that actually dials the peer. + +### Reconnect to peer + +The `reconnectToPeer` method is invoked when a connection attempt to a peer fails, +and the peer is configured as a persistent peer. + +The `reconnecting` synchronized map keeps the peer's in this state, identified +by their IDs (string). +This should ensure that a single instance of this method is running at any time. +The peer is kept in this map while this method is running for it: it is set on +the beginning, and removed when the method returns for whatever reason. +If the peer is already in the `reconnecting` state, nothing is done. + +The remaining of the method performs multiple connection attempts to the peer, +via `DialPeerWithAddress` method. +If a connection attempt succeeds, the methods returns and the routine finishes. +The same applies when an `ErrCurrentlyDialingOrExistingAddress` error is +returned by the dialing method, as it indicates that peer is already connected +or that another routine is attempting to (re)connect to it. + +A first set of connection attempts is done at (about) regular intervals. +More precisely, between two attempts, the switch waits for a interval of +`reconnectInterval`, hard-coded to 5 seconds, plus a random jitter up to +`dialRandomizerIntervalMilliseconds`, hard-coded to 3 seconds. +At most `reconnectAttempts`, hard-coded to 20, are made using this +regular-interval approach. + +A second set of connection attempts is done with exponentially increasing +intervals. +The base interval `reconnectBackOffBaseSeconds` is hard-coded to 3 seconds, +which is also the increasing factor. +The exponentially increasing dialing interval is adjusted as well by a random +jitter up to `dialRandomizerIntervalMilliseconds`. +At most `reconnectBackOffAttempts`, hard-coded to 10, are made using this approach. + +> Note: the first sleep interval, to which a random jitter is applied, is 1, +> not `reconnectBackOffBaseSeconds`, as the first exponent is `0`... + +## Accepting peers + +The `acceptRoutine` method is a persistent routine that handles connections +accepted by the transport configured for the switch. + +The [`Accept`](./transport.md#accept) method of the configured transport +returns a `Peer` with which an inbound connection was established. +The switch accepts a new peer if the maximum number of inbound peers was not +reached, or if the peer was configured as an _unconditional peer_. +The maximum number of inbound peers is determined by the `MaxNumInboundPeers` +configuration parameter, whose default value is `40`. + +If accepted, the peer is added to the switch using the [`addPeer`](#add-peer) method. +If the switch does not accept the established incoming connection, or if the +`addPeer` method returns an error, the switch invokes the transport's +[`Cleanup`](./transport.md#cleanup) method to clean any resources associated +with the peer. + +The transport's `Accept` method can also return a number of errors. +Errors of `ErrRejected` or `ErrFilterTimeout` types are ignored, +an `ErrTransportClosed` causes the accepted routine to be interrupted, +while other errors cause the routine to panic. + +> TODO: which errors can cause the routine to panic? + +## Add peer + +The `addPeer` method adds a peer to the switch, +either after dialing (by `addOutboundPeerWithConfig`, called by `DialPeerWithAddress`) +a peer and establishing an outbound connection, +or after accepting (`acceptRoutine`) a peer and establishing an inbound connection. + +The first step is to invoke the `filterPeer` method. +It checks whether the peer is already in the set of connected peers, +and whether any of the configured `peerFilter` methods reject the peer. +If the peer is already present or it is rejected by any filter, the `addPeer` +method fails and returns an error. + +Then, the new peer is started, added to the set of connected peers, and added +to all reactors. +More precisely, first the new peer's information is first provided to every +reactor (`InitPeer` method). +Next, the peer's sending and receiving routines are started, and the peer is +added to set of connected peers. +These two operations can fail, causing `addPeer` to return an error. +Then, in the absence of previous errors, the peer is added to every reactor (`AddPeer` method). + +> Adding the peer to the peer set returns a `ErrSwitchDuplicatePeerID` error +> when a peer with the same ID is already presented. +> +> TODO: Starting a peer could be reduced as starting the MConn with that peer? + +## Stop peer + +There are two methods for stopping a peer, namely disconnecting from it, and +removing it from the table of connected peers. + +The `StopPeerForError` method is invoked to stop a peer due to an external +error, which is provided to method as a generic "reason". + +The `StopPeerGracefully` method stops a peer in the absence of errors or, more +precisely, not providing to the switch any "reason" for that. + +In both cases the `Peer` instance is stopped, the peer is removed from all +registered reactors, and finally from the list of connected peers. + +> Issue https://github.com/tendermint/tendermint/issues/3338 is mentioned in +> the internal `stopAndRemovePeer` method explaining why removing the peer from +> the list of connected peers is the last action taken. + +When there is a "reason" for stopping the peer (`StopPeerForError` method) +and the peer is a persistent peer, the method creates a routine to attempt +reconnecting to the peer address, using the `reconnectToPeer` method. +If the peer is an outbound peer, the peer's address is know, since the switch +has dialed the peer. +Otherwise, the peer address is retrieved from the `NodeInfo` instance from the +connection handshake. + +## Add reactor + +The `AddReactor` method registers a `Reactor` to the switch. + +The reactor is associated to the set of channel ids it employs. +Two reactors (in the same node) cannot share the same channel id. + +There is a call back to the reactor, in which the switch passes itself to the +reactor. + +## Remove reactor + +The `RemoveReactor` method unregisters a `Reactor` from the switch. + +The reactor is disassociated from the set of channel ids it employs. + +There is a call back to the reactor, in which the switch passes `nil` to the +reactor. + +## OnStart + +This is a `BaseService` method. + +All registered reactors are started. + +The switch's `acceptRoutine` is started. + +## OnStop + +This is a `BaseService` method. + +All (connected) peers are stopped and removed from the peer's list using the +`stopAndRemovePeer` method. + +All registered reactors are stopped. + +## Broadcast + +This method broadcasts a message on a channel, by sending the message in +parallel to all connected peers. + +The method spawns a thread for each connected peer, invoking the `Send` method +provided by each `Peer` instance with the provided message and channel ID. +The return value (a boolean) of these calls are redirected to a channel that is +returned by the method. + +> TODO: detail where this method is invoked: +> +> - By the consensus protocol, in `broadcastNewRoundStepMessage`, +> `broadcastNewValidBlockMessage`, and `broadcastHasVoteMessage` +> - By the state sync protocol diff --git a/cometbft/v0.39/spec/p2p/implementation/transport.md b/cometbft/v0.39/spec/p2p/implementation/transport.md new file mode 100644 index 000000000..121e48467 --- /dev/null +++ b/cometbft/v0.39/spec/p2p/implementation/transport.md @@ -0,0 +1,221 @@ +# Transport + +The transport establishes secure and authenticated connections with peers. + +The transport [`Dial`](#dial)s peer addresses to establish outbound connections, +and [`Listen`](#listen)s in a configured network address +to [`Accept`](#accept) inbound connections from peers. + +The transport establishes raw TCP connections with peers +and [upgrade](#connection-upgrade) them into authenticated secret connections. +The established secret connection is then wrapped into `Peer` instance, which +is returned to the caller, typically the [switch](./switch.md). + +## Dial + +The `Dial` method is used by the switch to establish an outbound connection with a peer. +It is a synchronous method, which blocks until a connection is established or an error occurs. +The method returns an outbound `Peer` instance wrapping the established connection. + +The transport first dials the provided peer's address to establish a raw TCP connection. +The dialing maximum duration is determined by `dialTimeout`, hard-coded to 1 second. +The established raw connection is then submitted to a set of [filters](#connection-filtering), +which can reject it. +If the connection is not rejected, it is recorded in the table of established connections. + +The established raw TCP connection is then [upgraded](#connection-upgrade) into +an authenticated secret connection. +This procedure should ensure, in particular, that the public key of the remote peer +matches the ID of the dialed peer, which is part of peer address provided to this method. +In the absence of errors, +the established secret connection (`conn.SecretConnection` type) +and the information about the peer (`NodeInfo` record) retrieved and verified +during the version handshake, +are wrapped into an outbound `Peer` instance and returned to the switch. + +## Listen + +The `Listen` method produces a TCP listener instance for the provided network +address, and spawns an `acceptPeers` routine to handle the raw connections +accepted by the listener. +The `NetAddress` method exports the listen address configured for the transport. + +The maximum number of simultaneous incoming connections accepted by the listener +is bound to `MaxNumInboundPeer` plus the configured number of unconditional peers, +using the `MultiplexTransportMaxIncomingConnections` option, +in the node [initialization](https://github.com/cometbft/cometbft/blob/v0.34.x/node/node.go#L563). + +This method is called when a node is [started](https://github.com/cometbft/cometbft/blob/v0.34.x/node/node.go#L974). +In case of errors, the `acceptPeers` routine is not started and the error is returned. + +## Accept + +The `Accept` method returns to the switch inbound connections established with a peer. +It is a synchronous method, which blocks until a connection is accepted or an error occurs. +The method returns an inbound `Peer` instance wrapping the established connection. + +The transport handles incoming connections in the `acceptPeers` persistent routine. +This routine is started by the [`Listen`](#listen) method +and accepts raw connections from a TCP listener. +A new routine is spawned for each accepted connection. +The raw connection is submitted to a set of [filters](#connection-filtering), +which can reject it. +If the connection is not rejected, it is recorded in the table of established connections. + +The established raw TCP connection is then [upgraded](#connection-upgrade) into +an authenticated secret connection. +The established secret connection (`conn.SecretConnection` type), +the information about the peer (`NodeInfo` record) retrieved and verified +during the version handshake, +as well any error returned in this process are added to a queue of accepted connections. +This queue is consumed by the `Accept` method. + +Handling accepted connection asynchronously was introduced due to this issue: https://github.com/tendermint/tendermint/issues/2047 + +## Connection Filtering + +The `filterConn` method is invoked for every new raw connection established by the transport. +Its main goal is avoid the transport to maintain duplicated connections with the same peer. +It also runs a set of configured connection filters. + +The transports keeps a table `conns` of established connections. +The table maps the remote address returned by a generic connection to a list of +IP addresses, to which the connection remote address is resolved. +If the remote address of the new connection is already present in the table, +the connection is rejected. +Otherwise, the connection's remote address is resolved into a list of IPs, +which are recorded in the established connections table. + +The connection and the resolved IPs are then passed through a set of connection filters, +configured via the `MultiplexTransportConnFilters` transport option. +The maximum duration for the filters execution, which is performed in parallel, +is determined by `filterTimeout`. +Its default value is 5 seconds, +which can be changed using the `MultiplexTransportFilterTimeout` transport option. + +If the connection and the resolved remote addresses are not filtered out, +the transport registers them into the `conns` table and returns. + +In case of errors, the connection is removed from the table of established +connections and closed. + +### Errors + +If the address of the new connection is already present in the `conns` table, +an `ErrRejected` error with the `isDuplicate` reason is returned. + +If the IP resolution of the connection's remote address fails, +an `AddrError` or `DNSError` error is returned. + +If any of the filters reject the connection, +an `ErrRejected` error with the `isRejected` reason is returned. + +If the filters execution times out, +an `ErrFilterTimeout` error is returned. + +## Connection Upgrade + +The `upgrade` method is invoked for every new raw connection established by the +transport that was not [filtered out](#connection-filtering). +It upgrades an established raw TCP connection into a secret authenticated +connection, and validates the information provided by the peer. + +This is a complex procedure, that can be summarized by the following three +message exchanges between the node and the new peer: + +1. Encryption: the nodes produce ephemeral key pairs and exchange ephemeral + public keys, from which are derived: (i) a pair of secret keys used to + encrypt the data exchanged between the nodes, and (ii) a challenge message. +1. Authentication: the nodes exchange their persistent public keys and a + signature of the challenge message produced with the their persistent + private keys. This allows validating the peer's persistent public key, + which plays the role of node ID. +1. Version handshake: nodes exchange and validate each other `NodeInfo` records. + This records contain, among other fields, their node IDs, the network/chain + ID they are part of, and the list of supported channel IDs. + +Steps (1) and (2) are implemented in the `conn` package. +In case of success, they produce the secret connection that is actually used by +the node to communicate with the peer. +An overview of this procedure, which implements the station-to-station (STS) +[protocol][sts-paper] ([PDF][sts-paper-pdf]), can be found [here][peer-sts]. +The maximum duration for establishing a secret connection with the peer is +defined by `handshakeTimeout`, hard-coded to 3 seconds. + +The established secret connection stores the persistent public key of the peer, +which has been validated via the challenge authentication of step (2). +If the connection being upgraded is an outbound connection, i.e., if the node has +dialed the peer, the dialed peer's ID is compared to the peer's persistent public key: +if they do not match, the connection is rejected. +This verification is not performed in the case of inbound (accepted) connections, +as the node does not know a priori the remote node's ID. + +Step (3), the version handshake, is performed by the transport. +Its maximum duration is also defined by `handshakeTimeout`, hard-coded to 3 seconds. +The version handshake retrieves the `NodeInfo` record of the new peer, +which can be rejected for multiple reasons, listed [here][peer-handshake]. + +If the connection upgrade succeeds, the method returns the established secret +connection, an instance of `conn.SecretConnection` type, +and the `NodeInfo` record of the peer. + +In case of errors, the connection is removed from the table of established +connections and closed. + +### Errors + +The timeouts for steps (1) and (2), and for step (3), are configured as the +deadline for operations on the TCP connection that is being upgraded. +If this deadline it is reached, the connection produces an +`os.ErrDeadlineExceeded` error, returned by the corresponding step. + +Any error produced when establishing a secret connection with the peer (steps 1 and 2) or +during the version handshake (step 3), including timeouts, +is encapsulated into an `ErrRejected` error with reason `isAuthFailure` and returned. + +If the upgraded connection is an outbound connection, and the peer ID learned in step (2) +does not match the dialed peer's ID, +an `ErrRejected` error with reason `isAuthFailure` is returned. + +If the peer's `NodeInfo` record, retrieved in step (3), is invalid, +or if reports a node ID that does not match peer ID learned in step (2), +an `ErrRejected` error with reason `isAuthFailure` is returned. +If it reports a node ID equals to the local node ID, +an `ErrRejected` error with reason `isSelf` is returned. +If it is not compatible with the local `NodeInfo`, +an `ErrRejected` error with reason `isIncompatible` is returned. + +## Close + +The `Close` method closes the TCP listener created by the `Listen` method, +and sends a signal for interrupting the `acceptPeers` routine. + +This method is called when a node is [stopped](https://github.com/cometbft/cometbft/blob/v0.34.x/node/node.go#L1023). + +## Cleanup + +The `Cleanup` method receives a `Peer` instance, +and removes the connection established with a peer from the table of established connections. +It also invokes the `Peer` interface method to close the connection associated with a peer. + +It is invoked when the connection with a peer is closed. + +## Supported channels + +The `AddChannel` method registers a channel in the transport. + +The channel ID is added to the list of supported channel IDs, +stored in the local `NodeInfo` record. + +The `NodeInfo` record is exchanged with peers in the version handshake. +For this reason, this method is not invoked with a started transport. + +> The only call to this method is performed in the `CustomReactors` constructor +> option of a node, i.e., before the node is started. +> Note that the default list of supported channel IDs, including the default reactors, +> is provided to the transport as its original `NodeInfo` record. + +[peer-sts]: ../legacy-docs/peer.md#authenticated-encryption-handshake +[peer-handshake]: ../legacy-docs/peer.md#cometbft-version-handshake +[sts-paper]: https://link.springer.com/article/10.1007/BF00124891 +[sts-paper-pdf]: https://github.com/tendermint/tendermint/blob/0.1/docs/sts-final.pdf diff --git a/cometbft/v0.39/spec/p2p/implementation/types.md b/cometbft/v0.39/spec/p2p/implementation/types.md new file mode 100644 index 000000000..cef263293 --- /dev/null +++ b/cometbft/v0.39/spec/p2p/implementation/types.md @@ -0,0 +1,233 @@ +# Types adopted in the p2p implementation + +This document lists the packages and source files, excluding test units, that +implement the p2p layer, and summarizes the main types they implement. +Types play the role of classes in Go. + +The reference version for this documentation is the branch +[`v0.34.x`](https://github.com/cometbft/cometbft/tree/v0.34.x/p2p). + +State of August 2022. + +## Package `p2p` + +Implementation of the p2p layer of CometBFT. + +### `base_reactor.go` + +`Reactor` interface. + +`BaseReactor` implements `Reactor`. + +**Not documented yet**. + +### `conn_set.go` + +`ConnSet` interface, a "lookup table for connections and their ips". + +Internal type `connSet` implements the `ConnSet` interface. + +Used by the [transport](#transportgo) to store connected peers. + +### `errors.go` + +Defines several error types. + +`ErrRejected` enumerates a number of reason for which a peer was rejected. +Mainly produced by the [transport](#transportgo), +but also by the [switch](#switchgo). + +`ErrSwitchDuplicatePeerID` is produced by the `PeerSet` used by the [switch](#switchgo). + +`ErrSwitchConnectToSelf` is handled by the [switch](#switchgo), +but currently is not produced outside tests. + +`ErrSwitchAuthenticationFailure` is handled by the [PEX reactor](#pex_reactorgo), +but currently is not produced outside tests. + +`ErrTransportClosed` is produced by the [transport](#transportgo) +and handled by the [switch](#switchgo). + +`ErrNetAddressNoID`, `ErrNetAddressInvalid`, and `ErrNetAddressLookup` +are parsing a string to create an instance of `NetAddress`. +It can be returned in the setup of the [switch](#switchgo) +and of the [PEX reactor](#pex_reactorgo), +as well when the [transport](#transportgo) validates a `NodeInfo`, as part of +the connection handshake. + +`ErrCurrentlyDialingOrExistingAddress` is produced by the [switch](#switchgo), +and handled by the switch and the [PEX reactor](#pex_reactorgo). + +### `fuzz.go` + +For testing purposes. + +`FuzzedConnection` wraps a `net.Conn` and injects random delays. + +### `key.go` + +`NodeKey` is the persistent key of a node, namely its private key. + +The `ID` of a node is a string representing the node's public key. + +### `metrics.go` + +Prometheus `Metrics` exposed by the p2p layer. + +### `netaddress.go` + +Type `NetAddress` contains the `ID` and the network address (IP and port) of a node. + +The API of the [address book](#addrbookgo) receives and returns `NetAddress` instances. + +This source file was adapted from [`btcd`](https://github.com/btcsuite/btcd), +a Go implementation of Bitcoin. + +### `node_info.go` + +Interface `NodeInfo` stores the basic information about a node exchanged with a +peer during the handshake. + +It is implemented by `DefaultNodeInfo` type. + +The [switch](#switchgo) stores the local `NodeInfo`. + +The `NodeInfo` of connected peers is produced by the +[transport](#transportgo) during the handshake, and stored in [`Peer`](#peergo) instances. + +### `peer.go` + +Interface `Peer` represents a connected peer. + +It is implemented by the internal `peer` type. + +The [transport](#transportgo) API methods return `Peer` instances, +wrapping established secure connection with peers. + +The [switch](#switchgo) API methods receive `Peer` instances. +The switch stores connected peers in a `PeerSet`. + +The [`Reactor`](#base_reactorgo) methods, invoked by the switch, receive `Peer` instances. + +### `peer_set.go` + +Interface `IPeerSet` offers methods to access a table of [`Peer`](#peergo) instances. + +Type `PeerSet` implements a thread-safe table of [`Peer`](#peergo) instances, +used by the [switch](#switchgo). + +The switch provides limited access to this table by returing a `IPeerSet` +instance, used by the [PEX reactor](#pex_reactorgo). + +### `switch.go` + +Documented in [switch](./switch.md). + +The `Switch` implements the [peer manager](./peer_manager.md) role for inbound peers. + +[`Reactor`](#base_reactorgo)s have access to the `Switch` and may invoke its methods. +This includes the [PEX reactor](#pex_reactorgo). + +### `transport.go` + +Documented in [transport](./transport.md). + +The `Transport` interface is implemented by `MultiplexTransport`. + +The [switch](#switchgo) contains a `Transport` and uses it to establish +connections with peers. + +### `types.go` + +Aliases for p2p's `conn` package types. + +## Package `p2p.conn` + +Implements the connection between CometBFT nodes, +which is encrypted, authenticated, and multiplexed. + +### `connection.go` + +Implements the `MConnection` type and the `Channel` abstraction. + +A `MConnection` multiplexes a generic network connection (`net.Conn`) into +multiple independent `Channel`s, used by different [`Reactor`](#base_reactorgo)s. + +A [`Peer`](#peergo) stores the `MConnection` instance used to interact with a +peer, which multiplex a [`SecretConnection`](#secret_connectiongo). + +### `conn_go110.go` + +Support for go 1.10. + +### `secret_connection.go` + +Implements the `SecretConnection` type, which is an encrypted authenticated +connection built atop a raw network (TCP) connection. + +A [`Peer`](#peergo) stores the `SecretConnection` established by the transport, +which is the underlying connection multiplexed by [`MConnection`](#connectiongo). + +As briefly documented in the [transport](./transport.md#Connection-Upgrade), +a `SecretConnection` implements the Station-To-Station (STS) protocol. + +The `SecretConnection` type implements the `net.Conn` interface, +which is a generic network connection. + +## Package `p2p.mock` + +Mock implementations of [`Peer`](#peergo) and [`Reactor`](#base_reactorgo) interfaces. + +## Package `p2p.mocks` + +Code generated by `mockery`. + +## Package `p2p.pex` + +Implementation of the [PEX reactor](./pex.md). + +### `addrbook.go` + +Documented in [address book](./addressbook.md). + +This source file was adapted from [`btcd`](https://github.com/btcsuite/btcd), +a Go implementation of Bitcoin. + +### `errors.go` + +A number of errors produced and handled by the [address book](#addrbookgo). + +`ErrAddrBookNilAddr` is produced by the address book, but handled (logged) by +the [PEX reactor](#pex_reactorgo). + +`ErrUnsolicitedList` is produced and handled by the [PEX protocol](#pex_reactorgo). + +### `file.go` + +Implements the [address book](#addrbookgo) persistence. + +### `known_address.go` + +Type `knownAddress` represents an address stored in the [address book](#addrbookgo). + +### `params.go` + +Constants used by the [address book](#addrbookgo). + +### `pex_reactor.go` + +Implementation of the [PEX reactor](./pex.md), which is a [`Reactor`](#base_reactorgo). + +This includes the implementation of the [PEX protocol](./pex-protocol.md) +and of the [peer manager](./peer_manager.md) role for outbound peers. + +The PEX reactor also manages an [address book](#addrbookgo) instance. + +## Package `p2p.trust` + +Go documentation of `Metric` type: + +> // Metric - keeps track of peer reliability +> // See cometbft/docs/architecture/adr-006-trust-metric.md for details + +Not imported by any other CometBFT source file. diff --git a/cometbft/v0.39/spec/p2p/legacy-docs/Overview.mdx b/cometbft/v0.39/spec/p2p/legacy-docs/Overview.mdx new file mode 100644 index 000000000..355e398ec --- /dev/null +++ b/cometbft/v0.39/spec/p2p/legacy-docs/Overview.mdx @@ -0,0 +1,16 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/spec/p2p/legacy-docs/Overview' +order: 1 +title: Legacy Docs +--- + +This section contains useful information. However, part of this content is redundant, being more comprehensively covered +in more recent documents, and some implementation details might be outdated +(see issue [#981](https://github.com/cometbft/cometbft/issues/981)). + +- [Messages](/cometbft/v0.39/spec/p2p/legacy-docs/messages/Overview) +- [P2P Config](/cometbft/v0.39/spec/p2p/legacy-docs/P2P-Config) +- [P2P Multiplex Connection](/cometbft/v0.39/spec/p2p/legacy-docs/P2P-Multiplex-Connection) +- [Peer Discovery](/cometbft/v0.39/spec/p2p/legacy-docs/Peer-Discovery) +- [Peers](/cometbft/v0.39/spec/p2p/legacy-docs/Peers) diff --git a/cometbft/v0.39/spec/p2p/legacy-docs/P2P-Config.mdx b/cometbft/v0.39/spec/p2p/legacy-docs/P2P-Config.mdx new file mode 100644 index 000000000..eab03e288 --- /dev/null +++ b/cometbft/v0.39/spec/p2p/legacy-docs/P2P-Config.mdx @@ -0,0 +1,54 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/spec/p2p/legacy-docs/P2P-Config' +title: P2P Config +order: 1 +--- + +Here we describe configuration options around the Peer Exchange. +These can be set using flags or via the `$CMTHOME/config/config.toml` file. + +## Seed Mode + +`--p2p.seed_mode` + +The node operates in seed mode. In seed mode, a node continuously crawls the network for peers, +and upon incoming connection shares some peers and disconnects. + +## Seeds + +`--p2p.seeds “id100000000000000000000000000000000@1.2.3.4:26656,id200000000000000000000000000000000@2.3.4.5:4444”` + +Dials these seeds when we need more peers. They should return a list of peers and then disconnect. +If we already have enough peers in the address book, we may never need to dial them. + +## Persistent Peers + +`--p2p.persistent_peers “id100000000000000000000000000000000@1.2.3.4:26656,id200000000000000000000000000000000@2.3.4.5:26656”` + +Dial these peers and auto-redial them if the connection fails. +These are intended to be trusted persistent peers that can help +anchor us in the p2p network. The auto-redial uses exponential +backoff and will give up after a day of trying to connect. + +But If `persistent_peers_max_dial_period` is set greater than zero, +pause between each dial to each persistent peer will not exceed `persistent_peers_max_dial_period` +during exponential backoff and we keep trying again without giving up + +**Note:** If `seeds` and `persistent_peers` intersect, +the user will be warned that seeds may auto-close connections +and that the node may not be able to keep the connection persistent. + +## Private Peers + +`--p2p.private_peer_ids “id100000000000000000000000000000000,id200000000000000000000000000000000”` + +These are IDs of the peers that we do not add to the address book or gossip to +other peers. They stay private to us. + +## Unconditional Peers + +`--p2p.unconditional_peer_ids “id100000000000000000000000000000000,id200000000000000000000000000000000”` + +These are IDs of the peers which are allowed to be connected by both inbound or outbound regardless of +`max_num_inbound_peers` or `max_num_outbound_peers` of user's node reached or not. diff --git a/cometbft/v0.39/spec/p2p/legacy-docs/P2P-Multiplex-Connection.mdx b/cometbft/v0.39/spec/p2p/legacy-docs/P2P-Multiplex-Connection.mdx new file mode 100644 index 000000000..fd8b52115 --- /dev/null +++ b/cometbft/v0.39/spec/p2p/legacy-docs/P2P-Multiplex-Connection.mdx @@ -0,0 +1,116 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/spec/p2p/legacy-docs/P2P-Multiplex-Connection' +title: P2P Multiplex Connection +order: 1 +--- + +## MConnection + +`MConnection` is a multiplex connection that supports multiple independent streams +with distinct quality of service guarantees atop a single TCP connection. +Each stream is known as a `Channel` and each `Channel` has a globally unique _byte id_. +Each `Channel` also has a relative priority that determines the quality of service +of the `Channel` compared to other `Channel`s. +The _byte id_ and the relative priorities of each `Channel` are configured upon +initialization of the connection. + +The `MConnection` supports three packet types: + +- Ping +- Pong +- Msg + +### Ping and Pong + +The ping and pong messages consist of writing a single byte to the connection; 0x1 and 0x2, respectively. + +When we haven't received any messages on an `MConnection` in time `pingTimeout`, we send a ping message. +When a ping is received on the `MConnection`, a pong is sent in response only if there are no other messages +to send and the peer has not sent us too many pings (TODO). + +If a pong or message is not received in sufficient time after a ping, the peer is disconnected from. + +### Msg + +Messages in channels are chopped into smaller `msgPacket`s for multiplexing. + +```go +type msgPacket struct { + ChannelID byte + EOF byte // 1 means message ends here. + Bytes []byte +} +``` + +The `msgPacket` is serialized using [Proto3](https://developers.google.com/protocol-buffers/docs/proto3). +The received `Bytes` of a sequential set of packets are appended together +until a packet with `EOF=1` is received, then the complete serialized message +is returned for processing by the `onReceive` function of the corresponding channel. + +### Multiplexing + +Messages are sent from a single `sendRoutine`, which loops over a select statement and results in the sending +of a ping, a pong, or a batch of data messages. The batch of data messages may include messages from multiple channels. +Message bytes are queued for sending in their respective channel, with each channel holding one unsent message at a time. +Messages are chosen for a batch one at a time from the channel with the lowest ratio of recently sent bytes to channel priority. + +## Sending Messages + +There are two methods for sending messages: + +```go +func (m MConnection) Send(chID byte, msg interface{}) bool {} +func (m MConnection) TrySend(chID byte, msg interface{}) bool {} +``` + +`Send(chID, msg)` is a blocking call that waits until `msg` is successfully queued +for the channel with the given id byte `chID`. The message `msg` is serialized +using protobuf marshalling. + +`TrySend(chID, msg)` is a nonblocking call that queues the message msg in the channel +with the given id byte chID if the queue is not full; otherwise it returns false immediately. + +`Send()` and `TrySend()` are also exposed for each `Peer`. + +## Peer + +Each peer has one `MConnection` instance, and includes other information such as whether the connection +was outbound, whether the connection should be recreated if it closes, various identity information about the node, +and other higher level thread-safe data used by the reactors. + +## Switch/Reactor + +The `Switch` handles peer connections and exposes an API to receive incoming messages +on `Reactors`. Each `Reactor` is responsible for handling incoming messages of one +or more `Channels`. So while sending outgoing messages is typically performed on the peer, +incoming messages are received on the reactor. + +```go +// Declare a MyReactor reactor that handles messages on MyChannelID. +type MyReactor struct{} + +func (reactor MyReactor) GetChannels() []*ChannelDescriptor { + return []*ChannelDescriptor{ChannelDescriptor{ID:MyChannelID, Priority: 1}} +} + +func (reactor MyReactor) Receive(chID byte, peer *Peer, msgBytes []byte) { + r, n, err := bytes.NewBuffer(msgBytes), new(int64), new(error) + msgString := ReadString(r, n, err) + fmt.Println(msgString) +} + +// Other Reactor methods omitted for brevity +... + +switch := NewSwitch([]Reactor{MyReactor{}}) + +... + +// Send a random message to all outbound connections +for _, peer := range switch.Peers().List() { + if peer.IsOutbound() { + peer.Send(MyChannelID, "Here's a random message") + } +} +``` diff --git a/cometbft/v0.39/spec/p2p/legacy-docs/Peer-Discovery.mdx b/cometbft/v0.39/spec/p2p/legacy-docs/Peer-Discovery.mdx new file mode 100644 index 000000000..05ddf1f74 --- /dev/null +++ b/cometbft/v0.39/spec/p2p/legacy-docs/Peer-Discovery.mdx @@ -0,0 +1,70 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/spec/p2p/legacy-docs/Peer-Discovery' +title: Peer Discovery +order: 1 +--- + +A CometBFT P2P network has different kinds of nodes with different requirements for connectivity to one another. +This document describes what kind of nodes CometBFT should enable and how they should work. + +## Seeds + +Seeds are the first point of contact for a new node. +They return a list of known active peers and then disconnect. + +Seeds should operate full nodes with the PEX reactor in a "crawler" mode +that continuously explores to validate the availability of peers. + +Seeds should only respond with some top percentile of the best peers it knows about. + +## New Full Node + +A new node needs a few things to connect to the network: + +- a list of seeds, which can be provided to CometBFT via config file or flags, + or hardcoded into the software by in-process apps +- a `ChainID`, also called `Network` at the p2p layer +- a recent block height, H, and hash, HASH for the blockchain. + +The values `H` and `HASH` must be received and corroborated by means external to CometBFT, and specific to the user - ie. via the user's trusted social consensus. +This requirement to validate `H` and `HASH` out-of-band and via social consensus +is the essential difference in security models between Proof-of-Work and Proof-of-Stake blockchains. + +With the above, the node then queries some seeds for peers for its chain, +dials those peers, and runs the CometBFT protocols with those it successfully connects to. + +When the peer catches up to height H, it ensures the block hash matches HASH. +If not, CometBFT will exit, and the user must try again - either they are connected +to bad peers or their social consensus is invalid. + +## Restarted Full Node + +A node checks its address book on startup and attempts to connect to peers from there. +If it can't connect to any peers after some time, it falls back to the seeds to find more. + +Restarted full nodes can run the `blockchain` or `consensus` reactor protocols to sync up +to the latest state of the blockchain from wherever they were last. +In a Proof-of-Stake context, if they are sufficiently far behind (greater than the length +of the unbonding period), they will need to validate a recent `H` and `HASH` out-of-band again +so they know they have synced the correct chain. + +## Validator Node + +A validator node is a node that interfaces with a validator signing key. +These nodes require the highest security, and should not accept incoming connections. +They should maintain outgoing connections to a controlled set of "Sentry Nodes" that serve +as their proxy shield to the rest of the network. + +Validators that know and trust each other can accept incoming connections from one another and maintain direct private connectivity via VPN. + +## Sentry Node + +Sentry nodes are guardians of a validator node and provide it access to the rest of the network. +They should be well connected to other full nodes on the network. +Sentry nodes may be dynamic, but should maintain persistent connections to some evolving random subset of each other. +They should always expect to have direct incoming connections from the validator node and its backup(s). +They do not report the validator node's address in the PEX and +they may be more strict about the quality of peers they keep. + +Sentry nodes belonging to validators that trust each other may wish to maintain persistent connections via VPN with one another, but only report each other sparingly in the PEX. diff --git a/cometbft/v0.39/spec/p2p/legacy-docs/Peers.mdx b/cometbft/v0.39/spec/p2p/legacy-docs/Peers.mdx new file mode 100644 index 000000000..2ffd6b4b5 --- /dev/null +++ b/cometbft/v0.39/spec/p2p/legacy-docs/Peers.mdx @@ -0,0 +1,133 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/spec/p2p/legacy-docs/Peers' +title: Peers +order: 1 +--- + +This document explains how CometBFT Peers are identified and how they connect to one another. + +## Peer Identity + +CometBFT peers are expected to maintain long-term persistent identities in the form of a public key. +Each peer has an ID defined as `peer.ID == peer.PubKey.Address()`, where `Address` uses the scheme defined in `crypto` package. + +A single peer ID can have multiple IP addresses associated with it, but a node +will only ever connect to one at a time. + +When attempting to connect to a peer, we use the PeerURL: `@:`. +We will attempt to connect to the peer at IP:PORT, and verify, +via authenticated encryption, that it is in possession of the private key +corresponding to ``. This prevents man-in-the-middle attacks on the peer layer. + +## Connections + +All p2p connections use TCP. +Upon establishing a successful TCP connection with a peer, +two handshakes are performed: one for authenticated encryption, and one for CometBFT versioning. +Both handshakes have configurable timeouts (they should complete quickly). + +### Authenticated Encryption Handshake + +CometBFT implements the Station-to-Station protocol +using X25519 keys for Diffie-Helman key-exchange and chacha20poly1305 for encryption. + +Previous versions of this protocol (0.32 and below) suffered from malleability attacks whereas an active man +in the middle attacker could compromise confidentiality as described in [Prime, Order Please! +Revisiting Small Subgroup and Invalid Curve Attacks on +Protocols using Diffie-Hellman](https://eprint.iacr.org/2019/526.pdf). + +We have added dependency on the Merlin a keccak based transcript hashing protocol to ensure non-malleability. + +It goes as follows: + +- generate an ephemeral X25519 keypair +- send the ephemeral public key to the peer +- wait to receive the peer's ephemeral public key +- create a new Merlin Transcript with the string "TENDERMINT_SECRET_CONNECTION_TRANSCRIPT_HASH" +- Sort the ephemeral keys and add the high labeled "EPHEMERAL_UPPER_PUBLIC_KEY" and the low keys labeled "EPHEMERAL_LOWER_PUBLIC_KEY" to the Merlin transcript. +- compute the Diffie-Hellman shared secret using the peers ephemeral public key and our ephemeral private key +- add the DH secret to the transcript labeled DH_SECRET. +- generate two keys to use for encryption (sending and receiving) and a challenge for authentication as follows: + - create a hkdf-sha256 instance with the key being the diffie hellman shared secret, and info parameter as + `TENDERMINT_SECRET_CONNECTION_KEY_AND_CHALLENGE_GEN` + - get 64 bytes of output from hkdf-sha256 + - if we had the smaller ephemeral pubkey, use the first 32 bytes for the key for receiving, the second 32 bytes for sending; else the opposite. +- use a separate nonce for receiving and sending. Both nonces start at 0, and should support the full 96 bit nonce range +- all communications from now on are encrypted in 1400 byte frames (plus encoding overhead), + using the respective secret and nonce. Each nonce is incremented by one after each use. +- we now have an encrypted channel, but still need to authenticate +- extract a 32 bytes challenge from merlin transcript with the label "SECRET_CONNECTION_MAC" +- sign the common challenge obtained from the hkdf with our persistent private key +- send the amino encoded persistent pubkey and signature to the peer +- wait to receive the persistent public key and signature from the peer +- verify the signature on the challenge using the peer's persistent public key + +If this is an outgoing connection (we dialed the peer) and we used a peer ID, +then finally verify that the peer's persistent public key corresponds to the peer ID we dialed, +ie. `peer.PubKey.Address() == `. + +The connection has now been authenticated. All traffic is encrypted. + +Note: only the dialer can authenticate the identity of the peer, +but this is what we care about since when we join the network we wish to +ensure we have reached the intended peer (and are not being MITMd). + +### Peer Filter + +Before continuing, we check if the new peer has the same ID as ourselves or +an existing peer. If so, we disconnect. + +We also check the peer's address and public key against +an optional whitelist which can be managed through the ABCI app - +if the whitelist is enabled and the peer does not qualify, the connection is +terminated. + +### CometBFT Version Handshake + +The CometBFT Version Handshake allows the peers to exchange their NodeInfo: + +```golang +type NodeInfo struct { + Version p2p.Version + ID p2p.ID + ListenAddr string + + Network string + SoftwareVersion string + Channels []int8 + + Moniker string + Other NodeInfoOther +} + +type Version struct { + P2P uint64 + Block uint64 + App uint64 +} + +type NodeInfoOther struct { + TxIndex string + RPCAddress string +} +``` + +The connection is disconnected if: + +- `peer.NodeInfo.ID` is not equal `peerConn.ID` +- `peer.NodeInfo.Version.Block` does not match ours +- `peer.NodeInfo.Network` is not the same as ours +- `peer.Channels` does not intersect with our known Channels. +- `peer.NodeInfo.ListenAddr` is malformed or is a DNS host that cannot be + resolved + +At this point, if we have not disconnected, the peer is valid. +It is added to the switch and hence all reactors via the `AddPeer` method. +Note that each reactor may handle multiple channels. + +## Connection Activity + +Once a peer is added, incoming messages for a given reactor are handled through +that reactor's `Receive` method, and output messages are sent directly by the Reactors +on each peer. A typical reactor maintains per-peer go-routine(s) that handle this. diff --git a/cometbft/v0.39/spec/p2p/legacy-docs/messages/Overview.mdx b/cometbft/v0.39/spec/p2p/legacy-docs/messages/Overview.mdx new file mode 100644 index 000000000..a7f4fa36a --- /dev/null +++ b/cometbft/v0.39/spec/p2p/legacy-docs/messages/Overview.mdx @@ -0,0 +1,21 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/spec/p2p/legacy-docs/messages/Overview' +order: 1 +parent: + title: Messages + order: 1 +--- + +# Messages + +An implementation of the spec consists of many components. While many parts of these components are implementation specific, the p2p messages are not. In this section we will be covering all the p2p messages of components. + +There are two parts to the P2P messages, the message and the channel. The channel is message specific and messages are specific to components of CometBFT. When a node connect to a peer it will tell the other node which channels are available. This notifies the peer what services the connecting node offers. You can read more on channels in [connection.md](/cometbft/v0.39/spec/p2p/legacy-docs/P2P-Multiplex-Connection) + +- [Block Sync](/cometbft/v0.39/spec/p2p/legacy-docs/messages/block-sync) +- [Mempool](/cometbft/v0.39/spec/p2p/legacy-docs/messages/mempool) +- [Evidence](/cometbft/v0.39/spec/p2p/legacy-docs/messages/evidence) +- [State Sync](/cometbft/v0.39/spec/p2p/legacy-docs/messages/state-sync) +- [Pex](/cometbft/v0.39/spec/p2p/legacy-docs/messages/Peer-Exchange) +- [Consensus](/cometbft/v0.39/spec/p2p/legacy-docs/messages/consensus) diff --git a/cometbft/v0.39/spec/p2p/legacy-docs/messages/Peer-Exchange.mdx b/cometbft/v0.39/spec/p2p/legacy-docs/messages/Peer-Exchange.mdx new file mode 100644 index 000000000..aba68e6df --- /dev/null +++ b/cometbft/v0.39/spec/p2p/legacy-docs/messages/Peer-Exchange.mdx @@ -0,0 +1,78 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/spec/p2p/legacy-docs/messages/Peer-Exchange' +title: Peer Exchange +order: 6 +--- +{/* trigger rebuild */} + +## Channels + +Pex has one channel. The channel identifier is listed below. + +| Name | Number | +|------------|--------| +| PexChannel | 0 | + +## Message Types + +The current PEX service has two versions. The first uses IP/port pair but since the p2p stack is moving towards a transport agnostic approach, +node endpoints require a `Protocol` and `Path` hence the V2 version uses a [url](https://golang.org/pkg/net/url/#URL) instead. + +### PexRequest + +PexRequest is an empty message requesting a list of peers. + +> EmptyRequest + +### PexResponse + +PexResponse is an list of net addresses provided to a peer to dial. + +| Name | Type | Description | Field Number | +|-------|------------------------------------|------------------------------------------|--------------| +| addresses | repeated [PexAddress](#pexaddress) | List of peer addresses available to dial | 1 | + +### PexAddress + +PexAddress provides needed information for a node to dial a peer. + +| Name | Type | Description | Field Number | +|------|--------|------------------|--------------| +| id | string | NodeID of a peer | 1 | +| ip | string | The IP of a node | 2 | +| port | port | Port of a peer | 3 | + + +### PexRequestV2 + +PexRequest is an empty message requesting a list of peers. + +> EmptyRequest + +### PexResponseV2 + +PexResponse is an list of net addresses provided to a peer to dial. + +| Name | Type | Description | Field Number | +|-------|------------------------------------|------------------------------------------|--------------| +| addresses | repeated [PexAddressV2](#pexresponsev2) | List of peer addresses available to dial | 1 | + +### PexAddressV2 + +PexAddress provides needed information for a node to dial a peer. + +| Name | Type | Description | Field Number | +|------|--------|------------------|--------------| +| url | string | See [golang url](https://golang.org/pkg/net/url/#URL) | 1 | + +### Message + +Message is a [`oneof` protobuf type](https://developers.google.com/protocol-buffers/docs/proto#oneof). The one of consists of two messages. + +| Name | Type | Description | Field Number | +|--------------|---------------------------|------------------------------------------------------|--------------| +| pex_request | [PexRequest](#pexrequest) | Empty request asking for a list of addresses to dial | 1 | +| pex_response | [PexResponse](#pexresponse)| List of addresses to dial | 2 | +| pex_request_v2| [PexRequestV2](#pexrequestv2)| Empty request asking for a list of addresses to dial| 3 | +| pex_response_v2| [PexRespinseV2](#pexresponsev2)| List of addresses to dial | 4 | diff --git a/cometbft/v0.39/spec/p2p/legacy-docs/messages/block-sync.mdx b/cometbft/v0.39/spec/p2p/legacy-docs/messages/block-sync.mdx new file mode 100644 index 000000000..05b8fb8c7 --- /dev/null +++ b/cometbft/v0.39/spec/p2p/legacy-docs/messages/block-sync.mdx @@ -0,0 +1,72 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/spec/p2p/legacy-docs/messages/block-sync' +title: Block Sync +order: 2 +--- +{/* trigger rebuild */} + +## Channel + +Block sync has one channel. + +| Name | Number | +|-------------------|--------| +| BlocksyncChannel | 64 | + +## Message Types + +There are multiple message types for Block Sync + +### BlockRequest + +BlockRequest asks a peer for a block at the height specified. + +| Name | Type | Description | Field Number | +|--------|-------|---------------------------|--------------| +| Height | int64 | Height of requested block | 1 | + +### NoBlockResponse + +NoBlockResponse notifies the peer requesting a block that the node does not contain it. + +| Name | Type | Description | Field Number | +|--------|-------|---------------------------|--------------| +| Height | int64 | Height of requested block | 1 | + +### BlockResponse + +BlockResponse contains the block requested. +It also contains an extended commit _iff_ vote extensions are enabled at the block's height. + +| Name | Type | Description | Field Number | +|-----------|----------------------------------------------------------------|---------------------------------|--------------| +| Block | [Block](/cometbft/v0.39/spec/core/Data_structures#block) | Requested Block | 1 | +| ExtCommit | [ExtendedCommit](/cometbft/v0.39/spec/core/Data_structures#extendedcommit) | Sender's LastCommit information | 2 | + +### StatusRequest + +StatusRequest is an empty message that notifies the peer to respond with the highest and lowest blocks it has stored. + +> Empty message. + +### StatusResponse + +StatusResponse responds to a peer with the highest and lowest heights of any block it has in its blockstore. + +| Name | Type | Description | Field Number | +|--------|-------|-------------------------------------------------------------------|--------------| +| Height | int64 | Current Height of a node | 1 | +| Base | int64 | First known block, if pruning is enabled it will be higher than 1 | 2 | + +### Message + +Message is a [`oneof` protobuf type](https://developers.google.com/protocol-buffers/docs/proto#oneof). The `oneof` consists of five messages. + +| Name | Type | Description | Field Number | +|-------------------|-------------------------------------|--------------------------------------------------------------|--------------| +| block_request | [BlockRequest](#blockrequest) | Request a block from a peer | 1 | +| no_block_response | [NoBlockResponse](#noblockresponse) | Response saying it doe snot have the requested block | 2 | +| block_response | [BlockResponse](#blockresponse) | Response with requested block + (optionally) vote extensions | 3 | +| status_request | [StatusRequest](#statusrequest) | Request the highest and lowest block numbers from a peer | 4 | +| status_response | [StatusResponse](#statusresponse) | Response with the highest and lowest block numbers the store | 5 | diff --git a/cometbft/v0.39/spec/p2p/legacy-docs/messages/consensus.mdx b/cometbft/v0.39/spec/p2p/legacy-docs/messages/consensus.mdx new file mode 100644 index 000000000..8594ab9a5 --- /dev/null +++ b/cometbft/v0.39/spec/p2p/legacy-docs/messages/consensus.mdx @@ -0,0 +1,148 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/spec/p2p/legacy-docs/messages/consensus' +title: Consensus +order: 7 +--- +{/* trigger rebuild */} + +## Channel + +Consensus has four separate channels. The channel identifiers are listed below. + +| Name | Number | +|--------------------|--------| +| StateChannel | 32 | +| DataChannel | 33 | +| VoteChannel | 34 | +| VoteSetBitsChannel | 35 | + +## Message Types + +### Proposal + +Proposal is sent when a new block is proposed. It is a suggestion of what the +next block in the blockchain should be. + +| Name | Type | Description | Field Number | +|----------|----------------------------------------------------|----------------------------------------|--------------| +| proposal | [Proposal](/cometbft/v0.39/spec/core/Data_structures#proposal) | Proposed Block to come to consensus on | 1 | + +### Vote + +Vote is sent to vote for some block (or to inform others that a process does not vote in the +current round). Vote contains validator's information (validator address and index), height and +round for which the vote is sent, vote type, blockID if process vote for some block (`nil` otherwise) +and a timestamp when the vote is sent. The message is signed by the validator private key. + +| Name | Type | Description | Field Number | +|------|--------------------------------------------|---------------------------|--------------| +| vote | [Vote](/cometbft/v0.39/spec/core/Data_structures#vote) | Vote for a proposed Block | 1 | + +### BlockPart + +BlockPart is sent when gossiping a piece of the proposed block. It contains height, round +and the block part. + +| Name | Type | Description | Field Number | +|--------|--------------------------------------------|----------------------------------------|--------------| +| height | int64 | Height of corresponding block. | 1 | +| round | int32 | Round of voting to finalize the block. | 2 | +| part | [Part](/cometbft/v0.39/spec/core/Data_structures#part) | A part of the block. | 3 | + +### NewRoundStep + +NewRoundStep is sent for every step transition during the core consensus algorithm execution. +It is used in the gossip part of the CometBFT consensus protocol to inform peers about a current +height/round/step a process is in. + +| Name | Type | Description | Field Number | +|--------------------------|--------|----------------------------------------|--------------| +| height | int64 | Height of corresponding block | 1 | +| round | int32 | Round of voting to finalize the block. | 2 | +| step | uint32 | | 3 | +| seconds_since_start_time | int64 | | 4 | +| last_commit_round | int32 | | 5 | + +### NewValidBlock + +NewValidBlock is sent when a validator observes a valid block B in some round r, +i.e., there is a Proposal for block B and 2/3+ prevotes for the block B in the round r. +It contains height and round in which valid block is observed, block parts header that describes +the valid block and is used to obtain all +block parts, and a bit array of the block parts a process currently has, so its peers can know what +parts it is missing so they can send them. +In case the block is also committed, then IsCommit flag is set to true. + +| Name | Type | Description | Field Number | +|-----------------------|--------------------------------------------------------------|----------------------------------------|--------------| +| height | int64 | Height of corresponding block | 1 | +| round | int32 | Round of voting to finalize the block. | 2 | +| block_part_set_header | [PartSetHeader](/cometbft/v0.39/spec/core/Data_structures#partsetheader) | | 3 | +| block_parts | int32 | | 4 | +| is_commit | bool | | 5 | + +### ProposalPOL + +ProposalPOL is sent when a previous block is re-proposed. +It is used to inform peers in what round the process learned for this block (ProposalPOLRound), +and what prevotes for the re-proposed block the process has. + +| Name | Type | Description | Field Number | +|--------------------|----------|-------------------------------|--------------| +| height | int64 | Height of corresponding block | 1 | +| proposal_pol_round | int32 | | 2 | +| proposal_pol | bitarray | | 3 | + +### ReceivedVote + +ReceivedVote is sent to indicate that a particular vote has been received. It contains height, +round, vote type and the index of the validator that is the originator of the corresponding vote. + +| Name | Type | Description | Field Number | +|--------|------------------------------------------------------------------|----------------------------------------|--------------| +| height | int64 | Height of corresponding block | 1 | +| round | int32 | Round of voting to finalize the block. | 2 | +| type | [SignedMessageType](/cometbft/v0.39/spec/core/Data_structures#signedmsgtype) | | 3 | +| index | int32 | | 4 | + +### VoteSetMaj23 + +VoteSetMaj23 is sent to indicate that a process has seen +2/3 votes for some BlockID. +It contains height, round, vote type and the BlockID. + +| Name | Type | Description | Field Number | +|--------|------------------------------------------------------------------|----------------------------------------|--------------| +| height | int64 | Height of corresponding block | 1 | +| round | int32 | Round of voting to finalize the block. | 2 | +| type | [SignedMessageType](/cometbft/v0.39/spec/core/Data_structures#signedmsgtype) | | 3 | + +### VoteSetBits + +VoteSetBits is sent to communicate the bit-array of votes a process has seen for a given +BlockID. It contains height, round, vote type, BlockID and a bit array of +the votes a process has. + +| Name | Type | Description | Field Number | +|----------|------------------------------------------------------------------|----------------------------------------|--------------| +| height | int64 | Height of corresponding block | 1 | +| round | int32 | Round of voting to finalize the block. | 2 | +| type | [SignedMessageType](/cometbft/v0.39/spec/core/Data_structures#signedmsgtype) | | 3 | +| block_id | [BlockID](/cometbft/v0.39/spec/core/Data_structures#blockid) | | 4 | +| votes | BitArray | Round of voting to finalize the block. | 5 | + +### Message + +Message is a [`oneof` protobuf type](https://developers.google.com/protocol-buffers/docs/proto#oneof). + +| Name | Type | Description | Field Number | +|-----------------|---------------------------------|----------------------------------------|--------------| +| new_round_step | [NewRoundStep](#newroundstep) | Height of corresponding block | 1 | +| new_valid_block | [NewValidBlock](#newvalidblock) | Round of voting to finalize the block. | 2 | +| proposal | [Proposal](#proposal) | | 3 | +| proposal_pol | [ProposalPOL](#proposalpol) | | 4 | +| block_part | [BlockPart](#blockpart) | | 5 | +| vote | [Vote](#vote) | | 6 | +| received_vote | [ReceivedVote](#receivedvote) | | 7 | +| vote_set_maj23 | [VoteSetMaj23](#votesetmaj23) | | 8 | +| vote_set_bits | [VoteSetBits](#votesetbits) | | 9 | diff --git a/cometbft/v0.39/spec/p2p/legacy-docs/messages/evidence.mdx b/cometbft/v0.39/spec/p2p/legacy-docs/messages/evidence.mdx new file mode 100644 index 000000000..1b717aae9 --- /dev/null +++ b/cometbft/v0.39/spec/p2p/legacy-docs/messages/evidence.mdx @@ -0,0 +1,25 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/spec/p2p/legacy-docs/messages/evidence' +title: Evidence +order: 3 +--- +{/* trigger rebuild */} + +## Channel + +Evidence has one channel. The channel identifier is listed below. + +| Name | Number | +|-----------------|--------| +| EvidenceChannel | 56 | + +## Message Types + +### EvidenceList + +EvidenceList consists of a list of verified evidence. This evidence will already have been propagated throughout the network. EvidenceList is used in two places, as a p2p message and within the block [block](/cometbft/v0.39/spec/core/Data_structures#block) as well. + +| Name | Type | Description | Field Number | +|----------|-------------------------------------------------------------|------------------------|--------------| +| evidence | repeated [Evidence](/cometbft/v0.39/spec/core/Data_structures#evidence) | List of valid evidence | 1 | diff --git a/cometbft/v0.39/spec/p2p/legacy-docs/messages/mempool.mdx b/cometbft/v0.39/spec/p2p/legacy-docs/messages/mempool.mdx new file mode 100644 index 000000000..724d7bb31 --- /dev/null +++ b/cometbft/v0.39/spec/p2p/legacy-docs/messages/mempool.mdx @@ -0,0 +1,36 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/spec/p2p/legacy-docs/messages/mempool' +title: Mempool +order: 4 +--- +{/* trigger rebuild */} + +## Channel + +Mempool has one channel. The channel identifier is listed below. + +| Name | Number | +|----------------|--------| +| MempoolChannel | 48 | + +## Message Types + +There is currently only one message that Mempool broadcasts and receives over +the p2p gossip network (via the reactor): `TxsMessage` + +### Txs + +A list of transactions. These transactions have been checked against the application for validity. This does not mean that the transactions are valid, it is up to the application to check this. + +| Name | Type | Description | Field Number | +|------|----------------|----------------------|--------------| +| txs | repeated bytes | List of transactions | 1 | + +### Message + +Message is a [`oneof` protobuf type](https://developers.google.com/protocol-buffers/docs/proto#oneof). The one of consists of one message [`Txs`](#txs). + +| Name | Type | Description | Field Number | +|------|-------------|-----------------------|--------------| +| txs | [Txs](#txs) | List of transactions | 1 | diff --git a/cometbft/v0.39/spec/p2p/legacy-docs/messages/state-sync.mdx b/cometbft/v0.39/spec/p2p/legacy-docs/messages/state-sync.mdx new file mode 100644 index 000000000..c290cd597 --- /dev/null +++ b/cometbft/v0.39/spec/p2p/legacy-docs/messages/state-sync.mdx @@ -0,0 +1,134 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/spec/p2p/legacy-docs/messages/state-sync' +title: State Sync +order: 5 +--- +{/* trigger rebuild */} + +## Channels + +State sync has four distinct channels. The channel identifiers are listed below. + +| Name | Number | +|-------------------|--------| +| SnapshotChannel | 96 | +| ChunkChannel | 97 | +| LightBlockChannel | 98 | +| ParamsChannel | 99 | + +## Message Types + +### SnapshotRequest + +When a new node begin state syncing, it will ask all peers it encounters if it has any +available snapshots: + +| Name | Type | Description | Field Number | +|----------|--------|-------------|--------------| + +### SnapShotResponse + +The receiver will query the local ABCI application via `ListSnapshots`, and send a message +containing snapshot metadata (limited to 4 MB) for each of the 10 most recent snapshots: and stored at the application layer. When a peer is starting it will request snapshots. + +| Name | Type | Description | Field Number | +|----------|--------|-----------------------------------------------------------|--------------| +| height | uint64 | Height at which the snapshot was taken | 1 | +| format | uint32 | Format of the snapshot. | 2 | +| chunks | uint32 | How many chunks make up the snapshot | 3 | +| hash | bytes | Arbitrary snapshot hash | 4 | +| metadata | bytes | Arbitrary application data. **May be non-deterministic.** | 5 | + +### ChunkRequest + +The node running state sync will offer these snapshots to the local ABCI application via +`OfferSnapshot` ABCI calls, and keep track of which peers contain which snapshots. Once a snapshot +is accepted, the state syncer will request snapshot chunks from appropriate peers: + +| Name | Type | Description | Field Number | +|--------|--------|-------------------------------------------------------------|--------------| +| height | uint64 | Height at which the chunk was created | 1 | +| format | uint32 | Format chosen for the chunk. **May be non-deterministic.** | 2 | +| index | uint32 | Index of the chunk within the snapshot. | 3 | + +### ChunkResponse + +The receiver will load the requested chunk from its local application via `LoadSnapshotChunk`, +and respond with it (limited to 16 MB): + +| Name | Type | Description | Field Number | +|---------|--------|-------------------------------------------------------------|--------------| +| height | uint64 | Height at which the chunk was created | 1 | +| format | uint32 | Format chosen for the chunk. **May be non-deterministic.** | 2 | +| index | uint32 | Index of the chunk within the snapshot. | 3 | +| hash | bytes | Arbitrary snapshot hash | 4 | +| missing | bool | Arbitrary application data. **May be non-deterministic.** | 5 | + +Here, `Missing` is used to signify that the chunk was not found on the peer, since an empty +chunk is a valid (although unlikely) response. + +The returned chunk is given to the ABCI application via `ApplySnapshotChunk` until the snapshot +is restored. If a chunk response is not returned within some time, it will be re-requested, +possibly from a different peer. + +The ABCI application is able to request peer bans and chunk refetching as part of the ABCI protocol. + +### LightBlockRequest + +To verify state and to provide state relevant information for consensus, the node will ask peers for +light blocks at specified heights. + +| Name | Type | Description | Field Number | +|----------|--------|----------------------------|--------------| +| height | uint64 | Height of the light block | 1 | + +### LightBlockResponse + +The receiver will retrieve and construct the light block from both the block and state stores. The +receiver will verify the data by comparing the hashes and store the header, commit and validator set +if necessary. The light block at the height of the snapshot will be used to verify the `AppHash`. + +| Name | Type | Description | Field Number | +|---------------|---------------------------------------------------------|--------------------------------------|--------------| +| light_block | [LightBlock](/cometbft/v0.39/spec/core/Data_structures#lightblock) | Light block at the height requested | 1 | + +State sync will use [light client verification](/cometbft/v0.39/spec/light-client/verification) to verify +the light blocks. + +If no state sync is in progress (i.e. during normal operation), any unsolicited response messages +are discarded. + +### ParamsRequest + +In order to build the state, the state provider will request the params at the height of the snapshot and use the header to verify it. + +| Name | Type | Description | Field Number | +|----------|--------|----------------------------|--------------| +| height | uint64 | Height of the consensus params | 1 | + + +### ParamsResponse + +A reciever to the request will use the state store to fetch the consensus params at that height and return it to the sender. + +| Name | Type | Description | Field Number | +|----------|--------|---------------------------------|--------------| +| height | uint64 | Height of the consensus params | 1 | +| consensus_params | [ConsensusParams](/cometbft/v0.39/spec/core/Data_structures#blockparams) | Consensus params at the height requested | 2 | + + +### Message + +Message is a [`oneof` protobuf type](https://developers.google.com/protocol-buffers/docs/proto#oneof). The `oneof` consists of eight messages. + +| Name | Type | Description | Field Number | +|----------------------|--------------------------------------------|----------------------------------------------|--------------| +| snapshots_request | [SnapshotRequest](#snapshotrequest) | Request a recent snapshot from a peer | 1 | +| snapshots_response | [SnapshotResponse](#snapshotresponse) | Respond with the most recent snapshot stored | 2 | +| chunk_request | [ChunkRequest](#chunkrequest) | Request chunks of the snapshot. | 3 | +| chunk_response | [ChunkRequest](#chunkresponse) | Response of chunks used to recreate state. | 4 | +| light_block_request | [LightBlockRequest](#lightblockrequest) | Request a light block. | 5 | +| light_block_response | [LightBlockResponse](#lightblockresponse) | Respond with a light block | 6 | +| params_request | [ParamsRequest](#paramsrequest) | Request the consensus params at a height. | 7 | +| params_response | [ParamsResponse](#paramsresponse) | Respond with the consensus params | 8 | diff --git a/cometbft/v0.39/spec/p2p/reactor-api/API-for-Reactors.mdx b/cometbft/v0.39/spec/p2p/reactor-api/API-for-Reactors.mdx new file mode 100644 index 000000000..56631f27f --- /dev/null +++ b/cometbft/v0.39/spec/p2p/reactor-api/API-for-Reactors.mdx @@ -0,0 +1,336 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/spec/p2p/reactor-api/API-for-Reactors' +title: API for Reactors +order: 3 +--- + +This document describes the API provided by the p2p layer to the protocol +layer, namely to the registered reactors. + +This API consists of two interfaces: the one provided by the `Switch` instance, +and the ones provided by multiple `Peer` instances, one per connected peer. +The `Switch` instance is provided to every reactor as part of the reactor's +[registration procedure](/cometbft/v0.39/spec/p2p/reactor-api/Reactor-Api#registration). +The multiple `Peer` instances are provided to every registered reactor whenever +a [new connection with a peer](/cometbft/v0.39/spec/p2p/reactor-api/Reactor-Api#peer-management) is established. + +> **Note** +> +> The practical reasons that lead to the interface to be provided in two parts, +> `Switch` and `Peer` instances are discussed in more datail in the +> [knowledge-base repository](https://github.com/cometbft/knowledge-base/blob/main/p2p/reactors/switch-peer.md). + +## `Switch` API + +The [`Switch`][switch-type] is the central component of the p2p layer +implementation. It manages all the reactors running in a node and keeps track +of the connections with peers. +The table below summarizes the interaction of the standard reactors with the `Switch`: + +| `Switch` API method | consensus | block sync | state sync | mempool | evidence | PEX | +|--------------------------------------------|-----------|------------|------------|---------|-----------|-------| +| `Peers() IPeerSet` | x | x | | | | x | +| `NumPeers() (int, int, int)` | | x | | | | x | +| `Broadcast(Envelope) chan bool` | x | x | x | | | | +| `MarkPeerAsGood(Peer)` | x | | | | | | +| `StopPeerForError(Peer, interface{})` | x | x | x | x | x | x | +| `StopPeerGracefully(Peer)` | | | | | | x | +| `Reactor(string) Reactor` | | x | | | | | + +The above list is not exhaustive as it does not include all the `Switch` methods +invoked by the PEX reactor, a special component that should be considered part +of the p2p layer. This document does not cover the operation of the PEX reactor +as a connection manager. + +### Peers State + +The first two methods in the switch API allow reactors to query the state of +the p2p layer: the set of connected peers. +~~~ + + func (sw *Switch) Peers() IPeerSet +~~~ + +The `Peers()` method returns the current set of connected peers. +The returned `IPeerSet` is an immutable concurrency-safe copy of this set. +Observe that the `Peer` handlers returned by this method were previously +[added to the reactor](/cometbft/v0.39/spec/p2p/reactor-api/Reactor-Api#peer-management) via the `InitPeer(Peer)` method, +but not yet removed via the `RemovePeer(Peer)` method. +Thus, a priori, reactors should already have this information. +~~~ + + func (sw *Switch) NumPeers() (outbound, inbound, dialing int) +~~~ + +The `NumPeers()` method returns the current number of connected peers, +distinguished between `outbound` and `inbound` peers. +An `outbound` peer is a peer the node has dialed to, while an `inbound` peer is +a peer the node has accepted a connection from. +The third field `dialing` reports the number of peers to which the node is +currently attempting to connect, so not (yet) connected peers. + +> **Note** +> +> The third field returned by `NumPeers()`, the number of peers in `dialing` +> state, is not an information that should regard the protocol layer. +> In fact, with the exception of the PEX reactor, which can be considered part +> of the p2p layer implementation, no standard reactor actually uses this +> information, that could be removed when this interface is refactored. + +### Broadcast + +The switch provides, mostly for historical or retro-compatibility reasons, +a method for sending a message to all connected peers: +~~~ + + func (sw *Switch) Broadcast(e Envelope) chan bool +~~~ + +The `Broadcast()` method is not blocking and returns a channel of booleans. +For every connected `Peer`, it starts a background thread for sending the +message to that peer, using the `Peer.Send()` method +(which is blocking, as detailed in [Send Methods](#send-methods)). +The result of each unicast send operation (success or failure) is added to the +returned channel, which is closed when all operations are completed. + +> **Note** +> +> - The current _implementation_ of the `Switch.Broadcast(Envelope)` method is +> not efficient, as the marshalling of the provided message is performed as +> part of the `Peer.Send(Envelope)` helper method, that is, once per +> connected peer. +> - The return value of the broadcast method is not considered by any of the +> standard reactors that employ the method. One of the reasons is that is is +> not possible to associate each of the boolean outputs added to the +> returned channel to a peer. + +### Vetting Peers + +The p2p layer relies on the registered reactors to gauge the _quality_ of peers. +The following method can be invoked by a reactor to inform the p2p layer that a +peer has presented a "good" behaviour. +This information is registered in the node's address book and influences the +operation of the Peer Exchange (PEX) protocol, as node discovery adopts a bias +towards "good" peers: +~~~ + + func (sw *Switch) MarkPeerAsGood(peer Peer) +~~~ + +At the moment, it is up to the consensus reactor to vet a peer. +In the current logic, a peer is marked as good whenever the consensus protocol +collects a multiple of `votesToContributeToBecomeGoodPeer = 10000` useful votes +or `blocksToContributeToBecomeGoodPeer = 10000` useful block parts from that peer. +By "useful", the consensus implementation considers messages that are valid and +that are received by the node when the node is expected for such information, +which excludes duplicated or late received messages. + +> **Note** +> +> The switch doesn't currently provide a method to mark a peer as a bad peer. +> In fact, the peer quality management is really implemented in the current +> version of the p2p layer. +> This topic is being discussed in the [knowledge-base repository](https://github.com/cometbft/knowledge-base/blob/main/p2p/reactors/peer-quality.md). + +### Stopping Peers + +Reactors can instruct the p2p layer to disconnect from a peer. +Using the p2p layer's nomenclature, the reactor requests a peer to be stopped. +The peer's send and receive routines are in fact stopped, interrupting the +communication with the peer. +The `Peer` is then [removed from every registered reactor][reactor-removepeer], +using the `RemovePeer(Peer)` method, and from the set of connected peers. +~~~ + + func (sw *Switch) StopPeerForError(peer Peer, reason interface{}) +~~~ + +All the standard reactors employ the above method for disconnecting from a peer +in case of errors. +These are errors that occur when processing a message received from a `Peer`. +The produced `error` is provided to the method as the `reason`. + +The `StopPeerForError()` method has an important *caveat*: if the peer to be +stopped is configured as a _persistent peer_, the switch will attempt +reconnecting to that same peer. +While this behaviour makes sense when the method is invoked by other components +of the p2p layer (e.g., in the case of communication errors), it does not make +sense when it is invoked by a reactor. + +> **Note** +> +> A more comprehensive discussion regarding this topic can be found on the +> [knowledge-base repository](https://github.com/cometbft/knowledge-base/blob/main/p2p/reactors/stop-peer.md). + + func (sw *Switch) StopPeerGracefully(peer Peer) + +The second method instructs the switch to disconnect from a peer for no +particular reason. +This method is only adopted by the PEX reactor of a node operating in _seed mode_, +as seed nodes disconnect from a peer after exchanging peer addresses with it. + +### Reactors Table + +The switch keeps track of all registered reactors, indexed by unique reactor names. +A reactor can therefore use the switch to access another `Reactor` from its `name`: +~~~ + + func (sw *Switch) Reactor(name string) Reactor +~~~ + +This method is currently only used by the Block Sync reactor to access the +Consensus reactor implementation, from which it uses the exported +`SwitchToConsensus()` method. +While available, this inter-reactor interaction approach is discouraged and +should be avoided, as it violates the assumption that reactors are independent. + + +## `Peer` API + +The [`Peer`][peer-interface] interface represents a connected peer. +A `Peer` instance encapsulates a multiplex connection that implements the +actual communication (sending and receiving messages) with a peer. +When a connection is established with a peer, the `Switch` provides the +corresponding `Peer` instance to all registered reactors. +From this point, reactors can use the methods of the new `Peer` instance. + +The table below summarizes the interaction of the standard reactors with +connected peers, with the `Peer` methods used by them: + +| `Peer` API method | consensus | block sync | state sync | mempool | evidence | PEX | +|--------------------------------------------|-----------|------------|------------|---------|-----------|-------| +| `ID() ID` | x | x | x | x | x | x | +| `IsRunning() bool` | x | | | x | x | | +| `Quit() <-chan struct{}` | | | | x | x | | +| `Get(string) interface{}` | x | | | x | x | | +| `Set(string, interface{})` | x | | | | | | +| `Send(Envelope) bool` | x | x | x | x | x | x | +| `TrySend(Envelope) bool` | x | x | | | | | + +The above list is not exhaustive as it does not include all the `Peer` methods +invoked by the PEX reactor, a special component that should be considered part +of the p2p layer. This document does not cover the operation of the PEX reactor +as a connection manager. + +### Identification + +Nodes in the p2p network are configured with a unique cryptographic key pair. +The public part of this key pair is verified when establishing a connection +with the peer, as part of the authentication handshake, and constitutes the +peer's `ID`: +~~~ + + func (p Peer) ID() p2p.ID +~~~ + +Observe that each time the node connects to a peer (e.g., after disconnecting +from it), a new (distinct) `Peer` handler is provided to the reactors via +`InitPeer(Peer)` method. +In fact, the `Peer` handler is associated to a _connection_ with a peer, not to +the actual _node_ in the network. +To keep track of actual peers, the unique peer `p2p.ID` provided by the above +method should be employed. + +### Peer state + +The switch starts the peer's send and receive routines before adding the peer +to every registered reactor using the `AddPeer(Peer)` method. +The reactors then usually start routines to interact with the new connected +peer using the received `Peer` handler. +For these routines it is useful to check whether the peer is still connected +and its send and receive routines are still running: +~~~ + + func (p Peer) IsRunning() bool + func (p Peer) Quit() <-chan struct{} +~~~ + +The above two methods provide the same information about the state of a `Peer` +instance in two different ways. +Both of them are defined in the [`Service`][service-interface] interface. +The `IsRunning()` method is synchronous and returns whether the peer has been +started and has not been stopped. +The `Quit()` method returns a channel that is closed when the peer is stopped; +it is an asynchronous state query. + +### Key-value store + +Each `Peer` instance provides a synchronized key-value store that allows +sharing peer-specific state between reactors: + +~~~ + + func (p Peer) Get(key string) interface{} + func (p Peer) Set(key string, data interface{}) +~~~ + +This key-value store can be seen as an asynchronous mechanism to exchange the +state of a peer between reactors. +In the current use-case of this mechanism, the Consensus reactor populates the +key-value store with a `PeerState` instance for each connected peer. +The Consensus reactor routines interacting with a peer read and update the +shared peer state. +The Evidence and Mempool reactors, in their turn, periodically query the +key-value store of each peer for retrieving, in particular, the last height +reported by the peer. +This information, produced by the Consensus reactor, influences the interaction +of these two reactors with their peers. + +> **Note** +> +> More details of how this key-value store is used to share state between reactors can be found on the +> [knowledge-base repository](https://github.com/cometbft/knowledge-base/blob/main/p2p/reactors/peer-kvstore.md). + +### Send methods + +Finally, a `Peer` instance allows a reactor to send messages to companion +reactors running at that peer. +This is ultimately the goal of the switch when it provides `Peer` instances to +the registered reactors. +There are two methods for sending messages: +~~~ + + func (p Peer) Send(e Envelope) bool + func (p Peer) TrySend(e Envelope) bool +~~~ + +The two message-sending methods receive an `Envelope`, whose content should be +set as follows: + +- `ChannelID`: the channel the message should be sent through, which defines + the reactor that will process the message; +- `Src`: this field represents the source of an incoming message, which is + irrelevant for outgoing messages; +- `Message`: the actual message's payload, which is marshalled using protocol buffers. + +The two message-sending methods attempt to add the message (`e.Payload`) to the +send queue of the peer's destination channel (`e.ChannelID`). +There is a send queue for each registered channel supported by the peer, and +each send queue has a capacity. +The capacity of the send queues for each channel are [configured](/cometbft/v0.39/spec/p2p/reactor-api/Reactor-Api#registration) +by reactors via the corresponding `ChannelDescriptor`. + +The two message-sending methods return whether it was possible to enqueue +the marshalled message to the channel's send queue. +The most common reason for these methods to return `false` is the channel's +send queue being full. +Further reasons for returning `false` are: the peer being stopped, providing a +non-registered channel ID, or errors when marshalling the message's payload. + +The difference between the two message-sending methods is _when_ they return `false`. +The `Send()` method is a _blocking_ method, it returns `false` if the message +could not be enqueued, because the channel's send queue is still full, after a +10-second _timeout_. +The `TrySend()` method is a _non-blocking_ method, it _immediately_ returns +`false` when the channel's send queue is full. + +[peer-interface]: https://github.com/cometbft/cometbft/blob/v0.38.x/p2p/peer.go +[service-interface]: https://github.com/cometbft/cometbft/blob/v0.38.x/libs/service/service.go +[switch-type]: https://github.com/cometbft/cometbft/blob/v0.38.x/p2p/switch.go + +[reactor-interface]: https://github.com/cometbft/cometbft/blob/v0.38.x/p2p/base_reactor.go +[reactor-registration]: /cometbft/v0.39/spec/p2p/reactor-api/Reactor-Api#registration +[reactor-channels]: /cometbft/v0.39/spec/p2p/reactor-api/Reactor-Api#registration +[reactor-addpeer]: /cometbft/v0.39/spec/p2p/reactor-api/Reactor-Api#peer-management +[reactor-removepeer]: /cometbft/v0.39/spec/p2p/reactor-api/Reactor-Api#stop-peer diff --git a/cometbft/v0.39/spec/p2p/reactor-api/Reactor-Api.mdx b/cometbft/v0.39/spec/p2p/reactor-api/Reactor-Api.mdx new file mode 100644 index 000000000..a5ada62e2 --- /dev/null +++ b/cometbft/v0.39/spec/p2p/reactor-api/Reactor-Api.mdx @@ -0,0 +1,235 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/spec/p2p/reactor-api/Reactor-Api' +title: Reactor API +order: 2 +--- + +A component has to implement the [`p2p.Reactor` interface][reactor-interface] +in order to use communication services provided by the p2p layer. +This interface is currently the main source of documentation for a reactor. + +The goal of this document is to specify the behaviour of the p2p communication +layer when interacting with a reactor. +So while the [`Reactor interface`][reactor-interface] declares the methods +invoked and determines what the p2p layer expects from a reactor, +this documentation focuses on the **temporal behaviour** that a reactor implementation +should expect from the p2p layer. (That is, in which orders the functions may be called) + +This specification is accompanied by the [`reactor.qnt`](https://github.com/cometbft/cometbft/blob/v0.38.x/spec/p2p/reactor-api/reactor.qnt) file, +a more comprehensive model of the reactor's operation written in +[Quint][quint-repo], an executable specification language. +The methods declared in the [`Reactor`][reactor-interface] interface are +modeled in Quint, in the form of `pure def` methods, providing some examples of +how they should be implemented. +The behaviour of the p2p layer when interacting with a reactor, by invoking the +interface methods, is modeled in the form of state transitions, or `action`s in +the Quint nomenclature. + +## Overview + +The following _grammar_ is a simplified representation of the expected sequence of calls +from the p2p layer to a reactor. +Note that the grammar represents events referring to a _single reactor_, while +the p2p layer supports the execution of multiple reactors. +For a more detailed representation of the sequence of calls from the p2p layer +to reactors, please refer to the companion Quint model. + +While useful to provide an overview of the operation of a reactor, +grammars have some limitations in terms of the behaviour they can express. +For instance, the following grammar only represents the management of _a single peer_, +namely of a peer with a given ID which can connect, disconnect, and reconnect +multiple times to the node. +The p2p layer and every reactor should be able to handle multiple distinct peers in parallel. +This means that multiple occurrences of non-terminal `peer-management` of the +grammar below can "run" independently and in parallel, each one referring and +producing events associated to a different peer: + +```abnf +start = registration on-start *peer-management on-stop +registration = get-channels set-switch + +; Refers to a single peer, a reactor must support multiple concurrent peers +peer-management = init-peer start-peer stop-peer +start-peer = [*receive] (connected-peer / start-error) +connected-peer = add-peer *receive +stop-peer = [peer-error] remove-peer + +; Service interface +on-start = %s"OnStart()" +on-stop = %s"OnStop()" +; Reactor interface +get-channels = %s"GetChannels()" +set-switch = %s"SetSwitch(*Switch)" +init-peer = %s"InitPeer(Peer)" +add-peer = %s"AddPeer(Peer)" +remove-peer = %s"RemovePeer(Peer, reason)" +receive = %s"Receive(Envelope)" + +; Errors, for reference +start-error = %s"log(Error starting peer)" +peer-error = %s"log(Stopping peer for error)" +``` + +The grammar is written in case-sensitive Augmented Backus–Naur form (ABNF, +specified in [IETF RFC 7405](https://datatracker.ietf.org/doc/html/rfc7405)). +It is inspired on the grammar produced to specify the interaction of CometBFT +with an ABCI++ application, available [here](/cometbft/v0.39/spec/abci/CometBFTs-expected-behavior). + +## Registration + +To become a reactor, a component has first to implement the +[`Reactor`][reactor-interface] interface, +then to register the implementation with the p2p layer, using the +`Switch.AddReactor(name string, reactor Reactor)` method, +with a global unique `name` for the reactor. + +The registration must happen before the node, in general, and the p2p layer, +in particular, are started. +In other words, there is no support for registering a reactor on a running node: +reactors must be registered as part of the setup of a node. + +```abnf +registration = get-channels set-switch +``` + +The p2p layer retrieves from the reactor a list of channels the reactor is +responsible for, using the `GetChannels()` method. +The reactor implementation should thereafter expect the delivery of every +message received by the p2p layer in the informed channels. + +The second method `SetSwitch(Switch)` concludes the handshake between the +reactor and the p2p layer. +The `Switch` is the main component of the p2p layer, being responsible for +establishing connections with peers and routing messages. +The `Switch` instance provides a number of methods for all registered reactors, +documented in the companion [API for Reactors](/cometbft/v0.39/spec/p2p/reactor-api/API-for-Reactors#switch-api) document. + +## Service interface + +A reactor must implement the [`Service`](https://github.com/cometbft/cometbft/blob/v0.38.x/libs/service/service.go) interface, +in particular, a startup `OnStart()` and a shutdown `OnStop()` methods: + +```abnf +start = registration on-start *peer-management on-stop +``` + +As part of the startup of a node, all registered reactors are started by the p2p layer. +And when the node is shut down, all registered reactors are stopped by the p2p layer. +Observe that the `Service` interface specification establishes that a service +can be started and stopped only once. +So before being started or once stopped by the p2p layer, the reactor should +not expect any interaction. + +## Peer management + +The core of a reactor's operation is the interaction with peers or, more +precisely, with companion reactors operating on the same channels in peers connected to the node. +The grammar extract below represents the interaction of the reactor with a +single peer: + +```abnf +; Refers to a single peer, a reactor must support multiple concurrent peers +peer-management = init-peer start-peer stop-peer +``` + +The p2p layer informs all registered reactors when it establishes a connection +with a `Peer`, using the `InitPeer(Peer)` method. +When this method is invoked, the `Peer` has not yet been started, namely the +routines for sending messages to and receiving messages from the peer are not running. +This method should be used to initialize state or data related to the new +peer, but not to interact with it. + +The next step is to start the communication routines with the new `Peer`. +As detailed in the following, this procedure may or may not succeed. +In any case, the peer is eventually stopped, which concludes the management of +that `Peer` instance. + +## Start peer + +Once `InitPeer(Peer)` is invoked for every registered reactor, the p2p layer starts the peer's +communication routines and adds the `Peer` to the set of connected peers. +If both steps are concluded without errors, the reactor's `AddPeer(Peer)` is invoked: + +```abnf +start-peer = [*receive] (connected-peer / start-error) +connected-peer = add-peer *receive +``` + +In case of errors, a message is logged informing that the p2p layer failed to start the peer. +This is not a common scenario and it is only expected to happen when +interacting with a misbehaving or slow peer. A practical example is reported on this +[issue](https://github.com/tendermint/tendermint/pull/9500). + +It is up to the reactor to define how to process the `AddPeer(Peer)` event. +The typical behavior is to start routines that, given some conditions or events, +send messages to the added peer, using the provided `Peer` instance. +The companion [API for Reactors](/cometbft/v0.39/spec/p2p/reactor-api/API-for-Reactors#peer-api) documents the methods +provided by `Peer` instances, available from when they are added to the reactors. + +## Stop Peer + +The p2p layer informs all registered reactors when it disconnects from a `Peer`, +using the `RemovePeer(Peer, reason)` method: + +```abnf +stop-peer = [peer-error] remove-peer +``` + +This method is invoked after the p2p layer has stopped peer's send and receive routines. +Depending of the `reason` for which the peer was stopped, different log +messages can be produced. +After removing a peer from all reactors, the `Peer` instance is also removed from +the set of connected peers. +This enables the same peer to reconnect and `InitPeer(Peer)` to be invoked for +the new connection. + +From the removal of a `Peer` , the reactor should not receive any further message +from the peer and must not try sending messages to the removed peer. +This usually means stopping the routines that were started by the companion +`Add(Peer)` method. + +## Receive messages + +The main duty of a reactor is to handle incoming messages on the channels it +has registered with the p2p layer. + +The _pre-condition_ for receiving a message from a `Peer` is that the p2p layer +has previously invoked `InitPeer(Peer)`. +This means that the reactor must be able to receive a message from a `Peer` +_before_ `AddPeer(Peer)` is invoked. +This happens because the peer's send and receive routines are started before, +and should be already running when the p2p layer adds the peer to every +registered reactor. + +```abnf +start-peer = [*receive] (connected-peer / start-error) +connected-peer = add-peer *receive +``` + +The most common scenario, however, is to start receiving messages from a peer +after `AddPeer(Peer)` is invoked. +An arbitrary number of messages can be received, until the peer is stopped and +`RemovePeer(Peer)` is invoked. + +When a message is received from a connected peer on any of the channels +registered by the reactor, the p2p layer will deliver the message to the +reactor via the `Receive(Envelope)` method. +The message is packed into an `Envelope` that contains: + +- `ChannelID`: the channel the message belongs to +- `Src`: the source `Peer` handler, from which the message was received +- `Message`: the actual message's payload, unmarshalled using protocol buffers + +Two important observations regarding the implementation of the `Receive` method: + +1. Concurrency: the implementation should consider concurrent invocations of + the `Receive` method carrying messages from different peers, as the + interaction with different peers is independent and messages can be received in parallel. +1. Non-blocking: the implementation of the `Receive` method is expected not to block, + as it is invoked directly by the receive routines. + In other words, while `Receive` does not return, other messages from the + same sender are not delivered to any reactor. + +[reactor-interface]: https://github.com/cometbft/cometbft/blob/v0.38.x/p2p/base_reactor.go +[quint-repo]: https://github.com/informalsystems/quint diff --git a/cometbft/v0.39/spec/p2p/reactor-api/Reactors.mdx b/cometbft/v0.39/spec/p2p/reactor-api/Reactors.mdx new file mode 100644 index 000000000..f9e1d08e4 --- /dev/null +++ b/cometbft/v0.39/spec/p2p/reactor-api/Reactors.mdx @@ -0,0 +1,48 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/spec/p2p/reactor-api/Reactors' +title: Reactors +order: 1 +--- + +Reactor is the generic name for a component that employs the p2p communication layer. + +This section documents the interaction of the p2p communication layer with the +reactors. +The diagram below summarizes this interaction, namely the **northbound interface** +of the p2p communication layer, representing some relevant event flows: + +![P2P Reactors](../images/p2p-reactors.png) + +Each of the protocols running a CometBFT node implements a reactor and registers +the implementation with the p2p layer. +The p2p layer provides network events to the registered reactors, the main +two being new connections with peers and received messages. +The reactors provide to the p2p layer messages to be sent to +peers and commands to control the operation of the p2p layer. + +It is worth noting that the components depicted in the diagram below run +multiple routines and that the illustrated actions happen in parallel. +For instance, the connection establishment routines run in parallel, invoking +the depicted `AddPeer` method concurrently. +Once a connection is fully established, each `Peer` instance runs a send and a +receive routines. +The send routine collects messages from multiple reactors to a peer, packaging +then into raw messages which are transmitted to the peer. +The receive routine processes incoming messages and forwards them to the +destination reactors, invoking the depicted `Receive` methods. +In addition, the reactors run multiple routines for interacting +with the peers (for example, to send messages to them) or with the `Switch`. + +The remaining of the documentation is organized as follows: + +- [Reactor API](/cometbft/v0.39/spec/p2p/reactor-api/Reactor-Api): documents the [`p2p.Reactor`][reactor-interface] + interface and specifies the behaviour of the p2p layer when interacting with + a reactor. + In other words, the interaction of the p2p layer with the protocol layer (bottom-up). + +- [P2P API](/cometbft/v0.39/spec/p2p/reactor-api/API-for-Reactors): documents the interface provided by the p2p + layer to the reactors, through the `Switch` and `Peer` abstractions. + In other words, the interaction of the protocol layer with the p2p layer (top-down). + +[reactor-interface]: https://github.com/cometbft/cometbft/blob/v0.38.x/p2p/base_reactor.go diff --git a/cometbft/v0.39/spec/p2p/reactor-api/reactor.qnt b/cometbft/v0.39/spec/p2p/reactor-api/reactor.qnt new file mode 100644 index 000000000..002c57023 --- /dev/null +++ b/cometbft/v0.39/spec/p2p/reactor-api/reactor.qnt @@ -0,0 +1,276 @@ +// -*- mode: Bluespec; -*- +/* + * Reactor is responsible for handling incoming messages on one or more + * Channel. Switch calls GetChannels when reactor is added to it. When a new + * peer joins our node, InitPeer and AddPeer are called. RemovePeer is called + * when the peer is stopped. Receive is called when a message is received on a + * channel associated with this reactor. + */ +// Code: https://github.com/cometbft/cometbft/blob/main/p2p/base_reactor.go +module reactor { + + // Unique ID of a node. + type NodeID = str + + /* + * Peer is an interface representing a peer connected on a reactor. + */ + type Peer = { + ID: NodeID, + + // Other fields can be added to represent the p2p operation. + } + + // Byte ID used by channels, must be globally unique. + type Byte = str + + // Channel configuration. + type ChannelDescriptor = { + ID: Byte, + Priority: int, + } + + /* + * Envelope contains a message with sender routing info. + */ + type Envelope = { + Src: Peer, // Sender + Message: str, // Payload + ChannelID: Byte, + } + + // A Routine is used to interact with an active Peer. + type Routine = { + name: str, + peer: Peer, + } + + type ReactorState = { + // Peers that have been initialized but not yet removed. + // The reactor should expect receiving messages from them. + peers: Set[NodeID], + + // The reactor runs multiple routines. + routines: Set[Routine], + + // Values: init -> registered -> running -> stopped + state: str, + + // Name with which the reactor was registered. + name: str, + + // Channels the reactor is responsible for. + channels: Set[ChannelDescriptor], + } + + // Produces a new, uninitialized reactor. + pure def NewReactor(): ReactorState = { + { + peers: Set(), + routines: Set(), + state: "init", + name: "", + channels: Set(), + } + } + + // Pure definitions below represent the `p2p.Reactor` interface methods: + + /* + * GetChannels returns the list of MConnection.ChannelDescriptor. Make sure + * that each ID is unique across all the reactors added to the switch. + */ + pure def GetChannels(s: ReactorState): Set[ChannelDescriptor] = { + s.channels // Static list, configured at initialization. + } + + /* + * SetSwitch allows setting a switch. + */ + pure def SetSwitch(s: ReactorState, switch: bool): ReactorState = { + s.with("state", "registered") + } + + /* + * Start the service. + * If it's already started or stopped, will return an error. + */ + pure def OnStart(s: ReactorState): ReactorState = { + // Startup procedures should come here. + s.with("state", "running") + } + + /* + * Stop the service. + * If it's already stopped, will return an error. + */ + pure def OnStop(s: ReactorState): ReactorState = { + // Shutdown procedures should come here. + s.with("state", "stopped") + } + + /* + * InitPeer is called by the switch before the peer is started. Use it to + * initialize data for the peer (e.g. peer state). + */ + pure def InitPeer(s: ReactorState, peer: Peer): (ReactorState, Peer) = { + // This method can update the received peer, which is returned. + val updatedPeer = peer + (s.with("peers", s.peers.union(Set(peer.ID))), updatedPeer) + } + + /* + * AddPeer is called by the switch after the peer is added and successfully + * started. Use it to start goroutines communicating with the peer. + */ + pure def AddPeer(s: ReactorState, peer: Peer): ReactorState = { + // This method can be used to start routines to handle the peer. + // Below an example of an arbitrary 'ioRoutine' routine. + val startedRoutines = Set( {name: "ioRoutine", peer: peer} ) + s.with("routines", s.routines.union(startedRoutines)) + } + + /* + * RemovePeer is called by the switch when the peer is stopped (due to error + * or other reason). + */ + pure def RemovePeer(s: ReactorState, peer: Peer, reason: str): ReactorState = { + // This method should stop routines created by `AddPeer(Peer)`. + val stoppedRoutines = s.routines.filter(r => r.peer.ID == peer.ID) + s.with("peers", s.peers.exclude(Set(peer.ID))) + .with("routines", s.routines.exclude(stoppedRoutines)) + } + + /* + * Receive is called by the switch when an envelope is received from any connected + * peer on any of the channels registered by the reactor. + */ + pure def Receive(s: ReactorState, e: Envelope): ReactorState = { + // This method should process the message payload: e.Message. + s + } + + // Global state + + // Reactors are uniquely identified by their names. + var reactors: str -> ReactorState + + // Reactor (name) assigned to each channel ID. + var reactorsByCh: Byte -> str + + // Helper action to (only) update the state of a given reactor. + action updateReactorTo(reactor: ReactorState): bool = all { + reactors' = reactors.set(reactor.name, reactor), + reactorsByCh' = reactorsByCh + } + + // State transitions performed by the p2p layer, invoking `p2p.Reactor` methods: + + // Code: Switch.AddReactor(name string, reactor Reactor) + action register(name: str, reactor: ReactorState): bool = all { + reactor.state == "init", + // Assign the reactor as responsible for its channel IDs, which + // should not be already assigned to another reactor. + val chIDs = reactor.GetChannels().map(c => c.ID) + all { + size(chIDs.intersect(reactorsByCh.keys())) == 0, + reactorsByCh' = reactorsByCh.keys().union(chIDs). + mapBy(id => if (id.in(chIDs)) name + else reactorsByCh.get(id)), + }, + // Register the reactor by its name, which must be unique. + not(name.in(reactors.keys())), + reactors' = reactors.put(name, + reactor.SetSwitch(true).with("name", name)) + } + + // Code: Switch.OnStart() + action start(reactor: ReactorState): bool = all { + reactor.state == "registered", + updateReactorTo(reactor.OnStart()) + } + + // Code: Switch.addPeer(p Peer): preamble + action initPeer(reactor: ReactorState, peer: Peer): bool = all { + reactor.state == "running", + not(peer.ID.in(reactor.peers)), + updateReactorTo(reactor.InitPeer(peer)._1) + } + + // Code: Switch.addPeer(p Peer): conclusion + action addPeer(reactor: ReactorState, peer: Peer): bool = all { + reactor.state == "running", + peer.ID.in(reactor.peers), // InitPeer(peer) and not RemovePeer(peer) + reactor.routines.filter(r => r.peer.ID == peer.ID).size() == 0, + updateReactorTo(reactor.AddPeer(peer)) + } + + // Code: Switch.stopAndRemovePeer(peer Peer, reason interface{}) + action removePeer(reactor: ReactorState, peer: Peer, reason: str): bool = all { + reactor.state == "running", + peer.ID.in(reactor.peers), // InitPeer(peer) and not RemovePeer(peer) + // Routines might not be started, namely: not AddPeer(peer) + // Routines could also be already stopped if Peer has erroed. + updateReactorTo(reactor.RemovePeer(peer, reason)) + } + + // Code: Peer type, onReceive := func(chID byte, msgBytes []byte) + action receive(reactor: ReactorState, e: Envelope): bool = all { + reactor.state == "running", + // The message's sender is an active peer + e.Src.ID.in(reactor.peers), + // Reactor is assigned to the message's channel ID + e.ChannelID.in(reactorsByCh.keys()), + reactorsByCh.get(e.ChannelID) == reactor.name, + reactor.GetChannels().exists(c => c.ID == e.ChannelID), + updateReactorTo(reactor.Receive(e)) + } + + // Code: Switch.OnStop() + action stop(reactor: ReactorState): bool = all { + reactor.state == "running", + // Either no peer was added or all peers were removed + reactor.peers.size() == 0, + updateReactorTo(reactor.OnStop()) + } + + // Simulation support + + action init = all { + reactors' = Map(), + reactorsByCh' = Map(), + } + + // Modelled reactor configuration + pure val reactorName = "myReactor" + pure val reactorChannels = Set({ID: "3", Priority: 1}, {ID: "7", Priority: 2}) + + // For retro-compatibility: the state of the modelled reactor + def state(): ReactorState = { + reactors.get(reactorName) + } + + pure val samplePeers = Set({ID: "p1"}, {ID: "p3"}) + pure val sampleChIDs = Set("1", "3", "7") // ChannelID 1 not registered + pure val sampleMsgs = Set("ping", "pong") + + action step = any { + register(reactorName, NewReactor.with("channels", reactorChannels)), + val reactor = reactors.get(reactorName) + any { + reactor.start(), + reactor.stop(), + nondet peer = oneOf(samplePeers) + any { + // Peer-specific actions + reactor.initPeer(peer), + reactor.addPeer(peer), + reactor.removePeer(peer, "no reason"), + reactor.receive({Src: peer, + ChannelID: oneOf(sampleChIDs), + Message: oneOf(sampleMsgs)}), + } + } + } + +} diff --git a/cometbft/v0.39/spec/rpc/Rpc-Spe.mdx b/cometbft/v0.39/spec/rpc/Rpc-Spe.mdx new file mode 100644 index 000000000..fa0786e8b --- /dev/null +++ b/cometbft/v0.39/spec/rpc/Rpc-Spe.mdx @@ -0,0 +1,1262 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/cometbft/latest/spec/rpc/Rpc-Spe' +title: RPC spec +order: 1 +--- + +This file defines the JSON-RPC spec of CometBFT. This is meant to be implemented by all clients. + +## Support + + | | [CometBFT](https://github.com/cometbft/cometbft/) | [Tendermint-Rs](https://github.com/informalsystems/tendermint-rs) | + |--------------|:----------------------------------------------------------:|:----------------------------------------------------------------:| + | JSON-RPC 2.0 | ✅ | ✅ | + | HTTP | ✅ | ✅ | + | HTTPS | ✅ | ❌ | + | WS | ✅ | ✅ | + + | Routes | [CometBFT](https://github.com/cometbft/cometbft/) | [Tendermint-Rs](https://github.com/informalsystems/tendermint-rs) | + |-----------------------------------------|:----------------------------------------------------------:|:-----------------------------------------------------------------:| + | [Health](#health) | ✅ | ✅ | + | [Status](#status) | ✅ | ✅ | + | [NetInfo](#netinfo) | ✅ | ✅ | + | [Blockchain](#blockchain) | ✅ | ✅ | + | [Block](#block) | ✅ | ✅ | + | [BlockByHash](#blockbyhash) | ✅ | ❌ | + | [BlockResults](#blockresults) | ✅ | ✅ | + | [Commit](#commit) | ✅ | ✅ | + | [Validators](#validators) | ✅ | ✅ | + | [Genesis](#genesis) | ✅ | ✅ | + | [GenesisChunked](#genesischunked) | ✅ | ❌ | + | [ConsensusParams](#consensusparams) | ✅ | ❌ | + | [UnconfirmedTxs](#unconfirmedtxs) | ✅ | ❌ | + | [NumUnconfirmedTxs](#numunconfirmedtxs) | ✅ | ❌ | + | [Tx](#tx) | ✅ | ❌ | + | [BroadCastTxSync](#broadcasttxsync) | ✅ | ✅ | + | [BroadCastTxAsync](#broadcasttxasync) | ✅ | ✅ | + | [ABCIInfo](#abciinfo) | ✅ | ✅ | + | [ABCIQuery](#abciquery) | ✅ | ✅ | + | [BroadcastTxAsync](#broadcasttxasync) | ✅ | ✅ | + | [BroadcastEvidence](#broadcastevidence) | ✅ | ✅ | + +## Timestamps + +Timestamps in the RPC layer of CometBFT follows RFC3339Nano. The RFC3339Nano format removes trailing zeros from the seconds field. + +This means if a block has a timestamp like: `1985-04-12T23:20:50.5200000Z`, the value returned in the RPC will be `1985-04-12T23:20:50.52Z`. + + + +## Info Routes + +### Health + +Node heartbeat + +#### Parameters + +None + +#### Request + +##### HTTP + +```sh +curl http://127.0.0.1:26657/health +``` + +##### JSONRPC + +```sh +curl -X POST https://localhost:26657 -d "{\"jsonrpc\":\"2.0\",\"id\":1,\"method\":\"health\"}" +``` + +#### Response + +```json +{ + "jsonrpc": "2.0", + "id": -1, + "result": {} +} +``` + +### Status + +Get CometBFT status including node info, pubkey, latest block hash, app hash, block height and time. + +#### Parameters + +None + +#### Request + +##### HTTP + +```sh +curl http://127.0.0.1:26657/status +``` + +##### JSONRPC + +```sh +curl -X POST https://localhost:26657 -d "{\"jsonrpc\":\"2.0\",\"id\":1,\"method\":\"status\"}" +``` + +#### Response + +```json +{ + "jsonrpc": "2.0", + "id": -1, + "result": { + "node_info": { + "protocol_version": { + "p2p": "8", + "block": "11", + "app": "0" + }, + "id": "b93270b358a72a2db30089f3856475bb1f918d6d", + "listen_addr": "tcp://0.0.0.0:26656", + "network": "cosmoshub-4", + "version": "v0.34.8", + "channels": "40202122233038606100", + "moniker": "aib-hub-node", + "other": { + "tx_index": "on", + "rpc_address": "tcp://0.0.0.0:26657" + } + }, + "sync_info": { + "latest_block_hash": "50F03C0EAACA8BCA7F9C14189ACE9C05A9A1BBB5268DB63DC6A3C848D1ECFD27", + "latest_app_hash": "2316CFF7644219F4F15BEE456435F280E2B38955EEA6D4617CCB6D7ABF781C22", + "latest_block_height": "5622165", + "latest_block_time": "2021-03-25T14:00:43.356134226Z", + "earliest_block_hash": "1455A0C15AC49BB506992EC85A3CD4D32367E53A087689815E01A524231C3ADF", + "earliest_app_hash": "E3B0C44298FC1C149AFBF4C8996FB92427AE41E4649B934CA495991B7852B855", + "earliest_block_height": "5200791", + "earliest_block_time": "2019-12-11T16:11:34Z", + "catching_up": false + }, + "validator_info": { + "address": "38FB765D0092470989360ECA1C89CD06C2C1583C", + "pub_key": { + "type": "tendermint/PubKeyEd25519", + "value": "Z+8kntVegi1sQiWLYwFSVLNWqdAUGEy7lskL78gxLZI=" + }, + "voting_power": "0" + } + } +} +``` + +### NetInfo + +Network information + +#### Parameters + +None + +#### Request + +##### HTTP + +```sh +curl http://127.0.0.1:26657/net_info +``` + +##### JSONRPC + +```sh +curl -X POST https://localhost:26657 -d "{\"jsonrpc\":\"2.0\",\"id\":1,\"method\":\"net_info\"}" +``` + +#### Response + +```json +{ + "id": 0, + "jsonrpc": "2.0", + "result": { + "listening": true, + "listeners": [ + "Listener(@)" + ], + "n_peers": "1", + "peers": [ + { + "node_id": "5576458aef205977e18fd50b274e9b5d9014525a", + "url": "tcp://5576458aef205977e18fd50b274e9b5d9014525a@95.179.155.35:26656" + } + ] + } +} +``` + +### Blockchain + +Get block headers. Returned in descending order. May be limited in quantity. + +#### Parameters + +- `minHeight (integer)`: The lowest block to be returned in the response +- `maxHeight (integer)`: The highest block to be returned in the response + +#### Request + +##### HTTP + +```sh +curl http://127.0.0.1:26657/blockchain + +curl http://127.0.0.1:26657/blockchain?minHeight=1&maxHeight=2 +``` + +##### JSONRPC + +```sh +curl -X POST https://localhost:26657 -d "{\"jsonrpc\":\"2.0\",\"id\":1,\"method\":\"blockchain\",\"params\":{\"minHeight\":\"1\", \"maxHeight\":\"2\"}}" +``` + +#### Response + +```json +{ + "id": 0, + "jsonrpc": "2.0", + "result": { + "last_height": "1276718", + "block_metas": [ + { + "block_id": { + "hash": "112BC173FD838FB68EB43476816CD7B4C6661B6884A9E357B417EE957E1CF8F7", + "parts": { + "total": 1, + "hash": "38D4B26B5B725C4F13571EFE022C030390E4C33C8CF6F88EDD142EA769642DBD" + } + }, + "block_size": 1000000, + "header": { + "version": { + "block": "10", + "app": "0" + }, + "chain_id": "cosmoshub-2", + "height": "12", + "time": "2019-04-22T17:01:51.701356223Z", + "last_block_id": { + "hash": "112BC173FD838FB68EB43476816CD7B4C6661B6884A9E357B417EE957E1CF8F7", + "parts": { + "total": 1, + "hash": "38D4B26B5B725C4F13571EFE022C030390E4C33C8CF6F88EDD142EA769642DBD" + } + }, + "last_commit_hash": "21B9BC845AD2CB2C4193CDD17BFC506F1EBE5A7402E84AD96E64171287A34812", + "data_hash": "970886F99E77ED0D60DA8FCE0447C2676E59F2F77302B0C4AA10E1D02F18EF73", + "validators_hash": "D658BFD100CA8025CFD3BECFE86194322731D387286FBD26E059115FD5F2BCA0", + "next_validators_hash": "D658BFD100CA8025CFD3BECFE86194322731D387286FBD26E059115FD5F2BCA0", + "consensus_hash": "0F2908883A105C793B74495EB7D6DF2EEA479ED7FC9349206A65CB0F9987A0B8", + "app_hash": "223BF64D4A01074DC523A80E76B9BBC786C791FB0A1893AC5B14866356FCFD6C", + "last_results_hash": "", + "evidence_hash": "", + "proposer_address": "D540AB022088612AC74B287D076DBFBC4A377A2E" + }, + "num_txs": "54" + } + ] + } +} +``` + +### Block + +Get block at a specified height. + +#### Parameters + +- `height (integer)`: height of the requested block. If no height is specified the latest block will be used. + +#### Request + +##### HTTP + +```sh +curl http://127.0.0.1:26657/block + +curl http://127.0.0.1:26657/block?height=1 +``` + +##### JSONRPC + +```sh +curl -X POST https://localhost:26657 -d "{\"jsonrpc\":\"2.0\",\"id\":1,\"method\":\"block\",\"params\":{\"height\":\"1\"}}" +``` + +#### Response + +```json +{ + "id": 0, + "jsonrpc": "2.0", + "result": { + "block_id": { + "hash": "112BC173FD838FB68EB43476816CD7B4C6661B6884A9E357B417EE957E1CF8F7", + "parts": { + "total": 1, + "hash": "38D4B26B5B725C4F13571EFE022C030390E4C33C8CF6F88EDD142EA769642DBD" + } + }, + "block": { + "header": { + "version": { + "block": "10", + "app": "0" + }, + "chain_id": "cosmoshub-2", + "height": "12", + "time": "2019-04-22T17:01:51.701356223Z", + "last_block_id": { + "hash": "112BC173FD838FB68EB43476816CD7B4C6661B6884A9E357B417EE957E1CF8F7", + "parts": { + "total": 1, + "hash": "38D4B26B5B725C4F13571EFE022C030390E4C33C8CF6F88EDD142EA769642DBD" + } + }, + "last_commit_hash": "21B9BC845AD2CB2C4193CDD17BFC506F1EBE5A7402E84AD96E64171287A34812", + "data_hash": "970886F99E77ED0D60DA8FCE0447C2676E59F2F77302B0C4AA10E1D02F18EF73", + "validators_hash": "D658BFD100CA8025CFD3BECFE86194322731D387286FBD26E059115FD5F2BCA0", + "next_validators_hash": "D658BFD100CA8025CFD3BECFE86194322731D387286FBD26E059115FD5F2BCA0", + "consensus_hash": "0F2908883A105C793B74495EB7D6DF2EEA479ED7FC9349206A65CB0F9987A0B8", + "app_hash": "223BF64D4A01074DC523A80E76B9BBC786C791FB0A1893AC5B14866356FCFD6C", + "last_results_hash": "", + "evidence_hash": "", + "proposer_address": "D540AB022088612AC74B287D076DBFBC4A377A2E" + }, + "data": [ + "yQHwYl3uCkKoo2GaChRnd+THLQ2RM87nEZrE19910Z28ABIUWW/t8AtIMwcyU0sT32RcMDI9GF0aEAoFdWF0b20SBzEwMDAwMDASEwoNCgV1YXRvbRIEMzEwMRCd8gEaagom61rphyEDoJPxlcjRoNDtZ9xMdvs+lRzFaHe2dl2P5R2yVCWrsHISQKkqX5H1zXAIJuC57yw0Yb03Fwy75VRip0ZBtLiYsUqkOsPUoQZAhDNP+6LY+RUwz/nVzedkF0S29NZ32QXdGv0=" + ], + "evidence": [ + { + "type": "string", + "height": 0, + "time": 0, + "total_voting_power": 0, + "validator": { + "pub_key": { + "type": "tendermint/PubKeyEd25519", + "value": "A6DoBUypNtUAyEHWtQ9bFjfNg8Bo9CrnkUGl6k6OHN4=" + }, + "voting_power": 0, + "address": "string" + } + } + ], + "last_commit": { + "height": 0, + "round": 0, + "block_id": { + "hash": "112BC173FD838FB68EB43476816CD7B4C6661B6884A9E357B417EE957E1CF8F7", + "parts": { + "total": 1, + "hash": "38D4B26B5B725C4F13571EFE022C030390E4C33C8CF6F88EDD142EA769642DBD" + } + }, + "signatures": [ + { + "type": 2, + "height": "1262085", + "round": 0, + "block_id": { + "hash": "112BC173FD838FB68EB43476816CD7B4C6661B6884A9E357B417EE957E1CF8F7", + "parts": { + "total": 1, + "hash": "38D4B26B5B725C4F13571EFE022C030390E4C33C8CF6F88EDD142EA769642DBD" + } + }, + "timestamp": "2019-08-01T11:39:38.867269833Z", + "validator_address": "000001E443FD237E4B616E2FA69DF4EE3D49A94F", + "validator_index": 0, + "signature": "DBchvucTzAUEJnGYpNvMdqLhBAHG4Px8BsOBB3J3mAFCLGeuG7uJqy+nVngKzZdPhPi8RhmE/xcw/M9DOJjEDg==" + } + ] + } + } + } +} +``` + +### BlockByHash + +#### Parameters + +- `hash (string)`: Hash of the block to query for. + +#### Request + +##### HTTP + +```sh +curl http://127.0.0.1:26657/block_by_hash?hash=0xD70952032620CC4E2737EB8AC379806359D8E0B17B0488F627997A0B043ABDED +``` + +##### JSONRPC + +```sh +curl -X POST https://localhost:26657 -d "{\"jsonrpc\":\"2.0\",\"id\":1,\"method\":\"block_by_hash\",\"params\":{\"hash\":\"0xD70952032620CC4E2737EB8AC379806359D8E0B17B0488F627997A0B043ABDED\"}}" +``` + +#### Response + +```json +{ + "id": 0, + "jsonrpc": "2.0", + "result": { + "block_id": { + "hash": "112BC173FD838FB68EB43476816CD7B4C6661B6884A9E357B417EE957E1CF8F7", + "parts": { + "total": 1, + "hash": "38D4B26B5B725C4F13571EFE022C030390E4C33C8CF6F88EDD142EA769642DBD" + } + }, + "block": { + "header": { + "version": { + "block": "10", + "app": "0" + }, + "chain_id": "cosmoshub-2", + "height": "12", + "time": "2019-04-22T17:01:51.701356223Z", + "last_block_id": { + "hash": "112BC173FD838FB68EB43476816CD7B4C6661B6884A9E357B417EE957E1CF8F7", + "parts": { + "total": 1, + "hash": "38D4B26B5B725C4F13571EFE022C030390E4C33C8CF6F88EDD142EA769642DBD" + } + }, + "last_commit_hash": "21B9BC845AD2CB2C4193CDD17BFC506F1EBE5A7402E84AD96E64171287A34812", + "data_hash": "970886F99E77ED0D60DA8FCE0447C2676E59F2F77302B0C4AA10E1D02F18EF73", + "validators_hash": "D658BFD100CA8025CFD3BECFE86194322731D387286FBD26E059115FD5F2BCA0", + "next_validators_hash": "D658BFD100CA8025CFD3BECFE86194322731D387286FBD26E059115FD5F2BCA0", + "consensus_hash": "0F2908883A105C793B74495EB7D6DF2EEA479ED7FC9349206A65CB0F9987A0B8", + "app_hash": "223BF64D4A01074DC523A80E76B9BBC786C791FB0A1893AC5B14866356FCFD6C", + "last_results_hash": "", + "evidence_hash": "", + "proposer_address": "D540AB022088612AC74B287D076DBFBC4A377A2E" + }, + "data": [ + "yQHwYl3uCkKoo2GaChRnd+THLQ2RM87nEZrE19910Z28ABIUWW/t8AtIMwcyU0sT32RcMDI9GF0aEAoFdWF0b20SBzEwMDAwMDASEwoNCgV1YXRvbRIEMzEwMRCd8gEaagom61rphyEDoJPxlcjRoNDtZ9xMdvs+lRzFaHe2dl2P5R2yVCWrsHISQKkqX5H1zXAIJuC57yw0Yb03Fwy75VRip0ZBtLiYsUqkOsPUoQZAhDNP+6LY+RUwz/nVzedkF0S29NZ32QXdGv0=" + ], + "evidence": [ + { + "type": "string", + "height": 0, + "time": 0, + "total_voting_power": 0, + "validator": { + "pub_key": { + "type": "tendermint/PubKeyEd25519", + "value": "A6DoBUypNtUAyEHWtQ9bFjfNg8Bo9CrnkUGl6k6OHN4=" + }, + "voting_power": 0, + "address": "string" + } + } + ], + "last_commit": { + "height": 0, + "round": 0, + "block_id": { + "hash": "112BC173FD838FB68EB43476816CD7B4C6661B6884A9E357B417EE957E1CF8F7", + "parts": { + "total": 1, + "hash": "38D4B26B5B725C4F13571EFE022C030390E4C33C8CF6F88EDD142EA769642DBD" + } + }, + "signatures": [ + { + "type": 2, + "height": "1262085", + "round": 0, + "block_id": { + "hash": "112BC173FD838FB68EB43476816CD7B4C6661B6884A9E357B417EE957E1CF8F7", + "parts": { + "total": 1, + "hash": "38D4B26B5B725C4F13571EFE022C030390E4C33C8CF6F88EDD142EA769642DBD" + } + }, + "timestamp": "2019-08-01T11:39:38.867269833Z", + "validator_address": "000001E443FD237E4B616E2FA69DF4EE3D49A94F", + "validator_index": 0, + "signature": "DBchvucTzAUEJnGYpNvMdqLhBAHG4Px8BsOBB3J3mAFCLGeuG7uJqy+nVngKzZdPhPi8RhmE/xcw/M9DOJjEDg==" + } + ] + } + } + } +} +``` + +### BlockResults + +### Parameters + +- `height (integer)`: Height of the block which contains the results. If no height is specified, the latest block height will be used + +#### Request + +##### HTTP + +```sh +curl http://127.0.0.1:26657/block_results + + +curl http://127.0.0.1:26657/block_results?height=1 +``` + +##### JSONRPC + +```sh +curl -X POST https://localhost:26657 -d "{\"jsonrpc\":\"2.0\",\"id\":1,\"method\":\"block_results\",\"params\":{\"height\":\"1\"}}" +``` + +#### Response + +```json +{ + "jsonrpc": "2.0", + "id": 0, + "result": { + "height": "12", + "total_gas_used": "100", + "txs_results": [ + { + "code": "0", + "data": "", + "log": "not enough gas", + "info": "", + "gas_wanted": "100", + "gas_used": "100", + "events": [ + { + "type": "app", + "attributes": [ + { + "key": "YWN0aW9u", + "value": "c2VuZA==", + "index": false + } + ] + } + ], + "codespace": "ibc" + } + ], + "begin_block_events": [ + { + "type": "app", + "attributes": [ + { + "key": "YWN0aW9u", + "value": "c2VuZA==", + "index": false + } + ] + } + ], + "end_block": [ + { + "type": "app", + "attributes": [ + { + "key": "YWN0aW9u", + "value": "c2VuZA==", + "index": false + } + ] + } + ], + "validator_updates": [ + { + "pub_key": { + "type": "tendermint/PubKeyEd25519", + "value": "9tK9IT+FPdf2qm+5c2qaxi10sWP+3erWTKgftn2PaQM=" + }, + "power": "300" + } + ], + "consensus_params_updates": { + "block": { + "max_bytes": "22020096", + "max_gas": "1000", + "time_iota_ms": "1000" + }, + "evidence": { + "max_age": "100000" + }, + "validator": { + "pub_key_types": [ + "ed25519" + ] + } + } + } +} +``` + +### Commit + +#### Parameters + +- `height (integer)`: Height of the block the requested commit pertains to. If no height is set the latest commit will be returned. + +#### Request + +##### HTTP + +```sh +curl http://127.0.0.1:26657/commit + + +curl http://127.0.0.1:26657/commit?height=1 +``` + +##### JSONRPC + +```sh +curl -X POST https://localhost:26657 -d "{\"jsonrpc\":\"2.0\",\"id\":1,\"method\":\"commit\",\"params\":{\"height\":\"1\"}}" +``` + +#### Response + +```json +{ + "jsonrpc": "2.0", + "id": 0, + "result": { + "signed_header": { + "header": { + "version": { + "block": "10", + "app": "0" + }, + "chain_id": "cosmoshub-2", + "height": "12", + "time": "2019-04-22T17:01:51.701356223Z", + "last_block_id": { + "hash": "112BC173FD838FB68EB43476816CD7B4C6661B6884A9E357B417EE957E1CF8F7", + "parts": { + "total": 1, + "hash": "38D4B26B5B725C4F13571EFE022C030390E4C33C8CF6F88EDD142EA769642DBD" + } + }, + "last_commit_hash": "21B9BC845AD2CB2C4193CDD17BFC506F1EBE5A7402E84AD96E64171287A34812", + "data_hash": "970886F99E77ED0D60DA8FCE0447C2676E59F2F77302B0C4AA10E1D02F18EF73", + "validators_hash": "D658BFD100CA8025CFD3BECFE86194322731D387286FBD26E059115FD5F2BCA0", + "next_validators_hash": "D658BFD100CA8025CFD3BECFE86194322731D387286FBD26E059115FD5F2BCA0", + "consensus_hash": "0F2908883A105C793B74495EB7D6DF2EEA479ED7FC9349206A65CB0F9987A0B8", + "app_hash": "223BF64D4A01074DC523A80E76B9BBC786C791FB0A1893AC5B14866356FCFD6C", + "last_results_hash": "", + "evidence_hash": "", + "proposer_address": "D540AB022088612AC74B287D076DBFBC4A377A2E" + }, + "commit": { + "height": "1311801", + "round": 0, + "block_id": { + "hash": "112BC173FD838FB68EB43476816CD7B4C6661B6884A9E357B417EE957E1CF8F7", + "parts": { + "total": 1, + "hash": "38D4B26B5B725C4F13571EFE022C030390E4C33C8CF6F88EDD142EA769642DBD" + } + }, + "signatures": [ + { + "block_id_flag": 2, + "validator_address": "000001E443FD237E4B616E2FA69DF4EE3D49A94F", + "timestamp": "2019-04-22T17:01:58.376629719Z", + "signature": "14jaTQXYRt8kbLKEhdHq7AXycrFImiLuZx50uOjs2+Zv+2i7RTG/jnObD07Jo2ubZ8xd7bNBJMqkgtkd0oQHAw==" + } + ] + } + }, + "canonical": true + } +} +``` + +### Validators + +#### Parameters + +- `height (integer)`: Block height at which the validators were present on. If no height is set the latest commit will be returned. +- `page (integer)`: +- `per_page (integer)`: + +#### Request + +##### HTTP + +```sh +curl http://127.0.0.1:26657/validators +``` + +##### JSONRPC + +```sh +curl -X POST https://localhost:26657 -d "{\"jsonrpc\":\"2.0\",\"id\":1,\"method\":\"validators\",\"params\":{\"height\":\"1\", \"page\":\"1\", \"per_page\":\"20\"}}" +``` + +#### Response + +```json +{ + "jsonrpc": "2.0", + "id": 0, + "result": { + "block_height": "55", + "validators": [ + { + "address": "000001E443FD237E4B616E2FA69DF4EE3D49A94F", + "pub_key": { + "type": "tendermint/PubKeyEd25519", + "value": "9tK9IT+FPdf2qm+5c2qaxi10sWP+3erWTKgftn2PaQM=" + }, + "voting_power": "239727", + "proposer_priority": "-11896414" + } + ], + "count": "1", + "total": "25" + } +} +``` + +### Genesis + +Get Genesis of the chain. If the response is large, this operation +will return an error: use `genesis_chunked` instead. + +#### Request + +##### HTTP + +```sh +curl http://127.0.0.1:26657/genesis +``` + +##### JSONRPC + +```sh +curl -X POST https://localhost:26657 -d "{\"jsonrpc\":\"2.0\",\"id\":1,\"method\":\"genesis\"}" +``` + +#### Response + +```json +{ + "jsonrpc": "2.0", + "id": 0, + "result": { + "genesis": { + "genesis_time": "2019-04-22T17:00:00Z", + "chain_id": "cosmoshub-2", + "initial_height": "2", + "consensus_params": { + "block": { + "max_bytes": "22020096", + "max_gas": "1000", + "time_iota_ms": "1000" + }, + "evidence": { + "max_age": "100000" + }, + "validator": { + "pub_key_types": [ + "ed25519" + ] + } + }, + "validators": [ + { + "address": "B00A6323737F321EB0B8D59C6FD497A14B60938A", + "pub_key": { + "type": "tendermint/PubKeyEd25519", + "value": "cOQZvh/h9ZioSeUMZB/1Vy1Xo5x2sjrVjlE/qHnYifM=" + }, + "power": "9328525", + "name": "Certus One" + } + ], + "app_hash": "", + "app_state": {} + } + } +} +``` + +### GenesisChunked + +Get the genesis document in a chunks to support easily transfering larger documents. + +#### Parameters + +- `chunk` (integer): the index number of the chunk that you wish to + fetch. These IDs are 0 indexed. + +#### Request + +##### HTTP + +```sh +curl http://127.0.0.1:26657/genesis_chunked?chunk=0 +``` + +##### JSONRPC + +```sh +curl -X POST https://localhost:26657 -d "{\"jsonrpc\":\"2.0\",\"id\":1,\"method\":\"genesis_chunked\",\"params\":{\"chunk\":0}}" +``` + +#### Response + +```json +{ + "jsonrpc": "2.0", + "id": 0, + "result": { + "chunk": 0, + "total": 10, + "data": "dGVuZGVybWludAo=" + } +} +``` + +### ConsensusParams + +Get the consensus parameters. + +#### Parameters + +- `height (integer)`: Block height at which the consensus params would like to be fetched for. + +#### Request + +##### HTTP + +```sh +curl http://127.0.0.1:26657/consensus_params +``` + +##### JSONRPC + +```sh +curl -X POST https://localhost:26657 -d "{\"jsonrpc\":\"2.0\",\"id\":1,\"method\":\"consensus_params\"}" +``` + +#### Response + +```json +{ + "jsonrpc": "2.0", + "id": 0, + "result": { + "block_height": "1", + "consensus_params": { + "block": { + "max_bytes": "22020096", + "max_gas": "1000", + "time_iota_ms": "1000" + }, + "evidence": { + "max_age": "100000" + }, + "validator": { + "pub_key_types": [ + "ed25519" + ] + } + } + } +} +``` + +### UnconfirmedTxs + +Get a list of unconfirmed transactions. + +#### Parameters + +- `limit (integer)` The amount of txs to respond with. + +#### Request + +##### HTTP + +```sh +curl http://127.0.0.1:26657/unconfirmed_txs +``` + +##### JSONRPC + +```sh +curl -X POST https://localhost:26657 -d "{\"jsonrpc\":\"2.0\",\"id\":1,\"method\":\"unconfirmed_txs\, \"params\":{\"limit\":\"20\"}}" +``` + +#### Response + +```json +{ + "jsonrpc": "2.0", + "id": 0, + "result": { + "n_txs": "82", + "total": "82", + "total_bytes": "19974", + "txs": [ + "gAPwYl3uCjCMTXENChSMnIkb5ZpYHBKIZqecFEV2tuZr7xIUA75/FmYq9WymsOBJ0XSJ8yV8zmQKMIxNcQ0KFIyciRvlmlgcEohmp5wURXa25mvvEhQbrvwbvlNiT+Yjr86G+YQNx7kRVgowjE1xDQoUjJyJG+WaWBwSiGannBRFdrbma+8SFK2m+1oxgILuQLO55n8mWfnbIzyPCjCMTXENChSMnIkb5ZpYHBKIZqecFEV2tuZr7xIUQNGfkmhTNMis4j+dyMDIWXdIPiYKMIxNcQ0KFIyciRvlmlgcEohmp5wURXa25mvvEhS8sL0D0wwgGCItQwVowak5YB38KRIUCg4KBXVhdG9tEgUxMDA1NBDoxRgaagom61rphyECn8x7emhhKdRCB2io7aS/6Cpuq5NbVqbODmqOT3jWw6kSQKUresk+d+Gw0BhjiggTsu8+1voW+VlDCQ1GRYnMaFOHXhyFv7BCLhFWxLxHSAYT8a5XqoMayosZf9mANKdXArA=" + ] + } +} +``` + +### NumUnconfirmedTxs + +Get data about unconfirmed transactions. + +#### Parameters + +None + +#### Request + +##### HTTP + +```sh +curl http://127.0.0.1:26657/num_unconfirmed_txs +``` + +##### JSONRPC + +```sh +curl -X POST https://localhost:26657 -d "{\"jsonrpc\":\"2.0\",\"id\":1,\"method\":\"num_unconfirmed_txs\"}" +``` + +#### Response + +```json +{ + "jsonrpc": "2.0", + "id": 0, + "result": { + "n_txs": "31", + "total": "82", + "total_bytes": "19974" + } +} +``` + +### Tx + +#### Parameters + +- `hash (string)`: The hash of the transaction +- `prove (bool)`: If the response should include proof the transaction was included in a block. + +#### Request + +##### HTTP + +```sh +curl http://127.0.0.1:26657/num_unconfirmed_txs +``` + +##### JSONRPC + +```sh +curl -X POST https://localhost:26657 -d "{\"jsonrpc\":\"2.0\",\"id\":1,\"method\":\"num_unconfirmed_txs\"}" +``` + +#### Response + +```json +{ + "jsonrpc": "2.0", + "id": 0, + "result": { + "hash": "D70952032620CC4E2737EB8AC379806359D8E0B17B0488F627997A0B043ABDED", + "height": "1000", + "index": 0, + "tx_result": { + "log": "[{\"msg_index\":\"0\",\"success\":true,\"log\":\"\"}]", + "gas_wanted": "200000", + "gas_used": "28596", + "tags": [ + { + "key": "YWN0aW9u", + "value": "c2VuZA==", + "index": false + } + ] + }, + "tx": "5wHwYl3uCkaoo2GaChQmSIu8hxpJxLcCuIi8fiHN4TMwrRIU/Af1cEG7Rcs/6LjTl7YjRSymJfYaFAoFdWF0b20SCzE0OTk5OTk1MDAwEhMKDQoFdWF0b20SBDUwMDAQwJoMGmoKJuta6YchAwswBShaB1wkZBctLIhYqBC3JrAI28XGzxP+rVEticGEEkAc+khTkKL9CDE47aDvjEHvUNt+izJfT4KVF2v2JkC+bmlH9K08q3PqHeMI9Z5up+XMusnTqlP985KF+SI5J3ZOIhhNYWRlIGJ5IENpcmNsZSB3aXRoIGxvdmU=" + } +} +``` + +## Transaction Routes + +### BroadCastTxSync + +Returns with the response from CheckTx. Does not wait for DeliverTx result. + +#### Parameters + +- `tx (string)`: The transaction encoded + +#### Request + +##### HTTP + +```sh +curl http://127.0.0.1:26657/broadcast_tx_sync?tx=encoded_tx +``` + +##### JSONRPC + +```sh +curl -X POST https://localhost:26657 -d "{\"jsonrpc\":\"2.0\",\"id\":1,\"method\":\"broadcast_tx_sync\",\"params\":{\"tx\":\"a/encoded_tx/c\"}}" +``` + +#### Response + +```json +{ + "jsonrpc": "2.0", + "id": 0, + "result": { + "code": "0", + "data": "", + "log": "", + "codespace": "ibc", + "hash": "0D33F2F03A5234F38706E43004489E061AC40A2E" + }, + "error": "" +} +``` + +### BroadCastTxAsync + +Returns right away, with no response. Does not wait for CheckTx nor DeliverTx results. + +#### Parameters + +- `tx (string)`: The transaction encoded + +#### Request + +##### HTTP + +```sh +curl http://127.0.0.1:26657/broadcast_tx_async?tx=encoded_tx +``` + +##### JSONRPC + +```sh +curl -X POST https://localhost:26657 -d "{\"jsonrpc\":\"2.0\",\"id\":1,\"method\":\"broadcast_tx_async\",\"params\":{\"tx\":\"a/encoded_tx/c\"}}" +``` + +#### Response + +```json +{ + "jsonrpc": "2.0", + "id": 0, + "result": { + "code": "0", + "data": "", + "log": "", + "codespace": "ibc", + "hash": "0D33F2F03A5234F38706E43004489E061AC40A2E" + }, + "error": "" +} +``` + +### CheckTx + +Checks the transaction without executing it. + +#### Parameters + +- `tx (string)`: String of the encoded transaction + +#### Request + +##### HTTP + +```sh +curl http://127.0.0.1:26657/check_tx?tx=encoded_tx +``` + +##### JSONRPC + +```sh +curl -X POST https://localhost:26657 -d "{\"jsonrpc\":\"2.0\",\"id\":1,\"method\":\"check_tx\",\"params\":{\"tx\":\"a/encoded_tx/c\"}}" +``` + +#### Response + +```json +{ + "id": 0, + "jsonrpc": "2.0", + "error": "", + "result": { + "code": "0", + "data": "", + "log": "", + "info": "", + "gas_wanted": "1", + "gas_used": "0", + "events": [ + { + "type": "app", + "attributes": [ + { + "key": "YWN0aW9u", + "value": "c2VuZA==", + "index": false + } + ] + } + ], + "codespace": "bank" + } +} +``` + +## ABCI Routes + +### ABCIInfo + +Get some info about the application. + +#### Parameters + +None + +#### Request + +##### HTTP + +```sh +curl http://127.0.0.1:26657/abci_info +``` + +##### JSONRPC + +```sh +curl -X POST https://localhost:26657 -d "{\"jsonrpc\":\"2.0\",\"id\":1,\"method\":\"abci_info\"}" +``` + +#### Response + +```json +{ + "jsonrpc": "2.0", + "id": 0, + "result": { + "response": { + "data": "{\"size\":0}", + "version": "0.16.1", + "app_version": "1314126" + } + } +} +``` + +### ABCIQuery + +Query the application for some information. + +#### Parameters + +- `path (string)`: Path to the data. This is defined by the application. +- `data (string)`: The data requested +- `height (integer)`: Height at which the data is being requested for. +- `prove (bool)`: Include proofs of the transactions inclusion in the block + +#### Request + +##### HTTP + +```sh +curl http://127.0.0.1:26657/abci_query?path="a/b/c"=IHAVENOIDEA&height=1&prove=true +``` + +##### JSONRPC + +```sh +curl -X POST https://localhost:26657 -d "{\"jsonrpc\":\"2.0\",\"id\":1,\"method\":\"abci_query\",\"params\":{\"path\":\"a/b/c\", \"height\":\"1\", \"bool\":\"true\"}}" +``` + +#### Response + +```json +{ + "error": "", + "result": { + "response": { + "log": "exists", + "height": "0", + "proof": "010114FED0DAD959F36091AD761C922ABA3CBF1D8349990101020103011406AA2262E2F448242DF2C2607C3CDC705313EE3B0001149D16177BC71E445476174622EA559715C293740C", + "value": "61626364", + "key": "61626364", + "index": "-1", + "code": "0" + } + }, + "id": 0, + "jsonrpc": "2.0" +} +``` + +## Evidence Routes + +### BroadcastEvidence + +Broadcast evidence of the misbehavior. + +#### Parameters + +- `evidence (string)`: + +#### Request + +##### HTTP + +```sh +curl http://localhost:26657/broadcast_evidence?evidence=JSON_EVIDENCE_encoded +``` + +#### JSONRPC + +```sh +curl -X POST https://localhost:26657 -d "{\"jsonrpc\":\"2.0\",\"id\":1,\"method\":\"broadcast_evidence\",\"params\":{\"evidence\":\"JSON_EVIDENCE_encoded\"}}" +``` + +#### Response + +```json +{ + "error": "", + "result": "", + "id": 0, + "jsonrpc": "2.0" +} +``` diff --git a/docs.json b/docs.json index b93703093..a9905f0df 100644 --- a/docs.json +++ b/docs.json @@ -51,6 +51,26 @@ ] }, "redirects": [ + { + "source": "/sdk/next/upgrade/release", + "destination": "/sdk/next/upgrade/v0.55-release" + }, + { + "source": "/sdk/next/upgrade/upgrade", + "destination": "/sdk/next/upgrade/v0.55" + }, + { + "source": "/sdk/latest/upgrade/release", + "destination": "/sdk/latest/upgrade/v0.55-release" + }, + { + "source": "/sdk/latest/upgrade/upgrade", + "destination": "/sdk/latest/upgrade/v0.55" + }, + { + "source": "/sdk/next/modules/protocolpool/README", + "destination": "/sdk/next/modules/distribution/README" + }, { "source": "/enterprise/components/poa/overview", "destination": "/sdk/latest/enterprise/poa/overview" @@ -186,6 +206,58 @@ { "source": "/evm/next/documentation/getting-started/tooling-and-resources/development-environment", "destination": "/evm/latest/documentation/getting-started/tooling-and-resources" + }, + { + "source": "/sdk/latest/reference/rfc/rfc-template", + "destination": "/sdk/latest/reference/rfc" + }, + { + "source": "/sdk/next/reference/rfc/rfc-template", + "destination": "/sdk/next/reference/rfc" + }, + { + "source": "/sdk/v0.54/reference/rfc/rfc-template", + "destination": "/sdk/v0.54/reference/rfc" + }, + { + "source": "/sdk/v0.53/build/rfc/rfc-template", + "destination": "/sdk/v0.53/build/rfc" + }, + { + "source": "/sdk/v0.50/build/rfc/rfc-template", + "destination": "/sdk/v0.50/build/rfc" + }, + { + "source": "/sdk/v0.50/build/rfc/rfc/rfc-template", + "destination": "/sdk/v0.50/build/rfc" + }, + { + "source": "/sdk/v0.47/build/rfc/rfc-template", + "destination": "/sdk/v0.47/build/rfc" + }, + { + "source": "/sdk/latest/reference/architecture/adr-template", + "destination": "/sdk/latest/reference/architecture" + }, + { + "source": "/sdk/next/reference/architecture/adr-template", + "destination": "/sdk/next/reference/architecture" + }, + { + "source": "/sdk/v0.54/reference/architecture/adr-template", + "destination": "/sdk/v0.54/reference/architecture" + }, + { + "source": "/sdk/v0.53/build/architecture/adr-template", + "destination": "/sdk/v0.53/build/architecture" + }, + { + "source": "/sdk/v0.50/build/architecture/adr-template", + "destination": "/sdk/v0.50/build/architecture" + }, + { + "source": "/sdk/v0.47/build/architecture/adr-template", + "destination": "/sdk/v0.47/build/architecture" } ], "integrations": { @@ -305,7 +377,7 @@ "dropdown": "Cosmos SDK", "versions": [ { - "version": "v0.54", + "version": "v0.55", "tabs": [ { "tab": "Cosmos SDK", @@ -395,17 +467,53 @@ "sdk/latest/node/run-testnet", "sdk/latest/node/run-production" ] + }, + { + "group": "Key Rotation", + "pages": [ + "sdk/latest/keys/key-rotation", + "sdk/latest/keys/rotate-validator-key", + "sdk/latest/keys/rotate-validator-key-poa" + ] + }, + { + "group": "Post-quantum Keys", + "pages": [ + "sdk/latest/keys/post-quantum-keys", + "sdk/latest/keys/enable-ml-dsa-keys", + "sdk/latest/keys/migrate-validator-ml-dsa", + "sdk/latest/keys/create-ml-dsa-account" + ] + }, + { + "group": "Remote Signing: Cosmos-KMS", + "pages": [ + "sdk/latest/kms/remote-signing", + "sdk/latest/kms/tutorial-file-backend", + "sdk/latest/kms/configure-backend", + "sdk/latest/kms/rotate-key-remote-signer", + "sdk/latest/kms/migrate-from-tmkms", + "sdk/latest/kms/configuration-reference", + "sdk/latest/kms/best-practices" + ] } ] }, { - "group": "v0.54 Upgrade", + "group": "v0.55 Upgrade", "icon": "upload", "pages": [ - "sdk/latest/upgrade/release", - "sdk/latest/upgrade/upgrade", + "sdk/latest/upgrade/v0.55-release", + "sdk/latest/upgrade/v0.55", "sdk/latest/changelog/release-notes", - "sdk/latest/experimental/blockstm" + { + "group": "Previous versions", + "collapsed": true, + "pages": [ + "sdk/latest/upgrade/v0.54-release", + "sdk/latest/upgrade/v0.54" + ] + } ] }, { @@ -425,7 +533,8 @@ "pages": [ "sdk/latest/guides/abci/abci", "sdk/latest/guides/abci/app-mempool", - "sdk/latest/guides/abci/vote-extensions" + "sdk/latest/guides/abci/vote-extensions", + "sdk/latest/experimental/blockstm" ] }, { @@ -501,7 +610,6 @@ "sdk/latest/modules/mint/README", "sdk/latest/modules/nft/README", "sdk/latest/modules/params/README", - "sdk/latest/modules/protocolpool/README", "sdk/latest/modules/slashing/README", "sdk/latest/modules/staking/README", "sdk/latest/modules/upgrade/README", @@ -613,8 +721,7 @@ "pages": [ "sdk/latest/reference/rfc/README", "sdk/latest/reference/rfc/PROCESS", - "sdk/latest/reference/rfc/rfc-001-tx-validation", - "sdk/latest/reference/rfc/rfc-template" + "sdk/latest/reference/rfc/rfc-001-tx-validation" ] }, { @@ -647,7 +754,7 @@ { "tab": "Release Notes", "pages": [ - "sdk/latest/upgrade/release" + "sdk/latest/upgrade/v0.55-release" ] } ], @@ -745,17 +852,53 @@ "sdk/next/node/run-testnet", "sdk/next/node/run-production" ] + }, + { + "group": "Key Rotation", + "pages": [ + "sdk/next/keys/key-rotation", + "sdk/next/keys/rotate-validator-key", + "sdk/next/keys/rotate-validator-key-poa" + ] + }, + { + "group": "Post-quantum Keys", + "pages": [ + "sdk/next/keys/post-quantum-keys", + "sdk/next/keys/enable-ml-dsa-keys", + "sdk/next/keys/migrate-validator-ml-dsa", + "sdk/next/keys/create-ml-dsa-account" + ] + }, + { + "group": "Remote Signing: Cosmos-KMS", + "pages": [ + "sdk/next/kms/remote-signing", + "sdk/next/kms/tutorial-file-backend", + "sdk/next/kms/configure-backend", + "sdk/next/kms/rotate-key-remote-signer", + "sdk/next/kms/migrate-from-tmkms", + "sdk/next/kms/configuration-reference", + "sdk/next/kms/best-practices" + ] } ] }, { - "group": "v0.54 Upgrade", + "group": "v0.55 Upgrade", "icon": "upload", "pages": [ - "sdk/next/upgrade/release", - "sdk/next/upgrade/upgrade", + "sdk/next/upgrade/v0.55-release", + "sdk/next/upgrade/v0.55", "sdk/next/changelog/release-notes", - "sdk/next/experimental/blockstm" + { + "group": "Previous versions", + "collapsed": true, + "pages": [ + "sdk/next/upgrade/v0.54-release", + "sdk/next/upgrade/v0.54" + ] + } ] }, { @@ -775,7 +918,8 @@ "pages": [ "sdk/next/guides/abci/abci", "sdk/next/guides/abci/app-mempool", - "sdk/next/guides/abci/vote-extensions" + "sdk/next/guides/abci/vote-extensions", + "sdk/next/experimental/blockstm" ] }, { @@ -851,7 +995,6 @@ "sdk/next/modules/mint/README", "sdk/next/modules/nft/README", "sdk/next/modules/params/README", - "sdk/next/modules/protocolpool/README", "sdk/next/modules/slashing/README", "sdk/next/modules/staking/README", "sdk/next/modules/upgrade/README", @@ -963,8 +1106,7 @@ "pages": [ "sdk/next/reference/rfc/README", "sdk/next/reference/rfc/PROCESS", - "sdk/next/reference/rfc/rfc-001-tx-validation", - "sdk/next/reference/rfc/rfc-template" + "sdk/next/reference/rfc/rfc-001-tx-validation" ] }, { @@ -997,12 +1139,359 @@ { "tab": "Release Notes", "pages": [ - "sdk/next/upgrade/release" + "sdk/next/upgrade/v0.55-release" ] } ], "tag": "Unreleased" }, + { + "version": "v0.54", + "tabs": [ + { + "tab": "Cosmos SDK", + "groups": [ + { + "group": "Intro", + "icon": "earth", + "pages": [ + "sdk/v0.54/learn", + "sdk/v0.54/learn/start-here" + ] + }, + { + "group": "Overview", + "icon": "book-open", + "pages": [ + "sdk/v0.54/learn/intro/cosmos-stack", + "sdk/v0.54/learn/intro/overview", + "sdk/v0.54/learn/intro/blockchain-basics", + "sdk/v0.54/learn/intro/sdk-app-architecture" + ] + }, + { + "group": "Concepts", + "icon": "graduation-cap", + "pages": [ + { + "group": "Fundamentals", + "pages": [ + "sdk/v0.54/learn/concepts/accounts", + "sdk/v0.54/learn/concepts/transactions", + "sdk/v0.54/learn/concepts/lifecycle" + ] + }, + { + "group": "Modules", + "pages": [ + "sdk/v0.54/learn/concepts/modules", + "sdk/v0.54/learn/concepts/store", + "sdk/v0.54/learn/concepts/encoding", + "sdk/v0.54/learn/concepts/context-gas-events" + ] + }, + { + "group": "Cosmos SDK Internals", + "pages": [ + "sdk/v0.54/learn/concepts/sdk-structure", + "sdk/v0.54/learn/concepts/baseapp", + "sdk/v0.54/learn/concepts/app-go", + "sdk/v0.54/learn/concepts/cli-grpc-rest", + "sdk/v0.54/learn/concepts/testing" + ] + }, + "sdk/v0.54/release-family" + ] + }, + { + "group": "Build a Chain", + "icon": "hammer", + "pages": [ + "sdk/v0.54/tutorials/example/00-overview", + "sdk/v0.54/tutorials/example/01-prerequisites", + "sdk/v0.54/tutorials/example/02-quickstart", + "sdk/v0.54/tutorials/example/03-build-a-module", + "sdk/v0.54/tutorials/example/04-counter-walkthrough", + "sdk/v0.54/tutorials/example/05-run-and-test" + ] + }, + { + "group": "Run a Node", + "icon": "server", + "pages": [ + "sdk/v0.54/tutorials", + { + "group": "Get Started", + "pages": [ + "sdk/v0.54/node/prerequisites", + "sdk/v0.54/node/keyring", + "sdk/v0.54/node/run-node", + "sdk/v0.54/node/interact-node", + "sdk/v0.54/node/txs" + ] + }, + { + "group": "Testnet & Production", + "pages": [ + "sdk/v0.54/node/run-testnet", + "sdk/v0.54/node/run-production" + ] + } + ] + }, + { + "group": "v0.54 Upgrade", + "icon": "upload", + "pages": [ + "sdk/v0.54/upgrade/release", + "sdk/v0.54/upgrade/upgrade", + "sdk/v0.54/changelog/release-notes", + "sdk/v0.54/experimental/blockstm" + ] + }, + { + "group": "In-depth Guides", + "icon": "map", + "pages": [ + "sdk/v0.54/guides/guides", + { + "group": "Module Design", + "pages": [ + "sdk/v0.54/guides/module-design/module-design-considerations", + "sdk/v0.54/guides/module-design/ocap" + ] + }, + { + "group": "ABCI", + "pages": [ + "sdk/v0.54/guides/abci/abci", + "sdk/v0.54/guides/abci/app-mempool", + "sdk/v0.54/guides/abci/vote-extensions" + ] + }, + { + "group": "Tooling & CLI", + "pages": [ + "sdk/v0.54/guides/tooling/tool-guide", + "sdk/v0.54/guides/tooling/autocli", + "sdk/v0.54/guides/tooling/confix" + ] + }, + { + "group": "State", + "pages": [ + "sdk/v0.54/guides/state/store", + "sdk/v0.54/guides/state/collections" + ] + }, + { + "group": "Upgrades & Migrations", + "pages": [ + "sdk/v0.54/guides/upgrades/upgrade", + "sdk/v0.54/guides/upgrades/cosmovisor" + ] + }, + { + "group": "Testing and Observability", + "pages": [ + "sdk/v0.54/guides/testing/simulator", + "sdk/v0.54/guides/testing/telemetry", + "sdk/v0.54/guides/testing/log" + ] + } + ] + }, + { + "group": "Developer Reference", + "icon": "code", + "pages": [ + "sdk/v0.54/guides/reference/packages", + "sdk/v0.54/guides/reference/bech32", + "sdk/v0.54/guides/reference/protobuf-annotations", + "sdk/v0.54/guides/reference/proto-docs", + "sdk/v0.54/reference/cosmos-sdk-repo", + "sdk/v0.54/reference/example-repo" + ] + }, + { + "group": "Modules", + "icon": "puzzle", + "pages": [ + "sdk/v0.54/modules/modules", + { + "group": "Open-source Modules", + "pages": [ + { + "group": "x/auth", + "pages": [ + "sdk/v0.54/modules/auth/auth", + "sdk/v0.54/modules/auth/vesting", + "sdk/v0.54/modules/auth/tx" + ] + }, + "sdk/v0.54/modules/authz/README", + "sdk/v0.54/modules/bank/README", + "sdk/v0.54/modules/consensus/README", + "sdk/v0.54/modules/crisis/README", + "sdk/v0.54/modules/distribution/README", + "sdk/v0.54/modules/epochs/README", + "sdk/v0.54/modules/evidence/README", + "sdk/v0.54/modules/feegrant/README", + "sdk/v0.54/modules/gov/README", + "sdk/v0.54/modules/group/README", + "sdk/v0.54/modules/mint/README", + "sdk/v0.54/modules/nft/README", + "sdk/v0.54/modules/params/README", + "sdk/v0.54/modules/protocolpool/README", + "sdk/v0.54/modules/slashing/README", + "sdk/v0.54/modules/staking/README", + "sdk/v0.54/modules/upgrade/README", + "sdk/v0.54/modules/circuit/README", + "sdk/v0.54/modules/genutil/README" + ] + }, + { + "group": "Cosmos Enterprise Modules", + "pages": [ + "sdk/v0.54/enterprise/overview", + { + "group": "Permissioned Consensus", + "pages": [ + "sdk/v0.54/enterprise/poa/overview", + "sdk/v0.54/enterprise/poa/architecture", + "sdk/v0.54/enterprise/poa/governance", + "sdk/v0.54/enterprise/poa/api", + "sdk/v0.54/enterprise/poa/distribution" + ] + }, + { + "group": "Multi-sig", + "pages": [ + "sdk/v0.54/enterprise/group/overview", + "sdk/v0.54/enterprise/group/architecture", + "sdk/v0.54/enterprise/group/api" + ] + } + ] + } + ] + }, + { + "group": "Architecture Reference", + "icon": "building-2", + "pages": [ + { + "group": "ADRs", + "pages": [ + "sdk/v0.54/reference/architecture/README", + "sdk/v0.54/reference/architecture/PROCESS", + "sdk/v0.54/reference/architecture/adr-002-docs-structure", + "sdk/v0.54/reference/architecture/adr-003-dynamic-capability-store", + "sdk/v0.54/reference/architecture/adr-004-split-denomination-keys", + "sdk/v0.54/reference/architecture/adr-006-secret-store-replacement", + "sdk/v0.54/reference/architecture/adr-007-specialization-groups", + "sdk/v0.54/reference/architecture/adr-008-dCERT-group", + "sdk/v0.54/reference/architecture/adr-009-evidence-module", + "sdk/v0.54/reference/architecture/adr-010-modular-antehandler", + "sdk/v0.54/reference/architecture/adr-011-generalize-genesis-accounts", + "sdk/v0.54/reference/architecture/adr-012-state-accessors", + "sdk/v0.54/reference/architecture/adr-013-metrics", + "sdk/v0.54/reference/architecture/adr-014-proportional-slashing", + "sdk/v0.54/reference/architecture/adr-016-validator-consensus-key-rotation", + "sdk/v0.54/reference/architecture/adr-017-historical-header-module", + "sdk/v0.54/reference/architecture/adr-018-extendable-voting-period", + "sdk/v0.54/reference/architecture/adr-019-protobuf-state-encoding", + "sdk/v0.54/reference/architecture/adr-020-protobuf-transaction-encoding", + "sdk/v0.54/reference/architecture/adr-021-protobuf-query-encoding", + "sdk/v0.54/reference/architecture/adr-022-custom-panic-handling", + "sdk/v0.54/reference/architecture/adr-023-protobuf-naming", + "sdk/v0.54/reference/architecture/adr-024-coin-metadata", + "sdk/v0.54/reference/architecture/adr-027-deterministic-protobuf-serialization", + "sdk/v0.54/reference/architecture/adr-028-public-key-addresses", + "sdk/v0.54/reference/architecture/adr-029-fee-grant-module", + "sdk/v0.54/reference/architecture/adr-030-authz-module", + "sdk/v0.54/reference/architecture/adr-031-msg-service", + "sdk/v0.54/reference/architecture/adr-032-typed-events", + "sdk/v0.54/reference/architecture/adr-033-protobuf-inter-module-comm", + "sdk/v0.54/reference/architecture/adr-034-account-rekeying", + "sdk/v0.54/reference/architecture/adr-035-rosetta-api-support", + "sdk/v0.54/reference/architecture/adr-036-arbitrary-signature", + "sdk/v0.54/reference/architecture/adr-037-gov-split-vote", + "sdk/v0.54/reference/architecture/adr-038-state-listening", + "sdk/v0.54/reference/architecture/adr-039-epoched-staking", + "sdk/v0.54/reference/architecture/adr-040-storage-and-smt-state-commitments", + "sdk/v0.54/reference/architecture/adr-041-in-place-store-migrations", + "sdk/v0.54/reference/architecture/adr-042-group-module", + "sdk/v0.54/reference/architecture/adr-043-nft-module", + "sdk/v0.54/reference/architecture/adr-044-protobuf-updates-guidelines", + "sdk/v0.54/reference/architecture/adr-045-check-delivertx-middlewares", + "sdk/v0.54/reference/architecture/adr-046-module-params", + "sdk/v0.54/reference/architecture/adr-047-extend-upgrade-plan", + "sdk/v0.54/reference/architecture/adr-048-consensus-fees", + "sdk/v0.54/reference/architecture/adr-049-state-sync-hooks", + "sdk/v0.54/reference/architecture/adr-050-sign-mode-textual-annex1", + "sdk/v0.54/reference/architecture/adr-050-sign-mode-textual-annex2", + "sdk/v0.54/reference/architecture/adr-050-sign-mode-textual", + "sdk/v0.54/reference/architecture/adr-053-go-module-refactoring", + "sdk/v0.54/reference/architecture/adr-054-semver-compatible-modules", + "sdk/v0.54/reference/architecture/adr-055-orm", + "sdk/v0.54/reference/architecture/adr-057-app-wiring", + "sdk/v0.54/reference/architecture/adr-058-auto-generated-cli", + "sdk/v0.54/reference/architecture/adr-059-test-scopes", + "sdk/v0.54/reference/architecture/adr-060-abci-1.0", + "sdk/v0.54/reference/architecture/adr-061-liquid-staking", + "sdk/v0.54/reference/architecture/adr-062-collections-state-layer", + "sdk/v0.54/reference/architecture/adr-063-core-module-api", + "sdk/v0.54/reference/architecture/adr-064-abci-2.0", + "sdk/v0.54/reference/architecture/adr-065-store-v2", + "sdk/v0.54/reference/architecture/adr-068-preblock", + "sdk/v0.54/reference/architecture/adr-070-unordered-account", + "sdk/v0.54/reference/architecture/adr-076-tx-malleability" + ] + }, + { + "group": "RFCs", + "pages": [ + "sdk/v0.54/reference/rfc/README", + "sdk/v0.54/reference/rfc/PROCESS", + "sdk/v0.54/reference/rfc/rfc-001-tx-validation" + ] + }, + { + "group": "Specifications", + "pages": [ + "sdk/v0.54/reference/spec/README", + "sdk/v0.54/reference/spec/SPEC_MODULE", + "sdk/v0.54/reference/spec/SPEC_STANDARD" + ] + } + ] + }, + { + "group": "Security", + "icon": "shield", + "pages": [ + "sdk/v0.54/security/security-policy", + "sdk/v0.54/security/bug-bounty", + { + "group": "Audits", + "pages": [ + "sdk/v0.54/security/audits", + "sdk/v0.54/security/internal-audits" + ] + } + ] + } + ] + }, + { + "tab": "Release Notes", + "pages": [ + "sdk/v0.54/upgrade/release" + ] + } + ] + }, { "version": "v0.53", "tabs": [ @@ -1345,8 +1834,7 @@ "pages": [ "sdk/v0.53/build/rfc", "sdk/v0.53/build/rfc/PROCESS", - "sdk/v0.53/build/rfc/rfc-001-tx-validation", - "sdk/v0.53/build/rfc/rfc-template" + "sdk/v0.53/build/rfc/rfc-001-tx-validation" ] }, { @@ -1627,8 +2115,7 @@ "pages": [ "sdk/v0.50/build/rfc", "sdk/v0.50/build/rfc/PROCESS", - "sdk/v0.50/build/rfc/rfc-001-tx-validation", - "sdk/v0.50/build/rfc/rfc-template" + "sdk/v0.50/build/rfc/rfc-001-tx-validation" ] }, { @@ -1951,8 +2438,7 @@ "pages": [ "sdk/v0.47/build/rfc", "sdk/v0.47/build/rfc/PROCESS", - "sdk/v0.47/build/rfc/rfc-001-tx-validation", - "sdk/v0.47/build/rfc/rfc-template" + "sdk/v0.47/build/rfc/rfc-001-tx-validation" ] }, { @@ -6334,7 +6820,7 @@ "dropdown": "CometBFT", "versions": [ { - "version": "v0.39", + "version": "v0.40", "tabs": [ { "tab": "Learn", @@ -6764,6 +7250,220 @@ ], "tag": "Unreleased" }, + { + "version": "v0.39", + "tabs": [ + { + "tab": "Learn", + "groups": [ + { + "group": "CometBFT", + "pages": [ + "cometbft/v0.39/docs/README", + "cometbft/v0.39/docs/introduction/intro", + { + "group": "Guides", + "pages": [ + "cometbft/v0.39/docs/guides/Quick-Start", + "cometbft/v0.39/docs/guides/Install-CometBFT", + "cometbft/v0.39/docs/guides/Creating-an-application-in-Go", + "cometbft/v0.39/docs/guides/Creating-a-built-in-application-in-Go" + ] + }, + { + "group": "Apps", + "pages": [ + "cometbft/v0.39/docs/app-dev/Getting-Started", + "cometbft/v0.39/docs/app-dev/Application-Architecture-Guide", + "cometbft/v0.39/docs/app-dev/Using-ABCI-CLI", + "cometbft/v0.39/docs/app-dev/Indexing-Transactions" + ] + }, + { + "group": "Core", + "pages": [ + "cometbft/v0.39/docs/core/Using-CometBFT", + "cometbft/v0.39/docs/core/Running-in-production", + "cometbft/v0.39/docs/core/configuration", + "cometbft/v0.39/docs/core/mempool", + "cometbft/v0.39/docs/core/block-sync", + "cometbft/v0.39/docs/core/state-sync", + "cometbft/v0.39/docs/core/RPC", + "cometbft/v0.39/docs/core/Subscribing-to-events-via-Websocket", + "cometbft/v0.39/docs/core/metrics", + "cometbft/v0.39/docs/core/Validators", + "cometbft/v0.39/docs/core/light-client", + "cometbft/v0.39/docs/core/block-structure", + "cometbft/v0.39/docs/core/how-to-read-logs", + { + "group": "Experimental", + "pages": [ + "cometbft/v0.39/docs/experimental/lib-p2p" + ] + } + ] + }, + { + "group": "Developer Tools", + "pages": [ + "cometbft/v0.39/docs/tools/Overview", + "cometbft/v0.39/docs/tools/debugging" + ] + }, + { + "group": "Networks", + "pages": [ + "cometbft/v0.39/docs/networks/Docker-Compose" + ] + }, + { + "group": "CometBFT Quality Assurance", + "pages": [ + "cometbft/v0.39/docs/qa/CometBFT-qa", + "cometbft/v0.39/docs/qa/Method", + "cometbft/v0.39/docs/qa/CometBFT-QA-38", + "cometbft/v0.39/docs/qa/CometBFT-QA-37", + "cometbft/v0.39/docs/qa/CometBFT-QA-34", + "cometbft/v0.39/docs/qa/TMCore-QA-37", + "cometbft/v0.39/docs/qa/TMCore-QA-34" + ] + } + ] + } + ] + }, + { + "tab": "Specification", + "groups": [ + { + "group": "CometBFT Spec", + "pages": [ + "cometbft/v0.39/spec/CometBFT-Spec", + { + "group": "Core", + "pages": [ + "cometbft/v0.39/spec/core/Overview", + "cometbft/v0.39/spec/core/Data_structures", + "cometbft/v0.39/spec/core/encoding", + "cometbft/v0.39/spec/core/genesis", + "cometbft/v0.39/spec/core/state" + ] + }, + { + "group": "ABCI++", + "pages": [ + "cometbft/v0.39/spec/abci/Overview", + "cometbft/v0.39/spec/abci/Outline", + "cometbft/v0.39/spec/abci/Methods", + "cometbft/v0.39/spec/abci/Requirements-for-the-Application", + "cometbft/v0.39/spec/abci/CometBFTs-expected-behavior", + "cometbft/v0.39/spec/abci/Client-and-server", + "cometbft/v0.39/spec/abci/Introduction" + ] + }, + { + "group": "Consensus", + "pages": [ + "cometbft/v0.39/spec/consensus/Overview", + "cometbft/v0.39/spec/consensus/Consensus-Paper", + "cometbft/v0.39/spec/consensus/Byzantine-Consensus-Algorithm", + "cometbft/v0.39/spec/consensus/Light-Client", + "cometbft/v0.39/spec/consensus/Creating-Proposal", + "cometbft/v0.39/spec/consensus/BFT-Time", + "cometbft/v0.39/spec/consensus/Proposer-Selection", + "cometbft/v0.39/spec/consensus/Evidence", + "cometbft/v0.39/spec/consensus/Validator-Signing", + "cometbft/v0.39/spec/consensus/WAL" + ] + }, + { + "group": "Light Client", + "pages": [ + "cometbft/v0.39/spec/light-client/Light-Client-Specification", + "cometbft/v0.39/spec/light-client/verification", + "cometbft/v0.39/spec/light-client/Fork-Detection", + "cometbft/v0.39/spec/light-client/Accountability" + ] + }, + { + "group": "P2P", + "pages": [ + "cometbft/v0.39/spec/p2p/Peer-to-Peer", + "cometbft/v0.39/spec/p2p/Implementation-of-the-p2p-layer", + { + "group": "Legacy Docs", + "pages": [ + "cometbft/v0.39/spec/p2p/legacy-docs/Overview", + { + "group": "Messages", + "pages": [ + "cometbft/v0.39/spec/p2p/legacy-docs/messages/Overview", + "cometbft/v0.39/spec/p2p/legacy-docs/messages/block-sync", + "cometbft/v0.39/spec/p2p/legacy-docs/messages/evidence", + "cometbft/v0.39/spec/p2p/legacy-docs/messages/mempool", + "cometbft/v0.39/spec/p2p/legacy-docs/messages/state-sync", + "cometbft/v0.39/spec/p2p/legacy-docs/messages/Peer-Exchange", + "cometbft/v0.39/spec/p2p/legacy-docs/messages/consensus" + ] + }, + "cometbft/v0.39/spec/p2p/legacy-docs/P2P-Multiplex-Connection", + "cometbft/v0.39/spec/p2p/legacy-docs/Peers", + "cometbft/v0.39/spec/p2p/legacy-docs/P2P-Config", + "cometbft/v0.39/spec/p2p/legacy-docs/Peer-Discovery", + { + "group": "Reactors", + "pages": [ + "cometbft/v0.39/spec/p2p/reactor-api/Reactors", + "cometbft/v0.39/spec/p2p/reactor-api/Reactor-Api", + "cometbft/v0.39/spec/p2p/reactor-api/API-for-Reactors" + ] + } + ] + } + ] + }, + { + "group": "RPC", + "pages": [ + "cometbft/v0.39/spec/rpc/Rpc-Spe" + ] + }, + { + "group": "Blockchain", + "pages": [ + "cometbft/v0.39/spec/blockchain/Blockchain" + ] + } + ] + } + ] + }, + { + "tab": "API Reference", + "groups": [ + { + "group": "Overview", + "pages": [ + "cometbft/v0.39/api-reference/rpc/index" + ] + }, + { + "group": "RPC Methods", + "openapi": { + "source": "cometbft/v0.39/api-reference/rpc/openapi.yaml", + "directory": "cometbft/v0.39/api-reference/rpc" + } + } + ] + }, + { + "tab": "Changelog", + "pages": [ + "cometbft/v0.39/changelog/release-notes" + ] + } + ] + }, { "version": "v0.38", "tabs": [ diff --git a/evm/latest/documentation/concepts/accounts.mdx b/evm/latest/documentation/concepts/accounts.mdx index 6648014bf..8c6e19200 100644 --- a/evm/latest/documentation/concepts/accounts.mdx +++ b/evm/latest/documentation/concepts/accounts.mdx @@ -17,7 +17,7 @@ Cosmos blockchains support creating accounts with mnemonic phrases using [hierar ## EVM Accounts Cosmos EVM defines a custom `Account` type implementing an HD wallet compatible with Ethereum addresses using: -- Ethereum's ECDSA secp256k1 curve (`eth_secp265k1`) +- Ethereum's ECDSA secp256k1 curve (`eth_secp256k1`) - [EIP84](https://github.com/ethereum/EIPs/issues/84) for full [BIP44](https://github.com/bitcoin/bips/blob/master/bip-0044.mediawiki) paths - Root HD path: `m/44'/60'/0'/0` (Coin type `60` for Ethereum compatibility) @@ -84,6 +84,14 @@ cosmos1z3t55m0l9h0eupuz3dp5t5cypyv674jj7mz2jw - Address: `cosmosvalcons` prefix, 20 bytes - Pubkey: `cosmosvalconspub` prefix, 32 bytes +A chain can allow additional consensus key types through `consensus_params.validator.pub_key_types`: +- `secp256k1eth`: Ethereum-style validator consensus addresses, for chains that want consensus identities derived the Ethereum way. +- `ml_dsa_65`: post-quantum consensus keys. + +A Cosmos EVM validator can run any of these consensus signature types, including post-quantum ML-DSA. However, ML-DSA keys are not currently supported for EVM user accounts. As a fully EVM-compatible ledger, Cosmos EVM follows Ethereum's account conventions and will adopt user-side post-quantum support as it lands upstream. + +See [Enable ML-DSA keys](/sdk/latest/keys/enable-ml-dsa-keys) to allow a type and [Post-quantum keys](/sdk/latest/keys/post-quantum-keys) for the tradeoffs. Before enabling a new consensus key type on a chain with live IBC connections, confirm every counterparty runs a stack that can verify it. + ### Address Conversion Convert between formats using the CLI: diff --git a/evm/next/documentation/concepts/accounts.mdx b/evm/next/documentation/concepts/accounts.mdx index d33612fd5..16c752405 100644 --- a/evm/next/documentation/concepts/accounts.mdx +++ b/evm/next/documentation/concepts/accounts.mdx @@ -19,7 +19,7 @@ Cosmos blockchains support creating accounts with mnemonic phrases using [hierar ## EVM Accounts Cosmos EVM defines a custom `Account` type implementing an HD wallet compatible with Ethereum addresses using: -- Ethereum's ECDSA secp256k1 curve (`eth_secp265k1`) +- Ethereum's ECDSA secp256k1 curve (`eth_secp256k1`) - [EIP84](https://github.com/ethereum/EIPs/issues/84) for full [BIP44](https://github.com/bitcoin/bips/blob/master/bip-0044.mediawiki) paths - Root HD path: `m/44'/60'/0'/0` (Coin type `60` for Ethereum compatibility) @@ -86,6 +86,14 @@ cosmos1z3t55m0l9h0eupuz3dp5t5cypyv674jj7mz2jw - Address: `cosmosvalcons` prefix, 20 bytes - Pubkey: `cosmosvalconspub` prefix, 32 bytes +A chain can allow additional consensus key types through `consensus_params.validator.pub_key_types`: +- `secp256k1eth`: Ethereum-style validator consensus addresses, for chains that want consensus identities derived the Ethereum way. +- `ml_dsa_65`: post-quantum consensus keys. + +A Cosmos EVM validator can run any of these consensus signature types, including post-quantum ML-DSA. However, ML-DSA keys are not currently supported for EVM user accounts. As a fully EVM-compatible ledger, Cosmos EVM follows Ethereum's account conventions and will adopt user-side post-quantum support as it lands upstream. + +See [Enable ML-DSA keys](/sdk/latest/keys/enable-ml-dsa-keys) to allow a type and [Post-quantum keys](/sdk/latest/keys/post-quantum-keys) for the tradeoffs. Before enabling a new consensus key type on a chain with live IBC connections, confirm every counterparty runs a stack that can verify it. + ### Address Conversion Convert between formats using the CLI: diff --git a/scripts/versioning/CLAUDE.md b/scripts/versioning/CLAUDE.md index 35869383c..f0fc3646d 100644 --- a/scripts/versioning/CLAUDE.md +++ b/scripts/versioning/CLAUDE.md @@ -63,6 +63,103 @@ After every freeze, add a `latest` version entry to the product's dropdown in `d 3. Set `"version": ""`, `"tag": "Latest"`, `"default": true` 4. Give `next` the `"tag": "Unreleased"` field 5. Order: `latest [Latest, default]` → `next [Unreleased]` → stable archived newest-first +6. Check redirects to make sure they are redirected properly. + +### Manual checklist around every freeze + +The freeze script does not do any of the following. None of it is caught by `npx mint broken-links`. + +Order matters. Items 2 and 4 are content that lives in the pages themselves, so do them in `next/` **before** freezing and the promotion carries them into `latest/` for free. Doing them afterwards means editing `latest/` and then running `scripts/sync-latest-to-next.js` on every file touched, which is the same work twice with a chance to miss a file. Item 1 can only be done after, and item 3 must be done after because the freeze is what strips the front matter. + +**1. `docs.json` navigation** (after) — see the section above. + +**2. Version label in front matter (SDK only, do before).** Five pages carry the displayed version in their `description`, which is what renders as "Version: v0.54" under the page title. Bump all five in both `latest/` and `next/`: + +``` +sdk//learn.mdx +sdk//tutorials.mdx +sdk//reference/spec.mdx +sdk//reference/rfc.mdx +sdk//reference/architecture.mdx +``` + +```bash +grep -rn 'description: "Version: v' sdk/latest sdk/next --include='*.mdx' +``` + +Archived directories keep their own version and must not be touched. Only the SDK uses this pattern; the other products do not. + +**3. Changelog front matter (after).** `manage-changelogs.js` overwrites the whole file and does not preserve front matter, so a changelog regenerated after the freeze has already tagged it loses its `noindex` and `canonical`. Re-add them to `/next/changelog/release-notes.mdx` and re-run `tag-archived.js` for the archive directory. Verify with: + +```bash +for p in sdk cometbft; do for f in $(find $p/next -name '*.mdx'); do head -12 "$f" | grep -q noindex || echo "$f"; done; done +``` + +**4. GitHub links pinned to the previous version (do before).** Pages link into the product repo at a version-tracking ref, and those refs do not follow the freeze. After promoting SDK v0.55, `sdk/latest/` still held 96 links to `release/v0.54.x`; CometBFT at v0.40 still held 82 to `v0.38.x`. See the section below, because a blind find-and-replace is the wrong fix. + +### GitHub link refs — do not blind-replace + +Refs fall into three groups and only the first should ever be bumped: + +| Ref shape | Treatment | +| --------- | --------- | +| `release/v0..x` (SDK), `v0..x` (CometBFT) | Version-tracking. Bump on freeze. | +| `main`, `master` | Always current, never stale in the version sense, but line anchors silently rot as upstream moves. | +| Pinned tag (`v0.47.0-rc1`) or 40-char SHA | Deliberate historical citation. Leave alone. Bumping changes what the prose is referring to. | + +The hazard is line anchors (`#L127`). Bumping the ref while leaving the line number is worse than staying on the old ref, because the link keeps working while pointing at unrelated code. Measured on the v0.54 to v0.55 SDK freeze, of 39 line-anchored `release/v0.54.x` links: + +- 29 anchors still landed on the same line of code +- 7 had moved, with the original line findable uniquely elsewhere in the file +- 1 anchored line was deleted upstream, 1 file was deleted, 1 was ambiguous + +So roughly a quarter would have pointed somewhere wrong. CometBFT was worse, since it had drifted two minor versions: only 4 of 12 anchors survived. + +#### The check that works + +Per link: does the path exist at the new ref, then does the line number still mean the same thing. + +**Path existence.** Compare against the full file tree at each ref rather than probing URLs one at a time: + +```bash +curl -sL "https://api.github.com/repos/cosmos/cosmos-sdk/git/trees/release/v0.55.x?recursive=1" +``` + +Check the `truncated` field; it was `false` for both products at ~5,000 entries. Measured on this freeze, 93 of 96 SDK paths and 77 of 82 CometBFT paths survived the bump, so disappearance is the rare case. + +Two traps produce false positives here, and both bit on the first pass: + +- URL-decode the path first. `spec/abci/abci%2B%2B_methods.md` is `abci++_methods.md` in the tree and resolves fine. +- A `/blob/` URL pointing at a directory is valid. GitHub redirects it to `/tree/`, returning 200. + +**Line numbers.** Use git, not content matching. Content equality cannot tell a moved line from a coincidentally identical one, which is how a `query.proto#L108` link produced six candidate lines. Git resolves it deterministically: + +```bash +git init -q --bare refcheck && cd refcheck +git remote add origin https://github.com/cosmos/cosmos-sdk.git +git fetch -q --depth=1 origin 'release/v0.54.x:refs/heads/old' 'release/v0.55.x:refs/heads/new' + +# renames, with similarity scoring +git diff --find-renames --name-status old new + +# exact line mapping for one file +git diff -U0 old new -- +``` + +`--find-renames` is what makes rename detection usable. Matching on basename does not work: `store.go` had 23 candidates across the tree. + +For a line anchor, read the hunk headers from `git diff -U0`. A line outside every hunk moves by the cumulative offset, which is a safe automatic rewrite. A line inside a changed hunk was edited or deleted, which is a human decision. Do not guess. + +**Verdicts.** Four outcomes, not two: + +| Verdict | Action | +| ------- | ------ | +| Path exists, anchor unaffected | Bump the ref | +| Path exists, anchor moved outside any hunk | Bump the ref and rewrite the line number | +| Path renamed, deleted, or anchor inside a changed hunk | Flag for a human | +| Already 404 at the current ref | Flag, and read the page before repointing it | + +That last verdict is the valuable one. It found `store/tracekv/store.go`, dead since v0.54, which was the symptom of a page still documenting store tracing after upstream removed the API. ### Version format validation @@ -105,6 +202,82 @@ Only targets directories whose names match `/^v\d+/`. Never touches `next/` or ` --- +## check-github-refs.js — upstream link audit + +Audits GitHub links in a product's docs and bumps the ones it can prove are safe. Implements the rules in the "GitHub link refs" section above, so nothing there has to be done by hand. + +### Usage + +```bash +# report only (default) +node check-github-refs.js --product sdk + +# apply the auto-fixable verdicts +node check-github-refs.js --product sdk --targets latest,next --fix + +# write the machine-readable flag report for the adjudication skill +node check-github-refs.js --product cometbft --json /tmp/cometbft-flags.json +``` + +| Flag | Description | +| ---- | ----------- | +| `--product ` | `sdk` or `cometbft`. Only products with a version-tracking ref shape are supported. | +| `--targets ` | Comma-separated version dirs [default: `latest,next`] | +| `--new-ref ` | Override the target ref. Default is derived from `latestDisplayVersion` in `versions.json`. | +| `--fix` | Apply the auto-fixable verdicts. Never touches a flagged link. | +| `--json ` | Write the flag records for an agent to adjudicate | +| `--limit ` | Process only the first n links, for a smoke test | + +Set `GITHUB_TOKEN` to avoid unauthenticated API rate limits. Trees and the partial clone are cached under `.cache/refcheck/`, which is gitignored. + +Because the target ref is derived from `versions.json`, the script needs no arguments at freeze time. That also means it trusts what the freeze wrote: if `latestDisplayVersion` were wrong, the script would confidently bump every link to the wrong ref. Sanity-check a few verdicts against known cases on the first run after a freeze. + +### Verdicts + +Auto-fixable: + +| Verdict | Meaning | +| ------- | ------- | +| `bump` | Path exists at the new ref and there is no anchor to invalidate | +| `relocate` | Anchor fell outside every diff hunk, so it only shifted; the new line number is computed from the cumulative offset | + +Flagged, never auto-fixed: + +| Verdict | Meaning | +| ------- | ------- | +| `dead-now` | Already 404 at its current ref. Highest priority, because it usually means the prose describes something upstream removed. | +| `anchor-changed` | The anchored lines were edited between the refs, so the prose may no longer match | +| `path-gone` | Renamed or deleted at the new ref. No rename is guessed. | +| `drift` | A pinned ref whose anchored code has since changed. The link still resolves; the prose may not hold. | +| `unmaintainable` | A line anchor on `main`, which moves continuously. Repoint at a pinned sha or the release branch. | +| `skip` | Deliberately left alone, with the reason recorded | + +Flags carry the upstream diff hunk in `evidence`. Hand the `--json` output to `.claude/skills/update-stale-refs`, which decides whether each one needs a page edit. + +### How the line mapping works + +`git diff -U0 -- ` gives hunk headers. A line outside every hunk moved by the cumulative offset, which is a safe rewrite. A line inside a hunk was edited, which is a human decision. Content comparison cannot make that distinction, and produced six candidate lines for one `.proto` anchor before this approach replaced it. + +The clone is bare, shallow, and `--filter=blob:none`, so only the files actually diffed are downloaded. + +### Tests + +```bash +node check-github-refs.test.js +``` + +Covers the parsing and mapping logic, which is where the bugs were. Network and git behavior is not mocked; run the checker in report mode to exercise those. + +The tests live beside the script rather than in a top-level `tests/` directory because `.gitignore` ignores `tests/*`, and these need to be committed. + +Three bugs the tests now pin down, all of which would have produced silently wrong links: + +- A pure insertion hunk (`@@ -10,0 +11,5 @@`) inserts *after* old line 10, so line 10 must not shift. An earlier version shifted it. +- A bare URL ending a sentence pulls the trailing `.` into the path, which looked like a deleted file. +- An unanchored link is a string prefix of the anchored link to the same path. Rewriting the short URL first also rewrote the ref inside the long one, so the anchored record stopped matching and its line number stayed at the old value. Rewrites are applied longest-URL-first for this reason. + +--- + ## manage-changelogs.js — changelog generation Fetches `CHANGELOG.md` from a product's GitHub repository and generates Mintlify MDX release notes using `` components. diff --git a/scripts/versioning/check-github-refs.js b/scripts/versioning/check-github-refs.js new file mode 100644 index 000000000..e3782ca6a --- /dev/null +++ b/scripts/versioning/check-github-refs.js @@ -0,0 +1,660 @@ +#!/usr/bin/env node + +/** + * check-github-refs.js + * + * Audits GitHub links in a product's docs and, for links pinned to a + * version-tracking release branch, bumps them to the branch that is now + * shipping. Reports everything it will not touch. + * + * Usage: + * node check-github-refs.js --product sdk + * node check-github-refs.js --product sdk --targets latest,next --fix + * node check-github-refs.js --product cometbft --json flags.json + * + * Options: + * --product sdk | cometbft (products with a version-tracking ref shape) + * --targets comma-separated version dirs [default: latest,next] + * --new-ref override the target ref (default: derived from versions.json) + * --fix apply the two safe rewrites; never touches a flagged link + * --json write the machine-readable flag report + * --limit only process the first n links (for smoke tests) + * + * WHY THIS EXISTS + * + * A version-tracking ref does not follow a freeze, so after promoting v0.55 the + * pages still pointed at release/v0.54.x. Bumping them with sed is wrong: about a + * quarter of line-anchored links land on different code at the new ref, and the + * link keeps working, so the error is silent. See the GitHub link section of + * CLAUDE.md in this directory for the measurements. + * + * THE FLAG CONTRACT + * + * Every link produces one record. Records with an actionable verdict are written + * to --json for an agent to adjudicate (see .claude/skills/update-stale-refs). + * + * { + * doc: "sdk/latest/learn/concepts/store.mdx", // file containing the link + * line: 159, // 1-indexed line in doc + * url: "https://github.com/...", // the link as written + * repo: "cosmos/cosmos-sdk", + * kind: "blob" | "tree", + * refOld: "release/v0.54.x", + * refNew: "release/v0.55.x", // null when not bumpable + * path: "store/tracekv/store.go", // URL-decoded + * anchor: { start: 36, end: null } | null, + * category: "version-tracking" | "pinned" | "moving" | "other", + * verdict: see VERDICTS below, + * newAnchor: { start, end } | null, // when relocated + * evidence: string | null, // upstream diff hunk + * reason: string // one-line explanation + * } + */ + +import fs from 'fs'; +import path from 'path'; +import { execFileSync } from 'child_process'; +import { fileURLToPath } from 'url'; + +const __dirname = path.dirname(fileURLToPath(import.meta.url)); +const REPO_ROOT = path.join(__dirname, '..', '..'); +const CACHE = path.join(__dirname, '.cache', 'refcheck'); + +/** Verdicts, ordered from "safe to automate" to "needs a human". */ +export const VERDICTS = { + OK: 'ok', // already on the target ref, nothing to do + BUMP: 'bump', // path exists at new ref, no anchor to invalidate + RELOCATE: 'relocate', // anchor only shifted; new line number computed + ANCHOR_CHANGED: 'anchor-changed', // anchored code was edited; prose may be stale + PATH_GONE: 'path-gone', // renamed or deleted at the new ref + DEAD_NOW: 'dead-now', // already 404 at its current ref, before any bump + DRIFT: 'drift', // pinned ref whose anchored code has since changed + UNMAINTAINABLE: 'unmaintainable', // line anchor on a moving ref + UNASSESSED: 'unassessed', // could not be checked; not the same as "fine" + SKIP: 'skip', // deliberately left alone, and confirmed fine +}; + +const AUTO_FIX = new Set([VERDICTS.BUMP, VERDICTS.RELOCATE]); + +/** + * Pages whose content is ABOUT a specific version must keep that version's refs. + * A v0.54 upgrade guide citing the v0.54 CHANGELOG is correct; bumping it to + * v0.55 makes the page contradict itself. Bumping these produced exactly that: + * "breaking changes in v0.54.0, see the Changelog" pointing at v0.55's changelog, + * and "the v0.53.x to v0.54.x upgrade reference" pointing at v0.55's UPGRADING.md. + */ +const VERSION_SPECIFIC = [ + /\/upgrade\//, // //upgrade/v0.54.mdx and friends + /\/changelog\//, // generated release notes +]; + +export const isVersionSpecific = doc => VERSION_SPECIFIC.some(re => re.test(doc)); + +// Per-product ref shape. The version-tracking ref is the only one ever bumped. +const PRODUCTS = { + sdk: { + repo: 'cosmos/cosmos-sdk', + // release/v0.54.x + trackingRe: /^release\/v\d+\.\d+\.x$/, + refFor: v => `release/v${v.replace(/^v/, '')}.x`, + }, + cometbft: { + repo: 'cometbft/cometbft', + // v0.38.x + trackingRe: /^v\d+\.\d+\.x$/, + refFor: v => `v${v.replace(/^v/, '')}.x`, + }, +}; + +// --------------------------------------------------------------------------- +// Pure helpers (exported for the fixture tests) +// --------------------------------------------------------------------------- + +/** + * A two-segment ref (release/v0.55.x) must be tried before a single segment, + * or "release" is taken as the whole ref and "v0.55.x/..." becomes the path. + * That miscounted 126 SDK links as an unknown category on the first pass. + */ +const LINK_RE = new RegExp( + 'https://github\\.com/([\\w.-]+/[\\w.-]+)/(blob|tree)/' + + '(release/v\\d+\\.\\d+\\.x|[^/\\s)"\'\\]]+)' + + '/([^\\s)"\'\\]#]+)' + + '(?:#L(\\d+)(?:-L(\\d+))?)?', + 'g' +); + +/** TRAP 1: a path is URL-encoded in the link but literal in the git tree. */ +export function normalizePath(p) { + // A bare URL ending a sentence pulls the punctuation into the path. Markdown + // links are terminated by ")" so this only bites on unwrapped URLs, but a + // spurious trailing "." would look like a deleted file. + const trimmed = p.replace(/[.,;:]+$/, ''); + try { + return decodeURIComponent(trimmed); + } catch { + return trimmed; // malformed escape; compare as written + } +} + +export function classifyRef(ref, trackingRe) { + if (trackingRe.test(ref)) return 'version-tracking'; + if (ref === 'main' || ref === 'master') return 'moving'; + if (/^[0-9a-f]{7,40}$/.test(ref)) return 'pinned'; + if (/^v\d/.test(ref)) return 'pinned'; + return 'other'; +} + +/** + * TRAP 2: GitHub redirects /blob// to /tree/, so a "blob" link + * pointing at a directory resolves fine. Accept either type. + */ +export function treeHas(trees, p) { + const q = normalizePath(p); + return trees.blobs.has(q) || trees.dirs.has(q); +} + +export function parseHunks(diffText) { + const hunks = []; + for (const line of diffText.split('\n')) { + const m = line.match(/^@@ -(\d+)(?:,(\d+))? \+(\d+)(?:,(\d+))? @@/); + if (m) { + hunks.push({ + oldStart: +m[1], + oldCount: m[2] === undefined ? 1 : +m[2], + newStart: +m[3], + newCount: m[4] === undefined ? 1 : +m[4], + }); + } + } + return hunks; +} + +/** + * Map an old line number to its new position using diff hunk offsets. + * + * Outside every hunk the line only shifted, which is a safe rewrite. Inside a + * hunk the content itself changed, which is a human decision. This is what + * content comparison could not distinguish: identical text elsewhere in the file + * produced six candidate lines for one .proto anchor. + * + * @returns {{status:'same'|'moved'|'changed', line:number|null}} + */ +export function mapLine(oldLine, hunks) { + let offset = 0; + for (const h of hunks) { + if (h.oldCount === 0) { + // Pure insertion. Git writes "@@ -10,0 +11,5 @@" to mean the new lines go + // AFTER old line 10, so line 10 itself is untouched. + if (oldLine <= h.oldStart) break; + offset += h.newCount; + continue; + } + if (oldLine < h.oldStart) break; // no later hunk can affect it + if (oldLine < h.oldStart + h.oldCount) return { status: 'changed', line: null }; + offset += h.newCount - h.oldCount; + } + const mapped = oldLine + offset; + return { status: offset === 0 ? 'same' : 'moved', line: mapped }; +} + +/** + * Minor-version distance between two version-tracking refs, or null if either + * cannot be parsed. + * + * Hunk-offset mapping assumes git's diff alignment is semantic. For a file that + * has been heavily rewritten, git aligns on textual similarity instead, so a line + * can sit "outside every hunk" while no longer meaning the same thing. Measured: + * mapping node/node.go across v0.34.x to v0.40.x returned L684 and L730 where the + * semantically correct lines were L699 and L764. One minor version is reliable; + * six is not. + */ +export function minorGap(refA, refB) { + const pick = r => { + const m = String(r).match(/v(\d+)\.(\d+)/); + return m ? [+m[1], +m[2]] : null; + }; + const a = pick(refA); + const b = pick(refB); + if (!a || !b || a[0] !== b[0]) return null; + return Math.abs(b[1] - a[1]); +} + +/** Anchored auto-fixes are only trusted within this many minor versions. */ +export const MAX_ANCHOR_GAP = 1; + +export function extractLinks(text, docPath) { + const out = []; + const lines = text.split('\n'); + lines.forEach((lineText, i) => { + for (const m of lineText.matchAll(LINK_RE)) { + const [url, repo, kind, ref, rawPath, l1, l2] = m; + out.push({ + doc: docPath, + line: i + 1, + url, + repo, + kind, + refOld: ref, + path: normalizePath(rawPath), + rawPath, + anchor: l1 ? { start: +l1, end: l2 ? +l2 : null } : null, + }); + } + }); + return out; +} + +/** + * Rewrite one document's auto-fixable links. Pure, so the ordering hazard below + * is covered by fixture tests. + * + * @returns {{text:string, count:number, warnings:string[]}} + */ +export function applyRewrites(text, items) { + const warnings = []; + let count = 0; + // Longest URL first. An unanchored link is a string prefix of the anchored + // link to the same path, so replacing the short one first would rewrite the ref + // inside the long one; the anchored record would then no longer match and its + // line number would be stranded at the old value. That is the exact + // silent-wrong-line failure this tool exists to prevent. + const ordered = [...items].sort((a, b) => b.url.length - a.url.length); + for (const l of ordered) { + let replacement = l.url.replace(`/${l.kind}/${l.refOld}/`, `/${l.kind}/${l.refNew}/`); + if (l.verdict === VERDICTS.RELOCATE) { + const oldFrag = `#L${l.anchor.start}${l.anchor.end ? `-L${l.anchor.end}` : ''}`; + const newFrag = `#L${l.newAnchor.start}${l.newAnchor.end ? `-L${l.newAnchor.end}` : ''}`; + if (!replacement.endsWith(oldFrag)) { + warnings.push(`${l.doc}:${l.line} expected to end with ${oldFrag}; left alone`); + continue; + } + replacement = replacement.slice(0, -oldFrag.length) + newFrag; + } + // The same URL can appear on several lines and yields one record each; the + // first record rewrites them all, so count occurrences, not records. + const parts = text.split(l.url); + if (parts.length > 1) { + count += parts.length - 1; + text = parts.join(replacement); + } + } + return { text, count, warnings }; +} + +// --------------------------------------------------------------------------- +// Network and git +// --------------------------------------------------------------------------- + +async function fetchTree(repo, ref) { + const key = `${repo}@${ref}`.replace(/[^\w.@-]/g, '_'); + const cached = path.join(CACHE, `tree_${key}.json`); + if (fs.existsSync(cached)) return JSON.parse(fs.readFileSync(cached, 'utf8')); + + const url = `https://api.github.com/repos/${repo}/git/trees/${ref}?recursive=1`; + const headers = { 'User-Agent': 'cosmos-docs-check-github-refs' }; + if (process.env.GITHUB_TOKEN) headers.Authorization = `Bearer ${process.env.GITHUB_TOKEN}`; + const res = await fetch(url, { headers }); + if (!res.ok) throw new Error(`tree fetch failed for ${repo}@${ref}: ${res.status}`); + const data = await res.json(); + if (data.truncated) { + console.warn(` ⚠ tree for ${repo}@${ref} is TRUNCATED; path checks may be wrong`); + } + fs.mkdirSync(CACHE, { recursive: true }); + fs.writeFileSync(cached, JSON.stringify(data)); + return data; +} + +function toSets(treeData) { + const blobs = new Set(); + const dirs = new Set(); + for (const e of treeData.tree || []) { + (e.type === 'blob' ? blobs : dirs).add(e.path); + } + return { blobs, dirs }; +} + +/** Bare partial clone: blobs are fetched lazily, so only diffed files download. */ +function ensureRepo(repo) { + const dir = path.join(CACHE, repo.replace('/', '__') + '.git'); + if (!fs.existsSync(dir)) { + fs.mkdirSync(dir, { recursive: true }); + execFileSync('git', ['init', '-q', '--bare', dir]); + execFileSync('git', ['-C', dir, 'remote', 'add', 'origin', `https://github.com/${repo}.git`]); + } + return dir; +} + +function fetchRef(gitDir, ref) { + // Custom namespace, not refs/heads. An annotated tag resolves to a tag object, + // and git refuses to store one as a branch head ("invalid new value provided"), + // which silently failed every tag fetch and left 64 pinned links unassessed. + const local = `refs/docs/${ref.replace(/[^\w.-]/g, '_')}`; + try { + execFileSync('git', ['-C', gitDir, 'rev-parse', '--verify', '-q', local], { + stdio: 'ignore', + }); + return local; + } catch { + /* not fetched yet */ + } + // A bare name works for a branch; a tag needs its full refs/tags/ path. + const errors = []; + for (const src of [ref, `refs/tags/${ref}`, `refs/heads/${ref}`]) { + try { + execFileSync( + 'git', + ['-C', gitDir, 'fetch', '-q', '--filter=blob:none', '--depth=1', 'origin', `${src}:${local}`], + { stdio: ['ignore', 'ignore', 'pipe'] } + ); + execFileSync('git', ['-C', gitDir, 'rev-parse', '--verify', '-q', local], { stdio: 'ignore' }); + return local; + } catch (e) { + errors.push(`${src}: ${String(e.stderr || e.message).trim().split('\n')[0]}`); + } + } + throw new Error(`could not fetch ${ref} (tried ${errors.length} refspecs)`); +} + +function diffFile(gitDir, refA, refB, filePath) { + try { + return execFileSync( + 'git', + ['-C', gitDir, 'diff', '-U0', '--no-color', refA, refB, '--', filePath], + { encoding: 'utf8', maxBuffer: 64 * 1024 * 1024 } + ); + } catch (e) { + return e.stdout || ''; + } +} + +// --------------------------------------------------------------------------- +// Main +// --------------------------------------------------------------------------- + +function parseArgs(argv) { + const cfg = { product: null, targets: ['latest', 'next'], newRef: null, fix: false, json: null, limit: null }; + for (let i = 0; i < argv.length; i++) { + switch (argv[i]) { + case '--product': cfg.product = argv[++i]; break; + case '--targets': cfg.targets = argv[++i].split(','); break; + case '--new-ref': cfg.newRef = argv[++i]; break; + case '--fix': cfg.fix = true; break; + case '--json': cfg.json = argv[++i]; break; + case '--limit': cfg.limit = +argv[++i]; break; + case '--help': + console.log(fs.readFileSync(fileURLToPath(import.meta.url), 'utf8').split('*/')[0]); + process.exit(0); + } + } + return cfg; +} + +function walk(dir, acc = []) { + if (!fs.existsSync(dir)) return acc; + for (const e of fs.readdirSync(dir, { withFileTypes: true })) { + const p = path.join(dir, e.name); + if (e.isDirectory()) walk(p, acc); + else if (/\.(mdx|md)$/.test(e.name)) acc.push(p); + } + return acc; +} + +async function main() { + const cfg = parseArgs(process.argv.slice(2)); + const product = PRODUCTS[cfg.product]; + if (!product) { + console.error(`Error: --product must be one of ${Object.keys(PRODUCTS).join(', ')}`); + process.exit(1); + } + + const versions = JSON.parse(fs.readFileSync(path.join(REPO_ROOT, 'versions.json'), 'utf8')); + const display = versions.products[cfg.product].latestDisplayVersion; + const newRef = cfg.newRef || product.refFor(display); + + console.log(`check-github-refs: ${cfg.product} (${product.repo})`); + console.log(` shipping ref: ${newRef}`); + console.log(` targets: ${cfg.targets.join(', ')}`); + + // 1. extract + let links = []; + for (const t of cfg.targets) { + const root = path.join(REPO_ROOT, cfg.product, t); + for (const f of walk(root)) { + const rel = path.relative(REPO_ROOT, f); + links.push(...extractLinks(fs.readFileSync(f, 'utf8'), rel)); + } + } + links = links.filter(l => l.repo === product.repo); + if (cfg.limit) links = links.slice(0, cfg.limit); + console.log(` links: ${links.length} (${links.filter(l => l.anchor).length} anchored)\n`); + + // 2. classify, and collect the refs we need trees for + for (const l of links) l.category = classifyRef(l.refOld, product.trackingRe); + // Every ref gets a tree, including `main` and unrecognized branch names, so + // that every link is liveness-checked. A path present in the tree is a link + // that resolves; absent means the link is dead at the ref it carries. + const needTrees = new Set([newRef, ...links.map(l => l.refOld)]); + + const trees = {}; + for (const ref of needTrees) { + try { + trees[ref] = toSets(await fetchTree(product.repo, ref)); + console.log(` tree ${ref}: ${trees[ref].blobs.size} blobs`); + } catch (e) { + console.warn(` ⚠ ${e.message}`); + trees[ref] = null; + } + } + console.log(''); + + // 3. adjudicate + const gitDir = ensureRepo(product.repo); + const localRefs = {}; + const needDiff = l => + l.anchor && + (l.category === 'version-tracking' || l.category === 'pinned') && + l.refOld !== newRef; + const refsToClone = new Set(links.filter(needDiff).map(l => l.refOld)); + if (refsToClone.size) { + for (const ref of [newRef, ...refsToClone]) { + if (localRefs[ref]) continue; + try { + localRefs[ref] = fetchRef(gitDir, ref); + } catch (e) { + console.warn(` ⚠ could not fetch ${ref}: ${e.message}`); + localRefs[ref] = null; + } + } + } + + const diffCache = new Map(); + const getHunks = (refOld, p) => { + const key = `${refOld}${p}`; + if (!diffCache.has(key)) { + const a = localRefs[refOld]; + const b = localRefs[newRef]; + diffCache.set(key, a && b ? parseHunks(diffFile(gitDir, a, b, p)) : null); + } + return diffCache.get(key); + }; + + for (const l of links) { + l.refNew = null; + l.newAnchor = null; + l.evidence = null; + + // Liveness first, for every category. A link whose path is absent at its own + // ref is dead now, which matters more than which category it belongs to. + const ownTree = trees[l.refOld]; + if (ownTree && !treeHas(ownTree, l.path)) { + l.verdict = VERDICTS.DEAD_NOW; + l.reason = `path absent at its current ref ${l.refOld}; this link 404s today`; + continue; + } + + // A page about one version keeps that version's refs, even when superseded. + if (isVersionSpecific(l.doc)) { + l.verdict = VERDICTS.SKIP; + l.reason = `version-specific page; keeps its own version's refs and is never bumped`; + continue; + } + + if (l.category === 'other') { + l.verdict = VERDICTS.SKIP; + l.reason = `unrecognized ref "${l.refOld}"; needs a look`; + continue; + } + + if (l.category === 'moving') { + if (l.anchor) { + l.verdict = VERDICTS.UNMAINTAINABLE; + l.reason = `line anchor on "${l.refOld}", which moves continuously; repoint at a pinned sha or ${newRef}`; + } else { + l.verdict = VERDICTS.SKIP; + l.reason = `"${l.refOld}" tracks current code; no version bump applies`; + } + continue; + } + + if (l.category === 'pinned') { + if (!l.anchor) { + l.verdict = VERDICTS.SKIP; + l.reason = `pinned to ${l.refOld}; deliberate citation, not bumped`; + continue; + } + const newTree = trees[newRef]; + if (newTree && !treeHas(newTree, l.path)) { + l.verdict = VERDICTS.DRIFT; + l.reason = `pinned at ${l.refOld}; that path no longer exists at ${newRef}, so surrounding prose may be stale`; + continue; + } + // Line-level drift is only measurable within a small version distance, for + // the same reason anchored auto-fixes are. A SHA gives no distance at all. + const pgap = minorGap(l.refOld, newRef); + if (pgap === null || pgap > MAX_ANCHOR_GAP) { + l.verdict = VERDICTS.UNASSESSED; + l.reason = + `pinned at ${l.refOld}, ${pgap === null ? 'no comparable version distance' : pgap + ' minor versions'} ` + + `from ${newRef}; line-level drift cannot be measured reliably, so this needs a human read`; + continue; + } + const hunks = getHunks(l.refOld, l.path); + if (hunks === null) { + l.verdict = VERDICTS.UNASSESSED; + l.reason = `pinned at ${l.refOld}; could not diff the file, so drift is unknown`; + } else { + const span = [l.anchor.start, ...(l.anchor.end ? [l.anchor.end] : [])]; + const changed = span.some(n => mapLine(n, hunks).status === 'changed'); + if (changed) { + l.verdict = VERDICTS.DRIFT; + l.reason = `pinned at ${l.refOld}; the anchored code changed by ${newRef}, so check the prose still holds`; + l.evidence = diffFile(gitDir, localRefs[l.refOld], localRefs[newRef], l.path).slice(0, 4000); + } else { + l.verdict = VERDICTS.SKIP; + l.reason = `pinned at ${l.refOld}; anchored code unchanged at ${newRef}, prose still matches`; + } + } + continue; + } + + // version-tracking + if (l.refOld === newRef) { + l.verdict = VERDICTS.OK; + l.reason = 'already on the shipping ref'; + continue; + } + l.refNew = newRef; + + const newTree = trees[newRef]; + if (newTree && !treeHas(newTree, l.path)) { + l.verdict = VERDICTS.PATH_GONE; + l.reason = `path does not exist at ${newRef}; renamed or deleted upstream`; + continue; + } + + if (!l.anchor) { + l.verdict = VERDICTS.BUMP; + l.reason = `path exists at ${newRef} and there is no anchor to invalidate`; + continue; + } + + // Across more than one minor version, git's diff alignment stops being a + // reliable proxy for "the same line", so no anchored rewrite is trustworthy. + const gap = minorGap(l.refOld, newRef); + if (gap === null || gap > MAX_ANCHOR_GAP) { + l.verdict = VERDICTS.ANCHOR_CHANGED; + l.reason = + `anchored link spanning ${gap === null ? 'an unparseable ref pair' : gap + ' minor versions'} ` + + `(${l.refOld} to ${newRef}); line mapping is not reliable beyond ${MAX_ANCHOR_GAP}, verify by hand`; + continue; + } + + const hunks = getHunks(l.refOld, l.path); + if (hunks === null) { + l.verdict = VERDICTS.ANCHOR_CHANGED; + l.reason = 'could not diff the file to verify the anchor'; + continue; + } + const startMap = mapLine(l.anchor.start, hunks); + const endMap = l.anchor.end ? mapLine(l.anchor.end, hunks) : null; + if (startMap.status === 'changed' || endMap?.status === 'changed') { + l.verdict = VERDICTS.ANCHOR_CHANGED; + l.reason = `the anchored lines were edited between ${l.refOld} and ${newRef}`; + l.evidence = diffFile(gitDir, localRefs[l.refOld], localRefs[newRef], l.path).slice(0, 4000); + } else if (startMap.status === 'same' && (!endMap || endMap.status === 'same')) { + l.verdict = VERDICTS.BUMP; + l.reason = `anchor unaffected between ${l.refOld} and ${newRef}`; + } else { + l.verdict = VERDICTS.RELOCATE; + l.newAnchor = { start: startMap.line, end: endMap ? endMap.line : null }; + l.reason = `anchor shifted from L${l.anchor.start} to L${startMap.line}`; + } + } + + // 4. report + const byVerdict = {}; + for (const l of links) (byVerdict[l.verdict] ||= []).push(l); + console.log('VERDICTS'); + for (const v of Object.values(VERDICTS)) { + const n = byVerdict[v]?.length || 0; + if (n) console.log(` ${String(n).padStart(4)} ${v}${AUTO_FIX.has(v) ? ' (auto-fixable)' : ''}`); + } + + const flags = links.filter(l => !AUTO_FIX.has(l.verdict) && l.verdict !== VERDICTS.OK && l.verdict !== VERDICTS.SKIP); + if (flags.length) { + console.log(`\nNEEDS REVIEW (${flags.length})`); + for (const f of flags) { + console.log(` [${f.verdict}] ${f.doc}:${f.line}`); + console.log(` ${f.path}${f.anchor ? `#L${f.anchor.start}` : ''} ${f.refOld}`); + console.log(` ${f.reason}`); + } + } + + if (cfg.json) { + const out = links.filter(l => l.verdict !== VERDICTS.OK); + fs.writeFileSync(cfg.json, JSON.stringify({ product: cfg.product, repo: product.repo, newRef, records: out }, null, 2)); + console.log(`\nwrote ${out.length} records to ${cfg.json}`); + } + + // 5. fix + if (cfg.fix) { + const fixable = links.filter(l => AUTO_FIX.has(l.verdict) && l.refNew); + const byDoc = {}; + for (const l of fixable) (byDoc[l.doc] ||= []).push(l); + let count = 0; + for (const [doc, items] of Object.entries(byDoc)) { + const abs = path.join(REPO_ROOT, doc); + const res = applyRewrites(fs.readFileSync(abs, 'utf8'), items); + for (const w of res.warnings) console.warn(` ⚠ ${w}`); + count += res.count; + fs.writeFileSync(abs, res.text); + } + console.log(`\napplied ${count} rewrites across ${Object.keys(byDoc).length} files`); + console.log('flagged links were not touched'); + } else { + console.log('\n(report only; pass --fix to apply the auto-fixable verdicts)'); + } +} + +const isMain = process.argv[1] && fileURLToPath(import.meta.url) === path.resolve(process.argv[1]); +if (isMain) main(); diff --git a/scripts/versioning/check-github-refs.test.js b/scripts/versioning/check-github-refs.test.js new file mode 100644 index 000000000..5976e2452 --- /dev/null +++ b/scripts/versioning/check-github-refs.test.js @@ -0,0 +1,322 @@ +#!/usr/bin/env node + +/** + * Fixture tests for check-github-refs.js + * + * Run: node check-github-refs.test.js + * + * Covers the parsing and mapping logic only, which is where the real bugs were. + * Network and git behavior is not mocked; those are exercised by running the + * checker in report mode. + * + * These live here rather than in a top-level tests/ directory because + * .gitignore ignores `tests/*` and they need to be committed. + */ + +import assert from 'assert'; +import { + normalizePath, + classifyRef, + treeHas, + parseHunks, + mapLine, + extractLinks, + applyRewrites, + minorGap, + MAX_ANCHOR_GAP, + VERDICTS, +} from './check-github-refs.js'; + +let pass = 0; +const failures = []; +function t(name, fn) { + try { + fn(); + pass++; + } catch (e) { + failures.push(`${name}: ${e.message}`); + } +} + +const SDK_TRACKING = /^release\/v\d+\.\d+\.x$/; +const CMT_TRACKING = /^v\d+\.\d+\.x$/; + +// --- TRAP 1: URL-encoded paths ------------------------------------------- +// abci%2B%2B_methods.md is abci++_methods.md in the tree. Comparing the raw +// string reported a live CometBFT link as broken. + +t('normalizePath decodes %2B', () => { + assert.strictEqual( + normalizePath('spec/abci/abci%2B%2B_methods.md'), + 'spec/abci/abci++_methods.md' + ); +}); + +t('normalizePath leaves a plain path alone', () => { + assert.strictEqual(normalizePath('store/iavl/store.go'), 'store/iavl/store.go'); +}); + +t('normalizePath survives a malformed escape', () => { + assert.strictEqual(normalizePath('a/b%zz.md'), 'a/b%zz.md'); +}); + +t('treeHas matches an encoded path against the literal tree entry', () => { + const trees = { blobs: new Set(['spec/abci/abci++_methods.md']), dirs: new Set() }; + assert.ok(treeHas(trees, 'spec/abci/abci%2B%2B_methods.md')); +}); + +// --- TRAP 2: /blob/ pointing at a directory ------------------------------ +// GitHub redirects /blob// to /tree/, returning 200. Requiring a blob +// match reported four live CometBFT spec links as broken. + +t('treeHas accepts a directory for a blob-style link', () => { + const trees = { blobs: new Set(), dirs: new Set(['spec/light-client/accountability']) }; + assert.ok(treeHas(trees, 'spec/light-client/accountability')); +}); + +t('treeHas rejects a path in neither set', () => { + const trees = { blobs: new Set(['a/b.go']), dirs: new Set(['a']) }; + assert.ok(!treeHas(trees, 'store/tracekv/store.go')); +}); + +// --- Ref classification -------------------------------------------------- + +t('two-segment release ref is version-tracking, not "other"', () => { + assert.strictEqual(classifyRef('release/v0.55.x', SDK_TRACKING), 'version-tracking'); +}); + +t('cometbft bare version ref is version-tracking', () => { + assert.strictEqual(classifyRef('v0.40.x', CMT_TRACKING), 'version-tracking'); +}); + +t('main and master are moving', () => { + assert.strictEqual(classifyRef('main', SDK_TRACKING), 'moving'); + assert.strictEqual(classifyRef('master', SDK_TRACKING), 'moving'); +}); + +t('a 40-char sha is pinned', () => { + assert.strictEqual( + classifyRef('2bec9d2021918650d3938c3ab242f84289daef80', SDK_TRACKING), + 'pinned' + ); +}); + +t('a release-candidate tag is pinned, not version-tracking', () => { + assert.strictEqual(classifyRef('v0.47.0-rc1', SDK_TRACKING), 'pinned'); +}); + +t('a branch name like cosmovisor is "other"', () => { + assert.strictEqual(classifyRef('cosmovisor', SDK_TRACKING), 'other'); +}); + +// --- Link extraction ----------------------------------------------------- + +t('extracts a two-segment ref without eating the version into the path', () => { + const [l] = extractLinks( + 'see [keys](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/x/bank/types/keys.go)', + 'd.mdx' + ); + assert.strictEqual(l.refOld, 'release/v0.54.x'); + assert.strictEqual(l.path, 'x/bank/types/keys.go'); + assert.strictEqual(l.anchor, null); +}); + +t('extracts a single-line anchor', () => { + const [l] = extractLinks( + '[s](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/store/iavl/store.go#L36)', + 'd.mdx' + ); + assert.deepStrictEqual(l.anchor, { start: 36, end: null }); +}); + +t('extracts a line range anchor', () => { + const [l] = extractLinks( + '[p](https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/x/tx.proto#L29-L41)', + 'd.mdx' + ); + assert.deepStrictEqual(l.anchor, { start: 29, end: 41 }); +}); + +t('records the 1-indexed line the link sits on', () => { + const links = extractLinks( + 'intro\n\n[a](https://github.com/cosmos/cosmos-sdk/blob/main/go.mod)\n', + 'd.mdx' + ); + assert.strictEqual(links[0].line, 3); +}); + +t('does not swallow a trailing markdown paren or bracket', () => { + const [l] = extractLinks( + '[a](https://github.com/cosmos/cosmos-sdk/tree/main/store) and text', + 'd.mdx' + ); + assert.strictEqual(l.path, 'store'); + assert.strictEqual(l.kind, 'tree'); +}); + +t('finds multiple links on one line', () => { + const links = extractLinks( + '[a](https://github.com/cosmos/cosmos-sdk/blob/main/a.go) [b](https://github.com/cosmos/cosmos-sdk/blob/main/b.go)', + 'd.mdx' + ); + assert.strictEqual(links.length, 2); +}); + +// --- Hunk parsing and line mapping -------------------------------------- + +t('parseHunks reads counts, defaulting an omitted count to 1', () => { + const h = parseHunks('@@ -10 +10 @@\n@@ -20,3 +20,5 @@\n'); + assert.deepStrictEqual(h[0], { oldStart: 10, oldCount: 1, newStart: 10, newCount: 1 }); + assert.deepStrictEqual(h[1], { oldStart: 20, oldCount: 3, newStart: 20, newCount: 5 }); +}); + +t('a line before every hunk is unchanged', () => { + const h = parseHunks('@@ -50,2 +50,4 @@'); + assert.deepStrictEqual(mapLine(10, h), { status: 'same', line: 10 }); +}); + +t('a line after a net-negative hunk shifts up', () => { + // 3 old lines became 1: everything after moves up by 2 + const h = parseHunks('@@ -10,3 +10,1 @@'); + assert.deepStrictEqual(mapLine(50, h), { status: 'moved', line: 48 }); +}); + +t('a line after a net-positive hunk shifts down', () => { + const h = parseHunks('@@ -10,1 +10,4 @@'); + assert.deepStrictEqual(mapLine(50, h), { status: 'moved', line: 53 }); +}); + +t('a line inside a changed hunk is "changed", never guessed', () => { + const h = parseHunks('@@ -10,3 +10,3 @@'); + assert.strictEqual(mapLine(11, h).status, 'changed'); + assert.strictEqual(mapLine(11, h).line, null); +}); + +t('a pure insertion (oldCount 0) cannot swallow a line', () => { + const h = parseHunks('@@ -10,0 +11,5 @@'); + assert.strictEqual(mapLine(10, h).status, 'same'); + assert.deepStrictEqual(mapLine(11, h), { status: 'moved', line: 16 }); +}); + +t('offsets accumulate across several hunks', () => { + const h = parseHunks('@@ -10,5 +10,2 @@\n@@ -30,1 +27,3 @@\n'); + // -3 then +2 => net -1 for a line after both + assert.deepStrictEqual(mapLine(100, h), { status: 'moved', line: 99 }); +}); + +t('an equal-size rewrite still reports changed, not same', () => { + // net offset is 0, so a naive implementation would call this "same" + const h = parseHunks('@@ -10,2 +10,2 @@'); + assert.strictEqual(mapLine(10, h).status, 'changed'); +}); + +// --- Regression: the real cases we already know the answers to ----------- + +t('iavl#L36 case: no hunks means the anchor holds', () => { + assert.deepStrictEqual(mapLine(36, parseHunks('')), { status: 'same', line: 36 }); +}); + +t('root.go#L50 case: anchor shifted to 47 via a net -3 hunk above it', () => { + const h = parseHunks('@@ -20,5 +20,2 @@'); + assert.deepStrictEqual(mapLine(50, h), { status: 'moved', line: 47 }); +}); + +// --- Version gap guard --------------------------------------------------- +// Hunk mapping is only semantic for small deltas. Mapping node/node.go across +// v0.34.x to v0.40.x returned L684 and L730 where the right lines were L699 and +// L764, so anchored auto-fixes are refused beyond MAX_ANCHOR_GAP. + +t('minorGap measures adjacent versions as 1', () => { + assert.strictEqual(minorGap('release/v0.54.x', 'release/v0.55.x'), 1); + assert.strictEqual(minorGap('v0.39.x', 'v0.40.x'), 1); +}); + +t('minorGap measures a wide gap', () => { + assert.strictEqual(minorGap('v0.34.x', 'v0.40.x'), 6); + assert.strictEqual(minorGap('release/v0.50.x', 'release/v0.55.x'), 5); +}); + +t('minorGap is 0 for the same ref', () => { + assert.strictEqual(minorGap('v0.40.x', 'v0.40.x'), 0); +}); + +t('minorGap refuses across a major version', () => { + assert.strictEqual(minorGap('v0.40.x', 'v1.0.x'), null); +}); + +t('minorGap is null when a ref has no version', () => { + assert.strictEqual(minorGap('main', 'v0.40.x'), null); + assert.strictEqual(minorGap('2bec9d2021918650d3938c3ab242f84289daef80', 'v0.40.x'), null); +}); + +t('MAX_ANCHOR_GAP admits one minor version and no more', () => { + assert.strictEqual(MAX_ANCHOR_GAP, 1); + assert.ok(minorGap('release/v0.54.x', 'release/v0.55.x') <= MAX_ANCHOR_GAP); + assert.ok(minorGap('release/v0.50.x', 'release/v0.55.x') > MAX_ANCHOR_GAP); +}); + +// --- Rewrite application ------------------------------------------------- +// The prefix hazard: an unanchored link is a substring of the anchored link to +// the same path. store.mdx has exactly this shape. + +const BASE = 'https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/store/iavl/store.go'; +const rec = (over = {}) => ({ + doc: 'd.mdx', line: 1, kind: 'blob', + refOld: 'release/v0.54.x', refNew: 'release/v0.55.x', + url: BASE, anchor: null, newAnchor: null, verdict: VERDICTS.BUMP, ...over, +}); + +t('bumps a plain link', () => { + const r = applyRewrites(`see [s](${BASE})`, [rec()]); + assert.ok(r.text.includes('release/v0.55.x/store/iavl/store.go')); + assert.strictEqual(r.count, 1); +}); + +t('prefix collision: short link does not strand the anchored one', () => { + const anchored = `${BASE}#L36`; + const text = `[a](${BASE}) and [b](${anchored})`; + const r = applyRewrites(text, [ + rec(), + rec({ url: anchored, anchor: { start: 36, end: null }, newAnchor: { start: 40, end: null }, verdict: VERDICTS.RELOCATE }), + ]); + // the anchored link must have BOTH a new ref and the relocated line + assert.ok(r.text.includes('release/v0.55.x/store/iavl/store.go#L40'), 'anchor not relocated'); + assert.ok(!r.text.includes('#L36'), 'old line number survived'); + // and the plain link must still be bumped + assert.ok(r.text.includes(`[a](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/store/iavl/store.go)`)); + assert.ok(!r.text.includes('v0.54.x'), 'an old ref survived'); +}); + +t('same URL twice counts occurrences, not records', () => { + const text = `[a](${BASE}) then again [b](${BASE})`; + const r = applyRewrites(text, [rec(), rec({ line: 2 })]); + assert.strictEqual(r.count, 2); + assert.ok(!r.text.includes('v0.54.x')); +}); + +t('relocates a line range', () => { + const url = `${BASE}#L29-L41`; + const r = applyRewrites(`[p](${url})`, [ + rec({ url, anchor: { start: 29, end: 41 }, newAnchor: { start: 26, end: 38 }, verdict: VERDICTS.RELOCATE }), + ]); + assert.ok(r.text.includes('#L26-L38')); +}); + +t('warns and skips when the anchor is not where expected', () => { + const r = applyRewrites(`[p](${BASE})`, [ + rec({ anchor: { start: 99, end: null }, newAnchor: { start: 1, end: null }, verdict: VERDICTS.RELOCATE }), + ]); + assert.strictEqual(r.count, 0); + assert.strictEqual(r.warnings.length, 1); + assert.ok(r.text.includes('v0.54.x'), 'should have been left alone'); +}); + +t('leaves an unrelated link untouched', () => { + const other = 'https://github.com/cosmos/cosmos-sdk/blob/main/go.mod'; + const r = applyRewrites(`[a](${BASE}) [b](${other})`, [rec()]); + assert.ok(r.text.includes(other)); +}); + +console.log(`\n${pass} passed, ${failures.length} failed`); +for (const f of failures) console.log(` FAIL ${f}`); +process.exit(failures.length ? 1 : 0); diff --git a/scripts/versioning/manage-changelogs.js b/scripts/versioning/manage-changelogs.js index fc34277aa..6d8e71f3a 100644 --- a/scripts/versioning/manage-changelogs.js +++ b/scripts/versioning/manage-changelogs.js @@ -179,144 +179,177 @@ async function fetchChangelog(repo, source, changelogPath) { throw new Error(`Failed to fetch changelog from ${repo}. Tried: ${errors.join('; ')}`); } -// Sanitize line for MDX compatibility -function sanitizeLine(line) { - let cleaned = line.trim(); - const markdownLinks = []; - - // Preserve markdown links - cleaned = cleaned.replace(/\[([^\]]+)\]\(([^)]+)\)/g, (match, text, url) => { - const placeholder = `__LINK_${markdownLinks.length}__`; - markdownLinks.push({ text, url }); - return placeholder; - }); - - // Escape comparison operators - cleaned = cleaned - .replace(/ <= /g, ' <= ') - .replace(/ >= /g, ' >= ') - .replace(/ < /g, ' < ') - .replace(/ > /g, ' > '); +// --- MDX-safe passthrough ------------------------------------------------- +// The upstream changelog is already markdown, and MDX is a superset of it, so +// release-note bodies are copied through verbatim rather than decomposed into a +// data structure and rebuilt. Only two textual transforms are applied: heading +// demotion and escaping of characters MDX would read as JSX. Because the body is +// never reconstructed, no shape of markdown in it can be silently dropped. + +// `<` opens a JSX element and `{` opens an expression, so both break the MDX +// build when they appear in prose. Inside code they are literal, and must be +// left alone. +function escapeMdxOutsideCode(line) { + // String.split with a capturing group yields the code spans at odd indices. + return line + .split(/(`+[^`]*`+)/g) + .map((part, i) => + i % 2 ? part : part.replace(/ { - cleaned = cleaned.replace(`__LINK_${i}__`, `[${link.text}](${link.url})`); +function mapOutsideFences(lines, fn) { + let inFence = false; + return lines.map(line => { + if (/^\s*(?:```|~~~)/.test(line)) { + inFence = !inFence; + return line; + } + return inFence ? line : fn(line); }); - - return cleaned; } -// Parse changelog to extract version updates -function parseChangelog(content, versionFilter = null, unreleasedAs = null) { - const lines = content.split('\n'); - const updates = []; - let currentVersion = null; - let currentDate = null; - let sections = {}; - let currentSection = null; - let skipUntilVersion = true; +const escapeMdx = lines => mapOutsideFences(lines, escapeMdxOutsideCode); - for (const line of lines) { - // Skip main changelog header - if (line.match(/^#\s+Changelog/i)) continue; +// Products differ in bullet marker (`*` upstream in the SDK, `-` in CometBFT). +// Normalizing keeps generated output uniform and, more usefully, keeps a +// regeneration diff limited to real content changes. Indentation is preserved so +// nesting depth survives. +const normalizeBullets = lines => + mapOutsideFences(lines, line => + line.replace(/^(\s*)\*(\s)/, '$1-$2').replace(/\s+$/, '') + ); - // Handle unreleased section - if (line.match(/^##\s*\[?Unreleased\]?(?:\([^)]*\))?/i)) { - if (unreleasedAs) { - // Treat as a named version entry - if (currentVersion && Object.keys(sections).length > 0) { - updates.push({ version: currentVersion, date: currentDate, sections }); +// Upstream sections are h3; inside they should render a level higher. +const demoteHeadings = lines => + mapOutsideFences(lines, line => + line.replace(/^(#{3,6})(\s+)/, (_, hashes, sp) => hashes.slice(1) + sp) + ); + +// Upstream occasionally leaves a bullet marker with no text after it, which +// renders as a stray empty list item. +const dropEmptyBullets = lines => + mapOutsideFences(lines, line => (/^\s*[-*+]\s*$/.test(line) ? null : line)).filter( + line => line !== null + ); + +// Upstream ships section headers with nothing under them (e.g. a bare +// "### DEPENDENCIES"); rendering those as empty headings looks like a bug. +function dropEmptyHeadings(lines) { + const out = []; + for (let i = 0; i < lines.length; i++) { + if (/^#{2,6}\s+/.test(lines[i])) { + let hasContent = false; + for (let j = i + 1; j < lines.length; j++) { + if (/^#{2,6}\s+/.test(lines[j])) break; + if (lines[j].trim()) { + hasContent = true; + break; } - currentVersion = unreleasedAs; - currentDate = ''; - sections = {}; - currentSection = null; - skipUntilVersion = false; - } else { - skipUntilVersion = true; } - continue; + if (!hasContent) continue; } + out.push(lines[i]); + } + return out; +} - // Match version headers - const versionMatch = line.match(/^##\s*\[?([vV]?\d+\.\d+(?:\.(?:\d+|x))?)\]?(?:\([^)]*\))?\s*(?:-\s*(.+))?$/); +function trimBlankEdges(lines) { + let a = 0; + let b = lines.length; + while (a < b && !lines[a].trim()) a++; + while (b > a && !lines[b - 1].trim()) b--; + return lines.slice(a, b); +} - if (versionMatch) { - skipUntilVersion = false; - - // Save previous version if exists - if (currentVersion && Object.keys(sections).length > 0) { - updates.push({ - version: currentVersion, - date: currentDate, - sections: sections, - }); +// Slice the changelog on `##` version boundaries. Only header lines are +// interpreted; everything between them is body text. +function sliceByVersion(content, versionFilter = null, unreleasedAs = null) { + const lines = content.split('\n'); + const updates = []; + let current = null; + let skipping = true; + + const flush = () => { + if (!current) return; + let body = trimBlankEdges(current.lines); + + // Some products put the release date on its own italic line under the + // header rather than in it; lift it into the label instead of leaving it + // stranded at the top of the body. + if (body.length) { + // Italic or bold, and upstream is inconsistent about the space after the + // comma (e.g. "*January 23,2026*"). + const d = body[0].match(/^\*{1,2}([A-Z][a-z]+ \d{1,2},\s*\d{4})\*{1,2}$/); + if (d) { + if (!current.date) current.date = d[1].replace(/,\s*/, ', '); + body = trimBlankEdges(body.slice(1)); } + } - // Start new version - currentVersion = versionMatch[1]; - currentDate = versionMatch[2] || ''; - sections = {}; - currentSection = null; - continue; + body = trimBlankEdges( + dropEmptyHeadings(dropEmptyBullets(normalizeBullets(escapeMdx(demoteHeadings(body))))) + ); + if (body.length) { + updates.push({ + version: current.version, + date: current.date, + body: body.join('\n'), + }); } + current = null; + }; - if (skipUntilVersion) continue; - if (!line.trim() || line.match(/^[-=]+$/)) continue; + for (const line of lines) { + // Skip the document title + if (/^#\s+Changelog/i.test(line)) continue; - // Match section headers - const sectionMatch = line.match(/^###\s+(.+)$/); - if (sectionMatch) { - currentSection = sectionMatch[1].trim(); - if (!sections[currentSection]) { - sections[currentSection] = []; + if (/^##\s*\[?Unreleased\]?(?:\([^)]*\))?/i.test(line)) { + flush(); + if (unreleasedAs) { + current = { version: unreleasedAs, date: '', lines: [] }; + skipping = false; + } else { + skipping = true; } continue; } - // Collect changes - if (currentVersion && (line.startsWith('- ') || line.startsWith('* ') || line.match(/^\s+\*/))) { - const cleanedLine = sanitizeLine(line.trim().replace(/^[*-]\s*/, '')); - if (currentSection) { - sections[currentSection].push(cleanedLine); - } else { - if (!sections['Changes']) sections['Changes'] = []; - sections['Changes'].push(cleanedLine); - } + const versionMatch = line.match( + /^##\s*\[?([vV]?\d+\.\d+(?:\.(?:\d+|x))?)\]?(?:\([^)]*\))?\s*(?:-\s*(.+))?$/ + ); + if (versionMatch) { + flush(); + current = { + version: versionMatch[1], + date: (versionMatch[2] || '').trim(), + lines: [], + }; + skipping = false; + continue; } - } - // Add the last version - if (currentVersion && Object.keys(sections).length > 0) { - updates.push({ - version: currentVersion, - date: currentDate, - sections: sections, - }); + if (skipping || !current) continue; + current.lines.push(line); } - - // Fallback: if nothing parsed, create a single update with available content - if (updates.length === 0) { - console.warn(' ⚠ No versions parsed from changelog, creating fallback entry'); - const nonEmpty = lines.filter(l => l.trim().length).slice(0, 100); - updates.push({ - version: 'latest', - date: '', - sections: { - 'Changes': nonEmpty.map(l => sanitizeLine(l.trim())) - } - }); - } - - // Apply version filter if specified - if (versionFilter) { - return updates.filter(u => u.version.startsWith(versionFilter)); + flush(); + + // Fallback: a changelog with no recognizable version headers is emitted whole + // rather than dropped, so a format change surfaces as odd output, not silence. + if (updates.length === 0 && !versionFilter) { + const body = trimBlankEdges(dropEmptyBullets(normalizeBullets(escapeMdx(demoteHeadings(content.split('\n')))))); + if (body.length) { + console.warn(' ⚠ No version headers found; emitting changelog as one entry'); + return [{ version: 'latest', date: '', body: body.join('\n') }]; + } } - return updates; + return versionFilter + ? updates.filter(u => u.version.startsWith(versionFilter)) + : updates; } -// Generate Mintlify content function generateMintlifyContent(updates, repo, product, target) { const productLabel = product.toUpperCase(); const versionLabel = updates[0]?.version || target; @@ -332,20 +365,14 @@ mode: "wide" This page tracks releases and changes for ${versionLabel}. For the full release history, see the [CHANGELOG](${changelogUrl}) on GitHub. -${updates.map(update => { - const label = update.date || 'Release'; - const sectionsContent = Object.entries(update.sections) - .map(([sectionName, items]) => { - if (items.length === 0) return ''; - return `## ${sectionName}\n\n${items.map(item => `- ${item}`).join('\n')}`; - }) - .filter(s => s) - .join('\n\n'); - - return ` -${sectionsContent} +${updates + .map(update => { + const label = update.date || 'Release'; + return ` +${update.body} `; -}).join('\n\n')} + }) + .join('\n\n')} `; return content; @@ -403,7 +430,7 @@ async function generateChangelog(config, productConfig, target) { productConfig.changelogPath ); - let updates = parseChangelog(changelog, versionFilter, config.unreleasedAs); + let updates = sliceByVersion(changelog, versionFilter, config.unreleasedAs); if (config.currentOnly) updates = updates.slice(0, 1); diff --git a/scripts/versioning/verify-links-live.js b/scripts/versioning/verify-links-live.js new file mode 100644 index 000000000..e795852ac --- /dev/null +++ b/scripts/versioning/verify-links-live.js @@ -0,0 +1,190 @@ +#!/usr/bin/env node + +/** + * verify-links-live.js + * + * Requests every GitHub link in the docs and reports its real HTTP status, plus + * whether each `#Lnnn` anchor actually exists in the file. + * + * Usage: + * node verify-links-live.js sdk/latest cometbft/latest + * node verify-links-live.js sdk cometbft --json /tmp/live.json + * + * WHY THIS EXISTS + * + * `npx mint broken-links` validates internal page paths only. It never requests + * an external URL, so a dead GitHub link ships silently. check-github-refs.js + * infers liveness from the git tree, which is cheap and correct about paths but + * says nothing about whether a line anchor points past end of file: GitHub + * clamps `#L32` on a 31-line file rather than erroring, so a wrong anchor looks + * healthy. This makes the request and counts the lines. + */ + +import fs from 'fs'; +import path from 'path'; +import { fileURLToPath } from 'url'; + +const REPO_ROOT = path.join(path.dirname(fileURLToPath(import.meta.url)), '..', '..'); +const CONCURRENCY = 8; + +// Backtick is excluded from the path charset: a URL deliberately shown as inline +// code would otherwise absorb its own closing backtick into the path. +const LINK_RE = new RegExp( + 'https://github\\.com/([\\w.-]+/[\\w.-]+)/(blob|tree)/' + + '(release/v\\d+\\.\\d+\\.x|[^/\\s)"\'\\]`]+)' + + '/([^\\s)"\'\\]`#]+)' + + '(?:#L(\\d+)(?:-L(\\d+))?)?', + 'g' +); + +/** + * Is this occurrence an actual markdown link, or a URL shown as text? + * + * A dead URL that has deliberately been unlinked (shown as plain text or inline + * code, typically because the upstream repo moved and there is no successor) is + * not a broken link. Reporting it as one means the check can never go green, so + * the two cases have to be told apart. + */ +function isLinked(lineText, index) { + const before = lineText.slice(Math.max(0, index - 2), index); + return before.endsWith('](') || before.endsWith('(<'); +} + +function walk(dir, acc = []) { + if (!fs.existsSync(dir)) return acc; + for (const e of fs.readdirSync(dir, { withFileTypes: true })) { + const p = path.join(dir, e.name); + if (e.isDirectory()) walk(p, acc); + else if (/\.(mdx|md)$/.test(e.name)) acc.push(p); + } + return acc; +} + +function collect(roots) { + const seen = new Map(); + for (const root of roots) { + for (const f of walk(path.join(REPO_ROOT, root))) { + const rel = path.relative(REPO_ROOT, f); + const text = fs.readFileSync(f, 'utf8'); + text.split('\n').forEach((lineText, i) => { + for (const m of lineText.matchAll(LINK_RE)) { + const [url, repo, kind, ref, rawPath, l1, l2] = m; + const key = url; + if (!seen.has(key)) { + seen.set(key, { + url, repo, kind, ref, + path: decodeURIComponent(rawPath.replace(/[.,;:]+$/, '')), + anchor: l1 ? { start: +l1, end: l2 ? +l2 : null } : null, + sites: [], + linked: false, + }); + } + const rec = seen.get(key); + rec.sites.push(`${rel}:${i + 1}`); + // linked if ANY occurrence is a real markdown link + if (isLinked(lineText, m.index)) rec.linked = true; + } + }); + } + } + return [...seen.values()]; +} + +const headers = { 'User-Agent': 'cosmos-docs-verify-links' }; +if (process.env.GITHUB_TOKEN) headers.Authorization = `Bearer ${process.env.GITHUB_TOKEN}`; + +/** Line count per (repo, ref, path), so anchors can be range-checked. */ +const lineCache = new Map(); +async function lineCount(repo, ref, p) { + const key = `${repo}@${ref}/${p}`; + if (lineCache.has(key)) return lineCache.get(key); + let n = null; + try { + const r = await fetch(`https://raw.githubusercontent.com/${repo}/${ref}/${p}`, { headers }); + if (r.ok) n = (await r.text()).split('\n').length; + } catch { /* network */ } + lineCache.set(key, n); + return n; +} + +async function check(link) { + const out = { ...link, status: null, anchorOk: null, note: '' }; + try { + const r = await fetch(link.url, { headers, redirect: 'follow' }); + out.status = r.status; + } catch (e) { + out.status = 0; + out.note = e.message; + } + if (out.status === 200 && link.anchor && link.kind === 'blob') { + const n = await lineCount(link.repo, link.ref, link.path); + if (n === null) out.note = 'could not count lines'; + else { + const hi = link.anchor.end || link.anchor.start; + out.anchorOk = hi <= n; + if (!out.anchorOk) out.note = `anchor L${hi} exceeds ${n} lines`; + } + } + return out; +} + +async function main() { + const args = process.argv.slice(2); + const jsonAt = args.indexOf('--json'); + const jsonPath = jsonAt >= 0 ? args[jsonAt + 1] : null; + const roots = args.filter((a, i) => !a.startsWith('--') && i !== jsonAt + 1); + if (!roots.length) { + console.error('Usage: node verify-links-live.js [dir...] [--json out.json]'); + process.exit(1); + } + + const links = collect(roots); + console.log(`${links.length} unique GitHub URLs across ${roots.join(', ')}`); + console.log(`(${links.filter(l => l.anchor).length} carry line anchors)\n`); + + const results = []; + let done = 0; + const queue = [...links]; + await Promise.all( + Array.from({ length: CONCURRENCY }, async () => { + while (queue.length) { + const link = queue.shift(); + results.push(await check(link)); + if (++done % 50 === 0) console.log(` checked ${done}/${links.length}`); + } + }) + ); + + const allDead = results.filter(r => r.status !== 200); + const dead = allDead.filter(r => r.linked); + const deadUnlinked = allDead.filter(r => !r.linked); + const badAnchor = results.filter(r => r.anchorOk === false && r.linked); + console.log(`\n${results.length} checked`); + console.log(` ${results.length - allDead.length} returned 200`); + console.log(` ${dead.length} BROKEN LINKS (markdown links that do not resolve)`); + console.log(` ${badAnchor.length} resolve but the anchor is past end of file`); + console.log(` ${deadUnlinked.length} dead URLs shown as text, not linked (deliberate, not counted as broken)`); + + for (const r of dead) { + console.log(`\n[HTTP ${r.status}] ${r.url}`); + r.sites.forEach(s => console.log(` ${s}`)); + if (r.note) console.log(` ${r.note}`); + } + for (const r of badAnchor) { + console.log(`\n[BAD ANCHOR] ${r.url}`); + console.log(` ${r.note}`); + r.sites.forEach(s => console.log(` ${s}`)); + } + + if (jsonPath) { + fs.writeFileSync(jsonPath, JSON.stringify(results, null, 2)); + console.log(`\nwrote ${results.length} records to ${jsonPath}`); + } + if (deadUnlinked.length) { + console.log('\nDead URLs shown as text rather than linked (no action needed):'); + for (const r of deadUnlinked) console.log(` [${r.status}] ${r.url}\n ${r.sites[0]}`); + } + process.exit(dead.length || badAnchor.length ? 1 : 0); +} + +main(); diff --git a/sdk/latest/changelog/release-notes.mdx b/sdk/latest/changelog/release-notes.mdx index f93e5c0ee..7514cd2e7 100644 --- a/sdk/latest/changelog/release-notes.mdx +++ b/sdk/latest/changelog/release-notes.mdx @@ -5,111 +5,75 @@ mode: "wide" --- - This page tracks releases and changes for v0.54.1. For the full release history, see the [CHANGELOG](https://github.com/cosmos/cosmos-sdk/blob/main/CHANGELOG.md) on GitHub. + This page tracks releases and changes for v0.55.0. For the full release history, see the [CHANGELOG](https://github.com/cosmos/cosmos-sdk/blob/main/CHANGELOG.md) on GitHub. - -## Improvements - -- (x/auth) [#26297](https://github.com/cosmos/cosmos-sdk/pull/26297) Cap pagination limit at number of txs within block during `GetBlockWithTxs` instead of 100. - - - + ## Breaking Changes -- (x/consensus) [#25607](https://github.com/cosmos/cosmos-sdk/pull/25607) Add `AuthorityParams` to consensus params. When set, the consensus params authority takes precedence over per-keeper authority for all module parameter updates. Keeper constructor signatures are unchanged. -- (x/staking) [#25724](https://github.com/cosmos/cosmos-sdk/issues/25724) Validate `BondDenom` in `MsgUpdateParams` to prevent setting non-existent or zero-supply denoms. -- [#25778](https://github.com/cosmos/cosmos-sdk/pull/25778) Update `log` to log v2. -- [#25090](https://github.com/cosmos/cosmos-sdk/pull/25090) Moved deprecated modules to `./contrib`. These modules are still available but will no longer be actively maintained or supported in the Cosmos SDK Bug Bounty program. -- `x/group` -- `x/nft` -- `x/circuit` -- `x/crisis` -- (crypto) [#24414](https://github.com/cosmos/cosmos-sdk/pull/24414) Remove sr25519 support, since it was removed in CometBFT v1.x (see: CometBFT [#3646](https://github.com/cometbft/cometbft/pull/3646)). -- (x/mint) [#25599](https://github.com/cosmos/cosmos-sdk/pull/25599) Add max supply param. -- (x/gov) [#25615](https://github.com/cosmos/cosmos-sdk/pull/25615) Decouple `x/gov` from `x/staking` by making `CalculateVoteResultsAndVotingPowerFn` a required parameter to `keeper.NewKeeper` instead of `StakingKeeper`. -- (x/gov) [#25617](https://github.com/cosmos/cosmos-sdk/pull/25617) `AfterProposalSubmission` hook now includes proposer address as a parameter. -- (x/gov) [#25616](https://github.com/cosmos/cosmos-sdk/pull/25616) `DistrKeeper` `x/distribution` is now optional. Genesis validation ensures `distrKeeper` is set if distribution module is used as proposal cancel destination. -- (systemtests) [#25930]https://github.com/cosmos/cosmos-sdk/pull/25930) Move `systemtests` into `testutil` and no longer under its own `go.mod`. -- (baseapp) [#26060](https://github.com/cosmos/cosmos-sdk/pull/26060) Remove `BaseApp.SetStoreMetrics`. The `StoreMetrics` interface never worked, so removing dead code. -- (store) [#26061](https://github.com/cosmos/cosmos-sdk/pull/26061) Remove store tracing API and all related plumbing: -- Remove `SetTracer`, `SetTracingContext`, and `TracingEnabled` from `MultiStore` interface. -- Remove `CacheWrapWithTrace` from `CacheWrapper` interface. -- Remove `BaseApp.SetCommitMultiStoreTracer` and tracing context logic from `BaseApp.cacheTxContext` and `FinalizeBlock`. -- Remove `io.Writer` parameter from `servertypes.AppCreator` and `traceWriter io.Writer` from `servertypes.AppExporter`. -- Remove `traceStore io.Writer` parameter from `simapp.NewSimApp` and all enterprise simapp constructors. -- Remove `traceStore io.Writer` from all `testutil/simsx` app factory signatures. -- (store) [#26042](https://github.com/cosmos/cosmos-sdk/pull/26042) We are now importing `github.com/cosmos/cosmos-sdk/store/v2` as the store package instead of `cosmossdk.io/store` and all import paths have changed. -- (baseapp) [#26138](https://github.com/cosmos/cosmos-sdk/pull/26138) Default block gas meter to disabled. Adds checking to ensure block gas meter is not enabled while bstm parallel execution is configured and panics in these scenarios during parameter assignment. +- (mempool) [#25338](https://github.com/cosmos/cosmos-sdk/pull/25338) Respect gas wanted returned by the ante handler for block selection. Adds `InsertWithOption` to the `Mempool` interface (carries the ante-reported `GasWanted`) and changes the `SelectBy` callback to receive a `mempool.Tx` wrapper that exposes the stored value. +- (tx) [#26456](https://github.com/cosmos/cosmos-sdk/pull/26456) Remove `SIGN_MODE_TEXTUAL` and all associated implementation (`x/tx/signing/textual`, `x/auth/tx/textual.go`, `TextualCoinMetadataQueryFn`). The proto enum value is reserved to prevent future reuse. ADR-050 is marked archived. +- (modules) [#26421](https://github.com/cosmos/cosmos-sdk/pull/26421) Remove the `x/protocolpool` module and its API/proto surface from the SDK. Applications upgrading from v0.54 should include `protocolpool` in deleted store upgrades. +- (genutils) [#26468](https://github.com/cosmos/cosmos-sdk/pull/26468) Consolidate ExportGenesisFileWithTime arguments to preserve consensus params. ## Features -- [#25471](https://github.com/cosmos/cosmos-sdk/pull/25471) Full BLS 12-381 support enabled. -- [#24872](https://github.com/cosmos/cosmos-sdk/pull/24872) Support BLS 12-381 for cli `init`, `gentx`, `collect-gentx` -- (crypto) [#24919](https://github.com/cosmos/cosmos-sdk/pull/24919) add `NewPubKeyFromBytes` function to the `secp256r1` package to create `PubKey` from bytes -- (server) [#24720](https://github.com/cosmos/cosmos-sdk/pull/24720) add `verbose_log_level` flag for configuring the log level when switching to verbose logging mode during sensitive operations (such as chain upgrades). -- (crypto) [#24861](https://github.com/cosmos/cosmos-sdk/pull/24861) add `PubKeyFromCometTypeAndBytes` helper function to convert from `comet/v2` PubKeys to the `cryptotypes.Pubkey` interface. -- (abci_utils) [#25008](https://github.com/cosmos/cosmos-sdk/pull/25008) add the ability to assign a custom signer extraction adapter in `DefaultProposalHandler`. -- (x/distribution) [#25650](https://github.com/cosmos/cosmos-sdk/pull/25650) Add new gRPC query endpoints and CLI commands for `DelegatorStartingInfo`, `ValidatorHistoricalRewards`, and `ValidatorCurrentRewards`. -- [#25745](https://github.com/cosmos/cosmos-sdk/pull/25745) Add DiskIO telemetry via gopsutil. -- (grpc) [#25648](https://github.com/cosmos/cosmos-sdk/pull/25648) Add `earliest_block_height` and `latest_block_height` fields to `GetSyncingResponse`. -- (collections/codec) [#25614] (https://github.com/cosmos/cosmos-sdk/pull/25827) Add `TimeValue` (`ValueCodec[time.Time]`) to collections/codec. -- (enterprise/poa) [#25838](https://github.com/cosmos/cosmos-sdk/pull/25838) Add the `poa` module under the `enterprise` directory. -- (grpc) [#25850](https://github.com/cosmos/cosmos-sdk/pull/25850) Add `GetBlockResults` and `GetLatestBlockResults` gRPC endpoints to expose CometBFT block results including `finalize_block_events`. +- (abci) [#25620](https://github.com/cosmos/cosmos-sdk/pull/25620) Add support for new application side mempool ABCI methods. +- (abci) [#25969](https://github.com/cosmos/cosmos-sdk/pull/25969) Add support for new ABCI methods, `InsertTx` and `ReapTxs`. +- (blockstm) [#26208](https://github.com/cosmos/cosmos-sdk/pull/26208) Add Block-STM configuration support: `block-executor`, `block-stm-workers` and `block-stm-pre-estimate`. +- (blockstm) [#25909](https://github.com/cosmos/cosmos-sdk/pull/25909) Cache pre-state to optimize value-based validation. +- (deps) [#26388](https://github.com/cosmos/cosmos-sdk/pull/26388) Bump CometBFT version to v0.39.3. +- (staking) [#26440](https://github.com/cosmos/cosmos-sdk/pull/26440) Add basic key rotation for validator consensus keys. +- (crypto) [#26436](https://github.com/cosmos/cosmos-sdk/pull/26436) Add ML-DSA-65 (FIPS 204) post-quantum validator consensus key type, with SDK key wrappers, Amino + interface-registry registration, multisig support, and a `hd.MlDsa65Type` constant. +- (blockstm) [#26467](https://github.com/cosmos/cosmos-sdk/pull/26467) Track existence for `Has()` reads to reduce false conflicts. +- (staking) [#26485](https://github.com/cosmos/cosmos-sdk/pull/26485) Add `key_rotation_fee` to `x/staking` params and register associated 5->6 migration. +- (staking) [#26461](https://github.com/cosmos/cosmos-sdk/pull/26461) Wire `MsgRotateConsPubKey` into cli and add a happy path system test. +- (staking) [#26471](https://github.com/cosmos/cosmos-sdk/pull/26471) Add genesis import/export support for validator consensus key rotation. +- (crypto) [#26472](https://github.com/cosmos/cosmos-sdk/pull/26472) Add ML-DSA-65 (FIPS 204) support for user account keys: mnemonic-based keyring creation/recovery (`--algo ml_dsa_65`), transaction signing/verification, and an ante-handler signature-verification gas cost (`Params.SigVerifyCostMlDsa65`). +- (enterprise/poa) [#26590](https://github.com/cosmos/cosmos-sdk/pull/26590) Add `MsgRotateConsPubKey` for POA validator consensus key rotation (operator self-service plus admin override). +- (enterprise/poa) [#26614](https://github.com/cosmos/cosmos-sdk/pull/26614) Add ML-DSA-65 (mldsa65) validator key support to the PoA module via a `WithMlDsa65Support()` module option, raising `MaxPubKeyLength` to accommodate the larger keys. +- (crypto) [#26615](https://github.com/cosmos/cosmos-sdk/pull/26615) Add `secp256k1eth` validator consensus key type. ## Improvements -- (ci) Use softprops/action-gh-release for main-nightly instead of custom gh/git to avoid repository ruleset conflicts. -- (telemetry) [#26006](https://github.com/cosmos/cosmos-sdk/pull/26006) Export `ExtensionOptions` type for programmatic otel.yaml generation. -- [#25955](https://github.com/cosmos/cosmos-sdk/pull/25955) Use cosmos/btree directly instead of replacing it in go.mods -- (types) [#25342](https://github.com/cosmos/cosmos-sdk/pull/25342) Undeprecated `EmitEvent` and `EmitEvents` on the `EventManager`. These functions will continue to be maintained. -- (types) [#24668](https://github.com/cosmos/cosmos-sdk/pull/24668) Scope the global config to a particular binary so that multiple SDK binaries can be properly run on the same machine. -- (baseapp) [#24655](https://github.com/cosmos/cosmos-sdk/pull/24655) Add mutex locks for `state` and make `lastCommitInfo` atomic to prevent race conditions between `Commit` and `CreateQueryContext`. -- (proto) [#24161](https://github.com/cosmos/cosmos-sdk/pull/24161) Remove unnecessary annotations from `x/staking` authz proto. -- (x/bank) [#24660](https://github.com/cosmos/cosmos-sdk/pull/24660) Improve performance of the `GetAllBalances` and `GetAccountsBalances` keeper methods. -- (collections) [#25464](https://github.com/cosmos/cosmos-sdk/pull/25464) Add `IterateRaw` method to `Multi` index type to satisfty query `Collection` interface. -- (api) [#25613](https://github.com/cosmos/cosmos-sdk/pull/25613) Separated deprecated modules into the contrib directory, distinct from api, to enable and unblock new proto changes without affecting legacy code. -- (server) [#25740](https://github.com/cosmos/cosmos-sdk/pull/25740) Add variadic `grpc.DialOption` parameter to `StartGrpcServer` for custom gRPC client connection options. -- (blockstm) [#25765](https://github.com/cosmos/cosmos-sdk/pull/25765) Minor code readability improvement in block-stm. -- (blockstm) [#25786](https://github.com/cosmos/cosmos-sdk/pull/25786) Add pre-state checking in transaction state transition. -- (server/config) [#25807](https://github.com/cosmos/cosmos-sdk/pull/25807) fix(server): reject overlapping historical gRPC block ranges. -- [#25857](https://github.com/cosmos/cosmos-sdk/pull/25857) Reduce scope of mutex in `PriorityNonceMempool.Remove`. -- (baseapp) [#25862](https://github.com/cosmos/cosmos-sdk/pull/25862) Skip running validateBasic for rechecking txs. (Backport of https://github.com/cosmos/cosmos-sdk/pull/20208). -- (blockstm) [25883](https://github.com/cosmos/cosmos-sdk/pull/25883) Re-use decoded tx object in pre-estimates. -- (blockstm) [#25788](https://github.com/cosmos/cosmos-sdk/pull/25788) Only validate transactions that's executed at lease once. -- (blockstm) [#25767](https://github.com/cosmos/cosmos-sdk/pull/25767) Optimize block-stm MVMemory with bitmap index. +- (server/config) [#26572](https://github.com/cosmos/cosmos-sdk/pull/26572) Warn that `query-gas-limit = 0` (the default) is unbounded and exposes public RPC nodes to DoS via expensive queries. +- (docs) [#25918](https://github.com/cosmos/cosmos-sdk/issues/25918) Regenerate Swagger API spec to reflect current proto state, including `authority` field on consensus params and removal of stale module-config definitions. +- (baseapp) [#22368](https://github.com/cosmos/cosmos-sdk/issues/22368) Add `-race`-mode regression test (`TestABCI_Race_GRPC_Query_During_Commit`) covering concurrent `BaseApp.Query` and `FinalizeBlock`/`Commit`. Pins down the state-management mutex work added in #24655 and follow-ups so the data race reported against v0.50.x cannot regress silently. +- (x/staking, x/slashing) [#26481](https://github.com/cosmos/cosmos-sdk/pull/26481) Resolve evidence against recently rotated consensus keys and migrate slashing signing state to the active consensus key. +- (x/auth/tx) [#25221](https://github.com/cosmos/cosmos-sdk/issues/25221) Add `ConfigOptions.AminoJSONEncoder` so applications can configure a custom `aminojson.Encoder` (e.g. custom field encodings) for the `SIGN_MODE_LEGACY_AMINO_JSON` handler without replicating the SDK's `HandlerMap` construction. +- chore(x/auth) [#26567](https://github.com/cosmos/cosmos-sdk/pull/26567): add a human-readable error +- (blockstm) [#26592](https://github.com/cosmos/cosmos-sdk/pull/26592) Validate `ExecuteBlock` inputs (block size, store index mapping, and estimates) at the exported entry point so invalid input returns a descriptive error instead of an opaque "index out of range" panic. +- (cli) [#26604](https://github.com/cosmos/cosmos-sdk/pull/26604) Add consensus key algo to init and testnet CLIs. +- (crypto) [#26626](https://github.com/cosmos/cosmos-sdk/pull/26626) Update mldsa65 PubKey Address logic to validate length, remove unpacking. +- (staking) [#26619](https://github.com/cosmos/cosmos-sdk/pull/26619) Emit `rotate_cons_pubkey` and `apply_cons_pubkey_rotation` events during key rotation. ## Bug Fixes -- (baseapp) [#25331](https://github.com/cosmos/cosmos-sdk/issues/25331) Avoid noisy errors when gRPC response headers are already sent, set block height as a header when possible and fall back to a trailer. -- (blockstm) [#25789](https://github.com/cosmos/cosmos-sdk/issues/25789) Wake up suspended executors when scheduler doesn't complete to prevent goroutine leaks. -- (grpc) [#25647](https://github.com/cosmos/cosmos-sdk/pull/25647) Return actual `earliest_store_height` in `node.Status` gRPC endpoint instead of hardcoded `0`. -- (types/query) [#25665](https://github.com/cosmos/cosmos-sdk/issues/25665) Fix pagination offset when querying a collection with predicate function. -- (x/staking) [#25649](https://github.com/cosmos/cosmos-sdk/pull/25649) Add missing `defer iterator.Close()` calls in `IterateDelegatorRedelegations` and `GetRedelegations` to prevent resource leaks. -- (mempool) [#25563](https://github.com/cosmos/cosmos-sdk/pull/25563) Cleanup sender indices in case of tx replacement. -- (x/epochs) [#25425](https://github.com/cosmos/cosmos-sdk/pull/25425) Fix `InvokeSetHooks` being called with a nil keeper and `AppModule` containing a copy instead of a pointer (hooks set post creating the `AppModule` like with depinject didn't apply because it's a different instance). -- (client, client/rpc, x/auth/tx) [#24551](https://github.com/cosmos/cosmos-sdk/pull/24551) Handle cancellation properly when supplying context to client methods. -- (x/authz) [#24638](https://github.com/cosmos/cosmos-sdk/pull/24638) Fixed a minor bug where the grant key was cast as a string and dumped directly into the error message leading to an error string possibly containing invalid UTF-8. -- (client, client/rpc, x/auth/tx) [#24551](https://github.com/cosmos/cosmos-sdk/pull/24551) Handle cancellation properly when supplying context to client methods. -- (x/epochs) [#24770](https://github.com/cosmos/cosmos-sdk/pull/24770) Fix register of epoch hooks in `InvokeSetHooks`. -- (x/epochs) [#25087](https://github.com/cosmos/cosmos-sdk/pull/25087) Remove redundant error check in BeginBlocker. -- [GHSA-p22h-3m2v-cmgh](https://github.com/cosmos/cosmos-sdk/security/advisories/GHSA-p22h-3m2v-cmgh) Fix x/distribution can halt when historical rewards overflow. -- (x/staking) [#25258](https://github.com/cosmos/cosmos-sdk/pull/25258) Add delegator address to redelegate event. -- (x/bank) [#25751](https://github.com/cosmos/cosmos-sdk/pull/25751) Fix recipient address in events. -- (client) [#25811] (https://github.com/cosmos/cosmos-sdk/pull/25811) fix(client): fix file handle leaks in snapshot commands. -- (server/config) [#25806](https://github.com/cosmos/cosmos-sdk/pull/25806) fix: add missing commas in historical gRPC config template. -- (client) [#25804](https://github.com/cosmos/cosmos-sdk/pull/25804) Add `GetHeightFromMetadataStrict` API to `grpc` client for better error handling. -- (x/staking) [#25829](https://github.com/cosmos/cosmos-sdk/pull/25829) Validates case-sensitivity on authz grands in x/staking. -- (mempool) [#25869](https://github.com/cosmos/cosmos-sdk/pull/25869) fix(mempool): add thread safety to NextSenderTx. -- (blockstm) [#25912](https://github.com/cosmos/cosmos-sdk/pull/25912) Remove `SigVerificationDecorator` signature incarnation cache causing state divergence under blockstm. -- (x/group) [#25922](https://github.com/cosmos/cosmos-sdk/pull/25922) Add zero-total-weight check for ThresholdDecisionPolicy -- (x/group) [#25917](https://github.com/cosmos/cosmos-sdk/pull/25917) Prevent creation of zero-weight groups. -- (x/group) [#25919](https://github.com/cosmos/cosmos-sdk/pull/25919) add safer type assertions to group `DecisionPolicy` getter calls. -- (x/group) [#25920](https://github.com/cosmos/cosmos-sdk/pull/25920) Expand voting period check to verify period is positive instead of nonzero. -- (baseapp) [#26063](https://github.com/cosmos/cosmos-sdk/pull/26063) Fixes an issue where values embedded in context during ante handling were wiped after the handlers returned. - -## Deprecated - -- [#25948](https://github.com/cosmos/cosmos-sdk/pull/25948) Change default `app.go` code to not use `depinject` as we are phasing it out. -- (baseapp) [#26107](https://github.com/cosmos/cosmos-sdk/pull/26170) Deprecate baseapp test helper `app.NewUncachedContext`, consider using `app.NewNextBlockContext` or `app.NewContext` instead, see `UPGRADING.md` for more details. +- (codec) [#26587](https://github.com/cosmos/cosmos-sdk/pull/26587) Lower the nested `google.protobuf.Any` recursion depth cap in unknown-field validation from 10,000 to 64, reducing CPU-amplification DoS risk from deeply nested `Any` wrappers. No legitimate message nests `Any` anywhere near that deep. +- (x/feegrant) [#26596](https://github.com/cosmos/cosmos-sdk/pull/26596) Honor the `PageRequest` offset and `count_total` in the `Allowances` and `AllowancesByGranter` gRPC queries, which previously collected grants inside the pagination predicate and so returned offset-skipped and beyond-limit results. +- (x/authz) [#26588](https://github.com/cosmos/cosmos-sdk/pull/26588) Cap the number of expired grants pruned per `BeginBlocker` call to 200, matching `x/feegrant`'s existing pattern, so a block where many grants expire at once can't cause unbounded work. +- (client) [#26524](https://github.com/cosmos/cosmos-sdk/pull/26524) Fix file handle leak in the `snapshot dump` command where chunk files were deferred-closed inside the loop, keeping every chunk's handle open until the command returned (follow-up to #25811). +- (x/distribution) [#26518](https://github.com/cosmos/cosmos-sdk/pull/26518) Return an error from internal historical rewards reads when the record is absent, preventing recovered reference-count panics during BlockSTM speculative execution. +- (x/auth) [#26515](https://github.com/cosmos/cosmos-sdk/pull/26515) Bound the pubkey and signature indices in `ConsumeMultisignatureVerificationGas` and `VerifyMultisignature` so a multisig signature with a bit array larger than the key set, or with more set bits than supplied signatures, returns an error instead of panicking with index out of range. +- (x/distribution) [#26406](https://github.com/cosmos/cosmos-sdk/pull/26406) Add fallback paths (delegator/validator owner, then community pool) when withdrawing delegator rewards or validator commission to a blocked address during `Begin/EndBlockers`. user msg initiated paths still return `ErrUnauthorized` when withdrawing to blocked addresses. +- (x/gov) [#26353](https://github.com/cosmos/cosmos-sdk/pull/26353) Fix leading comma in `proposal_messages` event attribute emitted by `SubmitProposal`. +- (telemetry) [#26390](https://github.com/cosmos/cosmos-sdk/pull/26390) Fix env var for otel telemetry initialization. +- (x/staking) [#26408](https://github.com/cosmos/cosmos-sdk/pull/26408) Fix `MsgBeginRedelegate` failure when redelegating all shares from an unbonded source validator that is removed after unbonding. +- (x/auth/tx) [#26422](https://github.com/cosmos/cosmos-sdk/pull/26422) Reuse the signing context from the codec's `InterfaceRegistry` when `ConfigOptions.SigningOptions` is unset so that `CustomGetSigners` registered via `NewInterfaceRegistryWithOptions` are honored by `NewTxConfig` / `NewTxConfigWithOptions`. +- (x/staking) [#26460](https://github.com/cosmos/cosmos-sdk/pull/26460) Coalesce key rotation power updates to not emit duplicates. +- (x/staking) [#26483](https://github.com/cosmos/cosmos-sdk/pull/26483) Block `MsgCreateValidator` from creating validators with cons addrs locked by key rotations. +- (blockstm) [#25893](https://github.com/cosmos/cosmos-sdk/pull/25893) Fix CancelAll cancellation by clearing blocker ESTIMATE marks before waking suspended executors. +- (crypto) [#26529](https://github.com/cosmos/cosmos-sdk/pull/26529) Validate the SEC1 tag byte (`0x02`/`0x03`) when unmarshaling a `secp256k1.PubKey`, rejecting malformed compressed keys that previously passed the length-only check. +- (x/auth/tx) [#26571](https://github.com/cosmos/cosmos-sdk/pull/26571) Avoid nil pointer panic in `GetSigningTxData` for multisig `ModeInfo` with a nil `Multi` or nil `Bitarray`. +- (x/auth/tx) [#26527](https://github.com/cosmos/cosmos-sdk/pull/26527) Fix nil pointer panic in `GetSigningTxData` when a `SignerInfo` has a nil `PublicKey`. +- (x/auth/tx) [#26517](https://github.com/cosmos/cosmos-sdk/pull/26517) Return a decode error instead of panicking when a transaction's `SignerInfos` and `Signatures` counts disagree in `GetSignaturesV2`, or a multisig's `ModeInfos` and sub-signature counts disagree in `ModeInfoAndSigToSignatureData`. +- (x/auth/ante) [#26573](https://github.com/cosmos/cosmos-sdk/pull/26573) Reject tx with extra SignerInfos in SetPubKeyDecorator. +- (block-stm) [#26583](https://github.com/cosmos/cosmos-sdk/pull/26583) Fix count validation tasks before advancing validationIdx to prevent lost updates. +- (blockstm) [#26591](https://github.com/cosmos/cosmos-sdk/pull/26591) normalize non-positive worker count in `STMRunner.Run`. +- (x/staking) [#26613](https://github.com/cosmos/cosmos-sdk/pull/26613) Require `key_rotation_fee` denom to equal `bond_denom` in `Params.Validate` and derive the default fee denom from the configured bond denom. +- (x/staking) [#26611](https://github.com/cosmos/cosmos-sdk/pull/26611) Fix missing key rotation type tags on genesis import. +- (blockstm) [#26627](https://github.com/cosmos/cosmos-sdk/pull/26627) Guard against block-stm estimate panic. +- (x/staking) [#26616](https://github.com/cosmos/cosmos-sdk/pull/26616) Expire historical cons addr lookups only once equivocation evidence is no longer admissible. +- (x/poa) [#26642](https://github.com/cosmos/cosmos-sdk/pull/26642) Always return error when migrating fees to an occupied key. +- (x/staking) [#26641](https://github.com/cosmos/cosmos-sdk/pull/26641) Allow multiple history entires in genesis import, and fix labeling of historical entries. diff --git a/sdk/latest/enterprise/group/overview.mdx b/sdk/latest/enterprise/group/overview.mdx index dcc20a138..1f016a6aa 100644 --- a/sdk/latest/enterprise/group/overview.mdx +++ b/sdk/latest/enterprise/group/overview.mdx @@ -16,7 +16,7 @@ The Group module is designed for networks that require: ## Source Code -The source code for the Group module can be found [here](https://github.com/cosmos/cosmos-sdk/tree/main/enterprise/group). +The source code for the Group module can be found [here](https://github.com/cosmos/cosmos-sdk/tree/release/v0.55.x/enterprise/group). ## Available Documentation @@ -27,6 +27,6 @@ This section contains detailed documentation for the Group module. ## Licensing -The Group module source is published under the [Source Available Evaluation License](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/group/LICENSE), which permits evaluation and testing in non-production environments only. Production or commercial use requires an Enterprise License from Cosmos Labs. +The Group module source is published under the [Source Available Evaluation License](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/group/LICENSE), which permits evaluation and testing in non-production environments only. Production or commercial use requires an Enterprise License from Cosmos Labs. To use the Group module in production, contact sales@cosmoslabs.io. diff --git a/sdk/latest/enterprise/overview.mdx b/sdk/latest/enterprise/overview.mdx index 35b164b2b..4f84bb79a 100644 --- a/sdk/latest/enterprise/overview.mdx +++ b/sdk/latest/enterprise/overview.mdx @@ -5,7 +5,7 @@ description: "Source-available Cosmos SDK modules for permissioned and enterpris Cosmos Enterprise modules are production-ready modules for permissioned networks, institutional chains, and enterprise deployments that need features beyond a public blockchain architecture. They follow the same patterns as the core modules and integrate alongside them. -The module source is published in the [`enterprise` directory of the Cosmos SDK repository](https://github.com/cosmos/cosmos-sdk/tree/main/enterprise). +The module source is published in the [`enterprise` directory of the Cosmos SDK repository](https://github.com/cosmos/cosmos-sdk/tree/release/v0.55.x/enterprise). ## Available modules diff --git a/sdk/latest/enterprise/poa/api.mdx b/sdk/latest/enterprise/poa/api.mdx index 2c8a83a63..2c033591e 100644 --- a/sdk/latest/enterprise/poa/api.mdx +++ b/sdk/latest/enterprise/poa/api.mdx @@ -502,6 +502,46 @@ simd tx poa withdraw-fees \ --- +### RotateConsPubKey + +Replace a validator's consensus public key in place. + +**gRPC:** `cosmos.poa.v1.Msg/RotateConsPubKey` + +**Message:** +```protobuf +message MsgRotateConsPubKey { + string sender = 1; // Signer: the validator's operator or the chain admin + string validator_address = 2; // Operator address identifying the validator + google.protobuf.Any new_pub_key = 3; // New consensus public key +} +``` + +**Response:** +```protobuf +message MsgRotateConsPubKeyResponse {} +``` + +**CLI:** +```bash +simd tx poa rotate-cons-pub-key \ + --operator-address \ + --from \ + -y +``` + +**Authorization:** Must be signed by the validator's operator address or the chain admin. + +**Notes:** +- Re-keys the validator and migrates its accrued fees in the same block +- Power, metadata, and the operator address are unchanged +- No fee, no rate limit, and no rotation history, unlike `x/staking` rotation +- The new key's type must be in the chain's consensus params, must not equal the current key, and must not belong to another validator + +For the operational procedure, see [Rotate a consensus key, PoA](/sdk/latest/keys/rotate-validator-key-poa). + +--- + ## Common Use Cases ### 1. Query Current Admin diff --git a/sdk/latest/enterprise/poa/architecture.mdx b/sdk/latest/enterprise/poa/architecture.mdx index a8f613291..15a07aacc 100644 --- a/sdk/latest/enterprise/poa/architecture.mdx +++ b/sdk/latest/enterprise/poa/architecture.mdx @@ -9,6 +9,10 @@ description: "System architecture and module integration details for the PoA mod The Proof of Authority (PoA) permissioned consensus module is a Cosmos SDK module that implements a permissioned consensus mechanism where a designated admin controls the validator set. Unlike traditional Proof of Stake systems, PoA validators are explicitly authorized and managed by an administrative authority rather than being selected based on staked tokens. + +Vote extensions are not supported on PoA chains. A consensus key rotation leaves the SDK-side consensus address out of sync with CometBFT for two heights, and during that window the chain cannot verify a rotating validator's vote-extension signatures. If more than 2/3 of voting power rotates within one window, the chain halts. See [Key rotation](/sdk/latest/keys/key-rotation) for the mechanism. + + ## Table of Contents - [Architecture](#architecture) @@ -74,10 +78,9 @@ Standard SDK [governance](/sdk/latest/modules/gov/README) uses bonded tokens for **Storage Design Philosophy** -The module uses `cosmossdk.io/collections` with a composite key structure: -- Primary key: `(power, consensus_address)` enables efficient power-sorted iteration -- Secondary indexes on consensus and operator addresses for fast lookups -- Requires re-keying when power changes, but eliminates need for separate sorting +The module uses `cosmossdk.io/collections` with an indexed map: +- Primary key: consensus address +- Secondary indexes on operator address and power, for fast lookups and power-sorted iteration - See [Storage Design](#storage-design) for technical details ## Admin Control Flow @@ -91,13 +94,13 @@ The PoA module is controlled by a single admin address configured at genesis. Th The admin could be set to any authority that has an address. This includes a group from x/groups, the governance module account, and multisigs. -**Location**: Admin address stored in [`x/poa/types/keys.go:10`](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/x/poa/types/keys.go#L10) (params prefix) +**Location**: Admin address stored in [`x/poa/types/keys.go:26`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/poa/x/poa/types/keys.go#L26) (params prefix) Only the admin can update itself with a parameter change. ### Managing Validator Set -**MsgUpdateValidators** ([`x/poa/keeper/msg_server.go:72`](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/x/poa/keeper/msg_server.go#L72)) +**MsgUpdateValidators** ([`x/poa/keeper/msg_server.go:142`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/poa/x/poa/keeper/msg_server.go#L142)) The admin can batch update validators through a single transaction: @@ -111,11 +114,11 @@ The admin can batch update validators through a single transaction: - Fee checkpoint (allocates pending fees before power changes) - Total power recalculation - ABCI validator update queue -4. **Consensus Update**: Changes take effect at the end of the current block +4. **Consensus Update**: Changes take effect in the next block ### Updating Parameters -**MsgUpdateParams** ([`x/poa/keeper/msg_server.go:26`](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/x/poa/keeper/msg_server.go#L26)) +**MsgUpdateParams** ([`x/poa/keeper/msg_server.go:45`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/poa/x/poa/keeper/msg_server.go#L45)) The admin can update module parameters (currently only the admin address itself). This requires: - Transaction signed by current admin @@ -125,9 +128,9 @@ The admin can update module parameters (currently only the admin address itself) ### Validator Registration -**MsgCreateValidator** ([`x/poa/keeper/msg_server.go:45`](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/x/poa/keeper/msg_server.go#L45)) +**MsgCreateValidator** ([`x/poa/keeper/msg_server.go:79`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/poa/x/poa/keeper/msg_server.go#L79)) -**Permissionless Creation**: Any address can register as a validator candidate: +**Admin-Only Creation**: Only the admin can register a validator: 1. **Submit Registration**: Provide public key and metadata - **PubKey**: Ed25519 @@ -135,12 +138,12 @@ The admin can update module parameters (currently only the admin address itself) - **Moniker**: Human-readable name (max 256 chars) - **Description**: Additional details (max 256 chars) -2. **Initial State**: Created validators have **power = 0** until the admin updates it via `MsgUpdateValidators` +2. **Initial State**: The admin sets the validator's initial power. A validator with **power = 0** is inactive: - Not participating in consensus - Not earning fees - Cannot vote in governance -**Location**: [`x/poa/keeper/validator.go:95`](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/x/poa/keeper/validator.go#L95) +**Location**: [`x/poa/keeper/validator.go:121`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/poa/x/poa/keeper/validator.go#L121) ### Gaining Consensus Power @@ -158,7 +161,7 @@ Validators can only gain consensus power through admin action: - Power can be adjusted up or down by admin - Setting power = 0 removes validator from consensus without deleting -**Location**: [`x/poa/keeper/validator.go:19`](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/x/poa/keeper/validator.go#L19) +**Location**: [`x/poa/keeper/validator.go:34`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/poa/x/poa/keeper/validator.go#L34) ### Removing Validators @@ -183,7 +186,7 @@ The PoA module implements a custom checkpoint-based fee distribution system that **See [Fee Distribution Documentation](/sdk/latest/enterprise/poa/distribution)** for complete details. -**Location**: [`x/poa/keeper/distribution.go`](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/x/poa/keeper/distribution.go) +**Location**: [`x/poa/keeper/distribution.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/poa/x/poa/keeper/distribution.go) ## Governance @@ -200,31 +203,31 @@ The PoA module restricts governance participation to active validators only, usi **See [Governance Documentation](/sdk/latest/enterprise/poa/governance)** for complete details. -**Location**: [`x/poa/keeper/governance.go`](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/x/poa/keeper/governance.go) and [`x/poa/keeper/hooks.go`](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/x/poa/keeper/hooks.go) +**Location**: [`x/poa/keeper/governance.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/poa/x/poa/keeper/governance.go) and [`x/poa/keeper/hooks.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/poa/x/poa/keeper/hooks.go) ## Technical Implementation ### Storage Design -**Collections Schema** ([`x/poa/types/keys.go`](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/x/poa/types/keys.go)) +**Collections Schema** ([`x/poa/types/keys.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/poa/x/poa/types/keys.go)) The module uses `cosmossdk.io/collections` for type-safe state management: | Prefix | Collection | Key Type | Value Type | Purpose | |--------|------------|----------|------------|---------| | 0 | `params` | - | `Params` | Admin address and module config | -| 1 | `validators` | `(int64, string)` | `Validator` | Primary map, sorted by power | -| 2 | `validator_by_consensus` | `string` | `(int64, string)` | Index: consensus addr → composite key | -| 3 | `validator_by_operator` | `string` | `(int64, string)` | Index: operator addr → composite key | +| 1 | `validators` | `ConsAddress` | `Validator` | Primary map, keyed by consensus address | +| 2 | `validator_by_operator` | `string` | `ConsAddress` | Index: operator addr → consensus addr | +| 3 | `validator_by_power` | `(int64, ConsAddress)` | - | Index: power-sorted iteration | | 4 | `total_power` | - | `int64` | Sum of all validator power | | 5 | `total_allocated` | - | `ValidatorFees` | Sum of allocated fees | -**Location**: [`x/poa/keeper/keeper.go:16`](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/x/poa/keeper/keeper.go#L16) +**Location**: [`x/poa/keeper/keeper.go:38`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/poa/x/poa/keeper/keeper.go#L38) ### ABCI Integration -**EndBlocker** ([`x/poa/keeper/abci.go:9`](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/x/poa/keeper/abci.go#L9)) +**EndBlocker** ([`x/poa/keeper/abci.go:30`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/poa/x/poa/keeper/abci.go#L30)) The module integrates with CometBFT consensus through ABCI: @@ -242,7 +245,7 @@ ValidatorUpdate { } ``` -**Location**: [`x/poa/module.go:128`](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/x/poa/module.go#L128) +**Location**: [`x/poa/module.go:266`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/poa/x/poa/module.go#L266) ## Security Considerations @@ -250,8 +253,8 @@ ValidatorUpdate { - Admin address controls entire validator set 2. **Validator Registration**: - - Anyone can register as validator candidate - - Only admin can grant consensus power + - Only the admin can register a validator + - Only the admin can grant consensus power 3. **Total Power Invariant**: - Total power must remain > 0 diff --git a/sdk/latest/enterprise/poa/distribution.mdx b/sdk/latest/enterprise/poa/distribution.mdx index 5f2bfaaf5..e4f3d1678 100644 --- a/sdk/latest/enterprise/poa/distribution.mdx +++ b/sdk/latest/enterprise/poa/distribution.mdx @@ -13,7 +13,7 @@ The PoA module implements a custom fee distribution mechanism based on validator Fees flow through the PoA system differently than standard Cosmos SDK: -1. **Block Fees**: Transaction fees collected in each block go to the `fee_collector` module account by default, or to the PoA module account if configured (see [Fee Routing Setup](#fee-routing-setup)) +1. **Block Fees**: Transaction fees collected in each block go to the PoA module account (see [Fee Routing Setup](#fee-routing-setup)) 2. **Checkpoint System**: Allocated fees are updated for validators when: - Any validator power changes @@ -21,7 +21,7 @@ Fees flow through the PoA system differently than standard Cosmos SDK: **Why Checkpointing?**: Ensures fair distribution when power changes. If power changes mid-period, fees are allocated based on old power distribution before the change takes effect. -**Location**: [`x/poa/keeper/distribution.go:18`](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/x/poa/keeper/distribution.go#L18) +**Location**: [`x/poa/keeper/distribution.go:32`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/poa/x/poa/keeper/distribution.go#L32) ## Distribution Algorithm @@ -119,7 +119,7 @@ After this checkpoint, $A_{total}(t+1) = B_{collector}(t)$ (all fees are now all ## Withdrawing Fees -**MsgWithdrawFees** ([`x/poa/keeper/msg_server.go:91`](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/x/poa/keeper/msg_server.go#L91)) +**MsgWithdrawFees** ([`x/poa/keeper/msg_server.go:182`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/poa/x/poa/keeper/msg_server.go#L182)) Any validator operator can withdraw accumulated fees: @@ -137,7 +137,7 @@ Withdrawal: 100 utokens transferred to operator Remainder: 0.7543 utokens remain allocated (less than least significant utoken digit) ``` -**Location**: [`x/poa/keeper/distribution.go:106`](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/x/poa/keeper/distribution.go#L106) +**Location**: [`x/poa/keeper/distribution.go:135`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/poa/x/poa/keeper/distribution.go#L135) ## Withdrawal Formula @@ -164,7 +164,7 @@ Where: ## Fee Routing Setup -PoA has its own module account for collecting fees. Enabling the PoA module account is recommended to keep fee accounting isolated and accurate. If not enabled, fees are deposited into the standard `fee_collector` account by default. +PoA has its own module account for collecting fees. Enabling the PoA module account is required to keep fee accounting isolated and accurate. The chain panics at block 1 if the ante handler's fee recipient is not the PoA module. To enable the PoA module account, two wiring changes are required: @@ -186,7 +186,7 @@ app.AccountKeeper = authkeeper.NewAccountKeeper( ) ``` -**Source**: [`simapp/app.go`](https://github.com/cosmos/cosmos-sdk/blob/7bc1b146d437d834d971f415924104188203c96f/enterprise/poa/simapp/app.go#L191) +**Source**: [`simapp/app.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/poa/simapp/app.go#L182-L194) ### 2. Configure the Ante Handler @@ -210,9 +210,7 @@ anteDecorators := []sdk.AnteDecorator{ } ``` -**Source**: [`simapp/ante.go`](https://github.com/cosmos/cosmos-sdk/blob/7bc1b146d437d834d971f415924104188203c96f/enterprise/poa/simapp/ante.go#L49) - -`WithFeeRecipientModule` is backwards compatible — omitting it defaults to the standard `fee_collector` behavior. +**Source**: [`simapp/ante.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/poa/simapp/ante.go#L41-L56) ## Security Considerations diff --git a/sdk/latest/enterprise/poa/governance.mdx b/sdk/latest/enterprise/poa/governance.mdx index 1d1b6b9a6..ba109e23b 100644 --- a/sdk/latest/enterprise/poa/governance.mdx +++ b/sdk/latest/enterprise/poa/governance.mdx @@ -13,7 +13,7 @@ The PoA module integrates with Cosmos SDK governance to restrict participation t The PoA module restricts governance participation to authorized validators only through governance hooks. -**Governance Hooks** ([`x/poa/keeper/hooks.go`](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/x/poa/keeper/hooks.go)) +**Governance Hooks** ([`x/poa/keeper/hooks.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/poa/x/poa/keeper/hooks.go)) The module implements `govtypes.GovHooks`: @@ -29,13 +29,13 @@ The module implements `govtypes.GovHooks`: **Rejected Actions**: - If non-validator attempts governance action → transaction fails - If validator has power = 0 → transaction fails -- Error: "voter X is not an active PoA validator" +- Error: "voter X is not an active POA validator" -**Location**: [`x/poa/keeper/governance.go:92`](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/x/poa/keeper/governance.go#L92) +**Location**: [`x/poa/keeper/governance.go:115`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/poa/x/poa/keeper/governance.go#L115) ## Voting Power -**Custom Vote Tallying** ([`x/poa/keeper/governance.go:18`](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/x/poa/keeper/governance.go#L18)). An example of the wiring can be found in the [SimApp](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/simapp/app.go#L197-214). +**Custom Vote Tallying** ([`x/poa/keeper/governance.go:38`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/poa/x/poa/keeper/governance.go#L38)). An example of the wiring can be found in the [SimApp](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/poa/simapp/app.go#L214-L224). Standard governance uses staked tokens as voting weight. PoA governance uses validator power: @@ -96,7 +96,7 @@ Where: ### 1. Proposal Submission -**[MsgSubmitProposal](https://github.com/cosmos/cosmos-sdk/blob/main/proto/cosmos/gov/v1/tx.proto#L54-L65)** (standard x/gov module) +**[MsgSubmitProposal](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/gov/v1/tx.proto#L57-L88)** (standard x/gov module) When a proposal is submitted: @@ -112,7 +112,7 @@ When a proposal is submitted: ### 2. Deposit Period -**[MsgDeposit](https://github.com/cosmos/cosmos-sdk/blob/main/proto/cosmos/gov/v1/tx.proto#L90-L98)** (standard x/gov module) +**[MsgDeposit](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/gov/v1/tx.proto#L153-L166)** (standard x/gov module) When a deposit is made: @@ -125,7 +125,7 @@ When a deposit is made: ### 3. Voting Period -**[MsgVote](https://github.com/cosmos/cosmos-sdk/blob/main/proto/cosmos/gov/v1/tx.proto#L100-L108)** or **[MsgVoteWeighted](https://github.com/cosmos/cosmos-sdk/blob/main/proto/cosmos/gov/v1/tx.proto#L110-L118)** (standard x/gov module) +**[MsgVote](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/gov/v1/tx.proto#L111-L127)** or **[MsgVoteWeighted](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/gov/v1/tx.proto#L132-L148)** (standard x/gov module) When a vote is cast: @@ -146,7 +146,7 @@ When a vote is cast: At the end of the voting period, the [custom tally function](#vote-tallying-algorithm) is called: -**NewPoACalculateVoteResultsAndVotingPowerFn** ([`x/poa/keeper/governance.go:18`](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/x/poa/keeper/governance.go#L18)) +**NewPOACalculateVoteResultsAndVotingPowerFn** ([`x/poa/keeper/governance.go:38`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/poa/x/poa/keeper/governance.go#L38)) 1. Iterate all votes on the proposal 2. For each vote, look up the validator by voter address @@ -165,13 +165,13 @@ At the end of the voting period, the [custom tally function](#vote-tallying-algo ### Governance Hooks -**Location**: [`x/poa/keeper/hooks.go`](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/x/poa/keeper/hooks.go) +**Location**: [`x/poa/keeper/hooks.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/poa/x/poa/keeper/hooks.go) The module implements the `govtypes.GovHooks` interface: ``` type GovHooks interface { - AfterProposalSubmission(ctx, proposalID, depositorAddr) + AfterProposalSubmission(ctx, proposalID, proposerAddr) AfterProposalDeposit(ctx, proposalID, depositorAddr) AfterProposalVote(ctx, proposalID, voterAddr) // ... other hooks @@ -179,23 +179,23 @@ type GovHooks interface { ``` Each hook implementation: -1. Extracts the operator address from the context +1. Receives the operator address from the governance hook 2. Looks up the validator by operator address 3. Checks if validator exists and has power > 0 4. Returns error if validation fails ### Custom Tally Function -**Location**: [`x/poa/keeper/governance.go:18`](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/x/poa/keeper/governance.go#L18) +**Location**: [`x/poa/keeper/governance.go:38`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/poa/x/poa/keeper/governance.go#L38) The tally function replaces the standard governance tally: ```go -func NewPoACalculateVoteResultsAndVotingPowerFn(keeper) TallyFn { +func NewPOACalculateVoteResultsAndVotingPowerFn(keeper) TallyFn { return func(ctx, proposal) (totalVotingPower, results) { // Iterate votes for vote in votes(proposal) { - validator = keeper.GetValidatorByOperator(vote.voter) + validator = keeper.GetValidatorByOperatorAddress(vote.voter) if validator == nil || validator.Power <= 0 { continue // Skip non-authorized validators } @@ -239,6 +239,7 @@ The standard governance module parameters still apply: 3. **Admin Governance Control**: - Admin can change validator power at any time + - Admin can rotate any validator's consensus key on the operator's behalf - Admin can effectively control governance by adjusting power - Consider multi-sig admin or governance-controlled admin changes diff --git a/sdk/latest/enterprise/poa/overview.mdx b/sdk/latest/enterprise/poa/overview.mdx index e654d247c..af7facb5a 100644 --- a/sdk/latest/enterprise/poa/overview.mdx +++ b/sdk/latest/enterprise/poa/overview.mdx @@ -27,7 +27,7 @@ The PoA module is designed for networks that require: ## Source Code -The source code for the Proof of Authority module can be found [here](https://github.com/cosmos/cosmos-sdk/tree/main/enterprise/poa). +The source code for the Proof of Authority module can be found [here](https://github.com/cosmos/cosmos-sdk/tree/release/v0.55.x/enterprise/poa). ## Available Documentation @@ -37,9 +37,10 @@ This directory contains detailed documentation for the Proof of Authority module - **[Architecture](/sdk/latest/enterprise/poa/architecture)** - System architecture and module integration details - **[Distribution](/sdk/latest/enterprise/poa/distribution)** - Fee distribution mechanics and algorithms - **[Governance](/sdk/latest/enterprise/poa/governance)** - Governance integration and power-based voting +- **[Rotate a consensus key](/sdk/latest/keys/rotate-validator-key-poa)** - Rotate a PoA validator's consensus key as the operator or the admin ## Licensing -The Proof of Authority module source is published under the [Source Available Evaluation License](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/LICENSE), which permits evaluation and testing in non-production environments only. Production or commercial use requires an Enterprise License from Cosmos Labs. +The Proof of Authority module source is published under the [Source Available Evaluation License](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/poa/LICENSE), which permits evaluation and testing in non-production environments only. Production or commercial use requires an Enterprise License from Cosmos Labs. To use the Proof of Authority module in production, contact sales@cosmoslabs.io. \ No newline at end of file diff --git a/sdk/latest/experimental/blockstm.mdx b/sdk/latest/experimental/blockstm.mdx index cc9fc6573..20ca3ca34 100644 --- a/sdk/latest/experimental/blockstm.mdx +++ b/sdk/latest/experimental/blockstm.mdx @@ -115,9 +115,9 @@ NewSTMRunner( `NewSTMRunner` constructs a runner which uses parallel execution. Its parameters are: -* **`txDecoder`** — A standard `sdk.TxDecoder`, readily available in any SDK application. +* __`txDecoder`__ — A standard `sdk.TxDecoder`, readily available in any SDK application. -* **`stores`** — A list of every store key used in your application. Since Block-STM needs to track store usage across transactions, it must be passed all module-level store keys. Here is an example taken from the Cosmos EVM: +* __`stores`__ — A list of every store key used in your application. Since Block-STM needs to track store usage across transactions, it must be passed all module-level store keys. Here is an example taken from the Cosmos EVM: ```go keys := storetypes.NewKVStoreKeys( @@ -144,7 +144,7 @@ for _, k := range oKeys { } ``` -* **`workers`** — The number of parallel workers. Experimentation has shown diminishing returns above your system's hardware parallelism. The recommended value is: +* __`workers`__ — The number of parallel workers. Experimentation has shown diminishing returns above your system's hardware parallelism. The recommended value is: ```go import "runtime" @@ -152,9 +152,9 @@ import "runtime" workers := min(runtime.GOMAXPROCS(0), runtime.NumCPU()) ``` -* **`estimate`** — Controls whether the system should proactively determine transaction read/write conflicts before execution. Set this to `true` in all cases. +* __`estimate`__ — Controls whether the system should proactively determine transaction read/write conflicts before execution. Set this to `true` in all cases. -* **`coinDenom`** — A function that returns the staking coin denom at runtime. This is used during estimation to reason about which keys in the `bank` module will be modified when fees are collected. A hard-coded value is acceptable; the value should be your chain's bond denom. +* __`coinDenom`__ — A function that returns the staking coin denom at runtime. This is used during estimation to reason about which keys in the `bank` module will be modified when fees are collected. A hard-coded value is acceptable; the value should be your chain's bond denom. ### Full Wiring Example @@ -170,6 +170,28 @@ bApp.SetBlockSTMTxRunner(txnrunner.NewSTMRunner( )) ``` +### Configuration via app.toml + +The wiring above installs Block-STM programmatically. An application that uses `blockexec.Apply` can instead select the executor from `app.toml`, or from the equivalent `simd start` flags. Three keys control this: + +| Key | Type | Default | Description | +| --- | --- | --- | --- | +| `block-executor` | string | `sequential` | Selects the execution strategy. Set it to `block-stm` to enable parallel execution. | +| `block-stm-workers` | int | `0` | Sets the worker count. This maps to the `workers` runner parameter. A value of `0` resolves to `min(GOMAXPROCS, NumCPU)` at runtime. | +| `block-stm-pre-estimate` | bool | `false` | Enables pre-estimation of read and write conflicts. This maps to the `estimate` runner parameter. | + +When `block-executor` is set to `block-stm`, the block gas meter is disabled automatically. This is required, because the parallel runner panics if the block gas meter is still enabled. The programmatic wiring above does not disable it for you. Call `SetDisableBlockGasMeter(true)` when you wire the runner by hand. + +Example `app.toml` that enables Block-STM: + +```toml +block-executor = "block-stm" +block-stm-workers = 0 +block-stm-pre-estimate = true +``` + +The `block-stm-pre-estimate` value is set to `true` here to match the `estimate` guidance above. + ## Parallel Transaction Optimization Once Block-STM is wired in, you may initially notice that most blocks execute slower than with serial execution. This is due to the overhead of re-executing transactions when any two have conflicting reads or writes. To realize performance gains, you need to reduce storage access conflicts between transactions. diff --git a/sdk/latest/guides/abci/abci.mdx b/sdk/latest/guides/abci/abci.mdx index 4327c8755..80d5419d9 100644 --- a/sdk/latest/guides/abci/abci.mdx +++ b/sdk/latest/guides/abci/abci.mdx @@ -17,7 +17,7 @@ ABCI, Application Blockchain Interface is the interface between CometBFT and the * `VerifyVoteExtension` * `FinalizeBlock` -The Cosmos SDK's `BaseApp` implements the full ABCI interface. The source lives in [`baseapp/abci.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/baseapp/abci.go). +The Cosmos SDK's `BaseApp` implements the full ABCI interface. The source lives in [`baseapp/abci.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/baseapp/abci.go). ## CheckTx @@ -37,11 +37,11 @@ graph TD The default implementation runs the transaction through the `AnteHandler` chain, which performs signature verification, fee checks, and other stateless or lightweight stateful validation. If the `AnteHandler` returns an error, the transaction is rejected and never reaches the mempool. -See the implementation at [`baseapp/abci.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/baseapp/abci.go). +See the implementation at [`baseapp/abci.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/baseapp/abci.go). ### Custom CheckTx handler -`CheckTxHandler` lets you replace the default `CheckTx` logic entirely. The type is defined in [`types/abci.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/types/abci.go): +`CheckTxHandler` lets you replace the default `CheckTx` logic entirely. The type is defined in [`types/abci.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/types/abci.go): ```go type CheckTxHandler func(runTx RunTx, req *abci.RequestCheckTx) (*abci.ResponseCheckTx, error) @@ -69,7 +69,7 @@ CometBFT's own mempool uses FIFO ordering. `PrepareProposal` gives the applicati `PrepareProposal` MAY be non-deterministic and is only executed by the current block proposer. -The Cosmos SDK provides `DefaultProposalHandler` in [`baseapp/abci_utils.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/baseapp/abci_utils.go), which selects transactions from the app-side mempool up to `req.MaxTxBytes` and the block gas limit. +The Cosmos SDK provides `DefaultProposalHandler` in [`baseapp/abci_utils.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/baseapp/abci_utils.go), which selects transactions from the app-side mempool up to `req.MaxTxBytes` and the block gas limit. @@ -92,7 +92,7 @@ After the block proposer broadcasts a proposal, every validator calls `ProcessPr `ProcessProposal` MUST be deterministic. Non-deterministic results cause apphash mismatches across validators. If the handler panics or returns an error, honest validators prevote nil and CometBFT starts a new round with a new proposal. -See the default implementation in [`baseapp/abci_utils.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/baseapp/abci_utils.go). +See the default implementation in [`baseapp/abci_utils.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/baseapp/abci_utils.go). To wire a custom handler: @@ -116,4 +116,4 @@ See [Vote Extensions](/sdk/latest/guides/abci/vote-extensions) for implementatio `FinalizeBlock` is called once consensus is reached on a proposal. It executes all transactions in the block, runs `BeginBlock`/`EndBlock` equivalents, and commits the resulting state. It replaces the old `BeginBlock`, `DeliverTx`, and `EndBlock` methods from ABCI 1.0. -See the implementation at [`baseapp/abci.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/baseapp/abci.go). +See the implementation at [`baseapp/abci.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/baseapp/abci.go). diff --git a/sdk/latest/guides/abci/app-mempool.mdx b/sdk/latest/guides/abci/app-mempool.mdx index ed8d9f6d0..4a56ca2a2 100644 --- a/sdk/latest/guides/abci/app-mempool.mdx +++ b/sdk/latest/guides/abci/app-mempool.mdx @@ -37,6 +37,10 @@ chooses. So CometBFT decides what gets accepted into the network; the SDK app mempool decides how accepted transactions are ordered within a block. + +This in-process mempool should not be confused with CometBFT's `app` mempool. The mempool on this page orders transactions at `PrepareProposal` time. Setting `mempool.type = "app"` in CometBFT is a separate mechanism that routes transaction receipt itself to the application through the `InsertTx` and `ReapTxs` ABCI methods. That mechanism is documented in the [CometBFT mempool guide](/cometbft/next/docs/core/mempool). + + ## Mempool There are countless designs that an application developer can write for a mempool, the SDK opted to provide only simple mempool implementations. @@ -46,7 +50,7 @@ Namely, the SDK provides the following mempools: * [Sender Nonce Mempool](#sender-nonce-mempool) * [Priority Nonce Mempool](#priority-nonce-mempool) -By default, the SDK uses the [No-op Mempool](#no-op-mempool), but it can be replaced by the application developer in [`app.go`: +By default, the SDK uses the [No-op Mempool](#no-op-mempool), but it can be replaced by the application developer in `app.go`: ```go nonceMempool := mempool.NewSenderNonceMempool() @@ -66,7 +70,7 @@ which is FIFO-ordered by default. ### Sender Nonce Mempool -The nonce mempool is a mempool that keeps transactions from an sorted by nonce in order to avoid the issues with nonces. +The nonce mempool keeps each account's transactions sorted by nonce, so they are proposed in the order the account signed them. It works by storing the transaction in a list sorted by the transaction nonce. When the proposer asks for transactions to be included in a block it randomly selects a sender and gets the first transaction in the list. It repeats this until the mempool is empty or the block is full. It is configurable with the following parameters: @@ -85,7 +89,7 @@ Set the seed for the random number generator used to select transactions from th ### Priority Nonce Mempool -The [priority nonce mempool](https://github.com/cosmos/cosmos-sdk/blob/main/types/mempool/priority_nonce_spec.md) is a mempool implementation that stores txs in a partially ordered set by 2 dimensions: +The [priority nonce mempool](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/types/mempool/priority_nonce_spec.md) is a mempool implementation that stores txs in a partially ordered set by 2 dimensions: * priority * sender-nonce (sequence number) diff --git a/sdk/latest/guides/abci/vote-extensions.mdx b/sdk/latest/guides/abci/vote-extensions.mdx index 91a36bd2f..5286209cc 100644 --- a/sdk/latest/guides/abci/vote-extensions.mdx +++ b/sdk/latest/guides/abci/vote-extensions.mdx @@ -21,13 +21,13 @@ if cp.Abci != nil && req.Height > cp.Abci.VoteExtensionsEnableHeight { ## ExtendVote -The Cosmos SDK defines [`ExtendVoteHandler`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/types/abci.go#L48): +The Cosmos SDK defines [`ExtendVoteHandler`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/types/abci.go#L48): ```go type ExtendVoteHandler func(Context, *abci.RequestExtendVote) (*abci.ResponseExtendVote, error) ``` -Register a handler in `app.go` via `baseapp.SetExtendVoteHandler` (defined in [`baseapp/options.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/baseapp/options.go)): +Register a handler in `app.go` via `baseapp.SetExtendVoteHandler` (defined in [`baseapp/options.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/baseapp/options.go)): ```go app.SetExtendVoteHandler(myExtendVoteHandler) @@ -44,7 +44,7 @@ Keep extensions small — large extensions increase consensus latency. See [Come ## VerifyVoteExtension -The SDK defines [`VerifyVoteExtensionHandler`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/types/abci.go#L52): +The SDK defines [`VerifyVoteExtensionHandler`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/types/abci.go#L52): ```go type VerifyVoteExtensionHandler func(Context, *abci.RequestVerifyVoteExtension) (*abci.ResponseVerifyVoteExtension, error) @@ -62,7 +62,7 @@ Always validate the size of incoming extensions in this handler. ## Validating vote extension signatures -Before processing vote extensions in `PrepareProposal` or `ProcessProposal`, validate that they are properly signed. The SDK provides [`baseapp.ValidateVoteExtensions`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/baseapp/abci_utils.go) for this: +Before processing vote extensions in `PrepareProposal` or `ProcessProposal`, validate that they are properly signed. The SDK provides [`baseapp.ValidateVoteExtensions`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/baseapp/abci_utils.go) for this: ```go err := baseapp.ValidateVoteExtensions(ctx, valStore, req.Height, ctx.ChainID(), req.LocalLastCommit) @@ -71,7 +71,7 @@ if err != nil { } ``` -`ValidateVoteExtensions` verifies that each vote extension in the commit is correctly signed by its validator. `valStore` is a [`baseapp.ValidatorStore`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/baseapp/abci_utils.go), an interface with a single method: +`ValidateVoteExtensions` verifies that each vote extension in the commit is correctly signed by its validator. `valStore` is a [`baseapp.ValidatorStore`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/baseapp/abci_utils.go), an interface with a single method: ```go type ValidatorStore interface { @@ -101,7 +101,7 @@ proposalTxs = append([][]byte{bz}, proposalTxs...) `FinalizeBlock` ignores any byte slice that does not implement `sdk.Tx`, so injected extensions are safely skipped during message execution. -For more details on propagation design, see the [ABCI 2.0 ADR](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/docs/architecture/adr-064-abci-2.0.md#vote-extension-propagation--verification). +For more details on propagation design, see the [ABCI 2.0 ADR](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/docs/architecture/adr-064-abci-2.0.md#vote-extension-propagation--verification). ## Recovery via PreBlocker @@ -127,13 +127,13 @@ func (h *ProposalHandler) PreBlocker(ctx sdk.Context, req *abci.RequestFinalizeB } ``` -Register the PreBlocker in `app.go` (see [`baseapp/options.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/baseapp/options.go)): +Register the PreBlocker in `app.go` (see [`baseapp/options.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/baseapp/options.go)): ```go app.SetPreBlocker(proposalHandler.PreBlocker) ``` -The `sdk.PreBlocker` type is defined in [`types/abci.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/types/abci.go): +The `sdk.PreBlocker` type is defined in [`types/abci.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/types/abci.go): ```go type PreBlocker func(Context, *abci.RequestFinalizeBlock) (*ResponsePreBlock, error) diff --git a/sdk/latest/guides/module-design/module-design-considerations.mdx b/sdk/latest/guides/module-design/module-design-considerations.mdx index 6d769fa58..0f67cd7af 100644 --- a/sdk/latest/guides/module-design/module-design-considerations.mdx +++ b/sdk/latest/guides/module-design/module-design-considerations.mdx @@ -32,7 +32,7 @@ Ask: could a different chain reasonably use this module without modification? If ### Plan your state structure early -Every `KVStore` key your module defines is permanent: removing or renaming keys requires a migration. Use the [Collections](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/collections/README.md) library for structured state management, and name keys to be collision-resistant and self-documenting. +Every `KVStore` key your module defines is permanent: removing or renaming keys requires a migration. Use the [Collections](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/collections/README.md) library for structured state management, and name keys to be collision-resistant and self-documenting. Consider what your module needs to index. A value that is only ever looked up by a single key is simple. A value looked up by multiple dimensions (e.g. by owner and by ID) requires secondary indexes, which add complexity and storage overhead. diff --git a/sdk/latest/guides/module-design/ocap.mdx b/sdk/latest/guides/module-design/ocap.mdx index 5aa4ce261..986277029 100644 --- a/sdk/latest/guides/module-design/ocap.mdx +++ b/sdk/latest/guides/module-design/ocap.mdx @@ -93,6 +93,6 @@ if msg.Authority != k.authority { The authority address is set at wiring time in `app.go` and cannot be changed at runtime. This is ocap applied to governance: privileged capability is a reference, and only the holder of that reference can exercise it. -See [`simapp/app.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/simapp/app.go) for how keeper dependencies and authorities are wired in a complete application. +See [`simapp/app.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/simapp/app.go) for how keeper dependencies and authorities are wired in a complete application. For background, see the [Wikipedia article on object-capability model](https://en.wikipedia.org/wiki/Object-capability_model). diff --git a/sdk/latest/guides/reference/bech32.mdx b/sdk/latest/guides/reference/bech32.mdx index 08c9e2780..60ca00afb 100644 --- a/sdk/latest/guides/reference/bech32.mdx +++ b/sdk/latest/guides/reference/bech32.mdx @@ -99,7 +99,7 @@ bech32Address, _ := bech32.Encode("cosmos", converted) ## Configuring Bech32 prefixes -Every Cosmos SDK application sets its Bech32 prefixes and SLIP-44 coin type once at startup via `sdk.GetConfig()`, then seals the config so it cannot be changed at runtime. The defaults (`cosmos`, `cosmosvaloper`, etc.) are defined in [`types/config.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/types/config.go). Chain developers override them before the app starts: +Every Cosmos SDK application sets its Bech32 prefixes and SLIP-44 coin type once at startup via `sdk.GetConfig()`, then seals the config so it cannot be changed at runtime. The defaults (`cosmos`, `cosmosvaloper`, etc.) are defined in [`types/config.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/types/config.go). Chain developers override them before the app starts: ```go config := sdk.GetConfig() @@ -135,7 +135,7 @@ func (bc Bech32Codec) StringToBytes(text string) ([]byte, error) { ## Module Addresses -Module accounts use deterministic address derivation defined in [ADR-028](https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-028-public-key-addresses.md): +Module accounts use deterministic address derivation defined in [ADR-028](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/docs/architecture/adr-028-public-key-addresses.md): ```go // Module address without derivation keys diff --git a/sdk/latest/guides/reference/protobuf-annotations.mdx b/sdk/latest/guides/reference/protobuf-annotations.mdx index e78f6634b..a392135f0 100644 --- a/sdk/latest/guides/reference/protobuf-annotations.mdx +++ b/sdk/latest/guides/reference/protobuf-annotations.mdx @@ -32,7 +32,7 @@ Signer specifies which field should be used to determine the signer of a message Read more about the signer field [here](/sdk/latest/learn/concepts/encoding#message-signers). ```proto -// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/proto/cosmos/bank/v1beta1/tx.proto#L40 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/bank/v1beta1/tx.proto#L40 option (cosmos.msg.v1.signer) = "from_address"; ``` @@ -47,28 +47,28 @@ The scalar type defines a way for clients to understand how to construct protobu Example of account address string scalar: ```proto -// https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/proto/cosmos/bank/v1beta1/tx.proto#L46 +// https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/bank/v1beta1/tx.proto#L46 string from_address = 1 [(cosmos_proto.scalar) = "cosmos.AddressString"]; ``` Example of validator address string scalar: ```proto -// https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/proto/cosmos/distribution/v1beta1/query.proto#L108 +// https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/distribution/v1beta1/query.proto#L107 string validator_address = 1 [(cosmos_proto.scalar) = "cosmos.ValidatorAddressString"]; ``` Example of Dec scalar: ```proto -// https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/proto/cosmos/distribution/v1beta1/distribution.proto#L17 +// https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/distribution/v1beta1/distribution.proto#L17 string community_tax = 1 [(cosmos_proto.scalar) = "cosmos.Dec"]; ``` Example of Int scalar: ```proto -// https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/proto/cosmos/gov/v1/gov.proto#L127 +// https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/gov/v1/gov.proto#L127 string yes_count = 1 [(cosmos_proto.scalar) = "cosmos.Int"]; ``` @@ -109,7 +109,7 @@ The below annotations are used to provide information to the amino codec on how Name specifies the amino name that would show up for the user in order for them see which message they are signing. ```proto -// https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/proto/cosmos/bank/v1beta1/tx.proto#L41 +// https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/bank/v1beta1/tx.proto#L41 option (amino.name) = "cosmos-sdk/MsgSend"; ``` @@ -118,7 +118,7 @@ option (amino.name) = "cosmos-sdk/MsgSend"; Field name specifies the amino name that would show up for the user in order for them see which field they are signing. ```proto -// https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/proto/cosmos/distribution/v1beta1/distribution.proto#L165 +// https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/distribution/v1beta1/distribution.proto#L165 uint64 height = 3 [(amino.field_name) = "creation_height"]; ``` @@ -127,22 +127,22 @@ uint64 height = 3 [(amino.field_name) = "creation_height"]; Dont omitempty specifies that the field should not be omitted when encoding to amino. ```proto -// https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/proto/cosmos/bank/v1beta1/tx.proto#L48 +// https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/bank/v1beta1/tx.proto#L48 repeated cosmos.base.v1beta1.Coin amount = 3 [(amino.dont_omitempty) = true]; ``` ### Encoding -Encoding instructs the amino json marshaler how to encode certain fields that may differ from the standard encoding behavior. The most common example of this is how `repeated cosmos.base.v1beta1.Coin` is encoded when using the amino json encoding format. The `legacy_coins` option tells the json marshaler [how to encode a null slice](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/x/tx/signing/aminojson/json_marshal.go#L85) of `cosmos.base.v1beta1.Coin`. +Encoding instructs the amino json marshaler how to encode certain fields that may differ from the standard encoding behavior. The most common example of this is how `repeated cosmos.base.v1beta1.Coin` is encoded when using the amino json encoding format. The `legacy_coins` option tells the json marshaler [how to encode a null slice](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/x/tx/signing/aminojson/json_marshal.go#L85) of `cosmos.base.v1beta1.Coin`. ```proto -// https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/proto/cosmos/bank/v1beta1/genesis.proto#L23 +// https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/bank/v1beta1/genesis.proto#L23 (amino.encoding) = "legacy_coins", ``` ## Module Query Safe -The `cosmos.query.v1.module_query_safe` annotation ([source](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/proto/cosmos/query/v1/query.proto)) marks a query method as safe to call from within the state machine — for example from another module's keeper, via ADR-033 intermodule calls, or from CosmWasm contracts. +The `cosmos.query.v1.module_query_safe` annotation ([source](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/query/v1/query.proto)) marks a query method as safe to call from within the state machine — for example from another module's keeper, via ADR-033 intermodule calls, or from CosmWasm contracts. ```proto rpc Balance(QueryBalanceRequest) returns (QueryBalanceResponse) { diff --git a/sdk/latest/guides/state/collections.mdx b/sdk/latest/guides/state/collections.mdx index c846caf13..e19e9d676 100644 --- a/sdk/latest/guides/state/collections.mdx +++ b/sdk/latest/guides/state/collections.mdx @@ -112,7 +112,7 @@ Since a module can have multiple collections, the following is expected: We don't want a collection to write over the state of the other collection so we pass it a prefix, which defines a storage partition owned by the collection. -If you already built modules, the prefix translates to the items you were creating in your `types/keys.go` file, example: [Link](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/x/feegrant/key.go#L16-L22) +If you already built modules, the prefix translates to the items you were creating in your `types/keys.go` file, example: [Link](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/x/feegrant/key.go#L16-L22) your old: diff --git a/sdk/latest/guides/state/store.mdx b/sdk/latest/guides/state/store.mdx index b893b6cd0..1cdf4f299 100644 --- a/sdk/latest/guides/state/store.mdx +++ b/sdk/latest/guides/state/store.mdx @@ -12,7 +12,7 @@ abstractions. ### `Store` -The bulk of the store interfaces are defined [here](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/store/types/store.go), +The bulk of the store interfaces are defined [here](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/store/types/store.go), where the base primitive interface, for which other interfaces build off of, is the `Store` type. The `Store` interface defines the ability to tell the type of the implementing store and the ability to cache wrap via the `CacheWrapper` interface. diff --git a/sdk/latest/guides/testing/simulator.mdx b/sdk/latest/guides/testing/simulator.mdx index 13584cc43..c4f735ae2 100644 --- a/sdk/latest/guides/testing/simulator.mdx +++ b/sdk/latest/guides/testing/simulator.mdx @@ -54,9 +54,9 @@ type HasProposalMsgs interface { } ``` -See the full source at [`types/module/simulation.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/types/module/simulation.go). +See the full source at [`types/module/simulation.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/types/module/simulation.go). -See an example implementation of these methods from `x/distribution` [here](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/x/distribution/module.go#L170-L194). +See an example implementation of these methods from `x/distribution` [here](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/x/distribution/module.go#L158-L182). ## SimsX @@ -79,7 +79,7 @@ type ( ) ``` -See the full source at [`testutil/simsx/runner.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/testutil/simsx/runner.go). +See the full source at [`testutil/simsx/runner.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/testutil/simsx/runner.go). `SimMsgFactoryFn` is the default factory for most cases. It does not create future operations but ensures successful message delivery: @@ -88,22 +88,22 @@ See the full source at [`testutil/simsx/runner.go`](https://github.com/cosmos/co type SimMsgFactoryFn[T sdk.Msg] func(ctx context.Context, testData *ChainDataSource, reporter SimulationReporter) (signer []SimAccount, msg T) ``` -See the full source at [`testutil/simsx/msg_factory.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/testutil/simsx/msg_factory.go). +See the full source at [`testutil/simsx/msg_factory.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/testutil/simsx/msg_factory.go). These methods allow constructing randomized messages and/or proposal messages. Note that modules should **not** implement both `HasWeightedOperationsX` and `HasWeightedOperationsXWithProposals`. -See the runner code [here](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/testutil/simsx/runner.go#L330-L339) for details +See the runner code [here](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/testutil/simsx/runner.go#L330-L339) for details If the module does **not** have message handlers or governance proposal handlers, these interface methods do **not** need to be implemented. ### Example Implementations -* `HasWeightedOperationsXWithProposals`: [x/gov](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/x/gov/module.go#L242-L261) -* `HasWeightedOperationsX`: [x/bank](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/x/bank/module.go#L201-L205) -* `HasProposalMsgsX`: [x/bank](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/x/bank/module.go#L196-L199) +* `HasWeightedOperationsXWithProposals`: [x/gov](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/x/gov/module.go#L221-L240) +* `HasWeightedOperationsX`: [x/bank](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/x/bank/module.go#L179-L183) +* `HasProposalMsgsX`: [x/bank](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/x/bank/module.go#L174-L177) ## Store decoders @@ -111,7 +111,7 @@ Registering the store decoders is required for the `AppImportExport` simulation. for the key-value pairs from the stores to be decoded to their corresponding types. In particular, it matches the key to a concrete type and then unmarshalls the value from the `KVPair` to the type provided. -Modules using [collections](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/collections/README.md) can use the `NewStoreDecoderFuncFromCollectionsSchema` function that builds the decoder for you: +Modules using [collections](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/collections/README.md) can use the `NewStoreDecoderFuncFromCollectionsSchema` function that builds the decoder for you: ```go // RegisterStoreDecoder registers a decoder for supply module's types @@ -120,17 +120,17 @@ func (am AppModule) RegisterStoreDecoder(sdr simtypes.StoreDecoderRegistry) { } ``` -See the full source at [`types/simulation/collections.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/types/simulation/collections.go) and the bank module example at [`x/bank/module.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/x/bank/module.go#L183-L186). +See the full source at [`types/simulation/collections.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/types/simulation/collections.go) and the bank module example at [`x/bank/module.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/x/bank/module.go#L161-L164). Modules not using collections must manually build the store decoder. -See the implementation [here](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/x/distribution/simulation/decoder.go) from the distribution module for an example. +See the implementation [here](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/x/distribution/simulation/decoder.go) from the distribution module for an example. ## Randomized genesis The simulator tests different scenarios and values for genesis parameters. App modules must implement a `GenerateGenesisState` method to generate the initial random `GenesisState` from a given seed. -See an example from `x/auth` [here](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/x/auth/module.go#L174-L177). +See an example from `x/auth` [here](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/x/auth/module.go#L171-L174). Once the module's genesis parameters are generated randomly (or with the key and values defined in a `params` file), they are marshaled to JSON format and added @@ -169,7 +169,7 @@ Note that the name passed in to `weights.Get` must match the name of the operati For example, if the module contains an operation `op_weight_msg_set_withdraw_address`, the name passed to `weights.Get` should be `msg_set_withdraw_address`. -See the `x/distribution` for an example of implementing message factories [here](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/x/distribution/simulation/msg_factory.go) +See the `x/distribution` for an example of implementing message factories [here](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/x/distribution/simulation/msg_factory.go) ## App Simulator manager @@ -207,7 +207,7 @@ func (app *SimApp) SimulationManager() *module.SimulationManager { } ``` -See the full simapp setup at [`simapp/app.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/simapp/app.go). +See the full simapp setup at [`simapp/app.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/simapp/app.go). ## Running Simulations @@ -229,7 +229,7 @@ func TestAppImportExport(t *testing.T) { These functions should be called in tests (i.e., `app_test.go`, `app_sim_test.go`, etc.). -See the full simapp test file at [`simapp/sim_test.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/simapp/sim_test.go). +See the full simapp test file at [`simapp/sim_test.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/simapp/sim_test.go). ### Simulation test types @@ -246,7 +246,7 @@ Simulations run in three modes: 1. **Fully random** -- initial state, module parameters, and simulation parameters are all pseudo-randomly generated. 2. **From a `genesis.json` file** -- initial state and module parameters are defined by the file. Useful for testing against a known state such as a live network export. -3. **From a `params.json` file** -- initial state is pseudo-randomly generated but module and simulation parameters are set manually. Available parameters are listed [here](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/x/simulation/client/cli/flags.go#L43-L70). +3. **From a `params.json` file** -- initial state is pseudo-randomly generated but module and simulation parameters are set manually. Available parameters are listed [here](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/x/simulation/client/cli/flags.go#L43-L70). These modes are not mutually exclusive. For example, you can combine a randomly generated genesis state (mode 1) with manually defined simulation params (mode 3). @@ -263,7 +263,7 @@ go test -mod=readonly github.com/cosmos/cosmos-sdk/simapp \ -v -timeout 24h ``` -The full list of available flags is defined [here](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/x/simulation/client/cli/flags.go#L43-L70). For Makefile examples, see the Cosmos SDK [`Makefile`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/Makefile#L280-L340). +The full list of available flags is defined [here](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/x/simulation/client/cli/flags.go#L43-L70). For Makefile examples, see the Cosmos SDK [`Makefile`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/Makefile#L280-L340). ### Debugging tips @@ -273,4 +273,4 @@ When encountering a simulation failure: * **Use `-Verbose` logs** for a fuller picture of all operations involved. * **Try a different `-Seed`**. If the same error reproduces sooner, you will spend less time on each run. * **Reduce `-NumBlocks`** to isolate what the app state looks like at the block before failure. -* **Add a [`Logger`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/x/staking/keeper/keeper.go#L78-L82)** to operations that are not being logged. +* **Add a [`Logger`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/x/staking/keeper/keeper.go#L82-L86)** to operations that are not being logged. diff --git a/sdk/latest/guides/testing/telemetry.mdx b/sdk/latest/guides/testing/telemetry.mdx index f8dad96b6..b74255dfc 100644 --- a/sdk/latest/guides/testing/telemetry.mdx +++ b/sdk/latest/guides/testing/telemetry.mdx @@ -69,10 +69,10 @@ extensions: **Option A: Environment Variable (Recommended)** -Set `OTEL_EXPERIMENTAL_CONFIG_FILE` to your config path. This initializes the SDK before any meters/tracers are created, avoiding atomic load overhead. +Set `OTEL_CONFIG_FILE` to your config path. This initializes the SDK before any meters/tracers are created, avoiding atomic load overhead. ```bash -export OTEL_EXPERIMENTAL_CONFIG_FILE=/path/to/otel.yaml +export OTEL_CONFIG_FILE=/path/to/otel.yaml ``` **Option B: Node Config Directory** diff --git a/sdk/latest/guides/tooling/autocli.mdx b/sdk/latest/guides/tooling/autocli.mdx index 548bb8f71..f67e1783c 100644 --- a/sdk/latest/guides/tooling/autocli.mdx +++ b/sdk/latest/guides/tooling/autocli.mdx @@ -102,7 +102,7 @@ Users can however use the `--no-proposal` flag to disable the proposal creation By default, `autocli` generates a command for each method in your gRPC service. However, you can specify subcommands to group related commands together. To specify subcommands, use the `autocliv1.ServiceCommandDescriptor` struct. -For a real-world example, see the `gov` module's [`autocli.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/x/gov/autocli.go) in the Cosmos SDK. It demonstrates `ServiceCommandDescriptor` with `RpcCommandOptions`, `PositionalArgs`, `SubCommands`, `EnhanceCustomCommand`, and `GovProposal` all in one file. +For a real-world example, see the `gov` module's [`autocli.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/x/gov/autocli.go) in the Cosmos SDK. It demonstrates `ServiceCommandDescriptor` with `RpcCommandOptions`, `PositionalArgs`, `SubCommands`, `EnhanceCustomCommand`, and `GovProposal` all in one file. ### Positional Arguments @@ -110,7 +110,7 @@ By default `autocli` generates a flag for each field in your protobuf message. H To add positional arguments to a command, use the `autocliv1.PositionalArgDescriptor` struct, as seen in the example below. Specify the `ProtoField` parameter, which is the name of the protobuf field that should be used as the positional argument. In addition, if the parameter is a variable-length argument, you can specify the `Varargs` parameter as `true`. This can only be applied to the last positional parameter, and the `ProtoField` must be a repeated field. -For a real-world example, see the `auth` module's [`autocli.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/x/auth/autocli.go) in the Cosmos SDK. It shows positional args wired for every query method, with `address` as a positional argument on the `Account` method. +For a real-world example, see the `auth` module's [`autocli.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/x/auth/autocli.go) in the Cosmos SDK. It shows positional args wired for every query method, with `address` as a positional argument on the `Account` method. After wiring positional args, the command can be used as follows, instead of having to specify the `--address` flag: @@ -227,11 +227,11 @@ autoCliOpts.ModuleOptions[nodeCmds.Name()] = nodeCmds.AutoCLIOptions() `AutoCliOpts()` only picks up modules registered with the module manager — non-module commands always need to be added to `ModuleOptions` manually, as the example chain does with `nodeservice.NewNodeCommands()`. -For a more complete example of this pattern, see [`client/grpc/cmtservice/autocli.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/client/grpc/cmtservice/autocli.go) and [`client/grpc/node/autocli.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/client/grpc/node/autocli.go) in the Cosmos SDK. +For a more complete example of this pattern, see [`client/grpc/cmtservice/autocli.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/client/grpc/cmtservice/autocli.go) and [`client/grpc/node/autocli.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/client/grpc/node/autocli.go) in the Cosmos SDK. ## Root Command Setup -For AutoCLI-generated commands (and hand-written commands) to work correctly — signing transactions, querying the chain, reading configuration — the root command must set up the `client.Context` and `server.Context` in a `PersistentPreRunE` function. This runs before every subcommand and makes both contexts available to all child commands. See [`simapp/simd/cmd/root.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/simapp/simd/cmd/root.go#L50-L93) for a complete example. +For AutoCLI-generated commands (and hand-written commands) to work correctly — signing transactions, querying the chain, reading configuration — the root command must set up the `client.Context` and `server.Context` in a `PersistentPreRunE` function. This runs before every subcommand and makes both contexts available to all child commands. See [`simapp/simd/cmd/root.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/simapp/simd/cmd/root.go#L47-L70) for a complete example. The two key calls inside `PersistentPreRun` are: diff --git a/sdk/latest/guides/upgrades/upgrade.mdx b/sdk/latest/guides/upgrades/upgrade.mdx index eccb1b5c6..7faba91a3 100644 --- a/sdk/latest/guides/upgrades/upgrade.mdx +++ b/sdk/latest/guides/upgrades/upgrade.mdx @@ -103,7 +103,7 @@ func (m Migrator) Migrate1to2(ctx sdk.Context) error { } ``` -To see example code of changes that were implemented in a migration of balance keys, check out [migrateBalanceKeys](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/x/bank/migrations/v2/store.go#L55-L76). For context, this code introduced migrations of the bank store that updated addresses to be prefixed by their length in bytes as outlined in [ADR-028](/sdk/latest/reference/architecture/adr-028-public-key-addresses). +To see example code of changes that were implemented in a migration of balance keys, check out [migrateBalanceKeys](https://github.com/cosmos/cosmos-sdk/blob/v0.54.0/x/bank/migrations/v2/store.go#L55-L76). For context, this code introduced migrations of the bank store that updated addresses to be prefixed by their length in bytes as outlined in [ADR-028](/sdk/latest/reference/architecture/adr-028-public-key-addresses). ## Running Migrations in the App diff --git a/sdk/latest/keys/create-ml-dsa-account.mdx b/sdk/latest/keys/create-ml-dsa-account.mdx new file mode 100644 index 000000000..24febf943 --- /dev/null +++ b/sdk/latest/keys/create-ml-dsa-account.mdx @@ -0,0 +1,86 @@ +--- +title: "Create an ML-DSA account" +description: "Create a post-quantum user account with the keyring and move funds into it; there is no in-place migration for accounts." +--- + +This guide creates a user account backed by `ml_dsa_65`, the post-quantum signature algorithm, and moves funds into it. Accounts do not rotate. An existing account keeps its key for life, so moving to post-quantum means a new account and a transfer. For what ML-DSA is and when account migration matters, see [Post-quantum keys](/sdk/latest/keys/post-quantum-keys). + +Commands use `simd`. Substitute your chain's binary and adjust key names and denoms. + +{/* +Manual verification setup (hidden, macOS). Reproduces a running chain with a funded source account, using simd/simapp. +Build: from the cosmos-sdk repo run `make build`, then put ./build/simd on PATH as `simd`. + +simd init mynode --chain-id my-chain-1 --home ~/.node +simd config set client chain-id my-chain-1 --home ~/.node +simd config set client keyring-backend test --home ~/.node +simd keys add old-account --keyring-backend test --home ~/.node +simd keys add validator --keyring-backend test --home ~/.node +simd genesis add-genesis-account old-account 100000000000stake --keyring-backend test --home ~/.node +simd genesis add-genesis-account validator 100000000000stake --keyring-backend test --home ~/.node +simd genesis gentx validator 1000000000stake --chain-id my-chain-1 --keyring-backend test --home ~/.node +simd genesis collect-gentxs --home ~/.node +sed -i '' 's/timeout_commit = ".*"/timeout_commit = "1s"/' ~/.node/config/config.toml +sed -i '' 's/minimum-gas-prices = ".*"/minimum-gas-prices = "0stake"/' ~/.node/config/app.toml +simd start --home ~/.node +# then follow the guide (use --home ~/.node on each command). The bank sends need --chain-id/--keyring-backend +# from client config, and the first send FROM pq-account needs --gas auto (already shown in step 3). +*/} + +## Know the limits first + +- CLI only. Wallet support for ML-DSA accounts is minimal; expect to manage the account with the chain CLI. +- No hardware wallets. Ledger devices sign with `secp256k1` only. Recovery works from the mnemonic alone. +- No EVM. EVM transactions require `eth_secp256k1` account keys, so ML-DSA accounts do not work with the EVM. See the EVM section of [Post-quantum keys](/sdk/latest/keys/post-quantum-keys). + +## Prerequisites + +- The chain runs Cosmos SDK 0.55 or later, on every node. +- A running chain to send transactions to, with its CLI binary on your PATH and a funded account in the keyring. To stand up a local chain first, see [Run a node](/sdk/latest/node/run-node). + +Enabling ML-DSA accounts needs no chain configuration beyond the SDK version. Upgrading to 0.55 is sufficient, but every validator and node must run it: a binary built with an older SDK cannot process ML-DSA signatures. + +## 1. Create the account + +Generate the account with the keyring and select the ML-DSA algorithm: + +```shell +simd keys add pq-account --key-type ml_dsa_65 --home ~/.node --keyring-backend test +``` + +Use the same `--home` and `--keyring-backend` on every command in this guide. Without them the key lands in a different keyring from your funded account, and the transfer in step 2 fails to find it. + +The output shows the new address and a mnemonic. Store the mnemonic securely. No hardware wallet can hold this key, so the mnemonic is the only recovery path. To recover the account later: + +```shell +simd keys add pq-account --recover --key-type ml_dsa_65 --home ~/.node --keyring-backend test +``` + +Do not create an ML-DSA account from a mnemonic already used for a `secp256k1` account. ML-DSA key generation derives its seed from the same secp256k1 BIP32 path, so at that path the secp256k1 private key and the ML-DSA seed are the same secret. Reusing the mnemonic links the two keys: whoever obtains that secret controls both. Generate each ML-DSA account from a fresh mnemonic. + +## 2. Move funds in + +There is no in-place migration for accounts by design. Send funds from the old account with an ordinary transfer: + +```shell +simd tx bank send old-account "$(simd keys show pq-account -a --home ~/.node --keyring-backend test)" 1000000stake --from old-account --home ~/.node --keyring-backend test +``` + +Add `--chain-id` if your client config does not supply it, and `--fees` (or `--gas-prices`) to meet the chain's minimum gas price. + +## 3. Verify the account signs + +Prove the new key works by sending from it: + +```shell +simd tx bank send pq-account "$(simd keys show old-account -a --home ~/.node --keyring-backend test)" 1stake --from pq-account --gas auto --gas-adjustment 1.5 --home ~/.node --keyring-backend test +``` + +You may need to raise the gas limit for this transfer. The account's first transaction writes its public key to state, and an ML-DSA public key is large enough that a default limit of 200000 runs out of gas. `--gas auto` sizes the limit to fit. Later transactions from the account are smaller and fit within the default. + +A successful send from `pq-account` means the chain accepted an ML-DSA signature for the account. Drain and retire the old account afterwards. + +## Next steps + +- Understand what post-quantum protection the account now has. See [Post-quantum keys](/sdk/latest/keys/post-quantum-keys). +- Validators migrate differently, by key rotation. See [Migrate a validator to ML-DSA](/sdk/latest/keys/migrate-validator-ml-dsa). diff --git a/sdk/latest/keys/enable-ml-dsa-keys.mdx b/sdk/latest/keys/enable-ml-dsa-keys.mdx new file mode 100644 index 000000000..7853ee2a9 --- /dev/null +++ b/sdk/latest/keys/enable-ml-dsa-keys.mdx @@ -0,0 +1,132 @@ +--- +title: "Enable ML-DSA keys" +description: "Add ml_dsa_65 to a chain's accepted consensus key types, through genesis on a new chain or governance on a live one." +--- + +Validator key types are a consensus parameter, so allowing post-quantum validator keys is a chain-level change. This guide adds `ml_dsa_65` to the accepted consensus key types: on a new chain through genesis, or on a live chain through a governance proposal, with no coordinated restart. + +For background on ML-DSA and the key types, see [Post-quantum keys](/sdk/latest/keys/post-quantum-keys). + +Before any validator rotates to an ML-DSA key, every counterparty chain that verifies this chain over IBC must run CometBFT v0.40 or later. See [IBC considerations](/sdk/latest/keys/post-quantum-keys#ibc-considerations). + +## Prerequisites + +- The chain runs Cosmos SDK 0.55 and CometBFT 0.40 or later. See the [release notes](/sdk/latest/upgrade/v0.55-release). +- [jq](https://jqlang.org/) and [curl](https://curl.se/). The commands derive the proposal payload with jq and read the validator set with curl. +- The chain's CLI binary on your PATH, with RPC access to a node. To stand up a local chain, see [Run a node](/sdk/latest/node/run-node). +- For a live chain: the ability to pass a governance proposal, and an account funded for the proposal deposit and gas. + +Commands use `simd`; substitute your chain's binary, and adjust key names and denoms to your own. + +{/* +Manual verification setup (hidden, macOS). Reproduces a running chain with governance for the live-chain path, using simd/simapp. +Build: from the cosmos-sdk repo run `make build`, then put ./build/simd on PATH as `simd`. + +simd init mynode --chain-id my-chain-1 --home ~/.node +simd config set client chain-id my-chain-1 --home ~/.node +simd config set client keyring-backend test --home ~/.node +simd keys add val --keyring-backend test --home ~/.node +simd genesis add-genesis-account val 100000000000stake --keyring-backend test --home ~/.node +simd genesis gentx val 50000000000stake --chain-id my-chain-1 --keyring-backend test --home ~/.node +simd genesis collect-gentxs --home ~/.node +# short voting period so the proposal can pass quickly during testing +jq '.app_state.gov.params.voting_period="15s" | .app_state.gov.params.expedited_voting_period="10s"' ~/.node/config/genesis.json > /tmp/g && mv /tmp/g ~/.node/config/genesis.json +sed -i '' 's/timeout_commit = ".*"/timeout_commit = "1s"/' ~/.node/config/config.toml +sed -i '' 's/minimum-gas-prices = ".*"/minimum-gas-prices = "0stake"/' ~/.node/config/app.toml +simd start --home ~/.node +# then follow the guide (use --home ~/.node). Submit the proposal and vote quickly, within the 15s window. +# All commands run from one directory so params.json is shared. +*/} + +## Check the current state + +Consensus params list the allowed key types under `validator.pub_key_types`. Query them: + +```shell +simd query consensus params +``` + +The default is `ed25519` only. To see which types the validator set is currently running, list the validators and read each one's pubkey type: + +```shell +curl -s localhost:26657/validators | jq -r '.result.validators[].pub_key.type' +``` + +The output shows registered type names, one per validator: `tendermint/PubKeyEd25519` for ed25519 keys, `cometbft/PubKeyMlDsa65` for ML-DSA keys. + +## New chain: set the types in genesis + +Add `ml_dsa_65` to `consensus.params.validator.pub_key_types` in `genesis.json` before launch: + +```json +"validator": { + "pub_key_types": ["ed25519", "ml_dsa_65"] +} +``` + +Keep `ed25519` in the list unless every genesis validator starts on an ML-DSA key. Validators whose key type is not in the list cannot join the set. A genesis validator starts on an ML-DSA key by initializing its node with `simd init --consensus-key-algo ml_dsa_65`, and a local ML-DSA testnet comes from the same flag on `simd testnet init-files` or `simd testnet start`. + +`simd init --consensus-key-algo` replaces the genesis `pub_key_types` list with only the chosen algorithm. A chain that should accept both types must re-add `ed25519` to the list after init, or ed25519 validators cannot join. + +## Live chain: expand the types through governance + +The change is a parameter update executed by governance. It takes effect when the proposal passes, with no node restarts and no coordinated upgrade. + +All steps read and write `params.json` in the current directory, so run them from one place. + +1. Build the proposal params from the live chain state. The update message replaces the entire params object, so it must carry every current value. The following command derives the params from a query, adds `ml_dsa_65` to `pub_key_types`, and converts the evidence duration to the format the proposal parser accepts: + +```shell +simd query consensus params -o json \ + | jq '.params + | {block, evidence, validator, abci} + | .validator.pub_key_types += ["ml_dsa_65"] + | .evidence.max_age_duration |= ( + capture("((?[0-9]+)h)?((?[0-9]+)m)?((?[0-9]+)s)?") + | ((((.h // "0") | tonumber) * 3600 + + ((.m // "0") | tonumber) * 60 + + ((.s // "0") | tonumber)) | tostring) + "s" + )' \ + > params.json +``` + +2. You can then submit the file's contents as a governance proposal. The command takes the four param groups as separate arguments, sliced from the same file. Make sure to update the following command to include your key and correct deposit amount/denomination. Add `--home` pointing at your node's home, since the transaction commands on this page read the keyring from it. Also add `--chain-id` and `--keyring-backend` if your client config does not supply them, and `--fees` (or `--gas-prices`) to meet the chain's minimum gas price: + +```shell +simd tx consensus update-params-proposal "$(jq -c .block params.json)" "$(jq -c .evidence params.json)" "$(jq -c .validator params.json)" "$(jq -c .abci params.json)" --title "Allow ML-DSA validator keys" --summary "Add ml_dsa_65 to consensus pub_key_types" --deposit 10000000stake --from mykey -y +``` + +3. Vote as with any governance proposal, using the proposal ID from `simd query gov proposals`. + +```shell +simd tx gov vote yes --from mykey +``` + +When it passes, the new list is live. + + +The update replaces the entire params object, so every group must be present. If `params.json` is missing `block`, `evidence`, or `validator`, the `jq -c` substitution above emits `null` and the CLI rejects the command before it is submitted, with `invalid argument "null": proto: syntax error`. Always start from the queried current params and change only the key type list. + + +## Verify + +Query the params again and confirm the list includes `ml_dsa_65`: + +```shell +simd query consensus params +``` + +New validators can now join with ML-DSA keys. Existing validators can migrate by rotation. See [Key rotation](/sdk/latest/keys/key-rotation). + +## Remove a key type + +To remove a key type, update the `params.json` file to remove the key type from the `pub_key_types` array. Then, submit the updated `params.json` file as a governance proposal. + + +Do not remove a key type while validators still use it. Existing validators are not re-checked when a type leaves the list, but every later voting-power update for such a validator fails validation, and a failed validator update halts the chain. Routine activity is enough to trigger it: any delegation that changes the validator's voting power emits one of these updates. + + +## Next steps + +- Understand the rotation mechanics before touching a production validator. See [Key rotation](/sdk/latest/keys/key-rotation). +- Migrate a validator to the new key type. See [Migrate a validator to ML-DSA](/sdk/latest/keys/migrate-validator-ml-dsa). diff --git a/sdk/latest/keys/key-rotation.mdx b/sdk/latest/keys/key-rotation.mdx new file mode 100644 index 000000000..43db7426d --- /dev/null +++ b/sdk/latest/keys/key-rotation.mdx @@ -0,0 +1,77 @@ +--- +title: "Key rotation" +description: "How consensus key rotation works: which keys rotate, the two-height delay, the fee, the rotation limit, and how slashing follows a rotated key." +--- + +Key rotation replaces a validator's consensus key in place: no unbonding, no downtime, and no change to the validator's identity, power, or delegations. Rotation is a single operator-signed message on the [`x/staking`](/sdk/latest/keys/rotate-validator-key) and [`enterprise/poa`](/sdk/latest/keys/rotate-validator-key-poa) modules, and realizes a design first proposed in 2019 as [ADR-016](/sdk/latest/reference/architecture/adr-016-validator-consensus-key-rotation). + +## Which keys rotate + +A validator runs on three keys, each with a different owner and job: + +1. **Operator key**: an ordinary account key that the operator generates and custodies, often on a hardware wallet or behind a multisig. It owns the validator: it stakes, sets commission, edits the validator's description, withdraws rewards, votes, and signs the rotation message itself. Like any account key, it has no in-place replacement; retiring one means creating a new account and moving funds. +2. **Consensus key**: generated by the stack when a node initializes, stored in `priv_validator_key.json` or held by a remote signer. It is a raw keypair with no mnemonic behind it; the operator custodies the file, not a seed phrase. It signs the validator's vote on every block, and a block proposal whenever the validator's turn comes to propose. Evidence of misbehavior identifies the validator through this key's consensus address. This is the only key a rotation touches. +3. **Node key**: the node's peer-to-peer identity. It exists so peers can address and authenticate each other: its public key hashes to the node ID used in peer addresses, and it secures the connection handshake between nodes. It carries no on-chain state and regenerates freely. + +On chain, a validator is the pairing of an operator address with a consensus key. Rotation replaces the consensus half of that pairing and leaves everything the operator key controls alone. For the full survey of keys and algorithms, see [Post-quantum keys](/sdk/latest/keys/post-quantum-keys). + +## How a rotation works + +The operator submits `MsgRotateConsPubKey`, carrying the new consensus public key. Everything that defines the validator stays put: the operator address, voting power, delegations, and commission. Only the consensus key and its index change. + +The new key does not take effect immediately. The chain emits the power hand-off to CometBFT right away, setting the old key to zero and giving the new key the validator's full power, and CometBFT's validator update rule makes it effective two heights later, at the same height the chain swaps its stored key. Zero downtime rides on this: the operator runs a second node with the new key alongside the old one. Until the rotation takes effect, the second node follows the chain as a non-signing full node, because a CometBFT node whose key is outside the validator set produces no votes. The moment the new key enters the set, the second node starts signing, the old key holds no power, and the operator retires the old node. The step-by-step procedure is in [Rotate a consensus key, Staking](/sdk/latest/keys/rotate-validator-key). + +## Safety rails + +Three rules bound what a rotation can do: + +1. A rotation burns a flat fee, set by the `key_rotation_fee` staking param, to make rotation spam expensive. +2. A validator can rotate once per unbonding period. Until the unbonding period ends, the previous key remains accountable, so the chain rejects a second rotation inside the window. +3. A rotation cannot be undone. No cancel message exists, and the once-per-unbonding-period limit blocks an immediate rotation back, so an applied rotation stands until the window expires. + +## Security implications + +Read this section carefully. Key rotation introduces security and performance tradeoffs that chains must be aware of before rotating keys. + +### Slashing window length + +Slashing follows a validator's history, not the key. Evidence of a double sign under the old key still slashes the validator after rotation. When a rotation lands, the chain records the old consensus address. It keeps that address tied to the validator. It tracks the address for as long as evidence against it can still be admitted. It computes this window at rotation time from the evidence params `MaxAgeNumBlocks` and `MaxAgeDuration`. Once both elapse, the chain stops tracking the address. The window can be months, depending on the chain's evidence settings. Rotating away from a key does not let a validator escape slashing within that window. + +Downtime slashing carries over too. On a rotation, the validator's missed-block record and jailed status move to the new consensus key. A rotation does not reset them. + +The chain computes a rotation tracking window once at the time of rotation. It never updates this window. If governance extends `MaxAgeNumBlocks` or `MaxAgeDuration` after a rotation, the original window for the rotation remains in effect. In this scenario, the old consensus address will stop being tracked before the current evidence window closes. During that gap, a double sign under the old key cannot be slashed. This is a risk that chains should be aware of. Do not extend the evidence params while rotations are in flight without accounting for it. + +### Increased IBC light client updates + +Frequent rotations may raise the cost of keeping a light client current. Each rotation changes the validator set. Tendermint light clients advance using the overlap between successive validator sets. Heavy rotation churn shrinks that overlap. With less overlap, a relayer must submit more update-client messages to advance the client across the same span. The once-per-unbonding-period limit exists partly to bound this cost. + +### Proposer priority reset + +Rotation resets the validator's proposer priority. CometBFT orders validators by a proposer priority value. That value decides when a validator's turn to propose comes up. A rotation sends the validator to the back of that order. This holds even if the validator was next in line to propose. + +### Slower signature verification + +A validator set with mixed consensus key types verifies signatures more slowly. CometBFT batch-verifies signatures when every validator uses the same key type. Batch verification is faster than checking each signature on its own. One validator on a different key type breaks batching. Verification then falls back to one signature at a time, which can slow block times. This applies whenever the set holds more than one key type, not only during a rotation. + +## Exports and restarts + +In-progress rotations survive a genesis export. A chain exported mid-rotation carries the pending rotation and its remaining history window into the new genesis, so restarting a chain does not lose slashing accountability or drop a queued key change. + +## Staking and PoA chains + +Rotation ships in both validator models. On staked chains, it is the `x/staking` implementation described above; for the procedure, see [Rotate a consensus key, Staking](/sdk/latest/keys/rotate-validator-key). Chains running `enterprise/poa` get the same rotation with two differences. The admin can rotate any validator's key, not only the operator. And because PoA has no slashing or evidence handling, the safety rails above do not apply: no fee, no rate limit, no rotation history, and the state swap happens in the block the transaction lands. For the procedure, see [Rotate a consensus key, PoA](/sdk/latest/keys/rotate-validator-key-poa). + + +Vote extensions are not supported on PoA chains. PoA chains running custom logic that resolves a validator from a `LastCommit` address or that uses vote extensions may experience unexpected behavior during the two-height delay after a rotation. Read on for more details. + + +The PoS and PoA models differ in how they handle CometBFT's two-height delay. After a key rotation, CometBFT keeps signing `LastCommit` with the old consensus address for two heights. Staking waits out those heights before swapping its own state and keeps a historical address mapping, so the old address still resolves to its validator. PoA swaps immediately and keeps no mapping, so during those two heights it cannot resolve the old address that `LastCommit` still carries. A stock PoA chain never notices, because it runs neither x/distribution nor x/slashing (the modules that generally read those addresses). However, custom logic that resolves a validator from a `LastCommit` address on a PoA chain will not find a rotating validator for those two heights, which can lead to unexpected behavior. + +For this reason, vote extensions are not supported on PoA chains. The chain verifies a vote-extension signature by the `LastCommit` address. If a validator rotates its key during the two-height delay, the chain will reject the vote extension because the `LastCommit` address will not resolve to the validator. The standard `ValidateVoteExtensions` helper returns an error on the first commit vote whose `LastCommit` address it cannot resolve, before it tallies any voting power. One rotating validator that signed the previous block is therefore enough to have the whole extended commit rejected, whatever its share of voting power. + +## Next steps + +- Perform a rotation on a staked chain. See [Rotate a consensus key, Staking](/sdk/latest/keys/rotate-validator-key). +- Perform a rotation on a PoA chain. See [Rotate a consensus key, PoA](/sdk/latest/keys/rotate-validator-key-poa). +- Understand the key types. See [Post-quantum keys](/sdk/latest/keys/post-quantum-keys). +- Look up the message, parameters, and state layout. See the [x/staking module reference](/sdk/latest/modules/staking/README#msgrotateconspubkey). diff --git a/sdk/latest/keys/migrate-validator-ml-dsa.mdx b/sdk/latest/keys/migrate-validator-ml-dsa.mdx new file mode 100644 index 000000000..1732e247a --- /dev/null +++ b/sdk/latest/keys/migrate-validator-ml-dsa.mdx @@ -0,0 +1,85 @@ +--- +title: "Migrate a validator to ML-DSA" +description: "Move a validator's consensus key to the post-quantum ml_dsa_65 algorithm through an ordinary key rotation." +--- + +Migrating a validator to post-quantum signing is an ordinary key rotation with an ML-DSA target key. This guide adds the ML-DSA-specific steps around the standard procedure in [Rotate a consensus key, Staking](/sdk/latest/keys/rotate-validator-key). For more information on ML-DSA, see [Post-quantum keys](/sdk/latest/keys/post-quantum-keys). + +Key rotation can introduce security implications for your chain. Read the [Key rotation](/sdk/latest/keys/key-rotation) overview in its entirety before proceeding. + +Before rotating any validator to ML-DSA, confirm every counterparty chain that verifies this chain over IBC runs CometBFT v0.40 or later. An older `07-tendermint` light client cannot verify ML-DSA consensus signatures. See [IBC considerations](/sdk/latest/keys/post-quantum-keys#ibc-considerations) for more information. + +## Prerequisites + +- All prerequisites of the rotation procedure: [jq](https://jqlang.org/) and [curl](https://curl.se/), no rotation in the current unbonding period, and fee funds on the operator account. See [Rotate a consensus key, Staking](/sdk/latest/keys/rotate-validator-key). +- The chain's binary; the examples use `simd`. To build it and run a node, see [Run a node](/sdk/latest/node/run-node). + +## 1. Confirm the chain allows ML-DSA + +Validator key types are a consensus parameter. Check that `ml_dsa_65` is in the list: + +```shell +simd query consensus params +``` + +If `validator.pub_key_types` does not include `ml_dsa_65`, the rotation is rejected. To add the type, see [Enable ML-DSA keys](/sdk/latest/keys/enable-ml-dsa-keys). + +## 2. Rotate to an ML-DSA key + +### On a staking chain + +Follow [Rotate a consensus key, Staking](/sdk/latest/keys/rotate-validator-key), replacing the `simd init` command in its step 1 with one that adds `--consensus-key-algo ml_dsa_65` to create an ML-DSA key: + +```shell +simd init rotation-node --chain-id my-chain-1 --consensus-key-algo ml_dsa_65 --home ~/.rotation-node +``` + +Everything else runs as written. The guide's rotation command derives the public key with `simd comet show-validator` on the second node's home, which now prints the ML-DSA key, so the rotation message carries it automatically. + +### On a PoA chain + +Follow [Rotate a consensus key, PoA](/sdk/latest/keys/rotate-validator-key-poa) with two changes. Replace the `simd init` command in its step 1 with: + +```shell +simd init poa-newkey --chain-id my-chain-1 --consensus-key-algo ml_dsa_65 --home ~/.poa-newkey +``` + +And pass `ml_dsa_65` instead of `ed25519` as the key type when submitting. The ML-DSA public key may be large enough that the default gas limit runs out, so add `--gas auto`: + +```shell +simd tx poa rotate-cons-pub-key "$(simd comet show-validator --home ~/.poa-newkey | jq -r .key)" ml_dsa_65 --operator-address "$(simd keys show val -a --home ~/.node)" --from val --home ~/.node --gas auto --gas-adjustment 1.5 +``` + +The cutover timing is unchanged: keep the node on the old key until the validator set switches, then swap the key file in place, exactly as the guide's [steps 3 and 4](/sdk/latest/keys/rotate-validator-key-poa#3-wait-for-the-validator-set-to-switch) describe. + +### On a remote signer + +If the validator's consensus key lives in Cosmos-KMS rather than a local file, the second node gets its own signer and the public key derivation differs. See [Rotate a consensus key held in Cosmos-KMS](/sdk/latest/kms/rotate-key-remote-signer). + +{/* +CURRENT VERIFICATION: blind execution audit 2026-07-28 at shipping refs, cosmos-sdk v0.55.0 (64fd208a11), cometbft v0.40.0, PoA via enterprise/poa/simapp, macOS 26.5.2 arm64. Verdict PASS: both the staking and PoA ML-DSA rotations execute as written. Staking rotation returned code 0 at height 33 and /validators reported cometbft/PubKeyMlDsa65 with the chain still producing. Confirmed that --gas auto is required, since an ML-DSA pubkey exceeds the 200000 default (observed gasUsed 217674 to 233565). Full run: ~/Documents/tests/keys-docs-audit-findings.md. +Earlier run, retained for provenance: verified 2026-07-17 with simd from cosmos-sdk pr-26604 @ 46a177139a. #26604 has since merged and is in v0.55.0, so the "re-verify at merge" instruction is discharged. +*/} +{/* PoA ML-DSA rotation run-verified 2026-07-28 on cosmos-sdk v0.55.0 (includes #26614, merged as d6a3c6e27a): the command below executes as written on a PoA simapp binary, the module reports /cosmos.crypto.mldsa65.PubKey, and the chain resumes producing on the ML-DSA consensus key. Needs --gas auto (an ML-DSA pubkey exceeds the 200000 default). */} + +## 3. Verify + +Check the key type in the validator set: + +```shell +curl -s localhost:26657/validators | jq -r '.result.validators[].pub_key.type' +``` + +A migrated validator reports `cometbft/PubKeyMlDsa65` instead of `tendermint/PubKeyEd25519`. The chain's consensus is post-quantum secure once validators holding at least two thirds of voting power report a post-quantum type. For more information, see [Post-quantum keys](/sdk/latest/keys/post-quantum-keys). + +## What can go wrong + +- The rotation is rejected for an unsupported key type: the chain does not list `ml_dsa_65` yet. See [Enable ML-DSA keys](/sdk/latest/keys/enable-ml-dsa-keys). +- Anything else follows the standard rotation failure modes. See [Rotate a consensus key, Staking](/sdk/latest/keys/rotate-validator-key). + +The consensus key this rotation installs is a raw keypair with no mnemonic behind it. Custody the key file, not a seed phrase. + +## Next steps + +- Check which key types the chain allows and which the validator set is running. See [Enable ML-DSA keys](/sdk/latest/keys/enable-ml-dsa-keys). +- Understand the storage and bandwidth costs the chain takes on as the set migrates. See [Post-quantum keys](/sdk/latest/keys/post-quantum-keys). diff --git a/sdk/latest/keys/post-quantum-keys.mdx b/sdk/latest/keys/post-quantum-keys.mdx new file mode 100644 index 000000000..a5da59c3e --- /dev/null +++ b/sdk/latest/keys/post-quantum-keys.mdx @@ -0,0 +1,86 @@ +--- +title: "Post-quantum keys" +description: "The keys and signature algorithms of a Cosmos chain, what post-quantum security means, and what adopting ML-DSA costs." +--- + +A Cosmos chain uses various keys backed by a set of signature algorithms, including ML-DSA, a native post-quantum option for consensus keys and user accounts. This page surveys the keys and algorithms and explains what post-quantum security means and how it applies. + +A post-quantum key signs with an algorithm that stays secure against an attacker equipped with a quantum computer. + +## Keys and algorithms + +Keys are the foundation of a chain's security: funds, consensus votes, and governance are only as safe as the keys that sign for them. Every signature on a Cosmos chain comes from one of four keys, each held by a different party and signing different things: + +| Key | Held by | Signs | Algorithms | +| --- | --- | --- | --- | +| User account key | Anyone with an account | Transactions | | +| Validator operator key | The validator's operator | Staking transactions |
  • Same as the user account key.
| +| Validator consensus key | The validator node | Votes and proposals | | +| Node key | Every node | Peer-to-peer identity | | + +Consensus params decide which algorithms consensus keys may use on a given chain, while users pick an account algorithm each time they create a key. To check what a chain currently allows and which algorithms its validator set is running, see [Enable ML-DSA keys](/sdk/latest/keys/enable-ml-dsa-keys#check-the-current-state). For information on validator key rotation, see [Key rotation](/sdk/latest/keys/key-rotation). + +## What post-quantum means + +Every algorithm in the stack except `ml_dsa_65` is based on an elliptic curve. A sufficiently powerful quantum computer running [Shor's algorithm](https://en.wikipedia.org/wiki/Shor%27s_algorithm) breaks elliptic curve cryptography outright: no key size makes a curve safe. Account keys, operator keys, and consensus keys therefore share the same long-term exposure, and when practical quantum hardware arrives, curve-based signatures stop being trustworthy. To counter this, the `ml_dsa_65` key algorithm is introduced. + +## How ML-DSA works + +The Module-Lattice-Based Digital Signature Algorithm (ML-DSA) is NIST's lattice-based signature standard, published as [FIPS 204](https://csrc.nist.gov/pubs/fips/204/final) in 2024 and the finalized form of CRYSTALS-Dilithium. Instead of deriving security from elliptic curves, it builds keys and signatures on [lattice problems](https://en.wikipedia.org/wiki/Lattice-based_cryptography), a class of mathematics with no known quantum attack. + +The Cosmos stack uses the middle FIPS 204 parameter set, ML-DSA-65 (NIST security category 3), as the algorithm `ml_dsa_65` to implement post-quantum security. + +With the addition of this key algorithm, nothing about the signing workflow changes. The keyring generates and recovers an ML-DSA account like any other, and consensus treats an ML-DSA consensus key like any other key type. What differs is the math underneath and the size of the keys and signatures it produces. + +## Is hashing post-quantum secure? + +Yes, the SHA-256 hash function that underpins the Cosmos SDK and CometBFT (block transaction hashes, merkle trees over application state, etc.) are considered post-quantum secure. Unlike RSA and elliptic-curve cryptography, which are broken by Shor's algorithm, the best known quantum attack against generic hash functions is [Grover's algorithm](https://en.wikipedia.org/wiki/Grover%27s_algorithm). Grover's algorithm provides only a quadratic speedup, reducing SHA-256's preimage resistance from 256 bits to about 128 bits, which is still considered secure. Collision resistance, the property that matters for merkle trees and transaction hashes, was already about 128 bits classically and is essentially unaffected. + +## Who can adopt ML-DSA? + +Only account keys and consensus keys can use `ml_dsa_65`. The node key stays `ed25519` and merely identifies a node to its peers, and module accounts and smart contract accounts hold funds without any key at all, so none of them has anything to migrate. + +Adoption differs by role. A user can generate a new ML-DSA account and move funds into it at any time, with no chain-level permission required; there is no in-place migration for accounts. A validator migrates its consensus key in place through key rotation, which does require the chain to allow `ml_dsa_65` in consensus params first. + +## When is a chain considered post-quantum? + +Consensus security follows voting power. A chain's consensus becomes post-quantum secure once at least two thirds of voting power signs with post-quantum consensus keys, because two thirds is the threshold an attacker must forge to break finality. Account security is individual: each account is exactly as secure as its own key. + +## The cost of post-quantum keys + +Post-quantum security trades larger keys and signatures for quantum resistance. Signatures dominate the added cost because every block commit carries one per validator, so the totals below scale with the validator set. + +A single ML-DSA signature is about 3,300 bytes, over 50 times the size of an ed25519 signature. Every block stores one per validator, which drives the block-data growth shown below. + +The example below assumes 100 validators and six-second blocks. + +| Measure | `ed25519` | `ml_dsa_65` | +| --- | --- | --- | +| Public key | 32 B | 1,952 B | +| Signature | 64 B | 3,309 B | +| Signature data per block | ~6 KB | ~331 KB | +| Total block size, with ~4 KB fixed overhead | ~10 KB | ~335 KB | +| Total block data per year | ~56 GB | ~1.8 TB | + +Signing and verification are slightly slower than with `ed25519`. This is unlikely to affect most chains. + +## IBC considerations + +A chain migrating to ML-DSA consensus keys must be aware that this change affects IBC verification. + +Any counterparty chain that verifies an ML-DSA-enabled chain with an `07-tendermint` light client must be upgraded to CometBFT v0.40 or later. An older client fails as soon as the first ML-DSA validator joins the set, so every counterparty must upgrade to v0.40 before any validator can rotate to ML-DSA. + +ML-DSA signatures also enlarge block headers, which enlarges the IBC client updates that carry them. CometBFT v0.40 raises its signature-size limits to accommodate the larger signatures. + +## EVM chains + +Validators on EVM chains can run ML-DSA consensus keys, as on any other chain. User accounts cannot: the EVM requires `eth_secp256k1` account keys, and those cannot move to a post-quantum scheme in place. + +Ethereum's path to post-quantum accounts runs through account abstraction instead. [EIP-7702](https://eips.ethereum.org/EIPS/eip-7702) already lets an account delegate to contract code, so a contract can verify a post-quantum signature inside the VM. [EIP-8051](https://eips.ethereum.org/EIPS/eip-8051) proposes a native precompile for ML-DSA verification, and [EIP-8141](https://eips.ethereum.org/EIPS/eip-8141) proposes frame transactions, which let individual accounts adopt new signature schemes. As a fully EVM-compatible ledger, Cosmos EVM conforms to that roadmap and ships user-side post-quantum support as it lands upstream. + +## Next steps + +- Upgrade first; every flow on this page requires SDK 0.55 and CometBFT 0.40. See the [release notes](/sdk/latest/upgrade/v0.55-release). +- Create a post-quantum user account and move funds into it. See [Create an ML-DSA account](/sdk/latest/keys/create-ml-dsa-account). +- Allow `ml_dsa_65` on a new or live chain. See [Enable ML-DSA keys](/sdk/latest/keys/enable-ml-dsa-keys). +- Learn how rotation works before touching a production validator. See [Key rotation](/sdk/latest/keys/key-rotation). diff --git a/sdk/latest/keys/rotate-validator-key-poa.mdx b/sdk/latest/keys/rotate-validator-key-poa.mdx new file mode 100644 index 000000000..003e06e71 --- /dev/null +++ b/sdk/latest/keys/rotate-validator-key-poa.mdx @@ -0,0 +1,148 @@ +--- +title: "Rotate a consensus key, PoA" +description: "Rotate a PoA validator's consensus key as the operator or the admin: generate the key, submit the rotation, and time the node cutover." +--- + +This guide rotates a PoA validator's consensus key using the `enterprise/poa` module. PoA rotation drops three of the staking rails: there is no fee, no rate limit, and no rotation history. PoA chains have no slashing or evidence handling to protect. The operator can rotate its own key, and the chain admin can rotate any validator's key. For how rotation works in general, see [Key rotation](/sdk/latest/keys/key-rotation). + +Key rotation can introduce security implications for your chain. Read the [Key rotation](/sdk/latest/keys/key-rotation) overview in its entirety before proceeding. In particular, vote extensions are not supported on PoA chains, and custom logic that resolves a validator from a `LastCommit` address will not find a rotating validator for two heights after a rotation. See [Staking and PoA chains](/sdk/latest/keys/key-rotation#staking-and-poa-chains). + +Commands use `simd`. Substitute your chain's binary. Examples use `~/.node` for the validator node's home, `~/.poa-newkey` for the scratch home holding the new key, and `val` for the operator key name. + +{/* +Manual verification setup (hidden, macOS). Reproduces a single-validator PoA chain, using the enterprise/poa simapp. +Build: cd enterprise/poa/simapp && go build -o /tmp/poa-simd ./simd, then use /tmp/poa-simd as `simd`. +PoA validators come from poa genesis (no gentx); operator_address is a plain acc address (cosmos1...), not cosmosvaloper1.... + +simd init mynode --chain-id my-chain-1 --home ~/.node +simd config set client chain-id my-chain-1 --home ~/.node +simd config set client keyring-backend test --home ~/.node +simd keys add val --keyring-backend test --home ~/.node +simd keys add admin --keyring-backend test --home ~/.node +VAL=$(simd keys show val -a --keyring-backend test --home ~/.node) +ADMIN=$(simd keys show admin -a --keyring-backend test --home ~/.node) +simd genesis add-genesis-account $VAL 1000000000000stake --home ~/.node +simd genesis add-genesis-account $ADMIN 1000000000000stake --home ~/.node +G=~/.node/config/genesis.json +jq --arg admin "$ADMIN" '.app_state.poa.params.admin=$admin' $G > /tmp/g && mv /tmp/g $G +jq --arg op "$VAL" --slurpfile pk <(jq '{pub_key:{"@type":"/cosmos.crypto.ed25519.PubKey", key:.pub_key.value}, power:10000000, metadata:{moniker:"mynode", operator_address:"'$VAL'"}}' ~/.node/config/priv_validator_key.json) '.app_state.poa.validators=$pk' $G > /tmp/g && mv /tmp/g $G +sed -i '' 's/timeout_commit = ".*"/timeout_commit = "1s"/' ~/.node/config/config.toml +sed -i '' 's/minimum-gas-prices = ".*"/minimum-gas-prices = "0stake"/' ~/.node/config/app.toml +simd start --home ~/.node +# then follow the guide. Note: on a single-validator chain the block production halts between the set +# switch (step 3) and the node key swap (step 4); that is expected and step 4 recovers it. +*/} + +{/* CURRENT VERIFICATION: blind execution audit 2026-07-28 at cosmos-sdk v0.55.0 (64fd208a11) with the PoA binary from enterprise/poa/simapp. Verdict PARTIAL, the only defect being the unsatisfiable "v1.1 or later" prerequisite, since corrected on this page. Every procedural step verified: module state swapped in the landing block h44 with the CometBFT set switching at h46, power/moniker/operator preserved, the >1/3 halt Danger reproduced (chain froze at h45 until the swap), step 4 recovery resumed to h67, and all four documented rejections fired including the unauthorized-signer and nothing-to-rotate cases. Full run: ~/Documents/tests/keys-docs-audit-findings.md. Earlier run, retained for provenance: ed25519 run-verified 2026-07-20 against merged enterprise/poa (cosmos-sdk origin/main 3d3b901ce5, includes #26590 merge c52e67a39d; poa binary built via `cd enterprise/poa/simapp && go build ./simd`). Operator self-rotation and admin override PASS: rotate_cons_pubkey event with operator/old/new, power+operator preserved, set switch at tx height +2, fees migrated; all four rejections reproduced (already-in-use, nothing-to-rotate, unauthorized, type-not-in-params); power-0 note holds; cutover timing confirmed. Full ed25519 run: ~/Documents/tests/poa-rotation-test-findings.md. ML-DSA: run-verified 2026-07-28 on cosmos-sdk v0.55.0, which includes #26614 (merged as d6a3c6e27a). Operator and admin ML-DSA rotation both execute, the set switches to cometbft/PubKeyMlDsa65, and --gas auto is required (an ML-DSA pubkey exceeds the 200000 default). The earlier `unknown pubkey type: ml_dsa_65` failure no longer reproduces; ml_dsa_65 is registered in the pubkey factory at enterprise/poa/x/poa/module.go:134 behind the module option the PoA simapp enables. */} + +## Prerequisites + +- A chain binary built with the `poa` module. Clone the cosmos-sdk repo and build it: `cd enterprise/poa/simapp && go build -o /tmp/poa-simd ./simd`, then use that binary as `simd`. The `-o` is required, since `./simd` is the package directory. Check with `simd tx poa --help`, which must list `rotate-cons-pub-key`. +- [jq](https://jqlang.org/) and [curl](https://curl.se/). The commands extract the new public key with jq and watch the validator set with curl. +- A running PoA validator you operate, with the chain's binary on your PATH and a secondary machine or scratch directory for the new key. PoA validators are set in genesis under `app_state.poa`. + {/* app_state.poa shape: "poa": { "params": { "admin": "cosmos1..." }, "validators": [{ "pub_key": { "@type": "/cosmos.crypto.ed25519.PubKey", "key": "" }, "power": "10000000", "metadata": { "moniker": "mynode", "operator_address": "cosmos1..." } }] } */} +- For an ML-DSA rotation, `ml_dsa_65` is in the chain's consensus params. See [Enable ML-DSA keys](/sdk/latest/keys/enable-ml-dsa-keys). +- The signer is the validator's operator, or the chain admin. Any other sender is rejected. + +## 1. Generate a new consensus key + +Create the new key on a secondary machine or offline. Do not touch the live node's `priv_validator_key.json` yet: + +```shell +simd init poa-newkey --chain-id my-chain-1 --home ~/.poa-newkey +``` + +To rotate to a post-quantum key, add `--consensus-key-algo ml_dsa_65` to the command above. See [Migrate a validator to ML-DSA](/sdk/latest/keys/migrate-validator-ml-dsa). + +The important output is `~/.poa-newkey/config/priv_validator_key.json`, which is the new consensus key. Confirm it and read its public key: + +```shell +simd comet show-validator --home ~/.poa-newkey +``` + +## 2. Submit the rotation + +The command takes the new public key as base64 plus its type, and identifies the validator by operator address. Both derive from earlier steps: + +```shell +simd tx poa rotate-cons-pub-key "$(simd comet show-validator --home ~/.poa-newkey | jq -r .key)" ed25519 --operator-address "$(simd keys show val -a --home ~/.node)" --from val --home ~/.node +``` + +To rotate to a post-quantum key, pass `ml_dsa_65` instead of `ed25519` in the command above, and add `--gas auto --gas-adjustment 1.5` since the ML-DSA public key exceeds the default gas limit. See [Migrate a validator to ML-DSA](/sdk/latest/keys/migrate-validator-ml-dsa). + +The operator address is a regular account address (`cosmos1...`), not a `cosmosvaloper1...` address, so read it with `simd keys show val -a`. + +Add standard transaction flags as your setup requires: `--chain-id` and `--keyring-backend` if your client config does not supply them, and `--fees` (or `--gas-prices`) to meet the chain's minimum gas price. + +The transaction re-keys the validator's state and migrates its accrued fees in the same block. Power, metadata, and the operator address are unchanged. + +To rotate as the admin instead, see [Rotate as the admin](#rotate-as-the-admin). + +If the rotated validator holds more than 1/3 of voting power, the chain halts during the cutover period (steps 3 and 4) and does not resume until the node with the rotated key starts signing. + +## 3. Wait for the validator set to switch + +The chain's state swaps immediately, so `simd q poa validators` shows the new key in the same block the transaction lands. CometBFT applies the actual validator set change two blocks later, and only that switch governs when the node must sign with the new key. Until it happens, CometBFT still expects the old key, so keep the live node running untouched. Watch the CometBFT validator set until the new consensus key appears and the old one is gone: + +```shell +curl -s localhost:26657/validators | jq -r '.result.validators[].pub_key.value' +``` + +The value should match the key you read in step 1. + +Do not swap the node's key before the set switches (wait at least 2 blocks). Swapping early makes the node sign with a key CometBFT does not yet expect, and the validator misses blocks. + +## 4. Swap the node's key + +Once the new consensus address is in the set, stop the node, replace its key, and restart: + +```shell +cp ~/.poa-newkey/config/priv_validator_key.json ~/.node/config/priv_validator_key.json +``` + +Confirm the node signs under the new consensus address after the restart. If you run a redundant standby, make a single switch. Keep the old-key node signing until the set updates, stop it fully, and only then let the new-key node start signing. + +A validator with power 0 is outside the active set. Its rotation emits no validator set update, so there is no transition to time. Swap the node's key first, then have the admin grant power. + +## Rotate as the admin + +The admin override changes on-chain state only. Whoever runs the node must still swap `priv_validator_key.json` with the timing in steps 3 and 4 above. Otherwise the validator goes dark until its node signs with the new key. Coordinate the node-side swap with the operator before submitting, unless the goal is to cut off a compromised key. + +The admin rotates any validator's key with the same command, signed by the admin key. Generate a fresh key home for it: + +```shell +simd init poa-adminkey --chain-id my-chain-1 --home ~/.poa-adminkey +``` + +Then rotate the key: + +```shell +simd tx poa rotate-cons-pub-key "$(simd comet show-validator --home ~/.poa-adminkey | jq -r .key)" ed25519 --operator-address --from admin --home ~/.node +``` + +## Verify + +Confirm the module carries the new key: + +```shell +simd q poa validators --home ~/.node +``` + +The rotation also emits a `rotate_cons_pubkey` event with the operator address and the old and new consensus addresses. Read it from the transaction: + +```shell +simd q tx --home ~/.node +``` + +## What can go wrong + +- The transaction is rejected as unauthorized: the signer is neither the validator's operator nor the admin. +- The transaction is rejected for the key itself: the new key equals the current one, is already used by another validator, or its type is not in the chain's consensus params. +- The validator misses blocks right after the swap: the node's key was replaced before the set switched. Restore the old key, wait for the set, then swap again. +- The validator goes dark after an admin rotation: the node still holds the old key. Swap `priv_validator_key.json` and restart. + +## Next steps + +- Rotate to a post-quantum key. See [Migrate a validator to ML-DSA](/sdk/latest/keys/migrate-validator-ml-dsa). +- Understand the mechanics and the staking differences. See [Key rotation](/sdk/latest/keys/key-rotation). +- Look up the message. See the [PoA API reference](/sdk/latest/enterprise/poa/api#rotateconspubkey). diff --git a/sdk/latest/keys/rotate-validator-key.mdx b/sdk/latest/keys/rotate-validator-key.mdx new file mode 100644 index 000000000..379b2eb97 --- /dev/null +++ b/sdk/latest/keys/rotate-validator-key.mdx @@ -0,0 +1,124 @@ +--- +title: "Rotate a consensus key, Staking" +description: "Rotate a staked validator's consensus key with no downtime: run a second node, submit the rotation, verify, and retire the old node." +--- + +This guide rotates a staked validator's consensus key with no downtime: run a second node on the new key, submit the rotation, verify, and retire the old node. For how rotation works and its limits, see [Key rotation](/sdk/latest/keys/key-rotation). + +Key rotation can introduce security implications for your chain. Read the [Key rotation](/sdk/latest/keys/key-rotation) overview in its entirety before proceeding. + +Commands use `simd`. Substitute your chain's binary. Examples use `~/.node` for the existing node's home and `~/.rotation-node` for the new one; those paths, the key name `val`, and host addresses are the only values to adjust. + +{/* +Manual verification setup (hidden, macOS). Reproduces the running staked chain this guide assumes, using simd/simapp. +Build: from the cosmos-sdk repo run `make build`, then put ./build/simd on PATH as `simd`. + +simd init mynode --chain-id my-chain-1 --home ~/.node +simd config set client chain-id my-chain-1 --home ~/.node +simd config set client keyring-backend test --home ~/.node +simd keys add val --keyring-backend test --home ~/.node +simd genesis add-genesis-account val 100000000000stake --keyring-backend test --home ~/.node +simd genesis gentx val 1000000000stake --chain-id my-chain-1 --keyring-backend test --home ~/.node +simd genesis collect-gentxs --home ~/.node +# fast blocks + zero min-gas so the guide's commands work without extra flags +# (substitute --fees in step 3 with e.g. 2000stake, or omit it on this localnet) +sed -i '' 's/timeout_commit = ".*"/timeout_commit = "1s"/' ~/.node/config/config.toml +sed -i '' 's/minimum-gas-prices = ".*"/minimum-gas-prices = "0stake"/' ~/.node/config/app.toml +simd start --home ~/.node +# then follow the guide. The new node in step 1 shares this host, so it also needs +# --p2p.laddr/--rpc.laddr/--grpc.address and a distinct pprof_laddr (see step 1's note). +*/} + +## Prerequisites + +- The chain runs Cosmos SDK 0.55 and CometBFT 0.40 or later, and allows your target key type. See [Enable ML-DSA keys](/sdk/latest/keys/enable-ml-dsa-keys). +- [jq](https://jqlang.org/) and [curl](https://curl.se/) for the verification steps. +- A running validator you operate, and a second machine (or spare ports on the same host) for the new node, with the chain's binary installed on it. To build `simd` and run a node, see [Run a node](/sdk/latest/node/run-node). +- No rotation in the current unbonding period. Only one rotation is allowed per unbonding period. +- The operator account holds enough funds for two separate charges: the rotation fee, which is burned, and the ordinary gas fee for the transaction itself. Check the rotation fee: + +```shell +simd query staking params +``` + +The `key_rotation_fee` field shows the fee amount. + +## 1. Start a second node with the new key + +Initialize a fresh node home. The init command generates a new consensus key in `priv_validator_key.json`: + +```shell +simd init rotation-node --chain-id my-chain-1 --home ~/.rotation-node +``` + +To rotate to a post-quantum key, add `--consensus-key-algo ml_dsa_65` to the command above. See [Migrate a validator to ML-DSA](/sdk/latest/keys/migrate-validator-ml-dsa). + +Never copy the old `priv_validator_key.json` to the new node. Two nodes signing with the same consensus key is a double sign, which tombstones the validator. The new node must have its own freshly generated key. + +A fresh init writes a placeholder genesis. Replace it with the chain's genesis: + +```shell +cp ~/.node/config/genesis.json ~/.rotation-node/config/genesis.json +``` + +Start the new node peered with the existing one, and let it sync to the chain head: + +```shell +simd start --home ~/.rotation-node --p2p.persistent_peers "$(simd comet show-node-id --home ~/.node)@127.0.0.1:26656" +``` + +Adjust the peer address to the existing node's host. If both nodes share a host, also give the new node its own ports with `--p2p.laddr`, `--rpc.laddr`, `--grpc.address`, and `--rpc.pprof_laddr`. The pprof port is required, not optional: without it the new node exits at startup with `address already in use` for the default port 6060, which the first node already holds. Until the rotation applies, the node follows the chain as a non-signing full node. + +On a chain with history, a fresh node takes days to sync from genesis. Use state sync or a snapshot to reach the chain head quickly. See [State sync](/sdk/latest/node/run-node#state-sync). + +## 2. Confirm both nodes are healthy + +Before you submit, the old node must still be signing and the new node must be caught up to the chain head. If the new node is still syncing when the rotation applies, the validator will miss blocks until it catches up. Check sync status: + +```shell +curl -s localhost:26657/status | jq '.result.sync_info.catching_up' +``` + +Run this against each node, using the RPC port each one listens on (a co-located new node answers on the `--rpc.laddr` port you gave it, not `26657`). Both must report `false`. + +## 3. Submit the rotation + +The rotation message carries the new node's public key, read directly from that node's home: + +```shell +simd tx staking rotate-cons-pub-key "$(simd comet show-validator --home ~/.rotation-node)" --from val --home ~/.node --gas auto --gas-adjustment 1.5 --fees --yes +``` + +Set `--fees` (or `--gas-prices`) to meet the chain's minimum gas price; without it the node rejects the transaction with `insufficient fees`. This gas fee is separate from the burned rotation fee. The command reads the operator key and chain ID from your client config; add `--chain-id`, `--keyring-backend`, and `--home` if that config does not already supply them. + +A rotation cannot be undone. After it applies, the validator is committed to the new key for the rest of the unbonding period. Keep the old node running until step 4 verifies the new key is signing. + +## 4. Verify the new key is signing + +The rotation applies two heights after the message executes, so the new key can appear within seconds on a fast chain. A successful broadcast confirms only that the chain accepted the transaction; confirm the validator set actually carries the new key: + +```shell +curl -s localhost:26657/validators | jq -r '.result.validators[].pub_key' +``` + +The `value` matches the `key` field shown by `simd comet show-validator --home ~/.rotation-node`, and the old key is gone. If the set still shows the old key, wait a few blocks and check again. Watch the new node's logs to see it signing votes. + +## 5. Retire the old node + +Stop the old node and decommission it. Its consensus key holds no power, but it remains slashable for past behavior until equivocation evidence for it can no longer be admitted. That window is at least the unbonding period and can be longer, depending on the chain's evidence params. See [Key rotation](/sdk/latest/keys/key-rotation). Store its key material securely rather than leaving it on shared infrastructure. + +## What can go wrong + +- The transaction is rejected with a rotation limit error: a rotation already happened this unbonding period. Wait out the window. +- The transaction is rejected for an unsupported key type: the target type is not in the chain's consensus params. See [Enable ML-DSA keys](/sdk/latest/keys/enable-ml-dsa-keys). +- The fee cannot be paid: fund the operator account with at least `key_rotation_fee`. +- The transaction is rejected because the new key is unavailable: another validator already uses it, or a recent rotation still holds it locked. Generate a fresh key. +- The transaction is rejected because the validator is jailed: unjail it first. +- The validator misses blocks after the rotation applies: the new node was not caught up. It resumes signing once synced. + +## Next steps + +- Understand the mechanics behind each step. See [Key rotation](/sdk/latest/keys/key-rotation). +- Rotate to a post-quantum key. See [Migrate a validator to ML-DSA](/sdk/latest/keys/migrate-validator-ml-dsa). +- Rotate a key held in a remote signer. See [Rotate a consensus key held in Cosmos-KMS](/sdk/latest/kms/rotate-key-remote-signer). +- Look up the message and parameters. See the [x/staking module reference](/sdk/latest/modules/staking/README#msgrotateconspubkey). diff --git a/sdk/latest/kms/best-practices.mdx b/sdk/latest/kms/best-practices.mdx new file mode 100644 index 000000000..6cabdd3a7 --- /dev/null +++ b/sdk/latest/kms/best-practices.mdx @@ -0,0 +1,62 @@ +--- +title: "Remote signing best practices" +description: "Choose and defend a signing architecture: placement, transport, key separation, and the one-signer rule." +--- + +A remote signing setup is a set of trust decisions: where the signer runs, how the connection is secured, and which keys are protected to what degree. This page states the recommended defaults and the reasoning, so a setup can be defended in a security review rather than inherited by accident. + +## Place the signer in its own trust domain + +Run the signer on a separate host, in a separate network segment, with network ACLs between it and the validator. The validator node is the exposed machine: it peers with the public network and sees frequent maintenance. The signing host should do exactly one job, accept no inbound connections, and be reachable by as few people and systems as possible. + +The signer dials out to the validator, so this layout costs nothing: the signing host needs no open ports at all. The validator's privval listener is the only listening side; bind it to a private interface and firewall it so only the signing host can reach it. + +Choose the layout by the deployment: + +- A separate host with a network firewall is the default and reduces the attack surface the most. +- If the validator sits behind a sentry node, network isolation is already in place, so running the signer locally is reasonable. +- If cost is a constraint, running both processes on the same host still keeps the key in custody and off the node's disk, an acceptable tradeoff for the core benefit of remote signing. + +In the two local layouts the key still never touches disk, but the signer process and its credentials live on the exposed machine. + +Every layout adds some network latency between node and signer, so weigh that against the isolation each one provides. + +## Prefer the Noise transport + +Two transports secure the privval connection; the address scheme in `kms.yaml` selects between them. The default `tcp://` uses CometBFT's SecretConnection: the signer authenticates itself to the validator with its identity key. The validator's listener uses an ephemeral key, so the signer cannot verify it is talking to the right validator. + +The `noise://` transport closes that gap with mutual pinning. Each side asserts a stable peer ID: the signer's derives from its identity key, the validator's from its node key. Each side refuses a connection from any unexpected peer. Exchange the two peer IDs out of band and pin them: + +```shell +kms peer-id --home ~/.kms +``` + +```shell +cometbft show-node-id --libp2p --home ~/.node +``` + +Use `noise://` for any deployment where the signer and validator cross a network you do not fully control. + +## Protect keys according to their power + +The consensus key is the asset; keep it in the HSM or cloud KMS and never in a file on production hosts. The signer's identity key (`identity.json`) only authenticates the connection: it signs no consensus messages, and losing it means re-pinning a new peer ID, not a compromise. The two keys do not need the same protection level. Treating the identity key as low value keeps operational friction down. + +## Signer, chain, and key topology + +One signer can sign for several chains at once. Each chain is backed by exactly one key, so a single signer can hold several keys, one per chain. For redundancy, one signer can also connect to more than one node on the same chain, such as a primary and a backup. Only one live signer may hold a given key, as the next section explains. + +## Run exactly one signer per validator + +Double-sign protection lives in the signer's per-chain state file, which records the highest height, round, and step ever signed. That protection assumes one writer. Two signer instances holding the same consensus key with separate or missing state files can each sign the same height, which is a double sign. + +Never run two signer instances against the same validator key, because this can cause double signing. + +## Keep the gRPC listener off consensus paths + +The optional gRPC SignerService performs no caller authentication or authorization: any client that reaches the listener can use every configured key. If the service is enabled, front it with TLS, restrict it with network policy, and give it only the keys it exists to serve. + +## Next steps + +- Set up the backend that holds the key. See [Configure a signing backend](/sdk/latest/kms/configure-backend). +- Look up transport and connection fields. See the [configuration reference](/sdk/latest/kms/configuration-reference). +- Rotate the consensus key without moving it out of custody. See [Rotate a consensus key held in Cosmos-KMS](/sdk/latest/kms/rotate-key-remote-signer). diff --git a/sdk/latest/kms/configuration-reference.mdx b/sdk/latest/kms/configuration-reference.mdx new file mode 100644 index 000000000..d4bc14c5a --- /dev/null +++ b/sdk/latest/kms/configuration-reference.mdx @@ -0,0 +1,127 @@ +--- +title: "Cosmos-KMS configuration reference" +description: "Every field of kms.yaml: chains, validators, keys with per-backend parameters, and the gRPC block." +--- + +The signer reads one file, `/kms.yaml`, at startup. Relative paths anywhere in the file resolve against the `--home` directory. Validation runs at `kms start`; a rejected field is named in the error. + +For task-shaped setup, see [Configure a signing backend](/sdk/latest/kms/configure-backend); this page is the complete field list. + +## chains + +Declares one chain to sign for. One entry per chain. + +| Field | Type | Required | Description | +| --- | --- | --- | --- | +| `id` | string | yes | The chain ID, matching the chain's genesis. | +| `state_file` | string | no | Path to the double-sign protection state file. Defaults to `/state/.json`. | + +## validators + +Declares one outbound connection to a validator node's privval listener. A chain can have multiple entries, for example a primary and a backup node. + +| Field | Type | Required | Description | +| --- | --- | --- | --- | +| `chain_id` | string | yes | Must match a declared `chains[].id`. | +| `addr` | string | yes | The listener address. `tcp://host:port` selects the SecretConnection transport; `noise://@host:port` selects the Noise transport with mutual peer pinning. | +| `identity_key` | string | yes | Path to the signer's identity key file, generated by `kms init`. Authenticates the SecretConnection, and doubles as the signer's Noise identity. | +| `reconnect` | bool | no | Reconnect automatically after a dropped connection. Defaults to `true`. | + +## keys + +Binds one signing key to one or more chains. Each chain must be backed by exactly one key. The `backend` field selects the custodian, and the remaining fields depend on it; fields belonging to other backends are ignored. + +Fields shared by every backend: + +| Field | Type | Required | Description | +| --- | --- | --- | --- | +| `chain_ids` | list of strings | yes | Chains this key signs for. Each must match a declared `chains[].id`. | +| `backend` | string | no | `file` (default), `pkcs11`, or `awskms`. | +| `algorithm` | string | yes | Key algorithm: `ed25519`, `secp256k1`, `secp256k1eth`, or `mldsa65`. Set it explicitly. | +| `key_id` | string | per backend | For `pkcs11`: hex `CKA_ID` of the key object. For `awskms`: KMS key ID, ARN, or `alias/`. | + +Consensus signing supports `ed25519`, `secp256k1eth`, and `mldsa65` on every backend, and `secp256k1` on the AWS KMS backend only. The algorithm name `mldsa65` has no underscores; the chain-side key type `ml_dsa_65` does. + +### backend: file + +A key read from disk into memory. Development and testing only; the key is held in plaintext. + +| Field | Type | Required | Description | +| --- | --- | --- | --- | +| `key_file` | string | yes | Path to the key. Accepts a CometBFT `priv_validator_key.json`, or a raw private key file (base64-encoded for `ed25519` and `mldsa65`, hex-encoded for `secp256k1eth`). | + +### backend: pkcs11 + +A key on a PKCS#11 token or HSM. Signing happens on-device; the signer uses an existing key and never generates or imports one. + +| Field | Type | Required | Description | +| --- | --- | --- | --- | +| `module` | string | yes | Path to the PKCS#11 module shared library. | +| `token_label` | string | exactly one of the two | `CKA_LABEL` of the token. | +| `slot` | integer | exactly one of the two | Slot number of the token. | +| `key_label` | string | at least one of `key_label`/`key_id` | `CKA_LABEL` of the key object. | +| `pin` | string | exactly one PIN source | User PIN, inline. Prefer the alternatives below. | +| `pin_env` | string | exactly one PIN source | Environment variable holding the PIN. | +| `pin_file` | string | exactly one PIN source | Path to a file holding the PIN. | + +### backend: awskms + +A key held in AWS KMS. Signing happens through the KMS Sign API; credentials resolve through the AWS default credential chain, and no secret material appears in the config. + +| Field | Type | Required | Description | +| --- | --- | --- | --- | +| `region` | string | no | AWS region of the key. Falls back to the AWS default chain. | +| `profile` | string | no | Shared-config profile name. Falls back to the AWS default chain. | +| `endpoint` | string | no | Custom KMS endpoint URL, for LocalStack-style testing. Leave unset for AWS. | + +## grpc + +Optional. When present, the signer also serves the SignerService gRPC API alongside privval. Its usage documentation ships with the interoperability release; the fields are listed here for completeness. + +| Field | Type | Required | Description | +| --- | --- | --- | --- | +| `listen` | string | yes | `host:port` the gRPC server binds to. | +| `tls_cert` | string | no | TLS server certificate file. Omitting both TLS fields serves plaintext, for local testing only. | +| `tls_key` | string | no | TLS server private key file. | +| `keys` | list | yes | The keys the service exposes; see below. | + +Each `grpc.keys` entry: + +| Field | Type | Required | Description | +| --- | --- | --- | --- | +| `id` | string | yes | Logical key identifier returned to clients. | +| `backend` | string | yes | `file`, `awskms`, or `pkcs11`. | +| `algorithm` | string | yes | For `file` and `pkcs11`: `ed25519` or `secp256k1eth`. For `awskms`: `ed25519`, `secp256k1`, or `secp256k1eth`. `mldsa65` has no gRPC signature scheme and is privval-only. | +| `key_id` | string | awskms | KMS key ID, ARN, or `alias/`. | +| `key_file` | string | file | Path to the key. Accepts a CometBFT `priv_validator_key.json`, or a raw private key file (base64-encoded for `ed25519`, hex-encoded for `secp256k1eth`). | + +A `grpc.keys` entry with `backend: pkcs11` also takes the PKCS#11 fields, with the same rules and the same validation as a privval key. See [backend: pkcs11](#backend-pkcs11) for `module`, `token_label`, `slot`, `key_label`, `key_id`, and the PIN sources. + +The gRPC server performs no caller authentication or authorization. Any client that can reach the listener can use every configured key. Restrict access with TLS and network controls. + +## Constraints checked at startup + +- Every `validators[].chain_id` and every entry in `keys[].chain_ids` must match a declared `chains[].id`. +- Each chain must be backed by exactly one key. +- PKCS#11 keys must select the token with exactly one of `token_label` or `slot`, select the key with `key_label` or `key_id`, and supply exactly one PIN source. +- A declared chain with no `validators` entry is not rejected. The signer starts, binds nothing, and signs nothing, with no warning, so check that every chain has a validator entry. +- Every chain's sign-state file must exist and be non-empty. A missing or empty file fails closed with `sign-state file is missing or empty; refusing to start at height 0`, so the signer cannot re-sign a height it has no record of. See [First start on a new chain](#first-start-on-a-new-chain). + +This list is not exhaustive. Config-level rejections are prefixed `config:` and name the field at fault; errors raised later, when a backend or chain signer is opened, use their own prefixes such as `app:` or `file:`. + +## First start on a new chain + +A key that has never signed on a chain has no sign-state file, so the checks above block its first start. Write the height-0 floor with: + +```shell +kms start --home --allow-fresh-state +``` + +`kms state init` writes a floor too, but it loads and validates `kms.yaml` first, so it only works once the chain is declared and the config is complete. It also defaults `--step` to 3, which refuses everything at that height and round, where `--allow-fresh-state` writes step 0. Pass `--step 0` to match. See [Migrate from TMKMS](/sdk/latest/kms/migrate-from-tmkms) for more details. + +Pass `--allow-fresh-state` only for a chain the key has genuinely never signed on. It will not overwrite an existing floor, but a service definition that carries it permanently removes the protection: a deleted or truncated state file resets the double-sign floor to zero instead of stopping the signer. + +## Next steps + +- Task-shaped backend setup. See [Configure a signing backend](/sdk/latest/kms/configure-backend). +- First-time setup end to end. See [Remote signing tutorial](/sdk/latest/kms/tutorial-file-backend). diff --git a/sdk/latest/kms/configure-backend.mdx b/sdk/latest/kms/configure-backend.mdx new file mode 100644 index 000000000..6e244cebb --- /dev/null +++ b/sdk/latest/kms/configure-backend.mdx @@ -0,0 +1,168 @@ +--- +title: "Configure a signing backend" +description: "Point Cosmos-KMS at the custodian holding the consensus key: AWS KMS, a PKCS#11 HSM, or a file." +--- + +A signing backend is the custodian that holds the validator's consensus key and signs with it. Cosmos-KMS supports three: AWS KMS, a PKCS#11 hardware module, and a file on disk. The `keys` block in `kms.yaml` selects one, and this guide configures each in turn. + +## Prerequisites + +- A running signer and node, which [Remote signing tutorial](/sdk/latest/kms/tutorial-file-backend) sets up. Between backends only the `keys` block changes in `kms.yaml`. The file backend reuses the tutorial's existing key, but the AWS KMS and PKCS#11 backends hold a new consensus key the validator must adopt first. See [Adopt the key on a validator](#adopt-the-key-on-a-validator). +- Per backend: the [AWS CLI](https://aws.amazon.com/cli/) and an AWS account for AWS KMS; your HSM's tooling plus [OpenSC](https://github.com/OpenSC/OpenSC)'s `pkcs11-tool` for PKCS#11, with [SoftHSM2](https://www.opendnssec.org/softhsm/) as a local test rig. +- The chain's binary; the examples use `simd`. To build it and run a node, see [Run a node](/sdk/latest/node/run-node). + +The `algorithm` field is required for all backends. + +## AWS KMS + +AWS KMS caps messages it signs in raw form at 4096 bytes. This binds `ed25519` and `secp256k1`, which send the raw consensus message, so be careful with features that enlarge it, such as vote extensions. The signer checks the size itself and fails before calling AWS. `mldsa65` and `secp256k1eth` are not bound by the cap. + +AWS Key Management Service (KMS) is a managed service that stores cryptographic keys and signs with them on request. With this backend, the consensus key lives in KMS and never leaves it. The signer calls the KMS Sign API to produce each signature. Credentials come from the standard AWS default chain: environment, shared config, SSO, or an IAM role. No secrets enter `kms.yaml`. + +This backend signs with any key type Cosmos-KMS supports: `ed25519`, `secp256k1`, `secp256k1eth`, and post-quantum `mldsa65`. Provision the AWS key with the key spec that matches the algorithm, such as `ECC_NIST_EDWARDS25519` for `ed25519` or `ML_DSA_65` for `mldsa65`. + +Create an ML-DSA-65 signing key and give it an alias (or skip these and point `key_id` at a key you already have): + +```shell +aws kms create-key --key-spec ML_DSA_65 --key-usage SIGN_VERIFY +``` + +```shell +aws kms create-alias --alias-name alias/validator --target-key-id +``` + +Then bind it in the `keys` block: + +```yaml +keys: + - chain_ids: [my-chain-1] + backend: awskms + algorithm: mldsa65 + key_id: alias/validator + region: us-east-1 +``` + +The `key_id` accepts a key ID, a full ARN, or an alias. `region` is optional and falls back to the AWS default chain. Two more optional fields, not shown above, are `profile` (a named shared-config profile) and `endpoint` (for LocalStack-style testing only). + +The signer needs only two IAM permissions on the key: `kms:GetPublicKey`, called once at startup, and `kms:Sign`, called per block. `kms:DescribeKey` is not required. Attach a least-privilege policy scoped to the key ARN, not the alias; AWS resolves the alias to the key server-side: + +```json +{ + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Action": ["kms:GetPublicKey", "kms:Sign"], + "Resource": "arn:aws:kms:::key/" + } + ] +} +``` + +A freshly created KMS key is a new consensus key. To move an existing validator onto it, rotate the validator to the new key; to stand up a new validator, register it with the new key's public key. See [Rotate a consensus key held in Cosmos-KMS](/sdk/latest/kms/rotate-key-remote-signer). Creating the key inside KMS keeps it from ever leaving the service. + +{/* AWS IS STILL THE 2026-07-20 RUN. The 2026-07-28 blind audit deliberately skipped AWS (no account, no simulating), so the run below remains the only end-to-end evidence for this backend and must not be restamped with a later date. Its claims were re-checked from source on 2026-07-28 at kms v0.1.0 and still hold. Manual verification (awskms backend): run-verified 2026-07-20 against real AWS KMS (us-east-1, account 326804803147; simd cosmos-sdk @ 46a177139a, kms @ 0007b0d). Both ed25519 (ECC_NIST_EDWARDS25519) and mldsa65 (ML_DSA_65) keys signed consensus end to end: create-key + alias as written, validator built around the KMS key's pubkey, blocks flowed, stop-kms→stall / restart→resume passed; CloudTrail confirmed kms:GetPublicKey at startup and kms:Sign per block. Full run: ~/Documents/tests/kms-aws-verification-findings.md. */} + +## `PKCS#11` + +The `PKCS#11` backend keeps the consensus key on a hardware security module (HSM) or token and signs on the device through `PKCS#11`, the standard interface for cryptographic hardware. The key never leaves the module. The signer uses an existing key only, so provision one with your HSM tooling first. The example commands below show how to provision an ed25519 key with SoftHSM2 for testing: + +```shell +softhsm2-util --init-token --free --label validator-token --pin 1234 --so-pin 4321 +``` + +```shell +pkcs11-tool --module /usr/lib/softhsm/libsofthsm2.so --login --pin 1234 --keypairgen --key-type EC:edwards25519 --label validator --id 01 +``` + +The `--module` path is platform-specific: Linux uses `/usr/lib/softhsm/libsofthsm2.so`, and macOS Homebrew uses `/opt/homebrew/lib/softhsm/libsofthsm2.so` (`/usr/local/lib/softhsm/...` on Intel). Use the same path for the `module` field below. + +Then bind it: + +```yaml +keys: + - chain_ids: [my-chain-1] + backend: pkcs11 + algorithm: ed25519 + module: /usr/lib/softhsm/libsofthsm2.so + token_label: validator-token + key_label: validator + pin_env: KMS_PIN +``` + +The signer enforces three field rules at startup: + +- Select the token with exactly one of `token_label` or `slot`. +- Select the key with `key_label`, `key_id` (the hex `CKA_ID`), or both. +- Supply the PIN through exactly one of `pin`, `pin_env`, or `pin_file`. + +Prefer `pin_env` or `pin_file`. An inline `pin` puts the PIN in the config file. + +Set the PIN in the environment the signer runs in so it matches `pin_env`: `export KMS_PIN=`. If you are using SoftHSM2 as the test rig, also `export SOFTHSM2_CONF=` so the module can find the token. + +As with AWS KMS, a key generated in the HSM is a new consensus key. Move an existing validator onto it by rotation, or register a new validator with its public key. See [Rotate a consensus key held in Cosmos-KMS](/sdk/latest/kms/rotate-key-remote-signer). + +{/* Re-verified 2026-07-28 by blind execution audit at shipping refs (kms v0.1.0 4bd83922ee, cometbft v0.40.0, simd v0.55.0 64fd208a11, SoftHSM2 2.7.0): token provisioning ran verbatim, the HSM-held ed25519 key signed every block, `gentx --pubkey` adoption worked, and stop-signer/stall plus restart/resume both passed. The macOS module-path note on this page is correct and load-bearing. mldsa65 on PKCS#11 remains untestable locally: SoftHSM2 exposes no ML-DSA mechanism. Full run: ~/Documents/tests/kms-docs-audit-findings.md. Original run, retained for provenance: run-verified 2026-07-20 on macOS arm64 (simd cosmos-sdk @ 46a177139a, kms @ 0007b0d, SoftHSM2 2.7.0, OpenSC 0.27.1). Token inited and ed25519 key generated with the commands as written (module path /opt/homebrew/lib/softhsm/libsofthsm2.so on macOS); the validator's consensus pubkey was set to the HSM key via `simd genesis gentx --pubkey`, the kms pkcs11 backend signed every block, blocks flowed, and stop-kms→stall / restart→resume passed. Full run: ~/Documents/tests/kms-backend-verification-findings.md. */} + +## File + +The file backend reads the consensus key from a file on the signer's disk into memory. It is the development and testing backend. The key sits in plaintext, so it is not production custody. The [Remote signing tutorial](/sdk/latest/kms/tutorial-file-backend) covers it end to end: + +```yaml +keys: + - chain_ids: [my-chain-1] + backend: file + algorithm: ed25519 + key_file: priv_validator_key.json +``` + +The `key_file` accepts a CometBFT `priv_validator_key.json` or a raw base64-encoded private key. The file backend also signs post-quantum consensus keys. Generate the key with `simd init --consensus-key-algo ml_dsa_65` and bind it: + +```yaml +keys: + - chain_ids: [my-chain-1] + backend: file + algorithm: mldsa65 + key_file: priv_validator_key.json +``` + +{/* Manual verification (mldsa65 file backend): re-verified 2026-07-28 at shipping refs (kms v0.1.0 4bd83922ee, simd v0.55.0 64fd208a11) by blind execution audit: chain inited with --consensus-key-algo ml_dsa_65 produced cometbft/PubKeyMlDsa65, the signer with algorithm: mldsa65 signed to height 3-4, and key_file accepted a raw base64 private key. Original run 2026-07-17 on macOS (simd from cosmos-sdk pr-26604 @ 46a177139a, kms @ 0007b0d) also covered the stop/resume proof. Full runs: ~/Documents/tests/kms-docs-audit-findings.md and mldsa-keygen-test-findings.md. */} +{/* Verification status (mldsa65 per backend): file — run-verified above (2026-07-17). awskms — run-verified 2026-07-20 against real AWS KMS (ML_DSA_65 key signed consensus end to end; see the AWS manual-verification note). pkcs11 — source/owner-confirmed but NOT run-verified: SoftHSM2 has no ML-DSA mechanism, so it needs an ML-DSA-capable HSM. */} + +## Adopt the key on a validator + +The AWS KMS and PKCS#11 backends generate the key inside the custodian, so it is a new consensus key, not the one your validator already runs. Do not just repoint an existing validator's signer at a fresh key. The node finds its consensus key absent from the validator set and demotes itself, so it never proposes and there is no signature error to look for. The symptom is `This node is not a validator` in the node log and a chain that does not advance. Only the file backend, pointed at the validator's existing `priv_validator_key.json`, skips this step. + +Adopt the key one of two ways: + +- Existing validator: rotate its consensus key to the new one, which derives the new public key from a shadow node and swaps it in with no downtime. See [Rotate a consensus key held in Cosmos-KMS](/sdk/latest/kms/rotate-key-remote-signer). +- New validator: register it with the new key's consensus public key using `gentx --pubkey` (or the `pubkey` field of `create-validator`'s validator.json). Read the public key from the custodian itself, since `gentx` runs before the chain exists and there is no node to query. For PKCS#11, read the object and strip the DER wrapper, leaving the raw 32 bytes to base64: + +```shell +pkcs11-tool --module --login --pin --read-object --type pubkey --label validator \ + | xxd -p | tr -d '\n' | sed 's/^302a300506032b6570032100//' | xxd -r -p | base64 +``` + +For AWS KMS, `aws kms get-public-key` returns a DER `SubjectPublicKeyInfo`; strip the same wrapper before encoding. + +## Verify any backend + +Verification is the same regardless of custodian. Start the signer, start the node, and confirm blocks flow, exactly as in [the tutorial](/sdk/latest/kms/tutorial-file-backend). + +If this key has never signed on the chain, the signer's first start needs `--allow-fresh-state ` to write the height-0 double-sign floor, otherwise it exits with `sign-state file ... is missing or empty; refusing to start at height 0`. Pass it only on that first start. See [Start the signer](/sdk/latest/kms/tutorial-file-backend) in the tutorial for the full explanation, and the [configuration reference](/sdk/latest/kms/configuration-reference) for `kms state init`. + +```shell +curl -s localhost:26657/status | jq '.result.sync_info.latest_block_height' +``` + +## What can go wrong + +- The signer rejects the config at startup: both `token_label` and `slot` set, or more than one PIN source. The error names the offending field. A missing `algorithm` on the file backend instead fails with the less specific `file: unknown key type`. +- The signer starts but cannot reach the key: wrong `module` path, wrong `key_id` or alias, or AWS credentials with no permission (denied `kms:GetPublicKey` fails at `kms start` with `awskms: get public key for "": `). These also fail at startup. +- The node exits with a pubkey timeout: the signer is not running or not reachable. Start the signer first. It dials and retries. + +## Next steps + +- Look up any config field, its type, and its constraints. See the [configuration reference](/sdk/latest/kms/configuration-reference). +- Run the whole flow once with the file backend. See [Remote signing tutorial](/sdk/latest/kms/tutorial-file-backend). +- Move an existing validator onto a key in a new backend by rotation. See [Rotate a consensus key held in Cosmos-KMS](/sdk/latest/kms/rotate-key-remote-signer). diff --git a/sdk/latest/kms/migrate-from-tmkms.mdx b/sdk/latest/kms/migrate-from-tmkms.mdx new file mode 100644 index 000000000..aaded00df --- /dev/null +++ b/sdk/latest/kms/migrate-from-tmkms.mdx @@ -0,0 +1,143 @@ +--- +title: "Migrate from TMKMS" +description: "Move a validator from TMKMS to Cosmos-KMS: translate the config, move or rotate the key, and cut over without double signing." +--- + +This guide moves a validator's signing from TMKMS to Cosmos-KMS, the recommended remote signer going forward. Cosmos-KMS includes several upgrades over TMKMS: it adds AWS KMS and PKCS#11 backends and post-quantum ML-DSA signing. The migration translates the config, gets the key into a Cosmos-KMS backend, and cuts over with exactly one signer alive at every moment. For what Cosmos-KMS is and how it relates to TMKMS, see [Cosmos-KMS and remote signing](/sdk/latest/kms/remote-signing). + +The validator node itself needs no changes: both signers speak the same privval protocol to the same `priv_validator_laddr` listener. TMKMS connects over CometBFT's SecretConnection, and the Cosmos-KMS `tcp://` transport is the same, so the listener works unchanged. + +## Prerequisites + +- A validator currently signing through TMKMS, with access to its `tmkms.toml` and state file. +- [jq](https://jqlang.org/), used to translate the state file in step 4. +- Cosmos-KMS installed and initialized with `kms init`. [Remote signing tutorial](/sdk/latest/kms/tutorial-file-backend) covers installation, which needs [Go](https://go.dev/doc/install) 1.26 or later, [make](https://www.gnu.org/software/make/), and [git](https://git-scm.com/). + +## 1. Translate the config + +Create `kms.yaml` and translate each block from your `tmkms.toml`: + +| tmkms.toml | kms.yaml | Notes | +| --- | --- | --- | +| `[[chain]]` `id` | `chains[].id` | Same value. | +| `[[chain]]` `state_file` | `chains[].state_file` | See [step 4](#4-translate-the-double-sign-state) before reusing a path. | +| `[[chain]]` `key_format` | none | Not needed; Cosmos-KMS has no per-chain serialization config. | +| `[[validator]]` `addr` | `validators[].addr` | Drop the `@` prefix; `tcp://host:port` uses CometBFT's SecretConnection, matching TMKMS. For a node running libp2p, use `noise://@:` instead, where the peer ID is the validator's, from `cometbft show-node-id --libp2p`. A `noise://` host must be an IP literal, bracketed for IPv6; hostnames are rejected. Noise is mutual, so the validator must also carry the signer's peer ID, from `kms peer-id`, in its allowlist, or it rejects the connection. | +| `[[validator]]` `chain_id` | `validators[].chain_id` | Same value. | +| `[[validator]]` `secret_key` | `validators[].identity_key` | Different format; use the `identity.json` from `kms init` rather than converting. | +| `[[validator]]` `protocol_version` | none | Not needed. | +| `[[providers.softsign]]` | `keys[]` with `backend: file` | See [step 2](#from-softsign). | +| `[[providers.yubihsm]]` | `keys[]` with `backend: pkcs11` | PKCS#11 HSM; see [step 2](#from-yubihsm-or-another-hsm). | +| `[[providers.ledgertm]]` | none | Not yet supported in Cosmos-KMS; see [step 2](#from-yubihsm-or-another-hsm). | + +A standard cosmos-sdk node has no peer ID on its `priv_validator_laddr` listener, so there is usually no `@` prefix to carry over, and both TMKMS and Cosmos-KMS use a bare `tcp://host:port`. + +Below is a complete `kms.yaml` example for a single validator on the file backend: + +```yaml +chains: + - id: my-chain-1 + state_file: state/my-chain-1.json +validators: + - chain_id: my-chain-1 + addr: tcp://127.0.0.1:26659 + identity_key: identity.json +keys: + - chain_ids: [my-chain-1] + backend: file + algorithm: ed25519 + key_file: priv_validator_key.json +``` + +The `keys` block shown is the file backend. Its fields differ per backend, which step 2 covers. It also needs `keys[].algorithm`, which has no `tmkms.toml` equivalent, so set it explicitly (`ed25519` for a softsign key), as the example shows. The [configuration reference](/sdk/latest/kms/configuration-reference) lists every field. + +## 2. Move the consensus key into a backend + +Get the consensus key into the backend named in your `keys` block. The path depends on where TMKMS holds it today. + +### From softsign + +The softsign backend keeps the key in a file, but the format differs from what the Cosmos-KMS file backend reads. A TMKMS softsign key is the base64-encoded 32-byte ed25519 seed, while the file backend expects a CometBFT `priv_validator_key.json` or the base64-encoded 64-byte ed25519 key. Pointing `key_file` at a softsign key directly fails with `expected 64-byte ed25519 key, got 32`. + +If you still have the validator's original `priv_validator_key.json` (TMKMS softsign was imported from it), point `key_file` at that file directly. No conversion is needed. + +From only the softsign key (the file named by the `path` in your `[[providers.softsign]]` block), expand the 32-byte seed into the 64-byte `seed||pubkey` form the file backend accepts. Replace `tmkms_softsign.key` in the command with that file: + +```shell +python3 - tmkms_softsign.key converted.key <<'PY' +import base64, sys +from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey +from cryptography.hazmat.primitives.serialization import Encoding, PublicFormat +seed = base64.b64decode(open(sys.argv[1]).read().strip()) +assert len(seed) == 32, f"expected 32-byte seed, got {len(seed)}" +sk = Ed25519PrivateKey.from_private_bytes(seed) +pub = sk.public_key().public_bytes(Encoding.Raw, PublicFormat.Raw) +open(sys.argv[2], "w").write(base64.b64encode(seed + pub).decode()) +PY +``` + +Point `key_file` at `converted.key`. The command needs Python 3 with the `cryptography` package (`pip install cryptography`). + +`converted.key` and the exported softsign key are unencrypted private keys. Shred them after import (`shred -u` or `rm -P`) and keep them out of shell history and backups. + +{/* REAL TMKMS IS STILL THE 2026-07-20 RUN. The 2026-07-28 blind audit completed a full cutover using this page alone and confirmed the config map, the seed conversion, the state jq and all three error strings at shipping refs (kms v0.1.0 4bd83922ee, simd v0.55.0 64fd208a11), but tmkms was not installed on that host: its softsign key and state file were synthesized in the formats this page documents. So the run below remains the only evidence involving real tmkms and must not be restamped. The 2026-07-28 audit also found that the raw jq at step 4 silently lowers an existing double-sign floor, now covered by a Warning on the page. Full run: ~/Documents/tests/kms-docs-audit-findings.md. Original: run-verified 2026-07-20 on macOS. tmkms 0.15.0 (softsign-only), simd cosmos-sdk 46a17713 (v0.54.0), kms 0007b0d1, python cryptography 46.0.5. Verified: the config map, both key paths (original priv_validator_key.json, and the seed→64-byte conversion above which was byte-identical to the original), the state jq (step+1 remap), all three error strings, and cutover with no double sign (kms first signed at tmkms high-water +1). NOT run: the PKCS#11/HSM and AWS KMS migration paths. Full run: ~/Documents/tests/tmkms-migration-findings.md. */} + +### From YubiHSM or another HSM + +Most HSM-held keys do not move. A YubiHSM, a Fortanix device, or any HSM that exposes the key over PKCS#11 works with the Cosmos-KMS `pkcs11` backend directly: wire up a `keys` block against the same module and keep signing with the same key. Keys generated inside an HSM are typically non-exportable, which is the point of an HSM, so this no-move path is the normal case. For the PKCS#11 config, see [Configure a signing backend](/sdk/latest/kms/configure-backend). + +When you are changing custodian rather than keeping the key in place, migrate by rotation: generate a new key in the target custodian and rotate the validator to it on chain, which retires the TMKMS-held key entirely. See [Rotate a consensus key, Staking](/sdk/latest/keys/rotate-validator-key). + +Ledger-held consensus keys are not currently supported in Cosmos-KMS. If you need it, open a [feature request](https://github.com/cosmos/kms/issues) so demand for it can be gauged. + +## 3. Stop TMKMS + +Stop TMKMS and confirm the process is gone. The validator misses blocks until Cosmos-KMS takes over. That gap is expected: missed blocks are recoverable, a double sign from two live signers is not. Keep it short to avoid downtime jailing. + +Never run TMKMS and Cosmos-KMS at the same time against the same validator key. Each keeps its own last-signed state, so together they can sign the same height, which is a double sign. Stop one fully before starting the other, in both directions, including any rollback. + +## 4. Translate the double-sign state + +TMKMS and Cosmos-KMS both track the last signed height, round, and step per chain, and the protection only works if the new signer starts at or above the old signer's high-water mark. The two store this state differently, so it needs translating. + +Point the following command at the TMKMS state file named by `state_file` in your `[[chain]]` block (`tmkms_state.json` is a placeholder). It writes the translated state into the Cosmos-KMS location: + +```shell +mkdir -p /state +jq '{height, round: (.round|tonumber), step: (.step + 1)}' tmkms_state.json > /state/.json +``` + +The `jq` expression makes two conversions: `round` from string to number (the source of the `int32` error otherwise), and `step + 1` to remap TMKMS's 0/1/2 signing steps to CometBFT's 1/2/3. + +The redirection above overwrites whatever is at that path, so a re-run or a stale `tmkms_state.json` silently lowers the double-sign floor. Check the file does not already exist first. + +## 5. Start Cosmos-KMS + +Start Cosmos-KMS with the completed `kms.yaml`. It dials the validator and resumes signing. + +```shell +kms start --home +``` + +## 6. Confirm signing resumed + +Confirm the chain is producing again and the signer's state file is advancing. Run the status check twice; a climbing height means signing resumed: + +```shell +curl -s localhost:26657/status | jq '.result.sync_info.latest_block_height' +``` + +The signer's `/state/.json` should also advance past the last height TMKMS signed. + +## What can go wrong + +- Both signers briefly alive: the double-sign risk above. Cut over with the old process confirmed dead, not just signaled. +- The validator stays dark after cutover: the new signer is not reaching the listener. Check the `validators[].addr` translation and the firewall between the hosts. +- `dial tcp: lookup node-id@host: no such host`: the `@` prefix was left in `validators[].addr`. Drop it and keep only host and port. +- `file: parse key file "": expected 64-byte ed25519 key, got 32`: a TMKMS softsign key was pointed at directly. See the [key section](#from-softsign). +- `chain "": reload sign-state: json: cannot unmarshal string into Go value of type int32`: the TMKMS state file was reused as is. Translate it first; see [step 4](#4-translate-the-double-sign-state). + +## Next steps + +- Harden the new setup. See [Remote signing best practices](/sdk/latest/kms/best-practices). +- Full field reference for the translated config. See the [configuration reference](/sdk/latest/kms/configuration-reference). diff --git a/sdk/latest/kms/remote-signing.mdx b/sdk/latest/kms/remote-signing.mdx new file mode 100644 index 000000000..f10b49563 --- /dev/null +++ b/sdk/latest/kms/remote-signing.mdx @@ -0,0 +1,58 @@ +--- +title: "Overview" +description: "What privval is, what Cosmos-KMS is and is not, and why validator keys belong in a KMS or HSM rather than local files." +--- + +This page covers remote signing for validators and Cosmos-KMS, the Cosmos stack's remote signer. The guides in this section set both up. + +## What is remote signing? + +Remote signing splits a validator into two processes: a node that participates in consensus, and a signer that holds the consensus key and produces signatures on the node's behalf. The node handles blocks, gossip, and peers; the signer holds the one secret that matters. + +Separating the two keeps signing secure and makes the node replaceable. The key moves off the node's exposed filesystem into a hardware security module (HSM) or a cloud key service, where the people and automation that maintain the node never touch it. And because the node holds no secrets, it can be rebuilt, upgraded, or replaced at will. Once it is replaced, the signer reconnects and signing continues. + +CometBFT supports this through privval, the seam between the node and whatever holds the key. With `priv_validator_laddr` set in `config.toml`, the node signs nothing locally: it listens for a signer connection and sends each vote and proposal out for signature. The wire protocol carries four requests: sign this vote, sign this proposal, return the public key, and a keepalive ping. The node does not necessarily care what is on the other end. + +```mermaid +sequenceDiagram + participant N as Validator node + participant S as Cosmos-KMS signer + participant B as Backend (file / PKCS11 / AWS KMS) + N->>S: vote or proposal to sign (privval) + S->>B: sign with consensus key + B-->>S: signature + S-->>N: signature +``` + +The node forwards each vote or proposal to the signer over the privval connection. The signer has its backend produce the signature and returns only that, so the consensus key never leaves the backend. + +## Cosmos-KMS + +The [`cosmos-kms`](https://github.com/cosmos/kms) repo is a remote signer for CometBFT, written in Go. It implements the signer side of privval and answers a node's signing requests from a backend that holds the key. One signer process can sign for multiple chains, with each chain backed by exactly one key, and can hold connections to multiple nodes per chain. It is built for operators who must keep validator keys in real custody, an HSM or a cloud key service, rather than in files on chain infrastructure. Three backends are supported: + +- File: a key file on the signer's disk, for development and testing, not production custody. +- PKCS#11: a hardware security module, signing on-device through the standard HSM interface. +- AWS KMS: a key that never leaves AWS KMS, using standard AWS credentials and IAM. + +Cosmos-KMS is designed to work with existing validator infrastructure, and validators should prefer it over previous remote-signing solutions. If you run [TMKMS](https://github.com/iqlusioninc/tmkms) today, see [Migrate from TMKMS](/sdk/latest/kms/migrate-from-tmkms). + +Cosmos-KMS speaks the CometBFT privval protocol, so it can sign for any node that implements that protocol. + +## How it works + +A running signer comes down to five moving parts: + +- Configuration: one file, `kms.yaml`, in three blocks: the chains it signs for, the validators it dials, and the keys binding each chain to exactly one backend. The `kms init` command scaffolds it; `kms start` serves until stopped. +- Signing: requests travel the privval connection, the backend signs in place, on disk, on the HSM, or inside AWS KMS, and only the signature returns. The private key never crosses the wire. +- Connection: the signer dials out, so the signing host needs no inbound ports. The address scheme selects the transport: `tcp://` uses CometBFT's SecretConnection, and `noise://` adds mutual peer pinning, where each side refuses any connection from an unexpected peer. +- Key types: over privval, the signer signs `ed25519`, `secp256k1eth`, and post-quantum `mldsa65`, plus `secp256k1` on the AWS KMS backend; the gRPC signer service signs `ed25519`, `secp256k1eth`, and `secp256k1`, but not `mldsa65`, which has no gRPC signature scheme and is privval-only. For per-backend support, see [Configure a signing backend](/sdk/latest/kms/configure-backend). +- Double-sign protection: a per-chain last-signed state file refuses anything at or below a height, round, and step already signed. The protection lives with the key, so even a misbehaving or duplicated validator node cannot force a double sign. Because this protection is per state file, it is recommended never to run two signers for the same key. For a backup, connect a single signer to more than one validator node. + +## Next steps + +- Run a remote signer against a local chain. See [Remote signing tutorial](/sdk/latest/kms/tutorial-file-backend). +- Move the key into real custody, AWS KMS or an HSM. See [Configure a signing backend](/sdk/latest/kms/configure-backend). +- Harden the signer's placement and transport. See [Remote signing best practices](/sdk/latest/kms/best-practices). +- Look up any `kms.yaml` field. See the [configuration reference](/sdk/latest/kms/configuration-reference). +- Understand which key the signer holds and how it rotates. See [Key rotation](/sdk/latest/keys/key-rotation). +- Understand post-quantum consensus keys and their costs. See [Post-quantum keys](/sdk/latest/keys/post-quantum-keys). diff --git a/sdk/latest/kms/rotate-key-remote-signer.mdx b/sdk/latest/kms/rotate-key-remote-signer.mdx new file mode 100644 index 000000000..af3f6ac95 --- /dev/null +++ b/sdk/latest/kms/rotate-key-remote-signer.mdx @@ -0,0 +1,179 @@ +--- +title: "Rotate a consensus key held in Cosmos-KMS" +description: "Rotate a validator's consensus key when the current key lives in a remote signer: new key in the backend, a second node and signer, rotate, cut over." +--- + +This tutorial rotates a staked validator's consensus key when Cosmos-KMS holds the current key rather than a local file. It continues the [remote signing tutorial](/sdk/latest/kms/tutorial-file-backend) and reuses that setup: the single-node chain `kms-demo-1`, with node home `~/.kms-demo-node` (RPC on port 26657, privval listener on port 26659) and signer home `~/.kms-demo`, whose signer dials that listener. + +The rotation is the standard zero-downtime rotation with three differences. The new key is generated in a signing backend. The second node gets its own signer process. The new public key comes from the second node's RPC instead of a key file. For the standard procedure and the rotation rules, see [Rotate a consensus key, Staking](/sdk/latest/keys/rotate-validator-key). + +Key rotation can introduce security implications for your chain. Read the [Key rotation](/sdk/latest/keys/key-rotation) overview in its entirety before proceeding. + +Commands use `simd` as the chain binary and `val` as the validator key name. Substitute your own for a real chain. The live pair and the new pair run on one host, so the second node and its new signer take spare ports: the second node listens for its signer on port 26669 and serves RPC on port 26667. + +{/* +CURRENT VERIFICATION: blind execution audit 2026-07-28 against shipping refs, kms v0.1.0 (4bd83922ee), cometbft v0.40.0 (0880b4d378), simd v0.55.0 (64fd208a11), macOS 26.5.2 arm64. Verdict PASS, the only page in the KMS set to pass outright: every command ran verbatim on the first attempt and every checkpoint was observable. Rotation tx at height 39; old key served heights 39 and 40, new key from 41; CometBFT stopped requesting signatures from the old pair on its own; the chain kept producing on the new pair alone after both old processes were stopped. Full run: ~/Documents/tests/kms-docs-audit-findings.md. +Earlier run, retained for provenance: verified 2026-07-15 on Darwin 25.5.0 (arm64), Go 1.26.5, with simd from cosmos-sdk @ b9a11304cf and kms @ 538e5c5, which predates the fail-closed sign-state change in kms v0.1.0. +Base setup: the remote signing tutorial's single-validator localnet signing through kms (file backend), 1s timeout_commit, 0stake min gas. +Every command below was run verbatim. Checkpoints observed: second node caught up with zero signatures from the new signer; +rotation tx executed (code 0, 1000000stake fee burned); set swapped to the new key with no overlap; old signer stopped on its own; chain kept producing on the new pair alone. + +*/} + +## Prerequisites + +- A validator already signing through Cosmos-KMS, from the [remote signing tutorial](/sdk/latest/kms/tutorial-file-backend), still running. +- All prerequisites of the standard rotation: Cosmos SDK 0.55 and CometBFT 0.40 or later, the target key type in consensus params, no rotation in the current unbonding period, and funds for the burned rotation fee plus gas. See [Rotate a consensus key, Staking](/sdk/latest/keys/rotate-validator-key). +- [jq](https://jqlang.org/) and [curl](https://curl.se/), used to derive the new public key. +- Spare ports on the host for the second node and its signer. +- The chain's binary; the examples use `simd`. To build it and run a node, see [Run a node](/sdk/latest/node/run-node). + +## 1. Generate the new key in a backend + +For an HSM or AWS KMS, generate the key with the backend's own tooling. The commands are in [Configure a signing backend](/sdk/latest/kms/configure-backend). This guide uses the file backend, so a scratch `simd init` generates a fresh key file. + +```shell +simd init scratch --chain-id kms-demo-1 --home ~/.kms-demo-scratch +``` + +For a post-quantum `mldsa65` target on the file backend, add `--consensus-key-algo ml_dsa_65` to the `simd init` command above and set `algorithm: mldsa65` in the signer's `keys` block in step 2. The PKCS#11 and AWS KMS backends also sign `mldsa65`; see [Configure a signing backend](/sdk/latest/kms/configure-backend) for generating the key on those. + +The new key is `~/.kms-demo-scratch/config/priv_validator_key.json`. The rest of the scratch home is disposable. + +The new key must be freshly generated. Reusing key material any signer has signed with risks a double sign, which tombstones the validator. + +## 2. Configure a second signer with the new key + +One Cosmos-KMS process cannot hold two keys for the same chain. The new key must run in its own process, with its own home and double-sign state: + +```shell +kms init --home ~/.kms-demo2 +``` + +Copy the new key to the second signer's home: + +```shell +cp ~/.kms-demo-scratch/config/priv_validator_key.json ~/.kms-demo2/priv_validator_key.json +``` + +Replace the contents of `~/.kms-demo2/kms.yaml` with the following. The `addr` is the port the second node listens on for its signer. For an HSM or AWS KMS key, the `keys` block instead binds the backend entry from step 1: + +```yaml +chains: + - id: kms-demo-1 + +validators: + - chain_id: kms-demo-1 + addr: tcp://127.0.0.1:26669 + identity_key: identity.json + +keys: + - chain_ids: [kms-demo-1] + backend: file + algorithm: ed25519 + key_file: priv_validator_key.json +``` + +For a post-quantum `mldsa65` key, set `algorithm: mldsa65` in the signer's `keys` block above. + +Do not start the signer yet. It starts in step 3, right before the second node, so its connection retries are still fast when the node comes up. + +## 3. Bring up the second node and its signer + +Initialize a fresh node home and give it the chain's genesis: + +On a real chain with history, a fresh node takes days to sync from genesis. Use state sync or a snapshot to reach the chain head quickly. See [State sync](/sdk/latest/node/run-node#state-sync) for more info. + +```shell +simd init shadow --chain-id kms-demo-1 --home ~/.kms-demo-node2 +``` + +```shell +cp ~/.kms-demo-node/config/genesis.json ~/.kms-demo-node2/config/genesis.json +``` + +With the prep done, start the new signer and the second node in quick succession. In one terminal, start the signer. It logs `dial failed` and retries until the node exists: + +```shell +kms start --home ~/.kms-demo2 --allow-fresh-state kms-demo-1 +``` + +The new key has never signed on this chain, and this signer home is new, so it holds no sign-state file. `--allow-fresh-state` writes the height-0 double-sign floor so the signer will start. Without it the signer exits with `sign-state file ... is missing or empty; refusing to start at height 0`. + +Use `--allow-fresh-state` only here, for the new key's first start. Do not add it to the live signer or to any later start of this one. Once a state file exists the flag has no effect, so it fails quietly, but it means a lost or truncated state file resets the double-sign floor to zero rather than stopping the signer. After the cutover in step 5, run the new signer with the bare command. + +Right away, in a second terminal, start the second node. The flags point it at the new signer on port 26669. Move its listeners off the live node's ports, and peer it with the live node so it syncs: + +```shell +simd start --home ~/.kms-demo-node2 \ + --priv_validator_laddr tcp://127.0.0.1:26669 \ + --p2p.laddr tcp://0.0.0.0:26666 \ + --rpc.laddr tcp://127.0.0.1:26667 \ + --grpc.address localhost:9092 \ + --proxy_app tcp://127.0.0.1:26668 \ + --rpc.pprof_laddr localhost:6061 \ + --p2p.persistent_peers "$(simd comet show-node-id --home ~/.kms-demo-node)@127.0.0.1:26656" +``` + +If the second node exits with `can't get pubkey: ... endpoint connection timed out`, the signer has backed off to slow retries. Restart the signer, then start the node again. + +Confirm the second node has caught up. The value is `false` once it is synced: + +```shell +curl -s localhost:26667/status | jq '.result.sync_info.catching_up' +``` + +Until the rotation applies, the second node follows the chain without signing. Its signer logs `served pubkey request` but no signatures. That is correct. + +## 4. Derive the new public key and rotate + +The second node fetched its key from the new signer at startup and reports it at `/status`. Read it and reformat it into the proto-JSON the rotation command accepts: + +```shell +PK=$(curl -s localhost:26667/status | jq -c '{"@type":"/cosmos.crypto.ed25519.PubKey", key: .result.validator_info.pub_key.value}') +``` + +For an ML-DSA key, the proto type differs: use `"@type":"/cosmos.crypto.mldsa65.PubKey"` in the jq expression instead. + +Submit the rotation from the operator account and capture the transaction hash: + +```shell +TXHASH=$(simd tx staking rotate-cons-pub-key "$PK" --from val --keyring-backend test --home ~/.kms-demo-node --chain-id kms-demo-1 --node tcp://localhost:26657 --gas auto --gas-adjustment 1.5 --fees 2000stake --yes --output json | jq -r .txhash) +``` + +After the transaction lands in a block, confirm the code is `0`: + +```shell +simd query tx "$TXHASH" --node tcp://localhost:26657 --output json | jq '{height: .height, code: .code}' +``` + +The burned rotation fee is charged separately from the gas fee above. + +## 5. Verify and retire the old pair + +The validator set swaps to the new key atomically two heights after execution. Confirm the set carries only the new key. The value matches `$PK`, and the old key is gone: + +```shell +curl -s localhost:26657/validators | jq -r '.result.validators[].pub_key.value' +``` + +At the swap, CometBFT stops requesting signatures from the old pair on its own. Stop the live node (`~/.kms-demo-node`) and the live signer (`~/.kms-demo`) with Ctrl-C in their terminals. + +Confirm blocks keep flowing through the new pair. Run this twice a few seconds apart and watch the height climb: + +```shell +curl -s localhost:26667/status | jq '.result.sync_info.latest_block_height' +``` + +Keep the old key in its backend until the unbonding period ends. The validator remains slashable for its past behavior until then. + +## What can go wrong + +- The new signer exits with `app: multiple signers bound to chain`: both keys are in one `kms.yaml`. Run the new key in its own process. +- The validator is jailed after the rotation and neither signer is signing: the rotation targeted the stray local key. Rotate to the key from the second node's `/status`, once the unbonding window allows it. +- The transaction is rejected: the standard failure modes apply, including the rotation limit and unsupported key types. See [Rotate a consensus key, Staking](/sdk/latest/keys/rotate-validator-key). + +## Next steps + +- Harden the new signer's placement and transport. See [Remote signing best practices](/sdk/latest/kms/best-practices). +- Rotate to a post-quantum key with the same procedure. See [Migrate a validator to ML-DSA](/sdk/latest/keys/migrate-validator-ml-dsa). diff --git a/sdk/latest/kms/tutorial-file-backend.mdx b/sdk/latest/kms/tutorial-file-backend.mdx new file mode 100644 index 000000000..0b9a79044 --- /dev/null +++ b/sdk/latest/kms/tutorial-file-backend.mdx @@ -0,0 +1,171 @@ +--- +title: "Remote signing tutorial" +description: "Tutorial: stand up a local chain whose validator signs through Cosmos-KMS, using the file backend." +--- + +This tutorial builds a working remote signer from scratch: a single-node local chain where a Cosmos-KMS process signs the votes instead of the node itself. It uses the file backend, which needs no HSM or cloud account and exists for exactly this kind of learning setup. At the end, you stop the signer and watch the chain stall, which proves where signing really happens. This tutorial uses one node, one signer, and one key. + +Commands use `simd` for the chain binary. The node home is `~/.kms-demo-node` and the signer home is `~/.kms-demo`. + +{/* +CURRENT VERIFICATION: blind execution audit 2026-07-28 against shipping refs, kms v0.1.0 (4bd83922ee), cometbft v0.40.0 (0880b4d378), simd v0.55.0 (64fd208a11), go1.26.0, macOS 26.5.2 arm64. Verdict PARTIAL: all three checkpoints pass (no blocks before the signer; height climbs once the node connects; production stalls when kms stops and resumes on restart), and the state file is confirmed written at signer start rather than at first signature. The one blocker found was the step-8 timing cliff, since fixed on this page. Full run: ~/Documents/tests/kms-docs-audit-findings.md. +Note that kms v0.1.0 fails closed on a missing sign-state file (PR #35, c952fac), which the pre-2026-07-28 runs below predate: they were verified against kms commits where a bare `kms start` still worked on a fresh home. Any future re-verification must use a ref at or after v0.1.0. +Earlier runs, retained for provenance: verified 2026-07-14 on Darwin 25.5.0 (arm64), Go 1.26.5, with simd from cosmos-sdk main @ b9a11304cf (reported 0.0.0-dev; 0.55 not tagged then) and kms @ 7932ceb. +Step 8 startup timing re-verified 2026-07-20 on a clean single-localnet machine (simd @ 46a17713, kms @ 0007b0d): node started within ~5s of the signer passes 5/5; longer gaps hit dead zones (~6-9s, ~13-21s) and time out with `can't get pubkey`. Re-verified again 2026-07-21 (kms-tutorial-reverify-findings.md): a plain node rerun is unreliable (~30%), worsened on a node's first boot by a one-time IAVL storage upgrade that delays its listener; restarting the signer (resets its backoff to fast dials) reliably recovers. Root cause: kms dial backoff (200ms→10s cap, internal/manager) vs the node's single ~3s pubkey fetch (cometbft node/setup.go:736). Note updated to the restart-the-signer recovery. Fix flagged to eng (lower kms defaultBackoffMax to ~1s). Full runs: ~/Documents/tests/kms-tutorial-step8-reverify-findings.md and kms-tutorial-reverify-findings.md. +*/} + +## Prerequisites + +- [Go](https://go.dev/doc/install) 1.26 or later, [make](https://www.gnu.org/software/make/), [git](https://git-scm.com/), [jq](https://jqlang.org/), and [curl](https://curl.se/). +- A chain binary at Cosmos SDK 0.55 or later. The tutorial uses `simd`, built with `make install` in the [cosmos-sdk repo](https://github.com/cosmos/cosmos-sdk). To build it and run a node, see [Run a node](/sdk/latest/node/run-node). + +## 1. Install Cosmos-KMS + +Clone and install the signer: + +```shell +git clone https://github.com/cosmos/kms +``` + +```shell +cd kms && make install +``` + +Confirm the binary works: + +```shell +kms version +``` + +## 2. Create a single-node chain + +Set up a fresh chain home with one validator. Do not start the node yet: + +```shell +# Initialize the node home with the chain ID +simd init signer-demo --chain-id kms-demo-1 --home ~/.kms-demo-node + +# Create the validator key in the test keyring +simd keys add val --keyring-backend test --home ~/.kms-demo-node + +# Fund the validator account in genesis +simd genesis add-genesis-account val 1000000000stake --keyring-backend test --home ~/.kms-demo-node + +# Register the validator with a genesis staking transaction +simd genesis gentx val 500000000stake --chain-id kms-demo-1 --keyring-backend test --home ~/.kms-demo-node + +# Collect the gentx into the genesis file +simd genesis collect-gentxs --home ~/.kms-demo-node +``` + +## 3. Initialize the signer + +Scaffold the signer's home. This writes a stub `kms.yaml` and generates `identity.json`, the key the signer uses to authenticate its connection: + +```shell +kms init --home ~/.kms-demo +``` + +The command prints `initialized kms in /Users/you/.kms-demo`. Pass the `--home` flag on every `kms` command. Without it, the signer uses the current directory. + +## 4. Give the signer the consensus key + +Copy the consensus key that `simd init` generated into the signer's home: + +```shell +cp ~/.kms-demo-node/config/priv_validator_key.json ~/.kms-demo/priv_validator_key.json +``` + +Once the node is configured for remote signing, it never reads its local key file again. In production, move the key instead of copying it. Note that the node regenerates a fresh, unused consensus key file if it finds none, so moving the key reduces what is on the node host rather than leaving it key-free. For this tutorial, the copy keeps things simple. + +## 5. Configure the signer + +Replace the contents of `~/.kms-demo/kms.yaml` with: + +```yaml +chains: + - id: kms-demo-1 + +validators: + - chain_id: kms-demo-1 + addr: tcp://127.0.0.1:26659 + identity_key: identity.json + +keys: + - chain_ids: [kms-demo-1] + backend: file + algorithm: ed25519 + key_file: priv_validator_key.json +``` + +The three blocks say: sign for the chain `kms-demo-1`, dial its node at port 26659, and read the copied key file as an `ed25519` key. The file backend has no default algorithm, so the `algorithm` line is required. Relative paths resolve against the signer's home. + +## 6. Point the node at the signer + +Open `~/.kms-demo-node/config/config.toml`, find the `priv_validator_laddr` line, and set it: + +```toml +priv_validator_laddr = "tcp://127.0.0.1:26659" +``` + +With this set, the node signs nothing locally. It listens on that port for a signer connection and forwards every vote and proposal to it. + +## 7. Start the signer + +The node needs its signer available the moment it starts, so bring the signer up first: + +```shell +kms start --home ~/.kms-demo --allow-fresh-state kms-demo-1 +``` + +This validator has never signed on `kms-demo-1`, so no sign-state file exists yet. `--allow-fresh-state` writes the height-0 double-sign floor on this first start. Without it the signer refuses to start rather than risk re-signing a height it cannot prove it has already passed. + +Pass `--allow-fresh-state` only on a first start, and only for a chain the key has never signed on. It will not overwrite an existing floor, but leaving it in a service definition means a deleted or truncated state file resets the floor to zero instead of stopping the signer. Later starts in this tutorial use the bare command. + +To seed the floor as a separate step instead, run `kms state init --chain kms-demo-1 --height 0 --home ~/.kms-demo` and then start the signer with no flag. + +The signer logs `kms started` and dials the node. The node is not running yet, so the signer logs `dial failed; backing off` and keeps retrying. That is expected. Leave it running. + +## 8. Start the node + +In a second terminal, start the node within five seconds of starting the signer: + +```shell +simd start --home ~/.kms-demo-node +``` + +If `simd start` exits with `can't get pubkey: ... endpoint connection timed out`, the signer has backed off to slow retries. Restart the signer, then start the node again within five seconds. + +The node opens its private-validator listener on port 26659. The signer's next dial connects, the node fetches its consensus public key from the signer, and block production begins. Confirm the height is climbing: + +```shell +curl -s localhost:26657/status | jq '.result.sync_info.latest_block_height' +``` + +Also confirm the signer's double-sign protection state file is in place, written when the signer started: + +```shell +ls ~/.kms-demo/state/kms-demo-1.json +``` + +## 9. Prove the signer is doing the signing + +Stop the signer with Ctrl-C and watch the node's logs. Block production stalls because the validator can no longer sign. Start the signer again, this time with no `--allow-fresh-state`, because the state file now exists and carries the highest height signed so far: + +```shell +kms start --home ~/.kms-demo +``` + +The signer reconnects and the chain resumes. The node never touches a private key. Every signature comes from the signer. + +## What you built + +A validator whose consensus key lives outside the node. The node handles consensus and networking. The signer holds the key and signs, and the double-sign state file travels with it. The file backend keeps this tutorial self-contained, but it holds the key in plaintext on disk and is not production custody. The production version of this setup swaps one config block to move the key into an HSM or AWS KMS. + +## Next steps + +- Swap the file backend for real custody, AWS KMS or an HSM. See [Configure a signing backend](/sdk/latest/kms/configure-backend). +- Harden the signer's placement and transport. See [Remote signing best practices](/sdk/latest/kms/best-practices). +- Understand the architecture you just ran. See [Cosmos-KMS and remote signing](/sdk/latest/kms/remote-signing). + +- Look up any config field. See the [configuration reference](/sdk/latest/kms/configuration-reference). diff --git a/sdk/latest/learn.mdx b/sdk/latest/learn.mdx index 1b01c70da..bbaa5f498 100644 --- a/sdk/latest/learn.mdx +++ b/sdk/latest/learn.mdx @@ -1,6 +1,6 @@ --- title: "Cosmos SDK Docs" -description: "Version: v0.54" +description: "Version: v0.55" --- The Cosmos SDK is the most widely adopted, battle-tested Layer 1 blockchain stack, trusted by 200+ chains live in production. This modular framework enables you to build secure, high-performance blockchains with comprehensive guides covering everything from core concepts to advanced implementation patterns. diff --git a/sdk/latest/learn/concepts/accounts.mdx b/sdk/latest/learn/concepts/accounts.mdx index f5be1e6d5..9db7c8324 100644 --- a/sdk/latest/learn/concepts/accounts.mdx +++ b/sdk/latest/learn/concepts/accounts.mdx @@ -10,7 +10,7 @@ Every account is controlled by a cryptographic keypair derived from a seed phras ## What is an account -An account is an on-chain identity used to authorize transactions. Each account stores an address, a public key, an account number, and a sequence number, as defined by [`BaseAccount`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/x/auth/types/auth.pb.go#L32) in the `x/auth` module: +An account is an on-chain identity used to authorize transactions. Each account stores an address, a public key, an account number, and a sequence number, as defined by [`BaseAccount`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/x/auth/types/auth.pb.go#L32) in the `x/auth` module: ```go type BaseAccount struct { diff --git a/sdk/latest/learn/concepts/baseapp.mdx b/sdk/latest/learn/concepts/baseapp.mdx index 95c8441c1..dc5c08091 100644 --- a/sdk/latest/learn/concepts/baseapp.mdx +++ b/sdk/latest/learn/concepts/baseapp.mdx @@ -26,7 +26,7 @@ CometBFT drives the block lifecycle by calling ABCI methods on `BaseApp`. `BaseA ## Key fields -[`BaseApp`](https://github.com/cosmos/cosmos-sdk/blob/main/baseapp/baseapp.go) is defined in `baseapp/baseapp.go`. It holds references to everything needed to run a chain: +[`BaseApp`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/baseapp/baseapp.go#L63-L168) is defined in `baseapp/baseapp.go`. It holds references to everything needed to run a chain: ```go type BaseApp struct { @@ -49,7 +49,7 @@ type BaseApp struct { } ``` -For a complete list of fields, see the [`BaseApp` struct definition](https://github.com/cosmos/cosmos-sdk/blob/main/baseapp/baseapp.go). +For a complete list of fields, see the [`BaseApp` struct definition](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/baseapp/baseapp.go#L63-L168). - `cms` (CommitMultiStore): the root state store. All module substores are mounted here, and all state reads and writes during block execution pass through it. - `storeLoader`: a function that opens and mounts the individual module stores at application startup. @@ -113,7 +113,7 @@ For the application wiring side, including `SetAnteHandler`, `HandlerOptions`, a If the `AnteHandler` fails, the transaction is rejected and its messages never execute. If the `AnteHandler` succeeds but a message later fails, the `AnteHandler`'s state writes, such as fee deduction and sequence increment for ordered transactions, are already flushed to `finalizeBlockState` and will be committed with the block. Fees are charged even for transactions whose messages fail. -`BaseApp.runTx()` also handles Go panics that occur during execution — for example, when a keeper encounters an invalid state. By default, panics are caught and logged as errors. Applications can register custom panic recovery logic via `BaseApp.AddRunTxRecoveryHandler`, which adds a `RecoveryHandler` to the chain. See [ADR-022](/sdk/latest/reference/architecture/adr-022-custom-panic-handling) and [`baseapp/recovery.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/baseapp/recovery.go) for details. +`BaseApp.runTx()` also handles Go panics that occur during execution — for example, when a keeper encounters an invalid state. By default, panics are caught and logged as errors. Applications can register custom panic recovery logic via `BaseApp.AddRunTxRecoveryHandler`, which adds a `RecoveryHandler` to the chain. See [ADR-022](/sdk/latest/reference/architecture/adr-022-custom-panic-handling) and [`baseapp/recovery.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/baseapp/recovery.go) for details. ## Message routing diff --git a/sdk/latest/learn/concepts/cli-grpc-rest.mdx b/sdk/latest/learn/concepts/cli-grpc-rest.mdx index 42ccaf2d9..ffd57c08d 100644 --- a/sdk/latest/learn/concepts/cli-grpc-rest.mdx +++ b/sdk/latest/learn/concepts/cli-grpc-rest.mdx @@ -238,7 +238,7 @@ api.enable = true api.swagger = true ``` -To generate Swagger documentation for your own custom modules, see the [`proto-swagger-gen` script](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/scripts/protoc-swagger-gen.sh) in the Cosmos SDK. +To generate Swagger documentation for your own custom modules, see the [`proto-swagger-gen` script](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/scripts/protoc-swagger-gen.sh) in the Cosmos SDK. ## CometBFT RPC diff --git a/sdk/latest/learn/concepts/context-gas-events.mdx b/sdk/latest/learn/concepts/context-gas-events.mdx index 89d0a5c05..4d67078d9 100644 --- a/sdk/latest/learn/concepts/context-gas-events.mdx +++ b/sdk/latest/learn/concepts/context-gas-events.mdx @@ -8,7 +8,7 @@ In the previous section, [Encoding and Protobuf](/sdk/latest/learn/concepts/enco Every message handler, keeper method, and block hook in the Cosmos SDK receives an `sdk.Context`. It is the execution environment for a single unit of work (a transaction, a query, or a block hook) and carries everything that code needs to read state, emit events, and consume gas. Rather than passing the store, gas meter, and block header as separate arguments to every function, `Context` bundles them into a single value. -The `Context` struct is defined in [`types/context.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/types/context.go): +The `Context` struct is defined in [`types/context.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/types/context.go): ```go type Context struct { @@ -25,7 +25,7 @@ Context is a value type. It is passed by value and mutated through `With*` metho ### Block metadata -Context exposes read-only access to the current block's metadata (see [`types/context.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/types/context.go)): +Context exposes read-only access to the current block's metadata (see [`types/context.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/types/context.go)): - `ctx.BlockHeight()` returns the current block number. - `ctx.BlockTime()` returns the block's timestamp. @@ -34,7 +34,7 @@ Context exposes read-only access to the current block's metadata (see [`types/co These values are populated by [`BaseApp`](/sdk/latest/learn/concepts/baseapp) from the block header provided by CometBFT before any block logic runs. Modules read them to implement time-dependent logic (for example, checking whether a vesting period has elapsed) or to tag events with the block height. -`ctx.IsCheckTx()` returns true when the context is being used for mempool validation rather than block execution. For finer-grained branching, `ctx.ExecMode()` returns the precise execution mode: `ExecModeCheck`, `ExecModeReCheck`, `ExecModeSimulate`, `ExecModePrepareProposal`, `ExecModeProcessProposal`, `ExecModeFinalize`, and others (see [`types/context.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/types/context.go#L21) for more details). Modules that need to behave differently during simulation or proposal handling use `ExecMode()` instead of `IsCheckTx()`. +`ctx.IsCheckTx()` returns true when the context is being used for mempool validation rather than block execution. For finer-grained branching, `ctx.ExecMode()` returns the precise execution mode: `ExecModeCheck`, `ExecModeReCheck`, `ExecModeSimulate`, `ExecModePrepareProposal`, `ExecModeProcessProposal`, `ExecModeFinalize`, and others (see [`types/context.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/types/context.go#L21) for more details). Modules that need to behave differently during simulation or proposal handling use `ExecMode()` instead of `IsCheckTx()`. ### Context and state access @@ -50,7 +50,7 @@ The keeper does not hold a direct reference to the live multistore; it opens its ### Atomic sub-execution with `CacheContext` -Modules that need to attempt a sub-operation and revert it on failure can call [`ctx.CacheContext()`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/types/context.go#L412), which returns a branched copy of the context and a `writeCache` function. All state changes in the sub-operation go into the branch. Calling `writeCache()` flushes them to the parent context; not calling it discards them atomically. +Modules that need to attempt a sub-operation and revert it on failure can call [`ctx.CacheContext()`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/types/context.go#L412), which returns a branched copy of the context and a `writeCache` function. All state changes in the sub-operation go into the branch. Calling `writeCache()` flushes them to the parent context; not calling it discards them atomically. ```go cacheCtx, writeCache := ctx.CacheContext() @@ -72,7 +72,7 @@ The gas system exists to prevent abuse. Without a gas limit, a single transactio Every transaction specifies a gas limit in its `auth_info.fee.gas_limit` field. When `BaseApp` begins executing a transaction, it creates a `GasMeter` initialized with that limit and attaches it to the context. -The [`GasMeter`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/store/types/gas.go#L42) interface provides two key methods: +The [`GasMeter`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/store/types/gas.go#L42) interface provides two key methods: ```go type GasMeter interface { @@ -88,7 +88,7 @@ When submitting a transaction, users specify two of the three values `fees`, `ga ### How gas is consumed -Gas is consumed automatically at the store layer. Every read and write through the [`GasKVStore`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/store/gaskv/store.go#L12) wrapper charges gas before delegating to the underlying store: +Gas is consumed automatically at the store layer. Every read and write through the [`GasKVStore`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/store/gaskv/store.go#L12) wrapper charges gas before delegating to the underlying store: - A `Get` (store read) charges a flat read cost plus a per-byte cost for the key and value. - A `Set` (store write) charges a flat write cost plus a per-byte cost for the key and value. @@ -113,7 +113,7 @@ Events are not part of consensus state. They are not stored in the KVStore, do n ### EventManager -Modules emit events through the [`EventManager`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/types/events.go#L25), which is attached to the context. +Modules emit events through the [`EventManager`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/types/events.go#L25), which is attached to the context. The `EventManager` is created fresh for each transaction and collects all events emitted during that execution. @@ -125,11 +125,11 @@ The SDK automatically emits a `message` event for every transaction, with these - `message.module` — the module name, derived from the type URL - `message.sender` — the signer address, if present -These are defined as constants in [`types/events.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/types/events.go#L249-L263). Modules follow the same convention when emitting their own events. +These are defined as constants in [`types/events.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/types/events.go#L249-L263). Modules follow the same convention when emitting their own events. ### Emitting events -Modules emit events using [`EmitEvent`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/types/events.go#L35) or [`EmitTypedEvent`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/types/events.go#L58): +Modules emit events using [`EmitEvent`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/types/events.go#L35) or [`EmitTypedEvent`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/types/events.go#L58): ```go // emit an untyped event diff --git a/sdk/latest/learn/concepts/encoding.mdx b/sdk/latest/learn/concepts/encoding.mdx index d29327414..5f6db2bbf 100644 --- a/sdk/latest/learn/concepts/encoding.mdx +++ b/sdk/latest/learn/concepts/encoding.mdx @@ -26,7 +26,7 @@ The Cosmos SDK uses protobuf for a fundamental reason: consensus requires determ Every validator in the network independently executes each block. After execution, each validator computes the [app hash](/sdk/latest/learn/concepts/store#app-hash), a cryptographic hash of the application state. For validators to agree on the app hash, they must all produce exactly the same bytes for every piece of state they write. -Protobuf alone does not guarantee this. The Cosmos SDK uses protobuf **with additional deterministic encoding rules** formalized in [ADR-027 (Deterministic Protobuf Serialization)](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/docs/architecture/adr-027-deterministic-protobuf-serialization.md). ADR-027 specifies constraints such as requiring fields to appear in ascending field-number order and varint encodings to be as short as possible. The SDK validates incoming transactions against these rules before processing them, so a non-deterministically encoded transaction is rejected rather than producing divergent state. Every validator encoding the same data under these rules produces an identical byte sequence. +Protobuf alone does not guarantee this. The Cosmos SDK uses protobuf **with additional deterministic encoding rules** formalized in [ADR-027 (Deterministic Protobuf Serialization)](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/docs/architecture/adr-027-deterministic-protobuf-serialization.md). ADR-027 specifies constraints such as requiring fields to appear in ascending field-number order and varint encodings to be as short as possible. The SDK validates incoming transactions against these rules before processing them, so a non-deterministically encoded transaction is rejected rather than producing divergent state. Every validator encoding the same data under these rules produces an identical byte sequence. Beyond determinism, protobuf provides: @@ -57,7 +57,7 @@ Note: genesis data is distributed as JSON in `genesis.json`, but during chain in ## Transaction encoding -Transactions are protobuf messages defined in [`cosmos.tx.v1beta1`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/proto/cosmos/tx/v1beta1/tx.proto). A transaction is composed of three parts: +Transactions are protobuf messages defined in [`cosmos.tx.v1beta1`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/tx/v1beta1/tx.proto). A transaction is composed of three parts: ```text Tx @@ -105,11 +105,9 @@ A **sign mode** determines what bytes a signer commits to when signing a transac - `SIGN_MODE_LEGACY_AMINO_JSON`: the signer signs over an Amino JSON-encoded `StdSignDoc` instead of the protobuf `SignDoc`. This exists for backward compatibility with hardware wallets (e.g., older Ledger firmware) and client tooling that predates protobuf. New modules and chains should not depend on it. -- `SIGN_MODE_TEXTUAL`: the signer signs over a human-readable CBOR-encoded representation of the transaction, designed to display legibly on hardware wallet screens (introduced in v0.50, see [ADR-050](/sdk/latest/reference/architecture/adr-050-sign-mode-textual)). This is the SDK's newer direction for human-readable signing on hardware wallets, intended to replace `SIGN_MODE_LEGACY_AMINO_JSON` over time. Its specification is versioned and has evolved across SDK releases. - - `SIGN_MODE_DIRECT_AUX`: allows N-1 signers in a multi-signer transaction to sign over only `TxBody` and their own `SignerInfo`, without specifying fees. The designated fee payer signs last using `SIGN_MODE_DIRECT`. This simplifies multi-signature UX. -The sign mode is negotiated at transaction construction time and does not affect how state is stored or how validators execute transactions. It only affects what bytes are signed. The full list of sign modes is defined in [`signing.proto`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/proto/cosmos/tx/signing/v1beta1/signing.proto#L17). +The sign mode is negotiated at transaction construction time and does not affect how state is stored or how validators execute transactions. It only affects what bytes are signed. The full list of sign modes is defined in [`signing.proto`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/tx/signing/v1beta1/signing.proto#L17). **For module developers:** `SIGN_MODE_DIRECT` requires no extra work. If you want your module's messages to be signable on Ledger hardware wallets using `SIGN_MODE_LEGACY_AMINO_JSON`, register your message types with the Amino codec via `RegisterLegacyAminoCodec` in your module's `codec.go`. @@ -119,7 +117,7 @@ The sign mode is negotiated at transaction construction time and does not affect Every transaction message must declare which addresses are authorized to sign it. In v0.50+, this is done via the `cosmos.msg.v1.signer` protobuf annotation — the SDK reads the annotation at startup and automatically extracts signer addresses from that field. See [Protobuf Annotations](/sdk/latest/guides/reference/protobuf-annotations) for the full annotation reference. -For messages that cannot use the annotation — for example, messages with non-standard signing logic such as EVM-compatible transactions — you can register a custom signer function using [`signing.CustomGetSigner`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/x/tx/signing/context.go#L127): +For messages that cannot use the annotation — for example, messages with non-standard signing logic such as EVM-compatible transactions — you can register a custom signer function using [`signing.CustomGetSigner`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/x/tx/signing/context.go#L127): ```go signer := signing.CustomGetSigner{ @@ -175,7 +173,7 @@ The codec (`k.cdc`) is the protobuf codec described in the next section. ## The codec and interface registry -The Cosmos SDK wraps protobuf in a **codec** that modules use for marshaling and unmarshaling. The primary implementation is [`ProtoCodec`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/codec/proto_codec.go), which calls protobuf's `Marshal` and `Unmarshal` under the hood. +The Cosmos SDK wraps protobuf in a **codec** that modules use for marshaling and unmarshaling. The primary implementation is [`ProtoCodec`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/codec/proto_codec.go), which calls protobuf's `Marshal` and `Unmarshal` under the hood. ```go type ProtoCodec struct { @@ -239,7 +237,7 @@ This lookup is handled by the **interface registry**. ### Interface registry -The [`InterfaceRegistry`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/codec/types/interface_registry.go) is a runtime map from type URLs to Go types. When the SDK encounters an `Any` value, it queries the registry with the type URL to find the concrete Go type, then uses protobuf to unmarshal the bytes. +The [`InterfaceRegistry`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/codec/types/interface_registry.go) is a runtime map from type URLs to Go types. When the SDK encounters an `Any` value, it queries the registry with the type URL to find the concrete Go type, then uses protobuf to unmarshal the bytes. ```text Any { type_url, value_bytes } diff --git a/sdk/latest/learn/concepts/store.mdx b/sdk/latest/learn/concepts/store.mdx index 769de87c7..2958d413a 100644 --- a/sdk/latest/learn/concepts/store.mdx +++ b/sdk/latest/learn/concepts/store.mdx @@ -32,7 +32,7 @@ key: 0x2 | 20 | cosmos1abc...xyz | uatom value: ProtocolBuffer(1000000) ``` -The key encodes the store prefix, address length, address, and denomination. The value is a Protocol Buffer-encoded amount. See [`x/bank/types/keys.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/x/bank/types/keys.go) for the actual implementation. +The key encodes the store prefix, address length, address, and denomination. The value is a Protocol Buffer-encoded amount. See [`x/bank/types/keys.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/x/bank/types/keys.go) for the actual implementation. Each module owns its own namespace in the key-value store. Keys are defined by the module and typically begin with a byte prefix that distinguishes them from other module keys. @@ -40,7 +40,7 @@ Each module owns its own namespace in the key-value store. Keys are defined by t A single module store is only part of the picture. At the application level, all module stores are committed together. -Every module has its own KVStore, and all module stores are mounted inside a [multistore](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/store/rootmulti/store.go) that is committed as a single state root. +Every module has its own KVStore, and all module stores are mounted inside a [multistore](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/store/rootmulti/store.go) that is committed as a single state root. A module can only read and write to its own store through its keeper. Access is gated by a `StoreKey`, which is a typed capability object registered at app startup. Modules that don't hold the key cannot open the store. @@ -57,7 +57,7 @@ The full storage stack from top to bottom is: ``` Module keeper ↓ -KVStore (namespaced, wrapped with gas/trace) +KVStore (namespaced, wrapped with gas metering) ↓ CommitMultiStore (multistore, computes app hash) ↓ @@ -68,7 +68,7 @@ Database backend (goleveldb by default) ## How state is stored (IAVL and commit stores) -Each module's KVStore is backed by a [`CommitKVStore`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/store/iavl/store.go#L36). [See the store spec for more details.](/sdk/latest/guides/state/store) +Each module's KVStore is backed by a [`CommitKVStore`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/store/iavl/store.go#L36). [See the store spec for more details.](/sdk/latest/guides/state/store) In the current SDK store implementation described here, the Cosmos SDK uses [IAVL](https://github.com/cosmos/iavl), a versioned AVL Merkle tree. @@ -116,16 +116,16 @@ Beyond the base KVStore, the SDK provides several specialized store wrappers. - [CommitKVStore](#commitkvstore-persistent-store) - [CacheMultiStore](#cachemultistore-transaction-isolation) - [Ephemeral store types](#ephemeral-store-types) -- [Gas and trace store wrappers](#gas-and-trace-store-wrappers) +- [Gas store wrapper](#gas-store-wrapper) - [Prefix store](#prefix-store) ### CommitKVStore (persistent store) -The [`CommitKVStore`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/store/iavl/store.go#L36) is the main persistent store backed by IAVL. It persists across blocks, produces versioned commits, and contributes to the app hash. +The [`CommitKVStore`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/store/iavl/store.go#L36) is the main persistent store backed by IAVL. It persists across blocks, produces versioned commits, and contributes to the app hash. ### CacheMultiStore (transaction isolation) -Before executing each transaction, the Cosmos SDK's `BaseApp` creates a [`CacheMultiStore`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/store/cachemulti/store.go) — a cached, copy-on-write view of the multistore. +Before executing each transaction, the Cosmos SDK's `BaseApp` creates a [`CacheMultiStore`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/store/cachemulti/store.go) — a cached, copy-on-write view of the multistore. All writes during that transaction occur in this cached layer: @@ -144,9 +144,9 @@ This is how transaction atomicity is implemented in the store layer. ### Ephemeral store types -[Transient stores](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/store/transient/store.go) are cleared at the end of each block. They are used for temporary per-block data such as counters or intermediate calculations, and do not affect the app hash. +[Transient stores](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/store/transient/store.go) are cleared at the end of each block. They are used for temporary per-block data such as counters or intermediate calculations, and do not affect the app hash. -[Memory stores](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/store/mem/store.go) survive block commits but reset when the node restarts — their `Commit()` is a no-op and data is never written to disk. They are used for in-process caching of data that is expensive to recompute each block but does not need to survive a restart. Modules access them via `MemoryStoreKey`, mounted with `MountMemoryStores` in `app.go`. +[Memory stores](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/store/mem/store.go) survive block commits but reset when the node restarts — their `Commit()` is a no-op and data is never written to disk. They are used for in-process caching of data that is expensive to recompute each block but does not need to survive a restart. Modules access them via `MemoryStoreKey`, mounted with `MountMemoryStores` in `app.go`. | Store type | Survives block commit | Survives restart | |---|---|---| @@ -154,18 +154,15 @@ This is how transaction atomicity is implemented in the store layer. | Memory | Yes | No | | IAVL (CommitKVStore) | Yes | Yes | -### Gas and trace store wrappers +### Gas store wrapper -All store accesses are wrapped with additional behavior by the [`GasKVStore`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/store/gaskv/store.go) and [`TraceKVStore`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/store/tracekv/store.go) wrappers. - -- `GasKVStore` charges gas for each read and write -- `TraceKVStore` logs each store operation for debugging +All store accesses are wrapped with additional behavior by the [`GasKVStore`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/store/gaskv/store.go) wrapper, which charges gas for each read and write. Because state access is the dominant cost of transaction execution, the SDK charges gas at the store layer so that expensive reads and writes are reflected in transaction fees. Every read and write of a KVStore costs gas, and expensive operations naturally cost more. [Execution Context, Gas, and Events](/sdk/latest/learn/concepts/context-gas-events) explains how gas metering works at runtime. ### Prefix store -A [**prefix store**](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/store/prefix/store.go) wraps a KVStore and automatically prepends a fixed byte prefix to every key. This lets keepers scope their reads and writes to a sub-namespace without manually constructing prefixed keys on every call. +A [**prefix store**](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/store/prefix/store.go) wraps a KVStore and automatically prepends a fixed byte prefix to every key. This lets keepers scope their reads and writes to a sub-namespace without manually constructing prefixed keys on every call. ```go prefixStore := prefix.NewStore(kvStore, types.KeyPrefix("balances")) @@ -197,7 +194,7 @@ k.Counter.Set(ctx, count+1) The Collections API defines the storage schema, handles encoding and decoding, ensures consistent key construction, and makes state access type-safe. -Under the hood, collections still store data in a KVStore. Collections are used to provide a safer abstraction over raw byte keys. See [`collections/collections.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/collections/collections.go) for the base interface definitions. For the full package guide, see [Collections](/sdk/latest/guides/state/collections). +Under the hood, collections still store data in a KVStore. Collections are used to provide a safer abstraction over raw byte keys. See [`collections/collections.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/collections/collections.go) for the base interface definitions. For the full package guide, see [Collections](/sdk/latest/guides/state/collections). ## How modules access state @@ -249,6 +246,6 @@ For a walkthrough of genesis implementation in a module, see [Step 2: Proto file ## Next steps -For more information on stores, pruning strategies, and store configuration, see the [store spec](/sdk/latest/guides/state/store). For the full store interface definitions, see [`store/types/store.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/store/types/store.go) in the SDK source. +For more information on stores, pruning strategies, and store configuration, see the [store spec](/sdk/latest/guides/state/store). For the full store interface definitions, see [`store/types/store.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/store/types/store.go) in the SDK source. Because KV stores only hold raw bytes, modules must serialize structured data before writing it. The next section, [Encoding and Protobuf](/sdk/latest/learn/concepts/encoding), explains how the Cosmos SDK uses Protocol Buffers to encode that data deterministically, and why every validator must produce exactly the same bytes. diff --git a/sdk/latest/learn/concepts/testing.mdx b/sdk/latest/learn/concepts/testing.mdx index 87d6d6208..0760a0569 100644 --- a/sdk/latest/learn/concepts/testing.mdx +++ b/sdk/latest/learn/concepts/testing.mdx @@ -306,7 +306,7 @@ app.sm.RegisterStoreDecoders() ### testutil -The [`testutil`](https://github.com/cosmos/cosmos-sdk/tree/main/testutil) package provides helpers for constructing in-memory contexts for unit tests: +The [`testutil`](https://github.com/cosmos/cosmos-sdk/tree/release/v0.55.x/testutil) package provides helpers for constructing in-memory contexts for unit tests: - `testutil.DefaultContextWithDB` creates a real `sdk.Context` backed by an in-memory KV store. Keeper unit tests use this to get a realistic execution context without starting a full node. - `moduletestutil.MakeTestEncodingConfig` returns a codec with standard interface registration, suitable for keeper tests. @@ -328,7 +328,7 @@ func TestKeeperTestSuite(t *testing.T) { For a full guide on configuring and running simulations, see the [Module Simulation](/sdk/latest/guides/testing/simulator) page. -[`simsx`](https://github.com/cosmos/cosmos-sdk/tree/main/testutil/simsx) is the simulation execution framework. It provides: +[`simsx`](https://github.com/cosmos/cosmos-sdk/tree/release/v0.55.x/testutil/simsx) is the simulation execution framework. It provides: - `SimMsgFactoryFn`: a function type that implements the `SimMsgFactoryX` interface for message factories. Each factory selects random accounts and parameters, constructs a message, and returns it for execution. - `ChainDataSource`: provides access to random accounts, balances, and other chain data during message construction. diff --git a/sdk/latest/modules/auth/auth.mdx b/sdk/latest/modules/auth/auth.mdx index 2c94d9f87..ae4a116c5 100644 --- a/sdk/latest/modules/auth/auth.mdx +++ b/sdk/latest/modules/auth/auth.mdx @@ -160,7 +160,7 @@ See [Vesting](/sdk/latest/modules/auth/auth). ## AnteHandlers The `x/auth` module presently has no transaction handlers of its own, but does expose the special `AnteHandler`, used for performing basic validity checks on a transaction, such that it could be thrown out of the mempool. -The `AnteHandler` can be seen as a set of decorators that check transactions within the current context, per [ADR 010](https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-010-modular-antehandler.md). +The `AnteHandler` can be seen as a set of decorators that check transactions within the current context, per [ADR 010](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/docs/architecture/adr-010-modular-antehandler.md). Note that the `AnteHandler` is called on both `CheckTx` and `DeliverTx`, as CometBFT proposers presently have the ability to include in their proposed block transactions which fail `CheckTx`. @@ -260,6 +260,7 @@ The auth module contains the following parameters: | TxSigLimit | uint64 | 7 | | TxSizeCostPerByte | uint64 | 10 | | SigVerifyCostED25519 | uint64 | 590 | +| SigVerifyCostMlDsa65 | uint64 | 750 | | SigVerifyCostSecp256k1 | uint64 | 1000 | ## Client @@ -420,6 +421,7 @@ Example Output: ```bash max_memo_characters: "256" sig_verify_cost_ed25519: "590" +sig_verify_cost_mldsa65: "750" sig_verify_cost_secp256k1: "1000" tx_sig_limit: "7" tx_size_cost_per_byte: "10" diff --git a/sdk/latest/modules/auth/tx.mdx b/sdk/latest/modules/auth/tx.mdx index 5e853fac9..099aff5dc 100644 --- a/sdk/latest/modules/auth/tx.mdx +++ b/sdk/latest/modules/auth/tx.mdx @@ -34,19 +34,19 @@ This package represents the Cosmos SDK implementation of the `client.TxConfig`, The interface defines a set of methods for creating a `client.TxBuilder`. ```go -// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/client/tx_config.go#L25-L31 +// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.55.x/client/tx_config.go#L26-L36 ``` The default implementation of `client.TxConfig` is instantiated by `NewTxConfig` in `x/auth/tx` module. ```go -// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/x/auth/tx/config.go#L22-L28 +// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.55.x/x/auth/tx/config.go#L67-L87 ``` ### `TxBuilder` ```go -// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/client/tx_config.go#L33-L50 +// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.55.x/client/tx_config.go#L38-L56 ``` The [`client.TxBuilder`](/sdk/latest/learn/concepts/lifecycle#transaction-generation) interface is as well implemented by `x/auth/tx`. diff --git a/sdk/latest/modules/auth/vesting.mdx b/sdk/latest/modules/auth/vesting.mdx index acdbe1d2b..99c09d4f8 100644 --- a/sdk/latest/modules/auth/vesting.mdx +++ b/sdk/latest/modules/auth/vesting.mdx @@ -78,25 +78,25 @@ int64 ### BaseVestingAccount ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/proto/cosmos/vesting/v1beta1/vesting.proto#L11-L35 +// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.55.x/proto/cosmos/vesting/v1beta1/vesting.proto#L12-L39 ``` ### ContinuousVestingAccount ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/proto/cosmos/vesting/v1beta1/vesting.proto#L37-L46 +// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.55.x/proto/cosmos/vesting/v1beta1/vesting.proto#L41-L50 ``` ### DelayedVestingAccount ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/proto/cosmos/vesting/v1beta1/vesting.proto#L48-L57 +// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.55.x/proto/cosmos/vesting/v1beta1/vesting.proto#L52-L60 ``` ### Period ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/proto/cosmos/vesting/v1beta1/vesting.proto#L59-L69 +// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.55.x/proto/cosmos/vesting/v1beta1/vesting.proto#L62-L72 ``` ```go @@ -107,7 +107,7 @@ type Periods []Period ### PeriodicVestingAccount ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/proto/cosmos/vesting/v1beta1/vesting.proto#L71-L81 +// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.55.x/proto/cosmos/vesting/v1beta1/vesting.proto#L74-L83 ``` In order to facilitate less ad-hoc type checking and assertions and to support flexibility in account balance usage, the existing `x/bank` `ViewKeeper` interface is updated to contain the following: @@ -131,7 +131,7 @@ sdk.Coins ### PermanentLockedAccount ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/proto/cosmos/vesting/v1beta1/vesting.proto#L83-L94 +// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.55.x/proto/cosmos/vesting/v1beta1/vesting.proto#L85-L94 ``` ## Vesting Account Specification diff --git a/sdk/latest/modules/authz/README.mdx b/sdk/latest/modules/authz/README.mdx index 7ca72c974..ae46e2838 100644 --- a/sdk/latest/modules/authz/README.mdx +++ b/sdk/latest/modules/authz/README.mdx @@ -4,7 +4,7 @@ title: 'x/authz' ## Abstract -`x/authz` is an implementation of a Cosmos SDK module, per [ADR 30](https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-030-authz-module.md), that allows +`x/authz` is an implementation of a Cosmos SDK module, per [ADR 30](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/docs/architecture/adr-030-authz-module.md), that allows granting arbitrary privileges from one account (the granter) to another account (the grantee). Authorizations must be granted for a particular Msg service method one by one using an implementation of the `Authorization` interface. ## Contents @@ -31,7 +31,7 @@ granting arbitrary privileges from one account (the granter) to another account ### Authorization and Grant The `x/authz` module defines interfaces and messages grant authorizations to perform actions -on behalf of one account to other accounts. The design is defined in the [ADR 030](https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-030-authz-module.md). +on behalf of one account to other accounts. The design is defined in the [ADR 030](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/docs/architecture/adr-030-authz-module.md). A *grant* is an allowance to execute a Msg by the grantee on behalf of the granter. Authorization is an interface that must be implemented by a concrete authorization logic to validate and execute grants. Authorizations are extensible and can be defined for any Msg service method even outside of the module where the Msg method is defined. See the `SendAuthorization` example in the next section for more details. @@ -93,7 +93,7 @@ The Cosmos SDK `x/authz` module comes with following authorization types: `GenericAuthorization` implements the `Authorization` interface that gives unrestricted permission to execute the provided Msg on behalf of granter's account. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/authz/v1beta1/authz.proto#L14-L22 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/authz/v1beta1/authz.proto#L13-L21 ``` ```go expandable @@ -152,7 +152,7 @@ error { * It takes an (optional) `AllowList` that specifies to which addresses a grantee can send token. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/bank/v1beta1/authz.proto#L11-L30 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/bank/v1beta1/authz.proto#L11-L29 ``` ```go expandable @@ -279,10 +279,10 @@ return allowedAddrs #### StakeAuthorization -`StakeAuthorization` implements the `Authorization` interface for messages in the [staking module](/sdk/latest/modules/staking). It takes an `AuthorizationType` to specify whether you want to authorise delegating, undelegating or redelegating (i.e. these have to be authorised separately). It also takes an optional `MaxTokens` that keeps track of a limit to the amount of tokens that can be delegated/undelegated/redelegated. If left empty, the amount is unlimited. Additionally, this Msg takes an `AllowList` or a `DenyList`, which allows you to select which validators you allow or deny grantees to stake with. +`StakeAuthorization` implements the `Authorization` interface for messages in the [staking module](/sdk/latest/modules/staking). It takes an `AuthorizationType` to specify whether you want to authorise delegating, undelegating, redelegating, or cancelling an unbonding delegation (i.e. these have to be authorised separately). It also takes an optional `MaxTokens` that keeps track of a limit to the amount of tokens that can be delegated/undelegated/redelegated. If left empty, the amount is unlimited. Additionally, this Msg takes an `AllowList` or a `DenyList`, which allows you to select which validators you allow or deny grantees to stake with. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/authz.proto#L11-L35 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/staking/v1beta1/authz.proto#L10-L33 ``` ```go expandable @@ -498,602 +498,19 @@ Grants are identified by combining granter address (the address bytes of the gra The grant object encapsulates an `Authorization` type and an expiration timestamp: ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/authz/v1beta1/authz.proto#L24-L32 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/authz/v1beta1/authz.proto#L23-L31 ``` ### GrantQueue We are maintaining a queue for authz pruning. Whenever a grant is created, an item will be added to `GrantQueue` with a key of expiration, granter, grantee. -In `EndBlock` (which runs for every block) we continuously check and prune the expired grants by forming a prefix key with current blocktime that passed the stored expiration in `GrantQueue`, we iterate through all the matched records from `GrantQueue` and delete them from the `GrantQueue` & `Grant`s store. +In `BeginBlock`, which runs for every block, the module prunes expired grants. It forms a prefix key from the current block time and matches the records in `GrantQueue` whose stored expiration has passed. It deletes those records from both the `GrantQueue` and the `Grant` store. Pruning is capped at 200 grants per block. Any remaining expired grants are pruned in later blocks. -```go expandable -package keeper - -import ( - - "fmt" - "strconv" - "time" - "github.com/cosmos/gogoproto/proto" - abci "github.com/tendermint/tendermint/abci/types" - "github.com/tendermint/tendermint/libs/log" - "github.com/cosmos/cosmos-sdk/baseapp" - "github.com/cosmos/cosmos-sdk/codec" - codectypes "github.com/cosmos/cosmos-sdk/codec/types" - storetypes "github.com/cosmos/cosmos-sdk/store/types" - sdk "github.com/cosmos/cosmos-sdk/types" - sdkerrors "github.com/cosmos/cosmos-sdk/types/errors" - "github.com/cosmos/cosmos-sdk/x/authz" -) - -// TODO: Revisit this once we have proper gas fee framework. -// Tracking issues https://github.com/cosmos/cosmos-sdk/issues/9054, -// https://github.com/cosmos/cosmos-sdk/discussions/9072 -const gasCostPerIteration = uint64(20) - -type Keeper struct { - storeKey storetypes.StoreKey - cdc codec.BinaryCodec - router *baseapp.MsgServiceRouter - authKeeper authz.AccountKeeper -} - -// NewKeeper constructs a message authorization Keeper -func NewKeeper(storeKey storetypes.StoreKey, cdc codec.BinaryCodec, router *baseapp.MsgServiceRouter, ak authz.AccountKeeper) - -Keeper { - return Keeper{ - storeKey: storeKey, - cdc: cdc, - router: router, - authKeeper: ak, -} -} - -// Logger returns a module-specific logger. -func (k Keeper) - -Logger(ctx sdk.Context) - -log.Logger { - return ctx.Logger().With("module", fmt.Sprintf("x/%s", authz.ModuleName)) -} - -// getGrant returns grant stored at skey. -func (k Keeper) - -getGrant(ctx sdk.Context, skey []byte) (grant authz.Grant, found bool) { - store := ctx.KVStore(k.storeKey) - bz := store.Get(skey) - if bz == nil { - return grant, false -} - -k.cdc.MustUnmarshal(bz, &grant) - -return grant, true -} - -func (k Keeper) - -update(ctx sdk.Context, grantee sdk.AccAddress, granter sdk.AccAddress, updated authz.Authorization) - -error { - skey := grantStoreKey(grantee, granter, updated.MsgTypeURL()) - -grant, found := k.getGrant(ctx, skey) - if !found { - return authz.ErrNoAuthorizationFound -} - -msg, ok := updated.(proto.Message) - if !ok { - return sdkerrors.ErrPackAny.Wrapf("cannot proto marshal %T", updated) -} - -any, err := codectypes.NewAnyWithValue(msg) - if err != nil { - return err -} - -grant.Authorization = any - store := ctx.KVStore(k.storeKey) - -store.Set(skey, k.cdc.MustMarshal(&grant)) - -return nil -} - -// DispatchActions attempts to execute the provided messages via authorization -// grants from the message signer to the grantee. -func (k Keeper) - -DispatchActions(ctx sdk.Context, grantee sdk.AccAddress, msgs []sdk.Msg) ([][]byte, error) { - results := make([][]byte, len(msgs)) - now := ctx.BlockTime() - for i, msg := range msgs { - signers := msg.GetSigners() - if len(signers) != 1 { - return nil, authz.ErrAuthorizationNumOfSigners -} - granter := signers[0] - - // If granter != grantee then check authorization.Accept, otherwise we - // implicitly accept. - if !granter.Equals(grantee) { - skey := grantStoreKey(grantee, granter, sdk.MsgTypeURL(msg)) - -grant, found := k.getGrant(ctx, skey) - if !found { - return nil, sdkerrors.Wrapf(authz.ErrNoAuthorizationFound, "failed to update grant with key %s", string(skey)) -} - if grant.Expiration != nil && grant.Expiration.Before(now) { - return nil, authz.ErrAuthorizationExpired -} - -authorization, err := grant.GetAuthorization() - if err != nil { - return nil, err -} - -resp, err := authorization.Accept(ctx, msg) - if err != nil { - return nil, err -} - if resp.Delete { - err = k.DeleteGrant(ctx, grantee, granter, sdk.MsgTypeURL(msg)) -} - -else if resp.Updated != nil { - err = k.update(ctx, grantee, granter, resp.Updated) -} - if err != nil { - return nil, err -} - if !resp.Accept { - return nil, sdkerrors.ErrUnauthorized -} - -} - handler := k.router.Handler(msg) - if handler == nil { - return nil, sdkerrors.ErrUnknownRequest.Wrapf("unrecognized message route: %s", sdk.MsgTypeURL(msg)) -} - -msgResp, err := handler(ctx, msg) - if err != nil { - return nil, sdkerrors.Wrapf(err, "failed to execute message; message %v", msg) -} - -results[i] = msgResp.Data - - // emit the events from the dispatched actions - events := msgResp.Events - sdkEvents := make([]sdk.Event, 0, len(events)) - for _, event := range events { - e := event - e.Attributes = append(e.Attributes, abci.EventAttribute{ - Key: "authz_msg_index", - Value: strconv.Itoa(i) -}) - -sdkEvents = append(sdkEvents, sdk.Event(e)) -} - -ctx.EventManager().EmitEvents(sdkEvents) -} - -return results, nil -} - -// SaveGrant method grants the provided authorization to the grantee on the granter's account -// with the provided expiration time and insert authorization key into the grants queue. If there is an existing authorization grant for the -// same `sdk.Msg` type, this grant overwrites that. -func (k Keeper) - -SaveGrant(ctx sdk.Context, grantee, granter sdk.AccAddress, authorization authz.Authorization, expiration *time.Time) - -error { - store := ctx.KVStore(k.storeKey) - msgType := authorization.MsgTypeURL() - skey := grantStoreKey(grantee, granter, msgType) - -grant, err := authz.NewGrant(ctx.BlockTime(), authorization, expiration) - if err != nil { - return err -} - -var oldExp *time.Time - if oldGrant, found := k.getGrant(ctx, skey); found { - oldExp = oldGrant.Expiration -} - if oldExp != nil && (expiration == nil || !oldExp.Equal(*expiration)) { - if err = k.removeFromGrantQueue(ctx, skey, granter, grantee, *oldExp); err != nil { - return err -} - -} - - // If the expiration didn't change, then we don't remove it and we should not insert again - if expiration != nil && (oldExp == nil || !oldExp.Equal(*expiration)) { - if err = k.insertIntoGrantQueue(ctx, granter, grantee, msgType, *expiration); err != nil { - return err -} - -} - bz := k.cdc.MustMarshal(&grant) - -store.Set(skey, bz) - -return ctx.EventManager().EmitTypedEvent(&authz.EventGrant{ - MsgTypeUrl: authorization.MsgTypeURL(), - Granter: granter.String(), - Grantee: grantee.String(), -}) -} - -// DeleteGrant revokes any authorization for the provided message type granted to the grantee -// by the granter. -func (k Keeper) - -DeleteGrant(ctx sdk.Context, grantee sdk.AccAddress, granter sdk.AccAddress, msgType string) - -error { - store := ctx.KVStore(k.storeKey) - skey := grantStoreKey(grantee, granter, msgType) - -grant, found := k.getGrant(ctx, skey) - if !found { - return sdkerrors.Wrapf(authz.ErrNoAuthorizationFound, "failed to delete grant with key %s", string(skey)) -} - if grant.Expiration != nil { - err := k.removeFromGrantQueue(ctx, skey, granter, grantee, *grant.Expiration) - if err != nil { - return err -} - -} - -store.Delete(skey) - -return ctx.EventManager().EmitTypedEvent(&authz.EventRevoke{ - MsgTypeUrl: msgType, - Granter: granter.String(), - Grantee: grantee.String(), -}) -} - -// GetAuthorizations Returns list of `Authorizations` granted to the grantee by the granter. -func (k Keeper) - -GetAuthorizations(ctx sdk.Context, grantee sdk.AccAddress, granter sdk.AccAddress) ([]authz.Authorization, error) { - store := ctx.KVStore(k.storeKey) - key := grantStoreKey(grantee, granter, "") - iter := sdk.KVStorePrefixIterator(store, key) - -defer iter.Close() - -var authorization authz.Grant - var authorizations []authz.Authorization - for ; iter.Valid(); iter.Next() { - if err := k.cdc.Unmarshal(iter.Value(), &authorization); err != nil { - return nil, err -} - -a, err := authorization.GetAuthorization() - if err != nil { - return nil, err -} - -authorizations = append(authorizations, a) -} - -return authorizations, nil -} - -// GetAuthorization returns an Authorization and it's expiration time. -// A nil Authorization is returned under the following circumstances: -// - No grant is found. -// - A grant is found, but it is expired. -// - There was an error getting the authorization from the grant. -func (k Keeper) - -GetAuthorization(ctx sdk.Context, grantee sdk.AccAddress, granter sdk.AccAddress, msgType string) (authz.Authorization, *time.Time) { - grant, found := k.getGrant(ctx, grantStoreKey(grantee, granter, msgType)) - if !found || (grant.Expiration != nil && grant.Expiration.Before(ctx.BlockHeader().Time)) { - return nil, nil -} - -auth, err := grant.GetAuthorization() - if err != nil { - return nil, nil -} - -return auth, grant.Expiration -} - -// IterateGrants iterates over all authorization grants -// This function should be used with caution because it can involve significant IO operations. -// It should not be used in query or msg services without charging additional gas. -// The iteration stops when the handler function returns true or the iterator exhaust. -func (k Keeper) - -IterateGrants(ctx sdk.Context, - handler func(granterAddr sdk.AccAddress, granteeAddr sdk.AccAddress, grant authz.Grant) - -bool, -) { - store := ctx.KVStore(k.storeKey) - iter := sdk.KVStorePrefixIterator(store, GrantKey) - -defer iter.Close() - for ; iter.Valid(); iter.Next() { - var grant authz.Grant - granterAddr, granteeAddr, _ := parseGrantStoreKey(iter.Key()) - -k.cdc.MustUnmarshal(iter.Value(), &grant) - if handler(granterAddr, granteeAddr, grant) { - break -} - -} -} - -func (k Keeper) - -getGrantQueueItem(ctx sdk.Context, expiration time.Time, granter, grantee sdk.AccAddress) (*authz.GrantQueueItem, error) { - store := ctx.KVStore(k.storeKey) - bz := store.Get(GrantQueueKey(expiration, granter, grantee)) - if bz == nil { - return &authz.GrantQueueItem{ -}, nil -} - -var queueItems authz.GrantQueueItem - if err := k.cdc.Unmarshal(bz, &queueItems); err != nil { - return nil, err -} - -return &queueItems, nil -} - -func (k Keeper) - -setGrantQueueItem(ctx sdk.Context, expiration time.Time, - granter sdk.AccAddress, grantee sdk.AccAddress, queueItems *authz.GrantQueueItem, -) - -error { - store := ctx.KVStore(k.storeKey) - -bz, err := k.cdc.Marshal(queueItems) - if err != nil { - return err -} - -store.Set(GrantQueueKey(expiration, granter, grantee), bz) - -return nil -} - -// insertIntoGrantQueue inserts a grant key into the grant queue -func (k Keeper) - -insertIntoGrantQueue(ctx sdk.Context, granter, grantee sdk.AccAddress, msgType string, expiration time.Time) - -error { - queueItems, err := k.getGrantQueueItem(ctx, expiration, granter, grantee) - if err != nil { - return err -} - if len(queueItems.MsgTypeUrls) == 0 { - k.setGrantQueueItem(ctx, expiration, granter, grantee, &authz.GrantQueueItem{ - MsgTypeUrls: []string{ - msgType -}, -}) -} - -else { - queueItems.MsgTypeUrls = append(queueItems.MsgTypeUrls, msgType) - -k.setGrantQueueItem(ctx, expiration, granter, grantee, queueItems) -} - -return nil -} - -// removeFromGrantQueue removes a grant key from the grant queue -func (k Keeper) - -removeFromGrantQueue(ctx sdk.Context, grantKey []byte, granter, grantee sdk.AccAddress, expiration time.Time) - -error { - store := ctx.KVStore(k.storeKey) - key := GrantQueueKey(expiration, granter, grantee) - bz := store.Get(key) - if bz == nil { - return sdkerrors.Wrap(authz.ErrNoGrantKeyFound, "can't remove grant from the expire queue, grant key not found") -} - -var queueItem authz.GrantQueueItem - if err := k.cdc.Unmarshal(bz, &queueItem); err != nil { - return err -} - - _, _, msgType := parseGrantStoreKey(grantKey) - queueItems := queueItem.MsgTypeUrls - for index, typeURL := range queueItems { - ctx.GasMeter().ConsumeGas(gasCostPerIteration, "grant queue") - if typeURL == msgType { - end := len(queueItem.MsgTypeUrls) - 1 - queueItems[index] = queueItems[end] - queueItems = queueItems[:end] - if err := k.setGrantQueueItem(ctx, expiration, granter, grantee, &authz.GrantQueueItem{ - MsgTypeUrls: queueItems, -}); err != nil { - return err -} - -break -} - -} - -return nil -} - -// DequeueAndDeleteExpiredGrants deletes expired grants from the state and grant queue. -func (k Keeper) - -DequeueAndDeleteExpiredGrants(ctx sdk.Context) - -error { - store := ctx.KVStore(k.storeKey) - iterator := store.Iterator(GrantQueuePrefix, sdk.InclusiveEndBytes(GrantQueueTimePrefix(ctx.BlockTime()))) - -defer iterator.Close() - for ; iterator.Valid(); iterator.Next() { - var queueItem authz.GrantQueueItem - if err := k.cdc.Unmarshal(iterator.Value(), &queueItem); err != nil { - return err -} - - _, granter, grantee, err := parseGrantQueueKey(iterator.Key()) - if err != nil { - return err -} - -store.Delete(iterator.Key()) - for _, typeURL := range queueItem.MsgTypeUrls { - store.Delete(grantStoreKey(grantee, granter, typeURL)) -} - -} - -return nil -} -``` - -* GrantQueue: `0x02 | expiration_bytes | granter_address_len (1 byte) | granter_address_bytes | grantee_address_len (1 byte) | grantee_address_bytes -> ProtocalBuffer(GrantQueueItem)` +* GrantQueue: `0x02 | expiration_bytes | granter_address_len (1 byte) | granter_address_bytes | grantee_address_len (1 byte) | grantee_address_bytes -> ProtocolBuffer(GrantQueueItem)` The `expiration_bytes` are the expiration date in UTC with the format `"2006-01-02T15:04:05.000000000"`. -```go expandable -package keeper - -import ( - - "time" - "github.com/cosmos/cosmos-sdk/internal/conv" - sdk "github.com/cosmos/cosmos-sdk/types" - "github.com/cosmos/cosmos-sdk/types/address" - "github.com/cosmos/cosmos-sdk/types/kv" - "github.com/cosmos/cosmos-sdk/x/authz" -) - -// Keys for store prefixes -// Items are stored with the following key: values -// -// - 0x01: Grant -// - 0x02: GrantQueueItem -var ( - GrantKey = []byte{0x01 -} // prefix for each key - GrantQueuePrefix = []byte{0x02 -} -) - -var lenTime = len(sdk.FormatTimeBytes(time.Now())) - -// StoreKey is the store key string for authz -const StoreKey = authz.ModuleName - -// grantStoreKey - return authorization store key -// Items are stored with the following key: values -// -// - 0x01: Grant -func grantStoreKey(grantee sdk.AccAddress, granter sdk.AccAddress, msgType string) []byte { - m := conv.UnsafeStrToBytes(msgType) - -granter = address.MustLengthPrefix(granter) - -grantee = address.MustLengthPrefix(grantee) - key := sdk.AppendLengthPrefixedBytes(GrantKey, granter, grantee, m) - -return key -} - -// parseGrantStoreKey - split granter, grantee address and msg type from the authorization key -func parseGrantStoreKey(key []byte) (granterAddr, granteeAddr sdk.AccAddress, msgType string) { - // key is of format: - // 0x01 - - granterAddrLen, granterAddrLenEndIndex := sdk.ParseLengthPrefixedBytes(key, 1, 1) // ignore key[0] since it is a prefix key - granterAddr, granterAddrEndIndex := sdk.ParseLengthPrefixedBytes(key, granterAddrLenEndIndex+1, int(granterAddrLen[0])) - -granteeAddrLen, granteeAddrLenEndIndex := sdk.ParseLengthPrefixedBytes(key, granterAddrEndIndex+1, 1) - -granteeAddr, granteeAddrEndIndex := sdk.ParseLengthPrefixedBytes(key, granteeAddrLenEndIndex+1, int(granteeAddrLen[0])) - -kv.AssertKeyAtLeastLength(key, granteeAddrEndIndex+1) - -return granterAddr, granteeAddr, conv.UnsafeBytesToStr(key[(granteeAddrEndIndex + 1):]) -} - -// parseGrantQueueKey split expiration time, granter and grantee from the grant queue key -func parseGrantQueueKey(key []byte) (time.Time, sdk.AccAddress, sdk.AccAddress, error) { - // key is of format: - // 0x02 - - expBytes, expEndIndex := sdk.ParseLengthPrefixedBytes(key, 1, lenTime) - -exp, err := sdk.ParseTimeBytes(expBytes) - if err != nil { - return exp, nil, nil, err -} - -granterAddrLen, granterAddrLenEndIndex := sdk.ParseLengthPrefixedBytes(key, expEndIndex+1, 1) - -granter, granterEndIndex := sdk.ParseLengthPrefixedBytes(key, granterAddrLenEndIndex+1, int(granterAddrLen[0])) - -granteeAddrLen, granteeAddrLenEndIndex := sdk.ParseLengthPrefixedBytes(key, granterEndIndex+1, 1) - -grantee, _ := sdk.ParseLengthPrefixedBytes(key, granteeAddrLenEndIndex+1, int(granteeAddrLen[0])) - -return exp, granter, grantee, nil -} - -// GrantQueueKey - return grant queue store key. If a given grant doesn't have a defined -// expiration, then it should not be used in the pruning queue. -// Key format is: -// -// 0x02: GrantQueueItem -func GrantQueueKey(expiration time.Time, granter sdk.AccAddress, grantee sdk.AccAddress) []byte { - exp := sdk.FormatTimeBytes(expiration) - -granter = address.MustLengthPrefix(granter) - -grantee = address.MustLengthPrefix(grantee) - -return sdk.AppendLengthPrefixedBytes(GrantQueuePrefix, exp, granter, grantee) -} - -// GrantQueueTimePrefix - return grant queue time prefix -func GrantQueueTimePrefix(expiration time.Time) []byte { - return append(GrantQueuePrefix, sdk.FormatTimeBytes(expiration)...) -} - -// firstAddressFromGrantStoreKey parses the first address only -func firstAddressFromGrantStoreKey(key []byte) - -sdk.AccAddress { - addrLen := key[0] - return sdk.AccAddress(key[1 : 1+addrLen]) -} -``` - The `GrantQueueItem` object contains the list of type urls between granter and grantee that expire at the time indicated in the key. ## Messages @@ -1106,7 +523,7 @@ An authorization grant is created using the `MsgGrant` message. If there is already a grant for the `(granter, grantee, Authorization)` triple, then the new grant overwrites the previous one. To update or extend an existing grant, a new grant with the same `(granter, grantee, Authorization)` triple should be created. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/authz/v1beta1/tx.proto#L35-L45 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/authz/v1beta1/tx.proto#L34-L44 ``` The message handling should fail if: @@ -1121,7 +538,7 @@ The message handling should fail if: A grant can be removed with the `MsgRevoke` message. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/authz/v1beta1/tx.proto#L69-L78 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/authz/v1beta1/tx.proto#L68-L77 ``` The message handling should fail if: @@ -1136,7 +553,7 @@ NOTE: The `MsgExec` message removes a grant if the grant has expired. When a grantee wants to execute a transaction on behalf of a granter, they must send `MsgExec`. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/authz/v1beta1/tx.proto#L52-L63 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/authz/v1beta1/tx.proto#L49-L61 ``` The message handling should fail if: diff --git a/sdk/latest/modules/bank/README.mdx b/sdk/latest/modules/bank/README.mdx index 9b3cdf60d..6e85056a5 100644 --- a/sdk/latest/modules/bank/README.mdx +++ b/sdk/latest/modules/bank/README.mdx @@ -134,7 +134,7 @@ it can be updated with governance or the address with authority. * Params: `0x05 | ProtocolBuffer(Params)` ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/bank/v1beta1/bank.proto#L12-L23 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/bank/v1beta1/bank.proto#L12-L22 ``` ## Keepers @@ -472,7 +472,7 @@ IterateAllBalances(ctx context.Context, cb func(address sdk.AccAddress, coin sdk Send coins from one address to another. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/bank/v1beta1/tx.proto#L38-L53 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/bank/v1beta1/tx.proto#L38-L54 ``` The message will fail under the following conditions: @@ -485,7 +485,7 @@ The message will fail under the following conditions: Send coins from one sender and to a series of different address. If any of the receiving addresses do not correspond to an existing account, a new account is created. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/bank/v1beta1/tx.proto#L58-L69 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/bank/v1beta1/tx.proto#L59-L70 ``` The message will fail under the following conditions: @@ -500,7 +500,7 @@ The message will fail under the following conditions: The `bank` module params can be updated through `MsgUpdateParams`, which can be done using governance proposal. The signer will always be the `gov` module account address. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/bank/v1beta1/tx.proto#L74-L88 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/bank/v1beta1/tx.proto#L75-L88 ``` The message handling can fail if: @@ -512,7 +512,7 @@ The message handling can fail if: Used with the x/gov module to set create/edit SendEnabled entries. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/bank/v1beta1/tx.proto#L96-L117 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/bank/v1beta1/tx.proto#L96-L117 ``` The message will fail under the following conditions: diff --git a/sdk/latest/modules/circuit/README.mdx b/sdk/latest/modules/circuit/README.mdx index 63a3723cb..26b515d67 100644 --- a/sdk/latest/modules/circuit/README.mdx +++ b/sdk/latest/modules/circuit/README.mdx @@ -3,7 +3,7 @@ title: 'x/circuit' --- -`x/circuit` has been moved to [`./contrib/x/circuit`](https://github.com/cosmos/cosmos-sdk/tree/main/contrib/x/circuit) and is no longer actively maintained as part of the core Cosmos SDK. It is still available for use but is not included in the SDK Bug Bounty program. It was moved because it was never widely adopted. +`x/circuit` has been moved to [`./contrib/x/circuit`](https://github.com/cosmos/cosmos-sdk/tree/release/v0.55.x/contrib/x/circuit) and is no longer actively maintained as part of the core Cosmos SDK. It is still available for use but is not included in the SDK Bug Bounty program. It was moved because it was never widely adopted. ## Concepts @@ -439,7 +439,7 @@ Reset is called by an authorized account to enable execution for a specific msgU ### MsgAuthorizeCircuitBreaker ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/main/proto/cosmos/circuit/v1/tx.proto#L25-L75 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/contrib/proto/circuit/v1/tx.proto#L25-L40 ``` This message is expected to fail if: @@ -449,7 +449,7 @@ This message is expected to fail if: ### MsgTripCircuitBreaker ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/main/proto/cosmos/circuit/v1/tx.proto#L77-L93 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/contrib/proto/circuit/v1/tx.proto#L47-L60 ``` This message is expected to fail if: @@ -459,7 +459,7 @@ This message is expected to fail if: ### MsgResetCircuitBreaker ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/main/proto/cosmos/circuit/v1/tx.proto#L95-109 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/contrib/proto/circuit/v1/tx.proto#L67-L78 ``` This message is expected to fail if: diff --git a/sdk/latest/modules/crisis/README.mdx b/sdk/latest/modules/crisis/README.mdx index f22f9c532..e7918af20 100644 --- a/sdk/latest/modules/crisis/README.mdx +++ b/sdk/latest/modules/crisis/README.mdx @@ -5,7 +5,7 @@ description: >- --- -`x/crisis` has been moved to [`./contrib/x/crisis`](https://github.com/cosmos/cosmos-sdk/tree/main/contrib/x/crisis) and is no longer actively maintained as part of the core Cosmos SDK. It is still available for use but is not included in the SDK Bug Bounty program. The module was moved because it never worked as intended. +`x/crisis` has been moved to [`./contrib/x/crisis`](https://github.com/cosmos/cosmos-sdk/tree/release/v0.55.x/contrib/x/crisis) and is no longer actively maintained as part of the core Cosmos SDK. It is still available for use but is not included in the SDK Bug Bounty program. The module was moved because it never worked as intended. ## Overview @@ -36,7 +36,7 @@ with the standard gas consumption method. The ConstantFee param is stored in the module params state with the prefix of `0x01`, it can be updated with governance or the address with authority. -* Params: `mint/params -> legacy_amino(sdk.Coin)` +* ConstantFee: `0x01 -> ProtocolBuffer(Coin)` ## Messages @@ -48,7 +48,7 @@ corresponding updates to the state. Blockchain invariants can be checked using the `MsgVerifyInvariant` message. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/crisis/v1beta1/tx.proto#L26-L42 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/contrib/proto/crisis/v1beta1/tx.proto#L26-L42 ``` This message is expected to fail if: diff --git a/sdk/latest/modules/distribution/README.mdx b/sdk/latest/modules/distribution/README.mdx index 7a01830b3..7c3ce19f6 100644 --- a/sdk/latest/modules/distribution/README.mdx +++ b/sdk/latest/modules/distribution/README.mdx @@ -98,7 +98,7 @@ In Proof of Stake (PoS) blockchains, rewards gained from transaction fees are pa Rewards are calculated per period. The period is updated each time a validator's delegation changes, for example, when the validator receives a new delegation. The rewards for a single validator can then be calculated by taking the total rewards for the period before the delegation started, minus the current total rewards. -To learn more, see the [F1 Fee Distribution paper](https://github.com/cosmos/cosmos-sdk/tree/main/docs/spec/fee_distribution/f1_fee_distr.pdf). +To learn more, see the [F1 Fee Distribution paper](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/docs/spec/fee_distribution/f1_fee_distr.pdf). The commission to the validator is paid when the validator is removed or when the validator requests a withdrawal. The commission is calculated and incremented at every `BeginBlock` operation to update accumulated fee amounts. @@ -123,35 +123,6 @@ is created which might need to reference the historical record, the reference co Each time one object which previously needed to reference the historical record is deleted, the reference count is decremented. If the reference count hits zero, the historical record is deleted. -### External Community Pool Keepers - -An external pool community keeper is defined as: - -```go expandable -// ExternalCommunityPoolKeeper is the interface that an external community pool module keeper must fulfill -// for x/distribution to properly accept it as a community pool fund destination. -type ExternalCommunityPoolKeeper interface { - // GetCommunityPoolModule gets the module name that funds should be sent to for the community pool. - // This is the address that x/distribution will send funds to for external management. - GetCommunityPoolModule() - -string - // FundCommunityPool allows an account to directly fund the community fund pool. - FundCommunityPool(ctx sdk.Context, amount sdk.Coins, senderAddr sdk.AccAddress) - -error - // DistributeFromCommunityPool distributes funds from the community pool module account to - // a receiver address. - DistributeFromCommunityPool(ctx sdk.Context, amount sdk.Coins, receiveAddr sdk.AccAddress) - -error -} -``` - -By default, the distribution module will use a community pool implementation that is internal. An external community pool -can be provided to the module which will have funds be diverted to it instead of the internal implementation. The reference -external community pool maintained by the Cosmos SDK is [`x/protocolpool`](/sdk/latest/modules/protocolpool/README). - ## State ### FeePool @@ -178,7 +149,7 @@ type DecCoin struct { ``` ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/distribution/v1beta1/distribution.proto#L116-L123 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/distribution/v1beta1/distribution.proto#L117-L124 ``` ### Validator Distribution @@ -223,7 +194,7 @@ it can be updated with governance or the address with authority. * Params: `0x09 | ProtocolBuffer(Params)` ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/distribution/v1beta1/distribution.proto#L12-L42 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/distribution/v1beta1/distribution.proto#L12-L44 ``` ## Begin Block @@ -254,61 +225,6 @@ The community pool gets `community_tax * fees`, plus any remaining dust after validators get their rewards that are always rounded down to the nearest integer value. -#### Using an External Community Pool - -Starting with Cosmos SDK v0.53.0, an external community pool, such as `x/protocolpool`, can be used in place of the `x/distribution` managed community pool. - -Please view the warning in the next section before deciding to use an external community pool. - -```go expandable -// ExternalCommunityPoolKeeper is the interface that an external community pool module keeper must fulfill -// for x/distribution to properly accept it as a community pool fund destination. -type ExternalCommunityPoolKeeper interface { - // GetCommunityPoolModule gets the module name that funds should be sent to for the community pool. - // This is the address that x/distribution will send funds to for external management. - GetCommunityPoolModule() - -string - // FundCommunityPool allows an account to directly fund the community fund pool. - FundCommunityPool(ctx sdk.Context, amount sdk.Coins, senderAddr sdk.AccAddress) - -error - // DistributeFromCommunityPool distributes funds from the community pool module account to - // a receiver address. - DistributeFromCommunityPool(ctx sdk.Context, amount sdk.Coins, receiveAddr sdk.AccAddress) - -error -} -``` - -```go -app.DistrKeeper = distrkeeper.NewKeeper( - appCodec, - runtime.NewKVStoreService(keys[distrtypes.StoreKey]), - app.AccountKeeper, - app.BankKeeper, - app.StakingKeeper, - authtypes.FeeCollectorName, - authtypes.NewModuleAddress(govtypes.ModuleName).String(), - distrkeeper.WithExternalCommunityPool(app.ProtocolPoolKeeper), // New option. -) -``` - -#### External Community Pool Usage Warning - -When using an external community pool with `x/distribution`, the following handlers will return an error: - -**QueryService** - -* `CommunityPool` - -**MsgService** - -* `CommunityPoolSpend` -* `FundCommunityPool` - -If you have services that rely on this functionality from `x/distribution`, please update them to use the `x/protocolpool` equivalents. - #### Reward To the Validators The proposer receives no extra rewards. All fees are distributed among all the @@ -359,30 +275,12 @@ community tax rate) * (1 - validator commission rate) By default, the withdraw address is the delegator address. To change its withdraw address, a delegator must send a `MsgSetWithdrawAddress` message. Changing the withdraw address is possible only if the parameter `WithdrawAddrEnabled` is set to `true`. -The withdraw address cannot be any of the module accounts. These accounts are blocked from being withdraw addresses by being added to the distribution keeper's `blockedAddrs` array at initialization. +The withdraw address cannot be any of the module accounts. The distribution keeper does not track these itself; it asks the bank keeper through `BlockedAddr`, so the blocked set is the one bank maintains. -Response: +A blocked withdraw address is handled differently depending on how the withdrawal is triggered. A withdrawal triggered by a user message fails with `ErrUnauthorized`. An automatic withdrawal during `BeginBlock` or `EndBlock` does not fail. Instead, the funds fall back to the owner's own address. The owner is the delegator for rewards, or the validator for commission. If the owner's address is also blocked, the funds go to the community pool. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/distribution/v1beta1/tx.proto#L49-L60 -``` - -```go -func (k Keeper) - -SetWithdrawAddr(ctx context.Context, delegatorAddr sdk.AccAddress, withdrawAddr sdk.AccAddress) - -error - if k.blockedAddrs[withdrawAddr.String()] { - fail with "`{ - withdrawAddr -}` is not allowed to receive external funds" -} - if !k.GetWithdrawAddrEnabled(ctx) { - fail with `ErrSetWithdrawAddrDisabled` -} - -k.SetDelegatorWithdrawAddr(ctx, delegatorAddr, withdrawAddr) +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/distribution/v1beta1/tx.proto#L55-L66 ``` ### MsgWithdrawDelegatorReward @@ -419,10 +317,8 @@ rewards = rewards + (R(B) - R(PN)) * stake The historical rewards are calculated retroactively by playing back all the slashes and then attenuating the delegator's stake at each step. The final calculated stake is equivalent to the actual staked coins in the delegation with a margin of error due to rounding errors. -Response: - ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/distribution/v1beta1/tx.proto#L66-L77 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/distribution/v1beta1/tx.proto#L72-L83 ``` ### WithdrawValidatorCommission @@ -434,12 +330,6 @@ Only integer amounts can be sent. If the accumulated awards have decimals, the a ### FundCommunityPool - - -This handler will return an error if an `ExternalCommunityPool` is used. - - - This message sends coins directly from the sender to the community pool. The transaction fails if the amount cannot be transferred from the sender to the distribution module account. @@ -504,7 +394,7 @@ k.SetDelegatorStartingInfo(ctx, val, del, types.NewDelegatorStartingInfo(previou Distribution module params can be updated through `MsgUpdateParams`, which can be done using governance proposal and the signer will always be gov module account address. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/distribution/v1beta1/tx.proto#L133-L147 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/distribution/v1beta1/tx.proto#L142-L155 ``` The message handling can fail if: @@ -1300,4 +1190,3 @@ Example Output: } } ``` -```` diff --git a/sdk/latest/modules/evidence/README.mdx b/sdk/latest/modules/evidence/README.mdx index e481de12f..78da2b29c 100644 --- a/sdk/latest/modules/evidence/README.mdx +++ b/sdk/latest/modules/evidence/README.mdx @@ -16,7 +16,7 @@ description: Concepts State Messages Events Parameters BeginBlock Client CLI RES ## Abstract -`x/evidence` is an implementation of a Cosmos SDK module, per [ADR 009](https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-009-evidence-module.md), +`x/evidence` is an implementation of a Cosmos SDK module, per [ADR 009](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/docs/architecture/adr-009-evidence-module.md), that allows for the submission and handling of arbitrary evidence of misbehavior such as equivocation and counterfactual signing. @@ -228,7 +228,7 @@ The evidence module does not contain any parameters. ### Evidence Handling CometBFT blocks can include -[Evidence](https://github.com/cometbft/cometbft/blob/main/spec/abci/abci%2B%2B_basic_concepts.md#evidence) that indicates if a validator committed malicious behavior. The relevant information is forwarded to the application as ABCI Evidence in `abci.RequestBeginBlock` so that the validator can be punished accordingly. +[Evidence](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/abci/abci%2B%2B_basic_concepts.md#evidence) that indicates if a validator committed malicious behavior. The relevant information is forwarded to the application as ABCI Evidence in `abci.RequestBeginBlock` so that the validator can be punished accordingly. #### Equivocation @@ -240,7 +240,7 @@ The Cosmos SDK handles two types of evidence inside the ABCI `BeginBlock`: The evidence module handles these two evidence types the same way. First, the Cosmos SDK converts the CometBFT concrete evidence type to an SDK `Evidence` interface using `Equivocation` as the concrete type. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/proto/cosmos/evidence/v1beta1/evidence.proto#L12-L32 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/evidence/v1beta1/evidence.proto#L12-L31 ``` For some `Equivocation` submitted in `block` to be valid, it must satisfy: @@ -264,7 +264,7 @@ validator to ever re-enter the validator set. The `Equivocation` evidence is handled as follows: ```go -// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/x/evidence/keeper/infraction.go#L26-L140 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/x/evidence/keeper/infraction.go#L13-L155 ``` **Note:** The slashing, jailing, and tombstoning calls are delegated through the `x/slashing` module diff --git a/sdk/latest/modules/feegrant/README.mdx b/sdk/latest/modules/feegrant/README.mdx index 5fd772778..547c2382e 100644 --- a/sdk/latest/modules/feegrant/README.mdx +++ b/sdk/latest/modules/feegrant/README.mdx @@ -7,7 +7,7 @@ description: >- ## Abstract -This document specifies the fee grant module. For the full ADR, please see [Fee Grant ADR-029](https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-029-fee-grant-module.md). +This document specifies the fee grant module. For the full ADR, please see [Fee Grant ADR-029](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/docs/architecture/adr-029-fee-grant-module.md). This module allows accounts to grant fee allowances and to use fees from their accounts. Grantees can execute any transaction without the need to maintain sufficient fees. @@ -33,10 +33,10 @@ This module allows accounts to grant fee allowances and to use fees from their a ### Grant -`Grant` is stored in the KVStore to record a grant with full context. Every grant will contain `granter`, `grantee` and what kind of `allowance` is granted. `granter` is an account address who is giving permission to `grantee` (the beneficiary account address) to pay for some or all of `grantee`'s transaction fees. `allowance` defines what kind of fee allowance (`BasicAllowance` or `PeriodicAllowance`, see below) is granted to `grantee`. `allowance` accepts an interface which implements `FeeAllowanceI`, encoded as `Any` type. There can be only one existing fee grant allowed for a `grantee` and `granter`, self grants are not allowed. +`Grant` is stored in the KVStore to record a grant with full context. Every grant will contain `granter`, `grantee` and what kind of `allowance` is granted. `granter` is an account address who is giving permission to `grantee` (the beneficiary account address) to pay for some or all of `grantee`'s transaction fees. `allowance` defines what kind of fee allowance is granted to `grantee`. `allowance` accepts an interface which implements `FeeAllowanceI`, encoded as `Any` type. There can be only one existing fee grant allowed for a `grantee` and `granter`, self grants are not allowed. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/feegrant/v1beta1/feegrant.proto#L83-L93 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/feegrant/v1beta1/feegrant.proto#L85-L95 ``` `FeeAllowanceI` looks like: @@ -80,7 +80,7 @@ error ### Fee Allowance types -There are two types of fee allowances present at the moment: +There are three types of fee allowances: * `BasicAllowance` * `PeriodicAllowance` @@ -91,7 +91,7 @@ There are two types of fee allowances present at the moment: `BasicAllowance` is permission for `grantee` to use fee from a `granter`'s account. If any of the `spend_limit` or `expiration` reaches its limit, the grant will be removed from the state. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/feegrant/v1beta1/feegrant.proto#L15-L28 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/feegrant/v1beta1/feegrant.proto#L14-L32 ``` * `spend_limit` is the limit of coins that are allowed to be used from the `granter` account. If it is empty, it assumes there's no spend limit, `grantee` can use any number of available coins from `granter` account address before the expiration. @@ -105,7 +105,7 @@ There are two types of fee allowances present at the moment: `PeriodicAllowance` is a repeating fee allowance for the mentioned period, we can mention when the grant can expire as well as when a period can reset. We can also define the maximum number of coins that can be used in a mentioned period of time. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/feegrant/v1beta1/feegrant.proto#L34-L68 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/feegrant/v1beta1/feegrant.proto#L34-L70 ``` * `basic` is the instance of `BasicAllowance` which is optional for periodic fee allowance. If empty, the grant will have no `expiration` and no `spend_limit`. @@ -123,7 +123,7 @@ There are two types of fee allowances present at the moment: `AllowedMsgAllowance` is a fee allowance, it can be any of `BasicFeeAllowance`, `PeriodicAllowance` but restricted only to the allowed messages mentioned by the granter. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/feegrant/v1beta1/feegrant.proto#L70-L81 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/feegrant/v1beta1/feegrant.proto#L72-L83 ``` * `allowance` is either `BasicAllowance` or `PeriodicAllowance`. @@ -1586,7 +1586,7 @@ return nil ``` ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/tx/v1beta1/tx.proto#L203-L224 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/tx/v1beta1/tx.proto#L221-L248 ``` Example cmd: @@ -3409,7 +3409,7 @@ Fee allowance queue keys are stored in the state as follows: A fee allowance grant will be created with the `MsgGrantAllowance` message. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/feegrant/v1beta1/tx.proto#L25-L39 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/feegrant/v1beta1/tx.proto#L29-L43 ``` ### Msg/RevokeAllowance @@ -3417,7 +3417,7 @@ A fee allowance grant will be created with the `MsgGrantAllowance` message. An allowed grant fee allowance can be removed with the `MsgRevokeAllowance` message. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/feegrant/v1beta1/tx.proto#L41-L54 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/feegrant/v1beta1/tx.proto#L48-L58 ``` ## Events diff --git a/sdk/latest/modules/gov/README.mdx b/sdk/latest/modules/gov/README.mdx index ae8fc6b25..6926c3059 100644 --- a/sdk/latest/modules/gov/README.mdx +++ b/sdk/latest/modules/gov/README.mdx @@ -37,8 +37,8 @@ staking token of the chain. * [Proposal submission](#proposal-submission) * [Deposit](#deposit) * [Vote](#vote) - * [Software Upgrade](#software-upgrade) * [State](#state) + * [Constitution](#constitution) * [Proposals](#proposals) * [Parameters and base types](#parameters-and-base-types) * [Deposit](#deposit-1) @@ -171,18 +171,18 @@ proposal but accept the result of the vote. #### Weighted Votes -[ADR-037](https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-037-gov-split-vote.md) introduces the weighted vote feature which allows a staker to split their votes into several voting options. For example, it could use 70% of its voting power to vote Yes and 30% of its voting power to vote No. +[ADR-037](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/docs/architecture/adr-037-gov-split-vote.md) introduces the weighted vote feature which allows a staker to split their votes into several voting options. For example, it could use 70% of its voting power to vote Yes and 30% of its voting power to vote No. Often times the entity owning that address might not be a single individual. For example, a company might have different stakeholders who want to vote differently, and so it makes sense to allow them to split their voting power. Currently, it is not possible for them to do "passthrough voting" and giving their users voting rights over their tokens. However, with this system, exchanges can poll their users for voting preferences, and then vote on-chain proportionally to the results of the poll. To represent weighted vote on chain, we use the following Protobuf message. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/gov/v1beta1/gov.proto#L34-L47 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/gov/v1beta1/gov.proto#L32-L45 ``` ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/gov/v1beta1/gov.proto#L181-L201 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/gov/v1beta1/gov.proto#L180-L198 ``` For a weighted vote to be valid, the `options` field must not contain duplicate vote options, and the sum of weights of all options must be equal to 1. @@ -466,7 +466,7 @@ Threshold is defined as the minimum proportion of `Yes` votes (excluding Initially, the threshold is set at 50% of `Yes` votes, excluding `Abstain` votes. A possibility to veto exists if more than 1/3rd of all votes are -`NoWithVeto` votes. Note, both of these values are derived from the `TallyParams` +`NoWithVeto` votes. Note, both of these values are derived from the `Params` on-chain parameter, which is modifiable by governance. This means that proposals are accepted iff: @@ -553,7 +553,7 @@ unique id and contains a series of timestamps: `submit_time`, `deposit_end_time` `voting_start_time`, `voting_end_time` which track the lifecycle of a proposal ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/gov/v1/gov.proto#L51-L99 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/gov/v1/gov.proto#L50-L101 ``` A proposal will generally require more than just a set of messages to explain its @@ -598,26 +598,12 @@ be one active parameter set at any given time. If governance wants to change a parameter set, either to modify a value or add/remove a parameter field, a new parameter set has to be created and the previous one rendered inactive. -#### DepositParams - -```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/gov/v1/gov.proto#L152-L162 -``` - -#### VotingParams - -```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/gov/v1/gov.proto#L164-L168 -``` - -#### TallyParams +#### Params ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/gov/v1/gov.proto#L170-L182 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/gov/v1/gov.proto#L193-L255 ``` -Parameters are stored in a global `GlobalParams` KVStore. - Additionally, we introduce some basic types: ```go expandable @@ -652,7 +638,7 @@ const ( ### Deposit ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/gov/v1/gov.proto#L38-L49 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/gov/v1/gov.proto#L37-L48 ``` ### ValidatorGovInfo @@ -733,7 +719,7 @@ in voterIterator tmpValMap(voterAddress).Vote = vote - tallyingParam = load(GlobalParams, 'TallyingParam') + tallyingParam = load(Params, 'TallyingParam') // Update tally if validator voted for each validator in validators @@ -785,7 +771,7 @@ More information on how to submit proposals in the [client section](#client). Proposals can be submitted by any account via a `MsgSubmitProposal` transaction. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/gov/v1/tx.proto#L42-L69 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/gov/v1/tx.proto#L57-L88 ``` All `sdk.Msgs` passed into the `messages` field of a `MsgSubmitProposal` message @@ -816,7 +802,7 @@ A deposit is accepted iff: * The deposited coins are conform to the accepted denom from the `MinDeposit` param ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/gov/v1/tx.proto#L134-L147 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/gov/v1/tx.proto#L153-L166 ``` **State modifications:** @@ -835,7 +821,7 @@ bonded Atom holders are able to send `MsgVote` transactions to cast their vote on the proposal. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/gov/v1/tx.proto#L92-L108 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/gov/v1/tx.proto#L111-L127 ``` **State modifications:** diff --git a/sdk/latest/modules/group/README.mdx b/sdk/latest/modules/group/README.mdx index 301552b62..79e6c95fc 100644 --- a/sdk/latest/modules/group/README.mdx +++ b/sdk/latest/modules/group/README.mdx @@ -104,7 +104,7 @@ custom decision policies, as long as they adhere to the `DecisionPolicy` interface: ```go -// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/x/group/types.go#L27-L45 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/group/x/group/types.go#L44-L62 ``` #### Threshold decision policy @@ -340,7 +340,7 @@ The metadata has a maximum length that is chosen by the app developer, and passed into the group keeper as a config. ```go -// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/proto/cosmos/group/v1/tx.proto#L67-L80 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/group/proto/cosmos/group/v1/tx.proto#L80-L93 ``` It's expected to fail if @@ -353,7 +353,7 @@ It's expected to fail if Group members can be updated with the `UpdateGroupMembers`. ```go -// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/proto/cosmos/group/v1/tx.proto#L88-L102 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/group/proto/cosmos/group/v1/tx.proto#L101-L115 ``` In the list of `MemberUpdates`, an existing member can be removed by setting its weight to 0. @@ -368,7 +368,7 @@ It's expected to fail if: The `UpdateGroupAdmin` can be used to update a group admin. ```go -// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/proto/cosmos/group/v1/tx.proto#L107-L120 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/group/proto/cosmos/group/v1/tx.proto#L120-L133 ``` It's expected to fail if the signer is not the admin of the group. @@ -378,7 +378,7 @@ It's expected to fail if the signer is not the admin of the group. The `UpdateGroupMetadata` can be used to update a group metadata. ```go -// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/proto/cosmos/group/v1/tx.proto#L125-L138 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/group/proto/cosmos/group/v1/tx.proto#L138-L151 ``` It's expected to fail if: @@ -391,7 +391,7 @@ It's expected to fail if: A new group policy can be created with the `MsgCreateGroupPolicy`, which has an admin address, a group id, a decision policy and some optional metadata. ```go -// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/proto/cosmos/group/v1/tx.proto#L147-L165 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/group/proto/cosmos/group/v1/tx.proto#L160-L178 ``` It's expected to fail if: @@ -405,7 +405,7 @@ It's expected to fail if: A new group with policy can be created with the `MsgCreateGroupWithPolicy`, which has an admin address, a list of members, a decision policy, a `group_policy_as_admin` field to optionally set group and group policy admin with group policy address and some optional metadata for group and group policy. ```go -// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/proto/cosmos/group/v1/tx.proto#L191-L215 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/group/proto/cosmos/group/v1/tx.proto#L204-L228 ``` It's expected to fail for the same reasons as `Msg/CreateGroup` and `Msg/CreateGroupPolicy`. @@ -415,7 +415,7 @@ It's expected to fail for the same reasons as `Msg/CreateGroup` and `Msg/CreateG The `UpdateGroupPolicyAdmin` can be used to update a group policy admin. ```go -// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/proto/cosmos/group/v1/tx.proto#L173-L186 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/group/proto/cosmos/group/v1/tx.proto#L186-L199 ``` It's expected to fail if the signer is not the admin of the group policy. @@ -425,7 +425,7 @@ It's expected to fail if the signer is not the admin of the group policy. The `UpdateGroupPolicyDecisionPolicy` can be used to update a decision policy. ```go -// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/proto/cosmos/group/v1/tx.proto#L226-L241 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/group/proto/cosmos/group/v1/tx.proto#L239-L254 ``` It's expected to fail if: @@ -438,7 +438,7 @@ It's expected to fail if: The `UpdateGroupPolicyMetadata` can be used to update a group policy metadata. ```go -// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/proto/cosmos/group/v1/tx.proto#L246-L259 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/group/proto/cosmos/group/v1/tx.proto#L259-L272 ``` It's expected to fail if: @@ -452,7 +452,7 @@ A new proposal can be created with the `MsgSubmitProposal`, which has a group po An optional `Exec` value can be provided to try to execute the proposal immediately after proposal creation. Proposers signatures are considered as yes votes in this case. ```go -// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/proto/cosmos/group/v1/tx.proto#L281-L315 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/group/proto/cosmos/group/v1/tx.proto#L294-L324 ``` It's expected to fail if: @@ -465,7 +465,7 @@ It's expected to fail if: A proposal can be withdrawn using `MsgWithdrawProposal` which has an `address` (can be either a proposer or the group policy admin) and a `proposal_id` (which has to be withdrawn). ```go -// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/proto/cosmos/group/v1/tx.proto#L323-L333 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/group/proto/cosmos/group/v1/tx.proto#L332-L342 ``` It's expected to fail if: @@ -479,7 +479,7 @@ A new vote can be created with the `MsgVote`, given a proposal id, a voter addre An optional `Exec` value can be provided to try to execute the proposal immediately after voting. ```go -// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/proto/cosmos/group/v1/tx.proto#L338-L358 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/group/proto/cosmos/group/v1/tx.proto#L347-L367 ``` It's expected to fail if: @@ -492,7 +492,7 @@ It's expected to fail if: A proposal can be executed with the `MsgExec`. ```go -// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/proto/cosmos/group/v1/tx.proto#L363-L373 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/group/proto/cosmos/group/v1/tx.proto#L372-L382 ``` The messages that are part of this proposal won't be executed if: @@ -505,7 +505,7 @@ The messages that are part of this proposal won't be executed if: The `MsgLeaveGroup` allows group member to leave a group. ```go -// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/proto/cosmos/group/v1/tx.proto#L381-L391 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/group/proto/cosmos/group/v1/tx.proto#L390-L400 ``` It's expected to fail if: diff --git a/sdk/latest/modules/mint/README.mdx b/sdk/latest/modules/mint/README.mdx index 232b0fd5e..a41048d6b 100644 --- a/sdk/latest/modules/mint/README.mdx +++ b/sdk/latest/modules/mint/README.mdx @@ -143,7 +143,7 @@ The minter is a space for holding current inflation information. * Minter: `0x00 -> ProtocolBuffer(minter)` ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/mint/v1beta1/mint.proto#L10-L24 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/mint/v1beta1/mint.proto#L10-L24 ``` ### Params @@ -153,10 +153,10 @@ it can be updated with governance or the address with authority. **Note:** The `MaxSupply` parameter controls the maximum supply of tokens the module can mint. A value of `0` indicates an unlimited supply. -* Params: `mint/params -> legacy_amino(params)` +* Params: `0x01 -> ProtocolBuffer(Params)` ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/mint/v1beta1/mint.proto#L26-L59 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/mint/v1beta1/mint.proto#L26-L71 ``` ## Begin-Block diff --git a/sdk/latest/modules/modules.mdx b/sdk/latest/modules/modules.mdx index f432ac7b2..0ca0039cd 100644 --- a/sdk/latest/modules/modules.mdx +++ b/sdk/latest/modules/modules.mdx @@ -37,12 +37,11 @@ capabilities of your blockchain or further specialize it. * [Feegrant](/sdk/latest/modules/feegrant/README) - Grant fee allowances for executing transactions. * [Group](/sdk/latest/modules/group/README) - Allows for the creation and management of on-chain multisig accounts. * [NFT](/sdk/latest/modules/nft/README) - NFT module implemented based on [ADR43](/sdk/latest/reference/architecture/adr-043-nft-module). -* [ProtocolPool](/sdk/latest/modules/protocolpool/README) - Extended management of community pool functionality. ## Deprecated Modules The following modules are deprecated. They will no longer be maintained and eventually will be removed -in an upcoming release of the Cosmos SDK per our [release process](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/RELEASE_PROCESS.md). +in an upcoming release of the Cosmos SDK per the [release family lifecycle](/sdk/latest/release-family). * [Crisis](/sdk/latest/modules/crisis/README) - *Deprecated* halting the blockchain under certain circumstances (e.g. if an invariant is broken). * [Params](/sdk/latest/modules/params/README) - *Deprecated* Globally available parameter store. diff --git a/sdk/latest/modules/nft/README.mdx b/sdk/latest/modules/nft/README.mdx index b116985fd..6da9ba241 100644 --- a/sdk/latest/modules/nft/README.mdx +++ b/sdk/latest/modules/nft/README.mdx @@ -4,14 +4,14 @@ description: '## Abstract' --- -`x/nft` has been moved to [`./contrib/x/nft`](https://github.com/cosmos/cosmos-sdk/tree/main/contrib/x/nft) and is no longer actively maintained as part of the core Cosmos SDK. It is still available for use but is not included in the SDK Bug Bounty program. It was moved because it was never widely adopted. +`x/nft` has been moved to [`./contrib/x/nft`](https://github.com/cosmos/cosmos-sdk/tree/release/v0.55.x/contrib/x/nft) and is no longer actively maintained as part of the core Cosmos SDK. It is still available for use but is not included in the SDK Bug Bounty program. It was moved because it was never widely adopted. ## Contents ## Abstract -`x/nft` is an implementation of a Cosmos SDK module, per [ADR 43](https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-043-nft-module.md), that allows you to create nft classification, create nft, transfer nft, update nft, and support various queries by integrating the module. It is fully compatible with the ERC721 specification. +`x/nft` is an implementation of a Cosmos SDK module, per [ADR 43](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/docs/architecture/adr-043-nft-module.md), that allows you to create nft classification, create nft, transfer nft, update nft, and support various queries by integrating the module. It is fully compatible with the ERC721 specification. * [Concepts](#concepts) * [Class](#class) @@ -30,7 +30,7 @@ description: '## Abstract' ### Class -`x/nft` module defines a struct `Class` to describe the common characteristics of a class of nft, under this class, you can create a variety of nft, which is equivalent to an erc721 contract for Ethereum. The design is defined in the [ADR 043](https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-043-nft-module.md). +`x/nft` module defines a struct `Class` to describe the common characteristics of a class of nft, under this class, you can create a variety of nft, which is equivalent to an erc721 contract for Ethereum. The design is defined in the [ADR 043](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/docs/architecture/adr-043-nft-module.md). ### NFT diff --git a/sdk/latest/modules/protocolpool/README.mdx b/sdk/latest/modules/protocolpool/README.mdx deleted file mode 100644 index a967d238e..000000000 --- a/sdk/latest/modules/protocolpool/README.mdx +++ /dev/null @@ -1,657 +0,0 @@ ---- -title: 'x/protocolpool' ---- - -## Concepts - -`x/protocolpool` is a supplemental Cosmos SDK module that handles functionality for community pool funds. The module provides a separate module account for the community pool making it easier to track the pool assets. Starting with v0.53 of the Cosmos SDK, community funds can be tracked using this module instead of the `x/distribution` module. Funds are migrated from the `x/distribution` module's community pool to `x/protocolpool`'s module account automatically. - -This module is `supplemental`; it is not required to run a Cosmos SDK chain. `x/protocolpool` enhances the community pool functionality provided by `x/distribution` and enables custom modules to further extend the community pool. - -Note: *as long as an external commmunity pool keeper (here, `x/protocolpool`) is wired in DI configs, `x/distribution` will automatically use it for its external pool.* - -## Usage Limitations - -The following `x/distribution` handlers will now return an error when the `protocolpool` module is used with `x/distribution`: - -**QueryService** - -* `CommunityPool` - -**MsgService** - -* `CommunityPoolSpend` -* `FundCommunityPool` - -If you have services that rely on this functionality from `x/distribution`, please update them to use the `x/protocolpool` equivalents. - -## State Transitions - -### FundCommunityPool - -FundCommunityPool can be called by any valid account to send funds to the `x/protocolpool` module account. - -```protobuf - // FundCommunityPool defines a method to allow an account to directly - // fund the community pool. - rpc FundCommunityPool(MsgFundCommunityPool) returns (MsgFundCommunityPoolResponse); -``` - -### CommunityPoolSpend - -CommunityPoolSpend can be called by the module authority (default governance module account) or any account with authorization to spend funds from the `x/protocolpool` module account to a receiver address. - -```protobuf - // CommunityPoolSpend defines a governance operation for sending tokens from - // the community pool in the x/protocolpool module to another account, which - // could be the governance module itself. The authority is defined in the - // keeper. - rpc CommunityPoolSpend(MsgCommunityPoolSpend) returns (MsgCommunityPoolSpendResponse); -``` - -### CreateContinuousFund - -CreateContinuousFund is a message used to initiate a continuous fund for a specific recipient. The proposed percentage of funds will be distributed only on withdraw request for the recipient. The fund distribution continues until expiry time is reached or continuous fund request is canceled. -NOTE: This feature is designed to work with the SDK's default bond denom. - -```protobuf - // CreateContinuousFund defines a method to distribute a percentage of funds to an address continuously. - // This ContinuousFund can be indefinite or run until a given expiry time. - // Funds come from validator block rewards from x/distribution, but may also come from - // any user who funds the ProtocolPoolEscrow module account directly through x/bank. - rpc CreateContinuousFund(MsgCreateContinuousFund) returns (MsgCreateContinuousFundResponse); -``` - -### CancelContinuousFund - -CancelContinuousFund is a message used to cancel an existing continuous fund proposal for a specific recipient. Cancelling a continuous fund stops further distribution of funds, and the state object is removed from storage. - -```protobuf - // CancelContinuousFund defines a method for cancelling continuous fund. - rpc CancelContinuousFund(MsgCancelContinuousFund) returns (MsgCancelContinuousFundResponse); -``` - -## Messages - -### MsgFundCommunityPool - -This message sends coins directly from the sender to the community pool. - - -If you know the `x/protocolpool` module account address, you can directly use bank `send` transaction instead. - - -```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/proto/cosmos/protocolpool/v1/tx.proto#L43-L53 -``` - -* The msg will fail if the amount cannot be transferred from the sender to the `x/protocolpool` module account. - -```go -func (k Keeper) - -FundCommunityPool(ctx context.Context, amount sdk.Coins, sender sdk.AccAddress) - -error { - return k.bankKeeper.SendCoinsFromAccountToModule(ctx, sender, types.ModuleName, amount) -} -``` - -### MsgCommunityPoolSpend - -This message distributes funds from the `x/protocolpool` module account to the recipient using `DistributeFromCommunityPool` keeper method. - -```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/proto/cosmos/protocolpool/v1/tx.proto#L58-L69 -``` - -The message will fail under the following conditions: - -* The amount cannot be transferred to the recipient from the `x/protocolpool` module account. -* The `recipient` address is restricted - -```go -func (k Keeper) - -DistributeFromCommunityPool(ctx context.Context, amount sdk.Coins, receiveAddr sdk.AccAddress) - -error { - return k.bankKeeper.SendCoinsFromModuleToAccount(ctx, types.ModuleName, receiveAddr, amount) -} -``` - -### MsgCreateContinuousFund - -This message is used to create a continuous fund for a specific recipient. The proposed percentage of funds will be distributed only on withdraw request for the recipient. This fund distribution continues until expiry time is reached or continuous fund request is canceled. - -```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/proto/cosmos/protocolpool/v1/tx.proto#L114-L130 -``` - -The message will fail under the following conditions: - -* The recipient address is empty or restricted. -* The percentage is zero/negative/greater than one. -* The Expiry time is less than the current block time. - - -If two continuous fund proposals to the same address are created, the previous ContinuousFund will be updated with the new ContinuousFund. - - -```go expandable -package keeper - -import ( - - "context" - "fmt" - "cosmossdk.io/math" - - sdk "github.com/cosmos/cosmos-sdk/types" - sdkerrors "github.com/cosmos/cosmos-sdk/types/errors" - "github.com/cosmos/cosmos-sdk/x/protocolpool/types" -) - -type MsgServer struct { - Keeper -} - -var _ types.MsgServer = MsgServer{ -} - -// NewMsgServerImpl returns an implementation of the protocolpool MsgServer interface -// for the provided Keeper. -func NewMsgServerImpl(keeper Keeper) - -types.MsgServer { - return &MsgServer{ - Keeper: keeper -} -} - -func (k MsgServer) - -FundCommunityPool(ctx context.Context, msg *types.MsgFundCommunityPool) (*types.MsgFundCommunityPoolResponse, error) { - sdkCtx := sdk.UnwrapSDKContext(ctx) - -depositor, err := k.authKeeper.AddressCodec().StringToBytes(msg.Depositor) - if err != nil { - return nil, sdkerrors.ErrInvalidAddress.Wrapf("invalid depositor address: %s", err) -} - if err := validateAmount(msg.Amount); err != nil { - return nil, err -} - - // send funds to community pool module account - if err := k.Keeper.FundCommunityPool(sdkCtx, msg.Amount, depositor); err != nil { - return nil, err -} - -return &types.MsgFundCommunityPoolResponse{ -}, nil -} - -func (k MsgServer) - -CommunityPoolSpend(ctx context.Context, msg *types.MsgCommunityPoolSpend) (*types.MsgCommunityPoolSpendResponse, error) { - sdkCtx := sdk.UnwrapSDKContext(ctx) - if err := k.validateAuthority(msg.Authority); err != nil { - return nil, err -} - if err := validateAmount(msg.Amount); err != nil { - return nil, err -} - -recipient, err := k.authKeeper.AddressCodec().StringToBytes(msg.Recipient) - if err != nil { - return nil, err -} - - // distribute funds from community pool module account - if err := k.DistributeFromCommunityPool(sdkCtx, msg.Amount, recipient); err != nil { - return nil, err -} - -sdkCtx.Logger().Debug("transferred from the community pool", "amount", msg.Amount.String(), "recipient", msg.Recipient) - -return &types.MsgCommunityPoolSpendResponse{ -}, nil -} - -func (k MsgServer) - -CreateContinuousFund(ctx context.Context, msg *types.MsgCreateContinuousFund) (*types.MsgCreateContinuousFundResponse, error) { - sdkCtx := sdk.UnwrapSDKContext(ctx) - if err := k.validateAuthority(msg.Authority); err != nil { - return nil, err -} - -recipient, err := k.Keeper.authKeeper.AddressCodec().StringToBytes(msg.Recipient) - if err != nil { - return nil, err -} - - // deny creation if we know this address is blocked from receiving funds - if k.bankKeeper.BlockedAddr(recipient) { - return nil, fmt.Errorf("recipient is blocked in the bank keeper: %s", msg.Recipient) -} - -has, err := k.ContinuousFunds.Has(sdkCtx, recipient) - if err != nil { - return nil, err -} - if has { - return nil, fmt.Errorf("continuous fund already exists for recipient %s", msg.Recipient) -} - - // Validate the message fields - err = validateContinuousFund(sdkCtx, *msg) - if err != nil { - return nil, err -} - - // Check if total funds percentage exceeds 100% - // If exceeds, we should not setup continuous fund proposal. - totalStreamFundsPercentage := math.LegacyZeroDec() - -err = k.ContinuousFunds.Walk(sdkCtx, nil, func(key sdk.AccAddress, value types.ContinuousFund) (stop bool, err error) { - totalStreamFundsPercentage = totalStreamFundsPercentage.Add(value.Percentage) - -return false, nil -}) - if err != nil { - return nil, err -} - -totalStreamFundsPercentage = totalStreamFundsPercentage.Add(msg.Percentage) - if totalStreamFundsPercentage.GT(math.LegacyOneDec()) { - return nil, fmt.Errorf("cannot set continuous fund proposal\ntotal funds percentage exceeds 100\ncurrent total percentage: %s", totalStreamFundsPercentage.Sub(msg.Percentage).MulInt64(100).TruncateInt().String()) -} - - // Create continuous fund proposal - cf := types.ContinuousFund{ - Recipient: msg.Recipient, - Percentage: msg.Percentage, - Expiry: msg.Expiry, -} - - // Set continuous fund to the state - err = k.ContinuousFunds.Set(sdkCtx, recipient, cf) - if err != nil { - return nil, err -} - -return &types.MsgCreateContinuousFundResponse{ -}, nil -} - -func (k MsgServer) - -CancelContinuousFund(ctx context.Context, msg *types.MsgCancelContinuousFund) (*types.MsgCancelContinuousFundResponse, error) { - sdkCtx := sdk.UnwrapSDKContext(ctx) - if err := k.validateAuthority(msg.Authority); err != nil { - return nil, err -} - -recipient, err := k.Keeper.authKeeper.AddressCodec().StringToBytes(msg.Recipient) - if err != nil { - return nil, err -} - canceledHeight := sdkCtx.BlockHeight() - canceledTime := sdkCtx.BlockTime() - -has, err := k.ContinuousFunds.Has(sdkCtx, recipient) - if err != nil { - return nil, fmt.Errorf("cannot get continuous fund for recipient %w", err) -} - if !has { - return nil, fmt.Errorf("cannot cancel continuous fund for recipient %s - does not exist", msg.Recipient) -} - if err := k.ContinuousFunds.Remove(sdkCtx, recipient); err != nil { - return nil, fmt.Errorf("failed to remove continuous fund for recipient %s: %w", msg.Recipient, err) -} - -return &types.MsgCancelContinuousFundResponse{ - CanceledTime: canceledTime, - CanceledHeight: uint64(canceledHeight), - Recipient: msg.Recipient, -}, nil -} - -func (k MsgServer) - -UpdateParams(ctx context.Context, msg *types.MsgUpdateParams) (*types.MsgUpdateParamsResponse, error) { - sdkCtx := sdk.UnwrapSDKContext(ctx) - if err := k.validateAuthority(msg.GetAuthority()); err != nil { - return nil, err -} - if err := msg.Params.Validate(); err != nil { - return nil, fmt.Errorf("invalid params: %w", err) -} - if err := k.Params.Set(sdkCtx, msg.Params); err != nil { - return nil, fmt.Errorf("failed to set params: %w", err) -} - -return &types.MsgUpdateParamsResponse{ -}, nil -} -``` - -### MsgCancelContinuousFund - -This message is used to cancel an existing continuous fund proposal for a specific recipient. Once canceled, the continuous fund will no longer distribute funds at each begin block, and the state object will be removed. - -```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/proto/cosmos/protocolpool/v1/tx.proto#L118-L129 -``` - -The message will fail under the following conditions: - -* The recipient address is empty or restricted. -* The ContinuousFund for the recipient does not exist. - -```go expandable -package keeper - -import ( - - "context" - "fmt" - "cosmossdk.io/math" - - sdk "github.com/cosmos/cosmos-sdk/types" - sdkerrors "github.com/cosmos/cosmos-sdk/types/errors" - "github.com/cosmos/cosmos-sdk/x/protocolpool/types" -) - -type MsgServer struct { - Keeper -} - -var _ types.MsgServer = MsgServer{ -} - -// NewMsgServerImpl returns an implementation of the protocolpool MsgServer interface -// for the provided Keeper. -func NewMsgServerImpl(keeper Keeper) - -types.MsgServer { - return &MsgServer{ - Keeper: keeper -} -} - -func (k MsgServer) - -FundCommunityPool(ctx context.Context, msg *types.MsgFundCommunityPool) (*types.MsgFundCommunityPoolResponse, error) { - sdkCtx := sdk.UnwrapSDKContext(ctx) - -depositor, err := k.authKeeper.AddressCodec().StringToBytes(msg.Depositor) - if err != nil { - return nil, sdkerrors.ErrInvalidAddress.Wrapf("invalid depositor address: %s", err) -} - if err := validateAmount(msg.Amount); err != nil { - return nil, err -} - - // send funds to community pool module account - if err := k.Keeper.FundCommunityPool(sdkCtx, msg.Amount, depositor); err != nil { - return nil, err -} - -return &types.MsgFundCommunityPoolResponse{ -}, nil -} - -func (k MsgServer) - -CommunityPoolSpend(ctx context.Context, msg *types.MsgCommunityPoolSpend) (*types.MsgCommunityPoolSpendResponse, error) { - sdkCtx := sdk.UnwrapSDKContext(ctx) - if err := k.validateAuthority(msg.Authority); err != nil { - return nil, err -} - if err := validateAmount(msg.Amount); err != nil { - return nil, err -} - -recipient, err := k.authKeeper.AddressCodec().StringToBytes(msg.Recipient) - if err != nil { - return nil, err -} - - // distribute funds from community pool module account - if err := k.DistributeFromCommunityPool(sdkCtx, msg.Amount, recipient); err != nil { - return nil, err -} - -sdkCtx.Logger().Debug("transferred from the community pool", "amount", msg.Amount.String(), "recipient", msg.Recipient) - -return &types.MsgCommunityPoolSpendResponse{ -}, nil -} - -func (k MsgServer) - -CreateContinuousFund(ctx context.Context, msg *types.MsgCreateContinuousFund) (*types.MsgCreateContinuousFundResponse, error) { - sdkCtx := sdk.UnwrapSDKContext(ctx) - if err := k.validateAuthority(msg.Authority); err != nil { - return nil, err -} - -recipient, err := k.Keeper.authKeeper.AddressCodec().StringToBytes(msg.Recipient) - if err != nil { - return nil, err -} - - // deny creation if we know this address is blocked from receiving funds - if k.bankKeeper.BlockedAddr(recipient) { - return nil, fmt.Errorf("recipient is blocked in the bank keeper: %s", msg.Recipient) -} - -has, err := k.ContinuousFunds.Has(sdkCtx, recipient) - if err != nil { - return nil, err -} - if has { - return nil, fmt.Errorf("continuous fund already exists for recipient %s", msg.Recipient) -} - - // Validate the message fields - err = validateContinuousFund(sdkCtx, *msg) - if err != nil { - return nil, err -} - - // Check if total funds percentage exceeds 100% - // If exceeds, we should not setup continuous fund proposal. - totalStreamFundsPercentage := math.LegacyZeroDec() - -err = k.ContinuousFunds.Walk(sdkCtx, nil, func(key sdk.AccAddress, value types.ContinuousFund) (stop bool, err error) { - totalStreamFundsPercentage = totalStreamFundsPercentage.Add(value.Percentage) - -return false, nil -}) - if err != nil { - return nil, err -} - -totalStreamFundsPercentage = totalStreamFundsPercentage.Add(msg.Percentage) - if totalStreamFundsPercentage.GT(math.LegacyOneDec()) { - return nil, fmt.Errorf("cannot set continuous fund proposal\ntotal funds percentage exceeds 100\ncurrent total percentage: %s", totalStreamFundsPercentage.Sub(msg.Percentage).MulInt64(100).TruncateInt().String()) -} - - // Create continuous fund proposal - cf := types.ContinuousFund{ - Recipient: msg.Recipient, - Percentage: msg.Percentage, - Expiry: msg.Expiry, -} - - // Set continuous fund to the state - err = k.ContinuousFunds.Set(sdkCtx, recipient, cf) - if err != nil { - return nil, err -} - -return &types.MsgCreateContinuousFundResponse{ -}, nil -} - -func (k MsgServer) - -CancelContinuousFund(ctx context.Context, msg *types.MsgCancelContinuousFund) (*types.MsgCancelContinuousFundResponse, error) { - sdkCtx := sdk.UnwrapSDKContext(ctx) - if err := k.validateAuthority(msg.Authority); err != nil { - return nil, err -} - -recipient, err := k.Keeper.authKeeper.AddressCodec().StringToBytes(msg.Recipient) - if err != nil { - return nil, err -} - canceledHeight := sdkCtx.BlockHeight() - canceledTime := sdkCtx.BlockTime() - -has, err := k.ContinuousFunds.Has(sdkCtx, recipient) - if err != nil { - return nil, fmt.Errorf("cannot get continuous fund for recipient %w", err) -} - if !has { - return nil, fmt.Errorf("cannot cancel continuous fund for recipient %s - does not exist", msg.Recipient) -} - if err := k.ContinuousFunds.Remove(sdkCtx, recipient); err != nil { - return nil, fmt.Errorf("failed to remove continuous fund for recipient %s: %w", msg.Recipient, err) -} - -return &types.MsgCancelContinuousFundResponse{ - CanceledTime: canceledTime, - CanceledHeight: uint64(canceledHeight), - Recipient: msg.Recipient, -}, nil -} - -func (k MsgServer) - -UpdateParams(ctx context.Context, msg *types.MsgUpdateParams) (*types.MsgUpdateParamsResponse, error) { - sdkCtx := sdk.UnwrapSDKContext(ctx) - if err := k.validateAuthority(msg.GetAuthority()); err != nil { - return nil, err -} - if err := msg.Params.Validate(); err != nil { - return nil, fmt.Errorf("invalid params: %w", err) -} - if err := k.Params.Set(sdkCtx, msg.Params); err != nil { - return nil, fmt.Errorf("failed to set params: %w", err) -} - -return &types.MsgUpdateParamsResponse{ -}, nil -} -``` - -## Client - -It takes the advantage of `AutoCLI` - -```go expandable -package protocolpool - -import ( - - "fmt" - - autocliv1 "cosmossdk.io/api/cosmos/autocli/v1" - poolv1 "cosmossdk.io/api/cosmos/protocolpool/v1" - "github.com/cosmos/cosmos-sdk/version" -) - -// AutoCLIOptions implements the autocli.HasAutoCLIConfig interface. -func (am AppModule) - -AutoCLIOptions() *autocliv1.ModuleOptions { - return &autocliv1.ModuleOptions{ - Query: &autocliv1.ServiceCommandDescriptor{ - Service: poolv1.Query_ServiceDesc.ServiceName, - RpcCommandOptions: []*autocliv1.RpcCommandOptions{ - { - RpcMethod: "CommunityPool", - Use: "community-pool", - Short: "Query the amount of coins in the community pool", - Example: fmt.Sprintf(`%s query protocolpool community-pool`, version.AppName), -}, - { - RpcMethod: "ContinuousFunds", - Use: "continuous-funds", - Short: "Query all continuous funds", - Example: fmt.Sprintf(`%s query protocolpool continuous-funds`, version.AppName), -}, - { - RpcMethod: "ContinuousFund", - Use: "continuous-fund ", - Short: "Query a continuous fund by its recipient address", - Example: fmt.Sprintf(`%s query protocolpool continuous-fund cosmos1...`, version.AppName), - PositionalArgs: []*autocliv1.PositionalArgDescriptor{{ - ProtoField: "recipient" -}}, -}, -}, -}, - Tx: &autocliv1.ServiceCommandDescriptor{ - Service: poolv1.Msg_ServiceDesc.ServiceName, - RpcCommandOptions: []*autocliv1.RpcCommandOptions{ - { - RpcMethod: "FundCommunityPool", - Use: "fund-community-pool ", - Short: "Funds the community pool with the specified amount", - Example: fmt.Sprintf(`%s tx protocolpool fund-community-pool 100uatom --from mykey`, version.AppName), - PositionalArgs: []*autocliv1.PositionalArgDescriptor{{ - ProtoField: "amount" -}}, -}, - { - RpcMethod: "CreateContinuousFund", - Use: "create-continuous-fund ", - Short: "Create continuous fund for a recipient with optional expiry", - Example: fmt.Sprintf(`%s tx protocolpool create-continuous-fund cosmos1... 0.2 2023-11-31T12:34:56.789Z --from mykey`, version.AppName), - PositionalArgs: []*autocliv1.PositionalArgDescriptor{ - { - ProtoField: "recipient" -}, - { - ProtoField: "percentage" -}, - { - ProtoField: "expiry", - Optional: true -}, -}, - GovProposal: true, -}, - { - RpcMethod: "CancelContinuousFund", - Use: "cancel-continuous-fund ", - Short: "Cancel continuous fund for a specific recipient", - Example: fmt.Sprintf(`%s tx protocolpool cancel-continuous-fund cosmos1... --from mykey`, version.AppName), - PositionalArgs: []*autocliv1.PositionalArgDescriptor{ - { - ProtoField: "recipient" -}, -}, - GovProposal: true, -}, - { - RpcMethod: "UpdateParams", - Use: "update-params-proposal ", - Short: "Submit a proposal to update protocolpool module params. Note: the entire params must be provided.", - Example: fmt.Sprintf(`%s tx protocolpool update-params-proposal '{ "enabled_distribution_denoms": ["stake", "foo"] -}'`, version.AppName), - PositionalArgs: []*autocliv1.PositionalArgDescriptor{{ - ProtoField: "params" -}}, - GovProposal: true, -}, -}, -}, -} -} -``` diff --git a/sdk/latest/modules/slashing/README.mdx b/sdk/latest/modules/slashing/README.mdx index cd08a2f80..9b2ff227e 100644 --- a/sdk/latest/modules/slashing/README.mdx +++ b/sdk/latest/modules/slashing/README.mdx @@ -107,7 +107,7 @@ long as it contains precommits from +2/3 of total voting power. Proposers are incentivized to include precommits from all validators in the CometBFT `LastCommitInfo` by receiving additional fees proportional to the difference between the voting -power included in the `LastCommitInfo` and +2/3 (see [fee distribution](/sdk/v0.47/build/modules/distribution/README#begin-block)). +power included in the `LastCommitInfo` and +2/3 (see [fee distribution](/sdk/latest/modules/distribution/README#begin-block)). ```go type LastCommitInfo struct { @@ -144,7 +144,7 @@ bonded validator. The `SignedBlocksWindow` parameter defines the size The information stored for tracking validator liveness is as follows: ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/slashing/v1beta1/slashing.proto#L13-L35 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/slashing/v1beta1/slashing.proto#L13-L35 ``` ### Params @@ -155,7 +155,7 @@ it can be updated with governance or the address with authority. * Params: `0x00 | ProtocolBuffer(Params)` ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/slashing/v1beta1/slashing.proto#L37-L59 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/slashing/v1beta1/slashing.proto#L37-L62 ``` ## Messages diff --git a/sdk/latest/modules/staking/README.mdx b/sdk/latest/modules/staking/README.mdx index 73bb5c76a..33c408c12 100644 --- a/sdk/latest/modules/staking/README.mdx +++ b/sdk/latest/modules/staking/README.mdx @@ -46,6 +46,7 @@ network. * [MsgCancelUnbondingDelegation](#msgcancelunbondingdelegation) * [MsgBeginRedelegate](#msgbeginredelegate) * [MsgUpdateParams](#msgupdateparams) + * [MsgRotateConsPubKey](#msgrotateconspubkey) * [Begin-Block](#begin-block) * [Historical Info Tracking](#historical-info-tracking) * [End-Block](#end-block) @@ -95,7 +96,7 @@ it can be updated with governance or the address with authority. * Params: `0x51 | ProtocolBuffer(Params)` ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/staking.proto#L310-L333 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/staking/v1beta1/staking.proto#L298-L324 ``` ### Validator @@ -158,11 +159,11 @@ is updated during the validator set update process which takes place in [`EndBlo Each validator's state is stored in a `Validator` struct: ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/staking.proto#L82-L138 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/staking/v1beta1/staking.proto#L82-L136 ``` ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/staking.proto#L26-L80 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/staking/v1beta1/staking.proto#L26-L80 ``` ### Delegation @@ -178,7 +179,7 @@ delegator, and is associated with the shares for one validator. The sender of the transaction is the owner of the bond. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/staking.proto#L198-L216 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/staking/v1beta1/staking.proto#L191-L208 ``` #### Delegator Shares @@ -225,7 +226,7 @@ unbonding delegation entries. A UnbondingDelegation object is created every time an unbonding is initiated. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/staking.proto#L218-L261 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/staking/v1beta1/staking.proto#L210-L251 ``` ### Redelegation @@ -267,7 +268,7 @@ A redelegation object is created every time a redelegation occurs. To prevent where the source validator for this new redelegation is `Validator X`. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/staking.proto#L263-L308 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/staking/v1beta1/staking.proto#L253-L296 ``` ### Queues @@ -291,7 +292,7 @@ delegations queue is kept. * UnbondingDelegation: `0x41 | format(time) -> []DVPair` ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/staking.proto#L162-L172 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/staking/v1beta1/staking.proto#L157-L166 ``` #### RedelegationQueue @@ -302,7 +303,7 @@ kept. * RedelegationQueue: `0x42 | format(time) -> []DVVTriplet` ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/staking.proto#L179-L191 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/staking/v1beta1/staking.proto#L173-L184 ``` #### ValidatorQueue @@ -655,6 +656,8 @@ message Params { (gogoproto.customtype) = "github.com/cosmos/cosmos-sdk/types.Dec", (gogoproto.nullable) = false ]; + // key_rotation_fee is the fee charged when rotating a validator's consensus key. + cosmos.base.v1beta1.Coin key_rotation_fee = 7 [(gogoproto.nullable) = false, (amino.dont_omitempty) = true]; } // DelegationResponse is equivalent to Delegation except that it contains a @@ -736,6 +739,16 @@ they are in a deterministic order. The oldest HistoricalEntries will be pruned to ensure that there only exist the parameter-defined number of historical entries. +### Consensus key rotation + +Pending and historical consensus key rotations are tracked in five stores: + +* ConsKeyRotationQueue: `0x91 | format(maturityTime) | ValAddress` - rotations awaiting the end of their unbonding period, at which point the re-rotation rate limit is retired and the entry is pruned +* ValidatorConsKeyRotation: `0x92 | ValAddress` - a marker that the validator has rotated within the current unbonding period, enforcing the one-rotation-per-unbonding-period limit. The value is empty, and the entry is removed when the maturity queue retires it +* RotationLockedConsAddrIndex: `0x93 | ConsAddress` - consensus addresses a rotation has claimed, valued with a lock kind and the validator's operator address. A rotated-away address stays locked until equivocation evidence for it can no longer be admitted, and resolves back to the validator for slashing. A pending rotation's target address is reserved so no other validator can claim it, and that entry is released once the rotation applies in the end blocker +* ConsKeyRotationApplyQueue: `0x94 | BigEndian(applyHeight) | ValAddress` - height-keyed queue of rotations, valued with the new consensus public key, applied two heights after the rotation message +* ConsKeyEvidenceExpiryQueue: `0x95 | format(evidenceExpiryTime) | ConsAddress` - queue that retires an old address's lock once equivocation evidence for the rotated-away key can no longer be admitted, using the evidence time and block-height windows captured at rotation time + ## State Transitions ### Validators @@ -923,11 +936,11 @@ A validator is created using the `MsgCreateValidator` message. The validator must be created with an initial delegation from the operator. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/tx.proto#L20-L21 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/staking/v1beta1/tx.proto#L20-L21 ``` ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/tx.proto#L50-L73 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/staking/v1beta1/tx.proto#L55-L78 ``` This message is expected to fail if: @@ -952,11 +965,11 @@ The `Description`, `CommissionRate` of a validator can be updated using the `MsgEditValidator` message. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/tx.proto#L23-L24 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/staking/v1beta1/tx.proto#L23-L24 ``` ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/tx.proto#L78-L97 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/staking/v1beta1/tx.proto#L83-L102 ``` This message is expected to fail if: @@ -975,11 +988,11 @@ some amount of their validator's (newly created) delegator-shares that are assigned to `Delegation.Shares`. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/tx.proto#L26-L28 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/staking/v1beta1/tx.proto#L26-L28 ``` ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/tx.proto#L102-L114 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/staking/v1beta1/tx.proto#L107-L119 ``` This message is expected to fail if: @@ -1011,17 +1024,17 @@ The `MsgUndelegate` message allows delegators to undelegate their tokens from validator. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/tx.proto#L34-L36 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/staking/v1beta1/tx.proto#L34-L36 ``` ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/tx.proto#L140-L152 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/staking/v1beta1/tx.proto#L145-L157 ``` This message returns a response containing the completion time of the undelegation: ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/tx.proto#L154-L158 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/staking/v1beta1/tx.proto#L159-L167 ``` This message is expected to fail if: @@ -1050,11 +1063,11 @@ When this message is processed the following actions occur: The `MsgCancelUnbondingDelegation` message allows delegators to cancel the `unbondingDelegation` entry and delegate back to a previous validator. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/tx.proto#L38-L42 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/staking/v1beta1/tx.proto#L38-L42 ``` ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/tx.proto#L160-L175 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/staking/v1beta1/tx.proto#L169-L183 ``` This message is expected to fail if: @@ -1077,17 +1090,17 @@ the unbonding period has passed, the redelegation is automatically completed in the EndBlocker. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/tx.proto#L30-L32 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/staking/v1beta1/tx.proto#L30-L32 ``` ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/tx.proto#L119-L132 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/staking/v1beta1/tx.proto#L124-L137 ``` This message returns a response containing the completion time of the redelegation: ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/tx.proto#L133-L138 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/staking/v1beta1/tx.proto#L139-L143 ``` This message is expected to fail if: @@ -1119,7 +1132,7 @@ The `MsgUpdateParams` update the staking module parameters. The params are updated through a governance proposal where the signer is the gov module account address. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/tx.proto#L182-L195 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/staking/v1beta1/tx.proto#L190-L202 ``` The message handling can fail if: @@ -1127,6 +1140,28 @@ The message handling can fail if: * signer is not the authority defined in the staking keeper (usually the gov module account). * the `bond_denom` in the updated params has zero supply in the bank module (i.e., the denom does not exist on-chain). +### MsgRotateConsPubKey + +The `MsgRotateConsPubKey` message replaces a validator's consensus public key in place. The message is signed by the validator's operator address and carries the new public key. Power, delegations, commission, and the operator address are unchanged. + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/staking/v1beta1/tx.proto#L210-L220 +``` + +Handling the message burns the `KeyRotationFee` param from the operator account and enqueues the rotation. The new key enters the CometBFT validator set two heights after the message executes. The rotated-away consensus address is retained until equivocation evidence for it can no longer be admitted, which is at least the unbonding period and often longer, so evidence against the old key still slashes the validator. + +The message handling can fail if: + +* the fee cannot be deducted from the operator account. +* the validator already rotated within the current unbonding period. +* the new key's type is not in the chain's consensus params `validator.pub_key_types`. +* the validator does not exist. +* the validator is jailed. +* the new key is already in use by another validator. +* the new key is locked by a rotation, either because a validator rotated away from it and evidence against it can still be admitted, or because a pending rotation already targets it. + +For the operational procedure, see [Rotate a validator consensus key, x/staking](/sdk/latest/keys/rotate-validator-key). For the concepts, see [Key rotation](/sdk/latest/keys/key-rotation). + ## Begin-Block Each abci begin block call, the historical info will get stored and pruned @@ -1220,6 +1255,10 @@ Complete the unbonding of all mature `Redelegation.Entries` within the * remove the `Redelegation` object from the store if there are no remaining entries. +### Consensus Key Rotations + +At the height a rotation message executes, the end blocker emits the validator update handing the validator's power from the old consensus key to the new one. Two heights later, when CometBFT makes the update effective, the apply queue swaps the stored consensus key. When a rotation's unbonding period ends, the maturity queue retires the re-rotation rate limit and the validator may rotate again. The old consensus address stays locked on its own, longer schedule, until equivocation evidence for it can no longer be admitted. + ## Hooks Other modules may register operations to execute when a certain event has @@ -1342,11 +1381,14 @@ The staking module contains the following parameters: | Key | Type | Example | | ----------------- | ---------------- | ---------------------- | | UnbondingTime | string (time ns) | "259200000000000" | -| MaxValidators | uint16 | 100 | -| KeyMaxEntries | uint16 | 7 | -| HistoricalEntries | uint16 | 3 | +| MaxValidators | uint32 | 100 | +| MaxEntries | uint32 | 7 | +| HistoricalEntries | uint32 | 3 | | BondDenom | string | "stake" | | MinCommissionRate | string | "0.000000000000000000" | +| KeyRotationFee | sdk.Coin | `{"denom":"stake","amount":"1000000"}` | + +The limit of one consensus key rotation per unbonding period is fixed and is not a parameter. ## Client @@ -1554,8 +1596,12 @@ Example Output: ```bash bond_denom: stake historical_entries: 10000 +key_rotation_fee: + amount: "1000000" + denom: stake max_entries: 7 max_validators: 50 +min_commission_rate: "0.000000000000000000" unbonding_time: 1814400s ``` @@ -2088,6 +2134,22 @@ Example: simd tx staking cancel-unbond cosmosvaloper1gghjut3ccd8ay0zduzj64hwre2fxs9ldmqhffj 100stake 123123 --from mykey ``` +##### rotate-cons-pub-key + +The command `rotate-cons-pub-key` allows a validator operator to replace the validator's consensus public key. The new key is given as proto-JSON, and the validator address is derived from the `--from` signer. Handling the message burns the `key_rotation_fee` param. + +Usage: + +```bash +simd tx staking rotate-cons-pub-key [new-pubkey] [flags] +``` + +Example: + +```bash +simd tx staking rotate-cons-pub-key '{"@type":"/cosmos.crypto.ed25519.PubKey","key":"..."}' --from myvalidator +``` + ### gRPC A user can query the `staking` module using gRPC endpoints. @@ -2706,7 +2768,12 @@ Example Output: "maxValidators": 100, "maxEntries": 7, "historicalEntries": 10000, - "bondDenom": "stake" + "bondDenom": "stake", + "minCommissionRate": "0.000000000000000000", + "keyRotationFee": { + "denom": "stake", + "amount": "1000000" + } } } ``` @@ -2717,7 +2784,7 @@ A user can query the `staking` module using REST endpoints. #### DelegatorDelegations -The `DelegtaorDelegations` REST endpoint queries all delegations of a given delegator address. +The `DelegatorDelegations` REST endpoint queries all delegations of a given delegator address. ```bash /cosmos/staking/v1beta1/delegations/{delegatorAddr} diff --git a/sdk/latest/modules/upgrade/README.mdx b/sdk/latest/modules/upgrade/README.mdx index 33c1aa718..372076830 100644 --- a/sdk/latest/modules/upgrade/README.mdx +++ b/sdk/latest/modules/upgrade/README.mdx @@ -51,7 +51,7 @@ type Plan struct { If an operator running the application binary also runs a sidecar process to assist in the automatic download and upgrade of a binary, the `Info` allows this process to -be seamless. This tool is [Cosmovisor](https://github.com/cosmos/cosmos-sdk/tree/main/tools/cosmovisor#readme). +be seamless. This tool is [Cosmovisor](https://github.com/cosmos/cosmos-sdk/tree/release/v0.55.x/tools/cosmovisor#readme). ### Handler @@ -103,7 +103,7 @@ the `Plan`, which targets a specific `Handler`, is persisted and scheduled. The upgrade can be delayed or hastened by updating the `Plan.Height` in a new proposal. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/upgrade/v1beta1/tx.proto#L29-L41 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/upgrade/v1beta1/tx.proto#L28-L39 ``` #### Cancelling Upgrade Proposals @@ -115,7 +115,7 @@ Of course this requires that the upgrade was known to be a bad idea well before upgrade itself, to allow time for a vote. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/upgrade/v1beta1/tx.proto#L48-L57 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/upgrade/v1beta1/tx.proto#L46-L54 ``` If such a possibility is desired, the upgrade height is to be diff --git a/sdk/latest/node/keyring.mdx b/sdk/latest/node/keyring.mdx index b48ae1a71..94fcebfc8 100644 --- a/sdk/latest/node/keyring.mdx +++ b/sdk/latest/node/keyring.mdx @@ -29,7 +29,7 @@ MY_VALIDATOR_ADDRESS=$(simd keys show my_validator -a --keyring-backend test) This generates a 24-word mnemonic phrase and stores your key. **Save the mnemonic** if you'll use this key for value-bearing tokens. -This tutorial uses the `test` backend (unencrypted, for testing only). For production, use the `os` backend which integrates with your system's secure keyring. See [keyring backends](#reference:-keyring-backends) below for more information. +This tutorial uses the `test` backend (unencrypted, for testing only). For production, use the `os` backend which integrates with your system's secure keyring. See [keyring backends](#reference-keyring-backends) below for more information. ## Next steps @@ -150,6 +150,6 @@ You can set the keyring-backend using an environment variable: `BINNAME_KEYRING_ ### Additional key management -By default, the keyring generates a `secp256k1` keypair. The keyring also supports `ed25519` keys, which may be created by passing the `--algo ed25519` flag. A keyring can hold both types of keys simultaneously, and the Cosmos SDK's `x/auth` module supports both public key algorithms natively. +By default, the keyring generates a `secp256k1` keypair. The keyring also supports `ml_dsa_65`, the post-quantum signature algorithm, selected with the `--key-type` flag. A keyring can hold both types of keys simultaneously. For the algorithm and its tradeoffs, see [Post-quantum keys](/sdk/latest/keys/post-quantum-keys); to create and fund a post-quantum account, see [Create an ML-DSA account](/sdk/latest/keys/create-ml-dsa-account). -For help with key management commands, use `simd keys --help` or `simd keys [command] --help`. +List the key types your binary supports with `simd keys list-key-types`. For help with key management commands, use `simd keys --help` or `simd keys [command] --help`. diff --git a/sdk/latest/node/run-node.mdx b/sdk/latest/node/run-node.mdx index e055321c7..17732ebf1 100644 --- a/sdk/latest/node/run-node.mdx +++ b/sdk/latest/node/run-node.mdx @@ -5,7 +5,7 @@ title: Running a Node **Synopsis** -This section explains how to run a blockchain node. The application used in this tutorial is [`simapp`](https://github.com/cosmos/cosmos-sdk/tree/main/simapp), and its corresponding CLI binary `simd`. +This section explains how to run a blockchain node. The application used in this tutorial is [`simapp`](https://github.com/cosmos/cosmos-sdk/tree/release/v0.55.x/simapp), and its corresponding CLI binary `simd`. @@ -28,6 +28,8 @@ simd init --chain-id my-test-chain The command above creates all the configuration files needed for your node to run, as well as a default genesis file, which defines the initial state of the network. +The `init` command also selects the validator consensus key algorithm with the optional `--consensus-key-algo` flag. It defaults to `ed25519`. To initialize the node on a post-quantum key instead, see [the ML-DSA consensus key guides](/sdk/latest/keys/post-quantum-keys). + All these configuration files are in `~/.simapp` by default, but you can overwrite the location of this folder by passing the `--home` flag to each command, or set an `$APPD_HOME` environment variable (where `APPD` is the name of the binary). @@ -52,7 +54,7 @@ The `~/.simapp` folder has the following structure: ## 2. Update configuration settings (optional) -To change field values in configuration files (for example, genesis.json), use `jq` ([installation](https://stedolan.github.io/jq/download/) & [docs](https://stedolan.github.io/jq/manual/#Assignment)) and `sed` commands. A few examples are listed here. +To change field values in configuration files (for example, genesis.json), use `jq` ([installation](https://jqlang.org/download/) & [docs](https://jqlang.org/manual/#assignment)) and `sed` commands. A few examples are listed here. ```bash expandable # to change the chain-id @@ -86,7 +88,7 @@ Now, you can grant this account some `stake` tokens in your chain's genesis file simd genesis add-genesis-account $MY_VALIDATOR_ADDRESS 100000000000stake ``` -Recall that `$MY_VALIDATOR_ADDRESS` is a variable that holds the address of the `my_validator` key in the [keyring](/sdk/latest/node/keyring#create-a-key). Also note that the tokens in the Cosmos SDK have the `{amount}{denom}` format: `amount` is an 18-digit-precision decimal number, and `denom` is the unique token identifier with its denomination key (e.g., `atom` or `uatom`). Here, `stake` tokens are granted, as `stake` is the token identifier used for staking in [`simapp`](https://github.com/cosmos/cosmos-sdk/tree/main/simapp). For your own chain with its own staking denom, that token identifier should be used instead. +Recall that `$MY_VALIDATOR_ADDRESS` is a variable that holds the address of the `my_validator` key in the [keyring](/sdk/latest/node/keyring#create-a-key). Also note that the tokens in the Cosmos SDK have the `{amount}{denom}` format: `amount` is an 18-digit-precision decimal number, and `denom` is the unique token identifier with its denomination key (e.g., `atom` or `uatom`). Here, `stake` tokens are granted, as `stake` is the token identifier used for staking in [`simapp`](https://github.com/cosmos/cosmos-sdk/tree/release/v0.55.x/simapp). For your own chain with its own staking denom, that token identifier should be used instead. ## 4. Create genesis transaction @@ -121,16 +123,16 @@ simd genesis gentx --help The Cosmos SDK automatically generates two configuration files inside `~/.simapp/config`: * `config.toml`: used to configure the CometBFT, learn more on [CometBFT's documentation](/cometbft/latest/docs/core/configuration), -* `app.toml`: generated by the Cosmos SDK, and used to configure your app, such as state pruning strategies, telemetry, gRPC and REST server configuration, state sync... +* `app.toml`: generated by the Cosmos SDK, and used to configure your app, such as state pruning strategies, telemetry, gRPC and REST server configuration, state sync, etc. See the [default `app.toml` template](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/server/config/toml.go) for every field and its inline documentation. -Both files are heavily commented, please refer to them directly to tweak your node. +Both files are heavily commented, please refer to them directly to tweak your node. Each field carries an inline comment that explains it. This includes operational settings such as `query-gas-limit` that matter for nodes serving public RPC. One example config to tweak is the `minimum-gas-prices` field inside `app.toml`, which defines the minimum gas prices the validator node is willing to accept for processing a transaction. Depending on the chain, it might be an empty string or not. If it's empty, make sure to edit the field with some value, for example `10token`, or else the node will halt on startup. For the purposes of this tutorial, the minimum gas price is set to 0: ```toml # The minimum gas prices a validator is willing to accept for processing a # transaction. A transaction's fees must meet the minimum of any denomination - # specified in this config (e.g. 0.25token1;0.0001token2). + # specified in this config (e.g. 0.25token1,0.0001token2). minimum-gas-prices = "0stake" ``` @@ -140,7 +142,7 @@ When running a node (not a validator!) and not wanting to run the application me ```toml [mempool] # Setting max-txs to 0 will allow for an unbounded amount of transactions in the mempool. -# Setting max_txs to negative 1 (-1) will disable transactions from being inserted into the mempool. +# Setting max_txs to negative 1 (-1) will disable transactions from being inserted into the mempool (no-op mempool). # Setting max_txs to a positive number (> 0) will limit the number of transactions in the mempool, by the specified amount. # # Note, this configuration only applies to SDK built-in app-side mempool @@ -162,7 +164,7 @@ You should see blocks come in. ### What happens when the node starts -The `start` command (defined in [`server/start.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/server/start.go)) boots up the full-node in the following sequence: +The `start` command (defined in [`server/start.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/server/start.go)) boots up the full-node in the following sequence: 1. It opens the `db` (LevelDB by default) containing the latest persisted state. On first start, this is empty. 2. It creates a new instance of the application via an `appCreator` function, which is the [application constructor](/sdk/latest/learn/intro/sdk-app-architecture#constructor-function). @@ -171,7 +173,7 @@ The `start` command (defined in [`server/start.go`](https://github.com/cosmos/co The previous command allows you to run a single node. This is enough for the next section on interacting with this node, but you may wish to run multiple nodes at the same time, and see how consensus happens between them. -The naive way would be to run the same commands again in separate terminal windows. This is possible. However, [Docker Compose](https://docs.docker.com/compose/) can be leveraged to run a localnet. If you need inspiration on how to set up your own localnet with Docker Compose, refer to the Cosmos SDK's [`docker-compose.yml`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/docker-compose.yml). +The naive way would be to run the same commands again in separate terminal windows. This is possible. However, [Docker Compose](https://docs.docker.com/compose/) can be leveraged to run a localnet. If you need inspiration on how to set up your own localnet with Docker Compose, refer to the Cosmos SDK's [`docker-compose.yml`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/docker-compose.yml). ### Standalone App/CometBFT @@ -204,15 +206,34 @@ See the [Log Overview](/sdk/latest/guides/testing/log) for more information on l State sync is the act in which a node syncs the latest or close to the latest state of a blockchain. This is useful for users who don't want to sync all the blocks in history. Read more in [CometBFT documentation](/cometbft/latest/docs/core/state-sync). -State sync works thanks to snapshots. Read how the SDK handles snapshots [here](https://github.com/cosmos/cosmos-sdk/blob/825245d/store/snapshots/README.md). +State sync works thanks to snapshots. For how the SDK produces and stores them, see the [store/snapshots README](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/store/snapshots/README.md). + +### Produce and serve snapshots + +A node serves snapshots to state-syncing peers only after it takes them. Configure snapshots in `app.toml` under `[state-sync]`: + +```toml +[state-sync] +# Block interval at which the node takes a local snapshot (0 disables snapshots). +snapshot-interval = 1000 + +# Number of recent snapshots to keep and serve (0 keeps all). +snapshot-keep-recent = 2 +``` + +A node with `snapshot-interval = 0` takes no snapshots and cannot serve state sync. The `snapshot-keep-recent` setting bounds how many snapshots the node retains and offers to peers. Restart the node after changing these values. + +Snapshots work alongside pruning. When pruning is enabled, the SDK retains heights that are multiples of `snapshot-interval` until the snapshot at that height completes, so a pruning node can still produce snapshots. + +To take a snapshot on demand instead of waiting for the interval, run ` snapshots export`. It snapshots the latest committed height by default, or the height set with `--height`. This command opens the application database directly, so stop the node before running it. The automatic `snapshot-interval` is the only way to produce snapshots without downtime. ### Local State Sync Local state sync works similarly to normal state sync except that it works off a local snapshot of state instead of one provided via the p2p network. The steps to start local state sync are similar to normal state sync with a few different design considerations. 1. As mentioned in the [state sync documentation](/cometbft/latest/docs/core/state-sync), one must set a height and hash in the config.toml along with a few RPC servers (the aforementioned link has instructions on how to do this). -2. Run ` snapshot restore ` to restore a local snapshot (note: first load it from a file with the *load* command). -3. Bootstrapping Comet state to start the node after the snapshot has been ingested. This can be done with the bootstrap command ` comet bootstrap-state` +2. Run ` snapshots restore ` to restore a local snapshot (first load it from a file with ` snapshots load`). +3. Bootstrap Comet state to start the node after the snapshot has been ingested. Run ` comet bootstrap-state`. ### Snapshots Commands @@ -249,4 +270,6 @@ Your node is now running and producing blocks. You have successfully initialized ## Next steps - [Interact with the node](/sdk/latest/node/interact-node) to send transactions and query state -- [Generate and sign transactions](/sdk/latest/node/txs) to learn advanced transaction workflows \ No newline at end of file +- [Generate and sign transactions](/sdk/latest/node/txs) to learn advanced transaction workflows +- [Key rotation](/sdk/latest/keys/key-rotation) to understand the consensus key in `priv_validator_key.json` and how a validator replaces it +- [Cosmos-KMS and remote signing](/sdk/latest/kms/remote-signing) to move that key off the node entirely \ No newline at end of file diff --git a/sdk/latest/node/run-production.mdx b/sdk/latest/node/run-production.mdx index 3eb5f3789..655ffaa58 100644 --- a/sdk/latest/node/run-production.mdx +++ b/sdk/latest/node/run-production.mdx @@ -117,151 +117,14 @@ If the node that is being started is a validator there are multiple ways a valid #### File -File-based signing is the simplest and default approach. This approach works by storing the consensus key generated on initialization to sign blocks. This approach is only as safe as your server setup, as if the server is compromised, so is your key. This key is located in the `config/priv_val_key.json` directory generated on initialization. +File-based signing is the simplest and default approach. This approach works by storing the consensus key generated on initialization to sign blocks. This approach is only as safe as your server setup, as if the server is compromised, so is your key. This key is located in the `config/priv_validator_key.json` file generated on initialization. -A second file exists that users must be aware of; the file is located in the data directory `data/priv_val_state.json`. This file protects your node from double signing. It keeps track of the consensus key's last sign height, round, and latest signature. If the node crashes and needs to be recovered, this file must be kept in order to ensure that the consensus key will not be used for signing a block that was previously signed. +A second file exists that users must be aware of; the file is located in the data directory `data/priv_validator_state.json`. This file protects your node from double signing. It keeps track of the consensus key's last sign height, round, and latest signature. If the node crashes and needs to be recovered, this file must be kept in order to ensure that the consensus key will not be used for signing a block that was previously signed. -#### Remote Signer +#### Remote signer A remote signer is a secondary server that is separate from the running node that signs blocks with the consensus key. This means that the consensus key does not live on the node itself. This increases security because your full node which is connected to the remote signer can be swapped without missing blocks. -The two most used remote signers are [tmkms](https://github.com/iqlusioninc/tmkms) from [Iqlusion](https://www.iqlusion.io) and [horcrux](https://github.com/strangelove-ventures/horcrux) from [Strangelove](https://strange.love). +The Cosmos stack's remote signer is Cosmos-KMS, which holds the consensus key in a file, a PKCS#11 HSM, or AWS KMS. For what remote signing is and how it works, see [Cosmos-KMS and remote signing](/sdk/latest/kms/remote-signing). To set a signer up end to end, follow the [remote signing tutorial](/sdk/latest/kms/tutorial-file-backend), then harden the setup with [remote signing best practices](/sdk/latest/kms/best-practices). -##### TMKMS - -###### Dependencies - -1. Update server dependencies and install extras needed. - -```sh -sudo apt update -y && sudo apt install build-essential curl jq -y -``` - -2. Install Rust: - -```sh -curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -``` - -3. Install Libusb: - -```sh -sudo apt install libusb-1.0-0-dev -``` - -###### Setup - -There are two ways to install tmkms, from source or `cargo install`. In the examples we will cover downloading or building from source and using softsign. Softsign stands for software signing, but you could use a [yubihsm](https://www.yubico.com/products/hardware-security-module/) as your signing key if you wish. - -1. Build: - -From source: - -```bash -cd $HOME -git clone https://github.com/iqlusioninc/tmkms.git -cd $HOME/tmkms -cargo install tmkms --features=softsign -tmkms init config -tmkms softsign keygen ./config/secrets/secret_connection_key -``` - -or - -Cargo install: - -```bash -cargo install tmkms --features=softsign -tmkms init config -tmkms softsign keygen ./config/secrets/secret_connection_key -``` - - -To use tmkms with a yubikey install the binary with `--features=yubihsm`. - - -2. Migrate the validator key from the full node to the new tmkms instance. - -```bash -scp user@123.456.32.123:~/.simd/config/priv_validator_key.json ~/tmkms/config/secrets -``` - -3. Import the validator key into tmkms. - -```bash -tmkms softsign import $HOME/tmkms/config/secrets/priv_validator_key.json $HOME/tmkms/config/secrets/priv_validator_key -``` - -At this point, it is necessary to delete the `priv_validator_key.json` from the validator node and the tmkms node. Since the key has been imported into tmkms (above) it is no longer necessary on the nodes. The key can be safely stored offline. - -4. Modify the `tmkms.toml`. - -```bash -vim $HOME/tmkms/config/tmkms.toml -``` - -This example shows a configuration that could be used for soft signing. The example has an IP of `123.456.12.345` with a port of `26659` and a chain\_id of `test-chain-waSDSe`. These are items that must be modified for the use case of tmkms and the network. - -```toml expandable -# CometBFT KMS configuration file - -## Chain Configuration - -[[chain]] -id = "osmosis-1" -key_format = { type = "bech32", account_key_prefix = "cosmospub", consensus_key_prefix = "cosmosvalconspub" } -state_file = "/root/tmkms/config/state/priv_validator_state.json" - -## Signing Provider Configuration - -### Software-based Signer Configuration - -[[providers.softsign]] -chain_ids = ["test-chain-waSDSe"] -key_type = "consensus" -path = "/root/tmkms/config/secrets/priv_validator_key" - -## Validator Configuration - -[[validator]] -chain_id = "test-chain-waSDSe" -addr = "tcp://123.456.12.345:26659" -secret_key = "/root/tmkms/config/secrets/secret_connection_key" -protocol_version = "v0.34" -reconnect = true -``` - -5. Set the address of the tmkms instance. - -```bash -vim $HOME/.simd/config/config.toml - -priv_validator_laddr = "tcp://0.0.0.0:26659" -``` - - -The above address is set to `0.0.0.0`, but it is recommended to set the tmkms server address to secure the startup. - - - -It is recommended to comment or delete the lines that specify the path of the validator key and validator: - -```toml -# Path to the JSON file containing the private key to use as a validator in the consensus protocol -# priv_validator_key_file = "config/priv_validator_key.json" - -# Path to the JSON file containing the last sign state of a validator -# priv_validator_state_file = "data/priv_validator_state.json" -``` - - - -6. Start the two processes. - -```bash -tmkms start -c $HOME/tmkms/config/tmkms.toml -``` - -```bash -simd start -``` +TMKMS is the previous remote signer; Cosmos-KMS is the recommended one going forward. Validators running TMKMS should [migrate](/sdk/latest/kms/migrate-from-tmkms). diff --git a/sdk/latest/node/txs.mdx b/sdk/latest/node/txs.mdx index b575521cd..3c4504b14 100644 --- a/sdk/latest/node/txs.mdx +++ b/sdk/latest/node/txs.mdx @@ -332,7 +332,7 @@ txb.SetTimeoutTimestamp(time.Now().Add(expiration + (1 * time.Nanosecond))) ### Signing a Transaction -The encoding config is set to use Protobuf, which will use `SIGN_MODE_DIRECT` by default. As per [ADR-020](https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-020-protobuf-transaction-encoding.md), each signer needs to sign the `SignerInfo`s of all other signers. This means that two steps must be performed sequentially: +The encoding config is set to use Protobuf, which will use `SIGN_MODE_DIRECT` by default. As per [ADR-020](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/docs/architecture/adr-020-protobuf-transaction-encoding.md), each signer needs to sign the `SignerInfo`s of all other signers. This means that two steps must be performed sequentially: * for each signer, populate the signer's `SignerInfo` inside `TxBuilder` * once all `SignerInfo`s are populated, for each signer, sign the `SignDoc` (the payload to be signed). diff --git a/sdk/latest/reference/architecture.mdx b/sdk/latest/reference/architecture.mdx index e77effd8f..4a6303765 100644 --- a/sdk/latest/reference/architecture.mdx +++ b/sdk/latest/reference/architecture.mdx @@ -1,6 +1,6 @@ --- title: "Architecture Decision Records (ADR)" -description: "Version: v0.54" +description: "Version: v0.55" --- This is a location to record all high-level architecture decisions in the Cosmos-SDK. diff --git a/sdk/latest/reference/architecture/adr-009-evidence-module.mdx b/sdk/latest/reference/architecture/adr-009-evidence-module.mdx index 0b22fbf7c..b041e21fc 100644 --- a/sdk/latest/reference/architecture/adr-009-evidence-module.mdx +++ b/sdk/latest/reference/architecture/adr-009-evidence-module.mdx @@ -20,7 +20,7 @@ evidence can be submitted, evaluated and verified resulting in some agreed upon penalty for any misbehavior committed by a validator, such as equivocation (double-voting), signing when unbonded, signing an incorrect state transition (in the future), etc. Furthermore, such a mechanism is paramount for any -[IBC](https://github.com/cosmos/ics/blob/master/ibc/2_IBC_ARCHITECTURE.md) or +IBC (`https://github.com/cosmos/ics/blob/master/ibc/2_IBC_ARCHITECTURE.md`) or cross-chain validation protocol implementation in order to support the ability for any misbehavior to be relayed back from a collateralized chain to a primary chain so that the equivocating validator(s) can be slashed. @@ -214,5 +214,5 @@ type GenesisState struct { ## References * [ICS](https://github.com/cosmos/ics) -* [IBC Architecture](https://github.com/cosmos/ics/blob/master/ibc/1_IBC_ARCHITECTURE.md) +* IBC Architecture: `https://github.com/cosmos/ics/blob/master/ibc/1_IBC_ARCHITECTURE.md` * [Tendermint Fork Accountability](https://github.com/tendermint/spec/blob/7b3138e69490f410768d9b1ffc7a17abc23ea397/spec/consensus/fork-accountability.md) diff --git a/sdk/latest/reference/architecture/adr-030-authz-module.mdx b/sdk/latest/reference/architecture/adr-030-authz-module.mdx index 1c328b39b..a84fa6ea7 100644 --- a/sdk/latest/reference/architecture/adr-030-authz-module.mdx +++ b/sdk/latest/reference/architecture/adr-030-authz-module.mdx @@ -27,7 +27,7 @@ The concrete use cases which motivated this module include: delegated stake * "sub-keys" functionality, as originally proposed in [#4480](https://github.com/cosmos/cosmos-sdk/issues/4480) which is a term used to describe the functionality provided by this module together with - the `fee_grant` module from [ADR 029](/sdk/v0.50/build/architecture/adr-029-fee-grant-module) and the [group module](https://github.com/cosmos/cosmos-sdk/tree/main/x/group). + the `fee_grant` module from [ADR 029](/sdk/v0.50/build/architecture/adr-029-fee-grant-module) and the [group module](https://github.com/cosmos/cosmos-sdk/tree/release/v0.53.x/x/group). The "sub-keys" functionality roughly refers to the ability for one account to grant some subset of its capabilities to other accounts with possibly less robust, but easier to use security measures. For instance, a master account representing diff --git a/sdk/latest/reference/architecture/adr-038-state-listening.mdx b/sdk/latest/reference/architecture/adr-038-state-listening.mdx index eb1e42b0e..eedb1c2fc 100644 --- a/sdk/latest/reference/architecture/adr-038-state-listening.mdx +++ b/sdk/latest/reference/architecture/adr-038-state-listening.mdx @@ -22,7 +22,7 @@ This ADR defines a set of changes to enable listening to state changes of indivi ## Context -Currently, KVStore data can be remotely accessed through [Queries](https://github.com/cosmos/cosmos-sdk/blob/master/docs/building-modules/messages-and-queries.md#queries) +Currently, KVStore data can be remotely accessed through [Queries](https://github.com/cosmos/cosmos-sdk/blob/release/v0.46.x/docs/building-modules/messages-and-queries.md#queries) which proceed either through Tendermint and the ABCI, or through the gRPC server. In addition to these request/response queries, it would be beneficial to have a means of listening to state changes as they occur in real time. diff --git a/sdk/latest/reference/architecture/adr-042-group-module.mdx b/sdk/latest/reference/architecture/adr-042-group-module.mdx index e44e204ac..8303e178a 100644 --- a/sdk/latest/reference/architecture/adr-042-group-module.mdx +++ b/sdk/latest/reference/architecture/adr-042-group-module.mdx @@ -27,11 +27,11 @@ The legacy amino multi-signature mechanism of the Cosmos SDK has certain limitat While the group module is not meant to be a total replacement for the current multi-signature accounts, it provides a solution to the limitations described above, with a more flexible key management system where keys can be added, updated or removed, as well as configurable thresholds. It's meant to be used with other access control modules such as [`x/feegrant`](/sdk/v0.50/build/architecture/adr-029-fee-grant-module) ans [`x/authz`](/sdk/latest/reference/architecture/adr-030-authz-module) to simplify key management for individuals and organizations. -The proof of concept of the group module can be found in [Link](https://github.com/regen-network/regen-ledger/tree/master/proto/regen/group/v1alpha1) and [Link](https://github.com/regen-network/regen-ledger/tree/master/x/group). +The proof of concept of the group module can be found in `https://github.com/regen-network/regen-ledger/tree/master/proto/regen/group/v1alpha1` and `https://github.com/regen-network/regen-ledger/tree/master/x/group`. ## Decision -We propose merging the `x/group` module with its supporting [ORM/Table Store package](https://github.com/regen-network/regen-ledger/tree/master/orm) ([#7098](https://github.com/cosmos/cosmos-sdk/issues/7098)) into the Cosmos SDK and continuing development here. There will be a dedicated ADR for the ORM package. +We propose merging the `x/group` module with its supporting ORM/Table Store package (`https://github.com/regen-network/regen-ledger/tree/master/orm`) ([#7098](https://github.com/cosmos/cosmos-sdk/issues/7098)) into the Cosmos SDK and continuing development here. There will be a dedicated ADR for the ORM package. ### Group diff --git a/sdk/latest/reference/architecture/adr-050-sign-mode-textual-annex1.mdx b/sdk/latest/reference/architecture/adr-050-sign-mode-textual-annex1.mdx index b253f7d73..96bbd9fc3 100644 --- a/sdk/latest/reference/architecture/adr-050-sign-mode-textual-annex1.mdx +++ b/sdk/latest/reference/architecture/adr-050-sign-mode-textual-annex1.mdx @@ -12,7 +12,7 @@ title: 'ADR 050: SIGN_MODE_TEXTUAL: Annex 1 Value Renderers' ## Status -Accepted. Implementation started. Small value renderers details still need to be polished. +Archived. `SIGN_MODE_TEXTUAL` was removed in Cosmos SDK v0.55, and the proto enum value is reserved. This ADR is retained for historical reference. ## Abstract @@ -65,7 +65,7 @@ Value Renderers describe how values of different Protobuf types should be encode ### `repeated` -* Applies to all `repeated` fields, except `cosmos.tx.v1beta1.TxBody#Messages`, which has a particular encoding (see [ADR-050](/sdk/v0.50/build/architecture/adr-050-sign-mode-textual)). +* Applies to all `repeated` fields, except `cosmos.tx.v1beta1.TxBody#Messages`, which has a particular encoding (see [ADR-050](/sdk/latest/reference/architecture/adr-050-sign-mode-textual)). * A repeated type has the following template: ``` @@ -282,7 +282,7 @@ The number 35 was chosen because it is the longest length where the hashed-and-p * byte arrays starting from length 36 will be be hashed to 32 bytes, which is 64 hex characters plus 15 spaces, and with the `SHA-256=` prefix, it takes 87 characters. Also, secp256k1 public keys have length 33, so their Textual representation is not their hashed value, which we would like to avoid. -Note: Data longer than 35 bytes are not rendered in a way that can be inverted. See ADR-050's [section about invertability](/sdk/v0.50/build/architecture/adr-050-sign-mode-textual#invertible-rendering) for a discussion. +Note: Data longer than 35 bytes are not rendered in a way that can be inverted. See ADR-050's [section about invertibility](/sdk/latest/reference/architecture/adr-050-sign-mode-textual#invertible-rendering) for a discussion. #### Examples diff --git a/sdk/latest/reference/architecture/adr-050-sign-mode-textual-annex2.mdx b/sdk/latest/reference/architecture/adr-050-sign-mode-textual-annex2.mdx index b7070587e..4c9a318c2 100644 --- a/sdk/latest/reference/architecture/adr-050-sign-mode-textual-annex2.mdx +++ b/sdk/latest/reference/architecture/adr-050-sign-mode-textual-annex2.mdx @@ -1,6 +1,6 @@ --- -title: 'ADR 050: SIGN_MODE_TEXTUAL: Annex 2 XXX' -description: 'Oct 3, 2022: Initial Draft' +title: 'ADR 050: SIGN_MODE_TEXTUAL: Annex 2 Device Rendering' +description: 'Normative guidance on how hardware devices should render a SIGN_MODE_TEXTUAL document.' --- ## Changelog @@ -9,7 +9,7 @@ description: 'Oct 3, 2022: Initial Draft' ## Status -DRAFT +Archived. `SIGN_MODE_TEXTUAL` was removed in Cosmos SDK v0.55, and the proto enum value is reserved. This ADR is retained for historical reference. ## Abstract diff --git a/sdk/latest/reference/architecture/adr-050-sign-mode-textual.mdx b/sdk/latest/reference/architecture/adr-050-sign-mode-textual.mdx index ffea66c3a..52969d2bc 100644 --- a/sdk/latest/reference/architecture/adr-050-sign-mode-textual.mdx +++ b/sdk/latest/reference/architecture/adr-050-sign-mode-textual.mdx @@ -20,7 +20,7 @@ title: 'ADR 050: SIGN_MODE_TEXTUAL' ## Status -Accepted. Implementation started. Small value renderers details still need to be polished. +Archived. `SIGN_MODE_TEXTUAL` was removed in Cosmos SDK v0.55, and the proto enum value is reserved. This ADR is retained for historical reference. Spec version: 0. @@ -30,7 +30,7 @@ This ADR specifies SIGN\_MODE\_TEXTUAL, a new string-based sign mode that is tar ## Context -Protobuf-based SIGN\_MODE\_DIRECT was introduced in [ADR-020](/sdk/v0.50/build/architecture/adr-020-protobuf-transaction-encoding) and is intended to replace SIGN\_MODE\_LEGACY\_AMINO\_JSON in most situations, such as mobile wallets and CLI keyrings. However, the [Ledger](https://www.ledger.com/) hardware wallet is still using SIGN\_MODE\_LEGACY\_AMINO\_JSON for displaying the sign bytes to the user. Hardware wallets cannot transition to SIGN\_MODE\_DIRECT as: +Protobuf-based SIGN\_MODE\_DIRECT was introduced in [ADR-020](/sdk/latest/reference/architecture/adr-020-protobuf-transaction-encoding) and is intended to replace SIGN\_MODE\_LEGACY\_AMINO\_JSON in most situations, such as mobile wallets and CLI keyrings. However, the [Ledger](https://www.ledger.com/) hardware wallet is still using SIGN\_MODE\_LEGACY\_AMINO\_JSON for displaying the sign bytes to the user. Hardware wallets cannot transition to SIGN\_MODE\_DIRECT as: * SIGN\_MODE\_DIRECT is binary-based and thus not suitable for display to end-users. Technically, hardware wallets could simply display the sign bytes to the user. But this would be considered as blind signing, and is a security concern. * hardware cannot decode the protobuf sign bytes due to memory constraints, as the Protobuf definitions would need to be embedded on the hardware device. @@ -56,7 +56,7 @@ or to introduce or conclude a larger grouping. The text can contain the full range of Unicode code points, including control characters and nul. The device is responsible for deciding how to display characters it cannot render natively. -See [annex 2](/sdk/v0.50/build/architecture/adr-050-sign-mode-textual-annex2) for guidance. +See [annex 2](/sdk/latest/reference/architecture/adr-050-sign-mode-textual-annex2) for guidance. Screens have a non-negative indentation level to signal composite or nested structures. Indentation level zero is the top level. @@ -287,7 +287,7 @@ Moreover, the renderer must provide 2 functions: one for formatting from Protobu ### Require signing over the `TxBody` and `AuthInfo` raw bytes -Recall that the transaction bytes merklelized on chain are the Protobuf binary serialization of [TxRaw](hhttps://buf.build/cosmos/cosmos-sdk/sdk/v0.50/main:cosmos.tx.v1beta1#cosmos.tx.v1beta1.TxRaw), which contains the `body_bytes` and `auth_info_bytes`. Moreover, the transaction hash is defined as the SHA256 hash of the `TxRaw` bytes. We require that the user signs over these bytes in SIGN\_MODE\_TEXTUAL, more specifically over the following string: +Recall that the transaction bytes merklelized on chain are the Protobuf binary serialization of [TxRaw](https://buf.build/cosmos/cosmos-sdk/sdk/v0.50/main:cosmos.tx.v1beta1#cosmos.tx.v1beta1.TxRaw), which contains the `body_bytes` and `auth_info_bytes`. Moreover, the transaction hash is defined as the SHA256 hash of the `TxRaw` bytes. We require that the user signs over these bytes in SIGN\_MODE\_TEXTUAL, more specifically over the following string: ``` *Hash of raw bytes: @@ -301,7 +301,7 @@ where: This is to prevent transaction hash malleability. The point #1 about invertiblity assures that transaction `body` and `auth_info` values are not malleable, but the transaction hash still might be malleable with point #1 only, because the SIGN\_MODE\_TEXTUAL strings don't follow the byte ordering defined in `body_bytes` and `auth_info_bytes`. Without this hash, a malicious validator or exchange could intercept a transaction, modify its transaction hash *after* the user signed it using SIGN\_MODE\_TEXTUAL (by tweaking the byte ordering inside `body_bytes` or `auth_info_bytes`), and then submit it to Tendermint. -By including this hash in the SIGN\_MODE\_TEXTUAL signing payload, we keep the same level of guarantees as [SIGN\_MODE\_DIRECT](/sdk/v0.50/build/architecture/adr-020-protobuf-transaction-encoding). +By including this hash in the SIGN\_MODE\_TEXTUAL signing payload, we keep the same level of guarantees as [SIGN\_MODE\_DIRECT](/sdk/latest/reference/architecture/adr-020-protobuf-transaction-encoding). These bytes are only shown in expert mode, hence the leading `*`. @@ -322,7 +322,7 @@ The current spec version is defined in the "Status" section, on the top of this ## Additional Formatting by the Hardware Device -See [annex 2](/sdk/v0.50/build/architecture/adr-050-sign-mode-textual-annex2). +See [annex 2](/sdk/latest/reference/architecture/adr-050-sign-mode-textual-annex2). ## Examples @@ -357,14 +357,14 @@ SIGN\_MODE\_TEXTUAL is purely additive, and doesn't break any backwards compatib ## Further Discussions -* Some details on value renderers need to be polished, see [Annex 1](/sdk/v0.50/build/architecture/adr-050-sign-mode-textual-annex1). +* Some details on value renderers need to be polished, see [Annex 1](/sdk/latest/reference/architecture/adr-050-sign-mode-textual-annex1). * Are ledger apps able to support both SIGN\_MODE\_LEGACY\_AMINO\_JSON and SIGN\_MODE\_TEXTUAL at the same time? * Open question: should we add a Protobuf field option to allow app developers to overwrite the textual representation of certain Protobuf fields and message? This would be similar to Ethereum's [EIP4430](https://github.com/ethereum/EIPs/pull/4430), where the contract developer decides on the textual representation. * Internationalization. ## References -* [Annex 1](/sdk/v0.50/build/architecture/adr-050-sign-mode-textual-annex1) +* [Annex 1](/sdk/latest/reference/architecture/adr-050-sign-mode-textual-annex1) * Initial discussion: [Link](https://github.com/cosmos/cosmos-sdk/issues/6513) diff --git a/sdk/latest/reference/architecture/adr-062-collections-state-layer.mdx b/sdk/latest/reference/architecture/adr-062-collections-state-layer.mdx index 37c565dd8..a54f3d9f7 100644 --- a/sdk/latest/reference/architecture/adr-062-collections-state-layer.mdx +++ b/sdk/latest/reference/architecture/adr-062-collections-state-layer.mdx @@ -83,7 +83,7 @@ These default implementations also offer safety around proper lexicographic orde Examples of the collections API can be found here: * introduction: [Link](https://github.com/NibiruChain/collections/tree/main/examples) -* usage in nibiru: [x/oracle](https://github.com/NibiruChain/nibiru/blob/master/x/oracle/keeper/keeper.go#L32), [x/perp](https://github.com/NibiruChain/nibiru/blob/master/x/perp/keeper/keeper.go#L31) +* usage in nibiru: [x/oracle](https://github.com/NibiruChain/nibiru/blob/master/x/oracle/keeper/keeper.go#L32), x/perp (`https://github.com/NibiruChain/nibiru/blob/master/x/perp/keeper/keeper.go#L31`) * cosmos-sdk's x/staking migrated: [Link](https://github.com/testinginprod/cosmos-sdk/pull/22) ## Consequences diff --git a/sdk/latest/reference/architecture/adr-template.mdx b/sdk/latest/reference/architecture/adr-template.mdx deleted file mode 100644 index 1ce9549f1..000000000 --- a/sdk/latest/reference/architecture/adr-template.mdx +++ /dev/null @@ -1,82 +0,0 @@ -## Changelog - -* `{date}`: `{changelog}` - -## Status - -{DRAFT | PROPOSED} Not Implemented - -> Please have a look at the [PROCESS](/sdk/v0.50/build/rfc/PROCESS#adr-status) page. -> Use DRAFT if the ADR is in a draft stage (draft PR) or PROPOSED if it's in review. - -## Abstract - -> "If you can't explain it simply, you don't understand it well enough." Provide -> a simplified and layman-accessible explanation of the ADR. -> A short (\~200 word) description of the issue being addressed. - -## Context - -> This section describes the forces at play, including technological, political, -> social, and project local. These forces are probably in tension, and should be -> called out as such. The language in this section is value-neutral. It is simply -> describing facts. It should clearly explain the problem and motivation that the -> proposal aims to resolve. - -`{context body}` - -## Alternatives - -> This section describes alternative designs to the chosen design. This section -> is important and if an adr does not have any alternatives then it should be -> considered that the ADR was not thought through. - -## Decision - -> This section describes our response to these forces. It is stated in full -> sentences, with active voice. "We will ..." -> `{decision body}` - -## Consequences - -> This section describes the resulting context, after applying the decision. All -> consequences should be listed here, not just the "positive" ones. A particular -> decision may have positive, negative, and neutral consequences, but all of them -> affect the team and project in the future. - -### Backwards Compatibility - -> All ADRs that introduce backwards incompatibilities must include a section -> describing these incompatibilities and their severity. The ADR must explain -> how the author proposes to deal with these incompatibilities. ADR submissions -> without a sufficient backwards compatibility treatise may be rejected outright. - -### Positive - -> `{positive consequences}` - -### Negative - -> `{negative consequences}` - -### Neutral - -> `{neutral consequences}` - -## Further Discussions - -> While an ADR is in the DRAFT or PROPOSED stage, this section should contain a -> summary of issues to be solved in future iterations (usually referencing comments -> from a pull-request discussion). -> -> Later, this section can optionally list ideas or improvements the author or -> reviewers found during the analysis of this ADR. - -## Test Cases \[optional] - -Test cases for an implementation are mandatory for ADRs that are affecting consensus -changes. Other ADRs can choose to include links to test cases if applicable. - -## References - -* `{reference link}` diff --git a/sdk/latest/reference/rfc.mdx b/sdk/latest/reference/rfc.mdx index 624faf008..53e4ce6f9 100644 --- a/sdk/latest/reference/rfc.mdx +++ b/sdk/latest/reference/rfc.mdx @@ -1,6 +1,6 @@ --- title: "Requests for Comments" -description: "Version: v0.54" +description: "Version: v0.55" --- A Request for Comments (RFC) is a record of discussion on an open-ended topic related to the design and implementation of the Cosmos SDK, for which no immediate decision is required. @@ -18,7 +18,7 @@ An RFC should provide: * Any **background** a reader will need to understand and participate in the substance of the discussion (links to other documents are fine here). * The **discussion**, the primary content of the document. -The [rfc-template.md](/sdk/latest/reference/rfc/rfc-template) file includes placeholders for these sections. +The `rfc-template.md` file includes placeholders for these sections. ## Table of Contents[​](#table-of-contents "Direct link to Table of Contents") diff --git a/sdk/latest/reference/rfc/README.mdx b/sdk/latest/reference/rfc/README.mdx index b347d4662..3e978090a 100644 --- a/sdk/latest/reference/rfc/README.mdx +++ b/sdk/latest/reference/rfc/README.mdx @@ -32,7 +32,7 @@ An RFC should provide: substance of the discussion (links to other documents are fine here). * The **discussion**, the primary content of the document. -The [rfc-template.md](/sdk/v0.50/build/rfc/rfc-template) file includes placeholders for these +The `rfc-template.md` file includes placeholders for these sections. ## Table of Contents diff --git a/sdk/latest/reference/rfc/rfc-template.mdx b/sdk/latest/reference/rfc/rfc-template.mdx deleted file mode 100644 index 94f13c6a0..000000000 --- a/sdk/latest/reference/rfc/rfc-template.mdx +++ /dev/null @@ -1,77 +0,0 @@ -## Changelog - -* `{date}`: `{changelog}` - -## Background - -> The next section is the "Background" section. This section should be at least two paragraphs and can take up to a whole -> page in some cases. The guiding goal of the background section is: as a newcomer to this project (new employee, team -> transfer), can I read the background section and follow any links to get the full context of why this change is\ -> necessary? -> -> If you can't show a random engineer the background section and have them acquire nearly full context on the necessity -> for the RFC, then the background section is not full enough. To help achieve this, link to prior RFCs, discussions, and -> more here as necessary to provide context so you don't have to simply repeat yourself. - -## Proposal - -> The next required section is "Proposal" or "Goal". Given the background above, this section proposes a solution. -> This should be an overview of the "how" for the solution, but for details further sections will be used. - -## Abandoned Ideas (Optional) - -> As RFCs evolve, it is common that there are ideas that are abandoned. Rather than simply deleting them from the -> document, you should try to organize them into sections that make it clear they're abandoned while explaining why they -> were abandoned. -> -> When sharing your RFC with others or having someone look back on your RFC in the future, it is common to walk the same -> path and fall into the same pitfalls that we've since matured from. Abandoned ideas are a way to recognize that path -> and explain the pitfalls and why they were abandoned. - -## Decision - -> This section describes alternative designs to the chosen design. This section -> is important and if an ADR does not have any alternatives then it should be -> considered that the ADR was not thought through. - -## Consequences (optional) - -> This section describes the resulting context, after applying the decision. All -> consequences should be listed here, not just the "positive" ones. A particular -> decision may have positive, negative, and neutral consequences, but all of them -> affect the team and project in the future. - -### Backwards Compatibility - -> All ADRs that introduce backwards incompatibilities must include a section -> describing these incompatibilities and their severity. The ADR must explain -> how the author proposes to deal with these incompatibilities. ADR submissions -> without a sufficient backwards compatibility treatise may be rejected outright. - -### Positive - -> `{positive consequences}` - -### Negative - -> `{negative consequences}` - -### Neutral - -> `{neutral consequences}` - -### References - -> Links to external materials needed to follow the discussion may be added here. -> -> In addition, if the discussion in a request for comments leads to any design -> decisions, it may be helpful to add links to the ADR documents here after the -> discussion has settled. - -## Discussion - -> This section contains the core of the discussion. -> -> There is no fixed format for this section, but ideally changes to this -> section should be updated before merging to reflect any discussion that took -> place on the PR that made those changes. diff --git a/sdk/latest/reference/spec.mdx b/sdk/latest/reference/spec.mdx index f63e57d1e..3064cf20b 100644 --- a/sdk/latest/reference/spec.mdx +++ b/sdk/latest/reference/spec.mdx @@ -1,6 +1,6 @@ --- title: "Specifications" -description: "Version: v0.54" +description: "Version: v0.55" --- This directory contains specifications for the modules of the Cosmos SDK as well as Interchain Standards (ICS) and other specifications. @@ -18,4 +18,4 @@ Go the [module directory](/sdk/latest/modules/modules) ## CometBFT[​](#cometbft "Direct link to CometBFT") -For details on the underlying blockchain and p2p protocols, see the [CometBFT specification](https://github.com/cometbft/cometbft/tree/main/spec). +For details on the underlying blockchain and p2p protocols, see the [CometBFT specification](https://github.com/cometbft/cometbft/tree/v0.40.x/spec). diff --git a/sdk/latest/reference/spec/README.mdx b/sdk/latest/reference/spec/README.mdx index 573ad8edb..dceaa8ee1 100644 --- a/sdk/latest/reference/spec/README.mdx +++ b/sdk/latest/reference/spec/README.mdx @@ -23,4 +23,4 @@ Go the [module directory](/sdk/latest/modules/modules) ## CometBFT For details on the underlying blockchain and p2p protocols, see -the [CometBFT specification](https://github.com/cometbft/cometbft/tree/main/spec). +the [CometBFT specification](https://github.com/cometbft/cometbft/tree/v0.40.x/spec). diff --git a/sdk/latest/reference/spec/_ics/ics-030-signed-messages.mdx b/sdk/latest/reference/spec/_ics/ics-030-signed-messages.mdx index 1b3d2481a..ff9d88fe0 100644 --- a/sdk/latest/reference/spec/_ics/ics-030-signed-messages.mdx +++ b/sdk/latest/reference/spec/_ics/ics-030-signed-messages.mdx @@ -63,7 +63,7 @@ pre-image attacks, as well as being [deterministic](https://en.wikipedia.org/wik ## Specification CometBFT has a well established protocol for signing messages using a canonical -JSON representation as defined [here](https://github.com/cometbft/cometbft/blob/master/types/canonical.go). +JSON representation as defined [here](https://github.com/cometbft/cometbft/blob/v0.40.x/types/canonical.go). An example of such a canonical JSON structure is CometBFT's vote structure: diff --git a/sdk/latest/release-family.mdx b/sdk/latest/release-family.mdx index cffe7b65f..51da34d64 100644 --- a/sdk/latest/release-family.mdx +++ b/sdk/latest/release-family.mdx @@ -31,14 +31,14 @@ Certain packages within the SDK may not be listed as Cosmos Labs consolidates se | Component | Version | | --------- | ------- | -| [Cosmos SDK](https://github.com/cosmos/cosmos-sdk) | 0.54.x | -| [Enterprise Groups](https://github.com/cosmos/cosmos-sdk/tree/main/enterprise/group) | 1.x.y | -| [Enterprise PoA](https://github.com/cosmos/cosmos-sdk/tree/main/enterprise/poa) | 1.x.y | -| [CometBFT](https://github.com/cometbft/cometbft) | 0.39.x | +| [Cosmos SDK](https://github.com/cosmos/cosmos-sdk) | 0.55.x | +| [Enterprise Groups](https://github.com/cosmos/cosmos-sdk/tree/release/v0.55.x/enterprise/group) | 1.x.y | +| [Enterprise PoA](https://github.com/cosmos/cosmos-sdk/tree/release/v0.55.x/enterprise/poa) | 1.x.y | +| [CometBFT](https://github.com/cometbft/cometbft) | 0.40.x | | [IBC Go](https://github.com/cosmos/ibc-go) | v11.x.y | -| [Solidity IBC Eureka](https://github.com/cosmos/solidity-ibc-eureka) | 0.1.x | -| [Relayer](https://github.com/cosmos/ibc-relayer) | 0.1.x | -| [Attestor](https://github.com/cosmos/ibc-attestor) | 0.1.x | +| [Solidity IBC Eureka](https://github.com/cosmos/solidity-ibc-eureka) | 3.0.x | +| [Relayer](https://github.com/cosmos/ibc-relayer) | 1.1.x | +| [Attestor](https://github.com/cosmos/ibc-attestor) | 1.0.x | ### 2025.1 @@ -63,30 +63,6 @@ Lifecycle policy applies to families, not individual component versions in isola For security reporting and vulnerability handling details, see the [Security and Maintenance Policy](/sdk/latest/security/security-policy). -## Examples - -An example release family might look like: - -| Component | Version | -| --------- | ------- | -| Cosmos SDK | 0.54.0 | -| CometBFT | 0.39.0 | -| IBC Go | v11.0.0 | -| Solidity IBC Eureka | 0.1.0 | -| Relayer | 0.1.0 | -| Attestor | 0.1.0 | - -Upgrades that would update versions within this family without creating a new one: - -- SDK 0.54.0 to 0.54.1 -- Relayer 0.1.0 to 0.1.1 - -Upgrades that would require a new release family: - -- SDK 0.54.x to 0.55.0 -- CometBFT 0.39.x to 0.40.x -- Relayer 0.1.x to 1.0.0 - ## End of Life Notices The following releases are end of life and no longer receive maintenance, security patches, or compatibility support from Cosmos Labs: diff --git a/sdk/latest/security/audits.mdx b/sdk/latest/security/audits.mdx index 2fe157d63..3e8b66f27 100644 --- a/sdk/latest/security/audits.mdx +++ b/sdk/latest/security/audits.mdx @@ -6,7 +6,7 @@ description: "Security audits and transparency reports for Cosmos Stack componen This page is auto-generated from the [cosmos/security](https://github.com/cosmos/security) repository. -**Last synced:** Apr 27, 2026 | [View all audits](https://github.com/cosmos/security/tree/main/audits) +**Last synced:** Apr 10, 2026 | [View all audits](https://github.com/cosmos/security/tree/main/audits) Cosmos Labs maintains a comprehensive security program for all Cosmos Stack components. This page provides links to third-party security audits and transparency reports. diff --git a/sdk/latest/security/bug-bounty.mdx b/sdk/latest/security/bug-bounty.mdx index 512591b0a..338e70499 100644 --- a/sdk/latest/security/bug-bounty.mdx +++ b/sdk/latest/security/bug-bounty.mdx @@ -6,7 +6,7 @@ description: "Security and maintenance policy documentation for the Cosmos Stack This content is sourced from the official [Cosmos Security](https://github.com/cosmos/security) repository. -**Last sync:** Apr 27, 2026 | [View source](https://github.com/cosmos/security/blob/main/SECURITY.md) +**Last sync:** Apr 10, 2026 | [View source](https://github.com/cosmos/security/blob/main/SECURITY.md) ## Introduction diff --git a/sdk/latest/security/security-policy.mdx b/sdk/latest/security/security-policy.mdx index 53b5c7f67..d58ebab38 100644 --- a/sdk/latest/security/security-policy.mdx +++ b/sdk/latest/security/security-policy.mdx @@ -6,7 +6,7 @@ description: "Security and maintenance policy documentation for the Cosmos Stack This content is sourced from the official [Cosmos Security](https://github.com/cosmos/security) repository. -**Last sync:** Apr 27, 2026 | [View source](https://github.com/cosmos/security/blob/main/POLICY.md) +**Last sync:** Apr 10, 2026 | [View source](https://github.com/cosmos/security/blob/main/POLICY.md) ## Overview diff --git a/sdk/latest/tutorials.mdx b/sdk/latest/tutorials.mdx index 4dfc457f0..dfacc18b0 100644 --- a/sdk/latest/tutorials.mdx +++ b/sdk/latest/tutorials.mdx @@ -1,11 +1,11 @@ --- title: "Node Tutorial" -description: "Version: v0.54" +description: "Version: v0.55" --- This guide covers everything you need to run, configure, and maintain a Cosmos SDK node. Whether you're setting up a local development node, deploying to a testnet, or running production infrastructure, you'll find step-by-step instructions and best practices. -The node tutorial uses the `simapp` example application and its corresponding CLI binary `simd` as the blockchain application and CLI. You can view the source code for `simapp` [on GitHub](https://github.com/cosmos/cosmos-sdk/tree/main/simapp). +The node tutorial uses the `simapp` example application and its corresponding CLI binary `simd` as the blockchain application and CLI. You can view the source code for `simapp` [on GitHub](https://github.com/cosmos/cosmos-sdk/tree/release/v0.55.x/simapp). diff --git a/sdk/latest/upgrade/release.mdx b/sdk/latest/upgrade/v0.54-release.mdx similarity index 98% rename from sdk/latest/upgrade/release.mdx rename to sdk/latest/upgrade/v0.54-release.mdx index 612cb0d45..152e5cb07 100644 --- a/sdk/latest/upgrade/release.mdx +++ b/sdk/latest/upgrade/v0.54-release.mdx @@ -4,7 +4,7 @@ description: "What's new in the latest Cosmos SDK release, including performance --- - If you are upgrading to v0.54, see the [upgrade guide](/sdk/latest/upgrade/upgrade). For a full list of changes, see the [changelog](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/CHANGELOG.md). + If you are upgrading to v0.54, see the [upgrade guide](/sdk/latest/upgrade/v0.54). For a full list of changes, see the [changelog](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/CHANGELOG.md). ## Overview diff --git a/sdk/latest/upgrade/upgrade.mdx b/sdk/latest/upgrade/v0.54.mdx similarity index 99% rename from sdk/latest/upgrade/upgrade.mdx rename to sdk/latest/upgrade/v0.54.mdx index 103342c3b..3959a9aff 100644 --- a/sdk/latest/upgrade/upgrade.mdx +++ b/sdk/latest/upgrade/v0.54.mdx @@ -81,7 +81,7 @@ This guide provides an overview of the major changes in v0.54.0. However, this g #### x/gov -#### Keeper Initialization +##### Keeper Initialization The `x/gov` module has been decoupled from `x/staking`. The `keeper.NewKeeper` constructor now requires a `CalculateVoteResultsAndVotingPowerFn` parameter instead of a `StakingKeeper`. @@ -117,7 +117,7 @@ govKeeper := govkeeper.NewKeeper( For applications using depinject, the governance module now accepts an optional `CalculateVoteResultsAndVotingPowerFn`. If not provided, it will use the `StakingKeeper` (also optional) to create the default function. -#### GovHooks Interface +##### GovHooks Interface The `AfterProposalSubmission` hook now includes the proposer address as a parameter. diff --git a/sdk/latest/upgrade/v0.55-release.mdx b/sdk/latest/upgrade/v0.55-release.mdx new file mode 100644 index 000000000..48b5ca847 --- /dev/null +++ b/sdk/latest/upgrade/v0.55-release.mdx @@ -0,0 +1,78 @@ +--- +title: "v0.55 Release Notes" +description: "What's new in the 2026.1 Ledger Security release: post-quantum keys, validator consensus key rotation, and remote signing with Cosmos-KMS." +--- + + + If you are upgrading to v0.55, see the [upgrade guide](/sdk/latest/upgrade/v0.55). For a full list of changes, see the [changelog](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/CHANGELOG.md). + + + +## Overview + +This release is a holistic upgrade to the security of the Cosmos Stack. It adds the first native post-quantum key option in Cosmos, in-place validator consensus key rotation with no downtime, and a remote signer that keeps validator keys in your own KMS or HSM. + +All four artifacts ship together and join the existing 2026.1 release family. For the versions each family pins, see [Release Families](/sdk/latest/release-family). + +## What ships + +| Artifact | Version | What changed | +| -------- | ------- | ------------ | +| [Cosmos SDK](https://github.com/cosmos/cosmos-sdk) | v0.55.0 | ML-DSA account and consensus keys, consensus key rotation through `x/staking`, keyring key types | +| [CometBFT](https://github.com/cometbft/cometbft) | v0.40.0 | ML-DSA consensus key support and remote signer compatibility | +| [enterprise/poa](https://github.com/cosmos/cosmos-sdk/tree/release/v0.55.x/enterprise/poa) | v1.1.0 | Consensus key rotation for PoA validators, by the operator or the chain admin | +| [cosmos-kms](https://github.com/cosmos/kms) | v1.0.0 | First release of the remote signer | + +## Features + +### Post-quantum keys (ML-DSA) + +Chains can run ML-DSA for consensus and user-account keys. ML-DSA keys use lattice-based signatures, which are considered more quantum resistant than elliptic-curve-based keys. New chains set the allowed key types through consensus params; existing chains migrate one validator at a time, and a validator-led path moves a classical key to ML-DSA in place with no hard fork. + +A chain reaches post-quantum security once validators holding two-thirds of voting power have rotated to ML-DSA keys, the same threshold CometBFT uses to finalize blocks. + +See [Post-quantum keys](/sdk/latest/keys/post-quantum-keys) for the tradeoffs, [Enable ML-DSA keys](/sdk/latest/keys/enable-ml-dsa-keys) to allow the type on a chain, and [Migrate a validator to ML-DSA](/sdk/latest/keys/migrate-validator-ml-dsa) for the per-validator path. + +### Validator consensus key rotation + +Staked validators rotate a consensus key in place, keeping the validator's address, voting power, and accumulated fees, so the rotation stays invisible to delegators. Before this, a compromised or policy-expired consensus key meant standing up a new validator and rebuilding the delegator base. + +The operator submits `MsgRotateConsPubKey` with the new consensus public key, and CometBFT applies the change two heights later, which lets the operator bring up the new node with no downtime. Each rotation burns the `key_rotation_fee` staking parameter, a validator can rotate once per unbonding period, and a rotated-away key stays attributable for slashing until equivocation evidence for it can no longer be admitted. + +See [Key rotation](/sdk/latest/keys/key-rotation) for the mechanics and security implications, and [Rotate a consensus key, Staking](/sdk/latest/keys/rotate-validator-key) for the procedure. PoA chains follow [Rotate a consensus key, PoA](/sdk/latest/keys/rotate-validator-key-poa). + +### Remote signing with Cosmos-KMS + +`cosmos-kms` is a new remote signing solution that signs on the validator's behalf while keys stay in your own HSM or cloud KMS rather than in local files on the node. It adds AWS KMS and PKCS#11 backends and post-quantum ML-DSA signing, none of which TMKMS supported. + +See [Cosmos-KMS and remote signing](/sdk/latest/kms/remote-signing) for the architecture, and the [remote signing tutorial](/sdk/latest/kms/tutorial-file-backend) to run one against a local chain. + +## Removals and deprecations + +### TMKMS deprecation notice + +This release begins the deprecation of TMKMS. TMKMS reaches official deprecation six months from this release, so operators running it have that window to move to `cosmos-kms`. + +Validators using TMKMS should migrate. See [Migrate from TMKMS](/sdk/latest/kms/migrate-from-tmkms), which covers moving each TMKMS backend to `cosmos-kms`. + +### Removed in v0.55 + +- `x/params`, replaced by per-module params. +- `x/protocolpool`, with the community pool returning to `x/distribution`. +- `SIGN_MODE_TEXTUAL`. + +See the [upgrade guide](/sdk/latest/upgrade/v0.55) for the wiring changes each removal requires. + +## Upgrading + +Upgrading to Cosmos SDK v0.55.0 bumps CometBFT to v0.40.0 automatically, so you do not upgrade CometBFT separately. Coordinate the upgrade across the validator set, since it moves the SDK and CometBFT together. + +We document the [0.54 to 0.55 upgrade path](/sdk/latest/upgrade/v0.55), which also includes a [section on upgrading from 0.53 directly to 0.55](/sdk/latest/upgrade/v0.55#upgrading-from-v0-53-x). Module upgrades work across the last two SDK versions. + +## Upcoming + +The following features are planned for a future release: + +- Enterprise HSM and key custody. AWS KMS supports ML-DSA signatures through Cosmos-KMS in this release. Other HSM and KMS solutions will be supported in a future release. +- Post-quantum support for attestors and signers. +- Ledger-layer confidential transactions. diff --git a/sdk/latest/upgrade/v0.55.mdx b/sdk/latest/upgrade/v0.55.mdx new file mode 100644 index 000000000..70f30f233 --- /dev/null +++ b/sdk/latest/upgrade/v0.55.mdx @@ -0,0 +1,290 @@ +--- +title: "v0.55 Upgrade Guide" +description: "Reference for upgrading to v0.55 of Cosmos SDK" +--- + +This document provides a reference for upgrading from `v0.54.x` to `v0.55.x` of Cosmos SDK. If you are upgrading directly from `v0.53.x`, see [Upgrading from v0.53.x](#upgrading-from-v0-53-x) after reading the breaking changes below. + +For a full list of changes, see the [Changelog](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/CHANGELOG.md). + +The headline changes in this release are the removal of three legacy surfaces (`x/params`, `x/protocolpool`, and `SIGN_MODE_TEXTUAL`), a reworked app-side mempool interface, and validator consensus key rotation in `x/staking`. Key rotation ships enabled for every chain that upgrades — see [Validator Consensus Key Rotation](#validator-consensus-key-rotation) — and requires one line of wiring in `app.go`. Everything else in the new-features list (ML-DSA-65 keys, secp256k1eth keys, config-driven Block-STM wiring) is opt-in. + +## Table of Contents + +* [Breaking Changes](#breaking-changes) + * [CometBFT Upgrade](#cometbft-upgrade) + * [Removed: x/params](#removed-xparams) + * [Removed: x/protocolpool](#removed-xprotocolpool) + * [Removed: SIGN_MODE_TEXTUAL](#removed-sign_mode_textual) + * [Mempool Interface Changes](#mempool-interface-changes) + * [Staking: Key Rotation Wiring and Interface Changes](#staking-key-rotation-wiring-and-interface-changes) + * [genutil: ExportGenesisFileWithTime Signature](#genutil-exportgenesisfilewithtime-signature) + * [Upgrade Handler and Store Migrations](#upgrade-handler-and-store-migrations) +* [Upgrading from v0.53.x](#upgrading-from-v053x) +* [New Features and Non-Breaking Changes](#new-features-and-non-breaking-changes) + * [Validator Consensus Key Rotation](#validator-consensus-key-rotation) + * [ML-DSA-65 Validator Consensus Keys](#ml-dsa-65-validator-consensus-keys) + * [ML-DSA-65 Account Keys](#ml-dsa-65-account-keys) + * [secp256k1eth Validator Consensus Keys](#secp256k1eth-validator-consensus-keys) + * [Block-STM Configuration](#block-stm-configuration) +* [Behavior Changes Affecting Dapps and Indexers](#behavior-changes-affecting-dapps-and-indexers) + +## Breaking Changes + +### CometBFT Upgrade + +Cosmos SDK v0.55 requires CometBFT `v0.40.0` (the v0.54.x line shipped with `v0.39.x`, ending at `v0.39.3` in v0.54.3). Bump your app's `go.mod` to match the SDK's pin. Relevant changes in CometBFT v0.40.0: + +* Expanded `MaxSignatureSize` and per-validator `MaxCommitSigBytes` to accommodate post-quantum (ML-DSA-65) signatures. +* A fix for the application-side mempool (`mempool.type = "app"`, supported since CometBFT v0.39.2 / SDK v0.54.3): the default socket transport was missing the `InsertTx` / `ReapTxs` cases, causing node self-kill ([cometbft#5958](https://github.com/cometbft/cometbft/pull/5958)). Chains using an app-side mempool over the socket transport need v0.40.0. +* Updated `DefaultBlockParams` ([cometbft#5987](https://github.com/cometbft/cometbft/pull/5987)). This changes defaults for new chains only; existing chains keep their on-chain consensus params. + +See the [CometBFT changelog](https://github.com/cometbft/cometbft/blob/main/CHANGELOG.md) for the full list. + +### Removed: x/params + +[#25546](https://github.com/cosmos/cosmos-sdk/pull/25546) removes the `x/params` module entirely (only a tombstone README remains). Module parameters have been managed by each module since v0.47; v0.55 removes the leftover machinery: + +1. If your app still imports `x/params` (a `paramskeeper.Keeper`, per-module `Subspace`s, or the legacy gov proposal handler), remove that wiring. If the `params` store is still mounted, delete it in your store upgrades (see [Upgrade Handler and Store Migrations](#upgrade-handler-and-store-migrations)). Chains that have not yet migrated legacy subspace params to module-managed params must complete that migration **before** upgrading to v0.55 — the migration code is gone. + +2. Drop the trailing `exported.Subspace` argument (typically passed as `nil`) from the module constructors that carried it for legacy migrations: + +```go +// Before // After +auth.NewAppModule(cdc, accountKeeper, randGenAccountsFn, nil) auth.NewAppModule(cdc, accountKeeper, randGenAccountsFn) +bank.NewAppModule(cdc, bankKeeper, accountKeeper, nil) bank.NewAppModule(cdc, bankKeeper, accountKeeper) +gov.NewAppModule(cdc, &govKeeper, accountKeeper, bankKeeper, nil) gov.NewAppModule(cdc, &govKeeper, accountKeeper, bankKeeper) +mint.NewAppModule(cdc, mintKeeper, accountKeeper, nil, nil) mint.NewAppModule(cdc, mintKeeper, accountKeeper, nil) +slashing.NewAppModule(cdc, keeper, ak, bk, sk, nil, registry) slashing.NewAppModule(cdc, keeper, ak, bk, sk, registry) +distr.NewAppModule(cdc, keeper, ak, bk, stakingKeeper, nil) distr.NewAppModule(cdc, keeper, ak, bk, stakingKeeper) +staking.NewAppModule(cdc, keeper, ak, bk, nil) staking.NewAppModule(cdc, keeper, ak, bk) +``` + +(`mint.NewAppModule` retains its deprecated `InflationCalculationFn` parameter; only the subspace argument is removed.) + +### Removed: x/protocolpool + +[#26421](https://github.com/cosmos/cosmos-sdk/pull/26421) removes the `x/protocolpool` module and its proto/API surface from the SDK. The `distrkeeper.WithExternalCommunityPool` extension point is removed with it — `x/distribution` always uses its internal `FeePool` community pool again, and `MsgFundCommunityPool` / `MsgCommunityPoolSpend` operate on it directly. + +**Required action** if your app wired `x/protocolpool` (the v0.54 SimApp default): + +1. Remove all `protocolpool` wiring from `app.go`: the imports, the `ProtocolPoolKeeper` field and its `NewKeeper` call, the `protocolpooltypes.ModuleName` and `protocolpooltypes.ProtocolPoolEscrowAccount` entries in `maccPerms`, the module manager entry, and its entries in the begin-block, end-block, init-genesis, and export orders. +2. Remove `distrkeeper.WithExternalCommunityPool(app.ProtocolPoolKeeper)` from your `distrkeeper.NewKeeper` call. +3. Delete the `protocolpool` store in your store upgrades (see [Upgrade Handler and Store Migrations](#upgrade-handler-and-store-migrations)). +4. Balances held by the protocolpool module accounts are bank state and are **not** migrated automatically. Decide where those funds go and move them in your upgrade handler — e.g. transfer them to the `x/distribution` community pool so community-pool spend proposals keep working. + +If your app never wired `x/protocolpool`, no action is needed beyond not being able to import it. + +### Removed: SIGN_MODE_TEXTUAL + +`SIGN_MODE_TEXTUAL` (proto enum value `2`) and its entire implementation have been removed ([#26456](https://github.com/cosmos/cosmos-sdk/pull/26456)): + +* `x/tx/signing/textual/` — all renderers, the CBOR encoder, test data, and internal protos +* `x/auth/tx/textual.go` and `ConfigOptions.TextualCoinMetadataQueryFn` +* Ledger + SIGN_MODE_TEXTUAL integration in `client/` flags and tx factory + +The proto enum value `2` and string `"SIGN_MODE_TEXTUAL"` are **reserved** to prevent future reuse. ADR-050 is archived. + +**Required action** if your app enabled SIGN_MODE_TEXTUAL: + +1. Remove `TextualCoinMetadataQueryFn` from your `tx.ConfigOptions`: + + ```go + // Before + txConfig, err := tx.NewTxConfigWithOptions(cdc, tx.ConfigOptions{ + TextualCoinMetadataQueryFn: ..., + }) + + // After — field removed, omit it + txConfig, err := tx.NewTxConfigWithOptions(cdc, tx.ConfigOptions{...}) + ``` + +2. Remove any `SIGN_MODE_TEXTUAL` cases from signing mode handler switch statements. + +3. Remove Ledger wiring that depended on `SIGN_MODE_TEXTUAL`. Client-side root command wiring that constructed a textual-enabled tx config for online mode (as v0.54 SimApp did in `simd/cmd/root.go`) should be deleted as well. + +### Mempool Interface Changes + +[#25338](https://github.com/cosmos/cosmos-sdk/pull/25338) changes the `types/mempool` interfaces so the mempool stores the gas wanted reported by the ante handler at `CheckTx` time, and block selection uses that value instead of the tx-declared gas limit. + +**Required action** if you implement a custom mempool (chains using the SDK's built-in mempools or no app-side mempool just recompile): + +* `Insert` gains an `InsertOption` parameter carrying the ante-reported gas: `Insert(context.Context, sdk.Tx, InsertOption) error`. +* `Iterator.Tx()` now returns a `PooledTx` (`{Tx sdk.Tx; GasWanted uint64}`) instead of `sdk.Tx`. +* `ExtMempool.SelectBy`'s callback now receives a `PooledTx`: `SelectBy(context.Context, [][]byte, func(PooledTx) bool)`. +* `ExtMempool.RemoveWithReason` and the `RemoveReason` type, introduced in v0.54, are unchanged. + +Custom `PrepareProposal` handlers that iterate the mempool should read gas from `PooledTx.GasWanted` rather than re-deriving it from the tx. + +### Staking: Key Rotation Wiring and Interface Changes + +`x/staking` now requires a `key_rotation_fee_pool` module account with burn permissions — the staking keeper panics at construction if it is missing (`x/staking/keeper/keeper.go`). Add it to your `maccPerms`: + +```go +maccPerms = map[string][]string{ + // ...existing entries... + stakingtypes.KeyRotationFeePoolName: {authtypes.Burner}, +} +``` + +This is required for **all** chains upgrading to v0.55, whether or not validators are expected to use [key rotation](#validator-consensus-key-rotation). + +Key rotation also touches two staking keeper surfaces that external modules may implement or consume: + +* The `StakingHooks` interface gains `AfterValidatorConsKeyUpdated(ctx context.Context, oldConsAddr, newConsAddr sdk.ConsAddress, valAddr sdk.ValAddress) error`, called when a rotation is applied. Custom `StakingHooks` implementations must add this method (returning `nil` is fine if you don't need the notification). +* The staking keeper adds `ValidatorByHistoricalConsAddr(ctx, consAddr)`, which resolves a validator from a consensus address it used before a rotation. Modules that map consensus addresses to validators can no longer assume that mapping is immutable — see [Validator Consensus Key Rotation](#validator-consensus-key-rotation). + +### genutil: ExportGenesisFileWithTime Signature + +[#26468](https://github.com/cosmos/cosmos-sdk/pull/26468) consolidates `ExportGenesisFileWithTime`'s arguments so the exported file preserves consensus params (previously they were rebuilt from defaults, dropping the caller's values): + +```go +// Before +func ExportGenesisFileWithTime(genFile, chainID string, validators []cmttypes.GenesisValidator, + appState json.RawMessage, genTime time.Time) error + +// After — build the AppGenesis yourself; everything you set on it is preserved +func ExportGenesisFileWithTime(genFile string, appGenesis *types.AppGenesis, genTime time.Time) error +``` + +### Upgrade Handler and Store Migrations + +#### Module Migrations + +Two module consensus-version bumps ship in this release and run automatically via `RunMigrations` in your upgrade handler: + +* `x/staking` 5 → 6: adds the `key_rotation_fee` param, defaulting to `1000000` of the bond denom ([#26485](https://github.com/cosmos/cosmos-sdk/pull/26485)). `Params.Validate` requires the fee denom to equal `bond_denom` ([#26613](https://github.com/cosmos/cosmos-sdk/pull/26613)). +* `x/auth` 6 → 7: adds the `SigVerifyCostMlDsa65` param with its default value ([#26472](https://github.com/cosmos/cosmos-sdk/pull/26472)). + +#### Reference Upgrade Handler + +A reference upgrade handler for this release (see `simapp/upgrades.go`): + +```go +const UpgradeName = "v054-to-v055" + +func (app SimApp) RegisterUpgradeHandlers() { + app.UpgradeKeeper.SetUpgradeHandler( + UpgradeName, + func(ctx context.Context, _ upgradetypes.Plan, fromVM module.VersionMap) (module.VersionMap, error) { + return app.ModuleManager.RunMigrations(ctx, app.Configurator(), fromVM) + }, + ) + + upgradeInfo, err := app.UpgradeKeeper.ReadUpgradeInfoFromDisk() + if err != nil { + panic(err) + } + + if upgradeInfo.Name == UpgradeName && !app.UpgradeKeeper.IsSkipHeight(upgradeInfo.Height) { + storeUpgrades := storetypes.StoreUpgrades{ + Added: []string{}, + Deleted: []string{"protocolpool"}, + } + app.SetStoreLoader(upgradetypes.UpgradeStoreLoader(upgradeInfo.Height, &storeUpgrades)) + } +} +``` + +Add `"params"` to `Deleted` as well if your app still had the `x/params` store mounted. + +## Upgrading from v0.53.x + +Skipping v0.54 and upgrading directly from `v0.53.x` to `v0.55.x` is supported as a single coordinated upgrade: one binary swap, one upgrade handler, one halt height. Work through the [v0.53.x → v0.54.x upgrade reference](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/UPGRADING.md) first — all of its required changes still apply — then apply this guide on top. The v0.54 hop's highlights, so you know what you're signing up for: + +* CometBFT `v0.38.x` → `v0.39.x` (LibP2P, `AdaptiveSync`); from v0.53 you jump straight to the `v0.40.0` release v0.55 pins. +* Consolidation of `cosmossdk.io/x/*` vanity modules into `github.com/cosmos/cosmos-sdk/x/*`, plus the Log v2 and Store v2 moves. +* `x/gov` keeper-initialization and `GovHooks` interface changes, `x/epochs` and `x/bank` wiring updates, and the `x/circuit` / `x/nft` / `x/crisis` deprecations. +* IBC v11 (if your chain uses IBC). + +Where the two hops interact, land directly on the v0.55 state instead of transiting through v0.54's: + +* **Skip transient wiring.** Don't adopt v0.54 reference-app wiring that v0.55 removes in the same hop: the SIGN_MODE_TEXTUAL tx-config setup, `x/protocolpool` (if your v0.53 app didn't already wire it), and `distrkeeper.WithExternalCommunityPool`. Go straight to the v0.55 forms shown in this guide. +* **Module constructors.** v0.54's constructor signatures still carried the legacy `exported.Subspace` arguments; use the v0.55 signatures from [Removed: x/params](#removed-xparams) directly. +* **Custom mempools.** Implement the v0.55 `Mempool` interface ([Mempool Interface Changes](#mempool-interface-changes)) directly; don't bother with the v0.54 shape. +* **Module migrations are cumulative.** `RunMigrations` walks each module from its v0.53 consensus version to the v0.55 target in one pass (`x/auth` 5 → 6 → 7, `x/staking` 5 → 6). No manual intervention is needed beyond the standard upgrade handler. +* **Store upgrades.** The v0.53 → v0.54 hop required no store additions or deletions, so the combined store upgrade is exactly the snippet in [Upgrade Handler and Store Migrations](#upgrade-handler-and-store-migrations): delete `protocolpool` only if your v0.53 app had wired it, and `params` if its store was still mounted (more likely on a v0.53-era app). Use a single upgrade name, e.g. `v053-to-v055`. + +Test the full jump on a mainnet-state export before scheduling it: the two-version migration path gets far less ecosystem mileage than the single-version one. + +## New Features and Non-Breaking Changes + +These changes are optional to adopt during the upgrade; they are not required for a successful migration. The exception is key rotation, which is active on every v0.55 chain once the required wiring above is in place. + +### Validator Consensus Key Rotation + +v0.55 adds consensus key rotation to `x/staking` ([#26440](https://github.com/cosmos/cosmos-sdk/pull/26440)): a validator operator can submit `MsgRotateConsPubKey` (wired into the CLI, [#26461](https://github.com/cosmos/cosmos-sdk/pull/26461)) to replace their consensus key without unbonding. Key properties: + +* **Fee.** Each rotation charges the `key_rotation_fee` staking param (default `1000000` of the bond denom) from the operator account; the fee is burned via the `key_rotation_fee_pool` module account. +* **Rate limit.** One rotation per validator per unbonding period. +* **Applied in the end blocker.** The rotation is scheduled by the msg server and applied at the end of the block; CometBFT is informed through a validator-set update. +* **Evidence and slashing.** Equivocation evidence against a rotated-away (historical) consensus address remains attributable to the validator until the evidence is no longer admissible — i.e. until both `evidence.max_age_num_blocks` and `evidence.max_age_duration` have elapsed since the rotation, which can be later than the unbonding time ([#26481](https://github.com/cosmos/cosmos-sdk/pull/26481), [#26616](https://github.com/cosmos/cosmos-sdk/pull/26616)). Slashing signing info is migrated to the active consensus key. Governance changes that extend the evidence-age params after a rotation's expiry has been computed are not retroactively applied; chains should account for this when tuning evidence params. +* **Genesis.** Rotation history and pending-rotation state are included in staking genesis import/export ([#26471](https://github.com/cosmos/cosmos-sdk/pull/26471)); genesis export tooling that parses staking genesis JSON should expect the new fields. +* **Events.** `rotate_cons_pubkey` is emitted when a rotation is scheduled (including apply height, maturity time, evidence-expiry time/height, and the burned fee) and `apply_cons_pubkey_rotation` when it is applied (validator, old and new consensus addresses) ([#26619](https://github.com/cosmos/cosmos-sdk/pull/26619)). + +Indexers, exchanges, and monitoring that key validators by consensus address must handle the mapping changing over a validator's lifetime. On-chain, `keeper.ValidatorByHistoricalConsAddr` resolves a validator from a rotated-away consensus address. + +Chains built on the enterprise `x/poa` module have their own `MsgRotateConsPubKey` with different semantics — no fee, no rate limit, an admin override, and a same-block swap with no rotation history. Because the old consensus address is gone immediately, modules that attribute `LastCommit` signatures or vote extensions by consensus address need extra care across the swap; see the PoA guide below for the caveats and the operator runbook. + +For an overview of key rotation and the operator procedures, see [Key rotation](https://docs.cosmos.network/sdk/latest/keys/key-rotation), [Rotate a consensus key, Staking](https://docs.cosmos.network/sdk/latest/keys/rotate-validator-key), and [Rotate a consensus key, PoA](https://docs.cosmos.network/sdk/latest/keys/rotate-validator-key-poa). + +### ML-DSA-65 Validator Consensus Keys + +Cosmos SDK v0.55 registers the NIST ML-DSA-65 (FIPS 204) post-quantum signature scheme as a supported validator consensus key type ([#26436](https://github.com/cosmos/cosmos-sdk/pull/26436)). The new `cosmos.crypto.mldsa65.PubKey` / `PrivKey` proto messages, Amino routes (`cometbft/PubKeyMlDsa65`, `cometbft/PrivKeyMlDsa65`), interface-registry registration, multisig amino route, and `hd.MlDsa65Type` constant are all enabled by default. + +**Action required:** none. Existing chains continue to accept only the consensus key types listed in `genesis.consensus_params.validator.pub_key_types` (still `["ed25519"]` by default). No state-machine-relevant behavior changes for chains that do not opt in. + +**To opt in (new chains):** set `genesis.consensus_params.validator.pub_key_types` to `["ml_dsa_65"]` (or a list including it). Validators must then submit `MsgCreateValidator` with a `mldsa65.PubKey`. The `init` and `testnet` commands accept `--consensus-key-algo ml_dsa_65` to generate matching validator files ([#26604](https://github.com/cosmos/cosmos-sdk/pull/26604)). Test harnesses can use the new `testutil/network.Config.ValidatorConsensusKeyType` field together with `genutil.InitializeNodeValidatorFilesFromMnemonicWithKeyType` to spin up an in-process testnet pinned to ML-DSA-65. + +**Operational considerations:** ML-DSA-65 keys and signatures are substantially larger than ed25519 (pubkey 1952 bytes vs 32, signature 3309 bytes vs 64). Chains enabling this key type should review `consensus_params.block.max_bytes` and gossip framing limits accordingly. The cometbft commit lift in this release expanded `MaxSignatureSize` and the per-validator `MaxCommitSigBytes` to accommodate the larger signatures; downstream applications relying on the previous fixed values may need to be re-examined. + +**Warning — IBC counterparties must upgrade first.** IBC light clients on counterparty chains verify your validator set's commit signatures using the counterparty's own compiled-in crypto. A counterparty running a stack that predates ML-DSA-65 support cannot verify signatures from the new key type: once validators holding sufficient voting power sign with it, your headers fail verification there, IBC packet flow with that chain stops, and the client eventually expires. Before enabling a new consensus key type on a chain with live IBC connections, coordinate so every counterparty chain is running a CometBFT/SDK stack that can verify it — the counterparty only needs the verification code on its nodes, not the key type in its own `pub_key_types`. + +Existing chains can combine this with [key rotation](#validator-consensus-key-rotation) to move validators to post-quantum keys: add `ml_dsa_65` to `pub_key_types` via a consensus-params update, then have validators rotate. + +For the concepts and operator guides, see [Post-quantum keys](https://docs.cosmos.network/sdk/latest/keys/post-quantum-keys), [Enable ML-DSA keys](https://docs.cosmos.network/sdk/latest/keys/enable-ml-dsa-keys), and [Migrate a validator to ML-DSA](https://docs.cosmos.network/sdk/latest/keys/migrate-validator-ml-dsa). + +### ML-DSA-65 Account Keys + +[#26472](https://github.com/cosmos/cosmos-sdk/pull/26472) extends ML-DSA-65 support to user account keys: keyring creation and mnemonic recovery (`--algo ml_dsa_65`), transaction signing and verification, and a new ante-handler gas cost param `SigVerifyCostMlDsa65` (added to `x/auth` params by the automatic 6 → 7 migration). No action is required; accounts using existing key types are unaffected. + +See [Create an ML-DSA account](https://docs.cosmos.network/sdk/latest/keys/create-ml-dsa-account) and [Post-quantum keys](https://docs.cosmos.network/sdk/latest/keys/post-quantum-keys). + +### secp256k1eth Validator Consensus Keys + +[#26615](https://github.com/cosmos/cosmos-sdk/pull/26615) adds `crypto/keys/secp256k1eth`, wrapping CometBFT's Ethereum-style secp256k1 consensus key implementation with SDK codec registration. Intended for EVM-compatible chains that want validator consensus addresses derived the Ethereum way; opt in via `genesis.consensus_params.validator.pub_key_types`. + +The IBC counterparty warning from the [ML-DSA-65 section](#ml-dsa-65-validator-consensus-keys) applies here too: counterparty chains must run a stack that can verify secp256k1eth signatures before your validators adopt the key type, or IBC connections with them will break. + +See [Post-quantum keys](https://docs.cosmos.network/sdk/latest/keys/post-quantum-keys) for how the consensus key types compare. + +### Block-STM Configuration + +Block-STM parallel execution itself is not new — the engine (`baseapp/txnrunner`) and the `SetBlockSTMTxRunner` hook shipped in v0.54.x, wired programmatically per chain. v0.55 adds standard operator-facing configuration ([#26208](https://github.com/cosmos/cosmos-sdk/pull/26208)): `block-executor` (`"sequential"`, the default, or `"block-stm"`), `block-stm-workers`, and `block-stm-pre-estimate` in `app.toml`, plus a `baseapp/blockexec` helper that resolves them and installs the runner. Chains that already call `SetBlockSTMTxRunner` directly can keep that wiring or switch to the helper. + +To adopt the config-driven wiring, call `blockexec.Apply` after creating your store keys (see `simapp/app.go`): + +```go +stores := make([]storetypes.StoreKey, 0, len(keys)) +for _, k := range keys { + stores = append(stores, k) +} +blockexec.Apply(bApp, appOpts, stores, txConfig.TxDecoder(), + func(storetypes.MultiStore) string { return sdk.DefaultBondDenom }, +) +``` + +`Apply` resolves the executor from `app.toml`/flags and installs the corresponding `TxRunner`; with the default `sequential` executor it preserves today's behavior, so the wiring is safe to add unconditionally. Block-STM is incompatible with the block gas meter (disabled by default since v0.54): `Apply` disables the meter automatically when `block-stm` is selected, but chains wiring `SetBlockSTMTxRunner` directly must call `SetDisableBlockGasMeter(true)` first or the runner installation panics. + +Switching a running chain's executor is a per-node setting with identical state-transition results, but treat the first enablement as an operational rollout: test with your workload before flipping validators. + +## Behavior Changes Affecting Dapps and Indexers + +Observable changes between v0.54.x and v0.55.x that don't require code changes but may affect downstream consumers: + +* **Block selection uses ante-reported gas.** Proposals are packed using the gas wanted returned by the ante handler at `CheckTx` time rather than the tx-declared gas limit ([#25338](https://github.com/cosmos/cosmos-sdk/pull/25338)). Block composition can differ for txs whose ante-reported gas diverges from their declared limit. +* **Staking emits key-rotation events** (`rotate_cons_pubkey`, `apply_cons_pubkey_rotation`), and validator consensus addresses can change over time ([#26619](https://github.com/cosmos/cosmos-sdk/pull/26619)). +* **`x/gov` `proposal_messages` event attribute** no longer has a leading comma ([#26353](https://github.com/cosmos/cosmos-sdk/pull/26353)). +* **`x/authz` prunes at most 200 expired grants per begin block** ([#26588](https://github.com/cosmos/cosmos-sdk/pull/26588)); mass-expiry cleanup now spreads across blocks. +* **`x/distribution` reward withdrawals to blocked addresses** during begin/end block fall back to the delegator/validator owner and then the community pool instead of failing ([#26406](https://github.com/cosmos/cosmos-sdk/pull/26406)). User-initiated withdrawals to blocked addresses still return `ErrUnauthorized`. +* **`x/feegrant` `Allowances` and `AllowancesByGranter` queries** now honor `PageRequest.offset` and `count_total` correctly ([#26596](https://github.com/cosmos/cosmos-sdk/pull/26596)); clients that compensated for the old off-by-page results should re-check. \ No newline at end of file diff --git a/sdk/next/changelog/release-notes.mdx b/sdk/next/changelog/release-notes.mdx index f93e5c0ee..2049e3ede 100644 --- a/sdk/next/changelog/release-notes.mdx +++ b/sdk/next/changelog/release-notes.mdx @@ -1,115 +1,81 @@ --- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/changelog/release-notes' title: "Changelog" description: "Release history and changelog for Cosmos SDK" mode: "wide" --- - This page tracks releases and changes for v0.54.1. For the full release history, see the [CHANGELOG](https://github.com/cosmos/cosmos-sdk/blob/main/CHANGELOG.md) on GitHub. + This page tracks releases and changes for v0.55.0. For the full release history, see the [CHANGELOG](https://github.com/cosmos/cosmos-sdk/blob/main/CHANGELOG.md) on GitHub. - -## Improvements - -- (x/auth) [#26297](https://github.com/cosmos/cosmos-sdk/pull/26297) Cap pagination limit at number of txs within block during `GetBlockWithTxs` instead of 100. - - - + ## Breaking Changes -- (x/consensus) [#25607](https://github.com/cosmos/cosmos-sdk/pull/25607) Add `AuthorityParams` to consensus params. When set, the consensus params authority takes precedence over per-keeper authority for all module parameter updates. Keeper constructor signatures are unchanged. -- (x/staking) [#25724](https://github.com/cosmos/cosmos-sdk/issues/25724) Validate `BondDenom` in `MsgUpdateParams` to prevent setting non-existent or zero-supply denoms. -- [#25778](https://github.com/cosmos/cosmos-sdk/pull/25778) Update `log` to log v2. -- [#25090](https://github.com/cosmos/cosmos-sdk/pull/25090) Moved deprecated modules to `./contrib`. These modules are still available but will no longer be actively maintained or supported in the Cosmos SDK Bug Bounty program. -- `x/group` -- `x/nft` -- `x/circuit` -- `x/crisis` -- (crypto) [#24414](https://github.com/cosmos/cosmos-sdk/pull/24414) Remove sr25519 support, since it was removed in CometBFT v1.x (see: CometBFT [#3646](https://github.com/cometbft/cometbft/pull/3646)). -- (x/mint) [#25599](https://github.com/cosmos/cosmos-sdk/pull/25599) Add max supply param. -- (x/gov) [#25615](https://github.com/cosmos/cosmos-sdk/pull/25615) Decouple `x/gov` from `x/staking` by making `CalculateVoteResultsAndVotingPowerFn` a required parameter to `keeper.NewKeeper` instead of `StakingKeeper`. -- (x/gov) [#25617](https://github.com/cosmos/cosmos-sdk/pull/25617) `AfterProposalSubmission` hook now includes proposer address as a parameter. -- (x/gov) [#25616](https://github.com/cosmos/cosmos-sdk/pull/25616) `DistrKeeper` `x/distribution` is now optional. Genesis validation ensures `distrKeeper` is set if distribution module is used as proposal cancel destination. -- (systemtests) [#25930]https://github.com/cosmos/cosmos-sdk/pull/25930) Move `systemtests` into `testutil` and no longer under its own `go.mod`. -- (baseapp) [#26060](https://github.com/cosmos/cosmos-sdk/pull/26060) Remove `BaseApp.SetStoreMetrics`. The `StoreMetrics` interface never worked, so removing dead code. -- (store) [#26061](https://github.com/cosmos/cosmos-sdk/pull/26061) Remove store tracing API and all related plumbing: -- Remove `SetTracer`, `SetTracingContext`, and `TracingEnabled` from `MultiStore` interface. -- Remove `CacheWrapWithTrace` from `CacheWrapper` interface. -- Remove `BaseApp.SetCommitMultiStoreTracer` and tracing context logic from `BaseApp.cacheTxContext` and `FinalizeBlock`. -- Remove `io.Writer` parameter from `servertypes.AppCreator` and `traceWriter io.Writer` from `servertypes.AppExporter`. -- Remove `traceStore io.Writer` parameter from `simapp.NewSimApp` and all enterprise simapp constructors. -- Remove `traceStore io.Writer` from all `testutil/simsx` app factory signatures. -- (store) [#26042](https://github.com/cosmos/cosmos-sdk/pull/26042) We are now importing `github.com/cosmos/cosmos-sdk/store/v2` as the store package instead of `cosmossdk.io/store` and all import paths have changed. -- (baseapp) [#26138](https://github.com/cosmos/cosmos-sdk/pull/26138) Default block gas meter to disabled. Adds checking to ensure block gas meter is not enabled while bstm parallel execution is configured and panics in these scenarios during parameter assignment. +- (mempool) [#25338](https://github.com/cosmos/cosmos-sdk/pull/25338) Respect gas wanted returned by the ante handler for block selection. Adds `InsertWithOption` to the `Mempool` interface (carries the ante-reported `GasWanted`) and changes the `SelectBy` callback to receive a `mempool.Tx` wrapper that exposes the stored value. +- (tx) [#26456](https://github.com/cosmos/cosmos-sdk/pull/26456) Remove `SIGN_MODE_TEXTUAL` and all associated implementation (`x/tx/signing/textual`, `x/auth/tx/textual.go`, `TextualCoinMetadataQueryFn`). The proto enum value is reserved to prevent future reuse. ADR-050 is marked archived. +- (modules) [#26421](https://github.com/cosmos/cosmos-sdk/pull/26421) Remove the `x/protocolpool` module and its API/proto surface from the SDK. Applications upgrading from v0.54 should include `protocolpool` in deleted store upgrades. +- (genutils) [#26468](https://github.com/cosmos/cosmos-sdk/pull/26468) Consolidate ExportGenesisFileWithTime arguments to preserve consensus params. ## Features -- [#25471](https://github.com/cosmos/cosmos-sdk/pull/25471) Full BLS 12-381 support enabled. -- [#24872](https://github.com/cosmos/cosmos-sdk/pull/24872) Support BLS 12-381 for cli `init`, `gentx`, `collect-gentx` -- (crypto) [#24919](https://github.com/cosmos/cosmos-sdk/pull/24919) add `NewPubKeyFromBytes` function to the `secp256r1` package to create `PubKey` from bytes -- (server) [#24720](https://github.com/cosmos/cosmos-sdk/pull/24720) add `verbose_log_level` flag for configuring the log level when switching to verbose logging mode during sensitive operations (such as chain upgrades). -- (crypto) [#24861](https://github.com/cosmos/cosmos-sdk/pull/24861) add `PubKeyFromCometTypeAndBytes` helper function to convert from `comet/v2` PubKeys to the `cryptotypes.Pubkey` interface. -- (abci_utils) [#25008](https://github.com/cosmos/cosmos-sdk/pull/25008) add the ability to assign a custom signer extraction adapter in `DefaultProposalHandler`. -- (x/distribution) [#25650](https://github.com/cosmos/cosmos-sdk/pull/25650) Add new gRPC query endpoints and CLI commands for `DelegatorStartingInfo`, `ValidatorHistoricalRewards`, and `ValidatorCurrentRewards`. -- [#25745](https://github.com/cosmos/cosmos-sdk/pull/25745) Add DiskIO telemetry via gopsutil. -- (grpc) [#25648](https://github.com/cosmos/cosmos-sdk/pull/25648) Add `earliest_block_height` and `latest_block_height` fields to `GetSyncingResponse`. -- (collections/codec) [#25614] (https://github.com/cosmos/cosmos-sdk/pull/25827) Add `TimeValue` (`ValueCodec[time.Time]`) to collections/codec. -- (enterprise/poa) [#25838](https://github.com/cosmos/cosmos-sdk/pull/25838) Add the `poa` module under the `enterprise` directory. -- (grpc) [#25850](https://github.com/cosmos/cosmos-sdk/pull/25850) Add `GetBlockResults` and `GetLatestBlockResults` gRPC endpoints to expose CometBFT block results including `finalize_block_events`. +- (abci) [#25620](https://github.com/cosmos/cosmos-sdk/pull/25620) Add support for new application side mempool ABCI methods. +- (abci) [#25969](https://github.com/cosmos/cosmos-sdk/pull/25969) Add support for new ABCI methods, `InsertTx` and `ReapTxs`. +- (blockstm) [#26208](https://github.com/cosmos/cosmos-sdk/pull/26208) Add Block-STM configuration support: `block-executor`, `block-stm-workers` and `block-stm-pre-estimate`. +- (blockstm) [#25909](https://github.com/cosmos/cosmos-sdk/pull/25909) Cache pre-state to optimize value-based validation. +- (deps) [#26388](https://github.com/cosmos/cosmos-sdk/pull/26388) Bump CometBFT version to v0.39.3. +- (staking) [#26440](https://github.com/cosmos/cosmos-sdk/pull/26440) Add basic key rotation for validator consensus keys. +- (crypto) [#26436](https://github.com/cosmos/cosmos-sdk/pull/26436) Add ML-DSA-65 (FIPS 204) post-quantum validator consensus key type, with SDK key wrappers, Amino + interface-registry registration, multisig support, and a `hd.MlDsa65Type` constant. +- (blockstm) [#26467](https://github.com/cosmos/cosmos-sdk/pull/26467) Track existence for `Has()` reads to reduce false conflicts. +- (staking) [#26485](https://github.com/cosmos/cosmos-sdk/pull/26485) Add `key_rotation_fee` to `x/staking` params and register associated 5->6 migration. +- (staking) [#26461](https://github.com/cosmos/cosmos-sdk/pull/26461) Wire `MsgRotateConsPubKey` into cli and add a happy path system test. +- (staking) [#26471](https://github.com/cosmos/cosmos-sdk/pull/26471) Add genesis import/export support for validator consensus key rotation. +- (crypto) [#26472](https://github.com/cosmos/cosmos-sdk/pull/26472) Add ML-DSA-65 (FIPS 204) support for user account keys: mnemonic-based keyring creation/recovery (`--algo ml_dsa_65`), transaction signing/verification, and an ante-handler signature-verification gas cost (`Params.SigVerifyCostMlDsa65`). +- (enterprise/poa) [#26590](https://github.com/cosmos/cosmos-sdk/pull/26590) Add `MsgRotateConsPubKey` for POA validator consensus key rotation (operator self-service plus admin override). +- (enterprise/poa) [#26614](https://github.com/cosmos/cosmos-sdk/pull/26614) Add ML-DSA-65 (mldsa65) validator key support to the PoA module via a `WithMlDsa65Support()` module option, raising `MaxPubKeyLength` to accommodate the larger keys. +- (crypto) [#26615](https://github.com/cosmos/cosmos-sdk/pull/26615) Add `secp256k1eth` validator consensus key type. ## Improvements -- (ci) Use softprops/action-gh-release for main-nightly instead of custom gh/git to avoid repository ruleset conflicts. -- (telemetry) [#26006](https://github.com/cosmos/cosmos-sdk/pull/26006) Export `ExtensionOptions` type for programmatic otel.yaml generation. -- [#25955](https://github.com/cosmos/cosmos-sdk/pull/25955) Use cosmos/btree directly instead of replacing it in go.mods -- (types) [#25342](https://github.com/cosmos/cosmos-sdk/pull/25342) Undeprecated `EmitEvent` and `EmitEvents` on the `EventManager`. These functions will continue to be maintained. -- (types) [#24668](https://github.com/cosmos/cosmos-sdk/pull/24668) Scope the global config to a particular binary so that multiple SDK binaries can be properly run on the same machine. -- (baseapp) [#24655](https://github.com/cosmos/cosmos-sdk/pull/24655) Add mutex locks for `state` and make `lastCommitInfo` atomic to prevent race conditions between `Commit` and `CreateQueryContext`. -- (proto) [#24161](https://github.com/cosmos/cosmos-sdk/pull/24161) Remove unnecessary annotations from `x/staking` authz proto. -- (x/bank) [#24660](https://github.com/cosmos/cosmos-sdk/pull/24660) Improve performance of the `GetAllBalances` and `GetAccountsBalances` keeper methods. -- (collections) [#25464](https://github.com/cosmos/cosmos-sdk/pull/25464) Add `IterateRaw` method to `Multi` index type to satisfty query `Collection` interface. -- (api) [#25613](https://github.com/cosmos/cosmos-sdk/pull/25613) Separated deprecated modules into the contrib directory, distinct from api, to enable and unblock new proto changes without affecting legacy code. -- (server) [#25740](https://github.com/cosmos/cosmos-sdk/pull/25740) Add variadic `grpc.DialOption` parameter to `StartGrpcServer` for custom gRPC client connection options. -- (blockstm) [#25765](https://github.com/cosmos/cosmos-sdk/pull/25765) Minor code readability improvement in block-stm. -- (blockstm) [#25786](https://github.com/cosmos/cosmos-sdk/pull/25786) Add pre-state checking in transaction state transition. -- (server/config) [#25807](https://github.com/cosmos/cosmos-sdk/pull/25807) fix(server): reject overlapping historical gRPC block ranges. -- [#25857](https://github.com/cosmos/cosmos-sdk/pull/25857) Reduce scope of mutex in `PriorityNonceMempool.Remove`. -- (baseapp) [#25862](https://github.com/cosmos/cosmos-sdk/pull/25862) Skip running validateBasic for rechecking txs. (Backport of https://github.com/cosmos/cosmos-sdk/pull/20208). -- (blockstm) [25883](https://github.com/cosmos/cosmos-sdk/pull/25883) Re-use decoded tx object in pre-estimates. -- (blockstm) [#25788](https://github.com/cosmos/cosmos-sdk/pull/25788) Only validate transactions that's executed at lease once. -- (blockstm) [#25767](https://github.com/cosmos/cosmos-sdk/pull/25767) Optimize block-stm MVMemory with bitmap index. +- (server/config) [#26572](https://github.com/cosmos/cosmos-sdk/pull/26572) Warn that `query-gas-limit = 0` (the default) is unbounded and exposes public RPC nodes to DoS via expensive queries. +- (docs) [#25918](https://github.com/cosmos/cosmos-sdk/issues/25918) Regenerate Swagger API spec to reflect current proto state, including `authority` field on consensus params and removal of stale module-config definitions. +- (baseapp) [#22368](https://github.com/cosmos/cosmos-sdk/issues/22368) Add `-race`-mode regression test (`TestABCI_Race_GRPC_Query_During_Commit`) covering concurrent `BaseApp.Query` and `FinalizeBlock`/`Commit`. Pins down the state-management mutex work added in #24655 and follow-ups so the data race reported against v0.50.x cannot regress silently. +- (x/staking, x/slashing) [#26481](https://github.com/cosmos/cosmos-sdk/pull/26481) Resolve evidence against recently rotated consensus keys and migrate slashing signing state to the active consensus key. +- (x/auth/tx) [#25221](https://github.com/cosmos/cosmos-sdk/issues/25221) Add `ConfigOptions.AminoJSONEncoder` so applications can configure a custom `aminojson.Encoder` (e.g. custom field encodings) for the `SIGN_MODE_LEGACY_AMINO_JSON` handler without replicating the SDK's `HandlerMap` construction. +- chore(x/auth) [#26567](https://github.com/cosmos/cosmos-sdk/pull/26567): add a human-readable error +- (blockstm) [#26592](https://github.com/cosmos/cosmos-sdk/pull/26592) Validate `ExecuteBlock` inputs (block size, store index mapping, and estimates) at the exported entry point so invalid input returns a descriptive error instead of an opaque "index out of range" panic. +- (cli) [#26604](https://github.com/cosmos/cosmos-sdk/pull/26604) Add consensus key algo to init and testnet CLIs. +- (crypto) [#26626](https://github.com/cosmos/cosmos-sdk/pull/26626) Update mldsa65 PubKey Address logic to validate length, remove unpacking. +- (staking) [#26619](https://github.com/cosmos/cosmos-sdk/pull/26619) Emit `rotate_cons_pubkey` and `apply_cons_pubkey_rotation` events during key rotation. ## Bug Fixes -- (baseapp) [#25331](https://github.com/cosmos/cosmos-sdk/issues/25331) Avoid noisy errors when gRPC response headers are already sent, set block height as a header when possible and fall back to a trailer. -- (blockstm) [#25789](https://github.com/cosmos/cosmos-sdk/issues/25789) Wake up suspended executors when scheduler doesn't complete to prevent goroutine leaks. -- (grpc) [#25647](https://github.com/cosmos/cosmos-sdk/pull/25647) Return actual `earliest_store_height` in `node.Status` gRPC endpoint instead of hardcoded `0`. -- (types/query) [#25665](https://github.com/cosmos/cosmos-sdk/issues/25665) Fix pagination offset when querying a collection with predicate function. -- (x/staking) [#25649](https://github.com/cosmos/cosmos-sdk/pull/25649) Add missing `defer iterator.Close()` calls in `IterateDelegatorRedelegations` and `GetRedelegations` to prevent resource leaks. -- (mempool) [#25563](https://github.com/cosmos/cosmos-sdk/pull/25563) Cleanup sender indices in case of tx replacement. -- (x/epochs) [#25425](https://github.com/cosmos/cosmos-sdk/pull/25425) Fix `InvokeSetHooks` being called with a nil keeper and `AppModule` containing a copy instead of a pointer (hooks set post creating the `AppModule` like with depinject didn't apply because it's a different instance). -- (client, client/rpc, x/auth/tx) [#24551](https://github.com/cosmos/cosmos-sdk/pull/24551) Handle cancellation properly when supplying context to client methods. -- (x/authz) [#24638](https://github.com/cosmos/cosmos-sdk/pull/24638) Fixed a minor bug where the grant key was cast as a string and dumped directly into the error message leading to an error string possibly containing invalid UTF-8. -- (client, client/rpc, x/auth/tx) [#24551](https://github.com/cosmos/cosmos-sdk/pull/24551) Handle cancellation properly when supplying context to client methods. -- (x/epochs) [#24770](https://github.com/cosmos/cosmos-sdk/pull/24770) Fix register of epoch hooks in `InvokeSetHooks`. -- (x/epochs) [#25087](https://github.com/cosmos/cosmos-sdk/pull/25087) Remove redundant error check in BeginBlocker. -- [GHSA-p22h-3m2v-cmgh](https://github.com/cosmos/cosmos-sdk/security/advisories/GHSA-p22h-3m2v-cmgh) Fix x/distribution can halt when historical rewards overflow. -- (x/staking) [#25258](https://github.com/cosmos/cosmos-sdk/pull/25258) Add delegator address to redelegate event. -- (x/bank) [#25751](https://github.com/cosmos/cosmos-sdk/pull/25751) Fix recipient address in events. -- (client) [#25811] (https://github.com/cosmos/cosmos-sdk/pull/25811) fix(client): fix file handle leaks in snapshot commands. -- (server/config) [#25806](https://github.com/cosmos/cosmos-sdk/pull/25806) fix: add missing commas in historical gRPC config template. -- (client) [#25804](https://github.com/cosmos/cosmos-sdk/pull/25804) Add `GetHeightFromMetadataStrict` API to `grpc` client for better error handling. -- (x/staking) [#25829](https://github.com/cosmos/cosmos-sdk/pull/25829) Validates case-sensitivity on authz grands in x/staking. -- (mempool) [#25869](https://github.com/cosmos/cosmos-sdk/pull/25869) fix(mempool): add thread safety to NextSenderTx. -- (blockstm) [#25912](https://github.com/cosmos/cosmos-sdk/pull/25912) Remove `SigVerificationDecorator` signature incarnation cache causing state divergence under blockstm. -- (x/group) [#25922](https://github.com/cosmos/cosmos-sdk/pull/25922) Add zero-total-weight check for ThresholdDecisionPolicy -- (x/group) [#25917](https://github.com/cosmos/cosmos-sdk/pull/25917) Prevent creation of zero-weight groups. -- (x/group) [#25919](https://github.com/cosmos/cosmos-sdk/pull/25919) add safer type assertions to group `DecisionPolicy` getter calls. -- (x/group) [#25920](https://github.com/cosmos/cosmos-sdk/pull/25920) Expand voting period check to verify period is positive instead of nonzero. -- (baseapp) [#26063](https://github.com/cosmos/cosmos-sdk/pull/26063) Fixes an issue where values embedded in context during ante handling were wiped after the handlers returned. - -## Deprecated - -- [#25948](https://github.com/cosmos/cosmos-sdk/pull/25948) Change default `app.go` code to not use `depinject` as we are phasing it out. -- (baseapp) [#26107](https://github.com/cosmos/cosmos-sdk/pull/26170) Deprecate baseapp test helper `app.NewUncachedContext`, consider using `app.NewNextBlockContext` or `app.NewContext` instead, see `UPGRADING.md` for more details. +- (codec) [#26587](https://github.com/cosmos/cosmos-sdk/pull/26587) Lower the nested `google.protobuf.Any` recursion depth cap in unknown-field validation from 10,000 to 64, reducing CPU-amplification DoS risk from deeply nested `Any` wrappers. No legitimate message nests `Any` anywhere near that deep. +- (x/feegrant) [#26596](https://github.com/cosmos/cosmos-sdk/pull/26596) Honor the `PageRequest` offset and `count_total` in the `Allowances` and `AllowancesByGranter` gRPC queries, which previously collected grants inside the pagination predicate and so returned offset-skipped and beyond-limit results. +- (x/authz) [#26588](https://github.com/cosmos/cosmos-sdk/pull/26588) Cap the number of expired grants pruned per `BeginBlocker` call to 200, matching `x/feegrant`'s existing pattern, so a block where many grants expire at once can't cause unbounded work. +- (client) [#26524](https://github.com/cosmos/cosmos-sdk/pull/26524) Fix file handle leak in the `snapshot dump` command where chunk files were deferred-closed inside the loop, keeping every chunk's handle open until the command returned (follow-up to #25811). +- (x/distribution) [#26518](https://github.com/cosmos/cosmos-sdk/pull/26518) Return an error from internal historical rewards reads when the record is absent, preventing recovered reference-count panics during BlockSTM speculative execution. +- (x/auth) [#26515](https://github.com/cosmos/cosmos-sdk/pull/26515) Bound the pubkey and signature indices in `ConsumeMultisignatureVerificationGas` and `VerifyMultisignature` so a multisig signature with a bit array larger than the key set, or with more set bits than supplied signatures, returns an error instead of panicking with index out of range. +- (x/distribution) [#26406](https://github.com/cosmos/cosmos-sdk/pull/26406) Add fallback paths (delegator/validator owner, then community pool) when withdrawing delegator rewards or validator commission to a blocked address during `Begin/EndBlockers`. user msg initiated paths still return `ErrUnauthorized` when withdrawing to blocked addresses. +- (x/gov) [#26353](https://github.com/cosmos/cosmos-sdk/pull/26353) Fix leading comma in `proposal_messages` event attribute emitted by `SubmitProposal`. +- (telemetry) [#26390](https://github.com/cosmos/cosmos-sdk/pull/26390) Fix env var for otel telemetry initialization. +- (x/staking) [#26408](https://github.com/cosmos/cosmos-sdk/pull/26408) Fix `MsgBeginRedelegate` failure when redelegating all shares from an unbonded source validator that is removed after unbonding. +- (x/auth/tx) [#26422](https://github.com/cosmos/cosmos-sdk/pull/26422) Reuse the signing context from the codec's `InterfaceRegistry` when `ConfigOptions.SigningOptions` is unset so that `CustomGetSigners` registered via `NewInterfaceRegistryWithOptions` are honored by `NewTxConfig` / `NewTxConfigWithOptions`. +- (x/staking) [#26460](https://github.com/cosmos/cosmos-sdk/pull/26460) Coalesce key rotation power updates to not emit duplicates. +- (x/staking) [#26483](https://github.com/cosmos/cosmos-sdk/pull/26483) Block `MsgCreateValidator` from creating validators with cons addrs locked by key rotations. +- (blockstm) [#25893](https://github.com/cosmos/cosmos-sdk/pull/25893) Fix CancelAll cancellation by clearing blocker ESTIMATE marks before waking suspended executors. +- (crypto) [#26529](https://github.com/cosmos/cosmos-sdk/pull/26529) Validate the SEC1 tag byte (`0x02`/`0x03`) when unmarshaling a `secp256k1.PubKey`, rejecting malformed compressed keys that previously passed the length-only check. +- (x/auth/tx) [#26571](https://github.com/cosmos/cosmos-sdk/pull/26571) Avoid nil pointer panic in `GetSigningTxData` for multisig `ModeInfo` with a nil `Multi` or nil `Bitarray`. +- (x/auth/tx) [#26527](https://github.com/cosmos/cosmos-sdk/pull/26527) Fix nil pointer panic in `GetSigningTxData` when a `SignerInfo` has a nil `PublicKey`. +- (x/auth/tx) [#26517](https://github.com/cosmos/cosmos-sdk/pull/26517) Return a decode error instead of panicking when a transaction's `SignerInfos` and `Signatures` counts disagree in `GetSignaturesV2`, or a multisig's `ModeInfos` and sub-signature counts disagree in `ModeInfoAndSigToSignatureData`. +- (x/auth/ante) [#26573](https://github.com/cosmos/cosmos-sdk/pull/26573) Reject tx with extra SignerInfos in SetPubKeyDecorator. +- (block-stm) [#26583](https://github.com/cosmos/cosmos-sdk/pull/26583) Fix count validation tasks before advancing validationIdx to prevent lost updates. +- (blockstm) [#26591](https://github.com/cosmos/cosmos-sdk/pull/26591) normalize non-positive worker count in `STMRunner.Run`. +- (x/staking) [#26613](https://github.com/cosmos/cosmos-sdk/pull/26613) Require `key_rotation_fee` denom to equal `bond_denom` in `Params.Validate` and derive the default fee denom from the configured bond denom. +- (x/staking) [#26611](https://github.com/cosmos/cosmos-sdk/pull/26611) Fix missing key rotation type tags on genesis import. +- (blockstm) [#26627](https://github.com/cosmos/cosmos-sdk/pull/26627) Guard against block-stm estimate panic. +- (x/staking) [#26616](https://github.com/cosmos/cosmos-sdk/pull/26616) Expire historical cons addr lookups only once equivocation evidence is no longer admissible. +- (x/poa) [#26642](https://github.com/cosmos/cosmos-sdk/pull/26642) Always return error when migrating fees to an occupied key. +- (x/staking) [#26641](https://github.com/cosmos/cosmos-sdk/pull/26641) Allow multiple history entires in genesis import, and fix labeling of historical entries. diff --git a/sdk/next/enterprise/group/api.mdx b/sdk/next/enterprise/group/api.mdx index 38b693fdd..e3e56f641 100644 --- a/sdk/next/enterprise/group/api.mdx +++ b/sdk/next/enterprise/group/api.mdx @@ -1,4 +1,6 @@ --- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/enterprise/group/api' title: "API Reference" description: "Complete API reference for Group module queries and messages" --- diff --git a/sdk/next/enterprise/group/architecture.mdx b/sdk/next/enterprise/group/architecture.mdx index d1a3fc960..3b83cda96 100644 --- a/sdk/next/enterprise/group/architecture.mdx +++ b/sdk/next/enterprise/group/architecture.mdx @@ -1,4 +1,6 @@ --- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/enterprise/group/architecture' title: "Architecture" description: "System architecture, core concepts, and module integration details for the Group module" --- diff --git a/sdk/next/enterprise/group/overview.mdx b/sdk/next/enterprise/group/overview.mdx index 701ebf1c9..39cba3c85 100644 --- a/sdk/next/enterprise/group/overview.mdx +++ b/sdk/next/enterprise/group/overview.mdx @@ -1,4 +1,6 @@ --- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/enterprise/group/overview' title: "Overview" description: "On-Chain Multisig Accounts and Collective Decision-Making" --- @@ -16,7 +18,7 @@ The Group module is designed for networks that require: ## Source Code -The source code for the Group module can be found [here](https://github.com/cosmos/cosmos-sdk/tree/main/enterprise/group). +The source code for the Group module can be found [here](https://github.com/cosmos/cosmos-sdk/tree/release/v0.55.x/enterprise/group). ## Available Documentation @@ -27,6 +29,6 @@ This section contains detailed documentation for the Group module. ## Licensing -The Group module source is published under the [Source Available Evaluation License](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/group/LICENSE), which permits evaluation and testing in non-production environments only. Production or commercial use requires an Enterprise License from Cosmos Labs. +The Group module source is published under the [Source Available Evaluation License](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/group/LICENSE), which permits evaluation and testing in non-production environments only. Production or commercial use requires an Enterprise License from Cosmos Labs. To use the Group module in production, contact sales@cosmoslabs.io. diff --git a/sdk/next/enterprise/overview.mdx b/sdk/next/enterprise/overview.mdx index e80c1ec00..09dca1205 100644 --- a/sdk/next/enterprise/overview.mdx +++ b/sdk/next/enterprise/overview.mdx @@ -1,11 +1,13 @@ --- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/enterprise/overview' title: "Overview" description: "Source-available Cosmos SDK modules for permissioned and enterprise blockchain networks." --- Cosmos Enterprise modules are production-ready modules for permissioned networks, institutional chains, and enterprise deployments that need features beyond a public blockchain architecture. They follow the same patterns as the core modules and integrate alongside them. -The module source is published in the [`enterprise` directory of the Cosmos SDK repository](https://github.com/cosmos/cosmos-sdk/tree/main/enterprise). +The module source is published in the [`enterprise` directory of the Cosmos SDK repository](https://github.com/cosmos/cosmos-sdk/tree/release/v0.55.x/enterprise). ## Available modules diff --git a/sdk/next/enterprise/poa/api.mdx b/sdk/next/enterprise/poa/api.mdx index 2c8a83a63..13e26a431 100644 --- a/sdk/next/enterprise/poa/api.mdx +++ b/sdk/next/enterprise/poa/api.mdx @@ -1,4 +1,6 @@ --- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/enterprise/poa/api' title: "API Reference" description: "Complete API reference for PoA module gRPC queries and transactions" --- @@ -502,6 +504,46 @@ simd tx poa withdraw-fees \ --- +### RotateConsPubKey + +Replace a validator's consensus public key in place. + +**gRPC:** `cosmos.poa.v1.Msg/RotateConsPubKey` + +**Message:** +```protobuf +message MsgRotateConsPubKey { + string sender = 1; // Signer: the validator's operator or the chain admin + string validator_address = 2; // Operator address identifying the validator + google.protobuf.Any new_pub_key = 3; // New consensus public key +} +``` + +**Response:** +```protobuf +message MsgRotateConsPubKeyResponse {} +``` + +**CLI:** +```bash +simd tx poa rotate-cons-pub-key \ + --operator-address \ + --from \ + -y +``` + +**Authorization:** Must be signed by the validator's operator address or the chain admin. + +**Notes:** +- Re-keys the validator and migrates its accrued fees in the same block +- Power, metadata, and the operator address are unchanged +- No fee, no rate limit, and no rotation history, unlike `x/staking` rotation +- The new key's type must be in the chain's consensus params, must not equal the current key, and must not belong to another validator + +For the operational procedure, see [Rotate a consensus key, PoA](/sdk/next/keys/rotate-validator-key-poa). + +--- + ## Common Use Cases ### 1. Query Current Admin diff --git a/sdk/next/enterprise/poa/architecture.mdx b/sdk/next/enterprise/poa/architecture.mdx index dbf234a43..7eda2ddb3 100644 --- a/sdk/next/enterprise/poa/architecture.mdx +++ b/sdk/next/enterprise/poa/architecture.mdx @@ -1,4 +1,6 @@ --- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/enterprise/poa/architecture' title: "Architecture" description: "System architecture and module integration details for the PoA module" --- @@ -9,6 +11,10 @@ description: "System architecture and module integration details for the PoA mod The Proof of Authority (PoA) permissioned consensus module is a Cosmos SDK module that implements a permissioned consensus mechanism where a designated admin controls the validator set. Unlike traditional Proof of Stake systems, PoA validators are explicitly authorized and managed by an administrative authority rather than being selected based on staked tokens. + +Vote extensions are not supported on PoA chains. A consensus key rotation leaves the SDK-side consensus address out of sync with CometBFT for two heights, and during that window the chain cannot verify a rotating validator's vote-extension signatures. If more than 2/3 of voting power rotates within one window, the chain halts. See [Key rotation](/sdk/next/keys/key-rotation) for the mechanism. + + ## Table of Contents - [Architecture](#architecture) @@ -74,10 +80,9 @@ Standard SDK [governance](/sdk/next/modules/gov/README) uses bonded tokens for v **Storage Design Philosophy** -The module uses `cosmossdk.io/collections` with a composite key structure: -- Primary key: `(power, consensus_address)` enables efficient power-sorted iteration -- Secondary indexes on consensus and operator addresses for fast lookups -- Requires re-keying when power changes, but eliminates need for separate sorting +The module uses `cosmossdk.io/collections` with an indexed map: +- Primary key: consensus address +- Secondary indexes on operator address and power, for fast lookups and power-sorted iteration - See [Storage Design](#storage-design) for technical details ## Admin Control Flow @@ -91,13 +96,13 @@ The PoA module is controlled by a single admin address configured at genesis. Th The admin could be set to any authority that has an address. This includes a group from x/groups, the governance module account, and multisigs. -**Location**: Admin address stored in [`x/poa/types/keys.go:10`](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/x/poa/types/keys.go#L10) (params prefix) +**Location**: Admin address stored in [`x/poa/types/keys.go:26`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/poa/x/poa/types/keys.go#L26) (params prefix) Only the admin can update itself with a parameter change. ### Managing Validator Set -**MsgUpdateValidators** ([`x/poa/keeper/msg_server.go:72`](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/x/poa/keeper/msg_server.go#L72)) +**MsgUpdateValidators** ([`x/poa/keeper/msg_server.go:142`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/poa/x/poa/keeper/msg_server.go#L142)) The admin can batch update validators through a single transaction: @@ -111,11 +116,11 @@ The admin can batch update validators through a single transaction: - Fee checkpoint (allocates pending fees before power changes) - Total power recalculation - ABCI validator update queue -4. **Consensus Update**: Changes take effect at the end of the current block +4. **Consensus Update**: Changes take effect in the next block ### Updating Parameters -**MsgUpdateParams** ([`x/poa/keeper/msg_server.go:26`](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/x/poa/keeper/msg_server.go#L26)) +**MsgUpdateParams** ([`x/poa/keeper/msg_server.go:45`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/poa/x/poa/keeper/msg_server.go#L45)) The admin can update module parameters (currently only the admin address itself). This requires: - Transaction signed by current admin @@ -125,9 +130,9 @@ The admin can update module parameters (currently only the admin address itself) ### Validator Registration -**MsgCreateValidator** ([`x/poa/keeper/msg_server.go:45`](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/x/poa/keeper/msg_server.go#L45)) +**MsgCreateValidator** ([`x/poa/keeper/msg_server.go:79`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/poa/x/poa/keeper/msg_server.go#L79)) -**Permissionless Creation**: Any address can register as a validator candidate: +**Admin-Only Creation**: Only the admin can register a validator: 1. **Submit Registration**: Provide public key and metadata - **PubKey**: Ed25519 @@ -135,12 +140,12 @@ The admin can update module parameters (currently only the admin address itself) - **Moniker**: Human-readable name (max 256 chars) - **Description**: Additional details (max 256 chars) -2. **Initial State**: Created validators have **power = 0** until the admin updates it via `MsgUpdateValidators` +2. **Initial State**: The admin sets the validator's initial power. A validator with **power = 0** is inactive: - Not participating in consensus - Not earning fees - Cannot vote in governance -**Location**: [`x/poa/keeper/validator.go:95`](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/x/poa/keeper/validator.go#L95) +**Location**: [`x/poa/keeper/validator.go:121`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/poa/x/poa/keeper/validator.go#L121) ### Gaining Consensus Power @@ -158,7 +163,7 @@ Validators can only gain consensus power through admin action: - Power can be adjusted up or down by admin - Setting power = 0 removes validator from consensus without deleting -**Location**: [`x/poa/keeper/validator.go:19`](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/x/poa/keeper/validator.go#L19) +**Location**: [`x/poa/keeper/validator.go:34`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/poa/x/poa/keeper/validator.go#L34) ### Removing Validators @@ -183,7 +188,7 @@ The PoA module implements a custom checkpoint-based fee distribution system that **See [Fee Distribution Documentation](/sdk/next/enterprise/poa/distribution)** for complete details. -**Location**: [`x/poa/keeper/distribution.go`](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/x/poa/keeper/distribution.go) +**Location**: [`x/poa/keeper/distribution.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/poa/x/poa/keeper/distribution.go) ## Governance @@ -200,31 +205,31 @@ The PoA module restricts governance participation to active validators only, usi **See [Governance Documentation](/sdk/next/enterprise/poa/governance)** for complete details. -**Location**: [`x/poa/keeper/governance.go`](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/x/poa/keeper/governance.go) and [`x/poa/keeper/hooks.go`](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/x/poa/keeper/hooks.go) +**Location**: [`x/poa/keeper/governance.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/poa/x/poa/keeper/governance.go) and [`x/poa/keeper/hooks.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/poa/x/poa/keeper/hooks.go) ## Technical Implementation ### Storage Design -**Collections Schema** ([`x/poa/types/keys.go`](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/x/poa/types/keys.go)) +**Collections Schema** ([`x/poa/types/keys.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/poa/x/poa/types/keys.go)) The module uses `cosmossdk.io/collections` for type-safe state management: | Prefix | Collection | Key Type | Value Type | Purpose | |--------|------------|----------|------------|---------| | 0 | `params` | - | `Params` | Admin address and module config | -| 1 | `validators` | `(int64, string)` | `Validator` | Primary map, sorted by power | -| 2 | `validator_by_consensus` | `string` | `(int64, string)` | Index: consensus addr → composite key | -| 3 | `validator_by_operator` | `string` | `(int64, string)` | Index: operator addr → composite key | +| 1 | `validators` | `ConsAddress` | `Validator` | Primary map, keyed by consensus address | +| 2 | `validator_by_operator` | `string` | `ConsAddress` | Index: operator addr → consensus addr | +| 3 | `validator_by_power` | `(int64, ConsAddress)` | - | Index: power-sorted iteration | | 4 | `total_power` | - | `int64` | Sum of all validator power | | 5 | `total_allocated` | - | `ValidatorFees` | Sum of allocated fees | -**Location**: [`x/poa/keeper/keeper.go:16`](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/x/poa/keeper/keeper.go#L16) +**Location**: [`x/poa/keeper/keeper.go:38`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/poa/x/poa/keeper/keeper.go#L38) ### ABCI Integration -**EndBlocker** ([`x/poa/keeper/abci.go:9`](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/x/poa/keeper/abci.go#L9)) +**EndBlocker** ([`x/poa/keeper/abci.go:30`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/poa/x/poa/keeper/abci.go#L30)) The module integrates with CometBFT consensus through ABCI: @@ -242,7 +247,7 @@ ValidatorUpdate { } ``` -**Location**: [`x/poa/module.go:128`](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/x/poa/module.go#L128) +**Location**: [`x/poa/module.go:266`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/poa/x/poa/module.go#L266) ## Security Considerations @@ -250,8 +255,8 @@ ValidatorUpdate { - Admin address controls entire validator set 2. **Validator Registration**: - - Anyone can register as validator candidate - - Only admin can grant consensus power + - Only the admin can register a validator + - Only the admin can grant consensus power 3. **Total Power Invariant**: - Total power must remain > 0 diff --git a/sdk/next/enterprise/poa/distribution.mdx b/sdk/next/enterprise/poa/distribution.mdx index 5f2bfaaf5..6b18a86d9 100644 --- a/sdk/next/enterprise/poa/distribution.mdx +++ b/sdk/next/enterprise/poa/distribution.mdx @@ -1,4 +1,6 @@ --- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/enterprise/poa/distribution' title: "Fee Distribution" description: "Fee distribution mechanics and algorithms in the PoA module" --- @@ -13,7 +15,7 @@ The PoA module implements a custom fee distribution mechanism based on validator Fees flow through the PoA system differently than standard Cosmos SDK: -1. **Block Fees**: Transaction fees collected in each block go to the `fee_collector` module account by default, or to the PoA module account if configured (see [Fee Routing Setup](#fee-routing-setup)) +1. **Block Fees**: Transaction fees collected in each block go to the PoA module account (see [Fee Routing Setup](#fee-routing-setup)) 2. **Checkpoint System**: Allocated fees are updated for validators when: - Any validator power changes @@ -21,7 +23,7 @@ Fees flow through the PoA system differently than standard Cosmos SDK: **Why Checkpointing?**: Ensures fair distribution when power changes. If power changes mid-period, fees are allocated based on old power distribution before the change takes effect. -**Location**: [`x/poa/keeper/distribution.go:18`](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/x/poa/keeper/distribution.go#L18) +**Location**: [`x/poa/keeper/distribution.go:32`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/poa/x/poa/keeper/distribution.go#L32) ## Distribution Algorithm @@ -119,7 +121,7 @@ After this checkpoint, $A_{total}(t+1) = B_{collector}(t)$ (all fees are now all ## Withdrawing Fees -**MsgWithdrawFees** ([`x/poa/keeper/msg_server.go:91`](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/x/poa/keeper/msg_server.go#L91)) +**MsgWithdrawFees** ([`x/poa/keeper/msg_server.go:182`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/poa/x/poa/keeper/msg_server.go#L182)) Any validator operator can withdraw accumulated fees: @@ -137,7 +139,7 @@ Withdrawal: 100 utokens transferred to operator Remainder: 0.7543 utokens remain allocated (less than least significant utoken digit) ``` -**Location**: [`x/poa/keeper/distribution.go:106`](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/x/poa/keeper/distribution.go#L106) +**Location**: [`x/poa/keeper/distribution.go:135`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/poa/x/poa/keeper/distribution.go#L135) ## Withdrawal Formula @@ -164,7 +166,7 @@ Where: ## Fee Routing Setup -PoA has its own module account for collecting fees. Enabling the PoA module account is recommended to keep fee accounting isolated and accurate. If not enabled, fees are deposited into the standard `fee_collector` account by default. +PoA has its own module account for collecting fees. Enabling the PoA module account is required to keep fee accounting isolated and accurate. The chain panics at block 1 if the ante handler's fee recipient is not the PoA module. To enable the PoA module account, two wiring changes are required: @@ -186,7 +188,7 @@ app.AccountKeeper = authkeeper.NewAccountKeeper( ) ``` -**Source**: [`simapp/app.go`](https://github.com/cosmos/cosmos-sdk/blob/7bc1b146d437d834d971f415924104188203c96f/enterprise/poa/simapp/app.go#L191) +**Source**: [`simapp/app.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/poa/simapp/app.go#L182-L194) ### 2. Configure the Ante Handler @@ -210,9 +212,7 @@ anteDecorators := []sdk.AnteDecorator{ } ``` -**Source**: [`simapp/ante.go`](https://github.com/cosmos/cosmos-sdk/blob/7bc1b146d437d834d971f415924104188203c96f/enterprise/poa/simapp/ante.go#L49) - -`WithFeeRecipientModule` is backwards compatible — omitting it defaults to the standard `fee_collector` behavior. +**Source**: [`simapp/ante.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/poa/simapp/ante.go#L41-L56) ## Security Considerations diff --git a/sdk/next/enterprise/poa/governance.mdx b/sdk/next/enterprise/poa/governance.mdx index 1d1b6b9a6..f380558ba 100644 --- a/sdk/next/enterprise/poa/governance.mdx +++ b/sdk/next/enterprise/poa/governance.mdx @@ -1,4 +1,6 @@ --- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/enterprise/poa/governance' title: "Governance Integration" description: "Governance integration and power-based voting in the PoA module" --- @@ -13,7 +15,7 @@ The PoA module integrates with Cosmos SDK governance to restrict participation t The PoA module restricts governance participation to authorized validators only through governance hooks. -**Governance Hooks** ([`x/poa/keeper/hooks.go`](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/x/poa/keeper/hooks.go)) +**Governance Hooks** ([`x/poa/keeper/hooks.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/poa/x/poa/keeper/hooks.go)) The module implements `govtypes.GovHooks`: @@ -29,13 +31,13 @@ The module implements `govtypes.GovHooks`: **Rejected Actions**: - If non-validator attempts governance action → transaction fails - If validator has power = 0 → transaction fails -- Error: "voter X is not an active PoA validator" +- Error: "voter X is not an active POA validator" -**Location**: [`x/poa/keeper/governance.go:92`](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/x/poa/keeper/governance.go#L92) +**Location**: [`x/poa/keeper/governance.go:115`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/poa/x/poa/keeper/governance.go#L115) ## Voting Power -**Custom Vote Tallying** ([`x/poa/keeper/governance.go:18`](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/x/poa/keeper/governance.go#L18)). An example of the wiring can be found in the [SimApp](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/simapp/app.go#L197-214). +**Custom Vote Tallying** ([`x/poa/keeper/governance.go:38`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/poa/x/poa/keeper/governance.go#L38)). An example of the wiring can be found in the [SimApp](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/poa/simapp/app.go#L214-L224). Standard governance uses staked tokens as voting weight. PoA governance uses validator power: @@ -96,7 +98,7 @@ Where: ### 1. Proposal Submission -**[MsgSubmitProposal](https://github.com/cosmos/cosmos-sdk/blob/main/proto/cosmos/gov/v1/tx.proto#L54-L65)** (standard x/gov module) +**[MsgSubmitProposal](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/gov/v1/tx.proto#L57-L88)** (standard x/gov module) When a proposal is submitted: @@ -112,7 +114,7 @@ When a proposal is submitted: ### 2. Deposit Period -**[MsgDeposit](https://github.com/cosmos/cosmos-sdk/blob/main/proto/cosmos/gov/v1/tx.proto#L90-L98)** (standard x/gov module) +**[MsgDeposit](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/gov/v1/tx.proto#L153-L166)** (standard x/gov module) When a deposit is made: @@ -125,7 +127,7 @@ When a deposit is made: ### 3. Voting Period -**[MsgVote](https://github.com/cosmos/cosmos-sdk/blob/main/proto/cosmos/gov/v1/tx.proto#L100-L108)** or **[MsgVoteWeighted](https://github.com/cosmos/cosmos-sdk/blob/main/proto/cosmos/gov/v1/tx.proto#L110-L118)** (standard x/gov module) +**[MsgVote](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/gov/v1/tx.proto#L111-L127)** or **[MsgVoteWeighted](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/gov/v1/tx.proto#L132-L148)** (standard x/gov module) When a vote is cast: @@ -146,7 +148,7 @@ When a vote is cast: At the end of the voting period, the [custom tally function](#vote-tallying-algorithm) is called: -**NewPoACalculateVoteResultsAndVotingPowerFn** ([`x/poa/keeper/governance.go:18`](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/x/poa/keeper/governance.go#L18)) +**NewPOACalculateVoteResultsAndVotingPowerFn** ([`x/poa/keeper/governance.go:38`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/poa/x/poa/keeper/governance.go#L38)) 1. Iterate all votes on the proposal 2. For each vote, look up the validator by voter address @@ -165,13 +167,13 @@ At the end of the voting period, the [custom tally function](#vote-tallying-algo ### Governance Hooks -**Location**: [`x/poa/keeper/hooks.go`](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/x/poa/keeper/hooks.go) +**Location**: [`x/poa/keeper/hooks.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/poa/x/poa/keeper/hooks.go) The module implements the `govtypes.GovHooks` interface: ``` type GovHooks interface { - AfterProposalSubmission(ctx, proposalID, depositorAddr) + AfterProposalSubmission(ctx, proposalID, proposerAddr) AfterProposalDeposit(ctx, proposalID, depositorAddr) AfterProposalVote(ctx, proposalID, voterAddr) // ... other hooks @@ -179,23 +181,23 @@ type GovHooks interface { ``` Each hook implementation: -1. Extracts the operator address from the context +1. Receives the operator address from the governance hook 2. Looks up the validator by operator address 3. Checks if validator exists and has power > 0 4. Returns error if validation fails ### Custom Tally Function -**Location**: [`x/poa/keeper/governance.go:18`](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/x/poa/keeper/governance.go#L18) +**Location**: [`x/poa/keeper/governance.go:38`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/poa/x/poa/keeper/governance.go#L38) The tally function replaces the standard governance tally: ```go -func NewPoACalculateVoteResultsAndVotingPowerFn(keeper) TallyFn { +func NewPOACalculateVoteResultsAndVotingPowerFn(keeper) TallyFn { return func(ctx, proposal) (totalVotingPower, results) { // Iterate votes for vote in votes(proposal) { - validator = keeper.GetValidatorByOperator(vote.voter) + validator = keeper.GetValidatorByOperatorAddress(vote.voter) if validator == nil || validator.Power <= 0 { continue // Skip non-authorized validators } @@ -239,6 +241,7 @@ The standard governance module parameters still apply: 3. **Admin Governance Control**: - Admin can change validator power at any time + - Admin can rotate any validator's consensus key on the operator's behalf - Admin can effectively control governance by adjusting power - Consider multi-sig admin or governance-controlled admin changes diff --git a/sdk/next/enterprise/poa/overview.mdx b/sdk/next/enterprise/poa/overview.mdx index 7b333caa9..2411c69b3 100644 --- a/sdk/next/enterprise/poa/overview.mdx +++ b/sdk/next/enterprise/poa/overview.mdx @@ -1,4 +1,6 @@ --- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/enterprise/poa/overview' title: "Overview" description: "Enterprise-Ready Network Security and Operations" --- @@ -27,7 +29,7 @@ The PoA module is designed for networks that require: ## Source Code -The source code for the Proof of Authority module can be found [here](https://github.com/cosmos/cosmos-sdk/tree/main/enterprise/poa). +The source code for the Proof of Authority module can be found [here](https://github.com/cosmos/cosmos-sdk/tree/release/v0.55.x/enterprise/poa). ## Available Documentation @@ -37,9 +39,10 @@ This directory contains detailed documentation for the Proof of Authority module - **[Architecture](/sdk/next/enterprise/poa/architecture)** - System architecture and module integration details - **[Distribution](/sdk/next/enterprise/poa/distribution)** - Fee distribution mechanics and algorithms - **[Governance](/sdk/next/enterprise/poa/governance)** - Governance integration and power-based voting +- **[Rotate a consensus key](/sdk/next/keys/rotate-validator-key-poa)** - Rotate a PoA validator's consensus key as the operator or the admin ## Licensing -The Proof of Authority module source is published under the [Source Available Evaluation License](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/LICENSE), which permits evaluation and testing in non-production environments only. Production or commercial use requires an Enterprise License from Cosmos Labs. +The Proof of Authority module source is published under the [Source Available Evaluation License](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/poa/LICENSE), which permits evaluation and testing in non-production environments only. Production or commercial use requires an Enterprise License from Cosmos Labs. To use the Proof of Authority module in production, contact sales@cosmoslabs.io. \ No newline at end of file diff --git a/sdk/next/experimental/blockstm.mdx b/sdk/next/experimental/blockstm.mdx index ce9a49c9b..0d04306ce 100644 --- a/sdk/next/experimental/blockstm.mdx +++ b/sdk/next/experimental/blockstm.mdx @@ -171,6 +171,28 @@ bApp.SetBlockSTMTxRunner(txnrunner.NewSTMRunner( )) ``` +### Configuration via app.toml + +The wiring above installs Block-STM programmatically. An application that uses `blockexec.Apply` can instead select the executor from `app.toml`, or from the equivalent `simd start` flags. Three keys control this: + +| Key | Type | Default | Description | +| --- | --- | --- | --- | +| `block-executor` | string | `sequential` | Selects the execution strategy. Set it to `block-stm` to enable parallel execution. | +| `block-stm-workers` | int | `0` | Sets the worker count. This maps to the `workers` runner parameter. A value of `0` resolves to `min(GOMAXPROCS, NumCPU)` at runtime. | +| `block-stm-pre-estimate` | bool | `false` | Enables pre-estimation of read and write conflicts. This maps to the `estimate` runner parameter. | + +When `block-executor` is set to `block-stm`, the block gas meter is disabled automatically. This is required, because the parallel runner panics if the block gas meter is still enabled. The programmatic wiring above does not disable it for you. Call `SetDisableBlockGasMeter(true)` when you wire the runner by hand. + +Example `app.toml` that enables Block-STM: + +```toml +block-executor = "block-stm" +block-stm-workers = 0 +block-stm-pre-estimate = true +``` + +The `block-stm-pre-estimate` value is set to `true` here to match the `estimate` guidance above. + ## Parallel Transaction Optimization Once Block-STM is wired in, you may initially notice that most blocks execute slower than with serial execution. This is due to the overhead of re-executing transactions when any two have conflicting reads or writes. To realize performance gains, you need to reduce storage access conflicts between transactions. diff --git a/sdk/next/guides/abci/abci.mdx b/sdk/next/guides/abci/abci.mdx index 6563719fa..5fe23ab26 100644 --- a/sdk/next/guides/abci/abci.mdx +++ b/sdk/next/guides/abci/abci.mdx @@ -18,7 +18,7 @@ ABCI, Application Blockchain Interface is the interface between CometBFT and the * `VerifyVoteExtension` * `FinalizeBlock` -The Cosmos SDK's `BaseApp` implements the full ABCI interface. The source lives in [`baseapp/abci.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/baseapp/abci.go). +The Cosmos SDK's `BaseApp` implements the full ABCI interface. The source lives in [`baseapp/abci.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/baseapp/abci.go). ## CheckTx @@ -38,11 +38,11 @@ graph TD The default implementation runs the transaction through the `AnteHandler` chain, which performs signature verification, fee checks, and other stateless or lightweight stateful validation. If the `AnteHandler` returns an error, the transaction is rejected and never reaches the mempool. -See the implementation at [`baseapp/abci.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/baseapp/abci.go). +See the implementation at [`baseapp/abci.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/baseapp/abci.go). ### Custom CheckTx handler -`CheckTxHandler` lets you replace the default `CheckTx` logic entirely. The type is defined in [`types/abci.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/types/abci.go): +`CheckTxHandler` lets you replace the default `CheckTx` logic entirely. The type is defined in [`types/abci.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/types/abci.go): ```go type CheckTxHandler func(runTx RunTx, req *abci.RequestCheckTx) (*abci.ResponseCheckTx, error) @@ -70,7 +70,7 @@ CometBFT's own mempool uses FIFO ordering. `PrepareProposal` gives the applicati `PrepareProposal` MAY be non-deterministic and is only executed by the current block proposer. -The Cosmos SDK provides `DefaultProposalHandler` in [`baseapp/abci_utils.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/baseapp/abci_utils.go), which selects transactions from the app-side mempool up to `req.MaxTxBytes` and the block gas limit. +The Cosmos SDK provides `DefaultProposalHandler` in [`baseapp/abci_utils.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/baseapp/abci_utils.go), which selects transactions from the app-side mempool up to `req.MaxTxBytes` and the block gas limit. @@ -93,7 +93,7 @@ After the block proposer broadcasts a proposal, every validator calls `ProcessPr `ProcessProposal` MUST be deterministic. Non-deterministic results cause apphash mismatches across validators. If the handler panics or returns an error, honest validators prevote nil and CometBFT starts a new round with a new proposal. -See the default implementation in [`baseapp/abci_utils.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/baseapp/abci_utils.go). +See the default implementation in [`baseapp/abci_utils.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/baseapp/abci_utils.go). To wire a custom handler: @@ -117,4 +117,4 @@ See [Vote Extensions](/sdk/next/guides/abci/vote-extensions) for implementation `FinalizeBlock` is called once consensus is reached on a proposal. It executes all transactions in the block, runs `BeginBlock`/`EndBlock` equivalents, and commits the resulting state. It replaces the old `BeginBlock`, `DeliverTx`, and `EndBlock` methods from ABCI 1.0. -See the implementation at [`baseapp/abci.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/baseapp/abci.go). +See the implementation at [`baseapp/abci.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/baseapp/abci.go). diff --git a/sdk/next/guides/abci/app-mempool.mdx b/sdk/next/guides/abci/app-mempool.mdx index 3dc97b756..7a2e840ba 100644 --- a/sdk/next/guides/abci/app-mempool.mdx +++ b/sdk/next/guides/abci/app-mempool.mdx @@ -38,6 +38,10 @@ chooses. So CometBFT decides what gets accepted into the network; the SDK app mempool decides how accepted transactions are ordered within a block. + +This in-process mempool should not be confused with CometBFT's `app` mempool. The mempool on this page orders transactions at `PrepareProposal` time. Setting `mempool.type = "app"` in CometBFT is a separate mechanism that routes transaction receipt itself to the application through the `InsertTx` and `ReapTxs` ABCI methods. That mechanism is documented in the [CometBFT mempool guide](/cometbft/next/docs/core/mempool). + + ## Mempool There are countless designs that an application developer can write for a mempool, the SDK opted to provide only simple mempool implementations. @@ -47,7 +51,7 @@ Namely, the SDK provides the following mempools: * [Sender Nonce Mempool](#sender-nonce-mempool) * [Priority Nonce Mempool](#priority-nonce-mempool) -By default, the SDK uses the [No-op Mempool](#no-op-mempool), but it can be replaced by the application developer in [`app.go`: +By default, the SDK uses the [No-op Mempool](#no-op-mempool), but it can be replaced by the application developer in `app.go`: ```go nonceMempool := mempool.NewSenderNonceMempool() @@ -67,7 +71,7 @@ which is FIFO-ordered by default. ### Sender Nonce Mempool -The nonce mempool is a mempool that keeps transactions from an sorted by nonce in order to avoid the issues with nonces. +The nonce mempool keeps each account's transactions sorted by nonce, so they are proposed in the order the account signed them. It works by storing the transaction in a list sorted by the transaction nonce. When the proposer asks for transactions to be included in a block it randomly selects a sender and gets the first transaction in the list. It repeats this until the mempool is empty or the block is full. It is configurable with the following parameters: @@ -86,7 +90,7 @@ Set the seed for the random number generator used to select transactions from th ### Priority Nonce Mempool -The [priority nonce mempool](https://github.com/cosmos/cosmos-sdk/blob/main/types/mempool/priority_nonce_spec.md) is a mempool implementation that stores txs in a partially ordered set by 2 dimensions: +The [priority nonce mempool](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/types/mempool/priority_nonce_spec.md) is a mempool implementation that stores txs in a partially ordered set by 2 dimensions: * priority * sender-nonce (sequence number) diff --git a/sdk/next/guides/abci/vote-extensions.mdx b/sdk/next/guides/abci/vote-extensions.mdx index 46a5762a2..662152e39 100644 --- a/sdk/next/guides/abci/vote-extensions.mdx +++ b/sdk/next/guides/abci/vote-extensions.mdx @@ -22,13 +22,13 @@ if cp.Abci != nil && req.Height > cp.Abci.VoteExtensionsEnableHeight { ## ExtendVote -The Cosmos SDK defines [`ExtendVoteHandler`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/types/abci.go#L48): +The Cosmos SDK defines [`ExtendVoteHandler`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/types/abci.go#L48): ```go type ExtendVoteHandler func(Context, *abci.RequestExtendVote) (*abci.ResponseExtendVote, error) ``` -Register a handler in `app.go` via `baseapp.SetExtendVoteHandler` (defined in [`baseapp/options.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/baseapp/options.go)): +Register a handler in `app.go` via `baseapp.SetExtendVoteHandler` (defined in [`baseapp/options.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/baseapp/options.go)): ```go app.SetExtendVoteHandler(myExtendVoteHandler) @@ -45,7 +45,7 @@ Keep extensions small — large extensions increase consensus latency. See [Come ## VerifyVoteExtension -The SDK defines [`VerifyVoteExtensionHandler`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/types/abci.go#L52): +The SDK defines [`VerifyVoteExtensionHandler`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/types/abci.go#L52): ```go type VerifyVoteExtensionHandler func(Context, *abci.RequestVerifyVoteExtension) (*abci.ResponseVerifyVoteExtension, error) @@ -63,7 +63,7 @@ Always validate the size of incoming extensions in this handler. ## Validating vote extension signatures -Before processing vote extensions in `PrepareProposal` or `ProcessProposal`, validate that they are properly signed. The SDK provides [`baseapp.ValidateVoteExtensions`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/baseapp/abci_utils.go) for this: +Before processing vote extensions in `PrepareProposal` or `ProcessProposal`, validate that they are properly signed. The SDK provides [`baseapp.ValidateVoteExtensions`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/baseapp/abci_utils.go) for this: ```go err := baseapp.ValidateVoteExtensions(ctx, valStore, req.Height, ctx.ChainID(), req.LocalLastCommit) @@ -72,7 +72,7 @@ if err != nil { } ``` -`ValidateVoteExtensions` verifies that each vote extension in the commit is correctly signed by its validator. `valStore` is a [`baseapp.ValidatorStore`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/baseapp/abci_utils.go), an interface with a single method: +`ValidateVoteExtensions` verifies that each vote extension in the commit is correctly signed by its validator. `valStore` is a [`baseapp.ValidatorStore`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/baseapp/abci_utils.go), an interface with a single method: ```go type ValidatorStore interface { @@ -102,7 +102,7 @@ proposalTxs = append([][]byte{bz}, proposalTxs...) `FinalizeBlock` ignores any byte slice that does not implement `sdk.Tx`, so injected extensions are safely skipped during message execution. -For more details on propagation design, see the [ABCI 2.0 ADR](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/docs/architecture/adr-064-abci-2.0.md#vote-extension-propagation--verification). +For more details on propagation design, see the [ABCI 2.0 ADR](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/docs/architecture/adr-064-abci-2.0.md#vote-extension-propagation--verification). ## Recovery via PreBlocker @@ -128,13 +128,13 @@ func (h *ProposalHandler) PreBlocker(ctx sdk.Context, req *abci.RequestFinalizeB } ``` -Register the PreBlocker in `app.go` (see [`baseapp/options.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/baseapp/options.go)): +Register the PreBlocker in `app.go` (see [`baseapp/options.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/baseapp/options.go)): ```go app.SetPreBlocker(proposalHandler.PreBlocker) ``` -The `sdk.PreBlocker` type is defined in [`types/abci.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/types/abci.go): +The `sdk.PreBlocker` type is defined in [`types/abci.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/types/abci.go): ```go type PreBlocker func(Context, *abci.RequestFinalizeBlock) (*ResponsePreBlock, error) diff --git a/sdk/next/guides/module-design/module-design-considerations.mdx b/sdk/next/guides/module-design/module-design-considerations.mdx index 9c19188be..b2eee1a73 100644 --- a/sdk/next/guides/module-design/module-design-considerations.mdx +++ b/sdk/next/guides/module-design/module-design-considerations.mdx @@ -33,7 +33,7 @@ Ask: could a different chain reasonably use this module without modification? If ### Plan your state structure early -Every `KVStore` key your module defines is permanent: removing or renaming keys requires a migration. Use the [Collections](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/collections/README.md) library for structured state management, and name keys to be collision-resistant and self-documenting. +Every `KVStore` key your module defines is permanent: removing or renaming keys requires a migration. Use the [Collections](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/collections/README.md) library for structured state management, and name keys to be collision-resistant and self-documenting. Consider what your module needs to index. A value that is only ever looked up by a single key is simple. A value looked up by multiple dimensions (e.g. by owner and by ID) requires secondary indexes, which add complexity and storage overhead. diff --git a/sdk/next/guides/module-design/ocap.mdx b/sdk/next/guides/module-design/ocap.mdx index 25d53219c..01e157e9d 100644 --- a/sdk/next/guides/module-design/ocap.mdx +++ b/sdk/next/guides/module-design/ocap.mdx @@ -94,6 +94,6 @@ if msg.Authority != k.authority { The authority address is set at wiring time in `app.go` and cannot be changed at runtime. This is ocap applied to governance: privileged capability is a reference, and only the holder of that reference can exercise it. -See [`simapp/app.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/simapp/app.go) for how keeper dependencies and authorities are wired in a complete application. +See [`simapp/app.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/simapp/app.go) for how keeper dependencies and authorities are wired in a complete application. For background, see the [Wikipedia article on object-capability model](https://en.wikipedia.org/wiki/Object-capability_model). diff --git a/sdk/next/guides/reference/bech32.mdx b/sdk/next/guides/reference/bech32.mdx index 5efc95c10..1f18cba38 100644 --- a/sdk/next/guides/reference/bech32.mdx +++ b/sdk/next/guides/reference/bech32.mdx @@ -100,7 +100,7 @@ bech32Address, _ := bech32.Encode("cosmos", converted) ## Configuring Bech32 prefixes -Every Cosmos SDK application sets its Bech32 prefixes and SLIP-44 coin type once at startup via `sdk.GetConfig()`, then seals the config so it cannot be changed at runtime. The defaults (`cosmos`, `cosmosvaloper`, etc.) are defined in [`types/config.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/types/config.go). Chain developers override them before the app starts: +Every Cosmos SDK application sets its Bech32 prefixes and SLIP-44 coin type once at startup via `sdk.GetConfig()`, then seals the config so it cannot be changed at runtime. The defaults (`cosmos`, `cosmosvaloper`, etc.) are defined in [`types/config.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/types/config.go). Chain developers override them before the app starts: ```go config := sdk.GetConfig() @@ -136,7 +136,7 @@ func (bc Bech32Codec) StringToBytes(text string) ([]byte, error) { ## Module Addresses -Module accounts use deterministic address derivation defined in [ADR-028](https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-028-public-key-addresses.md): +Module accounts use deterministic address derivation defined in [ADR-028](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/docs/architecture/adr-028-public-key-addresses.md): ```go // Module address without derivation keys diff --git a/sdk/next/guides/reference/protobuf-annotations.mdx b/sdk/next/guides/reference/protobuf-annotations.mdx index 156a7077e..4fe727a56 100644 --- a/sdk/next/guides/reference/protobuf-annotations.mdx +++ b/sdk/next/guides/reference/protobuf-annotations.mdx @@ -33,7 +33,7 @@ Signer specifies which field should be used to determine the signer of a message Read more about the signer field [here](/sdk/next/learn/concepts/encoding#message-signers). ```proto -// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/proto/cosmos/bank/v1beta1/tx.proto#L40 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/bank/v1beta1/tx.proto#L40 option (cosmos.msg.v1.signer) = "from_address"; ``` @@ -48,28 +48,28 @@ The scalar type defines a way for clients to understand how to construct protobu Example of account address string scalar: ```proto -// https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/proto/cosmos/bank/v1beta1/tx.proto#L46 +// https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/bank/v1beta1/tx.proto#L46 string from_address = 1 [(cosmos_proto.scalar) = "cosmos.AddressString"]; ``` Example of validator address string scalar: ```proto -// https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/proto/cosmos/distribution/v1beta1/query.proto#L108 +// https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/distribution/v1beta1/query.proto#L107 string validator_address = 1 [(cosmos_proto.scalar) = "cosmos.ValidatorAddressString"]; ``` Example of Dec scalar: ```proto -// https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/proto/cosmos/distribution/v1beta1/distribution.proto#L17 +// https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/distribution/v1beta1/distribution.proto#L17 string community_tax = 1 [(cosmos_proto.scalar) = "cosmos.Dec"]; ``` Example of Int scalar: ```proto -// https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/proto/cosmos/gov/v1/gov.proto#L127 +// https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/gov/v1/gov.proto#L127 string yes_count = 1 [(cosmos_proto.scalar) = "cosmos.Int"]; ``` @@ -110,7 +110,7 @@ The below annotations are used to provide information to the amino codec on how Name specifies the amino name that would show up for the user in order for them see which message they are signing. ```proto -// https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/proto/cosmos/bank/v1beta1/tx.proto#L41 +// https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/bank/v1beta1/tx.proto#L41 option (amino.name) = "cosmos-sdk/MsgSend"; ``` @@ -119,7 +119,7 @@ option (amino.name) = "cosmos-sdk/MsgSend"; Field name specifies the amino name that would show up for the user in order for them see which field they are signing. ```proto -// https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/proto/cosmos/distribution/v1beta1/distribution.proto#L165 +// https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/distribution/v1beta1/distribution.proto#L165 uint64 height = 3 [(amino.field_name) = "creation_height"]; ``` @@ -128,22 +128,22 @@ uint64 height = 3 [(amino.field_name) = "creation_height"]; Dont omitempty specifies that the field should not be omitted when encoding to amino. ```proto -// https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/proto/cosmos/bank/v1beta1/tx.proto#L48 +// https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/bank/v1beta1/tx.proto#L48 repeated cosmos.base.v1beta1.Coin amount = 3 [(amino.dont_omitempty) = true]; ``` ### Encoding -Encoding instructs the amino json marshaler how to encode certain fields that may differ from the standard encoding behavior. The most common example of this is how `repeated cosmos.base.v1beta1.Coin` is encoded when using the amino json encoding format. The `legacy_coins` option tells the json marshaler [how to encode a null slice](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/x/tx/signing/aminojson/json_marshal.go#L85) of `cosmos.base.v1beta1.Coin`. +Encoding instructs the amino json marshaler how to encode certain fields that may differ from the standard encoding behavior. The most common example of this is how `repeated cosmos.base.v1beta1.Coin` is encoded when using the amino json encoding format. The `legacy_coins` option tells the json marshaler [how to encode a null slice](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/x/tx/signing/aminojson/json_marshal.go#L85) of `cosmos.base.v1beta1.Coin`. ```proto -// https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/proto/cosmos/bank/v1beta1/genesis.proto#L23 +// https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/bank/v1beta1/genesis.proto#L23 (amino.encoding) = "legacy_coins", ``` ## Module Query Safe -The `cosmos.query.v1.module_query_safe` annotation ([source](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/proto/cosmos/query/v1/query.proto)) marks a query method as safe to call from within the state machine — for example from another module's keeper, via ADR-033 intermodule calls, or from CosmWasm contracts. +The `cosmos.query.v1.module_query_safe` annotation ([source](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/query/v1/query.proto)) marks a query method as safe to call from within the state machine — for example from another module's keeper, via ADR-033 intermodule calls, or from CosmWasm contracts. ```proto rpc Balance(QueryBalanceRequest) returns (QueryBalanceResponse) { diff --git a/sdk/next/guides/state/collections.mdx b/sdk/next/guides/state/collections.mdx index c1349f1cf..b53bc48e7 100644 --- a/sdk/next/guides/state/collections.mdx +++ b/sdk/next/guides/state/collections.mdx @@ -113,7 +113,7 @@ Since a module can have multiple collections, the following is expected: We don't want a collection to write over the state of the other collection so we pass it a prefix, which defines a storage partition owned by the collection. -If you already built modules, the prefix translates to the items you were creating in your `types/keys.go` file, example: [Link](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/x/feegrant/key.go#L16-L22) +If you already built modules, the prefix translates to the items you were creating in your `types/keys.go` file, example: [Link](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/x/feegrant/key.go#L16-L22) your old: diff --git a/sdk/next/guides/state/store.mdx b/sdk/next/guides/state/store.mdx index 093f4e498..fc042cb29 100644 --- a/sdk/next/guides/state/store.mdx +++ b/sdk/next/guides/state/store.mdx @@ -13,7 +13,7 @@ abstractions. ### `Store` -The bulk of the store interfaces are defined [here](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/store/types/store.go), +The bulk of the store interfaces are defined [here](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/store/types/store.go), where the base primitive interface, for which other interfaces build off of, is the `Store` type. The `Store` interface defines the ability to tell the type of the implementing store and the ability to cache wrap via the `CacheWrapper` interface. diff --git a/sdk/next/guides/testing/simulator.mdx b/sdk/next/guides/testing/simulator.mdx index b937310e1..0f30793ab 100644 --- a/sdk/next/guides/testing/simulator.mdx +++ b/sdk/next/guides/testing/simulator.mdx @@ -55,9 +55,9 @@ type HasProposalMsgs interface { } ``` -See the full source at [`types/module/simulation.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/types/module/simulation.go). +See the full source at [`types/module/simulation.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/types/module/simulation.go). -See an example implementation of these methods from `x/distribution` [here](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/x/distribution/module.go#L170-L194). +See an example implementation of these methods from `x/distribution` [here](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/x/distribution/module.go#L158-L182). ## SimsX @@ -80,7 +80,7 @@ type ( ) ``` -See the full source at [`testutil/simsx/runner.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/testutil/simsx/runner.go). +See the full source at [`testutil/simsx/runner.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/testutil/simsx/runner.go). `SimMsgFactoryFn` is the default factory for most cases. It does not create future operations but ensures successful message delivery: @@ -89,22 +89,22 @@ See the full source at [`testutil/simsx/runner.go`](https://github.com/cosmos/co type SimMsgFactoryFn[T sdk.Msg] func(ctx context.Context, testData *ChainDataSource, reporter SimulationReporter) (signer []SimAccount, msg T) ``` -See the full source at [`testutil/simsx/msg_factory.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/testutil/simsx/msg_factory.go). +See the full source at [`testutil/simsx/msg_factory.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/testutil/simsx/msg_factory.go). These methods allow constructing randomized messages and/or proposal messages. Note that modules should **not** implement both `HasWeightedOperationsX` and `HasWeightedOperationsXWithProposals`. -See the runner code [here](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/testutil/simsx/runner.go#L330-L339) for details +See the runner code [here](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/testutil/simsx/runner.go#L330-L339) for details If the module does **not** have message handlers or governance proposal handlers, these interface methods do **not** need to be implemented. ### Example Implementations -* `HasWeightedOperationsXWithProposals`: [x/gov](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/x/gov/module.go#L242-L261) -* `HasWeightedOperationsX`: [x/bank](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/x/bank/module.go#L201-L205) -* `HasProposalMsgsX`: [x/bank](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/x/bank/module.go#L196-L199) +* `HasWeightedOperationsXWithProposals`: [x/gov](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/x/gov/module.go#L221-L240) +* `HasWeightedOperationsX`: [x/bank](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/x/bank/module.go#L179-L183) +* `HasProposalMsgsX`: [x/bank](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/x/bank/module.go#L174-L177) ## Store decoders @@ -112,7 +112,7 @@ Registering the store decoders is required for the `AppImportExport` simulation. for the key-value pairs from the stores to be decoded to their corresponding types. In particular, it matches the key to a concrete type and then unmarshalls the value from the `KVPair` to the type provided. -Modules using [collections](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/collections/README.md) can use the `NewStoreDecoderFuncFromCollectionsSchema` function that builds the decoder for you: +Modules using [collections](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/collections/README.md) can use the `NewStoreDecoderFuncFromCollectionsSchema` function that builds the decoder for you: ```go // RegisterStoreDecoder registers a decoder for supply module's types @@ -121,17 +121,17 @@ func (am AppModule) RegisterStoreDecoder(sdr simtypes.StoreDecoderRegistry) { } ``` -See the full source at [`types/simulation/collections.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/types/simulation/collections.go) and the bank module example at [`x/bank/module.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/x/bank/module.go#L183-L186). +See the full source at [`types/simulation/collections.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/types/simulation/collections.go) and the bank module example at [`x/bank/module.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/x/bank/module.go#L161-L164). Modules not using collections must manually build the store decoder. -See the implementation [here](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/x/distribution/simulation/decoder.go) from the distribution module for an example. +See the implementation [here](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/x/distribution/simulation/decoder.go) from the distribution module for an example. ## Randomized genesis The simulator tests different scenarios and values for genesis parameters. App modules must implement a `GenerateGenesisState` method to generate the initial random `GenesisState` from a given seed. -See an example from `x/auth` [here](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/x/auth/module.go#L174-L177). +See an example from `x/auth` [here](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/x/auth/module.go#L171-L174). Once the module's genesis parameters are generated randomly (or with the key and values defined in a `params` file), they are marshaled to JSON format and added @@ -170,7 +170,7 @@ Note that the name passed in to `weights.Get` must match the name of the operati For example, if the module contains an operation `op_weight_msg_set_withdraw_address`, the name passed to `weights.Get` should be `msg_set_withdraw_address`. -See the `x/distribution` for an example of implementing message factories [here](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/x/distribution/simulation/msg_factory.go) +See the `x/distribution` for an example of implementing message factories [here](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/x/distribution/simulation/msg_factory.go) ## App Simulator manager @@ -208,7 +208,7 @@ func (app *SimApp) SimulationManager() *module.SimulationManager { } ``` -See the full simapp setup at [`simapp/app.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/simapp/app.go). +See the full simapp setup at [`simapp/app.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/simapp/app.go). ## Running Simulations @@ -230,7 +230,7 @@ func TestAppImportExport(t *testing.T) { These functions should be called in tests (i.e., `app_test.go`, `app_sim_test.go`, etc.). -See the full simapp test file at [`simapp/sim_test.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/simapp/sim_test.go). +See the full simapp test file at [`simapp/sim_test.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/simapp/sim_test.go). ### Simulation test types @@ -247,7 +247,7 @@ Simulations run in three modes: 1. **Fully random** -- initial state, module parameters, and simulation parameters are all pseudo-randomly generated. 2. **From a `genesis.json` file** -- initial state and module parameters are defined by the file. Useful for testing against a known state such as a live network export. -3. **From a `params.json` file** -- initial state is pseudo-randomly generated but module and simulation parameters are set manually. Available parameters are listed [here](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/x/simulation/client/cli/flags.go#L43-L70). +3. **From a `params.json` file** -- initial state is pseudo-randomly generated but module and simulation parameters are set manually. Available parameters are listed [here](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/x/simulation/client/cli/flags.go#L43-L70). These modes are not mutually exclusive. For example, you can combine a randomly generated genesis state (mode 1) with manually defined simulation params (mode 3). @@ -264,7 +264,7 @@ go test -mod=readonly github.com/cosmos/cosmos-sdk/simapp \ -v -timeout 24h ``` -The full list of available flags is defined [here](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/x/simulation/client/cli/flags.go#L43-L70). For Makefile examples, see the Cosmos SDK [`Makefile`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/Makefile#L280-L340). +The full list of available flags is defined [here](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/x/simulation/client/cli/flags.go#L43-L70). For Makefile examples, see the Cosmos SDK [`Makefile`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/Makefile#L280-L340). ### Debugging tips @@ -274,4 +274,4 @@ When encountering a simulation failure: * **Use `-Verbose` logs** for a fuller picture of all operations involved. * **Try a different `-Seed`**. If the same error reproduces sooner, you will spend less time on each run. * **Reduce `-NumBlocks`** to isolate what the app state looks like at the block before failure. -* **Add a [`Logger`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/x/staking/keeper/keeper.go#L78-L82)** to operations that are not being logged. +* **Add a [`Logger`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/x/staking/keeper/keeper.go#L82-L86)** to operations that are not being logged. diff --git a/sdk/next/guides/testing/telemetry.mdx b/sdk/next/guides/testing/telemetry.mdx index 9cbcd6497..f5a27695d 100644 --- a/sdk/next/guides/testing/telemetry.mdx +++ b/sdk/next/guides/testing/telemetry.mdx @@ -70,10 +70,10 @@ extensions: **Option A: Environment Variable (Recommended)** -Set `OTEL_EXPERIMENTAL_CONFIG_FILE` to your config path. This initializes the SDK before any meters/tracers are created, avoiding atomic load overhead. +Set `OTEL_CONFIG_FILE` to your config path. This initializes the SDK before any meters/tracers are created, avoiding atomic load overhead. ```bash -export OTEL_EXPERIMENTAL_CONFIG_FILE=/path/to/otel.yaml +export OTEL_CONFIG_FILE=/path/to/otel.yaml ``` **Option B: Node Config Directory** diff --git a/sdk/next/guides/tooling/autocli.mdx b/sdk/next/guides/tooling/autocli.mdx index 713a7cbe6..18de69e03 100644 --- a/sdk/next/guides/tooling/autocli.mdx +++ b/sdk/next/guides/tooling/autocli.mdx @@ -103,7 +103,7 @@ Users can however use the `--no-proposal` flag to disable the proposal creation By default, `autocli` generates a command for each method in your gRPC service. However, you can specify subcommands to group related commands together. To specify subcommands, use the `autocliv1.ServiceCommandDescriptor` struct. -For a real-world example, see the `gov` module's [`autocli.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/x/gov/autocli.go) in the Cosmos SDK. It demonstrates `ServiceCommandDescriptor` with `RpcCommandOptions`, `PositionalArgs`, `SubCommands`, `EnhanceCustomCommand`, and `GovProposal` all in one file. +For a real-world example, see the `gov` module's [`autocli.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/x/gov/autocli.go) in the Cosmos SDK. It demonstrates `ServiceCommandDescriptor` with `RpcCommandOptions`, `PositionalArgs`, `SubCommands`, `EnhanceCustomCommand`, and `GovProposal` all in one file. ### Positional Arguments @@ -111,7 +111,7 @@ By default `autocli` generates a flag for each field in your protobuf message. H To add positional arguments to a command, use the `autocliv1.PositionalArgDescriptor` struct, as seen in the example below. Specify the `ProtoField` parameter, which is the name of the protobuf field that should be used as the positional argument. In addition, if the parameter is a variable-length argument, you can specify the `Varargs` parameter as `true`. This can only be applied to the last positional parameter, and the `ProtoField` must be a repeated field. -For a real-world example, see the `auth` module's [`autocli.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/x/auth/autocli.go) in the Cosmos SDK. It shows positional args wired for every query method, with `address` as a positional argument on the `Account` method. +For a real-world example, see the `auth` module's [`autocli.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/x/auth/autocli.go) in the Cosmos SDK. It shows positional args wired for every query method, with `address` as a positional argument on the `Account` method. After wiring positional args, the command can be used as follows, instead of having to specify the `--address` flag: @@ -228,11 +228,11 @@ autoCliOpts.ModuleOptions[nodeCmds.Name()] = nodeCmds.AutoCLIOptions() `AutoCliOpts()` only picks up modules registered with the module manager — non-module commands always need to be added to `ModuleOptions` manually, as the example chain does with `nodeservice.NewNodeCommands()`. -For a more complete example of this pattern, see [`client/grpc/cmtservice/autocli.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/client/grpc/cmtservice/autocli.go) and [`client/grpc/node/autocli.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/client/grpc/node/autocli.go) in the Cosmos SDK. +For a more complete example of this pattern, see [`client/grpc/cmtservice/autocli.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/client/grpc/cmtservice/autocli.go) and [`client/grpc/node/autocli.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/client/grpc/node/autocli.go) in the Cosmos SDK. ## Root Command Setup -For AutoCLI-generated commands (and hand-written commands) to work correctly — signing transactions, querying the chain, reading configuration — the root command must set up the `client.Context` and `server.Context` in a `PersistentPreRunE` function. This runs before every subcommand and makes both contexts available to all child commands. See [`simapp/simd/cmd/root.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/simapp/simd/cmd/root.go#L50-L93) for a complete example. +For AutoCLI-generated commands (and hand-written commands) to work correctly — signing transactions, querying the chain, reading configuration — the root command must set up the `client.Context` and `server.Context` in a `PersistentPreRunE` function. This runs before every subcommand and makes both contexts available to all child commands. See [`simapp/simd/cmd/root.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/simapp/simd/cmd/root.go#L47-L70) for a complete example. The two key calls inside `PersistentPreRun` are: diff --git a/sdk/next/guides/upgrades/upgrade.mdx b/sdk/next/guides/upgrades/upgrade.mdx index f893d1804..96cc3b270 100644 --- a/sdk/next/guides/upgrades/upgrade.mdx +++ b/sdk/next/guides/upgrades/upgrade.mdx @@ -104,7 +104,7 @@ func (m Migrator) Migrate1to2(ctx sdk.Context) error { } ``` -To see example code of changes that were implemented in a migration of balance keys, check out [migrateBalanceKeys](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/x/bank/migrations/v2/store.go#L55-L76). For context, this code introduced migrations of the bank store that updated addresses to be prefixed by their length in bytes as outlined in [ADR-028](/sdk/next/reference/architecture/adr-028-public-key-addresses). +To see example code of changes that were implemented in a migration of balance keys, check out [migrateBalanceKeys](https://github.com/cosmos/cosmos-sdk/blob/v0.54.0/x/bank/migrations/v2/store.go#L55-L76). For context, this code introduced migrations of the bank store that updated addresses to be prefixed by their length in bytes as outlined in [ADR-028](/sdk/next/reference/architecture/adr-028-public-key-addresses). ## Running Migrations in the App diff --git a/sdk/next/keys/create-ml-dsa-account.mdx b/sdk/next/keys/create-ml-dsa-account.mdx new file mode 100644 index 000000000..2fc61f3a4 --- /dev/null +++ b/sdk/next/keys/create-ml-dsa-account.mdx @@ -0,0 +1,87 @@ +--- +noindex: true +title: "Create an ML-DSA account" +description: "Create a post-quantum user account with the keyring and move funds into it; there is no in-place migration for accounts." +--- + +This guide creates a user account backed by `ml_dsa_65`, the post-quantum signature algorithm, and moves funds into it. Accounts do not rotate. An existing account keeps its key for life, so moving to post-quantum means a new account and a transfer. For what ML-DSA is and when account migration matters, see [Post-quantum keys](/sdk/next/keys/post-quantum-keys). + +Commands use `simd`. Substitute your chain's binary and adjust key names and denoms. + +{/* +Manual verification setup (hidden, macOS). Reproduces a running chain with a funded source account, using simd/simapp. +Build: from the cosmos-sdk repo run `make build`, then put ./build/simd on PATH as `simd`. + +simd init mynode --chain-id my-chain-1 --home ~/.node +simd config set client chain-id my-chain-1 --home ~/.node +simd config set client keyring-backend test --home ~/.node +simd keys add old-account --keyring-backend test --home ~/.node +simd keys add validator --keyring-backend test --home ~/.node +simd genesis add-genesis-account old-account 100000000000stake --keyring-backend test --home ~/.node +simd genesis add-genesis-account validator 100000000000stake --keyring-backend test --home ~/.node +simd genesis gentx validator 1000000000stake --chain-id my-chain-1 --keyring-backend test --home ~/.node +simd genesis collect-gentxs --home ~/.node +sed -i '' 's/timeout_commit = ".*"/timeout_commit = "1s"/' ~/.node/config/config.toml +sed -i '' 's/minimum-gas-prices = ".*"/minimum-gas-prices = "0stake"/' ~/.node/config/app.toml +simd start --home ~/.node +# then follow the guide (use --home ~/.node on each command). The bank sends need --chain-id/--keyring-backend +# from client config, and the first send FROM pq-account needs --gas auto (already shown in step 3). +*/} + +## Know the limits first + +- CLI only. Wallet support for ML-DSA accounts is minimal; expect to manage the account with the chain CLI. +- No hardware wallets. Ledger devices sign with `secp256k1` only. Recovery works from the mnemonic alone. +- No EVM. EVM transactions require `eth_secp256k1` account keys, so ML-DSA accounts do not work with the EVM. See the EVM section of [Post-quantum keys](/sdk/next/keys/post-quantum-keys). + +## Prerequisites + +- The chain runs Cosmos SDK 0.55 or later, on every node. +- A running chain to send transactions to, with its CLI binary on your PATH and a funded account in the keyring. To stand up a local chain first, see [Run a node](/sdk/next/node/run-node). + +Enabling ML-DSA accounts needs no chain configuration beyond the SDK version. Upgrading to 0.55 is sufficient, but every validator and node must run it: a binary built with an older SDK cannot process ML-DSA signatures. + +## 1. Create the account + +Generate the account with the keyring and select the ML-DSA algorithm: + +```shell +simd keys add pq-account --key-type ml_dsa_65 --home ~/.node --keyring-backend test +``` + +Use the same `--home` and `--keyring-backend` on every command in this guide. Without them the key lands in a different keyring from your funded account, and the transfer in step 2 fails to find it. + +The output shows the new address and a mnemonic. Store the mnemonic securely. No hardware wallet can hold this key, so the mnemonic is the only recovery path. To recover the account later: + +```shell +simd keys add pq-account --recover --key-type ml_dsa_65 --home ~/.node --keyring-backend test +``` + +Do not create an ML-DSA account from a mnemonic already used for a `secp256k1` account. ML-DSA key generation derives its seed from the same secp256k1 BIP32 path, so at that path the secp256k1 private key and the ML-DSA seed are the same secret. Reusing the mnemonic links the two keys: whoever obtains that secret controls both. Generate each ML-DSA account from a fresh mnemonic. + +## 2. Move funds in + +There is no in-place migration for accounts by design. Send funds from the old account with an ordinary transfer: + +```shell +simd tx bank send old-account "$(simd keys show pq-account -a --home ~/.node --keyring-backend test)" 1000000stake --from old-account --home ~/.node --keyring-backend test +``` + +Add `--chain-id` if your client config does not supply it, and `--fees` (or `--gas-prices`) to meet the chain's minimum gas price. + +## 3. Verify the account signs + +Prove the new key works by sending from it: + +```shell +simd tx bank send pq-account "$(simd keys show old-account -a --home ~/.node --keyring-backend test)" 1stake --from pq-account --gas auto --gas-adjustment 1.5 --home ~/.node --keyring-backend test +``` + +You may need to raise the gas limit for this transfer. The account's first transaction writes its public key to state, and an ML-DSA public key is large enough that a default limit of 200000 runs out of gas. `--gas auto` sizes the limit to fit. Later transactions from the account are smaller and fit within the default. + +A successful send from `pq-account` means the chain accepted an ML-DSA signature for the account. Drain and retire the old account afterwards. + +## Next steps + +- Understand what post-quantum protection the account now has. See [Post-quantum keys](/sdk/next/keys/post-quantum-keys). +- Validators migrate differently, by key rotation. See [Migrate a validator to ML-DSA](/sdk/next/keys/migrate-validator-ml-dsa). diff --git a/sdk/next/keys/enable-ml-dsa-keys.mdx b/sdk/next/keys/enable-ml-dsa-keys.mdx new file mode 100644 index 000000000..bc5cfee28 --- /dev/null +++ b/sdk/next/keys/enable-ml-dsa-keys.mdx @@ -0,0 +1,133 @@ +--- +noindex: true +title: "Enable ML-DSA keys" +description: "Add ml_dsa_65 to a chain's accepted consensus key types, through genesis on a new chain or governance on a live one." +--- + +Validator key types are a consensus parameter, so allowing post-quantum validator keys is a chain-level change. This guide adds `ml_dsa_65` to the accepted consensus key types: on a new chain through genesis, or on a live chain through a governance proposal, with no coordinated restart. + +For background on ML-DSA and the key types, see [Post-quantum keys](/sdk/next/keys/post-quantum-keys). + +Before any validator rotates to an ML-DSA key, every counterparty chain that verifies this chain over IBC must run CometBFT v0.40 or later. See [IBC considerations](/sdk/next/keys/post-quantum-keys#ibc-considerations). + +## Prerequisites + +- The chain runs Cosmos SDK 0.55 and CometBFT 0.40 or later. See the [release notes](/sdk/next/upgrade/v0.55-release). +- [jq](https://jqlang.org/) and [curl](https://curl.se/). The commands derive the proposal payload with jq and read the validator set with curl. +- The chain's CLI binary on your PATH, with RPC access to a node. To stand up a local chain, see [Run a node](/sdk/next/node/run-node). +- For a live chain: the ability to pass a governance proposal, and an account funded for the proposal deposit and gas. + +Commands use `simd`; substitute your chain's binary, and adjust key names and denoms to your own. + +{/* +Manual verification setup (hidden, macOS). Reproduces a running chain with governance for the live-chain path, using simd/simapp. +Build: from the cosmos-sdk repo run `make build`, then put ./build/simd on PATH as `simd`. + +simd init mynode --chain-id my-chain-1 --home ~/.node +simd config set client chain-id my-chain-1 --home ~/.node +simd config set client keyring-backend test --home ~/.node +simd keys add val --keyring-backend test --home ~/.node +simd genesis add-genesis-account val 100000000000stake --keyring-backend test --home ~/.node +simd genesis gentx val 50000000000stake --chain-id my-chain-1 --keyring-backend test --home ~/.node +simd genesis collect-gentxs --home ~/.node +# short voting period so the proposal can pass quickly during testing +jq '.app_state.gov.params.voting_period="15s" | .app_state.gov.params.expedited_voting_period="10s"' ~/.node/config/genesis.json > /tmp/g && mv /tmp/g ~/.node/config/genesis.json +sed -i '' 's/timeout_commit = ".*"/timeout_commit = "1s"/' ~/.node/config/config.toml +sed -i '' 's/minimum-gas-prices = ".*"/minimum-gas-prices = "0stake"/' ~/.node/config/app.toml +simd start --home ~/.node +# then follow the guide (use --home ~/.node). Submit the proposal and vote quickly, within the 15s window. +# All commands run from one directory so params.json is shared. +*/} + +## Check the current state + +Consensus params list the allowed key types under `validator.pub_key_types`. Query them: + +```shell +simd query consensus params +``` + +The default is `ed25519` only. To see which types the validator set is currently running, list the validators and read each one's pubkey type: + +```shell +curl -s localhost:26657/validators | jq -r '.result.validators[].pub_key.type' +``` + +The output shows registered type names, one per validator: `tendermint/PubKeyEd25519` for ed25519 keys, `cometbft/PubKeyMlDsa65` for ML-DSA keys. + +## New chain: set the types in genesis + +Add `ml_dsa_65` to `consensus.params.validator.pub_key_types` in `genesis.json` before launch: + +```json +"validator": { + "pub_key_types": ["ed25519", "ml_dsa_65"] +} +``` + +Keep `ed25519` in the list unless every genesis validator starts on an ML-DSA key. Validators whose key type is not in the list cannot join the set. A genesis validator starts on an ML-DSA key by initializing its node with `simd init --consensus-key-algo ml_dsa_65`, and a local ML-DSA testnet comes from the same flag on `simd testnet init-files` or `simd testnet start`. + +`simd init --consensus-key-algo` replaces the genesis `pub_key_types` list with only the chosen algorithm. A chain that should accept both types must re-add `ed25519` to the list after init, or ed25519 validators cannot join. + +## Live chain: expand the types through governance + +The change is a parameter update executed by governance. It takes effect when the proposal passes, with no node restarts and no coordinated upgrade. + +All steps read and write `params.json` in the current directory, so run them from one place. + +1. Build the proposal params from the live chain state. The update message replaces the entire params object, so it must carry every current value. The following command derives the params from a query, adds `ml_dsa_65` to `pub_key_types`, and converts the evidence duration to the format the proposal parser accepts: + +```shell +simd query consensus params -o json \ + | jq '.params + | {block, evidence, validator, abci} + | .validator.pub_key_types += ["ml_dsa_65"] + | .evidence.max_age_duration |= ( + capture("((?[0-9]+)h)?((?[0-9]+)m)?((?[0-9]+)s)?") + | ((((.h // "0") | tonumber) * 3600 + + ((.m // "0") | tonumber) * 60 + + ((.s // "0") | tonumber)) | tostring) + "s" + )' \ + > params.json +``` + +2. You can then submit the file's contents as a governance proposal. The command takes the four param groups as separate arguments, sliced from the same file. Make sure to update the following command to include your key and correct deposit amount/denomination. Add `--home` pointing at your node's home, since the transaction commands on this page read the keyring from it. Also add `--chain-id` and `--keyring-backend` if your client config does not supply them, and `--fees` (or `--gas-prices`) to meet the chain's minimum gas price: + +```shell +simd tx consensus update-params-proposal "$(jq -c .block params.json)" "$(jq -c .evidence params.json)" "$(jq -c .validator params.json)" "$(jq -c .abci params.json)" --title "Allow ML-DSA validator keys" --summary "Add ml_dsa_65 to consensus pub_key_types" --deposit 10000000stake --from mykey -y +``` + +3. Vote as with any governance proposal, using the proposal ID from `simd query gov proposals`. + +```shell +simd tx gov vote yes --from mykey +``` + +When it passes, the new list is live. + + +The update replaces the entire params object, so every group must be present. If `params.json` is missing `block`, `evidence`, or `validator`, the `jq -c` substitution above emits `null` and the CLI rejects the command before it is submitted, with `invalid argument "null": proto: syntax error`. Always start from the queried current params and change only the key type list. + + +## Verify + +Query the params again and confirm the list includes `ml_dsa_65`: + +```shell +simd query consensus params +``` + +New validators can now join with ML-DSA keys. Existing validators can migrate by rotation. See [Key rotation](/sdk/next/keys/key-rotation). + +## Remove a key type + +To remove a key type, update the `params.json` file to remove the key type from the `pub_key_types` array. Then, submit the updated `params.json` file as a governance proposal. + + +Do not remove a key type while validators still use it. Existing validators are not re-checked when a type leaves the list, but every later voting-power update for such a validator fails validation, and a failed validator update halts the chain. Routine activity is enough to trigger it: any delegation that changes the validator's voting power emits one of these updates. + + +## Next steps + +- Understand the rotation mechanics before touching a production validator. See [Key rotation](/sdk/next/keys/key-rotation). +- Migrate a validator to the new key type. See [Migrate a validator to ML-DSA](/sdk/next/keys/migrate-validator-ml-dsa). diff --git a/sdk/next/keys/key-rotation.mdx b/sdk/next/keys/key-rotation.mdx new file mode 100644 index 000000000..610183cf0 --- /dev/null +++ b/sdk/next/keys/key-rotation.mdx @@ -0,0 +1,78 @@ +--- +noindex: true +title: "Key rotation" +description: "How consensus key rotation works: which keys rotate, the two-height delay, the fee, the rotation limit, and how slashing follows a rotated key." +--- + +Key rotation replaces a validator's consensus key in place: no unbonding, no downtime, and no change to the validator's identity, power, or delegations. Rotation is a single operator-signed message on the [`x/staking`](/sdk/next/keys/rotate-validator-key) and [`enterprise/poa`](/sdk/next/keys/rotate-validator-key-poa) modules, and realizes a design first proposed in 2019 as [ADR-016](/sdk/next/reference/architecture/adr-016-validator-consensus-key-rotation). + +## Which keys rotate + +A validator runs on three keys, each with a different owner and job: + +1. **Operator key**: an ordinary account key that the operator generates and custodies, often on a hardware wallet or behind a multisig. It owns the validator: it stakes, sets commission, edits the validator's description, withdraws rewards, votes, and signs the rotation message itself. Like any account key, it has no in-place replacement; retiring one means creating a new account and moving funds. +2. **Consensus key**: generated by the stack when a node initializes, stored in `priv_validator_key.json` or held by a remote signer. It is a raw keypair with no mnemonic behind it; the operator custodies the file, not a seed phrase. It signs the validator's vote on every block, and a block proposal whenever the validator's turn comes to propose. Evidence of misbehavior identifies the validator through this key's consensus address. This is the only key a rotation touches. +3. **Node key**: the node's peer-to-peer identity. It exists so peers can address and authenticate each other: its public key hashes to the node ID used in peer addresses, and it secures the connection handshake between nodes. It carries no on-chain state and regenerates freely. + +On chain, a validator is the pairing of an operator address with a consensus key. Rotation replaces the consensus half of that pairing and leaves everything the operator key controls alone. For the full survey of keys and algorithms, see [Post-quantum keys](/sdk/next/keys/post-quantum-keys). + +## How a rotation works + +The operator submits `MsgRotateConsPubKey`, carrying the new consensus public key. Everything that defines the validator stays put: the operator address, voting power, delegations, and commission. Only the consensus key and its index change. + +The new key does not take effect immediately. The chain emits the power hand-off to CometBFT right away, setting the old key to zero and giving the new key the validator's full power, and CometBFT's validator update rule makes it effective two heights later, at the same height the chain swaps its stored key. Zero downtime rides on this: the operator runs a second node with the new key alongside the old one. Until the rotation takes effect, the second node follows the chain as a non-signing full node, because a CometBFT node whose key is outside the validator set produces no votes. The moment the new key enters the set, the second node starts signing, the old key holds no power, and the operator retires the old node. The step-by-step procedure is in [Rotate a consensus key, Staking](/sdk/next/keys/rotate-validator-key). + +## Safety rails + +Three rules bound what a rotation can do: + +1. A rotation burns a flat fee, set by the `key_rotation_fee` staking param, to make rotation spam expensive. +2. A validator can rotate once per unbonding period. Until the unbonding period ends, the previous key remains accountable, so the chain rejects a second rotation inside the window. +3. A rotation cannot be undone. No cancel message exists, and the once-per-unbonding-period limit blocks an immediate rotation back, so an applied rotation stands until the window expires. + +## Security implications + +Read this section carefully. Key rotation introduces security and performance tradeoffs that chains must be aware of before rotating keys. + +### Slashing window length + +Slashing follows a validator's history, not the key. Evidence of a double sign under the old key still slashes the validator after rotation. When a rotation lands, the chain records the old consensus address. It keeps that address tied to the validator. It tracks the address for as long as evidence against it can still be admitted. It computes this window at rotation time from the evidence params `MaxAgeNumBlocks` and `MaxAgeDuration`. Once both elapse, the chain stops tracking the address. The window can be months, depending on the chain's evidence settings. Rotating away from a key does not let a validator escape slashing within that window. + +Downtime slashing carries over too. On a rotation, the validator's missed-block record and jailed status move to the new consensus key. A rotation does not reset them. + +The chain computes a rotation tracking window once at the time of rotation. It never updates this window. If governance extends `MaxAgeNumBlocks` or `MaxAgeDuration` after a rotation, the original window for the rotation remains in effect. In this scenario, the old consensus address will stop being tracked before the current evidence window closes. During that gap, a double sign under the old key cannot be slashed. This is a risk that chains should be aware of. Do not extend the evidence params while rotations are in flight without accounting for it. + +### Increased IBC light client updates + +Frequent rotations may raise the cost of keeping a light client current. Each rotation changes the validator set. Tendermint light clients advance using the overlap between successive validator sets. Heavy rotation churn shrinks that overlap. With less overlap, a relayer must submit more update-client messages to advance the client across the same span. The once-per-unbonding-period limit exists partly to bound this cost. + +### Proposer priority reset + +Rotation resets the validator's proposer priority. CometBFT orders validators by a proposer priority value. That value decides when a validator's turn to propose comes up. A rotation sends the validator to the back of that order. This holds even if the validator was next in line to propose. + +### Slower signature verification + +A validator set with mixed consensus key types verifies signatures more slowly. CometBFT batch-verifies signatures when every validator uses the same key type. Batch verification is faster than checking each signature on its own. One validator on a different key type breaks batching. Verification then falls back to one signature at a time, which can slow block times. This applies whenever the set holds more than one key type, not only during a rotation. + +## Exports and restarts + +In-progress rotations survive a genesis export. A chain exported mid-rotation carries the pending rotation and its remaining history window into the new genesis, so restarting a chain does not lose slashing accountability or drop a queued key change. + +## Staking and PoA chains + +Rotation ships in both validator models. On staked chains, it is the `x/staking` implementation described above; for the procedure, see [Rotate a consensus key, Staking](/sdk/next/keys/rotate-validator-key). Chains running `enterprise/poa` get the same rotation with two differences. The admin can rotate any validator's key, not only the operator. And because PoA has no slashing or evidence handling, the safety rails above do not apply: no fee, no rate limit, no rotation history, and the state swap happens in the block the transaction lands. For the procedure, see [Rotate a consensus key, PoA](/sdk/next/keys/rotate-validator-key-poa). + + +Vote extensions are not supported on PoA chains. PoA chains running custom logic that resolves a validator from a `LastCommit` address or that uses vote extensions may experience unexpected behavior during the two-height delay after a rotation. Read on for more details. + + +The PoS and PoA models differ in how they handle CometBFT's two-height delay. After a key rotation, CometBFT keeps signing `LastCommit` with the old consensus address for two heights. Staking waits out those heights before swapping its own state and keeps a historical address mapping, so the old address still resolves to its validator. PoA swaps immediately and keeps no mapping, so during those two heights it cannot resolve the old address that `LastCommit` still carries. A stock PoA chain never notices, because it runs neither x/distribution nor x/slashing (the modules that generally read those addresses). However, custom logic that resolves a validator from a `LastCommit` address on a PoA chain will not find a rotating validator for those two heights, which can lead to unexpected behavior. + +For this reason, vote extensions are not supported on PoA chains. The chain verifies a vote-extension signature by the `LastCommit` address. If a validator rotates its key during the two-height delay, the chain will reject the vote extension because the `LastCommit` address will not resolve to the validator. The standard `ValidateVoteExtensions` helper returns an error on the first commit vote whose `LastCommit` address it cannot resolve, before it tallies any voting power. One rotating validator that signed the previous block is therefore enough to have the whole extended commit rejected, whatever its share of voting power. + +## Next steps + +- Perform a rotation on a staked chain. See [Rotate a consensus key, Staking](/sdk/next/keys/rotate-validator-key). +- Perform a rotation on a PoA chain. See [Rotate a consensus key, PoA](/sdk/next/keys/rotate-validator-key-poa). +- Understand the key types. See [Post-quantum keys](/sdk/next/keys/post-quantum-keys). +- Look up the message, parameters, and state layout. See the [x/staking module reference](/sdk/next/modules/staking/README#msgrotateconspubkey). diff --git a/sdk/next/keys/migrate-validator-ml-dsa.mdx b/sdk/next/keys/migrate-validator-ml-dsa.mdx new file mode 100644 index 000000000..7ce67bdc3 --- /dev/null +++ b/sdk/next/keys/migrate-validator-ml-dsa.mdx @@ -0,0 +1,86 @@ +--- +noindex: true +title: "Migrate a validator to ML-DSA" +description: "Move a validator's consensus key to the post-quantum ml_dsa_65 algorithm through an ordinary key rotation." +--- + +Migrating a validator to post-quantum signing is an ordinary key rotation with an ML-DSA target key. This guide adds the ML-DSA-specific steps around the standard procedure in [Rotate a consensus key, Staking](/sdk/next/keys/rotate-validator-key). For more information on ML-DSA, see [Post-quantum keys](/sdk/next/keys/post-quantum-keys). + +Key rotation can introduce security implications for your chain. Read the [Key rotation](/sdk/next/keys/key-rotation) overview in its entirety before proceeding. + +Before rotating any validator to ML-DSA, confirm every counterparty chain that verifies this chain over IBC runs CometBFT v0.40 or later. An older `07-tendermint` light client cannot verify ML-DSA consensus signatures. See [IBC considerations](/sdk/next/keys/post-quantum-keys#ibc-considerations) for more information. + +## Prerequisites + +- All prerequisites of the rotation procedure: [jq](https://jqlang.org/) and [curl](https://curl.se/), no rotation in the current unbonding period, and fee funds on the operator account. See [Rotate a consensus key, Staking](/sdk/next/keys/rotate-validator-key). +- The chain's binary; the examples use `simd`. To build it and run a node, see [Run a node](/sdk/next/node/run-node). + +## 1. Confirm the chain allows ML-DSA + +Validator key types are a consensus parameter. Check that `ml_dsa_65` is in the list: + +```shell +simd query consensus params +``` + +If `validator.pub_key_types` does not include `ml_dsa_65`, the rotation is rejected. To add the type, see [Enable ML-DSA keys](/sdk/next/keys/enable-ml-dsa-keys). + +## 2. Rotate to an ML-DSA key + +### On a staking chain + +Follow [Rotate a consensus key, Staking](/sdk/next/keys/rotate-validator-key), replacing the `simd init` command in its step 1 with one that adds `--consensus-key-algo ml_dsa_65` to create an ML-DSA key: + +```shell +simd init rotation-node --chain-id my-chain-1 --consensus-key-algo ml_dsa_65 --home ~/.rotation-node +``` + +Everything else runs as written. The guide's rotation command derives the public key with `simd comet show-validator` on the second node's home, which now prints the ML-DSA key, so the rotation message carries it automatically. + +### On a PoA chain + +Follow [Rotate a consensus key, PoA](/sdk/next/keys/rotate-validator-key-poa) with two changes. Replace the `simd init` command in its step 1 with: + +```shell +simd init poa-newkey --chain-id my-chain-1 --consensus-key-algo ml_dsa_65 --home ~/.poa-newkey +``` + +And pass `ml_dsa_65` instead of `ed25519` as the key type when submitting. The ML-DSA public key may be large enough that the default gas limit runs out, so add `--gas auto`: + +```shell +simd tx poa rotate-cons-pub-key "$(simd comet show-validator --home ~/.poa-newkey | jq -r .key)" ml_dsa_65 --operator-address "$(simd keys show val -a --home ~/.node)" --from val --home ~/.node --gas auto --gas-adjustment 1.5 +``` + +The cutover timing is unchanged: keep the node on the old key until the validator set switches, then swap the key file in place, exactly as the guide's [steps 3 and 4](/sdk/next/keys/rotate-validator-key-poa#3-wait-for-the-validator-set-to-switch) describe. + +### On a remote signer + +If the validator's consensus key lives in Cosmos-KMS rather than a local file, the second node gets its own signer and the public key derivation differs. See [Rotate a consensus key held in Cosmos-KMS](/sdk/next/kms/rotate-key-remote-signer). + +{/* +CURRENT VERIFICATION: blind execution audit 2026-07-28 at shipping refs, cosmos-sdk v0.55.0 (64fd208a11), cometbft v0.40.0, PoA via enterprise/poa/simapp, macOS 26.5.2 arm64. Verdict PASS: both the staking and PoA ML-DSA rotations execute as written. Staking rotation returned code 0 at height 33 and /validators reported cometbft/PubKeyMlDsa65 with the chain still producing. Confirmed that --gas auto is required, since an ML-DSA pubkey exceeds the 200000 default (observed gasUsed 217674 to 233565). Full run: ~/Documents/tests/keys-docs-audit-findings.md. +Earlier run, retained for provenance: verified 2026-07-17 with simd from cosmos-sdk pr-26604 @ 46a177139a. #26604 has since merged and is in v0.55.0, so the "re-verify at merge" instruction is discharged. +*/} +{/* PoA ML-DSA rotation run-verified 2026-07-28 on cosmos-sdk v0.55.0 (includes #26614, merged as d6a3c6e27a): the command below executes as written on a PoA simapp binary, the module reports /cosmos.crypto.mldsa65.PubKey, and the chain resumes producing on the ML-DSA consensus key. Needs --gas auto (an ML-DSA pubkey exceeds the 200000 default). */} + +## 3. Verify + +Check the key type in the validator set: + +```shell +curl -s localhost:26657/validators | jq -r '.result.validators[].pub_key.type' +``` + +A migrated validator reports `cometbft/PubKeyMlDsa65` instead of `tendermint/PubKeyEd25519`. The chain's consensus is post-quantum secure once validators holding at least two thirds of voting power report a post-quantum type. For more information, see [Post-quantum keys](/sdk/next/keys/post-quantum-keys). + +## What can go wrong + +- The rotation is rejected for an unsupported key type: the chain does not list `ml_dsa_65` yet. See [Enable ML-DSA keys](/sdk/next/keys/enable-ml-dsa-keys). +- Anything else follows the standard rotation failure modes. See [Rotate a consensus key, Staking](/sdk/next/keys/rotate-validator-key). + +The consensus key this rotation installs is a raw keypair with no mnemonic behind it. Custody the key file, not a seed phrase. + +## Next steps + +- Check which key types the chain allows and which the validator set is running. See [Enable ML-DSA keys](/sdk/next/keys/enable-ml-dsa-keys). +- Understand the storage and bandwidth costs the chain takes on as the set migrates. See [Post-quantum keys](/sdk/next/keys/post-quantum-keys). diff --git a/sdk/next/keys/post-quantum-keys.mdx b/sdk/next/keys/post-quantum-keys.mdx new file mode 100644 index 000000000..59e617f0e --- /dev/null +++ b/sdk/next/keys/post-quantum-keys.mdx @@ -0,0 +1,87 @@ +--- +noindex: true +title: "Post-quantum keys" +description: "The keys and signature algorithms of a Cosmos chain, what post-quantum security means, and what adopting ML-DSA costs." +--- + +A Cosmos chain uses various keys backed by a set of signature algorithms, including ML-DSA, a native post-quantum option for consensus keys and user accounts. This page surveys the keys and algorithms and explains what post-quantum security means and how it applies. + +A post-quantum key signs with an algorithm that stays secure against an attacker equipped with a quantum computer. + +## Keys and algorithms + +Keys are the foundation of a chain's security: funds, consensus votes, and governance are only as safe as the keys that sign for them. Every signature on a Cosmos chain comes from one of four keys, each held by a different party and signing different things: + +| Key | Held by | Signs | Algorithms | +| --- | --- | --- | --- | +| User account key | Anyone with an account | Transactions | | +| Validator operator key | The validator's operator | Staking transactions |
  • Same as the user account key.
| +| Validator consensus key | The validator node | Votes and proposals | | +| Node key | Every node | Peer-to-peer identity | | + +Consensus params decide which algorithms consensus keys may use on a given chain, while users pick an account algorithm each time they create a key. To check what a chain currently allows and which algorithms its validator set is running, see [Enable ML-DSA keys](/sdk/next/keys/enable-ml-dsa-keys#check-the-current-state). For information on validator key rotation, see [Key rotation](/sdk/next/keys/key-rotation). + +## What post-quantum means + +Every algorithm in the stack except `ml_dsa_65` is based on an elliptic curve. A sufficiently powerful quantum computer running [Shor's algorithm](https://en.wikipedia.org/wiki/Shor%27s_algorithm) breaks elliptic curve cryptography outright: no key size makes a curve safe. Account keys, operator keys, and consensus keys therefore share the same long-term exposure, and when practical quantum hardware arrives, curve-based signatures stop being trustworthy. To counter this, the `ml_dsa_65` key algorithm is introduced. + +## How ML-DSA works + +The Module-Lattice-Based Digital Signature Algorithm (ML-DSA) is NIST's lattice-based signature standard, published as [FIPS 204](https://csrc.nist.gov/pubs/fips/204/final) in 2024 and the finalized form of CRYSTALS-Dilithium. Instead of deriving security from elliptic curves, it builds keys and signatures on [lattice problems](https://en.wikipedia.org/wiki/Lattice-based_cryptography), a class of mathematics with no known quantum attack. + +The Cosmos stack uses the middle FIPS 204 parameter set, ML-DSA-65 (NIST security category 3), as the algorithm `ml_dsa_65` to implement post-quantum security. + +With the addition of this key algorithm, nothing about the signing workflow changes. The keyring generates and recovers an ML-DSA account like any other, and consensus treats an ML-DSA consensus key like any other key type. What differs is the math underneath and the size of the keys and signatures it produces. + +## Is hashing post-quantum secure? + +Yes, the SHA-256 hash function that underpins the Cosmos SDK and CometBFT (block transaction hashes, merkle trees over application state, etc.) are considered post-quantum secure. Unlike RSA and elliptic-curve cryptography, which are broken by Shor's algorithm, the best known quantum attack against generic hash functions is [Grover's algorithm](https://en.wikipedia.org/wiki/Grover%27s_algorithm). Grover's algorithm provides only a quadratic speedup, reducing SHA-256's preimage resistance from 256 bits to about 128 bits, which is still considered secure. Collision resistance, the property that matters for merkle trees and transaction hashes, was already about 128 bits classically and is essentially unaffected. + +## Who can adopt ML-DSA? + +Only account keys and consensus keys can use `ml_dsa_65`. The node key stays `ed25519` and merely identifies a node to its peers, and module accounts and smart contract accounts hold funds without any key at all, so none of them has anything to migrate. + +Adoption differs by role. A user can generate a new ML-DSA account and move funds into it at any time, with no chain-level permission required; there is no in-place migration for accounts. A validator migrates its consensus key in place through key rotation, which does require the chain to allow `ml_dsa_65` in consensus params first. + +## When is a chain considered post-quantum? + +Consensus security follows voting power. A chain's consensus becomes post-quantum secure once at least two thirds of voting power signs with post-quantum consensus keys, because two thirds is the threshold an attacker must forge to break finality. Account security is individual: each account is exactly as secure as its own key. + +## The cost of post-quantum keys + +Post-quantum security trades larger keys and signatures for quantum resistance. Signatures dominate the added cost because every block commit carries one per validator, so the totals below scale with the validator set. + +A single ML-DSA signature is about 3,300 bytes, over 50 times the size of an ed25519 signature. Every block stores one per validator, which drives the block-data growth shown below. + +The example below assumes 100 validators and six-second blocks. + +| Measure | `ed25519` | `ml_dsa_65` | +| --- | --- | --- | +| Public key | 32 B | 1,952 B | +| Signature | 64 B | 3,309 B | +| Signature data per block | ~6 KB | ~331 KB | +| Total block size, with ~4 KB fixed overhead | ~10 KB | ~335 KB | +| Total block data per year | ~56 GB | ~1.8 TB | + +Signing and verification are slightly slower than with `ed25519`. This is unlikely to affect most chains. + +## IBC considerations + +A chain migrating to ML-DSA consensus keys must be aware that this change affects IBC verification. + +Any counterparty chain that verifies an ML-DSA-enabled chain with an `07-tendermint` light client must be upgraded to CometBFT v0.40 or later. An older client fails as soon as the first ML-DSA validator joins the set, so every counterparty must upgrade to v0.40 before any validator can rotate to ML-DSA. + +ML-DSA signatures also enlarge block headers, which enlarges the IBC client updates that carry them. CometBFT v0.40 raises its signature-size limits to accommodate the larger signatures. + +## EVM chains + +Validators on EVM chains can run ML-DSA consensus keys, as on any other chain. User accounts cannot: the EVM requires `eth_secp256k1` account keys, and those cannot move to a post-quantum scheme in place. + +Ethereum's path to post-quantum accounts runs through account abstraction instead. [EIP-7702](https://eips.ethereum.org/EIPS/eip-7702) already lets an account delegate to contract code, so a contract can verify a post-quantum signature inside the VM. [EIP-8051](https://eips.ethereum.org/EIPS/eip-8051) proposes a native precompile for ML-DSA verification, and [EIP-8141](https://eips.ethereum.org/EIPS/eip-8141) proposes frame transactions, which let individual accounts adopt new signature schemes. As a fully EVM-compatible ledger, Cosmos EVM conforms to that roadmap and ships user-side post-quantum support as it lands upstream. + +## Next steps + +- Upgrade first; every flow on this page requires SDK 0.55 and CometBFT 0.40. See the [release notes](/sdk/next/upgrade/v0.55-release). +- Create a post-quantum user account and move funds into it. See [Create an ML-DSA account](/sdk/next/keys/create-ml-dsa-account). +- Allow `ml_dsa_65` on a new or live chain. See [Enable ML-DSA keys](/sdk/next/keys/enable-ml-dsa-keys). +- Learn how rotation works before touching a production validator. See [Key rotation](/sdk/next/keys/key-rotation). diff --git a/sdk/next/keys/rotate-validator-key-poa.mdx b/sdk/next/keys/rotate-validator-key-poa.mdx new file mode 100644 index 000000000..5a0a2b58c --- /dev/null +++ b/sdk/next/keys/rotate-validator-key-poa.mdx @@ -0,0 +1,149 @@ +--- +noindex: true +title: "Rotate a consensus key, PoA" +description: "Rotate a PoA validator's consensus key as the operator or the admin: generate the key, submit the rotation, and time the node cutover." +--- + +This guide rotates a PoA validator's consensus key using the `enterprise/poa` module. PoA rotation drops three of the staking rails: there is no fee, no rate limit, and no rotation history. PoA chains have no slashing or evidence handling to protect. The operator can rotate its own key, and the chain admin can rotate any validator's key. For how rotation works in general, see [Key rotation](/sdk/next/keys/key-rotation). + +Key rotation can introduce security implications for your chain. Read the [Key rotation](/sdk/next/keys/key-rotation) overview in its entirety before proceeding. In particular, vote extensions are not supported on PoA chains, and custom logic that resolves a validator from a `LastCommit` address will not find a rotating validator for two heights after a rotation. See [Staking and PoA chains](/sdk/next/keys/key-rotation#staking-and-poa-chains). + +Commands use `simd`. Substitute your chain's binary. Examples use `~/.node` for the validator node's home, `~/.poa-newkey` for the scratch home holding the new key, and `val` for the operator key name. + +{/* +Manual verification setup (hidden, macOS). Reproduces a single-validator PoA chain, using the enterprise/poa simapp. +Build: cd enterprise/poa/simapp && go build -o /tmp/poa-simd ./simd, then use /tmp/poa-simd as `simd`. +PoA validators come from poa genesis (no gentx); operator_address is a plain acc address (cosmos1...), not cosmosvaloper1.... + +simd init mynode --chain-id my-chain-1 --home ~/.node +simd config set client chain-id my-chain-1 --home ~/.node +simd config set client keyring-backend test --home ~/.node +simd keys add val --keyring-backend test --home ~/.node +simd keys add admin --keyring-backend test --home ~/.node +VAL=$(simd keys show val -a --keyring-backend test --home ~/.node) +ADMIN=$(simd keys show admin -a --keyring-backend test --home ~/.node) +simd genesis add-genesis-account $VAL 1000000000000stake --home ~/.node +simd genesis add-genesis-account $ADMIN 1000000000000stake --home ~/.node +G=~/.node/config/genesis.json +jq --arg admin "$ADMIN" '.app_state.poa.params.admin=$admin' $G > /tmp/g && mv /tmp/g $G +jq --arg op "$VAL" --slurpfile pk <(jq '{pub_key:{"@type":"/cosmos.crypto.ed25519.PubKey", key:.pub_key.value}, power:10000000, metadata:{moniker:"mynode", operator_address:"'$VAL'"}}' ~/.node/config/priv_validator_key.json) '.app_state.poa.validators=$pk' $G > /tmp/g && mv /tmp/g $G +sed -i '' 's/timeout_commit = ".*"/timeout_commit = "1s"/' ~/.node/config/config.toml +sed -i '' 's/minimum-gas-prices = ".*"/minimum-gas-prices = "0stake"/' ~/.node/config/app.toml +simd start --home ~/.node +# then follow the guide. Note: on a single-validator chain the block production halts between the set +# switch (step 3) and the node key swap (step 4); that is expected and step 4 recovers it. +*/} + +{/* CURRENT VERIFICATION: blind execution audit 2026-07-28 at cosmos-sdk v0.55.0 (64fd208a11) with the PoA binary from enterprise/poa/simapp. Verdict PARTIAL, the only defect being the unsatisfiable "v1.1 or later" prerequisite, since corrected on this page. Every procedural step verified: module state swapped in the landing block h44 with the CometBFT set switching at h46, power/moniker/operator preserved, the >1/3 halt Danger reproduced (chain froze at h45 until the swap), step 4 recovery resumed to h67, and all four documented rejections fired including the unauthorized-signer and nothing-to-rotate cases. Full run: ~/Documents/tests/keys-docs-audit-findings.md. Earlier run, retained for provenance: ed25519 run-verified 2026-07-20 against merged enterprise/poa (cosmos-sdk origin/main 3d3b901ce5, includes #26590 merge c52e67a39d; poa binary built via `cd enterprise/poa/simapp && go build ./simd`). Operator self-rotation and admin override PASS: rotate_cons_pubkey event with operator/old/new, power+operator preserved, set switch at tx height +2, fees migrated; all four rejections reproduced (already-in-use, nothing-to-rotate, unauthorized, type-not-in-params); power-0 note holds; cutover timing confirmed. Full ed25519 run: ~/Documents/tests/poa-rotation-test-findings.md. ML-DSA: run-verified 2026-07-28 on cosmos-sdk v0.55.0, which includes #26614 (merged as d6a3c6e27a). Operator and admin ML-DSA rotation both execute, the set switches to cometbft/PubKeyMlDsa65, and --gas auto is required (an ML-DSA pubkey exceeds the 200000 default). The earlier `unknown pubkey type: ml_dsa_65` failure no longer reproduces; ml_dsa_65 is registered in the pubkey factory at enterprise/poa/x/poa/module.go:134 behind the module option the PoA simapp enables. */} + +## Prerequisites + +- A chain binary built with the `poa` module. Clone the cosmos-sdk repo and build it: `cd enterprise/poa/simapp && go build -o /tmp/poa-simd ./simd`, then use that binary as `simd`. The `-o` is required, since `./simd` is the package directory. Check with `simd tx poa --help`, which must list `rotate-cons-pub-key`. +- [jq](https://jqlang.org/) and [curl](https://curl.se/). The commands extract the new public key with jq and watch the validator set with curl. +- A running PoA validator you operate, with the chain's binary on your PATH and a secondary machine or scratch directory for the new key. PoA validators are set in genesis under `app_state.poa`. + {/* app_state.poa shape: "poa": { "params": { "admin": "cosmos1..." }, "validators": [{ "pub_key": { "@type": "/cosmos.crypto.ed25519.PubKey", "key": "" }, "power": "10000000", "metadata": { "moniker": "mynode", "operator_address": "cosmos1..." } }] } */} +- For an ML-DSA rotation, `ml_dsa_65` is in the chain's consensus params. See [Enable ML-DSA keys](/sdk/next/keys/enable-ml-dsa-keys). +- The signer is the validator's operator, or the chain admin. Any other sender is rejected. + +## 1. Generate a new consensus key + +Create the new key on a secondary machine or offline. Do not touch the live node's `priv_validator_key.json` yet: + +```shell +simd init poa-newkey --chain-id my-chain-1 --home ~/.poa-newkey +``` + +To rotate to a post-quantum key, add `--consensus-key-algo ml_dsa_65` to the command above. See [Migrate a validator to ML-DSA](/sdk/next/keys/migrate-validator-ml-dsa). + +The important output is `~/.poa-newkey/config/priv_validator_key.json`, which is the new consensus key. Confirm it and read its public key: + +```shell +simd comet show-validator --home ~/.poa-newkey +``` + +## 2. Submit the rotation + +The command takes the new public key as base64 plus its type, and identifies the validator by operator address. Both derive from earlier steps: + +```shell +simd tx poa rotate-cons-pub-key "$(simd comet show-validator --home ~/.poa-newkey | jq -r .key)" ed25519 --operator-address "$(simd keys show val -a --home ~/.node)" --from val --home ~/.node +``` + +To rotate to a post-quantum key, pass `ml_dsa_65` instead of `ed25519` in the command above, and add `--gas auto --gas-adjustment 1.5` since the ML-DSA public key exceeds the default gas limit. See [Migrate a validator to ML-DSA](/sdk/next/keys/migrate-validator-ml-dsa). + +The operator address is a regular account address (`cosmos1...`), not a `cosmosvaloper1...` address, so read it with `simd keys show val -a`. + +Add standard transaction flags as your setup requires: `--chain-id` and `--keyring-backend` if your client config does not supply them, and `--fees` (or `--gas-prices`) to meet the chain's minimum gas price. + +The transaction re-keys the validator's state and migrates its accrued fees in the same block. Power, metadata, and the operator address are unchanged. + +To rotate as the admin instead, see [Rotate as the admin](#rotate-as-the-admin). + +If the rotated validator holds more than 1/3 of voting power, the chain halts during the cutover period (steps 3 and 4) and does not resume until the node with the rotated key starts signing. + +## 3. Wait for the validator set to switch + +The chain's state swaps immediately, so `simd q poa validators` shows the new key in the same block the transaction lands. CometBFT applies the actual validator set change two blocks later, and only that switch governs when the node must sign with the new key. Until it happens, CometBFT still expects the old key, so keep the live node running untouched. Watch the CometBFT validator set until the new consensus key appears and the old one is gone: + +```shell +curl -s localhost:26657/validators | jq -r '.result.validators[].pub_key.value' +``` + +The value should match the key you read in step 1. + +Do not swap the node's key before the set switches (wait at least 2 blocks). Swapping early makes the node sign with a key CometBFT does not yet expect, and the validator misses blocks. + +## 4. Swap the node's key + +Once the new consensus address is in the set, stop the node, replace its key, and restart: + +```shell +cp ~/.poa-newkey/config/priv_validator_key.json ~/.node/config/priv_validator_key.json +``` + +Confirm the node signs under the new consensus address after the restart. If you run a redundant standby, make a single switch. Keep the old-key node signing until the set updates, stop it fully, and only then let the new-key node start signing. + +A validator with power 0 is outside the active set. Its rotation emits no validator set update, so there is no transition to time. Swap the node's key first, then have the admin grant power. + +## Rotate as the admin + +The admin override changes on-chain state only. Whoever runs the node must still swap `priv_validator_key.json` with the timing in steps 3 and 4 above. Otherwise the validator goes dark until its node signs with the new key. Coordinate the node-side swap with the operator before submitting, unless the goal is to cut off a compromised key. + +The admin rotates any validator's key with the same command, signed by the admin key. Generate a fresh key home for it: + +```shell +simd init poa-adminkey --chain-id my-chain-1 --home ~/.poa-adminkey +``` + +Then rotate the key: + +```shell +simd tx poa rotate-cons-pub-key "$(simd comet show-validator --home ~/.poa-adminkey | jq -r .key)" ed25519 --operator-address --from admin --home ~/.node +``` + +## Verify + +Confirm the module carries the new key: + +```shell +simd q poa validators --home ~/.node +``` + +The rotation also emits a `rotate_cons_pubkey` event with the operator address and the old and new consensus addresses. Read it from the transaction: + +```shell +simd q tx --home ~/.node +``` + +## What can go wrong + +- The transaction is rejected as unauthorized: the signer is neither the validator's operator nor the admin. +- The transaction is rejected for the key itself: the new key equals the current one, is already used by another validator, or its type is not in the chain's consensus params. +- The validator misses blocks right after the swap: the node's key was replaced before the set switched. Restore the old key, wait for the set, then swap again. +- The validator goes dark after an admin rotation: the node still holds the old key. Swap `priv_validator_key.json` and restart. + +## Next steps + +- Rotate to a post-quantum key. See [Migrate a validator to ML-DSA](/sdk/next/keys/migrate-validator-ml-dsa). +- Understand the mechanics and the staking differences. See [Key rotation](/sdk/next/keys/key-rotation). +- Look up the message. See the [PoA API reference](/sdk/next/enterprise/poa/api#rotateconspubkey). diff --git a/sdk/next/keys/rotate-validator-key.mdx b/sdk/next/keys/rotate-validator-key.mdx new file mode 100644 index 000000000..f6b55a9f7 --- /dev/null +++ b/sdk/next/keys/rotate-validator-key.mdx @@ -0,0 +1,125 @@ +--- +noindex: true +title: "Rotate a consensus key, Staking" +description: "Rotate a staked validator's consensus key with no downtime: run a second node, submit the rotation, verify, and retire the old node." +--- + +This guide rotates a staked validator's consensus key with no downtime: run a second node on the new key, submit the rotation, verify, and retire the old node. For how rotation works and its limits, see [Key rotation](/sdk/next/keys/key-rotation). + +Key rotation can introduce security implications for your chain. Read the [Key rotation](/sdk/next/keys/key-rotation) overview in its entirety before proceeding. + +Commands use `simd`. Substitute your chain's binary. Examples use `~/.node` for the existing node's home and `~/.rotation-node` for the new one; those paths, the key name `val`, and host addresses are the only values to adjust. + +{/* +Manual verification setup (hidden, macOS). Reproduces the running staked chain this guide assumes, using simd/simapp. +Build: from the cosmos-sdk repo run `make build`, then put ./build/simd on PATH as `simd`. + +simd init mynode --chain-id my-chain-1 --home ~/.node +simd config set client chain-id my-chain-1 --home ~/.node +simd config set client keyring-backend test --home ~/.node +simd keys add val --keyring-backend test --home ~/.node +simd genesis add-genesis-account val 100000000000stake --keyring-backend test --home ~/.node +simd genesis gentx val 1000000000stake --chain-id my-chain-1 --keyring-backend test --home ~/.node +simd genesis collect-gentxs --home ~/.node +# fast blocks + zero min-gas so the guide's commands work without extra flags +# (substitute --fees in step 3 with e.g. 2000stake, or omit it on this localnet) +sed -i '' 's/timeout_commit = ".*"/timeout_commit = "1s"/' ~/.node/config/config.toml +sed -i '' 's/minimum-gas-prices = ".*"/minimum-gas-prices = "0stake"/' ~/.node/config/app.toml +simd start --home ~/.node +# then follow the guide. The new node in step 1 shares this host, so it also needs +# --p2p.laddr/--rpc.laddr/--grpc.address and a distinct pprof_laddr (see step 1's note). +*/} + +## Prerequisites + +- The chain runs Cosmos SDK 0.55 and CometBFT 0.40 or later, and allows your target key type. See [Enable ML-DSA keys](/sdk/next/keys/enable-ml-dsa-keys). +- [jq](https://jqlang.org/) and [curl](https://curl.se/) for the verification steps. +- A running validator you operate, and a second machine (or spare ports on the same host) for the new node, with the chain's binary installed on it. To build `simd` and run a node, see [Run a node](/sdk/next/node/run-node). +- No rotation in the current unbonding period. Only one rotation is allowed per unbonding period. +- The operator account holds enough funds for two separate charges: the rotation fee, which is burned, and the ordinary gas fee for the transaction itself. Check the rotation fee: + +```shell +simd query staking params +``` + +The `key_rotation_fee` field shows the fee amount. + +## 1. Start a second node with the new key + +Initialize a fresh node home. The init command generates a new consensus key in `priv_validator_key.json`: + +```shell +simd init rotation-node --chain-id my-chain-1 --home ~/.rotation-node +``` + +To rotate to a post-quantum key, add `--consensus-key-algo ml_dsa_65` to the command above. See [Migrate a validator to ML-DSA](/sdk/next/keys/migrate-validator-ml-dsa). + +Never copy the old `priv_validator_key.json` to the new node. Two nodes signing with the same consensus key is a double sign, which tombstones the validator. The new node must have its own freshly generated key. + +A fresh init writes a placeholder genesis. Replace it with the chain's genesis: + +```shell +cp ~/.node/config/genesis.json ~/.rotation-node/config/genesis.json +``` + +Start the new node peered with the existing one, and let it sync to the chain head: + +```shell +simd start --home ~/.rotation-node --p2p.persistent_peers "$(simd comet show-node-id --home ~/.node)@127.0.0.1:26656" +``` + +Adjust the peer address to the existing node's host. If both nodes share a host, also give the new node its own ports with `--p2p.laddr`, `--rpc.laddr`, `--grpc.address`, and `--rpc.pprof_laddr`. The pprof port is required, not optional: without it the new node exits at startup with `address already in use` for the default port 6060, which the first node already holds. Until the rotation applies, the node follows the chain as a non-signing full node. + +On a chain with history, a fresh node takes days to sync from genesis. Use state sync or a snapshot to reach the chain head quickly. See [State sync](/sdk/next/node/run-node#state-sync). + +## 2. Confirm both nodes are healthy + +Before you submit, the old node must still be signing and the new node must be caught up to the chain head. If the new node is still syncing when the rotation applies, the validator will miss blocks until it catches up. Check sync status: + +```shell +curl -s localhost:26657/status | jq '.result.sync_info.catching_up' +``` + +Run this against each node, using the RPC port each one listens on (a co-located new node answers on the `--rpc.laddr` port you gave it, not `26657`). Both must report `false`. + +## 3. Submit the rotation + +The rotation message carries the new node's public key, read directly from that node's home: + +```shell +simd tx staking rotate-cons-pub-key "$(simd comet show-validator --home ~/.rotation-node)" --from val --home ~/.node --gas auto --gas-adjustment 1.5 --fees --yes +``` + +Set `--fees` (or `--gas-prices`) to meet the chain's minimum gas price; without it the node rejects the transaction with `insufficient fees`. This gas fee is separate from the burned rotation fee. The command reads the operator key and chain ID from your client config; add `--chain-id`, `--keyring-backend`, and `--home` if that config does not already supply them. + +A rotation cannot be undone. After it applies, the validator is committed to the new key for the rest of the unbonding period. Keep the old node running until step 4 verifies the new key is signing. + +## 4. Verify the new key is signing + +The rotation applies two heights after the message executes, so the new key can appear within seconds on a fast chain. A successful broadcast confirms only that the chain accepted the transaction; confirm the validator set actually carries the new key: + +```shell +curl -s localhost:26657/validators | jq -r '.result.validators[].pub_key' +``` + +The `value` matches the `key` field shown by `simd comet show-validator --home ~/.rotation-node`, and the old key is gone. If the set still shows the old key, wait a few blocks and check again. Watch the new node's logs to see it signing votes. + +## 5. Retire the old node + +Stop the old node and decommission it. Its consensus key holds no power, but it remains slashable for past behavior until equivocation evidence for it can no longer be admitted. That window is at least the unbonding period and can be longer, depending on the chain's evidence params. See [Key rotation](/sdk/next/keys/key-rotation). Store its key material securely rather than leaving it on shared infrastructure. + +## What can go wrong + +- The transaction is rejected with a rotation limit error: a rotation already happened this unbonding period. Wait out the window. +- The transaction is rejected for an unsupported key type: the target type is not in the chain's consensus params. See [Enable ML-DSA keys](/sdk/next/keys/enable-ml-dsa-keys). +- The fee cannot be paid: fund the operator account with at least `key_rotation_fee`. +- The transaction is rejected because the new key is unavailable: another validator already uses it, or a recent rotation still holds it locked. Generate a fresh key. +- The transaction is rejected because the validator is jailed: unjail it first. +- The validator misses blocks after the rotation applies: the new node was not caught up. It resumes signing once synced. + +## Next steps + +- Understand the mechanics behind each step. See [Key rotation](/sdk/next/keys/key-rotation). +- Rotate to a post-quantum key. See [Migrate a validator to ML-DSA](/sdk/next/keys/migrate-validator-ml-dsa). +- Rotate a key held in a remote signer. See [Rotate a consensus key held in Cosmos-KMS](/sdk/next/kms/rotate-key-remote-signer). +- Look up the message and parameters. See the [x/staking module reference](/sdk/next/modules/staking/README#msgrotateconspubkey). diff --git a/sdk/next/kms/best-practices.mdx b/sdk/next/kms/best-practices.mdx new file mode 100644 index 000000000..9707ece38 --- /dev/null +++ b/sdk/next/kms/best-practices.mdx @@ -0,0 +1,63 @@ +--- +noindex: true +title: "Remote signing best practices" +description: "Choose and defend a signing architecture: placement, transport, key separation, and the one-signer rule." +--- + +A remote signing setup is a set of trust decisions: where the signer runs, how the connection is secured, and which keys are protected to what degree. This page states the recommended defaults and the reasoning, so a setup can be defended in a security review rather than inherited by accident. + +## Place the signer in its own trust domain + +Run the signer on a separate host, in a separate network segment, with network ACLs between it and the validator. The validator node is the exposed machine: it peers with the public network and sees frequent maintenance. The signing host should do exactly one job, accept no inbound connections, and be reachable by as few people and systems as possible. + +The signer dials out to the validator, so this layout costs nothing: the signing host needs no open ports at all. The validator's privval listener is the only listening side; bind it to a private interface and firewall it so only the signing host can reach it. + +Choose the layout by the deployment: + +- A separate host with a network firewall is the default and reduces the attack surface the most. +- If the validator sits behind a sentry node, network isolation is already in place, so running the signer locally is reasonable. +- If cost is a constraint, running both processes on the same host still keeps the key in custody and off the node's disk, an acceptable tradeoff for the core benefit of remote signing. + +In the two local layouts the key still never touches disk, but the signer process and its credentials live on the exposed machine. + +Every layout adds some network latency between node and signer, so weigh that against the isolation each one provides. + +## Prefer the Noise transport + +Two transports secure the privval connection; the address scheme in `kms.yaml` selects between them. The default `tcp://` uses CometBFT's SecretConnection: the signer authenticates itself to the validator with its identity key. The validator's listener uses an ephemeral key, so the signer cannot verify it is talking to the right validator. + +The `noise://` transport closes that gap with mutual pinning. Each side asserts a stable peer ID: the signer's derives from its identity key, the validator's from its node key. Each side refuses a connection from any unexpected peer. Exchange the two peer IDs out of band and pin them: + +```shell +kms peer-id --home ~/.kms +``` + +```shell +cometbft show-node-id --libp2p --home ~/.node +``` + +Use `noise://` for any deployment where the signer and validator cross a network you do not fully control. + +## Protect keys according to their power + +The consensus key is the asset; keep it in the HSM or cloud KMS and never in a file on production hosts. The signer's identity key (`identity.json`) only authenticates the connection: it signs no consensus messages, and losing it means re-pinning a new peer ID, not a compromise. The two keys do not need the same protection level. Treating the identity key as low value keeps operational friction down. + +## Signer, chain, and key topology + +One signer can sign for several chains at once. Each chain is backed by exactly one key, so a single signer can hold several keys, one per chain. For redundancy, one signer can also connect to more than one node on the same chain, such as a primary and a backup. Only one live signer may hold a given key, as the next section explains. + +## Run exactly one signer per validator + +Double-sign protection lives in the signer's per-chain state file, which records the highest height, round, and step ever signed. That protection assumes one writer. Two signer instances holding the same consensus key with separate or missing state files can each sign the same height, which is a double sign. + +Never run two signer instances against the same validator key, because this can cause double signing. + +## Keep the gRPC listener off consensus paths + +The optional gRPC SignerService performs no caller authentication or authorization: any client that reaches the listener can use every configured key. If the service is enabled, front it with TLS, restrict it with network policy, and give it only the keys it exists to serve. + +## Next steps + +- Set up the backend that holds the key. See [Configure a signing backend](/sdk/next/kms/configure-backend). +- Look up transport and connection fields. See the [configuration reference](/sdk/next/kms/configuration-reference). +- Rotate the consensus key without moving it out of custody. See [Rotate a consensus key held in Cosmos-KMS](/sdk/next/kms/rotate-key-remote-signer). diff --git a/sdk/next/kms/configuration-reference.mdx b/sdk/next/kms/configuration-reference.mdx new file mode 100644 index 000000000..b8eb16b52 --- /dev/null +++ b/sdk/next/kms/configuration-reference.mdx @@ -0,0 +1,128 @@ +--- +noindex: true +title: "Cosmos-KMS configuration reference" +description: "Every field of kms.yaml: chains, validators, keys with per-backend parameters, and the gRPC block." +--- + +The signer reads one file, `/kms.yaml`, at startup. Relative paths anywhere in the file resolve against the `--home` directory. Validation runs at `kms start`; a rejected field is named in the error. + +For task-shaped setup, see [Configure a signing backend](/sdk/next/kms/configure-backend); this page is the complete field list. + +## chains + +Declares one chain to sign for. One entry per chain. + +| Field | Type | Required | Description | +| --- | --- | --- | --- | +| `id` | string | yes | The chain ID, matching the chain's genesis. | +| `state_file` | string | no | Path to the double-sign protection state file. Defaults to `/state/.json`. | + +## validators + +Declares one outbound connection to a validator node's privval listener. A chain can have multiple entries, for example a primary and a backup node. + +| Field | Type | Required | Description | +| --- | --- | --- | --- | +| `chain_id` | string | yes | Must match a declared `chains[].id`. | +| `addr` | string | yes | The listener address. `tcp://host:port` selects the SecretConnection transport; `noise://@host:port` selects the Noise transport with mutual peer pinning. | +| `identity_key` | string | yes | Path to the signer's identity key file, generated by `kms init`. Authenticates the SecretConnection, and doubles as the signer's Noise identity. | +| `reconnect` | bool | no | Reconnect automatically after a dropped connection. Defaults to `true`. | + +## keys + +Binds one signing key to one or more chains. Each chain must be backed by exactly one key. The `backend` field selects the custodian, and the remaining fields depend on it; fields belonging to other backends are ignored. + +Fields shared by every backend: + +| Field | Type | Required | Description | +| --- | --- | --- | --- | +| `chain_ids` | list of strings | yes | Chains this key signs for. Each must match a declared `chains[].id`. | +| `backend` | string | no | `file` (default), `pkcs11`, or `awskms`. | +| `algorithm` | string | yes | Key algorithm: `ed25519`, `secp256k1`, `secp256k1eth`, or `mldsa65`. Set it explicitly. | +| `key_id` | string | per backend | For `pkcs11`: hex `CKA_ID` of the key object. For `awskms`: KMS key ID, ARN, or `alias/`. | + +Consensus signing supports `ed25519`, `secp256k1eth`, and `mldsa65` on every backend, and `secp256k1` on the AWS KMS backend only. The algorithm name `mldsa65` has no underscores; the chain-side key type `ml_dsa_65` does. + +### backend: file + +A key read from disk into memory. Development and testing only; the key is held in plaintext. + +| Field | Type | Required | Description | +| --- | --- | --- | --- | +| `key_file` | string | yes | Path to the key. Accepts a CometBFT `priv_validator_key.json`, or a raw private key file (base64-encoded for `ed25519` and `mldsa65`, hex-encoded for `secp256k1eth`). | + +### backend: pkcs11 + +A key on a PKCS#11 token or HSM. Signing happens on-device; the signer uses an existing key and never generates or imports one. + +| Field | Type | Required | Description | +| --- | --- | --- | --- | +| `module` | string | yes | Path to the PKCS#11 module shared library. | +| `token_label` | string | exactly one of the two | `CKA_LABEL` of the token. | +| `slot` | integer | exactly one of the two | Slot number of the token. | +| `key_label` | string | at least one of `key_label`/`key_id` | `CKA_LABEL` of the key object. | +| `pin` | string | exactly one PIN source | User PIN, inline. Prefer the alternatives below. | +| `pin_env` | string | exactly one PIN source | Environment variable holding the PIN. | +| `pin_file` | string | exactly one PIN source | Path to a file holding the PIN. | + +### backend: awskms + +A key held in AWS KMS. Signing happens through the KMS Sign API; credentials resolve through the AWS default credential chain, and no secret material appears in the config. + +| Field | Type | Required | Description | +| --- | --- | --- | --- | +| `region` | string | no | AWS region of the key. Falls back to the AWS default chain. | +| `profile` | string | no | Shared-config profile name. Falls back to the AWS default chain. | +| `endpoint` | string | no | Custom KMS endpoint URL, for LocalStack-style testing. Leave unset for AWS. | + +## grpc + +Optional. When present, the signer also serves the SignerService gRPC API alongside privval. Its usage documentation ships with the interoperability release; the fields are listed here for completeness. + +| Field | Type | Required | Description | +| --- | --- | --- | --- | +| `listen` | string | yes | `host:port` the gRPC server binds to. | +| `tls_cert` | string | no | TLS server certificate file. Omitting both TLS fields serves plaintext, for local testing only. | +| `tls_key` | string | no | TLS server private key file. | +| `keys` | list | yes | The keys the service exposes; see below. | + +Each `grpc.keys` entry: + +| Field | Type | Required | Description | +| --- | --- | --- | --- | +| `id` | string | yes | Logical key identifier returned to clients. | +| `backend` | string | yes | `file`, `awskms`, or `pkcs11`. | +| `algorithm` | string | yes | For `file` and `pkcs11`: `ed25519` or `secp256k1eth`. For `awskms`: `ed25519`, `secp256k1`, or `secp256k1eth`. `mldsa65` has no gRPC signature scheme and is privval-only. | +| `key_id` | string | awskms | KMS key ID, ARN, or `alias/`. | +| `key_file` | string | file | Path to the key. Accepts a CometBFT `priv_validator_key.json`, or a raw private key file (base64-encoded for `ed25519`, hex-encoded for `secp256k1eth`). | + +A `grpc.keys` entry with `backend: pkcs11` also takes the PKCS#11 fields, with the same rules and the same validation as a privval key. See [backend: pkcs11](#backend-pkcs11) for `module`, `token_label`, `slot`, `key_label`, `key_id`, and the PIN sources. + +The gRPC server performs no caller authentication or authorization. Any client that can reach the listener can use every configured key. Restrict access with TLS and network controls. + +## Constraints checked at startup + +- Every `validators[].chain_id` and every entry in `keys[].chain_ids` must match a declared `chains[].id`. +- Each chain must be backed by exactly one key. +- PKCS#11 keys must select the token with exactly one of `token_label` or `slot`, select the key with `key_label` or `key_id`, and supply exactly one PIN source. +- A declared chain with no `validators` entry is not rejected. The signer starts, binds nothing, and signs nothing, with no warning, so check that every chain has a validator entry. +- Every chain's sign-state file must exist and be non-empty. A missing or empty file fails closed with `sign-state file is missing or empty; refusing to start at height 0`, so the signer cannot re-sign a height it has no record of. See [First start on a new chain](#first-start-on-a-new-chain). + +This list is not exhaustive. Config-level rejections are prefixed `config:` and name the field at fault; errors raised later, when a backend or chain signer is opened, use their own prefixes such as `app:` or `file:`. + +## First start on a new chain + +A key that has never signed on a chain has no sign-state file, so the checks above block its first start. Write the height-0 floor with: + +```shell +kms start --home --allow-fresh-state +``` + +`kms state init` writes a floor too, but it loads and validates `kms.yaml` first, so it only works once the chain is declared and the config is complete. It also defaults `--step` to 3, which refuses everything at that height and round, where `--allow-fresh-state` writes step 0. Pass `--step 0` to match. See [Migrate from TMKMS](/sdk/next/kms/migrate-from-tmkms) for more details. + +Pass `--allow-fresh-state` only for a chain the key has genuinely never signed on. It will not overwrite an existing floor, but a service definition that carries it permanently removes the protection: a deleted or truncated state file resets the double-sign floor to zero instead of stopping the signer. + +## Next steps + +- Task-shaped backend setup. See [Configure a signing backend](/sdk/next/kms/configure-backend). +- First-time setup end to end. See [Remote signing tutorial](/sdk/next/kms/tutorial-file-backend). diff --git a/sdk/next/kms/configure-backend.mdx b/sdk/next/kms/configure-backend.mdx new file mode 100644 index 000000000..dcdcb8200 --- /dev/null +++ b/sdk/next/kms/configure-backend.mdx @@ -0,0 +1,169 @@ +--- +noindex: true +title: "Configure a signing backend" +description: "Point Cosmos-KMS at the custodian holding the consensus key: AWS KMS, a PKCS#11 HSM, or a file." +--- + +A signing backend is the custodian that holds the validator's consensus key and signs with it. Cosmos-KMS supports three: AWS KMS, a PKCS#11 hardware module, and a file on disk. The `keys` block in `kms.yaml` selects one, and this guide configures each in turn. + +## Prerequisites + +- A running signer and node, which [Remote signing tutorial](/sdk/next/kms/tutorial-file-backend) sets up. Between backends only the `keys` block changes in `kms.yaml`. The file backend reuses the tutorial's existing key, but the AWS KMS and PKCS#11 backends hold a new consensus key the validator must adopt first. See [Adopt the key on a validator](#adopt-the-key-on-a-validator). +- Per backend: the [AWS CLI](https://aws.amazon.com/cli/) and an AWS account for AWS KMS; your HSM's tooling plus [OpenSC](https://github.com/OpenSC/OpenSC)'s `pkcs11-tool` for PKCS#11, with [SoftHSM2](https://www.opendnssec.org/softhsm/) as a local test rig. +- The chain's binary; the examples use `simd`. To build it and run a node, see [Run a node](/sdk/next/node/run-node). + +The `algorithm` field is required for all backends. + +## AWS KMS + +AWS KMS caps messages it signs in raw form at 4096 bytes. This binds `ed25519` and `secp256k1`, which send the raw consensus message, so be careful with features that enlarge it, such as vote extensions. The signer checks the size itself and fails before calling AWS. `mldsa65` and `secp256k1eth` are not bound by the cap. + +AWS Key Management Service (KMS) is a managed service that stores cryptographic keys and signs with them on request. With this backend, the consensus key lives in KMS and never leaves it. The signer calls the KMS Sign API to produce each signature. Credentials come from the standard AWS default chain: environment, shared config, SSO, or an IAM role. No secrets enter `kms.yaml`. + +This backend signs with any key type Cosmos-KMS supports: `ed25519`, `secp256k1`, `secp256k1eth`, and post-quantum `mldsa65`. Provision the AWS key with the key spec that matches the algorithm, such as `ECC_NIST_EDWARDS25519` for `ed25519` or `ML_DSA_65` for `mldsa65`. + +Create an ML-DSA-65 signing key and give it an alias (or skip these and point `key_id` at a key you already have): + +```shell +aws kms create-key --key-spec ML_DSA_65 --key-usage SIGN_VERIFY +``` + +```shell +aws kms create-alias --alias-name alias/validator --target-key-id +``` + +Then bind it in the `keys` block: + +```yaml +keys: + - chain_ids: [my-chain-1] + backend: awskms + algorithm: mldsa65 + key_id: alias/validator + region: us-east-1 +``` + +The `key_id` accepts a key ID, a full ARN, or an alias. `region` is optional and falls back to the AWS default chain. Two more optional fields, not shown above, are `profile` (a named shared-config profile) and `endpoint` (for LocalStack-style testing only). + +The signer needs only two IAM permissions on the key: `kms:GetPublicKey`, called once at startup, and `kms:Sign`, called per block. `kms:DescribeKey` is not required. Attach a least-privilege policy scoped to the key ARN, not the alias; AWS resolves the alias to the key server-side: + +```json +{ + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Action": ["kms:GetPublicKey", "kms:Sign"], + "Resource": "arn:aws:kms:::key/" + } + ] +} +``` + +A freshly created KMS key is a new consensus key. To move an existing validator onto it, rotate the validator to the new key; to stand up a new validator, register it with the new key's public key. See [Rotate a consensus key held in Cosmos-KMS](/sdk/next/kms/rotate-key-remote-signer). Creating the key inside KMS keeps it from ever leaving the service. + +{/* AWS IS STILL THE 2026-07-20 RUN. The 2026-07-28 blind audit deliberately skipped AWS (no account, no simulating), so the run below remains the only end-to-end evidence for this backend and must not be restamped with a later date. Its claims were re-checked from source on 2026-07-28 at kms v0.1.0 and still hold. Manual verification (awskms backend): run-verified 2026-07-20 against real AWS KMS (us-east-1, account 326804803147; simd cosmos-sdk @ 46a177139a, kms @ 0007b0d). Both ed25519 (ECC_NIST_EDWARDS25519) and mldsa65 (ML_DSA_65) keys signed consensus end to end: create-key + alias as written, validator built around the KMS key's pubkey, blocks flowed, stop-kms→stall / restart→resume passed; CloudTrail confirmed kms:GetPublicKey at startup and kms:Sign per block. Full run: ~/Documents/tests/kms-aws-verification-findings.md. */} + +## `PKCS#11` + +The `PKCS#11` backend keeps the consensus key on a hardware security module (HSM) or token and signs on the device through `PKCS#11`, the standard interface for cryptographic hardware. The key never leaves the module. The signer uses an existing key only, so provision one with your HSM tooling first. The example commands below show how to provision an ed25519 key with SoftHSM2 for testing: + +```shell +softhsm2-util --init-token --free --label validator-token --pin 1234 --so-pin 4321 +``` + +```shell +pkcs11-tool --module /usr/lib/softhsm/libsofthsm2.so --login --pin 1234 --keypairgen --key-type EC:edwards25519 --label validator --id 01 +``` + +The `--module` path is platform-specific: Linux uses `/usr/lib/softhsm/libsofthsm2.so`, and macOS Homebrew uses `/opt/homebrew/lib/softhsm/libsofthsm2.so` (`/usr/local/lib/softhsm/...` on Intel). Use the same path for the `module` field below. + +Then bind it: + +```yaml +keys: + - chain_ids: [my-chain-1] + backend: pkcs11 + algorithm: ed25519 + module: /usr/lib/softhsm/libsofthsm2.so + token_label: validator-token + key_label: validator + pin_env: KMS_PIN +``` + +The signer enforces three field rules at startup: + +- Select the token with exactly one of `token_label` or `slot`. +- Select the key with `key_label`, `key_id` (the hex `CKA_ID`), or both. +- Supply the PIN through exactly one of `pin`, `pin_env`, or `pin_file`. + +Prefer `pin_env` or `pin_file`. An inline `pin` puts the PIN in the config file. + +Set the PIN in the environment the signer runs in so it matches `pin_env`: `export KMS_PIN=`. If you are using SoftHSM2 as the test rig, also `export SOFTHSM2_CONF=` so the module can find the token. + +As with AWS KMS, a key generated in the HSM is a new consensus key. Move an existing validator onto it by rotation, or register a new validator with its public key. See [Rotate a consensus key held in Cosmos-KMS](/sdk/next/kms/rotate-key-remote-signer). + +{/* Re-verified 2026-07-28 by blind execution audit at shipping refs (kms v0.1.0 4bd83922ee, cometbft v0.40.0, simd v0.55.0 64fd208a11, SoftHSM2 2.7.0): token provisioning ran verbatim, the HSM-held ed25519 key signed every block, `gentx --pubkey` adoption worked, and stop-signer/stall plus restart/resume both passed. The macOS module-path note on this page is correct and load-bearing. mldsa65 on PKCS#11 remains untestable locally: SoftHSM2 exposes no ML-DSA mechanism. Full run: ~/Documents/tests/kms-docs-audit-findings.md. Original run, retained for provenance: run-verified 2026-07-20 on macOS arm64 (simd cosmos-sdk @ 46a177139a, kms @ 0007b0d, SoftHSM2 2.7.0, OpenSC 0.27.1). Token inited and ed25519 key generated with the commands as written (module path /opt/homebrew/lib/softhsm/libsofthsm2.so on macOS); the validator's consensus pubkey was set to the HSM key via `simd genesis gentx --pubkey`, the kms pkcs11 backend signed every block, blocks flowed, and stop-kms→stall / restart→resume passed. Full run: ~/Documents/tests/kms-backend-verification-findings.md. */} + +## File + +The file backend reads the consensus key from a file on the signer's disk into memory. It is the development and testing backend. The key sits in plaintext, so it is not production custody. The [Remote signing tutorial](/sdk/next/kms/tutorial-file-backend) covers it end to end: + +```yaml +keys: + - chain_ids: [my-chain-1] + backend: file + algorithm: ed25519 + key_file: priv_validator_key.json +``` + +The `key_file` accepts a CometBFT `priv_validator_key.json` or a raw base64-encoded private key. The file backend also signs post-quantum consensus keys. Generate the key with `simd init --consensus-key-algo ml_dsa_65` and bind it: + +```yaml +keys: + - chain_ids: [my-chain-1] + backend: file + algorithm: mldsa65 + key_file: priv_validator_key.json +``` + +{/* Manual verification (mldsa65 file backend): re-verified 2026-07-28 at shipping refs (kms v0.1.0 4bd83922ee, simd v0.55.0 64fd208a11) by blind execution audit: chain inited with --consensus-key-algo ml_dsa_65 produced cometbft/PubKeyMlDsa65, the signer with algorithm: mldsa65 signed to height 3-4, and key_file accepted a raw base64 private key. Original run 2026-07-17 on macOS (simd from cosmos-sdk pr-26604 @ 46a177139a, kms @ 0007b0d) also covered the stop/resume proof. Full runs: ~/Documents/tests/kms-docs-audit-findings.md and mldsa-keygen-test-findings.md. */} +{/* Verification status (mldsa65 per backend): file — run-verified above (2026-07-17). awskms — run-verified 2026-07-20 against real AWS KMS (ML_DSA_65 key signed consensus end to end; see the AWS manual-verification note). pkcs11 — source/owner-confirmed but NOT run-verified: SoftHSM2 has no ML-DSA mechanism, so it needs an ML-DSA-capable HSM. */} + +## Adopt the key on a validator + +The AWS KMS and PKCS#11 backends generate the key inside the custodian, so it is a new consensus key, not the one your validator already runs. Do not just repoint an existing validator's signer at a fresh key. The node finds its consensus key absent from the validator set and demotes itself, so it never proposes and there is no signature error to look for. The symptom is `This node is not a validator` in the node log and a chain that does not advance. Only the file backend, pointed at the validator's existing `priv_validator_key.json`, skips this step. + +Adopt the key one of two ways: + +- Existing validator: rotate its consensus key to the new one, which derives the new public key from a shadow node and swaps it in with no downtime. See [Rotate a consensus key held in Cosmos-KMS](/sdk/next/kms/rotate-key-remote-signer). +- New validator: register it with the new key's consensus public key using `gentx --pubkey` (or the `pubkey` field of `create-validator`'s validator.json). Read the public key from the custodian itself, since `gentx` runs before the chain exists and there is no node to query. For PKCS#11, read the object and strip the DER wrapper, leaving the raw 32 bytes to base64: + +```shell +pkcs11-tool --module --login --pin --read-object --type pubkey --label validator \ + | xxd -p | tr -d '\n' | sed 's/^302a300506032b6570032100//' | xxd -r -p | base64 +``` + +For AWS KMS, `aws kms get-public-key` returns a DER `SubjectPublicKeyInfo`; strip the same wrapper before encoding. + +## Verify any backend + +Verification is the same regardless of custodian. Start the signer, start the node, and confirm blocks flow, exactly as in [the tutorial](/sdk/next/kms/tutorial-file-backend). + +If this key has never signed on the chain, the signer's first start needs `--allow-fresh-state ` to write the height-0 double-sign floor, otherwise it exits with `sign-state file ... is missing or empty; refusing to start at height 0`. Pass it only on that first start. See [Start the signer](/sdk/next/kms/tutorial-file-backend) in the tutorial for the full explanation, and the [configuration reference](/sdk/next/kms/configuration-reference) for `kms state init`. + +```shell +curl -s localhost:26657/status | jq '.result.sync_info.latest_block_height' +``` + +## What can go wrong + +- The signer rejects the config at startup: both `token_label` and `slot` set, or more than one PIN source. The error names the offending field. A missing `algorithm` on the file backend instead fails with the less specific `file: unknown key type`. +- The signer starts but cannot reach the key: wrong `module` path, wrong `key_id` or alias, or AWS credentials with no permission (denied `kms:GetPublicKey` fails at `kms start` with `awskms: get public key for "": `). These also fail at startup. +- The node exits with a pubkey timeout: the signer is not running or not reachable. Start the signer first. It dials and retries. + +## Next steps + +- Look up any config field, its type, and its constraints. See the [configuration reference](/sdk/next/kms/configuration-reference). +- Run the whole flow once with the file backend. See [Remote signing tutorial](/sdk/next/kms/tutorial-file-backend). +- Move an existing validator onto a key in a new backend by rotation. See [Rotate a consensus key held in Cosmos-KMS](/sdk/next/kms/rotate-key-remote-signer). diff --git a/sdk/next/kms/migrate-from-tmkms.mdx b/sdk/next/kms/migrate-from-tmkms.mdx new file mode 100644 index 000000000..89998aa7b --- /dev/null +++ b/sdk/next/kms/migrate-from-tmkms.mdx @@ -0,0 +1,144 @@ +--- +noindex: true +title: "Migrate from TMKMS" +description: "Move a validator from TMKMS to Cosmos-KMS: translate the config, move or rotate the key, and cut over without double signing." +--- + +This guide moves a validator's signing from TMKMS to Cosmos-KMS, the recommended remote signer going forward. Cosmos-KMS includes several upgrades over TMKMS: it adds AWS KMS and PKCS#11 backends and post-quantum ML-DSA signing. The migration translates the config, gets the key into a Cosmos-KMS backend, and cuts over with exactly one signer alive at every moment. For what Cosmos-KMS is and how it relates to TMKMS, see [Cosmos-KMS and remote signing](/sdk/next/kms/remote-signing). + +The validator node itself needs no changes: both signers speak the same privval protocol to the same `priv_validator_laddr` listener. TMKMS connects over CometBFT's SecretConnection, and the Cosmos-KMS `tcp://` transport is the same, so the listener works unchanged. + +## Prerequisites + +- A validator currently signing through TMKMS, with access to its `tmkms.toml` and state file. +- [jq](https://jqlang.org/), used to translate the state file in step 4. +- Cosmos-KMS installed and initialized with `kms init`. [Remote signing tutorial](/sdk/next/kms/tutorial-file-backend) covers installation, which needs [Go](https://go.dev/doc/install) 1.26 or later, [make](https://www.gnu.org/software/make/), and [git](https://git-scm.com/). + +## 1. Translate the config + +Create `kms.yaml` and translate each block from your `tmkms.toml`: + +| tmkms.toml | kms.yaml | Notes | +| --- | --- | --- | +| `[[chain]]` `id` | `chains[].id` | Same value. | +| `[[chain]]` `state_file` | `chains[].state_file` | See [step 4](#4-translate-the-double-sign-state) before reusing a path. | +| `[[chain]]` `key_format` | none | Not needed; Cosmos-KMS has no per-chain serialization config. | +| `[[validator]]` `addr` | `validators[].addr` | Drop the `@` prefix; `tcp://host:port` uses CometBFT's SecretConnection, matching TMKMS. For a node running libp2p, use `noise://@:` instead, where the peer ID is the validator's, from `cometbft show-node-id --libp2p`. A `noise://` host must be an IP literal, bracketed for IPv6; hostnames are rejected. Noise is mutual, so the validator must also carry the signer's peer ID, from `kms peer-id`, in its allowlist, or it rejects the connection. | +| `[[validator]]` `chain_id` | `validators[].chain_id` | Same value. | +| `[[validator]]` `secret_key` | `validators[].identity_key` | Different format; use the `identity.json` from `kms init` rather than converting. | +| `[[validator]]` `protocol_version` | none | Not needed. | +| `[[providers.softsign]]` | `keys[]` with `backend: file` | See [step 2](#from-softsign). | +| `[[providers.yubihsm]]` | `keys[]` with `backend: pkcs11` | PKCS#11 HSM; see [step 2](#from-yubihsm-or-another-hsm). | +| `[[providers.ledgertm]]` | none | Not yet supported in Cosmos-KMS; see [step 2](#from-yubihsm-or-another-hsm). | + +A standard cosmos-sdk node has no peer ID on its `priv_validator_laddr` listener, so there is usually no `@` prefix to carry over, and both TMKMS and Cosmos-KMS use a bare `tcp://host:port`. + +Below is a complete `kms.yaml` example for a single validator on the file backend: + +```yaml +chains: + - id: my-chain-1 + state_file: state/my-chain-1.json +validators: + - chain_id: my-chain-1 + addr: tcp://127.0.0.1:26659 + identity_key: identity.json +keys: + - chain_ids: [my-chain-1] + backend: file + algorithm: ed25519 + key_file: priv_validator_key.json +``` + +The `keys` block shown is the file backend. Its fields differ per backend, which step 2 covers. It also needs `keys[].algorithm`, which has no `tmkms.toml` equivalent, so set it explicitly (`ed25519` for a softsign key), as the example shows. The [configuration reference](/sdk/next/kms/configuration-reference) lists every field. + +## 2. Move the consensus key into a backend + +Get the consensus key into the backend named in your `keys` block. The path depends on where TMKMS holds it today. + +### From softsign + +The softsign backend keeps the key in a file, but the format differs from what the Cosmos-KMS file backend reads. A TMKMS softsign key is the base64-encoded 32-byte ed25519 seed, while the file backend expects a CometBFT `priv_validator_key.json` or the base64-encoded 64-byte ed25519 key. Pointing `key_file` at a softsign key directly fails with `expected 64-byte ed25519 key, got 32`. + +If you still have the validator's original `priv_validator_key.json` (TMKMS softsign was imported from it), point `key_file` at that file directly. No conversion is needed. + +From only the softsign key (the file named by the `path` in your `[[providers.softsign]]` block), expand the 32-byte seed into the 64-byte `seed||pubkey` form the file backend accepts. Replace `tmkms_softsign.key` in the command with that file: + +```shell +python3 - tmkms_softsign.key converted.key <<'PY' +import base64, sys +from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey +from cryptography.hazmat.primitives.serialization import Encoding, PublicFormat +seed = base64.b64decode(open(sys.argv[1]).read().strip()) +assert len(seed) == 32, f"expected 32-byte seed, got {len(seed)}" +sk = Ed25519PrivateKey.from_private_bytes(seed) +pub = sk.public_key().public_bytes(Encoding.Raw, PublicFormat.Raw) +open(sys.argv[2], "w").write(base64.b64encode(seed + pub).decode()) +PY +``` + +Point `key_file` at `converted.key`. The command needs Python 3 with the `cryptography` package (`pip install cryptography`). + +`converted.key` and the exported softsign key are unencrypted private keys. Shred them after import (`shred -u` or `rm -P`) and keep them out of shell history and backups. + +{/* REAL TMKMS IS STILL THE 2026-07-20 RUN. The 2026-07-28 blind audit completed a full cutover using this page alone and confirmed the config map, the seed conversion, the state jq and all three error strings at shipping refs (kms v0.1.0 4bd83922ee, simd v0.55.0 64fd208a11), but tmkms was not installed on that host: its softsign key and state file were synthesized in the formats this page documents. So the run below remains the only evidence involving real tmkms and must not be restamped. The 2026-07-28 audit also found that the raw jq at step 4 silently lowers an existing double-sign floor, now covered by a Warning on the page. Full run: ~/Documents/tests/kms-docs-audit-findings.md. Original: run-verified 2026-07-20 on macOS. tmkms 0.15.0 (softsign-only), simd cosmos-sdk 46a17713 (v0.54.0), kms 0007b0d1, python cryptography 46.0.5. Verified: the config map, both key paths (original priv_validator_key.json, and the seed→64-byte conversion above which was byte-identical to the original), the state jq (step+1 remap), all three error strings, and cutover with no double sign (kms first signed at tmkms high-water +1). NOT run: the PKCS#11/HSM and AWS KMS migration paths. Full run: ~/Documents/tests/tmkms-migration-findings.md. */} + +### From YubiHSM or another HSM + +Most HSM-held keys do not move. A YubiHSM, a Fortanix device, or any HSM that exposes the key over PKCS#11 works with the Cosmos-KMS `pkcs11` backend directly: wire up a `keys` block against the same module and keep signing with the same key. Keys generated inside an HSM are typically non-exportable, which is the point of an HSM, so this no-move path is the normal case. For the PKCS#11 config, see [Configure a signing backend](/sdk/next/kms/configure-backend). + +When you are changing custodian rather than keeping the key in place, migrate by rotation: generate a new key in the target custodian and rotate the validator to it on chain, which retires the TMKMS-held key entirely. See [Rotate a consensus key, Staking](/sdk/next/keys/rotate-validator-key). + +Ledger-held consensus keys are not currently supported in Cosmos-KMS. If you need it, open a [feature request](https://github.com/cosmos/kms/issues) so demand for it can be gauged. + +## 3. Stop TMKMS + +Stop TMKMS and confirm the process is gone. The validator misses blocks until Cosmos-KMS takes over. That gap is expected: missed blocks are recoverable, a double sign from two live signers is not. Keep it short to avoid downtime jailing. + +Never run TMKMS and Cosmos-KMS at the same time against the same validator key. Each keeps its own last-signed state, so together they can sign the same height, which is a double sign. Stop one fully before starting the other, in both directions, including any rollback. + +## 4. Translate the double-sign state + +TMKMS and Cosmos-KMS both track the last signed height, round, and step per chain, and the protection only works if the new signer starts at or above the old signer's high-water mark. The two store this state differently, so it needs translating. + +Point the following command at the TMKMS state file named by `state_file` in your `[[chain]]` block (`tmkms_state.json` is a placeholder). It writes the translated state into the Cosmos-KMS location: + +```shell +mkdir -p /state +jq '{height, round: (.round|tonumber), step: (.step + 1)}' tmkms_state.json > /state/.json +``` + +The `jq` expression makes two conversions: `round` from string to number (the source of the `int32` error otherwise), and `step + 1` to remap TMKMS's 0/1/2 signing steps to CometBFT's 1/2/3. + +The redirection above overwrites whatever is at that path, so a re-run or a stale `tmkms_state.json` silently lowers the double-sign floor. Check the file does not already exist first. + +## 5. Start Cosmos-KMS + +Start Cosmos-KMS with the completed `kms.yaml`. It dials the validator and resumes signing. + +```shell +kms start --home +``` + +## 6. Confirm signing resumed + +Confirm the chain is producing again and the signer's state file is advancing. Run the status check twice; a climbing height means signing resumed: + +```shell +curl -s localhost:26657/status | jq '.result.sync_info.latest_block_height' +``` + +The signer's `/state/.json` should also advance past the last height TMKMS signed. + +## What can go wrong + +- Both signers briefly alive: the double-sign risk above. Cut over with the old process confirmed dead, not just signaled. +- The validator stays dark after cutover: the new signer is not reaching the listener. Check the `validators[].addr` translation and the firewall between the hosts. +- `dial tcp: lookup node-id@host: no such host`: the `@` prefix was left in `validators[].addr`. Drop it and keep only host and port. +- `file: parse key file "": expected 64-byte ed25519 key, got 32`: a TMKMS softsign key was pointed at directly. See the [key section](#from-softsign). +- `chain "": reload sign-state: json: cannot unmarshal string into Go value of type int32`: the TMKMS state file was reused as is. Translate it first; see [step 4](#4-translate-the-double-sign-state). + +## Next steps + +- Harden the new setup. See [Remote signing best practices](/sdk/next/kms/best-practices). +- Full field reference for the translated config. See the [configuration reference](/sdk/next/kms/configuration-reference). diff --git a/sdk/next/kms/remote-signing.mdx b/sdk/next/kms/remote-signing.mdx new file mode 100644 index 000000000..6c5e0313e --- /dev/null +++ b/sdk/next/kms/remote-signing.mdx @@ -0,0 +1,59 @@ +--- +noindex: true +title: "Overview" +description: "What privval is, what Cosmos-KMS is and is not, and why validator keys belong in a KMS or HSM rather than local files." +--- + +This page covers remote signing for validators and Cosmos-KMS, the Cosmos stack's remote signer. The guides in this section set both up. + +## What is remote signing? + +Remote signing splits a validator into two processes: a node that participates in consensus, and a signer that holds the consensus key and produces signatures on the node's behalf. The node handles blocks, gossip, and peers; the signer holds the one secret that matters. + +Separating the two keeps signing secure and makes the node replaceable. The key moves off the node's exposed filesystem into a hardware security module (HSM) or a cloud key service, where the people and automation that maintain the node never touch it. And because the node holds no secrets, it can be rebuilt, upgraded, or replaced at will. Once it is replaced, the signer reconnects and signing continues. + +CometBFT supports this through privval, the seam between the node and whatever holds the key. With `priv_validator_laddr` set in `config.toml`, the node signs nothing locally: it listens for a signer connection and sends each vote and proposal out for signature. The wire protocol carries four requests: sign this vote, sign this proposal, return the public key, and a keepalive ping. The node does not necessarily care what is on the other end. + +```mermaid +sequenceDiagram + participant N as Validator node + participant S as Cosmos-KMS signer + participant B as Backend (file / PKCS11 / AWS KMS) + N->>S: vote or proposal to sign (privval) + S->>B: sign with consensus key + B-->>S: signature + S-->>N: signature +``` + +The node forwards each vote or proposal to the signer over the privval connection. The signer has its backend produce the signature and returns only that, so the consensus key never leaves the backend. + +## Cosmos-KMS + +The [`cosmos-kms`](https://github.com/cosmos/kms) repo is a remote signer for CometBFT, written in Go. It implements the signer side of privval and answers a node's signing requests from a backend that holds the key. One signer process can sign for multiple chains, with each chain backed by exactly one key, and can hold connections to multiple nodes per chain. It is built for operators who must keep validator keys in real custody, an HSM or a cloud key service, rather than in files on chain infrastructure. Three backends are supported: + +- File: a key file on the signer's disk, for development and testing, not production custody. +- PKCS#11: a hardware security module, signing on-device through the standard HSM interface. +- AWS KMS: a key that never leaves AWS KMS, using standard AWS credentials and IAM. + +Cosmos-KMS is designed to work with existing validator infrastructure, and validators should prefer it over previous remote-signing solutions. If you run [TMKMS](https://github.com/iqlusioninc/tmkms) today, see [Migrate from TMKMS](/sdk/next/kms/migrate-from-tmkms). + +Cosmos-KMS speaks the CometBFT privval protocol, so it can sign for any node that implements that protocol. + +## How it works + +A running signer comes down to five moving parts: + +- Configuration: one file, `kms.yaml`, in three blocks: the chains it signs for, the validators it dials, and the keys binding each chain to exactly one backend. The `kms init` command scaffolds it; `kms start` serves until stopped. +- Signing: requests travel the privval connection, the backend signs in place, on disk, on the HSM, or inside AWS KMS, and only the signature returns. The private key never crosses the wire. +- Connection: the signer dials out, so the signing host needs no inbound ports. The address scheme selects the transport: `tcp://` uses CometBFT's SecretConnection, and `noise://` adds mutual peer pinning, where each side refuses any connection from an unexpected peer. +- Key types: over privval, the signer signs `ed25519`, `secp256k1eth`, and post-quantum `mldsa65`, plus `secp256k1` on the AWS KMS backend; the gRPC signer service signs `ed25519`, `secp256k1eth`, and `secp256k1`, but not `mldsa65`, which has no gRPC signature scheme and is privval-only. For per-backend support, see [Configure a signing backend](/sdk/next/kms/configure-backend). +- Double-sign protection: a per-chain last-signed state file refuses anything at or below a height, round, and step already signed. The protection lives with the key, so even a misbehaving or duplicated validator node cannot force a double sign. Because this protection is per state file, it is recommended never to run two signers for the same key. For a backup, connect a single signer to more than one validator node. + +## Next steps + +- Run a remote signer against a local chain. See [Remote signing tutorial](/sdk/next/kms/tutorial-file-backend). +- Move the key into real custody, AWS KMS or an HSM. See [Configure a signing backend](/sdk/next/kms/configure-backend). +- Harden the signer's placement and transport. See [Remote signing best practices](/sdk/next/kms/best-practices). +- Look up any `kms.yaml` field. See the [configuration reference](/sdk/next/kms/configuration-reference). +- Understand which key the signer holds and how it rotates. See [Key rotation](/sdk/next/keys/key-rotation). +- Understand post-quantum consensus keys and their costs. See [Post-quantum keys](/sdk/next/keys/post-quantum-keys). diff --git a/sdk/next/kms/rotate-key-remote-signer.mdx b/sdk/next/kms/rotate-key-remote-signer.mdx new file mode 100644 index 000000000..08f274f87 --- /dev/null +++ b/sdk/next/kms/rotate-key-remote-signer.mdx @@ -0,0 +1,180 @@ +--- +noindex: true +title: "Rotate a consensus key held in Cosmos-KMS" +description: "Rotate a validator's consensus key when the current key lives in a remote signer: new key in the backend, a second node and signer, rotate, cut over." +--- + +This tutorial rotates a staked validator's consensus key when Cosmos-KMS holds the current key rather than a local file. It continues the [remote signing tutorial](/sdk/next/kms/tutorial-file-backend) and reuses that setup: the single-node chain `kms-demo-1`, with node home `~/.kms-demo-node` (RPC on port 26657, privval listener on port 26659) and signer home `~/.kms-demo`, whose signer dials that listener. + +The rotation is the standard zero-downtime rotation with three differences. The new key is generated in a signing backend. The second node gets its own signer process. The new public key comes from the second node's RPC instead of a key file. For the standard procedure and the rotation rules, see [Rotate a consensus key, Staking](/sdk/next/keys/rotate-validator-key). + +Key rotation can introduce security implications for your chain. Read the [Key rotation](/sdk/next/keys/key-rotation) overview in its entirety before proceeding. + +Commands use `simd` as the chain binary and `val` as the validator key name. Substitute your own for a real chain. The live pair and the new pair run on one host, so the second node and its new signer take spare ports: the second node listens for its signer on port 26669 and serves RPC on port 26667. + +{/* +CURRENT VERIFICATION: blind execution audit 2026-07-28 against shipping refs, kms v0.1.0 (4bd83922ee), cometbft v0.40.0 (0880b4d378), simd v0.55.0 (64fd208a11), macOS 26.5.2 arm64. Verdict PASS, the only page in the KMS set to pass outright: every command ran verbatim on the first attempt and every checkpoint was observable. Rotation tx at height 39; old key served heights 39 and 40, new key from 41; CometBFT stopped requesting signatures from the old pair on its own; the chain kept producing on the new pair alone after both old processes were stopped. Full run: ~/Documents/tests/kms-docs-audit-findings.md. +Earlier run, retained for provenance: verified 2026-07-15 on Darwin 25.5.0 (arm64), Go 1.26.5, with simd from cosmos-sdk @ b9a11304cf and kms @ 538e5c5, which predates the fail-closed sign-state change in kms v0.1.0. +Base setup: the remote signing tutorial's single-validator localnet signing through kms (file backend), 1s timeout_commit, 0stake min gas. +Every command below was run verbatim. Checkpoints observed: second node caught up with zero signatures from the new signer; +rotation tx executed (code 0, 1000000stake fee burned); set swapped to the new key with no overlap; old signer stopped on its own; chain kept producing on the new pair alone. + +*/} + +## Prerequisites + +- A validator already signing through Cosmos-KMS, from the [remote signing tutorial](/sdk/next/kms/tutorial-file-backend), still running. +- All prerequisites of the standard rotation: Cosmos SDK 0.55 and CometBFT 0.40 or later, the target key type in consensus params, no rotation in the current unbonding period, and funds for the burned rotation fee plus gas. See [Rotate a consensus key, Staking](/sdk/next/keys/rotate-validator-key). +- [jq](https://jqlang.org/) and [curl](https://curl.se/), used to derive the new public key. +- Spare ports on the host for the second node and its signer. +- The chain's binary; the examples use `simd`. To build it and run a node, see [Run a node](/sdk/next/node/run-node). + +## 1. Generate the new key in a backend + +For an HSM or AWS KMS, generate the key with the backend's own tooling. The commands are in [Configure a signing backend](/sdk/next/kms/configure-backend). This guide uses the file backend, so a scratch `simd init` generates a fresh key file. + +```shell +simd init scratch --chain-id kms-demo-1 --home ~/.kms-demo-scratch +``` + +For a post-quantum `mldsa65` target on the file backend, add `--consensus-key-algo ml_dsa_65` to the `simd init` command above and set `algorithm: mldsa65` in the signer's `keys` block in step 2. The PKCS#11 and AWS KMS backends also sign `mldsa65`; see [Configure a signing backend](/sdk/next/kms/configure-backend) for generating the key on those. + +The new key is `~/.kms-demo-scratch/config/priv_validator_key.json`. The rest of the scratch home is disposable. + +The new key must be freshly generated. Reusing key material any signer has signed with risks a double sign, which tombstones the validator. + +## 2. Configure a second signer with the new key + +One Cosmos-KMS process cannot hold two keys for the same chain. The new key must run in its own process, with its own home and double-sign state: + +```shell +kms init --home ~/.kms-demo2 +``` + +Copy the new key to the second signer's home: + +```shell +cp ~/.kms-demo-scratch/config/priv_validator_key.json ~/.kms-demo2/priv_validator_key.json +``` + +Replace the contents of `~/.kms-demo2/kms.yaml` with the following. The `addr` is the port the second node listens on for its signer. For an HSM or AWS KMS key, the `keys` block instead binds the backend entry from step 1: + +```yaml +chains: + - id: kms-demo-1 + +validators: + - chain_id: kms-demo-1 + addr: tcp://127.0.0.1:26669 + identity_key: identity.json + +keys: + - chain_ids: [kms-demo-1] + backend: file + algorithm: ed25519 + key_file: priv_validator_key.json +``` + +For a post-quantum `mldsa65` key, set `algorithm: mldsa65` in the signer's `keys` block above. + +Do not start the signer yet. It starts in step 3, right before the second node, so its connection retries are still fast when the node comes up. + +## 3. Bring up the second node and its signer + +Initialize a fresh node home and give it the chain's genesis: + +On a real chain with history, a fresh node takes days to sync from genesis. Use state sync or a snapshot to reach the chain head quickly. See [State sync](/sdk/next/node/run-node#state-sync) for more info. + +```shell +simd init shadow --chain-id kms-demo-1 --home ~/.kms-demo-node2 +``` + +```shell +cp ~/.kms-demo-node/config/genesis.json ~/.kms-demo-node2/config/genesis.json +``` + +With the prep done, start the new signer and the second node in quick succession. In one terminal, start the signer. It logs `dial failed` and retries until the node exists: + +```shell +kms start --home ~/.kms-demo2 --allow-fresh-state kms-demo-1 +``` + +The new key has never signed on this chain, and this signer home is new, so it holds no sign-state file. `--allow-fresh-state` writes the height-0 double-sign floor so the signer will start. Without it the signer exits with `sign-state file ... is missing or empty; refusing to start at height 0`. + +Use `--allow-fresh-state` only here, for the new key's first start. Do not add it to the live signer or to any later start of this one. Once a state file exists the flag has no effect, so it fails quietly, but it means a lost or truncated state file resets the double-sign floor to zero rather than stopping the signer. After the cutover in step 5, run the new signer with the bare command. + +Right away, in a second terminal, start the second node. The flags point it at the new signer on port 26669. Move its listeners off the live node's ports, and peer it with the live node so it syncs: + +```shell +simd start --home ~/.kms-demo-node2 \ + --priv_validator_laddr tcp://127.0.0.1:26669 \ + --p2p.laddr tcp://0.0.0.0:26666 \ + --rpc.laddr tcp://127.0.0.1:26667 \ + --grpc.address localhost:9092 \ + --proxy_app tcp://127.0.0.1:26668 \ + --rpc.pprof_laddr localhost:6061 \ + --p2p.persistent_peers "$(simd comet show-node-id --home ~/.kms-demo-node)@127.0.0.1:26656" +``` + +If the second node exits with `can't get pubkey: ... endpoint connection timed out`, the signer has backed off to slow retries. Restart the signer, then start the node again. + +Confirm the second node has caught up. The value is `false` once it is synced: + +```shell +curl -s localhost:26667/status | jq '.result.sync_info.catching_up' +``` + +Until the rotation applies, the second node follows the chain without signing. Its signer logs `served pubkey request` but no signatures. That is correct. + +## 4. Derive the new public key and rotate + +The second node fetched its key from the new signer at startup and reports it at `/status`. Read it and reformat it into the proto-JSON the rotation command accepts: + +```shell +PK=$(curl -s localhost:26667/status | jq -c '{"@type":"/cosmos.crypto.ed25519.PubKey", key: .result.validator_info.pub_key.value}') +``` + +For an ML-DSA key, the proto type differs: use `"@type":"/cosmos.crypto.mldsa65.PubKey"` in the jq expression instead. + +Submit the rotation from the operator account and capture the transaction hash: + +```shell +TXHASH=$(simd tx staking rotate-cons-pub-key "$PK" --from val --keyring-backend test --home ~/.kms-demo-node --chain-id kms-demo-1 --node tcp://localhost:26657 --gas auto --gas-adjustment 1.5 --fees 2000stake --yes --output json | jq -r .txhash) +``` + +After the transaction lands in a block, confirm the code is `0`: + +```shell +simd query tx "$TXHASH" --node tcp://localhost:26657 --output json | jq '{height: .height, code: .code}' +``` + +The burned rotation fee is charged separately from the gas fee above. + +## 5. Verify and retire the old pair + +The validator set swaps to the new key atomically two heights after execution. Confirm the set carries only the new key. The value matches `$PK`, and the old key is gone: + +```shell +curl -s localhost:26657/validators | jq -r '.result.validators[].pub_key.value' +``` + +At the swap, CometBFT stops requesting signatures from the old pair on its own. Stop the live node (`~/.kms-demo-node`) and the live signer (`~/.kms-demo`) with Ctrl-C in their terminals. + +Confirm blocks keep flowing through the new pair. Run this twice a few seconds apart and watch the height climb: + +```shell +curl -s localhost:26667/status | jq '.result.sync_info.latest_block_height' +``` + +Keep the old key in its backend until the unbonding period ends. The validator remains slashable for its past behavior until then. + +## What can go wrong + +- The new signer exits with `app: multiple signers bound to chain`: both keys are in one `kms.yaml`. Run the new key in its own process. +- The validator is jailed after the rotation and neither signer is signing: the rotation targeted the stray local key. Rotate to the key from the second node's `/status`, once the unbonding window allows it. +- The transaction is rejected: the standard failure modes apply, including the rotation limit and unsupported key types. See [Rotate a consensus key, Staking](/sdk/next/keys/rotate-validator-key). + +## Next steps + +- Harden the new signer's placement and transport. See [Remote signing best practices](/sdk/next/kms/best-practices). +- Rotate to a post-quantum key with the same procedure. See [Migrate a validator to ML-DSA](/sdk/next/keys/migrate-validator-ml-dsa). diff --git a/sdk/next/kms/tutorial-file-backend.mdx b/sdk/next/kms/tutorial-file-backend.mdx new file mode 100644 index 000000000..cc85f5f19 --- /dev/null +++ b/sdk/next/kms/tutorial-file-backend.mdx @@ -0,0 +1,172 @@ +--- +noindex: true +title: "Remote signing tutorial" +description: "Tutorial: stand up a local chain whose validator signs through Cosmos-KMS, using the file backend." +--- + +This tutorial builds a working remote signer from scratch: a single-node local chain where a Cosmos-KMS process signs the votes instead of the node itself. It uses the file backend, which needs no HSM or cloud account and exists for exactly this kind of learning setup. At the end, you stop the signer and watch the chain stall, which proves where signing really happens. This tutorial uses one node, one signer, and one key. + +Commands use `simd` for the chain binary. The node home is `~/.kms-demo-node` and the signer home is `~/.kms-demo`. + +{/* +CURRENT VERIFICATION: blind execution audit 2026-07-28 against shipping refs, kms v0.1.0 (4bd83922ee), cometbft v0.40.0 (0880b4d378), simd v0.55.0 (64fd208a11), go1.26.0, macOS 26.5.2 arm64. Verdict PARTIAL: all three checkpoints pass (no blocks before the signer; height climbs once the node connects; production stalls when kms stops and resumes on restart), and the state file is confirmed written at signer start rather than at first signature. The one blocker found was the step-8 timing cliff, since fixed on this page. Full run: ~/Documents/tests/kms-docs-audit-findings.md. +Note that kms v0.1.0 fails closed on a missing sign-state file (PR #35, c952fac), which the pre-2026-07-28 runs below predate: they were verified against kms commits where a bare `kms start` still worked on a fresh home. Any future re-verification must use a ref at or after v0.1.0. +Earlier runs, retained for provenance: verified 2026-07-14 on Darwin 25.5.0 (arm64), Go 1.26.5, with simd from cosmos-sdk main @ b9a11304cf (reported 0.0.0-dev; 0.55 not tagged then) and kms @ 7932ceb. +Step 8 startup timing re-verified 2026-07-20 on a clean single-localnet machine (simd @ 46a17713, kms @ 0007b0d): node started within ~5s of the signer passes 5/5; longer gaps hit dead zones (~6-9s, ~13-21s) and time out with `can't get pubkey`. Re-verified again 2026-07-21 (kms-tutorial-reverify-findings.md): a plain node rerun is unreliable (~30%), worsened on a node's first boot by a one-time IAVL storage upgrade that delays its listener; restarting the signer (resets its backoff to fast dials) reliably recovers. Root cause: kms dial backoff (200ms→10s cap, internal/manager) vs the node's single ~3s pubkey fetch (cometbft node/setup.go:736). Note updated to the restart-the-signer recovery. Fix flagged to eng (lower kms defaultBackoffMax to ~1s). Full runs: ~/Documents/tests/kms-tutorial-step8-reverify-findings.md and kms-tutorial-reverify-findings.md. +*/} + +## Prerequisites + +- [Go](https://go.dev/doc/install) 1.26 or later, [make](https://www.gnu.org/software/make/), [git](https://git-scm.com/), [jq](https://jqlang.org/), and [curl](https://curl.se/). +- A chain binary at Cosmos SDK 0.55 or later. The tutorial uses `simd`, built with `make install` in the [cosmos-sdk repo](https://github.com/cosmos/cosmos-sdk). To build it and run a node, see [Run a node](/sdk/next/node/run-node). + +## 1. Install Cosmos-KMS + +Clone and install the signer: + +```shell +git clone https://github.com/cosmos/kms +``` + +```shell +cd kms && make install +``` + +Confirm the binary works: + +```shell +kms version +``` + +## 2. Create a single-node chain + +Set up a fresh chain home with one validator. Do not start the node yet: + +```shell +# Initialize the node home with the chain ID +simd init signer-demo --chain-id kms-demo-1 --home ~/.kms-demo-node + +# Create the validator key in the test keyring +simd keys add val --keyring-backend test --home ~/.kms-demo-node + +# Fund the validator account in genesis +simd genesis add-genesis-account val 1000000000stake --keyring-backend test --home ~/.kms-demo-node + +# Register the validator with a genesis staking transaction +simd genesis gentx val 500000000stake --chain-id kms-demo-1 --keyring-backend test --home ~/.kms-demo-node + +# Collect the gentx into the genesis file +simd genesis collect-gentxs --home ~/.kms-demo-node +``` + +## 3. Initialize the signer + +Scaffold the signer's home. This writes a stub `kms.yaml` and generates `identity.json`, the key the signer uses to authenticate its connection: + +```shell +kms init --home ~/.kms-demo +``` + +The command prints `initialized kms in /Users/you/.kms-demo`. Pass the `--home` flag on every `kms` command. Without it, the signer uses the current directory. + +## 4. Give the signer the consensus key + +Copy the consensus key that `simd init` generated into the signer's home: + +```shell +cp ~/.kms-demo-node/config/priv_validator_key.json ~/.kms-demo/priv_validator_key.json +``` + +Once the node is configured for remote signing, it never reads its local key file again. In production, move the key instead of copying it. Note that the node regenerates a fresh, unused consensus key file if it finds none, so moving the key reduces what is on the node host rather than leaving it key-free. For this tutorial, the copy keeps things simple. + +## 5. Configure the signer + +Replace the contents of `~/.kms-demo/kms.yaml` with: + +```yaml +chains: + - id: kms-demo-1 + +validators: + - chain_id: kms-demo-1 + addr: tcp://127.0.0.1:26659 + identity_key: identity.json + +keys: + - chain_ids: [kms-demo-1] + backend: file + algorithm: ed25519 + key_file: priv_validator_key.json +``` + +The three blocks say: sign for the chain `kms-demo-1`, dial its node at port 26659, and read the copied key file as an `ed25519` key. The file backend has no default algorithm, so the `algorithm` line is required. Relative paths resolve against the signer's home. + +## 6. Point the node at the signer + +Open `~/.kms-demo-node/config/config.toml`, find the `priv_validator_laddr` line, and set it: + +```toml +priv_validator_laddr = "tcp://127.0.0.1:26659" +``` + +With this set, the node signs nothing locally. It listens on that port for a signer connection and forwards every vote and proposal to it. + +## 7. Start the signer + +The node needs its signer available the moment it starts, so bring the signer up first: + +```shell +kms start --home ~/.kms-demo --allow-fresh-state kms-demo-1 +``` + +This validator has never signed on `kms-demo-1`, so no sign-state file exists yet. `--allow-fresh-state` writes the height-0 double-sign floor on this first start. Without it the signer refuses to start rather than risk re-signing a height it cannot prove it has already passed. + +Pass `--allow-fresh-state` only on a first start, and only for a chain the key has never signed on. It will not overwrite an existing floor, but leaving it in a service definition means a deleted or truncated state file resets the floor to zero instead of stopping the signer. Later starts in this tutorial use the bare command. + +To seed the floor as a separate step instead, run `kms state init --chain kms-demo-1 --height 0 --home ~/.kms-demo` and then start the signer with no flag. + +The signer logs `kms started` and dials the node. The node is not running yet, so the signer logs `dial failed; backing off` and keeps retrying. That is expected. Leave it running. + +## 8. Start the node + +In a second terminal, start the node within five seconds of starting the signer: + +```shell +simd start --home ~/.kms-demo-node +``` + +If `simd start` exits with `can't get pubkey: ... endpoint connection timed out`, the signer has backed off to slow retries. Restart the signer, then start the node again within five seconds. + +The node opens its private-validator listener on port 26659. The signer's next dial connects, the node fetches its consensus public key from the signer, and block production begins. Confirm the height is climbing: + +```shell +curl -s localhost:26657/status | jq '.result.sync_info.latest_block_height' +``` + +Also confirm the signer's double-sign protection state file is in place, written when the signer started: + +```shell +ls ~/.kms-demo/state/kms-demo-1.json +``` + +## 9. Prove the signer is doing the signing + +Stop the signer with Ctrl-C and watch the node's logs. Block production stalls because the validator can no longer sign. Start the signer again, this time with no `--allow-fresh-state`, because the state file now exists and carries the highest height signed so far: + +```shell +kms start --home ~/.kms-demo +``` + +The signer reconnects and the chain resumes. The node never touches a private key. Every signature comes from the signer. + +## What you built + +A validator whose consensus key lives outside the node. The node handles consensus and networking. The signer holds the key and signs, and the double-sign state file travels with it. The file backend keeps this tutorial self-contained, but it holds the key in plaintext on disk and is not production custody. The production version of this setup swaps one config block to move the key into an HSM or AWS KMS. + +## Next steps + +- Swap the file backend for real custody, AWS KMS or an HSM. See [Configure a signing backend](/sdk/next/kms/configure-backend). +- Harden the signer's placement and transport. See [Remote signing best practices](/sdk/next/kms/best-practices). +- Understand the architecture you just ran. See [Cosmos-KMS and remote signing](/sdk/next/kms/remote-signing). + +- Look up any config field. See the [configuration reference](/sdk/next/kms/configuration-reference). diff --git a/sdk/next/learn.mdx b/sdk/next/learn.mdx index 1ae17ead5..ba4f18857 100644 --- a/sdk/next/learn.mdx +++ b/sdk/next/learn.mdx @@ -1,7 +1,7 @@ --- noindex: true title: "Cosmos SDK Docs" -description: "Version: v0.54" +description: "Version: v0.55" --- The Cosmos SDK is the most widely adopted, battle-tested Layer 1 blockchain stack, trusted by 200+ chains live in production. This modular framework enables you to build secure, high-performance blockchains with comprehensive guides covering everything from core concepts to advanced implementation patterns. diff --git a/sdk/next/learn/concepts/accounts.mdx b/sdk/next/learn/concepts/accounts.mdx index 2bcac5afa..97e6530a4 100644 --- a/sdk/next/learn/concepts/accounts.mdx +++ b/sdk/next/learn/concepts/accounts.mdx @@ -12,7 +12,7 @@ Every account is controlled by a cryptographic keypair derived from a seed phras ## What is an account -An account is an on-chain identity used to authorize transactions. Each account stores an address, a public key, an account number, and a sequence number, as defined by [`BaseAccount`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/x/auth/types/auth.pb.go#L32) in the `x/auth` module: +An account is an on-chain identity used to authorize transactions. Each account stores an address, a public key, an account number, and a sequence number, as defined by [`BaseAccount`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/x/auth/types/auth.pb.go#L32) in the `x/auth` module: ```go type BaseAccount struct { diff --git a/sdk/next/learn/concepts/baseapp.mdx b/sdk/next/learn/concepts/baseapp.mdx index 41859d2e2..80fdd5272 100644 --- a/sdk/next/learn/concepts/baseapp.mdx +++ b/sdk/next/learn/concepts/baseapp.mdx @@ -28,7 +28,7 @@ CometBFT drives the block lifecycle by calling ABCI methods on `BaseApp`. `BaseA ## Key fields -[`BaseApp`](https://github.com/cosmos/cosmos-sdk/blob/main/baseapp/baseapp.go) is defined in `baseapp/baseapp.go`. It holds references to everything needed to run a chain: +[`BaseApp`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/baseapp/baseapp.go#L63-L168) is defined in `baseapp/baseapp.go`. It holds references to everything needed to run a chain: ```go type BaseApp struct { @@ -51,7 +51,7 @@ type BaseApp struct { } ``` -For a complete list of fields, see the [`BaseApp` struct definition](https://github.com/cosmos/cosmos-sdk/blob/main/baseapp/baseapp.go). +For a complete list of fields, see the [`BaseApp` struct definition](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/baseapp/baseapp.go#L63-L168). - `cms` (CommitMultiStore): the root state store. All module substores are mounted here, and all state reads and writes during block execution pass through it. - `storeLoader`: a function that opens and mounts the individual module stores at application startup. @@ -115,7 +115,7 @@ For the application wiring side, including `SetAnteHandler`, `HandlerOptions`, a If the `AnteHandler` fails, the transaction is rejected and its messages never execute. If the `AnteHandler` succeeds but a message later fails, the `AnteHandler`'s state writes, such as fee deduction and sequence increment for ordered transactions, are already flushed to `finalizeBlockState` and will be committed with the block. Fees are charged even for transactions whose messages fail. -`BaseApp.runTx()` also handles Go panics that occur during execution — for example, when a keeper encounters an invalid state. By default, panics are caught and logged as errors. Applications can register custom panic recovery logic via `BaseApp.AddRunTxRecoveryHandler`, which adds a `RecoveryHandler` to the chain. See [ADR-022](/sdk/next/reference/architecture/adr-022-custom-panic-handling) and [`baseapp/recovery.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/baseapp/recovery.go) for details. +`BaseApp.runTx()` also handles Go panics that occur during execution — for example, when a keeper encounters an invalid state. By default, panics are caught and logged as errors. Applications can register custom panic recovery logic via `BaseApp.AddRunTxRecoveryHandler`, which adds a `RecoveryHandler` to the chain. See [ADR-022](/sdk/next/reference/architecture/adr-022-custom-panic-handling) and [`baseapp/recovery.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/baseapp/recovery.go) for details. ## Message routing diff --git a/sdk/next/learn/concepts/cli-grpc-rest.mdx b/sdk/next/learn/concepts/cli-grpc-rest.mdx index e8bdff14c..9a9508b0d 100644 --- a/sdk/next/learn/concepts/cli-grpc-rest.mdx +++ b/sdk/next/learn/concepts/cli-grpc-rest.mdx @@ -240,7 +240,7 @@ api.enable = true api.swagger = true ``` -To generate Swagger documentation for your own custom modules, see the [`proto-swagger-gen` script](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/scripts/protoc-swagger-gen.sh) in the Cosmos SDK. +To generate Swagger documentation for your own custom modules, see the [`proto-swagger-gen` script](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/scripts/protoc-swagger-gen.sh) in the Cosmos SDK. ## CometBFT RPC diff --git a/sdk/next/learn/concepts/context-gas-events.mdx b/sdk/next/learn/concepts/context-gas-events.mdx index e976870e9..9f6f94e47 100644 --- a/sdk/next/learn/concepts/context-gas-events.mdx +++ b/sdk/next/learn/concepts/context-gas-events.mdx @@ -10,7 +10,7 @@ In the previous section, [Encoding and Protobuf](/sdk/next/learn/concepts/encodi Every message handler, keeper method, and block hook in the Cosmos SDK receives an `sdk.Context`. It is the execution environment for a single unit of work (a transaction, a query, or a block hook) and carries everything that code needs to read state, emit events, and consume gas. Rather than passing the store, gas meter, and block header as separate arguments to every function, `Context` bundles them into a single value. -The `Context` struct is defined in [`types/context.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/types/context.go): +The `Context` struct is defined in [`types/context.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/types/context.go): ```go type Context struct { @@ -27,7 +27,7 @@ Context is a value type. It is passed by value and mutated through `With*` metho ### Block metadata -Context exposes read-only access to the current block's metadata (see [`types/context.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/types/context.go)): +Context exposes read-only access to the current block's metadata (see [`types/context.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/types/context.go)): - `ctx.BlockHeight()` returns the current block number. - `ctx.BlockTime()` returns the block's timestamp. @@ -36,7 +36,7 @@ Context exposes read-only access to the current block's metadata (see [`types/co These values are populated by [`BaseApp`](/sdk/next/learn/concepts/baseapp) from the block header provided by CometBFT before any block logic runs. Modules read them to implement time-dependent logic (for example, checking whether a vesting period has elapsed) or to tag events with the block height. -`ctx.IsCheckTx()` returns true when the context is being used for mempool validation rather than block execution. For finer-grained branching, `ctx.ExecMode()` returns the precise execution mode: `ExecModeCheck`, `ExecModeReCheck`, `ExecModeSimulate`, `ExecModePrepareProposal`, `ExecModeProcessProposal`, `ExecModeFinalize`, and others (see [`types/context.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/types/context.go#L21) for more details). Modules that need to behave differently during simulation or proposal handling use `ExecMode()` instead of `IsCheckTx()`. +`ctx.IsCheckTx()` returns true when the context is being used for mempool validation rather than block execution. For finer-grained branching, `ctx.ExecMode()` returns the precise execution mode: `ExecModeCheck`, `ExecModeReCheck`, `ExecModeSimulate`, `ExecModePrepareProposal`, `ExecModeProcessProposal`, `ExecModeFinalize`, and others (see [`types/context.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/types/context.go#L21) for more details). Modules that need to behave differently during simulation or proposal handling use `ExecMode()` instead of `IsCheckTx()`. ### Context and state access @@ -52,7 +52,7 @@ The keeper does not hold a direct reference to the live multistore; it opens its ### Atomic sub-execution with `CacheContext` -Modules that need to attempt a sub-operation and revert it on failure can call [`ctx.CacheContext()`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/types/context.go#L412), which returns a branched copy of the context and a `writeCache` function. All state changes in the sub-operation go into the branch. Calling `writeCache()` flushes them to the parent context; not calling it discards them atomically. +Modules that need to attempt a sub-operation and revert it on failure can call [`ctx.CacheContext()`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/types/context.go#L412), which returns a branched copy of the context and a `writeCache` function. All state changes in the sub-operation go into the branch. Calling `writeCache()` flushes them to the parent context; not calling it discards them atomically. ```go cacheCtx, writeCache := ctx.CacheContext() @@ -74,7 +74,7 @@ The gas system exists to prevent abuse. Without a gas limit, a single transactio Every transaction specifies a gas limit in its `auth_info.fee.gas_limit` field. When `BaseApp` begins executing a transaction, it creates a `GasMeter` initialized with that limit and attaches it to the context. -The [`GasMeter`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/store/types/gas.go#L42) interface provides two key methods: +The [`GasMeter`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/store/types/gas.go#L42) interface provides two key methods: ```go type GasMeter interface { @@ -90,7 +90,7 @@ When submitting a transaction, users specify two of the three values `fees`, `ga ### How gas is consumed -Gas is consumed automatically at the store layer. Every read and write through the [`GasKVStore`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/store/gaskv/store.go#L12) wrapper charges gas before delegating to the underlying store: +Gas is consumed automatically at the store layer. Every read and write through the [`GasKVStore`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/store/gaskv/store.go#L12) wrapper charges gas before delegating to the underlying store: - A `Get` (store read) charges a flat read cost plus a per-byte cost for the key and value. - A `Set` (store write) charges a flat write cost plus a per-byte cost for the key and value. @@ -115,7 +115,7 @@ Events are not part of consensus state. They are not stored in the KVStore, do n ### EventManager -Modules emit events through the [`EventManager`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/types/events.go#L25), which is attached to the context. +Modules emit events through the [`EventManager`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/types/events.go#L25), which is attached to the context. The `EventManager` is created fresh for each transaction and collects all events emitted during that execution. @@ -127,11 +127,11 @@ The SDK automatically emits a `message` event for every transaction, with these - `message.module` — the module name, derived from the type URL - `message.sender` — the signer address, if present -These are defined as constants in [`types/events.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/types/events.go#L249-L263). Modules follow the same convention when emitting their own events. +These are defined as constants in [`types/events.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/types/events.go#L249-L263). Modules follow the same convention when emitting their own events. ### Emitting events -Modules emit events using [`EmitEvent`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/types/events.go#L35) or [`EmitTypedEvent`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/types/events.go#L58): +Modules emit events using [`EmitEvent`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/types/events.go#L35) or [`EmitTypedEvent`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/types/events.go#L58): ```go // emit an untyped event diff --git a/sdk/next/learn/concepts/encoding.mdx b/sdk/next/learn/concepts/encoding.mdx index 339dbdcec..6f87846cd 100644 --- a/sdk/next/learn/concepts/encoding.mdx +++ b/sdk/next/learn/concepts/encoding.mdx @@ -28,7 +28,7 @@ The Cosmos SDK uses protobuf for a fundamental reason: consensus requires determ Every validator in the network independently executes each block. After execution, each validator computes the [app hash](/sdk/next/learn/concepts/store#app-hash), a cryptographic hash of the application state. For validators to agree on the app hash, they must all produce exactly the same bytes for every piece of state they write. -Protobuf alone does not guarantee this. The Cosmos SDK uses protobuf **with additional deterministic encoding rules** formalized in [ADR-027 (Deterministic Protobuf Serialization)](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/docs/architecture/adr-027-deterministic-protobuf-serialization.md). ADR-027 specifies constraints such as requiring fields to appear in ascending field-number order and varint encodings to be as short as possible. The SDK validates incoming transactions against these rules before processing them, so a non-deterministically encoded transaction is rejected rather than producing divergent state. Every validator encoding the same data under these rules produces an identical byte sequence. +Protobuf alone does not guarantee this. The Cosmos SDK uses protobuf **with additional deterministic encoding rules** formalized in [ADR-027 (Deterministic Protobuf Serialization)](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/docs/architecture/adr-027-deterministic-protobuf-serialization.md). ADR-027 specifies constraints such as requiring fields to appear in ascending field-number order and varint encodings to be as short as possible. The SDK validates incoming transactions against these rules before processing them, so a non-deterministically encoded transaction is rejected rather than producing divergent state. Every validator encoding the same data under these rules produces an identical byte sequence. Beyond determinism, protobuf provides: @@ -59,7 +59,7 @@ Note: genesis data is distributed as JSON in `genesis.json`, but during chain in ## Transaction encoding -Transactions are protobuf messages defined in [`cosmos.tx.v1beta1`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/proto/cosmos/tx/v1beta1/tx.proto). A transaction is composed of three parts: +Transactions are protobuf messages defined in [`cosmos.tx.v1beta1`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/tx/v1beta1/tx.proto). A transaction is composed of three parts: ```text Tx @@ -107,11 +107,9 @@ A **sign mode** determines what bytes a signer commits to when signing a transac - `SIGN_MODE_LEGACY_AMINO_JSON`: the signer signs over an Amino JSON-encoded `StdSignDoc` instead of the protobuf `SignDoc`. This exists for backward compatibility with hardware wallets (e.g., older Ledger firmware) and client tooling that predates protobuf. New modules and chains should not depend on it. -- `SIGN_MODE_TEXTUAL`: the signer signs over a human-readable CBOR-encoded representation of the transaction, designed to display legibly on hardware wallet screens (introduced in v0.50, see [ADR-050](/sdk/next/reference/architecture/adr-050-sign-mode-textual)). This is the SDK's newer direction for human-readable signing on hardware wallets, intended to replace `SIGN_MODE_LEGACY_AMINO_JSON` over time. Its specification is versioned and has evolved across SDK releases. - - `SIGN_MODE_DIRECT_AUX`: allows N-1 signers in a multi-signer transaction to sign over only `TxBody` and their own `SignerInfo`, without specifying fees. The designated fee payer signs last using `SIGN_MODE_DIRECT`. This simplifies multi-signature UX. -The sign mode is negotiated at transaction construction time and does not affect how state is stored or how validators execute transactions. It only affects what bytes are signed. The full list of sign modes is defined in [`signing.proto`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/proto/cosmos/tx/signing/v1beta1/signing.proto#L17). +The sign mode is negotiated at transaction construction time and does not affect how state is stored or how validators execute transactions. It only affects what bytes are signed. The full list of sign modes is defined in [`signing.proto`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/tx/signing/v1beta1/signing.proto#L17). **For module developers:** `SIGN_MODE_DIRECT` requires no extra work. If you want your module's messages to be signable on Ledger hardware wallets using `SIGN_MODE_LEGACY_AMINO_JSON`, register your message types with the Amino codec via `RegisterLegacyAminoCodec` in your module's `codec.go`. @@ -121,7 +119,7 @@ The sign mode is negotiated at transaction construction time and does not affect Every transaction message must declare which addresses are authorized to sign it. In v0.50+, this is done via the `cosmos.msg.v1.signer` protobuf annotation — the SDK reads the annotation at startup and automatically extracts signer addresses from that field. See [Protobuf Annotations](/sdk/next/guides/reference/protobuf-annotations) for the full annotation reference. -For messages that cannot use the annotation — for example, messages with non-standard signing logic such as EVM-compatible transactions — you can register a custom signer function using [`signing.CustomGetSigner`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/x/tx/signing/context.go#L127): +For messages that cannot use the annotation — for example, messages with non-standard signing logic such as EVM-compatible transactions — you can register a custom signer function using [`signing.CustomGetSigner`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/x/tx/signing/context.go#L127): ```go signer := signing.CustomGetSigner{ @@ -177,7 +175,7 @@ The codec (`k.cdc`) is the protobuf codec described in the next section. ## The codec and interface registry -The Cosmos SDK wraps protobuf in a **codec** that modules use for marshaling and unmarshaling. The primary implementation is [`ProtoCodec`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/codec/proto_codec.go), which calls protobuf's `Marshal` and `Unmarshal` under the hood. +The Cosmos SDK wraps protobuf in a **codec** that modules use for marshaling and unmarshaling. The primary implementation is [`ProtoCodec`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/codec/proto_codec.go), which calls protobuf's `Marshal` and `Unmarshal` under the hood. ```go type ProtoCodec struct { @@ -241,7 +239,7 @@ This lookup is handled by the **interface registry**. ### Interface registry -The [`InterfaceRegistry`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/codec/types/interface_registry.go) is a runtime map from type URLs to Go types. When the SDK encounters an `Any` value, it queries the registry with the type URL to find the concrete Go type, then uses protobuf to unmarshal the bytes. +The [`InterfaceRegistry`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/codec/types/interface_registry.go) is a runtime map from type URLs to Go types. When the SDK encounters an `Any` value, it queries the registry with the type URL to find the concrete Go type, then uses protobuf to unmarshal the bytes. ```text Any { type_url, value_bytes } diff --git a/sdk/next/learn/concepts/store.mdx b/sdk/next/learn/concepts/store.mdx index 93b8133c3..61c6d995a 100644 --- a/sdk/next/learn/concepts/store.mdx +++ b/sdk/next/learn/concepts/store.mdx @@ -34,7 +34,7 @@ key: 0x2 | 20 | cosmos1abc...xyz | uatom value: ProtocolBuffer(1000000) ``` -The key encodes the store prefix, address length, address, and denomination. The value is a Protocol Buffer-encoded amount. See [`x/bank/types/keys.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/x/bank/types/keys.go) for the actual implementation. +The key encodes the store prefix, address length, address, and denomination. The value is a Protocol Buffer-encoded amount. See [`x/bank/types/keys.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/x/bank/types/keys.go) for the actual implementation. Each module owns its own namespace in the key-value store. Keys are defined by the module and typically begin with a byte prefix that distinguishes them from other module keys. @@ -42,7 +42,7 @@ Each module owns its own namespace in the key-value store. Keys are defined by t A single module store is only part of the picture. At the application level, all module stores are committed together. -Every module has its own KVStore, and all module stores are mounted inside a [multistore](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/store/rootmulti/store.go) that is committed as a single state root. +Every module has its own KVStore, and all module stores are mounted inside a [multistore](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/store/rootmulti/store.go) that is committed as a single state root. A module can only read and write to its own store through its keeper. Access is gated by a `StoreKey`, which is a typed capability object registered at app startup. Modules that don't hold the key cannot open the store. @@ -59,7 +59,7 @@ The full storage stack from top to bottom is: ``` Module keeper ↓ -KVStore (namespaced, wrapped with gas/trace) +KVStore (namespaced, wrapped with gas metering) ↓ CommitMultiStore (multistore, computes app hash) ↓ @@ -70,7 +70,7 @@ Database backend (goleveldb by default) ## How state is stored (IAVL and commit stores) -Each module's KVStore is backed by a [`CommitKVStore`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/store/iavl/store.go#L36). [See the store spec for more details.](/sdk/next/guides/state/store) +Each module's KVStore is backed by a [`CommitKVStore`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/store/iavl/store.go#L36). [See the store spec for more details.](/sdk/next/guides/state/store) In the current SDK store implementation described here, the Cosmos SDK uses [IAVL](https://github.com/cosmos/iavl), a versioned AVL Merkle tree. @@ -118,16 +118,16 @@ Beyond the base KVStore, the SDK provides several specialized store wrappers. - [CommitKVStore](#commitkvstore-persistent-store) - [CacheMultiStore](#cachemultistore-transaction-isolation) - [Ephemeral store types](#ephemeral-store-types) -- [Gas and trace store wrappers](#gas-and-trace-store-wrappers) +- [Gas store wrapper](#gas-store-wrapper) - [Prefix store](#prefix-store) ### CommitKVStore (persistent store) -The [`CommitKVStore`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/store/iavl/store.go#L36) is the main persistent store backed by IAVL. It persists across blocks, produces versioned commits, and contributes to the app hash. +The [`CommitKVStore`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/store/iavl/store.go#L36) is the main persistent store backed by IAVL. It persists across blocks, produces versioned commits, and contributes to the app hash. ### CacheMultiStore (transaction isolation) -Before executing each transaction, the Cosmos SDK's `BaseApp` creates a [`CacheMultiStore`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/store/cachemulti/store.go) — a cached, copy-on-write view of the multistore. +Before executing each transaction, the Cosmos SDK's `BaseApp` creates a [`CacheMultiStore`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/store/cachemulti/store.go) — a cached, copy-on-write view of the multistore. All writes during that transaction occur in this cached layer: @@ -146,9 +146,9 @@ This is how transaction atomicity is implemented in the store layer. ### Ephemeral store types -[Transient stores](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/store/transient/store.go) are cleared at the end of each block. They are used for temporary per-block data such as counters or intermediate calculations, and do not affect the app hash. +[Transient stores](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/store/transient/store.go) are cleared at the end of each block. They are used for temporary per-block data such as counters or intermediate calculations, and do not affect the app hash. -[Memory stores](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/store/mem/store.go) survive block commits but reset when the node restarts — their `Commit()` is a no-op and data is never written to disk. They are used for in-process caching of data that is expensive to recompute each block but does not need to survive a restart. Modules access them via `MemoryStoreKey`, mounted with `MountMemoryStores` in `app.go`. +[Memory stores](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/store/mem/store.go) survive block commits but reset when the node restarts — their `Commit()` is a no-op and data is never written to disk. They are used for in-process caching of data that is expensive to recompute each block but does not need to survive a restart. Modules access them via `MemoryStoreKey`, mounted with `MountMemoryStores` in `app.go`. | Store type | Survives block commit | Survives restart | |---|---|---| @@ -156,18 +156,15 @@ This is how transaction atomicity is implemented in the store layer. | Memory | Yes | No | | IAVL (CommitKVStore) | Yes | Yes | -### Gas and trace store wrappers +### Gas store wrapper -All store accesses are wrapped with additional behavior by the [`GasKVStore`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/store/gaskv/store.go) and [`TraceKVStore`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/store/tracekv/store.go) wrappers. - -- `GasKVStore` charges gas for each read and write -- `TraceKVStore` logs each store operation for debugging +All store accesses are wrapped with additional behavior by the [`GasKVStore`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/store/gaskv/store.go) wrapper, which charges gas for each read and write. Because state access is the dominant cost of transaction execution, the SDK charges gas at the store layer so that expensive reads and writes are reflected in transaction fees. Every read and write of a KVStore costs gas, and expensive operations naturally cost more. [Execution Context, Gas, and Events](/sdk/next/learn/concepts/context-gas-events) explains how gas metering works at runtime. ### Prefix store -A [**prefix store**](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/store/prefix/store.go) wraps a KVStore and automatically prepends a fixed byte prefix to every key. This lets keepers scope their reads and writes to a sub-namespace without manually constructing prefixed keys on every call. +A [**prefix store**](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/store/prefix/store.go) wraps a KVStore and automatically prepends a fixed byte prefix to every key. This lets keepers scope their reads and writes to a sub-namespace without manually constructing prefixed keys on every call. ```go prefixStore := prefix.NewStore(kvStore, types.KeyPrefix("balances")) @@ -199,7 +196,7 @@ k.Counter.Set(ctx, count+1) The Collections API defines the storage schema, handles encoding and decoding, ensures consistent key construction, and makes state access type-safe. -Under the hood, collections still store data in a KVStore. Collections are used to provide a safer abstraction over raw byte keys. See [`collections/collections.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/collections/collections.go) for the base interface definitions. For the full package guide, see [Collections](/sdk/next/guides/state/collections). +Under the hood, collections still store data in a KVStore. Collections are used to provide a safer abstraction over raw byte keys. See [`collections/collections.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/collections/collections.go) for the base interface definitions. For the full package guide, see [Collections](/sdk/next/guides/state/collections). ## How modules access state @@ -251,6 +248,6 @@ For a walkthrough of genesis implementation in a module, see [Step 2: Proto file ## Next steps -For more information on stores, pruning strategies, and store configuration, see the [store spec](/sdk/next/guides/state/store). For the full store interface definitions, see [`store/types/store.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/store/types/store.go) in the SDK source. +For more information on stores, pruning strategies, and store configuration, see the [store spec](/sdk/next/guides/state/store). For the full store interface definitions, see [`store/types/store.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/store/types/store.go) in the SDK source. Because KV stores only hold raw bytes, modules must serialize structured data before writing it. The next section, [Encoding and Protobuf](/sdk/next/learn/concepts/encoding), explains how the Cosmos SDK uses Protocol Buffers to encode that data deterministically, and why every validator must produce exactly the same bytes. diff --git a/sdk/next/learn/concepts/testing.mdx b/sdk/next/learn/concepts/testing.mdx index 89e8d582d..822e7d705 100644 --- a/sdk/next/learn/concepts/testing.mdx +++ b/sdk/next/learn/concepts/testing.mdx @@ -308,7 +308,7 @@ app.sm.RegisterStoreDecoders() ### testutil -The [`testutil`](https://github.com/cosmos/cosmos-sdk/tree/main/testutil) package provides helpers for constructing in-memory contexts for unit tests: +The [`testutil`](https://github.com/cosmos/cosmos-sdk/tree/release/v0.55.x/testutil) package provides helpers for constructing in-memory contexts for unit tests: - `testutil.DefaultContextWithDB` creates a real `sdk.Context` backed by an in-memory KV store. Keeper unit tests use this to get a realistic execution context without starting a full node. - `moduletestutil.MakeTestEncodingConfig` returns a codec with standard interface registration, suitable for keeper tests. @@ -330,7 +330,7 @@ func TestKeeperTestSuite(t *testing.T) { For a full guide on configuring and running simulations, see the [Module Simulation](/sdk/next/guides/testing/simulator) page. -[`simsx`](https://github.com/cosmos/cosmos-sdk/tree/main/testutil/simsx) is the simulation execution framework. It provides: +[`simsx`](https://github.com/cosmos/cosmos-sdk/tree/release/v0.55.x/testutil/simsx) is the simulation execution framework. It provides: - `SimMsgFactoryFn`: a function type that implements the `SimMsgFactoryX` interface for message factories. Each factory selects random accounts and parameters, constructs a message, and returns it for execution. - `ChainDataSource`: provides access to random accounts, balances, and other chain data during message construction. diff --git a/sdk/next/modules/auth/auth.mdx b/sdk/next/modules/auth/auth.mdx index 3973d2193..e90f30064 100644 --- a/sdk/next/modules/auth/auth.mdx +++ b/sdk/next/modules/auth/auth.mdx @@ -161,7 +161,7 @@ See [Vesting](/sdk/next/modules/auth/auth). ## AnteHandlers The `x/auth` module presently has no transaction handlers of its own, but does expose the special `AnteHandler`, used for performing basic validity checks on a transaction, such that it could be thrown out of the mempool. -The `AnteHandler` can be seen as a set of decorators that check transactions within the current context, per [ADR 010](https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-010-modular-antehandler.md). +The `AnteHandler` can be seen as a set of decorators that check transactions within the current context, per [ADR 010](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/docs/architecture/adr-010-modular-antehandler.md). Note that the `AnteHandler` is called on both `CheckTx` and `DeliverTx`, as CometBFT proposers presently have the ability to include in their proposed block transactions which fail `CheckTx`. @@ -261,6 +261,7 @@ The auth module contains the following parameters: | TxSigLimit | uint64 | 7 | | TxSizeCostPerByte | uint64 | 10 | | SigVerifyCostED25519 | uint64 | 590 | +| SigVerifyCostMlDsa65 | uint64 | 750 | | SigVerifyCostSecp256k1 | uint64 | 1000 | ## Client @@ -421,6 +422,7 @@ Example Output: ```bash max_memo_characters: "256" sig_verify_cost_ed25519: "590" +sig_verify_cost_mldsa65: "750" sig_verify_cost_secp256k1: "1000" tx_sig_limit: "7" tx_size_cost_per_byte: "10" diff --git a/sdk/next/modules/auth/tx.mdx b/sdk/next/modules/auth/tx.mdx index 40cc88ed6..5e0c4b091 100644 --- a/sdk/next/modules/auth/tx.mdx +++ b/sdk/next/modules/auth/tx.mdx @@ -35,19 +35,19 @@ This package represents the Cosmos SDK implementation of the `client.TxConfig`, The interface defines a set of methods for creating a `client.TxBuilder`. ```go -// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/client/tx_config.go#L25-L31 +// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.55.x/client/tx_config.go#L26-L36 ``` The default implementation of `client.TxConfig` is instantiated by `NewTxConfig` in `x/auth/tx` module. ```go -// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/x/auth/tx/config.go#L22-L28 +// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.55.x/x/auth/tx/config.go#L67-L87 ``` ### `TxBuilder` ```go -// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/client/tx_config.go#L33-L50 +// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.55.x/client/tx_config.go#L38-L56 ``` The [`client.TxBuilder`](/sdk/next/learn/concepts/lifecycle#transaction-generation) interface is as well implemented by `x/auth/tx`. diff --git a/sdk/next/modules/auth/vesting.mdx b/sdk/next/modules/auth/vesting.mdx index cbdb1d47e..c12254cf7 100644 --- a/sdk/next/modules/auth/vesting.mdx +++ b/sdk/next/modules/auth/vesting.mdx @@ -79,25 +79,25 @@ int64 ### BaseVestingAccount ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/proto/cosmos/vesting/v1beta1/vesting.proto#L11-L35 +// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.55.x/proto/cosmos/vesting/v1beta1/vesting.proto#L12-L39 ``` ### ContinuousVestingAccount ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/proto/cosmos/vesting/v1beta1/vesting.proto#L37-L46 +// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.55.x/proto/cosmos/vesting/v1beta1/vesting.proto#L41-L50 ``` ### DelayedVestingAccount ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/proto/cosmos/vesting/v1beta1/vesting.proto#L48-L57 +// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.55.x/proto/cosmos/vesting/v1beta1/vesting.proto#L52-L60 ``` ### Period ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/proto/cosmos/vesting/v1beta1/vesting.proto#L59-L69 +// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.55.x/proto/cosmos/vesting/v1beta1/vesting.proto#L62-L72 ``` ```go @@ -108,7 +108,7 @@ type Periods []Period ### PeriodicVestingAccount ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/proto/cosmos/vesting/v1beta1/vesting.proto#L71-L81 +// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.55.x/proto/cosmos/vesting/v1beta1/vesting.proto#L74-L83 ``` In order to facilitate less ad-hoc type checking and assertions and to support flexibility in account balance usage, the existing `x/bank` `ViewKeeper` interface is updated to contain the following: @@ -132,7 +132,7 @@ sdk.Coins ### PermanentLockedAccount ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/proto/cosmos/vesting/v1beta1/vesting.proto#L83-L94 +// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.55.x/proto/cosmos/vesting/v1beta1/vesting.proto#L85-L94 ``` ## Vesting Account Specification diff --git a/sdk/next/modules/authz/README.mdx b/sdk/next/modules/authz/README.mdx index 670e9f16c..d6cb762ae 100644 --- a/sdk/next/modules/authz/README.mdx +++ b/sdk/next/modules/authz/README.mdx @@ -5,7 +5,7 @@ noindex: true ## Abstract -`x/authz` is an implementation of a Cosmos SDK module, per [ADR 30](https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-030-authz-module.md), that allows +`x/authz` is an implementation of a Cosmos SDK module, per [ADR 30](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/docs/architecture/adr-030-authz-module.md), that allows granting arbitrary privileges from one account (the granter) to another account (the grantee). Authorizations must be granted for a particular Msg service method one by one using an implementation of the `Authorization` interface. ## Contents @@ -32,7 +32,7 @@ granting arbitrary privileges from one account (the granter) to another account ### Authorization and Grant The `x/authz` module defines interfaces and messages grant authorizations to perform actions -on behalf of one account to other accounts. The design is defined in the [ADR 030](https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-030-authz-module.md). +on behalf of one account to other accounts. The design is defined in the [ADR 030](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/docs/architecture/adr-030-authz-module.md). A *grant* is an allowance to execute a Msg by the grantee on behalf of the granter. Authorization is an interface that must be implemented by a concrete authorization logic to validate and execute grants. Authorizations are extensible and can be defined for any Msg service method even outside of the module where the Msg method is defined. See the `SendAuthorization` example in the next section for more details. @@ -94,7 +94,7 @@ The Cosmos SDK `x/authz` module comes with following authorization types: `GenericAuthorization` implements the `Authorization` interface that gives unrestricted permission to execute the provided Msg on behalf of granter's account. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/authz/v1beta1/authz.proto#L14-L22 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/authz/v1beta1/authz.proto#L13-L21 ``` ```go expandable @@ -153,7 +153,7 @@ error { * It takes an (optional) `AllowList` that specifies to which addresses a grantee can send token. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/bank/v1beta1/authz.proto#L11-L30 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/bank/v1beta1/authz.proto#L11-L29 ``` ```go expandable @@ -280,10 +280,10 @@ return allowedAddrs #### StakeAuthorization -`StakeAuthorization` implements the `Authorization` interface for messages in the [staking module](/sdk/next/modules/staking). It takes an `AuthorizationType` to specify whether you want to authorise delegating, undelegating or redelegating (i.e. these have to be authorised separately). It also takes an optional `MaxTokens` that keeps track of a limit to the amount of tokens that can be delegated/undelegated/redelegated. If left empty, the amount is unlimited. Additionally, this Msg takes an `AllowList` or a `DenyList`, which allows you to select which validators you allow or deny grantees to stake with. +`StakeAuthorization` implements the `Authorization` interface for messages in the [staking module](/sdk/next/modules/staking). It takes an `AuthorizationType` to specify whether you want to authorise delegating, undelegating, redelegating, or cancelling an unbonding delegation (i.e. these have to be authorised separately). It also takes an optional `MaxTokens` that keeps track of a limit to the amount of tokens that can be delegated/undelegated/redelegated. If left empty, the amount is unlimited. Additionally, this Msg takes an `AllowList` or a `DenyList`, which allows you to select which validators you allow or deny grantees to stake with. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/authz.proto#L11-L35 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/staking/v1beta1/authz.proto#L10-L33 ``` ```go expandable @@ -499,602 +499,19 @@ Grants are identified by combining granter address (the address bytes of the gra The grant object encapsulates an `Authorization` type and an expiration timestamp: ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/authz/v1beta1/authz.proto#L24-L32 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/authz/v1beta1/authz.proto#L23-L31 ``` ### GrantQueue We are maintaining a queue for authz pruning. Whenever a grant is created, an item will be added to `GrantQueue` with a key of expiration, granter, grantee. -In `EndBlock` (which runs for every block) we continuously check and prune the expired grants by forming a prefix key with current blocktime that passed the stored expiration in `GrantQueue`, we iterate through all the matched records from `GrantQueue` and delete them from the `GrantQueue` & `Grant`s store. +In `BeginBlock`, which runs for every block, the module prunes expired grants. It forms a prefix key from the current block time and matches the records in `GrantQueue` whose stored expiration has passed. It deletes those records from both the `GrantQueue` and the `Grant` store. Pruning is capped at 200 grants per block. Any remaining expired grants are pruned in later blocks. -```go expandable -package keeper - -import ( - - "fmt" - "strconv" - "time" - "github.com/cosmos/gogoproto/proto" - abci "github.com/tendermint/tendermint/abci/types" - "github.com/tendermint/tendermint/libs/log" - "github.com/cosmos/cosmos-sdk/baseapp" - "github.com/cosmos/cosmos-sdk/codec" - codectypes "github.com/cosmos/cosmos-sdk/codec/types" - storetypes "github.com/cosmos/cosmos-sdk/store/types" - sdk "github.com/cosmos/cosmos-sdk/types" - sdkerrors "github.com/cosmos/cosmos-sdk/types/errors" - "github.com/cosmos/cosmos-sdk/x/authz" -) - -// TODO: Revisit this once we have proper gas fee framework. -// Tracking issues https://github.com/cosmos/cosmos-sdk/issues/9054, -// https://github.com/cosmos/cosmos-sdk/discussions/9072 -const gasCostPerIteration = uint64(20) - -type Keeper struct { - storeKey storetypes.StoreKey - cdc codec.BinaryCodec - router *baseapp.MsgServiceRouter - authKeeper authz.AccountKeeper -} - -// NewKeeper constructs a message authorization Keeper -func NewKeeper(storeKey storetypes.StoreKey, cdc codec.BinaryCodec, router *baseapp.MsgServiceRouter, ak authz.AccountKeeper) - -Keeper { - return Keeper{ - storeKey: storeKey, - cdc: cdc, - router: router, - authKeeper: ak, -} -} - -// Logger returns a module-specific logger. -func (k Keeper) - -Logger(ctx sdk.Context) - -log.Logger { - return ctx.Logger().With("module", fmt.Sprintf("x/%s", authz.ModuleName)) -} - -// getGrant returns grant stored at skey. -func (k Keeper) - -getGrant(ctx sdk.Context, skey []byte) (grant authz.Grant, found bool) { - store := ctx.KVStore(k.storeKey) - bz := store.Get(skey) - if bz == nil { - return grant, false -} - -k.cdc.MustUnmarshal(bz, &grant) - -return grant, true -} - -func (k Keeper) - -update(ctx sdk.Context, grantee sdk.AccAddress, granter sdk.AccAddress, updated authz.Authorization) - -error { - skey := grantStoreKey(grantee, granter, updated.MsgTypeURL()) - -grant, found := k.getGrant(ctx, skey) - if !found { - return authz.ErrNoAuthorizationFound -} - -msg, ok := updated.(proto.Message) - if !ok { - return sdkerrors.ErrPackAny.Wrapf("cannot proto marshal %T", updated) -} - -any, err := codectypes.NewAnyWithValue(msg) - if err != nil { - return err -} - -grant.Authorization = any - store := ctx.KVStore(k.storeKey) - -store.Set(skey, k.cdc.MustMarshal(&grant)) - -return nil -} - -// DispatchActions attempts to execute the provided messages via authorization -// grants from the message signer to the grantee. -func (k Keeper) - -DispatchActions(ctx sdk.Context, grantee sdk.AccAddress, msgs []sdk.Msg) ([][]byte, error) { - results := make([][]byte, len(msgs)) - now := ctx.BlockTime() - for i, msg := range msgs { - signers := msg.GetSigners() - if len(signers) != 1 { - return nil, authz.ErrAuthorizationNumOfSigners -} - granter := signers[0] - - // If granter != grantee then check authorization.Accept, otherwise we - // implicitly accept. - if !granter.Equals(grantee) { - skey := grantStoreKey(grantee, granter, sdk.MsgTypeURL(msg)) - -grant, found := k.getGrant(ctx, skey) - if !found { - return nil, sdkerrors.Wrapf(authz.ErrNoAuthorizationFound, "failed to update grant with key %s", string(skey)) -} - if grant.Expiration != nil && grant.Expiration.Before(now) { - return nil, authz.ErrAuthorizationExpired -} - -authorization, err := grant.GetAuthorization() - if err != nil { - return nil, err -} - -resp, err := authorization.Accept(ctx, msg) - if err != nil { - return nil, err -} - if resp.Delete { - err = k.DeleteGrant(ctx, grantee, granter, sdk.MsgTypeURL(msg)) -} - -else if resp.Updated != nil { - err = k.update(ctx, grantee, granter, resp.Updated) -} - if err != nil { - return nil, err -} - if !resp.Accept { - return nil, sdkerrors.ErrUnauthorized -} - -} - handler := k.router.Handler(msg) - if handler == nil { - return nil, sdkerrors.ErrUnknownRequest.Wrapf("unrecognized message route: %s", sdk.MsgTypeURL(msg)) -} - -msgResp, err := handler(ctx, msg) - if err != nil { - return nil, sdkerrors.Wrapf(err, "failed to execute message; message %v", msg) -} - -results[i] = msgResp.Data - - // emit the events from the dispatched actions - events := msgResp.Events - sdkEvents := make([]sdk.Event, 0, len(events)) - for _, event := range events { - e := event - e.Attributes = append(e.Attributes, abci.EventAttribute{ - Key: "authz_msg_index", - Value: strconv.Itoa(i) -}) - -sdkEvents = append(sdkEvents, sdk.Event(e)) -} - -ctx.EventManager().EmitEvents(sdkEvents) -} - -return results, nil -} - -// SaveGrant method grants the provided authorization to the grantee on the granter's account -// with the provided expiration time and insert authorization key into the grants queue. If there is an existing authorization grant for the -// same `sdk.Msg` type, this grant overwrites that. -func (k Keeper) - -SaveGrant(ctx sdk.Context, grantee, granter sdk.AccAddress, authorization authz.Authorization, expiration *time.Time) - -error { - store := ctx.KVStore(k.storeKey) - msgType := authorization.MsgTypeURL() - skey := grantStoreKey(grantee, granter, msgType) - -grant, err := authz.NewGrant(ctx.BlockTime(), authorization, expiration) - if err != nil { - return err -} - -var oldExp *time.Time - if oldGrant, found := k.getGrant(ctx, skey); found { - oldExp = oldGrant.Expiration -} - if oldExp != nil && (expiration == nil || !oldExp.Equal(*expiration)) { - if err = k.removeFromGrantQueue(ctx, skey, granter, grantee, *oldExp); err != nil { - return err -} - -} - - // If the expiration didn't change, then we don't remove it and we should not insert again - if expiration != nil && (oldExp == nil || !oldExp.Equal(*expiration)) { - if err = k.insertIntoGrantQueue(ctx, granter, grantee, msgType, *expiration); err != nil { - return err -} - -} - bz := k.cdc.MustMarshal(&grant) - -store.Set(skey, bz) - -return ctx.EventManager().EmitTypedEvent(&authz.EventGrant{ - MsgTypeUrl: authorization.MsgTypeURL(), - Granter: granter.String(), - Grantee: grantee.String(), -}) -} - -// DeleteGrant revokes any authorization for the provided message type granted to the grantee -// by the granter. -func (k Keeper) - -DeleteGrant(ctx sdk.Context, grantee sdk.AccAddress, granter sdk.AccAddress, msgType string) - -error { - store := ctx.KVStore(k.storeKey) - skey := grantStoreKey(grantee, granter, msgType) - -grant, found := k.getGrant(ctx, skey) - if !found { - return sdkerrors.Wrapf(authz.ErrNoAuthorizationFound, "failed to delete grant with key %s", string(skey)) -} - if grant.Expiration != nil { - err := k.removeFromGrantQueue(ctx, skey, granter, grantee, *grant.Expiration) - if err != nil { - return err -} - -} - -store.Delete(skey) - -return ctx.EventManager().EmitTypedEvent(&authz.EventRevoke{ - MsgTypeUrl: msgType, - Granter: granter.String(), - Grantee: grantee.String(), -}) -} - -// GetAuthorizations Returns list of `Authorizations` granted to the grantee by the granter. -func (k Keeper) - -GetAuthorizations(ctx sdk.Context, grantee sdk.AccAddress, granter sdk.AccAddress) ([]authz.Authorization, error) { - store := ctx.KVStore(k.storeKey) - key := grantStoreKey(grantee, granter, "") - iter := sdk.KVStorePrefixIterator(store, key) - -defer iter.Close() - -var authorization authz.Grant - var authorizations []authz.Authorization - for ; iter.Valid(); iter.Next() { - if err := k.cdc.Unmarshal(iter.Value(), &authorization); err != nil { - return nil, err -} - -a, err := authorization.GetAuthorization() - if err != nil { - return nil, err -} - -authorizations = append(authorizations, a) -} - -return authorizations, nil -} - -// GetAuthorization returns an Authorization and it's expiration time. -// A nil Authorization is returned under the following circumstances: -// - No grant is found. -// - A grant is found, but it is expired. -// - There was an error getting the authorization from the grant. -func (k Keeper) - -GetAuthorization(ctx sdk.Context, grantee sdk.AccAddress, granter sdk.AccAddress, msgType string) (authz.Authorization, *time.Time) { - grant, found := k.getGrant(ctx, grantStoreKey(grantee, granter, msgType)) - if !found || (grant.Expiration != nil && grant.Expiration.Before(ctx.BlockHeader().Time)) { - return nil, nil -} - -auth, err := grant.GetAuthorization() - if err != nil { - return nil, nil -} - -return auth, grant.Expiration -} - -// IterateGrants iterates over all authorization grants -// This function should be used with caution because it can involve significant IO operations. -// It should not be used in query or msg services without charging additional gas. -// The iteration stops when the handler function returns true or the iterator exhaust. -func (k Keeper) - -IterateGrants(ctx sdk.Context, - handler func(granterAddr sdk.AccAddress, granteeAddr sdk.AccAddress, grant authz.Grant) - -bool, -) { - store := ctx.KVStore(k.storeKey) - iter := sdk.KVStorePrefixIterator(store, GrantKey) - -defer iter.Close() - for ; iter.Valid(); iter.Next() { - var grant authz.Grant - granterAddr, granteeAddr, _ := parseGrantStoreKey(iter.Key()) - -k.cdc.MustUnmarshal(iter.Value(), &grant) - if handler(granterAddr, granteeAddr, grant) { - break -} - -} -} - -func (k Keeper) - -getGrantQueueItem(ctx sdk.Context, expiration time.Time, granter, grantee sdk.AccAddress) (*authz.GrantQueueItem, error) { - store := ctx.KVStore(k.storeKey) - bz := store.Get(GrantQueueKey(expiration, granter, grantee)) - if bz == nil { - return &authz.GrantQueueItem{ -}, nil -} - -var queueItems authz.GrantQueueItem - if err := k.cdc.Unmarshal(bz, &queueItems); err != nil { - return nil, err -} - -return &queueItems, nil -} - -func (k Keeper) - -setGrantQueueItem(ctx sdk.Context, expiration time.Time, - granter sdk.AccAddress, grantee sdk.AccAddress, queueItems *authz.GrantQueueItem, -) - -error { - store := ctx.KVStore(k.storeKey) - -bz, err := k.cdc.Marshal(queueItems) - if err != nil { - return err -} - -store.Set(GrantQueueKey(expiration, granter, grantee), bz) - -return nil -} - -// insertIntoGrantQueue inserts a grant key into the grant queue -func (k Keeper) - -insertIntoGrantQueue(ctx sdk.Context, granter, grantee sdk.AccAddress, msgType string, expiration time.Time) - -error { - queueItems, err := k.getGrantQueueItem(ctx, expiration, granter, grantee) - if err != nil { - return err -} - if len(queueItems.MsgTypeUrls) == 0 { - k.setGrantQueueItem(ctx, expiration, granter, grantee, &authz.GrantQueueItem{ - MsgTypeUrls: []string{ - msgType -}, -}) -} - -else { - queueItems.MsgTypeUrls = append(queueItems.MsgTypeUrls, msgType) - -k.setGrantQueueItem(ctx, expiration, granter, grantee, queueItems) -} - -return nil -} - -// removeFromGrantQueue removes a grant key from the grant queue -func (k Keeper) - -removeFromGrantQueue(ctx sdk.Context, grantKey []byte, granter, grantee sdk.AccAddress, expiration time.Time) - -error { - store := ctx.KVStore(k.storeKey) - key := GrantQueueKey(expiration, granter, grantee) - bz := store.Get(key) - if bz == nil { - return sdkerrors.Wrap(authz.ErrNoGrantKeyFound, "can't remove grant from the expire queue, grant key not found") -} - -var queueItem authz.GrantQueueItem - if err := k.cdc.Unmarshal(bz, &queueItem); err != nil { - return err -} - - _, _, msgType := parseGrantStoreKey(grantKey) - queueItems := queueItem.MsgTypeUrls - for index, typeURL := range queueItems { - ctx.GasMeter().ConsumeGas(gasCostPerIteration, "grant queue") - if typeURL == msgType { - end := len(queueItem.MsgTypeUrls) - 1 - queueItems[index] = queueItems[end] - queueItems = queueItems[:end] - if err := k.setGrantQueueItem(ctx, expiration, granter, grantee, &authz.GrantQueueItem{ - MsgTypeUrls: queueItems, -}); err != nil { - return err -} - -break -} - -} - -return nil -} - -// DequeueAndDeleteExpiredGrants deletes expired grants from the state and grant queue. -func (k Keeper) - -DequeueAndDeleteExpiredGrants(ctx sdk.Context) - -error { - store := ctx.KVStore(k.storeKey) - iterator := store.Iterator(GrantQueuePrefix, sdk.InclusiveEndBytes(GrantQueueTimePrefix(ctx.BlockTime()))) - -defer iterator.Close() - for ; iterator.Valid(); iterator.Next() { - var queueItem authz.GrantQueueItem - if err := k.cdc.Unmarshal(iterator.Value(), &queueItem); err != nil { - return err -} - - _, granter, grantee, err := parseGrantQueueKey(iterator.Key()) - if err != nil { - return err -} - -store.Delete(iterator.Key()) - for _, typeURL := range queueItem.MsgTypeUrls { - store.Delete(grantStoreKey(grantee, granter, typeURL)) -} - -} - -return nil -} -``` - -* GrantQueue: `0x02 | expiration_bytes | granter_address_len (1 byte) | granter_address_bytes | grantee_address_len (1 byte) | grantee_address_bytes -> ProtocalBuffer(GrantQueueItem)` +* GrantQueue: `0x02 | expiration_bytes | granter_address_len (1 byte) | granter_address_bytes | grantee_address_len (1 byte) | grantee_address_bytes -> ProtocolBuffer(GrantQueueItem)` The `expiration_bytes` are the expiration date in UTC with the format `"2006-01-02T15:04:05.000000000"`. -```go expandable -package keeper - -import ( - - "time" - "github.com/cosmos/cosmos-sdk/internal/conv" - sdk "github.com/cosmos/cosmos-sdk/types" - "github.com/cosmos/cosmos-sdk/types/address" - "github.com/cosmos/cosmos-sdk/types/kv" - "github.com/cosmos/cosmos-sdk/x/authz" -) - -// Keys for store prefixes -// Items are stored with the following key: values -// -// - 0x01: Grant -// - 0x02: GrantQueueItem -var ( - GrantKey = []byte{0x01 -} // prefix for each key - GrantQueuePrefix = []byte{0x02 -} -) - -var lenTime = len(sdk.FormatTimeBytes(time.Now())) - -// StoreKey is the store key string for authz -const StoreKey = authz.ModuleName - -// grantStoreKey - return authorization store key -// Items are stored with the following key: values -// -// - 0x01: Grant -func grantStoreKey(grantee sdk.AccAddress, granter sdk.AccAddress, msgType string) []byte { - m := conv.UnsafeStrToBytes(msgType) - -granter = address.MustLengthPrefix(granter) - -grantee = address.MustLengthPrefix(grantee) - key := sdk.AppendLengthPrefixedBytes(GrantKey, granter, grantee, m) - -return key -} - -// parseGrantStoreKey - split granter, grantee address and msg type from the authorization key -func parseGrantStoreKey(key []byte) (granterAddr, granteeAddr sdk.AccAddress, msgType string) { - // key is of format: - // 0x01 - - granterAddrLen, granterAddrLenEndIndex := sdk.ParseLengthPrefixedBytes(key, 1, 1) // ignore key[0] since it is a prefix key - granterAddr, granterAddrEndIndex := sdk.ParseLengthPrefixedBytes(key, granterAddrLenEndIndex+1, int(granterAddrLen[0])) - -granteeAddrLen, granteeAddrLenEndIndex := sdk.ParseLengthPrefixedBytes(key, granterAddrEndIndex+1, 1) - -granteeAddr, granteeAddrEndIndex := sdk.ParseLengthPrefixedBytes(key, granteeAddrLenEndIndex+1, int(granteeAddrLen[0])) - -kv.AssertKeyAtLeastLength(key, granteeAddrEndIndex+1) - -return granterAddr, granteeAddr, conv.UnsafeBytesToStr(key[(granteeAddrEndIndex + 1):]) -} - -// parseGrantQueueKey split expiration time, granter and grantee from the grant queue key -func parseGrantQueueKey(key []byte) (time.Time, sdk.AccAddress, sdk.AccAddress, error) { - // key is of format: - // 0x02 - - expBytes, expEndIndex := sdk.ParseLengthPrefixedBytes(key, 1, lenTime) - -exp, err := sdk.ParseTimeBytes(expBytes) - if err != nil { - return exp, nil, nil, err -} - -granterAddrLen, granterAddrLenEndIndex := sdk.ParseLengthPrefixedBytes(key, expEndIndex+1, 1) - -granter, granterEndIndex := sdk.ParseLengthPrefixedBytes(key, granterAddrLenEndIndex+1, int(granterAddrLen[0])) - -granteeAddrLen, granteeAddrLenEndIndex := sdk.ParseLengthPrefixedBytes(key, granterEndIndex+1, 1) - -grantee, _ := sdk.ParseLengthPrefixedBytes(key, granteeAddrLenEndIndex+1, int(granteeAddrLen[0])) - -return exp, granter, grantee, nil -} - -// GrantQueueKey - return grant queue store key. If a given grant doesn't have a defined -// expiration, then it should not be used in the pruning queue. -// Key format is: -// -// 0x02: GrantQueueItem -func GrantQueueKey(expiration time.Time, granter sdk.AccAddress, grantee sdk.AccAddress) []byte { - exp := sdk.FormatTimeBytes(expiration) - -granter = address.MustLengthPrefix(granter) - -grantee = address.MustLengthPrefix(grantee) - -return sdk.AppendLengthPrefixedBytes(GrantQueuePrefix, exp, granter, grantee) -} - -// GrantQueueTimePrefix - return grant queue time prefix -func GrantQueueTimePrefix(expiration time.Time) []byte { - return append(GrantQueuePrefix, sdk.FormatTimeBytes(expiration)...) -} - -// firstAddressFromGrantStoreKey parses the first address only -func firstAddressFromGrantStoreKey(key []byte) - -sdk.AccAddress { - addrLen := key[0] - return sdk.AccAddress(key[1 : 1+addrLen]) -} -``` - The `GrantQueueItem` object contains the list of type urls between granter and grantee that expire at the time indicated in the key. ## Messages @@ -1107,7 +524,7 @@ An authorization grant is created using the `MsgGrant` message. If there is already a grant for the `(granter, grantee, Authorization)` triple, then the new grant overwrites the previous one. To update or extend an existing grant, a new grant with the same `(granter, grantee, Authorization)` triple should be created. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/authz/v1beta1/tx.proto#L35-L45 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/authz/v1beta1/tx.proto#L34-L44 ``` The message handling should fail if: @@ -1122,7 +539,7 @@ The message handling should fail if: A grant can be removed with the `MsgRevoke` message. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/authz/v1beta1/tx.proto#L69-L78 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/authz/v1beta1/tx.proto#L68-L77 ``` The message handling should fail if: @@ -1137,7 +554,7 @@ NOTE: The `MsgExec` message removes a grant if the grant has expired. When a grantee wants to execute a transaction on behalf of a granter, they must send `MsgExec`. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/authz/v1beta1/tx.proto#L52-L63 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/authz/v1beta1/tx.proto#L49-L61 ``` The message handling should fail if: diff --git a/sdk/next/modules/bank/README.mdx b/sdk/next/modules/bank/README.mdx index e6959fb50..709679433 100644 --- a/sdk/next/modules/bank/README.mdx +++ b/sdk/next/modules/bank/README.mdx @@ -135,7 +135,7 @@ it can be updated with governance or the address with authority. * Params: `0x05 | ProtocolBuffer(Params)` ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/bank/v1beta1/bank.proto#L12-L23 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/bank/v1beta1/bank.proto#L12-L22 ``` ## Keepers @@ -473,7 +473,7 @@ IterateAllBalances(ctx context.Context, cb func(address sdk.AccAddress, coin sdk Send coins from one address to another. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/bank/v1beta1/tx.proto#L38-L53 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/bank/v1beta1/tx.proto#L38-L54 ``` The message will fail under the following conditions: @@ -486,7 +486,7 @@ The message will fail under the following conditions: Send coins from one sender and to a series of different address. If any of the receiving addresses do not correspond to an existing account, a new account is created. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/bank/v1beta1/tx.proto#L58-L69 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/bank/v1beta1/tx.proto#L59-L70 ``` The message will fail under the following conditions: @@ -501,7 +501,7 @@ The message will fail under the following conditions: The `bank` module params can be updated through `MsgUpdateParams`, which can be done using governance proposal. The signer will always be the `gov` module account address. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/bank/v1beta1/tx.proto#L74-L88 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/bank/v1beta1/tx.proto#L75-L88 ``` The message handling can fail if: @@ -513,7 +513,7 @@ The message handling can fail if: Used with the x/gov module to set create/edit SendEnabled entries. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/bank/v1beta1/tx.proto#L96-L117 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/bank/v1beta1/tx.proto#L96-L117 ``` The message will fail under the following conditions: diff --git a/sdk/next/modules/circuit/README.mdx b/sdk/next/modules/circuit/README.mdx index 08302768f..01cc8e641 100644 --- a/sdk/next/modules/circuit/README.mdx +++ b/sdk/next/modules/circuit/README.mdx @@ -4,7 +4,7 @@ noindex: true --- -`x/circuit` has been moved to [`./contrib/x/circuit`](https://github.com/cosmos/cosmos-sdk/tree/main/contrib/x/circuit) and is no longer actively maintained as part of the core Cosmos SDK. It is still available for use but is not included in the SDK Bug Bounty program. It was moved because it was never widely adopted. +`x/circuit` has been moved to [`./contrib/x/circuit`](https://github.com/cosmos/cosmos-sdk/tree/release/v0.55.x/contrib/x/circuit) and is no longer actively maintained as part of the core Cosmos SDK. It is still available for use but is not included in the SDK Bug Bounty program. It was moved because it was never widely adopted. ## Concepts @@ -440,7 +440,7 @@ Reset is called by an authorized account to enable execution for a specific msgU ### MsgAuthorizeCircuitBreaker ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/main/proto/cosmos/circuit/v1/tx.proto#L25-L75 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/contrib/proto/circuit/v1/tx.proto#L25-L40 ``` This message is expected to fail if: @@ -450,7 +450,7 @@ This message is expected to fail if: ### MsgTripCircuitBreaker ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/main/proto/cosmos/circuit/v1/tx.proto#L77-L93 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/contrib/proto/circuit/v1/tx.proto#L47-L60 ``` This message is expected to fail if: @@ -460,7 +460,7 @@ This message is expected to fail if: ### MsgResetCircuitBreaker ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/main/proto/cosmos/circuit/v1/tx.proto#L95-109 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/contrib/proto/circuit/v1/tx.proto#L67-L78 ``` This message is expected to fail if: diff --git a/sdk/next/modules/crisis/README.mdx b/sdk/next/modules/crisis/README.mdx index 3daa17b60..bb8890509 100644 --- a/sdk/next/modules/crisis/README.mdx +++ b/sdk/next/modules/crisis/README.mdx @@ -6,7 +6,7 @@ noindex: true --- -`x/crisis` has been moved to [`./contrib/x/crisis`](https://github.com/cosmos/cosmos-sdk/tree/main/contrib/x/crisis) and is no longer actively maintained as part of the core Cosmos SDK. It is still available for use but is not included in the SDK Bug Bounty program. The module was moved because it never worked as intended. +`x/crisis` has been moved to [`./contrib/x/crisis`](https://github.com/cosmos/cosmos-sdk/tree/release/v0.55.x/contrib/x/crisis) and is no longer actively maintained as part of the core Cosmos SDK. It is still available for use but is not included in the SDK Bug Bounty program. The module was moved because it never worked as intended. ## Overview @@ -37,7 +37,7 @@ with the standard gas consumption method. The ConstantFee param is stored in the module params state with the prefix of `0x01`, it can be updated with governance or the address with authority. -* Params: `mint/params -> legacy_amino(sdk.Coin)` +* ConstantFee: `0x01 -> ProtocolBuffer(Coin)` ## Messages @@ -49,7 +49,7 @@ corresponding updates to the state. Blockchain invariants can be checked using the `MsgVerifyInvariant` message. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/crisis/v1beta1/tx.proto#L26-L42 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/contrib/proto/crisis/v1beta1/tx.proto#L26-L42 ``` This message is expected to fail if: diff --git a/sdk/next/modules/distribution/README.mdx b/sdk/next/modules/distribution/README.mdx index 8d11f6722..689fb4339 100644 --- a/sdk/next/modules/distribution/README.mdx +++ b/sdk/next/modules/distribution/README.mdx @@ -99,7 +99,7 @@ In Proof of Stake (PoS) blockchains, rewards gained from transaction fees are pa Rewards are calculated per period. The period is updated each time a validator's delegation changes, for example, when the validator receives a new delegation. The rewards for a single validator can then be calculated by taking the total rewards for the period before the delegation started, minus the current total rewards. -To learn more, see the [F1 Fee Distribution paper](https://github.com/cosmos/cosmos-sdk/tree/main/docs/spec/fee_distribution/f1_fee_distr.pdf). +To learn more, see the [F1 Fee Distribution paper](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/docs/spec/fee_distribution/f1_fee_distr.pdf). The commission to the validator is paid when the validator is removed or when the validator requests a withdrawal. The commission is calculated and incremented at every `BeginBlock` operation to update accumulated fee amounts. @@ -124,35 +124,6 @@ is created which might need to reference the historical record, the reference co Each time one object which previously needed to reference the historical record is deleted, the reference count is decremented. If the reference count hits zero, the historical record is deleted. -### External Community Pool Keepers - -An external pool community keeper is defined as: - -```go expandable -// ExternalCommunityPoolKeeper is the interface that an external community pool module keeper must fulfill -// for x/distribution to properly accept it as a community pool fund destination. -type ExternalCommunityPoolKeeper interface { - // GetCommunityPoolModule gets the module name that funds should be sent to for the community pool. - // This is the address that x/distribution will send funds to for external management. - GetCommunityPoolModule() - -string - // FundCommunityPool allows an account to directly fund the community fund pool. - FundCommunityPool(ctx sdk.Context, amount sdk.Coins, senderAddr sdk.AccAddress) - -error - // DistributeFromCommunityPool distributes funds from the community pool module account to - // a receiver address. - DistributeFromCommunityPool(ctx sdk.Context, amount sdk.Coins, receiveAddr sdk.AccAddress) - -error -} -``` - -By default, the distribution module will use a community pool implementation that is internal. An external community pool -can be provided to the module which will have funds be diverted to it instead of the internal implementation. The reference -external community pool maintained by the Cosmos SDK is [`x/protocolpool`](/sdk/next/modules/protocolpool/README). - ## State ### FeePool @@ -179,7 +150,7 @@ type DecCoin struct { ``` ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/distribution/v1beta1/distribution.proto#L116-L123 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/distribution/v1beta1/distribution.proto#L117-L124 ``` ### Validator Distribution @@ -224,7 +195,7 @@ it can be updated with governance or the address with authority. * Params: `0x09 | ProtocolBuffer(Params)` ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/distribution/v1beta1/distribution.proto#L12-L42 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/distribution/v1beta1/distribution.proto#L12-L44 ``` ## Begin Block @@ -255,61 +226,6 @@ The community pool gets `community_tax * fees`, plus any remaining dust after validators get their rewards that are always rounded down to the nearest integer value. -#### Using an External Community Pool - -Starting with Cosmos SDK v0.53.0, an external community pool, such as `x/protocolpool`, can be used in place of the `x/distribution` managed community pool. - -Please view the warning in the next section before deciding to use an external community pool. - -```go expandable -// ExternalCommunityPoolKeeper is the interface that an external community pool module keeper must fulfill -// for x/distribution to properly accept it as a community pool fund destination. -type ExternalCommunityPoolKeeper interface { - // GetCommunityPoolModule gets the module name that funds should be sent to for the community pool. - // This is the address that x/distribution will send funds to for external management. - GetCommunityPoolModule() - -string - // FundCommunityPool allows an account to directly fund the community fund pool. - FundCommunityPool(ctx sdk.Context, amount sdk.Coins, senderAddr sdk.AccAddress) - -error - // DistributeFromCommunityPool distributes funds from the community pool module account to - // a receiver address. - DistributeFromCommunityPool(ctx sdk.Context, amount sdk.Coins, receiveAddr sdk.AccAddress) - -error -} -``` - -```go -app.DistrKeeper = distrkeeper.NewKeeper( - appCodec, - runtime.NewKVStoreService(keys[distrtypes.StoreKey]), - app.AccountKeeper, - app.BankKeeper, - app.StakingKeeper, - authtypes.FeeCollectorName, - authtypes.NewModuleAddress(govtypes.ModuleName).String(), - distrkeeper.WithExternalCommunityPool(app.ProtocolPoolKeeper), // New option. -) -``` - -#### External Community Pool Usage Warning - -When using an external community pool with `x/distribution`, the following handlers will return an error: - -**QueryService** - -* `CommunityPool` - -**MsgService** - -* `CommunityPoolSpend` -* `FundCommunityPool` - -If you have services that rely on this functionality from `x/distribution`, please update them to use the `x/protocolpool` equivalents. - #### Reward To the Validators The proposer receives no extra rewards. All fees are distributed among all the @@ -360,30 +276,12 @@ community tax rate) * (1 - validator commission rate) By default, the withdraw address is the delegator address. To change its withdraw address, a delegator must send a `MsgSetWithdrawAddress` message. Changing the withdraw address is possible only if the parameter `WithdrawAddrEnabled` is set to `true`. -The withdraw address cannot be any of the module accounts. These accounts are blocked from being withdraw addresses by being added to the distribution keeper's `blockedAddrs` array at initialization. +The withdraw address cannot be any of the module accounts. The distribution keeper does not track these itself; it asks the bank keeper through `BlockedAddr`, so the blocked set is the one bank maintains. -Response: +A blocked withdraw address is handled differently depending on how the withdrawal is triggered. A withdrawal triggered by a user message fails with `ErrUnauthorized`. An automatic withdrawal during `BeginBlock` or `EndBlock` does not fail. Instead, the funds fall back to the owner's own address. The owner is the delegator for rewards, or the validator for commission. If the owner's address is also blocked, the funds go to the community pool. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/distribution/v1beta1/tx.proto#L49-L60 -``` - -```go -func (k Keeper) - -SetWithdrawAddr(ctx context.Context, delegatorAddr sdk.AccAddress, withdrawAddr sdk.AccAddress) - -error - if k.blockedAddrs[withdrawAddr.String()] { - fail with "`{ - withdrawAddr -}` is not allowed to receive external funds" -} - if !k.GetWithdrawAddrEnabled(ctx) { - fail with `ErrSetWithdrawAddrDisabled` -} - -k.SetDelegatorWithdrawAddr(ctx, delegatorAddr, withdrawAddr) +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/distribution/v1beta1/tx.proto#L55-L66 ``` ### MsgWithdrawDelegatorReward @@ -420,10 +318,8 @@ rewards = rewards + (R(B) - R(PN)) * stake The historical rewards are calculated retroactively by playing back all the slashes and then attenuating the delegator's stake at each step. The final calculated stake is equivalent to the actual staked coins in the delegation with a margin of error due to rounding errors. -Response: - ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/distribution/v1beta1/tx.proto#L66-L77 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/distribution/v1beta1/tx.proto#L72-L83 ``` ### WithdrawValidatorCommission @@ -435,12 +331,6 @@ Only integer amounts can be sent. If the accumulated awards have decimals, the a ### FundCommunityPool - - -This handler will return an error if an `ExternalCommunityPool` is used. - - - This message sends coins directly from the sender to the community pool. The transaction fails if the amount cannot be transferred from the sender to the distribution module account. @@ -505,7 +395,7 @@ k.SetDelegatorStartingInfo(ctx, val, del, types.NewDelegatorStartingInfo(previou Distribution module params can be updated through `MsgUpdateParams`, which can be done using governance proposal and the signer will always be gov module account address. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/distribution/v1beta1/tx.proto#L133-L147 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/distribution/v1beta1/tx.proto#L142-L155 ``` The message handling can fail if: @@ -1301,4 +1191,3 @@ Example Output: } } ``` -```` diff --git a/sdk/next/modules/evidence/README.mdx b/sdk/next/modules/evidence/README.mdx index 9cb9238c4..0a2549e47 100644 --- a/sdk/next/modules/evidence/README.mdx +++ b/sdk/next/modules/evidence/README.mdx @@ -17,7 +17,7 @@ noindex: true ## Abstract -`x/evidence` is an implementation of a Cosmos SDK module, per [ADR 009](https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-009-evidence-module.md), +`x/evidence` is an implementation of a Cosmos SDK module, per [ADR 009](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/docs/architecture/adr-009-evidence-module.md), that allows for the submission and handling of arbitrary evidence of misbehavior such as equivocation and counterfactual signing. @@ -229,7 +229,7 @@ The evidence module does not contain any parameters. ### Evidence Handling CometBFT blocks can include -[Evidence](https://github.com/cometbft/cometbft/blob/main/spec/abci/abci%2B%2B_basic_concepts.md#evidence) that indicates if a validator committed malicious behavior. The relevant information is forwarded to the application as ABCI Evidence in `abci.RequestBeginBlock` so that the validator can be punished accordingly. +[Evidence](https://github.com/cometbft/cometbft/blob/v0.40.x/spec/abci/abci%2B%2B_basic_concepts.md#evidence) that indicates if a validator committed malicious behavior. The relevant information is forwarded to the application as ABCI Evidence in `abci.RequestBeginBlock` so that the validator can be punished accordingly. #### Equivocation @@ -241,7 +241,7 @@ The Cosmos SDK handles two types of evidence inside the ABCI `BeginBlock`: The evidence module handles these two evidence types the same way. First, the Cosmos SDK converts the CometBFT concrete evidence type to an SDK `Evidence` interface using `Equivocation` as the concrete type. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/proto/cosmos/evidence/v1beta1/evidence.proto#L12-L32 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/evidence/v1beta1/evidence.proto#L12-L31 ``` For some `Equivocation` submitted in `block` to be valid, it must satisfy: @@ -265,7 +265,7 @@ validator to ever re-enter the validator set. The `Equivocation` evidence is handled as follows: ```go -// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/x/evidence/keeper/infraction.go#L26-L140 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/x/evidence/keeper/infraction.go#L13-L155 ``` **Note:** The slashing, jailing, and tombstoning calls are delegated through the `x/slashing` module diff --git a/sdk/next/modules/feegrant/README.mdx b/sdk/next/modules/feegrant/README.mdx index 638722424..de0eee601 100644 --- a/sdk/next/modules/feegrant/README.mdx +++ b/sdk/next/modules/feegrant/README.mdx @@ -8,7 +8,7 @@ noindex: true ## Abstract -This document specifies the fee grant module. For the full ADR, please see [Fee Grant ADR-029](https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-029-fee-grant-module.md). +This document specifies the fee grant module. For the full ADR, please see [Fee Grant ADR-029](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/docs/architecture/adr-029-fee-grant-module.md). This module allows accounts to grant fee allowances and to use fees from their accounts. Grantees can execute any transaction without the need to maintain sufficient fees. @@ -34,10 +34,10 @@ This module allows accounts to grant fee allowances and to use fees from their a ### Grant -`Grant` is stored in the KVStore to record a grant with full context. Every grant will contain `granter`, `grantee` and what kind of `allowance` is granted. `granter` is an account address who is giving permission to `grantee` (the beneficiary account address) to pay for some or all of `grantee`'s transaction fees. `allowance` defines what kind of fee allowance (`BasicAllowance` or `PeriodicAllowance`, see below) is granted to `grantee`. `allowance` accepts an interface which implements `FeeAllowanceI`, encoded as `Any` type. There can be only one existing fee grant allowed for a `grantee` and `granter`, self grants are not allowed. +`Grant` is stored in the KVStore to record a grant with full context. Every grant will contain `granter`, `grantee` and what kind of `allowance` is granted. `granter` is an account address who is giving permission to `grantee` (the beneficiary account address) to pay for some or all of `grantee`'s transaction fees. `allowance` defines what kind of fee allowance is granted to `grantee`. `allowance` accepts an interface which implements `FeeAllowanceI`, encoded as `Any` type. There can be only one existing fee grant allowed for a `grantee` and `granter`, self grants are not allowed. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/feegrant/v1beta1/feegrant.proto#L83-L93 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/feegrant/v1beta1/feegrant.proto#L85-L95 ``` `FeeAllowanceI` looks like: @@ -81,7 +81,7 @@ error ### Fee Allowance types -There are two types of fee allowances present at the moment: +There are three types of fee allowances: * `BasicAllowance` * `PeriodicAllowance` @@ -92,7 +92,7 @@ There are two types of fee allowances present at the moment: `BasicAllowance` is permission for `grantee` to use fee from a `granter`'s account. If any of the `spend_limit` or `expiration` reaches its limit, the grant will be removed from the state. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/feegrant/v1beta1/feegrant.proto#L15-L28 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/feegrant/v1beta1/feegrant.proto#L14-L32 ``` * `spend_limit` is the limit of coins that are allowed to be used from the `granter` account. If it is empty, it assumes there's no spend limit, `grantee` can use any number of available coins from `granter` account address before the expiration. @@ -106,7 +106,7 @@ There are two types of fee allowances present at the moment: `PeriodicAllowance` is a repeating fee allowance for the mentioned period, we can mention when the grant can expire as well as when a period can reset. We can also define the maximum number of coins that can be used in a mentioned period of time. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/feegrant/v1beta1/feegrant.proto#L34-L68 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/feegrant/v1beta1/feegrant.proto#L34-L70 ``` * `basic` is the instance of `BasicAllowance` which is optional for periodic fee allowance. If empty, the grant will have no `expiration` and no `spend_limit`. @@ -124,7 +124,7 @@ There are two types of fee allowances present at the moment: `AllowedMsgAllowance` is a fee allowance, it can be any of `BasicFeeAllowance`, `PeriodicAllowance` but restricted only to the allowed messages mentioned by the granter. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/feegrant/v1beta1/feegrant.proto#L70-L81 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/feegrant/v1beta1/feegrant.proto#L72-L83 ``` * `allowance` is either `BasicAllowance` or `PeriodicAllowance`. @@ -1587,7 +1587,7 @@ return nil ``` ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/tx/v1beta1/tx.proto#L203-L224 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/tx/v1beta1/tx.proto#L221-L248 ``` Example cmd: @@ -3410,7 +3410,7 @@ Fee allowance queue keys are stored in the state as follows: A fee allowance grant will be created with the `MsgGrantAllowance` message. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/feegrant/v1beta1/tx.proto#L25-L39 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/feegrant/v1beta1/tx.proto#L29-L43 ``` ### Msg/RevokeAllowance @@ -3418,7 +3418,7 @@ A fee allowance grant will be created with the `MsgGrantAllowance` message. An allowed grant fee allowance can be removed with the `MsgRevokeAllowance` message. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/feegrant/v1beta1/tx.proto#L41-L54 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/feegrant/v1beta1/tx.proto#L48-L58 ``` ## Events diff --git a/sdk/next/modules/gov/README.mdx b/sdk/next/modules/gov/README.mdx index ecb35086b..c25ae7d69 100644 --- a/sdk/next/modules/gov/README.mdx +++ b/sdk/next/modules/gov/README.mdx @@ -38,8 +38,8 @@ staking token of the chain. * [Proposal submission](#proposal-submission) * [Deposit](#deposit) * [Vote](#vote) - * [Software Upgrade](#software-upgrade) * [State](#state) + * [Constitution](#constitution) * [Proposals](#proposals) * [Parameters and base types](#parameters-and-base-types) * [Deposit](#deposit-1) @@ -172,18 +172,18 @@ proposal but accept the result of the vote. #### Weighted Votes -[ADR-037](https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-037-gov-split-vote.md) introduces the weighted vote feature which allows a staker to split their votes into several voting options. For example, it could use 70% of its voting power to vote Yes and 30% of its voting power to vote No. +[ADR-037](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/docs/architecture/adr-037-gov-split-vote.md) introduces the weighted vote feature which allows a staker to split their votes into several voting options. For example, it could use 70% of its voting power to vote Yes and 30% of its voting power to vote No. Often times the entity owning that address might not be a single individual. For example, a company might have different stakeholders who want to vote differently, and so it makes sense to allow them to split their voting power. Currently, it is not possible for them to do "passthrough voting" and giving their users voting rights over their tokens. However, with this system, exchanges can poll their users for voting preferences, and then vote on-chain proportionally to the results of the poll. To represent weighted vote on chain, we use the following Protobuf message. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/gov/v1beta1/gov.proto#L34-L47 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/gov/v1beta1/gov.proto#L32-L45 ``` ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/gov/v1beta1/gov.proto#L181-L201 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/gov/v1beta1/gov.proto#L180-L198 ``` For a weighted vote to be valid, the `options` field must not contain duplicate vote options, and the sum of weights of all options must be equal to 1. @@ -467,7 +467,7 @@ Threshold is defined as the minimum proportion of `Yes` votes (excluding Initially, the threshold is set at 50% of `Yes` votes, excluding `Abstain` votes. A possibility to veto exists if more than 1/3rd of all votes are -`NoWithVeto` votes. Note, both of these values are derived from the `TallyParams` +`NoWithVeto` votes. Note, both of these values are derived from the `Params` on-chain parameter, which is modifiable by governance. This means that proposals are accepted iff: @@ -554,7 +554,7 @@ unique id and contains a series of timestamps: `submit_time`, `deposit_end_time` `voting_start_time`, `voting_end_time` which track the lifecycle of a proposal ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/gov/v1/gov.proto#L51-L99 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/gov/v1/gov.proto#L50-L101 ``` A proposal will generally require more than just a set of messages to explain its @@ -599,26 +599,12 @@ be one active parameter set at any given time. If governance wants to change a parameter set, either to modify a value or add/remove a parameter field, a new parameter set has to be created and the previous one rendered inactive. -#### DepositParams - -```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/gov/v1/gov.proto#L152-L162 -``` - -#### VotingParams - -```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/gov/v1/gov.proto#L164-L168 -``` - -#### TallyParams +#### Params ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/gov/v1/gov.proto#L170-L182 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/gov/v1/gov.proto#L193-L255 ``` -Parameters are stored in a global `GlobalParams` KVStore. - Additionally, we introduce some basic types: ```go expandable @@ -653,7 +639,7 @@ const ( ### Deposit ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/gov/v1/gov.proto#L38-L49 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/gov/v1/gov.proto#L37-L48 ``` ### ValidatorGovInfo @@ -734,7 +720,7 @@ in voterIterator tmpValMap(voterAddress).Vote = vote - tallyingParam = load(GlobalParams, 'TallyingParam') + tallyingParam = load(Params, 'TallyingParam') // Update tally if validator voted for each validator in validators @@ -786,7 +772,7 @@ More information on how to submit proposals in the [client section](#client). Proposals can be submitted by any account via a `MsgSubmitProposal` transaction. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/gov/v1/tx.proto#L42-L69 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/gov/v1/tx.proto#L57-L88 ``` All `sdk.Msgs` passed into the `messages` field of a `MsgSubmitProposal` message @@ -817,7 +803,7 @@ A deposit is accepted iff: * The deposited coins are conform to the accepted denom from the `MinDeposit` param ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/gov/v1/tx.proto#L134-L147 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/gov/v1/tx.proto#L153-L166 ``` **State modifications:** @@ -836,7 +822,7 @@ bonded Atom holders are able to send `MsgVote` transactions to cast their vote on the proposal. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/gov/v1/tx.proto#L92-L108 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/gov/v1/tx.proto#L111-L127 ``` **State modifications:** diff --git a/sdk/next/modules/group/README.mdx b/sdk/next/modules/group/README.mdx index 6e1816dae..35824f409 100644 --- a/sdk/next/modules/group/README.mdx +++ b/sdk/next/modules/group/README.mdx @@ -105,7 +105,7 @@ custom decision policies, as long as they adhere to the `DecisionPolicy` interface: ```go -// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/x/group/types.go#L27-L45 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/group/x/group/types.go#L44-L62 ``` #### Threshold decision policy @@ -341,7 +341,7 @@ The metadata has a maximum length that is chosen by the app developer, and passed into the group keeper as a config. ```go -// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/proto/cosmos/group/v1/tx.proto#L67-L80 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/group/proto/cosmos/group/v1/tx.proto#L80-L93 ``` It's expected to fail if @@ -354,7 +354,7 @@ It's expected to fail if Group members can be updated with the `UpdateGroupMembers`. ```go -// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/proto/cosmos/group/v1/tx.proto#L88-L102 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/group/proto/cosmos/group/v1/tx.proto#L101-L115 ``` In the list of `MemberUpdates`, an existing member can be removed by setting its weight to 0. @@ -369,7 +369,7 @@ It's expected to fail if: The `UpdateGroupAdmin` can be used to update a group admin. ```go -// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/proto/cosmos/group/v1/tx.proto#L107-L120 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/group/proto/cosmos/group/v1/tx.proto#L120-L133 ``` It's expected to fail if the signer is not the admin of the group. @@ -379,7 +379,7 @@ It's expected to fail if the signer is not the admin of the group. The `UpdateGroupMetadata` can be used to update a group metadata. ```go -// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/proto/cosmos/group/v1/tx.proto#L125-L138 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/group/proto/cosmos/group/v1/tx.proto#L138-L151 ``` It's expected to fail if: @@ -392,7 +392,7 @@ It's expected to fail if: A new group policy can be created with the `MsgCreateGroupPolicy`, which has an admin address, a group id, a decision policy and some optional metadata. ```go -// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/proto/cosmos/group/v1/tx.proto#L147-L165 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/group/proto/cosmos/group/v1/tx.proto#L160-L178 ``` It's expected to fail if: @@ -406,7 +406,7 @@ It's expected to fail if: A new group with policy can be created with the `MsgCreateGroupWithPolicy`, which has an admin address, a list of members, a decision policy, a `group_policy_as_admin` field to optionally set group and group policy admin with group policy address and some optional metadata for group and group policy. ```go -// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/proto/cosmos/group/v1/tx.proto#L191-L215 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/group/proto/cosmos/group/v1/tx.proto#L204-L228 ``` It's expected to fail for the same reasons as `Msg/CreateGroup` and `Msg/CreateGroupPolicy`. @@ -416,7 +416,7 @@ It's expected to fail for the same reasons as `Msg/CreateGroup` and `Msg/CreateG The `UpdateGroupPolicyAdmin` can be used to update a group policy admin. ```go -// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/proto/cosmos/group/v1/tx.proto#L173-L186 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/group/proto/cosmos/group/v1/tx.proto#L186-L199 ``` It's expected to fail if the signer is not the admin of the group policy. @@ -426,7 +426,7 @@ It's expected to fail if the signer is not the admin of the group policy. The `UpdateGroupPolicyDecisionPolicy` can be used to update a decision policy. ```go -// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/proto/cosmos/group/v1/tx.proto#L226-L241 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/group/proto/cosmos/group/v1/tx.proto#L239-L254 ``` It's expected to fail if: @@ -439,7 +439,7 @@ It's expected to fail if: The `UpdateGroupPolicyMetadata` can be used to update a group policy metadata. ```go -// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/proto/cosmos/group/v1/tx.proto#L246-L259 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/group/proto/cosmos/group/v1/tx.proto#L259-L272 ``` It's expected to fail if: @@ -453,7 +453,7 @@ A new proposal can be created with the `MsgSubmitProposal`, which has a group po An optional `Exec` value can be provided to try to execute the proposal immediately after proposal creation. Proposers signatures are considered as yes votes in this case. ```go -// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/proto/cosmos/group/v1/tx.proto#L281-L315 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/group/proto/cosmos/group/v1/tx.proto#L294-L324 ``` It's expected to fail if: @@ -466,7 +466,7 @@ It's expected to fail if: A proposal can be withdrawn using `MsgWithdrawProposal` which has an `address` (can be either a proposer or the group policy admin) and a `proposal_id` (which has to be withdrawn). ```go -// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/proto/cosmos/group/v1/tx.proto#L323-L333 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/group/proto/cosmos/group/v1/tx.proto#L332-L342 ``` It's expected to fail if: @@ -480,7 +480,7 @@ A new vote can be created with the `MsgVote`, given a proposal id, a voter addre An optional `Exec` value can be provided to try to execute the proposal immediately after voting. ```go -// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/proto/cosmos/group/v1/tx.proto#L338-L358 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/group/proto/cosmos/group/v1/tx.proto#L347-L367 ``` It's expected to fail if: @@ -493,7 +493,7 @@ It's expected to fail if: A proposal can be executed with the `MsgExec`. ```go -// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/proto/cosmos/group/v1/tx.proto#L363-L373 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/group/proto/cosmos/group/v1/tx.proto#L372-L382 ``` The messages that are part of this proposal won't be executed if: @@ -506,7 +506,7 @@ The messages that are part of this proposal won't be executed if: The `MsgLeaveGroup` allows group member to leave a group. ```go -// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/proto/cosmos/group/v1/tx.proto#L381-L391 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/enterprise/group/proto/cosmos/group/v1/tx.proto#L390-L400 ``` It's expected to fail if: diff --git a/sdk/next/modules/mint/README.mdx b/sdk/next/modules/mint/README.mdx index f0b8180a1..8129f11a9 100644 --- a/sdk/next/modules/mint/README.mdx +++ b/sdk/next/modules/mint/README.mdx @@ -144,7 +144,7 @@ The minter is a space for holding current inflation information. * Minter: `0x00 -> ProtocolBuffer(minter)` ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/mint/v1beta1/mint.proto#L10-L24 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/mint/v1beta1/mint.proto#L10-L24 ``` ### Params @@ -154,10 +154,10 @@ it can be updated with governance or the address with authority. **Note:** The `MaxSupply` parameter controls the maximum supply of tokens the module can mint. A value of `0` indicates an unlimited supply. -* Params: `mint/params -> legacy_amino(params)` +* Params: `0x01 -> ProtocolBuffer(Params)` ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/mint/v1beta1/mint.proto#L26-L59 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/mint/v1beta1/mint.proto#L26-L71 ``` ## Begin-Block diff --git a/sdk/next/modules/modules.mdx b/sdk/next/modules/modules.mdx index 1303b54a3..6beb573b5 100644 --- a/sdk/next/modules/modules.mdx +++ b/sdk/next/modules/modules.mdx @@ -38,12 +38,11 @@ capabilities of your blockchain or further specialize it. * [Feegrant](/sdk/next/modules/feegrant/README) - Grant fee allowances for executing transactions. * [Group](/sdk/next/modules/group/README) - Allows for the creation and management of on-chain multisig accounts. * [NFT](/sdk/next/modules/nft/README) - NFT module implemented based on [ADR43](/sdk/next/reference/architecture/adr-043-nft-module). -* [ProtocolPool](/sdk/next/modules/protocolpool/README) - Extended management of community pool functionality. ## Deprecated Modules The following modules are deprecated. They will no longer be maintained and eventually will be removed -in an upcoming release of the Cosmos SDK per our [release process](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/RELEASE_PROCESS.md). +in an upcoming release of the Cosmos SDK per the [release family lifecycle](/sdk/next/release-family). * [Crisis](/sdk/next/modules/crisis/README) - *Deprecated* halting the blockchain under certain circumstances (e.g. if an invariant is broken). * [Params](/sdk/next/modules/params/README) - *Deprecated* Globally available parameter store. diff --git a/sdk/next/modules/nft/README.mdx b/sdk/next/modules/nft/README.mdx index 2d6af4d69..c7283bd0a 100644 --- a/sdk/next/modules/nft/README.mdx +++ b/sdk/next/modules/nft/README.mdx @@ -5,14 +5,14 @@ noindex: true --- -`x/nft` has been moved to [`./contrib/x/nft`](https://github.com/cosmos/cosmos-sdk/tree/main/contrib/x/nft) and is no longer actively maintained as part of the core Cosmos SDK. It is still available for use but is not included in the SDK Bug Bounty program. It was moved because it was never widely adopted. +`x/nft` has been moved to [`./contrib/x/nft`](https://github.com/cosmos/cosmos-sdk/tree/release/v0.55.x/contrib/x/nft) and is no longer actively maintained as part of the core Cosmos SDK. It is still available for use but is not included in the SDK Bug Bounty program. It was moved because it was never widely adopted. ## Contents ## Abstract -`x/nft` is an implementation of a Cosmos SDK module, per [ADR 43](https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-043-nft-module.md), that allows you to create nft classification, create nft, transfer nft, update nft, and support various queries by integrating the module. It is fully compatible with the ERC721 specification. +`x/nft` is an implementation of a Cosmos SDK module, per [ADR 43](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/docs/architecture/adr-043-nft-module.md), that allows you to create nft classification, create nft, transfer nft, update nft, and support various queries by integrating the module. It is fully compatible with the ERC721 specification. * [Concepts](#concepts) * [Class](#class) @@ -31,7 +31,7 @@ noindex: true ### Class -`x/nft` module defines a struct `Class` to describe the common characteristics of a class of nft, under this class, you can create a variety of nft, which is equivalent to an erc721 contract for Ethereum. The design is defined in the [ADR 043](https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-043-nft-module.md). +`x/nft` module defines a struct `Class` to describe the common characteristics of a class of nft, under this class, you can create a variety of nft, which is equivalent to an erc721 contract for Ethereum. The design is defined in the [ADR 043](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/docs/architecture/adr-043-nft-module.md). ### NFT diff --git a/sdk/next/modules/slashing/README.mdx b/sdk/next/modules/slashing/README.mdx index 216c59278..a38af1130 100644 --- a/sdk/next/modules/slashing/README.mdx +++ b/sdk/next/modules/slashing/README.mdx @@ -108,7 +108,7 @@ long as it contains precommits from +2/3 of total voting power. Proposers are incentivized to include precommits from all validators in the CometBFT `LastCommitInfo` by receiving additional fees proportional to the difference between the voting -power included in the `LastCommitInfo` and +2/3 (see [fee distribution](/sdk/v0.47/build/modules/distribution/README#begin-block)). +power included in the `LastCommitInfo` and +2/3 (see [fee distribution](/sdk/next/modules/distribution/README#begin-block)). ```go type LastCommitInfo struct { @@ -145,7 +145,7 @@ bonded validator. The `SignedBlocksWindow` parameter defines the size The information stored for tracking validator liveness is as follows: ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/slashing/v1beta1/slashing.proto#L13-L35 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/slashing/v1beta1/slashing.proto#L13-L35 ``` ### Params @@ -156,7 +156,7 @@ it can be updated with governance or the address with authority. * Params: `0x00 | ProtocolBuffer(Params)` ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/slashing/v1beta1/slashing.proto#L37-L59 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/slashing/v1beta1/slashing.proto#L37-L62 ``` ## Messages diff --git a/sdk/next/modules/staking/README.mdx b/sdk/next/modules/staking/README.mdx index c2b7e7a9b..44e07ce5b 100644 --- a/sdk/next/modules/staking/README.mdx +++ b/sdk/next/modules/staking/README.mdx @@ -47,6 +47,7 @@ network. * [MsgCancelUnbondingDelegation](#msgcancelunbondingdelegation) * [MsgBeginRedelegate](#msgbeginredelegate) * [MsgUpdateParams](#msgupdateparams) + * [MsgRotateConsPubKey](#msgrotateconspubkey) * [Begin-Block](#begin-block) * [Historical Info Tracking](#historical-info-tracking) * [End-Block](#end-block) @@ -96,7 +97,7 @@ it can be updated with governance or the address with authority. * Params: `0x51 | ProtocolBuffer(Params)` ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/staking.proto#L310-L333 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/staking/v1beta1/staking.proto#L298-L324 ``` ### Validator @@ -159,11 +160,11 @@ is updated during the validator set update process which takes place in [`EndBlo Each validator's state is stored in a `Validator` struct: ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/staking.proto#L82-L138 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/staking/v1beta1/staking.proto#L82-L136 ``` ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/staking.proto#L26-L80 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/staking/v1beta1/staking.proto#L26-L80 ``` ### Delegation @@ -179,7 +180,7 @@ delegator, and is associated with the shares for one validator. The sender of the transaction is the owner of the bond. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/staking.proto#L198-L216 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/staking/v1beta1/staking.proto#L191-L208 ``` #### Delegator Shares @@ -226,7 +227,7 @@ unbonding delegation entries. A UnbondingDelegation object is created every time an unbonding is initiated. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/staking.proto#L218-L261 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/staking/v1beta1/staking.proto#L210-L251 ``` ### Redelegation @@ -268,7 +269,7 @@ A redelegation object is created every time a redelegation occurs. To prevent where the source validator for this new redelegation is `Validator X`. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/staking.proto#L263-L308 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/staking/v1beta1/staking.proto#L253-L296 ``` ### Queues @@ -292,7 +293,7 @@ delegations queue is kept. * UnbondingDelegation: `0x41 | format(time) -> []DVPair` ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/staking.proto#L162-L172 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/staking/v1beta1/staking.proto#L157-L166 ``` #### RedelegationQueue @@ -303,7 +304,7 @@ kept. * RedelegationQueue: `0x42 | format(time) -> []DVVTriplet` ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/staking.proto#L179-L191 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/staking/v1beta1/staking.proto#L173-L184 ``` #### ValidatorQueue @@ -656,6 +657,8 @@ message Params { (gogoproto.customtype) = "github.com/cosmos/cosmos-sdk/types.Dec", (gogoproto.nullable) = false ]; + // key_rotation_fee is the fee charged when rotating a validator's consensus key. + cosmos.base.v1beta1.Coin key_rotation_fee = 7 [(gogoproto.nullable) = false, (amino.dont_omitempty) = true]; } // DelegationResponse is equivalent to Delegation except that it contains a @@ -737,6 +740,16 @@ they are in a deterministic order. The oldest HistoricalEntries will be pruned to ensure that there only exist the parameter-defined number of historical entries. +### Consensus key rotation + +Pending and historical consensus key rotations are tracked in five stores: + +* ConsKeyRotationQueue: `0x91 | format(maturityTime) | ValAddress` - rotations awaiting the end of their unbonding period, at which point the re-rotation rate limit is retired and the entry is pruned +* ValidatorConsKeyRotation: `0x92 | ValAddress` - a marker that the validator has rotated within the current unbonding period, enforcing the one-rotation-per-unbonding-period limit. The value is empty, and the entry is removed when the maturity queue retires it +* RotationLockedConsAddrIndex: `0x93 | ConsAddress` - consensus addresses a rotation has claimed, valued with a lock kind and the validator's operator address. A rotated-away address stays locked until equivocation evidence for it can no longer be admitted, and resolves back to the validator for slashing. A pending rotation's target address is reserved so no other validator can claim it, and that entry is released once the rotation applies in the end blocker +* ConsKeyRotationApplyQueue: `0x94 | BigEndian(applyHeight) | ValAddress` - height-keyed queue of rotations, valued with the new consensus public key, applied two heights after the rotation message +* ConsKeyEvidenceExpiryQueue: `0x95 | format(evidenceExpiryTime) | ConsAddress` - queue that retires an old address's lock once equivocation evidence for the rotated-away key can no longer be admitted, using the evidence time and block-height windows captured at rotation time + ## State Transitions ### Validators @@ -924,11 +937,11 @@ A validator is created using the `MsgCreateValidator` message. The validator must be created with an initial delegation from the operator. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/tx.proto#L20-L21 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/staking/v1beta1/tx.proto#L20-L21 ``` ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/tx.proto#L50-L73 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/staking/v1beta1/tx.proto#L55-L78 ``` This message is expected to fail if: @@ -953,11 +966,11 @@ The `Description`, `CommissionRate` of a validator can be updated using the `MsgEditValidator` message. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/tx.proto#L23-L24 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/staking/v1beta1/tx.proto#L23-L24 ``` ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/tx.proto#L78-L97 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/staking/v1beta1/tx.proto#L83-L102 ``` This message is expected to fail if: @@ -976,11 +989,11 @@ some amount of their validator's (newly created) delegator-shares that are assigned to `Delegation.Shares`. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/tx.proto#L26-L28 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/staking/v1beta1/tx.proto#L26-L28 ``` ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/tx.proto#L102-L114 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/staking/v1beta1/tx.proto#L107-L119 ``` This message is expected to fail if: @@ -1012,17 +1025,17 @@ The `MsgUndelegate` message allows delegators to undelegate their tokens from validator. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/tx.proto#L34-L36 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/staking/v1beta1/tx.proto#L34-L36 ``` ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/tx.proto#L140-L152 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/staking/v1beta1/tx.proto#L145-L157 ``` This message returns a response containing the completion time of the undelegation: ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/tx.proto#L154-L158 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/staking/v1beta1/tx.proto#L159-L167 ``` This message is expected to fail if: @@ -1051,11 +1064,11 @@ When this message is processed the following actions occur: The `MsgCancelUnbondingDelegation` message allows delegators to cancel the `unbondingDelegation` entry and delegate back to a previous validator. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/tx.proto#L38-L42 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/staking/v1beta1/tx.proto#L38-L42 ``` ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/tx.proto#L160-L175 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/staking/v1beta1/tx.proto#L169-L183 ``` This message is expected to fail if: @@ -1078,17 +1091,17 @@ the unbonding period has passed, the redelegation is automatically completed in the EndBlocker. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/tx.proto#L30-L32 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/staking/v1beta1/tx.proto#L30-L32 ``` ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/tx.proto#L119-L132 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/staking/v1beta1/tx.proto#L124-L137 ``` This message returns a response containing the completion time of the redelegation: ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/tx.proto#L133-L138 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/staking/v1beta1/tx.proto#L139-L143 ``` This message is expected to fail if: @@ -1120,7 +1133,7 @@ The `MsgUpdateParams` update the staking module parameters. The params are updated through a governance proposal where the signer is the gov module account address. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/tx.proto#L182-L195 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/staking/v1beta1/tx.proto#L190-L202 ``` The message handling can fail if: @@ -1128,6 +1141,28 @@ The message handling can fail if: * signer is not the authority defined in the staking keeper (usually the gov module account). * the `bond_denom` in the updated params has zero supply in the bank module (i.e., the denom does not exist on-chain). +### MsgRotateConsPubKey + +The `MsgRotateConsPubKey` message replaces a validator's consensus public key in place. The message is signed by the validator's operator address and carries the new public key. Power, delegations, commission, and the operator address are unchanged. + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/staking/v1beta1/tx.proto#L210-L220 +``` + +Handling the message burns the `KeyRotationFee` param from the operator account and enqueues the rotation. The new key enters the CometBFT validator set two heights after the message executes. The rotated-away consensus address is retained until equivocation evidence for it can no longer be admitted, which is at least the unbonding period and often longer, so evidence against the old key still slashes the validator. + +The message handling can fail if: + +* the fee cannot be deducted from the operator account. +* the validator already rotated within the current unbonding period. +* the new key's type is not in the chain's consensus params `validator.pub_key_types`. +* the validator does not exist. +* the validator is jailed. +* the new key is already in use by another validator. +* the new key is locked by a rotation, either because a validator rotated away from it and evidence against it can still be admitted, or because a pending rotation already targets it. + +For the operational procedure, see [Rotate a validator consensus key, x/staking](/sdk/next/keys/rotate-validator-key). For the concepts, see [Key rotation](/sdk/next/keys/key-rotation). + ## Begin-Block Each abci begin block call, the historical info will get stored and pruned @@ -1221,6 +1256,10 @@ Complete the unbonding of all mature `Redelegation.Entries` within the * remove the `Redelegation` object from the store if there are no remaining entries. +### Consensus Key Rotations + +At the height a rotation message executes, the end blocker emits the validator update handing the validator's power from the old consensus key to the new one. Two heights later, when CometBFT makes the update effective, the apply queue swaps the stored consensus key. When a rotation's unbonding period ends, the maturity queue retires the re-rotation rate limit and the validator may rotate again. The old consensus address stays locked on its own, longer schedule, until equivocation evidence for it can no longer be admitted. + ## Hooks Other modules may register operations to execute when a certain event has @@ -1343,11 +1382,14 @@ The staking module contains the following parameters: | Key | Type | Example | | ----------------- | ---------------- | ---------------------- | | UnbondingTime | string (time ns) | "259200000000000" | -| MaxValidators | uint16 | 100 | -| KeyMaxEntries | uint16 | 7 | -| HistoricalEntries | uint16 | 3 | +| MaxValidators | uint32 | 100 | +| MaxEntries | uint32 | 7 | +| HistoricalEntries | uint32 | 3 | | BondDenom | string | "stake" | | MinCommissionRate | string | "0.000000000000000000" | +| KeyRotationFee | sdk.Coin | `{"denom":"stake","amount":"1000000"}` | + +The limit of one consensus key rotation per unbonding period is fixed and is not a parameter. ## Client @@ -1555,8 +1597,12 @@ Example Output: ```bash bond_denom: stake historical_entries: 10000 +key_rotation_fee: + amount: "1000000" + denom: stake max_entries: 7 max_validators: 50 +min_commission_rate: "0.000000000000000000" unbonding_time: 1814400s ``` @@ -2089,6 +2135,22 @@ Example: simd tx staking cancel-unbond cosmosvaloper1gghjut3ccd8ay0zduzj64hwre2fxs9ldmqhffj 100stake 123123 --from mykey ``` +##### rotate-cons-pub-key + +The command `rotate-cons-pub-key` allows a validator operator to replace the validator's consensus public key. The new key is given as proto-JSON, and the validator address is derived from the `--from` signer. Handling the message burns the `key_rotation_fee` param. + +Usage: + +```bash +simd tx staking rotate-cons-pub-key [new-pubkey] [flags] +``` + +Example: + +```bash +simd tx staking rotate-cons-pub-key '{"@type":"/cosmos.crypto.ed25519.PubKey","key":"..."}' --from myvalidator +``` + ### gRPC A user can query the `staking` module using gRPC endpoints. @@ -2707,7 +2769,12 @@ Example Output: "maxValidators": 100, "maxEntries": 7, "historicalEntries": 10000, - "bondDenom": "stake" + "bondDenom": "stake", + "minCommissionRate": "0.000000000000000000", + "keyRotationFee": { + "denom": "stake", + "amount": "1000000" + } } } ``` @@ -2718,7 +2785,7 @@ A user can query the `staking` module using REST endpoints. #### DelegatorDelegations -The `DelegtaorDelegations` REST endpoint queries all delegations of a given delegator address. +The `DelegatorDelegations` REST endpoint queries all delegations of a given delegator address. ```bash /cosmos/staking/v1beta1/delegations/{delegatorAddr} diff --git a/sdk/next/modules/upgrade/README.mdx b/sdk/next/modules/upgrade/README.mdx index f3ebd2644..a690e80a0 100644 --- a/sdk/next/modules/upgrade/README.mdx +++ b/sdk/next/modules/upgrade/README.mdx @@ -52,7 +52,7 @@ type Plan struct { If an operator running the application binary also runs a sidecar process to assist in the automatic download and upgrade of a binary, the `Info` allows this process to -be seamless. This tool is [Cosmovisor](https://github.com/cosmos/cosmos-sdk/tree/main/tools/cosmovisor#readme). +be seamless. This tool is [Cosmovisor](https://github.com/cosmos/cosmos-sdk/tree/release/v0.55.x/tools/cosmovisor#readme). ### Handler @@ -104,7 +104,7 @@ the `Plan`, which targets a specific `Handler`, is persisted and scheduled. The upgrade can be delayed or hastened by updating the `Plan.Height` in a new proposal. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/upgrade/v1beta1/tx.proto#L29-L41 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/upgrade/v1beta1/tx.proto#L28-L39 ``` #### Cancelling Upgrade Proposals @@ -116,7 +116,7 @@ Of course this requires that the upgrade was known to be a bad idea well before upgrade itself, to allow time for a vote. ```protobuf -// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/upgrade/v1beta1/tx.proto#L48-L57 +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/proto/cosmos/upgrade/v1beta1/tx.proto#L46-L54 ``` If such a possibility is desired, the upgrade height is to be diff --git a/sdk/next/node/keyring.mdx b/sdk/next/node/keyring.mdx index 3a8d73765..6966ce33a 100644 --- a/sdk/next/node/keyring.mdx +++ b/sdk/next/node/keyring.mdx @@ -30,7 +30,7 @@ MY_VALIDATOR_ADDRESS=$(simd keys show my_validator -a --keyring-backend test) This generates a 24-word mnemonic phrase and stores your key. **Save the mnemonic** if you'll use this key for value-bearing tokens. -This tutorial uses the `test` backend (unencrypted, for testing only). For production, use the `os` backend which integrates with your system's secure keyring. See [keyring backends](#reference:-keyring-backends) below for more information. +This tutorial uses the `test` backend (unencrypted, for testing only). For production, use the `os` backend which integrates with your system's secure keyring. See [keyring backends](#reference-keyring-backends) below for more information. ## Next steps @@ -151,6 +151,6 @@ You can set the keyring-backend using an environment variable: `BINNAME_KEYRING_ ### Additional key management -By default, the keyring generates a `secp256k1` keypair. The keyring also supports `ed25519` keys, which may be created by passing the `--algo ed25519` flag. A keyring can hold both types of keys simultaneously, and the Cosmos SDK's `x/auth` module supports both public key algorithms natively. +By default, the keyring generates a `secp256k1` keypair. The keyring also supports `ml_dsa_65`, the post-quantum signature algorithm, selected with the `--key-type` flag. A keyring can hold both types of keys simultaneously. For the algorithm and its tradeoffs, see [Post-quantum keys](/sdk/next/keys/post-quantum-keys); to create and fund a post-quantum account, see [Create an ML-DSA account](/sdk/next/keys/create-ml-dsa-account). -For help with key management commands, use `simd keys --help` or `simd keys [command] --help`. +List the key types your binary supports with `simd keys list-key-types`. For help with key management commands, use `simd keys --help` or `simd keys [command] --help`. diff --git a/sdk/next/node/run-node.mdx b/sdk/next/node/run-node.mdx index 75d557ece..1ee9b2b84 100644 --- a/sdk/next/node/run-node.mdx +++ b/sdk/next/node/run-node.mdx @@ -6,7 +6,7 @@ title: Running a Node **Synopsis** -This section explains how to run a blockchain node. The application used in this tutorial is [`simapp`](https://github.com/cosmos/cosmos-sdk/tree/main/simapp), and its corresponding CLI binary `simd`. +This section explains how to run a blockchain node. The application used in this tutorial is [`simapp`](https://github.com/cosmos/cosmos-sdk/tree/release/v0.55.x/simapp), and its corresponding CLI binary `simd`. @@ -29,6 +29,8 @@ simd init --chain-id my-test-chain The command above creates all the configuration files needed for your node to run, as well as a default genesis file, which defines the initial state of the network. +The `init` command also selects the validator consensus key algorithm with the optional `--consensus-key-algo` flag. It defaults to `ed25519`. To initialize the node on a post-quantum key instead, see [the ML-DSA consensus key guides](/sdk/next/keys/post-quantum-keys). + All these configuration files are in `~/.simapp` by default, but you can overwrite the location of this folder by passing the `--home` flag to each command, or set an `$APPD_HOME` environment variable (where `APPD` is the name of the binary). @@ -53,7 +55,7 @@ The `~/.simapp` folder has the following structure: ## 2. Update configuration settings (optional) -To change field values in configuration files (for example, genesis.json), use `jq` ([installation](https://stedolan.github.io/jq/download/) & [docs](https://stedolan.github.io/jq/manual/#Assignment)) and `sed` commands. A few examples are listed here. +To change field values in configuration files (for example, genesis.json), use `jq` ([installation](https://jqlang.org/download/) & [docs](https://jqlang.org/manual/#assignment)) and `sed` commands. A few examples are listed here. ```bash expandable # to change the chain-id @@ -87,7 +89,7 @@ Now, you can grant this account some `stake` tokens in your chain's genesis file simd genesis add-genesis-account $MY_VALIDATOR_ADDRESS 100000000000stake ``` -Recall that `$MY_VALIDATOR_ADDRESS` is a variable that holds the address of the `my_validator` key in the [keyring](/sdk/next/node/keyring#create-a-key). Also note that the tokens in the Cosmos SDK have the `{amount}{denom}` format: `amount` is an 18-digit-precision decimal number, and `denom` is the unique token identifier with its denomination key (e.g., `atom` or `uatom`). Here, `stake` tokens are granted, as `stake` is the token identifier used for staking in [`simapp`](https://github.com/cosmos/cosmos-sdk/tree/main/simapp). For your own chain with its own staking denom, that token identifier should be used instead. +Recall that `$MY_VALIDATOR_ADDRESS` is a variable that holds the address of the `my_validator` key in the [keyring](/sdk/next/node/keyring#create-a-key). Also note that the tokens in the Cosmos SDK have the `{amount}{denom}` format: `amount` is an 18-digit-precision decimal number, and `denom` is the unique token identifier with its denomination key (e.g., `atom` or `uatom`). Here, `stake` tokens are granted, as `stake` is the token identifier used for staking in [`simapp`](https://github.com/cosmos/cosmos-sdk/tree/release/v0.55.x/simapp). For your own chain with its own staking denom, that token identifier should be used instead. ## 4. Create genesis transaction @@ -122,16 +124,16 @@ simd genesis gentx --help The Cosmos SDK automatically generates two configuration files inside `~/.simapp/config`: * `config.toml`: used to configure the CometBFT, learn more on [CometBFT's documentation](/cometbft/latest/docs/core/configuration), -* `app.toml`: generated by the Cosmos SDK, and used to configure your app, such as state pruning strategies, telemetry, gRPC and REST server configuration, state sync... +* `app.toml`: generated by the Cosmos SDK, and used to configure your app, such as state pruning strategies, telemetry, gRPC and REST server configuration, state sync, etc. See the [default `app.toml` template](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/server/config/toml.go) for every field and its inline documentation. -Both files are heavily commented, please refer to them directly to tweak your node. +Both files are heavily commented, please refer to them directly to tweak your node. Each field carries an inline comment that explains it. This includes operational settings such as `query-gas-limit` that matter for nodes serving public RPC. One example config to tweak is the `minimum-gas-prices` field inside `app.toml`, which defines the minimum gas prices the validator node is willing to accept for processing a transaction. Depending on the chain, it might be an empty string or not. If it's empty, make sure to edit the field with some value, for example `10token`, or else the node will halt on startup. For the purposes of this tutorial, the minimum gas price is set to 0: ```toml # The minimum gas prices a validator is willing to accept for processing a # transaction. A transaction's fees must meet the minimum of any denomination - # specified in this config (e.g. 0.25token1;0.0001token2). + # specified in this config (e.g. 0.25token1,0.0001token2). minimum-gas-prices = "0stake" ``` @@ -141,7 +143,7 @@ When running a node (not a validator!) and not wanting to run the application me ```toml [mempool] # Setting max-txs to 0 will allow for an unbounded amount of transactions in the mempool. -# Setting max_txs to negative 1 (-1) will disable transactions from being inserted into the mempool. +# Setting max_txs to negative 1 (-1) will disable transactions from being inserted into the mempool (no-op mempool). # Setting max_txs to a positive number (> 0) will limit the number of transactions in the mempool, by the specified amount. # # Note, this configuration only applies to SDK built-in app-side mempool @@ -163,7 +165,7 @@ You should see blocks come in. ### What happens when the node starts -The `start` command (defined in [`server/start.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/server/start.go)) boots up the full-node in the following sequence: +The `start` command (defined in [`server/start.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/server/start.go)) boots up the full-node in the following sequence: 1. It opens the `db` (LevelDB by default) containing the latest persisted state. On first start, this is empty. 2. It creates a new instance of the application via an `appCreator` function, which is the [application constructor](/sdk/next/learn/intro/sdk-app-architecture#constructor-function). @@ -172,7 +174,7 @@ The `start` command (defined in [`server/start.go`](https://github.com/cosmos/co The previous command allows you to run a single node. This is enough for the next section on interacting with this node, but you may wish to run multiple nodes at the same time, and see how consensus happens between them. -The naive way would be to run the same commands again in separate terminal windows. This is possible. However, [Docker Compose](https://docs.docker.com/compose/) can be leveraged to run a localnet. If you need inspiration on how to set up your own localnet with Docker Compose, refer to the Cosmos SDK's [`docker-compose.yml`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/docker-compose.yml). +The naive way would be to run the same commands again in separate terminal windows. This is possible. However, [Docker Compose](https://docs.docker.com/compose/) can be leveraged to run a localnet. If you need inspiration on how to set up your own localnet with Docker Compose, refer to the Cosmos SDK's [`docker-compose.yml`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/docker-compose.yml). ### Standalone App/CometBFT @@ -205,15 +207,34 @@ See the [Log Overview](/sdk/next/guides/testing/log) for more information on log State sync is the act in which a node syncs the latest or close to the latest state of a blockchain. This is useful for users who don't want to sync all the blocks in history. Read more in [CometBFT documentation](/cometbft/latest/docs/core/state-sync). -State sync works thanks to snapshots. Read how the SDK handles snapshots [here](https://github.com/cosmos/cosmos-sdk/blob/825245d/store/snapshots/README.md). +State sync works thanks to snapshots. For how the SDK produces and stores them, see the [store/snapshots README](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/store/snapshots/README.md). + +### Produce and serve snapshots + +A node serves snapshots to state-syncing peers only after it takes them. Configure snapshots in `app.toml` under `[state-sync]`: + +```toml +[state-sync] +# Block interval at which the node takes a local snapshot (0 disables snapshots). +snapshot-interval = 1000 + +# Number of recent snapshots to keep and serve (0 keeps all). +snapshot-keep-recent = 2 +``` + +A node with `snapshot-interval = 0` takes no snapshots and cannot serve state sync. The `snapshot-keep-recent` setting bounds how many snapshots the node retains and offers to peers. Restart the node after changing these values. + +Snapshots work alongside pruning. When pruning is enabled, the SDK retains heights that are multiples of `snapshot-interval` until the snapshot at that height completes, so a pruning node can still produce snapshots. + +To take a snapshot on demand instead of waiting for the interval, run ` snapshots export`. It snapshots the latest committed height by default, or the height set with `--height`. This command opens the application database directly, so stop the node before running it. The automatic `snapshot-interval` is the only way to produce snapshots without downtime. ### Local State Sync Local state sync works similarly to normal state sync except that it works off a local snapshot of state instead of one provided via the p2p network. The steps to start local state sync are similar to normal state sync with a few different design considerations. 1. As mentioned in the [state sync documentation](/cometbft/latest/docs/core/state-sync), one must set a height and hash in the config.toml along with a few RPC servers (the aforementioned link has instructions on how to do this). -2. Run ` snapshot restore ` to restore a local snapshot (note: first load it from a file with the *load* command). -3. Bootstrapping Comet state to start the node after the snapshot has been ingested. This can be done with the bootstrap command ` comet bootstrap-state` +2. Run ` snapshots restore ` to restore a local snapshot (first load it from a file with ` snapshots load`). +3. Bootstrap Comet state to start the node after the snapshot has been ingested. Run ` comet bootstrap-state`. ### Snapshots Commands @@ -250,4 +271,6 @@ Your node is now running and producing blocks. You have successfully initialized ## Next steps - [Interact with the node](/sdk/next/node/interact-node) to send transactions and query state -- [Generate and sign transactions](/sdk/next/node/txs) to learn advanced transaction workflows \ No newline at end of file +- [Generate and sign transactions](/sdk/next/node/txs) to learn advanced transaction workflows +- [Key rotation](/sdk/next/keys/key-rotation) to understand the consensus key in `priv_validator_key.json` and how a validator replaces it +- [Cosmos-KMS and remote signing](/sdk/next/kms/remote-signing) to move that key off the node entirely \ No newline at end of file diff --git a/sdk/next/node/run-production.mdx b/sdk/next/node/run-production.mdx index 1900a2701..aa7d2b46f 100644 --- a/sdk/next/node/run-production.mdx +++ b/sdk/next/node/run-production.mdx @@ -118,151 +118,14 @@ If the node that is being started is a validator there are multiple ways a valid #### File -File-based signing is the simplest and default approach. This approach works by storing the consensus key generated on initialization to sign blocks. This approach is only as safe as your server setup, as if the server is compromised, so is your key. This key is located in the `config/priv_val_key.json` directory generated on initialization. +File-based signing is the simplest and default approach. This approach works by storing the consensus key generated on initialization to sign blocks. This approach is only as safe as your server setup, as if the server is compromised, so is your key. This key is located in the `config/priv_validator_key.json` file generated on initialization. -A second file exists that users must be aware of; the file is located in the data directory `data/priv_val_state.json`. This file protects your node from double signing. It keeps track of the consensus key's last sign height, round, and latest signature. If the node crashes and needs to be recovered, this file must be kept in order to ensure that the consensus key will not be used for signing a block that was previously signed. +A second file exists that users must be aware of; the file is located in the data directory `data/priv_validator_state.json`. This file protects your node from double signing. It keeps track of the consensus key's last sign height, round, and latest signature. If the node crashes and needs to be recovered, this file must be kept in order to ensure that the consensus key will not be used for signing a block that was previously signed. -#### Remote Signer +#### Remote signer A remote signer is a secondary server that is separate from the running node that signs blocks with the consensus key. This means that the consensus key does not live on the node itself. This increases security because your full node which is connected to the remote signer can be swapped without missing blocks. -The two most used remote signers are [tmkms](https://github.com/iqlusioninc/tmkms) from [Iqlusion](https://www.iqlusion.io) and [horcrux](https://github.com/strangelove-ventures/horcrux) from [Strangelove](https://strange.love). +The Cosmos stack's remote signer is Cosmos-KMS, which holds the consensus key in a file, a PKCS#11 HSM, or AWS KMS. For what remote signing is and how it works, see [Cosmos-KMS and remote signing](/sdk/next/kms/remote-signing). To set a signer up end to end, follow the [remote signing tutorial](/sdk/next/kms/tutorial-file-backend), then harden the setup with [remote signing best practices](/sdk/next/kms/best-practices). -##### TMKMS - -###### Dependencies - -1. Update server dependencies and install extras needed. - -```sh -sudo apt update -y && sudo apt install build-essential curl jq -y -``` - -2. Install Rust: - -```sh -curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -``` - -3. Install Libusb: - -```sh -sudo apt install libusb-1.0-0-dev -``` - -###### Setup - -There are two ways to install tmkms, from source or `cargo install`. In the examples we will cover downloading or building from source and using softsign. Softsign stands for software signing, but you could use a [yubihsm](https://www.yubico.com/products/hardware-security-module/) as your signing key if you wish. - -1. Build: - -From source: - -```bash -cd $HOME -git clone https://github.com/iqlusioninc/tmkms.git -cd $HOME/tmkms -cargo install tmkms --features=softsign -tmkms init config -tmkms softsign keygen ./config/secrets/secret_connection_key -``` - -or - -Cargo install: - -```bash -cargo install tmkms --features=softsign -tmkms init config -tmkms softsign keygen ./config/secrets/secret_connection_key -``` - - -To use tmkms with a yubikey install the binary with `--features=yubihsm`. - - -2. Migrate the validator key from the full node to the new tmkms instance. - -```bash -scp user@123.456.32.123:~/.simd/config/priv_validator_key.json ~/tmkms/config/secrets -``` - -3. Import the validator key into tmkms. - -```bash -tmkms softsign import $HOME/tmkms/config/secrets/priv_validator_key.json $HOME/tmkms/config/secrets/priv_validator_key -``` - -At this point, it is necessary to delete the `priv_validator_key.json` from the validator node and the tmkms node. Since the key has been imported into tmkms (above) it is no longer necessary on the nodes. The key can be safely stored offline. - -4. Modify the `tmkms.toml`. - -```bash -vim $HOME/tmkms/config/tmkms.toml -``` - -This example shows a configuration that could be used for soft signing. The example has an IP of `123.456.12.345` with a port of `26659` and a chain\_id of `test-chain-waSDSe`. These are items that must be modified for the use case of tmkms and the network. - -```toml expandable -# CometBFT KMS configuration file - -## Chain Configuration - -[[chain]] -id = "osmosis-1" -key_format = { type = "bech32", account_key_prefix = "cosmospub", consensus_key_prefix = "cosmosvalconspub" } -state_file = "/root/tmkms/config/state/priv_validator_state.json" - -## Signing Provider Configuration - -### Software-based Signer Configuration - -[[providers.softsign]] -chain_ids = ["test-chain-waSDSe"] -key_type = "consensus" -path = "/root/tmkms/config/secrets/priv_validator_key" - -## Validator Configuration - -[[validator]] -chain_id = "test-chain-waSDSe" -addr = "tcp://123.456.12.345:26659" -secret_key = "/root/tmkms/config/secrets/secret_connection_key" -protocol_version = "v0.34" -reconnect = true -``` - -5. Set the address of the tmkms instance. - -```bash -vim $HOME/.simd/config/config.toml - -priv_validator_laddr = "tcp://0.0.0.0:26659" -``` - - -The above address is set to `0.0.0.0`, but it is recommended to set the tmkms server address to secure the startup. - - - -It is recommended to comment or delete the lines that specify the path of the validator key and validator: - -```toml -# Path to the JSON file containing the private key to use as a validator in the consensus protocol -# priv_validator_key_file = "config/priv_validator_key.json" - -# Path to the JSON file containing the last sign state of a validator -# priv_validator_state_file = "data/priv_validator_state.json" -``` - - - -6. Start the two processes. - -```bash -tmkms start -c $HOME/tmkms/config/tmkms.toml -``` - -```bash -simd start -``` +TMKMS is the previous remote signer; Cosmos-KMS is the recommended one going forward. Validators running TMKMS should [migrate](/sdk/next/kms/migrate-from-tmkms). diff --git a/sdk/next/node/txs.mdx b/sdk/next/node/txs.mdx index 86748185b..ba6a5d80c 100644 --- a/sdk/next/node/txs.mdx +++ b/sdk/next/node/txs.mdx @@ -333,7 +333,7 @@ txb.SetTimeoutTimestamp(time.Now().Add(expiration + (1 * time.Nanosecond))) ### Signing a Transaction -The encoding config is set to use Protobuf, which will use `SIGN_MODE_DIRECT` by default. As per [ADR-020](https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-020-protobuf-transaction-encoding.md), each signer needs to sign the `SignerInfo`s of all other signers. This means that two steps must be performed sequentially: +The encoding config is set to use Protobuf, which will use `SIGN_MODE_DIRECT` by default. As per [ADR-020](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/docs/architecture/adr-020-protobuf-transaction-encoding.md), each signer needs to sign the `SignerInfo`s of all other signers. This means that two steps must be performed sequentially: * for each signer, populate the signer's `SignerInfo` inside `TxBuilder` * once all `SignerInfo`s are populated, for each signer, sign the `SignDoc` (the payload to be signed). diff --git a/sdk/next/reference/architecture.mdx b/sdk/next/reference/architecture.mdx index 037c22c2b..20481eb01 100644 --- a/sdk/next/reference/architecture.mdx +++ b/sdk/next/reference/architecture.mdx @@ -1,6 +1,6 @@ --- title: "Architecture Decision Records (ADR)" -description: "Version: v0.54" +description: "Version: v0.55" noindex: true --- diff --git a/sdk/next/reference/architecture/adr-009-evidence-module.mdx b/sdk/next/reference/architecture/adr-009-evidence-module.mdx index c9539be50..c8eae5e47 100644 --- a/sdk/next/reference/architecture/adr-009-evidence-module.mdx +++ b/sdk/next/reference/architecture/adr-009-evidence-module.mdx @@ -21,7 +21,7 @@ evidence can be submitted, evaluated and verified resulting in some agreed upon penalty for any misbehavior committed by a validator, such as equivocation (double-voting), signing when unbonded, signing an incorrect state transition (in the future), etc. Furthermore, such a mechanism is paramount for any -[IBC](https://github.com/cosmos/ics/blob/master/ibc/2_IBC_ARCHITECTURE.md) or +IBC (`https://github.com/cosmos/ics/blob/master/ibc/2_IBC_ARCHITECTURE.md`) or cross-chain validation protocol implementation in order to support the ability for any misbehavior to be relayed back from a collateralized chain to a primary chain so that the equivocating validator(s) can be slashed. @@ -215,5 +215,5 @@ type GenesisState struct { ## References * [ICS](https://github.com/cosmos/ics) -* [IBC Architecture](https://github.com/cosmos/ics/blob/master/ibc/1_IBC_ARCHITECTURE.md) +* IBC Architecture: `https://github.com/cosmos/ics/blob/master/ibc/1_IBC_ARCHITECTURE.md` * [Tendermint Fork Accountability](https://github.com/tendermint/spec/blob/7b3138e69490f410768d9b1ffc7a17abc23ea397/spec/consensus/fork-accountability.md) diff --git a/sdk/next/reference/architecture/adr-030-authz-module.mdx b/sdk/next/reference/architecture/adr-030-authz-module.mdx index 8710c6705..5ea64949e 100644 --- a/sdk/next/reference/architecture/adr-030-authz-module.mdx +++ b/sdk/next/reference/architecture/adr-030-authz-module.mdx @@ -28,7 +28,7 @@ The concrete use cases which motivated this module include: delegated stake * "sub-keys" functionality, as originally proposed in [#4480](https://github.com/cosmos/cosmos-sdk/issues/4480) which is a term used to describe the functionality provided by this module together with - the `fee_grant` module from [ADR 029](/sdk/v0.50/build/architecture/adr-029-fee-grant-module) and the [group module](https://github.com/cosmos/cosmos-sdk/tree/main/x/group). + the `fee_grant` module from [ADR 029](/sdk/v0.50/build/architecture/adr-029-fee-grant-module) and the [group module](https://github.com/cosmos/cosmos-sdk/tree/release/v0.53.x/x/group). The "sub-keys" functionality roughly refers to the ability for one account to grant some subset of its capabilities to other accounts with possibly less robust, but easier to use security measures. For instance, a master account representing diff --git a/sdk/next/reference/architecture/adr-038-state-listening.mdx b/sdk/next/reference/architecture/adr-038-state-listening.mdx index c67ec4b8b..7c8576ee0 100644 --- a/sdk/next/reference/architecture/adr-038-state-listening.mdx +++ b/sdk/next/reference/architecture/adr-038-state-listening.mdx @@ -23,7 +23,7 @@ This ADR defines a set of changes to enable listening to state changes of indivi ## Context -Currently, KVStore data can be remotely accessed through [Queries](https://github.com/cosmos/cosmos-sdk/blob/master/docs/building-modules/messages-and-queries.md#queries) +Currently, KVStore data can be remotely accessed through [Queries](https://github.com/cosmos/cosmos-sdk/blob/release/v0.46.x/docs/building-modules/messages-and-queries.md#queries) which proceed either through Tendermint and the ABCI, or through the gRPC server. In addition to these request/response queries, it would be beneficial to have a means of listening to state changes as they occur in real time. diff --git a/sdk/next/reference/architecture/adr-042-group-module.mdx b/sdk/next/reference/architecture/adr-042-group-module.mdx index 8cfd07f77..c81017662 100644 --- a/sdk/next/reference/architecture/adr-042-group-module.mdx +++ b/sdk/next/reference/architecture/adr-042-group-module.mdx @@ -28,11 +28,11 @@ The legacy amino multi-signature mechanism of the Cosmos SDK has certain limitat While the group module is not meant to be a total replacement for the current multi-signature accounts, it provides a solution to the limitations described above, with a more flexible key management system where keys can be added, updated or removed, as well as configurable thresholds. It's meant to be used with other access control modules such as [`x/feegrant`](/sdk/v0.50/build/architecture/adr-029-fee-grant-module) ans [`x/authz`](/sdk/next/reference/architecture/adr-030-authz-module) to simplify key management for individuals and organizations. -The proof of concept of the group module can be found in [Link](https://github.com/regen-network/regen-ledger/tree/master/proto/regen/group/v1alpha1) and [Link](https://github.com/regen-network/regen-ledger/tree/master/x/group). +The proof of concept of the group module can be found in `https://github.com/regen-network/regen-ledger/tree/master/proto/regen/group/v1alpha1` and `https://github.com/regen-network/regen-ledger/tree/master/x/group`. ## Decision -We propose merging the `x/group` module with its supporting [ORM/Table Store package](https://github.com/regen-network/regen-ledger/tree/master/orm) ([#7098](https://github.com/cosmos/cosmos-sdk/issues/7098)) into the Cosmos SDK and continuing development here. There will be a dedicated ADR for the ORM package. +We propose merging the `x/group` module with its supporting ORM/Table Store package (`https://github.com/regen-network/regen-ledger/tree/master/orm`) ([#7098](https://github.com/cosmos/cosmos-sdk/issues/7098)) into the Cosmos SDK and continuing development here. There will be a dedicated ADR for the ORM package. ### Group diff --git a/sdk/next/reference/architecture/adr-050-sign-mode-textual-annex1.mdx b/sdk/next/reference/architecture/adr-050-sign-mode-textual-annex1.mdx index eb8e5ab83..dcbf77f1a 100644 --- a/sdk/next/reference/architecture/adr-050-sign-mode-textual-annex1.mdx +++ b/sdk/next/reference/architecture/adr-050-sign-mode-textual-annex1.mdx @@ -13,7 +13,7 @@ noindex: true ## Status -Accepted. Implementation started. Small value renderers details still need to be polished. +Archived. `SIGN_MODE_TEXTUAL` was removed in Cosmos SDK v0.55, and the proto enum value is reserved. This ADR is retained for historical reference. ## Abstract @@ -66,7 +66,7 @@ Value Renderers describe how values of different Protobuf types should be encode ### `repeated` -* Applies to all `repeated` fields, except `cosmos.tx.v1beta1.TxBody#Messages`, which has a particular encoding (see [ADR-050](/sdk/v0.50/build/architecture/adr-050-sign-mode-textual)). +* Applies to all `repeated` fields, except `cosmos.tx.v1beta1.TxBody#Messages`, which has a particular encoding (see [ADR-050](/sdk/next/reference/architecture/adr-050-sign-mode-textual)). * A repeated type has the following template: ``` @@ -283,7 +283,7 @@ The number 35 was chosen because it is the longest length where the hashed-and-p * byte arrays starting from length 36 will be be hashed to 32 bytes, which is 64 hex characters plus 15 spaces, and with the `SHA-256=` prefix, it takes 87 characters. Also, secp256k1 public keys have length 33, so their Textual representation is not their hashed value, which we would like to avoid. -Note: Data longer than 35 bytes are not rendered in a way that can be inverted. See ADR-050's [section about invertability](/sdk/v0.50/build/architecture/adr-050-sign-mode-textual#invertible-rendering) for a discussion. +Note: Data longer than 35 bytes are not rendered in a way that can be inverted. See ADR-050's [section about invertibility](/sdk/next/reference/architecture/adr-050-sign-mode-textual#invertible-rendering) for a discussion. #### Examples diff --git a/sdk/next/reference/architecture/adr-050-sign-mode-textual-annex2.mdx b/sdk/next/reference/architecture/adr-050-sign-mode-textual-annex2.mdx index 167c6078a..e85664abc 100644 --- a/sdk/next/reference/architecture/adr-050-sign-mode-textual-annex2.mdx +++ b/sdk/next/reference/architecture/adr-050-sign-mode-textual-annex2.mdx @@ -1,6 +1,6 @@ --- -title: 'ADR 050: SIGN_MODE_TEXTUAL: Annex 2 XXX' -description: 'Oct 3, 2022: Initial Draft' +title: 'ADR 050: SIGN_MODE_TEXTUAL: Annex 2 Device Rendering' +description: 'Normative guidance on how hardware devices should render a SIGN_MODE_TEXTUAL document.' noindex: true --- @@ -10,7 +10,7 @@ noindex: true ## Status -DRAFT +Archived. `SIGN_MODE_TEXTUAL` was removed in Cosmos SDK v0.55, and the proto enum value is reserved. This ADR is retained for historical reference. ## Abstract diff --git a/sdk/next/reference/architecture/adr-050-sign-mode-textual.mdx b/sdk/next/reference/architecture/adr-050-sign-mode-textual.mdx index f1f1caa13..376beef09 100644 --- a/sdk/next/reference/architecture/adr-050-sign-mode-textual.mdx +++ b/sdk/next/reference/architecture/adr-050-sign-mode-textual.mdx @@ -21,7 +21,7 @@ noindex: true ## Status -Accepted. Implementation started. Small value renderers details still need to be polished. +Archived. `SIGN_MODE_TEXTUAL` was removed in Cosmos SDK v0.55, and the proto enum value is reserved. This ADR is retained for historical reference. Spec version: 0. @@ -31,7 +31,7 @@ This ADR specifies SIGN\_MODE\_TEXTUAL, a new string-based sign mode that is tar ## Context -Protobuf-based SIGN\_MODE\_DIRECT was introduced in [ADR-020](/sdk/v0.50/build/architecture/adr-020-protobuf-transaction-encoding) and is intended to replace SIGN\_MODE\_LEGACY\_AMINO\_JSON in most situations, such as mobile wallets and CLI keyrings. However, the [Ledger](https://www.ledger.com/) hardware wallet is still using SIGN\_MODE\_LEGACY\_AMINO\_JSON for displaying the sign bytes to the user. Hardware wallets cannot transition to SIGN\_MODE\_DIRECT as: +Protobuf-based SIGN\_MODE\_DIRECT was introduced in [ADR-020](/sdk/next/reference/architecture/adr-020-protobuf-transaction-encoding) and is intended to replace SIGN\_MODE\_LEGACY\_AMINO\_JSON in most situations, such as mobile wallets and CLI keyrings. However, the [Ledger](https://www.ledger.com/) hardware wallet is still using SIGN\_MODE\_LEGACY\_AMINO\_JSON for displaying the sign bytes to the user. Hardware wallets cannot transition to SIGN\_MODE\_DIRECT as: * SIGN\_MODE\_DIRECT is binary-based and thus not suitable for display to end-users. Technically, hardware wallets could simply display the sign bytes to the user. But this would be considered as blind signing, and is a security concern. * hardware cannot decode the protobuf sign bytes due to memory constraints, as the Protobuf definitions would need to be embedded on the hardware device. @@ -57,7 +57,7 @@ or to introduce or conclude a larger grouping. The text can contain the full range of Unicode code points, including control characters and nul. The device is responsible for deciding how to display characters it cannot render natively. -See [annex 2](/sdk/v0.50/build/architecture/adr-050-sign-mode-textual-annex2) for guidance. +See [annex 2](/sdk/next/reference/architecture/adr-050-sign-mode-textual-annex2) for guidance. Screens have a non-negative indentation level to signal composite or nested structures. Indentation level zero is the top level. @@ -288,7 +288,7 @@ Moreover, the renderer must provide 2 functions: one for formatting from Protobu ### Require signing over the `TxBody` and `AuthInfo` raw bytes -Recall that the transaction bytes merklelized on chain are the Protobuf binary serialization of [TxRaw](hhttps://buf.build/cosmos/cosmos-sdk/sdk/v0.50/main:cosmos.tx.v1beta1#cosmos.tx.v1beta1.TxRaw), which contains the `body_bytes` and `auth_info_bytes`. Moreover, the transaction hash is defined as the SHA256 hash of the `TxRaw` bytes. We require that the user signs over these bytes in SIGN\_MODE\_TEXTUAL, more specifically over the following string: +Recall that the transaction bytes merklelized on chain are the Protobuf binary serialization of [TxRaw](https://buf.build/cosmos/cosmos-sdk/sdk/v0.50/main:cosmos.tx.v1beta1#cosmos.tx.v1beta1.TxRaw), which contains the `body_bytes` and `auth_info_bytes`. Moreover, the transaction hash is defined as the SHA256 hash of the `TxRaw` bytes. We require that the user signs over these bytes in SIGN\_MODE\_TEXTUAL, more specifically over the following string: ``` *Hash of raw bytes: @@ -302,7 +302,7 @@ where: This is to prevent transaction hash malleability. The point #1 about invertiblity assures that transaction `body` and `auth_info` values are not malleable, but the transaction hash still might be malleable with point #1 only, because the SIGN\_MODE\_TEXTUAL strings don't follow the byte ordering defined in `body_bytes` and `auth_info_bytes`. Without this hash, a malicious validator or exchange could intercept a transaction, modify its transaction hash *after* the user signed it using SIGN\_MODE\_TEXTUAL (by tweaking the byte ordering inside `body_bytes` or `auth_info_bytes`), and then submit it to Tendermint. -By including this hash in the SIGN\_MODE\_TEXTUAL signing payload, we keep the same level of guarantees as [SIGN\_MODE\_DIRECT](/sdk/v0.50/build/architecture/adr-020-protobuf-transaction-encoding). +By including this hash in the SIGN\_MODE\_TEXTUAL signing payload, we keep the same level of guarantees as [SIGN\_MODE\_DIRECT](/sdk/next/reference/architecture/adr-020-protobuf-transaction-encoding). These bytes are only shown in expert mode, hence the leading `*`. @@ -323,7 +323,7 @@ The current spec version is defined in the "Status" section, on the top of this ## Additional Formatting by the Hardware Device -See [annex 2](/sdk/v0.50/build/architecture/adr-050-sign-mode-textual-annex2). +See [annex 2](/sdk/next/reference/architecture/adr-050-sign-mode-textual-annex2). ## Examples @@ -358,14 +358,14 @@ SIGN\_MODE\_TEXTUAL is purely additive, and doesn't break any backwards compatib ## Further Discussions -* Some details on value renderers need to be polished, see [Annex 1](/sdk/v0.50/build/architecture/adr-050-sign-mode-textual-annex1). +* Some details on value renderers need to be polished, see [Annex 1](/sdk/next/reference/architecture/adr-050-sign-mode-textual-annex1). * Are ledger apps able to support both SIGN\_MODE\_LEGACY\_AMINO\_JSON and SIGN\_MODE\_TEXTUAL at the same time? * Open question: should we add a Protobuf field option to allow app developers to overwrite the textual representation of certain Protobuf fields and message? This would be similar to Ethereum's [EIP4430](https://github.com/ethereum/EIPs/pull/4430), where the contract developer decides on the textual representation. * Internationalization. ## References -* [Annex 1](/sdk/v0.50/build/architecture/adr-050-sign-mode-textual-annex1) +* [Annex 1](/sdk/next/reference/architecture/adr-050-sign-mode-textual-annex1) * Initial discussion: [Link](https://github.com/cosmos/cosmos-sdk/issues/6513) diff --git a/sdk/next/reference/architecture/adr-062-collections-state-layer.mdx b/sdk/next/reference/architecture/adr-062-collections-state-layer.mdx index c3ce9dcd8..3645d06f4 100644 --- a/sdk/next/reference/architecture/adr-062-collections-state-layer.mdx +++ b/sdk/next/reference/architecture/adr-062-collections-state-layer.mdx @@ -84,7 +84,7 @@ These default implementations also offer safety around proper lexicographic orde Examples of the collections API can be found here: * introduction: [Link](https://github.com/NibiruChain/collections/tree/main/examples) -* usage in nibiru: [x/oracle](https://github.com/NibiruChain/nibiru/blob/master/x/oracle/keeper/keeper.go#L32), [x/perp](https://github.com/NibiruChain/nibiru/blob/master/x/perp/keeper/keeper.go#L31) +* usage in nibiru: [x/oracle](https://github.com/NibiruChain/nibiru/blob/master/x/oracle/keeper/keeper.go#L32), x/perp (`https://github.com/NibiruChain/nibiru/blob/master/x/perp/keeper/keeper.go#L31`) * cosmos-sdk's x/staking migrated: [Link](https://github.com/testinginprod/cosmos-sdk/pull/22) ## Consequences diff --git a/sdk/next/reference/architecture/adr-template.mdx b/sdk/next/reference/architecture/adr-template.mdx deleted file mode 100644 index f6e3eb89e..000000000 --- a/sdk/next/reference/architecture/adr-template.mdx +++ /dev/null @@ -1,86 +0,0 @@ ---- -noindex: true -canonical: 'https://docs.cosmos.network/sdk/latest/reference/architecture/adr-template' ---- -## Changelog - -* `{date}`: `{changelog}` - -## Status - -{DRAFT | PROPOSED} Not Implemented - -> Please have a look at the [PROCESS](/sdk/v0.50/build/rfc/PROCESS#adr-status) page. -> Use DRAFT if the ADR is in a draft stage (draft PR) or PROPOSED if it's in review. - -## Abstract - -> "If you can't explain it simply, you don't understand it well enough." Provide -> a simplified and layman-accessible explanation of the ADR. -> A short (\~200 word) description of the issue being addressed. - -## Context - -> This section describes the forces at play, including technological, political, -> social, and project local. These forces are probably in tension, and should be -> called out as such. The language in this section is value-neutral. It is simply -> describing facts. It should clearly explain the problem and motivation that the -> proposal aims to resolve. - -`{context body}` - -## Alternatives - -> This section describes alternative designs to the chosen design. This section -> is important and if an adr does not have any alternatives then it should be -> considered that the ADR was not thought through. - -## Decision - -> This section describes our response to these forces. It is stated in full -> sentences, with active voice. "We will ..." -> `{decision body}` - -## Consequences - -> This section describes the resulting context, after applying the decision. All -> consequences should be listed here, not just the "positive" ones. A particular -> decision may have positive, negative, and neutral consequences, but all of them -> affect the team and project in the future. - -### Backwards Compatibility - -> All ADRs that introduce backwards incompatibilities must include a section -> describing these incompatibilities and their severity. The ADR must explain -> how the author proposes to deal with these incompatibilities. ADR submissions -> without a sufficient backwards compatibility treatise may be rejected outright. - -### Positive - -> `{positive consequences}` - -### Negative - -> `{negative consequences}` - -### Neutral - -> `{neutral consequences}` - -## Further Discussions - -> While an ADR is in the DRAFT or PROPOSED stage, this section should contain a -> summary of issues to be solved in future iterations (usually referencing comments -> from a pull-request discussion). -> -> Later, this section can optionally list ideas or improvements the author or -> reviewers found during the analysis of this ADR. - -## Test Cases \[optional] - -Test cases for an implementation are mandatory for ADRs that are affecting consensus -changes. Other ADRs can choose to include links to test cases if applicable. - -## References - -* `{reference link}` diff --git a/sdk/next/reference/rfc.mdx b/sdk/next/reference/rfc.mdx index 05112ecec..8bac9a3b2 100644 --- a/sdk/next/reference/rfc.mdx +++ b/sdk/next/reference/rfc.mdx @@ -1,6 +1,6 @@ --- title: "Requests for Comments" -description: "Version: v0.54" +description: "Version: v0.55" noindex: true --- @@ -19,7 +19,7 @@ An RFC should provide: * Any **background** a reader will need to understand and participate in the substance of the discussion (links to other documents are fine here). * The **discussion**, the primary content of the document. -The [rfc-template.md](/sdk/next/reference/rfc/rfc-template) file includes placeholders for these sections. +The `rfc-template.md` file includes placeholders for these sections. ## Table of Contents[​](#table-of-contents "Direct link to Table of Contents") diff --git a/sdk/next/reference/rfc/README.mdx b/sdk/next/reference/rfc/README.mdx index 5fab97977..c40bcd634 100644 --- a/sdk/next/reference/rfc/README.mdx +++ b/sdk/next/reference/rfc/README.mdx @@ -33,7 +33,7 @@ An RFC should provide: substance of the discussion (links to other documents are fine here). * The **discussion**, the primary content of the document. -The [rfc-template.md](/sdk/v0.50/build/rfc/rfc-template) file includes placeholders for these +The `rfc-template.md` file includes placeholders for these sections. ## Table of Contents diff --git a/sdk/next/reference/rfc/rfc-template.mdx b/sdk/next/reference/rfc/rfc-template.mdx deleted file mode 100644 index a6c279d05..000000000 --- a/sdk/next/reference/rfc/rfc-template.mdx +++ /dev/null @@ -1,81 +0,0 @@ ---- -noindex: true -canonical: 'https://docs.cosmos.network/sdk/latest/reference/rfc/rfc-template' ---- -## Changelog - -* `{date}`: `{changelog}` - -## Background - -> The next section is the "Background" section. This section should be at least two paragraphs and can take up to a whole -> page in some cases. The guiding goal of the background section is: as a newcomer to this project (new employee, team -> transfer), can I read the background section and follow any links to get the full context of why this change is\ -> necessary? -> -> If you can't show a random engineer the background section and have them acquire nearly full context on the necessity -> for the RFC, then the background section is not full enough. To help achieve this, link to prior RFCs, discussions, and -> more here as necessary to provide context so you don't have to simply repeat yourself. - -## Proposal - -> The next required section is "Proposal" or "Goal". Given the background above, this section proposes a solution. -> This should be an overview of the "how" for the solution, but for details further sections will be used. - -## Abandoned Ideas (Optional) - -> As RFCs evolve, it is common that there are ideas that are abandoned. Rather than simply deleting them from the -> document, you should try to organize them into sections that make it clear they're abandoned while explaining why they -> were abandoned. -> -> When sharing your RFC with others or having someone look back on your RFC in the future, it is common to walk the same -> path and fall into the same pitfalls that we've since matured from. Abandoned ideas are a way to recognize that path -> and explain the pitfalls and why they were abandoned. - -## Decision - -> This section describes alternative designs to the chosen design. This section -> is important and if an ADR does not have any alternatives then it should be -> considered that the ADR was not thought through. - -## Consequences (optional) - -> This section describes the resulting context, after applying the decision. All -> consequences should be listed here, not just the "positive" ones. A particular -> decision may have positive, negative, and neutral consequences, but all of them -> affect the team and project in the future. - -### Backwards Compatibility - -> All ADRs that introduce backwards incompatibilities must include a section -> describing these incompatibilities and their severity. The ADR must explain -> how the author proposes to deal with these incompatibilities. ADR submissions -> without a sufficient backwards compatibility treatise may be rejected outright. - -### Positive - -> `{positive consequences}` - -### Negative - -> `{negative consequences}` - -### Neutral - -> `{neutral consequences}` - -### References - -> Links to external materials needed to follow the discussion may be added here. -> -> In addition, if the discussion in a request for comments leads to any design -> decisions, it may be helpful to add links to the ADR documents here after the -> discussion has settled. - -## Discussion - -> This section contains the core of the discussion. -> -> There is no fixed format for this section, but ideally changes to this -> section should be updated before merging to reflect any discussion that took -> place on the PR that made those changes. diff --git a/sdk/next/reference/spec.mdx b/sdk/next/reference/spec.mdx index 11c14913c..49f6778ab 100644 --- a/sdk/next/reference/spec.mdx +++ b/sdk/next/reference/spec.mdx @@ -1,6 +1,6 @@ --- title: "Specifications" -description: "Version: v0.54" +description: "Version: v0.55" noindex: true --- @@ -19,4 +19,4 @@ Go the [module directory](/sdk/next/modules/modules) ## CometBFT[​](#cometbft "Direct link to CometBFT") -For details on the underlying blockchain and p2p protocols, see the [CometBFT specification](https://github.com/cometbft/cometbft/tree/main/spec). +For details on the underlying blockchain and p2p protocols, see the [CometBFT specification](https://github.com/cometbft/cometbft/tree/v0.40.x/spec). diff --git a/sdk/next/reference/spec/README.mdx b/sdk/next/reference/spec/README.mdx index fcdfa7e45..f8534efa7 100644 --- a/sdk/next/reference/spec/README.mdx +++ b/sdk/next/reference/spec/README.mdx @@ -24,4 +24,4 @@ Go the [module directory](/sdk/next/modules/modules) ## CometBFT For details on the underlying blockchain and p2p protocols, see -the [CometBFT specification](https://github.com/cometbft/cometbft/tree/main/spec). +the [CometBFT specification](https://github.com/cometbft/cometbft/tree/v0.40.x/spec). diff --git a/sdk/next/reference/spec/_ics/ics-030-signed-messages.mdx b/sdk/next/reference/spec/_ics/ics-030-signed-messages.mdx index 1f70513f1..5d60fc138 100644 --- a/sdk/next/reference/spec/_ics/ics-030-signed-messages.mdx +++ b/sdk/next/reference/spec/_ics/ics-030-signed-messages.mdx @@ -64,7 +64,7 @@ pre-image attacks, as well as being [deterministic](https://en.wikipedia.org/wik ## Specification CometBFT has a well established protocol for signing messages using a canonical -JSON representation as defined [here](https://github.com/cometbft/cometbft/blob/master/types/canonical.go). +JSON representation as defined [here](https://github.com/cometbft/cometbft/blob/v0.40.x/types/canonical.go). An example of such a canonical JSON structure is CometBFT's vote structure: diff --git a/sdk/next/release-family.mdx b/sdk/next/release-family.mdx index d5a3691bb..ad2fb69ea 100644 --- a/sdk/next/release-family.mdx +++ b/sdk/next/release-family.mdx @@ -1,11 +1,15 @@ --- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/release-family' title: "Release Families" description: "What release families are, what they contain, and how upgrades work." --- ## Overview -A release family is a curated set of component versions across the Cosmos Stack that are tested for compatibility with one another. Cosmos Labs provides maintenance and bug fixes only for active families. For lifecycle policy and maintenance windows, see the [Security and Maintenance Policy](/sdk/next/security/security-policy). +A release family is a curated set of component versions across the Cosmos Stack that are tested for compatibility with one another. Cosmos Labs provides maintenance and bug fixes only for active families. + +This page is the canonical source of truth for release family lifecycle, active support windows, and retirement expectations. ## What a Release Family Contains @@ -29,14 +33,14 @@ Certain packages within the SDK may not be listed as Cosmos Labs consolidates se | Component | Version | | --------- | ------- | -| [Cosmos SDK](https://github.com/cosmos/cosmos-sdk) | 0.54.x | -| [Enterprise Groups](https://github.com/cosmos/cosmos-sdk/tree/main/enterprise/group) | 1.x.y | -| [Enterprise PoA](https://github.com/cosmos/cosmos-sdk/tree/main/enterprise/poa) | 1.x.y | -| [CometBFT](https://github.com/cometbft/cometbft) | 0.39.x | +| [Cosmos SDK](https://github.com/cosmos/cosmos-sdk) | 0.55.x | +| [Enterprise Groups](https://github.com/cosmos/cosmos-sdk/tree/release/v0.55.x/enterprise/group) | 1.x.y | +| [Enterprise PoA](https://github.com/cosmos/cosmos-sdk/tree/release/v0.55.x/enterprise/poa) | 1.x.y | +| [CometBFT](https://github.com/cometbft/cometbft) | 0.40.x | | [IBC Go](https://github.com/cosmos/ibc-go) | v11.x.y | -| [Solidity IBC Eureka](https://github.com/cosmos/solidity-ibc-eureka) | 0.1.x | -| [Relayer](https://github.com/cosmos/ibc-relayer) | 0.1.x | -| [Attestor](https://github.com/cosmos/ibc-attestor) | 0.1.x | +| [Solidity IBC Eureka](https://github.com/cosmos/solidity-ibc-eureka) | 3.0.x | +| [Relayer](https://github.com/cosmos/ibc-relayer) | 1.1.x | +| [Attestor](https://github.com/cosmos/ibc-attestor) | 1.0.x | ### 2025.1 @@ -60,29 +64,6 @@ Lifecycle policy applies to families, not individual component versions in isola For security reporting and vulnerability handling details, see the [Security and Maintenance Policy](/sdk/next/security/security-policy). -## Examples - -An example release family might look like: - -| Component | Version | -| --------- | ------- | -| Cosmos SDK | 0.54.0 | -| CometBFT | 0.39.0 | -| IBC Go | v11.0.0 | -| Solidity IBC Eureka | 0.1.0 | -| Relayer | 0.1.0 | -| Attestor | 0.1.0 | - -Upgrades that would update versions within this family without creating a new one: - -- SDK 0.54.0 to 0.54.1 -- Relayer 0.1.0 to 0.1.1 - -Upgrades that would require a new release family: - -- SDK 0.54.x to 0.55.0 -- CometBFT 0.39.x to 0.40.x -- Relayer 0.1.x to 1.0.0 ## End of Life Notices diff --git a/sdk/next/tutorials.mdx b/sdk/next/tutorials.mdx index 8d95b6389..a617be250 100644 --- a/sdk/next/tutorials.mdx +++ b/sdk/next/tutorials.mdx @@ -1,12 +1,12 @@ --- noindex: true title: "Node Tutorial" -description: "Version: v0.54" +description: "Version: v0.55" --- This guide covers everything you need to run, configure, and maintain a Cosmos SDK node. Whether you're setting up a local development node, deploying to a testnet, or running production infrastructure, you'll find step-by-step instructions and best practices. -The node tutorial uses the `simapp` example application and its corresponding CLI binary `simd` as the blockchain application and CLI. You can view the source code for `simapp` [on GitHub](https://github.com/cosmos/cosmos-sdk/tree/main/simapp). +The node tutorial uses the `simapp` example application and its corresponding CLI binary `simd` as the blockchain application and CLI. You can view the source code for `simapp` [on GitHub](https://github.com/cosmos/cosmos-sdk/tree/release/v0.55.x/simapp). diff --git a/sdk/next/upgrade/release.mdx b/sdk/next/upgrade/v0.54-release.mdx similarity index 98% rename from sdk/next/upgrade/release.mdx rename to sdk/next/upgrade/v0.54-release.mdx index bbce4872a..2f5987616 100644 --- a/sdk/next/upgrade/release.mdx +++ b/sdk/next/upgrade/v0.54-release.mdx @@ -5,7 +5,7 @@ description: "What's new in the latest Cosmos SDK release, including performance --- - If you are upgrading to v0.54, see the [upgrade guide](/sdk/next/upgrade/upgrade). For a full list of changes, see the [changelog](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/CHANGELOG.md). + If you are upgrading to v0.54, see the [upgrade guide](/sdk/next/upgrade/v0.54). For a full list of changes, see the [changelog](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/CHANGELOG.md). ## Overview diff --git a/sdk/next/upgrade/upgrade.mdx b/sdk/next/upgrade/v0.54.mdx similarity index 100% rename from sdk/next/upgrade/upgrade.mdx rename to sdk/next/upgrade/v0.54.mdx diff --git a/sdk/next/upgrade/v0.55-release.mdx b/sdk/next/upgrade/v0.55-release.mdx new file mode 100644 index 000000000..db64d6ce2 --- /dev/null +++ b/sdk/next/upgrade/v0.55-release.mdx @@ -0,0 +1,79 @@ +--- +noindex: true +title: "v0.55 Release Notes" +description: "What's new in the 2026.1 Ledger Security release: post-quantum keys, validator consensus key rotation, and remote signing with Cosmos-KMS." +--- + + + If you are upgrading to v0.55, see the [upgrade guide](/sdk/next/upgrade/v0.55). For a full list of changes, see the [changelog](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/CHANGELOG.md). + + + +## Overview + +This release is a holistic upgrade to the security of the Cosmos Stack. It adds the first native post-quantum key option in Cosmos, in-place validator consensus key rotation with no downtime, and a remote signer that keeps validator keys in your own KMS or HSM. + +All four artifacts ship together and join the existing 2026.1 release family. For the versions each family pins, see [Release Families](/sdk/next/release-family). + +## What ships + +| Artifact | Version | What changed | +| -------- | ------- | ------------ | +| [Cosmos SDK](https://github.com/cosmos/cosmos-sdk) | v0.55.0 | ML-DSA account and consensus keys, consensus key rotation through `x/staking`, keyring key types | +| [CometBFT](https://github.com/cometbft/cometbft) | v0.40.0 | ML-DSA consensus key support and remote signer compatibility | +| [enterprise/poa](https://github.com/cosmos/cosmos-sdk/tree/release/v0.55.x/enterprise/poa) | v1.1.0 | Consensus key rotation for PoA validators, by the operator or the chain admin | +| [cosmos-kms](https://github.com/cosmos/kms) | v1.0.0 | First release of the remote signer | + +## Features + +### Post-quantum keys (ML-DSA) + +Chains can run ML-DSA for consensus and user-account keys. ML-DSA keys use lattice-based signatures, which are considered more quantum resistant than elliptic-curve-based keys. New chains set the allowed key types through consensus params; existing chains migrate one validator at a time, and a validator-led path moves a classical key to ML-DSA in place with no hard fork. + +A chain reaches post-quantum security once validators holding two-thirds of voting power have rotated to ML-DSA keys, the same threshold CometBFT uses to finalize blocks. + +See [Post-quantum keys](/sdk/next/keys/post-quantum-keys) for the tradeoffs, [Enable ML-DSA keys](/sdk/next/keys/enable-ml-dsa-keys) to allow the type on a chain, and [Migrate a validator to ML-DSA](/sdk/next/keys/migrate-validator-ml-dsa) for the per-validator path. + +### Validator consensus key rotation + +Staked validators rotate a consensus key in place, keeping the validator's address, voting power, and accumulated fees, so the rotation stays invisible to delegators. Before this, a compromised or policy-expired consensus key meant standing up a new validator and rebuilding the delegator base. + +The operator submits `MsgRotateConsPubKey` with the new consensus public key, and CometBFT applies the change two heights later, which lets the operator bring up the new node with no downtime. Each rotation burns the `key_rotation_fee` staking parameter, a validator can rotate once per unbonding period, and a rotated-away key stays attributable for slashing until equivocation evidence for it can no longer be admitted. + +See [Key rotation](/sdk/next/keys/key-rotation) for the mechanics and security implications, and [Rotate a consensus key, Staking](/sdk/next/keys/rotate-validator-key) for the procedure. PoA chains follow [Rotate a consensus key, PoA](/sdk/next/keys/rotate-validator-key-poa). + +### Remote signing with Cosmos-KMS + +`cosmos-kms` is a new remote signing solution that signs on the validator's behalf while keys stay in your own HSM or cloud KMS rather than in local files on the node. It adds AWS KMS and PKCS#11 backends and post-quantum ML-DSA signing, none of which TMKMS supported. + +See [Cosmos-KMS and remote signing](/sdk/next/kms/remote-signing) for the architecture, and the [remote signing tutorial](/sdk/next/kms/tutorial-file-backend) to run one against a local chain. + +## Removals and deprecations + +### TMKMS deprecation notice + +This release begins the deprecation of TMKMS. TMKMS reaches official deprecation six months from this release, so operators running it have that window to move to `cosmos-kms`. + +Validators using TMKMS should migrate. See [Migrate from TMKMS](/sdk/next/kms/migrate-from-tmkms), which covers moving each TMKMS backend to `cosmos-kms`. + +### Removed in v0.55 + +- `x/params`, replaced by per-module params. +- `x/protocolpool`, with the community pool returning to `x/distribution`. +- `SIGN_MODE_TEXTUAL`. + +See the [upgrade guide](/sdk/next/upgrade/v0.55) for the wiring changes each removal requires. + +## Upgrading + +Upgrading to Cosmos SDK v0.55.0 bumps CometBFT to v0.40.0 automatically, so you do not upgrade CometBFT separately. Coordinate the upgrade across the validator set, since it moves the SDK and CometBFT together. + +We document the [0.54 to 0.55 upgrade path](/sdk/next/upgrade/v0.55), which also includes a [section on upgrading from 0.53 directly to 0.55](/sdk/next/upgrade/v0.55#upgrading-from-v0-53-x). Module upgrades work across the last two SDK versions. + +## Upcoming + +The following features are planned for a future release: + +- Enterprise HSM and key custody. AWS KMS supports ML-DSA signatures through Cosmos-KMS in this release. Other HSM and KMS solutions will be supported in a future release. +- Post-quantum support for attestors and signers. +- Ledger-layer confidential transactions. diff --git a/sdk/next/upgrade/v0.55.mdx b/sdk/next/upgrade/v0.55.mdx new file mode 100644 index 000000000..435087ff0 --- /dev/null +++ b/sdk/next/upgrade/v0.55.mdx @@ -0,0 +1,292 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/upgrade/upgrade' +title: "v0.55 Upgrade Guide" +description: "Reference for upgrading to v0.55 of Cosmos SDK" +--- + +This document provides a reference for upgrading from `v0.54.x` to `v0.55.x` of Cosmos SDK. If you are upgrading directly from `v0.53.x`, see [Upgrading from v0.53.x](#upgrading-from-v0-53-x) after reading the breaking changes below. + +For a full list of changes, see the [Changelog](https://github.com/cosmos/cosmos-sdk/blob/release/v0.55.x/CHANGELOG.md). + +The headline changes in this release are the removal of three legacy surfaces (`x/params`, `x/protocolpool`, and `SIGN_MODE_TEXTUAL`), a reworked app-side mempool interface, and validator consensus key rotation in `x/staking`. Key rotation ships enabled for every chain that upgrades — see [Validator Consensus Key Rotation](#validator-consensus-key-rotation) — and requires one line of wiring in `app.go`. Everything else in the new-features list (ML-DSA-65 keys, secp256k1eth keys, config-driven Block-STM wiring) is opt-in. + +## Table of Contents + +* [Breaking Changes](#breaking-changes) + * [CometBFT Upgrade](#cometbft-upgrade) + * [Removed: x/params](#removed-xparams) + * [Removed: x/protocolpool](#removed-xprotocolpool) + * [Removed: SIGN_MODE_TEXTUAL](#removed-sign_mode_textual) + * [Mempool Interface Changes](#mempool-interface-changes) + * [Staking: Key Rotation Wiring and Interface Changes](#staking-key-rotation-wiring-and-interface-changes) + * [genutil: ExportGenesisFileWithTime Signature](#genutil-exportgenesisfilewithtime-signature) + * [Upgrade Handler and Store Migrations](#upgrade-handler-and-store-migrations) +* [Upgrading from v0.53.x](#upgrading-from-v053x) +* [New Features and Non-Breaking Changes](#new-features-and-non-breaking-changes) + * [Validator Consensus Key Rotation](#validator-consensus-key-rotation) + * [ML-DSA-65 Validator Consensus Keys](#ml-dsa-65-validator-consensus-keys) + * [ML-DSA-65 Account Keys](#ml-dsa-65-account-keys) + * [secp256k1eth Validator Consensus Keys](#secp256k1eth-validator-consensus-keys) + * [Block-STM Configuration](#block-stm-configuration) +* [Behavior Changes Affecting Dapps and Indexers](#behavior-changes-affecting-dapps-and-indexers) + +## Breaking Changes + +### CometBFT Upgrade + +Cosmos SDK v0.55 requires CometBFT `v0.40.0` (the v0.54.x line shipped with `v0.39.x`, ending at `v0.39.3` in v0.54.3). Bump your app's `go.mod` to match the SDK's pin. Relevant changes in CometBFT v0.40.0: + +* Expanded `MaxSignatureSize` and per-validator `MaxCommitSigBytes` to accommodate post-quantum (ML-DSA-65) signatures. +* A fix for the application-side mempool (`mempool.type = "app"`, supported since CometBFT v0.39.2 / SDK v0.54.3): the default socket transport was missing the `InsertTx` / `ReapTxs` cases, causing node self-kill ([cometbft#5958](https://github.com/cometbft/cometbft/pull/5958)). Chains using an app-side mempool over the socket transport need v0.40.0. +* Updated `DefaultBlockParams` ([cometbft#5987](https://github.com/cometbft/cometbft/pull/5987)). This changes defaults for new chains only; existing chains keep their on-chain consensus params. + +See the [CometBFT changelog](https://github.com/cometbft/cometbft/blob/main/CHANGELOG.md) for the full list. + +### Removed: x/params + +[#25546](https://github.com/cosmos/cosmos-sdk/pull/25546) removes the `x/params` module entirely (only a tombstone README remains). Module parameters have been managed by each module since v0.47; v0.55 removes the leftover machinery: + +1. If your app still imports `x/params` (a `paramskeeper.Keeper`, per-module `Subspace`s, or the legacy gov proposal handler), remove that wiring. If the `params` store is still mounted, delete it in your store upgrades (see [Upgrade Handler and Store Migrations](#upgrade-handler-and-store-migrations)). Chains that have not yet migrated legacy subspace params to module-managed params must complete that migration **before** upgrading to v0.55 — the migration code is gone. + +2. Drop the trailing `exported.Subspace` argument (typically passed as `nil`) from the module constructors that carried it for legacy migrations: + +```go +// Before // After +auth.NewAppModule(cdc, accountKeeper, randGenAccountsFn, nil) auth.NewAppModule(cdc, accountKeeper, randGenAccountsFn) +bank.NewAppModule(cdc, bankKeeper, accountKeeper, nil) bank.NewAppModule(cdc, bankKeeper, accountKeeper) +gov.NewAppModule(cdc, &govKeeper, accountKeeper, bankKeeper, nil) gov.NewAppModule(cdc, &govKeeper, accountKeeper, bankKeeper) +mint.NewAppModule(cdc, mintKeeper, accountKeeper, nil, nil) mint.NewAppModule(cdc, mintKeeper, accountKeeper, nil) +slashing.NewAppModule(cdc, keeper, ak, bk, sk, nil, registry) slashing.NewAppModule(cdc, keeper, ak, bk, sk, registry) +distr.NewAppModule(cdc, keeper, ak, bk, stakingKeeper, nil) distr.NewAppModule(cdc, keeper, ak, bk, stakingKeeper) +staking.NewAppModule(cdc, keeper, ak, bk, nil) staking.NewAppModule(cdc, keeper, ak, bk) +``` + +(`mint.NewAppModule` retains its deprecated `InflationCalculationFn` parameter; only the subspace argument is removed.) + +### Removed: x/protocolpool + +[#26421](https://github.com/cosmos/cosmos-sdk/pull/26421) removes the `x/protocolpool` module and its proto/API surface from the SDK. The `distrkeeper.WithExternalCommunityPool` extension point is removed with it — `x/distribution` always uses its internal `FeePool` community pool again, and `MsgFundCommunityPool` / `MsgCommunityPoolSpend` operate on it directly. + +**Required action** if your app wired `x/protocolpool` (the v0.54 SimApp default): + +1. Remove all `protocolpool` wiring from `app.go`: the imports, the `ProtocolPoolKeeper` field and its `NewKeeper` call, the `protocolpooltypes.ModuleName` and `protocolpooltypes.ProtocolPoolEscrowAccount` entries in `maccPerms`, the module manager entry, and its entries in the begin-block, end-block, init-genesis, and export orders. +2. Remove `distrkeeper.WithExternalCommunityPool(app.ProtocolPoolKeeper)` from your `distrkeeper.NewKeeper` call. +3. Delete the `protocolpool` store in your store upgrades (see [Upgrade Handler and Store Migrations](#upgrade-handler-and-store-migrations)). +4. Balances held by the protocolpool module accounts are bank state and are **not** migrated automatically. Decide where those funds go and move them in your upgrade handler — e.g. transfer them to the `x/distribution` community pool so community-pool spend proposals keep working. + +If your app never wired `x/protocolpool`, no action is needed beyond not being able to import it. + +### Removed: SIGN_MODE_TEXTUAL + +`SIGN_MODE_TEXTUAL` (proto enum value `2`) and its entire implementation have been removed ([#26456](https://github.com/cosmos/cosmos-sdk/pull/26456)): + +* `x/tx/signing/textual/` — all renderers, the CBOR encoder, test data, and internal protos +* `x/auth/tx/textual.go` and `ConfigOptions.TextualCoinMetadataQueryFn` +* Ledger + SIGN_MODE_TEXTUAL integration in `client/` flags and tx factory + +The proto enum value `2` and string `"SIGN_MODE_TEXTUAL"` are **reserved** to prevent future reuse. ADR-050 is archived. + +**Required action** if your app enabled SIGN_MODE_TEXTUAL: + +1. Remove `TextualCoinMetadataQueryFn` from your `tx.ConfigOptions`: + + ```go + // Before + txConfig, err := tx.NewTxConfigWithOptions(cdc, tx.ConfigOptions{ + TextualCoinMetadataQueryFn: ..., + }) + + // After — field removed, omit it + txConfig, err := tx.NewTxConfigWithOptions(cdc, tx.ConfigOptions{...}) + ``` + +2. Remove any `SIGN_MODE_TEXTUAL` cases from signing mode handler switch statements. + +3. Remove Ledger wiring that depended on `SIGN_MODE_TEXTUAL`. Client-side root command wiring that constructed a textual-enabled tx config for online mode (as v0.54 SimApp did in `simd/cmd/root.go`) should be deleted as well. + +### Mempool Interface Changes + +[#25338](https://github.com/cosmos/cosmos-sdk/pull/25338) changes the `types/mempool` interfaces so the mempool stores the gas wanted reported by the ante handler at `CheckTx` time, and block selection uses that value instead of the tx-declared gas limit. + +**Required action** if you implement a custom mempool (chains using the SDK's built-in mempools or no app-side mempool just recompile): + +* `Insert` gains an `InsertOption` parameter carrying the ante-reported gas: `Insert(context.Context, sdk.Tx, InsertOption) error`. +* `Iterator.Tx()` now returns a `PooledTx` (`{Tx sdk.Tx; GasWanted uint64}`) instead of `sdk.Tx`. +* `ExtMempool.SelectBy`'s callback now receives a `PooledTx`: `SelectBy(context.Context, [][]byte, func(PooledTx) bool)`. +* `ExtMempool.RemoveWithReason` and the `RemoveReason` type, introduced in v0.54, are unchanged. + +Custom `PrepareProposal` handlers that iterate the mempool should read gas from `PooledTx.GasWanted` rather than re-deriving it from the tx. + +### Staking: Key Rotation Wiring and Interface Changes + +`x/staking` now requires a `key_rotation_fee_pool` module account with burn permissions — the staking keeper panics at construction if it is missing (`x/staking/keeper/keeper.go`). Add it to your `maccPerms`: + +```go +maccPerms = map[string][]string{ + // ...existing entries... + stakingtypes.KeyRotationFeePoolName: {authtypes.Burner}, +} +``` + +This is required for **all** chains upgrading to v0.55, whether or not validators are expected to use [key rotation](#validator-consensus-key-rotation). + +Key rotation also touches two staking keeper surfaces that external modules may implement or consume: + +* The `StakingHooks` interface gains `AfterValidatorConsKeyUpdated(ctx context.Context, oldConsAddr, newConsAddr sdk.ConsAddress, valAddr sdk.ValAddress) error`, called when a rotation is applied. Custom `StakingHooks` implementations must add this method (returning `nil` is fine if you don't need the notification). +* The staking keeper adds `ValidatorByHistoricalConsAddr(ctx, consAddr)`, which resolves a validator from a consensus address it used before a rotation. Modules that map consensus addresses to validators can no longer assume that mapping is immutable — see [Validator Consensus Key Rotation](#validator-consensus-key-rotation). + +### genutil: ExportGenesisFileWithTime Signature + +[#26468](https://github.com/cosmos/cosmos-sdk/pull/26468) consolidates `ExportGenesisFileWithTime`'s arguments so the exported file preserves consensus params (previously they were rebuilt from defaults, dropping the caller's values): + +```go +// Before +func ExportGenesisFileWithTime(genFile, chainID string, validators []cmttypes.GenesisValidator, + appState json.RawMessage, genTime time.Time) error + +// After — build the AppGenesis yourself; everything you set on it is preserved +func ExportGenesisFileWithTime(genFile string, appGenesis *types.AppGenesis, genTime time.Time) error +``` + +### Upgrade Handler and Store Migrations + +#### Module Migrations + +Two module consensus-version bumps ship in this release and run automatically via `RunMigrations` in your upgrade handler: + +* `x/staking` 5 → 6: adds the `key_rotation_fee` param, defaulting to `1000000` of the bond denom ([#26485](https://github.com/cosmos/cosmos-sdk/pull/26485)). `Params.Validate` requires the fee denom to equal `bond_denom` ([#26613](https://github.com/cosmos/cosmos-sdk/pull/26613)). +* `x/auth` 6 → 7: adds the `SigVerifyCostMlDsa65` param with its default value ([#26472](https://github.com/cosmos/cosmos-sdk/pull/26472)). + +#### Reference Upgrade Handler + +A reference upgrade handler for this release (see `simapp/upgrades.go`): + +```go +const UpgradeName = "v054-to-v055" + +func (app SimApp) RegisterUpgradeHandlers() { + app.UpgradeKeeper.SetUpgradeHandler( + UpgradeName, + func(ctx context.Context, _ upgradetypes.Plan, fromVM module.VersionMap) (module.VersionMap, error) { + return app.ModuleManager.RunMigrations(ctx, app.Configurator(), fromVM) + }, + ) + + upgradeInfo, err := app.UpgradeKeeper.ReadUpgradeInfoFromDisk() + if err != nil { + panic(err) + } + + if upgradeInfo.Name == UpgradeName && !app.UpgradeKeeper.IsSkipHeight(upgradeInfo.Height) { + storeUpgrades := storetypes.StoreUpgrades{ + Added: []string{}, + Deleted: []string{"protocolpool"}, + } + app.SetStoreLoader(upgradetypes.UpgradeStoreLoader(upgradeInfo.Height, &storeUpgrades)) + } +} +``` + +Add `"params"` to `Deleted` as well if your app still had the `x/params` store mounted. + +## Upgrading from v0.53.x + +Skipping v0.54 and upgrading directly from `v0.53.x` to `v0.55.x` is supported as a single coordinated upgrade: one binary swap, one upgrade handler, one halt height. Work through the [v0.53.x → v0.54.x upgrade reference](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/UPGRADING.md) first — all of its required changes still apply — then apply this guide on top. The v0.54 hop's highlights, so you know what you're signing up for: + +* CometBFT `v0.38.x` → `v0.39.x` (LibP2P, `AdaptiveSync`); from v0.53 you jump straight to the `v0.40.0` release v0.55 pins. +* Consolidation of `cosmossdk.io/x/*` vanity modules into `github.com/cosmos/cosmos-sdk/x/*`, plus the Log v2 and Store v2 moves. +* `x/gov` keeper-initialization and `GovHooks` interface changes, `x/epochs` and `x/bank` wiring updates, and the `x/circuit` / `x/nft` / `x/crisis` deprecations. +* IBC v11 (if your chain uses IBC). + +Where the two hops interact, land directly on the v0.55 state instead of transiting through v0.54's: + +* **Skip transient wiring.** Don't adopt v0.54 reference-app wiring that v0.55 removes in the same hop: the SIGN_MODE_TEXTUAL tx-config setup, `x/protocolpool` (if your v0.53 app didn't already wire it), and `distrkeeper.WithExternalCommunityPool`. Go straight to the v0.55 forms shown in this guide. +* **Module constructors.** v0.54's constructor signatures still carried the legacy `exported.Subspace` arguments; use the v0.55 signatures from [Removed: x/params](#removed-xparams) directly. +* **Custom mempools.** Implement the v0.55 `Mempool` interface ([Mempool Interface Changes](#mempool-interface-changes)) directly; don't bother with the v0.54 shape. +* **Module migrations are cumulative.** `RunMigrations` walks each module from its v0.53 consensus version to the v0.55 target in one pass (`x/auth` 5 → 6 → 7, `x/staking` 5 → 6). No manual intervention is needed beyond the standard upgrade handler. +* **Store upgrades.** The v0.53 → v0.54 hop required no store additions or deletions, so the combined store upgrade is exactly the snippet in [Upgrade Handler and Store Migrations](#upgrade-handler-and-store-migrations): delete `protocolpool` only if your v0.53 app had wired it, and `params` if its store was still mounted (more likely on a v0.53-era app). Use a single upgrade name, e.g. `v053-to-v055`. + +Test the full jump on a mainnet-state export before scheduling it: the two-version migration path gets far less ecosystem mileage than the single-version one. + +## New Features and Non-Breaking Changes + +These changes are optional to adopt during the upgrade; they are not required for a successful migration. The exception is key rotation, which is active on every v0.55 chain once the required wiring above is in place. + +### Validator Consensus Key Rotation + +v0.55 adds consensus key rotation to `x/staking` ([#26440](https://github.com/cosmos/cosmos-sdk/pull/26440)): a validator operator can submit `MsgRotateConsPubKey` (wired into the CLI, [#26461](https://github.com/cosmos/cosmos-sdk/pull/26461)) to replace their consensus key without unbonding. Key properties: + +* **Fee.** Each rotation charges the `key_rotation_fee` staking param (default `1000000` of the bond denom) from the operator account; the fee is burned via the `key_rotation_fee_pool` module account. +* **Rate limit.** One rotation per validator per unbonding period. +* **Applied in the end blocker.** The rotation is scheduled by the msg server and applied at the end of the block; CometBFT is informed through a validator-set update. +* **Evidence and slashing.** Equivocation evidence against a rotated-away (historical) consensus address remains attributable to the validator until the evidence is no longer admissible — i.e. until both `evidence.max_age_num_blocks` and `evidence.max_age_duration` have elapsed since the rotation, which can be later than the unbonding time ([#26481](https://github.com/cosmos/cosmos-sdk/pull/26481), [#26616](https://github.com/cosmos/cosmos-sdk/pull/26616)). Slashing signing info is migrated to the active consensus key. Governance changes that extend the evidence-age params after a rotation's expiry has been computed are not retroactively applied; chains should account for this when tuning evidence params. +* **Genesis.** Rotation history and pending-rotation state are included in staking genesis import/export ([#26471](https://github.com/cosmos/cosmos-sdk/pull/26471)); genesis export tooling that parses staking genesis JSON should expect the new fields. +* **Events.** `rotate_cons_pubkey` is emitted when a rotation is scheduled (including apply height, maturity time, evidence-expiry time/height, and the burned fee) and `apply_cons_pubkey_rotation` when it is applied (validator, old and new consensus addresses) ([#26619](https://github.com/cosmos/cosmos-sdk/pull/26619)). + +Indexers, exchanges, and monitoring that key validators by consensus address must handle the mapping changing over a validator's lifetime. On-chain, `keeper.ValidatorByHistoricalConsAddr` resolves a validator from a rotated-away consensus address. + +Chains built on the enterprise `x/poa` module have their own `MsgRotateConsPubKey` with different semantics — no fee, no rate limit, an admin override, and a same-block swap with no rotation history. Because the old consensus address is gone immediately, modules that attribute `LastCommit` signatures or vote extensions by consensus address need extra care across the swap; see the PoA guide below for the caveats and the operator runbook. + +For an overview of key rotation and the operator procedures, see [Key rotation](https://docs.cosmos.network/sdk/latest/keys/key-rotation), [Rotate a consensus key, Staking](https://docs.cosmos.network/sdk/latest/keys/rotate-validator-key), and [Rotate a consensus key, PoA](https://docs.cosmos.network/sdk/latest/keys/rotate-validator-key-poa). + +### ML-DSA-65 Validator Consensus Keys + +Cosmos SDK v0.55 registers the NIST ML-DSA-65 (FIPS 204) post-quantum signature scheme as a supported validator consensus key type ([#26436](https://github.com/cosmos/cosmos-sdk/pull/26436)). The new `cosmos.crypto.mldsa65.PubKey` / `PrivKey` proto messages, Amino routes (`cometbft/PubKeyMlDsa65`, `cometbft/PrivKeyMlDsa65`), interface-registry registration, multisig amino route, and `hd.MlDsa65Type` constant are all enabled by default. + +**Action required:** none. Existing chains continue to accept only the consensus key types listed in `genesis.consensus_params.validator.pub_key_types` (still `["ed25519"]` by default). No state-machine-relevant behavior changes for chains that do not opt in. + +**To opt in (new chains):** set `genesis.consensus_params.validator.pub_key_types` to `["ml_dsa_65"]` (or a list including it). Validators must then submit `MsgCreateValidator` with a `mldsa65.PubKey`. The `init` and `testnet` commands accept `--consensus-key-algo ml_dsa_65` to generate matching validator files ([#26604](https://github.com/cosmos/cosmos-sdk/pull/26604)). Test harnesses can use the new `testutil/network.Config.ValidatorConsensusKeyType` field together with `genutil.InitializeNodeValidatorFilesFromMnemonicWithKeyType` to spin up an in-process testnet pinned to ML-DSA-65. + +**Operational considerations:** ML-DSA-65 keys and signatures are substantially larger than ed25519 (pubkey 1952 bytes vs 32, signature 3309 bytes vs 64). Chains enabling this key type should review `consensus_params.block.max_bytes` and gossip framing limits accordingly. The cometbft commit lift in this release expanded `MaxSignatureSize` and the per-validator `MaxCommitSigBytes` to accommodate the larger signatures; downstream applications relying on the previous fixed values may need to be re-examined. + +**Warning — IBC counterparties must upgrade first.** IBC light clients on counterparty chains verify your validator set's commit signatures using the counterparty's own compiled-in crypto. A counterparty running a stack that predates ML-DSA-65 support cannot verify signatures from the new key type: once validators holding sufficient voting power sign with it, your headers fail verification there, IBC packet flow with that chain stops, and the client eventually expires. Before enabling a new consensus key type on a chain with live IBC connections, coordinate so every counterparty chain is running a CometBFT/SDK stack that can verify it — the counterparty only needs the verification code on its nodes, not the key type in its own `pub_key_types`. + +Existing chains can combine this with [key rotation](#validator-consensus-key-rotation) to move validators to post-quantum keys: add `ml_dsa_65` to `pub_key_types` via a consensus-params update, then have validators rotate. + +For the concepts and operator guides, see [Post-quantum keys](https://docs.cosmos.network/sdk/latest/keys/post-quantum-keys), [Enable ML-DSA keys](https://docs.cosmos.network/sdk/latest/keys/enable-ml-dsa-keys), and [Migrate a validator to ML-DSA](https://docs.cosmos.network/sdk/latest/keys/migrate-validator-ml-dsa). + +### ML-DSA-65 Account Keys + +[#26472](https://github.com/cosmos/cosmos-sdk/pull/26472) extends ML-DSA-65 support to user account keys: keyring creation and mnemonic recovery (`--algo ml_dsa_65`), transaction signing and verification, and a new ante-handler gas cost param `SigVerifyCostMlDsa65` (added to `x/auth` params by the automatic 6 → 7 migration). No action is required; accounts using existing key types are unaffected. + +See [Create an ML-DSA account](https://docs.cosmos.network/sdk/latest/keys/create-ml-dsa-account) and [Post-quantum keys](https://docs.cosmos.network/sdk/latest/keys/post-quantum-keys). + +### secp256k1eth Validator Consensus Keys + +[#26615](https://github.com/cosmos/cosmos-sdk/pull/26615) adds `crypto/keys/secp256k1eth`, wrapping CometBFT's Ethereum-style secp256k1 consensus key implementation with SDK codec registration. Intended for EVM-compatible chains that want validator consensus addresses derived the Ethereum way; opt in via `genesis.consensus_params.validator.pub_key_types`. + +The IBC counterparty warning from the [ML-DSA-65 section](#ml-dsa-65-validator-consensus-keys) applies here too: counterparty chains must run a stack that can verify secp256k1eth signatures before your validators adopt the key type, or IBC connections with them will break. + +See [Post-quantum keys](https://docs.cosmos.network/sdk/latest/keys/post-quantum-keys) for how the consensus key types compare. + +### Block-STM Configuration + +Block-STM parallel execution itself is not new — the engine (`baseapp/txnrunner`) and the `SetBlockSTMTxRunner` hook shipped in v0.54.x, wired programmatically per chain. v0.55 adds standard operator-facing configuration ([#26208](https://github.com/cosmos/cosmos-sdk/pull/26208)): `block-executor` (`"sequential"`, the default, or `"block-stm"`), `block-stm-workers`, and `block-stm-pre-estimate` in `app.toml`, plus a `baseapp/blockexec` helper that resolves them and installs the runner. Chains that already call `SetBlockSTMTxRunner` directly can keep that wiring or switch to the helper. + +To adopt the config-driven wiring, call `blockexec.Apply` after creating your store keys (see `simapp/app.go`): + +```go +stores := make([]storetypes.StoreKey, 0, len(keys)) +for _, k := range keys { + stores = append(stores, k) +} +blockexec.Apply(bApp, appOpts, stores, txConfig.TxDecoder(), + func(storetypes.MultiStore) string { return sdk.DefaultBondDenom }, +) +``` + +`Apply` resolves the executor from `app.toml`/flags and installs the corresponding `TxRunner`; with the default `sequential` executor it preserves today's behavior, so the wiring is safe to add unconditionally. Block-STM is incompatible with the block gas meter (disabled by default since v0.54): `Apply` disables the meter automatically when `block-stm` is selected, but chains wiring `SetBlockSTMTxRunner` directly must call `SetDisableBlockGasMeter(true)` first or the runner installation panics. + +Switching a running chain's executor is a per-node setting with identical state-transition results, but treat the first enablement as an operational rollout: test with your workload before flipping validators. + +## Behavior Changes Affecting Dapps and Indexers + +Observable changes between v0.54.x and v0.55.x that don't require code changes but may affect downstream consumers: + +* **Block selection uses ante-reported gas.** Proposals are packed using the gas wanted returned by the ante handler at `CheckTx` time rather than the tx-declared gas limit ([#25338](https://github.com/cosmos/cosmos-sdk/pull/25338)). Block composition can differ for txs whose ante-reported gas diverges from their declared limit. +* **Staking emits key-rotation events** (`rotate_cons_pubkey`, `apply_cons_pubkey_rotation`), and validator consensus addresses can change over time ([#26619](https://github.com/cosmos/cosmos-sdk/pull/26619)). +* **`x/gov` `proposal_messages` event attribute** no longer has a leading comma ([#26353](https://github.com/cosmos/cosmos-sdk/pull/26353)). +* **`x/authz` prunes at most 200 expired grants per begin block** ([#26588](https://github.com/cosmos/cosmos-sdk/pull/26588)); mass-expiry cleanup now spreads across blocks. +* **`x/distribution` reward withdrawals to blocked addresses** during begin/end block fall back to the delegator/validator owner and then the community pool instead of failing ([#26406](https://github.com/cosmos/cosmos-sdk/pull/26406)). User-initiated withdrawals to blocked addresses still return `ErrUnauthorized`. +* **`x/feegrant` `Allowances` and `AllowancesByGranter` queries** now honor `PageRequest.offset` and `count_total` correctly ([#26596](https://github.com/cosmos/cosmos-sdk/pull/26596)); clients that compensated for the old off-by-page results should re-check. \ No newline at end of file diff --git a/sdk/v0.47/build/architecture/adr-template.mdx b/sdk/v0.47/build/architecture/adr-template.mdx deleted file mode 100644 index f3491d0c6..000000000 --- a/sdk/v0.47/build/architecture/adr-template.mdx +++ /dev/null @@ -1,87 +0,0 @@ ---- -noindex: true -canonical: 'https://docs.cosmos.network/sdk/latest/' ---- - -## Changelog - -* `{date}`: `{changelog}` - -## Status - -{DRAFT | PROPOSED} Not Implemented - -> Please have a look at the [PROCESS](/sdk/v0.47/build/architecture/PROCESS#adr-status) page. -> Use DRAFT if the ADR is in a draft stage (draft PR) or PROPOSED if it's in review. - -## Abstract - -> "If you can't explain it simply, you don't understand it well enough." Provide -> a simplified and layman-accessible explanation of the ADR. -> A short (\~200 word) description of the issue being addressed. - -## Context - -> This section describes the forces at play, including technological, political, -> social, and project local. These forces are probably in tension, and should be -> called out as such. The language in this section is value-neutral. It is simply -> describing facts. It should clearly explain the problem and motivation that the -> proposal aims to resolve. - -`{context body}` - -## Alternatives - -> This section describes alternative designs to the chosen design. This section -> is important and if an adr does not have any alternatives then it should be -> considered that the ADR was not thought through. - -## Decision - -> This section describes our response to these forces. It is stated in full -> sentences, with active voice. "We will ..." -> `{decision body}` - -## Consequences - -> This section describes the resulting context, after applying the decision. All -> consequences should be listed here, not just the "positive" ones. A particular -> decision may have positive, negative, and neutral consequences, but all of them -> affect the team and project in the future. - -### Backwards Compatibility - -> All ADRs that introduce backwards incompatibilities must include a section -> describing these incompatibilities and their severity. The ADR must explain -> how the author proposes to deal with these incompatibilities. ADR submissions -> without a sufficient backwards compatibility treatise may be rejected outright. - -### Positive - -> `{positive consequences}` - -### Negative - -> `{negative consequences}` - -### Neutral - -> `{neutral consequences}` - -## Further Discussions - -> While an ADR is in the DRAFT or PROPOSED stage, this section should contain a -> summary of issues to be solved in future iterations (usually referencing comments -> from a pull-request discussion). -> -> Later, this section can optionally list ideas or improvements the author or -> reviewers found during the analysis of this ADR. - -## Test Cases \[optional] - -Test cases for an implementation are mandatory for ADRs that are affecting consensus -changes. Other ADRs can choose to include links to test cases if applicable. - -## References - -* `{reference link}` diff --git a/sdk/v0.47/build/rfc.mdx b/sdk/v0.47/build/rfc.mdx index a094e77e1..e58bf93f1 100644 --- a/sdk/v0.47/build/rfc.mdx +++ b/sdk/v0.47/build/rfc.mdx @@ -20,4 +20,4 @@ An RFC should provide: * Any **background** a reader will need to understand and participate in the substance of the discussion (links to other documents are fine here). * The **discussion**, the primary content of the document. -The [rfc-template.md](/sdk/v0.47/build/rfc/rfc-template) file includes placeholders for these sections. +The `rfc-template.md` file includes placeholders for these sections. diff --git a/sdk/v0.47/build/rfc/README.mdx b/sdk/v0.47/build/rfc/README.mdx index f48f68841..d437c76b9 100644 --- a/sdk/v0.47/build/rfc/README.mdx +++ b/sdk/v0.47/build/rfc/README.mdx @@ -34,5 +34,5 @@ An RFC should provide: substance of the discussion (links to other documents are fine here). * The **discussion**, the primary content of the document. -The [rfc-template.md](/sdk/v0.47/build/rfc/rfc-template) file includes placeholders for these +The `rfc-template.md` file includes placeholders for these sections. diff --git a/sdk/v0.47/build/rfc/rfc-template.mdx b/sdk/v0.47/build/rfc/rfc-template.mdx deleted file mode 100644 index da5eeb8f3..000000000 --- a/sdk/v0.47/build/rfc/rfc-template.mdx +++ /dev/null @@ -1,82 +0,0 @@ ---- -noindex: true -canonical: 'https://docs.cosmos.network/sdk/latest/' ---- - -## Changelog - -* `{date}`: `{changelog}` - -## Background - -> The next section is the "Background" section. This section should be at least two paragraphs and can take up to a whole -> page in some cases. The guiding goal of the background section is: as a newcomer to this project (new employee, team -> transfer), can I read the background section and follow any links to get the full context of why this change is\ -> necessary? -> -> If you can't show a random engineer the background section and have them acquire nearly full context on the necessity -> for the RFC, then the background section is not full enough. To help achieve this, link to prior RFCs, discussions, and -> more here as necessary to provide context so you don't have to simply repeat yourself. - -## Proposal - -> The next required section is "Proposal" or "Goal". Given the background above, this section proposes a solution. -> This should be an overview of the "how" for the solution, but for details further sections will be used. - -## Abandoned Ideas (Optional) - -> As RFCs evolve, it is common that there are ideas that are abandoned. Rather than simply deleting them from the -> document, you should try to organize them into sections that make it clear they're abandoned while explaining why they -> were abandoned. -> -> When sharing your RFC with others or having someone look back on your RFC in the future, it is common to walk the same -> path and fall into the same pitfalls that we've since matured from. Abandoned ideas are a way to recognize that path -> and explain the pitfalls and why they were abandoned. - -## Descision - -> This section describes alternative designs to the chosen design. This section -> is important and if an adr does not have any alternatives then it should be -> considered that the ADR was not thought through. - -## Consequences (optional) - -> This section describes the resulting context, after applying the decision. All -> consequences should be listed here, not just the "positive" ones. A particular -> decision may have positive, negative, and neutral consequences, but all of them -> affect the team and project in the future. - -### Backwards Compatibility - -> All ADRs that introduce backwards incompatibilities must include a section -> describing these incompatibilities and their severity. The ADR must explain -> how the author proposes to deal with these incompatibilities. ADR submissions -> without a sufficient backwards compatibility treatise may be rejected outright. - -### Positive - -> `{positive consequences}` - -### Negative - -> `{negative consequences}` - -### Neutral - -> `{neutral consequences}` - -### References - -> Links to external materials needed to follow the discussion may be added here. -> -> In addition, if the discussion in a request for comments leads to any design -> decisions, it may be helpful to add links to the ADR documents here after the -> discussion has settled. - -## Discussion - -> This section contains the core of the discussion. -> -> There is no fixed format for this section, but ideally changes to this -> section should be updated before merging to reflect any discussion that took -> place on the PR that made those changes. diff --git a/sdk/v0.50/build/architecture/adr-template.mdx b/sdk/v0.50/build/architecture/adr-template.mdx deleted file mode 100644 index 2ee20e4cc..000000000 --- a/sdk/v0.50/build/architecture/adr-template.mdx +++ /dev/null @@ -1,87 +0,0 @@ ---- -noindex: true -canonical: 'https://docs.cosmos.network/sdk/latest/' ---- - -## Changelog - -* `{date}`: `{changelog}` - -## Status - -{DRAFT | PROPOSED} Not Implemented - -> Please have a look at the [PROCESS](/sdk/v0.50/build/architecture/PROCESS#adr-status) page. -> Use DRAFT if the ADR is in a draft stage (draft PR) or PROPOSED if it's in review. - -## Abstract - -> "If you can't explain it simply, you don't understand it well enough." Provide -> a simplified and layman-accessible explanation of the ADR. -> A short (\~200 word) description of the issue being addressed. - -## Context - -> This section describes the forces at play, including technological, political, -> social, and project local. These forces are probably in tension, and should be -> called out as such. The language in this section is value-neutral. It is simply -> describing facts. It should clearly explain the problem and motivation that the -> proposal aims to resolve. - -`{context body}` - -## Alternatives - -> This section describes alternative designs to the chosen design. This section -> is important and if an adr does not have any alternatives then it should be -> considered that the ADR was not thought through. - -## Decision - -> This section describes our response to these forces. It is stated in full -> sentences, with active voice. "We will ..." -> `{decision body}` - -## Consequences - -> This section describes the resulting context, after applying the decision. All -> consequences should be listed here, not just the "positive" ones. A particular -> decision may have positive, negative, and neutral consequences, but all of them -> affect the team and project in the future. - -### Backwards Compatibility - -> All ADRs that introduce backwards incompatibilities must include a section -> describing these incompatibilities and their severity. The ADR must explain -> how the author proposes to deal with these incompatibilities. ADR submissions -> without a sufficient backwards compatibility treatise may be rejected outright. - -### Positive - -> `{positive consequences}` - -### Negative - -> `{negative consequences}` - -### Neutral - -> `{neutral consequences}` - -## Further Discussions - -> While an ADR is in the DRAFT or PROPOSED stage, this section should contain a -> summary of issues to be solved in future iterations (usually referencing comments -> from a pull-request discussion). -> -> Later, this section can optionally list ideas or improvements the author or -> reviewers found during the analysis of this ADR. - -## Test Cases \[optional] - -Test cases for an implementation are mandatory for ADRs that are affecting consensus -changes. Other ADRs can choose to include links to test cases if applicable. - -## References - -* `{reference link}` diff --git a/sdk/v0.50/build/rfc.mdx b/sdk/v0.50/build/rfc.mdx index 89e7fbe36..09fbd79b7 100644 --- a/sdk/v0.50/build/rfc.mdx +++ b/sdk/v0.50/build/rfc.mdx @@ -20,7 +20,7 @@ An RFC should provide: * Any **background** a reader will need to understand and participate in the substance of the discussion (links to other documents are fine here). * The **discussion**, the primary content of the document. -The [rfc-template.md](/sdk/v0.50/build/rfc/rfc-template) file includes placeholders for these sections. +The `rfc-template.md` file includes placeholders for these sections. ## Table of Contents[​](#table-of-contents "Direct link to Table of Contents") diff --git a/sdk/v0.50/build/rfc/README.mdx b/sdk/v0.50/build/rfc/README.mdx index 55b2e66b8..1da672011 100644 --- a/sdk/v0.50/build/rfc/README.mdx +++ b/sdk/v0.50/build/rfc/README.mdx @@ -34,7 +34,7 @@ An RFC should provide: substance of the discussion (links to other documents are fine here). * The **discussion**, the primary content of the document. -The [rfc-template.md](/sdk/v0.50/build/rfc/rfc-template) file includes placeholders for these +The `rfc-template.md` file includes placeholders for these sections. ## Table of Contents diff --git a/sdk/v0.50/build/rfc/rfc-template.mdx b/sdk/v0.50/build/rfc/rfc-template.mdx deleted file mode 100644 index da5eeb8f3..000000000 --- a/sdk/v0.50/build/rfc/rfc-template.mdx +++ /dev/null @@ -1,82 +0,0 @@ ---- -noindex: true -canonical: 'https://docs.cosmos.network/sdk/latest/' ---- - -## Changelog - -* `{date}`: `{changelog}` - -## Background - -> The next section is the "Background" section. This section should be at least two paragraphs and can take up to a whole -> page in some cases. The guiding goal of the background section is: as a newcomer to this project (new employee, team -> transfer), can I read the background section and follow any links to get the full context of why this change is\ -> necessary? -> -> If you can't show a random engineer the background section and have them acquire nearly full context on the necessity -> for the RFC, then the background section is not full enough. To help achieve this, link to prior RFCs, discussions, and -> more here as necessary to provide context so you don't have to simply repeat yourself. - -## Proposal - -> The next required section is "Proposal" or "Goal". Given the background above, this section proposes a solution. -> This should be an overview of the "how" for the solution, but for details further sections will be used. - -## Abandoned Ideas (Optional) - -> As RFCs evolve, it is common that there are ideas that are abandoned. Rather than simply deleting them from the -> document, you should try to organize them into sections that make it clear they're abandoned while explaining why they -> were abandoned. -> -> When sharing your RFC with others or having someone look back on your RFC in the future, it is common to walk the same -> path and fall into the same pitfalls that we've since matured from. Abandoned ideas are a way to recognize that path -> and explain the pitfalls and why they were abandoned. - -## Descision - -> This section describes alternative designs to the chosen design. This section -> is important and if an adr does not have any alternatives then it should be -> considered that the ADR was not thought through. - -## Consequences (optional) - -> This section describes the resulting context, after applying the decision. All -> consequences should be listed here, not just the "positive" ones. A particular -> decision may have positive, negative, and neutral consequences, but all of them -> affect the team and project in the future. - -### Backwards Compatibility - -> All ADRs that introduce backwards incompatibilities must include a section -> describing these incompatibilities and their severity. The ADR must explain -> how the author proposes to deal with these incompatibilities. ADR submissions -> without a sufficient backwards compatibility treatise may be rejected outright. - -### Positive - -> `{positive consequences}` - -### Negative - -> `{negative consequences}` - -### Neutral - -> `{neutral consequences}` - -### References - -> Links to external materials needed to follow the discussion may be added here. -> -> In addition, if the discussion in a request for comments leads to any design -> decisions, it may be helpful to add links to the ADR documents here after the -> discussion has settled. - -## Discussion - -> This section contains the core of the discussion. -> -> There is no fixed format for this section, but ideally changes to this -> section should be updated before merging to reflect any discussion that took -> place on the PR that made those changes. diff --git a/sdk/v0.50/build/rfc/rfc/README.mdx b/sdk/v0.50/build/rfc/rfc/README.mdx index 55b2e66b8..1da672011 100644 --- a/sdk/v0.50/build/rfc/rfc/README.mdx +++ b/sdk/v0.50/build/rfc/rfc/README.mdx @@ -34,7 +34,7 @@ An RFC should provide: substance of the discussion (links to other documents are fine here). * The **discussion**, the primary content of the document. -The [rfc-template.md](/sdk/v0.50/build/rfc/rfc-template) file includes placeholders for these +The `rfc-template.md` file includes placeholders for these sections. ## Table of Contents diff --git a/sdk/v0.50/build/rfc/rfc/rfc-template.mdx b/sdk/v0.50/build/rfc/rfc/rfc-template.mdx deleted file mode 100644 index da5eeb8f3..000000000 --- a/sdk/v0.50/build/rfc/rfc/rfc-template.mdx +++ /dev/null @@ -1,82 +0,0 @@ ---- -noindex: true -canonical: 'https://docs.cosmos.network/sdk/latest/' ---- - -## Changelog - -* `{date}`: `{changelog}` - -## Background - -> The next section is the "Background" section. This section should be at least two paragraphs and can take up to a whole -> page in some cases. The guiding goal of the background section is: as a newcomer to this project (new employee, team -> transfer), can I read the background section and follow any links to get the full context of why this change is\ -> necessary? -> -> If you can't show a random engineer the background section and have them acquire nearly full context on the necessity -> for the RFC, then the background section is not full enough. To help achieve this, link to prior RFCs, discussions, and -> more here as necessary to provide context so you don't have to simply repeat yourself. - -## Proposal - -> The next required section is "Proposal" or "Goal". Given the background above, this section proposes a solution. -> This should be an overview of the "how" for the solution, but for details further sections will be used. - -## Abandoned Ideas (Optional) - -> As RFCs evolve, it is common that there are ideas that are abandoned. Rather than simply deleting them from the -> document, you should try to organize them into sections that make it clear they're abandoned while explaining why they -> were abandoned. -> -> When sharing your RFC with others or having someone look back on your RFC in the future, it is common to walk the same -> path and fall into the same pitfalls that we've since matured from. Abandoned ideas are a way to recognize that path -> and explain the pitfalls and why they were abandoned. - -## Descision - -> This section describes alternative designs to the chosen design. This section -> is important and if an adr does not have any alternatives then it should be -> considered that the ADR was not thought through. - -## Consequences (optional) - -> This section describes the resulting context, after applying the decision. All -> consequences should be listed here, not just the "positive" ones. A particular -> decision may have positive, negative, and neutral consequences, but all of them -> affect the team and project in the future. - -### Backwards Compatibility - -> All ADRs that introduce backwards incompatibilities must include a section -> describing these incompatibilities and their severity. The ADR must explain -> how the author proposes to deal with these incompatibilities. ADR submissions -> without a sufficient backwards compatibility treatise may be rejected outright. - -### Positive - -> `{positive consequences}` - -### Negative - -> `{negative consequences}` - -### Neutral - -> `{neutral consequences}` - -### References - -> Links to external materials needed to follow the discussion may be added here. -> -> In addition, if the discussion in a request for comments leads to any design -> decisions, it may be helpful to add links to the ADR documents here after the -> discussion has settled. - -## Discussion - -> This section contains the core of the discussion. -> -> There is no fixed format for this section, but ideally changes to this -> section should be updated before merging to reflect any discussion that took -> place on the PR that made those changes. diff --git a/sdk/v0.53/build/architecture/adr-template.mdx b/sdk/v0.53/build/architecture/adr-template.mdx deleted file mode 100644 index 99c4b3269..000000000 --- a/sdk/v0.53/build/architecture/adr-template.mdx +++ /dev/null @@ -1,87 +0,0 @@ ---- -noindex: true -canonical: 'https://docs.cosmos.network/sdk/latest/' ---- - -## Changelog - -* `{date}`: `{changelog}` - -## Status - -{DRAFT | PROPOSED} Not Implemented - -> Please have a look at the [PROCESS](/sdk/v0.50/build/rfc/PROCESS#adr-status) page. -> Use DRAFT if the ADR is in a draft stage (draft PR) or PROPOSED if it's in review. - -## Abstract - -> "If you can't explain it simply, you don't understand it well enough." Provide -> a simplified and layman-accessible explanation of the ADR. -> A short (\~200 word) description of the issue being addressed. - -## Context - -> This section describes the forces at play, including technological, political, -> social, and project local. These forces are probably in tension, and should be -> called out as such. The language in this section is value-neutral. It is simply -> describing facts. It should clearly explain the problem and motivation that the -> proposal aims to resolve. - -`{context body}` - -## Alternatives - -> This section describes alternative designs to the chosen design. This section -> is important and if an adr does not have any alternatives then it should be -> considered that the ADR was not thought through. - -## Decision - -> This section describes our response to these forces. It is stated in full -> sentences, with active voice. "We will ..." -> `{decision body}` - -## Consequences - -> This section describes the resulting context, after applying the decision. All -> consequences should be listed here, not just the "positive" ones. A particular -> decision may have positive, negative, and neutral consequences, but all of them -> affect the team and project in the future. - -### Backwards Compatibility - -> All ADRs that introduce backwards incompatibilities must include a section -> describing these incompatibilities and their severity. The ADR must explain -> how the author proposes to deal with these incompatibilities. ADR submissions -> without a sufficient backwards compatibility treatise may be rejected outright. - -### Positive - -> `{positive consequences}` - -### Negative - -> `{negative consequences}` - -### Neutral - -> `{neutral consequences}` - -## Further Discussions - -> While an ADR is in the DRAFT or PROPOSED stage, this section should contain a -> summary of issues to be solved in future iterations (usually referencing comments -> from a pull-request discussion). -> -> Later, this section can optionally list ideas or improvements the author or -> reviewers found during the analysis of this ADR. - -## Test Cases \[optional] - -Test cases for an implementation are mandatory for ADRs that are affecting consensus -changes. Other ADRs can choose to include links to test cases if applicable. - -## References - -* `{reference link}` diff --git a/sdk/v0.53/build/rfc.mdx b/sdk/v0.53/build/rfc.mdx index 5a46be331..fbce7bc0d 100644 --- a/sdk/v0.53/build/rfc.mdx +++ b/sdk/v0.53/build/rfc.mdx @@ -20,7 +20,7 @@ An RFC should provide: * Any **background** a reader will need to understand and participate in the substance of the discussion (links to other documents are fine here). * The **discussion**, the primary content of the document. -The [rfc-template.md](/sdk/v0.53/build/rfc/rfc-template) file includes placeholders for these sections. +The `rfc-template.md` file includes placeholders for these sections. ## Table of Contents[​](#table-of-contents "Direct link to Table of Contents") diff --git a/sdk/v0.53/build/rfc/README.mdx b/sdk/v0.53/build/rfc/README.mdx index b701dd52e..eecfab17a 100644 --- a/sdk/v0.53/build/rfc/README.mdx +++ b/sdk/v0.53/build/rfc/README.mdx @@ -34,7 +34,7 @@ An RFC should provide: substance of the discussion (links to other documents are fine here). * The **discussion**, the primary content of the document. -The [rfc-template.md](/sdk/v0.50/build/rfc/rfc-template) file includes placeholders for these +The `rfc-template.md` file includes placeholders for these sections. ## Table of Contents diff --git a/sdk/v0.53/build/rfc/rfc-template.mdx b/sdk/v0.53/build/rfc/rfc-template.mdx deleted file mode 100644 index d134c90a9..000000000 --- a/sdk/v0.53/build/rfc/rfc-template.mdx +++ /dev/null @@ -1,82 +0,0 @@ ---- -noindex: true -canonical: 'https://docs.cosmos.network/sdk/latest/' ---- - -## Changelog - -* `{date}`: `{changelog}` - -## Background - -> The next section is the "Background" section. This section should be at least two paragraphs and can take up to a whole -> page in some cases. The guiding goal of the background section is: as a newcomer to this project (new employee, team -> transfer), can I read the background section and follow any links to get the full context of why this change is\ -> necessary? -> -> If you can't show a random engineer the background section and have them acquire nearly full context on the necessity -> for the RFC, then the background section is not full enough. To help achieve this, link to prior RFCs, discussions, and -> more here as necessary to provide context so you don't have to simply repeat yourself. - -## Proposal - -> The next required section is "Proposal" or "Goal". Given the background above, this section proposes a solution. -> This should be an overview of the "how" for the solution, but for details further sections will be used. - -## Abandoned Ideas (Optional) - -> As RFCs evolve, it is common that there are ideas that are abandoned. Rather than simply deleting them from the -> document, you should try to organize them into sections that make it clear they're abandoned while explaining why they -> were abandoned. -> -> When sharing your RFC with others or having someone look back on your RFC in the future, it is common to walk the same -> path and fall into the same pitfalls that we've since matured from. Abandoned ideas are a way to recognize that path -> and explain the pitfalls and why they were abandoned. - -## Decision - -> This section describes alternative designs to the chosen design. This section -> is important and if an ADR does not have any alternatives then it should be -> considered that the ADR was not thought through. - -## Consequences (optional) - -> This section describes the resulting context, after applying the decision. All -> consequences should be listed here, not just the "positive" ones. A particular -> decision may have positive, negative, and neutral consequences, but all of them -> affect the team and project in the future. - -### Backwards Compatibility - -> All ADRs that introduce backwards incompatibilities must include a section -> describing these incompatibilities and their severity. The ADR must explain -> how the author proposes to deal with these incompatibilities. ADR submissions -> without a sufficient backwards compatibility treatise may be rejected outright. - -### Positive - -> `{positive consequences}` - -### Negative - -> `{negative consequences}` - -### Neutral - -> `{neutral consequences}` - -### References - -> Links to external materials needed to follow the discussion may be added here. -> -> In addition, if the discussion in a request for comments leads to any design -> decisions, it may be helpful to add links to the ADR documents here after the -> discussion has settled. - -## Discussion - -> This section contains the core of the discussion. -> -> There is no fixed format for this section, but ideally changes to this -> section should be updated before merging to reflect any discussion that took -> place on the PR that made those changes. diff --git a/sdk/v0.54/changelog/release-notes.mdx b/sdk/v0.54/changelog/release-notes.mdx new file mode 100644 index 000000000..70f268b85 --- /dev/null +++ b/sdk/v0.54/changelog/release-notes.mdx @@ -0,0 +1,126 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/changelog/release-notes' +title: "Changelog" +description: "Release history and changelog for Cosmos SDK" +mode: "wide" +--- + + + This page tracks releases and changes for v0.54.2. For the full release history, see the [CHANGELOG](https://github.com/cosmos/cosmos-sdk/blob/main/CHANGELOG.md) on GitHub. + + + +This patch release contains only minor dependency bumps. + + + +## Improvements + +- (x/auth) [#26297](https://github.com/cosmos/cosmos-sdk/pull/26297) Cap pagination limit at number of txs within block during `GetBlockWithTxs` instead of 100. + + + +## Breaking Changes + +- (x/consensus) [#25607](https://github.com/cosmos/cosmos-sdk/pull/25607) Add `AuthorityParams` to consensus params. When set, the consensus params authority takes precedence over per-keeper authority for all module parameter updates. Keeper constructor signatures are unchanged. +- (x/staking) [#25724](https://github.com/cosmos/cosmos-sdk/issues/25724) Validate `BondDenom` in `MsgUpdateParams` to prevent setting non-existent or zero-supply denoms. +- [#25778](https://github.com/cosmos/cosmos-sdk/pull/25778) Update `log` to log v2. +- [#25546](https://github.com/cosmos/cosmos-sdk/pull/25546) Removed `x/params`: + - Removes all `legacySubspace` arguments from Keeper and Module instantiation +- [#25090](https://github.com/cosmos/cosmos-sdk/pull/25090) Moved deprecated modules to `./contrib`. These modules are still available but will no longer be actively maintained or supported in the Cosmos SDK Bug Bounty program. + - `x/group` + - `x/nft` + - `x/circuit` + - `x/crisis` +- (crypto) [#24414](https://github.com/cosmos/cosmos-sdk/pull/24414) Remove sr25519 support, since it was removed in CometBFT v1.x (see: CometBFT [#3646](https://github.com/cometbft/cometbft/pull/3646)). +- (x/mint) [#25599](https://github.com/cosmos/cosmos-sdk/pull/25599) Add max supply param. +- (x/gov) [#25615](https://github.com/cosmos/cosmos-sdk/pull/25615) Decouple `x/gov` from `x/staking` by making `CalculateVoteResultsAndVotingPowerFn` a required parameter to `keeper.NewKeeper` instead of `StakingKeeper`. +`BondedTokens` has been renamed to `ValidatorPower` and `TotalBondedTokens` has been renamed to `TotalValidatorPower` to allow for multiple validator power representations. +- (x/gov) [#25617](https://github.com/cosmos/cosmos-sdk/pull/25617) `AfterProposalSubmission` hook now includes proposer address as a parameter. +- (x/gov) [#25616](https://github.com/cosmos/cosmos-sdk/pull/25616) `DistrKeeper` `x/distribution` is now optional. Genesis validation ensures `distrKeeper` is set if distribution module is used as proposal cancel destination. +- (systemtests) [#25930]https://github.com/cosmos/cosmos-sdk/pull/25930) Move `systemtests` into `testutil` and no longer under its own `go.mod`. +- (baseapp) [#26060](https://github.com/cosmos/cosmos-sdk/pull/26060) Remove `BaseApp.SetStoreMetrics`. The `StoreMetrics` interface never worked, so removing dead code. +- (store) [#26061](https://github.com/cosmos/cosmos-sdk/pull/26061) Remove store tracing API and all related plumbing: + - Remove `SetTracer`, `SetTracingContext`, and `TracingEnabled` from `MultiStore` interface. + - Remove `CacheWrapWithTrace` from `CacheWrapper` interface. + - Remove `BaseApp.SetCommitMultiStoreTracer` and tracing context logic from `BaseApp.cacheTxContext` and `FinalizeBlock`. + - Remove `io.Writer` parameter from `servertypes.AppCreator` and `traceWriter io.Writer` from `servertypes.AppExporter`. + - Remove `traceStore io.Writer` parameter from `simapp.NewSimApp` and all enterprise simapp constructors. + - Remove `traceStore io.Writer` from all `testutil/simsx` app factory signatures. +- (store) [#26042](https://github.com/cosmos/cosmos-sdk/pull/26042) We are now importing `github.com/cosmos/cosmos-sdk/store/v2` as the store package instead of `cosmossdk.io/store` and all import paths have changed. +- (baseapp) [#26138](https://github.com/cosmos/cosmos-sdk/pull/26138) Default block gas meter to disabled. Adds checking to ensure block gas meter is not enabled while bstm parallel execution is configured and panics in these scenarios during parameter assignment. + +## Features + +- [#25471](https://github.com/cosmos/cosmos-sdk/pull/25471) Full BLS 12-381 support enabled. +- [#24872](https://github.com/cosmos/cosmos-sdk/pull/24872) Support BLS 12-381 for cli `init`, `gentx`, `collect-gentx` +- (crypto) [#24919](https://github.com/cosmos/cosmos-sdk/pull/24919) add `NewPubKeyFromBytes` function to the `secp256r1` package to create `PubKey` from bytes +- (server) [#24720](https://github.com/cosmos/cosmos-sdk/pull/24720) add `verbose_log_level` flag for configuring the log level when switching to verbose logging mode during sensitive operations (such as chain upgrades). +- (crypto) [#24861](https://github.com/cosmos/cosmos-sdk/pull/24861) add `PubKeyFromCometTypeAndBytes` helper function to convert from `comet/v2` PubKeys to the `cryptotypes.Pubkey` interface. +- (abci_utils) [#25008](https://github.com/cosmos/cosmos-sdk/pull/25008) add the ability to assign a custom signer extraction adapter in `DefaultProposalHandler`. +- (x/distribution) [#25650](https://github.com/cosmos/cosmos-sdk/pull/25650) Add new gRPC query endpoints and CLI commands for `DelegatorStartingInfo`, `ValidatorHistoricalRewards`, and `ValidatorCurrentRewards`. +- [#25745](https://github.com/cosmos/cosmos-sdk/pull/25745) Add DiskIO telemetry via gopsutil. +- (grpc) [#25648](https://github.com/cosmos/cosmos-sdk/pull/25648) Add `earliest_block_height` and `latest_block_height` fields to `GetSyncingResponse`. +- (collections/codec) [#25614] (https://github.com/cosmos/cosmos-sdk/pull/25827) Add `TimeValue` (`ValueCodec[time.Time]`) to collections/codec. +- (enterprise/poa) [#25838](https://github.com/cosmos/cosmos-sdk/pull/25838) Add the `poa` module under the `enterprise` directory. +- (grpc) [#25850](https://github.com/cosmos/cosmos-sdk/pull/25850) Add `GetBlockResults` and `GetLatestBlockResults` gRPC endpoints to expose CometBFT block results including `finalize_block_events`. + +## Improvements + +- (ci) Use softprops/action-gh-release for main-nightly instead of custom gh/git to avoid repository ruleset conflicts. +- (telemetry) [#26006](https://github.com/cosmos/cosmos-sdk/pull/26006) Export `ExtensionOptions` type for programmatic otel.yaml generation. +- [#25955](https://github.com/cosmos/cosmos-sdk/pull/25955) Use cosmos/btree directly instead of replacing it in go.mods +- (types) [#25342](https://github.com/cosmos/cosmos-sdk/pull/25342) Undeprecated `EmitEvent` and `EmitEvents` on the `EventManager`. These functions will continue to be maintained. +- (types) [#24668](https://github.com/cosmos/cosmos-sdk/pull/24668) Scope the global config to a particular binary so that multiple SDK binaries can be properly run on the same machine. +- (baseapp) [#24655](https://github.com/cosmos/cosmos-sdk/pull/24655) Add mutex locks for `state` and make `lastCommitInfo` atomic to prevent race conditions between `Commit` and `CreateQueryContext`. +- (proto) [#24161](https://github.com/cosmos/cosmos-sdk/pull/24161) Remove unnecessary annotations from `x/staking` authz proto. +- (x/bank) [#24660](https://github.com/cosmos/cosmos-sdk/pull/24660) Improve performance of the `GetAllBalances` and `GetAccountsBalances` keeper methods. +- (collections) [#25464](https://github.com/cosmos/cosmos-sdk/pull/25464) Add `IterateRaw` method to `Multi` index type to satisfty query `Collection` interface. +- (api) [#25613](https://github.com/cosmos/cosmos-sdk/pull/25613) Separated deprecated modules into the contrib directory, distinct from api, to enable and unblock new proto changes without affecting legacy code. +- (server) [#25740](https://github.com/cosmos/cosmos-sdk/pull/25740) Add variadic `grpc.DialOption` parameter to `StartGrpcServer` for custom gRPC client connection options. +- (blockstm) [#25765](https://github.com/cosmos/cosmos-sdk/pull/25765) Minor code readability improvement in block-stm. +- (blockstm) [#25786](https://github.com/cosmos/cosmos-sdk/pull/25786) Add pre-state checking in transaction state transition. +- (server/config) [#25807](https://github.com/cosmos/cosmos-sdk/pull/25807) fix(server): reject overlapping historical gRPC block ranges. +- [#25857](https://github.com/cosmos/cosmos-sdk/pull/25857) Reduce scope of mutex in `PriorityNonceMempool.Remove`. +- (baseapp) [#25862](https://github.com/cosmos/cosmos-sdk/pull/25862) Skip running validateBasic for rechecking txs. (Backport of https://github.com/cosmos/cosmos-sdk/pull/20208). +- (blockstm) [25883](https://github.com/cosmos/cosmos-sdk/pull/25883) Re-use decoded tx object in pre-estimates. +- (blockstm) [#25788](https://github.com/cosmos/cosmos-sdk/pull/25788) Only validate transactions that's executed at lease once. +- (blockstm) [#25767](https://github.com/cosmos/cosmos-sdk/pull/25767) Optimize block-stm MVMemory with bitmap index. + +## Bug Fixes + +- (baseapp) [#25331](https://github.com/cosmos/cosmos-sdk/issues/25331) Avoid noisy errors when gRPC response headers are already sent, set block height as a header when possible and fall back to a trailer. +- (blockstm) [#25789](https://github.com/cosmos/cosmos-sdk/issues/25789) Wake up suspended executors when scheduler doesn't complete to prevent goroutine leaks. +- (grpc) [#25647](https://github.com/cosmos/cosmos-sdk/pull/25647) Return actual `earliest_store_height` in `node.Status` gRPC endpoint instead of hardcoded `0`. +- (types/query) [#25665](https://github.com/cosmos/cosmos-sdk/issues/25665) Fix pagination offset when querying a collection with predicate function. +- (x/staking) [#25649](https://github.com/cosmos/cosmos-sdk/pull/25649) Add missing `defer iterator.Close()` calls in `IterateDelegatorRedelegations` and `GetRedelegations` to prevent resource leaks. +- (mempool) [#25563](https://github.com/cosmos/cosmos-sdk/pull/25563) Cleanup sender indices in case of tx replacement. +- (x/epochs) [#25425](https://github.com/cosmos/cosmos-sdk/pull/25425) Fix `InvokeSetHooks` being called with a nil keeper and `AppModule` containing a copy instead of a pointer (hooks set post creating the `AppModule` like with depinject didn't apply because it's a different instance). +- (client, client/rpc, x/auth/tx) [#24551](https://github.com/cosmos/cosmos-sdk/pull/24551) Handle cancellation properly when supplying context to client methods. +- (x/authz) [#24638](https://github.com/cosmos/cosmos-sdk/pull/24638) Fixed a minor bug where the grant key was cast as a string and dumped directly into the error message leading to an error string possibly containing invalid UTF-8. +- (client, client/rpc, x/auth/tx) [#24551](https://github.com/cosmos/cosmos-sdk/pull/24551) Handle cancellation properly when supplying context to client methods. +- (x/epochs) [#24770](https://github.com/cosmos/cosmos-sdk/pull/24770) Fix register of epoch hooks in `InvokeSetHooks`. +- (x/epochs) [#25087](https://github.com/cosmos/cosmos-sdk/pull/25087) Remove redundant error check in BeginBlocker. +- [GHSA-p22h-3m2v-cmgh](https://github.com/cosmos/cosmos-sdk/security/advisories/GHSA-p22h-3m2v-cmgh) Fix x/distribution can halt when historical rewards overflow. +- (x/staking) [#25258](https://github.com/cosmos/cosmos-sdk/pull/25258) Add delegator address to redelegate event. +- (x/bank) [#25751](https://github.com/cosmos/cosmos-sdk/pull/25751) Fix recipient address in events. +- (client) [#25811] (https://github.com/cosmos/cosmos-sdk/pull/25811) fix(client): fix file handle leaks in snapshot commands. +- (server/config) [#25806](https://github.com/cosmos/cosmos-sdk/pull/25806) fix: add missing commas in historical gRPC config template. +- (client) [#25804](https://github.com/cosmos/cosmos-sdk/pull/25804) Add `GetHeightFromMetadataStrict` API to `grpc` client for better error handling. +- (x/staking) [#25829](https://github.com/cosmos/cosmos-sdk/pull/25829) Validates case-sensitivity on authz grands in x/staking. +- (mempool) [#25869](https://github.com/cosmos/cosmos-sdk/pull/25869) fix(mempool): add thread safety to NextSenderTx. +- (blockstm) [#25912](https://github.com/cosmos/cosmos-sdk/pull/25912) Remove `SigVerificationDecorator` signature incarnation cache causing state divergence under blockstm. +- (x/group) [#25922](https://github.com/cosmos/cosmos-sdk/pull/25922) Add zero-total-weight check for ThresholdDecisionPolicy +- (x/group) [#25917](https://github.com/cosmos/cosmos-sdk/pull/25917) Prevent creation of zero-weight groups. +- (x/group) [#25919](https://github.com/cosmos/cosmos-sdk/pull/25919) add safer type assertions to group `DecisionPolicy` getter calls. +- (x/group) [#25920](https://github.com/cosmos/cosmos-sdk/pull/25920) Expand voting period check to verify period is positive instead of nonzero. +- (types/address) [#25944] (https://github.com/cosmos/cosmos-sdk/pull/25944) correct sort comparator in Compose to satisfy strict weak ordering. +- (baseapp) [#26063](https://github.com/cosmos/cosmos-sdk/pull/26063) Fixes an issue where values embedded in context during ante handling were wiped after the handlers returned. +- (collections/indexes) [#25942](https://github.com/cosmos/cosmos-sdk/pull/25942) handle iterator close errors in index helpers. + +## Deprecated + +- [#25948](https://github.com/cosmos/cosmos-sdk/pull/25948) Change default `app.go` code to not use `depinject` as we are phasing it out. +- (baseapp) [#26107](https://github.com/cosmos/cosmos-sdk/pull/26170) Deprecate baseapp test helper `app.NewUncachedContext`, consider using `app.NewNextBlockContext` or `app.NewContext` instead, see `UPGRADING.md` for more details. + diff --git a/sdk/v0.54/enterprise/group/api.mdx b/sdk/v0.54/enterprise/group/api.mdx new file mode 100644 index 000000000..e3e56f641 --- /dev/null +++ b/sdk/v0.54/enterprise/group/api.mdx @@ -0,0 +1,758 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/enterprise/group/api' +title: "API Reference" +description: "Complete API reference for Group module queries and messages" +--- + +# Group Module API Reference + +## Overview + +The Group module provides a comprehensive API for managing on-chain multisig groups and collective decision-making. + +**Package:** `cosmos.group.v1` +**Go Import:** `github.com/cosmos/cosmos-sdk/enterprise/group/x/group` + +--- + +## Data Types + +### GroupInfo + +Represents a group on-chain. + +```protobuf +message GroupInfo { + uint64 id = 1; + string admin = 2; + bytes metadata = 3; + uint64 version = 4; + string total_weight = 5; + google.protobuf.Timestamp created_at = 6; +} +``` + +**Fields:** +- `id` (uint64): Unique group identifier, auto-assigned on creation +- `admin` (string): Cosmos SDK address of the group administrator +- `metadata` (bytes): Optional group metadata +- `version` (uint64): Incremented on every group update; used to detect stale proposals +- `total_weight` (string): Sum of all member weights +- `created_at` (Timestamp): Block time when the group was created + +--- + +### GroupMember + +Represents a member's relationship to a group. + +```protobuf +message GroupMember { + uint64 group_id = 1; + Member member = 2; +} + +message Member { + string address = 1; + string weight = 2; + bytes metadata = 3; + google.protobuf.Timestamp added_at = 4; +} +``` + +**Fields:** +- `address` (string): Cosmos SDK address of the member +- `weight` (string): Voting weight. Set to `"0"` to remove a member. +- `metadata` (bytes): Optional member metadata +- `added_at` (Timestamp): Block time when the member was added + +--- + +### GroupPolicyInfo + +Represents a group policy account. + +```protobuf +message GroupPolicyInfo { + string address = 1; + uint64 group_id = 2; + string admin = 3; + bytes metadata = 4; + uint64 version = 5; + google.protobuf.Any decision_policy = 6; + google.protobuf.Timestamp created_at = 7; +} +``` + +**Fields:** +- `address` (string): The group policy's account address (auto-generated) +- `group_id` (uint64): The group this policy is associated with +- `admin` (string): Address with authority to update the policy +- `decision_policy` (Any): The policy's decision logic (threshold or percentage) +- `version` (uint64): Incremented on every update; used to detect aborted proposals + +--- + +### Proposal + +Represents an on-chain proposal submitted to a group policy. + +```protobuf +message Proposal { + uint64 id = 1; + string group_policy_address = 2; + bytes metadata = 3; + repeated string proposers = 4; + google.protobuf.Timestamp submit_time = 5; + uint64 group_version = 6; + uint64 group_policy_version = 7; + ProposalStatus status = 8; + TallyResult final_tally_result = 9; + google.protobuf.Timestamp voting_period_end = 10; + ProposalExecutorResult executor_result = 11; + repeated google.protobuf.Any messages = 12; + string title = 13; + string summary = 14; +} +``` + +**ProposalStatus values:** +- `PROPOSAL_STATUS_SUBMITTED` - Open for voting +- `PROPOSAL_STATUS_ACCEPTED` - Passed; ready for execution +- `PROPOSAL_STATUS_REJECTED` - Failed tally +- `PROPOSAL_STATUS_ABORTED` - Group or policy updated during voting +- `PROPOSAL_STATUS_WITHDRAWN` - Withdrawn by proposer or policy admin + +**ProposalExecutorResult values:** +- `PROPOSAL_EXECUTOR_RESULT_NOT_RUN` +- `PROPOSAL_EXECUTOR_RESULT_SUCCESS` +- `PROPOSAL_EXECUTOR_RESULT_FAILURE` + +--- + +### TallyResult + +The accumulated vote counts for a proposal. + +```protobuf +message TallyResult { + string yes_count = 1; + string abstain_count = 2; + string no_count = 3; + string no_with_veto_count = 4; +} +``` + +--- + +## Query API + +The Query service provides read-only access to Group module state. + +### GroupInfo + +Get information about a group by ID. + +**gRPC:** `cosmos.group.v1.Query/GroupInfo` +**REST:** `GET /cosmos/group/v1/groups/{group_id}` + +**CLI:** +```bash +simd q group group-info [group-id] +``` + +**Example:** +```bash +simd q group group-info 1 +``` + +--- + +### GroupPolicyInfo + +Get information about a group policy account. + +**gRPC:** `cosmos.group.v1.Query/GroupPolicyInfo` +**REST:** `GET /cosmos/group/v1/group_policies/{address}` + +**CLI:** +```bash +simd q group group-policy-info [group-policy-account] +``` + +--- + +### GroupMembers + +List all members of a group. + +**gRPC:** `cosmos.group.v1.Query/GroupMembers` +**REST:** `GET /cosmos/group/v1/groups/{group_id}/members` + +**CLI:** +```bash +simd q group group-members [group-id] +``` + +--- + +### GroupsByAdmin + +List all groups administered by a given address. + +**gRPC:** `cosmos.group.v1.Query/GroupsByAdmin` +**REST:** `GET /cosmos/group/v1/groups/by_admin/{admin}` + +**CLI:** +```bash +simd q group groups-by-admin [admin] +``` + +--- + +### GroupPoliciesByGroup + +List all group policies associated with a group. + +**gRPC:** `cosmos.group.v1.Query/GroupPoliciesByGroup` +**REST:** `GET /cosmos/group/v1/groups/{group_id}/group_policies` + +**CLI:** +```bash +simd q group group-policies-by-group [group-id] +``` + +--- + +### GroupPoliciesByAdmin + +List all group policies administered by a given address. + +**gRPC:** `cosmos.group.v1.Query/GroupPoliciesByAdmin` +**REST:** `GET /cosmos/group/v1/group_policies/by_admin/{admin}` + +**CLI:** +```bash +simd q group group-policies-by-admin [admin] +``` + +--- + +### Proposal + +Get a proposal by ID. + +**gRPC:** `cosmos.group.v1.Query/Proposal` +**REST:** `GET /cosmos/group/v1/proposals/{proposal_id}` + +**CLI:** +```bash +simd q group proposal [proposal-id] +``` + +--- + +### ProposalsByGroupPolicy + +List all proposals for a given group policy account. + +**gRPC:** `cosmos.group.v1.Query/ProposalsByGroupPolicy` +**REST:** `GET /cosmos/group/v1/proposals/by_group_policy/{address}` + +**CLI:** +```bash +simd q group proposals-by-group-policy [group-policy-account] +``` + +--- + +### VoteByProposalVoter + +Get a specific vote on a proposal. + +**gRPC:** `cosmos.group.v1.Query/VoteByProposalVoter` +**REST:** `GET /cosmos/group/v1/votes/{proposal_id}/{voter}` + +**CLI:** +```bash +simd q group vote [proposal-id] [voter] +``` + +--- + +### VotesByProposal + +List all votes on a proposal. + +**gRPC:** `cosmos.group.v1.Query/VotesByProposal` +**REST:** `GET /cosmos/group/v1/votes/by_proposal/{proposal_id}` + +**CLI:** +```bash +simd q group votes-by-proposal [proposal-id] +``` + +--- + +### TallyResult + +Get the current tally for a proposal. + +**gRPC:** `cosmos.group.v1.Query/TallyResult` +**REST:** `GET /cosmos/group/v1/proposals/{proposal_id}/tally` + +**CLI:** +```bash +simd q group tally-result [proposal-id] +``` + +**Example Response:** +```json +{ + "tally": { + "yes_count": "2", + "abstain_count": "0", + "no_count": "1", + "no_with_veto_count": "0" + } +} +``` + +--- + +### Groups + +List all groups on chain. + +**gRPC:** `cosmos.group.v1.Query/Groups` +**REST:** `GET /cosmos/group/v1/groups` + +**CLI:** +```bash +simd q group groups +``` + +--- + +## Transaction Messages (Msg Service) + +### CreateGroup + +Create a new group with an admin and initial members. + +**Msg:** `MsgCreateGroup` + +**CLI:** +```bash +simd tx group create-group [admin] [metadata] [members-json-file] +``` + +**Members JSON format:** +```json +{ + "members": [ + { + "address": "cosmos1...", + "weight": "1", + "metadata": "member description" + } + ] +} +``` + +**Authorization:** Any address can create a group. + +**Failure conditions:** +- Metadata length exceeds `MaxMetadataLen` +- Members have invalid addresses, duplicate entries, or zero weight + +--- + +### UpdateGroupMembers + +Add, remove, or reweight members in a group. + +**Msg:** `MsgUpdateGroupMembers` + +**CLI:** +```bash +simd tx group update-group-members [admin] [group-id] [members-json-file] +``` + +**Note:** Set a member's weight to `"0"` to remove them from the group. + +**Authorization:** Must be signed by the group admin. + +**Failure conditions:** +- Signer is not the group admin +- Any associated group policy's `Validate()` method fails against the updated member set + +--- + +### UpdateGroupAdmin + +Transfer group administration to a new address. + +**Msg:** `MsgUpdateGroupAdmin` + +**CLI:** +```bash +simd tx group update-group-admin [admin] [group-id] [new-admin] +``` + +**Authorization:** Must be signed by the current group admin. + +--- + +### UpdateGroupMetadata + +Update a group's metadata. + +**Msg:** `MsgUpdateGroupMetadata` + +**CLI:** +```bash +simd tx group update-group-metadata [admin] [group-id] [metadata] +``` + +**Authorization:** Must be signed by the group admin. + +--- + +### CreateGroupPolicy + +Create a new group policy account with a decision policy. + +**Msg:** `MsgCreateGroupPolicy` + +**CLI:** +```bash +simd tx group create-group-policy [admin] [group-id] [metadata] [decision-policy-json] +``` + +**Threshold policy example:** +```json +{ + "@type": "/cosmos.group.v1.ThresholdDecisionPolicy", + "threshold": "2", + "windows": { + "voting_period": "24h", + "min_execution_period": "0s" + } +} +``` + +**Percentage policy example:** +```json +{ + "@type": "/cosmos.group.v1.PercentageDecisionPolicy", + "percentage": "0.5", + "windows": { + "voting_period": "48h", + "min_execution_period": "0s" + } +} +``` + +**Authorization:** Must be signed by the group admin. + +**Failure conditions:** +- Signer is not the group admin +- Metadata length exceeds `MaxMetadataLen` +- Decision policy's `Validate()` method fails against the group + +--- + +### CreateGroupWithPolicy + +Create a group and a group policy in a single transaction. + +**Msg:** `MsgCreateGroupWithPolicy` + +**CLI:** +```bash +simd tx group create-group-with-policy [admin] [group-metadata] [group-policy-metadata] [members-json-file] [decision-policy-json] +``` + +Set `--group-policy-as-admin` to make the group policy account the group admin (enabling a self-governed group). + +--- + +### UpdateGroupPolicyAdmin + +Transfer group policy administration to a new address. + +**Msg:** `MsgUpdateGroupPolicyAdmin` + +**CLI:** +```bash +simd tx group update-group-policy-admin [admin] [group-policy-account] [new-admin] +``` + +**Authorization:** Must be signed by the group policy admin. + +--- + +### UpdateGroupPolicyDecisionPolicy + +Update the decision policy for a group policy account. + +**Msg:** `MsgUpdateGroupPolicyDecisionPolicy` + +**CLI:** +```bash +simd tx group update-group-policy-decision-policy [admin] [group-policy-account] [decision-policy-json] +``` + +**Authorization:** Must be signed by the group policy admin. + +**Note:** Updating the decision policy aborts any in-flight proposals for that policy. + +--- + +### UpdateGroupPolicyMetadata + +Update a group policy's metadata. + +**Msg:** `MsgUpdateGroupPolicyMetadata` + +**CLI:** +```bash +simd tx group update-group-policy-metadata [admin] [group-policy-account] [metadata] +``` + +**Authorization:** Must be signed by the group policy admin. + +--- + +### SubmitProposal + +Submit a proposal to a group policy account. + +**Msg:** `MsgSubmitProposal` + +**CLI:** +```bash +simd tx group submit-proposal [proposal-json-file] \ + --from proposer \ + --keyring-backend test +``` + +**Proposal JSON format:** +```json +{ + "group_policy_address": "cosmos1...", + "proposers": ["cosmos1..."], + "metadata": "proposal description", + "title": "My Proposal", + "summary": "A brief description of the proposal", + "messages": [ + { + "@type": "/cosmos.bank.v1beta1.MsgSend", + "from_address": "cosmos1...", + "to_address": "cosmos1...", + "amount": [{"denom": "uatom", "amount": "1000"}] + } + ], + "exec": 0 +} +``` + +Set `"exec": 1` (`EXEC_TRY`) to attempt immediate execution. When using `EXEC_TRY`, proposers are automatically counted as yes votes. + +**Authorization:** Must be signed by at least one group member. + +**Failure conditions:** +- Metadata, title, or summary length exceeds `MaxMetadataLen` +- Proposer is not a group member + +--- + +### WithdrawProposal + +Withdraw a pending proposal. + +**Msg:** `MsgWithdrawProposal` + +**CLI:** +```bash +simd tx group withdraw-proposal [proposal-id] [group-policy-admin-or-proposer] +``` + +**Authorization:** Must be signed by a proposer or the group policy admin. + +**Failure conditions:** +- Signer is neither a proposer nor the group policy admin +- Proposal is already closed or aborted + +--- + +### Vote + +Cast a vote on an open proposal. + +**Msg:** `MsgVote` + +**CLI:** +```bash +simd tx group vote [proposal-id] [voter] [vote-option] [metadata] +``` + +**Vote options:** +- `VOTE_OPTION_YES` +- `VOTE_OPTION_NO` +- `VOTE_OPTION_ABSTAIN` +- `VOTE_OPTION_NO_WITH_VETO` + +Set `--exec 1` to attempt immediate execution after voting. + +**Authorization:** Must be signed by a group member. + +**Failure conditions:** +- Metadata length exceeds `MaxMetadataLen` +- Proposal is no longer in the voting period + +--- + +### Exec + +Execute an accepted proposal. + +**Msg:** `MsgExec` + +**CLI:** +```bash +simd tx group exec [proposal-id] \ + --from executor \ + --keyring-backend test +``` + +**Authorization:** Any address can execute an accepted proposal. + +**Notes:** +- Proposal must be in `ACCEPTED` status +- Execution must occur before `MaxExecutionPeriod` after the voting period ends +- A failed execution (`PROPOSAL_EXECUTOR_RESULT_FAILURE`) can be retried until expiry + +--- + +### LeaveGroup + +Remove yourself from a group. + +**Msg:** `MsgLeaveGroup` + +**CLI:** +```bash +simd tx group leave-group [member-address] [group-id] +``` + +**Authorization:** Must be signed by the member leaving. + +**Failure conditions:** +- Signer is not a group member +- Any associated group policy's `Validate()` method fails against the updated member set + +--- + +## Events + +The Group module emits the following events: + +| Event Type | Key | Value | +|------------|-----|-------| +| `cosmos.group.v1.EventCreateGroup` | `group_id` | `{groupId}` | +| `cosmos.group.v1.EventUpdateGroup` | `group_id` | `{groupId}` | +| `cosmos.group.v1.EventCreateGroupPolicy` | `address` | `{groupPolicyAddress}` | +| `cosmos.group.v1.EventUpdateGroupPolicy` | `address` | `{groupPolicyAddress}` | +| `cosmos.group.v1.EventCreateProposal` | `proposal_id` | `{proposalId}` | +| `cosmos.group.v1.EventWithdrawProposal` | `proposal_id` | `{proposalId}` | +| `cosmos.group.v1.EventVote` | `proposal_id` | `{proposalId}` | +| `cosmos.group.v1.EventExec` | `proposal_id`, `logs` | `{proposalId}`, `{logs}` | +| `cosmos.group.v1.EventLeaveGroup` | `proposal_id`, `address` | `{proposalId}`, `{address}` | +| `cosmos.group.v1.EventProposalPruned` | `proposal_id`, `status`, `tally_result` | pruning details | + +--- + +## REST API Endpoints + +| Method | Endpoint | Description | +|--------|----------|-------------| +| GET | `/cosmos/group/v1/groups/{group_id}` | Get group info | +| GET | `/cosmos/group/v1/groups/by_admin/{admin}` | List groups by admin | +| GET | `/cosmos/group/v1/groups` | List all groups | +| GET | `/cosmos/group/v1/groups/{group_id}/members` | List group members | +| GET | `/cosmos/group/v1/group_policies/{address}` | Get group policy info | +| GET | `/cosmos/group/v1/groups/{group_id}/group_policies` | List policies for a group | +| GET | `/cosmos/group/v1/group_policies/by_admin/{admin}` | List policies by admin | +| GET | `/cosmos/group/v1/proposals/{proposal_id}` | Get proposal | +| GET | `/cosmos/group/v1/proposals/by_group_policy/{address}` | List proposals for a policy | +| GET | `/cosmos/group/v1/proposals/{proposal_id}/tally` | Get tally result | +| GET | `/cosmos/group/v1/votes/{proposal_id}/{voter}` | Get a specific vote | +| GET | `/cosmos/group/v1/votes/by_proposal/{proposal_id}` | List votes for a proposal | + +--- + +## Common Use Cases + +### 1. Create a 2-of-3 Multisig Group + +```bash +# Create the group with 3 members of equal weight +simd tx group create-group cosmos1admin "" members.json --from admin + +# members.json +{ + "members": [ + {"address": "cosmos1alice...", "weight": "1"}, + {"address": "cosmos1bob...", "weight": "1"}, + {"address": "cosmos1carol...", "weight": "1"} + ] +} + +# Create a policy requiring 2 of 3 yes votes +simd tx group create-group-policy cosmos1admin 1 "" policy.json --from admin + +# policy.json (threshold = 2) +{ + "@type": "/cosmos.group.v1.ThresholdDecisionPolicy", + "threshold": "2", + "windows": {"voting_period": "72h", "min_execution_period": "0s"} +} +``` + +### 2. Submit and Execute a Proposal + +```bash +# Alice submits a proposal +simd tx group submit-proposal proposal.json --from alice + +# Bob and Carol vote yes +simd tx group vote 1 cosmos1bob YES "" --from bob +simd tx group vote 1 cosmos1carol YES "" --from carol + +# Anyone executes the accepted proposal +simd tx group exec 1 --from alice +``` + +### 3. Self-Governing Group (Policy as Admin) + +```bash +# Create group with policy as its own admin +simd tx group create-group-with-policy cosmos1admin "" "" members.json policy.json \ + --group-policy-as-admin \ + --from admin +``` + +### 4. Multiple Policies for Different Actions + +```bash +# Low-threshold policy for routine actions (1-of-3) +simd tx group create-group-policy cosmos1admin 1 "routine" low_policy.json --from admin + +# High-threshold policy for critical actions (3-of-3) +simd tx group create-group-policy cosmos1admin 1 "critical" high_policy.json --from admin +``` + +--- \ No newline at end of file diff --git a/sdk/v0.54/enterprise/group/architecture.mdx b/sdk/v0.54/enterprise/group/architecture.mdx new file mode 100644 index 000000000..215a93a5d --- /dev/null +++ b/sdk/v0.54/enterprise/group/architecture.mdx @@ -0,0 +1,133 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/enterprise/group/architecture' +title: "Architecture" +description: "System architecture, core concepts, and module integration details for the Group module" +--- + +# Group Module Architecture + +## Overview + +The Group module enables collective decision-making through a proposal-and-vote system. Groups are collections of accounts with associated voting weights. Each group can have one or more policy accounts, each with its own decision policy governing how proposals are accepted or rejected. You can think of it like a dynamic multi-signature account. + +## Architecture Diagram + +![Group Module Architecture](/sdk/v0.54/enterprise/group/architecture.png) + +*The diagram above shows the Group module's actor model, data structures, and proposal lifecycle — from submission through voting to execution.* + +## Core Concepts + +### Group + +A group is an aggregation of accounts with associated voting weights. It is not itself an account and does not hold a balance. A group has an **administrator** who can add, remove, and update members. + +Key points: +- The administrator does not need to be a member of the group +- A group policy account can itself be the administrator of a group, enabling self-governed groups +- Members have weights that determine their relative voting power within proposals + +### Group Policy + +A group policy is an account associated with a group and a decision policy. Group policies are abstracted from groups so that a single group can have **multiple decision policies** for different types of actions. + +This separation keeps membership consistent across policies while allowing different authorization thresholds for different operations. The recommended pattern is: + +1. Create a **master group policy** for a given group +2. Create additional group policies with different decision policies for specific action types +3. Delegate permissions from the master account to sub-accounts using the `x/authz` module + +### Decision Policy + +A decision policy is the mechanism by which group members vote on proposals and the rules that determine whether a proposal passes based on its tally outcome. + +All decision policies have: +- **Minimum Execution Period**: The minimum time after submission before a proposal can be executed. Can be set to `0` to allow immediate execution. +- **Maximum Voting Window**: The maximum time after submission during which members can vote. + +The chain developer also defines an **app-wide maximum execution period** — the window after a proposal's voting period ends during which execution is permitted. + +#### Threshold Decision Policy + +A threshold decision policy defines a minimum total weight of yes votes required for a proposal to pass. Abstain and veto votes are treated as no votes. + +```json +{ + "@type": "/cosmos.group.v1.ThresholdDecisionPolicy", + "threshold": "2", + "windows": { + "voting_period": "24h", + "min_execution_period": "0s" + } +} +``` + +#### Percentage Decision Policy + +A percentage decision policy defines acceptance as a minimum percentage of total group weight voting yes. This policy is better suited for groups with dynamic membership, since the percentage threshold remains meaningful as member weights change. + +```json +{ + "@type": "/cosmos.group.v1.PercentageDecisionPolicy", + "percentage": "0.5", + "windows": { + "voting_period": "24h", + "min_execution_period": "0s" + } +} +``` + +#### Custom Decision Policies + +Chain developers can implement custom decision policies by implementing the `DecisionPolicy` interface. This enables encoding arbitrary acceptance logic into a group policy. + +### Proposal + +Any group member can submit a proposal to a group policy account. A proposal consists of: +- A list of messages to execute if the proposal is accepted +- Optional metadata, title, and summary +- An optional `Exec` field to attempt immediate execution on submission + +#### Voting + +Members vote with one of four options: +- `VOTE_OPTION_YES` +- `VOTE_OPTION_NO` +- `VOTE_OPTION_ABSTAIN` +- `VOTE_OPTION_NO_WITH_VETO` + +The voting window opens immediately on proposal submission and closes at the time defined by the group policy's decision policy. + +#### Tallying + +Tallying occurs when either: +1. A `Msg/Exec`, `Msg/SubmitProposal` (with `TRY_EXEC`), or `Msg/Vote` (with `TRY_EXEC`) triggers an execution attempt +2. The proposal's voting period end is reached during `EndBlock` + +If the tally passes the decision policy's rules, the proposal is marked `PROPOSAL_STATUS_ACCEPTED`. Otherwise it is marked `PROPOSAL_STATUS_REJECTED`. No further voting is permitted after tallying. + +#### Executing Proposals + +Accepted proposals must be executed before `MaxExecutionPeriod` after the voting period ends. Any account (not just group members) can submit a `Msg/Exec` transaction to execute an accepted proposal. + +When `Exec` is set to `EXEC_TRY` on a submit or vote message, the chain attempts immediate execution. If the proposal doesn't yet pass, it remains open for further votes. + +#### Withdrawn and Aborted Proposals + +- **Withdrawn**: Any proposer or the group policy admin can withdraw a proposal before the voting period ends. Withdrawn proposals cannot be executed. A proposal can be withdrawn using `MsgWithdrawProposal` which has an `address` (can be either a proposer or the group policy admin) and a `proposal_id` (which has to be withdrawn). +- **Aborted**: If the group or group policy is updated during the voting period, the proposal is automatically marked as `PROPOSAL_STATUS_ABORTED` since the rules it was created under no longer apply. + +### Pruning + +Proposals and votes are automatically pruned to prevent unbounded state growth. + +**Votes are pruned:** +- After a successful tally triggered by `Msg/Exec` or a submit/vote with `TRY_EXEC` +- On `EndBlock` immediately after the proposal's voting period ends (including aborted and withdrawn proposals) + +**Proposals are pruned:** +- On `EndBlock` when a withdrawn or aborted proposal's voting period ends +- After a successful proposal execution +- On `EndBlock` after `voting_period_end + max_execution_period` has passed + diff --git a/sdk/v0.54/enterprise/group/architecture.png b/sdk/v0.54/enterprise/group/architecture.png new file mode 100644 index 000000000..2d5373812 Binary files /dev/null and b/sdk/v0.54/enterprise/group/architecture.png differ diff --git a/sdk/v0.54/enterprise/group/overview.mdx b/sdk/v0.54/enterprise/group/overview.mdx new file mode 100644 index 000000000..7b04567a5 --- /dev/null +++ b/sdk/v0.54/enterprise/group/overview.mdx @@ -0,0 +1,34 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/enterprise/group/overview' +title: "Overview" +description: "On-Chain Multisig Accounts and Collective Decision-Making" +--- + +The Group module is a Cosmos SDK module that enables on-chain multisig accounts and collective decision-making through configurable voting policies. Any set of accounts can form a named group, attach one or more decision policies to it, and collectively authorize the execution of arbitrary messages through a proposal-and-vote workflow. + +Unlike chain-wide governance proposals, group proposals are scoped to a specific group policy account — enabling organizations, DAOs, and consortiums to manage their on-chain operations with flexible, programmable authorization rules. + +The Group module is designed for networks that require: + +1. **Multi-Party Authorization:** Groups aggregate accounts with weighted voting power, enabling multiple parties to collectively authorize on-chain actions without relying on a single key. +2. **Flexible Decision Policies:** Each group can have multiple policy accounts with independent threshold or percentage-based rules, allowing different authorization requirements for different types of actions. +3. **Permissioned Execution:** Proposals are only executed when they meet the policy's acceptance criteria, ensuring on-chain actions reflect genuine collective agreement. +4. **DAO and Consortium Support:** Ideal for coordinating on-chain operations across organizations, multisig signers, and governance participants. + +## Source Code + +The source code for the Group module can be found [here](https://github.com/cosmos/cosmos-sdk/tree/main/enterprise/group). + +## Available Documentation + +This section contains detailed documentation for the Group module. + +- **[API Reference](/sdk/v0.54/enterprise/group/api)** - Complete API reference for queries and messages +- **[Architecture](/sdk/v0.54/enterprise/group/architecture)** - System architecture, core concepts, and module integration details + +## Licensing + +The Group module source is published under the [Source Available Evaluation License](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/group/LICENSE), which permits evaluation and testing in non-production environments only. Production or commercial use requires an Enterprise License from Cosmos Labs. + +To use the Group module in production, contact sales@cosmoslabs.io. diff --git a/sdk/v0.54/enterprise/overview.mdx b/sdk/v0.54/enterprise/overview.mdx new file mode 100644 index 000000000..d117e78da --- /dev/null +++ b/sdk/v0.54/enterprise/overview.mdx @@ -0,0 +1,25 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/enterprise/overview' +title: "Overview" +description: "Source-available Cosmos SDK modules for permissioned and enterprise blockchain networks." +--- + +Cosmos Enterprise modules are production-ready modules for permissioned networks, institutional chains, and enterprise deployments that need features beyond a public blockchain architecture. They follow the same patterns as the core modules and integrate alongside them. + +The module source is published in the [`enterprise` directory of the Cosmos SDK repository](https://github.com/cosmos/cosmos-sdk/tree/main/enterprise). + +## Available modules + + + + A Proof of Authority (PoA) module enabling permissioned consensus for networks run by a known set of operators without staking or tokens. + + + On-chain multisig accounts and collective decision-making with configurable, weighted voting policies. + + + +## Licensing + +Cosmos Enterprise modules are published under the Source Available Evaluation License. For production use, please contact [sales@cosmoslabs.io](mailto:sales@cosmoslabs.io). diff --git a/sdk/v0.54/enterprise/poa/api.mdx b/sdk/v0.54/enterprise/poa/api.mdx new file mode 100644 index 000000000..57ecb27b4 --- /dev/null +++ b/sdk/v0.54/enterprise/poa/api.mdx @@ -0,0 +1,756 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/enterprise/poa/api' +title: "API Reference" +description: "Complete API reference for PoA module gRPC queries and transactions" +--- + +# PoA Module API Documentation + +## Overview + +The Proof of Authority (PoA) permissioned consensus module provides a governance mechanism for managing validators in a Cosmos SDK blockchain. Unlike traditional Proof of Stake, PoA allows a designated admin to control validator set membership and voting power distribution. + +**Package:** `cosmos.poa.v1` +**Go Import:** `github.com/cosmos/cosmos-sdk/enterprise/poa/types` + +--- + +## Core Concepts + +- **Admin Control:** A single admin address has exclusive authority to manage validators and module parameters +- **Validator Management:** Create validators, update voting power, and manage the active validator set +- **Fee Distribution:** Validators accumulate fees that can be withdrawn by their operators +- **Dynamic Updates:** Changes to the validator set are applied without stopping the chain + +--- + +## Data Types + +### Validator + +Represents a validator in the PoA system. + +```protobuf +message Validator { + google.protobuf.Any pub_key = 1; + int64 power = 2; + ValidatorMetadata metadata = 3; + repeated cosmos.base.v1beta1.DecCoin allocated_fees = 4; +} +``` + +**Fields:** +- `pub_key` (Any): The validator's consensus public key (typically `/cosmos.crypto.ed25519.PubKey`) +- `power` (int64): Voting power for this validator (use `0` to remove a validator) +- `metadata` (ValidatorMetadata): Additional validator information +- `allocated_fees` (DecCoin[]): Accumulated fees allocated to this validator + +**Example:** +```json +{ + "pub_key": { + "@type": "/cosmos.crypto.ed25519.PubKey", + "key": "YUzyiqZzKN8BmLbl75gdXfbxQ2QtSYpPSwA85bZ3xuE=" + }, + "power": "10000", + "metadata": { + "moniker": "validator-1", + "description": "First validator node", + "operator_address": "cosmos1x0mm8rws8lm46xay3zyyznzr6lvu5um3kht0x7" + }, + "allocated_fees": [] +} +``` + +### ValidatorMetadata + +Metadata information about a validator. + +```protobuf +message ValidatorMetadata { + string moniker = 3; + string description = 4; + string operator_address = 5; +} +``` + +**Fields:** +- `moniker` (string): Human-readable name for the validator +- `description` (string): Optional description of the validator +- `operator_address` (string): Cosmos SDK address that operates this validator + +### Params + +Module parameters. + +```protobuf +message Params { + string admin = 1; +} +``` + +**Fields:** +- `admin` (string): Cosmos SDK address with administrative privileges + +### ValidatorFees + +Represents fee allocations for a validator operator. + +```protobuf +message ValidatorFees { + repeated cosmos.base.v1beta1.DecCoin fees = 1; +} +``` + +**Fields:** +- `fees` (DecCoin[]): List of coins representing allocated fees + +--- + +## Query API + +The Query service provides read-only access to PoA module state. + +### Params + +Get module parameters. + +**gRPC:** `cosmos.poa.v1.Query/Params` +**REST:** `GET /cosmos/poa/v1/params` + +**Request:** +```protobuf +message QueryParamsRequest {} +``` + +**Response:** +```protobuf +message QueryParamsResponse { + Params params = 1; +} +``` + +**CLI:** +```bash +simd q poa params +``` + +**Example Response:** +```json +{ + "params": { + "admin": "cosmos1x0mm8rws8lm46xay3zyyznzr6lvu5um3kht0x7" + } +} +``` + +--- + +### Validator + +Query a single validator by address. + +**gRPC:** `cosmos.poa.v1.Query/Validator` +**REST:** `GET /cosmos/poa/v1/validator/{address}` + +**Request:** +```protobuf +message QueryValidatorRequest { + string address = 1; // Consensus or operator address +} +``` + +**Response:** +```protobuf +message QueryValidatorResponse { + Validator validator = 1; +} +``` + +**CLI:** +```bash +simd q poa validator
+``` + +**Notes:** +- `address` can be either a consensus address or operator address + +--- + +### Validators + +List all validators in the system. + +**gRPC:** `cosmos.poa.v1.Query/Validators` +**REST:** `GET /cosmos/poa/v1/validators` + +**Request:** +```protobuf +message QueryValidatorsRequest { + cosmos.base.query.v1beta1.PageRequest pagination = 2; +} +``` + +**Response:** +```protobuf +message QueryValidatorsResponse { + repeated Validator validators = 1; + cosmos.base.query.v1beta1.PageResponse pagination = 2; +} +``` + +**CLI:** +```bash +simd q poa validators +``` + +**Notes:** +- Results are always returned in descending order by voting power +- Supports pagination for large validator sets + +**Example Response:** +```json +{ + "validators": [ + { + "pub_key": { + "@type": "/cosmos.crypto.ed25519.PubKey", + "key": "YUzyiqZzKN8BmLbl75gdXfbxQ2QtSYpPSwA85bZ3xuE=" + }, + "power": "10000", + "metadata": { + "moniker": "validator-1", + "operator_address": "cosmos1..." + } + } + ] +} +``` + +--- + +### WithdrawableFees + +Query fees available for withdrawal by a validator operator. + +**gRPC:** `cosmos.poa.v1.Query/WithdrawableFees` +**REST:** `GET /cosmos/poa/v1/allocated_fees/{operator_address}` + +**Request:** +```protobuf +message QueryWithdrawableFeesRequest { + string operator_address = 1; +} +``` + +**Response:** +```protobuf +message QueryWithdrawableFeesResponse { + ValidatorFees fees = 1; +} +``` + +**CLI:** +```bash +simd q poa allocated-fees +``` + +**Example Response:** +```json +{ + "fees": { + "fees": [ + { + "denom": "token", + "amount": "1000.500000000000000000" + } + ] + } +} +``` + +--- + +### TotalPower + +Get the total voting power across all validators. + +**gRPC:** `cosmos.poa.v1.Query/TotalPower` +**REST:** `GET /cosmos/poa/v1/total_power` + +**Request:** +```protobuf +message QueryTotalPowerRequest {} +``` + +**Response:** +```protobuf +message QueryTotalPowerResponse { + int64 total_power = 1; +} +``` + +**CLI:** +```bash +simd q poa total-power +``` + +**Example Response:** +```json +{ + "total_power": "50000" +} +``` + +--- + +## Transaction Messages (Msg Service) + +The Msg service handles state-changing operations. + +### UpdateParams + +Update module parameters (admin only). + +**gRPC:** `cosmos.poa.v1.Msg/UpdateParams` + +**Message:** +```protobuf +message MsgUpdateParams { + Params params = 1; + string admin = 2; // Signer must be current admin +} +``` + +**Response:** +```protobuf +message MsgUpdateParamsResponse {} +``` + +**CLI:** +```bash +simd tx poa update-params \ + --admin \ + --from \ + --keyring-backend test \ + --chain-id \ + -y +``` + +**Authorization:** Only the current admin can execute this transaction. + +--- + +### CreateValidator + +Create a new validator with zero voting power (operator initiates, admin must activate). + +**gRPC:** `cosmos.poa.v1.Msg/CreateValidator` + +**Message:** +```protobuf +message MsgCreateValidator { + google.protobuf.Any pub_key = 1; + string moniker = 2; + string description = 3; + string operator_address = 4; // Signer +} +``` + +**Response:** +```protobuf +message MsgCreateValidatorResponse {} +``` + +**CLI:** +```bash +simd tx poa create-validator \ + --pubkey \ + --moniker "my-validator" \ + --description "Validator description" \ + --from \ + --keyring-backend test \ + --chain-id \ + -y +``` + +**Authorization:** Any account can create a validator, but it starts with power=0. + +**Notes:** +- The validator will not participate in consensus until the admin updates its power to a non-zero value +- Public key must be a valid consensus public key (typically Ed25519) + +--- + +### UpdateValidators + +Update validator set (admin only). This is the primary mechanism for managing validators. + +**gRPC:** `cosmos.poa.v1.Msg/UpdateValidators` + +**Message:** +```protobuf +message MsgUpdateValidators { + repeated Validator validators = 1; + string admin = 2; // Signer must be admin +} +``` + +**Response:** +```protobuf +message MsgUpdateValidatorsResponse {} +``` + +**CLI (inline):** +```bash +simd tx poa update-validators \ + --validator '{ + "pub_key": { + "@type": "/cosmos.crypto.ed25519.PubKey", + "key": "YUzyiqZzKN8BmLbl75gdXfbxQ2QtSYpPSwA85bZ3xuE=" + }, + "power": 10000 + }' \ + --validator '{ + "pub_key": { + "@type": "/cosmos.crypto.ed25519.PubKey", + "key": "lSR1GEByJtzgiuCevrWgcyBWjhQXjycsuzzIdf56Oa4=" + }, + "power": 0 + }' \ + --from account \ + --keyring-backend test \ + --chain-id \ + -y +``` + +**CLI (from file):** +```bash +simd tx poa update-validators validators.json \ + --from account \ + --keyring-backend test \ + --chain-id \ + -y +``` + +**File Format (validators.json):** +```json +[ + { + "pub_key": { + "@type": "/cosmos.crypto.ed25519.PubKey", + "key": "YUzyiqZzKN8BmLbl75gdXfbxQ2QtSYpPSwA85bZ3xuE=" + }, + "power": 10000, + "metadata": { + "moniker": "validator-1", + "description": "First validator", + "operator_address": "cosmos1x0mm8rws8lm46xay3zyyznzr6lvu5um3kht0x7" + } + }, + { + "pub_key": { + "@type": "/cosmos.crypto.ed25519.PubKey", + "key": "lSR1GEByJtzgiuCevrWgcyBWjhQXjycsuzzIdf56Oa4=" + }, + "power": 0 + } +] +``` + +**Authorization:** Only the admin can execute this transaction. + +**Notes:** +- Can update multiple validators in a single transaction +- Setting `power: 0` removes a validator from the active set +- Changes propagate to CometBFT consensus in the next block +- Missing fields in metadata are preserved from existing state + +--- + +### WithdrawFees + +Withdraw accumulated fees to the operator's account. + +**gRPC:** `cosmos.poa.v1.Msg/WithdrawFees` + +**Message:** +```protobuf +message MsgWithdrawFees { + string operator = 1; // Signer +} +``` + +**Response:** +```protobuf +message MsgWithdrawFeesResponse {} +``` + +**CLI:** +```bash +simd tx poa withdraw-fees \ + --from \ + --keyring-backend test \ + --chain-id \ + -y +``` + +**Authorization:** Must be signed by the validator's operator address. + +**Notes:** +- Transfers all accumulated fees to the operator's account +- Fees are denominated in the chain's native token(s) + +--- + +## Common Use Cases + +### 1. Query Current Admin + +```bash +simd q poa params +``` + +### 2. List All Active Validators + +```bash +simd q poa validators +``` + +### 3. Add a New Validator + +**Step 1:** Operator creates the validator: +```bash +simd tx poa create-validator \ + --pubkey \ + --moniker "new-validator" \ + --from operator-account \ + --keyring-backend test \ + -y +``` + +**Step 2:** Admin activates with voting power: +```bash +simd tx poa update-validators \ + --validator '{ + "pub_key": {"@type": "/cosmos.crypto.ed25519.PubKey", "key": "..."}, + "power": 10000 + }' \ + --from admin \ + --keyring-backend test \ + -y +``` + +### 4. Change Validator Voting Power + +```bash +simd tx poa update-validators \ + --validator '{ + "pub_key": {"@type": "/cosmos.crypto.ed25519.PubKey", "key": "..."}, + "power": 20000 + }' \ + --from admin \ + --keyring-backend test \ + -y +``` + +### 5. Remove a Validator + +```bash +simd tx poa update-validators \ + --validator '{ + "pub_key": {"@type": "/cosmos.crypto.ed25519.PubKey", "key": "..."}, + "power": 0 + }' \ + --from admin \ + --keyring-backend test \ + -y +``` + +### 6. Withdraw Validator Fees + +```bash +simd tx poa withdraw-fees \ + --from validator-operator \ + --keyring-backend test \ + -y +``` + +### 7. Transfer Admin Rights + +```bash +simd tx poa update-params \ + --admin cosmos1newadminaddress... \ + --from current-admin \ + --keyring-backend test \ + -y +``` + +--- + +## REST API Endpoints + +All query endpoints are available via REST: + +| Method | Endpoint | Description | +|--------|----------|-------------| +| GET | `/cosmos/poa/v1/params` | Get module parameters | +| GET | `/cosmos/poa/v1/validator/{address}` | Get single validator | +| GET | `/cosmos/poa/v1/validators` | List all validators | +| GET | `/cosmos/poa/v1/allocated_fees/{operator_address}` | Get withdrawable fees | +| GET | `/cosmos/poa/v1/total_power` | Get total voting power | + +**Example REST Query:** +```bash +curl http://localhost:1317/cosmos/poa/v1/validators +``` + +--- + +## Error Handling + +Common error scenarios: + +### Unauthorized Admin Action +**Error:** Transaction rejected +**Cause:** Non-admin attempted to call admin-only function +**Solution:** Ensure transaction is signed by the admin account + +### Invalid Public Key +**Error:** Invalid validator public key +**Cause:** Malformed or wrong type of public key +**Solution:** Use Ed25519 public key in correct format + +### Validator Not Found +**Error:** Validator does not exist +**Cause:** Querying non-existent validator +**Solution:** Verify validator address/public key + +### Insufficient Fees +**Error:** No fees to withdraw +**Cause:** Validator has no accumulated fees +**Solution:** Wait for fees to accumulate from block rewards + +--- + +## Integration Examples + +### JavaScript/TypeScript (CosmJS) + +```typescript +import { SigningStargateClient } from "@cosmjs/stargate"; + +// Query validators +const client = await StargateClient.connect("http://localhost:26657"); +const response = await client.queryContractSmart( + "cosmos.poa.v1.Query/Validators", + {} +); + +// Update validators (requires signing) +const signingClient = await SigningStargateClient.connectWithSigner( + "http://localhost:26657", + wallet +); + +const msg = { + typeUrl: "/cosmos.poa.v1.MsgUpdateValidators", + value: { + validators: [{ + pubKey: { typeUrl: "/cosmos.crypto.ed25519.PubKey", value: ... }, + power: 10000, + metadata: { moniker: "validator-1", operatorAddress: "cosmos1..." } + }], + admin: "cosmos1adminaddress..." + } +}; + +const result = await signingClient.signAndBroadcast( + adminAddress, + [msg], + "auto" +); +``` + +### Python (cosmpy) + +```python +from cosmpy.aerial.client import LedgerClient, NetworkConfig +from cosmpy.aerial.wallet import LocalWallet + +# Create client +client = LedgerClient(NetworkConfig.fetchai_mainnet()) + +# Query validators +response = client.query_contract( + "cosmos.poa.v1.Query/Validators", + {} +) + +print(response) +``` + +### Go + +```go +import ( + "context" + poatypes "github.com/cosmos/cosmos-sdk/enterprise/poa/types" + "google.golang.org/grpc" +) + +// Query client +conn, _ := grpc.Dial("localhost:9090", grpc.WithInsecure()) +queryClient := poatypes.NewQueryClient(conn) + +// Get validators +resp, err := queryClient.Validators(context.Background(), &poatypes.QueryValidatorsRequest{}) +if err != nil { + panic(err) +} + +for _, val := range resp.Validators { + fmt.Printf("Validator: %s, Power: %d\n", val.Metadata.Moniker, val.Power) +} +``` + +--- + +## Security Considerations + +1. **Admin Key Security:** The admin private key has complete control over the validator set. Use hardware wallets or secure key management systems. + +2. **Validator Public Keys:** Ensure validator public keys are correctly generated and stored securely. + +3. **Power Distribution:** Consider the security implications of power concentration. Avoid giving a single validator >67% of total power. + +4. **Operator Separation:** Use separate accounts for operator and admin roles to limit exposure. + +5. **Fee Withdrawal:** Operators should regularly withdraw fees to prevent accumulation in the module. + +--- + +## Appendix + +### Public Key Formats + +Ed25519 public keys should be base64-encoded: + +```json +{ + "@type": "/cosmos.crypto.ed25519.PubKey", + "key": "YUzyiqZzKN8BmLbl75gdXfbxQ2QtSYpPSwA85bZ3xuE=" +} +``` + +### Address Formats + +- **Operator Address:** Standard Cosmos SDK bech32 address (e.g., `cosmos1...`) +- **Consensus Address:** Can be derived from public key or use operator address for queries + +### Power Units + +- Voting power is represented as `int64` +- Total power affects block signing requirements (typically need >2/3 of total power for consensus) +- Zero power effectively removes a validator from the active set diff --git a/sdk/v0.54/enterprise/poa/architecture.mdx b/sdk/v0.54/enterprise/poa/architecture.mdx new file mode 100644 index 000000000..493c42ed8 --- /dev/null +++ b/sdk/v0.54/enterprise/poa/architecture.mdx @@ -0,0 +1,270 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/enterprise/poa/architecture' +title: "Architecture" +description: "System architecture and module integration details for the PoA module" +--- + +# PoA Module Architecture + +## Overview + +The Proof of Authority (PoA) permissioned consensus module is a Cosmos SDK module that implements a permissioned consensus mechanism where a designated admin controls the validator set. Unlike traditional Proof of Stake systems, PoA validators are explicitly authorized and managed by an administrative authority rather than being selected based on staked tokens. + +## Table of Contents + +- [Architecture](#architecture) + - [SDK Integration Points](#sdk-integration-points) + - [Architectural Decisions](#architectural-decisions) +- [Admin Control Flow](#admin-control-flow) + - [Setting Admin Authority](#setting-admin-authority) + - [Managing Validator Set](#managing-validator-set) + - [Updating Parameters](#updating-parameters) +- [Validator Lifecycle](#validator-lifecycle) + - [Validator Registration](#validator-registration) + - [Gaining Consensus Power](#gaining-consensus-power) + - [Removing Validators](#removing-validators) +- [Fee Distribution](#fee-distribution) → See [the Distribution page](/sdk/v0.54/enterprise/poa/distribution) +- [Governance](#governance) → See [the Governance page](/sdk/v0.54/enterprise/poa/governance) +- [Technical Implementation](#technical-implementation) + - [Storage Design](#storage-design) + - [ABCI Integration](#abci-integration) + - [Dependencies](#dependencies) +- [Security Considerations](#security-considerations) + +## Architecture + +### SDK Integration Points + +The PoA module plugs into the Cosmos SDK as a replacement for the standard staking module, providing an alternative consensus mechanism: + +![PoA Module Architecture](/sdk/v0.54/enterprise/poa/architecture.png) + +*The diagram above shows how the PoA module integrates with Cosmos SDK modules (x/auth, x/bank, x/gov), the fee_collector account, and CometBFT consensus engine.* + +**Key Integration Points:** + +1. **Replaces [x/staking](/sdk/v0.54/modules/staking/README)**: PoA provides validator management without token delegation or bonding +2. **Integrates with [x/gov](/sdk/v0.54/modules/gov/README)**: Custom governance hooks ensure only active validators can participate and tally function override allocates vote weight to validator power ([details](#governance)) +3. **Uses [x/auth](/sdk/v0.54/modules/auth/auth) & [x/bank](/sdk/v0.54/modules/bank/README)**: Standard account and token management for fee distribution ([details](#fee-distribution)) +4. **ABCI Lifecycle**: Implements `EndBlocker` to communicate validator updates to CometBFT ([details](#abci-integration)) + +### Architectural Decisions + +**Admin-Controlled Validator Set** + +Unlike proof-of-stake where validators are determined by token weight, PoA uses a single admin address to authorize validators. This design choice: +- Enables permissioned networks with known validator identities +- Removes token requirement from validator participation (no token bonding required) +- Centralizes trust in the admin address (see [Security Considerations](#security-considerations)) + +**Custom Fee Distribution** + +Rather than using the standard [x/distribution](/sdk/v0.54/modules/distribution/README) module, PoA implements its own fee mechanism: +- Fees are routed to the PoA module account via a custom ante handler (see [Fee Routing Setup](/sdk/v0.54/enterprise/poa/distribution#fee-routing-setup) for complete details). +- Fees allocated proportionally to validator power (not delegated stake) +- Validators withdraw fees on-demand +- See [Fee Distribution](#fee-distribution) for complete details + +**Governance Without Staking** + +Standard SDK [governance](/sdk/v0.54/modules/gov/README) uses bonded tokens for voting weight. PoA replaces this with validator power: +- Only active validators (power > 0) can submit, deposit, or vote on proposals +- Voting weight determined by validator power, not token holdings +- Prevents non-validator governance participation +- See [Governance](#governance) for implementation details + +**Storage Design Philosophy** + +The module uses `cosmossdk.io/collections` with a composite key structure: +- Primary key: `(power, consensus_address)` enables efficient power-sorted iteration +- Secondary indexes on consensus and operator addresses for fast lookups +- Requires re-keying when power changes, but eliminates need for separate sorting +- See [Storage Design](#storage-design) for technical details + +## Admin Control Flow + +### Setting Admin Authority + +The PoA module is controlled by a single admin address configured at genesis. This admin has exclusive authority to: +- Update validator power (grant/revoke consensus participation) +- Modify module parameters +- Batch update the entire validator set + +The admin could be set to any authority that has an address. This includes a group from x/groups, the governance module account, and multisigs. + +**Location**: Admin address stored in [`x/poa/types/keys.go:10`](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/x/poa/types/keys.go#L10) (params prefix) + +Only the admin can update itself with a parameter change. + +### Managing Validator Set + +**MsgUpdateValidators** ([`x/poa/keeper/msg_server.go:72`](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/x/poa/keeper/msg_server.go#L72)) + +The admin can batch update validators through a single transaction: + +1. **Authentication**: Transaction must be signed by the admin address +2. **Validation**: Each validator update is validated for: + - Valid public key + - Non-negative power + - Valid metadata (operator address, moniker, description) + - No duplicate operator addresses +3. **Power Changes**: Any power change triggers: + - Fee checkpoint (allocates pending fees before power changes) + - Total power recalculation + - ABCI validator update queue +4. **Consensus Update**: Changes take effect at the end of the current block + +### Updating Parameters + +**MsgUpdateParams** ([`x/poa/keeper/msg_server.go:26`](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/x/poa/keeper/msg_server.go#L26)) + +The admin can update module parameters (currently only the admin address itself). This requires: +- Transaction signed by current admin +- Validation of new parameters + +## Validator Lifecycle + +### Validator Registration + +**MsgCreateValidator** ([`x/poa/keeper/msg_server.go:45`](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/x/poa/keeper/msg_server.go#L45)) + +**Permissionless Creation**: Any address can register as a validator candidate: + +1. **Submit Registration**: Provide public key and metadata + - **PubKey**: Ed25519 + - **Operator Address**: Account that will receive fees and manage the validator + - **Moniker**: Human-readable name (max 256 chars) + - **Description**: Additional details (max 256 chars) + +2. **Initial State**: Created validators have **power = 0** until the admin updates it via `MsgUpdateValidators` + - Not participating in consensus + - Not earning fees + - Cannot vote in governance + +**Location**: [`x/poa/keeper/validator.go:95`](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/x/poa/keeper/validator.go#L95) + +### Gaining Consensus Power + +Validators can only gain consensus power through admin action: + +1. **Admin Updates Power**: Via [`MsgUpdateValidators`](#managing-validator-set) +2. **Power > 0**: Validator becomes active +3. **ABCI Update**: CometBFT adds validator to active set at next block +4. **Fee Eligibility**: Validator starts accumulating fees proportionally +5. **Governance Rights**: Validator can submit proposals, deposit, and vote + +**Power Mechanics**: +- Power is an integer representing voting weight +- Higher power = more consensus influence and fee share +- Power can be adjusted up or down by admin +- Setting power = 0 removes validator from consensus without deleting + +**Location**: [`x/poa/keeper/validator.go:19`](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/x/poa/keeper/validator.go#L19) + +### Removing Validators + +**Soft Removal** (Removing power): + - Admin sets validator power to 0 + - Validator remains registered but inactive + - Can be reactivated by admin later + - Validator entry is preserved in the map of validators + +## Fee Distribution + +The PoA module implements a custom checkpoint-based fee distribution system that allocates block fees proportionally to validator power. + +**Key Features**: +- Fees accumulate in [the PoA module account](/sdk/v0.54/enterprise/poa/distribution#fee-routing-setup) +- Allocated proportionally to validator power at checkpoints +- Checkpoints triggered by power changes or withdrawals +- Validators withdraw accumulated fees on-demand +- Uses DecCoins for precision to prevent dust accumulation + +**Why Checkpointing?**: Allows for efficient, lazy distribution rather than actively moving funds every block. + +**See [Fee Distribution Documentation](/sdk/v0.54/enterprise/poa/distribution)** for complete details. + +**Location**: [`x/poa/keeper/distribution.go`](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/x/poa/keeper/distribution.go) + +## Governance + +The PoA module restricts governance participation to active validators only, using validator power as voting weight instead of bonded tokens. + +**Key Features**: +- Uses existing x/gov module +- Only active validators (power > 0) can submit, deposit, or vote on proposals +- Voting weight equals validator power +- Custom tally function replaces standard governance tallying +- Admin indirectly controls governance through power distribution + +**Power-Based Voting**: Each validator's vote is weighted by their validator power set in the x/poa module. + +**See [Governance Documentation](/sdk/v0.54/enterprise/poa/governance)** for complete details. + +**Location**: [`x/poa/keeper/governance.go`](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/x/poa/keeper/governance.go) and [`x/poa/keeper/hooks.go`](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/x/poa/keeper/hooks.go) + +## Technical Implementation + +### Storage Design + +**Collections Schema** ([`x/poa/types/keys.go`](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/x/poa/types/keys.go)) + +The module uses `cosmossdk.io/collections` for type-safe state management: + +| Prefix | Collection | Key Type | Value Type | Purpose | +|--------|------------|----------|------------|---------| +| 0 | `params` | - | `Params` | Admin address and module config | +| 1 | `validators` | `(int64, string)` | `Validator` | Primary map, sorted by power | +| 2 | `validator_by_consensus` | `string` | `(int64, string)` | Index: consensus addr → composite key | +| 3 | `validator_by_operator` | `string` | `(int64, string)` | Index: operator addr → composite key | +| 4 | `total_power` | - | `int64` | Sum of all validator power | +| 5 | `total_allocated` | - | `ValidatorFees` | Sum of allocated fees | + + +**Location**: [`x/poa/keeper/keeper.go:16`](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/x/poa/keeper/keeper.go#L16) + +### ABCI Integration + +**EndBlocker** ([`x/poa/keeper/abci.go:9`](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/x/poa/keeper/abci.go#L9)) + +The module integrates with CometBFT consensus through ABCI: + +1. **Power Changes**: When validator power changes, create `ValidatorUpdate` +2. **Queue Updates**: Store updates in memory queue +3. **EndBlock**: At end of block, return all queued updates +4. **CometBFT Processing**: Consensus engine applies updates for next block +5. **Clear Queue**: After returning, clear the queue + +**ValidatorUpdate Format**: +``` +ValidatorUpdate { + PubKey: PublicKey // Consensus public key + Power: int64 // New power (0 = remove) +} +``` + +**Location**: [`x/poa/module.go:128`](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/x/poa/module.go#L128) + +## Security Considerations + +1. **Single Point of Control**: + - Admin address controls entire validator set + +2. **Validator Registration**: + - Anyone can register as validator candidate + - Only admin can grant consensus power + +3. **Total Power Invariant**: + - Total power must remain > 0 + - Prevents zero-power chain halts + - Validated on every power adjustment via a checkpoint trigger + +4. **Governance Restrictions**: + - Only active validators (power > 0) can participate + - Prevents governance spam from unauthorized users + - Ensures governance represents actual consensus participants + +5. **Validator Indexing**: + - Unique consensus address prevents duplicate validators + - Unique operator address prevents fee confusion diff --git a/sdk/v0.54/enterprise/poa/architecture.png b/sdk/v0.54/enterprise/poa/architecture.png new file mode 100644 index 000000000..8338a57aa Binary files /dev/null and b/sdk/v0.54/enterprise/poa/architecture.png differ diff --git a/sdk/v0.54/enterprise/poa/distribution.mdx b/sdk/v0.54/enterprise/poa/distribution.mdx new file mode 100644 index 000000000..2c9d2cd1f --- /dev/null +++ b/sdk/v0.54/enterprise/poa/distribution.mdx @@ -0,0 +1,225 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/enterprise/poa/distribution' +title: "Fee Distribution" +description: "Fee distribution mechanics and algorithms in the PoA module" +--- + +# Fee Distribution + +## Overview + +The PoA module implements a custom fee distribution mechanism based on validator power. Unlike the standard Cosmos SDK x/distribution module, PoA uses a checkpoint-based system to allocate fees proportionally to validators without automatic distribution. + +## How Fees Accumulate + +Fees flow through the PoA system differently than standard Cosmos SDK: + +1. **Block Fees**: Transaction fees collected in each block go to the `fee_collector` module account by default, or to the PoA module account if configured (see [Fee Routing Setup](#fee-routing-setup)) + +2. **Checkpoint System**: Allocated fees are updated for validators when: + - Any validator power changes + - Any validator withdraws fees + +**Why Checkpointing?**: Ensures fair distribution when power changes. If power changes mid-period, fees are allocated based on old power distribution before the change takes effect. + +**Location**: [`x/poa/keeper/distribution.go:18`](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/x/poa/keeper/distribution.go#L18) + +## Distribution Algorithm + +### Checkpoint-Based Allocation + +The PoA module uses a checkpoint system to allocate fees fairly when validator power changes. Rather than distributing fees actively at every block, allocation efficiently happens at discrete checkpoints. + +**Checkpoint Triggers**: +- Any validator power change (via `MsgUpdateValidators`) +- Any fee withdrawal (via `MsgWithdrawFees`) + +**Unallocated Fees Calculation**: + +At checkpoint time $t$, calculate unallocated fees: + +$$ +U_t = B_{collector}(t) - A_{total}(t) +$$ + +Where: +- $U_t$ = unallocated fees at checkpoint $t$ +- $B_{collector}(t)$ = current balance in the PoA module account +- $A_{total}(t) = \sum_{i=1}^{n} F_i(t)$ = sum of all previously- allocated fees across all validators (0 if no checkpoints have been done) + +**Proportional Share Allocation**: + +For each active validator $i$ (where $P_i(t) > 0$), allocate a share proportional to their power: + +$$ +S_i(t) = U_t \times \frac{P_i(t)}{P_{total}(t)} +$$ + +Where: +- $S_i(t)$ = share allocated to validator $i$ at checkpoint $t$ +- $P_i(t)$ = voting power of validator $i$ at checkpoint $t$ +- $P_{total}(t) = \sum_{j=1}^{n} P_j(t)$ = sum of all validator powers + +**Accumulated Fees Update**: + +After allocation, update each validator's accumulated fees: + +$$ +F_i(t+1) = F_i(t) + S_i(t) +$$ + +Where: +- $F_i(t)$ = validator $i$'s accumulated fees before checkpoint +- $F_i(t+1)$ = validator $i$'s accumulated fees after checkpoint +- $S_i(t)$ = share allocated in this checkpoint + +**Total Allocated Tracking**: + +Update the global allocated tracker: + +$$ +A_{total}(t+1) = A_{total}(t) + U_t +$$ + +After this checkpoint, $A_{total}(t+1) = B_{collector}(t)$ (all fees are now allocated). + +### Example Checkpoint Sequence + +**Initial State** (before checkpoint): +- PoA module account balance: $B_{collector} = 1000$ tokens +- Total allocated: $A_{total} = 400$ tokens (from previous checkpoints) +- Validator A: $P_A = 50$, $F_A = 200$ tokens allocated +- Validator B: $P_B = 50$, $F_B = 200$ tokens allocated +- Total power: $P_{total} = 100$ + +**Admin Action**: Admin submits `MsgUpdateValidators` to change power distribution to 30/70 + +**Checkpoint Triggered** (before power change takes effect): + +1. Calculate unallocated: $U = 1000 - 400 = 600$ tokens + +2. Allocate shares based on **current power** (50/50): + - Validator A: $S_A = 600 \times \frac{50}{100} = 300$ tokens + - Validator B: $S_B = 600 \times \frac{50}{100} = 300$ tokens + +3. Update accumulated fees: + - Validator A: $F_A = 200 + 300 = 500$ tokens + - Validator B: $F_B = 200 + 300 = 500$ tokens + +4. Update total allocated: $A_{total} = 400 + 600 = 1000$ tokens + +**After Checkpoint** - Power Change Applied: +- Validator A: $P_A = 30$ (new power for future blocks) +- Validator B: $P_B = 70$ (new power for future blocks) +- All 1000 tokens now allocated ($A_{total} = B_{collector}$) +- Each validator has updated $F_i$ available for withdrawal + +**Why This Matters**: Validator A earned 300 tokens (50% share) based on their power during the period when those fees were collected. After the checkpoint, their power drops to 30%, so future fees will be split 30/70. Checkpointing ensures validators are rewarded based on the work they actually performed. + +**Precision**: Uses `DecCoins` (decimal coins) to prevent rounding dust accumulation. Each validator tracks fractional amounts that are too small to withdraw. + +## Withdrawing Fees + +**MsgWithdrawFees** ([`x/poa/keeper/msg_server.go:91`](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/x/poa/keeper/msg_server.go#L91)) + +Any validator operator can withdraw accumulated fees: + +1. **Submit Withdrawal**: Signed by operator address +2. **Checkpoint**: System checkpoints all validators first (allocates any pending fees) +3. **Truncate**: Decimal coins truncated to whole coins +4. **Transfer**: Coins transferred from the PoA module account to operator address +5. **Update Tracking**: Total allocated decreases by withdrawn amount +6. **Remainder**: Decimal remainder stays in validator's allocated balance + +**Example**: +``` +Validator has: 100.7543 utokens allocated +Withdrawal: 100 utokens transferred to operator +Remainder: 0.7543 utokens remain allocated (less than least significant utoken digit) +``` + +**Location**: [`x/poa/keeper/distribution.go:106`](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/x/poa/keeper/distribution.go#L106) + +## Withdrawal Formula + +When validator $i$ withdraws fees: + +$$ +W_i = \lfloor F_i \rfloor +$$ + +$$ +F_i' = F_i - W_i +$$ + +$$ +A_{total}' = A_{total} - W_i +$$ + +Where: +- $W_i$ = amount withdrawn (truncated to integer coins) +- $F_i$ = validator's allocated fees before withdrawal +- $F_i'$ = validator's allocated fees after withdrawal (decimal remainder) +- $\lfloor F_i \rfloor$ = floor function (truncate decimals) +- $A_{total}'$ = updated total allocated across all validators + +## Fee Routing Setup + +PoA has its own module account for collecting fees. Enabling the PoA module account is recommended to keep fee accounting isolated and accurate. If not enabled, fees are deposited into the standard `fee_collector` account by default. + +To enable the PoA module account, two wiring changes are required: + +### 1. Register the PoA Module Account + +Register `poatypes.ModuleName` in the `maccPerms` map passed to `authkeeper.NewAccountKeeper`: + +```go +app.AccountKeeper = authkeeper.NewAccountKeeper( + appCodec, + runtime.NewKVStoreService(storeKeys[authtypes.StoreKey]), + authtypes.ProtoBaseAccount, + map[string][]string{ + authtypes.FeeCollectorName: nil, + govtypes.ModuleName: {authtypes.Burner, authtypes.Staking}, + poatypes.ModuleName: nil, // register PoA module account + }, + // ... +) +``` + +**Source**: [`simapp/app.go`](https://github.com/cosmos/cosmos-sdk/blob/7bc1b146d437d834d971f415924104188203c96f/enterprise/poa/simapp/app.go#L191) + +### 2. Configure the Ante Handler + +Use `WithFeeRecipientModule` on `NewDeductFeeDecorator` to route fees to the PoA module account: + +```go {9} +anteDecorators := []sdk.AnteDecorator{ + ante.NewSetUpContextDecorator(), + ante.NewExtensionOptionsDecorator(options.ExtensionOptionChecker), + ante.NewValidateBasicDecorator(), + ante.NewTxTimeoutHeightDecorator(), + ante.NewValidateMemoDecorator(options.AccountKeeper), + ante.NewConsumeGasForTxSizeDecorator(options.AccountKeeper), + ante.NewDeductFeeDecorator(options.AccountKeeper, options.BankKeeper, options.FeegrantKeeper, options.TxFeeChecker). + WithFeeRecipientModule(poatypes.ModuleName), // redirect fees to PoA module account + ante.NewSetPubKeyDecorator(options.AccountKeeper), + ante.NewValidateSigCountDecorator(options.AccountKeeper), + ante.NewSigGasConsumeDecorator(options.AccountKeeper, options.SigGasConsumer), + ante.NewSigVerificationDecorator(options.AccountKeeper, options.SignModeHandler, options.SigVerifyOptions...), + ante.NewIncrementSequenceDecorator(options.AccountKeeper), +} +``` + +**Source**: [`simapp/ante.go`](https://github.com/cosmos/cosmos-sdk/blob/7bc1b146d437d834d971f415924104188203c96f/enterprise/poa/simapp/ante.go#L49) + +`WithFeeRecipientModule` is backwards compatible — omitting it defaults to the standard `fee_collector` behavior. + +## Security Considerations + +1. **Decimal Precision**: + - Uses DecCoins to prevent dust accumulation + - Validators track fractional amounts + - Remainders preserved across withdrawals + - Prevents rounding errors from accumulating diff --git a/sdk/v0.54/enterprise/poa/governance.mdx b/sdk/v0.54/enterprise/poa/governance.mdx new file mode 100644 index 000000000..b01e993f2 --- /dev/null +++ b/sdk/v0.54/enterprise/poa/governance.mdx @@ -0,0 +1,289 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/enterprise/poa/governance' +title: "Governance Integration" +description: "Governance integration and power-based voting in the PoA module" +--- + +# Governance + +## Overview + +The PoA module integrates with Cosmos SDK governance to restrict participation to authorized validators only. Unlike standard governance that uses bonded tokens for voting weight, PoA governance uses validator power as the basis for voting. + +## Validator-Only Governance + +The PoA module restricts governance participation to authorized validators only through governance hooks. + +**Governance Hooks** ([`x/poa/keeper/hooks.go`](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/x/poa/keeper/hooks.go)) + +The module implements `govtypes.GovHooks`: + +1. **AfterProposalSubmission**: Only authorized validators can submit proposals +2. **AfterProposalDeposit**: Only authorized validators can deposit on proposals +3. **AfterProposalVote**: Only authorized validators can vote + +**Authorized Validator Definition**: +- Registered in PoA module +- Power > 0 +- Has valid operator address + +**Rejected Actions**: +- If non-validator attempts governance action → transaction fails +- If validator has power = 0 → transaction fails +- Error: "voter X is not an active PoA validator" + +**Location**: [`x/poa/keeper/governance.go:92`](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/x/poa/keeper/governance.go#L92) + +## Voting Power + +**Custom Vote Tallying** ([`x/poa/keeper/governance.go:18`](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/x/poa/keeper/governance.go#L18)). An example of the wiring can be found in the [SimApp](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/simapp/app.go#L197-214). + +Standard governance uses staked tokens as voting weight. PoA governance uses validator power: + +1. **Vote Collection**: System iterates all votes on a proposal +2. **Validator Check**: For each vote, verify voter is active PoA validator +3. **Weight Calculation**: Use validator's power as voting weight +4. **Weighted Options**: Supports split votes, exactly like x/staking in traditional POS governance (e.g., 70% Yes, 30% Abstain) +5. **Tally Results**: Sum weighted votes by option + +### Vote Tallying Algorithm + +**Voting Power Formula**: + +$$ +V_i = P_i +$$ + +Where: +- $V_i$ = voting power of validator $i$ +- $P_i$ = validator power (consensus weight) + +**Weighted Vote Calculation**: + +For a validator casting a split vote across multiple options: + +$$ +W_{i,o} = V_i \times w_{i,o} +$$ + +Where: +- $W_{i,o}$ = vote weight from validator $i$ for option $o$ +- $w_{i,o}$ = weight assigned to option $o$ by validator $i$ (where $\sum_{o} w_{i,o} = 1$) + +**Total Tally per Option**: + +$$ +T_o = \sum_{i \in voters} W_{i,o} +$$ + +Where: +- $T_o$ = total votes for option $o$ +- Sum over all validators who voted + +### Example + +**Validator A**: $P_A = 100$, votes 100% Yes +- $W_{A,Yes} = 100 \times 1.0 = 100$ + +**Validator B**: $P_B = 50$, votes 60% Yes, 40% No +- $W_{B,Yes} = 50 \times 0.6 = 30$ +- $W_{B,No} = 50 \times 0.4 = 20$ + +**Totals**: +- $T_{Yes} = 130$ +- $T_{No} = 20$ + +## Proposal Lifecycle + +### 1. Proposal Submission + +**[MsgSubmitProposal](https://github.com/cosmos/cosmos-sdk/blob/main/proto/cosmos/gov/v1/tx.proto#L54-L65)** (standard x/gov module) + +When a proposal is submitted: + +1. Standard governance validates the proposal content +2. `AfterProposalSubmission` hook is called +3. PoA module checks if proposer is authorized validator: + - Look up proposer by operator address + - Verify validator exists and has $P > 0$ + - If not active, reject with error +4. If valid, proposal enters deposit period + +**Restriction**: Only authorized validators can submit proposals, preventing spam from non-consensus participants. + +### 2. Deposit Period + +**[MsgDeposit](https://github.com/cosmos/cosmos-sdk/blob/main/proto/cosmos/gov/v1/tx.proto#L90-L98)** (standard x/gov module) + +When a deposit is made: + +1. Standard governance processes the deposit +2. `AfterProposalDeposit` hook is called +3. PoA module checks if depositor is authorized validator +4. If deposit threshold reached, proposal moves to voting period + +**Restriction**: Only authorized validators can deposit, ensuring only consensus participants can advance proposals. + +### 3. Voting Period + +**[MsgVote](https://github.com/cosmos/cosmos-sdk/blob/main/proto/cosmos/gov/v1/tx.proto#L100-L108)** or **[MsgVoteWeighted](https://github.com/cosmos/cosmos-sdk/blob/main/proto/cosmos/gov/v1/tx.proto#L110-L118)** (standard x/gov module) + +When a vote is cast: + +1. Standard governance records the vote +2. `AfterProposalVote` hook is called +3. PoA module validates voter is authorized validator +4. If invalid, transaction fails + +**Vote Options**: +- `Yes`: Support the proposal +- `No`: Oppose the proposal +- `NoWithVeto`: Oppose and veto (can burn deposits if threshold met) +- `Abstain`: Participate in quorum without taking a position + +**Weighted Voting**: Validators can split their vote across multiple options, with weights summing to 1. + +### 4. Vote Tallying + +At the end of the voting period, the [custom tally function](#vote-tallying-algorithm) is called: + +**NewPoACalculateVoteResultsAndVotingPowerFn** ([`x/poa/keeper/governance.go:18`](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/x/poa/keeper/governance.go#L18)) + +1. Iterate all votes on the proposal +2. For each vote, look up the validator by voter address +3. If validator is not active ($P \leq 0$), skip the vote +4. Otherwise, use validator power as voting weight +5. For weighted votes, distribute power across options +6. Sum all weighted votes by option +7. Apply standard governance thresholds: + - Quorum: Minimum participation percentage + - Threshold: Minimum "Yes" percentage to pass + - Veto: Maximum "NoWithVeto" percentage before rejection + +**Result**: Proposal passes, fails, or is vetoed based on power-weighted votes. + +## Implementation Details + +### Governance Hooks + +**Location**: [`x/poa/keeper/hooks.go`](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/x/poa/keeper/hooks.go) + +The module implements the `govtypes.GovHooks` interface: + +``` +type GovHooks interface { + AfterProposalSubmission(ctx, proposalID, depositorAddr) + AfterProposalDeposit(ctx, proposalID, depositorAddr) + AfterProposalVote(ctx, proposalID, voterAddr) + // ... other hooks +} +``` + +Each hook implementation: +1. Extracts the operator address from the context +2. Looks up the validator by operator address +3. Checks if validator exists and has power > 0 +4. Returns error if validation fails + +### Custom Tally Function + +**Location**: [`x/poa/keeper/governance.go:18`](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/x/poa/keeper/governance.go#L18) + +The tally function replaces the standard governance tally: + +```go +func NewPoACalculateVoteResultsAndVotingPowerFn(keeper) TallyFn { + return func(ctx, proposal) (totalVotingPower, results) { + // Iterate votes + for vote in votes(proposal) { + validator = keeper.GetValidatorByOperator(vote.voter) + if validator == nil || validator.Power <= 0 { + continue // Skip non-authorized validators + } + + // Add validator power to total + totalVotingPower += validator.Power + + // Apply vote weights + for option, weight in vote.options { + results[option] += validator.Power * weight + } + } + return totalVotingPower, results + } +} +``` + +## Governance Parameters + +The standard governance module parameters still apply: + +- **MinDeposit**: Minimum tokens required to enter voting period +- **MaxDepositPeriod**: Time limit for reaching minimum deposit +- **VotingPeriod**: Duration of the voting period +- **Quorum**: Minimum participation rate (fraction of total power) +- **Threshold**: Minimum "Yes" rate to pass (fraction of non-abstain votes) +- **VetoThreshold**: Maximum "NoWithVeto" rate before rejection + +**Key Difference**: Quorum is calculated as a percentage of total validator power, not total bonded tokens. + +## Security Considerations + +1. **Validator Exclusivity**: + - Only authorized validators (power > 0) can participate + - Prevents sybil attacks through unauthorized validator spam + - Ensures governance represents actual consensus participants + +2. **Power-Based Voting**: + - Voting weight tied to consensus power + - Admin controls power distribution, thus controls governance indirectly + +3. **Admin Governance Control**: + - Admin can change validator power at any time + - Admin can effectively control governance by adjusting power + - Consider multi-sig admin or governance-controlled admin changes + +4. **Proposal Spam Prevention**: + - Restricting submissions to authorized validators reduces spam + - Deposit requirements still apply + - Validators have reputational stake in proposal quality + +## Comparison to Standard Governance + +| Aspect | Standard Cosmos Governance | PoA Governance | +|--------|---------------------------|----------------| +| Who can vote | Token holders (delegators + validators) | Authorized validators only | +| Voting weight | Bonded tokens | Validator power | +| Who can propose | Anyone with min deposit | Authorized validators only | +| Who can deposit | Anyone | Authorized validators only | +| Vote tallying | Sum of bonded tokens | Sum of validator power | +| Quorum calculation | % of bonded tokens | % of total validator power | +| Admin control | No direct control | Admin controls power → controls votes | + +## Example Governance Flow + +**Scenario**: Validator A wants to propose a parameter change + +1. **Submit Proposal**: + - Validator A (power = 40) submits `MsgSubmitProposal` + - Hook verifies A is authorized validator + - Proposal enters deposit period + +2. **Reach Deposit**: + - Validator B (power = 30) deposits + - Validator C (power = 30) deposits + - Deposit threshold reached → voting period starts + +3. **Voting**: + - Validator A: 100% Yes (40 power → 40 Yes votes) + - Validator B: 60% Yes, 40% No (30 power → 18 Yes, 12 No) + - Validator C: 100% Abstain (30 power → 30 Abstain) + - Total power: 100 (all authorized validators) + +4. **Tally**: + - Total voting power: 100 (all voted) + - Quorum: 100/100 = 100% ✓ (assuming 33% quorum) + - Results: 58 Yes, 12 No, 30 Abstain (out of 70 non-abstain) + - Threshold: 58/70 = 82.9% Yes ✓ (assuming 50% threshold) + - **Proposal passes** diff --git a/sdk/v0.54/enterprise/poa/overview.mdx b/sdk/v0.54/enterprise/poa/overview.mdx new file mode 100644 index 000000000..7f7689553 --- /dev/null +++ b/sdk/v0.54/enterprise/poa/overview.mdx @@ -0,0 +1,47 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/enterprise/poa/overview' +title: "Overview" +description: "Enterprise-Ready Network Security and Operations" +--- + +The Proof of Authority (PoA) permissioned consensus module is a Cosmos SDK module that enables permissioned consensus for networks requiring controlled participation. A designated administrative authority manages the validator set directly, ensuring that only approved operators participate in block production and governance. + +Unlike traditional Proof-of-Stake systems, validator membership is not determined by token staking. Validators are explicitly authorized, updated, and removed through on-chain administrative actions, enabling predictable operations and compliance-aligned governance. + +The PoA module is designed for networks that require: + +1. **Permissioned Operators:** A configurable administrative authority defines validators and governance participants to meet organizational security, compliance, or consortium requirements. +2. **Instant Validator Updates:** Add, remove, or replace validators, adjust relative validator weights, and rotate keys in a single atomic on-chain action. +3. **Token-Free Operation:** Launch, operate, and govern a network without issuing or managing a native token. +4. **Future-Proof Architecture:** Seamlessly transition to Proof-of-Stake and introduce a token when needed. + +## The best available option for Proof of Authority + +| Characteristic | Alternatives | Cosmos PoA Module | +|----------------|-------------|-------------------| +| Compatibility with Cosmos SDK v0.53+ | ✗ | ✓ | +| Support for token-free operation | ✗ | ✓ | +| Flexible governance authority | ✗ | ✓ | +| Programmable penalties (jailing, slashing) | ✗ | ✓ | +| Included in Cosmos bug bounty program | ✗ | ✓ | +| Ongoing development by Cosmos core developers | ✗ | ✓ | + +## Source Code + +The source code for the Proof of Authority module can be found [here](https://github.com/cosmos/cosmos-sdk/tree/main/enterprise/poa). + +## Available Documentation + +This directory contains detailed documentation for the Proof of Authority module. + +- **[API Reference](/sdk/v0.54/enterprise/poa/api)** - Complete API reference for gRPC queries and transactions +- **[Architecture](/sdk/v0.54/enterprise/poa/architecture)** - System architecture and module integration details +- **[Distribution](/sdk/v0.54/enterprise/poa/distribution)** - Fee distribution mechanics and algorithms +- **[Governance](/sdk/v0.54/enterprise/poa/governance)** - Governance integration and power-based voting + +## Licensing + +The Proof of Authority module source is published under the [Source Available Evaluation License](https://github.com/cosmos/cosmos-sdk/blob/main/enterprise/poa/LICENSE), which permits evaluation and testing in non-production environments only. Production or commercial use requires an Enterprise License from Cosmos Labs. + +To use the Proof of Authority module in production, contact sales@cosmoslabs.io. \ No newline at end of file diff --git a/sdk/v0.54/experimental/blockstm.mdx b/sdk/v0.54/experimental/blockstm.mdx new file mode 100644 index 000000000..3c2edc5f5 --- /dev/null +++ b/sdk/v0.54/experimental/blockstm.mdx @@ -0,0 +1,267 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/experimental/blockstm' +title: "Block-STM: Parallel Transaction Execution" +--- + + +**Synopsis** +Block-STM enables parallel execution of transactions during `FinalizeBlock`, using optimistic concurrency control to improve block processing throughput. + + + +**Prerequisite Readings** + +* [BaseApp](/sdk/v0.54/learn/concepts/baseapp) +* [Transactions](/sdk/v0.54/learn/concepts/transactions) +* [Store](/sdk/v0.54/learn/concepts/store) + + + +## Background + +Block-STM is an algorithm originally published in the [Block-STM paper](https://arxiv.org/pdf/2203.06871) and implemented for the Aptos blockchain. The algorithm was then written for Cosmos SDK compatible chains in Go by developers for the Cronos blockchain in [go-block-stm](https://github.com/crypto-org-chain/go-block-stm). + +This library was forked and directly integrated into the Cosmos SDK with accompanying changes to the `baseapp` and `store` packages. Subsequent changes and improvements have been made on top of the original implementation to further optimize performance in both memory and time. + +## Algorithm Overview + +Block-STM implements a form of optimistic concurrency control to enable parallel execution of transactions. It does this by implementing read and write set tracking on top of the SDK's IAVL storage layer. This, combined with the absolute ordering of transactions provided by the block proposal, is used in a validation phase which determines if any two executed transactions have conflicting storage access. In the case of conflicting storage access, the algorithm provides a means for re-execution and re-validation of the conflicting transactions based on the ordering in the proposal. + +Block-STM is currently only integrated into the `FinalizeBlock` phase of execution, meaning the code path is never accessed until the block is agreed upon in consensus. It is possible that the algorithm may be extended in the future to support different execution models, but as of right now it expects a complete block and returns its result after the entire block has been executed. For this reason, Block-STM is expected to produce identical results to serial execution. In other words, the `AppHash` produced by Block-STM's parallel execution should be equal to the `AppHash` produced by the default serial transaction runner. + +## Safe Deployment Practices + +Given the Block-STM executor is a general purpose parallel execution engine, we recommend a phased rollout with +extensive +testing for each application individually. + +* _Phased Rollout_ + +Since parallel execution is purely a performance optimization, applications should expect to calculate the same +AppHash when using Block-STM as they would when serial execution is enabled via the default TxRunner. This allows teams +to turn on parallel execution for a fraction of their nodes--API nodes instead of validators or on a portion of a +distributed validator cluster for example. + +Running with a mixed fleet of parallel and serial execution for an extended time should minimize blast radius in the +event that a failure occurs. + +* _Message Type Support_ + +We have done testing on as many of the core SDK message types as possible, but given the Cosmos SDK allows arbitrary +message creation it will be impossible to validate all message types that exist and all combinations of workflows. +Each team integrating Block-STM in production should validate their own message types for both correctness and performance. + +NOTE: We specifically have __not__ validated support for CosmWasm message types run using Block-STM. Run independent +validation before enabling if your chain uses CosmWasm. + +* _Caching Risks_ + +Block-STM works via dependency tracking within the SDK's `MultiStore` interface. Any data which could cause stateful +changes to execution that lives outside the store poses the biggest risk for indeterminism. We recommend in general +avoiding the use of cached data, in memory stores, or persisting any state outside the scope of a `Store`. + +## App Integration + +Integration of parallel execution is abstracted into two interfaces: `DeliverTxFunc` and `TxRunner`. + +```go +// DeliverTxFunc is the function called for each transaction in order to produce +// a single ExecTxResult. `memTx` is an optional in-memory representation of +// the transaction, which can be used to avoid decoding the transaction. +type DeliverTxFunc func( + tx []byte, + memTx Tx, + ms storetypes.MultiStore, + txIndex int, + incarnationCache map[string]any, +) *abci.ExecTxResult + +// TxRunner defines an interface for types which can be used to execute the +// DeliverTxFunc. It should return an array of *abci.ExecTxResult corresponding +// to the result of executing each transaction provided to the Run function. +type TxRunner interface { + Run( + ctx context.Context, + ms storetypes.MultiStore, + txs [][]byte, + deliverTx DeliverTxFunc, + ) ([]*abci.ExecTxResult, error) +} +``` + +The `TxRunner` is the primary interface that developers wire into their application. The `baseapp` package provides an option to set it up: + +```go +func (app *BaseApp) SetBlockSTMTxRunner(txRunner sdk.TxRunner) { + app.txRunner = txRunner +} +``` + +### Runner Implementations + +Two implementations of `TxRunner` are provided in the `baseapp/txnrunner` package: + +```go +NewDefaultRunner(txDecoder sdk.TxDecoder) *DefaultRunner +NewSTMRunner( + txDecoder sdk.TxDecoder, + stores []storetypes.StoreKey, + workers int, + estimate bool, + coinDenom func(storetypes.MultiStore) string, +) *STMRunner +``` + +`NewDefaultRunner` is used by `BaseApp` by default and provides serial execution without using the Block-STM code paths. You do not need to wire this in explicitly. + +`NewSTMRunner` constructs a runner which uses parallel execution. Its parameters are: + +* **`txDecoder`** — A standard `sdk.TxDecoder`, readily available in any SDK application. + +* **`stores`** — A list of every store key used in your application. Since Block-STM needs to track store usage across transactions, it must be passed all module-level store keys. Here is an example taken from the Cosmos EVM: + +```go +keys := storetypes.NewKVStoreKeys( + authtypes.StoreKey, banktypes.StoreKey, stakingtypes.StoreKey, + minttypes.StoreKey, distrtypes.StoreKey, slashingtypes.StoreKey, + govtypes.StoreKey, consensusparamtypes.StoreKey, + upgradetypes.StoreKey, feegrant.StoreKey, evidencetypes.StoreKey, + authzkeeper.StoreKey, + // IBC keys + ibcexported.StoreKey, ibctransfertypes.StoreKey, + // Cosmos EVM store keys + evmtypes.StoreKey, feemarkettypes.StoreKey, erc20types.StoreKey, +) +oKeys := storetypes.NewObjectStoreKeys( + banktypes.ObjectStoreKey, evmtypes.ObjectKey, +) + +var nonTransientKeys []storetypes.StoreKey +for _, k := range keys { + nonTransientKeys = append(nonTransientKeys, k) +} +for _, k := range oKeys { + nonTransientKeys = append(nonTransientKeys, k) +} +``` + +* **`workers`** — The number of parallel workers. Experimentation has shown diminishing returns above your system's hardware parallelism. The recommended value is: + +```go +import "runtime" + +workers := min(runtime.GOMAXPROCS(0), runtime.NumCPU()) +``` + +* **`estimate`** — Controls whether the system should proactively determine transaction read/write conflicts before execution. Set this to `true` in all cases. + +* **`coinDenom`** — A function that returns the staking coin denom at runtime. This is used during estimation to reason about which keys in the `bank` module will be modified when fees are collected. A hard-coded value is acceptable; the value should be your chain's bond denom. + +### Full Wiring Example + +Here is a complete example taken from the Cosmos EVM's `evmd` application: + +```go +bApp.SetBlockSTMTxRunner(txnrunner.NewSTMRunner( + encodingConfig.TxConfig.TxDecoder(), + nonTransientKeys, + min(goruntime.GOMAXPROCS(0), goruntime.NumCPU()), + true, + func(ms storetypes.MultiStore) string { return sdk.DefaultBondDenom }, +)) +``` + +## Parallel Transaction Optimization + +Once Block-STM is wired in, you may initially notice that most blocks execute slower than with serial execution. This is due to the overhead of re-executing transactions when any two have conflicting reads or writes. To realize performance gains, you need to reduce storage access conflicts between transactions. + +An example of this can be seen in [PR #26005](https://github.com/cosmos/cosmos-sdk/pull/26005) where new account creation involved assigning an ID whose value was retrieved and incremented via a single key in the `x/auth` module. The linked PR converts account ID generation to use deterministic UUID generation instead of relying on a conflicting storage location. The result is that multiple transactions in the same block which each create new accounts no longer access this key and can be run in parallel without re-executions. + +Work has already been done within the SDK and Cosmos EVM for common transaction types such as bank sends and EVM gas sends. The following steps describe the additional configuration needed. + +### Enable Virtual Fee Collection (EVM-specific) + +This alters how fee collection works for EVM transactions, accumulating fees to the fee collector module in the `EndBlocker` instead of using regular sends during transaction execution. + +```go +app.EVMKeeper.EnableVirtualFeeCollection() +``` + +### Set Up the Object Store in the Bank Keeper + +This enables the bank keeper to collect fees in the `EndBlocker` instead of requiring every transaction to send fees directly to the `FeeCollector` module account. + +```go +app.BankKeeper = app.BankKeeper.WithObjStoreKey(oKeys[banktypes.ObjectStoreKey]) +``` + +### Custom Modules + +All other changes to parallelize common transactions were done in a way that does not require configuration. + +For custom transaction types or custom modules, additional changes to KV store access patterns may be required. There is no generalized approach for this yet. The common pattern for functionality that requires access to the same storage key is to write intermediate values to transient or object storage and use an `EndBlocker` to collect the values after all transaction execution completes. + +## Benchmarks + +### Environment + +- **Machine:** Apple M3 Pro, 11 cores +- **OS:** macOS (Darwin 25.3.0) +- **Package:** `github.com/cosmos/cosmos-sdk/internal/blockstm` + +--- + +### Random Workload (10k txs, 100 keys) + +| Workers | ns/op | B/op | allocs/op | Speedup | +|---------|-------|------|-----------|---------| +| sequential | 1,169M | 9.9M | 220K | 1.0x | +| 1 | 1,208M | 36.3M | 544K | 0.97x | +| 5 | 324M | 37.1M | 552K | **3.6x** | +| 10 | 218M | 43.7M | 621K | **5.4x** | +| 15 | 211M | 77.4M | 975K | **5.5x** | +| 20 | 226M | 78.0M | 982K | **5.2x** | + +### No-Conflict Workload (10k txs) + +| Workers | ns/op | B/op | allocs/op | Speedup | +|---------|-------|------|-----------|---------| +| sequential | 1,381M | 11.7M | 221K | 1.0x | +| 1 | 1,358M | 80.3M | 1,095K | 1.0x | +| 5 | 291M | 81.1M | 1,103K | **4.8x** | +| 10 | 209M | 83.5M | 1,131K | **6.6x** | +| 15 | 200M | 83.7M | 1,135K | **6.9x** | +| 20 | 204M | 83.8M | 1,136K | **6.8x** | + +### Worst-Case Workload (full conflict, 10k txs) + +| Workers | ns/op | B/op | allocs/op | Speedup | +|---------|-------|------|-----------|---------| +| sequential | 1,239M | 9.6M | 220K | 1.0x | +| 1 | 1,280M | 34.5M | 520K | 0.97x | +| 5 | 295M | 42.8M | 607K | **4.2x** | +| 10 | 224M | 70.1M | 899K | **5.5x** | +| 15 | 246M | 77.3M | 980K | **5.0x** | +| 20 | 262M | 77.4M | 980K | **4.7x** | + +### Iterate Workload (10k txs, 100 keys) + +| Workers | ns/op | B/op | allocs/op | Speedup | +|---------|-------|------|-----------|---------| +| sequential | 1,286M | 16.5M | 290K | 1.0x | +| 1 | 1,332M | 75.2M | 843K | 0.97x | +| 5 | 288M | 76.5M | 855K | **4.5x** | +| 10 | 252M | 123.1M | 1,280K | **5.1x** | +| 15 | 317M | 359.5M | 3,405K | **4.1x** | +| 20 | 319M | 363.5M | 3,419K | **4.0x** | + +--- + +### Key Takeaways + +- Peak speedup is **6.9x** at 15 workers on the no-conflict workload +- Diminishing returns beyond 10–15 workers, with memory usage increasing significantly +- Even the worst-case (full conflict) scenario achieves ~5x speedup at 10 workers +- The iterate workload shows performance degradation beyond 10 workers, likely due to increased contention on range reads (memory usage jumps ~3x at 15+ workers) + diff --git a/sdk/v0.54/guides/abci/abci.mdx b/sdk/v0.54/guides/abci/abci.mdx new file mode 100644 index 000000000..6e2a04466 --- /dev/null +++ b/sdk/v0.54/guides/abci/abci.mdx @@ -0,0 +1,121 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/guides/abci/abci' +title: ABCI Overview +description: >- + ABCI, Application Blockchain Interface is the interface between CometBFT and + the application. More information about ABCI can be found here. CometBFT + version 0.38 included a new version of ABCI (called ABCI 2.0) which added + several new methods. +--- + +## What is ABCI? + +ABCI, Application Blockchain Interface is the interface between CometBFT and the application. More information about ABCI can be found [here](/cometbft/latest/spec/abci/Overview). CometBFT version 0.38 introduced ABCI 2.0, which added several new methods: + +* `PrepareProposal` +* `ProcessProposal` +* `ExtendVote` +* `VerifyVoteExtension` +* `FinalizeBlock` + +The Cosmos SDK's `BaseApp` implements the full ABCI interface. The source lives in [`baseapp/abci.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/baseapp/abci.go). + +## CheckTx + +```mermaid +graph TD + subgraph SDK[Cosmos SDK] + B[BaseApp] + A[AnteHandlers] + B <-->|Validate TX| A + end + C[CometBFT] <-->|CheckTx|SDK + U((User)) -->|Submit TX| C + N[P2P] -->|Receive TX| C +``` + +`CheckTx` is called by `BaseApp` whenever CometBFT receives a transaction from a client, over the p2p network, or via RPC. Its sole job is to decide whether the transaction is valid enough to enter the mempool. It does not execute messages. + +The default implementation runs the transaction through the `AnteHandler` chain, which performs signature verification, fee checks, and other stateless or lightweight stateful validation. If the `AnteHandler` returns an error, the transaction is rejected and never reaches the mempool. + +See the implementation at [`baseapp/abci.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/baseapp/abci.go). + +### Custom CheckTx handler + +`CheckTxHandler` lets you replace the default `CheckTx` logic entirely. The type is defined in [`types/abci.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/types/abci.go): + +```go +type CheckTxHandler func(runTx RunTx, req *abci.RequestCheckTx) (*abci.ResponseCheckTx, error) +``` + +Where `RunTx` is: + +```go +type RunTx = func(txBytes []byte, tx Tx) (gInfo GasInfo, result *Result, anteEvents []abci.Event, err error) +``` + +The handler receives the `runTx` closure from `BaseApp` (bound to the correct execution mode) and the raw ABCI request. It must return deterministic results for the same input bytes. + +Register a custom handler from `app.go`: + +```go +app.SetCheckTxHandler(myCheckTxHandler) +``` + +## PrepareProposal + +Based on validator voting power, CometBFT selects a block proposer and calls `PrepareProposal` on that validator's application. The proposer collects outstanding transactions from the mempool and returns a proposal to CometBFT. + +CometBFT's own mempool uses FIFO ordering. `PrepareProposal` gives the application full control to reorder, drop, or inject transactions before the proposal is sent. For example, an application can inject vote extension data from the previous block as synthetic transactions. What the application does here has no effect on CometBFT's mempool state. + +`PrepareProposal` MAY be non-deterministic and is only executed by the current block proposer. + +The Cosmos SDK provides `DefaultProposalHandler` in [`baseapp/abci_utils.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/baseapp/abci_utils.go), which selects transactions from the app-side mempool up to `req.MaxTxBytes` and the block gas limit. + + + +If you implement a custom `PrepareProposal` handler, the selected transactions MUST NOT exceed the maximum block gas (if set) or `req.MaxTxBytes`. + + + +To wire the default handler (or swap in a custom one) from `app.go`: + +```go +abciPropHandler := baseapp.NewDefaultProposalHandler(mempool, app) +app.SetPrepareProposal(abciPropHandler.PrepareProposalHandler()) +``` + +Vote extensions are only available at the height after they are enabled. See [Vote Extensions](/sdk/v0.54/guides/abci/vote-extensions) for details. + +## ProcessProposal + +After the block proposer broadcasts a proposal, every validator calls `ProcessProposal` to accept or reject it. The default implementation checks that each transaction decodes correctly and passes the `AnteHandler`. + +`ProcessProposal` MUST be deterministic. Non-deterministic results cause apphash mismatches across validators. If the handler panics or returns an error, honest validators prevote nil and CometBFT starts a new round with a new proposal. + +See the default implementation in [`baseapp/abci_utils.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/baseapp/abci_utils.go). + +To wire a custom handler: + +```go +app.SetProcessProposal(myProcessProposalHandler) +``` + +## ExtendVote and VerifyVoteExtensions + +These methods allow applications to extend the voting process by requiring validators to perform additional actions beyond simply validating blocks. + +If vote extensions are enabled, `ExtendVote` is called on every validator and each one returns its vote extension — an arbitrary byte slice. This data is only available in the next block height during `PrepareProposal`. Common use cases include prices for a price oracle or encryption shares for an encrypted transaction mempool. `ExtendVote` CAN be non-deterministic. + +`VerifyVoteExtension` is called on every validator to verify other validators' vote extensions. It MUST be deterministic. + +Applications must keep vote extension data concise, as large extensions degrade chain performance. See the [CometBFT QA results](/cometbft/latest/docs/qa/CometBFT-QA-38#vote-extensions-testbed) for benchmarks. + +See [Vote Extensions](/sdk/v0.54/guides/abci/vote-extensions) for implementation details. + +## FinalizeBlock + +`FinalizeBlock` is called once consensus is reached on a proposal. It executes all transactions in the block, runs `BeginBlock`/`EndBlock` equivalents, and commits the resulting state. It replaces the old `BeginBlock`, `DeliverTx`, and `EndBlock` methods from ABCI 1.0. + +See the implementation at [`baseapp/abci.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/baseapp/abci.go). diff --git a/sdk/v0.54/guides/abci/app-mempool.mdx b/sdk/v0.54/guides/abci/app-mempool.mdx new file mode 100644 index 000000000..8fa25448e --- /dev/null +++ b/sdk/v0.54/guides/abci/app-mempool.mdx @@ -0,0 +1,114 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/guides/abci/app-mempool' +title: Application Mempool +--- + + +**Synopsis** +This section describes how the app-side mempool can be used and replaced. + + +Since `v0.47` the application has its own mempool to allow much more granular +block building than previous versions. This change was enabled by +[ABCI 1.0](https://github.com/cometbft/cometbft/blob/v0.37.0/spec/abci). +Notably it introduces the `PrepareProposal` and `ProcessProposal` steps of ABCI++. + + +**Prerequisite Readings** + +* [BaseApp](/sdk/v0.54/learn/concepts/baseapp) +* [ABCI](/sdk/v0.54/guides/abci/abci) + + + +## Overview + +The application mempool is an in-process transaction store within the application that gives developers control over how transactions are ordered and selected for block inclusion. Unlike the [CometBFT mempool](/cometbft/latest/docs/core/mempool), which handles transaction receipt and gossip at the network layer, the SDK's application mempool operates at block proposal time to determine the ordering of transactions within a block. + +When a transaction is submitted to a node, CometBFT receives it first. CometBFT +calls `CheckTx` on the application to validate the transaction, then stores it +in its own mempool (the `flood` mempool by default) and gossips it to peers. + +When it is time to build a block, CometBFT calls `PrepareProposal` on the +application. This is where the SDK app mempool comes in: the application pulls +from its own internal mempool and decides which transactions to include and in +what order: by priority, nonce, fees, or any custom logic the developer +chooses. + +So CometBFT decides what gets accepted into the network; the SDK app mempool +decides how accepted transactions are ordered within a block. + +## Mempool + +There are countless designs that an application developer can write for a mempool, the SDK opted to provide only simple mempool implementations. +Namely, the SDK provides the following mempools: + +* [No-op Mempool](#no-op-mempool) +* [Sender Nonce Mempool](#sender-nonce-mempool) +* [Priority Nonce Mempool](#priority-nonce-mempool) + +By default, the SDK uses the [No-op Mempool](#no-op-mempool), but it can be replaced by the application developer in [`app.go`: + +```go +nonceMempool := mempool.NewSenderNonceMempool() + mempoolOpt := baseapp.SetMempool(nonceMempool) + +baseAppOptions = append(baseAppOptions, mempoolOpt) +``` + +### No-op Mempool + +A no-op mempool is a mempool where transactions are completely discarded and ignored when BaseApp interacts with the mempool. +When this mempool is used, it is assumed that an application will rely on CometBFT's transaction ordering defined in `RequestPrepareProposal`, +which is FIFO-ordered by default. + +> Note: If a NoOp mempool is used, PrepareProposal and ProcessProposal both should be aware of this as +> PrepareProposal could include transactions that could fail verification in ProcessProposal. + +### Sender Nonce Mempool + +The nonce mempool is a mempool that keeps transactions from an sorted by nonce in order to avoid the issues with nonces. +It works by storing the transaction in a list sorted by the transaction nonce. When the proposer asks for transactions to be included in a block it randomly selects a sender and gets the first transaction in the list. It repeats this until the mempool is empty or the block is full. + +It is configurable with the following parameters: + +#### MaxTxs + +It is an integer value that sets the mempool in one of three modes, *bounded*, *unbounded*, or *disabled*. + +* **negative**: Disabled, mempool does not insert new transaction and return early. +* **zero**: Unbounded mempool has no transaction limit and will never fail with `ErrMempoolTxMaxCapacity`. +* **positive**: Bounded, it fails with `ErrMempoolTxMaxCapacity` when `maxTx` value is the same as `CountTx()` + +#### Seed + +Set the seed for the random number generator used to select transactions from the mempool. + +### Priority Nonce Mempool + +The [priority nonce mempool](https://github.com/cosmos/cosmos-sdk/blob/main/types/mempool/priority_nonce_spec.md) is a mempool implementation that stores txs in a partially ordered set by 2 dimensions: + +* priority +* sender-nonce (sequence number) + +Internally it uses one priority ordered [skip list](https://pkg.go.dev/github.com/huandu/skiplist) and one skip list per sender ordered by sender-nonce (sequence number). When there are multiple txs from the same sender, they are not always comparable by priority to other sender txs and must be partially ordered by both sender-nonce and priority. + +It is configurable with the following parameters: + +#### MaxTxs + +It is an integer value that sets the mempool in one of three modes, *bounded*, *unbounded*, or *disabled*. + +* **negative**: Disabled, mempool does not insert new transaction and return early. +* **zero**: Unbounded mempool has no transaction limit and will never fail with `ErrMempoolTxMaxCapacity`. +* **positive**: Bounded, it fails with `ErrMempoolTxMaxCapacity` when `maxTx` value is the same as `CountTx()` + +#### Callback + +The priority nonce mempool provides mempool options allowing the application sets callback(s). + +* **OnRead**: Set a callback to be called when a transaction is read from the mempool. +* **TxReplacement**: Sets a callback to be called when duplicated transaction nonce detected during mempool insert. Application can define a transaction replacement rule based on tx priority or certain transaction fields. + +More information on the SDK mempool implementation can be found in the [godocs](https://pkg.go.dev/github.com/cosmos/cosmos-sdk/types/mempool). diff --git a/sdk/v0.54/guides/abci/vote-extensions.mdx b/sdk/v0.54/guides/abci/vote-extensions.mdx new file mode 100644 index 000000000..9ee958567 --- /dev/null +++ b/sdk/v0.54/guides/abci/vote-extensions.mdx @@ -0,0 +1,144 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/guides/abci/vote-extensions' +title: Vote Extensions +--- + +Vote extensions are arbitrary bytes that validators can attach to their pre-commit vote at block height `H`. They are part of ABCI 2.0 and are available starting from CometBFT v0.38 and Cosmos SDK v0.50. + +## Enabling vote extensions + +Vote extensions are controlled by the `VoteExtensionsEnableHeight` consensus parameter. At the configured height, CometBFT begins calling `ExtendVote` and `VerifyVoteExtension` on every validator. Extensions produced at height `H` are available to the block proposer at height `H+1` via `PrepareProposal`. + +To check whether vote extensions are active in a handler: + +```go +cp := ctx.ConsensusParams() +if cp.Abci != nil && req.Height > cp.Abci.VoteExtensionsEnableHeight { + // vote extensions are available +} +``` + +`ConsensusParams().Abci` is a pointer and must be nil-checked before use. + +## ExtendVote + +The Cosmos SDK defines [`ExtendVoteHandler`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/types/abci.go#L48): + +```go +type ExtendVoteHandler func(Context, *abci.RequestExtendVote) (*abci.ResponseExtendVote, error) +``` + +Register a handler in `app.go` via `baseapp.SetExtendVoteHandler` (defined in [`baseapp/options.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/baseapp/options.go)): + +```go +app.SetExtendVoteHandler(myExtendVoteHandler) +``` + +If `ExtendVoteHandler` is set, it **must** return a non-nil `VoteExtension`. An empty byte slice is valid. + +`ExtendVote` is called only on the local validator and does **not** need to be deterministic. Common uses include: + +- Submitting prices for an oracle +- Sharing encryption shares for an encrypted mempool + +Keep extensions small — large extensions increase consensus latency. See [CometBFT QA results](/cometbft/latest/docs/qa/CometBFT-QA-38#vote-extensions-testbed) for benchmarks. + +## VerifyVoteExtension + +The SDK defines [`VerifyVoteExtensionHandler`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/types/abci.go#L52): + +```go +type VerifyVoteExtensionHandler func(Context, *abci.RequestVerifyVoteExtension) (*abci.ResponseVerifyVoteExtension, error) +``` + +Register it in `app.go`: + +```go +app.SetVerifyVoteExtensionHandler(myVerifyVoteExtensionHandler) +``` + +`VerifyVoteExtension` is called on every validator for every peer's pre-commit. It **must** be deterministic — the same extension must produce the same result on every validator. If an application defines `ExtendVoteHandler`, it should also define a `VerifyVoteExtensionHandler`. + +Always validate the size of incoming extensions in this handler. + +## Validating vote extension signatures + +Before processing vote extensions in `PrepareProposal` or `ProcessProposal`, validate that they are properly signed. The SDK provides [`baseapp.ValidateVoteExtensions`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/baseapp/abci_utils.go) for this: + +```go +err := baseapp.ValidateVoteExtensions(ctx, valStore, req.Height, ctx.ChainID(), req.LocalLastCommit) +if err != nil { + return nil, err +} +``` + +`ValidateVoteExtensions` verifies that each vote extension in the commit is correctly signed by its validator. `valStore` is a [`baseapp.ValidatorStore`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/baseapp/abci_utils.go), an interface with a single method: + +```go +type ValidatorStore interface { + GetPubKeyByConsAddr(context.Context, sdk.ConsAddress) (cmtprotocrypto.PublicKey, error) +} +``` + +Call `ValidateVoteExtensions` in both `PrepareProposal` (on `req.LocalLastCommit`) and `ProcessProposal` (on the `ExtendedCommitInfo` recovered from the injected transaction) before trusting any extension data. + +## Vote extension propagation + +Vote extensions from height `H` are provided only to the block proposer at height `H+1` via `req.LocalLastCommit` in `PrepareProposal`. They are **not** provided to other validators during `ProcessProposal`. + +If all validators need to use extension data at `H+1`, the proposer must inject it into the block proposal. Since the `Txs` field in `PrepareProposal` is a `[][]byte`, any byte slice — including a serialized extensions summary — can be prepended to the proposal: + +```go +injectedVoteExtTx := StakeWeightedPrices{ + StakeWeightedPrices: stakeWeightedPrices, + ExtendedCommitInfo: req.LocalLastCommit, +} +bz, err := json.Marshal(injectedVoteExtTx) +if err != nil { + return nil, err +} +proposalTxs = append([][]byte{bz}, proposalTxs...) +``` + +`FinalizeBlock` ignores any byte slice that does not implement `sdk.Tx`, so injected extensions are safely skipped during message execution. + +For more details on propagation design, see the [ABCI 2.0 ADR](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/docs/architecture/adr-064-abci-2.0.md#vote-extension-propagation--verification). + +## Recovery via PreBlocker + +The SDK's `PreBlocker` runs before any message execution in `FinalizeBlock`. Use it to recover injected vote extensions and make the results available to modules during the block: + +```go +func (h *ProposalHandler) PreBlocker(ctx sdk.Context, req *abci.RequestFinalizeBlock) (*sdk.ResponsePreBlock, error) { + res := &sdk.ResponsePreBlock{} + if len(req.Txs) == 0 { + return res, nil + } + cp := ctx.ConsensusParams() + if cp.Abci != nil && req.Height > cp.Abci.VoteExtensionsEnableHeight { + var injectedVoteExtTx StakeWeightedPrices + if err := json.Unmarshal(req.Txs[0], &injectedVoteExtTx); err != nil { + return nil, err + } + if err := h.keeper.SetOraclePrices(ctx, injectedVoteExtTx.StakeWeightedPrices); err != nil { + return nil, err + } + } + return res, nil +} +``` + +Register the PreBlocker in `app.go` (see [`baseapp/options.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/baseapp/options.go)): + +```go +app.SetPreBlocker(proposalHandler.PreBlocker) +``` + +The `sdk.PreBlocker` type is defined in [`types/abci.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/types/abci.go): + +```go +type PreBlocker func(Context, *abci.RequestFinalizeBlock) (*ResponsePreBlock, error) +``` + +State written to the context inside `PreBlocker` is available to all `BeginBlock` and message handlers in the same block. diff --git a/sdk/v0.54/guides/guides.mdx b/sdk/v0.54/guides/guides.mdx new file mode 100644 index 000000000..381f1a232 --- /dev/null +++ b/sdk/v0.54/guides/guides.mdx @@ -0,0 +1,53 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/guides/guides' +title: Guides Overview +description: Deep dives into specific Cosmos SDK topics for developers who have completed the tutorial. +--- + +These guides go deeper on specific topics. If you've completed the [Build a Chain Tutorial](/sdk/v0.54/tutorials/example/00-overview) and want to learn more about a particular area, this is where to look. + +## Module Design + +Best practices and architectural patterns for building well-structured modules. + +- [Module Design Considerations](/sdk/v0.54/guides/module-design/module-design-considerations): module boundaries, state layout, privileged operations, and inter-module dependencies +- [Object-Capability Model](/sdk/v0.54/guides/module-design/ocap): how the SDK uses keeper interfaces to scope access between modules + +## ABCI + +How your application interacts with CometBFT at the protocol level, including advanced features such as mempool design and vote extensions. + +- [ABCI Overview](/sdk/v0.54/guides/abci/abci): CheckTx, PrepareProposal, ProcessProposal, FinalizeBlock +- [Application Mempool](/sdk/v0.54/guides/abci/app-mempool): custom mempool implementations and transaction ordering +- [Vote Extensions](/sdk/v0.54/guides/abci/vote-extensions): injecting application data into the consensus process + +## Tooling + +Tools available to Cosmos SDK developers. + +- [Tool Guide](/sdk/v0.54/guides/tooling/tool-guide): overview of all available tools by category +- [Writing CLI Commands](/sdk/v0.54/guides/tooling/autocli): AutoCLI and hand-written commands +- [Confix](/sdk/v0.54/guides/tooling/confix): managing and migrating node configuration + +## State + +How modules store and access state. + +- [Module Store Internals](/sdk/v0.54/guides/state/store): KVStore, prefix stores, and the multistore +- [Collections API](/sdk/v0.54/guides/state/collections): the modern Collections framework for module state + +## Upgrades and Migrations + +How to upgrade modules and chains without downtime. + +- [Upgrading Modules](/sdk/v0.54/guides/upgrades/upgrade): consensus versions, migration handlers, and store upgrades +- [Cosmovisor](/sdk/v0.54/guides/upgrades/cosmovisor): automated binary upgrade management + +## Testing and Observability + +Testing your modules and monitoring a running chain. + +- [Module Simulation](/sdk/v0.54/guides/testing/simulator): fuzz testing with the SDK's simulation framework +- [Telemetry](/sdk/v0.54/guides/testing/telemetry): metrics and instrumentation +- [Log v2](/sdk/v0.54/guides/testing/log): structured logging with zerolog and OpenTelemetry diff --git a/sdk/v0.54/guides/module-design/module-design-considerations.mdx b/sdk/v0.54/guides/module-design/module-design-considerations.mdx new file mode 100644 index 000000000..4fa9cbafa --- /dev/null +++ b/sdk/v0.54/guides/module-design/module-design-considerations.mdx @@ -0,0 +1,349 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/guides/module-design/module-design-considerations' +title: Module Design Considerations +--- + + +**Synopsis** +Modules define most of the logic of Cosmos SDK applications. Developers compose modules together using the Cosmos SDK to build their custom application-specific blockchains. This document outlines the basic concepts behind SDK modules and how to approach module management. + + +This page discusses some of the design considerations for building modules in the Cosmos SDK. + +For more in-depth information on modules, see the following pages: + + + + Deep dive into how modules work -- keepers, message handlers, query services, and the module manager. + + + Follow a step-by-step tutorial to build a custom module from scratch on an example Cosmos SDK chain. + + + +## Design Considerations + +Before writing any code, these are the key design decisions that shape how a module will behave, interoperate, and evolve. + +### Define clear module boundaries + +A module should own a single, well-scoped piece of application state. Resist the temptation to bundle unrelated functionality into one module because it is convenient. Narrow modules are easier to audit, re-use across chains, and upgrade independently. + +Ask: could a different chain reasonably use this module without modification? If the answer depends on removing half the features, the module is probably doing too much. + +### Plan your state structure early + +Every `KVStore` key your module defines is permanent: removing or renaming keys requires a migration. Use the [Collections](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/collections/README.md) library for structured state management, and name keys to be collision-resistant and self-documenting. + +Consider what your module needs to index. A value that is only ever looked up by a single key is simple. A value looked up by multiple dimensions (e.g. by owner and by ID) requires secondary indexes, which add complexity and storage overhead. + +### Design your message and query surface + +Keep the `Msg` service minimal. Every message your module accepts becomes part of your public API and must be handled across upgrades. Prefer fewer, general-purpose messages over many narrow ones. + +Queries are cheaper to add later than messages, but consider what clients need from day one. Poorly designed queries often lead to excessive on-chain state that exists solely to support a query no one else needs. + +### Decide how privileged operations are controlled + +Most modules have parameters that governance should be able to update. Use the standard `MsgUpdateParams` pattern with an `Authority` field, and set that authority to the governance module address at genesis. This ensures parameter changes go through on-chain governance rather than being hardcoded or requiring a chain upgrade. + +If your module needs to call into another module's privileged functions, establish those permissions through keeper references at app initialization -- not through dynamic lookups at runtime. + +### Model inter-module dependencies carefully + +List every other module your module needs access to. Each dependency becomes a keeper reference injected into your keeper at construction. Avoid circular dependencies: if module A needs B and B needs A, one of them is doing too much. Introduce a third module or restructure the shared logic. + +Prefer accepting interfaces over concrete keeper types. This makes your module testable in isolation and re-usable across chains with different module implementations. + +### Plan for upgrades from the start + +If your module defines state, it will eventually need a migration. Write migration logic in `x//migrations/` from the first version, even if v1 to v2 is a no-op. Establish the pattern early so upgrades are not an afterthought. + +See [Module Upgrades](/sdk/v0.54/guides/upgrades/upgrade) for implementation details. + +## Role of Modules in a Cosmos SDK Application + +The Cosmos SDK can be thought of as the Ruby-on-Rails of blockchain development. It comes with a core that provides the basic functionalities every blockchain application needs, like a [boilerplate implementation of the ABCI](/sdk/v0.54/learn/concepts/baseapp) to communicate with the underlying consensus engine, a [`multistore`](/sdk/v0.54/learn/concepts/store#multistore) to persist state, a [server](/sdk/v0.54/node/run-node) to form a full-node and interfaces to handle queries. + +On top of this core, the Cosmos SDK enables developers to build modules that implement the business logic of their application. In other words, SDK modules implement the bulk of the logic of applications, while the core does the wiring and enables modules to be composed together. The end goal is to build a robust ecosystem of open-source Cosmos SDK modules, making it increasingly easier to build complex blockchain applications. + +Cosmos SDK modules can be seen as little state-machines within the state-machine. They generally define a subset of the state using one or more `KVStore`s in the [main multistore](/sdk/v0.54/learn/concepts/store#multistore), as well as a subset of [message types](/sdk/v0.54/learn/concepts/transactions#messages). These messages are routed by one of the main components of Cosmos SDK core, [`BaseApp`](/sdk/v0.54/learn/concepts/baseapp), to a module Protobuf [`Msg` service](/sdk/v0.54/learn/concepts/transactions#messages) that defines them. + +```mermaid expandable +flowchart TD + A[Transaction relayed from the full-node's consensus engine to the node's application via FinalizeBlock] + A --> B[APPLICATION] + B --> C["Using baseapp's methods: Decode the Tx, extract and route the message(s)"] + C --> D[Message routed to the correct module to be processed] + D --> E[AUTH MODULE] + D --> F[BANK MODULE] + D --> G[STAKING MODULE] + D --> H[GOV MODULE] + H --> I[Handles message, Updates state] + E --> I + F --> I + G --> I + I --> J["Return result to the underlying consensus engine (e.g. CometBFT) (0=Ok, 1=Err)"] +``` + +As a result of this architecture, building a Cosmos SDK application usually revolves around writing modules to implement the specialized logic of the application and composing them with existing modules to complete the application. Developers will generally work on modules that implement logic needed for their specific use case that do not exist yet, and will use existing modules for more generic functionalities like staking, accounts, or token management. + +### Modules as super-users + +Modules have the ability to perform actions that are not available to regular users. This is because modules are given sudo permissions by the state machine. Modules can reject another modules desire to execute a function but this logic must be explicit. Examples of this can be seen when modules create functions to modify parameters: + +```go expandable +package keeper + +import ( + + "context" + "github.com/hashicorp/go-metrics" + + errorsmod "cosmossdk.io/errors" + "cosmossdk.io/x/bank/types" + "github.com/cosmos/cosmos-sdk/telemetry" + sdk "github.com/cosmos/cosmos-sdk/types" + sdkerrors "github.com/cosmos/cosmos-sdk/types/errors" +) + +type msgServer struct { + Keeper +} + +var _ types.MsgServer = msgServer{ +} + +// NewMsgServerImpl returns an implementation of the bank MsgServer interface +// for the provided Keeper. +func NewMsgServerImpl(keeper Keeper) + +types.MsgServer { + return &msgServer{ + Keeper: keeper +} +} + +func (k msgServer) + +Send(ctx context.Context, msg *types.MsgSend) (*types.MsgSendResponse, error) { + var ( + from, to []byte + err error + ) + if base, ok := k.Keeper.(BaseKeeper); ok { + from, err = base.ak.AddressCodec().StringToBytes(msg.FromAddress) + if err != nil { + return nil, sdkerrors.ErrInvalidAddress.Wrapf("invalid from address: %s", err) +} + +to, err = base.ak.AddressCodec().StringToBytes(msg.ToAddress) + if err != nil { + return nil, sdkerrors.ErrInvalidAddress.Wrapf("invalid to address: %s", err) +} + +} + +else { + return nil, sdkerrors.ErrInvalidRequest.Wrapf("invalid keeper type: %T", k.Keeper) +} + if !msg.Amount.IsValid() { + return nil, errorsmod.Wrap(sdkerrors.ErrInvalidCoins, msg.Amount.String()) +} + if !msg.Amount.IsAllPositive() { + return nil, errorsmod.Wrap(sdkerrors.ErrInvalidCoins, msg.Amount.String()) +} + if err := k.IsSendEnabledCoins(ctx, msg.Amount...); err != nil { + return nil, err +} + if k.BlockedAddr(to) { + return nil, errorsmod.Wrapf(sdkerrors.ErrUnauthorized, "%s is not allowed to receive funds", msg.ToAddress) +} + +err = k.SendCoins(ctx, from, to, msg.Amount) + if err != nil { + return nil, err +} + +defer func() { + for _, a := range msg.Amount { + if a.Amount.IsInt64() { + telemetry.SetGaugeWithLabels( + []string{"tx", "msg", "send" +}, + float32(a.Amount.Int64()), + []metrics.Label{ + telemetry.NewLabel("denom", a.Denom) +}, + ) +} + +} + +}() + +return &types.MsgSendResponse{ +}, nil +} + +func (k msgServer) + +MultiSend(ctx context.Context, msg *types.MsgMultiSend) (*types.MsgMultiSendResponse, error) { + if len(msg.Inputs) == 0 { + return nil, types.ErrNoInputs +} + if len(msg.Inputs) != 1 { + return nil, types.ErrMultipleSenders +} + if len(msg.Outputs) == 0 { + return nil, types.ErrNoOutputs +} + if err := types.ValidateInputOutputs(msg.Inputs[0], msg.Outputs); err != nil { + return nil, err +} + + // NOTE: totalIn == totalOut should already have been checked + for _, in := range msg.Inputs { + if err := k.IsSendEnabledCoins(ctx, in.Coins...); err != nil { + return nil, err +} + +} + for _, out := range msg.Outputs { + if base, ok := k.Keeper.(BaseKeeper); ok { + accAddr, err := base.ak.AddressCodec().StringToBytes(out.Address) + if err != nil { + return nil, err +} + if k.BlockedAddr(accAddr) { + return nil, errorsmod.Wrapf(sdkerrors.ErrUnauthorized, "%s is not allowed to receive funds", out.Address) +} + +} + +else { + return nil, sdkerrors.ErrInvalidRequest.Wrapf("invalid keeper type: %T", k.Keeper) +} + +} + err := k.InputOutputCoins(ctx, msg.Inputs[0], msg.Outputs) + if err != nil { + return nil, err +} + +return &types.MsgMultiSendResponse{ +}, nil +} + +func (k msgServer) + +UpdateParams(ctx context.Context, req *types.MsgUpdateParams) (*types.MsgUpdateParamsResponse, error) { + if k.GetAuthority() != req.Authority { + return nil, errorsmod.Wrapf(types.ErrInvalidSigner, "invalid authority; expected %s, got %s", k.GetAuthority(), req.Authority) +} + if err := req.Params.Validate(); err != nil { + return nil, err +} + if err := k.SetParams(ctx, req.Params); err != nil { + return nil, err +} + +return &types.MsgUpdateParamsResponse{ +}, nil +} + +func (k msgServer) + +SetSendEnabled(ctx context.Context, msg *types.MsgSetSendEnabled) (*types.MsgSetSendEnabledResponse, error) { + if k.GetAuthority() != msg.Authority { + return nil, errorsmod.Wrapf(types.ErrInvalidSigner, "invalid authority; expected %s, got %s", k.GetAuthority(), msg.Authority) +} + seen := map[string]bool{ +} + for _, se := range msg.SendEnabled { + if _, alreadySeen := seen[se.Denom]; alreadySeen { + return nil, sdkerrors.ErrInvalidRequest.Wrapf("duplicate denom entries found for %q", se.Denom) +} + +seen[se.Denom] = true + if err := se.Validate(); err != nil { + return nil, sdkerrors.ErrInvalidRequest.Wrapf("invalid SendEnabled denom %q: %s", se.Denom, err) +} + +} + for _, denom := range msg.UseDefaultFor { + if err := sdk.ValidateDenom(denom); err != nil { + return nil, sdkerrors.ErrInvalidRequest.Wrapf("invalid UseDefaultFor denom %q: %s", denom, err) +} + +} + if len(msg.SendEnabled) > 0 { + k.SetAllSendEnabled(ctx, msg.SendEnabled) +} + if len(msg.UseDefaultFor) > 0 { + k.DeleteSendEnabled(ctx, msg.UseDefaultFor...) +} + +return &types.MsgSetSendEnabledResponse{ +}, nil +} + +func (k msgServer) + +Burn(goCtx context.Context, msg *types.MsgBurn) (*types.MsgBurnResponse, error) { + var ( + from []byte + err error + ) + +var coins sdk.Coins + for _, coin := range msg.Amount { + coins = coins.Add(sdk.NewCoin(coin.Denom, coin.Amount)) +} + if base, ok := k.Keeper.(BaseKeeper); ok { + from, err = base.ak.AddressCodec().StringToBytes(msg.FromAddress) + if err != nil { + return nil, sdkerrors.ErrInvalidAddress.Wrapf("invalid from address: %s", err) +} + +} + +else { + return nil, sdkerrors.ErrInvalidRequest.Wrapf("invalid keeper type: %T", k.Keeper) +} + if !coins.IsValid() { + return nil, errorsmod.Wrap(sdkerrors.ErrInvalidCoins, coins.String()) +} + if !coins.IsAllPositive() { + return nil, errorsmod.Wrap(sdkerrors.ErrInvalidCoins, coins.String()) +} + +err = k.BurnCoins(goCtx, from, coins) + if err != nil { + return nil, err +} + +return &types.MsgBurnResponse{ +}, nil +} +``` + +## How to Approach Building Modules as a Developer + +While there are no definitive guidelines for writing modules, here are some important design principles developers should keep in mind when building them: + +* **Composability**: Cosmos SDK applications are almost always composed of multiple modules. This means developers need to carefully consider the integration of their module not only with the core of the Cosmos SDK, but also with other modules. The former is achieved by following standard design patterns outlined [here](#main-components-of-cosmos-sdk-modules), while the latter is achieved by properly exposing the store(s) of the module via the [`keeper`](/sdk/v0.54/learn/concepts/modules#keeper). +* **Specialization**: A direct consequence of the **composability** feature is that modules should be **specialized**. Developers should carefully establish the scope of their module and not batch multiple functionalities into the same module. This separation of concerns enables modules to be re-used in other projects and improves the upgradability of the application. **Specialization** also plays an important role in the [object-capabilities model](/sdk/v0.54/guides/module-design/ocap) of the Cosmos SDK. +* **Capabilities**: Most modules need to read and/or write to the store(s) of other modules. However, in an open-source environment, it is possible for some modules to be malicious. That is why module developers need to carefully think not only about how their module interacts with other modules, but also about how to give access to the module's store(s). The Cosmos SDK takes a capabilities-oriented approach to inter-module security. This means that each store defined by a module is accessed by a `key`, which is held by the module's [`keeper`](/sdk/v0.54/learn/concepts/modules#keeper). This `keeper` defines how to access the store(s) and under what conditions. Access to the module's store(s) is done by passing a reference to the module's `keeper`. + +## Main Components of Cosmos SDK Modules + +Modules are by convention defined in the `./x/` subfolder (e.g. the `bank` module will be defined in the `./x/bank` folder). They generally share the same core components: + +* A [`keeper`](/sdk/v0.54/learn/concepts/modules#keeper), used to access the module's store(s) and update the state. +* A [`Msg` service](/sdk/v0.54/learn/concepts/transactions#messages), used to process messages when they are routed to the module by [`BaseApp`](/sdk/v0.54/learn/concepts/baseapp#message-routing) and trigger state-transitions. +* A [query service](/sdk/v0.54/learn/concepts/transactions#queries), used to process user queries when they are routed to the module by [`BaseApp`](/sdk/v0.54/learn/concepts/baseapp#query-routing). +* Interfaces, for end users to query the subset of the state defined by the module and create `message`s of the custom types defined in the module. + +In addition to these components, modules implement the `AppModule` interface in order to be managed by the [`module manager`](/sdk/v0.54/learn/concepts/app-go#module-manager). diff --git a/sdk/v0.54/guides/module-design/ocap.mdx b/sdk/v0.54/guides/module-design/ocap.mdx new file mode 100644 index 000000000..9639c9579 --- /dev/null +++ b/sdk/v0.54/guides/module-design/ocap.mdx @@ -0,0 +1,100 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/guides/module-design/ocap' +title: Object-Capability Model +description: How the Cosmos SDK uses object capabilities to isolate modules and limit the blast radius of faulty or malicious code. +--- + +The Cosmos SDK is built around the **object-capability model** (ocap) — a security model designed for systems that compose untrusted components. + +The threat model is explicit: a thriving ecosystem of Cosmos SDK modules will eventually include faulty or malicious ones. Ocap limits the damage any single module can do. + +## How it works + +The model has two rules: + +1. An object can send a message to another object only if it holds a reference to it. +2. An object can obtain a reference to another object only by receiving it through a message. + +In practice: a module can only affect the state it has been explicitly handed access to. If the bank keeper was not passed to your module, your module cannot touch balances — full stop. There is no global registry to reach into. + +This makes security analysis local. You can audit what a module can do by looking at what references it was given at wiring time, without reading its implementation. + +## Pointer vs. value + +Only pass what a module needs. If you pass a pointer, you grant write access. If you pass a value, you grant read access. + +This code violates the principle — passing a pointer to an external module grants it the ability to mutate the account: + +```go +account := &AppAccount{ + Address: pub.Address(), + Coins: sdk.Coins{sdk.NewInt64Coin("ATM", 100)}, +} +sumValue := externalModule.ComputeSumValue(account) // can modify account +``` + +Pass a copy instead: + +```go +sumValue := externalModule.ComputeSumValue(*account) // read-only +``` + +## Keeper interfaces + +The most common place to apply ocap in SDK modules is at keeper boundaries. Instead of accepting a concrete keeper type from another module, define a narrow interface containing only the methods your module actually calls. + +For example, `x/distribution` needs to query balances and send coins, but it does not need the full bank keeper. It defines its own interface: + +```go +// x/distribution/types/expected_keepers.go +type BankKeeper interface { + GetAllBalances(ctx context.Context, addr sdk.AccAddress) sdk.Coins + SpendableCoins(ctx context.Context, addr sdk.AccAddress) sdk.Coins + SendCoinsFromModuleToModule(ctx context.Context, senderModule, recipientModule string, amt sdk.Coins) error + SendCoinsFromModuleToAccount(ctx context.Context, senderModule string, recipientAddr sdk.AccAddress, amt sdk.Coins) error + SendCoinsFromAccountToModule(ctx context.Context, senderAddr sdk.AccAddress, recipientModule string, amt sdk.Coins) error + BlockedAddr(addr sdk.AccAddress) bool +} +``` + +By convention these live in `types/expected_keepers.go`. The benefit is twofold: the interface documents exactly what cross-module access your module requires, and it makes the dependency easy to mock in tests. + +## Store isolation + +Modules do not receive direct access to the global multistore. Instead, each module gets a `store.KVStoreService` scoped to its own prefix — it can only read and write within that namespace. + +```go +type Keeper struct { + storeService store.KVStoreService + // ... +} +``` + +This means a bug or malicious call in one module's keeper cannot read or corrupt another module's state. The scoping is enforced at the store layer, not by convention. + +## Authority + +Some operations — updating parameters, pausing a module, triggering emergency actions — should only be callable by governance or another trusted account. The SDK handles this with an explicit `authority` string stored in the keeper. + +```go +type Keeper struct { + // the address capable of executing privileged messages, + // typically the x/gov module account + authority string +} +``` + +Message handlers check the caller against this address before proceeding: + +```go +if msg.Authority != k.authority { + return nil, errors.Wrapf(sdkerrors.ErrUnauthorized, "expected %s, got %s", k.authority, msg.Authority) +} +``` + +The authority address is set at wiring time in `app.go` and cannot be changed at runtime. This is ocap applied to governance: privileged capability is a reference, and only the holder of that reference can exercise it. + +See [`simapp/app.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/simapp/app.go) for how keeper dependencies and authorities are wired in a complete application. + +For background, see the [Wikipedia article on object-capability model](https://en.wikipedia.org/wiki/Object-capability_model). diff --git a/sdk/v0.54/guides/reference/bech32.mdx b/sdk/v0.54/guides/reference/bech32.mdx new file mode 100644 index 000000000..d982d080c --- /dev/null +++ b/sdk/v0.54/guides/reference/bech32.mdx @@ -0,0 +1,230 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/guides/reference/bech32' +title: Address Encoding +--- + +The Cosmos SDK uses the Bech32 address format for all user-facing addresses. Bech32 encoding provides robust integrity checks through checksums and includes a human-readable prefix (HRP) that provides contextual information about the address type. + +## Address Types + +The SDK defines three distinct address types, each with its own Bech32 prefix: + +| Address Type | Bech32 Prefix | Example | Purpose | +|--------------|---------------|---------|---------| +| Account Address | `cosmos` | `cosmos1r5v5sr...` | User accounts, balances, transactions | +| Validator Operator Address | `cosmosvaloper` | `cosmosvaloper1r5v5sr...` | Validator operator identity, staking operations | +| Consensus Address | `cosmosvalcons` | `cosmosvalcons1r5v5sr...` | Validator consensus participation, block signing | + +Each address type also has a corresponding public key prefix: +- Account public keys: `cosmospub` +- Validator public keys: `cosmosvaloperpub` +- Consensus public keys: `cosmosvalconspub` + +## Supported Key Schemes + +The Cosmos SDK supports three key schemes. The choice of scheme affects address length and whether it can be used for transactions or consensus: + +| | Address length in bytes | Public key length in bytes | Used for transaction authentication | Used for consensus (CometBFT) | +| :----------: | :---------------------: | :------------------------: | :---------------------------------: | :---------------------------: | +| `secp256k1` | 20 | 33 | yes | no | +| `secp256r1` | 32 | 33 | yes | no | +| `tm-ed25519` | -- not used -- | 32 | no | yes | + +`secp256k1` is the default for user accounts. `secp256r1` is supported as an alternative and produces longer addresses (32 bytes). `tm-ed25519` is used exclusively for validator consensus keys and does not produce a user-facing address. + +## Address Derivation + +Addresses are derived from public keys through cryptographic hashing. The process differs based on the key algorithm: + +### Secp256k1 Keys (Account Addresses) + +Account addresses use Bitcoin-style address derivation: + +``` +1. Public Key: 33 bytes (compressed secp256k1 public key) +2. SHA-256 hash of public key: 32 bytes +3. RIPEMD-160 hash of result: 20 bytes (final address) +``` + +**Implementation:** `crypto/keys/secp256k1/secp256k1.go` + +```go +func (pubKey *PubKey) Address() crypto.Address { + sha := sha256.Sum256(pubKey.Key) // Step 1: SHA-256 + hasherRIPEMD160 := ripemd160.New() + hasherRIPEMD160.Write(sha[:]) + return hasherRIPEMD160.Sum(nil) // Step 2: RIPEMD-160 = 20 bytes +} +``` + +### Ed25519 Keys (Consensus Addresses) + +Consensus addresses use truncated SHA-256: + +``` +1. Public Key: 32 bytes (Ed25519 public key) +2. SHA-256 hash, truncated to first 20 bytes +``` + +**Implementation:** `crypto/keys/ed25519/ed25519.go` + +```go +func (pubKey *PubKey) Address() crypto.Address { + return crypto.Address(tmhash.SumTruncated(pubKey.Key)) // SHA-256-20 +} +``` + +## Bech32 Encoding Process + +Once address bytes are derived, they're converted to Bech32 format: + +**Step 1: Convert from 8-bit to 5-bit encoding** + +```go +// Address bytes (20 bytes = 160 bits) +addressBytes := []byte{0x12, 0x34, ..., 0xab} // 20 bytes + +// Convert to 5-bit groups for Bech32 +converted, _ := bech32.ConvertBits(addressBytes, 8, 5, true) +``` + +**Step 2: Encode with Human-Readable Prefix** + +```go +// Combine HRP with converted bytes +bech32Address, _ := bech32.Encode("cosmos", converted) +// Result: "cosmos1r5v5srda7xfth3uckstjst6k05kmeyzptewwdk" +``` + +**Implementation:** `types/bech32/bech32.go` + +## Configuring Bech32 prefixes + +Every Cosmos SDK application sets its Bech32 prefixes and SLIP-44 coin type once at startup via `sdk.GetConfig()`, then seals the config so it cannot be changed at runtime. The defaults (`cosmos`, `cosmosvaloper`, etc.) are defined in [`types/config.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/types/config.go). Chain developers override them before the app starts: + +```go +config := sdk.GetConfig() +config.SetBech32PrefixForAccount("cosmos", "cosmospub") +config.SetBech32PrefixForValidator("cosmosvaloper", "cosmosvaloperpub") +config.SetBech32PrefixForConsensusNode("cosmosvalcons", "cosmosvalconspub") +config.SetCoinType(118) // SLIP-44 coin type +config.Seal() +``` + +## Address Validation + +The SDK validates addresses through: + +1. **Format validation**: Ensures valid Bech32 encoding +2. **Prefix validation**: Confirms correct HRP for address type +3. **Length validation**: Verifies address is exactly 20 bytes when decoded + +```go +func (bc Bech32Codec) StringToBytes(text string) ([]byte, error) { + hrp, bz, err := bech32.DecodeAndConvert(text) + if err != nil { + return nil, err + } + + if hrp != bc.Bech32Prefix { + return nil, fmt.Errorf("invalid prefix") + } + + return bz, sdk.VerifyAddressFormat(bz) // Checks length = 20 bytes +} +``` + +## Module Addresses + +Module accounts use deterministic address derivation defined in [ADR-028](https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-028-public-key-addresses.md): + +```go +// Module address without derivation keys +func Module(moduleName string) []byte { + return crypto.AddressHash([]byte(moduleName)) +} + +// Module address with derivation keys (new method) +func Module(moduleName string, derivationKeys ...[]byte) []byte { + mKey := append([]byte(moduleName), 0) // Null byte separator + addr := Hash("module", append(mKey, derivationKeys[0]...)) + return addr // 32 bytes (not 20 bytes like user addresses) +} +``` + +Module addresses are longer (32 bytes vs 20 bytes) to reduce collision probability. + +## Validator Address Relationships + +A validator has three related addresses: + +1. **Operator Address** (`cosmosvaloper1...`): The validator's operational identity, derived from the operator's account key +2. **Consensus Address** (`cosmosvalcons1...`): Derived from the validator's consensus public key (Ed25519), used for block signing +3. **Account Address** (`cosmos1...`): The operator's account for receiving rewards + +```go +// Validator stores its consensus pubkey +type Validator struct { + OperatorAddress string // cosmosvaloper1... (from operator's account) + ConsensusPubkey *Any // Ed25519 public key for signing + // ... +} + +// Consensus address is derived from the consensus pubkey +func (v Validator) GetConsAddr() ([]byte, error) { + pk := v.ConsensusPubkey.GetCachedValue().(cryptotypes.PubKey) + return pk.Address().Bytes(), nil // SHA-256-20 of Ed25519 pubkey +} +``` + +## Performance: Address Caching + +The SDK caches Bech32-encoded addresses to optimize repeated conversions: + +```go +var ( + accAddrCache *simplelru.LRU // 60,000 entries + valAddrCache *simplelru.LRU // 500 entries + consAddrCache *simplelru.LRU // 500 entries +) +``` + +When `Address.String()` is called, the SDK: +1. Checks the LRU cache for the encoded address +2. Returns cached value if found +3. Otherwise, performs Bech32 encoding and caches the result + +This significantly improves performance during block processing and state queries. + +## Complete Example + +Here's the full pipeline for creating an account address: + +```go +// 1. Generate keypair +privKey := secp256k1.GenPrivKey() // 32 bytes +pubKey := privKey.PubKey() // 33 bytes (compressed) + +// 2. Derive address bytes +sha := sha256.Sum256(pubKey.Bytes()) // 32 bytes +ripemd := ripemd160.Sum(sha[:]) // 20 bytes +addrBytes := ripemd[:] + +// 3. Create AccAddress type +accAddr := sdk.AccAddress(addrBytes) + +// 4. Convert to Bech32 string +// Internally: bech32.ConvertAndEncode("cosmos", addrBytes) +addressStr := accAddr.String() +// Result: "cosmos1r5v5srda7xfth3uckstjst6k05kmeyzptewwdk" + +// 5. Use in account +account := auth.NewBaseAccount(accAddr, pubKey, accountNumber, sequence) +``` + +## Related Concepts + +- [Accounts](/sdk/v0.54/learn/concepts/accounts) - Understanding account types and management +- [Store](/sdk/v0.54/learn/concepts/store) - How addresses are used as keys in state storage +- [Transactions](/sdk/v0.54/learn/concepts/transactions) - How addresses are used in transaction signing diff --git a/sdk/v0.54/guides/reference/packages.mdx b/sdk/v0.54/guides/reference/packages.mdx new file mode 100644 index 000000000..c694d424a --- /dev/null +++ b/sdk/v0.54/guides/reference/packages.mdx @@ -0,0 +1,45 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/guides/reference/packages' +title: SDK Go Packages +description: >- + The Cosmos SDK is a collection of Go modules. This section provides + documentation on various packages that can be used when developing a Cosmos SDK + chain. It lists all standalone Go modules that are part of the Cosmos SDK. +--- + +The Cosmos SDK is a collection of Go modules. This section provides documentation on various packages that can be used when developing a Cosmos SDK chain. + + +For more information on SDK modules, see the [SDK Modules](/sdk/v0.54/modules/modules) section. +For more information on SDK tooling, see the [Tooling](/sdk/v0.54/guides/tooling/tool-guide) section. + + +## Core + +* [Core](https://pkg.go.dev/cosmossdk.io/core) - Core library defining SDK interfaces ([ADR-063](/sdk/v0.54/reference/architecture/adr-063-core-module-api)) +* [API](https://pkg.go.dev/cosmossdk.io/api) - API library containing generated SDK Pulsar API +* [Store](https://pkg.go.dev/cosmossdk.io/store) - Implementation of the Cosmos SDK store + +## State Management + +* [Collections](https://pkg.go.dev/cosmossdk.io/collections) - Typed state management library with automatic key encoding, iteration, and secondary indexes. See the [Collections guide](/sdk/v0.54/guides/state/collections). +* [ORM](https://pkg.go.dev/cosmossdk.io/orm) - ORM-style state layer built on top of collections, providing table abstractions with primary and secondary indexes. Based on [ADR-055](/sdk/v0.54/reference/architecture/adr-055-orm). + +## Automation + +* [Client/v2](https://pkg.go.dev/cosmossdk.io/client/v2) - Library powering [AutoCLI](/sdk/v0.54/guides/tooling/autocli) + +## Transactions + +* [x/tx](https://pkg.go.dev/cosmossdk.io/x/tx) - Transaction signing types, sign mode implementations (direct, amino JSON, textual), and transaction decoder utilities. + +## Utilities + +* [Log](https://pkg.go.dev/cosmossdk.io/log) - Logging library +* [Errors](https://pkg.go.dev/cosmossdk.io/errors) - Error handling library +* [Math](https://pkg.go.dev/cosmossdk.io/math) - Math library for SDK arithmetic operations + +## SimApp + +* [SimApp](https://pkg.go.dev/cosmossdk.io/simapp) - SimApp is a sample Cosmos SDK chain used for testing and development. diff --git a/sdk/v0.54/guides/reference/proto-docs.mdx b/sdk/v0.54/guides/reference/proto-docs.mdx new file mode 100644 index 000000000..29eb6d39d --- /dev/null +++ b/sdk/v0.54/guides/reference/proto-docs.mdx @@ -0,0 +1,6 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/guides/reference/proto-docs' +title: "Cosmos Protobuf Docs" +url: "https://buf.build/cosmos/cosmos-sdk/docs/main" +--- diff --git a/sdk/v0.54/guides/reference/protobuf-annotations.mdx b/sdk/v0.54/guides/reference/protobuf-annotations.mdx new file mode 100644 index 000000000..81762b079 --- /dev/null +++ b/sdk/v0.54/guides/reference/protobuf-annotations.mdx @@ -0,0 +1,162 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/guides/reference/protobuf-annotations' +title: Protobuf Annotations +description: >- + This document explains the various protobuf scalars that have been added to + make working with protobuf easier for Cosmos SDK application developers +--- + +This document explains the various protobuf scalars that have been added to make working with protobuf easier for Cosmos SDK application developers + +### Gogoproto + +Modules are encouraged to utilize Protobuf encoding for their respective types. In the Cosmos SDK, we use the [Gogoproto](https://github.com/cosmos/gogoproto) specific implementation of the Protobuf spec that offers speed and developer experience improvements compared to the official [Google protobuf implementation](https://github.com/protocolbuffers/protobuf). + +### Guidelines for protobuf message definitions + +In addition to [following official Protocol Buffer guidelines](https://developers.google.com/protocol-buffers/docs/proto3#simple), we recommend using these annotations in `.proto` files when dealing with interfaces: + +* Use `cosmos_proto.accepts_interface` to annotate `Any` fields that accept interfaces: + * Pass the same fully qualified name as `protoName` to `InterfaceRegistry.RegisterInterface`. + * Example: `(cosmos_proto.accepts_interface) = "cosmos.gov.v1beta1.Content"` (not just `Content`). +* Annotate interface implementations with `cosmos_proto.implements_interface`: + * Pass the same fully qualified name as `protoName` to `InterfaceRegistry.RegisterInterface`. + * Example: `(cosmos_proto.implements_interface) = "cosmos.authz.v1beta1.Authorization"` (not just `Authorization`). + +Code generators can then match the `accepts_interface` and `implements_interface` annotations to determine whether some Protobuf messages are allowed to be packed in a given `Any` field. + + +## Signer + +Signer specifies which field should be used to determine the signer of a message for the Cosmos SDK. This field can be used for clients as well to infer which field should be used to determine the signer of a message. + +Read more about the signer field [here](/sdk/v0.54/learn/concepts/encoding#message-signers). + +```proto +// Reference: https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/proto/cosmos/bank/v1beta1/tx.proto#L40 +option (cosmos.msg.v1.signer) = "from_address"; +``` + +## Scalar + +The scalar type defines a way for clients to understand how to construct protobuf messages according to what is expected by the module and sdk. + +```proto +(cosmos_proto.scalar) = "cosmos.AddressString" +``` + +Example of account address string scalar: + +```proto +// https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/proto/cosmos/bank/v1beta1/tx.proto#L46 +string from_address = 1 [(cosmos_proto.scalar) = "cosmos.AddressString"]; +``` + +Example of validator address string scalar: + +```proto +// https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/proto/cosmos/distribution/v1beta1/query.proto#L108 +string validator_address = 1 [(cosmos_proto.scalar) = "cosmos.ValidatorAddressString"]; +``` + +Example of Dec scalar: + +```proto +// https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/proto/cosmos/distribution/v1beta1/distribution.proto#L17 +string community_tax = 1 [(cosmos_proto.scalar) = "cosmos.Dec"]; +``` + +Example of Int scalar: + +```proto +// https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/proto/cosmos/gov/v1/gov.proto#L127 +string yes_count = 1 [(cosmos_proto.scalar) = "cosmos.Int"]; +``` + +There are a few options for what can be provided as a scalar: `cosmos.AddressString`, `cosmos.ValidatorAddressString`, `cosmos.ConsensusAddressString`, `cosmos.Int`, `cosmos.Dec`. + +## Implements\_Interface + +Implement interface is used to provide information to client tooling like [telescope](https://github.com/cosmology-tech/telescope) on how to encode and decode protobuf messages. + +```proto +option (cosmos_proto.implements_interface) = "cosmos.auth.v1beta1.AccountI"; +``` + +## Method,Field,Message Added In + +`method_added_in`, `field_added_in` and `message_added_in` are annotations to indicate to clients that a method, field, or message has been supported since a later version. This is useful when new methods or fields are added in later versions and the client needs to be aware of what it can call. + +The annotations are used as follows: + +```proto +option (cosmos_proto.method_added_in) = "cosmos-sdk 0.50.1"; +option (cosmos_proto.field_added_in) = "cosmos-sdk 0.50.1"; +option (cosmos_proto.message_added_in) = "cosmos-sdk 0.50.1"; +``` + +## Amino + +The amino codec was removed in `v0.50+`, this means there is not a need register `legacyAminoCodec`. To replace the amino codec, Amino protobuf annotations are used to provide information to the amino codec on how to encode and decode protobuf messages. + + +Amino annotations are only used for backwards compatibility with amino. New modules are not required use amino annotations. + + +The below annotations are used to provide information to the amino codec on how to encode and decode protobuf messages in a backwards compatible manner. + +### Name + +Name specifies the amino name that would show up for the user in order for them see which message they are signing. + +```proto +// https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/proto/cosmos/bank/v1beta1/tx.proto#L41 +option (amino.name) = "cosmos-sdk/MsgSend"; +``` + +### Field\_Name + +Field name specifies the amino name that would show up for the user in order for them see which field they are signing. + +```proto +// https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/proto/cosmos/distribution/v1beta1/distribution.proto#L165 +uint64 height = 3 [(amino.field_name) = "creation_height"]; +``` + +### Dont\_OmitEmpty + +Dont omitempty specifies that the field should not be omitted when encoding to amino. + +```proto +// https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/proto/cosmos/bank/v1beta1/tx.proto#L48 +repeated cosmos.base.v1beta1.Coin amount = 3 [(amino.dont_omitempty) = true]; +``` + +### Encoding + +Encoding instructs the amino json marshaler how to encode certain fields that may differ from the standard encoding behavior. The most common example of this is how `repeated cosmos.base.v1beta1.Coin` is encoded when using the amino json encoding format. The `legacy_coins` option tells the json marshaler [how to encode a null slice](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/x/tx/signing/aminojson/json_marshal.go#L85) of `cosmos.base.v1beta1.Coin`. + +```proto +// https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/proto/cosmos/bank/v1beta1/genesis.proto#L23 +(amino.encoding) = "legacy_coins", +``` + +## Module Query Safe + +The `cosmos.query.v1.module_query_safe` annotation ([source](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/proto/cosmos/query/v1/query.proto)) marks a query method as safe to call from within the state machine — for example from another module's keeper, via ADR-033 intermodule calls, or from CosmWasm contracts. + +```proto +rpc Balance(QueryBalanceRequest) returns (QueryBalanceResponse) { + option (cosmos.query.v1.module_query_safe) = true; +} +``` + +When set to `true`, the annotation asserts that the query is: + +1. **Deterministic**: given a block height, it returns the exact same response on every call and does not introduce state-machine-breaking changes across SDK patch versions. +2. **Gas-tracked**: gas consumption is correctly accounted for, preventing attack vectors where high-computation queries consume no gas. + +If you add this annotation to your own query, you must ensure both conditions hold. For queries that may consume significant gas (for example those with pagination that could be misconfigured), add a Protobuf comment warning downstream module developers. + +This annotation was introduced in v0.47. diff --git a/sdk/v0.54/guides/state/collections.mdx b/sdk/v0.54/guides/state/collections.mdx new file mode 100644 index 000000000..e6299da9d --- /dev/null +++ b/sdk/v0.54/guides/state/collections.mdx @@ -0,0 +1,1390 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/guides/state/collections' +title: Collections API +description: >- + Collections is a library meant to simplify the experience with respect to + module state handling. +--- + +Collections is a library meant to simplify the experience with respect to module state handling. + +Cosmos SDK modules handle their state using the `KVStore` interface. The problem with working with +`KVStore` is that it forces you to think of state as a bytes KV pairings when in reality the majority of +state comes from complex concrete golang objects (strings, ints, structs, etc.). + +Collections allows you to work with state as if they were normal golang objects and removes the need +for you to think of your state as raw bytes in your code. + +It also allows you to migrate your existing state without causing any state breakage that forces you into +tedious and complex chain state migrations. + +## Installation + +To install collections in your cosmos-sdk chain project, run the following command: + +```shell +go get cosmossdk.io/collections +``` + +## Core types + +Collections offers 5 different APIs to work with state, which will be explored in the next sections, these APIs are: + +* `Map`: to work with typed arbitrary KV pairings. +* `KeySet`: to work with just typed keys +* `Item`: to work with just one typed value +* `Sequence`: which is a monotonically increasing number. +* `IndexedMap`: which combines `Map` and `KeySet` to provide a `Map` with indexing capabilities. + +## Preliminary components + +Before exploring the different collections types and their capability it is necessary to introduce +the three components that every collection shares. In fact when instantiating a collection type by doing, for example, +`collections.NewMap/collections.NewItem/...` you will find yourself having to pass them some common arguments. + +For example, in code: + +```go expandable +package collections + +import ( + + "cosmossdk.io/collections" + store "cosmossdk.io/core/store" +) + +var AllowListPrefix = collections.NewPrefix(0) + +type Keeper struct { + Schema collections.Schema + AllowList collections.KeySet[string] +} + +func NewKeeper(storeService store.KVStoreService) + +Keeper { + sb := collections.NewSchemaBuilder(storeService) + +return Keeper{ + AllowList: collections.NewKeySet(sb, AllowListPrefix, "allow_list", collections.StringKey), +} +} +``` + +Let's analyze the shared arguments, what they do, and why we need them. + +### SchemaBuilder + +The first argument passed is the `SchemaBuilder` + +`SchemaBuilder` is a structure that keeps track of all the state of a module, it is not required by the collections +to deal with state but it offers a dynamic and reflective way for clients to explore a module's state. + +We instantiate a `SchemaBuilder` by passing it a `store.KVStoreService`, which is the module's store service obtained via dependency injection or `runtime.NewKVStoreService`. + +We then need to pass the schema builder to every collection type we instantiate in our keeper, in our case the `AllowList`. + +After creating all collections, call `sb.Build()` to validate prefix uniqueness and finalize the schema. Store the returned `collections.Schema` in the keeper's `Schema` field: + +```go +k := Keeper{ + AllowList: collections.NewKeySet(sb, AllowListPrefix, "allow_list", collections.StringKey), +} +schema, err := sb.Build() +if err != nil { + panic(err) +} +k.Schema = schema +return k +``` + +The code examples in this document show the collection instantiation patterns but omit the `sb.Build()` call for brevity. In production code, `sb.Build()` is required. + +### Prefix + +The second argument passed to our `KeySet` is a `collections.Prefix`, a prefix represents a partition of the module's `KVStore` +where all the state of a specific collection will be saved. + +Since a module can have multiple collections, the following is expected: + +* module params will become a `collections.Item` +* the `AllowList` is a `collections.KeySet` + +We don't want a collection to write over the state of the other collection so we pass it a prefix, which defines a storage +partition owned by the collection. + +If you already built modules, the prefix translates to the items you were creating in your `types/keys.go` file, example: [Link](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/x/feegrant/key.go#L16-L22) + +your old: + +```go +var ( + // FeeAllowanceKeyPrefix is the set of the kvstore for fee allowance data + // - 0x00: allowance + FeeAllowanceKeyPrefix = []byte{0x00 +} + + // FeeAllowanceQueueKeyPrefix is the set of the kvstore for fee allowance keys data + // - 0x01: + FeeAllowanceQueueKeyPrefix = []byte{0x01 +} +) +``` + +becomes: + +```go +var ( + // FeeAllowanceKeyPrefix is the set of the kvstore for fee allowance data + // - 0x00: allowance + FeeAllowanceKeyPrefix = collections.NewPrefix(0) + + // FeeAllowanceQueueKeyPrefix is the set of the kvstore for fee allowance keys data + // - 0x01: + FeeAllowanceQueueKeyPrefix = collections.NewPrefix(1) +) +``` + +#### Rules + +`collections.NewPrefix` accepts either `int`, `string` or `[]byte`. It is good practice to use a monotonically increasing `int` (values 0–255) for disk space efficiency. + +A collection **MUST NOT** share the same prefix as another collection in the same module, and a collection prefix **MUST NEVER** start with the same prefix as another, examples: + +```go +prefix1 := collections.NewPrefix("prefix") + +prefix2 := collections.NewPrefix("prefix") // THIS IS BAD! +``` + +```go +prefix1 := collections.NewPrefix("a") + +prefix2 := collections.NewPrefix("aa") // prefix2 starts with the same as prefix1: BAD!!! +``` + +### Human-Readable Name + +The third parameter we pass to a collection is a string, which is a human-readable name. +It is needed to make the role of a collection understandable by clients who have no clue about +what a module is storing in state. + +#### Rules + +Each collection in a module **MUST** have a unique humanized name. + +## Key and Value Codecs + +A collection is generic over the type you can use as keys or values. +This makes collections dumb, but also means that hypothetically we can store everything +that can be a go type into a collection. We are not bounded to any type of encoding (be it proto, json or whatever) + +So a collection needs to be given a way to understand how to convert your keys and values to bytes. +This is achieved through `KeyCodec` and `ValueCodec`, which are arguments that you pass to your +collections when you're instantiating them using the `collections.NewMap/collections.NewItem/...` +instantiation functions. + +NOTE: Generally speaking you will never be required to implement your own `Key/ValueCodec` as +the SDK and collections libraries already come with default, safe and fast implementation of those. +You might need to implement them only if you're migrating to collections and there are state layout incompatibilities. + +Let's explore an example: + +```go expandable +package collections + +import ( + + "cosmossdk.io/collections" + store "cosmossdk.io/core/store" + sdk "github.com/cosmos/cosmos-sdk/types" +) + +var IDsPrefix = collections.NewPrefix(0) + +type Keeper struct { + Schema collections.Schema + IDs collections.Map[string, uint64] +} + +func NewKeeper(storeService store.KVStoreService) + +Keeper { + sb := collections.NewSchemaBuilder(storeService) + +return Keeper{ + IDs: collections.NewMap(sb, IDsPrefix, "ids", collections.StringKey, collections.Uint64Value), +} +} +``` + +We're now instantiating a map where the key is string and the value is `uint64`. +We already know the first three arguments of the `NewMap` function. + +The fourth parameter is our `KeyCodec`, we know that the `Map` has `string` as key so we pass it a `KeyCodec` that handles strings as keys. + +The fifth parameter is our `ValueCodec`, we know that the `Map` has a `uint64` as value so we pass it a `ValueCodec` that handles uint64. + +Collections already comes with all the required implementations for golang primitive types. + +Let's make another example, this falls closer to what we build using cosmos SDK, let's say we want +to create a `collections.Map` that maps account addresses to their base account. So we want to map an `sdk.AccAddress` to an `auth.BaseAccount` (which is a proto): + +```go expandable +package collections + +import ( + + "cosmossdk.io/collections" + store "cosmossdk.io/core/store" + "github.com/cosmos/cosmos-sdk/codec" + sdk "github.com/cosmos/cosmos-sdk/types" + authtypes "github.com/cosmos/cosmos-sdk/x/auth/types" +) + +var AccountsPrefix = collections.NewPrefix(0) + +type Keeper struct { + Schema collections.Schema + Accounts collections.Map[sdk.AccAddress, authtypes.BaseAccount] +} + +func NewKeeper(storeService store.KVStoreService, cdc codec.BinaryCodec) + +Keeper { + sb := collections.NewSchemaBuilder(storeService) + +return Keeper{ + Accounts: collections.NewMap(sb, AccountsPrefix, "accounts", + sdk.AccAddressKey, codec.CollValue[authtypes.BaseAccount](cdc)), +} +} +``` + +As we can see here since our `collections.Map` maps `sdk.AccAddress` to `authtypes.BaseAccount`, +we use the `sdk.AccAddressKey` which is the `KeyCodec` implementation for `AccAddress` and we use `codec.CollValue` to +encode our proto type `BaseAccount`. + +Generally speaking you will always find the respective key and value codecs for types in the `go.mod` path you're using +to import that type. If you want to encode proto values refer to the codec `codec.CollValue` function, which allows you +to encode any type implement the `proto.Message` interface. + +## Map + +We analyze the first and most important collection type, the `collections.Map`. +This is the type that everything else builds on top of. + +### Use case + +A `collections.Map` is used to map arbitrary keys with arbitrary values. + +### Example + +It's easier to explain a `collections.Map` capabilities through an example: + +```go expandable +package collections + +import ( + + "cosmossdk.io/collections" + store "cosmossdk.io/core/store" + "fmt" + "github.com/cosmos/cosmos-sdk/codec" + sdk "github.com/cosmos/cosmos-sdk/types" + authtypes "github.com/cosmos/cosmos-sdk/x/auth/types" +) + +var AccountsPrefix = collections.NewPrefix(0) + +type Keeper struct { + Schema collections.Schema + Accounts collections.Map[sdk.AccAddress, authtypes.BaseAccount] +} + +func NewKeeper(storeService store.KVStoreService, cdc codec.BinaryCodec) + +Keeper { + sb := collections.NewSchemaBuilder(storeService) + +return Keeper{ + Accounts: collections.NewMap(sb, AccountsPrefix, "accounts", + sdk.AccAddressKey, codec.CollValue[authtypes.BaseAccount](cdc)), +} +} + +func (k Keeper) + +CreateAccount(ctx sdk.Context, addr sdk.AccAddress, account authtypes.BaseAccount) + +error { + has, err := k.Accounts.Has(ctx, addr) + if err != nil { + return err +} + if has { + return fmt.Errorf("account already exists: %s", addr) +} + +err = k.Accounts.Set(ctx, addr, account) + if err != nil { + return err +} + +return nil +} + +func (k Keeper) + +GetAccount(ctx sdk.Context, addr sdk.AccAddress) (authtypes.BaseAccount, error) { + acc, err := k.Accounts.Get(ctx, addr) + if err != nil { + return authtypes.BaseAccount{ +}, err +} + +return acc, nil +} + +func (k Keeper) + +RemoveAccount(ctx sdk.Context, addr sdk.AccAddress) + +error { + err := k.Accounts.Remove(ctx, addr) + if err != nil { + return err +} + +return nil +} +``` + +#### Set method + +Set maps with the provided `AccAddress` (the key) to the `auth.BaseAccount` (the value). + +Under the hood the `collections.Map` will convert the key and value to bytes using the [key and value codec](#key-and-value-codecs). +It will prepend to our bytes key the [prefix](#prefix) and store it in the KVStore of the module. + +#### Has method + +The has method reports if the provided key exists in the store. + +#### Get method + +The get method accepts the `AccAddress` and returns the associated `auth.BaseAccount` if it exists, otherwise it errors. + +#### Remove method + +The remove method accepts the `AccAddress` and removes it from the store. It won't report errors +if it does not exist, to check for existence before removal use the `Has` method. + +#### Iteration + +Iteration has a separate section. + +## KeySet + +The second type of collection is `collections.KeySet`, as the word suggests it maintains +only a set of keys without values. + +#### Implementation curiosity + +A `collections.KeySet` is just a `collections.Map` with a `key` but no value. +The value internally is always the same and is represented as an empty byte slice `[]byte{}`. + +### Example + +As always we explore the collection type through an example: + +```go expandable +package collections + +import ( + + "cosmossdk.io/collections" + store "cosmossdk.io/core/store" + "fmt" + sdk "github.com/cosmos/cosmos-sdk/types" +) + +var ValidatorsSetPrefix = collections.NewPrefix(0) + +type Keeper struct { + Schema collections.Schema + ValidatorsSet collections.KeySet[sdk.ValAddress] +} + +func NewKeeper(storeService store.KVStoreService) + +Keeper { + sb := collections.NewSchemaBuilder(storeService) + +return Keeper{ + ValidatorsSet: collections.NewKeySet(sb, ValidatorsSetPrefix, "validators_set", sdk.ValAddressKey), +} +} + +func (k Keeper) + +AddValidator(ctx sdk.Context, validator sdk.ValAddress) + +error { + has, err := k.ValidatorsSet.Has(ctx, validator) + if err != nil { + return err +} + if has { + return fmt.Errorf("validator already in set: %s", validator) +} + +err = k.ValidatorsSet.Set(ctx, validator) + if err != nil { + return err +} + +return nil +} + +func (k Keeper) + +RemoveValidator(ctx sdk.Context, validator sdk.ValAddress) + +error { + err := k.ValidatorsSet.Remove(ctx, validator) + if err != nil { + return err +} + +return nil +} +``` + +The first difference we notice is that `KeySet` needs use to specify only one type parameter: the key (`sdk.ValAddress` in this case). +The second difference we notice is that `KeySet` in its `NewKeySet` function does not require +us to specify a `ValueCodec` but only a `KeyCodec`. This is because a `KeySet` only saves keys and not values. + +Let's explore the methods. + +#### Has method + +Has allows us to understand if a key is present in the `collections.KeySet` or not, functions in the same way as `collections.Map.Has +` + +#### Set method + +Set inserts the provided key in the `KeySet`. + +#### Remove method + +Remove removes the provided key from the `KeySet`, it does not error if the key does not exist, +if existence check before removal is required it needs to be coupled with the `Has` method. + +## Item + +The third type of collection is the `collections.Item`. +It stores only one single item, it's useful for example for parameters, there's only one instance +of parameters in state always. + +#### implementation curiosity + +A `collections.Item` is just a `collections.Map` with no key but just a value. +The key is the prefix of the collection! + +### Example + +```go expandable +package collections + +import ( + + "cosmossdk.io/collections" + store "cosmossdk.io/core/store" + "github.com/cosmos/cosmos-sdk/codec" + sdk "github.com/cosmos/cosmos-sdk/types" + stakingtypes "cosmossdk.io/x/staking/types" +) + +var ParamsPrefix = collections.NewPrefix(0) + +type Keeper struct { + Schema collections.Schema + Params collections.Item[stakingtypes.Params] +} + +func NewKeeper(storeService store.KVStoreService, cdc codec.BinaryCodec) + +Keeper { + sb := collections.NewSchemaBuilder(storeService) + +return Keeper{ + Params: collections.NewItem(sb, ParamsPrefix, "params", codec.CollValue[stakingtypes.Params](cdc)), +} +} + +func (k Keeper) + +UpdateParams(ctx sdk.Context, params stakingtypes.Params) + +error { + err := k.Params.Set(ctx, params) + if err != nil { + return err +} + +return nil +} + +func (k Keeper) + +GetParams(ctx sdk.Context) (stakingtypes.Params, error) { + return k.Params.Get(ctx) +} +``` + +The first key difference we notice is that we specify only one type parameter, which is the value we're storing. +The second key difference is that we don't specify the `KeyCodec`, since we store only one item we already know the key +and the fact that it is constant. + +## Iteration + +One of the key features of the `KVStore` is iterating over keys. + +Collections which deal with keys (so `Map`, `KeySet` and `IndexedMap`) allow you to iterate +over keys in a safe and typed way. They all share the same API, the only difference being +that `KeySet` returns a different type of `Iterator` because `KeySet` only deals with keys. + + + +Every collection shares the same `Iterator` semantics. + + + +Let's have a look at the `Map.Iterate` method: + +```go +func (m Map[K, V]) + +Iterate(ctx context.Context, ranger Ranger[K]) (Iterator[K, V], error) +``` + +It accepts a `collections.Ranger[K]`, which is an API that instructs map on how to iterate over keys. +As always we don't need to implement anything here as `collections` already provides some generic `Ranger` implementers +that expose all you need to work with ranges. + +### Example + +We have a `collections.Map` that maps accounts using `uint64` IDs. + +```go expandable +package collections + +import ( + + "cosmossdk.io/collections" + store "cosmossdk.io/core/store" + "github.com/cosmos/cosmos-sdk/codec" + sdk "github.com/cosmos/cosmos-sdk/types" + authtypes "github.com/cosmos/cosmos-sdk/x/auth/types" +) + +var AccountsPrefix = collections.NewPrefix(0) + +type Keeper struct { + Schema collections.Schema + Accounts collections.Map[uint64, authtypes.BaseAccount] +} + +func NewKeeper(storeService store.KVStoreService, cdc codec.BinaryCodec) + +Keeper { + sb := collections.NewSchemaBuilder(storeService) + +return Keeper{ + Accounts: collections.NewMap(sb, AccountsPrefix, "accounts", collections.Uint64Key, codec.CollValue[authtypes.BaseAccount](cdc)), +} +} + +func (k Keeper) + +GetAllAccounts(ctx sdk.Context) ([]authtypes.BaseAccount, error) { + // passing a nil Ranger equals to: iterate over every possible key + iter, err := k.Accounts.Iterate(ctx, nil) + if err != nil { + return nil, err +} + +accounts, err := iter.Values() + if err != nil { + return nil, err +} + +return accounts, err +} + +func (k Keeper) + +IterateAccountsBetween(ctx sdk.Context, start, end uint64) ([]authtypes.BaseAccount, error) { + // The collections.Range API offers a lot of capabilities + // like defining where the iteration starts or ends. + rng := new(collections.Range[uint64]). + StartInclusive(start). + EndExclusive(end). + Descending() + +iter, err := k.Accounts.Iterate(ctx, rng) + if err != nil { + return nil, err +} + +accounts, err := iter.Values() + if err != nil { + return nil, err +} + +return accounts, nil +} + +func (k Keeper) + +IterateAccounts(ctx sdk.Context, do func(id uint64, acc authtypes.BaseAccount) (stop bool)) + +error { + iter, err := k.Accounts.Iterate(ctx, nil) + if err != nil { + return err +} + +defer iter.Close() + for ; iter.Valid(); iter.Next() { + kv, err := iter.KeyValue() + if err != nil { + return err +} + if do(kv.Key, kv.Value) { + break +} + +} + +return nil +} +``` + +Let's analyze each method in the example and how it makes use of the `Iterate` and the returned `Iterator` API. + +#### GetAllAccounts + +In `GetAllAccounts` we pass to our `Iterate` a nil `Ranger`. This means that the returned `Iterator` will include +all the existing keys within the collection. + +Then we use the `Values` method from the returned `Iterator` API to collect all the values into a slice. + +`Iterator` offers other methods such as `Keys()` to collect only the keys and not the values and `KeyValues` to collect +all the keys and values. + +#### IterateAccountsBetween + +Here we make use of the `collections.Range` helper to specialize our range. +We make it start in a point through `StartInclusive` and end in the other with `EndExclusive`, then +we instruct it to report us results in reverse order through `Descending` + +Then we pass the range instruction to `Iterate` and get an `Iterator`, which will contain only the results +we specified in the range. + +Then we use again the `Values` method of the `Iterator` to collect all the results. + +`collections.Range` also offers a `Prefix` API which is not applicable to all keys types, +for example uint64 cannot be prefix because it is of constant size, but a `string` key +can be prefixed. + +#### IterateAccounts + +Here we showcase how to lazily collect values from an Iterator. + + + +`Keys/Values/KeyValues` fully consume and close the `Iterator`, here we need to explicitly do a `defer iterator.Close()` call. + + + +`Iterator` also exposes a `Value` and `Key` method to collect only the current value or key, if collecting both is not needed. + + + +For this `callback` pattern, collections expose a `Walk` API. + + + +## Composite keys + +So far we've worked only with simple keys, like `uint64`, the account address, etc. +There are some more complex cases in, which we need to deal with composite keys. + +A key is composite when it is composed of multiple keys, for example bank balances as stored as the composite key +`(AccAddress, string)` where the first part is the address holding the coins and the second part is the denom. + +Example, let's say address `BOB` holds `10atom,15osmo`, this is how it is stored in state: + +```javascript +(bob, atom) => 10 +(bob, osmos) => 15 +``` + +Now this allows to efficiently get a specific denom balance of an address, by simply `getting` `(address, denom)`, or getting all the balances +of an address by prefixing over `(address)`. + +Let's see now how we can work with composite keys using collections. + +### Example + +In our example we will showcase how we can use collections when we are dealing with balances, similar to bank, +a balance is a mapping between `(address, denom) => math.Int` the composite key in our case is `(address, denom)`. + +## Instantiation of a composite key collection + +```go expandable +package collections + +import ( + + "cosmossdk.io/collections" + "cosmossdk.io/math" + store "cosmossdk.io/core/store" + sdk "github.com/cosmos/cosmos-sdk/types" +) + +var BalancesPrefix = collections.NewPrefix(1) + +type Keeper struct { + Schema collections.Schema + Balances collections.Map[collections.Pair[sdk.AccAddress, string], math.Int] +} + +func NewKeeper(storeService store.KVStoreService) + +Keeper { + sb := collections.NewSchemaBuilder(storeService) + +return Keeper{ + Balances: collections.NewMap( + sb, BalancesPrefix, "balances", + collections.PairKeyCodec(sdk.AccAddressKey, collections.StringKey), + sdk.IntValue, + ), +} +} +``` + +#### The Map Key definition + +First of all we can see that in order to define a composite key of two elements we use the `collections.Pair` type: + +```go +collections.Map[collections.Pair[sdk.AccAddress, string], math.Int] +``` + +`collections.Pair` defines a key composed of two other keys, in our case the first part is `sdk.AccAddress`, the second +part is `string`. + +#### The Key Codec instantiation + +The arguments to instantiate are always the same, the only thing that changes is how we instantiate +the `KeyCodec`, since this key is composed of two keys we use `collections.PairKeyCodec`, which generates +a `KeyCodec` composed of two key codecs. The first one will encode the first part of the key, the second one will +encode the second part of the key. + +### Working with composite key collections + +Let's expand on the example we used before: + +```go expandable +var BalancesPrefix = collections.NewPrefix(1) + +type Keeper struct { + Schema collections.Schema + Balances collections.Map[collections.Pair[sdk.AccAddress, string], math.Int] +} + +func NewKeeper(storeService store.KVStoreService) + +Keeper { + sb := collections.NewSchemaBuilder(storeService) + +return Keeper{ + Balances: collections.NewMap( + sb, BalancesPrefix, "balances", + collections.PairKeyCodec(sdk.AccAddressKey, collections.StringKey), + sdk.IntValue, + ), +} +} + +func (k Keeper) + +SetBalance(ctx sdk.Context, address sdk.AccAddress, denom string, amount math.Int) + +error { + key := collections.Join(address, denom) + +return k.Balances.Set(ctx, key, amount) +} + +func (k Keeper) + +GetBalance(ctx sdk.Context, address sdk.AccAddress, denom string) (math.Int, error) { + return k.Balances.Get(ctx, collections.Join(address, denom)) +} + +func (k Keeper) + +GetAllAddressBalances(ctx sdk.Context, address sdk.AccAddress) (sdk.Coins, error) { + balances := sdk.NewCoins() + rng := collections.NewPrefixedPairRange[sdk.AccAddress, string](address) + +iter, err := k.Balances.Iterate(ctx, rng) + if err != nil { + return nil, err +} + +kvs, err := iter.KeyValues() + if err != nil { + return nil, err +} + for _, kv := range kvs { + balances = balances.Add(sdk.NewCoin(kv.Key.K2(), kv.Value)) +} + +return balances, nil +} + +func (k Keeper) + +GetAllAddressBalancesBetween(ctx sdk.Context, address sdk.AccAddress, startDenom, endDenom string) (sdk.Coins, error) { + rng := collections.NewPrefixedPairRange[sdk.AccAddress, string](address). + StartInclusive(startDenom). + EndInclusive(endDenom) + +iter, err := k.Balances.Iterate(ctx, rng) + if err != nil { + return nil, err +} + ... +} +``` + +#### SetBalance + +As we can see here we're setting the balance of an address for a specific denom. +We use the `collections.Join` function to generate the composite key. +`collections.Join` returns a `collections.Pair` (which is the key of our `collections.Map`) + +`collections.Pair` contains the two keys we have joined, it also exposes two methods: `K1` to fetch the 1st part of the +key and `K2` to fetch the second part. + +As always, we use the `collections.Map.Set` method to map the composite key to our value (`math.Int` in this case) + +#### GetBalance + +To get a value in composite key collection, we simply use `collections.Join` to compose the key. + +#### GetAllAddressBalances + +We use `collections.PrefixedPairRange` to iterate over all the keys starting with the provided address. +Concretely the iteration will report all the balances belonging to the provided address. + +The first part is that we instantiate a `PrefixedPairRange`, which is a `Ranger` implementer aimed to help +in `Pair` keys iterations. + +```go +rng := collections.NewPrefixedPairRange[sdk.AccAddress, string](address) +``` + +As we can see here we're passing the type parameters of the `collections.Pair` because golang type inference +with respect to generics is not as permissive as other languages, so we need to explicitly say what are the types of the pair key. + +#### GetAllAddressesBalancesBetween + +This showcases how we can further specialize our range to limit the results further, by specifying +the range between the second part of the key (in our case the denoms, which are strings). + +## IndexedMap + +`collections.IndexedMap` is a collection that uses under the hood a `collections.Map`, and has a struct, which contains the indexes that we need to define. + +### Example + +Let's say we have an `auth.BaseAccount` struct which looks like the following: + +```go +type BaseAccount struct { + AccountNumber uint64 `protobuf:"varint,3,opt,name=account_number,json=accountNumber,proto3" json:"account_number,omitempty"` + Sequence uint64 `protobuf:"varint,4,opt,name=sequence,proto3" json:"sequence,omitempty"` +} +``` + +First of all, when we save our accounts in state we map them using a primary key `sdk.AccAddress`. +If it were to be a `collections.Map` it would be `collections.Map[sdk.AccAddress, authtypes.BaseAccount]`. + +Then we also want to be able to get an account not only by its `sdk.AccAddress`, but also by its `AccountNumber`. + +So we can say we want to create an `Index` that maps our `BaseAccount` to its `AccountNumber`. + +We also know that this `Index` is unique. Unique means that there can only be one `BaseAccount` that maps to a specific +`AccountNumber`. + +First of all, we start by defining the object that contains our index: + +```go expandable +var AccountsNumberIndexPrefix = collections.NewPrefix(1) + +type AccountsIndexes struct { + Number *indexes.Unique[uint64, sdk.AccAddress, authtypes.BaseAccount] +} + +func NewAccountIndexes(sb *collections.SchemaBuilder) + +AccountsIndexes { + return AccountsIndexes{ + Number: indexes.NewUnique( + sb, AccountsNumberIndexPrefix, "accounts_by_number", + collections.Uint64Key, sdk.AccAddressKey, + func(_ sdk.AccAddress, v authtypes.BaseAccount) (uint64, error) { + return v.AccountNumber, nil +}, + ), +} +} +``` + +We create an `AccountIndexes` struct which contains a field: `Number`. This field represents our `AccountNumber` index. +`AccountNumber` is a field of `authtypes.BaseAccount` and it's a `uint64`. + +Then we can see in our `AccountIndexes` struct the `Number` field is defined as: + +```go +*indexes.Unique[uint64, sdk.AccAddress, authtypes.BaseAccount] +``` + +Where the first type parameter is `uint64`, which is the field type of our index. +The second type parameter is the primary key `sdk.AccAddress`. +And the third type parameter is the actual object we're storing `authtypes.BaseAccount`. + +Then we create a `NewAccountIndexes` function that instantiates and returns the `AccountsIndexes` struct. + +The function takes a `SchemaBuilder`. Then we instantiate our `indexes.Unique`, let's analyze the arguments we pass to +`indexes.NewUnique`. + +#### NOTE: indexes list + +The `AccountsIndexes` struct contains the indexes, the `NewIndexedMap` function will infer the indexes form that struct +using reflection, this happens only at init and is not computationally expensive. In case you want to explicitly declare +indexes: implement the `Indexes` interface in the `AccountsIndexes` struct: + +```go +func (a AccountsIndexes) + +IndexesList() []collections.Index[sdk.AccAddress, authtypes.BaseAccount] { + return []collections.Index[sdk.AccAddress, authtypes.BaseAccount]{ + a.Number +} +} +``` + +#### Instantiating a `indexes.Unique` + +The first three arguments, we already know them, they are: `SchemaBuilder`, `Prefix` which is our index prefix (the partition +where index keys relationship for the `Number` index will be maintained), and the human name for the `Number` index. + +The second argument is a `collections.Uint64Key` which is a key codec to deal with `uint64` keys, we pass that because +the key we're trying to index is a `uint64` key (the account number), and then we pass as fifth argument the primary key codec, +which in our case is `sdk.AccAddress` (remember: we're mapping `sdk.AccAddress` => `BaseAccount`). + +Then as last parameter we pass a function that: given the `BaseAccount` returns its `AccountNumber`. + +After this we can proceed instantiating our `IndexedMap`. + +```go expandable +var AccountsPrefix = collections.NewPrefix(0) + +type Keeper struct { + Schema collections.Schema + Accounts *collections.IndexedMap[sdk.AccAddress, authtypes.BaseAccount, AccountsIndexes] +} + +func NewKeeper(storeService store.KVStoreService, cdc codec.BinaryCodec) + +Keeper { + sb := collections.NewSchemaBuilder(storeService) + +return Keeper{ + Accounts: collections.NewIndexedMap( + sb, AccountsPrefix, "accounts", + sdk.AccAddressKey, codec.CollValue[authtypes.BaseAccount](cdc), + NewAccountIndexes(sb), + ), +} +} +``` + +As we can see here what we do, for now, is the same thing as we did for `collections.Map`. +We pass it the `SchemaBuilder`, the `Prefix` where we plan to store the mapping between `sdk.AccAddress` and `authtypes.BaseAccount`, +the human name and the respective `sdk.AccAddress` key codec and `authtypes.BaseAccount` value codec. + +Then we pass the instantiation of our `AccountIndexes` through `NewAccountIndexes`. + +Full example: + +```go expandable +package docs + +import ( + + "cosmossdk.io/collections" + "cosmossdk.io/collections/indexes" + store "cosmossdk.io/core/store" + "github.com/cosmos/cosmos-sdk/codec" + sdk "github.com/cosmos/cosmos-sdk/types" + authtypes "github.com/cosmos/cosmos-sdk/x/auth/types" +) + +var AccountsNumberIndexPrefix = collections.NewPrefix(1) + +type AccountsIndexes struct { + Number *indexes.Unique[uint64, sdk.AccAddress, authtypes.BaseAccount] +} + +func (a AccountsIndexes) + +IndexesList() []collections.Index[sdk.AccAddress, authtypes.BaseAccount] { + return []collections.Index[sdk.AccAddress, authtypes.BaseAccount]{ + a.Number +} +} + +func NewAccountIndexes(sb *collections.SchemaBuilder) + +AccountsIndexes { + return AccountsIndexes{ + Number: indexes.NewUnique( + sb, AccountsNumberIndexPrefix, "accounts_by_number", + collections.Uint64Key, sdk.AccAddressKey, + func(_ sdk.AccAddress, v authtypes.BaseAccount) (uint64, error) { + return v.AccountNumber, nil +}, + ), +} +} + +var AccountsPrefix = collections.NewPrefix(0) + +type Keeper struct { + Schema collections.Schema + Accounts *collections.IndexedMap[sdk.AccAddress, authtypes.BaseAccount, AccountsIndexes] +} + +func NewKeeper(storeService store.KVStoreService, cdc codec.BinaryCodec) + +Keeper { + sb := collections.NewSchemaBuilder(storeService) + +return Keeper{ + Accounts: collections.NewIndexedMap( + sb, AccountsPrefix, "accounts", + sdk.AccAddressKey, codec.CollValue[authtypes.BaseAccount](cdc), + NewAccountIndexes(sb), + ), +} +} +``` + +### Working with IndexedMaps + +While instantiating `collections.IndexedMap` is tedious, working with them is extremely smooth. + +Let's take the full example, and expand it with some use-cases. + +```go expandable +package docs + +import ( + + "cosmossdk.io/collections" + "cosmossdk.io/collections/indexes" + store "cosmossdk.io/core/store" + "github.com/cosmos/cosmos-sdk/codec" + sdk "github.com/cosmos/cosmos-sdk/types" + authtypes "github.com/cosmos/cosmos-sdk/x/auth/types" +) + +var AccountsNumberIndexPrefix = collections.NewPrefix(1) + +type AccountsIndexes struct { + Number *indexes.Unique[uint64, sdk.AccAddress, authtypes.BaseAccount] +} + +func (a AccountsIndexes) + +IndexesList() []collections.Index[sdk.AccAddress, authtypes.BaseAccount] { + return []collections.Index[sdk.AccAddress, authtypes.BaseAccount]{ + a.Number +} +} + +func NewAccountIndexes(sb *collections.SchemaBuilder) + +AccountsIndexes { + return AccountsIndexes{ + Number: indexes.NewUnique( + sb, AccountsNumberIndexPrefix, "accounts_by_number", + collections.Uint64Key, sdk.AccAddressKey, + func(_ sdk.AccAddress, v authtypes.BaseAccount) (uint64, error) { + return v.AccountNumber, nil +}, + ), +} +} + +var AccountsPrefix = collections.NewPrefix(0) + +type Keeper struct { + Schema collections.Schema + Accounts *collections.IndexedMap[sdk.AccAddress, authtypes.BaseAccount, AccountsIndexes] +} + +func NewKeeper(storeService store.KVStoreService, cdc codec.BinaryCodec) + +Keeper { + sb := collections.NewSchemaBuilder(storeService) + +return Keeper{ + Accounts: collections.NewIndexedMap( + sb, AccountsPrefix, "accounts", + sdk.AccAddressKey, codec.CollValue[authtypes.BaseAccount](cdc), + NewAccountIndexes(sb), + ), +} +} + +func (k Keeper) + +CreateAccount(ctx sdk.Context, addr sdk.AccAddress) + +error { + nextAccountNumber := k.getNextAccountNumber() + newAcc := authtypes.BaseAccount{ + AccountNumber: nextAccountNumber, + Sequence: 0, +} + +return k.Accounts.Set(ctx, addr, newAcc) +} + +func (k Keeper) + +RemoveAccount(ctx sdk.Context, addr sdk.AccAddress) + +error { + return k.Accounts.Remove(ctx, addr) +} + +func (k Keeper) + +GetAccountByNumber(ctx sdk.Context, accNumber uint64) (sdk.AccAddress, authtypes.BaseAccount, error) { + accAddress, err := k.Accounts.Indexes.Number.MatchExact(ctx, accNumber) + if err != nil { + return nil, authtypes.BaseAccount{ +}, err +} + +acc, err := k.Accounts.Get(ctx, accAddress) + +return accAddress, acc, nil +} + +func (k Keeper) + +GetAccountsByNumber(ctx sdk.Context, startAccNum, endAccNum uint64) ([]authtypes.BaseAccount, error) { + rng := new(collections.Range[uint64]). + StartInclusive(startAccNum). + EndInclusive(endAccNum) + +iter, err := k.Accounts.Indexes.Number.Iterate(ctx, rng) + if err != nil { + return nil, err +} + +return indexes.CollectValues(ctx, k.Accounts, iter) +} + +func (k Keeper) + +getNextAccountNumber() + +uint64 { + return 0 +} +``` + +## Collections with interfaces as values + +Although cosmos-sdk is shifting away from the usage of interface registry, there are still some places where it is used. +In order to support old code, we have to support collections with interface values. + +The generic `codec.CollValue` is not able to handle interface values, so we need to use a special type `codec.CollInterfaceValue`. +`codec.CollInterfaceValue` takes a `codec.BinaryCodec` as an argument, and uses it to marshal and unmarshal values as interfaces. +The `codec.CollInterfaceValue` lives in the `codec` package, whose import path is `github.com/cosmos/cosmos-sdk/codec`. + +### Instantiating Collections with interface values + +In order to instantiate a collection with interface values, we need to use `codec.CollInterfaceValue` instead of `codec.CollValue`. + +```go expandable +package example + +import ( + + "cosmossdk.io/collections" + store "cosmossdk.io/core/store" + "github.com/cosmos/cosmos-sdk/codec" + sdk "github.com/cosmos/cosmos-sdk/types" + authtypes "github.com/cosmos/cosmos-sdk/x/auth/types" +) + +var AccountsPrefix = collections.NewPrefix(0) + +type Keeper struct { + Schema collections.Schema + Accounts *collections.Map[sdk.AccAddress, sdk.AccountI] +} + +func NewKeeper(cdc codec.BinaryCodec, storeService store.KVStoreService) + +Keeper { + sb := collections.NewSchemaBuilder(storeService) + +return Keeper{ + Accounts: collections.NewMap( + sb, AccountsPrefix, "accounts", + sdk.AccAddressKey, codec.CollInterfaceValue[sdk.AccountI](cdc), + ), +} +} + +func (k Keeper) + +SaveBaseAccount(ctx sdk.Context, account authtypes.BaseAccount) + +error { + return k.Accounts.Set(ctx, account.GetAddress(), account) +} + +func (k Keeper) + +SaveModuleAccount(ctx sdk.Context, account authtypes.ModuleAccount) + +error { + return k.Accounts.Set(ctx, account.GetAddress(), account) +} + +func (k Keeper) + +GetAccount(ctx sdk.Context, addr sdk.AccAddress) (sdk.AccountI, error) { + return k.Accounts.Get(ctx, addr) +} +``` + +## Triple key + +The `collections.Triple` is a special type of key composed of three keys, it's identical to `collections.Pair`. + +Let's see an example. + +```go expandable +package example + +import ( + + "context" + "cosmossdk.io/collections" + store "cosmossdk.io/core/store" +) + +type AccAddress = string +type ValAddress = string + +type Keeper struct { + // let's simulate we have redelegations which are stored as a triple key composed of + // the delegator, the source validator and the destination validator. + Redelegations collections.KeySet[collections.Triple[AccAddress, ValAddress, ValAddress]] +} + +func NewKeeper(storeService store.KVStoreService) + +Keeper { + sb := collections.NewSchemaBuilder(storeService) + +return Keeper{ + Redelegations: collections.NewKeySet(sb, collections.NewPrefix(0), "redelegations", collections.TripleKeyCodec(collections.StringKey, collections.StringKey, collections.StringKey) +} +} + +// RedelegationsByDelegator iterates over all the redelegations of a given delegator and calls onResult providing +// each redelegation from source validator towards the destination validator. +func (k Keeper) + +RedelegationsByDelegator(ctx context.Context, delegator AccAddress, onResult func(src, dst ValAddress) (stop bool, err error)) + +error { + rng := collections.NewPrefixedTripleRange[AccAddress, ValAddress, ValAddress](delegator) + +return k.Redelegations.Walk(ctx, rng, func(key collections.Triple[AccAddress, ValAddress, ValAddress]) (stop bool, err error) { + return onResult(key.K2(), key.K3()) +}) +} + +// RedelegationsByDelegatorAndValidator iterates over all the redelegations of a given delegator and its source validator and calls onResult for each +// destination validator. +func (k Keeper) + +RedelegationsByDelegatorAndValidator(ctx context.Context, delegator AccAddress, validator ValAddress, onResult func(dst ValAddress) (stop bool, err error)) + +error { + rng := collections.NewSuperPrefixedTripleRange[AccAddress, ValAddress, ValAddress](delegator, validator) + +return k.Redelegations.Walk(ctx, rng, func(key collections.Triple[AccAddress, ValAddress, ValAddress]) (stop bool, err error) { + return onResult(key.K3()) +}) +} +``` + +## Advanced Usages + +### Alternative Value Codec + +The `codec.AltValueCodec` allows a collection to decode values using a different codec than the one used to encode them. +Basically it enables to decode two different byte representations of the same concrete value. +It can be used to lazily migrate values from one bytes representation to another, as long as the new representation is +not able to decode the old one. + +A concrete example can be found in `x/bank` where the balance was initially stored as `Coin` and then migrated to `Int`. + +```go +var BankBalanceValueCodec = codec.NewAltValueCodec(sdk.IntValue, func(b []byte) (sdk.Int, error) { + coin := sdk.Coin{ +} + err := coin.Unmarshal(b) + if err != nil { + return sdk.Int{ +}, err +} + +return coin.Amount, nil +}) +``` + +The above example shows how to create an `AltValueCodec` that can decode both `sdk.Int` and `sdk.Coin` values. The provided +decoder function will be used as a fallback in case the default decoder fails. When the value will be encoded back into state +it will use the default encoder. This allows to lazily migrate values to a new bytes representation. diff --git a/sdk/v0.54/guides/state/store.mdx b/sdk/v0.54/guides/state/store.mdx new file mode 100644 index 000000000..5f6a883d8 --- /dev/null +++ b/sdk/v0.54/guides/state/store.mdx @@ -0,0 +1,235 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/guides/state/store' +title: Module Store Internals +--- + +The store package defines the interfaces, types and abstractions for Cosmos SDK +modules to read and write to Merkleized state within a Cosmos SDK application. +The store package provides many primitives for developers to use in order to +work with both state storage and state commitment. Below we describe the various +abstractions. + +## Types + +### `Store` + +The bulk of the store interfaces are defined [here](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/store/types/store.go), +where the base primitive interface, for which other interfaces build off of, is +the `Store` type. The `Store` interface defines the ability to tell the type of +the implementing store and the ability to cache wrap via the `CacheWrapper` interface. + +### `CacheWrapper` & `CacheWrap` + +One of the most important features a store has the ability to perform is the +ability to cache wrap. Cache wrapping is essentially the underlying store wrapping +itself within another store type that performs caching for both reads and writes +with the ability to flush writes via `Write()`. + +### `KVStore` & `CacheKVStore` + +One of the most important interfaces that both developers and modules interface +with, which also provides the basis of most state storage and commitment operations, +is the `KVStore`. The `KVStore` interface provides basic CRUD abilities and +prefix-based iteration, including reverse iteration. + +Typically, each module has its own dedicated `KVStore` instance, which it can +get access to via the `sdk.Context` and the use of a pointer-based named key -- +`KVStoreKey`. The `KVStoreKey` provides pseudo-OCAP. How exactly a `KVStoreKey` +maps to a `KVStore` will be illustrated below through the `CommitMultiStore`. + +Note, a `KVStore` cannot directly commit state. Instead, a `KVStore` can be wrapped +by a `CacheKVStore` which extends a `KVStore` and provides the ability for the +caller to execute `Write()` which flushes pending writes to the parent `KVStore` in memory. +Note, this doesn't actually flush writes to disk as writes are held in memory +until `Commit()` is called on the `CommitMultiStore`. + +### `CommitMultiStore` + +The `CommitMultiStore` interface exposes the top-level interface that is used +to manage state commitment and storage by an SDK application and abstracts the +concept of multiple `KVStore`s which are used by multiple modules. Specifically, +it supports the following high-level primitives: + +* Allows for a caller to retrieve a `KVStore` by providing a `KVStoreKey`. +* Exposes pruning mechanisms to remove state pinned against a specific height/version + in the past. +* Allows for loading state storage at a particular height/version in the past to + provide current head and historical queries. +* Provides the ability to rollback state to a previous height/version. +* Provides the ability to load state storage at a particular height/version + while also performing store upgrades, which are used during live hard-fork + application state migrations. +* Provides the ability to commit all current accumulated state to disk and performs + Merkle commitment. + +## Implementation Details + +While there are many interfaces that the `store` package provides, there is +typically a core implementation for each main interface that modules and +developers interact with that are defined in the Cosmos SDK. + +### `iavl.Store` + +The `iavl.Store` provides the core implementation for state storage and commitment +by implementing the following interfaces: + +* `KVStore` +* `CommitStore` +* `CommitKVStore` +* `Queryable` +* `StoreWithInitialVersion` + +It allows for all CRUD operations to be performed along with allowing current +and historical state queries, prefix iteration, and state commitment along with +Merkle proof operations. The `iavl.Store` also provides the ability to remove +historical state from the state commitment layer. + +An overview of the IAVL implementation can be found [here](https://github.com/cosmos/iavl/blob/master/docs/overview.md). +It is important to note that the IAVL store provides both state commitment and +logical storage operations, which comes with drawbacks as there are various +performance impacts, some of which are very drastic, when it comes to the +operations mentioned above. + +When dealing with state management in modules and clients, the Cosmos SDK provides +various layers of abstractions or "store wrapping", where the `iavl.Store` is the +bottom most layer. When requesting a store to perform reads or writes in a module, +the typical abstraction layer in order is defined as follows: + +```text +rootmulti.Store -> cachemulti.Store -> gaskv.Store -> cachekv.Store -> iavl.Store +``` + +### Concurrent use of IAVL store + +The tree under `iavl.Store` is not safe for concurrent use. It is the +responsibility of the caller to ensure that concurrent access to the store is +not performed. + +The main issue with concurrent use is when data is written at the same time as +it's being iterated over. Doing so will cause an irrecoverable fatal error because +of concurrent reads and writes to an internal map. + +Although it's not recommended, you can iterate through values while writing to +it by disabling "FastNode" **without guarantees that the values being written will +be returned during the iteration** (if you need this, you might want to reconsider +the design of your application). This is done by setting `iavl-disable-fastnode` +to `true` in the config TOML file. + +### `cachekv.Store` + +The `cachekv.Store` store wraps an underlying `KVStore`, typically a `iavl.Store` +and contains an in-memory cache for storing pending writes to underlying `KVStore`. +`Set` and `Delete` calls are executed on the in-memory cache. `Has` checks the cache first, falling through to the underlying `KVStore` only on a cache miss. + +One of the most important calls to a `cachekv.Store` is `Write()`, which ensures +that key-value pairs are written to the underlying `KVStore` in a deterministic +and ordered manner by sorting the keys first. The store keeps track of "dirty" +keys and uses these to determine what keys to sort. Deletions are represented as zero-value (nil) entries; `Write()` detects these and calls `Delete` on the underlying `KVStore` for each one. + +The `cachekv.Store` also provides the ability to perform iteration and reverse +iteration. Iteration is performed through the `cacheMergeIterator` type and uses +both the dirty cache and underlying `KVStore` to iterate over key-value pairs. + +Note, all calls to CRUD and iteration operations on a `cachekv.Store` are thread-safe. + +### `gaskv.Store` + +The `gaskv.Store` store provides a simple implementation of a `KVStore`. +Specifically, it just wraps an existing `KVStore`, such as a cache-wrapped +`iavl.Store`, and incurs configurable gas costs for CRUD operations via +`ConsumeGas()` calls on a `GasMeter` passed at construction time, then proxies the underlying CRUD call to the wrapped store. + +### `cachemulti.Store` & `rootmulti.Store` + +The `rootmulti.Store` acts as an abstraction around a series of stores. Namely, +it implements the `CommitMultiStore` an `Queryable` interfaces. Through the +`rootmulti.Store`, an SDK module can request access to a `KVStore` to perform +state CRUD operations and queries by holding access to a unique `KVStoreKey`. + +The `rootmulti.Store` ensures these queries and state operations are performed +through cached-wrapped instances of `cachekv.Store` which is described above. The +`rootmulti.Store` implementation is also responsible for committing all accumulated +state from each `KVStore` to disk and returning an application state Merkle root. + +Queries can be performed to return state data along with associated state +commitment proofs for both previous heights/versions and the current state root. +Queries are routed based on store name, i.e. a module, along with other parameters defined in the SDK's `RequestQuery` type. + +The `rootmulti.Store` also provides primitives for pruning data at a given +height/version from state storage. When a height is committed, the `rootmulti.Store` +will determine if other previous heights should be considered for removal based +on the operator's pruning settings defined by `PruningOptions`, which defines +how many recent versions to keep on disk and the interval at which to remove +"staged" pruned heights from disk. During each interval, the staged heights are +removed from each `KVStore`. Note, it is up to the underlying `KVStore` +implementation to determine how pruning is actually performed. The `PruningOptions` +are defined as follows: + +```go +type PruningOptions struct { + // KeepRecent defines how many recent heights to keep on disk. + KeepRecent uint64 + + // Interval defines when the pruned heights are removed from disk. + Interval uint64 + + // Strategy defines the kind of pruning strategy. See below for more information on each. + Strategy PruningStrategy +} +``` + +The Cosmos SDK defines a preset number of pruning "strategies": `default`, `everything`, `nothing`, and `custom`. + +It is important to note that the `rootmulti.Store` considers each `KVStore` as a +separate logical store. In other words, they do not share a Merkle tree or +comparable data structure. This means that when state is committed via +`rootmulti.Store`, each store is committed in sequence and thus is not atomic. + +In terms of store construction and wiring, each Cosmos SDK application contains +a `BaseApp` instance which internally has a reference to a `CommitMultiStore` +that is implemented by a `rootmulti.Store`. The application then registers one or +more `KVStoreKey` that pertain to a unique module and thus a `KVStore`. Through +the use of an `sdk.Context` and a `KVStoreKey`, each module can get direct access +to it's respective `KVStore` instance. + +Example: + +```go expandable +func NewApp(...) + +Application { + // ... + bApp := baseapp.NewBaseApp(appName, logger, db, txConfig.TxDecoder(), baseAppOptions...) + +bApp.SetVersion(version.Version) + +bApp.SetInterfaceRegistry(interfaceRegistry) + + // ... + keys := sdk.NewKVStoreKeys(...) + transientKeys := sdk.NewTransientStoreKeys(...) + memKeys := sdk.NewMemoryStoreKeys(...) + + // ... + + // initialize stores + app.MountKVStores(keys) + +app.MountTransientStores(transientKeys) + +app.MountMemoryStores(memKeys) + + // ... +} +``` + +The `rootmulti.Store` itself can be cache-wrapped which returns an instance of a +`cachemulti.Store`. For each block, `BaseApp` ensures that the proper abstractions +are created on the `CommitMultiStore`, i.e. ensuring that the `rootmulti.Store` +is cached-wrapped and uses the resulting `cachemulti.Store` to be set on the +`sdk.Context` which is then used for block and transaction execution. As a result, +all state mutations due to block and transaction execution are actually held +ephemerally until `Commit()` is called by the ABCI client. This concept is further +expanded upon when the AnteHandler is executed per transaction to ensure state +is not committed for transactions that failed CheckTx. diff --git a/sdk/v0.54/guides/testing/log.mdx b/sdk/v0.54/guides/testing/log.mdx new file mode 100644 index 000000000..86cc31046 --- /dev/null +++ b/sdk/v0.54/guides/testing/log.mdx @@ -0,0 +1,204 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/guides/testing/log' +title: Log v2 +--- + +`cosmossdk.io/log/v2` is the Cosmos SDK logging package. + +At a high level, there are three pieces to understand: + +1. `log.NewLogger(...)` creates the default Cosmos SDK logger. It is backed by `zerolog`. +2. `cosmossdk.io/log/v2/slog` lets you satisfy the same SDK `Logger` interface with a standard library `*slog.Logger`. +3. `log.NewMultiLogger(...)` fans one log call out to multiple SDK loggers. The SDK uses this during server startup when OpenTelemetry log exporting is enabled. To learn more about how we support OpenTelemetry, read the [Telemetry docs](/sdk/v0.54/guides/testing/telemetry). + +If you only need ordinary SDK logging, you usually only need `log.NewLogger`, which is automatically provisioned and set on `sdk.Context`. + +## Default Logger + +The default implementation is a small wrapper around `zerolog`. + +```go +logger := log.NewLogger(os.Stderr) + +logger.Info("starting app", "chain_id", chainID) +logger.Error("failed to load state", "err", err) +``` + +`NewLogger` writes human-readable console output by default. The server command wiring switches options based on CLI configuration, for example: + +- `OutputJSONOption()` for JSON logs +- `LevelOption(...)` for a global log level +- `FilterOption(...)` for module-based filtering +- `TraceOption(true)` to include stack traces on error logs +- `VerboseLevelOption(...)` for temporary verbose mode + +The SDK also uses the `module` field consistently. The package exposes `log.ModuleKey` for this: + +```go +logger = logger.With(log.ModuleKey, "bank") +logger.Info("send coins", "from", from, "to", to) +``` + +That matters because the log filter implementation keys off the `module` field when parsing values such as `consensus:debug,*:error`. + +## Structured Context + +`Logger.With(...)` returns a derived logger with additional fields: + +```go +keeperLogger := logger.With(log.ModuleKey, "staking", "component", "keeper") +keeperLogger.Info("validator updated", "operator", valAddr) +``` + +This is the normal way to attach stable metadata to a logger instance. + +## Context-Aware Logging + +The v2 `Logger` interface adds `*Context` methods: + +```go +type Logger interface { + Info(msg string, keyVals ...any) + InfoContext(ctx context.Context, msg string, keyVals ...any) + Warn(msg string, keyVals ...any) + WarnContext(ctx context.Context, msg string, keyVals ...any) + Error(msg string, keyVals ...any) + ErrorContext(ctx context.Context, msg string, keyVals ...any) + Debug(msg string, keyVals ...any) + DebugContext(ctx context.Context, msg string, keyVals ...any) + With(keyVals ...any) Logger + Impl() any +} +``` + +The important distinction is: + +- `Info`, `Warn`, `Error`, and `Debug` log without inspecting a `context.Context` +- `InfoContext`, `WarnContext`, `ErrorContext`, and `DebugContext` use the provided context for trace correlation + +For the default `zerolog` implementation, the `*Context` methods extract the active OpenTelemetry span from `ctx` and add: + +- `trace_id` +- `span_id` +- `trace_flags` when present + +If there is no valid span in the context, they behave like normal log calls. + +## Trace Correlation + +When you want logs to line up with spans, use the context-aware methods. + +```go +func (k Keeper) UpdateBalance(ctx sdk.Context, addr sdk.AccAddress, coins sdk.Coins) error { + ctx, span := ctx.StartSpan(tracer, "UpdateBalance") + defer span.End() + + logger := ctx.Logger().With(log.ModuleKey, "bank") + logger.InfoContext(ctx, "updating balance", "address", addr.String()) + + return nil +} +``` + +Two details matter here: + +1. `sdk.Context.StartSpan(...)` returns a new `sdk.Context` with the Go `context.Context` updated to include the span. +2. The logger only sees trace information when you call one of the logger's `*Context` methods with that updated context. + +Without the `*Context` call, the default logger will not add trace fields to the log record. + +## `log/slog` + +`cosmossdk.io/log/v2/slog` is an adapter for code that already has a standard library `*slog.Logger`. + +```go +base := slog.New(handler) +logger := sdklogSlog.NewCustomLogger(base) +``` + +This does not add extra SDK behavior by itself. It simply makes a `*slog.Logger` satisfy the Cosmos SDK `Logger` interface. Filtering, formatting, sinks, and handler behavior are whatever the underlying `slog.Logger` is configured to do. + +## `MultiLogger` + +`log.NewMultiLogger(loggers...)` returns a logger that dispatches each log call to every wrapped logger. + +That includes: + +- ordinary log methods such as `Info(...)` +- context-aware methods such as `InfoContext(...)` +- `With(...)`, which derives a child logger for each wrapped logger + +If an underlying logger implements `VerboseModeLogger`, `SetVerboseMode(...)` is also forwarded. + +In other words, `MultiLogger` is just fanout. It does not merge records or add new fields on its own. + +## When The SDK Configures `MultiLogger` + +`MultiLogger` is not created for every app automatically. + +During the node's server start, the SDK first builds the normal server logger from CLI/config flags. That logger is the usual `zerolog`-backed logger. + +Then the SDK initializes OpenTelemetry from `config/otel.yaml`. If `telemetry.IsOtelLoggerEnabled()` reports that the global OpenTelemetry logger provider has active log processors/exporters, the SDK wraps the existing server logger like this: + +```go +otelLogger := sdkSlog.NewCustomLogger(otelslog.NewLogger("")) +svrCtx.Logger = log.NewMultiLogger(svrCtx.Logger, otelLogger) +``` + +So when OpenTelemetry log exporting is enabled, one log call is sent to: + +- the existing console/stdout logger +- an OpenTelemetry-backed logger for export + +If OpenTelemetry logging is not enabled, the server continues using only the normal logger. + +## What `otelslog` Is + +`otelslog` is an OpenTelemetry bridge for Go's `log/slog` package. + +More specifically, it provides a `slog.Handler` and `slog.Logger` that convert `slog.Record` values into OpenTelemetry log records and sends them to the configured OpenTelemetry logger provider. + +In the Cosmos SDK startup path: + +- `otelslog.NewLogger("")` creates an `*slog.Logger` backed by that bridge +- `cosmossdk.io/log/v2/slog.NewCustomLogger(...)` wraps it so it satisfies the SDK `Logger` interface +- `log.NewMultiLogger(...)` fans logs out to both the normal `zerolog` logger and the OpenTelemetry bridge + +Because `slog` has native `InfoContext`/`WarnContext`/`ErrorContext`/`DebugContext` methods, the `otelslog` side receives the context directly. That means trace/span correlation is handled by the OpenTelemetry logging pipeline without the SDK needing to manually inject `trace_id` fields into that branch. + +## Two Common Setups + +### 1. Stdout only + +If you do not configure an OpenTelemetry logger provider, logs only go to the normal SDK logger output. This does not restrict you from log correlation, however. + +For trace correlation in tools such as Grafana Tempo and Loki, you can: + +1. Emit JSON logs to stdout/stderr. +2. Scrape those logs with an agent such as the OpenTelemetry Collector filelog receiver. +3. Forward them to Loki. +4. Query by the `trace_id` field in the logs. + +Remember, `trace_id` is only injected into the log if a contextual method was called with a context that contains an active span. + +### 2. OpenTelemetry log exporter enabled + +If `otel.yaml` enables an OpenTelemetry log pipeline with real log processors/exporters, the SDK configures a `MultiLogger`. + +In that setup: + +- console logging still works as before +- logs are also exported through OpenTelemetry +- context-aware log calls carry trace context into the OpenTelemetry branch as well + +This is the path to use when you want the SDK to write logs directly into an OpenTelemetry logging backend, which eliminates the need to set up scraping infrastructure. + +## Future Direction + +Today the SDK uses a `MultiLogger` because the default logger is `zerolog`, while OpenTelemetry currently offers a bridge for `slog` rather than `zerolog`. + +If a first-class `zerolog` bridge becomes available and suitable, that would likely be a simpler export path than maintaining a separate fanout logger. Relevant discussion: + +- https://github.com/rs/zerolog/pull/682 +- https://github.com/open-telemetry/opentelemetry-go-contrib/issues/5969 \ No newline at end of file diff --git a/sdk/v0.54/guides/testing/simulator.mdx b/sdk/v0.54/guides/testing/simulator.mdx new file mode 100644 index 000000000..28a9ebe18 --- /dev/null +++ b/sdk/v0.54/guides/testing/simulator.mdx @@ -0,0 +1,278 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/guides/testing/simulator' +title: Module Simulation +--- + + +**Prerequisite Readings** + +* [Testing in the SDK](/sdk/v0.54/learn/concepts/testing) + + + +## Synopsis + +This document guides developers on integrating their custom modules with the Cosmos SDK `Simulations`. +Simulations are useful for testing edge cases in module implementations. + +* [Simulation Package](#simulation-package) +* [Simulation App Module](#simulation-app-module) +* [SimsX](#simsx) + * [Example Implementations](#example-implementations) +* [Store decoders](#store-decoders) +* [Randomized genesis](#randomized-genesis) +* [Random weighted operations](#random-weighted-operations) + * [Using Simsx](#using-simsx) +* [App Simulator manager](#app-simulator-manager) +* [Running Simulations](#running-simulations) + +## Simulation Package + +The Cosmos SDK suggests organizing your simulation related code in a `x//simulation` package. + +## Simulation App Module + +To integrate with the Cosmos SDK `SimulationManager`, app modules must implement the `AppModuleSimulation` interface. + +```go +// AppModuleSimulation defines the standard functions that every module should expose +// for the SDK blockchain simulator +type AppModuleSimulation interface { + // randomized genesis states + GenerateGenesisState(input *SimulationState) + + // register a func to decode the each module's defined types from their corresponding store key + RegisterStoreDecoder(simulation.StoreDecoderRegistry) + + // simulation operations (i.e msgs) with their respective weight + WeightedOperations(simState SimulationState) []simulation.WeightedOperation +} + +// HasProposalMsgs defines the messages that can be used to simulate governance (v1) proposals +type HasProposalMsgs interface { + // msg functions used to simulate governance proposals + ProposalMsgs(simState SimulationState) []simulation.WeightedProposalMsg +} +``` + +See the full source at [`types/module/simulation.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/types/module/simulation.go). + +See an example implementation of these methods from `x/distribution` [here](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/x/distribution/module.go#L170-L194). + +## SimsX + +Cosmos SDK v0.53.0 introduced a new package, `simsx`, providing improved DevX for writing simulation code. + +It exposes the following extension interfaces that modules may implement to integrate with the new `simsx` runner. + +```go +type ( + HasWeightedOperationsX interface { + WeightedOperationsX(weight WeightSource, reg Registry) + } + HasWeightedOperationsXWithProposals interface { + WeightedOperationsX(weights WeightSource, reg Registry, proposals WeightedProposalMsgIter, + legacyProposals []simtypes.WeightedProposalContent) + } + HasProposalMsgsX interface { + ProposalMsgsX(weights WeightSource, reg Registry) + } +) +``` + +See the full source at [`testutil/simsx/runner.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/testutil/simsx/runner.go). + +`SimMsgFactoryFn` is the default factory for most cases. It does not create future operations but ensures successful message delivery: + +```go +// SimMsgFactoryFn is the default factory for most cases. It does not create future operations but ensures successful message delivery. +type SimMsgFactoryFn[T sdk.Msg] func(ctx context.Context, testData *ChainDataSource, reporter SimulationReporter) (signer []SimAccount, msg T) +``` + +See the full source at [`testutil/simsx/msg_factory.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/testutil/simsx/msg_factory.go). + +These methods allow constructing randomized messages and/or proposal messages. + + +Note that modules should **not** implement both `HasWeightedOperationsX` and `HasWeightedOperationsXWithProposals`. +See the runner code [here](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/testutil/simsx/runner.go#L330-L339) for details + +If the module does **not** have message handlers or governance proposal handlers, these interface methods do **not** need to be implemented. + + +### Example Implementations + +* `HasWeightedOperationsXWithProposals`: [x/gov](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/x/gov/module.go#L242-L261) +* `HasWeightedOperationsX`: [x/bank](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/x/bank/module.go#L201-L205) +* `HasProposalMsgsX`: [x/bank](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/x/bank/module.go#L196-L199) + +## Store decoders + +Registering the store decoders is required for the `AppImportExport` simulation. This allows +for the key-value pairs from the stores to be decoded to their corresponding types. +In particular, it matches the key to a concrete type and then unmarshalls the value from the `KVPair` to the type provided. + +Modules using [collections](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/collections/README.md) can use the `NewStoreDecoderFuncFromCollectionsSchema` function that builds the decoder for you: + +```go +// RegisterStoreDecoder registers a decoder for supply module's types +func (am AppModule) RegisterStoreDecoder(sdr simtypes.StoreDecoderRegistry) { + sdr[types.StoreKey] = simtypes.NewStoreDecoderFuncFromCollectionsSchema(am.keeper.(keeper.BaseKeeper).Schema) +} +``` + +See the full source at [`types/simulation/collections.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/types/simulation/collections.go) and the bank module example at [`x/bank/module.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/x/bank/module.go#L183-L186). + +Modules not using collections must manually build the store decoder. +See the implementation [here](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/x/distribution/simulation/decoder.go) from the distribution module for an example. + +## Randomized genesis + +The simulator tests different scenarios and values for genesis parameters. +App modules must implement a `GenerateGenesisState` method to generate the initial random `GenesisState` from a given seed. + +See an example from `x/auth` [here](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/x/auth/module.go#L174-L177). + +Once the module's genesis parameters are generated randomly (or with the key and +values defined in a `params` file), they are marshaled to JSON format and added +to the app genesis JSON for the simulation. + +## Random weighted operations + +Operations are one of the crucial parts of the Cosmos SDK simulation. They are the transactions +(`Msg`) that are simulated with random field values. The sender of the operation +is also assigned randomly. + +Operations on the simulation are simulated using the full [transaction cycle](/sdk/v0.54/learn/concepts/lifecycle) of a +`ABCI` application that exposes the `BaseApp`. + +### Using Simsx + +Simsx introduces the ability to define a `MsgFactory` for each of a module's messages. + +These factories are registered in `WeightedOperationsX` and/or `ProposalMsgsX`. + +```go +// ProposalMsgsX registers governance proposal messages in the simulation registry. +func (AppModule) ProposalMsgsX(weights simsx.WeightSource, reg simsx.Registry) { + reg.Add(weights.Get("msg_update_params", 100), simulation.MsgUpdateParamsFactory()) +} + +// WeightedOperationsX registers weighted distribution module operations for simulation. +func (am AppModule) WeightedOperationsX(weights simsx.WeightSource, reg simsx.Registry) { + reg.Add(weights.Get("msg_set_withdraw_address", 50), simulation.MsgSetWithdrawAddressFactory(am.keeper)) + reg.Add(weights.Get("msg_withdraw_delegation_reward", 50), simulation.MsgWithdrawDelegatorRewardFactory(am.keeper, am.stakingKeeper)) + reg.Add(weights.Get("msg_withdraw_validator_commission", 50), simulation.MsgWithdrawValidatorCommissionFactory(am.keeper, am.stakingKeeper)) +} +``` + +Note that the name passed in to `weights.Get` must match the name of the operation set in the `WeightedOperations`. + +For example, if the module contains an operation `op_weight_msg_set_withdraw_address`, the name passed to `weights.Get` should be `msg_set_withdraw_address`. + +See the `x/distribution` for an example of implementing message factories [here](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/x/distribution/simulation/msg_factory.go) + +## App Simulator manager + +The following step is setting up the `SimulationManager` at the app level. This +is required for the simulation test files in the next step. + +```go +type CoolApp struct { + ... + sm *module.SimulationManager +} +``` + +Within the constructor of the application, construct the simulation manager using the modules from `ModuleManager` and call the `RegisterStoreDecoders` method. + +```go +overrideModules := map[string]module.AppModuleSimulation{ + authtypes.ModuleName: auth.NewAppModule(app.appCodec, app.AccountKeeper, authsims.RandomGenesisAccounts, nil), +} + +app.sm = module.NewSimulationManagerFromAppModules(app.ModuleManager.Modules, overrideModules) + +app.sm.RegisterStoreDecoders() +``` + +Note that you may override some modules. +This is useful if the existing module configuration in the `ModuleManager` should be different in the `SimulationManager`. + +Finally, the application should expose the `SimulationManager` via the following method defined in the `AppI` interface: + +```go +// SimulationManager implements the SimulationApp interface +func (app *SimApp) SimulationManager() *module.SimulationManager { + return app.sm +} +``` + +See the full simapp setup at [`simapp/app.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/simapp/app.go). + +## Running Simulations + +To run the simulation, use the `simsx` runner. + +Call `simsx.Run` to begin simulating with the default seeds, or `simsx.RunWithSeeds` to provide specific seeds: + +```go +func TestFullAppSimulation(t *testing.T) { + sims.Run(t, NewSimApp, setupStateFactory) +} + +func TestAppImportExport(t *testing.T) { + sims.Run(t, NewSimApp, setupStateFactory, func(tb testing.TB, ti sims.TestInstance[*SimApp], accs []simtypes.Account) { + // post-run assertions: export and compare stores + }) +} +``` + +These functions should be called in tests (i.e., `app_test.go`, `app_sim_test.go`, etc.). + +See the full simapp test file at [`simapp/sim_test.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/simapp/sim_test.go). + +### Simulation test types + +The simulation framework provides four test functions, each testing a different failure scenario: + +* `TestFullAppSimulation`: General simulation mode. Runs the chain and specified operations for a given number of blocks, checking for panics. +* `TestAppImportExport`: Exports the initial app state and creates a new app with the exported `genesis.json` as input, checking for store inconsistencies between the two. +* `TestAppSimulationAfterImport`: Chains two simulations -- the first provides its app state to the second. Useful for testing software upgrades or hard-forks from a live chain. +* `TestAppStateDeterminism`: Checks that all nodes return the same values in the same order. + +### Simulator modes + +Simulations run in three modes: + +1. **Fully random** -- initial state, module parameters, and simulation parameters are all pseudo-randomly generated. +2. **From a `genesis.json` file** -- initial state and module parameters are defined by the file. Useful for testing against a known state such as a live network export. +3. **From a `params.json` file** -- initial state is pseudo-randomly generated but module and simulation parameters are set manually. Available parameters are listed [here](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/x/simulation/client/cli/flags.go#L43-L70). + + +These modes are not mutually exclusive. For example, you can combine a randomly generated genesis state (mode 1) with manually defined simulation params (mode 3). + + +### Running via go test + +Simulations can be run directly with `go test`: + +```bash +go test -mod=readonly github.com/cosmos/cosmos-sdk/simapp \ + -run=TestApp \ + ... \ + -v -timeout 24h +``` + +The full list of available flags is defined [here](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/x/simulation/client/cli/flags.go#L43-L70). For Makefile examples, see the Cosmos SDK [`Makefile`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/Makefile#L280-L340). + +### Debugging tips + +When encountering a simulation failure: + +* **Export app state** at the failure height using the `-ExportStatePath` flag. +* **Use `-Verbose` logs** for a fuller picture of all operations involved. +* **Try a different `-Seed`**. If the same error reproduces sooner, you will spend less time on each run. +* **Reduce `-NumBlocks`** to isolate what the app state looks like at the block before failure. +* **Add a [`Logger`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/x/staking/keeper/keeper.go#L78-L82)** to operations that are not being logged. diff --git a/sdk/v0.54/guides/testing/telemetry.mdx b/sdk/v0.54/guides/testing/telemetry.mdx new file mode 100644 index 000000000..43989052f --- /dev/null +++ b/sdk/v0.54/guides/testing/telemetry.mdx @@ -0,0 +1,395 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/guides/testing/telemetry' +title: Telemetry +--- + + +Gather relevant insights about your application and modules with custom metrics and telemetry. + + +## Overview + +The `telemetry` package provides observability tooling for Cosmos SDK applications using [OpenTelemetry](https://opentelemetry.io/docs/). It offers a unified initialization point for traces, metrics, and logs via the OpenTelemetry declarative configuration API. + +This package: +- Initializes OpenTelemetry SDK using YAML configuration files +- Provides backward compatibility with Cosmos SDK's legacy `go-metrics` wrapper API +- Includes built-in instrumentation for host, runtime, and disk I/O metrics + +## Quick Start + +### 1. Start a Local Telemetry Backend + +```bash +docker run -p 3000:3000 -p 4317:4317 -p 4318:4318 --rm -ti grafana/otel-lgtm +``` + +### 2. Create Configuration File + +Create an `otel.yaml` file: + +```yaml +file_format: "1.0-rc.3" +resource: + attributes: + - name: service.name + value: my-cosmos-app + +tracer_provider: + processors: + - batch: + exporter: + otlp_grpc: + endpoint: http://localhost:4317 + +meter_provider: + readers: + - pull: + exporter: + prometheus/development: + host: 0.0.0.0 + port: 9464 + +logger_provider: + processors: + - batch: + exporter: + otlp_grpc: + endpoint: http://localhost:4317 + +extensions: + instruments: + host: {} + runtime: {} + diskio: {} + propagators: + - tracecontext +``` + +### 3. Initialize Telemetry + +**Option A: Environment Variable (Recommended)** + +Set `OTEL_EXPERIMENTAL_CONFIG_FILE` to your config path. This initializes the SDK before any meters/tracers are created, avoiding atomic load overhead. + +```bash +export OTEL_EXPERIMENTAL_CONFIG_FILE=/path/to/otel.yaml +``` + +**Option B: Node Config Directory** + +An empty `otel.yaml` will now be generated in `~/./config/`. Place the desired configuration in `otel.yaml`. + +**Option C: Programmatic Initialization** + +The SDK will first attempt to initialize via env var, then using the config in the node's home directory. +You may optionally initialize telemetry yourself using the `telemetry.InitializeOpenTelemetry` function: + +```go +err := telemetry.InitializeOpenTelemetry("/path/to/otel.yaml") +if err != nil { + log.Fatal(err) +} +defer telemetry.Shutdown(context.Background()) +``` + +## Configuration + +### OpenTelemetry Configuration + +The package uses the [OpenTelemetry declarative configuration spec](https://opentelemetry.io/docs/languages/sdk-configuration/declarative-configuration/). Key sections: + +| Section | Purpose | +|-------------------|---------------------------------| +| `resource` | Service identity and attributes | +| `tracer_provider` | Trace export configuration | +| `meter_provider` | Metrics export configuration | +| `logger_provider` | Log export configuration | + +For examples containing available options, see the [OpenTelemetry configuration examples](https://github.com/open-telemetry/opentelemetry-configuration/tree/main/examples). + +### Extensions + +The `extensions` section of the `otel.yaml` configuration file provides additional features not yet supported by the standard otelconf: + +```yaml +extensions: + # Optional file-based exporters + trace_file: "/path/to/traces.json" + metrics_file: "/path/to/metrics.json" + metrics_file_interval: "10s" + logs_file: "/path/to/logs.json" + + # Custom instrumentation additions + instruments: + host: {} + runtime: {} + diskio: + disable_virtual_device_filter: true # removes the automatic filtering of virtual disks. Operating systems such as Linux typically add virtual disks, which can add duplication to disk io data. These disks usually take the form of loopback, RAID, partitions, etc. + + # Trace context propagation + propagators: + - tracecontext + - baggage + - b3 + - jaeger +``` + +## Custom Instruments + +### Host Instrumentation (`host`) + +Reports host-level metrics using `go.opentelemetry.io/contrib/instrumentation/host`: +- CPU usage +- Memory usage +- Network I/O + +```yaml +extensions: + instruments: + host: {} +``` + +### Runtime Instrumentation (`runtime`) + +Reports Go runtime metrics using `go.opentelemetry.io/contrib/instrumentation/runtime`: +- Goroutine count +- GC statistics +- Memory allocations + +```yaml +extensions: + instruments: + runtime: {} +``` + +### Disk I/O Instrumentation (`diskio`) + +Reports disk I/O metrics using gopsutil: + +| Metric | Description | +|------------------------------|-------------------------------| +| `system.disk.io` | Bytes read/written | +| `system.disk.operations` | Read/write operation counts | +| `system.disk.io_time` | Time spent on I/O operations | +| `system.disk.operation_time` | Time per read/write operation | +| `system.disk.merged` | Merged read/write operations | + +```yaml +extensions: + instruments: + diskio: {} + # Or with options: + diskio: + disable_virtual_device_filter: true # Include loopback, RAID, partitions on Linux +``` + +By default, virtual devices (loopback, RAID, partitions) are filtered out on Linux to avoid double-counting I/O. + +## Propagators + +Configure trace context propagation for distributed tracing: + +| Propagator | Description | +|----------------|-----------------------------| +| `tracecontext` | W3C Trace Context (default) | +| `baggage` | W3C Baggage | +| `b3` | Zipkin B3 single header | +| `b3multi` | Zipkin B3 multi-header | +| `jaeger` | Jaeger propagation | + +## Developer Usage + +### Using Meters and Tracers + +After initialization, use standard OpenTelemetry APIs: + +```go +import ( + "go.opentelemetry.io/otel" + "go.opentelemetry.io/otel/metric" +) + +var ( + tracer = otel.Tracer("my-package") + meter = otel.Meter("my-package") + + myCounter metric.Int64Counter +) + +func init() { + var err error + myCounter, err = meter.Int64Counter("my.counter") + if err != nil { + panic(err) + } +} + +func MyFunction(ctx context.Context) error { + ctx, span := tracer.Start(ctx, "MyFunction") + defer span.End() + + myCounter.Add(ctx, 1) + + // ... your code + return nil +} +``` + +### Shutdown + +Always call `Shutdown()` when the application exits: + +```go +func (a *App) Close() { + telemetry.Shutdown(ctx) +} + +``` + +## Legacy API (Deprecated) + +The package provides backward-compatible wrappers for `github.com/hashicorp/go-metrics`. These are **deprecated** and users should migrate to OpenTelemetry APIs directly. + +### OpenTelemetry Bridge + +Cosmos SDK v0.54.0+ provides a bridge to send existing go-metrics to the meter provider defined in your OpenTelemetry config. +To bridge your metrics, set the `metrics-sink` in `app.toml` to "otel". + +```go + +############################################################################### +### Telemetry Configuration ### +############################################################################### +[telemetry] + +# other fields... + +metrics-sink = "otel" +``` + +### Legacy Configuration + +```go +cfg := telemetry.Config{ + ServiceName: "my-service", + Enabled: true, + EnableHostname: true, + EnableHostnameLabel: true, + EnableServiceLabel: true, + PrometheusRetentionTime: 60, // seconds + GlobalLabels: [][]string{{"chain_id", "cosmoshub-1"}}, + MetricsSink: "otel", // "mem", "statsd", "dogstatsd", "otel" + StatsdAddr: "localhost:8125", +} + +m, err := telemetry.New(cfg) +``` + +### Legacy Metrics Functions + +All are deprecated; prefer OpenTelemetry: + +```go +// Counters +telemetry.IncrCounter(1.0, "tx", "count") +telemetry.IncrCounterWithLabels([]string{"tx", "count"}, 1.0, labels) + +// Gauges +telemetry.SetGauge(42.0, "mempool", "size") +telemetry.SetGaugeWithLabels([]string{"mempool", "size"}, 42.0, labels) + +// Timing +start := telemetry.Now() +// ... operation +telemetry.MeasureSince(start, "tx", "process_time") + +// Module-specific helpers +telemetry.ModuleMeasureSince("bank", start, "send", "time") +telemetry.ModuleSetGauge("bank", 100.0, "balance", "total") +``` + +## Metrics Sink Types + +| Sink | Description | +|-------------|-----------------------------------------------------| +| `mem` | In-memory sink with SIGUSR1 dump support (default) | +| `statsd` | StatsD protocol | +| `dogstatsd` | Datadog DogStatsD | +| `otel` | OpenTelemetry (bridges to configured MeterProvider) | + +## Best Practices + +1. **Use environment variable initialization** for production to avoid atomic load overhead +2. **Always call `Shutdown()`** to ensure metrics/traces are flushed +3. **Thread `context.Context`** properly for correct span correlation + +## Viewing Telemetry Data + +With Grafana LGTM running: + +1. Open http://localhost:3000 +2. Use the Drilldown views to explore: + - **Traces**: Distributed trace visualization + - **Metrics**: Query and dashboard metrics + - **Logs**: Structured log search + +## Related Documentation + +- [OpenTelemetry Go SDK](https://opentelemetry.io/docs/languages/go/) +- [OpenTelemetry Configuration Spec](https://opentelemetry.io/docs/languages/sdk-configuration/declarative-configuration/) +- [otelconf Go Package](https://pkg.go.dev/go.opentelemetry.io/contrib/otelconf) + + +## Cosmos SDK Metrics + +The following metrics are emitted from the Cosmos SDK. + +| Metric | Description | Unit | Type | +| :------------------------------ | :---------------------------------------------------------------------------------------- | :-------------- | :------ | +| `tx_count` | Total number of txs processed via `FinalizeBlock` | tx | counter | +| `tx_successful` | Total number of successful txs processed via `FinalizeBlock` | tx | counter | +| `tx_failed` | Total number of failed txs processed via `FinalizeBlock` | tx | counter | +| `tx_gas_used` | The total amount of gas used by a tx | gas | gauge | +| `tx_gas_wanted` | The total amount of gas requested by a tx | gas | gauge | +| `tx_msg_send` | The total amount of tokens sent in a `MsgSend` (per denom) | token | gauge | +| `tx_msg_withdraw_reward` | The total amount of tokens withdrawn in a `MsgWithdrawDelegatorReward` (per denom) | token | gauge | +| `tx_msg_withdraw_commission` | The total amount of tokens withdrawn in a `MsgWithdrawValidatorCommission` (per denom) | token | gauge | +| `tx_msg_delegate` | The total amount of tokens delegated in a `MsgDelegate` | token | gauge | +| `tx_msg_begin_unbonding` | The total amount of tokens undelegated in a `MsgUndelegate` | token | gauge | +| `tx_msg_begin_begin_redelegate` | The total amount of tokens redelegated in a `MsgBeginRedelegate` | token | gauge | +| `tx_msg_ibc_transfer` | The total amount of tokens transferred via IBC in a `MsgTransfer` (source or sink chain) | token | gauge | +| `ibc_transfer_packet_receive` | The total amount of tokens received in a `FungibleTokenPacketData` (source or sink chain) | token | gauge | +| `new_account` | Total number of new accounts created | account | counter | +| `gov_proposal` | Total number of governance proposals | proposal | counter | +| `gov_vote` | Total number of governance votes for a proposal | vote | counter | +| `gov_deposit` | Total number of governance deposits for a proposal | deposit | counter | +| `staking_delegate` | Total number of delegations | delegation | counter | +| `staking_undelegate` | Total number of undelegations | undelegation | counter | +| `staking_redelegate` | Total number of redelegations | redelegation | counter | +| `ibc_transfer_send` | Total number of IBC transfers sent from a chain (source or sink) | transfer | counter | +| `ibc_transfer_receive` | Total number of IBC transfers received to a chain (source or sink) | transfer | counter | +| `ibc_client_create` | Total number of clients created | create | counter | +| `ibc_client_update` | Total number of client updates | update | counter | +| `ibc_client_upgrade` | Total number of client upgrades | upgrade | counter | +| `ibc_client_misbehaviour` | Total number of client misbehaviors | misbehaviour | counter | +| `ibc_connection_open-init` | Total number of connection `OpenInit` handshakes | handshake | counter | +| `ibc_connection_open-try` | Total number of connection `OpenTry` handshakes | handshake | counter | +| `ibc_connection_open-ack` | Total number of connection `OpenAck` handshakes | handshake | counter | +| `ibc_connection_open-confirm` | Total number of connection `OpenConfirm` handshakes | handshake | counter | +| `ibc_channel_open-init` | Total number of channel `OpenInit` handshakes | handshake | counter | +| `ibc_channel_open-try` | Total number of channel `OpenTry` handshakes | handshake | counter | +| `ibc_channel_open-ack` | Total number of channel `OpenAck` handshakes | handshake | counter | +| `ibc_channel_open-confirm` | Total number of channel `OpenConfirm` handshakes | handshake | counter | +| `ibc_channel_close-init` | Total number of channel `CloseInit` handshakes | handshake | counter | +| `ibc_channel_close-confirm` | Total number of channel `CloseConfirm` handshakes | handshake | counter | +| `tx_msg_ibc_recv_packet` | Total number of IBC packets received | packet | counter | +| `tx_msg_ibc_acknowledge_packet` | Total number of IBC packets acknowledged | acknowledgement | counter | +| `ibc_timeout_packet` | Total number of IBC timeout packets | timeout | counter | +| `store_iavl_get` | Duration of an IAVL `Store#Get` call | ms | summary | +| `store_iavl_set` | Duration of an IAVL `Store#Set` call | ms | summary | +| `store_iavl_has` | Duration of an IAVL `Store#Has` call | ms | summary | +| `store_iavl_delete` | Duration of an IAVL `Store#Delete` call | ms | summary | +| `store_iavl_commit` | Duration of an IAVL `Store#Commit` call | ms | summary | +| `store_iavl_query` | Duration of an IAVL `Store#Query` call | ms | summary | \ No newline at end of file diff --git a/sdk/v0.54/guides/tooling/autocli.mdx b/sdk/v0.54/guides/tooling/autocli.mdx new file mode 100644 index 000000000..9cdae750e --- /dev/null +++ b/sdk/v0.54/guides/tooling/autocli.mdx @@ -0,0 +1,336 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/guides/tooling/autocli' +title: Writing CLI Commands +--- + + + For a conceptual overview of how CLI, gRPC, and REST fit together in a Cosmos SDK app, see [CLI, gRPC & REST](/sdk/v0.54/learn/concepts/cli-grpc-rest). + + +## Overview + +`autocli` generates CLI commands and flags for each method defined in your gRPC service. By default, it generates a command for each gRPC service method. The commands are named based on the name of the service method. + +For example, given the following protobuf definition for a service: + +```protobuf +service MyService { + rpc MyMethod(MyRequest) returns (MyResponse) {} +} +``` + +The `autocli` package will generate a command named `my-method` for the `MyMethod` method. The command will have flags for each field in the `MyRequest` message. + +It is possible to customize the generation of transactions and queries by defining options for each service. + +## Application Wiring + +Here are the steps to use AutoCLI: + +1. Ensure your app's modules implement the `appmodule.AppModule` interface. +2. (optional) Configure how `autocli` behaves during command generation, by implementing the `func (am AppModule) AutoCLIOptions() *autocliv1.ModuleOptions` method on the module. +3. Call `app.AutoCliOpts()` to get an `autocli.AppOptions` populated from the module manager, then set `ClientCtx` on it to wire in the keyring. +4. Call `EnhanceRootCommand()` to add the generated CLI commands to your root command. + + +AutoCLI is additive only, meaning *enhancing* the root command will only add subcommands that are not already registered. This means that you can use AutoCLI alongside other custom commands within your app. + + +In practice this looks like (from the [example chain](https://github.com/cosmos/example/blob/main/exampled/cmd/root.go)): + +```go +autoCliOpts := app.AutoCliOpts() +autoCliOpts.ClientCtx = initClientCtx // wires keyring + node connection + +if err := autoCliOpts.EnhanceRootCommand(rootCmd); err != nil { + panic(err) +} +``` + +### Keyring + +AutoCLI resolves key names and signs transactions using the keyring from `client.Context`. At runtime, it reads the keyring from the command's live context (set by `SetCmdClientContextHandler` in `PersistentPreRunE` — see [Root Command Setup](#root-command-setup)) and adapts it to the [`cosmossdk.io/client/v2/autocli/keyring`](https://pkg.go.dev/cosmossdk.io/client/v2/autocli/keyring) interface via `keyring.NewAutoCLIKeyring` internally. + +If no keyring is provided, AutoCLI-generated commands can still query the chain but cannot sign transactions. + + +Because AutoCLI resolves key names from the keyring, you can use account names directly instead of addresses: + +```sh + q bank balances alice + tx bank send alice bob 1000denom +``` + + + +## Signing + +`autocli` supports signing transactions with the keyring. +The [`cosmos.msg.v1.signer` protobuf annotation](/sdk/v0.54/guides/reference/protobuf-annotations) defines the signer field of the message. +This field is automatically filled when using the `--from` flag or defining the signer as a positional argument. + + +AutoCLI currently supports only one signer per transaction. + + +## Module wiring & Customization + +The `AutoCLIOptions()` method on your module allows to specify custom commands, sub-commands or flags for each service, as it was a `cobra.Command` instance, within the `RpcCommandOptions` struct. Defining such options will customize the behavior of the `autocli` command generation, which by default generates a command for each method in your gRPC service. + +```go +autocliv1.RpcCommandOptions{ + RpcMethod: "Params", // The name of the gRPC service + Use: "params", // Command usage that is displayed in the help + Short: "Query the parameters of the governance process", // Short description of the command + Long: "Query the parameters of the governance process. Specify specific param types (voting|tallying|deposit) + +to filter results.", // Long description of the command + PositionalArgs: []*autocliv1.PositionalArgDescriptor{ + { + ProtoField: "params_type", + Optional: true +}, // Transform a flag into a positional argument +}, +} +``` + + +AutoCLI can create a gov proposal of any tx by simply setting the `GovProposal` field to `true` in the `autocliv1.RpcCommandOptions` struct. +Users can however use the `--no-proposal` flag to disable the proposal creation (which is useful if the authority isn't the gov module on a chain). + + +### Specifying Subcommands + +By default, `autocli` generates a command for each method in your gRPC service. However, you can specify subcommands to group related commands together. To specify subcommands, use the `autocliv1.ServiceCommandDescriptor` struct. + +For a real-world example, see the `gov` module's [`autocli.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/x/gov/autocli.go) in the Cosmos SDK. It demonstrates `ServiceCommandDescriptor` with `RpcCommandOptions`, `PositionalArgs`, `SubCommands`, `EnhanceCustomCommand`, and `GovProposal` all in one file. + +### Positional Arguments + +By default `autocli` generates a flag for each field in your protobuf message. However, you can choose to use positional arguments instead of flags for certain fields. + +To add positional arguments to a command, use the `autocliv1.PositionalArgDescriptor` struct, as seen in the example below. Specify the `ProtoField` parameter, which is the name of the protobuf field that should be used as the positional argument. In addition, if the parameter is a variable-length argument, you can specify the `Varargs` parameter as `true`. This can only be applied to the last positional parameter, and the `ProtoField` must be a repeated field. + +For a real-world example, see the `auth` module's [`autocli.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/x/auth/autocli.go) in the Cosmos SDK. It shows positional args wired for every query method, with `address` as a positional argument on the `Account` method. + +After wiring positional args, the command can be used as follows, instead of having to specify the `--address` flag: + +```bash + query auth account cosmos1abcd...xyz +``` + +#### Flattened Fields in Positional Arguments + +AutoCLI also supports flattening nested message fields as positional arguments. This means you can access nested fields +using dot notation in the `ProtoField` parameter. This is particularly useful when you want to directly set nested +message fields as positional arguments. + +For example, if you have a nested message structure like this: + +```protobuf +message Permissions { + string level = 1; + repeated string limit_type_urls = 2; +} + +message MsgAuthorizeCircuitBreaker { + string grantee = 1; + Permissions permissions = 2; +} +``` + +You can flatten the fields in your AutoCLI configuration: + +```go +{ + RpcMethod: "AuthorizeCircuitBreaker", + Use: "authorize [grantee] [level] [msg_type_urls]", + PositionalArgs: []*autocliv1.PositionalArgDescriptor{ + {ProtoField: "grantee"}, + {ProtoField: "permissions.level"}, + {ProtoField: "permissions.limit_type_urls", Varargs: true}, + }, +} +``` + +This allows users to provide values for nested fields directly as positional arguments: + +```bash + tx circuit authorize cosmos1... super-admin "/cosmos.bank.v1beta1.MsgSend" "/cosmos.bank.v1beta1.MsgMultiSend" +``` + +Instead of having to provide a complex JSON structure for nested fields, flattening makes the CLI more user-friendly by allowing direct access to nested fields. + +#### Customizing Flag Names + +By default, `autocli` generates flag names based on the names of the fields in your protobuf message. However, you can customize the flag names by providing a `FlagOptions`. This parameter allows you to specify custom names for flags based on the names of the message fields. + +For example, if you have a message with the fields `test` and `test1`, you can use the following naming options to customize the flags: + +```go +autocliv1.RpcCommandOptions{ + FlagOptions: map[string]*autocliv1.FlagOptions{ + "test": { + Name: "custom_name", +}, + "test1": { + Name: "other_name", +}, +}, +} +``` + +### Combining AutoCLI with Other Commands Within A Module + +AutoCLI can be used alongside other commands within a module. For example, the `gov` module uses AutoCLI for its query commands while also keeping hand-written tx commands for `submit-proposal`, `weighted-vote`, and similar. + +Set `EnhanceCustomCommand: true` on each `ServiceCommandDescriptor` where you want AutoCLI to add generated commands alongside existing ones: + +```go +func (am AppModule) AutoCLIOptions() *autocliv1.ModuleOptions { + return &autocliv1.ModuleOptions{ + Query: &autocliv1.ServiceCommandDescriptor{ + Service: govv1.Query_ServiceDesc.ServiceName, + EnhanceCustomCommand: true, // keep hand-written gov query commands + RpcCommandOptions: []*autocliv1.RpcCommandOptions{ /* ... */ }, + }, + Tx: &autocliv1.ServiceCommandDescriptor{ + Service: govv1.Msg_ServiceDesc.ServiceName, + EnhanceCustomCommand: true, // keep hand-written gov tx commands + }, + } +} +``` + +If `EnhanceCustomCommand` is not set to `true`, AutoCLI skips command generation for any service that already has commands registered via `GetTxCmd()` or `GetQueryCmd()`. + +### Skip a command + +AutoCLI checks the [`cosmos_proto.method_added_in` protobuf annotation](/sdk/v0.54/guides/reference/protobuf-annotations) and skips commands that were introduced in a newer SDK version than the one currently running. + +Additionally, a command can be manually skipped using the `autocliv1.RpcCommandOptions`: + +```go +autocliv1.RpcCommandOptions{ + RpcMethod: "Params", // The name of the gRPC method + Skip: true, +} +``` + +### Use AutoCLI for non module commands + +It is possible to use `AutoCLI` for non-module commands. The pattern is to add the options directly to `autoCliOpts.ModuleOptions` after calling `AutoCliOpts()`: + +```go +nodeCmds := nodeservice.NewNodeCommands() +autoCliOpts.ModuleOptions[nodeCmds.Name()] = nodeCmds.AutoCLIOptions() +``` + +`AutoCliOpts()` only picks up modules registered with the module manager — non-module commands always need to be added to `ModuleOptions` manually, as the example chain does with `nodeservice.NewNodeCommands()`. + +For a more complete example of this pattern, see [`client/grpc/cmtservice/autocli.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/client/grpc/cmtservice/autocli.go) and [`client/grpc/node/autocli.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/client/grpc/node/autocli.go) in the Cosmos SDK. + +## Root Command Setup + +For AutoCLI-generated commands (and hand-written commands) to work correctly — signing transactions, querying the chain, reading configuration — the root command must set up the `client.Context` and `server.Context` in a `PersistentPreRunE` function. This runs before every subcommand and makes both contexts available to all child commands. See [`simapp/simd/cmd/root.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/simapp/simd/cmd/root.go#L50-L93) for a complete example. + +The two key calls inside `PersistentPreRun` are: + +- `SetCmdClientContextHandler` reads persistent flags via `ReadPersistentCommandFlags`, creates a `client.Context`, and sets it on the command context. This is what AutoCLI and hand-written commands use to sign transactions and connect to a node. +- `InterceptConfigsPreRunHandler` creates the `server.Context`, loads `app.toml` and `config.toml` from the node home directory, and binds them to the server context's viper instance. This is what makes application configuration available at startup. + +### Custom logger + +By default, `InterceptConfigsPreRunHandler` sets the default SDK logger. To use a custom logger, use `InterceptConfigsAndCreateContext` instead and set the logger manually: + +```diff expandable +-return server.InterceptConfigsPreRunHandler(cmd, customAppTemplate, customAppConfig, customCMTConfig) + ++serverCtx, err := server.InterceptConfigsAndCreateContext(cmd, customAppTemplate, customAppConfig, customCMTConfig) ++if err != nil { ++ return err ++} + ++// overwrite default server logger ++logger, err := server.CreateSDKLogger(serverCtx, cmd.OutOrStdout()) ++if err != nil { ++ return err ++} ++serverCtx.Logger = logger.With(log.ModuleKey, "server") + ++// set server context ++return server.SetCmdServerContext(cmd, serverCtx) +``` + +## Environment Variables + +Every CLI flag is automatically bound to an environment variable. The variable name is the app's `basename` in uppercase followed by the flag name, with `-` replaced by `_`. For example, `--node` for an app with basename `GAIA` binds to `GAIA_NODE`. + +This lets you pre-configure common flags instead of passing them on every command: + +```shell +# set once in .env or shell profile +GAIA_HOME= +GAIA_NODE= +GAIA_CHAIN_ID="cosmoshub-4" +GAIA_KEYRING_BACKEND="test" + +# then just run +gaiad tx bank send alice bob 1000uatom --fees 500uatom +``` + +## Hand-Written Commands + +AutoCLI covers the standard case: one protobuf RPC method maps to one CLI command. For commands that don't fit that model, you can write Cobra commands manually and combine them with AutoCLI using `EnhanceCustomCommand: true`. + +Common reasons to write a command manually: + +- **Complex argument parsing** — multiple positional args that require custom validation or coin parsing before the message is built +- **Commands that span multiple RPC calls** — e.g., building a transaction from inputs that require a preceding query +- **Non-standard UX** — interactive prompts, offline signing flows, or commands that generate output rather than broadcast + +### Pattern + +A manual transaction command uses `client.GetClientTxContext` to retrieve the signing context, constructs a message, and passes it to `tx.GenerateOrBroadcastTxCLI`: + +```go +func NewSendTxCmd(ac address.Codec) *cobra.Command { + cmd := &cobra.Command{ + Use: "send [from_key_or_address] [to_address] [amount]", + Short: "Send tokens from one account to another", + Args: cobra.ExactArgs(3), + RunE: func(cmd *cobra.Command, args []string) error { + // set --from from the first positional arg + if err := cmd.Flags().Set(flags.FlagFrom, args[0]); err != nil { + return err + } + clientCtx, err := client.GetClientTxContext(cmd) + if err != nil { + return err + } + + toAddr, err := ac.StringToBytes(args[1]) + if err != nil { + return err + } + + coins, err := sdk.ParseCoinsNormalized(args[2]) + if err != nil { + return err + } + + msg := types.NewMsgSend(clientCtx.GetFromAddress(), toAddr, coins) + return tx.GenerateOrBroadcastTxCLI(clientCtx, cmd.Flags(), msg) + }, + } + + flags.AddTxFlagsToCmd(cmd) + return cmd +} +``` + +Key elements: +- `client.GetClientTxContext(cmd)` retrieves the client context (signer, node connection, codec) +- `flags.AddTxFlagsToCmd(cmd)` adds standard transaction flags (`--from`, `--fees`, `--gas`, etc.) +- `tx.GenerateOrBroadcastTxCLI` handles both `--generate-only` (offline) and live broadcast modes diff --git a/sdk/v0.54/guides/tooling/confix.mdx b/sdk/v0.54/guides/tooling/confix.mdx new file mode 100644 index 000000000..5ab24aa01 --- /dev/null +++ b/sdk/v0.54/guides/tooling/confix.mdx @@ -0,0 +1,143 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/guides/tooling/confix' +title: Confix +description: >- + Confix is a configuration management tool that allows you to manage your + configuration via CLI. +--- + +`Confix` is a configuration management tool that allows you to manage your configuration via CLI. + +It is based on the [CometBFT RFC 019](https://github.com/cometbft/cometbft/blob/5013bc3f4a6d64dcc2bf02ccc002ebc9881c62e4/docs/rfc/rfc-019-config-version.md). + +## Installation + +### Add Config Command + +To add the confix tool, it's required to add the `ConfigCommand` to your application's root command file (e.g. `/cmd/root.go`). + +Import the `confixCmd` package: + +```go +import confixcmd "cosmossdk.io/tools/confix/cmd" +``` + +Inside your `initRootCmd` function, add the command to the root: + +```go +rootCmd.AddCommand( + confixcmd.ConfigCommand(), +) +``` + +The `ConfigCommand` function builds the `config` root command and is defined in the `confixcmd` package (`cosmossdk.io/tools/confix/cmd`). +An implementation example can be found in `simapp`. + +The command will be available as `simd config`. + + +Using confix directly in the application can have less features than using it standalone. +This is because confix is versioned with the SDK, while `latest` is the standalone version. + + +### Using Confix Standalone + +To use Confix standalone, without having to add it in your application, install it with the following command: + +```bash +go install cosmossdk.io/tools/confix/cmd/confix@latest +``` + +Alternatively, for building from source, simply run `make confix`. The binary will be located in `tools/confix`. + +## Usage + +Use standalone: + +```shell +confix --help +``` + +Use in simd: + +```shell +simd config --help +``` + +### Get + +Get a configuration value, e.g.: + +```shell +simd config get app pruning # gets the value pruning from app.toml +simd config get client chain-id # gets the value chain-id from client.toml +``` + +```shell +confix get ~/.simapp/config/app.toml pruning # gets the value pruning from app.toml +confix get ~/.simapp/config/client.toml chain-id # gets the value chain-id from client.toml +``` + +### Set + +Set a configuration value, e.g.: + +```shell +simd config set app pruning "enabled" # sets the value pruning from app.toml +simd config set client chain-id "foo-1" # sets the value chain-id from client.toml +``` + +```shell +confix set ~/.simapp/config/app.toml pruning "enabled" # sets the value pruning from app.toml +confix set ~/.simapp/config/client.toml chain-id "foo-1" # sets the value chain-id from client.toml +``` + +### Migrate + +Migrate a configuration file to a new version, config type defaults to `app.toml`, if you want to change it to `client.toml`, please indicate it by adding the optional parameter, e.g.: + +```shell +simd config migrate v0.53 # migrates defaultHome/config/app.toml to the latest v0.53 config +simd config migrate v0.53 --client # migrates defaultHome/config/client.toml to the latest v0.53 config +``` + +```shell +confix migrate v0.53 ~/.simapp/config/app.toml # migrate ~/.simapp/config/app.toml to the latest v0.53 config +confix migrate v0.53 ~/.simapp/config/client.toml --client # migrate ~/.simapp/config/client.toml to the latest v0.53 config +``` + +### Diff + +Get the diff between a given configuration file and the default configuration file, e.g.: + +```shell +simd config diff v0.53 # gets the diff between defaultHome/config/app.toml and the latest v0.53 config +simd config diff v0.53 --client # gets the diff between defaultHome/config/client.toml and the latest v0.53 config +``` + +```shell +confix diff v0.53 ~/.simapp/config/app.toml # gets the diff between ~/.simapp/config/app.toml and the latest v0.53 config +confix diff v0.53 ~/.simapp/config/client.toml --client # gets the diff between ~/.simapp/config/client.toml and the latest v0.53 config +``` + +### View + +View a configuration file, e.g: + +```shell +simd config view client # views the current app client config +``` + +```shell +confix view ~/.simapp/config/client.toml # views the current app client config +``` + +### Maintainer + +At each SDK modification of the default configuration, add the default SDK config under `data/vXX-app.toml`. +This allows users to use the tool standalone. + +## Credits + +This project is based on the [CometBFT RFC 019](https://github.com/cometbft/cometbft/blob/5013bc3f4a6d64dcc2bf02ccc002ebc9881c62e4/docs/rfc/rfc-019-config-version.md) and their never released own implementation of [confix](https://github.com/cometbft/cometbft/blob/v0.36.x/scripts/confix/confix.go). diff --git a/sdk/v0.54/guides/tooling/tool-guide.mdx b/sdk/v0.54/guides/tooling/tool-guide.mdx new file mode 100644 index 000000000..2d2c4b5d7 --- /dev/null +++ b/sdk/v0.54/guides/tooling/tool-guide.mdx @@ -0,0 +1,66 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/guides/tooling/tool-guide' +title: Tool Guide +description: What tools should I use and for what? A practical guide to the Cosmos SDK developer toolbox. +--- + +A practical reference for Cosmos chain and module developers: what each tool does and when to reach for it. + +## Code generation + +**[Buf](https://buf.build/cosmos/cosmos-sdk/docs/main)** — Compiles `.proto` files into Go types, gRPC stubs, and REST gateway code. The standard way to run `proto-gen` in a Cosmos project. Also lints and formats proto files, and publishes generated docs to the Buf registry. See the [Protobuf Documentation](https://buf.build/cosmos/cosmos-sdk/docs/main) on the Buf registry. + +**[Protobuf Annotations](/sdk/v0.54/guides/reference/protobuf-annotations)** — Cosmos SDK-specific proto field options (scalar descriptors, amino names, query pagination, etc.) that affect code generation output. Consult this when writing `.proto` files for a new module. + +**[AutoCLI](/sdk/v0.54/guides/tooling/autocli)** — Generates CLI commands and gRPC-gateway routes for your module's messages and queries directly from proto definitions. Use it instead of hand-writing CLI commands — it also handles pagination, output formatting, and custom flag mappings. + +## Client library + +**[CosmJS](https://github.com/cosmos/cosmjs)** — The official JavaScript and TypeScript library for building clients, frontends, and scripts that interact with Cosmos chains. Handles transaction signing, broadcasting, querying, and wallet integration in browser and Node.js environments. + + +## State management + +**[Collections](/sdk/v0.54/guides/state/collections)** — A typed abstraction over raw `KVStore` access. Handles key encoding, prefix isolation, iteration, and secondary indexes. Also produces a schema used automatically by simulation decoders. Use it for all new module state instead of raw byte keys. + +**[Store](/sdk/v0.54/guides/state/store)** — Reference documentation for the SDK store layer: `KVStore`, `CommitMultiStore`, `CacheKVStore`, IAVL, pruning strategies, and store versioning. Read this when you need to understand what is happening under the collections abstraction or need to work with stores directly. + +## Testing + +**[Testing](/sdk/v0.54/learn/concepts/testing)** — The SDK's testing conventions: unit tests for keepers and message servers, integration tests wired with `depinject`, and end-to-end tests using the `testnet` package. + +**[Module Simulation](/sdk/v0.54/guides/testing/simulator)** — A fuzz-testing framework that runs your module's messages with randomized inputs and genesis states. Checks for panics, non-determinism, and import/export inconsistencies. Use it to catch edge cases that unit tests miss. + +## Node setup and operations + +**[Prerequisites](/sdk/v0.54/node/prerequisites)** — Required software and environment setup before running a node. + +**[Run a Node](/sdk/v0.54/node/run-node)** — How to initialize a chain, configure genesis, and start a node with `simd`. + +**[Run a Testnet](/sdk/v0.54/node/run-testnet)** — Running a local multi-node testnet using `simd testnet`. + +**[Production Deployment](/sdk/v0.54/node/run-production)** — Hardening and deployment guidance for running a node in production: systemd, state sync, backup strategies, and security considerations. + +**[Cosmovisor](/sdk/v0.54/guides/upgrades/cosmovisor)** — A process manager for your chain binary that watches for on-chain upgrade proposals and automatically swaps in the new binary at the correct upgrade height. Required for zero-downtime upgrades in production. + +**[Confix](/sdk/v0.54/guides/tooling/confix)** — A CLI tool for reading, setting, migrating, and diffing `app.toml` and `client.toml` configuration files across SDK versions. Use it when upgrading a node between SDK versions or scripting config changes. + +## Keys and transactions + +**[Keyring](/sdk/v0.54/node/keyring)** — The SDK's key management layer. Covers keyring backends (`os`, `file`, `test`, `memory`), key types, and how to manage keys via `simd keys`. Use this to understand key storage security trade-offs in production deployments. + +**[Building Transactions](/sdk/v0.54/node/txs)** — How to programmatically construct, sign, encode, and broadcast transactions using the SDK's `TxBuilder` and `TxConfig` APIs. + +**[Interacting with a Node](/sdk/v0.54/node/interact-node)** — Using the CLI and gRPC to query state and broadcast transactions against a running node. + +## Observability + +**[Telemetry](/sdk/v0.54/guides/testing/telemetry)** — OpenTelemetry-based metrics for the SDK and your modules. Emit counters, gauges, and histograms from keeper methods. Integrates with Prometheus and any OTLP-compatible backend. + +**[Logging](/sdk/v0.54/guides/testing/log)** — Structured logging via `cosmossdk.io/log` (backed by zerolog). Use it in keepers and servers to emit structured log lines, with support for log correlation and OpenTelemetry log export. + +## IBC + +**[IBC Go](/ibc)** — The canonical IBC implementation for Cosmos SDK chains. Use it to add cross-chain token transfers and arbitrary message passing to your chain. + diff --git a/sdk/v0.54/guides/upgrades/cosmovisor.mdx b/sdk/v0.54/guides/upgrades/cosmovisor.mdx new file mode 100644 index 000000000..b7199805d --- /dev/null +++ b/sdk/v0.54/guides/upgrades/cosmovisor.mdx @@ -0,0 +1,458 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/guides/upgrades/cosmovisor' +title: Cosmovisor +--- + +`cosmovisor` is a process manager for Cosmos SDK application binaries that automates application binary switch at chain upgrades. +It polls the `upgrade-info.json` file that is created by the x/upgrade module at upgrade height, and then can automatically download the new binary, stop the current binary, switch from the old binary to the new one, and finally restart the node with the new binary. + +* [Design](#design) +* [Contributing](#contributing) +* [Setup](#setup) + * [Installation](#installation) + * [Command Line Arguments And Environment Variables](#command-line-arguments-and-environment-variables) + * [Folder Layout](#folder-layout) +* [Usage](#usage) + * [Initialization](#initialization) + * [Detecting Upgrades](#detecting-upgrades) + * [Adding Upgrade Binary](#adding-upgrade-binary) + * [Auto-Download](#auto-download) + * [Preparing for an Upgrade](#preparing-for-an-upgrade) +* [Example: SimApp Upgrade](#example-simapp-upgrade) + * [Chain Setup](#chain-setup) + * [Prepare Cosmovisor and Start the Chain](#prepare-cosmovisor-and-start-the-chain) + * [Update App](#update-app) + +## Design + +Cosmovisor is designed to be used as a wrapper for a `Cosmos SDK` app: + +* it will pass arguments to the associated app (configured by `DAEMON_NAME` env variable). + Running `cosmovisor run arg1 arg2 ....` will run `app arg1 arg2 ...`; +* it will manage an app by restarting and upgrading if needed; +* it is configured using environment variables, not positional arguments. + +*Note: If new versions of the application are not set up to run in-place store migrations, migrations will need to be run manually before restarting `cosmovisor` with the new binary. For this reason, we recommend applications adopt in-place store migrations.* + + +Only the latest version of cosmovisor is actively developed/maintained. + + + +Versions prior to v1.0.0 have a vulnerability that could lead to a DOS. Please upgrade to the latest version. + + +## Contributing + +Cosmovisor is part of the Cosmos SDK monorepo, but it's a separate module with its own release schedule. + +Release branches have the following format `release/cosmovisor/vA.B.x`, where A and B are a number (e.g. `release/cosmovisor/v1.3.x`). Releases are tagged using the following format: `cosmovisor/vA.B.C`. + +## Setup + +### Installation + +You can download Cosmovisor from the [GitHub releases](https://github.com/cosmos/cosmos-sdk/releases/tag/cosmovisor%2Fv1.5.0). + +To install the latest version of `cosmovisor`, run the following command: + +```shell +go install cosmossdk.io/tools/cosmovisor/cmd/cosmovisor@latest +``` + +To install a specific version, you can specify the version: + +```shell +go install cosmossdk.io/tools/cosmovisor/cmd/cosmovisor@v1.5.0 +``` + +Run `cosmovisor version` to check the cosmovisor version. + +Alternatively, for building from source, simply run `make cosmovisor`. The binary will be located in `tools/cosmovisor`. + + +Installing cosmovisor using `go install` will display the correct `cosmovisor` version. +Building from source (`make cosmovisor`) or installing `cosmovisor` by other means won't display the correct version. + + +### Command Line Arguments And Environment Variables + +The first argument passed to `cosmovisor` is the action for `cosmovisor` to take. Options are: + +* `help`, `--help`, or `-h` - Output `cosmovisor` help information and check your `cosmovisor` configuration. +* `run` - Run the configured binary using the rest of the provided arguments. +* `version` - Output the `cosmovisor` version and also run the binary with the `version` argument. +* `config` - Display the current `cosmovisor` configuration, that means displaying the environment variables value that `cosmovisor` is using. +* `add-upgrade` - Add an upgrade manually to `cosmovisor`. This command allow you to easily add the binary corresponding to an upgrade in cosmovisor. +* `add-batch-upgrade` - Add multiple upgrades at once. +* `show-upgrade-info` - Show the current upgrade info from the upgrade-info.json file. + +All arguments passed to `cosmovisor run` will be passed to the application binary (as a subprocess). `cosmovisor` will return `/dev/stdout` and `/dev/stderr` of the subprocess as its own. For this reason, `cosmovisor run` cannot accept any command-line arguments other than those available to the application binary. + +`cosmovisor` reads its configuration from environment variables, or its configuration file (use `--cosmovisor-config `): + +* `DAEMON_HOME` is the location where the `cosmovisor/` directory is kept that contains the genesis binary, the upgrade binaries, and any additional auxiliary files associated with each binary (e.g. `$HOME/.gaiad`, `$HOME/.regend`, `$HOME/.simd`, etc.). +* `DAEMON_NAME` is the name of the binary itself (e.g. `gaiad`, `regend`, `simd`, etc.). +* `DAEMON_ALLOW_DOWNLOAD_BINARIES` (*optional*), if set to `true`, will enable auto-downloading of new binaries (for security reasons, this is intended for full nodes rather than validators). By default, `cosmovisor` will not auto-download new binaries. +* `DAEMON_DOWNLOAD_MUST_HAVE_CHECKSUM` (*optional*, default = `false`), if `true` cosmovisor will require that a checksum is provided in the upgrade plan for the binary to be downloaded. If `false`, cosmovisor will not require a checksum to be provided, but still check the checksum if one is provided. +* `DAEMON_RESTART_AFTER_UPGRADE` (*optional*, default = `true`), if `true`, restarts the subprocess with the same command-line arguments and flags (but with the new binary) after a successful upgrade. Otherwise (`false`), `cosmovisor` stops running after an upgrade and requires the system administrator to manually restart it. Note restart is only after the upgrade and does not auto-restart the subprocess after an error occurs. +* `DAEMON_RESTART_DELAY` (*optional*, default none), allow a node operator to define a delay between the node halt (for upgrade) and backup by the specified time. The value must be a duration (e.g. `1s`). +* `DAEMON_SHUTDOWN_GRACE` (*optional*, default none), if set, send interrupt to binary and wait the specified time to allow for cleanup/cache flush to disk before sending the kill signal. The value must be a duration (e.g. `1s`). +* `DAEMON_POLL_INTERVAL` (*optional*, default 300 milliseconds), is the interval length for polling the upgrade plan file. The value must be a duration (e.g. `1s`). +* `DAEMON_DATA_BACKUP_DIR` option to set a custom backup directory. If not set, `DAEMON_HOME` is used. +* `UNSAFE_SKIP_BACKUP` (defaults to `false`), if set to `true`, upgrades directly without performing a backup. Otherwise (`false`, default) backs up the data before trying the upgrade. The default value of false is useful and recommended in case of failures and when a backup needed to rollback. We recommend using the default backup option `UNSAFE_SKIP_BACKUP=false`. +* `DAEMON_PREUPGRADE_MAX_RETRIES` (defaults to `0`). The maximum number of times to retry [`pre-upgrade`](#pre-upgrade-handling) after exit status of `31`. With the default of `0`, a single exit-31 result immediately fails the upgrade. After retries are exhausted, Cosmovisor fails the upgrade. +* `DAEMON_GRPC_ADDRESS` (*optional*, default `localhost:9090`). The gRPC address of the node, used by the `prepare-upgrade` command and the batch upgrade watcher. +* `COSMOVISOR_DISABLE_LOGS` (defaults to `false`). If set to true, this will disable Cosmovisor logs (but not the underlying process) completely. This may be useful, for example, when a Cosmovisor subcommand you are executing returns a valid JSON you are then parsing, as logs added by Cosmovisor make this output not a valid JSON. +* `COSMOVISOR_COLOR_LOGS` (defaults to `true`). If set to true, this will colorize Cosmovisor logs (but not the underlying process). +* `COSMOVISOR_TIMEFORMAT_LOGS` (defaults to `kitchen`). If set to a value (`layout|ansic|unixdate|rubydate|rfc822|rfc822z|rfc850|rfc1123|rfc1123z|rfc3339|rfc3339nano|kitchen`), this will add timestamp prefix to Cosmovisor logs (but not the underlying process). +* `COSMOVISOR_CUSTOM_PREUPGRADE` (defaults to \`\`). If set, this will run $DAEMON\_HOME/cosmovisor/$COSMOVISOR\_CUSTOM\_PREUPGRADE prior to upgrade with the arguments \[ upgrade.Name, upgrade.Height ]. Executes a custom script (separate and prior to the chain daemon pre-upgrade command) +* `COSMOVISOR_DISABLE_RECASE` (defaults to `false`). If set to true, the upgrade directory will expected to match the upgrade plan name without any case changes + +### Folder Layout + +`$DAEMON_HOME/cosmovisor` is expected to belong completely to `cosmovisor` and the subprocesses that are controlled by it. The folder content is organized as follows: + +```text expandable +. +├── current -> genesis or upgrades/ +├── genesis +│   └── bin +│   └── $DAEMON_NAME +└── upgrades +│ └── +│ ├── bin +│ │   └── $DAEMON_NAME +│ └── upgrade-info.json +└── preupgrade.sh (optional) +``` + +The `cosmovisor/` directory includes a subdirectory for each version of the application (i.e. `genesis` or `upgrades/`). Within each subdirectory is the application binary (i.e. `bin/$DAEMON_NAME`) and any additional auxiliary files associated with each binary. `current` is a symbolic link to the currently active directory (i.e. `genesis` or `upgrades/`). The `name` variable in `upgrades/` is the lowercased URI-encoded name of the upgrade as specified in the upgrade module plan. Note that the upgrade name path are normalized to be lowercased: for instance, `MyUpgrade` is normalized to `myupgrade`, and its path is `upgrades/myupgrade`. + +Please note that `$DAEMON_HOME/cosmovisor` only stores the *application binaries*. The `cosmovisor` binary itself can be stored in any typical location (e.g. `/usr/local/bin`). The application will continue to store its data in the default data directory (e.g. `$HOME/.simapp`) or the data directory specified with the `--home` flag. `$DAEMON_HOME` is dependent of the data directory and must be set to the same directory as the data directory, you will end up with a configuration like the following: + +```text +.simapp +├── config +├── data +└── cosmovisor +``` + +## Usage + +The system administrator is responsible for: + +* installing the `cosmovisor` binary +* configuring the host's init system (e.g. `systemd`, `launchd`, etc.) +* appropriately setting the environmental variables +* creating the `/cosmovisor` directory +* creating the `/cosmovisor/genesis/bin` folder +* creating the `/cosmovisor/upgrades//bin` folders +* placing the different versions of the `` executable in the appropriate `bin` folders. + +`cosmovisor` will set the `current` link to point to `genesis` at first start (i.e. when no `current` link exists) and then handle switching binaries at the correct points in time so that the system administrator can prepare days in advance and relax at upgrade time. + +In order to support downloadable binaries, a tarball for each upgrade binary will need to be packaged up and made available through a canonical URL. Additionally, a tarball that includes the genesis binary and all available upgrade binaries can be packaged up and made available so that all the necessary binaries required to sync a fullnode from start can be easily downloaded. + +The `DAEMON` specific code and operations (e.g. CometBFT config, the application db, syncing blocks, etc.) all work as expected. The application binaries' directives such as command-line flags and environment variables also work as expected. + +### Initialization + +The `cosmovisor init ` command creates the folder structure required for using cosmovisor. + +It does the following: + +* creates the `/cosmovisor` folder if it doesn't yet exist +* creates the `/cosmovisor/genesis/bin` folder if it doesn't yet exist +* copies the provided executable file to `/cosmovisor/genesis/bin/` +* creates the `current` link, pointing to the `genesis` folder + +It uses the `DAEMON_HOME` and `DAEMON_NAME` environment variables for folder location and executable name. + +The `cosmovisor init` command is specifically for initializing cosmovisor, and should not be confused with a chain's `init` command (e.g. `cosmovisor run init`). + +### Detecting Upgrades + +`cosmovisor` is polling the `$DAEMON_HOME/data/upgrade-info.json` file for new upgrade instructions. The file is created by the x/upgrade module in `BeginBlocker` when an upgrade is detected and the blockchain reaches the upgrade height. +The following heuristic is applied to detect the upgrade: + +* When starting, `cosmovisor` doesn't know much about currently running upgrade, except the binary which is `current/bin/`. It tries to read the `current/upgrade-info.json` file to get information about the current upgrade name. +* If neither `cosmovisor/current/upgrade-info.json` nor `data/upgrade-info.json` exist, then `cosmovisor` will wait for `data/upgrade-info.json` file to trigger an upgrade. +* If `cosmovisor/current/upgrade-info.json` doesn't exist but `data/upgrade-info.json` exists, then `cosmovisor` assumes that whatever is in `data/upgrade-info.json` is a valid upgrade request. In this case `cosmovisor` tries immediately to make an upgrade according to the `name` attribute in `data/upgrade-info.json`. +* Otherwise, `cosmovisor` waits for changes in `upgrade-info.json`. As soon as a new upgrade name is recorded in the file, `cosmovisor` will trigger an upgrade mechanism. + +When the upgrade mechanism is triggered, `cosmovisor` will: + +1. if `DAEMON_ALLOW_DOWNLOAD_BINARIES` is enabled, start by auto-downloading a new binary into `cosmovisor//bin` (where `` is the `upgrade-info.json:name` attribute); +2. update the `current` symbolic link to point to the new directory and save `data/upgrade-info.json` to `cosmovisor/current/upgrade-info.json`. + +### Adding Upgrade Binary + +`cosmovisor` has an `add-upgrade` command that allows to easily link a binary to an upgrade. It creates a new folder in `cosmovisor/upgrades/` and copies the provided executable file to `cosmovisor/upgrades//bin/`. + +Using the `--upgrade-height` flag allows you to specify at which height the binary should be switched, without going via a governance proposal. +This enables support for an emergency coordinated upgrades where the binary must be switched at a specific height, but there is no time to go through a governance proposal. + + +`--upgrade-height` creates an `upgrade-info.json` file. This means if a chain upgrade via governance proposal is executed before the specified height with `--upgrade-height`, the governance proposal will overwrite the `upgrade-info.json` plan created by `add-upgrade --upgrade-height `. +Take this into consideration when using `--upgrade-height`. + + +### Auto-Download + +Generally, `cosmovisor` requires that the system administrator place all relevant binaries on disk before the upgrade happens. However, for people who don't need such control and want an automated setup (maybe they are syncing a non-validating fullnode and want to do little maintenance), there is another option. + +**NOTE: we don't recommend using auto-download** because it doesn't verify in advance if a binary is available. If there will be any issue with downloading a binary, the cosmovisor will stop and won't restart an App (which could lead to a chain halt). + +If `DAEMON_ALLOW_DOWNLOAD_BINARIES` is set to `true`, and no local binary can be found when an upgrade is triggered, `cosmovisor` will attempt to download and install the binary itself based on the instructions in the `info` attribute in the `data/upgrade-info.json` file. The files is constructed by the x/upgrade module and contains data from the upgrade `Plan` object. The `Plan` has an info field that is expected to have one of the following two valid formats to specify a download: + +1. Store an os/architecture -> binary URI map in the upgrade plan info field as JSON under the `"binaries"` key. For example: + + ```json + { + "binaries": { + "linux/amd64": "https://example.com/gaia.zip?checksum=sha256:aec070645fe53ee3b3763059376134f058cc337247c978add178b6ccdfb0019f" + } + } + ``` + + You can include multiple binaries at once to ensure more than one environment will receive the correct binaries: + + ```json + { + "binaries": { + "linux/amd64": "https://example.com/gaia.zip?checksum=sha256:aec070645fe53ee3b3763059376134f058cc337247c978add178b6ccdfb0019f", + "linux/arm64": "https://example.com/gaia.zip?checksum=sha256:aec070645fe53ee3b3763059376134f058cc337247c978add178b6ccdfb0019f", + "darwin/amd64": "https://example.com/gaia.zip?checksum=sha256:aec070645fe53ee3b3763059376134f058cc337247c978add178b6ccdfb0019f" + } + } + ``` + + When submitting this as a proposal ensure there are no spaces. An example command using `gaiad` could look like: + + ```shell expandable + > gaiad tx upgrade software-upgrade Vega \ + --title Vega \ + --deposit 100uatom \ + --upgrade-height 7368420 \ + --upgrade-info '{"binaries":{"linux/amd64":"https://github.com/cosmos/gaia/releases/download/v6.0.0-rc1/gaiad-v6.0.0-rc1-linux-amd64","linux/arm64":"https://github.com/cosmos/gaia/releases/download/v6.0.0-rc1/gaiad-v6.0.0-rc1-linux-arm64","darwin/amd64":"https://github.com/cosmos/gaia/releases/download/v6.0.0-rc1/gaiad-v6.0.0-rc1-darwin-amd64"}}' \ + --summary "upgrade to Vega" \ + --gas 400000 \ + --from user \ + --chain-id test \ + --home test/val2 \ + --node tcp://localhost:36657 \ + --yes + ``` + +2. Store a link to a file that contains all information in the above format (e.g. if you want to specify lots of binaries, changelog info, etc. without filling up the blockchain). For example: + + ```text + https://example.com/testnet-1001-info.json?checksum=sha256:deaaa99fda9407c4dbe1d04bd49bab0cc3c1dd76fa392cd55a9425be074af01e + ``` + +When `cosmovisor` is triggered to download the new binary, `cosmovisor` will parse the `"binaries"` field, download the new binary with [go-getter](https://github.com/hashicorp/go-getter), and unpack the new binary in the `upgrades/` folder so that it can be run as if it was installed manually. + +Note that for this mechanism to provide strong security guarantees, all URLs should include a SHA 256/512 checksum. This ensures that no false binary is run, even if someone hacks the server or hijacks the DNS. `go-getter` will always ensure the downloaded file matches the checksum if it is provided. `go-getter` will also handle unpacking archives into directories (in this case the download link should point to a `zip` file of all data in the `bin` directory). + +To properly create a sha256 checksum on linux, you can use the `sha256sum` utility. For example: + +```shell +sha256sum ./testdata/repo/zip_directory/autod.zip +``` + +The result will look something like the following: `29139e1381b8177aec909fab9a75d11381cab5adf7d3af0c05ff1c9c117743a7`. + +You can also use `sha512sum` if you would prefer to use longer hashes, or `md5sum` if you would prefer to use broken hashes. Whichever you choose, make sure to set the hash algorithm properly in the checksum argument to the URL. + +### Preparing for an Upgrade + +To prepare for an upgrade, use the `prepare-upgrade` command: + +```shell +cosmovisor prepare-upgrade +``` + +This command performs the following actions: + +1. Retrieves upgrade information directly from the blockchain about the next scheduled upgrade. +2. Downloads the new binary specified in the upgrade plan. +3. Verifies the binary's checksum (if required by configuration). +4. Places the new binary in the appropriate directory for Cosmovisor to use during the upgrade. + +This command requires gRPC to be enabled on the node (configured via `DAEMON_GRPC_ADDRESS`, default `localhost:9090`). + +The `prepare-upgrade` command logs the following: + +* The name and height of the upcoming upgrade +* The URL from which the new binary is being downloaded +* Confirmation of successful completion + +Example output: + +```bash +INFO Preparing for upgrade name=v1.0.0 height=1000000 +INFO Downloading upgrade binary url=https://example.com/binary/v1.0.0?checksum=sha256:339911508de5e20b573ce902c500ee670589073485216bee8b045e853f24bce8 +INFO Upgrade preparation complete name=v1.0.0 height=1000000 +``` + +*Note: The current way of downloading manually and placing the binary at the right place would still work.* + +## Example: SimApp Upgrade + +The following instructions provide a demonstration of `cosmovisor` using the simulation application (`simapp`) shipped with the Cosmos SDK's source code. The following commands are to be run from within the `cosmos-sdk` repository. + +### Chain Setup + +Let's create a new chain using the `v0.47.4` version of simapp (the Cosmos SDK demo app): + +```shell +git checkout v0.47.4 +make build +``` + +Clean `~/.simapp` (never do this in a production environment): + +```shell +./build/simd tendermint unsafe-reset-all +``` + +Set up app config: + +```shell +./build/simd config chain-id test +./build/simd config keyring-backend test +./build/simd config broadcast-mode sync +``` + +Initialize the node and overwrite any previous genesis file (never do this in a production environment): + +```shell +./build/simd init test --chain-id test --overwrite +``` + +For the sake of this demonstration, amend `voting_period` in `genesis.json` to a reduced time of 20 seconds (`20s`): + +```shell +cat <<< $(jq '.app_state.gov.params.voting_period = "20s"' $HOME/.simapp/config/genesis.json) > $HOME/.simapp/config/genesis.json +``` + +Create a validator, and setup genesis transaction: + +```shell +./build/simd keys add validator +./build/simd genesis add-genesis-account validator 1000000000stake --keyring-backend test +./build/simd genesis gentx validator 1000000stake --chain-id test +./build/simd genesis collect-gentxs +``` + +#### Prepare Cosmovisor and Start the Chain + +Set the required environment variables: + +```shell +export DAEMON_NAME=simd +export DAEMON_HOME=$HOME/.simapp +``` + +Set the optional environment variable to trigger an automatic app restart: + +```shell +export DAEMON_RESTART_AFTER_UPGRADE=true +``` + +Initialize cosmovisor with the current binary: + +```shell +cosmovisor init ./build/simd +``` + +Now you can run cosmovisor with simapp v0.47.4: + +```shell +cosmovisor run start +``` + +### Update App + +Update app to the latest version (e.g. v0.50.0). + + + +Migration plans are defined using the `x/upgrade` module and described in [Upgrading Modules](/sdk/v0.54/guides/upgrades/upgrade). Migrations can perform any deterministic state change. + +The migration plan to upgrade the simapp from v0.47 to v0.50 is defined in `simapp/upgrade.go`. + + + +Build the new version `simd` binary: + +```shell +make build +``` + +Add the new `simd` binary and the upgrade name: + + + +The migration name must match the one defined in the migration plan. + + + +```shell +cosmovisor add-upgrade v047-to-v050 ./build/simd +``` + +Open a new terminal window and submit an upgrade proposal along with a deposit and a vote (these commands must be run within 20 seconds of each other): + +```shell +./build/simd tx upgrade software-upgrade v047-to-v050 --title upgrade --summary upgrade --upgrade-height 200 --upgrade-info "{}" --no-validate --from validator --yes +./build/simd tx gov deposit 1 10000000stake --from validator --yes +./build/simd tx gov vote 1 yes --from validator --yes +``` + +The upgrade will occur automatically at height 200. Note: you may need to change the upgrade height in the snippet above if your test play takes more time. + +## Pre-Upgrade Handling + +Cosmovisor supports custom pre-upgrade handling. Use pre-upgrade handling when you need to implement application config changes that are required in the newer version before you perform the upgrade. If pre-upgrade handling is not implemented, the upgrade continues normally. + +Before the application binary is upgraded, Cosmovisor calls a `pre-upgrade` command that can be implemented by the application. The `pre-upgrade` command does not take in any command-line arguments and is expected to terminate with the following exit codes: + +| Exit status code | How it is handled in Cosmovisor | +| --- | --- | +| `0` | `pre-upgrade` command executed successfully. Cosmovisor continues the upgrade. | +| `1` | `pre-upgrade` command is not implemented. Cosmovisor continues the upgrade normally. | +| `30` | `pre-upgrade` command failed. Cosmovisor fails the entire upgrade. | +| `31` | `pre-upgrade` command failed. Cosmovisor retries until exit code `1` or `30` are returned, or until `DAEMON_PREUPGRADE_MAX_RETRIES` retries are exhausted (at which point the upgrade fails). | + +The number of allowed retries for exit code `31` is configured via `DAEMON_PREUPGRADE_MAX_RETRIES` (defaults to `0`, meaning no retries -- a single exit-31 result immediately fails the upgrade). + +Sample `pre-upgrade` command implementation: + +```go +func preUpgradeCommand() *cobra.Command { + return &cobra.Command{ + Use: "pre-upgrade", + Short: "Pre-upgrade command", + Run: func(cmd *cobra.Command, args []string) { + if err := HandlePreUpgrade(); err != nil { + os.Exit(30) + } + os.Exit(0) + }, + } +} +``` + +Register it in the root command: + +```go +rootCmd.AddCommand( + // .. + preUpgradeCommand(), +) +``` + +When not using Cosmovisor, install the new binary first, then run ` pre-upgrade` before starting it. The pre-upgrade command is part of the new binary, not the old one. diff --git a/sdk/v0.54/guides/upgrades/upgrade.mdx b/sdk/v0.54/guides/upgrades/upgrade.mdx new file mode 100644 index 000000000..e8bd27b35 --- /dev/null +++ b/sdk/v0.54/guides/upgrades/upgrade.mdx @@ -0,0 +1,199 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/guides/upgrades/upgrade' +title: Upgrades and Store Migrations +--- + + +Read and understand all of this page before running a migration on a live chain. + + + +**Synopsis** +In-place store migrations allow modules to upgrade to new versions that include breaking changes. This document covers both the module-side (writing migrations) and the app-side (running migrations during an upgrade). + + +The Cosmos SDK supports two approaches to chain upgrades: exporting the entire application state to JSON and starting fresh with a modified genesis file, or performing in-place store migrations that update state directly. In-place migrations are significantly faster for chains with large state and are the standard approach for live networks. + +This page covers how to write module migrations and how to run them inside an upgrade handler in your app. + +## Consensus Version + +Successful upgrades of existing modules require each `AppModule` to implement the function `ConsensusVersion() uint64`. + +* The versions must be hard-coded by the module developer. +* The initial version **must** be set to 1. + +Consensus versions serve as state-breaking versions of app modules and must be incremented when the module introduces breaking changes. + +## Registering Migrations + +To register the functionality that takes place during a module upgrade, you must register which migrations you want to take place. + +Migration registration takes place in the `Configurator` using the `RegisterMigration` method. The `AppModule` reference to the configurator is in the `RegisterServices` method. + +You can register one or more migrations. If you register more than one migration script, list the migrations in increasing order and ensure there are enough migrations that lead to the desired consensus version. For example, to migrate to version 3 of a module, register separate migrations for version 1 and version 2 as shown in the following example: + +```go +func (am AppModule) RegisterServices(cfg module.Configurator) { + // --snip-- + if err := cfg.RegisterMigration(types.ModuleName, 1, func(ctx sdk.Context) error { + // Perform in-place store migrations from ConsensusVersion 1 to 2. + return nil + }); err != nil { + panic(fmt.Sprintf("failed to migrate %s from version 1 to 2: %v", types.ModuleName, err)) + } + + if err := cfg.RegisterMigration(types.ModuleName, 2, func(ctx sdk.Context) error { + // Perform in-place store migrations from ConsensusVersion 2 to 3. + return nil + }); err != nil { + panic(fmt.Sprintf("failed to migrate %s from version 2 to 3: %v", types.ModuleName, err)) + } +} +``` + +Since these migrations are functions that need access to a Keeper's store, use a wrapper around the keepers called `Migrator` as shown in this example: + +```go expandable +package keeper + +import ( + sdk "github.com/cosmos/cosmos-sdk/types" + "github.com/cosmos/cosmos-sdk/x/bank/exported" + v2 "github.com/cosmos/cosmos-sdk/x/bank/migrations/v2" + v3 "github.com/cosmos/cosmos-sdk/x/bank/migrations/v3" + v4 "github.com/cosmos/cosmos-sdk/x/bank/migrations/v4" +) + +// Migrator is a struct for handling in-place store migrations. +type Migrator struct { + keeper BaseKeeper + legacySubspace exported.Subspace +} + +// NewMigrator returns a new Migrator. +func NewMigrator(keeper BaseKeeper, legacySubspace exported.Subspace) Migrator { + return Migrator{keeper: keeper, legacySubspace: legacySubspace} +} + +// Migrate1to2 migrates from version 1 to 2. +func (m Migrator) Migrate1to2(ctx sdk.Context) error { + return v2.MigrateStore(ctx, m.keeper.storeService, m.keeper.cdc) +} + +// Migrate2to3 migrates x/bank storage from version 2 to 3. +func (m Migrator) Migrate2to3(ctx sdk.Context) error { + return v3.MigrateStore(ctx, m.keeper.storeService, m.keeper.cdc) +} + +// Migrate3to4 migrates x/bank storage from version 3 to 4. +func (m Migrator) Migrate3to4(ctx sdk.Context) error { + m.MigrateSendEnabledParams(ctx) + return v4.MigrateStore(ctx, m.keeper.storeService, m.legacySubspace, m.keeper.cdc) +} +``` + +## Writing Migration Scripts + +To define the functionality that takes place during an upgrade, write a migration script and place the functions in a `migrations/` directory. For example, to write migration scripts for the bank module, place the functions in `x/bank/migrations/`. Import each version package and call its `MigrateStore` function from the corresponding `Migrator` method: + +```go +// Migrating bank module from version 1 to 2 +func (m Migrator) Migrate1to2(ctx sdk.Context) error { + return v2.MigrateStore(ctx, m.keeper.storeService, m.keeper.cdc) // v2 is package `x/bank/migrations/v2`. +} +``` + +To see example code of changes that were implemented in a migration of balance keys, check out [migrateBalanceKeys](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/x/bank/migrations/v2/store.go#L55-L76). For context, this code introduced migrations of the bank store that updated addresses to be prefixed by their length in bytes as outlined in [ADR-028](/sdk/v0.54/reference/architecture/adr-028-public-key-addresses). + +## Running Migrations in the App + +Once modules have registered their migrations, the app runs them inside an `UpgradeHandler`. The upgrade handler type is: + +```go +type UpgradeHandler func(ctx context.Context, plan upgradetypes.Plan, fromVM module.VersionMap) (module.VersionMap, error) +``` + +The handler receives the `VersionMap` stored by `x/upgrade` (reflecting the consensus versions from the previous binary), performs any additional upgrade logic, and must return the updated `VersionMap` from `RunMigrations`. Register the handler in `app.go`: + +```go +app.UpgradeKeeper.SetUpgradeHandler("my-plan", func(ctx context.Context, plan upgradetypes.Plan, fromVM module.VersionMap) (module.VersionMap, error) { + // optional: additional upgrade logic here + return app.ModuleManager.RunMigrations(ctx, app.Configurator(), fromVM) +}) +``` + +`RunMigrations` iterates over all registered modules in order, checks each module's version in the `VersionMap`, and runs all registered migration scripts for modules whose consensus version has increased. The updated `VersionMap` is returned to the upgrade keeper, which persists it in the `x/upgrade` store. + +### Order of migrations + +By default, migrations run in alphabetical order by module name, with one exception: `x/auth` runs last due to state dependencies with other modules (see [cosmos/cosmos-sdk#10591](https://github.com/cosmos/cosmos-sdk/issues/10591)). To change the order, call `app.ModuleManager.SetOrderMigrations(module1, module2, ...)` in `app.go`. The function panics if any registered module is omitted. + +### Adding new modules during an upgrade + +New modules are recognized because they have no entry in the `x/upgrade` `VersionMap` store. `RunMigrations` calls `InitGenesis` for them automatically. + +If you need to add stores for a new module, configure the store loader before the upgrade runs: + +```go +upgradeInfo, err := app.UpgradeKeeper.ReadUpgradeInfoFromDisk() +if err != nil { + panic(err) +} +if upgradeInfo.Name == "my-plan" && !app.UpgradeKeeper.IsSkipHeight(upgradeInfo.Height) { + storeUpgrades := storetypes.StoreUpgrades{ + Added: []string{"newmodule"}, + } + app.SetStoreLoader(upgradetypes.UpgradeStoreLoader(upgradeInfo.Height, &storeUpgrades)) +} +``` + +To skip `InitGenesis` for a new module (for example, if you are manually initializing state in the handler), set its version in `fromVM` before calling `RunMigrations`: + +```go +fromVM["newmodule"] = newmodule.AppModule{}.ConsensusVersion() +return app.ModuleManager.RunMigrations(ctx, app.Configurator(), fromVM) +``` + +### Genesis state + +When starting a new chain, the consensus version of each module must be saved to state during genesis. Add this to `InitChainer` in `app.go`: + +```go +func (app *MyApp) InitChainer(ctx sdk.Context, req *abci.RequestInitChain) (*abci.ResponseInitChain, error) { + // ... + app.UpgradeKeeper.SetModuleVersionMap(ctx, app.ModuleManager.GetVersionMap()) + // ... +} +``` + +This lets the Cosmos SDK detect when modules with newer consensus versions are introduced in a future upgrade. + +### Overwriting genesis functions + +The SDK provides modules that app developers can import, and those modules often already have an `InitGenesis` function. If you want to run a custom genesis function for one of those modules during an upgrade instead of the default one, you must both call your custom function in the handler AND manually set that module's consensus version in `fromVM`. Without the second step, `RunMigrations` will run the module's existing `InitGenesis` even though you already initialized it. + + +You must manually set the consensus version in `fromVM` for any module whose `InitGenesis` you are overriding. If you don't, the SDK will call the module's default `InitGenesis` in addition to your custom one. + + +```go +import foo "github.com/my/module/foo" + +app.UpgradeKeeper.SetUpgradeHandler("my-plan", func(ctx context.Context, plan upgradetypes.Plan, fromVM module.VersionMap) (module.VersionMap, error) { + // Prevent RunMigrations from calling foo's default InitGenesis. + fromVM["foo"] = foo.AppModule{}.ConsensusVersion() + + // Run your custom genesis initialization for foo. + app.ModuleManager.Modules["foo"].(module.HasGenesis).InitGenesis(ctx, app.appCodec, myCustomGenesisState) + + return app.ModuleManager.RunMigrations(ctx, app.Configurator(), fromVM) +}) +``` + +## Syncing a Full Node to an Upgraded Blockchain + +A full node joining an already-upgraded chain must start from the initial binary that the chain used at genesis and replay all historical upgrades. If all upgrade plans include binary download instructions, Cosmovisor's auto-download mode handles this automatically. Otherwise, you must provide each historical binary manually. + +See the [Cosmovisor](/sdk/v0.54/guides/upgrades/cosmovisor) guide for setup and configuration. diff --git a/sdk/v0.54/learn.mdx b/sdk/v0.54/learn.mdx new file mode 100644 index 000000000..46fa5a7b4 --- /dev/null +++ b/sdk/v0.54/learn.mdx @@ -0,0 +1,29 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/learn' +title: "Cosmos SDK Docs" +description: "Version: v0.54" +--- + +The Cosmos SDK is the most widely adopted, battle-tested Layer 1 blockchain stack, trusted by 200+ chains live in production. This modular framework enables you to build secure, high-performance blockchains with comprehensive guides covering everything from core concepts to advanced implementation patterns. + + + + New to the Cosmos SDK? Find the right starting point based on your background and what you want to build. + + + Learn essential concepts including application anatomy, transaction lifecycles, accounts, and gas mechanics. + + + Build and run a Cosmos chain from scratch, with step-by-step guidance from setup to a working custom module. + + + Develop custom modules with comprehensive guides on module architecture, message handling, and state management. + + + Set up, configure, and maintain nodes from local development environments to production deployments. + + + Understand the fundamentals of Cosmos SDK, application-specific blockchains, and the SDK's architecture. + + diff --git a/sdk/v0.54/learn/concepts/accounts.mdx b/sdk/v0.54/learn/concepts/accounts.mdx new file mode 100644 index 000000000..7f68a13f8 --- /dev/null +++ b/sdk/v0.54/learn/concepts/accounts.mdx @@ -0,0 +1,166 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/learn/concepts/accounts' +title: Accounts +--- + +In [Cosmos Architecture](/sdk/v0.54/learn/intro/sdk-app-architecture), you learned that transactions change state and must be signed and validated. But who creates and signs these transactions? The answer is **accounts**. + +Accounts represent identities on a Cosmos SDK chain. They hold balances, authorize transactions with digital signatures, and prevent transaction replay using sequence numbers. Accounts are managed by the auth module (`x/auth`), which tracks account metadata like addresses, public keys, account numbers, and sequence numbers. + +Every account is controlled by a cryptographic keypair derived from a seed phrase. A seed phrase yields one or more private keys, each of which produces a public key and an account address. + +## What is an account + +An account is an on-chain identity used to authorize transactions. Each account stores an address, a public key, an account number, and a sequence number, as defined by [`BaseAccount`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/x/auth/types/auth.pb.go#L32) in the `x/auth` module: + +```go +type BaseAccount struct { + Address string `protobuf:"bytes,1,opt,name=address,proto3" json:"address,omitempty"` + PubKey *anypb.Any `protobuf:"bytes,2,opt,name=pub_key,json=pubKey,proto3" json:"pub_key,omitempty"` + AccountNumber uint64 `protobuf:"varint,3,opt,name=account_number,json=accountNumber,proto3" json:"account_number,omitempty"` + Sequence uint64 `protobuf:"varint,4,opt,name=sequence,proto3" json:"sequence,omitempty"` +} +``` + +Accounts can be used in other modules to associate on-chain state with an identity. For example, the bank module (`x/bank`) maps account addresses to token balances, and the staking module maps them to delegations. + +The private key and [seed phrase](#seed-phrases) are never stored on-chain; they are kept locally by the user or wallet. + +An account does not execute logic itself; instead, it authorizes [transactions](/sdk/v0.54/learn/concepts/transactions). Balance changes for accounts are handled by the modules that process the transaction's messages. An account's sequence number is used for [replay protection](#sequences-and-replay-protection) during transaction processing. + +## Public and private keys + +Accounts are rooted in cryptographic keypairs. Cosmos SDK uses asymmetric cryptography, where a private key and public key form a pair. This is a fundamental concept in cryptography and is used to secure data and transactions. + +- A **private key** is used to sign transactions. Before signing, the transaction data is serialized and hashed; the private key then produces a digital signature over this hash. This signature proves ownership of the private key without revealing it. Private keys must always remain secret. + +- A **public key** is derived mathematically from the private key. The network uses it to verify signatures produced by the corresponding private key. Because the public key is derived through a one-way function, it is not possible to derive the private key from the public key. + +## Seed phrases + +Most wallets do not generate raw private keys directly. Instead, they start from a seed phrase (mnemonic), a list of human-readable words such as: + +``` text +apple maple river stone cloud frame picnic ladder jungle orbit solar velvet +``` + +A private key is then derived from the seed phrase using a deterministic algorithm. Cosmos wallets follow common standards such as: + +- [BIP-39 (mnemonic phrases)](https://github.com/bitcoin/bips/blob/master/bip-0039.mediawiki) +- [BIP-32 (hierarchical deterministic wallets)](https://github.com/bitcoin/bips/blob/master/bip-0032.mediawiki) +- [BIP-44 (multi-account derivation paths)](https://github.com/bitcoin/bips/blob/master/bip-0044.mediawiki) + +From the seed phrase, a binary seed is computed and used to derive a master private key. From that master key, specific private keys are derived along a path (for example: `m/44'/118'/0'/0/0`, where `118` is the Cosmos coin type). Each private key produces a public key. + +Control of the seed phrase means control of the derived private keys and therefore control of the corresponding accounts. Losing the seed phrase without backing it up means losing access to the account forever. + +## Addresses + +An address is a shortened identifier derived from the public key. The public key is hashed and encoded, typically in [Bech32](/sdk/v0.54/guides/reference/bech32) format, with a prefix that indicates the chain, for example `cosmos`. This address is what users share and what appears in state and transactions: + +```text +cosmos1qnk2n4nlkpw9xfqntladh74er2xa62wgas7mv0 +``` + +An address is not the same as a public key. Because an address is only a hash of the public key, users can generate addresses and receive funds entirely offline. The public key is revealed on-chain the first time the account signs a transaction, at which point validators can verify the signature and the chain stores the public key alongside the account metadata. + +```text +Seed Phrase + ↓ (BIP-39/BIP-32/BIP-44) +Private Key (secp256k1) + ↓ (elliptic curve math) +Public Key + ↓ (hash + Bech32 encoding) +Address +``` + +## Sequences and replay protection + +There are two types of transactions in the Cosmos SDK: ordered and unordered. Ordered transactions are the default. Each account tracks a sequence number starting at zero that increments with each transaction. The network rejects any transaction whose sequence number does not match the current value, preventing replay attacks and ensuring that dependent transactions from the same account execute in order (for example, sending tokens then immediately staking them). Unordered transactions bypass this check and use a timeout-based mechanism instead. + +Example: + +```text +Initial state: + sequence = 0 + +After first accepted transaction: + sequence = 1 + +After second accepted transaction: + sequence = 2 +``` + +If a signed transaction carries `sequence = 1` but the account's current sequence is `2`, the transaction is rejected, ensuring that ordered transactions are applied in order and cannot be reused. + +The Cosmos SDK also supports optional unordered transactions, which allow transactions from the same account to be submitted and processed without strict sequence ordering. When a chain enables unordered transactions, replay protection uses a timeout timestamp and unordered nonce tracking instead of the normal per-signer sequence check. + +See [Transactions, Messages, and Queries](/sdk/v0.54/learn/concepts/transactions#message-execution-and-atomicity) for more information. + +## Balances + +Accounts are associated with token balances stored on-chain. Balances are managed by the bank module (`x/bank`) and indexed by account address. While account metadata (address, public key, sequence number) is stored in the auth module's state, token balances are stored separately in the bank module's state. + +When tokens are sent from one account to another, the bank module updates balances in state. Conceptually, a token transfer decreases the sender's balance and increases the recipient's balance. + +An account must have sufficient balance to cover the tokens being sent and any associated transaction fees. If the balance is insufficient, the transaction is rejected during validation. + +## Types of accounts + +Cosmos SDK supports several account types that extend the base account model: + +- **Base account**: A standard account that holds balances and signs transactions. This is the most common account type for users. + +- **Module account**: Owned by a [module](/sdk/v0.54/learn/concepts/modules) rather than a user. Module accounts are derived from the module name and cannot be controlled by a private key. For example, the staking module uses a module account to hold all delegated tokens, and the distribution module uses a module account to hold rewards before they are distributed. This design allows protocol logic to custody tokens without requiring a private key holder, which is essential for decentralized operations. For a working example of adding a module account to receive fees, see [Module accounts](/sdk/v0.54/tutorials/example/04-counter-walkthrough#module-accounts) in the Full Counter Module Walkthrough. + +- **Vesting account**: Holds tokens that unlock gradually over time according to a schedule. Vesting accounts are often used for team allocations or investor tokens that vest over months or years. They restrict spending to only unlocked tokens while still allowing the account to participate in staking and governance. + +All account types rely on the same key and address structure but may impose additional rules on balance usage. + +## Accounts and transaction authorization + +Accounts authorize [transactions](/sdk/v0.54/learn/concepts/transactions) by producing digital signatures. + +A transaction includes: + +- One or more messages +- A signature created using the private key +- A sequence number +- Associated fees + +When a transaction is signed, the transaction bytes are serialized and hashed. The private key then generates a digital signature over that hash. This signature proves that the holder of the private key approved the transaction, without revealing the private key itself. + +During execution of a standard ordered transaction: + +1. The signature is verified using the account's public key. +2. The sequence number is checked against the account's current sequence. +3. Fees are deducted from the account's balance. +4. If validation passes, messages execute and may update state. +5. If execution succeeds, the sequence number increments and state updates are committed. + +High-level flow: + +``` +Seed Phrase + ↓ +Private Key + ↓ signs +Transaction + ↓ verified with +Public Key + ↓ identifies +Address + ↓ updates +State +``` + +Accounts provide identity and authorization, transactions carry intent, and modules execute the logic. The result is stored in state. + +To learn more about the transaction flow in a Cosmos blockchain, visit the [Transaction Lifecycle page](/sdk/v0.54/learn/concepts/lifecycle) + +## Summary + +Accounts are the foundation of user interaction with a Cosmos SDK chain. They connect cryptographic keys to on-chain identity, authorize transaction execution, and prevent replay attacks. + +Understanding keys, addresses, balances, and sequence numbers provides the basis for understanding how transactions flow through the system. The next page, [Transactions, Messages, and Queries](/sdk/v0.54/learn/concepts/transactions), explains how accounts authorize the actions a transaction carries. diff --git a/sdk/v0.54/learn/concepts/app-go.mdx b/sdk/v0.54/learn/concepts/app-go.mdx new file mode 100644 index 000000000..cbf33c46f --- /dev/null +++ b/sdk/v0.54/learn/concepts/app-go.mdx @@ -0,0 +1,328 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/learn/concepts/app-go' +title: app.go Overview +--- + +`app.go` is where an application is assembled into a working chain. It creates the `BaseApp` instance that talks to CometBFT, allocates store keys, initializes keepers, registers modules, configures execution ordering, mounts stores, and sets lifecycle hooks and the `AnteHandler`. Finally, it seals the application with `LoadLatestVersion`. + +The result is a single constructor, `NewExampleApp`, that returns a fully wired, ready-to-run chain. + +Most examples on this page come from the counter module example in the `example` repo, where `x/counter` is wired into a fuller chain. The minimal counter module example shows the smaller `app.go` delta needed to add `x/counter` to a stripped-down app. See [Step 10: Wire into app.go](/sdk/v0.54/tutorials/example/03-build-a-module#step-10-wire-into-appgo) in the Build a Module tutorial. + +## What `app.go` does + +`app.go` performs a one-time, ordered initialization of the entire chain: + +``` +1. Create BaseApp and codecs +2. Allocate store keys +3. Initialize keepers +4. Create the ModuleManager +5. Configure execution ordering +6. Register module services +7. Mount KV stores +8. Set lifecycle hooks (InitChainer, PreBlocker, BeginBlocker, EndBlocker, AnteHandler) +9. Load latest version +``` + +This sequence is strict: + - Keepers require store keys, so keys come first. + - The `ModuleManager` depends on keepers, so modules come after keeper construction. + - Lifecycle hooks depend on the `ModuleManager`, so hook wiring comes later. + - `LoadLatestVersion` seals `BaseApp`, so it runs last. + +## The app struct + +The application struct embeds [`BaseApp`](/sdk/v0.54/learn/concepts/baseapp) and holds all keepers and the `ModuleManager`: + +```go +type ExampleApp struct { + *baseapp.BaseApp + appCodec codec.Codec + interfaceRegistry codectypes.InterfaceRegistry + + keys map[string]*storetypes.KVStoreKey + + // representative keepers + AccountKeeper authkeeper.AccountKeeper + BankKeeper bankkeeper.Keeper + ConsensusParamsKeeper consensusparamkeeper.Keeper + CounterKeeper *counterkeeper.Keeper + + // application wiring helpers + ModuleManager *module.Manager + BasicModuleManager module.BasicManager + configurator module.Configurator +} +``` + +Embedding `*baseapp.BaseApp` gives `ExampleApp` the full `BaseApp` interface: ABCI methods, message and query routers, store management, and lifecycle hooks. The keeper fields are exported so test code and CLI helpers can reference them. The `keys` map holds the KV store keys allocated during initialization. The real example app includes additional keepers and helper fields; this excerpt shows the part of the struct that matters for understanding the wiring pattern. + +## Creating `BaseApp` + +`NewExampleApp` begins by setting up codecs and creating the `BaseApp` instance: + +```go +appCodec := codec.NewProtoCodec(interfaceRegistry) +txConfig := authtx.NewTxConfig(appCodec, authtx.DefaultSignModes) + +bApp := baseapp.NewBaseApp(appName, logger, db, txConfig.TxDecoder(), baseAppOptions...) +bApp.SetVersion(version.Version) +bApp.SetInterfaceRegistry(interfaceRegistry) +bApp.SetTxEncoder(txConfig.TxEncoder()) +``` + +`baseapp.NewBaseApp` creates the `BaseApp` with a name, logger, database, and `TxDecoder`. The `TxDecoder` is how `BaseApp` turns raw transaction bytes from CometBFT into an `sdk.Tx` it can inspect and route. Additional functional options (`baseAppOptions`) let callers configure pruning, minimum gas prices, chain ID, and optimistic execution without modifying `NewExampleApp` directly. The full example app also wires legacy Amino support, tracing, and interface registration around this excerpt. See [`BaseApp` Overview](/sdk/v0.54/learn/concepts/baseapp) for a fuller description of its fields and behavior. + + +## Allocating store keys + +Each module that persists state needs a dedicated KV store key. All keys are allocated together before any keeper is created: + +```go +keys := storetypes.NewKVStoreKeys( + authtypes.StoreKey, + banktypes.StoreKey, + stakingtypes.StoreKey, + distrtypes.StoreKey, + slashingtypes.StoreKey, + govtypes.StoreKey, + consensusparamtypes.StoreKey, + countertypes.StoreKey, +) +``` + +Each module defines its store key name as a string constant in `types/keys.go` (for example, `countertypes.StoreKey = "counter"` — see [Step 4: Types](/sdk/v0.54/tutorials/example/03-build-a-module#step-4-types) in the Build a Module tutorial). `NewKVStoreKeys` takes those names and allocates a `*storetypes.KVStoreKey` for each one. Keys are passed to keeper constructors and later mounted on the `CommitMultiStore` via `MountKVStores`. No two modules share a key; that isolation is what keeps module state separate. + +## Initializing keepers + +Each keeper is initialized with its store key, codec, and any dependencies on other keepers. `ConsensusParamsKeeper` is initialized first because it must call `bApp.SetParamStore` before any other keeper is created: + +```go +app.ConsensusParamsKeeper = consensusparamkeeper.NewKeeper(...) +bApp.SetParamStore(app.ConsensusParamsKeeper.ParamsStore) + +app.AccountKeeper = authkeeper.NewAccountKeeper(...) +app.BankKeeper = bankkeeper.NewBaseKeeper(..., app.AccountKeeper, ...) +app.CounterKeeper = counterkeeper.NewKeeper( + runtime.NewKVStoreService(keys[countertypes.StoreKey]), + appCodec, + app.BankKeeper, +) +``` + +`runtime.NewKVStoreService(key)` wraps the raw store key in a service interface that keepers use to open their store from a context. This keeps keepers from holding direct references to the underlying store. Instead, they retrieve it at runtime from the context passed into each method. + +The keeper initialization order matters: `BankKeeper` receives `app.AccountKeeper` as an argument, so `AccountKeeper` must be initialized first. The same dependency ordering applies throughout. The counter module example also passes `app.BankKeeper` into `counterkeeper.NewKeeper`, showing how custom modules depend on existing module services (see [Expected keepers and fee collection](/sdk/v0.54/tutorials/example/04-counter-walkthrough#expected-keepers-and-fee-collection) in the Full Counter Module Walkthrough). Where modules are interdependent, hooks connect them after both keepers exist: + +```go +app.StakingKeeper.SetHooks( + stakingtypes.NewMultiStakingHooks( + app.DistrKeeper.Hooks(), + app.SlashingKeeper.Hooks(), + ), +) +``` + +The authority address passed to most keepers (`authtypes.NewModuleAddress(govtypes.ModuleName).String()`) is the address that is allowed to call privileged messages such as `MsgUpdateParams`. Governance controls parameter changes by sending messages from the governance module account. See [Params](/sdk/v0.54/learn/concepts/modules#params) for how this pattern works. + +## Registering modules + +After all keepers are initialized, the module manager is created with every module the application uses: + +```go +app.ModuleManager = module.NewManager( + auth.NewAppModule(appCodec, app.AccountKeeper, authsims.RandomGenesisAccounts, nil), + bank.NewAppModule(appCodec, app.BankKeeper, app.AccountKeeper, nil), + consensus.NewAppModule(appCodec, app.ConsensusParamsKeeper), + counter.NewAppModule(appCodec, app.CounterKeeper), + // ...other modules... +) +``` + +`module.NewManager` takes a list of `AppModule` implementations. Each `AppModule` wraps a keeper and satisfies the interfaces the `ModuleManager` uses: genesis, block hooks, message and query service registration, and simulation support. The real example app includes the full built-in module set around `x/counter` — see [Step 10: Wire into app.go](/sdk/v0.54/tutorials/example/03-build-a-module#step-10-wire-into-appgo) for a walkthrough of module registration; this excerpt shows the basic registration pattern. The `BasicModuleManager` is then derived from the `ModuleManager` for codec registration and default genesis handling. + + +## Module Manager + +The `ModuleManager` is the application's registry of modules. It holds references to all `AppModule` instances and coordinates their participation in the block lifecycle. When `BaseApp` fires a lifecycle hook (`PreBlock`, `BeginBlock`, `EndBlock`, `InitGenesis`), it delegates to the `ModuleManager`, which calls each module's corresponding method in the configured order. + +The `ModuleManager` is also responsible for service registration: it iterates all modules and calls each module's `RegisterServices` to register `MsgServer` and `QueryServer` implementations with `BaseApp`'s routers. For the execution-model view, see [Module Manager in `BaseApp`](/sdk/v0.54/learn/concepts/baseapp#module-manager). + +## Execution ordering + +The order in which modules run their block hooks and genesis initialization matters. Some modules depend on others having already updated state. Ordering is configured explicitly after the module manager is created: + +```go +app.ModuleManager.SetOrderPreBlockers( + authtypes.ModuleName, +) +app.ModuleManager.SetOrderBeginBlockers( + distrtypes.ModuleName, + slashingtypes.ModuleName, + stakingtypes.ModuleName, + countertypes.ModuleName, + genutiltypes.ModuleName, +) +app.ModuleManager.SetOrderEndBlockers( + banktypes.ModuleName, + govtypes.ModuleName, + stakingtypes.ModuleName, + countertypes.ModuleName, + genutiltypes.ModuleName, +) +``` + +Genesis initialization order is separate and equally important: + +```go +genesisModuleOrder := []string{ + authtypes.ModuleName, + banktypes.ModuleName, + distrtypes.ModuleName, + stakingtypes.ModuleName, + slashingtypes.ModuleName, + govtypes.ModuleName, + consensusparamtypes.ModuleName, + vestingtypes.ModuleName, + countertypes.ModuleName, + genutiltypes.ModuleName, +} +app.ModuleManager.SetOrderInitGenesis(genesisModuleOrder...) +app.ModuleManager.SetOrderExportGenesis(exportModuleOrder...) +``` + +`SetOrderExportGenesis` controls the order modules serialize their state when the chain is exported to a genesis file, for example during a hard fork or when creating a snapshot-based testnet. The export order can differ from the init genesis order; in the example chain they use different orderings. + +The comments in the example app explain the reasoning: `genutil` must run after `staking` so that staking pools are initialized before genesis transactions are processed, and after `auth` so that it can access auth parameters. + + +Each hook type has its own ordering constraint: + +- [`PreBlock`](/sdk/v0.54/learn/concepts/lifecycle#preblock): runs before `BeginBlock`. Used for upgrades and consensus parameter changes that must take effect before the block begins. +- [`BeginBlock`](/sdk/v0.54/learn/concepts/lifecycle#beginblock): runs at the start of each block. Used for per-block housekeeping such as minting inflation rewards and distributing staking rewards. +- [`EndBlock`](/sdk/v0.54/learn/concepts/lifecycle#endblock): runs after all transactions in the block. Used for logic that depends on cumulative block state, such as tallying governance votes or recalculating validator power. +- [`InitGenesis`](/sdk/v0.54/learn/concepts/store#genesis-and-chain-initialization): runs once at chain start, populating each module's store from `genesis.json`. + +For a worked example of implementing these hooks in a custom module, see [BeginBlock and EndBlock](/sdk/v0.54/tutorials/example/04-counter-walkthrough#beginblock-and-endblock) in the Full Counter Module Walkthrough. + +## Routing setup + +After execution ordering is configured, module services are registered with `BaseApp`'s routers: + +```go +app.configurator = module.NewConfigurator(app.appCodec, app.MsgServiceRouter(), app.GRPCQueryRouter()) +err := app.ModuleManager.RegisterServices(app.configurator) +``` + +`RegisterServices` iterates all modules and calls each module's `RegisterServices(cfg)` method. Each module uses the configurator to register its `MsgServer` with the message router and its `QueryServer` with the gRPC query router. After this step, `BaseApp` can route any registered message type to the correct module handler, and any registered query to the correct query handler. The example app also registers an AutoCLI query service after this step. + +The AutoCLI query service registration lets the CLI introspect module options without requiring per-module CLI command boilerplate. For how these services are exposed to clients, see [CLI, gRPC, and REST](/sdk/v0.54/learn/concepts/cli-grpc-rest). + +## Block proposal and vote extension handlers + +`BaseApp` exposes four handlers for the ABCI 2.0 proposal phase: `SetPrepareProposal`, `SetProcessProposal`, `SetExtendVoteHandler`, and `SetVerifyVoteExtensionHandler`. All have sensible defaults. Chains that need custom behavior wire their handlers in `app.go` after the module manager is configured: + +```go +app.SetPrepareProposal(myPrepareProposalHandler) +app.SetProcessProposal(myProcessProposalHandler) +``` + +For a full explanation of what each handler does, see [Block proposal and vote extensions](/sdk/v0.54/learn/concepts/baseapp#block-proposal-and-vote-extensions). + +## Mounting stores and setting hooks + +With routing configured, stores are mounted and the application's lifecycle hooks are set: + +```go +// initialize stores +app.MountKVStores(keys) + +// initialize BaseApp +app.SetInitChainer(app.InitChainer) +app.SetPreBlocker(app.PreBlocker()) +app.SetBeginBlocker(app.BeginBlocker) +app.SetEndBlocker(app.EndBlocker) +app.setAnteHandler(txConfig) +``` + +`MountKVStores` registers each key with `BaseApp`'s `CommitMultiStore`. The hooks delegate to the `ModuleManager`: + +```go +func (app *ExampleApp) BeginBlocker(ctx sdk.Context) (sdk.BeginBlock, error) { + return app.ModuleManager.BeginBlock(ctx) +} +``` + +`EndBlocker` delegates in the same way, and `InitChainer` delegates to `ModuleManager.InitGenesis` after decoding `genesis.json`. The `AnteHandler` is configured separately because it takes a `TxConfig` dependency: + +```go +func (app *ExampleApp) setAnteHandler(txConfig client.TxConfig) { + anteHandler, err := ante.NewAnteHandler( + ante.HandlerOptions{ + AccountKeeper: app.AccountKeeper, + BankKeeper: app.BankKeeper, + SignModeHandler: txConfig.SignModeHandler(), + SigGasConsumer: ante.DefaultSigVerificationGasConsumer, + }, + ) + if err != nil { + panic(err) + } + app.SetAnteHandler(anteHandler) +} +``` + +The `AnteHandler` runs before any message in a transaction executes. It verifies signatures, validates and increments the account sequence number, deducts fees, and meters gas. If it fails, the transaction is rejected before any module logic runs. + +A `PostHandler` can also be registered with `app.SetPostHandler`. It runs after all messages in a transaction execute (regardless of whether they succeeded), in the same state branch, and is reverted if it fails. The SDK's default `PostHandler` chain is minimal. For a deeper look at how the `AnteHandler` fits into transaction execution, see [AnteHandler](/sdk/v0.54/learn/concepts/baseapp#antehandler). + +## Sealing with LoadLatestVersion + +The final step in `NewExampleApp` is loading the latest committed state: + +```go +if loadLatest { + if err := app.LoadLatestVersion(); err != nil { + panic(fmt.Errorf("error loading last version: %w", err)) + } +} +``` + +`LoadLatestVersion` calls `storeLoader` to load the latest committed store state from the database, then calls `Init`, which validates that required components are configured, initializes the check state, and sets `BaseApp.sealed` to `true`. Any setter called after this point panics. This enforces that all wiring happens before the application starts serving requests. + +On first launch, CometBFT calls `InitChain` which triggers `InitChainer`, which calls `ModuleManager.InitGenesis` to populate each module's state from `genesis.json`. + +## How everything fits together + +A Cosmos SDK chain is assembled into a single constructor function that returns a fully wired, ready-to-run chain. Each step builds on the previous: + +``` +NewBaseApp + ↓ +NewKVStoreKeys → one key per module + ↓ +NewKeeper(key, ...) → one keeper per module, dependencies wired explicitly + ↓ +NewManager(modules) → module manager holds all AppModule instances + ↓ +SetOrder*(...) → configure hook and genesis execution ordering + ↓ +RegisterServices(configurator) → wire MsgServer and QueryServer into BaseApp routers + ↓ +MountKVStores(keys) → attach module stores to CommitMultiStore + ↓ +SetInitChainer / SetPreBlocker / SetBeginBlocker / SetEndBlocker / SetAnteHandler + ↓ +LoadLatestVersion → seal BaseApp, ready to serve +``` + +At runtime, CometBFT drives the application through ABCI. Each ABCI call dispatches through `BaseApp`: + +- `InitChain` calls `InitChainer`, which runs `ModuleManager.InitGenesis`. +- `FinalizeBlock` calls `PreBlocker`, `BeginBlocker`, each transaction's `AnteHandler` and message handlers, then `EndBlocker`. +- `CheckTx` validates a transaction through the `AnteHandler` and writes to the internal `CheckTx` state if it passes. +- `Commit` persists the finalized block state. + +Modules never call each other directly. They interact through keeper interfaces wired at initialization time, and they participate in the block lifecycle through hooks that the `ModuleManager` coordinates in a fixed, declared order. See [BaseApp Overview](/sdk/v0.54/learn/concepts/baseapp) for how ABCI calls flow through the application, and [Intro to Modules](/sdk/v0.54/learn/concepts/modules) for how individual modules are structured. The next section, [CLI, gRPC, and REST API](/sdk/v0.54/learn/concepts/cli-grpc-rest), explains how clients interact with the chain once that wiring is in place. diff --git a/sdk/v0.54/learn/concepts/baseapp.mdx b/sdk/v0.54/learn/concepts/baseapp.mdx new file mode 100644 index 000000000..65124cc7b --- /dev/null +++ b/sdk/v0.54/learn/concepts/baseapp.mdx @@ -0,0 +1,211 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/learn/concepts/baseapp' +title: BaseApp Overview +--- + +`BaseApp` is the execution engine of every Cosmos SDK chain. It implements [ABCI (Application Blockchain Interface)](/sdk/v0.54/learn/intro/sdk-app-architecture#abci-application-blockchain-interface), the protocol CometBFT uses to communicate with the application, and translates those calls into module execution, transaction processing, and state transitions. + +Every Cosmos SDK chain embeds `BaseApp`. Your `app.go` creates a `BaseApp` instance, configures it with modules, keepers, and middleware, and the resulting struct is what CometBFT communicates with directly. `BaseApp` provides the base layer of execution infrastructure to your blockchain application. Without it, every chain would need to independently implement ABCI handling, signature verification, gas metering, message routing, block hook orchestration, and state commitment. + +## Architectural position + +`BaseApp` sits between CometBFT and the modules: + +``` +CometBFT (consensus engine) + ↓ ABCI (InitChain, CheckTx, FinalizeBlock, Commit, ...) +BaseApp + ↓ orchestrates block execution +ModuleManager + ↓ dispatches to individual modules +Modules (x/auth, x/bank, x/staking, ...) + ↓ read/write +State (KVStores) +``` + +CometBFT drives the block lifecycle by calling ABCI methods on `BaseApp`. `BaseApp` handles each call, delegating to registered lifecycle hooks and routing messages to the appropriate module handlers. Modules contain the business logic, and KVStores hold the resulting state. + +## Key fields + +[`BaseApp`](https://github.com/cosmos/cosmos-sdk/blob/main/baseapp/baseapp.go) is defined in `baseapp/baseapp.go`. It holds references to everything needed to run a chain: + +```go +type BaseApp struct { + logger log.Logger + name string // application name from abci.BlockInfo + db dbm.DB // common DB backend + cms storetypes.CommitMultiStore // Main (uncached) state + storeLoader StoreLoader // function to handle store loading + grpcQueryRouter *GRPCQueryRouter // router for redirecting gRPC query calls + msgServiceRouter *MsgServiceRouter // router for redirecting Msg service messages + txDecoder sdk.TxDecoder // unmarshal []byte into sdk.Tx + mempool mempool.Mempool + anteHandler sdk.AnteHandler // ante handler for fee and auth + postHandler sdk.PostHandler // post handler, optional + // ... + sealed bool + // ... + chainID string + // ... +} +``` + +For a complete list of fields, see the [`BaseApp` struct definition](https://github.com/cosmos/cosmos-sdk/blob/main/baseapp/baseapp.go). + +- `cms` (CommitMultiStore): the root state store. All module substores are mounted here, and all state reads and writes during block execution pass through it. +- `storeLoader`: a function that opens and mounts the individual module stores at application startup. +- `grpcQueryRouter`: routes incoming gRPC queries to the correct module's query handler. +- `msgServiceRouter`: routes each message in a transaction to the correct module's `MsgServer` handler. +- `txDecoder`: decodes raw transaction bytes from CometBFT into an `sdk.Tx`. +- `anteHandler`: runs before message execution to handle cross-cutting concerns: signature verification, sequence validation, and fee deduction. +- `postHandler`: optional middleware that runs after message execution — used for tasks such as tipping or post-execution state adjustments. +- `sealed`: set to `true` after `LoadLatestVersion` is called. Setter methods panic if called after sealing. + +## Initialization and sealing + +`BaseApp` enforces a configuration lifecycle: setter methods must be called before `LoadLatestVersion` is invoked. When `LoadLatestVersion` runs, it validates required components, initializes the check state, and sets `sealed` to `true`. Any setter called after sealing panics. On first launch, CometBFT calls `InitChain`. It stores `ConsensusParams` from the genesis file — block gas limit, max block size, evidence rules — in the `ParamStore`, where they can later be adjusted via on-chain governance. It initializes all volatile states by branching the root store, sets the block gas meter to infinite so genesis transactions are not gas-constrained, and calls the application's `initChainer`, which runs each module's `InitGenesis` to populate initial state. How this shapes the structure of `app.go` is covered in the next section. + + +## Transaction decoding + +Transactions arrive from CometBFT as raw bytes. Before `BaseApp` can validate or execute them, it must decode them into the SDK's transaction type using the `TxDecoder`: + +``` +[]byte tx + ↓ +TxDecoder + ↓ +sdk.Tx +``` + +This step happens before the transaction enters the execution pipeline. Without it, `BaseApp` cannot inspect messages, run the `AnteHandler`, or route execution to the correct module. + +## Execution modes + +`BaseApp` does not execute everything against the same mutable state. It maintains branched, copy-on-write views of the committed root state for different execution contexts: + +- `CheckTx` (`ExecModeCheck`): validates a transaction before it enters the mempool, without committing state. +- `FinalizeBlock` (`ExecModeFinalize`): executes transactions in a proposed block against a branched state that is committed at the end. +- `PrepareProposal` (`ExecModePrepareProposal`): runs when the node is the block proposer, assembling a candidate block. Executes against a branched state that is never committed. +- `ProcessProposal` (`ExecModeProcessProposal`): runs on every validator to validate an incoming proposal. Also executes against a branched state that is never committed. +- `Simulate` (`ExecModeSimulate`): runs a transaction for gas estimation without committing state. + +This separation ensures that validation, proposal handling, and simulation cannot accidentally mutate committed application state. + +## The transaction execution pipeline + +When `BaseApp` processes a transaction, it runs through a structured pipeline: + +``` +RunTx + ├─ DecodeTx → raw bytes → sdk.Tx + ├─ AnteHandler → signatures, sequence, fees, gas setup + ├─ RunMsgs → route each message to the correct module handler + └─ PostHandler → optional post-execution middleware +``` + +If the `AnteHandler` fails, message execution does not begin. If any message fails, message execution reverts atomically; all message writes commit or none do. + +## `AnteHandler` + +The `AnteHandler` is middleware that runs before any message in a transaction executes. It verifies cryptographic signatures, validates and increments the account sequence number, deducts transaction fees, and sets up the gas meter for the transaction. + +For the application wiring side, including `SetAnteHandler`, `HandlerOptions`, and constructor ordering, see [Mounting stores and setting hooks in `app.go`](/sdk/v0.54/learn/concepts/app-go#mounting-stores-and-setting-hooks). + +If the `AnteHandler` fails, the transaction is rejected and its messages never execute. If the `AnteHandler` succeeds but a message later fails, the `AnteHandler`'s state writes, such as fee deduction and sequence increment for ordered transactions, are already flushed to `finalizeBlockState` and will be committed with the block. Fees are charged even for transactions whose messages fail. + +`BaseApp.runTx()` also handles Go panics that occur during execution — for example, when a keeper encounters an invalid state. By default, panics are caught and logged as errors. Applications can register custom panic recovery logic via `BaseApp.AddRunTxRecoveryHandler`, which adds a `RecoveryHandler` to the chain. See [ADR-022](/sdk/v0.54/reference/architecture/adr-022-custom-panic-handling) and [`baseapp/recovery.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/baseapp/recovery.go) for details. + +## Message routing + +When a transaction contains messages, `BaseApp` routes each one to the appropriate module handler using the `MsgServiceRouter`. + +```go +type MsgServiceRouter struct { + routes map[string]MsgServiceHandler + // ... +} +``` + +The routing process has three steps: + +1. Registration: During app startup, each module calls `RegisterService`, which registers its message handlers keyed by message type URL (e.g., `/cosmos.bank.v1beta1.MsgSend`). +2. Lookup: At execution time, `Handler` looks up the registered handler for the incoming message's type URL. +3. Execution: The retrieved handler invokes the module's `MsgServer` implementation, which validates inputs, applies business rules, and updates state through the keeper. + +This routing is entirely type-URL-based. Modules do not need to know about each other at the routing level; `BaseApp` is the neutral coordinator. + +## Queries + +For read-only access to application state, `BaseApp` uses the **`GRPCQueryRouter`** to route incoming gRPC queries to the correct module query service. Queries bypass the transaction execution pipeline and directly read committed state. They do not go through the `AnteHandler`, do not consume gas in the same way, and do not mutate state. + +## Store management + +`BaseApp` owns the `CommitMultiStore` that holds all module state. At app startup, each module registers its store key, and `BaseApp` mounts the corresponding store: + +```go +app.MountKVStores(keys) +``` + +Before executing each transaction, `BaseApp` creates a cached, copy-on-write view of the multistore. All writes during that transaction occur in the cache. If the transaction succeeds, the cache is flushed to the underlying store. If the transaction fails at any point, the cache is discarded and no state changes are applied. + +## CheckTx and mempool validation + +Before a transaction reaches block execution, it goes through `CheckTx`. `BaseApp` runs the `AnteHandler` in `CheckTx` mode to validate signatures, check sequence numbers, and verify fees. Each validator also enforces a configurable `minGasPrices` floor, and transactions offering less than the minimum gas price are rejected here as a spam protection measure. Transactions that fail `CheckTx` are rejected and do not enter the mempool. + +`CheckTx` does not execute messages. It does run the `AnteHandler`, and if ante succeeds the resulting writes are persisted to `BaseApp`'s internal `CheckTx` state rather than to committed chain state. This is how the mempool tracks transaction validity before block execution. After each block commits, CometBFT triggers a recheck pass (`ReCheckTx`) that re-validates all pending mempool transactions against the new state, and any transactions that became invalid (for example, because their sequence number was consumed by a competing transaction) are evicted at this point. + +## Coordinating block execution + +When CometBFT calls `FinalizeBlock`, `BaseApp` runs the full block execution pipeline in order: + +``` +FinalizeBlock + ├─ PreBlock → module pre-block hooks + ├─ BeginBlock → module begin-block hooks + ├─ For each transaction: + │ ├─ AnteHandler (signature verification, fee deduction, gas setup) + │ ├─ Message routing and execution + │ └─ Commit or revert (atomic per-transaction) + └─ EndBlock → module end-block hooks + → returns AppHash +``` + +[`PreBlock`](/sdk/v0.54/learn/concepts/lifecycle#preblock) runs before any block logic. It handles changes that must take effect before the block begins, such as activating a chain upgrade or modifying consensus parameters. + +[`BeginBlock`](/sdk/v0.54/learn/concepts/modules#beginblock) runs after [`PreBlock`](/sdk/v0.54/learn/concepts/lifecycle#preblock) and handles per-block housekeeping: minting inflation rewards, distributing staking rewards, resetting per-block counters. + +[Transactions](/sdk/v0.54/learn/concepts/transactions) execute sequentially in block order. Message execution for each transaction is atomic: if any message fails, the message execution branch reverts. `AnteHandler` side effects may already have been applied. + +[`EndBlock`](/sdk/v0.54/learn/concepts/modules#endblock) runs after all transactions. It handles logic that depends on the block's cumulative state — for example, tallying governance votes after all vote messages have been processed, or updating validator power after all delegation changes. + +After `FinalizeBlock` completes, `BaseApp` computes and returns the app hash — the Merkle root of all committed state. See [App hash](/sdk/v0.54/learn/concepts/store#app-hash) for how it relates to the multistore and deterministic execution. When CometBFT subsequently calls `Commit`, `BaseApp` writes `finalizeBlockState` to the root store, resets `checkState` to the newly committed state, and clears `finalizeBlockState` to `nil` in preparation for the next block. + +## Module Manager + +`BaseApp` exposes `PreBlock`, `BeginBlock`, and `EndBlock` as lifecycle hook points. Every standard SDK application wires these to a `ModuleManager`, which holds the full set of registered modules and their execution ordering. When a hook fires, `ModuleManager` iterates its ordered module list and calls each module's corresponding hook in sequence. Ordering matters: some modules depend on others having already updated state before they run. + +The `app.go` page shows how the application constructs the `ModuleManager`, wires it into `BaseApp`, and configures ordering in practice. See [Module Manager in `app.go`](/sdk/v0.54/learn/concepts/app-go#module-manager). + +## Block proposal and vote extensions + +[ABCI 2.0](/sdk/v0.54/guides/abci/abci#abci-20) added a proposal phase that runs during consensus rounds, before `FinalizeBlock` executes. `BaseApp` exposes four handlers for this phase, with default implementations wired at construction: + +- `PrepareProposal`: called on the current block proposer to assemble a block from the mempool. The default selects transactions up to the block gas limit. Chains can override this to implement custom ordering, filtering, or injection of protocol-level transactions. +- `ProcessProposal`: called on every validator to validate an incoming proposal. The default accepts any structurally valid proposal. Chains that use `PrepareProposal` to inject data typically also override this to verify that data is present and valid. +- `ExtendVote` / `VerifyVoteExtension`: allow validators to attach arbitrary data to their precommit votes and verify other validators' extensions. One major use case is oracle price feeds: validators inject off-chain data into consensus so it becomes available on-chain at block start. + +All four are configurable in `app.go` via `SetPrepareProposal`, `SetProcessProposal`, `SetExtendVoteHandler`, and `SetVerifyVoteExtensionHandler`. Chains that do not need custom behavior can leave the defaults in place. + +## Putting it all together + +`BaseApp` is the execution engine of a Cosmos SDK chain: + +``` +CometBFT → ABCI → BaseApp → Modules → State +``` + +It implements ABCI, coordinates the block lifecycle (`PreBlock` → `BeginBlock` → transactions → `EndBlock`), routes messages to module handlers via the `MsgServiceRouter`, routes queries via the `GRPCQueryRouter`, runs the `AnteHandler` before each transaction, and manages the multistore with copy-on-write caching for atomicity. State changes are committed at block end; validation and simulation run against branched state and never touch committed data. + +The next section, [app.go Overview](/sdk/v0.54/learn/concepts/app-go), explains how `BaseApp` is instantiated, configured, and wired with modules to produce a complete, running chain. diff --git a/sdk/v0.54/learn/concepts/cli-grpc-rest.mdx b/sdk/v0.54/learn/concepts/cli-grpc-rest.mdx new file mode 100644 index 000000000..897b08797 --- /dev/null +++ b/sdk/v0.54/learn/concepts/cli-grpc-rest.mdx @@ -0,0 +1,320 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/learn/concepts/cli-grpc-rest' +title: CLI, gRPC, and REST API +--- + +A Cosmos SDK chain exposes three external interfaces for interacting with it: a command-line interface (CLI), a gRPC API, and a REST API. Each is a different surface over the same underlying chain logic. Users and developers can choose whichever interface suits their use case without affecting how the chain processes or validates transactions. + +## How users interact with a chain + +Every operation a user performs falls into one of two categories: + +- **Transactions**: state-changing operations broadcast to the network and included in blocks (send tokens, delegate stake, vote on a proposal) +- **Queries**: read-only requests that return data from the current chain state without going through consensus + +Both categories are accessible through the CLI, gRPC, and REST interfaces. The interfaces differ in how requests are constructed and transmitted, not in what they can do. + +None of these interfaces affect consensus. Transactions are validated and ordered by the consensus engine (CometBFT); the interfaces are simply delivery mechanisms that carry signed transactions to the network and return results. + +For the transaction and query model underneath these interfaces, see [Transactions, Messages, and Queries](/sdk/v0.54/learn/concepts/transactions). + +## Interface comparison + +All endpoints default to `localhost` and must be configured to be accessible over the public internet. + +| Interface | Default port | Best for | Notes | +|---|---|---|---| +| **CLI** | — | Development, testing, and node operations | Best for operator and developer workflows | +| **gRPC** | 9090 | Wallets, backend services, and SDK clients | Not supported in browsers (requires HTTP/2) | +| **REST** | 1317 | Web applications, scripts, and environments without gRPC support | Use when gRPC is unavailable; REST is disabled by default | +| **CometBFT RPC** | 26657 | Consensus and blockchain data queries | Limited to consensus-layer data | + +## CLI + +The CLI is the primary tool for developers and operators interacting with a chain from the terminal. Most Cosmos SDK chains ship a single binary that acts as both the server process and the CLI client. It is common to append a `d` suffix to the binary name to indicate that it is a daemon process, such as `exampled` or `simd`. + +For a hands-on walkthrough of running a local chain and using the CLI, see the [Running and Testing](/sdk/v0.54/tutorials/example/05-run-and-test) tutorial. + +When used as a client, the CLI constructs a transaction or query, signs it if required, and submits it through the node client interface. + +To learn how to run a local node and use the CLI, see [Run a Local Node](/sdk/v0.54/node/prerequisites). + +### Using the CLI + +CLI commands are organized into two categories: + +- `query` commands retrieve information from chain state +- `tx` commands construct and broadcast transactions + +Example commands: + +``` +exampled query counter count +``` + +``` +exampled tx counter add 10 \ + --from mykey \ + --chain-id example-1 \ + --gas auto \ + --gas-adjustment 1.3 \ + --fees 1000stake +``` + +- `--from` specifies the signing key +- `--gas auto` asks the CLI to estimate gas usage +- `--gas-adjustment` applies a safety multiplier to the estimate +- `--fees` specifies the transaction fee + +Gas limits the computational work a transaction can perform. The full gas model is explained in [Execution Context, Gas, and Events](/sdk/v0.54/learn/concepts/context-gas-events). + +For a full CLI reference for the example chain, see [CLI reference](/sdk/v0.54/tutorials/example/05-run-and-test#cli-reference) in the Running and Testing tutorial. + +### How modules expose CLI commands with `AutoCLI` + +In modern Cosmos SDK applications, modules expose CLI commands through **`AutoCLI`**. `AutoCLI` reads a module's protobuf service definitions and generates CLI commands automatically, without requiring modules to hand-write Cobra command boilerplate. The counter snippets in this section are from the minimal counter module example. See the [Build a Module from Scratch](/sdk/v0.54/tutorials/example/03-build-a-module) tutorial. + +A module opts into `AutoCLI` by implementing `AutoCLIOptions()` on its `AppModule`: + +```go +func (a AppModule) AutoCLIOptions() *autocliv1.ModuleOptions { + return &autocliv1.ModuleOptions{ + Query: &autocliv1.ServiceCommandDescriptor{ + Service: "example.counter.Query", + EnhanceCustomCommand: true, + RpcCommandOptions: []*autocliv1.RpcCommandOptions{ + { + RpcMethod: "Count", + Use: "count", + Short: "Query the current counter value", + }, + }, + }, + Tx: &autocliv1.ServiceCommandDescriptor{ + Service: "example.counter.Msg", + EnhanceCustomCommand: true, + RpcCommandOptions: []*autocliv1.RpcCommandOptions{ + { + RpcMethod: "Add", + Use: "add [amount]", + Short: "Add to the counter", + PositionalArgs: []*autocliv1.PositionalArgDescriptor{{ProtoField: "add"}}, + }, + }, + }, + } +} +``` + +`AutoCLI` uses this configuration to generate the `exampled tx counter add` and `exampled query counter count` commands. The `Service` field names the protobuf service, and `RpcCommandOptions` maps individual RPC methods to CLI subcommands with positional arguments, flags, and help text. + +The `AutoCliOpts()` method on the application struct collects these options from all modules and passes them to the `AutoCLI` framework at startup: + +```go +func (app *ExampleApp) AutoCliOpts() autocli.AppOptions { + modules := make(map[string]appmodule.AppModule) + for _, m := range app.ModuleManager.Modules { + if moduleWithName, ok := m.(module.HasName); ok { + moduleName := moduleWithName.Name() + if appModule, ok := moduleWithName.(appmodule.AppModule); ok { + modules[moduleName] = appModule + } + } + } + + return autocli.AppOptions{ + Modules: modules, + ModuleOptions: runtimeservices.ExtractAutoCLIOptions(app.ModuleManager.Modules), + AddressCodec: authcodec.NewBech32Codec(sdk.GetConfig().GetBech32AccountAddrPrefix()), + ValidatorAddressCodec: authcodec.NewBech32Codec(sdk.GetConfig().GetBech32ValidatorAddrPrefix()), + ConsensusAddressCodec: authcodec.NewBech32Codec(sdk.GetConfig().GetBech32ConsensusAddrPrefix()), + } +} +``` + +This collects module options and address codecs and hands them to `AutoCLI`, which wires the generated commands into the root command. + +## gRPC + +gRPC is the primary programmatic interface for interacting with a Cosmos chain. It uses Protocol Buffers to define strongly typed request and response structures and supports generated clients for many programming languages. + +Each module exposes its functionality through two protobuf services: + +- A `Query` service for read-only access to module state +- A `Msg` service for state-changing operations + +These services are defined in the module's `query.proto` and `tx.proto` files. The protobuf definitions for the Cosmos SDK are published at [buf.build/cosmos/cosmos-sdk](https://buf.build/cosmos/cosmos-sdk). + +### How modules expose gRPC services + +Modules register their gRPC services during application startup via `RegisterServices`: + +```go +app.configurator = module.NewConfigurator(app.appCodec, app.MsgServiceRouter(), app.GRPCQueryRouter()) +err := app.ModuleManager.RegisterServices(app.configurator) +``` + +Each module implements `RegisterServices` to connect service implementations to the application routers: + +```go +func (am AppModule) RegisterServices(cfg module.Configurator) { + types.RegisterMsgServer(cfg.MsgServer(), keeper.NewMsgServerImpl(am.keeper)) + types.RegisterQueryServer(cfg.QueryServer(), keeper.NewQueryServer(am.keeper)) +} +``` + +`RegisterMsgServer` routes incoming `Msg` service calls to the module's `MsgServer` implementation. `RegisterQueryServer` routes incoming `Query` service calls to the module's `QueryServer` implementation. + +### How to interact with gRPC + +Connect to the node's gRPC endpoint (default: `localhost:9090`) using a generated client: + +```go +conn, _ := grpc.NewClient("localhost:9090", grpc.WithTransportCredentials(insecure.NewCredentials())) +queryClient := countertypes.NewQueryClient(conn) + +resp, _ := queryClient.Count(ctx, &countertypes.QueryCountRequest{}) +``` + +The gRPC server can be configured in `app.toml`: + +- `grpc.enable = true|false` — enables or disables the gRPC server (default: `true`) +- `grpc.address = {string}` — the `ip:port` the server binds to (default: `localhost:9090`) +- `grpc.max-recv-msg-size` — maximum message size in bytes the server can receive (default: 10MB) +- `grpc.max-send-msg-size` — maximum message size in bytes the server can send (default: `math.MaxInt32`) + +For archive node setups, `grpc.historical-grpc-address-block-range` maps gRPC backend addresses to inclusive block height ranges, so historical queries are routed to the node holding that slice of chain history. The value is a JSON string, for example: `'{"archive-node-1:9090": [0, 1000000]}'`. Leave it empty (the default) to disable. + +For more usage examples, see [Interact with the Node](/sdk/v0.54/node/interact-node#using-grpc). + +## REST via gRPC-gateway + +The Cosmos SDK also exposes a REST API. REST endpoints are not written by hand; they are generated automatically from the same protobuf definitions used by gRPC, using **gRPC-gateway**. + +gRPC-gateway reads HTTP annotations in the `.proto` files and generates a reverse proxy that translates REST requests into gRPC calls: + +```protobuf +service Query { + rpc Count(QueryCountRequest) returns (QueryCountResponse) { + option (google.api.http) = { + get: "/example/counter/v1/count" + }; + } +} +``` + +This annotation causes gRPC-gateway to generate a `GET /example/counter/v1/count` HTTP endpoint. The gateway receives the HTTP request, marshals it into a `QueryCountRequest`, calls the gRPC `Count` handler, and returns the response as JSON. + +### Registering REST routes + +REST routes are registered in `RegisterAPIRoutes`: + +```go +func (app *ExampleApp) RegisterAPIRoutes(apiSvr *api.Server, apiConfig config.APIConfig) { + clientCtx := apiSvr.ClientCtx + // Register new tx routes from grpc-gateway. + authtx.RegisterGRPCGatewayRoutes(clientCtx, apiSvr.GRPCGatewayRouter) + // Register new CometBFT queries routes from grpc-gateway. + cmtservice.RegisterGRPCGatewayRoutes(clientCtx, apiSvr.GRPCGatewayRouter) + // Register node gRPC service for grpc-gateway. + nodeservice.RegisterGRPCGatewayRoutes(clientCtx, apiSvr.GRPCGatewayRouter) + // Register grpc-gateway routes for all modules. + app.BasicModuleManager.RegisterGRPCGatewayRoutes(clientCtx, apiSvr.GRPCGatewayRouter) +} +``` + +The REST server can be configured in `app.toml`: + +- `api.enable = true|false` — enables or disables the REST server (default: `false`) +- `api.address = {string}` — the `ip:port` the server binds to (default: `tcp://localhost:1317`) + +### Swagger + +When the REST server and Swagger are both enabled, the node exposes a Swagger (OpenAPI v2) specification at `http://localhost:1317/swagger/`. Swagger lists all REST endpoints, request parameters, and response schemas, and provides a browser-based interface for exploring the REST API. + +Both are disabled by default. Enable them in `app.toml`: + +``` +api.enable = true +api.swagger = true +``` + +To generate Swagger documentation for your own custom modules, see the [`proto-swagger-gen` script](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/scripts/protoc-swagger-gen.sh) in the Cosmos SDK. + +## CometBFT RPC + +CometBFT also exposes its own RPC server, independent of the Cosmos SDK. It serves consensus and blockchain data and is configured under the `rpc` table in `config.toml` (default: `tcp://localhost:26657`). An OpenAPI specification of all CometBFT RPC endpoints is available in the [CometBFT documentation](/cometbft/latest/docs/core/RPC). + +Some CometBFT RPC endpoints are directly related to the Cosmos SDK: + +- `/abci_query` — queries the application for state. The `path` parameter accepts: + - any protobuf fully-qualified service method, for example `/cosmos.bank.v1beta1.Query/AllBalances` + - `/app/simulate` — simulate a transaction and return gas usage + - `/app/version` — return the application version + - `/store/{storeName}/key` — direct key lookup in a named store + - `/store/{storeName}/subspace` — prefix scan in a named store + - `/p2p/filter/addr/{addr}` and `/p2p/filter/id/{id}` — filter peers by address or node ID +- `/broadcast_tx_sync`, `/broadcast_tx_async`, `/broadcast_tx_commit` — broadcast a signed transaction to peers. The CLI, gRPC, and REST interfaces all use these CometBFT RPCs under the hood. + +Two gRPC methods on the CometBFT service return ABCI block results: `GetBlockResults` (by height) and `GetLatestBlockResults`. These expose `finalize_block_events` and per-transaction results. + + +## End-to-end interaction flow + +To illustrate how these interfaces connect, here is the path of a `counter add` transaction from the user's terminal to a state change on the chain. This example follows the minimal counter module example's CLI shape. See the [Build a Module from Scratch](/sdk/v0.54/tutorials/example/03-build-a-module#step-9-autocli) tutorial. + +``` +User runs: exampled tx counter add 10 --from mykey --chain-id example-1 + ↓ +CLI (AutoCLI generated command) + Constructs MsgAddRequest{Sender: mykey, Add: 10} + Signs the transaction with mykey + Encodes to protobuf bytes + ↓ +Broadcast through the node client interface + ↓ +Node: CheckTx + AnteHandler verifies signature, deducts fee, meters gas + Transaction enters the mempool + ↓ +CometBFT: block proposal and consensus + ↓ +FinalizeBlock: transaction executed + AnteHandler runs again (finalizeBlock mode) + MsgServiceRouter routes MsgAddRequest → counter module MsgServer + MsgServer.Add calls keeper.AddCount + Keeper reads current count, adds 10, writes new count + ↓ +Commit: state change persisted + ↓ +User receives TxResponse with code 0 +``` + +A query follows a shorter path that bypasses consensus entirely: + +``` +User runs: exampled query counter count + ↓ +CLI (AutoCLI generated command) + Constructs QueryCountRequest{} + Sends directly to node gRPC query endpoint + ↓ +GRPCQueryRouter routes to counter QueryServer + QueryServer.Count calls keeper.GetCount + Keeper reads current count from store + ↓ +QueryCountResponse{Count: 10} returned to user +``` + +Queries do not enter the mempool, are not included in blocks, and do not pass through the `AnteHandler`. They read committed state and return immediately. + +## Interfaces and consensus + +The CLI, gRPC, and REST interfaces are transport layers. They construct, sign, and deliver messages, but they do not participate in consensus and cannot affect the determinism of block execution. + +- Transactions become part of consensus only after they pass `CheckTx` and are included in a proposed block. The interface used to submit the transaction has no bearing on how it is validated or ordered. +- Queries bypass the transaction pipeline entirely. They read committed state from a node and never reach the consensus engine. +- Any node in the network can serve queries or accept transaction submissions. The result is always the same committed state, regardless of which node or which interface is used. + +This separation means that changing the CLI or REST surface of a module (renaming a command, adding a new query) never requires a chain upgrade. Only changes to message types, keeper logic, or state schema affect consensus. The next section, [Testing in the SDK](/sdk/v0.54/learn/concepts/testing), shows how to test those behaviors once they are wired up. diff --git a/sdk/v0.54/learn/concepts/context-gas-events.mdx b/sdk/v0.54/learn/concepts/context-gas-events.mdx new file mode 100644 index 000000000..5bb08c827 --- /dev/null +++ b/sdk/v0.54/learn/concepts/context-gas-events.mdx @@ -0,0 +1,203 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/learn/concepts/context-gas-events' +title: Execution Context, Gas, and Events +--- + +In the previous section, [Encoding and Protobuf](/sdk/v0.54/learn/concepts/encoding) explained how data is serialized and why every validator must encode state identically. This page covers the runtime environment that modules execute within: the context object that carries block metadata and state access, the gas system that limits computation, and the event system that allows modules to emit observable signals. + +## What is `sdk.Context` + +Every message handler, keeper method, and block hook in the Cosmos SDK receives an `sdk.Context`. It is the execution environment for a single unit of work (a transaction, a query, or a block hook) and carries everything that code needs to read state, emit events, and consume gas. Rather than passing the store, gas meter, and block header as separate arguments to every function, `Context` bundles them into a single value. + +The `Context` struct is defined in [`types/context.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/types/context.go): + +```go +type Context struct { + ms storetypes.MultiStore + chainID string + gasMeter storetypes.GasMeter + blockGasMeter storetypes.GasMeter + eventManager EventManagerI + // ... additional fields +} +``` + +Context is a value type. It is passed by value and mutated through `With*` methods that return a new copy. This means a module can safely derive a sub-context (for example, with a different gas meter) without affecting the caller's context. + +### Block metadata + +Context exposes read-only access to the current block's metadata (see [`types/context.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/types/context.go)): + +- `ctx.BlockHeight()` returns the current block number. +- `ctx.BlockTime()` returns the block's timestamp. +- `ctx.ChainID()` returns the chain identifier string. +- `ctx.Logger()` returns a structured logger scoped to the current execution context. Modules use this for operational logging (e.g., logging an upgrade activation or an unexpected state) without affecting consensus. + +These values are populated by [`BaseApp`](/sdk/v0.54/learn/concepts/baseapp) from the block header provided by CometBFT before any block logic runs. Modules read them to implement time-dependent logic (for example, checking whether a vesting period has elapsed) or to tag events with the block height. + +`ctx.IsCheckTx()` returns true when the context is being used for mempool validation rather than block execution. For finer-grained branching, `ctx.ExecMode()` returns the precise execution mode: `ExecModeCheck`, `ExecModeReCheck`, `ExecModeSimulate`, `ExecModePrepareProposal`, `ExecModeProcessProposal`, `ExecModeFinalize`, and others (see [`types/context.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/types/context.go#L21) for more details). Modules that need to behave differently during simulation or proposal handling use `ExecMode()` instead of `IsCheckTx()`. + +### Context and state access + +State is accessed through context. The context holds a reference to the multistore, and each keeper opens its own store through the context: + +```go +func (k Keeper) GetCount(ctx context.Context) (uint64, error) { + return k.counter.Get(ctx) +} +``` + +The keeper does not hold a direct reference to the live multistore; it opens its module's store from the context on each call. This is why context must be passed to every keeper method: it is the gateway to the current block's state, the gas meter, and the event manager for that execution unit. + +### Atomic sub-execution with `CacheContext` + +Modules that need to attempt a sub-operation and revert it on failure can call [`ctx.CacheContext()`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/types/context.go#L412), which returns a branched copy of the context and a `writeCache` function. All state changes in the sub-operation go into the branch. Calling `writeCache()` flushes them to the parent context; not calling it discards them atomically. + +```go +cacheCtx, writeCache := ctx.CacheContext() +if err := doRiskyOperation(cacheCtx); err != nil { + return err // branch is discarded, no state changes applied +} +writeCache() // flush branch to parent context +``` + +## Gas metering + +### What gas measures + +Gas is a unit of computation. In the Cosmos SDK, gas accounts for both computation and state access. Every store read, store write, and iterator step costs gas. Complex computations such as signature verification in the `AnteHandler` also cost gas. + +The gas system exists to prevent abuse. Without a gas limit, a single transaction could exhaust a node's resources with an unbounded computation or an unindexed state scan. + +### Gas limit and the transaction gas meter + +Every transaction specifies a gas limit in its `auth_info.fee.gas_limit` field. When `BaseApp` begins executing a transaction, it creates a `GasMeter` initialized with that limit and attaches it to the context. + +The [`GasMeter`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/store/types/gas.go#L42) interface provides two key methods: + +```go +type GasMeter interface { + GasConsumed() Gas + ConsumeGas(amount Gas, descriptor string) + // ... +} +``` + +`GasConsumed` returns the total gas used so far in the current execution unit. `ConsumeGas` adds to the running total and panics with `ErrorOutOfGas` if consumption exceeds the limit. + +When submitting a transaction, users specify two of the three values `fees`, `gas`, and `gas-prices` — the third is derived from the equation `fees = gas * gas-prices`. The `gas` value becomes `GasWanted`: the maximum gas the transaction is allowed to consume. The actual gas consumed during execution is `GasUsed`. Both `GasWanted` and `GasUsed` are returned to CometBFT when `FinalizeBlock` completes. + +### How gas is consumed + +Gas is consumed automatically at the store layer. Every read and write through the [`GasKVStore`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/store/gaskv/store.go#L12) wrapper charges gas before delegating to the underlying store: + +- A `Get` (store read) charges a flat read cost plus a per-byte cost for the key and value. +- A `Set` (store write) charges a flat write cost plus a per-byte cost for the key and value. + +Modules do not need to manually track gas for ordinary state access — the store layer handles it automatically. Modules call `ctx.GasMeter().ConsumeGas(...)` directly only for computation costs that are not captured by store operations (for example, a module that performs a cryptographic operation outside the store). + +### When gas runs out + +If gas is exhausted during execution, `ConsumeGas` panics with `ErrorOutOfGas`. `BaseApp` recovers from this panic, discards the current message execution branch, and returns an error to the user. Fees may still be charged for the gas consumed up to the point of failure, and `AnteHandler` side effects may already have been applied before message execution started. + +### Block gas limit + +In addition to the per-transaction gas meter, there is a block-level gas meter that tracks total gas consumed by all transactions in a block. The block gas limit prevents a single block from consuming unbounded computation. If a transaction would cause the block's gas total to exceed the limit, it is excluded from the block. The block gas limit and minimum gas prices are configured in [`app.toml`](/sdk/v0.54/tutorials/example/05-run-and-test#apptoml). + +## Events + +### What events are + +Events are observable signals emitted during transaction and block execution. A module emits events to describe what happened: tokens were transferred, a validator was slashed, a governance proposal passed. Events carry structured key-value data alongside a type string. + +Events are not part of consensus state. They are not stored in the KVStore, do not affect the app hash, and are not required for deterministic execution. Instead, they are collected by `BaseApp` and included in the block result, where indexers, explorers, and relayers consume them. + +### EventManager + +Modules emit events through the [`EventManager`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/types/events.go#L25), which is attached to the context. + +The `EventManager` is created fresh for each transaction and collects all events emitted during that execution. + +### Standard event types + +The SDK automatically emits a `message` event for every transaction, with these attributes set by `BaseApp`: + +- `message.action` — the full type URL of the message (e.g., `/cosmos.bank.v1beta1.Msg/Send`) +- `message.module` — the module name, derived from the type URL +- `message.sender` — the signer address, if present + +These are defined as constants in [`types/events.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/types/events.go#L249-L263). Modules follow the same convention when emitting their own events. + +### Emitting events + +Modules emit events using [`EmitEvent`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/types/events.go#L35) or [`EmitTypedEvent`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/types/events.go#L58): + +```go +// emit an untyped event +ctx.EventManager().EmitEvent(sdk.NewEvent( + "increment", + sdk.NewAttribute("new_count", strconv.FormatUint(newCount, 10)), +)) +``` + +`EmitEvent` appends a raw key-value event to the manager's accumulated list. + +For events backed by protobuf message types, `EmitTypedEvent` serializes the message's fields into event attributes automatically: + +```go +ctx.EventManager().EmitTypedEvent(&types.EventCounterIncremented{ + NewCount: newCount, +}) +``` + +Using `EmitTypedEvent` is the modern approach. It provides type safety and makes the event schema explicit through proto definitions, allowing clients to deserialize events back into typed structs. + +### Block events and transaction events + +Events emitted during `BeginBlock` or `EndBlock` hooks are **block events**: they describe things that happened at the block level (inflation minted, validator updates applied). Events emitted inside a message handler are **transaction events**: they describe what a specific transaction did. + +Both types are included in the `FinalizeBlock` response that CometBFT returns to the network, but they are reported separately so clients can distinguish block-level activity from per-transaction activity. + +### Who consumes events + +Events are consumed outside the node: + +- **Block explorers** index events to show users what happened in a transaction (which tokens moved, which validator was slashed, which proposal passed). +- **Relayers** (IBC) subscribe to specific event types to detect packet sends and acknowledgments. +- **Indexers and off-chain services** build queryable databases of chain activity from event streams. Events can also be queried via the node's REST API and WebSocket endpoint. +- **Wallets and UIs** display event data to users as transaction receipts. + +Events are included in the block result that CometBFT returns after each block. They are not replayed or reprocessed; once a block is finalized, its events are fixed. + +### Querying events + +Events are indexed using the format `{type}.{key}={value}` and can be filtered when querying transactions. String values must be wrapped in single quotes. + +| Filter | Description | +|---|---| +| `tx.height=23` | All transactions at block height 23 | +| `message.action='/cosmos.bank.v1beta1.Msg/Send'` | Transactions containing a bank Send message | +| `message.module='bank'` | Transactions from the x/bank module | + +## Putting it together + +During transaction execution, context, gas, and events work together as the runtime layer: + +``` +BaseApp creates Context for the transaction + ↓ +AnteHandler runs + → signature verification, fee deduction, gas meter initialized + ↓ +Message handler runs + → each store read/write consumes gas via GasKVStore + → module logic emits events via EventManager + ↓ +If gas exhausted → panic → state reverted, fees charged for gas consumed +If execution succeeds → state changes committed, events returned in block result +``` + +The context carries the gas meter and event manager into every keeper call. Gas is consumed transparently at the store layer. Events accumulate and are returned as part of the block result once execution completes. + +The next section, [Intro to SDK Structure](/sdk/v0.54/learn/concepts/sdk-structure), explains how an SDK application is structured as a codebase: where modules live, what goes in `app/`, and how all the pieces are assembled. diff --git a/sdk/v0.54/learn/concepts/encoding.mdx b/sdk/v0.54/learn/concepts/encoding.mdx new file mode 100644 index 000000000..28468b1f1 --- /dev/null +++ b/sdk/v0.54/learn/concepts/encoding.mdx @@ -0,0 +1,411 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/learn/concepts/encoding' +title: Protobuf and Signing +--- + +As described in [State, Storage, and Genesis](/sdk/v0.54/learn/concepts/store), modules write structured state values into the KV store as raw bytes. Encoding defines how those structured values are serialized into bytes, and why every validator must produce exactly the same bytes. This page explains how that encoding works, why the Cosmos SDK chose Protocol Buffers, and what that means for module development. + +## What is Protobuf? + +[Protocol Buffers](https://protobuf.dev/) (protobuf) is a language-neutral, binary serialization format developed by Google. You define your data structures in `.proto` files using a schema language, then generate code in your target language from that schema. The generated code handles serialization (converting structured data into bytes) and deserialization (converting bytes back into structured data). + +A simple protobuf message looks like this: + +```proto +message MsgSend { + string from_address = 1; + string to_address = 2; + repeated Coin amount = 3; +} +``` + +Each field has a name, a type, and a field number. The field numbers are what protobuf actually uses during encoding; field names are only present in the schema, not in the serialized bytes. + +## Why the Cosmos SDK uses protobuf + +The Cosmos SDK uses protobuf for a fundamental reason: consensus requires determinism. + +Every validator in the network independently executes each block. After execution, each validator computes the [app hash](/sdk/v0.54/learn/concepts/store#app-hash), a cryptographic hash of the application state. For validators to agree on the app hash, they must all produce exactly the same bytes for every piece of state they write. + +Protobuf alone does not guarantee this. The Cosmos SDK uses protobuf **with additional deterministic encoding rules** formalized in [ADR-027 (Deterministic Protobuf Serialization)](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/docs/architecture/adr-027-deterministic-protobuf-serialization.md). ADR-027 specifies constraints such as requiring fields to appear in ascending field-number order and varint encodings to be as short as possible. The SDK validates incoming transactions against these rules before processing them, so a non-deterministically encoded transaction is rejected rather than producing divergent state. Every validator encoding the same data under these rules produces an identical byte sequence. + +Beyond determinism, protobuf provides: + +- **Compact encoding**: binary wire format is smaller than JSON or XML, which matters for transaction throughput and block size. +- **Schema evolution**: fields can be added or deprecated without breaking existing clients, which is critical for chain upgrades. +- **Code generation**: `.proto` files generate Go structs, gRPC service stubs, and REST gateway handlers automatically. +- **Cross-language support**: clients in any language can interact with the chain by generating code from the same `.proto` files. + +## Binary and JSON encoding + +The Cosmos SDK uses protobuf in two encoding modes: + +**Binary encoding** is the default for everything that participates in consensus: transactions written to blocks, state stored in KV stores, and genesis data. Binary encoding is compact and deterministic. When a transaction is broadcast to the network, it travels as protobuf binary. When a module writes state, it serializes values to protobuf binary before calling `Set` on the store. + +**JSON encoding** is used for human-readable output: the [CLI, gRPC-gateway REST endpoints](/sdk/v0.54/learn/concepts/cli-grpc-rest), and off-chain tooling. The Cosmos SDK uses protobuf's JSON encoding (`ProtoMarshalJSON`) rather than standard Go JSON, which preserves field names from the `.proto` schema and handles special types like `Any` correctly. + +It is important to keep in mind that **binary encoding is consensus-critical**. Two validators must produce identical binary bytes for identical data. JSON is only used where humans or external clients need to read the data; it never influences the AppHash. + +```text +Consensus-critical path Human-readable path +───────────────────────── ───────────────────────── +Transaction bytes (binary) CLI output (JSON) +State KV values (binary) REST API responses (JSON) +Genesis KV state (binary) Block explorers (JSON) +``` + +Note: genesis data is distributed as JSON in `genesis.json`, but during chain initialization `InitGenesis` deserializes that JSON into protobuf structs and writes them to the KV store as binary. The KV store (and therefore the AppHash) only ever contains the binary form. + +## Transaction encoding + +Transactions are protobuf messages defined in [`cosmos.tx.v1beta1`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/proto/cosmos/tx/v1beta1/tx.proto). A transaction is composed of three parts: + +```text +Tx + ├─ TxBody + │ └─ repeated google.protobuf.Any messages + ├─ AuthInfo + │ ├─ repeated SignerInfo (each with sequence) + │ └─ Fee + └─ repeated bytes signatures +``` + +- **TxBody** contains the messages to execute, serialized as `repeated google.protobuf.Any messages`. +- **AuthInfo** contains signer information (including the per-signer sequence number) and fee. +- **signatures** contains the cryptographic signatures, one per signer. + +Messages inside the transaction are stored as `google.protobuf.Any` values so that a single transaction can contain multiple message types from different modules. + +When a user submits a transaction, the SDK encodes it as a `TxRaw`—a flat structure with the `TxBody` bytes, `AuthInfo` bytes, and signatures already serialized. It then broadcasts that binary representation over the network. + +## Transaction signing and `SignDoc` + +Transactions are not signed directly. Instead, the SDK constructs a deterministic structure called a **`SignDoc`**, which defines exactly what bytes the signer commits to: + +```text +SignDoc + ├─ body_bytes (serialized TxBody) + ├─ auth_info_bytes (serialized AuthInfo, includes sequence per signer) + ├─ chain_id (prevents cross-chain replay) + └─ account_number (ties the signature to a specific on-chain account) +``` + +The `SignDoc` is serialized to protobuf binary and then signed with the user's private key: + +```text +signature = Sign(proto.Marshal(SignDoc)) +``` + +Because `SignDoc` is serialized deterministically, all validators verify the exact same bytes when checking transaction signatures. The per-signer sequence number lives in `AuthInfo.SignerInfo.sequence` and is included in `auth_info_bytes`, which is part of `SignDoc`—this is what prevents replay attacks. + +## Sign modes + +A **sign mode** determines what bytes a signer commits to when signing a transaction. The SDK supports multiple sign modes to accommodate different clients and hardware: + +- `SIGN_MODE_DIRECT` (default): the signer signs over the protobuf-binary-serialized `SignDoc` described above. This is compact, deterministic, and the correct choice for all new development. + +- `SIGN_MODE_LEGACY_AMINO_JSON`: the signer signs over an Amino JSON-encoded `StdSignDoc` instead of the protobuf `SignDoc`. This exists for backward compatibility with hardware wallets (e.g., older Ledger firmware) and client tooling that predates protobuf. New modules and chains should not depend on it. + +- `SIGN_MODE_TEXTUAL`: the signer signs over a human-readable CBOR-encoded representation of the transaction, designed to display legibly on hardware wallet screens (introduced in v0.50, see [ADR-050](/sdk/v0.54/reference/architecture/adr-050-sign-mode-textual)). This is the SDK's newer direction for human-readable signing on hardware wallets, intended to replace `SIGN_MODE_LEGACY_AMINO_JSON` over time. Its specification is versioned and has evolved across SDK releases. + +- `SIGN_MODE_DIRECT_AUX`: allows N-1 signers in a multi-signer transaction to sign over only `TxBody` and their own `SignerInfo`, without specifying fees. The designated fee payer signs last using `SIGN_MODE_DIRECT`. This simplifies multi-signature UX. + +The sign mode is negotiated at transaction construction time and does not affect how state is stored or how validators execute transactions. It only affects what bytes are signed. The full list of sign modes is defined in [`signing.proto`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/proto/cosmos/tx/signing/v1beta1/signing.proto#L17). + + +**For module developers:** `SIGN_MODE_DIRECT` requires no extra work. If you want your module's messages to be signable on Ledger hardware wallets using `SIGN_MODE_LEGACY_AMINO_JSON`, register your message types with the Amino codec via `RegisterLegacyAminoCodec` in your module's `codec.go`. + + +## Message signers + +Every transaction message must declare which addresses are authorized to sign it. In v0.50+, this is done via the `cosmos.msg.v1.signer` protobuf annotation — the SDK reads the annotation at startup and automatically extracts signer addresses from that field. See [Protobuf Annotations](/sdk/v0.54/guides/reference/protobuf-annotations) for the full annotation reference. + +For messages that cannot use the annotation — for example, messages with non-standard signing logic such as EVM-compatible transactions — you can register a custom signer function using [`signing.CustomGetSigner`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/x/tx/signing/context.go#L127): + +```go +signer := signing.CustomGetSigner{ + MsgType: proto.MessageName(&MyMsg{}), + Fn: func(msg proto.Message) ([][]byte, error) { + m := msg.(*MyMsg) + // extract and return signer address bytes + return [][]byte{m.SignerBytes()}, nil + }, +} +``` + +To register it, call `signingOptions.DefineCustomGetSigners(msgType, fn)` on the `txsigning.Options` you pass to `authtx.NewTxConfigWithOptions` when building your app's `TxConfig`. + +## How protobuf is used in modules + +Most public and persisted data types in modern SDK modules are defined in `.proto` files and serialized with protobuf. This covers the core API surface: transaction messages, query request/response types, stored state values, and genesis state. + +### Messages and transactions + +Each module defines its transaction messages in a `tx.proto` file. The `MsgSend` definition above is an example. When a user submits a transaction, the SDK serializes the transaction body (including its messages) to binary using protobuf before broadcasting it. + +For a hands-on example, see [tx.proto](/sdk/v0.54/tutorials/example/03-build-a-module#txproto) in the Build a Module tutorial. + +### Queries + +Modules define their query services in `query.proto`. Request and response types are protobuf messages. The SDK uses gRPC for queries, and gRPC uses protobuf as its serialization format by definition. + +For a hands-on example, see [query.proto](/sdk/v0.54/tutorials/example/03-build-a-module#queryproto) in the Build a Module tutorial. + +### State types + +Data stored in the KV store is protobuf-encoded. A module that stores a custom struct first marshals it to bytes using the codec, then writes those bytes to the store. When reading, it unmarshals the bytes back into the struct. Note that only *values* are protobuf-encoded; *keys* are manually constructed byte sequences, not protobuf. Key layout is covered in the [State, Storage, and Genesis](/sdk/v0.54/learn/concepts/store) section. + +### Genesis + +Genesis state is defined in `genesis.proto`. `InitGenesis` and `ExportGenesis` use protobuf to deserialize genesis state from `genesis.json` and serialize it back. + +A concrete example shows how a module reads and writes typed state as bytes: + +```go +// write: marshal the coin amount to bytes, then set in store +bz, err := k.cdc.Marshal(&amount) +store.Set(key, bz) + +// read: get bytes from store, unmarshal back to coin +var amount sdk.Coin +bz := store.Get(key) +k.cdc.Unmarshal(bz, &amount) +``` + +The codec (`k.cdc`) is the protobuf codec described in the next section. + +## The codec and interface registry + +The Cosmos SDK wraps protobuf in a **codec** that modules use for marshaling and unmarshaling. The primary implementation is [`ProtoCodec`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/codec/proto_codec.go), which calls protobuf's `Marshal` and `Unmarshal` under the hood. + +```go +type ProtoCodec struct { + interfaceRegistry types.InterfaceRegistry +} + +func (pc *ProtoCodec) Marshal(o ProtoMarshaler) ([]byte, error) +func (pc *ProtoCodec) Unmarshal(bz []byte, ptr ProtoMarshaler) error +``` + +Keepers hold a reference to the codec and use it to encode and decode state: + +```go +type Keeper struct { + cdc codec.BinaryCodec + store storetypes.StoreKey +} +``` + +The codec is initialized once at app startup and passed to each keeper during initialization. + +### Interface types and `Any` + +Protobuf is strongly typed. You cannot store a field as "some implementation of an interface" directly in a protobuf message. The Cosmos SDK solves this using protobuf's [`google.protobuf.Any`](https://protobuf.dev/programming-guides/proto3/#any), which wraps an arbitrary message type alongside a URL that identifies what type it contains. + +`Any` is used anywhere the SDK needs to serialize a value whose concrete type is not known at compile time. The most common example is public keys. An account might use a secp256k1 key, an ed25519 key, or a multisig key. The `BaseAccount` stores the public key as `Any`: + +```proto +message BaseAccount { + string address = 1; + google.protobuf.Any pub_key = 2; + uint64 account_number = 3; + uint64 sequence = 4; +} +``` + +The `Any` field holds the serialized public key bytes plus a type URL like `/cosmos.crypto.secp256k1.PubKey`. When the SDK reads the account, it uses the type URL to look up the concrete Go type, then unmarshals the bytes into that type. + +#### Messages inside transactions + +Transaction messages are the most common use of `Any` in the SDK. A transaction can carry multiple message types from different modules (`bank.MsgSend`, `staking.MsgDelegate`, `gov.MsgVote`) in a single `TxBody`. Because protobuf requires concrete types at the field level, each message is packed into an `Any` before being placed inside the transaction: + +```text +MsgSend + ↓ pack into Any +Any { + type_url: "/cosmos.bank.v1beta1.MsgSend" + value: +} + ↓ placed in TxBody.messages +repeated google.protobuf.Any messages +``` + +During decoding, the SDK reads the `type_url`, looks up the concrete type in the interface registry, and unmarshals the bytes into the correct message struct. This is why every `sdk.Msg` implementation must be registered with `RegisterInterfaces` before the application starts. + + +The Cosmos SDK uses type URLs with a leading `/` but without the `type.googleapis.com` prefix (e.g. `/cosmos.bank.v1beta1.MsgSend`, not `type.googleapis.com/cosmos.bank.v1beta1.MsgSend`). If you need to pack a value into an `Any` manually, use `anyutil.New` from `github.com/cosmos/cosmos-proto/anyutil` rather than `anypb.New` from `google.golang.org/protobuf/types/known/anypb` — the standard library helper inserts the `type.googleapis.com` prefix, which breaks SDK type resolution. + + +This lookup is handled by the **interface registry**. + +### Interface registry + +The [`InterfaceRegistry`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/codec/types/interface_registry.go) is a runtime map from type URLs to Go types. When the SDK encounters an `Any` value, it queries the registry with the type URL to find the concrete Go type, then uses protobuf to unmarshal the bytes. + +```text +Any { type_url, value_bytes } + ↓ + InterfaceRegistry.Resolve(type_url) + ↓ + concrete Go type + ↓ + proto.Unmarshal(value_bytes, concreteType) +``` + +Without the interface registry, the SDK cannot decode `Any` values. This is why types must be explicitly registered before they can be deserialized. + +## Registering interface implementations + +Because the interface registry is a runtime lookup table, every concrete type that implements an SDK interface must be registered before the application starts. This is done with `RegisterInterfaces`: + +```go +// in codec registration, typically in module.go or types/codec.go +func RegisterInterfaces(registry codectypes.InterfaceRegistry) { + registry.RegisterImplementations( + (*cryptotypes.PubKey)(nil), + &secp256k1.PubKey{}, + &ed25519.PubKey{}, + ) +} +``` + +This tells the registry: "a `PubKey` interface can be a `secp256k1.PubKey` or an `ed25519.PubKey`." If a type is used in an `Any` field anywhere in the application and is not registered, the codec will fail to unmarshal it and return an error. + +Each module calls `RegisterInterfaces` during app initialization, and `app.go` calls these registration functions through the module manager when building the app. Custom types that implement SDK interfaces must follow the same pattern. + +### `codec.go` + +By convention, modules collect all codec registration in a single file: `x/mymodule/types/codec.go`. This file typically contains two functions: + +```go +// RegisterInterfaces registers protobuf interface implementations with the registry. +// Called during app initialization so the SDK can decode Any values at runtime. +func RegisterInterfaces(registry codectypes.InterfaceRegistry) { + registry.RegisterImplementations((*sdk.Msg)(nil), + &MsgAdd{}, + &MsgUpdateParams{}, + ) +} + +// RegisterLegacyAminoCodec registers message types for Amino JSON encoding. +// Required only if you want messages signable via SIGN_MODE_LEGACY_AMINO_JSON +// (e.g., Ledger hardware wallets using older firmware). +func RegisterLegacyAminoCodec(cdc *codec.LegacyAmino) { + cdc.RegisterConcrete(&MsgAdd{}, "mymodule/Add", nil) +} +``` + +`RegisterInterfaces` is required for every module that defines message types. Without it, the SDK cannot decode those messages from transactions. `RegisterLegacyAminoCodec` is optional and only needed for Ledger hardware wallet support via `SIGN_MODE_LEGACY_AMINO_JSON`. + +For an example of interface registration in a working module, see [Interface Registration](/sdk/v0.54/tutorials/example/03-build-a-module#interface-registration) in the Build a Module tutorial. + +## Proto-to-code generation workflow + +Writing `.proto` files produces `.pb.go` files through a code generation step. The generated Go code contains struct definitions, marshal/unmarshal methods, and gRPC service stubs. You never edit these generated files directly. + +The workflow is: + +**1. Write the `.proto` file** + +Proto files for a module live in the `proto/` directory at the repository root: + +``` +proto/myapp/mymodule/v1/ +├── tx.proto # message types (MsgAdd, MsgAddResponse, ...) +├── query.proto # query service (QueryCount, ...) +├── state.proto # on-chain state types +└── genesis.proto # genesis state +``` + +A message definition: + +```proto +syntax = "proto3"; +package myapp.mymodule.v1; + +message MsgAdd { + string sender = 1; + uint64 add = 2; +} + +message MsgAddResponse { + uint64 updated_count = 1; +} + +service Msg { + rpc Add(MsgAdd) returns (MsgAddResponse); +} +``` + +**2. Run code generation** + +```bash +# example — the exact target varies by project +make proto-gen +``` + +This runs [`buf`](https://buf.build/cosmos/cosmos-sdk/docs/main) (or `protoc` with plugins) against the `.proto` files and produces Go code under the module's `types/` directory. The full generated API reference for the Cosmos SDK is published at [buf.build/cosmos/cosmos-sdk/docs/main](https://buf.build/cosmos/cosmos-sdk/docs/main). + +``` +x/mymodule/types/ +├── tx.pb.go # generated: MsgAdd, MsgAddResponse, Marshal/Unmarshal methods +├── query.pb.go # generated: query request/response types +├── query.pb.gw.go # generated: gRPC-gateway REST handlers +└── state.pb.go # generated: on-chain state types +``` + +**3. Use the generated types** + +The generated structs implement `proto.Message` and can be passed directly to the codec for marshaling, registered with the interface registry, and used in keeper methods and message handlers: + +```go +// handler receives the generated type +func (m msgServer) Add(ctx context.Context, req *types.MsgAdd) (*types.MsgAddResponse, error) { + count, err := m.AddCount(ctx, req.Sender, req.Add) + if err != nil { + return nil, err + } + return &types.MsgAddResponse{UpdatedCount: count}, nil +} +``` + +The generated gRPC service stub is registered with BaseApp's message router, connecting the handler to the transaction execution pipeline automatically. + +To learn how to build a module from scratch using this workflow, visit the [Module building tutorial](/sdk/v0.54/tutorials/example/00-overview). + +## Legacy Amino encoding + +Before protobuf, the Cosmos SDK used a custom serialization format called **Amino** for transaction encoding, JSON signing documents, and interface serialization. Protobuf has replaced it in all of those roles. The `LegacyAmino` codec still exists for backward compatibility, but is not used in the consensus-critical path. + +Some legacy components still reference it: + +- `LegacyAmino` is still present in the codec package for backward-compatibility +- `LegacyAminoPubKey` (multisig) is registered alongside protobuf public key types +- Some older chains, hardware wallets, and client tooling depend on Amino JSON signing + +New modules and chains should use protobuf exclusively. + +## Encoding in context + +Every layer of the Cosmos SDK depends on encoding: + +``` +Transaction (binary protobuf) + ↓ broadcast over p2p +CometBFT + ↓ passes raw bytes to application +BaseApp + ↓ decodes transaction, extracts messages +Module MsgServer + ↓ processes message, calls keeper +Keeper + ↓ marshals state value to bytes +KVStore (raw bytes) + ↓ committed to disk +AppHash (Merkle root over all KV bytes) +``` + +Determinism comes from the combination of canonical transaction encoding (ADR-027), deterministic application logic, and consistent protobuf serialization of stored state. Two validators executing the same transactions under these rules always produce the same bytes at every layer, and therefore always arrive at the same AppHash. + +The next section, [Execution Context, Gas, and Events](/sdk/v0.54/learn/concepts/context-gas-events), explains the runtime execution environment that modules operate within: `sdk.Context`, gas metering, and events. diff --git a/sdk/v0.54/learn/concepts/lifecycle.mdx b/sdk/v0.54/learn/concepts/lifecycle.mdx new file mode 100644 index 000000000..abd23bfcd --- /dev/null +++ b/sdk/v0.54/learn/concepts/lifecycle.mdx @@ -0,0 +1,214 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/learn/concepts/lifecycle' +title: Transaction Lifecycle +--- + +In the [Transactions, Messages, and Queries](/sdk/v0.54/learn/concepts/transactions) page, you learned that transactions are the actual mechanism that authorizes and executes logic on the chain. This page explains how transactions are validated, executed, and committed in the Cosmos SDK. + +Before building with the Cosmos SDK, it's important to connect the high-level architecture from [SDK Application Architecture](/sdk/v0.54/learn/intro/sdk-app-architecture) with how blocks and transactions actually execute in code. + +The following components are essential for understanding the lifecycle of a transaction in the Cosmos SDK: + +- [CometBFT](/cometbft) (consensus engine) — orders and proposes blocks +- [ABCI](/sdk/v0.54/learn/intro/sdk-app-architecture#abci-application-blockchain-interface) (Application-Blockchain Interface) — the protocol CometBFT uses to talk to the Cosmos SDK application +- SDK application ([`BaseApp`](/sdk/v0.54/learn/concepts/baseapp) + [modules](/sdk/v0.54/learn/concepts/modules)) — the deterministic state machine that executes transactions +- [Protobuf schemas](/sdk/v0.54/learn/concepts/encoding) — define transactions, messages, state, and query types + +This page maps the block and transaction lifecycle back to those components. + +## ABCI overview + +CometBFT and the SDK application are two separate processes with distinct responsibilities. +- [CometBFT](/cometbft/latest/docs/introduction/intro) handles consensus: ordering transactions, managing validators, and driving block production. +- The [SDK application](/sdk/v0.54/learn/concepts/sdk-structure) handles state: executing transactions and updating the chain's data. + +The [ABCI](/sdk/v0.54/learn/intro/sdk-app-architecture#abci-application-blockchain-interface) (Application Blockchain Interface) is the protocol that connects them: CometBFT calls ABCI methods on the application to drive each phase of the block lifecycle, and the application responds. + +[`BaseApp`](/sdk/v0.54/learn/concepts/baseapp) is the SDK's implementation of the ABCI interface. It receives these calls from CometBFT and orchestrates execution across modules. Modules plug into `BaseApp` and execute their logic during the appropriate phases. + +```python ++---------------------+ | +-------------------------+ +| CometBFT | | | SDK Application | +| (Consensus) | ABCI | (BaseApp + modules) | ++---------------------+ | +-------------------------+ + | +InitChain (once) | + Chain start -------------------|------> InitGenesis per module + | +CheckTx (per submitted tx) | + Mempool validation ------------|------> decode · verify · validate + |<------ accept → Mempool + | +PrepareProposal (proposer only) | + Build block proposal ----------|------> select txs (MaxTxBytes, MaxGas) + | +ProcessProposal (all validators) | + Evaluate proposal -------------|------> verify txs → ACCEPT / REJECT + | +FinalizeBlock (per block) | + Execute block -----------------|------> PreBlock hooks + | BeginBlock hooks + | For each tx: + | AnteHandler + | → message routing + | → MsgServer (module logic) + | EndBlock hooks + | Return AppHash +Commit | + Persist state -----------------|------> persist state to disk + |<------ return AppHash +``` + +## InitChain (genesis only) + +`InitChain` runs once when the chain starts for the first time. `BaseApp` loads `genesis.json`, which defines the chain's initial state, and calls each module's `InitGenesis` to populate its store. The initial validator set is established. Genesis runs before the first block begins. + +For how `genesis.json` becomes module state, see [Genesis and chain initialization](/sdk/v0.54/learn/concepts/store#genesis-and-chain-initialization). + +## CheckTx and the mempool + +Before a transaction can enter a block, it goes through `CheckTx`: + +```text +User + ↓ +Node + ↓ +ABCI: CheckTx + ↓ +Mempool +``` + +Transactions are sent as raw protobuf-encoded bytes. +For how those bytes are encoded deterministically, see [Encoding and Protobuf](/sdk/v0.54/learn/concepts/encoding). + +During `CheckTx`, the SDK application's `BaseApp` decodes the transaction, verifies signatures and sequences, validates fees and gas, and performs basic message validation. +For the account sequence model, see [Accounts](/sdk/v0.54/learn/concepts/accounts). For gas metering and fee-related execution details, see [Execution Context, Gas, and Events](/sdk/v0.54/learn/concepts/context-gas-events). + +If validation fails, the transaction is rejected. If it passes, it enters the mempool. The mempool is a node's in-memory pool of validated transactions waiting to be included in a block. + +Validated transactions wait in the mempool until CometBFT selects a block proposer for the next round. + +## PrepareProposal + +Each round, [CometBFT](/cometbft/latest/docs/introduction/intro#intro-to-abci) selects one validator to propose a block. `PrepareProposal` is called on that validator only. `BaseApp` selects transactions from the mempool respecting the block's `MaxTxBytes` and `MaxGas` limits and returns the final transaction list. For where this handler is configured, see [Block proposal and vote extensions](/sdk/v0.54/learn/concepts/baseapp#block-proposal-and-vote-extensions). + +## ProcessProposal + +Once the other validators receive the proposed block, CometBFT calls `ProcessProposal`. `BaseApp` verifies each transaction and returns `ACCEPT` or `REJECT`. No state is written. Once more than two-thirds of voting power accepts the block and consensus is reached, CometBFT calls `FinalizeBlock`. For the execution-model view of these handlers, see [Block proposal and vote extensions](/sdk/v0.54/learn/concepts/baseapp#block-proposal-and-vote-extensions). + +## FinalizeBlock + +CometBFT calls `FinalizeBlock` once per block. Inside `FinalizeBlock`, `BaseApp` runs these phases in order: + +```text +PreBlock → BeginBlock → transaction execution → EndBlock +``` + +### `PreBlock` + +`PreBlock` runs before `BeginBlock` and is generally used for logic that must affect consensus-critical state before the block begins, such as activating a chain upgrade or modifying consensus parameters. Because these changes need to take effect before any block logic runs, they cannot happen inside `BeginBlock`. Modules may implement this via the `HasPreBlocker` extension interface on their `AppModule` (typically in `x//module.go`), and the application's `ModuleManager` invokes all registered PreBlockers during `FinalizeBlock`. + +If a `PreBlocker` modifies consensus parameters, it signals this by returning `ConsensusParamsChanged=true` in its `ResponsePreBlock`. `BaseApp` then refreshes the consensus params in the current context before proceeding to `BeginBlock`: + +```go +app.finalizeBlockState.ctx = app.finalizeBlockState.ctx.WithConsensusParams(app.GetConsensusParams()) +``` + +### `BeginBlock` + +`BeginBlock` runs after `PreBlock` and handles per-block housekeeping that must happen before any transactions execute, regardless of the transactions in the block. Common uses include minting inflation rewards, distributing staking rewards, and resetting per-block state. Modules implement this via the `BeginBlock` function in `x//module.go`. Because `BeginBlock` and `EndBlock` run on every block, complex or expensive logic in these hooks can slow block execution; keep their work lightweight. + +### Transaction execution + +After `BeginBlock`, `BaseApp` iterates over each transaction in the block and runs it through a fixed pipeline. + +#### Step 1: `AnteHandler` + +Configured in a Cosmos SDK chain's [`app.go`](/sdk/v0.54/learn/concepts/app-go), the `AnteHandler` runs first for every transaction. For standard ordered transactions, it verifies signatures, checks sequence numbers, deducts fees, and meters gas. See [BaseApp](/sdk/v0.54/learn/concepts/baseapp#antehandler) for the full middleware model. + +If the `AnteHandler` fails, the transaction aborts and its messages do not execute. + +#### Step 2: Message routing and execution + +Each message in a transaction is routed via `BaseApp`'s `MsgServiceRouter` to the appropriate module's protobuf `Msg` service. Messages are module-specific and typically defined in a module's `tx.proto`. `BaseApp` routes these messages to the module's registered protobuf `Msg` service handler, which calls the module's `MsgServer` implementation. See [Message routing](/sdk/v0.54/learn/concepts/baseapp#message-routing) for the router's role in the execution pipeline. + +The `MsgServer` contains the execution logic for that message type. It validates the message content, applies business rules, and updates state. State is read and written through the module's keeper, which manages access to the module's KV store and encapsulates its storage keys. [Intro to Modules](/sdk/v0.54/learn/concepts/modules) explains how `MsgServer` and `Keeper` divide responsibilities. + +Messages execute sequentially in the order they appear in the transaction. + +#### Step 3: Atomicity + +Message execution is atomic: all messages succeed or none of the message execution writes are committed. + +```text +Tx + ├─ Msg 1 + ├─ Msg 2 + └─ Msg 3 +``` + +If any message fails, the message execution branch for that transaction is discarded and the transaction returns an error. The next transaction in the block is then executed. `BaseApp` uses cached stores internally to implement this. `AnteHandler` side effects may already have been applied before message execution begins. + +If the chain enables unordered transactions, the normal sequence check is bypassed and replay protection uses a timeout timestamp plus unordered nonce tracking. For the client-facing flow, see [Generating an Unordered Transaction](/sdk/v0.54/node/txs#generating-an-unordered-transaction). + +### `EndBlock` + +`EndBlock` runs after all transactions in the block have executed. It is used for logic that depends on the block's cumulative state, like tallying governance votes after all vote transactions have been processed, or recalculating validator power after all delegation changes in the block. Modules implement this via the `EndBlock` function in `x//module.go`. + +## Commit + +After `FinalizeBlock` returns, CometBFT calls `Commit`. This persists the state changes to the node's local disk. + +## Deterministic execution + +Across all validators, the block execution is deterministic. Blocks must contain the same ordered transactions, and transactions must use canonical protobuf binary encoding. State transitions must be deterministic, which ensures that every validator computes the same app hash during `FinalizeBlock`, which guarantees consensus safety. If validators holding more than 1/3 of voting power disagree on the app hash, consensus halts. + +## Complete lifecycle overview + +```go +CometBFT + ↓ ABCI InitChain +BaseApp → x//InitGenesis + +For each submitted transaction (async): + ↓ ABCI CheckTx + → decode, verify, validate + → insert into mempool + +For every block: + ↓ ABCI PrepareProposal (proposer only) + → select txs from mempool (MaxTxBytes, MaxGas) + → return tx list to CometBFT + ↓ ABCI ProcessProposal (all validators) + → verify txs, check gas limit + → ACCEPT or REJECT + ↓ ABCI FinalizeBlock + → PreBlock + → x//BeginBlock + For each tx (in the block): + → AnteHandler + → Message routing + → Message execution (atomic) + → x//EndBlock + ↓ ABCI Commit +BaseApp commits KVStores +``` + + +The hooks that run at each phase (the `AnteHandler`, `BeginBlocker`, `EndBlocker`, and `InitChainer`) are registered in your chain's [`app.go`](/sdk/v0.54/learn/concepts/app-go) before any block executes. `app.go` is the configuration layer that wires modules into `BaseApp`. + + +CometBFT drives block processing through ABCI. `BaseApp` implements ABCI and orchestrates execution. + +- Transactions are validated in `CheckTx` before entering the mempool +- `PrepareProposal` runs on the proposer to build the final tx set for the block +- `ProcessProposal` runs on all validators to accept or reject the proposed block +- Each block is executed inside a single `FinalizeBlock` call +- Within `FinalizeBlock`: `PreBlock` → `BeginBlock` → transactions → `EndBlock` +- Each transaction runs through `AnteHandler` → message routing → message execution +- Message execution within a transaction is atomic: all messages commit or none do +- `FinalizeBlock` computes and returns the app hash; `Commit` persists state to disk + +[Protobuf](/sdk/v0.54/learn/concepts/encoding) ensures canonical encoding so all validators interpret transactions identically. The next section, [Intro to Modules](/sdk/v0.54/learn/concepts/modules), turns from block execution to the module structure that actually implements chain logic. \ No newline at end of file diff --git a/sdk/v0.54/learn/concepts/modules.mdx b/sdk/v0.54/learn/concepts/modules.mdx new file mode 100644 index 000000000..c1d7a3007 --- /dev/null +++ b/sdk/v0.54/learn/concepts/modules.mdx @@ -0,0 +1,347 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/learn/concepts/modules' +title: Intro to Modules +--- + +In the previous section, you saw how blocks and transactions are processed. But where does the actual application logic of a blockchain live? + +In the Cosmos SDK, **modules** define that logic. + +Modules are the fundamental building blocks of a Cosmos SDK application. Each module encapsulates a specific piece of functionality, such as accounts, token transfers, validator management, governance, or any custom logic you define. + +To see a complete working module, follow the [Build a module tutorial series](/sdk/v0.54/tutorials/example/00-overview). + +## Why modules exist + +A blockchain application needs to manage many independent concerns (accounts, balances, validator management, etc). Instead of placing all logic in a single monolithic state machine, the Cosmos SDK divides the application into modules. The SDK provides a base layer that allows these modules to operate together as a cohesive blockchain. + +Each module owns a slice of state, defines its messages and queries, implements its business rules, and hooks into the block lifecycle and genesis as needed. This keeps the application organized, composable, and easier to reason about. It also separates safety concerns between modules, creating a more secure system. + +## What a module defines + +A module is a self-contained unit of state and logic. + +At a high level, a module defines: + +- [State](#state): a `KVStore` namespace that contains the module's data +- [Messages](#messages): the actions the module allows +- [Queries](#queries): read-only access to the module's state +- [`MsgServer`](#message-execution-msgserver): validates, applies business logic, and delegates to the keeper +- [Keeper](#keeper): the state access layer: the only sanctioned path to the store +- [Params](#params): governance-controlled configuration stored on-chain + +### State + +State is the data persisted on the chain: everything that transactions can read from or write to. When a transaction is executed, modules apply state transitions or deterministic updates to this stored data. + +Each module owns its own part of the blockchain state. For example: + +- The `x/auth` module stores account metadata. +- The `x/bank` module stores account balances. + +Modules do not share storage directly. Each module has its own key-value store namespace located in a `multistore`. For example, the bank module's store might contain entries like: + +``` +// x/bank store (conceptual) +balances | cosmos1abc...xyz | uatom → 1000000 +balances | cosmos1def...uvw | uatom → 500000 +``` + +The key encodes the namespace, address, and denomination. The value is the encoded amount. No other module can read or write these entries directly, only the [module's keeper](#keeper) can. To learn more about how state is stored and accessed, see [Store](/sdk/v0.54/learn/concepts/store). + +### Messages + +As you learned in the [Transactions, Messages, and Queries](/sdk/v0.54/learn/concepts/transactions) section, each module defines the actions it allows via messages (`sdk.Msg`). + +Messages are defined in the module's `tx.proto` file under a `service Msg` block, and implemented by that module's `MsgServer`. Here is a simplified example from the bank module: + +```protobuf +// Message type definition +message MsgSend { + string from_address = 1; + string to_address = 2; + repeated cosmos.base.v1beta1.Coin amount = 3; +} + +// Msg service — groups all messages for this module +service Msg { + rpc Send(MsgSend) returns (MsgSendResponse); +} +``` + +The `service Msg` block is referred to as the `Msg service` in the Cosmos SDK. The protobuf compiler generates a `MsgServer` interface from it, which the module implements in Go. [`BaseApp`](/sdk/v0.54/learn/concepts/baseapp) routes each incoming message by its type URL (for example, `/cosmos.bank.v1beta1.MsgSend`) to the correct implementation. + +`MsgSend` above is an example of a message type definition. It represents a request to transfer tokens from one account to another. When included in a transaction and executed: + +1. The sender's balance is checked. +2. The amount is deducted from `from_address`. +3. The amount is credited to `to_address`. + +### Queries + +Modules expose read-only access to their state through query services, defined in `query.proto`. Queries do not modify state and do not go through block execution. For example: + +``` +// query.proto +rpc Balance(QueryBalanceRequest) returns (QueryBalanceResponse); +``` + +In this case, the caller provides an address and denomination; the query reads the balance from the x/bank keeper and returns it without modifying state. + +### Business logic + +While messages define intent, the `MsgServer` and `Keeper` work together to execute that intent and apply state transitions to the module. + +Business logic is conceptually split across two layers: + +- The [`MsgServer`](#message-execution-msgserver) handles the transaction-facing logic: it validates inputs, applies message-level business rules, and where required checks authorization. Once satisfied, it delegates state transitions to the keeper. +- The [`Keeper`](#keeper) owns the module's state: it defines the storage schema and provides the only authorized path for reading and writing it. Message handlers, block hooks, and governance proposals all go through keeper methods to make state changes. All state changes must go through the keeper, nothing accesses the store directly. + +### Message execution (`MsgServer`) + +Each module implements a `MsgServer`, which is invoked by `BaseApp`'s message router when a message is routed to that module. + +The `MsgServer` is responsible for: + +- Checking authorization when required +- Delegating to keeper methods that validate inputs, enforce business rules, and perform state transitions +- Returning a response + +The following example is from the minimal counter module tutorial example, which walks you through building a module that lets users increment a shared counter. See the [Build a Module from Scratch](/sdk/v0.54/tutorials/example/03-build-a-module) tutorial. + +The following is the counter module's `Add` handler which is the `MsgServer` method that processes a request to increment the counter: + +```go +func (m msgServer) Add(ctx context.Context, request *types.MsgAddRequest) (*types.MsgAddResponse, error) { + newCount, err := m.AddCount(ctx, request.GetAdd()) + if err != nil { + return nil, err + } + + return &types.MsgAddResponse{UpdatedCount: newCount}, nil +} +``` + +In the minimal tutorial, `Add` is permissionless and delegates directly to the keeper. The handler itself contains almost no business logic. That keeps `msg_server.go` focused on request handling, while the keeper owns the actual state transition. + +The full counter module example adds richer patterns on top of that minimal shape. For privileged messages like `MsgUpdateParams`, the `MsgServer` checks the caller against the stored authority before proceeding. See the [Full Counter Module Walkthrough](/sdk/v0.54/tutorials/example/04-counter-walkthrough#params-and-authority). + +```go +func (m msgServer) UpdateParams(ctx context.Context, msg *types.MsgUpdateParams) (*types.MsgUpdateParamsResponse, error) { + if m.authority != msg.Authority { + return nil, sdkerrors.Wrapf( + govtypes.ErrInvalidSigner, + "invalid authority; expected %s, got %s", + m.authority, + msg.Authority, + ) + } + + if err := m.SetParams(ctx, msg.Params); err != nil { + return nil, err + } + + return &types.MsgUpdateParamsResponse{}, nil +} +``` + +## Keeper + +A module's **`Keeper`** is its state access layer. It owns the module's `KVStore` and provides typed methods for reading and writing state. The store fields are unexported, so nothing outside the `keeper` package can access them directly. + +The `MsgServer` and `QueryServer` both embed the keeper. It is best practice for business logic to be implemented in keeper methods rather than the `MsgServer`, so the same rules apply whether the caller is a message handler, a block hook, or a governance proposal. + +The following keeper example is from the minimal counter module example. It holds a single state item and shows the smallest useful keeper shape. See [Step 5: Keeper](/sdk/v0.54/tutorials/example/03-build-a-module#step-5-keeper) in the tutorial. + +```go +type Keeper struct { + Schema collections.Schema + counter collections.Item[uint64] +} + +func (k *Keeper) AddCount(ctx context.Context, amount uint64) (uint64, error) { + count, err := k.GetCount(ctx) + if err != nil { + return 0, err + } + newCount := count + amount + return newCount, k.counter.Set(ctx, newCount) +} +``` + +`counter` uses `collections.Item[uint64]`, a typed single-value entry backed by the module's KV store using the [Collections API](/sdk/v0.54/learn/concepts/store#collections-api-typed-state-access). `AddCount` reads the current value, increments it, and writes it back. All state access goes through the keeper: nothing outside the `keeper` package can reach `counter` directly. For details on how the keeper opens its store from the context it receives, see [Execution Context](/sdk/v0.54/learn/concepts/context-gas-events). + +### Inter-module access + +Modules are isolated by default. Each module owns its state, and direct access to that state is restricted through the module's keeper. Other modules cannot arbitrarily mutate another module's storage. + +Instead, modules interact through explicitly defined keeper interfaces. + +For example: + +- The staking module calls methods on the bank keeper to transfer tokens. +- The governance module calls parameter update methods on other modules. + +Each module defines an `expected_keepers.go` file that declares the interfaces it requires from other modules. This makes cross-module dependencies explicit: a module can only call methods the other module has chosen to expose. + +This design keeps dependencies auditable and prevents accidental or unsafe cross-module state mutation. + +To see expected keepers and cross-module fee collection in practice, see [Expected keepers and fee collection](/sdk/v0.54/tutorials/example/04-counter-walkthrough#expected-keepers-and-fee-collection) in the Full Counter Module Walkthrough. + +### Params + +Most modules expose a `Params` struct: a set of configuration values stored on-chain that control the module's behavior. Unlike regular state, params are intentionally stable: they only change through governance, not through user transactions. Examples include the minimum governance deposit, the maximum number of validators, or mint module inflation bounds. Params are stored under a single key (typically a `collections.Item[Params]`) and updated by submitting a governance proposal. + +To update params, a governance proposal submits a [`MsgUpdateParams`](/sdk/v0.54/modules/consensus/README#msgupdateparams) message. The `MsgServer` checks that the caller is the designated authority address (usually the governance module account) before writing the new values to the store. See [Message execution (`MsgServer`)](#message-execution-msgserver) for a code example of this pattern. + +The authority address is set at keeper construction time in [`app.go`](/sdk/v0.54/learn/concepts/app-go#initializing-keepers). + +For chain-level consensus parameters, the [`x/consensus`](/sdk/v0.54/modules/consensus) module manages them centrally; it also supports [`AuthorityParams`](/sdk/v0.54/modules/consensus/README#authorityparams), which lets governance update the authority address on-chain without a software upgrade. + +To see params implemented in a working module, see [Params and authority](/sdk/v0.54/tutorials/example/04-counter-walkthrough#params-and-authority) in the Full Counter Module Walkthrough. + +### Block hooks + +Modules may execute logic at specific points in the block lifecycle by implementing optional hook interfaces in the `AppModule` struct in `module.go`: + +- `HasBeginBlocker` — runs logic at the start of each block +- `HasEndBlocker` — runs logic at the end of each block +- `HasPreBlocker` — runs logic before `BeginBlock`, used for consensus parameter changes + +Hooks are optional, and modules should only implement the hooks they need. These hooks are invoked during block execution by the `ModuleManager` in [`BaseApp`](/sdk/v0.54/learn/concepts/baseapp#module-manager), which calls each registered module's hooks in a configured order. For the application wiring side, see [Module Manager in `app.go`](/sdk/v0.54/learn/concepts/app-go#module-manager). + +A module implements a hook by defining the corresponding method on its `AppModule` struct in `module.go`: + +```go +func (a AppModule) PreBlock(ctx context.Context) (appmodule.ResponsePreBlock, error) { + // runs before BeginBlock; used for logic that must take effect before block execution + return &sdk.ResponsePreBlock{}, nil +} + +func (a AppModule) BeginBlock(ctx context.Context) error { + // runs at the start of each block + return nil +} + +func (a AppModule) EndBlock(ctx context.Context) error { + // runs at the end of each block + return nil +} +``` + +Defining these methods opts the module into the corresponding block hooks. The module also needs to be registered with the `ModuleManager` in [`app.go`](/sdk/v0.54/learn/concepts/app-go#module-manager) for the hooks to be called. + +### Genesis initialization + +Modules define how their state is initialized when the chain starts. + +Each module implements: + +- `DefaultGenesis`: returns the module's default genesis state +- `ValidateGenesis`: validates the genesis state before the chain starts +- `InitGenesis`: writes the genesis state into the module's store at chain start +- `ExportGenesis`: reads the module's current state and serializes it as genesis data + +During `InitChain`, `BaseApp` calls each module's `InitGenesis` to populate its state from `genesis.json`. + +For where this happens in the block lifecycle, see [InitChain (genesis only)](/sdk/v0.54/learn/concepts/lifecycle#initchain-genesis-only). +For a walkthrough of genesis implementation, see [Step 2: Proto files](/sdk/v0.54/tutorials/example/03-build-a-module#step-2-proto-files) and [Step 8: module.go](/sdk/v0.54/tutorials/example/03-build-a-module#step-8-modulego) in the Build a Module tutorial. + +## Built-in and custom modules + +The Cosmos SDK ships with a set of core modules that most chains use. Click any module to learn more: + +| Module | What it does | +| --- | --- | +| [x/auth](/sdk/v0.54/modules/auth) | Accounts, authentication, and transaction signing | +| [x/bank](/sdk/v0.54/modules/bank) | Token balances and transfers | +| [x/staking](/sdk/v0.54/modules/staking) | Validator set and delegations | +| [x/gov](/sdk/v0.54/modules/gov) | On-chain governance and proposals | +| [x/distribution](/sdk/v0.54/modules/distribution) | Staking reward distribution | +| [x/slashing](/sdk/v0.54/modules/slashing) | Validator penalty enforcement | +| [x/mint](/sdk/v0.54/modules/mint) | Token issuance | +| [x/evidence](/sdk/v0.54/modules/evidence) | Submission and handling of validator misbehavior | +| [x/upgrade](/sdk/v0.54/modules/upgrade) | Coordinated chain upgrades | +| [x/authz](/sdk/v0.54/modules/authz) | Delegated message authorization | +| [x/feegrant](/sdk/v0.54/modules/feegrant) | Fee allowances between accounts | +| [x/consensus](/sdk/v0.54/modules/consensus) | On-chain management of CometBFT consensus params | + +The above are just a few of the modules available. Applications can include any subset of these modules and can define entirely new custom modules. For the full list, see [List of Modules](/sdk/v0.54/modules/modules). + +[Cosmos Enterprise](/sdk/v0.54/enterprise/overview) provides additional hardened modules for production networks with more demanding requirements: + +| Module | What it does | +| --- | --- | +| [Permissioned Consensus](/sdk/v0.54/enterprise/poa/overview) | Permissioned validator set managed by an on-chain authority, replacing token-based staking | +| [Multi-sig](/sdk/v0.54/enterprise/group/overview) | On-chain multisig accounts and collective decision-making with configurable voting policies | + +A Cosmos SDK blockchain is ultimately a collection of modules assembled into a single application. This customization of modules and application logic down to the lowest levels of a chain is what makes the Cosmos SDK so flexible and powerful. + +## Anatomy of a module (high-level) + +Modules live under the `x/` directory of an SDK application: + +``` +x/ +├── auth/ # Accounts and authentication +├── bank/ # Token balances and transfers +├── poa/ # Proof-of-authority validator management +├── gov/ # Governance system +└── mymodule/ # Your custom module +``` + +Each subdirectory under `x/` is a self-contained module. +An application composes multiple modules together to form a complete blockchain. + +Inside a module, you will typically see a structure like this: + +``` +x/mymodule/ +├── keeper/ +│ ├── keeper.go # Keeper struct and state access methods +│ ├── msg_server.go # MsgServer implementation +│ └── query_server.go # QueryServer implementation +├── types/ +│ ├── expected_keepers.go # Interfaces for other modules' keepers +│ ├── keys.go # Store key definitions +│ └── *.pb.go # Generated from proto definitions +└── module.go # AppModule implementation and hook registration +``` + +Proto files live separately at the repository root, not inside `x/`: + +``` +proto/myapp/mymodule/v1/ +├── tx.proto # Message definitions +├── query.proto # Query service definitions +├── state.proto # On-chain state types +└── genesis.proto # Genesis state definition +``` + +- `keeper/`: Contains the `Keeper` struct (state access) and implementations of the `MsgServer` and `QueryServer` interfaces. +- `types/`: Defines the module's public types: generated protobuf structs, store keys, and the `expected_keepers.go` interfaces that declare what this module needs from other modules. +- `module.go`: Connects the module to the application and registers genesis handlers, block hooks, and message and query services. +- `.proto` files: Define messages, queries, state schemas, and genesis state. Go code is generated from these files and used throughout the module. + +Check out the [Module Tutorial](/sdk/v0.54/tutorials/example/00-overview) to learn how to build a module from scratch. + +## Modules in context + +Putting everything together you've learned so far: + +- **Accounts** authorize transactions. +- **Transactions** carry messages and execution constraints. +- **Blocks** order transactions and define commit boundaries. +- **Modules** define the business rules of execution. +- **MsgServer** validates messages and orchestrates state transitions. +- **Keeper** performs controlled reads and writes to module state. +- **State** persists the deterministic result of execution. + +``` +Account → Transaction → Message → Module → MsgServer → Keeper → State +``` + +In the next section, [State, Storage, and Genesis](/sdk/v0.54/learn/concepts/store), you will look more closely at how module state is stored, how genesis initializes it, and how the application commits deterministic state transitions. + +Before writing a module, review [Module Design Considerations](/sdk/v0.54/guides/module-design/module-design-considerations) for guidance on state structure, message surface, inter-module dependencies, and upgrade planning. When you are ready to build, follow the [Build a Module tutorial](/sdk/v0.54/tutorials/example/00-overview) for a step-by-step walkthrough. diff --git a/sdk/v0.54/learn/concepts/sdk-structure.mdx b/sdk/v0.54/learn/concepts/sdk-structure.mdx new file mode 100644 index 000000000..7a61b61f0 --- /dev/null +++ b/sdk/v0.54/learn/concepts/sdk-structure.mdx @@ -0,0 +1,160 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/learn/concepts/sdk-structure' +title: Cosmos Blockchain Structure +--- + +Before writing a module or chain, it helps to understand how the Cosmos SDK organizes code and how the pieces connect. This page maps the directory structure of a Cosmos SDK application, explains what lives inside a module, and shows how modules are assembled into a running application. + +## What is an SDK application + +A Cosmos SDK application is a Go binary that implements a deterministic state machine. It runs alongside CometBFT inside the node daemon process. CometBFT drives consensus and the SDK application executes transactions and maintains state. + +Every SDK application is composed of three main elements: + +- [`BaseApp`](/sdk/v0.54/learn/concepts/baseapp): the execution engine that implements the ABCI and orchestrates transaction processing +- [Modules](/sdk/v0.54/learn/concepts/modules): self-contained units of business logic, state, messages, and queries +- [`app.go`](/sdk/v0.54/learn/concepts/app-go): the wiring layer that instantiates `BaseApp`, registers modules, and configures the application at startup + +`BaseApp` and `app.go` are covered in depth in the next two sections. This page focuses on how they are organized in the codebase. To see all of this in action, follow the [Build a Chain tutorial series](/sdk/v0.54/tutorials/example/00-overview). + +## Repository structure + +SDK applications repositories generally use the following layout: + +``` +myapp/ +├── app/ +│ └── app.go # Application wiring: configures BaseApp, registers modules +├── cmd/ +│ └── main.go # Node binary entrypoint +├── x/ +│ ├── mymodule/ # Custom module +│ └── ... +└── proto/ + └── myapp/ + └── mymodule/ + └── v1/ + ├── tx.proto + ├── query.proto + ├── state.proto + └── genesis.proto +``` + +Each directory has a distinct responsibility: + +- `x/` contains the modules. Each subdirectory is a separate, self-contained module. Built-in Cosmos SDK modules (`x/auth`, `x/bank`, `x/staking`, etc.) follow the same layout and are imported as Go packages. Your custom modules live alongside them. + +- `app/` contains `app.go`, which assembles the application: creating `BaseApp`, mounting stores, initializing keepers, and registering all modules with the `ModuleManager`. + +- `cmd/` contains `main.go`, the entrypoint for the node binary. It parses command-line flags, reads configuration files, and starts the daemon process that runs both the CometBFT node and the SDK application. + +- `proto/` contains the Protobuf definitions for all custom types: messages, queries, state schemas, and genesis. Go code is generated from these files and consumed throughout the module. Proto files live at the repository root, not inside `x/`, so they can be shared across languages and tooling. + +## What lives inside a module + +Each module under `x/` follows a consistent internal layout: + +``` +x/mymodule/ +├── keeper/ # Keeper (state access), MsgServer, QueryServer +├── types/ # Generated proto types, store keys, expected_keepers.go +└── module.go # AppModule: wires the module into the application +``` + +Proto definitions live separately in `proto/`, not inside `x/`. See [Intro to Modules](/sdk/v0.54/learn/concepts/modules#anatomy-of-a-module-high-level) for a complete walkthrough of each file and the role it plays. + +## How modules are assembled into an application + +Modules are assembled through the `ModuleManager` in [`app.go`](/sdk/v0.54/learn/concepts/app-go). The `ModuleManager` holds the full set of registered modules and coordinates their lifecycle hooks (`InitGenesis`, `BeginBlock`, `EndBlock`, and service registration) across the application. Module ordering is configured explicitly in `app.go` and matters: for example, in `simapp` the distribution module runs before slashing in `BeginBlock` so validator rewards are handled before slashing updates are applied. See [Module Manager](/sdk/v0.54/learn/concepts/baseapp#module-manager) for details on how `BaseApp` integrates with it. + +`BaseApp` implements the ABCI interface that CometBFT calls to drive block execution. When CometBFT calls `FinalizeBlock`, `BaseApp` runs the block through all its phases (`PreBlock`, `BeginBlock`, transactions, `EndBlock`) and returns the resulting app hash. `BaseApp` is covered in detail in [BaseApp Overview](/sdk/v0.54/learn/concepts/baseapp). + +## The role of `app.go` + +[`app.go`](/sdk/v0.54/learn/concepts/app-go) is the single file that defines a specific chain. It is where the application is assembled from its parts. Here is a breakdown of the steps it takes: + +1. Create a `BaseApp` instance with the application name, logger, database, and codec. +2. Create a `StoreKey` for each module and mount it to the multistore. +3. Instantiate each `Keeper`, passing in the codec, store key, and references to other keepers the module depends on. +4. Create the `ModuleManager` with all module instances. +5. Configure execution ordering: which modules run first during genesis, `BeginBlock`, and `EndBlock`. +6. Register all gRPC services (message and query handlers) through the `ModuleManager`. +7. Set the `AnteHandler` and other middleware. + +Because `app.go` is plain Go code, it is fully customizable. A chain includes exactly the modules it needs, wires keepers together as required, and controls the execution order of all lifecycle hooks. + +## Other files in a chain + +A complete SDK chain repository contains more than just `x/`, `app/`, `cmd/`, and `proto/`. Below are some other files you will typically find in a Cosmos SDK chain repository: + +### Additional files in `app/` + +Real-world applications typically split the `app/` directory across multiple files to keep `app.go` focused on wiring: + +``` +app/ +├── app.go # Main wiring: BaseApp, keepers, module registration +├── export.go # Exports current state as a genesis file (hard forks, snapshots) +├── upgrades.go # Upgrade handlers for consensus-breaking software changes +└── genesis.go # Helpers for genesis state initialization (optional) +``` + +- `export.go`: Implements `ExportAppStateAndValidators`, which serializes all module state into a `genesis.json`. This is used when migrating to a new chain version (hard fork) or creating a testnet from a live chain snapshot. +- `upgrades.go`: Registers named upgrade handlers consumed by the `x/upgrade` module. Each handler runs exactly once, at the block where the governance-approved upgrade height is reached, and performs any necessary state migrations. + +### At the repository root + +``` +myapp/ +├── go.mod # Go module definition: SDK version and all dependencies +├── go.sum # Cryptographic checksums for all dependencies +├── Makefile # Build, test, and codegen tasks +└── scripts/ # Automation scripts (proto generation, linting) +``` + +- `go.mod` / `go.sum`: Standard Go module files. `go.mod` declares the Cosmos SDK version and all other imported packages. `go.sum` provides verifiable checksums for the full dependency tree. +- `Makefile`: The standard entry point for development tasks: `make build` compiles the binary, `make test` runs unit tests, `make proto-gen` regenerates Go code from `.proto` files. Most SDK chains include targets for linting, simulation tests, and Docker builds. +- `scripts/`: Shell scripts and configuration for tooling that the Makefile invokes. + +### The node binary (`cmd/`) + +``` +cmd/ +└── myappdaemon/ + ├── main.go # Binary entrypoint + └── root.go # Root Cobra command: subcommands (start, tx, query, keys, ...) +``` + +The `cmd/` directory produces the node daemon binary (e.g., `simd`, `gaiad`, `wasmd`). It uses [Cobra](https://github.com/spf13/cobra) to expose subcommands for starting the node, submitting transactions, querying state, managing keys, and running genesis initialization. The `start` command spins up CometBFT and the SDK application together in a single process. + +### Node home directory + +A typical repository contains the source code for a chain. When you actually run a node, the binary generates a separate home directory on disk that holds runtime configuration and chain data. Running `myappdaemon init` creates this directory: + +``` +~/.myapp/ # Node home directory (configurable with --home) +├── config/ +│ ├── app.toml # SDK server configuration +│ ├── config.toml # CometBFT configuration +│ ├── client.toml # CLI client defaults +│ └── genesis.json # Initial chain state +└── data/ # Database files (block store, state store, snapshots) +``` + +The location defaults to `~/.myapp` but can be overridden with the `--home` flag or the `MYAPP_HOME` environment variable. + +Each configuration file controls a distinct layer of the node: + +- `app.toml`: SDK-level server settings. Controls whether the gRPC server and REST API are enabled, their bind addresses, state sync configuration, pruning strategy, and mempool parameters. +- `config.toml`: CometBFT-level settings. Controls P2P networking (seeds, peers, listen address), consensus timeouts, the CometBFT RPC server address, and block size limits. +- `client.toml`: Default values for CLI client commands. Stores the chain ID, keyring backend, and the node RPC address so you don't have to pass `--chain-id` and `--node` on every command. +- `genesis.json`: The initial state of the chain at block 0. It is distributed out-of-band when joining a network, or generated locally for a new chain. Once the chain starts, this file is no longer read. + +## Summary + +An SDK application is a deterministic state machine composed of modules assembled in `app.go`. The codebase follows a conventional layout: modules in `x/`, application wiring in `app/`, the binary entrypoint in `cmd/`, and Protobuf definitions in `proto/`. + +The `ModuleManager` assembles modules and coordinates their lifecycle hooks across the application. `BaseApp` provides the ABCI implementation that connects the state machine to CometBFT's consensus engine. + +The next section, [BaseApp Overview](/sdk/v0.54/learn/concepts/baseapp), explains what `BaseApp` is and how it coordinates transaction execution in detail. diff --git a/sdk/v0.54/learn/concepts/store.mdx b/sdk/v0.54/learn/concepts/store.mdx new file mode 100644 index 000000000..0866b8a7d --- /dev/null +++ b/sdk/v0.54/learn/concepts/store.mdx @@ -0,0 +1,256 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/learn/concepts/store' +title: State, Storage, and Genesis +--- + +In the previous section, you learned that modules define business logic and that keepers are responsible for reading and writing module state. This page explains how that state is actually stored, committed, and made verifiable across the network. + +## What is state? + +State is the persistent data of the blockchain: account balances, delegations, governance proposals, [module parameters](/sdk/v0.54/learn/concepts/modules#params), and any other data that survives between blocks. When a transaction executes, modules update state. When a block is committed, that updated state becomes the starting point for the next block: + +```text +State0 + ↓ apply Block 1 +State1 + ↓ apply Block 2 +State2 +``` + +## The KVStore model + +At its lowest level, the Cosmos SDK stores state as **key-value pairs**. Both keys and values are byte arrays. Modules encode structured data into those bytes using Protocol Buffers, and decode them back when reading. + +See [Encoding and Protobuf](/sdk/v0.54/learn/concepts/encoding) for details on how modules serialize data into bytes. +The following example shows a conceptual example of how the bank module stores balances: + +```go +key: 0x2 | len(address) | address_bytes | denom_bytes +value: ProtocolBuffer(amount) + +# example +key: 0x2 | 20 | cosmos1abc...xyz | uatom +value: ProtocolBuffer(1000000) +``` + +The key encodes the store prefix, address length, address, and denomination. The value is a Protocol Buffer-encoded amount. See [`x/bank/types/keys.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/x/bank/types/keys.go) for the actual implementation. + +Each module owns its own namespace in the key-value store. Keys are defined by the module and typically begin with a byte prefix that distinguishes them from other module keys. + +## Multistore + +A single module store is only part of the picture. At the application level, all module stores are committed together. + +Every module has its own KVStore, and all module stores are mounted inside a [multistore](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/store/rootmulti/store.go) that is committed as a single state root. + +A module can only read and write to its own store through its keeper. Access is gated by a `StoreKey`, which is a typed capability object registered at app startup. Modules that don't hold the key cannot open the store. + +This isolation follows an object-capabilities model: + +1. Modules cannot directly mutate another module's state +2. Cross-module interaction must go through exposed keeper methods + +When a block finishes executing, the multistore computes a new root hash (the **app hash**) that represents the entire application state. That hash is returned to CometBFT, included in the block header, and is what makes the chain's state verifiable. [Transaction Lifecycle](/sdk/v0.54/learn/concepts/lifecycle) explains where that app hash is produced and committed. + + +The full storage stack from top to bottom is: + +``` +Module keeper + ↓ +KVStore (namespaced, wrapped with gas/trace) + ↓ +CommitMultiStore (multistore, computes app hash) + ↓ +IAVL tree (versioned Merkle tree) + ↓ +Database backend (goleveldb by default) +``` + +## How state is stored (IAVL and commit stores) + +Each module's KVStore is backed by a [`CommitKVStore`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/store/iavl/store.go#L36). [See the store spec for more details.](/sdk/v0.54/guides/state/store) + +In the current SDK store implementation described here, the Cosmos SDK uses [IAVL](https://github.com/cosmos/iavl), a versioned AVL Merkle tree. + +IAVL gives every read and write of the tree `O(log n)` complexity, meaning the time to read or write a key scales with the height of the tree, not the total number of keys. It also versions state on each block commit, and produces deterministic root hashes that can be used to generate Merkle proofs for light clients. + +Each block commit produces a new tree version with a new root hash: + +```text +Block 1 Block 2 Block 3 + + [root h1] [root h2] [root h3] + / \ / \ / \ + [branch1] [branch2] [branch1] [branch2'] [branch1] [branch2''] + / \ / \ / \ / \ / \ / \ + [a] [b] [c] [d] [a] [b] [c] [d'] [a] [b] [c'] [d'] + ↑ ↑ + (updated) (updated) + +// branch1 and branch2 are internal nodes (they store hashes, not data). +// Leaf nodes (a, b, c, d) are actual key-value entries. +// When a leaf changes, only nodes on the path to the root are rewritten (marked '). +// Unmodified subtrees (branch1, a, b) are shared across all three versions. +``` + +Only modified nodes are rewritten, and unchanged nodes are shared across versions. The root hash changes any time any leaf changes. All validators must compute the same root hash. If they disagree, consensus halts. + +Because of this, state transitions must be deterministic, encoding must be deterministic, and transaction ordering must be consistent. + +### App hash + +The **app hash** is the cryptographic root hash of the application's committed state. It summarizes all module stores together through the `CommitMultiStore`. Because every validator executes the same state transitions deterministically, they should all compute the same app hash for a given block. + +### Database backend + +The IAVL tree does not store data in memory. It writes versioned nodes to a database backend, which is a key-value store on disk. + +The Cosmos SDK uses [CometBFT's `db` package](https://github.com/cometbft/cometbft-db) to abstract over the database implementation. The default backend is [goleveldb](https://github.com/syndtr/goleveldb). Other supported backends include [PebbleDB](https://github.com/cockroachdb/pebble), [RocksDB](https://github.com/facebook/rocksdb), and memDB (in-memory, for testing). + +The database backend is selected at node startup and configured in [`app.toml`](/sdk/v0.54/tutorials/example/05-run-and-test#apptoml). Application code never interacts with it directly; the store layer owns that boundary. + +## Store types in the SDK + +Beyond the base KVStore, the SDK provides several specialized store wrappers. + +- [CommitKVStore](#commitkvstore-persistent-store) +- [CacheMultiStore](#cachemultistore-transaction-isolation) +- [Ephemeral store types](#ephemeral-store-types) +- [Gas and trace store wrappers](#gas-and-trace-store-wrappers) +- [Prefix store](#prefix-store) + +### CommitKVStore (persistent store) + +The [`CommitKVStore`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/store/iavl/store.go#L36) is the main persistent store backed by IAVL. It persists across blocks, produces versioned commits, and contributes to the app hash. + +### CacheMultiStore (transaction isolation) + +Before executing each transaction, the Cosmos SDK's `BaseApp` creates a [`CacheMultiStore`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/store/cachemulti/store.go) — a cached, copy-on-write view of the multistore. + +All writes during that transaction occur in this cached layer: + +``` +Multistore + ↓ CacheWrap (per transaction) + ↓ Execute tx + → Success → commit changes + → Failure → discard +``` + +- If the transaction succeeds, changes are written to the underlying store. +- If the transaction fails, the cache is discarded and no state changes are committed. + +This is how transaction atomicity is implemented in the store layer. + +### Ephemeral store types + +[Transient stores](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/store/transient/store.go) are cleared at the end of each block. They are used for temporary per-block data such as counters or intermediate calculations, and do not affect the app hash. + +[Memory stores](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/store/mem/store.go) survive block commits but reset when the node restarts — their `Commit()` is a no-op and data is never written to disk. They are used for in-process caching of data that is expensive to recompute each block but does not need to survive a restart. Modules access them via `MemoryStoreKey`, mounted with `MountMemoryStores` in `app.go`. + +| Store type | Survives block commit | Survives restart | +|---|---|---| +| Transient | No (cleared each block) | No | +| Memory | Yes | No | +| IAVL (CommitKVStore) | Yes | Yes | + +### Gas and trace store wrappers + +All store accesses are wrapped with additional behavior by the [`GasKVStore`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/store/gaskv/store.go) and [`TraceKVStore`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/store/tracekv/store.go) wrappers. + +- `GasKVStore` charges gas for each read and write +- `TraceKVStore` logs each store operation for debugging + +Because state access is the dominant cost of transaction execution, the SDK charges gas at the store layer so that expensive reads and writes are reflected in transaction fees. Every read and write of a KVStore costs gas, and expensive operations naturally cost more. [Execution Context, Gas, and Events](/sdk/v0.54/learn/concepts/context-gas-events) explains how gas metering works at runtime. + +### Prefix store + +A [**prefix store**](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/store/prefix/store.go) wraps a KVStore and automatically prepends a fixed byte prefix to every key. This lets keepers scope their reads and writes to a sub-namespace without manually constructing prefixed keys on every call. + +```go +prefixStore := prefix.NewStore(kvStore, types.KeyPrefix("balances")) +prefixStore.Set(key, value) // stored as "balances" + key +``` + +This is how modules avoid key collisions within their own store. + +## Collections API (typed state access) + +In the Cosmos SDK, modules commonly use the collections API to define typed state access. + +Instead of manually constructing byte keys, modules define typed collections such as: + +- `collections.Item[T]` +- `collections.Map[K, V]` +- `collections.Sequence` + +Example: + +```go +// declared in the keeper struct +Counter collections.Item[uint64] + +// read and write in message handlers +count, _ := k.Counter.Get(ctx) +k.Counter.Set(ctx, count+1) +``` + +The Collections API defines the storage schema, handles encoding and decoding, ensures consistent key construction, and makes state access type-safe. + +Under the hood, collections still store data in a KVStore. Collections are used to provide a safer abstraction over raw byte keys. See [`collections/collections.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/collections/collections.go) for the base interface definitions. For the full package guide, see [Collections](/sdk/v0.54/guides/state/collections). + +## How modules access state + +Modules do not interact with the multistore directly. Instead, each module defines a keeper that opens its KVStore through the execution `Context` it receives on each call. For details on how `Context` carries the store reference at runtime, see [Execution Context](/sdk/v0.54/learn/concepts/context-gas-events). + +A keeper typically holds: + +- the module's **store key** (an object-capability used to open the module's `KVStore` from `Context`), +- a **Protobuf codec** used to encode and decode values stored as bytes, +- **references (interfaces) to other keepers** the module depends on. + +State access typically flows through the keeper: + +```go +MsgServer / QueryServer + ↓ + Keeper + ↓ + KVStore +``` + +The keeper exposes high-level methods that construct keys, encode values, and enforce business logic: + +```go +func (k Keeper) GetBalance(ctx sdk.Context, addr sdk.AccAddress) sdk.Coins +func (k Keeper) SetParams(ctx sdk.Context, params types.Params) +``` +For the keeper's role within a module, see [Keeper](/sdk/v0.54/learn/concepts/modules#keeper). + +## Genesis and chain initialization + +Before the first block executes, the chain must start with an initial state called **genesis**, defined in `genesis.json`. Genesis is the first write to the KVStores — it is how every module's state exists before any transaction runs. + +During `InitChain`, `BaseApp` calls each module's `InitGenesis` to populate its store: + +```text +genesis.json + ↓ +BaseApp.InitChain + ↓ +Module.InitGenesis + ↓ +KVStores populated +``` + +For information on how modules define their genesis methods (`DefaultGenesis`, `ValidateGenesis`, `InitGenesis`, `ExportGenesis`) and initialization ordering, see [Intro to Modules](/sdk/v0.54/learn/concepts/modules) and [Transaction Lifecycle](/sdk/v0.54/learn/concepts/lifecycle). + +For a walkthrough of genesis implementation in a module, see [Step 2: Proto files](/sdk/v0.54/tutorials/example/03-build-a-module#step-2-proto-files) and [Step 8: module.go](/sdk/v0.54/tutorials/example/03-build-a-module#step-8-modulego) in the Build a Module tutorial. + +## Next steps + +For more information on stores, pruning strategies, and store configuration, see the [store spec](/sdk/v0.54/guides/state/store). For the full store interface definitions, see [`store/types/store.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/store/types/store.go) in the SDK source. + +Because KV stores only hold raw bytes, modules must serialize structured data before writing it. The next section, [Encoding and Protobuf](/sdk/v0.54/learn/concepts/encoding), explains how the Cosmos SDK uses Protocol Buffers to encode that data deterministically, and why every validator must produce exactly the same bytes. diff --git a/sdk/v0.54/learn/concepts/testing.mdx b/sdk/v0.54/learn/concepts/testing.mdx new file mode 100644 index 000000000..734a91ec3 --- /dev/null +++ b/sdk/v0.54/learn/concepts/testing.mdx @@ -0,0 +1,362 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/learn/concepts/testing' +title: Testing in the SDK +--- + +The Cosmos SDK provides a layered testing approach that mirrors the architecture of the framework itself. Tests are organized into three levels, each testing a progressively larger slice of the application. This page uses the counter module example in the `example` repo, not the minimal counter module example, because the fuller module includes the testing surfaces needed for these examples. + +The examples on this page come from the [Full Counter Module Walkthrough](/sdk/v0.54/tutorials/example/04-counter-walkthrough#unit-tests) and [Running and Testing](/sdk/v0.54/tutorials/example/05-run-and-test) tutorials. + +## Three testing levels + +### Keeper unit tests + +Keeper unit tests verify keeper logic in isolation, without starting a full application. They construct a minimal in-memory context with a real KV store, initialize the keeper under test, and call its methods directly. No server, no network, no block processing. + +The counter module keeper tests live in `x/counter/keeper/keeper_test.go`. The test suite sets up a keeper with a live store and mock dependencies: + +```go +type KeeperTestSuite struct { + suite.Suite + + ctx sdk.Context + keeper *keeper.Keeper + queryClient types.QueryClient + msgServer types.MsgServer + bankKeeper *MockBankKeeper + authority string +} + +func (s *KeeperTestSuite) SetupTest() { + key := storetypes.NewKVStoreKey("counter") + storeService := runtime.NewKVStoreService(key) + testCtx := testutil.DefaultContextWithDB(s.T(), key, storetypes.NewTransientStoreKey("transient_test")) + ctx := testCtx.Ctx.WithBlockHeader(cmtproto.Header{Time: cmttime.Now()}) + encCfg := moduletestutil.MakeTestEncodingConfig() + + s.authority = "cosmos10d07y265gmmuvt4z0w9aw880jnsr700j6zn9kn" + s.bankKeeper = &MockBankKeeper{} + k := keeper.NewKeeper(storeService, encCfg.Codec, s.bankKeeper, keeper.WithAuthority(s.authority)) + + s.ctx = ctx + s.keeper = k + + queryHelper := baseapp.NewQueryServerTestHelper(ctx, encCfg.InterfaceRegistry) + types.RegisterQueryServer(queryHelper, keeper.NewQueryServer(k)) + s.queryClient = types.NewQueryClient(queryHelper) + s.msgServer = keeper.NewMsgServerImpl(k) +} +``` + +`testutil.DefaultContextWithDB` creates a real KV store backed by an in-memory database. `moduletestutil.MakeTestEncodingConfig` returns a codec configured for the test. The `MockBankKeeper` replaces the real bank keeper with a struct whose behavior can be controlled per test case: + +```go +type MockBankKeeper struct { + SendCoinsFromAccountToModuleFn func(ctx context.Context, senderAddr sdk.AccAddress, recipientModule string, amt sdk.Coins) error +} +``` + +A typical keeper test case covers the happy path and the error conditions with table-driven tests: + +```go +func (s *KeeperTestSuite) TestAddCount() { + testCases := []struct { + name string + setup func() + sender string + amount uint64 + expErr bool + expErrMsg string + expPostCount uint64 + }{ + { + name: "add to zero counter", + setup: func() { + err := s.keeper.InitGenesis(s.ctx, &types.GenesisState{ + Count: 0, + Params: types.Params{MaxAddValue: 100}, + }) + s.Require().NoError(err) + }, + sender: "cosmos1test", + amount: 10, + expErr: false, + expPostCount: 10, + }, + { + name: "add exceeds max_add_value - should error", + setup: func() { + err := s.keeper.InitGenesis(s.ctx, &types.GenesisState{ + Count: 0, + Params: types.Params{MaxAddValue: 50}, + }) + s.Require().NoError(err) + }, + sender: "cosmos1test", + amount: 100, + expErr: true, + expErrMsg: "exceeds max allowed", + }, + } + + for _, tc := range testCases { + s.Run(tc.name, func() { + s.SetupTest() + tc.setup() + + newCount, err := s.keeper.AddCount(s.ctx, tc.sender, tc.amount) + if tc.expErr { + s.Require().Error(err) + if tc.expErrMsg != "" { + s.Require().Contains(err.Error(), tc.expErrMsg) + } + } else { + s.Require().NoError(err) + s.Require().Equal(tc.expPostCount, newCount) + + count, err := s.keeper.GetCount(s.ctx) + s.Require().NoError(err) + s.Require().Equal(tc.expPostCount, count) + } + }) + } +} +``` + +`msg_server_test.go` uses the same suite to test the `MsgServer` layer, including event emission: + +```go +func (s *KeeperTestSuite) TestMsgAddEmitsEvent() { + s.SetupTest() + err := s.keeper.InitGenesis(s.ctx, &types.GenesisState{ + Count: 0, + Params: types.Params{MaxAddValue: 100}, + }) + s.Require().NoError(err) + + _, err = s.msgServer.Add(s.ctx, &types.MsgAddRequest{Sender: "cosmos1test", Add: 42}) + s.Require().NoError(err) + + events := s.ctx.EventManager().Events() + s.Require().NotEmpty(events) + + found := false + for _, event := range events { + if event.Type == "count_increased" { + found = true + } + } + s.Require().True(found, "count_increased event not found") +} +``` + +Keeper unit tests are fast, deterministic, and surgical. They are the right level for testing business logic, error conditions, edge cases, and event emission. + +### Integration tests + +Integration tests verify behavior across the full application stack. They start a real in-memory network with one or more validators, wait for blocks to be produced, broadcast actual signed transactions via gRPC, and query the resulting state. These tests exercise the AnteHandler, message routing, block execution, and state commitment together. + +The counter module integration tests live in `tests/counter_test.go`. The test suite uses `testutil/network` from the Cosmos SDK to spin up a full in-memory chain: + +```go +type E2ETestSuite struct { + suite.Suite + + cfg network.Config + network *network.Network + conn *grpc.ClientConn +} + +func (s *E2ETestSuite) SetupSuite() { + s.T().Log("setting up e2e test suite") + + var err error + s.cfg = network.DefaultConfig(NewTestNetworkFixture) + s.cfg.NumValidators = 1 + + // Customize counter genesis to set initial count and permissive params + genesisState := s.cfg.GenesisState + counterGenesis := countertypes.GenesisState{ + Count: 0, + Params: countertypes.Params{ + MaxAddValue: 1000, + AddCost: nil, + }, + } + counterGenesisBz, err := s.cfg.Codec.MarshalJSON(&counterGenesis) + s.Require().NoError(err) + genesisState[countertypes.ModuleName] = counterGenesisBz + s.cfg.GenesisState = genesisState + + s.network, err = network.New(s.T(), s.T().TempDir(), s.cfg) + s.Require().NoError(err) + + _, err = s.network.WaitForHeight(2) + s.Require().NoError(err) + + val0 := s.network.Validators[0] + s.conn, err = grpc.NewClient( + val0.AppConfig.GRPC.Address, + grpc.WithTransportCredentials(insecure.NewCredentials()), + grpc.WithDefaultCallOptions(grpc.ForceCodec(codec.NewProtoCodec(s.cfg.InterfaceRegistry).GRPCCodec())), + ) + s.Require().NoError(err) +} +``` + +`NewTestNetworkFixture` (in `tests/test_helpers.go`) constructs the `ExampleApp` with `dbm.NewMemDB()` and returns a `network.TestFixture` that configures the in-memory validator. This lets the SDK's network test helper start a real application with real consensus. + +A test that exercises the full transaction path: + +```go +func (s *E2ETestSuite) TestAddCounter() { + val := s.network.Validators[0] + + initialCount := s.getCurrentCount() + + txBuilder := s.mkCounterAddTx(val, 42) + txBytes, err := val.ClientCtx.TxConfig.TxEncoder()(txBuilder.GetTx()) + s.Require().NoError(err) + + txClient := txtypes.NewServiceClient(s.conn) + grpcRes, err := txClient.BroadcastTx( + context.Background(), + &txtypes.BroadcastTxRequest{ + Mode: txtypes.BroadcastMode_BROADCAST_MODE_SYNC, + TxBytes: txBytes, + }, + ) + s.Require().NoError(err) + s.Require().Equal(uint32(0), grpcRes.TxResponse.Code, "tx failed: %s", grpcRes.TxResponse.RawLog) + + s.Require().NoError(s.network.WaitForNextBlock()) + + finalCount := s.getCurrentCount() + s.Require().Equal(initialCount+42, finalCount) +} +``` + +Integration tests are slower than keeper unit tests because they start a real consensus engine and wait for blocks. They exist to catch failures at the boundaries: AnteHandler rejections, routing errors, genesis state mismatches, and cross-module interactions that only manifest when the full stack is running. + +### Simulation tests + +Simulation tests are property-based tests. Instead of testing specific inputs, they generate large volumes of random operations and verify that the application's invariants hold throughout. They catch bugs that deterministic test cases miss: unexpected ordering effects, state corruption under high load, and invariant violations that only appear after many sequential operations. + +The Cosmos SDK simulation framework drives this through [`simsx`](#simsx-and-simd). The counter module defines a message factory that generates random `MsgAddRequest` messages: + +```go +// x/counter/simulation/msg_factory.go + +func MsgAddFactory() simsx.SimMsgFactoryFn[*types.MsgAddRequest] { + return func(ctx context.Context, testData *simsx.ChainDataSource, reporter simsx.SimulationReporter) ([]simsx.SimAccount, *types.MsgAddRequest) { + sender := testData.AnyAccount(reporter) + if reporter.IsSkipped() { + return nil, nil + } + + r := testData.Rand() + addAmount := uint64(r.Intn(100) + 1) + + msg := &types.MsgAddRequest{ + Sender: sender.AddressBech32, + Add: addAmount, + } + + return []simsx.SimAccount{sender}, msg + } +} +``` + +The simulation runner selects a random account and a random add amount within valid bounds, then executes the message against the live application. This runs thousands of times across a simulated block sequence. + +The top-level simulation test in `sim_test.go` wires everything together: + +```go +//go:build sims + +func TestFullAppSimulation(t *testing.T) { + simsx.Run(t, NewExampleApp, setupStateFactory) +} + +func setupStateFactory(app *ExampleApp) simsx.SimStateFactory { + return simsx.SimStateFactory{ + Codec: app.AppCodec(), + AppStateFn: simtestutil.AppStateFn(app.AppCodec(), app.SimulationManager(), app.DefaultGenesis()), + BlockedAddr: BlockedAddresses(), + AccountSource: app.AccountKeeper, + BalanceSource: app.BankKeeper, + } +} +``` + +The `//go:build sims` build tag means simulation tests are excluded from regular `go test` runs and only execute when explicitly requested with `-tags sims`. This keeps CI fast. + +The simulation manager is initialized in `app.go`: + +```go +overrideModules := map[string]module.AppModuleSimulation{ + authtypes.ModuleName: auth.NewAppModule(appCodec, app.AccountKeeper, authsims.RandomGenesisAccounts, nil), +} +app.sm = module.NewSimulationManagerFromAppModules(app.ModuleManager.Modules, overrideModules) +app.sm.RegisterStoreDecoders() +``` + +`NewSimulationManagerFromAppModules` collects simulation support from all modules that implement `AppModuleSimulation`. `RegisterStoreDecoders` registers human-readable decoders for each module's store entries, used when the simulation framework logs state for debugging. + +## Test utilities + +### testutil + +The [`testutil`](https://github.com/cosmos/cosmos-sdk/tree/main/testutil) package provides helpers for constructing in-memory contexts for unit tests: + +- `testutil.DefaultContextWithDB` creates a real `sdk.Context` backed by an in-memory KV store. Keeper unit tests use this to get a realistic execution context without starting a full node. +- `moduletestutil.MakeTestEncodingConfig` returns a codec with standard interface registration, suitable for keeper tests. +- `baseapp.NewQueryServerTestHelper` creates a `QueryServiceTestHelper` that implements both the gRPC Server and ClientConn interfaces, allowing keeper tests to register query services and invoke them directly without a network connection. + +### testify suite + +The SDK's test files use the [`testify/suite`](https://pkg.go.dev/github.com/stretchr/testify/suite) package. A `suite.Suite` groups test setup, teardown, and test methods into a single struct. `SetupTest` runs before each test method; `SetupSuite` runs once before all tests in the suite. + +```go +func TestKeeperTestSuite(t *testing.T) { + suite.Run(t, new(KeeperTestSuite)) +} +``` + +`suite.Run` discovers methods on the struct whose names start with `Test` and runs them as individual test cases. `s.Require()` returns assertion helpers that stop the test immediately on failure, while `s.Assert()` continues after a failure. + +## simsx and simd + +For a full guide on configuring and running simulations, see the [Module Simulation](/sdk/v0.54/guides/testing/simulator) page. + +[`simsx`](https://github.com/cosmos/cosmos-sdk/tree/main/testutil/simsx) is the simulation execution framework. It provides: + +- `SimMsgFactoryFn`: a function type that implements the `SimMsgFactoryX` interface for message factories. Each factory selects random accounts and parameters, constructs a message, and returns it for execution. +- `ChainDataSource`: provides access to random accounts, balances, and other chain data during message construction. +- `SimulationReporter`: allows a factory to signal that it should be skipped (for example, if no suitable account exists). +- `simsx.Run`: the top-level entry point that drives a full simulation run against the application. + +[`simd`](/sdk/v0.54/node/prerequisites) is the reference simulation binary provided by the Cosmos SDK. It is a fully configured simapp (`simapp`) compiled as a standalone binary, used to run simulations against the SDK's own module set without setting up a custom chain. For a custom chain like the example app, you use your own binary with the `sims` build tag. To learn how to run an example chain, visit the [simd node tutorial](/sdk/v0.54/node/run-node). + +To run simulations against the example app: + +```bash +go test -tags sims -run TestFullAppSimulation ./... +``` + +To add simulation support to your own module — implementing `AppModuleSimulation`, writing message factories, and wiring the `SimulationManager` — see [Module Simulation](/sdk/v0.54/guides/testing/simulator). + +## Telemetry + +The counter module uses OpenTelemetry to emit metrics from keeper operations: + +```go +var ( + meter = otel.Meter("github.com/cosmos/example/x/counter") + + countMetric metric.Int64Counter +) +``` + +The SDK provides a telemetry package built around OpenTelemetry, with legacy support for go-metrics. Modules can emit counters, gauges, and histograms from keeper methods to expose runtime behavior for monitoring. See [Telemetry](/sdk/v0.54/guides/testing/telemetry) for full details. diff --git a/sdk/v0.54/learn/concepts/transactions.mdx b/sdk/v0.54/learn/concepts/transactions.mdx new file mode 100644 index 000000000..aefa3ab2d --- /dev/null +++ b/sdk/v0.54/learn/concepts/transactions.mdx @@ -0,0 +1,208 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/learn/concepts/transactions' +title: Transactions, Messages, and Queries +--- + +In the previous section, you learned that [accounts](/sdk/v0.54/learn/concepts/accounts) authorize activity on a chain using digital signatures and sequence numbers. Accounts provide identity and permission, but transactions are the actual mechanism that authorizes and executes logic on the chain. + +## Interacting with a chain + +A Cosmos SDK blockchain is a deterministic state machine. Its state changes only when transactions are executed and committed in blocks. + +Users and applications interact with the blockchain in two fundamental ways: + +- **Transactions** modify state and are included in blocks. When a user wants to **change** something (transfer tokens, delegate stake, submit a governance proposal), they submit a transaction. +- **Queries** read state and are not included in blocks. When a user wants to **inspect** something (check a balance, view delegations, read proposal details), they perform a query. + +Only transactions affect consensus state. + +## Transactions + +A **transaction** is a signed container that carries one or more actions to be executed on the blockchain. + +A transaction includes: + +- Messages: one or more actions you want to execute (send tokens, delegate stake, vote on a proposal) +- Signatures: cryptographic proof that you authorize these actions +- Sequence number: prevents someone from resubmitting your transaction (replay protection) +- Gas limit: the maximum computational resources you're willing to spend +- Fees: what you pay for the transaction to be processed + +The transaction itself does not define business logic. Instead, it packages intent (messages) to change state, proves authorization (signatures), and specifies execution limits (gas and fees). You can think of a transaction as an envelope you send to the blockchain, with a message inside containing instructions, a signature to prove authenticity, and a stamp to pay for postage. + + +```text +Transaction + ├── Message 1 + ├── Message 2 + ├── ... + ├── Signature(s) + ├── Sequence + ├── Gas limit + └── Fees +``` + +In the Cosmos SDK, account metadata and transaction authorization are handled by the `x/auth` module. Transaction construction and encoding are configured through the SDK's transaction system (commonly via `x/auth/tx`). + +## Messages + +A **message** (`sdk.Msg`) is the actual instruction inside a transaction. Each message is defined by a specific module and represents a single action. Messages are located in that module's `types` package (like `x/bank/types` or `x/staking/types`). Modules define which messages they support and the rules for executing them. While the transaction provides the envelope with signatures and fees, the message defines the specific action to execute. + +Examples include `MsgSend` (transfer tokens), `MsgDelegate` (delegate stake), and `MsgVote` (vote on proposals). + +If a transaction contains multiple messages, they execute in order. See [Message execution and atomicity](#message-execution-and-atomicity) below for details. + +### How messages are defined + +Messages in the Cosmos SDK are defined in each module's [`tx.proto` file](/sdk/v0.54/tutorials/example/03-build-a-module#txproto) using [Protocol Buffers (protobuf)](/sdk/v0.54/learn/concepts/encoding), which provides deterministic serialization, backward compatibility, and cross-language support. Each message is defined in a `.proto` file that specifies its fields, data types, and unique identifiers. From this schema, code is generated that allows the message to be constructed, serialized, and validated. + +Here's an example of a transaction in JSON format: + +```json +{ + "body": { + "messages": [ + { + "@type": "/cosmos.bank.v1beta1.MsgSend", + "from_address": "cosmos1...", + "to_address": "cosmos1...", + "amount": [{"denom": "uatom", "amount": "1000000"}] + } + ], + "memo": "", + "timeout_height": "0", + "extension_options": [], + "non_critical_extension_options": [] + }, + "auth_info": { + "signer_infos": [ + { + "public_key": { + "@type": "/cosmos.crypto.secp256k1.PubKey", + "key": "A..." + }, + "mode_info": {"single": {"mode": "SIGN_MODE_DIRECT"}}, + "sequence": "0" + } + ], + "fee": { + "amount": [{"denom": "uatom", "amount": "500"}], + "gas_limit": "200000", + "payer": "", + "granter": "" + } + }, + "signatures": ["MEUCIQDx..."] +} +``` + +This transaction transfers 1 ATOM (1,000,000 uatom) from one account to another. You can see the message in the `body.messages` array, the sender's public key and sequence in `auth_info.signer_infos`, the fee and gas limit in `auth_info.fee`, and the cryptographic signature in the `signatures` array. + +When broadcast, this JSON is serialized into bytes using protobuf, ensuring every validator interprets the transaction identically. + +### Message execution and atomicity + +When a transaction contains multiple messages, they are executed **in the order they appear** in the transaction. + +For example, a transaction might: + +1. Send tokens to another account. +2. Delegate those tokens to a validator. + +If the order were reversed, the delegation could fail due to insufficient balance. + +At execution time, messages inside a transaction are applied sequentially. The transaction succeeds only if all messages execute successfully. + +Conceptually: + +```text +Transaction + ├── Msg 1 → execute + ├── Msg 2 → execute + ├── Msg 3 → execute +``` + +If any message fails, the transaction returns an error and none of the message execution writes from that transaction are committed. + +Message execution inside a transaction is atomic: all messages commit or none do. The [transaction lifecycle](/sdk/v0.54/learn/concepts/lifecycle) page covers this execution pipeline in more detail. + + + +In v0.53, transactions support an optional **unordered** mode. When `unordered=true`, the normal per-signer sequence check is bypassed and replay protection is handled through `timeout_timestamp` plus unordered nonce tracking in `x/auth`. This enables fire-and-forget and concurrent transaction submission without coordinating sequence numbers. Unordered transactions must have a `timeout_timestamp` set and a sequence of `0`. For how clients build and submit them, see [Generating an Unordered Transaction](/sdk/v0.54/node/txs#generating-an-unordered-transaction). + + +## Blocks and transactions + +A blockchain can be understood as a sequence of blocks. Each block contains an ordered list of transactions. + +When a new block is committed: + +1. Each transaction in the block is applied to the current state. +2. Each transaction executes its messages in order. +3. Modules update their portion of state. +4. The resulting state becomes the starting point for the next block. + +Conceptually: + +```text +State₀ + ↓ apply Block 1 (Tx₁, Tx₂, Tx₃) +State₁ + ↓ apply Block 2 (Tx₄, Tx₅) +State₂ + ↓ apply Block 3 (...) +State₃ +``` + +In this way, the blockchain is a deterministic sequence of state transitions driven entirely by transactions. + +Blocks group transactions, transactions drive execution, and execution updates state. + +## Queries + +A **query** retrieves data from the blockchain's state without modifying it. + +Queries are read-only. They don't require signatures, aren't included in blocks, and don't affect consensus state. Modules define query services using protobuf in a [`query.proto` file](/sdk/v0.54/tutorials/example/03-build-a-module#queryproto), exposed over gRPC and REST. + +For example: + +- Query an account's balance (the `x/bank` module) +- Query staking delegations (the `x/staking` module) +- Query governance proposal details (the `x/gov` module) + + +## Transaction and query flow + + + + + + + + + + +
Transaction FlowQuery Flow
+
User
+  ↓ signs
+Transaction
+  ↓ contains
+Message(s)
+  ↓ handled by
+Module(s)
+  ↓ update
+State
+ 		+
User
+  ↓
+Query
+  ↓
+Module
+  ↓
+State (read-only)
+
+ +Transactions modify the blockchain. Messages define what modifications occur. Modules execute those modifications in order. Queries allow anyone to observe the resulting state. To see this flow in action with a working chain, see the [Quickstart](/sdk/v0.54/tutorials/example/02-quickstart) tutorial. + +The next section, [Transaction Lifecycle](/sdk/v0.54/learn/concepts/lifecycle), follows a transaction from broadcast through validation, block inclusion, execution, and state commitment to show how these components work together in practice. diff --git a/sdk/v0.54/learn/intro/blockchain-basics.mdx b/sdk/v0.54/learn/intro/blockchain-basics.mdx new file mode 100644 index 000000000..250f5855f --- /dev/null +++ b/sdk/v0.54/learn/intro/blockchain-basics.mdx @@ -0,0 +1,170 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/learn/intro/blockchain-basics' +title: Blockchain Basics +description: 'Learn the fundamentals of blockchains, state machines, and how Cosmos SDK applications work.' +--- + +import { BlockchainDemo } from '/snippets/blockchain-demo.jsx'; + +## What Is a Blockchain? + +A blockchain is a decentralized ledger that multiple independent computers (called nodes) maintain together. Instead of relying on a single authority to track transactions and maintain state, blockchain networks distribute this responsibility across many nodes. Each node keeps its own copy of the ledger and works with other nodes to agree on what transactions are valid and in what order they should be applied. + +You can think of a blockchain or decentralized ledger as a shared spreadsheet that dozens of people maintain independently. Everyone has their own copy, and they all follow the same rules for updating it. When someone wants to make a change, the group agrees on whether that change is valid and what order it should happen in. If everyone follows the rules correctly, all copies end up identical. If someone tries to modify their copy without following the consensus rules, the other nodes will reject their version because it doesn't match what the network agreed upon. This makes blockchains resistant to tampering: you'd need to control a majority of the network to force through an invalid change. + +## Why Blockchains? + +Traditional digital systems usually rely on a central authority to maintain accurate records. A bank, for example, maintains the definitive record of account balances. Users trust the bank to process transactions correctly and prevent problems like spending the same money twice (also known as the double-spend problem). + +Blockchains solve a more difficult challenge: maintaining accurate, trustworthy records without relying on a singular, central authority. In a decentralized network, no single entity has the final say. Instead, independent nodes must agree on the state of the ledger even though they don't trust each other. This requires solving several problems simultaneously: + +- [Agreement through consensus](#consensus): How do nodes agree on which transactions are included and in what order they’re applied? +- [Security through tamper-evident cryptography](#how-blocks-are-linked): How can the network prevent malicious nodes from creating fraudulent transactions or rewriting history? +- [Consistency through deterministic execution](#why-“deterministic”): How do all nodes maintain identical copies of the ledger despite network delays and potential failures? + +Blockchains address these challenges through cryptographic linking, deterministic execution, and decentralized consensus mechanisms. The result is a system where no single party controls the ledger, yet all participants can verify its accuracy and trust its contents. + +## State Machines: The Foundation of Blockchains + +At their core, blockchains are **replicated, deterministic state machines**. + +### What Is a State Machine? + +In computer science, **State** represents all the current data in a system at a specific point in time. For example, in a bank application, the state includes all account balances. In the context of a blockchain or decentralized ledger, the state includes all account balances, smart contract data, and other information the chain tracks. + +A **state machine** is a system that moves from one state to another by applying transactions. Each transaction describes an action that should change the state. + +Here's a simple example of a state machine using a bank account: + +```text +Current State: + User A's balance: $100 + User B's balance: $50 + +Transaction: User A sends $30 to User B + +New State: + User A's balance: $70 + User B's balance: $80 +``` + +The state machine takes the current state (User A has \$100, User B has \$50), applies a transaction (transfer \$30), and produces a new state (User A has \$70, User B has \$80). + +### Why "Deterministic"? + +**Deterministic** means that the same transaction applied to the same state will always produce the same result. This property is critical for blockchains and decentralized ledgers. + +Using the bank example: if User A starts with \$100 and sends User B \$30, their balance will always become \$70. It doesn't matter who processes this transaction, when they process it, or how many times they recalculate it from the initial state: the result will always be the same. + +In a blockchain, determinism ensures that all nodes independently arrive at the same final state. If the logic weren't deterministic, different nodes would end up with different versions of the ledger, and the network would break down. +In practice, blockchain applications must avoid sources of non-determinism such as local time, floating-point math, or external network calls. + +### Why "Replicated"? + +**Replicated** refers to the fact that many independent nodes each run their own copy of the same state machine. Instead of one central server maintaining the state, multiple independent nodes each maintain their own complete copy. + +When a new block is added to the blockchain, every node: +1. Receives the block with its ordered list of transactions +2. Independently executes each transaction through their local state machine +3. Arrives at the same new state (thanks to determinism). + +This replication is what makes blockchains decentralized and resilient. If any single node fails, goes offline, or acts maliciously, the network continues operating as long as a majority of the network's consensus power still have complete, accurate copies of the state. The network doesn't depend on any one node being available or trustworthy. + +## How Blockchains Work + +With an understanding of state machines, the next step is to see how blockchains use them to maintain a shared ledger across many independent nodes. + +### Nodes + +A **node** is a computer that participates in the blockchain network. Each node stores a complete copy of the blockchain's state, receives and validates new transactions, participates in consensus to agree on new blocks, and executes transactions to update its local state. Some nodes, called validators, participate directly in consensus by proposing and voting on blocks, while other nodes simply replicate and verify the chain. +In public, permissionless blockchains, anyone can typically run a node, which makes the network decentralized: no single entity controls the ledger. + + +### Transactions + +A **transaction (tx)** is a request to change the blockchain's state. In Cosmos SDK blockchains, transactions contain one or more **messages** that represent the specific actions to be executed. These messages can represent many different actions: +- Transferring tokens from one account to another +- Creating or updating a smart contract +- Staking tokens to become a validator +- Voting on a governance proposal + +When a user creates a transaction, it gets broadcast to nodes in the network. Nodes verify that the transaction is valid (proper signature, sufficient balance, etc.) before accepting it into their mempool. + +### Blocks + +Transactions are grouped together into **blocks** for efficiency. A block is a batch of transactions that the network processes together. Each block is cryptographically linked to the previous block, forming a **chain of blocks**. This chain structure creates a permanent, tamper-evident history: if someone tries to alter a past transaction, it would break the cryptographic link to all subsequent blocks, making the tampering obvious to the network. + +### From Transactions to Blocks + +Rather than processing transactions one at a time, blockchains group them into **blocks** for efficiency. Here's how it works: + +1. **Transaction pool (Mempool)**: Nodes collect valid transactions into a waiting area called the mempool +2. **Block proposal**: A designated node (called a validator or block proposer) selects transactions from the mempool and proposes them as the next block +3. **Consensus**: Nodes run a consensus algorithm to agree on which proposed block to accept and in what order +4. **Block commitment**: Once consensus is reached, the block becomes final and is added to the blockchain +5. **State transition**: Each node applies the transactions in the new block to their local state machine, updating their copy of the state + +```text +Mempool (pending txs) + ↓ + Block B +[Tx1, Tx2, Tx3, ...] + ↓ + Consensus + ↓ +Apply to State Machine + ↓ + New State +``` + +This process repeats for every block, creating a chain of blocks, or a "blockchain". + +### Consensus + +**Consensus** is the mechanism by which nodes agree on a single, authoritative version of the blockchain despite operating independently. In step 3 above, nodes must reach consensus on which block to add next and in what order. + +Transaction ordering is critical. Consider two transactions: "User A sends 100 tokens to User B" and "User A sends 100 tokens to User C." If User A only has 100 tokens, the order matters—only the first transaction can succeed. Different nodes might receive these transactions in different orders, so consensus is used to establish a single, canonical ordering that all nodes follow. This prevents the double-spend problem and ensures that deterministic execution produces identical results on every node. + +Consensus algorithms ensure that: +- All honest nodes agree on the same sequence of blocks +- The network can continue operating even if some nodes are offline or malicious +- Transactions are ordered consistently across all nodes + +Most Cosmos SDK blockchains use the CometBFT consensus engine, which implements a Byzantine Fault Tolerant (BFT) consensus algorithm. This means the network can reach agreement as long as more than two-thirds of the voting power comes from honest validators. The specifics of how consensus works are covered in the [Blockchain Architecture](/sdk/v0.54/learn/intro/sdk-app-architecture) section. It's important to note that consensus only determines the ordering and inclusion of transactions into blocks. Whether a transaction is valid is ultimately determined by the application’s state machine when the block is executed. + +### How Blocks Are Linked + +Each block contains a **block header** with metadata about the block. Critically, every block header includes a cryptographic hash of the previous block's header. + +A **hash** is like a digital fingerprint: it takes data of any size and produces a unique, fixed-length string of characters. For example, hashing the text "Hello World" might produce something like "a591a6d4...". The key property is that even a tiny change to the input (like changing "Hello World" to "Hello World!") produces a completely different hash. Hash functions are one-way, which means you can't reverse a hash back to the original data. Hash functions are also collision-resistant: no two different inputs produce the same hash. + +Cosmos blockchains use [SHA-256](https://nvlpubs.nist.gov/nistpubs/FIPS/NIST.FIPS.180-4.pdf) which was created by the NSA as the hash function for block headers and other cryptographic operations to securely link blocks together. This provides **cryptographic security**: finding a different input that produces the same hash output is computationally infeasible, making it virtually impossible to tamper with block data without detection. + +Block headers also include Merkle roots that commit to the block’s transactions and state, allowing nodes and light clients to verify data efficiently. + +This hashing mechanism creates a tamper-evident chain. You can see this in action in the demo in the next section. + +### Blockchain Demo: Immutability + +The demo below shows a blockchain with three blocks. You can see how each block is linked to the previous block by the hash in the block header. Try changing the data in a block to see how it changes the hash of that block and invalidates all subsequent blocks. You can add new blocks to the chain by clicking the "Add Block" button. + + + + +This is a simplified demonstration. Actual Cosmos SDK blocks include additional security features like validator signatures, timestamps, consensus information, and Merkle roots for transaction verification. The cryptographic linking shown here is just one part of blockchain security. + + +If someone tries to alter a transaction in Block 1, it would change the contents of Block 1, which would change Block 1's hash. But Block 2 stores Block 1's original hash in its header. The mismatch would be immediately obvious, and Block 2 would be pointing to a hash that no longer matches Block 1. This broken link would invalidate Block 2 and all subsequent blocks, making the tampering evident to the entire network. This is why blockchains are resistant to any changes: you’d need to control a supermajority of the network’s consensus power to rewrite history. + +This cryptographic linking is what makes blockchain history **immutable**, or unchangeable. The further back in history a block is, the more subsequent blocks depend on it remaining unchanged, making older blocks increasingly difficult to tamper with. In BFT-based systems like CometBFT, blocks have instant finality: once a block is committed, it cannot be reverted without violating consensus assumptions. + +## What's Next? + +Now that you understand blockchain fundamentals (state machines, deterministic execution, replication, and cryptographic linking), the next step is to learn how Cosmos SDK actually implements these concepts. + +In [Blockchain Architecture](/sdk/v0.54/learn/intro/sdk-app-architecture), you'll explore: +- How CometBFT handles consensus and networking to maintain the replicated state machine +- The Application Blockchain Interface (ABCI) that connects consensus to application logic +- How the Cosmos SDK implements the state machine layer +- The complete architecture of a Cosmos blockchain application \ No newline at end of file diff --git a/sdk/v0.54/learn/intro/cosmos-stack.mdx b/sdk/v0.54/learn/intro/cosmos-stack.mdx new file mode 100644 index 000000000..0fdfe0ea4 --- /dev/null +++ b/sdk/v0.54/learn/intro/cosmos-stack.mdx @@ -0,0 +1,52 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/learn/intro/cosmos-stack' +title: "The Cosmos Stack" +description: "Understanding the modular architecture of the Cosmos blockchain stack" +--- + +Performant, customizable, and EVM-compatible, the Cosmos stack offers builders full control of their blockchain infrastructure and implementation. Its stable and secure open-source codebase enables blockchains to achieve high throughput of 10,000+ TPS, tuned, and fast finality for instant transaction settlement. Development on the Cosmos stack began in 2016, and today, hundreds of public and private blockchains use the Cosmos stack in production. + +The stack is modular: leverage pre-built components or integrate custom features for your specific use case, from consensus mechanisms to governance and compliance. The components of the stack work together to create a complete blockchain network solution that is secure, performant, scalable, and endlessly customizable. + +![Cosmos Stack Architecture](/assets/public/cosmos-stack.png) + +At its core, the Cosmos stack is composed of several interoperable layers: the [Cosmos SDK](/sdk/v0.54/learn/intro/overview) for application logic, [CometBFT](/cometbft/latest/docs/README) for consensus and networking, [Cosmos EVM](/evm/v0.5.0/documentation/overview) for Ethereum compatibility, and [the Inter-Blockchain Communication Protocol (IBC)](/ibc) for trust-minimized cross-chain communication. + +For teams running permissioned or production networks, [Cosmos Enterprise modules](/sdk/v0.54/enterprise/overview) add hardened, licensed Cosmos SDK modules for controlled participation and collective on-chain authorization. + +Together, these components form a flexible, battle-tested stack for developing performant, reliable, interoperable, and secure blockchains + +## Cosmos SDK: Business Logic Layer + +The Cosmos SDK is the business logic layer of the Cosmos stack. It provides a customizable base layer for building blockchains and digital ledgers, made up of interoperable modules that work together to define how a blockchain behaves, from accounts and transactions to tokenization, compliance, and custom application logic. Builders can compose [pre-built modules](/sdk/v0.54/modules/modules) and [develop bespoke ones](/sdk/v0.54/guides/module-design/module-design-considerations) to embed their unique business logic directly into the foundation of the chain, rather than deploying it as isolated smart contracts. + +This approach enables a level of customization, interoperability, and performance that traditional smart contract platforms and Layer-2 blockchains cannot offer. Developers gain access to block lifecycle hooks ([BeginBlocker and EndBlocker](/sdk/v0.54/learn/intro/sdk-app-architecture#block-lifecycle-hooks)), fine-grained state separation, and scoped permissions through a security-first [Object Capability Model](/sdk/v0.54/guides/module-design/ocap). Native execution alongside [CometBFT](/cometbft/latest/docs/README) consensus unlocks significantly higher throughput and deterministic behavior, while upgrade tools like [Cosmovisor](/sdk/v0.54/guides/upgrades/cosmovisor) make chains easy to maintain and evolve over time. + +[Explore the Cosmos SDK →](/sdk/v0.54/learn) + +## Cosmos Enterprise modules + +Cosmos Enterprise modules are hardened Cosmos SDK modules for permissioned and production networks, including permissioned consensus (PoA) and multi-sig (Groups). The module source is published under the Source Available Evaluation License, and production use requires an Enterprise License from Cosmos Labs. + +[Learn about Cosmos Enterprise modules →](/sdk/v0.54/enterprise/overview) + +## Cosmos EVM: Ethereum Compatibility Layer + +Cosmos EVM enables plug-and-play Ethereum Virtual Machine compatibility for Cosmos SDK–based chains. It allows developers to deploy Solidity smart contracts, use familiar Ethereum tooling, and interact with native Cosmos modules (including IBC) through precompiles and extensions. It provides software engineers with functionality beyond standard EVM for new use cases and workflows by allowing them to run existing Ethereum contracts without modification while also extending the EVM with new capabilities at the chain level. + +[Learn about Cosmos EVM →](/evm) + +## IBC Protocol: Interoperability Layer + +The Inter-Blockchain Communication (IBC) protocol is the interoperability layer of the Cosmos stack, enabling blockchains to securely transfer tokens, messages, and arbitrary data. Blockchains communicate over IBC with self-hosted infrastructure through point-to-point connections. It connects them into an interoperable network through trust-minimized communication with configurable permissioning while maintaining secure, independent execution. + +[Explore IBC Documentation →](/ibc) + +## CometBFT: Highly-Performant Consensus Layer + +CometBFT is the consensus layer of the Cosmos stack and one of the most widely adopted, battle-tested consensus engines for building blockchains and decentralized ledger networks. It is a Byzantine Fault Tolerant (BFT) middleware that takes a deterministic state transition machine (which can be written in any programming language) and securely replicates it across a distributed set of nodes. By separating consensus from application logic, CometBFT allows developers to build custom blockchains without implementing their own networking or consensus protocols. + +Responsible for proposing blocks, ordering transactions, and finalizing state transitions, CometBFT ensures that all nodes reach agreement on the canonical state of the chain. Highly performant and deterministic, it provides fast finality and can achieve throughput of up to 10,000 transactions per second (TPS), making it well-suited for high-performance, application-specific blockchains. + +[Learn about CometBFT →](/cometbft/latest/docs/README) diff --git a/sdk/v0.54/learn/intro/overview.mdx b/sdk/v0.54/learn/intro/overview.mdx new file mode 100644 index 000000000..f35b4d023 --- /dev/null +++ b/sdk/v0.54/learn/intro/overview.mdx @@ -0,0 +1,71 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/learn/intro/overview' +title: What is the Cosmos SDK +--- + +The [Cosmos SDK](https://github.com/cosmos/cosmos-sdk) is a secure, open-source framework for building application-specific blockchains and digital ledgers. It gives engineers full control over access control, security, business logic, and governance, while providing a robust set of pre-built modules covering common blockchain functionality. Organizations can use the Cosmos SDK to easily build and maintain any type of blockchain network, including private permissioned networks, public networks, and consortia. Cosmos SDK blockchains are natively interoperable through [IBC](/ibc). + +As the business logic layer of the Cosmos stack, the Cosmos SDK provides a fully customizable foundation for building blockchains from customizable modules that define how a chain behaves, from accounts and transactions to tokenization, compliance, and custom application logic. + +Development on the Cosmos SDK has been continuous since 2016. Today, companies in banking and finance, SaaS, AI, and other industries use the Cosmos SDK’s open-source codebase with proven consensus via [CometBFT](/cometbft) and native interoperability through IBC for business use cases like interbank networks, asset tokenization, and business automation. It offers fast performance of 10,000+ transactions per second, strong security, and resiliency in production. + +## Purpose of the Cosmos SDK + +At its core, the Cosmos SDK is designed to give developers full flexibility across the entire blockchain stack. Business logic can run natively at the protocol level, be exposed through modular components, or be extended with optional [virtual machine layers](/evm), depending on the needs of the application. + +The Cosmos SDK is interoperable by design. Chains built with the SDK can communicate with other blockchains via [IBC](/ibc) while maintaining independent execution and security. + +## Modularity of the Cosmos SDK + +## Interoperable Modules + +Blockchains built with the Cosmos SDK are composed of interoperable modules, each responsible for a specific function, such as [accounts](/sdk/v0.54/learn/concepts/accounts), transactions, governance, tokenization, or compliance logic. These modules are built on top of the [SDK's base application framework](/sdk/v0.54/learn/intro/sdk-app-architecture), which provides the shared execution environment that allows modules to operate together as a cohesive blockchain. + +The Cosmos SDK offers engineers a robust set of [predefined modules](/sdk/v0.54/modules/modules) that cover standard blockchain features, such as consensus, accounts, transfers, governance permissioning, fee distribution, and more. In addition, engineers can [build their own modules](/sdk/v0.54/guides/module-design/module-design-considerations) tailored to their application's requirements. By composing and extending modules, developers can build blockchains and ledgers that are optimized for performance, security, and long-term maintainability. + +The SDK's base layer handles core concerns such as [message routing, module lifecycle orchestration, and interaction with the underlying consensus engine](/sdk/v0.54/learn/intro/sdk-app-architecture). It defines clear boundaries between modules by isolating their state into [independent stores](/sdk/v0.54/learn/intro/sdk-app-architecture), while providing secure, well-defined interfaces for [cross-module communication](/sdk/v0.54/learn/concepts/modules#keeper). This structure allows modules to operate and evolve independently while preserving the overall system. + +## Cosmos SDK and the modularity of the Cosmos stack + +While SDK modules define how application logic is composed within a chain, the Cosmos SDK also supports modularity at the component level. Consensus, networking, and data availability are provided by external components like CometBFT. Execution logic is defined by SDK modules. + +We recommend building Cosmos SDK-based blockchains using [CometBFT](/cometbft/latest/docs/README) for consensus because it offers best-in-class performance and out-of-the-box interoperability between components. Alternatively, engineers can pair the Cosmos SDK with other consensus engines like modular execution and settlement architectures depending on their performance and security requirements. This flexibility allows ledger chains to evolve alongside their application and operational needs. + +## Application-Specific Blockchains + +A common development paradigm in blockchain ecosystems is the use of general-purpose virtual machine chains, where applications are deployed as smart contracts on top of a shared execution environment. While this approach is well suited for some use cases, it imposes constraints around performance, customization, and protocol-level control, which impact an organization’s infrastructure costs and security and compliance profile. + +Application-specific blockchains offer a different model. An application-specific blockchain is a blockchain or digital ledger that runs custom business logic at the protocol or chain level to accomplish a particular business use case. With the Cosmos SDK, developers can tailor execution logic, fee models, governance rules, and state transitions directly at the protocol level, enabling greater flexibility and performance. + + +## Virtual Machine Layers + +The Cosmos SDK allows for the use of smart contracts. Developers can add a virtual machine layer to add smart contract support. The [Cosmos EVM](/evm) supports Ethereum-compatible smart contracts and tooling. Engineers can also choose other VMs. + +## Security via the Object-Capability Model + +The Cosmos SDK uses a capabilities-based security model to enforce strict boundaries between modules. Rather than granting broad access to shared state, modules are given only the specific capabilities they require, making it easier to reason about authority, permissions, and potential attack surfaces. + +This design improves the security and composability of complex blockchain applications. For a deeper dive into this model, see the [Object-Capability Model](/sdk/v0.54/guides/module-design/ocap). + +## Cosmos Enterprise modules + +Cosmos Enterprise modules are hardened Cosmos SDK modules for permissioned and production networks, including permissioned consensus (PoA) and multi-sig (Groups). The module source is published under the Source Available Evaluation License, and production use requires an Enterprise License from Cosmos Labs. + +[Learn about Cosmos Enterprise modules →](/sdk/v0.54/enterprise/overview) + +## Why Build with the Cosmos SDK + +The Cosmos SDK is one of the most mature and widely adopted frameworks for building custom, modular blockchains. Key advantages include: + +- **Proven in production**: 200+ blockchains use the Cosmos SDK in production today for use cases such as interbank networks, regulated lending, and banking asset tokenization. +- **Strong security foundations**: Capabilities-based security informed by years of production experience. +- **Protocol-level customization**: Define application logic, governance, and economic models directly in the blockchain, not just in smart contracts. +- **Built-in interoperability**: Native interoperability through [IBC](/ibc) and extensible chain-level integration. +- **Flexible execution models**: Combine native modules with optional VM layers such as [Cosmos EVM](/evm/v0.5.0/documentation/overview). + +## Getting Started with the Cosmos SDK + +- Learn about the [architecture of a Cosmos SDK application](/sdk/v0.54/learn/intro/sdk-app-architecture) +- Run a blockchain in under 5 minutes with the [Cosmos SDK Node Tutorial](/sdk/v0.54/tutorials) diff --git a/sdk/v0.54/learn/intro/sdk-app-architecture.mdx b/sdk/v0.54/learn/intro/sdk-app-architecture.mdx new file mode 100644 index 000000000..4223d1239 --- /dev/null +++ b/sdk/v0.54/learn/intro/sdk-app-architecture.mdx @@ -0,0 +1,289 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/learn/intro/sdk-app-architecture' +title: Cosmos Architecture +description: How Cosmos SDK implements blockchains through separation of consensus, interface, and application logic. +--- + +In [Blockchain Basics](/sdk/v0.54/learn/intro/blockchain-basics), you learned that a blockchain is a replicated, deterministic state machine maintained by independent nodes through consensus. The Cosmos SDK implements this model through a clean separation of concerns: CometBFT handles consensus, networking, and block production; ABCI (Application Blockchain Interface) defines the boundary between consensus and application; and the Cosmos SDK implements the application logic and state machine. + +This page explains the high-level architecture of Cosmos SDK blockchains: how components interact, what each layer is responsible for, and why this separation matters. + +## Cosmos Application Architecture + +A Cosmos blockchain consists of 2 distinct layers: CometBFT for consensus and block production, and the Cosmos SDK layer which contains the application logic, modules, and transaction execution logic. Between these layers is the ABCI (Application Blockchain Interface), which is the interface that connects them. + +```text ++---------------------------------------+ +| | +| Cosmos SDK Application | <- State machine +| (Modules, Keepers, State) | - Transaction execution +| | - Business logic ++------------------+--------------------+ + | + ABCI <- Application Blockchain Interface + | ++------------------+--------------------+ +| | +| CometBFT | <- Consensus engine +| (Consensus, Networking, | - Block production +| Block Replication) | - p2p networking +| | ++---------------------------------------+ +``` + +- **[CometBFT](#cometbft)**: the consensus engine that handles networking, block production, and Byzantine Fault Tolerant consensus. + +- **[ABCI](#abci-application-blockchain-interface)**: the interface boundary that defines when and how CometBFT invokes the application, enforcing separation between consensus and execution. The ABCI is implemented in a Cosmos application via [`BaseApp`](#baseapp-and-appgo). + +- **[Cosmos SDK](#cosmos-sdk-application)**: the framework for building the blockchain application (often simply called "the application"), which is a state machine assembled from composable [modules](/sdk/v0.54/learn/concepts/modules). Each module owns a specific domain of logic, state, and transactions; together, modules define the chain's business logic, execute transactions, and produce cryptographic state commitments. This layer is defined in the [`app.go`](#baseapp-and-appgo) file of a Cosmos application. + +### Separating Consensus and Application Logic + +Separating consensus from application logic provides several benefits. Security improves because isolating consensus logic prevents application bugs from affecting block production or network stability. Flexibility increases as developers can build application-specific blockchains without reimplementing consensus. The modular design allows consensus and application layers to evolve independently, and reusability means one consensus engine (CometBFT) can power many different blockchains without modification. + +## Nodes and Daemons + +A blockchain is made up of nodes, or participants in the blockchain network. Each node runs a daemon process that includes both a CometBFT instance for consensus and networking, and a Cosmos SDK application for the state machine and execution logic. The daemon participates in networking, reaches consensus with other nodes, and executes transactions to update the application state. + +Nodes can operate in different roles. There are two main types of nodes, **validators** and **full nodes**: + +- **Validators** are nodes that participate in consensus by proposing and voting on blocks, with consensus power staked or attributed to them making them responsible for block production. They are in charge of validating blocks before voting, ensuring that only valid blocks are finalized. + +- **Full nodes** replicate and verify blocks without participating in consensus voting, maintaining complete state and answering queries but not voting on proposals. + +Although anyone can typically run a full node, becoming a validator depends on the chain's staking, governance, or permissioning rules. In a proof-of-stake blockchain, validator candidates are selected based on their stake, and they must meet certain criteria to be elected as validators. In a proof-of-authority blockchain, validators are selected based on permissioned criteria. + +To learn how to run a node, visit the [Cosmos Node Tutorial](/sdk/v0.54/tutorials). + +## CometBFT + +[CometBFT](/cometbft/latest/docs/README) is a Byzantine Fault Tolerant consensus engine that provides fast, deterministic finality. It's used by Cosmos SDK blockchains to replicate the state machine across a decentralized network. + +### Core Responsibilities + +CometBFT handles several core responsibilities: + +- **Peer-to-peer networking**: CometBFT manages node discovery, establishes and maintains connections with other validators and full nodes, and implements gossip protocols for propagating information across the network. + +- **Transaction propagation and mempool management**: When users submit transactions to a node, CometBFT gossips them to other nodes. Each node maintains a **mempool** (memory pool), which is a waiting area for valid transactions that have not yet been included in a block. The mempool holds transactions temporarily until a validator includes them in a block proposal. + +- **Block proposal and transaction ordering**: CometBFT uses deterministic proposer selection to choose which validator will propose the next block. The proposer selects transactions from the mempool, orders them, and packages them into a block proposal. + +- **Byzantine Fault Tolerant consensus**: CometBFT coordinates voting rounds where validators vote on block proposals. When a proposal receives votes from more than two-thirds of voting power, consensus is reached and the block is finalized. Validators cryptographically sign their votes, ensuring against double-voting and other forms of malicious behavior. + +- **Block replication**: Once finalized, CometBFT ensures the block is replicated across all nodes in the network, maintaining a consistent, ordered history of all committed blocks. + +To learn more about how CometbFT works, visit the [CometBFT Documentation](/cometbft/latest/docs/README). + +### Byzantine Fault Tolerance and Finality + +Byzantine Fault Tolerance (BFT) is a system's ability to reach consensus even when some participants crash, send conflicting messages, or act maliciously. The name comes from the [Byzantine Generals Problem](https://lamport.azurewebsites.net/pubs/byz.pdf). CometBFT tolerates up to one-third of voting power being faulty while still producing valid blocks, and maintains liveness as long as more than two-thirds of validators are online. Unlike proof-of-work blockchains, CometBFT provides instant finality: once a block is committed, it cannot be reverted. + +### Orchestrating Block Production + +CometBFT drives the entire block production lifecycle. It determines when blocks are produced (maintaining consistent block times), which validator proposes each block (through deterministic proposer selection), and the order in which transactions are included in blocks. CometBFT coordinates the consensus process by managing the voting rounds where validators evaluate and vote on block proposals. + +To learn more about how consensus in CometBFT works, visit the [CometBFT Documentation](/cometbft/latest/docs/introduction/intro). + +### Content Agnosticism + +CometBFT is agnostic to block content and application implementation. It treats transactions as opaque byte arrays, ensuring all nodes receive the same ordered sequence without interpreting what they mean. All application-specific logic lives in the SDK layer, which uses [Protocol Buffers](/sdk/v0.54/learn/concepts/encoding) to serialize structured messages into bytes CometBFT can carry. + +## ABCI (Application Blockchain Interface) + +ABCI is a strict request/response interface between CometBFT and a Cosmos SDK blockchain application. It defines the block execution lifecycle and ensures a clean separation between consensus and application logic to ensure secure, reliable block production that cannot be compromised by application faults or bugs. + +The ABCI is unidirectional: all calls flow from CometBFT to the application, and the application cannot call into or control CometBFT. The Cosmos SDK application must respond deterministically to all ABCI calls, producing the same results given the same inputs. + +The ABCI itself is a stateless protocol containing no business logic. It only provides the interface definitions between the CometBFT consensus engine, which handles block production, and the Cosmos SDK application, which defines the business logic and state machine. + +### Block Lifecycle Methods + +As the driver of block production, CometBFT controls when the Cosmos SDK application is invoked. It calls the application through ABCI at specific points during block production to validate transactions for the mempool (CheckTx), to construct or evaluate block proposals (PrepareProposal and ProcessProposal), to execute finalized blocks (FinalizeBlock), and to persist state (Commit). The SDK application responds to these calls but cannot initiate them. This means CometBFT, not the SDK application, determines the timing and cadence of block production and state transitions. This ABCI boundary also prevents application logic from influencing consensus. + + +```python ++---------------+ | +--------------------------+ +| CometBFT | | | SDK Application | +| (Consensus) | ABCI | (State Machine Logic) | ++---------------+ | +--------------------------+ +1) CheckTx | + Mempool validation ---------|-------> Validate tx (sigs, fees) + | +2) PrepareProposal | + Proposer builds block ------|--------> Construct block proposal + | +3) ProcessProposal | + Validators evaluate --------|--------> Validate proposal + | +4) Consensus | + BFT voting, finalizing block| + (SDK not involved) | + | +5) FinalizeBlock | + Execute block --------------|--------> PreBlock hooks + | BeginBlock hooks + | Execute transactions + | EndBlock hooks + | Return AppHash +6) Commit | + Persist state --------------|--------> Persist to disk + |<-------- Return AppHash +``` + +1. **CheckTx** validates transactions before adding them to the mempool. It checks that transactions are well-formed and economically viable (proper signature, sufficient fees) without making state changes. This protects the mempool against spam. + +2. **PrepareProposal** is invoked when a validator constructs a new block proposal. This gives the application limited control over block construction, allowing it to reorder transactions or add application-specific data. + +3. **ProcessProposal** is called when validators evaluate a block proposal from another validator. The application can validate the proposed block according to application-specific rules before voting to accept it. + +4. **Consensus** occurs entirely within CometBFT. After validators evaluate the proposal (via ProcessProposal), they participate in BFT voting rounds. If the proposal receives votes from more than two-thirds of voting power, consensus is reached and the block is finalized. + +5. **FinalizeBlock** is invoked after consensus on a block is reached. This is where state transitions occur, executing the entire block atomically. `BaseApp` invokes lifecycle hooks in this order: PreBlock hooks, BeginBlock hooks, transaction execution, and EndBlock hooks. The application returns the new AppHash (a cryptographic commitment to the state) and any validator set changes. + +6. **Commit** is called after FinalizeBlock to persist the finalized state to the nodes' local disk and return the [AppHash](#state) that gets included in the next block header. + + +**PreBlock** hooks were introduced in SDK v0.50, which run before BeginBlock. PreBlockers must be explicitly ordered using `SetOrderPreBlockers`. Some core modules (notably `x/auth`) require PreBlock execution; missing PreBlock wiring will cause runtime errors. + + +CometBFT decides when blocks happen and what order transactions appear in, the Cosmos SDK decides whether those transactions are valid and how they change state, and the ABCI defines the interface between the two. + +For a more in-depth look at the ABCI, visit the [ABCI page](/sdk/v0.54/guides/abci/abci). + +## Cosmos SDK Application + +A Cosmos SDK application is a deterministic state machine that defines a blockchain's behavior. It focuses entirely on defining what state the blockchain tracks, what transactions are valid, and how transactions change state. The applications defines transaction and message formats using Protocol Buffers for serialization, validates transactions by checking signatures and fees, executes message handlers to apply state transitions, maintains state across all modules, and produces the AppHash that cryptographically commits to the current state. + +### Application Structure + +A typical Cosmos SDK application consists of: + - **[`BaseApp`](#baseapp-and-app-go)**: boilerplate code that provides the ABCI implementation and execution framework for a chain to interact with CometBFT. + - **[Modules](#modules-transactions-and-application-logic)**: building blocks of domain-specific logic such as transactions, custom business logic, and governance/permissioning. + - **[State Multistore](#kv-stores-and-multistore)**: a collection of key-value stores that store the state of the application, isolated by module. + - **[Keepers](#keepers)**: providing interfaces for accessing module state while enforcing access control. + - **[app.go](#baseapp-and-app-go)**: serving as the composition root that wires everything together. + +For a complete overview of Cosmos SDK structure, visit the [Intro to SDK Structure](/sdk/v0.54/learn/concepts/sdk-structure) + +### BaseApp and app.go + +`BaseApp` is the Cosmos SDK's standard implementation of the ABCI interface. It handles all ABCI method calls from CometBFT, routes messages to the appropriate [module handlers](#modules-transactions-and-application-logic), manages state versioning and caching, and enforces transaction execution semantics. + +Developers do not implement ABCI directly; instead, they extend `BaseApp` and register their modules, handlers, and execution logic with it. + +To learn more about `BaseApp`, visit the [`BaseApp` page](/sdk/v0.54/learn/concepts/baseapp). + +The `app.go` file is the composition root of a Cosmos SDK application. This is where a specific blockchain is assembled by creating the `BaseApp` instance, instantiating all module keepers with their dependencies, registering store keys for each module's state, wiring module [lifecycle hooks](#block-lifecycle-hooks), and configuring transaction processing. + +To learn more about `app.go`, visit the [`app.go` page](/sdk/v0.54/learn/concepts/app-go). + +Modules also define genesis state initialization and migration logic to support chain upgrades, allowing application state to evolve safely over time. + +### Modules, Transactions, and Application Logic + +Modules are the building blocks of Cosmos SDK applications. Each module implements a specific domain of functionality: the bank module handles token transfers, the staking module manages validator delegation, the governance module implements on-chain proposals, and so on. Every module acts as its own mini state machine, processing transactions and updating state according to its own rules. Together, modules for the entire application form a single, cohesive state machine. + +A module provides its business logic through message handlers. Messages work like function calls that specify an operation (like "send 100 tokens to address X") with typed parameters. Users invoke module logic by submitting transactions that contain these messages. During execution, each message gets routed to its module's handler, which runs the business logic and updates state. + +In the Cosmos SDK, a **transaction** is a signed, serialized container that wraps one or more **messages**. Messages represent the actual operations to execute (like "send tokens" or "delegate stake"), while the transaction adds metadata like signatures, fees, and gas limit. Blocks contain transactions, and during block execution (FinalizeBlock), each transaction's messages are extracted and routed to the appropriate module handlers for execution. Transactions contain signatures from their creators authorizing the requested state change. + +Modules define message types using [Protocol Buffers (Protobuf)](/sdk/v0.54/learn/concepts/encoding), which provide type-safe, cross-language serialization. Since CometBFT treats transactions as raw bytes and [KV stores](#kv-stores-and-multistore) only accept byte arrays, Protobuf serializes structured messages and state data into bytes for transmission and storage. Modules also define state schemas (what data the module stores), state transitions (how messages modify state), queries (allowing clients to read module state), and optional lifecycle hooks for tasks that run at block boundaries. + +To learn more about the transactions and messages, visit the [Transactions page](/sdk/v0.54/learn/concepts/transactions). For more in-depth information on modules, visit the [Intro to Modules page](/sdk/v0.54/learn/concepts/modules) or check out [the Module Tutorial](/sdk/v0.54/tutorials/example/00-overview) to learn how to build a module from scratch. + +### Block Lifecycle Hooks + +Beyond processing individual transactions, modules can define lifecycle hooks that run at specific points during block execution. These hooks allow modules to perform tasks at block boundaries, such as minting rewards, updating validator sets, or preparing state before transactions execute. + +During FinalizeBlock, `BaseApp` invokes module hooks in this sequence: +1. **PreBlock** - Prepare state before block execution begins +2. **BeginBlock** - Perform tasks at block start (e.g., minting rewards) +3. **Transaction execution** - For each transaction: run AnteHandler, execute message handlers, run PostHandler (if configured) +4. **EndBlock** - Perform tasks at block end (e.g., updating validator sets) + +Modules are coordinated by a [`ModuleManager`](/sdk/v0.54/learn/concepts/baseapp#module-manager), which orchestrates these lifecycle events along with genesis initialization and module upgrades. + +### State + +In a Cosmos SDK application, state is stored as a collection of key-value pairs in a **multistore**. + +State changes occur during block execution. After consensus finalizes a block, `FinalizeBlock` is invoked by CometBFT via the ABCI, which executes each transaction in the Cosmos SDK application in order. For each transaction, the messages are extracted and routed to the `MsgServer` of the corresponding module. The `MsgServer` validates messages, and the [keepers](#keepers) executes the business logic of the module and updates the state. + +After executing the transactions in the block and updating state, each node computes the `AppHash` from its local state. The **`AppHash`** is a cryptographic proof of the state of the application at the end of the block, and is included in the next block header. This ensures that state updates only take effect once consensus is reached. By design, all state changes in a Cosmos SDK application are deterministic and replayable: executing the same block against the same initial state will always produce the same final state. + +#### Keepers + +Keepers are the gatekeepers to module state. They provide the only interface for accessing and mutating a module's state, enforce access control between modules, and encapsulate state access logic. This aligns with the [object capability model](/sdk/v0.54/guides/module-design/ocap), where modules can only access capabilities (other keepers) explicitly passed to them during initialization. Modules interact through keeper interfaces rather than directly accessing each other's state, enforcing modularity and preventing tight coupling. To learn more about keepers, visit the [Intro to Modules page](/sdk/v0.54/learn/concepts/modules#keeper). + +#### KV Stores and Multistore + +The state of a Cosmos SDK application is stored in a **multistore**, which is a collection of key-value stores. Each module owns a namespaced key-value store, isolating its data from other modules. The multistore combines all module stores into a single, unified state representation with height-based versioning for historical queries. + +KV stores only accept byte arrays (`[]byte`) as values, so any custom data structures must be marshaled using a [codec](/sdk/v0.54/learn/concepts/encoding) before being stored. This ensures consistent serialization across the application, typically using Protocol Buffers. + +```text ++-----------------------------------------+ +| Multistore (Root) | +| | +| +----------+ +----------+ +-----+ | +| | Bank | | Staking | | ... | | +| | Store | | Store | | | | +| +----------+ +----------+ +-----+ | +| | ++-----------------------------------------+ +``` + +To learn more, visit the [Store page](/sdk/v0.54/learn/concepts/store). + +#### Merkle Trees and Commitments + +Application state is committed using Merkle trees. Each module store is organized as a Merkle tree (implemented as an IAVL tree), and the multistore root is a tree of module store roots. The AppHash is the root hash of this multistore structure, uniquely identifying a state version. + +```text + AppHash (Root) + | + +-------------+-------------+ + | | + BankRoot StakingRoot + | | + +----+----+ +----+----+ + | | | | + Key1 Key2 Key3 Key4 +``` + +CometBFT includes the AppHash in block headers, allowing anyone to verify state commitments. This is crucial for light clients, which can verify state without downloading all blocks. + +Visit the [Store page](/sdk/v0.54/learn/concepts/store#how-state-is-stored-iavl-and-commit-stores) to learn more about how Cosmos uses IAVL trees. + +### How State Replicates Across Nodes + +The state changes described above happen independently on every node in the network. CometBFT does not replicate application state directly—instead, it replicates blocks (ordered transactions) and consensus decisions. Each node then independently executes these blocks through its local Cosmos SDK application. + +This design relies entirely on deterministic execution. When all nodes execute the same ordered transactions, deterministic execution guarantees they arrive at identical state. Nodes verify this agreement by comparing AppHash commitments. If execution were non-deterministic, nodes would compute different AppHashes and consensus would fail. + +After executing a block, each node computes the AppHash from its local state. This AppHash is included in the next block header. Validators cryptographically sign block headers that include the AppHash, attesting that they executed the block and arrived at that state. Nodes with divergent state produce different AppHashes, and validators will not sign blocks with incorrect commitments. This makes state disagreement detectable and ensures that consensus implies state agreement across all honest nodes. + +To maintain determinism, applications should avoid common pitfalls: use block time instead of local timestamps (which vary across nodes), use integer math instead of floating-point arithmetic (which can differ by hardware), use deterministic randomness seeded with block data, and include all necessary data in transactions rather than making external API calls. + +## Communication Layers + +Cosmos SDK blockchains use different communication mechanisms depending on the context. + +- **Node-to-Node Communication**: Nodes communicate with each other using CometBFT's custom peer-to-peer gossip protocol. This handles transaction propagation across the network, block replication to all nodes, and consensus messages like votes and proposals. This communication is entirely within CometBFT and is application-agnostic. + +- **Consensus to Application Communication**: CometBFT communicates with the Cosmos SDK application through ABCI using local function calls. This is in-process communication within the same daemon, not a network protocol. CometBFT invokes the application at key points in the block lifecycle, and the application returns execution results and state commitments. + +- **Client to Node Communication**: Clients (wallets, explorers, and other external applications) interact with nodes through the Cosmos SDK's gRPC API, with HTTP/REST available via gRPC-Gateway. This external API allows clients to query application state and submit transactions for mempool inclusion. This communication is completely separate from consensus—clients never directly interact with CometBFT's consensus protocols. + +To learn more, visit [the CLI, gRPC, and REST API page](/sdk/v0.54/learn/concepts/cli-grpc-rest). + +## Summary + +The architecture of a Cosmos SDK blockchain cleanly separates three concerns. CometBFT determines when blocks happen and which transactions they contain through its consensus mechanism. ABCI defines how and when the application is invoked as part of the block lifecycle. The Cosmos SDK defines what those transactions mean and how they change state deterministically. + +## What's Next + +- The [Transaction Lifecycle](/sdk/v0.54/learn/concepts/lifecycle) page follows a transaction from submission through mempool admission, consensus, and execution. +- The [Application Anatomy](/sdk/v0.54/learn/intro/sdk-app-architecture) page provides a deep dive into building a Cosmos SDK application, exploring modules, keepers, and the composition of app.go in detail. diff --git a/sdk/v0.54/learn/start-here.mdx b/sdk/v0.54/learn/start-here.mdx new file mode 100644 index 000000000..cace46446 --- /dev/null +++ b/sdk/v0.54/learn/start-here.mdx @@ -0,0 +1,73 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/learn/start-here' +title: "Start Here" +description: Pick the path that matches what you want to do with the Cosmos SDK. +icon: play +--- + +These docs are designed to get you building quickly. Choose the path that best matches your goal: + +## What do you want to do? + +- Understand how the SDK works → [Concepts + Tutorial](#learn-+-build-recommended-path) +- New to Cosmos or Blockchain? → [Start here](#new-to-cosmos) +- Build something quickly → [Quickstart](#get-running-fast) +- Run a node → [Operations](#run-a-node) +- Explore modules → [Module Directory](#browse-modules) +- Go deeper on advanced topics → [In-depth Guides](#go-deeper) + + +## Learn + Build (Recommended Path) + +Get a solid mental model of the SDK and build your first chain. + +1. Read the [intro pages](/sdk/v0.54/learn/intro/overview) and [Cosmos Architecture](/sdk/v0.54/learn/intro/sdk-app-architecture) for an overview of how a Cosmos chain is structured +2. Read the [Concepts](/sdk/v0.54/learn/concepts/accounts) section (Fundamentals, Modules, SDK Internals) +3. Follow the [Build a Chain Tutorial](/sdk/v0.54/tutorials/example/00-overview) + +This is the most complete path for developers new to the Cosmos SDK. + +## Get running fast + +Spin up a local chain in minutes. + +1. [Prerequisites](/sdk/v0.54/tutorials/example/01-prerequisites) +2. [Chain Quickstart](/sdk/v0.54/tutorials/example/02-quickstart) + +From there, continue the tutorial to learn more about building modules: + +- [Build a Module from Scratch](/sdk/v0.54/tutorials/example/03-build-a-module): write your first custom module with messages, queries, and state +- [Full Counter Module Walkthrough](/sdk/v0.54/tutorials/example/04-counter-walkthrough): add advanced features like fees, events, and module accounts +- [Run, Test, and Configure](/sdk/v0.54/tutorials/example/05-run-and-test): test your chain and configure it for different environments + +## New to Cosmos? + +Start here if you're new to the ecosystem. + +- [The Cosmos Stack](/sdk/v0.54/learn/intro/cosmos-stack) +- [What is the Cosmos SDK](/sdk/v0.54/learn/intro/overview) + +New to blockchain development? Read +[Blockchain Basics](/sdk/v0.54/learn/intro/blockchain-basics) + +For a technical overview of how a Cosmos chain is structured: +[Cosmos Architecture](/sdk/v0.54/learn/intro/sdk-app-architecture) + +## Run a node + +Set up and operate a Cosmos node (validators, operators). + +- [Run a Node](/sdk/v0.54/tutorials) + +## Browse modules + +Explore production-ready SDK modules and their documentation. + +- [Module Directory](/sdk/v0.54/modules/modules) + +## Go deeper + +Dive into advanced topics like ABCI++, vote extensions, mempool design, upgrades, and observability. + +- [In-depth Guides](/sdk/v0.54/guides/guides) diff --git a/sdk/v0.54/modules/auth/auth.mdx b/sdk/v0.54/modules/auth/auth.mdx new file mode 100644 index 000000000..d94ac7c38 --- /dev/null +++ b/sdk/v0.54/modules/auth/auth.mdx @@ -0,0 +1,740 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/modules/auth/auth' +title: 'x/auth' +description: This document specifies the auth module of the Cosmos SDK. +--- + +## Abstract + +This document specifies the auth module of the Cosmos SDK. + +The auth module is responsible for specifying the base transaction and account types +for an application, since the SDK itself is agnostic to these particulars. It contains +the middlewares, where all basic transaction validity checks (signatures, nonces, auxiliary fields) +are performed, and exposes the account keeper, which allows other modules to read, write, and modify accounts. + +This module is used in the Cosmos Hub. + +## Contents + +* [Concepts](#concepts) + * [Gas & Fees](#gas-&-fees) +* [State](#state) + * [Accounts](#accounts) +* [AnteHandlers](#antehandlers) +* [Keepers](#keepers) + * [Account Keeper](#account-keeper) +* [Parameters](#parameters) +* [Client](#client) + * [CLI](#cli) + * [gRPC](#grpc) + * [REST](#rest) + +## Concepts + +**Note:** The auth module is different from the [authz module](/sdk/v0.54/modules/authz/README). + +The differences are: + +* `auth` - authentication of accounts and transactions for Cosmos SDK applications and is responsible for specifying the base transaction and account types. +* `authz` - authorization for accounts to perform actions on behalf of other accounts and enables a granter to grant authorizations to a grantee that allows the grantee to execute messages on behalf of the granter. + +### Gas & Fees + +Fees serve two purposes for an operator of the network. + +Fees limit the growth of the state stored by every full node and allow for +general purpose censorship of transactions of little economic value. Fees +are best suited as an anti-spam mechanism where validators are disinterested in +the use of the network and identities of users. + +Fees are determined by the gas limits and gas prices transactions provide, where +`fees = ceil(gasLimit * gasPrices)`. Txs incur gas costs for all state reads/writes, +signature verification, as well as costs proportional to the tx size. Operators +should set minimum gas prices when starting their nodes. They must set the unit +costs of gas in each token denomination they wish to support: + +`simd start ... --minimum-gas-prices=0.00001stake;0.05photinos` + +When adding transactions to mempool or gossipping transactions, validators check +if the transaction's gas prices, which are determined by the provided fees, meet +any of the validator's minimum gas prices. In other words, a transaction must +provide a fee of at least one denomination that matches a validator's minimum +gas price. + +CometBFT does not currently provide fee based mempool prioritization, and fee +based mempool filtering is local to node and not part of consensus. But with +minimum gas prices set, such a mechanism could be implemented by node operators. + +Because the market value for tokens will fluctuate, validators are expected to +dynamically adjust their minimum gas prices to a level that would encourage the +use of the network. + +## State + +### Accounts + +Accounts contain authentication information for a uniquely identified external user of an SDK blockchain, +including public key, address, and account number / sequence number for replay protection. For efficiency, +since account balances must also be fetched to pay fees, account structs also store the balance of a user +as `sdk.Coins`. + +Accounts are exposed externally as an interface, and stored internally as +either a base account or vesting account. Module clients wishing to add more +account types may do so. + +* `0x01 | Address -> ProtocolBuffer(account)` + +#### Account Interface + +The account interface exposes methods to read and write standard account information. +Note that all of these methods operate on an account struct conforming to the +interface - in order to write the account to the store, the account keeper will +need to be used. + +```go expandable +// AccountI is an interface used to store coins at a given address within state. +// It presumes a notion of sequence numbers for replay protection, +// a notion of account numbers for replay protection for previously pruned accounts, +// and a pubkey for authentication purposes. +// +// Many complex conditions can be used in the concrete struct which implements AccountI. +type AccountI interface { + proto.Message + + GetAddress() + +sdk.AccAddress + SetAddress(sdk.AccAddress) + +error // errors if already set. + + GetPubKey() + +crypto.PubKey // can return nil. + SetPubKey(crypto.PubKey) + +error + + GetAccountNumber() + +uint64 + SetAccountNumber(uint64) + +error + + GetSequence() + +uint64 + SetSequence(uint64) + +error + + // Ensure that account implements stringer + String() + +string +} +``` + +##### Base Account + +A base account is the simplest and most common account type, which just stores all requisite +fields directly in a struct. + +```protobuf +// BaseAccount defines a base account type. It contains all the necessary fields +// for basic account functionality. Any custom account type should extend this +// type for additional functionality (e.g. vesting). +message BaseAccount { + string address = 1; + google.protobuf.Any pub_key = 2; + uint64 account_number = 3; + uint64 sequence = 4; +} +``` + +### Vesting Account + +See [Vesting](/sdk/v0.54/modules/auth/auth). + +## AnteHandlers + +The `x/auth` module presently has no transaction handlers of its own, but does expose the special `AnteHandler`, used for performing basic validity checks on a transaction, such that it could be thrown out of the mempool. +The `AnteHandler` can be seen as a set of decorators that check transactions within the current context, per [ADR 010](https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-010-modular-antehandler.md). + +Note that the `AnteHandler` is called on both `CheckTx` and `DeliverTx`, as CometBFT proposers presently have the ability to include in their proposed block transactions which fail `CheckTx`. + +### Decorators + +The auth module provides `AnteDecorator`s that are recursively chained together into a single `AnteHandler` in the following order: + +* `SetUpContextDecorator`: Sets the `GasMeter` in the `Context` and wraps the next `AnteHandler` with a defer clause to recover from any downstream `OutOfGas` panics in the `AnteHandler` chain to return an error with information on gas provided and gas used. + +* `RejectExtensionOptionsDecorator`: Rejects all extension options which can optionally be included in protobuf transactions. + +* `MempoolFeeDecorator`: Checks if the `tx` fee is above local mempool `minFee` parameter during `CheckTx`. + +* `ValidateBasicDecorator`: Calls `tx.ValidateBasic` and returns any non-nil error. + +* `TxTimeoutHeightDecorator`: Check for a `tx` height timeout. + +* `ValidateMemoDecorator`: Validates `tx` memo with application parameters and returns any non-nil error. + +* `ConsumeGasTxSizeDecorator`: Consumes gas proportional to the `tx` size based on application parameters. + +* `DeductFeeDecorator`: Deducts the `FeeAmount` from first signer of the `tx`. If the `x/feegrant` module is enabled and a fee granter is set, it deducts fees from the fee granter account. + +* `SetPubKeyDecorator`: Sets the pubkey from a `tx`'s signers that does not already have its corresponding pubkey saved in the state machine and in the current context. + +* `ValidateSigCountDecorator`: Validates the number of signatures in `tx` based on app-parameters. + +* `SigGasConsumeDecorator`: Consumes parameter-defined amount of gas for each signature. This requires pubkeys to be set in context for all signers as part of `SetPubKeyDecorator`. + +* `SigVerificationDecorator`: Verifies all signatures are valid. This requires pubkeys to be set in context for all signers as part of `SetPubKeyDecorator`. + +* `IncrementSequenceDecorator`: Increments the account sequence for each signer to prevent replay attacks. + +## Keepers + +The auth module only exposes one keeper, the account keeper, which can be used to read and write accounts. + +### Account Keeper + +Presently only one fully-permissioned account keeper is exposed, which has the ability to both read and write +all fields of all accounts, and to iterate over all stored accounts. + +```go expandable +// AccountKeeperI is the interface contract that x/auth's keeper implements. +type AccountKeeperI interface { + // Return a new account with the next account number and the specified address. Does not save the new account to the store. + NewAccountWithAddress(sdk.Context, sdk.AccAddress) + +types.AccountI + + // Return a new account with the next account number. Does not save the new account to the store. + NewAccount(sdk.Context, types.AccountI) + +types.AccountI + + // Check if an account exists in the store. + HasAccount(sdk.Context, sdk.AccAddress) + +bool + + // Retrieve an account from the store. + GetAccount(sdk.Context, sdk.AccAddress) + +types.AccountI + + // Set an account in the store. + SetAccount(sdk.Context, types.AccountI) + + // Remove an account from the store. + RemoveAccount(sdk.Context, types.AccountI) + + // Iterate over all accounts, calling the provided function. Stop iteration when it returns true. + IterateAccounts(sdk.Context, func(types.AccountI) + +bool) + + // Fetch the public key of an account at a specified address + GetPubKey(sdk.Context, sdk.AccAddress) (crypto.PubKey, error) + + // Fetch the sequence of an account at a specified address. + GetSequence(sdk.Context, sdk.AccAddress) (uint64, error) + + // Fetch the next account number, and increment the internal counter. + NextAccountNumber(sdk.Context) + +uint64 +} +``` + +## Parameters + +The auth module contains the following parameters: + +| Key | Type | Example | +| ---------------------- | ------ | ------- | +| MaxMemoCharacters | uint64 | 256 | +| TxSigLimit | uint64 | 7 | +| TxSizeCostPerByte | uint64 | 10 | +| SigVerifyCostED25519 | uint64 | 590 | +| SigVerifyCostSecp256k1 | uint64 | 1000 | + +## Client + +### CLI + +A user can query and interact with the `auth` module using the CLI. + +### Query + +The `query` commands allow users to query `auth` state. + +```bash +simd query auth --help +``` + +#### account + +The `account` command allow users to query for an account by it's address. + +```bash +simd query auth account [address] [flags] +``` + +Example: + +```bash +simd query auth account cosmos1... +``` + +Example Output: + +```bash +'@type': /cosmos.auth.v1beta1.BaseAccount +account_number: "0" +address: cosmos1zwg6tpl8aw4rawv8sgag9086lpw5hv33u5ctr2 +pub_key: + '@type': /cosmos.crypto.secp256k1.PubKey + key: ApDrE38zZdd7wLmFS9YmqO684y5DG6fjZ4rVeihF/AQD +sequence: "1" +``` + +#### accounts + +The `accounts` command allow users to query all the available accounts. + +```bash +simd query auth accounts [flags] +``` + +Example: + +```bash +simd query auth accounts +``` + +Example Output: + +```bash expandable +accounts: +- '@type': /cosmos.auth.v1beta1.BaseAccount + account_number: "0" + address: cosmos1zwg6tpl8aw4rawv8sgag9086lpw5hv33u5ctr2 + pub_key: + '@type': /cosmos.crypto.secp256k1.PubKey + key: ApDrE38zZdd7wLmFS9YmqO684y5DG6fjZ4rVeihF/AQD + sequence: "1" +- '@type': /cosmos.auth.v1beta1.ModuleAccount + base_account: + account_number: "8" + address: cosmos1yl6hdjhmkf37639730gffanpzndzdpmhwlkfhr + pub_key: null + sequence: "0" + name: transfer + permissions: + - minter + - burner +- '@type': /cosmos.auth.v1beta1.ModuleAccount + base_account: + account_number: "4" + address: cosmos1fl48vsnmsdzcv85q5d2q4z5ajdha8yu34mf0eh + pub_key: null + sequence: "0" + name: bonded_tokens_pool + permissions: + - burner + - staking +- '@type': /cosmos.auth.v1beta1.ModuleAccount + base_account: + account_number: "5" + address: cosmos1tygms3xhhs3yv487phx3dw4a95jn7t7lpm470r + pub_key: null + sequence: "0" + name: not_bonded_tokens_pool + permissions: + - burner + - staking +- '@type': /cosmos.auth.v1beta1.ModuleAccount + base_account: + account_number: "6" + address: cosmos10d07y265gmmuvt4z0w9aw880jnsr700j6zn9kn + pub_key: null + sequence: "0" + name: gov + permissions: + - burner +- '@type': /cosmos.auth.v1beta1.ModuleAccount + base_account: + account_number: "3" + address: cosmos1jv65s3grqf6v6jl3dp4t6c9t9rk99cd88lyufl + pub_key: null + sequence: "0" + name: distribution + permissions: [] +- '@type': /cosmos.auth.v1beta1.BaseAccount + account_number: "1" + address: cosmos147k3r7v2tvwqhcmaxcfql7j8rmkrlsemxshd3j + pub_key: null + sequence: "0" +- '@type': /cosmos.auth.v1beta1.ModuleAccount + base_account: + account_number: "7" + address: cosmos1m3h30wlvsf8llruxtpukdvsy0km2kum8g38c8q + pub_key: null + sequence: "0" + name: mint + permissions: + - minter +- '@type': /cosmos.auth.v1beta1.ModuleAccount + base_account: + account_number: "2" + address: cosmos17xpfvakm2amg962yls6f84z3kell8c5lserqta + pub_key: null + sequence: "0" + name: fee_collector + permissions: [] +pagination: + next_key: null + total: "0" +``` + +#### params + +The `params` command allow users to query the current auth parameters. + +```bash +simd query auth params [flags] +``` + +Example: + +```bash +simd query auth params +``` + +Example Output: + +```bash +max_memo_characters: "256" +sig_verify_cost_ed25519: "590" +sig_verify_cost_secp256k1: "1000" +tx_sig_limit: "7" +tx_size_cost_per_byte: "10" +``` + +### Transactions + +The `auth` module supports transactions commands to help you with signing and more. Compared to other modules you can access directly the `auth` module transactions commands using the only `tx` command. + +Use directly the `--help` flag to get more information about the `tx` command. + +```bash +simd tx --help +``` + +#### `sign` + +The `sign` command allows users to sign transactions that was generated offline. + +```bash +simd tx sign tx.json --from $ALICE > tx.signed.json +``` + +The result is a signed transaction that can be broadcasted to the network thanks to the broadcast command. + +More information about the `sign` command can be found running `simd tx sign --help`. + +#### `sign-batch` + +The `sign-batch` command allows users to sign multiples offline generated transactions. +The transactions can be in one file, with one tx per line, or in multiple files. + +```bash +simd tx sign txs.json --from $ALICE > tx.signed.json +``` + +or + +```bash +simd tx sign tx1.json tx2.json tx3.json --from $ALICE > tx.signed.json +``` + +The result is multiples signed transactions. For combining the signed transactions into one transactions, use the `--append` flag. + +More information about the `sign-batch` command can be found running `simd tx sign-batch --help`. + +#### `multi-sign` + +The `multi-sign` command allows users to sign transactions that was generated offline by a multisig account. + +```bash +simd tx multisign transaction.json k1k2k3 k1sig.json k2sig.json k3sig.json +``` + +Where `k1k2k3` is the multisig account address, `k1sig.json` is the signature of the first signer, `k2sig.json` is the signature of the second signer, and `k3sig.json` is the signature of the third signer. + +##### Nested multisig transactions + +To allow transactions to be signed by nested multisigs, meaning that a participant of a multisig account can be another multisig account, the `--skip-signature-verification` flag must be used. + +```bash +# First aggregate signatures of the multisig participant +simd tx multi-sign transaction.json ms1 ms1p1sig.json ms1p2sig.json --signature-only --skip-signature-verification > ms1sig.json + +# Then use the aggregated signatures and the other signatures to sign the final transaction +simd tx multi-sign transaction.json k1ms1 k1sig.json ms1sig.json --skip-signature-verification +``` + +Where `ms1` is the nested multisig account address, `ms1p1sig.json` is the signature of the first participant of the nested multisig account, `ms1p2sig.json` is the signature of the second participant of the nested multisig account, and `ms1sig.json` is the aggregated signature of the nested multisig account. + +`k1ms1` is a multisig account comprised of an individual signer and another nested multisig account (`ms1`). `k1sig.json` is the signature of the first signer of the individual member. + +More information about the `multi-sign` command can be found running `simd tx multi-sign --help`. + +#### `multisign-batch` + +The `multisign-batch` works the same way as `sign-batch`, but for multisig accounts. +With the difference that the `multisign-batch` command requires all transactions to be in one file, and the `--append` flag does not exist. + +More information about the `multisign-batch` command can be found running `simd tx multisign-batch --help`. + +#### `validate-signatures` + +The `validate-signatures` command allows users to validate the signatures of a signed transaction. + +```bash +$ simd tx validate-signatures tx.signed.json +Signers: + 0: cosmos1l6vsqhh7rnwsyr2kyz3jjg3qduaz8gwgyl8275 + +Signatures: + 0: cosmos1l6vsqhh7rnwsyr2kyz3jjg3qduaz8gwgyl8275 [OK] +``` + +More information about the `validate-signatures` command can be found running `simd tx validate-signatures --help`. + +#### `broadcast` + +The `broadcast` command allows users to broadcast a signed transaction to the network. + +```bash +simd tx broadcast tx.signed.json +``` + +More information about the `broadcast` command can be found running `simd tx broadcast --help`. + +### gRPC + +A user can query the `auth` module using gRPC endpoints. + +#### Account + +The `account` endpoint allow users to query for an account by it's address. + +```bash +cosmos.auth.v1beta1.Query/Account +``` + +Example: + +```bash +grpcurl -plaintext \ + -d '{"address":"cosmos1.."}' \ + localhost:9090 \ + cosmos.auth.v1beta1.Query/Account +``` + +Example Output: + +```bash expandable +{ + "account":{ + "@type":"/cosmos.auth.v1beta1.BaseAccount", + "address":"cosmos1zwg6tpl8aw4rawv8sgag9086lpw5hv33u5ctr2", + "pubKey":{ + "@type":"/cosmos.crypto.secp256k1.PubKey", + "key":"ApDrE38zZdd7wLmFS9YmqO684y5DG6fjZ4rVeihF/AQD" + }, + "sequence":"1" + } +} +``` + +#### Accounts + +The `accounts` endpoint allow users to query all the available accounts. + +```bash +cosmos.auth.v1beta1.Query/Accounts +``` + +Example: + +```bash +grpcurl -plaintext \ + localhost:9090 \ + cosmos.auth.v1beta1.Query/Accounts +``` + +Example Output: + +```bash expandable +{ + "accounts":[ + { + "@type":"/cosmos.auth.v1beta1.BaseAccount", + "address":"cosmos1zwg6tpl8aw4rawv8sgag9086lpw5hv33u5ctr2", + "pubKey":{ + "@type":"/cosmos.crypto.secp256k1.PubKey", + "key":"ApDrE38zZdd7wLmFS9YmqO684y5DG6fjZ4rVeihF/AQD" + }, + "sequence":"1" + }, + { + "@type":"/cosmos.auth.v1beta1.ModuleAccount", + "baseAccount":{ + "address":"cosmos1yl6hdjhmkf37639730gffanpzndzdpmhwlkfhr", + "accountNumber":"8" + }, + "name":"transfer", + "permissions":[ + "minter", + "burner" + ] + }, + { + "@type":"/cosmos.auth.v1beta1.ModuleAccount", + "baseAccount":{ + "address":"cosmos1fl48vsnmsdzcv85q5d2q4z5ajdha8yu34mf0eh", + "accountNumber":"4" + }, + "name":"bonded_tokens_pool", + "permissions":[ + "burner", + "staking" + ] + }, + { + "@type":"/cosmos.auth.v1beta1.ModuleAccount", + "baseAccount":{ + "address":"cosmos1tygms3xhhs3yv487phx3dw4a95jn7t7lpm470r", + "accountNumber":"5" + }, + "name":"not_bonded_tokens_pool", + "permissions":[ + "burner", + "staking" + ] + }, + { + "@type":"/cosmos.auth.v1beta1.ModuleAccount", + "baseAccount":{ + "address":"cosmos10d07y265gmmuvt4z0w9aw880jnsr700j6zn9kn", + "accountNumber":"6" + }, + "name":"gov", + "permissions":[ + "burner" + ] + }, + { + "@type":"/cosmos.auth.v1beta1.ModuleAccount", + "baseAccount":{ + "address":"cosmos1jv65s3grqf6v6jl3dp4t6c9t9rk99cd88lyufl", + "accountNumber":"3" + }, + "name":"distribution" + }, + { + "@type":"/cosmos.auth.v1beta1.BaseAccount", + "accountNumber":"1", + "address":"cosmos147k3r7v2tvwqhcmaxcfql7j8rmkrlsemxshd3j" + }, + { + "@type":"/cosmos.auth.v1beta1.ModuleAccount", + "baseAccount":{ + "address":"cosmos1m3h30wlvsf8llruxtpukdvsy0km2kum8g38c8q", + "accountNumber":"7" + }, + "name":"mint", + "permissions":[ + "minter" + ] + }, + { + "@type":"/cosmos.auth.v1beta1.ModuleAccount", + "baseAccount":{ + "address":"cosmos17xpfvakm2amg962yls6f84z3kell8c5lserqta", + "accountNumber":"2" + }, + "name":"fee_collector" + } + ], + "pagination":{ + "total":"9" + } +} +``` + +#### Params + +The `params` endpoint allow users to query the current auth parameters. + +```bash +cosmos.auth.v1beta1.Query/Params +``` + +Example: + +```bash +grpcurl -plaintext \ + localhost:9090 \ + cosmos.auth.v1beta1.Query/Params +``` + +Example Output: + +```bash +{ + "params": { + "maxMemoCharacters": "256", + "txSigLimit": "7", + "txSizeCostPerByte": "10", + "sigVerifyCostEd25519": "590", + "sigVerifyCostSecp256k1": "1000" + } +} +``` + +### REST + +A user can query the `auth` module using REST endpoints. + +#### Account + +The `account` endpoint allow users to query for an account by it's address. + +```bash +/cosmos/auth/v1beta1/account?address={address} +``` + +#### Accounts + +The `accounts` endpoint allow users to query all the available accounts. + +```bash +/cosmos/auth/v1beta1/accounts +``` + +#### Params + +The `params` endpoint allow users to query the current auth parameters. + +```bash +/cosmos/auth/v1beta1/params +``` diff --git a/sdk/v0.54/modules/auth/tx.mdx b/sdk/v0.54/modules/auth/tx.mdx new file mode 100644 index 000000000..afb5522aa --- /dev/null +++ b/sdk/v0.54/modules/auth/tx.mdx @@ -0,0 +1,274 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/modules/auth/tx' +title: 'x/auth/tx' +--- + + +**Prerequisite Readings** + +* [Transactions](/sdk/v0.54/learn/concepts/lifecycle#transaction-generation) +* [Encoding](/sdk/v0.54/learn/concepts/encoding#transaction-encoding) + + + +## Abstract + +This document specifies the `x/auth/tx` package of the Cosmos SDK. + +This package represents the Cosmos SDK implementation of the `client.TxConfig`, `client.TxBuilder`, `client.TxEncoder` and `client.TxDecoder` interfaces. + +## Contents + +* [Transactions](#transactions) + * [`TxConfig`](#txconfig) + * [`TxBuilder`](#txbuilder) + * [`TxEncoder`/ `TxDecoder`](#txencoder-txdecoder) +* [Client](#client) + * [CLI](#cli) + * [gRPC](#grpc) + +## Transactions + +### `TxConfig` + +`client.TxConfig` defines an interface a client can utilize to generate an application-defined concrete transaction type. +The interface defines a set of methods for creating a `client.TxBuilder`. + +```go +// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/client/tx_config.go#L25-L31 +``` + +The default implementation of `client.TxConfig` is instantiated by `NewTxConfig` in `x/auth/tx` module. + +```go +// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/x/auth/tx/config.go#L22-L28 +``` + +### `TxBuilder` + +```go +// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/client/tx_config.go#L33-L50 +``` + +The [`client.TxBuilder`](/sdk/v0.54/learn/concepts/lifecycle#transaction-generation) interface is as well implemented by `x/auth/tx`. +A `client.TxBuilder` can be accessed with `TxConfig.NewTxBuilder()`. + +### `TxEncoder`/ `TxDecoder` + +More information about `TxEncoder` and `TxDecoder` can be found [here](/sdk/v0.54/learn/concepts/encoding#transaction-encoding). + +## Client + +### CLI + +#### Query + +The `x/auth/tx` module provides a CLI command to query any transaction, given its hash, transaction sequence or signature. + +Without any argument, the command will query the transaction using the transaction hash. + +```shell +simd query tx DFE87B78A630C0EFDF76C80CD24C997E252792E0317502AE1A02B9809F0D8685 +``` + +When querying a transaction from an account given its sequence, use the `--type=acc_seq` flag: + +```shell +simd query tx --type=acc_seq cosmos1u69uyr6v9qwe6zaaeaqly2h6wnedac0xpxq325/1 +``` + +When querying a transaction given its signature, use the `--type=signature` flag: + +```shell +simd query tx --type=signature Ofjvgrqi8twZfqVDmYIhqwRLQjZZ40XbxEamk/veH3gQpRF0hL2PH4ejRaDzAX+2WChnaWNQJQ41ekToIi5Wqw== +``` + +When querying a transaction given its events, use the `--type=events` flag: + +```shell +simd query txs --events 'message.sender=cosmos...' --page 1 --limit 30 +``` + +The `x/auth/block` module provides a CLI command to query any block, given its hash, height, or events. + +When querying a block by its hash, use the `--type=hash` flag: + +```shell +simd query block --type=hash DFE87B78A630C0EFDF76C80CD24C997E252792E0317502AE1A02B9809F0D8685 +``` + +When querying a block by its height, use the `--type=height` flag: + +```shell +simd query block --type=height 1357 +``` + +When querying a block by its events, use the `--query` flag: + +```shell +simd query blocks --query 'message.sender=cosmos...' --page 1 --limit 30 +``` + +#### Transactions + +The `x/auth/tx` module provides a convenient CLI command for decoding and encoding transactions. + +#### `encode` + +The `encode` command encodes a transaction created with the `--generate-only` flag or signed with the sign command. +The transaction is serialized to Protobuf and returned as base64. + +```bash +$ simd tx encode tx.json +Co8BCowBChwvY29zbW9zLmJhbmsudjFiZXRhMS5Nc2dTZW5kEmwKLWNvc21vczFsNnZzcWhoN3Jud3N5cjJreXozampnM3FkdWF6OGd3Z3lsODI3NRItY29zbW9zMTU4c2FsZHlnOHBteHU3Znd2dDBkNng3amVzd3A0Z3d5a2xrNnkzGgwKBXN0YWtlEgMxMDASBhIEEMCaDA== +$ simd tx encode tx.signed.json +``` + +More information about the `encode` command can be found running `simd tx encode --help`. + +#### `decode` + +The `decode` command decodes a transaction encoded with the `encode` command. + +```bash +simd tx decode Co8BCowBChwvY29zbW9zLmJhbmsudjFiZXRhMS5Nc2dTZW5kEmwKLWNvc21vczFsNnZzcWhoN3Jud3N5cjJreXozampnM3FkdWF6OGd3Z3lsODI3NRItY29zbW9zMTU4c2FsZHlnOHBteHU3Znd2dDBkNng3amVzd3A0Z3d5a2xrNnkzGgwKBXN0YWtlEgMxMDASBhIEEMCaDA== +``` + +More information about the `decode` command can be found running `simd tx decode --help`. + +### gRPC + +A user can query the `x/auth/tx` module using gRPC endpoints. + +#### `TxDecode` + +The `TxDecode` endpoint allows to decode a transaction. + +```shell +cosmos.tx.v1beta1.Service/TxDecode +``` + +Example: + +```shell +grpcurl -plaintext \ + -d '{"tx_bytes":"Co8BCowBChwvY29zbW9zLmJhbmsudjFiZXRhMS5Nc2dTZW5kEmwKLWNvc21vczFsNnZzcWhoN3Jud3N5cjJreXozampnM3FkdWF6OGd3Z3lsODI3NRItY29zbW9zMTU4c2FsZHlnOHBteHU3Znd2dDBkNng3amVzd3A0Z3d5a2xrNnkzGgwKBXN0YWtlEgMxMDASBhIEEMCaDA=="}' \ + localhost:9090 \ + cosmos.tx.v1beta1.Service/TxDecode +``` + +Example Output: + +```json expandable +{ + "tx": { + "body": { + "messages": [ + { + "@type": "/cosmos.bank.v1beta1.MsgSend", + "amount": [ + { + "denom": "stake", + "amount": "100" + } + ], + "fromAddress": "cosmos1l6vsqhh7rnwsyr2kyz3jjg3qduaz8gwgyl8275", + "toAddress": "cosmos158saldyg8pmxu7fwvt0d6x7jeswp4gwyklk6y3" + } + ] + }, + "authInfo": { + "fee": { + "gasLimit": "200000" + } + } + } +} +``` + +#### `TxEncode` + +The `TxEncode` endpoint allows to encode a transaction. + +```shell +cosmos.tx.v1beta1.Service/TxEncode +``` + +Example: + +```shell expandable +grpcurl -plaintext \ + -d '{"tx": { + "body": { + "messages": [ + {"@type":"/cosmos.bank.v1beta1.MsgSend","amount":[{"denom":"stake","amount":"100"}],"fromAddress":"cosmos1l6vsqhh7rnwsyr2kyz3jjg3qduaz8gwgyl8275","toAddress":"cosmos158saldyg8pmxu7fwvt0d6x7jeswp4gwyklk6y3"} + ] + }, + "authInfo": { + "fee": { + "gasLimit": "200000" + } + } + }}' \ + localhost:9090 \ + cosmos.tx.v1beta1.Service/TxEncode +``` + +Example Output: + +```json +{ + "txBytes": "Co8BCowBChwvY29zbW9zLmJhbmsudjFiZXRhMS5Nc2dTZW5kEmwKLWNvc21vczFsNnZzcWhoN3Jud3N5cjJreXozampnM3FkdWF6OGd3Z3lsODI3NRItY29zbW9zMTU4c2FsZHlnOHBteHU3Znd2dDBkNng3amVzd3A0Z3d5a2xrNnkzGgwKBXN0YWtlEgMxMDASBhIEEMCaDA==" +} +``` + +#### `TxDecodeAmino` + +The `TxDecode` endpoint allows to decode an amino transaction. + +```shell +cosmos.tx.v1beta1.Service/TxDecodeAmino +``` + +Example: + +```shell +grpcurl -plaintext \ + -d '{"amino_binary": "KCgWqQpvqKNhmgotY29zbW9zMXRzeno3cDJ6Z2Q3dnZrYWh5ZnJlNHduNXh5dTgwcnB0ZzZ2OWg1Ei1jb3Ntb3MxdHN6ejdwMnpnZDd2dmthaHlmcmU0d241eHl1ODBycHRnNnY5aDUaCwoFc3Rha2USAjEwEhEKCwoFc3Rha2USAjEwEMCaDCIGZm9vYmFy"}' \ + localhost:9090 \ + cosmos.tx.v1beta1.Service/TxDecodeAmino +``` + +Example Output: + +```json +{ + "aminoJson": "{\"type\":\"cosmos-sdk/StdTx\",\"value\":{\"msg\":[{\"type\":\"cosmos-sdk/MsgSend\",\"value\":{\"from_address\":\"cosmos1tszz7p2zgd7vvkahyfre4wn5xyu80rptg6v9h5\",\"to_address\":\"cosmos1tszz7p2zgd7vvkahyfre4wn5xyu80rptg6v9h5\",\"amount\":[{\"denom\":\"stake\",\"amount\":\"10\"}]}}],\"fee\":{\"amount\":[{\"denom\":\"stake\",\"amount\":\"10\"}],\"gas\":\"200000\"},\"signatures\":null,\"memo\":\"foobar\",\"timeout_height\":\"0\"}}" +} +``` + +#### `TxEncodeAmino` + +The `TxEncodeAmino` endpoint allows to encode an amino transaction. + +```shell +cosmos.tx.v1beta1.Service/TxEncodeAmino +``` + +Example: + +```shell +grpcurl -plaintext \ + -d '{"amino_json":"{\"type\":\"cosmos-sdk/StdTx\",\"value\":{\"msg\":[{\"type\":\"cosmos-sdk/MsgSend\",\"value\":{\"from_address\":\"cosmos1tszz7p2zgd7vvkahyfre4wn5xyu80rptg6v9h5\",\"to_address\":\"cosmos1tszz7p2zgd7vvkahyfre4wn5xyu80rptg6v9h5\",\"amount\":[{\"denom\":\"stake\",\"amount\":\"10\"}]}}],\"fee\":{\"amount\":[{\"denom\":\"stake\",\"amount\":\"10\"}],\"gas\":\"200000\"},\"signatures\":null,\"memo\":\"foobar\",\"timeout_height\":\"0\"}}"}' \ + localhost:9090 \ + cosmos.tx.v1beta1.Service/TxEncodeAmino +``` + +Example Output: + +```json +{ + "amino_binary": "KCgWqQpvqKNhmgotY29zbW9zMXRzeno3cDJ6Z2Q3dnZrYWh5ZnJlNHduNXh5dTgwcnB0ZzZ2OWg1Ei1jb3Ntb3MxdHN6ejdwMnpnZDd2dmthaHlmcmU0d241eHl1ODBycHRnNnY5aDUaCwoFc3Rha2USAjEwEhEKCwoFc3Rha2USAjEwEMCaDCIGZm9vYmFy" +} +``` diff --git a/sdk/v0.54/modules/auth/vesting.mdx b/sdk/v0.54/modules/auth/vesting.mdx new file mode 100644 index 000000000..e9a5429d0 --- /dev/null +++ b/sdk/v0.54/modules/auth/vesting.mdx @@ -0,0 +1,681 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/modules/auth/vesting' +title: 'x/auth/vesting' +--- + +* [Intro and Requirements](#intro-and-requirements) +* [Note](#note) +* [Vesting Account Types](#vesting-account-types) + * [BaseVestingAccount](#basevestingaccount) + * [ContinuousVestingAccount](#continuousvestingaccount) + * [DelayedVestingAccount](#delayedvestingaccount) + * [Period](#period) + * [PeriodicVestingAccount](#periodicvestingaccount) + * [PermanentLockedAccount](#permanentlockedaccount) +* [Vesting Account Specification](#vesting-account-specification) + * [Determining Vesting & Vested Amounts](#determining-vesting-&-vested-amounts) + * [Periodic Vesting Accounts](#periodic-vesting-accounts) + * [Transferring/Sending](#transferringsending) + * [Delegating](#delegating) + * [Undelegating](#undelegating) +* [Keepers & Handlers](#keepers-&-handlers) +* [Genesis Initialization](#genesis-initialization) +* [Examples](#examples) + * [Simple](#simple) + * [Slashing](#slashing) + * [Periodic Vesting](#periodic-vesting) +* [Glossary](#glossary) + +## Intro and Requirements + +This specification defines the vesting account implementation that is used by the Cosmos Hub. The requirements for this vesting account is that it should be initialized during genesis with a starting balance `X` and a vesting end time `ET`. A vesting account may be initialized with a vesting start time `ST` and a number of vesting periods `P`. If a vesting start time is included, the vesting period does not begin until start time is reached. If vesting periods are included, the vesting occurs over the specified number of periods. + +For all vesting accounts, the owner of the vesting account is able to delegate and undelegate from validators, however they cannot transfer coins to another account until those coins are vested. This specification allows for four different kinds of vesting: + +* Delayed vesting, where all coins are vested once `ET` is reached. +* Continuous vesting, where coins begin to vest at `ST` and vest linearly with respect to time until `ET` is reached +* Periodic vesting, where coins begin to vest at `ST` and vest periodically according to number of periods and the vesting amount per period. The number of periods, length per period, and amount per period are configurable. A periodic vesting account is distinguished from a continuous vesting account in that coins can be released in staggered tranches. For example, a periodic vesting account could be used for vesting arrangements where coins are released quarterly, yearly, or over any other function of tokens over time. +* Permanent locked vesting, where coins are locked forever. Coins in this account can still be used for delegating and for governance votes even while locked. + +## Note + +Vesting accounts can be initialized with some vesting and non-vesting coins. The non-vesting coins would be immediately transferable. DelayedVesting ContinuousVesting, PeriodicVesting and PermanentVesting accounts can be created with normal messages after genesis. Other types of vesting accounts must be created at genesis, or as part of a manual network upgrade. The current specification only allows for *unconditional* vesting (ie. there is no possibility of reaching `ET` and +having coins fail to vest). + +## Vesting Account Types + +```go expandable +// VestingAccount defines an interface that any vesting account type must +// implement. +type VestingAccount interface { + Account + + GetVestedCoins(Time) + +Coins + GetVestingCoins(Time) + +Coins + + // TrackDelegation performs internal vesting accounting necessary when + // delegating from a vesting account. It accepts the current block time, the + // delegation amount and balance of all coins whose denomination exists in + // the account's original vesting balance. + TrackDelegation(Time, Coins, Coins) + + // TrackUndelegation performs internal vesting accounting necessary when a + // vesting account performs an undelegation. + TrackUndelegation(Coins) + +GetStartTime() + +int64 + GetEndTime() + +int64 +} +``` + +### BaseVestingAccount + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/proto/cosmos/vesting/v1beta1/vesting.proto#L11-L35 +``` + +### ContinuousVestingAccount + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/proto/cosmos/vesting/v1beta1/vesting.proto#L37-L46 +``` + +### DelayedVestingAccount + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/proto/cosmos/vesting/v1beta1/vesting.proto#L48-L57 +``` + +### Period + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/proto/cosmos/vesting/v1beta1/vesting.proto#L59-L69 +``` + +```go +// Stores all vesting periods passed as part of a PeriodicVestingAccount +type Periods []Period +``` + +### PeriodicVestingAccount + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/proto/cosmos/vesting/v1beta1/vesting.proto#L71-L81 +``` + +In order to facilitate less ad-hoc type checking and assertions and to support flexibility in account balance usage, the existing `x/bank` `ViewKeeper` interface is updated to contain the following: + +```go +type ViewKeeper interface { + // ... + + // Calculates the total locked account balance. + LockedCoins(ctx sdk.Context, addr sdk.AccAddress) + +sdk.Coins + + // Calculates the total spendable balance that can be sent to other accounts. + SpendableCoins(ctx sdk.Context, addr sdk.AccAddress) + +sdk.Coins +} +``` + +### PermanentLockedAccount + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/proto/cosmos/vesting/v1beta1/vesting.proto#L83-L94 +``` + +## Vesting Account Specification + +Given a vesting account, we define the following in the proceeding operations: + +* `OV`: The original vesting coin amount. It is a constant value. +* `V`: The number of `OV` coins that are still *vesting*. It is derived by + `OV`, `StartTime` and `EndTime`. This value is computed on demand and not on a per-block basis. +* `V'`: The number of `OV` coins that are *vested* (unlocked). This value is computed on demand and not a per-block basis. +* `DV`: The number of delegated *vesting* coins. It is a variable value. It is stored and modified directly in the vesting account. +* `DF`: The number of delegated *vested* (unlocked) coins. It is a variable value. It is stored and modified directly in the vesting account. +* `BC`: The number of `OV` coins less any coins that are transferred + (which can be negative or delegated). It is considered to be balance of the embedded base account. It is stored and modified directly in the vesting account. + +### Determining Vesting & Vested Amounts + +It is important to note that these values are computed on demand and not on a mandatory per-block basis (e.g. `BeginBlocker` or `EndBlocker`). + +#### Continuously Vesting Accounts + +To determine the amount of coins that are vested for a given block time `T`, the +following is performed: + +1. Compute `X := T - StartTime` +2. Compute `Y := EndTime - StartTime` +3. Compute `V' := OV * (X / Y)` +4. Compute `V := OV - V'` + +Thus, the total amount of *vested* coins is `V'` and the remaining amount, `V`, +is *vesting*. + +```go expandable +func (cva ContinuousVestingAccount) + +GetVestedCoins(t Time) + +Coins { + if t <= cva.StartTime { + // We must handle the case where the start time for a vesting account has + // been set into the future or when the start of the chain is not exactly + // known. + return ZeroCoins +} + +else if t >= cva.EndTime { + return cva.OriginalVesting +} + x := t - cva.StartTime + y := cva.EndTime - cva.StartTime + + return cva.OriginalVesting * (x / y) +} + +func (cva ContinuousVestingAccount) + +GetVestingCoins(t Time) + +Coins { + return cva.OriginalVesting - cva.GetVestedCoins(t) +} +``` + +### Periodic Vesting Accounts + +Periodic vesting accounts require calculating the coins released during each period for a given block time `T`. Note that multiple periods could have passed when calling `GetVestedCoins`, so we must iterate over each period until the end of that period is after `T`. + +1. Set `CT := StartTime` +2. Set `V' := 0` + +For each Period P: + +1. Compute `X := T - CT` +2. IF `X >= P.Length` + 1. Compute `V' += P.Amount` + 2. Compute `CT += P.Length` + 3. ELSE break +3. Compute `V := OV - V'` + +```go expandable +func (pva PeriodicVestingAccount) + +GetVestedCoins(t Time) + +Coins { + if t < pva.StartTime { + return ZeroCoins +} + ct := pva.StartTime // The start of the vesting schedule + vested := 0 + periods = pva.GetPeriods() + for _, period := range periods { + if t - ct < period.Length { + break +} + +vested += period.Amount + ct += period.Length // increment ct to the start of the next vesting period +} + +return vested +} + +func (pva PeriodicVestingAccount) + +GetVestingCoins(t Time) + +Coins { + return pva.OriginalVesting - cva.GetVestedCoins(t) +} +``` + +#### Delayed/Discrete Vesting Accounts + +Delayed vesting accounts are easier to reason about as they only have the full amount vesting up until a certain time, then all the coins become vested (unlocked). This does not include any unlocked coins the account may have initially. + +```go expandable +func (dva DelayedVestingAccount) + +GetVestedCoins(t Time) + +Coins { + if t >= dva.EndTime { + return dva.OriginalVesting +} + +return ZeroCoins +} + +func (dva DelayedVestingAccount) + +GetVestingCoins(t Time) + +Coins { + return dva.OriginalVesting - dva.GetVestedCoins(t) +} +``` + +### Transferring/Sending + +At any given time, a vesting account may transfer: `min((BC + DV) - V, BC)`. + +In other words, a vesting account may transfer the minimum of the base account balance and the base account balance plus the number of currently delegated vesting coins less the number of coins vested so far. + +However, given that account balances are tracked via the `x/bank` module and that we want to avoid loading the entire account balance, we can instead determine the locked balance, which can be defined as `max(V - DV, 0)`, and infer the spendable balance from that. + +```go +func (va VestingAccount) + +LockedCoins(t Time) + +Coins { + return max(va.GetVestingCoins(t) - va.DelegatedVesting, 0) +} +``` + +The `x/bank` `ViewKeeper` can then provide APIs to determine locked and spendable coins for any account: + +```go expandable +func (k Keeper) + +LockedCoins(ctx Context, addr AccAddress) + +Coins { + acc := k.GetAccount(ctx, addr) + if acc != nil { + if acc.IsVesting() { + return acc.LockedCoins(ctx.BlockTime()) +} + +} + + // non-vesting accounts do not have any locked coins + return NewCoins() +} +``` + +#### Keepers/Handlers + +The corresponding `x/bank` keeper should appropriately handle sending coins based on if the account is a vesting account or not. + +```go expandable +func (k Keeper) + +SendCoins(ctx Context, from Account, to Account, amount Coins) { + bc := k.GetBalances(ctx, from) + v := k.LockedCoins(ctx, from) + spendable := bc - v + newCoins := spendable - amount + assert(newCoins >= 0) + +from.SetBalance(newCoins) + +to.AddBalance(amount) + + // save balances... +} +``` + +### Delegating + +For a vesting account attempting to delegate `D` coins, the following is performed: + +1. Verify `BC >= D > 0` +2. Compute `X := min(max(V - DV, 0), D)` (portion of `D` that is vesting) +3. Compute `Y := D - X` (portion of `D` that is free) +4. Set `DV += X` +5. Set `DF += Y` + +```go +func (va VestingAccount) + +TrackDelegation(t Time, balance Coins, amount Coins) { + assert(balance <= amount) + x := min(max(va.GetVestingCoins(t) - va.DelegatedVesting, 0), amount) + y := amount - x + + va.DelegatedVesting += x + va.DelegatedFree += y +} +``` + +**Note** `TrackDelegation` only modifies the `DelegatedVesting` and `DelegatedFree` fields, so upstream callers MUST modify the `Coins` field by subtracting `amount`. + +#### Keepers/Handlers + +```go +func DelegateCoins(t Time, from Account, amount Coins) { + if isVesting(from) { + from.TrackDelegation(t, amount) +} + +else { + from.SetBalance(sc - amount) +} + + // save account... +} +``` + +### Undelegating + +For a vesting account attempting to undelegate `D` coins, the following is performed: + +> NOTE: `DV < D` and `(DV + DF) < D` may be possible due to quirks in the rounding of delegation/undelegation logic. + +1. Verify `D > 0` +2. Compute `X := min(DF, D)` (portion of `D` that should become free, prioritizing free coins) +3. Compute `Y := min(DV, D - X)` (portion of `D` that should remain vesting) +4. Set `DF -= X` +5. Set `DV -= Y` + +```go +func (cva ContinuousVestingAccount) + +TrackUndelegation(amount Coins) { + x := min(cva.DelegatedFree, amount) + y := amount - x + + cva.DelegatedFree -= x + cva.DelegatedVesting -= y +} +``` + +**Note** `TrackUnDelegation` only modifies the `DelegatedVesting` and `DelegatedFree` fields, so upstream callers MUST modify the `Coins` field by adding `amount`. + +**Note**: If a delegation is slashed, the continuous vesting account ends up with an excess `DV` amount, even after all its coins have vested. This is because undelegating free coins are prioritized. + +**Note**: The undelegation (bond refund) amount may exceed the delegated vesting (bond) amount due to the way undelegation truncates the bond refund, which can increase the validator's exchange rate (tokens/shares) slightly if the undelegated tokens are non-integral. + +#### Keepers/Handlers + +```go expandable +func UndelegateCoins(to Account, amount Coins) { + if isVesting(to) { + if to.DelegatedFree + to.DelegatedVesting >= amount { + to.TrackUndelegation(amount) + // save account ... +} + +} + +else { + AddBalance(to, amount) + // save account... +} +} +``` + +## Keepers & Handlers + +The `VestingAccount` implementations reside in `x/auth`. However, any keeper in a module (e.g. staking in `x/staking`) wishing to potentially utilize any vesting coins, must call explicit methods on the `x/bank` keeper (e.g. `DelegateCoins`) opposed to `SendCoins` and `SubtractCoins`. + +In addition, the vesting account should also be able to spend any coins it receives from other users. Thus, the bank module's `MsgSend` handler should error if a vesting account is trying to send an amount that exceeds their unlocked coin amount. + +See the above specification for full implementation details. + +## Genesis Initialization + +To initialize both vesting and non-vesting accounts, the `GenesisAccount` struct includes new fields: `Vesting`, `StartTime`, and `EndTime`. Accounts meant to be of type `BaseAccount` or any non-vesting type have `Vesting = false`. The genesis initialization logic (e.g. `initFromGenesisState`) must parse and return the correct accounts accordingly based off of these fields. + +```go expandable +type GenesisAccount struct { + // ... + + // vesting account fields + OriginalVesting sdk.Coins `json:"original_vesting"` + DelegatedFree sdk.Coins `json:"delegated_free"` + DelegatedVesting sdk.Coins `json:"delegated_vesting"` + StartTime int64 `json:"start_time"` + EndTime int64 `json:"end_time"` +} + +func ToAccount(gacc GenesisAccount) + +Account { + bacc := NewBaseAccount(gacc) + if gacc.OriginalVesting > 0 { + if ga.StartTime != 0 && ga.EndTime != 0 { + // return a continuous vesting account +} + +else if ga.EndTime != 0 { + // return a delayed vesting account +} + +else { + // invalid genesis vesting account provided + panic() +} + +} + +return bacc +} +``` + +## Examples + +### Simple + +Given a continuous vesting account with 10 vesting coins. + +```text +OV = 10 +DF = 0 +DV = 0 +BC = 10 +V = 10 +V' = 0 +``` + +1. Immediately receives 1 coin + + ```text + BC = 11 + ``` + +2. Time passes, 2 coins vest + + ```text + V = 8 + V' = 2 + ``` + +3. Delegates 4 coins to validator A + + ```text + DV = 4 + BC = 7 + ``` + +4. Sends 3 coins + + ```text + BC = 4 + ``` + +5. More time passes, 2 more coins vest + + ```text + V = 6 + V' = 4 + ``` + +6. Sends 2 coins. At this point the account cannot send anymore until further + coins vest or it receives additional coins. It can still however, delegate. + + ```text + BC = 2 + ``` + +### Slashing + +Same initial starting conditions as the simple example. + +1. Time passes, 5 coins vest + + ```text + V = 5 + V' = 5 + ``` + +2. Delegate 5 coins to validator A + + ```text + DV = 5 + BC = 5 + ``` + +3. Delegate 5 coins to validator B + + ```text + DF = 5 + BC = 0 + ``` + +4. Validator A gets slashed by 50%, making the delegation to A now worth 2.5 coins + +5. Undelegate from validator A (2.5 coins) + + ```text + DF = 5 - 2.5 = 2.5 + BC = 0 + 2.5 = 2.5 + ``` + +6. Undelegate from validator B (5 coins). The account at this point can only + send 2.5 coins unless it receives more coins or until more coins vest. + It can still however, delegate. + + ```text + DV = 5 - 2.5 = 2.5 + DF = 2.5 - 2.5 = 0 + BC = 2.5 + 5 = 7.5 + ``` + + Notice how we have an excess amount of `DV`. + +### Periodic Vesting + +A vesting account is created where 100 tokens will be released over 1 year, with +1/4 of tokens vesting each quarter. The vesting schedule would be as follows: + +```yaml +Periods: +- amount: 25stake, length: 7884000 +- amount: 25stake, length: 7884000 +- amount: 25stake, length: 7884000 +- amount: 25stake, length: 7884000 +``` + +```text +OV = 100 +DF = 0 +DV = 0 +BC = 100 +V = 100 +V' = 0 +``` + +1. Immediately receives 1 coin + + ```text + BC = 101 + ``` + +2. Vesting period 1 passes, 25 coins vest + + ```text + V = 75 + V' = 25 + ``` + +3. During vesting period 2, 5 coins are transferred and 5 coins are delegated + + ```text + DV = 5 + BC = 91 + ``` + +4. Vesting period 2 passes, 25 coins vest + + ```text + V = 50 + V' = 50 + ``` + +## Glossary + +* OriginalVesting: The amount of coins (per denomination) that are initially + part of a vesting account. These coins are set at genesis. +* StartTime: The BFT time at which a vesting account starts to vest. +* EndTime: The BFT time at which a vesting account is fully vested. +* DelegatedFree: The tracked amount of coins (per denomination) that are + delegated from a vesting account that have been fully vested at time of delegation. +* DelegatedVesting: The tracked amount of coins (per denomination) that are + delegated from a vesting account that were vesting at time of delegation. +* ContinuousVestingAccount: A vesting account implementation that vests coins + linearly over time. +* DelayedVestingAccount: A vesting account implementation that only fully vests + all coins at a given time. +* PeriodicVestingAccount: A vesting account implementation that vests coins + according to a custom vesting schedule. +* PermanentLockedAccount: It does not ever release coins, locking them indefinitely. + Coins in this account can still be used for delegating and for governance votes even while locked. + +## CLI + +A user can query and interact with the `vesting` module using the CLI. + +### Transactions + +The `tx` commands allow users to interact with the `vesting` module. + +```bash +simd tx vesting --help +``` + +#### create-periodic-vesting-account + +The `create-periodic-vesting-account` command creates a new vesting account funded with an allocation of tokens, where a sequence of coins and period length in seconds. Periods are sequential, in that the duration of a period only starts at the end of the previous period. The duration of the first period starts upon account creation. + +```bash +simd tx vesting create-periodic-vesting-account [to_address] [periods_json_file] [flags] +``` + +Example: + +```bash +simd tx vesting create-periodic-vesting-account cosmos1.. periods.json +``` + +#### create-vesting-account + +The `create-vesting-account` command creates a new vesting account funded with an allocation of tokens. The account can either be a delayed or continuous vesting account, which is determined by the '--delayed' flag. All vesting accouts created will have their start time set by the committed block's time. The end\_time must be provided as a UNIX epoch timestamp. + +```bash +simd tx vesting create-vesting-account [to_address] [amount] [end_time] [flags] +``` + +Example: + +```bash +simd tx vesting create-vesting-account cosmos1.. 100stake 2592000 +``` diff --git a/sdk/v0.54/modules/authz/README.mdx b/sdk/v0.54/modules/authz/README.mdx new file mode 100644 index 000000000..a65fd851f --- /dev/null +++ b/sdk/v0.54/modules/authz/README.mdx @@ -0,0 +1,1343 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/modules/authz/README' +title: 'x/authz' +--- + +## Abstract + +`x/authz` is an implementation of a Cosmos SDK module, per [ADR 30](https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-030-authz-module.md), that allows +granting arbitrary privileges from one account (the granter) to another account (the grantee). Authorizations must be granted for a particular Msg service method one by one using an implementation of the `Authorization` interface. + +## Contents + +* [Concepts](#concepts) + * [Authorization and Grant](#authorization-and-grant) + * [Built-in Authorizations](#built-in-authorizations) + * [Gas](#gas) +* [State](#state) + * [Grant](#grant) + * [GrantQueue](#grantqueue) +* [Messages](#messages) + * [MsgGrant](#msggrant) + * [MsgRevoke](#msgrevoke) + * [MsgExec](#msgexec) +* [Events](#events) +* [Client](#client) + * [CLI](#cli) + * [gRPC](#grpc) + * [REST](#rest) + +## Concepts + +### Authorization and Grant + +The `x/authz` module defines interfaces and messages grant authorizations to perform actions +on behalf of one account to other accounts. The design is defined in the [ADR 030](https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-030-authz-module.md). + +A *grant* is an allowance to execute a Msg by the grantee on behalf of the granter. +Authorization is an interface that must be implemented by a concrete authorization logic to validate and execute grants. Authorizations are extensible and can be defined for any Msg service method even outside of the module where the Msg method is defined. See the `SendAuthorization` example in the next section for more details. + +**Note:** The authz module is different from the [auth (authentication)](/sdk/v0.54/modules/auth/auth/) module that is responsible for specifying the base transaction and account types. + +```go expandable +package authz + +import ( + + "github.com/cosmos/gogoproto/proto" + + sdk "github.com/cosmos/cosmos-sdk/types" +) + +// Authorization represents the interface of various Authorization types implemented +// by other modules. +type Authorization interface { + proto.Message + + // MsgTypeURL returns the fully-qualified Msg service method URL (as described in ADR 031), + // which will process and accept or reject a request. + MsgTypeURL() + +string + + // Accept determines whether this grant permits the provided sdk.Msg to be performed, + // and if so provides an upgraded authorization instance. + Accept(ctx sdk.Context, msg sdk.Msg) (AcceptResponse, error) + + // ValidateBasic does a simple validation check that + // doesn't require access to any other information. + ValidateBasic() + +error +} + +// AcceptResponse instruments the controller of an authz message if the request is accepted +// and if it should be updated or deleted. +type AcceptResponse struct { + // If Accept=true, the controller can accept and authorization and handle the update. + Accept bool + // If Delete=true, the controller must delete the authorization object and release + // storage resources. + Delete bool + // Controller, who is calling Authorization.Accept must check if `Updated != nil`. If yes, + // it must use the updated version and handle the update on the storage level. + Updated Authorization +} +``` + +### Built-in Authorizations + +The Cosmos SDK `x/authz` module comes with following authorization types: + +#### GenericAuthorization + +`GenericAuthorization` implements the `Authorization` interface that gives unrestricted permission to execute the provided Msg on behalf of granter's account. + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/authz/v1beta1/authz.proto#L14-L22 +``` + +```go expandable +package authz + +import ( + + sdk "github.com/cosmos/cosmos-sdk/types" +) + +var _ Authorization = &GenericAuthorization{ +} + +// NewGenericAuthorization creates a new GenericAuthorization object. +func NewGenericAuthorization(msgTypeURL string) *GenericAuthorization { + return &GenericAuthorization{ + Msg: msgTypeURL, +} +} + +// MsgTypeURL implements Authorization.MsgTypeURL. +func (a GenericAuthorization) + +MsgTypeURL() + +string { + return a.Msg +} + +// Accept implements Authorization.Accept. +func (a GenericAuthorization) + +Accept(ctx sdk.Context, msg sdk.Msg) (AcceptResponse, error) { + return AcceptResponse{ + Accept: true +}, nil +} + +// ValidateBasic implements Authorization.ValidateBasic. +func (a GenericAuthorization) + +ValidateBasic() + +error { + return nil +} +``` + +* `msg` stores Msg type URL. + +#### SendAuthorization + +`SendAuthorization` implements the `Authorization` interface for the `cosmos.bank.v1beta1.MsgSend` Msg. + +* It takes a (positive) `SpendLimit` that specifies the maximum amount of tokens the grantee can spend. The `SpendLimit` is updated as the tokens are spent. +* It takes an (optional) `AllowList` that specifies to which addresses a grantee can send token. + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/bank/v1beta1/authz.proto#L11-L30 +``` + +```go expandable +package types + +import ( + + sdk "github.com/cosmos/cosmos-sdk/types" + sdkerrors "github.com/cosmos/cosmos-sdk/types/errors" + "github.com/cosmos/cosmos-sdk/x/authz" +) + +// TODO: Revisit this once we have proper gas fee framework. +// Ref: https://github.com/cosmos/cosmos-sdk/issues/9054 +// Ref: https://github.com/cosmos/cosmos-sdk/discussions/9072 +const gasCostPerIteration = uint64(10) + +var _ authz.Authorization = &SendAuthorization{ +} + +// NewSendAuthorization creates a new SendAuthorization object. +func NewSendAuthorization(spendLimit sdk.Coins, allowed []sdk.AccAddress) *SendAuthorization { + return &SendAuthorization{ + AllowList: toBech32Addresses(allowed), + SpendLimit: spendLimit, +} +} + +// MsgTypeURL implements Authorization.MsgTypeURL. +func (a SendAuthorization) + +MsgTypeURL() + +string { + return sdk.MsgTypeURL(&MsgSend{ +}) +} + +// Accept implements Authorization.Accept. +func (a SendAuthorization) + +Accept(ctx sdk.Context, msg sdk.Msg) (authz.AcceptResponse, error) { + mSend, ok := msg.(*MsgSend) + if !ok { + return authz.AcceptResponse{ +}, sdkerrors.ErrInvalidType.Wrap("type mismatch") +} + toAddr := mSend.ToAddress + + limitLeft, isNegative := a.SpendLimit.SafeSub(mSend.Amount...) + if isNegative { + return authz.AcceptResponse{ +}, sdkerrors.ErrInsufficientFunds.Wrapf("requested amount is more than spend limit") +} + if limitLeft.IsZero() { + return authz.AcceptResponse{ + Accept: true, + Delete: true +}, nil +} + isAddrExists := false + allowedList := a.GetAllowList() + for _, addr := range allowedList { + ctx.GasMeter().ConsumeGas(gasCostPerIteration, "send authorization") + if addr == toAddr { + isAddrExists = true + break +} + +} + if len(allowedList) > 0 && !isAddrExists { + return authz.AcceptResponse{ +}, sdkerrors.ErrUnauthorized.Wrapf("cannot send to %s address", toAddr) +} + +return authz.AcceptResponse{ + Accept: true, + Delete: false, + Updated: &SendAuthorization{ + SpendLimit: limitLeft, + AllowList: allowedList +}}, nil +} + +// ValidateBasic implements Authorization.ValidateBasic. +func (a SendAuthorization) + +ValidateBasic() + +error { + if a.SpendLimit == nil { + return sdkerrors.ErrInvalidCoins.Wrap("spend limit cannot be nil") +} + if !a.SpendLimit.IsAllPositive() { + return sdkerrors.ErrInvalidCoins.Wrapf("spend limit must be positive") +} + found := make(map[string]bool, 0) + for i := 0; i < len(a.AllowList); i++ { + if found[a.AllowList[i]] { + return ErrDuplicateEntry +} + +found[a.AllowList[i]] = true +} + +return nil +} + +func toBech32Addresses(allowed []sdk.AccAddress) []string { + if len(allowed) == 0 { + return nil +} + allowedAddrs := make([]string, len(allowed)) + for i, addr := range allowed { + allowedAddrs[i] = addr.String() +} + +return allowedAddrs +} +``` + +* `spend_limit` keeps track of how many coins are left in the authorization. +* `allow_list` specifies an optional list of addresses to whom the grantee can send tokens on behalf of the granter. + +#### StakeAuthorization + +`StakeAuthorization` implements the `Authorization` interface for messages in the [staking module](/sdk/v0.54/modules/staking). It takes an `AuthorizationType` to specify whether you want to authorise delegating, undelegating or redelegating (i.e. these have to be authorised separately). It also takes an optional `MaxTokens` that keeps track of a limit to the amount of tokens that can be delegated/undelegated/redelegated. If left empty, the amount is unlimited. Additionally, this Msg takes an `AllowList` or a `DenyList`, which allows you to select which validators you allow or deny grantees to stake with. + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/authz.proto#L11-L35 +``` + +```go expandable +package types + +import ( + + sdk "github.com/cosmos/cosmos-sdk/types" + sdkerrors "github.com/cosmos/cosmos-sdk/types/errors" + "github.com/cosmos/cosmos-sdk/x/authz" +) + +// TODO: Revisit this once we have proper gas fee framework. +// Tracking issues https://github.com/cosmos/cosmos-sdk/issues/9054, https://github.com/cosmos/cosmos-sdk/discussions/9072 +const gasCostPerIteration = uint64(10) + +var _ authz.Authorization = &StakeAuthorization{ +} + +// NewStakeAuthorization creates a new StakeAuthorization object. +func NewStakeAuthorization(allowed []sdk.ValAddress, denied []sdk.ValAddress, authzType AuthorizationType, amount *sdk.Coin) (*StakeAuthorization, error) { + allowedValidators, deniedValidators, err := validateAllowAndDenyValidators(allowed, denied) + if err != nil { + return nil, err +} + a := StakeAuthorization{ +} + if allowedValidators != nil { + a.Validators = &StakeAuthorization_AllowList{ + AllowList: &StakeAuthorization_Validators{ + Address: allowedValidators +}} + +} + +else { + a.Validators = &StakeAuthorization_DenyList{ + DenyList: &StakeAuthorization_Validators{ + Address: deniedValidators +}} + +} + if amount != nil { + a.MaxTokens = amount +} + +a.AuthorizationType = authzType + + return &a, nil +} + +// MsgTypeURL implements Authorization.MsgTypeURL. +func (a StakeAuthorization) + +MsgTypeURL() + +string { + authzType, err := normalizeAuthzType(a.AuthorizationType) + if err != nil { + panic(err) +} + +return authzType +} + +func (a StakeAuthorization) + +ValidateBasic() + +error { + if a.MaxTokens != nil && a.MaxTokens.IsNegative() { + return sdkerrors.Wrapf(authz.ErrNegativeMaxTokens, "negative coin amount: %v", a.MaxTokens) +} + if a.AuthorizationType == AuthorizationType_AUTHORIZATION_TYPE_UNSPECIFIED { + return authz.ErrUnknownAuthorizationType +} + +return nil +} + +// Accept implements Authorization.Accept. +func (a StakeAuthorization) + +Accept(ctx sdk.Context, msg sdk.Msg) (authz.AcceptResponse, error) { + var validatorAddress string + var amount sdk.Coin + switch msg := msg.(type) { + case *MsgDelegate: + validatorAddress = msg.ValidatorAddress + amount = msg.Amount + case *MsgUndelegate: + validatorAddress = msg.ValidatorAddress + amount = msg.Amount + case *MsgBeginRedelegate: + validatorAddress = msg.ValidatorDstAddress + amount = msg.Amount + default: + return authz.AcceptResponse{ +}, sdkerrors.ErrInvalidRequest.Wrap("unknown msg type") +} + isValidatorExists := false + allowedList := a.GetAllowList().GetAddress() + for _, validator := range allowedList { + ctx.GasMeter().ConsumeGas(gasCostPerIteration, "stake authorization") + if validator == validatorAddress { + isValidatorExists = true + break +} + +} + denyList := a.GetDenyList().GetAddress() + for _, validator := range denyList { + ctx.GasMeter().ConsumeGas(gasCostPerIteration, "stake authorization") + if validator == validatorAddress { + return authz.AcceptResponse{ +}, sdkerrors.ErrUnauthorized.Wrapf("cannot delegate/undelegate to %s validator", validator) +} + +} + if len(allowedList) > 0 && !isValidatorExists { + return authz.AcceptResponse{ +}, sdkerrors.ErrUnauthorized.Wrapf("cannot delegate/undelegate to %s validator", validatorAddress) +} + if a.MaxTokens == nil { + return authz.AcceptResponse{ + Accept: true, + Delete: false, + Updated: &StakeAuthorization{ + Validators: a.GetValidators(), + AuthorizationType: a.GetAuthorizationType() +}, +}, nil +} + +limitLeft, err := a.MaxTokens.SafeSub(amount) + if err != nil { + return authz.AcceptResponse{ +}, err +} + if limitLeft.IsZero() { + return authz.AcceptResponse{ + Accept: true, + Delete: true +}, nil +} + +return authz.AcceptResponse{ + Accept: true, + Delete: false, + Updated: &StakeAuthorization{ + Validators: a.GetValidators(), + AuthorizationType: a.GetAuthorizationType(), + MaxTokens: &limitLeft +}, +}, nil +} + +func validateAllowAndDenyValidators(allowed []sdk.ValAddress, denied []sdk.ValAddress) ([]string, []string, error) { + if len(allowed) == 0 && len(denied) == 0 { + return nil, nil, sdkerrors.ErrInvalidRequest.Wrap("both allowed & deny list cannot be empty") +} + if len(allowed) > 0 && len(denied) > 0 { + return nil, nil, sdkerrors.ErrInvalidRequest.Wrap("cannot set both allowed & deny list") +} + allowedValidators := make([]string, len(allowed)) + if len(allowed) > 0 { + for i, validator := range allowed { + allowedValidators[i] = validator.String() +} + +return allowedValidators, nil, nil +} + deniedValidators := make([]string, len(denied)) + for i, validator := range denied { + deniedValidators[i] = validator.String() +} + +return nil, deniedValidators, nil +} + +// Normalized Msg type URLs +func normalizeAuthzType(authzType AuthorizationType) (string, error) { + switch authzType { + case AuthorizationType_AUTHORIZATION_TYPE_DELEGATE: + return sdk.MsgTypeURL(&MsgDelegate{ +}), nil + case AuthorizationType_AUTHORIZATION_TYPE_UNDELEGATE: + return sdk.MsgTypeURL(&MsgUndelegate{ +}), nil + case AuthorizationType_AUTHORIZATION_TYPE_REDELEGATE: + return sdk.MsgTypeURL(&MsgBeginRedelegate{ +}), nil + default: + return "", sdkerrors.Wrapf(authz.ErrUnknownAuthorizationType, "cannot normalize authz type with %T", authzType) +} +} +``` + +### Gas + +In order to prevent DoS attacks, granting `StakeAuthorization`s with `x/authz` incurs gas. `StakeAuthorization` allows you to authorize another account to delegate, undelegate, or redelegate to validators. The authorizer can define a list of validators they allow or deny delegations to. The Cosmos SDK iterates over these lists and charge 10 gas for each validator in both of the lists. + +Since the state maintains a list for granter, grantee pair with the same expiration, we are iterating over the list to remove the grant (in case of any revoke of a particular `msgType`) from the list and we are charging 20 gas per iteration. + +## State + +### Grant + +Grants are identified by combining granter address (the address bytes of the granter), grantee address (the address bytes of the grantee) and Authorization type (its type URL). Hence we only allow one grant for the (granter, grantee, Authorization) triple. + +* Grant: `0x01 | granter_address_len (1 byte) | granter_address_bytes | grantee_address_len (1 byte) | grantee_address_bytes | msgType_bytes -> ProtocolBuffer(AuthorizationGrant)` + +The grant object encapsulates an `Authorization` type and an expiration timestamp: + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/authz/v1beta1/authz.proto#L24-L32 +``` + +### GrantQueue + +We are maintaining a queue for authz pruning. Whenever a grant is created, an item will be added to `GrantQueue` with a key of expiration, granter, grantee. + +In `EndBlock` (which runs for every block) we continuously check and prune the expired grants by forming a prefix key with current blocktime that passed the stored expiration in `GrantQueue`, we iterate through all the matched records from `GrantQueue` and delete them from the `GrantQueue` & `Grant`s store. + +```go expandable +package keeper + +import ( + + "fmt" + "strconv" + "time" + "github.com/cosmos/gogoproto/proto" + abci "github.com/tendermint/tendermint/abci/types" + "github.com/tendermint/tendermint/libs/log" + "github.com/cosmos/cosmos-sdk/baseapp" + "github.com/cosmos/cosmos-sdk/codec" + codectypes "github.com/cosmos/cosmos-sdk/codec/types" + storetypes "github.com/cosmos/cosmos-sdk/store/types" + sdk "github.com/cosmos/cosmos-sdk/types" + sdkerrors "github.com/cosmos/cosmos-sdk/types/errors" + "github.com/cosmos/cosmos-sdk/x/authz" +) + +// TODO: Revisit this once we have proper gas fee framework. +// Tracking issues https://github.com/cosmos/cosmos-sdk/issues/9054, +// https://github.com/cosmos/cosmos-sdk/discussions/9072 +const gasCostPerIteration = uint64(20) + +type Keeper struct { + storeKey storetypes.StoreKey + cdc codec.BinaryCodec + router *baseapp.MsgServiceRouter + authKeeper authz.AccountKeeper +} + +// NewKeeper constructs a message authorization Keeper +func NewKeeper(storeKey storetypes.StoreKey, cdc codec.BinaryCodec, router *baseapp.MsgServiceRouter, ak authz.AccountKeeper) + +Keeper { + return Keeper{ + storeKey: storeKey, + cdc: cdc, + router: router, + authKeeper: ak, +} +} + +// Logger returns a module-specific logger. +func (k Keeper) + +Logger(ctx sdk.Context) + +log.Logger { + return ctx.Logger().With("module", fmt.Sprintf("x/%s", authz.ModuleName)) +} + +// getGrant returns grant stored at skey. +func (k Keeper) + +getGrant(ctx sdk.Context, skey []byte) (grant authz.Grant, found bool) { + store := ctx.KVStore(k.storeKey) + bz := store.Get(skey) + if bz == nil { + return grant, false +} + +k.cdc.MustUnmarshal(bz, &grant) + +return grant, true +} + +func (k Keeper) + +update(ctx sdk.Context, grantee sdk.AccAddress, granter sdk.AccAddress, updated authz.Authorization) + +error { + skey := grantStoreKey(grantee, granter, updated.MsgTypeURL()) + +grant, found := k.getGrant(ctx, skey) + if !found { + return authz.ErrNoAuthorizationFound +} + +msg, ok := updated.(proto.Message) + if !ok { + return sdkerrors.ErrPackAny.Wrapf("cannot proto marshal %T", updated) +} + +any, err := codectypes.NewAnyWithValue(msg) + if err != nil { + return err +} + +grant.Authorization = any + store := ctx.KVStore(k.storeKey) + +store.Set(skey, k.cdc.MustMarshal(&grant)) + +return nil +} + +// DispatchActions attempts to execute the provided messages via authorization +// grants from the message signer to the grantee. +func (k Keeper) + +DispatchActions(ctx sdk.Context, grantee sdk.AccAddress, msgs []sdk.Msg) ([][]byte, error) { + results := make([][]byte, len(msgs)) + now := ctx.BlockTime() + for i, msg := range msgs { + signers := msg.GetSigners() + if len(signers) != 1 { + return nil, authz.ErrAuthorizationNumOfSigners +} + granter := signers[0] + + // If granter != grantee then check authorization.Accept, otherwise we + // implicitly accept. + if !granter.Equals(grantee) { + skey := grantStoreKey(grantee, granter, sdk.MsgTypeURL(msg)) + +grant, found := k.getGrant(ctx, skey) + if !found { + return nil, sdkerrors.Wrapf(authz.ErrNoAuthorizationFound, "failed to update grant with key %s", string(skey)) +} + if grant.Expiration != nil && grant.Expiration.Before(now) { + return nil, authz.ErrAuthorizationExpired +} + +authorization, err := grant.GetAuthorization() + if err != nil { + return nil, err +} + +resp, err := authorization.Accept(ctx, msg) + if err != nil { + return nil, err +} + if resp.Delete { + err = k.DeleteGrant(ctx, grantee, granter, sdk.MsgTypeURL(msg)) +} + +else if resp.Updated != nil { + err = k.update(ctx, grantee, granter, resp.Updated) +} + if err != nil { + return nil, err +} + if !resp.Accept { + return nil, sdkerrors.ErrUnauthorized +} + +} + handler := k.router.Handler(msg) + if handler == nil { + return nil, sdkerrors.ErrUnknownRequest.Wrapf("unrecognized message route: %s", sdk.MsgTypeURL(msg)) +} + +msgResp, err := handler(ctx, msg) + if err != nil { + return nil, sdkerrors.Wrapf(err, "failed to execute message; message %v", msg) +} + +results[i] = msgResp.Data + + // emit the events from the dispatched actions + events := msgResp.Events + sdkEvents := make([]sdk.Event, 0, len(events)) + for _, event := range events { + e := event + e.Attributes = append(e.Attributes, abci.EventAttribute{ + Key: "authz_msg_index", + Value: strconv.Itoa(i) +}) + +sdkEvents = append(sdkEvents, sdk.Event(e)) +} + +ctx.EventManager().EmitEvents(sdkEvents) +} + +return results, nil +} + +// SaveGrant method grants the provided authorization to the grantee on the granter's account +// with the provided expiration time and insert authorization key into the grants queue. If there is an existing authorization grant for the +// same `sdk.Msg` type, this grant overwrites that. +func (k Keeper) + +SaveGrant(ctx sdk.Context, grantee, granter sdk.AccAddress, authorization authz.Authorization, expiration *time.Time) + +error { + store := ctx.KVStore(k.storeKey) + msgType := authorization.MsgTypeURL() + skey := grantStoreKey(grantee, granter, msgType) + +grant, err := authz.NewGrant(ctx.BlockTime(), authorization, expiration) + if err != nil { + return err +} + +var oldExp *time.Time + if oldGrant, found := k.getGrant(ctx, skey); found { + oldExp = oldGrant.Expiration +} + if oldExp != nil && (expiration == nil || !oldExp.Equal(*expiration)) { + if err = k.removeFromGrantQueue(ctx, skey, granter, grantee, *oldExp); err != nil { + return err +} + +} + + // If the expiration didn't change, then we don't remove it and we should not insert again + if expiration != nil && (oldExp == nil || !oldExp.Equal(*expiration)) { + if err = k.insertIntoGrantQueue(ctx, granter, grantee, msgType, *expiration); err != nil { + return err +} + +} + bz := k.cdc.MustMarshal(&grant) + +store.Set(skey, bz) + +return ctx.EventManager().EmitTypedEvent(&authz.EventGrant{ + MsgTypeUrl: authorization.MsgTypeURL(), + Granter: granter.String(), + Grantee: grantee.String(), +}) +} + +// DeleteGrant revokes any authorization for the provided message type granted to the grantee +// by the granter. +func (k Keeper) + +DeleteGrant(ctx sdk.Context, grantee sdk.AccAddress, granter sdk.AccAddress, msgType string) + +error { + store := ctx.KVStore(k.storeKey) + skey := grantStoreKey(grantee, granter, msgType) + +grant, found := k.getGrant(ctx, skey) + if !found { + return sdkerrors.Wrapf(authz.ErrNoAuthorizationFound, "failed to delete grant with key %s", string(skey)) +} + if grant.Expiration != nil { + err := k.removeFromGrantQueue(ctx, skey, granter, grantee, *grant.Expiration) + if err != nil { + return err +} + +} + +store.Delete(skey) + +return ctx.EventManager().EmitTypedEvent(&authz.EventRevoke{ + MsgTypeUrl: msgType, + Granter: granter.String(), + Grantee: grantee.String(), +}) +} + +// GetAuthorizations Returns list of `Authorizations` granted to the grantee by the granter. +func (k Keeper) + +GetAuthorizations(ctx sdk.Context, grantee sdk.AccAddress, granter sdk.AccAddress) ([]authz.Authorization, error) { + store := ctx.KVStore(k.storeKey) + key := grantStoreKey(grantee, granter, "") + iter := sdk.KVStorePrefixIterator(store, key) + +defer iter.Close() + +var authorization authz.Grant + var authorizations []authz.Authorization + for ; iter.Valid(); iter.Next() { + if err := k.cdc.Unmarshal(iter.Value(), &authorization); err != nil { + return nil, err +} + +a, err := authorization.GetAuthorization() + if err != nil { + return nil, err +} + +authorizations = append(authorizations, a) +} + +return authorizations, nil +} + +// GetAuthorization returns an Authorization and it's expiration time. +// A nil Authorization is returned under the following circumstances: +// - No grant is found. +// - A grant is found, but it is expired. +// - There was an error getting the authorization from the grant. +func (k Keeper) + +GetAuthorization(ctx sdk.Context, grantee sdk.AccAddress, granter sdk.AccAddress, msgType string) (authz.Authorization, *time.Time) { + grant, found := k.getGrant(ctx, grantStoreKey(grantee, granter, msgType)) + if !found || (grant.Expiration != nil && grant.Expiration.Before(ctx.BlockHeader().Time)) { + return nil, nil +} + +auth, err := grant.GetAuthorization() + if err != nil { + return nil, nil +} + +return auth, grant.Expiration +} + +// IterateGrants iterates over all authorization grants +// This function should be used with caution because it can involve significant IO operations. +// It should not be used in query or msg services without charging additional gas. +// The iteration stops when the handler function returns true or the iterator exhaust. +func (k Keeper) + +IterateGrants(ctx sdk.Context, + handler func(granterAddr sdk.AccAddress, granteeAddr sdk.AccAddress, grant authz.Grant) + +bool, +) { + store := ctx.KVStore(k.storeKey) + iter := sdk.KVStorePrefixIterator(store, GrantKey) + +defer iter.Close() + for ; iter.Valid(); iter.Next() { + var grant authz.Grant + granterAddr, granteeAddr, _ := parseGrantStoreKey(iter.Key()) + +k.cdc.MustUnmarshal(iter.Value(), &grant) + if handler(granterAddr, granteeAddr, grant) { + break +} + +} +} + +func (k Keeper) + +getGrantQueueItem(ctx sdk.Context, expiration time.Time, granter, grantee sdk.AccAddress) (*authz.GrantQueueItem, error) { + store := ctx.KVStore(k.storeKey) + bz := store.Get(GrantQueueKey(expiration, granter, grantee)) + if bz == nil { + return &authz.GrantQueueItem{ +}, nil +} + +var queueItems authz.GrantQueueItem + if err := k.cdc.Unmarshal(bz, &queueItems); err != nil { + return nil, err +} + +return &queueItems, nil +} + +func (k Keeper) + +setGrantQueueItem(ctx sdk.Context, expiration time.Time, + granter sdk.AccAddress, grantee sdk.AccAddress, queueItems *authz.GrantQueueItem, +) + +error { + store := ctx.KVStore(k.storeKey) + +bz, err := k.cdc.Marshal(queueItems) + if err != nil { + return err +} + +store.Set(GrantQueueKey(expiration, granter, grantee), bz) + +return nil +} + +// insertIntoGrantQueue inserts a grant key into the grant queue +func (k Keeper) + +insertIntoGrantQueue(ctx sdk.Context, granter, grantee sdk.AccAddress, msgType string, expiration time.Time) + +error { + queueItems, err := k.getGrantQueueItem(ctx, expiration, granter, grantee) + if err != nil { + return err +} + if len(queueItems.MsgTypeUrls) == 0 { + k.setGrantQueueItem(ctx, expiration, granter, grantee, &authz.GrantQueueItem{ + MsgTypeUrls: []string{ + msgType +}, +}) +} + +else { + queueItems.MsgTypeUrls = append(queueItems.MsgTypeUrls, msgType) + +k.setGrantQueueItem(ctx, expiration, granter, grantee, queueItems) +} + +return nil +} + +// removeFromGrantQueue removes a grant key from the grant queue +func (k Keeper) + +removeFromGrantQueue(ctx sdk.Context, grantKey []byte, granter, grantee sdk.AccAddress, expiration time.Time) + +error { + store := ctx.KVStore(k.storeKey) + key := GrantQueueKey(expiration, granter, grantee) + bz := store.Get(key) + if bz == nil { + return sdkerrors.Wrap(authz.ErrNoGrantKeyFound, "can't remove grant from the expire queue, grant key not found") +} + +var queueItem authz.GrantQueueItem + if err := k.cdc.Unmarshal(bz, &queueItem); err != nil { + return err +} + + _, _, msgType := parseGrantStoreKey(grantKey) + queueItems := queueItem.MsgTypeUrls + for index, typeURL := range queueItems { + ctx.GasMeter().ConsumeGas(gasCostPerIteration, "grant queue") + if typeURL == msgType { + end := len(queueItem.MsgTypeUrls) - 1 + queueItems[index] = queueItems[end] + queueItems = queueItems[:end] + if err := k.setGrantQueueItem(ctx, expiration, granter, grantee, &authz.GrantQueueItem{ + MsgTypeUrls: queueItems, +}); err != nil { + return err +} + +break +} + +} + +return nil +} + +// DequeueAndDeleteExpiredGrants deletes expired grants from the state and grant queue. +func (k Keeper) + +DequeueAndDeleteExpiredGrants(ctx sdk.Context) + +error { + store := ctx.KVStore(k.storeKey) + iterator := store.Iterator(GrantQueuePrefix, sdk.InclusiveEndBytes(GrantQueueTimePrefix(ctx.BlockTime()))) + +defer iterator.Close() + for ; iterator.Valid(); iterator.Next() { + var queueItem authz.GrantQueueItem + if err := k.cdc.Unmarshal(iterator.Value(), &queueItem); err != nil { + return err +} + + _, granter, grantee, err := parseGrantQueueKey(iterator.Key()) + if err != nil { + return err +} + +store.Delete(iterator.Key()) + for _, typeURL := range queueItem.MsgTypeUrls { + store.Delete(grantStoreKey(grantee, granter, typeURL)) +} + +} + +return nil +} +``` + +* GrantQueue: `0x02 | expiration_bytes | granter_address_len (1 byte) | granter_address_bytes | grantee_address_len (1 byte) | grantee_address_bytes -> ProtocalBuffer(GrantQueueItem)` + +The `expiration_bytes` are the expiration date in UTC with the format `"2006-01-02T15:04:05.000000000"`. + +```go expandable +package keeper + +import ( + + "time" + "github.com/cosmos/cosmos-sdk/internal/conv" + sdk "github.com/cosmos/cosmos-sdk/types" + "github.com/cosmos/cosmos-sdk/types/address" + "github.com/cosmos/cosmos-sdk/types/kv" + "github.com/cosmos/cosmos-sdk/x/authz" +) + +// Keys for store prefixes +// Items are stored with the following key: values +// +// - 0x01: Grant +// - 0x02: GrantQueueItem +var ( + GrantKey = []byte{0x01 +} // prefix for each key + GrantQueuePrefix = []byte{0x02 +} +) + +var lenTime = len(sdk.FormatTimeBytes(time.Now())) + +// StoreKey is the store key string for authz +const StoreKey = authz.ModuleName + +// grantStoreKey - return authorization store key +// Items are stored with the following key: values +// +// - 0x01: Grant +func grantStoreKey(grantee sdk.AccAddress, granter sdk.AccAddress, msgType string) []byte { + m := conv.UnsafeStrToBytes(msgType) + +granter = address.MustLengthPrefix(granter) + +grantee = address.MustLengthPrefix(grantee) + key := sdk.AppendLengthPrefixedBytes(GrantKey, granter, grantee, m) + +return key +} + +// parseGrantStoreKey - split granter, grantee address and msg type from the authorization key +func parseGrantStoreKey(key []byte) (granterAddr, granteeAddr sdk.AccAddress, msgType string) { + // key is of format: + // 0x01 + + granterAddrLen, granterAddrLenEndIndex := sdk.ParseLengthPrefixedBytes(key, 1, 1) // ignore key[0] since it is a prefix key + granterAddr, granterAddrEndIndex := sdk.ParseLengthPrefixedBytes(key, granterAddrLenEndIndex+1, int(granterAddrLen[0])) + +granteeAddrLen, granteeAddrLenEndIndex := sdk.ParseLengthPrefixedBytes(key, granterAddrEndIndex+1, 1) + +granteeAddr, granteeAddrEndIndex := sdk.ParseLengthPrefixedBytes(key, granteeAddrLenEndIndex+1, int(granteeAddrLen[0])) + +kv.AssertKeyAtLeastLength(key, granteeAddrEndIndex+1) + +return granterAddr, granteeAddr, conv.UnsafeBytesToStr(key[(granteeAddrEndIndex + 1):]) +} + +// parseGrantQueueKey split expiration time, granter and grantee from the grant queue key +func parseGrantQueueKey(key []byte) (time.Time, sdk.AccAddress, sdk.AccAddress, error) { + // key is of format: + // 0x02 + + expBytes, expEndIndex := sdk.ParseLengthPrefixedBytes(key, 1, lenTime) + +exp, err := sdk.ParseTimeBytes(expBytes) + if err != nil { + return exp, nil, nil, err +} + +granterAddrLen, granterAddrLenEndIndex := sdk.ParseLengthPrefixedBytes(key, expEndIndex+1, 1) + +granter, granterEndIndex := sdk.ParseLengthPrefixedBytes(key, granterAddrLenEndIndex+1, int(granterAddrLen[0])) + +granteeAddrLen, granteeAddrLenEndIndex := sdk.ParseLengthPrefixedBytes(key, granterEndIndex+1, 1) + +grantee, _ := sdk.ParseLengthPrefixedBytes(key, granteeAddrLenEndIndex+1, int(granteeAddrLen[0])) + +return exp, granter, grantee, nil +} + +// GrantQueueKey - return grant queue store key. If a given grant doesn't have a defined +// expiration, then it should not be used in the pruning queue. +// Key format is: +// +// 0x02: GrantQueueItem +func GrantQueueKey(expiration time.Time, granter sdk.AccAddress, grantee sdk.AccAddress) []byte { + exp := sdk.FormatTimeBytes(expiration) + +granter = address.MustLengthPrefix(granter) + +grantee = address.MustLengthPrefix(grantee) + +return sdk.AppendLengthPrefixedBytes(GrantQueuePrefix, exp, granter, grantee) +} + +// GrantQueueTimePrefix - return grant queue time prefix +func GrantQueueTimePrefix(expiration time.Time) []byte { + return append(GrantQueuePrefix, sdk.FormatTimeBytes(expiration)...) +} + +// firstAddressFromGrantStoreKey parses the first address only +func firstAddressFromGrantStoreKey(key []byte) + +sdk.AccAddress { + addrLen := key[0] + return sdk.AccAddress(key[1 : 1+addrLen]) +} +``` + +The `GrantQueueItem` object contains the list of type urls between granter and grantee that expire at the time indicated in the key. + +## Messages + +In this section we describe the processing of messages for the authz module. + +### MsgGrant + +An authorization grant is created using the `MsgGrant` message. +If there is already a grant for the `(granter, grantee, Authorization)` triple, then the new grant overwrites the previous one. To update or extend an existing grant, a new grant with the same `(granter, grantee, Authorization)` triple should be created. + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/authz/v1beta1/tx.proto#L35-L45 +``` + +The message handling should fail if: + +* both granter and grantee have the same address. +* provided `Expiration` time is less than current unix timestamp (but a grant will be created if no `expiration` time is provided since `expiration` is optional). +* provided `Grant.Authorization` is not implemented. +* `Authorization.MsgTypeURL()` is not defined in the router (there is no defined handler in the app router to handle that Msg types). + +### MsgRevoke + +A grant can be removed with the `MsgRevoke` message. + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/authz/v1beta1/tx.proto#L69-L78 +``` + +The message handling should fail if: + +* both granter and grantee have the same address. +* provided `MsgTypeUrl` is empty. + +NOTE: The `MsgExec` message removes a grant if the grant has expired. + +### MsgExec + +When a grantee wants to execute a transaction on behalf of a granter, they must send `MsgExec`. + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/authz/v1beta1/tx.proto#L52-L63 +``` + +The message handling should fail if: + +* provided `Authorization` is not implemented. +* grantee doesn't have permission to run the transaction. +* if granted authorization is expired. + +## Events + +The authz module emits proto events defined in [the Protobuf reference](https://buf.build/cosmos/cosmos-sdk/docs/main/cosmos.authz.v1beta1#cosmos.authz.v1beta1.EventGrant). + +## Client + +### CLI + +A user can query and interact with the `authz` module using the CLI. + +#### Query + +The `query` commands allow users to query `authz` state. + +```bash +simd query authz --help +``` + +##### grants + +The `grants` command allows users to query grants for a granter-grantee pair. If the message type URL is set, it selects grants only for that message type. + +```bash +simd query authz grants [granter-addr] [grantee-addr] [msg-type-url]? [flags] +``` + +Example: + +```bash +simd query authz grants cosmos1.. cosmos1.. /cosmos.bank.v1beta1.MsgSend +``` + +Example Output: + +```bash +grants: +- authorization: + '@type': /cosmos.bank.v1beta1.SendAuthorization + spend_limit: + - amount: "100" + denom: stake + expiration: "2022-01-01T00:00:00Z" +pagination: null +``` + +#### Transactions + +The `tx` commands allow users to interact with the `authz` module. + +```bash +simd tx authz --help +``` + +##### exec + +The `exec` command allows a grantee to execute a transaction on behalf of granter. + +```bash + simd tx authz exec [tx-json-file] --from [grantee] [flags] +``` + +Example: + +```bash +simd tx authz exec tx.json --from=cosmos1.. +``` + +##### grant + +The `grant` command allows a granter to grant an authorization to a grantee. + +```bash +simd tx authz grant --from [flags] +``` + +* The `send` authorization\_type refers to the built-in `SendAuthorization` type. The custom flags available are `spend-limit` (required) and `allow-list` (optional) , documented [here](#SendAuthorization) + +Example: + +```bash + simd tx authz grant cosmos1.. send --spend-limit=100stake --allow-list=cosmos1...,cosmos2... --from=cosmos1.. +``` + +* The `generic` authorization\_type refers to the built-in `GenericAuthorization` type. The custom flag available is `msg-type` ( required) documented [here](#GenericAuthorization). + +> Note: `msg-type` is any valid Cosmos SDK `Msg` type url. + +Example: + +```bash + simd tx authz grant cosmos1.. generic --msg-type=/cosmos.bank.v1beta1.MsgSend --from=cosmos1.. +``` + +* The `delegate`,`unbond`,`redelegate` authorization\_types refer to the built-in `StakeAuthorization` type. The custom flags available are `spend-limit` (optional), `allowed-validators` (optional) and `deny-validators` (optional) documented [here](#StakeAuthorization). + +> Note: `allowed-validators` and `deny-validators` cannot both be empty. `spend-limit` represents the `MaxTokens` + +Example: + +```bash +simd tx authz grant cosmos1.. delegate --spend-limit=100stake --allowed-validators=cosmos...,cosmos... --deny-validators=cosmos... --from=cosmos1.. +``` + +##### revoke + +The `revoke` command allows a granter to revoke an authorization from a grantee. + +```bash +simd tx authz revoke [grantee] [msg-type-url] --from=[granter] [flags] +``` + +Example: + +```bash +simd tx authz revoke cosmos1.. /cosmos.bank.v1beta1.MsgSend --from=cosmos1.. +``` + +### gRPC + +A user can query the `authz` module using gRPC endpoints. + +#### Grants + +The `Grants` endpoint allows users to query grants for a granter-grantee pair. If the message type URL is set, it selects grants only for that message type. + +```bash +cosmos.authz.v1beta1.Query/Grants +``` + +Example: + +```bash +grpcurl -plaintext \ + -d '{"granter":"cosmos1..","grantee":"cosmos1..","msg_type_url":"/cosmos.bank.v1beta1.MsgSend"}' \ + localhost:9090 \ + cosmos.authz.v1beta1.Query/Grants +``` + +Example Output: + +```bash expandable +{ + "grants": [ + { + "authorization": { + "@type": "/cosmos.bank.v1beta1.SendAuthorization", + "spendLimit": [ + { + "denom":"stake", + "amount":"100" + } + ] + }, + "expiration": "2022-01-01T00:00:00Z" + } + ] +} +``` + +### REST + +A user can query the `authz` module using REST endpoints. + +```bash +/cosmos/authz/v1beta1/grants +``` + +Example: + +```bash +curl "localhost:1317/cosmos/authz/v1beta1/grants?granter=cosmos1..&grantee=cosmos1..&msg_type_url=/cosmos.bank.v1beta1.MsgSend" +``` + +Example Output: + +```bash expandable +{ + "grants": [ + { + "authorization": { + "@type": "/cosmos.bank.v1beta1.SendAuthorization", + "spend_limit": [ + { + "denom": "stake", + "amount": "100" + } + ] + }, + "expiration": "2022-01-01T00:00:00Z" + } + ], + "pagination": null +} +``` diff --git a/sdk/v0.54/modules/bank/README.mdx b/sdk/v0.54/modules/bank/README.mdx new file mode 100644 index 000000000..125be1005 --- /dev/null +++ b/sdk/v0.54/modules/bank/README.mdx @@ -0,0 +1,1139 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/modules/bank/README' +title: 'x/bank' +description: This document specifies the bank module of the Cosmos SDK. +--- + +## Abstract + +This document specifies the bank module of the Cosmos SDK. + +The bank module is responsible for handling multi-asset coin transfers between +accounts and tracking special-case pseudo-transfers which must work differently +with particular kinds of accounts (notably delegating/undelegating for vesting +accounts). It exposes several interfaces with varying capabilities for secure +interaction with other modules which must alter user balances. + +In addition, the bank module tracks and provides query support for the total +supply of all assets used in the application. + +This module is used in the Cosmos Hub. + +## Contents + +* [Supply](#supply) + * [Total Supply](#total-supply) +* [Module Accounts](#module-accounts) + * [Permissions](#permissions) +* [State](#state) +* [Params](#params) +* [Keepers](#keepers) +* [Messages](#messages) +* [Events](#events) + * [Message Events](#message-events) + * [Keeper Events](#keeper-events) +* [Parameters](#parameters) + * [SendEnabled](#sendenabled) + * [DefaultSendEnabled](#defaultsendenabled) +* [Client](#client) + * [CLI](#cli) + * [Query](#query) + * [Transactions](#transactions) +* [gRPC](#grpc) + +## Supply + +The `supply` functionality: + +* passively tracks the total supply of coins within a chain, +* provides a pattern for modules to hold/interact with `Coins`, and +* introduces the invariant check to verify a chain's total supply. + +### Total Supply + +The total `Supply` of the network is equal to the sum of all coins from the +account. The total supply is updated every time a `Coin` is minted (eg: as part +of the inflation mechanism) or burned (eg: due to slashing or if a governance +proposal is vetoed). + +## Module Accounts + +The supply functionality introduces a new type of `auth.Account` which can be used by +modules to allocate tokens and in special cases mint or burn tokens. At a base +level these module accounts are capable of sending/receiving tokens to and from +`auth.Account`s and other module accounts. This design replaces previous +alternative designs where, to hold tokens, modules would burn the incoming +tokens from the sender account, and then track those tokens internally. Later, +in order to send tokens, the module would need to effectively mint tokens +within a destination account. The new design removes duplicate logic between +modules to perform this accounting. + +The `ModuleAccount` interface is defined as follows: + +```go +type ModuleAccount interface { + auth.Account // same methods as the Account interface + + GetName() + +string // name of the module; used to obtain the address + GetPermissions() []string // permissions of module account + HasPermission(string) + +bool +} +``` + +> **WARNING!** +> Any module or message handler that allows either direct or indirect sending of funds must explicitly guarantee those funds cannot be sent to module accounts (unless allowed). + +The supply `Keeper` also introduces new wrapper functions for the auth `Keeper` +and the bank `Keeper` that are related to `ModuleAccount`s in order to be able +to: + +* Get and set `ModuleAccount`s by providing the `Name`. +* Send coins from and to other `ModuleAccount`s or standard `Account`s + (`BaseAccount` or `VestingAccount`) by passing only the `Name`. +* `Mint` or `Burn` coins for a `ModuleAccount` (restricted to its permissions). + +### Permissions + +Each `ModuleAccount` has a different set of permissions that provide different +object capabilities to perform certain actions. Permissions need to be +registered upon the creation of the supply `Keeper` so that every time a +`ModuleAccount` calls the allowed functions, the `Keeper` can lookup the +permissions to that specific account and perform or not perform the action. + +The available permissions are: + +* `Minter`: allows for a module to mint a specific amount of coins. +* `Burner`: allows for a module to burn a specific amount of coins. +* `Staking`: allows for a module to delegate and undelegate a specific amount of coins. + +## State + +The `x/bank` module keeps state of the following primary objects: + +1. Account balances +2. Denomination metadata +3. The total supply of all balances +4. Information on which denominations are allowed to be sent. + +In addition, the `x/bank` module keeps the following indexes to manage the +aforementioned state: + +* Supply Index: `0x0 | byte(denom) -> byte(amount)` +* Denom Metadata Index: `0x1 | byte(denom) -> ProtocolBuffer(Metadata)` +* Balances Index: `0x2 | byte(address length) | []byte(address) | []byte(balance.Denom) -> ProtocolBuffer(balance)` +* Reverse Denomination to Address Index: `0x03 | byte(denom) | 0x00 | []byte(address) -> 0` + +## Params + +The bank module stores its params in state with the prefix of `0x05`, +it can be updated with governance or the address with authority. + +* Params: `0x05 | ProtocolBuffer(Params)` + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/bank/v1beta1/bank.proto#L12-L23 +``` + +## Keepers + +The bank module provides these exported keeper interfaces that can be +passed to other modules that read or update account balances. Modules +should use the least-permissive interface that provides the functionality they +require. + +Best practices dictate careful review of `bank` module code to ensure that +permissions are limited in the way that you expect. + +### Denied Addresses + +The `x/bank` module accepts a map of addresses that are considered blocklisted +from directly and explicitly receiving funds through means such as `MsgSend` and +`MsgMultiSend` and direct API calls like `SendCoinsFromModuleToAccount`. + +Typically, these addresses are module accounts. If these addresses receive funds +outside the expected rules of the state machine, invariants are likely to be +broken and could result in a halted network. + +By providing the `x/bank` module with a blocklisted set of addresses, an error occurs for the operation if a user or client attempts to directly or indirectly send funds to a blocklisted account, for example, by using [IBC](/ibc/latest/intro). + +### Common Types + +#### Input + +An input of a multiparty transfer + +```protobuf +// Input models transaction input. +message Input { + string address = 1; + repeated cosmos.base.v1beta1.Coin coins = 2; +} +``` + +#### Output + +An output of a multiparty transfer. + +```protobuf +// Output models transaction outputs. +message Output { + string address = 1; + repeated cosmos.base.v1beta1.Coin coins = 2; +} +``` + +### BaseKeeper + +The base keeper provides full-permission access: the ability to arbitrary modify any account's balance and mint or burn coins. + +Restricted permission to mint per module could be achieved by using baseKeeper with `WithMintCoinsRestriction` to give specific restrictions to mint (e.g. only minting certain denom). + +```go expandable +// Keeper defines a module interface that facilitates the transfer of coins +// between accounts. +type Keeper interface { + SendKeeper + WithMintCoinsRestriction(MintingRestrictionFn) + +BaseKeeper + + InitGenesis(context.Context, *types.GenesisState) + +ExportGenesis(context.Context) *types.GenesisState + + GetSupply(ctx context.Context, denom string) + +sdk.Coin + HasSupply(ctx context.Context, denom string) + +bool + GetPaginatedTotalSupply(ctx context.Context, pagination *query.PageRequest) (sdk.Coins, *query.PageResponse, error) + +IterateTotalSupply(ctx context.Context, cb func(sdk.Coin) + +bool) + +GetDenomMetaData(ctx context.Context, denom string) (types.Metadata, bool) + +HasDenomMetaData(ctx context.Context, denom string) + +bool + SetDenomMetaData(ctx context.Context, denomMetaData types.Metadata) + +IterateAllDenomMetaData(ctx context.Context, cb func(types.Metadata) + +bool) + +SendCoinsFromModuleToAccount(ctx context.Context, senderModule string, recipientAddr sdk.AccAddress, amt sdk.Coins) + +error + SendCoinsFromModuleToModule(ctx context.Context, senderModule, recipientModule string, amt sdk.Coins) + +error + SendCoinsFromAccountToModule(ctx context.Context, senderAddr sdk.AccAddress, recipientModule string, amt sdk.Coins) + +error + DelegateCoinsFromAccountToModule(ctx context.Context, senderAddr sdk.AccAddress, recipientModule string, amt sdk.Coins) + +error + UndelegateCoinsFromModuleToAccount(ctx context.Context, senderModule string, recipientAddr sdk.AccAddress, amt sdk.Coins) + +error + MintCoins(ctx context.Context, moduleName string, amt sdk.Coins) + +error + BurnCoins(ctx context.Context, moduleName string, amt sdk.Coins) + +error + + DelegateCoins(ctx context.Context, delegatorAddr, moduleAccAddr sdk.AccAddress, amt sdk.Coins) + +error + UndelegateCoins(ctx context.Context, moduleAccAddr, delegatorAddr sdk.AccAddress, amt sdk.Coins) + +error + + // GetAuthority gets the address capable of executing governance proposal messages. Usually the gov module account. + GetAuthority() + +string + + types.QueryServer +} +``` + +### SendKeeper + +The send keeper provides access to account balances and the ability to transfer coins between +accounts. The send keeper does not alter the total supply (mint or burn coins). + +```go expandable +// SendKeeper defines a module interface that facilitates the transfer of coins +// between accounts without the possibility of creating coins. +type SendKeeper interface { + ViewKeeper + + AppendSendRestriction(restriction SendRestrictionFn) + +PrependSendRestriction(restriction SendRestrictionFn) + +ClearSendRestriction() + +InputOutputCoins(ctx context.Context, input types.Input, outputs []types.Output) + +error + SendCoins(ctx context.Context, fromAddr, toAddr sdk.AccAddress, amt sdk.Coins) + +error + + GetParams(ctx context.Context) + +types.Params + SetParams(ctx context.Context, params types.Params) + +error + + IsSendEnabledDenom(ctx context.Context, denom string) + +bool + SetSendEnabled(ctx context.Context, denom string, value bool) + +SetAllSendEnabled(ctx context.Context, sendEnableds []*types.SendEnabled) + +DeleteSendEnabled(ctx context.Context, denom string) + +IterateSendEnabledEntries(ctx context.Context, cb func(denom string, sendEnabled bool) (stop bool)) + +GetAllSendEnabledEntries(ctx context.Context) []types.SendEnabled + + IsSendEnabledCoin(ctx context.Context, coin sdk.Coin) + +bool + IsSendEnabledCoins(ctx context.Context, coins ...sdk.Coin) + +error + + BlockedAddr(addr sdk.AccAddress) + +bool +} +``` + +#### Send Restrictions + +The `SendKeeper` applies a `SendRestrictionFn` before each transfer of funds. + +```golang +// A SendRestrictionFn can restrict sends and/or provide a new receiver address. +type SendRestrictionFn func(ctx context.Context, fromAddr, toAddr sdk.AccAddress, amt sdk.Coins) (newToAddr sdk.AccAddress, err error) +``` + +After the `SendKeeper` (or `BaseKeeper`) has been created, send restrictions can be added to it using the `AppendSendRestriction` or `PrependSendRestriction` functions. +Both functions compose the provided restriction with any previously provided restrictions. +`AppendSendRestriction` adds the provided restriction to be run after any previously provided send restrictions. +`PrependSendRestriction` adds the restriction to be run before any previously provided send restrictions. +The composition will short-circuit when an error is encountered. I.e. if the first one returns an error, the second is not run. + +During `SendCoins`, the send restriction is applied before coins are removed from the from address and adding them to the to address. +During `InputOutputCoins`, the send restriction is applied after the input coins are removed and once for each output before the funds are added. + +A send restriction function should make use of a custom value in the context to allow bypassing that specific restriction. + +Send Restrictions are not placed on `ModuleToAccount` or `ModuleToModule` transfers. This is done due to modules needing to move funds to user accounts and other module accounts. This is a design decision to allow for more flexibility in the state machine. The state machine should be able to move funds between module accounts and user accounts without restrictions. + +Secondly this limitation would limit the usage of the state machine even for itself. users would not be able to receive rewards, not be able to move funds between module accounts. In the case that a user sends funds from a user account to the community pool and then a governance proposal is used to get those tokens into the users account this would fall under the discretion of the app chain developer to what they would like to do here. We can not make strong assumptions here. +Thirdly, this issue could lead into a chain halt if a token is disabled and the token is moved in the begin/endblock. This is the last reason we see the current change and more damaging then beneficial for users. + +For example, in your module's keeper package, you'd define the send restriction function: + +```golang expandable +var _ banktypes.SendRestrictionFn = Keeper{ +}.SendRestrictionFn + +func (k Keeper) + +SendRestrictionFn(ctx context.Context, fromAddr, toAddr sdk.AccAddress, amt sdk.Coins) (sdk.AccAddress, error) { + // Bypass if the context says to. + if mymodule.HasBypass(ctx) { + return toAddr, nil +} + + // Your custom send restriction logic goes here. + return nil, errors.New("not implemented") +} +``` + +The bank keeper should be provided to your keeper's constructor so the send restriction can be added to it: + +```golang +func NewKeeper(cdc codec.BinaryCodec, storeKey storetypes.StoreKey, bankKeeper mymodule.BankKeeper) + +Keeper { + rv := Keeper{/*...*/ +} + +bankKeeper.AppendSendRestriction(rv.SendRestrictionFn) + +return rv +} +``` + +Then, in the `mymodule` package, define the context helpers: + +```golang expandable +const bypassKey = "bypass-mymodule-restriction" + +// WithBypass returns a new context that will cause the mymodule bank send restriction to be skipped. +func WithBypass(ctx context.Context) + +context.Context { + return sdk.UnwrapSDKContext(ctx).WithValue(bypassKey, true) +} + +// WithoutBypass returns a new context that will cause the mymodule bank send restriction to not be skipped. +func WithoutBypass(ctx context.Context) + +context.Context { + return sdk.UnwrapSDKContext(ctx).WithValue(bypassKey, false) +} + +// HasBypass checks the context to see if the mymodule bank send restriction should be skipped. +func HasBypass(ctx context.Context) + +bool { + bypassValue := ctx.Value(bypassKey) + if bypassValue == nil { + return false +} + +bypass, isBool := bypassValue.(bool) + +return isBool && bypass +} +``` + +Now, anywhere where you want to use `SendCoins` or `InputOutputCoins`, but you don't want your send restriction applied: + +```golang +func (k Keeper) + +DoThing(ctx context.Context, fromAddr, toAddr sdk.AccAddress, amt sdk.Coins) + +error { + return k.bankKeeper.SendCoins(mymodule.WithBypass(ctx), fromAddr, toAddr, amt) +} +``` + +### ViewKeeper + +The view keeper provides read-only access to account balances. The view keeper does not have balance alteration functionality. All balance lookups are `O(1)`. + +```go expandable +// ViewKeeper defines a module interface that facilitates read only access to +// account balances. +type ViewKeeper interface { + ValidateBalance(ctx context.Context, addr sdk.AccAddress) + +error + HasBalance(ctx context.Context, addr sdk.AccAddress, amt sdk.Coin) + +bool + + GetAllBalances(ctx context.Context, addr sdk.AccAddress) + +sdk.Coins + GetAccountsBalances(ctx context.Context) []types.Balance + GetBalance(ctx context.Context, addr sdk.AccAddress, denom string) + +sdk.Coin + LockedCoins(ctx context.Context, addr sdk.AccAddress) + +sdk.Coins + SpendableCoins(ctx context.Context, addr sdk.AccAddress) + +sdk.Coins + SpendableCoin(ctx context.Context, addr sdk.AccAddress, denom string) + +sdk.Coin + + IterateAccountBalances(ctx context.Context, addr sdk.AccAddress, cb func(coin sdk.Coin) (stop bool)) + +IterateAllBalances(ctx context.Context, cb func(address sdk.AccAddress, coin sdk.Coin) (stop bool)) +} +``` + +## Messages + +### MsgSend + +Send coins from one address to another. + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/bank/v1beta1/tx.proto#L38-L53 +``` + +The message will fail under the following conditions: + +* The coins do not have sending enabled +* The `to` address is restricted + +### MsgMultiSend + +Send coins from one sender and to a series of different address. If any of the receiving addresses do not correspond to an existing account, a new account is created. + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/bank/v1beta1/tx.proto#L58-L69 +``` + +The message will fail under the following conditions: + +* Any of the coins do not have sending enabled +* Any of the `to` addresses are restricted +* Any of the coins are locked +* The inputs and outputs do not correctly correspond to one another + +### MsgUpdateParams + +The `bank` module params can be updated through `MsgUpdateParams`, which can be done using governance proposal. The signer will always be the `gov` module account address. + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/bank/v1beta1/tx.proto#L74-L88 +``` + +The message handling can fail if: + +* signer is not the gov module account address. + +### MsgSetSendEnabled + +Used with the x/gov module to set create/edit SendEnabled entries. + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/bank/v1beta1/tx.proto#L96-L117 +``` + +The message will fail under the following conditions: + +* The authority is not a bech32 address. +* The authority is not x/gov module's address. +* There are multiple SendEnabled entries with the same Denom. +* One or more SendEnabled entries has an invalid Denom. + +## Events + +The bank module emits the following events: + +### Message Events + +#### MsgSend + +| Type | Attribute Key | Attribute Value | +| -------- | ------------- | ------------------ | +| transfer | recipient | `{recipientAddress}` | +| transfer | amount | `{amount}` | +| message | module | bank | +| message | action | send | +| message | sender | `{senderAddress}` | + +#### MsgMultiSend + +| Type | Attribute Key | Attribute Value | +| -------- | ------------- | ------------------ | +| transfer | recipient | `{recipientAddress}` | +| transfer | amount | `{amount}` | +| message | module | bank | +| message | action | multisend | +| message | sender | `{senderAddress}` | + +### Keeper Events + +In addition to message events, the bank keeper will produce events when the following methods are called (or any method which ends up calling them) + +#### MintCoins + +```json expandable +{ + "type": "coinbase", + "attributes": [ + { + "key": "minter", + "value": "{{sdk.AccAddress of the module minting coins}}", + "index": true + }, + { + "key": "amount", + "value": "{{sdk.Coins being minted}}", + "index": true + } + ] +} +``` + +```json expandable +{ + "type": "coin_received", + "attributes": [ + { + "key": "receiver", + "value": "{{sdk.AccAddress of the module minting coins}}", + "index": true + }, + { + "key": "amount", + "value": "{{sdk.Coins being received}}", + "index": true + } + ] +} +``` + +#### BurnCoins + +```json expandable +{ + "type": "burn", + "attributes": [ + { + "key": "burner", + "value": "{{sdk.AccAddress of the module burning coins}}", + "index": true + }, + { + "key": "amount", + "value": "{{sdk.Coins being burned}}", + "index": true + } + ] +} +``` + +```json expandable +{ + "type": "coin_spent", + "attributes": [ + { + "key": "spender", + "value": "{{sdk.AccAddress of the module burning coins}}", + "index": true + }, + { + "key": "amount", + "value": "{{sdk.Coins being burned}}", + "index": true + } + ] +} +``` + +#### addCoins + +```json expandable +{ + "type": "coin_received", + "attributes": [ + { + "key": "receiver", + "value": "{{sdk.AccAddress of the address beneficiary of the coins}}", + "index": true + }, + { + "key": "amount", + "value": "{{sdk.Coins being received}}", + "index": true + } + ] +} +``` + +#### subUnlockedCoins/DelegateCoins + +```json expandable +{ + "type": "coin_spent", + "attributes": [ + { + "key": "spender", + "value": "{{sdk.AccAddress of the address which is spending coins}}", + "index": true + }, + { + "key": "amount", + "value": "{{sdk.Coins being spent}}", + "index": true + } + ] +} +``` + +## Parameters + +The bank module contains the following parameters + +### SendEnabled + +The SendEnabled parameter is now deprecated and not to be use. It is replaced +with state store records. + +### DefaultSendEnabled + +The default send enabled value controls send transfer capability for all +coin denominations unless specifically included in the array of `SendEnabled` +parameters. + +## Client + +### CLI + +A user can query and interact with the `bank` module using the CLI. + +#### Query + +The `query` commands allow users to query `bank` state. + +```shell +simd query bank --help +``` + +##### balances + +The `balances` command allows users to query account balances by address. + +```shell +simd query bank balances [address] [flags] +``` + +Example: + +```shell +simd query bank balances cosmos1.. +``` + +Example Output: + +```yml +balances: +- amount: "1000000000" + denom: stake +pagination: + next_key: null + total: "0" +``` + +##### denom-metadata + +The `denom-metadata` command allows users to query metadata for coin denominations. A user can query metadata for a single denomination using the `--denom` flag or all denominations without it. + +```shell +simd query bank denom-metadata [flags] +``` + +Example: + +```shell +simd query bank denom-metadata --denom stake +``` + +Example Output: + +```yml +metadata: + base: stake + denom_units: + - aliases: + - STAKE + denom: stake + description: native staking token of simulation app + display: stake + name: SimApp Token + symbol: STK +``` + +##### total + +The `total` command allows users to query the total supply of coins. A user can query the total supply for a single coin using the `--denom` flag or all coins without it. + +```shell +simd query bank total [flags] +``` + +Example: + +```shell +simd query bank total --denom stake +``` + +Example Output: + +```yml +amount: "10000000000" +denom: stake +``` + +##### send-enabled + +The `send-enabled` command allows users to query for all or some SendEnabled entries. + +```shell +simd query bank send-enabled [denom1 ...] [flags] +``` + +Example: + +```shell +simd query bank send-enabled +``` + +Example output: + +```yml +send_enabled: +- denom: foocoin + enabled: true +- denom: barcoin +pagination: + next-key: null + total: 2 +``` + +#### Transactions + +The `tx` commands allow users to interact with the `bank` module. + +```shell +simd tx bank --help +``` + +##### send + +The `send` command allows users to send funds from one account to another. + +```shell +simd tx bank send [from_key_or_address] [to_address] [amount] [flags] +``` + +Example: + +```shell +simd tx bank send cosmos1.. cosmos1.. 100stake +``` + +## gRPC + +A user can query the `bank` module using gRPC endpoints. + +### Balance + +The `Balance` endpoint allows users to query account balance by address for a given denomination. + +```shell +cosmos.bank.v1beta1.Query/Balance +``` + +Example: + +```shell +grpcurl -plaintext \ + -d '{"address":"cosmos1..","denom":"stake"}' \ + localhost:9090 \ + cosmos.bank.v1beta1.Query/Balance +``` + +Example Output: + +```json +{ + "balance": { + "denom": "stake", + "amount": "1000000000" + } +} +``` + +### AllBalances + +The `AllBalances` endpoint allows users to query account balance by address for all denominations. + +```shell +cosmos.bank.v1beta1.Query/AllBalances +``` + +Example: + +```shell +grpcurl -plaintext \ + -d '{"address":"cosmos1.."}' \ + localhost:9090 \ + cosmos.bank.v1beta1.Query/AllBalances +``` + +Example Output: + +```json expandable +{ + "balances": [ + { + "denom": "stake", + "amount": "1000000000" + } + ], + "pagination": { + "total": "1" + } +} +``` + +### DenomMetadata + +The `DenomMetadata` endpoint allows users to query metadata for a single coin denomination. + +```shell +cosmos.bank.v1beta1.Query/DenomMetadata +``` + +Example: + +```shell +grpcurl -plaintext \ + -d '{"denom":"stake"}' \ + localhost:9090 \ + cosmos.bank.v1beta1.Query/DenomMetadata +``` + +Example Output: + +```json expandable +{ + "metadata": { + "description": "native staking token of simulation app", + "denomUnits": [ + { + "denom": "stake", + "aliases": [ + "STAKE" + ] + } + ], + "base": "stake", + "display": "stake", + "name": "SimApp Token", + "symbol": "STK" + } +} +``` + +### DenomsMetadata + +The `DenomsMetadata` endpoint allows users to query metadata for all coin denominations. + +```shell +cosmos.bank.v1beta1.Query/DenomsMetadata +``` + +Example: + +```shell +grpcurl -plaintext \ + localhost:9090 \ + cosmos.bank.v1beta1.Query/DenomsMetadata +``` + +Example Output: + +```json expandable +{ + "metadatas": [ + { + "description": "native staking token of simulation app", + "denomUnits": [ + { + "denom": "stake", + "aliases": [ + "STAKE" + ] + } + ], + "base": "stake", + "display": "stake", + "name": "SimApp Token", + "symbol": "STK" + } + ], + "pagination": { + "total": "1" + } +} +``` + +### DenomOwners + +The `DenomOwners` endpoint allows users to query metadata for a single coin denomination. + +```shell +cosmos.bank.v1beta1.Query/DenomOwners +``` + +Example: + +```shell +grpcurl -plaintext \ + -d '{"denom":"stake"}' \ + localhost:9090 \ + cosmos.bank.v1beta1.Query/DenomOwners +``` + +Example Output: + +```json expandable +{ + "denomOwners": [ + { + "address": "cosmos1..", + "balance": { + "denom": "stake", + "amount": "5000000000" + } + +}, + { + "address": "cosmos1..", + "balance": { + "denom": "stake", + "amount": "5000000000" + } + +}, + ], + "pagination": { + "total": "2" + } +} +``` + +### TotalSupply + +The `TotalSupply` endpoint allows users to query the total supply of all coins. + +```shell +cosmos.bank.v1beta1.Query/TotalSupply +``` + +Example: + +```shell +grpcurl -plaintext \ + localhost:9090 \ + cosmos.bank.v1beta1.Query/TotalSupply +``` + +Example Output: + +```json expandable +{ + "supply": [ + { + "denom": "stake", + "amount": "10000000000" + } + ], + "pagination": { + "total": "1" + } +} +``` + +### SupplyOf + +The `SupplyOf` endpoint allows users to query the total supply of a single coin. + +```shell +cosmos.bank.v1beta1.Query/SupplyOf +``` + +Example: + +```shell +grpcurl -plaintext \ + -d '{"denom":"stake"}' \ + localhost:9090 \ + cosmos.bank.v1beta1.Query/SupplyOf +``` + +Example Output: + +```json +{ + "amount": { + "denom": "stake", + "amount": "10000000000" + } +} +``` + +### Params + +The `Params` endpoint allows users to query the parameters of the `bank` module. + +```shell +cosmos.bank.v1beta1.Query/Params +``` + +Example: + +```shell +grpcurl -plaintext \ + localhost:9090 \ + cosmos.bank.v1beta1.Query/Params +``` + +Example Output: + +```json +{ + "params": { + "defaultSendEnabled": true + } +} +``` + +### SendEnabled + +The `SendEnabled` enpoints allows users to query the SendEnabled entries of the `bank` module. + +Any denominations NOT returned, use the `Params.DefaultSendEnabled` value. + +```shell +cosmos.bank.v1beta1.Query/SendEnabled +``` + +Example: + +```shell +grpcurl -plaintext \ + localhost:9090 \ + cosmos.bank.v1beta1.Query/SendEnabled +``` + +Example Output: + +```json expandable +{ + "send_enabled": [ + { + "denom": "foocoin", + "enabled": true + }, + { + "denom": "barcoin" + } + ], + "pagination": { + "next-key": null, + "total": 2 + } +} +``` diff --git a/sdk/v0.54/modules/circuit/README.mdx b/sdk/v0.54/modules/circuit/README.mdx new file mode 100644 index 000000000..b6c33cbd0 --- /dev/null +++ b/sdk/v0.54/modules/circuit/README.mdx @@ -0,0 +1,597 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/modules/circuit/README' +title: 'x/circuit' +--- + + +`x/circuit` has been moved to [`./contrib/x/circuit`](https://github.com/cosmos/cosmos-sdk/tree/main/contrib/x/circuit) and is no longer actively maintained as part of the core Cosmos SDK. It is still available for use but is not included in the SDK Bug Bounty program. It was moved because it was never widely adopted. + + +## Concepts + +Circuit Breaker is a module that is meant to avoid a chain needing to halt/shut down in the presence of a vulnerability, instead the module will allow specific messages or all messages to be disabled. When operating a chain, if it is app specific then a halt of the chain is less detrimental, but if there are applications built on top of the chain then halting is expensive due to the disturbance to applications. + +Circuit Breaker works with the idea that an address or set of addresses have the right to block messages from being executed and/or included in the mempool. Any address with a permission is able to reset the circuit breaker for the message. + +The transactions are checked and can be rejected at two points: + +* In `CircuitBreakerDecorator` [ante handler](/sdk/v0.54/learn/concepts/baseapp#antehandler): + +```go expandable +package ante + +import ( + + "context" + "github.com/cockroachdb/errors" + + sdk "github.com/cosmos/cosmos-sdk/types" +) + +// CircuitBreaker is an interface that defines the methods for a circuit breaker. +type CircuitBreaker interface { + IsAllowed(ctx context.Context, typeURL string) (bool, error) +} + +// CircuitBreakerDecorator is an AnteDecorator that checks if the transaction type is allowed to enter the mempool or be executed +type CircuitBreakerDecorator struct { + circuitKeeper CircuitBreaker +} + +func NewCircuitBreakerDecorator(ck CircuitBreaker) + +CircuitBreakerDecorator { + return CircuitBreakerDecorator{ + circuitKeeper: ck, +} +} + +func (cbd CircuitBreakerDecorator) + +AnteHandle(ctx sdk.Context, tx sdk.Tx, simulate bool, next sdk.AnteHandler) (sdk.Context, error) { + // loop through all the messages and check if the message type is allowed + for _, msg := range tx.GetMsgs() { + isAllowed, err := cbd.circuitKeeper.IsAllowed(ctx, sdk.MsgTypeURL(msg)) + if err != nil { + return ctx, err +} + if !isAllowed { + return ctx, errors.New("tx type not allowed") +} + +} + +return next(ctx, tx, simulate) +} +``` + +* With a [message router check](/sdk/v0.54/learn/concepts/baseapp#msg-service-router): + +```go expandable +package baseapp + +import ( + + "context" + "fmt" + + gogogrpc "github.com/cosmos/gogoproto/grpc" + "github.com/cosmos/gogoproto/proto" + "google.golang.org/grpc" + "google.golang.org/protobuf/runtime/protoiface" + + errorsmod "cosmossdk.io/errors" + "github.com/cosmos/cosmos-sdk/baseapp/internal/protocompat" + "github.com/cosmos/cosmos-sdk/codec" + codectypes "github.com/cosmos/cosmos-sdk/codec/types" + sdk "github.com/cosmos/cosmos-sdk/types" + sdkerrors "github.com/cosmos/cosmos-sdk/types/errors" +) + +// MessageRouter ADR 031 request type routing +// https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-031-msg-service.md +type MessageRouter interface { + Handler(msg sdk.Msg) + +MsgServiceHandler + HandlerByTypeURL(typeURL string) + +MsgServiceHandler +} + +// MsgServiceRouter routes fully-qualified Msg service methods to their handler. +type MsgServiceRouter struct { + interfaceRegistry codectypes.InterfaceRegistry + routes map[string]MsgServiceHandler + hybridHandlers map[string]func(ctx context.Context, req, resp protoiface.MessageV1) + +error + circuitBreaker CircuitBreaker +} + +var _ gogogrpc.Server = &MsgServiceRouter{ +} + +// NewMsgServiceRouter creates a new MsgServiceRouter. +func NewMsgServiceRouter() *MsgServiceRouter { + return &MsgServiceRouter{ + routes: map[string]MsgServiceHandler{ +}, + hybridHandlers: map[string]func(ctx context.Context, req, resp protoiface.MessageV1) + +error{ +}, +} +} + +func (msr *MsgServiceRouter) + +SetCircuit(cb CircuitBreaker) { + msr.circuitBreaker = cb +} + +// MsgServiceHandler defines a function type which handles Msg service message. +type MsgServiceHandler = func(ctx sdk.Context, req sdk.Msg) (*sdk.Result, error) + +// Handler returns the MsgServiceHandler for a given msg or nil if not found. +func (msr *MsgServiceRouter) + +Handler(msg sdk.Msg) + +MsgServiceHandler { + return msr.routes[sdk.MsgTypeURL(msg)] +} + +// HandlerByTypeURL returns the MsgServiceHandler for a given query route path or nil +// if not found. +func (msr *MsgServiceRouter) + +HandlerByTypeURL(typeURL string) + +MsgServiceHandler { + return msr.routes[typeURL] +} + +// RegisterService implements the gRPC Server.RegisterService method. sd is a gRPC +// service description, handler is an object which implements that gRPC service. +// +// This function PANICs: +// - if it is called before the service `Msg`s have been registered using +// RegisterInterfaces, +// - or if a service is being registered twice. +func (msr *MsgServiceRouter) + +RegisterService(sd *grpc.ServiceDesc, handler interface{ +}) { + // Adds a top-level query handler based on the gRPC service name. + for _, method := range sd.Methods { + err := msr.registerMsgServiceHandler(sd, method, handler) + if err != nil { + panic(err) +} + +err = msr.registerHybridHandler(sd, method, handler) + if err != nil { + panic(err) +} + +} +} + +func (msr *MsgServiceRouter) + +HybridHandlerByMsgName(msgName string) + +func(ctx context.Context, req, resp protoiface.MessageV1) + +error { + return msr.hybridHandlers[msgName] +} + +func (msr *MsgServiceRouter) + +registerHybridHandler(sd *grpc.ServiceDesc, method grpc.MethodDesc, handler interface{ +}) + +error { + inputName, err := protocompat.RequestFullNameFromMethodDesc(sd, method) + if err != nil { + return err +} + cdc := codec.NewProtoCodec(msr.interfaceRegistry) + +hybridHandler, err := protocompat.MakeHybridHandler(cdc, sd, method, handler) + if err != nil { + return err +} + // if circuit breaker is not nil, then we decorate the hybrid handler with the circuit breaker + if msr.circuitBreaker == nil { + msr.hybridHandlers[string(inputName)] = hybridHandler + return nil +} + // decorate the hybrid handler with the circuit breaker + circuitBreakerHybridHandler := func(ctx context.Context, req, resp protoiface.MessageV1) + +error { + messageName := codectypes.MsgTypeURL(req) + +allowed, err := msr.circuitBreaker.IsAllowed(ctx, messageName) + if err != nil { + return err +} + if !allowed { + return fmt.Errorf("circuit breaker disallows execution of message %s", messageName) +} + +return hybridHandler(ctx, req, resp) +} + +msr.hybridHandlers[string(inputName)] = circuitBreakerHybridHandler + return nil +} + +func (msr *MsgServiceRouter) + +registerMsgServiceHandler(sd *grpc.ServiceDesc, method grpc.MethodDesc, handler interface{ +}) + +error { + fqMethod := fmt.Sprintf("/%s/%s", sd.ServiceName, method.MethodName) + methodHandler := method.Handler + + var requestTypeName string + + // NOTE: This is how we pull the concrete request type for each handler for registering in the InterfaceRegistry. + // This approach is maybe a bit hacky, but less hacky than reflecting on the handler object itself. + // We use a no-op interceptor to avoid actually calling into the handler itself. + _, _ = methodHandler(nil, context.Background(), func(i interface{ +}) + +error { + msg, ok := i.(sdk.Msg) + if !ok { + // We panic here because there is no other alternative and the app cannot be initialized correctly + // this should only happen if there is a problem with code generation in which case the app won't + // work correctly anyway. + panic(fmt.Errorf("unable to register service method %s: %T does not implement sdk.Msg", fqMethod, i)) +} + +requestTypeName = sdk.MsgTypeURL(msg) + +return nil +}, noopInterceptor) + + // Check that the service Msg fully-qualified method name has already + // been registered (via RegisterInterfaces). If the user registers a + // service without registering according service Msg type, there might be + // some unexpected behavior down the road. Since we can't return an error + // (`Server.RegisterService` interface restriction) + +we panic (at startup). + reqType, err := msr.interfaceRegistry.Resolve(requestTypeName) + if err != nil || reqType == nil { + return fmt.Errorf( + "type_url %s has not been registered yet. "+ + "Before calling RegisterService, you must register all interfaces by calling the `RegisterInterfaces` "+ + "method on module.BasicManager. Each module should call `msgservice.RegisterMsgServiceDesc` inside its "+ + "`RegisterInterfaces` method with the `_Msg_serviceDesc` generated by proto-gen", + requestTypeName, + ) +} + + // Check that each service is only registered once. If a service is + // registered more than once, then we should error. Since we can't + // return an error (`Server.RegisterService` interface restriction) + +we + // panic (at startup). + _, found := msr.routes[requestTypeName] + if found { + return fmt.Errorf( + "msg service %s has already been registered. Please make sure to only register each service once. "+ + "This usually means that there are conflicting modules registering the same msg service", + fqMethod, + ) +} + +msr.routes[requestTypeName] = func(ctx sdk.Context, msg sdk.Msg) (*sdk.Result, error) { + ctx = ctx.WithEventManager(sdk.NewEventManager()) + interceptor := func(goCtx context.Context, _ interface{ +}, _ *grpc.UnaryServerInfo, handler grpc.UnaryHandler) (interface{ +}, error) { + goCtx = context.WithValue(goCtx, sdk.SdkContextKey, ctx) + +return handler(goCtx, msg) +} + if m, ok := msg.(sdk.HasValidateBasic); ok { + if err := m.ValidateBasic(); err != nil { + return nil, err +} + +} + if msr.circuitBreaker != nil { + msgURL := sdk.MsgTypeURL(msg) + +isAllowed, err := msr.circuitBreaker.IsAllowed(ctx, msgURL) + if err != nil { + return nil, err +} + if !isAllowed { + return nil, fmt.Errorf("circuit breaker disables execution of this message: %s", msgURL) +} + +} + + // Call the method handler from the service description with the handler object. + // We don't do any decoding here because the decoding was already done. + res, err := methodHandler(handler, ctx, noopDecoder, interceptor) + if err != nil { + return nil, err +} + +resMsg, ok := res.(proto.Message) + if !ok { + return nil, errorsmod.Wrapf(sdkerrors.ErrInvalidType, "Expecting proto.Message, got %T", resMsg) +} + +return sdk.WrapServiceResult(ctx, resMsg, err) +} + +return nil +} + +// SetInterfaceRegistry sets the interface registry for the router. +func (msr *MsgServiceRouter) + +SetInterfaceRegistry(interfaceRegistry codectypes.InterfaceRegistry) { + msr.interfaceRegistry = interfaceRegistry +} + +func noopDecoder(_ interface{ +}) + +error { + return nil +} + +func noopInterceptor(_ context.Context, _ interface{ +}, _ *grpc.UnaryServerInfo, _ grpc.UnaryHandler) (interface{ +}, error) { + return nil, nil +} +``` + + +The `CircuitBreakerDecorator` works for most use cases, but [does not check the inner messages of a transaction](/sdk/v0.54/learn/concepts/lifecycle#antehandler). This some transactions (such as `x/authz` transactions or some `x/gov` transactions) may pass the ante handler. **This does not affect the circuit breaker** as the message router check will still fail the transaction. +This tradeoff is to avoid introducing more dependencies in the `x/circuit` module. Chains can re-define the `CircuitBreakerDecorator` to check for inner messages if they wish to do so. + + +## State + +### Accounts + +* AccountPermissions `0x1 | account_address -> ProtocolBuffer(CircuitBreakerPermissions)` + +```go expandable +type level int32 + +const ( + // LEVEL_NONE_UNSPECIFIED indicates that the account will have no circuit + // breaker permissions. + LEVEL_NONE_UNSPECIFIED = iota + // LEVEL_SOME_MSGS indicates that the account will have permission to + // trip or reset the circuit breaker for some Msg type URLs. If this level + // is chosen, a non-empty list of Msg type URLs must be provided in + // limit_type_urls. + LEVEL_SOME_MSGS + // LEVEL_ALL_MSGS indicates that the account can trip or reset the circuit + // breaker for Msg's of all type URLs. + LEVEL_ALL_MSGS + // LEVEL_SUPER_ADMIN indicates that the account can take all circuit breaker + // actions and can grant permissions to other accounts. + LEVEL_SUPER_ADMIN +) + +type Access struct { + level int32 + msgs []string // if full permission, msgs can be empty +} +``` + +### Disable List + +List of type urls that are disabled. + +* DisableList `0x2 | msg_type_url -> []byte{}` {/* - should this be stored in json to skip encoding and decoding each block, does it matter? */} + +## State Transitions + +### Authorize + +Authorize, is called by the module authority (default governance module account) or any account with `LEVEL_SUPER_ADMIN` to give permission to disable/enable messages to another account. There are three levels of permissions that can be granted. `LEVEL_SOME_MSGS` limits the number of messages that can be disabled. `LEVEL_ALL_MSGS` permits all messages to be disabled. `LEVEL_SUPER_ADMIN` allows an account to take all circuit breaker actions including authorizing and deauthorizing other accounts. + +```protobuf + // AuthorizeCircuitBreaker allows a super-admin to grant (or revoke) another + // account's circuit breaker permissions. + rpc AuthorizeCircuitBreaker(MsgAuthorizeCircuitBreaker) returns (MsgAuthorizeCircuitBreakerResponse); +``` + +### Trip + +Trip, is called by an authorized account to disable message execution for a specific msgURL. If empty, all the msgs will be disabled. + +```protobuf + // TripCircuitBreaker pauses processing of Msg's in the state machine. + rpc TripCircuitBreaker(MsgTripCircuitBreaker) returns (MsgTripCircuitBreakerResponse); +``` + +### Reset + +Reset is called by an authorized account to enable execution for a specific msgURL of previously disabled message. If empty, all the disabled messages will be enabled. + +```protobuf + // ResetCircuitBreaker resumes processing of Msg's in the state machine that + // have been paused using TripCircuitBreaker. + rpc ResetCircuitBreaker(MsgResetCircuitBreaker) returns (MsgResetCircuitBreakerResponse); +``` + +## Messages + +### MsgAuthorizeCircuitBreaker + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/blob/main/proto/cosmos/circuit/v1/tx.proto#L25-L75 +``` + +This message is expected to fail if: + +* the granter is not an account with permission level `LEVEL_SUPER_ADMIN` or the module authority + +### MsgTripCircuitBreaker + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/blob/main/proto/cosmos/circuit/v1/tx.proto#L77-L93 +``` + +This message is expected to fail if: + +* if the signer does not have a permission level with the ability to disable the specified type url message + +### MsgResetCircuitBreaker + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/blob/main/proto/cosmos/circuit/v1/tx.proto#L95-109 +``` + +This message is expected to fail if: + +* if the type url is not disabled + +## Events - list and describe event tags + +The circuit module emits the following events: + +### Message Events + +#### MsgAuthorizeCircuitBreaker + +| Type | Attribute Key | Attribute Value | +| ------- | ------------- | --------------------------- | +| string | granter | `{granterAddress}` | +| string | grantee | `{granteeAddress}` | +| string | permission | `{granteePermissions}` | +| message | module | circuit | +| message | action | authorize\_circuit\_breaker | + +#### MsgTripCircuitBreaker + +| Type | Attribute Key | Attribute Value | +| --------- | ------------- | ---------------------- | +| string | authority | `{authorityAddress}` | +| \[]string | msg\_urls | \[]string`{msg\_urls}` | +| message | module | circuit | +| message | action | trip\_circuit\_breaker | + +#### ResetCircuitBreaker + +| Type | Attribute Key | Attribute Value | +| --------- | ------------- | ----------------------- | +| string | authority | `{authorityAddress}` | +| \[]string | msg\_urls | \[]string`{msg\_urls}` | +| message | module | circuit | +| message | action | reset\_circuit\_breaker | + +## Keys - list of key prefixes used by the circuit module + +* `AccountPermissionPrefix` - `0x01` +* `DisableListPrefix` - `0x02` + +## Client - list and describe CLI commands and gRPC and REST endpoints + +## Examples: Using Circuit Breaker CLI Commands + +This section provides practical examples for using the Circuit Breaker module through the command-line interface (CLI). These examples demonstrate how to authorize accounts, disable (trip) specific message types, and re-enable (reset) them when needed. + +### Querying Circuit Breaker Permissions + +Check an account's current circuit breaker permissions: + +```bash +# Query permissions for a specific account + query circuit account-permissions + +# Example: +simd query circuit account-permissions cosmos1... +``` + +Check which message types are currently disabled: + +```bash +# Query all disabled message types + query circuit disabled-list + +# Example: +simd query circuit disabled-list +``` + +### Authorizing an Account as Circuit Breaker + +Only a super-admin or the module authority (typically the governance module account) can grant circuit breaker permissions to other accounts: + +```bash +# Grant LEVEL_ALL_MSGS permission (can disable any message type) + tx circuit authorize --level=ALL_MSGS --from= --gas=auto --gas-adjustment=1.5 + +# Grant LEVEL_SOME_MSGS permission (can only disable specific message types) + tx circuit authorize --level=SOME_MSGS --limit-type-urls="/cosmos.bank.v1beta1.MsgSend,/cosmos.staking.v1beta1.MsgDelegate" --from= --gas=auto --gas-adjustment=1.5 + +# Grant LEVEL_SUPER_ADMIN permission (can disable messages and authorize other accounts) + tx circuit authorize --level=SUPER_ADMIN --from= --gas=auto --gas-adjustment=1.5 +``` + +### Disabling Message Processing (Trip) + +Disable specific message types to prevent their execution (requires authorization): + +```bash +# Disable a single message type + tx circuit trip --type-urls="/cosmos.bank.v1beta1.MsgSend" --from= --gas=auto --gas-adjustment=1.5 + +# Disable multiple message types + tx circuit trip --type-urls="/cosmos.bank.v1beta1.MsgSend,/cosmos.staking.v1beta1.MsgDelegate" --from= --gas=auto --gas-adjustment=1.5 + +# Disable all message types (emergency measure) + tx circuit trip --from= --gas=auto --gas-adjustment=1.5 +``` + +### Re-enabling Message Processing (Reset) + +Re-enable previously disabled message types (requires authorization): + +```bash +# Re-enable a single message type + tx circuit reset --type-urls="/cosmos.bank.v1beta1.MsgSend" --from= --gas=auto --gas-adjustment=1.5 + +# Re-enable multiple message types + tx circuit reset --type-urls="/cosmos.bank.v1beta1.MsgSend,/cosmos.staking.v1beta1.MsgDelegate" --from= --gas=auto --gas-adjustment=1.5 + +# Re-enable all disabled message types + tx circuit reset --from= --gas=auto --gas-adjustment=1.5 +``` + +### Usage in Emergency Scenarios + +In case of a critical vulnerability in a specific message type: + +1. Quickly disable the vulnerable message type: + ```bash + tx circuit trip --type-urls="/cosmos.vulnerable.v1beta1.MsgVulnerable" --from= --gas=auto --gas-adjustment=1.5 + ``` + +2. After a fix is deployed, re-enable the message type: + ```bash + tx circuit reset --type-urls="/cosmos.vulnerable.v1beta1.MsgVulnerable" --from= --gas=auto --gas-adjustment=1.5 + ``` + +This allows chains to surgically disable problematic functionality without halting the entire chain, providing time for developers to implement and deploy fixes. diff --git a/sdk/v0.54/modules/consensus/README.mdx b/sdk/v0.54/modules/consensus/README.mdx new file mode 100644 index 000000000..b346b9f6a --- /dev/null +++ b/sdk/v0.54/modules/consensus/README.mdx @@ -0,0 +1,179 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/modules/consensus/README' +title: 'x/consensus' +description: Functionality to modify CometBFT's ABCI consensus params. +--- + +The `x/consensus` module allows governance to update CometBFT's ABCI consensus parameters on a live chain without a software upgrade. + +## Consensus Parameters + +The module manages the following CometBFT consensus parameters: + +### Block Parameters + +| Parameter | Description | +| --- | --- | +| `MaxBytes` | Maximum block size in bytes | +| `MaxGas` | Maximum gas per block (`-1` for unlimited) | + +### Evidence Parameters + +| Parameter | Description | +| --- | --- | +| `MaxAgeNumBlocks` | Maximum age of evidence in blocks | +| `MaxAgeDuration` | Maximum age of evidence as a duration | +| `MaxBytes` | Maximum total evidence size per block in bytes | + +### Validator Parameters + +| Parameter | Description | +| --- | --- | +| `PubKeyTypes` | Supported public key types for validators (e.g., `ed25519`, `secp256k1`, `bls12381`) | + +### ABCI Parameters + +| Parameter | Description | +| --- | --- | +| `VoteExtensionsEnableHeight` | Block height at which vote extensions are enabled (`0` to disable) | + +## Messages + +### MsgUpdateParams + +Updates consensus parameters via governance. All of `block`, `evidence`, and `validator` must be provided. `abci` is optional. + +```go +msg := &types.MsgUpdateParams{ + Authority: authtypes.NewModuleAddress(govtypes.ModuleName).String(), + Block: &cmtproto.BlockParams{ + MaxBytes: 200000, + MaxGas: 100000000, + }, + Evidence: &cmtproto.EvidenceParams{ + MaxAgeNumBlocks: 302400, + MaxAgeDuration: 504 * time.Hour, + MaxBytes: 10000, + }, + Validator: &cmtproto.ValidatorParams{ + PubKeyTypes: []string{"ed25519"}, + }, + Abci: &cmtproto.ABCIParams{ + VoteExtensionsEnableHeight: 0, + }, +} +``` + +## AuthorityParams + +Authority management can be centralized via the `x/consensus` module using `AuthorityParams`. The `AuthorityParams` field in `ConsensusParams` stores the authority address on-chain. When set, it takes precedence over the per-keeper authority parameter. + +Keeper constructors still accept the `authority` parameter. It is used as a fallback when no authority is configured in consensus params. + +### How It Works + +When a module validates authority (e.g., in `UpdateParams`), it checks consensus params first. If no authority is set there, it falls back to the keeper's `authority` field: + +```go +authority := sdkCtx.Authority() // from consensus params +if authority == "" { + authority = k.authority // fallback to keeper field +} +if authority != msg.Authority { + return nil, errors.Wrapf(...) +} +``` + +To enable centralized authority, set the `AuthorityParams` in consensus params via a governance proposal targeting the `x/consensus` module's `MsgUpdateParams`. + +## CLI + +### Query + +#### params + +Query the current consensus parameters: + +```shell +simd query consensus params +``` + +Example Output: + +```yml +params: + abci: + vote_extensions_enable_height: "0" + block: + max_bytes: "200000" + max_gas: "-1" + evidence: + max_age_duration: 1814400s + max_age_num_blocks: "302400" + max_bytes: "10000" + validator: + pub_key_types: + - ed25519 +``` + +### Transactions + +#### update-params-proposal + +Submit a governance proposal to update consensus parameters: + +```shell +simd tx consensus update-params-proposal [block] [evidence] [validator] [abci] [flags] +``` + +Example: + +```shell +simd tx consensus update-params-proposal \ + '{"max_bytes":"200000","max_gas":"100000000"}' \ + '{"max_age_num_blocks":"302400","max_age_duration":"1814400s","max_bytes":"10000"}' \ + '{"pub_key_types":["ed25519"]}' \ + '{"vote_extensions_enable_height":"0"}' \ + --from mykey +``` + +## gRPC + +### Params + +Query the current consensus parameters: + +```shell +grpcurl -plaintext localhost:9090 cosmos.consensus.v1.Query/Params +``` + +Example Output: + +```json +{ + "params": { + "block": { + "maxBytes": "200000", + "maxGas": "-1" + }, + "evidence": { + "maxAgeNumBlocks": "302400", + "maxAgeDuration": "1814400s", + "maxBytes": "10000" + }, + "validator": { + "pubKeyTypes": ["ed25519"] + }, + "abci": { + "voteExtensionsEnableHeight": "0" + } + } +} +``` + +## REST + +``` +GET /cosmos/consensus/v1/params +``` diff --git a/sdk/v0.54/modules/crisis/README.mdx b/sdk/v0.54/modules/crisis/README.mdx new file mode 100644 index 000000000..c3ba65e16 --- /dev/null +++ b/sdk/v0.54/modules/crisis/README.mdx @@ -0,0 +1,116 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/modules/crisis/README' +title: 'x/crisis' +description: >- + x/crisis has been moved to ./contrib/x/crisis and is no longer part of the core Cosmos SDK. +--- + + +`x/crisis` has been moved to [`./contrib/x/crisis`](https://github.com/cosmos/cosmos-sdk/tree/main/contrib/x/crisis) and is no longer actively maintained as part of the core Cosmos SDK. It is still available for use but is not included in the SDK Bug Bounty program. The module was moved because it never worked as intended. + + +## Overview + +The crisis module halts the blockchain under the circumstance that a blockchain +invariant is broken. Invariants can be registered with the application during the +application initialization process. + +## Contents + +* [State](#state) +* [Messages](#messages) +* [Events](#events) +* [Parameters](#parameters) +* [Client](#client) + * [CLI](#cli) + +## State + +### ConstantFee + +Due to the anticipated large gas cost requirement to verify an invariant (and +potential to exceed the maximum allowable block gas limit) a constant fee is +used instead of the standard gas consumption method. The constant fee is +intended to be larger than the anticipated gas cost of running the invariant +with the standard gas consumption method. + +The ConstantFee param is stored in the module params state with the prefix of `0x01`, +it can be updated with governance or the address with authority. + +* Params: `mint/params -> legacy_amino(sdk.Coin)` + +## Messages + +In this section we describe the processing of the crisis messages and the +corresponding updates to the state. + +### MsgVerifyInvariant + +Blockchain invariants can be checked using the `MsgVerifyInvariant` message. + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/crisis/v1beta1/tx.proto#L26-L42 +``` + +This message is expected to fail if: + +* the sender does not have enough coins for the constant fee +* the invariant route is not registered + +This message checks the invariant provided, and if the invariant is broken it +panics, halting the blockchain. If the invariant is broken, the constant fee is +never deducted as the transaction is never committed to a block (equivalent to +being refunded). However, if the invariant is not broken, the constant fee will +not be refunded. + +## Events + +The crisis module emits the following events: + +### Handlers + +#### MsgVerifyInvariant + +| Type | Attribute Key | Attribute Value | +| --------- | ------------- | ----------------- | +| invariant | route | `{invariantRoute}` | +| message | module | crisis | +| message | action | verify\_invariant | +| message | sender | `{senderAddress}` | + +## Parameters + +The crisis module contains the following parameters: + +| Key | Type | Example | +| ----------- | ------------- | --------------------------------- | +| ConstantFee | object (coin) | `{"denom":"uatom","amount":"1000"}` | + +## Client + +### CLI + +A user can query and interact with the `crisis` module using the CLI. + +#### Transactions + +The `tx` commands allow users to interact with the `crisis` module. + +```bash +simd tx crisis --help +``` + +##### invariant-broken + +The `invariant-broken` command submits proof when an invariant was broken to halt the chain + +```bash +simd tx crisis invariant-broken [module-name] [invariant-route] [flags] +``` + +Example: + +```bash +simd tx crisis invariant-broken bank total-supply --from=[keyname or address] +``` diff --git a/sdk/v0.54/modules/distribution/README.mdx b/sdk/v0.54/modules/distribution/README.mdx new file mode 100644 index 000000000..3efe41a8b --- /dev/null +++ b/sdk/v0.54/modules/distribution/README.mdx @@ -0,0 +1,1305 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/modules/distribution/README' +title: 'x/distribution' +--- + +## Overview + +This *simple* distribution mechanism describes a functional way to passively +distribute rewards between validators and delegators. Note that this mechanism does +not distribute funds in as precisely as active reward distribution mechanisms and +will therefore be upgraded in the future. + +The mechanism operates as follows. Collected rewards are pooled globally and +divided out passively to validators and delegators. Each validator has the +opportunity to charge commission to the delegators on the rewards collected on +behalf of the delegators. Fees are collected directly into a global reward pool +and validator proposer-reward pool. Due to the nature of passive accounting, +whenever changes to parameters which affect the rate of reward distribution +occurs, withdrawal of rewards must also occur. + +* Whenever withdrawing, one must withdraw the maximum amount they are entitled + to, leaving nothing in the pool. +* Whenever bonding, unbonding, or re-delegating tokens to an existing account, a + full withdrawal of the rewards must occur (as the rules for lazy accounting + change). +* Whenever a validator chooses to change the commission on rewards, all accumulated + commission rewards must be simultaneously withdrawn. + +The above scenarios are covered in `hooks.md`. + +The distribution mechanism outlined herein is used to lazily distribute the +following rewards between validators and associated delegators: + +* multi-token fees to be socially distributed +* inflated staked asset provisions +* validator commission on all rewards earned by their delegators stake + +Fees are pooled within a global pool. The mechanisms used allow for validators +and delegators to independently and lazily withdraw their rewards. + +## Shortcomings + +As a part of the lazy computations, each delegator holds an accumulation term +specific to each validator which is used to estimate what their approximate +fair portion of tokens held in the global fee pool is owed to them. + +```text +entitlement = delegator-accumulation / all-delegators-accumulation +``` + +Under the circumstance that there was constant and equal flow of incoming +reward tokens every block, this distribution mechanism would be equal to the +active distribution (distribute individually to all delegators each block). +However, this is unrealistic so deviations from the active distribution will +occur based on fluctuations of incoming reward tokens as well as timing of +reward withdrawal by other delegators. + +If you happen to know that incoming rewards are about to significantly increase, +you are incentivized to not withdraw until after this event, increasing the +worth of your existing *accum*. See [#2764](https://github.com/cosmos/cosmos-sdk/issues/2764) +for further details. + +## Effect on Staking + +Charging commission on Atom provisions while also allowing for Atom-provisions +to be auto-bonded (distributed directly to the validators bonded stake) is +problematic within BPoS. Fundamentally, these two mechanisms are mutually +exclusive. If both commission and auto-bonding mechanisms are simultaneously +applied to the staking-token then the distribution of staking-tokens between +any validator and its delegators will change with each block. This then +necessitates a calculation for each delegation records for each block - +which is considered computationally expensive. + +In conclusion, we can only have Atom commission and unbonded atoms +provisions or bonded atom provisions with no Atom commission, and we elect to +implement the former. Stakeholders wishing to rebond their provisions may elect +to set up a script to periodically withdraw and rebond rewards. + +## Contents + +* [Concepts](#concepts) +* [State](#state) + * [FeePool](#feepool) + * [Validator Distribution](#validator-distribution) + * [Delegation Distribution](#delegation-distribution) + * [Params](#params) +* [Begin Block](#begin-block) +* [Messages](#messages) +* [Hooks](#hooks) +* [Events](#events) +* [Parameters](#parameters) +* [Client](#client) + * [CLI](#cli) + * [gRPC](#grpc) + +## Concepts + +In Proof of Stake (PoS) blockchains, rewards gained from transaction fees are paid to validators. The fee distribution module fairly distributes the rewards to the validators' constituent delegators. + +Rewards are calculated per period. The period is updated each time a validator's delegation changes, for example, when the validator receives a new delegation. +The rewards for a single validator can then be calculated by taking the total rewards for the period before the delegation started, minus the current total rewards. +To learn more, see the [F1 Fee Distribution paper](https://github.com/cosmos/cosmos-sdk/tree/main/docs/spec/fee_distribution/f1_fee_distr.pdf). + +The commission to the validator is paid when the validator is removed or when the validator requests a withdrawal. +The commission is calculated and incremented at every `BeginBlock` operation to update accumulated fee amounts. + +The rewards to a delegator are distributed when the delegation is changed or removed, or a withdrawal is requested. +Before rewards are distributed, all slashes to the validator that occurred during the current delegation are applied. + +### Reference Counting in F1 Fee Distribution + +In F1 fee distribution, the rewards a delegator receives are calculated when their delegation is withdrawn. This calculation must read the terms of the summation of rewards divided by the share of tokens from the period which they ended when they delegated, and the final period that was created for the withdrawal. + +Additionally, as slashes change the amount of tokens a delegation will have (but we calculate this lazily, +only when a delegator un-delegates), we must calculate rewards in separate periods before / after any slashes +which occurred in between when a delegator delegated and when they withdrew their rewards. Thus slashes, like +delegations, reference the period which was ended by the slash event. + +All stored historical rewards records for periods which are no longer referenced by any delegations +or any slashes can thus be safely removed, as they will never be read (future delegations and future +slashes will always reference future periods). This is implemented by tracking a `ReferenceCount` +along with each historical reward storage entry. Each time a new object (delegation or slash) +is created which might need to reference the historical record, the reference count is incremented. +Each time one object which previously needed to reference the historical record is deleted, the reference +count is decremented. If the reference count hits zero, the historical record is deleted. + +### External Community Pool Keepers + +An external pool community keeper is defined as: + +```go expandable +// ExternalCommunityPoolKeeper is the interface that an external community pool module keeper must fulfill +// for x/distribution to properly accept it as a community pool fund destination. +type ExternalCommunityPoolKeeper interface { + // GetCommunityPoolModule gets the module name that funds should be sent to for the community pool. + // This is the address that x/distribution will send funds to for external management. + GetCommunityPoolModule() + +string + // FundCommunityPool allows an account to directly fund the community fund pool. + FundCommunityPool(ctx sdk.Context, amount sdk.Coins, senderAddr sdk.AccAddress) + +error + // DistributeFromCommunityPool distributes funds from the community pool module account to + // a receiver address. + DistributeFromCommunityPool(ctx sdk.Context, amount sdk.Coins, receiveAddr sdk.AccAddress) + +error +} +``` + +By default, the distribution module will use a community pool implementation that is internal. An external community pool +can be provided to the module which will have funds be diverted to it instead of the internal implementation. The reference +external community pool maintained by the Cosmos SDK is [`x/protocolpool`](/sdk/v0.54/modules/protocolpool/README). + +## State + +### FeePool + +All globally tracked parameters for distribution are stored within +`FeePool`. Rewards are collected and added to the reward pool and +distributed to validators/delegators from here. + +Note that the reward pool holds decimal coins (`DecCoins`) to allow +for fractions of coins to be received from operations like inflation. +When coins are distributed from the pool they are truncated back to +`sdk.Coins` which are non-decimal. + +* FeePool: `0x00 -> ProtocolBuffer(FeePool)` + +```go +// coins with decimal +type DecCoins []DecCoin + +type DecCoin struct { + Amount math.LegacyDec + Denom string +} +``` + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/distribution/v1beta1/distribution.proto#L116-L123 +``` + +### Validator Distribution + +Validator distribution information for the relevant validator is updated each time: + +1. delegation amount to a validator is updated, +2. any delegator withdraws from a validator, or +3. the validator withdraws its commission. + +* ValidatorDistInfo: `0x02 | ValOperatorAddrLen (1 byte) | ValOperatorAddr -> ProtocolBuffer(validatorDistribution)` + +```go +type ValidatorDistInfo struct { + OperatorAddress sdk.AccAddress + SelfBondRewards sdkmath.DecCoins + ValidatorCommission types.ValidatorAccumulatedCommission +} +``` + +### Delegation Distribution + +Each delegation distribution only needs to record the height at which it last +withdrew fees. Because a delegation must withdraw fees each time it's +properties change (aka bonded tokens etc.) its properties will remain constant +and the delegator's *accumulation* factor can be calculated passively knowing +only the height of the last withdrawal and its current properties. + +* DelegationDistInfo: `0x02 | DelegatorAddrLen (1 byte) | DelegatorAddr | ValOperatorAddrLen (1 byte) | ValOperatorAddr -> ProtocolBuffer(delegatorDist)` + +```go +type DelegationDistInfo struct { + WithdrawalHeight int64 // last time this delegation withdrew rewards +} +``` + +### Params + +The distribution module stores its params in state with the prefix of `0x09`, +it can be updated with governance or the address with authority. + +* Params: `0x09 | ProtocolBuffer(Params)` + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/distribution/v1beta1/distribution.proto#L12-L42 +``` + +## Begin Block + +At each `BeginBlock`, all fees received in the previous block are transferred to +the distribution `ModuleAccount` account. When a delegator or validator +withdraws their rewards, they are taken out of the `ModuleAccount`. During begin +block, the different claims on the fees collected are updated as follows: + +* The reserve community tax is charged. +* The remainder is distributed proportionally by voting power to all bonded validators + +### The Distribution Scheme + +See [params](#params) for description of parameters. + +Let `fees` be the total fees collected in the previous block, including +inflationary rewards to the stake. All fees are collected in a specific module +account during the block. During `BeginBlock`, they are sent to the +`"distribution"` `ModuleAccount`. No other sending of tokens occurs. Instead, the +rewards each account is entitled to are stored, and withdrawals can be triggered +through the messages `FundCommunityPool`, `WithdrawValidatorCommission` and +`WithdrawDelegatorReward`. + +#### Reward to the Community Pool + +The community pool gets `community_tax * fees`, plus any remaining dust after +validators get their rewards that are always rounded down to the nearest +integer value. + +#### Using an External Community Pool + +Starting with Cosmos SDK v0.53.0, an external community pool, such as `x/protocolpool`, can be used in place of the `x/distribution` managed community pool. + +Please view the warning in the next section before deciding to use an external community pool. + +```go expandable +// ExternalCommunityPoolKeeper is the interface that an external community pool module keeper must fulfill +// for x/distribution to properly accept it as a community pool fund destination. +type ExternalCommunityPoolKeeper interface { + // GetCommunityPoolModule gets the module name that funds should be sent to for the community pool. + // This is the address that x/distribution will send funds to for external management. + GetCommunityPoolModule() + +string + // FundCommunityPool allows an account to directly fund the community fund pool. + FundCommunityPool(ctx sdk.Context, amount sdk.Coins, senderAddr sdk.AccAddress) + +error + // DistributeFromCommunityPool distributes funds from the community pool module account to + // a receiver address. + DistributeFromCommunityPool(ctx sdk.Context, amount sdk.Coins, receiveAddr sdk.AccAddress) + +error +} +``` + +```go +app.DistrKeeper = distrkeeper.NewKeeper( + appCodec, + runtime.NewKVStoreService(keys[distrtypes.StoreKey]), + app.AccountKeeper, + app.BankKeeper, + app.StakingKeeper, + authtypes.FeeCollectorName, + authtypes.NewModuleAddress(govtypes.ModuleName).String(), + distrkeeper.WithExternalCommunityPool(app.ProtocolPoolKeeper), // New option. +) +``` + +#### External Community Pool Usage Warning + +When using an external community pool with `x/distribution`, the following handlers will return an error: + +**QueryService** + +* `CommunityPool` + +**MsgService** + +* `CommunityPoolSpend` +* `FundCommunityPool` + +If you have services that rely on this functionality from `x/distribution`, please update them to use the `x/protocolpool` equivalents. + +#### Reward To the Validators + +The proposer receives no extra rewards. All fees are distributed among all the +bonded validators, including the proposer, in proportion to their consensus power. + +```text +powFrac = validator power / total bonded validator power +voteMul = 1 - community_tax +``` + +All validators receive `fees * voteMul * powFrac`. + +#### Rewards to Delegators + +Each validator's rewards are distributed to its delegators. The validator also +has a self-delegation that is treated like a regular delegation in +distribution calculations. + +The validator sets a commission rate. The commission rate is flexible, but each +validator sets a maximum rate and a maximum daily increase. These maximums cannot be exceeded and protect delegators from sudden increases of validator commission rates to prevent validators from taking all of the rewards. + +The outstanding rewards that the operator is entitled to are stored in +`ValidatorAccumulatedCommission`, while the rewards the delegators are entitled +to are stored in `ValidatorCurrentRewards`. The [F1 fee distribution scheme](#concepts) is used to calculate the rewards per delegator as they +withdraw or update their delegation, and is thus not handled in `BeginBlock`. + +#### Example Distribution + +For this example distribution, the underlying consensus engine selects block proposers in +proportion to their power relative to the entire bonded power. + +All validators are equally performant at including pre-commits in their proposed +blocks. Then hold `(pre_commits included) / (total bonded validator power)` +constant so that the amortized block reward for the validator is `( validator power / total bonded power) * (1 - community tax rate)` of +the total rewards. Consequently, the reward for a single delegator is: + +```text +(delegator proportion of the validator power / validator power) * (validator power / total bonded power) + * (1 - community tax rate) * (1 - validator commission rate) += (delegator proportion of the validator power / total bonded power) * (1 - +community tax rate) * (1 - validator commission rate) +``` + +## Messages + +### MsgSetWithdrawAddress + +By default, the withdraw address is the delegator address. To change its withdraw address, a delegator must send a `MsgSetWithdrawAddress` message. +Changing the withdraw address is possible only if the parameter `WithdrawAddrEnabled` is set to `true`. + +The withdraw address cannot be any of the module accounts. These accounts are blocked from being withdraw addresses by being added to the distribution keeper's `blockedAddrs` array at initialization. + +Response: + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/distribution/v1beta1/tx.proto#L49-L60 +``` + +```go +func (k Keeper) + +SetWithdrawAddr(ctx context.Context, delegatorAddr sdk.AccAddress, withdrawAddr sdk.AccAddress) + +error + if k.blockedAddrs[withdrawAddr.String()] { + fail with "`{ + withdrawAddr +}` is not allowed to receive external funds" +} + if !k.GetWithdrawAddrEnabled(ctx) { + fail with `ErrSetWithdrawAddrDisabled` +} + +k.SetDelegatorWithdrawAddr(ctx, delegatorAddr, withdrawAddr) +``` + +### MsgWithdrawDelegatorReward + +A delegator can withdraw its rewards. +Internally in the distribution module, this transaction simultaneously removes the previous delegation with associated rewards, the same as if the delegator simply started a new delegation of the same value. +The rewards are sent immediately from the distribution `ModuleAccount` to the withdraw address. +Any remainder (truncated decimals) are sent to the community pool. +The starting height of the delegation is set to the current validator period, and the reference count for the previous period is decremented. +The amount withdrawn is deducted from the `ValidatorOutstandingRewards` variable for the validator. + +In the F1 distribution, the total rewards are calculated per validator period, and a delegator receives a piece of those rewards in proportion to their stake in the validator. +In basic F1, the total rewards that all the delegators are entitled to between to periods is calculated the following way. +Let `R(X)` be the total accumulated rewards up to period `X` divided by the tokens staked at that time. The delegator allocation is `R(X) * delegator_stake`. +Then the rewards for all the delegators for staking between periods `A` and `B` are `(R(B) - R(A)) * total stake`. +However, these calculated rewards don't account for slashing. + +Taking the slashes into account requires iteration. +Let `F(X)` be the fraction a validator is to be slashed for a slashing event that happened at period `X`. +If the validator was slashed at periods `P1, ..., PN`, where `A < P1`, `PN < B`, the distribution module calculates the individual delegator's rewards, `T(A, B)`, as follows: + +```go +stake := initial stake + rewards := 0 + previous := A + for P in P1, ..., PN`: + rewards = (R(P) - previous) * stake + stake = stake * F(P) + +previous = P +rewards = rewards + (R(B) - R(PN)) * stake +``` + +The historical rewards are calculated retroactively by playing back all the slashes and then attenuating the delegator's stake at each step. +The final calculated stake is equivalent to the actual staked coins in the delegation with a margin of error due to rounding errors. + +Response: + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/distribution/v1beta1/tx.proto#L66-L77 +``` + +### WithdrawValidatorCommission + +The validator can send the WithdrawValidatorCommission message to withdraw their accumulated commission. +The commission is calculated in every block during `BeginBlock`, so no iteration is required to withdraw. +The amount withdrawn is deducted from the `ValidatorOutstandingRewards` variable for the validator. +Only integer amounts can be sent. If the accumulated awards have decimals, the amount is truncated before the withdrawal is sent, and the remainder is left to be withdrawn later. + +### FundCommunityPool + + + +This handler will return an error if an `ExternalCommunityPool` is used. + + + +This message sends coins directly from the sender to the community pool. + +The transaction fails if the amount cannot be transferred from the sender to the distribution module account. + +```go expandable +func (k Keeper) + +FundCommunityPool(ctx context.Context, amount sdk.Coins, sender sdk.AccAddress) + +error { + if err := k.bankKeeper.SendCoinsFromAccountToModule(ctx, sender, types.ModuleName, amount); err != nil { + return err +} + +feePool, err := k.FeePool.Get(ctx) + if err != nil { + return err +} + +feePool.CommunityPool = feePool.CommunityPool.Add(sdk.NewDecCoinsFromCoins(amount...)...) + if err := k.FeePool.Set(ctx, feePool); err != nil { + return err +} + +return nil +} +``` + +### Common distribution operations + +These operations take place during many different messages. + +#### Initialize delegation + +Each time a delegation is changed, the rewards are withdrawn and the delegation is reinitialized. +Initializing a delegation increments the validator period and keeps track of the starting period of the delegation. + +```go expandable +// initialize starting info for a new delegation +func (k Keeper) + +initializeDelegation(ctx context.Context, val sdk.ValAddress, del sdk.AccAddress) { + // period has already been incremented - we want to store the period ended by this delegation action + previousPeriod := k.GetValidatorCurrentRewards(ctx, val).Period - 1 + + // increment reference count for the period we're going to track + k.incrementReferenceCount(ctx, val, previousPeriod) + validator := k.stakingKeeper.Validator(ctx, val) + delegation := k.stakingKeeper.Delegation(ctx, del, val) + + // calculate delegation stake in tokens + // we don't store directly, so multiply delegation shares * (tokens per share) + // note: necessary to truncate so we don't allow withdrawing more rewards than owed + stake := validator.TokensFromSharesTruncated(delegation.GetShares()) + +k.SetDelegatorStartingInfo(ctx, val, del, types.NewDelegatorStartingInfo(previousPeriod, stake, uint64(ctx.BlockHeight()))) +} +``` + +### MsgUpdateParams + +Distribution module params can be updated through `MsgUpdateParams`, which can be done using governance proposal and the signer will always be gov module account address. + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/distribution/v1beta1/tx.proto#L133-L147 +``` + +The message handling can fail if: + +* signer is not the gov module account address. + +## Hooks + +Available hooks that can be called by and from this module. + +### Create or modify delegation distribution + +* triggered-by: `staking.MsgDelegate`, `staking.MsgBeginRedelegate`, `staking.MsgUndelegate` + +#### Before + +* The delegation rewards are withdrawn to the withdraw address of the delegator. + The rewards include the current period and exclude the starting period. +* The validator period is incremented. + The validator period is incremented because the validator's power and share distribution might have changed. +* The reference count for the delegator's starting period is decremented. + +#### After + +The starting height of the delegation is set to the previous period. +Because of the `Before`-hook, this period is the last period for which the delegator was rewarded. + +### Validator created + +* triggered-by: `staking.MsgCreateValidator` + +When a validator is created, the following validator variables are initialized: + +* Historical rewards +* Current accumulated rewards +* Accumulated commission +* Total outstanding rewards +* Period + +By default, all values are set to a `0`, except period, which is set to `1`. + +### Validator removed + +* triggered-by: `staking.RemoveValidator` + +Outstanding commission is sent to the validator's self-delegation withdrawal address. +Remaining delegator rewards get sent to the community fee pool. + +Note: The validator gets removed only when it has no remaining delegations. +At that time, all outstanding delegator rewards will have been withdrawn. +Any remaining rewards are dust amounts. + +### Validator is slashed + +* triggered-by: `staking.Slash` +* The current validator period reference count is incremented. + The reference count is incremented because the slash event has created a reference to it. +* The validator period is incremented. +* The slash event is stored for later use. + The slash event will be referenced when calculating delegator rewards. + +## Events + +The distribution module emits the following events: + +### BeginBlocker + +| Type | Attribute Key | Attribute Value | +| ---------------- | ------------- | ------------------ | +| proposer\_reward | validator | `{validatorAddress}` | +| proposer\_reward | reward | `{proposerReward}` | +| commission | amount | `{commissionAmount}` | +| commission | validator | `{validatorAddress}` | +| rewards | amount | `{rewardAmount}` | +| rewards | validator | `{validatorAddress}` | + +### Handlers + +#### MsgSetWithdrawAddress + +| Type | Attribute Key | Attribute Value | +| ---------------------- | ----------------- | ---------------------- | +| set\_withdraw\_address | withdraw\_address | `{withdrawAddress}` | +| message | module | distribution | +| message | action | set\_withdraw\_address | +| message | sender | `{senderAddress}` | + +#### MsgWithdrawDelegatorReward + +| Type | Attribute Key | Attribute Value | +| ----------------- | ------------- | --------------------------- | +| withdraw\_rewards | amount | `{rewardAmount}` | +| withdraw\_rewards | validator | `{validatorAddress}` | +| message | module | distribution | +| message | action | withdraw\_delegator\_reward | +| message | sender | `{senderAddress}` | + +#### MsgWithdrawValidatorCommission + +| Type | Attribute Key | Attribute Value | +| -------------------- | ------------- | ------------------------------- | +| withdraw\_commission | amount | `{commissionAmount}` | +| message | module | distribution | +| message | action | withdraw\_validator\_commission | +| message | sender | `{senderAddress}` | + +## Parameters + +The distribution module contains the following parameters: + +| Key | Type | Example | +| ------------------- | ------------ | --------------------------- | +| communitytax | string (dec) | "0.020000000000000000" \[0] | +| withdrawaddrenabled | bool | true | + +* \[0] `communitytax` must be positive and cannot exceed 1.00. +* `baseproposerreward` and `bonusproposerreward` were parameters that are deprecated in v0.47 and are not used. + + +The reserve pool is the pool of collected funds for use by governance taken via the `CommunityTax`. +Currently with the Cosmos SDK, tokens collected by the CommunityTax are accounted for but unspendable. + + +## Client + +## CLI + +A user can query and interact with the `distribution` module using the CLI. + +#### Query + +The `query` commands allow users to query `distribution` state. + +```shell +simd query distribution --help +``` + +##### commission + +The `commission` command allows users to query validator commission rewards by address. + +```shell +simd query distribution commission [address] [flags] +``` + +Example: + +```shell +simd query distribution commission cosmosvaloper1... +``` + +Example Output: + +```yml +commission: +- amount: "1000000.000000000000000000" + denom: stake +``` + +##### community-pool + +The `community-pool` command allows users to query all coin balances within the community pool. + +```shell +simd query distribution community-pool [flags] +``` + +Example: + +```shell +simd query distribution community-pool +``` + +Example Output: + +```yml +pool: +- amount: "1000000.000000000000000000" + denom: stake +``` + +##### params + +The `params` command allows users to query the parameters of the `distribution` module. + +```shell +simd query distribution params [flags] +``` + +Example: + +```shell +simd query distribution params +``` + +Example Output: + +```yml +base_proposer_reward: "0.000000000000000000" +bonus_proposer_reward: "0.000000000000000000" +community_tax: "0.020000000000000000" +withdraw_addr_enabled: true +``` + +##### rewards + +The `rewards` command allows users to query delegator rewards. Users can optionally include the validator address to query rewards earned from a specific validator. + +```shell +simd query distribution rewards [delegator-addr] [validator-addr] [flags] +``` + +Example: + +```shell +simd query distribution rewards cosmos1... +``` + +Example Output: + +```yml +rewards: +- reward: + - amount: "1000000.000000000000000000" + denom: stake + validator_address: cosmosvaloper1.. +total: +- amount: "1000000.000000000000000000" + denom: stake +``` + +##### slashes + +The `slashes` command allows users to query all slashes for a given block range. + +```shell +simd query distribution slashes [validator] [start-height] [end-height] [flags] +``` + +Example: + +```shell +simd query distribution slashes cosmosvaloper1... 1 1000 +``` + +Example Output: + +```yml +pagination: + next_key: null + total: "0" +slashes: +- validator_period: 20, + fraction: "0.009999999999999999" +``` + +##### validator-outstanding-rewards + +The `validator-outstanding-rewards` command allows users to query all outstanding (un-withdrawn) rewards for a validator and all their delegations. + +```shell +simd query distribution validator-outstanding-rewards [validator] [flags] +``` + +Example: + +```shell +simd query distribution validator-outstanding-rewards cosmosvaloper1... +``` + +Example Output: + +```yml +rewards: +- amount: "1000000.000000000000000000" + denom: stake +``` + +##### validator-distribution-info + +The `validator-distribution-info` command allows users to query validator commission and self-delegation rewards for validator. + +```shell expandable +simd query distribution validator-distribution-info cosmosvaloper1... +``` + +Example Output: + +```yml +commission: +- amount: "100000.000000000000000000" + denom: stake +operator_address: cosmosvaloper1... +self_bond_rewards: +- amount: "100000.000000000000000000" + denom: stake +``` + +##### validator-historical-rewards + +The `validator-historical-rewards` command allows users to query historical rewards for a validator at a specific period. + +```shell +simd query distribution validator-historical-rewards [validator] [period] [flags] +``` + +Example: + +```shell +simd query distribution validator-historical-rewards cosmosvaloper1... 5 +``` + +Example Output: + +```yml +rewards: + cumulative_reward_ratio: + - amount: "1000000.000000000000000000" + denom: stake + reference_count: 2 +``` + +##### validator-current-rewards + +The `validator-current-rewards` command allows users to query current rewards for a validator. + +```shell +simd query distribution validator-current-rewards [validator] [flags] +``` + +Example: + +```shell +simd query distribution validator-current-rewards cosmosvaloper1... +``` + +Example Output: + +```yml +rewards: + period: "3" + rewards: + - amount: "1000000.000000000000000000" + denom: stake +``` + +##### delegator-starting-info + +The `delegator-starting-info` command allows users to query the starting info for a delegator on a given validator. + +```shell +simd query distribution delegator-starting-info [delegator-address] [validator-address] [flags] +``` + +Example: + +```shell +simd query distribution delegator-starting-info cosmos1... cosmosvaloper1... +``` + +Example Output: + +```yml +starting_info: + creation_height: "10" + previous_period: "2" + stake: "1000000.000000000000000000" +``` + +#### Transactions + +The `tx` commands allow users to interact with the `distribution` module. + +```shell +simd tx distribution --help +``` + +##### fund-community-pool + +The `fund-community-pool` command allows users to send funds to the community pool. + +```shell +simd tx distribution fund-community-pool [amount] [flags] +``` + +Example: + +```shell +simd tx distribution fund-community-pool 100stake --from cosmos1... +``` + +##### set-withdraw-addr + +The `set-withdraw-addr` command allows users to set the withdraw address for rewards associated with a delegator address. + +```shell +simd tx distribution set-withdraw-addr [withdraw-addr] [flags] +``` + +Example: + +```shell +simd tx distribution set-withdraw-addr cosmos1... --from cosmos1... +``` + +##### withdraw-all-rewards + +The `withdraw-all-rewards` command allows users to withdraw all rewards for a delegator. + +```shell +simd tx distribution withdraw-all-rewards [flags] +``` + +Example: + +```shell +simd tx distribution withdraw-all-rewards --from cosmos1... +``` + +##### withdraw-rewards + +The `withdraw-rewards` command allows users to withdraw all rewards from a given delegation address, +and optionally withdraw validator commission if the delegation address given is a validator operator and the user proves the `--commission` flag. + +```shell +simd tx distribution withdraw-rewards [validator-addr] [flags] +``` + +Example: + +```shell +simd tx distribution withdraw-rewards cosmosvaloper1... --from cosmos1... --commission +``` + +### gRPC + +A user can query the `distribution` module using gRPC endpoints. + +#### Params + +The `Params` endpoint allows users to query parameters of the `distribution` module. + +Example: + +```shell +grpcurl -plaintext \ + localhost:9090 \ + cosmos.distribution.v1beta1.Query/Params +``` + +Example Output: + +```json +{ + "params": { + "communityTax": "20000000000000000", + "baseProposerReward": "00000000000000000", + "bonusProposerReward": "00000000000000000", + "withdrawAddrEnabled": true + } +} +``` + +#### ValidatorDistributionInfo + +The `ValidatorDistributionInfo` queries validator commission and self-delegation rewards for validator. + +Example: + +```shell +grpcurl -plaintext \ + -d '{"validator_address":"cosmosvalop1..."}' \ + localhost:9090 \ + cosmos.distribution.v1beta1.Query/ValidatorDistributionInfo +``` + +Example Output: + +```json +{ + "commission": { + "commission": [ + { + "denom": "stake", + "amount": "1000000000000000" + } + ] + }, + "self_bond_rewards": [ + { + "denom": "stake", + "amount": "1000000000000000" + } + ], + "validator_address": "cosmosvalop1..." +} +``` + +#### ValidatorOutstandingRewards + +The `ValidatorOutstandingRewards` endpoint allows users to query rewards of a validator address. + +Example: + +```shell +grpcurl -plaintext \ + -d '{"validator_address":"cosmosvalop1.."}' \ + localhost:9090 \ + cosmos.distribution.v1beta1.Query/ValidatorOutstandingRewards +``` + +Example Output: + +```json +{ + "rewards": { + "rewards": [ + { + "denom": "stake", + "amount": "1000000000000000" + } + ] + } +} +``` + +#### ValidatorCommission + +The `ValidatorCommission` endpoint allows users to query accumulated commission for a validator. + +Example: + +```shell +grpcurl -plaintext \ + -d '{"validator_address":"cosmosvalop1.."}' \ + localhost:9090 \ + cosmos.distribution.v1beta1.Query/ValidatorCommission +``` + +Example Output: + +```json +{ + "commission": { + "commission": [ + { + "denom": "stake", + "amount": "1000000000000000" + } + ] + } +} +``` + +#### ValidatorSlashes + +The `ValidatorSlashes` endpoint allows users to query slash events of a validator. + +Example: + +```shell +grpcurl -plaintext \ + -d '{"validator_address":"cosmosvalop1.."}' \ + localhost:9090 \ + cosmos.distribution.v1beta1.Query/ValidatorSlashes +``` + +Example Output: + +```json +{ + "slashes": [ + { + "validator_period": "20", + "fraction": "0.009999999999999999" + } + ], + "pagination": { + "total": "1" + } +} +``` + +#### DelegationRewards + +The `DelegationRewards` endpoint allows users to query the total rewards accrued by a delegation. + +Example: + +```shell +grpcurl -plaintext \ + -d '{"delegator_address":"cosmos1...","validator_address":"cosmosvalop1..."}' \ + localhost:9090 \ + cosmos.distribution.v1beta1.Query/DelegationRewards +``` + +Example Output: + +```json +{ + "rewards": [ + { + "denom": "stake", + "amount": "1000000000000000" + } + ] +} +``` + +#### DelegationTotalRewards + +The `DelegationTotalRewards` endpoint allows users to query the total rewards accrued by each validator. + +Example: + +```shell +grpcurl -plaintext \ + -d '{"delegator_address":"cosmos1..."}' \ + localhost:9090 \ + cosmos.distribution.v1beta1.Query/DelegationTotalRewards +``` + +Example Output: + +```json +{ + "rewards": [ + { + "validatorAddress": "cosmosvaloper1...", + "reward": [ + { + "denom": "stake", + "amount": "1000000000000000" + } + ] + } + ], + "total": [ + { + "denom": "stake", + "amount": "1000000000000000" + } + ] +} +``` + +#### DelegatorValidators + +The `DelegatorValidators` endpoint allows users to query all validators for given delegator. + +Example: + +```shell +grpcurl -plaintext \ + -d '{"delegator_address":"cosmos1..."}' \ + localhost:9090 \ + cosmos.distribution.v1beta1.Query/DelegatorValidators +``` + +Example Output: + +```json +{ + "validators": ["cosmosvaloper1..."] +} +``` + +#### DelegatorWithdrawAddress + +The `DelegatorWithdrawAddress` endpoint allows users to query the withdraw address of a delegator. + +Example: + +```shell +grpcurl -plaintext \ + -d '{"delegator_address":"cosmos1..."}' \ + localhost:9090 \ + cosmos.distribution.v1beta1.Query/DelegatorWithdrawAddress +``` + +Example Output: + +```json +{ + "withdrawAddress": "cosmos1..." +} +``` + +#### CommunityPool + +The `CommunityPool` endpoint allows users to query the community pool coins. + +Example: + +```shell +grpcurl -plaintext \ + localhost:9090 \ + cosmos.distribution.v1beta1.Query/CommunityPool +``` + +Example Output: + +```json +{ + "pool": [ + { + "denom": "stake", + "amount": "1000000000000000000" + } + ] +} +``` + +#### ValidatorHistoricalRewards + +The `ValidatorHistoricalRewards` endpoint allows users to query historical rewards for a validator at a specific period. This is useful for debugging reward calculations by inspecting internal distribution state. + +Example: + +```shell +grpcurl -plaintext \ + -d '{"validator_address":"cosmosvaloper1...","period":"5"}' \ + localhost:9090 \ + cosmos.distribution.v1beta1.Query/ValidatorHistoricalRewards +``` + +Example Output: + +```json +{ + "rewards": { + "cumulativeRewardRatio": [ + { + "denom": "stake", + "amount": "1000000000000000" + } + ], + "referenceCount": 2 + } +} +``` + +#### ValidatorCurrentRewards + +The `ValidatorCurrentRewards` endpoint allows users to query current rewards for a validator. + +Example: + +```shell +grpcurl -plaintext \ + -d '{"validator_address":"cosmosvaloper1..."}' \ + localhost:9090 \ + cosmos.distribution.v1beta1.Query/ValidatorCurrentRewards +``` + +Example Output: + +```json +{ + "rewards": { + "rewards": [ + { + "denom": "stake", + "amount": "1000000000000000" + } + ], + "period": "3" + } +} +``` + +#### DelegatorStartingInfo + +The `DelegatorStartingInfo` endpoint allows users to query the starting info for a delegator on a given validator. Combined with `ValidatorHistoricalRewards`, this enables verification of reward calculations by retrieving the previous period and stake, then looking up cumulative reward ratios for that period. + +Example: + +```shell +grpcurl -plaintext \ + -d '{"delegator_address":"cosmos1...","validator_address":"cosmosvaloper1..."}' \ + localhost:9090 \ + cosmos.distribution.v1beta1.Query/DelegatorStartingInfo +``` + +Example Output: + +```json +{ + "startingInfo": { + "previousPeriod": "2", + "stake": "1000000000000000000", + "creationHeight": "10" + } +} +``` +```` diff --git a/sdk/v0.54/modules/epochs/README.mdx b/sdk/v0.54/modules/epochs/README.mdx new file mode 100644 index 000000000..151141ba9 --- /dev/null +++ b/sdk/v0.54/modules/epochs/README.mdx @@ -0,0 +1,181 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/modules/epochs/README' +title: 'x/epochs' +--- + +## Abstract + +Often in the SDK, we would like to run certain code every-so often. The +purpose of `epochs` module is to allow other modules to set that they +would like to be signaled once every period. So another module can +specify it wants to execute code once a week, starting at UTC-time = x. +`epochs` creates a generalized epoch interface to other modules so that +they can easily be signaled upon such events. + +## Contents + +1. **[Concept](#concepts)** +2. **[State](#state)** +3. **[Events](#events)** +4. **[Keeper](#keepers)** +5. **[Hooks](#hooks)** +6. **[Queries](#queries)** + +## Concepts + +The epochs module defines on-chain timers that execute at fixed time intervals. +Other SDK modules can then register logic to be executed at the timer ticks. +We refer to the period in between two timer ticks as an "epoch". + +Every timer has a unique identifier. +Every epoch will have a start time, and an end time, where `end time = start time + timer interval`. +On mainnet, we only utilize one identifier, with a time interval of `one day`. + +The timer will tick at the first block whose block time is greater than the timer end time, +and set the start as the prior timer end time. (Notably, it's not set to the block time!) +This means that if the chain has been down for a while, you will get one timer tick per block, +until the timer has caught up. + +## State + +The Epochs module keeps a single `EpochInfo` per identifier. +This contains the current state of the timer with the corresponding identifier. +Its fields are modified at every timer tick. +EpochInfos are initialized as part of genesis initialization or upgrade logic, +and are only modified on begin blockers. + +## Events + +The `epochs` module emits the following events: + +### BeginBlocker + +| Type | Attribute Key | Attribute Value | +| ------------ | ------------- | --------------- | +| epoch\_start | epoch\_number | `{epoch\_number}` | +| epoch\_start | start\_time | `{start\_time}` | + +### EndBlocker + +| Type | Attribute Key | Attribute Value | +| ---------- | ------------- | --------------- | +| epoch\_end | epoch\_number | `{epoch\_number}` | + +## Keepers + +### Keeper functions + +Epochs keeper module provides utility functions to manage epochs. + +## Hooks + +```go +// the first block whose timestamp is after the duration is counted as the end of the epoch + AfterEpochEnd(ctx sdk.Context, epochIdentifier string, epochNumber int64) + // new epoch is next block of epoch end block + BeforeEpochStart(ctx sdk.Context, epochIdentifier string, epochNumber int64) +``` + +### How modules receive hooks + +On hook receiver function of other modules, they need to filter +`epochIdentifier` and only do executions for only specific +epochIdentifier. Filtering epochIdentifier could be in `Params` of other +modules so that they can be modified by governance. + +This is the standard dev UX of this: + +```golang +func (k MyModuleKeeper) + +AfterEpochEnd(ctx sdk.Context, epochIdentifier string, epochNumber int64) { + params := k.GetParams(ctx) + if epochIdentifier == params.DistrEpochIdentifier { + // my logic +} +} +``` + +### Panic isolation + +If a given epoch hook panics, its state update is reverted, but we keep +proceeding through the remaining hooks. This allows more advanced epoch +logic to be used, without concern over state machine halting, or halting +subsequent modules. + +This does mean that if there is behavior you expect from a prior epoch +hook, and that epoch hook reverted, your hook may also have an issue. So +do keep in mind "what if a prior hook didn't get executed" in the safety +checks you consider for a new epoch hook. + +## Queries + +The Epochs module provides the following queries to check the module's state. + +```protobuf +service Query { + // EpochInfos provide running epochInfos + rpc EpochInfos(QueryEpochsInfoRequest) returns (QueryEpochsInfoResponse) {} + // CurrentEpoch provide current epoch of specified identifier + rpc CurrentEpoch(QueryCurrentEpochRequest) returns (QueryCurrentEpochResponse) {} +} +``` + +### Epoch Infos + +Query the currently running epochInfos + +```sh + query epochs epoch-infos +``` + + +**Example** + +An example output: + +```sh expandable +epochs: +- current_epoch: "183" + current_epoch_start_height: "2438409" + current_epoch_start_time: "2021-12-18T17:16:09.898160996Z" + duration: 86400s + epoch_counting_started: true + identifier: day + start_time: "2021-06-18T17:00:00Z" +- current_epoch: "26" + current_epoch_start_height: "2424854" + current_epoch_start_time: "2021-12-17T17:02:07.229632445Z" + duration: 604800s + epoch_counting_started: true + identifier: week + start_time: "2021-06-18T17:00:00Z" +``` + + + +### Current Epoch + +Query the current epoch by the specified identifier + +```sh + query epochs current-epoch [identifier] +``` + + +**Example** + +Query the current `day` epoch: + +```sh + query epochs current-epoch day +``` + +Which in this example outputs: + +```sh +current_epoch: "183" +``` + + diff --git a/sdk/v0.54/modules/evidence/README.mdx b/sdk/v0.54/modules/evidence/README.mdx new file mode 100644 index 000000000..0db81d214 --- /dev/null +++ b/sdk/v0.54/modules/evidence/README.mdx @@ -0,0 +1,463 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/modules/evidence/README' +title: 'x/evidence' +description: Concepts State Messages Events Parameters BeginBlock Client CLI REST gRPC +--- + +* [Concepts](#concepts) +* [State](#state) +* [Messages](#messages) +* [Events](#events) +* [Parameters](#parameters) +* [BeginBlock](#beginblock) +* [Client](#client) + * [CLI](#cli) + * [REST](#rest) + * [gRPC](#grpc) + +## Abstract + +`x/evidence` is an implementation of a Cosmos SDK module, per [ADR 009](https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-009-evidence-module.md), +that allows for the submission and handling of arbitrary evidence of misbehavior such +as equivocation and counterfactual signing. + +The evidence module differs from standard evidence handling which typically expects the +underlying consensus engine, e.g. CometBFT, to automatically submit evidence when +it is discovered by allowing clients and foreign chains to submit more complex evidence +directly. + +All concrete evidence types must implement the `Evidence` interface contract. Submitted +`Evidence` is first routed through the evidence module's `Router` in which it attempts +to find a corresponding registered `Handler` for that specific `Evidence` type. +Each `Evidence` type must have a `Handler` registered with the evidence module's +keeper in order for it to be successfully routed and executed. + +Each corresponding handler must also fulfill the `Handler` interface contract. The +`Handler` for a given `Evidence` type can perform any arbitrary state transitions +such as slashing, jailing, and tombstoning. + +## Concepts + +### Evidence + +Any concrete type of evidence submitted to the `x/evidence` module must fulfill the +`Evidence` contract outlined below. Not all concrete types of evidence will fulfill +this contract in the same way and some data may be entirely irrelevant to certain +types of evidence. An additional `ValidatorEvidence`, which extends `Evidence`, +has also been created to define a contract for evidence against malicious validators. + +```go expandable +// Evidence defines the contract which concrete evidence types of misbehavior +// must implement. +type Evidence interface { + proto.Message + + Route() + +string + String() + +string + Hash() []byte + ValidateBasic() + +error + + // Height at which the infraction occurred + GetHeight() + +int64 +} + +// ValidatorEvidence extends Evidence interface to define contract +// for evidence against malicious validators +type ValidatorEvidence interface { + Evidence + + // The consensus address of the malicious validator at time of infraction + GetConsensusAddress() + +sdk.ConsAddress + + // The total power of the malicious validator at time of infraction + GetValidatorPower() + +int64 + + // The total validator set power at time of infraction + GetTotalPower() + +int64 +} +``` + +### Registration & Handling + +The `x/evidence` module must first know about all types of evidence it is expected +to handle. This is accomplished by registering the `Route` method in the `Evidence` +contract with what is known as a `Router` (defined below). The `Router` accepts +`Evidence` and attempts to find the corresponding `Handler` for the `Evidence` +via the `Route` method. + +```go +type Router interface { + AddRoute(r string, h Handler) + +Router + HasRoute(r string) + +bool + GetRoute(path string) + +Handler + Seal() + +Sealed() + +bool +} +``` + +The `Handler` (defined below) is responsible for executing the entirety of the +business logic for handling `Evidence`. This typically includes validating the +evidence, both stateless checks via `ValidateBasic` and stateful checks via any +keepers provided to the `Handler`. In addition, the `Handler` may also perform +capabilities such as slashing and jailing a validator. All `Evidence` handled +by the `Handler` should be persisted. + +```go +// Handler defines an agnostic Evidence handler. The handler is responsible +// for executing all corresponding business logic necessary for verifying the +// evidence as valid. In addition, the Handler may execute any necessary +// slashing and potential jailing. +type Handler func(context.Context, Evidence) + +error +``` + +## State + +Currently the `x/evidence` module only stores valid submitted `Evidence` in state. +The evidence state is also stored and exported in the `x/evidence` module's `GenesisState`. + +```protobuf +// GenesisState defines the evidence module's genesis state. +message GenesisState { + // evidence defines all the evidence at genesis. + repeated google.protobuf.Any evidence = 1; +} + +``` + +All `Evidence` is retrieved and stored via a prefix `KVStore` using prefix `0x00` (`KeyPrefixEvidence`). + +## Messages + +### MsgSubmitEvidence + +Evidence is submitted through a `MsgSubmitEvidence` message: + +```protobuf +// MsgSubmitEvidence represents a message that supports submitting arbitrary +// Evidence of misbehavior such as equivocation or counterfactual signing. +message MsgSubmitEvidence { + string submitter = 1; + google.protobuf.Any evidence = 2; +} +``` + +Note, the `Evidence` of a `MsgSubmitEvidence` message must have a corresponding +`Handler` registered with the `x/evidence` module's `Router` in order to be processed +and routed correctly. + +Given the `Evidence` is registered with a corresponding `Handler`, it is processed +as follows: + +```go expandable +func SubmitEvidence(ctx Context, evidence Evidence) + +error { + if _, err := GetEvidence(ctx, evidence.Hash()); err == nil { + return errorsmod.Wrap(types.ErrEvidenceExists, strings.ToUpper(hex.EncodeToString(evidence.Hash()))) +} + if !router.HasRoute(evidence.Route()) { + return errorsmod.Wrap(types.ErrNoEvidenceHandlerExists, evidence.Route()) +} + handler := router.GetRoute(evidence.Route()) + if err := handler(ctx, evidence); err != nil { + return errorsmod.Wrap(types.ErrInvalidEvidence, err.Error()) +} + +ctx.EventManager().EmitEvent( + sdk.NewEvent( + types.EventTypeSubmitEvidence, + sdk.NewAttribute(types.AttributeKeyEvidenceHash, strings.ToUpper(hex.EncodeToString(evidence.Hash()))), + ), + ) + +SetEvidence(ctx, evidence) + +return nil +} +``` + +First, there must not already exist valid submitted `Evidence` of the exact same +type. Secondly, the `Evidence` is routed to the `Handler` and executed. Finally, +if there is no error in handling the `Evidence`, an event is emitted and it is persisted to state. + +## Events + +The `x/evidence` module emits the following events: + +### Handlers + +#### MsgSubmitEvidence + +| Type | Attribute Key | Attribute Value | +| ---------------- | -------------- | ---------------- | +| submit\_evidence | evidence\_hash | `{evidenceHash}` | +| message | module | evidence | +| message | sender | `{senderAddress}` | +| message | action | submit\_evidence | + +## Parameters + +The evidence module does not contain any parameters. + +## BeginBlock + +### Evidence Handling + +CometBFT blocks can include +[Evidence](https://github.com/cometbft/cometbft/blob/main/spec/abci/abci%2B%2B_basic_concepts.md#evidence) that indicates if a validator committed malicious behavior. The relevant information is forwarded to the application as ABCI Evidence in `abci.RequestBeginBlock` so that the validator can be punished accordingly. + +#### Equivocation + +The Cosmos SDK handles two types of evidence inside the ABCI `BeginBlock`: + +* `DuplicateVoteEvidence`, +* `LightClientAttackEvidence`. + +The evidence module handles these two evidence types the same way. First, the Cosmos SDK converts the CometBFT concrete evidence type to an SDK `Evidence` interface using `Equivocation` as the concrete type. + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/proto/cosmos/evidence/v1beta1/evidence.proto#L12-L32 +``` + +For some `Equivocation` submitted in `block` to be valid, it must satisfy: + +`Evidence.Timestamp >= block.Timestamp - MaxEvidenceAge` + +Where: + +* `Evidence.Timestamp` is the timestamp in the block at height `Evidence.Height` +* `block.Timestamp` is the current block timestamp. + +If valid `Equivocation` evidence is included in a block, the validator's stake is +reduced (slashed) by `SlashFractionDoubleSign` as defined by the `x/slashing` module +of what their stake was when the infraction occurred, rather than when the evidence was discovered. +We want to "follow the stake", i.e., the stake that contributed to the infraction +should be slashed, even if it has since been redelegated or started unbonding. + +In addition, the validator is permanently jailed and tombstoned to make it impossible for that +validator to ever re-enter the validator set. + +The `Equivocation` evidence is handled as follows: + +```go +// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/x/evidence/keeper/infraction.go#L26-L140 +``` + +**Note:** The slashing, jailing, and tombstoning calls are delegated through the `x/slashing` module +that emits informative events and finally delegates calls to the `x/staking` module. See documentation +on slashing and jailing in [State Transitions](/sdk/v0.54/modules/staking/README#state-transitions). + +## Client + +### CLI + +A user can query and interact with the `evidence` module using the CLI. + +#### Query + +The `query` command allows users to query `evidence` state. + +```bash +simd query evidence --help +``` + +#### evidence + +The `evidence` command allows users to list all evidence or evidence by hash. + +Usage: + +```bash +simd query evidence evidence [flags] +``` + +To query evidence by hash + +Example: + +```bash +simd query evidence evidence "DF0C23E8634E480F84B9D5674A7CDC9816466DEC28A3358F73260F68D28D7660" +``` + +Example Output: + +```bash +evidence: + consensus_address: cosmosvalcons1ntk8eualewuprz0gamh8hnvcem2nrcdsgz563h + height: 11 + power: 100 + time: "2021-10-20T16:08:38.194017624Z" +``` + +To get all evidence + +Example: + +```bash +simd query evidence list +``` + +Example Output: + +```bash +evidence: + consensus_address: cosmosvalcons1ntk8eualewuprz0gamh8hnvcem2nrcdsgz563h + height: 11 + power: 100 + time: "2021-10-20T16:08:38.194017624Z" +pagination: + next_key: null + total: "1" +``` + +### REST + +A user can query the `evidence` module using REST endpoints. + +#### Evidence + +Get evidence by hash + +```bash +/cosmos/evidence/v1beta1/evidence/{hash} +``` + +Example: + +```bash +curl -X GET "http://localhost:1317/cosmos/evidence/v1beta1/evidence/DF0C23E8634E480F84B9D5674A7CDC9816466DEC28A3358F73260F68D28D7660" +``` + +Example Output: + +```bash +{ + "evidence": { + "consensus_address": "cosmosvalcons1ntk8eualewuprz0gamh8hnvcem2nrcdsgz563h", + "height": "11", + "power": "100", + "time": "2021-10-20T16:08:38.194017624Z" + } +} +``` + +#### All evidence + +Get all evidence + +```bash +/cosmos/evidence/v1beta1/evidence +``` + +Example: + +```bash +curl -X GET "http://localhost:1317/cosmos/evidence/v1beta1/evidence" +``` + +Example Output: + +```bash expandable +{ + "evidence": [ + { + "consensus_address": "cosmosvalcons1ntk8eualewuprz0gamh8hnvcem2nrcdsgz563h", + "height": "11", + "power": "100", + "time": "2021-10-20T16:08:38.194017624Z" + } + ], + "pagination": { + "total": "1" + } +} +``` + +### gRPC + +A user can query the `evidence` module using gRPC endpoints. + +#### Evidence + +Get evidence by hash + +```bash +cosmos.evidence.v1beta1.Query/Evidence +``` + +Example: + +```bash +grpcurl -plaintext -d '{"evidence_hash":"DF0C23E8634E480F84B9D5674A7CDC9816466DEC28A3358F73260F68D28D7660"}' localhost:9090 cosmos.evidence.v1beta1.Query/Evidence +``` + +Example Output: + +```bash +{ + "evidence": { + "consensus_address": "cosmosvalcons1ntk8eualewuprz0gamh8hnvcem2nrcdsgz563h", + "height": "11", + "power": "100", + "time": "2021-10-20T16:08:38.194017624Z" + } +} +``` + +#### All evidence + +Get all evidence + +```bash +cosmos.evidence.v1beta1.Query/AllEvidence +``` + +Example: + +```bash +grpcurl -plaintext localhost:9090 cosmos.evidence.v1beta1.Query/AllEvidence +``` + +Example Output: + +```bash expandable +{ + "evidence": [ + { + "consensus_address": "cosmosvalcons1ntk8eualewuprz0gamh8hnvcem2nrcdsgz563h", + "height": "11", + "power": "100", + "time": "2021-10-20T16:08:38.194017624Z" + } + ], + "pagination": { + "total": "1" + } +} +``` diff --git a/sdk/v0.54/modules/feegrant/README.mdx b/sdk/v0.54/modules/feegrant/README.mdx new file mode 100644 index 000000000..c77835d4c --- /dev/null +++ b/sdk/v0.54/modules/feegrant/README.mdx @@ -0,0 +1,3657 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/modules/feegrant/README' +title: 'x/feegrant' +description: >- + This document specifies the fee grant module. For the full ADR, please see Fee + Grant ADR-029. +--- + +## Abstract + +This document specifies the fee grant module. For the full ADR, please see [Fee Grant ADR-029](https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-029-fee-grant-module.md). + +This module allows accounts to grant fee allowances and to use fees from their accounts. Grantees can execute any transaction without the need to maintain sufficient fees. + +## Contents + +* [Concepts](#concepts) +* [State](#state) + * [FeeAllowance](#feeallowance) + * [FeeAllowanceQueue](#feeallowancequeue) +* [Messages](#messages) + * [Msg/GrantAllowance](#msggrantallowance) + * [Msg/RevokeAllowance](#msgrevokeallowance) +* [Events](#events) +* [Msg Server](#msg-server) + * [MsgGrantAllowance](#msggrantallowance-1) + * [MsgRevokeAllowance](#msgrevokeallowance-1) + * [Exec fee allowance](#exec-fee-allowance) +* [Client](#client) + * [CLI](#cli) + * [gRPC](#grpc) + +## Concepts + +### Grant + +`Grant` is stored in the KVStore to record a grant with full context. Every grant will contain `granter`, `grantee` and what kind of `allowance` is granted. `granter` is an account address who is giving permission to `grantee` (the beneficiary account address) to pay for some or all of `grantee`'s transaction fees. `allowance` defines what kind of fee allowance (`BasicAllowance` or `PeriodicAllowance`, see below) is granted to `grantee`. `allowance` accepts an interface which implements `FeeAllowanceI`, encoded as `Any` type. There can be only one existing fee grant allowed for a `grantee` and `granter`, self grants are not allowed. + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/feegrant/v1beta1/feegrant.proto#L83-L93 +``` + +`FeeAllowanceI` looks like: + +```go expandable +package feegrant + +import ( + + "time" + + sdk "github.com/cosmos/cosmos-sdk/types" +) + +// FeeAllowance implementations are tied to a given fee delegator and delegatee, +// and are used to enforce fee grant limits. +type FeeAllowanceI interface { + // Accept can use fee payment requested as well as timestamp of the current block + // to determine whether or not to process this. This is checked in + // Keeper.UseGrantedFees and the return values should match how it is handled there. + // + // If it returns an error, the fee payment is rejected, otherwise it is accepted. + // The FeeAllowance implementation is expected to update its internal state + // and will be saved again after an acceptance. + // + // If remove is true (regardless of the error), the FeeAllowance will be deleted from storage + // (eg. when it is used up). (See call to RevokeAllowance in Keeper.UseGrantedFees) + +Accept(ctx sdk.Context, fee sdk.Coins, msgs []sdk.Msg) (remove bool, err error) + + // ValidateBasic should evaluate this FeeAllowance for internal consistency. + // Don't allow negative amounts, or negative periods for example. + ValidateBasic() + +error + + // ExpiresAt returns the expiry time of the allowance. + ExpiresAt() (*time.Time, error) +} +``` + +### Fee Allowance types + +There are two types of fee allowances present at the moment: + +* `BasicAllowance` +* `PeriodicAllowance` +* `AllowedMsgAllowance` + +### BasicAllowance + +`BasicAllowance` is permission for `grantee` to use fee from a `granter`'s account. If any of the `spend_limit` or `expiration` reaches its limit, the grant will be removed from the state. + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/feegrant/v1beta1/feegrant.proto#L15-L28 +``` + +* `spend_limit` is the limit of coins that are allowed to be used from the `granter` account. If it is empty, it assumes there's no spend limit, `grantee` can use any number of available coins from `granter` account address before the expiration. + +* `expiration` specifies an optional time when this allowance expires. If the value is left empty, there is no expiry for the grant. + +* When a grant is created with empty values for `spend_limit` and `expiration`, it is still a valid grant. It won't restrict the `grantee` to use any number of coins from `granter` and it won't have any expiration. The only way to restrict the `grantee` is by revoking the grant. + +### PeriodicAllowance + +`PeriodicAllowance` is a repeating fee allowance for the mentioned period, we can mention when the grant can expire as well as when a period can reset. We can also define the maximum number of coins that can be used in a mentioned period of time. + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/feegrant/v1beta1/feegrant.proto#L34-L68 +``` + +* `basic` is the instance of `BasicAllowance` which is optional for periodic fee allowance. If empty, the grant will have no `expiration` and no `spend_limit`. + +* `period` is the specific period of time, after each period passes, `period_can_spend` will be reset. + +* `period_spend_limit` specifies the maximum number of coins that can be spent in the period. + +* `period_can_spend` is the number of coins left to be spent before the period\_reset time. + +* `period_reset` keeps track of when a next period reset should happen. + +### AllowedMsgAllowance + +`AllowedMsgAllowance` is a fee allowance, it can be any of `BasicFeeAllowance`, `PeriodicAllowance` but restricted only to the allowed messages mentioned by the granter. + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/feegrant/v1beta1/feegrant.proto#L70-L81 +``` + +* `allowance` is either `BasicAllowance` or `PeriodicAllowance`. + +* `allowed_messages` is array of messages allowed to execute the given allowance. + +### FeeGranter flag + +`feegrant` module introduces a `FeeGranter` flag for CLI for the sake of executing transactions with fee granter. When this flag is set, `clientCtx` will append the granter account address for transactions generated through CLI. + +```go expandable +package client + +import ( + + "crypto/tls" + "fmt" + "strings" + "github.com/pkg/errors" + "github.com/spf13/cobra" + "github.com/spf13/pflag" + "github.com/tendermint/tendermint/libs/cli" + "google.golang.org/grpc" + "google.golang.org/grpc/credentials" + "google.golang.org/grpc/credentials/insecure" + "github.com/cosmos/cosmos-sdk/client/flags" + "github.com/cosmos/cosmos-sdk/crypto/keyring" + sdk "github.com/cosmos/cosmos-sdk/types" +) + +// ClientContextKey defines the context key used to retrieve a client.Context from +// a command's Context. +const ClientContextKey = sdk.ContextKey("client.context") + +// SetCmdClientContextHandler is to be used in a command pre-hook execution to +// read flags that populate a Context and sets that to the command's Context. +func SetCmdClientContextHandler(clientCtx Context, cmd *cobra.Command) (err error) { + clientCtx, err = ReadPersistentCommandFlags(clientCtx, cmd.Flags()) + if err != nil { + return err +} + +return SetCmdClientContext(cmd, clientCtx) +} + +// ValidateCmd returns unknown command error or Help display if help flag set +func ValidateCmd(cmd *cobra.Command, args []string) + +error { + var unknownCmd string + var skipNext bool + for _, arg := range args { + // search for help flag + if arg == "--help" || arg == "-h" { + return cmd.Help() +} + + // check if the current arg is a flag + switch { + case len(arg) > 0 && (arg[0] == '-'): + // the next arg should be skipped if the current arg is a + // flag and does not use "=" to assign the flag's value + if !strings.Contains(arg, "=") { + skipNext = true +} + +else { + skipNext = false +} + case skipNext: + // skip current arg + skipNext = false + case unknownCmd == "": + // unknown command found + // continue searching for help flag + unknownCmd = arg +} + +} + + // return the help screen if no unknown command is found + if unknownCmd != "" { + err := fmt.Sprintf("unknown command \"%s\" for \"%s\"", unknownCmd, cmd.CalledAs()) + + // build suggestions for unknown argument + if suggestions := cmd.SuggestionsFor(unknownCmd); len(suggestions) > 0 { + err += "\n\nDid you mean this?\n" + for _, s := range suggestions { + err += fmt.Sprintf("\t%v\n", s) +} + +} + +return errors.New(err) +} + +return cmd.Help() +} + +// ReadPersistentCommandFlags returns a Context with fields set for "persistent" +// or common flags that do not necessarily change with context. +// +// Note, the provided clientCtx may have field pre-populated. The following order +// of precedence occurs: +// +// - client.Context field not pre-populated & flag not set: uses default flag value +// - client.Context field not pre-populated & flag set: uses set flag value +// - client.Context field pre-populated & flag not set: uses pre-populated value +// - client.Context field pre-populated & flag set: uses set flag value +func ReadPersistentCommandFlags(clientCtx Context, flagSet *pflag.FlagSet) (Context, error) { + if clientCtx.OutputFormat == "" || flagSet.Changed(cli.OutputFlag) { + output, _ := flagSet.GetString(cli.OutputFlag) + +clientCtx = clientCtx.WithOutputFormat(output) +} + if clientCtx.HomeDir == "" || flagSet.Changed(flags.FlagHome) { + homeDir, _ := flagSet.GetString(flags.FlagHome) + +clientCtx = clientCtx.WithHomeDir(homeDir) +} + if !clientCtx.Simulate || flagSet.Changed(flags.FlagDryRun) { + dryRun, _ := flagSet.GetBool(flags.FlagDryRun) + +clientCtx = clientCtx.WithSimulation(dryRun) +} + if clientCtx.KeyringDir == "" || flagSet.Changed(flags.FlagKeyringDir) { + keyringDir, _ := flagSet.GetString(flags.FlagKeyringDir) + + // The keyring directory is optional and falls back to the home directory + // if omitted. + if keyringDir == "" { + keyringDir = clientCtx.HomeDir +} + +clientCtx = clientCtx.WithKeyringDir(keyringDir) +} + if clientCtx.ChainID == "" || flagSet.Changed(flags.FlagChainID) { + chainID, _ := flagSet.GetString(flags.FlagChainID) + +clientCtx = clientCtx.WithChainID(chainID) +} + if clientCtx.Keyring == nil || flagSet.Changed(flags.FlagKeyringBackend) { + keyringBackend, _ := flagSet.GetString(flags.FlagKeyringBackend) + if keyringBackend != "" { + kr, err := NewKeyringFromBackend(clientCtx, keyringBackend) + if err != nil { + return clientCtx, err +} + +clientCtx = clientCtx.WithKeyring(kr) +} + +} + if clientCtx.Client == nil || flagSet.Changed(flags.FlagNode) { + rpcURI, _ := flagSet.GetString(flags.FlagNode) + if rpcURI != "" { + clientCtx = clientCtx.WithNodeURI(rpcURI) + +client, err := NewClientFromNode(rpcURI) + if err != nil { + return clientCtx, err +} + +clientCtx = clientCtx.WithClient(client) +} + +} + if clientCtx.GRPCClient == nil || flagSet.Changed(flags.FlagGRPC) { + grpcURI, _ := flagSet.GetString(flags.FlagGRPC) + if grpcURI != "" { + var dialOpts []grpc.DialOption + + useInsecure, _ := flagSet.GetBool(flags.FlagGRPCInsecure) + if useInsecure { + dialOpts = append(dialOpts, grpc.WithTransportCredentials(insecure.NewCredentials())) +} + +else { + dialOpts = append(dialOpts, grpc.WithTransportCredentials(credentials.NewTLS(&tls.Config{ + MinVersion: tls.VersionTLS12, +}))) +} + +grpcClient, err := grpc.Dial(grpcURI, dialOpts...) + if err != nil { + return Context{ +}, err +} + +clientCtx = clientCtx.WithGRPCClient(grpcClient) +} + +} + +return clientCtx, nil +} + +// readQueryCommandFlags returns an updated Context with fields set based on flags +// defined in AddQueryFlagsToCmd. An error is returned if any flag query fails. +// +// Note, the provided clientCtx may have field pre-populated. The following order +// of precedence occurs: +// +// - client.Context field not pre-populated & flag not set: uses default flag value +// - client.Context field not pre-populated & flag set: uses set flag value +// - client.Context field pre-populated & flag not set: uses pre-populated value +// - client.Context field pre-populated & flag set: uses set flag value +func readQueryCommandFlags(clientCtx Context, flagSet *pflag.FlagSet) (Context, error) { + if clientCtx.Height == 0 || flagSet.Changed(flags.FlagHeight) { + height, _ := flagSet.GetInt64(flags.FlagHeight) + +clientCtx = clientCtx.WithHeight(height) +} + if !clientCtx.UseLedger || flagSet.Changed(flags.FlagUseLedger) { + useLedger, _ := flagSet.GetBool(flags.FlagUseLedger) + +clientCtx = clientCtx.WithUseLedger(useLedger) +} + +return ReadPersistentCommandFlags(clientCtx, flagSet) +} + +// readTxCommandFlags returns an updated Context with fields set based on flags +// defined in AddTxFlagsToCmd. An error is returned if any flag query fails. +// +// Note, the provided clientCtx may have field pre-populated. The following order +// of precedence occurs: +// +// - client.Context field not pre-populated & flag not set: uses default flag value +// - client.Context field not pre-populated & flag set: uses set flag value +// - client.Context field pre-populated & flag not set: uses pre-populated value +// - client.Context field pre-populated & flag set: uses set flag value +func readTxCommandFlags(clientCtx Context, flagSet *pflag.FlagSet) (Context, error) { + clientCtx, err := ReadPersistentCommandFlags(clientCtx, flagSet) + if err != nil { + return clientCtx, err +} + if !clientCtx.GenerateOnly || flagSet.Changed(flags.FlagGenerateOnly) { + genOnly, _ := flagSet.GetBool(flags.FlagGenerateOnly) + +clientCtx = clientCtx.WithGenerateOnly(genOnly) +} + if !clientCtx.Offline || flagSet.Changed(flags.FlagOffline) { + offline, _ := flagSet.GetBool(flags.FlagOffline) + +clientCtx = clientCtx.WithOffline(offline) +} + if !clientCtx.UseLedger || flagSet.Changed(flags.FlagUseLedger) { + useLedger, _ := flagSet.GetBool(flags.FlagUseLedger) + +clientCtx = clientCtx.WithUseLedger(useLedger) +} + if clientCtx.BroadcastMode == "" || flagSet.Changed(flags.FlagBroadcastMode) { + bMode, _ := flagSet.GetString(flags.FlagBroadcastMode) + +clientCtx = clientCtx.WithBroadcastMode(bMode) +} + if !clientCtx.SkipConfirm || flagSet.Changed(flags.FlagSkipConfirmation) { + skipConfirm, _ := flagSet.GetBool(flags.FlagSkipConfirmation) + +clientCtx = clientCtx.WithSkipConfirmation(skipConfirm) +} + if clientCtx.SignModeStr == "" || flagSet.Changed(flags.FlagSignMode) { + signModeStr, _ := flagSet.GetString(flags.FlagSignMode) + +clientCtx = clientCtx.WithSignModeStr(signModeStr) +} + if clientCtx.FeePayer == nil || flagSet.Changed(flags.FlagFeePayer) { + payer, _ := flagSet.GetString(flags.FlagFeePayer) + if payer != "" { + payerAcc, err := sdk.AccAddressFromBech32(payer) + if err != nil { + return clientCtx, err +} + +clientCtx = clientCtx.WithFeePayerAddress(payerAcc) +} + +} + if clientCtx.FeeGranter == nil || flagSet.Changed(flags.FlagFeeGranter) { + granter, _ := flagSet.GetString(flags.FlagFeeGranter) + if granter != "" { + granterAcc, err := sdk.AccAddressFromBech32(granter) + if err != nil { + return clientCtx, err +} + +clientCtx = clientCtx.WithFeeGranterAddress(granterAcc) +} + +} + if clientCtx.From == "" || flagSet.Changed(flags.FlagFrom) { + from, _ := flagSet.GetString(flags.FlagFrom) + +fromAddr, fromName, keyType, err := GetFromFields(clientCtx, clientCtx.Keyring, from) + if err != nil { + return clientCtx, err +} + +clientCtx = clientCtx.WithFrom(from).WithFromAddress(fromAddr).WithFromName(fromName) + + // If the `from` signer account is a ledger key, we need to use + // SIGN_MODE_AMINO_JSON, because ledger doesn't support proto yet. + // ref: https://github.com/cosmos/cosmos-sdk/issues/8109 + if keyType == keyring.TypeLedger && clientCtx.SignModeStr != flags.SignModeLegacyAminoJSON && !clientCtx.LedgerHasProtobuf { + fmt.Println("Default sign-mode 'direct' not supported by Ledger, using sign-mode 'amino-json'.") + +clientCtx = clientCtx.WithSignModeStr(flags.SignModeLegacyAminoJSON) +} + +} + if !clientCtx.IsAux || flagSet.Changed(flags.FlagAux) { + isAux, _ := flagSet.GetBool(flags.FlagAux) + +clientCtx = clientCtx.WithAux(isAux) + if isAux { + // If the user didn't explicitly set an --output flag, use JSON by + // default. + if clientCtx.OutputFormat == "" || !flagSet.Changed(cli.OutputFlag) { + clientCtx = clientCtx.WithOutputFormat("json") +} + + // If the user didn't explicitly set a --sign-mode flag, use + // DIRECT_AUX by default. + if clientCtx.SignModeStr == "" || !flagSet.Changed(flags.FlagSignMode) { + clientCtx = clientCtx.WithSignModeStr(flags.SignModeDirectAux) +} + +} + +} + +return clientCtx, nil +} + +// GetClientQueryContext returns a Context from a command with fields set based on flags +// defined in AddQueryFlagsToCmd. An error is returned if any flag query fails. +// +// - client.Context field not pre-populated & flag not set: uses default flag value +// - client.Context field not pre-populated & flag set: uses set flag value +// - client.Context field pre-populated & flag not set: uses pre-populated value +// - client.Context field pre-populated & flag set: uses set flag value +func GetClientQueryContext(cmd *cobra.Command) (Context, error) { + ctx := GetClientContextFromCmd(cmd) + +return readQueryCommandFlags(ctx, cmd.Flags()) +} + +// GetClientTxContext returns a Context from a command with fields set based on flags +// defined in AddTxFlagsToCmd. An error is returned if any flag query fails. +// +// - client.Context field not pre-populated & flag not set: uses default flag value +// - client.Context field not pre-populated & flag set: uses set flag value +// - client.Context field pre-populated & flag not set: uses pre-populated value +// - client.Context field pre-populated & flag set: uses set flag value +func GetClientTxContext(cmd *cobra.Command) (Context, error) { + ctx := GetClientContextFromCmd(cmd) + +return readTxCommandFlags(ctx, cmd.Flags()) +} + +// GetClientContextFromCmd returns a Context from a command or an empty Context +// if it has not been set. +func GetClientContextFromCmd(cmd *cobra.Command) + +Context { + if v := cmd.Context().Value(ClientContextKey); v != nil { + clientCtxPtr := v.(*Context) + +return *clientCtxPtr +} + +return Context{ +} +} + +// SetCmdClientContext sets a command's Context value to the provided argument. +func SetCmdClientContext(cmd *cobra.Command, clientCtx Context) + +error { + v := cmd.Context().Value(ClientContextKey) + if v == nil { + return errors.New("client context not set") +} + clientCtxPtr := v.(*Context) + *clientCtxPtr = clientCtx + + return nil +} +``` + +```go expandable +package tx + +import ( + + "bufio" + "context" + "encoding/json" + "errors" + "fmt" + "os" + + gogogrpc "github.com/cosmos/gogoproto/grpc" + "github.com/spf13/pflag" + "github.com/cosmos/cosmos-sdk/client" + "github.com/cosmos/cosmos-sdk/client/input" + cryptotypes "github.com/cosmos/cosmos-sdk/crypto/types" + sdk "github.com/cosmos/cosmos-sdk/types" + sdkerrors "github.com/cosmos/cosmos-sdk/types/errors" + "github.com/cosmos/cosmos-sdk/types/tx" + "github.com/cosmos/cosmos-sdk/types/tx/signing" + authsigning "github.com/cosmos/cosmos-sdk/x/auth/signing" +) + +// GenerateOrBroadcastTxCLI will either generate and print and unsigned transaction +// or sign it and broadcast it returning an error upon failure. +func GenerateOrBroadcastTxCLI(clientCtx client.Context, flagSet *pflag.FlagSet, msgs ...sdk.Msg) + +error { + txf := NewFactoryCLI(clientCtx, flagSet) + +return GenerateOrBroadcastTxWithFactory(clientCtx, txf, msgs...) +} + +// GenerateOrBroadcastTxWithFactory will either generate and print and unsigned transaction +// or sign it and broadcast it returning an error upon failure. +func GenerateOrBroadcastTxWithFactory(clientCtx client.Context, txf Factory, msgs ...sdk.Msg) + +error { + // Validate all msgs before generating or broadcasting the tx. + // We were calling ValidateBasic separately in each CLI handler before. + // Right now, we're factorizing that call inside this function. + // ref: https://github.com/cosmos/cosmos-sdk/pull/9236#discussion_r623803504 + for _, msg := range msgs { + if err := msg.ValidateBasic(); err != nil { + return err +} + +} + + // If the --aux flag is set, we simply generate and print the AuxSignerData. + if clientCtx.IsAux { + auxSignerData, err := makeAuxSignerData(clientCtx, txf, msgs...) + if err != nil { + return err +} + +return clientCtx.PrintProto(&auxSignerData) +} + if clientCtx.GenerateOnly { + return txf.PrintUnsignedTx(clientCtx, msgs...) +} + +return BroadcastTx(clientCtx, txf, msgs...) +} + +// BroadcastTx attempts to generate, sign and broadcast a transaction with the +// given set of messages. It will also simulate gas requirements if necessary. +// It will return an error upon failure. +func BroadcastTx(clientCtx client.Context, txf Factory, msgs ...sdk.Msg) + +error { + txf, err := txf.Prepare(clientCtx) + if err != nil { + return err +} + if txf.SimulateAndExecute() || clientCtx.Simulate { + _, adjusted, err := CalculateGas(clientCtx, txf, msgs...) + if err != nil { + return err +} + +txf = txf.WithGas(adjusted) + _, _ = fmt.Fprintf(os.Stderr, "%s\n", GasEstimateResponse{ + GasEstimate: txf.Gas() +}) +} + if clientCtx.Simulate { + return nil +} + +tx, err := txf.BuildUnsignedTx(msgs...) + if err != nil { + return err +} + if !clientCtx.SkipConfirm { + txBytes, err := clientCtx.TxConfig.TxJSONEncoder()(tx.GetTx()) + if err != nil { + return err +} + if err := clientCtx.PrintRaw(json.RawMessage(txBytes)); err != nil { + _, _ = fmt.Fprintf(os.Stderr, "%s\n", txBytes) +} + buf := bufio.NewReader(os.Stdin) + +ok, err := input.GetConfirmation("confirm transaction before signing and broadcasting", buf, os.Stderr) + if err != nil || !ok { + _, _ = fmt.Fprintf(os.Stderr, "%s\n", "cancelled transaction") + +return err +} + +} + +err = Sign(txf, clientCtx.GetFromName(), tx, true) + if err != nil { + return err +} + +txBytes, err := clientCtx.TxConfig.TxEncoder()(tx.GetTx()) + if err != nil { + return err +} + + // broadcast to a Tendermint node + res, err := clientCtx.BroadcastTx(txBytes) + if err != nil { + return err +} + +return clientCtx.PrintProto(res) +} + +// CalculateGas simulates the execution of a transaction and returns the +// simulation response obtained by the query and the adjusted gas amount. +func CalculateGas( + clientCtx gogogrpc.ClientConn, txf Factory, msgs ...sdk.Msg, +) (*tx.SimulateResponse, uint64, error) { + txBytes, err := txf.BuildSimTx(msgs...) + if err != nil { + return nil, 0, err +} + txSvcClient := tx.NewServiceClient(clientCtx) + +simRes, err := txSvcClient.Simulate(context.Background(), &tx.SimulateRequest{ + TxBytes: txBytes, +}) + if err != nil { + return nil, 0, err +} + +return simRes, uint64(txf.GasAdjustment() * float64(simRes.GasInfo.GasUsed)), nil +} + +// SignWithPrivKey signs a given tx with the given private key, and returns the +// corresponding SignatureV2 if the signing is successful. +func SignWithPrivKey( + signMode signing.SignMode, signerData authsigning.SignerData, + txBuilder client.TxBuilder, priv cryptotypes.PrivKey, txConfig client.TxConfig, + accSeq uint64, +) (signing.SignatureV2, error) { + var sigV2 signing.SignatureV2 + + // Generate the bytes to be signed. + signBytes, err := txConfig.SignModeHandler().GetSignBytes(signMode, signerData, txBuilder.GetTx()) + if err != nil { + return sigV2, err +} + + // Sign those bytes + signature, err := priv.Sign(signBytes) + if err != nil { + return sigV2, err +} + + // Construct the SignatureV2 struct + sigData := signing.SingleSignatureData{ + SignMode: signMode, + Signature: signature, +} + +sigV2 = signing.SignatureV2{ + PubKey: priv.PubKey(), + Data: &sigData, + Sequence: accSeq, +} + +return sigV2, nil +} + +// countDirectSigners counts the number of DIRECT signers in a signature data. +func countDirectSigners(data signing.SignatureData) + +int { + switch data := data.(type) { + case *signing.SingleSignatureData: + if data.SignMode == signing.SignMode_SIGN_MODE_DIRECT { + return 1 +} + +return 0 + case *signing.MultiSignatureData: + directSigners := 0 + for _, d := range data.Signatures { + directSigners += countDirectSigners(d) +} + +return directSigners + default: + panic("unreachable case") +} +} + +// checkMultipleSigners checks that there can be maximum one DIRECT signer in +// a tx. +func checkMultipleSigners(tx authsigning.Tx) + +error { + directSigners := 0 + sigsV2, err := tx.GetSignaturesV2() + if err != nil { + return err +} + for _, sig := range sigsV2 { + directSigners += countDirectSigners(sig.Data) + if directSigners > 1 { + return sdkerrors.ErrNotSupported.Wrap("txs signed with CLI can have maximum 1 DIRECT signer") +} + +} + +return nil +} + +// Sign signs a given tx with a named key. The bytes signed over are canconical. +// The resulting signature will be added to the transaction builder overwriting the previous +// ones if overwrite=true (otherwise, the signature will be appended). +// Signing a transaction with mutltiple signers in the DIRECT mode is not supprted and will +// return an error. +// An error is returned upon failure. +func Sign(txf Factory, name string, txBuilder client.TxBuilder, overwriteSig bool) + +error { + if txf.keybase == nil { + return errors.New("keybase must be set prior to signing a transaction") +} + signMode := txf.signMode + if signMode == signing.SignMode_SIGN_MODE_UNSPECIFIED { + // use the SignModeHandler's default mode if unspecified + signMode = txf.txConfig.SignModeHandler().DefaultMode() +} + +k, err := txf.keybase.Key(name) + if err != nil { + return err +} + +pubKey, err := k.GetPubKey() + if err != nil { + return err +} + signerData := authsigning.SignerData{ + ChainID: txf.chainID, + AccountNumber: txf.accountNumber, + Sequence: txf.sequence, + PubKey: pubKey, + Address: sdk.AccAddress(pubKey.Address()).String(), +} + + // For SIGN_MODE_DIRECT, calling SetSignatures calls setSignerInfos on + // TxBuilder under the hood, and SignerInfos is needed to generated the + // sign bytes. This is the reason for setting SetSignatures here, with a + // nil signature. + // + // Note: this line is not needed for SIGN_MODE_LEGACY_AMINO, but putting it + // also doesn't affect its generated sign bytes, so for code's simplicity + // sake, we put it here. + sigData := signing.SingleSignatureData{ + SignMode: signMode, + Signature: nil, +} + sig := signing.SignatureV2{ + PubKey: pubKey, + Data: &sigData, + Sequence: txf.Sequence(), +} + +var prevSignatures []signing.SignatureV2 + if !overwriteSig { + prevSignatures, err = txBuilder.GetTx().GetSignaturesV2() + if err != nil { + return err +} + +} + // Overwrite or append signer infos. + var sigs []signing.SignatureV2 + if overwriteSig { + sigs = []signing.SignatureV2{ + sig +} + +} + +else { + sigs = append(sigs, prevSignatures...) + +sigs = append(sigs, sig) +} + if err := txBuilder.SetSignatures(sigs...); err != nil { + return err +} + if err := checkMultipleSigners(txBuilder.GetTx()); err != nil { + return err +} + + // Generate the bytes to be signed. + bytesToSign, err := txf.txConfig.SignModeHandler().GetSignBytes(signMode, signerData, txBuilder.GetTx()) + if err != nil { + return err +} + + // Sign those bytes + sigBytes, _, err := txf.keybase.Sign(name, bytesToSign) + if err != nil { + return err +} + + // Construct the SignatureV2 struct + sigData = signing.SingleSignatureData{ + SignMode: signMode, + Signature: sigBytes, +} + +sig = signing.SignatureV2{ + PubKey: pubKey, + Data: &sigData, + Sequence: txf.Sequence(), +} + if overwriteSig { + err = txBuilder.SetSignatures(sig) +} + +else { + prevSignatures = append(prevSignatures, sig) + +err = txBuilder.SetSignatures(prevSignatures...) +} + if err != nil { + return fmt.Errorf("unable to set signatures on payload: %w", err) +} + + // Run optional preprocessing if specified. By default, this is unset + // and will return nil. + return txf.PreprocessTx(name, txBuilder) +} + +// GasEstimateResponse defines a response definition for tx gas estimation. +type GasEstimateResponse struct { + GasEstimate uint64 `json:"gas_estimate" yaml:"gas_estimate"` +} + +func (gr GasEstimateResponse) + +String() + +string { + return fmt.Sprintf("gas estimate: %d", gr.GasEstimate) +} + +// makeAuxSignerData generates an AuxSignerData from the client inputs. +func makeAuxSignerData(clientCtx client.Context, f Factory, msgs ...sdk.Msg) (tx.AuxSignerData, error) { + b := NewAuxTxBuilder() + +fromAddress, name, _, err := client.GetFromFields(clientCtx, clientCtx.Keyring, clientCtx.From) + if err != nil { + return tx.AuxSignerData{ +}, err +} + +b.SetAddress(fromAddress.String()) + if clientCtx.Offline { + b.SetAccountNumber(f.accountNumber) + +b.SetSequence(f.sequence) +} + +else { + accNum, seq, err := clientCtx.AccountRetriever.GetAccountNumberSequence(clientCtx, fromAddress) + if err != nil { + return tx.AuxSignerData{ +}, err +} + +b.SetAccountNumber(accNum) + +b.SetSequence(seq) +} + +err = b.SetMsgs(msgs...) + if err != nil { + return tx.AuxSignerData{ +}, err +} + if f.tip != nil { + if _, err := sdk.AccAddressFromBech32(f.tip.Tipper); err != nil { + return tx.AuxSignerData{ +}, sdkerrors.ErrInvalidAddress.Wrap("tipper must be a bech32 address") +} + +b.SetTip(f.tip) +} + +err = b.SetSignMode(f.SignMode()) + if err != nil { + return tx.AuxSignerData{ +}, err +} + +key, err := clientCtx.Keyring.Key(name) + if err != nil { + return tx.AuxSignerData{ +}, err +} + +pub, err := key.GetPubKey() + if err != nil { + return tx.AuxSignerData{ +}, err +} + +err = b.SetPubKey(pub) + if err != nil { + return tx.AuxSignerData{ +}, err +} + +b.SetChainID(clientCtx.ChainID) + +signBz, err := b.GetSignBytes() + if err != nil { + return tx.AuxSignerData{ +}, err +} + +sig, _, err := clientCtx.Keyring.Sign(name, signBz) + if err != nil { + return tx.AuxSignerData{ +}, err +} + +b.SetSignature(sig) + +return b.GetAuxSignerData() +} +``` + +```go expandable +package tx + +import ( + + "github.com/cosmos/gogoproto/proto" + "github.com/cosmos/cosmos-sdk/client" + "github.com/cosmos/cosmos-sdk/codec" + codectypes "github.com/cosmos/cosmos-sdk/codec/types" + cryptotypes "github.com/cosmos/cosmos-sdk/crypto/types" + sdk "github.com/cosmos/cosmos-sdk/types" + sdkerrors "github.com/cosmos/cosmos-sdk/types/errors" + "github.com/cosmos/cosmos-sdk/types/tx" + "github.com/cosmos/cosmos-sdk/types/tx/signing" + "github.com/cosmos/cosmos-sdk/x/auth/ante" + authsigning "github.com/cosmos/cosmos-sdk/x/auth/signing" +) + +// wrapper is a wrapper around the tx.Tx proto.Message which retain the raw +// body and auth_info bytes. +type wrapper struct { + cdc codec.Codec + + tx *tx.Tx + + // bodyBz represents the protobuf encoding of TxBody. This should be encoding + // from the client using TxRaw if the tx was decoded from the wire + bodyBz []byte + + // authInfoBz represents the protobuf encoding of TxBody. This should be encoding + // from the client using TxRaw if the tx was decoded from the wire + authInfoBz []byte + + txBodyHasUnknownNonCriticals bool +} + +var ( + _ authsigning.Tx = &wrapper{ +} + _ client.TxBuilder = &wrapper{ +} + _ tx.TipTx = &wrapper{ +} + _ ante.HasExtensionOptionsTx = &wrapper{ +} + _ ExtensionOptionsTxBuilder = &wrapper{ +} + _ tx.TipTx = &wrapper{ +} +) + +// ExtensionOptionsTxBuilder defines a TxBuilder that can also set extensions. +type ExtensionOptionsTxBuilder interface { + client.TxBuilder + + SetExtensionOptions(...*codectypes.Any) + +SetNonCriticalExtensionOptions(...*codectypes.Any) +} + +func newBuilder(cdc codec.Codec) *wrapper { + return &wrapper{ + cdc: cdc, + tx: &tx.Tx{ + Body: &tx.TxBody{ +}, + AuthInfo: &tx.AuthInfo{ + Fee: &tx.Fee{ +}, +}, +}, +} +} + +func (w *wrapper) + +GetMsgs() []sdk.Msg { + return w.tx.GetMsgs() +} + +func (w *wrapper) + +ValidateBasic() + +error { + return w.tx.ValidateBasic() +} + +func (w *wrapper) + +getBodyBytes() []byte { + if len(w.bodyBz) == 0 { + // if bodyBz is empty, then marshal the body. bodyBz will generally + // be set to nil whenever SetBody is called so the result of calling + // this method should always return the correct bytes. Note that after + // decoding bodyBz is derived from TxRaw so that it matches what was + // transmitted over the wire + var err error + w.bodyBz, err = proto.Marshal(w.tx.Body) + if err != nil { + panic(err) +} + +} + +return w.bodyBz +} + +func (w *wrapper) + +getAuthInfoBytes() []byte { + if len(w.authInfoBz) == 0 { + // if authInfoBz is empty, then marshal the body. authInfoBz will generally + // be set to nil whenever SetAuthInfo is called so the result of calling + // this method should always return the correct bytes. Note that after + // decoding authInfoBz is derived from TxRaw so that it matches what was + // transmitted over the wire + var err error + w.authInfoBz, err = proto.Marshal(w.tx.AuthInfo) + if err != nil { + panic(err) +} + +} + +return w.authInfoBz +} + +func (w *wrapper) + +GetSigners() []sdk.AccAddress { + return w.tx.GetSigners() +} + +func (w *wrapper) + +GetPubKeys() ([]cryptotypes.PubKey, error) { + signerInfos := w.tx.AuthInfo.SignerInfos + pks := make([]cryptotypes.PubKey, len(signerInfos)) + for i, si := range signerInfos { + // NOTE: it is okay to leave this nil if there is no PubKey in the SignerInfo. + // PubKey's can be left unset in SignerInfo. + if si.PublicKey == nil { + continue +} + pkAny := si.PublicKey.GetCachedValue() + +pk, ok := pkAny.(cryptotypes.PubKey) + if ok { + pks[i] = pk +} + +else { + return nil, sdkerrors.Wrapf(sdkerrors.ErrLogic, "Expecting PubKey, got: %T", pkAny) +} + +} + +return pks, nil +} + +func (w *wrapper) + +GetGas() + +uint64 { + return w.tx.AuthInfo.Fee.GasLimit +} + +func (w *wrapper) + +GetFee() + +sdk.Coins { + return w.tx.AuthInfo.Fee.Amount +} + +func (w *wrapper) + +FeePayer() + +sdk.AccAddress { + feePayer := w.tx.AuthInfo.Fee.Payer + if feePayer != "" { + return sdk.MustAccAddressFromBech32(feePayer) +} + // use first signer as default if no payer specified + return w.GetSigners()[0] +} + +func (w *wrapper) + +FeeGranter() + +sdk.AccAddress { + feePayer := w.tx.AuthInfo.Fee.Granter + if feePayer != "" { + return sdk.MustAccAddressFromBech32(feePayer) +} + +return nil +} + +func (w *wrapper) + +GetTip() *tx.Tip { + return w.tx.AuthInfo.Tip +} + +func (w *wrapper) + +GetMemo() + +string { + return w.tx.Body.Memo +} + +// GetTimeoutHeight returns the transaction's timeout height (if set). +func (w *wrapper) + +GetTimeoutHeight() + +uint64 { + return w.tx.Body.TimeoutHeight +} + +func (w *wrapper) + +GetSignaturesV2() ([]signing.SignatureV2, error) { + signerInfos := w.tx.AuthInfo.SignerInfos + sigs := w.tx.Signatures + pubKeys, err := w.GetPubKeys() + if err != nil { + return nil, err +} + n := len(signerInfos) + res := make([]signing.SignatureV2, n) + for i, si := range signerInfos { + // handle nil signatures (in case of simulation) + if si.ModeInfo == nil { + res[i] = signing.SignatureV2{ + PubKey: pubKeys[i], +} + +} + +else { + var err error + sigData, err := ModeInfoAndSigToSignatureData(si.ModeInfo, sigs[i]) + if err != nil { + return nil, err +} + // sequence number is functionally a transaction nonce and referred to as such in the SDK + nonce := si.GetSequence() + +res[i] = signing.SignatureV2{ + PubKey: pubKeys[i], + Data: sigData, + Sequence: nonce, +} + + +} + +} + +return res, nil +} + +func (w *wrapper) + +SetMsgs(msgs ...sdk.Msg) + +error { + anys, err := tx.SetMsgs(msgs) + if err != nil { + return err +} + +w.tx.Body.Messages = anys + + // set bodyBz to nil because the cached bodyBz no longer matches tx.Body + w.bodyBz = nil + + return nil +} + +// SetTimeoutHeight sets the transaction's height timeout. +func (w *wrapper) + +SetTimeoutHeight(height uint64) { + w.tx.Body.TimeoutHeight = height + + // set bodyBz to nil because the cached bodyBz no longer matches tx.Body + w.bodyBz = nil +} + +func (w *wrapper) + +SetMemo(memo string) { + w.tx.Body.Memo = memo + + // set bodyBz to nil because the cached bodyBz no longer matches tx.Body + w.bodyBz = nil +} + +func (w *wrapper) + +SetGasLimit(limit uint64) { + if w.tx.AuthInfo.Fee == nil { + w.tx.AuthInfo.Fee = &tx.Fee{ +} + +} + +w.tx.AuthInfo.Fee.GasLimit = limit + + // set authInfoBz to nil because the cached authInfoBz no longer matches tx.AuthInfo + w.authInfoBz = nil +} + +func (w *wrapper) + +SetFeeAmount(coins sdk.Coins) { + if w.tx.AuthInfo.Fee == nil { + w.tx.AuthInfo.Fee = &tx.Fee{ +} + +} + +w.tx.AuthInfo.Fee.Amount = coins + + // set authInfoBz to nil because the cached authInfoBz no longer matches tx.AuthInfo + w.authInfoBz = nil +} + +func (w *wrapper) + +SetTip(tip *tx.Tip) { + w.tx.AuthInfo.Tip = tip + + // set authInfoBz to nil because the cached authInfoBz no longer matches tx.AuthInfo + w.authInfoBz = nil +} + +func (w *wrapper) + +SetFeePayer(feePayer sdk.AccAddress) { + if w.tx.AuthInfo.Fee == nil { + w.tx.AuthInfo.Fee = &tx.Fee{ +} + +} + +w.tx.AuthInfo.Fee.Payer = feePayer.String() + + // set authInfoBz to nil because the cached authInfoBz no longer matches tx.AuthInfo + w.authInfoBz = nil +} + +func (w *wrapper) + +SetFeeGranter(feeGranter sdk.AccAddress) { + if w.tx.AuthInfo.Fee == nil { + w.tx.AuthInfo.Fee = &tx.Fee{ +} + +} + +w.tx.AuthInfo.Fee.Granter = feeGranter.String() + + // set authInfoBz to nil because the cached authInfoBz no longer matches tx.AuthInfo + w.authInfoBz = nil +} + +func (w *wrapper) + +SetSignatures(signatures ...signing.SignatureV2) + +error { + n := len(signatures) + signerInfos := make([]*tx.SignerInfo, n) + rawSigs := make([][]byte, n) + for i, sig := range signatures { + var modeInfo *tx.ModeInfo + modeInfo, rawSigs[i] = SignatureDataToModeInfoAndSig(sig.Data) + +any, err := codectypes.NewAnyWithValue(sig.PubKey) + if err != nil { + return err +} + +signerInfos[i] = &tx.SignerInfo{ + PublicKey: any, + ModeInfo: modeInfo, + Sequence: sig.Sequence, +} + +} + +w.setSignerInfos(signerInfos) + +w.setSignatures(rawSigs) + +return nil +} + +func (w *wrapper) + +setSignerInfos(infos []*tx.SignerInfo) { + w.tx.AuthInfo.SignerInfos = infos + // set authInfoBz to nil because the cached authInfoBz no longer matches tx.AuthInfo + w.authInfoBz = nil +} + +func (w *wrapper) + +setSignerInfoAtIndex(index int, info *tx.SignerInfo) { + if w.tx.AuthInfo.SignerInfos == nil { + w.tx.AuthInfo.SignerInfos = make([]*tx.SignerInfo, len(w.GetSigners())) +} + +w.tx.AuthInfo.SignerInfos[index] = info + // set authInfoBz to nil because the cached authInfoBz no longer matches tx.AuthInfo + w.authInfoBz = nil +} + +func (w *wrapper) + +setSignatures(sigs [][]byte) { + w.tx.Signatures = sigs +} + +func (w *wrapper) + +setSignatureAtIndex(index int, sig []byte) { + if w.tx.Signatures == nil { + w.tx.Signatures = make([][]byte, len(w.GetSigners())) +} + +w.tx.Signatures[index] = sig +} + +func (w *wrapper) + +GetTx() + +authsigning.Tx { + return w +} + +func (w *wrapper) + +GetProtoTx() *tx.Tx { + return w.tx +} + +// Deprecated: AsAny extracts proto Tx and wraps it into Any. +// NOTE: You should probably use `GetProtoTx` if you want to serialize the transaction. +func (w *wrapper) + +AsAny() *codectypes.Any { + return codectypes.UnsafePackAny(w.tx) +} + +// WrapTx creates a TxBuilder wrapper around a tx.Tx proto message. +func WrapTx(protoTx *tx.Tx) + +client.TxBuilder { + return &wrapper{ + tx: protoTx, +} +} + +func (w *wrapper) + +GetExtensionOptions() []*codectypes.Any { + return w.tx.Body.ExtensionOptions +} + +func (w *wrapper) + +GetNonCriticalExtensionOptions() []*codectypes.Any { + return w.tx.Body.NonCriticalExtensionOptions +} + +func (w *wrapper) + +SetExtensionOptions(extOpts ...*codectypes.Any) { + w.tx.Body.ExtensionOptions = extOpts + w.bodyBz = nil +} + +func (w *wrapper) + +SetNonCriticalExtensionOptions(extOpts ...*codectypes.Any) { + w.tx.Body.NonCriticalExtensionOptions = extOpts + w.bodyBz = nil +} + +func (w *wrapper) + +AddAuxSignerData(data tx.AuxSignerData) + +error { + err := data.ValidateBasic() + if err != nil { + return err +} + +w.bodyBz = data.SignDoc.BodyBytes + + var body tx.TxBody + err = w.cdc.Unmarshal(w.bodyBz, &body) + if err != nil { + return err +} + if w.tx.Body.Memo != "" && w.tx.Body.Memo != body.Memo { + return sdkerrors.ErrInvalidRequest.Wrapf("TxBuilder has memo %s, got %s in AuxSignerData", w.tx.Body.Memo, body.Memo) +} + if w.tx.Body.TimeoutHeight != 0 && w.tx.Body.TimeoutHeight != body.TimeoutHeight { + return sdkerrors.ErrInvalidRequest.Wrapf("TxBuilder has timeout height %d, got %d in AuxSignerData", w.tx.Body.TimeoutHeight, body.TimeoutHeight) +} + if len(w.tx.Body.ExtensionOptions) != 0 { + if len(w.tx.Body.ExtensionOptions) != len(body.ExtensionOptions) { + return sdkerrors.ErrInvalidRequest.Wrapf("TxBuilder has %d extension options, got %d in AuxSignerData", len(w.tx.Body.ExtensionOptions), len(body.ExtensionOptions)) +} + for i, o := range w.tx.Body.ExtensionOptions { + if !o.Equal(body.ExtensionOptions[i]) { + return sdkerrors.ErrInvalidRequest.Wrapf("TxBuilder has extension option %+v at index %d, got %+v in AuxSignerData", o, i, body.ExtensionOptions[i]) +} + +} + +} + if len(w.tx.Body.NonCriticalExtensionOptions) != 0 { + if len(w.tx.Body.NonCriticalExtensionOptions) != len(body.NonCriticalExtensionOptions) { + return sdkerrors.ErrInvalidRequest.Wrapf("TxBuilder has %d non-critical extension options, got %d in AuxSignerData", len(w.tx.Body.NonCriticalExtensionOptions), len(body.NonCriticalExtensionOptions)) +} + for i, o := range w.tx.Body.NonCriticalExtensionOptions { + if !o.Equal(body.NonCriticalExtensionOptions[i]) { + return sdkerrors.ErrInvalidRequest.Wrapf("TxBuilder has non-critical extension option %+v at index %d, got %+v in AuxSignerData", o, i, body.NonCriticalExtensionOptions[i]) +} + +} + +} + if len(w.tx.Body.Messages) != 0 { + if len(w.tx.Body.Messages) != len(body.Messages) { + return sdkerrors.ErrInvalidRequest.Wrapf("TxBuilder has %d Msgs, got %d in AuxSignerData", len(w.tx.Body.Messages), len(body.Messages)) +} + for i, o := range w.tx.Body.Messages { + if !o.Equal(body.Messages[i]) { + return sdkerrors.ErrInvalidRequest.Wrapf("TxBuilder has Msg %+v at index %d, got %+v in AuxSignerData", o, i, body.Messages[i]) +} + +} + +} + if w.tx.AuthInfo.Tip != nil && data.SignDoc.Tip != nil { + if !w.tx.AuthInfo.Tip.Amount.IsEqual(data.SignDoc.Tip.Amount) { + return sdkerrors.ErrInvalidRequest.Wrapf("TxBuilder has tip %+v, got %+v in AuxSignerData", w.tx.AuthInfo.Tip.Amount, data.SignDoc.Tip.Amount) +} + if w.tx.AuthInfo.Tip.Tipper != data.SignDoc.Tip.Tipper { + return sdkerrors.ErrInvalidRequest.Wrapf("TxBuilder has tipper %s, got %s in AuxSignerData", w.tx.AuthInfo.Tip.Tipper, data.SignDoc.Tip.Tipper) +} + +} + +w.SetMemo(body.Memo) + +w.SetTimeoutHeight(body.TimeoutHeight) + +w.SetExtensionOptions(body.ExtensionOptions...) + +w.SetNonCriticalExtensionOptions(body.NonCriticalExtensionOptions...) + msgs := make([]sdk.Msg, len(body.Messages)) + for i, msgAny := range body.Messages { + msgs[i] = msgAny.GetCachedValue().(sdk.Msg) +} + +w.SetMsgs(msgs...) + +w.SetTip(data.GetSignDoc().GetTip()) + + // Get the aux signer's index in GetSigners. + signerIndex := -1 + for i, signer := range w.GetSigners() { + if signer.String() == data.Address { + signerIndex = i +} + +} + if signerIndex < 0 { + return sdkerrors.ErrLogic.Wrapf("address %s is not a signer", data.Address) +} + +w.setSignerInfoAtIndex(signerIndex, &tx.SignerInfo{ + PublicKey: data.SignDoc.PublicKey, + ModeInfo: &tx.ModeInfo{ + Sum: &tx.ModeInfo_Single_{ + Single: &tx.ModeInfo_Single{ + Mode: data.Mode +}}}, + Sequence: data.SignDoc.Sequence, +}) + +w.setSignatureAtIndex(signerIndex, data.Sig) + +return nil +} +``` + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/tx/v1beta1/tx.proto#L203-L224 +``` + +Example cmd: + +```go +./simd tx gov submit-proposal --title="Test Proposal" --description="My awesome proposal" --type="Text" --from validator-key --fee-granter=cosmos1xh44hxt7spr67hqaa7nyx5gnutrz5fraw6grxn --chain-id=testnet --fees="10stake" +``` + +### Granted Fee Deductions + +Fees are deducted from grants in the `x/auth` ante handler. To learn more about how ante handlers work, read the [Auth Module AnteHandlers Guide](/sdk/v0.54/modules/auth/auth#antehandlers). + +### Gas + +In order to prevent DoS attacks, using a filtered `x/feegrant` incurs gas. The SDK must assure that the `grantee`'s transactions all conform to the filter set by the `granter`. The SDK does this by iterating over the allowed messages in the filter and charging 10 gas per filtered message. The SDK will then iterate over the messages being sent by the `grantee` to ensure the messages adhere to the filter, also charging 10 gas per message. The SDK will stop iterating and fail the transaction if it finds a message that does not conform to the filter. + +**WARNING**: The gas is charged against the granted allowance. Ensure your messages conform to the filter, if any, before sending transactions using your allowance. + +### Pruning + +A queue in the state maintained with the prefix of expiration of the grants and checks them on EndBlock with the current block time for every block to prune. + +## State + +### FeeAllowance + +Fee Allowances are identified by combining `Grantee` (the account address of fee allowance grantee) with the `Granter` (the account address of fee allowance granter). + +Fee allowance grants are stored in the state as follows: + +* Grant: `0x00 | grantee_addr_len (1 byte) | grantee_addr_bytes | granter_addr_len (1 byte) | granter_addr_bytes -> ProtocolBuffer(Grant)` + +```go expandable +// Code generated by protoc-gen-gogo. DO NOT EDIT. +// source: cosmos/feegrant/v1beta1/feegrant.proto + +package feegrant + +import ( + + fmt "fmt" + _ "github.com/cosmos/cosmos-proto" + types1 "github.com/cosmos/cosmos-sdk/codec/types" + github_com_cosmos_cosmos_sdk_types "github.com/cosmos/cosmos-sdk/types" + types "github.com/cosmos/cosmos-sdk/types" + _ "github.com/cosmos/cosmos-sdk/types/tx/amino" + _ "github.com/cosmos/gogoproto/gogoproto" + proto "github.com/cosmos/gogoproto/proto" + github_com_cosmos_gogoproto_types "github.com/cosmos/gogoproto/types" + _ "google.golang.org/protobuf/types/known/durationpb" + _ "google.golang.org/protobuf/types/known/timestamppb" + io "io" + math "math" + math_bits "math/bits" + time "time" +) + +// Reference imports to suppress errors if they are not otherwise used. +var _ = proto.Marshal +var _ = fmt.Errorf +var _ = math.Inf +var _ = time.Kitchen + +// This is a compile-time assertion to ensure that this generated file +// is compatible with the proto package it is being compiled against. +// A compilation error at this line likely means your copy of the +// proto package needs to be updated. +const _ = proto.GoGoProtoPackageIsVersion3 // please upgrade the proto package + +// BasicAllowance implements Allowance with a one-time grant of coins +// that optionally expires. The grantee can use up to SpendLimit to cover fees. +type BasicAllowance struct { + // spend_limit specifies the maximum amount of coins that can be spent + // by this allowance and will be updated as coins are spent. If it is + // empty, there is no spend limit and any amount of coins can be spent. + SpendLimit github_com_cosmos_cosmos_sdk_types.Coins `protobuf:"bytes,1,rep,name=spend_limit,json=spendLimit,proto3,castrepeated=github.com/cosmos/cosmos-sdk/types.Coins" json:"spend_limit"` + // expiration specifies an optional time when this allowance expires + Expiration *time.Time `protobuf:"bytes,2,opt,name=expiration,proto3,stdtime" json:"expiration,omitempty"` +} + +func (m *BasicAllowance) + +Reset() { *m = BasicAllowance{ +} +} + +func (m *BasicAllowance) + +String() + +string { + return proto.CompactTextString(m) +} + +func (*BasicAllowance) + +ProtoMessage() { +} + +func (*BasicAllowance) + +Descriptor() ([]byte, []int) { + return fileDescriptor_7279582900c30aea, []int{0 +} +} + +func (m *BasicAllowance) + +XXX_Unmarshal(b []byte) + +error { + return m.Unmarshal(b) +} + +func (m *BasicAllowance) + +XXX_Marshal(b []byte, deterministic bool) ([]byte, error) { + if deterministic { + return xxx_messageInfo_BasicAllowance.Marshal(b, m, deterministic) +} + +else { + b = b[:cap(b)] + n, err := m.MarshalToSizedBuffer(b) + if err != nil { + return nil, err +} + +return b[:n], nil +} +} + +func (m *BasicAllowance) + +XXX_Merge(src proto.Message) { + xxx_messageInfo_BasicAllowance.Merge(m, src) +} + +func (m *BasicAllowance) + +XXX_Size() + +int { + return m.Size() +} + +func (m *BasicAllowance) + +XXX_DiscardUnknown() { + xxx_messageInfo_BasicAllowance.DiscardUnknown(m) +} + +var xxx_messageInfo_BasicAllowance proto.InternalMessageInfo + +func (m *BasicAllowance) + +GetSpendLimit() + +github_com_cosmos_cosmos_sdk_types.Coins { + if m != nil { + return m.SpendLimit +} + +return nil +} + +func (m *BasicAllowance) + +GetExpiration() *time.Time { + if m != nil { + return m.Expiration +} + +return nil +} + +// PeriodicAllowance extends Allowance to allow for both a maximum cap, +// as well as a limit per time period. +type PeriodicAllowance struct { + // basic specifies a struct of `BasicAllowance` + Basic BasicAllowance `protobuf:"bytes,1,opt,name=basic,proto3" json:"basic"` + // period specifies the time duration in which period_spend_limit coins can + // be spent before that allowance is reset + Period time.Duration `protobuf:"bytes,2,opt,name=period,proto3,stdduration" json:"period"` + // period_spend_limit specifies the maximum number of coins that can be spent + // in the period + PeriodSpendLimit github_com_cosmos_cosmos_sdk_types.Coins `protobuf:"bytes,3,rep,name=period_spend_limit,json=periodSpendLimit,proto3,castrepeated=github.com/cosmos/cosmos-sdk/types.Coins" json:"period_spend_limit"` + // period_can_spend is the number of coins left to be spent before the period_reset time + PeriodCanSpend github_com_cosmos_cosmos_sdk_types.Coins `protobuf:"bytes,4,rep,name=period_can_spend,json=periodCanSpend,proto3,castrepeated=github.com/cosmos/cosmos-sdk/types.Coins" json:"period_can_spend"` + // period_reset is the time at which this period resets and a new one begins, + // it is calculated from the start time of the first transaction after the + // last period ended + PeriodReset time.Time `protobuf:"bytes,5,opt,name=period_reset,json=periodReset,proto3,stdtime" json:"period_reset"` +} + +func (m *PeriodicAllowance) + +Reset() { *m = PeriodicAllowance{ +} +} + +func (m *PeriodicAllowance) + +String() + +string { + return proto.CompactTextString(m) +} + +func (*PeriodicAllowance) + +ProtoMessage() { +} + +func (*PeriodicAllowance) + +Descriptor() ([]byte, []int) { + return fileDescriptor_7279582900c30aea, []int{1 +} +} + +func (m *PeriodicAllowance) + +XXX_Unmarshal(b []byte) + +error { + return m.Unmarshal(b) +} + +func (m *PeriodicAllowance) + +XXX_Marshal(b []byte, deterministic bool) ([]byte, error) { + if deterministic { + return xxx_messageInfo_PeriodicAllowance.Marshal(b, m, deterministic) +} + +else { + b = b[:cap(b)] + n, err := m.MarshalToSizedBuffer(b) + if err != nil { + return nil, err +} + +return b[:n], nil +} +} + +func (m *PeriodicAllowance) + +XXX_Merge(src proto.Message) { + xxx_messageInfo_PeriodicAllowance.Merge(m, src) +} + +func (m *PeriodicAllowance) + +XXX_Size() + +int { + return m.Size() +} + +func (m *PeriodicAllowance) + +XXX_DiscardUnknown() { + xxx_messageInfo_PeriodicAllowance.DiscardUnknown(m) +} + +var xxx_messageInfo_PeriodicAllowance proto.InternalMessageInfo + +func (m *PeriodicAllowance) + +GetBasic() + +BasicAllowance { + if m != nil { + return m.Basic +} + +return BasicAllowance{ +} +} + +func (m *PeriodicAllowance) + +GetPeriod() + +time.Duration { + if m != nil { + return m.Period +} + +return 0 +} + +func (m *PeriodicAllowance) + +GetPeriodSpendLimit() + +github_com_cosmos_cosmos_sdk_types.Coins { + if m != nil { + return m.PeriodSpendLimit +} + +return nil +} + +func (m *PeriodicAllowance) + +GetPeriodCanSpend() + +github_com_cosmos_cosmos_sdk_types.Coins { + if m != nil { + return m.PeriodCanSpend +} + +return nil +} + +func (m *PeriodicAllowance) + +GetPeriodReset() + +time.Time { + if m != nil { + return m.PeriodReset +} + +return time.Time{ +} +} + +// AllowedMsgAllowance creates allowance only for specified message types. +type AllowedMsgAllowance struct { + // allowance can be any of basic and periodic fee allowance. + Allowance *types1.Any `protobuf:"bytes,1,opt,name=allowance,proto3" json:"allowance,omitempty"` + // allowed_messages are the messages for which the grantee has the access. + AllowedMessages []string `protobuf:"bytes,2,rep,name=allowed_messages,json=allowedMessages,proto3" json:"allowed_messages,omitempty"` +} + +func (m *AllowedMsgAllowance) + +Reset() { *m = AllowedMsgAllowance{ +} +} + +func (m *AllowedMsgAllowance) + +String() + +string { + return proto.CompactTextString(m) +} + +func (*AllowedMsgAllowance) + +ProtoMessage() { +} + +func (*AllowedMsgAllowance) + +Descriptor() ([]byte, []int) { + return fileDescriptor_7279582900c30aea, []int{2 +} +} + +func (m *AllowedMsgAllowance) + +XXX_Unmarshal(b []byte) + +error { + return m.Unmarshal(b) +} + +func (m *AllowedMsgAllowance) + +XXX_Marshal(b []byte, deterministic bool) ([]byte, error) { + if deterministic { + return xxx_messageInfo_AllowedMsgAllowance.Marshal(b, m, deterministic) +} + +else { + b = b[:cap(b)] + n, err := m.MarshalToSizedBuffer(b) + if err != nil { + return nil, err +} + +return b[:n], nil +} +} + +func (m *AllowedMsgAllowance) + +XXX_Merge(src proto.Message) { + xxx_messageInfo_AllowedMsgAllowance.Merge(m, src) +} + +func (m *AllowedMsgAllowance) + +XXX_Size() + +int { + return m.Size() +} + +func (m *AllowedMsgAllowance) + +XXX_DiscardUnknown() { + xxx_messageInfo_AllowedMsgAllowance.DiscardUnknown(m) +} + +var xxx_messageInfo_AllowedMsgAllowance proto.InternalMessageInfo + +// Grant is stored in the KVStore to record a grant with full context +type Grant struct { + // granter is the address of the user granting an allowance of their funds. + Granter string `protobuf:"bytes,1,opt,name=granter,proto3" json:"granter,omitempty"` + // grantee is the address of the user being granted an allowance of another user's funds. + Grantee string `protobuf:"bytes,2,opt,name=grantee,proto3" json:"grantee,omitempty"` + // allowance can be any of basic, periodic, allowed fee allowance. + Allowance *types1.Any `protobuf:"bytes,3,opt,name=allowance,proto3" json:"allowance,omitempty"` +} + +func (m *Grant) + +Reset() { *m = Grant{ +} +} + +func (m *Grant) + +String() + +string { + return proto.CompactTextString(m) +} + +func (*Grant) + +ProtoMessage() { +} + +func (*Grant) + +Descriptor() ([]byte, []int) { + return fileDescriptor_7279582900c30aea, []int{3 +} +} + +func (m *Grant) + +XXX_Unmarshal(b []byte) + +error { + return m.Unmarshal(b) +} + +func (m *Grant) + +XXX_Marshal(b []byte, deterministic bool) ([]byte, error) { + if deterministic { + return xxx_messageInfo_Grant.Marshal(b, m, deterministic) +} + +else { + b = b[:cap(b)] + n, err := m.MarshalToSizedBuffer(b) + if err != nil { + return nil, err +} + +return b[:n], nil +} +} + +func (m *Grant) + +XXX_Merge(src proto.Message) { + xxx_messageInfo_Grant.Merge(m, src) +} + +func (m *Grant) + +XXX_Size() + +int { + return m.Size() +} + +func (m *Grant) + +XXX_DiscardUnknown() { + xxx_messageInfo_Grant.DiscardUnknown(m) +} + +var xxx_messageInfo_Grant proto.InternalMessageInfo + +func (m *Grant) + +GetGranter() + +string { + if m != nil { + return m.Granter +} + +return "" +} + +func (m *Grant) + +GetGrantee() + +string { + if m != nil { + return m.Grantee +} + +return "" +} + +func (m *Grant) + +GetAllowance() *types1.Any { + if m != nil { + return m.Allowance +} + +return nil +} + +func init() { + proto.RegisterType((*BasicAllowance)(nil), "cosmos.feegrant.v1beta1.BasicAllowance") + +proto.RegisterType((*PeriodicAllowance)(nil), "cosmos.feegrant.v1beta1.PeriodicAllowance") + +proto.RegisterType((*AllowedMsgAllowance)(nil), "cosmos.feegrant.v1beta1.AllowedMsgAllowance") + +proto.RegisterType((*Grant)(nil), "cosmos.feegrant.v1beta1.Grant") +} + +func init() { + proto.RegisterFile("cosmos/feegrant/v1beta1/feegrant.proto", fileDescriptor_7279582900c30aea) +} + +var fileDescriptor_7279582900c30aea = []byte{ + // 639 bytes of a gzipped FileDescriptorProto + 0x1f, 0x8b, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0xff, 0xb4, 0x55, 0x3f, 0x6f, 0xd3, 0x40, + 0x14, 0x8f, 0x9b, 0xb6, 0x28, 0x17, 0x28, 0xad, 0xa9, 0x84, 0x53, 0x21, 0xbb, 0x8a, 0x04, 0x4d, + 0x2b, 0xd5, 0x56, 0x8b, 0x58, 0x3a, 0x35, 0x2e, 0xa2, 0x80, 0x5a, 0xa9, 0x72, 0x99, 0x90, 0x50, + 0x74, 0xb6, 0xaf, 0xe6, 0x44, 0xec, 0x33, 0x3e, 0x17, 0x1a, 0x06, 0x66, 0xc4, 0x80, 0x32, 0x32, + 0x32, 0x22, 0xa6, 0x0e, 0xe5, 0x3b, 0x54, 0x0c, 0xa8, 0x62, 0x62, 0x22, 0x28, 0x19, 0x3a, 0xf3, + 0x0d, 0x90, 0xef, 0xce, 0x8e, 0x9b, 0x50, 0x68, 0x25, 0xba, 0x24, 0x77, 0xef, 0xde, 0xfb, 0xfd, + 0x79, 0xef, 0x45, 0x01, 0xb7, 0x1c, 0x42, 0x7d, 0x42, 0x8d, 0x1d, 0x84, 0xbc, 0x08, 0x06, 0xb1, + 0xf1, 0x62, 0xc9, 0x46, 0x31, 0x5c, 0xca, 0x02, 0x7a, 0x18, 0x91, 0x98, 0xc8, 0xd7, 0x79, 0x9e, + 0x9e, 0x85, 0x45, 0xde, 0xcc, 0xb4, 0x47, 0x3c, 0xc2, 0x72, 0x8c, 0xe4, 0xc4, 0xd3, 0x67, 0x2a, + 0x1e, 0x21, 0x5e, 0x13, 0x19, 0xec, 0x66, 0xef, 0xee, 0x18, 0x30, 0x68, 0xa5, 0x4f, 0x1c, 0xa9, + 0xc1, 0x6b, 0x04, 0x2c, 0x7f, 0x52, 0x85, 0x18, 0x1b, 0x52, 0x94, 0x09, 0x71, 0x08, 0x0e, 0xc4, + 0xfb, 0x14, 0xf4, 0x71, 0x40, 0x0c, 0xf6, 0x29, 0x42, 0xda, 0x20, 0x51, 0x8c, 0x7d, 0x44, 0x63, + 0xe8, 0x87, 0x29, 0xe6, 0x60, 0x82, 0xbb, 0x1b, 0xc1, 0x18, 0x13, 0x81, 0x59, 0x7d, 0x37, 0x02, + 0x26, 0x4c, 0x48, 0xb1, 0x53, 0x6f, 0x36, 0xc9, 0x4b, 0x18, 0x38, 0x48, 0x7e, 0x0e, 0xca, 0x34, + 0x44, 0x81, 0xdb, 0x68, 0x62, 0x1f, 0xc7, 0x8a, 0x34, 0x5b, 0xac, 0x95, 0x97, 0x2b, 0xba, 0x90, + 0x9a, 0x88, 0x4b, 0xdd, 0xeb, 0x6b, 0x04, 0x07, 0xe6, 0x9d, 0xc3, 0x1f, 0x5a, 0xe1, 0x53, 0x47, + 0xab, 0x79, 0x38, 0x7e, 0xba, 0x6b, 0xeb, 0x0e, 0xf1, 0x85, 0x2f, 0xf1, 0xb5, 0x48, 0xdd, 0x67, + 0x46, 0xdc, 0x0a, 0x11, 0x65, 0x05, 0xf4, 0xe3, 0xf1, 0xfe, 0x82, 0x64, 0x01, 0x46, 0xb2, 0x91, + 0x70, 0xc8, 0xab, 0x00, 0xa0, 0xbd, 0x10, 0x73, 0x65, 0xca, 0xc8, 0xac, 0x54, 0x2b, 0x2f, 0xcf, + 0xe8, 0x5c, 0xba, 0x9e, 0x4a, 0xd7, 0x1f, 0xa5, 0xde, 0xcc, 0xd1, 0x76, 0x47, 0x93, 0xac, 0x5c, + 0xcd, 0xca, 0xfa, 0x97, 0x83, 0xc5, 0x9b, 0xa7, 0x0c, 0x49, 0xbf, 0x87, 0x50, 0x66, 0xef, 0xc1, + 0xdb, 0xe3, 0xfd, 0x85, 0x4a, 0x4e, 0xd8, 0x49, 0xf7, 0xd5, 0xcf, 0xa3, 0x60, 0x6a, 0x0b, 0x45, + 0x98, 0xb8, 0xf9, 0x9e, 0xdc, 0x07, 0x63, 0x76, 0x92, 0xa7, 0x48, 0x4c, 0xdb, 0x9c, 0x7e, 0x1a, + 0xd5, 0x49, 0x34, 0xb3, 0x94, 0xf4, 0x86, 0xfb, 0xe5, 0x00, 0xf2, 0x2a, 0x18, 0x0f, 0x19, 0xbc, + 0xb0, 0x59, 0x19, 0xb2, 0x79, 0x57, 0x4c, 0xc8, 0xbc, 0x92, 0x14, 0xbf, 0xef, 0x68, 0x12, 0x07, + 0x10, 0x75, 0xf2, 0x6b, 0x20, 0xf3, 0x53, 0x23, 0x3f, 0xa6, 0xe2, 0x05, 0x8d, 0x69, 0x92, 0x73, + 0x6d, 0xf7, 0x87, 0xf5, 0x0a, 0x88, 0x58, 0xc3, 0x81, 0x01, 0xd7, 0xa0, 0x8c, 0x5e, 0x10, 0xfb, + 0x04, 0x67, 0x5a, 0x83, 0x01, 0x13, 0x20, 0x6f, 0x80, 0xcb, 0x82, 0x3b, 0x42, 0x14, 0xc5, 0xca, + 0xd8, 0x3f, 0x57, 0x85, 0x35, 0xb1, 0x9d, 0x35, 0xb1, 0xcc, 0xcb, 0xad, 0xa4, 0x7a, 0xe5, 0xe1, + 0xb9, 0x96, 0xe6, 0x46, 0x4e, 0xe8, 0xd0, 0x86, 0x54, 0x7f, 0x49, 0xe0, 0x1a, 0xbb, 0x21, 0x77, + 0x93, 0x7a, 0xfd, 0xcd, 0x79, 0x02, 0x4a, 0x30, 0xbd, 0x88, 0xed, 0x99, 0x1e, 0x92, 0x5b, 0x0f, + 0x5a, 0xe6, 0xfc, 0x99, 0xc5, 0x58, 0x7d, 0x44, 0x79, 0x1e, 0x4c, 0x42, 0xce, 0xda, 0xf0, 0x11, + 0xa5, 0xd0, 0x43, 0x54, 0x19, 0x99, 0x2d, 0xd6, 0x4a, 0xd6, 0x55, 0x11, 0xdf, 0x14, 0xe1, 0x95, + 0xad, 0x37, 0x1f, 0xb4, 0xc2, 0xb9, 0x1c, 0xab, 0x39, 0xc7, 0x7f, 0xf0, 0x56, 0xfd, 0x2a, 0x81, + 0xb1, 0xf5, 0x04, 0x42, 0x5e, 0x06, 0x97, 0x18, 0x16, 0x8a, 0x98, 0xc7, 0x92, 0xa9, 0x7c, 0x3b, + 0x58, 0x9c, 0x16, 0x44, 0x75, 0xd7, 0x8d, 0x10, 0xa5, 0xdb, 0x71, 0x84, 0x03, 0xcf, 0x4a, 0x13, + 0xfb, 0x35, 0x88, 0xfd, 0x14, 0xce, 0x50, 0x33, 0xd0, 0xcd, 0xe2, 0xff, 0xee, 0xa6, 0x59, 0x3f, + 0xec, 0xaa, 0xd2, 0x51, 0x57, 0x95, 0x7e, 0x76, 0x55, 0xa9, 0xdd, 0x53, 0x0b, 0x47, 0x3d, 0xb5, + 0xf0, 0xbd, 0xa7, 0x16, 0x1e, 0xcf, 0xfd, 0x75, 0x6f, 0xf7, 0xb2, 0xff, 0x0b, 0x7b, 0x9c, 0xc9, + 0xb8, 0xfd, 0x3b, 0x00, 0x00, 0xff, 0xff, 0xe4, 0x3d, 0x09, 0x1d, 0x5a, 0x06, 0x00, 0x00, +} + +func (m *BasicAllowance) + +Marshal() (dAtA []byte, err error) { + size := m.Size() + +dAtA = make([]byte, size) + +n, err := m.MarshalToSizedBuffer(dAtA[:size]) + if err != nil { + return nil, err +} + +return dAtA[:n], nil +} + +func (m *BasicAllowance) + +MarshalTo(dAtA []byte) (int, error) { + size := m.Size() + +return m.MarshalToSizedBuffer(dAtA[:size]) +} + +func (m *BasicAllowance) + +MarshalToSizedBuffer(dAtA []byte) (int, error) { + i := len(dAtA) + _ = i + var l int + _ = l + if m.Expiration != nil { + n1, err1 := github_com_cosmos_gogoproto_types.StdTimeMarshalTo(*m.Expiration, dAtA[i-github_com_cosmos_gogoproto_types.SizeOfStdTime(*m.Expiration):]) + if err1 != nil { + return 0, err1 +} + +i -= n1 + i = encodeVarintFeegrant(dAtA, i, uint64(n1)) + +i-- + dAtA[i] = 0x12 +} + if len(m.SpendLimit) > 0 { + for iNdEx := len(m.SpendLimit) - 1; iNdEx >= 0; iNdEx-- { + { + size, err := m.SpendLimit[iNdEx].MarshalToSizedBuffer(dAtA[:i]) + if err != nil { + return 0, err +} + +i -= size + i = encodeVarintFeegrant(dAtA, i, uint64(size)) +} + +i-- + dAtA[i] = 0xa +} + +} + +return len(dAtA) - i, nil +} + +func (m *PeriodicAllowance) + +Marshal() (dAtA []byte, err error) { + size := m.Size() + +dAtA = make([]byte, size) + +n, err := m.MarshalToSizedBuffer(dAtA[:size]) + if err != nil { + return nil, err +} + +return dAtA[:n], nil +} + +func (m *PeriodicAllowance) + +MarshalTo(dAtA []byte) (int, error) { + size := m.Size() + +return m.MarshalToSizedBuffer(dAtA[:size]) +} + +func (m *PeriodicAllowance) + +MarshalToSizedBuffer(dAtA []byte) (int, error) { + i := len(dAtA) + _ = i + var l int + _ = l + n2, err2 := github_com_cosmos_gogoproto_types.StdTimeMarshalTo(m.PeriodReset, dAtA[i-github_com_cosmos_gogoproto_types.SizeOfStdTime(m.PeriodReset):]) + if err2 != nil { + return 0, err2 +} + +i -= n2 + i = encodeVarintFeegrant(dAtA, i, uint64(n2)) + +i-- + dAtA[i] = 0x2a + if len(m.PeriodCanSpend) > 0 { + for iNdEx := len(m.PeriodCanSpend) - 1; iNdEx >= 0; iNdEx-- { + { + size, err := m.PeriodCanSpend[iNdEx].MarshalToSizedBuffer(dAtA[:i]) + if err != nil { + return 0, err +} + +i -= size + i = encodeVarintFeegrant(dAtA, i, uint64(size)) +} + +i-- + dAtA[i] = 0x22 +} + +} + if len(m.PeriodSpendLimit) > 0 { + for iNdEx := len(m.PeriodSpendLimit) - 1; iNdEx >= 0; iNdEx-- { + { + size, err := m.PeriodSpendLimit[iNdEx].MarshalToSizedBuffer(dAtA[:i]) + if err != nil { + return 0, err +} + +i -= size + i = encodeVarintFeegrant(dAtA, i, uint64(size)) +} + +i-- + dAtA[i] = 0x1a +} + +} + +n3, err3 := github_com_cosmos_gogoproto_types.StdDurationMarshalTo(m.Period, dAtA[i-github_com_cosmos_gogoproto_types.SizeOfStdDuration(m.Period):]) + if err3 != nil { + return 0, err3 +} + +i -= n3 + i = encodeVarintFeegrant(dAtA, i, uint64(n3)) + +i-- + dAtA[i] = 0x12 + { + size, err := m.Basic.MarshalToSizedBuffer(dAtA[:i]) + if err != nil { + return 0, err +} + +i -= size + i = encodeVarintFeegrant(dAtA, i, uint64(size)) +} + +i-- + dAtA[i] = 0xa + return len(dAtA) - i, nil +} + +func (m *AllowedMsgAllowance) + +Marshal() (dAtA []byte, err error) { + size := m.Size() + +dAtA = make([]byte, size) + +n, err := m.MarshalToSizedBuffer(dAtA[:size]) + if err != nil { + return nil, err +} + +return dAtA[:n], nil +} + +func (m *AllowedMsgAllowance) + +MarshalTo(dAtA []byte) (int, error) { + size := m.Size() + +return m.MarshalToSizedBuffer(dAtA[:size]) +} + +func (m *AllowedMsgAllowance) + +MarshalToSizedBuffer(dAtA []byte) (int, error) { + i := len(dAtA) + _ = i + var l int + _ = l + if len(m.AllowedMessages) > 0 { + for iNdEx := len(m.AllowedMessages) - 1; iNdEx >= 0; iNdEx-- { + i -= len(m.AllowedMessages[iNdEx]) + +copy(dAtA[i:], m.AllowedMessages[iNdEx]) + +i = encodeVarintFeegrant(dAtA, i, uint64(len(m.AllowedMessages[iNdEx]))) + +i-- + dAtA[i] = 0x12 +} + +} + if m.Allowance != nil { + { + size, err := m.Allowance.MarshalToSizedBuffer(dAtA[:i]) + if err != nil { + return 0, err +} + +i -= size + i = encodeVarintFeegrant(dAtA, i, uint64(size)) +} + +i-- + dAtA[i] = 0xa +} + +return len(dAtA) - i, nil +} + +func (m *Grant) + +Marshal() (dAtA []byte, err error) { + size := m.Size() + +dAtA = make([]byte, size) + +n, err := m.MarshalToSizedBuffer(dAtA[:size]) + if err != nil { + return nil, err +} + +return dAtA[:n], nil +} + +func (m *Grant) + +MarshalTo(dAtA []byte) (int, error) { + size := m.Size() + +return m.MarshalToSizedBuffer(dAtA[:size]) +} + +func (m *Grant) + +MarshalToSizedBuffer(dAtA []byte) (int, error) { + i := len(dAtA) + _ = i + var l int + _ = l + if m.Allowance != nil { + { + size, err := m.Allowance.MarshalToSizedBuffer(dAtA[:i]) + if err != nil { + return 0, err +} + +i -= size + i = encodeVarintFeegrant(dAtA, i, uint64(size)) +} + +i-- + dAtA[i] = 0x1a +} + if len(m.Grantee) > 0 { + i -= len(m.Grantee) + +copy(dAtA[i:], m.Grantee) + +i = encodeVarintFeegrant(dAtA, i, uint64(len(m.Grantee))) + +i-- + dAtA[i] = 0x12 +} + if len(m.Granter) > 0 { + i -= len(m.Granter) + +copy(dAtA[i:], m.Granter) + +i = encodeVarintFeegrant(dAtA, i, uint64(len(m.Granter))) + +i-- + dAtA[i] = 0xa +} + +return len(dAtA) - i, nil +} + +func encodeVarintFeegrant(dAtA []byte, offset int, v uint64) + +int { + offset -= sovFeegrant(v) + base := offset + for v >= 1<<7 { + dAtA[offset] = uint8(v&0x7f | 0x80) + +v >>= 7 + offset++ +} + +dAtA[offset] = uint8(v) + +return base +} + +func (m *BasicAllowance) + +Size() (n int) { + if m == nil { + return 0 +} + +var l int + _ = l + if len(m.SpendLimit) > 0 { + for _, e := range m.SpendLimit { + l = e.Size() + +n += 1 + l + sovFeegrant(uint64(l)) +} + +} + if m.Expiration != nil { + l = github_com_cosmos_gogoproto_types.SizeOfStdTime(*m.Expiration) + +n += 1 + l + sovFeegrant(uint64(l)) +} + +return n +} + +func (m *PeriodicAllowance) + +Size() (n int) { + if m == nil { + return 0 +} + +var l int + _ = l + l = m.Basic.Size() + +n += 1 + l + sovFeegrant(uint64(l)) + +l = github_com_cosmos_gogoproto_types.SizeOfStdDuration(m.Period) + +n += 1 + l + sovFeegrant(uint64(l)) + if len(m.PeriodSpendLimit) > 0 { + for _, e := range m.PeriodSpendLimit { + l = e.Size() + +n += 1 + l + sovFeegrant(uint64(l)) +} + +} + if len(m.PeriodCanSpend) > 0 { + for _, e := range m.PeriodCanSpend { + l = e.Size() + +n += 1 + l + sovFeegrant(uint64(l)) +} + +} + +l = github_com_cosmos_gogoproto_types.SizeOfStdTime(m.PeriodReset) + +n += 1 + l + sovFeegrant(uint64(l)) + +return n +} + +func (m *AllowedMsgAllowance) + +Size() (n int) { + if m == nil { + return 0 +} + +var l int + _ = l + if m.Allowance != nil { + l = m.Allowance.Size() + +n += 1 + l + sovFeegrant(uint64(l)) +} + if len(m.AllowedMessages) > 0 { + for _, s := range m.AllowedMessages { + l = len(s) + +n += 1 + l + sovFeegrant(uint64(l)) +} + +} + +return n +} + +func (m *Grant) + +Size() (n int) { + if m == nil { + return 0 +} + +var l int + _ = l + l = len(m.Granter) + if l > 0 { + n += 1 + l + sovFeegrant(uint64(l)) +} + +l = len(m.Grantee) + if l > 0 { + n += 1 + l + sovFeegrant(uint64(l)) +} + if m.Allowance != nil { + l = m.Allowance.Size() + +n += 1 + l + sovFeegrant(uint64(l)) +} + +return n +} + +func sovFeegrant(x uint64) (n int) { + return (math_bits.Len64(x|1) + 6) / 7 +} + +func sozFeegrant(x uint64) (n int) { + return sovFeegrant(uint64((x << 1) ^ uint64((int64(x) >> 63)))) +} + +func (m *BasicAllowance) + +Unmarshal(dAtA []byte) + +error { + l := len(dAtA) + iNdEx := 0 + for iNdEx < l { + preIndex := iNdEx + var wire uint64 + for shift := uint(0); ; shift += 7 { + if shift >= 64 { + return ErrIntOverflowFeegrant +} + if iNdEx >= l { + return io.ErrUnexpectedEOF +} + b := dAtA[iNdEx] + iNdEx++ + wire |= uint64(b&0x7F) << shift + if b < 0x80 { + break +} + +} + fieldNum := int32(wire >> 3) + wireType := int(wire & 0x7) + if wireType == 4 { + return fmt.Errorf("proto: BasicAllowance: wiretype end group for non-group") +} + if fieldNum <= 0 { + return fmt.Errorf("proto: BasicAllowance: illegal tag %d (wire type %d)", fieldNum, wire) +} + switch fieldNum { + case 1: + if wireType != 2 { + return fmt.Errorf("proto: wrong wireType = %d for field SpendLimit", wireType) +} + +var msglen int + for shift := uint(0); ; shift += 7 { + if shift >= 64 { + return ErrIntOverflowFeegrant +} + if iNdEx >= l { + return io.ErrUnexpectedEOF +} + b := dAtA[iNdEx] + iNdEx++ + msglen |= int(b&0x7F) << shift + if b < 0x80 { + break +} + +} + if msglen < 0 { + return ErrInvalidLengthFeegrant +} + postIndex := iNdEx + msglen + if postIndex < 0 { + return ErrInvalidLengthFeegrant +} + if postIndex > l { + return io.ErrUnexpectedEOF +} + +m.SpendLimit = append(m.SpendLimit, types.Coin{ +}) + if err := m.SpendLimit[len(m.SpendLimit)-1].Unmarshal(dAtA[iNdEx:postIndex]); err != nil { + return err +} + +iNdEx = postIndex + case 2: + if wireType != 2 { + return fmt.Errorf("proto: wrong wireType = %d for field Expiration", wireType) +} + +var msglen int + for shift := uint(0); ; shift += 7 { + if shift >= 64 { + return ErrIntOverflowFeegrant +} + if iNdEx >= l { + return io.ErrUnexpectedEOF +} + b := dAtA[iNdEx] + iNdEx++ + msglen |= int(b&0x7F) << shift + if b < 0x80 { + break +} + +} + if msglen < 0 { + return ErrInvalidLengthFeegrant +} + postIndex := iNdEx + msglen + if postIndex < 0 { + return ErrInvalidLengthFeegrant +} + if postIndex > l { + return io.ErrUnexpectedEOF +} + if m.Expiration == nil { + m.Expiration = new(time.Time) +} + if err := github_com_cosmos_gogoproto_types.StdTimeUnmarshal(m.Expiration, dAtA[iNdEx:postIndex]); err != nil { + return err +} + +iNdEx = postIndex + default: + iNdEx = preIndex + skippy, err := skipFeegrant(dAtA[iNdEx:]) + if err != nil { + return err +} + if (skippy < 0) || (iNdEx+skippy) < 0 { + return ErrInvalidLengthFeegrant +} + if (iNdEx + skippy) > l { + return io.ErrUnexpectedEOF +} + +iNdEx += skippy +} + +} + if iNdEx > l { + return io.ErrUnexpectedEOF +} + +return nil +} + +func (m *PeriodicAllowance) + +Unmarshal(dAtA []byte) + +error { + l := len(dAtA) + iNdEx := 0 + for iNdEx < l { + preIndex := iNdEx + var wire uint64 + for shift := uint(0); ; shift += 7 { + if shift >= 64 { + return ErrIntOverflowFeegrant +} + if iNdEx >= l { + return io.ErrUnexpectedEOF +} + b := dAtA[iNdEx] + iNdEx++ + wire |= uint64(b&0x7F) << shift + if b < 0x80 { + break +} + +} + fieldNum := int32(wire >> 3) + wireType := int(wire & 0x7) + if wireType == 4 { + return fmt.Errorf("proto: PeriodicAllowance: wiretype end group for non-group") +} + if fieldNum <= 0 { + return fmt.Errorf("proto: PeriodicAllowance: illegal tag %d (wire type %d)", fieldNum, wire) +} + switch fieldNum { + case 1: + if wireType != 2 { + return fmt.Errorf("proto: wrong wireType = %d for field Basic", wireType) +} + +var msglen int + for shift := uint(0); ; shift += 7 { + if shift >= 64 { + return ErrIntOverflowFeegrant +} + if iNdEx >= l { + return io.ErrUnexpectedEOF +} + b := dAtA[iNdEx] + iNdEx++ + msglen |= int(b&0x7F) << shift + if b < 0x80 { + break +} + +} + if msglen < 0 { + return ErrInvalidLengthFeegrant +} + postIndex := iNdEx + msglen + if postIndex < 0 { + return ErrInvalidLengthFeegrant +} + if postIndex > l { + return io.ErrUnexpectedEOF +} + if err := m.Basic.Unmarshal(dAtA[iNdEx:postIndex]); err != nil { + return err +} + +iNdEx = postIndex + case 2: + if wireType != 2 { + return fmt.Errorf("proto: wrong wireType = %d for field Period", wireType) +} + +var msglen int + for shift := uint(0); ; shift += 7 { + if shift >= 64 { + return ErrIntOverflowFeegrant +} + if iNdEx >= l { + return io.ErrUnexpectedEOF +} + b := dAtA[iNdEx] + iNdEx++ + msglen |= int(b&0x7F) << shift + if b < 0x80 { + break +} + +} + if msglen < 0 { + return ErrInvalidLengthFeegrant +} + postIndex := iNdEx + msglen + if postIndex < 0 { + return ErrInvalidLengthFeegrant +} + if postIndex > l { + return io.ErrUnexpectedEOF +} + if err := github_com_cosmos_gogoproto_types.StdDurationUnmarshal(&m.Period, dAtA[iNdEx:postIndex]); err != nil { + return err +} + +iNdEx = postIndex + case 3: + if wireType != 2 { + return fmt.Errorf("proto: wrong wireType = %d for field PeriodSpendLimit", wireType) +} + +var msglen int + for shift := uint(0); ; shift += 7 { + if shift >= 64 { + return ErrIntOverflowFeegrant +} + if iNdEx >= l { + return io.ErrUnexpectedEOF +} + b := dAtA[iNdEx] + iNdEx++ + msglen |= int(b&0x7F) << shift + if b < 0x80 { + break +} + +} + if msglen < 0 { + return ErrInvalidLengthFeegrant +} + postIndex := iNdEx + msglen + if postIndex < 0 { + return ErrInvalidLengthFeegrant +} + if postIndex > l { + return io.ErrUnexpectedEOF +} + +m.PeriodSpendLimit = append(m.PeriodSpendLimit, types.Coin{ +}) + if err := m.PeriodSpendLimit[len(m.PeriodSpendLimit)-1].Unmarshal(dAtA[iNdEx:postIndex]); err != nil { + return err +} + +iNdEx = postIndex + case 4: + if wireType != 2 { + return fmt.Errorf("proto: wrong wireType = %d for field PeriodCanSpend", wireType) +} + +var msglen int + for shift := uint(0); ; shift += 7 { + if shift >= 64 { + return ErrIntOverflowFeegrant +} + if iNdEx >= l { + return io.ErrUnexpectedEOF +} + b := dAtA[iNdEx] + iNdEx++ + msglen |= int(b&0x7F) << shift + if b < 0x80 { + break +} + +} + if msglen < 0 { + return ErrInvalidLengthFeegrant +} + postIndex := iNdEx + msglen + if postIndex < 0 { + return ErrInvalidLengthFeegrant +} + if postIndex > l { + return io.ErrUnexpectedEOF +} + +m.PeriodCanSpend = append(m.PeriodCanSpend, types.Coin{ +}) + if err := m.PeriodCanSpend[len(m.PeriodCanSpend)-1].Unmarshal(dAtA[iNdEx:postIndex]); err != nil { + return err +} + +iNdEx = postIndex + case 5: + if wireType != 2 { + return fmt.Errorf("proto: wrong wireType = %d for field PeriodReset", wireType) +} + +var msglen int + for shift := uint(0); ; shift += 7 { + if shift >= 64 { + return ErrIntOverflowFeegrant +} + if iNdEx >= l { + return io.ErrUnexpectedEOF +} + b := dAtA[iNdEx] + iNdEx++ + msglen |= int(b&0x7F) << shift + if b < 0x80 { + break +} + +} + if msglen < 0 { + return ErrInvalidLengthFeegrant +} + postIndex := iNdEx + msglen + if postIndex < 0 { + return ErrInvalidLengthFeegrant +} + if postIndex > l { + return io.ErrUnexpectedEOF +} + if err := github_com_cosmos_gogoproto_types.StdTimeUnmarshal(&m.PeriodReset, dAtA[iNdEx:postIndex]); err != nil { + return err +} + +iNdEx = postIndex + default: + iNdEx = preIndex + skippy, err := skipFeegrant(dAtA[iNdEx:]) + if err != nil { + return err +} + if (skippy < 0) || (iNdEx+skippy) < 0 { + return ErrInvalidLengthFeegrant +} + if (iNdEx + skippy) > l { + return io.ErrUnexpectedEOF +} + +iNdEx += skippy +} + +} + if iNdEx > l { + return io.ErrUnexpectedEOF +} + +return nil +} + +func (m *AllowedMsgAllowance) + +Unmarshal(dAtA []byte) + +error { + l := len(dAtA) + iNdEx := 0 + for iNdEx < l { + preIndex := iNdEx + var wire uint64 + for shift := uint(0); ; shift += 7 { + if shift >= 64 { + return ErrIntOverflowFeegrant +} + if iNdEx >= l { + return io.ErrUnexpectedEOF +} + b := dAtA[iNdEx] + iNdEx++ + wire |= uint64(b&0x7F) << shift + if b < 0x80 { + break +} + +} + fieldNum := int32(wire >> 3) + wireType := int(wire & 0x7) + if wireType == 4 { + return fmt.Errorf("proto: AllowedMsgAllowance: wiretype end group for non-group") +} + if fieldNum <= 0 { + return fmt.Errorf("proto: AllowedMsgAllowance: illegal tag %d (wire type %d)", fieldNum, wire) +} + switch fieldNum { + case 1: + if wireType != 2 { + return fmt.Errorf("proto: wrong wireType = %d for field Allowance", wireType) +} + +var msglen int + for shift := uint(0); ; shift += 7 { + if shift >= 64 { + return ErrIntOverflowFeegrant +} + if iNdEx >= l { + return io.ErrUnexpectedEOF +} + b := dAtA[iNdEx] + iNdEx++ + msglen |= int(b&0x7F) << shift + if b < 0x80 { + break +} + +} + if msglen < 0 { + return ErrInvalidLengthFeegrant +} + postIndex := iNdEx + msglen + if postIndex < 0 { + return ErrInvalidLengthFeegrant +} + if postIndex > l { + return io.ErrUnexpectedEOF +} + if m.Allowance == nil { + m.Allowance = &types1.Any{ +} + +} + if err := m.Allowance.Unmarshal(dAtA[iNdEx:postIndex]); err != nil { + return err +} + +iNdEx = postIndex + case 2: + if wireType != 2 { + return fmt.Errorf("proto: wrong wireType = %d for field AllowedMessages", wireType) +} + +var stringLen uint64 + for shift := uint(0); ; shift += 7 { + if shift >= 64 { + return ErrIntOverflowFeegrant +} + if iNdEx >= l { + return io.ErrUnexpectedEOF +} + b := dAtA[iNdEx] + iNdEx++ + stringLen |= uint64(b&0x7F) << shift + if b < 0x80 { + break +} + +} + intStringLen := int(stringLen) + if intStringLen < 0 { + return ErrInvalidLengthFeegrant +} + postIndex := iNdEx + intStringLen + if postIndex < 0 { + return ErrInvalidLengthFeegrant +} + if postIndex > l { + return io.ErrUnexpectedEOF +} + +m.AllowedMessages = append(m.AllowedMessages, string(dAtA[iNdEx:postIndex])) + +iNdEx = postIndex + default: + iNdEx = preIndex + skippy, err := skipFeegrant(dAtA[iNdEx:]) + if err != nil { + return err +} + if (skippy < 0) || (iNdEx+skippy) < 0 { + return ErrInvalidLengthFeegrant +} + if (iNdEx + skippy) > l { + return io.ErrUnexpectedEOF +} + +iNdEx += skippy +} + +} + if iNdEx > l { + return io.ErrUnexpectedEOF +} + +return nil +} + +func (m *Grant) + +Unmarshal(dAtA []byte) + +error { + l := len(dAtA) + iNdEx := 0 + for iNdEx < l { + preIndex := iNdEx + var wire uint64 + for shift := uint(0); ; shift += 7 { + if shift >= 64 { + return ErrIntOverflowFeegrant +} + if iNdEx >= l { + return io.ErrUnexpectedEOF +} + b := dAtA[iNdEx] + iNdEx++ + wire |= uint64(b&0x7F) << shift + if b < 0x80 { + break +} + +} + fieldNum := int32(wire >> 3) + wireType := int(wire & 0x7) + if wireType == 4 { + return fmt.Errorf("proto: Grant: wiretype end group for non-group") +} + if fieldNum <= 0 { + return fmt.Errorf("proto: Grant: illegal tag %d (wire type %d)", fieldNum, wire) +} + switch fieldNum { + case 1: + if wireType != 2 { + return fmt.Errorf("proto: wrong wireType = %d for field Granter", wireType) +} + +var stringLen uint64 + for shift := uint(0); ; shift += 7 { + if shift >= 64 { + return ErrIntOverflowFeegrant +} + if iNdEx >= l { + return io.ErrUnexpectedEOF +} + b := dAtA[iNdEx] + iNdEx++ + stringLen |= uint64(b&0x7F) << shift + if b < 0x80 { + break +} + +} + intStringLen := int(stringLen) + if intStringLen < 0 { + return ErrInvalidLengthFeegrant +} + postIndex := iNdEx + intStringLen + if postIndex < 0 { + return ErrInvalidLengthFeegrant +} + if postIndex > l { + return io.ErrUnexpectedEOF +} + +m.Granter = string(dAtA[iNdEx:postIndex]) + +iNdEx = postIndex + case 2: + if wireType != 2 { + return fmt.Errorf("proto: wrong wireType = %d for field Grantee", wireType) +} + +var stringLen uint64 + for shift := uint(0); ; shift += 7 { + if shift >= 64 { + return ErrIntOverflowFeegrant +} + if iNdEx >= l { + return io.ErrUnexpectedEOF +} + b := dAtA[iNdEx] + iNdEx++ + stringLen |= uint64(b&0x7F) << shift + if b < 0x80 { + break +} + +} + intStringLen := int(stringLen) + if intStringLen < 0 { + return ErrInvalidLengthFeegrant +} + postIndex := iNdEx + intStringLen + if postIndex < 0 { + return ErrInvalidLengthFeegrant +} + if postIndex > l { + return io.ErrUnexpectedEOF +} + +m.Grantee = string(dAtA[iNdEx:postIndex]) + +iNdEx = postIndex + case 3: + if wireType != 2 { + return fmt.Errorf("proto: wrong wireType = %d for field Allowance", wireType) +} + +var msglen int + for shift := uint(0); ; shift += 7 { + if shift >= 64 { + return ErrIntOverflowFeegrant +} + if iNdEx >= l { + return io.ErrUnexpectedEOF +} + b := dAtA[iNdEx] + iNdEx++ + msglen |= int(b&0x7F) << shift + if b < 0x80 { + break +} + +} + if msglen < 0 { + return ErrInvalidLengthFeegrant +} + postIndex := iNdEx + msglen + if postIndex < 0 { + return ErrInvalidLengthFeegrant +} + if postIndex > l { + return io.ErrUnexpectedEOF +} + if m.Allowance == nil { + m.Allowance = &types1.Any{ +} + +} + if err := m.Allowance.Unmarshal(dAtA[iNdEx:postIndex]); err != nil { + return err +} + +iNdEx = postIndex + default: + iNdEx = preIndex + skippy, err := skipFeegrant(dAtA[iNdEx:]) + if err != nil { + return err +} + if (skippy < 0) || (iNdEx+skippy) < 0 { + return ErrInvalidLengthFeegrant +} + if (iNdEx + skippy) > l { + return io.ErrUnexpectedEOF +} + +iNdEx += skippy +} + +} + if iNdEx > l { + return io.ErrUnexpectedEOF +} + +return nil +} + +func skipFeegrant(dAtA []byte) (n int, err error) { + l := len(dAtA) + iNdEx := 0 + depth := 0 + for iNdEx < l { + var wire uint64 + for shift := uint(0); ; shift += 7 { + if shift >= 64 { + return 0, ErrIntOverflowFeegrant +} + if iNdEx >= l { + return 0, io.ErrUnexpectedEOF +} + b := dAtA[iNdEx] + iNdEx++ + wire |= (uint64(b) & 0x7F) << shift + if b < 0x80 { + break +} + +} + wireType := int(wire & 0x7) + switch wireType { + case 0: + for shift := uint(0); ; shift += 7 { + if shift >= 64 { + return 0, ErrIntOverflowFeegrant +} + if iNdEx >= l { + return 0, io.ErrUnexpectedEOF +} + +iNdEx++ + if dAtA[iNdEx-1] < 0x80 { + break +} + +} + case 1: + iNdEx += 8 + case 2: + var length int + for shift := uint(0); ; shift += 7 { + if shift >= 64 { + return 0, ErrIntOverflowFeegrant +} + if iNdEx >= l { + return 0, io.ErrUnexpectedEOF +} + b := dAtA[iNdEx] + iNdEx++ + length |= (int(b) & 0x7F) << shift + if b < 0x80 { + break +} + +} + if length < 0 { + return 0, ErrInvalidLengthFeegrant +} + +iNdEx += length + case 3: + depth++ + case 4: + if depth == 0 { + return 0, ErrUnexpectedEndOfGroupFeegrant +} + +depth-- + case 5: + iNdEx += 4 + default: + return 0, fmt.Errorf("proto: illegal wireType %d", wireType) +} + if iNdEx < 0 { + return 0, ErrInvalidLengthFeegrant +} + if depth == 0 { + return iNdEx, nil +} + +} + +return 0, io.ErrUnexpectedEOF +} + +var ( + ErrInvalidLengthFeegrant = fmt.Errorf("proto: negative length found during unmarshaling") + +ErrIntOverflowFeegrant = fmt.Errorf("proto: integer overflow") + +ErrUnexpectedEndOfGroupFeegrant = fmt.Errorf("proto: unexpected end of group") +) +``` + +### FeeAllowanceQueue + +Fee Allowances queue items are identified by combining the `FeeAllowancePrefixQueue` (i.e., 0x01), `expiration`, `grantee` (the account address of fee allowance grantee), `granter` (the account address of fee allowance granter). Endblocker checks `FeeAllowanceQueue` state for the expired grants and prunes them from `FeeAllowance` if there are any found. + +Fee allowance queue keys are stored in the state as follows: + +* Grant: `0x01 | expiration_bytes | grantee_addr_len (1 byte) | grantee_addr_bytes | granter_addr_len (1 byte) | granter_addr_bytes -> EmptyBytes` + +## Messages + +### Msg/GrantAllowance + +A fee allowance grant will be created with the `MsgGrantAllowance` message. + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/feegrant/v1beta1/tx.proto#L25-L39 +``` + +### Msg/RevokeAllowance + +An allowed grant fee allowance can be removed with the `MsgRevokeAllowance` message. + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/feegrant/v1beta1/tx.proto#L41-L54 +``` + +## Events + +The feegrant module emits the following events: + +## Msg Server + +### MsgGrantAllowance + +| Type | Attribute Key | Attribute Value | +| ------- | ------------- | ---------------- | +| message | action | set\_feegrant | +| message | granter | `{granterAddress}` | +| message | grantee | `{granteeAddress}` | + +### MsgRevokeAllowance + +| Type | Attribute Key | Attribute Value | +| ------- | ------------- | ---------------- | +| message | action | revoke\_feegrant | +| message | granter | `{granterAddress}` | +| message | grantee | `{granteeAddress}` | + +### Exec fee allowance + +| Type | Attribute Key | Attribute Value | +| ------- | ------------- | ---------------- | +| message | action | use\_feegrant | +| message | granter | `{granterAddress}` | +| message | grantee | `{granteeAddress}` | + +### Prune fee allowances + +| Type | Attribute Key | Attribute Value | +| ------- | ------------- | --------------- | +| message | action | prune\_feegrant | +| message | pruner | `{prunerAddress}` | + +## Client + +### CLI + +A user can query and interact with the `feegrant` module using the CLI. + +#### Query + +The `query` commands allow users to query `feegrant` state. + +```shell +simd query feegrant --help +``` + +##### grant + +The `grant` command allows users to query a grant for a given granter-grantee pair. + +```shell +simd query feegrant grant [granter] [grantee] [flags] +``` + +Example: + +```shell +simd query feegrant grant cosmos1.. cosmos1.. +``` + +Example Output: + +```yml +allowance: + '@type': /cosmos.feegrant.v1beta1.BasicAllowance + expiration: null + spend_limit: + - amount: "100" + denom: stake +grantee: cosmos1.. +granter: cosmos1.. +``` + +##### grants + +The `grants` command allows users to query all grants for a given grantee. + +```shell +simd query feegrant grants [grantee] [flags] +``` + +Example: + +```shell +simd query feegrant grants cosmos1.. +``` + +Example Output: + +```yml expandable +allowances: +- allowance: + '@type': /cosmos.feegrant.v1beta1.BasicAllowance + expiration: null + spend_limit: + - amount: "100" + denom: stake + grantee: cosmos1.. + granter: cosmos1.. +pagination: + next_key: null + total: "0" +``` + +#### Transactions + +The `tx` commands allow users to interact with the `feegrant` module. + +```shell +simd tx feegrant --help +``` + +##### grant + +The `grant` command allows users to grant fee allowances to another account. The fee allowance can have an expiration date, a total spend limit, and/or a periodic spend limit. + +```shell +simd tx feegrant grant [granter] [grantee] [flags] +``` + +Example (one-time spend limit): + +```shell +simd tx feegrant grant cosmos1.. cosmos1.. --spend-limit 100stake +``` + +Example (periodic spend limit): + +```shell +simd tx feegrant grant cosmos1.. cosmos1.. --period 3600 --period-limit 10stake +``` + +##### revoke + +The `revoke` command allows users to revoke a granted fee allowance. + +```shell +simd tx feegrant revoke [granter] [grantee] [flags] +``` + +Example: + +```shell +simd tx feegrant revoke cosmos1.. cosmos1.. +``` + +### gRPC + +A user can query the `feegrant` module using gRPC endpoints. + +#### Allowance + +The `Allowance` endpoint allows users to query a granted fee allowance. + +```shell +cosmos.feegrant.v1beta1.Query/Allowance +``` + +Example: + +```shell +grpcurl -plaintext \ + -d '{"grantee":"cosmos1..","granter":"cosmos1.."}' \ + localhost:9090 \ + cosmos.feegrant.v1beta1.Query/Allowance +``` + +Example Output: + +```json +{ + "allowance": { + "granter": "cosmos1..", + "grantee": "cosmos1..", + "allowance": { + "@type": "/cosmos.feegrant.v1beta1.BasicAllowance", + "spendLimit": [ + { + "denom": "stake", + "amount": "100" + } + ] + } + } +} +``` + +#### Allowances + +The `Allowances` endpoint allows users to query all granted fee allowances for a given grantee. + +```shell +cosmos.feegrant.v1beta1.Query/Allowances +``` + +Example: + +```shell +grpcurl -plaintext \ + -d '{"address":"cosmos1.."}' \ + localhost:9090 \ + cosmos.feegrant.v1beta1.Query/Allowances +``` + +Example Output: + +```json expandable +{ + "allowances": [ + { + "granter": "cosmos1..", + "grantee": "cosmos1..", + "allowance": { + "@type": "/cosmos.feegrant.v1beta1.BasicAllowance", + "spendLimit": [ + { + "denom": "stake", + "amount": "100" + } + ] + } + } + ], + "pagination": { + "total": "1" + } +} +``` diff --git a/sdk/v0.54/modules/genutil/README.mdx b/sdk/v0.54/modules/genutil/README.mdx new file mode 100644 index 000000000..f6553586f --- /dev/null +++ b/sdk/v0.54/modules/genutil/README.mdx @@ -0,0 +1,1253 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/modules/genutil/README' +title: 'x/genutil' +description: >- + The genutil package contains a variety of genesis utility functionalities for + usage within a blockchain application. Namely: +--- + +## Concepts + +The `genutil` package contains a variety of genesis utility functionalities for usage within a blockchain application. Namely: + +* Genesis transactions related (gentx) +* Commands for collection and creation of gentxs +* `InitChain` processing of gentxs +* Genesis file creation +* Genesis file validation +* Genesis file migration +* CometBFT related initialization + * Translation of an app genesis to a CometBFT genesis + +## Genesis + +Genutil contains the data structure that defines an application genesis. +An application genesis consist of a consensus genesis (g.e. CometBFT genesis) and application related genesis data. + +```go expandable +package types + +import ( + + "bytes" + "encoding/json" + "errors" + "fmt" + "os" + "time" + + cmtjson "github.com/cometbft/cometbft/libs/json" + cmtproto "github.com/cometbft/cometbft/proto/tendermint/types" + cmttypes "github.com/cometbft/cometbft/types" + cmttime "github.com/cometbft/cometbft/types/time" + "github.com/cosmos/cosmos-sdk/version" +) + +const ( + // MaxChainIDLen is the maximum length of a chain ID. + MaxChainIDLen = cmttypes.MaxChainIDLen +) + +// AppGenesis defines the app's genesis. +type AppGenesis struct { + AppName string `json:"app_name"` + AppVersion string `json:"app_version"` + GenesisTime time.Time `json:"genesis_time"` + ChainID string `json:"chain_id"` + InitialHeight int64 `json:"initial_height"` + AppHash []byte `json:"app_hash"` + AppState json.RawMessage `json:"app_state,omitempty"` + Consensus *ConsensusGenesis `json:"consensus,omitempty"` +} + +// NewAppGenesisWithVersion returns a new AppGenesis with the app name and app version already. +func NewAppGenesisWithVersion(chainID string, appState json.RawMessage) *AppGenesis { + return &AppGenesis{ + AppName: version.AppName, + AppVersion: version.Version, + ChainID: chainID, + AppState: appState, + Consensus: &ConsensusGenesis{ + Validators: nil, +}, +} +} + +// ValidateAndComplete performs validation and completes the AppGenesis. +func (ag *AppGenesis) + +ValidateAndComplete() + +error { + if ag.ChainID == "" { + return errors.New("genesis doc must include non-empty chain_id") +} + if len(ag.ChainID) > MaxChainIDLen { + return fmt.Errorf("chain_id in genesis doc is too long (max: %d)", MaxChainIDLen) +} + if ag.InitialHeight < 0 { + return fmt.Errorf("initial_height cannot be negative (got %v)", ag.InitialHeight) +} + if ag.InitialHeight == 0 { + ag.InitialHeight = 1 +} + if ag.GenesisTime.IsZero() { + ag.GenesisTime = cmttime.Now() +} + if err := ag.Consensus.ValidateAndComplete(); err != nil { + return err +} + +return nil +} + +// SaveAs is a utility method for saving AppGenesis as a JSON file. +func (ag *AppGenesis) + +SaveAs(file string) + +error { + appGenesisBytes, err := json.MarshalIndent(ag, "", " + ") + if err != nil { + return err +} + +return os.WriteFile(file, appGenesisBytes, 0o600) +} + +// AppGenesisFromFile reads the AppGenesis from the provided file. +func AppGenesisFromFile(genFile string) (*AppGenesis, error) { + jsonBlob, err := os.ReadFile(genFile) + if err != nil { + return nil, fmt.Errorf("couldn't read AppGenesis file (%s): %w", genFile, err) +} + +var appGenesis AppGenesis + if err := json.Unmarshal(jsonBlob, &appGenesis); err != nil { + // fallback to CometBFT genesis + var ctmGenesis cmttypes.GenesisDoc + if err2 := cmtjson.Unmarshal(jsonBlob, &ctmGenesis); err2 != nil { + return nil, fmt.Errorf("error unmarshalling AppGenesis at %s: %w\n failed fallback to CometBFT GenDoc: %w", genFile, err, err2) +} + +appGenesis = AppGenesis{ + AppName: version.AppName, + // AppVersion is not filled as we do not know it from a CometBFT genesis + GenesisTime: ctmGenesis.GenesisTime, + ChainID: ctmGenesis.ChainID, + InitialHeight: ctmGenesis.InitialHeight, + AppHash: ctmGenesis.AppHash, + AppState: ctmGenesis.AppState, + Consensus: &ConsensusGenesis{ + Validators: ctmGenesis.Validators, + Params: ctmGenesis.ConsensusParams, +}, +} + +} + +return &appGenesis, nil +} + +// -------------------------- +// CometBFT Genesis Handling +// -------------------------- + +// ToGenesisDoc converts the AppGenesis to a CometBFT GenesisDoc. +func (ag *AppGenesis) + +ToGenesisDoc() (*cmttypes.GenesisDoc, error) { + return &cmttypes.GenesisDoc{ + GenesisTime: ag.GenesisTime, + ChainID: ag.ChainID, + InitialHeight: ag.InitialHeight, + AppHash: ag.AppHash, + AppState: ag.AppState, + Validators: ag.Consensus.Validators, + ConsensusParams: ag.Consensus.Params, +}, nil +} + +// ConsensusGenesis defines the consensus layer's genesis. +// TODO(@julienrbrt) + +eventually abstract from CometBFT types +type ConsensusGenesis struct { + Validators []cmttypes.GenesisValidator `json:"validators,omitempty"` + Params *cmttypes.ConsensusParams `json:"params,omitempty"` +} + +// NewConsensusGenesis returns a ConsensusGenesis with given values. +// It takes a proto consensus params so it can called from server export command. +func NewConsensusGenesis(params cmtproto.ConsensusParams, validators []cmttypes.GenesisValidator) *ConsensusGenesis { + return &ConsensusGenesis{ + Params: &cmttypes.ConsensusParams{ + Block: cmttypes.BlockParams{ + MaxBytes: params.Block.MaxBytes, + MaxGas: params.Block.MaxGas, +}, + Evidence: cmttypes.EvidenceParams{ + MaxAgeNumBlocks: params.Evidence.MaxAgeNumBlocks, + MaxAgeDuration: params.Evidence.MaxAgeDuration, + MaxBytes: params.Evidence.MaxBytes, +}, + Validator: cmttypes.ValidatorParams{ + PubKeyTypes: params.Validator.PubKeyTypes, +}, +}, + Validators: validators, +} +} + +func (cs *ConsensusGenesis) + +MarshalJSON() ([]byte, error) { + type Alias ConsensusGenesis + return cmtjson.Marshal(&Alias{ + Validators: cs.Validators, + Params: cs.Params, +}) +} + +func (cs *ConsensusGenesis) + +UnmarshalJSON(b []byte) + +error { + type Alias ConsensusGenesis + result := Alias{ +} + if err := cmtjson.Unmarshal(b, &result); err != nil { + return err +} + +cs.Params = result.Params + cs.Validators = result.Validators + + return nil +} + +func (cs *ConsensusGenesis) + +ValidateAndComplete() + +error { + if cs == nil { + return fmt.Errorf("consensus genesis cannot be nil") +} + if cs.Params == nil { + cs.Params = cmttypes.DefaultConsensusParams() +} + +else if err := cs.Params.ValidateBasic(); err != nil { + return err +} + for i, v := range cs.Validators { + if v.Power == 0 { + return fmt.Errorf("the genesis file cannot contain validators with no voting power: %v", v) +} + if len(v.Address) > 0 && !bytes.Equal(v.PubKey.Address(), v.Address) { + return fmt.Errorf("incorrect address for validator %v in the genesis file, should be %v", v, v.PubKey.Address()) +} + if len(v.Address) == 0 { + cs.Validators[i].Address = v.PubKey.Address() +} + +} + +return nil +} +``` + +The application genesis can then be translated to the consensus engine to the right format: + +```go expandable +package types + +import ( + + "bytes" + "encoding/json" + "errors" + "fmt" + "os" + "time" + + cmtjson "github.com/cometbft/cometbft/libs/json" + cmtproto "github.com/cometbft/cometbft/proto/tendermint/types" + cmttypes "github.com/cometbft/cometbft/types" + cmttime "github.com/cometbft/cometbft/types/time" + "github.com/cosmos/cosmos-sdk/version" +) + +const ( + // MaxChainIDLen is the maximum length of a chain ID. + MaxChainIDLen = cmttypes.MaxChainIDLen +) + +// AppGenesis defines the app's genesis. +type AppGenesis struct { + AppName string `json:"app_name"` + AppVersion string `json:"app_version"` + GenesisTime time.Time `json:"genesis_time"` + ChainID string `json:"chain_id"` + InitialHeight int64 `json:"initial_height"` + AppHash []byte `json:"app_hash"` + AppState json.RawMessage `json:"app_state,omitempty"` + Consensus *ConsensusGenesis `json:"consensus,omitempty"` +} + +// NewAppGenesisWithVersion returns a new AppGenesis with the app name and app version already. +func NewAppGenesisWithVersion(chainID string, appState json.RawMessage) *AppGenesis { + return &AppGenesis{ + AppName: version.AppName, + AppVersion: version.Version, + ChainID: chainID, + AppState: appState, + Consensus: &ConsensusGenesis{ + Validators: nil, +}, +} +} + +// ValidateAndComplete performs validation and completes the AppGenesis. +func (ag *AppGenesis) + +ValidateAndComplete() + +error { + if ag.ChainID == "" { + return errors.New("genesis doc must include non-empty chain_id") +} + if len(ag.ChainID) > MaxChainIDLen { + return fmt.Errorf("chain_id in genesis doc is too long (max: %d)", MaxChainIDLen) +} + if ag.InitialHeight < 0 { + return fmt.Errorf("initial_height cannot be negative (got %v)", ag.InitialHeight) +} + if ag.InitialHeight == 0 { + ag.InitialHeight = 1 +} + if ag.GenesisTime.IsZero() { + ag.GenesisTime = cmttime.Now() +} + if err := ag.Consensus.ValidateAndComplete(); err != nil { + return err +} + +return nil +} + +// SaveAs is a utility method for saving AppGenesis as a JSON file. +func (ag *AppGenesis) + +SaveAs(file string) + +error { + appGenesisBytes, err := json.MarshalIndent(ag, "", " + ") + if err != nil { + return err +} + +return os.WriteFile(file, appGenesisBytes, 0o600) +} + +// AppGenesisFromFile reads the AppGenesis from the provided file. +func AppGenesisFromFile(genFile string) (*AppGenesis, error) { + jsonBlob, err := os.ReadFile(genFile) + if err != nil { + return nil, fmt.Errorf("couldn't read AppGenesis file (%s): %w", genFile, err) +} + +var appGenesis AppGenesis + if err := json.Unmarshal(jsonBlob, &appGenesis); err != nil { + // fallback to CometBFT genesis + var ctmGenesis cmttypes.GenesisDoc + if err2 := cmtjson.Unmarshal(jsonBlob, &ctmGenesis); err2 != nil { + return nil, fmt.Errorf("error unmarshalling AppGenesis at %s: %w\n failed fallback to CometBFT GenDoc: %w", genFile, err, err2) +} + +appGenesis = AppGenesis{ + AppName: version.AppName, + // AppVersion is not filled as we do not know it from a CometBFT genesis + GenesisTime: ctmGenesis.GenesisTime, + ChainID: ctmGenesis.ChainID, + InitialHeight: ctmGenesis.InitialHeight, + AppHash: ctmGenesis.AppHash, + AppState: ctmGenesis.AppState, + Consensus: &ConsensusGenesis{ + Validators: ctmGenesis.Validators, + Params: ctmGenesis.ConsensusParams, +}, +} + +} + +return &appGenesis, nil +} + +// -------------------------- +// CometBFT Genesis Handling +// -------------------------- + +// ToGenesisDoc converts the AppGenesis to a CometBFT GenesisDoc. +func (ag *AppGenesis) + +ToGenesisDoc() (*cmttypes.GenesisDoc, error) { + return &cmttypes.GenesisDoc{ + GenesisTime: ag.GenesisTime, + ChainID: ag.ChainID, + InitialHeight: ag.InitialHeight, + AppHash: ag.AppHash, + AppState: ag.AppState, + Validators: ag.Consensus.Validators, + ConsensusParams: ag.Consensus.Params, +}, nil +} + +// ConsensusGenesis defines the consensus layer's genesis. +// TODO(@julienrbrt) + +eventually abstract from CometBFT types +type ConsensusGenesis struct { + Validators []cmttypes.GenesisValidator `json:"validators,omitempty"` + Params *cmttypes.ConsensusParams `json:"params,omitempty"` +} + +// NewConsensusGenesis returns a ConsensusGenesis with given values. +// It takes a proto consensus params so it can called from server export command. +func NewConsensusGenesis(params cmtproto.ConsensusParams, validators []cmttypes.GenesisValidator) *ConsensusGenesis { + return &ConsensusGenesis{ + Params: &cmttypes.ConsensusParams{ + Block: cmttypes.BlockParams{ + MaxBytes: params.Block.MaxBytes, + MaxGas: params.Block.MaxGas, +}, + Evidence: cmttypes.EvidenceParams{ + MaxAgeNumBlocks: params.Evidence.MaxAgeNumBlocks, + MaxAgeDuration: params.Evidence.MaxAgeDuration, + MaxBytes: params.Evidence.MaxBytes, +}, + Validator: cmttypes.ValidatorParams{ + PubKeyTypes: params.Validator.PubKeyTypes, +}, +}, + Validators: validators, +} +} + +func (cs *ConsensusGenesis) + +MarshalJSON() ([]byte, error) { + type Alias ConsensusGenesis + return cmtjson.Marshal(&Alias{ + Validators: cs.Validators, + Params: cs.Params, +}) +} + +func (cs *ConsensusGenesis) + +UnmarshalJSON(b []byte) + +error { + type Alias ConsensusGenesis + result := Alias{ +} + if err := cmtjson.Unmarshal(b, &result); err != nil { + return err +} + +cs.Params = result.Params + cs.Validators = result.Validators + + return nil +} + +func (cs *ConsensusGenesis) + +ValidateAndComplete() + +error { + if cs == nil { + return fmt.Errorf("consensus genesis cannot be nil") +} + if cs.Params == nil { + cs.Params = cmttypes.DefaultConsensusParams() +} + +else if err := cs.Params.ValidateBasic(); err != nil { + return err +} + for i, v := range cs.Validators { + if v.Power == 0 { + return fmt.Errorf("the genesis file cannot contain validators with no voting power: %v", v) +} + if len(v.Address) > 0 && !bytes.Equal(v.PubKey.Address(), v.Address) { + return fmt.Errorf("incorrect address for validator %v in the genesis file, should be %v", v, v.PubKey.Address()) +} + if len(v.Address) == 0 { + cs.Validators[i].Address = v.PubKey.Address() +} + +} + +return nil +} +``` + +```go expandable +package server + +import ( + + "context" + "errors" + "fmt" + "io" + "net" + "os" + "runtime/pprof" + "github.com/cometbft/cometbft/abci/server" + cmtcmd "github.com/cometbft/cometbft/cmd/cometbft/commands" + cmtcfg "github.com/cometbft/cometbft/config" + "github.com/cometbft/cometbft/node" + "github.com/cometbft/cometbft/p2p" + pvm "github.com/cometbft/cometbft/privval" + "github.com/cometbft/cometbft/proxy" + "github.com/cometbft/cometbft/rpc/client/local" + cmttypes "github.com/cometbft/cometbft/types" + dbm "github.com/cosmos/cosmos-db" + "github.com/hashicorp/go-metrics" + "github.com/spf13/cobra" + "github.com/spf13/pflag" + "golang.org/x/sync/errgroup" + "google.golang.org/grpc" + "google.golang.org/grpc/credentials/insecure" + + pruningtypes "cosmossdk.io/store/pruning/types" + "github.com/cosmos/cosmos-sdk/client" + "github.com/cosmos/cosmos-sdk/client/flags" + "github.com/cosmos/cosmos-sdk/codec" + "github.com/cosmos/cosmos-sdk/server/api" + serverconfig "github.com/cosmos/cosmos-sdk/server/config" + servergrpc "github.com/cosmos/cosmos-sdk/server/grpc" + servercmtlog "github.com/cosmos/cosmos-sdk/server/log" + "github.com/cosmos/cosmos-sdk/server/types" + "github.com/cosmos/cosmos-sdk/telemetry" + "github.com/cosmos/cosmos-sdk/types/mempool" + "github.com/cosmos/cosmos-sdk/version" + genutiltypes "github.com/cosmos/cosmos-sdk/x/genutil/types" +) + +const ( + // CometBFT full-node start flags + flagWithComet = "with-comet" + flagAddress = "address" + flagTransport = "transport" + flagTraceStore = "trace-store" + flagCPUProfile = "cpu-profile" + FlagMinGasPrices = "minimum-gas-prices" + FlagQueryGasLimit = "query-gas-limit" + FlagHaltHeight = "halt-height" + FlagHaltTime = "halt-time" + FlagInterBlockCache = "inter-block-cache" + FlagUnsafeSkipUpgrades = "unsafe-skip-upgrades" + FlagTrace = "trace" + FlagInvCheckPeriod = "inv-check-period" + + FlagPruning = "pruning" + FlagPruningKeepRecent = "pruning-keep-recent" + FlagPruningInterval = "pruning-interval" + FlagIndexEvents = "index-events" + FlagMinRetainBlocks = "min-retain-blocks" + FlagIAVLCacheSize = "iavl-cache-size" + FlagDisableIAVLFastNode = "iavl-disable-fastnode" + + // state sync-related flags + FlagStateSyncSnapshotInterval = "state-sync.snapshot-interval" + FlagStateSyncSnapshotKeepRecent = "state-sync.snapshot-keep-recent" + + // api-related flags + FlagAPIEnable = "api.enable" + FlagAPISwagger = "api.swagger" + FlagAPIAddress = "api.address" + FlagAPIMaxOpenConnections = "api.max-open-connections" + FlagRPCReadTimeout = "api.rpc-read-timeout" + FlagRPCWriteTimeout = "api.rpc-write-timeout" + FlagRPCMaxBodyBytes = "api.rpc-max-body-bytes" + FlagAPIEnableUnsafeCORS = "api.enabled-unsafe-cors" + + // gRPC-related flags + flagGRPCOnly = "grpc-only" + flagGRPCEnable = "grpc.enable" + flagGRPCAddress = "grpc.address" + flagGRPCWebEnable = "grpc-web.enable" + + // mempool flags + FlagMempoolMaxTxs = "mempool.max-txs" +) + +// StartCmdOptions defines options that can be customized in `StartCmdWithOptions`, +type StartCmdOptions struct { + // DBOpener can be used to customize db opening, for example customize db options or support different db backends, + // default to the builtin db opener. + DBOpener func(rootDir string, backendType dbm.BackendType) (dbm.DB, error) + // PostSetup can be used to setup extra services under the same cancellable context, + // it's not called in stand-alone mode, only for in-process mode. + PostSetup func(svrCtx *Context, clientCtx client.Context, ctx context.Context, g *errgroup.Group) + +error + // AddFlags add custom flags to start cmd + AddFlags func(cmd *cobra.Command) +} + +// StartCmd runs the service passed in, either stand-alone or in-process with +// CometBFT. +func StartCmd(appCreator types.AppCreator, defaultNodeHome string) *cobra.Command { + return StartCmdWithOptions(appCreator, defaultNodeHome, StartCmdOptions{ +}) +} + +// StartCmdWithOptions runs the service passed in, either stand-alone or in-process with +// CometBFT. +func StartCmdWithOptions(appCreator types.AppCreator, defaultNodeHome string, opts StartCmdOptions) *cobra.Command { + if opts.DBOpener == nil { + opts.DBOpener = openDB +} + cmd := &cobra.Command{ + Use: "start", + Short: "Run the full node", + Long: `Run the full node application with CometBFT in or out of process. By +default, the application will run with CometBFT in process. + +Pruning options can be provided via the '--pruning' flag or alternatively with '--pruning-keep-recent', and +'pruning-interval' together. + +For '--pruning' the options are as follows: + +default: the last 362880 states are kept, pruning at 10 block intervals +nothing: all historic states will be saved, nothing will be deleted (i.e. archiving node) + +everything: 2 latest states will be kept; pruning at 10 block intervals. +custom: allow pruning options to be manually specified through 'pruning-keep-recent', and 'pruning-interval' + +Node halting configurations exist in the form of two flags: '--halt-height' and '--halt-time'. During +the ABCI Commit phase, the node will check if the current block height is greater than or equal to +the halt-height or if the current block time is greater than or equal to the halt-time. If so, the +node will attempt to gracefully shutdown and the block will not be committed. In addition, the node +will not be able to commit subsequent blocks. + +For profiling and benchmarking purposes, CPU profiling can be enabled via the '--cpu-profile' flag +which accepts a path for the resulting pprof file. + +The node may be started in a 'query only' mode where only the gRPC and JSON HTTP +API services are enabled via the 'grpc-only' flag. In this mode, CometBFT is +bypassed and can be used when legacy queries are needed after an on-chain upgrade +is performed. Note, when enabled, gRPC will also be automatically enabled. +`, + PreRunE: func(cmd *cobra.Command, _ []string) + +error { + serverCtx := GetServerContextFromCmd(cmd) + + // Bind flags to the Context's Viper so the app construction can set + // options accordingly. + if err := serverCtx.Viper.BindPFlags(cmd.Flags()); err != nil { + return err +} + + _, err := GetPruningOptionsFromFlags(serverCtx.Viper) + +return err +}, + RunE: func(cmd *cobra.Command, _ []string) + +error { + serverCtx := GetServerContextFromCmd(cmd) + +clientCtx, err := client.GetClientQueryContext(cmd) + if err != nil { + return err +} + +withCMT, _ := cmd.Flags().GetBool(flagWithComet) + if !withCMT { + serverCtx.Logger.Info("starting ABCI without CometBFT") +} + +return wrapCPUProfile(serverCtx, func() + +error { + return start(serverCtx, clientCtx, appCreator, withCMT, opts) +}) +}, +} + +cmd.Flags().String(flags.FlagHome, defaultNodeHome, "The application home directory") + +cmd.Flags().Bool(flagWithComet, true, "Run abci app embedded in-process with CometBFT") + +cmd.Flags().String(flagAddress, "tcp://0.0.0.0:26658", "Listen address") + +cmd.Flags().String(flagTransport, "socket", "Transport protocol: socket, grpc") + +cmd.Flags().String(flagTraceStore, "", "Enable KVStore tracing to an output file") + +cmd.Flags().String(FlagMinGasPrices, "", "Minimum gas prices to accept for transactions; Any fee in a tx must meet this minimum (e.g. 0.01photino;0.0001stake)") + +cmd.Flags().Uint64(FlagQueryGasLimit, 0, "Maximum gas a Rest/Grpc query can consume. Blank and 0 imply unbounded.") + +cmd.Flags().IntSlice(FlagUnsafeSkipUpgrades, []int{ +}, "Skip a set of upgrade heights to continue the old binary") + +cmd.Flags().Uint64(FlagHaltHeight, 0, "Block height at which to gracefully halt the chain and shutdown the node") + +cmd.Flags().Uint64(FlagHaltTime, 0, "Minimum block time (in Unix seconds) + +at which to gracefully halt the chain and shutdown the node") + +cmd.Flags().Bool(FlagInterBlockCache, true, "Enable inter-block caching") + +cmd.Flags().String(flagCPUProfile, "", "Enable CPU profiling and write to the provided file") + +cmd.Flags().Bool(FlagTrace, false, "Provide full stack traces for errors in ABCI Log") + +cmd.Flags().String(FlagPruning, pruningtypes.PruningOptionDefault, "Pruning strategy (default|nothing|everything|custom)") + +cmd.Flags().Uint64(FlagPruningKeepRecent, 0, "Number of recent heights to keep on disk (ignored if pruning is not 'custom')") + +cmd.Flags().Uint64(FlagPruningInterval, 0, "Height interval at which pruned heights are removed from disk (ignored if pruning is not 'custom')") + +cmd.Flags().Uint(FlagInvCheckPeriod, 0, "Assert registered invariants every N blocks") + +cmd.Flags().Uint64(FlagMinRetainBlocks, 0, "Minimum block height offset during ABCI commit to prune CometBFT blocks") + +cmd.Flags().Bool(FlagAPIEnable, false, "Define if the API server should be enabled") + +cmd.Flags().Bool(FlagAPISwagger, false, "Define if swagger documentation should automatically be registered (Note: the API must also be enabled)") + +cmd.Flags().String(FlagAPIAddress, serverconfig.DefaultAPIAddress, "the API server address to listen on") + +cmd.Flags().Uint(FlagAPIMaxOpenConnections, 1000, "Define the number of maximum open connections") + +cmd.Flags().Uint(FlagRPCReadTimeout, 10, "Define the CometBFT RPC read timeout (in seconds)") + +cmd.Flags().Uint(FlagRPCWriteTimeout, 0, "Define the CometBFT RPC write timeout (in seconds)") + +cmd.Flags().Uint(FlagRPCMaxBodyBytes, 1000000, "Define the CometBFT maximum request body (in bytes)") + +cmd.Flags().Bool(FlagAPIEnableUnsafeCORS, false, "Define if CORS should be enabled (unsafe - use it at your own risk)") + +cmd.Flags().Bool(flagGRPCOnly, false, "Start the node in gRPC query only mode (no CometBFT process is started)") + +cmd.Flags().Bool(flagGRPCEnable, true, "Define if the gRPC server should be enabled") + +cmd.Flags().String(flagGRPCAddress, serverconfig.DefaultGRPCAddress, "the gRPC server address to listen on") + +cmd.Flags().Bool(flagGRPCWebEnable, true, "Define if the gRPC-Web server should be enabled. (Note: gRPC must also be enabled)") + +cmd.Flags().Uint64(FlagStateSyncSnapshotInterval, 0, "State sync snapshot interval") + +cmd.Flags().Uint32(FlagStateSyncSnapshotKeepRecent, 2, "State sync snapshot to keep") + +cmd.Flags().Bool(FlagDisableIAVLFastNode, false, "Disable fast node for IAVL tree") + +cmd.Flags().Int(FlagMempoolMaxTxs, mempool.DefaultMaxTx, "Sets MaxTx value for the app-side mempool") + + // support old flags name for backwards compatibility + cmd.Flags().SetNormalizeFunc(func(f *pflag.FlagSet, name string) + +pflag.NormalizedName { + if name == "with-tendermint" { + name = flagWithComet +} + +return pflag.NormalizedName(name) +}) + + // add support for all CometBFT-specific command line options + cmtcmd.AddNodeFlags(cmd) + if opts.AddFlags != nil { + opts.AddFlags(cmd) +} + +return cmd +} + +func start(svrCtx *Context, clientCtx client.Context, appCreator types.AppCreator, withCmt bool, opts StartCmdOptions) + +error { + svrCfg, err := getAndValidateConfig(svrCtx) + if err != nil { + return err +} + +app, appCleanupFn, err := startApp(svrCtx, appCreator, opts) + if err != nil { + return err +} + +defer appCleanupFn() + +metrics, err := startTelemetry(svrCfg) + if err != nil { + return err +} + +emitServerInfoMetrics() + if !withCmt { + return startStandAlone(svrCtx, app, opts) +} + +return startInProcess(svrCtx, svrCfg, clientCtx, app, metrics, opts) +} + +func startStandAlone(svrCtx *Context, app types.Application, opts StartCmdOptions) + +error { + addr := svrCtx.Viper.GetString(flagAddress) + transport := svrCtx.Viper.GetString(flagTransport) + cmtApp := NewCometABCIWrapper(app) + +svr, err := server.NewServer(addr, transport, cmtApp) + if err != nil { + return fmt.Errorf("error creating listener: %v", err) +} + +svr.SetLogger(servercmtlog.CometLoggerWrapper{ + Logger: svrCtx.Logger.With("module", "abci-server") +}) + +g, ctx := getCtx(svrCtx, false) + +g.Go(func() + +error { + if err := svr.Start(); err != nil { + svrCtx.Logger.Error("failed to start out-of-process ABCI server", "err", err) + +return err +} + + // Wait for the calling process to be canceled or close the provided context, + // so we can gracefully stop the ABCI server. + <-ctx.Done() + +svrCtx.Logger.Info("stopping the ABCI server...") + +return errors.Join(svr.Stop(), app.Close()) +}) + +return g.Wait() +} + +func startInProcess(svrCtx *Context, svrCfg serverconfig.Config, clientCtx client.Context, app types.Application, + metrics *telemetry.Metrics, opts StartCmdOptions, +) + +error { + cmtCfg := svrCtx.Config + home := cmtCfg.RootDir + gRPCOnly := svrCtx.Viper.GetBool(flagGRPCOnly) + +g, ctx := getCtx(svrCtx, true) + if gRPCOnly { + // TODO: Generalize logic so that gRPC only is really in startStandAlone + svrCtx.Logger.Info("starting node in gRPC only mode; CometBFT is disabled") + +svrCfg.GRPC.Enable = true +} + +else { + svrCtx.Logger.Info("starting node with ABCI CometBFT in-process") + +tmNode, cleanupFn, err := startCmtNode(ctx, cmtCfg, app, svrCtx) + if err != nil { + return err +} + +defer cleanupFn() + + // Add the tx service to the gRPC router. We only need to register this + // service if API or gRPC is enabled, and avoid doing so in the general + // case, because it spawns a new local CometBFT RPC client. + if svrCfg.API.Enable || svrCfg.GRPC.Enable { + // Re-assign for making the client available below do not use := to avoid + // shadowing the clientCtx variable. + clientCtx = clientCtx.WithClient(local.New(tmNode)) + +app.RegisterTxService(clientCtx) + +app.RegisterTendermintService(clientCtx) + +app.RegisterNodeService(clientCtx, svrCfg) +} + +} + +grpcSrv, clientCtx, err := startGrpcServer(ctx, g, svrCfg.GRPC, clientCtx, svrCtx, app) + if err != nil { + return err +} + +err = startAPIServer(ctx, g, cmtCfg, svrCfg, clientCtx, svrCtx, app, home, grpcSrv, metrics) + if err != nil { + return err +} + if opts.PostSetup != nil { + if err := opts.PostSetup(svrCtx, clientCtx, ctx, g); err != nil { + return err +} + +} + + // wait for signal capture and gracefully return + // we are guaranteed to be waiting for the "ListenForQuitSignals" goroutine. + return g.Wait() +} + +// TODO: Move nodeKey into being created within the function. +func startCmtNode( + ctx context.Context, + cfg *cmtcfg.Config, + app types.Application, + svrCtx *Context, +) (tmNode *node.Node, cleanupFn func(), err error) { + nodeKey, err := p2p.LoadOrGenNodeKey(cfg.NodeKeyFile()) + if err != nil { + return nil, cleanupFn, err +} + cmtApp := NewCometABCIWrapper(app) + +tmNode, err = node.NewNodeWithContext( + ctx, + cfg, + pvm.LoadOrGenFilePV(cfg.PrivValidatorKeyFile(), cfg.PrivValidatorStateFile()), + nodeKey, + proxy.NewLocalClientCreator(cmtApp), + getGenDocProvider(cfg), + cmtcfg.DefaultDBProvider, + node.DefaultMetricsProvider(cfg.Instrumentation), + servercmtlog.CometLoggerWrapper{ + Logger: svrCtx.Logger +}, + ) + if err != nil { + return tmNode, cleanupFn, err +} + if err := tmNode.Start(); err != nil { + return tmNode, cleanupFn, err +} + +cleanupFn = func() { + if tmNode != nil && tmNode.IsRunning() { + _ = tmNode.Stop() + _ = app.Close() +} + +} + +return tmNode, cleanupFn, nil +} + +func getAndValidateConfig(svrCtx *Context) (serverconfig.Config, error) { + config, err := serverconfig.GetConfig(svrCtx.Viper) + if err != nil { + return config, err +} + if err := config.ValidateBasic(); err != nil { + return config, err +} + +return config, nil +} + +// returns a function which returns the genesis doc from the genesis file. +func getGenDocProvider(cfg *cmtcfg.Config) + +func() (*cmttypes.GenesisDoc, error) { + return func() (*cmttypes.GenesisDoc, error) { + appGenesis, err := genutiltypes.AppGenesisFromFile(cfg.GenesisFile()) + if err != nil { + return nil, err +} + +return appGenesis.ToGenesisDoc() +} +} + +func setupTraceWriter(svrCtx *Context) (traceWriter io.WriteCloser, cleanup func(), err error) { + // clean up the traceWriter when the server is shutting down + cleanup = func() { +} + traceWriterFile := svrCtx.Viper.GetString(flagTraceStore) + +traceWriter, err = openTraceWriter(traceWriterFile) + if err != nil { + return traceWriter, cleanup, err +} + + // if flagTraceStore is not used then traceWriter is nil + if traceWriter != nil { + cleanup = func() { + if err = traceWriter.Close(); err != nil { + svrCtx.Logger.Error("failed to close trace writer", "err", err) +} + +} + +} + +return traceWriter, cleanup, nil +} + +func startGrpcServer( + ctx context.Context, + g *errgroup.Group, + config serverconfig.GRPCConfig, + clientCtx client.Context, + svrCtx *Context, + app types.Application, +) (*grpc.Server, client.Context, error) { + if !config.Enable { + // return grpcServer as nil if gRPC is disabled + return nil, clientCtx, nil +} + _, port, err := net.SplitHostPort(config.Address) + if err != nil { + return nil, clientCtx, err +} + maxSendMsgSize := config.MaxSendMsgSize + if maxSendMsgSize == 0 { + maxSendMsgSize = serverconfig.DefaultGRPCMaxSendMsgSize +} + maxRecvMsgSize := config.MaxRecvMsgSize + if maxRecvMsgSize == 0 { + maxRecvMsgSize = serverconfig.DefaultGRPCMaxRecvMsgSize +} + grpcAddress := fmt.Sprintf("127.0.0.1:%s", port) + + // if gRPC is enabled, configure gRPC client for gRPC gateway + grpcClient, err := grpc.Dial( + grpcAddress, + grpc.WithTransportCredentials(insecure.NewCredentials()), + grpc.WithDefaultCallOptions( + grpc.ForceCodec(codec.NewProtoCodec(clientCtx.InterfaceRegistry).GRPCCodec()), + grpc.MaxCallRecvMsgSize(maxRecvMsgSize), + grpc.MaxCallSendMsgSize(maxSendMsgSize), + ), + ) + if err != nil { + return nil, clientCtx, err +} + +clientCtx = clientCtx.WithGRPCClient(grpcClient) + +svrCtx.Logger.Debug("gRPC client assigned to client context", "target", grpcAddress) + +grpcSrv, err := servergrpc.NewGRPCServer(clientCtx, app, config) + if err != nil { + return nil, clientCtx, err +} + + // Start the gRPC server in a goroutine. Note, the provided ctx will ensure + // that the server is gracefully shut down. + g.Go(func() + +error { + return servergrpc.StartGRPCServer(ctx, svrCtx.Logger.With("module", "grpc-server"), config, grpcSrv) +}) + +return grpcSrv, clientCtx, nil +} + +func startAPIServer( + ctx context.Context, + g *errgroup.Group, + cmtCfg *cmtcfg.Config, + svrCfg serverconfig.Config, + clientCtx client.Context, + svrCtx *Context, + app types.Application, + home string, + grpcSrv *grpc.Server, + metrics *telemetry.Metrics, +) + +error { + if !svrCfg.API.Enable { + return nil +} + +clientCtx = clientCtx.WithHomeDir(home) + apiSrv := api.New(clientCtx, svrCtx.Logger.With("module", "api-server"), grpcSrv) + +app.RegisterAPIRoutes(apiSrv, svrCfg.API) + if svrCfg.Telemetry.Enabled { + apiSrv.SetTelemetry(metrics) +} + +g.Go(func() + +error { + return apiSrv.Start(ctx, svrCfg) +}) + +return nil +} + +func startTelemetry(cfg serverconfig.Config) (*telemetry.Metrics, error) { + if !cfg.Telemetry.Enabled { + return nil, nil +} + +return telemetry.New(cfg.Telemetry) +} + +// wrapCPUProfile starts CPU profiling, if enabled, and executes the provided +// callbackFn in a separate goroutine, then will wait for that callback to +// return. +// +// NOTE: We expect the caller to handle graceful shutdown and signal handling. +func wrapCPUProfile(svrCtx *Context, callbackFn func() + +error) + +error { + if cpuProfile := svrCtx.Viper.GetString(flagCPUProfile); cpuProfile != "" { + f, err := os.Create(cpuProfile) + if err != nil { + return err +} + +svrCtx.Logger.Info("starting CPU profiler", "profile", cpuProfile) + if err := pprof.StartCPUProfile(f); err != nil { + return err +} + +defer func() { + svrCtx.Logger.Info("stopping CPU profiler", "profile", cpuProfile) + +pprof.StopCPUProfile() + if err := f.Close(); err != nil { + svrCtx.Logger.Info("failed to close cpu-profile file", "profile", cpuProfile, "err", err.Error()) +} + +}() +} + +return callbackFn() +} + +// emitServerInfoMetrics emits server info related metrics using application telemetry. +func emitServerInfoMetrics() { + var ls []metrics.Label + versionInfo := version.NewInfo() + if len(versionInfo.GoVersion) > 0 { + ls = append(ls, telemetry.NewLabel("go", versionInfo.GoVersion)) +} + if len(versionInfo.CosmosSdkVersion) > 0 { + ls = append(ls, telemetry.NewLabel("version", versionInfo.CosmosSdkVersion)) +} + if len(ls) == 0 { + return +} + +telemetry.SetGaugeWithLabels([]string{"server", "info" +}, 1, ls) +} + +func getCtx(svrCtx *Context, block bool) (*errgroup.Group, context.Context) { + ctx, cancelFn := context.WithCancel(context.Background()) + +g, ctx := errgroup.WithContext(ctx) + // listen for quit signals so the calling parent process can gracefully exit + ListenForQuitSignals(g, block, cancelFn, svrCtx.Logger) + +return g, ctx +} + +func startApp(svrCtx *Context, appCreator types.AppCreator, opts StartCmdOptions) (app types.Application, cleanupFn func(), err error) { + traceWriter, traceCleanupFn, err := setupTraceWriter(svrCtx) + if err != nil { + return app, traceCleanupFn, err +} + home := svrCtx.Config.RootDir + db, err := opts.DBOpener(home, GetAppDBBackend(svrCtx.Viper)) + if err != nil { + return app, traceCleanupFn, err +} + +app = appCreator(svrCtx.Logger, db, traceWriter, svrCtx.Viper) + +cleanupFn = func() { + traceCleanupFn() + if localErr := app.Close(); localErr != nil { + svrCtx.Logger.Error(localErr.Error()) +} + +} + +return app, cleanupFn, nil +} +``` + +## Client + +### CLI + +The genutil commands are available under the `genesis` subcommand. + +#### add-genesis-account + +Add a genesis account to `genesis.json`. Learn more [here](/sdk/v0.54/node/run-node#adding-genesis-accounts). + +#### collect-gentxs + +Collect genesis txs and output a `genesis.json` file. + +```shell +simd genesis collect-gentxs +``` + +This will create a new `genesis.json` file that includes data from all the validators (we sometimes call it the "super genesis file" to distinguish it from single-validator genesis files). + +#### gentx + +Generate a genesis tx carrying a self delegation. + +```shell +simd genesis gentx [key_name] [amount] --chain-id [chain-id] +``` + +This will create the genesis transaction for your new chain. Here `amount` should be at least `1000000000stake`. +If you provide too much or too little, you will encounter an error when starting a node. + +#### migrate + +Migrate genesis to a specified target (SDK) version. + +```shell +simd genesis migrate [target-version] +``` + + +The `migrate` command is extensible and takes a `MigrationMap`. This map is a mapping of target versions to genesis migrations functions. +When not using the default `MigrationMap`, it is recommended to still call the default `MigrationMap` corresponding the SDK version of the chain and prepend/append your own genesis migrations. + + +#### validate-genesis + +Validates the genesis file at the default location or at the location passed as an argument. + +```shell +simd genesis validate-genesis +``` + + +Validate genesis only validates if the genesis is valid at the **current application binary**. For validating a genesis from a previous version of the application, use the `migrate` command to migrate the genesis to the current version. + diff --git a/sdk/v0.54/modules/gov/README.mdx b/sdk/v0.54/modules/gov/README.mdx new file mode 100644 index 000000000..fde969bb0 --- /dev/null +++ b/sdk/v0.54/modules/gov/README.mdx @@ -0,0 +1,2876 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/modules/gov/README' +title: 'x/gov' +description: >- + This paper specifies the Governance module of the Cosmos SDK, which was first + described in the Cosmos Whitepaper in June 2016. +--- + +## Abstract + +This paper specifies the Governance module of the Cosmos SDK, which was first +described in the [Cosmos Whitepaper](https://github.com/cosmos/cosmos/blob/master/WHITEPAPER.md) in +June 2016. + +The module enables Cosmos SDK based blockchain to support an on-chain governance +system. In this system, holders of the native staking token of the chain can vote +on proposals on a 1 token 1 vote basis. Next is a list of features the module +currently supports: + +* **Proposal submission:** Users can submit proposals with a deposit. Once the + minimum deposit is reached, the proposal enters voting period. The minimum deposit can be reached by collecting deposits from different users (including proposer) within deposit period. +* **Vote:** Participants can vote on proposals that reached MinDeposit and entered voting period. +* **Inheritance and penalties:** Delegators inherit their validator's vote if + they don't vote themselves. +* **Claiming deposit:** Users that deposited on proposals can recover their + deposits if the proposal was accepted or rejected. If the proposal was vetoed, or never entered voting period (minimum deposit not reached within deposit period), the deposit is burned. + +This module is in use on the Cosmos Hub (a.k.a [gaia](https://github.com/cosmos/gaia)). +Features that may be added in the future are described in [Future Improvements](#future-improvements). + +## Contents + +The following specification uses *ATOM* as the native staking token. The module +can be adapted to any Proof-Of-Stake blockchain by replacing *ATOM* with the native +staking token of the chain. + +* [Concepts](#concepts) + * [Proposal submission](#proposal-submission) + * [Deposit](#deposit) + * [Vote](#vote) + * [Software Upgrade](#software-upgrade) +* [State](#state) + * [Proposals](#proposals) + * [Parameters and base types](#parameters-and-base-types) + * [Deposit](#deposit-1) + * [ValidatorGovInfo](#validatorgovinfo) + * [Stores](#stores) + * [Proposal Processing Queue](#proposal-processing-queue) + * [Legacy Proposal](#legacy-proposal) +* [Messages](#messages) + * [Proposal Submission](#proposal-submission-1) + * [Deposit](#deposit-2) + * [Vote](#vote-1) +* [Events](#events) + * [EndBlocker](#endblocker) + * [Handlers](#handlers) +* [Hooks](#hooks) + * [AfterProposalSubmission](#afterproposalsubmission) + * [AfterProposalDeposit](#afterproposaldeposit) + * [AfterProposalVote](#afterproposalvote) + * [AfterProposalFailedMinDeposit](#afterproposalfailedmindeposit) + * [AfterProposalVotingPeriodEnded](#afterproposalvotingperiodended) +* [Parameters](#parameters) +* [Client](#client) + * [CLI](#cli) + * [gRPC](#grpc) + * [REST](#rest) +* [Metadata](#metadata) + * [Proposal](#proposal-3) + * [Vote](#vote-5) +* [Future Improvements](#future-improvements) + +## Concepts + +{/* *Disclaimer: This is work in progress. Mechanisms are susceptible to change.* */} + +The governance process is divided in a few steps that are outlined below: + +* **Proposal submission:** Proposal is submitted to the blockchain with a + deposit. +* **Vote:** Once deposit reaches a certain value (`MinDeposit`), proposal is + confirmed and vote opens. Bonded Atom holders can then send `TxGovVote` + transactions to vote on the proposal. +* **Execution** After a period of time, the votes are tallied and depending + on the result, the messages in the proposal will be executed. + +### Proposal submission + +#### Right to submit a proposal + +Every account can submit proposals by sending a `MsgSubmitProposal` transaction. +Once a proposal is submitted, it is identified by its unique `proposalID`. + +#### Proposal Messages + +A proposal includes an array of `sdk.Msg`s which are executed automatically if the +proposal passes. The messages are executed by the governance `ModuleAccount` itself. Modules +such as `x/upgrade`, that want to allow certain messages to be executed by governance +only should add a whitelist within the respective msg server, granting the governance +module the right to execute the message once a quorum has been reached. The governance +module uses the `MsgServiceRouter` to check that these messages are correctly constructed +and have a respective path to execute on but do not perform a full validity check. + +### Deposit + +To prevent spam, proposals must be submitted with a deposit in the coins defined by +the `MinDeposit` param. + +When a proposal is submitted, it has to be accompanied with a deposit that must be +strictly positive, but can be inferior to `MinDeposit`. The submitter doesn't need +to pay for the entire deposit on their own. The newly created proposal is stored in +an *inactive proposal queue* and stays there until its deposit passes the `MinDeposit`. +Other token holders can increase the proposal's deposit by sending a `Deposit` +transaction. If a proposal doesn't pass the `MinDeposit` before the deposit end time +(the time when deposits are no longer accepted), the proposal will be destroyed: the +proposal will be removed from state and the deposit will be burned (see x/gov `EndBlocker`). +When a proposal deposit passes the `MinDeposit` threshold (even during the proposal +submission) before the deposit end time, the proposal will be moved into the +*active proposal queue* and the voting period will begin. + +The deposit is kept in escrow and held by the governance `ModuleAccount` until the +proposal is finalized (passed or rejected). + +#### Deposit refund and burn + +When a proposal is finalized, the coins from the deposit are either refunded or burned +according to the final tally of the proposal: + +* If the proposal is approved or rejected but *not* vetoed, each deposit will be + automatically refunded to its respective depositor (transferred from the governance + `ModuleAccount`). +* When the proposal is vetoed with greater than 1/3, deposits will be burned from the + governance `ModuleAccount` and the proposal information along with its deposit + information will be removed from state. +* All refunded or burned deposits are removed from the state. Events are issued when + burning or refunding a deposit. + +### Vote + +#### Participants + +*Participants* are users that have the right to vote on proposals. On the +Cosmos Hub, participants are bonded Atom holders. Unbonded Atom holders and +other users do not get the right to participate in governance. However, they +can submit and deposit on proposals. + +Note that when *participants* have bonded and unbonded Atoms, their voting power is calculated from their bonded Atom holdings only. + +#### Voting period + +Once a proposal reaches `MinDeposit`, it immediately enters `Voting period`. We +define `Voting period` as the interval between the moment the vote opens and +the moment the vote closes. The initial value of `Voting period` is 2 weeks. + +#### Option set + +The option set of a proposal refers to the set of choices a participant can +choose from when casting its vote. + +The initial option set includes the following options: + +* `Yes` +* `No` +* `NoWithVeto` +* `Abstain` + +`NoWithVeto` counts as `No` but also adds a `Veto` vote. `Abstain` option +allows voters to signal that they do not intend to vote in favor or against the +proposal but accept the result of the vote. + +*Note: from the UI, for urgent proposals we should maybe add a ‘Not Urgent’ option that casts a `NoWithVeto` vote.* + +#### Weighted Votes + +[ADR-037](https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-037-gov-split-vote.md) introduces the weighted vote feature which allows a staker to split their votes into several voting options. For example, it could use 70% of its voting power to vote Yes and 30% of its voting power to vote No. + +Often times the entity owning that address might not be a single individual. For example, a company might have different stakeholders who want to vote differently, and so it makes sense to allow them to split their voting power. Currently, it is not possible for them to do "passthrough voting" and giving their users voting rights over their tokens. However, with this system, exchanges can poll their users for voting preferences, and then vote on-chain proportionally to the results of the poll. + +To represent weighted vote on chain, we use the following Protobuf message. + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/gov/v1beta1/gov.proto#L34-L47 +``` + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/gov/v1beta1/gov.proto#L181-L201 +``` + +For a weighted vote to be valid, the `options` field must not contain duplicate vote options, and the sum of weights of all options must be equal to 1. + +#### Custom Vote Calculation + +Cosmos SDK v0.53.0 introduced an option for developers to define a custom vote result and voting power calculation function. As of v0.54, `x/gov` has been decoupled from `x/staking`: the `keeper.NewKeeper` constructor now requires a `CalculateVoteResultsAndVotingPowerFn` as a required parameter instead of a `StakingKeeper`. To use the default staking-based tally logic, wrap your staking keeper with `keeper.NewDefaultCalculateVoteResultsAndVotingPower(stakingKeeper)`. + +```go expandable +package keeper + +import ( + + "context" + "fmt" + "cosmossdk.io/collections" + "cosmossdk.io/math" + + sdk "github.com/cosmos/cosmos-sdk/types" + v1 "github.com/cosmos/cosmos-sdk/x/gov/types/v1" + stakingtypes "github.com/cosmos/cosmos-sdk/x/staking/types" +) + +// CalculateVoteResultsAndVotingPowerFn is a function signature for calculating vote results and voting power +// It can be overridden to customize the voting power calculation for proposals +// It gets the proposal tallied and the validators governance infos (validator power, voting power, etc.) +// It must return the total voting power and the results of the vote +type CalculateVoteResultsAndVotingPowerFn func( + ctx context.Context, + k Keeper, + proposal v1.Proposal, + validators map[string]v1.ValidatorGovInfo, +) (totalVoterPower math.LegacyDec, results map[v1.VoteOption]math.LegacyDec, err error) + +func defaultCalculateVoteResultsAndVotingPower( + ctx context.Context, + k Keeper, + proposal v1.Proposal, + validators map[string]v1.ValidatorGovInfo, +) (totalVoterPower math.LegacyDec, results map[v1.VoteOption]math.LegacyDec, err error) { + totalVotingPower := math.LegacyZeroDec() + +results = make(map[v1.VoteOption]math.LegacyDec) + +results[v1.OptionYes] = math.LegacyZeroDec() + +results[v1.OptionAbstain] = math.LegacyZeroDec() + +results[v1.OptionNo] = math.LegacyZeroDec() + +results[v1.OptionNoWithVeto] = math.LegacyZeroDec() + rng := collections.NewPrefixedPairRange[uint64, sdk.AccAddress](proposal.Id) + votesToRemove := []collections.Pair[uint64, sdk.AccAddress]{ +} + +err = k.Votes.Walk(ctx, rng, func(key collections.Pair[uint64, sdk.AccAddress], vote v1.Vote) (bool, error) { + // if validator, just record it in the map + voter, err := k.authKeeper.AddressCodec().StringToBytes(vote.Voter) + if err != nil { + return false, err +} + +valAddrStr, err := k.sk.ValidatorAddressCodec().BytesToString(voter) + if err != nil { + return false, err +} + if val, ok := validators[valAddrStr]; ok { + val.Vote = vote.Options + validators[valAddrStr] = val +} + + // iterate over all delegations from voter, deduct from any delegated-to validators + err = k.sk.IterateDelegations(ctx, voter, func(index int64, delegation stakingtypes.DelegationI) (stop bool) { + valAddrStr := delegation.GetValidatorAddr() + if val, ok := validators[valAddrStr]; ok { + // There is no need to handle the special case that validator address equal to voter address. + // Because voter's voting power will tally again even if there will be deduction of voter's voting power from validator. + val.DelegatorDeductions = val.DelegatorDeductions.Add(delegation.GetShares()) + +validators[valAddrStr] = val + + // delegation shares * bonded / total shares + votingPower := delegation.GetShares().MulInt(val.ValidatorPower).Quo(val.DelegatorShares) + for _, option := range vote.Options { + weight, _ := math.LegacyNewDecFromStr(option.Weight) + subPower := votingPower.Mul(weight) + +results[option.Option] = results[option.Option].Add(subPower) +} + +totalVotingPower = totalVotingPower.Add(votingPower) +} + +return false +}) + if err != nil { + return false, err +} + +votesToRemove = append(votesToRemove, key) + +return false, nil +}) + if err != nil { + return math.LegacyZeroDec(), nil, fmt.Errorf("error while iterating delegations: %w", err) +} + + // remove all votes from store + for _, key := range votesToRemove { + if err := k.Votes.Remove(ctx, key); err != nil { + return math.LegacyDec{ +}, nil, fmt.Errorf("error while removing vote (%d/%s): %w", key.K1(), key.K2(), err) +} + +} + + // iterate over the validators again to tally their voting power + for _, val := range validators { + if len(val.Vote) == 0 { + continue +} + sharesAfterDeductions := val.DelegatorShares.Sub(val.DelegatorDeductions) + votingPower := sharesAfterDeductions.MulInt(val.ValidatorPower).Quo(val.DelegatorShares) + for _, option := range val.Vote { + weight, _ := math.LegacyNewDecFromStr(option.Weight) + subPower := votingPower.Mul(weight) + +results[option.Option] = results[option.Option].Add(subPower) +} + +totalVotingPower = totalVotingPower.Add(votingPower) +} + +return totalVotingPower, results, nil +} + +// getCurrentValidators fetches all the bonded validators, insert them into currValidators +func (k Keeper) + +getCurrentValidators(ctx context.Context) (map[string]v1.ValidatorGovInfo, error) { + currValidators := make(map[string]v1.ValidatorGovInfo) + if err := k.sk.IterateBondedValidatorsByPower(ctx, func(index int64, validator stakingtypes.ValidatorI) (stop bool) { + valBz, err := k.sk.ValidatorAddressCodec().StringToBytes(validator.GetOperator()) + if err != nil { + return false +} + +currValidators[validator.GetOperator()] = v1.NewValidatorGovInfo( + valBz, + validator.GetValidatorPower(), + validator.GetDelegatorShares(), + math.LegacyZeroDec(), + v1.WeightedVoteOptions{ +}, + ) + +return false +}); err != nil { + return nil, err +} + +return currValidators, nil +} + +// Tally iterates over the votes and updates the tally of a proposal based on the voting power of the +// voters +func (k Keeper) + +Tally(ctx context.Context, proposal v1.Proposal) (passes, burnDeposits bool, tallyResults v1.TallyResult, err error) { + currValidators, err := k.getCurrentValidators(ctx) + if err != nil { + return false, false, tallyResults, fmt.Errorf("error while getting current validators: %w", err) +} + tallyFn := k.calculateVoteResultsAndVotingPowerFn + totalVotingPower, results, err := tallyFn(ctx, k, proposal, currValidators) + if err != nil { + return false, false, tallyResults, fmt.Errorf("error while calculating tally results: %w", err) +} + +tallyResults = v1.NewTallyResultFromMap(results) + + // TODO: Upgrade the spec to cover all of these cases & remove pseudocode. + // If there is no staked coins, the proposal fails + totalBonded, err := k.sk.TotalValidatorPower(ctx) + if err != nil { + return false, false, tallyResults, err +} + if totalBonded.IsZero() { + return false, false, tallyResults, nil +} + +params, err := k.Params.Get(ctx) + if err != nil { + return false, false, tallyResults, fmt.Errorf("error while getting params: %w", err) +} + + // If there is not enough quorum of votes, the proposal fails + percentVoting := totalVotingPower.Quo(math.LegacyNewDecFromInt(totalBonded)) + +quorum, _ := math.LegacyNewDecFromStr(params.Quorum) + if percentVoting.LT(quorum) { + return false, params.BurnVoteQuorum, tallyResults, nil +} + + // If no one votes (everyone abstains), proposal fails + if totalVotingPower.Sub(results[v1.OptionAbstain]).Equal(math.LegacyZeroDec()) { + return false, false, tallyResults, nil +} + + // If more than 1/3 of voters veto, proposal fails + vetoThreshold, _ := math.LegacyNewDecFromStr(params.VetoThreshold) + if results[v1.OptionNoWithVeto].Quo(totalVotingPower).GT(vetoThreshold) { + return false, params.BurnVoteVeto, tallyResults, nil +} + + // If more than 1/2 of non-abstaining voters vote Yes, proposal passes + // For expedited 2/3 + var thresholdStr string + if proposal.Expedited { + thresholdStr = params.GetExpeditedThreshold() +} + +else { + thresholdStr = params.GetThreshold() +} + +threshold, _ := math.LegacyNewDecFromStr(thresholdStr) + if results[v1.OptionYes].Quo(totalVotingPower.Sub(results[v1.OptionAbstain])).GT(threshold) { + return true, false, tallyResults, nil +} + + // If more than 1/2 of non-abstaining voters vote No, proposal fails + return false, false, tallyResults, nil +} +``` + +This gives developers a more expressive way to handle governance on their appchains. +Developers can now build systems with: + +* Quadratic Voting +* Time-weighted Voting +* Reputation-Based voting + +##### Example + +```go expandable +func myCustomVotingFunction( + ctx context.Context, + k Keeper, + proposal v1.Proposal, + validators map[string]v1.ValidatorGovInfo, +) (totalVoterPower math.LegacyDec, results map[v1.VoteOption]math.LegacyDec, err error) { + // ... tally logic +} + govKeeper := govkeeper.NewKeeper( + appCodec, + runtime.NewKVStoreService(keys[govtypes.StoreKey]), + app.AccountKeeper, + app.BankKeeper, + app.DistrKeeper, // optional: can be nil if the module address is not used as a cancellation fee destination + app.MsgServiceRouter(), + govConfig, + authtypes.NewModuleAddress(govtypes.ModuleName).String(), + myCustomVotingFunction, // required: CalculateVoteResultsAndVotingPowerFn +) +``` + +### Quorum + +Quorum is defined as the minimum percentage of voting power that needs to be +cast on a proposal for the result to be valid. + +### Expedited Proposals + +A proposal can be expedited, making the proposal use shorter voting duration and a higher tally threshold by its default. If an expedited proposal fails to meet the threshold within the scope of shorter voting duration, the expedited proposal is then converted to a regular proposal and restarts voting under regular voting conditions. + +#### Threshold + +Threshold is defined as the minimum proportion of `Yes` votes (excluding +`Abstain` votes) for the proposal to be accepted. + +Initially, the threshold is set at 50% of `Yes` votes, excluding `Abstain` +votes. A possibility to veto exists if more than 1/3rd of all votes are +`NoWithVeto` votes. Note, both of these values are derived from the `TallyParams` +on-chain parameter, which is modifiable by governance. +This means that proposals are accepted iff: + +* There exist bonded tokens. +* Quorum has been achieved. +* The proportion of `Abstain` votes is inferior to 1/1. +* The proportion of `NoWithVeto` votes is inferior to 1/3, including + `Abstain` votes. +* The proportion of `Yes` votes, excluding `Abstain` votes, at the end of + the voting period is superior to 1/2. + +For expedited proposals, by default, the threshold is higher than with a *normal proposal*, namely, 66.7%. + +#### Inheritance + +If a delegator does not vote, it will inherit its validator vote. + +* If the delegator votes before its validator, it will not inherit from the + validator's vote. +* If the delegator votes after its validator, it will override its validator + vote with its own. If the proposal is urgent, it is possible + that the vote will close before delegators have a chance to react and + override their validator's vote. This is not a problem, as proposals require more than 2/3rd of the total voting power to pass, when tallied at the end of the voting period. Because as little as 1/3 + 1 validation power could collude to censor transactions, non-collusion is already assumed for ranges exceeding this threshold. + +#### Validator’s punishment for non-voting + +At present, validators are not punished for failing to vote. + +#### Governance address + +Later, we may add permissioned keys that could only sign txs from certain modules. For the MVP, the `Governance address` will be the main validator address generated at account creation. This address corresponds to a different PrivKey than the CometBFT PrivKey which is responsible for signing consensus messages. Validators thus do not have to sign governance transactions with the sensitive CometBFT PrivKey. + +#### Burnable Params + +There are three parameters that define if the deposit of a proposal should be burned or returned to the depositors. + +* `BurnVoteVeto` burns the proposal deposit if the proposal gets vetoed. +* `BurnVoteQuorum` burns the proposal deposit if the proposal deposit if the vote does not reach quorum. +* `BurnProposalDepositPrevote` burns the proposal deposit if it does not enter the voting phase. + +> Note: These parameters are modifiable via governance. + +## State + +### Constitution + +`Constitution` is found in the genesis state. It is a string field intended to be used to describe the purpose of a particular blockchain, and its expected norms. A few examples of how the constitution field can be used: + +* define the purpose of the chain, laying a foundation for its future development +* set expectations for delegators +* set expectations for validators +* define the chain's relationship to "meatspace" entities, like a foundation or corporation + +Since this is more of a social feature than a technical feature, we'll now get into some items that may have been useful to have in a genesis constitution: + +* What limitations on governance exist, if any? + * is it okay for the community to slash the wallet of a whale that they no longer feel that they want around? (viz: Juno Proposal 4 and 16) + * can governance "socially slash" a validator who is using unapproved MEV? (viz: commonwealth.im/osmosis) + * In the event of an economic emergency, what should validators do? + * Terra crash of May, 2022, saw validators choose to run a new binary with code that had not been approved by governance, because the governance token had been inflated to nothing. +* What is the purpose of the chain, specifically? + * best example of this is the Cosmos hub, where different founding groups, have different interpertations of the purpose of the network. + +This genesis entry, "constitution" hasn't been designed for existing chains, who should likely just ratify a constitution using their governance system. Instead, this is for new chains. It will allow for validators to have a much clearer idea of purpose and the expectations placed on them while operating their nodes. Likewise, for community members, the constitution will give them some idea of what to expect from both the "chain team" and the validators, respectively. + +This constitution is designed to be immutable, and placed only in genesis, though that could change over time by a pull request to the cosmos-sdk that allows for the constitution to be changed by governance. Communities wishing to make amendments to their original constitution should use the governance mechanism and a "signaling proposal" to do exactly that. + +**Ideal use scenario for a cosmos chain constitution** + +As a chain developer, you decide that you'd like to provide clarity to your key user groups: + +* validators +* token holders +* developers (yourself) + +You use the constitution to immutably store some Markdown in genesis, so that when difficult questions come up, the constitution can provide guidance to the community. + +### Proposals + +`Proposal` objects are used to tally votes and generally track the proposal's state. +They contain an array of arbitrary `sdk.Msg`'s which the governance module will attempt +to resolve and then execute if the proposal passes. `Proposal`'s are identified by a +unique id and contains a series of timestamps: `submit_time`, `deposit_end_time`, +`voting_start_time`, `voting_end_time` which track the lifecycle of a proposal + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/gov/v1/gov.proto#L51-L99 +``` + +A proposal will generally require more than just a set of messages to explain its +purpose but need some greater justification and allow a means for interested participants +to discuss and debate the proposal. +In most cases, **it is encouraged to have an off-chain system that supports the on-chain governance process**. +To accommodate for this, a proposal contains a special **`metadata`** field, a string, +which can be used to add context to the proposal. The `metadata` field allows custom use for networks, +however, it is expected that the field contains a URL or some form of CID using a system such as +[IPFS](https://docs.ipfs.io/concepts/content-addressing/). To support the case of +interoperability across networks, the SDK recommends that the `metadata` represents +the following `JSON` template: + +```json +{ + "title": "...", + "description": "...", + "forum": "...", // a link to the discussion platform (i.e. Discord) + "other": "..." // any extra data that doesn't correspond to the other fields +} +``` + +This makes it far easier for clients to support multiple networks. + +The metadata has a maximum length that is chosen by the app developer, and +passed into the gov keeper as a config. The default maximum length in the SDK is 255 characters. + +#### Writing a module that uses governance + +There are many aspects of a chain, or of the individual modules that you may want to +use governance to perform such as changing various parameters. This is very simple +to do. First, write out your message types and `MsgServer` implementation. Add an +`authority` field to the keeper which will be populated in the constructor with the +governance module account: `govKeeper.GetGovernanceAccount().GetAddress()`. Then for +the methods in the `msg_server.go`, perform a check on the message that the signer +matches `authority`. This will prevent any user from executing that message. + +### Parameters and base types + +`Parameters` define the rules according to which votes are run. There can only +be one active parameter set at any given time. If governance wants to change a +parameter set, either to modify a value or add/remove a parameter field, a new +parameter set has to be created and the previous one rendered inactive. + +#### DepositParams + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/gov/v1/gov.proto#L152-L162 +``` + +#### VotingParams + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/gov/v1/gov.proto#L164-L168 +``` + +#### TallyParams + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/gov/v1/gov.proto#L170-L182 +``` + +Parameters are stored in a global `GlobalParams` KVStore. + +Additionally, we introduce some basic types: + +```go expandable +type Vote byte + +const ( + VoteYes = 0x1 + VoteNo = 0x2 + VoteNoWithVeto = 0x3 + VoteAbstain = 0x4 +) + +type ProposalType string + +const ( + ProposalTypePlainText = "Text" + ProposalTypeSoftwareUpgrade = "SoftwareUpgrade" +) + +type ProposalStatus byte + +const ( + StatusNil ProposalStatus = 0x00 + StatusDepositPeriod ProposalStatus = 0x01 // Proposal is submitted. Participants can deposit on it but not vote + StatusVotingPeriod ProposalStatus = 0x02 // MinDeposit is reached, participants can vote + StatusPassed ProposalStatus = 0x03 // Proposal passed and successfully executed + StatusRejected ProposalStatus = 0x04 // Proposal has been rejected + StatusFailed ProposalStatus = 0x05 // Proposal passed but failed execution +) +``` + +### Deposit + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/gov/v1/gov.proto#L38-L49 +``` + +### ValidatorGovInfo + +This type is used in a temp map when tallying + +```go +type ValidatorGovInfo struct { + Minus sdk.Dec + Vote Vote +} +``` + +## Stores + + +Stores are KVStores in the multi-store. The key to find the store is the first parameter in the list + + +We will use one KVStore `Governance` to store four mappings: + +* A mapping from `proposalID|'proposal'` to `Proposal`. +* A mapping from `proposalID|'addresses'|address` to `Vote`. This mapping allows + us to query all addresses that voted on the proposal along with their vote by + doing a range query on `proposalID:addresses`. +* A mapping from `ParamsKey|'Params'` to `Params`. This map allows to query all + x/gov params. +* A mapping from `VotingPeriodProposalKeyPrefix|proposalID` to a single byte. This allows + us to know if a proposal is in the voting period or not with very low gas cost. + +For pseudocode purposes, here are the two function we will use to read or write in stores: + +* `load(StoreKey, Key)`: Retrieve item stored at key `Key` in store found at key `StoreKey` in the multistore +* `store(StoreKey, Key, value)`: Write value `Value` at key `Key` in store found at key `StoreKey` in the multistore + +### Proposal Processing Queue + +**Store:** + +* `ProposalProcessingQueue`: A queue `queue[proposalID]` containing all the + `ProposalIDs` of proposals that reached `MinDeposit`. During each `EndBlock`, + all the proposals that have reached the end of their voting period are processed. + To process a finished proposal, the application tallies the votes, computes the + votes of each validator and checks if every validator in the validator set has + voted. If the proposal is accepted, deposits are refunded. Finally, the proposal + content `Handler` is executed. + +And the pseudocode for the `ProposalProcessingQueue`: + +```go expandable +in EndBlock do + for finishedProposalID in GetAllFinishedProposalIDs(block.Time) + +proposal = load(Governance, ) // proposal is a const key + + validators = Keeper.getAllValidators() + tmpValMap := map(sdk.AccAddress) + +ValidatorGovInfo + + // Initiate mapping at 0. This is the amount of shares of the validator's vote that will be overridden by their delegator's votes + for each validator in validators + tmpValMap(validator.OperatorAddr).Minus = 0 + + // Tally + voterIterator = rangeQuery(Governance, ) //return all the addresses that voted on the proposal + for each (voterAddress, vote) + +in voterIterator + delegations = stakingKeeper.getDelegations(voterAddress) // get all delegations for current voter + for each delegation in delegations + // make sure delegation.Shares does NOT include shares being unbonded + tmpValMap(delegation.ValidatorAddr).Minus += delegation.Shares + proposal.updateTally(vote, delegation.Shares) + + _, isVal = stakingKeeper.getValidator(voterAddress) + if (isVal) + +tmpValMap(voterAddress).Vote = vote + + tallyingParam = load(GlobalParams, 'TallyingParam') + + // Update tally if validator voted + for each validator in validators + if tmpValMap(validator).HasVoted + proposal.updateTally(tmpValMap(validator).Vote, (validator.TotalShares - tmpValMap(validator).Minus)) + + // Check if proposal is accepted or rejected + totalNonAbstain := proposal.YesVotes + proposal.NoVotes + proposal.NoWithVetoVotes + if (proposal.Votes.YesVotes/totalNonAbstain > tallyingParam.Threshold AND proposal.Votes.NoWithVetoVotes/totalNonAbstain < tallyingParam.Veto) + // proposal was accepted at the end of the voting period + // refund deposits (non-voters already punished) + for each (amount, depositor) + +in proposal.Deposits + depositor.AtomBalance += amount + + stateWriter, err := proposal.Handler() + if err != nil + // proposal passed but failed during state execution + proposal.CurrentStatus = ProposalStatusFailed + else + // proposal pass and state is persisted + proposal.CurrentStatus = ProposalStatusAccepted + stateWriter.save() + +else + // proposal was rejected + proposal.CurrentStatus = ProposalStatusRejected + + store(Governance, , proposal) +``` + +### Legacy Proposal + + +Legacy proposals are deprecated. Use the new proposal flow by granting the governance module the right to execute the message. + + +A legacy proposal is the old implementation of governance proposal. +Contrary to proposal that can contain any messages, a legacy proposal allows to submit a set of pre-defined proposals. +These proposals are defined by their types and handled by handlers that are registered in the gov v1beta1 router. + +More information on how to submit proposals in the [client section](#client). + +## Messages + +### Proposal Submission + +Proposals can be submitted by any account via a `MsgSubmitProposal` transaction. + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/gov/v1/tx.proto#L42-L69 +``` + +All `sdk.Msgs` passed into the `messages` field of a `MsgSubmitProposal` message +must be registered in the app's `MsgServiceRouter`. Each of these messages must +have one signer, namely the gov module account. And finally, the metadata length +must not be larger than the `maxMetadataLen` config passed into the gov keeper. +The `initialDeposit` must be strictly positive and conform to the accepted denom of the `MinDeposit` param. + +**State modifications:** + +* Generate new `proposalID` +* Create new `Proposal` +* Initialize `Proposal`'s attributes +* Decrease balance of sender by `InitialDeposit` +* If `MinDeposit` is reached: + * Push `proposalID` in `ProposalProcessingQueue` +* Transfer `InitialDeposit` from the `Proposer` to the governance `ModuleAccount` + +### Deposit + +Once a proposal is submitted, if `Proposal.TotalDeposit < ActiveParam.MinDeposit`, Atom holders can send +`MsgDeposit` transactions to increase the proposal's deposit. + +A deposit is accepted iff: + +* The proposal exists +* The proposal is not in the voting period +* The deposited coins are conform to the accepted denom from the `MinDeposit` param + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/gov/v1/tx.proto#L134-L147 +``` + +**State modifications:** + +* Decrease balance of sender by `deposit` +* Add `deposit` of sender in `proposal.Deposits` +* Increase `proposal.TotalDeposit` by sender's `deposit` +* If `MinDeposit` is reached: + * Push `proposalID` in `ProposalProcessingQueueEnd` +* Transfer `Deposit` from the `proposer` to the governance `ModuleAccount` + +### Vote + +Once `ActiveParam.MinDeposit` is reached, voting period starts. From there, +bonded Atom holders are able to send `MsgVote` transactions to cast their +vote on the proposal. + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/gov/v1/tx.proto#L92-L108 +``` + +**State modifications:** + +* Record `Vote` of sender + + +Gas cost for this message has to take into account the future tallying of the vote in EndBlocker. + + +## Events + +The governance module emits the following events: + +### EndBlocker + +| Type | Attribute Key | Attribute Value | +| ------------------ | ---------------- | ---------------- | +| inactive\_proposal | proposal\_id | `{proposalID}` | +| inactive\_proposal | proposal\_result | `{proposalResult}` | +| active\_proposal | proposal\_id | `{proposalID}` | +| active\_proposal | proposal\_result | `{proposalResult}` | + +### Handlers + +#### MsgSubmitProposal + +| Type | Attribute Key | Attribute Value | +| --------------------- | --------------------- | ---------------- | +| submit\_proposal | proposal\_id | `{proposalID}` | +| submit\_proposal \[0] | voting\_period\_start | `{proposalID}` | +| proposal\_deposit | amount | `{depositAmount}` | +| proposal\_deposit | proposal\_id | `{proposalID}` | +| message | module | governance | +| message | action | submit\_proposal | +| message | sender | `{senderAddress}` | + +* \[0] Event only emitted if the voting period starts during the submission. + +#### MsgVote + +| Type | Attribute Key | Attribute Value | +| -------------- | ------------- | --------------- | +| proposal\_vote | option | `{voteOption}` | +| proposal\_vote | proposal\_id | `{proposalID}` | +| message | module | governance | +| message | action | vote | +| message | sender | `{senderAddress}` | + +#### MsgVoteWeighted + +| Type | Attribute Key | Attribute Value | +| -------------- | ------------- | --------------------- | +| proposal\_vote | option | `{weightedVoteOptions}` | +| proposal\_vote | proposal\_id | `{proposalID}` | +| message | module | governance | +| message | action | vote | +| message | sender | `{senderAddress}` | + +#### MsgDeposit + +| Type | Attribute Key | Attribute Value | +| ---------------------- | --------------------- | --------------- | +| proposal\_deposit | amount | `{depositAmount}` | +| proposal\_deposit | proposal\_id | `{proposalID}` | +| proposal\_deposit \[0] | voting\_period\_start | `{proposalID}` | +| message | module | governance | +| message | action | deposit | +| message | sender | `{senderAddress}` | + +* \[0] Event only emitted if the voting period starts during the submission. + +## Hooks + +The governance module exposes a `GovHooks` interface that allows other modules to react to governance events. + +```go +type GovHooks interface { + AfterProposalSubmission(ctx context.Context, proposalID uint64, proposerAddr sdk.AccAddress) error + AfterProposalDeposit(ctx context.Context, proposalID uint64, depositorAddr sdk.AccAddress) error + AfterProposalVote(ctx context.Context, proposalID uint64, voterAddr sdk.AccAddress) error + AfterProposalFailedMinDeposit(ctx context.Context, proposalID uint64) error + AfterProposalVotingPeriodEnded(ctx context.Context, proposalID uint64) error +} +``` + +### AfterProposalSubmission + +Called after a proposal is submitted. The hook receives the proposal ID and the proposer's address. + +**Note:** The `proposerAddr` parameter was added in a recent release. If you are implementing `GovHooks`, you must update your `AfterProposalSubmission` method signature to include `proposerAddr sdk.AccAddress` as a third parameter. + +**Before:** + +```go +func (h MyGovHooks) AfterProposalSubmission(ctx context.Context, proposalID uint64) error { + // implementation +} +``` + +**After:** + +```go +func (h MyGovHooks) AfterProposalSubmission(ctx context.Context, proposalID uint64, proposerAddr sdk.AccAddress) error { + // implementation +} +``` + +### AfterProposalDeposit + +Called after a deposit is made on a proposal. + +### AfterProposalVote + +Called after a vote is cast on a proposal. + +### AfterProposalFailedMinDeposit + +Called when a proposal fails to reach the minimum deposit within the deposit period. + +### AfterProposalVotingPeriodEnded + +Called when a proposal's voting period ends. + +## Parameters + +The governance module contains the following parameters: + +| Key | Type | Example | +| -------------------------------- | ---------------- | ---------------------------------------- | +| min\_deposit | array (coins) | \[`{"denom":"uatom","amount":"10000000"}`] | +| max\_deposit\_period | string (time ns) | "172800000000000" (17280s) | +| voting\_period | string (time ns) | "172800000000000" (17280s) | +| quorum | string (dec) | "0.334000000000000000" | +| threshold | string (dec) | "0.500000000000000000" | +| veto | string (dec) | "0.334000000000000000" | +| expedited\_threshold | string (time ns) | "0.667000000000000000" | +| expedited\_voting\_period | string (time ns) | "86400000000000" (8600s) | +| expedited\_min\_deposit | array (coins) | \[`{"denom":"uatom","amount":"50000000"}`] | +| burn\_proposal\_deposit\_prevote | bool | false | +| burn\_vote\_quorum | bool | false | +| burn\_vote\_veto | bool | true | +| min\_initial\_deposit\_ratio | string | "0.1" | + +**NOTE**: The governance module contains parameters that are objects unlike other +modules. If only a subset of parameters are desired to be changed, only they need +to be included and not the entire parameter object structure. + +## Client + +### CLI + +A user can query and interact with the `gov` module using the CLI. + +#### Query + +The `query` commands allow users to query `gov` state. + +```bash +simd query gov --help +``` + +##### deposit + +The `deposit` command allows users to query a deposit for a given proposal from a given depositor. + +```bash +simd query gov deposit [proposal-id] [depositer-addr] [flags] +``` + +Example: + +```bash +simd query gov deposit 1 cosmos1.. +``` + +Example Output: + +```bash +amount: +- amount: "100" + denom: stake +depositor: cosmos1.. +proposal_id: "1" +``` + +##### deposits + +The `deposits` command allows users to query all deposits for a given proposal. + +```bash +simd query gov deposits [proposal-id] [flags] +``` + +Example: + +```bash +simd query gov deposits 1 +``` + +Example Output: + +```bash +deposits: +- amount: + - amount: "100" + denom: stake + depositor: cosmos1.. + proposal_id: "1" +pagination: + next_key: null + total: "0" +``` + +##### param + +The `param` command allows users to query a given parameter for the `gov` module. + +```bash +simd query gov param [param-type] [flags] +``` + +Example: + +```bash +simd query gov param voting +``` + +Example Output: + +```bash +voting_period: "172800000000000" +``` + +##### params + +The `params` command allows users to query all parameters for the `gov` module. + +```bash +simd query gov params [flags] +``` + +Example: + +```bash +simd query gov params +``` + +Example Output: + +```bash expandable +deposit_params: + max_deposit_period: 172800s + min_deposit: + - amount: "10000000" + denom: stake +params: + expedited_min_deposit: + - amount: "50000000" + denom: stake + expedited_threshold: "0.670000000000000000" + expedited_voting_period: 86400s + max_deposit_period: 172800s + min_deposit: + - amount: "10000000" + denom: stake + min_initial_deposit_ratio: "0.000000000000000000" + proposal_cancel_burn_rate: "0.500000000000000000" + quorum: "0.334000000000000000" + threshold: "0.500000000000000000" + veto_threshold: "0.334000000000000000" + voting_period: 172800s +tally_params: + quorum: "0.334000000000000000" + threshold: "0.500000000000000000" + veto_threshold: "0.334000000000000000" +voting_params: + voting_period: 172800s +``` + +##### proposal + +The `proposal` command allows users to query a given proposal. + +```bash +simd query gov proposal [proposal-id] [flags] +``` + +Example: + +```bash +simd query gov proposal 1 +``` + +Example Output: + +```bash expandable +deposit_end_time: "2022-03-30T11:50:20.819676256Z" +final_tally_result: + abstain_count: "0" + no_count: "0" + no_with_veto_count: "0" + yes_count: "0" +id: "1" +messages: +- '@type': /cosmos.bank.v1beta1.MsgSend + amount: + - amount: "10" + denom: stake + from_address: cosmos1.. + to_address: cosmos1.. +metadata: AQ== +status: PROPOSAL_STATUS_DEPOSIT_PERIOD +submit_time: "2022-03-28T11:50:20.819676256Z" +total_deposit: +- amount: "10" + denom: stake +voting_end_time: null +voting_start_time: null +``` + +##### proposals + +The `proposals` command allows users to query all proposals with optional filters. + +```bash +simd query gov proposals [flags] +``` + +Example: + +```bash +simd query gov proposals +``` + +Example Output: + +```bash expandable +pagination: + next_key: null + total: "0" +proposals: +- deposit_end_time: "2022-03-30T11:50:20.819676256Z" + final_tally_result: + abstain_count: "0" + no_count: "0" + no_with_veto_count: "0" + yes_count: "0" + id: "1" + messages: + - '@type': /cosmos.bank.v1beta1.MsgSend + amount: + - amount: "10" + denom: stake + from_address: cosmos1.. + to_address: cosmos1.. + metadata: AQ== + status: PROPOSAL_STATUS_DEPOSIT_PERIOD + submit_time: "2022-03-28T11:50:20.819676256Z" + total_deposit: + - amount: "10" + denom: stake + voting_end_time: null + voting_start_time: null +- deposit_end_time: "2022-03-30T14:02:41.165025015Z" + final_tally_result: + abstain_count: "0" + no_count: "0" + no_with_veto_count: "0" + yes_count: "0" + id: "2" + messages: + - '@type': /cosmos.bank.v1beta1.MsgSend + amount: + - amount: "10" + denom: stake + from_address: cosmos1.. + to_address: cosmos1.. + metadata: AQ== + status: PROPOSAL_STATUS_DEPOSIT_PERIOD + submit_time: "2022-03-28T14:02:41.165025015Z" + total_deposit: + - amount: "10" + denom: stake + voting_end_time: null + voting_start_time: null +``` + +##### proposer + +The `proposer` command allows users to query the proposer for a given proposal. + +```bash +simd query gov proposer [proposal-id] [flags] +``` + +Example: + +```bash +simd query gov proposer 1 +``` + +Example Output: + +```bash +proposal_id: "1" +proposer: cosmos1.. +``` + +##### tally + +The `tally` command allows users to query the tally of a given proposal vote. + +```bash +simd query gov tally [proposal-id] [flags] +``` + +Example: + +```bash +simd query gov tally 1 +``` + +Example Output: + +```bash +abstain: "0" +"no": "0" +no_with_veto: "0" +"yes": "1" +``` + +##### vote + +The `vote` command allows users to query a vote for a given proposal. + +```bash +simd query gov vote [proposal-id] [voter-addr] [flags] +``` + +Example: + +```bash +simd query gov vote 1 cosmos1.. +``` + +Example Output: + +```bash +option: VOTE_OPTION_YES +options: +- option: VOTE_OPTION_YES + weight: "1.000000000000000000" +proposal_id: "1" +voter: cosmos1.. +``` + +##### votes + +The `votes` command allows users to query all votes for a given proposal. + +```bash +simd query gov votes [proposal-id] [flags] +``` + +Example: + +```bash +simd query gov votes 1 +``` + +Example Output: + +```bash +pagination: + next_key: null + total: "0" +votes: +- option: VOTE_OPTION_YES + options: + - option: VOTE_OPTION_YES + weight: "1.000000000000000000" + proposal_id: "1" + voter: cosmos1.. +``` + +#### Transactions + +The `tx` commands allow users to interact with the `gov` module. + +```bash +simd tx gov --help +``` + +##### deposit + +The `deposit` command allows users to deposit tokens for a given proposal. + +```bash +simd tx gov deposit [proposal-id] [deposit] [flags] +``` + +Example: + +```bash +simd tx gov deposit 1 10000000stake --from cosmos1.. +``` + +##### draft-proposal + +The `draft-proposal` command allows users to draft any type of proposal. +The command returns a `draft_proposal.json`, to be used by `submit-proposal` after being completed. +The `draft_metadata.json` is meant to be uploaded to [IPFS](#metadata). + +```bash +simd tx gov draft-proposal +``` + +##### submit-proposal + +The `submit-proposal` command allows users to submit a governance proposal along with some messages and metadata. +Messages, metadata and deposit are defined in a JSON file. + +```bash +simd tx gov submit-proposal [path-to-proposal-json] [flags] +``` + +Example: + +```bash +simd tx gov submit-proposal /path/to/proposal.json --from cosmos1.. +``` + +where `proposal.json` contains: + +```json expandable +{ + "messages": [ + { + "@type": "/cosmos.bank.v1beta1.MsgSend", + "from_address": "cosmos1...", // The gov module module address + "to_address": "cosmos1...", + "amount":[{ + "denom": "stake", + "amount": "10"}] + } + ], + "metadata": "AQ==", + "deposit": "10stake", + "title": "Proposal Title", + "summary": "Proposal Summary" +} +``` + + +By default the metadata, summary and title are both limited by 255 characters, this can be overridden by the application developer. + + + +When metadata is not specified, the title is limited to 255 characters and the summary 40x the title length. + + +##### submit-legacy-proposal + +The `submit-legacy-proposal` command allows users to submit a governance legacy proposal along with an initial deposit. + +```bash +simd tx gov submit-legacy-proposal [command] [flags] +``` + +Example: + +```bash +simd tx gov submit-legacy-proposal --title="Test Proposal" --description="testing" --type="Text" --deposit="100000000stake" --from cosmos1.. +``` + +Example (`param-change`): + +```bash +simd tx gov submit-legacy-proposal param-change proposal.json --from cosmos1.. +``` + +```json expandable +{ + "title": "Test Proposal", + "description": "testing, testing, 1, 2, 3", + "changes": [ + { + "subspace": "staking", + "key": "MaxValidators", + "value": 100 + } + ], + "deposit": "10000000stake" +} +``` + +#### cancel-proposal + +Once proposal is canceled, from the deposits of proposal `deposits * proposal_cancel_ratio` will be burned or sent to `ProposalCancelDest` address , if `ProposalCancelDest` is empty then deposits will be burned. The `remaining deposits` will be sent to depositers. + +```bash +simd tx gov cancel-proposal [proposal-id] [flags] +``` + +Example: + +```bash +simd tx gov cancel-proposal 1 --from cosmos1... +``` + +##### vote + +The `vote` command allows users to submit a vote for a given governance proposal. + +```bash +simd tx gov vote [command] [flags] +``` + +Example: + +```bash +simd tx gov vote 1 yes --from cosmos1.. +``` + +##### weighted-vote + +The `weighted-vote` command allows users to submit a weighted vote for a given governance proposal. + +```bash +simd tx gov weighted-vote [proposal-id] [weighted-options] [flags] +``` + +Example: + +```bash +simd tx gov weighted-vote 1 yes=0.5,no=0.5 --from cosmos1.. +``` + +### gRPC + +A user can query the `gov` module using gRPC endpoints. + +#### Proposal + +The `Proposal` endpoint allows users to query a given proposal. + +Using legacy v1beta1: + +```bash +cosmos.gov.v1beta1.Query/Proposal +``` + +Example: + +```bash +grpcurl -plaintext \ + -d '{"proposal_id":"1"}' \ + localhost:9090 \ + cosmos.gov.v1beta1.Query/Proposal +``` + +Example Output: + +```bash expandable +{ + "proposal": { + "proposalId": "1", + "content": {"@type":"/cosmos.gov.v1beta1.TextProposal","description":"testing, testing, 1, 2, 3","title":"Test Proposal"}, + "status": "PROPOSAL_STATUS_VOTING_PERIOD", + "finalTallyResult": { + "yes": "0", + "abstain": "0", + "no": "0", + "noWithVeto": "0" + }, + "submitTime": "2021-09-16T19:40:08.712440474Z", + "depositEndTime": "2021-09-18T19:40:08.712440474Z", + "totalDeposit": [ + { + "denom": "stake", + "amount": "10000000" + } + ], + "votingStartTime": "2021-09-16T19:40:08.712440474Z", + "votingEndTime": "2021-09-18T19:40:08.712440474Z", + "title": "Test Proposal", + "summary": "testing, testing, 1, 2, 3" + } +} +``` + +Using v1: + +```bash +cosmos.gov.v1.Query/Proposal +``` + +Example: + +```bash +grpcurl -plaintext \ + -d '{"proposal_id":"1"}' \ + localhost:9090 \ + cosmos.gov.v1.Query/Proposal +``` + +Example Output: + +```bash expandable +{ + "proposal": { + "id": "1", + "messages": [ + {"@type":"/cosmos.bank.v1beta1.MsgSend","amount":[{"denom":"stake","amount":"10"}],"fromAddress":"cosmos1..","toAddress":"cosmos1.."} + ], + "status": "PROPOSAL_STATUS_VOTING_PERIOD", + "finalTallyResult": { + "yesCount": "0", + "abstainCount": "0", + "noCount": "0", + "noWithVetoCount": "0" + }, + "submitTime": "2022-03-28T11:50:20.819676256Z", + "depositEndTime": "2022-03-30T11:50:20.819676256Z", + "totalDeposit": [ + { + "denom": "stake", + "amount": "10000000" + } + ], + "votingStartTime": "2022-03-28T14:25:26.644857113Z", + "votingEndTime": "2022-03-30T14:25:26.644857113Z", + "metadata": "AQ==", + "title": "Test Proposal", + "summary": "testing, testing, 1, 2, 3" + } +} +``` + +#### Proposals + +The `Proposals` endpoint allows users to query all proposals with optional filters. + +Using legacy v1beta1: + +```bash +cosmos.gov.v1beta1.Query/Proposals +``` + +Example: + +```bash +grpcurl -plaintext \ + localhost:9090 \ + cosmos.gov.v1beta1.Query/Proposals +``` + +Example Output: + +```bash expandable +{ + "proposals": [ + { + "proposalId": "1", + "status": "PROPOSAL_STATUS_VOTING_PERIOD", + "finalTallyResult": { + "yes": "0", + "abstain": "0", + "no": "0", + "noWithVeto": "0" + }, + "submitTime": "2022-03-28T11:50:20.819676256Z", + "depositEndTime": "2022-03-30T11:50:20.819676256Z", + "totalDeposit": [ + { + "denom": "stake", + "amount": "10000000010" + } + ], + "votingStartTime": "2022-03-28T14:25:26.644857113Z", + "votingEndTime": "2022-03-30T14:25:26.644857113Z" + }, + { + "proposalId": "2", + "status": "PROPOSAL_STATUS_DEPOSIT_PERIOD", + "finalTallyResult": { + "yes": "0", + "abstain": "0", + "no": "0", + "noWithVeto": "0" + }, + "submitTime": "2022-03-28T14:02:41.165025015Z", + "depositEndTime": "2022-03-30T14:02:41.165025015Z", + "totalDeposit": [ + { + "denom": "stake", + "amount": "10" + } + ], + "votingStartTime": "0001-01-01T00:00:00Z", + "votingEndTime": "0001-01-01T00:00:00Z" + } + ], + "pagination": { + "total": "2" + } +} + +``` + +Using v1: + +```bash +cosmos.gov.v1.Query/Proposals +``` + +Example: + +```bash +grpcurl -plaintext \ + localhost:9090 \ + cosmos.gov.v1.Query/Proposals +``` + +Example Output: + +```bash expandable +{ + "proposals": [ + { + "id": "1", + "messages": [ + {"@type":"/cosmos.bank.v1beta1.MsgSend","amount":[{"denom":"stake","amount":"10"}],"fromAddress":"cosmos1..","toAddress":"cosmos1.."} + ], + "status": "PROPOSAL_STATUS_VOTING_PERIOD", + "finalTallyResult": { + "yesCount": "0", + "abstainCount": "0", + "noCount": "0", + "noWithVetoCount": "0" + }, + "submitTime": "2022-03-28T11:50:20.819676256Z", + "depositEndTime": "2022-03-30T11:50:20.819676256Z", + "totalDeposit": [ + { + "denom": "stake", + "amount": "10000000010" + } + ], + "votingStartTime": "2022-03-28T14:25:26.644857113Z", + "votingEndTime": "2022-03-30T14:25:26.644857113Z", + "metadata": "AQ==", + "title": "Proposal Title", + "summary": "Proposal Summary" + }, + { + "id": "2", + "messages": [ + {"@type":"/cosmos.bank.v1beta1.MsgSend","amount":[{"denom":"stake","amount":"10"}],"fromAddress":"cosmos1..","toAddress":"cosmos1.."} + ], + "status": "PROPOSAL_STATUS_DEPOSIT_PERIOD", + "finalTallyResult": { + "yesCount": "0", + "abstainCount": "0", + "noCount": "0", + "noWithVetoCount": "0" + }, + "submitTime": "2022-03-28T14:02:41.165025015Z", + "depositEndTime": "2022-03-30T14:02:41.165025015Z", + "totalDeposit": [ + { + "denom": "stake", + "amount": "10" + } + ], + "metadata": "AQ==", + "title": "Proposal Title", + "summary": "Proposal Summary" + } + ], + "pagination": { + "total": "2" + } +} +``` + +#### Vote + +The `Vote` endpoint allows users to query a vote for a given proposal. + +Using legacy v1beta1: + +```bash +cosmos.gov.v1beta1.Query/Vote +``` + +Example: + +```bash +grpcurl -plaintext \ + -d '{"proposal_id":"1","voter":"cosmos1.."}' \ + localhost:9090 \ + cosmos.gov.v1beta1.Query/Vote +``` + +Example Output: + +```bash expandable +{ + "vote": { + "proposalId": "1", + "voter": "cosmos1..", + "option": "VOTE_OPTION_YES", + "options": [ + { + "option": "VOTE_OPTION_YES", + "weight": "1000000000000000000" + } + ] + } +} +``` + +Using v1: + +```bash +cosmos.gov.v1.Query/Vote +``` + +Example: + +```bash +grpcurl -plaintext \ + -d '{"proposal_id":"1","voter":"cosmos1.."}' \ + localhost:9090 \ + cosmos.gov.v1.Query/Vote +``` + +Example Output: + +```bash expandable +{ + "vote": { + "proposalId": "1", + "voter": "cosmos1..", + "option": "VOTE_OPTION_YES", + "options": [ + { + "option": "VOTE_OPTION_YES", + "weight": "1.000000000000000000" + } + ] + } +} +``` + +#### Votes + +The `Votes` endpoint allows users to query all votes for a given proposal. + +Using legacy v1beta1: + +```bash +cosmos.gov.v1beta1.Query/Votes +``` + +Example: + +```bash +grpcurl -plaintext \ + -d '{"proposal_id":"1"}' \ + localhost:9090 \ + cosmos.gov.v1beta1.Query/Votes +``` + +Example Output: + +```bash expandable +{ + "votes": [ + { + "proposalId": "1", + "voter": "cosmos1..", + "options": [ + { + "option": "VOTE_OPTION_YES", + "weight": "1000000000000000000" + } + ] + } + ], + "pagination": { + "total": "1" + } +} +``` + +Using v1: + +```bash +cosmos.gov.v1.Query/Votes +``` + +Example: + +```bash +grpcurl -plaintext \ + -d '{"proposal_id":"1"}' \ + localhost:9090 \ + cosmos.gov.v1.Query/Votes +``` + +Example Output: + +```bash expandable +{ + "votes": [ + { + "proposalId": "1", + "voter": "cosmos1..", + "options": [ + { + "option": "VOTE_OPTION_YES", + "weight": "1.000000000000000000" + } + ] + } + ], + "pagination": { + "total": "1" + } +} +``` + +#### Params + +The `Params` endpoint allows users to query all parameters for the `gov` module. + +{/* TODO: #10197 Querying governance params outputs nil values */} + +Using legacy v1beta1: + +```bash +cosmos.gov.v1beta1.Query/Params +``` + +Example: + +```bash +grpcurl -plaintext \ + -d '{"params_type":"voting"}' \ + localhost:9090 \ + cosmos.gov.v1beta1.Query/Params +``` + +Example Output: + +```bash expandable +{ + "votingParams": { + "votingPeriod": "172800s" + }, + "depositParams": { + "maxDepositPeriod": "0s" + }, + "tallyParams": { + "quorum": "MA==", + "threshold": "MA==", + "vetoThreshold": "MA==" + } +} +``` + +Using v1: + +```bash +cosmos.gov.v1.Query/Params +``` + +Example: + +```bash +grpcurl -plaintext \ + -d '{"params_type":"voting"}' \ + localhost:9090 \ + cosmos.gov.v1.Query/Params +``` + +Example Output: + +```bash +{ + "votingParams": { + "votingPeriod": "172800s" + } +} +``` + +#### Deposit + +The `Deposit` endpoint allows users to query a deposit for a given proposal from a given depositor. + +Using legacy v1beta1: + +```bash +cosmos.gov.v1beta1.Query/Deposit +``` + +Example: + +```bash +grpcurl -plaintext \ + '{"proposal_id":"1","depositor":"cosmos1.."}' \ + localhost:9090 \ + cosmos.gov.v1beta1.Query/Deposit +``` + +Example Output: + +```bash expandable +{ + "deposit": { + "proposalId": "1", + "depositor": "cosmos1..", + "amount": [ + { + "denom": "stake", + "amount": "10000000" + } + ] + } +} +``` + +Using v1: + +```bash +cosmos.gov.v1.Query/Deposit +``` + +Example: + +```bash +grpcurl -plaintext \ + '{"proposal_id":"1","depositor":"cosmos1.."}' \ + localhost:9090 \ + cosmos.gov.v1.Query/Deposit +``` + +Example Output: + +```bash expandable +{ + "deposit": { + "proposalId": "1", + "depositor": "cosmos1..", + "amount": [ + { + "denom": "stake", + "amount": "10000000" + } + ] + } +} +``` + +#### deposits + +The `Deposits` endpoint allows users to query all deposits for a given proposal. + +Using legacy v1beta1: + +```bash +cosmos.gov.v1beta1.Query/Deposits +``` + +Example: + +```bash +grpcurl -plaintext \ + -d '{"proposal_id":"1"}' \ + localhost:9090 \ + cosmos.gov.v1beta1.Query/Deposits +``` + +Example Output: + +```bash expandable +{ + "deposits": [ + { + "proposalId": "1", + "depositor": "cosmos1..", + "amount": [ + { + "denom": "stake", + "amount": "10000000" + } + ] + } + ], + "pagination": { + "total": "1" + } +} +``` + +Using v1: + +```bash +cosmos.gov.v1.Query/Deposits +``` + +Example: + +```bash +grpcurl -plaintext \ + -d '{"proposal_id":"1"}' \ + localhost:9090 \ + cosmos.gov.v1.Query/Deposits +``` + +Example Output: + +```bash expandable +{ + "deposits": [ + { + "proposalId": "1", + "depositor": "cosmos1..", + "amount": [ + { + "denom": "stake", + "amount": "10000000" + } + ] + } + ], + "pagination": { + "total": "1" + } +} +``` + +#### TallyResult + +The `TallyResult` endpoint allows users to query the tally of a given proposal. + +Using legacy v1beta1: + +```bash +cosmos.gov.v1beta1.Query/TallyResult +``` + +Example: + +```bash +grpcurl -plaintext \ + -d '{"proposal_id":"1"}' \ + localhost:9090 \ + cosmos.gov.v1beta1.Query/TallyResult +``` + +Example Output: + +```bash +{ + "tally": { + "yes": "1000000", + "abstain": "0", + "no": "0", + "noWithVeto": "0" + } +} +``` + +Using v1: + +```bash +cosmos.gov.v1.Query/TallyResult +``` + +Example: + +```bash +grpcurl -plaintext \ + -d '{"proposal_id":"1"}' \ + localhost:9090 \ + cosmos.gov.v1.Query/TallyResult +``` + +Example Output: + +```bash +{ + "tally": { + "yes": "1000000", + "abstain": "0", + "no": "0", + "noWithVeto": "0" + } +} +``` + +### REST + +A user can query the `gov` module using REST endpoints. + +#### proposal + +The `proposals` endpoint allows users to query a given proposal. + +Using legacy v1beta1: + +```bash +/cosmos/gov/v1beta1/proposals/{proposal_id} +``` + +Example: + +```bash +curl localhost:1317/cosmos/gov/v1beta1/proposals/1 +``` + +Example Output: + +```bash expandable +{ + "proposal": { + "proposal_id": "1", + "content": null, + "status": "PROPOSAL_STATUS_VOTING_PERIOD", + "final_tally_result": { + "yes": "0", + "abstain": "0", + "no": "0", + "no_with_veto": "0" + }, + "submit_time": "2022-03-28T11:50:20.819676256Z", + "deposit_end_time": "2022-03-30T11:50:20.819676256Z", + "total_deposit": [ + { + "denom": "stake", + "amount": "10000000010" + } + ], + "voting_start_time": "2022-03-28T14:25:26.644857113Z", + "voting_end_time": "2022-03-30T14:25:26.644857113Z" + } +} +``` + +Using v1: + +```bash +/cosmos/gov/v1/proposals/{proposal_id} +``` + +Example: + +```bash +curl localhost:1317/cosmos/gov/v1/proposals/1 +``` + +Example Output: + +```bash expandable +{ + "proposal": { + "id": "1", + "messages": [ + { + "@type": "/cosmos.bank.v1beta1.MsgSend", + "from_address": "cosmos1..", + "to_address": "cosmos1..", + "amount": [ + { + "denom": "stake", + "amount": "10" + } + ] + } + ], + "status": "PROPOSAL_STATUS_VOTING_PERIOD", + "final_tally_result": { + "yes_count": "0", + "abstain_count": "0", + "no_count": "0", + "no_with_veto_count": "0" + }, + "submit_time": "2022-03-28T11:50:20.819676256Z", + "deposit_end_time": "2022-03-30T11:50:20.819676256Z", + "total_deposit": [ + { + "denom": "stake", + "amount": "10000000" + } + ], + "voting_start_time": "2022-03-28T14:25:26.644857113Z", + "voting_end_time": "2022-03-30T14:25:26.644857113Z", + "metadata": "AQ==", + "title": "Proposal Title", + "summary": "Proposal Summary" + } +} +``` + +#### proposals + +The `proposals` endpoint also allows users to query all proposals with optional filters. + +Using legacy v1beta1: + +```bash +/cosmos/gov/v1beta1/proposals +``` + +Example: + +```bash +curl localhost:1317/cosmos/gov/v1beta1/proposals +``` + +Example Output: + +```bash expandable +{ + "proposals": [ + { + "proposal_id": "1", + "content": null, + "status": "PROPOSAL_STATUS_VOTING_PERIOD", + "final_tally_result": { + "yes": "0", + "abstain": "0", + "no": "0", + "no_with_veto": "0" + }, + "submit_time": "2022-03-28T11:50:20.819676256Z", + "deposit_end_time": "2022-03-30T11:50:20.819676256Z", + "total_deposit": [ + { + "denom": "stake", + "amount": "10000000" + } + ], + "voting_start_time": "2022-03-28T14:25:26.644857113Z", + "voting_end_time": "2022-03-30T14:25:26.644857113Z" + }, + { + "proposal_id": "2", + "content": null, + "status": "PROPOSAL_STATUS_DEPOSIT_PERIOD", + "final_tally_result": { + "yes": "0", + "abstain": "0", + "no": "0", + "no_with_veto": "0" + }, + "submit_time": "2022-03-28T14:02:41.165025015Z", + "deposit_end_time": "2022-03-30T14:02:41.165025015Z", + "total_deposit": [ + { + "denom": "stake", + "amount": "10" + } + ], + "voting_start_time": "0001-01-01T00:00:00Z", + "voting_end_time": "0001-01-01T00:00:00Z" + } + ], + "pagination": { + "next_key": null, + "total": "2" + } +} +``` + +Using v1: + +```bash +/cosmos/gov/v1/proposals +``` + +Example: + +```bash +curl localhost:1317/cosmos/gov/v1/proposals +``` + +Example Output: + +```bash expandable +{ + "proposals": [ + { + "id": "1", + "messages": [ + { + "@type": "/cosmos.bank.v1beta1.MsgSend", + "from_address": "cosmos1..", + "to_address": "cosmos1..", + "amount": [ + { + "denom": "stake", + "amount": "10" + } + ] + } + ], + "status": "PROPOSAL_STATUS_VOTING_PERIOD", + "final_tally_result": { + "yes_count": "0", + "abstain_count": "0", + "no_count": "0", + "no_with_veto_count": "0" + }, + "submit_time": "2022-03-28T11:50:20.819676256Z", + "deposit_end_time": "2022-03-30T11:50:20.819676256Z", + "total_deposit": [ + { + "denom": "stake", + "amount": "10000000010" + } + ], + "voting_start_time": "2022-03-28T14:25:26.644857113Z", + "voting_end_time": "2022-03-30T14:25:26.644857113Z", + "metadata": "AQ==", + "title": "Proposal Title", + "summary": "Proposal Summary" + }, + { + "id": "2", + "messages": [ + { + "@type": "/cosmos.bank.v1beta1.MsgSend", + "from_address": "cosmos1..", + "to_address": "cosmos1..", + "amount": [ + { + "denom": "stake", + "amount": "10" + } + ] + } + ], + "status": "PROPOSAL_STATUS_DEPOSIT_PERIOD", + "final_tally_result": { + "yes_count": "0", + "abstain_count": "0", + "no_count": "0", + "no_with_veto_count": "0" + }, + "submit_time": "2022-03-28T14:02:41.165025015Z", + "deposit_end_time": "2022-03-30T14:02:41.165025015Z", + "total_deposit": [ + { + "denom": "stake", + "amount": "10" + } + ], + "voting_start_time": null, + "voting_end_time": null, + "metadata": "AQ==", + "title": "Proposal Title", + "summary": "Proposal Summary" + } + ], + "pagination": { + "next_key": null, + "total": "2" + } +} +``` + +#### voter vote + +The `votes` endpoint allows users to query a vote for a given proposal. + +Using legacy v1beta1: + +```bash +/cosmos/gov/v1beta1/proposals/{proposal_id}/votes/{voter} +``` + +Example: + +```bash +curl localhost:1317/cosmos/gov/v1beta1/proposals/1/votes/cosmos1.. +``` + +Example Output: + +```bash expandable +{ + "vote": { + "proposal_id": "1", + "voter": "cosmos1..", + "option": "VOTE_OPTION_YES", + "options": [ + { + "option": "VOTE_OPTION_YES", + "weight": "1.000000000000000000" + } + ] + } +} +``` + +Using v1: + +```bash +/cosmos/gov/v1/proposals/{proposal_id}/votes/{voter} +``` + +Example: + +```bash +curl localhost:1317/cosmos/gov/v1/proposals/1/votes/cosmos1.. +``` + +Example Output: + +```bash expandable +{ + "vote": { + "proposal_id": "1", + "voter": "cosmos1..", + "options": [ + { + "option": "VOTE_OPTION_YES", + "weight": "1.000000000000000000" + } + ], + "metadata": "" + } +} +``` + +#### votes + +The `votes` endpoint allows users to query all votes for a given proposal. + +Using legacy v1beta1: + +```bash +/cosmos/gov/v1beta1/proposals/{proposal_id}/votes +``` + +Example: + +```bash +curl localhost:1317/cosmos/gov/v1beta1/proposals/1/votes +``` + +Example Output: + +```bash expandable +{ + "votes": [ + { + "proposal_id": "1", + "voter": "cosmos1..", + "option": "VOTE_OPTION_YES", + "options": [ + { + "option": "VOTE_OPTION_YES", + "weight": "1.000000000000000000" + } + ] + } + ], + "pagination": { + "next_key": null, + "total": "1" + } +} +``` + +Using v1: + +```bash +/cosmos/gov/v1/proposals/{proposal_id}/votes +``` + +Example: + +```bash +curl localhost:1317/cosmos/gov/v1/proposals/1/votes +``` + +Example Output: + +```bash expandable +{ + "votes": [ + { + "proposal_id": "1", + "voter": "cosmos1..", + "options": [ + { + "option": "VOTE_OPTION_YES", + "weight": "1.000000000000000000" + } + ], + "metadata": "" + } + ], + "pagination": { + "next_key": null, + "total": "1" + } +} +``` + +#### params + +The `params` endpoint allows users to query all parameters for the `gov` module. + +{/* TODO: #10197 Querying governance params outputs nil values */} + +Using legacy v1beta1: + +```bash +/cosmos/gov/v1beta1/params/{params_type} +``` + +Example: + +```bash +curl localhost:1317/cosmos/gov/v1beta1/params/voting +``` + +Example Output: + +```bash expandable +{ + "voting_params": { + "voting_period": "172800s" + }, + "deposit_params": { + "min_deposit": [ + ], + "max_deposit_period": "0s" + }, + "tally_params": { + "quorum": "0.000000000000000000", + "threshold": "0.000000000000000000", + "veto_threshold": "0.000000000000000000" + } +} +``` + +Using v1: + +```bash +/cosmos/gov/v1/params/{params_type} +``` + +Example: + +```bash +curl localhost:1317/cosmos/gov/v1/params/voting +``` + +Example Output: + +```bash expandable +{ + "voting_params": { + "voting_period": "172800s" + }, + "deposit_params": { + "min_deposit": [ + ], + "max_deposit_period": "0s" + }, + "tally_params": { + "quorum": "0.000000000000000000", + "threshold": "0.000000000000000000", + "veto_threshold": "0.000000000000000000" + } +} +``` + +#### deposits + +The `deposits` endpoint allows users to query a deposit for a given proposal from a given depositor. + +Using legacy v1beta1: + +```bash +/cosmos/gov/v1beta1/proposals/{proposal_id}/deposits/{depositor} +``` + +Example: + +```bash +curl localhost:1317/cosmos/gov/v1beta1/proposals/1/deposits/cosmos1.. +``` + +Example Output: + +```bash expandable +{ + "deposit": { + "proposal_id": "1", + "depositor": "cosmos1..", + "amount": [ + { + "denom": "stake", + "amount": "10000000" + } + ] + } +} +``` + +Using v1: + +```bash +/cosmos/gov/v1/proposals/{proposal_id}/deposits/{depositor} +``` + +Example: + +```bash +curl localhost:1317/cosmos/gov/v1/proposals/1/deposits/cosmos1.. +``` + +Example Output: + +```bash expandable +{ + "deposit": { + "proposal_id": "1", + "depositor": "cosmos1..", + "amount": [ + { + "denom": "stake", + "amount": "10000000" + } + ] + } +} +``` + +#### proposal deposits + +The `deposits` endpoint allows users to query all deposits for a given proposal. + +Using legacy v1beta1: + +```bash +/cosmos/gov/v1beta1/proposals/{proposal_id}/deposits +``` + +Example: + +```bash +curl localhost:1317/cosmos/gov/v1beta1/proposals/1/deposits +``` + +Example Output: + +```bash expandable +{ + "deposits": [ + { + "proposal_id": "1", + "depositor": "cosmos1..", + "amount": [ + { + "denom": "stake", + "amount": "10000000" + } + ] + } + ], + "pagination": { + "next_key": null, + "total": "1" + } +} +``` + +Using v1: + +```bash +/cosmos/gov/v1/proposals/{proposal_id}/deposits +``` + +Example: + +```bash +curl localhost:1317/cosmos/gov/v1/proposals/1/deposits +``` + +Example Output: + +```bash expandable +{ + "deposits": [ + { + "proposal_id": "1", + "depositor": "cosmos1..", + "amount": [ + { + "denom": "stake", + "amount": "10000000" + } + ] + } + ], + "pagination": { + "next_key": null, + "total": "1" + } +} +``` + +#### tally + +The `tally` endpoint allows users to query the tally of a given proposal. + +Using legacy v1beta1: + +```bash +/cosmos/gov/v1beta1/proposals/{proposal_id}/tally +``` + +Example: + +```bash +curl localhost:1317/cosmos/gov/v1beta1/proposals/1/tally +``` + +Example Output: + +```bash +{ + "tally": { + "yes": "1000000", + "abstain": "0", + "no": "0", + "no_with_veto": "0" + } +} +``` + +Using v1: + +```bash +/cosmos/gov/v1/proposals/{proposal_id}/tally +``` + +Example: + +```bash +curl localhost:1317/cosmos/gov/v1/proposals/1/tally +``` + +Example Output: + +```bash +{ + "tally": { + "yes": "1000000", + "abstain": "0", + "no": "0", + "no_with_veto": "0" + } +} +``` + +## Metadata + +The gov module has two locations for metadata where users can provide further context about the on-chain actions they are taking. By default all metadata fields have a 255 character length field where metadata can be stored in json format, either on-chain or off-chain depending on the amount of data required. Here we provide a recommendation for the json structure and where the data should be stored. There are two important factors in making these recommendations. First, that the gov and group modules are consistent with one another, note the number of proposals made by all groups may be quite large. Second, that client applications such as block explorers and governance interfaces have confidence in the consistency of metadata structure accross chains. + +### Proposal + +Location: off-chain as json object stored on IPFS (mirrors [group proposal](/sdk/v0.54/modules/group/README#metadata)) + +```json +{ + "title": "", + "authors": [""], + "summary": "", + "details": "", + "proposal_forum_url": "", + "vote_option_context": "", +} +``` + + +The `authors` field is an array of strings, this is to allow for multiple authors to be listed in the metadata. +In v0.46, the `authors` field is a comma-separated string. Frontends are encouraged to support both formats for backwards compatibility. + + +### Vote + +Location: on-chain as json within 255 character limit (mirrors [group vote](/sdk/v0.54/modules/group/README#metadata)) + +```json +{ + "justification": "", +} +``` + +## Future Improvements + +The current documentation only describes the minimum viable product for the +governance module. Future improvements may include: + +* **`BountyProposals`:** If accepted, a `BountyProposal` creates an open + bounty. The `BountyProposal` specifies how many Atoms will be given upon + completion. These Atoms will be taken from the `reserve pool`. After a + `BountyProposal` is accepted by governance, anybody can submit a + `SoftwareUpgradeProposal` with the code to claim the bounty. Note that once a + `BountyProposal` is accepted, the corresponding funds in the `reserve pool` + are locked so that payment can always be honored. In order to link a + `SoftwareUpgradeProposal` to an open bounty, the submitter of the + `SoftwareUpgradeProposal` will use the `Proposal.LinkedProposal` attribute. + If a `SoftwareUpgradeProposal` linked to an open bounty is accepted by + governance, the funds that were reserved are automatically transferred to the + submitter. +* **Complex delegation:** Delegators could choose other representatives than + their validators. Ultimately, the chain of representatives would always end + up to a validator, but delegators could inherit the vote of their chosen + representative before they inherit the vote of their validator. In other + words, they would only inherit the vote of their validator if their other + appointed representative did not vote. +* **Better process for proposal review:** There would be two parts to + `proposal.Deposit`, one for anti-spam (same as in MVP) and an other one to + reward third party auditors. diff --git a/sdk/v0.54/modules/group/README.mdx b/sdk/v0.54/modules/group/README.mdx new file mode 100644 index 000000000..55f6800ef --- /dev/null +++ b/sdk/v0.54/modules/group/README.mdx @@ -0,0 +1,2170 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/modules/group/README' +title: 'x/group' +description: The following documents specify the group module. +--- + + +The `x/group` module is now maintained under the Cosmos Enterprise offering. If your application uses `x/group`, you will need to migrate your code to the Enterprise-distributed package and obtain a Cosmos Enterprise license to continue using it. Please see [Cosmos Enterprise](/sdk/v0.54/enterprise/overview) to learn more. + + +## Abstract + +The following documents specify the group module. + +This module allows the creation and management of on-chain multisig accounts and enables voting for message execution based on configurable decision policies. + +## Contents + +* [Concepts](#concepts) + * [Group](#group) + * [Group Policy](#group-policy) + * [Decision Policy](#decision-policy) + * [Proposal](#proposal) + * [Pruning](#pruning) +* [State](#state) + * [Group Table](#group-table) + * [Group Member Table](#group-member-table) + * [Group Policy Table](#group-policy-table) + * [Proposal Table](#proposal-table) + * [Vote Table](#vote-table) +* [Msg Service](#msg-service) + * [Msg/CreateGroup](#msgcreategroup) + * [Msg/UpdateGroupMembers](#msgupdategroupmembers) + * [Msg/UpdateGroupAdmin](#msgupdategroupadmin) + * [Msg/UpdateGroupMetadata](#msgupdategroupmetadata) + * [Msg/CreateGroupPolicy](#msgcreategrouppolicy) + * [Msg/CreateGroupWithPolicy](#msgcreategroupwithpolicy) + * [Msg/UpdateGroupPolicyAdmin](#msgupdategrouppolicyadmin) + * [Msg/UpdateGroupPolicyDecisionPolicy](#msgupdategrouppolicydecisionpolicy) + * [Msg/UpdateGroupPolicyMetadata](#msgupdategrouppolicymetadata) + * [Msg/SubmitProposal](#msgsubmitproposal) + * [Msg/WithdrawProposal](#msgwithdrawproposal) + * [Msg/Vote](#msgvote) + * [Msg/Exec](#msgexec) + * [Msg/LeaveGroup](#msgleavegroup) +* [Events](#events) + * [EventCreateGroup](#eventcreategroup) + * [EventUpdateGroup](#eventupdategroup) + * [EventCreateGroupPolicy](#eventcreategrouppolicy) + * [EventUpdateGroupPolicy](#eventupdategrouppolicy) + * [EventCreateProposal](#eventcreateproposal) + * [EventWithdrawProposal](#eventwithdrawproposal) + * [EventVote](#eventvote) + * [EventExec](#eventexec) + * [EventLeaveGroup](#eventleavegroup) + * [EventProposalPruned](#eventproposalpruned) +* [Client](#client) + * [CLI](#cli) + * [gRPC](#grpc) + * [REST](#rest) +* [Metadata](#metadata) + +## Concepts + +### Group + +A group is simply an aggregation of accounts with associated weights. It is not +an account and doesn't have a balance. It doesn't in and of itself have any +sort of voting or decision weight. It does have an "administrator" which has +the ability to add, remove and update members in the group. Note that a +group policy account could be an administrator of a group, and that the +administrator doesn't necessarily have to be a member of the group. + +### Group Policy + +A group policy is an account associated with a group and a decision policy. +Group policies are abstracted from groups because a single group may have +multiple decision policies for different types of actions. Managing group +membership separately from decision policies results in the least overhead +and keeps membership consistent across different policies. The pattern that +is recommended is to have a single master group policy for a given group, +and then to create separate group policies with different decision policies +and delegate the desired permissions from the master account to +those "sub-accounts" using the `x/authz` module. + +### Decision Policy + +A decision policy is the mechanism by which members of a group can vote on +proposals, as well as the rules that dictate whether a proposal should pass +or not based on its tally outcome. + +All decision policies generally would have a mininum execution period and a +maximum voting window. The minimum execution period is the minimum amount of time +that must pass after submission in order for a proposal to potentially be executed, and it may +be set to 0. The maximum voting window is the maximum time after submission that a proposal may +be voted on before it is tallied. + +The chain developer also defines an app-wide maximum execution period, which is +the maximum amount of time after a proposal's voting period end where users are +allowed to execute a proposal. + +The current group module comes shipped with two decision policies: threshold +and percentage. Any chain developer can extend upon these two, by creating +custom decision policies, as long as they adhere to the `DecisionPolicy` +interface: + +```go +// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/x/group/types.go#L27-L45 +``` + +#### Threshold decision policy + +A threshold decision policy defines a threshold of yes votes (based on a tally +of voter weights) that must be achieved in order for a proposal to pass. For +this decision policy, abstain and veto are simply treated as no's. + +This decision policy also has a VotingPeriod window and a MinExecutionPeriod +window. The former defines the duration after proposal submission where members +are allowed to vote, after which tallying is performed. The latter specifies +the minimum duration after proposal submission where the proposal can be +executed. If set to 0, then the proposal is allowed to be executed immediately +on submission (using the `TRY_EXEC` option). Obviously, MinExecutionPeriod +cannot be greater than VotingPeriod+MaxExecutionPeriod (where MaxExecution is +the app-defined duration that specifies the window after voting ended where a +proposal can be executed). + +#### Percentage decision policy + +A percentage decision policy is similar to a threshold decision policy, except +that the threshold is not defined as a constant weight, but as a percentage. +It's more suited for groups where the group members' weights can be updated, as +the percentage threshold stays the same, and doesn't depend on how those member +weights get updated. + +Same as the Threshold decision policy, the percentage decision policy has the +two VotingPeriod and MinExecutionPeriod parameters. + +### Proposal + +Any member(s) of a group can submit a proposal for a group policy account to decide upon. +A proposal consists of a set of messages that will be executed if the proposal +passes as well as any metadata associated with the proposal. + +#### Voting + +There are four choices to choose while voting - yes, no, abstain and veto. Not +all decision policies will take the four choices into account. Votes can contain some optional metadata. +In the current implementation, the voting window begins as soon as a proposal +is submitted, and the end is defined by the group policy's decision policy. + +#### Withdrawing Proposals + +Proposals can be withdrawn any time before the voting period end, either by the +admin of the group policy or by one of the proposers. Once withdrawn, it is +marked as `PROPOSAL_STATUS_WITHDRAWN`, and no more voting or execution is +allowed on it. + +#### Aborted Proposals + +If the group policy is updated during the voting period of the proposal, then +the proposal is marked as `PROPOSAL_STATUS_ABORTED`, and no more voting or +execution is allowed on it. This is because the group policy defines the rules +of proposal voting and execution, so if those rules change during the lifecycle +of a proposal, then the proposal should be marked as stale. + +#### Tallying + +Tallying is the counting of all votes on a proposal. It happens only once in +the lifecycle of a proposal, but can be triggered by two factors, whichever +happens first: + +* either someone tries to execute the proposal (see next section), which can + happen on a `Msg/Exec` transaction, or a `Msg/{SubmitProposal,Vote}` + transaction with the `Exec` field set. When a proposal execution is attempted, + a tally is done first to make sure the proposal passes. +* or on `EndBlock` when the proposal's voting period end just passed. + +If the tally result passes the decision policy's rules, then the proposal is +marked as `PROPOSAL_STATUS_ACCEPTED`, or else it is marked as +`PROPOSAL_STATUS_REJECTED`. In any case, no more voting is allowed anymore, and the tally +result is persisted to state in the proposal's `FinalTallyResult`. + +#### Executing Proposals + +Proposals are executed only when the tallying is done, and the group account's +decision policy allows the proposal to pass based on the tally outcome. They +are marked by the status `PROPOSAL_STATUS_ACCEPTED`. Execution must happen +before a duration of `MaxExecutionPeriod` (set by the chain developer) after +each proposal's voting period end. + +Proposals will not be automatically executed by the chain in this current design, +but rather a user must submit a `Msg/Exec` transaction to attempt to execute the +proposal based on the current votes and decision policy. Any user (not only the +group members) can execute proposals that have been accepted, and execution fees are +paid by the proposal executor. +It's also possible to try to execute a proposal immediately on creation or on +new votes using the `Exec` field of `Msg/SubmitProposal` and `Msg/Vote` requests. +In the former case, proposers signatures are considered as yes votes. +In these cases, if the proposal can't be executed (i.e. it didn't pass the +decision policy's rules), it will still be opened for new votes and +could be tallied and executed later on. + +A successful proposal execution will have its `ExecutorResult` marked as +`PROPOSAL_EXECUTOR_RESULT_SUCCESS`. The proposal will be automatically pruned +after execution. On the other hand, a failed proposal execution will be marked +as `PROPOSAL_EXECUTOR_RESULT_FAILURE`. Such a proposal can be re-executed +multiple times, until it expires after `MaxExecutionPeriod` after voting period +end. + +### Pruning + +Proposals and votes are automatically pruned to avoid state bloat. + +Votes are pruned: + +* either after a successful tally, i.e. a tally whose result passes the decision + policy's rules, which can be trigged by a `Msg/Exec` or a + `Msg/{SubmitProposal,Vote}` with the `Exec` field set, +* or on `EndBlock` right after the proposal's voting period end. This applies to proposals with status `aborted` or `withdrawn` too. + +whichever happens first. + +Proposals are pruned: + +* on `EndBlock` whose proposal status is `withdrawn` or `aborted` on proposal's voting period end before tallying, +* and either after a successful proposal execution, +* or on `EndBlock` right after the proposal's `voting_period_end` + + `max_execution_period` (defined as an app-wide configuration) is passed, + +whichever happens first. + +## State + +The `group` module uses the `orm` package which provides table storage with support for +primary keys and secondary indexes. `orm` also defines `Sequence` which is a persistent unique key generator based on a counter that can be used along with `Table`s. + +Here's the list of tables and associated sequences and indexes stored as part of the `group` module. + +### Group Table + +The `groupTable` stores `GroupInfo`: `0x0 | BigEndian(GroupId) -> ProtocolBuffer(GroupInfo)`. + +#### groupSeq + +The value of `groupSeq` is incremented when creating a new group and corresponds to the new `GroupId`: `0x1 | 0x1 -> BigEndian`. + +The second `0x1` corresponds to the ORM `sequenceStorageKey`. + +#### groupByAdminIndex + +`groupByAdminIndex` allows to retrieve groups by admin address: +`0x2 | len([]byte(group.Admin)) | []byte(group.Admin) | BigEndian(GroupId) -> []byte()`. + +### Group Member Table + +The `groupMemberTable` stores `GroupMember`s: `0x10 | BigEndian(GroupId) | []byte(member.Address) -> ProtocolBuffer(GroupMember)`. + +The `groupMemberTable` is a primary key table and its `PrimaryKey` is given by +`BigEndian(GroupId) | []byte(member.Address)` which is used by the following indexes. + +#### groupMemberByGroupIndex + +`groupMemberByGroupIndex` allows to retrieve group members by group id: +`0x11 | BigEndian(GroupId) | PrimaryKey -> []byte()`. + +#### groupMemberByMemberIndex + +`groupMemberByMemberIndex` allows to retrieve group members by member address: +`0x12 | len([]byte(member.Address)) | []byte(member.Address) | PrimaryKey -> []byte()`. + +### Group Policy Table + +The `groupPolicyTable` stores `GroupPolicyInfo`: `0x20 | len([]byte(Address)) | []byte(Address) -> ProtocolBuffer(GroupPolicyInfo)`. + +The `groupPolicyTable` is a primary key table and its `PrimaryKey` is given by +`len([]byte(Address)) | []byte(Address)` which is used by the following indexes. + +#### groupPolicySeq + +The value of `groupPolicySeq` is incremented when creating a new group policy and is used to generate the new group policy account `Address`: +`0x21 | 0x1 -> BigEndian`. + +The second `0x1` corresponds to the ORM `sequenceStorageKey`. + +#### groupPolicyByGroupIndex + +`groupPolicyByGroupIndex` allows to retrieve group policies by group id: +`0x22 | BigEndian(GroupId) | PrimaryKey -> []byte()`. + +#### groupPolicyByAdminIndex + +`groupPolicyByAdminIndex` allows to retrieve group policies by admin address: +`0x23 | len([]byte(Address)) | []byte(Address) | PrimaryKey -> []byte()`. + +### Proposal Table + +The `proposalTable` stores `Proposal`s: `0x30 | BigEndian(ProposalId) -> ProtocolBuffer(Proposal)`. + +#### proposalSeq + +The value of `proposalSeq` is incremented when creating a new proposal and corresponds to the new `ProposalId`: `0x31 | 0x1 -> BigEndian`. + +The second `0x1` corresponds to the ORM `sequenceStorageKey`. + +#### proposalByGroupPolicyIndex + +`proposalByGroupPolicyIndex` allows to retrieve proposals by group policy account address: +`0x32 | len([]byte(account.Address)) | []byte(account.Address) | BigEndian(ProposalId) -> []byte()`. + +#### ProposalsByVotingPeriodEndIndex + +`proposalsByVotingPeriodEndIndex` allows to retrieve proposals sorted by chronological `voting_period_end`: +`0x33 | sdk.FormatTimeBytes(proposal.VotingPeriodEnd) | BigEndian(ProposalId) -> []byte()`. + +This index is used when tallying the proposal votes at the end of the voting period, and for pruning proposals at `VotingPeriodEnd + MaxExecutionPeriod`. + +### Vote Table + +The `voteTable` stores `Vote`s: `0x40 | BigEndian(ProposalId) | []byte(voter.Address) -> ProtocolBuffer(Vote)`. + +The `voteTable` is a primary key table and its `PrimaryKey` is given by +`BigEndian(ProposalId) | []byte(voter.Address)` which is used by the following indexes. + +#### voteByProposalIndex + +`voteByProposalIndex` allows to retrieve votes by proposal id: +`0x41 | BigEndian(ProposalId) | PrimaryKey -> []byte()`. + +#### voteByVoterIndex + +`voteByVoterIndex` allows to retrieve votes by voter address: +`0x42 | len([]byte(voter.Address)) | []byte(voter.Address) | PrimaryKey -> []byte()`. + +## Msg Service + +### Msg/CreateGroup + +A new group can be created with the `MsgCreateGroup`, which has an admin address, a list of members and some optional metadata. + +The metadata has a maximum length that is chosen by the app developer, and +passed into the group keeper as a config. + +```go +// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/proto/cosmos/group/v1/tx.proto#L67-L80 +``` + +It's expected to fail if + +* metadata length is greater than `MaxMetadataLen` config +* members are not correctly set (e.g. wrong address format, duplicates, or with 0 weight). + +### Msg/UpdateGroupMembers + +Group members can be updated with the `UpdateGroupMembers`. + +```go +// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/proto/cosmos/group/v1/tx.proto#L88-L102 +``` + +In the list of `MemberUpdates`, an existing member can be removed by setting its weight to 0. + +It's expected to fail if: + +* the signer is not the admin of the group. +* for any one of the associated group policies, if its decision policy's `Validate()` method fails against the updated group. + +### Msg/UpdateGroupAdmin + +The `UpdateGroupAdmin` can be used to update a group admin. + +```go +// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/proto/cosmos/group/v1/tx.proto#L107-L120 +``` + +It's expected to fail if the signer is not the admin of the group. + +### Msg/UpdateGroupMetadata + +The `UpdateGroupMetadata` can be used to update a group metadata. + +```go +// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/proto/cosmos/group/v1/tx.proto#L125-L138 +``` + +It's expected to fail if: + +* new metadata length is greater than `MaxMetadataLen` config. +* the signer is not the admin of the group. + +### Msg/CreateGroupPolicy + +A new group policy can be created with the `MsgCreateGroupPolicy`, which has an admin address, a group id, a decision policy and some optional metadata. + +```go +// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/proto/cosmos/group/v1/tx.proto#L147-L165 +``` + +It's expected to fail if: + +* the signer is not the admin of the group. +* metadata length is greater than `MaxMetadataLen` config. +* the decision policy's `Validate()` method doesn't pass against the group. + +### Msg/CreateGroupWithPolicy + +A new group with policy can be created with the `MsgCreateGroupWithPolicy`, which has an admin address, a list of members, a decision policy, a `group_policy_as_admin` field to optionally set group and group policy admin with group policy address and some optional metadata for group and group policy. + +```go +// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/proto/cosmos/group/v1/tx.proto#L191-L215 +``` + +It's expected to fail for the same reasons as `Msg/CreateGroup` and `Msg/CreateGroupPolicy`. + +### Msg/UpdateGroupPolicyAdmin + +The `UpdateGroupPolicyAdmin` can be used to update a group policy admin. + +```go +// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/proto/cosmos/group/v1/tx.proto#L173-L186 +``` + +It's expected to fail if the signer is not the admin of the group policy. + +### Msg/UpdateGroupPolicyDecisionPolicy + +The `UpdateGroupPolicyDecisionPolicy` can be used to update a decision policy. + +```go +// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/proto/cosmos/group/v1/tx.proto#L226-L241 +``` + +It's expected to fail if: + +* the signer is not the admin of the group policy. +* the new decision policy's `Validate()` method doesn't pass against the group. + +### Msg/UpdateGroupPolicyMetadata + +The `UpdateGroupPolicyMetadata` can be used to update a group policy metadata. + +```go +// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/proto/cosmos/group/v1/tx.proto#L246-L259 +``` + +It's expected to fail if: + +* new metadata length is greater than `MaxMetadataLen` config. +* the signer is not the admin of the group. + +### Msg/SubmitProposal + +A new proposal can be created with the `MsgSubmitProposal`, which has a group policy account address, a list of proposers addresses, a list of messages to execute if the proposal is accepted and some optional metadata. +An optional `Exec` value can be provided to try to execute the proposal immediately after proposal creation. Proposers signatures are considered as yes votes in this case. + +```go +// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/proto/cosmos/group/v1/tx.proto#L281-L315 +``` + +It's expected to fail if: + +* metadata, title, or summary length is greater than `MaxMetadataLen` config. +* if any of the proposers is not a group member. + +### Msg/WithdrawProposal + +A proposal can be withdrawn using `MsgWithdrawProposal` which has an `address` (can be either a proposer or the group policy admin) and a `proposal_id` (which has to be withdrawn). + +```go +// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/proto/cosmos/group/v1/tx.proto#L323-L333 +``` + +It's expected to fail if: + +* the signer is neither the group policy admin nor proposer of the proposal. +* the proposal is already closed or aborted. + +### Msg/Vote + +A new vote can be created with the `MsgVote`, given a proposal id, a voter address, a choice (yes, no, veto or abstain) and some optional metadata. +An optional `Exec` value can be provided to try to execute the proposal immediately after voting. + +```go +// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/proto/cosmos/group/v1/tx.proto#L338-L358 +``` + +It's expected to fail if: + +* metadata length is greater than `MaxMetadataLen` config. +* the proposal is not in voting period anymore. + +### Msg/Exec + +A proposal can be executed with the `MsgExec`. + +```go +// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/proto/cosmos/group/v1/tx.proto#L363-L373 +``` + +The messages that are part of this proposal won't be executed if: + +* the proposal has not been accepted by the group policy. +* the proposal has already been successfully executed. + +### Msg/LeaveGroup + +The `MsgLeaveGroup` allows group member to leave a group. + +```go +// Reference: https://github.com/cosmos/cosmos-sdk/tree/release/v0.50.x/proto/cosmos/group/v1/tx.proto#L381-L391 +``` + +It's expected to fail if: + +* the group member is not part of the group. +* for any one of the associated group policies, if its decision policy's `Validate()` method fails against the updated group. + +## Events + +The group module emits the following events: + +### EventCreateGroup + +| Type | Attribute Key | Attribute Value | +| -------------------------------- | ------------- | -------------------------------- | +| message | action | /cosmos.group.v1.Msg/CreateGroup | +| cosmos.group.v1.EventCreateGroup | group\_id | `{groupId}` | + +### EventUpdateGroup + +| Type | Attribute Key | Attribute Value | +| -------------------------------- | ------------- | ---------------------------------------------------------- | +| message | action | `/cosmos.group.v1.Msg/UpdateGroup{Admin\|Metadata\|Members}` | +| cosmos.group.v1.EventUpdateGroup | group\_id | `{groupId}` | + +### EventCreateGroupPolicy + +| Type | Attribute Key | Attribute Value | +| -------------------------------------- | ------------- | -------------------------------------- | +| message | action | /cosmos.group.v1.Msg/CreateGroupPolicy | +| cosmos.group.v1.EventCreateGroupPolicy | address | `{groupPolicyAddress}` | + +### EventUpdateGroupPolicy + +| Type | Attribute Key | Attribute Value | +| -------------------------------------- | ------------- | ----------------------------------------------------------------------- | +| message | action | `/cosmos.group.v1.Msg/UpdateGroupPolicy{Admin\|Metadata\|DecisionPolicy}` | +| cosmos.group.v1.EventUpdateGroupPolicy | address | `{groupPolicyAddress}` | + +### EventCreateProposal + +| Type | Attribute Key | Attribute Value | +| ----------------------------------- | ------------- | ----------------------------------- | +| message | action | /cosmos.group.v1.Msg/CreateProposal | +| cosmos.group.v1.EventCreateProposal | proposal\_id | `{proposalId}` | + +### EventWithdrawProposal + +| Type | Attribute Key | Attribute Value | +| ------------------------------------- | ------------- | ------------------------------------- | +| message | action | /cosmos.group.v1.Msg/WithdrawProposal | +| cosmos.group.v1.EventWithdrawProposal | proposal\_id | `{proposalId}` | + +### EventVote + +| Type | Attribute Key | Attribute Value | +| ------------------------- | ------------- | ------------------------- | +| message | action | /cosmos.group.v1.Msg/Vote | +| cosmos.group.v1.EventVote | proposal\_id | `{proposalId}` | + +## EventExec + +| Type | Attribute Key | Attribute Value | +| ------------------------- | ------------- | ------------------------- | +| message | action | /cosmos.group.v1.Msg/Exec | +| cosmos.group.v1.EventExec | proposal\_id | `{proposalId}` | +| cosmos.group.v1.EventExec | logs | `{logs\_string}` | + +### EventLeaveGroup + +| Type | Attribute Key | Attribute Value | +| ------------------------------- | ------------- | ------------------------------- | +| message | action | /cosmos.group.v1.Msg/LeaveGroup | +| cosmos.group.v1.EventLeaveGroup | proposal\_id | `{proposalId}` | +| cosmos.group.v1.EventLeaveGroup | address | `{address}` | + +### EventProposalPruned + +| Type | Attribute Key | Attribute Value | +| ----------------------------------- | ------------- | ------------------------------- | +| message | action | /cosmos.group.v1.Msg/LeaveGroup | +| cosmos.group.v1.EventProposalPruned | proposal\_id | `{proposalId}` | +| cosmos.group.v1.EventProposalPruned | status | `{ProposalStatus}` | +| cosmos.group.v1.EventProposalPruned | tally\_result | `{TallyResult}` | + +## Client + +### CLI + +A user can query and interact with the `group` module using the CLI. + +#### Query + +The `query` commands allow users to query `group` state. + +```bash +simd query group --help +``` + +##### group-info + +The `group-info` command allows users to query for group info by given group id. + +```bash +simd query group group-info [id] [flags] +``` + +Example: + +```bash +simd query group group-info 1 +``` + +Example Output: + +```bash +admin: cosmos1.. +group_id: "1" +metadata: AQ== +total_weight: "3" +version: "1" +``` + +##### group-policy-info + +The `group-policy-info` command allows users to query for group policy info by account address of group policy . + +```bash +simd query group group-policy-info [group-policy-account] [flags] +``` + +Example: + +```bash +simd query group group-policy-info cosmos1.. +``` + +Example Output: + +```bash expandable +address: cosmos1.. +admin: cosmos1.. +decision_policy: + '@type': /cosmos.group.v1.ThresholdDecisionPolicy + threshold: "1" + windows: + min_execution_period: 0s + voting_period: 432000s +group_id: "1" +metadata: AQ== +version: "1" +``` + +##### group-members + +The `group-members` command allows users to query for group members by group id with pagination flags. + +```bash +simd query group group-members [id] [flags] +``` + +Example: + +```bash +simd query group group-members 1 +``` + +Example Output: + +```bash expandable +members: +- group_id: "1" + member: + address: cosmos1.. + metadata: AQ== + weight: "2" +- group_id: "1" + member: + address: cosmos1.. + metadata: AQ== + weight: "1" +pagination: + next_key: null + total: "2" +``` + +##### groups-by-admin + +The `groups-by-admin` command allows users to query for groups by admin account address with pagination flags. + +```bash +simd query group groups-by-admin [admin] [flags] +``` + +Example: + +```bash +simd query group groups-by-admin cosmos1.. +``` + +Example Output: + +```bash expandable +groups: +- admin: cosmos1.. + group_id: "1" + metadata: AQ== + total_weight: "3" + version: "1" +- admin: cosmos1.. + group_id: "2" + metadata: AQ== + total_weight: "3" + version: "1" +pagination: + next_key: null + total: "2" +``` + +##### group-policies-by-group + +The `group-policies-by-group` command allows users to query for group policies by group id with pagination flags. + +```bash +simd query group group-policies-by-group [group-id] [flags] +``` + +Example: + +```bash +simd query group group-policies-by-group 1 +``` + +Example Output: + +```bash expandable +group_policies: +- address: cosmos1.. + admin: cosmos1.. + decision_policy: + '@type': /cosmos.group.v1.ThresholdDecisionPolicy + threshold: "1" + windows: + min_execution_period: 0s + voting_period: 432000s + group_id: "1" + metadata: AQ== + version: "1" +- address: cosmos1.. + admin: cosmos1.. + decision_policy: + '@type': /cosmos.group.v1.ThresholdDecisionPolicy + threshold: "1" + windows: + min_execution_period: 0s + voting_period: 432000s + group_id: "1" + metadata: AQ== + version: "1" +pagination: + next_key: null + total: "2" +``` + +##### group-policies-by-admin + +The `group-policies-by-admin` command allows users to query for group policies by admin account address with pagination flags. + +```bash +simd query group group-policies-by-admin [admin] [flags] +``` + +Example: + +```bash +simd query group group-policies-by-admin cosmos1.. +``` + +Example Output: + +```bash expandable +group_policies: +- address: cosmos1.. + admin: cosmos1.. + decision_policy: + '@type': /cosmos.group.v1.ThresholdDecisionPolicy + threshold: "1" + windows: + min_execution_period: 0s + voting_period: 432000s + group_id: "1" + metadata: AQ== + version: "1" +- address: cosmos1.. + admin: cosmos1.. + decision_policy: + '@type': /cosmos.group.v1.ThresholdDecisionPolicy + threshold: "1" + windows: + min_execution_period: 0s + voting_period: 432000s + group_id: "1" + metadata: AQ== + version: "1" +pagination: + next_key: null + total: "2" +``` + +##### proposal + +The `proposal` command allows users to query for proposal by id. + +```bash +simd query group proposal [id] [flags] +``` + +Example: + +```bash +simd query group proposal 1 +``` + +Example Output: + +```bash expandable +proposal: + address: cosmos1.. + executor_result: EXECUTOR_RESULT_NOT_RUN + group_policy_version: "1" + group_version: "1" + metadata: AQ== + msgs: + - '@type': /cosmos.bank.v1beta1.MsgSend + amount: + - amount: "100000000" + denom: stake + from_address: cosmos1.. + to_address: cosmos1.. + proposal_id: "1" + proposers: + - cosmos1.. + result: RESULT_UNFINALIZED + status: STATUS_SUBMITTED + submitted_at: "2021-12-17T07:06:26.310638964Z" + windows: + min_execution_period: 0s + voting_period: 432000s + vote_state: + abstain_count: "0" + no_count: "0" + veto_count: "0" + yes_count: "0" + summary: "Summary" + title: "Title" +``` + +##### proposals-by-group-policy + +The `proposals-by-group-policy` command allows users to query for proposals by account address of group policy with pagination flags. + +```bash +simd query group proposals-by-group-policy [group-policy-account] [flags] +``` + +Example: + +```bash +simd query group proposals-by-group-policy cosmos1.. +``` + +Example Output: + +```bash expandable +pagination: + next_key: null + total: "1" +proposals: +- address: cosmos1.. + executor_result: EXECUTOR_RESULT_NOT_RUN + group_policy_version: "1" + group_version: "1" + metadata: AQ== + msgs: + - '@type': /cosmos.bank.v1beta1.MsgSend + amount: + - amount: "100000000" + denom: stake + from_address: cosmos1.. + to_address: cosmos1.. + proposal_id: "1" + proposers: + - cosmos1.. + result: RESULT_UNFINALIZED + status: STATUS_SUBMITTED + submitted_at: "2021-12-17T07:06:26.310638964Z" + windows: + min_execution_period: 0s + voting_period: 432000s + vote_state: + abstain_count: "0" + no_count: "0" + veto_count: "0" + yes_count: "0" + summary: "Summary" + title: "Title" +``` + +##### vote + +The `vote` command allows users to query for vote by proposal id and voter account address. + +```bash +simd query group vote [proposal-id] [voter] [flags] +``` + +Example: + +```bash +simd query group vote 1 cosmos1.. +``` + +Example Output: + +```bash +vote: + choice: CHOICE_YES + metadata: AQ== + proposal_id: "1" + submitted_at: "2021-12-17T08:05:02.490164009Z" + voter: cosmos1.. +``` + +##### votes-by-proposal + +The `votes-by-proposal` command allows users to query for votes by proposal id with pagination flags. + +```bash +simd query group votes-by-proposal [proposal-id] [flags] +``` + +Example: + +```bash +simd query group votes-by-proposal 1 +``` + +Example Output: + +```bash +pagination: + next_key: null + total: "1" +votes: +- choice: CHOICE_YES + metadata: AQ== + proposal_id: "1" + submitted_at: "2021-12-17T08:05:02.490164009Z" + voter: cosmos1.. +``` + +##### votes-by-voter + +The `votes-by-voter` command allows users to query for votes by voter account address with pagination flags. + +```bash +simd query group votes-by-voter [voter] [flags] +``` + +Example: + +```bash +simd query group votes-by-voter cosmos1.. +``` + +Example Output: + +```bash +pagination: + next_key: null + total: "1" +votes: +- choice: CHOICE_YES + metadata: AQ== + proposal_id: "1" + submitted_at: "2021-12-17T08:05:02.490164009Z" + voter: cosmos1.. +``` + +### Transactions + +The `tx` commands allow users to interact with the `group` module. + +```bash +simd tx group --help +``` + +#### create-group + +The `create-group` command allows users to create a group which is an aggregation of member accounts with associated weights and +an administrator account. + +```bash +simd tx group create-group [admin] [metadata] [members-json-file] +``` + +Example: + +```bash +simd tx group create-group cosmos1.. "AQ==" members.json +``` + +#### update-group-admin + +The `update-group-admin` command allows users to update a group's admin. + +```bash +simd tx group update-group-admin [admin] [group-id] [new-admin] [flags] +``` + +Example: + +```bash +simd tx group update-group-admin cosmos1.. 1 cosmos1.. +``` + +#### update-group-members + +The `update-group-members` command allows users to update a group's members. + +```bash +simd tx group update-group-members [admin] [group-id] [members-json-file] [flags] +``` + +Example: + +```bash +simd tx group update-group-members cosmos1.. 1 members.json +``` + +#### update-group-metadata + +The `update-group-metadata` command allows users to update a group's metadata. + +```bash +simd tx group update-group-metadata [admin] [group-id] [metadata] [flags] +``` + +Example: + +```bash +simd tx group update-group-metadata cosmos1.. 1 "AQ==" +``` + +#### create-group-policy + +The `create-group-policy` command allows users to create a group policy which is an account associated with a group and a decision policy. + +```bash +simd tx group create-group-policy [admin] [group-id] [metadata] [decision-policy] [flags] +``` + +Example: + +```bash +simd tx group create-group-policy cosmos1.. 1 "AQ==" '{"@type":"/cosmos.group.v1.ThresholdDecisionPolicy", "threshold":"1", "windows": {"voting_period": "120h", "min_execution_period": "0s"}}' +``` + +#### create-group-with-policy + +The `create-group-with-policy` command allows users to create a group which is an aggregation of member accounts with associated weights and an administrator account with decision policy. If the `--group-policy-as-admin` flag is set to `true`, the group policy address becomes the group and group policy admin. + +```bash +simd tx group create-group-with-policy [admin] [group-metadata] [group-policy-metadata] [members-json-file] [decision-policy] [flags] +``` + +Example: + +```bash +simd tx group create-group-with-policy cosmos1.. "AQ==" "AQ==" members.json '{"@type":"/cosmos.group.v1.ThresholdDecisionPolicy", "threshold":"1", "windows": {"voting_period": "120h", "min_execution_period": "0s"}}' +``` + +#### update-group-policy-admin + +The `update-group-policy-admin` command allows users to update a group policy admin. + +```bash +simd tx group update-group-policy-admin [admin] [group-policy-account] [new-admin] [flags] +``` + +Example: + +```bash +simd tx group update-group-policy-admin cosmos1.. cosmos1.. cosmos1.. +``` + +#### update-group-policy-metadata + +The `update-group-policy-metadata` command allows users to update a group policy metadata. + +```bash +simd tx group update-group-policy-metadata [admin] [group-policy-account] [new-metadata] [flags] +``` + +Example: + +```bash +simd tx group update-group-policy-metadata cosmos1.. cosmos1.. "AQ==" +``` + +#### update-group-policy-decision-policy + +The `update-group-policy-decision-policy` command allows users to update a group policy's decision policy. + +```bash +simd tx group update-group-policy-decision-policy [admin] [group-policy-account] [decision-policy] [flags] +``` + +Example: + +```bash +simd tx group update-group-policy-decision-policy cosmos1.. cosmos1.. '{"@type":"/cosmos.group.v1.ThresholdDecisionPolicy", "threshold":"2", "windows": {"voting_period": "120h", "min_execution_period": "0s"}}' +``` + +#### submit-proposal + +The `submit-proposal` command allows users to submit a new proposal. + +```bash +simd tx group submit-proposal [group-policy-account] [proposer[,proposer]*] [msg_tx_json_file] [metadata] [flags] +``` + +Example: + +```bash +simd tx group submit-proposal cosmos1.. cosmos1.. msg_tx.json "AQ==" +``` + +#### withdraw-proposal + +The `withdraw-proposal` command allows users to withdraw a proposal. + +```bash +simd tx group withdraw-proposal [proposal-id] [group-policy-admin-or-proposer] +``` + +Example: + +```bash +simd tx group withdraw-proposal 1 cosmos1.. +``` + +#### vote + +The `vote` command allows users to vote on a proposal. + +```bash +simd tx group vote proposal-id] [voter] [choice] [metadata] [flags] +``` + +Example: + +```bash +simd tx group vote 1 cosmos1.. CHOICE_YES "AQ==" +``` + +#### exec + +The `exec` command allows users to execute a proposal. + +```bash +simd tx group exec [proposal-id] [flags] +``` + +Example: + +```bash +simd tx group exec 1 +``` + +#### leave-group + +The `leave-group` command allows group member to leave the group. + +```bash +simd tx group leave-group [member-address] [group-id] +``` + +Example: + +```bash +simd tx group leave-group cosmos1... 1 +``` + +### gRPC + +A user can query the `group` module using gRPC endpoints. + +#### GroupInfo + +The `GroupInfo` endpoint allows users to query for group info by given group id. + +```bash +cosmos.group.v1.Query/GroupInfo +``` + +Example: + +```bash +grpcurl -plaintext \ + -d '{"group_id":1}' localhost:9090 cosmos.group.v1.Query/GroupInfo +``` + +Example Output: + +```bash +{ + "info": { + "groupId": "1", + "admin": "cosmos1..", + "metadata": "AQ==", + "version": "1", + "totalWeight": "3" + } +} +``` + +#### GroupPolicyInfo + +The `GroupPolicyInfo` endpoint allows users to query for group policy info by account address of group policy. + +```bash +cosmos.group.v1.Query/GroupPolicyInfo +``` + +Example: + +```bash +grpcurl -plaintext \ + -d '{"address":"cosmos1.."}' localhost:9090 cosmos.group.v1.Query/GroupPolicyInfo +``` + +Example Output: + +```bash +{ + "info": { + "address": "cosmos1..", + "groupId": "1", + "admin": "cosmos1..", + "version": "1", + "decisionPolicy": {"@type":"/cosmos.group.v1.ThresholdDecisionPolicy","threshold":"1","windows": {"voting_period": "120h", "min_execution_period": "0s"}}, + } +} +``` + +#### GroupMembers + +The `GroupMembers` endpoint allows users to query for group members by group id with pagination flags. + +```bash +cosmos.group.v1.Query/GroupMembers +``` + +Example: + +```bash +grpcurl -plaintext \ + -d '{"group_id":"1"}' localhost:9090 cosmos.group.v1.Query/GroupMembers +``` + +Example Output: + +```bash expandable +{ + "members": [ + { + "groupId": "1", + "member": { + "address": "cosmos1..", + "weight": "1" + } + }, + { + "groupId": "1", + "member": { + "address": "cosmos1..", + "weight": "2" + } + } + ], + "pagination": { + "total": "2" + } +} +``` + +#### GroupsByAdmin + +The `GroupsByAdmin` endpoint allows users to query for groups by admin account address with pagination flags. + +```bash +cosmos.group.v1.Query/GroupsByAdmin +``` + +Example: + +```bash +grpcurl -plaintext \ + -d '{"admin":"cosmos1.."}' localhost:9090 cosmos.group.v1.Query/GroupsByAdmin +``` + +Example Output: + +```bash expandable +{ + "groups": [ + { + "groupId": "1", + "admin": "cosmos1..", + "metadata": "AQ==", + "version": "1", + "totalWeight": "3" + }, + { + "groupId": "2", + "admin": "cosmos1..", + "metadata": "AQ==", + "version": "1", + "totalWeight": "3" + } + ], + "pagination": { + "total": "2" + } +} +``` + +#### GroupPoliciesByGroup + +The `GroupPoliciesByGroup` endpoint allows users to query for group policies by group id with pagination flags. + +```bash +cosmos.group.v1.Query/GroupPoliciesByGroup +``` + +Example: + +```bash +grpcurl -plaintext \ + -d '{"group_id":"1"}' localhost:9090 cosmos.group.v1.Query/GroupPoliciesByGroup +``` + +Example Output: + +```bash expandable +{ + "GroupPolicies": [ + { + "address": "cosmos1..", + "groupId": "1", + "admin": "cosmos1..", + "version": "1", + "decisionPolicy": {"@type":"/cosmos.group.v1.ThresholdDecisionPolicy","threshold":"1","windows":{"voting_period": "120h", "min_execution_period": "0s"}}, + }, + { + "address": "cosmos1..", + "groupId": "1", + "admin": "cosmos1..", + "version": "1", + "decisionPolicy": {"@type":"/cosmos.group.v1.ThresholdDecisionPolicy","threshold":"1","windows":{"voting_period": "120h", "min_execution_period": "0s"}}, + } + ], + "pagination": { + "total": "2" + } +} +``` + +#### GroupPoliciesByAdmin + +The `GroupPoliciesByAdmin` endpoint allows users to query for group policies by admin account address with pagination flags. + +```bash +cosmos.group.v1.Query/GroupPoliciesByAdmin +``` + +Example: + +```bash +grpcurl -plaintext \ + -d '{"admin":"cosmos1.."}' localhost:9090 cosmos.group.v1.Query/GroupPoliciesByAdmin +``` + +Example Output: + +```bash expandable +{ + "GroupPolicies": [ + { + "address": "cosmos1..", + "groupId": "1", + "admin": "cosmos1..", + "version": "1", + "decisionPolicy": {"@type":"/cosmos.group.v1.ThresholdDecisionPolicy","threshold":"1","windows":{"voting_period": "120h", "min_execution_period": "0s"}}, + }, + { + "address": "cosmos1..", + "groupId": "1", + "admin": "cosmos1..", + "version": "1", + "decisionPolicy": {"@type":"/cosmos.group.v1.ThresholdDecisionPolicy","threshold":"1","windows":{"voting_period": "120h", "min_execution_period": "0s"}}, + } + ], + "pagination": { + "total": "2" + } +} +``` + +#### Proposal + +The `Proposal` endpoint allows users to query for proposal by id. + +```bash +cosmos.group.v1.Query/Proposal +``` + +Example: + +```bash +grpcurl -plaintext \ + -d '{"proposal_id":"1"}' localhost:9090 cosmos.group.v1.Query/Proposal +``` + +Example Output: + +```bash expandable +{ + "proposal": { + "proposalId": "1", + "address": "cosmos1..", + "proposers": [ + "cosmos1.." + ], + "submittedAt": "2021-12-17T07:06:26.310638964Z", + "groupVersion": "1", + "GroupPolicyVersion": "1", + "status": "STATUS_SUBMITTED", + "result": "RESULT_UNFINALIZED", + "voteState": { + "yesCount": "0", + "noCount": "0", + "abstainCount": "0", + "vetoCount": "0" + }, + "windows": { + "min_execution_period": "0s", + "voting_period": "432000s" + }, + "executorResult": "EXECUTOR_RESULT_NOT_RUN", + "messages": [ + {"@type":"/cosmos.bank.v1beta1.MsgSend","amount":[{"denom":"stake","amount":"100000000"}],"fromAddress":"cosmos1..","toAddress":"cosmos1.."} + ], + "title": "Title", + "summary": "Summary", + } +} +``` + +#### ProposalsByGroupPolicy + +The `ProposalsByGroupPolicy` endpoint allows users to query for proposals by account address of group policy with pagination flags. + +```bash +cosmos.group.v1.Query/ProposalsByGroupPolicy +``` + +Example: + +```bash +grpcurl -plaintext \ + -d '{"address":"cosmos1.."}' localhost:9090 cosmos.group.v1.Query/ProposalsByGroupPolicy +``` + +Example Output: + +```bash expandable +{ + "proposals": [ + { + "proposalId": "1", + "address": "cosmos1..", + "proposers": [ + "cosmos1.." + ], + "submittedAt": "2021-12-17T08:03:27.099649352Z", + "groupVersion": "1", + "GroupPolicyVersion": "1", + "status": "STATUS_CLOSED", + "result": "RESULT_ACCEPTED", + "voteState": { + "yesCount": "1", + "noCount": "0", + "abstainCount": "0", + "vetoCount": "0" + }, + "windows": { + "min_execution_period": "0s", + "voting_period": "432000s" + }, + "executorResult": "EXECUTOR_RESULT_NOT_RUN", + "messages": [ + {"@type":"/cosmos.bank.v1beta1.MsgSend","amount":[{"denom":"stake","amount":"100000000"}],"fromAddress":"cosmos1..","toAddress":"cosmos1.."} + ], + "title": "Title", + "summary": "Summary", + } + ], + "pagination": { + "total": "1" + } +} +``` + +#### VoteByProposalVoter + +The `VoteByProposalVoter` endpoint allows users to query for vote by proposal id and voter account address. + +```bash +cosmos.group.v1.Query/VoteByProposalVoter +``` + +Example: + +```bash +grpcurl -plaintext \ + -d '{"proposal_id":"1","voter":"cosmos1.."}' localhost:9090 cosmos.group.v1.Query/VoteByProposalVoter +``` + +Example Output: + +```bash +{ + "vote": { + "proposalId": "1", + "voter": "cosmos1..", + "choice": "CHOICE_YES", + "submittedAt": "2021-12-17T08:05:02.490164009Z" + } +} +``` + +#### VotesByProposal + +The `VotesByProposal` endpoint allows users to query for votes by proposal id with pagination flags. + +```bash +cosmos.group.v1.Query/VotesByProposal +``` + +Example: + +```bash +grpcurl -plaintext \ + -d '{"proposal_id":"1"}' localhost:9090 cosmos.group.v1.Query/VotesByProposal +``` + +Example Output: + +```bash expandable +{ + "votes": [ + { + "proposalId": "1", + "voter": "cosmos1..", + "choice": "CHOICE_YES", + "submittedAt": "2021-12-17T08:05:02.490164009Z" + } + ], + "pagination": { + "total": "1" + } +} +``` + +#### VotesByVoter + +The `VotesByVoter` endpoint allows users to query for votes by voter account address with pagination flags. + +```bash +cosmos.group.v1.Query/VotesByVoter +``` + +Example: + +```bash +grpcurl -plaintext \ + -d '{"voter":"cosmos1.."}' localhost:9090 cosmos.group.v1.Query/VotesByVoter +``` + +Example Output: + +```bash expandable +{ + "votes": [ + { + "proposalId": "1", + "voter": "cosmos1..", + "choice": "CHOICE_YES", + "submittedAt": "2021-12-17T08:05:02.490164009Z" + } + ], + "pagination": { + "total": "1" + } +} +``` + +### REST + +A user can query the `group` module using REST endpoints. + +#### GroupInfo + +The `GroupInfo` endpoint allows users to query for group info by given group id. + +```bash +/cosmos/group/v1/group_info/{group_id} +``` + +Example: + +```bash +curl localhost:1317/cosmos/group/v1/group_info/1 +``` + +Example Output: + +```bash +{ + "info": { + "id": "1", + "admin": "cosmos1..", + "metadata": "AQ==", + "version": "1", + "total_weight": "3" + } +} +``` + +#### GroupPolicyInfo + +The `GroupPolicyInfo` endpoint allows users to query for group policy info by account address of group policy. + +```bash +/cosmos/group/v1/group_policy_info/{address} +``` + +Example: + +```bash +curl localhost:1317/cosmos/group/v1/group_policy_info/cosmos1.. +``` + +Example Output: + +```bash expandable +{ + "info": { + "address": "cosmos1..", + "group_id": "1", + "admin": "cosmos1..", + "metadata": "AQ==", + "version": "1", + "decision_policy": { + "@type": "/cosmos.group.v1.ThresholdDecisionPolicy", + "threshold": "1", + "windows": { + "voting_period": "120h", + "min_execution_period": "0s" + } + }, + } +} +``` + +#### GroupMembers + +The `GroupMembers` endpoint allows users to query for group members by group id with pagination flags. + +```bash +/cosmos/group/v1/group_members/{group_id} +``` + +Example: + +```bash +curl localhost:1317/cosmos/group/v1/group_members/1 +``` + +Example Output: + +```bash expandable +{ + "members": [ + { + "group_id": "1", + "member": { + "address": "cosmos1..", + "weight": "1", + "metadata": "AQ==" + } + }, + { + "group_id": "1", + "member": { + "address": "cosmos1..", + "weight": "2", + "metadata": "AQ==" + } + ], + "pagination": { + "next_key": null, + "total": "2" + } +} +``` + +#### GroupsByAdmin + +The `GroupsByAdmin` endpoint allows users to query for groups by admin account address with pagination flags. + +```bash +/cosmos/group/v1/groups_by_admin/{admin} +``` + +Example: + +```bash +curl localhost:1317/cosmos/group/v1/groups_by_admin/cosmos1.. +``` + +Example Output: + +```bash expandable +{ + "groups": [ + { + "id": "1", + "admin": "cosmos1..", + "metadata": "AQ==", + "version": "1", + "total_weight": "3" + }, + { + "id": "2", + "admin": "cosmos1..", + "metadata": "AQ==", + "version": "1", + "total_weight": "3" + } + ], + "pagination": { + "next_key": null, + "total": "2" + } +} +``` + +#### GroupPoliciesByGroup + +The `GroupPoliciesByGroup` endpoint allows users to query for group policies by group id with pagination flags. + +```bash +/cosmos/group/v1/group_policies_by_group/{group_id} +``` + +Example: + +```bash +curl localhost:1317/cosmos/group/v1/group_policies_by_group/1 +``` + +Example Output: + +```bash expandable +{ + "group_policies": [ + { + "address": "cosmos1..", + "group_id": "1", + "admin": "cosmos1..", + "metadata": "AQ==", + "version": "1", + "decision_policy": { + "@type": "/cosmos.group.v1.ThresholdDecisionPolicy", + "threshold": "1", + "windows": { + "voting_period": "120h", + "min_execution_period": "0s" + } + }, + }, + { + "address": "cosmos1..", + "group_id": "1", + "admin": "cosmos1..", + "metadata": "AQ==", + "version": "1", + "decision_policy": { + "@type": "/cosmos.group.v1.ThresholdDecisionPolicy", + "threshold": "1", + "windows": { + "voting_period": "120h", + "min_execution_period": "0s" + } + }, + } + ], + "pagination": { + "next_key": null, + "total": "2" + } +} +``` + +#### GroupPoliciesByAdmin + +The `GroupPoliciesByAdmin` endpoint allows users to query for group policies by admin account address with pagination flags. + +```bash +/cosmos/group/v1/group_policies_by_admin/{admin} +``` + +Example: + +```bash +curl localhost:1317/cosmos/group/v1/group_policies_by_admin/cosmos1.. +``` + +Example Output: + +```bash expandable +{ + "group_policies": [ + { + "address": "cosmos1..", + "group_id": "1", + "admin": "cosmos1..", + "metadata": "AQ==", + "version": "1", + "decision_policy": { + "@type": "/cosmos.group.v1.ThresholdDecisionPolicy", + "threshold": "1", + "windows": { + "voting_period": "120h", + "min_execution_period": "0s" + } + }, + }, + { + "address": "cosmos1..", + "group_id": "1", + "admin": "cosmos1..", + "metadata": "AQ==", + "version": "1", + "decision_policy": { + "@type": "/cosmos.group.v1.ThresholdDecisionPolicy", + "threshold": "1", + "windows": { + "voting_period": "120h", + "min_execution_period": "0s" + } + }, + } + ], + "pagination": { + "next_key": null, + "total": "2" + } +``` + +#### Proposal + +The `Proposal` endpoint allows users to query for proposal by id. + +```bash +/cosmos/group/v1/proposal/{proposal_id} +``` + +Example: + +```bash +curl localhost:1317/cosmos/group/v1/proposal/1 +``` + +Example Output: + +```bash expandable +{ + "proposal": { + "proposal_id": "1", + "address": "cosmos1..", + "metadata": "AQ==", + "proposers": [ + "cosmos1.." + ], + "submitted_at": "2021-12-17T07:06:26.310638964Z", + "group_version": "1", + "group_policy_version": "1", + "status": "STATUS_SUBMITTED", + "result": "RESULT_UNFINALIZED", + "vote_state": { + "yes_count": "0", + "no_count": "0", + "abstain_count": "0", + "veto_count": "0" + }, + "windows": { + "min_execution_period": "0s", + "voting_period": "432000s" + }, + "executor_result": "EXECUTOR_RESULT_NOT_RUN", + "messages": [ + { + "@type": "/cosmos.bank.v1beta1.MsgSend", + "from_address": "cosmos1..", + "to_address": "cosmos1..", + "amount": [ + { + "denom": "stake", + "amount": "100000000" + } + ] + } + ], + "title": "Title", + "summary": "Summary", + } +} +``` + +#### ProposalsByGroupPolicy + +The `ProposalsByGroupPolicy` endpoint allows users to query for proposals by account address of group policy with pagination flags. + +```bash +/cosmos/group/v1/proposals_by_group_policy/{address} +``` + +Example: + +```bash +curl localhost:1317/cosmos/group/v1/proposals_by_group_policy/cosmos1.. +``` + +Example Output: + +```bash expandable +{ + "proposals": [ + { + "id": "1", + "group_policy_address": "cosmos1..", + "metadata": "AQ==", + "proposers": [ + "cosmos1.." + ], + "submit_time": "2021-12-17T08:03:27.099649352Z", + "group_version": "1", + "group_policy_version": "1", + "status": "STATUS_CLOSED", + "result": "RESULT_ACCEPTED", + "vote_state": { + "yes_count": "1", + "no_count": "0", + "abstain_count": "0", + "veto_count": "0" + }, + "windows": { + "min_execution_period": "0s", + "voting_period": "432000s" + }, + "executor_result": "EXECUTOR_RESULT_NOT_RUN", + "messages": [ + { + "@type": "/cosmos.bank.v1beta1.MsgSend", + "from_address": "cosmos1..", + "to_address": "cosmos1..", + "amount": [ + { + "denom": "stake", + "amount": "100000000" + } + ] + } + ] + } + ], + "pagination": { + "next_key": null, + "total": "1" + } +} +``` + +#### VoteByProposalVoter + +The `VoteByProposalVoter` endpoint allows users to query for vote by proposal id and voter account address. + +```bash +/cosmos/group/v1/vote_by_proposal_voter/{proposal_id}/{voter} +``` + +Example: + +```bash +curl localhost:1317/cosmos/group/v1beta1/vote_by_proposal_voter/1/cosmos1.. +``` + +Example Output: + +```bash +{ + "vote": { + "proposal_id": "1", + "voter": "cosmos1..", + "choice": "CHOICE_YES", + "metadata": "AQ==", + "submitted_at": "2021-12-17T08:05:02.490164009Z" + } +} +``` + +#### VotesByProposal + +The `VotesByProposal` endpoint allows users to query for votes by proposal id with pagination flags. + +```bash +/cosmos/group/v1/votes_by_proposal/{proposal_id} +``` + +Example: + +```bash +curl localhost:1317/cosmos/group/v1/votes_by_proposal/1 +``` + +Example Output: + +```bash expandable +{ + "votes": [ + { + "proposal_id": "1", + "voter": "cosmos1..", + "option": "CHOICE_YES", + "metadata": "AQ==", + "submit_time": "2021-12-17T08:05:02.490164009Z" + } + ], + "pagination": { + "next_key": null, + "total": "1" + } +} +``` + +#### VotesByVoter + +The `VotesByVoter` endpoint allows users to query for votes by voter account address with pagination flags. + +```bash +/cosmos/group/v1/votes_by_voter/{voter} +``` + +Example: + +```bash +curl localhost:1317/cosmos/group/v1/votes_by_voter/cosmos1.. +``` + +Example Output: + +```bash expandable +{ + "votes": [ + { + "proposal_id": "1", + "voter": "cosmos1..", + "choice": "CHOICE_YES", + "metadata": "AQ==", + "submitted_at": "2021-12-17T08:05:02.490164009Z" + } + ], + "pagination": { + "next_key": null, + "total": "1" + } +} +``` + +## Metadata + +The group module has four locations for metadata where users can provide further context about the on-chain actions they are taking. By default all metadata fields have a 255 character length field where metadata can be stored in json format, either on-chain or off-chain depending on the amount of data required. Here we provide a recommendation for the json structure and where the data should be stored. There are two important factors in making these recommendations. First, that the group and gov modules are consistent with one another, note the number of proposals made by all groups may be quite large. Second, that client applications such as block explorers and governance interfaces have confidence in the consistency of metadata structure across chains. + +### Proposal + +Location: off-chain as json object stored on IPFS (mirrors [gov proposal](/sdk/v0.54/modules/gov/README#metadata)) + +```json +{ + "title": "", + "authors": [""], + "summary": "", + "details": "", + "proposal_forum_url": "", + "vote_option_context": "", +} +``` + + +The `authors` field is an array of strings, this is to allow for multiple authors to be listed in the metadata. +In v0.46, the `authors` field is a comma-separated string. Frontends are encouraged to support both formats for backwards compatibility. + + +### Vote + +Location: on-chain as json within 255 character limit (mirrors [gov vote](/sdk/v0.54/modules/gov/README#metadata)) + +```json +{ + "justification": "", +} +``` + +### Group + +Location: off-chain as json object stored on IPFS + +```json +{ + "name": "", + "description": "", + "group_website_url": "", + "group_forum_url": "", +} +``` + +### Decision policy + +Location: on-chain as json within 255 character limit + +```json +{ + "name": "", + "description": "", +} +``` diff --git a/sdk/v0.54/modules/mint/README.mdx b/sdk/v0.54/modules/mint/README.mdx new file mode 100644 index 000000000..dcb8f789e --- /dev/null +++ b/sdk/v0.54/modules/mint/README.mdx @@ -0,0 +1,481 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/modules/mint/README' +title: 'x/mint' +description: >- + The x/mint module handles the regular minting of new tokens in a configurable + manner. +--- + +The `x/mint` module handles the regular minting of new tokens in a configurable manner. + +## Contents + +* [State](#state) + * [Minter](#minter) + * [Params](#params) +* [Begin-Block](#begin-block) + * [NextInflationRate](#nextinflationrate) + * [NextAnnualProvisions](#nextannualprovisions) + * [BlockProvision](#blockprovision) +* [Parameters](#parameters) +* [Events](#events) + * [BeginBlocker](#beginblocker) +* [Client](#client) + * [CLI](#cli) + * [gRPC](#grpc) + * [REST](#rest) + +## Concepts + +### The Minting Mechanism + +The default minting mechanism was designed to: + +* allow for a flexible inflation rate determined by market demand targeting a particular bonded-stake ratio +* effect a balance between market liquidity and staked supply + +In order to best determine the appropriate market rate for inflation rewards, a +moving change rate is used. The moving change rate mechanism ensures that if +the % bonded is either over or under the goal %-bonded, the inflation rate will +adjust to further incentivize or disincentivize being bonded, respectively. Setting the goal +%-bonded at less than 100% encourages the network to maintain some non-staked tokens +which should help provide some liquidity. + +It can be broken down in the following way: + +* If the actual percentage of bonded tokens is below the goal %-bonded the inflation rate will + increase until a maximum value is reached +* If the goal % bonded (67% in Cosmos-Hub) is maintained, then the inflation + rate will stay constant +* If the actual percentage of bonded tokens is above the goal %-bonded the inflation rate will + decrease until a minimum value is reached + +### Custom Minters + +As of Cosmos SDK v0.53.0, developers can set a custom `MintFn` for the module for specialized token minting logic. + +The function signature that a `MintFn` must implement is as follows: + +```go +// MintFn defines the function that needs to be implemented in order to customize the minting process. +type MintFn func(ctx sdk.Context, k *Keeper) + +error +``` + +This can be passed to the `Keeper` upon creation with an additional `Option`: + +```go +app.MintKeeper = mintkeeper.NewKeeper( + appCodec, + runtime.NewKVStoreService(keys[minttypes.StoreKey]), + app.StakingKeeper, + app.AccountKeeper, + app.BankKeeper, + authtypes.FeeCollectorName, + authtypes.NewModuleAddress(govtypes.ModuleName).String(), + // mintkeeper.WithMintFn(CUSTOM_MINT_FN), // custom mintFn can be added here + ) +``` + +#### Custom Minter DI Example + +Below is a simple approach to creating a custom mint function with extra dependencies in DI configurations. +For this basic example, we will make the minter simply double the supply of `foo` coin. + +First, we will define a function that takes our required dependencies, and returns a `MintFn`. + +```go expandable +// MyCustomMintFunction is a custom mint function that doubles the supply of `foo` coin. +func MyCustomMintFunction(bank bankkeeper.BaseKeeper) + +mintkeeper.MintFn { + return func(ctx sdk.Context, k *mintkeeper.Keeper) + +error { + supply := bank.GetSupply(ctx, "foo") + err := k.MintCoins(ctx, sdk.NewCoins(supply.Add(supply))) + if err != nil { + return err +} + +return nil +} +} +``` + +Then, pass the function defined above into the `depinject.Supply` function with the required dependencies. + +```go expandable +// NewSimApp returns a reference to an initialized SimApp. +func NewSimApp( + logger log.Logger, + db dbm.DB, + traceStore io.Writer, + loadLatest bool, + appOpts servertypes.AppOptions, + baseAppOptions ...func(*baseapp.BaseApp), +) *SimApp { + var ( + app = &SimApp{ +} + +appBuilder *runtime.AppBuilder + appConfig = depinject.Configs( + AppConfig, + depinject.Supply( + appOpts, + logger, + // our custom mint function with the necessary dependency passed in. + MyCustomMintFunction(app.BankKeeper), + ), + ) + ) + // ... +} +``` + +## State + +### Minter + +The minter is a space for holding current inflation information. + +* Minter: `0x00 -> ProtocolBuffer(minter)` + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/mint/v1beta1/mint.proto#L10-L24 +``` + +### Params + +The mint module stores its params in state with the prefix of `0x01`, +it can be updated with governance or the address with authority. + +**Note:** The `MaxSupply` parameter controls the maximum supply of tokens the module can mint. A value of `0` indicates an unlimited supply. + +* Params: `mint/params -> legacy_amino(params)` + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/mint/v1beta1/mint.proto#L26-L59 +``` + +## Begin-Block + +Minting parameters are recalculated and inflation paid at the beginning of each block. + +### Inflation rate calculation + +Inflation rate is calculated using an "inflation calculation function" that's +passed to the `NewAppModule` function. If no function is passed, then the SDK's +default inflation function will be used (`NextInflationRate`). In case a custom +inflation calculation logic is needed, this can be achieved by defining and +passing a function that matches `InflationCalculationFn`'s signature. + +```go +type InflationCalculationFn func(ctx sdk.Context, minter Minter, params Params, bondedRatio math.LegacyDec) + +math.LegacyDec +``` + +#### NextInflationRate + +The target annual inflation rate is recalculated each block. +The inflation is also subject to a rate change (positive or negative) +depending on the distance from the desired ratio (67%). The maximum rate change +possible is defined to be 13% per year, however, the annual inflation is capped +as between 7% and 20%. + +```go expandable +NextInflationRate(params Params, bondedRatio math.LegacyDec) (inflation math.LegacyDec) { + inflationRateChangePerYear = (1 - bondedRatio/params.GoalBonded) * params.InflationRateChange + inflationRateChange = inflationRateChangePerYear/blocksPerYr + + // increase the new annual inflation for this next block + inflation += inflationRateChange + if inflation > params.InflationMax { + inflation = params.InflationMax +} + if inflation < params.InflationMin { + inflation = params.InflationMin +} + +return inflation +} +``` + +### NextAnnualProvisions + +Calculate the annual provisions based on current total supply and inflation +rate. This parameter is calculated once per block. + +```go +NextAnnualProvisions(params Params, totalSupply math.LegacyDec) (provisions math.LegacyDec) { + return Inflation * totalSupply +``` + +### BlockProvision + +Calculate the provisions generated for each block based on current annual provisions. The provisions are then minted by the `mint` module's `ModuleMinterAccount` and then transferred to the `auth`'s `FeeCollector` `ModuleAccount`. + +```go +BlockProvision(params Params) + +sdk.Coin { + provisionAmt = AnnualProvisions/ params.BlocksPerYear + return sdk.NewCoin(params.MintDenom, provisionAmt.Truncate()) +``` + +## Parameters + +The minting module contains the following parameters: + +| Key | Type | Example | +| ------------------- | ----------------- | ---------------------- | +| MintDenom | string | "uatom" | +| InflationRateChange | string (dec) | "0.130000000000000000" | +| InflationMax | string (dec) | "0.200000000000000000" | +| InflationMin | string (dec) | "0.070000000000000000" | +| GoalBonded | string (dec) | "0.670000000000000000" | +| BlocksPerYear | string (uint64) | "6311520" | +| MaxSupply | string (math.Int) | "0" | + +A `MaxSupply` value of `0` means no maximum supply is enforced. Minting stops automatically once the total supply reaches the configured `MaxSupply`. For legacy Amino JSON compatibility, `max_supply` is encoded even when set to `"0"`. + +## Events + +The minting module emits the following events: + +### BeginBlocker + +| Type | Attribute Key | Attribute Value | +| ---- | ------------------ | ------------------ | +| mint | bonded\_ratio | `{bondedRatio}` | +| mint | inflation | `{inflation}` | +| mint | annual\_provisions | `{annualProvisions}` | +| mint | amount | `{amount}` | + +## Client + +### CLI + +A user can query and interact with the `mint` module using the CLI. + +#### Query + +The `query` commands allows users to query `mint` state. + +```shell +simd query mint --help +``` + +##### annual-provisions + +The `annual-provisions` command allows users to query the current minting annual provisions value + +```shell +simd query mint annual-provisions [flags] +``` + +Example: + +```shell +simd query mint annual-provisions +``` + +Example Output: + +```shell +22268504368893.612100895088410693 +``` + +##### inflation + +The `inflation` command allows users to query the current minting inflation value + +```shell +simd query mint inflation [flags] +``` + +Example: + +```shell +simd query mint inflation +``` + +Example Output: + +```shell +0.199200302563256955 +``` + +##### params + +The `params` command allows users to query the current minting parameters + +```shell +simd query mint params [flags] +``` + +Example: + +```yml +blocks_per_year: "4360000" +goal_bonded: "0.670000000000000000" +inflation_max: "0.200000000000000000" +inflation_min: "0.070000000000000000" +inflation_rate_change: "0.130000000000000000" +max_supply: "0" +mint_denom: stake +``` + +### gRPC + +A user can query the `mint` module using gRPC endpoints. + +#### AnnualProvisions + +The `AnnualProvisions` endpoint allows users to query the current minting annual provisions value + +```shell +/cosmos.mint.v1beta1.Query/AnnualProvisions +``` + +Example: + +```shell +grpcurl -plaintext localhost:9090 cosmos.mint.v1beta1.Query/AnnualProvisions +``` + +Example Output: + +```json +{ + "annualProvisions": "1432452520532626265712995618" +} +``` + +#### Inflation + +The `Inflation` endpoint allows users to query the current minting inflation value + +```shell +/cosmos.mint.v1beta1.Query/Inflation +``` + +Example: + +```shell +grpcurl -plaintext localhost:9090 cosmos.mint.v1beta1.Query/Inflation +``` + +Example Output: + +```json +{ + "inflation": "130197115720711261" +} +``` + +#### Params + +The `Params` endpoint allows users to query the current minting parameters + +```shell +/cosmos.mint.v1beta1.Query/Params +``` + +Example: + +```shell +grpcurl -plaintext localhost:9090 cosmos.mint.v1beta1.Query/Params +``` + +Example Output: + +```json +{ + "params": { + "mintDenom": "stake", + "inflationRateChange": "130000000000000000", + "inflationMax": "200000000000000000", + "inflationMin": "70000000000000000", + "goalBonded": "670000000000000000", + "blocksPerYear": "6311520", + "maxSupply": "0" + } +} +``` + +### REST + +A user can query the `mint` module using REST endpoints. + +#### annual-provisions + +```shell +/cosmos/mint/v1beta1/annual_provisions +``` + +Example: + +```shell +curl "localhost:1317/cosmos/mint/v1beta1/annual_provisions" +``` + +Example Output: + +```json +{ + "annualProvisions": "1432452520532626265712995618" +} +``` + +#### inflation + +```shell +/cosmos/mint/v1beta1/inflation +``` + +Example: + +```shell +curl "localhost:1317/cosmos/mint/v1beta1/inflation" +``` + +Example Output: + +```json +{ + "inflation": "130197115720711261" +} +``` + +#### params + +```shell +/cosmos/mint/v1beta1/params +``` + +Example: + +```shell +curl "localhost:1317/cosmos/mint/v1beta1/params" +``` + +Example Output: + +```json +{ + "params": { + "mintDenom": "stake", + "inflationRateChange": "130000000000000000", + "inflationMax": "200000000000000000", + "inflationMin": "70000000000000000", + "goalBonded": "670000000000000000", + "blocksPerYear": "6311520", + "maxSupply": "0" + } +} +``` diff --git a/sdk/v0.54/modules/modules.mdx b/sdk/v0.54/modules/modules.mdx new file mode 100644 index 000000000..2d0059b23 --- /dev/null +++ b/sdk/v0.54/modules/modules.mdx @@ -0,0 +1,66 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/modules/modules' +title: Module Directory +description: >- + Here are some production-grade modules that can be used in Cosmos SDK + applications, along with their respective documentation. +--- + +Here are some production-grade modules that can be used in Cosmos SDK applications, along with their respective documentation: + +## Essential Modules + +Essential modules include functionality that *must* be included in your Cosmos SDK blockchain. +These modules provide the core behaviors that are needed for users and operators such as balance tracking, +proof-of-stake capabilities and governance. + +* [Auth](/sdk/v0.54/modules/auth/auth) - Authentication of accounts and transactions for Cosmos SDK applications. +* [Bank](/sdk/v0.54/modules/bank/README) - Token transfer functionalities. +* [Circuit](/sdk/v0.54/modules/circuit/README) - Circuit breaker module for pausing messages. +* [Consensus](/sdk/v0.54/modules/consensus/README) - Consensus module for modifying CometBFT's ABCI consensus params. +* [Distribution](/sdk/v0.54/modules/distribution/README) - Fee distribution, and staking token provision distribution. +* [Evidence](/sdk/v0.54/modules/evidence/README) - Evidence handling for double signing, misbehaviour, etc. +* [Governance](/sdk/v0.54/modules/gov/README) - On-chain proposals and voting. +* [Genutil](/sdk/v0.54/modules/genutil/README) - Genesis utilities for the Cosmos SDK. +* [Mint](/sdk/v0.54/modules/mint/README) - Creation of new units of staking token. +* [Slashing](/sdk/v0.54/modules/slashing/README) - Validator punishment mechanisms. +* [Staking](/sdk/v0.54/modules/staking/README) - Proof-of-Stake layer for public blockchains. +* [Upgrade](/sdk/v0.54/modules/upgrade/README) - Software upgrades handling and coordination. + +## Supplementary Modules + +Supplementary modules are modules that are maintained in the Cosmos SDK but are not necessary for +the core functionality of your blockchain. They can be thought of as ways to extend the +capabilities of your blockchain or further specialize it. + +* [Authz](/sdk/v0.54/modules/authz/README) - Authorization for accounts to perform actions on behalf of other accounts. +* [Epochs](/sdk/v0.54/modules/epochs/README) - Registration so SDK modules can have logic to be executed at the timed tickers. +* [Feegrant](/sdk/v0.54/modules/feegrant/README) - Grant fee allowances for executing transactions. +* [Group](/sdk/v0.54/modules/group/README) - Allows for the creation and management of on-chain multisig accounts. +* [NFT](/sdk/v0.54/modules/nft/README) - NFT module implemented based on [ADR43](/sdk/v0.54/reference/architecture/adr-043-nft-module). +* [ProtocolPool](/sdk/v0.54/modules/protocolpool/README) - Extended management of community pool functionality. + +## Deprecated Modules + +The following modules are deprecated. They will no longer be maintained and eventually will be removed +in an upcoming release of the Cosmos SDK per our [release process](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/RELEASE_PROCESS.md). + +* [Crisis](/sdk/v0.54/modules/crisis/README) - *Deprecated* halting the blockchain under certain circumstances (e.g. if an invariant is broken). +* [Params](/sdk/v0.54/modules/params/README) - *Deprecated* Globally available parameter store. + +To learn more about the process of building modules, visit the [building modules reference documentation](/sdk/v0.54/guides/module-design/module-design-considerations). + +## IBC + +The IBC module for the SDK is maintained by the IBC Go team in its [own repository](https://github.com/cosmos/ibc-go). + +Additionally, the [capability module](https://github.com/cosmos/ibc-go/tree/fdd664698d79864f1e00e147f9879e58497b5ef1/modules/capability) is from v0.50+ maintained by the IBC Go team in its [own repository](https://github.com/cosmos/ibc-go/tree/fdd664698d79864f1e00e147f9879e58497b5ef1/modules/capability). + +## CosmWasm + +The CosmWasm module enables smart contracts, learn more by going to their [documentation site](https://book.cosmwasm.com/), or visit [the repository](https://github.com/CosmWasm/cosmwasm). + +## EVM + +Read more about writing smart contracts with solidity at the official [`evm` documentation page](https://evm.cosmos.network/). \ No newline at end of file diff --git a/sdk/v0.54/modules/nft/README.mdx b/sdk/v0.54/modules/nft/README.mdx new file mode 100644 index 000000000..190b0a874 --- /dev/null +++ b/sdk/v0.54/modules/nft/README.mdx @@ -0,0 +1,94 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/modules/nft/README' +title: 'x/nft' +description: '## Abstract' +--- + + +`x/nft` has been moved to [`./contrib/x/nft`](https://github.com/cosmos/cosmos-sdk/tree/main/contrib/x/nft) and is no longer actively maintained as part of the core Cosmos SDK. It is still available for use but is not included in the SDK Bug Bounty program. It was moved because it was never widely adopted. + + +## Contents + +## Abstract + +`x/nft` is an implementation of a Cosmos SDK module, per [ADR 43](https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-043-nft-module.md), that allows you to create nft classification, create nft, transfer nft, update nft, and support various queries by integrating the module. It is fully compatible with the ERC721 specification. + +* [Concepts](#concepts) + * [Class](#class) + * [NFT](#nft) +* [State](#state) + * [Class](#class-1) + * [NFT](#nft-1) + * [NFTOfClassByOwner](#nftofclassbyowner) + * [Owner](#owner) + * [TotalSupply](#totalsupply) +* [Messages](#messages) + * [MsgSend](#msgsend) +* [Events](#events) + +## Concepts + +### Class + +`x/nft` module defines a struct `Class` to describe the common characteristics of a class of nft, under this class, you can create a variety of nft, which is equivalent to an erc721 contract for Ethereum. The design is defined in the [ADR 043](https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-043-nft-module.md). + +### NFT + +The full name of NFT is Non-Fungible Tokens. Because of the irreplaceable nature of NFT, it means that it can be used to represent unique things. The nft implemented by this module is fully compatible with Ethereum ERC721 standard. + +## State + +### Class + +Class is mainly composed of `id`, `name`, `symbol`, `description`, `uri`, `uri_hash`,`data` where `id` is the unique identifier of the class, similar to the Ethereum ERC721 contract address, the others are optional. + +* Class: `0x01 | classID | -> ProtocolBuffer(Class)` + +### NFT + +NFT is mainly composed of `class_id`, `id`, `uri`, `uri_hash` and `data`. Among them, `class_id` and `id` are two-tuples that identify the uniqueness of nft, `uri` and `uri_hash` is optional, which identifies the off-chain storage location of the nft, and `data` is an Any type. Use Any chain of `x/nft` modules can be customized by extending this field + +* NFT: `0x02 | classID | 0x00 | nftID |-> ProtocolBuffer(NFT)` + +### NFTOfClassByOwner + +NFTOfClassByOwner is mainly to realize the function of querying all nfts using classID and owner, without other redundant functions. + +* NFTOfClassByOwner: `0x03 | owner | 0x00 | classID | 0x00 | nftID |-> 0x01` + +### Owner + +Since there is no extra field in NFT to indicate the owner of nft, an additional key-value pair is used to save the ownership of nft. With the transfer of nft, the key-value pair is updated synchronously. + +* OwnerKey: `0x04 | classID | 0x00 | nftID |-> owner` + +### TotalSupply + +TotalSupply is responsible for tracking the number of all nfts under a certain class. Mint operation is performed under the changed class, supply increases by one, burn operation, and supply decreases by one. + +* OwnerKey: `0x05 | classID |-> totalSupply` + +## Messages + +In this section we describe the processing of messages for the NFT module. + + +The validation of `ClassID` and `NftID` is left to the app developer.\ +The SDK does not provide any validation for these fields. + + +### MsgSend + +You can use the `MsgSend` message to transfer the ownership of nft. This is a function provided by the `x/nft` module. Of course, you can use the `Transfer` method to implement your own transfer logic, but you need to pay extra attention to the transfer permissions. + +The message handling should fail if: + +* provided `ClassID` does not exist. +* provided `Id` does not exist. +* provided `Sender` does not the owner of nft. + +## Events + +The nft module emits proto events defined in [the Protobuf reference](https://buf.build/cosmos/cosmos-sdk/docs/main:cosmos.nft.v1beta1). diff --git a/sdk/v0.54/modules/params/README.mdx b/sdk/v0.54/modules/params/README.mdx new file mode 100644 index 000000000..8c609c897 --- /dev/null +++ b/sdk/v0.54/modules/params/README.mdx @@ -0,0 +1,84 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/modules/params/README' +title: 'x/params' +description: >- + NOTE: x/params is deprecated as of Cosmos SDK v0.53 and will be removed in the + next release. +--- + +NOTE: `x/params` is deprecated as of Cosmos SDK v0.53 and will be removed in the next release. + +## Abstract + +Package params provides a globally available parameter store. + +There are two main types, Keeper and Subspace. Subspace is an isolated namespace for a +paramstore, where keys are prefixed by preconfigured spacename. Keeper has a +permission to access all existing spaces. + +Subspace can be used by the individual keepers, which need a private parameter store +that the other keepers cannot modify. The params Keeper can be used to add a route to `x/gov` router in order to modify any parameter in case a proposal passes. + +The following contents explains how to use params module for master and user modules. + +## Contents + +* [Keeper](#keeper) +* [Subspace](#subspace) + * [Key](#key) + * [KeyTable](#keytable) + * [ParamSet](#paramset) + +## Keeper + +In the app initialization stage, [subspaces](#subspace) can be allocated for other modules' keeper using `Keeper.Subspace` and are stored in `Keeper.spaces`. Then, those modules can have a reference to their specific parameter store through `Keeper.GetSubspace`. + +Example: + +```go +type ExampleKeeper struct { + paramSpace paramtypes.Subspace +} + +func (k ExampleKeeper) + +SetParams(ctx sdk.Context, params types.Params) { + k.paramSpace.SetParamSet(ctx, ¶ms) +} +``` + +## Subspace + +`Subspace` is a prefixed subspace of the parameter store. Each module which uses the +parameter store will take a `Subspace` to isolate permission to access. + +### Key + +Parameter keys are human readable alphanumeric strings. A parameter for the key +`"ExampleParameter"` is stored under `[]byte("SubspaceName" + "/" + "ExampleParameter")`, +where `"SubspaceName"` is the name of the subspace. + +Subkeys are secondary parameter keys those are used along with a primary parameter key. +Subkeys can be used for grouping or dynamic parameter key generation during runtime. + +### KeyTable + +All of the parameter keys that will be used should be registered at the compile +time. `KeyTable` is essentially a `map[string]attribute`, where the `string` is a parameter key. + +Currently, `attribute` consists of a `reflect.Type`, which indicates the parameter +type to check that provided key and value are compatible and registered, as well as a function `ValueValidatorFn` to validate values. + +Only primary keys have to be registered on the `KeyTable`. Subkeys inherit the +attribute of the primary key. + +### ParamSet + +Modules often define parameters as a proto message. The generated struct can implement +`ParamSet` interface to be used with the following methods: + +* `KeyTable.RegisterParamSet()`: registers all parameters in the struct +* `Subspace.{Get, Set}ParamSet()`: Get to & Set from the struct + +The implementor should be a pointer in order to use `GetParamSet()`. diff --git a/sdk/next/modules/protocolpool/README.mdx b/sdk/v0.54/modules/protocolpool/README.mdx similarity index 99% rename from sdk/next/modules/protocolpool/README.mdx rename to sdk/v0.54/modules/protocolpool/README.mdx index 43f57d325..07e207f66 100644 --- a/sdk/next/modules/protocolpool/README.mdx +++ b/sdk/v0.54/modules/protocolpool/README.mdx @@ -1,6 +1,7 @@ --- -title: 'x/protocolpool' noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/modules/protocolpool/README' +title: 'x/protocolpool' --- ## Concepts diff --git a/sdk/v0.54/modules/slashing/README.mdx b/sdk/v0.54/modules/slashing/README.mdx new file mode 100644 index 000000000..d1ec2a874 --- /dev/null +++ b/sdk/v0.54/modules/slashing/README.mdx @@ -0,0 +1,816 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/modules/slashing/README' +title: 'x/slashing' +description: >- + This section specifies the slashing module of the Cosmos SDK, which implements + functionality first outlined in the Cosmos Whitepaper in June 2016. +--- + +## Abstract + +This section specifies the slashing module of the Cosmos SDK, which implements functionality +first outlined in the [Cosmos Whitepaper](https://github.com/cosmos/cosmos/blob/master/WHITEPAPER.md) in June 2016. + +The slashing module enables Cosmos SDK-based blockchains to disincentivize any attributable action +by a protocol-recognized actor with value at stake by penalizing them ("slashing"). + +Penalties may include, but are not limited to: + +* Burning some amount of their stake +* Removing their ability to vote on future blocks for a period of time. + +This module will be used by the Cosmos Hub, the first hub in the Cosmos ecosystem. + +## Contents + +* [Concepts](#concepts) + * [States](#states) + * [Tombstone Caps](#tombstone-caps) + * [Infraction Timelines](#infraction-timelines) +* [State](#state) + * [Signing Info (Liveness)](#signing-info-liveness) + * [Params](#params) +* [Messages](#messages) + * [Unjail](#unjail) +* [BeginBlock](#beginblock) + * [Liveness Tracking](#liveness-tracking) +* [Hooks](#hooks) +* [Events](#events) +* [Staking Tombstone](#staking-tombstone) +* [Parameters](#parameters) +* [CLI](#cli) + * [Query](#query) + * [Transactions](#transactions) + * [gRPC](#grpc) + * [REST](#rest) + +## Concepts + +### States + +At any given time, there are any number of validators registered in the state +machine. Each block, the top `MaxValidators` (defined by `x/staking`) validators +who are not jailed become *bonded*, meaning that they may propose and vote on +blocks. Validators who are *bonded* are *at stake*, meaning that part or all of +their stake and their delegators' stake is at risk if they commit a protocol fault. + +For each of these validators we keep a `ValidatorSigningInfo` record that contains +information pertaining to validator's liveness and other infraction related +attributes. + +### Tombstone Caps + +In order to mitigate the impact of initially likely categories of non-malicious +protocol faults, the Cosmos Hub implements for each validator +a *tombstone* cap, which only allows a validator to be slashed once for a double +sign fault. For example, if you misconfigure your HSM and double-sign a bunch of +old blocks, you'll only be punished for the first double-sign (and then immediately tombstoned). This will still be quite expensive and desirable to avoid, but tombstone caps +somewhat blunt the economic impact of unintentional misconfiguration. + +Liveness faults do not have caps, as they can't stack upon each other. Liveness bugs are "detected" as soon as the infraction occurs, and the validators are immediately put in jail, so it is not possible for them to commit multiple liveness faults without unjailing in between. + +### Infraction Timelines + +To illustrate how the `x/slashing` module handles submitted evidence through +CometBFT consensus, consider the following examples: + +**Definitions**: + +*\[* : timeline start\ +*]* : timeline end\ +*Cn* : infraction `n` committed\ +*Dn* : infraction `n` discovered\ +*Vb* : validator bonded\ +*Vu* : validator unbonded + +#### Single Double Sign Infraction + +\[----------C1----D1,Vu-----] + +A single infraction is committed then later discovered, at which point the +validator is unbonded and slashed at the full amount for the infraction. + +#### Multiple Double Sign Infractions + +\[----------C1--C2---C3---D1,D2,D3Vu-----] + +Multiple infractions are committed and then later discovered, at which point the +validator is jailed and slashed for only one infraction. Because the validator +is also tombstoned, they can not rejoin the validator set. + +## State + +### Signing Info (Liveness) + +Every block includes a set of precommits by the validators for the previous block, +known as the `LastCommitInfo` provided by CometBFT. A `LastCommitInfo` is valid so +long as it contains precommits from +2/3 of total voting power. + +Proposers are incentivized to include precommits from all validators in the CometBFT `LastCommitInfo` +by receiving additional fees proportional to the difference between the voting +power included in the `LastCommitInfo` and +2/3 (see [fee distribution](/sdk/v0.47/build/modules/distribution/README#begin-block)). + +```go +type LastCommitInfo struct { + Round int32 + Votes []VoteInfo +} +``` + +Validators are penalized for failing to be included in the `LastCommitInfo` for some +number of blocks by being automatically jailed, potentially slashed, and unbonded. + +Information about validator's liveness activity is tracked through `ValidatorSigningInfo`. +It is indexed in the store as follows: + +* ValidatorSigningInfo: `0x01 | ConsAddrLen (1 byte) | ConsAddress -> ProtocolBuffer(ValSigningInfo)` +* MissedBlocksBitArray: `0x02 | ConsAddrLen (1 byte) | ConsAddress | LittleEndianUint64(signArrayIndex) -> VarInt(didMiss)` (varint is a number encoding format) + +The first mapping allows us to easily lookup the recent signing info for a +validator based on the validator's consensus address. + +The second mapping (`MissedBlocksBitArray`) acts +as a bit-array of size `SignedBlocksWindow` that tells us if the validator missed +the block for a given index in the bit-array. The index in the bit-array is given +as little endian uint64. +The result is a `varint` that takes on `0` or `1`, where `0` indicates the +validator did not miss (did sign) the corresponding block, and `1` indicates +they missed the block (did not sign). + +Note that the `MissedBlocksBitArray` is not explicitly initialized up-front. Keys +are added as we progress through the first `SignedBlocksWindow` blocks for a newly +bonded validator. The `SignedBlocksWindow` parameter defines the size +(number of blocks) of the sliding window used to track validator liveness. + +The information stored for tracking validator liveness is as follows: + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/slashing/v1beta1/slashing.proto#L13-L35 +``` + +### Params + +The slashing module stores it's params in state with the prefix of `0x00`, +it can be updated with governance or the address with authority. + +* Params: `0x00 | ProtocolBuffer(Params)` + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/slashing/v1beta1/slashing.proto#L37-L59 +``` + +## Messages + +In this section we describe the processing of messages for the `slashing` module. + +### Unjail + +If a validator was automatically unbonded due to downtime and wishes to come back online & +possibly rejoin the bonded set, it must send `MsgUnjail`: + +```protobuf +// MsgUnjail is an sdk.Msg used for unjailing a jailed validator, thus returning +// them into the bonded validator set, so they can begin receiving provisions +// and rewards again. +message MsgUnjail { + string validator_addr = 1; +} +``` + +Below is a pseudocode of the `MsgSrv/Unjail` RPC: + +```go expandable +unjail(tx MsgUnjail) + +validator = getValidator(tx.ValidatorAddr) + if validator == nil + fail with "No validator found" + if getSelfDelegation(validator) == 0 + fail with "validator must self delegate before unjailing" + if !validator.Jailed + fail with "Validator not jailed, cannot unjail" + + info = GetValidatorSigningInfo(operator) + if info.Tombstoned + fail with "Tombstoned validator cannot be unjailed" + if block time < info.JailedUntil + fail with "Validator still jailed, cannot unjail until period has expired" + + validator.Jailed = false + setValidator(validator) + +return +``` + +If the validator has enough stake to be in the top `n = MaximumBondedValidators`, it will be automatically rebonded, +and all delegators still delegated to the validator will be rebonded and begin to again collect +provisions and rewards. + +## BeginBlock + +### Liveness Tracking + +At the beginning of each block, we update the `ValidatorSigningInfo` for each +validator and check if they've crossed below the liveness threshold over a +sliding window. This sliding window is defined by `SignedBlocksWindow` and the +index in this window is determined by `IndexOffset` found in the validator's +`ValidatorSigningInfo`. For each block processed, the `IndexOffset` is incremented +regardless if the validator signed or not. Once the index is determined, the +`MissedBlocksBitArray` and `MissedBlocksCounter` are updated accordingly. + +Finally, in order to determine if a validator crosses below the liveness threshold, +we fetch the maximum number of blocks missed, `maxMissed`, which is +`SignedBlocksWindow - (MinSignedPerWindow * SignedBlocksWindow)` and the minimum +height at which we can determine liveness, `minHeight`. If the current block is +greater than `minHeight` and the validator's `MissedBlocksCounter` is greater than +`maxMissed`, they will be slashed by `SlashFractionDowntime`, will be jailed +for `DowntimeJailDuration`, and have the following values reset: +`MissedBlocksBitArray`, `MissedBlocksCounter`, and `IndexOffset`. + +**Note**: Liveness slashes do **NOT** lead to a tombstoning. + +```go expandable +height := block.Height + for vote in block.LastCommitInfo.Votes { + signInfo := GetValidatorSigningInfo(vote.Validator.Address) + + // This is a relative index, so we counts blocks the validator SHOULD have + // signed. We use the 0-value default signing info if not present, except for + // start height. + index := signInfo.IndexOffset % SignedBlocksWindow() + +signInfo.IndexOffset++ + + // Update MissedBlocksBitArray and MissedBlocksCounter. The MissedBlocksCounter + // just tracks the sum of MissedBlocksBitArray. That way we avoid needing to + // read/write the whole array each time. + missedPrevious := GetValidatorMissedBlockBitArray(vote.Validator.Address, index) + missed := !signed + switch { + case !missedPrevious && missed: + // array index has changed from not missed to missed, increment counter + SetValidatorMissedBlockBitArray(vote.Validator.Address, index, true) + +signInfo.MissedBlocksCounter++ + case missedPrevious && !missed: + // array index has changed from missed to not missed, decrement counter + SetValidatorMissedBlockBitArray(vote.Validator.Address, index, false) + +signInfo.MissedBlocksCounter-- + + default: + // array index at this index has not changed; no need to update counter +} + if missed { + // emit events... +} + minHeight := signInfo.StartHeight + SignedBlocksWindow() + maxMissed := SignedBlocksWindow() - MinSignedPerWindow() + + // If we are past the minimum height and the validator has missed too many + // jail and slash them. + if height > minHeight && signInfo.MissedBlocksCounter > maxMissed { + validator := ValidatorByConsAddr(vote.Validator.Address) + + // emit events... + + // We need to retrieve the stake distribution which signed the block, so we + // subtract ValidatorUpdateDelay from the block height, and subtract an + // additional 1 since this is the LastCommit. + // + // Note, that this CAN result in a negative "distributionHeight" up to + // -ValidatorUpdateDelay-1, i.e. at the end of the pre-genesis block (none) = at the beginning of the genesis block. + // That's fine since this is just used to filter unbonding delegations & redelegations. + distributionHeight := height - sdk.ValidatorUpdateDelay - 1 + + SlashWithInfractionReason(vote.Validator.Address, distributionHeight, vote.Validator.Power, SlashFractionDowntime(), stakingtypes.Downtime) + +Jail(vote.Validator.Address) + +signInfo.JailedUntil = block.Time.Add(DowntimeJailDuration()) + + // We need to reset the counter & array so that the validator won't be + // immediately slashed for downtime upon rebonding. + signInfo.MissedBlocksCounter = 0 + signInfo.IndexOffset = 0 + ClearValidatorMissedBlockBitArray(vote.Validator.Address) +} + +SetValidatorSigningInfo(vote.Validator.Address, signInfo) +} +``` + +## Hooks + +This section contains a description of the module's `hooks`. Hooks are operations that are executed automatically when events are raised. + +### Staking hooks + +The slashing module implements the `StakingHooks` defined in `x/staking` and are used as record-keeping of validators information. During the app initialization, these hooks should be registered in the staking module struct. + +The following hooks impact the slashing state: + +* `AfterValidatorBonded` creates a `ValidatorSigningInfo` instance as described in the following section. +* `AfterValidatorCreated` stores a validator's consensus key. +* `AfterValidatorRemoved` removes a validator's consensus key. + +### Validator Bonded + +Upon successful first-time bonding of a new validator, we create a new `ValidatorSigningInfo` structure for the +now-bonded validator, which `StartHeight` of the current block. + +If the validator was out of the validator set and gets bonded again, its new bonded height is set. + +```go expandable +onValidatorBonded(address sdk.ValAddress) + +signingInfo, found = GetValidatorSigningInfo(address) + if !found { + signingInfo = ValidatorSigningInfo { + StartHeight : CurrentHeight, + IndexOffset : 0, + JailedUntil : time.Unix(0, 0), + Tombstone : false, + MissedBlocksCounter : 0 +} + +else { + signingInfo.StartHeight = CurrentHeight +} + +setValidatorSigningInfo(signingInfo) +} + +return +``` + +## Events + +The slashing module emits the following events: + +### MsgServer + +#### MsgUnjail + +| Type | Attribute Key | Attribute Value | +| ------- | ------------- | ------------------ | +| message | module | slashing | +| message | sender | `{validatorAddress}` | + +### Keeper + +### BeginBlocker: HandleValidatorSignature + +| Type | Attribute Key | Attribute Value | +| ----- | ------------- | --------------------------- | +| slash | address | `{validatorConsensusAddress}` | +| slash | power | `{validatorPower}` | +| slash | reason | `{slashReason}` | +| slash | jailed \[0] | `{validatorConsensusAddress}` | +| slash | burned coins | `{math.Int}` | + +* \[0] Only included if the validator is jailed. + +| Type | Attribute Key | Attribute Value | +| -------- | -------------- | --------------------------- | +| liveness | address | `{validatorConsensusAddress}` | +| liveness | missed\_blocks | `{missedBlocksCounter}` | +| liveness | height | `{blockHeight}` | + +#### Slash + +* same as `"slash"` event from `HandleValidatorSignature`, but without the `jailed` attribute. + +#### Jail + +| Type | Attribute Key | Attribute Value | +| ----- | ------------- | ------------------ | +| slash | jailed | `{validatorAddress}` | + +## Staking Tombstone + +### Abstract + +In the current implementation of the `slashing` module, when the consensus engine +informs the state machine of a validator's consensus fault, the validator is +partially slashed, and put into a "jail period", a period of time in which they +are not allowed to rejoin the validator set. However, because of the nature of +consensus faults and ABCI, there can be a delay between an infraction occurring, +and evidence of the infraction reaching the state machine (this is one of the +primary reasons for the existence of the unbonding period). + +> Note: The tombstone concept, only applies to faults that have a delay between +> the infraction occurring and evidence reaching the state machine. For example, +> evidence of a validator double signing may take a while to reach the state machine +> due to unpredictable evidence gossip layer delays and the ability of validators to +> selectively reveal double-signatures (e.g. to infrequently-online light clients). +> Liveness slashing, on the other hand, is detected immediately as soon as the +> infraction occurs, and therefore no slashing period is needed. A validator is +> immediately put into jail period, and they cannot commit another liveness fault +> until they unjail. In the future, there may be other types of byzantine faults +> that have delays (for example, submitting evidence of an invalid proposal as a transaction). +> When implemented, it will have to be decided whether these future types of +> byzantine faults will result in a tombstoning (and if not, the slash amounts +> will not be capped by a slashing period). + +In the current system design, once a validator is put in the jail for a consensus +fault, after the `JailPeriod` they are allowed to send a transaction to `unjail` +themselves, and thus rejoin the validator set. + +One of the "design desires" of the `slashing` module is that if multiple +infractions occur before evidence is executed (and a validator is put in jail), +they should only be punished for single worst infraction, but not cumulatively. +For example, if the sequence of events is: + +1. Validator A commits Infraction 1 (worth 30% slash) +2. Validator A commits Infraction 2 (worth 40% slash) +3. Validator A commits Infraction 3 (worth 35% slash) +4. Evidence for Infraction 1 reaches state machine (and validator is put in jail) +5. Evidence for Infraction 2 reaches state machine +6. Evidence for Infraction 3 reaches state machine + +Only Infraction 2 should have its slash take effect, as it is the highest. This +is done, so that in the case of the compromise of a validator's consensus key, +they will only be punished once, even if the hacker double-signs many blocks. +Because, the unjailing has to be done with the validator's operator key, they +have a chance to re-secure their consensus key, and then signal that they are +ready using their operator key. We call this period during which we track only +the max infraction, the "slashing period". + +Once, a validator rejoins by unjailing themselves, we begin a new slashing period; +if they commit a new infraction after unjailing, it gets slashed cumulatively on +top of the worst infraction from the previous slashing period. + +However, while infractions are grouped based off of the slashing periods, because +evidence can be submitted up to an `unbondingPeriod` after the infraction, we +still have to allow for evidence to be submitted for previous slashing periods. +For example, if the sequence of events is: + +1. Validator A commits Infraction 1 (worth 30% slash) +2. Validator A commits Infraction 2 (worth 40% slash) +3. Evidence for Infraction 1 reaches state machine (and Validator A is put in jail) +4. Validator A unjails + +We are now in a new slashing period, however we still have to keep the door open +for the previous infraction, as the evidence for Infraction 2 may still come in. +As the number of slashing periods increase, it creates more complexity as we have +to keep track of the highest infraction amount for every single slashing period. + +> Note: Currently, according to the `slashing` module spec, a new slashing period +> is created every time a validator is unbonded then rebonded. This should probably +> be changed to jailed/unjailed. See issue [#3205](https://github.com/cosmos/cosmos-sdk/issues/3205) +> for further details. For the remainder of this, I will assume that we only start +> a new slashing period when a validator gets unjailed. + +The maximum number of slashing periods is the `len(UnbondingPeriod) / len(JailPeriod)`. +The current defaults in Gaia for the `UnbondingPeriod` and `JailPeriod` are 3 weeks +and 2 days, respectively. This means there could potentially be up to 11 slashing +periods concurrently being tracked per validator. If we set the `JailPeriod >= UnbondingPeriod`, +we only have to track 1 slashing period (i.e not have to track slashing periods). + +Currently, in the jail period implementation, once a validator unjails, all of +their delegators who are delegated to them (haven't unbonded / redelegated away), +stay with them. Given that consensus safety faults are so egregious +(way more so than liveness faults), it is probably prudent to have delegators not +"auto-rebond" to the validator. + +#### Proposal: infinite jail + +We propose setting the "jail time" for a +validator who commits a consensus safety fault, to `infinite` (i.e. a tombstone state). +This essentially kicks the validator out of the validator set and does not allow +them to re-enter the validator set. All of their delegators (including the operator themselves) +have to either unbond or redelegate away. The validator operator can create a new +validator if they would like, with a new operator key and consensus key, but they +have to "re-earn" their delegations back. + +Implementing the tombstone system and getting rid of the slashing period tracking +will make the `slashing` module way simpler, especially because we can remove all +of the hooks defined in the `slashing` module consumed by the `staking` module +(the `slashing` module still consumes hooks defined in `staking`). + +#### Single slashing amount + +Another optimization that can be made is that if we assume that all ABCI faults +for CometBFT consensus are slashed at the same level, we don't have to keep +track of "max slash". Once an ABCI fault happens, we don't have to worry about +comparing potential future ones to find the max. + +Currently the only CometBFT ABCI fault is: + +* Unjustified precommits (double signs) + +It is currently planned to include the following fault in the near future: + +* Signing a precommit when you're in unbonding phase (needed to make light client bisection safe) + +Given that these faults are both attributable byzantine faults, we will likely +want to slash them equally, and thus we can enact the above change. + +> Note: This change may make sense for current CometBFT consensus, but maybe +> not for a different consensus algorithm or future versions of CometBFT that +> may want to punish at different levels (for example, partial slashing). + +## Parameters + +The slashing module contains the following parameters: + +| Key | Type | Example | +| ----------------------- | -------------- | ---------------------- | +| SignedBlocksWindow | string (int64) | "100" | +| MinSignedPerWindow | string (dec) | "0.500000000000000000" | +| DowntimeJailDuration | string (ns) | "600000000000" | +| SlashFractionDoubleSign | string (dec) | "0.050000000000000000" | +| SlashFractionDowntime | string (dec) | "0.010000000000000000" | + +## CLI + +A user can query and interact with the `slashing` module using the CLI. + +### Query + +The `query` commands allow users to query `slashing` state. + +```shell +simd query slashing --help +``` + +#### params + +The `params` command allows users to query genesis parameters for the slashing module. + +```shell +simd query slashing params [flags] +``` + +Example: + +```shell +simd query slashing params +``` + +Example Output: + +```yml +downtime_jail_duration: 600s +min_signed_per_window: "0.500000000000000000" +signed_blocks_window: "100" +slash_fraction_double_sign: "0.050000000000000000" +slash_fraction_downtime: "0.010000000000000000" +``` + +#### signing-info + +The `signing-info` command allows users to query signing-info of the validator using consensus public key. + +```shell +simd query slashing signing-infos [flags] +``` + +Example: + +```shell +simd query slashing signing-info '{"@type":"/cosmos.crypto.ed25519.PubKey","key":"Auxs3865HpB/EfssYOzfqNhEJjzys6jD5B6tPgC8="}' + +``` + +Example Output: + +```yml +address: cosmosvalcons1nrqsld3aw6lh6t082frdqc84uwxn0t958c +index_offset: "2068" +jailed_until: "1970-01-01T00:00:00Z" +missed_blocks_counter: "0" +start_height: "0" +tombstoned: false +``` + +#### signing-infos + +The `signing-infos` command allows users to query signing infos of all validators. + +```shell +simd query slashing signing-infos [flags] +``` + +Example: + +```shell +simd query slashing signing-infos +``` + +Example Output: + +```yml +info: +- address: cosmosvalcons1nrqsld3aw6lh6t082frdqc84uwxn0t958c + index_offset: "2075" + jailed_until: "1970-01-01T00:00:00Z" + missed_blocks_counter: "0" + start_height: "0" + tombstoned: false +pagination: + next_key: null + total: "0" +``` + +### Transactions + +The `tx` commands allow users to interact with the `slashing` module. + +```bash +simd tx slashing --help +``` + +#### unjail + +The `unjail` command allows users to unjail a validator previously jailed for downtime. + +```bash +simd tx slashing unjail --from mykey [flags] +``` + +Example: + +```bash +simd tx slashing unjail --from mykey +``` + +### gRPC + +A user can query the `slashing` module using gRPC endpoints. + +#### Params + +The `Params` endpoint allows users to query the parameters of slashing module. + +```shell +cosmos.slashing.v1beta1.Query/Params +``` + +Example: + +```shell +grpcurl -plaintext localhost:9090 cosmos.slashing.v1beta1.Query/Params +``` + +Example Output: + +```json +{ + "params": { + "signedBlocksWindow": "100", + "minSignedPerWindow": "NTAwMDAwMDAwMDAwMDAwMDAw", + "downtimeJailDuration": "600s", + "slashFractionDoubleSign": "NTAwMDAwMDAwMDAwMDAwMDA=", + "slashFractionDowntime": "MTAwMDAwMDAwMDAwMDAwMDA=" + } +} +``` + +#### SigningInfo + +The SigningInfo queries the signing info of given cons address. + +```shell +cosmos.slashing.v1beta1.Query/SigningInfo +``` + +Example: + +```shell +grpcurl -plaintext -d '{"cons_address":"cosmosvalcons1nrqsld3aw6lh6t082frdqc84uwxn0t958c"}' localhost:9090 cosmos.slashing.v1beta1.Query/SigningInfo +``` + +Example Output: + +```json +{ + "valSigningInfo": { + "address": "cosmosvalcons1nrqsld3aw6lh6t082frdqc84uwxn0t958c", + "indexOffset": "3493", + "jailedUntil": "1970-01-01T00:00:00Z" + } +} +``` + +#### SigningInfos + +The SigningInfos queries signing info of all validators. + +```shell +cosmos.slashing.v1beta1.Query/SigningInfos +``` + +Example: + +```shell +grpcurl -plaintext localhost:9090 cosmos.slashing.v1beta1.Query/SigningInfos +``` + +Example Output: + +```json expandable +{ + "info": [ + { + "address": "cosmosvalcons1nrqslkwd3pz096lh6t082frdqc84uwxn0t958c", + "indexOffset": "2467", + "jailedUntil": "1970-01-01T00:00:00Z" + } + ], + "pagination": { + "total": "1" + } +} +``` + +### REST + +A user can query the `slashing` module using REST endpoints. + +#### Params + +```shell +/cosmos/slashing/v1beta1/params +``` + +Example: + +```shell +curl "localhost:1317/cosmos/slashing/v1beta1/params" +``` + +Example Output: + +```json +{ + "params": { + "signed_blocks_window": "100", + "min_signed_per_window": "0.500000000000000000", + "downtime_jail_duration": "600s", + "slash_fraction_double_sign": "0.050000000000000000", + "slash_fraction_downtime": "0.010000000000000000" +} +``` + +#### signing\_info + +```shell +/cosmos/slashing/v1beta1/signing_infos/%s +``` + +Example: + +```shell +curl "localhost:1317/cosmos/slashing/v1beta1/signing_infos/cosmosvalcons1nrqslkwd3pz096lh6t082frdqc84uwxn0t958c" +``` + +Example Output: + +```json +{ + "val_signing_info": { + "address": "cosmosvalcons1nrqslkwd3pz096lh6t082frdqc84uwxn0t958c", + "start_height": "0", + "index_offset": "4184", + "jailed_until": "1970-01-01T00:00:00Z", + "tombstoned": false, + "missed_blocks_counter": "0" + } +} +``` + +#### signing\_infos + +```shell +/cosmos/slashing/v1beta1/signing_infos +``` + +Example: + +```shell +curl "localhost:1317/cosmos/slashing/v1beta1/signing_infos +``` + +Example Output: + +```json expandable +{ + "info": [ + { + "address": "cosmosvalcons1nrqslkwd3pz096lh6t082frdqc84uwxn0t958c", + "start_height": "0", + "index_offset": "4169", + "jailed_until": "1970-01-01T00:00:00Z", + "tombstoned": false, + "missed_blocks_counter": "0" + } + ], + "pagination": { + "next_key": null, + "total": "1" + } +} +``` diff --git a/sdk/v0.54/modules/staking/README.mdx b/sdk/v0.54/modules/staking/README.mdx new file mode 100644 index 000000000..9dcbaefaa --- /dev/null +++ b/sdk/v0.54/modules/staking/README.mdx @@ -0,0 +1,3465 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/modules/staking/README' +title: 'x/staking' +description: >- + This paper specifies the Staking module of the Cosmos SDK that was first + described in the Cosmos Whitepaper in June 2016. +--- + +## Abstract + +This paper specifies the Staking module of the Cosmos SDK that was first +described in the [Cosmos Whitepaper](https://github.com/cosmos/cosmos/blob/master/WHITEPAPER.md) +in June 2016. + +The module enables Cosmos SDK-based blockchain to support an advanced +Proof-of-Stake (PoS) system. In this system, holders of the native staking token of +the chain can become validators and can delegate tokens to validators, +ultimately determining the effective validator set for the system. + +This module is used in the Cosmos Hub, the first Hub in the Cosmos +network. + +## Contents + +* [State](#state) + * [Pool](#pool) + * [LastTotalPower](#lasttotalpower) + * [ValidatorUpdates](#validatorupdates) + * [UnbondingID](#unbondingid) + * [Params](#params) + * [Validator](#validator) + * [Delegation](#delegation) + * [UnbondingDelegation](#unbondingdelegation) + * [Redelegation](#redelegation) + * [Queues](#queues) + * [HistoricalInfo](#historicalinfo) +* [State Transitions](#state-transitions) + * [Validators](#validators) + * [Delegations](#delegations) + * [Slashing](#slashing) + * [How Shares are calculated](#how-shares-are-calculated) +* [Messages](#messages) + * [MsgCreateValidator](#msgcreatevalidator) + * [MsgEditValidator](#msgeditvalidator) + * [MsgDelegate](#msgdelegate) + * [MsgUndelegate](#msgundelegate) + * [MsgCancelUnbondingDelegation](#msgcancelunbondingdelegation) + * [MsgBeginRedelegate](#msgbeginredelegate) + * [MsgUpdateParams](#msgupdateparams) +* [Begin-Block](#begin-block) + * [Historical Info Tracking](#historical-info-tracking) +* [End-Block](#end-block) + * [Validator Set Changes](#validator-set-changes) + * [Queues](#queues-1) +* [Hooks](#hooks) +* [Events](#events) + * [EndBlocker](#endblocker) + * [Msg's](#msgs) +* [Parameters](#parameters) +* [Client](#client) + * [CLI](#cli) + * [gRPC](#grpc) + * [REST](#rest) + +## State + +### Pool + +Pool is used for tracking bonded and not-bonded token supply of the bond denomination. + +### LastTotalPower + +LastTotalPower tracks the total amounts of bonded tokens recorded during the previous end block. +Store entries prefixed with "Last" must remain unchanged until EndBlock. + +* LastTotalPower: `0x12 -> ProtocolBuffer(math.Int)` + +### ValidatorUpdates + +ValidatorUpdates contains the validator updates returned to ABCI at the end of every block. +The values are overwritten in every block. + +* ValidatorUpdates `0x61 -> []abci.ValidatorUpdate` + +### UnbondingID + +UnbondingID stores the ID of the latest unbonding operation. It enables creating unique IDs for unbonding operations, i.e., UnbondingID is incremented every time a new unbonding operation (validator unbonding, unbonding delegation, redelegation) is initiated. + +* UnbondingID: `0x37 -> uint64` + +### Params + +The staking module stores its params in state with the prefix of `0x51`, +it can be updated with governance or the address with authority. + +* Params: `0x51 | ProtocolBuffer(Params)` + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/staking.proto#L310-L333 +``` + +### Validator + +Validators can have one of three statuses + +* `Unbonded`: The validator is not in the active set. They cannot sign blocks and do not earn + rewards. They can receive delegations. +* `Bonded`: Once the validator receives sufficient bonded tokens they automatically join the + active set during [`EndBlock`](#validator-set-changes) and their status is updated to `Bonded`. + They are signing blocks and receiving rewards. They can receive further delegations. + They can be slashed for misbehavior. Delegators to this validator who unbond their delegation + must wait the duration of the UnbondingTime, a chain-specific param, during which time + they are still slashable for offences of the source validator if those offences were committed + during the period of time that the tokens were bonded. +* `Unbonding`: When a validator leaves the active set, either by choice or due to slashing, jailing or + tombstoning, an unbonding of all their delegations begins. All delegations must then wait the UnbondingTime + before their tokens are moved to their accounts from the `BondedPool`. + + +Tombstoning is permanent, once tombstoned a validator's consensus key can not be reused within the chain where the tombstoning happened. + + +Validators objects should be primarily stored and accessed by the +`OperatorAddr`, an SDK validator address for the operator of the validator. Two +additional indices are maintained per validator object in order to fulfill +required lookups for slashing and validator-set updates. A third special index +(`LastValidatorPower`) is also maintained which however remains constant +throughout each block, unlike the first two indices which mirror the validator +records within a block. + +* Validators: `0x21 | OperatorAddrLen (1 byte) | OperatorAddr -> ProtocolBuffer(validator)` +* ValidatorsByConsAddr: `0x22 | ConsAddrLen (1 byte) | ConsAddr -> OperatorAddr` +* ValidatorsByPower: `0x23 | BigEndian(ConsensusPower) | OperatorAddrLen (1 byte) | OperatorAddr -> OperatorAddr` +* LastValidatorsPower: `0x11 | OperatorAddrLen (1 byte) | OperatorAddr -> ProtocolBuffer(ConsensusPower)` +* ValidatorsByUnbondingID: `0x38 | UnbondingID -> 0x21 | OperatorAddrLen (1 byte) | OperatorAddr` + +`Validators` is the primary index - it ensures that each operator can have only one +associated validator, where the public key of that validator can change in the +future. Delegators can refer to the immutable operator of the validator, without +concern for the changing public key. + +`ValidatorsByUnbondingID` is an additional index that enables lookups for +validators by the unbonding IDs corresponding to their current unbonding. + +`ValidatorByConsAddr` is an additional index that enables lookups for slashing. +When CometBFT reports evidence, it provides the validator address, so this +map is needed to find the operator. Note that the `ConsAddr` corresponds to the +address which can be derived from the validator's `ConsPubKey`. + +`ValidatorsByPower` is an additional index that provides a sorted list of +potential validators to quickly determine the current active set. Here +ConsensusPower is validator.Tokens/10^6 by default. Note that all validators +where `Jailed` is true are not stored within this index. + +`LastValidatorsPower` is a special index that provides a historical list of the +last-block's bonded validators. This index remains constant during a block but +is updated during the validator set update process which takes place in [`EndBlock`](#end-block). + +Each validator's state is stored in a `Validator` struct: + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/staking.proto#L82-L138 +``` + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/staking.proto#L26-L80 +``` + +### Delegation + +Delegations are identified by combining `DelegatorAddr` (the address of the delegator) +with the `ValidatorAddr` Delegators are indexed in the store as follows: + +* Delegation: `0x31 | DelegatorAddrLen (1 byte) | DelegatorAddr | ValidatorAddrLen (1 byte) | ValidatorAddr -> ProtocolBuffer(delegation)` + +Stake holders may delegate coins to validators; under this circumstance their +funds are held in a `Delegation` data structure. It is owned by one +delegator, and is associated with the shares for one validator. The sender of +the transaction is the owner of the bond. + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/staking.proto#L198-L216 +``` + +#### Delegator Shares + +When one delegates tokens to a Validator, they are issued a number of delegator shares based on a +dynamic exchange rate, calculated as follows from the total number of tokens delegated to the +validator and the number of shares issued so far: + +`Shares per Token = validator.TotalShares() / validator.Tokens()` + +Only the number of shares received is stored on the DelegationEntry. When a delegator then +Undelegates, the token amount they receive is calculated from the number of shares they currently +hold and the inverse exchange rate: + +`Tokens per Share = validator.Tokens() / validatorShares()` + +These `Shares` are simply an accounting mechanism. They are not a fungible asset. The reason for +this mechanism is to simplify the accounting around slashing. Rather than iteratively slashing the +tokens of every delegation entry, instead the Validator's total bonded tokens can be slashed, +effectively reducing the value of each issued delegator share. + +### UnbondingDelegation + +Shares in a `Delegation` can be unbonded, but they must for some time exist as +an `UnbondingDelegation`, where shares can be reduced if Byzantine behavior is +detected. + +`UnbondingDelegation` are indexed in the store as: + +* UnbondingDelegation: `0x32 | DelegatorAddrLen (1 byte) | DelegatorAddr | ValidatorAddrLen (1 byte) | ValidatorAddr -> ProtocolBuffer(unbondingDelegation)` +* UnbondingDelegationsFromValidator: `0x33 | ValidatorAddrLen (1 byte) | ValidatorAddr | DelegatorAddrLen (1 byte) | DelegatorAddr -> nil` +* UnbondingDelegationByUnbondingId: `0x38 | UnbondingId -> 0x32 | DelegatorAddrLen (1 byte) | DelegatorAddr | ValidatorAddrLen (1 byte) | ValidatorAddr` + `UnbondingDelegation` is used in queries, to lookup all unbonding delegations for + a given delegator. + +`UnbondingDelegationsFromValidator` is used in slashing, to lookup all +unbonding delegations associated with a given validator that need to be +slashed. + +`UnbondingDelegationByUnbondingId` is an additional index that enables +lookups for unbonding delegations by the unbonding IDs of the containing +unbonding delegation entries. + +A UnbondingDelegation object is created every time an unbonding is initiated. + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/staking.proto#L218-L261 +``` + +### Redelegation + +The bonded tokens worth of a `Delegation` may be instantly redelegated from a +source validator to a different validator (destination validator). However when +this occurs they must be tracked in a `Redelegation` object, whereby their +shares can be slashed if their tokens have contributed to a Byzantine fault +committed by the source validator. + +`Redelegation` are indexed in the store as: + +* Redelegations: `0x34 | DelegatorAddrLen (1 byte) | DelegatorAddr | ValidatorAddrLen (1 byte) | ValidatorSrcAddr | ValidatorDstAddr -> ProtocolBuffer(redelegation)` +* RedelegationsBySrc: `0x35 | ValidatorSrcAddrLen (1 byte) | ValidatorSrcAddr | ValidatorDstAddrLen (1 byte) | ValidatorDstAddr | DelegatorAddrLen (1 byte) | DelegatorAddr -> nil` +* RedelegationsByDst: `0x36 | ValidatorDstAddrLen (1 byte) | ValidatorDstAddr | ValidatorSrcAddrLen (1 byte) | ValidatorSrcAddr | DelegatorAddrLen (1 byte) | DelegatorAddr -> nil` +* RedelegationByUnbondingId: `0x38 | UnbondingId -> 0x34 | DelegatorAddrLen (1 byte) | DelegatorAddr | ValidatorAddrLen (1 byte) | ValidatorSrcAddr | ValidatorDstAddr` + +`Redelegations` is used for queries, to lookup all redelegations for a given +delegator. + +`RedelegationsBySrc` is used for slashing based on the `ValidatorSrcAddr`. + +`RedelegationsByDst` is used for slashing based on the `ValidatorDstAddr` + +The first map here is used for queries, to lookup all redelegations for a given +delegator. The second map is used for slashing based on the `ValidatorSrcAddr`, +while the third map is for slashing based on the `ValidatorDstAddr`. + +`RedelegationByUnbondingId` is an additional index that enables +lookups for redelegations by the unbonding IDs of the containing +redelegation entries. + +A redelegation object is created every time a redelegation occurs. To prevent +"redelegation hopping" redelegations may not occur under the situation that: + +* the (re)delegator already has another immature redelegation in progress + with a destination to a validator (let's call it `Validator X`) +* and, the (re)delegator is attempting to create a *new* redelegation + where the source validator for this new redelegation is `Validator X`. + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/staking.proto#L263-L308 +``` + +### Queues + +All queue objects are sorted by timestamp. The time used within any queue is +firstly converted to UTC, rounded to the nearest nanosecond then sorted. The sortable time format +used is a slight modification of the RFC3339Nano and uses the format string +`"2006-01-02T15:04:05.000000000"`. Notably this format: + +* right pads all zeros +* drops the time zone info (we already use UTC) + +In all cases, the stored timestamp represents the maturation time of the queue +element. + +#### UnbondingDelegationQueue + +For the purpose of tracking progress of unbonding delegations the unbonding +delegations queue is kept. + +* UnbondingDelegation: `0x41 | format(time) -> []DVPair` + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/staking.proto#L162-L172 +``` + +#### RedelegationQueue + +For the purpose of tracking progress of redelegations the redelegation queue is +kept. + +* RedelegationQueue: `0x42 | format(time) -> []DVVTriplet` + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/staking.proto#L179-L191 +``` + +#### ValidatorQueue + +For the purpose of tracking progress of unbonding validators the validator +queue is kept. + +* ValidatorQueueTime: `0x43 | format(time) -> []sdk.ValAddress` + +The stored object by each key is an array of validator operator addresses from +which the validator object can be accessed. Typically it is expected that only +a single validator record will be associated with a given timestamp however it is possible +that multiple validators exist in the queue at the same location. + +### HistoricalInfo + +HistoricalInfo objects are stored and pruned at each block such that the staking keeper persists +the `n` most recent historical info defined by staking module parameter: `HistoricalEntries`. + +```go expandable +syntax = "proto3"; +package cosmos.staking.v1beta1; + +import "gogoproto/gogo.proto"; +import "google/protobuf/any.proto"; +import "google/protobuf/duration.proto"; +import "google/protobuf/timestamp.proto"; + +import "cosmos_proto/cosmos.proto"; +import "cosmos/base/v1beta1/coin.proto"; +import "amino/amino.proto"; +import "tendermint/types/types.proto"; +import "tendermint/abci/types.proto"; + +option go_package = "github.com/cosmos/cosmos-sdk/x/staking/types"; + +// HistoricalInfo contains header and validator information for a given block. +// It is stored as part of staking module's state, which persists the `n` most +// recent HistoricalInfo +// (`n` is set by the staking module's `historical_entries` parameter). +message HistoricalInfo { + tendermint.types.Header header = 1 [(gogoproto.nullable) = false, (amino.dont_omitempty) = true]; + repeated Validator valset = 2 [(gogoproto.nullable) = false, (amino.dont_omitempty) = true]; +} + +// CommissionRates defines the initial commission rates to be used for creating +// a validator. +message CommissionRates { + option (gogoproto.equal) = true; + option (gogoproto.goproto_stringer) = false; + + // rate is the commission rate charged to delegators, as a fraction. + string rate = 1 [ + (cosmos_proto.scalar) = "cosmos.Dec", + (gogoproto.customtype) = "github.com/cosmos/cosmos-sdk/types.Dec", + (gogoproto.nullable) = false + ]; + // max_rate defines the maximum commission rate which validator can ever charge, as a fraction. + string max_rate = 2 [ + (cosmos_proto.scalar) = "cosmos.Dec", + (gogoproto.customtype) = "github.com/cosmos/cosmos-sdk/types.Dec", + (gogoproto.nullable) = false + ]; + // max_change_rate defines the maximum daily increase of the validator commission, as a fraction. + string max_change_rate = 3 [ + (cosmos_proto.scalar) = "cosmos.Dec", + (gogoproto.customtype) = "github.com/cosmos/cosmos-sdk/types.Dec", + (gogoproto.nullable) = false + ]; +} + +// Commission defines commission parameters for a given validator. +message Commission { + option (gogoproto.equal) = true; + option (gogoproto.goproto_stringer) = false; + + // commission_rates defines the initial commission rates to be used for creating a validator. + CommissionRates commission_rates = 1 + [(gogoproto.embed) = true, (gogoproto.nullable) = false, (amino.dont_omitempty) = true]; + // update_time is the last time the commission rate was changed. + google.protobuf.Timestamp update_time = 2 + [(gogoproto.nullable) = false, (amino.dont_omitempty) = true, (gogoproto.stdtime) = true]; +} + +// Description defines a validator description. +message Description { + option (gogoproto.equal) = true; + option (gogoproto.goproto_stringer) = false; + + // moniker defines a human-readable name for the validator. + string moniker = 1; + // identity defines an optional identity signature (ex. UPort or Keybase). + string identity = 2; + // website defines an optional website link. + string website = 3; + // security_contact defines an optional email for security contact. + string security_contact = 4; + // details define other optional details. + string details = 5; +} + +// Validator defines a validator, together with the total amount of the +// Validator's bond shares and their exchange rate to coins. Slashing results in +// a decrease in the exchange rate, allowing correct calculation of future +// undelegations without iterating over delegators. When coins are delegated to +// this validator, the validator is credited with a delegation whose number of +// bond shares is based on the amount of coins delegated divided by the current +// exchange rate. Voting power can be calculated as total bonded shares +// multiplied by exchange rate. +message Validator { + option (gogoproto.equal) = false; + option (gogoproto.goproto_stringer) = false; + option (gogoproto.goproto_getters) = false; + + // operator_address defines the address of the validator's operator; bech encoded in JSON. + string operator_address = 1 [(cosmos_proto.scalar) = "cosmos.AddressString"]; + // consensus_pubkey is the consensus public key of the validator, as a Protobuf Any. + google.protobuf.Any consensus_pubkey = 2 [(cosmos_proto.accepts_interface) = "cosmos.crypto.PubKey"]; + // jailed defined whether the validator has been jailed from bonded status or not. + bool jailed = 3; + // status is the validator status (bonded/unbonding/unbonded). + BondStatus status = 4; + // tokens define the delegated tokens (incl. self-delegation). + string tokens = 5 [ + (cosmos_proto.scalar) = "cosmos.Int", + (gogoproto.customtype) = "github.com/cosmos/cosmos-sdk/types.Int", + (gogoproto.nullable) = false + ]; + // delegator_shares defines total shares issued to a validator's delegators. + string delegator_shares = 6 [ + (cosmos_proto.scalar) = "cosmos.Dec", + (gogoproto.customtype) = "github.com/cosmos/cosmos-sdk/types.Dec", + (gogoproto.nullable) = false + ]; + // description defines the description terms for the validator. + Description description = 7 [(gogoproto.nullable) = false, (amino.dont_omitempty) = true]; + // unbonding_height defines, if unbonding, the height at which this validator has begun unbonding. + int64 unbonding_height = 8; + // unbonding_time defines, if unbonding, the min time for the validator to complete unbonding. + google.protobuf.Timestamp unbonding_time = 9 + [(gogoproto.nullable) = false, (amino.dont_omitempty) = true, (gogoproto.stdtime) = true]; + // commission defines the commission parameters. + Commission commission = 10 [(gogoproto.nullable) = false, (amino.dont_omitempty) = true]; + // min_self_delegation is the validator's self declared minimum self delegation. + // + // Since: cosmos-sdk 0.46 + string min_self_delegation = 11 [ + (cosmos_proto.scalar) = "cosmos.Int", + (gogoproto.customtype) = "github.com/cosmos/cosmos-sdk/types.Int", + (gogoproto.nullable) = false + ]; + + // strictly positive if this validator's unbonding has been stopped by external modules + int64 unbonding_on_hold_ref_count = 12; + + // list of unbonding ids, each uniquely identifing an unbonding of this validator + repeated uint64 unbonding_ids = 13; +} + +// BondStatus is the status of a validator. +enum BondStatus { + option (gogoproto.goproto_enum_prefix) = false; + + // UNSPECIFIED defines an invalid validator status. + BOND_STATUS_UNSPECIFIED = 0 [(gogoproto.enumvalue_customname) = "Unspecified"]; + // UNBONDED defines a validator that is not bonded. + BOND_STATUS_UNBONDED = 1 [(gogoproto.enumvalue_customname) = "Unbonded"]; + // UNBONDING defines a validator that is unbonding. + BOND_STATUS_UNBONDING = 2 [(gogoproto.enumvalue_customname) = "Unbonding"]; + // BONDED defines a validator that is bonded. + BOND_STATUS_BONDED = 3 [(gogoproto.enumvalue_customname) = "Bonded"]; +} + +// ValAddresses defines a repeated set of validator addresses. +message ValAddresses { + option (gogoproto.goproto_stringer) = false; + option (gogoproto.stringer) = true; + + repeated string addresses = 1 [(cosmos_proto.scalar) = "cosmos.AddressString"]; +} + +// DVPair is struct that just has a delegator-validator pair with no other data. +// It is intended to be used as a marshalable pointer. For example, a DVPair can +// be used to construct the key to getting an UnbondingDelegation from state. +message DVPair { + option (gogoproto.equal) = false; + option (gogoproto.goproto_getters) = false; + option (gogoproto.goproto_stringer) = false; + + string delegator_address = 1 [(cosmos_proto.scalar) = "cosmos.AddressString"]; + string validator_address = 2 [(cosmos_proto.scalar) = "cosmos.AddressString"]; +} + +// DVPairs defines an array of DVPair objects. +message DVPairs { + repeated DVPair pairs = 1 [(gogoproto.nullable) = false, (amino.dont_omitempty) = true]; +} + +// DVVTriplet is struct that just has a delegator-validator-validator triplet +// with no other data. It is intended to be used as a marshalable pointer. For +// example, a DVVTriplet can be used to construct the key to getting a +// Redelegation from state. +message DVVTriplet { + option (gogoproto.equal) = false; + option (gogoproto.goproto_getters) = false; + option (gogoproto.goproto_stringer) = false; + + string delegator_address = 1 [(cosmos_proto.scalar) = "cosmos.AddressString"]; + string validator_src_address = 2 [(cosmos_proto.scalar) = "cosmos.AddressString"]; + string validator_dst_address = 3 [(cosmos_proto.scalar) = "cosmos.AddressString"]; +} + +// DVVTriplets defines an array of DVVTriplet objects. +message DVVTriplets { + repeated DVVTriplet triplets = 1 [(gogoproto.nullable) = false, (amino.dont_omitempty) = true]; +} + +// Delegation represents the bond with tokens held by an account. It is +// owned by one delegator, and is associated with the voting power of one +// validator. +message Delegation { + option (gogoproto.equal) = false; + option (gogoproto.goproto_getters) = false; + option (gogoproto.goproto_stringer) = false; + + // delegator_address is the bech32-encoded address of the delegator. + string delegator_address = 1 [(cosmos_proto.scalar) = "cosmos.AddressString"]; + // validator_address is the bech32-encoded address of the validator. + string validator_address = 2 [(cosmos_proto.scalar) = "cosmos.AddressString"]; + // shares define the delegation shares received. + string shares = 3 [ + (cosmos_proto.scalar) = "cosmos.Dec", + (gogoproto.customtype) = "github.com/cosmos/cosmos-sdk/types.Dec", + (gogoproto.nullable) = false + ]; +} + +// UnbondingDelegation stores all of a single delegator's unbonding bonds +// for a single validator in an time-ordered list. +message UnbondingDelegation { + option (gogoproto.equal) = false; + option (gogoproto.goproto_getters) = false; + option (gogoproto.goproto_stringer) = false; + + // delegator_address is the bech32-encoded address of the delegator. + string delegator_address = 1 [(cosmos_proto.scalar) = "cosmos.AddressString"]; + // validator_address is the bech32-encoded address of the validator. + string validator_address = 2 [(cosmos_proto.scalar) = "cosmos.AddressString"]; + // entries are the unbonding delegation entries. + repeated UnbondingDelegationEntry entries = 3 + [(gogoproto.nullable) = false, (amino.dont_omitempty) = true]; // unbonding delegation entries +} + +// UnbondingDelegationEntry defines an unbonding object with relevant metadata. +message UnbondingDelegationEntry { + option (gogoproto.equal) = true; + option (gogoproto.goproto_stringer) = false; + + // creation_height is the height which the unbonding took place. + int64 creation_height = 1; + // completion_time is the unix time for unbonding completion. + google.protobuf.Timestamp completion_time = 2 + [(gogoproto.nullable) = false, (amino.dont_omitempty) = true, (gogoproto.stdtime) = true]; + // initial_balance defines the tokens initially scheduled to receive at completion. + string initial_balance = 3 [ + (cosmos_proto.scalar) = "cosmos.Int", + (gogoproto.customtype) = "github.com/cosmos/cosmos-sdk/types.Int", + (gogoproto.nullable) = false + ]; + // balance defines the tokens to receive at completion. + string balance = 4 [ + (cosmos_proto.scalar) = "cosmos.Int", + (gogoproto.customtype) = "github.com/cosmos/cosmos-sdk/types.Int", + (gogoproto.nullable) = false + ]; + // Incrementing id that uniquely identifies this entry + uint64 unbonding_id = 5; + + // Strictly positive if this entry's unbonding has been stopped by external modules + int64 unbonding_on_hold_ref_count = 6; +} + +// RedelegationEntry defines a redelegation object with relevant metadata. +message RedelegationEntry { + option (gogoproto.equal) = true; + option (gogoproto.goproto_stringer) = false; + + // creation_height defines the height which the redelegation took place. + int64 creation_height = 1; + // completion_time defines the unix time for redelegation completion. + google.protobuf.Timestamp completion_time = 2 + [(gogoproto.nullable) = false, (amino.dont_omitempty) = true, (gogoproto.stdtime) = true]; + // initial_balance defines the initial balance when redelegation started. + string initial_balance = 3 [ + (cosmos_proto.scalar) = "cosmos.Int", + (gogoproto.customtype) = "github.com/cosmos/cosmos-sdk/types.Int", + (gogoproto.nullable) = false + ]; + // shares_dst is the amount of destination-validator shares created by redelegation. + string shares_dst = 4 [ + (cosmos_proto.scalar) = "cosmos.Dec", + (gogoproto.customtype) = "github.com/cosmos/cosmos-sdk/types.Dec", + (gogoproto.nullable) = false + ]; + // Incrementing id that uniquely identifies this entry + uint64 unbonding_id = 5; + + // Strictly positive if this entry's unbonding has been stopped by external modules + int64 unbonding_on_hold_ref_count = 6; +} + +// Redelegation contains the list of a particular delegator's redelegating bonds +// from a particular source validator to a particular destination validator. +message Redelegation { + option (gogoproto.equal) = false; + option (gogoproto.goproto_getters) = false; + option (gogoproto.goproto_stringer) = false; + + // delegator_address is the bech32-encoded address of the delegator. + string delegator_address = 1 [(cosmos_proto.scalar) = "cosmos.AddressString"]; + // validator_src_address is the validator redelegation source operator address. + string validator_src_address = 2 [(cosmos_proto.scalar) = "cosmos.AddressString"]; + // validator_dst_address is the validator redelegation destination operator address. + string validator_dst_address = 3 [(cosmos_proto.scalar) = "cosmos.AddressString"]; + // entries are the redelegation entries. + repeated RedelegationEntry entries = 4 + [(gogoproto.nullable) = false, (amino.dont_omitempty) = true]; // redelegation entries +} + +// Params defines the parameters for the x/staking module. +message Params { + option (amino.name) = "cosmos-sdk/x/staking/Params"; + option (gogoproto.equal) = true; + option (gogoproto.goproto_stringer) = false; + + // unbonding_time is the time duration of unbonding. + google.protobuf.Duration unbonding_time = 1 + [(gogoproto.nullable) = false, (amino.dont_omitempty) = true, (gogoproto.stdduration) = true]; + // max_validators is the maximum number of validators. + uint32 max_validators = 2; + // max_entries is the max entries for either unbonding delegation or redelegation (per pair/trio). + uint32 max_entries = 3; + // historical_entries is the number of historical entries to persist. + uint32 historical_entries = 4; + // bond_denom defines the bondable coin denomination. + string bond_denom = 5; + // min_commission_rate is the chain-wide minimum commission rate that a validator can charge their delegators + string min_commission_rate = 6 [ + (gogoproto.moretags) = "yaml:\"min_commission_rate\"", + (gogoproto.customtype) = "github.com/cosmos/cosmos-sdk/types.Dec", + (gogoproto.nullable) = false + ]; +} + +// DelegationResponse is equivalent to Delegation except that it contains a +// balance in addition to shares which is more suitable for client responses. +message DelegationResponse { + option (gogoproto.equal) = false; + option (gogoproto.goproto_stringer) = false; + + Delegation delegation = 1 [(gogoproto.nullable) = false, (amino.dont_omitempty) = true]; + + cosmos.base.v1beta1.Coin balance = 2 [(gogoproto.nullable) = false, (amino.dont_omitempty) = true]; +} + +// RedelegationEntryResponse is equivalent to a RedelegationEntry except that it +// contains a balance in addition to shares which is more suitable for client +// responses. +message RedelegationEntryResponse { + option (gogoproto.equal) = true; + + RedelegationEntry redelegation_entry = 1 [(gogoproto.nullable) = false, (amino.dont_omitempty) = true]; + string balance = 4 [ + (cosmos_proto.scalar) = "cosmos.Int", + (gogoproto.customtype) = "github.com/cosmos/cosmos-sdk/types.Int", + (gogoproto.nullable) = false + ]; +} + +// RedelegationResponse is equivalent to a Redelegation except that its entries +// contain a balance in addition to shares which is more suitable for client +// responses. +message RedelegationResponse { + option (gogoproto.equal) = false; + + Redelegation redelegation = 1 [(gogoproto.nullable) = false, (amino.dont_omitempty) = true]; + repeated RedelegationEntryResponse entries = 2 [(gogoproto.nullable) = false, (amino.dont_omitempty) = true]; +} + +// Pool is used for tracking bonded and not-bonded token supply of the bond +// denomination. +message Pool { + option (gogoproto.description) = true; + option (gogoproto.equal) = true; + string not_bonded_tokens = 1 [ + (cosmos_proto.scalar) = "cosmos.Int", + (gogoproto.customtype) = "github.com/cosmos/cosmos-sdk/types.Int", + (gogoproto.nullable) = false, + (gogoproto.jsontag) = "not_bonded_tokens", + (amino.dont_omitempty) = true + ]; + string bonded_tokens = 2 [ + (cosmos_proto.scalar) = "cosmos.Int", + (gogoproto.customtype) = "github.com/cosmos/cosmos-sdk/types.Int", + (gogoproto.nullable) = false, + (gogoproto.jsontag) = "bonded_tokens", + (amino.dont_omitempty) = true + ]; +} + +// Infraction indicates the infraction a validator committed. +enum Infraction { + // UNSPECIFIED defines an empty infraction. + INFRACTION_UNSPECIFIED = 0; + // DOUBLE_SIGN defines a validator that double-signs a block. + INFRACTION_DOUBLE_SIGN = 1; + // DOWNTIME defines a validator that missed signing too many blocks. + INFRACTION_DOWNTIME = 2; +} + +// ValidatorUpdates defines an array of abci.ValidatorUpdate objects. +// TODO: explore moving this to proto/cosmos/base to separate modules from tendermint dependence +message ValidatorUpdates { + repeated tendermint.abci.ValidatorUpdate updates = 1 [(gogoproto.nullable) = false, (amino.dont_omitempty) = true]; +} +``` + +At each BeginBlock, the staking keeper will persist the current Header and the Validators that committed +the current block in a `HistoricalInfo` object. The Validators are sorted on their address to ensure that +they are in a deterministic order. +The oldest HistoricalEntries will be pruned to ensure that there only exist the parameter-defined number of +historical entries. + +## State Transitions + +### Validators + +State transitions in validators are performed on every [`EndBlock`](#validator-set-changes) +in order to check for changes in the active `ValidatorSet`. + +A validator can be `Unbonded`, `Unbonding` or `Bonded`. `Unbonded` +and `Unbonding` are collectively called `Not Bonded`. A validator can move +directly between all the states, except for from `Bonded` to `Unbonded`. + +#### Not bonded to Bonded + +The following transition occurs when a validator's ranking in the `ValidatorPowerIndex` surpasses +that of the `LastValidator`. + +* set `validator.Status` to `Bonded` +* send the `validator.Tokens` from the `NotBondedTokens` to the `BondedPool` `ModuleAccount` +* delete the existing record from `ValidatorByPowerIndex` +* add a new updated record to the `ValidatorByPowerIndex` +* update the `Validator` object for this validator +* if it exists, delete any `ValidatorQueue` record for this validator + +#### Bonded to Unbonding + +When a validator begins the unbonding process the following operations occur: + +* send the `validator.Tokens` from the `BondedPool` to the `NotBondedTokens` `ModuleAccount` +* set `validator.Status` to `Unbonding` +* delete the existing record from `ValidatorByPowerIndex` +* add a new updated record to the `ValidatorByPowerIndex` +* update the `Validator` object for this validator +* insert a new record into the `ValidatorQueue` for this validator + +#### Unbonding to Unbonded + +A validator moves from unbonding to unbonded when the `ValidatorQueue` object +moves from bonded to unbonded + +* update the `Validator` object for this validator +* set `validator.Status` to `Unbonded` + +#### Jail/Unjail + +when a validator is jailed it is effectively removed from the CometBFT set. +this process may be also be reversed. the following operations occur: + +* set `Validator.Jailed` and update object +* if jailed delete record from `ValidatorByPowerIndex` +* if unjailed add record to `ValidatorByPowerIndex` + +Jailed validators are not present in any of the following stores: + +* the power store (from consensus power to address) + +### Delegations + +#### Delegate + +When a delegation occurs both the validator and the delegation objects are affected + +* determine the delegators shares based on tokens delegated and the validator's exchange rate +* remove tokens from the sending account +* add shares the delegation object or add them to a created validator object +* add new delegator shares and update the `Validator` object +* transfer the `delegation.Amount` from the delegator's account to the `BondedPool` or the `NotBondedPool` `ModuleAccount` depending if the `validator.Status` is `Bonded` or not +* delete the existing record from `ValidatorByPowerIndex` +* add an new updated record to the `ValidatorByPowerIndex` + +#### Begin Unbonding + +As a part of the Undelegate and Complete Unbonding state transitions Unbond +Delegation may be called. + +* subtract the unbonded shares from delegator +* add the unbonded tokens to an `UnbondingDelegationEntry` +* update the delegation or remove the delegation if there are no more shares +* if the delegation is the operator of the validator and no more shares exist then trigger a jail validator +* update the validator with removed the delegator shares and associated coins +* if the validator state is `Bonded`, transfer the `Coins` worth of the unbonded + shares from the `BondedPool` to the `NotBondedPool` `ModuleAccount` +* remove the validator if it is unbonded and there are no more delegation shares. +* remove the validator if it is unbonded and there are no more delegation shares +* get a unique `unbondingId` and map it to the `UnbondingDelegationEntry` in `UnbondingDelegationByUnbondingId` +* call the `AfterUnbondingInitiated(unbondingId)` hook +* add the unbonding delegation to `UnbondingDelegationQueue` with the completion time set to `UnbondingTime` + +#### Cancel an `UnbondingDelegation` Entry + +When a `cancel unbond delegation` occurs both the `validator`, the `delegation` and an `UnbondingDelegationQueue` state will be updated. + +* if cancel unbonding delegation amount equals to the `UnbondingDelegation` entry `balance`, then the `UnbondingDelegation` entry deleted from `UnbondingDelegationQueue`. +* if the `cancel unbonding delegation amount is less than the `UnbondingDelegation`entry balance, then the`UnbondingDelegation`entry will be updated with new balance in the`UnbondingDelegationQueue\`. +* cancel `amount` is [Delegated](#delegations) back to the original `validator`. + +#### Complete Unbonding + +For undelegations which do not complete immediately, the following operations +occur when the unbonding delegation queue element matures: + +* remove the entry from the `UnbondingDelegation` object +* transfer the tokens from the `NotBondedPool` `ModuleAccount` to the delegator `Account` + +#### Begin Redelegation + +Redelegations affect the delegation, source and destination validators. + +* perform an `unbond` delegation from the source validator to retrieve the tokens worth of the unbonded shares +* using the unbonded tokens, `Delegate` them to the destination validator +* if the `sourceValidator.Status` is `Bonded`, and the `destinationValidator` is not, + transfer the newly delegated tokens from the `BondedPool` to the `NotBondedPool` `ModuleAccount` +* otherwise, if the `sourceValidator.Status` is not `Bonded`, and the `destinationValidator` + is `Bonded`, transfer the newly delegated tokens from the `NotBondedPool` to the `BondedPool` `ModuleAccount` +* record the token amount in an new entry in the relevant `Redelegation` + +From when a redelegation begins until it completes, the delegator is in a state of "pseudo-unbonding", and can still be +slashed for infractions that occurred before the redelegation began. + +#### Complete Redelegation + +When a redelegations complete the following occurs: + +* remove the entry from the `Redelegation` object + +### Slashing + +#### Slash Validator + +When a Validator is slashed, the following occurs: + +* The total `slashAmount` is calculated as the `slashFactor` (a chain parameter) \* `TokensFromConsensusPower`, + the total number of tokens bonded to the validator at the time of the infraction. +* Every unbonding delegation and pseudo-unbonding redelegation such that the infraction occurred before the unbonding or + redelegation began from the validator are slashed by the `slashFactor` percentage of the initialBalance. +* Each amount slashed from redelegations and unbonding delegations is subtracted from the + total slash amount. +* The `remaingSlashAmount` is then slashed from the validator's tokens in the `BondedPool` or + `NonBondedPool` depending on the validator's status. This reduces the total supply of tokens. + +In the case of a slash due to any infraction that requires evidence to submitted (for example double-sign), the slash +occurs at the block where the evidence is included, not at the block where the infraction occurred. +Put otherwise, validators are not slashed retroactively, only when they are caught. + +#### Slash Unbonding Delegation + +When a validator is slashed, so are those unbonding delegations from the validator that began unbonding +after the time of the infraction. Every entry in every unbonding delegation from the validator +is slashed by `slashFactor`. The amount slashed is calculated from the `InitialBalance` of the +delegation and is capped to prevent a resulting negative balance. Completed (or mature) unbondings are not slashed. + +#### Slash Redelegation + +When a validator is slashed, so are all redelegations from the validator that began after the +infraction. Redelegations are slashed by `slashFactor`. +Redelegations that began before the infraction are not slashed. +The amount slashed is calculated from the `InitialBalance` of the delegation and is capped to +prevent a resulting negative balance. +Mature redelegations (that have completed pseudo-unbonding) are not slashed. + +### How Shares are calculated + +At any given point in time, each validator has a number of tokens, `T`, and has a number of shares issued, `S`. +Each delegator, `i`, holds a number of shares, `S_i`. +The number of tokens is the sum of all tokens delegated to the validator, plus the rewards, minus the slashes. + +The delegator is entitled to a portion of the underlying tokens proportional to their proportion of shares. +So delegator `i` is entitled to `T * S_i / S` of the validator's tokens. + +When a delegator delegates new tokens to the validator, they receive a number of shares proportional to their contribution. +So when delegator `j` delegates `T_j` tokens, they receive `S_j = S * T_j / T` shares. +The total number of tokens is now `T + T_j`, and the total number of shares is `S + S_j`. +`j`s proportion of the shares is the same as their proportion of the total tokens contributed: `(S + S_j) / S = (T + T_j) / T`. + +A special case is the initial delegation, when `T = 0` and `S = 0`, so `T_j / T` is undefined. +For the initial delegation, delegator `j` who delegates `T_j` tokens receive `S_j = T_j` shares. +So a validator that hasn't received any rewards and has not been slashed will have `T = S`. + +## Messages + +In this section we describe the processing of the staking messages and the corresponding updates to the state. All created/modified state objects specified by each message are defined within the [state](#state) section. + +### MsgCreateValidator + +A validator is created using the `MsgCreateValidator` message. +The validator must be created with an initial delegation from the operator. + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/tx.proto#L20-L21 +``` + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/tx.proto#L50-L73 +``` + +This message is expected to fail if: + +* another validator with this operator address is already registered +* another validator with this pubkey is already registered +* the initial self-delegation tokens are of a denom not specified as the bonding denom +* the commission parameters are faulty, namely: + * `MaxRate` is either > 1 or < 0 + * the initial `Rate` is either negative or > `MaxRate` + * the initial `MaxChangeRate` is either negative or > `MaxRate` +* the description fields are too large + +This message creates and stores the `Validator` object at appropriate indexes. +Additionally a self-delegation is made with the initial tokens delegation +tokens `Delegation`. The validator always starts as unbonded but may be bonded +in the first end-block. + +### MsgEditValidator + +The `Description`, `CommissionRate` of a validator can be updated using the +`MsgEditValidator` message. + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/tx.proto#L23-L24 +``` + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/tx.proto#L78-L97 +``` + +This message is expected to fail if: + +* the initial `CommissionRate` is either negative or > `MaxRate` +* the `CommissionRate` has already been updated within the previous 24 hours +* the `CommissionRate` is > `MaxChangeRate` +* the description fields are too large + +This message stores the updated `Validator` object. + +### MsgDelegate + +Within this message the delegator provides coins, and in return receives +some amount of their validator's (newly created) delegator-shares that are +assigned to `Delegation.Shares`. + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/tx.proto#L26-L28 +``` + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/tx.proto#L102-L114 +``` + +This message is expected to fail if: + +* the validator does not exist +* the `Amount` `Coin` has a denomination different than one defined by `params.BondDenom` +* the exchange rate is invalid, meaning the validator has no tokens (due to slashing) but there are outstanding shares +* the amount delegated is less than the minimum allowed delegation + +If an existing `Delegation` object for provided addresses does not already +exist then it is created as part of this message otherwise the existing +`Delegation` is updated to include the newly received shares. + +The delegator receives newly minted shares at the current exchange rate. +The exchange rate is the number of existing shares in the validator divided by +the number of currently delegated tokens. + +The validator is updated in the `ValidatorByPower` index, and the delegation is +tracked in validator object in the `Validators` index. + +It is possible to delegate to a jailed validator, the only difference being it +will not be added to the power index until it is unjailed. + +![Delegation sequence](https://raw.githubusercontent.com/cosmos/cosmos-sdk/release/v0.46.x/docs/uml/svg/delegation_sequence.svg) + +### MsgUndelegate + +The `MsgUndelegate` message allows delegators to undelegate their tokens from +validator. + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/tx.proto#L34-L36 +``` + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/tx.proto#L140-L152 +``` + +This message returns a response containing the completion time of the undelegation: + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/tx.proto#L154-L158 +``` + +This message is expected to fail if: + +* the delegation doesn't exist +* the validator doesn't exist +* the delegation has less shares than the ones worth of `Amount` +* existing `UnbondingDelegation` has maximum entries as defined by `params.MaxEntries` +* the `Amount` has a denomination different than one defined by `params.BondDenom` + +When this message is processed the following actions occur: + +* validator's `DelegatorShares` and the delegation's `Shares` are both reduced by the message `SharesAmount` +* calculate the token worth of the shares remove that amount tokens held within the validator +* with those removed tokens, if the validator is: + * `Bonded` - add them to an entry in `UnbondingDelegation` (create `UnbondingDelegation` if it doesn't exist) with a completion time a full unbonding period from the current time. Update pool shares to reduce BondedTokens and increase NotBondedTokens by token worth of the shares. + * `Unbonding` - add them to an entry in `UnbondingDelegation` (create `UnbondingDelegation` if it doesn't exist) with the same completion time as the validator (`UnbondingMinTime`). + * `Unbonded` - then send the coins the message `DelegatorAddr` +* if there are no more `Shares` in the delegation, then the delegation object is removed from the store + * under this situation if the delegation is the validator's self-delegation then also jail the validator. + +![Unbond sequence](https://raw.githubusercontent.com/cosmos/cosmos-sdk/release/v0.46.x/docs/uml/svg/unbond_sequence.svg) + +### MsgCancelUnbondingDelegation + +The `MsgCancelUnbondingDelegation` message allows delegators to cancel the `unbondingDelegation` entry and delegate back to a previous validator. + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/tx.proto#L38-L42 +``` + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/tx.proto#L160-L175 +``` + +This message is expected to fail if: + +* the `unbondingDelegation` entry is already processed. +* the `cancel unbonding delegation` amount is greater than the `unbondingDelegation` entry balance. +* the `cancel unbonding delegation` height doesn't exist in the `unbondingDelegationQueue` of the delegator. + +When this message is processed the following actions occur: + +* if the `unbondingDelegation` Entry balance is zero + * in this condition `unbondingDelegation` entry will be removed from `unbondingDelegationQueue`. + * otherwise `unbondingDelegationQueue` will be updated with new `unbondingDelegation` entry balance and initial balance +* the validator's `DelegatorShares` and the delegation's `Shares` are both increased by the message `Amount`. + +### MsgBeginRedelegate + +The redelegation command allows delegators to instantly switch validators. Once +the unbonding period has passed, the redelegation is automatically completed in +the EndBlocker. + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/tx.proto#L30-L32 +``` + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/tx.proto#L119-L132 +``` + +This message returns a response containing the completion time of the redelegation: + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/tx.proto#L133-L138 +``` + +This message is expected to fail if: + +* the delegation doesn't exist +* the source or destination validators don't exist +* the delegation has less shares than the ones worth of `Amount` +* the source validator has a receiving redelegation which is not matured (aka. the redelegation may be transitive) +* existing `Redelegation` has maximum entries as defined by `params.MaxEntries` +* the `Amount` `Coin` has a denomination different than one defined by `params.BondDenom` + +When this message is processed the following actions occur: + +* the source validator's `DelegatorShares` and the delegations `Shares` are both reduced by the message `SharesAmount` +* calculate the token worth of the shares remove that amount tokens held within the source validator. +* if the source validator is: + * `Bonded` - add an entry to the `Redelegation` (create `Redelegation` if it doesn't exist) with a completion time a full unbonding period from the current time. Update pool shares to reduce BondedTokens and increase NotBondedTokens by token worth of the shares (this may be effectively reversed in the next step however). + * `Unbonding` - add an entry to the `Redelegation` (create `Redelegation` if it doesn't exist) with the same completion time as the validator (`UnbondingMinTime`). + * `Unbonded` - no action required in this step +* Delegate the token worth to the destination validator, possibly moving tokens back to the bonded state. +* if there are no more `Shares` in the source delegation, then the source delegation object is removed from the store + * under this situation if the delegation is the validator's self-delegation then also jail the validator. + +![Begin redelegation sequence](https://raw.githubusercontent.com/cosmos/cosmos-sdk/release/v0.46.x/docs/uml/svg/begin_redelegation_sequence.svg) + +### MsgUpdateParams + +The `MsgUpdateParams` update the staking module parameters. +The params are updated through a governance proposal where the signer is the gov module account address. + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/staking/v1beta1/tx.proto#L182-L195 +``` + +The message handling can fail if: + +* signer is not the authority defined in the staking keeper (usually the gov module account). +* the `bond_denom` in the updated params has zero supply in the bank module (i.e., the denom does not exist on-chain). + +## Begin-Block + +Each abci begin block call, the historical info will get stored and pruned +according to the `HistoricalEntries` parameter. + +### Historical Info Tracking + +If the `HistoricalEntries` parameter is 0, then the `BeginBlock` performs a no-op. + +Otherwise, the latest historical info is stored under the key `historicalInfoKey|height`, while any entries older than `height - HistoricalEntries` is deleted. +In most cases, this results in a single entry being pruned per block. +However, if the parameter `HistoricalEntries` has changed to a lower value there will be multiple entries in the store that must be pruned. + +## End-Block + +Each abci end block call, the operations to update queues and validator set +changes are specified to execute. + +### Validator Set Changes + +The staking validator set is updated during this process by state transitions +that run at the end of every block. As a part of this process any updated +validators are also returned back to CometBFT for inclusion in the CometBFT +validator set which is responsible for validating CometBFT messages at the +consensus layer. Operations are as following: + +* the new validator set is taken as the top `params.MaxValidators` number of + validators retrieved from the `ValidatorsByPower` index +* the previous validator set is compared with the new validator set: + * missing validators begin unbonding and their `Tokens` are transferred from the + `BondedPool` to the `NotBondedPool` `ModuleAccount` + * new validators are instantly bonded and their `Tokens` are transferred from the + `NotBondedPool` to the `BondedPool` `ModuleAccount` + +In all cases, any validators leaving or entering the bonded validator set or +changing balances and staying within the bonded validator set incur an update +message reporting their new consensus power which is passed back to CometBFT. + +The `LastTotalPower` and `LastValidatorsPower` hold the state of the total power +and validator power from the end of the last block, and are used to check for +changes that have occurred in `ValidatorsByPower` and the total new power, which +is calculated during `EndBlock`. + +### Queues + +Within staking, certain state-transitions are not instantaneous but take place +over a duration of time (typically the unbonding period). When these +transitions are mature certain operations must take place in order to complete +the state operation. This is achieved through the use of queues which are +checked/processed at the end of each block. + +#### Unbonding Validators + +When a validator is kicked out of the bonded validator set (either through +being jailed, or not having sufficient bonded tokens) it begins the unbonding +process along with all its delegations begin unbonding (while still being +delegated to this validator). At this point the validator is said to be an +"unbonding validator", whereby it will mature to become an "unbonded validator" +after the unbonding period has passed. + +Each block the validator queue is to be checked for mature unbonding validators +(namely with a completion time `<=` current time and completion height `<=` current +block height). At this point any mature validators which do not have any +delegations remaining are deleted from state. For all other mature unbonding +validators that still have remaining delegations, the `validator.Status` is +switched from `types.Unbonding` to +`types.Unbonded`. + +Unbonding operations can be put on hold by external modules via the `PutUnbondingOnHold(unbondingId)` method. +As a result, an unbonding operation (e.g., an unbonding delegation) that is on hold, cannot complete +even if it reaches maturity. For an unbonding operation with `unbondingId` to eventually complete +(after it reaches maturity), every call to `PutUnbondingOnHold(unbondingId)` must be matched +by a call to `UnbondingCanComplete(unbondingId)`. + +#### Unbonding Delegations + +Complete the unbonding of all mature `UnbondingDelegations.Entries` within the +`UnbondingDelegations` queue with the following procedure: + +* transfer the balance coins to the delegator's wallet address +* remove the mature entry from `UnbondingDelegation.Entries` +* remove the `UnbondingDelegation` object from the store if there are no + remaining entries. + +#### Redelegations + +Complete the unbonding of all mature `Redelegation.Entries` within the +`Redelegations` queue with the following procedure: + +* remove the mature entry from `Redelegation.Entries` +* remove the `Redelegation` object from the store if there are no + remaining entries. + +## Hooks + +Other modules may register operations to execute when a certain event has +occurred within staking. These events can be registered to execute either +right `Before` or `After` the staking event (as per the hook name). The +following hooks can registered with staking: + +* `AfterValidatorCreated(Context, ValAddress) error` + * called when a validator is created +* `BeforeValidatorModified(Context, ValAddress) error` + * called when a validator's state is changed +* `AfterValidatorRemoved(Context, ConsAddress, ValAddress) error` + * called when a validator is deleted +* `AfterValidatorBonded(Context, ConsAddress, ValAddress) error` + * called when a validator is bonded +* `AfterValidatorBeginUnbonding(Context, ConsAddress, ValAddress) error` + * called when a validator begins unbonding +* `BeforeDelegationCreated(Context, AccAddress, ValAddress) error` + * called when a delegation is created +* `BeforeDelegationSharesModified(Context, AccAddress, ValAddress) error` + * called when a delegation's shares are modified +* `AfterDelegationModified(Context, AccAddress, ValAddress) error` + * called when a delegation is created or modified +* `BeforeDelegationRemoved(Context, AccAddress, ValAddress) error` + * called when a delegation is removed +* `AfterUnbondingInitiated(Context, UnbondingID)` + * called when an unbonding operation (validator unbonding, unbonding delegation, redelegation) was initiated + +## Events + +The staking module emits the following events: + +### EndBlocker + +| Type | Attribute Key | Attribute Value | +| ---------------------- | ---------------------- | ------------------------- | +| complete\_unbonding | amount | `{totalUnbondingAmount}` | +| complete\_unbonding | validator | `{validatorAddress}` | +| complete\_unbonding | delegator | `{delegatorAddress}` | +| complete\_redelegation | amount | `{totalRedelegationAmount}` | +| complete\_redelegation | source\_validator | `{srcValidatorAddress}` | +| complete\_redelegation | destination\_validator | `{dstValidatorAddress}` | +| complete\_redelegation | delegator | `{delegatorAddress}` | + +## Msg's + +### MsgCreateValidator + +| Type | Attribute Key | Attribute Value | +| ----------------- | ------------- | ------------------ | +| create\_validator | validator | `{validatorAddress}` | +| create\_validator | amount | `{delegationAmount}` | +| message | module | staking | +| message | action | create\_validator | +| message | sender | `{senderAddress}` | + +### MsgEditValidator + +| Type | Attribute Key | Attribute Value | +| --------------- | --------------------- | ------------------- | +| edit\_validator | commission\_rate | `{commissionRate}` | +| edit\_validator | min\_self\_delegation | `{minSelfDelegation}` | +| message | module | staking | +| message | action | edit\_validator | +| message | sender | `{senderAddress}` | + +### MsgDelegate + +| Type | Attribute Key | Attribute Value | +| -------- | ------------- | ------------------ | +| delegate | validator | `{validatorAddress}` | +| delegate | amount | `{delegationAmount}` | +| message | module | staking | +| message | action | delegate | +| message | sender | `{senderAddress}` | + +### MsgUndelegate + +| Type | Attribute Key | Attribute Value | +| ------- | --------------------- | ------------------ | +| unbond | validator | `{validatorAddress}` | +| unbond | amount | `{unbondAmount}` | +| unbond | completion\_time \[0] | `{completionTime}` | +| message | module | staking | +| message | action | begin\_unbonding | +| message | sender | `{senderAddress}` | + +* \[0] Time is formatted in the RFC3339 standard + +### MsgCancelUnbondingDelegation + +| Type | Attribute Key | Attribute Value | +| ----------------------------- | ---------------- | --------------------------------- | +| cancel\_unbonding\_delegation | validator | `{validatorAddress}` | +| cancel\_unbonding\_delegation | delegator | `{delegatorAddress}` | +| cancel\_unbonding\_delegation | amount | `{cancelUnbondingDelegationAmount}` | +| cancel\_unbonding\_delegation | creation\_height | `{unbondingCreationHeight}` | +| message | module | staking | +| message | action | cancel\_unbond | +| message | sender | `{senderAddress}` | + +### MsgBeginRedelegate + +| Type | Attribute Key | Attribute Value | +| ---------- | ---------------------- | --------------------- | +| redelegate | source\_validator | `{srcValidatorAddress}` | +| redelegate | destination\_validator | `{dstValidatorAddress}` | +| redelegate | amount | `{unbondAmount}` | +| redelegate | completion\_time \[0] | `{completionTime}` | +| message | module | staking | +| message | action | begin\_redelegate | +| message | sender | `{senderAddress}` | + +* \[0] Time is formatted in the RFC3339 standard + +## Parameters + +The staking module contains the following parameters: + +| Key | Type | Example | +| ----------------- | ---------------- | ---------------------- | +| UnbondingTime | string (time ns) | "259200000000000" | +| MaxValidators | uint16 | 100 | +| KeyMaxEntries | uint16 | 7 | +| HistoricalEntries | uint16 | 3 | +| BondDenom | string | "stake" | +| MinCommissionRate | string | "0.000000000000000000" | + +## Client + +### CLI + +A user can query and interact with the `staking` module using the CLI. + +#### Query + +The `query` commands allows users to query `staking` state. + +```bash +simd query staking --help +``` + +##### delegation + +The `delegation` command allows users to query delegations for an individual delegator on an individual validator. + +Usage: + +```bash +simd query staking delegation [delegator-addr] [validator-addr] [flags] +``` + +Example: + +```bash +simd query staking delegation cosmos1gghjut3ccd8ay0zduzj64hwre2fxs9ld75ru9p cosmosvaloper1gghjut3ccd8ay0zduzj64hwre2fxs9ldmqhffj +``` + +Example Output: + +```bash +balance: + amount: "10000000000" + denom: stake +delegation: + delegator_address: cosmos1gghjut3ccd8ay0zduzj64hwre2fxs9ld75ru9p + shares: "10000000000.000000000000000000" + validator_address: cosmosvaloper1gghjut3ccd8ay0zduzj64hwre2fxs9ldmqhffj +``` + +##### delegations + +The `delegations` command allows users to query delegations for an individual delegator on all validators. + +Usage: + +```bash +simd query staking delegations [delegator-addr] [flags] +``` + +Example: + +```bash +simd query staking delegations cosmos1gghjut3ccd8ay0zduzj64hwre2fxs9ld75ru9p +``` + +Example Output: + +```bash expandable +delegation_responses: +- balance: + amount: "10000000000" + denom: stake + delegation: + delegator_address: cosmos1gghjut3ccd8ay0zduzj64hwre2fxs9ld75ru9p + shares: "10000000000.000000000000000000" + validator_address: cosmosvaloper1gghjut3ccd8ay0zduzj64hwre2fxs9ldmqhffj +- balance: + amount: "10000000000" + denom: stake + delegation: + delegator_address: cosmos1gghjut3ccd8ay0zduzj64hwre2fxs9ld75ru9p + shares: "10000000000.000000000000000000" + validator_address: cosmosvaloper1x20lytyf6zkcrv5edpkfkn8sz578qg5sqfyqnp +pagination: + next_key: null + total: "0" +``` + +##### delegations-to + +The `delegations-to` command allows users to query delegations on an individual validator. + +Usage: + +```bash +simd query staking delegations-to [validator-addr] [flags] +``` + +Example: + +```bash +simd query staking delegations-to cosmosvaloper1gghjut3ccd8ay0zduzj64hwre2fxs9ldmqhffj +``` + +Example Output: + +```bash expandable +- balance: + amount: "504000000" + denom: stake + delegation: + delegator_address: cosmos1q2qwwynhv8kh3lu5fkeex4awau9x8fwt45f5cp + shares: "504000000.000000000000000000" + validator_address: cosmosvaloper1gghjut3ccd8ay0zduzj64hwre2fxs9ldmqhffj +- balance: + amount: "78125000000" + denom: uixo + delegation: + delegator_address: cosmos1qvppl3479hw4clahe0kwdlfvf8uvjtcd99m2ca + shares: "78125000000.000000000000000000" + validator_address: cosmosvaloper1gghjut3ccd8ay0zduzj64hwre2fxs9ldmqhffj +pagination: + next_key: null + total: "0" +``` + +##### historical-info + +The `historical-info` command allows users to query historical information at given height. + +Usage: + +```bash +simd query staking historical-info [height] [flags] +``` + +Example: + +```bash +simd query staking historical-info 10 +``` + +Example Output: + +```bash expandable +header: + app_hash: Lbx8cXpI868wz8sgp4qPYVrlaKjevR5WP/IjUxwp3oo= + chain_id: testnet + consensus_hash: BICRvH3cKD93v7+R1zxE2ljD34qcvIZ0Bdi389qtoi8= + data_hash: 47DEQpj8HBSa+/TImW+5JCeuQeRkm5NMpJWZG3hSuFU= + evidence_hash: 47DEQpj8HBSa+/TImW+5JCeuQeRkm5NMpJWZG3hSuFU= + height: "10" + last_block_id: + hash: RFbkpu6pWfSThXxKKl6EZVDnBSm16+U0l0xVjTX08Fk= + part_set_header: + hash: vpIvXD4rxD5GM4MXGz0Sad9I7//iVYLzZsEU4BVgWIU= + total: 1 + last_commit_hash: Ne4uXyx4QtNp4Zx89kf9UK7oG9QVbdB6e7ZwZkhy8K0= + last_results_hash: 47DEQpj8HBSa+/TImW+5JCeuQeRkm5NMpJWZG3hSuFU= + next_validators_hash: nGBgKeWBjoxeKFti00CxHsnULORgKY4LiuQwBuUrhCs= + proposer_address: mMEP2c2IRPLr99LedSRtBg9eONM= + time: "2021-10-01T06:00:49.785790894Z" + validators_hash: nGBgKeWBjoxeKFti00CxHsnULORgKY4LiuQwBuUrhCs= + version: + app: "0" + block: "11" +valset: +- commission: + commission_rates: + max_change_rate: "0.010000000000000000" + max_rate: "0.200000000000000000" + rate: "0.100000000000000000" + update_time: "2021-10-01T05:52:50.380144238Z" + consensus_pubkey: + '@type': /cosmos.crypto.ed25519.PubKey + key: Auxs3865HpB/EfssYOzfqNhEJjzys2Fo6jD5B8tPgC8= + delegator_shares: "10000000.000000000000000000" + description: + details: "" + identity: "" + moniker: myvalidator + security_contact: "" + website: "" + jailed: false + min_self_delegation: "1" + operator_address: cosmosvaloper1rne8lgs98p0jqe82sgt0qr4rdn4hgvmgp9ggcc + status: BOND_STATUS_BONDED + tokens: "10000000" + unbonding_height: "0" + unbonding_time: "1970-01-01T00:00:00Z" +``` + +##### params + +The `params` command allows users to query values set as staking parameters. + +Usage: + +```bash +simd query staking params [flags] +``` + +Example: + +```bash +simd query staking params +``` + +Example Output: + +```bash +bond_denom: stake +historical_entries: 10000 +max_entries: 7 +max_validators: 50 +unbonding_time: 1814400s +``` + +##### pool + +The `pool` command allows users to query values for amounts stored in the staking pool. + +Usage: + +```bash +simd q staking pool [flags] +``` + +Example: + +```bash +simd q staking pool +``` + +Example Output: + +```bash +bonded_tokens: "10000000" +not_bonded_tokens: "0" +``` + +##### redelegation + +The `redelegation` command allows users to query a redelegation record based on delegator and a source and destination validator address. + +Usage: + +```bash +simd query staking redelegation [delegator-addr] [src-validator-addr] [dst-validator-addr] [flags] +``` + +Example: + +```bash +simd query staking redelegation cosmos1gghjut3ccd8ay0zduzj64hwre2fxs9ld75ru9p cosmosvaloper1l2rsakp388kuv9k8qzq6lrm9taddae7fpx59wm cosmosvaloper1gghjut3ccd8ay0zduzj64hwre2fxs9ldmqhffj +``` + +Example Output: + +```bash expandable +pagination: null +redelegation_responses: +- entries: + - balance: "50000000" + redelegation_entry: + completion_time: "2021-10-24T20:33:21.960084845Z" + creation_height: 2.382847e+06 + initial_balance: "50000000" + shares_dst: "50000000.000000000000000000" + - balance: "5000000000" + redelegation_entry: + completion_time: "2021-10-25T21:33:54.446846862Z" + creation_height: 2.397271e+06 + initial_balance: "5000000000" + shares_dst: "5000000000.000000000000000000" + redelegation: + delegator_address: cosmos1gghjut3ccd8ay0zduzj64hwre2fxs9ld75ru9p + entries: null + validator_dst_address: cosmosvaloper1l2rsakp388kuv9k8qzq6lrm9taddae7fpx59wm + validator_src_address: cosmosvaloper1l2rsakp388kuv9k8qzq6lrm9taddae7fpx59wm +``` + +##### redelegations + +The `redelegations` command allows users to query all redelegation records for an individual delegator. + +Usage: + +```bash +simd query staking redelegations [delegator-addr] [flags] +``` + +Example: + +```bash +simd query staking redelegation cosmos1gghjut3ccd8ay0zduzj64hwre2fxs9ld75ru9p +``` + +Example Output: + +```bash expandable +pagination: + next_key: null + total: "0" +redelegation_responses: +- entries: + - balance: "50000000" + redelegation_entry: + completion_time: "2021-10-24T20:33:21.960084845Z" + creation_height: 2.382847e+06 + initial_balance: "50000000" + shares_dst: "50000000.000000000000000000" + - balance: "5000000000" + redelegation_entry: + completion_time: "2021-10-25T21:33:54.446846862Z" + creation_height: 2.397271e+06 + initial_balance: "5000000000" + shares_dst: "5000000000.000000000000000000" + redelegation: + delegator_address: cosmos1gghjut3ccd8ay0zduzj64hwre2fxs9ld75ru9p + entries: null + validator_dst_address: cosmosvaloper1uccl5ugxrm7vqlzwqr04pjd320d2fz0z3hc6vm + validator_src_address: cosmosvaloper1zppjyal5emta5cquje8ndkpz0rs046m7zqxrpp +- entries: + - balance: "562770000000" + redelegation_entry: + completion_time: "2021-10-25T21:42:07.336911677Z" + creation_height: 2.39735e+06 + initial_balance: "562770000000" + shares_dst: "562770000000.000000000000000000" + redelegation: + delegator_address: cosmos1gghjut3ccd8ay0zduzj64hwre2fxs9ld75ru9p + entries: null + validator_dst_address: cosmosvaloper1uccl5ugxrm7vqlzwqr04pjd320d2fz0z3hc6vm + validator_src_address: cosmosvaloper1zppjyal5emta5cquje8ndkpz0rs046m7zqxrpp +``` + +##### redelegations-from + +The `redelegations-from` command allows users to query delegations that are redelegating *from* a validator. + +Usage: + +```bash +simd query staking redelegations-from [validator-addr] [flags] +``` + +Example: + +```bash +simd query staking redelegations-from cosmosvaloper1y4rzzrgl66eyhzt6gse2k7ej3zgwmngeleucjy +``` + +Example Output: + +```bash expandable +pagination: + next_key: null + total: "0" +redelegation_responses: +- entries: + - balance: "50000000" + redelegation_entry: + completion_time: "2021-10-24T20:33:21.960084845Z" + creation_height: 2.382847e+06 + initial_balance: "50000000" + shares_dst: "50000000.000000000000000000" + - balance: "5000000000" + redelegation_entry: + completion_time: "2021-10-25T21:33:54.446846862Z" + creation_height: 2.397271e+06 + initial_balance: "5000000000" + shares_dst: "5000000000.000000000000000000" + redelegation: + delegator_address: cosmos1pm6e78p4pgn0da365plzl4t56pxy8hwtqp2mph + entries: null + validator_dst_address: cosmosvaloper1uccl5ugxrm7vqlzwqr04pjd320d2fz0z3hc6vm + validator_src_address: cosmosvaloper1y4rzzrgl66eyhzt6gse2k7ej3zgwmngeleucjy +- entries: + - balance: "221000000" + redelegation_entry: + completion_time: "2021-10-05T21:05:45.669420544Z" + creation_height: 2.120693e+06 + initial_balance: "221000000" + shares_dst: "221000000.000000000000000000" + redelegation: + delegator_address: cosmos1zqv8qxy2zgn4c58fz8jt8jmhs3d0attcussrf6 + entries: null + validator_dst_address: cosmosvaloper10mseqwnwtjaqfrwwp2nyrruwmjp6u5jhah4c3y + validator_src_address: cosmosvaloper1y4rzzrgl66eyhzt6gse2k7ej3zgwmngeleucjy +``` + +##### unbonding-delegation + +The `unbonding-delegation` command allows users to query unbonding delegations for an individual delegator on an individual validator. + +Usage: + +```bash +simd query staking unbonding-delegation [delegator-addr] [validator-addr] [flags] +``` + +Example: + +```bash +simd query staking unbonding-delegation cosmos1gghjut3ccd8ay0zduzj64hwre2fxs9ld75ru9p cosmosvaloper1gghjut3ccd8ay0zduzj64hwre2fxs9ldmqhffj +``` + +Example Output: + +```bash +delegator_address: cosmos1gghjut3ccd8ay0zduzj64hwre2fxs9ld75ru9p +entries: +- balance: "52000000" + completion_time: "2021-11-02T11:35:55.391594709Z" + creation_height: "55078" + initial_balance: "52000000" +validator_address: cosmosvaloper1gghjut3ccd8ay0zduzj64hwre2fxs9ldmqhffj +``` + +##### unbonding-delegations + +The `unbonding-delegations` command allows users to query all unbonding-delegations records for one delegator. + +Usage: + +```bash +simd query staking unbonding-delegations [delegator-addr] [flags] +``` + +Example: + +```bash +simd query staking unbonding-delegations cosmos1gghjut3ccd8ay0zduzj64hwre2fxs9ld75ru9p +``` + +Example Output: + +```bash expandable +pagination: + next_key: null + total: "0" +unbonding_responses: +- delegator_address: cosmos1gghjut3ccd8ay0zduzj64hwre2fxs9ld75ru9p + entries: + - balance: "52000000" + completion_time: "2021-11-02T11:35:55.391594709Z" + creation_height: "55078" + initial_balance: "52000000" + validator_address: cosmosvaloper1t8ehvswxjfn3ejzkjtntcyrqwvmvuknzmvtaaa + +``` + +##### unbonding-delegations-from + +The `unbonding-delegations-from` command allows users to query delegations that are unbonding *from* a validator. + +Usage: + +```bash +simd query staking unbonding-delegations-from [validator-addr] [flags] +``` + +Example: + +```bash +simd query staking unbonding-delegations-from cosmosvaloper1gghjut3ccd8ay0zduzj64hwre2fxs9ldmqhffj +``` + +Example Output: + +```bash expandable +pagination: + next_key: null + total: "0" +unbonding_responses: +- delegator_address: cosmos1qqq9txnw4c77sdvzx0tkedsafl5s3vk7hn53fn + entries: + - balance: "150000000" + completion_time: "2021-11-01T21:41:13.098141574Z" + creation_height: "46823" + initial_balance: "150000000" + validator_address: cosmosvaloper1gghjut3ccd8ay0zduzj64hwre2fxs9ldmqhffj +- delegator_address: cosmos1peteje73eklqau66mr7h7rmewmt2vt99y24f5z + entries: + - balance: "24000000" + completion_time: "2021-10-31T02:57:18.192280361Z" + creation_height: "21516" + initial_balance: "24000000" + validator_address: cosmosvaloper1gghjut3ccd8ay0zduzj64hwre2fxs9ldmqhffj +``` + +##### validator + +The `validator` command allows users to query details about an individual validator. + +Usage: + +```bash +simd query staking validator [validator-addr] [flags] +``` + +Example: + +```bash +simd query staking validator cosmosvaloper1gghjut3ccd8ay0zduzj64hwre2fxs9ldmqhffj +``` + +Example Output: + +```bash expandable +commission: + commission_rates: + max_change_rate: "0.020000000000000000" + max_rate: "0.200000000000000000" + rate: "0.050000000000000000" + update_time: "2021-10-01T19:24:52.663191049Z" +consensus_pubkey: + '@type': /cosmos.crypto.ed25519.PubKey + key: sIiexdJdYWn27+7iUHQJDnkp63gq/rzUq1Y+fxoGjXc= +delegator_shares: "32948270000.000000000000000000" +description: + details: Witval is the validator arm from Vitwit. Vitwit is into software consulting + and services business since 2015. We are working closely with Cosmos ecosystem + since 2018. We are also building tools for the ecosystem, Aneka is our explorer + for the cosmos ecosystem. + identity: 51468B615127273A + moniker: Witval + security_contact: "" + website: "" +jailed: false +min_self_delegation: "1" +operator_address: cosmosvaloper1gghjut3ccd8ay0zduzj64hwre2fxs9ldmqhffj +status: BOND_STATUS_BONDED +tokens: "32948270000" +unbonding_height: "0" +unbonding_time: "1970-01-01T00:00:00Z" +``` + +##### validators + +The `validators` command allows users to query details about all validators on a network. + +Usage: + +```bash +simd query staking validators [flags] +``` + +Example: + +```bash +simd query staking validators +``` + +Example Output: + +```bash expandable +pagination: + next_key: FPTi7TKAjN63QqZh+BaXn6gBmD5/ + total: "0" +validators: +commission: + commission_rates: + max_change_rate: "0.020000000000000000" + max_rate: "0.200000000000000000" + rate: "0.050000000000000000" + update_time: "2021-10-01T19:24:52.663191049Z" +consensus_pubkey: + '@type': /cosmos.crypto.ed25519.PubKey + key: sIiexdJdYWn27+7iUHQJDnkp63gq/rzUq1Y+fxoGjXc= +delegator_shares: "32948270000.000000000000000000" +description: + details: Witval is the validator arm from Vitwit. Vitwit is into software consulting + and services business since 2015. We are working closely with Cosmos ecosystem + since 2018. We are also building tools for the ecosystem, Aneka is our explorer + for the cosmos ecosystem. + identity: 51468B615127273A + moniker: Witval + security_contact: "" + website: "" + jailed: false + min_self_delegation: "1" + operator_address: cosmosvaloper1gghjut3ccd8ay0zduzj64hwre2fxs9ldmqhffj + status: BOND_STATUS_BONDED + tokens: "32948270000" + unbonding_height: "0" + unbonding_time: "1970-01-01T00:00:00Z" +- commission: + commission_rates: + max_change_rate: "0.100000000000000000" + max_rate: "0.200000000000000000" + rate: "0.050000000000000000" + update_time: "2021-10-04T18:02:21.446645619Z" + consensus_pubkey: + '@type': /cosmos.crypto.ed25519.PubKey + key: GDNpuKDmCg9GnhnsiU4fCWktuGUemjNfvpCZiqoRIYA= + delegator_shares: "559343421.000000000000000000" + description: + details: Noderunners is a professional validator in POS networks. We have a huge + node running experience, reliable soft and hardware. Our commissions are always + low, our support to delegators is always full. Stake with us and start receiving + your Cosmos rewards now! + identity: 812E82D12FEA3493 + moniker: Noderunners + security_contact: info@noderunners.biz + website: http://noderunners.biz + jailed: false + min_self_delegation: "1" + operator_address: cosmosvaloper1q5ku90atkhktze83j9xjaks2p7uruag5zp6wt7 + status: BOND_STATUS_BONDED + tokens: "559343421" + unbonding_height: "0" + unbonding_time: "1970-01-01T00:00:00Z" +``` + +#### Transactions + +The `tx` commands allows users to interact with the `staking` module. + +```bash +simd tx staking --help +``` + +##### create-validator + +The command `create-validator` allows users to create new validator initialized with a self-delegation to it. + +Usage: + +```bash +simd tx staking create-validator [path/to/validator.json] [flags] +``` + +Example: + +```bash +simd tx staking create-validator /path/to/validator.json \ + --chain-id="name_of_chain_id" \ + --gas="auto" \ + --gas-adjustment="1.2" \ + --gas-prices="0.025stake" \ + --from=mykey +``` + +where `validator.json` contains: + +```json expandable +{ + "pubkey": { + "@type": "/cosmos.crypto.ed25519.PubKey", + "key": "BnbwFpeONLqvWqJb3qaUbL5aoIcW3fSuAp9nT3z5f20=" + }, + "amount": "1000000stake", + "moniker": "my-moniker", + "website": "https://myweb.site", + "security": "security-contact@gmail.com", + "details": "description of your validator", + "commission-rate": "0.10", + "commission-max-rate": "0.20", + "commission-max-change-rate": "0.01", + "min-self-delegation": "1" +} +``` + +and pubkey can be obtained by using `simd tendermint show-validator` command. + +##### delegate + +The command `delegate` allows users to delegate liquid tokens to a validator. + +Usage: + +```bash +simd tx staking delegate [validator-addr] [amount] [flags] +``` + +Example: + +```bash +simd tx staking delegate cosmosvaloper1l2rsakp388kuv9k8qzq6lrm9taddae7fpx59wm 1000stake --from mykey +``` + +##### edit-validator + +The command `edit-validator` allows users to edit an existing validator account. + +Usage: + +```bash +simd tx staking edit-validator [flags] +``` + +Example: + +```bash +simd tx staking edit-validator --moniker "new_moniker_name" --website "new_webiste_url" --from mykey +``` + +##### redelegate + +The command `redelegate` allows users to redelegate illiquid tokens from one validator to another. + +Usage: + +```bash +simd tx staking redelegate [src-validator-addr] [dst-validator-addr] [amount] [flags] +``` + +Example: + +```bash +simd tx staking redelegate cosmosvaloper1gghjut3ccd8ay0zduzj64hwre2fxs9ldmqhffj cosmosvaloper1l2rsakp388kuv9k8qzq6lrm9taddae7fpx59wm 100stake --from mykey +``` + +##### unbond + +The command `unbond` allows users to unbond shares from a validator. + +Usage: + +```bash +simd tx staking unbond [validator-addr] [amount] [flags] +``` + +Example: + +```bash +simd tx staking unbond cosmosvaloper1gghjut3ccd8ay0zduzj64hwre2fxs9ldmqhffj 100stake --from mykey +``` + +##### cancel unbond + +The command `cancel-unbond` allow users to cancel the unbonding delegation entry and delegate back to the original validator. + +Usage: + +```bash +simd tx staking cancel-unbond [validator-addr] [amount] [creation-height] +``` + +Example: + +```bash +simd tx staking cancel-unbond cosmosvaloper1gghjut3ccd8ay0zduzj64hwre2fxs9ldmqhffj 100stake 123123 --from mykey +``` + +### gRPC + +A user can query the `staking` module using gRPC endpoints. + +#### Validators + +The `Validators` endpoint queries all validators that match the given status. + +```bash +cosmos.staking.v1beta1.Query/Validators +``` + +Example: + +```bash +grpcurl -plaintext localhost:9090 cosmos.staking.v1beta1.Query/Validators +``` + +Example Output: + +```bash expandable +{ + "validators": [ + { + "operatorAddress": "cosmosvaloper1rne8lgs98p0jqe82sgt0qr4rdn4hgvmgp9ggcc", + "consensusPubkey": {"@type":"/cosmos.crypto.ed25519.PubKey","key":"Auxs3865HpB/EfssYOzfqNhEJjzys2Fo6jD5B8tPgC8="}, + "status": "BOND_STATUS_BONDED", + "tokens": "10000000", + "delegatorShares": "10000000000000000000000000", + "description": { + "moniker": "myvalidator" + }, + "unbondingTime": "1970-01-01T00:00:00Z", + "commission": { + "commissionRates": { + "rate": "100000000000000000", + "maxRate": "200000000000000000", + "maxChangeRate": "10000000000000000" + }, + "updateTime": "2021-10-01T05:52:50.380144238Z" + }, + "minSelfDelegation": "1" + } + ], + "pagination": { + "total": "1" + } +} +``` + +#### Validator + +The `Validator` endpoint queries validator information for given validator address. + +```bash +cosmos.staking.v1beta1.Query/Validator +``` + +Example: + +```bash +grpcurl -plaintext -d '{"validator_addr":"cosmosvaloper1rne8lgs98p0jqe82sgt0qr4rdn4hgvmgp9ggcc"}' \ +localhost:9090 cosmos.staking.v1beta1.Query/Validator +``` + +Example Output: + +```bash expandable +{ + "validator": { + "operatorAddress": "cosmosvaloper1rne8lgs98p0jqe82sgt0qr4rdn4hgvmgp9ggcc", + "consensusPubkey": {"@type":"/cosmos.crypto.ed25519.PubKey","key":"Auxs3865HpB/EfssYOzfqNhEJjzys2Fo6jD5B8tPgC8="}, + "status": "BOND_STATUS_BONDED", + "tokens": "10000000", + "delegatorShares": "10000000000000000000000000", + "description": { + "moniker": "myvalidator" + }, + "unbondingTime": "1970-01-01T00:00:00Z", + "commission": { + "commissionRates": { + "rate": "100000000000000000", + "maxRate": "200000000000000000", + "maxChangeRate": "10000000000000000" + }, + "updateTime": "2021-10-01T05:52:50.380144238Z" + }, + "minSelfDelegation": "1" + } +} +``` + +#### ValidatorDelegations + +The `ValidatorDelegations` endpoint queries delegate information for given validator. + +```bash +cosmos.staking.v1beta1.Query/ValidatorDelegations +``` + +Example: + +```bash +grpcurl -plaintext -d '{"validator_addr":"cosmosvaloper1rne8lgs98p0jqe82sgt0qr4rdn4hgvmgp9ggcc"}' \ +localhost:9090 cosmos.staking.v1beta1.Query/ValidatorDelegations +``` + +Example Output: + +```bash expandable +{ + "delegationResponses": [ + { + "delegation": { + "delegatorAddress": "cosmos1rne8lgs98p0jqe82sgt0qr4rdn4hgvmgy3ua5t", + "validatorAddress": "cosmosvaloper1rne8lgs98p0jqe82sgt0qr4rdn4hgvmgp9ggcc", + "shares": "10000000000000000000000000" + }, + "balance": { + "denom": "stake", + "amount": "10000000" + } + } + ], + "pagination": { + "total": "1" + } +} +``` + +#### ValidatorUnbondingDelegations + +The `ValidatorUnbondingDelegations` endpoint queries delegate information for given validator. + +```bash +cosmos.staking.v1beta1.Query/ValidatorUnbondingDelegations +``` + +Example: + +```bash +grpcurl -plaintext -d '{"validator_addr":"cosmosvaloper1rne8lgs98p0jqe82sgt0qr4rdn4hgvmgp9ggcc"}' \ +localhost:9090 cosmos.staking.v1beta1.Query/ValidatorUnbondingDelegations +``` + +Example Output: + +```bash expandable +{ + "unbonding_responses": [ + { + "delegator_address": "cosmos1z3pzzw84d6xn00pw9dy3yapqypfde7vg6965fy", + "validator_address": "cosmosvaloper1rne8lgs98p0jqe82sgt0qr4rdn4hgvmgp9ggcc", + "entries": [ + { + "creation_height": "25325", + "completion_time": "2021-10-31T09:24:36.797320636Z", + "initial_balance": "20000000", + "balance": "20000000" + } + ] + }, + { + "delegator_address": "cosmos1y8nyfvmqh50p6ldpzljk3yrglppdv3t8phju77", + "validator_address": "cosmosvaloper1rne8lgs98p0jqe82sgt0qr4rdn4hgvmgp9ggcc", + "entries": [ + { + "creation_height": "13100", + "completion_time": "2021-10-30T12:53:02.272266791Z", + "initial_balance": "1000000", + "balance": "1000000" + } + ] + }, + ], + "pagination": { + "next_key": null, + "total": "8" + } +} +``` + +#### Delegation + +The `Delegation` endpoint queries delegate information for given validator delegator pair. + +```bash +cosmos.staking.v1beta1.Query/Delegation +``` + +Example: + +```bash +grpcurl -plaintext \ +-d '{"delegator_addr": "cosmos1y8nyfvmqh50p6ldpzljk3yrglppdv3t8phju77", validator_addr":"cosmosvaloper1rne8lgs98p0jqe82sgt0qr4rdn4hgvmgp9ggcc"}' \ +localhost:9090 cosmos.staking.v1beta1.Query/Delegation +``` + +Example Output: + +```bash expandable +{ + "delegation_response": + { + "delegation": + { + "delegator_address":"cosmos1y8nyfvmqh50p6ldpzljk3yrglppdv3t8phju77", + "validator_address":"cosmosvaloper1rne8lgs98p0jqe82sgt0qr4rdn4hgvmgp9ggcc", + "shares":"25083119936.000000000000000000" + }, + "balance": + { + "denom":"stake", + "amount":"25083119936" + } + } +} +``` + +#### UnbondingDelegation + +The `UnbondingDelegation` endpoint queries unbonding information for given validator delegator. + +```bash +cosmos.staking.v1beta1.Query/UnbondingDelegation +``` + +Example: + +```bash +grpcurl -plaintext \ +-d '{"delegator_addr": "cosmos1y8nyfvmqh50p6ldpzljk3yrglppdv3t8phju77", validator_addr":"cosmosvaloper1rne8lgs98p0jqe82sgt0qr4rdn4hgvmgp9ggcc"}' \ +localhost:9090 cosmos.staking.v1beta1.Query/UnbondingDelegation +``` + +Example Output: + +```bash expandable +{ + "unbond": { + "delegator_address": "cosmos1y8nyfvmqh50p6ldpzljk3yrglppdv3t8phju77", + "validator_address": "cosmosvaloper1rne8lgs98p0jqe82sgt0qr4rdn4hgvmgp9ggcc", + "entries": [ + { + "creation_height": "136984", + "completion_time": "2021-11-08T05:38:47.505593891Z", + "initial_balance": "400000000", + "balance": "400000000" + }, + { + "creation_height": "137005", + "completion_time": "2021-11-08T05:40:53.526196312Z", + "initial_balance": "385000000", + "balance": "385000000" + } + ] + } +} +``` + +#### DelegatorDelegations + +The `DelegatorDelegations` endpoint queries all delegations of a given delegator address. + +```bash +cosmos.staking.v1beta1.Query/DelegatorDelegations +``` + +Example: + +```bash +grpcurl -plaintext \ +-d '{"delegator_addr": "cosmos1y8nyfvmqh50p6ldpzljk3yrglppdv3t8phju77"}' \ +localhost:9090 cosmos.staking.v1beta1.Query/DelegatorDelegations +``` + +Example Output: + +```bash +{ + "delegation_responses": [ + {"delegation":{"delegator_address":"cosmos1y8nyfvmqh50p6ldpzljk3yrglppdv3t8phju77","validator_address":"cosmosvaloper1eh5mwu044gd5ntkkc2xgfg8247mgc56fww3vc8","shares":"25083339023.000000000000000000"},"balance":{"denom":"stake","amount":"25083339023"}} + ], + "pagination": { + "next_key": null, + "total": "1" + } +} +``` + +#### DelegatorUnbondingDelegations + +The `DelegatorUnbondingDelegations` endpoint queries all unbonding delegations of a given delegator address. + +```bash +cosmos.staking.v1beta1.Query/DelegatorUnbondingDelegations +``` + +Example: + +```bash +grpcurl -plaintext \ +-d '{"delegator_addr": "cosmos1y8nyfvmqh50p6ldpzljk3yrglppdv3t8phju77"}' \ +localhost:9090 cosmos.staking.v1beta1.Query/DelegatorUnbondingDelegations +``` + +Example Output: + +```bash expandable +{ + "unbonding_responses": [ + { + "delegator_address": "cosmos1y8nyfvmqh50p6ldpzljk3yrglppdv3t8phju77", + "validator_address": "cosmosvaloper1sjllsnramtg3ewxqwwrwjxfgc4n4ef9uxyejze", + "entries": [ + { + "creation_height": "136984", + "completion_time": "2021-11-08T05:38:47.505593891Z", + "initial_balance": "400000000", + "balance": "400000000" + }, + { + "creation_height": "137005", + "completion_time": "2021-11-08T05:40:53.526196312Z", + "initial_balance": "385000000", + "balance": "385000000" + } + ] + } + ], + "pagination": { + "next_key": null, + "total": "1" + } +} +``` + +#### Redelegations + +The `Redelegations` endpoint queries redelegations of given address. + +```bash +cosmos.staking.v1beta1.Query/Redelegations +``` + +Example: + +```bash +grpcurl -plaintext \ +-d '{"delegator_addr": "cosmos1ld5p7hn43yuh8ht28gm9pfjgj2fctujp2tgwvf", "src_validator_addr" : "cosmosvaloper1j7euyj85fv2jugejrktj540emh9353ltgppc3g", "dst_validator_addr" : "cosmosvaloper1yy3tnegzmkdcm7czzcy3flw5z0zyr9vkkxrfse"}' \ +localhost:9090 cosmos.staking.v1beta1.Query/Redelegations +``` + +Example Output: + +```bash expandable +{ + "redelegation_responses": [ + { + "redelegation": { + "delegator_address": "cosmos1ld5p7hn43yuh8ht28gm9pfjgj2fctujp2tgwvf", + "validator_src_address": "cosmosvaloper1j7euyj85fv2jugejrktj540emh9353ltgppc3g", + "validator_dst_address": "cosmosvaloper1yy3tnegzmkdcm7czzcy3flw5z0zyr9vkkxrfse", + "entries": null + }, + "entries": [ + { + "redelegation_entry": { + "creation_height": 135932, + "completion_time": "2021-11-08T03:52:55.299147901Z", + "initial_balance": "2900000", + "shares_dst": "2900000.000000000000000000" + }, + "balance": "2900000" + } + ] + } + ], + "pagination": null +} +``` + +#### DelegatorValidators + +The `DelegatorValidators` endpoint queries all validators information for given delegator. + +```bash +cosmos.staking.v1beta1.Query/DelegatorValidators +``` + +Example: + +```bash +grpcurl -plaintext \ +-d '{"delegator_addr": "cosmos1ld5p7hn43yuh8ht28gm9pfjgj2fctujp2tgwvf"}' \ +localhost:9090 cosmos.staking.v1beta1.Query/DelegatorValidators +``` + +Example Output: + +```bash expandable +{ + "validators": [ + { + "operator_address": "cosmosvaloper1eh5mwu044gd5ntkkc2xgfg8247mgc56fww3vc8", + "consensus_pubkey": { + "@type": "/cosmos.crypto.ed25519.PubKey", + "key": "UPwHWxH1zHJWGOa/m6JB3f5YjHMvPQPkVbDqqi+U7Uw=" + }, + "jailed": false, + "status": "BOND_STATUS_BONDED", + "tokens": "347260647559", + "delegator_shares": "347260647559.000000000000000000", + "description": { + "moniker": "BouBouNode", + "identity": "", + "website": "https://boubounode.com", + "security_contact": "", + "details": "AI-based Validator. #1 AI Validator on Game of Stakes. Fairly priced. Don't trust (humans), verify. Made with BouBou love." + }, + "unbonding_height": "0", + "unbonding_time": "1970-01-01T00:00:00Z", + "commission": { + "commission_rates": { + "rate": "0.061000000000000000", + "max_rate": "0.300000000000000000", + "max_change_rate": "0.150000000000000000" + }, + "update_time": "2021-10-01T15:00:00Z" + }, + "min_self_delegation": "1" + } + ], + "pagination": { + "next_key": null, + "total": "1" + } +} +``` + +#### DelegatorValidator + +The `DelegatorValidator` endpoint queries validator information for given delegator validator + +```bash +cosmos.staking.v1beta1.Query/DelegatorValidator +``` + +Example: + +```bash +grpcurl -plaintext \ +-d '{"delegator_addr": "cosmos1eh5mwu044gd5ntkkc2xgfg8247mgc56f3n8rr7", "validator_addr": "cosmosvaloper1eh5mwu044gd5ntkkc2xgfg8247mgc56fww3vc8"}' \ +localhost:9090 cosmos.staking.v1beta1.Query/DelegatorValidator +``` + +Example Output: + +```bash expandable +{ + "validator": { + "operator_address": "cosmosvaloper1eh5mwu044gd5ntkkc2xgfg8247mgc56fww3vc8", + "consensus_pubkey": { + "@type": "/cosmos.crypto.ed25519.PubKey", + "key": "UPwHWxH1zHJWGOa/m6JB3f5YjHMvPQPkVbDqqi+U7Uw=" + }, + "jailed": false, + "status": "BOND_STATUS_BONDED", + "tokens": "347262754841", + "delegator_shares": "347262754841.000000000000000000", + "description": { + "moniker": "BouBouNode", + "identity": "", + "website": "https://boubounode.com", + "security_contact": "", + "details": "AI-based Validator. #1 AI Validator on Game of Stakes. Fairly priced. Don't trust (humans), verify. Made with BouBou love." + }, + "unbonding_height": "0", + "unbonding_time": "1970-01-01T00:00:00Z", + "commission": { + "commission_rates": { + "rate": "0.061000000000000000", + "max_rate": "0.300000000000000000", + "max_change_rate": "0.150000000000000000" + }, + "update_time": "2021-10-01T15:00:00Z" + }, + "min_self_delegation": "1" + } +} +``` + +#### HistoricalInfo + +```bash +cosmos.staking.v1beta1.Query/HistoricalInfo +``` + +Example: + +```bash +grpcurl -plaintext -d '{"height" : 1}' localhost:9090 cosmos.staking.v1beta1.Query/HistoricalInfo +``` + +Example Output: + +```bash expandable +{ + "hist": { + "header": { + "version": { + "block": "11", + "app": "0" + }, + "chain_id": "simd-1", + "height": "140142", + "time": "2021-10-11T10:56:29.720079569Z", + "last_block_id": { + "hash": "9gri/4LLJUBFqioQ3NzZIP9/7YHR9QqaM6B2aJNQA7o=", + "part_set_header": { + "total": 1, + "hash": "Hk1+C864uQkl9+I6Zn7IurBZBKUevqlVtU7VqaZl1tc=" + } + }, + "last_commit_hash": "VxrcS27GtvGruS3I9+AlpT7udxIT1F0OrRklrVFSSKc=", + "data_hash": "80BjOrqNYUOkTnmgWyz9AQ8n7SoEmPVi4QmAe8RbQBY=", + "validators_hash": "95W49n2hw8RWpr1GPTAO5MSPi6w6Wjr3JjjS7AjpBho=", + "next_validators_hash": "95W49n2hw8RWpr1GPTAO5MSPi6w6Wjr3JjjS7AjpBho=", + "consensus_hash": "BICRvH3cKD93v7+R1zxE2ljD34qcvIZ0Bdi389qtoi8=", + "app_hash": "ZZaxnSY3E6Ex5Bvkm+RigYCK82g8SSUL53NymPITeOE=", + "last_results_hash": "47DEQpj8HBSa+/TImW+5JCeuQeRkm5NMpJWZG3hSuFU=", + "evidence_hash": "47DEQpj8HBSa+/TImW+5JCeuQeRkm5NMpJWZG3hSuFU=", + "proposer_address": "aH6dO428B+ItuoqPq70efFHrSMY=" + }, + "valset": [ + { + "operator_address": "cosmosvaloper196ax4vc0lwpxndu9dyhvca7jhxp70rmcqcnylw", + "consensus_pubkey": { + "@type": "/cosmos.crypto.ed25519.PubKey", + "key": "/O7BtNW0pafwfvomgR4ZnfldwPXiFfJs9mHg3gwfv5Q=" + }, + "jailed": false, + "status": "BOND_STATUS_BONDED", + "tokens": "1426045203613", + "delegator_shares": "1426045203613.000000000000000000", + "description": { + "moniker": "SG-1", + "identity": "48608633F99D1B60", + "website": "https://sg-1.online", + "security_contact": "", + "details": "SG-1 - your favorite validator on Witval. We offer 100% Soft Slash protection." + }, + "unbonding_height": "0", + "unbonding_time": "1970-01-01T00:00:00Z", + "commission": { + "commission_rates": { + "rate": "0.037500000000000000", + "max_rate": "0.200000000000000000", + "max_change_rate": "0.030000000000000000" + }, + "update_time": "2021-10-01T15:00:00Z" + }, + "min_self_delegation": "1" + } + ] + } +} + +``` + +#### Pool + +The `Pool` endpoint queries the pool information. + +```bash +cosmos.staking.v1beta1.Query/Pool +``` + +Example: + +```bash +grpcurl -plaintext -d localhost:9090 cosmos.staking.v1beta1.Query/Pool +``` + +Example Output: + +```bash +{ + "pool": { + "not_bonded_tokens": "369054400189", + "bonded_tokens": "15657192425623" + } +} +``` + +#### Params + +The `Params` endpoint queries the pool information. + +```bash +cosmos.staking.v1beta1.Query/Params +``` + +Example: + +```bash +grpcurl -plaintext localhost:9090 cosmos.staking.v1beta1.Query/Params +``` + +Example Output: + +```bash +{ + "params": { + "unbondingTime": "1814400s", + "maxValidators": 100, + "maxEntries": 7, + "historicalEntries": 10000, + "bondDenom": "stake" + } +} +``` + +### REST + +A user can query the `staking` module using REST endpoints. + +#### DelegatorDelegations + +The `DelegtaorDelegations` REST endpoint queries all delegations of a given delegator address. + +```bash +/cosmos/staking/v1beta1/delegations/{delegatorAddr} +``` + +Example: + +```bash +curl -X GET "http://localhost:1317/cosmos/staking/v1beta1/delegations/cosmos1vcs68xf2tnqes5tg0khr0vyevm40ff6zdxatp5" -H "accept: application/json" +``` + +Example Output: + +```bash expandable +{ + "delegation_responses": [ + { + "delegation": { + "delegator_address": "cosmos1vcs68xf2tnqes5tg0khr0vyevm40ff6zdxatp5", + "validator_address": "cosmosvaloper1quqxfrxkycr0uzt4yk0d57tcq3zk7srm7sm6r8", + "shares": "256250000.000000000000000000" + }, + "balance": { + "denom": "stake", + "amount": "256250000" + } + }, + { + "delegation": { + "delegator_address": "cosmos1vcs68xf2tnqes5tg0khr0vyevm40ff6zdxatp5", + "validator_address": "cosmosvaloper194v8uwee2fvs2s8fa5k7j03ktwc87h5ym39jfv", + "shares": "255150000.000000000000000000" + }, + "balance": { + "denom": "stake", + "amount": "255150000" + } + } + ], + "pagination": { + "next_key": null, + "total": "2" + } +} +``` + +#### Redelegations + +The `Redelegations` REST endpoint queries redelegations of given address. + +```bash +/cosmos/staking/v1beta1/delegators/{delegatorAddr}/redelegations +``` + +Example: + +```bash +curl -X GET \ +"http://localhost:1317/cosmos/staking/v1beta1/delegators/cosmos1thfntksw0d35n2tkr0k8v54fr8wxtxwxl2c56e/redelegations?srcValidatorAddr=cosmosvaloper1lzhlnpahvznwfv4jmay2tgaha5kmz5qx4cuznf&dstValidatorAddr=cosmosvaloper1vq8tw77kp8lvxq9u3c8eeln9zymn68rng8pgt4" \ +-H "accept: application/json" +``` + +Example Output: + +```bash expandable +{ + "redelegation_responses": [ + { + "redelegation": { + "delegator_address": "cosmos1thfntksw0d35n2tkr0k8v54fr8wxtxwxl2c56e", + "validator_src_address": "cosmosvaloper1lzhlnpahvznwfv4jmay2tgaha5kmz5qx4cuznf", + "validator_dst_address": "cosmosvaloper1vq8tw77kp8lvxq9u3c8eeln9zymn68rng8pgt4", + "entries": null + }, + "entries": [ + { + "redelegation_entry": { + "creation_height": 151523, + "completion_time": "2021-11-09T06:03:25.640682116Z", + "initial_balance": "200000000", + "shares_dst": "200000000.000000000000000000" + }, + "balance": "200000000" + } + ] + } + ], + "pagination": null +} +``` + +#### DelegatorUnbondingDelegations + +The `DelegatorUnbondingDelegations` REST endpoint queries all unbonding delegations of a given delegator address. + +```bash +/cosmos/staking/v1beta1/delegators/{delegatorAddr}/unbonding_delegations +``` + +Example: + +```bash +curl -X GET \ +"http://localhost:1317/cosmos/staking/v1beta1/delegators/cosmos1nxv42u3lv642q0fuzu2qmrku27zgut3n3z7lll/unbonding_delegations" \ +-H "accept: application/json" +``` + +Example Output: + +```bash expandable +{ + "unbonding_responses": [ + { + "delegator_address": "cosmos1nxv42u3lv642q0fuzu2qmrku27zgut3n3z7lll", + "validator_address": "cosmosvaloper1e7mvqlz50ch6gw4yjfemsc069wfre4qwmw53kq", + "entries": [ + { + "creation_height": "2442278", + "completion_time": "2021-10-12T10:59:03.797335857Z", + "initial_balance": "50000000000", + "balance": "50000000000" + } + ] + } + ], + "pagination": { + "next_key": null, + "total": "1" + } +} +``` + +#### DelegatorValidators + +The `DelegatorValidators` REST endpoint queries all validators information for given delegator address. + +```bash +/cosmos/staking/v1beta1/delegators/{delegatorAddr}/validators +``` + +Example: + +```bash +curl -X GET \ +"http://localhost:1317/cosmos/staking/v1beta1/delegators/cosmos1xwazl8ftks4gn00y5x3c47auquc62ssune9ppv/validators" \ +-H "accept: application/json" +``` + +Example Output: + +```bash expandable +{ + "validators": [ + { + "operator_address": "cosmosvaloper1xwazl8ftks4gn00y5x3c47auquc62ssuvynw64", + "consensus_pubkey": { + "@type": "/cosmos.crypto.ed25519.PubKey", + "key": "5v4n3px3PkfNnKflSgepDnsMQR1hiNXnqOC11Y72/PQ=" + }, + "jailed": false, + "status": "BOND_STATUS_BONDED", + "tokens": "21592843799", + "delegator_shares": "21592843799.000000000000000000", + "description": { + "moniker": "jabbey", + "identity": "", + "website": "https://twitter.com/JoeAbbey", + "security_contact": "", + "details": "just another dad in the cosmos" + }, + "unbonding_height": "0", + "unbonding_time": "1970-01-01T00:00:00Z", + "commission": { + "commission_rates": { + "rate": "0.100000000000000000", + "max_rate": "0.200000000000000000", + "max_change_rate": "0.100000000000000000" + }, + "update_time": "2021-10-09T19:03:54.984821705Z" + }, + "min_self_delegation": "1" + } + ], + "pagination": { + "next_key": null, + "total": "1" + } +} +``` + +#### DelegatorValidator + +The `DelegatorValidator` REST endpoint queries validator information for given delegator validator pair. + +```bash +/cosmos/staking/v1beta1/delegators/{delegatorAddr}/validators/{validatorAddr} +``` + +Example: + +```bash +curl -X GET \ +"http://localhost:1317/cosmos/staking/v1beta1/delegators/cosmos1xwazl8ftks4gn00y5x3c47auquc62ssune9ppv/validators/cosmosvaloper1xwazl8ftks4gn00y5x3c47auquc62ssuvynw64" \ +-H "accept: application/json" +``` + +Example Output: + +```bash expandable +{ + "validator": { + "operator_address": "cosmosvaloper1xwazl8ftks4gn00y5x3c47auquc62ssuvynw64", + "consensus_pubkey": { + "@type": "/cosmos.crypto.ed25519.PubKey", + "key": "5v4n3px3PkfNnKflSgepDnsMQR1hiNXnqOC11Y72/PQ=" + }, + "jailed": false, + "status": "BOND_STATUS_BONDED", + "tokens": "21592843799", + "delegator_shares": "21592843799.000000000000000000", + "description": { + "moniker": "jabbey", + "identity": "", + "website": "https://twitter.com/JoeAbbey", + "security_contact": "", + "details": "just another dad in the cosmos" + }, + "unbonding_height": "0", + "unbonding_time": "1970-01-01T00:00:00Z", + "commission": { + "commission_rates": { + "rate": "0.100000000000000000", + "max_rate": "0.200000000000000000", + "max_change_rate": "0.100000000000000000" + }, + "update_time": "2021-10-09T19:03:54.984821705Z" + }, + "min_self_delegation": "1" + } +} +``` + +#### HistoricalInfo + +The `HistoricalInfo` REST endpoint queries the historical information for given height. + +```bash +/cosmos/staking/v1beta1/historical_info/{height} +``` + +Example: + +```bash +curl -X GET "http://localhost:1317/cosmos/staking/v1beta1/historical_info/153332" -H "accept: application/json" +``` + +Example Output: + +```bash expandable +{ + "hist": { + "header": { + "version": { + "block": "11", + "app": "0" + }, + "chain_id": "cosmos-1", + "height": "153332", + "time": "2021-10-12T09:05:35.062230221Z", + "last_block_id": { + "hash": "NX8HevR5khb7H6NGKva+jVz7cyf0skF1CrcY9A0s+d8=", + "part_set_header": { + "total": 1, + "hash": "zLQ2FiKM5tooL3BInt+VVfgzjlBXfq0Hc8Iux/xrhdg=" + } + }, + "last_commit_hash": "P6IJrK8vSqU3dGEyRHnAFocoDGja0bn9euLuy09s350=", + "data_hash": "eUd+6acHWrNXYju8Js449RJ99lOYOs16KpqQl4SMrEM=", + "validators_hash": "mB4pravvMsJKgi+g8aYdSeNlt0kPjnRFyvtAQtaxcfw=", + "next_validators_hash": "mB4pravvMsJKgi+g8aYdSeNlt0kPjnRFyvtAQtaxcfw=", + "consensus_hash": "BICRvH3cKD93v7+R1zxE2ljD34qcvIZ0Bdi389qtoi8=", + "app_hash": "fuELArKRK+CptnZ8tu54h6xEleSWenHNmqC84W866fU=", + "last_results_hash": "p/BPexV4LxAzlVcPRvW+lomgXb6Yze8YLIQUo/4Kdgc=", + "evidence_hash": "47DEQpj8HBSa+/TImW+5JCeuQeRkm5NMpJWZG3hSuFU=", + "proposer_address": "G0MeY8xQx7ooOsni8KE/3R/Ib3Q=" + }, + "valset": [ + { + "operator_address": "cosmosvaloper196ax4vc0lwpxndu9dyhvca7jhxp70rmcqcnylw", + "consensus_pubkey": { + "@type": "/cosmos.crypto.ed25519.PubKey", + "key": "/O7BtNW0pafwfvomgR4ZnfldwPXiFfJs9mHg3gwfv5Q=" + }, + "jailed": false, + "status": "BOND_STATUS_BONDED", + "tokens": "1416521659632", + "delegator_shares": "1416521659632.000000000000000000", + "description": { + "moniker": "SG-1", + "identity": "48608633F99D1B60", + "website": "https://sg-1.online", + "security_contact": "", + "details": "SG-1 - your favorite validator on cosmos. We offer 100% Soft Slash protection." + }, + "unbonding_height": "0", + "unbonding_time": "1970-01-01T00:00:00Z", + "commission": { + "commission_rates": { + "rate": "0.037500000000000000", + "max_rate": "0.200000000000000000", + "max_change_rate": "0.030000000000000000" + }, + "update_time": "2021-10-01T15:00:00Z" + }, + "min_self_delegation": "1" + }, + { + "operator_address": "cosmosvaloper1t8ehvswxjfn3ejzkjtntcyrqwvmvuknzmvtaaa", + "consensus_pubkey": { + "@type": "/cosmos.crypto.ed25519.PubKey", + "key": "uExZyjNLtr2+FFIhNDAMcQ8+yTrqE7ygYTsI7khkA5Y=" + }, + "jailed": false, + "status": "BOND_STATUS_BONDED", + "tokens": "1348298958808", + "delegator_shares": "1348298958808.000000000000000000", + "description": { + "moniker": "Cosmostation", + "identity": "AE4C403A6E7AA1AC", + "website": "https://www.cosmostation.io", + "security_contact": "admin@stamper.network", + "details": "Cosmostation validator node. Delegate your tokens and Start Earning Staking Rewards" + }, + "unbonding_height": "0", + "unbonding_time": "1970-01-01T00:00:00Z", + "commission": { + "commission_rates": { + "rate": "0.050000000000000000", + "max_rate": "1.000000000000000000", + "max_change_rate": "0.200000000000000000" + }, + "update_time": "2021-10-01T15:06:38.821314287Z" + }, + "min_self_delegation": "1" + } + ] + } +} +``` + +#### Parameters + +The `Parameters` REST endpoint queries the staking parameters. + +```bash +/cosmos/staking/v1beta1/params +``` + +Example: + +```bash +curl -X GET "http://localhost:1317/cosmos/staking/v1beta1/params" -H "accept: application/json" +``` + +Example Output: + +```bash +{ + "params": { + "unbonding_time": "2419200s", + "max_validators": 100, + "max_entries": 7, + "historical_entries": 10000, + "bond_denom": "stake" + } +} +``` + +#### Pool + +The `Pool` REST endpoint queries the pool information. + +```bash +/cosmos/staking/v1beta1/pool +``` + +Example: + +```bash +curl -X GET "http://localhost:1317/cosmos/staking/v1beta1/pool" -H "accept: application/json" +``` + +Example Output: + +```bash +{ + "pool": { + "not_bonded_tokens": "432805737458", + "bonded_tokens": "15783637712645" + } +} +``` + +#### Validators + +The `Validators` REST endpoint queries all validators that match the given status. + +```bash +/cosmos/staking/v1beta1/validators +``` + +Example: + +```bash +curl -X GET "http://localhost:1317/cosmos/staking/v1beta1/validators" -H "accept: application/json" +``` + +Example Output: + +```bash expandable +{ + "validators": [ + { + "operator_address": "cosmosvaloper1q3jsx9dpfhtyqqgetwpe5tmk8f0ms5qywje8tw", + "consensus_pubkey": { + "@type": "/cosmos.crypto.ed25519.PubKey", + "key": "N7BPyek2aKuNZ0N/8YsrqSDhGZmgVaYUBuddY8pwKaE=" + }, + "jailed": false, + "status": "BOND_STATUS_BONDED", + "tokens": "383301887799", + "delegator_shares": "383301887799.000000000000000000", + "description": { + "moniker": "SmartNodes", + "identity": "D372724899D1EDC8", + "website": "https://smartnodes.co", + "security_contact": "", + "details": "Earn Rewards with Crypto Staking & Node Deployment" + }, + "unbonding_height": "0", + "unbonding_time": "1970-01-01T00:00:00Z", + "commission": { + "commission_rates": { + "rate": "0.050000000000000000", + "max_rate": "0.200000000000000000", + "max_change_rate": "0.100000000000000000" + }, + "update_time": "2021-10-01T15:51:31.596618510Z" + }, + "min_self_delegation": "1" + }, + { + "operator_address": "cosmosvaloper1q5ku90atkhktze83j9xjaks2p7uruag5zp6wt7", + "consensus_pubkey": { + "@type": "/cosmos.crypto.ed25519.PubKey", + "key": "GDNpuKDmCg9GnhnsiU4fCWktuGUemjNfvpCZiqoRIYA=" + }, + "jailed": false, + "status": "BOND_STATUS_UNBONDING", + "tokens": "1017819654", + "delegator_shares": "1017819654.000000000000000000", + "description": { + "moniker": "Noderunners", + "identity": "812E82D12FEA3493", + "website": "http://noderunners.biz", + "security_contact": "info@noderunners.biz", + "details": "Noderunners is a professional validator in POS networks. We have a huge node running experience, reliable soft and hardware. Our commissions are always low, our support to delegators is always full. Stake with us and start receiving your cosmos rewards now!" + }, + "unbonding_height": "147302", + "unbonding_time": "2021-11-08T22:58:53.718662452Z", + "commission": { + "commission_rates": { + "rate": "0.050000000000000000", + "max_rate": "0.200000000000000000", + "max_change_rate": "0.100000000000000000" + }, + "update_time": "2021-10-04T18:02:21.446645619Z" + }, + "min_self_delegation": "1" + } + ], + "pagination": { + "next_key": "FONDBFkE4tEEf7yxWWKOD49jC2NK", + "total": "2" + } +} +``` + +#### Validator + +The `Validator` REST endpoint queries validator information for given validator address. + +```bash +/cosmos/staking/v1beta1/validators/{validatorAddr} +``` + +Example: + +```bash +curl -X GET \ +"http://localhost:1317/cosmos/staking/v1beta1/validators/cosmosvaloper16msryt3fqlxtvsy8u5ay7wv2p8mglfg9g70e3q" \ +-H "accept: application/json" +``` + +Example Output: + +```bash expandable +{ + "validator": { + "operator_address": "cosmosvaloper16msryt3fqlxtvsy8u5ay7wv2p8mglfg9g70e3q", + "consensus_pubkey": { + "@type": "/cosmos.crypto.ed25519.PubKey", + "key": "sIiexdJdYWn27+7iUHQJDnkp63gq/rzUq1Y+fxoGjXc=" + }, + "jailed": false, + "status": "BOND_STATUS_BONDED", + "tokens": "33027900000", + "delegator_shares": "33027900000.000000000000000000", + "description": { + "moniker": "Witval", + "identity": "51468B615127273A", + "website": "", + "security_contact": "", + "details": "Witval is the validator arm from Vitwit. Vitwit is into software consulting and services business since 2015. We are working closely with Cosmos ecosystem since 2018. We are also building tools for the ecosystem, Aneka is our explorer for the cosmos ecosystem." + }, + "unbonding_height": "0", + "unbonding_time": "1970-01-01T00:00:00Z", + "commission": { + "commission_rates": { + "rate": "0.050000000000000000", + "max_rate": "0.200000000000000000", + "max_change_rate": "0.020000000000000000" + }, + "update_time": "2021-10-01T19:24:52.663191049Z" + }, + "min_self_delegation": "1" + } +} +``` + +#### ValidatorDelegations + +The `ValidatorDelegations` REST endpoint queries delegate information for given validator. + +```bash +/cosmos/staking/v1beta1/validators/{validatorAddr}/delegations +``` + +Example: + +```bash +curl -X GET "http://localhost:1317/cosmos/staking/v1beta1/validators/cosmosvaloper16msryt3fqlxtvsy8u5ay7wv2p8mglfg9g70e3q/delegations" -H "accept: application/json" +``` + +Example Output: + +```bash expandable +{ + "delegation_responses": [ + { + "delegation": { + "delegator_address": "cosmos190g5j8aszqhvtg7cprmev8xcxs6csra7xnk3n3", + "validator_address": "cosmosvaloper16msryt3fqlxtvsy8u5ay7wv2p8mglfg9g70e3q", + "shares": "31000000000.000000000000000000" + }, + "balance": { + "denom": "stake", + "amount": "31000000000" + } + }, + { + "delegation": { + "delegator_address": "cosmos1ddle9tczl87gsvmeva3c48nenyng4n56qwq4ee", + "validator_address": "cosmosvaloper16msryt3fqlxtvsy8u5ay7wv2p8mglfg9g70e3q", + "shares": "628470000.000000000000000000" + }, + "balance": { + "denom": "stake", + "amount": "628470000" + } + }, + { + "delegation": { + "delegator_address": "cosmos10fdvkczl76m040smd33lh9xn9j0cf26kk4s2nw", + "validator_address": "cosmosvaloper16msryt3fqlxtvsy8u5ay7wv2p8mglfg9g70e3q", + "shares": "838120000.000000000000000000" + }, + "balance": { + "denom": "stake", + "amount": "838120000" + } + }, + { + "delegation": { + "delegator_address": "cosmos1n8f5fknsv2yt7a8u6nrx30zqy7lu9jfm0t5lq8", + "validator_address": "cosmosvaloper16msryt3fqlxtvsy8u5ay7wv2p8mglfg9g70e3q", + "shares": "500000000.000000000000000000" + }, + "balance": { + "denom": "stake", + "amount": "500000000" + } + }, + { + "delegation": { + "delegator_address": "cosmos16msryt3fqlxtvsy8u5ay7wv2p8mglfg9hrek2e", + "validator_address": "cosmosvaloper16msryt3fqlxtvsy8u5ay7wv2p8mglfg9g70e3q", + "shares": "61310000.000000000000000000" + }, + "balance": { + "denom": "stake", + "amount": "61310000" + } + } + ], + "pagination": { + "next_key": null, + "total": "5" + } +} +``` + +#### Delegation + +The `Delegation` REST endpoint queries delegate information for given validator delegator pair. + +```bash +/cosmos/staking/v1beta1/validators/{validatorAddr}/delegations/{delegatorAddr} +``` + +Example: + +```bash +curl -X GET \ +"http://localhost:1317/cosmos/staking/v1beta1/validators/cosmosvaloper16msryt3fqlxtvsy8u5ay7wv2p8mglfg9g70e3q/delegations/cosmos1n8f5fknsv2yt7a8u6nrx30zqy7lu9jfm0t5lq8" \ +-H "accept: application/json" +``` + +Example Output: + +```bash expandable +{ + "delegation_response": { + "delegation": { + "delegator_address": "cosmos1n8f5fknsv2yt7a8u6nrx30zqy7lu9jfm0t5lq8", + "validator_address": "cosmosvaloper16msryt3fqlxtvsy8u5ay7wv2p8mglfg9g70e3q", + "shares": "500000000.000000000000000000" + }, + "balance": { + "denom": "stake", + "amount": "500000000" + } + } +} +``` + +#### UnbondingDelegation + +The `UnbondingDelegation` REST endpoint queries unbonding information for given validator delegator pair. + +```bash +/cosmos/staking/v1beta1/validators/{validatorAddr}/delegations/{delegatorAddr}/unbonding_delegation +``` + +Example: + +```bash +curl -X GET \ +"http://localhost:1317/cosmos/staking/v1beta1/validators/cosmosvaloper13v4spsah85ps4vtrw07vzea37gq5la5gktlkeu/delegations/cosmos1ze2ye5u5k3qdlexvt2e0nn0508p04094ya0qpm/unbonding_delegation" \ +-H "accept: application/json" +``` + +Example Output: + +```bash expandable +{ + "unbond": { + "delegator_address": "cosmos1ze2ye5u5k3qdlexvt2e0nn0508p04094ya0qpm", + "validator_address": "cosmosvaloper13v4spsah85ps4vtrw07vzea37gq5la5gktlkeu", + "entries": [ + { + "creation_height": "153687", + "completion_time": "2021-11-09T09:41:18.352401903Z", + "initial_balance": "525111", + "balance": "525111" + } + ] + } +} +``` + +#### ValidatorUnbondingDelegations + +The `ValidatorUnbondingDelegations` REST endpoint queries unbonding delegations of a validator. + +```bash +/cosmos/staking/v1beta1/validators/{validatorAddr}/unbonding_delegations +``` + +Example: + +```bash +curl -X GET \ +"http://localhost:1317/cosmos/staking/v1beta1/validators/cosmosvaloper13v4spsah85ps4vtrw07vzea37gq5la5gktlkeu/unbonding_delegations" \ +-H "accept: application/json" +``` + +Example Output: + +```bash expandable +{ + "unbonding_responses": [ + { + "delegator_address": "cosmos1q9snn84jfrd9ge8t46kdcggpe58dua82vnj7uy", + "validator_address": "cosmosvaloper13v4spsah85ps4vtrw07vzea37gq5la5gktlkeu", + "entries": [ + { + "creation_height": "90998", + "completion_time": "2021-11-05T00:14:37.005841058Z", + "initial_balance": "24000000", + "balance": "24000000" + } + ] + }, + { + "delegator_address": "cosmos1qf36e6wmq9h4twhdvs6pyq9qcaeu7ye0s3dqq2", + "validator_address": "cosmosvaloper13v4spsah85ps4vtrw07vzea37gq5la5gktlkeu", + "entries": [ + { + "creation_height": "47478", + "completion_time": "2021-11-01T22:47:26.714116854Z", + "initial_balance": "8000000", + "balance": "8000000" + } + ] + } + ], + "pagination": { + "next_key": null, + "total": "2" + } +} +``` diff --git a/sdk/v0.54/modules/upgrade/README.mdx b/sdk/v0.54/modules/upgrade/README.mdx new file mode 100644 index 000000000..3ba52fa78 --- /dev/null +++ b/sdk/v0.54/modules/upgrade/README.mdx @@ -0,0 +1,611 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/modules/upgrade/README' +title: 'x/upgrade' +--- + +## Abstract + +`x/upgrade` is an implementation of a Cosmos SDK module that facilitates smoothly +upgrading a live Cosmos chain to a new (breaking) software version. It accomplishes this by +providing a `PreBlocker` hook that prevents the blockchain state machine from +proceeding once a pre-defined upgrade block height has been reached. + +The module does not prescribe anything regarding how governance decides to do an +upgrade, but just the mechanism for coordinating the upgrade safely. Without software +support for upgrades, upgrading a live chain is risky because all of the validators +need to pause their state machines at exactly the same point in the process. If +this is not done correctly, there can be state inconsistencies which are hard to +recover from. + +* [Concepts](#concepts) +* [State](#state) +* [Events](#events) +* [Client](#client) + * [CLI](#cli) + * [REST](#rest) + * [gRPC](#grpc) +* [Resources](#resources) + +## Concepts + +### Plan + +The `x/upgrade` module defines a `Plan` type in which a live upgrade is scheduled +to occur. A `Plan` can be scheduled at a specific block height. +A `Plan` is created once a (frozen) release candidate along with an appropriate upgrade +`Handler` (see below) is agreed upon, where the `Name` of a `Plan` corresponds to a +specific `Handler`. Typically, a `Plan` is created through a governance proposal +process, where if voted upon and passed, will be scheduled. The `Info` of a `Plan` +may contain various metadata about the upgrade, typically application specific +upgrade info to be included on-chain such as a git commit that validators could +automatically upgrade to. + +```go +type Plan struct { + Name string + Height int64 + Info string +} +``` + +#### Sidecar Process + +If an operator running the application binary also runs a sidecar process to assist +in the automatic download and upgrade of a binary, the `Info` allows this process to +be seamless. This tool is [Cosmovisor](https://github.com/cosmos/cosmos-sdk/tree/main/tools/cosmovisor#readme). + +### Handler + +The `x/upgrade` module facilitates upgrading from major version X to major version Y. To +accomplish this, node operators must first upgrade their current binary to a new +binary that has a corresponding `Handler` for the new version Y. It is assumed that +this version has fully been tested and approved by the community at large. This +`Handler` defines what state migrations need to occur before the new binary Y +can successfully run the chain. Naturally, this `Handler` is application specific +and not defined on a per-module basis. Registering a `Handler` is done via +`Keeper#SetUpgradeHandler` in the application. + +```go +type UpgradeHandler func(Context, Plan, VersionMap) (VersionMap, error) +``` + +During each `EndBlock` execution, the `x/upgrade` module checks if there exists a +`Plan` that should execute (is scheduled at that height). If so, the corresponding +`Handler` is executed. If the `Plan` is expected to execute but no `Handler` is registered +or if the binary was upgraded too early, the node will gracefully panic and exit. + +### StoreLoader + +The `x/upgrade` module also facilitates store migrations as part of the upgrade. The +`StoreLoader` sets the migrations that need to occur before the new binary can +successfully run the chain. This `StoreLoader` is also application specific and +not defined on a per-module basis. Registering this `StoreLoader` is done via +`app#SetStoreLoader` in the application. + +```go +func UpgradeStoreLoader (upgradeHeight int64, storeUpgrades *store.StoreUpgrades) + +baseapp.StoreLoader +``` + +If there's a planned upgrade and the upgrade height is reached, the old binary writes `Plan` to the disk before panicking. + +This information is critical to ensure the `StoreUpgrades` happens smoothly at the correct height and +expected upgrade. It eliminates the chances for the new binary to execute `StoreUpgrades` multiple +times every time on restart. Also, if there are multiple upgrades planned on the same height, the `Name` +will ensure these `StoreUpgrades` take place only in the planned upgrade handler. + +### Proposal + +Typically, a `Plan` is proposed and submitted through governance via a proposal +containing a `MsgSoftwareUpgrade` message. +This proposal prescribes to the standard governance process. If the proposal passes, +the `Plan`, which targets a specific `Handler`, is persisted and scheduled. The +upgrade can be delayed or hastened by updating the `Plan.Height` in a new proposal. + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/upgrade/v1beta1/tx.proto#L29-L41 +``` + +#### Cancelling Upgrade Proposals + +Upgrade proposals can be cancelled. There exists a gov-enabled `MsgCancelUpgrade` +message type, which can be embedded in a proposal, voted on and, if passed, will +remove the scheduled upgrade `Plan`. +Of course this requires that the upgrade was known to be a bad idea well before the +upgrade itself, to allow time for a vote. + +```protobuf +// Reference: https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/proto/cosmos/upgrade/v1beta1/tx.proto#L48-L57 +``` + +If such a possibility is desired, the upgrade height is to be +`2 * (VotingPeriod + DepositPeriod) + (SafetyDelta)` from the beginning of the +upgrade proposal. The `SafetyDelta` is the time available from the success of an +upgrade proposal and the realization it was a bad idea (due to external social consensus). + +A `MsgCancelUpgrade` proposal can also be made while the original +`MsgSoftwareUpgrade` proposal is still being voted upon, as long as the `VotingPeriod` +ends after the `MsgSoftwareUpgrade` proposal. + +## State + +The internal state of the `x/upgrade` module is relatively minimal and simple. The +state contains the currently active upgrade `Plan` (if one exists) by key +`0x0` and if a `Plan` is marked as "done" by key `0x1`. The state +contains the consensus versions of all app modules in the application. The versions +are stored as big endian `uint64`, and can be accessed with prefix `0x2` appended +by the corresponding module name of type `string`. The state maintains a +`Protocol Version` which can be accessed by key `0x3`. + +* Plan: `0x0 -> Plan` +* Done: `0x1 | byte(plan name) -> BigEndian(Block Height)` +* ConsensusVersion: `0x2 | byte(module name) -> BigEndian(Module Consensus Version)` +* ProtocolVersion: `0x3 -> BigEndian(Protocol Version)` + +The `x/upgrade` module contains no genesis state. + +## Events + +The `x/upgrade` does not emit any events by itself. Any and all proposal related +events are emitted through the `x/gov` module. + +## Client + +### CLI + +A user can query and interact with the `upgrade` module using the CLI. + +#### Query + +The `query` commands allow users to query `upgrade` state. + +```bash +simd query upgrade --help +``` + +##### applied + +The `applied` command allows users to query the block header for height at which a completed upgrade was applied. + +```bash +simd query upgrade applied [upgrade-name] [flags] +``` + +If upgrade-name was previously executed on the chain, this returns the header for the block at which it was applied. +This helps a client determine which binary was valid over a given range of blocks, as well as more context to understand past migrations. + +Example: + +```bash +simd query upgrade applied "test-upgrade" +``` + +Example Output: + +```bash expandable +"block_id": { + "hash": "A769136351786B9034A5F196DC53F7E50FCEB53B48FA0786E1BFC45A0BB646B5", + "parts": { + "total": 1, + "hash": "B13CBD23011C7480E6F11BE4594EE316548648E6A666B3575409F8F16EC6939E" + } + }, + "block_size": "7213", + "header": { + "version": { + "block": "11" + }, + "chain_id": "testnet-2", + "height": "455200", + "time": "2021-04-10T04:37:57.085493838Z", + "last_block_id": { + "hash": "0E8AD9309C2DC411DF98217AF59E044A0E1CCEAE7C0338417A70338DF50F4783", + "parts": { + "total": 1, + "hash": "8FE572A48CD10BC2CBB02653CA04CA247A0F6830FF19DC972F64D339A355E77D" + } + }, + "last_commit_hash": "DE890239416A19E6164C2076B837CC1D7F7822FC214F305616725F11D2533140", + "data_hash": "E3B0C44298FC1C149AFBF4C8996FB92427AE41E4649B934CA495991B7852B855", + "validators_hash": "A31047ADE54AE9072EE2A12FF260A8990BA4C39F903EAF5636B50D58DBA72582", + "next_validators_hash": "A31047ADE54AE9072EE2A12FF260A8990BA4C39F903EAF5636B50D58DBA72582", + "consensus_hash": "048091BC7DDC283F77BFBF91D73C44DA58C3DF8A9CBC867405D8B7F3DAADA22F", + "app_hash": "28ECC486AFC332BA6CC976706DBDE87E7D32441375E3F10FD084CD4BAF0DA021", + "last_results_hash": "E3B0C44298FC1C149AFBF4C8996FB92427AE41E4649B934CA495991B7852B855", + "evidence_hash": "E3B0C44298FC1C149AFBF4C8996FB92427AE41E4649B934CA495991B7852B855", + "proposer_address": "2ABC4854B1A1C5AA8403C4EA853A81ACA901CC76" + }, + "num_txs": "0" +} +``` + +##### module versions + +The `module_versions` command gets a list of module names and their respective consensus versions. + +Following the command with a specific module name will return only +that module's information. + +```bash +simd query upgrade module_versions [optional module_name] [flags] +``` + +Example: + +```bash +simd query upgrade module_versions +``` + +Example Output: + +```bash expandable +module_versions: +- name: auth + version: "2" +- name: authz + version: "1" +- name: bank + version: "2" +- name: distribution + version: "2" +- name: evidence + version: "1" +- name: feegrant + version: "1" +- name: genutil + version: "1" +- name: gov + version: "2" +- name: ibc + version: "2" +- name: mint + version: "1" +- name: params + version: "1" +- name: slashing + version: "2" +- name: staking + version: "2" +- name: transfer + version: "1" +- name: upgrade + version: "1" +- name: vesting + version: "1" +``` + +Example: + +```bash +regen query upgrade module_versions ibc +``` + +Example Output: + +```bash +module_versions: +- name: ibc + version: "2" +``` + +##### plan + +The `plan` command gets the currently scheduled upgrade plan, if one exists. + +```bash +regen query upgrade plan [flags] +``` + +Example: + +```bash +simd query upgrade plan +``` + +Example Output: + +```bash +height: "130" +info: "" +name: test-upgrade +time: "0001-01-01T00:00:00Z" +upgraded_client_state: null +``` + +#### Transactions + +The upgrade module supports the following transactions: + +* `software-proposal` - submits an upgrade proposal: + +```bash +simd tx upgrade software-upgrade v2 --title="Test Proposal" --summary="testing" --deposit="100000000stake" --upgrade-height 1000000 \ +--upgrade-info '{ "binaries": { "linux/amd64":"https://example.com/simd.zip?checksum=sha256:aec070645fe53ee3b3763059376134f058cc337247c978add178b6ccdfb0019f" } }' --from cosmos1.. +``` + +* `cancel-software-upgrade` - cancels a previously submitted upgrade proposal: + +```bash +simd tx upgrade cancel-software-upgrade --title="Test Proposal" --summary="testing" --deposit="100000000stake" --from cosmos1.. +``` + +### REST + +A user can query the `upgrade` module using REST endpoints. + +#### Applied Plan + +`AppliedPlan` queries a previously applied upgrade plan by its name. + +```bash +/cosmos/upgrade/v1beta1/applied_plan/{name} +``` + +Example: + +```bash +curl -X GET "http://localhost:1317/cosmos/upgrade/v1beta1/applied_plan/v2.0-upgrade" -H "accept: application/json" +``` + +Example Output: + +```bash +{ + "height": "30" +} +``` + +#### Current Plan + +`CurrentPlan` queries the current upgrade plan. + +```bash +/cosmos/upgrade/v1beta1/current_plan +``` + +Example: + +```bash +curl -X GET "http://localhost:1317/cosmos/upgrade/v1beta1/current_plan" -H "accept: application/json" +``` + +Example Output: + +```bash +{ + "plan": "v2.1-upgrade" +} +``` + +#### Module versions + +`ModuleVersions` queries the list of module versions from state. + +```bash +/cosmos/upgrade/v1beta1/module_versions +``` + +Example: + +```bash +curl -X GET "http://localhost:1317/cosmos/upgrade/v1beta1/module_versions" -H "accept: application/json" +``` + +Example Output: + +```bash expandable +{ + "module_versions": [ + { + "name": "auth", + "version": "2" + }, + { + "name": "authz", + "version": "1" + }, + { + "name": "bank", + "version": "2" + }, + { + "name": "distribution", + "version": "2" + }, + { + "name": "evidence", + "version": "1" + }, + { + "name": "feegrant", + "version": "1" + }, + { + "name": "genutil", + "version": "1" + }, + { + "name": "gov", + "version": "2" + }, + { + "name": "ibc", + "version": "2" + }, + { + "name": "mint", + "version": "1" + }, + { + "name": "params", + "version": "1" + }, + { + "name": "slashing", + "version": "2" + }, + { + "name": "staking", + "version": "2" + }, + { + "name": "transfer", + "version": "1" + }, + { + "name": "upgrade", + "version": "1" + }, + { + "name": "vesting", + "version": "1" + } + ] +} +``` + +### gRPC + +A user can query the `upgrade` module using gRPC endpoints. + +#### Applied Plan + +`AppliedPlan` queries a previously applied upgrade plan by its name. + +```bash +cosmos.upgrade.v1beta1.Query/AppliedPlan +``` + +Example: + +```bash +grpcurl -plaintext \ + -d '{"name":"v2.0-upgrade"}' \ + localhost:9090 \ + cosmos.upgrade.v1beta1.Query/AppliedPlan +``` + +Example Output: + +```bash +{ + "height": "30" +} +``` + +#### Current Plan + +`CurrentPlan` queries the current upgrade plan. + +```bash +cosmos.upgrade.v1beta1.Query/CurrentPlan +``` + +Example: + +```bash +grpcurl -plaintext localhost:9090 cosmos.slashing.v1beta1.Query/CurrentPlan +``` + +Example Output: + +```bash +{ + "plan": "v2.1-upgrade" +} +``` + +#### Module versions + +`ModuleVersions` queries the list of module versions from state. + +```bash +cosmos.upgrade.v1beta1.Query/ModuleVersions +``` + +Example: + +```bash +grpcurl -plaintext localhost:9090 cosmos.slashing.v1beta1.Query/ModuleVersions +``` + +Example Output: + +```bash expandable +{ + "module_versions": [ + { + "name": "auth", + "version": "2" + }, + { + "name": "authz", + "version": "1" + }, + { + "name": "bank", + "version": "2" + }, + { + "name": "distribution", + "version": "2" + }, + { + "name": "evidence", + "version": "1" + }, + { + "name": "feegrant", + "version": "1" + }, + { + "name": "genutil", + "version": "1" + }, + { + "name": "gov", + "version": "2" + }, + { + "name": "ibc", + "version": "2" + }, + { + "name": "mint", + "version": "1" + }, + { + "name": "params", + "version": "1" + }, + { + "name": "slashing", + "version": "2" + }, + { + "name": "staking", + "version": "2" + }, + { + "name": "transfer", + "version": "1" + }, + { + "name": "upgrade", + "version": "1" + }, + { + "name": "vesting", + "version": "1" + } + ] +} +``` + +## Resources + +A list of (external) resources to learn more about the `x/upgrade` module. + +* [Cosmos Dev Series: Cosmos Blockchain Upgrade](https://medium.com/web3-surfers/cosmos-dev-series-cosmos-sdk-based-blockchain-upgrade-b5e99181554c) - The blog post that explains how software upgrades work in detail. diff --git a/sdk/v0.54/node/interact-node.mdx b/sdk/v0.54/node/interact-node.mdx new file mode 100644 index 000000000..516924eaf --- /dev/null +++ b/sdk/v0.54/node/interact-node.mdx @@ -0,0 +1,322 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/node/interact-node' +title: Interacting with a Node +--- + + +**Synopsis** +There are multiple ways to interact with a node: using the CLI, gRPC, or REST endpoints. + + + +**Prerequisite Readings** + +* [gRPC, REST and CometBFT Endpoints](/sdk/v0.54/learn/concepts/cli-grpc-rest) +* [Running a Node](/sdk/v0.54/node/run-node) + + + +## Using the CLI + +Now that your chain is running, it is time to try sending tokens from the first account you created to a second account. In a new terminal window, start by running the following query command: + +```bash +simd query bank balances $MY_VALIDATOR_ADDRESS +``` + +You should see the current balance of the account you created, equal to the original balance of `stake` you granted it minus the amount you delegated via the `gentx`. Now, create a second account: + +```bash +simd keys add recipient --keyring-backend test + +# Put the generated address in a variable for later use. +RECIPIENT=$(simd keys show recipient -a --keyring-backend test) +``` + +The command above creates a local key-pair that is not yet registered on the chain. An account is created the first time it receives tokens from another account. Now, run the following command to send tokens to the `recipient` account: + +```bash +simd tx bank send $MY_VALIDATOR_ADDRESS $RECIPIENT 1000000stake --chain-id my-test-chain --keyring-backend test + +# Check that the recipient account did receive the tokens. +simd query bank balances $RECIPIENT +``` + + +Add the `-y` or `--yes` flag to skip the confirmation prompt, which is useful for scripts and automation: + +```bash +simd tx bank send $MY_VALIDATOR_ADDRESS $RECIPIENT 1000000stake --chain-id my-test-chain --keyring-backend test -y +``` + + +Finally, delegate some of the stake tokens sent to the `recipient` account to the validator: + +```bash +simd tx staking delegate $(simd keys show my_validator --bech val -a --keyring-backend test) 500stake --from recipient --chain-id my-test-chain --keyring-backend test + +# Query the total delegations to `validator`. +simd query staking delegations-to $(simd keys show my_validator --bech val -a --keyring-backend test) +``` + +You should see two delegations, the first one made from the `gentx`, and the second one you just performed from the `recipient` account. + +## Using gRPC + +The Protobuf ecosystem developed tools for different use cases, including code-generation from `*.proto` files into various languages. These tools allow the building of clients easily. Often, the client connection (i.e. the transport) can be plugged and replaced very easily. This section explores one of the most popular transports: [gRPC](/sdk/v0.54/learn/concepts/cli-grpc-rest). + +Since the code generation library largely depends on your own tech stack, three alternatives are presented: + +* `grpcurl` for generic debugging and testing, +* programmatically via Go, +* CosmJS for JavaScript/TypeScript developers. + +### grpcurl + +[grpcurl](https://github.com/fullstorydev/grpcurl) is like `curl` but for gRPC. It is also available as a Go library, but this tutorial uses it only as a CLI command for debugging and testing purposes. Follow the instructions in the previous link to install it. + +Assuming you have a local node running (either a localnet, or connected to a live network), you should be able to run the following command to list the Protobuf services available (you can replace `localhost:9090` with the gRPC server endpoint of another node, which is configured under the `grpc.address` field inside [`app.toml`](/sdk/v0.54/node/run-node#configuring-the-node-using-apptoml-and-configtoml)): + +```bash +grpcurl -plaintext localhost:9090 list +``` + +You should see a list of gRPC services, like `cosmos.bank.v1beta1.Query`. This is called reflection, which is a Protobuf endpoint returning a description of all available endpoints. Each of these represents a different Protobuf service, and each service exposes multiple RPC methods you can query against. + +In order to get a description of the service you can run the following command: + +```bash +grpcurl -plaintext \ + localhost:9090 \ + describe cosmos.bank.v1beta1.Query # Service we want to inspect +``` + +It's also possible to execute an RPC call to query the node for information: + +```bash +grpcurl \ + -plaintext \ + -d "{\"address\":\"$MY_VALIDATOR_ADDRESS\"}" \ + localhost:9090 \ + cosmos.bank.v1beta1.Query/AllBalances +``` + +The list of all available gRPC query endpoints is [coming soon](https://github.com/cosmos/cosmos-sdk/issues/7786). + +#### Query for historical state using grpcurl + +You may also query for historical data by passing some [gRPC metadata](https://github.com/grpc/grpc-go/blob/master/Documentation/grpc-metadata.md) to the query: the `x-cosmos-block-height` metadata should contain the block to query. Using grpcurl as above, the command looks like: + +```bash +grpcurl \ + -plaintext \ + -H "x-cosmos-block-height: 123" \ + -d "{\"address\":\"$MY_VALIDATOR_ADDRESS\"}" \ + localhost:9090 \ + cosmos.bank.v1beta1.Query/AllBalances +``` + +Assuming the state at that block has not yet been pruned by the node, this query should return a non-empty response. + +### Programmatically via Go + +The following snippet shows how to query the state using gRPC inside a Go program. The idea is to create a gRPC connection, and use the Protobuf-generated client code to query the gRPC server. + +#### Install Cosmos SDK + +```bash +go get github.com/cosmos/cosmos-sdk@main +``` + +```go expandable +package main + +import ( + + "context" + "fmt" + "google.golang.org/grpc" + "github.com/cosmos/cosmos-sdk/codec" + sdk "github.com/cosmos/cosmos-sdk/types" + banktypes "github.com/cosmos/cosmos-sdk/x/bank/types" +) + +func queryState() + +error { + myAddress, err := sdk.AccAddressFromBech32("cosmos1...") // the my_validator or recipient address. + if err != nil { + return err +} + + // Create a connection to the gRPC server. + grpcConn, err := grpc.Dial( + "127.0.0.1:9090", // your gRPC server address. + grpc.WithInsecure(), // The Cosmos SDK doesn't support any transport security mechanisms. + // This instantiates a general gRPC codec which handles proto bytes. We pass in a nil interface registry + // if the request/response types contain an interface instead of 'nil' you should pass the application specific codec. + grpc.WithDefaultCallOptions(grpc.ForceCodec(codec.NewProtoCodec(nil).GRPCCodec())), + ) + if err != nil { + return err +} + +defer grpcConn.Close() + + // This creates a gRPC client to query the x/bank service. + bankClient := banktypes.NewQueryClient(grpcConn) + +bankRes, err := bankClient.Balance( + context.Background(), + &banktypes.QueryBalanceRequest{ + Address: myAddress.String(), + Denom: "stake" +}, + ) + if err != nil { + return err +} + +fmt.Println(bankRes.GetBalance()) // Prints the account balance + + return nil +} + +func main() { + if err := queryState(); err != nil { + panic(err) +} +} +``` + +You can replace the query client (here we are using `x/bank`'s) with one generated from any other Protobuf service. The list of all available gRPC query endpoints is [coming soon](https://github.com/cosmos/cosmos-sdk/issues/7786). + +#### Query for historical state using Go + +Querying for historical blocks is done by adding the block height metadata in the gRPC request. + +```go expandable +package main + +import ( + + "context" + "fmt" + "google.golang.org/grpc" + "google.golang.org/grpc/metadata" + "github.com/cosmos/cosmos-sdk/codec" + sdk "github.com/cosmos/cosmos-sdk/types" + grpctypes "github.com/cosmos/cosmos-sdk/types/grpc" + banktypes "github.com/cosmos/cosmos-sdk/x/bank/types" +) + +func queryState() + +error { + myAddress, err := sdk.AccAddressFromBech32("cosmos1yerherx4d43gj5wa3zl5vflj9d4pln42n7kuzu") // the my_validator or recipient address. + if err != nil { + return err +} + + // Create a connection to the gRPC server. + grpcConn, err := grpc.Dial( + "127.0.0.1:9090", // your gRPC server address. + grpc.WithInsecure(), // The Cosmos SDK doesn't support any transport security mechanisms. + // This instantiates a general gRPC codec which handles proto bytes. We pass in a nil interface registry + // if the request/response types contain an interface instead of 'nil' you should pass the application specific codec. + grpc.WithDefaultCallOptions(grpc.ForceCodec(codec.NewProtoCodec(nil).GRPCCodec())), + ) + if err != nil { + return err +} + +defer grpcConn.Close() + + // This creates a gRPC client to query the x/bank service. + bankClient := banktypes.NewQueryClient(grpcConn) + +var header metadata.MD + _, err = bankClient.Balance( + metadata.AppendToOutgoingContext(context.Background(), grpctypes.GRPCBlockHeightHeader, "12"), // Add metadata to request + &banktypes.QueryBalanceRequest{ + Address: myAddress.String(), + Denom: "stake" +}, + grpc.Header(&header), // Retrieve header from response + ) + if err != nil { + return err +} + blockHeight := header.Get(grpctypes.GRPCBlockHeightHeader) + +fmt.Println(blockHeight) // Prints the block height (12) + +return nil +} + +func main() { + if err := queryState(); err != nil { + panic(err) +} +} +``` + +### CosmJS + +CosmJS documentation can be found at [Link](https://cosmos.github.io/cosmjs). {/* As of January 2021, CosmJS documentation is still work in progress. */} + +## Using the REST Endpoints + +As described in the [gRPC guide](/sdk/v0.54/learn/concepts/cli-grpc-rest), all gRPC services on the Cosmos SDK are made available for more convenient REST-based queries through gRPC-gateway. The format of the URL path is based on the Protobuf service method's full-qualified name, but may contain small customizations so that final URLs look more idiomatic. For example, the REST endpoint for the `cosmos.bank.v1beta1.Query/AllBalances` method is `GET /cosmos/bank/v1beta1/balances/{address}`. Request arguments are passed as query parameters. + +Note that the REST endpoints are not enabled by default. To enable them, edit the `api` section of your `~/.simapp/config/app.toml` file: + +```toml +# Enable defines if the API server should be enabled. +enable = true +``` + + +After enabling the API, you must restart your node for the changes to take effect. Stop the node with `Ctrl+C` and run `simd start` again. + + +As a concrete example, the `curl` command to make balances request is: + +```bash +curl \ + -X GET \ + -H "Content-Type: application/json" \ + http://localhost:1317/cosmos/bank/v1beta1/balances/$MY_VALIDATOR_ADDRESS +``` + +Make sure to replace `localhost:1317` with the REST endpoint of your node, configured under the `api.address` field. + +The list of all available REST endpoints is available as a Swagger specification file, which can be viewed at `localhost:1317/swagger`. Make sure that the `api.swagger` field is set to true in your [`app.toml`](/sdk/v0.54/node/run-node#configuring-the-node-using-apptoml-and-configtoml) file. + +### Query for historical state using REST + +Querying for historical state is done using the HTTP header `x-cosmos-block-height`. For example, a curl command would look like: + +```bash +curl \ + -X GET \ + -H "Content-Type: application/json" \ + -H "x-cosmos-block-height: 123" \ + http://localhost:1317/cosmos/bank/v1beta1/balances/$MY_VALIDATOR_ADDRESS +``` + +Assuming the state at that block has not yet been pruned by the node, this query should return a non-empty response. + +### Cross-Origin Resource Sharing (CORS) + +[CORS policies](https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS) are not enabled by default to help with security. If you would like to use the rest-server in a public environment, we recommend you provide a reverse proxy, which can be done with [nginx](https://www.nginx.com/). For testing and development purposes, there is an `enabled-unsafe-cors` field inside [`app.toml`](/sdk/v0.54/node/run-node#configuring-the-node-using-apptoml-and-configtoml). + +## Congratulations! + +You have successfully interacted with your Cosmos SDK node using the CLI, gRPC, and REST endpoints. You can now query state and submit transactions through multiple interfaces. + +## Next steps + +- [Generate and sign transactions](/sdk/v0.54/node/txs) to learn manual transaction workflows +- Explore the [gRPC and REST guide](/sdk/v0.54/learn/concepts/cli-grpc-rest) for more advanced querying techniques diff --git a/sdk/v0.54/node/keyring.mdx b/sdk/v0.54/node/keyring.mdx new file mode 100644 index 000000000..fa287d9a2 --- /dev/null +++ b/sdk/v0.54/node/keyring.mdx @@ -0,0 +1,157 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/node/keyring' +title: Setting up the keyring +--- + + +**Prerequisite Readings** + +* [Prerequisites](/sdk/v0.54/node/prerequisites) - Set up Go and build the `simd` binary + + + +The keyring holds the private/public key pairs used to interact with a node. A validator key needs to be set up before running the blockchain node so that blocks can be correctly signed. + +## Create a key + +1. Create a new key for your validator: + +```bash +simd keys add my_validator --keyring-backend test +``` + +2. Store the address for later use: + +```bash + +MY_VALIDATOR_ADDRESS=$(simd keys show my_validator -a --keyring-backend test) +``` + +This generates a 24-word mnemonic phrase and stores your key. **Save the mnemonic** if you'll use this key for value-bearing tokens. + + +This tutorial uses the `test` backend (unencrypted, for testing only). For production, use the `os` backend which integrates with your system's secure keyring. See [keyring backends](#reference:-keyring-backends) below for more information. + + +## Next steps + +You have just created your first key. The keyring is now ready to manage keys for interacting with your blockchain node. + +If you are running through this tutorial as a test, continue to [Run a node](/sdk/v0.54/node/run-node) to initialize your blockchain and start your node. + +For more information on the keyring and its various backends, continue reading below. + +## Reference: Keyring backends + +The Cosmos SDK keyring supports multiple storage backends. The private key can be stored in different locations such as a file or the operating system's own key storage. + +### The `os` backend + +The `os` backend relies on operating system-specific defaults to handle key storage +securely. Typically, an operating system's credential subsystem handles password prompts, +private key storage, and user sessions according to the user's password policies. Here +is a list of the most popular operating systems and their respective password managers: + +* macOS: [Keychain](https://support.apple.com/en-gb/guide/keychain-access/welcome/mac) +* Windows: [Credentials Management API](https://docs.microsoft.com/en-us/windows/win32/secauthn/credentials-management) +* GNU/Linux: + * [libsecret](https://gitlab.gnome.org/GNOME/libsecret) + * [kwallet](https://api.kde.org/kwallet-index.html) + * [keyctl](https://www.kernel.org/doc/html/latest/security/keys/core.html) + +GNU/Linux distributions that use GNOME as the default desktop environment typically come with +[Seahorse](https://wiki.gnome.org/Apps/Seahorse). Users of KDE based distributions are +commonly provided with [KDE Wallet Manager](https://userbase.kde.org/KDE_Wallet_Manager). +Whilst the former is in fact a `libsecret` convenient frontend, the latter is a `kwallet` +client. `keyctl` is a secure backend that leverages the Linux's kernel security key management system +to store cryptographic keys securely in memory. + +`os` is the default option since operating systems' default credentials managers are +designed to meet users' most common needs and provide them with a comfortable +experience without compromising on security. + +The recommended backends for headless environments are `file` and `pass`. + +### The `file` backend + +The `file` backend more closely resembles the keybase implementation used prior to +v0.38.1. It stores the keyring encrypted within the app's configuration directory. This +keyring will request a password each time it is accessed, which may occur multiple +times in a single command, resulting in repeated password prompts. If using bash scripts +to execute commands using the `file` option, you may want to utilize the following format +for multiple prompts: + +```shell +# assuming that KEYPASSWD is set in the environment +$ gaiacli config keyring-backend file # use file backend +$ (echo $KEYPASSWD; echo $KEYPASSWD) | gaiacli keys add me # multiple prompts +$ echo $KEYPASSWD | gaiacli keys show me # single prompt +``` + + +The first time you add a key to an empty keyring, you will be prompted to type the password twice. + + +### The `pass` backend + +The `pass` backend uses the [pass](https://www.passwordstore.org/) utility to manage on-disk +encryption of keys' sensitive data and metadata. Keys are stored inside `gpg`-encrypted files +within app-specific directories. `pass` is available for the most popular UNIX +operating systems as well as GNU/Linux distributions. Please refer to its manual page for +information on how to download and install it. + + +**pass** uses [GnuPG](https://gnupg.org/) for encryption. `gpg` automatically invokes the `gpg-agent` +daemon upon execution, which handles the caching of GnuPG credentials. Please refer to `gpg-agent` +man page for more information on how to configure cache parameters such as credentials TTL and +passphrase expiration. + + +The password store must be set up prior to first use: + +```shell +pass init +``` + +Replace `` with your GPG key ID. You can use your personal GPG key or an alternative +one you may want to use specifically to encrypt the password store. + +### The `kwallet` backend + +The `kwallet` backend uses `KDE Wallet Manager`, which comes installed by default on the +GNU/Linux distributions that ships KDE as default desktop environment. Please refer to +[KWallet Handbook](https://userbase.kde.org/KDE_Wallet_Manager) for more +information. + +### The `keyctl` backend + +The *Kernel Key Retention Service* is a security facility that +has been added to the Linux kernel relatively recently. It allows sensitive +cryptographic data such as passwords, private keys, authentication tokens, etc. +to be stored securely in memory. + +The `keyctl` backend is available on Linux platforms only. + +### The `test` backend + +The `test` backend is a password-less variation of the `file` backend. Keys are stored +unencrypted on disk. + +**Provided for testing purposes only. The `test` backend is not recommended for use in production environments**. + +### The `memory` backend + +The `memory` backend stores keys in memory. The keys are immediately deleted after the program has exited. + +**Provided for testing purposes only. The `memory` backend is not recommended for use in production environments**. + +### Setting backend using the env variable + +You can set the keyring-backend using an environment variable: `BINNAME_KEYRING_BACKEND`. For example, if your binary name is `gaia-v5`, then set: `export GAIA_V5_KEYRING_BACKEND=pass` + +### Additional key management + +By default, the keyring generates a `secp256k1` keypair. The keyring also supports `ed25519` keys, which may be created by passing the `--algo ed25519` flag. A keyring can hold both types of keys simultaneously, and the Cosmos SDK's `x/auth` module supports both public key algorithms natively. + +For help with key management commands, use `simd keys --help` or `simd keys [command] --help`. diff --git a/sdk/v0.54/node/prerequisites.mdx b/sdk/v0.54/node/prerequisites.mdx new file mode 100644 index 000000000..2eb500099 --- /dev/null +++ b/sdk/v0.54/node/prerequisites.mdx @@ -0,0 +1,124 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/node/prerequisites' +title: Prerequisites +--- + +## Introduction + +The Cosmos SDK requires Go and a built binary to run a blockchain node. This tutorial walks through installing Go, building the `simd` binary, and configuring your environment to run a Cosmos SDK node. + +## Prerequisites + +This tutorial assumes you have the following installed: + +- A terminal application +- A code editor +- Basic familiarity with command-line operations + +## 1. Install Go + +The Cosmos SDK requires [Go](https://go.dev/) version 1.25 or higher. Download the installer from the [official Go downloads page](https://go.dev/dl/) and follow the installation instructions for your operating system. + +Verify the installation: + +```bash +go version +``` + +### Configure Go environment variables + + + + +Open your shell config file (`~/.bashrc` or `~/.zshrc`) and add: + +```bash +export GOPATH=$HOME/go +export PATH=$PATH:$GOPATH/bin +``` + +Apply changes: `source ~/.bashrc` + + + + +Open `~/.zshrc` and add: + +```bash +export GOPATH=$HOME/go +export PATH=$PATH:$GOPATH/bin +``` + +Apply changes: `source ~/.zshrc` + + + + +In PowerShell (as Administrator): + +```powershell +[System.Environment]::SetEnvironmentVariable('GOPATH', "$HOME\go", 'User') +[System.Environment]::SetEnvironmentVariable('Path', "$env:Path;$HOME\go\bin", 'User') +``` + +Restart PowerShell after setting. + + + + +Verify: `go env GOPATH` + +## 2. Clone the Cosmos SDK repository + +This tutorial uses `simapp`, the Cosmos SDK example application. Clone the [Cosmos SDK repository](https://github.com/cosmos/cosmos-sdk) to access `simapp`. + +1. Navigate to your preferred directory. This example uses `~/Documents/GitHub`: + +```bash +cd ~/Documents/GitHub +``` + +2. Clone the Cosmos SDK repository: + +```bash +git clone https://github.com/cosmos/cosmos-sdk.git +``` + +3. Navigate into the cloned repository: + +```bash +cd cosmos-sdk +``` + + +If you are building your own chain, clone your chain's repository instead. Replace `simd` with your chain's binary name throughout this tutorial. + + + +## 3. Build the simd binary + +The `simd` binary is the command-line interface for interacting with the Cosmos SDK blockchain. + +Build and install the `simd` binary: + +```bash +make install +``` + + +**Windows users**: If `make` is not available, you can install it via [Chocolatey](https://chocolatey.org/) (`choco install make`) or use WSL2. + + +Verify `simd` is working: + +```bash +simd version +``` + +Your environment is now set up to run a Cosmos SDK node. + +## Next steps + +- [Set up the keyring](/sdk/v0.54/node/keyring) to create and manage cryptographic keys + diff --git a/sdk/v0.54/node/run-node.mdx b/sdk/v0.54/node/run-node.mdx new file mode 100644 index 000000000..f5a37780b --- /dev/null +++ b/sdk/v0.54/node/run-node.mdx @@ -0,0 +1,254 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/node/run-node' +title: Running a Node +--- + + +**Synopsis** + +This section explains how to run a blockchain node. The application used in this tutorial is [`simapp`](https://github.com/cosmos/cosmos-sdk/tree/main/simapp), and its corresponding CLI binary `simd`. + + + +**Prerequisite Readings** + +* [Prerequisites](/sdk/v0.54/node/prerequisites) - Set up Go and build the `simd` binary +* [Anatomy of a Cosmos SDK Application](/sdk/v0.54/learn/intro/sdk-app-architecture) +* [Setting up the keyring](/sdk/v0.54/node/keyring) + + + +## 1. Initialize the chain + +Before running the node, initialize the chain and its genesis file. Use the `init` subcommand: + +```bash +# The argument is the custom username of your node, it should be human-readable. +simd init --chain-id my-test-chain +``` + +The command above creates all the configuration files needed for your node to run, as well as a default genesis file, which defines the initial state of the network. + + +All these configuration files are in `~/.simapp` by default, but you can overwrite the location of this folder by passing the `--home` flag to each command, +or set an `$APPD_HOME` environment variable (where `APPD` is the name of the binary). + + + +**Windows users**: Replace `~/.simapp` with `%USERPROFILE%\.simapp` (or `$HOME\.simapp` in PowerShell). For `jq` and `sed` commands in this tutorial, install via [Chocolatey](https://chocolatey.org/) (`choco install jq sed`) or use [Git Bash](https://gitforwindows.org/)/WSL2. + + +The `~/.simapp` folder has the following structure: + +```bash +. # ~/.simapp + |- data # Contains the databases used by the node. + |- config/ + |- app.toml # Application-related configuration file. + |- config.toml # CometBFT-related configuration file. + |- genesis.json # The genesis file. + |- node_key.json # Private key to use for node authentication in the p2p protocol. + |- priv_validator_key.json # Private key to use as a validator in the consensus protocol. +``` + +## 2. Update configuration settings (optional) + +To change field values in configuration files (for example, genesis.json), use `jq` ([installation](https://stedolan.github.io/jq/download/) & [docs](https://stedolan.github.io/jq/manual/#Assignment)) and `sed` commands. A few examples are listed here. + +```bash expandable +# to change the chain-id +jq '.chain_id = "testing"' genesis.json > temp.json && mv temp.json genesis.json + +# to enable the api server +sed -i '/\[api\]/,+3 s/enable = false/enable = true/' app.toml + +# to change the voting_period +jq '.app_state.gov.voting_params.voting_period = "600s"' genesis.json > temp.json && mv temp.json genesis.json + +# to change the inflation +jq '.app_state.mint.minter.inflation = "0.300000000000000000"' genesis.json > temp.json && mv temp.json genesis.json +``` + +### Client Interaction + +When instantiating a node, gRPC and REST are defaulted to localhost to avoid unknown exposure of your node to the public. It is recommended not to expose these endpoints without a proxy that can handle load balancing or authentication set up between your node and the public. + + +A commonly used tool for this is [nginx](https://nginx.org). + + +## 3. Add genesis accounts + +Earlier in this tutorial, you [created an account in the keyring](/sdk/v0.54/node/keyring#create-a-key) named `my_validator` under the `test` keyring backend. + +Now, you can grant this account some `stake` tokens in your chain's genesis file. Doing so will also make sure your chain is aware of this account's existence: + +```bash +simd genesis add-genesis-account $MY_VALIDATOR_ADDRESS 100000000000stake +``` + +Recall that `$MY_VALIDATOR_ADDRESS` is a variable that holds the address of the `my_validator` key in the [keyring](/sdk/v0.54/node/keyring#create-a-key). Also note that the tokens in the Cosmos SDK have the `{amount}{denom}` format: `amount` is an 18-digit-precision decimal number, and `denom` is the unique token identifier with its denomination key (e.g., `atom` or `uatom`). Here, `stake` tokens are granted, as `stake` is the token identifier used for staking in [`simapp`](https://github.com/cosmos/cosmos-sdk/tree/main/simapp). For your own chain with its own staking denom, that token identifier should be used instead. + +## 4. Create genesis transaction + +Now that your account has some tokens, you need to add a validator to your chain. Validators are special full-nodes that participate in the consensus process (implemented in the [underlying consensus engine](/sdk/v0.54/learn/intro/sdk-app-architecture#cometbft)) in order to add new blocks to the chain. Any account can declare its intention to become a validator operator, but only those with sufficient delegation get to enter the active set (for example, only the top 125 validator candidates with the most delegation get to be validators in the Cosmos Hub). For this guide, your local node (created via the `init` command above) will be added as a validator of your chain. Validators can be declared before a chain is first started via a special transaction included in the genesis file called a `gentx`: + +1. Create a gentx. + +```bash +simd genesis gentx my_validator 100000000stake --chain-id my-test-chain --keyring-backend test +``` + +2. Add the gentx to the genesis file: + +```bash +simd genesis collect-gentxs +``` + +A `gentx` does three things: + +1. Registers the validator account you created as a validator operator account (i.e., the account that controls the validator). +2. Self-delegates the provided `amount` of staking tokens. +3. Link the operator account with a CometBFT node pubkey that will be used for signing blocks. If no `--pubkey` flag is provided, it defaults to the local node pubkey created via the `simd init` command above. + +For more information on `gentx`, use the following command: + +```bash +simd genesis gentx --help +``` + +## 5. Configure the node using `app.toml` and `config.toml` + +The Cosmos SDK automatically generates two configuration files inside `~/.simapp/config`: + +* `config.toml`: used to configure the CometBFT, learn more on [CometBFT's documentation](/cometbft/latest/docs/core/configuration), +* `app.toml`: generated by the Cosmos SDK, and used to configure your app, such as state pruning strategies, telemetry, gRPC and REST server configuration, state sync... + +Both files are heavily commented, please refer to them directly to tweak your node. + +One example config to tweak is the `minimum-gas-prices` field inside `app.toml`, which defines the minimum gas prices the validator node is willing to accept for processing a transaction. Depending on the chain, it might be an empty string or not. If it's empty, make sure to edit the field with some value, for example `10token`, or else the node will halt on startup. For the purposes of this tutorial, the minimum gas price is set to 0: + +```toml + # The minimum gas prices a validator is willing to accept for processing a + # transaction. A transaction's fees must meet the minimum of any denomination + # specified in this config (e.g. 0.25token1;0.0001token2). + minimum-gas-prices = "0stake" +``` + + +When running a node (not a validator!) and not wanting to run the application mempool, set the `max-txs` field to `-1`. + +```toml +[mempool] +# Setting max-txs to 0 will allow for an unbounded amount of transactions in the mempool. +# Setting max_txs to negative 1 (-1) will disable transactions from being inserted into the mempool. +# Setting max_txs to a positive number (> 0) will limit the number of transactions in the mempool, by the specified amount. +# +# Note, this configuration only applies to SDK built-in app-side mempool +# implementations. +max-txs = "-1" +``` + + + +## 6. Start the node + +Now that everything is set up, you can finally start your node: + +```bash +simd start +``` + +You should see blocks come in. + +### What happens when the node starts + +The `start` command (defined in [`server/start.go`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/server/start.go)) boots up the full-node in the following sequence: + +1. It opens the `db` (LevelDB by default) containing the latest persisted state. On first start, this is empty. +2. It creates a new instance of the application via an `appCreator` function, which is the [application constructor](/sdk/v0.54/learn/intro/sdk-app-architecture#constructor-function). +3. It instantiates a CometBFT node using the application. As part of `node.New`, CometBFT checks that the application's block height matches its own. If the application is behind, it replays blocks to catch up. If the height is `0`, it calls [`InitChain`](/sdk/v0.54/learn/concepts/baseapp#initchain) to initialize state from the genesis file. +4. Once in sync, the node starts its RPC and P2P servers and begins dialing peers. During the handshake, if the node is behind its peers, it queries missing blocks sequentially. Once caught up, it waits for new block proposals and validator signatures. + +The previous command allows you to run a single node. This is enough for the next section on interacting with this node, but you may wish to run multiple nodes at the same time, and see how consensus happens between them. + +The naive way would be to run the same commands again in separate terminal windows. This is possible. However, [Docker Compose](https://docs.docker.com/compose/) can be leveraged to run a localnet. If you need inspiration on how to set up your own localnet with Docker Compose, refer to the Cosmos SDK's [`docker-compose.yml`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/docker-compose.yml). + +### Standalone App/CometBFT + +By default, the Cosmos SDK runs CometBFT in-process with the application +If you want to run the application and CometBFT in separate processes, +start the application with the `--with-comet=false` flag +and set `rpc.laddr` in `config.toml` to the CometBFT node's RPC address. + +## Logging + +Logging provides a way to see what is going on with a node. The default logging level is `info`. This is a global level and all info logs will be outputted to the terminal. If you would like to filter specific logs to the terminal instead of all, then setting `module:log_level` is how this can work. + +Example in `config.toml`: + +```toml +log_level: "state:info,p2p:info,consensus:info,x/staking:info,x/ibc:info,*error" +``` + +### Verbose log level + +Some operations, such as chain upgrades, emit additional log messages when a higher log level is active. You can control this with the `--verbose_log_level` flag: + +```bash +simd start --verbose_log_level debug +``` + +See the [Log Overview](/sdk/v0.54/guides/testing/log) for more information on logging. + +## State Sync + +State sync is the act in which a node syncs the latest or close to the latest state of a blockchain. This is useful for users who don't want to sync all the blocks in history. Read more in [CometBFT documentation](/cometbft/latest/docs/core/state-sync). + +State sync works thanks to snapshots. Read how the SDK handles snapshots [here](https://github.com/cosmos/cosmos-sdk/blob/825245d/store/snapshots/README.md). + +### Local State Sync + +Local state sync works similarly to normal state sync except that it works off a local snapshot of state instead of one provided via the p2p network. The steps to start local state sync are similar to normal state sync with a few different design considerations. + +1. As mentioned in the [state sync documentation](/cometbft/latest/docs/core/state-sync), one must set a height and hash in the config.toml along with a few RPC servers (the aforementioned link has instructions on how to do this). +2. Run ` snapshot restore ` to restore a local snapshot (note: first load it from a file with the *load* command). +3. Bootstrapping Comet state to start the node after the snapshot has been ingested. This can be done with the bootstrap command ` comet bootstrap-state` + +### Snapshots Commands + +The Cosmos SDK provides commands for managing snapshots. +These commands can be added in an app with the following snippet in `cmd//root.go`: + +```go +import ( + + "github.com/cosmos/cosmos-sdk/client/snapshot" +) + +func initRootCmd(/* ... */) { + // ... + rootCmd.AddCommand( + snapshot.Cmd(appCreator), + ) +} +``` + +Then the following commands are available at ` snapshots [command]`: + +* **list**: list local snapshots +* **load**: Load a snapshot archive file into snapshot store +* **restore**: Restore app state from local snapshot +* **export**: Export app state to snapshot store +* **dump**: Dump the snapshot as portable archive format +* **delete**: Delete a local snapshot + +## Congratulations! + +Your node is now running and producing blocks. You have successfully initialized a Cosmos SDK blockchain from scratch. + +## Next steps + +- [Interact with the node](/sdk/v0.54/node/interact-node) to send transactions and query state +- [Generate and sign transactions](/sdk/v0.54/node/txs) to learn advanced transaction workflows \ No newline at end of file diff --git a/sdk/v0.54/node/run-production.mdx b/sdk/v0.54/node/run-production.mdx new file mode 100644 index 000000000..573aa3218 --- /dev/null +++ b/sdk/v0.54/node/run-production.mdx @@ -0,0 +1,269 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/node/run-production' +title: Running in Production +--- + + +**Synopsis** +This section describes how to securely run a node in a public setting and/or on a mainnet on one of the many Cosmos SDK public blockchains. + + + +When operating a node, full node, or validator in production it is important to set your server up securely. + +This walkthrough assumes the underlying operating system is Ubuntu. + +There are many different ways to secure a server and your node. The steps described here are for informational purposes only. + + + +## Server Setup + +### User + +When creating a server most times it is created as user `root`. This user has heightened privileges on the server. When operating a node, it is recommended to not run your node as the root user. + +1. Create a new user + +```bash +sudo adduser change_me +``` + +2. We want to allow this user to perform sudo tasks + +```bash +sudo usermod -aG sudo change_me +``` + +Now when logging into the server, the non `root` user can be used. + +### Go + +1. Install the [Go](https://go.dev/doc/install) version recommended by the application. + + +In the past, validators [have had issues](https://github.com/cosmos/cosmos-sdk/issues/13976) when using different versions of Go. It is recommended that the whole validator set uses the version of Go that is recommended by the application. + + +### Firewall + +Nodes should not have all ports open to the public; this is a simple way to get DDoS'd. Secondly, it is recommended by [CometBFT](https://github.com/cometbft/cometbft) to never expose ports that are not required to operate a node. + +When setting up a firewall, there are a few ports that can be open when operating a Cosmos SDK node. These include the CometBFT JSON-RPC, Prometheus, p2p, remote signer, and Cosmos SDK gRPC and REST. If the node is being operated as a node that does not offer endpoints to be used for submission or querying, then a maximum of three endpoints are needed. + +Most, if not all servers come equipped with [ufw](https://help.ubuntu.com/community/UFW). Ufw will be used in this tutorial. + +1. Reset UFW to disallow all incoming connections and allow outgoing + +```bash +sudo ufw default deny incoming +sudo ufw default allow outgoing +``` + +2. Let's make sure that port 22 (SSH) stays open. + +```bash +sudo ufw allow ssh +``` + +or + +```bash +sudo ufw allow 22 +``` + +Both of the above commands are the same. + +3. Allow Port 26656 (cometbft p2p port). If the node has a modified p2p port then that port must be used here. + +```bash +sudo ufw allow 26656/tcp +``` + +4. Allow port 26660 (CometBFT [Prometheus](https://prometheus.io)). This acts as the application's monitoring port as well. + +```bash +sudo ufw allow 26660/tcp +``` + +5. If the node which is being set up would like to expose CometBFT's JSON-RPC and Cosmos SDK gRPC and REST, then follow this step. (Optional) + +##### CometBFT JSON-RPC + +```bash +sudo ufw allow 26657/tcp +``` + +##### Cosmos SDK gRPC + +```bash +sudo ufw allow 9090/tcp +``` + +##### Cosmos SDK REST + +```bash +sudo ufw allow 1317/tcp +``` + +6. Lastly, enable ufw + +```bash +sudo ufw enable +``` + +### Signing + +If the node that is being started is a validator there are multiple ways a validator could sign blocks. + +#### File + +File-based signing is the simplest and default approach. This approach works by storing the consensus key generated on initialization to sign blocks. This approach is only as safe as your server setup, as if the server is compromised, so is your key. This key is located in the `config/priv_val_key.json` directory generated on initialization. + +A second file exists that users must be aware of; the file is located in the data directory `data/priv_val_state.json`. This file protects your node from double signing. It keeps track of the consensus key's last sign height, round, and latest signature. If the node crashes and needs to be recovered, this file must be kept in order to ensure that the consensus key will not be used for signing a block that was previously signed. + +#### Remote Signer + +A remote signer is a secondary server that is separate from the running node that signs blocks with the consensus key. This means that the consensus key does not live on the node itself. This increases security because your full node which is connected to the remote signer can be swapped without missing blocks. + +The two most used remote signers are [tmkms](https://github.com/iqlusioninc/tmkms) from [Iqlusion](https://www.iqlusion.io) and [horcrux](https://github.com/strangelove-ventures/horcrux) from [Strangelove](https://strange.love). + +##### TMKMS + +###### Dependencies + +1. Update server dependencies and install extras needed. + +```sh +sudo apt update -y && sudo apt install build-essential curl jq -y +``` + +2. Install Rust: + +```sh +curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh +``` + +3. Install Libusb: + +```sh +sudo apt install libusb-1.0-0-dev +``` + +###### Setup + +There are two ways to install tmkms, from source or `cargo install`. In the examples we will cover downloading or building from source and using softsign. Softsign stands for software signing, but you could use a [yubihsm](https://www.yubico.com/products/hardware-security-module/) as your signing key if you wish. + +1. Build: + +From source: + +```bash +cd $HOME +git clone https://github.com/iqlusioninc/tmkms.git +cd $HOME/tmkms +cargo install tmkms --features=softsign +tmkms init config +tmkms softsign keygen ./config/secrets/secret_connection_key +``` + +or + +Cargo install: + +```bash +cargo install tmkms --features=softsign +tmkms init config +tmkms softsign keygen ./config/secrets/secret_connection_key +``` + + +To use tmkms with a yubikey install the binary with `--features=yubihsm`. + + +2. Migrate the validator key from the full node to the new tmkms instance. + +```bash +scp user@123.456.32.123:~/.simd/config/priv_validator_key.json ~/tmkms/config/secrets +``` + +3. Import the validator key into tmkms. + +```bash +tmkms softsign import $HOME/tmkms/config/secrets/priv_validator_key.json $HOME/tmkms/config/secrets/priv_validator_key +``` + +At this point, it is necessary to delete the `priv_validator_key.json` from the validator node and the tmkms node. Since the key has been imported into tmkms (above) it is no longer necessary on the nodes. The key can be safely stored offline. + +4. Modify the `tmkms.toml`. + +```bash +vim $HOME/tmkms/config/tmkms.toml +``` + +This example shows a configuration that could be used for soft signing. The example has an IP of `123.456.12.345` with a port of `26659` and a chain\_id of `test-chain-waSDSe`. These are items that must be modified for the use case of tmkms and the network. + +```toml expandable +# CometBFT KMS configuration file + +## Chain Configuration + +[[chain]] +id = "osmosis-1" +key_format = { type = "bech32", account_key_prefix = "cosmospub", consensus_key_prefix = "cosmosvalconspub" } +state_file = "/root/tmkms/config/state/priv_validator_state.json" + +## Signing Provider Configuration + +### Software-based Signer Configuration + +[[providers.softsign]] +chain_ids = ["test-chain-waSDSe"] +key_type = "consensus" +path = "/root/tmkms/config/secrets/priv_validator_key" + +## Validator Configuration + +[[validator]] +chain_id = "test-chain-waSDSe" +addr = "tcp://123.456.12.345:26659" +secret_key = "/root/tmkms/config/secrets/secret_connection_key" +protocol_version = "v0.34" +reconnect = true +``` + +5. Set the address of the tmkms instance. + +```bash +vim $HOME/.simd/config/config.toml + +priv_validator_laddr = "tcp://0.0.0.0:26659" +``` + + +The above address is set to `0.0.0.0`, but it is recommended to set the tmkms server address to secure the startup. + + + +It is recommended to comment or delete the lines that specify the path of the validator key and validator: + +```toml +# Path to the JSON file containing the private key to use as a validator in the consensus protocol +# priv_validator_key_file = "config/priv_validator_key.json" + +# Path to the JSON file containing the last sign state of a validator +# priv_validator_state_file = "data/priv_validator_state.json" +``` + + + +6. Start the two processes. + +```bash +tmkms start -c $HOME/tmkms/config/tmkms.toml +``` + +```bash +simd start +``` diff --git a/sdk/v0.54/node/run-testnet.mdx b/sdk/v0.54/node/run-testnet.mdx new file mode 100644 index 000000000..cb438452c --- /dev/null +++ b/sdk/v0.54/node/run-testnet.mdx @@ -0,0 +1,100 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/node/run-testnet' +title: Running a Testnet +--- + + +**Synopsis** +The `simd testnet` subcommand makes it easy to initialize and start a simulated test network for testing purposes. + + +In addition to the commands for [running a node](/sdk/v0.54/node/run-node), the `simd` binary also includes a `testnet` command that allows you to start a simulated test network in-process or to initialize files for a simulated test network that runs in a separate process. + +## Initialize Files + +First, let's take a look at the `init-files` subcommand. + +This is similar to the `init` command when initializing a single node, but in this case we are initializing multiple nodes, generating the genesis transactions for each node, and then collecting those transactions. + +The `init-files` subcommand initializes the necessary files to run a test network in a separate process (i.e. using a Docker container). Running this command is not a prerequisite for the `start` subcommand ([see below](#start-testnet)). + +In order to initialize the files for a test network, run the following command: + +```bash +simd testnet init-files +``` + +You should see the following output in your terminal: + +```bash +Successfully initialized 4 node directories +``` + +The default output directory is a relative `.testnets` directory. Let's take a look at the files created within the `.testnets` directory. + +### gentxs + +The `gentxs` directory includes a genesis transaction for each validator node. Each file includes a JSON encoded genesis transaction used to register a validator node at the time of genesis. The genesis transactions are added to the `genesis.json` file within each node directory during the initialization process. + +### nodes + +A node directory is created for each validator node. Within each node directory is a `simd` directory. The `simd` directory is the home directory for each node, which includes the configuration and data files for that node (i.e. the same files included in the default `~/.simapp` directory when running a single node). + +## Start Testnet + +Now, let's take a look at the `start` subcommand. + +The `start` subcommand both initializes and starts an in-process test network. This is the fastest way to spin up a local test network for testing purposes. + +You can start the local test network by running the following command: + +```bash +simd testnet start +``` + +You should see something similar to the following: + +```bash expandable +acquiring test network lock +preparing test network with chain-id "chain-mtoD9v" + ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ +++ THIS MNEMONIC IS FOR TESTING PURPOSES ONLY ++ +++ DO NOT USE IN PRODUCTION ++ +++ ++ +++ sustain know debris minute gate hybrid stereo custom ++ +++ divorce cross spoon machine latin vibrant term oblige ++ +++ moment beauty laundry repeat grab game bronze truly ++ ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + +starting test network... +started test network +press the Enter Key to terminate +``` + +The first validator node is now running in-process, which means the test network will terminate once you either close the terminal window or you press the Enter key. In the output, the mnemonic phrase for the first validator node is provided for testing purposes. The validator node is using the same default addresses being used when initializing and starting a single node (no need to provide a `--node` flag). + +Check the status of the first validator node: + +```shell +simd status +``` + +Import the key from the provided mnemonic: + +```shell +simd keys add test --recover --keyring-backend test +``` + +Check the balance of the account address: + +```shell +simd q bank balances [address] +``` + +Use this test account to manually test against the test network. + +## Testnet Options + +You can customize the configuration of the test network with flags. In order to see all flag options, append the `--help` flag to each command. diff --git a/sdk/v0.54/node/txs.mdx b/sdk/v0.54/node/txs.mdx new file mode 100644 index 000000000..c74c3b2b6 --- /dev/null +++ b/sdk/v0.54/node/txs.mdx @@ -0,0 +1,568 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/node/txs' +title: 'Generating, Signing and Broadcasting Transactions' +--- + + +**Synopsis** +This document describes how to generate an (unsigned) transaction, signing it (with one or multiple keys), and broadcasting it to the network. + + +## Using the CLI + +The easiest way to send transactions is using the CLI, as shown in the previous page when [interacting with a node](/sdk/v0.54/node/interact-node#using-the-cli). For example, running the following command + +```bash +simd tx bank send $MY_VALIDATOR_ADDRESS $RECIPIENT 1000stake --chain-id my-test-chain --keyring-backend test +``` + +will run the following steps: + +* generate a transaction with one `Msg` (`x/bank`'s `MsgSend`), and print the generated transaction to the console. +* ask the user for confirmation to send the transaction from the `$MY_VALIDATOR_ADDRESS` account. +* fetch `$MY_VALIDATOR_ADDRESS` from the keyring. This is possible because the [CLI's keyring was set up](/sdk/v0.54/node/keyring) in a previous step. +* sign the generated transaction with the keyring's account. +* broadcast the signed transaction to the network. This is possible because the CLI connects to the node's CometBFT RPC endpoint. + +The CLI bundles all the necessary steps into a simple-to-use user experience. However, it is possible to run all the steps individually too. + +### Generating a Transaction + +Generating a transaction can simply be done by appending the `--generate-only` flag on any `tx` command, e.g.: + +```bash +simd tx bank send $MY_VALIDATOR_ADDRESS $RECIPIENT 1000stake --chain-id my-test-chain --generate-only +``` + +This will output the unsigned transaction as JSON in the console. The unsigned transaction can also be saved to a file (to be passed around between signers more easily) by appending `> unsigned_tx.json` to the above command. + +### Signing a Transaction + +Signing a transaction using the CLI requires the unsigned transaction to be saved in a file. For this example, assume the unsigned transaction is in a file called `unsigned_tx.json` in the current directory (see previous paragraph on how to do that). Then, simply run the following command: + +```bash +simd tx sign unsigned_tx.json --chain-id my-test-chain --keyring-backend test --from $MY_VALIDATOR_ADDRESS +``` + +This command will decode the unsigned transaction and sign it with `SIGN_MODE_DIRECT` with `$MY_VALIDATOR_ADDRESS`'s key, which was already set up in the keyring. The signed transaction will be output as JSON to the console, and, as above, it can be saved to a file by appending `--output-document signed_tx.json`. + +Some useful flags to consider in the `tx sign` command: + +* `--sign-mode`: you may use `amino-json` to sign the transaction using `SIGN_MODE_LEGACY_AMINO_JSON`, +* `--offline`: sign in offline mode. This means that the `tx sign` command doesn't connect to the node to retrieve the signer's account number and sequence, both needed for signing. In this case, you must manually supply the `--account-number` and `--sequence` flags. This is useful for offline signing, i.e. signing in a secure environment which doesn't have access to the internet. + +#### Signing with Multiple Signers + + +Please note that signing a transaction with multiple signers or with a multisig account, where at least one signer uses `SIGN_MODE_DIRECT`, is not yet possible. You may follow [this Github issue](https://github.com/cosmos/cosmos-sdk/issues/8141) for more info. + + +Signing with multiple signers is done with the `tx multisign` command. This command assumes that all signers use `SIGN_MODE_LEGACY_AMINO_JSON`. The flow is similar to the `tx sign` command flow, but instead of signing an unsigned transaction file, each signer signs the file signed by previous signer(s). The `tx multisign` command will append signatures to the existing transactions. It is important that signers sign the transaction **in the same order** as given by the transaction, which is retrievable using the `GetSigners()` method. + +For example, starting with the `unsigned_tx.json`, and assuming the transaction has 4 signers, we would run: + +```bash +# Let signer1 sign the unsigned tx. +simd tx multisign unsigned_tx.json signer_key_1 --chain-id my-test-chain --keyring-backend test > partial_tx_1.json +# Now signer1 will send the partial_tx_1.json to the signer2. +# Signer2 appends their signature: +simd tx multisign partial_tx_1.json signer_key_2 --chain-id my-test-chain --keyring-backend test > partial_tx_2.json +# Signer2 sends the partial_tx_2.json file to signer3, and signer3 can append his signature: +simd tx multisign partial_tx_2.json signer_key_3 --chain-id my-test-chain --keyring-backend test > partial_tx_3.json +``` + +### Broadcasting a Transaction + +Broadcasting a transaction is done using the following command: + +```bash +simd tx broadcast tx_signed.json +``` + +You may optionally pass the `--broadcast-mode` flag to specify which response to receive from the node: + +* `sync`: the CLI waits for a CheckTx execution response only. +* `async`: the CLI returns immediately (transaction might fail). + +### Encoding a Transaction + +In order to broadcast a transaction using the gRPC or REST endpoints, the transaction will need to be encoded first. This can be done using the CLI. + +Encoding a transaction is done using the following command: + +```bash +simd tx encode tx_signed.json +``` + +This will read the transaction from the file, serialize it using Protobuf, and output the transaction bytes as base64 in the console. + +### Decoding a Transaction + +The CLI can also be used to decode transaction bytes. + +Decoding a transaction is done using the following command: + +```bash +simd tx decode [protobuf-byte-string] +``` + +This will decode the transaction bytes and output the transaction as JSON in the console. You can also save the transaction to a file by appending `> tx.json` to the above command. + +## Programmatically with Go + +It is possible to manipulate transactions programmatically via Go using the Cosmos SDK's `TxBuilder` interface. + +### Generating a Transaction + +Before generating a transaction, a new instance of a `TxBuilder` needs to be created. Since the Cosmos SDK supports both Amino and Protobuf transactions, the first step would be to decide which encoding scheme to use. All the subsequent steps remain unchanged, whether you're using Amino or Protobuf, as `TxBuilder` abstracts the encoding mechanisms. In the following snippet, we will use Protobuf. + +```go expandable +import ( + + "github.com/cosmos/cosmos-sdk/simapp" +) + +func sendTx() + +error { + // Choose your codec: Amino or Protobuf. Here, we use Protobuf, given by the following function. + app := simapp.NewSimApp(...) + + // Create a new TxBuilder. + txBuilder := app.TxConfig().NewTxBuilder() + + // --snip-- +} +``` + +The following example sets up some keys and addresses that will send and receive the transactions. For the purpose of this tutorial, dummy data is used to create keys. + +```go +import ( + + "github.com/cosmos/cosmos-sdk/testutil/testdata" +) + +priv1, _, addr1 := testdata.KeyTestPubAddr() + +priv2, _, addr2 := testdata.KeyTestPubAddr() + +priv3, _, addr3 := testdata.KeyTestPubAddr() +``` + +Populating the `TxBuilder` can be done via its methods: + +```go expandable +package client + +import ( + + "time" + + txsigning "cosmossdk.io/x/tx/signing" + + codectypes "github.com/cosmos/cosmos-sdk/codec/types" + sdk "github.com/cosmos/cosmos-sdk/types" + "github.com/cosmos/cosmos-sdk/types/tx" + signingtypes "github.com/cosmos/cosmos-sdk/types/tx/signing" + "github.com/cosmos/cosmos-sdk/x/auth/signing" +) + +type ( + // TxEncodingConfig defines an interface that contains transaction + // encoders and decoders + TxEncodingConfig interface { + TxEncoder() + +sdk.TxEncoder + TxDecoder() + +sdk.TxDecoder + TxJSONEncoder() + +sdk.TxEncoder + TxJSONDecoder() + +sdk.TxDecoder + MarshalSignatureJSON([]signingtypes.SignatureV2) ([]byte, error) + +UnmarshalSignatureJSON([]byte) ([]signingtypes.SignatureV2, error) +} + + // TxConfig defines an interface a client can utilize to generate an + // application-defined concrete transaction type. The type returned must + // implement TxBuilder. + TxConfig interface { + TxEncodingConfig + + NewTxBuilder() + +TxBuilder + WrapTxBuilder(sdk.Tx) (TxBuilder, error) + +SignModeHandler() *txsigning.HandlerMap + SigningContext() *txsigning.Context +} + + // TxBuilder defines an interface which an application-defined concrete transaction + // type must implement. Namely, it must be able to set messages, generate + // signatures, and provide canonical bytes to sign over. The transaction must + // also know how to encode itself. + TxBuilder interface { + GetTx() + +signing.Tx + + SetMsgs(msgs ...sdk.Msg) + +error + SetSignatures(signatures ...signingtypes.SignatureV2) + +error + SetMemo(memo string) + +SetFeeAmount(amount sdk.Coins) + +SetFeePayer(feePayer sdk.AccAddress) + +SetGasLimit(limit uint64) + +SetTimeoutHeight(height uint64) + +SetTimeoutTimestamp(timestamp time.Time) + +SetUnordered(v bool) + +SetFeeGranter(feeGranter sdk.AccAddress) + +AddAuxSignerData(tx.AuxSignerData) + +error +} + + // ExtendedTxBuilder extends the TxBuilder interface, + // which is used to set extension options to be included in a transaction. + ExtendedTxBuilder interface { + SetExtensionOptions(extOpts ...*codectypes.Any) +} +) +``` + +```go expandable +import ( + + banktypes "github.com/cosmos/cosmos-sdk/x/bank/types" +) + +func sendTx() + +error { + // --snip-- + + // Define two x/bank MsgSend messages: + // - from addr1 to addr3 + // - from addr2 to addr3 + // This means that the transaction needs two signers: addr1 and addr2. + msg1 := banktypes.NewMsgSend(addr1, addr3, sdk.NewCoins(sdk.NewInt64Coin("atom", 12))) + msg2 := banktypes.NewMsgSend(addr2, addr3, sdk.NewCoins(sdk.NewInt64Coin("atom", 34))) + err := txBuilder.SetMsgs(msg1, msg2) + if err != nil { + return err +} + +txBuilder.SetGasLimit(...) + +txBuilder.SetFeeAmount(...) + +txBuilder.SetMemo(...) + +txBuilder.SetTimeoutHeight(...) +} +``` + +At this point, `TxBuilder`'s underlying transaction is ready to be signed. + +#### Generating an Unordered Transaction + +Starting with Cosmos SDK v0.53.0, users may send unordered transactions to chains that have the feature enabled. + + + +Unordered transactions MUST leave sequence values unset. When a transaction is both unordered and contains a non-zero sequence value, +the transaction will be rejected. External services that operate on prior assumptions about transaction sequence values should be updated to handle unordered transactions. +Services should be aware that when the transaction is unordered, the transaction sequence will always be zero. + + + +Using the example above, we can set the required fields to mark a transaction as unordered. +By default, unordered transactions charge an extra 2240 units of gas to offset the additional storage overhead that supports their functionality. +The extra units of gas are customizable and therefore vary by chain, so be sure to check the chain's ante handler for the gas value set, if any. + +```go +func sendTx() + +error { + // --snip-- + expiration := 5 * time.Minute + txBuilder.SetUnordered(true) + +txBuilder.SetTimeoutTimestamp(time.Now().Add(expiration + (1 * time.Nanosecond))) +} +``` + +Unordered transactions from the same account must use a unique timeout timestamp value. The difference between each timeout timestamp value may be as small as a nanosecond, however. + +```go expandable +import ( + + "github.com/cosmos/cosmos-sdk/client" +) + +func sendMessages(txBuilders []client.TxBuilder) + +error { + // --snip-- + expiration := 5 * time.Minute + for _, txb := range txBuilders { + txb.SetUnordered(true) + +txb.SetTimeoutTimestamp(time.Now().Add(expiration + (1 * time.Nanosecond))) +} +} +``` + +### Signing a Transaction + +The encoding config is set to use Protobuf, which will use `SIGN_MODE_DIRECT` by default. As per [ADR-020](https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-020-protobuf-transaction-encoding.md), each signer needs to sign the `SignerInfo`s of all other signers. This means that two steps must be performed sequentially: + +* for each signer, populate the signer's `SignerInfo` inside `TxBuilder` +* once all `SignerInfo`s are populated, for each signer, sign the `SignDoc` (the payload to be signed). + +In the current `TxBuilder`'s API, both steps are done using the same method: `SetSignatures()`. The current API requires a first round of `SetSignatures()` *with empty signatures*, only to populate `SignerInfo`s, and a second round of `SetSignatures()` to actually sign the correct payload. + +```go expandable +import ( + + cryptotypes "github.com/cosmos/cosmos-sdk/crypto/types" + "github.com/cosmos/cosmos-sdk/types/tx/signing" + xauthsigning "github.com/cosmos/cosmos-sdk/x/auth/signing" +) + +func sendTx() + +error { + // --snip-- + privs := []cryptotypes.PrivKey{ + priv1, priv2 +} + accNums:= []uint64{..., ... +} // The accounts' account numbers + accSeqs:= []uint64{..., ... +} // The accounts' sequence numbers + + // First round: we gather all the signer infos. We use the "set empty + // signature" hack to do that. + var sigsV2 []signing.SignatureV2 + for i, priv := range privs { + sigV2 := signing.SignatureV2{ + PubKey: priv.PubKey(), + Data: &signing.SingleSignatureData{ + SignMode: encCfg.TxConfig.SignModeHandler().DefaultMode(), + Signature: nil, +}, + Sequence: accSeqs[i], +} + +sigsV2 = append(sigsV2, sigV2) +} + err := txBuilder.SetSignatures(sigsV2...) + if err != nil { + return err +} + + // Second round: all signer infos are set, so each signer can sign. + sigsV2 = []signing.SignatureV2{ +} + for i, priv := range privs { + signerData := xauthsigning.SignerData{ + ChainID: chainID, + AccountNumber: accNums[i], + Sequence: accSeqs[i], +} + +sigV2, err := tx.SignWithPrivKey( + encCfg.TxConfig.SignModeHandler().DefaultMode(), signerData, + txBuilder, priv, encCfg.TxConfig, accSeqs[i]) + if err != nil { + return nil, err +} + +sigsV2 = append(sigsV2, sigV2) +} + +err = txBuilder.SetSignatures(sigsV2...) + if err != nil { + return err +} +} +``` + +The `TxBuilder` is now correctly populated. To print it, you can use the `TxConfig` interface from the initial encoding config `encCfg`: + +```go expandable +func sendTx() + +error { + // --snip-- + + // Generated Protobuf-encoded bytes. + txBytes, err := encCfg.TxConfig.TxEncoder()(txBuilder.GetTx()) + if err != nil { + return err +} + + // Generate a JSON string. + txJSONBytes, err := encCfg.TxConfig.TxJSONEncoder()(txBuilder.GetTx()) + if err != nil { + return err +} + txJSON := string(txJSONBytes) +} +``` + +### Broadcasting a Transaction + +The preferred way to broadcast a transaction is to use gRPC, though using REST (via `gRPC-gateway`) or the CometBFT RPC is also possible. An overview of the differences between these methods is exposed [here](/sdk/v0.54/learn/concepts/cli-grpc-rest). For this tutorial, we will only describe the gRPC method. + +```go expandable +import ( + + "context" + "fmt" + "google.golang.org/grpc" + "github.com/cosmos/cosmos-sdk/types/tx" +) + +func sendTx(ctx context.Context) + +error { + // --snip-- + + // Create a connection to the gRPC server. + grpcConn, err := grpc.Dial( + "127.0.0.1:9090", // Or your gRPC server address. + grpc.WithInsecure(), // The Cosmos SDK doesn't support any transport security mechanisms. + ) + if err != nil { + return err + } + +defer grpcConn.Close() + + // Broadcast the tx via gRPC. We create a new client for the Protobuf Tx + // service. + txClient := tx.NewServiceClient(grpcConn) + // We then call the BroadcastTx method on this client. + grpcRes, err := txClient.BroadcastTx( + ctx, + &tx.BroadcastTxRequest{ + Mode: tx.BroadcastMode_BROADCAST_MODE_SYNC, + TxBytes: txBytes, // Proto-binary of the signed transaction, see previous step. +}, + ) + if err != nil { + return err +} + +fmt.Println(grpcRes.TxResponse.Code) // Should be `0` if the tx is successful + + return nil +} +``` + +#### Simulating a Transaction + +Before broadcasting a transaction, we sometimes may want to dry-run the transaction to estimate some information about the transaction without actually committing it. This is called simulating a transaction, and can be done as follows: + +```go expandable +import ( + + "context" + "fmt" + "testing" + "github.com/cosmos/cosmos-sdk/client" + "github.com/cosmos/cosmos-sdk/types/tx" + authtx "github.com/cosmos/cosmos-sdk/x/auth/tx" +) + +func simulateTx() + +error { + // --snip-- + + // Simulate the tx via gRPC. We create a new client for the Protobuf Tx + // service. + txClient := tx.NewServiceClient(grpcConn) + txBytes := /* Fill in with your signed transaction bytes. */ + + // We then call the Simulate method on this client. + grpcRes, err := txClient.Simulate( + context.Background(), + &tx.SimulateRequest{ + TxBytes: txBytes, +}, + ) + if err != nil { + return err +} + +fmt.Println(grpcRes.GasInfo) // Prints estimated gas used. + + return nil +} +``` + +## Using gRPC + +It is not possible to generate or sign a transaction using gRPC, only to broadcast one. In order to broadcast a transaction using gRPC, you will need to generate, sign, and encode the transaction using either the CLI or programmatically with Go. + +### Broadcasting a Transaction + +Broadcasting a transaction using the gRPC endpoint can be done by sending a `BroadcastTx` request as follows, where the `txBytes` are the protobuf-encoded bytes of a signed transaction: + +```bash +grpcurl -plaintext \ + -d '{"tx_bytes":"{{txBytes}}","mode":"BROADCAST_MODE_SYNC"}' \ + localhost:9090 \ + cosmos.tx.v1beta1.Service/BroadcastTx +``` + +## Using REST + +It is not possible to generate or sign a transaction using REST, only to broadcast one. In order to broadcast a transaction using REST, you will need to generate, sign, and encode the transaction using either the CLI or programmatically with Go. + +### Broadcasting a Transaction + +Broadcasting a transaction using the REST endpoint (served by `gRPC-gateway`) can be done by sending a POST request as follows, where the `txBytes` are the protobuf-encoded bytes of a signed transaction: + +```bash +curl -X POST \ + -H "Content-Type: application/json" \ + -d'{"tx_bytes":"{{txBytes}}","mode":"BROADCAST_MODE_SYNC"}' \ + localhost:1317/cosmos/tx/v1beta1/txs +``` + +## Using CosmJS (JavaScript & TypeScript) + +CosmJS aims to build client libraries in JavaScript that can be embedded in web applications. Please see [Link](https://cosmos.github.io/cosmjs) for more information. + +## Congratulations! + +You have learned how to manually generate, sign, and broadcast transactions using the Cosmos SDK. These workflows provide the foundation for building custom transaction tools and integrations. + +## Next steps + +- [Run in production](/sdk/v0.54/node/run-production) for security and deployment best practices. +- [Run a testnet](/sdk/v0.54/node/run-testnet) to test your blockchain. + diff --git a/sdk/v0.54/reference/architecture.mdx b/sdk/v0.54/reference/architecture.mdx new file mode 100644 index 000000000..19172ebf9 --- /dev/null +++ b/sdk/v0.54/reference/architecture.mdx @@ -0,0 +1,91 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/architecture' +title: "Architecture Decision Records (ADR)" +description: "Version: v0.54" +--- + +This is a location to record all high-level architecture decisions in the Cosmos-SDK. + +An Architectural Decision (**AD**) is a software design choice that addresses a functional or non-functional requirement that is architecturally significant. An Architecturally Significant Requirement (**ASR**) is a requirement that has a measurable effect on a software system’s architecture and quality. An Architectural Decision Record (**ADR**) captures a single AD, such as often done when writing personal notes or meeting minutes; the collection of ADRs created and maintained in a project constitute its decision log. All these are within the topic of Architectural Knowledge Management (AKM). + +You can read more about the ADR concept in this [blog post](https://product.reverb.com/documenting-architecture-decisions-the-reverb-way-a3563bb24bd0#.78xhdix6t). + +## Rationale[​](#rationale "Direct link to Rationale") + +ADRs are intended to be the primary mechanism for proposing new feature designs and new processes, for collecting community input on an issue, and for documenting the design decisions. An ADR should provide: + +* Context on the relevant goals and the current state +* Proposed changes to achieve the goals +* Summary of pros and cons +* References +* Changelog + +Note the distinction between an ADR and a spec. The ADR provides the context, intuition, reasoning, and justification for a change in architecture, or for the architecture of something new. The spec is a much more compressed and streamlined summary of everything as it stands today. + +If recorded decisions turned out to be lacking, convene a discussion, record the new decisions here, and then modify the code to match. + +## Creating new ADR[​](#creating-new-adr "Direct link to Creating new ADR") + +Read about the [PROCESS](/sdk/v0.54/reference/architecture/PROCESS). + +### Use RFC 2119 Keywords[​](#use-rfc-2119-keywords "Direct link to Use RFC 2119 Keywords") + +When writing ADRs, follow the same best practices for writing RFCs. When writing RFCs, key words are used to signify the requirements in the specification. These words are often capitalized: "MUST," "MUST NOT," "REQUIRED," "SHALL," "SHALL NOT," "SHOULD," "SHOULD NOT," "RECOMMENDED," "MAY," and "OPTIONAL." They are to be interpreted as described in [RFC 2119](https://datatracker.ietf.org/doc/html/rfc2119). + +## ADR Table of Contents[​](#adr-table-of-contents "Direct link to ADR Table of Contents") + +### Accepted[​](#accepted "Direct link to Accepted") + +* [ADR 002: SDK Documentation Structure](/sdk/v0.54/reference/architecture/adr-002-docs-structure) +* [ADR 004: Split Denomination Keys](/sdk/v0.54/reference/architecture/adr-004-split-denomination-keys) +* [ADR 006: Secret Store Replacement](/sdk/v0.54/reference/architecture/adr-006-secret-store-replacement) +* [ADR 009: Evidence Module](/sdk/v0.54/reference/architecture/adr-009-evidence-module) +* [ADR 010: Modular AnteHandler](/sdk/v0.54/reference/architecture/adr-010-modular-antehandler) +* [ADR 019: Protocol Buffer State Encoding](/sdk/v0.54/reference/architecture/adr-019-protobuf-state-encoding) +* [ADR 020: Protocol Buffer Transaction Encoding](/sdk/v0.54/reference/architecture/adr-020-protobuf-transaction-encoding) +* [ADR 021: Protocol Buffer Query Encoding](/sdk/v0.54/reference/architecture/adr-021-protobuf-query-encoding) +* [ADR 023: Protocol Buffer Naming and Versioning](/sdk/v0.54/reference/architecture/adr-023-protobuf-naming) +* [ADR 029: Fee Grant Module](/sdk/v0.54/reference/architecture/adr-029-fee-grant-module) +* [ADR 030: Message Authorization Module](/sdk/v0.54/reference/architecture/adr-030-authz-module) +* [ADR 031: Protobuf Msg Services](/sdk/v0.54/reference/architecture/adr-031-msg-service) +* [ADR 055: ORM](/sdk/v0.54/reference/architecture/adr-055-orm) +* [ADR 058: Auto-Generated CLI](/sdk/v0.54/reference/architecture/adr-058-auto-generated-cli) +* [ADR 060: ABCI 1.0 (Phase I)](/sdk/v0.54/reference/architecture/adr-060-abci-1.0) +* [ADR 061: Liquid Staking](/sdk/v0.54/reference/architecture/adr-061-liquid-staking) + +### Proposed[​](#proposed "Direct link to Proposed") + +* [ADR 003: Dynamic Capability Store](/sdk/v0.54/reference/architecture/adr-003-dynamic-capability-store) +* [ADR 011: Generalize Genesis Accounts](/sdk/v0.54/reference/architecture/adr-011-generalize-genesis-accounts) +* [ADR 012: State Accessors](/sdk/v0.54/reference/architecture/adr-012-state-accessors) +* [ADR 013: Metrics](/sdk/v0.54/reference/architecture/adr-013-metrics) +* [ADR 016: Validator Consensus Key Rotation](/sdk/v0.54/reference/architecture/adr-016-validator-consensus-key-rotation) +* [ADR 017: Historical Header Module](/sdk/v0.54/reference/architecture/adr-017-historical-header-module) +* [ADR 018: Extendable Voting Periods](/sdk/v0.54/reference/architecture/adr-018-extendable-voting-period) +* [ADR 022: Custom baseapp panic handling](/sdk/v0.54/reference/architecture/adr-022-custom-panic-handling) +* [ADR 024: Coin Metadata](/sdk/v0.54/reference/architecture/adr-024-coin-metadata) +* [ADR 027: Deterministic Protobuf Serialization](/sdk/v0.54/reference/architecture/adr-027-deterministic-protobuf-serialization) +* [ADR 028: Public Key Addresses](/sdk/v0.54/reference/architecture/adr-028-public-key-addresses) +* [ADR 032: Typed Events](/sdk/v0.54/reference/architecture/adr-032-typed-events) +* [ADR 033: Inter-module RPC](/sdk/v0.54/reference/architecture/adr-033-protobuf-inter-module-comm) +* [ADR 035: Rosetta API Support](/sdk/v0.54/reference/architecture/adr-035-rosetta-api-support) +* [ADR 037: Governance Split Votes](/sdk/v0.54/reference/architecture/adr-037-gov-split-vote) +* [ADR 038: State Listening](/sdk/v0.54/reference/architecture/adr-038-state-listening) +* [ADR 039: Epoched Staking](/sdk/v0.54/reference/architecture/adr-039-epoched-staking) +* [ADR 040: Storage and SMT State Commitments](/sdk/v0.54/reference/architecture/adr-040-storage-and-smt-state-commitments) +* [ADR 046: Module Params](/sdk/v0.54/reference/architecture/adr-046-module-params) +* [ADR 054: Semver Compatible SDK Modules](/sdk/v0.54/reference/architecture/adr-054-semver-compatible-modules) +* [ADR 057: App Wiring](/sdk/v0.54/reference/architecture/adr-057-app-wiring) +* [ADR 059: Test Scopes](/sdk/v0.54/reference/architecture/adr-059-test-scopes) +* [ADR 062: Collections State Layer](/sdk/v0.54/reference/architecture/adr-062-collections-state-layer) +* [ADR 063: Core Module API](/sdk/v0.54/reference/architecture/adr-063-core-module-api) +* [ADR 065: Store V2](/sdk/v0.54/reference/architecture/adr-065-store-v2) +* [ADR 076: Transaction Malleability Risk Review and Recommendations](/sdk/v0.54/reference/architecture/adr-076-tx-malleability) + +### Draft[​](#draft "Direct link to Draft") + +* [ADR 044: Guidelines for Updating Protobuf Definitions](/sdk/v0.54/reference/architecture/adr-044-protobuf-updates-guidelines) +* [ADR 047: Extend Upgrade Plan](/sdk/v0.54/reference/architecture/adr-047-extend-upgrade-plan) +* [ADR 053: Go Module Refactoring](/sdk/v0.54/reference/architecture/adr-053-go-module-refactoring) +* [ADR 068: Preblock](/sdk/v0.54/reference/architecture/adr-068-preblock) diff --git a/sdk/v0.54/reference/architecture/PROCESS.mdx b/sdk/v0.54/reference/architecture/PROCESS.mdx new file mode 100644 index 000000000..bb4942c0e --- /dev/null +++ b/sdk/v0.54/reference/architecture/PROCESS.mdx @@ -0,0 +1,62 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/architecture/PROCESS' +title: ADR Creation Process +--- + +1. Copy the `adr-template.md` file. Use the following filename pattern: `adr-next_number-title.md` +2. Create a draft Pull Request if you want to get early feedback. +3. Make sure the context and solution are clear and well documented. +4. Add an entry to the list in the [README](/sdk/v0.54/reference/architecture/README) file. +5. Create a Pull Request to propose a new ADR. + +## What is an ADR? + +An ADR is a document to document an implementation and design that may or may not have been discussed in an RFC. While an RFC is meant to replace synchronous communication in a distributed environment, an ADR is meant to document an already made decision. An ADR won't come with much of a communication overhead because the discussion was recorded in an RFC or a synchronous discussion. If the consensus came from a synchronous discussion, then a short excerpt should be added to the ADR to explain the goals. + +## ADR life cycle + +ADR creation is an **iterative** process. Instead of having a high amount of communication overhead, an ADR is used when there is already a decision made and implementation details need to be added. The ADR should document what the collective consensus for the specific issue is and how to solve it. + +1. Every ADR should start with either an RFC or a discussion where consensus has been met. + +2. Once consensus is met, a GitHub Pull Request (PR) is created with a new document based on the `adr-template.md`. + +3. If a *proposed* ADR is merged, then it should clearly document outstanding issues either in ADR document notes or in a GitHub Issue. + +4. The PR SHOULD always be merged. In the case of a faulty ADR, we still prefer to merge it with a *rejected* status. The only time the ADR SHOULD NOT be merged is if the author abandons it. + +5. Merged ADRs SHOULD NOT be pruned. + +### ADR status + +Status has two components: + +```text +{CONSENSUS STATUS} {IMPLEMENTATION STATUS} +``` + +IMPLEMENTATION STATUS is either `Implemented` or `Not Implemented`. + +#### Consensus Status + +```text +DRAFT -> PROPOSED -> LAST CALL yyyy-mm-dd -> ACCEPTED | REJECTED -> SUPERSEDED by ADR-xxx + \ | + \ | + v v + ABANDONED +``` + +* `DRAFT`: \[optional] an ADR which is a work in progress, not being ready for a general review. This is to present an early work and get early feedback in a Draft Pull Request form. +* `PROPOSED`: an ADR covering a full solution architecture and still in the review - project stakeholders haven't reached an agreement yet. +* `LAST CALL `: \[optional] Notifies that we are close to accepting updates. Changing a status to `LAST CALL` means that social consensus (of Cosmos SDK maintainers) has been reached, and we still want to give it a time to let the community react or analyze. +* `ACCEPTED`: ADR which will represent a currently implemented or to be implemented architecture design. +* `REJECTED`: ADR can go from PROPOSED or ACCEPTED to rejected if the consensus among project stakeholders will decide so. +* `SUPERSEDED by ADR-xxx`: ADR which has been superseded by a new ADR. +* `ABANDONED`: the ADR is no longer pursued by the original authors. + +## Language used in ADR + +* The context/background should be written in the present tense. +* Avoid using a first, personal form. diff --git a/sdk/v0.54/reference/architecture/README.mdx b/sdk/v0.54/reference/architecture/README.mdx new file mode 100644 index 000000000..4ac4e07e1 --- /dev/null +++ b/sdk/v0.54/reference/architecture/README.mdx @@ -0,0 +1,99 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/architecture/README' +title: Architecture Decision Records (ADR) +description: >- + This is a location to record all high-level architecture decisions in the + Cosmos-SDK. +--- + +This is a location to record all high-level architecture decisions in the Cosmos-SDK. + +An Architectural Decision (**AD**) is a software design choice that addresses a functional or non-functional requirement that is architecturally significant. +An Architecturally Significant Requirement (**ASR**) is a requirement that has a measurable effect on a software system’s architecture and quality. +An Architectural Decision Record (**ADR**) captures a single AD, such as often done when writing personal notes or meeting minutes; the collection of ADRs created and maintained in a project constitute its decision log. All these are within the topic of Architectural Knowledge Management (AKM). + +You can read more about the ADR concept in this [blog post](https://product.reverb.com/documenting-architecture-decisions-the-reverb-way-a3563bb24bd0#.78xhdix6t). + +## Rationale + +ADRs are intended to be the primary mechanism for proposing new feature designs and new processes, for collecting community input on an issue, and for documenting the design decisions. +An ADR should provide: + +* Context on the relevant goals and the current state +* Proposed changes to achieve the goals +* Summary of pros and cons +* References +* Changelog + +Note the distinction between an ADR and a spec. The ADR provides the context, intuition, reasoning, and +justification for a change in architecture, or for the architecture of something +new. The spec is a much more compressed and streamlined summary of everything as +it stands today. + +If recorded decisions turned out to be lacking, convene a discussion, record the new decisions here, and then modify the code to match. + +## Creating new ADR + +Read about the [PROCESS](/sdk/v0.54/reference/architecture/PROCESS). + +### Use RFC 2119 Keywords + +When writing ADRs, follow the same best practices for writing RFCs. When writing RFCs, key words are used to signify the requirements in the specification. These words are often capitalized: "MUST," "MUST NOT," "REQUIRED," "SHALL," "SHALL NOT," "SHOULD," "SHOULD NOT," "RECOMMENDED," "MAY," and "OPTIONAL." They are to be interpreted as described in [RFC 2119](https://datatracker.ietf.org/doc/html/rfc2119). + +## ADR Table of Contents + +### Accepted + +* [ADR 002: SDK Documentation Structure](/sdk/v0.54/reference/architecture/adr-002-docs-structure) +* [ADR 004: Split Denomination Keys](/sdk/v0.54/reference/architecture/adr-004-split-denomination-keys) +* [ADR 006: Secret Store Replacement](/sdk/v0.54/reference/architecture/adr-006-secret-store-replacement) +* [ADR 009: Evidence Module](/sdk/v0.54/reference/architecture/adr-009-evidence-module) +* [ADR 010: Modular AnteHandler](/sdk/v0.54/reference/architecture/adr-010-modular-antehandler) +* [ADR 019: Protocol Buffer State Encoding](/sdk/v0.54/reference/architecture/adr-019-protobuf-state-encoding) +* [ADR 020: Protocol Buffer Transaction Encoding](/sdk/v0.54/reference/architecture/adr-020-protobuf-transaction-encoding) +* [ADR 021: Protocol Buffer Query Encoding](/sdk/v0.54/reference/architecture/adr-021-protobuf-query-encoding) +* [ADR 023: Protocol Buffer Naming and Versioning](/sdk/v0.54/reference/architecture/adr-023-protobuf-naming) +* [ADR 029: Fee Grant Module](/sdk/v0.54/reference/architecture/adr-029-fee-grant-module) +* [ADR 030: Message Authorization Module](/sdk/v0.54/reference/architecture/adr-030-authz-module) +* [ADR 031: Protobuf Msg Services](/sdk/v0.54/reference/architecture/adr-031-msg-service) +* [ADR 055: ORM](/sdk/v0.54/reference/architecture/adr-055-orm) +* [ADR 058: Auto-Generated CLI](/sdk/v0.54/reference/architecture/adr-058-auto-generated-cli) +* [ADR 060: ABCI 1.0 (Phase I)](/sdk/v0.54/reference/architecture/adr-060-abci-1.0) +* [ADR 061: Liquid Staking](/sdk/v0.54/reference/architecture/adr-061-liquid-staking) + +### Proposed + +* [ADR 003: Dynamic Capability Store](/sdk/v0.54/reference/architecture/adr-003-dynamic-capability-store) +* [ADR 011: Generalize Genesis Accounts](/sdk/v0.54/reference/architecture/adr-011-generalize-genesis-accounts) +* [ADR 012: State Accessors](/sdk/v0.54/reference/architecture/adr-012-state-accessors) +* [ADR 013: Metrics](/sdk/v0.54/reference/architecture/adr-013-metrics) +* [ADR 016: Validator Consensus Key Rotation](/sdk/v0.54/reference/architecture/adr-016-validator-consensus-key-rotation) +* [ADR 017: Historical Header Module](/sdk/v0.54/reference/architecture/adr-017-historical-header-module) +* [ADR 018: Extendable Voting Periods](/sdk/v0.54/reference/architecture/adr-018-extendable-voting-period) +* [ADR 022: Custom baseapp panic handling](/sdk/v0.54/reference/architecture/adr-022-custom-panic-handling) +* [ADR 024: Coin Metadata](/sdk/v0.54/reference/architecture/adr-024-coin-metadata) +* [ADR 027: Deterministic Protobuf Serialization](/sdk/v0.54/reference/architecture/adr-027-deterministic-protobuf-serialization) +* [ADR 028: Public Key Addresses](/sdk/v0.54/reference/architecture/adr-028-public-key-addresses) +* [ADR 032: Typed Events](/sdk/v0.54/reference/architecture/adr-032-typed-events) +* [ADR 033: Inter-module RPC](/sdk/v0.54/reference/architecture/adr-033-protobuf-inter-module-comm) +* [ADR 035: Rosetta API Support](/sdk/v0.54/reference/architecture/adr-035-rosetta-api-support) +* [ADR 037: Governance Split Votes](/sdk/v0.54/reference/architecture/adr-037-gov-split-vote) +* [ADR 038: State Listening](/sdk/v0.54/reference/architecture/adr-038-state-listening) +* [ADR 039: Epoched Staking](/sdk/v0.54/reference/architecture/adr-039-epoched-staking) +* [ADR 040: Storage and SMT State Commitments](/sdk/v0.54/reference/architecture/adr-040-storage-and-smt-state-commitments) +* [ADR 046: Module Params](/sdk/v0.54/reference/architecture/adr-046-module-params) +* [ADR 054: Semver Compatible SDK Modules](/sdk/v0.54/reference/architecture/adr-054-semver-compatible-modules) +* [ADR 057: App Wiring](/sdk/v0.54/reference/architecture/adr-057-app-wiring) +* [ADR 059: Test Scopes](/sdk/v0.54/reference/architecture/adr-059-test-scopes) +* [ADR 062: Collections State Layer](/sdk/v0.54/reference/architecture/adr-062-collections-state-layer) +* [ADR 063: Core Module API](/sdk/v0.54/reference/architecture/adr-063-core-module-api) +* [ADR 065: Store V2](/sdk/v0.54/reference/architecture/adr-065-store-v2) +* [ADR 076: Transaction Malleability Risk Review and Recommendations](/sdk/v0.54/reference/architecture/adr-076-tx-malleability) + +### Draft + +* [ADR 044: Guidelines for Updating Protobuf Definitions](/sdk/v0.54/reference/architecture/adr-044-protobuf-updates-guidelines) +* [ADR 047: Extend Upgrade Plan](/sdk/v0.54/reference/architecture/adr-047-extend-upgrade-plan) +* [ADR 053: Go Module Refactoring](/sdk/v0.54/reference/architecture/adr-053-go-module-refactoring) +* [ADR 068: Preblock](/sdk/v0.54/reference/architecture/adr-068-preblock) diff --git a/sdk/v0.54/reference/architecture/adr-002-docs-structure.mdx b/sdk/v0.54/reference/architecture/adr-002-docs-structure.mdx new file mode 100644 index 000000000..69122acae --- /dev/null +++ b/sdk/v0.54/reference/architecture/adr-002-docs-structure.mdx @@ -0,0 +1,94 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/architecture/adr-002-docs-structure' +title: 'ADR 002: SDK Documentation Structure' +description: >- + There is a need for a scalable structure of the Cosmos SDK documentation. + Current documentation includes a lot of non-related Cosmos SDK material, is + difficult to maintain and hard to follow as a user. +--- + +## Context + +There is a need for a scalable structure of the Cosmos SDK documentation. Current documentation includes a lot of non-related Cosmos SDK material, is difficult to maintain and hard to follow as a user. + +Ideally, we would have: + +* All docs related to dev frameworks or tools live in their respective github repos (sdk repo would contain sdk docs, hub repo would contain hub docs, lotion repo would contain lotion docs, etc.) +* All other docs (faqs, whitepaper, high-level material about Cosmos) would live on the website. + +## Decision + +Re-structure the `/docs` folder of the Cosmos SDK github repo as follows: + +```text expandable +docs/ +├── README +├── intro/ +├── concepts/ +│ ├── baseapp +│ ├── types +│ ├── store +│ ├── server +│ ├── modules/ +│ │ ├── keeper +│ │ ├── handler +│ │ ├── cli +│ ├── gas +│ └── commands +├── clients/ +│ ├── lite/ +│ ├── service-providers +├── modules/ +├── spec/ +├── translations/ +└── architecture/ +``` + +The files in each sub-folder do not matter and will likely change. What matters is the sectioning: + +* `README`: Landing page of the docs. +* `intro`: Introductory material. Goal is to have a short explainer of the Cosmos SDK and then channel people to the resource they need. The [Cosmos SDK tutorial](https://github.com/cosmos/sdk-application-tutorial/) will be highlighted, as well as the `godocs`. +* `concepts`: Contains high-level explanations of the abstractions of the Cosmos SDK. It does not contain specific code implementation and does not need to be updated often. **It is not an API specification of the interfaces**. API spec is the `godoc`. +* `clients`: Contains specs and info about the various Cosmos SDK clients. +* `spec`: Contains specs of modules, and others. +* `modules`: Contains links to `godocs` and the spec of the modules. +* `architecture`: Contains architecture-related docs like the present one. +* `translations`: Contains different translations of the documentation. + +Website docs sidebar will only include the following sections: + +* `README` +* `intro` +* `concepts` +* `clients` + +`architecture` need not be displayed on the website. + +## Status + +Accepted + +## Consequences + +### Positive + +* Much clearer organization of the Cosmos SDK docs. +* The `/docs` folder now only contains Cosmos SDK and gaia related material. Later, it will only contain Cosmos SDK related material. +* Developers only have to update `/docs` folder when they open a PR (and not `/examples` for example). +* Easier for developers to find what they need to update in the docs thanks to reworked architecture. +* Cleaner vuepress build for website docs. +* Will help build an executable doc (cf [Link](https://github.com/cosmos/cosmos-sdk/issues/2611)) + +### Neutral + +* We need to move a bunch of deprecated stuff to `/_attic` folder. +* We need to integrate content in `sdk/docs/core` in `concepts`. +* We need to move all the content that currently lives in `docs` and does not fit in new structure (like `lotion`, intro material, whitepaper) to the website repository. +* Update `DOCS_README.md` + +## References + +* [Link](https://github.com/cosmos/cosmos-sdk/issues/1460) +* [Link](https://github.com/cosmos/cosmos-sdk/pull/2695) +* [Link](https://github.com/cosmos/cosmos-sdk/issues/2611) diff --git a/sdk/v0.54/reference/architecture/adr-003-dynamic-capability-store.mdx b/sdk/v0.54/reference/architecture/adr-003-dynamic-capability-store.mdx new file mode 100644 index 000000000..6deb163a4 --- /dev/null +++ b/sdk/v0.54/reference/architecture/adr-003-dynamic-capability-store.mdx @@ -0,0 +1,394 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/architecture/adr-003-dynamic-capability-store' +title: 'ADR 003: Dynamic Capability Store' +description: '12 December 2019: Initial version 02 April 2020: Memory Store Revisions' +--- + +## Changelog + +* 12 December 2019: Initial version +* 02 April 2020: Memory Store Revisions + +## Context + +Full implementation of the [IBC specification](https://github.com/cosmos/ibc) requires the ability to create and authenticate object-capability keys at runtime (i.e., during transaction execution), +as described in [ICS 5](https://github.com/cosmos/ibc/tree/master/spec/core/ics-005-port-allocation#technical-specification). In the IBC specification, capability keys are created for each newly initialized +port & channel, and are used to authenticate future usage of the port or channel. Since channels and potentially ports can be initialized during transaction execution, the state machine must be able to create +object-capability keys at this time. + +At present, the Cosmos SDK does not have the ability to do this. Object-capability keys are currently pointers (memory addresses) of `StoreKey` structs created at application initialisation in `app.go` ([example](https://github.com/cosmos/gaia/blob/dcbddd9f04b3086c0ad07ee65de16e7adedc7da4/app/app.go#L132)) +and passed to Keepers as fixed arguments ([example](https://github.com/cosmos/gaia/blob/dcbddd9f04b3086c0ad07ee65de16e7adedc7da4/app/app.go#L160)). Keepers cannot create or store capability keys during transaction execution — although they could call `NewKVStoreKey` and take the memory address +of the returned struct, storing this in the Merklised store would result in a consensus fault, since the memory address will be different on each machine (this is intentional — were this not the case, the keys would be predictable and couldn't serve as object capabilities). + +Keepers need a way to keep a private map of store keys which can be altered during transaction execution, along with a suitable mechanism for regenerating the unique memory addresses (capability keys) in this map whenever the application is started or restarted, along with a mechanism to revert capability creation on tx failure. +This ADR proposes such an interface & mechanism. + +## Decision + +The Cosmos SDK will include a new `CapabilityKeeper` abstraction, which is responsible for provisioning, +tracking, and authenticating capabilities at runtime. During application initialisation in `app.go`, +the `CapabilityKeeper` will be hooked up to modules through unique function references +(by calling `ScopeToModule`, defined below) so that it can identify the calling module when later +invoked. + +When the initial state is loaded from disk, the `CapabilityKeeper`'s `Initialize` function will create +new capability keys for all previously allocated capability identifiers (allocated during execution of +past transactions and assigned to particular modes), and keep them in a memory-only store while the +chain is running. + +The `CapabilityKeeper` will include a persistent `KVStore`, a `MemoryStore`, and an in-memory map. +The persistent `KVStore` tracks which capability is owned by which modules. +The `MemoryStore` stores a forward mapping that map from module name, capability tuples to capability names and +a reverse mapping that map from module name, capability name to the capability index. +Since we cannot marshal the capability into a `KVStore` and unmarshal without changing the memory location of the capability, +the reverse mapping in the KVStore will simply map to an index. This index can then be used as a key in the ephemeral +go-map to retrieve the capability at the original memory location. + +The `CapabilityKeeper` will define the following types & functions: + +The `Capability` is similar to `StoreKey`, but has a globally unique `Index()` instead of +a name. A `String()` method is provided for debugging. + +A `Capability` is simply a struct, the address of which is taken for the actual capability. + +```go +type Capability struct { + index uint64 +} +``` + +A `CapabilityKeeper` contains a persistent store key, memory store key, and mapping of allocated module names. + +```go +type CapabilityKeeper struct { + persistentKey StoreKey + memKey StoreKey + capMap map[uint64]*Capability + moduleNames map[string]interface{ +} + +sealed bool +} +``` + +The `CapabilityKeeper` provides the ability to create *scoped* sub-keepers which are tied to a +particular module name. These `ScopedCapabilityKeeper`s must be created at application initialisation +and passed to modules, which can then use them to claim capabilities they receive and retrieve +capabilities which they own by name, in addition to creating new capabilities & authenticating capabilities +passed by other modules. + +```go +type ScopedCapabilityKeeper struct { + persistentKey StoreKey + memKey StoreKey + capMap map[uint64]*Capability + moduleName string +} +``` + +`ScopeToModule` is used to create a scoped sub-keeper with a particular name, which must be unique. +It MUST be called before `InitializeAndSeal`. + +```go expandable +func (ck CapabilityKeeper) + +ScopeToModule(moduleName string) + +ScopedCapabilityKeeper { + if k.sealed { + panic("cannot scope to module via a sealed capability keeper") +} + if _, ok := k.scopedModules[moduleName]; ok { + panic(fmt.Sprintf("cannot create multiple scoped keepers for the same module name: %s", moduleName)) +} + +k.scopedModules[moduleName] = struct{ +}{ +} + +return ScopedKeeper{ + cdc: k.cdc, + storeKey: k.storeKey, + memKey: k.memKey, + capMap: k.capMap, + module: moduleName, +} +} +``` + +`InitializeAndSeal` MUST be called exactly once, after loading the initial state and creating all +necessary `ScopedCapabilityKeeper`s, in order to populate the memory store with newly-created +capability keys in accordance with the keys previously claimed by particular modules and prevent the +creation of any new `ScopedCapabilityKeeper`s. + +```go expandable +func (ck CapabilityKeeper) + +InitializeAndSeal(ctx Context) { + if ck.sealed { + panic("capability keeper is sealed") +} + persistentStore := ctx.KVStore(ck.persistentKey) + map := ctx.KVStore(ck.memKey) + + // initialise memory store for all names in persistent store + for index, value := range persistentStore.Iter() { + capability = &CapabilityKey{ + index: index +} + for moduleAndCapability := range value { + moduleName, capabilityName := moduleAndCapability.Split("/") + +memStore.Set(moduleName + "/fwd/" + capability, capabilityName) + +memStore.Set(moduleName + "/rev/" + capabilityName, index) + +ck.capMap[index] = capability +} + +} + +ck.sealed = true +} +``` + +`NewCapability` can be called by any module to create a new unique, unforgeable object-capability +reference. The newly created capability is automatically persisted; the calling module need not +call `ClaimCapability`. + +```go expandable +func (sck ScopedCapabilityKeeper) + +NewCapability(ctx Context, name string) (Capability, error) { + // check name not taken in memory store + if capStore.Get("rev/" + name) != nil { + return nil, errors.New("name already taken") +} + + // fetch the current index + index := persistentStore.Get("index") + + // create a new capability + capability := &CapabilityKey{ + index: index +} + + // set persistent store + persistentStore.Set(index, Set.singleton(sck.moduleName + "/" + name)) + + // update the index + index++ + persistentStore.Set("index", index) + + // set forward mapping in memory store from capability to name + memStore.Set(sck.moduleName + "/fwd/" + capability, name) + + // set reverse mapping in memory store from name to index + memStore.Set(sck.moduleName + "/rev/" + name, index) + + // set the in-memory mapping from index to capability pointer + capMap[index] = capability + + // return the newly created capability + return capability +} +``` + +`AuthenticateCapability` can be called by any module to check that a capability +does in fact correspond to a particular name (the name can be untrusted user input) +with which the calling module previously associated it. + +```go +func (sck ScopedCapabilityKeeper) + +AuthenticateCapability(name string, capability Capability) + +bool { + // return whether forward mapping in memory store matches name + return memStore.Get(sck.moduleName + "/fwd/" + capability) === name +} +``` + +`ClaimCapability` allows a module to claim a capability key which it has received from another module +so that future `GetCapability` calls will succeed. + +`ClaimCapability` MUST be called if a module which receives a capability wishes to access it by name +in the future. Capabilities are multi-owner, so if multiple modules have a single `Capability` reference, +they will all own it. + +```go expandable +func (sck ScopedCapabilityKeeper) + +ClaimCapability(ctx Context, capability Capability, name string) + +error { + persistentStore := ctx.KVStore(sck.persistentKey) + + // set forward mapping in memory store from capability to name + memStore.Set(sck.moduleName + "/fwd/" + capability, name) + + // set reverse mapping in memory store from name to capability + memStore.Set(sck.moduleName + "/rev/" + name, capability) + + // update owner set in persistent store + owners := persistentStore.Get(capability.Index()) + +owners.add(sck.moduleName + "/" + name) + +persistentStore.Set(capability.Index(), owners) +} +``` + +`GetCapability` allows a module to fetch a capability which it has previously claimed by name. +The module is not allowed to retrieve capabilities which it does not own. + +```go +func (sck ScopedCapabilityKeeper) + +GetCapability(ctx Context, name string) (Capability, error) { + // fetch the index of capability using reverse mapping in memstore + index := memStore.Get(sck.moduleName + "/rev/" + name) + + // fetch capability from go-map using index + capability := capMap[index] + + // return the capability + return capability +} +``` + +`ReleaseCapability` allows a module to release a capability which it had previously claimed. If no +more owners exist, the capability will be deleted globally. + +```go expandable +func (sck ScopedCapabilityKeeper) + +ReleaseCapability(ctx Context, capability Capability) + +err { + persistentStore := ctx.KVStore(sck.persistentKey) + name := capStore.Get(sck.moduleName + "/fwd/" + capability) + if name == nil { + return error("capability not owned by module") +} + + // delete forward mapping in memory store + memoryStore.Delete(sck.moduleName + "/fwd/" + capability, name) + + // delete reverse mapping in memory store + memoryStore.Delete(sck.moduleName + "/rev/" + name, capability) + + // update owner set in persistent store + owners := persistentStore.Get(capability.Index()) + +owners.remove(sck.moduleName + "/" + name) + if owners.size() > 0 { + // there are still other owners, keep the capability around + persistentStore.Set(capability.Index(), owners) +} + +else { + // no more owners, delete the capability + persistentStore.Delete(capability.Index()) + +delete(capMap[capability.Index()]) +} +} +``` + +### Usage patterns + +#### Initialisation + +Any modules which use dynamic capabilities must be provided a `ScopedCapabilityKeeper` in `app.go`: + +```go +ck := NewCapabilityKeeper(persistentKey, memoryKey) + +mod1Keeper := NewMod1Keeper(ck.ScopeToModule("mod1"), ....) + +mod2Keeper := NewMod2Keeper(ck.ScopeToModule("mod2"), ....) + +// other initialisation logic ... + +// load initial state... + +ck.InitializeAndSeal(initialContext) +``` + +#### Creating, passing, claiming and using capabilities + +Consider the case where `mod1` wants to create a capability, associate it with a resource (e.g. an IBC channel) by name, then pass it to `mod2` which will use it later: + +Module 1 would have the following code: + +```go +capability := scopedCapabilityKeeper.NewCapability(ctx, "resourceABC") + +mod2Keeper.SomeFunction(ctx, capability, args...) +``` + +`SomeFunction`, running in module 2, could then claim the capability: + +```go +func (k Mod2Keeper) + +SomeFunction(ctx Context, capability Capability) { + k.sck.ClaimCapability(ctx, capability, "resourceABC") + // other logic... +} +``` + +Later on, module 2 can retrieve that capability by name and pass it to module 1, which will authenticate it against the resource: + +```go +func (k Mod2Keeper) + +SomeOtherFunction(ctx Context, name string) { + capability := k.sck.GetCapability(ctx, name) + +mod1.UseResource(ctx, capability, "resourceABC") +} +``` + +Module 1 will then check that this capability key is authenticated to use the resource before allowing module 2 to use it: + +```go +func (k Mod1Keeper) + +UseResource(ctx Context, capability Capability, resource string) { + if !k.sck.AuthenticateCapability(name, capability) { + return errors.New("unauthenticated") +} + // do something with the resource +} +``` + +If module 2 passed the capability key to module 3, module 3 could then claim it and call module 1 just like module 2 did +(in which case module 1, module 2, and module 3 would all be able to use this capability). + +## Status + +Proposed. + +## Consequences + +### Positive + +* Dynamic capability support. +* Allows CapabilityKeeper to return same capability pointer from go-map while reverting any writes to the persistent `KVStore` and in-memory `MemoryStore` on tx failure. + +### Negative + +* Requires an additional keeper. +* Some overlap with existing `StoreKey` system (in the future they could be combined, since this is a superset functionality-wise). +* Requires an extra level of indirection in the reverse mapping, since MemoryStore must map to index which must then be used as key in a go map to retrieve the actual capability + +### Neutral + +(none known) + +## References + +* [Original discussion](https://github.com/cosmos/cosmos-sdk/pull/5230#discussion_r343978513) diff --git a/sdk/v0.54/reference/architecture/adr-004-split-denomination-keys.mdx b/sdk/v0.54/reference/architecture/adr-004-split-denomination-keys.mdx new file mode 100644 index 000000000..b812b0e5f --- /dev/null +++ b/sdk/v0.54/reference/architecture/adr-004-split-denomination-keys.mdx @@ -0,0 +1,131 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/architecture/adr-004-split-denomination-keys' +title: 'ADR 004: Split Denomination Keys' +description: >- + 2020-01-08: Initial version 2020-01-09: Alterations to handle vesting accounts + 2020-01-14: Updates from review feedback 2020-01-30: Updates from + implementation +--- + +## Changelog + +* 2020-01-08: Initial version +* 2020-01-09: Alterations to handle vesting accounts +* 2020-01-14: Updates from review feedback +* 2020-01-30: Updates from implementation + +### Glossary + +* denom / denomination key -- unique token identifier. + +## Context + +With permissionless IBC, anyone will be able to send arbitrary denominations to any other account. Currently, all non-zero balances are stored along with the account in an `sdk.Coins` struct, which creates a potential denial-of-service concern, as too many denominations will become expensive to load & store each time the account is modified. See issues [5467](https://github.com/cosmos/cosmos-sdk/issues/5467) and [4982](https://github.com/cosmos/cosmos-sdk/issues/4982) for additional context. + +Simply rejecting incoming deposits after a denomination count limit doesn't work, since it opens up a griefing vector: someone could send a user lots of nonsensical coins over IBC, and then prevent the user from receiving real denominations (such as staking rewards). + +## Decision + +Balances shall be stored per-account & per-denomination under a denomination- and account-unique key, thus enabling O(1) read & write access to the balance of a particular account in a particular denomination. + +### Account interface (x/auth) + +`GetCoins()` and `SetCoins()` will be removed from the account interface, since coin balances will +now be stored in & managed by the bank module. + +The vesting account interface will replace `SpendableCoins` in favor of `LockedCoins` which does +not require the account balance anymore. In addition, `TrackDelegation()` will now accept the +account balance of all tokens denominated in the vesting balance instead of loading the entire +account balance. + +Vesting accounts will continue to store original vesting, delegated free, and delegated +vesting coins (which is safe since these cannot contain arbitrary denominations). + +### Bank keeper (x/bank) + +The following APIs will be added to the `x/bank` keeper: + +* `GetAllBalances(ctx Context, addr AccAddress) Coins` +* `GetBalance(ctx Context, addr AccAddress, denom string) Coin` +* `SetBalance(ctx Context, addr AccAddress, coin Coin)` +* `LockedCoins(ctx Context, addr AccAddress) Coins` +* `SpendableCoins(ctx Context, addr AccAddress) Coins` + +Additional APIs may be added to facilitate iteration and auxiliary functionality not essential to +core functionality or persistence. + +Balances will be stored first by the address, then by the denomination (the reverse is also possible, +but retrieval of all balances for a single account is presumed to be more frequent): + +```go expandable +var BalancesPrefix = []byte("balances") + +func (k Keeper) + +SetBalance(ctx Context, addr AccAddress, balance Coin) + +error { + if !balance.IsValid() { + return err +} + store := ctx.KVStore(k.storeKey) + balancesStore := prefix.NewStore(store, BalancesPrefix) + accountStore := prefix.NewStore(balancesStore, addr.Bytes()) + bz := Marshal(balance) + +accountStore.Set([]byte(balance.Denom), bz) + +return nil +} +``` + +This will result in the balances being indexed by the byte representation of +`balances/{address}/{denom}`. + +`DelegateCoins()` and `UndelegateCoins()` will be altered to only load each individual +account balance by denomination found in the (un)delegation amount. As a result, +any mutations to the account balance by will made by denomination. + +`SubtractCoins()` and `AddCoins()` will be altered to read & write the balances +directly instead of calling `GetCoins()` / `SetCoins()` (which no longer exist). + +`trackDelegation()` and `trackUndelegation()` will be altered to no longer update +account balances. + +External APIs will need to scan all balances under an account to retain backwards-compatibility. It +is advised that these APIs use `GetBalance` and `SetBalance` instead of `GetAllBalances` when +possible as to not load the entire account balance. + +### Supply module + +The supply module, in order to implement the total supply invariant, will now need +to scan all accounts & call `GetAllBalances` using the `x/bank` Keeper, then sum +the balances and check that they match the expected total supply. + +## Status + +Accepted. + +## Consequences + +### Positive + +* O(1) reads & writes of balances (with respect to the number of denominations for + which an account has non-zero balances). Note, this does not relate to the actual + I/O cost, rather the total number of direct reads needed. + +### Negative + +* Slightly less efficient reads/writes when reading & writing all balances of a + single account in a transaction. + +### Neutral + +None in particular. + +## References + +* Ref: [Link](https://github.com/cosmos/cosmos-sdk/issues/4982) +* Ref: [Link](https://github.com/cosmos/cosmos-sdk/issues/5467) +* Ref: [Link](https://github.com/cosmos/cosmos-sdk/issues/5492) diff --git a/sdk/v0.54/reference/architecture/adr-006-secret-store-replacement.mdx b/sdk/v0.54/reference/architecture/adr-006-secret-store-replacement.mdx new file mode 100644 index 000000000..99a83710a --- /dev/null +++ b/sdk/v0.54/reference/architecture/adr-006-secret-store-replacement.mdx @@ -0,0 +1,61 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/architecture/adr-006-secret-store-replacement' +title: 'ADR 006: Secret Store Replacement' +description: >- + July 29th, 2019: Initial draft September 11th, 2019: Work has started November + 4th: Cosmos SDK changes merged in November 18th: Gaia changes merged in +--- + +## Changelog + +* July 29th, 2019: Initial draft +* September 11th, 2019: Work has started +* November 4th: Cosmos SDK changes merged in +* November 18th: Gaia changes merged in + +## Context + +Currently, a Cosmos SDK application's CLI directory stores key material and metadata in a plain text database in the user's home directory. Key material is encrypted by a passphrase, protected by the bcrypt hashing algorithm. Metadata (e.g. addresses, public keys, key storage details) is available in plain text. + +This is not desirable for a number of reasons. Perhaps the biggest reason is insufficient security protection of key material and metadata. Leaking the plain text allows an attacker to surveil what keys a given computer controls via a number of techniques, like compromised dependencies without any privileged execution. This could be followed by a more targeted attack on a particular user/computer. + +All modern desktop operating systems (Ubuntu, Debian, macOS, Windows) provide a built-in secret store that is designed to allow applications to store information that is isolated from all other applications and requires passphrase entry to access the data. + +We are seeking a solution that provides a common abstraction layer to the many different backends and reasonable fallback for minimal platforms that don't provide a native secret store. + +## Decision + +We recommend replacing the current Keybase backend based on LevelDB with [Keyring](https://github.com/99designs/keyring) by 99designs. This application is designed to provide a common abstraction and uniform interface between many secret stores and is used by the AWS Vault application by 99designs. + +This appears to fulfill the requirement of protecting both key material and metadata from rogue software on a user's machine. + +## Status + +Accepted + +## Consequences + +### Positive + +Increased safety for users. + +### Negative + +Users must manually migrate. + +Testing against all supported backends is difficult. + +Running tests locally on a Mac requires numerous repetitive password entries. + +### Neutral + +No neutral consequences identified. + +## References + +* \#4754 Switch secret store to the keyring secret store (original PR by @poldsam) \[**CLOSED**] +* \#5029 Add support for github.com/99designs/keyring-backed keybases \[**MERGED**] +* \#5097 Add keys migrate command \[**MERGED**] +* \#5180 Drop on-disk keybase in favor of keyring \[*PENDING\_REVIEW*] +* cosmos/gaia#164 Drop on-disk keybase in favor of keyring (gaia's changes) \[*PENDING\_REVIEW*] diff --git a/sdk/v0.54/reference/architecture/adr-007-specialization-groups.mdx b/sdk/v0.54/reference/architecture/adr-007-specialization-groups.mdx new file mode 100644 index 000000000..ecdc7c61d --- /dev/null +++ b/sdk/v0.54/reference/architecture/adr-007-specialization-groups.mdx @@ -0,0 +1,200 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/architecture/adr-007-specialization-groups' +title: 'ADR 007: Specialization Groups' +description: '2019 Jul 31: Initial Draft' +--- + +## Changelog + +* 2019 Jul 31: Initial Draft + +## Context + +This idea was first conceived of in order to fulfill the use case of the +creation of a decentralized Computer Emergency Response Team (dCERT), whose +members would be elected by a governing community and would fulfill the role of +coordinating the community under emergency situations. This thinking +can be further abstracted into the conception of "blockchain specialization +groups". + +The creation of these groups are the beginning of specialization capabilities +within a wider blockchain community which could be used to enable a certain +level of delegated responsibilities. Examples of specialization which could be +beneficial to a blockchain community include: code auditing, emergency response, +code development etc. This type of community organization paves the way for +individual stakeholders to delegate votes by issue type, if in the future +governance proposals include a field for issue type. + +## Decision + +A specialization group can be broadly broken down into the following functions +(herein containing examples): + +* Membership Admittance +* Membership Acceptance +* Membership Revocation + * (probably) Without Penalty + * member steps down (self-Revocation) + * replaced by new member from governance + * (probably) With Penalty + * due to breach of soft-agreement (determined through governance) + * due to breach of hard-agreement (determined by code) +* Execution of Duties + * Special transactions which only execute for members of a specialization + group (for example, dCERT members voting to turn off transaction routes in + an emergency scenario) +* Compensation + * Group compensation (further distribution decided by the specialization group) + * Individual compensation for all constituents of a group from the + greater community + +Membership admittance to a specialization group could take place over a wide +variety of mechanisms. The most obvious example is through a general vote among +the entire community, however in certain systems a community may want to allow +the members already in a specialization group to internally elect new members, +or maybe the community may assign a permission to a particular specialization +group to appoint members to other 3rd party groups. The sky is really the limit +as to how membership admittance can be structured. We attempt to capture +some of these possiblities in a common interface dubbed the `Electionator`. For +its initial implementation as a part of this ADR we recommend that the general +election abstraction (`Electionator`) is provided as well as a basic +implementation of that abstraction which allows for a continuous election of +members of a specialization group. + +```golang expandable +// The Electionator abstraction covers the concept space for +// a wide variety of election kinds. +type Electionator interface { + + // is the election object accepting votes. + Active() + +bool + + // functionality to execute for when a vote is cast in this election, here + // the vote field is anticipated to be marshalled into a vote type used + // by an election. + // + // NOTE There are no explicit ids here. Just votes which pertain specifically + // to one electionator. Anyone can create and send a vote to the electionator item + // which will presumably attempt to marshal those bytes into a particular struct + // and apply the vote information in some arbitrary way. There can be multiple + // Electionators within the Cosmos-Hub for multiple specialization groups, votes + // would need to be routed to the Electionator upstream of here. + Vote(addr sdk.AccAddress, vote []byte) + + // here lies all functionality to authenticate and execute changes for + // when a member accepts being elected + AcceptElection(sdk.AccAddress) + + // Register a revoker object + RegisterRevoker(Revoker) + + // No more revokers may be registered after this function is called + SealRevokers() + + // register hooks to call when an election actions occur + RegisterHooks(ElectionatorHooks) + + // query for the current winner(s) + +of this election based on arbitrary + // election ruleset + QueryElected() []sdk.AccAddress + + // query metadata for an address in the election this + // could include for example position that an address + // is being elected for within a group + // + // this metadata may be directly related to + // voting information and/or privileges enabled + // to members within a group. + QueryMetadata(sdk.AccAddress) []byte +} + +// ElectionatorHooks, once registered with an Electionator, +// trigger execution of relevant interface functions when +// Electionator events occur. +type ElectionatorHooks interface { + AfterVoteCast(addr sdk.AccAddress, vote []byte) + +AfterMemberAccepted(addr sdk.AccAddress) + +AfterMemberRevoked(addr sdk.AccAddress, cause []byte) +} + +// Revoker defines the function required for a membership revocation rule-set +// used by a specialization group. This could be used to create self revoking, +// and evidence based revoking, etc. Revokers types may be created and +// reused for different election types. +// +// When revoking the "cause" bytes may be arbitrarily marshalled into evidence, +// memos, etc. +type Revoker interface { + RevokeName() + +string // identifier for this revoker type + RevokeMember(addr sdk.AccAddress, cause []byte) + +error +} +``` + +Certain level of commonality likely exists between the existing code within +`x/governance` and required functionality of elections. This common +functionality should be abstracted during implementation. Similarly for each +vote implementation client CLI/REST functionality should be abstracted +to be reused for multiple elections. + +The specialization group abstraction firstly extends the `Electionator` +but also further defines traits of the group. + +```golang expandable +type SpecializationGroup interface { + Electionator + GetName() + +string + GetDescription() + +string + + // general soft contract the group is expected + // to fulfill with the greater community + GetContract() + +string + + // messages which can be executed by the members of the group + Handler(ctx sdk.Context, msg sdk.Msg) + +sdk.Result + + // logic to be executed at endblock, this may for instance + // include payment of a stipend to the group members + // for participation in the security group. + EndBlocker(ctx sdk.Context) +} +``` + +## Status + +> Proposed + +## Consequences + +### Positive + +* increases specialization capabilities of a blockchain +* improve abstractions in `x/gov/` such that they can be used with specialization groups + +### Negative + +* could be used to increase centralization within a community + +### Neutral + +## References + +* [dCERT ADR](/sdk/v0.50/build/architecture/adr-008-dCERT-group) diff --git a/sdk/v0.54/reference/architecture/adr-008-dCERT-group.mdx b/sdk/v0.54/reference/architecture/adr-008-dCERT-group.mdx new file mode 100644 index 000000000..b499ab98b --- /dev/null +++ b/sdk/v0.54/reference/architecture/adr-008-dCERT-group.mdx @@ -0,0 +1,176 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/architecture/adr-008-dCERT-group' +title: 'ADR 008: Decentralized Computer Emergency Response Team (dCERT) Group' +description: '2019 Jul 31: Initial Draft' +--- + +## Changelog + +* 2019 Jul 31: Initial Draft + +## Context + +In order to reduce the number of parties involved with handling sensitive +information in an emergency scenario, we propose the creation of a +specialization group named The Decentralized Computer Emergency Response Team +(dCERT). Initially this group's role is intended to serve as coordinators +between various actors within a blockchain community such as validators, +bug-hunters, and developers. During a time of crisis, the dCERT group would +aggregate and relay input from a variety of stakeholders to the developers who +are actively devising a patch to the software, this way sensitive information +does not need to be publicly disclosed while some input from the community can +still be gained. + +Additionally, a special privilege is proposed for the dCERT group: the capacity +to "circuit-break" (aka. temporarily disable) a particular message path. Note +that this privilege should be enabled/disabled globally with a governance +parameter such that this privilege could start disabled and later be enabled +through a parameter change proposal, once a dCERT group has been established. + +In the future it is foreseeable that the community may wish to expand the roles +of dCERT with further responsibilities such as the capacity to "pre-approve" a +security update on behalf of the community prior to a full community +wide vote whereby the sensitive information would be revealed prior to a +vulnerability being patched on the live network. + +## Decision + +The dCERT group is proposed to include an implementation of a `SpecializationGroup` +as defined in [ADR 007](/sdk/v0.50/build/architecture/adr-007-specialization-groups). This will include the +implementation of: + +* continuous voting +* slashing due to breach of soft contract +* revoking a member due to breach of soft contract +* emergency disband of the entire dCERT group (ex. for colluding maliciously) +* compensation stipend from the community pool or other means decided by + governance + +This system necessitates the following new parameters: + +* blockly stipend allowance per dCERT member +* maximum number of dCERT members +* required staked slashable tokens for each dCERT member +* quorum for suspending a particular member +* proposal wager for disbanding the dCERT group +* stabilization period for dCERT member transition +* circuit break dCERT privileges enabled + +These parameters are expected to be implemented through the param keeper such +that governance may change them at any given point. + +### Continuous Voting Electionator + +An `Electionator` object is to be implemented as continuous voting and with the +following specifications: + +* All delegation addresses may submit votes at any point which updates their + preferred representation on the dCERT group. +* Preferred representation may be arbitrarily split between addresses (ex. 50% + to John, 25% to Sally, 25% to Carol) +* In order for a new member to be added to the dCERT group they must + send a transaction accepting their admission at which point the validity of + their admission is to be confirmed. + * A sequence number is assigned when a member is added to dCERT group. + If a member leaves the dCERT group and then enters back, a new sequence number + is assigned. +* Addresses which control the greatest amount of preferred-representation are + eligible to join the dCERT group (up the *maximum number of dCERT members*). + If the dCERT group is already full and new member is admitted, the existing + dCERT member with the lowest amount of votes is kicked from the dCERT group. + * In the split situation where the dCERT group is full but a vying candidate + has the same amount of vote as an existing dCERT member, the existing + member should maintain its position. + * In the split situation where somebody must be kicked out but the two + addresses with the smallest number of votes have the same number of votes, + the address with the smallest sequence number maintains its position. +* A stabilization period can be optionally included to reduce the + "flip-flopping" of the dCERT membership tail members. If a stabilization + period is provided which is greater than 0, when members are kicked due to + insufficient support, a queue entry is created which documents which member is + to replace which other member. While this entry is in the queue, no new entries + to kick that same dCERT member can be made. When the entry matures at the + duration of the stabilization period, the new member is instantiated, and old + member kicked. + +### Staking/Slashing + +All members of the dCERT group must stake tokens *specifically* to maintain +eligibility as a dCERT member. These tokens can be staked directly by the vying +dCERT member or out of the good will of a 3rd party (who shall gain no on-chain +benefits for doing so). This staking mechanism should use the existing global +unbonding time of tokens staked for network validator security. A dCERT member +can *only be* a member if it has the required tokens staked under this +mechanism. If those tokens are unbonded then the dCERT member must be +automatically kicked from the group. + +Slashing of a particular dCERT member due to soft-contract breach should be +performed by governance on a per member basis based on the magnitude of the +breach. The process flow is anticipated to be that a dCERT member is suspended +by the dCERT group prior to being slashed by governance. + +Membership suspension by the dCERT group takes place through a voting procedure +by the dCERT group members. After this suspension has taken place, a governance +proposal to slash the dCERT member must be submitted, if the proposal is not +approved by the time the rescinding member has completed unbonding their +tokens, then the tokens are no longer staked and unable to be slashed. + +Additionally in the case of an emergency situation of a colluding and malicious +dCERT group, the community needs the capability to disband the entire dCERT +group and likely fully slash them. This could be achieved though a special new +proposal type (implemented as a general governance proposal) which would halt +the functionality of the dCERT group until the proposal was concluded. This +special proposal type would likely need to also have a fairly large wager which +could be slashed if the proposal creator was malicious. The reason a large +wager should be required is because as soon as the proposal is made, the +capability of the dCERT group to halt message routes is put on temporarily +suspended, meaning that a malicious actor who created such a proposal could +then potentially exploit a bug during this period of time, with no dCERT group +capable of shutting down the exploitable message routes. + +### dCERT membership transactions + +Active dCERT members + +* change of the description of the dCERT group +* circuit break a message route +* vote to suspend a dCERT member. + +Here circuit-breaking refers to the capability to disable a groups of messages, +This could for instance mean: "disable all staking-delegation messages", or +"disable all distribution messages". This could be accomplished by verifying +that the message route has not been "circuit-broken" at CheckTx time (in +`baseapp/baseapp.go`). + +"unbreaking" a circuit is anticipated only to occur during a hard fork upgrade +meaning that no capability to unbreak a message route on a live chain is +required. + +Note also, that if there was a problem with governance voting (for instance a +capability to vote many times) then governance would be broken and should be +halted with this mechanism, it would be then up to the validator set to +coordinate and hard-fork upgrade to a patched version of the software where +governance is re-enabled (and fixed). If the dCERT group abuses this privilege +they should all be severely slashed. + +## Status + +> Proposed + +## Consequences + +### Positive + +* Potential to reduces the number of parties to coordinate with during an emergency +* Reduction in possibility of disclosing sensitive information to malicious parties + +### Negative + +* Centralization risks + +### Neutral + +## References + +[Specialization Groups ADR](/sdk/v0.50/build/architecture/adr-007-specialization-groups) diff --git a/sdk/v0.54/reference/architecture/adr-009-evidence-module.mdx b/sdk/v0.54/reference/architecture/adr-009-evidence-module.mdx new file mode 100644 index 000000000..4d24e1a8d --- /dev/null +++ b/sdk/v0.54/reference/architecture/adr-009-evidence-module.mdx @@ -0,0 +1,220 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/architecture/adr-009-evidence-module' +title: 'ADR 009: Evidence Module' +description: '2019 July 31: Initial draft 2019 October 24: Initial implementation' +--- + +## Changelog + +* 2019 July 31: Initial draft +* 2019 October 24: Initial implementation + +## Status + +Accepted + +## Context + +In order to support building highly secure, robust and interoperable blockchain +applications, it is vital for the Cosmos SDK to expose a mechanism in which arbitrary +evidence can be submitted, evaluated and verified resulting in some agreed upon +penalty for any misbehavior committed by a validator, such as equivocation (double-voting), +signing when unbonded, signing an incorrect state transition (in the future), etc. +Furthermore, such a mechanism is paramount for any +[IBC](https://github.com/cosmos/ics/blob/master/ibc/2_IBC_ARCHITECTURE.md) or +cross-chain validation protocol implementation in order to support the ability +for any misbehavior to be relayed back from a collateralized chain to a primary +chain so that the equivocating validator(s) can be slashed. + +## Decision + +We will implement an evidence module in the Cosmos SDK supporting the following +functionality: + +* Provide developers with the abstractions and interfaces necessary to define + custom evidence messages, message handlers, and methods to slash and penalize + accordingly for misbehavior. +* Support the ability to route evidence messages to handlers in any module to + determine the validity of submitted misbehavior. +* Support the ability, through governance, to modify slashing penalties of any + evidence type. +* Querier implementation to support querying params, evidence types, params, and + all submitted valid misbehavior. + +### Types + +First, we define the `Evidence` interface type. The `x/evidence` module may implement +its own types that can be used by many chains (e.g. `CounterFactualEvidence`). +In addition, other modules may implement their own `Evidence` types in a similar +manner in which governance is extensible. It is important to note any concrete +type implementing the `Evidence` interface may include arbitrary fields such as +an infraction time. We want the `Evidence` type to remain as flexible as possible. + +When submitting evidence to the `x/evidence` module, the concrete type must provide +the validator's consensus address, which should be known by the `x/slashing` +module (assuming the infraction is valid), the height at which the infraction +occurred and the validator's power at same height in which the infraction occurred. + +```go expandable +type Evidence interface { + Route() + +string + Type() + +string + String() + +string + Hash() + +HexBytes + ValidateBasic() + +error + + // The consensus address of the malicious validator at time of infraction + GetConsensusAddress() + +ConsAddress + + // Height at which the infraction occurred + GetHeight() + +int64 + + // The total power of the malicious validator at time of infraction + GetValidatorPower() + +int64 + + // The total validator set power at time of infraction + GetTotalPower() + +int64 +} +``` + +### Routing & Handling + +Each `Evidence` type must map to a specific unique route and be registered with +the `x/evidence` module. It accomplishes this through the `Router` implementation. + +```go +type Router interface { + AddRoute(r string, h Handler) + +Router + HasRoute(r string) + +bool + GetRoute(path string) + +Handler + Seal() +} +``` + +Upon successful routing through the `x/evidence` module, the `Evidence` type +is passed through a `Handler`. This `Handler` is responsible for executing all +corresponding business logic necessary for verifying the evidence as valid. In +addition, the `Handler` may execute any necessary slashing and potential jailing. +Since slashing fractions will typically result from some form of static functions, +allow the `Handler` to do this provides the greatest flexibility. An example could +be `k * evidence.GetValidatorPower()` where `k` is an on-chain parameter controlled +by governance. The `Evidence` type should provide all the external information +necessary in order for the `Handler` to make the necessary state transitions. +If no error is returned, the `Evidence` is considered valid. + +```go +type Handler func(Context, Evidence) + +error +``` + +### Submission + +`Evidence` is submitted through a `MsgSubmitEvidence` message type which is internally +handled by the `x/evidence` module's `SubmitEvidence`. + +```go expandable +type MsgSubmitEvidence struct { + Evidence +} + +func handleMsgSubmitEvidence(ctx Context, keeper Keeper, msg MsgSubmitEvidence) + +Result { + if err := keeper.SubmitEvidence(ctx, msg.Evidence); err != nil { + return err.Result() +} + + // emit events... + + return Result{ + // ... +} +} +``` + +The `x/evidence` module's keeper is responsible for matching the `Evidence` against +the module's router and invoking the corresponding `Handler` which may include +slashing and jailing the validator. Upon success, the submitted evidence is persisted. + +```go +func (k Keeper) + +SubmitEvidence(ctx Context, evidence Evidence) + +error { + handler := keeper.router.GetRoute(evidence.Route()) + if err := handler(ctx, evidence); err != nil { + return ErrInvalidEvidence(keeper.codespace, err) +} + +keeper.setEvidence(ctx, evidence) + +return nil +} +``` + +### Genesis + +Finally, we need to represent the genesis state of the `x/evidence` module. The +module only needs a list of all submitted valid infractions and any necessary params +for which the module needs in order to handle submitted evidence. The `x/evidence` +module will naturally define and route native evidence types for which it'll most +likely need slashing penalty constants for. + +```go +type GenesisState struct { + Params Params + Infractions []Evidence +} +``` + +## Consequences + +### Positive + +* Allows the state machine to process misbehavior submitted on-chain and penalize + validators based on agreed upon slashing parameters. +* Allows evidence types to be defined and handled by any module. This further allows + slashing and jailing to be defined by more complex mechanisms. +* Does not solely rely on Tendermint to submit evidence. + +### Negative + +* No easy way to introduce new evidence types through governance on a live chain + due to the inability to introduce the new evidence type's corresponding handler + +### Neutral + +* Should we persist infractions indefinitely? Or should we rather rely on events? + +## References + +* [ICS](https://github.com/cosmos/ics) +* [IBC Architecture](https://github.com/cosmos/ics/blob/master/ibc/1_IBC_ARCHITECTURE.md) +* [Tendermint Fork Accountability](https://github.com/tendermint/spec/blob/7b3138e69490f410768d9b1ffc7a17abc23ea397/spec/consensus/fork-accountability.md) diff --git a/sdk/v0.54/reference/architecture/adr-010-modular-antehandler.mdx b/sdk/v0.54/reference/architecture/adr-010-modular-antehandler.mdx new file mode 100644 index 000000000..ff5b5e4f1 --- /dev/null +++ b/sdk/v0.54/reference/architecture/adr-010-modular-antehandler.mdx @@ -0,0 +1,322 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/architecture/adr-010-modular-antehandler' +title: 'ADR 010: Modular AnteHandler' +description: '2019 Aug 31: Initial draft 2021 Sep 14: Superseded by ADR-045' +--- + +## Changelog + +* 2019 Aug 31: Initial draft +* 2021 Sep 14: Superseded by ADR-045 + +## Status + +SUPERSEDED by ADR-045 + +## Context + +The current AnteHandler design allows users to either use the default AnteHandler provided in `x/auth` or to build their own AnteHandler from scratch. Ideally AnteHandler functionality is split into multiple, modular functions that can be chained together along with custom ante-functions so that users do not have to rewrite common antehandler logic when they want to implement custom behavior. + +For example, let's say a user wants to implement some custom signature verification logic. In the current codebase, the user would have to write their own Antehandler from scratch largely reimplementing much of the same code and then set their own custom, monolithic antehandler in the baseapp. Instead, we would like to allow users to specify custom behavior when necessary and combine them with default ante-handler functionality in a way that is as modular and flexible as possible. + +## Proposals + +### Per-Module AnteHandler + +One approach is to use the [ModuleManager](https://pkg.go.dev/github.com/cosmos/cosmos-sdk/types/module) and have each module implement its own antehandler if it requires custom antehandler logic. The ModuleManager can then be passed in an AnteHandler order in the same way it has an order for BeginBlockers and EndBlockers. The ModuleManager returns a single AnteHandler function that will take in a tx and run each module's `AnteHandle` in the specified order. The module manager's AnteHandler is set as the baseapp's AnteHandler. + +Pros: + +1. Simple to implement +2. Utilizes the existing ModuleManager architecture + +Cons: + +1. Improves granularity but still cannot get more granular than a per-module basis. e.g. If auth's `AnteHandle` function is in charge of validating memo and signatures, users cannot swap the signature-checking functionality while keeping the rest of auth's `AnteHandle` functionality. +2. Module AnteHandler are run one after the other. There is no way for one AnteHandler to wrap or "decorate" another. + +### Decorator Pattern + +The [weave project](https://github.com/iov-one/weave) achieves AnteHandler modularity through the use of a decorator pattern. The interface is designed as follows: + +```go +// Decorator wraps a Handler to provide common functionality +// like authentication, or fee-handling, to many Handlers +type Decorator interface { + Check(ctx Context, store KVStore, tx Tx, next Checker) (*CheckResult, error) + +Deliver(ctx Context, store KVStore, tx Tx, next Deliverer) (*DeliverResult, error) +} +``` + +Each decorator works like a modularized Cosmos SDK antehandler function, but it can take in a `next` argument that may be another decorator or a Handler (which does not take in a next argument). These decorators can be chained together, one decorator being passed in as the `next` argument of the previous decorator in the chain. The chain ends in a Router which can take a tx and route to the appropriate msg handler. + +A key benefit of this approach is that one Decorator can wrap its internal logic around the next Checker/Deliverer. A weave Decorator may do the following: + +```go +// Example Decorator's Deliver function +func (example Decorator) + +Deliver(ctx Context, store KVStore, tx Tx, next Deliverer) { + // Do some pre-processing logic + + res, err := next.Deliver(ctx, store, tx) + + // Do some post-processing logic given the result and error +} +``` + +Pros: + +1. Weave Decorators can wrap over the next decorator/handler in the chain. The ability to both pre-process and post-process may be useful in certain settings. +2. Provides a nested modular structure that isn't possible in the solution above, while also allowing for a linear one-after-the-other structure like the solution above. + +Cons: + +1. It is hard to understand at first glance the state updates that would occur after a Decorator runs given the `ctx`, `store`, and `tx`. A Decorator can have an arbitrary number of nested Decorators being called within its function body, each possibly doing some pre- and post-processing before calling the next decorator on the chain. Thus to understand what a Decorator is doing, one must also understand what every other decorator further along the chain is also doing. This can get quite complicated to understand. A linear, one-after-the-other approach while less powerful, may be much easier to reason about. + +### Chained Micro-Functions + +The benefit of Weave's approach is that the Decorators can be very concise, which when chained together allows for maximum customizability. However, the nested structure can get quite complex and thus hard to reason about. + +Another approach is to split the AnteHandler functionality into tightly scoped "micro-functions", while preserving the one-after-the-other ordering that would come from the ModuleManager approach. + +We can then have a way to chain these micro-functions so that they run one after the other. Modules may define multiple ante micro-functions and then also provide a default per-module AnteHandler that implements a default, suggested order for these micro-functions. + +Users can order the AnteHandlers easily by simply using the ModuleManager. The ModuleManager will take in a list of AnteHandlers and return a single AnteHandler that runs each AnteHandler in the order of the list provided. If the user is comfortable with the default ordering of each module, this is as simple as providing a list with each module's antehandler (exactly the same as BeginBlocker and EndBlocker). + +If however, users wish to change the order or add, modify, or delete ante micro-functions in anyway; they can always define their own ante micro-functions and add them explicitly to the list that gets passed into module manager. + +#### Default Workflow + +This is an example of a user's AnteHandler if they choose not to make any custom micro-functions. + +##### Cosmos SDK code + +```go expandable +// Chains together a list of AnteHandler micro-functions that get run one after the other. +// Returned AnteHandler will abort on first error. +func Chainer(order []AnteHandler) + +AnteHandler { + return func(ctx Context, tx Tx, simulate bool) (newCtx Context, err error) { + for _, ante := range order { + ctx, err := ante(ctx, tx, simulate) + if err != nil { + return ctx, err +} + +} + +return ctx, err +} +} +``` + +```go expandable +// AnteHandler micro-function to verify signatures +func VerifySignatures(ctx Context, tx Tx, simulate bool) (newCtx Context, err error) { + // verify signatures + // Returns InvalidSignature Result and abort=true if sigs invalid + // Return OK result and abort=false if sigs are valid +} + +// AnteHandler micro-function to validate memo +func ValidateMemo(ctx Context, tx Tx, simulate bool) (newCtx Context, err error) { + // validate memo +} + +// Auth defines its own default ante-handler by chaining its micro-functions in a recommended order +AuthModuleAnteHandler := Chainer([]AnteHandler{ + VerifySignatures, ValidateMemo +}) +``` + +```go expandable +// Distribution micro-function to deduct fees from tx +func DeductFees(ctx Context, tx Tx, simulate bool) (newCtx Context, err error) { + // Deduct fees from tx + // Abort if insufficient funds in account to pay for fees +} + +// Distribution micro-function to check if fees > mempool parameter +func CheckMempoolFees(ctx Context, tx Tx, simulate bool) (newCtx Context, err error) { + // If CheckTx: Abort if the fees are less than the mempool's minFee parameter +} + +// Distribution defines its own default ante-handler by chaining its micro-functions in a recommended order +DistrModuleAnteHandler := Chainer([]AnteHandler{ + CheckMempoolFees, DeductFees +}) +``` + +```go +type ModuleManager struct { + // other fields + AnteHandlerOrder []AnteHandler +} + +func (mm ModuleManager) + +GetAnteHandler() + +AnteHandler { + retun Chainer(mm.AnteHandlerOrder) +} +``` + +##### User Code + +```go +// Note: Since user is not making any custom modifications, we can just SetAnteHandlerOrder with the default AnteHandlers provided by each module in our preferred order +moduleManager.SetAnteHandlerOrder([]AnteHandler(AuthModuleAnteHandler, DistrModuleAnteHandler)) + +app.SetAnteHandler(mm.GetAnteHandler()) +``` + +#### Custom Workflow + +This is an example workflow for a user that wants to implement custom antehandler logic. In this example, the user wants to implement custom signature verification and change the order of antehandler so that validate memo runs before signature verification. + +##### User Code + +```go +// User can implement their own custom signature verification antehandler micro-function +func CustomSigVerify(ctx Context, tx Tx, simulate bool) (newCtx Context, err error) { + // do some custom signature verification logic +} +``` + +```go +// Micro-functions allow users to change order of when they get executed, and swap out default ante-functionality with their own custom logic. +// Note that users can still chain the default distribution module handler, and auth micro-function along with their custom ante function +moduleManager.SetAnteHandlerOrder([]AnteHandler(ValidateMemo, CustomSigVerify, DistrModuleAnteHandler)) +``` + +Pros: + +1. Allows for ante functionality to be as modular as possible. +2. For users that do not need custom ante-functionality, there is little difference between how antehandlers work and how BeginBlock and EndBlock work in ModuleManager. +3. Still easy to understand + +Cons: + +1. Cannot wrap antehandlers with decorators like you can with Weave. + +### Simple Decorators + +This approach takes inspiration from Weave's decorator design while trying to minimize the number of breaking changes to the Cosmos SDK and maximizing simplicity. Like Weave decorators, this approach allows one `AnteDecorator` to wrap the next AnteHandler to do pre- and post-processing on the result. This is useful since decorators can do defer/cleanups after an AnteHandler returns as well as perform some setup beforehand. Unlike Weave decorators, these `AnteDecorator` functions can only wrap over the AnteHandler rather than the entire handler execution path. This is deliberate as we want decorators from different modules to perform authentication/validation on a `tx`. However, we do not want decorators being capable of wrapping and modifying the results of a `MsgHandler`. + +In addition, this approach will not break any core Cosmos SDK API's. Since we preserve the notion of an AnteHandler and still set a single AnteHandler in baseapp, the decorator is simply an additional approach available for users that desire more customization. The API of modules (namely `x/auth`) may break with this approach, but the core API remains untouched. + +Allow Decorator interface that can be chained together to create a Cosmos SDK AnteHandler. + +This allows users to choose between implementing an AnteHandler by themselves and setting it in the baseapp, or use the decorator pattern to chain their custom decorators with the Cosmos SDK provided decorators in the order they wish. + +```go +// An AnteDecorator wraps an AnteHandler, and can do pre- and post-processing on the next AnteHandler +type AnteDecorator interface { + AnteHandle(ctx Context, tx Tx, simulate bool, next AnteHandler) (newCtx Context, err error) +} +``` + +```go expandable +// ChainAnteDecorators will recursively link all of the AnteDecorators in the chain and return a final AnteHandler function +// This is done to preserve the ability to set a single AnteHandler function in the baseapp. +func ChainAnteDecorators(chain ...AnteDecorator) + +AnteHandler { + if len(chain) == 1 { + return func(ctx Context, tx Tx, simulate bool) { + chain[0].AnteHandle(ctx, tx, simulate, nil) +} + +} + +return func(ctx Context, tx Tx, simulate bool) { + chain[0].AnteHandle(ctx, tx, simulate, ChainAnteDecorators(chain[1:])) +} +} +``` + +#### Example Code + +Define AnteDecorator functions + +```go expandable +// Setup GasMeter, catch OutOfGasPanic and handle appropriately +type SetUpContextDecorator struct{ +} + +func (sud SetUpContextDecorator) + +AnteHandle(ctx Context, tx Tx, simulate bool, next AnteHandler) (newCtx Context, err error) { + ctx.GasMeter = NewGasMeter(tx.Gas) + +defer func() { + // recover from OutOfGas panic and handle appropriately +} + +return next(ctx, tx, simulate) +} + +// Signature Verification decorator. Verify Signatures and move on +type SigVerifyDecorator struct{ +} + +func (svd SigVerifyDecorator) + +AnteHandle(ctx Context, tx Tx, simulate bool, next AnteHandler) (newCtx Context, err error) { + // verify sigs. Return error if invalid + + // call next antehandler if sigs ok + return next(ctx, tx, simulate) +} + +// User-defined Decorator. Can choose to pre- and post-process on AnteHandler +type UserDefinedDecorator struct{ + // custom fields +} + +func (udd UserDefinedDecorator) + +AnteHandle(ctx Context, tx Tx, simulate bool, next AnteHandler) (newCtx Context, err error) { + // pre-processing logic + + ctx, err = next(ctx, tx, simulate) + + // post-processing logic +} +``` + +Link AnteDecorators to create a final AnteHandler. Set this AnteHandler in baseapp. + +```go +// Create final antehandler by chaining the decorators together + antehandler := ChainAnteDecorators(NewSetUpContextDecorator(), NewSigVerifyDecorator(), NewUserDefinedDecorator()) + +// Set chained Antehandler in the baseapp +bapp.SetAnteHandler(antehandler) +``` + +Pros: + +1. Allows one decorator to pre- and post-process the next AnteHandler, similar to the Weave design. +2. Do not need to break baseapp API. Users can still set a single AnteHandler if they choose. + +Cons: + +1. Decorator pattern may have a deeply nested structure that is hard to understand, this is mitigated by having the decorator order explicitly listed in the `ChainAnteDecorators` function. +2. Does not make use of the ModuleManager design. Since this is already being used for BeginBlocker/EndBlocker, this proposal seems unaligned with that design pattern. + +## Consequences + +Since pros and cons are written for each approach, it is omitted from this section + +## References + +* [#4572](https://github.com/cosmos/cosmos-sdk/issues/4572): Modular AnteHandler Issue +* [#4582](https://github.com/cosmos/cosmos-sdk/pull/4583): Initial Implementation of Per-Module AnteHandler Approach +* [Weave Decorator Code](https://github.com/iov-one/weave/blob/master/handler.go#L35) +* [Weave Design Videos](https://vimeo.com/showcase/6189877) diff --git a/sdk/v0.54/reference/architecture/adr-011-generalize-genesis-accounts.mdx b/sdk/v0.54/reference/architecture/adr-011-generalize-genesis-accounts.mdx new file mode 100644 index 000000000..40964c3f1 --- /dev/null +++ b/sdk/v0.54/reference/architecture/adr-011-generalize-genesis-accounts.mdx @@ -0,0 +1,190 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/architecture/adr-011-generalize-genesis-accounts' +title: 'ADR 011: Generalize Genesis Accounts' +description: '2019-08-30: initial draft' +--- + +## Changelog + +* 2019-08-30: initial draft + +## Context + +Currently, the Cosmos SDK allows for custom account types; the `auth` keeper stores any type fulfilling its `Account` interface. However `auth` does not handle exporting or loading accounts to/from a genesis file, this is done by `genaccounts`, which only handles one of 4 concrete account types (`BaseAccount`, `ContinuousVestingAccount`, `DelayedVestingAccount` and `ModuleAccount`). + +Projects desiring to use custom accounts (say custom vesting accounts) need to fork and modify `genaccounts`. + +## Decision + +In summary, we will (un)marshal all accounts (interface types) directly using amino, rather than converting to `genaccounts`’s `GenesisAccount` type. Since doing this removes the majority of `genaccounts`'s code, we will merge `genaccounts` into `auth`. Marshalled accounts will be stored in `auth`'s genesis state. + +Detailed changes: + +### 1) (Un)Marshal accounts directly using amino + +The `auth` module's `GenesisState` gains a new field `Accounts`. Note these aren't of type `exported.Account` for reasons outlined in section 3. + +```go +// GenesisState - all auth state that must be provided at genesis +type GenesisState struct { + Params Params `json:"params" yaml:"params"` + Accounts []GenesisAccount `json:"accounts" yaml:"accounts"` +} +``` + +Now `auth`'s `InitGenesis` and `ExportGenesis` (un)marshal accounts as well as the defined params. + +```go expandable +// InitGenesis - Init store state from genesis data +func InitGenesis(ctx sdk.Context, ak AccountKeeper, data GenesisState) { + ak.SetParams(ctx, data.Params) + // load the accounts + for _, a := range data.Accounts { + acc := ak.NewAccount(ctx, a) // set account number + ak.SetAccount(ctx, acc) +} +} + +// ExportGenesis returns a GenesisState for a given context and keeper +func ExportGenesis(ctx sdk.Context, ak AccountKeeper) + +GenesisState { + params := ak.GetParams(ctx) + +var genAccounts []exported.GenesisAccount + ak.IterateAccounts(ctx, func(account exported.Account) + +bool { + genAccount := account.(exported.GenesisAccount) + +genAccounts = append(genAccounts, genAccount) + +return false +}) + +return NewGenesisState(params, genAccounts) +} +``` + +### 2) Register custom account types on the `auth` codec + +The `auth` codec must have all custom account types registered to marshal them. We will follow the pattern established in `gov` for proposals. + +An example custom account definition: + +```go +import authtypes "github.com/cosmos/cosmos-sdk/x/auth/types" + +// Register the module account type with the auth module codec so it can decode module accounts stored in a genesis file +func init() { + authtypes.RegisterAccountTypeCodec(ModuleAccount{ +}, "cosmos-sdk/ModuleAccount") +} + +type ModuleAccount struct { + ... +``` + +The `auth` codec definition: + +```go expandable +var ModuleCdc *codec.LegacyAmino + +func init() { + ModuleCdc = codec.NewLegacyAmino() + // register module msg's and Account interface + ... + // leave the codec unsealed +} + +// RegisterAccountTypeCodec registers an external account type defined in another module for the internal ModuleCdc. +func RegisterAccountTypeCodec(o interface{ +}, name string) { + ModuleCdc.RegisterConcrete(o, name, nil) +} +``` + +### 3) Genesis validation for custom account types + +Modules implement a `ValidateGenesis` method. As `auth` does not know of account implementations, accounts will need to validate themselves. + +We will unmarshal accounts into a `GenesisAccount` interface that includes a `Validate` method. + +```go +type GenesisAccount interface { + exported.Account + Validate() + +error +} +``` + +Then the `auth` `ValidateGenesis` function becomes: + +```go expandable +// ValidateGenesis performs basic validation of auth genesis data returning an +// error for any failed validation criteria. +func ValidateGenesis(data GenesisState) + +error { + // Validate params + ... + + // Validate accounts + addrMap := make(map[string]bool, len(data.Accounts)) + for _, acc := range data.Accounts { + + // check for duplicated accounts + addrStr := acc.GetAddress().String() + if _, ok := addrMap[addrStr]; ok { + return fmt.Errorf("duplicate account found in genesis state; address: %s", addrStr) +} + +addrMap[addrStr] = true + + // check account specific validation + if err := acc.Validate(); err != nil { + return fmt.Errorf("invalid account found in genesis state; address: %s, error: %s", addrStr, err.Error()) +} + + +} + +return nil +} +``` + +### 4) Move add-genesis-account cli to `auth` + +The `genaccounts` module contains a cli command to add base or vesting accounts to a genesis file. + +This will be moved to `auth`. We will leave it to projects to write their own commands to add custom accounts. An extensible cli handler, similar to `gov`, could be created but it is not worth the complexity for this minor use case. + +### 5) Update module and vesting accounts + +Under the new scheme, module and vesting account types need some minor updates: + +* Type registration on `auth`'s codec (shown above) +* A `Validate` method for each `Account` concrete type + +## Status + +Proposed + +## Consequences + +### Positive + +* custom accounts can be used without needing to fork `genaccounts` +* reduction in lines of code + +### Negative + +### Neutral + +* `genaccounts` module no longer exists +* accounts in genesis files are stored under `accounts` in `auth` rather than in the `genaccounts` module. + -`add-genesis-account` cli command now in `auth` + +## References diff --git a/sdk/v0.54/reference/architecture/adr-012-state-accessors.mdx b/sdk/v0.54/reference/architecture/adr-012-state-accessors.mdx new file mode 100644 index 000000000..5994661e6 --- /dev/null +++ b/sdk/v0.54/reference/architecture/adr-012-state-accessors.mdx @@ -0,0 +1,221 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/architecture/adr-012-state-accessors' +title: 'ADR 012: State Accessors' +description: '2019 Sep 04: Initial draft' +--- + +## Changelog + +* 2019 Sep 04: Initial draft + +## Context + +Cosmos SDK modules currently use the `KVStore` interface and `Codec` to access their respective state. While +this provides a large degree of freedom to module developers, it is hard to modularize and the UX is +mediocre. + +First, each time a module tries to access the state, it has to marshal the value and set or get the +value and finally unmarshal. Usually this is done by declaring `Keeper.GetXXX` and `Keeper.SetXXX` functions, +which are repetitive and hard to maintain. + +Second, this makes it harder to align with the object capability theorem: the right to access the +state is defined as a `StoreKey`, which gives full access on the entire Merkle tree, so a module cannot +send the access right to a specific key-value pair (or a set of key-value pairs) to another module safely. + +Finally, because the getter/setter functions are defined as methods of a module's `Keeper`, the reviewers +have to consider the whole Merkle tree space when they reviewing a function accessing any part of the state. +There is no static way to know which part of the state that the function is accessing (and which is not). + +## Decision + +We will define a type named `Value`: + +```go +type Value struct { + m Mapping + key []byte +} +``` + +The `Value` works as a reference for a key-value pair in the state, where `Value.m` defines the key-value +space it will access and `Value.key` defines the exact key for the reference. + +We will define a type named `Mapping`: + +```go +type Mapping struct { + storeKey sdk.StoreKey + cdc *codec.LegacyAmino + prefix []byte +} +``` + +The `Mapping` works as a reference for a key-value space in the state, where `Mapping.storeKey` defines +the IAVL (sub-)tree and `Mapping.prefix` defines the optional subspace prefix. + +We will define the following core methods for the `Value` type: + +```go expandable +// Get and unmarshal stored data, noop if not exists, panic if cannot unmarshal +func (Value) + +Get(ctx Context, ptr interface{ +}) { +} + +// Get and unmarshal stored data, return error if not exists or cannot unmarshal +func (Value) + +GetSafe(ctx Context, ptr interface{ +}) { +} + +// Get stored data as raw byte slice +func (Value) + +GetRaw(ctx Context) []byte { +} + +// Marshal and set a raw value +func (Value) + +Set(ctx Context, o interface{ +}) { +} + +// Check if a raw value exists +func (Value) + +Exists(ctx Context) + +bool { +} + +// Delete a raw value value +func (Value) + +Delete(ctx Context) { +} +``` + +We will define the following core methods for the `Mapping` type: + +```go expandable +// Constructs key-value pair reference corresponding to the key argument in the Mapping space +func (Mapping) + +Value(key []byte) + +Value { +} + +// Get and unmarshal stored data, noop if not exists, panic if cannot unmarshal +func (Mapping) + +Get(ctx Context, key []byte, ptr interface{ +}) { +} + +// Get and unmarshal stored data, return error if not exists or cannot unmarshal +func (Mapping) + +GetSafe(ctx Context, key []byte, ptr interface{ +}) + +// Get stored data as raw byte slice +func (Mapping) + +GetRaw(ctx Context, key []byte) []byte { +} + +// Marshal and set a raw value +func (Mapping) + +Set(ctx Context, key []byte, o interface{ +}) { +} + +// Check if a raw value exists +func (Mapping) + +Has(ctx Context, key []byte) + +bool { +} + +// Delete a raw value value +func (Mapping) + +Delete(ctx Context, key []byte) { +} +``` + +Each method of the `Mapping` type that is passed the arguments `ctx`, `key`, and `args...` will proxy +the call to `Mapping.Value(key)` with arguments `ctx` and `args...`. + +In addition, we will define and provide a common set of types derived from the `Value` type: + +```go +type Boolean struct { + Value +} + +type Enum struct { + Value +} + +type Integer struct { + Value; enc IntEncoding +} + +type String struct { + Value +} +// ... +``` + +Where the encoding schemes can be different, `o` arguments in core methods are typed, and `ptr` arguments +in core methods are replaced by explicit return types. + +Finally, we will define a family of types derived from the `Mapping` type: + +```go +type Indexer struct { + m Mapping + enc IntEncoding +} +``` + +Where the `key` argument in core method is typed. + +Some of the properties of the accessor types are: + +* State access happens only when a function which takes a `Context` as an argument is invoked +* Accessor type structs give rights to access the state only that the struct is referring, no other +* Marshalling/Unmarshalling happens implicitly within the core methods + +## Status + +Proposed + +## Consequences + +### Positive + +* Serialization will be done automatically +* Shorter code size, less boilerplate, better UX +* References to the state can be transferred safely +* Explicit scope of accessing + +### Negative + +* Serialization format will be hidden +* Different architecture from the current, but the use of accessor types can be opt-in +* Type-specific types (e.g. `Boolean` and `Integer`) have to be defined manually + +### Neutral + +## References + +* [#4554](https://github.com/cosmos/cosmos-sdk/issues/4554) diff --git a/sdk/v0.54/reference/architecture/adr-013-metrics.mdx b/sdk/v0.54/reference/architecture/adr-013-metrics.mdx new file mode 100644 index 000000000..440d5435a --- /dev/null +++ b/sdk/v0.54/reference/architecture/adr-013-metrics.mdx @@ -0,0 +1,173 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/architecture/adr-013-metrics' +title: 'ADR 013: Observability' +description: '20-01-2020: Initial Draft' +--- + +## Changelog + +* 20-01-2020: Initial Draft + +## Status + +Proposed + +## Context + +Telemetry is paramount into debugging and understanding what the application is doing and how it is +performing. We aim to expose metrics from modules and other core parts of the Cosmos SDK. + +In addition, we should aim to support multiple configurable sinks that an operator may choose from. +By default, when telemetry is enabled, the application should track and expose metrics that are +stored in-memory. The operator may choose to enable additional sinks, where we support only +[Prometheus](https://prometheus.io/) for now, as it's battle-tested, simple to setup, open source, +and is rich with ecosystem tooling. + +We must also aim to integrate metrics into the Cosmos SDK in the most seamless way possible such that +metrics may be added or removed at will and without much friction. To do this, we will use the +[go-metrics](https://github.com/hashicorp/go-metrics) library. + +Finally, operators may enable telemetry along with specific configuration options. If enabled, metrics +will be exposed via `/metrics?format={text|prometheus}` via the API server. + +## Decision + +We will add an additional configuration block to `app.toml` that defines telemetry settings: + +```toml expandable +############################################################################### +### Telemetry Configuration ### +############################################################################### + +[telemetry] + +# Prefixed with keys to separate services +service-name = {{ .Telemetry.ServiceName }} + +# Enabled enables the application telemetry functionality. When enabled, +# an in-memory sink is also enabled by default. Operators may also enabled +# other sinks such as Prometheus. +enabled = {{ .Telemetry.Enabled }} + +# Enable prefixing gauge values with hostname +enable-hostname = {{ .Telemetry.EnableHostname }} + +# Enable adding hostname to labels +enable-hostname-label = {{ .Telemetry.EnableHostnameLabel }} + +# Enable adding service to labels +enable-service-label = {{ .Telemetry.EnableServiceLabel }} + +# PrometheusRetentionTime, when positive, enables a Prometheus metrics sink. +prometheus-retention-time = {{ .Telemetry.PrometheusRetentionTime }} +``` + +The given configuration allows for two sinks -- in-memory and Prometheus. We create a `Metrics` +type that performs all the bootstrapping for the operator, so capturing metrics becomes seamless. + +```go expandable +// Metrics defines a wrapper around application telemetry functionality. It allows +// metrics to be gathered at any point in time. When creating a Metrics object, +// internally, a global metrics is registered with a set of sinks as configured +// by the operator. In addition to the sinks, when a process gets a SIGUSR1, a +// dump of formatted recent metrics will be sent to STDERR. +type Metrics struct { + memSink *metrics.InmemSink + prometheusEnabled bool +} + +// Gather collects all registered metrics and returns a GatherResponse where the +// metrics are encoded depending on the type. Metrics are either encoded via +// Prometheus or JSON if in-memory. +func (m *Metrics) + +Gather(format string) (GatherResponse, error) { + switch format { + case FormatPrometheus: + return m.gatherPrometheus() + case FormatText: + return m.gatherGeneric() + case FormatDefault: + return m.gatherGeneric() + +default: + return GatherResponse{ +}, fmt.Errorf("unsupported metrics format: %s", format) +} +} +``` + +In addition, `Metrics` allows us to gather the current set of metrics at any given point in time. An +operator may also choose to send a signal, SIGUSR1, to dump and print formatted metrics to STDERR. + +During an application's bootstrapping and construction phase, if `Telemetry.Enabled` is `true`, the +API server will create an instance of a reference to `Metrics` object and will register a metrics +handler accordingly. + +```go expandable +func (s *Server) + +Start(cfg config.Config) + +error { + // ... + if cfg.Telemetry.Enabled { + m, err := telemetry.New(cfg.Telemetry) + if err != nil { + return err +} + +s.metrics = m + s.registerMetrics() +} + + // ... +} + +func (s *Server) + +registerMetrics() { + metricsHandler := func(w http.ResponseWriter, r *http.Request) { + format := strings.TrimSpace(r.FormValue("format")) + +gr, err := s.metrics.Gather(format) + if err != nil { + rest.WriteErrorResponse(w, http.StatusBadRequest, fmt.Sprintf("failed to gather metrics: %s", err)) + +return +} + +w.Header().Set("Content-Type", gr.ContentType) + _, _ = w.Write(gr.Metrics) +} + +s.Router.HandleFunc("/metrics", metricsHandler).Methods("GET") +} +``` + +Application developers may track counters, gauges, summaries, and key/value metrics. There is no +additional lifting required by modules to leverage profiling metrics. To do so, it's as simple as: + +```go +func (k BaseKeeper) + +MintCoins(ctx sdk.Context, moduleName string, amt sdk.Coins) + +error { + defer metrics.MeasureSince(time.Now(), "MintCoins") + // ... +} +``` + +## Consequences + +### Positive + +* Exposure into the performance and behavior of an application + +### Negative + +### Neutral + +## References diff --git a/sdk/v0.54/reference/architecture/adr-014-proportional-slashing.mdx b/sdk/v0.54/reference/architecture/adr-014-proportional-slashing.mdx new file mode 100644 index 000000000..28d43b713 --- /dev/null +++ b/sdk/v0.54/reference/architecture/adr-014-proportional-slashing.mdx @@ -0,0 +1,92 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/architecture/adr-014-proportional-slashing' +title: 'ADR 14: Proportional Slashing' +description: >- + 2019-10-15: Initial draft 2020-05-25: Removed correlation root slashing + 2020-07-01: Updated to include S-curve function instead of linear +--- + +## Changelog + +* 2019-10-15: Initial draft +* 2020-05-25: Removed correlation root slashing +* 2020-07-01: Updated to include S-curve function instead of linear + +## Context + +In Proof of Stake-based chains, centralization of consensus power amongst a small set of validators can cause harm to the network due to increased risk of censorship, liveness failure, fork attacks, etc. However, while this centralization causes a negative externality to the network, it is not directly felt by the delegators contributing towards delegating towards already large validators. We would like a way to pass on the negative externality cost of centralization onto those large validators and their delegators. + +## Decision + +### Design + +To solve this problem, we will implement a procedure called Proportional Slashing. The desire is that the larger a validator is, the more they should be slashed. The first naive attempt is to make a validator's slash percent proportional to their share of consensus voting power. + +```text +slash_amount = k * power // power is the faulting validator's voting power and k is some on-chain constant +``` + +However, this will incentivize validators with large amounts of stake to split up their voting power amongst accounts (sybil attack), so that if they fault, they all get slashed at a lower percent. The solution to this is to take into account not just a validator's own voting percentage, but also the voting percentage of all the other validators who get slashed in a specified time frame. + +```text +slash_amount = k * (power_1 + power_2 + ... + power_n) // where power_i is the voting power of the ith validator faulting in the specified time frame and k is some on-chain constant +``` + +Now, if someone splits a validator of 10% into two validators of 5% each which both fault, then they both fault in the same time frame, they both will get slashed at the sum 10% amount. + +However in practice, we likely don't want a linear relation between amount of stake at fault, and the percentage of stake to slash. In particular, solely 5% of stake double signing effectively did nothing to majorly threaten security, whereas 30% of stake being at fault clearly merits a large slashing factor, due to being very close to the point at which Tendermint security is threatened. A linear relation would require a factor of 6 gap between these two, whereas the difference in risk posed to the network is much larger. We propose using S-curves (formally [logistic functions](https://en.wikipedia.org/wiki/Logistic_function) to solve this). S-Curves capture the desired criterion quite well. They allow the slashing factor to be minimal for small values, and then grow very rapidly near some threshold point where the risk posed becomes notable. + +#### Parameterization + +This requires parameterizing a logistic function. It is very well understood how to parameterize this. It has four parameters: + +1. A minimum slashing factor +2. A maximum slashing factor +3. The inflection point of the S-curve (essentially where do you want to center the S) +4. The rate of growth of the S-curve (How elongated is the S) + +#### Correlation across non-sybil validators + +One will note, that this model doesn't differentiate between multiple validators run by the same operators vs validators run by different operators. This can be seen as an additional benefit in fact. It incentivizes validators to differentiate their setups from other validators, to avoid having correlated faults with them or else they risk a higher slash. So for example, operators should avoid using the same popular cloud hosting platforms or using the same Staking as a Service providers. This will lead to a more resilient and decentralized network. + +#### Griefing + +Griefing, the act of intentionally getting oneself slashed in order to make another's slash worse, could be a concern here. However, using the protocol described here, the attacker also gets equally impacted by the grief as the victim, so it would not provide much benefit to the griefer. + +### Implementation + +In the slashing module, we will add two queues that will track all of the recent slash events. For double sign faults, we will define "recent slashes" as ones that have occurred within the last `unbonding period`. For liveness faults, we will define "recent slashes" as ones that have occurred withing the last `jail period`. + +```go +type SlashEvent struct { + Address sdk.ValAddress + ValidatorVotingPercent sdk.Dec + SlashedSoFar sdk.Dec +} +``` + +These slash events will be pruned from the queue once they are older than their respective "recent slash period". + +Whenever a new slash occurs, a `SlashEvent` struct is created with the faulting validator's voting percent and a `SlashedSoFar` of 0. Because recent slash events are pruned before the unbonding period and unjail period expires, it should not be possible for the same validator to have multiple SlashEvents in the same Queue at the same time. + +We then will iterate over all the SlashEvents in the queue, adding their `ValidatorVotingPercent` to calculate the new percent to slash all the validators in the queue at, using the "Square of Sum of Roots" formula introduced above. + +Once we have the `NewSlashPercent`, we then iterate over all the `SlashEvent`s in the queue once again, and if `NewSlashPercent > SlashedSoFar` for that SlashEvent, we call the `staking.Slash(slashEvent.Address, slashEvent.Power, Math.Min(Math.Max(minSlashPercent, NewSlashPercent - SlashedSoFar), maxSlashPercent)` (we pass in the power of the validator before any slashes occurred, so that we slash the right amount of tokens). We then set `SlashEvent.SlashedSoFar` amount to `NewSlashPercent`. + +## Status + +Proposed + +## Consequences + +### Positive + +* Increases decentralization by disincentivizing delegating to large validators +* Incentivizes Decorrelation of Validators +* More severely punishes attacks than accidental faults +* More flexibility in slashing rates parameterization + +### Negative + +* More computationally expensive than current implementation. Will require more data about "recent slashing events" to be stored on chain. diff --git a/sdk/v0.54/reference/architecture/adr-016-validator-consensus-key-rotation.mdx b/sdk/v0.54/reference/architecture/adr-016-validator-consensus-key-rotation.mdx new file mode 100644 index 000000000..5941412f9 --- /dev/null +++ b/sdk/v0.54/reference/architecture/adr-016-validator-consensus-key-rotation.mdx @@ -0,0 +1,134 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/architecture/adr-016-validator-consensus-key-rotation' +title: 'ADR 016: Validator Consensus Key Rotation' +description: '2019 Oct 23: Initial draft 2019 Nov 28: Add key rotation fee' +--- + +## Changelog + +* 2019 Oct 23: Initial draft +* 2019 Nov 28: Add key rotation fee + +## Context + +Validator consensus key rotation feature has been discussed and requested for a long time, for the sake of safer validator key management policy (e.g. [Link](https://github.com/tendermint/tendermint/issues/1136)). So, we suggest one of the simplest form of validator consensus key rotation implementation mostly onto Cosmos SDK. + +We don't need to make any update on consensus logic in Tendermint because Tendermint does not have any mapping information of consensus key and validator operator key, meaning that from Tendermint point of view, a consensus key rotation of a validator is simply a replacement of a consensus key to another. + +Also, it should be noted that this ADR includes only the simplest form of consensus key rotation without considering multiple consensus keys concept. Such multiple consensus keys concept shall remain a long term goal of Tendermint and Cosmos SDK. + +## Decision + +### Pseudo procedure for consensus key rotation + +* create new random consensus key. +* create and broadcast a transaction with a `MsgRotateConsPubKey` that states the new consensus key is now coupled with the validator operator with signature from the validator's operator key. +* old consensus key becomes unable to participate on consensus immediately after the update of key mapping state on-chain. +* start validating with new consensus key. +* validators using HSM and KMS should update the consensus key in HSM to use the new rotated key after the height `h` when `MsgRotateConsPubKey` committed to the blockchain. + +### Considerations + +* consensus key mapping information management strategy + * store history of each key mapping changes in the kvstore. + * the state machine can search corresponding consensus key paired with given validator operator for any arbitrary height in a recent unbonding period. + * the state machine does not need any historical mapping information which is past more than unbonding period. +* key rotation costs related to LCD and IBC + * LCD and IBC will have traffic/computation burden when there exists frequent power changes + * In current Tendermint design, consensus key rotations are seen as power changes from LCD or IBC perspective + * Therefore, to minimize unnecessary frequent key rotation behavior, we limited maximum number of rotation in recent unbonding period and also applied exponentially increasing rotation fee +* limits + * a validator cannot rotate its consensus key more than `MaxConsPubKeyRotations` time for any unbonding period, to prevent spam. + * parameters can be decided by governance and stored in genesis file. +* key rotation fee + * a validator should pay `KeyRotationFee` to rotate the consensus key which is calculated as below + * `KeyRotationFee` = (max(`VotingPowerPercentage` *100, 1)* `InitialKeyRotationFee`) \* 2^(number of rotations in `ConsPubKeyRotationHistory` in recent unbonding period) +* evidence module + * evidence module can search corresponding consensus key for any height from slashing keeper so that it can decide which consensus key is supposed to be used for given height. +* abci.ValidatorUpdate + * tendermint already has ability to change a consensus key by ABCI communication(`ValidatorUpdate`). + * validator consensus key update can be done via creating new + delete old by change the power to zero. + * therefore, we expect we even do not need to change tendermint codebase at all to implement this feature. +* new genesis parameters in `staking` module + * `MaxConsPubKeyRotations` : maximum number of rotation can be executed by a validator in recent unbonding period. default value 10 is suggested(11th key rotation will be rejected) + * `InitialKeyRotationFee` : the initial key rotation fee when no key rotation has happened in recent unbonding period. default value 1atom is suggested(1atom fee for the first key rotation in recent unbonding period) + +### Workflow + +1. The validator generates a new consensus keypair. + +2. The validator generates and signs a `MsgRotateConsPubKey` tx with their operator key and new ConsPubKey + + ```go + type MsgRotateConsPubKey struct { + ValidatorAddress sdk.ValAddress + NewPubKey crypto.PubKey + } + ``` + +3. `handleMsgRotateConsPubKey` gets `MsgRotateConsPubKey`, calls `RotateConsPubKey` with emits event + +4. `RotateConsPubKey` + + * checks if `NewPubKey` is not duplicated on `ValidatorsByConsAddr` + * checks if the validator is does not exceed parameter `MaxConsPubKeyRotations` by iterating `ConsPubKeyRotationHistory` + * checks if the signing account has enough balance to pay `KeyRotationFee` + * pays `KeyRotationFee` to community fund + * overwrites `NewPubKey` in `validator.ConsPubKey` + * deletes old `ValidatorByConsAddr` + * `SetValidatorByConsAddr` for `NewPubKey` + * Add `ConsPubKeyRotationHistory` for tracking rotation + + ```go + type ConsPubKeyRotationHistory struct { + OperatorAddress sdk.ValAddress + OldConsPubKey crypto.PubKey + NewConsPubKey crypto.PubKey + RotatedHeight int64 + } + ``` + +5. `ApplyAndReturnValidatorSetUpdates` checks if there is `ConsPubKeyRotationHistory` with `ConsPubKeyRotationHistory.RotatedHeight == ctx.BlockHeight()` and if so, generates 2 `ValidatorUpdate` , one for a remove validator and one for create new validator + + ```go + abci.ValidatorUpdate{ + PubKey: cmttypes.TM2PB.PubKey(OldConsPubKey), + Power: 0, + } + + abci.ValidatorUpdate{ + PubKey: cmttypes.TM2PB.PubKey(NewConsPubKey), + Power: v.ConsensusPower(), + } + ``` + +6. at `previousVotes` Iteration logic of `AllocateTokens`, `previousVote` using `OldConsPubKey` match up with `ConsPubKeyRotationHistory`, and replace validator for token allocation + +7. Migrate `ValidatorSigningInfo` and `ValidatorMissedBlockBitArray` from `OldConsPubKey` to `NewConsPubKey` + +* Note : All above features shall be implemented in `staking` module. + +## Status + +Proposed + +## Consequences + +### Positive + +* Validators can immediately or periodically rotate their consensus key to have better security policy +* improved security against Long-Range attacks given a validator throws away the old consensus key(s) + +### Negative + +* Slash module needs more computation because it needs to lookup corresponding consensus key of validators for each height +* frequent key rotations will make light client bisection less efficient + +### Neutral + +## References + +* on tendermint repo : [Link](https://github.com/tendermint/tendermint/issues/1136) +* on cosmos-sdk repo : [Link](https://github.com/cosmos/cosmos-sdk/issues/5231) +* about multiple consensus keys : [Link](https://github.com/tendermint/tendermint/issues/1758#issuecomment-545291698) diff --git a/sdk/v0.54/reference/architecture/adr-017-historical-header-module.mdx b/sdk/v0.54/reference/architecture/adr-017-historical-header-module.mdx new file mode 100644 index 000000000..2be676fcd --- /dev/null +++ b/sdk/v0.54/reference/architecture/adr-017-historical-header-module.mdx @@ -0,0 +1,72 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/architecture/adr-017-historical-header-module' +title: 'ADR 17: Historical Header Module' +description: >- + 26 November 2019: Start of first version 2 December 2019: Final draft of first + version +--- + +## Changelog + +* 26 November 2019: Start of first version +* 2 December 2019: Final draft of first version + +## Context + +In order for the Cosmos SDK to implement the [IBC specification](https://github.com/cosmos/ics), modules within the Cosmos SDK must have the ability to introspect recent consensus states (validator sets & commitment roots) as proofs of these values on other chains must be checked during the handshakes. + +## Decision + +The application MUST store the most recent `n` headers in a persistent store. At first, this store MAY be the current Merklised store. A non-Merklised store MAY be used later as no proofs are necessary. + +The application MUST store this information by storing new headers immediately when handling `abci.RequestBeginBlock`: + +```go +func BeginBlock(ctx sdk.Context, keeper HistoricalHeaderKeeper, req abci.RequestBeginBlock) + +abci.ResponseBeginBlock { + info := HistoricalInfo{ + Header: ctx.BlockHeader(), + ValSet: keeper.StakingKeeper.GetAllValidators(ctx), // note that this must be stored in a canonical order +} + +keeper.SetHistoricalInfo(ctx, ctx.BlockHeight(), info) + n := keeper.GetParamRecentHeadersToStore() + +keeper.PruneHistoricalInfo(ctx, ctx.BlockHeight() - n) + // continue handling request +} +``` + +Alternatively, the application MAY store only the hash of the validator set. + +The application MUST make these past `n` committed headers available for querying by Cosmos SDK modules through the `Keeper`'s `GetHistoricalInfo` function. This MAY be implemented in a new module, or it MAY also be integrated into an existing one (likely `x/staking` or `x/ibc`). + +`n` MAY be configured as a parameter store parameter, in which case it could be changed by `ParameterChangeProposal`s, although it will take some blocks for the stored information to catch up if `n` is increased. + +## Status + +Proposed. + +## Consequences + +Implementation of this ADR will require changes to the Cosmos SDK. It will not require changes to Tendermint. + +### Positive + +* Easy retrieval of headers & state roots for recent past heights by modules anywhere in the Cosmos SDK. +* No RPC calls to Tendermint required. +* No ABCI alterations required. + +### Negative + +* Duplicates `n` headers data in Tendermint & the application (additional disk usage) - in the long term, an approach such as [this](https://github.com/tendermint/tendermint/issues/4210) might be preferable. + +### Neutral + +(none known) + +## References + +* [ICS 2: "Consensus state introspection"](https://github.com/cosmos/ibc/tree/master/spec/core/ics-002-client-semantics#consensus-state-introspection) diff --git a/sdk/v0.54/reference/architecture/adr-018-extendable-voting-period.mdx b/sdk/v0.54/reference/architecture/adr-018-extendable-voting-period.mdx new file mode 100644 index 000000000..74d7f1f44 --- /dev/null +++ b/sdk/v0.54/reference/architecture/adr-018-extendable-voting-period.mdx @@ -0,0 +1,71 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/architecture/adr-018-extendable-voting-period' +title: 'ADR 18: Extendable Voting Periods' +description: '1 January 2020: Start of first version' +--- + +## Changelog + +* 1 January 2020: Start of first version + +## Context + +Currently the voting period for all governance proposals is the same. However, this is suboptimal as all governance proposals do not require the same time period. For more non-contentious proposals, they can be dealt with more efficiently with a faster period, while more contentious or complex proposals may need a longer period for extended discussion/consideration. + +## Decision + +We would like to design a mechanism for making the voting period of a governance proposal variable based on the demand of voters. We would like it to be based on the view of the governance participants, rather than just the proposer of a governance proposal (thus, allowing the proposer to select the voting period length is not sufficient). + +However, we would like to avoid the creation of an entire second voting process to determine the length of the voting period, as it just pushed the problem to determining the length of that first voting period. + +Thus, we propose the following mechanism: + +### Params + +* The current gov param `VotingPeriod` is to be replaced by a `MinVotingPeriod` param. This is the default voting period that all governance proposal voting periods start with. +* There is a new gov param called `MaxVotingPeriodExtension`. + +### Mechanism + +There is a new `Msg` type called `MsgExtendVotingPeriod`, which can be sent by any staked account during a proposal's voting period. It allows the sender to unilaterally extend the length of the voting period by `MaxVotingPeriodExtension * sender's share of voting power`. Every address can only call `MsgExtendVotingPeriod` once per proposal. + +So for example, if the `MaxVotingPeriodExtension` is set to 100 Days, then anyone with 1% of voting power can extend the voting power by 1 day. If 33% of voting power has sent the message, the voting period will be extended by 33 days. Thus, if absolutely everyone chooses to extend the voting period, the absolute maximum voting period will be `MinVotingPeriod + MaxVotingPeriodExtension`. + +This system acts as a sort of distributed coordination, where individual stakers choosing to extend or not, allows the system the guage the conentiousness/complexity of the proposal. It is extremely unlikely that many stakers will choose to extend at the exact same time, it allows stakers to view how long others have already extended thus far, to decide whether or not to extend further. + +### Dealing with Unbonding/Redelegation + +There is one thing that needs to be addressed. How to deal with redelegation/unbonding during the voting period. If a staker of 5% calls `MsgExtendVotingPeriod` and then unbonds, does the voting period then decrease by 5 days again? This is not good as it can give people a false sense of how long they have to make their decision. For this reason, we want to design it such that the voting period length can only be extended, not shortened. To do this, the current extension amount is based on the highest percent that voted extension at any time. This is best explained by example: + +1. Let's say 2 stakers of voting power 4% and 3% respectively vote to extend. The voting period will be extended by 7 days. +2. Now the staker of 3% decides to unbond before the end of the voting period. The voting period extension remains 7 days. +3. Now, let's say another staker of 2% voting power decides to extend voting period. There is now 6% of active voting power choosing the extend. The voting power remains 7 days. +4. If a fourth staker of 10% chooses to extend now, there is a total of 16% of active voting power wishing to extend. The voting period will be extended to 16 days. + +### Delegators + +Just like votes in the actual voting period, delegators automatically inherit the extension of their validators. If their validator chooses to extend, their voting power will be used in the validator's extension. However, the delegator is unable to override their validator and "unextend" as that would contradict the "voting power length can only be ratcheted up" principle described in the previous section. However, a delegator may choose the extend using their personal voting power, if their validator has not done so. + +## Status + +Proposed + +## Consequences + +### Positive + +* More complex/contentious governance proposals will have more time to properly digest and deliberate + +### Negative + +* Governance process becomes more complex and requires more understanding to interact with effectively +* Can no longer predict when a governance proposal will end. Can't assume order in which governance proposals will end. + +### Neutral + +* The minimum voting period can be made shorter + +## References + +* [Cosmos Forum post where idea first originated](https://forum.cosmos.network/t/proposal-draft-reduce-governance-voting-period-to-7-days/3032/9) diff --git a/sdk/v0.54/reference/architecture/adr-019-protobuf-state-encoding.mdx b/sdk/v0.54/reference/architecture/adr-019-protobuf-state-encoding.mdx new file mode 100644 index 000000000..28e420e3b --- /dev/null +++ b/sdk/v0.54/reference/architecture/adr-019-protobuf-state-encoding.mdx @@ -0,0 +1,403 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/architecture/adr-019-protobuf-state-encoding' +title: 'ADR 019: Protocol Buffer State Encoding' +--- + +## Changelog + +* 2020 Feb 15: Initial Draft +* 2020 Feb 24: Updates to handle messages with interface fields +* 2020 Apr 27: Convert usages of `oneof` for interfaces to `Any` +* 2020 May 15: Describe `cosmos_proto` extensions and amino compatibility +* 2020 Dec 4: Move and rename `MarshalAny` and `UnmarshalAny` into the `codec.Codec` interface. +* 2021 Feb 24: Remove mentions of `HybridCodec`, which has been abandoned in [#6843](https://github.com/cosmos/cosmos-sdk/pull/6843). + +## Status + +Accepted + +## Context + +Currently, the Cosmos SDK utilizes [go-amino](https://github.com/tendermint/go-amino/) for binary +and JSON object encoding over the wire bringing parity between logical objects and persistence objects. + +From the Amino docs: + +> Amino is an object encoding specification. It is a subset of Proto3 with an extension for interface +> support. See the [Proto3 spec](https://developers.google.com/protocol-buffers/docs/proto3) for more +> information on Proto3, which Amino is largely compatible with (but not with Proto2). +> +> The goal of the Amino encoding protocol is to bring parity into logic objects and persistence objects. + +Amino also aims to have the following goals (not a complete list): + +* Binary bytes must be decode-able with a schema. +* Schema must be upgradeable. +* The encoder and decoder logic must be reasonably simple. + +However, we believe that Amino does not fulfill these goals completely and does not fully meet the +needs of a truly flexible cross-language and multi-client compatible encoding protocol in the Cosmos SDK. +Namely, Amino has proven to be a big pain-point in regards to supporting object serialization across +clients written in various languages while providing virtually little in the way of true backwards +compatibility and upgradeability. Furthermore, through profiling and various benchmarks, Amino has +been shown to be an extremely large performance bottleneck in the Cosmos SDK 1. This is +largely reflected in the performance of simulations and application transaction throughput. + +Thus, we need to adopt an encoding protocol that meets the following criteria for state serialization: + +* Language agnostic +* Platform agnostic +* Rich client support and thriving ecosystem +* High performance +* Minimal encoded message size +* Codegen-based over reflection-based +* Supports backward and forward compatibility + +Note, migrating away from Amino should be viewed as a two-pronged approach, state and client encoding. +This ADR focuses on state serialization in the Cosmos SDK state machine. A corresponding ADR will be +made to address client-side encoding. + +## Decision + +We will adopt [Protocol Buffers](https://developers.google.com/protocol-buffers) for serializing +persisted structured data in the Cosmos SDK while providing a clean mechanism and developer UX for +applications wishing to continue to use Amino. We will provide this mechanism by updating modules to +accept a codec interface, `Marshaler`, instead of a concrete Amino codec. Furthermore, the Cosmos SDK +will provide two concrete implementations of the `Marshaler` interface: `AminoCodec` and `ProtoCodec`. + +* `AminoCodec`: Uses Amino for both binary and JSON encoding. +* `ProtoCodec`: Uses Protobuf for both binary and JSON encoding. + +Modules will use whichever codec that is instantiated in the app. By default, the Cosmos SDK's `simapp` +instantiates a `ProtoCodec` as the concrete implementation of `Marshaler`, inside the `MakeTestEncodingConfig` +function. This can be easily overwritten by app developers if they so desire. + +The ultimate goal will be to replace Amino JSON encoding with Protobuf encoding and thus have +modules accept and/or extend `ProtoCodec`. Until then, Amino JSON is still provided for legacy use-cases. +A handful of places in the Cosmos SDK still have Amino JSON hardcoded, such as the Legacy API REST endpoints +and the `x/params` store. They are planned to be converted to Protobuf in a gradual manner. + +### Module Codecs + +Modules that do not require the ability to work with and serialize interfaces, the path to Protobuf +migration is pretty straightforward. These modules are to simply migrate any existing types that +are encoded and persisted via their concrete Amino codec to Protobuf and have their keeper accept a +`Marshaler` that will be a `ProtoCodec`. This migration is simple as things will just work as-is. + +Note, any business logic that needs to encode primitive types like `bool` or `int64` should use +[gogoprotobuf](https://github.com/cosmos/gogoproto) Value types. + +Example: + +```go +ts, err := gogotypes.TimestampProto(completionTime) + if err != nil { + // ... +} + bz := cdc.MustMarshal(ts) +``` + +However, modules can vary greatly in purpose and design and so we must support the ability for modules +to be able to encode and work with interfaces (e.g. `Account` or `Content`). For these modules, they +must define their own codec interface that extends `Marshaler`. These specific interfaces are unique +to the module and will contain method contracts that know how to serialize the needed interfaces. + +Example: + +```go expandable +// x/auth/types/codec.go + +type Codec interface { + codec.Codec + + MarshalAccount(acc exported.Account) ([]byte, error) + +UnmarshalAccount(bz []byte) (exported.Account, error) + +MarshalAccountJSON(acc exported.Account) ([]byte, error) + +UnmarshalAccountJSON(bz []byte) (exported.Account, error) +} +``` + +### Usage of `Any` to encode interfaces + +In general, module-level .proto files should define messages which encode interfaces +using [`google.protobuf.Any`](https://github.com/protocolbuffers/protobuf/blob/master/src/google/protobuf/any.proto). +After [extension discussion](https://github.com/cosmos/cosmos-sdk/issues/6030), +this was chosen as the preferred alternative to application-level `oneof`s +as in our original protobuf design. The arguments in favor of `Any` can be +summarized as follows: + +* `Any` provides a simpler, more consistent client UX for dealing with + interfaces than app-level `oneof`s that will need to be coordinated more + carefully across applications. Creating a generic transaction + signing library using `oneof`s may be cumbersome and critical logic may need + to be reimplemented for each chain +* `Any` provides more resistance against human error than `oneof` +* `Any` is generally simpler to implement for both modules and apps + +The main counter-argument to using `Any` centers around its additional space +and possibly performance overhead. The space overhead could be dealt with using +compression at the persistence layer in the future and the performance impact +is likely to be small. Thus, not using `Any` is seem as a pre-mature optimization, +with user experience as the higher order concern. + +Note, that given the Cosmos SDK's decision to adopt the `Codec` interfaces described +above, apps can still choose to use `oneof` to encode state and transactions +but it is not the recommended approach. If apps do choose to use `oneof`s +instead of `Any` they will likely lose compatibility with client apps that +support multiple chains. Thus developers should think carefully about whether +they care more about what is possibly a pre-mature optimization or end-user +and client developer UX. + +### Safe usage of `Any` + +By default, the [gogo protobuf implementation of `Any`](https://pkg.go.dev/github.com/cosmos/gogoproto/types) +uses [global type registration](https://github.com/cosmos/gogoproto/blob/master/proto/properties.go#L540) +to decode values packed in `Any` into concrete +go types. This introduces a vulnerability where any malicious module +in the dependency tree could register a type with the global protobuf registry +and cause it to be loaded and unmarshaled by a transaction that referenced +it in the `type_url` field. + +To prevent this, we introduce a type registration mechanism for decoding `Any` +values into concrete types through the `InterfaceRegistry` interface which +bears some similarity to type registration with Amino: + +```go expandable +type InterfaceRegistry interface { + // RegisterInterface associates protoName as the public name for the + // interface passed in as iface + // Ex: + // registry.RegisterInterface("cosmos_sdk.Msg", (*sdk.Msg)(nil)) + +RegisterInterface(protoName string, iface interface{ +}) + + // RegisterImplementations registers impls as a concrete implementations of + // the interface iface + // Ex: + // registry.RegisterImplementations((*sdk.Msg)(nil), &MsgSend{ +}, &MsgMultiSend{ +}) + +RegisterImplementations(iface interface{ +}, impls ...proto.Message) +} +``` + +In addition to serving as a whitelist, `InterfaceRegistry` can also serve +to communicate the list of concrete types that satisfy an interface to clients. + +In .proto files: + +* fields which accept interfaces should be annotated with `cosmos_proto.accepts_interface` + using the same full-qualified name passed as `protoName` to `InterfaceRegistry.RegisterInterface` +* interface implementations should be annotated with `cosmos_proto.implements_interface` + using the same full-qualified name passed as `protoName` to `InterfaceRegistry.RegisterInterface` + +In the future, `protoName`, `cosmos_proto.accepts_interface`, `cosmos_proto.implements_interface` +may be used via code generation, reflection &/or static linting. + +The same struct that implements `InterfaceRegistry` will also implement an +interface `InterfaceUnpacker` to be used for unpacking `Any`s: + +```go +type InterfaceUnpacker interface { + // UnpackAny unpacks the value in any to the interface pointer passed in as + // iface. Note that the type in any must have been registered with + // RegisterImplementations as a concrete type for that interface + // Ex: + // var msg sdk.Msg + // err := ctx.UnpackAny(any, &msg) + // ... + UnpackAny(any *Any, iface interface{ +}) + +error +} +``` + +Note that `InterfaceRegistry` usage does not deviate from standard protobuf +usage of `Any`, it just introduces a security and introspection layer for +golang usage. + +`InterfaceRegistry` will be a member of `ProtoCodec` +described above. In order for modules to register interface types, app modules +can optionally implement the following interface: + +```go +type InterfaceModule interface { + RegisterInterfaceTypes(InterfaceRegistry) +} +``` + +The module manager will include a method to call `RegisterInterfaceTypes` on +every module that implements it in order to populate the `InterfaceRegistry`. + +### Using `Any` to encode state + +The Cosmos SDK will provide support methods `MarshalInterface` and `UnmarshalInterface` to hide a complexity of wrapping interface types into `Any` and allow easy serialization. + +```go expandable +import "github.com/cosmos/cosmos-sdk/codec" + +// note: eviexported.Evidence is an interface type +func MarshalEvidence(cdc codec.BinaryCodec, e eviexported.Evidence) ([]byte, error) { + return cdc.MarshalInterface(e) +} + +func UnmarshalEvidence(cdc codec.BinaryCodec, bz []byte) (eviexported.Evidence, error) { + var evi eviexported.Evidence + err := cdc.UnmarshalInterface(&evi, bz) + +return err, nil +} +``` + +### Using `Any` in `sdk.Msg`s + +A similar concept is to be applied for messages that contain interfaces fields. +For example, we can define `MsgSubmitEvidence` as follows where `Evidence` is +an interface: + +```protobuf +// x/evidence/types/types.proto + +message MsgSubmitEvidence { + bytes submitter = 1 + [ + (gogoproto.casttype) = "github.com/cosmos/cosmos-sdk/types.AccAddress" + ]; + google.protobuf.Any evidence = 2; +} +``` + +Note that in order to unpack the evidence from `Any` we do need a reference to +`InterfaceRegistry`. In order to reference evidence in methods like +`ValidateBasic` which shouldn't have to know about the `InterfaceRegistry`, we +introduce an `UnpackInterfaces` phase to deserialization which unpacks +interfaces before they're needed. + +### Unpacking Interfaces + +To implement the `UnpackInterfaces` phase of deserialization which unpacks +interfaces wrapped in `Any` before they're needed, we create an interface +that `sdk.Msg`s and other types can implement: + +```go +type UnpackInterfacesMessage interface { + UnpackInterfaces(InterfaceUnpacker) + +error +} +``` + +We also introduce a private `cachedValue interface{}` field onto the `Any` +struct itself with a public getter `GetCachedValue() interface{}`. + +The `UnpackInterfaces` method is to be invoked during message deserialization right +after `Unmarshal` and any interface values packed in `Any`s will be decoded +and stored in `cachedValue` for reference later. + +Then unpacked interface values can safely be used in any code afterwards +without knowledge of the `InterfaceRegistry` +and messages can introduce a simple getter to cast the cached value to the +correct interface type. + +This has the added benefit that unmarshaling of `Any` values only happens once +during initial deserialization rather than every time the value is read. Also, +when `Any` values are first packed (for instance in a call to +`NewMsgSubmitEvidence`), the original interface value is cached so that +unmarshaling isn't needed to read it again. + +`MsgSubmitEvidence` could implement `UnpackInterfaces`, plus a convenience getter +`GetEvidence` as follows: + +```go +func (msg MsgSubmitEvidence) + +UnpackInterfaces(ctx sdk.InterfaceRegistry) + +error { + var evi eviexported.Evidence + return ctx.UnpackAny(msg.Evidence, *evi) +} + +func (msg MsgSubmitEvidence) + +GetEvidence() + +eviexported.Evidence { + return msg.Evidence.GetCachedValue().(eviexported.Evidence) +} +``` + +### Amino Compatibility + +Our custom implementation of `Any` can be used transparently with Amino if used +with the proper codec instance. What this means is that interfaces packed within +`Any`s will be amino marshaled like regular Amino interfaces (assuming they +have been registered properly with Amino). + +In order for this functionality to work: + +* **all legacy code must use `*codec.LegacyAmino` instead of `*amino.Codec` which is + now a wrapper which properly handles `Any`** +* **all new code should use `Marshaler` which is compatible with both amino and + protobuf** +* Also, before v0.39, `codec.LegacyAmino` will be renamed to `codec.LegacyAmino`. + +### Why Wasn't X Chosen Instead + +For a more complete comparison to alternative protocols, see [here](https://codeburst.io/json-vs-protocol-buffers-vs-flatbuffers-a4247f8bda6f). + +### Cap'n Proto + +While [Cap’n Proto](https://capnproto.org/) does seem like an advantageous alternative to Protobuf +due to it's native support for interfaces/generics and built in canonicalization, it does lack the +rich client ecosystem compared to Protobuf and is a bit less mature. + +### FlatBuffers + +[FlatBuffers](https://google.github.io/flatbuffers/) is also a potentially viable alternative, with the +primary difference being that FlatBuffers does not need a parsing/unpacking step to a secondary +representation before you can access data, often coupled with per-object memory allocation. + +However, it would require great efforts into research and full understanding the scope of the migration +and path forward -- which isn't immediately clear. In addition, FlatBuffers aren't designed for +untrusted inputs. + +## Future Improvements & Roadmap + +In the future we may consider a compression layer right above the persistence +layer which doesn't change tx or merkle tree hashes, but reduces the storage +overhead of `Any`. In addition, we may adopt protobuf naming conventions which +make type URLs a bit more concise while remaining descriptive. + +Additional code generation support around the usage of `Any` is something that +could also be explored in the future to make the UX for go developers more +seamless. + +## Consequences + +### Positive + +* Significant performance gains. +* Supports backward and forward type compatibility. +* Better support for cross-language clients. + +### Negative + +* Learning curve required to understand and implement Protobuf messages. +* Slightly larger message size due to use of `Any`, although this could be offset + by a compression layer in the future + +### Neutral + +## References + +1. [Link](https://github.com/cosmos/cosmos-sdk/issues/4977) +2. [Link](https://github.com/cosmos/cosmos-sdk/issues/5444) diff --git a/sdk/v0.54/reference/architecture/adr-020-protobuf-transaction-encoding.mdx b/sdk/v0.54/reference/architecture/adr-020-protobuf-transaction-encoding.mdx new file mode 100644 index 000000000..eb1dd055a --- /dev/null +++ b/sdk/v0.54/reference/architecture/adr-020-protobuf-transaction-encoding.mdx @@ -0,0 +1,493 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/architecture/adr-020-protobuf-transaction-encoding' +title: 'ADR 020: Protocol Buffer Transaction Encoding' +--- + +## Changelog + +* 2020 March 06: Initial Draft +* 2020 March 12: API Updates +* 2020 April 13: Added details on interface `oneof` handling +* 2020 April 30: Switch to `Any` +* 2020 May 14: Describe public key encoding +* 2020 June 08: Store `TxBody` and `AuthInfo` as bytes in `SignDoc`; Document `TxRaw` as broadcast and storage type. +* 2020 August 07: Use ADR 027 for serializing `SignDoc`. +* 2020 August 19: Move sequence field from `SignDoc` to `SignerInfo`, as discussed in [#6966](https://github.com/cosmos/cosmos-sdk/issues/6966). +* 2020 September 25: Remove `PublicKey` type in favor of `secp256k1.PubKey`, `ed25519.PubKey` and `multisig.LegacyAminoPubKey`. +* 2020 October 15: Add `GetAccount` and `GetAccountWithHeight` methods to the `AccountRetriever` interface. +* 2021 Feb 24: The Cosmos SDK does not use Tendermint's `PubKey` interface anymore, but its own `cryptotypes.PubKey`. Updates to reflect this. +* 2021 May 3: Rename `clientCtx.JSONMarshaler` to `clientCtx.JSONCodec`. +* 2021 June 10: Add `clientCtx.Codec: codec.Codec`. + +## Status + +Accepted + +## Context + +This ADR is a continuation of the motivation, design, and context established in +[ADR 019](/sdk/v0.50/build/architecture/adr-019-protobuf-state-encoding), namely, we aim to design the +Protocol Buffer migration path for the client-side of the Cosmos SDK. + +Specifically, the client-side migration path primarily includes tx generation and +signing, message construction and routing, in addition to CLI & REST handlers and +business logic (i.e. queriers). + +With this in mind, we will tackle the migration path via two main areas, txs and +querying. However, this ADR solely focuses on transactions. Querying should be +addressed in a future ADR, but it should build off of these proposals. + +Based on detailed discussions ([#6030](https://github.com/cosmos/cosmos-sdk/issues/6030) +and [#6078](https://github.com/cosmos/cosmos-sdk/issues/6078)), the original +design for transactions was changed substantially from an `oneof` /JSON-signing +approach to the approach described below. + +## Decision + +### Transactions + +Since interface values are encoded with `google.protobuf.Any` in state (see [ADR 019](/sdk/v0.54/reference/architecture/adr-019-protobuf-state-encoding)), +`sdk.Msg`s are encoding with `Any` in transactions. + +One of the main goals of using `Any` to encode interface values is to have a +core set of types which is reused by apps so that +clients can safely be compatible with as many chains as possible. + +It is one of the goals of this specification to provide a flexible cross-chain transaction +format that can serve a wide variety of use cases without breaking client +compatibility. + +In order to facilitate signing, transactions are separated into `TxBody`, +which will be re-used by `SignDoc` below, and `signatures`: + +```protobuf expandable +// types/types.proto +package cosmos_sdk.v1; + +message Tx { + TxBody body = 1; + AuthInfo auth_info = 2; + // A list of signatures that matches the length and order of AuthInfo's signer_infos to + // allow connecting signature meta information like public key and signing mode by position. + repeated bytes signatures = 3; +} + +// A variant of Tx that pins the signer's exact binary represenation of body and +// auth_info. This is used for signing, broadcasting and verification. The binary +// `serialize(tx: TxRaw)` is stored in Tendermint and the hash `sha256(serialize(tx: TxRaw))` +// becomes the "txhash", commonly used as the transaction ID. +message TxRaw { + // A protobuf serialization of a TxBody that matches the representation in SignDoc. + bytes body = 1; + // A protobuf serialization of an AuthInfo that matches the representation in SignDoc. + bytes auth_info = 2; + // A list of signatures that matches the length and order of AuthInfo's signer_infos to + // allow connecting signature meta information like public key and signing mode by position. + repeated bytes signatures = 3; +} + +message TxBody { + // A list of messages to be executed. The required signers of those messages define + // the number and order of elements in AuthInfo's signer_infos and Tx's signatures. + // Each required signer address is added to the list only the first time it occurs. + // + // By convention, the first required signer (usually from the first message) is referred + // to as the primary signer and pays the fee for the whole transaction. + repeated google.protobuf.Any messages = 1; + string memo = 2; + int64 timeout_height = 3; + repeated google.protobuf.Any extension_options = 1023; +} + +message AuthInfo { + // This list defines the signing modes for the required signers. The number + // and order of elements must match the required signers from TxBody's messages. + // The first element is the primary signer and the one which pays the fee. + repeated SignerInfo signer_infos = 1; + // The fee can be calculated based on the cost of evaluating the body and doing signature verification of the signers. This can be estimated via simulation. + Fee fee = 2; +} + +message SignerInfo { + // The public key is optional for accounts that already exist in state. If unset, the + // verifier can use the required signer address for this position and lookup the public key. + google.protobuf.Any public_key = 1; + // ModeInfo describes the signing mode of the signer and is a nested + // structure to support nested multisig pubkey's + ModeInfo mode_info = 2; + // sequence is the sequence of the account, which describes the + // number of committed transactions signed by a given address. It is used to prevent + // replay attacks. + uint64 sequence = 3; +} + +message ModeInfo { + oneof sum { + Single single = 1; + Multi multi = 2; + } + + // Single is the mode info for a single signer. It is structured as a message + // to allow for additional fields such as locale for SIGN_MODE_TEXTUAL in the future + message Single { + SignMode mode = 1; + } + + // Multi is the mode info for a multisig public key + message Multi { + // bitarray specifies which keys within the multisig are signing + CompactBitArray bitarray = 1; + // mode_infos is the corresponding modes of the signers of the multisig + // which could include nested multisig public keys + repeated ModeInfo mode_infos = 2; + } +} + +enum SignMode { + SIGN_MODE_UNSPECIFIED = 0; + + SIGN_MODE_DIRECT = 1; + + SIGN_MODE_TEXTUAL = 2; + + SIGN_MODE_LEGACY_AMINO_JSON = 127; +} +``` + +As will be discussed below, in order to include as much of the `Tx` as possible +in the `SignDoc`, `SignerInfo` is separated from signatures so that only the +raw signatures themselves live outside of what is signed over. + +Because we are aiming for a flexible, extensible cross-chain transaction +format, new transaction processing options should be added to `TxBody` as soon +those use cases are discovered, even if they can't be implemented yet. + +Because there is coordination overhead in this, `TxBody` includes an +`extension_options` field which can be used for any transaction processing +options that are not already covered. App developers should, nevertheless, +attempt to upstream important improvements to `Tx`. + +### Signing + +All of the signing modes below aim to provide the following guarantees: + +* **No Malleability**: `TxBody` and `AuthInfo` cannot change once the transaction + is signed +* **Predictable Gas**: if I am signing a transaction where I am paying a fee, + the final gas is fully dependent on what I am signing + +These guarantees give the maximum amount confidence to message signers that +manipulation of `Tx`s by intermediaries can't result in any meaningful changes. + +#### `SIGN_MODE_DIRECT` + +The "direct" signing behavior is to sign the raw `TxBody` bytes as broadcast over +the wire. This has the advantages of: + +* requiring the minimum additional client capabilities beyond a standard protocol + buffers implementation +* leaving effectively zero holes for transaction malleability (i.e. there are no + subtle differences between the signing and encoding formats which could + potentially be exploited by an attacker) + +Signatures are structured using the `SignDoc` below which reuses the serialization of +`TxBody` and `AuthInfo` and only adds the fields which are needed for signatures: + +```protobuf +// types/types.proto +message SignDoc { + // A protobuf serialization of a TxBody that matches the representation in TxRaw. + bytes body = 1; + // A protobuf serialization of an AuthInfo that matches the representation in TxRaw. + bytes auth_info = 2; + string chain_id = 3; + uint64 account_number = 4; +} +``` + +In order to sign in the default mode, clients take the following steps: + +1. Serialize `TxBody` and `AuthInfo` using any valid protobuf implementation. +2. Create a `SignDoc` and serialize it using [ADR 027](/sdk/v0.50/build/architecture/adr-027-deterministic-protobuf-serialization). +3. Sign the encoded `SignDoc` bytes. +4. Build a `TxRaw` and serialize it for broadcasting. + +Signature verification is based on comparing the raw `TxBody` and `AuthInfo` +bytes encoded in `TxRaw` not based on any ["canonicalization"](https://github.com/regen-network/canonical-proto3) +algorithm which creates added complexity for clients in addition to preventing +some forms of upgradeability (to be addressed later in this document). + +Signature verifiers do: + +1. Deserialize a `TxRaw` and pull out `body` and `auth_info`. +2. Create a list of required signer addresses from the messages. +3. For each required signer: + * Pull account number and sequence from the state. + * Obtain the public key either from state or `AuthInfo`'s `signer_infos`. + * Create a `SignDoc` and serialize it using [ADR 027](/sdk/v0.50/build/architecture/adr-027-deterministic-protobuf-serialization). + * Verify the signature at the same list position against the serialized `SignDoc`. + +#### `SIGN_MODE_LEGACY_AMINO` + +In order to support legacy wallets and exchanges, Amino JSON will be temporarily +supported transaction signing. Once wallets and exchanges have had a +chance to upgrade to protobuf based signing, this option will be disabled. In +the meantime, it is foreseen that disabling the current Amino signing would cause +too much breakage to be feasible. Note that this is mainly a requirement of the +Cosmos Hub and other chains may choose to disable Amino signing immediately. + +Legacy clients will be able to sign a transaction using the current Amino +JSON format and have it encoded to protobuf using the REST `/tx/encode` +endpoint before broadcasting. + +#### `SIGN_MODE_TEXTUAL` + +As was discussed extensively in [#6078](https://github.com/cosmos/cosmos-sdk/issues/6078), +there is a desire for a human-readable signing encoding, especially for hardware +wallets like the [Ledger](https://www.ledger.com) which display +transaction contents to users before signing. JSON was an attempt at this but +falls short of the ideal. + +`SIGN_MODE_TEXTUAL` is intended as a placeholder for a human-readable +encoding which will replace Amino JSON. This new encoding should be even more +focused on readability than JSON, possibly based on formatting strings like +[MessageFormat](http://userguide.icu-project.org/formatparse/messages). + +In order to ensure that the new human-readable format does not suffer from +transaction malleability issues, `SIGN_MODE_TEXTUAL` +requires that the *human-readable bytes are concatenated with the raw `SignDoc`* +to generate sign bytes. + +Multiple human-readable formats (maybe even localized messages) may be supported +by `SIGN_MODE_TEXTUAL` when it is implemented. + +### Unknown Field Filtering + +Unknown fields in protobuf messages should generally be rejected by transaction +processors because: + +* important data may be present in the unknown fields, that if ignored, will + cause unexpected behavior for clients +* they present a malleability vulnerability where attackers can bloat tx size + by adding random uninterpreted data to unsigned content (i.e. the master `Tx`, + not `TxBody`) + +There are also scenarios where we may choose to safely ignore unknown fields +([Link](https://github.com/cosmos/cosmos-sdk/issues/6078#issuecomment-624400188)) to +provide graceful forwards compatibility with newer clients. + +We propose that field numbers with bit 11 set (for most use cases this is +the range of 1024-2047) be considered non-critical fields that can safely be +ignored if unknown. + +To handle this we will need an unknown field filter that: + +* always rejects unknown fields in unsigned content (i.e. top-level `Tx` and + unsigned parts of `AuthInfo` if present based on the signing mode) +* rejects unknown fields in all messages (including nested `Any`s) other than + fields with bit 11 set + +This will likely need to be a custom protobuf parser pass that takes message bytes +and `FileDescriptor`s and returns a boolean result. + +### Public Key Encoding + +Public keys in the Cosmos SDK implement the `cryptotypes.PubKey` interface. +We propose to use `Any` for protobuf encoding as we are doing with other interfaces (for example, in `BaseAccount.PubKey` and `SignerInfo.PublicKey`). +The following public keys are implemented: secp256k1, secp256r1, ed25519 and legacy-multisignature. + +Ex: + +```protobuf +message PubKey { + bytes key = 1; +} +``` + +`multisig.LegacyAminoPubKey` has an array of `Any`'s member to support any +protobuf public key type. + +Apps should only attempt to handle a registered set of public keys that they +have tested. The provided signature verification ante handler decorators will +enforce this. + +### CLI & REST + +Currently, the REST and CLI handlers encode and decode types and txs via Amino +JSON encoding using a concrete Amino codec. Being that some of the types dealt with +in the client can be interfaces, similar to how we described in [ADR 019](/sdk/v0.50/build/architecture/adr-019-protobuf-state-encoding), +the client logic will now need to take a codec interface that knows not only how +to handle all the types, but also knows how to generate transactions, signatures, +and messages. + +```go expandable +type AccountRetriever interface { + GetAccount(clientCtx Context, addr sdk.AccAddress) (client.Account, error) + +GetAccountWithHeight(clientCtx Context, addr sdk.AccAddress) (client.Account, int64, error) + +EnsureExists(clientCtx client.Context, addr sdk.AccAddress) + +error + GetAccountNumberSequence(clientCtx client.Context, addr sdk.AccAddress) (uint64, uint64, error) +} + +type Generator interface { + NewTx() + +TxBuilder + NewFee() + +ClientFee + NewSignature() + +ClientSignature + MarshalTx(tx types.Tx) ([]byte, error) +} + +type TxBuilder interface { + GetTx() + +sdk.Tx + + SetMsgs(...sdk.Msg) + +error + GetSignatures() []sdk.Signature + SetSignatures(...sdk.Signature) + +GetFee() + +sdk.Fee + SetFee(sdk.Fee) + +GetMemo() + +string + SetMemo(string) +} +``` + +We then update `Context` to have new fields: `Codec`, `TxGenerator`, +and `AccountRetriever`, and we update `AppModuleBasic.GetTxCmd` to take +a `Context` which should have all of these fields pre-populated. + +Each client method should then use one of the `Init` methods to re-initialize +the pre-populated `Context`. `tx.GenerateOrBroadcastTx` can be used to +generate or broadcast a transaction. For example: + +```go expandable +import "github.com/spf13/cobra" +import "github.com/cosmos/cosmos-sdk/client" +import "github.com/cosmos/cosmos-sdk/client/tx" + +func NewCmdDoSomething(clientCtx client.Context) *cobra.Command { + return &cobra.Command{ + RunE: func(cmd *cobra.Command, args []string) + +error { + clientCtx := ctx.InitWithInput(cmd.InOrStdin()) + msg := NewSomeMsg{... +} + +tx.GenerateOrBroadcastTx(clientCtx, msg) +}, +} +} +``` + +## Future Improvements + +### `SIGN_MODE_TEXTUAL` specification + +A concrete specification and implementation of `SIGN_MODE_TEXTUAL` is intended +as a near-term future improvement so that the ledger app and other wallets +can gracefully transition away from Amino JSON. + +### `SIGN_MODE_DIRECT_AUX` + +(\*Documented as option (3) in [Link](https://github.com/cosmos/cosmos-sdk/issues/6078#issuecomment-628026933)) + +We could add a mode `SIGN_MODE_DIRECT_AUX` +to support scenarios where multiple signatures +are being gathered into a single transaction but the message composer does not +yet know which signatures will be included in the final transaction. For instance, +I may have a 3/5 multisig wallet and want to send a `TxBody` to all 5 +signers to see who signs first. As soon as I have 3 signatures then I will go +ahead and build the full transaction. + +With `SIGN_MODE_DIRECT`, each signer needs +to sign the full `AuthInfo` which includes the full list of all signers and +their signing modes, making the above scenario very hard. + +`SIGN_MODE_DIRECT_AUX` would allow "auxiliary" signers to create their signature +using only `TxBody` and their own `PublicKey`. This allows the full list of +signers in `AuthInfo` to be delayed until signatures have been collected. + +An "auxiliary" signer is any signer besides the primary signer who is paying +the fee. For the primary signer, the full `AuthInfo` is actually needed to calculate gas and fees +because that is dependent on how many signers and which key types and signing +modes they are using. Auxiliary signers, however, do not need to worry about +fees or gas and thus can just sign `TxBody`. + +To generate a signature in `SIGN_MODE_DIRECT_AUX` these steps would be followed: + +1. Encode `SignDocAux` (with the same requirement that fields must be serialized + in order): + + ```protobuf expandable + // types/types.proto + message SignDocAux { + bytes body_bytes = 1; + // PublicKey is included in SignDocAux : + // 1. as a special case for multisig public keys. For multisig public keys, + // the signer should use the top-level multisig public key they are signing + // against, not their own public key. This is to prevent against a form + // of malleability where a signature could be taken out of context of the + // multisig key that was intended to be signed for + // 2. to guard against scenario where configuration information is encoded + // in public keys (it has been proposed) such that two keys can generate + // the same signature but have different security properties + // + // By including it here, the composer of AuthInfo cannot reference the + // a public key variant the signer did not intend to use + PublicKey public_key = 2; + string chain_id = 3; + uint64 account_number = 4; + } + ``` + +2. Sign the encoded `SignDocAux` bytes + +3. Send their signature and `SignerInfo` to primary signer who will then + sign and broadcast the final transaction (with `SIGN_MODE_DIRECT` and `AuthInfo` + added) once enough signatures have been collected + +### `SIGN_MODE_DIRECT_RELAXED` + +(*Documented as option (1)(a) in [Link](https://github.com/cosmos/cosmos-sdk/issues/6078#issuecomment-628026933)*) + +This is a variation of `SIGN_MODE_DIRECT` where multiple signers wouldn't need to +coordinate public keys and signing modes in advance. It would involve an alternate +`SignDoc` similar to `SignDocAux` above with fee. This could be added in the future +if client developers found the burden of collecting public keys and modes in advance +too burdensome. + +## Consequences + +### Positive + +* Significant performance gains. +* Supports backward and forward type compatibility. +* Better support for cross-language clients. +* Multiple signing modes allow for greater protocol evolution + +### Negative + +* `google.protobuf.Any` type URLs increase transaction size although the effect + may be negligible or compression may be able to mitigate it. + +### Neutral + +## References diff --git a/sdk/v0.54/reference/architecture/adr-021-protobuf-query-encoding.mdx b/sdk/v0.54/reference/architecture/adr-021-protobuf-query-encoding.mdx new file mode 100644 index 000000000..6357a99ae --- /dev/null +++ b/sdk/v0.54/reference/architecture/adr-021-protobuf-query-encoding.mdx @@ -0,0 +1,276 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/architecture/adr-021-protobuf-query-encoding' +title: 'ADR 021: Protocol Buffer Query Encoding' +description: '2020 March 27: Initial Draft' +--- + +## Changelog + +* 2020 March 27: Initial Draft + +## Status + +Accepted + +## Context + +This ADR is a continuation of the motivation, design, and context established in +[ADR 019](/sdk/v0.50/build/architecture/adr-019-protobuf-state-encoding) and +[ADR 020](/sdk/v0.50/build/architecture/adr-020-protobuf-transaction-encoding), namely, we aim to design the +Protocol Buffer migration path for the client-side of the Cosmos SDK. + +This ADR continues from [ADD 020](/sdk/v0.50/build/architecture/adr-020-protobuf-transaction-encoding) +to specify the encoding of queries. + +## Decision + +### Custom Query Definition + +Modules define custom queries through a protocol buffers `service` definition. +These `service` definitions are generally associated with and used by the +GRPC protocol. However, the protocol buffers specification indicates that +they can be used more generically by any request/response protocol that uses +protocol buffer encoding. Thus, we can use `service` definitions for specifying +custom ABCI queries and even reuse a substantial amount of the GRPC infrastructure. + +Each module with custom queries should define a service canonically named `Query`: + +```protobuf +// x/bank/types/types.proto + +service Query { + rpc QueryBalance(QueryBalanceParams) returns (cosmos_sdk.v1.Coin) { } + rpc QueryAllBalances(QueryAllBalancesParams) returns (QueryAllBalancesResponse) { } +} +``` + +#### Handling of Interface Types + +Modules that use interface types and need true polymorphism generally force a +`oneof` up to the app-level that provides the set of concrete implementations of +that interface that the app supports. While app's are welcome to do the same for +queries and implement an app-level query service, it is recommended that modules +provide query methods that expose these interfaces via `google.protobuf.Any`. +There is a concern on the transaction level that the overhead of `Any` is too +high to justify its usage. However for queries this is not a concern, and +providing generic module-level queries that use `Any` does not preclude apps +from also providing app-level queries that return use the app-level `oneof`s. + +A hypothetical example for the `gov` module would look something like: + +```protobuf expandable +// x/gov/types/types.proto + +import "google/protobuf/any.proto"; + +service Query { + rpc GetProposal(GetProposalParams) returns (AnyProposal) { } +} + +message AnyProposal { + ProposalBase base = 1; + google.protobuf.Any content = 2; +} +``` + +### Custom Query Implementation + +In order to implement the query service, we can reuse the existing [gogo protobuf](https://github.com/cosmos/gogoproto) +grpc plugin, which for a service named `Query` generates an interface named +`QueryServer` as below: + +```go +type QueryServer interface { + QueryBalance(context.Context, *QueryBalanceParams) (*types.Coin, error) + +QueryAllBalances(context.Context, *QueryAllBalancesParams) (*QueryAllBalancesResponse, error) +} +``` + +The custom queries for our module are implemented by implementing this interface. + +The first parameter in this generated interface is a generic `context.Context`, +whereas querier methods generally need an instance of `sdk.Context` to read +from the store. Since arbitrary values can be attached to `context.Context` +using the `WithValue` and `Value` methods, the Cosmos SDK should provide a function +`sdk.UnwrapSDKContext` to retrieve the `sdk.Context` from the provided +`context.Context`. + +An example implementation of `QueryBalance` for the bank module as above would +look something like: + +```go +type Querier struct { + Keeper +} + +func (q Querier) + +QueryBalance(ctx context.Context, params *types.QueryBalanceParams) (*sdk.Coin, error) { + balance := q.GetBalance(sdk.UnwrapSDKContext(ctx), params.Address, params.Denom) + +return &balance, nil +} +``` + +### Custom Query Registration and Routing + +Query server implementations as above would be registered with `AppModule`s using +a new method `RegisterQueryService(grpc.Server)` which could be implemented simply +as below: + +```go +// x/bank/module.go +func (am AppModule) + +RegisterQueryService(server grpc.Server) { + types.RegisterQueryServer(server, keeper.Querier{ + am.keeper +}) +} +``` + +Underneath the hood, a new method `RegisterService(sd *grpc.ServiceDesc, handler interface{})` +will be added to the existing `baseapp.QueryRouter` to add the queries to the custom +query routing table (with the routing method being described below). +The signature for this method matches the existing +`RegisterServer` method on the GRPC `Server` type where `handler` is the custom +query server implementation described above. + +GRPC-like requests are routed by the service name (ex. `cosmos_sdk.x.bank.v1.Query`) +and method name (ex. `QueryBalance`) combined with `/`s to form a full +method name (ex. `/cosmos_sdk.x.bank.v1.Query/QueryBalance`). This gets translated +into an ABCI query as `custom/cosmos_sdk.x.bank.v1.Query/QueryBalance`. Service handlers +registered with `QueryRouter.RegisterService` will be routed this way. + +Beyond the method name, GRPC requests carry a protobuf encoded payload, which maps naturally +to `RequestQuery.Data`, and receive a protobuf encoded response or error. Thus +there is a quite natural mapping of GRPC-like rpc methods to the existing +`sdk.Query` and `QueryRouter` infrastructure. + +This basic specification allows us to reuse protocol buffer `service` definitions +for ABCI custom queries substantially reducing the need for manual decoding and +encoding in query methods. + +### GRPC Protocol Support + +In addition to providing an ABCI query pathway, we can easily provide a GRPC +proxy server that routes requests in the GRPC protocol to ABCI query requests +under the hood. In this way, clients could use their host languages' existing +GRPC implementations to make direct queries against Cosmos SDK app's using +these `service` definitions. In order for this server to work, the `QueryRouter` +on `BaseApp` will need to expose the service handlers registered with +`QueryRouter.RegisterService` to the proxy server implementation. Nodes could +launch the proxy server on a separate port in the same process as the ABCI app +with a command-line flag. + +### REST Queries and Swagger Generation + +[grpc-gateway](https://github.com/grpc-ecosystem/grpc-gateway) is a project that +translates REST calls into GRPC calls using special annotations on service +methods. Modules that want to expose REST queries should add `google.api.http` +annotations to their `rpc` methods as in this example below. + +```protobuf expandable +// x/bank/types/types.proto + +service Query { + rpc QueryBalance(QueryBalanceParams) returns (cosmos_sdk.v1.Coin) { + option (google.api.http) = { + get: "/x/bank/v1/balance/{address}/{denom}" + }; + } + rpc QueryAllBalances(QueryAllBalancesParams) returns (QueryAllBalancesResponse) { + option (google.api.http) = { + get: "/x/bank/v1/balances/{address}" + }; + } +} +``` + +grpc-gateway will work direcly against the GRPC proxy described above which will +translate requests to ABCI queries under the hood. grpc-gateway can also +generate Swagger definitions automatically. + +In the current implementation of REST queries, each module needs to implement +REST queries manually in addition to ABCI querier methods. Using the grpc-gateway +approach, there will be no need to generate separate REST query handlers, just +query servers as described above as grpc-gateway handles the translation of protobuf +to REST as well as Swagger definitions. + +The Cosmos SDK should provide CLI commands for apps to start GRPC gateway either in +a separate process or the same process as the ABCI app, as well as provide a +command for generating grpc-gateway proxy `.proto` files and the `swagger.json` +file. + +### Client Usage + +The gogo protobuf grpc plugin generates client interfaces in addition to server +interfaces. For the `Query` service defined above we would get a `QueryClient` +interface like: + +```go +type QueryClient interface { + QueryBalance(ctx context.Context, in *QueryBalanceParams, opts ...grpc.CallOption) (*types.Coin, error) + +QueryAllBalances(ctx context.Context, in *QueryAllBalancesParams, opts ...grpc.CallOption) (*QueryAllBalancesResponse, error) +} +``` + +Via a small patch to gogo protobuf ([gogo/protobuf#675](https://github.com/gogo/protobuf/pull/675)) +we have tweaked the grpc codegen to use an interface rather than concrete type +for the generated client struct. This allows us to also reuse the GRPC infrastructure +for ABCI client queries. + +1Context`will receive a new method`QueryConn`that returns a`ClientConn\` +that routes calls to ABCI queries + +Clients (such as CLI methods) will then be able to call query methods like this: + +```go +clientCtx := client.NewContext() + queryClient := types.NewQueryClient(clientCtx.QueryConn()) + params := &types.QueryBalanceParams{ + addr, denom +} + +result, err := queryClient.QueryBalance(gocontext.Background(), params) +``` + +### Testing + +Tests would be able to create a query client directly from keeper and `sdk.Context` +references using a `QueryServerTestHelper` as below: + +```go +queryHelper := baseapp.NewQueryServerTestHelper(ctx) + +types.RegisterQueryServer(queryHelper, keeper.Querier{ + app.BankKeeper +}) + queryClient := types.NewQueryClient(queryHelper) +``` + +## Future Improvements + +## Consequences + +### Positive + +* greatly simplified querier implementation (no manual encoding/decoding) +* easy query client generation (can use existing grpc and swagger tools) +* no need for REST query implementations +* type safe query methods (generated via grpc plugin) +* going forward, there will be less breakage of query methods because of the + backwards compatibility guarantees provided by buf + +### Negative + +* all clients using the existing ABCI/REST queries will need to be refactored + for both the new GRPC/REST query paths as well as protobuf/proto-json encoded + data, but this is more or less unavoidable in the protobuf refactoring + +### Neutral + +## References diff --git a/sdk/v0.54/reference/architecture/adr-022-custom-panic-handling.mdx b/sdk/v0.54/reference/architecture/adr-022-custom-panic-handling.mdx new file mode 100644 index 000000000..953cfbaf7 --- /dev/null +++ b/sdk/v0.54/reference/architecture/adr-022-custom-panic-handling.mdx @@ -0,0 +1,267 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/architecture/adr-022-custom-panic-handling' +title: "ADR 022: Custom BaseApp panic handling" +description: "2020 Apr 24: Initial Draft 2021 Sep 14: Superseded by ADR-045" +--- + +## Changelog + +- 2020 Apr 24: Initial Draft +- 2021 Sep 14: Superseded by ADR-045 + +## Status + +SUPERSEDED by ADR-045 + +## Context + +The current implementation of BaseApp does not allow developers to write custom error handlers during panic recovery +[runTx()](https://github.com/cosmos/cosmos-sdk/blob/bad4ca75f58b182f600396ca350ad844c18fc80b/baseapp/baseapp.go#L539) +method. We think that this method can be more flexible and can give Cosmos SDK users more options for customizations without +the need to rewrite whole BaseApp. Also there's one special case for `sdk.ErrorOutOfGas` error handling, that case +might be handled in a "standard" way (middleware) alongside the others. + +We propose middleware-solution, which could help developers implement the following cases: + +- add external logging (let's say sending reports to external services like [Sentry](https://sentry.io)); +- call panic for specific error cases; + +It will also make `OutOfGas` case and `default` case one of the middlewares. +`Default` case wraps recovery object to an error and logs it ([example middleware implementation](#Recovery-middleware)). + +Our project has a sidecar service running alongside the blockchain node (smart contracts virtual machine). It is +essential that node `<->` sidecar connectivity stays stable for TXs processing. So when the communication breaks we need +to crash the node and reboot it once the problem is solved. That behavior makes node's state machine execution +deterministic. As all keeper panics are caught by runTx's `defer()` handler, we have to adjust the BaseApp code +in order to customize it. + +## Decision + +### Design + +#### Overview + +Instead of hardcoding custom error handling into BaseApp we suggest using set of middlewares which can be customized +externally and will allow developers use as many custom error handlers as they want. Implementation with tests +can be found [here](https://github.com/cosmos/cosmos-sdk/pull/6053). + +#### Implementation details + +##### Recovery handler + +New `RecoveryHandler` type added. `recoveryObj` input argument is an object returned by the standard Go function +`recover()` from the `builtin` package. + +```go +type RecoveryHandler func(recoveryObj interface{ +}) + +error +``` + +Handler should type assert (or other methods) an object to define if object should be handled. +`nil` should be returned if input object can't be handled by that `RecoveryHandler` (not a handler's target type). +Not `nil` error should be returned if input object was handled and middleware chain execution should be stopped. + +An example: + +```go +func exampleErrHandler(recoveryObj interface{ +}) + +error { + err, ok := recoveryObj.(error) + if !ok { + return nil +} + if someSpecificError.Is(err) { + panic(customPanicMsg) +} + +else { + return nil +} +} +``` + +This example breaks the application execution, but it also might enrich the error's context like the `OutOfGas` handler. + +##### Recovery middleware + +We also add a middleware type (decorator). That function type wraps `RecoveryHandler` and returns the next middleware in +execution chain and handler's `error`. Type is used to separate actual `recovery()` object handling from middleware +chain processing. + +```go +type recoveryMiddleware func(recoveryObj interface{ +}) (recoveryMiddleware, error) + +func newRecoveryMiddleware(handler RecoveryHandler, next recoveryMiddleware) + +recoveryMiddleware { + return func(recoveryObj interface{ +}) (recoveryMiddleware, error) { + if err := handler(recoveryObj); err != nil { + return nil, err +} + +return next, nil +} +} +``` + +Function receives a `recoveryObj` object and returns: + +- (next `recoveryMiddleware`, `nil`) if object wasn't handled (not a target type) by `RecoveryHandler`; +- (`nil`, not nil `error`) if input object was handled and other middlewares in the chain should not be executed; +- (`nil`, `nil`) in case of invalid behavior. Panic recovery might not have been properly handled; + this can be avoided by always using a `default` as a rightmost middleware in the chain (always returns an `error`'); + +`OutOfGas` middleware example: + +```go expandable +func newOutOfGasRecoveryMiddleware(gasWanted uint64, ctx sdk.Context, next recoveryMiddleware) + +recoveryMiddleware { + handler := func(recoveryObj interface{ +}) + +error { + err, ok := recoveryObj.(sdk.ErrorOutOfGas) + if !ok { + return nil +} + +return errorsmod.Wrap( + sdkerrors.ErrOutOfGas, fmt.Sprintf( + "out of gas in location: %v; gasWanted: %d, gasUsed: %d", err.Descriptor, gasWanted, ctx.GasMeter().GasConsumed(), + ), + ) +} + +return newRecoveryMiddleware(handler, next) +} +``` + +`Default` middleware example: + +```go +func newDefaultRecoveryMiddleware() + +recoveryMiddleware { + handler := func(recoveryObj interface{ +}) + +error { + return errorsmod.Wrap( + sdkerrors.ErrPanic, fmt.Sprintf("recovered: %v\nstack:\n%v", recoveryObj, string(debug.Stack())), + ) +} + +return newRecoveryMiddleware(handler, nil) +} +``` + +##### Recovery processing + +Basic chain of middlewares processing would look like: + +```go +func processRecovery(recoveryObj interface{ +}, middleware recoveryMiddleware) + +error { + if middleware == nil { + return nil +} + +next, err := middleware(recoveryObj) + if err != nil { + return err +} + if next == nil { + return nil +} + +return processRecovery(recoveryObj, next) +} +``` + +That way we can create a middleware chain which is executed from left to right, the rightmost middleware is a +`default` handler which must return an `error`. + +##### BaseApp changes + +The `default` middleware chain must exist in a `BaseApp` object. `Baseapp` modifications: + +```go expandable +type BaseApp struct { + // ... + runTxRecoveryMiddleware recoveryMiddleware +} + +func NewBaseApp(...) { + // ... + app.runTxRecoveryMiddleware = newDefaultRecoveryMiddleware() +} + +func (app *BaseApp) + +runTx(...) { + // ... + defer func() { + if r := recover(); r != nil { + recoveryMW := newOutOfGasRecoveryMiddleware(gasWanted, ctx, app.runTxRecoveryMiddleware) + +err, result = processRecovery(r, recoveryMW), nil +} + +gInfo = sdk.GasInfo{ + GasWanted: gasWanted, + GasUsed: ctx.GasMeter().GasConsumed() +} + +}() + // ... +} +``` + +Developers can add their custom `RecoveryHandler`s by providing `AddRunTxRecoveryHandler` as a BaseApp option parameter to the `NewBaseapp` constructor: + +```go +func (app *BaseApp) + +AddRunTxRecoveryHandler(handlers ...RecoveryHandler) { + for _, h := range handlers { + app.runTxRecoveryMiddleware = newRecoveryMiddleware(h, app.runTxRecoveryMiddleware) +} +} +``` + +This method would prepend handlers to an existing chain. + +## Consequences + +### Positive + +- Developers of Cosmos SDK based projects can add custom panic handlers to: + - add error context for custom panic sources (panic inside of custom keepers); + - emit `panic()`: passthrough recovery object to the Tendermint core; + - other necessary handling; +- Developers can use standard Cosmos SDK `BaseApp` implementation, rather that rewriting it in their projects; +- Proposed solution doesn't break the current "standard" `runTx()` flow; + +### Negative + +- Introduces changes to the execution model design. + +### Neutral + +- `OutOfGas` error handler becomes one of the middlewares; +- Default panic handler becomes one of the middlewares; + +## References + +- [PR-6053 with proposed solution](https://github.com/cosmos/cosmos-sdk/pull/6053) +- [Similar solution. ADR-010 Modular AnteHandler](https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-010-modular-antehandler.md) diff --git a/sdk/v0.54/reference/architecture/adr-023-protobuf-naming.mdx b/sdk/v0.54/reference/architecture/adr-023-protobuf-naming.mdx new file mode 100644 index 000000000..5c45b3cc5 --- /dev/null +++ b/sdk/v0.54/reference/architecture/adr-023-protobuf-naming.mdx @@ -0,0 +1,268 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/architecture/adr-023-protobuf-naming' +title: 'ADR 023: Protocol Buffer Naming and Versioning Conventions' +description: '2020 April 27: Initial Draft 2020 August 5: Update guidelines' +--- + +## Changelog + +* 2020 April 27: Initial Draft +* 2020 August 5: Update guidelines + +## Status + +Accepted + +## Context + +Protocol Buffers provide a basic [style guide](https://developers.google.com/protocol-buffers/docs/style) +and [Buf](https://buf.build/docs/style-guide) builds upon that. To the +extent possible, we want to follow industry accepted guidelines and wisdom for +the effective usage of protobuf, deviating from those only when there is clear +rationale for our use case. + +### Adoption of `Any` + +The adoption of `google.protobuf.Any` as the recommended approach for encoding +interface types (as opposed to `oneof`) makes package naming a central part +of the encoding as fully-qualified message names now appear in encoded +messages. + +### Current Directory Organization + +Thus far we have mostly followed [Buf's](https://buf.build) [DEFAULT](https://buf.build/docs/lint-checkers#default) +recommendations, with the minor deviation of disabling [`PACKAGE_DIRECTORY_MATCH`](https://buf.build/docs/lint-checkers#file_layout) +which although being convenient for developing code comes with the warning +from Buf that: + +> you will have a very bad time with many Protobuf plugins across various languages if you do not do this + +### Adoption of gRPC Queries + +In [ADR 021](/sdk/v0.54/reference/architecture/adr-021-protobuf-query-encoding), gRPC was adopted for Protobuf +native queries. The full gRPC service path thus becomes a key part of ABCI query +path. In the future, gRPC queries may be allowed from within persistent scripts +by technologies such as CosmWasm and these query routes would be stored within +script binaries. + +## Decision + +The goal of this ADR is to provide thoughtful naming conventions that: + +* encourage a good user experience for when users interact directly with + .proto files and fully-qualified protobuf names +* balance conciseness against the possibility of either over-optimizing (making + names too short and cryptic) or under-optimizing (just accepting bloated names + with lots of redundant information) + +These guidelines are meant to act as a style guide for both the Cosmos SDK and +third-party modules. + +As a starting point, we should adopt all of the [DEFAULT](https://buf.build/docs/lint-checkers#default) +checkers in [Buf's](https://buf.build) including [`PACKAGE_DIRECTORY_MATCH`](https://buf.build/docs/lint-checkers#file_layout), +except: + +* [PACKAGE\_VERSION\_SUFFIX](https://buf.build/docs/lint-checkers#package_version_suffix) +* [SERVICE\_SUFFIX](https://buf.build/docs/lint-checkers#service_suffix) + +Further guidelines to be described below. + +### Principles + +#### Concise and Descriptive Names + +Names should be descriptive enough to convey their meaning and distinguish +them from other names. + +Given that we are using fully-qualifed names within +`google.protobuf.Any` as well as within gRPC query routes, we should aim to +keep names concise, without going overboard. The general rule of thumb should +be if a shorter name would convey more or else the same thing, pick the shorter +name. + +For instance, `cosmos.bank.MsgSend` (19 bytes) conveys roughly the same information +as `cosmos_sdk.x.bank.v1.MsgSend` (28 bytes) but is more concise. + +Such conciseness makes names both more pleasant to work with and take up less +space within transactions and on the wire. + +We should also resist the temptation to over-optimize, by making names +cryptically short with abbreviations. For instance, we shouldn't try to +reduce `cosmos.bank.MsgSend` to `csm.bk.MSnd` just to save a few bytes. + +The goal is to make names ***concise but not cryptic***. + +#### Names are for Clients First + +Package and type names should be chosen for the benefit of users, not +necessarily because of legacy concerns related to the go code-base. + +#### Plan for Longevity + +In the interests of long-term support, we should plan on the names we do +choose to be in usage for a long time, so now is the opportunity to make +the best choices for the future. + +### Versioning + +#### Guidelines on Stable Package Versions + +In general, schema evolution is the way to update protobuf schemas. That means that new fields, +messages, and RPC methods are *added* to existing schemas and old fields, messages and RPC methods +are maintained as long as possible. + +Breaking things is often unacceptable in a blockchain scenario. For instance, immutable smart contracts +may depend on certain data schemas on the host chain. If the host chain breaks those schemas, the smart +contract may be irreparably broken. Even when things can be fixed (for instance in client software), +this often comes at a high cost. + +Instead of breaking things, we should make every effort to evolve schemas rather than just breaking them. +[Buf](https://buf.build) breaking change detection should be used on all stable (non-alpha or beta) packages +to prevent such breakage. + +With that in mind, different stable versions (i.e. `v1` or `v2`) of a package should more or less be considered +different packages and this should be last resort approach for upgrading protobuf schemas. Scenarios where creating +a `v2` may make sense are: + +* we want to create a new module with similar functionality to an existing module and adding `v2` is the most natural + way to do this. In that case, there are really just two different, but similar modules with different APIs. +* we want to add a new revamped API for an existing module and it's just too cumbersome to add it to the existing package, + so putting it in `v2` is cleaner for users. In this case, care should be made to not deprecate support for + `v1` if it is actively used in immutable smart contracts. + +#### Guidelines on unstable (alpha and beta) package versions + +The following guidelines are recommended for marking packages as alpha or beta: + +* marking something as `alpha` or `beta` should be a last resort and just putting something in the + stable package (i.e. `v1` or `v2`) should be preferred +* a package *should* be marked as `alpha` *if and only if* there are active discussions to remove + or significantly alter the package in the near future +* a package *should* be marked as `beta` *if and only if* there is an active discussion to + significantly refactor/rework the functionality in the near future but not remove it +* modules *can and should* have types in both stable (i.e. `v1` or `v2`) and unstable (`alpha` or `beta`) packages. + +*`alpha` and `beta` should not be used to avoid responsibility for maintaining compatibility.* +Whenever code is released into the wild, especially on a blockchain, there is a high cost to changing things. In some +cases, for instance with immutable smart contracts, a breaking change may be impossible to fix. + +When marking something as `alpha` or `beta`, maintainers should ask the questions: + +* what is the cost of asking others to change their code vs the benefit of us maintaining the optionality to change it? +* what is the plan for moving this to `v1` and how will that affect users? + +`alpha` or `beta` should really be used to communicate "changes are planned". + +As a case study, gRPC reflection is in the package `grpc.reflection.v1alpha`. It hasn't been changed since +2017 and it is now used in other widely used software like gRPCurl. Some folks probably use it in production services +and so if they actually went and changed the package to `grpc.reflection.v1`, some software would break and +they probably don't want to do that... So now the `v1alpha` package is more or less the de-facto `v1`. Let's not do that. + +The following are guidelines for working with non-stable packages: + +* [Buf's recommended version suffix](https://buf.build/docs/lint-checkers#package_version_suffix) + (ex. `v1alpha1`) *should* be used for non-stable packages +* non-stable packages should generally be excluded from breaking change detection +* immutable smart contract modules (i.e. CosmWasm) *should* block smart contracts/persistent + scripts from interacting with `alpha`/`beta` packages + +#### Omit v1 suffix + +Instead of using [Buf's recommended version suffix](https://buf.build/docs/lint-checkers#package_version_suffix), +we can omit `v1` for packages that don't actually have a second version. This +allows for more concise names for common use cases like `cosmos.bank.Send`. +Packages that do have a second or third version can indicate that with `.v2` +or `.v3`. + +### Package Naming + +#### Adopt a short, unique top-level package name + +Top-level packages should adopt a short name that is known to not collide with +other names in common usage within the Cosmos ecosystem. In the near future, a +registry should be created to reserve and index top-level package names used +within the Cosmos ecosystem. Because the Cosmos SDK is intended to provide +the top-level types for the Cosmos project, the top-level package name `cosmos` +is recommended for usage within the Cosmos SDK instead of the longer `cosmos_sdk`. +[ICS](https://github.com/cosmos/ics) specifications could consider a +short top-level package like `ics23` based upon the standard number. + +#### Limit sub-package depth + +Sub-package depth should be increased with caution. Generally a single +sub-package is needed for a module or a library. Even though `x` or `modules` +is used in source code to denote modules, this is often unnecessary for .proto +files as modules are the primary thing sub-packages are used for. Only items which +are known to be used infrequently should have deep sub-package depths. + +For the Cosmos SDK, it is recommended that we simply write `cosmos.bank`, +`cosmos.gov`, etc. rather than `cosmos.x.bank`. In practice, most non-module +types can go straight in the `cosmos` package or we can introduce a +`cosmos.base` package if needed. Note that this naming *will not* change +go package names, i.e. the `cosmos.bank` protobuf package will still live in +`x/bank`. + +### Message Naming + +Message type names should be as concise possible without losing clarity. `sdk.Msg` +types which are used in transactions will retain the `Msg` prefix as that provides +helpful context. + +### Service and RPC Naming + +[ADR 021](/sdk/v0.54/reference/architecture/adr-021-protobuf-query-encoding) specifies that modules should +implement a gRPC query service. We should consider the principle of conciseness +for query service and RPC names as these may be called from persistent script +modules such as CosmWasm. Also, users may use these query paths from tools like +[gRPCurl](https://github.com/fullstorydev/grpcurl). As an example, we can shorten +`/cosmos_sdk.x.bank.v1.QueryService/QueryBalance` to +`/cosmos.bank.Query/Balance` without losing much useful information. + +RPC request and response types *should* follow the `ServiceNameMethodNameRequest`/ +`ServiceNameMethodNameResponse` naming convention. i.e. for an RPC method named `Balance` +on the `Query` service, the request and response types would be `QueryBalanceRequest` +and `QueryBalanceResponse`. This will be more self-explanatory than `BalanceRequest` +and `BalanceResponse`. + +#### Use just `Query` for the query service + +Instead of [Buf's default service suffix recommendation](https://github.com/cosmos/cosmos-sdk/pull/6033), +we should simply use the shorter `Query` for query services. + +For other types of gRPC services, we should consider sticking with Buf's +default recommendation. + +#### Omit `Get` and `Query` from query service RPC names + +`Get` and `Query` should be omitted from `Query` service names because they are +redundant in the fully-qualified name. For instance, `/cosmos.bank.Query/QueryBalance` +just says `Query` twice without any new information. + +## Future Improvements + +A registry of top-level package names should be created to coordinate naming +across the ecosystem, prevent collisions, and also help developers discover +useful schemas. A simple starting point would be a git repository with +community-based governance. + +## Consequences + +### Positive + +* names will be more concise and easier to read and type +* all transactions using `Any` will be at shorter (`_sdk.x` and `.v1` will be removed) +* `.proto` file imports will be more standard (without `"third_party/proto"` in + the path) +* code generation will be easier for clients because .proto files will be + in a single `proto/` directory which can be copied rather than scattered + throughout the Cosmos SDK + +### Negative + +### Neutral + +* `.proto` files will need to be reorganized and refactored +* some modules may need to be marked as alpha or beta + +## References diff --git a/sdk/v0.54/reference/architecture/adr-024-coin-metadata.mdx b/sdk/v0.54/reference/architecture/adr-024-coin-metadata.mdx new file mode 100644 index 000000000..86f415601 --- /dev/null +++ b/sdk/v0.54/reference/architecture/adr-024-coin-metadata.mdx @@ -0,0 +1,147 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/architecture/adr-024-coin-metadata' +title: 'ADR 024: Coin Metadata' +description: '05/19/2020: Initial draft' +--- + +## Changelog + +* 05/19/2020: Initial draft + +## Status + +Proposed + +## Context + +Assets in the Cosmos SDK are represented via a `Coins` type that consists of an `amount` and a `denom`, +where the `amount` can be any arbitrarily large or small value. In addition, the Cosmos SDK uses an +account-based model where there are two types of primary accounts -- basic accounts and module accounts. +All account types have a set of balances that are composed of `Coins`. The `x/bank` module keeps +track of all balances for all accounts and also keeps track of the total supply of balances in an +application. + +With regards to a balance `amount`, the Cosmos SDK assumes a static and fixed unit of denomination, +regardless of the denomination itself. In other words, clients and apps built atop a Cosmos-SDK-based +chain may choose to define and use arbitrary units of denomination to provide a richer UX, however, by +the time a tx or operation reaches the Cosmos SDK state machine, the `amount` is treated as a single +unit. For example, for the Cosmos Hub (Gaia), clients assume 1 ATOM = 10^6 uatom, and so all txs and +operations in the Cosmos SDK work off of units of 10^6. + +This clearly provides a poor and limited UX especially as interoperability of networks increases and +as a result the total amount of asset types increases. We propose to have `x/bank` additionally keep +track of metadata per `denom` in order to help clients, wallet providers, and explorers improve their +UX and remove the requirement for making any assumptions on the unit of denomination. + +## Decision + +The `x/bank` module will be updated to store and index metadata by `denom`, specifically the "base" or +smallest unit -- the unit the Cosmos SDK state-machine works with. + +Metadata may also include a non-zero length list of denominations. Each entry contains the name of +the denomination `denom`, the exponent to the base and a list of aliases. An entry is to be +interpreted as `1 denom = 10^exponent base_denom` (e.g. `1 ETH = 10^18 wei` and `1 uatom = 10^0 uatom`). + +There are two denominations that are of high importance for clients: the `base`, which is the smallest +possible unit and the `display`, which is the unit that is commonly referred to in human communication +and on exchanges. The values in those fields link to an entry in the list of denominations. + +The list in `denom_units` and the `display` entry may be changed via governance. + +As a result, we can define the type as follows: + +```protobuf expandable +message DenomUnit { + string denom = 1; + uint32 exponent = 2; + repeated string aliases = 3; +} + +message Metadata { + string description = 1; + repeated DenomUnit denom_units = 2; + string base = 3; + string display = 4; +} +``` + +As an example, the ATOM's metadata can be defined as follows: + +```json expandable +{ + "name": "atom", + "description": "The native staking token of the Cosmos Hub.", + "denom_units": [ + { + "denom": "uatom", + "exponent": 0, + "aliases": [ + "microatom" + ], + +}, + { + "denom": "matom", + "exponent": 3, + "aliases": [ + "milliatom" + ] + +}, + { + "denom": "atom", + "exponent": 6, + } + ], + "base": "uatom", + "display": "atom", +} +``` + +Given the above metadata, a client may infer the following things: + +* 4.3atom = 4.3 \* (10^6) = 4,300,000uatom +* The string "atom" can be used as a display name in a list of tokens. +* The balance 4300000 can be displayed as 4,300,000uatom or 4,300matom or 4.3atom. + The `display` denomination 4.3atom is a good default if the authors of the client don't make + an explicit decision to choose a different representation. + +A client should be able to query for metadata by denom both via the CLI and REST interfaces. In +addition, we will add handlers to these interfaces to convert from any unit to another given unit, +as the base framework for this already exists in the Cosmos SDK. + +Finally, we need to ensure metadata exists in the `GenesisState` of the `x/bank` module which is also +indexed by the base `denom`. + +```go +type GenesisState struct { + SendEnabled bool `json:"send_enabled" yaml:"send_enabled"` + Balances []Balance `json:"balances" yaml:"balances"` + Supply sdk.Coins `json:"supply" yaml:"supply"` + DenomMetadata []Metadata `json:"denom_metadata" yaml:"denom_metadata"` +} +``` + +## Future Work + +In order for clients to avoid having to convert assets to the base denomination -- either manually or +via an endpoint, we may consider supporting automatic conversion of a given unit input. + +## Consequences + +### Positive + +* Provides clients, wallet providers and block explorers with additional data on + asset denomination to improve UX and remove any need to make assumptions on + denomination units. + +### Negative + +* A small amount of required additional storage in the `x/bank` module. The amount + of additional storage should be minimal as the amount of total assets should not + be large. + +### Neutral + +## References diff --git a/sdk/v0.54/reference/architecture/adr-027-deterministic-protobuf-serialization.mdx b/sdk/v0.54/reference/architecture/adr-027-deterministic-protobuf-serialization.mdx new file mode 100644 index 000000000..2860d1951 --- /dev/null +++ b/sdk/v0.54/reference/architecture/adr-027-deterministic-protobuf-serialization.mdx @@ -0,0 +1,319 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/architecture/adr-027-deterministic-protobuf-serialization' +title: 'ADR 027: Deterministic Protobuf Serialization' +description: '2020-08-07: Initial Draft 2020-09-01: Further clarify rules' +--- + +## Changelog + +* 2020-08-07: Initial Draft +* 2020-09-01: Further clarify rules + +## Status + +Proposed + +## Abstract + +Fully deterministic structure serialization, which works across many languages and clients, +is needed when signing messages. We need to be sure that whenever we serialize +a data structure, no matter in which supported language, the raw bytes +will stay the same. +[Protobuf](https://developers.google.com/protocol-buffers/docs/proto3) +serialization is not bijective (i.e. there exist a practically unlimited number of +valid binary representations for a given protobuf document)1. + +This document describes a deterministic serialization scheme for +a subset of protobuf documents, that covers this use case but can be reused in +other cases as well. + +### Context + +For signature verification in Cosmos SDK, the signer and verifier need to agree on +the same serialization of a `SignDoc` as defined in +[ADR-020](/sdk/v0.50/build/architecture/adr-020-protobuf-transaction-encoding) without transmitting the +serialization. + +Currently, for block signatures we are using a workaround: we create a new [TxRaw](https://github.com/cosmos/cosmos-sdk/blob/9e85e81e0e8140067dd893421290c191529c148c/proto/cosmos/tx/v1beta1/tx.proto#L30) +instance (as defined in [adr-020-protobuf-transaction-encoding](https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-020-protobuf-transaction-encoding.md#transactions)) +by converting all [Tx](https://github.com/cosmos/cosmos-sdk/blob/9e85e81e0e8140067dd893421290c191529c148c/proto/cosmos/tx/v1beta1/tx.proto#L13) +fields to bytes on the client side. This adds an additional manual +step when sending and signing transactions. + +### Decision + +The following encoding scheme is to be used by other ADRs, +and in particular for `SignDoc` serialization. + +## Specification + +### Scope + +This ADR defines a protobuf3 serializer. The output is a valid protobuf +serialization, such that every protobuf parser can parse it. + +No maps are supported in version 1 due to the complexity of defining a +deterministic serialization. This might change in future. Implementations must +reject documents containing maps as invalid input. + +### Background - Protobuf3 Encoding + +Most numeric types in protobuf3 are encoded as +[varints](https://developers.google.com/protocol-buffers/docs/encoding#varints). +Varints are at most 10 bytes, and since each varint byte has 7 bits of data, +varints are a representation of `uint70` (70-bit unsigned integer). When +encoding, numeric values are casted from their base type to `uint70`, and when +decoding, the parsed `uint70` is casted to the appropriate numeric type. + +The maximum valid value for a varint that complies with protobuf3 is +`FF FF FF FF FF FF FF FF FF 7F` (i.e. `2**70 -1`). If the field type is +`{,u,s}int64`, the highest 6 bits of the 70 are dropped during decoding, +introducing 6 bits of malleability. If the field type is `{,u,s}int32`, the +highest 38 bits of the 70 are dropped during decoding, introducing 38 bits of +malleability. + +Among other sources of non-determinism, this ADR eliminates the possibility of +encoding malleability. + +### Serialization rules + +The serialization is based on the +[protobuf3 encoding](https://developers.google.com/protocol-buffers/docs/encoding) +with the following additions: + +1. Fields must be serialized only once in ascending order +2. Extra fields or any extra data must not be added +3. [Default values](https://developers.google.com/protocol-buffers/docs/proto3#default) + must be omitted +4. `repeated` fields of scalar numeric types must use + [packed encoding](https://developers.google.com/protocol-buffers/docs/encoding#packed) +5. Varint encoding must not be longer than needed: + * No trailing zero bytes (in little endian, i.e. no leading zeroes in big + endian). Per rule 3 above, the default value of `0` must be omitted, so + this rule does not apply in such cases. + * The maximum value for a varint must be `FF FF FF FF FF FF FF FF FF 01`. + In other words, when decoded, the highest 6 bits of the 70-bit unsigned + integer must be `0`. (10-byte varints are 10 groups of 7 bits, i.e. + 70 bits, of which only the lowest 70-6=64 are useful.) + * The maximum value for 32-bit values in varint encoding must be `FF FF FF FF 0F` + with one exception (below). In other words, when decoded, the highest 38 + bits of the 70-bit unsigned integer must be `0`. + * The one exception to the above is *negative* `int32`, which must be + encoded using the full 10 bytes for sign extension2. + * The maximum value for Boolean values in varint encoding must be `01` (i.e. + it must be `0` or `1`). Per rule 3 above, the default value of `0` must + be omitted, so if a Boolean is included it must have a value of `1`. + +While rule number 1. and 2. should be pretty straight forward and describe the +default behavior of all protobuf encoders the author is aware of, the 3rd rule +is more interesting. After a protobuf3 deserialization you cannot differentiate +between unset fields and fields set to the default value3. At +serialization level however, it is possible to set the fields with an empty +value or omitting them entirely. This is a significant difference to e.g. JSON +where a property can be empty (`""`, `0`), `null` or undefined, leading to 3 +different documents. + +Omitting fields set to default values is valid because the parser must assign +the default value to fields missing in the serialization4. For scalar +types, omitting defaults is required by the spec5. For `repeated` +fields, not serializing them is the only way to express empty lists. Enums must +have a first element of numeric value 0, which is the default6. And +message fields default to unset7. + +Omitting defaults allows for some amount of forward compatibility: users of +newer versions of a protobuf schema produce the same serialization as users of +older versions as long as newly added fields are not used (i.e. set to their +default value). + +### Implementation + +There are three main implementation strategies, ordered from the least to the +most custom development: + +* **Use a protobuf serializer that follows the above rules by default.** E.g. + [gogoproto](https://pkg.go.dev/github.com/cosmos/gogoproto/gogoproto) is known to + be compliant by in most cases, but not when certain annotations such as + `nullable = false` are used. It might also be an option to configure an + existing serializer accordingly. + +* **Normalize default values before encoding them.** If your serializer follows + rule 1. and 2. and allows you to explicitly unset fields for serialization, + you can normalize default values to unset. This can be done when working with + [protobuf.js](https://www.npmjs.com/package/protobufjs): + + ```js + const bytes = SignDoc.encode({ + bodyBytes: body.length > 0 ? body : null, // normalize empty bytes to unset + authInfoBytes: authInfo.length > 0 ? authInfo : null, // normalize empty bytes to unset + chainId: chainId || null, // normalize "" to unset + accountNumber: accountNumber || null, // normalize 0 to unset + accountSequence: accountSequence || null, // normalize 0 to unset + }).finish(); + ``` + +* **Use a hand-written serializer for the types you need.** If none of the above + ways works for you, you can write a serializer yourself. For SignDoc this + would look something like this in Go, building on existing protobuf utilities: + + ```go expandable + if !signDoc.body_bytes.empty() { + buf.WriteUVarInt64(0xA) // wire type and field number for body_bytes + buf.WriteUVarInt64(signDoc.body_bytes.length()) + + buf.WriteBytes(signDoc.body_bytes) + } + if !signDoc.auth_info.empty() { + buf.WriteUVarInt64(0x12) // wire type and field number for auth_info + buf.WriteUVarInt64(signDoc.auth_info.length()) + + buf.WriteBytes(signDoc.auth_info) + } + if !signDoc.chain_id.empty() { + buf.WriteUVarInt64(0x1a) // wire type and field number for chain_id + buf.WriteUVarInt64(signDoc.chain_id.length()) + + buf.WriteBytes(signDoc.chain_id) + } + if signDoc.account_number != 0 { + buf.WriteUVarInt64(0x20) // wire type and field number for account_number + buf.WriteUVarInt(signDoc.account_number) + } + if signDoc.account_sequence != 0 { + buf.WriteUVarInt64(0x28) // wire type and field number for account_sequence + buf.WriteUVarInt(signDoc.account_sequence) + } + ``` + +### Test vectors + +Given the protobuf definition `Article.proto` + +```protobuf expandable +package blog; +syntax = "proto3"; + +enum Type { + UNSPECIFIED = 0; + IMAGES = 1; + NEWS = 2; +}; + +enum Review { + UNSPECIFIED = 0; + ACCEPTED = 1; + REJECTED = 2; +}; + +message Article { + string title = 1; + string description = 2; + uint64 created = 3; + uint64 updated = 4; + bool public = 5; + bool promoted = 6; + Type type = 7; + Review review = 8; + repeated string comments = 9; + repeated string backlinks = 10; +}; +``` + +serializing the values + +```yaml +title: "The world needs change 🌳" +description: "" +created: 1596806111080 +updated: 0 +public: true +promoted: false +type: Type.NEWS +review: Review.UNSPECIFIED +comments: ["Nice one", "Thank you"] +backlinks: [] +``` + +must result in the serialization + +```text +0a1b54686520776f726c64206e65656473206368616e676520f09f8cb318e8bebec8bc2e280138024a084e696365206f6e654a095468616e6b20796f75 +``` + +When inspecting the serialized document, you see that every second field is +omitted: + +```shell +$ echo 0a1b54686520776f726c64206e65656473206368616e676520f09f8cb318e8bebec8bc2e280138024a084e696365206f6e654a095468616e6b20796f75 | xxd -r -p | protoc --decode_raw +1: "The world needs change \360\237\214\263" +3: 1596806111080 +5: 1 +7: 2 +9: "Nice one" +9: "Thank you" +``` + +## Consequences + +Having such an encoding available allows us to get deterministic serialization +for all protobuf documents we need in the context of Cosmos SDK signing. + +### Positive + +* Well defined rules that can be verified independent of a reference + implementation +* Simple enough to keep the barrier to implement transaction signing low +* It allows us to continue to use 0 and other empty values in SignDoc, avoiding + the need to work around 0 sequences. This does not imply the change from + [Link](https://github.com/cosmos/cosmos-sdk/pull/6949) should not be merged, but not + too important anymore. + +### Negative + +* When implementing transaction signing, the encoding rules above must be + understood and implemented. +* The need for rule number 3. adds some complexity to implementations. +* Some data structures may require custom code for serialization. Thus + the code is not very portable - it will require additional work for each + client implementing serialization to properly handle custom data structures. + +### Neutral + +### Usage in Cosmos SDK + +For the reasons mentioned above ("Negative" section) we prefer to keep workarounds +for shared data structure. Example: the aforementioned `TxRaw` is using raw bytes +as a workaround. This allows them to use any valid Protobuf library without +the need of implementing a custom serializer that adheres to this standard (and related risks of bugs). + +## References + +* 1 *When a message is serialized, there is no guaranteed order for + how its known or unknown fields should be written. Serialization order is an + implementation detail and the details of any particular implementation may + change in the future. Therefore, protocol buffer parsers must be able to parse + fields in any order.* from + [Link](https://developers.google.com/protocol-buffers/docs/encoding#order) +* 2 [Link](https://developers.google.com/protocol-buffers/docs/encoding#signed_integers) +* 3 *Note that for scalar message fields, once a message is parsed + there's no way of telling whether a field was explicitly set to the default + value (for example whether a boolean was set to false) or just not set at all: + you should bear this in mind when defining your message types. For example, + don't have a boolean that switches on some behavior when set to false if you + don't want that behavior to also happen by default.* from + [Link](https://developers.google.com/protocol-buffers/docs/proto3#default) +* 4 *When a message is parsed, if the encoded message does not + contain a particular singular element, the corresponding field in the parsed + object is set to the default value for that field.* from + [Link](https://developers.google.com/protocol-buffers/docs/proto3#default) +* 5 *Also note that if a scalar message field is set to its default, + the value will not be serialized on the wire.* from + [Link](https://developers.google.com/protocol-buffers/docs/proto3#default) +* 6 *For enums, the default value is the first defined enum value, + which must be 0.* from + [Link](https://developers.google.com/protocol-buffers/docs/proto3#default) +* 7 *For message fields, the field is not set. Its exact value is + language-dependent.* from + [Link](https://developers.google.com/protocol-buffers/docs/proto3#default) +* Encoding rules and parts of the reasoning taken from + [canonical-proto3 Aaron Craelius](https://github.com/regen-network/canonical-proto3) diff --git a/sdk/v0.54/reference/architecture/adr-028-public-key-addresses.mdx b/sdk/v0.54/reference/architecture/adr-028-public-key-addresses.mdx new file mode 100644 index 000000000..6f059e51b --- /dev/null +++ b/sdk/v0.54/reference/architecture/adr-028-public-key-addresses.mdx @@ -0,0 +1,360 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/architecture/adr-028-public-key-addresses' +title: 'ADR 028: Public Key Addresses' +description: '2020/08/18: Initial version 2021/01/15: Analysis and algorithm update' +--- + +## Changelog + +* 2020/08/18: Initial version +* 2021/01/15: Analysis and algorithm update + +## Status + +Proposed + +## Abstract + +This ADR defines an address format for all addressable Cosmos SDK accounts. That includes: new public key algorithms, multisig public keys, and module accounts. + +## Context + +Issue [#3685](https://github.com/cosmos/cosmos-sdk/issues/3685) identified that public key +address spaces are currently overlapping. We confirmed that it significantly decreases security of Cosmos SDK. + +### Problem + +An attacker can control an input for an address generation function. This leads to a birthday attack, which significantly decreases the security space. +To overcome this, we need to separate the inputs for different kind of account types: +a security break of one account type shouldn't impact the security of other account types. + +### Initial proposals + +One initial proposal was extending the address length and +adding prefixes for different types of addresses. + +@ethanfrey explained an alternate approach originally used in [Link](https://github.com/iov-one/weave): + +> I spent quite a bit of time thinking about this issue while building weave... The other cosmos Sdk. +> Basically I define a condition to be a type and format as human readable string with some binary data appended. This condition is hashed into an Address (again at 20 bytes). The use of this prefix makes it impossible to find a preimage for a given address with a different condition (eg ed25519 vs secp256k1). +> This is explained in depth here [Link](https://weave.readthedocs.io/en/latest/design/permissions.html) +> And the code is here, look mainly at the top where we process conditions. [Link](https://github.com/iov-one/weave/blob/master/conditions.go) + +And explained how this approach should be sufficiently collision resistant: + +> Yeah, AFAIK, 20 bytes should be collision resistance when the preimages are unique and not malleable. A space of 2^160 would expect some collision to be likely around 2^80 elements (birthday paradox). And if you want to find a collision for some existing element in the database, it is still 2^160. 2^80 only is if all these elements are written to state. +> The good example you brought up was eg. a public key bytes being a valid public key on two algorithms supported by the codec. Meaning if either was broken, you would break accounts even if they were secured with the safer variant. This is only as the issue when no differentiating type info is present in the preimage (before hashing into an address). +> I would like to hear an argument if the 20 bytes space is an actual issue for security, as I would be happy to increase my address sizes in weave. I just figured cosmos and ethereum and bitcoin all use 20 bytes, it should be good enough. And the arguments above which made me feel it was secure. But I have not done a deeper analysis. + +This led to the first proposal (which we proved to be not good enough): +we concatenate a key type with a public key, hash it and take the first 20 bytes of that hash, summarized as `sha256(keyTypePrefix || keybytes)[:20]`. + +### Review and Discussions + +In [#5694](https://github.com/cosmos/cosmos-sdk/issues/5694) we discussed various solutions. +We agreed that 20 bytes it's not future proof, and extending the address length is the only way to allow addresses of different types, various signature types, etc. +This disqualifies the initial proposal. + +In the issue we discussed various modifications: + +* Choice of the hash function. +* Move the prefix out of the hash function: `keyTypePrefix + sha256(keybytes)[:20]` \[post-hash-prefix-proposal]. +* Use double hashing: `sha256(keyTypePrefix + sha256(keybytes)[:20])`. +* Increase to keybytes hash slice from 20 byte to 32 or 40 bytes. We concluded that 32 bytes, produced by a good hash functions is future secure. + +### Requirements + +* Support currently used tools - we don't want to break an ecosystem, or add a long adaptation period. Ref: [Link](https://github.com/cosmos/cosmos-sdk/issues/8041) +* Try to keep the address length small - addresses are widely used in state, both as part of a key and object value. + +### Scope + +This ADR only defines a process for the generation of address bytes. For end-user interactions with addresses (through the API, or CLI, etc.), we still use bech32 to format these addresses as strings. This ADR doesn't change that. +Using Bech32 for string encoding gives us support for checksum error codes and handling of user typos. + +## Decision + +We define the following account types, for which we define the address function: + +1. simple accounts: represented by a regular public key (ie: secp256k1, sr25519) +2. naive multisig: accounts composed by other addressable objects (ie: naive multisig) +3. composed accounts with a native address key (ie: bls, group module accounts) +4. module accounts: basically any accounts which cannot sign transactions and which are managed internally by modules + +### Legacy Public Key Addresses Don't Change + +Currently (Jan 2021), the only officially supported Cosmos SDK user accounts are `secp256k1` basic accounts and legacy amino multisig. +They are used in existing Cosmos SDK zones. They use the following address formats: + +* secp256k1: `ripemd160(sha256(pk_bytes))[:20]` +* legacy amino multisig: `sha256(aminoCdc.Marshal(pk))[:20]` + +We don't want to change existing addresses. So the addresses for these two key types will remain the same. + +The current multisig public keys use amino serialization to generate the address. We will retain +those public keys and their address formatting, and call them "legacy amino" multisig public keys +in protobuf. We will also create multisig public keys without amino addresses to be described below. + +### Hash Function Choice + +As in other parts of the Cosmos SDK, we will use `sha256`. + +### Basic Address + +We start with defining a base algorithm for generating addresses which we will call `Hash`. Notably, it's used for accounts represented by a single key pair. For each public key schema we have to have an associated `typ` string, explained in the next section. `hash` is the cryptographic hash function defined in the previous section. + +```go +const A_LEN = 32 + +func Hash(typ string, key []byte) []byte { + return hash(hash(typ) + key)[:A_LEN] +} +``` + +The `+` is bytes concatenation, which doesn't use any separator. + +This algorithm is the outcome of a consultation session with a professional cryptographer. +Motivation: this algorithm keeps the address relatively small (length of the `typ` doesn't impact the length of the final address) +and it's more secure than \[post-hash-prefix-proposal] (which uses the first 20 bytes of a pubkey hash, significantly reducing the address space). +Moreover the cryptographer motivated the choice of adding `typ` in the hash to protect against a switch table attack. + +`address.Hash` is a low level function to generate *base* addresses for new key types. Example: + +* BLS: `address.Hash("bls", pubkey)` + +### Composed Addresses + +For simple composed accounts (like a new naive multisig) we generalize the `address.Hash`. The address is constructed by recursively creating addresses for the sub accounts, sorting the addresses and composing them into a single address. It ensures that the ordering of keys doesn't impact the resulting address. + +```go +// We don't need a PubKey interface - we need anything which is addressable. +type Addressable interface { + Address() []byte +} + +func Composed(typ string, subaccounts []Addressable) []byte { + addresses = map(subaccounts, \a -> LengthPrefix(a.Address())) + +addresses = sort(addresses) + +return address.Hash(typ, addresses[0] + ... + addresses[n]) +} +``` + +The `typ` parameter should be a schema descriptor, containing all significant attributes with deterministic serialization (eg: utf8 string). +`LengthPrefix` is a function which prepends 1 byte to the address. The value of that byte is the length of the address bits before prepending. The address must be at most 255 bits long. +We are using `LengthPrefix` to eliminate conflicts - it assures, that for 2 lists of addresses: `as = {a1, a2, ..., an}` and `bs = {b1, b2, ..., bm}` such that every `bi` and `ai` is at most 255 long, `concatenate(map(as, (a) => LengthPrefix(a))) = map(bs, (b) => LengthPrefix(b))` if `as = bs`. + +Implementation Tip: account implementations should cache addresses. + +#### Multisig Addresses + +For a new multisig public keys, we define the `typ` parameter not based on any encoding scheme (amino or protobuf). This avoids issues with non-determinism in the encoding scheme. + +Example: + +```protobuf +package cosmos.crypto.multisig; + +message PubKey { + uint32 threshold = 1; + repeated google.protobuf.Any pubkeys = 2; +} +``` + +```go expandable +func (multisig PubKey) + +Address() { + // first gather all nested pub keys + var keys []address.Addressable // cryptotypes.PubKey implements Addressable + for _, _key := range multisig.Pubkeys { + keys = append(keys, key.GetCachedValue().(cryptotypes.PubKey)) +} + + // form the type from the message name (cosmos.crypto.multisig.PubKey) + +and the threshold joined together + prefix := fmt.Sprintf("%s/%d", proto.MessageName(multisig), multisig.Threshold) + + // use the Composed function defined above + return address.Composed(prefix, keys) +} +``` + +### Derived Addresses + +We must be able to cryptographically derive one address from another one. The derivation process must guarantee hash properties, hence we use the already defined `Hash` function: + +```go +func Derive(address, derivationKey []byte) []byte { + return Hash(addres, derivationKey) +} +``` + +### Module Account Addresses + +A module account will have `"module"` type. Module accounts can have sub accounts. The submodule account will be created based on module name, and sequence of derivation keys. Typically, the first derivation key should be a class of the derived accounts. The derivation process has a defined order: module name, submodule key, subsubmodule key... An example module account is created using: + +```go +address.Module(moduleName, key) +``` + +An example sub-module account is created using: + +```go +groupPolicyAddresses := []byte{1 +} + +address.Module(moduleName, groupPolicyAddresses, policyID) +``` + +The `address.Module` function is using `address.Hash` with `"module"` as the type argument, and byte representation of the module name concatenated with submodule key. The two last component must be uniquely separated to avoid potential clashes (example: modulename="ab" & submodulekey="bc" will have the same derivation key as modulename="a" & submodulekey="bbc"). +We use a null byte (`'\x00'`) to separate module name from the submodule key. This works, because null byte is not a part of a valid module name. Finally, the sub-submodule accounts are created by applying the `Derive` function recursively. +We could use `Derive` function also in the first step (rather than concatenating module name with zero byte and the submodule key). We decided to do concatenation to avoid one level of derivation and speed up computation. + +For backward compatibility with the existing `authtypes.NewModuleAddress`, we add a special case in `Module` function: when no derivation key is provided, we fallback to the "legacy" implementation. + +```go +func Module(moduleName string, derivationKeys ...[]byte) []byte{ + if len(derivationKeys) == 0 { + return authtypes.NewModuleAddress(modulenName) // legacy case +} + submoduleAddress := Hash("module", []byte(moduleName) + 0 + key) + +return fold((a, k) => Derive(a, k), subsubKeys, submoduleAddress) +} +``` + +**Example 1** A lending BTC pool address would be: + +```go +btcPool := address.Module("lending", btc.Address() +}) +``` + +If we want to create an address for a module account depending on more than one key, we can concatenate them: + +```go +btcAtomAMM := address.Module("amm", btc.Address() + atom.Address() +}) +``` + +**Example 2** a smart-contract address could be constructed by: + +```go +smartContractAddr = Module("mySmartContractVM", smartContractsNamespace, smartContractKey +}) + +// which equals to: +smartContractAddr = Derived( + Module("mySmartContractVM", smartContractsNamespace), + []{ + smartContractKey +}) +``` + +### Schema Types + +A `typ` parameter used in `Hash` function SHOULD be unique for each account type. +Since all Cosmos SDK account types are serialized in the state, we propose to use the protobuf message name string. + +Example: all public key types have a unique protobuf message type similar to: + +```protobuf +package cosmos.crypto.sr25519; + +message PubKey { + bytes key = 1; +} +``` + +All protobuf messages have unique fully qualified names, in this example `cosmos.crypto.sr25519.PubKey`. +These names are derived directly from .proto files in a standardized way and used +in other places such as the type URL in `Any`s. We can easily obtain the name using +`proto.MessageName(msg)`. + +## Consequences + +### Backwards Compatibility + +This ADR is compatible with what was committed and directly supported in the Cosmos SDK repository. + +### Positive + +* a simple algorithm for generating addresses for new public keys, complex accounts and modules +* the algorithm generalizes *native composed keys* +* increased security and collision resistance of addresses +* the approach is extensible for future use-cases - one can use other address types, as long as they don't conflict with the address length specified here (20 or 32 bytes). +* support new account types. + +### Negative + +* addresses do not communicate key type, a prefixed approach would have done this +* addresses are 60% longer and will consume more storage space +* requires a refactor of KVStore store keys to handle variable length addresses + +### Neutral + +* protobuf message names are used as key type prefixes + +## Further Discussions + +Some accounts can have a fixed name or may be constructed in other way (eg: modules). We were discussing an idea of an account with a predefined name (eg: `me.regen`), which could be used by institutions. +Without going into details, these kinds of addresses are compatible with the hash based addresses described here as long as they don't have the same length. +More specifically, any special account address must not have a length equal to 20 or 32 bytes. + +## Appendix: Consulting session + +End of Dec 2020 we had a session with [Alan Szepieniec](https://scholar.google.be/citations?user=4LyZn8oAAAAJ\&hl=en) to consult the approach presented above. + +Alan general observations: + +* we don’t need 2-preimage resistance +* we need 32bytes address space for collision resistance +* when an attacker can control an input for object with an address then we have a problem with birthday attack +* there is an issue with smart-contracts for hashing +* sha2 mining can be use to breaking address pre-image + +Hashing algorithm + +* any attack breaking blake3 will break blake2 +* Alan is pretty confident about the current security analysis of the blake hash algorithm. It was a finalist, and the author is well known in security analysis. + +Algorithm: + +* Alan recommends to hash the prefix: `address(pub_key) = hash(hash(key_type) + pub_key)[:32]`, main benefits: + * we are free to user arbitrary long prefix names + * we still don’t risk collisions + * switch tables +* discussion about penalization -> about adding prefix post hash +* Aaron asked about post hash prefixes (`address(pub_key) = key_type + hash(pub_key)`) and differences. Alan noted that this approach has longer address space and it’s stronger. + +Algorithm for complex / composed keys: + +* merging tree like addresses with same algorithm are fine + +Module addresses: Should module addresses have different size to differentiate it? + +* we will need to set a pre-image prefix for module addresse to keept them in 32-byte space: `hash(hash('module') + module_key)` +* Aaron observation: we already need to deal with variable length (to not break secp256k1 keys). + +Discssion about arithmetic hash function for ZKP + +* Posseidon / Rescue +* Problem: much bigger risk because we don’t know much techniques and history of crypto-analysis of arithmetic constructions. It’s still a new ground and area of active research. + +Post quantum signature size + +* Alan suggestion: Falcon: speed / size ration - very good. +* Aaron - should we think about it? + Alan: based on early extrapolation this thing will get able to break EC cryptography in 2050 . But that’s a lot of uncertainty. But there is magic happening with recurions / linking / simulation and that can speedup the progress. + +Other ideas + +* Let’s say we use same key and two different address algorithms for 2 different use cases. Is it still safe to use it? Alan: if we want to hide the public key (which is not our use case), then it’s less secure but there are fixes. + +### References + +* [Notes](https://hackmd.io/_NGWI4xZSbKzj1BkCqyZMw) diff --git a/sdk/v0.54/reference/architecture/adr-029-fee-grant-module.mdx b/sdk/v0.54/reference/architecture/adr-029-fee-grant-module.mdx new file mode 100644 index 000000000..fa16c008f --- /dev/null +++ b/sdk/v0.54/reference/architecture/adr-029-fee-grant-module.mdx @@ -0,0 +1,163 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/architecture/adr-029-fee-grant-module' +title: 'ADR 029: Fee Grant Module' +description: >- + 2020/08/18: Initial Draft 2021/05/05: Removed height based expiration support + and simplified naming. +--- + +## Changelog + +* 2020/08/18: Initial Draft +* 2021/05/05: Removed height based expiration support and simplified naming. + +## Status + +Accepted + +## Context + +In order to make blockchain transactions, the signing account must possess a sufficient balance of the right denomination +in order to pay fees. There are classes of transactions where needing to maintain a wallet with sufficient fees is a +barrier to adoption. + +For instance, when proper permissions are setup, someone may temporarily delegate the ability to vote on proposals to +a "burner" account that is stored on a mobile phone with only minimal security. + +Other use cases include workers tracking items in a supply chain or farmers submitting field data for analytics +or compliance purposes. + +For all of these use cases, UX would be significantly enhanced by obviating the need for these accounts to always +maintain the appropriate fee balance. This is especially true if we wanted to achieve enterprise adoption for something +like supply chain tracking. + +While one solution would be to have a service that fills up these accounts automatically with the appropriate fees, a better UX +would be provided by allowing these accounts to pull from a common fee pool account with proper spending limits. +A single pool would reduce the churn of making lots of small "fill up" transactions and also more effectively leverages +the resources of the organization setting up the pool. + +## Decision + +As a solution we propose a module, `x/feegrant` which allows one account, the "granter" to grant another account, the "grantee" +an allowance to spend the granter's account balance for fees within certain well-defined limits. + +Fee allowances are defined by the extensible `FeeAllowanceI` interface: + +```go expandable +type FeeAllowanceI { + // Accept can use fee payment requested as well as timestamp of the current block + // to determine whether or not to process this. This is checked in + // Keeper.UseGrantedFees and the return values should match how it is handled there. + // + // If it returns an error, the fee payment is rejected, otherwise it is accepted. + // The FeeAllowance implementation is expected to update it's internal state + // and will be saved again after an acceptance. + // + // If remove is true (regardless of the error), the FeeAllowance will be deleted from storage + // (eg. when it is used up). (See call to RevokeFeeAllowance in Keeper.UseGrantedFees) + +Accept(ctx sdk.Context, fee sdk.Coins, msgs []sdk.Msg) (remove bool, err error) + + // ValidateBasic should evaluate this FeeAllowance for internal consistency. + // Don't allow negative amounts, or negative periods for example. + ValidateBasic() + +error +} +``` + +Two basic fee allowance types, `BasicAllowance` and `PeriodicAllowance` are defined to support known use cases: + +```protobuf expandable +// BasicAllowance implements FeeAllowanceI with a one-time grant of tokens +// that optionally expires. The delegatee can use up to SpendLimit to cover fees. +message BasicAllowance { + // spend_limit specifies the maximum amount of tokens that can be spent + // by this allowance and will be updated as tokens are spent. If it is + // empty, there is no spend limit and any amount of coins can be spent. + repeated cosmos_sdk.v1.Coin spend_limit = 1; + + // expiration specifies an optional time when this allowance expires + google.protobuf.Timestamp expiration = 2; +} + +// PeriodicAllowance extends FeeAllowanceI to allow for both a maximum cap, +// as well as a limit per time period. +message PeriodicAllowance { + BasicAllowance basic = 1; + + // period specifies the time duration in which period_spend_limit coins can + // be spent before that allowance is reset + google.protobuf.Duration period = 2; + + // period_spend_limit specifies the maximum number of coins that can be spent + // in the period + repeated cosmos_sdk.v1.Coin period_spend_limit = 3; + + // period_can_spend is the number of coins left to be spent before the period_reset time + repeated cosmos_sdk.v1.Coin period_can_spend = 4; + + // period_reset is the time at which this period resets and a new one begins, + // it is calculated from the start time of the first transaction after the + // last period ended + google.protobuf.Timestamp period_reset = 5; +} + +``` + +Allowances can be granted and revoked using `MsgGrantAllowance` and `MsgRevokeAllowance`: + +```protobuf expandable +// MsgGrantAllowance adds permission for Grantee to spend up to Allowance +// of fees from the account of Granter. +message MsgGrantAllowance { + string granter = 1; + string grantee = 2; + google.protobuf.Any allowance = 3; + } + + // MsgRevokeAllowance removes any existing FeeAllowance from Granter to Grantee. + message MsgRevokeAllowance { + string granter = 1; + string grantee = 2; + } +``` + +In order to use allowances in transactions, we add a new field `granter` to the transaction `Fee` type: + +```protobuf +package cosmos.tx.v1beta1; + +message Fee { + repeated cosmos.base.v1beta1.Coin amount = 1; + uint64 gas_limit = 2; + string payer = 3; + string granter = 4; +} +``` + +`granter` must either be left empty or must correspond to an account which has granted +a fee allowance to fee payer (either the first signer or the value of the `payer` field). + +A new `AnteDecorator` named `DeductGrantedFeeDecorator` will be created in order to process transactions with `fee_payer` +set and correctly deduct fees based on fee allowances. + +## Consequences + +### Positive + +* improved UX for use cases where it is cumbersome to maintain an account balance just for fees + +### Negative + +### Neutral + +* a new field must be added to the transaction `Fee` message and a new `AnteDecorator` must be + created to use it + +## References + +* Blog article describing initial work: [Link](https://medium.com/regen-network/hacking-the-cosmos-cosmwasm-and-key-management-a08b9f561d1b) +* Initial public specification: [Link](https://gist.github.com/aaronc/b60628017352df5983791cad30babe56) +* Original subkeys proposal from B-harvest which influenced this design: [Link](https://github.com/cosmos/cosmos-sdk/issues/4480) diff --git a/sdk/v0.54/reference/architecture/adr-030-authz-module.mdx b/sdk/v0.54/reference/architecture/adr-030-authz-module.mdx new file mode 100644 index 000000000..0daa5d8de --- /dev/null +++ b/sdk/v0.54/reference/architecture/adr-030-authz-module.mdx @@ -0,0 +1,289 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/architecture/adr-030-authz-module' +title: 'ADR 030: Authorization Module' +--- + +## Changelog + +* 2019-11-06: Initial Draft +* 2020-10-12: Updated Draft +* 2020-11-13: Accepted +* 2020-05-06: proto API updates, use `sdk.Msg` instead of `sdk.ServiceMsg` (the latter concept was removed from Cosmos SDK) +* 2022-04-20: Updated the `SendAuthorization` proto docs to clarify the `SpendLimit` is a required field. (Generic authorization can be used with bank msg type url to create limit less bank authorization) + +## Status + +Accepted + +## Abstract + +This ADR defines the `x/authz` module which allows accounts to grant authorizations to perform actions +on behalf of that account to other accounts. + +## Context + +The concrete use cases which motivated this module include: + +* the desire to delegate the ability to vote on proposals to other accounts besides the account which one has + delegated stake +* "sub-keys" functionality, as originally proposed in [#4480](https://github.com/cosmos/cosmos-sdk/issues/4480) which + is a term used to describe the functionality provided by this module together with + the `fee_grant` module from [ADR 029](/sdk/v0.50/build/architecture/adr-029-fee-grant-module) and the [group module](https://github.com/cosmos/cosmos-sdk/tree/main/x/group). + +The "sub-keys" functionality roughly refers to the ability for one account to grant some subset of its capabilities to +other accounts with possibly less robust, but easier to use security measures. For instance, a master account representing +an organization could grant the ability to spend small amounts of the organization's funds to individual employee accounts. +Or an individual (or group) with a multisig wallet could grant the ability to vote on proposals to any one of the member +keys. + +The current implementation is based on work done by the [Gaian's team at Hackatom Berlin 2019](https://github.com/cosmos-gaians/cosmos-sdk/tree/hackatom/x/delegation). + +## Decision + +We will create a module named `authz` which provides functionality for +granting arbitrary privileges from one account (the *granter*) to another account (the *grantee*). Authorizations +must be granted for a particular `Msg` service methods one by one using an implementation +of `Authorization` interface. + +### Types + +Authorizations determine exactly what privileges are granted. They are extensible +and can be defined for any `Msg` service method even outside of the module where +the `Msg` method is defined. `Authorization`s reference `Msg`s using their TypeURL. + +#### Authorization + +```go expandable +type Authorization interface { + proto.Message + + // MsgTypeURL returns the fully-qualified Msg TypeURL (as described in ADR 020), + // which will process and accept or reject a request. + MsgTypeURL() + +string + + // Accept determines whether this grant permits the provided sdk.Msg to be performed, and if + // so provides an upgraded authorization instance. + Accept(ctx sdk.Context, msg sdk.Msg) (AcceptResponse, error) + + // ValidateBasic does a simple validation check that + // doesn't require access to any other information. + ValidateBasic() + +error +} + +// AcceptResponse instruments the controller of an authz message if the request is accepted +// and if it should be updated or deleted. +type AcceptResponse struct { + // If Accept=true, the controller can accept and authorization and handle the update. + Accept bool + // If Delete=true, the controller must delete the authorization object and release + // storage resources. + Delete bool + // Controller, who is calling Authorization.Accept must check if `Updated != nil`. If yes, + // it must use the updated version and handle the update on the storage level. + Updated Authorization +} +``` + +For example a `SendAuthorization` like this is defined for `MsgSend` that takes +a `SpendLimit` and updates it down to zero: + +```go expandable +type SendAuthorization struct { + // SpendLimit specifies the maximum amount of tokens that can be spent + // by this authorization and will be updated as tokens are spent. This field is required. (Generic authorization + // can be used with bank msg type url to create limit less bank authorization). + SpendLimit sdk.Coins +} + +func (a SendAuthorization) + +MsgTypeURL() + +string { + return sdk.MsgTypeURL(&MsgSend{ +}) +} + +func (a SendAuthorization) + +Accept(ctx sdk.Context, msg sdk.Msg) (authz.AcceptResponse, error) { + mSend, ok := msg.(*MsgSend) + if !ok { + return authz.AcceptResponse{ +}, sdkerrors.ErrInvalidType.Wrap("type mismatch") +} + +limitLeft, isNegative := a.SpendLimit.SafeSub(mSend.Amount) + if isNegative { + return authz.AcceptResponse{ +}, sdkerrors.ErrInsufficientFunds.Wrapf("requested amount is more than spend limit") +} + if limitLeft.IsZero() { + return authz.AcceptResponse{ + Accept: true, + Delete: true +}, nil +} + +return authz.AcceptResponse{ + Accept: true, + Delete: false, + Updated: &SendAuthorization{ + SpendLimit: limitLeft +}}, nil +} +``` + +A different type of capability for `MsgSend` could be implemented +using the `Authorization` interface with no need to change the underlying +`bank` module. + +##### Small notes on `AcceptResponse` + +* The `AcceptResponse.Accept` field will be set to `true` if the authorization is accepted. + However, if it is rejected, the function `Accept` will raise an error (without setting `AcceptResponse.Accept` to `false`). + +* The `AcceptResponse.Updated` field will be set to a non-nil value only if there is a real change to the authorization. + If authorization remains the same (as is, for instance, always the case for a [`GenericAuthorization`](#genericauthorization)), + the field will be `nil`. + +### `Msg` Service + +```protobuf expandable +service Msg { + // Grant grants the provided authorization to the grantee on the granter's + // account with the provided expiration time. + rpc Grant(MsgGrant) returns (MsgGrantResponse); + + // Exec attempts to execute the provided messages using + // authorizations granted to the grantee. Each message should have only + // one signer corresponding to the granter of the authorization. + rpc Exec(MsgExec) returns (MsgExecResponse); + + // Revoke revokes any authorization corresponding to the provided method name on the + // granter's account that has been granted to the grantee. + rpc Revoke(MsgRevoke) returns (MsgRevokeResponse); +} + +// Grant gives permissions to execute +// the provided method with expiration time. +message Grant { + google.protobuf.Any authorization = 1 [(cosmos_proto.accepts_interface) = "cosmos.authz.v1beta1.Authorization"]; + google.protobuf.Timestamp expiration = 2 [(gogoproto.stdtime) = true, (gogoproto.nullable) = false]; +} + +message MsgGrant { + string granter = 1; + string grantee = 2; + + Grant grant = 3 [(gogoproto.nullable) = false]; +} + +message MsgExecResponse { + cosmos.base.abci.v1beta1.Result result = 1; +} + +message MsgExec { + string grantee = 1; + // Authorization Msg requests to execute. Each msg must implement Authorization interface + repeated google.protobuf.Any msgs = 2 [(cosmos_proto.accepts_interface) = "cosmos.base.v1beta1.Msg"];; +} +``` + +### Router Middleware + +The `authz` `Keeper` will expose a `DispatchActions` method which allows other modules to send `Msg`s +to the router based on `Authorization` grants: + +```go +type Keeper interface { + // DispatchActions routes the provided msgs to their respective handlers if the grantee was granted an authorization + // to send those messages by the first (and only) + +signer of each msg. + DispatchActions(ctx sdk.Context, grantee sdk.AccAddress, msgs []sdk.Msg) + +sdk.Result` +} +``` + +### CLI + +#### `tx exec` Method + +When a CLI user wants to run a transaction on behalf of another account using `MsgExec`, they +can use the `exec` method. For instance `gaiacli tx gov vote 1 yes --from --generate-only | gaiacli tx authz exec --send-as --from ` +would send a transaction like this: + +```go +MsgExec { + Grantee: mykey, + Msgs: []sdk.Msg{ + MsgVote { + ProposalID: 1, + Voter: cosmos3thsdgh983egh823 + Option: Yes +} + +} +} +``` + +#### `tx grant --from ` + +This CLI command will send a `MsgGrant` transaction. `authorization` should be encoded as +JSON on the CLI. + +#### `tx revoke --from ` + +This CLI command will send a `MsgRevoke` transaction. + +### Built-in Authorizations + +#### `SendAuthorization` + +```protobuf +// SendAuthorization allows the grantee to spend up to spend_limit coins from +// the granter's account. +message SendAuthorization { + repeated cosmos.base.v1beta1.Coin spend_limit = 1; +} +``` + +#### `GenericAuthorization` + +```protobuf +// GenericAuthorization gives the grantee unrestricted permissions to execute +// the provided method on behalf of the granter's account. +message GenericAuthorization { + option (cosmos_proto.implements_interface) = "Authorization"; + + // Msg, identified by it's type URL, to grant unrestricted permissions to execute + string msg = 1; +} +``` + +## Consequences + +### Positive + +* Users will be able to authorize arbitrary actions on behalf of their accounts to other + users, improving key management for many use cases +* The solution is more generic than previously considered approaches and the + `Authorization` interface approach can be extended to cover other use cases by + SDK users + +### Negative + +### Neutral + +## References + +* Initial Hackatom implementation: [Link](https://github.com/cosmos-gaians/cosmos-sdk/tree/hackatom/x/delegation) +* Post-Hackatom spec: [Link](https://gist.github.com/aaronc/b60628017352df5983791cad30babe56#delegation-module) +* B-Harvest subkeys spec: [Link](https://github.com/cosmos/cosmos-sdk/issues/4480) diff --git a/sdk/v0.54/reference/architecture/adr-031-msg-service.mdx b/sdk/v0.54/reference/architecture/adr-031-msg-service.mdx new file mode 100644 index 000000000..45aefed3b --- /dev/null +++ b/sdk/v0.54/reference/architecture/adr-031-msg-service.mdx @@ -0,0 +1,218 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/architecture/adr-031-msg-service' +title: 'ADR 031: Protobuf Msg Services' +description: >- + 2020-10-05: Initial Draft 2021-04-21: Remove ServiceMsgs to follow Protobuf + Any's spec, see #9063. +--- + +## Changelog + +* 2020-10-05: Initial Draft +* 2021-04-21: Remove `ServiceMsg`s to follow Protobuf `Any`'s spec, see [#9063](https://github.com/cosmos/cosmos-sdk/issues/9063). + +## Status + +Accepted + +## Abstract + +We want to leverage protobuf `service` definitions for defining `Msg`s which will give us significant developer UX +improvements in terms of the code that is generated and the fact that return types will now be well defined. + +## Context + +Currently `Msg` handlers in the Cosmos SDK do have return values that are placed in the `data` field of the response. +These return values, however, are not specified anywhere except in the golang handler code. + +In early conversations it was proposed +that `Msg` return types be captured using a protobuf extension field, ex: + +```protobuf +package cosmos.gov; + +message MsgSubmitProposal + option (cosmos_proto.msg_return) = “uint64”; + string delegator_address = 1; + string validator_address = 2; + repeated sdk.Coin amount = 3; +} +``` + +This was never adopted, however. + +Having a well-specified return value for `Msg`s would improve client UX. For instance, +in `x/gov`, `MsgSubmitProposal` returns the proposal ID as a big-endian `uint64`. +This isn’t really documented anywhere and clients would need to know the internals +of the Cosmos SDK to parse that value and return it to users. + +Also, there may be cases where we want to use these return values programatically. +For instance, [Link](https://github.com/cosmos/cosmos-sdk/issues/7093) proposes a method for +doing inter-module Ocaps using the `Msg` router. A well-defined return type would +improve the developer UX for this approach. + +In addition, handler registration of `Msg` types tends to add a bit of +boilerplate on top of keepers and is usually done through manual type switches. +This isn't necessarily bad, but it does add overhead to creating modules. + +## Decision + +We decide to use protobuf `service` definitions for defining `Msg`s as well as +the code generated by them as a replacement for `Msg` handlers. + +Below we define how this will look for the `SubmitProposal` message from `x/gov` module. +We start with a `Msg` `service` definition: + +```protobuf expandable +package cosmos.gov; + +service Msg { + rpc SubmitProposal(MsgSubmitProposal) returns (MsgSubmitProposalResponse); +} + +// Note that for backwards compatibility this uses MsgSubmitProposal as the request +// type instead of the more canonical MsgSubmitProposalRequest +message MsgSubmitProposal { + google.protobuf.Any content = 1; + string proposer = 2; +} + +message MsgSubmitProposalResponse { + uint64 proposal_id; +} +``` + +While this is most commonly used for gRPC, overloading protobuf `service` definitions like this does not violate +the intent of the [protobuf spec](https://developers.google.com/protocol-buffers/docs/proto3#services) which says: + +> If you don’t want to use gRPC, it’s also possible to use protocol buffers with your own RPC implementation. +> With this approach, we would get an auto-generated `MsgServer` interface: + +In addition to clearly specifying return types, this has the benefit of generating client and server code. On the server +side, this is almost like an automatically generated keeper method and could maybe be used intead of keepers eventually +(see [#7093](https://github.com/cosmos/cosmos-sdk/issues/7093)): + +```go +package gov + +type MsgServer interface { + SubmitProposal(context.Context, *MsgSubmitProposal) (*MsgSubmitProposalResponse, error) +} +``` + +On the client side, developers could take advantage of this by creating RPC implementations that encapsulate transaction +logic. Protobuf libraries that use asynchronous callbacks, like [protobuf.js](https://github.com/protobufjs/protobuf.js#using-services) +could use this to register callbacks for specific messages even for transactions that include multiple `Msg`s. + +Each `Msg` service method should have exactly one request parameter: its corresponding `Msg` type. For example, the `Msg` service method `/cosmos.gov.v1beta1.Msg/SubmitProposal` above has exactly one request parameter, namely the `Msg` type `/cosmos.gov.v1beta1.MsgSubmitProposal`. It is important the reader understands clearly the nomenclature difference between a `Msg` service (a Protobuf service) and a `Msg` type (a Protobuf message), and the differences in their fully-qualified name. + +This convention has been decided over the more canonical `Msg...Request` names mainly for backwards compatibility, but also for better readability in `TxBody.messages` (see [Encoding section](#encoding) below): transactions containing `/cosmos.gov.MsgSubmitProposal` read better than those containing `/cosmos.gov.v1beta1.MsgSubmitProposalRequest`. + +One consequence of this convention is that each `Msg` type can be the request parameter of only one `Msg` service method. However, we consider this limitation a good practice in explicitness. + +### Encoding + +Encoding of transactions generated with `Msg` services do not differ from current Protobuf transaction encoding as defined in [ADR-020](/sdk/v0.50/build/architecture/adr-020-protobuf-transaction-encoding). We are encoding `Msg` types (which are exactly `Msg` service methods' request parameters) as `Any` in `Tx`s which involves packing the +binary-encoded `Msg` with its type URL. + +### Decoding + +Since `Msg` types are packed into `Any`, decoding transactions messages are done by unpacking `Any`s into `Msg` types. For more information, please refer to [ADR-020](/sdk/v0.50/build/architecture/adr-020-protobuf-transaction-encoding#transactions). + +### Routing + +We propose to add a `msg_service_router` in BaseApp. This router is a key/value map which maps `Msg` types' `type_url`s to their corresponding `Msg` service method handler. Since there is a 1-to-1 mapping between `Msg` types and `Msg` service method, the `msg_service_router` has exactly one entry per `Msg` service method. + +When a transaction is processed by BaseApp (in CheckTx or in DeliverTx), its `TxBody.messages` are decoded as `Msg`s. Each `Msg`'s `type_url` is matched against an entry in the `msg_service_router`, and the respective `Msg` service method handler is called. + +For backward compatibility, the old handlers are not removed yet. If BaseApp receives a legacy `Msg` with no corresponding entry in the `msg_service_router`, it will be routed via its legacy `Route()` method into the legacy handler. + +### Module Configuration + +In [ADR 021](/sdk/v0.50/build/architecture/adr-021-protobuf-query-encoding), we introduced a method `RegisterQueryService` +to `AppModule` which allows for modules to register gRPC queriers. + +To register `Msg` services, we attempt a more extensible approach by converting `RegisterQueryService` +to a more generic `RegisterServices` method: + +```go expandable +type AppModule interface { + RegisterServices(Configurator) + ... +} + +type Configurator interface { + QueryServer() + +grpc.Server + MsgServer() + +grpc.Server +} + +// example module: +func (am AppModule) + +RegisterServices(cfg Configurator) { + types.RegisterQueryServer(cfg.QueryServer(), keeper) + +types.RegisterMsgServer(cfg.MsgServer(), keeper) +} +``` + +The `RegisterServices` method and the `Configurator` interface are intended to +evolve to satisfy the use cases discussed in [#7093](https://github.com/cosmos/cosmos-sdk/issues/7093) +and [#7122](https://github.com/cosmos/cosmos-sdk/issues/7421). + +When `Msg` services are registered, the framework *should* verify that all `Msg` types +implement the `sdk.Msg` interface and throw an error during initialization rather +than later when transactions are processed. + +### `Msg` Service Implementation + +Just like query services, `Msg` service methods can retrieve the `sdk.Context` +from the `context.Context` parameter method using the `sdk.UnwrapSDKContext` +method: + +```go +package gov + +func (k Keeper) + +SubmitProposal(goCtx context.Context, params *types.MsgSubmitProposal) (*MsgSubmitProposalResponse, error) { + ctx := sdk.UnwrapSDKContext(goCtx) + ... +} +``` + +The `sdk.Context` should have an `EventManager` already attached by BaseApp's `msg_service_router`. + +Separate handler definition is no longer needed with this approach. + +## Consequences + +This design changes how a module functionality is exposed and accessed. It deprecates the existing `Handler` interface and `AppModule.Route` in favor of [Protocol Buffer Services](https://developers.google.com/protocol-buffers/docs/proto3#services) and Service Routing described above. This dramatically simplifies the code. We don't need to create handlers and keepers any more. Use of Protocol Buffer auto-generated clients clearly separates the communication interfaces between the module and a modules user. The control logic (aka handlers and keepers) is not exposed any more. A module interface can be seen as a black box accessible through a client API. It's worth to note that the client interfaces are also generated by Protocol Buffers. + +This also allows us to change how we perform functional tests. Instead of mocking AppModules and Router, we will mock a client (server will stay hidden). More specifically: we will never mock `moduleA.MsgServer` in `moduleB`, but rather `moduleA.MsgClient`. One can think about it as working with external services (eg DBs, or online servers...). We assume that the transmission between clients and servers is correctly handled by generated Protocol Buffers. + +Finally, closing a module to client API opens desirable OCAP patterns discussed in ADR-033. Since server implementation and interface is hidden, nobody can hold "keepers"/servers and will be forced to relay on the client interface, which will drive developers for correct encapsulation and software engineering patterns. + +### Pros + +* communicates return type clearly +* manual handler registration and return type marshaling is no longer needed, just implement the interface and register it +* communication interface is automatically generated, the developer can now focus only on the state transition methods - this would improve the UX of [#7093](https://github.com/cosmos/cosmos-sdk/issues/7093) approach (1) if we chose to adopt that +* generated client code could be useful for clients and tests +* dramatically reduces and simplifies the code + +### Cons + +* using `service` definitions outside the context of gRPC could be confusing (but doesn’t violate the proto3 spec) + +## References + +* [Initial Github Issue #7122](https://github.com/cosmos/cosmos-sdk/issues/7122) +* [proto 3 Language Guide: Defining Services](https://developers.google.com/protocol-buffers/docs/proto3#services) +* [ADR 020](/sdk/v0.50/build/architecture/adr-020-protobuf-transaction-encoding) +* [ADR 021](/sdk/v0.50/build/architecture/adr-021-protobuf-query-encoding) diff --git a/sdk/v0.54/reference/architecture/adr-032-typed-events.mdx b/sdk/v0.54/reference/architecture/adr-032-typed-events.mdx new file mode 100644 index 000000000..75e3efc41 --- /dev/null +++ b/sdk/v0.54/reference/architecture/adr-032-typed-events.mdx @@ -0,0 +1,353 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/architecture/adr-032-typed-events' +title: 'ADR 032: Typed Events' +description: '28-Sept-2020: Initial Draft' +--- + +## Changelog + +* 28-Sept-2020: Initial Draft + +## Authors + +* Anil Kumar (@anilcse) +* Jack Zampolin (@jackzampolin) +* Adam Bozanich (@boz) + +## Status + +Proposed + +## Abstract + +Currently in the Cosmos SDK, events are defined in the handlers for each message as well as `BeginBlock` and `EndBlock`. Each module doesn't have types defined for each event, they are implemented as `map[string]string`. Above all else this makes these events difficult to consume as it requires a great deal of raw string matching and parsing. This proposal focuses on updating the events to use **typed events** defined in each module such that emiting and subscribing to events will be much easier. This workflow comes from the experience of the Akash Network team. + +## Context + +Currently in the Cosmos SDK, events are defined in the handlers for each message, meaning each module doesn't have a cannonical set of types for each event. Above all else this makes these events difficult to consume as it requires a great deal of raw string matching and parsing. This proposal focuses on updating the events to use **typed events** defined in each module such that emiting and subscribing to events will be much easier. This workflow comes from the experience of the Akash Network team. + +[Our platform](http://github.com/ovrclk/akash) requires a number of programatic on chain interactions both on the provider (datacenter - to bid on new orders and listen for leases created) and user (application developer - to send the app manifest to the provider) side. In addition the Akash team is now maintaining the IBC [`relayer`](https://github.com/ovrclk/relayer), another very event driven process. In working on these core pieces of infrastructure, and integrating lessons learned from Kubernetes developement, our team has developed a standard method for defining and consuming typed events in Cosmos SDK modules. We have found that it is extremely useful in building this type of event driven application. + +As the Cosmos SDK gets used more extensively for apps like `peggy`, other peg zones, IBC, DeFi, etc... there will be an exploding demand for event driven applications to support new features desired by users. We propose upstreaming our findings into the Cosmos SDK to enable all Cosmos SDK applications to quickly and easily build event driven apps to aid their core application. Wallets, exchanges, explorers, and defi protocols all stand to benefit from this work. + +If this proposal is accepted, users will be able to build event driven Cosmos SDK apps in go by just writing `EventHandler`s for their specific event types and passing them to `EventEmitters` that are defined in the Cosmos SDK. + +The end of this proposal contains a detailed example of how to consume events after this refactor. + +This proposal is specifically about how to consume these events as a client of the blockchain, not for intermodule communication. + +## Decision + +**Step-1**: Implement additional functionality in the `types` package: `EmitTypedEvent` and `ParseTypedEvent` functions + +```go expandable +// types/events.go + +// EmitTypedEvent takes typed event and emits converting it into sdk.Event +func (em *EventManager) + +EmitTypedEvent(event proto.Message) + +error { + evtType := proto.MessageName(event) + +evtJSON, err := codec.ProtoMarshalJSON(event) + if err != nil { + return err +} + +var attrMap map[string]json.RawMessage + err = json.Unmarshal(evtJSON, &attrMap) + if err != nil { + return err +} + +var attrs []abci.EventAttribute + for k, v := range attrMap { + attrs = append(attrs, abci.EventAttribute{ + Key: []byte(k), + Value: v, +}) +} + +em.EmitEvent(Event{ + Type: evtType, + Attributes: attrs, +}) + +return nil +} + +// ParseTypedEvent converts abci.Event back to typed event +func ParseTypedEvent(event abci.Event) (proto.Message, error) { + concreteGoType := proto.MessageType(event.Type) + if concreteGoType == nil { + return nil, fmt.Errorf("failed to retrieve the message of type %q", event.Type) +} + +var value reflect.Value + if concreteGoType.Kind() == reflect.Ptr { + value = reflect.New(concreteGoType.Elem()) +} + +else { + value = reflect.Zero(concreteGoType) +} + +protoMsg, ok := value.Interface().(proto.Message) + if !ok { + return nil, fmt.Errorf("%q does not implement proto.Message", event.Type) +} + attrMap := make(map[string]json.RawMessage) + for _, attr := range event.Attributes { + attrMap[string(attr.Key)] = attr.Value +} + +attrBytes, err := json.Marshal(attrMap) + if err != nil { + return nil, err +} + +err = jsonpb.Unmarshal(strings.NewReader(string(attrBytes)), protoMsg) + if err != nil { + return nil, err +} + +return protoMsg, nil +} +``` + +Here, the `EmitTypedEvent` is a method on `EventManager` which takes typed event as input and apply json serialization on it. Then it maps the JSON key/value pairs to `event.Attributes` and emits it in form of `sdk.Event`. `Event.Type` will be the type URL of the proto message. + +When we subscribe to emitted events on the CometBFT websocket, they are emitted in the form of an `abci.Event`. `ParseTypedEvent` parses the event back to it's original proto message. + +**Step-2**: Add proto definitions for typed events for msgs in each module: + +For example, let's take `MsgSubmitProposal` of `gov` module and implement this event's type. + +```protobuf +// proto/cosmos/gov/v1beta1/gov.proto +// Add typed event definition + +package cosmos.gov.v1beta1; + +message EventSubmitProposal { + string from_address = 1; + uint64 proposal_id = 2; + TextProposal proposal = 3; +} +``` + +**Step-3**: Refactor event emission to use the typed event created and emit using `sdk.EmitTypedEvent`: + +```go expandable +// x/gov/handler.go +func handleMsgSubmitProposal(ctx sdk.Context, keeper keeper.Keeper, msg types.MsgSubmitProposalI) (*sdk.Result, error) { + ... + types.Context.EventManager().EmitTypedEvent( + &EventSubmitProposal{ + FromAddress: fromAddress, + ProposalId: id, + Proposal: proposal, +}, + ) + ... +} +``` + +### How to subscribe to these typed events in `Client` + +> NOTE: Full code example below + +Users will be able to subscribe using `client.Context.Client.Subscribe` and consume events which are emitted using `EventHandler`s. + +Akash Network has built a simple [`pubsub`](https://github.com/ovrclk/akash/blob/90d258caeb933b611d575355b8df281208a214f8/pubsub/bus.go#L20). This can be used to subscribe to `abci.Events` and [publish](https://github.com/ovrclk/akash/blob/90d258caeb933b611d575355b8df281208a214f8/events/publish.go#L21) them as typed events. + +Please see the below code sample for more detail on this flow looks for clients. + +## Consequences + +### Positive + +* Improves consistency of implementation for the events currently in the Cosmos SDK +* Provides a much more ergonomic way to handle events and facilitates writing event driven applications +* This implementation will support a middleware ecosystem of `EventHandler`s + +### Negative + +## Detailed code example of publishing events + +This ADR also proposes adding affordances to emit and consume these events. This way developers will only need to write +`EventHandler`s which define the actions they desire to take. + +```go expandable +// EventEmitter is a type that describes event emitter functions +// This should be defined in `types/events.go` +type EventEmitter func(context.Context, client.Context, ...EventHandler) + +error + +// EventHandler is a type of function that handles events coming out of the event bus +// This should be defined in `types/events.go` +type EventHandler func(proto.Message) + +error + +// Sample use of the functions below +func main() { + ctx, cancel := context.WithCancel(context.Background()) + if err := TxEmitter(ctx, client.Context{ +}.WithNodeURI("tcp://localhost:26657"), SubmitProposalEventHandler); err != nil { + cancel() + +panic(err) +} + +return +} + +// SubmitProposalEventHandler is an example of an event handler that prints proposal details +// when any EventSubmitProposal is emitted. +func SubmitProposalEventHandler(ev proto.Message) (err error) { + switch event := ev.(type) { + // Handle governance proposal events creation events + case govtypes.EventSubmitProposal: + // Users define business logic here e.g. + fmt.Println(ev.FromAddress, ev.ProposalId, ev.Proposal) + +return nil + default: + return nil +} +} + +// TxEmitter is an example of an event emitter that emits just transaction events. This can and +// should be implemented somewhere in the Cosmos SDK. The Cosmos SDK can include an EventEmitters for tm.event='Tx' +// and/or tm.event='NewBlock' (the new block events may contain typed events) + +func TxEmitter(ctx context.Context, cliCtx client.Context, ehs ...EventHandler) (err error) { + // Instantiate and start CometBFT RPC client + client, err := cliCtx.GetNode() + if err != nil { + return err +} + if err = client.Start(); err != nil { + return err +} + + // Start the pubsub bus + bus := pubsub.NewBus() + +defer bus.Close() + + // Initialize a new error group + eg, ctx := errgroup.WithContext(ctx) + + // Publish chain events to the pubsub bus + eg.Go(func() + +error { + return PublishChainTxEvents(ctx, client, bus, simapp.ModuleBasics) +}) + + // Subscribe to the bus events + subscriber, err := bus.Subscribe() + if err != nil { + return err +} + + // Handle all the events coming out of the bus + eg.Go(func() + +error { + var err error + for { + select { + case <-ctx.Done(): + return nil + case <-subscriber.Done(): + return nil + case ev := <-subscriber.Events(): + for _, eh := range ehs { + if err = eh(ev); err != nil { + break +} + +} + +} + +} + +return nil +}) + +return group.Wait() +} + +// PublishChainTxEvents events using cmtclient. Waits on context shutdown signals to exit. +func PublishChainTxEvents(ctx context.Context, client cmtclient.EventsClient, bus pubsub.Bus, mb module.BasicManager) (err error) { + // Subscribe to transaction events + txch, err := client.Subscribe(ctx, "txevents", "tm.event='Tx'", 100) + if err != nil { + return err +} + + // Unsubscribe from transaction events on function exit + defer func() { + err = client.UnsubscribeAll(ctx, "txevents") +}() + + // Use errgroup to manage concurrency + g, ctx := errgroup.WithContext(ctx) + + // Publish transaction events in a goroutine + g.Go(func() + +error { + var err error + for { + select { + case <-ctx.Done(): + break + case ed := <-ch: + switch evt := ed.Data.(type) { + case cmttypes.EventDataTx: + if !evt.Result.IsOK() { + continue +} + // range over events, parse them using the basic manager and + // send them to the pubsub bus + for _, abciEv := range events { + typedEvent, err := sdk.ParseTypedEvent(abciEv) + if err != nil { + return er +} + if err := bus.Publish(typedEvent); err != nil { + bus.Close() + +return +} + +continue +} + +} + +} + +} + +return err +}) + + // Exit on error or context cancelation + return g.Wait() +} +``` + +## References + +* [Publish Custom Events via a bus](https://github.com/ovrclk/akash/blob/90d258caeb933b611d575355b8df281208a214f8/events/publish.go#L19-L58) +* [Consuming the events in `Client`](https://github.com/ovrclk/deploy/blob/bf6c633ab6c68f3026df59efd9982d6ca1bf0561/cmd/event-handlers.go#L57) diff --git a/sdk/v0.54/reference/architecture/adr-033-protobuf-inter-module-comm.mdx b/sdk/v0.54/reference/architecture/adr-033-protobuf-inter-module-comm.mdx new file mode 100644 index 000000000..b898b782b --- /dev/null +++ b/sdk/v0.54/reference/architecture/adr-033-protobuf-inter-module-comm.mdx @@ -0,0 +1,457 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/architecture/adr-033-protobuf-inter-module-comm' +title: 'ADR 033: Protobuf-based Inter-Module Communication' +description: '2020-10-05: Initial Draft' +--- + +## Changelog + +* 2020-10-05: Initial Draft + +## Status + +Proposed + +## Abstract + +This ADR introduces a system for permissioned inter-module communication leveraging the protobuf `Query` and `Msg` +service definitions defined in [ADR 021](/sdk/v0.50/build/architecture/adr-021-protobuf-query-encoding) and +[ADR 031](/sdk/v0.50/build/architecture/adr-031-msg-service) which provides: + +* stable protobuf based module interfaces to potentially later replace the keeper paradigm +* stronger inter-module object capabilities (OCAPs) guarantees +* module accounts and sub-account authorization + +## Context + +In the current Cosmos SDK documentation on the [Object-Capability Model](/sdk/v0.54/guides/module-design/ocap), it is stated that: + +> We assume that a thriving ecosystem of Cosmos SDK modules that are easy to compose into a blockchain application will contain faulty or malicious modules. + +There is currently not a thriving ecosystem of Cosmos SDK modules. We hypothesize that this is in part due to: + +1. lack of a stable v1.0 Cosmos SDK to build modules off of. Module interfaces are changing, sometimes dramatically, from + point release to point release, often for good reasons, but this does not create a stable foundation to build on. +2. lack of a properly implemented object capability or even object-oriented encapsulation system which makes refactors + of module keeper interfaces inevitable because the current interfaces are poorly constrained. + +### `x/bank` Case Study + +Currently the `x/bank` keeper gives pretty much unrestricted access to any module which references it. For instance, the +`SetBalance` method allows the caller to set the balance of any account to anything, bypassing even proper tracking of supply. + +There appears to have been some later attempts to implement some semblance of OCAPs using module-level minting, staking +and burning permissions. These permissions allow a module to mint, burn or delegate tokens with reference to the module’s +own account. These permissions are actually stored as a `[]string` array on the `ModuleAccount` type in state. + +However, these permissions don’t really do much. They control what modules can be referenced in the `MintCoins`, +`BurnCoins` and `DelegateCoins***` methods, but for one there is no unique object capability token that controls access — +just a simple string. So the `x/upgrade` module could mint tokens for the `x/staking` module simple by calling +`MintCoins(“staking”)`. Furthermore, all modules which have access to these keeper methods, also have access to +`SetBalance` negating any other attempt at OCAPs and breaking even basic object-oriented encapsulation. + +## Decision + +Based on [ADR-021](/sdk/v0.50/build/architecture/adr-021-protobuf-query-encoding) and [ADR-031](/sdk/v0.50/build/architecture/adr-031-msg-service), we introduce the +Inter-Module Communication framework for secure module authorization and OCAPs. +When implemented, this could also serve as an alternative to the existing paradigm of passing keepers between +modules. The approach outlined here-in is intended to form the basis of a Cosmos SDK v1.0 that provides the necessary +stability and encapsulation guarantees that allow a thriving module ecosystem to emerge. + +Of particular note — the decision is to *enable* this functionality for modules to adopt at their own discretion. +Proposals to migrate existing modules to this new paradigm will have to be a separate conversation, potentially +addressed as amendments to this ADR. + +### New "Keeper" Paradigm + +In [ADR 021](/sdk/v0.50/build/architecture/adr-021-protobuf-query-encoding), a mechanism for using protobuf service definitions to define queriers +was introduced and in [ADR 31](/sdk/v0.50/build/architecture/adr-031-msg-service), a mechanism for using protobuf service to define `Msg`s was added. +Protobuf service definitions generate two golang interfaces representing the client and server sides of a service plus +some helper code. Here is a minimal example for the bank `cosmos.bank.Msg/Send` message type: + +```go +package bank + +type MsgClient interface { + Send(context.Context, *MsgSend, opts ...grpc.CallOption) (*MsgSendResponse, error) +} + +type MsgServer interface { + Send(context.Context, *MsgSend) (*MsgSendResponse, error) +} +``` + +[ADR 021](/sdk/v0.50/build/architecture/adr-021-protobuf-query-encoding) and [ADR 31](/sdk/v0.50/build/architecture/adr-031-msg-service) specifies how modules can implement the generated `QueryServer` +and `MsgServer` interfaces as replacements for the legacy queriers and `Msg` handlers respectively. + +In this ADR we explain how modules can make queries and send `Msg`s to other modules using the generated `QueryClient` +and `MsgClient` interfaces and propose this mechanism as a replacement for the existing `Keeper` paradigm. To be clear, +this ADR does not necessitate the creation of new protobuf definitions or services. Rather, it leverages the same proto +based service interfaces already used by clients for inter-module communication. + +Using this `QueryClient`/`MsgClient` approach has the following key benefits over exposing keepers to external modules: + +1. Protobuf types are checked for breaking changes using [buf](https://buf.build/docs/breaking-overview) and because of + the way protobuf is designed this will give us strong backwards compatibility guarantees while allowing for forward + evolution. +2. The separation between the client and server interfaces will allow us to insert permission checking code in between + the two which checks if one module is authorized to send the specified `Msg` to the other module providing a proper + object capability system (see below). +3. The router for inter-module communication gives us a convenient place to handle rollback of transactions, + enabling atomicy of operations ([currently a problem](https://github.com/cosmos/cosmos-sdk/issues/8030)). Any failure within a module-to-module call would result in a failure of the entire + transaction + +This mechanism has the added benefits of: + +* reducing boilerplate through code generation, and +* allowing for modules in other languages either via a VM like CosmWasm or sub-processes using gRPC + +### Inter-module Communication + +To use the `Client` generated by the protobuf compiler we need a `grpc.ClientConn` [interface](https://github.com/grpc/grpc-go/blob/v1.49.x/clientconn.go#L441-L450) +implementation. For this we introduce +a new type, `ModuleKey`, which implements the `grpc.ClientConn` interface. `ModuleKey` can be thought of as the "private +key" corresponding to a module account, where authentication is provided through use of a special `Invoker()` function, +described in more detail below. + +Blockchain users (external clients) use their account's private key to sign transactions containing `Msg`s where they are listed as signers (each +message specifies required signers with `Msg.GetSigner`). The authentication checks is performed by `AnteHandler`. + +Here, we extend this process, by allowing modules to be identified in `Msg.GetSigners`. When a module wants to trigger the execution a `Msg` in another module, +its `ModuleKey` acts as the sender (through the `ClientConn` interface we describe below) and is set as a sole "signer". It's worth to note +that we don't use any cryptographic signature in this case. +For example, module `A` could use its `A.ModuleKey` to create `MsgSend` object for `/cosmos.bank.Msg/Send` transaction. `MsgSend` validation +will assure that the `from` account (`A.ModuleKey` in this case) is the signer. + +Here's an example of a hypothetical module `foo` interacting with `x/bank`: + +```go expandable +package foo + +type FooMsgServer { + // ... + + bankQuery bank.QueryClient + bankMsg bank.MsgClient +} + +func NewFooMsgServer(moduleKey RootModuleKey, ...) + +FooMsgServer { + // ... + + return FooMsgServer { + // ... + modouleKey: moduleKey, + bankQuery: bank.NewQueryClient(moduleKey), + bankMsg: bank.NewMsgClient(moduleKey), +} +} + +func (foo *FooMsgServer) + +Bar(ctx context.Context, req *MsgBarRequest) (*MsgBarResponse, error) { + balance, err := foo.bankQuery.Balance(&bank.QueryBalanceRequest{ + Address: fooMsgServer.moduleKey.Address(), + Denom: "foo" +}) + + ... + + res, err := foo.bankMsg.Send(ctx, &bank.MsgSendRequest{ + FromAddress: fooMsgServer.moduleKey.Address(), ... +}) + + ... +} +``` + +This design is also intended to be extensible to cover use cases of more fine grained permissioning like minting by +denom prefix being restricted to certain modules (as discussed in +[#7459](https://github.com/cosmos/cosmos-sdk/pull/7459#discussion_r529545528)). + +### `ModuleKey`s and `ModuleID`s + +A `ModuleKey` can be thought of as a "private key" for a module account and a `ModuleID` can be thought of as the +corresponding "public key". From the [ADR 028](/sdk/v0.50/build/architecture/adr-028-public-key-addresses), modules can have both a root module account and any number of sub-accounts +or derived accounts that can be used for different pools (ex. staking pools) or managed accounts (ex. group +accounts). We can also think of module sub-accounts as similar to derived keys - there is a root key and then some +derivation path. `ModuleID` is a simple struct which contains the module name and optional "derivation" path, +and forms its address based on the `AddressHash` method from [the ADR-028](https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-028-public-key-addresses.md): + +```go +type ModuleID struct { + ModuleName string + Path []byte +} + +func (key ModuleID) + +Address() []byte { + return AddressHash(key.ModuleName, key.Path) +} +``` + +In addition to being able to generate a `ModuleID` and address, a `ModuleKey` contains a special function called +`Invoker` which is the key to safe inter-module access. The `Invoker` creates an `InvokeFn` closure which is used as an `Invoke` method in +the `grpc.ClientConn` interface and under the hood is able to route messages to the appropriate `Msg` and `Query` handlers +performing appropriate security checks on `Msg`s. This allows for even safer inter-module access than keeper's whose +private member variables could be manipulated through reflection. Golang does not support reflection on a function +closure's captured variables and direct manipulation of memory would be needed for a truly malicious module to bypass +the `ModuleKey` security. + +The two `ModuleKey` types are `RootModuleKey` and `DerivedModuleKey`: + +```go expandable +type Invoker func(callInfo CallInfo) + +func(ctx context.Context, request, response interface{ +}, opts ...interface{ +}) + +error + +type CallInfo { + Method string + Caller ModuleID +} + +type RootModuleKey struct { + moduleName string + invoker Invoker +} + +func (rm RootModuleKey) + +Derive(path []byte) + +DerivedModuleKey { /* ... */ +} + +type DerivedModuleKey struct { + moduleName string + path []byte + invoker Invoker +} +``` + +A module can get access to a `DerivedModuleKey`, using the `Derive(path []byte)` method on `RootModuleKey` and then +would use this key to authenticate `Msg`s from a sub-account. Ex: + +```go +package foo + +func (fooMsgServer *MsgServer) + +Bar(ctx context.Context, req *MsgBar) (*MsgBarResponse, error) { + derivedKey := fooMsgServer.moduleKey.Derive(req.SomePath) + bankMsgClient := bank.NewMsgClient(derivedKey) + +res, err := bankMsgClient.Balance(ctx, &bank.MsgSend{ + FromAddress: derivedKey.Address(), ... +}) + ... +} +``` + +In this way, a module can gain permissioned access to a root account and any number of sub-accounts and send +authenticated `Msg`s from these accounts. The `Invoker` `callInfo.Caller` parameter is used under the hood to +distinguish between different module accounts, but either way the function returned by `Invoker` only allows `Msg`s +from either the root or a derived module account to pass through. + +Note that `Invoker` itself returns a function closure based on the `CallInfo` passed in. This will allow client implementations +in the future that cache the invoke function for each method type avoiding the overhead of hash table lookup. +This would reduce the performance overhead of this inter-module communication method to the bare minimum required for +checking permissions. + +To re-iterate, the closure only allows access to authorized calls. There is no access to anything else regardless of any +name impersonation. + +Below is a rough sketch of the implementation of `grpc.ClientConn.Invoke` for `RootModuleKey`: + +```go +func (key RootModuleKey) + +Invoke(ctx context.Context, method string, args, reply interface{ +}, opts ...grpc.CallOption) + +error { + f := key.invoker(CallInfo { + Method: method, + Caller: ModuleID { + ModuleName: key.moduleName +}}) + +return f(ctx, args, reply) +} +``` + +### `AppModule` Wiring and Requirements + +In [ADR 031](/sdk/v0.50/build/architecture/adr-031-msg-service), the `AppModule.RegisterService(Configurator)` method was introduced. To support +inter-module communication, we extend the `Configurator` interface to pass in the `ModuleKey` and to allow modules to +specify their dependencies on other modules using `RequireServer()`: + +```go +type Configurator interface { + MsgServer() + +grpc.Server + QueryServer() + +grpc.Server + + ModuleKey() + +ModuleKey + RequireServer(msgServer interface{ +}) +} +``` + +The `ModuleKey` is passed to modules in the `RegisterService` method itself so that `RegisterServices` serves as a single +entry point for configuring module services. This is intended to also have the side-effect of greatly reducing boilerplate in +`app.go`. For now, `ModuleKey`s will be created based on `AppModuleBasic.Name()`, but a more flexible system may be +introduced in the future. The `ModuleManager` will handle creation of module accounts behind the scenes. + +Because modules do not get direct access to each other anymore, modules may have unfulfilled dependencies. To make sure +that module dependencies are resolved at startup, the `Configurator.RequireServer` method should be added. The `ModuleManager` +will make sure that all dependencies declared with `RequireServer` can be resolved before the app starts. An example +module `foo` could declare it's dependency on `x/bank` like this: + +```go +package foo + +func (am AppModule) + +RegisterServices(cfg Configurator) { + cfg.RequireServer((*bank.QueryServer)(nil)) + +cfg.RequireServer((*bank.MsgServer)(nil)) +} +``` + +### Security Considerations + +In addition to checking for `ModuleKey` permissions, a few additional security precautions will need to be taken by +the underlying router infrastructure. + +#### Recursion and Re-entry + +Recursive or re-entrant method invocations pose a potential security threat. This can be a problem if Module A +calls Module B and Module B calls module A again in the same call. + +One basic way for the router system to deal with this is to maintain a call stack which prevents a module from +being referenced more than once in the call stack so that there is no re-entry. A `map[string]interface{}` table +in the router could be used to perform this security check. + +#### Queries + +Queries in Cosmos SDK are generally un-permissioned so allowing one module to query another module should not pose +any major security threats assuming basic precautions are taken. The basic precaution that the router system will +need to take is making sure that the `sdk.Context` passed to query methods does not allow writing to the store. This +can be done for now with a `CacheMultiStore` as is currently done for `BaseApp` queries. + +### Internal Methods + +In many cases, we may wish for modules to call methods on other modules which are not exposed to clients at all. For this +purpose, we add the `InternalServer` method to `Configurator`: + +```go +type Configurator interface { + MsgServer() + +grpc.Server + QueryServer() + +grpc.Server + InternalServer() + +grpc.Server +} +``` + +As an example, x/slashing's Slash must call x/staking's Slash, but we don't want to expose x/staking's Slash to end users +and clients. + +Internal protobuf services will be defined in a corresponding `internal.proto` file in the given module's +proto package. + +Services registered against `InternalServer` will be callable from other modules but not by external clients. + +An alternative solution to internal-only methods could involve hooks / plugins as discussed [here](https://github.com/cosmos/cosmos-sdk/pull/7459#issuecomment-733807753). +A more detailed evaluation of a hooks / plugin system will be addressed later in follow-ups to this ADR or as a separate +ADR. + +### Authorization + +By default, the inter-module router requires that messages are sent by the first signer returned by `GetSigners`. The +inter-module router should also accept authorization middleware such as that provided by [ADR 030](https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-030-authz-module.md). +This middleware will allow accounts to otherwise specific module accounts to perform actions on their behalf. +Authorization middleware should take into account the need to grant certain modules effectively "admin" privileges to +other modules. This will be addressed in separate ADRs or updates to this ADR. + +### Future Work + +Other future improvements may include: + +* custom code generation that: + * simplifies interfaces (ex. generates code with `sdk.Context` instead of `context.Context`) + * optimizes inter-module calls - for instance caching resolved methods after first invocation +* combining `StoreKey`s and `ModuleKey`s into a single interface so that modules have a single OCAPs handle +* code generation which makes inter-module communication more performant +* decoupling `ModuleKey` creation from `AppModuleBasic.Name()` so that app's can override root module account names +* inter-module hooks and plugins + +## Alternatives + +### MsgServices vs `x/capability` + +The `x/capability` module does provide a proper object-capability implementation that can be used by any module in the +Cosmos SDK and could even be used for inter-module OCAPs as described in [#5931](https://github.com/cosmos/cosmos-sdk/issues/5931). + +The advantages of the approach described in this ADR are mostly around how it integrates with other parts of the Cosmos SDK, +specifically: + +* protobuf so that: + * code generation of interfaces can be leveraged for a better dev UX + * module interfaces are versioned and checked for breakage using [buf](https://docs.buf.build/breaking-overview) +* sub-module accounts as per ADR 028 +* the general `Msg` passing paradigm and the way signers are specified by `GetSigners` + +Also, this is a complete replacement for keepers and could be applied to *all* inter-module communication whereas the +`x/capability` approach in #5931 would need to be applied method by method. + +## Consequences + +### Backwards Compatibility + +This ADR is intended to provide a pathway to a scenario where there is greater long term compatibility between modules. +In the short-term, this will likely result in breaking certain `Keeper` interfaces which are too permissive and/or +replacing `Keeper` interfaces altogether. + +### Positive + +* an alternative to keepers which can more easily lead to stable inter-module interfaces +* proper inter-module OCAPs +* improved module developer DevX, as commented on by several particpants on + [Architecture Review Call, Dec 3](https://hackmd.io/E0wxxOvRQ5qVmTf6N_k84Q) +* lays the groundwork for what can be a greatly simplified `app.go` +* router can be setup to enforce atomic transactions for module-to-module calls + +### Negative + +* modules which adopt this will need significant refactoring + +### Neutral + +## Test Cases \[optional] + +## References + +* [ADR 021](/sdk/v0.50/build/architecture/adr-021-protobuf-query-encoding) +* [ADR 031](/sdk/v0.50/build/architecture/adr-031-msg-service) +* [ADR 028](/sdk/v0.50/build/architecture/adr-028-public-key-addresses) +* [ADR 030 draft](https://github.com/cosmos/cosmos-sdk/pull/7105) +* [Object-Capability Model](/sdk/v0.54/guides/module-design/ocap) diff --git a/sdk/v0.54/reference/architecture/adr-034-account-rekeying.mdx b/sdk/v0.54/reference/architecture/adr-034-account-rekeying.mdx new file mode 100644 index 000000000..7e0f735d9 --- /dev/null +++ b/sdk/v0.54/reference/architecture/adr-034-account-rekeying.mdx @@ -0,0 +1,81 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/architecture/adr-034-account-rekeying' +title: 'ADR 034: Account Rekeying' +description: '30-09-2020: Initial Draft' +--- + +## Changelog + +* 30-09-2020: Initial Draft + +## Status + +PROPOSED + +## Abstract + +Account rekeying is a process hat allows an account to replace its authentication pubkey with a new one. + +## Context + +Currently, in the Cosmos SDK, the address of an auth `BaseAccount` is based on the hash of the public key. Once an account is created, the public key for the account is set in stone, and cannot be changed. This can be a problem for users, as key rotation is a useful security practice, but is not possible currently. Furthermore, as multisigs are a type of pubkey, once a multisig for an account is set, it can not be updated. This is problematic, as multisigs are often used by organizations or companies, who may need to change their set of multisig signers for internal reasons. + +Transferring all the assets of an account to a new account with the updated pubkey is not sufficient, because some "engagements" of an account are not easily transferable. For example, in staking, to transfer bonded Atoms, an account would have to unbond all delegations and wait the three week unbonding period. Even more significantly, for validator operators, ownership over a validator is not transferrable at all, meaning that the operator key for a validator can never be updated, leading to poor operational security for validators. + +## Decision + +We propose the addition of a new feature to `x/auth` that allows accounts to update the public key associated with their account, while keeping the address the same. + +This is possible because the Cosmos SDK `BaseAccount` stores the public key for an account in state, instead of making the assumption that the public key is included in the transaction (whether explicitly or implicitly through the signature) as in other blockchains such as Bitcoin and Ethereum. Because the public key is stored on chain, it is okay for the public key to not hash to the address of an account, as the address is not pertinent to the signature checking process. + +To build this system, we design a new Msg type as follows: + +```protobuf +service Msg { + rpc ChangePubKey(MsgChangePubKey) returns (MsgChangePubKeyResponse); +} + +message MsgChangePubKey { + string address = 1; + google.protobuf.Any pub_key = 2; +} + +message MsgChangePubKeyResponse {} +``` + +The MsgChangePubKey transaction needs to be signed by the existing pubkey in state. + +Once, approved, the handler for this message type, which takes in the AccountKeeper, will update the in-state pubkey for the account and replace it with the pubkey from the Msg. + +An account that has had its pubkey changed cannot be automatically pruned from state. This is because if pruned, the original pubkey of the account would be needed to recreate the same address, but the owner of the address may not have the original pubkey anymore. Currently, we do not automatically prune any accounts anyways, but we would like to keep this option open the road (this is the purpose of account numbers). To resolve this, we charge an additional gas fee for this operation to compensate for this this externality (this bound gas amount is configured as parameter `PubKeyChangeCost`). The bonus gas is charged inside the handler, using the `ConsumeGas` function. Furthermore, in the future, we can allow accounts that have rekeyed manually prune themselves using a new Msg type such as `MsgDeleteAccount`. Manually pruning accounts can give a gas refund as an incentive for performing the action. + +```go +amount := ak.GetParams(ctx).PubKeyChangeCost + ctx.GasMeter().ConsumeGas(amount, "pubkey change fee") +``` + +Every time a key for an address is changed, we will store a log of this change in the state of the chain, thus creating a stack of all previous keys for an address and the time intervals for which they were active. This allows dapps and clients to easily query past keys for an account which may be useful for features such as verifying timestamped off-chain signed messages. + +## Consequences + +### Positive + +* Will allow users and validator operators to employ better operational security practices with key rotation. +* Will allow organizations or groups to easily change and add/remove multisig signers. + +### Negative + +Breaks the current assumed relationship between address and pubkeys as H(pubkey) = address. This has a couple of consequences. + +* This makes wallets that support this feature more complicated. For example, if an address on chain was updated, the corresponding key in the CLI wallet also needs to be updated. +* Cannot automatically prune accounts with 0 balance that have had their pubkey changed. + +### Neutral + +* While the purpose of this is intended to allow the owner of an account to update to a new pubkey they own, this could technically also be used to transfer ownership of an account to a new owner. For example, this could be use used to sell a staked position without unbonding or an account that has vesting tokens. However, the friction of this is very high as this would essentially have to be done as a very specific OTC trade. Furthermore, additional constraints could be added to prevent accouns with Vesting tokens to use this feature. +* Will require that PubKeys for an account are included in the genesis exports. + +## References + +* [Link](https://www.algorand.com/resources/blog/announcing-rekeying) diff --git a/sdk/v0.54/reference/architecture/adr-035-rosetta-api-support.mdx b/sdk/v0.54/reference/architecture/adr-035-rosetta-api-support.mdx new file mode 100644 index 000000000..2c7254500 --- /dev/null +++ b/sdk/v0.54/reference/architecture/adr-035-rosetta-api-support.mdx @@ -0,0 +1,229 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/architecture/adr-035-rosetta-api-support' +title: 'ADR 035: Rosetta API Support' +description: >- + Jonathan Gimeno (@jgimeno) David Grierson (@senormonito) Alessio Treglia + (@alessio) Frojdy Dymylja (@fdymylja) +--- + +## Authors + +* Jonathan Gimeno (@jgimeno) +* David Grierson (@senormonito) +* Alessio Treglia (@alessio) +* Frojdy Dymylja (@fdymylja) + +## Changelog + +* 2021-05-12: the external library [cosmos-rosetta-gateway](https://github.com/tendermint/cosmos-rosetta-gateway) has been moved within the Cosmos SDK. + +## Context + +[Rosetta API](https://www.rosetta-api.org/) is an open-source specification and set of tools developed by Coinbase to +standardise blockchain interactions. + +Through the use of a standard API for integrating blockchain applications it will + +* Be easier for a user to interact with a given blockchain +* Allow exchanges to integrate new blockchains quickly and easily +* Enable application developers to build cross-blockchain applications such as block explorers, wallets and dApps at + considerably lower cost and effort. + +## Decision + +It is clear that adding Rosetta API support to the Cosmos SDK will bring value to all the developers and +Cosmos SDK based chains in the ecosystem. How it is implemented is key. + +The driving principles of the proposed design are: + +1. **Extensibility:** it must be as riskless and painless as possible for application developers to set-up network + configurations to expose Rosetta API-compliant services. +2. **Long term support:** This proposal aims to provide support for all the supported Cosmos SDK release series. +3. **Cost-efficiency:** Backporting changes to Rosetta API specifications from `master` to the various stable + branches of Cosmos SDK is a cost that needs to be reduced. + +We will achieve these delivering on these principles by the following: + +1. There will be a package `rosetta/lib` + for the implementation of the core Rosetta API features, particularly: + a. The types and interfaces (`Client`, `OfflineClient`...), this separates design from implementation detail. + b. The `Server` functionality as this is independent of the Cosmos SDK version. + c. The `Online/OfflineNetwork`, which is not exported, and implements the rosetta API using the `Client` interface to query the node, build tx and so on. + d. The `errors` package to extend rosetta errors. +2. Due to differences between the Cosmos release series, each series will have its own specific implementation of `Client` interface. +3. There will be two options for starting an API service in applications: + a. API shares the application process + b. API-specific process. + +## Architecture + +### The External Repo + +As section will describe the proposed external library, including the service implementation, plus the defined types and interfaces. + +#### Server + +`Server` is a simple `struct` that is started and listens to the port specified in the settings. This is meant to be used across all the Cosmos SDK versions that are actively supported. + +The constructor follows: + +`func NewServer(settings Settings) (Server, error)` + +`Settings`, which are used to construct a new server, are the following: + +```go expandable +// Settings define the rosetta server settings +type Settings struct { + // Network contains the information regarding the network + Network *types.NetworkIdentifier + // Client is the online API handler + Client crgtypes.Client + // Listen is the address the handler will listen at + Listen string + // Offline defines if the rosetta service should be exposed in offline mode + Offline bool + // Retries is the number of readiness checks that will be attempted when instantiating the handler + // valid only for online API + Retries int + // RetryWait is the time that will be waited between retries + RetryWait time.Duration +} +``` + +#### Types + +Package types uses a mixture of rosetta types and custom defined type wrappers, that the client must parse and return while executing operations. + +##### Interfaces + +Every SDK version uses a different format to connect (rpc, gRPC, etc), query and build transactions, we have abstracted this in what is the `Client` interface. +The client uses rosetta types, while the `Online/OfflineNetwork` takes care of returning correctly parsed rosetta responses and errors. + +Each Cosmos SDK release series will have their own `Client` implementations. +Developers can implement their own custom `Client`s as required. + +```go expandable +// Client defines the API the client implementation should provide. +type Client interface { + // Needed if the client needs to perform some action before connecting. + Bootstrap() + +error + // Ready checks if the servicer constraints for queries are satisfied + // for example the node might still not be ready, it's useful in process + // when the rosetta instance might come up before the node itself + // the servicer must return nil if the node is ready + Ready() + +error + + // Data API + + // Balances fetches the balance of the given address + // if height is not nil, then the balance will be displayed + // at the provided height, otherwise last block balance will be returned + Balances(ctx context.Context, addr string, height *int64) ([]*types.Amount, error) + // BlockByHashAlt gets a block and its transaction at the provided height + BlockByHash(ctx context.Context, hash string) (BlockResponse, error) + // BlockByHeightAlt gets a block given its height, if height is nil then last block is returned + BlockByHeight(ctx context.Context, height *int64) (BlockResponse, error) + // BlockTransactionsByHash gets the block, parent block and transactions + // given the block hash. + BlockTransactionsByHash(ctx context.Context, hash string) (BlockTransactionsResponse, error) + // BlockTransactionsByHash gets the block, parent block and transactions + // given the block hash. + BlockTransactionsByHeight(ctx context.Context, height *int64) (BlockTransactionsResponse, error) + // GetTx gets a transaction given its hash + GetTx(ctx context.Context, hash string) (*types.Transaction, error) + // GetUnconfirmedTx gets an unconfirmed Tx given its hash + // NOTE(fdymylja): NOT IMPLEMENTED YET! + GetUnconfirmedTx(ctx context.Context, hash string) (*types.Transaction, error) + // Mempool returns the list of the current non confirmed transactions + Mempool(ctx context.Context) ([]*types.TransactionIdentifier, error) + // Peers gets the peers currently connected to the node + Peers(ctx context.Context) ([]*types.Peer, error) + // Status returns the node status, such as sync data, version etc + Status(ctx context.Context) (*types.SyncStatus, error) + + // Construction API + + // PostTx posts txBytes to the node and returns the transaction identifier plus metadata related + // to the transaction itself. + PostTx(txBytes []byte) (res *types.TransactionIdentifier, meta map[string]interface{ +}, err error) + // ConstructionMetadataFromOptions + ConstructionMetadataFromOptions(ctx context.Context, options map[string]interface{ +}) (meta map[string]interface{ +}, err error) + +OfflineClient +} + +// OfflineClient defines the functionalities supported without having access to the node +type OfflineClient interface { + NetworkInformationProvider + // SignedTx returns the signed transaction given the tx bytes (msgs) + +plus the signatures + SignedTx(ctx context.Context, txBytes []byte, sigs []*types.Signature) (signedTxBytes []byte, err error) + // TxOperationsAndSignersAccountIdentifiers returns the operations related to a transaction and the account + // identifiers if the transaction is signed + TxOperationsAndSignersAccountIdentifiers(signed bool, hexBytes []byte) (ops []*types.Operation, signers []*types.AccountIdentifier, err error) + // ConstructionPayload returns the construction payload given the request + ConstructionPayload(ctx context.Context, req *types.ConstructionPayloadsRequest) (resp *types.ConstructionPayloadsResponse, err error) + // PreprocessOperationsToOptions returns the options given the preprocess operations + PreprocessOperationsToOptions(ctx context.Context, req *types.ConstructionPreprocessRequest) (options map[string]interface{ +}, err error) + // AccountIdentifierFromPublicKey returns the account identifier given the public key + AccountIdentifierFromPublicKey(pubKey *types.PublicKey) (*types.AccountIdentifier, error) +} +``` + +### 2. Cosmos SDK Implementation + +The Cosmos SDK implementation, based on version, takes care of satisfying the `Client` interface. +In Stargate, Launchpad and 0.37, we have introduced the concept of rosetta.Msg, this message is not in the shared repository as the sdk.Msg type differs between Cosmos SDK versions. + +The rosetta.Msg interface follows: + +```go +// Msg represents a cosmos-sdk message that can be converted from and to a rosetta operation. +type Msg interface { + sdk.Msg + ToOperations(withStatus, hasError bool) []*types.Operation + FromOperations(ops []*types.Operation) (sdk.Msg, error) +} +``` + +Hence developers who want to extend the rosetta set of supported operations just need to extend their module's sdk.Msgs with the `ToOperations` and `FromOperations` methods. + +### 3. API service invocation + +As stated at the start, application developers will have two methods for invocation of the Rosetta API service: + +1. Shared process for both application and API +2. Standalone API service + +#### Shared Process (Only Stargate) + +Rosetta API service could run within the same execution process as the application. This would be enabled via app.toml settings, and if gRPC is not enabled the rosetta instance would be spinned in offline mode (tx building capabilities only). + +#### Separate API service + +Client application developers can write a new command to launch a Rosetta API server as a separate process too, using the rosetta command contained in the `/server/rosetta` package. Construction of the command depends on Cosmos SDK version. Examples can be found inside `simd` for stargate, and `contrib/rosetta/simapp` for other release series. + +## Status + +Proposed + +## Consequences + +### Positive + +* Out-of-the-box Rosetta API support within Cosmos SDK. +* Blockchain interface standardisation + +## References + +* [Link](https://www.rosetta-api.org/) diff --git a/sdk/v0.54/reference/architecture/adr-036-arbitrary-signature.mdx b/sdk/v0.54/reference/architecture/adr-036-arbitrary-signature.mdx new file mode 100644 index 000000000..6d5e7de42 --- /dev/null +++ b/sdk/v0.54/reference/architecture/adr-036-arbitrary-signature.mdx @@ -0,0 +1,137 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/architecture/adr-036-arbitrary-signature' +title: 'ADR 036: Arbitrary Message Signature Specification' +description: 28/10/2020 - Initial draft +--- + +## Changelog + +* 28/10/2020 - Initial draft + +## Authors + +* Antoine Herzog (@antoineherzog) +* Zaki Manian (@zmanian) +* Aleksandr Bezobchuk (alexanderbez) \[1] +* Frojdi Dymylja (@fdymylja) + +## Status + +Draft + +## Abstract + +Currently, in the Cosmos SDK, there is no convention to sign arbitrary message like on Ethereum. We propose with this specification, for Cosmos SDK ecosystem, a way to sign and validate off-chain arbitrary messages. + +This specification serves the purpose of covering every use case, this means that cosmos-sdk applications developers decide how to serialize and represent `Data` to users. + +## Context + +Having the ability to sign messages off-chain has proven to be a fundamental aspect of nearly any blockchain. The notion of signing messages off-chain has many added benefits such as saving on computational costs and reducing transaction throughput and overhead. Within the context of the Cosmos, some of the major applications of signing such data includes, but is not limited to, providing a cryptographic secure and verifiable means of proving validator identity and possibly associating it with some other framework or organization. In addition, having the ability to sign Cosmos messages with a Ledger or similar HSM device. + +Further context and use cases can be found in the references links. + +## Decision + +The aim is being able to sign arbitrary messages, even using Ledger or similar HSM devices. + +As a result signed messages should look roughly like Cosmos SDK messages but **must not** be a valid on-chain transaction. `chain-id`, `account_number` and `sequence` can all be assigned invalid values. + +Cosmos SDK 0.40 also introduces a concept of “auth\_info” this can specify SIGN\_MODES. + +A spec should include an `auth_info` that supports SIGN\_MODE\_DIRECT and SIGN\_MODE\_LEGACY\_AMINO. + +Create the `offchain` proto definitions, we extend the auth module with `offchain` package to offer functionalities to verify and sign offline messages. + +An offchain transaction follows these rules: + +* the memo must be empty +* nonce, sequence number must be equal to 0 +* chain-id must be equal to “” +* fee gas must be equal to 0 +* fee amount must be an empty array + +Verification of an offchain transaction follows the same rules as an onchain one, except for the spec differences highlighted above. + +The first message added to the `offchain` package is `MsgSignData`. + +`MsgSignData` allows developers to sign arbitrary bytes valid offchain only. Where `Signer` is the account address of the signer. `Data` is arbitrary bytes which can represent `text`, `files`, `object`s. It's applications developers decision how `Data` should be deserialized, serialized and the object it can represent in their context. + +It's applications developers decision how `Data` should be treated, by treated we mean the serialization and deserialization process and the Object `Data` should represent. + +Proto definition: + +```protobuf +// MsgSignData defines an arbitrary, general-purpose, off-chain message +message MsgSignData { + // Signer is the sdk.AccAddress of the message signer + bytes Signer = 1 [(gogoproto.jsontag) = "signer", (gogoproto.casttype) = "github.com/cosmos/cosmos-sdk/types.AccAddress"]; + // Data represents the raw bytes of the content that is signed (text, json, etc) + bytes Data = 2 [(gogoproto.jsontag) = "data"]; +} +``` + +Signed MsgSignData json example: + +```json expandable +{ + "type": "cosmos-sdk/StdTx", + "value": { + "msg": [ + { + "type": "sign/MsgSignData", + "value": { + "signer": "cosmos1hftz5ugqmpg9243xeegsqqav62f8hnywsjr4xr", + "data": "cmFuZG9t" + } + } + ], + "fee": { + "amount": [], + "gas": "0" + }, + "signatures": [ + { + "pub_key": { + "type": "tendermint/PubKeySecp256k1", + "value": "AqnDSiRoFmTPfq97xxEb2VkQ/Hm28cPsqsZm9jEVsYK9" + }, + "signature": "8y8i34qJakkjse9pOD2De+dnlc4KvFgh0wQpes4eydN66D9kv7cmCEouRrkka9tlW9cAkIL52ErB+6ye7X5aEg==" + } + ], + "memo": "" + } +} +``` + +## Consequences + +There is a specification on how messages, that are not meant to be broadcast to a live chain, should be formed. + +### Backwards Compatibility + +Backwards compatibility is maintained as this is a new message spec definition. + +### Positive + +* A common format that can be used by multiple applications to sign and verify off-chain messages. +* The specification is primitive which means it can cover every use case without limiting what is possible to fit inside it. +* It gives room for other off-chain messages specifications that aim to target more specific and common use cases such as off-chain-based authN/authZ layers \[2]. + +### Negative + +* Current proposal requires a fixed relationship between an account address and a public key. +* Doesn't work with multisig accounts. + +## Further discussion + +* Regarding security in `MsgSignData`, the developer using `MsgSignData` is in charge of making the content laying in `Data` non-replayable when, and if, needed. +* the offchain package will be further extended with extra messages that target specific use cases such as, but not limited to, authentication in applications, payment channels, L2 solutions in general. + +## References + +1. [Link](https://github.com/cosmos/ics/pull/33) +2. [Link](https://github.com/cosmos/cosmos-sdk/pull/7727#discussion_r515668204) +3. [Link](https://github.com/cosmos/cosmos-sdk/pull/7727#issuecomment-722478477) +4. [Link](https://github.com/cosmos/cosmos-sdk/pull/7727#issuecomment-721062923) diff --git a/sdk/v0.54/reference/architecture/adr-037-gov-split-vote.mdx b/sdk/v0.54/reference/architecture/adr-037-gov-split-vote.mdx new file mode 100644 index 000000000..3b0eab95e --- /dev/null +++ b/sdk/v0.54/reference/architecture/adr-037-gov-split-vote.mdx @@ -0,0 +1,116 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/architecture/adr-037-gov-split-vote' +title: 'ADR 037: Governance split votes' +description: '2020/10/28: Intial draft' +--- + +## Changelog + +* 2020/10/28: Intial draft + +## Status + +Accepted + +## Abstract + +This ADR defines a modification to the governance module that would allow a staker to split their votes into several voting options. For example, it could use 70% of its voting power to vote Yes and 30% of its voting power to vote No. + +## Context + +Currently, an address can cast a vote with only one options (Yes/No/Abstain/NoWithVeto) and use their full voting power behind that choice. + +However, often times the entity owning that address might not be a single individual. For example, a company might have different stakeholders who want to vote differently, and so it makes sense to allow them to split their voting power. Another example use case is exchanges. Many centralized exchanges often stake a portion of their users' tokens in their custody. Currently, it is not possible for them to do "passthrough voting" and giving their users voting rights over their tokens. However, with this system, exchanges can poll their users for voting preferences, and then vote on-chain proportionally to the results of the poll. + +## Decision + +We modify the vote structs to be + +```go +type WeightedVoteOption struct { + Option string + Weight sdk.Dec +} + +type Vote struct { + ProposalID int64 + Voter sdk.Address + Options []WeightedVoteOption +} +``` + +And for backwards compatibility, we introduce `MsgVoteWeighted` while keeping `MsgVote`. + +```go expandable +type MsgVote struct { + ProposalID int64 + Voter sdk.Address + Option Option +} + +type MsgVoteWeighted struct { + ProposalID int64 + Voter sdk.Address + Options []WeightedVoteOption +} +``` + +The `ValidateBasic` of a `MsgVoteWeighted` struct would require that + +1. The sum of all the Rates is equal to 1.0 +2. No Option is repeated + +The governance tally function will iterate over all the options in a vote and add to the tally the result of the voter's voting power \* the rate for that option. + +```go +tally() { + results := map[types.VoteOption]sdk.Dec + for _, vote := range votes { + for i, weightedOption := range vote.Options { + results[weightedOption.Option] += getVotingPower(vote.voter) * weightedOption.Weight +} + +} +} +``` + +The CLI command for creating a multi-option vote would be as such: + +```shell +simd tx gov vote 1 "yes=0.6,no=0.3,abstain=0.05,no_with_veto=0.05" --from mykey +``` + +To create a single-option vote a user can do either + +```shell +simd tx gov vote 1 "yes=1" --from mykey +``` + +or + +```shell +simd tx gov vote 1 yes --from mykey +``` + +to maintain backwards compatibility. + +## Consequences + +### Backwards Compatibility + +* Previous VoteMsg types will remain the same and so clients will not have to update their procedure unless they want to support the WeightedVoteMsg feature. +* When querying a Vote struct from state, its structure will be different, and so clients wanting to display all voters and their respective votes will have to handle the new format and the fact that a single voter can have split votes. +* The result of querying the tally function should have the same API for clients. + +### Positive + +* Can make the voting process more accurate for addresses representing multiple stakeholders, often some of the largest addresses. + +### Negative + +* Is more complex than simple voting, and so may be harder to explain to users. However, this is mostly mitigated because the feature is opt-in. + +### Neutral + +* Relatively minor change to governance tally function. diff --git a/sdk/v0.54/reference/architecture/adr-038-state-listening.mdx b/sdk/v0.54/reference/architecture/adr-038-state-listening.mdx new file mode 100644 index 000000000..90d83bd57 --- /dev/null +++ b/sdk/v0.54/reference/architecture/adr-038-state-listening.mdx @@ -0,0 +1,860 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/architecture/adr-038-state-listening' +title: 'ADR 038: KVStore state listening' +--- + +## Changelog + +* 11/23/2020: Initial draft +* 10/06/2022: Introduce plugin system based on hashicorp/go-plugin +* 10/14/2022: + * Add `ListenCommit`, flatten the state writes in a block to a single batch. + * Remove listeners from cache stores, should only listen to `rootmulti.Store`. + * Remove `HaltAppOnDeliveryError()`, the errors are propagated by default, the implementations should return nil if don't want to propogate errors. +* 26/05/2023: Update with ABCI 2.0 + +## Status + +Proposed + +## Abstract + +This ADR defines a set of changes to enable listening to state changes of individual KVStores and exposing these data to consumers. + +## Context + +Currently, KVStore data can be remotely accessed through [Queries](https://github.com/cosmos/cosmos-sdk/blob/master/docs/building-modules/messages-and-queries.md#queries) +which proceed either through Tendermint and the ABCI, or through the gRPC server. +In addition to these request/response queries, it would be beneficial to have a means of listening to state changes as they occur in real time. + +## Decision + +We will modify the `CommitMultiStore` interface and its concrete (`rootmulti`) implementations and introduce a new `listenkv.Store` to allow listening to state changes in underlying KVStores. We don't need to listen to cache stores, because we can't be sure that the writes will be committed eventually, and the writes are duplicated in `rootmulti.Store` eventually, so we should only listen to `rootmulti.Store`. +We will introduce a plugin system for configuring and running streaming services that write these state changes and their surrounding ABCI message context to different destinations. + +### Listening + +In a new file, `store/types/listening.go`, we will create a `MemoryListener` struct for streaming out protobuf encoded KV pairs state changes from a KVStore. +The `MemoryListener` will be used internally by the concrete `rootmulti` implementation to collect state changes from KVStores. + +```go expandable +// MemoryListener listens to the state writes and accumulate the records in memory. +type MemoryListener struct { + stateCache []StoreKVPair +} + +// NewMemoryListener creates a listener that accumulate the state writes in memory. +func NewMemoryListener() *MemoryListener { + return &MemoryListener{ +} +} + +// OnWrite writes state change events to the internal cache +func (fl *MemoryListener) + +OnWrite(storeKey StoreKey, key []byte, value []byte, delete bool) { + fl.stateCache = append(fl.stateCache, StoreKVPair{ + StoreKey: storeKey.Name(), + Delete: delete, + Key: key, + Value: value, +}) +} + +// PopStateCache returns the current state caches and set to nil +func (fl *MemoryListener) + +PopStateCache() []StoreKVPair { + res := fl.stateCache + fl.stateCache = nil + return res +} +``` + +We will also define a protobuf type for the KV pairs. In addition to the key and value fields this message +will include the StoreKey for the originating KVStore so that we can collect information from separate KVStores and determine the source of each KV pair. + +```protobuf +message StoreKVPair { + optional string store_key = 1; // the store key for the KVStore this pair originates from + required bool set = 2; // true indicates a set operation, false indicates a delete operation + required bytes key = 3; + required bytes value = 4; +} +``` + +### ListenKVStore + +We will create a new `Store` type `listenkv.Store` that the `rootmulti` store will use to wrap a `KVStore` to enable state listening. +We will configure the `Store` with a `MemoryListener` which will collect state changes for output to specific destinations. + +```go expandable +// Store implements the KVStore interface with listening enabled. +// Operations are traced on each core KVStore call and written to any of the +// underlying listeners with the proper key and operation permissions +type Store struct { + parent types.KVStore + listener *types.MemoryListener + parentStoreKey types.StoreKey +} + +// NewStore returns a reference to a new traceKVStore given a parent +// KVStore implementation and a buffered writer. +func NewStore(parent types.KVStore, psk types.StoreKey, listener *types.MemoryListener) *Store { + return &Store{ + parent: parent, listener: listener, parentStoreKey: psk +} +} + +// Set implements the KVStore interface. It traces a write operation and +// delegates the Set call to the parent KVStore. +func (s *Store) + +Set(key []byte, value []byte) { + types.AssertValidKey(key) + +s.parent.Set(key, value) + +s.listener.OnWrite(s.parentStoreKey, key, value, false) +} + +// Delete implements the KVStore interface. It traces a write operation and +// delegates the Delete call to the parent KVStore. +func (s *Store) + +Delete(key []byte) { + s.parent.Delete(key) + +s.listener.OnWrite(s.parentStoreKey, key, nil, true) +} +``` + +### MultiStore interface updates + +We will update the `CommitMultiStore` interface to allow us to wrap a `Memorylistener` to a specific `KVStore`. +Note that the `MemoryListener` will be attached internally by the concrete `rootmulti` implementation. + +```go +type CommitMultiStore interface { + ... + + // AddListeners adds a listener for the KVStore belonging to the provided StoreKey + AddListeners(keys []StoreKey) + + // PopStateCache returns the accumulated state change messages from MemoryListener + PopStateCache() []StoreKVPair +} +``` + +### MultiStore implementation updates + +We will adjust the `rootmulti` `GetKVStore` method to wrap the returned `KVStore` with a `listenkv.Store` if listening is turned on for that `Store`. + +```go expandable +func (rs *Store) + +GetKVStore(key types.StoreKey) + +types.KVStore { + store := rs.stores[key].(types.KVStore) + if rs.TracingEnabled() { + store = tracekv.NewStore(store, rs.traceWriter, rs.traceContext) +} + if rs.ListeningEnabled(key) { + store = listenkv.NewStore(store, key, rs.listeners[key]) +} + +return store +} +``` + +We will implement `AddListeners` to manage KVStore listeners internally and implement `PopStateCache` +for a means of retrieving the current state. + +```go +// AddListeners adds state change listener for a specific KVStore +func (rs *Store) + +AddListeners(keys []types.StoreKey) { + listener := types.NewMemoryListener() + for i := range keys { + rs.listeners[keys[i]] = listener +} +} +``` + +```go +func (rs *Store) + +PopStateCache() []types.StoreKVPair { + var cache []types.StoreKVPair + for _, ls := range rs.listeners { + cache = append(cache, ls.PopStateCache()...) +} + +sort.SliceStable(cache, func(i, j int) + +bool { + return cache[i].StoreKey < cache[j].StoreKey +}) + +return cache +} +``` + +We will also adjust the `rootmulti` `CacheMultiStore` and `CacheMultiStoreWithVersion` methods to enable listening in +the cache layer. + +```go expandable +func (rs *Store) + +CacheMultiStore() + +types.CacheMultiStore { + stores := make(map[types.StoreKey]types.CacheWrapper) + for k, v := range rs.stores { + store := v.(types.KVStore) + // Wire the listenkv.Store to allow listeners to observe the writes from the cache store, + // set same listeners on cache store will observe duplicated writes. + if rs.ListeningEnabled(k) { + store = listenkv.NewStore(store, k, rs.listeners[k]) +} + +stores[k] = store +} + +return cachemulti.NewStore(rs.db, stores, rs.keysByName, rs.traceWriter, rs.getTracingContext()) +} +``` + +```go expandable +func (rs *Store) + +CacheMultiStoreWithVersion(version int64) (types.CacheMultiStore, error) { + // ... + + // Wire the listenkv.Store to allow listeners to observe the writes from the cache store, + // set same listeners on cache store will observe duplicated writes. + if rs.ListeningEnabled(key) { + cacheStore = listenkv.NewStore(cacheStore, key, rs.listeners[key]) +} + +cachedStores[key] = cacheStore +} + +return cachemulti.NewStore(rs.db, cachedStores, rs.keysByName, rs.traceWriter, rs.getTracingContext()), nil +} +``` + +### Exposing the data + +#### Streaming Service + +We will introduce a new `ABCIListener` interface that plugs into the BaseApp and relays ABCI requests and responses +so that the service can group the state changes with the ABCI requests. + +```go +// baseapp/streaming.go + +// ABCIListener is the interface that we're exposing as a streaming service. +type ABCIListener interface { + // ListenFinalizeBlock updates the streaming service with the latest FinalizeBlock messages + ListenFinalizeBlock(ctx context.Context, req abci.RequestFinalizeBlock, res abci.ResponseFinalizeBlock) + +error + // ListenCommit updates the steaming service with the latest Commit messages and state changes + ListenCommit(ctx context.Context, res abci.ResponseCommit, changeSet []*StoreKVPair) + +error +} +``` + +#### BaseApp Registration + +We will add a new method to the `BaseApp` to enable the registration of `StreamingService`s: + +```go +// SetStreamingService is used to set a streaming service into the BaseApp hooks and load the listeners into the multistore +func (app *BaseApp) + +SetStreamingService(s ABCIListener) { + // register the StreamingService within the BaseApp + // BaseApp will pass BeginBlock, DeliverTx, and EndBlock requests and responses to the streaming services to update their ABCI context + app.abciListeners = append(app.abciListeners, s) +} +``` + +We will add two new fields to the `BaseApp` struct: + +```go expandable +type BaseApp struct { + + ... + + // abciListenersAsync for determining if abciListeners will run asynchronously. + // When abciListenersAsync=false and stopNodeOnABCIListenerErr=false listeners will run synchronized but will not stop the node. + // When abciListenersAsync=true stopNodeOnABCIListenerErr will be ignored. + abciListenersAsync bool + + // stopNodeOnABCIListenerErr halts the node when ABCI streaming service listening results in an error. + // stopNodeOnABCIListenerErr=true must be paired with abciListenersAsync=false. + stopNodeOnABCIListenerErr bool +} +``` + +#### ABCI Event Hooks + +We will modify the `FinalizeBlock` and `Commit` methods to pass ABCI requests and responses +to any streaming service hooks registered with the `BaseApp`. + +```go expandable +func (app *BaseApp) + +FinalizeBlock(req abci.RequestFinalizeBlock) + +abci.ResponseFinalizeBlock { + var abciRes abci.ResponseFinalizeBlock + defer func() { + // call the streaming service hook with the FinalizeBlock messages + for _, abciListener := range app.abciListeners { + ctx := app.finalizeState.ctx + blockHeight := ctx.BlockHeight() + if app.abciListenersAsync { + go func(req abci.RequestFinalizeBlock, res abci.ResponseFinalizeBlock) { + if err := app.abciListener.FinalizeBlock(blockHeight, req, res); err != nil { + app.logger.Error("FinalizeBlock listening hook failed", "height", blockHeight, "err", err) +} + +}(req, abciRes) +} + +else { + if err := app.abciListener.ListenFinalizeBlock(blockHeight, req, res); err != nil { + app.logger.Error("FinalizeBlock listening hook failed", "height", blockHeight, "err", err) + if app.stopNodeOnABCIListenerErr { + os.Exit(1) +} + +} + +} + +} + +}() + + ... + + return abciRes +} +``` + +```go expandable +func (app *BaseApp) + +Commit() + +abci.ResponseCommit { + + ... + res := abci.ResponseCommit{ + Data: commitID.Hash, + RetainHeight: retainHeight, +} + + // call the streaming service hook with the Commit messages + for _, abciListener := range app.abciListeners { + ctx := app.deliverState.ctx + blockHeight := ctx.BlockHeight() + changeSet := app.cms.PopStateCache() + if app.abciListenersAsync { + go func(res abci.ResponseCommit, changeSet []store.StoreKVPair) { + if err := app.abciListener.ListenCommit(ctx, res, changeSet); err != nil { + app.logger.Error("ListenCommit listening hook failed", "height", blockHeight, "err", err) +} + +}(res, changeSet) +} + +else { + if err := app.abciListener.ListenCommit(ctx, res, changeSet); err != nil { + app.logger.Error("ListenCommit listening hook failed", "height", blockHeight, "err", err) + if app.stopNodeOnABCIListenerErr { + os.Exit(1) +} + +} + +} + +} + + ... + + return res +} +``` + +#### Go Plugin System + +We propose a plugin architecture to load and run `Streaming` plugins and other types of implementations. We will introduce a plugin +system over gRPC that is used to load and run Cosmos-SDK plugins. The plugin system uses [hashicorp/go-plugin](https://github.com/hashicorp/go-plugin). +Each plugin must have a struct that implements the `plugin.Plugin` interface and an `Impl` interface for processing messages over gRPC. +Each plugin must also have a message protocol defined for the gRPC service: + +```go expandable +// streaming/plugins/abci/{ + plugin_version +}/interface.go + +// Handshake is a common handshake that is shared by streaming and host. +// This prevents users from executing bad plugins or executing a plugin +// directory. It is a UX feature, not a security feature. +var Handshake = plugin.HandshakeConfig{ + ProtocolVersion: 1, + MagicCookieKey: "ABCI_LISTENER_PLUGIN", + MagicCookieValue: "ef78114d-7bdf-411c-868f-347c99a78345", +} + +// ListenerPlugin is the base struc for all kinds of go-plugin implementations +// It will be included in interfaces of different Plugins +type ABCIListenerPlugin struct { + // GRPCPlugin must still implement the Plugin interface + plugin.Plugin + // Concrete implementation, written in Go. This is only used for plugins + // that are written in Go. + Impl baseapp.ABCIListener +} + +func (p *ListenerGRPCPlugin) + +GRPCServer(_ *plugin.GRPCBroker, s *grpc.Server) + +error { + RegisterABCIListenerServiceServer(s, &GRPCServer{ + Impl: p.Impl +}) + +return nil +} + +func (p *ListenerGRPCPlugin) + +GRPCClient( + _ context.Context, + _ *plugin.GRPCBroker, + c *grpc.ClientConn, +) (interface{ +}, error) { + return &GRPCClient{ + client: NewABCIListenerServiceClient(c) +}, nil +} +``` + +The `plugin.Plugin` interface has two methods `Client` and `Server`. For our GRPC service these are `GRPCClient` and `GRPCServer` +The `Impl` field holds the concrete implementation of our `baseapp.ABCIListener` interface written in Go. +Note: this is only used for plugin implementations written in Go. + +The advantage of having such a plugin system is that within each plugin authors can define the message protocol in a way that fits their use case. +For example, when state change listening is desired, the `ABCIListener` message protocol can be defined as below (*for illustrative purposes only*). +When state change listening is not desired than `ListenCommit` can be omitted from the protocol. + +```protobuf expandable +syntax = "proto3"; + +... + +message Empty {} + +message ListenFinalizeBlockRequest { + RequestFinalizeBlock req = 1; + ResponseFinalizeBlock res = 2; +} +message ListenCommitRequest { + int64 block_height = 1; + ResponseCommit res = 2; + repeated StoreKVPair changeSet = 3; +} + +// plugin that listens to state changes +service ABCIListenerService { + rpc ListenFinalizeBlock(ListenFinalizeBlockRequest) returns (Empty); + rpc ListenCommit(ListenCommitRequest) returns (Empty); +} +``` + +```protobuf +... +// plugin that doesn't listen to state changes +service ABCIListenerService { + rpc ListenFinalizeBlock(ListenFinalizeBlockRequest) returns (Empty); + rpc ListenCommit(ListenCommitRequest) returns (Empty); +} +``` + +Implementing the service above: + +```go expandable +// streaming/plugins/abci/{ + plugin_version +}/grpc.go + +var ( + _ baseapp.ABCIListener = (*GRPCClient)(nil) +) + +// GRPCClient is an implementation of the ABCIListener and ABCIListenerPlugin interfaces that talks over RPC. +type GRPCClient struct { + client ABCIListenerServiceClient +} + +func (m *GRPCClient) + +ListenFinalizeBlock(goCtx context.Context, req abci.RequestFinalizeBlock, res abci.ResponseFinalizeBlock) + +error { + ctx := sdk.UnwrapSDKContext(goCtx) + _, err := m.client.ListenDeliverTx(ctx, &ListenDeliverTxRequest{ + BlockHeight: ctx.BlockHeight(), + Req: req, + Res: res +}) + +return err +} + +func (m *GRPCClient) + +ListenCommit(goCtx context.Context, res abci.ResponseCommit, changeSet []store.StoreKVPair) + +error { + ctx := sdk.UnwrapSDKContext(goCtx) + _, err := m.client.ListenCommit(ctx, &ListenCommitRequest{ + BlockHeight: ctx.BlockHeight(), + Res: res, + ChangeSet: changeSet +}) + +return err +} + +// GRPCServer is the gRPC server that GRPCClient talks to. +type GRPCServer struct { + // This is the real implementation + Impl baseapp.ABCIListener +} + +func (m *GRPCServer) + +ListenFinalizeBlock(ctx context.Context, req *ListenFinalizeBlockRequest) (*Empty, error) { + return &Empty{ +}, m.Impl.ListenFinalizeBlock(ctx, req.Req, req.Res) +} + +func (m *GRPCServer) + +ListenCommit(ctx context.Context, req *ListenCommitRequest) (*Empty, error) { + return &Empty{ +}, m.Impl.ListenCommit(ctx, req.Res, req.ChangeSet) +} +``` + +And the pre-compiled Go plugin `Impl`(*this is only used for plugins that are written in Go*): + +```go expandable +// streaming/plugins/abci/{ + plugin_version +}/impl/plugin.go + +// Plugins are pre-compiled and loaded by the plugin system + +// ABCIListener is the implementation of the baseapp.ABCIListener interface +type ABCIListener struct{ +} + +func (m *ABCIListenerPlugin) + +ListenFinalizeBlock(ctx context.Context, req abci.RequestFinalizeBlock, res abci.ResponseFinalizeBlock) + +error { + // send data to external system +} + +func (m *ABCIListenerPlugin) + +ListenCommit(ctx context.Context, res abci.ResponseCommit, changeSet []store.StoreKVPair) + +error { + // send data to external system +} + +func main() { + plugin.Serve(&plugin.ServeConfig{ + HandshakeConfig: grpc_abci_v1.Handshake, + Plugins: map[string]plugin.Plugin{ + "grpc_plugin_v1": &grpc_abci_v1.ABCIListenerGRPCPlugin{ + Impl: &ABCIListenerPlugin{ +}}, +}, + + // A non-nil value here enables gRPC serving for this streaming... + GRPCServer: plugin.DefaultGRPCServer, +}) +} +``` + +We will introduce a plugin loading system that will return `(interface{}, error)`. +This provides the advantage of using versioned plugins where the plugin interface and gRPC protocol change over time. +In addition, it allows for building independent plugin that can expose different parts of the system over gRPC. + +```go expandable +func NewStreamingPlugin(name string, logLevel string) (interface{ +}, error) { + logger := hclog.New(&hclog.LoggerOptions{ + Output: hclog.DefaultOutput, + Level: toHclogLevel(logLevel), + Name: fmt.Sprintf("plugin.%s", name), +}) + + // We're a host. Start by launching the streaming process. + env := os.Getenv(GetPluginEnvKey(name)) + client := plugin.NewClient(&plugin.ClientConfig{ + HandshakeConfig: HandshakeMap[name], + Plugins: PluginMap, + Cmd: exec.Command("sh", "-c", env), + Logger: logger, + AllowedProtocols: []plugin.Protocol{ + plugin.ProtocolNetRPC, plugin.ProtocolGRPC +}, +}) + + // Connect via RPC + rpcClient, err := client.Client() + if err != nil { + return nil, err +} + + // Request streaming plugin + return rpcClient.Dispense(name) +} +``` + +We propose a `RegisterStreamingPlugin` function for the App to register `NewStreamingPlugin`s with the App's BaseApp. +Streaming plugins can be of `Any` type; therefore, the function takes in an interface vs a concrete type. +For example, we could have plugins of `ABCIListener`, `WasmListener` or `IBCListener`. Note that `RegisterStreamingPluing` function +is helper function and not a requirement. Plugin registration can easily be moved from the App to the BaseApp directly. + +```go expandable +// baseapp/streaming.go + +// RegisterStreamingPlugin registers streaming plugins with the App. +// This method returns an error if a plugin is not supported. +func RegisterStreamingPlugin( + bApp *BaseApp, + appOpts servertypes.AppOptions, + keys map[string]*types.KVStoreKey, + streamingPlugin interface{ +}, +) + +error { + switch t := streamingPlugin.(type) { + case ABCIListener: + registerABCIListenerPlugin(bApp, appOpts, keys, t) + +default: + return fmt.Errorf("unexpected plugin type %T", t) +} + +return nil +} +``` + +```go expandable +func registerABCIListenerPlugin( + bApp *BaseApp, + appOpts servertypes.AppOptions, + keys map[string]*store.KVStoreKey, + abciListener ABCIListener, +) { + asyncKey := fmt.Sprintf("%s.%s.%s", StreamingTomlKey, StreamingABCITomlKey, StreamingABCIAsync) + async := cast.ToBool(appOpts.Get(asyncKey)) + stopNodeOnErrKey := fmt.Sprintf("%s.%s.%s", StreamingTomlKey, StreamingABCITomlKey, StreamingABCIStopNodeOnErrTomlKey) + stopNodeOnErr := cast.ToBool(appOpts.Get(stopNodeOnErrKey)) + keysKey := fmt.Sprintf("%s.%s.%s", StreamingTomlKey, StreamingABCITomlKey, StreamingABCIKeysTomlKey) + exposeKeysStr := cast.ToStringSlice(appOpts.Get(keysKey)) + exposedKeys := exposeStoreKeysSorted(exposeKeysStr, keys) + +bApp.cms.AddListeners(exposedKeys) + +app.SetStreamingManager( + storetypes.StreamingManager{ + ABCIListeners: []storetypes.ABCIListener{ + abciListener +}, + StopNodeOnErr: stopNodeOnErr, +}, + ) +} +``` + +```go expandable +func exposeAll(list []string) + +bool { + for _, ele := range list { + if ele == "*" { + return true +} + +} + +return false +} + +func exposeStoreKeys(keysStr []string, keys map[string]*types.KVStoreKey) []types.StoreKey { + var exposeStoreKeys []types.StoreKey + if exposeAll(keysStr) { + exposeStoreKeys = make([]types.StoreKey, 0, len(keys)) + for _, storeKey := range keys { + exposeStoreKeys = append(exposeStoreKeys, storeKey) +} + +} + +else { + exposeStoreKeys = make([]types.StoreKey, 0, len(keysStr)) + for _, keyStr := range keysStr { + if storeKey, ok := keys[keyStr]; ok { + exposeStoreKeys = append(exposeStoreKeys, storeKey) +} + +} + +} + // sort storeKeys for deterministic output + sort.SliceStable(exposeStoreKeys, func(i, j int) + +bool { + return exposeStoreKeys[i].Name() < exposeStoreKeys[j].Name() +}) + +return exposeStoreKeys +} +``` + +The `NewStreamingPlugin` and `RegisterStreamingPlugin` functions are used to register a plugin with the App's BaseApp. + +e.g. in `NewSimApp`: + +```go expandable +func NewSimApp( + logger log.Logger, + db dbm.DB, + traceStore io.Writer, + loadLatest bool, + appOpts servertypes.AppOptions, + baseAppOptions ...func(*baseapp.BaseApp), +) *SimApp { + + ... + keys := sdk.NewKVStoreKeys( + authtypes.StoreKey, banktypes.StoreKey, stakingtypes.StoreKey, + minttypes.StoreKey, distrtypes.StoreKey, slashingtypes.StoreKey, + govtypes.StoreKey, paramstypes.StoreKey, ibchost.StoreKey, upgradetypes.StoreKey, + evidencetypes.StoreKey, ibctransfertypes.StoreKey, capabilitytypes.StoreKey, + ) + + ... + + // register streaming services + streamingCfg := cast.ToStringMap(appOpts.Get(baseapp.StreamingTomlKey)) + for service := range streamingCfg { + pluginKey := fmt.Sprintf("%s.%s.%s", baseapp.StreamingTomlKey, service, baseapp.StreamingPluginTomlKey) + pluginName := strings.TrimSpace(cast.ToString(appOpts.Get(pluginKey))) + if len(pluginName) > 0 { + logLevel := cast.ToString(appOpts.Get(flags.FlagLogLevel)) + +plugin, err := streaming.NewStreamingPlugin(pluginName, logLevel) + if err != nil { + tmos.Exit(err.Error()) +} + if err := baseapp.RegisterStreamingPlugin(bApp, appOpts, keys, plugin); err != nil { + tmos.Exit(err.Error()) +} + +} + +} + +return app +``` + +#### Configuration + +The plugin system will be configured within an App's TOML configuration files. + +```toml expandable +# gRPC streaming +[streaming] + +# ABCI streaming service +[streaming.abci] + +# The plugin version to use for ABCI listening +plugin = "abci_v1" + +# List of kv store keys to listen to for state changes. +# Set to ["*"] to expose all keys. +keys = ["*"] + +# Enable abciListeners to run asynchronously. +# When abciListenersAsync=false and stopNodeOnABCIListenerErr=false listeners will run synchronized but will not stop the node. +# When abciListenersAsync=true stopNodeOnABCIListenerErr will be ignored. +async = false + +# Whether to stop the node on message deliver error. +stop-node-on-err = true +``` + +There will be four parameters for configuring `ABCIListener` plugin: `streaming.abci.plugin`, `streaming.abci.keys`, `streaming.abci.async` and `streaming.abci.stop-node-on-err`. +`streaming.abci.plugin` is the name of the plugin we want to use for streaming, `streaming.abci.keys` is a set of store keys for stores it listens to, +`streaming.abci.async` is bool enabling asynchronous listening and `streaming.abci.stop-node-on-err` is a bool that stops the node when true and when operating +on synchronized mode `streaming.abci.async=false`. Note that `streaming.abci.stop-node-on-err=true` will be ignored if `streaming.abci.async=true`. + +The configuration above support additional streaming plugins by adding the plugin to the `[streaming]` configuration section +and registering the plugin with `RegisterStreamingPlugin` helper function. + +Note the that each plugin must include `streaming.{service}.plugin` property as it is a requirement for doing the lookup and registration of the plugin +with the App. All other properties are unique to the individual services. + +#### Encoding and decoding streams + +ADR-038 introduces the interfaces and types for streaming state changes out from KVStores, associating this +data with their related ABCI requests and responses, and registering a service for consuming this data and streaming it to some destination in a final format. +Instead of prescribing a final data format in this ADR, it is left to a specific plugin implementation to define and document this format. +We take this approach because flexibility in the final format is necessary to support a wide range of streaming service plugins. For example, +the data format for a streaming service that writes the data out to a set of files will differ from the data format that is written to a Kafka topic. + +## Consequences + +These changes will provide a means of subscribing to KVStore state changes in real time. + +### Backwards Compatibility + +* This ADR changes the `CommitMultiStore` interface, implementations supporting the previous version of this interface will not support the new one + +### Positive + +* Ability to listen to KVStore state changes in real time and expose these events to external consumers + +### Negative + +* Changes `CommitMultiStore` interface and its implementations + +### Neutral + +* Introduces additional- but optional- complexity to configuring and running a cosmos application +* If an application developer opts to use these features to expose data, they need to be aware of the ramifications/risks of that data exposure as it pertains to the specifics of their application diff --git a/sdk/v0.54/reference/architecture/adr-039-epoched-staking.mdx b/sdk/v0.54/reference/architecture/adr-039-epoched-staking.mdx new file mode 100644 index 000000000..f29754b76 --- /dev/null +++ b/sdk/v0.54/reference/architecture/adr-039-epoched-staking.mdx @@ -0,0 +1,127 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/architecture/adr-039-epoched-staking' +title: 'ADR 039: Epoched Staking' +description: '10-Feb-2021: Initial Draft' +--- + +## Changelog + +* 10-Feb-2021: Initial Draft + +## Authors + +* Dev Ojha (@valardragon) +* Sunny Aggarwal (@sunnya97) + +## Status + +Proposed + +## Abstract + +This ADR updates the proof of stake module to buffer the staking weight updates for a number of blocks before updating the consensus' staking weights. The length of the buffer is dubbed an epoch. The prior functionality of the staking module is then a special case of the abstracted module, with the epoch being set to 1 block. + +## Context + +The current proof of stake module takes the design decision to apply staking weight changes to the consensus engine immediately. This means that delegations and unbonds get applied immediately to the validator set. This decision was primarily done as it was implementationally simplest, and because we at the time believed that this would lead to better UX for clients. + +An alternative design choice is to allow buffering staking updates (delegations, unbonds, validators joining) for a number of blocks. This 'epoch'd proof of stake consensus provides the guarantee that the consensus weights for validators will not change mid-epoch, except in the event of a slash condition. + +Additionally, the UX hurdle may not be as significant as was previously thought. This is because it is possible to provide users immediate acknowledgement that their bond was recorded and will be executed. + +Furthermore, it has become clearer over time that immediate execution of staking events comes with limitations, such as: + +* Threshold based cryptography. One of the main limitations is that because the validator set can change so regularly, it makes the running of multiparty computation by a fixed validator set difficult. Many threshold-based cryptographic features for blockchains such as randomness beacons and threshold decryption require a computationally-expensive DKG process (will take much longer than 1 block to create). To productively use these, we need to guarantee that the result of the DKG will be used for a reasonably long time. It wouldn't be feasible to rerun the DKG every block. By epoching staking, it guarantees we'll only need to run a new DKG once every epoch. + +* Light client efficiency. This would lessen the overhead for IBC when there is high churn in the validator set. In the Tendermint light client bisection algorithm, the number of headers you need to verify is related to bounding the difference in validator sets between a trusted header and the latest header. If the difference is too great, you verify more header in between the two. By limiting the frequency of validator set changes, we can reduce the worst case size of IBC lite client proofs, which occurs when a validator set has high churn. + +* Fairness of deterministic leader election. Currently we have no ways of reasoning of fairness of deterministic leader election in the presence of staking changes without epochs (tendermint/spec#217). Breaking fairness of leader election is profitable for validators, as they earn additional rewards from being the proposer. Adding epochs at least makes it easier for our deterministic leader election to match something we can prove secure. (Albeit, we still haven’t proven if our current algorithm is fair with > 2 validators in the presence of stake changes) + +* Staking derivative design. Currently, reward distribution is done lazily using the F1 fee distribution. While saving computational complexity, lazy accounting requires a more stateful staking implementation. Right now, each delegation entry has to track the time of last withdrawal. Handling this can be a challenge for some staking derivatives designs that seek to provide fungibility for all tokens staked to a single validator. Force-withdrawing rewards to users can help solve this, however it is infeasible to force-withdraw rewards to users on a per block basis. With epochs, a chain could more easily alter the design to have rewards be forcefully withdrawn (iterating over delegator accounts only once per-epoch), and can thus remove delegation timing from state. This may be useful for certain staking derivative designs. + +## Design considerations + +### Slashing + +There is a design consideration for whether to apply a slash immediately or at the end of an epoch. A slash event should apply to only members who are actually staked during the time of the infraction, namely during the epoch the slash event occurred. + +Applying it immediately can be viewed as offering greater consensus layer security, at potential costs to the aforementioned usecases. The benefits of immediate slashing for consensus layer security can be all be obtained by executing the validator jailing immediately (thus removing it from the validator set), and delaying the actual slash change to the validator's weight until the epoch boundary. For the use cases mentioned above, workarounds can be integrated to avoid problems, as follows: + +* For threshold based cryptography, this setting will have the threshold cryptography use the original epoch weights, while consensus has an update that lets it more rapidly benefit from additional security. If the threshold based cryptography blocks liveness of the chain, then we have effectively raised the liveness threshold of the remaining validators for the rest of the epoch. (Alternatively, jailed nodes could still contribute shares) This plan will fail in the extreme case that more than 1/3rd of the validators have been jailed within a single epoch. For such an extreme scenario, the chain already have its own custom incident response plan, and defining how to handle the threshold cryptography should be a part of that. +* For light client efficiency, there can be a bit included in the header indicating an intra-epoch slash (ala [Link](https://github.com/tendermint/spec/issues/199)). +* For fairness of deterministic leader election, applying a slash or jailing within an epoch would break the guarantee we were seeking to provide. This then re-introduces a new (but significantly simpler) problem for trying to provide fairness guarantees. Namely, that validators can adversarially elect to remove themself from the set of proposers. From a security perspective, this could potentially be handled by two different mechanisms (or prove to still be too difficult to achieve). One is making a security statement acknowledging the ability for an adversary to force an ahead-of-time fixed threshold of users to drop out of the proposer set within an epoch. The second method would be to parameterize such that the cost of a slash within the epoch far outweights benefits due to being a proposer. However, this latter criterion is quite dubious, since being a proposer can have many advantageous side-effects in chains with complex state machines. (Namely, DeFi games such as Fomo3D) +* For staking derivative design, there is no issue introduced. This does not increase the state size of staking records, since whether a slash has occurred is fully queryable given the validator address. + +### Token lockup + +When someone makes a transaction to delegate, even though they are not immediately staked, their tokens should be moved into a pool managed by the staking module which will then be used at the end of an epoch. This prevents concerns where they stake, and then spend those tokens not realizing they were already allocated for staking, and thus having their staking tx fail. + +### Pipelining the epochs + +For threshold based cryptography in particular, we need a pipeline for epoch changes. This is because when we are in epoch N, we want the epoch N+1 weights to be fixed so that the validator set can do the DKG accordingly. So if we are currently in epoch N, the stake weights for epoch N+1 should already be fixed, and new stake changes should be getting applied to epoch N + 2. + +This can be handled by making a parameter for the epoch pipeline length. This parameter should not be alterable except during hard forks, to mitigate implementation complexity of switching the pipeline length. + +With pipeline length 1, if I redelegate during epoch N, then my redelegation is applied prior to the beginning of epoch N+1. +With pipeline length 2, if I redelegate during epoch N, then my redelegation is applied prior to the beginning of epoch N+2. + +### Rewards + +Even though all staking updates are applied at epoch boundaries, rewards can still be distributed immediately when they are claimed. This is because they do not affect the current stake weights, as we do not implement auto-bonding of rewards. If such a feature were to be implemented, it would have to be setup so that rewards are auto-bonded at the epoch boundary. + +### Parameterizing the epoch length + +When choosing the epoch length, there is a trade-off queued state/computation buildup, and countering the previously discussed limitations of immediate execution if they apply to a given chain. + +Until an ABCI mechanism for variable block times is introduced, it is ill-advised to be using high epoch lengths due to the computation buildup. This is because when a block's execution time is greater than the expected block time from Tendermint, rounds may increment. + +## Decision + +**Step-1**: Implement buffering of all staking and slashing messages. + +First we create a pool for storing tokens that are being bonded, but should be applied at the epoch boundary called the `EpochDelegationPool`. Then, we have two separate queues, one for staking, one for slashing. We describe what happens on each message being delivered below: + +### Staking messages + +* **MsgCreateValidator**: Move user's self-bond to `EpochDelegationPool` immediately. Queue a message for the epoch boundary to handle the self-bond, taking the funds from the `EpochDelegationPool`. If Epoch execution fail, return back funds from `EpochDelegationPool` to user's account. +* **MsgEditValidator**: Validate message and if valid queue the message for execution at the end of the Epoch. +* **MsgDelegate**: Move user's funds to `EpochDelegationPool` immediately. Queue a message for the epoch boundary to handle the delegation, taking the funds from the `EpochDelegationPool`. If Epoch execution fail, return back funds from `EpochDelegationPool` to user's account. +* **MsgBeginRedelegate**: Validate message and if valid queue the message for execution at the end of the Epoch. +* **MsgUndelegate**: Validate message and if valid queue the message for execution at the end of the Epoch. + +### Slashing messages + +* **MsgUnjail**: Validate message and if valid queue the message for execution at the end of the Epoch. +* **Slash Event**: Whenever a slash event is created, it gets queued in the slashing module to apply at the end of the epoch. The queues should be setup such that this slash applies immediately. + +### Evidence Messages + +* **MsgSubmitEvidence**: This gets executed immediately, and the validator gets jailed immediately. However in slashing, the actual slash event gets queued. + +Then we add methods to the end blockers, to ensure that at the epoch boundary the queues are cleared and delegation updates are applied. + +**Step-2**: Implement querying of queued staking txs. + +When querying the staking activity of a given address, the status should return not only the amount of tokens staked, but also if there are any queued stake events for that address. This will require more work to be done in the querying logic, to trace the queued upcoming staking events. + +As an initial implementation, this can be implemented as a linear search over all queued staking events. However, for chains that need long epochs, they should eventually build additional support for nodes that support querying to be able to produce results in constant time. (This is do-able by maintaining an auxilliary hashmap for indexing upcoming staking events by address) + +**Step-3**: Adjust gas + +Currently gas represents the cost of executing a transaction when its done immediately. (Merging together costs of p2p overhead, state access overhead, and computational overhead) However, now a transaction can cause computation in a future block, namely at the epoch boundary. + +To handle this, we should initially include parameters for estimating the amount of future computation (denominated in gas), and add that as a flat charge needed for the message. +We leave it as out of scope for how to weight future computation versus current computation in gas pricing, and have it set such that the are weighted equally for now. + +## Consequences + +### Positive + +* Abstracts the proof of stake module that allows retaining the existing functionality +* Enables new features such as validator-set based threshold cryptography + +### Negative + +* Increases complexity of integrating more complex gas pricing mechanisms, as they now have to consider future execution costs as well. +* When epoch > 1, validators can no longer leave the network immediately, and must wait until an epoch boundary. diff --git a/sdk/v0.54/reference/architecture/adr-040-storage-and-smt-state-commitments.mdx b/sdk/v0.54/reference/architecture/adr-040-storage-and-smt-state-commitments.mdx new file mode 100644 index 000000000..acee076ff --- /dev/null +++ b/sdk/v0.54/reference/architecture/adr-040-storage-and-smt-state-commitments.mdx @@ -0,0 +1,298 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/architecture/adr-040-storage-and-smt-state-commitments' +title: 'ADR 040: Storage and SMT State Commitments' +description: '2020-01-15: Draft' +--- + +## Changelog + +* 2020-01-15: Draft + +## Status + +DRAFT Not Implemented + +## Abstract + +Sparse Merkle Tree ([SMT](https://osf.io/8mcnh/)) is a version of a Merkle Tree with various storage and performance optimizations. This ADR defines a separation of state commitments from data storage and the Cosmos SDK transition from IAVL to SMT. + +## Context + +Currently, Cosmos SDK uses IAVL for both state [commitments](https://cryptography.fandom.com/wiki/Commitment_scheme) and data storage. + +IAVL has effectively become an orphaned project within the Cosmos ecosystem and it's proven to be an inefficient state commitment data structure. +In the current design, IAVL is used for both data storage and as a Merkle Tree for state commitments. IAVL is meant to be a standalone Merkelized key/value database, however it's using a KV DB engine to store all tree nodes. So, each node is stored in a separate record in the KV DB. This causes many inefficiencies and problems: + +* Each object query requires a tree traversal from the root. Subsequent queries for the same object are cached on the Cosmos SDK level. +* Each edge traversal requires a DB query. +* Creating snapshots is [expensive](https://github.com/cosmos/cosmos-sdk/issues/7215#issuecomment-684804950). It takes about 30 seconds to export less than 100 MB of state (as of March 2020). +* Updates in IAVL may trigger tree reorganization and possible O(log(n)) hashes re-computation, which can become a CPU bottleneck. +* The node structure is pretty expensive - it contains a standard tree node elements (key, value, left and right element) and additional metadata such as height, version (which is not required by the Cosmos SDK). The entire node is hashed, and that hash is used as the key in the underlying database, [ref](https://github.com/cosmos/iavl/blob/master/docs/node/node.md). + +Moreover, the IAVL project lacks support and a maintainer and we already see better and well-established alternatives. Instead of optimizing the IAVL, we are looking into other solutions for both storage and state commitments. + +## Decision + +We propose to separate the concerns of state commitment (**SC**), needed for consensus, and state storage (**SS**), needed for state machine. Finally we replace IAVL with [Celestia's SMT](https://github.com/lazyledger/smt). Celestia SMT is based on Diem (called jellyfish) design \[\*] - it uses a compute-optimized SMT by replacing subtrees with only default values with a single node (same approach is used by Ethereum2) and implements compact proofs. + +The storage model presented here doesn't deal with data structure nor serialization. It's a Key-Value database, where both key and value are binaries. The storage user is responsible for data serialization. + +### Decouple state commitment from storage + +Separation of storage and commitment (by the SMT) will allow the optimization of different components according to their usage and access patterns. + +`SC` (SMT) is used to commit to a data and compute Merkle proofs. `SS` is used to directly access data. To avoid collisions, both `SS` and `SC` will use a separate storage namespace (they could use the same database underneath). `SS` will store each record directly (mapping `(key, value)` as `key → value`). + +SMT is a merkle tree structure: we don't store keys directly. For every `(key, value)` pair, `hash(key)` is used as leaf path (we hash a key to uniformly distribute leaves in the tree) and `hash(value)` as the leaf contents. The tree structure is specified in more depth [below](#smt-for-state-commitment). + +For data access we propose 2 additional KV buckets (implemented as namespaces for the key-value pairs, sometimes called [column family](https://github.com/facebook/rocksdb/wiki/Terminology)): + +1. B1: `key → value`: the principal object storage, used by a state machine, behind the Cosmos SDK `KVStore` interface: provides direct access by key and allows prefix iteration (KV DB backend must support it). +2. B2: `hash(key) → key`: a reverse index to get a key from an SMT path. Internally the SMT will store `(key, value)` as `prefix || hash(key) || hash(value)`. So, we can get an object value by composing `hash(key) → B2 → B1`. +3. We could use more buckets to optimize the app usage if needed. + +We propose to use a KV database for both `SS` and `SC`. The store interface will allow to use the same physical DB backend for both `SS` and `SC` as well two separate DBs. The latter option allows for the separation of `SS` and `SC` into different hardware units, providing support for more complex setup scenarios and improving overall performance: one can use different backends (eg RocksDB and Badger) as well as independently tuning the underlying DB configuration. + +### Requirements + +State Storage requirements: + +* range queries +* quick (key, value) access +* creating a snapshot +* historical versioning +* pruning (garbage collection) + +State Commitment requirements: + +* fast updates +* tree path should be short +* query historical commitment proofs using ICS-23 standard +* pruning (garbage collection) + +### SMT for State Commitment + +A Sparse Merkle tree is based on the idea of a complete Merkle tree of an intractable size. The assumption here is that as the size of the tree is intractable, there would only be a few leaf nodes with valid data blocks relative to the tree size, rendering a sparse tree. + +The full specification can be found at [Celestia](https://github.com/celestiaorg/celestia-specs/blob/ec98170398dfc6394423ee79b00b71038879e211/src/specs/data_structures.md#sparse-merkle-tree). In summary: + +* The SMT consists of a binary Merkle tree, constructed in the same fashion as described in [Certificate Transparency (RFC-6962)](https://tools.ietf.org/html/rfc6962), but using as the hashing function SHA-2-256 as defined in [FIPS 180-4](https://doi.org/10.6028/NIST.FIPS.180-4). +* Leaves and internal nodes are hashed differently: the one-byte `0x00` is prepended for leaf nodes while `0x01` is prepended for internal nodes. +* Default values are given to leaf nodes with empty leaves. +* While the above rule is sufficient to pre-compute the values of intermediate nodes that are roots of empty subtrees, a further simplification is to extend this default value to all nodes that are roots of empty subtrees. The 32-byte zero is used as the default value. This rule takes precedence over the above one. +* An internal node that is the root of a subtree that contains exactly one non-empty leaf is replaced by that leaf's leaf node. + +### Snapshots for storage sync and state versioning + +Below, with simple *snapshot* we refer to a database snapshot mechanism, not to a *ABCI snapshot sync*. The latter will be referred as *snapshot sync* (which will directly use DB snapshot as described below). + +Database snapshot is a view of DB state at a certain time or transaction. It's not a full copy of a database (it would be too big). Usually a snapshot mechanism is based on a *copy on write* and it allows DB state to be efficiently delivered at a certain stage. +Some DB engines support snapshotting. Hence, we propose to reuse that functionality for the state sync and versioning (described below). We limit the supported DB engines to ones which efficiently implement snapshots. In a final section we discuss the evaluated DBs. + +One of the Stargate core features is a *snapshot sync* delivered in the `/snapshot` package. It provides a way to trustlessly sync a blockchain without repeating all transactions from the genesis. This feature is implemented in Cosmos SDK and requires storage support. Currently IAVL is the only supported backend. It works by streaming to a client a snapshot of a `SS` at a certain version together with a header chain. + +A new database snapshot will be created in every `EndBlocker` and identified by a block height. The `root` store keeps track of the available snapshots to offer `SS` at a certain version. The `root` store implements the `RootStore` interface described below. In essence, `RootStore` encapsulates a `Committer` interface. `Committer` has a `Commit`, `SetPruning`, `GetPruning` functions which will be used for creating and removing snapshots. The `rootStore.Commit` function creates a new snapshot and increments the version on each call, and checks if it needs to remove old versions. We will need to update the SMT interface to implement the `Committer` interface. +NOTE: `Commit` must be called exactly once per block. Otherwise we risk going out of sync for the version number and block height. +NOTE: For the Cosmos SDK storage, we may consider splitting that interface into `Committer` and `PruningCommitter` - only the multiroot should implement `PruningCommitter` (cache and prefix store don't need pruning). + +Number of historical versions for `abci.RequestQuery` and state sync snapshots is part of a node configuration, not a chain configuration (configuration implied by the blockchain consensus). A configuration should allow to specify number of past blocks and number of past blocks modulo some number (eg: 100 past blocks and one snapshot every 100 blocks for past 2000 blocks). Archival nodes can keep all past versions. + +Pruning old snapshots is effectively done by a database. Whenever we update a record in `SC`, SMT won't update nodes - instead it creates new nodes on the update path, without removing the old one. Since we are snapshotting each block, we need to change that mechanism to immediately remove orphaned nodes from the database. This is a safe operation - snapshots will keep track of the records and make it available when accessing past versions. + +To manage the active snapshots we will either use a DB *max number of snapshots* option (if available), or we will remove DB snapshots in the `EndBlocker`. The latter option can be done efficiently by identifying snapshots with block height and calling a store function to remove past versions. + +#### Accessing old state versions + +One of the functional requirements is to access old state. This is done through `abci.RequestQuery` structure. The version is specified by a block height (so we query for an object by a key `K` at block height `H`). The number of old versions supported for `abci.RequestQuery` is configurable. Accessing an old state is done by using available snapshots. +`abci.RequestQuery` doesn't need old state of `SC` unless the `prove=true` parameter is set. The SMT merkle proof must be included in the `abci.ResponseQuery` only if both `SC` and `SS` have a snapshot for requested version. + +Moreover, Cosmos SDK could provide a way to directly access a historical state. However, a state machine shouldn't do that - since the number of snapshots is configurable, it would lead to nondeterministic execution. + +We positively [validated](https://github.com/cosmos/cosmos-sdk/discussions/8297) a versioning and snapshot mechanism for querying old state with regards to the database we evaluated. + +### State Proofs + +For any object stored in State Store (SS), we have corresponding object in `SC`. A proof for object `V` identified by a key `K` is a branch of `SC`, where the path corresponds to the key `hash(K)`, and the leaf is `hash(K, V)`. + +### Rollbacks + +We need to be able to process transactions and roll-back state updates if a transaction fails. This can be done in the following way: during transaction processing, we keep all state change requests (writes) in a `CacheWrapper` abstraction (as it's done today). Once we finish the block processing, in the `Endblocker`, we commit a root store - at that time, all changes are written to the SMT and to the `SS` and a snapshot is created. + +### Committing to an object without saving it + +We identified use-cases, where modules will need to save an object commitment without storing an object itself. Sometimes clients are receiving complex objects, and they have no way to prove a correctness of that object without knowing the storage layout. For those use cases it would be easier to commit to the object without storing it directly. + +### Refactor MultiStore + +The Stargate `/store` implementation (store/v1) adds an additional layer in the SDK store construction - the `MultiStore` structure. The multistore exists to support the modularity of the Cosmos SDK - each module is using its own instance of IAVL, but in the current implementation, all instances share the same database. The latter indicates, however, that the implementation doesn't provide true modularity. Instead it causes problems related to race condition and atomic DB commits (see: [#6370](https://github.com/cosmos/cosmos-sdk/issues/6370) and [discussion](https://github.com/cosmos/cosmos-sdk/discussions/8297#discussioncomment-757043)). + +We propose to reduce the multistore concept from the SDK, and to use a single instance of `SC` and `SS` in a `RootStore` object. To avoid confusion, we should rename the `MultiStore` interface to `RootStore`. The `RootStore` will have the following interface; the methods for configuring tracing and listeners are omitted for brevity. + +```go expandable +// Used where read-only access to versions is needed. +type BasicRootStore interface { + Store + GetKVStore(StoreKey) + +KVStore + CacheRootStore() + +CacheRootStore +} + +// Used as the main app state, replacing CommitMultiStore. +type CommitRootStore interface { + BasicRootStore + Committer + Snapshotter + + GetVersion(uint64) (BasicRootStore, error) + +SetInitialVersion(uint64) + +error + + ... // Trace and Listen methods +} + +// Replaces CacheMultiStore for branched state. +type CacheRootStore interface { + BasicRootStore + Write() + + ... // Trace and Listen methods +} + +// Example of constructor parameters for the concrete type. +type RootStoreConfig struct { + Upgrades *StoreUpgrades + InitialVersion uint64 + + ReservePrefix(StoreKey, StoreType) +} +``` + +{/* TODO: Review whether these types can be further reduced or simplified */} +{/* TODO: RootStorePersistentCache type */} + +In contrast to `MultiStore`, `RootStore` doesn't allow to dynamically mount sub-stores or provide an arbitrary backing DB for individual sub-stores. + +NOTE: modules will be able to use a special commitment and their own DBs. For example: a module which will use ZK proofs for state can store and commit this proof in the `RootStore` (usually as a single record) and manage the specialized store privately or using the `SC` low level interface. + +#### Compatibility support + +To ease the transition to this new interface for users, we can create a shim which wraps a `CommitMultiStore` but provides a `CommitRootStore` interface, and expose functions to safely create and access the underlying `CommitMultiStore`. + +The new `RootStore` and supporting types can be implemented in a `store/v2alpha1` package to avoid breaking existing code. + +#### Merkle Proofs and IBC + +Currently, an IBC (v1.0) Merkle proof path consists of two elements (`["", ""]`), with each key corresponding to a separate proof. These are each verified according to individual [ICS-23 specs](https://github.com/cosmos/ibc-go/blob/f7051429e1cf833a6f65d51e6c3df1609290a549/modules/core/23-commitment/types/merkle.go#L17), and the result hash of each step is used as the committed value of the next step, until a root commitment hash is obtained. +The root hash of the proof for `""` is hashed with the `""` to validate against the App Hash. + +This is not compatible with the `RootStore`, which stores all records in a single Merkle tree structure, and won't produce separate proofs for the store- and record-key. Ideally, the store-key component of the proof could just be omitted, and updated to use a "no-op" spec, so only the record-key is used. However, because the IBC verification code hardcodes the `"ibc"` prefix and applies it to the SDK proof as a separate element of the proof path, this isn't possible without a breaking change. Breaking this behavior would severely impact the Cosmos ecosystem which already widely adopts the IBC module. Requesting an update of the IBC module across the chains is a time consuming effort and not easily feasible. + +As a workaround, the `RootStore` will have to use two separate SMTs (they could use the same underlying DB): one for IBC state and one for everything else. A simple Merkle map that reference these SMTs will act as a Merkle Tree to create a final App hash. The Merkle map is not stored in a DBs - it's constructed in the runtime. The IBC substore key must be `"ibc"`. + +The workaround can still guarantee atomic syncs: the [proposed DB backends](#evaluated-kv-databases) support atomic transactions and efficient rollbacks, which will be used in the commit phase. + +The presented workaround can be used until the IBC module is fully upgraded to supports single-element commitment proofs. + +### Optimization: compress module key prefixes + +We consider a compression of prefix keys by creating a mapping from module key to an integer, and serializing the integer using varint coding. Varint coding assures that different values don't have common byte prefix. For Merkle Proofs we can't use prefix compression - so it should only apply for the `SS` keys. Moreover, the prefix compression should be only applied for the module namespace. More precisely: + +* each module has its own namespace; +* when accessing a module namespace we create a KVStore with embedded prefix; +* that prefix will be compressed only when accessing and managing `SS`. + +We need to assure that the codes won't change. We can fix the mapping in a static variable (provided by an app) or SS state under a special key. + +TODO: need to make decision about the key compression. + +## Optimization: SS key compression + +Some objects may be saved with key, which contains a Protobuf message type. Such keys are long. We could save a lot of space if we can map Protobuf message types in varints. + +TODO: finalize this or move to another ADR. + +## Migration + +Using the new store will require a migration. 2 Migrations are proposed: + +1. Genesis export -- it will reset the blockchain history. +2. In place migration: we can reuse `UpgradeKeeper.SetUpgradeHandler` to provide the migration logic: + +```go +app.UpgradeKeeper.SetUpgradeHandler("adr-40", func(ctx sdk.Context, plan upgradetypes.Plan, vm module.VersionMap) (module.VersionMap, error) { + storev2.Migrate(iavlstore, v2.store) + + // RunMigrations returns the VersionMap + // with the updated module ConsensusVersions + return app.mm.RunMigrations(ctx, vm) +}) +``` + +The `Migrate` function will read all entries from a store/v1 DB and save them to the AD-40 combined KV store. +Cache layer should not be used and the operation must finish with a single Commit call. + +Inserting records to the `SC` (SMT) component is the bottleneck. Unfortunately SMT doesn't support batch transactions. +Adding batch transactions to `SC` layer is considered as a feature after the main release. + +## Consequences + +### Backwards Compatibility + +This ADR doesn't introduce any Cosmos SDK level API changes. + +We change the storage layout of the state machine, a storage hard fork and network upgrade is required to incorporate these changes. SMT provides a merkle proof functionality, however it is not compatible with ICS23. Updating the proofs for ICS23 compatibility is required. + +### Positive + +* Decoupling state from state commitment introduce better engineering opportunities for further optimizations and better storage patterns. +* Performance improvements. +* Joining SMT based camp which has wider and proven adoption than IAVL. Example projects which decided on SMT: Ethereum2, Diem (Libra), Trillan, Tezos, Celestia. +* Multistore removal fixes a longstanding issue with the current MultiStore design. +* Simplifies merkle proofs - all modules, except IBC, have only one pass for merkle proof. + +### Negative + +* Storage migration +* LL SMT doesn't support pruning - we will need to add and test that functionality. +* `SS` keys will have an overhead of a key prefix. This doesn't impact `SC` because all keys in `SC` have same size (they are hashed). + +### Neutral + +* Deprecating IAVL, which is one of the core proposals of Cosmos Whitepaper. + +## Alternative designs + +Most of the alternative designs were evaluated in a prior state commitments and storage report. + +Ethereum research published [Verkle Trie](https://dankradfeist.de/ethereum/2021/06/18/verkle-trie-for-eth1.html) - an idea of combining polynomial commitments with merkle tree in order to reduce the tree height. This concept has a very good potential, but we think it's too early to implement it. The current, SMT based design could be easily updated to the Verkle Trie once other research implement all necessary libraries. The main advantage of the design described in this ADR is the separation of state commitments from the data storage and designing a more powerful interface. + +## Further Discussions + +### Evaluated KV Databases + +We verified existing databases KV databases for evaluating snapshot support. The following databases provide efficient snapshot mechanism: Badger, RocksDB, [Pebble](https://github.com/cockroachdb/pebble). Databases which don't provide such support or are not production ready: boltdb, leveldb, goleveldb, membdb, lmdb. + +### RDBMS + +Use of RDBMS instead of simple KV store for state. Use of RDBMS will require a Cosmos SDK API breaking change (`KVStore` interface) and will allow better data extraction and indexing solutions. Instead of saving an object as a single blob of bytes, we could save it as record in a table in the state storage layer, and as a `hash(key, protobuf(object))` in the SMT as outlined above. To verify that an object registered in RDBMS is same as the one committed to SMT, one will need to load it from RDBMS, marshal using protobuf, hash and do SMT search. + +### Off Chain Store + +We were discussing use case where modules can use a support database, which is not automatically committed. Module will responsible for having a sound storage model and can optionally use the feature discussed in \_*Committing to an object without saving it* section. + +## References + +* [IAVL What's Next?](https://github.com/cosmos/cosmos-sdk/issues/7100) +* [IAVL overview](https://docs.google.com/document/d/16Z_hW2rSAmoyMENO-RlAhQjAG3mSNKsQueMnKpmcBv0/edit#heading=h.yd2th7x3o1iv) of it's state v0.15 +* [Celestia (LazyLedger) SMT](https://github.com/lazyledger/smt) +* Facebook Diem (Libra) SMT [design](https://developers.diem.com/papers/jellyfish-merkle-tree/2021-01-14.pdf) +* [Trillian Revocation Transparency](https://github.com/google/trillian/blob/master/docs/papers/RevocationTransparency.pdf), [Trillian Verifiable Data Structures](https://github.com/google/trillian/blob/master/docs/papers/VerifiableDataStructures.pdf). +* Design and implementation [discussion](https://github.com/cosmos/cosmos-sdk/discussions/8297). +* [How to Upgrade IBC Chains and their Clients](https://github.com/cosmos/ibc-go/blob/main/docs/docs/01-ibc/05-upgrades/01-quick-guide.md) +* [ADR-40 Effect on IBC](https://github.com/cosmos/ibc-go/discussions/256) diff --git a/sdk/v0.54/reference/architecture/adr-041-in-place-store-migrations.mdx b/sdk/v0.54/reference/architecture/adr-041-in-place-store-migrations.mdx new file mode 100644 index 000000000..0bb1fb3b4 --- /dev/null +++ b/sdk/v0.54/reference/architecture/adr-041-in-place-store-migrations.mdx @@ -0,0 +1,185 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/architecture/adr-041-in-place-store-migrations' +title: 'ADR 041: In-Place Store Migrations' +description: '17.02.2021: Initial Draft' +--- + +## Changelog + +* 17.02.2021: Initial Draft + +## Status + +Accepted + +## Abstract + +This ADR introduces a mechanism to perform in-place state store migrations during chain software upgrades. + +## Context + +When a chain upgrade introduces state-breaking changes inside modules, the current procedure consists of exporting the whole state into a JSON file (via the `simd export` command), running migration scripts on the JSON file (`simd genesis migrate` command), clearing the stores (`simd unsafe-reset-all` command), and starting a new chain with the migrated JSON file as new genesis (optionally with a custom initial block height). An example of such a procedure can be seen [in the Cosmos Hub 3->4 migration guide](https://github.com/cosmos/gaia/blob/v4.0.3/docs/migration/cosmoshub-3.md#upgrade-procedure). + +This procedure is cumbersome for multiple reasons: + +* The procedure takes time. It can take hours to run the `export` command, plus some additional hours to run `InitChain` on the fresh chain using the migrated JSON. +* The exported JSON file can be heavy (\~100MB-1GB), making it difficult to view, edit and transfer, which in turn introduces additional work to solve these problems (such as [streaming genesis](https://github.com/cosmos/cosmos-sdk/issues/6936)). + +## Decision + +We propose a migration procedure based on modifying the KV store in-place without involving the JSON export-process-import flow described above. + +### Module `ConsensusVersion` + +We introduce a new method on the `AppModule` interface: + +```go +type AppModule interface { + // --snip-- + ConsensusVersion() + +uint64 +} +``` + +This methods returns an `uint64` which serves as state-breaking version of the module. It MUST be incremented on each consensus-breaking change introduced by the module. To avoid potential errors with default values, the initial version of a module MUST be set to 1. In the Cosmos SDK, version 1 corresponds to the modules in the v0.41 series. + +### Module-Specific Migration Functions + +For each consensus-breaking change introduced by the module, a migration script from ConsensusVersion `N` to version `N+1` MUST be registered in the `Configurator` using its newly-added `RegisterMigration` method. All modules receive a reference to the configurator in their `RegisterServices` method on `AppModule`, and this is where the migration functions should be registered. The migration functions should be registered in increasing order. + +```go +func (am AppModule) + +RegisterServices(cfg module.Configurator) { + // --snip-- + cfg.RegisterMigration(types.ModuleName, 1, func(ctx sdk.Context) + +error { + // Perform in-place store migrations from ConsensusVersion 1 to 2. +}) + +cfg.RegisterMigration(types.ModuleName, 2, func(ctx sdk.Context) + +error { + // Perform in-place store migrations from ConsensusVersion 2 to 3. +}) + // etc. +} +``` + +For example, if the new ConsensusVersion of a module is `N` , then `N-1` migration functions MUST be registered in the configurator. + +In the Cosmos SDK, the migration functions are handled by each module's keeper, because the keeper holds the `sdk.StoreKey` used to perform in-place store migrations. To not overload the keeper, a `Migrator` wrapper is used by each module to handle the migration functions: + +```go +// Migrator is a struct for handling in-place store migrations. +type Migrator struct { + BaseKeeper +} +``` + +Migration functions should live inside the `migrations/` folder of each module, and be called by the Migrator's methods. We propose the format `Migrate{M}to{N}` for method names. + +```go +// Migrate1to2 migrates from version 1 to 2. +func (m Migrator) + +Migrate1to2(ctx sdk.Context) + +error { + return v2bank.MigrateStore(ctx, m.keeper.storeKey) // v043bank is package `x/bank/migrations/v2`. +} +``` + +Each module's migration functions are specific to the module's store evolutions, and are not described in this ADR. An example of x/bank store key migrations after the introduction of ADR-028 length-prefixed addresses can be seen in this [store.go code](https://github.com/cosmos/cosmos-sdk/blob/36f68eb9e041e20a5bb47e216ac5eb8b91f95471/x/bank/legacy/v043/store.go#L41-L62). + +### Tracking Module Versions in `x/upgrade` + +We introduce a new prefix store in `x/upgrade`'s store. This store will track each module's current version, it can be modelized as a `map[string]uint64` of module name to module ConsensusVersion, and will be used when running the migrations (see next section for details). The key prefix used is `0x1`, and the key/value format is: + +```text +0x2 | {bytes(module_name)} => BigEndian(module_consensus_version) +``` + +The initial state of the store is set from `app.go`'s `InitChainer` method. + +The UpgradeHandler signature needs to be updated to take a `VersionMap`, as well as return an upgraded `VersionMap` and an error: + +```diff +- type UpgradeHandler func(ctx sdk.Context, plan Plan) ++ type UpgradeHandler func(ctx sdk.Context, plan Plan, versionMap VersionMap) (VersionMap, error) +``` + +To apply an upgrade, we query the `VersionMap` from the `x/upgrade` store and pass it into the handler. The handler runs the actual migration functions (see next section), and if successful, returns an updated `VersionMap` to be stored in state. + +```diff expandable +func (k UpgradeKeeper) ApplyUpgrade(ctx sdk.Context, plan types.Plan) { + // --snip-- +- handler(ctx, plan) ++ updatedVM, err := handler(ctx, plan, k.GetModuleVersionMap(ctx)) // k.GetModuleVersionMap() fetches the VersionMap stored in state. ++ if err != nil { ++ return err ++ } ++ ++ // Set the updated consensus versions to state ++ k.SetModuleVersionMap(ctx, updatedVM) +} +``` + +A gRPC query endpoint to query the `VersionMap` stored in `x/upgrade`'s state will also be added, so that app developers can double-check the `VersionMap` before the upgrade handler runs. + +### Running Migrations + +Once all the migration handlers are registered inside the configurator (which happens at startup), running migrations can happen by calling the `RunMigrations` method on `module.Manager`. This function will loop through all modules, and for each module: + +* Get the old ConsensusVersion of the module from its `VersionMap` argument (let's call it `M`). +* Fetch the new ConsensusVersion of the module from the `ConsensusVersion()` method on `AppModule` (call it `N`). +* If `N>M`, run all registered migrations for the module sequentially `M -> M+1 -> M+2...` until `N`. + * There is a special case where there is no ConsensusVersion for the module, as this means that the module has been newly added during the upgrade. In this case, no migration function is run, and the module's current ConsensusVersion is saved to `x/upgrade`'s store. + +If a required migration is missing (e.g. if it has not been registered in the `Configurator`), then the `RunMigrations` function will error. + +In practice, the `RunMigrations` method should be called from inside an `UpgradeHandler`. + +```go +app.UpgradeKeeper.SetUpgradeHandler("my-plan", func(ctx sdk.Context, plan upgradetypes.Plan, vm module.VersionMap) (module.VersionMap, error) { + return app.mm.RunMigrations(ctx, vm) +}) +``` + +Assuming a chain upgrades at block `n`, the procedure should run as follows: + +* the old binary will halt in `BeginBlock` when starting block `N`. In its store, the ConsensusVersions of the old binary's modules are stored. +* the new binary will start at block `N`. The UpgradeHandler is set in the new binary, so will run at `BeginBlock` of the new binary. Inside `x/upgrade`'s `ApplyUpgrade`, the `VersionMap` will be retrieved from the (old binary's) store, and passed into the `RunMigrations` functon, migrating all module stores in-place before the modules' own `BeginBlock`s. + +## Consequences + +### Backwards Compatibility + +This ADR introduces a new method `ConsensusVersion()` on `AppModule`, which all modules need to implement. It also alters the UpgradeHandler function signature. As such, it is not backwards-compatible. + +While modules MUST register their migration functions when bumping ConsensusVersions, running those scripts using an upgrade handler is optional. An application may perfectly well decide to not call the `RunMigrations` inside its upgrade handler, and continue using the legacy JSON migration path. + +### Positive + +* Perform chain upgrades without manipulating JSON files. +* While no benchmark has been made yet, it is probable that in-place store migrations will take less time than JSON migrations. The main reason supporting this claim is that both the `simd export` command on the old binary and the `InitChain` function on the new binary will be skipped. + +### Negative + +* Module developers MUST correctly track consensus-breaking changes in their modules. If a consensus-breaking change is introduced in a module without its corresponding `ConsensusVersion()` bump, then the `RunMigrations` function won't detect the migration, and the chain upgrade might be unsuccessful. Documentation should clearly reflect this. + +### Neutral + +* The Cosmos SDK will continue to support JSON migrations via the existing `simd export` and `simd genesis migrate` commands. +* The current ADR does not allow creating, renaming or deleting stores, only modifying existing store keys and values. The Cosmos SDK already has the `StoreLoader` for those operations. + +## Further Discussions + +## References + +* Initial discussion: [Link](https://github.com/cosmos/cosmos-sdk/discussions/8429) +* Implementation of `ConsensusVersion` and `RunMigrations`: [Link](https://github.com/cosmos/cosmos-sdk/pull/8485) +* Issue discussing `x/upgrade` design: [Link](https://github.com/cosmos/cosmos-sdk/issues/8514) diff --git a/sdk/v0.54/reference/architecture/adr-042-group-module.mdx b/sdk/v0.54/reference/architecture/adr-042-group-module.mdx new file mode 100644 index 000000000..a37791070 --- /dev/null +++ b/sdk/v0.54/reference/architecture/adr-042-group-module.mdx @@ -0,0 +1,291 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/architecture/adr-042-group-module' +title: 'ADR 042: Group Module' +description: '2020/04/09: Initial Draft' +--- + +## Changelog + +* 2020/04/09: Initial Draft + +## Status + +Draft + +## Abstract + +This ADR defines the `x/group` module which allows the creation and management of on-chain multi-signature accounts and enables voting for message execution based on configurable decision policies. + +## Context + +The legacy amino multi-signature mechanism of the Cosmos SDK has certain limitations: + +* Key rotation is not possible, although this can be solved with [account rekeying](/sdk/v0.54/reference/architecture/adr-034-account-rekeying). +* Thresholds can't be changed. +* UX is cumbersome for non-technical users ([#5661](https://github.com/cosmos/cosmos-sdk/issues/5661)). +* It requires `legacy_amino` sign mode ([#8141](https://github.com/cosmos/cosmos-sdk/issues/8141)). + +While the group module is not meant to be a total replacement for the current multi-signature accounts, it provides a solution to the limitations described above, with a more flexible key management system where keys can be added, updated or removed, as well as configurable thresholds. +It's meant to be used with other access control modules such as [`x/feegrant`](/sdk/v0.50/build/architecture/adr-029-fee-grant-module) ans [`x/authz`](/sdk/v0.54/reference/architecture/adr-030-authz-module) to simplify key management for individuals and organizations. + +The proof of concept of the group module can be found in [Link](https://github.com/regen-network/regen-ledger/tree/master/proto/regen/group/v1alpha1) and [Link](https://github.com/regen-network/regen-ledger/tree/master/x/group). + +## Decision + +We propose merging the `x/group` module with its supporting [ORM/Table Store package](https://github.com/regen-network/regen-ledger/tree/master/orm) ([#7098](https://github.com/cosmos/cosmos-sdk/issues/7098)) into the Cosmos SDK and continuing development here. There will be a dedicated ADR for the ORM package. + +### Group + +A group is a composition of accounts with associated weights. It is not +an account and doesn't have a balance. It doesn't in and of itself have any +sort of voting or decision weight. +Group members can create proposals and vote on them through group accounts using different decision policies. + +It has an `admin` account which can manage members in the group, update the group +metadata and set a new admin. + +```protobuf expandable +message GroupInfo { + + // group_id is the unique ID of this group. + uint64 group_id = 1; + + // admin is the account address of the group's admin. + string admin = 2; + + // metadata is any arbitrary metadata to attached to the group. + bytes metadata = 3; + + // version is used to track changes to a group's membership structure that + // would break existing proposals. Whenever a member weight has changed, + // or any member is added or removed, the version is incremented and will + // invalidate all proposals from older versions. + uint64 version = 4; + + // total_weight is the sum of the group members' weights. + string total_weight = 5; +} +``` + +```protobuf expandable +message GroupMember { + + // group_id is the unique ID of the group. + uint64 group_id = 1; + + // member is the member data. + Member member = 2; +} + +// Member represents a group member with an account address, +// non-zero weight and metadata. +message Member { + + // address is the member's account address. + string address = 1; + + // weight is the member's voting weight that should be greater than 0. + string weight = 2; + + // metadata is any arbitrary metadata to attached to the member. + bytes metadata = 3; +} +``` + +### Group Account + +A group account is an account associated with a group and a decision policy. +A group account does have a balance. + +Group accounts are abstracted from groups because a single group may have +multiple decision policies for different types of actions. Managing group +membership separately from decision policies results in the least overhead +and keeps membership consistent across different policies. The pattern that +is recommended is to have a single master group account for a given group, +and then to create separate group accounts with different decision policies +and delegate the desired permissions from the master account to +those "sub-accounts" using the [`x/authz` module](/sdk/v0.54/reference/architecture/adr-030-authz-module). + +```protobuf expandable +message GroupAccountInfo { + + // address is the group account address. + string address = 1; + + // group_id is the ID of the Group the GroupAccount belongs to. + uint64 group_id = 2; + + // admin is the account address of the group admin. + string admin = 3; + + // metadata is any arbitrary metadata of this group account. + bytes metadata = 4; + + // version is used to track changes to a group's GroupAccountInfo structure that + // invalidates active proposal from old versions. + uint64 version = 5; + + // decision_policy specifies the group account's decision policy. + google.protobuf.Any decision_policy = 6 [(cosmos_proto.accepts_interface) = "cosmos.group.v1.DecisionPolicy"]; +} +``` + +Similarly to a group admin, a group account admin can update its metadata, decision policy or set a new group account admin. + +A group account can also be an admin or a member of a group. +For instance, a group admin could be another group account which could "elects" the members or it could be the same group that elects itself. + +### Decision Policy + +A decision policy is the mechanism by which members of a group can vote on +proposals. + +All decision policies should have a minimum and maximum voting window. +The minimum voting window is the minimum duration that must pass in order +for a proposal to potentially pass, and it may be set to 0. The maximum voting +window is the maximum time that a proposal may be voted on and executed if +it reached enough support before it is closed. +Both of these values must be less than a chain-wide max voting window parameter. + +We define the `DecisionPolicy` interface that all decision policies must implement: + +```go expandable +type DecisionPolicy interface { + codec.ProtoMarshaler + + ValidateBasic() + +error + GetTimeout() + +types.Duration + Allow(tally Tally, totalPower string, votingDuration time.Duration) (DecisionPolicyResult, error) + +Validate(g GroupInfo) + +error +} + +type DecisionPolicyResult struct { + Allow bool + Final bool +} +``` + +#### Threshold decision policy + +A threshold decision policy defines a minimum support votes (*yes*), based on a tally +of voter weights, for a proposal to pass. For +this decision policy, abstain and veto are treated as no support (*no*). + +```protobuf +message ThresholdDecisionPolicy { + + // threshold is the minimum weighted sum of support votes for a proposal to succeed. + string threshold = 1; + + // voting_period is the duration from submission of a proposal to the end of voting period + // Within this period, votes and exec messages can be submitted. + google.protobuf.Duration voting_period = 2 [(gogoproto.nullable) = false]; +} +``` + +### Proposal + +Any member of a group can submit a proposal for a group account to decide upon. +A proposal consists of a set of `sdk.Msg`s that will be executed if the proposal +passes as well as any metadata associated with the proposal. These `sdk.Msg`s get validated as part of the `Msg/CreateProposal` request validation. They should also have their signer set as the group account. + +Internally, a proposal also tracks: + +* its current `Status`: submitted, closed or aborted +* its `Result`: unfinalized, accepted or rejected +* its `VoteState` in the form of a `Tally`, which is calculated on new votes and when executing the proposal. + +```protobuf expandable +// Tally represents the sum of weighted votes. +message Tally { + option (gogoproto.goproto_getters) = false; + + // yes_count is the weighted sum of yes votes. + string yes_count = 1; + + // no_count is the weighted sum of no votes. + string no_count = 2; + + // abstain_count is the weighted sum of abstainers. + string abstain_count = 3; + + // veto_count is the weighted sum of vetoes. + string veto_count = 4; +} +``` + +### Voting + +Members of a group can vote on proposals. There are four choices to choose while voting - yes, no, abstain and veto. Not +all decision policies will support them. Votes can contain some optional metadata. +In the current implementation, the voting window begins as soon as a proposal +is submitted. + +Voting internally updates the proposal `VoteState` as well as `Status` and `Result` if needed. + +### Executing Proposals + +Proposals will not be automatically executed by the chain in this current design, +but rather a user must submit a `Msg/Exec` transaction to attempt to execute the +proposal based on the current votes and decision policy. A future upgrade could +automate this and have the group account (or a fee granter) pay. + +#### Changing Group Membership + +In the current implementation, updating a group or a group account after submitting a proposal will make it invalid. It will simply fail if someone calls `Msg/Exec` and will eventually be garbage collected. + +### Notes on current implementation + +This section outlines the current implementation used in the proof of concept of the group module but this could be subject to changes and iterated on. + +#### ORM + +The [ORM package](https://github.com/cosmos/cosmos-sdk/discussions/9156) defines tables, sequences and secondary indexes which are used in the group module. + +Groups are stored in state as part of a `groupTable`, the `group_id` being an auto-increment integer. Group members are stored in a `groupMemberTable`. + +Group accounts are stored in a `groupAccountTable`. The group account address is generated based on an auto-increment integer which is used to derive the group module `RootModuleKey` into a `DerivedModuleKey`, as stated in [ADR-033](/sdk/v0.54/reference/architecture/adr-033-protobuf-inter-module-comm#modulekeys-and-moduleids). The group account is added as a new `ModuleAccount` through `x/auth`. + +Proposals are stored as part of the `proposalTable` using the `Proposal` type. The `proposal_id` is an auto-increment integer. + +Votes are stored in the `voteTable`. The primary key is based on the vote's `proposal_id` and `voter` account address. + +#### ADR-033 to route proposal messages + +Inter-module communication introduced by [ADR-033](/sdk/v0.54/reference/architecture/adr-033-protobuf-inter-module-comm) can be used to route a proposal's messages using the `DerivedModuleKey` corresponding to the proposal's group account. + +## Consequences + +### Positive + +* Improved UX for multi-signature accounts allowing key rotation and custom decision policies. + +### Negative + +### Neutral + +* It uses ADR 033 so it will need to be implemented within the Cosmos SDK, but this doesn't imply necessarily any large refactoring of existing Cosmos SDK modules. +* The current implementation of the group module uses the ORM package. + +## Further Discussions + +* Convergence of `/group` and `x/gov` as both support proposals and voting: [Link](https://github.com/cosmos/cosmos-sdk/discussions/9066) +* `x/group` possible future improvements: + * Execute proposals on submission ([Link](https://github.com/regen-network/regen-ledger/issues/288)) + * Withdraw a proposal ([Link](https://github.com/regen-network/cosmos-modules/issues/41)) + * Make `Tally` more flexible and support non-binary choices + +## References + +* Initial specification: + * [Link](https://gist.github.com/aaronc/b60628017352df5983791cad30babe56#group-module) + * [#5236](https://github.com/cosmos/cosmos-sdk/pull/5236) +* Proposal to add `x/group` into the Cosmos SDK: [#7633](https://github.com/cosmos/cosmos-sdk/issues/7633) diff --git a/sdk/v0.54/reference/architecture/adr-043-nft-module.mdx b/sdk/v0.54/reference/architecture/adr-043-nft-module.mdx new file mode 100644 index 000000000..67578faf4 --- /dev/null +++ b/sdk/v0.54/reference/architecture/adr-043-nft-module.mdx @@ -0,0 +1,383 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/architecture/adr-043-nft-module' +title: 'ADR 43: NFT Module' +description: >- + 2021-05-01: Initial Draft 2021-07-02: Review updates 2022-06-15: Add batch + operation 2022-11-11: Remove strict validation of classID and tokenID +--- + +## Changelog + +* 2021-05-01: Initial Draft +* 2021-07-02: Review updates +* 2022-06-15: Add batch operation +* 2022-11-11: Remove strict validation of classID and tokenID + +## Status + +PROPOSED + +## Abstract + +This ADR defines the `x/nft` module which is a generic implementation of NFTs, roughly "compatible" with ERC721. **Applications using the `x/nft` module must implement the following functions**: + +* `MsgNewClass` - Receive the user's request to create a class, and call the `NewClass` of the `x/nft` module. +* `MsgUpdateClass` - Receive the user's request to update a class, and call the `UpdateClass` of the `x/nft` module. +* `MsgMintNFT` - Receive the user's request to mint a nft, and call the `MintNFT` of the `x/nft` module. +* `BurnNFT` - Receive the user's request to burn a nft, and call the `BurnNFT` of the `x/nft` module. +* `UpdateNFT` - Receive the user's request to update a nft, and call the `UpdateNFT` of the `x/nft` module. + +## Context + +NFTs are more than just crypto art, which is very helpful for accruing value to the Cosmos ecosystem. As a result, Cosmos Hub should implement NFT functions and enable a unified mechanism for storing and sending the ownership representative of NFTs as discussed in [Link](https://github.com/cosmos/cosmos-sdk/discussions/9065). + +As discussed in [#9065](https://github.com/cosmos/cosmos-sdk/discussions/9065), several potential solutions can be considered: + +* irismod/nft and modules/incubator/nft +* CW721 +* DID NFTs +* interNFT + +Since functions/use cases of NFTs are tightly connected with their logic, it is almost impossible to support all the NFTs' use cases in one Cosmos SDK module by defining and implementing different transaction types. + +Considering generic usage and compatibility of interchain protocols including IBC and Gravity Bridge, it is preferred to have a generic NFT module design which handles the generic NFTs logic. +This design idea can enable composability that application-specific functions should be managed by other modules on Cosmos Hub or on other Zones by importing the NFT module. + +The current design is based on the work done by [IRISnet team](https://github.com/irisnet/irismod/tree/master/modules/nft) and an older implementation in the [Cosmos repository](https://github.com/cosmos/modules/tree/master/incubator/nft). + +## Decision + +We create a `x/nft` module, which contains the following functionality: + +* Store NFTs and track their ownership. +* Expose `Keeper` interface for composing modules to transfer, mint and burn NFTs. +* Expose external `Message` interface for users to transfer ownership of their NFTs. +* Query NFTs and their supply information. + +The proposed module is a base module for NFT app logic. It's goal it to provide a common layer for storage, basic transfer functionality and IBC. The module should not be used as a standalone. +Instead an app should create a specialized module to handle app specific logic (eg: NFT ID construction, royalty), user level minting and burning. Moreover an app specialized module should handle auxiliary data to support the app logic (eg indexes, ORM, business data). + +All data carried over IBC must be part of the `NFT` or `Class` type described below. The app specific NFT data should be encoded in `NFT.data` for cross-chain integrity. Other objects related to NFT, which are not important for integrity can be part of the app specific module. + +### Types + +We propose two main types: + +* `Class` -- describes NFT class. We can think about it as a smart contract address. +* `NFT` -- object representing unique, non fungible asset. Each NFT is associated with a Class. + +#### Class + +NFT **Class** is comparable to an ERC-721 smart contract (provides description of a smart contract), under which a collection of NFTs can be created and managed. + +```protobuf +message Class { + string id = 1; + string name = 2; + string symbol = 3; + string description = 4; + string uri = 5; + string uri_hash = 6; + google.protobuf.Any data = 7; +} +``` + +* `id` is used as the primary index for storing the class; *required* +* `name` is a descriptive name of the NFT class; *optional* +* `symbol` is the symbol usually shown on exchanges for the NFT class; *optional* +* `description` is a detailed description of the NFT class; *optional* +* `uri` is a URI for the class metadata stored off chain. It should be a JSON file that contains metadata about the NFT class and NFT data schema ([OpenSea example](https://docs.opensea.io/docs/contract-level-metadata)); *optional* +* `uri_hash` is a hash of the document pointed by uri; *optional* +* `data` is app specific metadata of the class; *optional* + +#### NFT + +We define a general model for `NFT` as follows. + +```protobuf +message NFT { + string class_id = 1; + string id = 2; + string uri = 3; + string uri_hash = 4; + google.protobuf.Any data = 10; +} +``` + +* `class_id` is the identifier of the NFT class where the NFT belongs; *required* + +* `id` is an identifier of the NFT, unique within the scope of its class. It is specified by the creator of the NFT and may be expanded to use DID in the future. `class_id` combined with `id` uniquely identifies an NFT and is used as the primary index for storing the NFT; *required* + + ```text + {class_id}/{id} --> NFT (bytes) + ``` + +* `uri` is a URI for the NFT metadata stored off chain. Should point to a JSON file that contains metadata about this NFT (Ref: [ERC721 standard and OpenSea extension](https://docs.opensea.io/docs/metadata-standards)); *required* + +* `uri_hash` is a hash of the document pointed by uri; *optional* + +* `data` is an app specific data of the NFT. CAN be used by composing modules to specify additional properties of the NFT; *optional* + +This ADR doesn't specify values that `data` can take; however, best practices recommend upper-level NFT modules clearly specify their contents. Although the value of this field doesn't provide the additional context required to manage NFT records, which means that the field can technically be removed from the specification, the field's existence allows basic informational/UI functionality. + +### `Keeper` Interface + +```go expandable +type Keeper interface { + NewClass(ctx sdk.Context,class Class) + +UpdateClass(ctx sdk.Context,class Class) + +Mint(ctx sdk.Context,nft NFT，receiver sdk.AccAddress) // updates totalSupply + BatchMint(ctx sdk.Context, tokens []NFT,receiver sdk.AccAddress) + +error + + Burn(ctx sdk.Context, classId string, nftId string) // updates totalSupply + BatchBurn(ctx sdk.Context, classID string, nftIDs []string) + +error + + Update(ctx sdk.Context, nft NFT) + +BatchUpdate(ctx sdk.Context, tokens []NFT) + +error + + Transfer(ctx sdk.Context, classId string, nftId string, receiver sdk.AccAddress) + +BatchTransfer(ctx sdk.Context, classID string, nftIDs []string, receiver sdk.AccAddress) + +error + + GetClass(ctx sdk.Context, classId string) + +Class + GetClasses(ctx sdk.Context) []Class + + GetNFT(ctx sdk.Context, classId string, nftId string) + +NFT + GetNFTsOfClassByOwner(ctx sdk.Context, classId string, owner sdk.AccAddress) []NFT + GetNFTsOfClass(ctx sdk.Context, classId string) []NFT + + GetOwner(ctx sdk.Context, classId string, nftId string) + +sdk.AccAddress + GetBalance(ctx sdk.Context, classId string, owner sdk.AccAddress) + +uint64 + GetTotalSupply(ctx sdk.Context, classId string) + +uint64 +} +``` + +Other business logic implementations should be defined in composing modules that import `x/nft` and use its `Keeper`. + +### `Msg` Service + +```protobuf expandable +service Msg { + rpc Send(MsgSend) returns (MsgSendResponse); +} + +message MsgSend { + string class_id = 1; + string id = 2; + string sender = 3; + string reveiver = 4; +} +message MsgSendResponse {} +``` + +`MsgSend` can be used to transfer the ownership of an NFT to another address. + +The implementation outline of the server is as follows: + +```go expandable +type msgServer struct{ + k Keeper +} + +func (m msgServer) + +Send(ctx context.Context, msg *types.MsgSend) (*types.MsgSendResponse, error) { + // check current ownership + assertEqual(msg.Sender, m.k.GetOwner(msg.ClassId, msg.Id)) + + // transfer ownership + m.k.Transfer(msg.ClassId, msg.Id, msg.Receiver) + +return &types.MsgSendResponse{ +}, nil +} +``` + +The query service methods for the `x/nft` module are: + +```protobuf expandable +service Query { + // Balance queries the number of NFTs of a given class owned by the owner, same as balanceOf in ERC721 + rpc Balance(QueryBalanceRequest) returns (QueryBalanceResponse) { + option (google.api.http).get = "/cosmos/nft/v1beta1/balance/{owner}/{class_id}"; + } + + // Owner queries the owner of the NFT based on its class and id, same as ownerOf in ERC721 + rpc Owner(QueryOwnerRequest) returns (QueryOwnerResponse) { + option (google.api.http).get = "/cosmos/nft/v1beta1/owner/{class_id}/{id}"; + } + + // Supply queries the number of NFTs from the given class, same as totalSupply of ERC721. + rpc Supply(QuerySupplyRequest) returns (QuerySupplyResponse) { + option (google.api.http).get = "/cosmos/nft/v1beta1/supply/{class_id}"; + } + + // NFTs queries all NFTs of a given class or owner,choose at least one of the two, similar to tokenByIndex in ERC721Enumerable + rpc NFTs(QueryNFTsRequest) returns (QueryNFTsResponse) { + option (google.api.http).get = "/cosmos/nft/v1beta1/nfts"; + } + + // NFT queries an NFT based on its class and id. + rpc NFT(QueryNFTRequest) returns (QueryNFTResponse) { + option (google.api.http).get = "/cosmos/nft/v1beta1/nfts/{class_id}/{id}"; + } + + // Class queries an NFT class based on its id + rpc Class(QueryClassRequest) returns (QueryClassResponse) { + option (google.api.http).get = "/cosmos/nft/v1beta1/classes/{class_id}"; + } + + // Classes queries all NFT classes + rpc Classes(QueryClassesRequest) returns (QueryClassesResponse) { + option (google.api.http).get = "/cosmos/nft/v1beta1/classes"; + } +} + +// QueryBalanceRequest is the request type for the Query/Balance RPC method +message QueryBalanceRequest { + string class_id = 1; + string owner = 2; +} + +// QueryBalanceResponse is the response type for the Query/Balance RPC method +message QueryBalanceResponse { + uint64 amount = 1; +} + +// QueryOwnerRequest is the request type for the Query/Owner RPC method +message QueryOwnerRequest { + string class_id = 1; + string id = 2; +} + +// QueryOwnerResponse is the response type for the Query/Owner RPC method +message QueryOwnerResponse { + string owner = 1; +} + +// QuerySupplyRequest is the request type for the Query/Supply RPC method +message QuerySupplyRequest { + string class_id = 1; +} + +// QuerySupplyResponse is the response type for the Query/Supply RPC method +message QuerySupplyResponse { + uint64 amount = 1; +} + +// QueryNFTstRequest is the request type for the Query/NFTs RPC method +message QueryNFTsRequest { + string class_id = 1; + string owner = 2; + cosmos.base.query.v1beta1.PageRequest pagination = 3; +} + +// QueryNFTsResponse is the response type for the Query/NFTs RPC methods +message QueryNFTsResponse { + repeated cosmos.nft.v1beta1.NFT nfts = 1; + cosmos.base.query.v1beta1.PageResponse pagination = 2; +} + +// QueryNFTRequest is the request type for the Query/NFT RPC method +message QueryNFTRequest { + string class_id = 1; + string id = 2; +} + +// QueryNFTResponse is the response type for the Query/NFT RPC method +message QueryNFTResponse { + cosmos.nft.v1beta1.NFT nft = 1; +} + +// QueryClassRequest is the request type for the Query/Class RPC method +message QueryClassRequest { + string class_id = 1; +} + +// QueryClassResponse is the response type for the Query/Class RPC method +message QueryClassResponse { + cosmos.nft.v1beta1.Class class = 1; +} + +// QueryClassesRequest is the request type for the Query/Classes RPC method +message QueryClassesRequest { + // pagination defines an optional pagination for the request. + cosmos.base.query.v1beta1.PageRequest pagination = 1; +} + +// QueryClassesResponse is the response type for the Query/Classes RPC method +message QueryClassesResponse { + repeated cosmos.nft.v1beta1.Class classes = 1; + cosmos.base.query.v1beta1.PageResponse pagination = 2; +} +``` + +### Interoperability + +Interoperability is all about reusing assets between modules and chains. The former one is achieved by ADR-33: Protobuf client - server communication. At the time of writing ADR-33 is not finalized. The latter is achieved by IBC. Here we will focus on the IBC side. +IBC is implemented per module. Here, we aligned that NFTs will be recorded and managed in the x/nft. This requires creation of a new IBC standard and implementation of it. + +For IBC interoperability, NFT custom modules MUST use the NFT object type understood by the IBC client. So, for x/nft interoperability, custom NFT implementations (example: x/cryptokitty) should use the canonical x/nft module and proxy all NFT balance keeping functionality to x/nft or else re-implement all functionality using the NFT object type understood by the IBC client. In other words: x/nft becomes the standard NFT registry for all Cosmos NFTs (example: x/cryptokitty will register a kitty NFT in x/nft and use x/nft for book keeping). This was [discussed](https://github.com/cosmos/cosmos-sdk/discussions/9065#discussioncomment-873206) in the context of using x/bank as a general asset balance book. Not using x/nft will require implementing another module for IBC. + +## Consequences + +### Backward Compatibility + +No backward incompatibilities. + +### Forward Compatibility + +This specification conforms to the ERC-721 smart contract specification for NFT identifiers. Note that ERC-721 defines uniqueness based on (contract address, uint256 tokenId), and we conform to this implicitly because a single module is currently aimed to track NFT identifiers. Note: use of the (mutable) data field to determine uniqueness is not safe.s + +### Positive + +* NFT identifiers available on Cosmos Hub. +* Ability to build different NFT modules for the Cosmos Hub, e.g., ERC-721. +* NFT module which supports interoperability with IBC and other cross-chain infrastructures like Gravity Bridge + +### Negative + +* New IBC app is required for x/nft +* CW721 adapter is required + +### Neutral + +* Other functions need more modules. For example, a custody module is needed for NFT trading function, a collectible module is needed for defining NFT properties. + +## Further Discussions + +For other kinds of applications on the Hub, more app-specific modules can be developed in the future: + +* `x/nft/custody`: custody of NFTs to support trading functionality. +* `x/nft/marketplace`: selling and buying NFTs using sdk.Coins. +* `x/fractional`: a module to split an ownership of an asset (NFT or other assets) for multiple stakeholder. `x/group` should work for most of the cases. + +Other networks in the Cosmos ecosystem could design and implement their own NFT modules for specific NFT applications and use cases. + +## References + +* Initial discussion: [Link](https://github.com/cosmos/cosmos-sdk/discussions/9065) +* x/nft: initialize module: [Link](https://github.com/cosmos/cosmos-sdk/pull/9174) +* [ADR 033](https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-033-protobuf-inter-module-comm.md) diff --git a/sdk/v0.54/reference/architecture/adr-044-protobuf-updates-guidelines.mdx b/sdk/v0.54/reference/architecture/adr-044-protobuf-updates-guidelines.mdx new file mode 100644 index 000000000..e75613874 --- /dev/null +++ b/sdk/v0.54/reference/architecture/adr-044-protobuf-updates-guidelines.mdx @@ -0,0 +1,136 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/architecture/adr-044-protobuf-updates-guidelines' +title: 'ADR 044: Guidelines for Updating Protobuf Definitions' +description: >- + 28.06.2021: Initial Draft 02.12.2021: Add Since: comment for new fields + 21.07.2022: Remove the rule of no new Msg in the same proto version. +--- + +## Changelog + +* 28.06.2021: Initial Draft +* 02.12.2021: Add `Since:` comment for new fields +* 21.07.2022: Remove the rule of no new `Msg` in the same proto version. + +## Status + +Draft + +## Abstract + +This ADR provides guidelines and recommended practices when updating Protobuf definitions. These guidelines are targeting module developers. + +## Context + +The Cosmos SDK maintains a set of [Protobuf definitions](https://github.com/cosmos/cosmos-sdk/tree/main/proto/cosmos). It is important to correctly design Protobuf definitions to avoid any breaking changes within the same version. The reasons are to not break tooling (including indexers and explorers), wallets and other third-party integrations. + +When making changes to these Protobuf definitions, the Cosmos SDK currently only follows [Buf's](https://docs.buf.build/) recommendations. We noticed however that Buf's recommendations might still result in breaking changes in the SDK in some cases. For example: + +* Adding fields to `Msg`s. Adding fields is a not a Protobuf spec-breaking operation. However, when adding new fields to `Msg`s, the unknown field rejection will throw an error when sending the new `Msg` to an older node. +* Marking fields as `reserved`. Protobuf proposes the `reserved` keyword for removing fields without the need to bump the package version. However, by doing so, client backwards compatibility is broken as Protobuf doesn't generate anything for `reserved` fields. See [#9446](https://github.com/cosmos/cosmos-sdk/issues/9446) for more details on this issue. + +Moreover, module developers often face other questions around Protobuf definitions such as "Can I rename a field?" or "Can I deprecate a field?" This ADR aims to answer all these questions by providing clear guidelines about allowed updates for Protobuf definitions. + +## Decision + +We decide to keep [Buf's](https://docs.buf.build/) recommendations with the following exceptions: + +* `UNARY_RPC`: the Cosmos SDK currently does not support streaming RPCs. +* `COMMENT_FIELD`: the Cosmos SDK allows fields with no comments. +* `SERVICE_SUFFIX`: we use the `Query` and `Msg` service naming convention, which doesn't use the `-Service` suffix. +* `PACKAGE_VERSION_SUFFIX`: some packages, such as `cosmos.crypto.ed25519`, don't use a version suffix. +* `RPC_REQUEST_STANDARD_NAME`: Requests for the `Msg` service don't have the `-Request` suffix to keep backwards compatibility. + +On top of Buf's recommendations we add the following guidelines that are specific to the Cosmos SDK. + +### Updating Protobuf Definition Without Bumping Version + +#### 1. Module developers MAY add new Protobuf definitions + +Module developers MAY add new `message`s, new `Service`s, new `rpc` endpoints, and new fields to existing messages. This recommendation follows the Protobuf specification, but is added in this document for clarity, as the SDK requires one additional change. + +The SDK requires the Protobuf comment of the new addition to contain one line with the following format: + +```protobuf +// Since: cosmos-sdk {, ...} +``` + +Where each `version` denotes a minor ("0.45") or patch ("0.44.5") version from which the field is available. This will greatly help client libraries, who can optionally use reflection or custom code generation to show/hide these fields depending on the targetted node version. + +As examples, the following comments are valid: + +```protobuf +// Since: cosmos-sdk 0.44 + +// Since: cosmos-sdk 0.42.11, 0.44.5 +``` + +and the following ones are NOT valid: + +```protobuf +// Since cosmos-sdk v0.44 + +// since: cosmos-sdk 0.44 + +// Since: cosmos-sdk 0.42.11 0.44.5 + +// Since: Cosmos SDK 0.42.11, 0.44.5 +``` + +#### 2. Fields MAY be marked as `deprecated`, and nodes MAY implement a protocol-breaking change for handling these fields + +Protobuf supports the [`deprecated` field option](https://developers.google.com/protocol-buffers/docs/proto#options), and this option MAY be used on any field, including `Msg` fields. If a node handles a Protobuf message with a non-empty deprecated field, the node MAY change its behavior upon processing it, even in a protocol-breaking way. When possible, the node MUST handle backwards compatibility without breaking the consensus (unless we increment the proto version). + +As an example, the Cosmos SDK v0.42 to v0.43 update contained two Protobuf-breaking changes, listed below. Instead of bumping the package versions from `v1beta1` to `v1`, the SDK team decided to follow this guideline, by reverting the breaking changes, marking those changes as deprecated, and modifying the node implementation when processing messages with deprecated fields. More specifically: + +* The Cosmos SDK recently removed support for [time-based software upgrades](https://github.com/cosmos/cosmos-sdk/pull/8849). As such, the `time` field has been marked as deprecated in `cosmos.upgrade.v1beta1.Plan`. Moreover, the node will reject any proposal containing an upgrade Plan whose `time` field is non-empty. +* The Cosmos SDK now supports [governance split votes](/sdk/v0.50/build/architecture/adr-037-gov-split-vote). When querying for votes, the returned `cosmos.gov.v1beta1.Vote` message has its `option` field (used for 1 vote option) deprecated in favor of its `options` field (allowing multiple vote options). Whenever possible, the SDK still populates the deprecated `option` field, that is, if and only if the `len(options) == 1` and `options[0].Weight == 1.0`. + +#### 3. Fields MUST NOT be renamed + +Whereas the official Protobuf recommendations do not prohibit renaming fields, as it does not break the Protobuf binary representation, the SDK explicitly forbids renaming fields in Protobuf structs. The main reason for this choice is to avoid introducing breaking changes for clients, which often rely on hard-coded fields from generated types. Moreover, renaming fields will lead to client-breaking JSON representations of Protobuf definitions, used in REST endpoints and in the CLI. + +### Incrementing Protobuf Package Version + +TODO, needs architecture review. Some topics: + +* Bumping versions frequency +* When bumping versions, should the Cosmos SDK support both versions? + * i.e. v1beta1 -> v1, should we have two folders in the Cosmos SDK, and handlers for both versions? +* mention ADR-023 Protobuf naming + +## Consequences + +> This section describes the resulting context, after applying the decision. All consequences should be listed here, not just the "positive" ones. A particular decision may have positive, negative, and neutral consequences, but all of them affect the team and project in the future. + +### Backwards Compatibility + +> All ADRs that introduce backwards incompatibilities must include a section describing these incompatibilities and their severity. The ADR must explain how the author proposes to deal with these incompatibilities. ADR submissions without a sufficient backwards compatibility treatise may be rejected outright. + +### Positive + +* less pain to tool developers +* more compatibility in the ecosystem +* ... + +### Negative + +`{negative consequences}` + +### Neutral + +* more rigor in Protobuf review + +## Further Discussions + +This ADR is still in the DRAFT stage, and the "Incrementing Protobuf Package Version" will be filled in once we make a decision on how to correctly do it. + +## Test Cases \[optional] + +Test cases for an implementation are mandatory for ADRs that are affecting consensus changes. Other ADRs can choose to include links to test cases if applicable. + +## References + +* [#9445](https://github.com/cosmos/cosmos-sdk/issues/9445) Release proto definitions v1 +* [#9446](https://github.com/cosmos/cosmos-sdk/issues/9446) Address v1beta1 proto breaking changes diff --git a/sdk/v0.54/reference/architecture/adr-045-check-delivertx-middlewares.mdx b/sdk/v0.54/reference/architecture/adr-045-check-delivertx-middlewares.mdx new file mode 100644 index 000000000..32d0d5abe --- /dev/null +++ b/sdk/v0.54/reference/architecture/adr-045-check-delivertx-middlewares.mdx @@ -0,0 +1,346 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/architecture/adr-045-check-delivertx-middlewares' +description: >- + 20.08.2021: Initial draft. 07.12.2021: Update tx.Handler interface (\#10693). + 17.05.2022: ADR is abandoned, as middlewares are deemed too hard to reason + about. +--- + +## Changelog + +* 20.08.2021: Initial draft. +* 07.12.2021: Update `tx.Handler` interface ([#10693](https://github.com/cosmos/cosmos-sdk/pull/10693)). +* 17.05.2022: ADR is abandoned, as middlewares are deemed too hard to reason about. + +## Status + +ABANDONED. Replacement is being discussed in [#11955](https://github.com/cosmos/cosmos-sdk/issues/11955). + +## Abstract + +This ADR replaces the current BaseApp `runTx` and antehandlers design with a middleware-based design. + +## Context + +BaseApp's implementation of ABCI `{Check,Deliver}Tx()` and its own `Simulate()` method call the `runTx` method under the hood, which first runs antehandlers, then executes `Msg`s. However, the [transaction Tips](https://github.com/cosmos/cosmos-sdk/issues/9406) and [refunding unused gas](https://github.com/cosmos/cosmos-sdk/issues/2150) use cases require custom logic to be run after the `Msg`s execution. There is currently no way to achieve this. + +An naive solution would be to add post-`Msg` hooks to BaseApp. However, the Cosmos SDK team thinks in parallel about the bigger picture of making app wiring simpler ([#9181](https://github.com/cosmos/cosmos-sdk/discussions/9182)), which includes making BaseApp more lightweight and modular. + +## Decision + +We decide to transform Baseapp's implementation of ABCI `{Check,Deliver}Tx` and its own `Simulate` methods to use a middleware-based design. + +The two following interfaces are the base of the middleware design, and are defined in `types/tx`: + +```go +type Handler interface { + CheckTx(ctx context.Context, req Request, checkReq RequestCheckTx) (Response, ResponseCheckTx, error) + +DeliverTx(ctx context.Context, req Request) (Response, error) + +SimulateTx(ctx context.Context, req Request (Response, error) +} + +type Middleware func(Handler) + +Handler +``` + +where we define the following arguments and return types: + +```go expandable +type Request struct { + Tx sdk.Tx + TxBytes []byte +} + +type Response struct { + GasWanted uint64 + GasUsed uint64 + // MsgResponses is an array containing each Msg service handler's response + // type, packed in an Any. This will get proto-serialized into the `Data` field + // in the ABCI Check/DeliverTx responses. + MsgResponses []*codectypes.Any + Log string + Events []abci.Event +} + +type RequestCheckTx struct { + Type abci.CheckTxType +} + +type ResponseCheckTx struct { + Priority int64 +} +``` + +Please note that because CheckTx handles separate logic related to mempool priotization, its signature is different than DeliverTx and SimulateTx. + +BaseApp holds a reference to a `tx.Handler`: + +```go +type BaseApp struct { + // other fields + txHandler tx.Handler +} +``` + +Baseapp's ABCI `{Check,Deliver}Tx()` and `Simulate()` methods simply call `app.txHandler.{Check,Deliver,Simulate}Tx()` with the relevant arguments. For example, for `DeliverTx`: + +```go expandable +func (app *BaseApp) + +DeliverTx(req abci.RequestDeliverTx) + +abci.ResponseDeliverTx { + var abciRes abci.ResponseDeliverTx + ctx := app.getContextForTx(runTxModeDeliver, req.Tx) + +res, err := app.txHandler.DeliverTx(ctx, tx.Request{ + TxBytes: req.Tx +}) + if err != nil { + abciRes = sdkerrors.ResponseDeliverTx(err, uint64(res.GasUsed), uint64(res.GasWanted), app.trace) + +return abciRes +} + +abciRes, err = convertTxResponseToDeliverTx(res) + if err != nil { + return sdkerrors.ResponseDeliverTx(err, uint64(res.GasUsed), uint64(res.GasWanted), app.trace) +} + +return abciRes +} + +// convertTxResponseToDeliverTx converts a tx.Response into a abci.ResponseDeliverTx. +func convertTxResponseToDeliverTx(txRes tx.Response) (abci.ResponseDeliverTx, error) { + data, err := makeABCIData(txRes) + if err != nil { + return abci.ResponseDeliverTx{ +}, nil +} + +return abci.ResponseDeliverTx{ + Data: data, + Log: txRes.Log, + Events: txRes.Events, +}, nil +} + +// makeABCIData generates the Data field to be sent to ABCI Check/DeliverTx. +func makeABCIData(txRes tx.Response) ([]byte, error) { + return proto.Marshal(&sdk.TxMsgData{ + MsgResponses: txRes.MsgResponses +}) +} +``` + +The implementations are similar for `BaseApp.CheckTx` and `BaseApp.Simulate`. + +`baseapp.txHandler`'s three methods' implementations can obviously be monolithic functions, but for modularity we propose a middleware composition design, where a middleware is simply a function that takes a `tx.Handler`, and returns another `tx.Handler` wrapped around the previous one. + +### Implementing a Middleware + +In practice, middlewares are created by Go function that takes as arguments some parameters needed for the middleware, and returns a `tx.Middleware`. + +For example, for creating an arbitrary `MyMiddleware`, we can implement: + +```go expandable +// myTxHandler is the tx.Handler of this middleware. Note that it holds a +// reference to the next tx.Handler in the stack. +type myTxHandler struct { + // next is the next tx.Handler in the middleware stack. + next tx.Handler + // some other fields that are relevant to the middleware can be added here +} + +// NewMyMiddleware returns a middleware that does this and that. +func NewMyMiddleware(arg1, arg2) + +tx.Middleware { + return func (txh tx.Handler) + +tx.Handler { + return myTxHandler{ + next: txh, + // optionally, set arg1, arg2... if they are needed in the middleware +} + +} +} + +// Assert myTxHandler is a tx.Handler. +var _ tx.Handler = myTxHandler{ +} + +func (h myTxHandler) + +CheckTx(ctx context.Context, req Request, checkReq RequestcheckTx) (Response, ResponseCheckTx, error) { + // CheckTx specific pre-processing logic + + // run the next middleware + res, checkRes, err := txh.next.CheckTx(ctx, req, checkReq) + + // CheckTx specific post-processing logic + + return res, checkRes, err +} + +func (h myTxHandler) + +DeliverTx(ctx context.Context, req Request) (Response, error) { + // DeliverTx specific pre-processing logic + + // run the next middleware + res, err := txh.next.DeliverTx(ctx, tx, req) + + // DeliverTx specific post-processing logic + + return res, err +} + +func (h myTxHandler) + +SimulateTx(ctx context.Context, req Request) (Response, error) { + // SimulateTx specific pre-processing logic + + // run the next middleware + res, err := txh.next.SimulateTx(ctx, tx, req) + + // SimulateTx specific post-processing logic + + return res, err +} +``` + +### Composing Middlewares + +While BaseApp simply holds a reference to a `tx.Handler`, this `tx.Handler` itself is defined using a middleware stack. The Cosmos SDK exposes a base (i.e. innermost) `tx.Handler` called `RunMsgsTxHandler`, which executes messages. + +Then, the app developer can compose multiple middlewares on top on the base `tx.Handler`. Each middleware can run pre-and-post-processing logic around its next middleware, as described in the section above. Conceptually, as an example, given the middlewares `A`, `B`, and `C` and the base `tx.Handler` `H` the stack looks like: + +```text +A.pre + B.pre + C.pre + H # The base tx.handler, for example `RunMsgsTxHandler` + C.post + B.post +A.post +``` + +We define a `ComposeMiddlewares` function for composing middlewares. It takes the base handler as first argument, and middlewares in the "outer to inner" order. For the above stack, the final `tx.Handler` is: + +```go +txHandler := middleware.ComposeMiddlewares(H, A, B, C) +``` + +The middleware is set in BaseApp via its `SetTxHandler` setter: + +```go +// simapp/app.go + txHandler := middleware.ComposeMiddlewares(...) + +app.SetTxHandler(txHandler) +``` + +The app developer can define their own middlewares, or use the Cosmos SDK's pre-defined middlewares from `middleware.NewDefaultTxHandler()`. + +### Middlewares Maintained by the Cosmos SDK + +While the app developer can define and compose the middlewares of their choice, the Cosmos SDK provides a set of middlewares that caters for the ecosystem's most common use cases. These middlewares are: + +| Middleware | Description | +| ----------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| RunMsgsTxHandler | This is the base `tx.Handler`. It replaces the old baseapp's `runMsgs`, and executes a transaction's `Msg`s. | +| TxDecoderMiddleware | This middleware takes in transaction raw bytes, and decodes them into a `sdk.Tx`. It replaces the `baseapp.txDecoder` field, so that BaseApp stays as thin as possible. Since most middlewares read the contents of the `sdk.Tx`, the TxDecoderMiddleware should be run first in the middleware stack. | +| `{Antehandlers}` | Each antehandler is converted to its own middleware. These middlewares perform signature verification, fee deductions and other validations on the incoming transaction. | +| IndexEventsTxMiddleware | This is a simple middleware that chooses which events to index in Tendermint. Replaces `baseapp.indexEvents` (which unfortunately still exists in baseapp too, because it's used to index Begin/EndBlock events) | +| RecoveryTxMiddleware | This index recovers from panics. It replaces baseapp.runTx's panic recovery described in [ADR-022](/sdk/v0.50/build/architecture/adr-022-custom-panic-handling). | +| GasTxMiddleware | This replaces the [`Setup`](https://github.com/cosmos/cosmos-sdk/blob/v0.43.0/x/auth/ante/setup.go) Antehandler. It sets a GasMeter on sdk.Context. Note that before, GasMeter was set on sdk.Context inside the antehandlers, and there was some mess around the fact that antehandlers had their own panic recovery system so that the GasMeter could be read by baseapp's recovery system. Now, this mess is all removed: one middleware sets GasMeter, another one handles recovery. | + +### Similarities and Differences between Antehandlers and Middlewares + +The middleware-based design builds upon the existing antehandlers design described in [ADR-010](/sdk/v0.50/build/architecture/adr-010-modular-antehandler). Even though the final decision of ADR-010 was to go with the "Simple Decorators" approach, the middleware design is actually very similar to the other [Decorator Pattern](/sdk/v0.50/build/architecture/adr-010-modular-antehandler#decorator-pattern) proposal, also used in [weave](https://github.com/iov-one/weave). + +#### Similarities with Antehandlers + +* Designed as chaining/composing small modular pieces. +* Allow code reuse for `{Check,Deliver}Tx` and for `Simulate`. +* Set up in `app.go`, and easily customizable by app developers. +* Order is important. + +#### Differences with Antehandlers + +* The Antehandlers are run before `Msg` execution, whereas middlewares can run before and after. +* The middleware approach uses separate methods for `{Check,Deliver,Simulate}Tx`, whereas the antehandlers pass a `simulate bool` flag and uses the `sdkCtx.Is{Check,Recheck}Tx()` flags to determine in which transaction mode we are. +* The middleware design lets each middleware hold a reference to the next middleware, whereas the antehandlers pass a `next` argument in the `AnteHandle` method. +* The middleware design use Go's standard `context.Context`, whereas the antehandlers use `sdk.Context`. + +## Consequences + +### Backwards Compatibility + +Since this refactor removes some logic away from BaseApp and into middlewares, it introduces API-breaking changes for app developers. Most notably, instead of creating an antehandler chain in `app.go`, app developers need to create a middleware stack: + +```diff expandable +- anteHandler, err := ante.NewAnteHandler( +- ante.HandlerOptions{ +- AccountKeeper: app.AccountKeeper, +- BankKeeper: app.BankKeeper, +- SignModeHandler: encodingConfig.TxConfig.SignModeHandler(), +- FeegrantKeeper: app.FeeGrantKeeper, +- SigGasConsumer: ante.DefaultSigVerificationGasConsumer, +- }, +-) ++txHandler, err := authmiddleware.NewDefaultTxHandler(authmiddleware.TxHandlerOptions{ ++ Debug: app.Trace(), ++ IndexEvents: indexEvents, ++ LegacyRouter: app.legacyRouter, ++ MsgServiceRouter: app.msgSvcRouter, ++ LegacyAnteHandler: anteHandler, ++ TxDecoder: encodingConfig.TxConfig.TxDecoder, ++}) +if err != nil { + panic(err) +} +- app.SetAnteHandler(anteHandler) ++ app.SetTxHandler(txHandler) +``` + +Other more minor API breaking changes will also be provided in the CHANGELOG. As usual, the Cosmos SDK will provide a release migration document for app developers. + +This ADR does not introduce any state-machine-, client- or CLI-breaking changes. + +### Positive + +* Allow custom logic to be run before an after `Msg` execution. This enables the [tips](https://github.com/cosmos/cosmos-sdk/issues/9406) and [gas refund](https://github.com/cosmos/cosmos-sdk/issues/2150) uses cases, and possibly other ones. +* Make BaseApp more lightweight, and defer complex logic to small modular components. +* Separate paths for `{Check,Deliver,Simulate}Tx` with different returns types. This allows for improved readability (replace `if sdkCtx.IsRecheckTx() && !simulate {...}` with separate methods) and more flexibility (e.g. returning a `priority` in `ResponseCheckTx`). + +### Negative + +* It is hard to understand at first glance the state updates that would occur after a middleware runs given the `sdk.Context` and `tx`. A middleware can have an arbitrary number of nested middleware being called within its function body, each possibly doing some pre- and post-processing before calling the next middleware on the chain. Thus to understand what a middleware is doing, one must also understand what every other middleware further along the chain is also doing, and the order of middlewares matters. This can get quite complicated to understand. +* API-breaking changes for app developers. + +### Neutral + +No neutral consequences. + +## Further Discussions + +* [#9934](https://github.com/cosmos/cosmos-sdk/discussions/9934) Decomposing BaseApp's other ABCI methods into middlewares. +* Replace `sdk.Tx` interface with the concrete protobuf Tx type in the `tx.Handler` methods signature. + +## Test Cases + +We update the existing baseapp and antehandlers tests to use the new middleware API, but keep the same test cases and logic, to avoid introducing regressions. Existing CLI tests will also be left untouched. + +For new middlewares, we introduce unit tests. Since middlewares are purposefully small, unit tests suit well. + +## References + +* Initial discussion: [Link](https://github.com/cosmos/cosmos-sdk/issues/9585) +* Implementation: [#9920 BaseApp refactor](https://github.com/cosmos/cosmos-sdk/pull/9920) and [#10028 Antehandlers migration](https://github.com/cosmos/cosmos-sdk/pull/10028) diff --git a/sdk/v0.54/reference/architecture/adr-046-module-params.mdx b/sdk/v0.54/reference/architecture/adr-046-module-params.mdx new file mode 100644 index 000000000..4bd02f4c2 --- /dev/null +++ b/sdk/v0.54/reference/architecture/adr-046-module-params.mdx @@ -0,0 +1,193 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/architecture/adr-046-module-params' +title: 'ADR 046: Module Params' +description: 'Sep 22, 2021: Initial Draft' +--- + +## Changelog + +* Sep 22, 2021: Initial Draft + +## Status + +Proposed + +## Abstract + +This ADR describes an alternative approach to how Cosmos SDK modules use, interact, +and store their respective parameters. + +## Context + +Currently, in the Cosmos SDK, modules that require the use of parameters use the +`x/params` module. The `x/params` works by having modules define parameters, +typically via a simple `Params` structure, and registering that structure in +the `x/params` module via a unique `Subspace` that belongs to the respective +registering module. The registering module then has unique access to its respective +`Subspace`. Through this `Subspace`, the module can get and set its `Params` +structure. + +In addition, the Cosmos SDK's `x/gov` module has direct support for changing +parameters on-chain via a `ParamChangeProposal` governance proposal type, where +stakeholders can vote on suggested parameter changes. + +There are various tradeoffs to using the `x/params` module to manage individual +module parameters. Namely, managing parameters essentially comes for "free" in +that developers only need to define the `Params` struct, the `Subspace`, and the +various auxiliary functions, e.g. `ParamSetPairs`, on the `Params` type. However, +there are some notable drawbacks. These drawbacks include the fact that parameters +are serialized in state via JSON which is extremely slow. In addition, parameter +changes via `ParamChangeProposal` governance proposals have no way of reading from +or writing to state. In other words, it is currently not possible to have any +state transitions in the application during an attempt to change param(s). + +## Decision + +We will build off of the alignment of `x/gov` and `x/authz` work per +[#9810](https://github.com/cosmos/cosmos-sdk/pull/9810). Namely, module developers +will create one or more unique parameter data structures that must be serialized +to state. The Param data structures must implement `sdk.Msg` interface with respective +Protobuf Msg service method which will validate and update the parameters with all +necessary changes. The `x/gov` module via the work done in +[#9810](https://github.com/cosmos/cosmos-sdk/pull/9810), will dispatch Param +messages, which will be handled by Protobuf Msg services. + +Note, it is up to developers to decide how to structure their parameters and +the respective `sdk.Msg` messages. Consider the parameters currently defined in +`x/auth` using the `x/params` module for parameter management: + +```protobuf +message Params { + uint64 max_memo_characters = 1; + uint64 tx_sig_limit = 2; + uint64 tx_size_cost_per_byte = 3; + uint64 sig_verify_cost_ed25519 = 4; + uint64 sig_verify_cost_secp256k1 = 5; +} +``` + +Developers can choose to either create a unique data structure for every field in +`Params` or they can create a single `Params` structure as outlined above in the +case of `x/auth`. + +In the former, `x/params`, approach, a `sdk.Msg` would need to be created for every single +field along with a handler. This can become burdensome if there are a lot of +parameter fields. In the latter case, there is only a single data structure and +thus only a single message handler, however, the message handler might have to be +more sophisticated in that it might need to understand what parameters are being +changed vs what parameters are untouched. + +Params change proposals are made using the `x/gov` module. Execution is done through +`x/authz` authorization to the root `x/gov` module's account. + +Continuing to use `x/auth`, we demonstrate a more complete example: + +```go expandable +type Params struct { + MaxMemoCharacters uint64 + TxSigLimit uint64 + TxSizeCostPerByte uint64 + SigVerifyCostED25519 uint64 + SigVerifyCostSecp256k1 uint64 +} + +type MsgUpdateParams struct { + MaxMemoCharacters uint64 + TxSigLimit uint64 + TxSizeCostPerByte uint64 + SigVerifyCostED25519 uint64 + SigVerifyCostSecp256k1 uint64 +} + +type MsgUpdateParamsResponse struct { +} + +func (ms msgServer) + +UpdateParams(goCtx context.Context, msg *types.MsgUpdateParams) (*types.MsgUpdateParamsResponse, error) { + ctx := sdk.UnwrapSDKContext(goCtx) + + // verification logic... + + // persist params + params := ParamsFromMsg(msg) + +ms.SaveParams(ctx, params) + +return &types.MsgUpdateParamsResponse{ +}, nil +} + +func ParamsFromMsg(msg *types.MsgUpdateParams) + +Params { + // ... +} +``` + +A gRPC `Service` query should also be provided, for example: + +```protobuf expandable +service Query { + // ... + + rpc Params(QueryParamsRequest) returns (QueryParamsResponse) { + option (google.api.http).get = "/cosmos//v1beta1/params"; + } +} + +message QueryParamsResponse { + Params params = 1 [(gogoproto.nullable) = false]; +} +``` + +## Consequences + +As a result of implementing the module parameter methodology, we gain the ability +for module parameter changes to be stateful and extensible to fit nearly every +application's use case. We will be able to emit events (and trigger hooks registered +to that events using the work proposed in [event hooks](https://github.com/cosmos/cosmos-sdk/discussions/9656)), +call other Msg service methods or perform migration. +In addition, there will be significant gains in performance when it comes to reading +and writing parameters from and to state, especially if a specific set of parameters +are read on a consistent basis. + +However, this methodology will require developers to implement more types and +Msg service methods which can become burdensome if many parameters exist. In addition, +developers are required to implement persistence logic for module parameters. +However, this should be trivial. + +### Backwards Compatibility + +The new method for working with module parameters is naturally not backwards +compatible with the existing `x/params` module. However, the `x/params` will +remain in the Cosmos SDK and will be marked as deprecated with no additional +functionality being added apart from potential bug fixes. Note, the `x/params` +module may be removed entirely in a future release. + +### Positive + +* Module parameters are serialized more efficiently +* Modules are able to react on parameters changes and perform additional actions. +* Special events can be emitted, allowing hooks to be triggered. + +### Negative + +* Module parameters becomes slightly more burdensome for module developers: + * Modules are now responsible for persisting and retrieving parameter state + * Modules are now required to have unique message handlers to handle parameter + changes per unique parameter data structure. + +### Neutral + +* Requires [#9810](https://github.com/cosmos/cosmos-sdk/pull/9810) to be reviewed + and merged. + +{/* ## Further Discussions While an ADR is in the DRAFT or PROPOSED stage, this section should contain a summary of issues to be solved in future iterations (usually referencing comments from a pull-request discussion). Later, this section can optionally list ideas or improvements the author or reviewers found during the analysis of this ADR. */} + +## References + +* [Link](https://github.com/cosmos/cosmos-sdk/pull/9810) +* [Link](https://github.com/cosmos/cosmos-sdk/issues/9438) +* [Link](https://github.com/cosmos/cosmos-sdk/discussions/9913) diff --git a/sdk/v0.54/reference/architecture/adr-047-extend-upgrade-plan.mdx b/sdk/v0.54/reference/architecture/adr-047-extend-upgrade-plan.mdx new file mode 100644 index 000000000..48232c25f --- /dev/null +++ b/sdk/v0.54/reference/architecture/adr-047-extend-upgrade-plan.mdx @@ -0,0 +1,261 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/architecture/adr-047-extend-upgrade-plan' +title: 'ADR 047: Extend Upgrade Plan' +description: >- + Nov, 23, 2021: Initial Draft May, 16, 2023: Proposal ABANDONED. prerun and + postrun are not necessary anymore and adding the artifacts brings minor + benefits. +--- + +## Changelog + +* Nov, 23, 2021: Initial Draft +* May, 16, 2023: Proposal ABANDONED. `pre_run` and `post_run` are not necessary anymore and adding the `artifacts` brings minor benefits. + +## Status + +ABANDONED + +## Abstract + +This ADR expands the existing x/upgrade `Plan` proto message to include new fields for defining pre-run and post-run processes within upgrade tooling. +It also defines a structure for providing downloadable artifacts involved in an upgrade. + +## Context + +The `upgrade` module in conjunction with Cosmovisor are designed to facilitate and automate a blockchain's transition from one version to another. + +Users submit a software upgrade governance proposal containing an upgrade `Plan`. +The [Plan](https://github.com/cosmos/cosmos-sdk/blob/v0.44.5/proto/cosmos/upgrade/v1beta1/upgrade.proto#L12) currently contains the following fields: + +* `name`: A short string identifying the new version. +* `height`: The chain height at which the upgrade is to be performed. +* `info`: A string containing information about the upgrade. + +The `info` string can be anything. +However, Cosmovisor will try to use the `info` field to automatically download a new version of the blockchain executable. +For the auto-download to work, Cosmovisor expects it to be either a stringified JSON object (with a specific structure defined through documentation), or a URL that will return such JSON. +The JSON object identifies URLs used to download the new blockchain executable for different platforms (OS and Architecture, e.g. "linux/amd64"). +Such a URL can either return the executable file directly or can return an archive containing the executable and possibly other assets. + +If the URL returns an archive, it is decompressed into `{DAEMON_HOME}/cosmovisor/{upgrade name}`. +Then, if `{DAEMON_HOME}/cosmovisor/{upgrade name}/bin/{DAEMON_NAME}` does not exist, but `{DAEMON_HOME}/cosmovisor/{upgrade name}/{DAEMON_NAME}` does, the latter is copied to the former. +If the URL returns something other than an archive, it is downloaded to `{DAEMON_HOME}/cosmovisor/{upgrade name}/bin/{DAEMON_NAME}`. + +If an upgrade height is reached and the new version of the executable version isn't available, Cosmovisor will stop running. + +Both `DAEMON_HOME` and `DAEMON_NAME` are [environment variables used to configure Cosmovisor](https://github.com/cosmos/cosmos-sdk/blob/cosmovisor/v1.0.0/cosmovisor/README.md#command-line-arguments-and-environment-variables). + +Currently, there is no mechanism that makes Cosmovisor run a command after the upgraded chain has been restarted. + +The current upgrade process has this timeline: + +1. An upgrade governance proposal is submitted and approved. +2. The upgrade height is reached. +3. The `x/upgrade` module writes the `upgrade_info.json` file. +4. The chain halts. +5. Cosmovisor backs up the data directory (if set up to do so). +6. Cosmovisor downloads the new executable (if not already in place). +7. Cosmovisor executes the `${DAEMON_NAME} pre-upgrade`. +8. Cosmovisor restarts the app using the new version and same args originally provided. + +## Decision + +### Protobuf Updates + +We will update the `x/upgrade.Plan` message for providing upgrade instructions. +The upgrade instructions will contain a list of artifacts available for each platform. +It allows for the definition of a pre-run and post-run commands. +These commands are not consensus guaranteed; they will be executed by Cosmosvisor (or other) during its upgrade handling. + +```protobuf +message Plan { + // ... (existing fields) + + UpgradeInstructions instructions = 6; +} +``` + +The new `UpgradeInstructions instructions` field MUST be optional. + +```protobuf +message UpgradeInstructions { + string pre_run = 1; + string post_run = 2; + repeated Artifact artifacts = 3; + string description = 4; +} +``` + +All fields in the `UpgradeInstructions` are optional. + +* `pre_run` is a command to run prior to the upgraded chain restarting. + If defined, it will be executed after halting and downloading the new artifact but before restarting the upgraded chain. + The working directory this command runs from MUST be `{DAEMON_HOME}/cosmovisor/{upgrade name}`. + This command MUST behave the same as the current [pre-upgrade](https://github.com/cosmos/cosmos-sdk/blob/v0.44.5/docs/migrations/pre-upgrade.md) command. + It does not take in any command-line arguments and is expected to terminate with the following exit codes: + + | Exit status code | How it is handled in Cosmosvisor | + | ---------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------- | + | `0` | Assumes `pre-upgrade` command executed successfully and continues the upgrade. | + | `1` | Default exit code when `pre-upgrade` command has not been implemented. | + | `30` | `pre-upgrade` command was executed but failed. This fails the entire upgrade. | + | `31` | `pre-upgrade` command was executed but failed. But the command is retried until exit code `1` or `30` are returned. | + | If defined, then the app supervisors (e.g. Cosmovisor) MUST NOT run `app pre-run`. | | +* `post_run` is a command to run after the upgraded chain has been started. If defined, this command MUST be only executed at most once by an upgrading node. + The output and exit code SHOULD be logged but SHOULD NOT affect the running of the upgraded chain. + The working directory this command runs from MUST be `{DAEMON_HOME}/cosmovisor/{upgrade name}`. +* `artifacts` define items to be downloaded. + It SHOULD have only one entry per platform. +* `description` contains human-readable information about the upgrade and might contain references to external resources. + It SHOULD NOT be used for structured processing information. + +```protobuf +message Artifact { + string platform = 1; + string url = 2; + string checksum = 3; + string checksum_algo = 4; +} +``` + +* `platform` is a required string that SHOULD be in the format `{OS}/{CPU}`, e.g. `"linux/amd64"`. + The string `"any"` SHOULD also be allowed. + An `Artifact` with a `platform` of `"any"` SHOULD be used as a fallback when a specific `{OS}/{CPU}` entry is not found. + That is, if an `Artifact` exists with a `platform` that matches the system's OS and CPU, that should be used; + otherwise, if an `Artifact` exists with a `platform` of `any`, that should be used; + otherwise no artifact should be downloaded. +* `url` is a required URL string that MUST conform to [RFC 1738: Uniform Resource Locators](https://www.ietf.org/rfc/rfc1738.txt). + A request to this `url` MUST return either an executable file or an archive containing either `bin/{DAEMON_NAME}` or `{DAEMON_NAME}`. + The URL should not contain checksum - it should be specified by the `checksum` attribute. +* `checksum` is a checksum of the expected result of a request to the `url`. + It is not required, but is recommended. + If provided, it MUST be a hex encoded checksum string. + Tools utilizing these `UpgradeInstructions` MUST fail if a `checksum` is provided but is different from the checksum of the result returned by the `url`. +* `checksum_algo` is a string identify the algorithm used to generate the `checksum`. + Recommended algorithms: `sha256`, `sha512`. + Algorithms also supported (but not recommended): `sha1`, `md5`. + If a `checksum` is provided, a `checksum_algo` MUST also be provided. + +A `url` is not required to contain a `checksum` query parameter. +If the `url` does contain a `checksum` query parameter, the `checksum` and `checksum_algo` fields MUST also be populated, and their values MUST match the value of the query parameter. +For example, if the `url` is `"https://example.com?checksum=md5:d41d8cd98f00b204e9800998ecf8427e"`, then the `checksum` field must be `"d41d8cd98f00b204e9800998ecf8427e"` and the `checksum_algo` field must be `"md5"`. + +### Upgrade Module Updates + +If an upgrade `Plan` does not use the new `UpgradeInstructions` field, existing functionality will be maintained. +The parsing of the `info` field as either a URL or `binaries` JSON will be deprecated. +During validation, if the `info` field is used as such, a warning will be issued, but not an error. + +We will update the creation of the `upgrade-info.json` file to include the `UpgradeInstructions`. + +We will update the optional validation available via CLI to account for the new `Plan` structure. +We will add the following validation: + +1. If `UpgradeInstructions` are provided: + 1. There MUST be at least one entry in `artifacts`. + 2. All of the `artifacts` MUST have a unique `platform`. + 3. For each `Artifact`, if the `url` contains a `checksum` query parameter: + 1. The `checksum` query parameter value MUST be in the format of `{checksum_algo}:{checksum}`. + 2. The `{checksum}` from the query parameter MUST equal the `checksum` provided in the `Artifact`. + 3. The `{checksum_algo}` from the query parameter MUST equal the `checksum_algo` provided in the `Artifact`. +2. The following validation is currently done using the `info` field. We will apply similar validation to the `UpgradeInstructions`. + For each `Artifact`: + 1. The `platform` MUST have the format `{OS}/{CPU}` or be `"any"`. + 2. The `url` field MUST NOT be empty. + 3. The `url` field MUST be a proper URL. + 4. A `checksum` MUST be provided either in the `checksum` field or as a query parameter in the `url`. + 5. If the `checksum` field has a value and the `url` also has a `checksum` query parameter, the two values MUST be equal. + 6. The `url` MUST return either a file or an archive containing either `bin/{DAEMON_NAME}` or `{DAEMON_NAME}`. + 7. If a `checksum` is provided (in the field or as a query param), the checksum of the result of the `url` MUST equal the provided checksum. + +Downloading of an `Artifact` will happen the same way that URLs from `info` are currently downloaded. + +### Cosmovisor Updates + +If the `upgrade-info.json` file does not contain any `UpgradeInstructions`, existing functionality will be maintained. + +We will update Cosmovisor to look for and handle the new `UpgradeInstructions` in `upgrade-info.json`. +If the `UpgradeInstructions` are provided, we will do the following: + +1. The `info` field will be ignored. +2. The `artifacts` field will be used to identify the artifact to download based on the `platform` that Cosmovisor is running in. +3. If a `checksum` is provided (either in the field or as a query param in the `url`), and the downloaded artifact has a different checksum, the upgrade process will be interrupted and Cosmovisor will exit with an error. +4. If a `pre_run` command is defined, it will be executed at the same point in the process where the `app pre-upgrade` command would have been executed. + It will be executed using the same environment as other commands run by Cosmovisor. +5. If a `post_run` command is defined, it will be executed after executing the command that restarts the chain. + It will be executed in a background process using the same environment as the other commands. + Any output generated by the command will be logged. + Once complete, the exit code will be logged. + +We will deprecate the use of the `info` field for anything other than human readable information. +A warning will be logged if the `info` field is used to define the assets (either by URL or JSON). + +The new upgrade timeline is very similar to the current one. Changes are in bold: + +1. An upgrade governance proposal is submitted and approved. +2. The upgrade height is reached. +3. The `x/upgrade` module writes the `upgrade_info.json` file **(now possibly with `UpgradeInstructions`)**. +4. The chain halts. +5. Cosmovisor backs up the data directory (if set up to do so). +6. Cosmovisor downloads the new executable (if not already in place). +7. Cosmovisor executes **the `pre_run` command if provided**, or else the `${DAEMON_NAME} pre-upgrade` command. +8. Cosmovisor restarts the app using the new version and same args originally provided. +9. **Cosmovisor immediately runs the `post_run` command in a detached process.** + +## Consequences + +### Backwards Compatibility + +Since the only change to existing definitions is the addition of the `instructions` field to the `Plan` message, and that field is optional, there are no backwards incompatibilities with respects to the proto messages. +Additionally, current behavior will be maintained when no `UpgradeInstructions` are provided, so there are no backwards incompatibilities with respects to either the upgrade module or Cosmovisor. + +### Forwards Compatibility + +In order to utilize the `UpgradeInstructions` as part of a software upgrade, both of the following must be true: + +1. The chain must already be using a sufficiently advanced version of the Cosmos SDK. +2. The chain's nodes must be using a sufficiently advanced version of Cosmovisor. + +### Positive + +1. The structure for defining artifacts is clearer since it is now defined in the proto instead of in documentation. +2. Availability of a pre-run command becomes more obvious. +3. A post-run command becomes possible. + +### Negative + +1. The `Plan` message becomes larger. This is negligible because A) the `x/upgrades` module only stores at most one upgrade plan, and B) upgrades are rare enough that the increased gas cost isn't a concern. +2. There is no option for providing a URL that will return the `UpgradeInstructions`. +3. The only way to provide multiple assets (executables and other files) for a platform is to use an archive as the platform's artifact. + +### Neutral + +1. Existing functionality of the `info` field is maintained when the `UpgradeInstructions` aren't provided. + +## Further Discussions + +1. [Draft PR #10032 Comment](https://github.com/cosmos/cosmos-sdk/pull/10032/files?authenticity_token=pLtzpnXJJB%2Fif2UWiTp9Td3MvRrBF04DvjSuEjf1azoWdLF%2BSNymVYw9Ic7VkqHgNLhNj6iq9bHQYnVLzMXd4g%3D%3D\&file-filters%5B%5D=.go\&file-filters%5B%5D=.proto#r698708349): + Consider different names for `UpgradeInstructions instructions` (either the message type or field name). +2. [Draft PR #10032 Comment](https://github.com/cosmos/cosmos-sdk/pull/10032/files?authenticity_token=pLtzpnXJJB%2Fif2UWiTp9Td3MvRrBF04DvjSuEjf1azoWdLF%2BSNymVYw9Ic7VkqHgNLhNj6iq9bHQYnVLzMXd4g%3D%3D\&file-filters%5B%5D=.go\&file-filters%5B%5D=.proto#r754655072): + 1. Consider putting the `string platform` field inside `UpgradeInstructions` and make `UpgradeInstructions` a repeated field in `Plan`. + 2. Consider using a `oneof` field in the `Plan` which could either be `UpgradeInstructions` or else a URL that should return the `UpgradeInstructions`. + 3. Consider allowing `info` to either be a JSON serialized version of `UpgradeInstructions` or else a URL that returns that. +3. [Draft PR #10032 Comment](https://github.com/cosmos/cosmos-sdk/pull/10032/files?authenticity_token=pLtzpnXJJB%2Fif2UWiTp9Td3MvRrBF04DvjSuEjf1azoWdLF%2BSNymVYw9Ic7VkqHgNLhNj6iq9bHQYnVLzMXd4g%3D%3D\&file-filters%5B%5D=.go\&file-filters%5B%5D=.proto#r755462876): + Consider not including the `UpgradeInstructions.description` field, using the `info` field for that purpose instead. +4. [Draft PR #10032 Comment](https://github.com/cosmos/cosmos-sdk/pull/10032/files?authenticity_token=pLtzpnXJJB%2Fif2UWiTp9Td3MvRrBF04DvjSuEjf1azoWdLF%2BSNymVYw9Ic7VkqHgNLhNj6iq9bHQYnVLzMXd4g%3D%3D\&file-filters%5B%5D=.go\&file-filters%5B%5D=.proto#r754643691): + Consider allowing multiple artifacts to be downloaded for any given `platform` by adding a `name` field to the `Artifact` message. +5. [PR #10502 Comment](https://github.com/cosmos/cosmos-sdk/pull/10602#discussion_r781438288) + Allow the new `UpgradeInstructions` to be provided via URL. +6. [PR #10502 Comment](https://github.com/cosmos/cosmos-sdk/pull/10602#discussion_r781438288) + Allow definition of a `signer` for assets (as an alternative to using a `checksum`). + +## References + +* [Current upgrade.proto](https://github.com/cosmos/cosmos-sdk/blob/v0.44.5/proto/cosmos/upgrade/v1beta1/upgrade.proto) +* [Upgrade Module README](https://github.com/cosmos/cosmos-sdk/blob/v0.44.5/x/upgrade/spec/README.md) +* [Cosmovisor README](https://github.com/cosmos/cosmos-sdk/blob/cosmovisor/v1.0.0/cosmovisor/README.md) +* [Pre-upgrade README](https://github.com/cosmos/cosmos-sdk/blob/v0.44.5/docs/migrations/pre-upgrade.md) +* [Draft/POC PR #10032](https://github.com/cosmos/cosmos-sdk/pull/10032) +* [RFC 1738: Uniform Resource Locators](https://www.ietf.org/rfc/rfc1738.txt) diff --git a/sdk/v0.54/reference/architecture/adr-048-consensus-fees.mdx b/sdk/v0.54/reference/architecture/adr-048-consensus-fees.mdx new file mode 100644 index 000000000..90628575d --- /dev/null +++ b/sdk/v0.54/reference/architecture/adr-048-consensus-fees.mdx @@ -0,0 +1,209 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/architecture/adr-048-consensus-fees' +title: 'ADR 048: Multi Tire Gas Price System' +description: 'Dec 1, 2021: Initial Draft' +--- + +## Changelog + +* Dec 1, 2021: Initial Draft + +## Status + +Rejected + +## Abstract + +This ADR describes a flexible mechanism to maintain a consensus level gas prices, in which one can choose a multi-tier gas price system or EIP-1559 like one through configuration. + +## Context + +Currently, each validator configures its own `minimal-gas-prices` in `app.yaml`. But setting a proper minimal gas price is critical to protect network from DDoS attack, and it's hard for all the validators to pick a sensible value, so we propose to maintain a gas price in consensus level. + +Since tendermint 0.34.20 has supported mempool prioritization, we can take advantage of that to implement more sophisticated gas fee system. + +## Multi-Tier Price System + +We propose a multi-tier price system on consensus to provide maximum flexibility: + +* Tier 1: a constant gas price, which could only be modified occasionally through governance proposal. +* Tier 2: a dynamic gas price which is adjusted according to previous block load. +* Tier 3: a dynamic gas price which is adjusted according to previous block load at a higher speed. + +The gas price of higher tier should bigger than the lower tier. + +The transaction fees are charged with the exact gas price calculated on consensus. + +The parameter schema is like this: + +```protobuf expandable +message TierParams { + uint32 priority = 1 // priority in tendermint mempool + Coin initial_gas_price = 2 // + uint32 parent_gas_target = 3 // the target saturation of block + uint32 change_denominator = 4 // decides the change speed + Coin min_gas_price = 5 // optional lower bound of the price adjustment + Coin max_gas_price = 6 // optional upper bound of the price adjustment +} + +message Params { + repeated TierParams tiers = 1; +} +``` + +### Extension Options + +We need to allow user to specify the tier of service for the transaction, to support it in an extensible way, we add an extension option in `AuthInfo`: + +```protobuf +message ExtensionOptionsTieredTx { + uint32 fee_tier = 1 +} +``` + +The value of `fee_tier` is just the index to the `tiers` parameter list. + +We also change the semantic of existing `fee` field of `Tx`, instead of charging user the exact `fee` amount, we treat it as a fee cap, while the actual amount of fee charged is decided dynamically. If the `fee` is smaller than dynamic one, the transaction won't be included in current block and ideally should stay in the mempool until the consensus gas price drop. The mempool can eventually prune old transactions. + +### Tx Prioritization + +Transactions are prioritized based on the tier, the higher the tier, the higher the priority. + +Within the same tier, follow the default Tendermint order (currently FIFO). Be aware of that the mempool tx ordering logic is not part of consensus and can be modified by malicious validator. + +This mechanism can be easily composed with prioritization mechanisms: + +* we can add extra tiers out of a user control: + * Example 1: user can set tier 0, 10 or 20, but the protocol will create tiers 0, 1, 2 ... 29. For example IBC transactions will go to tier `user_tier + 5`: if user selected tier 1, then the transaction will go to tier 15. + * Example 2: we can reserve tier 4, 5, ... only for special transaction types. For example, tier 5 is reserved for evidence tx. So if submits a bank.Send transaction and set tier 5, it will be delegated to tier 3 (the max tier level available for any transaction). + * Example 3: we can enforce that all transactions of a sepecific type will go to specific tier. For example, tier 100 will be reserved for evidence transactions and all evidence transactions will always go to that tier. + +### `min-gas-prices` + +Deprecate the current per-validator `min-gas-prices` configuration, since it would confusing for it to work together with the consensus gas price. + +### Adjust For Block Load + +For tier 2 and tier 3 transactions, the gas price is adjusted according to previous block load, the logic could be similar to EIP-1559: + +```python expandable +def adjust_gas_price(gas_price, parent_gas_used, tier): + if parent_gas_used == tier.parent_gas_target: + return gas_price + elif parent_gas_used > tier.parent_gas_target: + gas_used_delta = parent_gas_used - tier.parent_gas_target + gas_price_delta = max(gas_price * gas_used_delta // tier.parent_gas_target // tier.change_speed, 1) + return gas_price + gas_price_delta + else: + gas_used_delta = parent_gas_target - parent_gas_used + gas_price_delta = gas_price * gas_used_delta // parent_gas_target // tier.change_speed + return gas_price - gas_price_delta +``` + +### Block Segment Reservation + +Ideally we should reserve block segments for each tier, so the lower tiered transactions won't be completely squeezed out by higher tier transactions, which will force user to use higher tier, and the system degraded to a single tier. + +We need help from tendermint to implement this. + +## Implementation + +We can make each tier's gas price strategy fully configurable in protocol parameters, while providing a sensible default one. + +Pseudocode in python-like syntax: + +```python expandable +interface TieredTx: + def tier(self) -> int: + pass + +def tx_tier(tx): + if isinstance(tx, TieredTx): + return tx.tier() + else: + # default tier for custom transactions + return 0 + # NOTE: we can add more rules here per "Tx Prioritization" section + +class TierParams: + 'gas price strategy parameters of one tier' + priority: int # priority in tendermint mempool + initial_gas_price: Coin + parent_gas_target: int + change_speed: Decimal # 0 means don't adjust for block load. + +class Params: + 'protocol parameters' + tiers: List[TierParams] + +class State: + 'consensus state' + # total gas used in last block, None when it's the first block + parent_gas_used: Optional[int] + # gas prices of last block for all tiers + gas_prices: List[Coin] + +def begin_block(): + 'Adjust gas prices' + for i, tier in enumerate(Params.tiers): + if State.parent_gas_used is None: + # initialized gas price for the first block + State.gas_prices[i] = tier.initial_gas_price + else: + # adjust gas price according to gas used in previous block + State.gas_prices[i] = adjust_gas_price(State.gas_prices[i], State.parent_gas_used, tier) + +def mempoolFeeTxHandler_checkTx(ctx, tx): + # the minimal-gas-price configured by validator, zero in deliver_tx context + validator_price = ctx.MinGasPrice() + consensus_price = State.gas_prices[tx_tier(tx)] + min_price = max(validator_price, consensus_price) + + # zero means infinity for gas price cap + if tx.gas_price() > 0 and tx.gas_price() < min_price: + return 'insufficient fees' + return next_CheckTx(ctx, tx) + +def txPriorityHandler_checkTx(ctx, tx): + res, err := next_CheckTx(ctx, tx) + # pass priority to tendermint + res.Priority = Params.tiers[tx_tier(tx)].priority + return res, err + +def end_block(): + 'Update block gas used' + State.parent_gas_used = block_gas_meter.consumed() +``` + +### DDoS attack protection + +To fully saturate the blocks and prevent other transactions from executing, attacker need to use transactions of highest tier, the cost would be significantly higher than the default tier. + +If attacker spam with lower tier transactions, user can mitigate by sending higher tier transactions. + +## Consequences + +### Backwards Compatibility + +* New protocol parameters. +* New consensus states. +* New/changed fields in transaction body. + +### Positive + +* The default tier keeps the same predictable gas price experience for client. +* The higher tier's gas price can adapt to block load. +* No priority conflict with custom priority based on transaction types, since this proposal only occupy three priority levels. +* Possibility to compose different priority rules with tiers + +### Negative + +* Wallets & tools need to update to support the new `tier` parameter, and semantic of `fee` field is changed. + +### Neutral + +## References + +* [Link](https://eips.ethereum.org/EIPS/eip-1559) +* [Link](https://iohk.io/en/blog/posts/2021/11/26/network-traffic-and-tiered-pricing/) diff --git a/sdk/v0.54/reference/architecture/adr-049-state-sync-hooks.mdx b/sdk/v0.54/reference/architecture/adr-049-state-sync-hooks.mdx new file mode 100644 index 000000000..adac55070 --- /dev/null +++ b/sdk/v0.54/reference/architecture/adr-049-state-sync-hooks.mdx @@ -0,0 +1,199 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/architecture/adr-049-state-sync-hooks' +title: 'ADR 049: State Sync Hooks' +description: >- + Jan 19, 2022: Initial Draft Apr 29, 2022: Safer extension snapshotter + interface +--- + +## Changelog + +* Jan 19, 2022: Initial Draft +* Apr 29, 2022: Safer extension snapshotter interface + +## Status + +Implemented + +## Abstract + +This ADR outlines a hooks-based mechanism for application modules to provide additional state (outside of the IAVL tree) to be used +during state sync. + +## Context + +New clients use state-sync to download snapshots of module state from peers. Currently, the snapshot consists of a +stream of `SnapshotStoreItem` and `SnapshotIAVLItem`, which means that application modules that define their state outside of the IAVL +tree cannot include their state as part of the state-sync process. + +Note, Even though the module state data is outside of the tree, for determinism we require that the hash of the external data should +be posted in the IAVL tree. + +## Decision + +A simple proposal based on our existing implementation is that, we can add two new message types: `SnapshotExtensionMeta` +and `SnapshotExtensionPayload`, and they are appended to the existing multi-store stream with `SnapshotExtensionMeta` +acting as a delimiter between extensions. As the chunk hashes should be able to ensure data integrity, we don't need +a delimiter to mark the end of the snapshot stream. + +Besides, we provide `Snapshotter` and `ExtensionSnapshotter` interface for modules to implement snapshotters, which will handle both taking +snapshot and the restoration. Each module could have mutiple snapshotters, and for modules with additional state, they should +implement `ExtensionSnapshotter` as extension snapshotters. When setting up the application, the snapshot `Manager` should call +`RegisterExtensions([]ExtensionSnapshotter…)` to register all the extension snapshotters. + +```protobuf expandable +// SnapshotItem is an item contained in a rootmulti.Store snapshot. +// On top of the exsiting SnapshotStoreItem and SnapshotIAVLItem, we add two new options for the item. +message SnapshotItem { + // item is the specific type of snapshot item. + oneof item { + SnapshotStoreItem store = 1; + SnapshotIAVLItem iavl = 2 [(gogoproto.customname) = "IAVL"]; + SnapshotExtensionMeta extension = 3; + SnapshotExtensionPayload extension_payload = 4; + } +} + +// SnapshotExtensionMeta contains metadata about an external snapshotter. +// One module may need multiple snapshotters, so each module may have multiple SnapshotExtensionMeta. +message SnapshotExtensionMeta { + // the name of the ExtensionSnapshotter, and it is registered to snapshotter manager when setting up the application + // name should be unique for each ExtensionSnapshotter as we need to alphabetically order their snapshots to get + // deterministic snapshot stream. + string name = 1; + // this is used by each ExtensionSnapshotter to decide the format of payloads included in SnapshotExtensionPayload message + // it is used within the snapshotter/namespace, not global one for all modules + uint32 format = 2; +} + +// SnapshotExtensionPayload contains payloads of an external snapshotter. +message SnapshotExtensionPayload { + bytes payload = 1; +} +``` + +When we create a snapshot stream, the `multistore` snapshot is always placed at the beginning of the binary stream, and other extension snapshots are alphabetically ordered by the name of the corresponding `ExtensionSnapshotter`. + +The snapshot stream would look like as follows: + +```go +// multi-store snapshot +{ + SnapshotStoreItem | SnapshotIAVLItem, ... +} +// extension1 snapshot +SnapshotExtensionMeta +{ + SnapshotExtensionPayload, ... +} +// extension2 snapshot +SnapshotExtensionMeta +{ + SnapshotExtensionPayload, ... +} +``` + +We add an `extensions` field to snapshot `Manager` for extension snapshotters. The `multistore` snapshotter is a special one and it doesn't need a name because it is always placed at the beginning of the binary stream. + +```go expandable +type Manager struct { + store *Store + multistore types.Snapshotter + extensions map[string]types.ExtensionSnapshotter + mtx sync.Mutex + operation operation + chRestore chan<- io.ReadCloser + chRestoreDone <-chan restoreDone + restoreChunkHashes [][]byte + restoreChunkIndex uint32 +} +``` + +For extension snapshotters that implement the `ExtensionSnapshotter` interface, their names should be registered to the snapshot `Manager` by +calling `RegisterExtensions` when setting up the application. The snapshotters will handle both taking snapshot and restoration. + +```go +// RegisterExtensions register extension snapshotters to manager +func (m *Manager) + +RegisterExtensions(extensions ...types.ExtensionSnapshotter) + +error +``` + +On top of the existing `Snapshotter` interface for the `multistore`, we add `ExtensionSnapshotter` interface for the extension snapshotters. Three more function signatures: `SnapshotFormat()`, `SupportedFormats()` and `SnapshotName()` are added to `ExtensionSnapshotter`. + +```go expandable +// ExtensionPayloadReader read extension payloads, +// it returns io.EOF when reached either end of stream or the extension boundaries. +type ExtensionPayloadReader = func() ([]byte, error) + +// ExtensionPayloadWriter is a helper to write extension payloads to underlying stream. +type ExtensionPayloadWriter = func([]byte) + +error + +// ExtensionSnapshotter is an extension Snapshotter that is appended to the snapshot stream. +// ExtensionSnapshotter has a unique name and manages its own internal formats. +type ExtensionSnapshotter interface { + // SnapshotName returns the name of snapshotter, it should be unique in the manager. + SnapshotName() + +string + + // SnapshotFormat returns the default format used to take a snapshot. + SnapshotFormat() + +uint32 + + // SupportedFormats returns a list of formats it can restore from. + SupportedFormats() []uint32 + + // SnapshotExtension writes extension payloads into the underlying protobuf stream. + SnapshotExtension(height uint64, payloadWriter ExtensionPayloadWriter) + +error + + // RestoreExtension restores an extension state snapshot, + // the payload reader returns `io.EOF` when reached the extension boundaries. + RestoreExtension(height uint64, format uint32, payloadReader ExtensionPayloadReader) + +error +} +``` + +## Consequences + +As a result of this implementation, we are able to create snapshots of binary chunk stream for the state that we maintain outside of the IAVL Tree, CosmWasm blobs for example. And new clients are able to fetch sanpshots of state for all modules that have implemented the corresponding interface from peer nodes. + +### Backwards Compatibility + +This ADR introduces new proto message types, add an `extensions` field in snapshot `Manager`, and add new `ExtensionSnapshotter` interface, so this is not backwards compatible if we have extensions. + +But for applications that does not have the state data outside of the IAVL tree for any module, the snapshot stream is backwards-compatible. + +### Positive + +* State maintained outside of IAVL tree like CosmWasm blobs can create snapshots by implementing extension snapshotters, and being fetched by new clients via state-sync. + +### Negative + +### Neutral + +* All modules that maintain state outside of IAVL tree need to implement `ExtensionSnapshotter` and the snapshot `Manager` need to call `RegisterExtensions` when setting up the application. + +## Further Discussions + +While an ADR is in the DRAFT or PROPOSED stage, this section should contain a summary of issues to be solved in future iterations (usually referencing comments from a pull-request discussion). +Later, this section can optionally list ideas or improvements the author or reviewers found during the analysis of this ADR. + +## Test Cases \[optional] + +Test cases for an implementation are mandatory for ADRs that are affecting consensus changes. Other ADRs can choose to include links to test cases if applicable. + +## References + +* [Link](https://github.com/cosmos/cosmos-sdk/pull/10961) +* [Link](https://github.com/cosmos/cosmos-sdk/issues/7340) + diff --git a/sdk/v0.54/reference/architecture/adr-050-sign-mode-textual-annex1.mdx b/sdk/v0.54/reference/architecture/adr-050-sign-mode-textual-annex1.mdx new file mode 100644 index 000000000..66c1c62d7 --- /dev/null +++ b/sdk/v0.54/reference/architecture/adr-050-sign-mode-textual-annex1.mdx @@ -0,0 +1,366 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/architecture/adr-050-sign-mode-textual-annex1' +title: 'ADR 050: SIGN_MODE_TEXTUAL: Annex 1 Value Renderers' +--- + +## Changelog + +* Dec 06, 2021: Initial Draft +* Feb 07, 2022: Draft read and concept-ACKed by the Ledger team. +* Dec 01, 2022: Remove `Object: ` prefix on Any header screen. +* Dec 13, 2022: Sign over bytes hash when bytes length > 32. +* Mar 27, 2023: Update `Any` value renderer to omit message header screen. + +## Status + +Accepted. Implementation started. Small value renderers details still need to be polished. + +## Abstract + +This Annex describes value renderers, which are used for displaying Protobuf values in a human-friendly way using a string array. + +## Value Renderers + +Value Renderers describe how values of different Protobuf types should be encoded as a string array. Value renderers can be formalized as a set of bijective functions `func renderT(value T) []string`, where `T` is one of the below Protobuf types for which this spec is defined. + +### Protobuf `number` + +* Applies to: + * protobuf numeric integer types (`int{32,64}`, `uint{32,64}`, `sint{32,64}`, `fixed{32,64}`, `sfixed{32,64}`) + * strings whose `customtype` is `github.com/cosmos/cosmos-sdk/types.Int` or `github.com/cosmos/cosmos-sdk/types.Dec` + * bytes whose `customtype` is `github.com/cosmos/cosmos-sdk/types.Int` or `github.com/cosmos/cosmos-sdk/types.Dec` +* Trailing decimal zeroes are always removed +* Formatting with `'`s for every three integral digits. +* Usage of `.` to denote the decimal delimiter. + +#### Examples + +* `1000` (uint64) -> `1'000` +* `"1000000.00"` (string representing a Dec) -> `1'000'000` +* `"1000000.10"` (string representing a Dec) -> `1'000'000.1` + +### `coin` + +* Applies to `cosmos.base.v1beta1.Coin`. +* Denoms are converted to `display` denoms using `Metadata` (if available). **This requires a state query**. The definition of `Metadata` can be found in the [bank protobuf definition](https://buf.build/cosmos/cosmos-sdk/docs/main:cosmos.bank.v1beta1#cosmos.bank.v1beta1.Metadata). If the `display` field is empty or nil, then we do not perform any denom conversion. +* Amounts are converted to `display` denom amounts and rendered as `number`s above + * We do not change the capitalization of the denom. In practice, `display` denoms are stored in lowercase in state (e.g. `10 atom`), however they are often showed in UPPERCASE in everyday life (e.g. `10 ATOM`). Value renderers keep the case used in state, but we may recommend chains changing the denom metadata to be uppercase for better user display. +* One space between the denom and amount (e.g. `10 atom`). +* In the future, IBC denoms could maybe be converted to DID/IIDs, if we can find a robust way for doing this (ex. `cosmos:cosmos:hub:bank:denom:atom`) + +#### Examples + +* `1000000000uatom` -> `["1'000 atom"]`, because atom is the metadata's display denom. + +### `coins` + +* an array of `coin` is display as the concatenation of each `coin` encoded as the specification above, the joined together with the delimiter `", "` (a comma and a space, no quotes around). +* the list of coins is ordered by unicode code point of the display denom: `A-Z` < `a-z`. For example, the string `aAbBcC` would be sorted `ABCabc`. + * if the coins list had 0 items in it then it'll be rendered as `zero` + +### Example + +* `["3cosm", "2000000uatom"]` -> `2 atom, 3 COSM` (assuming the display denoms are `atom` and `COSM`) +* `["10atom", "20Acoin"]` -> `20 Acoin, 10 atom` (assuming the display denoms are `atom` and `Acoin`) +* `[]` -> `zero` + +### `repeated` + +* Applies to all `repeated` fields, except `cosmos.tx.v1beta1.TxBody#Messages`, which has a particular encoding (see [ADR-050](/sdk/v0.50/build/architecture/adr-050-sign-mode-textual)). +* A repeated type has the following template: + +``` +: + (/): + + (/): + +End of . +``` + +where: + +* `field_name` is the Protobuf field name of the repeated field +* `field_kind`: + * if the type of the repeated field is a message, `field_kind` is the message name + * if the type of the repeated field is an enum, `field_kind` is the enum name + * in any other case, `field_kind` is the protobuf primitive type (e.g. "string" or "bytes") +* `int` is the length of the array +* `index` is one based index of the repeated field + +#### Examples + +Given the proto definition: + +```protobuf +message AllowedMsgAllowance { + repeated string allowed_messages = 1; +} +``` + +and initializing with: + +```go +x := []AllowedMsgAllowance{"cosmos.bank.v1beta1.MsgSend", "cosmos.gov.v1.MsgVote" +} +``` + +we have the following value-rendered encoding: + +``` +Allowed messages: 2 strings +Allowed messages (1/2): cosmos.bank.v1beta1.MsgSend +Allowed messages (2/2): cosmos.gov.v1.MsgVote +End of Allowed messages +``` + +### `message` + +* Applies to all Protobuf messages that do not have a custom encoding. +* Field names follow [sentence case](https://en.wiktionary.org/wiki/sentence_case) + * replace each `_` with a space + * capitalize first letter of the sentence +* Field names are ordered by their Protobuf field number +* Screen title is the field name, and screen content is the value. +* Nesting: + + * if a field contains a nested message, we value-render the underlying message using the template: + + ``` + : <1st line of value-rendered message> + > // Notice the `>` prefix. + ``` + + * `>` character is used to denote nesting. For each additional level of nesting, add `>`. + +#### Examples + +Given the following Protobuf messages: + +```protobuf expandable +enum VoteOption { + VOTE_OPTION_UNSPECIFIED = 0; + VOTE_OPTION_YES = 1; + VOTE_OPTION_ABSTAIN = 2; + VOTE_OPTION_NO = 3; + VOTE_OPTION_NO_WITH_VETO = 4; +} + +message WeightedVoteOption { + VoteOption option = 1; + string weight = 2 [(cosmos_proto.scalar) = "cosmos.Dec"]; +} + +message Vote { + uint64 proposal_id = 1; + string voter = 2 [(cosmos_proto.scalar) = "cosmos.AddressString"]; + reserved 3; + repeated WeightedVoteOption options = 4; +} +``` + +we get the following encoding for the `Vote` message: + +``` +Vote object +> Proposal id: 4 +> Voter: cosmos1abc...def +> Options: 2 WeightedVoteOptions +> Options (1/2): WeightedVoteOption object +>> Option: VOTE_OPTION_YES +>> Weight: 0.7 +> Options (2/2): WeightedVoteOption object +>> Option: VOTE_OPTION_NO +>> Weight: 0.3 +> End of Options +``` + +### Enums + +* Show the enum variant name as string. + +#### Examples + +See example above with `message Vote{}`. + +### `google.protobuf.Any` + +* Applies to `google.protobuf.Any` +* Rendered as: + +``` + +> +``` + +There is however one exception: when the underlying message is a Protobuf message that does not have a custom encoding, then the message header screen is omitted, and one level of indentation is removed. + +Messages that have a custom encoding, including `google.protobuf.Timestamp`, `google.protobuf.Duration`, `google.protobuf.Any`, `cosmos.base.v1beta1.Coin`, and messages that have an app-defined custom encoding, will preserve their header and indentation level. + +#### Examples + +Message header screen is stripped, one-level of indentation removed: + +``` +/cosmos.gov.v1.Vote +> Proposal id: 4 +> Vote: cosmos1abc...def +> Options: 2 WeightedVoteOptions +> Options (1/2): WeightedVoteOption object +>> Option: Yes +>> Weight: 0.7 +> Options (2/2): WeightedVoteOption object +>> Option: No +>> Weight: 0.3 +> End of Options +``` + +Message with custom encoding: + +``` +/cosmos.base.v1beta1.Coin +> 10uatom +``` + +### `google.protobuf.Timestamp` + +Rendered using [RFC 3339](https://www.rfc-editor.org/rfc/rfc3339) (a +simplification of ISO 8601), which is the current recommendation for portable +time values. The rendering always uses "Z" (UTC) as the timezone. It uses only +the necessary fractional digits of a second, omitting the fractional part +entirely if the timestamp has no fractional seconds. (The resulting timestamps +are not automatically sortable by standard lexicographic order, but we favor +the legibility of the shorter string.) + +#### Examples + +The timestamp with 1136214245 seconds and 700000000 nanoseconds is rendered +as `2006-01-02T15:04:05.7Z`. +The timestamp with 1136214245 seconds and zero nanoseconds is rendered +as `2006-01-02T15:04:05Z`. + +### `google.protobuf.Duration` + +The duration proto expresses a raw number of seconds and nanoseconds. +This will be rendered as longer time units of days, hours, and minutes, +plus any remaining seconds, in that order. +Leading and trailing zero-quantity units will be omitted, but all +units in between nonzero units will be shown, e.g. ` 3 days, 0 hours, 0 minutes, 5 seconds`. + +Even longer time units such as months or years are imprecise. +Weeks are precise, but not commonly used - `91 days` is more immediately +legible than `13 weeks`. Although `days` can be problematic, +e.g. noon to noon on subsequent days can be 23 or 25 hours depending on +daylight savings transitions, there is significant advantage in using +strict 24-hour days over using only hours (e.g. `91 days` vs `2184 hours`). + +When nanoseconds are nonzero, they will be shown as fractional seconds, +with only the minimum number of digits, e.g `0.5 seconds`. + +A duration of exactly zero is shown as `0 seconds`. + +Units will be given as singular (no trailing `s`) when the quantity is exactly one, +and will be shown in plural otherwise. + +Negative durations will be indicated with a leading minus sign (`-`). + +Examples: + +* `1 day` +* `30 days` +* `-1 day, 12 hours` +* `3 hours, 0 minutes, 53.025 seconds` + +### bytes + +* Bytes of length shorter or equal to 35 are rendered in hexadecimal, all capital letters, without the `0x` prefix. +* Bytes of length greater than 35 are hashed using SHA256. The rendered text is `SHA-256=`, followed by the 32-byte hash, in hexadecimal, all capital letters, without the `0x` prefix. +* The hexadecimal string is finally separated into groups of 4 digits, with a space `' '` as separator. If the bytes length is odd, the 2 remaining hexadecimal characters are at the end. + +The number 35 was chosen because it is the longest length where the hashed-and-prefixed representation is longer than the original data directly formatted, using the 3 rules above. More specifically: + +* a 35-byte array will have 70 hex characters, plus 17 space characters, resulting in 87 characters. +* byte arrays starting from length 36 will be be hashed to 32 bytes, which is 64 hex characters plus 15 spaces, and with the `SHA-256=` prefix, it takes 87 characters. + Also, secp256k1 public keys have length 33, so their Textual representation is not their hashed value, which we would like to avoid. + +Note: Data longer than 35 bytes are not rendered in a way that can be inverted. See ADR-050's [section about invertability](/sdk/v0.50/build/architecture/adr-050-sign-mode-textual#invertible-rendering) for a discussion. + +#### Examples + +Inputs are displayed as byte arrays. + +* `[0]`: `00` +* `[0,1,2]`: `0001 02` +* `[0,1,2,..,34]`: `0001 0203 0405 0607 0809 0A0B 0C0D 0E0F 1011 1213 1415 1617 1819 1A1B 1C1D 1E1F 2021 22` +* `[0,1,2,..,35]`: `SHA-256=5D7E 2D9B 1DCB C85E 7C89 0036 A2CF 2F9F E7B6 6554 F2DF 08CE C6AA 9C0A 25C9 9C21` + +### address bytes + +We currently use `string` types in protobuf for addresses so this may not be needed, but if any address bytes are used in sign mode textual they should be rendered with bech32 formatting + +### strings + +Strings are rendered as-is. + +### Default Values + +* Default Protobuf values for each field are skipped. + +#### Example + +```protobuf +message TestData { + string signer = 1; + string metadata = 2; +} +``` + +```go +myTestData := TestData{ + Signer: "cosmos1abc" +} +``` + +We get the following encoding for the `TestData` message: + +``` +TestData object +> Signer: cosmos1abc +``` + +### bool + +Boolean values are rendered as `True` or `False`. + +### \[ABANDONED] Custom `msg_title` instead of Msg `type_url` + +*This paragraph is in the Annex for informational purposes only, and will be removed in a next update of the ADR.* + + + +* all protobuf messages to be used with `SIGN_MODE_TEXTUAL` CAN have a short title associated with them that can be used in format strings whenever the type URL is explicitly referenced via the `cosmos.msg.v1.textual.msg_title` Protobuf message option. +* if this option is not specified for a Msg, then the Protobuf fully qualified name will be used. + +```protobuf +message MsgSend { + option (cosmos.msg.v1.textual.msg_title) = "bank send coins"; +} +``` + +* they MUST be unique per message, per chain + +#### Examples + +* `cosmos.gov.v1.MsgVote` -> `governance v1 vote` + +#### Best Pratices + +We recommend to use this option only for `Msg`s whose Protobuf fully qualified name can be hard to understand. As such, the two examples above (`MsgSend` and `MsgVote`) are not good examples to be used with `msg_title`. We still allow `msg_title` for chains who might have `Msg`s with complex or non-obvious names. + +In those cases, we recommend to drop the version (e.g. `v1`) in the string if there's only one version of the module on chain. This way, the bijective mapping can figure out which message each string corresponds to. If multiple Protobuf versions of the same module exist on the same chain, we recommend keeping the first `msg_title` with version, and the second `msg_title` with version (e.g. `v2`): + +* `mychain.mymodule.v1.MsgDo` -> `mymodule do something` +* `mychain.mymodule.v2.MsgDo` -> `mymodule v2 do something` + + diff --git a/sdk/v0.54/reference/architecture/adr-050-sign-mode-textual-annex2.mdx b/sdk/v0.54/reference/architecture/adr-050-sign-mode-textual-annex2.mdx new file mode 100644 index 000000000..10cb1c82b --- /dev/null +++ b/sdk/v0.54/reference/architecture/adr-050-sign-mode-textual-annex2.mdx @@ -0,0 +1,127 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/architecture/adr-050-sign-mode-textual-annex2' +title: 'ADR 050: SIGN_MODE_TEXTUAL: Annex 2 XXX' +description: 'Oct 3, 2022: Initial Draft' +--- + +## Changelog + +* Oct 3, 2022: Initial Draft + +## Status + +DRAFT + +## Abstract + +This annex provides normative guidance on how devices should render a +`SIGN_MODE_TEXTUAL` document. + +## Context + +`SIGN_MODE_TEXTUAL` allows a legible version of a transaction to be signed +on a hardware security device, such as a Ledger. Early versions of the +design rendered transactions directly to lines of ASCII text, but this +proved awkward from its in-band signaling, and for the need to display +Unicode text within the transaction. + +## Decision + +`SIGN_MODE_TEXTUAL` renders to an abstract representation, leaving it +up to device-specific software how to present this representation given the +capabilities, limitations, and conventions of the deivce. + +We offer the following normative guidance: + +1. The presentation should be as legible as possible to the user, given + the capabilities of the device. If legibility could be sacrificed for other + properties, we would recommend just using some other signing mode. + Legibility should focus on the common case - it is okay for unusual cases + to be less legible. + +2. The presentation should be invertible if possible without substantial + sacrifice of legibility. Any change to the rendered data should result + in a visible change to the presentation. This extends the integrity of the + signing to user-visible presentation. + +3. The presentation should follow normal conventions of the device, + without sacrificing legibility or invertibility. + +As an illustration of these principles, here is an example algorithm +for presentation on a device which can display a single 80-character +line of printable ASCII characters: + +* The presentation is broken into lines, and each line is presented in + sequence, with user controls for going forward or backward a line. + +* Expert mode screens are only presented if the device is in expert mode. + +* Each line of the screen starts with a number of `>` characters equal + to the screen's indentation level, followed by a `+` character if this + isn't the first line of the screen, followed by a space if either a + `>` or a `+` has been emitted, + or if this header is followed by a `>`, `+`, or space. + +* If the line ends with whitespace or an `@` character, an additional `@` + character is appended to the line. + +* The following ASCII control characters or backslash (`\`) are converted + to a backslash followed by a letter code, in the manner of string literals + in many languages: + + * a: U+0007 alert or bell + * b: U+0008 backspace + * f: U+000C form feed + * n: U+000A line feed + * r: U+000D carriage return + * t: U+0009 horizontal tab + * v: U+000B vertical tab + * `\`: U+005C backslash + +* All other ASCII control characters, plus non-ASCII Unicode code points, + are shown as either: + + * `\u` followed by 4 uppercase hex chacters for code points + in the basic multilingual plane (BMP). + + * `\U` followed by 8 uppercase hex characters for other code points. + +* The screen will be broken into multiple lines to fit the 80-character + limit, considering the above transformations in a way that attempts to + minimize the number of lines generated. Expanded control or Unicode characters + are never split across lines. + +Example output: + +``` +An introductory line. +key1: 123456 +key2: a string that ends in whitespace @ +key3: a string that ends in a single ampersand - @@ + >tricky key4<: note the leading space in the presentation +introducing an aggregate +> key5: false +> key6: a very long line of text, please co\u00F6perate and break into +>+ multiple lines. +> Can we do further nesting? +>> You bet we can! +``` + +The inverse mapping gives us the only input which could have +generated this output (JSON notation for string data): + +``` +Indent Text +------ ---- +0 "An introductory line." +0 "key1: 123456" +0 "key2: a string that ends in whitespace " +0 "key3: a string that ends in a single ampersand - @" +0 ">tricky key4<: note the leading space in the presentation" +0 "introducing an aggregate" +1 "key5: false" +1 "key6: a very long line of text, please coöperate and break into multiple lines." +1 "Can we do further nesting?" +2 "You bet we can!" +``` diff --git a/sdk/v0.54/reference/architecture/adr-050-sign-mode-textual.mdx b/sdk/v0.54/reference/architecture/adr-050-sign-mode-textual.mdx new file mode 100644 index 000000000..e7884e30f --- /dev/null +++ b/sdk/v0.54/reference/architecture/adr-050-sign-mode-textual.mdx @@ -0,0 +1,377 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/architecture/adr-050-sign-mode-textual' +title: 'ADR 050: SIGN_MODE_TEXTUAL' +--- + +## Changelog + +* Dec 06, 2021: Initial Draft. +* Feb 07, 2022: Draft read and concept-ACKed by the Ledger team. +* May 16, 2022: Change status to Accepted. +* Aug 11, 2022: Require signing over tx raw bytes. +* Sep 07, 2022: Add custom `Msg`-renderers. +* Sep 18, 2022: Structured format instead of lines of text +* Nov 23, 2022: Specify CBOR encoding. +* Dec 01, 2022: Link to examples in separate JSON file. +* Dec 06, 2022: Re-ordering of envelope screens. +* Dec 14, 2022: Mention exceptions for invertability. +* Jan 23, 2023: Switch Screen.Text to Title+Content. +* Mar 07, 2023: Change SignDoc from array to struct containing array. +* Mar 20, 2023: Introduce a spec version initialized to 0. + +## Status + +Accepted. Implementation started. Small value renderers details still need to be polished. + +Spec version: 0. + +## Abstract + +This ADR specifies SIGN\_MODE\_TEXTUAL, a new string-based sign mode that is targetted at signing with hardware devices. + +## Context + +Protobuf-based SIGN\_MODE\_DIRECT was introduced in [ADR-020](/sdk/v0.50/build/architecture/adr-020-protobuf-transaction-encoding) and is intended to replace SIGN\_MODE\_LEGACY\_AMINO\_JSON in most situations, such as mobile wallets and CLI keyrings. However, the [Ledger](https://www.ledger.com/) hardware wallet is still using SIGN\_MODE\_LEGACY\_AMINO\_JSON for displaying the sign bytes to the user. Hardware wallets cannot transition to SIGN\_MODE\_DIRECT as: + +* SIGN\_MODE\_DIRECT is binary-based and thus not suitable for display to end-users. Technically, hardware wallets could simply display the sign bytes to the user. But this would be considered as blind signing, and is a security concern. +* hardware cannot decode the protobuf sign bytes due to memory constraints, as the Protobuf definitions would need to be embedded on the hardware device. + +In an effort to remove Amino from the SDK, a new sign mode needs to be created for hardware devices. [Initial discussions](https://github.com/cosmos/cosmos-sdk/issues/6513) propose a text-based sign mode, which this ADR formally specifies. + +## Decision + +In SIGN\_MODE\_TEXTUAL, a transaction is rendered into a textual representation, +which is then sent to a secure device or subsystem for the user to review and sign. +Unlike `SIGN_MODE_DIRECT`, the transmitted data can be simply decoded into legible text +even on devices with limited processing and display. + +The textual representation is a sequence of *screens*. +Each screen is meant to be displayed in its entirety (if possible) even on a small device like a Ledger. +A screen is roughly equivalent to a short line of text. +Large screens can be displayed in several pieces, +much as long lines of text are wrapped, +so no hard guidance is given, though 40 characters is a good target. +A screen is used to display a single key/value pair for scalar values +(or composite values with a compact notation, such as `Coins`) +or to introduce or conclude a larger grouping. + +The text can contain the full range of Unicode code points, including control characters and nul. +The device is responsible for deciding how to display characters it cannot render natively. +See [annex 2](/sdk/v0.50/build/architecture/adr-050-sign-mode-textual-annex2) for guidance. + +Screens have a non-negative indentation level to signal composite or nested structures. +Indentation level zero is the top level. +Indentation is displayed via some device-specific mechanism. +Message quotation notation is an appropriate model, such as +leading `>` characters or vertical bars on more capable displays. + +Some screens are marked as *expert* screens, +meant to be displayed only if the viewer chooses to opt in for the extra detail. +Expert screens are meant for information that is rarely useful, +or needs to be present only for signature integrity (see below). + +### Invertible Rendering + +We require that the rendering of the transaction be invertible: +there must be a parsing function such that for every transaction, +when rendered to the textual representation, +parsing that representation yeilds a proto message equivalent +to the original under proto equality. + +Note that this inverse function does not need to perform correct +parsing or error signaling for the whole domain of textual data. +Merely that the range of valid transactions be invertible under +the composition of rendering and parsing. + +Note that the existence of an inverse function ensures that the +rendered text contains the full information of the original transaction, +not a hash or subset. + +We make an exception for invertibility for data which are too large to +meaningfully display, such as byte strings longer than 32 bytes. We may then +selectively render them with a cryptographically-strong hash. In these cases, +it is still computationally infeasible to find a different transaction which +has the same rendering. However, we must ensure that the hash computation is +simple enough to be reliably executed independently, so at least the hash is +itself reasonably verifiable when the raw byte string is not. + +### Chain State + +The rendering function (and parsing function) may depend on the current chain state. +This is useful for reading parameters, such as coin display metadata, +or for reading user-specific preferences such as language or address aliases. +Note that if the observed state changes between signature generation +and the transaction's inclusion in a block, the delivery-time rendering +might differ. If so, the signature will be invalid and the transaction +will be rejected. + +### Signature and Security + +For security, transaction signatures should have three properties: + +1. Given the transaction, signatures, and chain state, it must be possible to validate that the signatures matches the transaction, + to verify that the signers must have known their respective secret keys. + +2. It must be computationally infeasible to find a substantially different transaction for which the given signatures are valid, given the same chain state. + +3. The user should be able to give informed consent to the signed data via a simple, secure device with limited display capabilities. + +The correctness and security of `SIGN_MODE_TEXTUAL` is guaranteed by demonstrating an inverse function from the rendering to transaction protos. +This means that it is impossible for a different protocol buffer message to render to the same text. + +### Transaction Hash Malleability + +When client software forms a transaction, the "raw" transaction (`TxRaw`) is serialized as a proto +and a hash of the resulting byte sequence is computed. +This is the `TxHash`, and is used by various services to track the submitted transaction through its lifecycle. +Various misbehavior is possible if one can generate a modified transaction with a different TxHash +but for which the signature still checks out. + +SIGN\_MODE\_TEXTUAL prevents this transaction malleability by including the TxHash as an expert screen +in the rendering. + +### SignDoc + +The SignDoc for `SIGN_MODE_TEXTUAL` is formed from a data structure like: + +```go +type Screen struct { + Title string // possibly size limited to, advised to 64 characters + Content string // possibly size limited to, advised to 255 characters + Indent uint8 // size limited to something small like 16 or 32 + Expert bool +} + +type SignDocTextual struct { + Screens []Screen +} +``` + +We do not plan to use protobuf serialization to form the sequence of bytes +that will be tranmitted and signed, in order to keep the decoder simple. +We will use [CBOR](https://cbor.io) ([RFC 8949](https://www.rfc-editor.org/rfc/rfc8949.html)) instead. +The encoding is defined by the following CDDL ([RFC 8610](https://www.rfc-editor.org/rfc/rfc8610)): + +``` +;;; CDDL (RFC 8610) Specification of SignDoc for SIGN_MODE_TEXTUAL. +;;; Must be encoded using CBOR deterministic encoding (RFC 8949, section 4.2.1). + +;; A Textual document is a struct containing one field: an array of screens. +sign_doc = { + screens_key: [* screen], +} + +;; The key is an integer to keep the encoding small. +screens_key = 1 + +;; A screen consists of a text string, an indentation, and the expert flag, +;; represented as an integer-keyed map. All entries are optional +;; and MUST be omitted from the encoding if empty, zero, or false. +;; Text defaults to the empty string, indent defaults to zero, +;; and expert defaults to false. +screen = { + ? title_key: tstr, + ? content_key: tstr, + ? indent_key: uint, + ? expert_key: bool, +} + +;; Keys are small integers to keep the encoding small. +title_key = 1 +content_key = 2 +indent_key = 3 +expert_key = 4 +``` + +Defining the sign\_doc as directly an array of screens has also been considered. However, given the possibility of future iterations of this specification, using a single-keyed struct has been chosen over the former proposal, as structs allow for easier backwards-compatibility. + +## Details + +In the examples that follow, screens will be shown as lines of text, +indentation is indicated with a leading '>', +and expert screens are marked with a leading `*`. + +### Encoding of the Transaction Envelope + +We define "transaction envelope" as all data in a transaction that is not in the `TxBody.Messages` field. Transaction envelope includes fee, signer infos and memo, but don't include `Msg`s. `//` denotes comments and are not shown on the Ledger device. + +```protobuf expandable +Chain ID: +Account number: +Sequence: +Address: +*Public Key: +This transaction has Message(s) // Pluralize "Message" only when int>1 +> Message (/): // See value renderers for Any rendering. +End of Message +Memo: // Skipped if no memo set. +Fee: // See value renderers for coins rendering. +*Fee payer: // Skipped if no fee_payer set. +*Fee granter: // Skipped if no fee_granter set. +Tip: // Skippted if no tip. +Tipper: +*Gas Limit: +*Timeout Height: // Skipped if no timeout_height set. +*Other signer: SignerInfo // Skipped if the transaction only has 1 signer. +*> Other signer (/): +*End of other signers +*Extension options: Any: // Skipped if no body extension options +*> Extension options (/): +*End of extension options +*Non critical extension options: Any: // Skipped if no body non critical extension options +*> Non critical extension options (/): +*End of Non critical extension options +*Hash of raw bytes: // Hex encoding of bytes defined, to prevent tx hash malleability. +``` + +### Encoding of the Transaction Body + +Transaction Body is the `Tx.TxBody.Messages` field, which is an array of `Any`s, where each `Any` packs a `sdk.Msg`. Since `sdk.Msg`s are widely used, they have a slightly different encoding than usual array of `Any`s (Protobuf: `repeated google.protobuf.Any`) described in Annex 1. + +``` +This transaction has message: // Optional 's' for "message" if there's is >1 sdk.Msgs. +// For each Msg, print the following 2 lines: +Msg (/): // E.g. Msg (1/2): bank v1beta1 send coins + +End of transaction messages +``` + +#### Example + +Given the following Protobuf message: + +```protobuf expandable +message Grant { + google.protobuf.Any authorization = 1 [(cosmos_proto.accepts_interface) = "cosmos.authz.v1beta1.Authorization"]; + google.protobuf.Timestamp expiration = 2 [(gogoproto.stdtime) = true, (gogoproto.nullable) = false]; +} + +message MsgGrant { + option (cosmos.msg.v1.signer) = "granter"; + + string granter = 1 [(cosmos_proto.scalar) = "cosmos.AddressString"]; + string grantee = 2 [(cosmos_proto.scalar) = "cosmos.AddressString"]; +} +``` + +and a transaction containing 1 such `sdk.Msg`, we get the following encoding: + +``` +This transaction has 1 message: +Msg (1/1): authz v1beta1 grant +Granter: cosmos1abc...def +Grantee: cosmos1ghi...jkl +End of transaction messages +``` + +### Custom `Msg` Renderers + +Application developers may choose to not follow default renderer value output for their own `Msg`s. In this case, they can implement their own custom `Msg` renderer. This is similar to [EIP4430](https://github.com/ethereum/EIPs/blob/master/EIPS/eip-4430.md), where the smart contract developer chooses the description string to be shown to the end user. + +This is done by setting the `cosmos.msg.textual.v1.expert_custom_renderer` Protobuf option to a non-empty string. This option CAN ONLY be set on a Protobuf message representing transaction message object (implementing `sdk.Msg` interface). + +```protobuf +message MsgFooBar { + // Optional comments to describe in human-readable language the formatting + // rules of the custom renderer. + option (cosmos.msg.textual.v1.expert_custom_renderer) = ""; + + // proto fields +} +``` + +When this option is set on a `Msg`, a registered function will transform the `Msg` into an array of one or more strings, which MAY use the key/value format (described in point #3) with the expert field prefix (described in point #5) and arbitrary indentation (point #6). These strings MAY be rendered from a `Msg` field using a default value renderer, or they may be generated from several fields using custom logic. + +The `` is a string convention chosen by the application developer and is used to identify the custom `Msg` renderer. For example, the documentation or specification of this custom algorithm can reference this identifier. This identifier CAN have a versioned suffix (e.g. `_v1`) to adapt for future changes (which would be consensus-breaking). We also recommend adding Protobuf comments to describe in human language the custom logic used. + +Moreover, the renderer must provide 2 functions: one for formatting from Protobuf to string, and one for parsing string to Protobuf. These 2 functions are provided by the application developer. To satisfy point #1, the parse function MUST be the inverse of the formatting function. This property will not be checked by the SDK at runtime. However, we strongly recommend the application developer to include a comprehensive suite in their app repo to test invertibility, as to not introduce security bugs. + +### Require signing over the `TxBody` and `AuthInfo` raw bytes + +Recall that the transaction bytes merklelized on chain are the Protobuf binary serialization of [TxRaw](hhttps://buf.build/cosmos/cosmos-sdk/sdk/v0.50/main:cosmos.tx.v1beta1#cosmos.tx.v1beta1.TxRaw), which contains the `body_bytes` and `auth_info_bytes`. Moreover, the transaction hash is defined as the SHA256 hash of the `TxRaw` bytes. We require that the user signs over these bytes in SIGN\_MODE\_TEXTUAL, more specifically over the following string: + +``` +*Hash of raw bytes: +``` + +where: + +* `++` denotes concatenation, +* `HEX` is the hexadecimal representation of the bytes, all in capital letters, no `0x` prefix, +* and `len()` is encoded as a Big-Endian uint64. + +This is to prevent transaction hash malleability. The point #1 about invertiblity assures that transaction `body` and `auth_info` values are not malleable, but the transaction hash still might be malleable with point #1 only, because the SIGN\_MODE\_TEXTUAL strings don't follow the byte ordering defined in `body_bytes` and `auth_info_bytes`. Without this hash, a malicious validator or exchange could intercept a transaction, modify its transaction hash *after* the user signed it using SIGN\_MODE\_TEXTUAL (by tweaking the byte ordering inside `body_bytes` or `auth_info_bytes`), and then submit it to Tendermint. + +By including this hash in the SIGN\_MODE\_TEXTUAL signing payload, we keep the same level of guarantees as [SIGN\_MODE\_DIRECT](/sdk/v0.50/build/architecture/adr-020-protobuf-transaction-encoding). + +These bytes are only shown in expert mode, hence the leading `*`. + +## Updates to the current specification + +The current specification is not set in stone, and future iterations are to be expected. We distinguish two categories of updates to this specification: + +1. Updates that require changes of the hardware device embedded application. +2. Updates that only modify the envelope and the value renderers. + +Updates in the 1st category include changes of the `Screen` struct or its corresponding CBOR encoding. This type of updates require a modification of the hardware signer application, to be able to decode and parse the new types. Backwards-compatibility must also be guaranteed, so that the new hardware application works with existing versions of the SDK. These updates require the coordination of multiple parties: SDK developers, hardware application developers (currently: Zondax), and client-side developers (e.g. CosmJS). Furthermore, a new submission of the hardware device application may be necessary, which, dependending on the vendor, can take some time. As such, we recommend to avoid this type of updates as much as possible. + +Updates in the 2nd category include changes to any of the value renderers or to the transaction envelope. For example, the ordering of fields in the envelope can be swapped, or the timestamp formatting can be modified. Since SIGN\_MODE\_TEXTUAL sends `Screen`s to the hardware device, this type of change do not need a hardware wallet application update. They are however state-machine-breaking, and must be documented as such. They require the coordination of SDK developers with client-side developers (e.g. CosmJS), so that the updates are released on both sides close to each other in time. + +We define a spec version, which is an integer that must be incremented on each update of either category. This spec version will be exposed by the SDK's implementation, and can be communicated to clients. For example, SDK v0.50 might use the spec version 1, and SDK v0.51 might use 2; thanks to this versioning, clients can know how to craft SIGN\_MODE\_TEXTUAL transactions based on the target SDK version. + +The current spec version is defined in the "Status" section, on the top of this document. It is initialized to `0` to allow flexibility in choosing how to define future versions, as it would allow adding a field either in the SignDoc Go struct or in Protobuf in a backwards-compatible way. + +## Additional Formatting by the Hardware Device + +See [annex 2](/sdk/v0.50/build/architecture/adr-050-sign-mode-textual-annex2). + +## Examples + +1. A minimal MsgSend: [see transaction](https://github.com/cosmos/cosmos-sdk/blob/094abcd393379acbbd043996024d66cd65246fb1/tx/textual/internal/testdata/e2e.json#L2-L70). +2. A transaction with a bit of everything: [see transaction](https://github.com/cosmos/cosmos-sdk/blob/094abcd393379acbbd043996024d66cd65246fb1/tx/textual/internal/testdata/e2e.json#L71-L270). + +The examples below are stored in a JSON file with the following fields: + +* `proto`: the representation of the transaction in ProtoJSON, +* `screens`: the transaction rendered into SIGN\_MODE\_TEXTUAL screens, +* `cbor`: the sign bytes of the transaction, which is the CBOR encoding of the screens. + +## Consequences + +### Backwards Compatibility + +SIGN\_MODE\_TEXTUAL is purely additive, and doesn't break any backwards compatibility with other sign modes. + +### Positive + +* Human-friendly way of signing in hardware devices. +* Once SIGN\_MODE\_TEXTUAL is shipped, SIGN\_MODE\_LEGACY\_AMINO\_JSON can be deprecated and removed. On the longer term, once the ecosystem has totally migrated, Amino can be totally removed. + +### Negative + +* Some fields are still encoded in non-human-readable ways, such as public keys in hexadecimal. +* New ledger app needs to be released, still unclear + +### Neutral + +* If the transaction is complex, the string array can be arbitrarily long, and some users might just skip some screens and blind sign. + +## Further Discussions + +* Some details on value renderers need to be polished, see [Annex 1](/sdk/v0.50/build/architecture/adr-050-sign-mode-textual-annex1). +* Are ledger apps able to support both SIGN\_MODE\_LEGACY\_AMINO\_JSON and SIGN\_MODE\_TEXTUAL at the same time? +* Open question: should we add a Protobuf field option to allow app developers to overwrite the textual representation of certain Protobuf fields and message? This would be similar to Ethereum's [EIP4430](https://github.com/ethereum/EIPs/pull/4430), where the contract developer decides on the textual representation. +* Internationalization. + +## References + +* [Annex 1](/sdk/v0.50/build/architecture/adr-050-sign-mode-textual-annex1) + +* Initial discussion: [Link](https://github.com/cosmos/cosmos-sdk/issues/6513) + +* Living document used in the working group: [Link](https://hackmd.io/fsZAO-TfT0CKmLDtfMcKeA?both) + +* Working group meeting notes: [Link](https://hackmd.io/7RkGfv_rQAaZzEigUYhcXw) + +* Ethereum's "Described Transactions" [Link](https://github.com/ethereum/EIPs/pull/4430) diff --git a/sdk/v0.54/reference/architecture/adr-053-go-module-refactoring.mdx b/sdk/v0.54/reference/architecture/adr-053-go-module-refactoring.mdx new file mode 100644 index 000000000..c4facae33 --- /dev/null +++ b/sdk/v0.54/reference/architecture/adr-053-go-module-refactoring.mdx @@ -0,0 +1,115 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/architecture/adr-053-go-module-refactoring' +title: 'ADR 053: Go Module Refactoring' +description: '2022-04-27: First Draft' +--- + +## Changelog + +* 2022-04-27: First Draft + +## Status + +PROPOSED + +## Abstract + +The current SDK is built as a single monolithic go module. This ADR describes +how we refactor the SDK into smaller independently versioned go modules +for ease of maintenance. + +## Context + +Go modules impose certain requirements on software projects with respect to +stable version numbers (anything above 0.x) in that [any API breaking changes +necessitate a major version](https://go.dev/doc/modules/release-workflow#breaking) +increase which technically creates a new go module +(with a v2, v3, etc. suffix). + +[Keeping modules API compatible](https://go.dev/blog/module-compatibility) in +this way requires a fair amount of fair thought and discipline. + +The Cosmos SDK is a fairly large project which originated before go modules +came into existence and has always been under a v0.x release even though +it has been used in production for years now, not because it isn't production +quality software, but rather because the API compatibility guarantees required +by go modules are fairly complex to adhere to with such a large project. +Up to now, it has generally been deemed more important to be able to break the +API if needed rather than require all users update all package import paths +to accommodate breaking changes causing v2, v3, etc. releases. This is in +addition to the other complexities related to protobuf generated code that will +be addressed in a separate ADR. + +Nevertheless, the desire for semantic versioning has been [strong in the +community](https://github.com/cosmos/cosmos-sdk/discussions/10162) and the +single go module release process has made it very hard to +release small changes to isolated features in a timely manner. Release cycles +often exceed six months which means small improvements done in a day or +two get bottle-necked by everything else in the monolithic release cycle. + +## Decision + +To improve the current situation, the SDK is being refactored into multiple +go modules within the current repository. There has been a [fair amount of +debate](https://github.com/cosmos/cosmos-sdk/discussions/10582#discussioncomment-1813377) +as to how to do this, with some developers arguing for larger vs smaller +module scopes. There are pros and cons to both approaches (which will be +discussed below in the [Consequences](#consequences) section), but the +approach being adopted is the following: + +* a go module should generally be scoped to a specific coherent set of + functionality (such as math, errors, store, etc.) +* when code is removed from the core SDK and moved to a new module path, every + effort should be made to avoid API breaking changes in the existing code using + aliases and wrapper types (as done in [Link](https://github.com/cosmos/cosmos-sdk/pull/10779) + and [Link](https://github.com/cosmos/cosmos-sdk/pull/11788)) +* new go modules should be moved to a standalone domain (`cosmossdk.io`) before + being tagged as `v1.0.0` to accommodate the possibility that they may be + better served by a standalone repository in the future +* all go modules should follow the guidelines in [Link](https://go.dev/blog/module-compatibility) + before `v1.0.0` is tagged and should make use of `internal` packages to limit + the exposed API surface +* the new go module's API may deviate from the existing code where there are + clear improvements to be made or to remove legacy dependencies (for instance on + amino or gogo proto), as long the old package attempts + to avoid API breakage with aliases and wrappers +* care should be taken when simply trying to turn an existing package into a + new go module: [Link](https://github.com/golang/go/wiki/Modules#is-it-possible-to-add-a-module-to-a-multi-module-repository). + In general, it seems safer to just create a new module path (appending v2, v3, etc. + if necessary), rather than trying to make an old package a new module. + +## Consequences + +### Backwards Compatibility + +If the above guidelines are followed to use aliases or wrapper types pointing +in existing APIs that point back to the new go modules, there should be no or +very limited breaking changes to existing APIs. + +### Positive + +* standalone pieces of software will reach `v1.0.0` sooner +* new features to specific functionality will be released sooner + +### Negative + +* there will be more go module versions to update in the SDK itself and + per-project, although most of these will hopefully be indirect + +### Neutral + +## Further Discussions + +Further discussions are occurring in primarily in +[Link](https://github.com/cosmos/cosmos-sdk/discussions/10582) and within +the Cosmos SDK Framework Working Group. + +## References + +* [Link](https://go.dev/doc/modules/release-workflow) +* [Link](https://go.dev/blog/module-compatibility) +* [Link](https://github.com/cosmos/cosmos-sdk/discussions/10162) +* [Link](https://github.com/cosmos/cosmos-sdk/discussions/10582) +* [Link](https://github.com/cosmos/cosmos-sdk/pull/10779) +* [Link](https://github.com/cosmos/cosmos-sdk/pull/11788) diff --git a/sdk/v0.54/reference/architecture/adr-054-semver-compatible-modules.mdx b/sdk/v0.54/reference/architecture/adr-054-semver-compatible-modules.mdx new file mode 100644 index 000000000..cb75f3c3f --- /dev/null +++ b/sdk/v0.54/reference/architecture/adr-054-semver-compatible-modules.mdx @@ -0,0 +1,801 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/architecture/adr-054-semver-compatible-modules' +title: 'ADR 054: Semver Compatible SDK Modules' +description: '2022-04-27: First draft' +--- + +## Changelog + +* 2022-04-27: First draft + +## Status + +DRAFT + +## Abstract + +In order to move the Cosmos SDK to a system of decoupled semantically versioned +modules which can be composed in different combinations (ex. staking v3 with +bank v1 and distribution v2), we need to reassess how we organize the API surface +of modules to avoid problems with go semantic import versioning and +circular dependencies. This ADR explores various approaches we can take to +addressing these issues. + +## Context + +There has been [a fair amount of desire](https://github.com/cosmos/cosmos-sdk/discussions/10162) +in the community for semantic versioning in the SDK and there has been significant +movement to splitting SDK modules into [standalone go modules](https://github.com/cosmos/cosmos-sdk/issues/11899). +Both of these will ideally allow the ecosystem to move faster because we won't +be waiting for all dependencies to update synchronously. For instance, we could +have 3 versions of the core SDK compatible with the latest 2 releases of +CosmWasm as well as 4 different versions of staking . This sort of setup would +allow early adopters to aggressively integrate new versions, while allowing +more conservative users to be selective about which versions they're ready for. + +In order to achieve this, we need to solve the following problems: + +1. because of the way [go semantic import versioning](https://research.swtch.com/vgo-import) (SIV) + works, moving to SIV naively will actually make it harder to achieve these goals +2. circular dependencies between modules need to be broken to actually release + many modules in the SDK independently +3. pernicious minor version incompatibilities introduced through correctly + [evolving protobuf schemas](https://developers.google.com/protocol-buffers/docs/proto3#updating) + without correct [unknown field filtering](/sdk/v0.50/build/architecture/adr-020-protobuf-transaction-encoding#unknown-field-filtering) + +Note that all the following discussion assumes that the proto file versioning and state machine versioning of a module +are distinct in that: + +* proto files are maintained in a non-breaking way (using something + like [buf breaking](https://docs.buf.build/breaking/overview) + to ensure all changes are backwards compatible) +* proto file versions get bumped much less frequently, i.e. we might maintain `cosmos.bank.v1` through many versions + of the bank module state machine +* state machine breaking changes are more common and ideally this is what we'd want to semantically version with + go modules, ex. `x/bank/v2`, `x/bank/v3`, etc. + +### Problem 1: Semantic Import Versioning Compatibility + +Consider we have a module `foo` which defines the following `MsgDoSomething` and that we've released its state +machine in go module `example.com/foo`: + +```protobuf +package foo.v1; + +message MsgDoSomething { + string sender = 1; + uint64 amount = 2; +} + +service Msg { + DoSomething(MsgDoSomething) returns (MsgDoSomethingResponse); +} +``` + +Now consider that we make a revision to this module and add a new `condition` field to `MsgDoSomething` and also +add a new validation rule on `amount` requiring it to be non-zero, and that following go semantic versioning we +release the next state machine version of `foo` as `example.com/foo/v2`. + +```protobuf expandable +// Revision 1 +package foo.v1; + +message MsgDoSomething { + string sender = 1; + + // amount must be a non-zero integer. + uint64 amount = 2; + + // condition is an optional condition on doing the thing. + // + // Since: Revision 1 + Condition condition = 3; +} +``` + +Approaching this naively, we would generate the protobuf types for the initial +version of `foo` in `example.com/foo/types` and we would generate the protobuf +types for the second version in `example.com/foo/v2/types`. + +Now let's say we have a module `bar` which talks to `foo` using this keeper +interface which `foo` provides: + +```go +type FooKeeper interface { + DoSomething(MsgDoSomething) + +error +} +``` + +#### Scenario A: Backward Compatibility: Newer Foo, Older Bar + +Imagine we have a chain which uses both `foo` and `bar` and wants to upgrade to +`foo/v2`, but the `bar` module has not upgraded to `foo/v2`. + +In this case, the chain will not be able to upgrade to `foo/v2` until `bar` +has upgraded its references to `example.com/foo/types.MsgDoSomething` to +`example.com/foo/v2/types.MsgDoSomething`. + +Even if `bar`'s usage of `MsgDoSomething` has not changed at all, the upgrade +will be impossible without this change because `example.com/foo/types.MsgDoSomething` +and `example.com/foo/v2/types.MsgDoSomething` are fundamentally different +incompatible structs in the go type system. + +#### Scenario B: Forward Compatibility: Older Foo, Newer Bar + +Now let's consider the reverse scenario, where `bar` upgrades to `foo/v2` +by changing the `MsgDoSomething` reference to `example.com/foo/v2/types.MsgDoSomething` +and releases that as `bar/v2` with some other changes that a chain wants. +The chain, however, has decided that it thinks the changes in `foo/v2` are too +risky and that it'd prefer to stay on the initial version of `foo`. + +In this scenario, it is impossible to upgrade to `bar/v2` without upgrading +to `foo/v2` even if `bar/v2` would have worked 100% fine with `foo` other +than changing the import path to `MsgDoSomething` (meaning that `bar/v2` +doesn't actually use any new features of `foo/v2`). + +Now because of the way go semantic import versioning works, we are locked +into either using `foo` and `bar` OR `foo/v2` and `bar/v2`. We cannot have +`foo` + `bar/v2` OR `foo/v2` + `bar`. The go type system doesn't allow this +even if both versions of these modules are otherwise compatible with each +other. + +#### Naive Mitigation + +A naive approach to fixing this would be to not regenerate the protobuf types +in `example.com/foo/v2/types` but instead just update `example.com/foo/types` +to reflect the changes needed for `v2` (adding `condition` and requiring +`amount` to be non-zero). Then we could release a patch of `example.com/foo/types` +with this update and use that for `foo/v2`. But this change is state machine +breaking for `v1`. It requires changing the `ValidateBasic` method to reject +the case where `amount` is zero, and it adds the `condition` field which +should be rejected based +on [ADR 020 unknown field filtering](/sdk/v0.50/build/architecture/adr-020-protobuf-transaction-encoding#unknown-field-filtering). +So adding these changes as a patch on `v1` is actually incorrect based on semantic +versioning. Chains that want to stay on `v1` of `foo` should not +be importing these changes because they are incorrect for `v1.` + +### Problem 2: Circular dependencies + +None of the above approaches allow `foo` and `bar` to be separate modules +if for some reason `foo` and `bar` depend on each other in different ways. +For instance, we can't have `foo` import `bar/types` while `bar` imports +`foo/types`. + +We have several cases of circular module dependencies in the SDK +(ex. staking, distribution and slashing) that are legitimate from a state machine +perspective. Without separating the API types out somehow, there would be +no way to independently semantically version these modules without some other +mitigation. + +### Problem 3: Handling Minor Version Incompatibilities + +Imagine that we solve the first two problems but now have a scenario where +`bar/v2` wants the option to use `MsgDoSomething.condition` which only `foo/v2` +supports. If `bar/v2` works with `foo` `v1` and sets `condition` to some non-nil +value, then `foo` will silently ignore this field resulting in a silent logic +possibly dangerous logic error. If `bar/v2` were able to check whether `foo` was +on `v1` or `v2` and dynamically, it could choose to only use `condition` when +`foo/v2` is available. Even if `bar/v2` were able to perform this check, however, +how do we know that it is always performing the check properly. Without +some sort of +framework-level [unknown field filtering](/sdk/v0.50/build/architecture/adr-020-protobuf-transaction-encoding#unknown-field-filtering), +it is hard to know whether these pernicious hard to detect bugs are getting into +our app and a client-server layer such as [ADR 033: Inter-Module Communication](/sdk/v0.50/build/architecture/adr-033-protobuf-inter-module-comm) +may be needed to do this. + +## Solutions + +### Approach A) Separate API and State Machine Modules + +One solution (first proposed in [Link](https://github.com/cosmos/cosmos-sdk/discussions/10582)) is to isolate all protobuf +generated code into a separate module +from the state machine module. This would mean that we could have state machine +go modules `foo` and `foo/v2` which could use a types or API go module say +`foo/api`. This `foo/api` go module would be perpetually on `v1.x` and only +accept non-breaking changes. This would then allow other modules to be +compatible with either `foo` or `foo/v2` as long as the inter-module API only +depends on the types in `foo/api`. It would also allow modules `foo` and `bar` +to depend on each other in that both of them could depend on `foo/api` and +`bar/api` without `foo` directly depending on `bar` and vice versa. + +This is similar to the naive mitigation described above except that it separates +the types into separate go modules which in and of itself could be used to +break circular module dependencies. It has the same problems as the naive solution, +otherwise, which we could rectify by: + +1. removing all state machine breaking code from the API module (ex. `ValidateBasic` and any other interface methods) +2. embedding the correct file descriptors for unknown field filtering in the binary + +#### Migrate all interface methods on API types to handlers + +To solve 1), we need to remove all interface implementations from generated +types and instead use a handler approach which essentially means that given +a type `X`, we have some sort of resolver which allows us to resolve interface +implementations for that type (ex. `sdk.Msg` or `authz.Authorization`). For +example: + +```go +func (k Keeper) + +DoSomething(msg MsgDoSomething) + +error { + var validateBasicHandler ValidateBasicHandler + err := k.resolver.Resolve(&validateBasic, msg) + if err != nil { + return err +} + +err = validateBasicHandler.ValidateBasic() + ... +} +``` + +In the case of some methods on `sdk.Msg`, we could replace them with declarative +annotations. For instance, `GetSigners` can already be replaced by the protobuf +annotation `cosmos.msg.v1.signer`. In the future, we may consider some sort +of protobuf validation framework (like [Link](https://github.com/bufbuild/protoc-gen-validate) +but more Cosmos-specific) to replace `ValidateBasic`. + +#### Pinned FileDescriptor's + +To solve 2), state machine modules must be able to specify what the version of +the protobuf files was that they were built against. For instance if the API +module for `foo` upgrades to `foo/v2`, the original `foo` module still needs +a copy of the original protobuf files it was built with so that ADR 020 +unknown field filtering will reject `MsgDoSomething` when `condition` is +set. + +The simplest way to do this may be to embed the protobuf `FileDescriptor`s into +the module itself so that these `FileDescriptor`s are used at runtime rather +than the ones that are built into the `foo/api` which may be different. Using +[buf build](https://docs.buf.build/build/usage#output-format), [go embed](https://pkg.go.dev/embed), +and a build script we can probably come up with a solution for embedding +`FileDescriptor`s into modules that is fairly straightforward. + +#### Potential limitations to generated code + +One challenge with this approach is that it places heavy restrictions on what +can go in API modules and requires that most of this is state machine breaking. +All or most of the code in the API module would be generated from protobuf +files, so we can probably control this with how code generation is done, but +it is a risk to be aware of. + +For instance, we do code generation for the ORM that in the future could +contain optimizations that are state machine breaking. We +would either need to ensure very carefully that the optimizations aren't +actually state machine breaking in generated code or separate this generated code +out from the API module into the state machine module. Both of these mitigations +are potentially viable but the API module approach does require an extra level +of care to avoid these sorts of issues. + +#### Minor Version Incompatibilities + +This approach in and of itself does little to address any potential minor +version incompatibilities and the +requisite [unknown field filtering](/sdk/v0.50/build/architecture/adr-020-protobuf-transaction-encoding#unknown-field-filtering). +Likely some sort of client-server routing layer which does this check such as +[ADR 033: Inter-Module communication](/sdk/v0.50/build/architecture/adr-033-protobuf-inter-module-comm) +is required to make sure that this is done properly. We could then allow +modules to perform a runtime check given a `MsgClient`, ex: + +```go +func (k Keeper) + +CallFoo() + +error { + if k.interModuleClient.MinorRevision(k.fooMsgClient) >= 2 { + k.fooMsgClient.DoSomething(&MsgDoSomething{ + Condition: ... +}) +} + +else { + ... +} +} +``` + +To do the unknown field filtering itself, the ADR 033 router would need to use +the [protoreflect API](https://pkg.go.dev/google.golang.org/protobuf/reflect/protoreflect) +to ensure that no fields unknown to the receiving module are set. This could +result in an undesirable performance hit depending on how complex this logic is. + +### Approach B) Changes to Generated Code + +An alternate approach to solving the versioning problem is to change how protobuf code is generated and move modules +mostly or completely in the direction of inter-module communication as described +in [ADR 033](/sdk/v0.50/build/architecture/adr-033-protobuf-inter-module-comm). +In this paradigm, a module could generate all the types it needs internally - including the API types of other modules - +and talk to other modules via a client-server boundary. For instance, if `bar` needs to talk to `foo`, it could +generate its own version of `MsgDoSomething` as `bar/internal/foo/v1.MsgDoSomething` and just pass this to the +inter-module router which would somehow convert it to the version which foo needs (ex. `foo/internal.MsgDoSomething`). + +Currently, two generated structs for the same protobuf type cannot exist in the same go binary without special +build flags (see [Link](https://developers.google.com/protocol-buffers/docs/reference/go/faq#fix-namespace-conflict)). +A relatively simple mitigation to this issue would be to set up the protobuf code to not register protobuf types +globally if they are generated in an `internal/` package. This will require modules to register their types manually +with the app-level level protobuf registry, this is similar to what modules already do with the `InterfaceRegistry` +and amino codec. + +If modules *only* do ADR 033 message passing then a naive and non-performant solution for +converting `bar/internal/foo/v1.MsgDoSomething` +to `foo/internal.MsgDoSomething` would be marshaling and unmarshaling in the ADR 033 router. This would break down if +we needed to expose protobuf types in `Keeper` interfaces because the whole point is to try to keep these types +`internal/` so that we don't end up with all the import version incompatibilities we've described above. However, +because of the issue with minor version incompatibilities and the need +for [unknown field filtering](/sdk/v0.50/build/architecture/adr-020-protobuf-transaction-encoding#unknown-field-filtering), +sticking with the `Keeper` paradigm instead of ADR 033 may be unviable to begin with. + +A more performant solution (that could maybe be adapted to work with `Keeper` interfaces) would be to only expose +getters and setters for generated types and internally store data in memory buffers which could be passed from +one implementation to another in a zero-copy way. + +For example, imagine this protobuf API with only getters and setters is exposed for `MsgSend`: + +```go expandable +type MsgSend interface { + proto.Message + GetFromAddress() + +string + GetToAddress() + +string + GetAmount() []v1beta1.Coin + SetFromAddress(string) + +SetToAddress(string) + +SetAmount([]v1beta1.Coin) +} + +func NewMsgSend() + +MsgSend { + return &msgSendImpl{ + memoryBuffers: ... +} +} +``` + +Under the hood, `MsgSend` could be implemented based on some raw memory buffer in the same way +that [Cap'n Proto](https://capnproto.org) +and [FlatBuffers](https://google.github.io/flatbuffers/) so that we could convert between one version of `MsgSend` +and another without serialization (i.e. zero-copy). This approach would have the added benefits of allowing zero-copy +message passing to modules written in other languages such as Rust and accessed through a VM or FFI. It could also make +unknown field filtering in inter-module communication simpler if we require that all new fields are added in sequential +order, ex. just checking that no field `> 5` is set. + +Also, we wouldn't have any issues with state machine breaking code on generated types because all the generated +code used in the state machine would actually live in the state machine module itself. Depending on how interface +types and protobuf `Any`s are used in other languages, however, it may still be desirable to take the handler +approach described in approach A. Either way, types implementing interfaces would still need to be registered +with an `InterfaceRegistry` as they are now because there would be no way to retrieve them via the global registry. + +In order to simplify access to other modules using ADR 033, a public API module (maybe even one +[remotely generated by Buf](https://docs.buf.build/bsr/remote-generation/go)) could be used by client modules instead +of requiring to generate all client types internally. + +The big downsides of this approach are that it requires big changes to how people use protobuf types and would be a +substantial rewrite of the protobuf code generator. This new generated code, however, could still be made compatible +with +the [`google.golang.org/protobuf/reflect/protoreflect`](https://pkg.go.dev/google.golang.org/protobuf/reflect/protoreflect) +API in order to work with all standard golang protobuf tooling. + +It is possible that the naive approach of marshaling/unmarshaling in the ADR 033 router is an acceptable intermediate +solution if the changes to the code generator are seen as too complex. However, since all modules would likely need +to migrate to ADR 033 anyway with this approach, it might be better to do this all at once. + +### Approach C) Don't address these issues + +If the above solutions are seen as too complex, we can also decide not to do anything explicit to enable better module +version compatibility, and break circular dependencies. + +In this case, when developers are confronted with the issues described above they can require dependencies to update in +sync (what we do now) or attempt some ad-hoc potentially hacky solution. + +One approach is to ditch go semantic import versioning (SIV) altogether. Some people have commented that go's SIV +(i.e. changing the import path to `foo/v2`, `foo/v3`, etc.) is too restrictive and that it should be optional. The +golang maintainers disagree and only officially support semantic import versioning. We could, however, take the +contrarian perspective and get more flexibility by using 0.x-based versioning basically forever. + +Module version compatibility could then be achieved using go.mod replace directives to pin dependencies to specific +compatible 0.x versions. For instance if we knew `foo` 0.2 and 0.3 were both compatible with `bar` 0.3 and 0.4, we +could use replace directives in our go.mod to stick to the versions of `foo` and `bar` we want. This would work as +long as the authors of `foo` and `bar` avoid incompatible breaking changes between these modules. + +Or, if developers choose to use semantic import versioning, they can attempt the naive solution described above +and would also need to use special tags and replace directives to make sure that modules are pinned to the correct +versions. + +Note, however, that all of these ad-hoc approaches, would be vulnerable to the minor version compatibility issues +described above unless [unknown field filtering](/sdk/v0.50/build/architecture/adr-020-protobuf-transaction-encoding#unknown-field-filtering) +is properly addressed. + +### Approach D) Avoid protobuf generated code in public APIs + +An alternative approach would be to avoid protobuf generated code in public module APIs. This would help avoid the +discrepancy between state machine versions and client API versions at the module to module boundaries. It would mean +that we wouldn't do inter-module message passing based on ADR 033, but rather stick to the existing keeper approach +and take it one step further by avoiding any protobuf generated code in the keeper interface methods. + +Using this approach, our `foo.Keeper.DoSomething` method wouldn't have the generated `MsgDoSomething` struct (which +comes from the protobuf API), but instead positional parameters. Then in order for `foo/v2` to support the `foo/v1` +keeper it would simply need to implement both the v1 and v2 keeper APIs. The `DoSomething` method in v2 could have the +additional `condition` parameter, but this wouldn't be present in v1 at all so there would be no danger of a client +accidentally setting this when it isn't available. + +So this approach would avoid the challenge around minor version incompatibilities because the existing module keeper +API would not get new fields when they are added to protobuf files. + +Taking this approach, however, would likely require making all protobuf generated code internal in order to prevent +it from leaking into the keeper API. This means we would still need to modify the protobuf code generator to not +register `internal/` code with the global registry, and we would still need to manually register protobuf +`FileDescriptor`s (this is probably true in all scenarios). It may, however, be possible to avoid needing to refactor +interface methods on generated types to handlers. + +Also, this approach doesn't address what would be done in scenarios where modules still want to use the message router. +Either way, we probably still want a way to pass messages from one module to another router safely even if it's just for +use cases like `x/gov`, `x/authz`, CosmWasm, etc. That would still require most of the things outlined in approach (B), +although we could advise modules to prefer keepers for communicating with other modules. + +The biggest downside of this approach is probably that it requires a strict refactoring of keeper interfaces to avoid +generated code leaking into the API. This may result in cases where we need to duplicate types that are already defined +in proto files and then write methods for converting between the golang and protobuf version. This may end up in a lot +of unnecessary boilerplate and that may discourage modules from actually adopting it and achieving effective version +compatibility. Approaches (A) and (B), although heavy handed initially, aim to provide a system which once adopted +more or less gives the developer version compatibility for free with minimal boilerplate. Approach (D) may not be able +to provide such a straightforward system since it requires a golang API to be defined alongside a protobuf API in a +way that requires duplication and differing sets of design principles (protobuf APIs encourage additive changes +while golang APIs would forbid it). + +Other downsides to this approach are: + +* no clear roadmap to supporting modules in other languages like Rust +* doesn't get us any closer to proper object capability security (one of the goals of ADR 033) +* ADR 033 needs to be done properly anyway for the set of use cases which do need it + +## Decision + +The latest **DRAFT** proposal is: + +1. we are alignment on adopting [ADR 033](/sdk/v0.50/build/architecture/adr-033-protobuf-inter-module-comm) not just as an addition to the + framework, but as a core replacement to the keeper paradigm entirely. +2. the ADR 033 inter-module router will accommodate any variation of approach (A) or (B) given the following rules: + a. if the client type is the same as the server type then pass it directly through, + b. if both client and server use the zero-copy generated code wrappers (which still need to be defined), then pass + the memory buffers from one wrapper to the other, or + c. marshal/unmarshal types between client and server. + +This approach will allow for both maximal correctness and enable a clear path to enabling modules within in other +languages, possibly executed within a WASM VM. + +### Minor API Revisions + +To declare minor API revisions of proto files, we propose the following guidelines (which were already documented +in [cosmos.app.v1alpha module options](https://github.com/cosmos/cosmos-sdk/blob/v0.50.10/proto/cosmos/app/v1alpha1/module.proto)): + +* proto packages which are revised from their initial version (considered revision `0`) should include a `package` +* comment in some .proto file containing the test `Revision N` at the start of a comment line where `N` is the current + revision number. +* all fields, messages, etc. added in a version beyond the initial revision should add a comment at the start of a + comment line of the form `Since: Revision N` where `N` is the non-zero revision it was added. + +It is advised that there is a 1:1 correspondence between a state machine module and versioned set of proto files +which are versioned either as a buf module a go API module or both. If the buf schema registry is used, the version of +this buf module should always be `1.N` where `N` corresponds to the package revision. Patch releases should be used when +only documentation comments are updated. It is okay to include proto packages named `v2`, `v3`, etc. in this same +`1.N` versioned buf module (ex. `cosmos.bank.v2`) as long as all these proto packages consist of a single API intended +to be served by a single SDK module. + +### Introspecting Minor API Revisions + +In order for modules to introspect the minor API revision of peer modules, we propose adding the following method +to `cosmossdk.io/core/intermodule.Client`: + +```go +ServiceRevision(ctx context.Context, serviceName string) + +uint64 +``` + +Modules could all this using the service name statically generated by the go grpc code generator: + +```go +intermoduleClient.ServiceRevision(ctx, bankv1beta1.Msg_ServiceDesc.ServiceName) +``` + +In the future, we may decide to extend the code generator used for protobuf services to add a field +to client types which does this check more concisely, ex: + +```go +package bankv1beta1 + +type MsgClient interface { + Send(context.Context, MsgSend) (MsgSendResponse, error) + +ServiceRevision(context.Context) + +uint64 +} +``` + +### Unknown Field Filtering + +To correctly perform [unknown field filtering](/sdk/v0.50/build/architecture/adr-020-protobuf-transaction-encoding#unknown-field-filtering), +the inter-module router can do one of the following: + +* use the `protoreflect` API for messages which support that +* for gogo proto messages, marshal and use the existing `codec/unknownproto` code +* for zero-copy messages, do a simple check on the highest set field number (assuming we can require that fields are + adding consecutively in increasing order) + +### `FileDescriptor` Registration + +Because a single go binary may contain different versions of the same generated protobuf code, we cannot rely on the +global protobuf registry to contain the correct `FileDescriptor`s. Because `appconfig` module configuration is itself +written in protobuf, we would like to load the `FileDescriptor`s for a module before loading a module itself. So we +will provide ways to register `FileDescriptor`s at module registration time before instantiation. We propose the +following `cosmossdk.io/core/appmodule.Option` constructors for the various cases of how `FileDescriptor`s may be +packaged: + +```go expandable +package appmodule + +// this can be used when we are using google.golang.org/protobuf compatible generated code +// Ex: +// ProtoFiles(bankv1beta1.File_cosmos_bank_v1beta1_module_proto) + +func ProtoFiles(file []protoreflect.FileDescriptor) + +Option { +} + +// this can be used when we are using gogo proto generated code. +func GzippedProtoFiles(file [][]byte) + +Option { +} + +// this can be used when we are using buf build to generated a pinned file descriptor +func ProtoImage(protoImage []byte) + +Option { +} +``` + +This approach allows us to support several ways protobuf files might be generated: + +* proto files generated internally to a module (use `ProtoFiles`) +* the API module approach with pinned file descriptors (use `ProtoImage`) +* gogo proto (use `GzippedProtoFiles`) + +### Module Dependency Declaration + +One risk of ADR 033 is that dependencies are called at runtime which are not present in the loaded set of SDK modules.\ +Also we want modules to have a way to define a minimum dependency API revision that they require. Therefore, all +modules should declare their set of dependencies upfront. These dependencies could be defined when a module is +instantiated, but ideally we know what the dependencies are before instantiation and can statically look at an app +config and determine whether the set of modules. For example, if `bar` requires `foo` revision `>= 1`, then we +should be able to know this when creating an app config with two versions of `bar` and `foo`. + +We propose defining these dependencies in the proto options of the module config object itself. + +### Interface Registration + +We will also need to define how interface methods are defined on types that are serialized as `google.protobuf.Any`'s. +In light of the desire to support modules in other languages, we may want to think of solutions that will accommodate +other languages such as plugins described briefly in [ADR 033](/sdk/v0.50/build/architecture/adr-033-protobuf-inter-module-comm#internal-methods). + +### Testing + +In order to ensure that modules are indeed with multiple versions of their dependencies, we plan to provide specialized +unit and integration testing infrastructure that automatically tests multiple versions of dependencies. + +#### Unit Testing + +Unit tests should be conducted inside SDK modules by mocking their dependencies. In a full ADR 033 scenario, +this means that all interaction with other modules is done via the inter-module router, so mocking of dependencies +means mocking their msg and query server implementations. We will provide both a test runner and fixture to make this +streamlined. The key thing that the test runner should do to test compatibility is to test all combinations of +dependency API revisions. This can be done by taking the file descriptors for the dependencies, parsing their comments +to determine the revisions various elements were added, and then created synthetic file descriptors for each revision +by subtracting elements that were added later. + +Here is a proposed API for the unit test runner and fixture: + +```go expandable +package moduletesting + +import ( + + "context" + "testing" + "cosmossdk.io/core/intermodule" + "cosmossdk.io/depinject" + "google.golang.org/grpc" + "google.golang.org/protobuf/proto" + "google.golang.org/protobuf/reflect/protodesc" +) + +type TestFixture interface { + context.Context + intermodule.Client // for making calls to the module we're testing + BeginBlock() + +EndBlock() +} + +type UnitTestFixture interface { + TestFixture + grpc.ServiceRegistrar // for registering mock service implementations +} + +type UnitTestConfig struct { + ModuleConfig proto.Message // the module's config object + DepinjectConfig depinject.Config // optional additional depinject config options + DependencyFileDescriptors []protodesc.FileDescriptorProto // optional dependency file descriptors to use instead of the global registry +} + +// Run runs the test function for all combinations of dependency API revisions. +func (cfg UnitTestConfig) + +Run(t *testing.T, f func(t *testing.T, f UnitTestFixture)) { + // ... +} +``` + +Here is an example for testing bar calling foo which takes advantage of conditional service revisions in the expected +mock arguments: + +```go expandable +func TestBar(t *testing.T) { + UnitTestConfig{ + ModuleConfig: &foomodulev1.Module{ +}}.Run(t, func (t *testing.T, f moduletesting.UnitTestFixture) { + ctrl := gomock.NewController(t) + mockFooMsgServer := footestutil.NewMockMsgServer() + +foov1.RegisterMsgServer(f, mockFooMsgServer) + barMsgClient := barv1.NewMsgClient(f) + if f.ServiceRevision(foov1.Msg_ServiceDesc.ServiceName) >= 1 { + mockFooMsgServer.EXPECT().DoSomething(gomock.Any(), &foov1.MsgDoSomething{ + ..., + Condition: ..., // condition is expected in revision >= 1 +}).Return(&foov1.MsgDoSomethingResponse{ +}, nil) +} + +else { + mockFooMsgServer.EXPECT().DoSomething(gomock.Any(), &foov1.MsgDoSomething{... +}).Return(&foov1.MsgDoSomethingResponse{ +}, nil) +} + +res, err := barMsgClient.CallFoo(f, &MsgCallFoo{ +}) + ... +}) +} +``` + +The unit test runner would make sure that no dependency mocks return arguments which are invalid for the service +revision being tested to ensure that modules don't incorrectly depend on functionality not present in a given revision. + +#### Integration Testing + +An integration test runner and fixture would also be provided which instead of using mocks would test actual module +dependencies in various combinations. Here is the proposed API: + +```go expandable +type IntegrationTestFixture interface { + TestFixture +} + +type IntegrationTestConfig struct { + ModuleConfig proto.Message // the module's config object + DependencyMatrix map[string][]proto.Message // all the dependent module configs +} + +// Run runs the test function for all combinations of dependency modules. +func (cfg IntegationTestConfig) + +Run(t *testing.T, f func (t *testing.T, f IntegrationTestFixture)) { + // ... +} +``` + +And here is an example with foo and bar: + +```go expandable +func TestBarIntegration(t *testing.T) { + IntegrationTestConfig{ + ModuleConfig: &barmodulev1.Module{ +}, + DependencyMatrix: map[string][]proto.Message{ + "runtime": []proto.Message{ // test against two versions of runtime + &runtimev1.Module{ +}, + &runtimev2.Module{ +}, +}, + "foo": []proto.Message{ // test against three versions of foo + &foomodulev1.Module{ +}, + &foomodulev2.Module{ +}, + &foomodulev3.Module{ +}, +} + +} +}.Run(t, func (t *testing.T, f moduletesting.IntegrationTestFixture) { + barMsgClient := barv1.NewMsgClient(f) + +res, err := barMsgClient.CallFoo(f, &MsgCallFoo{ +}) + ... +}) +} +``` + +Unlike unit tests, integration tests actually pull in other module dependencies. So that modules can be written +without direct dependencies on other modules and because golang has no concept of development dependencies, integration +tests should be written in separate go modules, ex. `example.com/bar/v2/test`. Because this paradigm uses go semantic +versioning, it is possible to build a single go module which imports 3 versions of bar and 2 versions of runtime and +can test these all together in the six various combinations of dependencies. + +## Consequences + +### Backwards Compatibility + +Modules which migrate fully to ADR 033 will not be compatible with existing modules which use the keeper paradigm. +As a temporary workaround we may create some wrapper types that emulate the current keeper interface to minimize +the migration overhead. + +### Positive + +* we will be able to deliver interoperable semantically versioned modules which should dramatically increase the + ability of the Cosmos SDK ecosystem to iterate on new features +* it will be possible to write Cosmos SDK modules in other languages in the near future + +### Negative + +* all modules will need to be refactored somewhat dramatically + +### Neutral + +* the `cosmossdk.io/core/appconfig` framework will play a more central role in terms of how modules are defined, this + is likely generally a good thing but does mean additional changes for users wanting to stick to the pre-depinject way + of wiring up modules +* `depinject` is somewhat less needed or maybe even obviated because of the full ADR 033 approach. If we adopt the + core API proposed in [Link](https://github.com/cosmos/cosmos-sdk/pull/12239), then a module would probably always instantiate + itself with a method `ProvideModule(appmodule.Service) (appmodule.AppModule, error)`. There is no complex wiring of + keeper dependencies in this scenario and dependency injection may not have as much of (or any) use case. + +## Further Discussions + +The decision described above is considered in draft mode and is pending final buy-in from the team and key stakeholders. +Key outstanding discussions if we do adopt that direction are: + +* how do module clients introspect dependency module API revisions +* how do modules determine a minor dependency module API revision requirement +* how do modules appropriately test compatibility with different dependency versions +* how to register and resolve interface implementations +* how do modules register their protobuf file descriptors depending on the approach they take to generated code (the + API module approach may still be viable as a supported strategy and would need pinned file descriptors) + +## References + +* [Link](https://github.com/cosmos/cosmos-sdk/discussions/10162) +* [Link](https://github.com/cosmos/cosmos-sdk/discussions/10582) +* [Link](https://github.com/cosmos/cosmos-sdk/discussions/10368) +* [Link](https://github.com/cosmos/cosmos-sdk/pull/11340) +* [Link](https://github.com/cosmos/cosmos-sdk/issues/11899) +* [ADR 020](/sdk/v0.50/build/architecture/adr-020-protobuf-transaction-encoding) +* [ADR 033](/sdk/v0.50/build/architecture/adr-033-protobuf-inter-module-comm) diff --git a/sdk/v0.54/reference/architecture/adr-055-orm.mdx b/sdk/v0.54/reference/architecture/adr-055-orm.mdx new file mode 100644 index 000000000..6c330a8db --- /dev/null +++ b/sdk/v0.54/reference/architecture/adr-055-orm.mdx @@ -0,0 +1,118 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/architecture/adr-055-orm' +title: 'ADR 055: ORM' +description: '2022-04-27: First draft' +--- + +## Changelog + +* 2022-04-27: First draft + +## Status + +ACCEPTED Implemented + +## Abstract + +In order to make it easier for developers to build Cosmos SDK modules and for clients to query, index and verify proofs +against state data, we have implemented an ORM (object-relational mapping) layer for the Cosmos SDK. + +## Context + +Historically modules in the Cosmos SDK have always used the key-value store directly and created various handwritten +functions for managing key format as well as constructing secondary indexes. This consumes a significant amount of +time when building a module and is error-prone. Because key formats are non-standard, sometimes poorly documented, +and subject to change, it is hard for clients to generically index, query and verify merkle proofs against state data. + +The known first instance of an "ORM" in the Cosmos ecosystem was in [weave](https://github.com/iov-one/weave/tree/master/orm). +A later version was built for [regen-ledger](https://github.com/regen-network/regen-ledger/tree/157181f955823149e1825263a317ad8e16096da4/orm) for +use in the group module and later [ported to the SDK](https://github.com/cosmos/cosmos-sdk/tree/35d3312c3be306591fcba39892223f1244c8d108/x/group/internal/orm) +just for that purpose. + +While these earlier designs made it significantly easier to write state machines, they still required a lot of manual +configuration, didn't expose state format directly to clients, and were limited in their support of different types +of index keys, composite keys, and range queries. + +Discussions about the design continued in [Link](https://github.com/cosmos/cosmos-sdk/discussions/9156) and more +sophisticated proofs of concept were created in [Link](https://github.com/allinbits/cosmos-sdk-poc/tree/master/runtime/orm) +and [Link](https://github.com/cosmos/cosmos-sdk/pull/10454). + +## Decision + +These prior efforts culminated in the creation of the Cosmos SDK `orm` go module which uses protobuf annotations +for specifying ORM table definitions. This ORM is based on the new `google.golang.org/protobuf/reflect/protoreflect` +API and supports: + +* sorted indexes for all simple protobuf types (except `bytes`, `enum`, `float`, `double`) as well as `Timestamp` and `Duration` +* unsorted `bytes` and `enum` indexes +* composite primary and secondary keys +* unique indexes +* auto-incrementing `uint64` primary keys +* complex prefix and range queries +* paginated queries +* complete logical decoding of KV-store data + +Almost all the information needed to decode state directly is specified in .proto files. Each table definition specifies +an ID which is unique per .proto file and each index within a table is unique within that table. Clients then only need +to know the name of a module and the prefix ORM data for a specific .proto file within that module in order to decode +state data directly. This additional information will be exposed directly through app configs which will be explained +in a future ADR related to app wiring. + +The ORM makes optimizations around storage space by not repeating values in the primary key in the key value +when storing primary key records. For example, if the object `{"a":0,"b":1}` has the primary key `a`, it will +be stored in the key value store as `Key: '0', Value: {"b":1}` (with more efficient protobuf binary encoding). +Also, the generated code from [Link](https://github.com/cosmos/cosmos-proto) does optimizations around the +`google.golang.org/protobuf/reflect/protoreflect` API to improve performance. + +A code generator is included with the ORM which creates type safe wrappers around the ORM's dynamic `Table` +implementation and is the recommended way for modules to use the ORM. + +The ORM tests provide a simplified bank module demonstration which illustrates: + +* [ORM proto options](https://github.com/cosmos/cosmos-sdk/blob/0d846ae2f0424b2eb640f6679a703b52d407813d/orm/internal/testpb/bank.proto) +* [Generated Code](https://github.com/cosmos/cosmos-sdk/blob/0d846ae2f0424b2eb640f6679a703b52d407813d/orm/internal/testpb/bank.cosmos_orm.go) +* [Example Usage in a Module Keeper](https://github.com/cosmos/cosmos-sdk/blob/0d846ae2f0424b2eb640f6679a703b52d407813d/orm/model/ormdb/module_test.go) + +## Consequences + +### Backwards Compatibility + +State machine code that adopts the ORM will need migrations as the state layout is generally backwards incompatible. +These state machines will also need to migrate to [Link](https://github.com/cosmos/cosmos-proto) at least for state data. + +### Positive + +* easier to build modules +* easier to add secondary indexes to state +* possible to write a generic indexer for ORM state +* easier to write clients that do state proofs +* possible to automatically write query layers rather than needing to manually implement gRPC queries + +### Negative + +* worse performance than handwritten keys (for now). See [Further Discussions](#further-discussions) + for potential improvements + +### Neutral + +## Further Discussions + +Further discussions will happen within the Cosmos SDK Framework Working Group. Current planned and ongoing work includes: + +* automatically generate client-facing query layer +* client-side query libraries that transparently verify light client proofs +* index ORM data to SQL databases +* improve performance by: + * optimizing existing reflection based code to avoid unnecessary gets when doing deletes & updates of simple tables + * more sophisticated code generation such as making fast path reflection even faster (avoiding `switch` statements), + or even fully generating code that equals handwritten performance + +## References + +* [Link](https://github.com/iov-one/weave/tree/master/orm)). +* [Link](https://github.com/regen-network/regen-ledger/tree/157181f955823149e1825263a317ad8e16096da4/orm) +* [Link](https://github.com/cosmos/cosmos-sdk/tree/35d3312c3be306591fcba39892223f1244c8d108/x/group/internal/orm) +* [Link](https://github.com/cosmos/cosmos-sdk/discussions/9156) +* [Link](https://github.com/allinbits/cosmos-sdk-poc/tree/master/runtime/orm) +* [Link](https://github.com/cosmos/cosmos-sdk/pull/10454) diff --git a/sdk/v0.54/reference/architecture/adr-057-app-wiring.mdx b/sdk/v0.54/reference/architecture/adr-057-app-wiring.mdx new file mode 100644 index 000000000..ec59d8fb5 --- /dev/null +++ b/sdk/v0.54/reference/architecture/adr-057-app-wiring.mdx @@ -0,0 +1,391 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/architecture/adr-057-app-wiring' +title: 'ADR 057: App Wiring' +description: '2022-05-04: Initial Draft 2022-08-19: Updates' +--- + +## Changelog + +* 2022-05-04: Initial Draft +* 2022-08-19: Updates + +## Status + +PROPOSED Implemented + +## Abstract + +In order to make it easier to build Cosmos SDK modules and apps, we propose a new app wiring system based on +dependency injection and declarative app configurations to replace the current `app.go` code. + +## Context + +A number of factors have made the SDK and SDK apps in their current state hard to maintain. A symptom of the current +state of complexity is [`simapp/app.go`](https://github.com/cosmos/cosmos-sdk/blob/c3edbb22cab8678c35e21fe0253919996b780c01/simapp/app.go) +which contains almost 100 lines of imports and is otherwise over 600 lines of mostly boilerplate code that is +generally copied to each new project. (Not to mention the additional boilerplate which gets copied in `simapp/simd`.) + +The large amount of boilerplate needed to bootstrap an app has made it hard to release independently versioned go +modules for Cosmos SDK modules as described in [ADR 053: Go Module Refactoring](/sdk/v0.54/reference/architecture/adr-053-go-module-refactoring). + +In addition to being very verbose and repetitive, `app.go` also exposes a large surface area for breaking changes +as most modules instantiate themselves with positional parameters which forces breaking changes anytime a new parameter +(even an optional one) is needed. + +Several attempts were made to improve the current situation including [ADR 033: Internal-Module Communication](/sdk/v0.54/reference/architecture/adr-033-protobuf-inter-module-comm) +and [a proof-of-concept of a new SDK](https://github.com/allinbits/cosmos-sdk-poc). The discussions around these +designs led to the current solution described here. + +## Decision + +In order to improve the current situation, a new "app wiring" paradigm has been designed to replace `app.go` which +involves: + +* declaration configuration of the modules in an app which can be serialized to JSON or YAML +* a dependency-injection (DI) framework for instantiating apps from the that configuration + +### Dependency Injection + +When examining the code in `app.go` most of the code simply instantiates modules with dependencies provided either +by the framework (such as store keys) or by other modules (such as keepers). It is generally pretty obvious given +the context what the correct dependencies actually should be, so dependency-injection is an obvious solution. Rather +than making developers manually resolve dependencies, a module will tell the DI container what dependency it needs +and the container will figure out how to provide it. + +We explored several existing DI solutions in golang and felt that the reflection-based approach in [uber/dig](https://github.com/uber-go/dig) +was closest to what we needed but not quite there. Assessing what we needed for the SDK, we designed and built +the Cosmos SDK [depinject module](https://pkg.go.dev/github.com/cosmos/cosmos-sdk/depinject), which has the following +features: + +* dependency resolution and provision through functional constructors, ex: `func(need SomeDep) (AnotherDep, error)` +* dependency injection `In` and `Out` structs which support `optional` dependencies +* grouped-dependencies (many-per-container) through the `ManyPerContainerType` tag interface +* module-scoped dependencies via `ModuleKey`s (where each module gets a unique dependency) +* one-per-module dependencies through the `OnePerModuleType` tag interface +* sophisticated debugging information and container visualization via GraphViz + +Here are some examples of how these would be used in an SDK module: + +* `StoreKey` could be a module-scoped dependency which is unique per module +* a module's `AppModule` instance (or the equivalent) could be a `OnePerModuleType` +* CLI commands could be provided with `ManyPerContainerType`s + +Note that even though dependency resolution is dynamic and based on reflection, which could be considered a pitfall +of this approach, the entire dependency graph should be resolved immediately on app startup and only gets resolved +once (except in the case of dynamic config reloading which is a separate topic). This means that if there are any +errors in the dependency graph, they will get reported immediately on startup so this approach is only slightly worse +than fully static resolution in terms of error reporting and much better in terms of code complexity. + +### Declarative App Config + +In order to compose modules into an app, a declarative app configuration will be used. This configuration is based off +of protobuf and its basic structure is very simple: + +```protobuf +package cosmos.app.v1; + +message Config { + repeated ModuleConfig modules = 1; +} + +message ModuleConfig { + string name = 1; + google.protobuf.Any config = 2; +} +``` + +(See also [Link](https://github.com/cosmos/cosmos-sdk/blob/6e18f582bf69e3926a1e22a6de3c35ea327aadce/proto/cosmos/app/v1alpha1/config.proto)) + +The configuration for every module is itself a protobuf message and modules will be identified and loaded based +on the protobuf type URL of their config object (ex. `cosmos.bank.module.v1.Module`). Modules are given a unique short `name` +to share resources across different versions of the same module which might have a different protobuf package +versions (ex. `cosmos.bank.module.v2.Module`). All module config objects should define the `cosmos.app.v1alpha1.module` +descriptor option which will provide additional useful metadata for the framework and which can also be indexed +in module registries. + +An example app config in YAML might look like this: + +```yaml expandable +modules: + - name: baseapp + config: + "@type": cosmos.baseapp.module.v1.Module + begin_blockers: [staking, auth, bank] + end_blockers: [bank, auth, staking] + init_genesis: [bank, auth, staking] + - name: auth + config: + "@type": cosmos.auth.module.v1.Module + bech32_prefix: "foo" + - name: bank + config: + "@type": cosmos.bank.module.v1.Module + - name: staking + config: + "@type": cosmos.staking.module.v1.Module +``` + +In the above example, there is a hypothetical `baseapp` module which contains the information around ordering of +begin blockers, end blockers, and init genesis. Rather than lifting these concerns up to the module config layer, +they are themselves handled by a module which could allow a convenient way of swapping out different versions of +baseapp (for instance to target different versions of tendermint), without needing to change the rest of the config. +The `baseapp` module would then provide to the server framework (which sort of sits outside the ABCI app) an instance +of `abci.Application`. + +In this model, an app is *modules all the way down* and the dependency injection/app config layer is very much +protocol-agnostic and can adapt to even major breaking changes at the protocol layer. + +### Module & Protobuf Registration + +In order for the two components of dependency injection and declarative configuration to work together as described, +we need a way for modules to actually register themselves and provide dependencies to the container. + +One additional complexity that needs to be handled at this layer is protobuf registry initialization. Recall that +in both the current SDK `codec` and the proposed [ADR 054: Protobuf Semver Compatible Codegen](https://github.com/cosmos/cosmos-sdk/pull/11802), +protobuf types need to be explicitly registered. Given that the app config itself is based on protobuf and +uses protobuf `Any` types, protobuf registration needs to happen before the app config itself can be decoded. Because +we don't know which protobuf `Any` types will be needed a priori and modules themselves define those types, we need +to decode the app config in separate phases: + +1. parse app config JSON/YAML as raw JSON and collect required module type URLs (without doing proto JSON decoding) +2. build a [protobuf type registry](https://pkg.go.dev/google.golang.org/protobuf@v1.28.0/reflect/protoregistry) based + on file descriptors and types provided by each required module +3. decode the app config as proto JSON using the protobuf type registry + +Because in [ADR 054: Protobuf Semver Compatible Codegen](https://github.com/cosmos/cosmos-sdk/pull/11802), each module +might use `internal` generated code which is not registered with the global protobuf registry, this code should provide +an alternate way to register protobuf types with a type registry. In the same way that `.pb.go` files currently have a +`var File_foo_proto protoreflect.FileDescriptor` for the file `foo.proto`, generated code should have a new member +`var Types_foo_proto TypeInfo` where `TypeInfo` is an interface or struct with all the necessary info to register both +the protobuf generated types and file descriptor. + +So a module must provide dependency injection providers and protobuf types, and takes as input its module +config object which uniquely identifies the module based on its type URL. + +With this in mind, we define a global module register which allows module implementations to register themselves +with the following API: + +```go expandable +// Register registers a module with the provided type name (ex. cosmos.bank.module.v1.Module) +// and the provided options. +func Register(configTypeName protoreflect.FullName, option ...Option) { ... +} + +type Option { /* private methods */ +} + +// Provide registers dependency injection provider functions which work with the +// cosmos-sdk container module. These functions can also accept an additional +// parameter for the module's config object. +func Provide(providers ...interface{ +}) + +Option { ... +} + +// Types registers protobuf TypeInfo's with the protobuf registry. +func Types(types ...TypeInfo) + +Option { ... +} +``` + +Ex: + +```go expandable +func init() { + appmodule.Register("cosmos.bank.module.v1.Module", + appmodule.Types( + types.Types_tx_proto, + types.Types_query_proto, + types.Types_types_proto, + ), + appmodule.Provide( + provideBankModule, + ) + ) +} + +type Inputs struct { + container.In + + AuthKeeper auth.Keeper + DB ormdb.ModuleDB +} + +type Outputs struct { + Keeper bank.Keeper + AppModule appmodule.AppModule +} + +func ProvideBankModule(config *bankmodulev1.Module, Inputs) (Outputs, error) { ... +} +``` + +Note that in this module, a module configuration object *cannot* register different dependency providers at runtime +based on the configuration. This is intentional because it allows us to know globally which modules provide which +dependencies, and it will also allow us to do code generation of the whole app initialization. This +can help us figure out issues with missing dependencies in an app config if the needed modules are loaded at runtime. +In cases where required modules are not loaded at runtime, it may be possible to guide users to the correct module if +through a global Cosmos SDK module registry. + +The `*appmodule.Handler` type referenced above is a replacement for the legacy `AppModule` framework, and +described in [ADR 063: Core Module API](/sdk/v0.54/reference/architecture/adr-063-core-module-api). + +### New `app.go` + +With this setup, `app.go` might now look something like this: + +```go expandable +package main + +import ( + + // Each go package which registers a module must be imported just for side-effects + // so that module implementations are registered. + _ "github.com/cosmos/cosmos-sdk/x/auth/module" + _ "github.com/cosmos/cosmos-sdk/x/bank/module" + _ "github.com/cosmos/cosmos-sdk/x/staking/module" + "github.com/cosmos/cosmos-sdk/core/app" +) + +// go:embed app.yaml +var appConfigYAML []byte + +func main() { + app.Run(app.LoadYAML(appConfigYAML)) +} +``` + +### Application to existing SDK modules + +So far we have described a system which is largely agnostic to the specifics of the SDK such as store keys, `AppModule`, +`BaseApp`, etc. Improvements to these parts of the framework that integrate with the general app wiring framework +defined here are described in [ADR 063: Core Module API](/sdk/v0.54/reference/architecture/adr-063-core-module-api). + +### Registration of Inter-Module Hooks + +### Registration of Inter-Module Hooks + +Some modules define a hooks interface (ex. `StakingHooks`) which allows one module to call back into another module +when certain events happen. + +With the app wiring framework, these hooks interfaces can be defined as a `OnePerModuleType`s and then the module +which consumes these hooks can collect these hooks as a map of module name to hook type (ex. `map[string]FooHooks`). Ex: + +```go expandable +func init() { + appmodule.Register( + &foomodulev1.Module{ +}, + appmodule.Invoke(InvokeSetFooHooks), + ... + ) +} + +func InvokeSetFooHooks( + keeper *keeper.Keeper, + fooHooks map[string]FooHooks, +) + +error { + for k in sort.Strings(maps.Keys(fooHooks)) { + keeper.AddFooHooks(fooHooks[k]) +} +} +``` + +Optionally, the module consuming hooks can allow app's to define an order for calling these hooks based on module name +in its config object. + +An alternative way for registering hooks via reflection was considered where all keeper types are inspected to see if +they implement the hook interface by the modules exposing hooks. This has the downsides of: + +* needing to expose all the keepers of all modules to the module providing hooks, +* not allowing for encapsulating hooks on a different type which doesn't expose all keeper methods, +* harder to know statically which module expose hooks or are checking for them. + +With the approach proposed here, hooks registration will be obviously observable in `app.go` if `depinject` codegen +(described below) is used. + +### Code Generation + +The `depinject` framework will optionally allow the app configuration and dependency injection wiring to be code +generated. This will allow: + +* dependency injection wiring to be inspected as regular go code just like the existing `app.go`, +* dependency injection to be opt-in with manual wiring 100% still possible. + +Code generation requires that all providers and invokers and their parameters are exported and in non-internal packages. + +### Module Semantic Versioning + +When we start creating semantically versioned SDK modules that are in standalone go modules, a state machine breaking +change to a module should be handled as follows: + +* the semantic major version should be incremented, and +* a new semantically versioned module config protobuf type should be created. + +For instance, if we have the SDK module for bank in the go module `github.com/cosmos/cosmos-sdk/x/bank` with the module config type +`cosmos.bank.module.v1.Module`, and we want to make a state machine breaking change to the module, we would: + +* create a new go module `github.com/cosmos/cosmos-sdk/x/bank/v2`, +* with the module config protobuf type `cosmos.bank.module.v2.Module`. + +This *does not* mean that we need to increment the protobuf API version for bank. Both modules can support +`cosmos.bank.v1`, but `github.com/cosmos/cosmos-sdk/x/bank/v2` will be a separate go module with a separate module config type. + +This practice will eventually allow us to use appconfig to load new versions of a module via a configuration change. + +Effectively, there should be a 1:1 correspondence between a semantically versioned go module and a +versioned module config protobuf type, and major versioning bumps should occur whenever state machine breaking changes +are made to a module. + +NOTE: SDK modules that are standalone go modules *should not* adopt semantic versioning until the concerns described in +[ADR 054: Module Semantic Versioning](/sdk/v0.54/reference/architecture/adr-054-semver-compatible-modules) are +addressed. The short-term solution for this issue was left somewhat unresolved. However, the easiest tactic is +likely to use a standalone API go module and follow the guidelines described in this comment: [Link](https://github.com/cosmos/cosmos-sdk/pull/11802#issuecomment-1406815181). For the time-being, it is recommended that +Cosmos SDK modules continue to follow tried and true [0-based versioning](https://0ver.org) until an officially +recommended solution is provided. This section of the ADR will be updated when that happens and for now, this section +should be considered as a design recommendation for future adoption of semantic versioning. + +## Consequences + +### Backwards Compatibility + +Modules which work with the new app wiring system do not need to drop their existing `AppModule` and `NewKeeper` +registration paradigms. These two methods can live side-by-side for as long as is needed. + +### Positive + +* wiring up new apps will be simpler, more succinct and less error-prone +* it will be easier to develop and test standalone SDK modules without needing to replicate all of simapp +* it may be possible to dynamically load modules and upgrade chains without needing to do a coordinated stop and binary + upgrade using this mechanism +* easier plugin integration +* dependency injection framework provides more automated reasoning about dependencies in the project, with a graph visualization. + +### Negative + +* it may be confusing when a dependency is missing although error messages, the GraphViz visualization, and global + module registration may help with that + +### Neutral + +* it will require work and education + +## Further Discussions + +The protobuf type registration system described in this ADR has not been implemented and may need to be reconsidered in +light of code generation. It may be better to do this type registration with a DI provider. + +## References + +* [Link](https://github.com/cosmos/cosmos-sdk/blob/c3edbb22cab8678c35e21fe0253919996b780c01/simapp/app.go) +* [Link](https://github.com/allinbits/cosmos-sdk-poc) +* [Link](https://github.com/uber-go/dig) +* [Link](https://github.com/google/wire) +* [Link](https://pkg.go.dev/github.com/cosmos/cosmos-sdk/container) +* [Link](https://github.com/cosmos/cosmos-sdk/pull/11802) +* [ADR 063: Core Module API](/sdk/v0.54/reference/architecture/adr-063-core-module-api) diff --git a/sdk/v0.54/reference/architecture/adr-058-auto-generated-cli.mdx b/sdk/v0.54/reference/architecture/adr-058-auto-generated-cli.mdx new file mode 100644 index 000000000..5fa3e9ca2 --- /dev/null +++ b/sdk/v0.54/reference/architecture/adr-058-auto-generated-cli.mdx @@ -0,0 +1,103 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/architecture/adr-058-auto-generated-cli' +title: 'ADR 058: Auto-Generated CLI' +description: '2022-05-04: Initial Draft' +--- + +## Changelog + +* 2022-05-04: Initial Draft + +## Status + +ACCEPTED Partially Implemented + +## Abstract + +In order to make it easier for developers to write Cosmos SDK modules, we provide infrastructure which automatically +generates CLI commands based on protobuf definitions. + +## Context + +Current Cosmos SDK modules generally implement a CLI command for every transaction and every query supported by the +module. These are handwritten for each command and essentially amount to providing some CLI flags or positional +arguments for specific fields in protobuf messages. + +In order to make sure CLI commands are correctly implemented as well as to make sure that the application works +in end-to-end scenarios, we do integration tests using CLI commands. While these tests are valuable on some-level, +they can be hard to write and maintain, and run slowly. [Some teams have contemplated](https://github.com/regen-network/regen-ledger/issues/1041) +moving away from CLI-style integration tests (which are really end-to-end tests) towards narrower integration tests +which exercise `MsgClient` and `QueryClient` directly. This might involve replacing the current end-to-end CLI +tests with unit tests as there still needs to be some way to test these CLI commands for full quality assurance. + +## Decision + +To make module development simpler, we provide infrastructure - in the new [`client/v2`](https://github.com/cosmos/cosmos-sdk/tree/main/client/v2) +go module - for automatically generating CLI commands based on protobuf definitions to either replace or complement +handwritten CLI commands. This will mean that when developing a module, it will be possible to skip both writing and +testing CLI commands as that can all be taken care of by the framework. + +The basic design for automatically generating CLI commands is to: + +* create one CLI command for each `rpc` method in a protobuf `Query` or `Msg` service +* create a CLI flag for each field in the `rpc` request type +* for `query` commands call gRPC and print the response as protobuf JSON or YAML (via the `-o`/`--output` flag) +* for `tx` commands, create a transaction and apply common transaction flags + +In order to make the auto-generated CLI as easy to use (or easier) than handwritten CLI, we need to do custom handling +of specific protobuf field types so that the input format is easy for humans: + +* `Coin`, `Coins`, `DecCoin`, and `DecCoins` should be input using the existing format (i.e. `1000uatom`) +* it should be possible to specify an address using either the bech32 address string or a named key in the keyring +* `Timestamp` and `Duration` should accept strings like `2001-01-01T00:00:00Z` and `1h3m` respectively +* pagination should be handled with flags like `--page-limit`, `--page-offset`, etc. +* it should be possible to customize any other protobuf type either via its message name or a `cosmos_proto.scalar` annotation + +At a basic level it should be possible to generate a command for a single `rpc` method as well as all the commands for +a whole protobuf `service` definition. It should be possible to mix and match auto-generated and handwritten commands. + +## Consequences + +### Backwards Compatibility + +Existing modules can mix and match auto-generated and handwritten CLI commands so it is up to them as to whether they +make breaking changes by replacing handwritten commands with slightly different auto-generated ones. + +For now the SDK will maintain the existing set of CLI commands for backwards compatibility but new commands will use +this functionality. + +### Positive + +* module developers will not need to write CLI commands +* module developers will not need to test CLI commands +* [lens](https://github.com/strangelove-ventures/lens) may benefit from this + +### Negative + +### Neutral + +## Further Discussions + +We would like to be able to customize: + +* short and long usage strings for commands +* aliases for flags (ex. `-a` for `--amount`) +* which fields are positional parameters rather than flags + +It is an [open discussion](https://github.com/cosmos/cosmos-sdk/pull/11725#issuecomment-1108676129) +as to whether these customizations options should line in: + +* the .proto files themselves, +* separate config files (ex. YAML), or +* directly in code + +Providing the options in .proto files would allow a dynamic client to automatically generate +CLI commands on the fly. However, that may pollute the .proto files themselves with information that is only relevant +for a small subset of users. + +## References + +* [Link](https://github.com/regen-network/regen-ledger/issues/1041) +* [Link](https://github.com/cosmos/cosmos-sdk/tree/main/client/v2) +* [Link](https://github.com/cosmos/cosmos-sdk/pull/11725#issuecomment-1108676129) diff --git a/sdk/v0.54/reference/architecture/adr-059-test-scopes.mdx b/sdk/v0.54/reference/architecture/adr-059-test-scopes.mdx new file mode 100644 index 000000000..df0f1c173 --- /dev/null +++ b/sdk/v0.54/reference/architecture/adr-059-test-scopes.mdx @@ -0,0 +1,261 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/architecture/adr-059-test-scopes' +title: 'ADR 059: Test Scopes' +description: >- + 2022-08-02: Initial Draft 2023-03-02: Add precision for integration tests + 2023-03-23: Add precision for E2E tests +--- + +## Changelog + +* 2022-08-02: Initial Draft +* 2023-03-02: Add precision for integration tests +* 2023-03-23: Add precision for E2E tests + +## Status + +PROPOSED Partially Implemented + +## Abstract + +Recent work in the SDK aimed at breaking apart the monolithic root go module has highlighted +shortcomings and inconsistencies in our testing paradigm. This ADR clarifies a common +language for talking about test scopes and proposes an ideal state of tests at each scope. + +## Context + +[ADR-053: Go Module Refactoring](https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-053-go-module-refactoring.md) expresses our desire for an SDK composed of many +independently versioned Go modules, and [ADR-057: App Wiring](https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-057-app-wiring.md) offers a methodology +for breaking apart inter-module dependencies through the use of dependency injection. As +described in [EPIC: Separate all SDK modules into standalone go modules](https://github.com/cosmos/cosmos-sdk/issues/11899), module +dependencies are particularly complected in the test phase, where simapp is used as +the key test fixture in setting up and running tests. It is clear that the successful +completion of Phases 3 and 4 in that EPIC require the resolution of this dependency problem. + +In [EPIC: Unit Testing of Modules via Mocks](https://github.com/cosmos/cosmos-sdk/issues/12398) it was thought this Gordian knot could be +unwound by mocking all dependencies in the test phase for each module, but seeing how these +refactors were complete rewrites of test suites discussions began around the fate of the +existing integration tests. One perspective is that they ought to be thrown out, another is +that integration tests have some utility of their own and a place in the SDK's testing story. + +Another point of confusion has been the current state of CLI test suites, [x/auth](https://github.com/cosmos/cosmos-sdk/blob/0f7e56c6f9102cda0ca9aba5b6f091dbca976b5a/x/auth/client/testutil/suite.go#L44-L49) for +example. In code these are called integration tests, but in reality function as end to end +tests by starting up a tendermint node and full application. [EPIC: Rewrite and simplify +CLI tests](https://github.com/cosmos/cosmos-sdk/issues/12696) identifies the ideal state of CLI tests using mocks, but does not address the +place end to end tests may have in the SDK. + +From here we identify three scopes of testing, **unit**, **integration**, **e2e** (end to +end), seek to define the boundaries of each, their shortcomings (real and imposed), and their +ideal state in the SDK. + +### Unit tests + +Unit tests exercise the code contained in a single module (e.g. `/x/bank`) or package +(e.g. `/client`) in isolation from the rest of the code base. Within this we identify two +levels of unit tests, *illustrative* and *journey*. The definitions below lean heavily on +[The BDD Books - Formulation](https://leanpub.com/bddbooks-formulation) section 1.3. + +*Illustrative* tests exercise an atomic part of a module in isolation - in this case we +might do fixture setup/mocking of other parts of the module. + +Tests which exercise a whole module's function with dependencies mocked, are *journeys*. +These are almost like integration tests in that they exercise many things together but still +use mocks. + +Example 1 journey vs illustrative tests - depinject's BDD style tests, show how we can +rapidly build up many illustrative cases demonstrating behavioral rules without [very much code](https://github.com/cosmos/cosmos-sdk/blob/main/depinject/binding_test.go) while maintaining high level readability. + +Example 2 [depinject table driven tests](https://github.com/cosmos/cosmos-sdk/blob/main/depinject/provider_desc_test.go) + +Example 3 [Bank keeper tests](https://github.com/cosmos/cosmos-sdk/blob/2bec9d2021918650d3938c3ab242f84289daef80/x/bank/keeper/keeper_test.go#L94-L105) - A mock implementation of `AccountKeeper` is supplied to the keeper constructor. + +#### Limitations + +Certain modules are tightly coupled beyond the test phase. A recent dependency report for +`bank -> auth` found 274 total usages of `auth` in `bank`, 50 of which are in +production code and 224 in test. This tight coupling may suggest that either the modules +should be merged, or refactoring is required to abstract references to the core types tying +the modules together. It could also indicate that these modules should be tested together +in integration tests beyond mocked unit tests. + +In some cases setting up a test case for a module with many mocked dependencies can be quite +cumbersome and the resulting test may only show that the mocking framework works as expected +rather than working as a functional test of interdependent module behavior. + +### Integration tests + +Integration tests define and exercise relationships between an arbitrary number of modules +and/or application subsystems. + +Wiring for integration tests is provided by `depinject` and some [helper code](https://github.com/cosmos/cosmos-sdk/blob/2bec9d2021918650d3938c3ab242f84289daef80/testutil/sims/app_helpers.go#L95) starts up +a running application. A section of the running application may then be tested. Certain +inputs during different phases of the application life cycle are expected to produce +invariant outputs without too much concern for component internals. This type of black box +testing has a larger scope than unit testing. + +Example 1 [client/grpc\_query\_test/TestGRPCQuery](https://github.com/cosmos/cosmos-sdk/blob/2bec9d2021918650d3938c3ab242f84289daef80/client/grpc_query_test.go#L111-L129) - This test is misplaced in `/client`, +but tests the life cycle of (at least) `runtime` and `bank` as they progress through +startup, genesis and query time. It also exercises the fitness of the client and query +server without putting bytes on the wire through the use of [QueryServiceTestHelper](https://github.com/cosmos/cosmos-sdk/blob/2bec9d2021918650d3938c3ab242f84289daef80/baseapp/grpcrouter_helpers.go#L31). + +Example 2 `x/evidence` Keeper integration tests - Starts up an application composed of [8 +modules](https://github.com/cosmos/cosmos-sdk/blob/2bec9d2021918650d3938c3ab242f84289daef80/x/evidence/testutil/app.yaml#L1) with [5 keepers](https://github.com/cosmos/cosmos-sdk/blob/2bec9d2021918650d3938c3ab242f84289daef80/x/evidence/keeper/keeper_test.go#L101-L106) used in the integration test suite. One test in the suite +exercises [HandleEquivocationEvidence](https://github.com/cosmos/cosmos-sdk/blob/2bec9d2021918650d3938c3ab242f84289daef80/x/evidence/keeper/infraction_test.go#L42) which contains many interactions with the staking +keeper. + +Example 3 - Integration suite app configurations may also be specified via golang (not +YAML as above) statically or [dynamically](https://github.com/cosmos/cosmos-sdk/blob/8c23f6f957d1c0bedd314806d1ac65bea59b084c/tests/integration/bank/keeper/keeper_test.go#L129-L134). + +#### Limitations + +Setting up a particular input state may be more challenging since the application is +starting from a zero state. Some of this may be addressed by good test fixture +abstractions with testing of their own. Tests may also be more brittle, and larger +refactors could impact application initialization in unexpected ways with harder to +understand errors. This could also be seen as a benefit, and indeed the SDK's current +integration tests were helpful in tracking down logic errors during earlier stages +of app-wiring refactors. + +### Simulations + +Simulations (also called generative testing) are a special case of integration tests where +deterministically random module operations are executed against a running simapp, building +blocks on the chain until a specified height is reached. No *specific* assertions are +made for the state transitions resulting from module operations but any error will halt and +fail the simulation. Since `crisis` is included in simapp and the simulation runs +EndBlockers at the end of each block any module invariant violations will also fail +the simulation. + +Modules must implement [AppModuleSimulation.WeightedOperations](https://github.com/cosmos/cosmos-sdk/blob/2bec9d2021918650d3938c3ab242f84289daef80/types/module/simulation.go#L31) to define their +simulation operations. Note that not all modules implement this which may indicate a +gap in current simulation test coverage. + +Modules not returning simulation operations: + +* `auth` +* `evidence` +* `mint` +* `params` + +A separate binary, [runsim](https://github.com/cosmos/tools/tree/master/cmd/runsim), is responsible for kicking off some of these tests and +managing their life cycle. + +#### Limitations + +* A success may take a long time to run, 7-10 minutes per simulation in CI. +* Timeouts sometimes occur on apparent successes without any indication why. +* Useful error messages not provided on failure from CI, requiring a developer to run + the simulation locally to reproduce. + +### E2E tests + +End to end tests exercise the entire system as we understand it in as close an approximation +to a production environment as is practical. Presently these tests are located at +[tests/e2e](https://github.com/cosmos/cosmos-sdk/tree/main/tests/e2e) and rely on [testutil/network](https://github.com/cosmos/cosmos-sdk/tree/main/testutil/network) to start up an in-process Tendermint node. + +An application should be built as minimally as possible to exercise the desired functionality. +The SDK uses an application will only the required modules for the tests. The application developer is adviced to use its own application for e2e tests. + +#### Limitations + +In general the limitations of end to end tests are orchestration and compute cost. +Scaffolding is required to start up and run a prod-like environment and the this +process takes much longer to start and run than unit or integration tests. + +Global locks present in Tendermint code cause stateful starting/stopping to sometimes hang +or fail intermittently when run in a CI environment. + +The scope of e2e tests has been complected with command line interface testing. + +## Decision + +We accept these test scopes and identify the following decisions points for each. + +| Scope | App Type | Mocks? | +| ----------- | ------------------- | ------ | +| Unit | None | Yes | +| Integration | integration helpers | Some | +| Simulation | minimal app | No | +| E2E | minimal app | No | + +The decision above is valid for the SDK. An application developer should test their application with their full application instead of the minimal app. + +### Unit Tests + +All modules must have mocked unit test coverage. + +Illustrative tests should outnumber journeys in unit tests. + +Unit tests should outnumber integration tests. + +Unit tests must not introduce additional dependencies beyond those already present in +production code. + +When module unit test introduction as per [EPIC: Unit testing of modules via mocks](https://github.com/cosmos/cosmos-sdk/issues/12398) +results in a near complete rewrite of an integration test suite the test suite should be +retained and moved to `/tests/integration`. We accept the resulting test logic +duplication but recommend improving the unit test suite through the addition of +illustrative tests. + +### Integration Tests + +All integration tests shall be located in `/tests/integration`, even those which do not +introduce extra module dependencies. + +To help limit scope and complexity, it is recommended to use the smallest possible number of +modules in application startup, i.e. don't depend on simapp. + +Integration tests should outnumber e2e tests. + +### Simulations + +Simulations shall use a minimal application (usually via app wiring). They are located under `/x/{moduleName}/simulation`. + +### E2E Tests + +Existing e2e tests shall be migrated to integration tests by removing the dependency on the +test network and in-process Tendermint node to ensure we do not lose test coverage. + +The e2e rest runner shall transition from in process Tendermint to a runner powered by +Docker via [dockertest](https://github.com/ory/dockertest). + +E2E tests exercising a full network upgrade shall be written. + +The CLI testing aspect of existing e2e tests shall be rewritten using the network mocking +demonstrated in [PR#12706](https://github.com/cosmos/cosmos-sdk/pull/12706). + +## Consequences + +### Positive + +* test coverage is increased +* test organization is improved +* reduced dependency graph size in modules +* simapp removed as a dependency from modules +* inter-module dependencies introduced in test code are removed +* reduced CI run time after transitioning away from in process Tendermint + +### Negative + +* some test logic duplication between unit and integration tests during transition +* test written using dockertest DX may be a bit worse + +### Neutral + +* some discovery required for e2e transition to dockertest + +## Further Discussions + +It may be useful if test suites could be run in integration mode (with mocked tendermint) or +with e2e fixtures (with real tendermint and many nodes). Integration fixtures could be used +for quicker runs, e2e fixures could be used for more battle hardening. + +A PoC `x/gov` was completed in PR [#12847](https://github.com/cosmos/cosmos-sdk/pull/12847) +is in progress for unit tests demonstrating BDD \[Rejected]. +Observing that a strength of BDD specifications is their readability, and a con is the +cognitive load while writing and maintaining, current consensus is to reserve BDD use +for places in the SDK where complex rules and module interactions are demonstrated. +More straightforward or low level test cases will continue to rely on go table tests. + +Levels are network mocking in integration and e2e tests are still being worked on and formalized. diff --git a/sdk/v0.54/reference/architecture/adr-060-abci-1.0.mdx b/sdk/v0.54/reference/architecture/adr-060-abci-1.0.mdx new file mode 100644 index 000000000..0ec6e8390 --- /dev/null +++ b/sdk/v0.54/reference/architecture/adr-060-abci-1.0.mdx @@ -0,0 +1,260 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/architecture/adr-060-abci-1.0' +title: 'ADR 60: ABCI 1.0 Integration (Phase I)' +description: >- + 2022-08-10: Initial Draft (@alexanderbez, @tac0turtle) Nov 12, 2022: Update + PrepareProposal and ProcessProposal semantics per the initial implementation + PR (@alexanderbez) +--- + +## Changelog + +* 2022-08-10: Initial Draft (@alexanderbez, @tac0turtle) +* Nov 12, 2022: Update `PrepareProposal` and `ProcessProposal` semantics per the + initial implementation [PR](https://github.com/cosmos/cosmos-sdk/pull/13453) (@alexanderbez) + +## Status + +ACCEPTED + +## Abstract + +This ADR describes the initial adoption of [ABCI 1.0](https://github.com/tendermint/tendermint/blob/master/spec/abci%2B%2B/README.md), +the next evolution of ABCI, within the Cosmos SDK. ABCI 1.0 aims to provide +application developers with more flexibility and control over application and +consensus semantics, e.g. in-application mempools, in-process oracles, and +order-book style matching engines. + +## Context + +Tendermint will release ABCI 1.0. Notably, at the time of this writing, +Tendermint is releasing v0.37.0 which will include `PrepareProposal` and `ProcessProposal`. + +The `PrepareProposal` ABCI method is concerned with a block proposer requesting +the application to evaluate a series of transactions to be included in the next +block, defined as a slice of `TxRecord` objects. The application can either +accept, reject, or completely ignore some or all of these transactions. This is +an important consideration to make as the application can essentially define and +control its own mempool allowing it to define sophisticated transaction priority +and filtering mechanisms, by completely ignoring the `TxRecords` Tendermint +sends it, favoring its own transactions. This essentially means that the Tendermint +mempool acts more like a gossip data structure. + +The second ABCI method, `ProcessProposal`, is used to process the block proposer's +proposal as defined by `PrepareProposal`. It is important to note the following +with respect to `ProcessProposal`: + +* Execution of `ProcessProposal` must be deterministic. +* There must be coherence between `PrepareProposal` and `ProcessProposal`. In + other words, for any two correct processes *p* and *q*, if *q*'s Tendermint + calls `RequestProcessProposal` on *up*, *q*'s Application returns + ACCEPT in `ResponseProcessProposal`. + +It is important to note that in ABCI 1.0 integration, the application +is NOT responsible for locking semantics -- Tendermint will still be responsible +for that. In the future, however, the application will be responsible for locking, +which allows for parallel execution possibilities. + +## Decision + +We will integrate ABCI 1.0, which will be introduced in Tendermint +v0.37.0, in the next major release of the Cosmos SDK. We will integrate ABCI 1.0 +methods on the `BaseApp` type. We describe the implementations of the two methods +individually below. + +Prior to describing the implementation of the two new methods, it is important to +note that the existing ABCI methods, `CheckTx`, `DeliverTx`, etc, still exist and +serve the same functions as they do now. + +### `PrepareProposal` + +Prior to evaluating the decision for how to implement `PrepareProposal`, it is +important to note that `CheckTx` will still be executed and will be responsible +for evaluating transaction validity as it does now, with one very important +*additive* distinction. + +When executing transactions in `CheckTx`, the application will now add valid +transactions, i.e. passing the AnteHandler, to its own mempool data structure. +In order to provide a flexible approach to meet the varying needs of application +developers, we will define both a mempool interface and a data structure utilizing +Golang generics, allowing developers to focus only on transaction +ordering. Developers requiring absolute full control can implement their own +custom mempool implementation. + +We define the general mempool interface as follows (subject to change): + +```go expandable +type Mempool interface { + // Insert attempts to insert a Tx into the app-side mempool returning + // an error upon failure. + Insert(sdk.Context, sdk.Tx) + +error + + // Select returns an Iterator over the app-side mempool. If txs are specified, + // then they shall be incorporated into the Iterator. The Iterator must + // closed by the caller. + Select(sdk.Context, [][]byte) + +Iterator + + // CountTx returns the number of transactions currently in the mempool. + CountTx() + +int + + // Remove attempts to remove a transaction from the mempool, returning an error + // upon failure. + Remove(sdk.Tx) + +error +} + +// Iterator defines an app-side mempool iterator interface that is as minimal as +// possible. The order of iteration is determined by the app-side mempool +// implementation. +type Iterator interface { + // Next returns the next transaction from the mempool. If there are no more + // transactions, it returns nil. + Next() + +Iterator + + // Tx returns the transaction at the current position of the iterator. + Tx() + +sdk.Tx +} +``` + +We will define an implementation of `Mempool`, defined by `nonceMempool`, that +will cover most basic application use-cases. Namely, it will prioritize transactions +by transaction sender, allowing for multiple transactions from the same sender. + +The default app-side mempool implementation, `nonceMempool`, will operate on a +single skip list data structure. Specifically, transactions with the lowest nonce +globally are prioritized. Transactions with the same nonce are prioritized by +sender address. + +```go +type nonceMempool struct { + txQueue *huandu.SkipList +} +``` + +Previous discussions1 have come to the agreement that Tendermint will +perform a request to the application, via `RequestPrepareProposal`, with a certain +amount of transactions reaped from Tendermint's local mempool. The exact amount +of transactions reaped will be determined by a local operator configuration. +This is referred to as the "one-shot approach" seen in discussions. + +When Tendermint reaps transactions from the local mempool and sends them to the +application via `RequestPrepareProposal`, the application will have to evaluate +the transactions. Specifically, it will need to inform Tendermint if it should +reject and or include each transaction. Note, the application can even *replace* +transactions entirely with other transactions. + +When evaluating transactions from `RequestPrepareProposal`, the application will +ignore *ALL* transactions sent to it in the request and instead reap up to +`RequestPrepareProposal.max_tx_bytes` from its own mempool. + +Since an application can technically insert or inject transactions on `Insert` +during `CheckTx` execution, it is recommended that applications ensure transaction +validity when reaping transactions during `PrepareProposal`. However, what validity +exactly means is entirely determined by the application. + +The Cosmos SDK will provide a default `PrepareProposal` implementation that simply +select up to `MaxBytes` *valid* transactions. + +However, applications can override this default implementation with their own +implementation and set that on `BaseApp` via `SetPrepareProposal`. + +### `ProcessProposal` + +The `ProcessProposal` ABCI method is relatively straightforward. It is responsible +for ensuring validity of the proposed block containing transactions that were +selected from the `PrepareProposal` step. However, how an application determines +validity of a proposed block depends on the application and its varying use cases. +For most applications, simply calling the `AnteHandler` chain would suffice, but +there could easily be other applications that need more control over the validation +process of the proposed block, such as ensuring txs are in a certain order or +that certain transactions are included. While this theoretically could be achieved +with a custom `AnteHandler` implementation, it's not the cleanest UX or the most +efficient solution. + +Instead, we will define an additional ABCI interface method on the existing +`Application` interface, similar to the existing ABCI methods such as `BeginBlock` +or `EndBlock`. This new interface method will be defined as follows: + +```go +ProcessProposal(sdk.Context, abci.RequestProcessProposal) + +error { +} +``` + +Note, we must call `ProcessProposal` with a new internal branched state on the +`Context` argument as we cannot simply just use the existing `checkState` because +`BaseApp` already has a modified `checkState` at this point. So when executing +`ProcessProposal`, we create a similar branched state, `processProposalState`, +off of `deliverState`. Note, the `processProposalState` is never committed and +is completely discarded after `ProcessProposal` finishes execution. + +The Cosmos SDK will provide a default implementation of `ProcessProposal` in which +all transactions are validated using the CheckTx flow, i.e. the AnteHandler, and +will always return ACCEPT unless any transaction cannot be decoded. + +### `DeliverTx` + +Since transactions are not truly removed from the app-side mempool during +`PrepareProposal`, since `ProcessProposal` can fail or take multiple rounds and +we do not want to lose transactions, we need to finally remove the transaction +from the app-side mempool during `DeliverTx` since during this phase, the +transactions are being included in the proposed block. + +Alternatively, we can keep the transactions as truly being removed during the +reaping phase in `PrepareProposal` and add them back to the app-side mempool in +case `ProcessProposal` fails. + +## Consequences + +### Backwards Compatibility + +ABCI 1.0 is naturally not backwards compatible with prior versions of the Cosmos SDK +and Tendermint. For example, an application that requests `RequestPrepareProposal` +to the same application that does not speak ABCI 1.0 will naturally fail. + +However, in the first phase of the integration, the existing ABCI methods as we +know them today will still exist and function as they currently do. + +### Positive + +* Applications now have full control over transaction ordering and priority. +* Lays the groundwork for the full integration of ABCI 1.0, which will unlock more + app-side use cases around block construction and integration with the Tendermint + consensus engine. + +### Negative + +* Requires that the "mempool", as a general data structure that collects and stores + uncommitted transactions will be duplicated between both Tendermint and the + Cosmos SDK. +* Additional requests between Tendermint and the Cosmos SDK in the context of + block execution. Albeit, the overhead should be negligible. +* Not backwards compatible with previous versions of Tendermint and the Cosmos SDK. + +## Further Discussions + +It is possible to design the app-side implementation of the `Mempool[T MempoolTx]` +in many different ways using different data structures and implementations. All +of which have different tradeoffs. The proposed solution keeps things simple +and covers cases that would be required for most basic applications. There are +tradeoffs that can be made to improve performance of reaping and inserting into +the provided mempool implementation. + +## References + +* [Link](https://github.com/tendermint/tendermint/blob/master/spec/abci%2B%2B/README.md) +* \[1] [Link](https://github.com/tendermint/tendermint/issues/7750#issuecomment-1076806155) +* \[2] [Link](https://github.com/tendermint/tendermint/issues/7750#issuecomment-1075717151) diff --git a/sdk/v0.54/reference/architecture/adr-061-liquid-staking.mdx b/sdk/v0.54/reference/architecture/adr-061-liquid-staking.mdx new file mode 100644 index 000000000..d76bacc41 --- /dev/null +++ b/sdk/v0.54/reference/architecture/adr-061-liquid-staking.mdx @@ -0,0 +1,84 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/architecture/adr-061-liquid-staking' +title: 'ADR ADR-061: Liquid Staking' +description: '2022-09-10: Initial Draft (@zmanian)' +--- + +## Changelog + +* 2022-09-10: Initial Draft (@zmanian) + +## Status + +ACCEPTED + +## Abstract + +Add a semi-fungible liquid staking primitive to the default Cosmos SDK staking module. This upgrades proof of stake to enable safe designs with lower overall monetary issuance and integration with numerous liquid staking protocols like Stride, Persistence, Quicksilver, Lido etc. + +## Context + +The original release of the Cosmos Hub featured the implementation of a ground breaking proof of stake mechanism featuring delegation, slashing, in protocol reward distribution and adaptive issuance. This design was state of the art for 2016 and has been deployed without major changes by many L1 blockchains. + +As both Proof of Stake and blockchain use cases have matured, this design has aged poorly and should no longer be considered a good baseline Proof of Stake issuance. In the world of application specific blockchains, there cannot be a one size fits all blockchain but the Cosmos SDK does endeavour to provide a good baseline implementation and one that is suitable for the Cosmos Hub. + +The most important deficiency of the legacy staking design is that it composes poorly with on chain protocols for trading, lending, derivatives that are referred to collectively as DeFi. The legacy staking implementation starves these applications of liquidity by increasing the risk free rate adaptively. It basically makes DeFi and staking security somewhat incompatible. + +The Osmosis team has adopted the idea of Superfluid and Interfluid staking where assets that are participating in DeFi appliactions can also be used in proof of stake. This requires tight integration with an enshrined set of DeFi applications and thus is unsuitable for the Cosmos SDK. + +It's also important to note that Interchain Accounts are available in the default IBC implementation and can be used to [rehypothecate](https://www.investopedia.com/terms/h/hypothecation.asp#toc-what-is-rehypothecation) delegations. Thus liquid staking is already possible and these changes merely improve the UX of liquid staking. Centralized exchanges also rehypothecate staked assets, posing challenges for decentralization. This ADR takes the position that adoption of in-protocol liquid staking is the preferable outcome and provides new levers to incentivize decentralization of stake. + +These changes to the staking module have been in development for more than a year and have seen substantial industry adoption who plan to build staking UX. The internal economics at Informal team has also done a review of the impacts of these changes and this review led to the development of the exempt delegation system. This system provides governance with a tuneable parameter for modulating the risks of principal agent problem called the exemption factor. + +## Decision + +We implement the semi-fungible liquid staking system and exemption factor system within the cosmos sdk. Though registered as fungible assets, these tokenized shares have extremely limited fungibility, only among the specific delegation record that was created when shares were tokenized. These assets can be used for OTC trades but composability with DeFi is limited. The primary expected use case is improving the user experience of liquid staking providers. + +A new governance parameter is introduced that defines the ratio of exempt to issued tokenized shares. This is called the exemption factor. A larger exemption factor allows more tokenized shares to be issued for a smaller amount of exempt delegations. If governance is comfortable with how the liquid staking market is evolving, it makes sense to increase this value. + +Min self delegation is removed from the staking system with the expectation that it will be replaced by the exempt delegations system. The exempt delegation system allows multiple accounts to demonstrate economic alignment with the validator operator as team members, partners etc. without co-mingling funds. Delegation exemption will likely be required to grow the validators' business under widespread adoption of liquid staking once governance has adjusted the exemption factor. + +When shares are tokenized, the underlying shares are transferred to a module account and rewards go to the module account for the TokenizedShareRecord. + +There is no longer a mechanism to override the validators vote for TokenizedShares. + +### `MsgTokenizeShares` + +The MsgTokenizeShares message is used to create tokenize delegated tokens. This message can be executed by any delegator who has positive amount of delegation and after execution the specific amount of delegation disappear from the account and share tokens are provided. Share tokens are denominated in the validator and record id of the underlying delegation. + +A user may tokenize some or all of their delegation. + +They will receive shares with the denom of `cosmosvaloper1xxxx/5` where 5 is the record id for the validator operator. + +MsgTokenizeShares fails if the account is a VestingAccount. Users will have to move vested tokens to a new account and endure the unbonding period. We view this as an acceptable tradeoff vs. the complex book keeping required to track vested tokens. + +The total amount of outstanding tokenized shares for the validator is checked against the sum of exempt delegations multiplied by the exemption factor. If the tokenized shares exceeds this limit, execution fails. + +MsgTokenizeSharesResponse provides the number of tokens generated and their denom. + +### `MsgRedeemTokensforShares` + +The MsgRedeemTokensforShares message is used to redeem the delegation from share tokens. This message can be executed by any user who owns share tokens. After execution delegations will appear to the user. + +### `MsgTransferTokenizeShareRecord` + +The MsgTransferTokenizeShareRecord message is used to transfer the ownership of rewards generated from the tokenized amount of delegation. The tokenize share record is created when a user tokenize his/her delegation and deleted when the full amount of share tokens are redeemed. + +This is designed to work with liquid staking designs that do not redeem the tokenized shares and may instead want to keep the shares tokenized. + +### `MsgExemptDelegation` + +The MsgExemptDelegation message is used to exempt a delegation to a validator. If the exemption factor is greater than 0, this will allow more delegation shares to be issued from the validator. + +This design allows the chain to force an amount of self-delegation by validators participating in liquid staking schemes. + +## Consequences + +### Backwards Compatibility + +By setting the exemption factor to zero, this module works like legacy staking. The only substantial change is the removal of min-self-bond and without any tokenized shares, there is no incentive to exempt delegation. + +### Positive + +This approach should enable integration with liquid staking providers and improved user experience. It provides a pathway to security under non-exponential issuance policies in the baseline staking module. diff --git a/sdk/v0.54/reference/architecture/adr-062-collections-state-layer.mdx b/sdk/v0.54/reference/architecture/adr-062-collections-state-layer.mdx new file mode 100644 index 000000000..c9d553f75 --- /dev/null +++ b/sdk/v0.54/reference/architecture/adr-062-collections-state-layer.mdx @@ -0,0 +1,121 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/architecture/adr-062-collections-state-layer' +title: 'ADR 062: Collections, a simplified storage layer for cosmos-sdk modules.' +description: '30/11/2022: PROPOSED' +--- + +## Changelog + +* 30/11/2022: PROPOSED + +## Status + +PROPOSED - Implemented + +## Abstract + +We propose a simplified module storage layer which leverages golang generics to allow module developers to handle module +storage in a simple and straightforward manner, whilst offering safety, extensibility and standardisation. + +## Context + +Module developers are forced into manually implementing storage functionalities in their modules, those functionalities include +but are not limited to: + +* Defining key to bytes formats. +* Defining value to bytes formats. +* Defining secondary indexes. +* Defining query methods to expose outside to deal with storage. +* Defining local methods to deal with storage writing. +* Dealing with genesis imports and exports. +* Writing tests for all the above. + +This brings in a lot of problems: + +* It blocks developers from focusing on the most important part: writing business logic. +* Key to bytes formats are complex and their definition is error-prone, for example: + * how do I format time to bytes in such a way that bytes are sorted? + * how do I ensure when I don't have namespace collisions when dealing with secondary indexes? +* The lack of standardisation makes life hard for clients, and the problem is exacerbated when it comes to providing proofs for objects present in state. Clients are forced to maintain a list of object paths to gather proofs. + +### Current Solution: ORM + +The current SDK proposed solution to this problem is [ORM](https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-055-orm.md). +While ORM offers a lot of good functionality aimed at solving these specific problems, it has some downsides: + +* It requires migrations. +* It uses the newest protobuf golang API, whilst the SDK still mainly uses gogoproto. +* Integrating ORM into a module would require the developer to deal with two different golang frameworks (golang protobuf + gogoproto) representing the same API objects. +* It has a high learning curve, even for simple storage layers as it requires developers to have knowledge around protobuf options, custom cosmos-sdk storage extensions, and tooling download. Then after this they still need to learn the code-generated API. + +### CosmWasm Solution: cw-storage-plus + +The collections API takes inspiration from [cw-storage-plus](https://docs.cosmwasm.com/docs/1.0/smart-contracts/state/cw-plus/), +which has demonstrated to be a powerful tool for dealing with storage in CosmWasm contracts. +It's simple, does not require extra tooling, it makes it easy to deal with complex storage structures (indexes, snapshot, etc). +The API is straightforward and explicit. + +## Decision + +We propose to port the `collections` API, whose implementation lives in [NibiruChain/collections](https://github.com/NibiruChain/collections) to cosmos-sdk. + +Collections implements four different storage handlers types: + +* `Map`: which deals with simple `key=>object` mappings. +* `KeySet`: which acts as a `Set` and only retains keys and no object (usecase: allow-lists). +* `Item`: which always contains only one object (usecase: Params) +* `Sequence`: which implements a simple always increasing number (usecase: Nonces) +* `IndexedMap`: builds on top of `Map` and `KeySet` and allows to create relationships with `Objects` and `Objects` secondary keys. + +All the collection APIs build on top of the simple `Map` type. + +Collections is fully generic, meaning that anything can be used as `Key` and `Value`. It can be a protobuf object or not. + +Collections types, in fact, delegate the duty of serialisation of keys and values to a secondary collections API component called `ValueEncoders` and `KeyEncoders`. + +`ValueEncoders` take care of converting a value to bytes (relevant only for `Map`). And offers a plug and play layer which allows us to change how we encode objects, +which is relevant for swapping serialisation frameworks and enhancing performance. +`Collections` already comes in with default `ValueEncoders`, specifically for: protobuf objects, special SDK types (sdk.Int, sdk.Dec). + +`KeyEncoders` take care of converting keys to bytes, `collections` already comes in with some default `KeyEncoders` for some privimite golang types +(uint64, string, time.Time, ...) and some widely used sdk types (sdk.Acc/Val/ConsAddress, sdk.Int/Dec, ...). +These default implementations also offer safety around proper lexicographic ordering and namespace-collision. + +Examples of the collections API can be found here: + +* introduction: [Link](https://github.com/NibiruChain/collections/tree/main/examples) +* usage in nibiru: [x/oracle](https://github.com/NibiruChain/nibiru/blob/master/x/oracle/keeper/keeper.go#L32), [x/perp](https://github.com/NibiruChain/nibiru/blob/master/x/perp/keeper/keeper.go#L31) +* cosmos-sdk's x/staking migrated: [Link](https://github.com/testinginprod/cosmos-sdk/pull/22) + +## Consequences + +### Backwards Compatibility + +The design of `ValueEncoders` and `KeyEncoders` allows modules to retain the same `byte(key)=>byte(value)` mappings, making +the upgrade to the new storage layer non-state breaking. + +### Positive + +* ADR aimed at removing code from the SDK rather than adding it. Migrating just `x/staking` to collections would yield to a net decrease in LOC (even considering the addition of collections itself). +* Simplifies and standardises storage layers across modules in the SDK. +* Does not require to have to deal with protobuf. +* It's pure golang code. +* Generalisation over `KeyEncoders` and `ValueEncoders` allows us to not tie ourself to the data serialisation framework. +* `KeyEncoders` and `ValueEncoders` can be extended to provide schema reflection. + +### Negative + +* Golang generics are not as battle-tested as other Golang features, despite being used in production right now. +* Collection types instantiation needs to be improved. + +### Neutral + +`{neutral consequences}` + +## Further Discussions + +* Automatic genesis import/export (not implemented because of API breakage) +* Schema reflection + +## References diff --git a/sdk/v0.54/reference/architecture/adr-063-core-module-api.mdx b/sdk/v0.54/reference/architecture/adr-063-core-module-api.mdx new file mode 100644 index 000000000..6136c41c6 --- /dev/null +++ b/sdk/v0.54/reference/architecture/adr-063-core-module-api.mdx @@ -0,0 +1,617 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/architecture/adr-063-core-module-api' +title: 'ADR 063: Core Module API' +description: 2022-08-18 First Draft 2022-12-08 First Draft 2023-01-24 Updates +--- + +## Changelog + +* 2022-08-18 First Draft +* 2022-12-08 First Draft +* 2023-01-24 Updates + +## Status + +ACCEPTED Partially Implemented + +## Abstract + +A new core API is proposed as a way to develop cosmos-sdk applications that will eventually replace the existing +`AppModule` and `sdk.Context` frameworks a set of core services and extension interfaces. This core API aims to: + +* be simpler +* more extensible +* more stable than the current framework +* enable deterministic events and queries, +* support event listeners +* [ADR 033: Protobuf-based Inter-Module Communication](/sdk/v0.54/reference/architecture/adr-033-protobuf-inter-module-comm) clients. + +## Context + +Historically modules have exposed their functionality to the framework via the `AppModule` and `AppModuleBasic` +interfaces which have the following shortcomings: + +* both `AppModule` and `AppModuleBasic` need to be defined and registered which is counter-intuitive +* apps need to implement the full interfaces, even parts they don't need (although there are workarounds for this), +* interface methods depend heavily on unstable third party dependencies, in particular Comet, +* legacy required methods have littered these interfaces for far too long + +In order to interact with the state machine, modules have needed to do a combination of these things: + +* get store keys from the app +* call methods on `sdk.Context` which contains more or less the full set of capability available to modules. + +By isolating all the state machine functionality into `sdk.Context`, the set of functionalities available to +modules are tightly coupled to this type. If there are changes to upstream dependencies (such as Comet) +or new functionalities are desired (such as alternate store types), the changes need impact `sdk.Context` and all +consumers of it (basically all modules). Also, all modules now receive `context.Context` and need to convert these +to `sdk.Context`'s with a non-ergonomic unwrapping function. + +Any breaking changes to these interfaces, such as ones imposed by third-party dependencies like Comet, have the +side effect of forcing all modules in the ecosystem to update in lock-step. This means it is almost impossible to have +a version of the module which can be run with 2 or 3 different versions of the SDK or 2 or 3 different versions of +another module. This lock-step coupling slows down overall development within the ecosystem and causes updates to +components to be delayed longer than they would if things were more stable and loosely coupled. + +## Decision + +The `core` API proposes a set of core APIs that modules can rely on to interact with the state machine and expose their +functionalities to it that are designed in a principled way such that: + +* tight coupling of dependencies and unrelated functionalities is minimized or eliminated +* APIs can have long-term stability guarantees +* the SDK framework is extensible in a safe and straightforward way + +The design principles of the core API are as follows: + +* everything that a module wants to interact with in the state machine is a service +* all services coordinate state via `context.Context` and don't try to recreate the "bag of variables" approach of `sdk.Context` +* all independent services are isolated in independent packages with minimal APIs and minimal dependencies +* the core API should be minimalistic and designed for long-term support (LTS) +* a "runtime" module will implement all the "core services" defined by the core API and can handle all module + functionalities exposed by core extension interfaces +* other non-core and/or non-LTS services can be exposed by specific versions of runtime modules or other modules + following the same design principles, this includes functionality that interacts with specific non-stable versions of + third party dependencies such as Comet +* the core API doesn't implement *any* functionality, it just defines types +* go stable API compatibility guidelines are followed: [Link](https://go.dev/blog/module-compatibility) + +A "runtime" module is any module which implements the core functionality of composing an ABCI app, which is currently +handled by `BaseApp` and the `ModuleManager`. Runtime modules which implement the core API are *intentionally* separate +from the core API in order to enable more parallel versions and forks of the runtime module than is possible with the +SDK's current tightly coupled `BaseApp` design while still allowing for a high degree of composability and +compatibility. + +Modules which are built only against the core API don't need to know anything about which version of runtime, +`BaseApp` or Comet in order to be compatible. Modules from the core mainline SDK could be easily composed +with a forked version of runtime with this pattern. + +This design is intended to enable matrices of compatible dependency versions. Ideally a given version of any module +is compatible with multiple versions of the runtime module and other compatible modules. This will allow dependencies +to be selectively updated based on battle-testing. More conservative projects may want to update some dependencies +slower than more fast moving projects. + +### Core Services + +The following "core services" are defined by the core API. All valid runtime module implementations should provide +implementations of these services to modules via both [dependency injection](/sdk/v0.54/reference/architecture/adr-057-app-wiring) and +manual wiring. The individual services described below are all bundled in a convenient `appmodule.Service` +"bundle service" so that for simplicity modules can declare a dependency on a single service. + +#### Store Services + +Store services will be defined in the `cosmossdk.io/core/store` package. + +The generic `store.KVStore` interface is the same as current SDK `KVStore` interface. Store keys have been refactored +into store services which, instead of expecting the context to know about stores, invert the pattern and allow +retrieving a store from a generic context. There are three store services for the three types of currently supported +stores - regular kv-store, memory, and transient: + +```go +type KVStoreService interface { + OpenKVStore(context.Context) + +KVStore +} + +type MemoryStoreService interface { + OpenMemoryStore(context.Context) + +KVStore +} + +type TransientStoreService interface { + OpenTransientStore(context.Context) + +KVStore +} +``` + +Modules can use these services like this: + +```go +func (k msgServer) + +Send(ctx context.Context, msg *types.MsgSend) (*types.MsgSendResponse, error) { + store := k.kvStoreSvc.OpenKVStore(ctx) +} +``` + +Just as with the current runtime module implementation, modules will not need to explicitly name these store keys, +but rather the runtime module will choose an appropriate name for them and modules just need to request the +type of store they need in their dependency injection (or manual) constructors. + +#### Event Service + +The event `Service` will be defined in the `cosmossdk.io/core/event` package. + +The event `Service` allows modules to emit typed and legacy untyped events: + +```go expandable +package event + +type Service interface { + // EmitProtoEvent emits events represented as a protobuf message (as described in ADR 032). + // + // Callers SHOULD assume that these events may be included in consensus. These events + // MUST be emitted deterministically and adding, removing or changing these events SHOULD + // be considered state-machine breaking. + EmitProtoEvent(ctx context.Context, event protoiface.MessageV1) + +error + + // EmitKVEvent emits an event based on an event and kv-pair attributes. + // + // These events will not be part of consensus and adding, removing or changing these events is + // not a state-machine breaking change. + EmitKVEvent(ctx context.Context, eventType string, attrs ...KVEventAttribute) + +error + + // EmitProtoEventNonConsensus emits events represented as a protobuf message (as described in ADR 032), without + // including it in blockchain consensus. + // + // These events will not be part of consensus and adding, removing or changing events is + // not a state-machine breaking change. + EmitProtoEventNonConsensus(ctx context.Context, event protoiface.MessageV1) + +error +} +``` + +Typed events emitted with `EmitProto` should be assumed to be part of blockchain consensus (whether they are part of +the block or app hash is left to the runtime to specify). + +Events emitted by `EmitKVEvent` and `EmitProtoEventNonConsensus` are not considered to be part of consensus and cannot be observed +by other modules. If there is a client-side need to add events in patch releases, these methods can be used. + +#### Logger + +A logger (`cosmossdk.io/log`) must be supplied using `depinject`, and will +be made available for modules to use via `depinject.In`. +Modules using it should follow the current pattern in the SDK by adding the module name before using it. + +```go expandable +type ModuleInputs struct { + depinject.In + + Logger log.Logger +} + +func ProvideModule(in ModuleInputs) + +ModuleOutputs { + keeper := keeper.NewKeeper( + in.logger, + ) +} + +func NewKeeper(logger log.Logger) + +Keeper { + return Keeper{ + logger: logger.With(log.ModuleKey, "x/"+types.ModuleName), +} +} +``` + +### Core `AppModule` extension interfaces + +Modules will provide their core services to the runtime module via extension interfaces built on top of the +`cosmossdk.io/core/appmodule.AppModule` tag interface. This tag interface requires only two empty methods which +allow `depinject` to identify implementors as `depinject.OnePerModule` types and as app module implementations: + +```go +type AppModule interface { + depinject.OnePerModuleType + + // IsAppModule is a dummy method to tag a struct as implementing an AppModule. + IsAppModule() +} +``` + +Other core extension interfaces will be defined in `cosmossdk.io/core` should be supported by valid runtime +implementations. + +#### `MsgServer` and `QueryServer` registration + +`MsgServer` and `QueryServer` registration is done by implementing the `HasServices` extension interface: + +```go +type HasServices interface { + AppModule + + RegisterServices(grpc.ServiceRegistrar) +} +``` + +Because of the `cosmos.msg.v1.service` protobuf option, required for `Msg` services, the same `ServiceRegitrar` can be +used to register both `Msg` and query services. + +#### Genesis + +The genesis `Handler` functions - `DefaultGenesis`, `ValidateGenesis`, `InitGenesis` and `ExportGenesis` - are specified +against the `GenesisSource` and `GenesisTarget` interfaces which will abstract over genesis sources which may be a single +JSON object or collections of JSON objects that can be efficiently streamed. + +```go expandable +// GenesisSource is a source for genesis data in JSON format. It may abstract over a +// single JSON object or separate files for each field in a JSON object that can +// be streamed over. Modules should open a separate io.ReadCloser for each field that +// is required. When fields represent arrays they can efficiently be streamed +// over. If there is no data for a field, this function should return nil, nil. It is +// important that the caller closes the reader when done with it. +type GenesisSource = func(field string) (io.ReadCloser, error) + +// GenesisTarget is a target for writing genesis data in JSON format. It may +// abstract over a single JSON object or JSON in separate files that can be +// streamed over. Modules should open a separate io.WriteCloser for each field +// and should prefer writing fields as arrays when possible to support efficient +// iteration. It is important the caller closers the writer AND checks the error +// when done with it. It is expected that a stream of JSON data is written +// to the writer. +type GenesisTarget = func(field string) (io.WriteCloser, error) +``` + +All genesis objects for a given module are expected to conform to the semantics of a JSON object. +Each field in the JSON object should be read and written separately to support streaming genesis. +The [ORM](/sdk/v0.54/reference/architecture/adr-055-orm) and [collections](/sdk/v0.54/reference/architecture/adr-062-collections-state-layer) both support +streaming genesis and modules using these frameworks generally do not need to write any manual +genesis code. + +To support genesis, modules should implement the `HasGenesis` extension interface: + +```go expandable +type HasGenesis interface { + AppModule + + // DefaultGenesis writes the default genesis for this module to the target. + DefaultGenesis(GenesisTarget) + +error + + // ValidateGenesis validates the genesis data read from the source. + ValidateGenesis(GenesisSource) + +error + + // InitGenesis initializes module state from the genesis source. + InitGenesis(context.Context, GenesisSource) + +error + + // ExportGenesis exports module state to the genesis target. + ExportGenesis(context.Context, GenesisTarget) + +error +} +``` + +#### Pre Blockers + +Modules that have functionality that runs before BeginBlock and should implement the has `HasPreBlocker` interfaces: + +```go +type HasPreBlocker interface { + AppModule + PreBlock(context.Context) + +error +} +``` + +#### Begin and End Blockers + +Modules that have functionality that runs before transactions (begin blockers) or after transactions +(end blockers) should implement the has `HasBeginBlocker` and/or `HasEndBlocker` interfaces: + +```go +type HasBeginBlocker interface { + AppModule + BeginBlock(context.Context) + +error +} + +type HasEndBlocker interface { + AppModule + EndBlock(context.Context) + +error +} +``` + +The `BeginBlock` and `EndBlock` methods will take a `context.Context`, because: + +* most modules don't need Comet information other than `BlockInfo` so we can eliminate dependencies on specific + Comet versions +* for the few modules that need Comet block headers and/or return validator updates, specific versions of the + runtime module will provide specific functionality for interacting with the specific version(s) of Comet + supported + +In order for `BeginBlock`, `EndBlock` and `InitGenesis` to send back validator updates and retrieve full Comet +block headers, the runtime module for a specific version of Comet could provide services like this: + +```go +type ValidatorUpdateService interface { + SetValidatorUpdates(context.Context, []abci.ValidatorUpdate) +} +``` + +Header Service defines a way to get header information about a block. This information is generalized for all implementations: + +```go +type Service interface { + GetHeaderInfo(context.Context) + +Info +} + +type Info struct { + Height int64 // Height returns the height of the block + Hash []byte // Hash returns the hash of the block header + Time time.Time // Time returns the time of the block + ChainID string // ChainId returns the chain ID of the block +} +``` + +Comet Service provides a way to get comet specific information: + +```go expandable +type Service interface { + GetCometInfo(context.Context) + +Info +} + +type CometInfo struct { + Evidence []abci.Misbehavior // Misbehavior returns the misbehavior of the block + // ValidatorsHash returns the hash of the validators + // For Comet, it is the hash of the next validators + ValidatorsHash []byte + ProposerAddress []byte // ProposerAddress returns the address of the block proposer + DecidedLastCommit abci.CommitInfo // DecidedLastCommit returns the last commit info +} +``` + +If a user would like to provide a module other information they would need to implement another service like: + +```go +type RollKit Interface { + ... +} +``` + +We know these types will change at the Comet level and that also a very limited set of modules actually need this +functionality, so they are intentionally kept out of core to keep core limited to the necessary, minimal set of stable +APIs. + +#### Remaining Parts of AppModule + +The current `AppModule` framework handles a number of additional concerns which aren't addressed by this core API. +These include: + +* gas +* block headers +* upgrades +* registration of gogo proto and amino interface types +* cobra query and tx commands +* gRPC gateway +* crisis module invariants +* simulations + +Additional `AppModule` extension interfaces either inside or outside of core will need to be specified to handle +these concerns. + +In the case of gogo proto and amino interfaces, the registration of these generally should happen as early +as possible during initialization and in [ADR 057: App Wiring](/sdk/v0.54/reference/architecture/adr-057-app-wiring), protobuf type registration\ +happens before dependency injection (although this could alternatively be done dedicated DI providers). + +gRPC gateway registration should probably be handled by the runtime module, but the core API shouldn't depend on gRPC +gateway types as 1) we are already using an older version and 2) it's possible the framework can do this registration +automatically in the future. So for now, the runtime module should probably provide some sort of specific type for doing +this registration ex: + +```go +type GrpcGatewayInfo struct { + Handlers []GrpcGatewayHandler +} + +type GrpcGatewayHandler func(ctx context.Context, mux *runtime.ServeMux, client QueryClient) + +error +``` + +which modules can return in a provider: + +```go +func ProvideGrpcGateway() + +GrpcGatewayInfo { + return GrpcGatewayinfo { + Handlers: []Handler { + types.RegisterQueryHandlerClient +} + +} +} +``` + +Crisis module invariants and simulations are subject to potential redesign and should be managed with types +defined in the crisis and simulation modules respectively. + +Extension interface for CLI commands will be provided via the `cosmossdk.io/client/v2` module and its +[autocli](/sdk/v0.54/reference/architecture/adr-058-auto-generated-cli) framework. + +#### Example Usage + +Here is an example of setting up a hypothetical `foo` v2 module which uses the [ORM](/sdk/v0.54/reference/architecture/adr-055-orm) for its state +management and genesis. + +```go expandable +type Keeper struct { + db orm.ModuleDB + evtSrv event.Service +} + +func (k Keeper) + +RegisterServices(r grpc.ServiceRegistrar) { + foov1.RegisterMsgServer(r, k) + +foov1.RegisterQueryServer(r, k) +} + +func (k Keeper) + +BeginBlock(context.Context) + +error { + return nil +} + +func ProvideApp(config *foomodulev2.Module, evtSvc event.EventService, db orm.ModuleDB) (Keeper, appmodule.AppModule) { + k := &Keeper{ + db: db, evtSvc: evtSvc +} + +return k, k +} +``` + +### Runtime Compatibility Version + +The `core` module will define a static integer var, `cosmossdk.io/core.RuntimeCompatibilityVersion`, which is +a minor version indicator of the core module that is accessible at runtime. Correct runtime module implementations +should check this compatibility version and return an error if the current `RuntimeCompatibilityVersion` is higher +than the version of the core API that this runtime version can support. When new features are adding to the `core` +module API that runtime modules are required to support, this version should be incremented. + +### Runtime Modules + +The initial `runtime` module will simply be created within the existing `github.com/cosmos/cosmos-sdk` go module +under the `runtime` package. This module will be a small wrapper around the existing `BaseApp`, `sdk.Context` and +module manager and follow the Cosmos SDK's existing [0-based versioning](https://0ver.org). To move to semantic +versioning as well as runtime modularity, new officially supported runtime modules will be created under the +`cosmossdk.io/runtime` prefix. For each supported consensus engine a semantically-versioned go module should be created +with a runtime implementation for that consensus engine. For example: + +* `cosmossdk.io/runtime/comet` +* `cosmossdk.io/runtime/comet/v2` +* `cosmossdk.io/runtime/rollkit` +* etc. + +These runtime modules should attempt to be semantically versioned even if the underlying consensus engine is not. Also, +because a runtime module is also a first class Cosmos SDK module, it should have a protobuf module config type. +A new semantically versioned module config type should be created for each of these runtime module such that there is a +1:1 correspondence between the go module and module config type. This is the same practice should be followed for every +semantically versioned Cosmos SDK module as described in [ADR 057: App Wiring](/sdk/v0.54/reference/architecture/adr-057-app-wiring). + +Currently, `github.com/cosmos/cosmos-sdk/runtime` uses the protobuf config type `cosmos.app.runtime.v1alpha1.Module`. +When we have a standalone v1 comet runtime, we should use a dedicated protobuf module config type such as +`cosmos.runtime.comet.v1.Module1`. When we release v2 of the comet runtime (`cosmossdk.io/runtime/comet/v2`) we should +have a corresponding `cosmos.runtime.comet.v2.Module` protobuf type. + +In order to make it easier to support different consensus engines that support the same core module functionality as +described in this ADR, a common go module should be created with shared runtime components. The easiest runtime components +to share initially are probably the message/query router, inter-module client, service register, and event router. +This common runtime module should be created initially as the `cosmossdk.io/runtime/common` go module. + +When this new architecture has been implemented, the main dependency for a Cosmos SDK module would be +`cosmossdk.io/core` and that module should be able to be used with any supported consensus engine (to the extent +that it does not explicitly depend on consensus engine specific functionality such as Comet's block headers). An +app developer would then be able to choose which consensus engine they want to use by importing the corresponding +runtime module. The current `BaseApp` would be refactored into the `cosmossdk.io/runtime/comet` module, the router +infrastructure in `baseapp/` would be refactored into `cosmossdk.io/runtime/common` and support ADR 033, and eventually +a dependency on `github.com/cosmos/cosmos-sdk` would no longer be required. + +In short, modules would depend primarily on `cosmossdk.io/core`, and each `cosmossdk.io/runtime/{consensus-engine}` +would implement the `cosmossdk.io/core` functionality for that consensus engine. + +On additional piece that would need to be resolved as part of this architecture is how runtimes relate to the server. +Likely it would make sense to modularize the current server architecture so that it can be used with any runtime even +if that is based on a consensus engine besides Comet. This means that eventually the Comet runtime would need to +encapsulate the logic for starting Comet and the ABCI app. + +### Testing + +A mock implementation of all services should be provided in core to allow for unit testing of modules +without needing to depend on any particular version of runtime. Mock services should +allow tests to observe service behavior or provide a non-production implementation - for instance memory +stores can be used to mock stores. + +For integration testing, a mock runtime implementation should be provided that allows composing different app modules +together for testing without a dependency on runtime or Comet. + +## Consequences + +### Backwards Compatibility + +Early versions of runtime modules should aim to support as much as possible modules built with the existing +`AppModule`/`sdk.Context` framework. As the core API is more widely adopted, later runtime versions may choose to +drop support and only support the core API plus any runtime module specific APIs (like specific versions of Comet). + +The core module itself should strive to remain at the go semantic version `v1` as long as possible and follow design +principles that allow for strong long-term support (LTS). + +Older versions of the SDK can support modules built against core with adaptors that convert wrap core `AppModule` +implementations in implementations of `AppModule` that conform to that version of the SDK's semantics as well +as by providing service implementations by wrapping `sdk.Context`. + +### Positive + +* better API encapsulation and separation of concerns +* more stable APIs +* more framework extensibility +* deterministic events and queries +* event listeners +* inter-module msg and query execution support +* more explicit support for forking and merging of module versions (including runtime) + +### Negative + +### Neutral + +* modules will need to be refactored to use this API +* some replacements for `AppModule` functionality still need to be defined in follow-ups + (type registration, commands, invariants, simulations) and this will take additional design work + +## Further Discussions + +* gas +* block headers +* upgrades +* registration of gogo proto and amino interface types +* cobra query and tx commands +* gRPC gateway +* crisis module invariants +* simulations + +## References + +* [ADR 033: Protobuf-based Inter-Module Communication](/sdk/v0.54/reference/architecture/adr-033-protobuf-inter-module-comm) +* [ADR 057: App Wiring](/sdk/v0.54/reference/architecture/adr-057-app-wiring) +* [ADR 055: ORM](/sdk/v0.54/reference/architecture/adr-055-orm) +* [ADR 028: Public Key Addresses](/sdk/v0.54/reference/architecture/adr-028-public-key-addresses) +* [Keeping Your Modules Compatible](https://go.dev/blog/module-compatibility) diff --git a/sdk/v0.54/reference/architecture/adr-064-abci-2.0.mdx b/sdk/v0.54/reference/architecture/adr-064-abci-2.0.mdx new file mode 100644 index 000000000..1c5099a1b --- /dev/null +++ b/sdk/v0.54/reference/architecture/adr-064-abci-2.0.mdx @@ -0,0 +1,507 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/architecture/adr-064-abci-2.0' +title: 'ADR 64: ABCI 2.0 Integration (Phase II)' +--- + +## Changelog + +* 2023-01-17: Initial Draft (@alexanderbez) +* 2023-04-06: Add upgrading section (@alexanderbez) +* 2023-04-10: Simplify vote extension state persistence (@alexanderbez) +* 2023-07-07: Revise vote extension state persistence (@alexanderbez) +* 2023-08-24: Revise vote extension power calculations and staking interface (@davidterpay) + +## Status + +ACCEPTED + +## Abstract + +This ADR outlines the continuation of the efforts to implement ABCI++ in the Cosmos +SDK outlined in [ADR 060: ABCI 1.0 (Phase I)](/sdk/v0.54/reference/architecture/adr-060-abci-1.0). + +Specifically, this ADR outlines the design and implementation of ABCI 2.0, which +includes `ExtendVote`, `VerifyVoteExtension` and `FinalizeBlock`. + +## Context + +ABCI 2.0 continues the promised updates from ABCI++, specifically three additional +ABCI methods that the application can implement in order to gain further control, +insight and customization of the consensus process, unlocking many novel use-cases +that previously not possible. We describe these three new methods below: + +### `ExtendVote` + +This method allows each validator process to extend the pre-commit phase of the +CometBFT consensus process. Specifically, it allows the application to perform +custom business logic that extends the pre-commit vote and supply additional data +as part of the vote, although they are signed separately by the same key. + +The data, called vote extension, will be broadcast and received together with the +vote it is extending, and will be made available to the application in the next +height. Specifically, the proposer of the next block will receive the vote extensions +in `RequestPrepareProposal.local_last_commit.votes`. + +If the application does not have vote extension information to provide, it +returns a 0-length byte array as its vote extension. + +**NOTE**: + +* Although each validator process submits its own vote extension, ONLY the *proposer* + of the *next* block will receive all the vote extensions included as part of the + pre-commit phase of the previous block. This means only the proposer will + implicitly have access to all the vote extensions, via `RequestPrepareProposal`, + and that not all vote extensions may be included, since a validator does not + have to wait for all pre-commits, only 2/3. +* The pre-commit vote is signed independently from the vote extension. + +### `VerifyVoteExtension` + +This method allows validators to validate the vote extension data attached to +each pre-commit message it receives. If the validation fails, the whole pre-commit +message will be deemed invalid and ignored by CometBFT. + +CometBFT uses `VerifyVoteExtension` when validating a pre-commit vote. Specifically, +for a pre-commit, CometBFT will: + +* Reject the message if it doesn't contain a signed vote AND a signed vote extension +* Reject the message if the vote's signature OR the vote extension's signature fails to verify +* Reject the message if `VerifyVoteExtension` was rejected by the app + +Otherwise, CometBFT will accept the pre-commit message. + +Note, this has important consequences on liveness, i.e., if vote extensions repeatedly +cannot be verified by correct validators, CometBFT may not be able to finalize +a block even if sufficiently many (+2/3) validators send pre-commit votes for +that block. Thus, `VerifyVoteExtension` should be used with special care. + +CometBFT recommends that an application that detects an invalid vote extension +SHOULD accept it in `ResponseVerifyVoteExtension` and ignore it in its own logic. + +### `FinalizeBlock` + +This method delivers a decided block to the application. The application must +execute the transactions in the block deterministically and update its state +accordingly. Cryptographic commitments to the block and transaction results, +returned via the corresponding parameters in `ResponseFinalizeBlock`, are +included in the header of the next block. CometBFT calls it when a new block +is decided. + +In other words, `FinalizeBlock` encapsulates the current ABCI execution flow of +`BeginBlock`, one or more `DeliverTx`, and `EndBlock` into a single ABCI method. +CometBFT will no longer execute requests for these legacy methods and instead +will just simply call `FinalizeBlock`. + +## Decision + +We will discuss changes to the Cosmos SDK to implement ABCI 2.0 in two distinct +phases, `VoteExtensions` and `FinalizeBlock`. + +### `VoteExtensions` + +Similarly for `PrepareProposal` and `ProcessProposal`, we propose to introduce +two new handlers that an application can implement in order to provide and verify +vote extensions. + +We propose the following new handlers for applications to implement: + +```go +type ExtendVoteHandler func(sdk.Context, abci.RequestExtendVote) + +abci.ResponseExtendVote +type VerifyVoteExtensionHandler func(sdk.Context, abci.RequestVerifyVoteExtension) + +abci.ResponseVerifyVoteExtension +``` + +An ephemeral context and state will be supplied to both handlers. The +context will contain relevant metadata such as the block height and block hash. +The state will be a cached version of the committed state of the application and +will be discarded after the execution of the handler, this means that both handlers +get a fresh state view and no changes made to it will be written. + +If an application decides to implement `ExtendVoteHandler`, it must return a +non-nil `ResponseExtendVote.VoteExtension`. + +Recall, an implementation of `ExtendVoteHandler` does NOT need to be deterministic, +however, given a set of vote extensions, `VerifyVoteExtensionHandler` must be +deterministic, otherwise the chain may suffer from liveness faults. In addition, +recall CometBFT proceeds in rounds for each height, so if a decision cannot be +made about about a block proposal at a given height, CometBFT will proceed to the +next round and thus will execute `ExtendVote` and `VerifyVoteExtension` again for +the new round for each validator until 2/3 valid pre-commits can be obtained. + +Given the broad scope of potential implementations and use-cases of vote extensions, +and how to verify them, most applications should choose to implement the handlers +through a single handler type, which can have any number of dependencies injected +such as keepers. In addition, this handler type could contain some notion of +volatile vote extension state management which would assist in vote extension +verification. This state management could be ephemeral or could be some form of +on-disk persistence. + +Example: + +```go expandable +// VoteExtensionHandler implements an Oracle vote extension handler. +type VoteExtensionHandler struct { + cdc Codec + mk MyKeeper + state VoteExtState // This could be a map or a DB connection object +} + +// ExtendVoteHandler can do something with h.mk and possibly h.state to create +// a vote extension, such as fetching a series of prices for supported assets. +func (h VoteExtensionHandler) + +ExtendVoteHandler(ctx sdk.Context, req abci.RequestExtendVote) + +abci.ResponseExtendVote { + prices := GetPrices(ctx, h.mk.Assets()) + +bz, err := EncodePrices(h.cdc, prices) + if err != nil { + panic(fmt.Errorf("failed to encode prices for vote extension: %w", err)) +} + + // store our vote extension at the given height + // + // NOTE: Vote extensions can be overridden since we can timeout in a round. + SetPrices(h.state, req, bz) + +return abci.ResponseExtendVote{ + VoteExtension: bz +} +} + +// VerifyVoteExtensionHandler can do something with h.state and req to verify +// the req.VoteExtension field, such as ensuring the provided oracle prices are +// within some valid range of our prices. +func (h VoteExtensionHandler) + +VerifyVoteExtensionHandler(ctx sdk.Context, req abci.RequestVerifyVoteExtension) + +abci.ResponseVerifyVoteExtension { + prices, err := DecodePrices(h.cdc, req.VoteExtension) + if err != nil { + log("failed to decode vote extension", "err", err) + +return abci.ResponseVerifyVoteExtension{ + Status: REJECT +} + +} + if err := ValidatePrices(h.state, req, prices); err != nil { + log("failed to validate vote extension", "prices", prices, "err", err) + +return abci.ResponseVerifyVoteExtension{ + Status: REJECT +} + +} + + // store updated vote extensions at the given height + // + // NOTE: Vote extensions can be overridden since we can timeout in a round. + SetPrices(h.state, req, req.VoteExtension) + +return abci.ResponseVerifyVoteExtension{ + Status: ACCEPT +} +} +``` + +#### Vote Extension Propagation & Verification + +As mentioned previously, vote extensions for height `H` are only made available +to the proposer at height `H+1` during `PrepareProposal`. However, in order to +make vote extensions useful, all validators should have access to the agreed upon +vote extensions at height `H` during `H+1`. + +Since CometBFT includes all the vote extension signatures in `RequestPrepareProposal`, +we propose that the proposing validator manually "inject" the vote extensions +along with their respective signatures via a special transaction, `VoteExtsTx`, +into the block proposal during `PrepareProposal`. The `VoteExtsTx` will be +populated with a single `ExtendedCommitInfo` object which is received directly +from `RequestPrepareProposal`. + +For convention, the `VoteExtsTx` transaction should be the first transaction in +the block proposal, although chains can implement their own preferences. For +safety purposes, we also propose that the proposer itself verify all the vote +extension signatures it receives in `RequestPrepareProposal`. + +A validator, upon a `RequestProcessProposal`, will receive the injected `VoteExtsTx` +which includes the vote extensions along with their signatures. If no such transaction +exists, the validator MUST REJECT the proposal. + +When a validator inspects a `VoteExtsTx`, it will evaluate each `SignedVoteExtension`. +For each signed vote extension, the validator will generate the signed bytes and +verify the signature. At least 2/3 valid signatures, based on voting power, must +be received in order for the block proposal to be valid, otherwise the validator +MUST REJECT the proposal. + +In order to have the ability to validate signatures, `BaseApp` must have access +to the `x/staking` module, since this module stores an index from consensus +address to public key. However, we will avoid a direct dependency on `x/staking` +and instead rely on an interface instead. In addition, the Cosmos SDK will expose +a default signature verification method which applications can use: + +```go expandable +type ValidatorStore interface { + GetPubKeyByConsAddr(context.Context, sdk.ConsAddress) (cmtprotocrypto.PublicKey, error) +} + +// ValidateVoteExtensions is a function that an application can execute in +// ProcessProposal to verify vote extension signatures. +func (app *BaseApp) + +ValidateVoteExtensions(ctx sdk.Context, currentHeight int64, extCommit abci.ExtendedCommitInfo) + +error { + votingPower := 0 + totalVotingPower := 0 + for _, vote := range extCommit.Votes { + totalVotingPower += vote.Validator.Power + if !vote.SignedLastBlock || len(vote.VoteExtension) == 0 { + continue +} + valConsAddr := sdk.ConsAddress(vote.Validator.Address) + +pubKeyProto, err := valStore.GetPubKeyByConsAddr(ctx, valConsAddr) + if err != nil { + return fmt.Errorf("failed to get public key for validator %s: %w", valConsAddr, err) +} + if len(vote.ExtensionSignature) == 0 { + return fmt.Errorf("received a non-empty vote extension with empty signature for validator %s", valConsAddr) +} + +cmtPubKey, err := cryptoenc.PubKeyFromProto(pubKeyProto) + if err != nil { + return fmt.Errorf("failed to convert validator %X public key: %w", valConsAddr, err) +} + cve := cmtproto.CanonicalVoteExtension{ + Extension: vote.VoteExtension, + Height: currentHeight - 1, // the vote extension was signed in the previous height + Round: int64(extCommit.Round), + ChainId: app.GetChainID(), +} + +extSignBytes, err := cosmosio.MarshalDelimited(&cve) + if err != nil { + return fmt.Errorf("failed to encode CanonicalVoteExtension: %w", err) +} + if !cmtPubKey.VerifySignature(extSignBytes, vote.ExtensionSignature) { + return errors.New("received vote with invalid signature") +} + +votingPower += vote.Validator.Power +} + if (votingPower / totalVotingPower) < threshold { + return errors.New("not enough voting power for the vote extensions") +} + +return nil +} +``` + +Once at least 2/3 signatures, by voting power, are received and verified, the +validator can use the vote extensions to derive additional data or come to some +decision based on the vote extensions. + +> NOTE: It is very important to state, that neither the vote propagation technique +> nor the vote extension verification mechanism described above is required for +> applications to implement. In other words, a proposer is not required to verify +> and propagate vote extensions along with their signatures nor are proposers +> required to verify those signatures. An application can implement its own +> PKI mechanism and use that to sign and verify vote extensions. + +#### Vote Extension Persistence + +In certain contexts, it may be useful or necessary for applications to persist +data derived from vote extensions. In order to facilitate this use case, we propose +to allow app developers to define a pre-Blocker hook which will be called +at the very beginning of `FinalizeBlock`, i.e. before `BeginBlock` (see below). + +Note, we cannot allow applications to directly write to the application state +during `ProcessProposal` because during replay, CometBFT will NOT call `ProcessProposal`, +which would result in an incomplete state view. + +```go +func (a MyApp) + +PreBlocker(ctx sdk.Context, req *abci.RequestFinalizeBlock) + +error { + voteExts := GetVoteExtensions(ctx, req.Txs) + + // Process and perform some compute on vote extensions, storing any resulting + // state. + if err a.processVoteExtensions(ctx, voteExts); if err != nil { + return err +} +} +``` + +### `FinalizeBlock` + +The existing ABCI methods `BeginBlock`, `DeliverTx`, and `EndBlock` have existed +since the dawn of ABCI-based applications. Thus, applications, tooling, and developers +have grown used to these methods and their use-cases. Specifically, `BeginBlock` +and `EndBlock` have grown to be pretty integral and powerful within ABCI-based +applications. E.g. an application might want to run distribution and inflation +related operations prior to executing transactions and then have staking related +changes to happen after executing all transactions. + +We propose to keep `BeginBlock` and `EndBlock` within the SDK's core module +interfaces only so application developers can continue to build against existing +execution flows. However, we will remove `BeginBlock`, `DeliverTx` and `EndBlock` +from the SDK's `BaseApp` implementation and thus the ABCI surface area. + +What will then exist is a single `FinalizeBlock` execution flow. Specifically, in +`FinalizeBlock` we will execute the application's `BeginBlock`, followed by +execution of all the transactions, finally followed by execution of the application's +`EndBlock`. + +Note, we will still keep the existing transaction execution mechanics within +`BaseApp`, but all notions of `DeliverTx` will be removed, i.e. `deliverState` +will be replace with `finalizeState`, which will be committed on `Commit`. + +However, there are current parameters and fields that exist in the existing +`BeginBlock` and `EndBlock` ABCI types, such as votes that are used in distribution +and byzantine validators used in evidence handling. These parameters exist in the +`FinalizeBlock` request type, and will need to be passed to the application's +implementations of `BeginBlock` and `EndBlock`. + +This means the Cosmos SDK's core module interfaces will need to be updated to +reflect these parameters. The easiest and most straightforward way to achieve +this is to just pass `RequestFinalizeBlock` to `BeginBlock` and `EndBlock`. +Alternatively, we can create dedicated proxy types in the SDK that reflect these +legacy ABCI types, e.g. `LegacyBeginBlockRequest` and `LegacyEndBlockRequest`. Or, +we can come up with new types and names altogether. + +```go expandable +func (app *BaseApp) + +FinalizeBlock(req abci.RequestFinalizeBlock) (*abci.ResponseFinalizeBlock, error) { + ctx := ... + if app.preBlocker != nil { + ctx := app.finalizeBlockState.ctx + rsp, err := app.preBlocker(ctx, req) + if err != nil { + return nil, err +} + if rsp.ConsensusParamsChanged { + app.finalizeBlockState.ctx = ctx.WithConsensusParams(app.GetConsensusParams(ctx)) +} + +} + +beginBlockResp, err := app.beginBlock(req) + +appendBlockEventAttr(beginBlockResp.Events, "begin_block") + txExecResults := make([]abci.ExecTxResult, 0, len(req.Txs)) + for _, tx := range req.Txs { + result := app.runTx(runTxModeFinalize, tx) + +txExecResults = append(txExecResults, result) +} + +endBlockResp, err := app.endBlock(app.finalizeBlockState.ctx) + +appendBlockEventAttr(beginBlockResp.Events, "end_block") + +return abci.ResponseFinalizeBlock{ + TxResults: txExecResults, + Events: joinEvents(beginBlockResp.Events, endBlockResp.Events), + ValidatorUpdates: endBlockResp.ValidatorUpdates, + ConsensusParamUpdates: endBlockResp.ConsensusParamUpdates, + AppHash: nil, +} +} +``` + +#### Events + +Many tools, indexers and ecosystem libraries rely on the existence `BeginBlock` +and `EndBlock` events. Since CometBFT now only exposes `FinalizeBlockEvents`, we +find that it will still be useful for these clients and tools to still query for +and rely on existing events, especially since applications will still define +`BeginBlock` and `EndBlock` implementations. + +In order to facilitate existing event functionality, we propose that all `BeginBlock` +and `EndBlock` events have a dedicated `EventAttribute` with `key=block` and +`value=begin_block|end_block`. The `EventAttribute` will be appended to each event +in both `BeginBlock` and `EndBlock` events\`. + +### Upgrading + +CometBFT defines a consensus parameter, [`VoteExtensionsEnableHeight`](https://github.com/cometbft/cometbft/blob/v0.38.0-alpha.1/spec/abci/abci%2B%2B_app_requirements.md#abciparamsvoteextensionsenableheight), +which specifies the height at which vote extensions are enabled and **required**. +If the value is set to zero, which is the default, then vote extensions are +disabled and an application is not required to implement and use vote extensions. + +However, if the value `H` is positive, at all heights greater than the configured +height `H` vote extensions must be present (even if empty). When the configured +height `H` is reached, `PrepareProposal` will not include vote extensions yet, +but `ExtendVote` and `VerifyVoteExtension` will be called. Then, when reaching +height `H+1`, `PrepareProposal` will include the vote extensions from height `H`. + +It is very important to note, for all heights after H: + +* Vote extensions CANNOT be disabled +* They are mandatory, i.e. all pre-commit messages sent MUST have an extension + attached (even if empty) + +When an application updates to the Cosmos SDK version with CometBFT v0.38 support, +in the upgrade handler it must ensure to set the consensus parameter +`VoteExtensionsEnableHeight` to the correct value. E.g. if an application is set +to perform an upgrade at height `H`, then the value of `VoteExtensionsEnableHeight` +should be set to any value `>=H+1`. This means that at the upgrade height, `H`, +vote extensions will not be enabled yet, but at height `H+1` they will be enabled. + +## Consequences + +### Backwards Compatibility + +ABCI 2.0 is naturally not backwards compatible with prior versions of the Cosmos SDK +and CometBFT. For example, an application that requests `RequestFinalizeBlock` +to the same application that does not speak ABCI 2.0 will naturally fail. + +In addition, `BeginBlock`, `DeliverTx` and `EndBlock` will be removed from the +application ABCI interfaces and along with the inputs and outputs being modified +in the module interfaces. + +### Positive + +* `BeginBlock` and `EndBlock` semantics remain, so burden on application developers + should be limited. +* Less communication overhead as multiple ABCI requests are condensed into a single + request. +* Sets the groundwork for optimistic execution. +* Vote extensions allow for an entirely new set of application primitives to be + developed, such as in-process price oracles and encrypted mempools. + +### Negative + +* Some existing Cosmos SDK core APIs may need to be modified and thus broken. +* Signature verification in `ProcessProposal` of 100+ vote extension signatures + will add significant performance overhead to `ProcessProposal`. Granted, the + signature verification process can happen concurrently using an error group + with `GOMAXPROCS` goroutines. + +### Neutral + +* Having to manually "inject" vote extensions into the block proposal during + `PrepareProposal` is an awkward approach and takes up block space unnecessarily. +* The requirement of `ResetProcessProposalState` can create a footgun for + application developers if they're not careful, but this is necessary in order + for applications to be able to commit state from vote extension computation. + +## Further Discussions + +Future discussions include design and implementation of ABCI 3.0, which is a +continuation of ABCI++ and the general discussion of optimistic execution. + +## References + +* [ADR 060: ABCI 1.0 (Phase I)](/sdk/v0.54/reference/architecture/adr-060-abci-1.0) diff --git a/sdk/v0.54/reference/architecture/adr-065-store-v2.mdx b/sdk/v0.54/reference/architecture/adr-065-store-v2.mdx new file mode 100644 index 000000000..0969a0f37 --- /dev/null +++ b/sdk/v0.54/reference/architecture/adr-065-store-v2.mdx @@ -0,0 +1,295 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/architecture/adr-065-store-v2' +title: 'ADR-065: Store V2' +description: 'Feb 14, 2023: Initial Draft (@alexanderbez)' +--- + +## Changelog + +* Feb 14, 2023: Initial Draft (@alexanderbez) + +## Status + +DRAFT + +## Abstract + +The storage and state primitives that Cosmos SDK based applications have used have +by and large not changed since the launch of the inaugural Cosmos Hub. The demands +and needs of Cosmos SDK based applications, from both developer and client UX +perspectives, have evolved and outgrown the ecosystem since these primitives +were first introduced. + +Over time as these applications have gained significant adoption, many critical +shortcomings and flaws have been exposed in the state and storage primitives of +the Cosmos SDK. + +In order to keep up with the evolving demands and needs of both clients and developers, +a major overhaul to these primitives are necessary. + +## Context + +The Cosmos SDK provides application developers with various storage primitives +for dealing with application state. Specifically, each module contains its own +merkle commitment data structure -- an IAVL tree. In this data structure, a module +can store and retrieve key-value pairs along with Merkle commitments, i.e. proofs, +to those key-value pairs indicating that they do or do not exist in the global +application state. This data structure is the base layer `KVStore`. + +In addition, the SDK provides abstractions on top of this Merkle data structure. +Namely, a root multi-store (RMS) is a collection of each module's `KVStore`. +Through the RMS, the application can serve queries and provide proofs to clients +in addition to provide a module access to its own unique `KVStore` though the use +of `StoreKey`, which is an OCAP primitive. + +There are further layers of abstraction that sit between the RMS and the underlying +IAVL `KVStore`. A `GasKVStore` is responsible for tracking gas IO consumption for +state machine reads and writes. A `CacheKVStore` is responsible for providing a +way to cache reads and buffer writes to make state transitions atomic, e.g. +transaction execution or governance proposal execution. + +There are a few critical drawbacks to these layers of abstraction and the overall +design of storage in the Cosmos SDK: + +* Since each module has its own IAVL `KVStore`, commitments are not [atomic](https://github.com/cosmos/cosmos-sdk/issues/14625) + * Note, we can still allow modules to have their own IAVL `KVStore`, but the + IAVL library will need to support the ability to pass a DB instance as an + argument to various IAVL APIs. +* Since IAVL is responsible for both state storage and commitment, running an + archive node becomes increasingly expensive as disk space grows exponentially. +* As the size of a network increases, various performance bottlenecks start to + emerge in many areas such as query performance, network upgrades, state + migrations, and general application performance. +* Developer UX is poor as it does not allow application developers to experiment + with different types of approaches to storage and commitments, along with the + complications of many layers of abstractions referenced above. + +See the [Storage Discussion](https://github.com/cosmos/cosmos-sdk/discussions/13545) for more information. + +## Alternatives + +There was a previous attempt to refactor the storage layer described in [ADR-040](/sdk/v0.50/build/architecture/adr-040-storage-and-smt-state-commitments). +However, this approach mainly stems on the short comings of IAVL and various performance +issues around it. While there was a (partial) implementation of [ADR-040](/sdk/v0.50/build/architecture/adr-040-storage-and-smt-state-commitments), +it was never adopted for a variety of reasons, such as the reliance on using an +SMT, which was more in a research phase, and some design choices that couldn't +be fully agreed upon, such as the snap-shotting mechanism that would result in +massive state bloat. + +## Decision + +We propose to build upon some of the great ideas introduced in [ADR-040](/sdk/v0.50/build/architecture/adr-040-storage-and-smt-state-commitments), +while being a bit more flexible with the underlying implementations and overall +less intrusive. Specifically, we propose to: + +* Separate the concerns of state commitment (**SC**), needed for consensus, and + state storage (**SS**), needed for state machine and clients. +* Reduce layers of abstractions necessary between the RMS and underlying stores. +* Provide atomic module store commitments by providing a batch database object + to core IAVL APIs. +* Reduce complexities in the `CacheKVStore` implementation while also improving + performance\[3]. + +Furthermore, we will keep the IAVL is the backing [commitment](https://cryptography.fandom.com/wiki/Commitment_scheme) +store for the time being. While we might not fully settle on the use of IAVL in +the long term, we do not have strong empirical evidence to suggest a better +alternative. Given that the SDK provides interfaces for stores, it should be sufficient +to change the backing commitment store in the future should evidence arise to +warrant a better alternative. However there is promising work being done to IAVL +that should result in significant performance improvement \[1,2]. + +### Separating SS and SC + +By separating SS and SC, it will allow for us to optimize against primary use cases +and access patterns to state. Specifically, The SS layer will be responsible for +direct access to data in the form of (key, value) pairs, whereas the SC layer (IAVL) +will be responsible for committing to data and providing Merkle proofs. + +Note, the underlying physical storage database will be the same between both the +SS and SC layers. So to avoid collisions between (key, value) pairs, both layers +will be namespaced. + +#### State Commitment (SC) + +Given that the existing solution today acts as both SS and SC, we can simply +repurpose it to act solely as the SC layer without any significant changes to +access patterns or behavior. In other words, the entire collection of existing +IAVL-backed module `KVStore`s will act as the SC layer. + +However, in order for the SC layer to remain lightweight and not duplicate a +majority of the data held in the SS layer, we encourage node operators to keep +tight pruning strategies. + +#### State Storage (SS) + +In the RMS, we will expose a *single* `KVStore` backed by the same physical +database that backs the SC layer. This `KVStore` will be explicitly namespaced +to avoid collisions and will act as the primary storage for (key, value) pairs. + +While we most likely will continue the use of `cosmos-db`, or some local interface, +to allow for flexibility and iteration over preferred physical storage backends +as research and benchmarking continues. However, we propose to hardcode the use +of RocksDB as the primary physical storage backend. + +Since the SS layer will be implemented as a `KVStore`, it will support the +following functionality: + +* Range queries +* CRUD operations +* Historical queries and versioning +* Pruning + +The RMS will keep track of all buffered writes using a dedicated and internal +`MemoryListener` for each `StoreKey`. For each block height, upon `Commit`, the +SS layer will write all buffered (key, value) pairs under a [RocksDB user-defined timestamp](https://github.com/facebook/rocksdb/wiki/User-defined-Timestamp-%28Experimental%29) column +family using the block height as the timestamp, which is an unsigned integer. +This will allow a client to fetch (key, value) pairs at historical and current +heights along with making iteration and range queries relatively performant as +the timestamp is the key suffix. + +Note, we choose not to use a more general approach of allowing any embedded key/value +database, such as LevelDB or PebbleDB, using height key-prefixed keys to +effectively version state because most of these databases use variable length +keys which would effectively make actions likes iteration and range queries less +performant. + +Since operators might want pruning strategies to differ in SS compared to SC, +e.g. having a very tight pruning strategy in SC while having a looser pruning +strategy for SS, we propose to introduce an additional pruning configuration, +with parameters that are identical to what exists in the SDK today, and allow +operators to control the pruning strategy of the SS layer independently of the +SC layer. + +Note, the SC pruning strategy must be congruent with the operator's state sync +configuration. This is so as to allow state sync snapshots to execute successfully, +otherwise, a snapshot could be triggered on a height that is not available in SC. + +#### State Sync + +The state sync process should be largely unaffected by the separation of the SC +and SS layers. However, if a node syncs via state sync, the SS layer of the node +will not have the state synced height available, since the IAVL import process is +not setup in way to easily allow direct key/value insertion. A modification of +the IAVL import process would be necessary to facilitate having the state sync +height available. + +Note, this is not problematic for the state machine itself because when a query +is made, the RMS will automatically direct the query correctly (see [Queries](#queries)). + +#### Queries + +To consolidate the query routing between both the SC and SS layers, we propose to +have a notion of a "query router" that is constructed in the RMS. This query router +will be supplied to each `KVStore` implementation. The query router will route +queries to either the SC layer or the SS layer based on a few parameters. If +`prove: true`, then the query must be routed to the SC layer. Otherwise, if the +query height is available in the SS layer, the query will be served from the SS +layer. Otherwise, we fall back on the SC layer. + +If no height is provided, the SS layer will assume the latest height. The SS +layer will store a reverse index to lookup `LatestVersion -> timestamp(version)` +which is set on `Commit`. + +#### Proofs + +Since the SS layer is naturally a storage layer only, without any commitments +to (key, value) pairs, it cannot provide Merkle proofs to clients during queries. + +Since the pruning strategy against the SC layer is configured by the operator, +we can therefore have the RMS route the query SC layer if the version exists and +`prove: true`. Otherwise, the query will fall back to the SS layer without a proof. + +We could explore the idea of using state snapshots to rebuild an in-memory IAVL +tree in real time against a version closest to the one provided in the query. +However, it is not clear what the performance implications will be of this approach. + +### Atomic Commitment + +We propose to modify the existing IAVL APIs to accept a batch DB object instead +of relying on an internal batch object in `nodeDB`. Since each underlying IAVL +`KVStore` shares the same DB in the SC layer, this will allow commits to be +atomic. + +Specifically, we propose to: + +* Remove the `dbm.Batch` field from `nodeDB` +* Update the `SaveVersion` method of the `MutableTree` IAVL type to accept a batch object +* Update the `Commit` method of the `CommitKVStore` interface to accept a batch object +* Create a batch object in the RMS during `Commit` and pass this object to each + `KVStore` +* Write the database batch after all stores have committed successfully + +Note, this will require IAVL to be updated to not rely or assume on any batch +being present during `SaveVersion`. + +## Consequences + +As a result of a new store V2 package, we should expect to see improved performance +for queries and transactions due to the separation of concerns. We should also +expect to see improved developer UX around experimentation of commitment schemes +and storage backends for further performance, in addition to a reduced amount of +abstraction around KVStores making operations such as caching and state branching +more intuitive. + +However, due to the proposed design, there are drawbacks around providing state +proofs for historical queries. + +### Backwards Compatibility + +This ADR proposes changes to the storage implementation in the Cosmos SDK through +an entirely new package. Interfaces may be borrowed and extended from existing +types that exist in `store`, but no existing implementations or interfaces will +be broken or modified. + +### Positive + +* Improved performance of independent SS and SC layers +* Reduced layers of abstraction making storage primitives easier to understand +* Atomic commitments for SC +* Redesign of storage types and interfaces will allow for greater experimentation + such as different physical storage backends and different commitment schemes + for different application modules + +### Negative + +* Providing proofs for historical state is challenging + +### Neutral + +* Keeping IAVL as the primary commitment data structure, although drastic + performance improvements are being made + +## Further Discussions + +### Module Storage Control + +Many modules store secondary indexes that are typically solely used to support +client queries, but are actually not needed for the state machine's state +transitions. What this means is that these indexes technically have no reason to +exist in the SC layer at all, as they take up unnecessary space. It is worth +exploring what an API would look like to allow modules to indicate what (key, value) +pairs they want to be persisted in the SC layer, implicitly indicating the SS +layer as well, as opposed to just persisting the (key, value) pair only in the +SS layer. + +### Historical State Proofs + +It is not clear what the importance or demand is within the community of providing +commitment proofs for historical state. While solutions can be devised such as +rebuilding trees on the fly based on state snapshots, it is not clear what the +performance implications are for such solutions. + +### Physical DB Backends + +This ADR proposes usage of RocksDB to utilize user-defined timestamps as a +versioning mechanism. However, other physical DB backends are available that may +offer alternative ways to implement versioning while also providing performance +improvements over RocksDB. E.g. PebbleDB supports MVCC timestamps as well, but +we'll need to explore how PebbleDB handles compaction and state growth over time. + +## References + +* \[1] [Link](https://github.com/cosmos/iavl/pull/676) +* \[2] [Link](https://github.com/cosmos/iavl/pull/664) +* \[3] [Link](https://github.com/cosmos/cosmos-sdk/issues/14990) diff --git a/sdk/v0.54/reference/architecture/adr-068-preblock.mdx b/sdk/v0.54/reference/architecture/adr-068-preblock.mdx new file mode 100644 index 000000000..627a8d6e4 --- /dev/null +++ b/sdk/v0.54/reference/architecture/adr-068-preblock.mdx @@ -0,0 +1,67 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/architecture/adr-068-preblock' +title: 'ADR 068: Preblock' +description: 'Sept 13, 2023: Initial Draft' +--- + +## Changelog + +* Sept 13, 2023: Initial Draft + +## Status + +DRAFT + +## Abstract + +Introduce `PreBlock`, which runs before begin blocker other modules, and allows to modify consensus parameters, and the changes are visible to the following state machine logics. + +## Context + +When upgrading to sdk 0.47, the storage format for consensus parameters changed, but in the migration block, `ctx.ConsensusParams()` is always `nil`, because it fails to load the old format using new code, it's supposed to be migrated by the `x/upgrade` module first, but unfortunately, the migration happens in `BeginBlocker` handler, which runs after the `ctx` is initialized. +When we try to solve this, we find the `x/upgrade` module can't modify the context to make the consensus parameters visible for the other modules, the context is passed by value, and sdk team want to keep it that way, that's good for isolations between modules. + +## Alternatives + +The first alternative solution introduced a `MigrateModuleManager`, which only includes the `x/upgrade` module right now, and baseapp will run their `BeginBlocker`s before the other modules, and reload context's consensus parameters in between. + +## Decision + +Suggested this new lifecycle method. + +### `PreBlocker` + +There are two semantics around the new lifecycle method: + +* It runs before the `BeginBlocker` of all modules +* It can modify consensus parameters in storage, and signal the caller through the return value. + +When it returns `ConsensusParamsChanged=true`, the caller must refresh the consensus parameter in the finalize context: + +``` +app.finalizeBlockState.ctx = app.finalizeBlockState.ctx.WithConsensusParams(app.GetConsensusParams()) +``` + +The new ctx must be passed to all the other lifecycle methods. + +## Consequences + +### Backwards Compatibility + +### Positive + +### Negative + +### Neutral + +## Further Discussions + +## Test Cases + +## References + +* \[1] [Link](https://github.com/cosmos/cosmos-sdk/issues/16494) +* \[2] [Link](https://github.com/cosmos/cosmos-sdk/pull/16583) +* \[3] [Link](https://github.com/cosmos/cosmos-sdk/pull/17421) +* \[4] [Link](https://github.com/cosmos/cosmos-sdk/pull/17713) diff --git a/sdk/v0.54/reference/architecture/adr-070-unordered-account.mdx b/sdk/v0.54/reference/architecture/adr-070-unordered-account.mdx new file mode 100644 index 000000000..209938cab --- /dev/null +++ b/sdk/v0.54/reference/architecture/adr-070-unordered-account.mdx @@ -0,0 +1,353 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/architecture/adr-070-unordered-account' +title: 'ADR 070: Unordered Transactions' +--- + +## Changelog + +* Dec 4, 2023: Initial Draft (@yihuang, @tac0turtle, @alexanderbez) +* Jan 30, 2024: Include section on deterministic transaction encoding +* Mar 18, 2025: Revise implementation to use Cosmos SDK KV Store and require unique timeouts per-address (@technicallyty) +* Apr 25, 2025: Add note about rejecting unordered txs with sequence values. + +## Status + +ACCEPTED Not Implemented + +## Abstract + +We propose a way to do replay-attack protection without enforcing the order of +transactions and without requiring the use of monotonically increasing sequences. Instead, we propose +the use of a time-based, ephemeral sequence. + +## Context + +Account sequence values serve to prevent replay attacks and ensure transactions from the same sender are included into blocks and executed +in sequential order. Unfortunately, this makes it difficult to reliably send many concurrent transactions from the +same sender. Victims of such limitations include IBC relayers and crypto exchanges. + +## Decision + +We propose adding a boolean field `unordered` and a google.protobuf.Timestamp field `timeout_timestamp` to the transaction body. + +Unordered transactions will bypass the traditional account sequence rules and follow the rules described +below, without impacting traditional ordered transactions which will follow the same sequence rules as before. + +We will introduce new storage of time-based, ephemeral unordered sequences using the SDK's existing KV Store library. +Specifically, we will leverage the existing x/auth KV store to store the unordered sequences. + +When an unordered transaction is included in a block, a concatenation of the `timeout_timestamp` and sender’s address bytes +will be recorded to state (i.e. `542939323/`). In cases of multi-party signing, one entry per signer +will be recorded to state. + +New transactions will be checked against the state to prevent duplicate submissions. To prevent the state from growing indefinitely, we propose the following: + +* Define an upper bound for the value of `timeout_timestamp` (i.e. 10 minutes). +* Add PreBlocker method x/auth that removes state entries with a `timeout_timestamp` earlier than the current block time. + +### Transaction Format + +```protobuf +message TxBody { + ... + + bool unordered = 4; + google.protobuf.Timestamp timeout_timestamp = 5 +} +``` + +### Replay Protection + +We facilitate replay protection by storing the unordered sequence in the Cosmos SDK KV store. Upon transaction ingress, we check if the transaction's unordered +sequence exists in state, or if the TTL value is stale, i.e. before the current block time. If so, we reject it. Otherwise, +we add the unordered sequence to the state. This section of the state will belong to the `x/auth` module. + +The state is evaluated during x/auth's `PreBlocker`. All transactions with an unordered sequence earlier than the current block time +will be deleted. + +```go +func (am AppModule) + +PreBlock(ctx context.Context) (appmodule.ResponsePreBlock, error) { + err := am.accountKeeper.RemoveExpired(sdk.UnwrapSDKContext(ctx)) + if err != nil { + return nil, err +} + +return &sdk.ResponsePreBlock{ + ConsensusParamsChanged: false +}, nil +} +``` + +```golang expandable +package keeper + +import ( + + sdk "github.com/cosmos/cosmos-sdk/types" + "cosmossdk.io/collections" + "cosmossdk.io/core/store" +) + +var ( + // just arbitrarily picking some upper bound number. + unorderedSequencePrefix = collections.NewPrefix(90) +) + +type AccountKeeper struct { + // ... + unorderedSequences collections.KeySet[collections.Pair[uint64, []byte]] +} + +func (m *AccountKeeper) + +Contains(ctx sdk.Context, sender []byte, timestamp uint64) (bool, error) { + return m.unorderedSequences.Has(ctx, collections.Join(timestamp, sender)) +} + +func (m *AccountKeeper) + +Add(ctx sdk.Context, sender []byte, timestamp uint64) + +error { + return m.unorderedSequences.Set(ctx, collections.Join(timestamp, sender)) +} + +func (m *AccountKeeper) + +RemoveExpired(ctx sdk.Context) + +error { + blkTime := ctx.BlockTime().UnixNano() + +it, err := m.unorderedSequences.Iterate(ctx, collections.NewPrefixUntilPairRange[uint64, []byte](uint64(blkTime))) + if err != nil { + return err +} + +defer it.Close() + +keys, err := it.Keys() + if err != nil { + return err +} + for _, key := range keys { + if err := m.unorderedSequences.Remove(ctx, key); err != nil { + return err +} + +} + +return nil +} +``` + +### AnteHandler Decorator + +To facilitate bypassing nonce verification, we must modify the existing +`IncrementSequenceDecorator` AnteHandler decorator to skip the nonce verification +when the transaction is marked as unordered. + +```golang +func (isd IncrementSequenceDecorator) + +AnteHandle(ctx sdk.Context, tx sdk.Tx, simulate bool, next sdk.AnteHandler) (sdk.Context, error) { + if tx.UnOrdered() { + return next(ctx, tx, simulate) +} + + // ... +} +``` + +We also introduce a new decorator to perform the unordered transaction verification. + +```golang expandable +package ante + +import ( + + "slices" + "strings" + "time" + + sdk "github.com/cosmos/cosmos-sdk/types" + sdkerrors "github.com/cosmos/cosmos-sdk/types/errors" + authkeeper "github.com/cosmos/cosmos-sdk/x/auth/keeper" + authsigning "github.com/cosmos/cosmos-sdk/x/auth/signing" + + errorsmod "cosmossdk.io/errors" +) + +var _ sdk.AnteDecorator = (*UnorderedTxDecorator)(nil) + +// UnorderedTxDecorator defines an AnteHandler decorator that is responsible for +// checking if a transaction is intended to be unordered and, if so, evaluates +// the transaction accordingly. An unordered transaction will bypass having its +// nonce incremented, which allows fire-and-forget transaction broadcasting, +// removing the necessity of ordering on the sender-side. +// +// The transaction sender must ensure that unordered=true and a timeout_height +// is appropriately set. The AnteHandler will check that the transaction is not +// a duplicate and will evict it from state when the timeout is reached. +// +// The UnorderedTxDecorator should be placed as early as possible in the AnteHandler +// chain to ensure that during DeliverTx, the transaction is added to the unordered sequence state. +type UnorderedTxDecorator struct { + // maxUnOrderedTTL defines the maximum TTL a transaction can define. + maxTimeoutDuration time.Duration + txManager authkeeper.UnorderedTxManager +} + +func NewUnorderedTxDecorator( + utxm authkeeper.UnorderedTxManager, +) *UnorderedTxDecorator { + return &UnorderedTxDecorator{ + maxTimeoutDuration: 10 * time.Minute, + txManager: utxm, +} +} + +func (d *UnorderedTxDecorator) + +AnteHandle( + ctx sdk.Context, + tx sdk.Tx, + _ bool, + next sdk.AnteHandler, +) (sdk.Context, error) { + if err := d.ValidateTx(ctx, tx); err != nil { + return ctx, err +} + +return next(ctx, tx, false) +} + +func (d *UnorderedTxDecorator) + +ValidateTx(ctx sdk.Context, tx sdk.Tx) + +error { + unorderedTx, ok := tx.(sdk.TxWithUnordered) + if !ok || !unorderedTx.GetUnordered() { + // If the transaction does not implement unordered capabilities or has the + // unordered value as false, we bypass. + return nil +} + blockTime := ctx.BlockTime() + timeoutTimestamp := unorderedTx.GetTimeoutTimeStamp() + if timeoutTimestamp.IsZero() || timeoutTimestamp.Unix() == 0 { + return errorsmod.Wrap( + sdkerrors.ErrInvalidRequest, + "unordered transaction must have timeout_timestamp set", + ) +} + if timeoutTimestamp.Before(blockTime) { + return errorsmod.Wrap( + sdkerrors.ErrInvalidRequest, + "unordered transaction has a timeout_timestamp that has already passed", + ) +} + if timeoutTimestamp.After(blockTime.Add(d.maxTimeoutDuration)) { + return errorsmod.Wrapf( + sdkerrors.ErrInvalidRequest, + "unordered tx ttl exceeds %s", + d.maxTimeoutDuration.String(), + ) +} + execMode := ctx.ExecMode() + if execMode == sdk.ExecModeSimulate { + return nil +} + +signerAddrs, err := getSigners(tx) + if err != nil { + return err +} + for _, signer := range signerAddrs { + contains, err := d.txManager.Contains(ctx, signer, uint64(unorderedTx.GetTimeoutTimeStamp().Unix())) + if err != nil { + return errorsmod.Wrap( + sdkerrors.ErrIO, + "failed to check contains", + ) +} + if contains { + return errorsmod.Wrapf( + sdkerrors.ErrInvalidRequest, + "tx is duplicated for signer %x", signer, + ) +} + if err := d.txManager.Add(ctx, signer, uint64(unorderedTx.GetTimeoutTimeStamp().Unix())); err != nil { + return errorsmod.Wrap( + sdkerrors.ErrIO, + "failed to add unordered sequence to state", + ) +} + +} + +return nil +} + +func getSigners(tx sdk.Tx) ([][]byte, error) { + sigTx, ok := tx.(authsigning.SigVerifiableTx) + if !ok { + return nil, errorsmod.Wrap(sdkerrors.ErrTxDecode, "invalid tx type") +} + +return sigTx.GetSigners() +} +``` + +### Unordered Sequences + +Unordered sequences provide a simple, straightforward mechanism to protect against both transaction malleability and +transaction duplication. It is important to note that the unordered sequence must still be unique. However, +the value is not required to be strictly increasing as with regular sequences, and the order in which the node receives +the transactions no longer matters. Clients can handle building unordered transactions similarly to the code below: + +```go +for _, tx := range txs { + tx.SetUnordered(true) + +tx.SetTimeoutTimestamp(time.Now() + 1 * time.Nanosecond) +} +``` + +We will reject transactions that have both sequence and unordered timeouts set. We do this to avoid assuming the intent of the user. + +### State Management + +The storage of unordered sequences will be facilitated using the Cosmos SDK's KV Store service. + +## Note On Previous Design Iteration + +The previous iteration of unordered transactions worked by using an ad-hoc state-management system that posed severe +risks and a vector for duplicated tx processing. It relied on graceful app closure which would flush the current state +of the unordered sequence mapping. If the 2/3's of the network crashed, and the graceful closure did not trigger, +the system would lose track of all sequences in the mapping, allowing those transactions to be replayed. The +implementation proposed in the updated version of this ADR solves this by writing directly to the Cosmos KV Store. +While this is less performant, for the initial implementation, we opted to choose a safer path and postpone performance optimizations until we have more data on real-world impacts and a more battle-tested approach to optimization. + +Additionally, the previous iteration relied on using hashes to create what we call an "unordered sequence." There are known +issues with transaction malleability in Cosmos SDK signing modes. This ADR gets away from this problem by enforcing +single-use unordered nonces, instead of deriving nonces from bytes in the transaction. + +## Consequences + +### Positive + +* Support unordered transaction inclusion, enabling the ability to "fire and forget" many transactions at once. + +### Negative + +* Requires additional storage overhead. +* Requirement of unique timestamps per transaction causes a small amount of additional overhead for clients. Clients must ensure each transaction's timeout timestamp is different. However, nanosecond differentials suffice. +* Usage of Cosmos SDK KV store is slower in comparison to using a non-merklized store or ad-hoc methods, and block times may slow down as a result. + +## References + +* [Link](https://github.com/cosmos/cosmos-sdk/issues/13009) diff --git a/sdk/v0.54/reference/architecture/adr-076-tx-malleability.mdx b/sdk/v0.54/reference/architecture/adr-076-tx-malleability.mdx new file mode 100644 index 000000000..df350da6d --- /dev/null +++ b/sdk/v0.54/reference/architecture/adr-076-tx-malleability.mdx @@ -0,0 +1,175 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/architecture/adr-076-tx-malleability' +title: Cosmos SDK Transaction Malleability Risk Review and Recommendations +description: '2025-03-10: Initial draft (@aaronc)' +--- + +## Changelog + +* 2025-03-10: Initial draft (@aaronc) + +## Status + +PROPOSED: Not Implemented + +## Abstract + +Several encoding and sign mode related issues have historically resulted in the possibility +that Cosmos SDK transactions may be re-encoded in such a way as to change their hash +(and in rare cases, their meaning) without invalidating the signature. +This document details these cases, their potential risks, the extent to which they have been +addressed, and provides recommendations for future improvements. + +## Review + +One naive assumption about Cosmos SDK transactions is that hashing the raw bytes of a submitted transaction creates a safe unique identifier for the transaction. In reality, there are multiple ways in which transactions could be manipulated to create different transaction bytes (and as a result different hashes) that still pass signature verification. + +This document attempts to enumerate the various potential transaction "malleability" risks that we have identified and the extent to which they have or have not been addressed in various sign modes. We also identify vulnerabilities that could be introduced if developers make changes in the future without careful consideration of the complexities involved with transaction encoding, sign modes and signatures. + +### Risks Associated with Malleability + +The malleability of transactions poses the following potential risks to end users: + +* unsigned data could get added to transactions and be processed by state machines +* clients often rely on transaction hashes for checking transaction status, but whether or not submitted transaction hashes match processed transaction hashes depends primarily on good network actors rather than fundamental protocol guarantees +* transactions could potentially get executed more than once (faulty replay protection) + +If a client generates a transaction, keeps a record of its hash and then attempts to query nodes to check the transaction's status, this process may falsely conclude that the transaction had not been processed if an intermediary +processor decoded and re-encoded the transaction with different encoding rules (either maliciously or unintentionally). +As long as no malleability is present in the signature bytes themselves, clients *should* query transactions by signature instead of hash. + +Not being cognizant of this risk may lead clients to submit the same transaction multiple times if they believe that +earlier transactions had failed or gotten lost in processing. +This could be an attack vector against users if wallets primarily query transactions by hash. + +If the state machine were to rely on transaction hashes as a replay mechanism itself, this would be faulty and not +provide the intended replay protection. Instead, the state machine should rely on deterministic representations of +transactions rather than the raw encoding, or other nonces, +if they want to provide some replay protection that doesn't rely on a monotonically +increasing account sequence number. + +### Sources of Malleability + +#### Non-deterministic Protobuf Encoding + +Cosmos SDK transactions are encoded using protobuf binary encoding when they are submitted to the network. Protobuf binary is not inherently a deterministic encoding meaning that the same logical payload could have several valid bytes representations. In a basic sense, this means that protobuf in general can be decoded and re-encoded to produce a different byte stream (and thus different hash) without changing the logical meaning of the bytes. [ADR 027: Deterministic Protobuf Serialization](https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-027-deterministic-protobuf-serialization.md) describes in detail what needs to be done to produce what we consider to be a "canonical", deterministic protobuf serialization. Briefly, the following sources of malleability at the encoding level have been identified and are addressed by this specification: + +* fields can be emitted in any order +* default field values can be included or omitted, and this doesn't change meaning unless `optional` is used +* `repeated` fields of scalars may use packed or "regular" encoding +* `varint`s can include extra ignored bits +* extra fields may be added and are usually simply ignored by decoders. [ADR 020](https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-020-protobuf-transaction-encoding.md#unknown-field-filtering) specifies that in general such extra fields should cause messages and transactions to be rejected) + +When using `SIGN_MODE_DIRECT` none of the above malleabilities will be tolerated because: + +* signatures of messages and extensions must be done over the raw encoded bytes of those fields +* the outer tx envelope (`TxRaw`) must follow ADR 027 rules or be rejected + +Transactions signed with `SIGN_MODE_LEGACY_AMINO_JSON`, however, have no way of protecting against the above malleabilities because what is signed is a JSON representation of the logical contents of the transaction. These logical contents could have any number of valid protobuf binary encodings, so in general there are no guarantees regarding transaction hash with Amino JSON signing. + +In addition to being aware of the general non-determinism of protobuf binary, developers need to pay special attention to make sure that unknown protobuf fields get rejected when developing new capabilities related to protobuf transactions. The protobuf serialization format was designed with the assumption that unknown data known to encoders could safely be ignored by decoders. This assumption may have been fairly safe within the walled garden of Google's centralized infrastructure. However, in distributed blockchain systems, this assumption is generally unsafe. If a newer client encodes a protobuf message with data intended for a newer server, it is not safe for an older server to simply ignore and discard instructions that it does not understand. These instructions could include critical information that the transaction signer is relying upon and just assuming that it is unimportant is not safe. + +[ADR 020](https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-020-protobuf-transaction-encoding.md#unknown-field-filtering) specifies some provisions for "non-critical" fields which can safely be ignored by older servers. In practice, I have not seen any valid usages of this. It is something in the design that maintainers should be aware of, but it may not be necessary or even 100% safe. + +#### Non-deterministic Value Encoding + +In addition to the non-determinism present in protobuf binary itself, some protobuf field data is encoded using a micro-format which itself may not be deterministic. Consider for instance integer or decimal encoding. Some decoders may allow for the presence of leading or trailing zeros without changing the logical meaning, ex. `00100` vs `100` or `100.00` vs `100`. So if a sign mode encodes numbers deterministically, but decoders accept multiple representations, +a user may sign over the value `100` while `0100` gets encoded. This would be possible with Amino JSON to the extent that the integer decoder accepts leading zeros. I believe the current `Int` implementation will reject this, however, it is +probably possible to encode a octal or hexadecimal representation in the transaction whereas the user signs over a decimal integer. + +#### Signature Encoding + +Signatures themselves are encoded using a micro-format specific to the signature algorithm being used and sometimes these +micro-formats can allow for non-determinism (multiple valid bytes for the same signature). +Most of the signature algorithms supported by the SDK should reject non-canonical bytes in their current implementation. +However, the `Multisignature` protobuf type uses normal protobuf encoding and there is no check as to whether the +decoded bytes followed canonical ADR 027 rules or not. Therefore, multisig transactions can have malleability in +their signatures. +Any new or custom signature algorithms must make sure that they reject any non-canonical bytes, otherwise even +with `SIGN_MODE_DIRECT` there can be transaction hash malleability by re-encoding signatures with a non-canonical +representation. + +#### Fields not covered by Amino JSON + +Another area that needs to be addressed carefully is the discrepancy between `AminoSignDoc`(see [`aminojson.proto`](https://github.com/cosmos/cosmos-sdk/blob/v0.50.10/x/tx/signing/aminojson/internal/aminojsonpb/aminojson.proto)) used for `SIGN_MODE_LEGACY_AMINO_JSON` and the actual contents of `TxBody` and `AuthInfo` (see [`tx.proto`](https://github.com/cosmos/cosmos-sdk/blob/v0.50.10/proto/cosmos/tx/v1beta1/tx.proto)). +If fields get added to `TxBody` or `AuthInfo`, they must either have a corresponding representing in `AminoSignDoc` or Amino JSON signatures must be rejected when those new fields are set. Making sure that this is done is a +highly manual process, and developers could easily make the mistake of updating `TxBody` or `AuthInfo` +without paying any attention to the implementation of `GetSignBytes` for Amino JSON. This is a critical +vulnerability in which unsigned content can now get into the transaction and signature verification will +pass. + +## Sign Mode Summary and Recommendations + +The sign modes officially supported by the SDK are `SIGN_MODE_DIRECT`, `SIGN_MODE_TEXTUAL`, `SIGN_MODE_DIRECT_AUX`, +and `SIGN_MODE_LEGACY_AMINO_JSON`. +`SIGN_MODE_LEGACY_AMINO_JSON` is used commonly by wallets and is currently the only sign mode supported on Nano Ledger hardware devices +(although `SIGN_MODE_TEXTUAL` was designed to also support hardware devices). +`SIGN_MODE_DIRECT` is the simplest sign mode and its usage is also fairly common. +`SIGN_MODE_DIRECT_AUX` is a variant of `SIGN_MODE_DIRECT` that can be used by auxiliary signers in a multi-signer +transaction by those signers who are not paying gas. +`SIGN_MODE_TEXTUAL` was intended as a replacement for `SIGN_MODE_LEGACY_AMINO_JSON`, but as far as we know it +has not been adopted by any clients yet and thus is not in active use. + +All known malleability concerns have been addressed in the current implementation of `SIGN_MODE_DIRECT`. +The only known malleability that could occur with a transaction signed with `SIGN_MODE_DIRECT` would +need to be in the signature bytes themselves. +Since signatures are not signed over, it is impossible for any sign mode to address this directly +and instead signature algorithms need to take care to reject any non-canonically encoded signature bytes +to prevent malleability. +For the known malleability of the `Multisignature` type, we should make sure that any valid signatures +were encoded following canonical ADR 027 rules when doing signature verification. + +`SIGN_MODE_DIRECT_AUX` provides the same level of safety as `SIGN_MODE_DIRECT` because + +* the raw encoded `TxBody` bytes are signed over in `SignDocDirectAux`, and +* a transaction using `SIGN_MODE_DIRECT_AUX` still requires the primary signer to sign the transaction with `SIGN_MODE_DIRECT` + +`SIGN_MODE_TEXTUAL` also provides the same level of safety as `SIGN_MODE_DIRECT` because the hash of the raw encoded +`TxBody` and `AuthInfo` bytes are signed over. + +Unfortunately, the vast majority of unaddressed malleability risks affect `SIGN_MODE_LEGACY_AMINO_JSON` and this +sign mode is still commonly used. +It is recommended that the following improvements be made to Amino JSON signing: + +* hashes of `TxBody` and `AuthInfo` should be added to `AminoSignDoc` so that encoding-level malleablity is addressed +* when constructing `AminoSignDoc`, [protoreflect](https://pkg.go.dev/google.golang.org/protobuf/reflect/protoreflect) API should be used to ensure that there no fields in `TxBody` or `AuthInfo` which do not have a mapping in `AminoSignDoc` have been set +* fields present in `TxBody` or `AuthInfo` that are not present in `AminoSignDoc` (such as extension options) should + be added to `AminoSignDoc` if possible + +## Testing + +To test that transactions are resistant to malleability, +we can develop a test suite to run against all sign modes that +attempts to manipulate transaction bytes in the following ways: + +* changing protobuf encoding by + * reordering fields + * setting default values + * adding extra bits to varints, or + * setting new unknown fields +* modifying integer and decimal values encoded as strings with leading or trailing zeros + +Whenever any of these manipulations is done, we should observe that the sign doc bytes for the sign mode being +tested also change, meaning that the corresponding signatures will also have to change. + +In the case of Amino JSON, we should also develop tests which ensure that if any `TxBody` or `AuthInfo` +field not supported by Amino's `AminoSignDoc` is set that signing fails. + +In the general case of transaction decoding, we should have unit tests to ensure that + +* any `TxRaw` bytes which do not follow ADR 027 canonical encoding cause decoding to fail, and +* any top-level transaction elements including `TxBody`, `AuthInfo`, public keys, and messages which + have unknown fields set cause the transaction to be rejected + (this ensures that ADR 020 unknown field filtering is properly applied) + +For each supported signature algorithm, +there should also be unit tests to ensure that signatures must be encoded canonically +or get rejected. + +## References + +* [ADR 027: Deterministic Protobuf Serialization](https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-027-deterministic-protobuf-serialization.md) +* [ADR 020](https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-020-protobuf-transaction-encoding.md#unknown-field-filtering) +* [`aminojson.proto`](https://github.com/cosmos/cosmos-sdk/blob/v0.50.10/x/tx/signing/aminojson/internal/aminojsonpb/aminojson.proto) +* [`tx.proto`](https://github.com/cosmos/cosmos-sdk/blob/v0.50.10/proto/cosmos/tx/v1beta1/tx.proto) diff --git a/sdk/v0.54/reference/cosmos-sdk-repo.mdx b/sdk/v0.54/reference/cosmos-sdk-repo.mdx new file mode 100644 index 000000000..2d663bf61 --- /dev/null +++ b/sdk/v0.54/reference/cosmos-sdk-repo.mdx @@ -0,0 +1,6 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/cosmos-sdk-repo' +title: "Cosmos SDK Repository" +url: "https://github.com/cosmos/cosmos-sdk" +--- diff --git a/sdk/v0.54/reference/example-repo.mdx b/sdk/v0.54/reference/example-repo.mdx new file mode 100644 index 000000000..d3896d8df --- /dev/null +++ b/sdk/v0.54/reference/example-repo.mdx @@ -0,0 +1,6 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/example-repo' +title: "Tutorial Example Repository" +url: "https://github.com/cosmos/example" +--- diff --git a/sdk/v0.54/reference/rfc.mdx b/sdk/v0.54/reference/rfc.mdx new file mode 100644 index 000000000..9d64be54a --- /dev/null +++ b/sdk/v0.54/reference/rfc.mdx @@ -0,0 +1,27 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/rfc' +title: "Requests for Comments" +description: "Version: v0.54" +--- + +A Request for Comments (RFC) is a record of discussion on an open-ended topic related to the design and implementation of the Cosmos SDK, for which no immediate decision is required. + +The purpose of an RFC is to serve as a historical record of a high-level discussion that might otherwise only be recorded in an ad-hoc way (for example, via gists or Google docs) that are difficult to discover for someone after the fact. An RFC *may* give rise to more specific architectural *decisions* for the Cosmos SDK, but those decisions must be recorded separately in [Architecture Decision Records (ADR)](/sdk/v0.54/reference/architecture/README). + +As a rule of thumb, if you can articulate a specific question that needs to be answered, write an ADR. If you need to explore the topic and get input from others to know what questions need to be answered, an RFC may be appropriate. + +## RFC Content[​](#rfc-content "Direct link to RFC Content") + +An RFC should provide: + +* A **changelog**, documenting when and how the RFC has changed. +* An **abstract**, briefly summarizing the topic so the reader can quickly tell whether it is relevant to their interest. +* Any **background** a reader will need to understand and participate in the substance of the discussion (links to other documents are fine here). +* The **discussion**, the primary content of the document. + +The `rfc-template.md` file includes placeholders for these sections. + +## Table of Contents[​](#table-of-contents "Direct link to Table of Contents") + +* [RFC-001: Tx Validation](/sdk/v0.54/reference/rfc/rfc-001-tx-validation) diff --git a/sdk/v0.54/reference/rfc/PROCESS.mdx b/sdk/v0.54/reference/rfc/PROCESS.mdx new file mode 100644 index 000000000..a04cba9bf --- /dev/null +++ b/sdk/v0.54/reference/rfc/PROCESS.mdx @@ -0,0 +1,66 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/rfc/PROCESS' +title: RFC Creation Process +--- + +1. Copy the `rfc-template.md` file. Use the following filename pattern: `rfc-next_number-title.md` +2. Create a draft Pull Request if you want to get an early feedback. +3. Make sure the context and a solution is clear and well documented. +4. Add an entry to a list in the [README](/sdk/v0.50/build/rfc/README) file. +5. Create a Pull Request to propose a new ADR. + +## What is an RFC? + +An RFC is a sort of async whiteboarding session. It is meant to replace the need for a distributed team to come together to make a decision. Currently, the Cosmos SDK team and contributors are distributed around the world. The team conducts working groups to have a synchronous discussion and an RFC can be used to capture the discussion for a wider audience to better understand the changes that are coming to the software. + +The main difference the Cosmos SDK is defining as a differentiation between RFC and ADRs is that one is to come to consensus and circulate information about a potential change or feature. An ADR is used if there is already consensus on a feature or change and there is not a need to articulate the change coming to the software. An ADR will articulate the changes and have a lower amount of communication . + +## RFC life cycle + +RFC creation is an **iterative** process. An RFC is meant as a distributed collaboration session, it may have many comments and is usually the byproduct of no working group or synchronous communication + +1. Proposals could start with a new GitHub Issue, be a result of existing Issues or a discussion. + +2. An RFC doesn't have to arrive to `main` with an *accepted* status in a single PR. If the motivation is clear and the solution is sound, we SHOULD be able to merge it and keep a *proposed* status. It's preferable to have an iterative approach rather than long, not merged Pull Requests. + +3. If a *proposed* RFC is merged, then it should clearly document outstanding issues either in the RFC document notes or in a GitHub Issue. + +4. The PR SHOULD always be merged. In the case of a faulty RFC, we still prefer to merge it with a *rejected* status. The only time the RFC SHOULD NOT be merged is if the author abandons it. + +5. Merged RFCs SHOULD NOT be pruned. + +6. If there is consensus and enough feedback then the RFC can be accepted. + +> Note: An RFC is written when there is no working group or team session on the problem. RFC's are meant as a distributed whiteboarding session. If there is a working group on the proposal there is no need to have an RFC as there is synchronous whiteboarding going on. + +### RFC status + +Status has two components: + +```text +{CONSENSUS STATUS} +``` + +#### Consensus Status + +```text +DRAFT -> PROPOSED -> LAST CALL yyyy-mm-dd -> ACCEPTED | REJECTED -> SUPERSEDED by ADR-xxx + \ | + \ | + v v + ABANDONED +``` + +* `DRAFT`: \[optional] an ADR which is work in progress, not being ready for a general review. This is to present an early work and get an early feedback in a Draft Pull Request form. +* `PROPOSED`: an ADR covering a full solution architecture and still in the review - project stakeholders haven't reached agreement yet. +* `LAST CALL `: \[optional] clear notify that we are close to accept updates. Changing a status to `LAST CALL` means that social consensus (of Cosmos SDK maintainers) has been reached and we still want to give it a time to let the community react or analyze. +* `ACCEPTED`: ADR which will represent a currently implemented or to be implemented architecture design. +* `REJECTED`: ADR can go from PROPOSED or ACCEPTED to rejected if the consensus among project stakeholders will decide so. +* `SUPERSEDED by ADR-xxx`: ADR which has been superseded by a new ADR. +* `ABANDONED`: the ADR is no longer pursued by the original authors. + +## Language used in RFC + +* The background/goal should be written in the present tense. +* Avoid using a first, personal form. diff --git a/sdk/v0.54/reference/rfc/README.mdx b/sdk/v0.54/reference/rfc/README.mdx new file mode 100644 index 000000000..fb4d4c26f --- /dev/null +++ b/sdk/v0.54/reference/rfc/README.mdx @@ -0,0 +1,42 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/rfc/README' +title: Requests for Comments +description: >- + A Request for Comments (RFC) is a record of discussion on an open-ended topic + related to the design and implementation of the Cosmos SDK, for which no + immediate decision is required. +--- + +A Request for Comments (RFC) is a record of discussion on an open-ended topic +related to the design and implementation of the Cosmos SDK, for which no +immediate decision is required. + +The purpose of an RFC is to serve as a historical record of a high-level +discussion that might otherwise only be recorded in an ad-hoc way (for example, +via gists or Google docs) that are difficult to discover for someone after the +fact. An RFC *may* give rise to more specific architectural *decisions* for +the Cosmos SDK, but those decisions must be recorded separately in +[Architecture Decision Records (ADR)](/sdk/v0.54/reference/architecture/README). + +As a rule of thumb, if you can articulate a specific question that needs to be +answered, write an ADR. If you need to explore the topic and get input from +others to know what questions need to be answered, an RFC may be appropriate. + +## RFC Content + +An RFC should provide: + +* A **changelog**, documenting when and how the RFC has changed. +* An **abstract**, briefly summarizing the topic so the reader can quickly tell + whether it is relevant to their interest. +* Any **background** a reader will need to understand and participate in the + substance of the discussion (links to other documents are fine here). +* The **discussion**, the primary content of the document. + +The `rfc-template.md` file includes placeholders for these +sections. + +## Table of Contents + +* [RFC-001: Tx Validation](/sdk/v0.50/build/rfc/rfc-001-tx-validation) diff --git a/sdk/v0.54/reference/rfc/rfc-001-tx-validation.mdx b/sdk/v0.54/reference/rfc/rfc-001-tx-validation.mdx new file mode 100644 index 000000000..be26a78bc --- /dev/null +++ b/sdk/v0.54/reference/rfc/rfc-001-tx-validation.mdx @@ -0,0 +1,30 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/rfc/rfc-001-tx-validation' +title: 'RFC 001: Transaction Validation' +description: '2023-03-12: Proposed' +--- + +## Changelog + +* 2023-03-12: Proposed + +## Background + +Transaction Validation is crucial to a functioning state machine. Within the Cosmos SDK there are two validation flows, one is outside the message server and the other within. The flow outside of the message server is the `ValidateBasic` function. It is called in the antehandler on both `CheckTx` and `DeliverTx`. There is an overhead and sometimes duplication of validation within these two flows. This extra validation provides an additional check before entering the mempool. + +With the deprecation of [`GetSigners`](https://github.com/cosmos/cosmos-sdk/issues/11275) we have the optionality to remove [sdk.Msg](https://github.com/cosmos/cosmos-sdk/blob/16a5404f8e00ddcf8857c8a55dca2f7c109c29bc/types/tx_msg.go#L16) and the `ValidateBasic` function. + +With the separation of CometBFT and Cosmos-SDK, there is a lack of control of what transactions get broadcasted and included in a block. This extra validation in the antehandler is meant to help in this case. In most cases the transaction is or should be simulated against a node for validation. With this flow transactions will be treated the same. + +## Proposal + +The acceptance of this RFC would move validation within `ValidateBasic` to the message server in modules, update tutorials and docs to remove mention of using `ValidateBasic` in favour of handling all validation for a message where it is executed. + +We can and will still support the `ValidateBasic` function for users and provide an extension interface of the function once `sdk.Msg` is deprecated. + +> Note: This is how messages are handled in VMs like Ethereum and CosmWasm. + +### Consequences + +The consequence of updating the transaction flow is that transaction that may have failed before with the `ValidateBasic` flow will now be included in a block and fees charged. diff --git a/sdk/v0.54/reference/spec.mdx b/sdk/v0.54/reference/spec.mdx new file mode 100644 index 000000000..dc85af7a6 --- /dev/null +++ b/sdk/v0.54/reference/spec.mdx @@ -0,0 +1,23 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/spec' +title: "Specifications" +description: "Version: v0.54" +--- + +This directory contains specifications for the modules of the Cosmos SDK as well as Interchain Standards (ICS) and other specifications. + +Cosmos SDK applications hold this state in a Merkle store. Updates to the store may be made during transactions and at the beginning and end of every block. + +## Cosmos SDK specifications[​](#cosmos-sdk-specifications "Direct link to Cosmos SDK specifications") + +* [Store](/sdk/v0.54/guides/state/store) - The core Merkle store that holds the state. +* [Bech32](/sdk/v0.54/guides/reference/bech32) - Address format for Cosmos SDK applications. + +## Modules specifications[​](#modules-specifications "Direct link to Modules specifications") + +Go the [module directory](/sdk/v0.54/modules/modules) + +## CometBFT[​](#cometbft "Direct link to CometBFT") + +For details on the underlying blockchain and p2p protocols, see the [CometBFT specification](https://github.com/cometbft/cometbft/tree/main/spec). diff --git a/sdk/v0.54/reference/spec/README.mdx b/sdk/v0.54/reference/spec/README.mdx new file mode 100644 index 000000000..029fe8b94 --- /dev/null +++ b/sdk/v0.54/reference/spec/README.mdx @@ -0,0 +1,28 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/spec/README' +title: Specifications +description: >- + This directory contains specifications for the modules of the Cosmos SDK as + well as Interchain Standards (ICS) and other specifications. +--- + +This directory contains specifications for the modules of the Cosmos SDK as well as Interchain Standards (ICS) and other specifications. + +Cosmos SDK applications hold this state in a Merkle store. Updates to +the store may be made during transactions and at the beginning and end of every +block. + +## Cosmos SDK specifications + +* [Store](/sdk/v0.50/learn/advanced/store) - The core Merkle store that holds the state. +* [Bech32](/sdk/v0.50/build/spec/addresses/bech32) - Address format for Cosmos SDK applications. + +## Modules specifications + +Go the [module directory](/sdk/v0.54/modules/modules) + +## CometBFT + +For details on the underlying blockchain and p2p protocols, see +the [CometBFT specification](https://github.com/cometbft/cometbft/tree/main/spec). diff --git a/sdk/v0.54/reference/spec/SPEC_MODULE.mdx b/sdk/v0.54/reference/spec/SPEC_MODULE.mdx new file mode 100644 index 000000000..98858d7f2 --- /dev/null +++ b/sdk/v0.54/reference/spec/SPEC_MODULE.mdx @@ -0,0 +1,67 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/spec/SPEC_MODULE' +title: Specification of Modules +description: >- + This file intends to outline the common structure for specifications within + this directory. +--- + +This file intends to outline the common structure for specifications within +this directory. + +## Tense + +For consistency, specs should be written in passive present tense. + +## Pseudo-Code + +Generally, pseudo-code should be minimized throughout the spec. Often, simple +bulleted-lists which describe a function's operations are sufficient and should +be considered preferable. In certain instances, due to the complex nature of +the functionality being described pseudo-code may be the most suitable form of +specification. In these cases use of pseudo-code is permissible, but should be +presented in a concise manner, ideally restricted to only the complex +element as a part of a larger description. + +## Common Layout + +The following generalized `README` structure should be used to breakdown +specifications for modules. The following list is nonbinding and all sections are optional. + +* `# {Module Name}` - overview of the module +* `## Concepts` - describe specialized concepts and definitions used throughout the spec +* `## State` - specify and describe structures expected to be marshaled into the store, and their keys +* `## State Transitions` - standard state transition operations triggered by hooks, messages, etc. +* `## Messages` - specify message structure(s) and expected state machine behavior(s) +* `## Begin Block` - specify any begin-block operations +* `## End Block` - specify any end-block operations +* `## Hooks` - describe available hooks to be called by/from this module +* `## Events` - list and describe event tags used +* `## Client` - list and describe CLI commands and gRPC and REST endpoints +* `## Params` - list all module parameters, their types (in JSON) and examples +* `## Future Improvements` - describe future improvements of this module +* `## Tests` - acceptance tests +* `## Appendix` - supplementary details referenced elsewhere within the spec + +### Notation for key-value mapping + +Within `## State` the following notation `->` should be used to describe key to +value mapping: + +```text +key -> value +``` + +to represent byte concatenation the `|` may be used. In addition, encoding +type may be specified, for example: + +```text +0x00 | addressBytes | address2Bytes -> amino(value_object) +``` + +Additionally, index mappings may be specified by mapping to the `nil` value, for example: + +```text +0x01 | address2Bytes | addressBytes -> nil +``` diff --git a/sdk/v0.54/reference/spec/SPEC_STANDARD.mdx b/sdk/v0.54/reference/spec/SPEC_STANDARD.mdx new file mode 100644 index 000000000..3ae8381eb --- /dev/null +++ b/sdk/v0.54/reference/spec/SPEC_STANDARD.mdx @@ -0,0 +1,130 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/spec/SPEC_STANDARD' +title: What is an SDK standard? +--- + +An SDK standard is a design document describing a particular protocol, standard, or feature expected to be used by the Cosmos SDK. An SDK standard should list the desired properties of the standard, explain the design rationale, and provide a concise but comprehensive technical specification. The primary author is responsible for pushing the proposal through the standardization process, soliciting input and support from the community, and communicating with relevant stakeholders to ensure (social) consensus. + +## Sections + +An SDK standard consists of: + +* a synopsis, +* overview and basic concepts, +* technical specification, +* history log, and +* copyright notice. + +All top-level sections are required. References should be included inline as links, or tabulated at the bottom of the section if necessary. Included subsections should be listed in the order specified below. + +### Table Of Contents + +Provide a table of contents at the top of the file to help readers. + +### Synopsis + +The document should include a brief (\~200 word) synopsis providing a high-level description of and rationale for the specification. + +### Overview and basic concepts + +This section should include a motivation subsection and a definition subsection if required: + +* *Motivation* - A rationale for the existence of the proposed feature, or the proposed changes to an existing feature. +* *Definitions* - A list of new terms or concepts used in the document or required to understand it. + +### System model and properties + +This section should include an assumption subsection if any, the mandatory properties subsection, and a dependency subsection. Note that the first two subsections are tightly coupled: how to enforce a property will depend directly on the assumptions made. This subsection is important to capture the interactions of the specified feature with the "rest-of-the-world," i.e., with other features of the ecosystem. + +* *Assumptions* - A list of any assumptions made by the feature designer. It should capture which features are used by the feature under specification, and what do we expect from them. +* *Properties* - A list of the desired properties or characteristics of the feature specified, and expected effects or failures when the properties are violated. In case it is relevant, it can also include a list of properties that the feature does not guarantee. +* *Dependencies* - A list of the features that use the feature under specification and how. + +### Technical specification + +This is the main section of the document, and should contain protocol documentation, design rationale, required references, and technical details where appropriate. +The section may have any or all of the following subsections, as appropriate to the particular specification. The API subsection is especially encouraged when appropriate. + +* *API* - A detailed description of the feature's API. +* *Technical Details* - All technical details including syntax, diagrams, semantics, protocols, data structures, algorithms, and pseudocode as appropriate. The technical specification should be detailed enough such that separate correct implementations of the specification without knowledge of each other are compatible. +* *Backwards Compatibility* - A discussion of compatibility (or lack thereof) with previous feature or protocol versions. +* *Known Issues* - A list of known issues. This subsection is specially important for specifications of already in-use features. +* *Example Implementation* - A concrete example implementation or description of an expected implementation to serve as the primary reference for implementers. + +### History + +A specification should include a history section, listing any inspiring documents and a plaintext log of significant changes. + +See an example history section [below](#history-1). + +### Copyright + +A specification should include a copyright section waiving rights via [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0). + +## Formatting + +### General + +Specifications must be written in GitHub-flavored Markdown. + +For a GitHub-flavored Markdown cheat sheet, see [here](https://github.com/adam-p/markdown-here/wiki/Markdown-Cheatsheet). For a local Markdown renderer, see [here](https://github.com/joeyespo/grip). + +### Language + +Specifications should be written in Simple English, avoiding obscure terminology and unnecessary jargon. For excellent examples of Simple English, please see the [Simple English Wikipedia](https://simple.wikipedia.org/wiki/Main_Page). + +The keywords "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in specifications are to be interpreted as described in [RFC 2119](https://tools.ietf.org/html/rfc2119). + +### Pseudocode + +Pseudocode in specifications should be language-agnostic and formatted in a simple imperative standard, with line numbers, variables, simple conditional blocks, for loops, and +English fragments where necessary to explain further functionality such as scheduling timeouts. LaTeX images should be avoided because they are challenging to review in diff form. + +Pseudocode for structs can be written in a simple language like TypeScript or golang, as interfaces. + +Example Golang pseudocode struct: + +```go +type CacheKVStore interface { + cache: map[Key]Value + parent: KVStore + deleted: Key +} +``` + +Pseudocode for algorithms should be written in simple Golang, as functions. + +Example pseudocode algorithm: + +```go expandable +func get( + store CacheKVStore, + key Key) + +Value { + value = store.cache.get(Key) + if (value !== null) { + return value +} + +else { + value = store.parent.get(key) + +store.cache.set(key, value) + +return value +} +} +``` + +## History + +This specification was significantly inspired by and derived from IBC's [ICS](https://github.com/cosmos/ibc/blob/main/spec/ics-001-ics-standard/README.md), which +was in turn derived from Ethereum's [EIP 1](https://github.com/ethereum/EIPs/blob/master/EIPS/eip-1.md). + +Nov 24, 2022 - Initial draft finished and submitted as a PR + +## Copyright + +All content herein is licensed under [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0). diff --git a/sdk/v0.54/reference/spec/_ics/README.mdx b/sdk/v0.54/reference/spec/_ics/README.mdx new file mode 100644 index 000000000..596861caa --- /dev/null +++ b/sdk/v0.54/reference/spec/_ics/README.mdx @@ -0,0 +1,8 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/spec/_ics/README' +title: Cosmos ICS +description: ICS030 - Signed Messages +--- + +* [ICS030 - Signed Messages](/sdk/v0.50/build/spec/_ics/ics-030-signed-messages) diff --git a/sdk/v0.54/reference/spec/_ics/ics-030-signed-messages.mdx b/sdk/v0.54/reference/spec/_ics/ics-030-signed-messages.mdx new file mode 100644 index 000000000..e7364ecdc --- /dev/null +++ b/sdk/v0.54/reference/spec/_ics/ics-030-signed-messages.mdx @@ -0,0 +1,196 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/reference/spec/_ics/ics-030-signed-messages' +title: 'ICS 030: Cosmos Signed Messages' +--- + +> TODO: Replace with valid ICS number and possibly move to new location. + +* [Changelog](#changelog) +* [Abstract](#abstract) +* [Preliminary](#preliminary) +* [Specification](#specification) +* [Future Adaptations](#future-adaptations) +* [API](#api) +* [References](#references) + +## Status + +Proposed. + +## Changelog + +## Abstract + +Having the ability to sign messages off-chain has proven to be a fundamental aspect +of nearly any blockchain. The notion of signing messages off-chain has many +added benefits such as saving on computational costs and reducing transaction +throughput and overhead. Within the context of the Cosmos, some of the major +applications of signing such data includes, but is not limited to, providing a +cryptographic secure and verifiable means of proving validator identity and +possibly associating it with some other framework or organization. In addition, +having the ability to sign Cosmos messages with a Ledger or similar HSM device. + +A standardized protocol for hashing, signing, and verifying messages that can be +implemented by the Cosmos SDK and other third-party organizations is needed. Such a +standardized protocol subscribes to the following: + +* Contains a specification of human-readable and machine-verifiable typed structured data +* Contains a framework for deterministic and injective encoding of structured data +* Utilizes cryptographic secure hashing and signing algorithms +* A framework for supporting extensions and domain separation +* Is invulnerable to chosen ciphertext attacks +* Has protection against potentially signing transactions a user did not intend to + +This specification is only concerned with the rationale and the standardized +implementation of Cosmos signed messages. It does **not** concern itself with the +concept of replay attacks as that will be left up to the higher-level application +implementation. If you view signed messages in the means of authorizing some +action or data, then such an application would have to either treat this as +idempotent or have mechanisms in place to reject known signed messages. + +## Preliminary + +The Cosmos message signing protocol will be parameterized with a cryptographic +secure hashing algorithm `SHA-256` and a signing algorithm `S` that contains +the operations `sign` and `verify` which provide a digital signature over a set +of bytes and verification of a signature respectively. + +Note, our goal here is not to provide context and reasoning about why necessarily +these algorithms were chosen apart from the fact they are the de facto algorithms +used in CometBFT and the Cosmos SDK and that they satisfy our needs for such +cryptographic algorithms such as having resistance to collision and second +pre-image attacks, as well as being [deterministic](https://en.wikipedia.org/wiki/Hash_function#Determinism) and [uniform](https://en.wikipedia.org/wiki/Hash_function#Uniformity). + +## Specification + +CometBFT has a well established protocol for signing messages using a canonical +JSON representation as defined [here](https://github.com/cometbft/cometbft/blob/master/types/canonical.go). + +An example of such a canonical JSON structure is CometBFT's vote structure: + +```go +type CanonicalJSONVote struct { + ChainID string `json:"@chain_id"` + Type string `json:"@type"` + BlockID CanonicalJSONBlockID `json:"block_id"` + Height int64 `json:"height"` + Round int `json:"round"` + Timestamp string `json:"timestamp"` + VoteType byte `json:"type"` +} +``` + +With such canonical JSON structures, the specification requires that they include +meta fields: `@chain_id` and `@type`. These meta fields are reserved and must be +included. They are both of type `string`. In addition, fields must be ordered +in lexicographically ascending order. + +For the purposes of signing Cosmos messages, the `@chain_id` field must correspond +to the Cosmos chain identifier. The user-agent should **refuse** signing if the +`@chain_id` field does not match the currently active chain! The `@type` field +must equal the constant `"message"`. The `@type` field corresponds to the type of +structure the user will be signing in an application. For now, a user is only +allowed to sign bytes of valid ASCII text ([see here](https://github.com/cometbft/cometbft/blob/v0.37.0/libs/strings/string.go#L35-L64)). +However, this will change and evolve to support additional application-specific +structures that are human-readable and machine-verifiable ([see Future Adaptations](#future-adaptations)). + +Thus, we can have a canonical JSON structure for signing Cosmos messages using +the [JSON schema](http://json-schema.org/) specification as such: + +```json expandable +{ + "$schema": "http://json-schema.org/draft-04/schema#", + "$id": "cosmos/signing/typeData/schema", + "title": "The Cosmos signed message typed data schema.", + "type": "object", + "properties": { + "@chain_id": { + "type": "string", + "description": "The corresponding Cosmos chain identifier.", + "minLength": 1 + }, + "@type": { + "type": "string", + "description": "The message type. It must be 'message'.", + "enum": [ + "message" + ] + }, + "text": { + "type": "string", + "description": "The valid ASCII text to sign.", + "pattern": "^[\\x20-\\x7E]+$", + "minLength": 1 + } + }, + "required": [ + "@chain_id", + "@type", + "text" + ] +} +``` + +e.g. + +```json +{ + "@chain_id": "1", + "@type": "message", + "text": "Hello, you can identify me as XYZ on keybase." +} +``` + +## Future Adaptations + +As applications can vary greatly in domain, it will be vital to support both +domain separation and human-readable and machine-verifiable structures. + +Domain separation will allow for application developers to prevent collisions of +otherwise identical structures. It should be designed to be unique per application +use and should directly be used in the signature encoding itself. + +Human-readable and machine-verifiable structures will allow end users to sign +more complex structures, apart from just string messages, and still be able to +know exactly what they are signing (opposed to signing a bunch of arbitrary bytes). + +Thus, in the future, the Cosmos signing message specification will be expected +to expand upon its canonical JSON structure to include such functionality. + +## API + +Application developers and designers should formalize a standard set of APIs that +adhere to the following specification: + +*** + +### **cosmosSignBytes** + +Params: + +* `data`: the Cosmos signed message canonical JSON structure +* `address`: the Bech32 Cosmos account address to sign data with + +Returns: + +* `signature`: the Cosmos signature derived using signing algorithm `S` + +*** + +### Examples + +Using the `secp256k1` as the DSA, `S`: + +```javascript +data = { + "@chain_id": "1", + "@type": "message", + "text": "I hereby claim I am ABC on Keybase!" +} + +cosmosSignBytes(data, "cosmos1pvsch6cddahhrn5e8ekw0us50dpnugwnlfngt3") +> "0x7fc4a495473045022100dec81a9820df0102381cdbf7e8b0f1e2cb64c58e0ecda1324543742e0388e41a02200df37905a6505c1b56a404e23b7473d2c0bc5bcda96771d2dda59df6ed2b98f8" +``` + +## References diff --git a/sdk/v0.54/release-family.mdx b/sdk/v0.54/release-family.mdx new file mode 100644 index 000000000..2e1c9f5dd --- /dev/null +++ b/sdk/v0.54/release-family.mdx @@ -0,0 +1,76 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/release-family' +title: "Release Families" +description: "What release families are, what they contain, and how upgrades work." +--- + +## Overview + +A release family is a curated set of component versions across the Cosmos Stack that are tested for compatibility with one another. Cosmos Labs provides maintenance and bug fixes only for active families. + +This page is the canonical source of truth for release family lifecycle, active support windows, and retirement expectations. + +## What a Release Family Contains + +Each release family includes pinned versions of the following components: + +- [CometBFT](https://github.com/cometbft/cometbft) +- [Cosmos SDK](https://github.com/cosmos/cosmos-sdk) +- [Cosmos EVM](https://github.com/cosmos/evm) +- [IBC Go](https://github.com/cosmos/ibc-go) +- [Solidity IBC Eureka](https://github.com/cosmos/solidity-ibc-eureka) +- [Relayer](https://github.com/cosmos/ibc-relayer) +- [Attestor](https://github.com/cosmos/ibc-attestor) + +The goal is to guarantee that every version listed in a family is compatible with every other version in that family. + +Certain packages within the SDK may not be listed as Cosmos Labs consolidates separate Go modules over time. + +## Current Release Families + +### 2026.1 + +| Component | Version | +| --------- | ------- | +| [Cosmos SDK](https://github.com/cosmos/cosmos-sdk) | 0.55.x | +| [Enterprise Groups](https://github.com/cosmos/cosmos-sdk/tree/main/enterprise/group) | 1.x.y | +| [Enterprise PoA](https://github.com/cosmos/cosmos-sdk/tree/main/enterprise/poa) | 1.x.y | +| [CometBFT](https://github.com/cometbft/cometbft) | 0.40.x | +| [IBC Go](https://github.com/cosmos/ibc-go) | v11.x.y | +| [Solidity IBC Eureka](https://github.com/cosmos/solidity-ibc-eureka) | 3.0.x | +| [Relayer](https://github.com/cosmos/ibc-relayer) | 1.1.x | +| [Attestor](https://github.com/cosmos/ibc-attestor) | 1.0.x | + +### 2025.1 + +| Component | Version | +| --------- | ------- | +| [Cosmos SDK](https://github.com/cosmos/cosmos-sdk) | 0.53.x | +| [CometBFT](https://github.com/cometbft/cometbft) | 0.38.x | +| [IBC Go](https://github.com/cosmos/ibc-go) | v10.x.y | + +## Upgrades and Support + +Supported versions within a release family are updated over time, and upgrade paths are provided where generalized upgrades make sense. + +New release families include breaking changes from the previous family. New features are only considered for backporting to the most recent release family, and only when they are non-breaking. + +Cosmos Labs supports up to two release families at a time. + +Release cadence targets two new release families per year. If a planned successor family is delayed, the most recent supported family remains active until its successor is formally released. + +Lifecycle policy applies to families, not individual component versions in isolation. A component version is supported only when it appears in an active release family. + +For security reporting and vulnerability handling details, see the [Security and Maintenance Policy](/sdk/v0.54/security/security-policy). + + +## End of Life Notices + +The following releases are end of life and no longer receive maintenance, security patches, or compatibility support from Cosmos Labs: + +- CometBFT v0.37.x and lower +- ibc-go v0.7.x and lower +- Cosmos SDK v0.50.x and lower + +CometBFT v1.x is not supported. That release line was retracted and is not part of any supported release family. diff --git a/sdk/v0.54/security/audits.mdx b/sdk/v0.54/security/audits.mdx new file mode 100644 index 000000000..5e3f7a816 --- /dev/null +++ b/sdk/v0.54/security/audits.mdx @@ -0,0 +1,52 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/security/audits' +title: "Security Audits" +description: "Security audits and transparency reports for Cosmos Stack components" +--- + + +This page is auto-generated from the [cosmos/security](https://github.com/cosmos/security) repository. + +**Last synced:** Apr 27, 2026 | [View all audits](https://github.com/cosmos/security/tree/main/audits) + + +Cosmos Labs maintains a comprehensive security program for all Cosmos Stack components. This page provides links to third-party security audits and transparency reports. + + +## Cosmos EVM + +- [Sherlock 2025 07 28 Final](https://github.com/cosmos/security/blob/main/audits/evm/sherlock_2025_07_28_final.pdf) + +## Cosmos Hub (Gaia) + +- [2022 Liquid Staking Oak](https://github.com/cosmos/security/blob/main/audits/gaia/2022-liquid-staking-oak.pdf) + +## Interchain Security (ICS) + +- [Informal Ics 2023](https://github.com/cosmos/security/blob/main/audits/ics/informal-ics-2023.pdf) + +## Ledger + + +**ledger/** + +- [2023 Zondax](https://github.com/cosmos/security/blob/main/audits/ledger/ledger/2023-zondax.pdf) +- [2026 Zondax](https://github.com/cosmos/security/blob/main/audits/ledger/ledger/2026-zondax.pdf) + +## Cosmos SDK + +- [Cosmos Sdk 2019 Final](https://github.com/cosmos/security/blob/main/audits/sdk/cosmos_sdk_2019_final.pdf) +- [Cosmos Sdk V53 Audit Final](https://github.com/cosmos/security/blob/main/audits/sdk/cosmos_sdk_v53_audit_final.pdf) +- [Group Module Audit](https://github.com/cosmos/security/blob/main/audits/sdk/group_module_audit.pdf) + +## Transparency Reports + +- [Transparency Report 2023 2024](https://github.com/cosmos/security/blob/main/reports/transparency_report_2023_2024.pdf) + + +## Additional Resources + +- [Security and Maintenance Policy](/sdk/v0.54/security/security-policy) - Release and maintenance policy +- [Bug Bounty Program](/sdk/v0.54/security/bug-bounty) - Report vulnerabilities and earn rewards +- [cosmos/security Repository](https://github.com/cosmos/security) - Complete security documentation diff --git a/sdk/v0.54/security/bug-bounty.mdx b/sdk/v0.54/security/bug-bounty.mdx new file mode 100644 index 000000000..50c15d9c1 --- /dev/null +++ b/sdk/v0.54/security/bug-bounty.mdx @@ -0,0 +1,168 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/security/bug-bounty' +title: "Bug Bounty Program" +description: "Security and maintenance policy documentation for the Cosmos Stack" +--- + + +This content is sourced from the official [Cosmos Security](https://github.com/cosmos/security) repository. + +**Last sync:** Apr 27, 2026 | [View source](https://github.com/cosmos/security/blob/main/SECURITY.md) + + +## Introduction + +Cosmos Labs is committed to maintaining the security of the Cosmos Stack +and supporting responsible vulnerability disclosure. We operate a bug +bounty program to incentivize security researchers to identify and +report security issues. + +This document defines the process for reporting vulnerabilities, +describes the bug bounty program, and outlines Cosmos Labs’ approach to +patching and public disclosure. + +------------------------------------------------------------------------ + +## Reporting a Vulnerability + +**Private Disclosure Required** + +Security vulnerabilities affecting the Cosmos ecosystem—including the +Cosmos SDK, CometBFT, IBC, and other core components—must be reported +privately through the channels listed below. + +- **Preferred:** Submit reports through the + [Cosmos HackerOne Bug Bounty Program](https://hackerone.com/cosmos). +- If HackerOne submission is not possible, reports may be sent to + `security@cosmoslabs.io` with sufficient technical detail, including + impact and reproduction steps. + +> Reports submitted via email are *not eligible* for bounty rewards. +> Only reports submitted through HackerOne qualify for bounties. + +Public disclosure of vulnerabilities (including GitHub issues, blog +posts, or social media) is prohibited until Cosmos Labs has remediated +the issue and explicitly authorized disclosure. +Disclosure timelines may be coordinated with the reporter. + +Submission of a report constitutes agreement to participate in +**coordinated vulnerability disclosure**, allowing time for development, +testing, and deployment of a fix prior to public release of details. + +------------------------------------------------------------------------ + +## Bug Bounty Program Overview + +Cosmos Labs operates a bug bounty program through **HackerOne**. +Eligible reports are rewarded based on severity, impact, and quality. + +**In Scope:** Core Cosmos Stack components, including the Cosmos SDK, +CometBFT, IBC, Cosmos EVM, and other critical infrastructure components. + +The authoritative scope definition, severity classifications, and +reward ranges are maintained on the Cosmos +[HackerOne program page](https://hackerone.com/cosmos). + +The program is governed by **Safe Harbor** provisions for good-faith +research. The HackerOne page defines the applicable **Coordinated +Vulnerability Disclosure Policy** and **Safe Harbor terms**. + +> In the event of conflict, the HackerOne policy supersedes all other +> documentation. + +------------------------------------------------------------------------ + +## Vulnerability Severity Levels + +Reported vulnerabilities are assigned a severity classification that +determines handling priority and disclosure timing. + +| **Level** | **Description** | **Examples** | +|--------------|------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------| +| **Critical** | Permanent and irrecoverable loss of fund | Direct fund loss, unauthorized and unlimited token minting, irreversible theft of fund. | +| **High** | Severe impact affecting many nodes or users; often remotely exploitable. | Remote crash or chain halt vulnerabilities. | +| **Medium** | Limited or conditional impact; exploitation may require specific conditions. | Node halt requiring elevated permissions. | +| **Low** | Minor impact or impractical exploitation scenarios. | Slow block propagation, limited denial-of-service. | + +These classifications follow industry standards and inform response +urgency and disclosure policy. Additional details are available in the +[Classification Matrix](https://github.com/cosmos/security/blob/main/resources/CLASSIFICATION_MATRIX.md). + +------------------------------------------------------------------------ + +## Silent Patch and Disclosure Process + +Cosmos Labs follows a **silent patch** model for most security +vulnerabilities. Issues are addressed privately and remediated prior to +public disclosure. + +This approach aligns with practices +used by other major protocols, such as **Ethereum's Geth** (see +https://geth.ethereum.org/docs/developers/geth-developer/disclosures), +**Bitcoin Core** (see https://bitcoincore.org/en/security-advisories/), +and **Zcash** (see https://z.cash/technology/security-advisories/). + +Premature disclosure can place unpatched networks at risk. Silent +remediation allows operators time to upgrade before vulnerability +details become public. + +Vulnerabilities classified as **Critical** are handled on a case-by-case +basis. When an issue presents an immediate or network-wide risk, Cosmos +Labs will initiate emergency mitigations, private fix distribution, or +coordinated upgrades before any public disclosure occurs. + +If Cosmos Labs determines that a vulnerability with **network-wide +impact** (such as a chain halt or consensus failure) is already being +actively exploited, or that attacker awareness is confirmed prior to a +scheduled release, the issue is escalated and handled as **Critical** for +response and disclosure purposes, regardless of its original +classification. + +### Fix Distribution + +- Fixes are delivered through patch or minor releases. +- Release notes may omit explicit references to security implications. +- Validators and node operators may be notified privately to upgrade. +- For critical vulnerabilities, fixes may be distributed privately to + key operators or require emergency network upgrades. + +### Disclosure Timeline + +| **Severity** | **Disclosure Timing** | **Details** | +|------------------|----------------------------------------------------------------------------|-----------------------------------------------------------------------| +| **Low / Medium** | Approximately four weeks after public release of the fix | Full advisory published with impact and remediation details. | +| **High** | After the affected version reaches **End-of-Life (EOL)** (~1 year typical) | Disclosure delayed to reduce exploitation risk. | +| **Critical** | Case-by-case (At minimum after EOL) | Disclosure only when deemed safe; details may be limited or withheld. | + +------------------------------------------------------------------------ + +## Transparency and Post-Disclosure + +After expiration of the disclosure embargo, Cosmos Labs publishes a +**Security Advisory** (via GitHub advisories or official blog posts) +containing: + +- Vulnerability description +- Affected versions +- Severity classification +- Remediation guidance +- Reporter attribution (unless anonymity is requested) + +All advisories remain publicly available. This delayed disclosure model +balances ecosystem safety with long-term transparency. + +------------------------------------------------------------------------ + +Cosmos Labs acknowledges and appreciates the contributions of security +researchers, auditors, and white-hat hackers who strengthen the Cosmos +ecosystem. + +------------------------------------------------------------------------ + +### References + +- [Bitcoin Core Security Advisories](https://bitcoincore.org/en/security-advisories/) +- [Go Ethereum Vulnerability Disclosure](https://ethereumpow.github.io/go-ethereum/docs/vulnerabilities/vulnerabilities) +- [Bitcoin Core Security Disclosure Policy Announcement](https://bitexes.com/blog/124272) + diff --git a/sdk/v0.54/security/internal-audits.mdx b/sdk/v0.54/security/internal-audits.mdx new file mode 100644 index 000000000..50475f8ec --- /dev/null +++ b/sdk/v0.54/security/internal-audits.mdx @@ -0,0 +1,74 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/security/internal-audits' +title: "Internal Audit Process" +description: "Internal audit process for Cosmos Stack components" +--- + +This page outlines the internal audit process for Cosmos Stack components. + +## Overview + +Cosmos Labs runs a structured internal audit on complex features before they ship, in addition to any external audits. The goal is to verify correctness, security, and maintainability through a collaborative, line-by-line review led by the engineers with the most context on the code. + +The process has two stages: + +1. Pre-audit report +2. Internal audit + +Each audit is run by a designated audit lead, normally the person with the deepest context on the feature under review. The lead prepares the materials, briefs the team, facilitates the review, and splits the audit across multiple sessions when the feature is large. + +## Stage 1: Pre-audit report + +The pre-audit report is a prerequisite to the internal audit. It is a macro-level review that surfaces systemic and integration risks early, before the full team is brought in. Critical issues found here must be resolved before the internal audit is scheduled. + +The audit lead produces a pre-audit document covering: + +- Scope definition: every file added or modified, with attention to those that introduce or depend on integration points. +- Critical integration point analysis: changes in related or dependent modules outside the core feature that could introduce integration failures. +- Spec diff: updates to the feature specification, compared against the current implementation. +- Problem-first search: known problem areas and likely failure cases specific to the feature. + +Objectives of this stage: + +- Review code changes since the last stable milestone. +- Detect integration issues across system boundaries. +- Proactively investigate known vulnerability patterns and risk vectors, including state corruption, authorization failures, replay or race conditions, data consistency issues, resource exhaustion, callback or reentrancy risks, and other feature-specific failure modes. +- Evaluate the risk introduced by untrusted or novel external dependencies. + +## Stage 2: Internal audit + +Once the pre-audit report is complete and any critical issues are addressed, the audit lead schedules the internal audit: a live, line-by-line code review with the full team. The session is interactive. Participants ask questions, clarify assumptions, and flag concerns while the lead provides context on design intent and technical decisions. Large or complex features are split across multiple sessions by subsystem. + +### Context briefing + +The lead opens the audit by providing the context needed to review the feature: + +- A summary of the feature and its intended behavior. +- Key design decisions and tradeoffs. +- Integration risks identified during the pre-audit. +- Diagrams and visuals for complex workflows. + +Context briefings are recorded so they can be preserved as a reusable knowledge base. A briefing is skipped when an existing recording already covers the same ground. + +### Line-by-line review + +The review starts at user entry points and follows the flow of logic, with attention to: + +- Correctness of implementation and internal function logic. +- Input validation. +- Authentication, authorization, and access control. +- State consistency and post-conditions. +- Performance, resource usage, storage layout, and efficiency. +- Error handling and logging. +- Adherence to the specification and intended semantics. +- Event emission and observability. +- Test coverage, edge cases, and failure-path validation. +- Code readability and maintainability. +- Use of external libraries and dependencies. + +Security-critical external code that is not heavily battle-tested receives extra scrutiny. This includes parsing, serialization, cryptography, verification logic, and other security-sensitive components, where past audits have uncovered bugs in upstream dependencies. + +## Audit outcomes + +Throughout the audit, designated team members document every issue, concern, and open question. These notes are formalized into tracked issues and follow-up tasks scheduled in upcoming engineering iterations. Each item is categorized to support prioritization and triage after the audit, for example as a bug, optimization, specification mismatch, documentation improvement, or investigation needed. \ No newline at end of file diff --git a/sdk/v0.54/security/security-policy.mdx b/sdk/v0.54/security/security-policy.mdx new file mode 100644 index 000000000..1f8f0fb61 --- /dev/null +++ b/sdk/v0.54/security/security-policy.mdx @@ -0,0 +1,57 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/security/security-policy' +title: "Security and Maintenance Policy" +description: "Security and maintenance policy documentation for the Cosmos Stack" +--- + + +This content is sourced from the official [Cosmos Security](https://github.com/cosmos/security) repository. + +**Last sync:** Apr 27, 2026 | [View source](https://github.com/cosmos/security/blob/main/POLICY.md) + + +## Overview + +This policy defines how Cosmos Labs manages maintenance and support for the core Cosmos Stack components: + +- **CometBFT** +- **Cosmos SDK** +- **Cosmos EVM** +- **Inter-Blockchain Communication Protocol (IBC)** + +This release process aims to provide clarity and predictability to both developers using the Stack and the Cosmos Labs engineering team. Developers should know exactly which software combinations are supported and should be used in production. At the same time, the Cosmos Labs team can coordinate fixes, security patches, and upgrades across a smaller set of well-defined release families, allowing for faster response times and more predictable maintenance. + +To achieve this, we are introducing the concept of **Release Families**, curated sets of component versions of the Stack. Each family is fully tested for compatibility, stability, and long-term support. Maintenance and bug fixes are provided only for active families. + +--- + +## Release Families + +A **Release Family** is defined as a specific combination of component versions. + +The canonical source of truth for release family lifecycle, active support windows, and retirement policy is the [Release Families](/sdk/v0.54/release-family) page. + +This page intentionally does not duplicate lifecycle timelines to avoid policy drift across multiple pages. + +--- + +## What Is Supported + +- **Bug Fixes:** Critical security and stability issues are patched for all active families. +- **Compatibility:** All components within a family are guaranteed to work together. +- **Lifecycle and Retirement:** Lifecycle windows and retirement details are maintained on the [Release Families](/sdk/v0.54/release-family) page. +- **Upgradability:** We guarantee an upgrade path from one release family to the next adjacent family in the form of clear guides, compatibility guarantees, and tooling for assistance. + +--- + +## Security Fix Process + +Please read our [security policy](https://github.com/cosmos/security/blob/main/SECURITY.md) for a detailed breakdown of how bugs and vulnerabilities are to be handled for the Cosmos Stack. + +--- + +## End of Life (EOL) Notices + +Current and historical EOL notices for release families are maintained on the [Release Families](/sdk/v0.54/release-family) page to keep lifecycle policy centralized in one place. + diff --git a/sdk/v0.54/tutorials.mdx b/sdk/v0.54/tutorials.mdx new file mode 100644 index 000000000..b136db809 --- /dev/null +++ b/sdk/v0.54/tutorials.mdx @@ -0,0 +1,31 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/tutorials' +title: "Node Tutorial" +description: "Version: v0.54" +--- + +This guide covers everything you need to run, configure, and maintain a Cosmos SDK node. Whether you're setting up a local development node, deploying to a testnet, or running production infrastructure, you'll find step-by-step instructions and best practices. + +The node tutorial uses the `simapp` example application and its corresponding CLI binary `simd` as the blockchain application and CLI. You can view the source code for `simapp` [on GitHub](https://github.com/cosmos/cosmos-sdk/tree/main/simapp). + + + + Learn the fundamentals of running a node, from initial setup through keyring management and starting your node. + + + Connect to your node and query data using CLI, gRPC, or REST endpoints. + + + Create, sign, and broadcast transactions to your node using various methods. + + + Configure and deploy nodes for testnet environments and production networks. + + + Monitor your node's health and performance using built-in telemetry and metrics collection. + + + Manage chain upgrades and migrations safely using in-place upgrade mechanisms and Cosmovisor. + + diff --git a/sdk/v0.54/tutorials/example/00-overview.mdx b/sdk/v0.54/tutorials/example/00-overview.mdx new file mode 100644 index 000000000..9dc5c85cf --- /dev/null +++ b/sdk/v0.54/tutorials/example/00-overview.mdx @@ -0,0 +1,39 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/tutorials/example/00-overview' +title: Tutorial Intro +description: Build a module from scratch, wire it into a chain, and run it locally, all in minutes. +--- + +The Cosmos SDK is a developer-first framework for building custom blockchains. This tutorial series shows you how to build a module from scratch, wire it into a chain, and run it locally, all in minutes. + +By the end, you will have: + +- A working Cosmos SDK chain running on your machine +- A custom module you built yourself, wired into the chain +- A clear mental model of how modules, keepers, messages, and queries fit together + +This series starts from zero; you don't need any prior Cosmos SDK experience to follow along. + +## The example repo + +All tutorials in this series are based on [cosmos/example](https://github.com/cosmos/example), a reference Cosmos SDK chain built around a custom `x/counter` module. + +The repo has two main branches: + +- `main`: the complete chain with the full `x/counter` module wired in. This is used in the [Quickstart guide](/sdk/v0.54/tutorials/example/02-quickstart). +- `tutorial/start`: the same chain without the counter module. The `x/counter` directory and its app wiring are stripped out so you can build the module from scratch by [following the tutorial](/sdk/v0.54/tutorials/example/03-build-a-module). + +If you want to follow along and build the module yourself, start from `tutorial/start`. If you want to browse the finished implementation first, use `main`. + +## What's in this series + +1. [Prerequisites](/sdk/v0.54/tutorials/example/01-prerequisites): Install Go, Make, Docker, and Git. Clone the repo and get familiar with the layout. + +2. [Quickstart](/sdk/v0.54/tutorials/example/02-quickstart): Build and run the chain in minutes. Submit a transaction, query the result, and see the counter module in action before you build it yourself. + +3. [Build a Module from Scratch](/sdk/v0.54/tutorials/example/03-build-a-module): Build a minimal counter module step by step: proto definitions, keeper, message server, query server, and app wiring. Start here if you want to understand how a module comes together. + +4. [Full Module Walkthrough](/sdk/v0.54/tutorials/example/04-counter-walkthrough): Walk through the complete `x/counter` implementation on `main`. Covers everything added on top of the minimal module: params, governance-gated authority, validation, fees, sentinel errors, telemetry, AutoCLI, simulation, block hooks, and a full unit test suite. + +5. [Run and Test](/sdk/v0.54/tutorials/example/05-run-and-test): Learn the full development workflow: running a local chain, using the CLI, and working with the three layers of testing: unit tests, end-to-end tests, and simulation. diff --git a/sdk/v0.54/tutorials/example/01-prerequisites.mdx b/sdk/v0.54/tutorials/example/01-prerequisites.mdx new file mode 100644 index 000000000..e91c83d75 --- /dev/null +++ b/sdk/v0.54/tutorials/example/01-prerequisites.mdx @@ -0,0 +1,122 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/tutorials/example/01-prerequisites' +title: Prerequisites +description: Install dependencies +--- + +Before starting the tutorial, make sure you have the following tools installed. + + +This tutorial is intended for macOS and Linux systems. Other systems may have additional requirements. + + +## Go + +The example chain requires Go 1.25 or higher. + +```bash +go version +# go version go1.25.0 linux/amd64 # Linux +# go version go1.25.0 darwin/arm64 # macOS +``` + +If Go is not installed, download it from [go.dev/dl](https://go.dev/dl). + +### Configure Go Environment Variables + +After installing Go, make sure `$GOPATH/bin` is on your `PATH` so installed binaries (like `exampled`) are accessible. + +Open your shell config file (`~/.zshrc` on macOS or `~/.bashrc` on Linux) and add: + +```bash +export GOPATH=$HOME/go +export PATH=$PATH:$GOPATH/bin +``` + +Then apply the changes: + +```bash +source ~/.zshrc # macOS +source ~/.bashrc # Linux +``` + +Verify: `go env GOPATH` + +## Make + +Make is used to run build and development commands throughout the tutorial. + +```bash +make --version +# GNU Make 3.81 +``` + +Make is pre-installed on most Linux and macOS systems. If it is missing: + +- **macOS:** `xcode-select --install` +- **Linux (Debian/Ubuntu):** `sudo apt install build-essential` + +## Docker + +Docker is required to run `make proto-gen`, which generates Go code from the module's proto files using [buf](https://buf.build). + +```bash +docker --version +# Docker version 29.2.1 +``` + +Download Docker from [docs.docker.com/get-docker](https://docs.docker.com/get-docker). + +Docker must be running before you execute `make proto-gen`. + +## Git + +```bash +git --version +# git version 2.52.0 +``` + +## Clone the repository + +Clone [cosmos/example](https://github.com/cosmos/example) and navigate into it: + +```bash +git clone https://github.com/cosmos/example +cd example +``` + +The repo has two branches used in this tutorial series: + +- `main` — the complete chain with the full `x/counter` module wired in. +- `tutorial/start` — the same chain with the counter module stripped out. Start here if you want to build the module yourself from scratch. + +## Repository Layout + +After cloning, the repository looks like this: + +```text +example/ +├── exampled/ # Binary entrypoint (main.go + CLI root command) +├── app.go # Chain application, module wiring lives here +├── proto/ # Proto definitions for all modules +├── x/ # Module implementations +│ └── counter/ # The example counter module +├── tests/ # E2E and integration tests +├── scripts/ # Local node and proto generation scripts +├── docs/ # This tutorial series +└── Makefile # Build, test, and dev commands +``` + +## Where things live + +The tutorials in this section will walk you through the most common kinds of chain changes and show you where they usually live in the repo: + +- Add or modify a module: `x//` and `proto/` +- Wire a module into the chain: `app.go` +- Change the binary or CLI: `exampled/` +- Run the chain or tests: `Makefile` targets + +--- + +Next: [Quickstart →](/sdk/v0.54/tutorials/example/02-quickstart) diff --git a/sdk/v0.54/tutorials/example/02-quickstart.mdx b/sdk/v0.54/tutorials/example/02-quickstart.mdx new file mode 100644 index 000000000..8abf6fb93 --- /dev/null +++ b/sdk/v0.54/tutorials/example/02-quickstart.mdx @@ -0,0 +1,108 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/tutorials/example/02-quickstart' +title: Chain Quickstart +description: Start a chain, submit a transaction, and query the result in minutes +--- + +Building on Cosmos is simple: you can start a chain with a [single command](#start-the-chain). This quickstart gets you from zero to a running chain, a submitted transaction, and a queried result in minutes. + +`exampled` is a simple Cosmos SDK chain that shows the core pieces of a working app chain. It includes the basic building-block modules for accounts, bank, staking, distribution, slashing, governance, and more, plus a custom `x/counter` module. In the next tutorials, you'll build a simple version of that module yourself and then walk through the full implementation. + + +Before continuing, make sure you have completed the [Prerequisites](/sdk/v0.54/tutorials/example/01-prerequisites) to get your environment set up. + + +## Install the binary + +Run the following to compile the `exampled` binary and place it on your `$PATH`. + +```bash +make install +``` + +Verify the install by running: + +```bash +exampled version +``` + +You can also run the following to see all available node CLI commands: + +```bash +exampled +``` + +## Start the chain + +Run the following to start a single-node local chain. It handles all setup automatically: initializes the chain data, creates test accounts, and starts the node. Leave it running in this terminal. + +```bash +make start +``` + +## Query the counter + +Open a second terminal and query the current count: + +```bash +exampled query counter count +``` + +You should see the following output, which means the counter is starting at `0`: + + +```text +{} +``` + +You can also query the module parameters: + +```bash +exampled query counter params +``` + +This shows that the fee to increment the counter is stored as a module parameter. The base coin denomination for the `exampled` chain is `stake`. + +```yaml +params: + add_cost: + - amount: "100" + denom: stake + max_add_value: "100" +``` + +## Submit an add transaction + +Send an `Add` transaction to increment the counter. This charges a fee from the funded `alice` account you are sending the transaction from: + +```bash +exampled tx counter add 5 --from alice --chain-id demo --yes +``` + +## Query the counter again + +After submitting the transaction, query the counter again to see the updated module state: + +```bash +exampled query counter count +``` + +You should see the following: + +``` +count: "5" +``` + +Congratulations! You just ran a blockchain, submitted a transaction, and queried module state. + +## Next steps + +In the following tutorials, you will: + +1. Build a minimal version of this module from scratch to understand the core pattern +2. Walk through the full `x/counter` module example to see what it adds +3. See how modules are wired into a chain and how to run the full test suite + + +Next: [Build a Module from Scratch →](/sdk/v0.54/tutorials/example/03-build-a-module) diff --git a/sdk/v0.54/tutorials/example/03-build-a-module.mdx b/sdk/v0.54/tutorials/example/03-build-a-module.mdx new file mode 100644 index 000000000..e95ddd370 --- /dev/null +++ b/sdk/v0.54/tutorials/example/03-build-a-module.mdx @@ -0,0 +1,752 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/tutorials/example/03-build-a-module' +title: Build a Module from Scratch +description: Build a simple counter module from scratch in minutes +--- + +In [quickstart](/sdk/v0.54/tutorials/example/02-quickstart), you started a chain and submitted a transaction to increase the counter. In this tutorial, you'll build a simple counter module from scratch. It follows the same overall structure as the full `x/counter`, but uses a stripped-down version so you can focus on the core steps of building and wiring a module yourself. + +By the end, you'll have built a working module and wired it into a running chain. For a deeper dive into how modules work in the Cosmos SDK, see [Intro to Modules](/sdk/v0.54/learn/concepts/modules). + + +Before continuing, you must follow the [Prerequisites guide](/sdk/v0.54/tutorials/example/01-prerequisites) to make sure everything is installed. + + +## Making modules + +The Cosmos SDK makes it easy to build custom business logic directly into your chain through modules. Every module follows the same overall pattern: + +```text +proto files → code generation → keeper → msg server → query server → module.go → app wiring +``` + +First, you'll define what the module does: + +- Define messages: users can send `Add` to increase the counter +- Define queries: users can query `Count` to read the current value +- Define genesis state: the module starts with a count of `0` + +Then you'll wire that behavior into the SDK: + +- Run `proto-gen` to generate the Go types and interfaces +- Implement your business logic in a `keeper` to store the count and update it +- Implement `MsgServer` and `QueryServer` to pass messages and queries into the keeper +- Register the module in `module.go` +- Wire it into the chain in `app.go` + +You'll build the following module structure: + +```text +proto/example/counter/v1/ +├── tx.proto # Transaction message and Msg service definition +├── query.proto # Query message and Query service definition +└── genesis.proto # Genesis state definition + +x/counter/ +├── keeper/ +│ ├── keeper.go # Keeper struct and state methods +│ ├── msg_server.go # MsgServer implementation +│ └── query_server.go # QueryServer implementation +├── types/ +│ ├── keys.go # Module name and store key constants +│ ├── codec.go # Interface registration +│ └── *.pb.go # Generated from proto — do not edit +├── module.go # AppModule wiring +└── autocli.go # CLI command definitions +``` + +## Step 1: Setup + +This tutorial uses the `tutorial/start` branch, which is a blank template for you to create the module from scratch and wire it into `app.go`. + +1. Clone the repo if you haven't already: + +```bash +git clone https://github.com/cosmos/example +cd example +``` + +2. Check out the `tutorial/start` branch and make the new module directories: + +```bash +git checkout tutorial/start +mkdir -p x/counter/keeper x/counter/types proto/example/counter/v1 +``` + +You should see empty placeholder directories at `x/counter/` and `proto/example/counter/v1/`. + +## Step 2: Proto files + +Proto files are the source of truth for the module's public API. You define messages and services here. For a deeper look at how protobuf is used across modules, see [Encoding and Protobuf](/sdk/v0.54/learn/concepts/encoding#how-protobuf-is-used-in-modules). + +In this tutorial, the counter module stores one number, `Add` increases it by the amount the user submits, and the query returns the current value. + +First, create the three proto files: + +```bash +touch proto/example/counter/v1/tx.proto \ + proto/example/counter/v1/query.proto \ + proto/example/counter/v1/genesis.proto +``` + +Then add the following contents to each file. + +### tx.proto + +This is the first module file you define. It declares the transaction message shape for `Add`: what the user sends to increment the counter, and what the module returns after handling it. To learn more about how messages are defined and routed, see [Messages](/sdk/v0.54/learn/concepts/transactions#messages). Add the following code to `tx.proto`. + +```proto +syntax = "proto3"; + +// Matches the module's protobuf namespace. +package example.counter; + +// Provides Cosmos SDK message annotations like signer and service markers. +import "cosmos/msg/v1/msg.proto"; + +// Generated Go types are written into x/counter/types. +option go_package = "github.com/cosmos/example/x/counter/types"; + +service Msg { + // Marks this as a transaction service, not a normal gRPC service. + option (cosmos.msg.v1.service) = true; + // Add is the one transaction this minimal module supports. + rpc Add(MsgAddRequest) returns (MsgAddResponse); +} + +message MsgAddRequest { + // The sender signs this message. + option (cosmos.msg.v1.signer) = "sender"; + string sender = 1; + uint64 add = 2; +} + +message MsgAddResponse { + // Return the new counter value after the add succeeds. + uint64 updated_count = 1; +} +``` + +### query.proto + +This file defines the read-only gRPC query service and the response type for fetching the current count. To learn more about how queries differ from transactions, see [Queries](/sdk/v0.54/learn/concepts/transactions#queries). Add the following code to `query.proto`. + +```proto +syntax = "proto3"; + +// Matches the module's protobuf namespace. +package example.counter; + +// Enables the REST gateway route annotation below. +import "google/api/annotations.proto"; + +// Generated Go types are written into x/counter/types. +option go_package = "github.com/cosmos/example/x/counter/types"; + +service Query { + rpc Count(QueryCountRequest) returns (QueryCountResponse) { + // Exposes this query over the HTTP API as well as gRPC. + option (google.api.http).get = "/example/counter/v1/count"; + } +} + +// Empty because this query only needs the module's current state. +message QueryCountRequest {} + +message QueryCountResponse { + // The current counter value. + uint64 count = 1; +} +``` + +### genesis.proto + +This file defines the data the module stores in genesis so the counter can be initialized when the chain starts. Add the following code to `genesis.proto`. + +```proto +syntax = "proto3"; + +// Matches the module's protobuf namespace. +package example.counter; + +// Generated Go types are written into x/counter/types. +option go_package = "github.com/cosmos/example/x/counter/types"; + +message GenesisState { + // The counter value to load when the chain initializes. + uint64 count = 1; +} +``` + +## Step 3: Generate Code + +1. Make sure Docker is running. + +2. The first time you run proto-gen you need to build the builder image. Run the following commands: + +```bash +make proto-image-build +make proto-gen +``` + +This compiles the proto files using [buf](https://buf.build) inside Docker to produce the Go interfaces you will then implement. + +The generated files will appear in `x/counter/types/`: + +```text +x/counter/types/ +├── tx.pb.go # MsgAddRequest, MsgAddResponse, MsgServer interface +├── query.pb.go # QueryCountRequest, QueryCountResponse, QueryServer interface +├── query.pb.gw.go # REST gateway registration +└── genesis.pb.go # GenesisState +``` + +> **Do not edit generated files.** Changes to public types belong in the proto files. Re-run `make proto-gen` after any proto change. + +The most important generated output is the `MsgServer` and `QueryServer` interfaces. In Steps 5 and 6, you'll implement them in `keeper/msg_server.go` and `keeper/query_server.go`. + + +## Step 4: Types + +Next, you'll define the module types and identifiers in `x/counter/types` that the rest of the module depends on. + +Create the two files for this section: + +```bash +touch x/counter/types/keys.go \ + x/counter/types/codec.go +``` + +Then add the following contents to each file. + +### keys.go + +This file defines the module's basic identifiers: the module name used throughout the SDK, and the store key used to claim the module's KV store namespace. For more on how modules access state through store keys, see [How modules access state](/sdk/v0.54/learn/concepts/store#how-modules-access-state). + +```go +// x/counter/types/keys.go +package types + +const ( + // ModuleName is the name the SDK uses to refer to this module. + ModuleName = "counter" + // StoreKey is the key for this module's KV store. + StoreKey = ModuleName +) +``` + +`ModuleName` identifies the module throughout the SDK (routing, events, governance). `StoreKey` is the key used to claim the module's isolated namespace in the chain's KV store (set equal to `ModuleName` by convention). + +### Interface Registration + +This file registers your generated message types with the SDK interface registry so the application can decode and route your module's transactions correctly. + +```go +// x/counter/types/codec.go +package types + +import ( + codectypes "github.com/cosmos/cosmos-sdk/codec/types" + sdk "github.com/cosmos/cosmos-sdk/types" + "github.com/cosmos/cosmos-sdk/types/msgservice" +) + +func RegisterInterfaces(registry codectypes.InterfaceRegistry) { + // Register MsgAddRequest as an sdk.Msg so the app can decode it from transactions. + registry.RegisterImplementations((*sdk.Msg)(nil), + &MsgAddRequest{}, + ) + // Register the generated Msg service description for routing. + msgservice.RegisterMsgServiceDesc(registry, &_Msg_serviceDesc) +} +``` + +`_Msg_serviceDesc` is generated by `make proto-gen` — it describes the `Msg` gRPC service defined in `tx.proto`. + + +## Step 5: Keeper + +In this step, you create the keeper, which is the part of the module that owns the counter state and provides the methods the rest of the module will call. For a conceptual overview of the keeper's role, see [Keeper](/sdk/v0.54/learn/concepts/modules#keeper). + +Create the keeper file: + +```bash +touch x/counter/keeper/keeper.go +``` + +Then add the following contents. + +This file defines the keeper struct, sets up the counter's storage item, and implements the core state methods for reading, updating, and loading the counter at genesis. + +```go +// x/counter/keeper/keeper.go +package keeper + +import ( + "context" + "errors" + + "cosmossdk.io/collections" + "cosmossdk.io/core/store" + "github.com/cosmos/cosmos-sdk/codec" + "github.com/cosmos/example/x/counter/types" +) + +type Keeper struct { + Schema collections.Schema + counter collections.Item[uint64] +} + +func NewKeeper(storeService store.KVStoreService, cdc codec.Codec) *Keeper { + sb := collections.NewSchemaBuilder(storeService) + k := Keeper{ + // Store the counter under prefix 0 in this module's KV store. + counter: collections.NewItem(sb, collections.NewPrefix(0), "counter", collections.Uint64Value), + } + schema, err := sb.Build() + if err != nil { + panic(err) + } + k.Schema = schema + return &k +} + +func (k *Keeper) GetCount(ctx context.Context) (uint64, error) { + count, err := k.counter.Get(ctx) + // Treat missing state as zero so a fresh chain starts cleanly. + if err != nil && !errors.Is(err, collections.ErrNotFound) { + return 0, err + } + return count, nil +} + +func (k *Keeper) AddCount(ctx context.Context, amount uint64) (uint64, error) { + count, err := k.GetCount(ctx) + if err != nil { + return 0, err + } + // Increment the current count and write it back to state. + newCount := count + amount + return newCount, k.counter.Set(ctx, newCount) +} + +func (k *Keeper) InitGenesis(ctx context.Context, gs *types.GenesisState) error { + return k.counter.Set(ctx, gs.Count) +} + +func (k *Keeper) ExportGenesis(ctx context.Context) (*types.GenesisState, error) { + count, err := k.GetCount(ctx) + if err != nil { + return nil, err + } + return &types.GenesisState{Count: count}, nil +} +``` + +`collections.Item[uint64]` is a typed KV store entry; the `collections` package handles encoding and namespacing. `GetCount` treats `ErrNotFound` as zero so the counter starts at zero without explicit initialization. + +> **State layout** +> +> - `StoreKey` (`"counter"`) is the module's isolated namespace within the chain's global KV store. No other module can read or write this namespace. +> - `collections.NewPrefix(0)` is a single-byte prefix that identifies the `counter` item within the module's namespace. A module with multiple items would use `NewPrefix(0)`, `NewPrefix(1)`, etc. to keep them separate. +> - `ErrNotFound` treated as zero means the keeper never needs to explicitly set an initial value — the first `GetCount` call on a fresh chain returns `0` by convention. + + +## Step 6: MsgServer + +In this step, you implement the transaction handler for the generated `MsgServer` interface. This is the code path that runs when a user submits `tx counter add`. For a conceptual overview of message execution, see [Message execution](/sdk/v0.54/learn/concepts/modules#message-execution-msgserver). + +Create the message server file: + +```bash +touch x/counter/keeper/msg_server.go +``` + +Then add the following contents. + +This file implements the generated `MsgServer` interface and forwards the `Add` transaction to the keeper's `AddCount` method. + +```go +// x/counter/keeper/msg_server.go +package keeper + +import ( + "context" + + "github.com/cosmos/example/x/counter/types" +) + +type msgServer struct { + *Keeper +} + +func NewMsgServerImpl(k *Keeper) types.MsgServer { + return &msgServer{k} +} + +func (m msgServer) Add(ctx context.Context, req *types.MsgAddRequest) (*types.MsgAddResponse, error) { + // Delegate the state update to the keeper. + newCount, err := m.AddCount(ctx, req.GetAdd()) + if err != nil { + return nil, err + } + // Return the updated count back to the caller. + return &types.MsgAddResponse{UpdatedCount: newCount}, nil +} +``` + +`msgServer` embeds `*Keeper` and delegates directly to `AddCount`. The handler itself contains no business logic. + + +## Step 7: QueryServer + +In this step, you implement the read-only query handler for the generated `QueryServer` interface. This is the code path that runs when someone queries the current counter value. For more on how modules expose queries, see [Queries](/sdk/v0.54/learn/concepts/modules#queries). + +Create the query server file: + +```bash +touch x/counter/keeper/query_server.go +``` + +Then add the following contents. + +This file implements the generated `QueryServer` interface and returns the current counter value from the keeper. + +```go +// x/counter/keeper/query_server.go +package keeper + +import ( + "context" + + "github.com/cosmos/example/x/counter/types" +) + +type queryServer struct { + *Keeper +} + +func NewQueryServer(k *Keeper) types.QueryServer { + return &queryServer{k} +} + +func (q queryServer) Count(ctx context.Context, _ *types.QueryCountRequest) (*types.QueryCountResponse, error) { + // Read the current count from state and return it in the query response. + count, err := q.GetCount(ctx) + if err != nil { + return nil, err + } + return &types.QueryCountResponse{Count: count}, nil +} +``` + + +## Step 8: module.go + +In this step, you connect your keeper and generated services to the Cosmos SDK module framework so the application knows how to initialize the module, expose its query routes, and register its transaction handlers. + +Create the module file: + +```bash +touch x/counter/module.go +``` + +Then add the following contents. + +This file defines the app module types and wires your keeper into genesis handling, service registration, and gRPC gateway registration. + +```go +// x/counter/module.go +package counter + +import ( + "context" + "encoding/json" + + "cosmossdk.io/core/appmodule" + "github.com/cosmos/cosmos-sdk/client" + "github.com/cosmos/cosmos-sdk/codec" + codecTypes "github.com/cosmos/cosmos-sdk/codec/types" + sdk "github.com/cosmos/cosmos-sdk/types" + "github.com/cosmos/cosmos-sdk/types/module" + "github.com/grpc-ecosystem/grpc-gateway/runtime" + + "github.com/cosmos/example/x/counter/keeper" + countertypes "github.com/cosmos/example/x/counter/types" +) + +var ( + // Compile-time checks that AppModule implements the required module interfaces. + _ appmodule.AppModule = AppModule{} + _ module.HasConsensusVersion = AppModule{} + _ module.HasGenesis = AppModule{} + _ module.HasServices = AppModule{} +) + +type AppModuleBasic struct { + cdc codec.Codec +} + +func (a AppModuleBasic) Name() string { return countertypes.ModuleName } + +func (a AppModuleBasic) RegisterLegacyAminoCodec(*codec.LegacyAmino) {} + +func (a AppModuleBasic) RegisterInterfaces(registry codecTypes.InterfaceRegistry) { + countertypes.RegisterInterfaces(registry) +} + +func (a AppModuleBasic) DefaultGenesis(cdc codec.JSONCodec) json.RawMessage { + // Start the module with a zero counter by default. + return cdc.MustMarshalJSON(&countertypes.GenesisState{Count: 0}) +} + +func (a AppModuleBasic) ValidateGenesis(cdc codec.JSONCodec, _ client.TxEncodingConfig, bz json.RawMessage) error { + gs := countertypes.GenesisState{} + return cdc.UnmarshalJSON(bz, &gs) +} + +func (a AppModuleBasic) RegisterGRPCGatewayRoutes(clientCtx client.Context, mux *runtime.ServeMux) { + // Expose the Query service through the HTTP gateway. + if err := countertypes.RegisterQueryHandlerClient(context.Background(), mux, countertypes.NewQueryClient(clientCtx)); err != nil { + panic(err) + } +} + +type AppModule struct { + AppModuleBasic + keeper *keeper.Keeper +} + +func NewAppModule(cdc codec.Codec, k *keeper.Keeper) AppModule { + return AppModule{AppModuleBasic: AppModuleBasic{cdc: cdc}, keeper: k} +} + +func (a AppModule) IsOnePerModuleType() {} +func (a AppModule) IsAppModule() {} + +func (a AppModule) ConsensusVersion() uint64 { return 1 } + +func (a AppModule) RegisterServices(cfg module.Configurator) { + // Connect the generated service interfaces to your keeper-backed implementations. + countertypes.RegisterMsgServer(cfg.MsgServer(), keeper.NewMsgServerImpl(a.keeper)) + countertypes.RegisterQueryServer(cfg.QueryServer(), keeper.NewQueryServer(a.keeper)) +} + +func (a AppModule) InitGenesis(ctx sdk.Context, cdc codec.JSONCodec, bz json.RawMessage) { + gs := &countertypes.GenesisState{} + cdc.MustUnmarshalJSON(bz, gs) + // Load the initial counter value into state at chain start. + if err := a.keeper.InitGenesis(ctx, gs); err != nil { + panic(err) + } +} + +func (a AppModule) ExportGenesis(ctx sdk.Context, cdc codec.JSONCodec) json.RawMessage { + gs, err := a.keeper.ExportGenesis(ctx) + if err != nil { + panic(err) + } + // Write the current counter value back out for exports. + return cdc.MustMarshalJSON(gs) +} +``` + +The `var _ interface = Struct{}` block at the top is a Go compile-time check — if the struct is missing any required method, the build fails immediately. + +`RegisterServices` is the most important method. It connects the generated server interfaces to your implementations, making them reachable from the SDK's message and query routers. + + +## Step 9: AutoCLI + +In this step, you define the CLI metadata for your module. AutoCLI reads this configuration together with your proto services and generates the `exampled query counter` and `exampled tx counter` commands automatically. + +Create the AutoCLI file: + +```bash +touch x/counter/autocli.go +``` + +Then add the following contents. + +This file tells `AutoCLI` how to expose the `Count` query and `Add` transaction as simple command-line commands. + +```go +// x/counter/autocli.go +package counter + +import ( + autocliv1 "cosmossdk.io/api/cosmos/autocli/v1" +) + +func (a AppModule) AutoCLIOptions() *autocliv1.ModuleOptions { + return &autocliv1.ModuleOptions{ + Query: &autocliv1.ServiceCommandDescriptor{ + Service: "example.counter.Query", + RpcCommandOptions: []*autocliv1.RpcCommandOptions{ + // exampled query counter count + {RpcMethod: "Count", Use: "count", Short: "Query the current counter value"}, + }, + }, + Tx: &autocliv1.ServiceCommandDescriptor{ + Service: "example.counter.Msg", + RpcCommandOptions: []*autocliv1.RpcCommandOptions{ + // exampled tx counter add 4 --from alice + {RpcMethod: "Add", Use: "add [amount]", Short: "Add to the counter", + PositionalArgs: []*autocliv1.PositionalArgDescriptor{{ProtoField: "add"}}}, + }, + }, + } +} +``` + +`PositionalArgs` maps the first CLI argument to the `add` field in `MsgAddRequest`, so `add 4` works instead of `add --add 4`. + + +## Step 10: Wire into app.go + +In this step, you wire your new module into the application so the chain creates its store, constructs its keeper, and includes it in module startup and genesis handling. For a full explanation of what `app.go` does and why the wiring order matters, see [app.go Overview](/sdk/v0.54/learn/concepts/app-go). + +Open `app.go` and find each marker comment. Paste the code directly below it. + +### 1. Imports + +Add the counter module, keeper, and shared types imports to `app.go`. + +Find the comment in `app.go` and add the code directly below it. + +```go +// counter tutorial app wiring 1: add counter imports below + counter "github.com/cosmos/example/x/counter" + counterkeeper "github.com/cosmos/example/x/counter/keeper" + countertypes "github.com/cosmos/example/x/counter/types" +``` + +### 2. Keeper Field + +Store the counter keeper on `ExampleApp` so the rest of the app can reference it. + +```go +// counter tutorial app wiring 2: add the counter keeper field below +CounterKeeper *counterkeeper.Keeper +``` + +### 3. Store Key + +Give the counter module its own KV store namespace. + +```go +// counter tutorial app wiring 3: add the counter store key below +countertypes.StoreKey, +``` + +### 4. Keeper Instantiation + +Construct the counter keeper using the module store and app codec. + +```go +// counter tutorial app wiring 4: create the counter keeper below +app.CounterKeeper = counterkeeper.NewKeeper( + runtime.NewKVStoreService(keys[countertypes.StoreKey]), + appCodec, +) +``` + +### 5. Module Manager + +Register the counter module with the app's module manager. + +```go +// counter tutorial app wiring 5: register the counter module below +counter.NewAppModule(appCodec, app.CounterKeeper), +``` + +### 6. Genesis Order + +Include the counter module when the app initializes state from genesis. + +```go +// counter tutorial app wiring 6: add the counter module to genesis order below +countertypes.ModuleName, +``` + +### 7. Export Order + +Include the counter module when the app exports state back out to genesis. + +```go +// counter tutorial app wiring 7: add the counter module to export order below +countertypes.ModuleName, +``` + + +## Step 11: Build + +Run the following to compile the app and make sure the new module wiring is valid before you try to run the chain. + +```bash +go build ./... +``` + +Fix any compilation errors before continuing. + + +## Step 12: Test your module + +Now you'll run the app locally and use one transaction plus one query to confirm the module works end-to-end. + +### Start the chain + +First, install the binary and start the demo chain. + +```bash +make install +make start +``` + +This builds and installs `exampled` and then runs `scripts/local_node.sh`, which: +- resets the local chain data +- initializes genesis +- creates and funds the `alice` and `bob` test accounts +- creates a validator transaction +- starts the chain + +You'll see the chain running and it should start producing blocks. + +### Submit a transaction + +Open a second terminal and submit a transaction that adds `4` to the counter: + +```bash +exampled tx counter add 4 --from alice --chain-id demo --yes +``` + +If the transaction succeeds, the response should include `code: 0`, which means the chain accepted and executed the transaction without an application error: + +``` +code: 0 +``` + +### Query the chain + +Query the counter to confirm the stored value changed using the query command that `AutoCLI` generated earlier: + +```bash +exampled query counter count +``` + +You should see the following output: + +```text +count: "4" +``` + +Congratulations, you've just created a Cosmos module from scratch and wired it into a real chain! + +If you are planning to build a production module, see [Module Design Considerations](/sdk/v0.54/guides/module-design/module-design-considerations) for guidance on state structure, message surface, dependencies, and upgrade planning before you ship. + +## Next steps + +The simple counter module you built here follows the same structure as the full `x/counter` example in the `main` branch. Next, you'll see how the full module extends that foundation with features like params, fee collection, tests, and more. + +Next: [Full Counter Module Walkthrough →](/sdk/v0.54/tutorials/example/04-counter-walkthrough) diff --git a/sdk/v0.54/tutorials/example/04-counter-walkthrough.mdx b/sdk/v0.54/tutorials/example/04-counter-walkthrough.mdx new file mode 100644 index 000000000..14cffae1b --- /dev/null +++ b/sdk/v0.54/tutorials/example/04-counter-walkthrough.mdx @@ -0,0 +1,569 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/tutorials/example/04-counter-walkthrough' +title: Full Counter Module Walkthrough +--- + +If you came here from the module building tutorial, switch back to the `main` branch of the [`cosmos/example` repo](https://github.com/cosmos/example) first: + +```bash +git checkout main +``` + +The minimal counter you built in the previous tutorial captures the core SDK module pattern. The full `x/counter` module example in `main` follows the same pattern and adds several features on top. + +This walkthrough is meant to show you exactly what each feature is, what it does, and how you can add a similar feature to any module. + +## Minimal vs full counter + +The full counter in the `main` branch adds quite a bit of functionality to the minimal tutorial counter. + +| Feature | minimal x/counter | full x/counter | +|---|---|---| +| [State](#params-and-authority) | `count` | `count` + `params` | +| [Messages](#params-and-authority) | `Add` | `Add` + `UpdateParams` | +| [Queries](#params-and-authority) | `Count` | `Count` + `Params` | +| [Validation](#expected-keepers-and-fee-collection) | None | `MaxAddValue` limit, overflow check | +| [Fees](#expected-keepers-and-fee-collection) | None | `AddCost` charged via bank module | +| [Authority](#params-and-authority) | None | Governance-gated param updates | +| [Errors](#sentinel-errors) | Generic | Named sentinel errors | +| [Telemetry](#telemetry) | None | OpenTelemetry counter metric | +| [CLI](#autocli) | AutoCLI | AutoCLI + `EnhanceCustomCommand` | +| [Simulation](#simulation) | None | `simsx` weighted operations | +| [Block hooks](#beginblock-and-endblock) | None | `BeginBlock` + `EndBlock` | +| [Unit tests](#unit-tests) | None | Full keeper/msg/query test suite | + +The wiring code in [`msg_server.go`](https://github.com/cosmos/example/blob/main/x/counter/keeper/msg_server.go), [`query_server.go`](https://github.com/cosmos/example/blob/main/x/counter/keeper/query_server.go), [`module.go`](https://github.com/cosmos/example/blob/main/x/counter/module.go), and [`types/`](https://github.com/cosmos/example/tree/main/x/counter/types) is structurally similar between the two. Much of the new keeper logic lives in a single method: `AddCount` in [`keeper.go`](https://github.com/cosmos/example/blob/main/x/counter/keeper/keeper.go). + +## Params and authority + +A [module param](/sdk/v0.54/learn/concepts/modules#params) is on-chain configuration that controls how the module behaves without changing the code. + +The full counter adds a `Params` type that lets the chain governance configure the module's behavior at runtime. In the full module, params control how large an `Add` can be and how much it costs. + +### Where the code lives + +- [`proto/example/counter/v1/state.proto`](https://github.com/cosmos/example/blob/main/proto/example/counter/v1/state.proto) defines the `Params` type +- [`proto/example/counter/v1/tx.proto`](https://github.com/cosmos/example/blob/main/proto/example/counter/v1/tx.proto) adds the `UpdateParams` message +- [`proto/example/counter/v1/query.proto`](https://github.com/cosmos/example/blob/main/proto/example/counter/v1/query.proto) adds the `Params` query +- [`x/counter/keeper/keeper.go`](https://github.com/cosmos/example/blob/main/x/counter/keeper/keeper.go) stores the params and authority +- [`x/counter/keeper/msg_server.go`](https://github.com/cosmos/example/blob/main/x/counter/keeper/msg_server.go) checks the authority on updates +- [`x/counter/keeper/query_server.go`](https://github.com/cosmos/example/blob/main/x/counter/keeper/query_server.go) returns the current params + +### Try it + +You can inspect the current params with: + +```bash +exampled query counter params +``` + +### Add this to your module + +To add runtime-configurable params to your own module, make these changes: + +1. Define a `Params` type in proto +2. Add a privileged `UpdateParams` message +3. Add a query to read the current params +4. Store the params and authority in your keeper +5. Check the authority in `MsgServer` before writing new params + +### state.proto + +The relevant addition in [`state.proto`](https://github.com/cosmos/example/blob/main/proto/example/counter/v1/state.proto) is: + +```proto +message Params { + uint64 max_add_value = 1; + repeated cosmos.base.v1beta1.Coin add_cost = 2 [ + (gogoproto.nullable) = false, + (gogoproto.castrepeated) = "github.com/cosmos/cosmos-sdk/types.Coins", + (amino.dont_omitempty) = true + ]; +} +``` + +`MaxAddValue` caps how much a single `Add` call can increment the counter. `AddCost` sets an optional fee charged for each add operation. + +### tx.proto - UpdateParams + +The relevant addition in [`tx.proto`](https://github.com/cosmos/example/blob/main/proto/example/counter/v1/tx.proto) is: + +```proto +rpc UpdateParams(MsgUpdateParams) returns (MsgUpdateParamsResponse); + +message MsgUpdateParams { + option (cosmos.msg.v1.signer) = "authority"; + string authority = 1 [(cosmos_proto.scalar) = "cosmos.AddressString"]; + Params params = 2 [(gogoproto.nullable) = false]; +} + +message MsgUpdateParamsResponse {} +``` + +`UpdateParams` is a privileged message. Only the `authority` address can call it. By default that address is the governance module account, so params can only be changed through a governance proposal. + +### query.proto - Params + +[`query.proto`](https://github.com/cosmos/example/blob/main/proto/example/counter/v1/query.proto) adds a second query to expose the current params: + +```proto +rpc Params(QueryParamsRequest) returns (QueryParamsResponse); +``` + +### The authority pattern + +The keeper stores the authority address and checks it on every `UpdateParams` call: + +```go +type Keeper struct { + // ... + // authority is the address capable of executing a MsgUpdateParams message. + // Typically, this should be the x/gov module account. + authority string +} +``` + +```go +// msg_server.go +func (m msgServer) UpdateParams(ctx context.Context, msg *types.MsgUpdateParams) (*types.MsgUpdateParamsResponse, error) { + if m.authority != msg.Authority { + return nil, sdkerrors.Wrapf(govtypes.ErrInvalidSigner, + "invalid authority; expected %s, got %s", m.authority, msg.Authority) + } + return &types.MsgUpdateParamsResponse{}, m.SetParams(ctx, msg.Params) +} +``` + +The authority defaults to the governance module account at keeper construction: + +```go +authority: authtypes.NewModuleAddress(govtypes.ModuleName).String(), +``` + +This pattern, storing authority in the keeper and checking it in `MsgServer`, is the standard Cosmos SDK approach to governance-gated configuration. + + +## Expected keepers and fee collection + +This section shows the standard Cosmos SDK pattern for [module-to-module interaction](/sdk/v0.54/learn/concepts/modules#inter-module-access). `x/counter` uses an expected keeper to call into the bank module and charge a fee for each add operation. + +### Where the code lives + +- [`x/counter/types/expected_keepers.go`](https://github.com/cosmos/example/blob/main/x/counter/types/expected_keepers.go) defines the narrow bank keeper interface +- [`x/counter/keeper/keeper.go`](https://github.com/cosmos/example/blob/main/x/counter/keeper/keeper.go) stores the bank keeper dependency and charges the fee in `AddCount` +- [`app.go`](https://github.com/cosmos/example/blob/main/app.go) passes `app.BankKeeper` into `counterkeeper.NewKeeper` +- [`app.go`](https://github.com/cosmos/example/blob/main/app.go) adds a module account entry so the counter module can receive fees + +### app.go changes + +This feature requires two `app.go` changes: + +- add `countertypes.ModuleName: nil` to `maccPerms` +- pass `app.BankKeeper` into `counterkeeper.NewKeeper(...)` + +In [`app.go`](https://github.com/cosmos/example/blob/main/app.go), those changes look like this: + +```go +maccPerms = map[string][]string{ + // ... + countertypes.ModuleName: nil, +} +``` + +```go +app.CounterKeeper = counterkeeper.NewKeeper( + runtime.NewKVStoreService(keys[countertypes.StoreKey]), + appCodec, + app.BankKeeper, +) +``` + +### Try it + +Submit an add transaction and the configured `AddCost` fee will be charged from the sender: + +```bash +exampled tx counter add 5 --from alice --chain-id demo --yes +``` + +### Add this to your module + +To add fee collection through the bank module, make these changes: + +1. Define a narrow bank keeper interface in `types/expected_keepers.go` +2. Add a `bankKeeper` field to your keeper +3. Charge the fee inside your keeper business logic +4. Add a module account entry in `maccPerms` +5. Pass `app.BankKeeper` into your keeper constructor in `app.go` + +### expected_keepers.go + +Rather than importing the bank module directly, the counter module defines the minimal interface it needs: + +```go +// x/counter/types/expected_keepers.go +type BankKeeper interface { + SendCoinsFromAccountToModule(ctx context.Context, senderAddr sdk.AccAddress, recipientModule string, amt sdk.Coins) error +} +``` + +This keeps the dependency explicit and narrow. The counter module cannot accidentally call any other bank method. + +### Keeper struct + +```go +type Keeper struct { + Schema collections.Schema + counter collections.Item[uint64] + params collections.Item[types.Params] + bankKeeper types.BankKeeper + authority string +} +``` + +### Fee charging in AddCount + +```go +func (k *Keeper) AddCount(ctx context.Context, sender string, amount uint64) (uint64, error) { + if amount >= math.MaxUint64 { + return 0, ErrNumTooLarge + } + + params, err := k.GetParams(ctx) + if err != nil { + return 0, err + } + + if params.MaxAddValue > 0 && amount > params.MaxAddValue { + return 0, ErrExceedsMaxAdd + } + + if !params.AddCost.IsZero() { + senderAddr, err := sdk.AccAddressFromBech32(sender) + if err != nil { + return 0, err + } + if err := k.bankKeeper.SendCoinsFromAccountToModule(ctx, senderAddr, types.ModuleName, params.AddCost); err != nil { + return 0, sdkerrors.Wrap(ErrInsufficientFunds, err.Error()) + } + } + + count, err := k.GetCount(ctx) + if err != nil { + return 0, err + } + + newCount := count + amount + if err := k.counter.Set(ctx, newCount); err != nil { + return 0, err + } + + sdkCtx := sdk.UnwrapSDKContext(ctx) + sdkCtx.EventManager().EmitEvent( + sdk.NewEvent( + "count_increased", + sdk.NewAttribute("count", fmt.Sprintf("%v", newCount)), + ), + ) + + countMetric.Add(ctx, int64(amount)) + + return newCount, nil +} +``` + +All the business logic, validation, fee charging, state mutation, events, and telemetry, lives in `AddCount`. The `MsgServer` stays thin: + +```go +func (m msgServer) Add(ctx context.Context, req *types.MsgAddRequest) (*types.MsgAddResponse, error) { + newCount, err := m.AddCount(ctx, req.GetSender(), req.GetAdd()) + if err != nil { + return nil, err + } + return &types.MsgAddResponse{UpdatedCount: newCount}, nil +} +``` + +Because `AddCount` is a named keeper method, it can also be called from `BeginBlock`, governance hooks, or other modules, not just from the `MsgServer`. + +### Module accounts + +A module account is an on-chain account owned by a module instead of a user. Modules use module accounts to hold funds, receive fees, or get special permissions like minting or burning. + +Because `x/counter` receives fees from users, it needs a module account entry in [`app.go`](https://github.com/cosmos/example/blob/main/app.go): + +```go +maccPerms = map[string][]string{ + // ... + countertypes.ModuleName: nil, +} +``` + +This lives in the `maccPerms` map in [`app.go`](https://github.com/cosmos/example/blob/main/app.go). Here, `nil` means the module account can receive funds but does not get extra permissions like minting or burning. + +## Sentinel errors + +Rather than returning generic errors, `x/counter` defines named sentinel errors with registered codes. That makes failures easier to understand and easier for clients to match on programmatically. + +### Where the code lives + +- [`x/counter/keeper/errors.go`](https://github.com/cosmos/example/blob/main/x/counter/keeper/errors.go) defines the registered module errors +- [`x/counter/keeper/keeper.go`](https://github.com/cosmos/example/blob/main/x/counter/keeper/keeper.go) returns those errors from business logic checks + +```go +// keeper/errors.go +var ( + ErrNumTooLarge = errors.Register("counter", 0, "requested integer to add is too large") + ErrExceedsMaxAdd = errors.Register("counter", 1, "add value exceeds max allowed") + ErrInsufficientFunds = errors.Register("counter", 2, "insufficient funds to pay add cost") +) +``` + +Registered errors produce structured error responses on-chain that clients can match against by code, not just by string. Each error code must be unique within the module and greater than zero (code `1` is reserved for internal SDK errors). To check whether an error is of a specific sentinel type, use `errors.Is(err, ErrInsufficientFunds)` — this works correctly even when the error has been wrapped with additional context via `errorsmod.Wrap` or `errorsmod.Wrapf`. + +All validation — both stateless field checks and stateful business logic checks — should live in the `msgServer` method or the keeper function it calls. The older `ValidateBasic` method on message types is deprecated: prefer performing all validation inside the message server. If your message type does implement `ValidateBasic`, the SDK still calls it for backward compatibility, but new modules should not rely on it. + +## Telemetry + +[Telemetry](/sdk/v0.54/guides/testing/telemetry) records how often the counter is updated so you can observe module activity in an OpenTelemetry-compatible system. + +### Where the code lives + +- [`x/counter/keeper/telemetry.go`](https://github.com/cosmos/example/blob/main/x/counter/keeper/telemetry.go) defines the meter and counter metric +- [`x/counter/keeper/keeper.go`](https://github.com/cosmos/example/blob/main/x/counter/keeper/keeper.go) records the metric from `AddCount` + +```go +// x/counter/keeper/telemetry.go +var ( + meter = otel.Meter("github.com/cosmos/example/x/counter") + + countMetric metric.Int64Counter +) + +func init() { + var err error + countMetric, err = meter.Int64Counter("count") + if err != nil { + panic(err) + } +} +``` + +`countMetric.Add(ctx, int64(amount))` in `AddCount` increments an OpenTelemetry counter every time the module state is updated. This makes module activity visible in any OTel-compatible observability system. + +## AutoCLI + +[AutoCLI](/sdk/v0.54/guides/tooling/autocli) exposes the module's queries and transactions as CLI commands. The full module example keeps the same basic AutoCLI setup as the minimal module and adds the recommended setting for custom command integration. + +### Where the code lives + +- [`x/counter/autocli.go`](https://github.com/cosmos/example/blob/main/x/counter/autocli.go) defines the generated query and tx commands + +### Try it + +These commands come from the AutoCLI configuration. `count` and `add` are customized explicitly in `autocli.go`, and `params` is still available from the generated query service. + +```bash +exampled query counter count +exampled query counter params +exampled tx counter add 5 --from alice --chain-id demo --yes +``` + +Both modules use AutoCLI. The only difference is that `x/counter` sets `EnhanceCustomCommand: true`, which merges any hand-written CLI commands with the auto-generated ones. Since neither module has hand-written commands, it is a no-op here, but it is a good default for fuller modules. + +The [`autocli.go`](https://github.com/cosmos/example/blob/main/x/counter/autocli.go) file in `x/counter`: + +```go +// autocli.go +func (a AppModule) AutoCLIOptions() *autocliv1.ModuleOptions { + return &autocliv1.ModuleOptions{ + Query: &autocliv1.ServiceCommandDescriptor{ + Service: "example.counter.Query", + EnhanceCustomCommand: true, + RpcCommandOptions: []*autocliv1.RpcCommandOptions{ + {RpcMethod: "Count", Use: "count", Short: "Query the current counter value"}, + }, + }, + Tx: &autocliv1.ServiceCommandDescriptor{ + Service: "example.counter.Msg", + EnhanceCustomCommand: true, + RpcCommandOptions: []*autocliv1.RpcCommandOptions{ + {RpcMethod: "Add", Use: "add [amount]", Short: "Add to the counter", + PositionalArgs: []*autocliv1.PositionalArgDescriptor{{ProtoField: "add"}}}, + }, + }, + } +} +``` + + +## Simulation + +[Simulation](/sdk/v0.54/guides/testing/simulator) lets the SDK generate randomized transactions against the module during fuzz-style testing. + +### Where the code lives + +- [`x/counter/simulation/msg_factory.go`](https://github.com/cosmos/example/blob/main/x/counter/simulation/msg_factory.go) defines how to generate random `Add` messages +- [`x/counter/module.go`](https://github.com/cosmos/example/blob/main/x/counter/module.go) registers those weighted operations + +### Test it + +You can exercise simulation through the repo's simulation test targets described in the running and testing tutorial. + +`x/counter` implements `simsx`-based simulation, which lets the SDK's simulation framework generate random `Add` transactions during fuzz testing: + +```go +// x/counter/simulation/msg_factory.go +func MsgAddFactory() simsx.SimMsgFactoryFn[*types.MsgAddRequest] { + return func(ctx context.Context, testData *simsx.ChainDataSource, reporter simsx.SimulationReporter) ([]simsx.SimAccount, *types.MsgAddRequest) { + sender := testData.AnyAccount(reporter) + if reporter.IsSkipped() { + return nil, nil + } + + r := testData.Rand() + addAmount := uint64(r.Intn(100) + 1) + + msg := &types.MsgAddRequest{ + Sender: sender.AddressBech32, + Add: addAmount, + } + + return []simsx.SimAccount{sender}, msg + } +} +``` + +`module.go` registers this factory: + +```go +func (a AppModule) WeightedOperationsX(weights simsx.WeightSource, reg simsx.Registry) { + reg.Add(weights.Get("msg_add", 100), simulation.MsgAddFactory()) +} +``` + + +## BeginBlock and EndBlock + +These [hooks](/sdk/v0.54/learn/concepts/modules#block-hooks) let a module run code automatically at the start or end of every block. In `x/counter`, they are purposefully empty to demonstrate where and how these features can be added. + +### Where the code lives + +- [`x/counter/module.go`](https://github.com/cosmos/example/blob/main/x/counter/module.go) implements `BeginBlock` and `EndBlock` +- [`app.go`](https://github.com/cosmos/example/blob/main/app.go) adds the module to `SetOrderBeginBlockers` and `SetOrderEndBlockers` + +### app.go changes + +Because the module advertises block hooks, [`app.go`](https://github.com/cosmos/example/blob/main/app.go) must include `countertypes.ModuleName` in both blocker order lists. + +### Add this to your module + +To add begin and end blockers to your own module, make two changes: + +1. Implement the hooks in `x//module.go` +2. Add your module name to `SetOrderBeginBlockers` and `SetOrderEndBlockers` in `app.go` + +`module.go` implements `HasBeginBlocker` and `HasEndBlocker`: + +```go +func (a AppModule) BeginBlock(ctx context.Context) error { + // optional: logic to execute at the start of every block + return nil +} + +func (a AppModule) EndBlock(ctx context.Context) error { + // optional: logic to execute at the end of every block + return nil +} +``` + +In [`app.go`](https://github.com/cosmos/example/blob/main/app.go), the module is added to the blocker order lists like this: + +```go +app.ModuleManager.SetOrderBeginBlockers( + // ... + countertypes.ModuleName, +) + +app.ModuleManager.SetOrderEndBlockers( + // ... + countertypes.ModuleName, +) +``` + +`x/counter` has no per-block logic, so both methods return nil. They exist to demonstrate the pattern: modules that need per-block execution (staking, distribution) implement real logic here. For example, a counter that auto-increments every block would call `k.AddCount(ctx, 1)` from `BeginBlock` instead of exposing a message type. + +## Unit tests + +The full module example includes a real [test suite](/sdk/v0.54/learn/concepts/testing) for keeper logic, query behavior, message handling, and bank keeper interactions. + +### Where the code lives + +- [`x/counter/keeper/keeper_test.go`](https://github.com/cosmos/example/blob/main/x/counter/keeper/keeper_test.go) +- [`x/counter/keeper/msg_server_test.go`](https://github.com/cosmos/example/blob/main/x/counter/keeper/msg_server_test.go) +- [`x/counter/keeper/query_server_test.go`](https://github.com/cosmos/example/blob/main/x/counter/keeper/query_server_test.go) + +### Run them + +You can run the counter module tests directly with: + +```bash +go test ./x/counter/... +``` + +### Add this to your module + +Start with keeper, message server, and query server tests. If your module depends on another keeper, use a small mock interface like `MockBankKeeper` so you can control success and failure cases in isolation. + +`x/counter` ships a full test suite in [`x/counter/keeper/`](https://github.com/cosmos/example/tree/main/x/counter/keeper): + +| File | What it tests | +|---|---| +| `keeper_test.go` | `KeeperTestSuite` setup, `InitGenesis`, `ExportGenesis`, `GetCount`, `AddCount`, `SetParams` | +| `msg_server_test.go` | `MsgAdd`, event emission, `MsgUpdateParams` | +| `query_server_test.go` | `QueryCount`, `QueryParams` | + +All three files share the `KeeperTestSuite` struct defined in [`keeper_test.go`](https://github.com/cosmos/example/blob/main/x/counter/keeper/keeper_test.go), which sets up an isolated in-memory store, a mock bank keeper, and a real keeper instance: + +```go +type KeeperTestSuite struct { + suite.Suite + ctx sdk.Context + keeper *keeper.Keeper + queryClient types.QueryClient + msgServer types.MsgServer + bankKeeper *MockBankKeeper + authority string +} +``` + +`MockBankKeeper` lets tests control exactly what the bank keeper returns without needing a real bank module: + +```go +type MockBankKeeper struct { + SendCoinsFromAccountToModuleFn func(ctx context.Context, senderAddr sdk.AccAddress, recipientModule string, amt sdk.Coins) error +} +``` + +Tests set `SendCoinsFromAccountToModuleFn` to simulate success or failure: + +```go +s.bankKeeper.SendCoinsFromAccountToModuleFn = func(...) error { + return errors.New("insufficient funds") +} +``` + +## Gas + +`minimum-gas-prices` in `app.toml` sets the minimum fee a node requires before it will accept and relay a transaction. The local dev chain started by `make start` leaves this empty, so transactions are accepted with no fee beyond the `AddCost` module parameter. + +To require a minimum network fee, set it in `app.toml`: + +```toml +minimum-gas-prices = "0.025stake" +``` + +Transactions that don't meet the minimum will be rejected by the node before they reach your module. This is a per-node setting, not a chain-wide consensus rule, so validators on a live network each configure their own threshold. + +Next: [Running and Testing →](/sdk/v0.54/tutorials/example/05-run-and-test) diff --git a/sdk/v0.54/tutorials/example/05-run-and-test.mdx b/sdk/v0.54/tutorials/example/05-run-and-test.mdx new file mode 100644 index 000000000..b7283607a --- /dev/null +++ b/sdk/v0.54/tutorials/example/05-run-and-test.mdx @@ -0,0 +1,238 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/tutorials/example/05-run-and-test' +title: Run, Test, and Configure +description: Learn how to run and test a chain +--- + +Now that you've [built a module from scratch](/sdk/v0.54/tutorials/example/03-build-a-module) and walked through the [full counter module](/sdk/v0.54/tutorials/example/04-counter-walkthrough), the next step is learning the workflow for running and validating a production-ready chain. This page shows how to start the chain locally, interact with it through the CLI, and use the main layers of testing before shipping changes. + +## Single-node local chain + +Use a single-node chain for the fastest local development loop. It gives you one validator with predictable state so you can quickly test queries and transactions. + +### Start + +```bash +make start +``` + +This builds the binary, initializes chain data, and starts a single validator node. It handles cleanup automatically — existing chain state is reset on each run. + +The chain uses: +- Chain ID: `demo` +- Pre-funded accounts: `alice`, `bob` +- Default denomination: `stake` + +### Stop + +Press `Ctrl+C` in the terminal running `make start`. + +### Reset chain state + +```bash +make start +``` + +Re-running `make start` resets state automatically. There is no separate reset command. + +## Localnet (multi-validator) + +Use localnet when you want a setup that is closer to a real network. It runs multiple validators in Docker so you can test multi-node behavior locally. + +For a multi-validator setup using Docker: + +```bash +# Initialize localnet configuration +make localnet-init + +# Start all validators +make localnet-start + +# View logs +make localnet-logs + +# Stop +make localnet-stop + +# Clean all localnet data +make localnet-clean +``` + +## CLI reference + +Once the chain is running, these are the core [CLI](/sdk/v0.54/learn/concepts/cli-grpc-rest#cli) commands you'll use to inspect state and submit transactions. + +### Query commands + +Use query commands to read module state without changing anything on-chain. + +```bash +# Query the current counter value +exampled query counter count + +# Query the module parameters +exampled query counter params + +# Query with a specific node (if not using default localhost:26657) +exampled query counter count --node tcp://localhost:26657 +``` + +### Transaction commands + +Use transaction commands to submit state-changing messages to the chain. + +```bash +# Add to the counter +exampled tx counter add 10 --from alice --chain-id demo --yes + +# Add with a gas limit +exampled tx counter add 10 --from alice --chain-id demo --gas 200000 --yes + +# Update module parameters (requires governance authority) +exampled tx counter update-params --from alice --chain-id demo --yes +``` + +### Useful flags + +These flags are the ones you'll use most often while iterating locally. + +| Flag | Description | +|---|---| +| `--from` | Key name or address to sign with | +| `--chain-id` | Chain ID (use `demo` for local) | +| `--yes` | Skip confirmation prompt | +| `--gas` | Gas limit for the transaction | +| `--node` | RPC endpoint (default: `tcp://localhost:26657`) | +| `--output json` | Output response as JSON | + + +## Node Configuration + +When you run `make start`, the chain creates `~/.exampleapp/config/` automatically and initializes two config files inside it: + +| File | What it controls | +|---|---| +| `app.toml` | SDK application settings: gas prices, pruning, API/gRPC servers, telemetry | +| `config.toml` | CometBFT settings: peer networking, consensus timeouts, mempool, RPC | + +### app.toml + +The most common settings to change during development: + +| Setting | Default | Description | +|---|---|---| +| `minimum-gas-prices` | `"0stake"` | Minimum fee the node accepts before processing a transaction | +| `pruning` | `"default"` | How much historical state to keep (`default`, `nothing`, `everything`, `custom`) | +| `api.enable` | `true` | Enables the REST API on port 1317 | +| `grpc.enable` | `true` | Enables the gRPC server on port 9090 | + +### config.toml + +The settings most likely to change during development: + +| Setting | Default | Description | +|---|---|---| +| `moniker` | `"test"` | Human-readable name for the node | +| `log_level` | `"info"` | Log verbosity (`debug`, `info`, `error`) | +| `consensus.timeout_commit` | `"5s"` | How long to wait after a block is committed before starting the next one | +| `p2p.seeds` | `""` | Seed nodes to connect to on a live network | +| `p2p.persistent_peers` | `""` | Peers to maintain permanent connections to | + +## Unit tests + +Start here when you want fast feedback on module logic without running a chain. These tests isolate the [keeper](/sdk/v0.54/learn/concepts/testing#keeper-unit-tests) and gRPC servers from the rest of the app. + +The unit test logic lives in the counter keeper package on `main`: the shared suite setup is in [x/counter/keeper/keeper_test.go](https://github.com/cosmos/example/blob/main/x/counter/keeper/keeper_test.go), message-path tests are in [x/counter/keeper/msg_server_test.go](https://github.com/cosmos/example/blob/main/x/counter/keeper/msg_server_test.go), and query-path tests are in [x/counter/keeper/query_server_test.go](https://github.com/cosmos/example/blob/main/x/counter/keeper/query_server_test.go). + +The keeper test suite covers the keeper, msg server, and query server in isolation using an in-memory store and a mock bank keeper. No running chain is required. + +```bash +go test ./x/counter/... +``` + +To run with verbose output: + +```bash +go test -v ./x/counter/... +``` + +To run a specific test: + +```bash +go test -v -run TestKeeperTestSuite/TestAddCount ./x/counter/... +``` + +The test suite is structured around three files: + +| File | Tests | +|---|---| +| `keeper/keeper_test.go` | Genesis, `GetCount`, `AddCount`, `SetParams` | +| `keeper/msg_server_test.go` | `MsgAdd`, event emission, `MsgUpdateParams` | +| `keeper/query_server_test.go` | `QueryCount`, `QueryParams` | + +## E2E tests + +Run [E2E tests](/sdk/v0.54/learn/concepts/testing#integration-tests) when you want to verify the full request path against a real node. They give you higher confidence than unit tests, but take longer to complete. + +The E2E logic lives on `main` in [tests/counter_test.go](https://github.com/cosmos/example/blob/main/tests/counter_test.go), which starts an in-process network, builds signed transactions, and verifies query results. The shared network fixture it uses is defined in [tests/test_helpers.go](https://github.com/cosmos/example/blob/main/tests/test_helpers.go). + +The E2E test suite starts a real in-process validator network and submits actual transactions against it. This tests the full stack: transaction encoding, message routing, keeper logic, and query responses. + +```bash +go test -v -run TestE2ETestSuite ./tests/... +``` + +E2E tests take longer than unit tests because they spin up a real node. Run them before merging significant changes. + +## Simulation tests + +[Simulation tests](/sdk/v0.54/learn/concepts/testing#simulation-tests) stress the chain with randomized activity to catch edge cases that targeted tests can miss. In this repo, that simulation flow is built with `simsx`, the Cosmos SDK's higher-level simulation framework for defining random on-chain activity at the module level. + +The top-level simulation test commands on `main` run through [sim_test.go](https://github.com/cosmos/example/blob/main/sim_test.go). The counter module's `simsx` registration lives in [x/counter/module.go](https://github.com/cosmos/example/blob/main/x/counter/module.go), the random `MsgAdd` generation lives in [x/counter/simulation/msg_factory.go](https://github.com/cosmos/example/blob/main/x/counter/simulation/msg_factory.go), and randomized counter genesis lives in [x/counter/simulation/genesis.go](https://github.com/cosmos/example/blob/main/x/counter/simulation/genesis.go). + +In practice, `simsx` lets each module describe three things: how to generate random starting state, which operations can happen during simulation, and how often each operation should be chosen. For `x/counter`, that means generating a random initial counter value, registering `MsgAdd` as a simulation operation, and assigning it a weight so the simulator knows how frequently to try it relative to other module operations. + +When you run a simulation target, the test harness repeatedly builds app instances, creates random accounts and balances, generates random transactions from the registered module operations, and executes them over many blocks. That makes `simsx` useful for catching issues that are hard to cover with hand-written tests, like state machine bugs, unexpected panics, invariant violations, and non-deterministic behavior across runs. + +Simulation runs the chain with randomly generated transactions to detect non-determinism and invariant violations. + +```bash +# Full simulation +make test-sim-full + +# Determinism check +make test-sim-determinism + +# All simulation tests +make test-sim +``` + +Simulation requires the `sims` build tag, which the Makefile targets handle automatically. + +## Lint + +Linting is the quickest way to catch style problems and common code-quality issues before CI or code review does. + +The lint commands are defined in the repo [Makefile](https://github.com/cosmos/example/blob/main/Makefile), which installs `golangci-lint` and runs it across the full module tree. + +```bash +make lint +``` + +This installs and runs `golangci-lint` across the repository. To auto-fix issues where possible: + +```bash +make lint-fix +``` + +## Test summary + +Use this table as a quick reference for choosing the right validation command for the kind of change you made. + +| Command | What it validates | +|---|---| +| `go test ./x/counter/...` | Keeper, MsgServer, QueryServer in isolation | +| `go test -run TestE2ETestSuite ./tests/...` | Full transaction and query flow on a live node | +| `make test-sim-full` | Non-determinism and invariant violations | +| `make lint` | Code style and static analysis | diff --git a/sdk/v0.54/upgrade/release.mdx b/sdk/v0.54/upgrade/release.mdx new file mode 100644 index 000000000..447afb958 --- /dev/null +++ b/sdk/v0.54/upgrade/release.mdx @@ -0,0 +1,49 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/upgrade/release' +title: "v0.54 Release Notes" +description: "What's new in the latest Cosmos SDK release, including performance improvements, new features, and removals." +--- + + + If you are upgrading to v0.54, see the [upgrade guide](/sdk/v0.54/upgrade/upgrade). For a full list of changes, see the [changelog](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/CHANGELOG.md). + + +## Overview + +This release introduces order of magnitude improvements to network stability and throughput. In testing, we are able to support sustained 1K TPS on a variety of network configurations with no degradation in block time, whereas previously block production would have slowed / halted almost immediately after 200+ TPS. + +This is made possible through 2 critical performance improvements targeting different layers of the stack: + +- **Parallel transactions (BlockSTM)**: When applied to blocks containing fully parallelizable transactions, Block STM shows between 5-10x improvements in execution time depending on the available CPUs, size of the blocks, and types of transactions being run. We have modified the underlying implementations of Cosmos bank sends and EVM native sends to ensure they are parallelizable, so you will benefit from speed ups of these transactions immediately. It is possible to do the same for other common kinds of Cosmos transactions (e.g. governance, staking, auth), but we haven’t optimized them yet. Custom transaction types and EVM smart contracts may similarly require implementation modifications to benefit from parallelization. See our guide [here](/sdk/v0.54/experimental/blockstm) for more information. +- **Enhanced Networking (LibP2P):** The lib-p2p based reactor implementation outperforms Comet’s existing p2p implementation on latency benchmarks across a variety of workloads, reducing p99 latency metrics by a factor of 100 and up to 1000 in some cases. libp2p is industry-standard in peer-to-peer data exchange. Under the hood, it leverages QUIC, a modern low-latency UDP-based communication protocol. At this time, lib-p2p is meant for usage in centrally managed Cosmos networks, as peer exchange and upgradeability from comet’s networking stack are not supported yet. Please reach out if you are interested in testing libp2p in devnet or testnet environments and potentially contributing these improvements. We want to work closely with teams to gather feedback. See the [LibP2P guide](/cometbft/latest/docs/experimental/lib-p2p) for more information. + +## Additional Features + +1. **AdaptiveSync** helps nodes catchup when they fall behind by letting consensus and blocksync work simultaneously. During traffic spikes or short block times, this keeps nodes progressing with the network while preserving normal consensus safety and finality behavior. Especially valuable for RPC-heavy nodes. See the [block sync guide](/cometbft/latest/docs/core/block-sync#adaptivesync) for more information. +2. **Log/v2** supports the transition of the Cosmos SDK’s observability to OpenTelemetry, enabling automatic trace correlation across all log output (show via the logged keys `trace_id`, `span_id`, and `trace_flags`, if a span is present in the `ctx`). This is powered by four new required contextual logging methods on the `Logger` interface (`InfoContext`, `WarnContext`, etc). Additionally, a new `MultiLogger` allows fanning out to multiple logging backends simultaneously, which the server now uses automatically when OpenTelemetry is configured. See the [logging guide](/sdk/v0.54/guides/testing/log) and [telemetry guide](/sdk/v0.54/guides/testing/telemetry) for more information. +3. **IBC General Message Passing (GMP)**: General Message Passing in IBC enables calling arbitrary smart contracts on remote networks. Unlike Interchain Accounts, the caller does not need to own an account on the destination chain (though it is general enough to support this usage pattern). Instead, GMP directly calls contracts on the destination chain. This makes it especially useful for implementing mint/burn bridges (See [below](#upcoming-features-available-soon-in-minor-releases) for more details) + +## Enterprise Features + +The following features are released as part of [Cosmos Enterprise](/sdk/v0.54/enterprise/overview): + +1. The **Groups module** enables on-chain multisig and collective decision-making for any set of accounts. Groups are formed with weighted members and one or more configurable decision policies that define how proposals pass. Members submit proposals containing arbitrary SDK messages, vote, and any account can trigger execution once a proposal is accepted. Two built-in decision policies are included: threshold (absolute weighted vote count) and percentage (proportion of YES votes), each with configurable voting and minimum execution periods. The decision policy interface supports custom extensions. See the [Groups module docs](/sdk/v0.54/enterprise/group/overview) for more information. +2. The **POA module** provides an admin-managed validator set as a drop-in replacement for the staking, distribution, and slashing modules. Purpose-built for institutional deployments run by a known set of operators, it offers a streamlined validator lifecycle with no native token required. Fee distribution to validators and full governance compatibility are included out of the box. See the [POA module docs](/sdk/v0.54/enterprise/poa/overview) for more information. + +## Upcoming Features (Available soon in Minor Releases) + +1. **Krakatoa mempool (Cosmos EVM only)**: This mempool significantly improves transaction throughput and network stability by making the comet mempool stateless and introducing two new concurrent ABCI methods for transaction processing (`reapTxs` and `insertTx`). The upshot is that transaction processing is more concurrent and more lightweight, resulting in performance and stability gains. This will be available for Cosmos EVM chains at the end of April. +2. **Interchain Fungible Token Standard (IFT):** This is a more modern and flexible approach to token transfers in IBC compared to ICS20 that enables mint/burn based bridging. IFT decouples the contract or module that mints a token from the IBC channel. Importantly, this allows token issuers to establish canonical, owned deployments of their tokens on any networks they choose and manage cross-chain mints/burns with IBC, rather than using “wrapped” tokens that they cannot control. It also allows a single token to support fungibility over multiple IBC paths and to upgrade/change the IBC connection in the background without worrying about the “token path” changing. This is coming shortly to ibc-go, ibc-solidity, and ibc-sol. +3. **IBC support for any EVM network:** IBC functionality will extend directly to any EVM network as a collection of Solidity contracts that implement IBC Eureka. This will enable direct IBC connectivity without requiring any modifications to the EVM chain. This means Ethereum, Base, Arbitrum, Optimism, and other EVM networks can participate directly in IBC transfers. Combined with IFT, token issuers can manage canonical token deployments across Cosmos and any number of EVM chains from a single source of truth. +4. **IBC support for Solana:** Similar to EVM support, IBC connectivity will extend to Solana with a native program implementation. This will allow Solana to participate directly in IBC transfers with Cosmos and EVM chains, enabling cross-ecosystem token movement without wrapped tokens or intermediary chains. +5. **IBC v2 relayer:** A standalone, production-ready, request-driven relayer service for the IBC v2 protocol. This relayer will support interoperating between a Cosmos-based chain and major EVM networks (Ethereum, Base, Optimism, Arbitrum, Polygon, and more). Operators submit a source transaction hash and can track each packet's status in real time, from submission through relay completion, with full retry and failure recovery handled automatically. + +## Removals + +The following features have been removed from this release family: + +- **ibc-apps/async-icq:** We have never had official support for ibc-apps/async-icq middleware. This is us just stating this explicitly. We will not be updating it as a part of this release or going forward. We will not be testing its compatibility with IBC-go v11.0.0 +- **ibc-apps/pfm (packet forwarding middleware):** We have never had official support for PFM , but historically, we did update it and make a best effort to ensure compatibility with IBC in during previous release cycles. We will not be doing that as a part of this release or going forward. Instead, we are upstreaming PFM into IBC-Go to streamline our support. We will guarantee equivalent functionality and APIs as part of this migration. The upstreamed version will be available for you to migrate to in IBC-go v11.1.0, which we are planning to release towards the end of April 2026. +- **ibc-apps/rate-limits:** We have never had official support for ibc-apps/rate-limits middleware, but historically, we did update it and make a best effort to ensure compatibility with IBC in during previous release cycles. We will not be doing that as a part of this release or going forward. Instead, we are upstreaming PFM into IBC-Go to streamline our support. We will guarantee equivalent functionality and APIs as part of this migration. The upstreamed version will be available for you to migrate to in IBC-go v11.2.0, which we are planning to release in the first weeks of May 2026. +- **ibc-apps/ibc-hooks:** We have never had official support for ibc-apps/ibc-hooks middleware, but historically, we did update it and make a best effort to ensure compatibility with IBC in during previous release cycles. We will not be doing that as a part of this release or going forward. Instead, we are introducing and will maintain a new `callbacks` middleware that enables calling Cosmwasm contracts (like ibc-hooks) as well as Cosmos modules and EVM contracts when processing ICS20 packets. We are working to ensure the upcoming wasmd release will enable Cosmwasm contracts to adopt this without changing contract interfaces. \ No newline at end of file diff --git a/sdk/v0.54/upgrade/upgrade.mdx b/sdk/v0.54/upgrade/upgrade.mdx new file mode 100644 index 000000000..dfdb4eaa4 --- /dev/null +++ b/sdk/v0.54/upgrade/upgrade.mdx @@ -0,0 +1,515 @@ +--- +noindex: true +canonical: 'https://docs.cosmos.network/sdk/latest/upgrade/upgrade' +title: "v0.54 Upgrade Guide" +description: "Reference for upgrading from v0.53 to v0.54 of Cosmos SDK" +--- + +This document provides a reference for upgrading from `v0.53.x` to `v0.54.x` of Cosmos SDK. + +However, this guide is not exhaustive for all breaking changes. For a comprehensive list of all breaking changes in v0.54.0, see the [Changelog](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/CHANGELOG.md). + +Always read the [App Wiring Changes](#app-wiring-changes) section for more information on application wiring updates. + +## Table of Contents + +- [Upgrade Checklist](#upgrade-checklist) +- [Required Changes](#required-changes) + - [App Wiring Changes](#app-wiring-changes) + - [x/gov](#x/gov) + - [Keeper Initialization](#keeper-initialization) + - [GovHooks Interface](#govhooks-interface) + - [x/epochs](#x/epochs) + - [x/bank](#x/bank) + - [NodeService](#nodeservice) + - [Removed Go Modules](#removed-go-modules) + - [Renamed Go Modules](#renamed-go-modules) + - [Module Version Updates](#module-version-updates) + - [Log v2](#log-v2) + - [Store v2](#store-v2) +- [Conditional Changes](#conditional-changes) + - [Module Deprecations](#module-deprecations) + - [x/circuit](#x/circuit) + - [x/nft](#x/nft) + - [x/crisis](#x/crisis) + - [Cosmos Enterprise](#cosmos-enterprise) + - [Groups Module](#groups-module) + - [PoA Module](#poa-module) +- [New Features and Non-Breaking Changes](#new-features-and-non-breaking-changes) + - [Telemetry](#telemetry) + - [OpenTelemetry](#opentelemetry) + - [Centralized Authority via Consensus Params](#centralized-authority-via-consensus-params) + - [How AuthorityParams Works](#how-authorityparams-works) +- [Upgrade Handler](#upgrade-handler) +- [IBC v11 Updates](#ibc-v11-updates) +- [Cosmos Performance Upgrades (Experimental)](#cosmos-performance-upgrades-experimental) + - [Cosmos SDK](#cosmos-sdk) + - [BlockSTM](#blockstm) + - [CometBFT v0.39 Updates](#cometbft-v039-updates) + - [LibP2P](#libp2p) + - [`AdaptiveSync`](#adaptivesync) + + +## Upgrade Checklist + +Use this checklist first, then read the linked sections for the exact code or wiring changes. + +- [ ] Update `x/gov` keeper wiring, as the `x/gov` module has been decoupled from `x/staking`. See [Keeper Initialization](#keeper-initialization). +- [ ] Update your governance hooks if you implement `AfterProposalSubmission`. See [GovHooks Interface](#govhooks-interface). +- [ ] Update `x/epochs.NewAppModule` if your app includes `x/epochs`. See [x/epochs](#x/epochs). +- [ ] Put `x/bank` first in `SetOrderEndBlockers`. See [x/bank](#x/bank). +- [ ] Update your node service registration if your app exposes `NodeService`. See [NodeService](#nodeservice). +- [ ] Migrate imports for removed `x/` Go modules. See [Removed Go Modules](#removed-go-modules). +- [ ] Update required Cosmos SDK Go module dependencies. See [Module Version Updates](#module-version-updates). +- [ ] Migrate to `contrib/` imports if you use `x/circuit`, `x/nft`, or `x/crisis`. See [Module Deprecations](#module-deprecations). +- [ ] Migrate to Cosmos Enterprise if you use the `x/group` module. See [Groups Module](#groups-module). +- [ ] Update imports to `cosmossdk.io/log/v2` if your app imports the log package directly. See [Log v2](#log-v2). +- [ ] Migrate imports to `github.com/cosmos/cosmos-sdk/store/v2`. See [Store v2](#store-v2). +- [ ] Migrate any remaining `BaseApp.NewUncachedContext()` usage. See [Store v2](#store-v2). +- [ ] If using `systemtests` update import to `github.com/cosmos/cosmos-sdk/tools/systemtests`. See [Renamed Go Modules](#renamed-go-modules). +- [ ] Review [IBC v11 Updates](#ibc-v11-updates) if your chain uses IBC. Several APIs have been removed. +- [ ] Review [Centralized Authority via Consensus Params](#centralized-authority-via-consensus-params). No upgrade action is required to keep using per-keeper authorities. +- [ ] Review [Telemetry](#telemetry). No upgrade action is required to keep existing telemetry wiring, but upgrading to OpenTelemetry is strongly encouraged. +- [ ] Review [PoA Module](#poa-module) if you are interested in adopting the new Cosmos Enterprise Proof of Authority module. +- [ ] Review [Cosmos Performance Upgrades (Experimental)](#cosmos-performance-upgrades-experimental) if you are interested in experimenting with BlockSTM, LibP2P, or AdaptiveSync. + +## Required Changes + +All chains upgrading to `v0.54.x` should review and apply the changes in this section. + +This guide provides an overview of the major changes in v0.54.0. However, this guide is not exhaustive for all breaking changes. For a comprehensive list of all breaking changes in v0.54.0, see the [Changelog](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/CHANGELOG.md). + +### App Wiring Changes + +#### x/gov + +#### Keeper Initialization + +The `x/gov` module has been decoupled from `x/staking`. The `keeper.NewKeeper` constructor now requires a `CalculateVoteResultsAndVotingPowerFn` parameter instead of a `StakingKeeper`. + +**Before:** +```go +govKeeper := govkeeper.NewKeeper( + appCodec, + runtime.NewKVStoreService(keys[govtypes.StoreKey]), + app.AccountKeeper, + app.BankKeeper, + app.StakingKeeper, // REMOVED IN v0.54 + app.DistrKeeper, + app.MsgServiceRouter(), + govConfig, + authtypes.NewModuleAddress(govtypes.ModuleName).String(), +) +``` + +**After:** +```go +govKeeper := govkeeper.NewKeeper( + appCodec, + runtime.NewKVStoreService(keys[govtypes.StoreKey]), + app.AccountKeeper, + app.BankKeeper, + app.DistrKeeper, + app.MsgServiceRouter(), + govConfig, + authtypes.NewModuleAddress(govtypes.ModuleName).String(), + govkeeper.NewDefaultCalculateVoteResultsAndVotingPower(app.StakingKeeper), // ADDED IN v0.54 +) +``` + +For applications using depinject, the governance module now accepts an optional `CalculateVoteResultsAndVotingPowerFn`. If not provided, it will use the `StakingKeeper` (also optional) to create the default function. + +#### GovHooks Interface + +The `AfterProposalSubmission` hook now includes the proposer address as a parameter. + +**Before:** +```go +func (h MyGovHooks) AfterProposalSubmission(ctx context.Context, proposalID uint64) error { + // implementation +} +``` + +**After:** +```go +func (h MyGovHooks) AfterProposalSubmission(ctx context.Context, proposalID uint64, proposerAddr sdk.AccAddress) error { + // implementation +} +``` + +#### x/epochs + +The epochs module's `NewAppModule` function now requires the epoch keeper by pointer instead of value, fixing a bug related to setting hooks via depinject. + +#### x/bank + +The bank module now contains an `EndBlock` method to support the new BlockSTM experimental package. BlockSTM requires coordinating object store access across parallel execution workers, and `x/bank`'s `EndBlock` handles the finalization step for that. **All applications must make this change**, whether or not they enable BlockSTM, because the `EndBlock` registration is now part of the module's standard lifecycle. + +```go + app.ModuleManager.SetOrderEndBlockers( + banktypes.ModuleName, + // other modules... +) +``` + +#### NodeService + +The node service has been updated to return the node's earliest store height in the `Status` query. Please update your registration with the following code (make sure you are already updated to `github.com/cosmos/cosmos-sdk/store/v2`): + +```go +func (app *SimApp) RegisterNodeService(clientCtx client.Context, cfg config.Config) { + nodeservice.RegisterNodeService(clientCtx, app.GRPCQueryRouter(), cfg, func() int64 { + return app.CommitMultiStore().EarliestVersion() + }) +} +``` + +### Removed Go Modules + +Most `cosmossdk.io` vanity URLs for modules under `x/` have been removed. These separate Go modules caused dependency version management to be unpredictable; different modules could be pinned to different SDK versions, leading to compatibility issues. Consolidating everything under `github.com/cosmos/cosmos-sdk` gives developers a single, versioned dependency to manage. + +The following must be updated: + +- `cosmossdk.io/x/evidence` -> `github.com/cosmos/cosmos-sdk/x/evidence` +- `cosmossdk.io/x/feegrant` -> `github.com/cosmos/cosmos-sdk/x/feegrant` +- `cosmossdk.io/x/upgrade` -> `github.com/cosmos/cosmos-sdk/x/upgrade` +- `cosmossdk.io/x/tx` -> `github.com/cosmos/cosmos-sdk/x/tx` + +### Renamed Go Modules + +The `cosmossdk.io/systemtests` go module is now named `github.com/cosmos/cosmos-sdk/tools/systemtests`. + + +### Module Version Updates + +- `cosmossdk.io/client/v2` has been updated to v2.11.0 + +### Log v2 + +The log package has been updated to `v2`. Applications using v0.54.0+ of Cosmos SDK will be required to update imports to `cosmossdk.io/log/v2`. Usage of the logger itself does not need to be updated. +The v2 release of log adds contextual methods to the logger interface (InfoContext, DebugContext, etc.), allowing logs to be correlated with OpenTelemetry traces. +To learn more about the new features offered in `log/v2`, as well as setting up log correlation, see the [log package documentation](https://docs.cosmos.network/sdk/latest/guides/testing/log). + +### Store v2 + +Store v2 introduces breaking changes. For a comprehensive list of all breaking changes, see the [Changelog](https://github.com/cosmos/cosmos-sdk/blob/release/v0.54.x/CHANGELOG.md). + +The store package has been updated to `v2`. Applications using v0.54.0+ of +Cosmos SDK will be required to update imports to +`github.com/cosmos/cosmos-sdk/store/v2`. + +`BaseApp.NewUncachedContext()` was deprecated as part of this work. With store v2, writes must go through a cache/branch first; the SDK no longer exposes a helper that lets applications write directly against the root `CommitMultiStore`. + +If you previously used `BaseApp.NewUncachedContext()` in tests: + +- Replace `app.NewUncachedContext(false, header)` with `app.NewNextBlockContext(header)` when the test needs a writable context between `Commit()` and the next `FinalizeBlock()`. +- Replace `app.NewUncachedContext(true, header)` with `app.NewContext(true)` or `app.NewContextLegacy(true, header)` when the test only needs the `CheckTx` state. + +Below is an example of migrating away from `NewUncachedContext`. +```go +func TestApp(t *testing.T) { + db := dbm.NewMemDB() + logger := log.NewTestLogger(t) + app := NewSimappWithCustomOptions(t, false, SetupOptions{ + Logger: logger.With("instance", "first"), + DB: db, + AppOpts: simtestutil.NewAppOptionsWithFlagHome(t.TempDir()), + }) + + /* + Before the updates, most code would look like: + + ctx := gaiaApp.NewUncachedContext(true, tmproto.Header{}) // CheckTx context + app.MyKeeper.MyMethod(ctx, ...) + + The main thing to be aware of is when you are using checkTx state and finalizeState. + NewNextBlockContext will overwrite the finalize state and return a context that writes to that state. + Reading from checkTx state without committing will not reflect the changes made in finalizeBlock state UNLESS you have committed. + */ + ctx := app.BaseApp.NewNextBlockContext(cmtproto.Header{}) // gets finalize block state + app.BankKeeper.SetSendEnabled(ctx, "foobar", true) + _, err := app.Commit() // commit the out-of-band changes. + require.NoError(t, err) + + // since we committed, we can now read the out-of-band changes via checkTx state. + // If we didn't commit above, we could read this value by passing `false` to NewContext, which would give us a handle + // on the finalize block state. However, if you DID commit like we did above, you MUST use `true` here. + res, err := app.BankKeeper.SendEnabled(app.BaseApp.NewContext(true), &banktypes.QuerySendEnabledRequest{ + Denoms: []string{"foobar"}, + Pagination: nil, + }) + require.NoError(t, err) + require.Len(t, res.SendEnabled, 1) + require.Equal(t, "foobar", res.SendEnabled[0].Denom) +} +``` + +## Conditional Changes + +These changes apply if your chain uses the affected modules, packages, or integrations. + +### Module Deprecations + +Cosmos SDK v0.54.0 drops support for the circuit, nft, and crisis modules. Developers can still use these modules, +however, they will no longer be actively maintained by Cosmos Labs. + +#### x/circuit + +The circuit module is no longer being actively maintained by Cosmos Labs and was moved to `contrib/x/circuit`. + +#### x/nft + +The nft module is no longer being actively maintained by Cosmos Labs and was moved to `contrib/x/nft`. + +#### x/crisis + +The crisis module is no longer being actively maintained by Cosmos Labs and was moved to `contrib/x/crisis`. + +### Cosmos Enterprise + +[Cosmos Enterprise modules](/sdk/v0.54/enterprise/overview) are hardened Cosmos SDK modules for permissioned and production networks. The module source is published under the Source Available Evaluation License, and production use requires an Enterprise License from Cosmos Labs. + +#### Groups Module + +The groups module is now maintained under the Cosmos Enterprise offering. If your application uses `x/group`, you will need to migrate your code to the Enterprise-distributed package and obtain a Cosmos Enterprise license to continue using it. Please see [Cosmos Enterprise](/sdk/v0.54/enterprise/overview) to learn more. + +#### PoA Module + +Cosmos SDK v0.54 includes a Proof of Authority (POA) module under the Cosmos Enterprise offering. Please see [Cosmos Enterprise](/sdk/v0.54/enterprise/poa/overview) to learn more about using the PoA module in your application. + +## New Features and Non-Breaking Changes + +These changes are informational and optional to adopt during the upgrade; they are not required for a successful migration. + +### Telemetry + +The telemetry package has been deprecated and users are encouraged to switch to OpenTelemetry. + +#### OpenTelemetry + +Previously, Cosmos SDK telemetry support was provided by `github.com/hashicorp/go-metrics` which was undermaintained and only supported metrics instrumentation. + +OpenTelemetry provides an integrated solution for metrics, traces, and logging which is widely adopted and actively maintained. + +The existing wrapper functions in the `telemetry` package required acquiring mutex locks and map lookups for every metric operation which is suboptimal. OpenTelemetry's API uses atomic concurrency wherever possible and should introduce less performance overhead during metric collection. + +See the [telemetry documentation](https://docs.cosmos.network/sdk/latest/guides/testing/telemetry) to learn how to set up OpenTelemetry with Cosmos SDK v0.54.0+. + + +Below is a quick reference on setting up and using meters and traces with OpenTelemetry: + +```go +package mymodule + +import ( + "context" + + "go.opentelemetry.io/otel" + "go.opentelemetry.io/otel/attribute" + "go.opentelemetry.io/otel/codes" + "go.opentelemetry.io/otel/metric" + "go.opentelemetry.io/otel/trace" + + sdk "github.com/cosmos/cosmos-sdk/types" +) + +// Declare package-level meter and tracer using otel.Meter() and otel.Tracer(). +// Instruments should be created once at package initialization and reused. +var ( + tracer = otel.Tracer("cosmos-sdk/x/mymodule") + meter = otel.Meter("cosmos-sdk/x/mymodule") + txCounter metric.Int64Counter + latencyHist metric.Float64Histogram +) + +func init() { + var err error + txCounter, err = meter.Int64Counter( + "mymodule.tx.count", + metric.WithDescription("Number of transactions processed"), + ) + if err != nil { + panic(err) + } + latencyHist, err = meter.Float64Histogram( + "mymodule.tx.latency", + metric.WithDescription("Transaction processing latency"), + metric.WithUnit("ms"), + ) + if err != nil { + panic(err) + } +} + +// ExampleWithContext demonstrates tracing with a standard context.Context. +// Use tracer.Start directly when you have a Go context. +func ExampleWithContext(ctx context.Context) error { + ctx, span := tracer.Start(ctx, "ExampleWithContext", + trace.WithAttributes(attribute.String("key", "value")), + ) + defer span.End() + + // Record metrics + txCounter.Add(ctx, 1) + + if err := doWork(ctx); err != nil { + span.RecordError(err) + span.SetStatus(codes.Error, err.Error()) + return err + } + + return nil +} + +// ExampleWithSDKContext demonstrates tracing with sdk.Context. +// Use ctx.StartSpan to properly propagate the span through the SDK context. +func ExampleWithSDKContext(ctx sdk.Context) error { + ctx, span := ctx.StartSpan(tracer, "ExampleWithSDKContext", + trace.WithAttributes(attribute.String("module", "mymodule")), + ) + defer span.End() + + // Record metrics (sdk.Context implements context.Context) + txCounter.Add(ctx, 1) + + // Create child spans for sub-operations + ctx, childSpan := ctx.StartSpan(tracer, "ExampleWithSDKContext.SubOperation") + // ... do sub-operation work ... + childSpan.End() + + return nil +} +``` + +### Centralized Authority via Consensus Params + +Authority management can now be centralized via the `x/consensus` module. A new `AuthorityParams` field in `ConsensusParams` stores the authority address on-chain. When set, it takes precedence over the per-keeper authority parameter. + +**This feature introduces no breaking changes**: Keeper constructors still accept the `authority` parameter. It is now used as a **fallback** when no authority is configured in consensus params. Existing code continues to work without changes. + +#### How AuthorityParams Works + +When a module validates authority (e.g., in `UpdateParams`), it checks consensus params first. If no authority is set there, it falls back to the keeper's `authority` field: + +```go +authority := sdkCtx.Authority() // from consensus params +if authority == "" { + authority = k.authority // fallback to keeper field +} +if authority != msg.Authority { + return nil, errors.Wrapf(...) +} +``` + +To enable centralized authority, set the `AuthorityParams` in consensus params via a governance proposal targeting the `x/consensus` module's `MsgUpdateParams`. + +## Upgrade Handler + +This section provides a reference example for implementing the on-chain upgrade itself. + +The following is an example upgrade handler for upgrading from **v0.53.6** to **v0.54.0**. + +```go +const UpgradeName = "v0.53.6-to-v0.54.0" + +func (app SimApp) RegisterUpgradeHandlers() { + app.UpgradeKeeper.SetUpgradeHandler( + UpgradeName, + func(ctx context.Context, _ upgradetypes.Plan, fromVM module.VersionMap) (module.VersionMap, error) { + return app.ModuleManager.RunMigrations(ctx, app.Configurator(), fromVM) + }, + ) + + upgradeInfo, err := app.UpgradeKeeper.ReadUpgradeInfoFromDisk() + if err != nil { + panic(err) + } + + if upgradeInfo.Name == UpgradeName && !app.UpgradeKeeper.IsSkipHeight(upgradeInfo.Height) { + storeUpgrades := storetypes.StoreUpgrades{ + Added: []string{}, + } + // configure store loader that checks if version == upgradeHeight and applies store upgrades + app.SetStoreLoader(upgradetypes.UpgradeStoreLoader(upgradeInfo.Height, &storeUpgrades)) + } +} +``` + +## IBC v11 Updates + +IBC v11 introduces several improvements, removes long-deprecated APIs (`ParamSubspace` from all Keeper constructors, `MsgSubmitMisbehaviour`, and `ibcwasmtypes.Checksums`), and adds custom address codec support in the transfer module to enable Cosmos EVM compatibility with IBC transfers. + +Read the [Changelog](https://github.com/cosmos/ibc-go/blob/main/CHANGELOG.md) and [v11 Migration Guide](https://docs.cosmos.network/ibc/latest/migrations/v10-to-v11) for more information. + +## Cosmos Performance Upgrades (Experimental) + +For Q1 of 2026, Cosmos Labs has been focusing on greatly improving performance of Cosmos SDK applications. v0.54 of Cosmos SDK introduces support for several performance-related features across the stack. The SDK introduces [BlockSTM](#blockstm) for concurrent transactions, and CometBFT introduces [LibP2P](#libp2p) and [`AdaptiveSync`](#adaptivesync). + +NOTE: It is important to emphasize that the following are **experimental** features. We DO NOT recommend running chains with these features enabled in production without extensive testing. + +### Cosmos SDK + +#### BlockSTM + +BlockSTM enables deterministic, concurrent execution of transactions, improving block execution speeds and throughput. + +Developers interested in experimenting with BlockSTM should read the [documentation](https://docs.cosmos.network/sdk/latest/experimental/blockstm). + +Below is an example of setting up BlockSTM: + +> **⚠️ Warning:** BlockSTM is experimental. Ensure thorough testing before enabling in production. + +```go +import ( + "runtime" + + "github.com/cosmos/cosmos-sdk/baseapp/blockstm" +) + +oKeys := storetypes.NewObjectStoreKeys(banktypes.ObjectStoreKey) + +keys := storetypes.NewKVStoreKeys( + authtypes.StoreKey, banktypes.StoreKey, stakingtypes.StoreKey, + // ... other store keys +) + +// Collect non-transient store keys +var nonTransientKeys []storetypes.StoreKey +for _, k := range keys { + nonTransientKeys = append(nonTransientKeys, k) +} +for _, k := range oKeys { + nonTransientKeys = append(nonTransientKeys, k) +} + +// Enable BlockSTM runner +bApp.SetBlockSTMTxRunner(blockstm.NewSTMRunner( + txConfig.TxDecoder(), + nonTransientKeys, + min(runtime.GOMAXPROCS(0), runtime.NumCPU()), + true, // debug logging + sdk.DefaultBondDenom, +)) + +// Optionally disable block gas meter for better performance +bApp.SetDisableBlockGasMeter(true) + +// Set ObjectStoreKey on bank module +app.BankKeeper = app.BankKeeper.WithObjStoreKey(oKeys[banktypes.ObjectStoreKey]) +``` + +### CometBFT v0.39 Updates + +#### LibP2P + +libp2p replaces CometBFT's legacy `comet-p2p` transport layer with [go-libp2p](https://libp2p.io/). It adds native stream-oriented transport, concurrent receive pipelines, and autoscaled worker pools per reactor, reducing queue pressure and improving message flow under load. In benchmarks, libp2p has been a key contributor to reaching over 2000 TPS. Beyond raw throughput, it improves network liveness by making peer communication and block propagation more resilient under sustained congestion and sudden load spikes. + +Unlike other opt-in features, **to opt-in to libp2p, every validator in the network must upgrade together**. CometBFT p2p and libp2p are fundamentally incompatible and cannot interoperate. Because of this, a coordinated network-wide migration at a specific upgrade height is required. + +See the [libp2p page](https://docs.cosmos.network/cometbft/latest/docs/experimental/lib-p2p) in the CometBFT documentation for details. + +#### `AdaptiveSync` + +`AdaptiveSync` allows a node to run `blocksync` and consensus at the same time for faster recovery behavior. In the default flow, a node starts in `blocksync`, catches up, then switches to consensus. Under sustained load, a node can remain behind and struggle to catch up. With `adaptive_sync` enabled, consensus still works normally, but it can also ingest already available blocks from `blocksync`, allowing nodes to recover more quickly during traffic spikes. `AdaptiveSync` does not change consensus safety or finality rules. + +See the [`AdaptiveSync` documentation](https://docs.cosmos.network/cometbft/latest/docs/core/block-sync#adaptivesync) for details. diff --git a/versions.json b/versions.json index d24772fa6..56000dde3 100644 --- a/versions.json +++ b/versions.json @@ -18,6 +18,7 @@ "versions": [ "latest", "next", + "v0.54", "v0.53", "v0.50", "v0.47" @@ -25,7 +26,7 @@ "defaultVersion": "latest", "repository": "cosmos/cosmos-sdk", "changelogPath": "CHANGELOG.md", - "latestDisplayVersion": "v0.54" + "latestDisplayVersion": "v0.55" }, "ibc": { "versions": [ @@ -48,13 +49,14 @@ "versions": [ "latest", "next", + "v0.39", "v0.38", "v0.37" ], "defaultVersion": "latest", "repository": "cometbft/cometbft", "changelogPath": "CHANGELOG.md", - "latestDisplayVersion": "v0.39" + "latestDisplayVersion": "v0.40" }, "hub": { "versions": [ diff --git a/work-log/drift-backlog.md b/work-log/drift-backlog.md new file mode 100644 index 000000000..ef182f664 --- /dev/null +++ b/work-log/drift-backlog.md @@ -0,0 +1,84 @@ +# Upstream link backlog + +Regenerated after the v0.55 / v0.40 ref sweep. Counts are for `latest/` only; `next/` +mirrors it. Produced by `scripts/versioning/check-github-refs.js` and +`scripts/versioning/verify-links-live.js`. + +## Release surface is clean + +Every `cosmos-sdk` and `cometbft` link in `latest/` and `next/` returns 200, and no line +anchor points past end of file. 450 of 459 unique URLs verified 200 directly; the other 9 +were rate-limited rather than failing. Set `GITHUB_TOKEN` before running the live verifier +to avoid that, which raises the GitHub limit from 60 requests an hour to 5,000. + +SDK: 237 links on `release/v0.55.x`. CometBFT: 105 on `v0.40.x`, with nothing flagged. + +## Open: six dead third-party links + +These are not cosmos or cometbft repos, so the tree-based checker never saw them; only the +live verifier catches this class. All sit in ADR pages, where the repository being cited has +since moved or restructured. + +| page | dead URL | +| --- | --- | +| `sdk/latest/reference/architecture/adr-009-evidence-module.mdx:217` | `cosmos/ics/blob/master/ibc/1_IBC_ARCHITECTURE.md` | +| `sdk/latest/reference/architecture/adr-009-evidence-module.mdx:23` | `cosmos/ics/blob/master/ibc/2_IBC_ARCHITECTURE.md` | +| `sdk/latest/reference/architecture/adr-042-group-module.mdx:30` | `regen-network/regen-ledger/tree/master/proto/regen/group/v1alpha1` | +| `sdk/latest/reference/architecture/adr-042-group-module.mdx:30` | `regen-network/regen-ledger/tree/master/x/group` | +| `sdk/latest/reference/architecture/adr-042-group-module.mdx:34` | `regen-network/regen-ledger/tree/master/orm` | +| `sdk/latest/reference/architecture/adr-062-collections-state-layer.mdx:86` | `NibiruChain/nibiru/blob/master/x/perp/keeper/keeper.go#L31` | + +## Open: 20 ADR and RFC links not made current + +Architecture decision records legitimately cite code as it stood when the decision was taken, +so these are liveness-checked only and not bumped. All return 200. Listed so the state is +recorded rather than rediscovered. + +| verdict | page | target | +| --- | --- | --- | +| `drift` | `architecture/adr-041-in-place-store-migrations.mdx:94` | `x/bank/legacy/v043/store.go#L41` @ `36f68eb9e041` | +| `drift` | `architecture/adr-050-sign-mode-textual.mdx:329` | `tx/textual/internal/testdata/e2e.json#L2` @ `094abcd39337` | +| `drift` | `architecture/adr-050-sign-mode-textual.mdx:330` | `tx/textual/internal/testdata/e2e.json#L71` @ `094abcd39337` | +| `drift` | `architecture/adr-059-test-scopes.mdx:40` | `x/auth/client/testutil/suite.go#L44` @ `0f7e56c6f910` | +| `drift` | `architecture/adr-059-test-scopes.mdx:101` | `x/evidence/testutil/app.yaml#L1` @ `2bec9d202191` | +| `drift` | `architecture/adr-059-test-scopes.mdx:102` | `x/evidence/keeper/infraction_test.go#L42` @ `2bec9d202191` | +| `drift` | `architecture/adr-059-test-scopes.mdx:106` | `tests/integration/bank/keeper/keeper_test.go#L129` @ `8c23f6f957d1` | +| `path-gone` | `architecture/adr-030-authz-module.mdx:30` | `x/group` @ `release/v0.5` | +| `path-gone` | `architecture/adr-038-state-listening.mdx:25` | `docs/building-modules/messages-and-queries.md` @ `release/v0.4` | +| `unassessed` | `architecture/adr-022-custom-panic-handling.mdx:18` | `baseapp/baseapp.go#L539` @ `bad4ca75f58b` | +| `unassessed` | `architecture/adr-027-deterministic-protobuf-serialization.mdx:36` | `proto/cosmos/tx/v1beta1/tx.proto#L30` @ `9e85e81e0e81` | +| `unassessed` | `architecture/adr-027-deterministic-protobuf-serialization.mdx:38` | `proto/cosmos/tx/v1beta1/tx.proto#L13` @ `9e85e81e0e81` | +| `unassessed` | `architecture/adr-047-extend-upgrade-plan.mdx:28` | `proto/cosmos/upgrade/v1beta1/upgrade.proto#L12` @ `v0.44.5` | +| `unassessed` | `architecture/adr-059-test-scopes.mdx:69` | `x/bank/keeper/keeper_test.go#L94` @ `2bec9d202191` | +| `unassessed` | `architecture/adr-059-test-scopes.mdx:89` | `testutil/sims/app_helpers.go#L95` @ `2bec9d202191` | +| `unassessed` | `architecture/adr-059-test-scopes.mdx:95` | `client/grpc_query_test.go#L111` @ `2bec9d202191` | +| `unassessed` | `architecture/adr-059-test-scopes.mdx:98` | `baseapp/grpcrouter_helpers.go#L31` @ `2bec9d202191` | +| `unassessed` | `architecture/adr-059-test-scopes.mdx:101` | `x/evidence/keeper/keeper_test.go#L101` @ `2bec9d202191` | +| `unassessed` | `architecture/adr-059-test-scopes.mdx:128` | `types/module/simulation.go#L31` @ `2bec9d202191` | +| `unassessed` | `rfc/rfc-001-tx-validation.mdx:14` | `types/tx_msg.go#L16` @ `16a5404f8e00` | + +Two of those were 404 and were repointed at the newest ref where the path still exists, +`x/group` to `release/v0.53.x` and a building-modules doc to `release/v0.46.x`. + +## Deliberate pins, not backlog + +- `guides/upgrades/upgrade.mdx` cites `migrateBalanceKeys` at the `v0.54.0` tag. `x/bank/migrations/` does not exist at v0.55.x and the prose is past tense about a historical migration. +- Pages under `/latest/upgrade/` and `/latest/changelog/` keep their own version's refs. A v0.54 upgrade guide citing the v0.54 changelog is correct, and the checker now refuses to bump them. +- Three CometBFT QA pages cite the CHANGELOG of the exact alpha tag each campaign tested. +- One `main` URL in `modules/circuit/README.mdx` sits inside a verbatim copy of upstream source; rewriting it would falsify the excerpt. + +## Not in scope: 34 dead links in archived directories + +Almost all in `cometbft/v0.37/`, and of two kinds: paths under `tendermint/tendermint` that +stopped existing after the rename to CometBFT, and third-party ABCI implementations that have +since restructured. These pages carry `noindex`, so there is no search impact, and archived +directories are not edited. Recorded only because this sweep was the first thing to look. + +## What no tooling catches + +`npx mint broken-links` checks internal page paths only. It does not request external URLs and +does not validate heading anchors. `check-github-refs.js` infers liveness from the git tree, so +it only sees the two product repos and cannot tell that a `#Lnnn` anchor points past end of +file, because GitHub clamps rather than erroring. `verify-links-live.js` covers both gaps and +is the only check that sees third-party links. + diff --git a/work-log/release-versioning-v55.md b/work-log/release-versioning-v55.md new file mode 100644 index 000000000..04d2cf880 --- /dev/null +++ b/work-log/release-versioning-v55.md @@ -0,0 +1,230 @@ +# release-versioning-v55 + +Doc updates for the Cosmos SDK v0.55 release cut. Upstream references verified against `cosmos-sdk@release/v0.55.x`. + +## 2026-07-29 (GitHub ref audit: tooling, and a bug the tool caused) + +Built `scripts/versioning/check-github-refs.js` plus `check-github-refs.test.js` (40 fixture tests) and `.claude/skills/update-stale-refs/`. Both documented in `scripts/versioning/CLAUDE.md`, and the release sequence in the root `CLAUDE.md` now calls the checker at step 2 instead of describing a manual grep. + +Applied 386 rewrites across 106 files, then reverted 42 of them. The revert is the important part of this entry. + +### The tool produced the bug it was built to prevent + +Hunk-offset mapping assumes git's diff alignment is semantic. For a heavily rewritten file git aligns on textual similarity instead, so a line can sit outside every hunk while no longer meaning the same thing. Mapping `node/node.go` from v0.34.x to v0.40.x returned L684 and L730; the semantically correct lines were L699 and L764. L684 is an RPC listen-address check and L730 is a bare `return nil`, neither related to the prose. The links resolved, so nothing would have surfaced it. + +Caught by the adjudicating subagent, which read the whole page as the skill requires and checked two links the checker had already auto-fixed rather than only the ones it flagged. + +Fix: `minorGap()` plus `MAX_ANCHOR_GAP = 1`. Any anchored link whose ref gap exceeds one minor version is now flagged rather than rewritten, on the grounds that one version of drift is tractable and six is not. 42 rewrites were reverted under that rule: 22 SDK spanning v0.50.x to v0.55.x, 20 CometBFT spanning v0.34.x or v0.38.x to v0.40.x. + +Also fixed three bugs found by reading before the tool ever ran: a pure-insertion hunk shifted a line it should not have; a bare URL ending a sentence pulled the trailing period into the path; and an unanchored link, being a string prefix of the anchored link to the same path, caused the short URL's rewrite to strand the long one's line number. + +### Outcome + +| | SDK | CometBFT | +| --- | --- | --- | +| now on the shipping ref | 192 | 156 | +| retained, flagged | 56 | 24 | + +Retained refs and why they stay, so nobody re-investigates: + +- 52 at `release/v0.50.x` on `modules/group/README.mdx`. `x/group` did not move to `./contrib` as the changelog implies; at v0.55.x it is at `enterprise/group/x/group/` under an evaluation-only licence rather than Apache 2.0. The page is a verbatim copy of the upstream README, and upstream itself still pins these to v0.50.x. Retargeting would point public module docs at restrictively licensed source and would diverge from the file it is synced from. Needs a maintainer decision, options recorded below. +- 4 at `release/v0.54.x`. One is `x/bank/migrations/v2/store.go#L55`, correctly pinned: the package is deleted at v0.55.x, the link resolves, and the prose calls it an example of a past migration. The other is `RELEASE_PROCESS.md`, which 404s today and needs a decision on where to point. +- 24 CometBFT anchored links spanning two or more minor versions, held back by the new gap guard pending hand verification. + +The 48 `main` links carrying line anchors and the 22 pinned-ref drift cases are reported, not fixed, per the agreed scope. The `main` combination has no correct maintenance strategy: the ref moves continuously so the anchor is wrong within days and the link never breaks to say so. + +### Group README observations, no edit made + +`sdk/latest/modules/modules.mdx` misclassifies several modules after v0.55: Circuit and NFT are listed as SDK-maintained but moved to `contrib/x/`, `x/params` is listed as deprecated but is removed outright, and Group is listed as Supplementary though it is Enterprise. Reclassifying is editorial intent, so it was left alone. Separately, the 15 `go reference` fences on the group README render as empty code blocks: upstream uses a fence type that embeds the referenced source and the docs conversion dropped it, so the code was never rendered regardless of which ref the URL carried. + +## 2026-07-29 (ref audit completed: hand verification of the gap-guarded anchors) + +All 23 anchors held back by the gap guard were verified by hand against both refs, using the `update-stale-refs` skill via subagents. Result: 12 URL corrections applied, 11 deliberately retained, zero prose changes. + +### CometBFT: fully current + +All 180 version-tracking links now point at `v0.40.x`. Twelve were corrected by hand: + +- `blocksync/pool.go#L168` to `#L232`. `IsCaughtUp` moved; L168 was `pool.sendError` inside `removeTimedoutPeers`, so this anchor had been wrong since before the freeze. +- `node/node.go#L563` to `node/setup.go#L459`. `createTransport` changed file; the three anchored lines are byte-identical. +- `node/node.go#L974` to `#L699` and `#L1023` to `#L764`, the two the mapper had got wrong. +- `node/node.go#L987` to `#L712`. Another pre-existing mis-anchor: the prose describes the `DialPeersAsync` call for persistent peers, but L987 at v0.34.x was the comment `// Start the transport.`, which is the subject of a different page. Repointed at the line the sentence actually describes. +- `rpc/core/net.go#L47` to `#L51` and `#L87` to `#L94`; `netaddress.go#L258` to `#L259`. Ordinary shifts, each confirmed by reading both files. +- `ed25519.go#L36`, `socket_server.go#L20`, `local_client.go#L13`, `Dockerfile#L11`: ref bumped, line number unchanged and confirmed identical at both refs. + +Worth noting how few of these were simple drift. Of twelve, three were anchors that had been pointing at the wrong construct for years, and only one was a plain line shift. A ref sweep surfaces mis-anchoring as a side effect. + +### SDK: 26 links deliberately left at release/v0.50.x + +Verified, not bumped, for a reason that only became clear on inspection: the line numbers were already wrong at v0.50.x. They are inherited from v0.47-era files. `x/auth/tx/config.go#L22-L28` is exactly `NewTxConfig` at v0.47.x but a fragment of `type config struct` at v0.50.x, and `evidence.proto#L12-L32` addresses a line past the end of a 31-line file. Ten of the eleven ranges in the auth and evidence batch either straddle two declarations or truncate one. + +Three further facts make bumping the wrong move: + +- Upstream still pins these to `release/v0.50.x` with the same numbers, on both `release/v0.55.x` and `main`. These pages are conversions of upstream READMEs, so editing here forks them and a future content re-sync reverts it. +- Shifting the numbers literally would faithfully preserve a known-wrong snippet. Re-anchoring to the named construct gives the right code but is not a mechanical transform, so the checker can never reproduce it. +- Mintlify does not fetch these blocks. Upstream uses a fence type that embeds the referenced source; the docs conversion turned it into a code block containing a `// Reference: ` comment. So a wrong range costs click-through only and cannot make a rendered page wrong. + +Correct ranges are recorded should anyone want them: `client/tx_config.go#L26-L36` and `#L38-L56`, `x/auth/tx/config.go#L67-L87`, `vesting.proto#L12-L39`, `#L41-L50`, `#L52-L60`, `#L62-L72`, `#L74-L83`, `#L85-L94`, `evidence.proto#L12-L31`, `x/evidence/keeper/infraction.go#L13-L155`. + +The remaining 15 at v0.50.x are the `x/group` cluster, still pending the licensing decision recorded in the previous entry. + +### Two accuracy problems found, no edit made + +Both are spec-authoring questions rather than link fixes, and the skill's prohibition on adding prose applies. + +- `cometbft/latest/spec/abci/Requirements-for-the-Application.mdx` claims a global lock means ABCI messages "are received in sequence, one at a time". At v0.40.x `local_client.go` has three lock-free paths: `CheckTx` skips the mutex under `IsLockFreeContext`, and `InsertTx` and `ReapTxs` are documented thread-safe and take no lock. Both are now part of the exported `Application` interface and the mempool connection, so the guarantee no longer holds across the whole ABCI surface. The page does not mention either method. +- `sdk/latest/modules/evidence/README.mdx:231` says evidence reaches the application as ABCI `Evidence` in `abci.RequestBeginBlock`. That type was removed in CometBFT 0.38 and SDK 0.50; it is `FinalizeBlock` now. Upstream text. + +Also unrelated but adjacent: `cometbft/latest/docs/core/Using-CometBFT.mdx:529` uses `"cometbft/PubKeyEd25519"` where every other sample on the page, and the upstream constant the page links, use `"tendermint/PubKeyEd25519"`. + +### Verification + +`npx mint broken-links` clean. `latest`/`next` parity 0 unexplained diffs for both products. 40/40 fixture tests pass. No headings changed, so no anchor slugs are affected. Drift backlog written to `work-log/drift-backlog.md`. + +## 2026-07-29 (ref audit: final state) + +Reversed the earlier decision to leave the auth, vesting and evidence links at v0.50.x. The subagent had verified the correct v0.55.x ranges by reading both refs, so retaining a known-wrong anchor to preserve parity with an upstream file that is also wrong was the weaker choice. All 11 re-anchored to the named construct rather than shifted literally, since ten of the eleven original ranges either truncated a declaration or straddled two: + +`client/tx_config.go#L26-L36` and `#L38-L56`, `x/auth/tx/config.go#L67-L87` (was pointing at a `type config struct` fragment, not `NewTxConfig`), `vesting.proto#L12-L39`, `#L41-L50`, `#L52-L60`, `#L62-L72`, `#L74-L83`, `#L85-L94`, `evidence.proto#L12-L31` (was addressing L32 of a 31-line file), `x/evidence/keeper/infraction.go#L13-L155`. + +Consequence to be aware of: these three pages are conversions of upstream READMEs which still carry the old numbers on `main`, so a future content re-sync will revert this. The durable fix is a PR upstream against the reference fences. + +Two other links resolved: + +- `x/bank/migrations/v2/store.go#L55-L76` moved from `release/v0.54.x` to the `v0.54.0` tag. The package is deleted at v0.55.x and the prose presents it as an example of a past migration, so a pinned tag states that intent. A version-tracking ref on a branch that no longer contains the file only looked like drift. +- The dead `RELEASE_PROCESS.md` link now points at `/sdk/latest/release-family`, the docs' own canonical lifecycle page, which self-describes as the source of truth for support windows and retirement. Upstream deleted the file with no replacement, so pinning to v0.53.x would only defer the rot. Also drops a first-person "our" in passing. + +### Final state + +| | on shipping ref | retained | +| --- | --- | --- | +| SDK | 214 | 30 | +| CometBFT | 180 | 0 | + +The 30 retained are 15 links, mirrored across `latest/` and `next/`, all on `sdk/modules/group/README.mdx`. They stay pending a decision that is not a documentation call: at v0.55.x `x/group` lives at `enterprise/group/x/group/` under an evaluation-only licence rather than Apache 2.0, so retargeting would point a public module page at restrictively licensed source. All 15 anchor relocations are verified and recorded should option B be chosen. Options are in the earlier entry. + +Verified: `broken-links` clean, `latest`/`next` parity 0 for both products, 40/40 fixture tests, noindex invariants intact (0 in `latest/`, all in `next/`). No headings changed, so no anchor slugs are affected. + +## 2026-07-29 (ref audit: closed out) + +The last 15 `release/v0.50.x` links on `modules/group/README.mdx` are now pinned to the `v0.50.13` tag. This resolves the category without touching the licensing question. + +The reasoning is the same as the `x/bank/migrations` link earlier in the sweep. A version-tracking branch ref implies the page follows that branch, so once the branch is superseded the link reads as drift. These are not drift, they are a deliberate citation: `x/group` as it shipped in the final SDK release that carried it under Apache 2.0. A pinned tag says that outright. + +`x/group/types.go` and `proto/cosmos/group/v1/tx.proto` are byte-identical between `release/v0.50.x` and `v0.50.13` (same md5, same line counts, 629 and 394 lines), so every anchor is preserved exactly and no line number moved. Spot-checked one rendered link at 200. + +This deliberately does not decide the licensing question, and leaves both remaining options open: retargeting to `enterprise/group/x/group/` at v0.55.x with the 15 verified relocations, or retiring the page in favour of the existing `sdk/latest/enterprise/group/` set. Either can be done later without redoing this work. + +### Closing state + +Zero superseded version-tracking refs in `sdk/{latest,next}` and `cometbft/{latest,next}`. 214 SDK links on `release/v0.55.x`, 180 CometBFT links on `v0.40.x`. Everything else is either a pinned historical citation or a `main` ref, both of which are categories the sweep does not bump. + +Verified: `broken-links` clean, `latest`/`next` parity 0 for both products, 40/40 fixture tests, noindex invariants intact. Drift backlog at `work-log/drift-backlog.md`. + +Still open and genuinely editorial, recorded so they are not rediscovered: the `x/group` page's future (retarget or retire), the ABCI concurrency claim that v0.40.x's three lock-free paths have made too strong, and `evidence/README.mdx:231` citing `abci.RequestBeginBlock`, removed in SDK 0.50. + +## 2026-07-30 (line-anchor audit: x/staking, x/gov) + +Repointed all 34 upstream GitHub links on `sdk/{latest,next}/modules/staking/README.mdx` and `sdk/{latest,next}/modules/gov/README.mdx` from `v0.47.0-rc1` to `release/v0.55.x`, re-deriving every line range from the proto files at the new ref rather than carrying the old numbers forward. Fifteen ranges moved, eight were unchanged by coincidence, and one already-v0.55.x range was tightened off a second declaration. Every range now spans one complete declaration (or the same declaration group the prose describes) including its doc comment, verified against the fetched files. + +The 34 old ranges were, contrary to expectation, all clean at `v0.47.0-rc1`: each began at a doc-comment line and ended on a closing brace. The one exception was the `MsgBeginRedelegateResponse` link, which started one line early on a blank line. So the failure mode here was purely drift, not sloppy anchors. + +Also bumped the gov ADR-037 link off `main` to `release/v0.55.x`, where `docs/architecture/adr-037-gov-split-vote.md` still resolves. + +No prose was edited. Four accuracy defects were found and are recorded rather than fixed, three of them inherited from upstream and so at risk of being reverted by a content sync: + +- `gov/README.mdx` claims gov parameters live in a `GlobalParams` KVStore. At v0.55.x they are a single `collections.Item[v1.Params]` under `ParamsKey` (prefix 48). Upstream says the same thing. +- `gov/README.mdx` documents `DepositParams`, `VotingParams`, and `TallyParams` as the live parameter set. All three carry `option deprecated = true` at v0.55.x and the module reads `Params`. Upstream unchanged. +- `gov/README.mdx` table of contents links `#software-upgrade`, a heading the page does not have. Dead in-page anchor, inherited from upstream. +- `staking/README.mdx` table of contents omits `MsgRotateConsPubKey`, a section this repo added locally. Upstream has neither, so a Contents sync would keep dropping it. + +Separately, the `### HistoricalInfo` section of `staking/README.mdx` carries a 400-line inline copy of the whole `staking.proto` fenced as `go expandable`, where upstream references only `staking.proto#L17-L24`. The inlined copy is partly stale: `types.Dec` and `types.Int` custom types instead of `cosmossdk.io/math`, leftover `Since: cosmos-sdk 0.46` comments, and no `ConsKeyEvidenceExpiry`. That is a conversion defect, not a link, and is left for a decision on whether to trim it to the `HistoricalInfo` message. + +Both `release/v0.55.x` upstream module READMEs still carry the original `v0.47.0-rc1` links with the original line numbers, so these corrections are local and a future upstream sync will reintroduce the stale refs. + +## 2026-07-30 (line-anchor audit: x/authz, x/feegrant, x/group) + +Repointed all 32 upstream GitHub links on `sdk/{latest,next}/modules/authz/README.mdx`, `.../feegrant/README.mdx`, and `.../group/README.mdx` to `release/v0.55.x`, re-deriving every line range from the files at the new ref. The authz and feegrant links were pinned to `v0.47.0-rc1`; the group links to tag `v0.50.13`. + +`x/group` does not exist at `x/group/` on `release/v0.55.x`. The module lives at `enterprise/group/x/group/` with its protos at `enterprise/group/proto/`, so all 15 group links were repathed as well as rerefed, and switched from `/tree/` to `/blob/` so the line anchors resolve. That source carries `SPDX-License-Identifier: CosmosLabs-Evaluation-Only` under `enterprise/group/LICENSE`, not Apache 2.0, so the page now links readers at code they cannot use commercially. The page's existing Enterprise warning callout covers this, but the licence status of the linked source is not stated anywhere. + +Three old ranges were defective rather than merely drifted: authz `MsgExec` (`tx.proto#L52-L63`) cut the message's closing brace; feegrant `BasicAllowance` (`feegrant.proto#L15-L28`) stopped inside the `spend_limit` field and omitted `expiration`; feegrant `MsgRevokeAllowance` (`tx.proto#L41-L54`) straddled `MsgGrantAllowanceResponse` and `MsgRevokeAllowance`. The group `DecisionPolicy` link (`x/group/types.go#L27-L45`) started mid-`DecisionPolicyResult` and truncated the `DecisionPolicy` interface the prose promises. All 14 group proto ranges were clean at `v0.50.13` and moved only by the enterprise relocation's 13-line licence header. + +No prose was edited. Verified accurate at v0.55.x and left alone: the authz `BeginBlock` pruning cap of 200, the authz 20-gas grant-queue and 10-gas stake-authorization iteration costs, both authz state key prefixes, the feegrant `EndBlock` pruning and both queue prefixes, the feegrant 10-gas filtered-message cost, the authz CLI's `send|generic|delegate|unbond|redelegate` type list, and the group page's 14 Msg Service entries, two decision policies, and `EndBlock` tallying. + +Accuracy defects found and recorded rather than fixed: + +- `feegrant/README.mdx:83` says "There are two types of fee allowances present at the moment" above a list of three. `AllowedMsgAllowance` has its own section further down the same page. +- `feegrant/README.mdx:36` glosses `allowance` as `BasicAllowance` or `PeriodicAllowance`; the proto comment and the page's own section list include `AllowedMsgAllowance`. +- `feegrant/README.mdx` Messages section omits `Msg/PruneAllowances`, which the module exposes and whose event the page's own Events section already documents. +- `authz/README.mdx:282` describes `StakeAuthorization`'s `AuthorizationType` as delegate, undelegate, or redelegate. The enum has a fourth value, `AUTHORIZATION_TYPE_CANCEL_UNBONDING_DELEGATION`, which `Accept` and `normalizeAuthzType` both handle. The CLI does not expose it, so the CLI section is correct as written. +- `sdk/latest/modules/group/README.mdx` substantially duplicates `sdk/latest/enterprise/group/`: `architecture.mdx` repeats the same Group, Group Policy, Decision Policy, Proposal, and Pruning concepts, and `api.mdx` repeats the same 14 Msg Service entries, events, and REST endpoints. Left in place pending a decision on the page's future, already open in this log. + +The authz and feegrant embedded `go expandable` code blocks are `v0.47.0-rc1`-era snapshots: CometBFT still imported as `tendermint/tendermint`, `sdkerrors.Wrapf` for wrapping, `SetTip`/`GetTip` on the tx wrapper where the shipping proto marks `Tip` deprecated, and a `StakeAuthorization.Accept` switch missing the `MsgCancelUnbondingDelegation` case. Conversion defects rather than links, left for the same decision as the staking `HistoricalInfo` block. + +## 2026-07-30 (module and CometBFT page sweeps, by subagent) + +2026-07-30 + +- Repointed every upstream GitHub reference on ten SDK module pages (`distribution`, `bank`, `circuit`, `nft`, `upgrade`, `crisis`, `mint`, `slashing`, `evidence`, `auth/auth`) from `v0.47.0-rc1` and `main` to `release/v0.55.x`, with each line range re-derived from the file at that ref. `circuit`, `crisis`, and `nft` protos now resolve under `contrib/proto/`, matching their move to `contrib/x/`. Sixteen of the twenty-two anchored ranges shifted, and four were already malformed at the old ref: the distribution `FeePool` range truncated the closing brace and dropped the doc comment, and all three `circuit` ranges straddled message boundaries, with one pointing past end of file. Normalized `tree/` to `blob/` on file links. Synced all ten pages to `next/`. +- Left the `adr-031` URL in the circuit page's `msg_service_router.go` excerpt at `blob/main`: it is verbatim upstream source, unchanged at the release ref. +- Repointed the remaining floating and commit-pinned upstream references on ten CometBFT pages to `v0.40.x`: the four links pinned to commit `af3bc47` in the Byzantine consensus algorithm and WAL specs now carry re-derived struct ranges (`Vote`, `Commit`, `ValidatorSet`, `autofile.Group`, all four of which had shifted), the `abci/server` example link moved off the dead `master` branch, and the RFC-100, light-client spec, and ADR-025 references moved off `main`. Prose on all ten pages checked against the code at that ref and found accurate. Synced to `next/`. +- Repointed nine SDK pages (`node/run-node`, `node/txs`, `learn/concepts/baseapp`, `learn/concepts/testing`, `release-family`, `tutorials`, `guides/abci/app-mempool`, `guides/reference/bech32`, `upgrade/v0.55-release`) from `main` to `release/v0.55.x`, adding the verified `BaseApp` struct range. Synced to `next/`, which also carried four `release/v0.54.x` links on the run-node and baseapp pages forward. +- Left two historical citations pinned: the CometBFT QA v0.37 and v0.38 pages cite the CHANGELOG of the exact alpha tag each campaign tested, and the upgrade guide's `migrateBalanceKeys` example is pinned to `v0.54.0` because `x/bank/migrations/` no longer exists upstream. Pinned the same QA v0.38 page's end-to-end framework link to `v0.38.0-alpha.2` rather than letting it float on `main`. +- Open: the upgrade guide's `Migrator` example is built on `x/bank/migrations/v2`, `v3`, and `v4`, none of which exist at `release/v0.55.x`; `x/staking` is the remaining module with a live `Migrator` and `migrations/v6`. Left for a maintainer decision. The two `changelog/release-notes` pages point at `blob/main/CHANGELOG.md`, which is generated by `manage-changelogs.js` and correct for "full release history". + +## 2026-07-30 + +- Repointed every GitHub reference on the six Cosmos Enterprise pages (`sdk/latest/enterprise/overview`, `enterprise/group/overview`, `enterprise/poa/{overview,architecture,governance,distribution}`) from `main` and a pinned commit SHA to `release/v0.55.x`, and re-verified each line anchor against the code at that ref. 22 of the 23 anchored links had drifted, several landing in license headers or import blocks. Synced to `sdk/next/`. +- Adjudicated the prose behind those links against `release/v0.55.x`. Confirmed-false claims reported for maintainer review rather than applied: PoA validator creation is admin-only and carries an initial power (the pages describe it as permissionless and power-zero); the validators collection is an indexed map keyed by consensus address with operator and power secondary indexes (the pages describe a `(power, consensus_address)` composite primary key); the PoA module account is mandatory, since `EndBlocker` panics at block 1 when the ante handler fee recipient is not the PoA module (the pages describe it as recommended with a `fee_collector` fallback); and the tally constructor is `NewPOACalculateVoteResultsAndVotingPowerFn`, not `NewPoA...`. +- Left the vote-extension warning on `enterprise/poa/architecture` unverified. The claim is about a CometBFT interaction that the module source neither states nor contradicts. + + +## 2026-07-30 (ref sweep: outcome and what is deliberately not on the shipping ref) + +Every GitHub link in `sdk/latest`, `sdk/next`, `cometbft/latest` and `cometbft/next` was tested. 160 links were corrected across six page clusters, each new target confirmed to return 200 with its line range re-derived from the file at the shipping ref, and every touched page synced to `next/`. + +Final distribution in `latest/`: SDK 250 of 313 on `release/v0.55.x`, CometBFT 105 of 109 on `v0.40.x`. The remainder is deliberate: + +- 60 in ADR and RFC pages. Per maintainer instruction these are liveness-checked only, not made current: an architecture decision record legitimately cites code as it stood when the decision was taken. All 60 return 200. Two that did not were repointed to the newest ref where the path still exists, `x/group` to `release/v0.53.x` and a building-modules doc to `release/v0.46.x`, the latter with its `#queries` anchor confirmed present. +- Two `changelog/release-notes.mdx` pages cite `main/CHANGELOG.md`. The sentence promises the full release history, which only `main` carries, and the URL is emitted by `manage-changelogs.js` rather than written on the page. +- Three CometBFT QA citations stay pinned to the alpha tag each campaign actually tested. Bumping them would make the sentences false. One companion link on the same page was moved off floating `main` onto the matching tag for the same reason. +- `guides/upgrades/upgrade.mdx` keeps `v0.54.0` for `migrateBalanceKeys`. `x/bank/migrations/` does not exist at v0.55.x, and the prose is already past tense about a historical migration. +- `modules/circuit/README.mdx` keeps a `main` URL that sits inside a verbatim copy of `baseapp/msg_service_router.go`. Rewriting it would falsify the code excerpt. + +### What the sweep found beyond stale refs + +Line drift was the smaller problem. Anchors that had been pointing at the wrong thing for years were the larger one: + +- On the Enterprise PoA pages, 22 of 23 anchored links had the wrong line, several landing inside a licence header or an import block, even though `main` and `release/v0.55.x` are byte-identical for those files. +- Four ranges on the module pages were malformed at the ref they already carried: the distribution `FeePool` range truncated its closing brace, and all three `circuit` ranges straddled message boundaries, one running past end of file. +- `spec/abci/Client-and-server.mdx` pointed at `master`, a branch that no longer exists on `cometbft/cometbft`. It was a broken link, not a stale one. +- By contrast the large `v0.47.0-rc1` cluster on staking and gov was almost entirely well-formed, 34 of 35 ranges bounding their construct correctly. That cluster was pure version drift, which contradicted the expectation set by the earlier `v0.50.x` findings. + +### Upstream sync exposure + +The module reference pages under `sdk/latest/modules/` are conversions of upstream READMEs, and those READMEs still carry the original stale URLs byte-for-byte at `release/v0.55.x`. Corrections to staking, gov, authz, feegrant, group and the ten smaller module pages will be reverted by the next content sync unless the same fix lands upstream first. + +Related: `sync-latest-to-next.js` copies these links verbatim, so `next/` now cites `release/v0.55.x` while documenting the unreleased version. Harmless while the two are close, and the next freeze re-bumps them, but it means `next/` does not point at the branch it describes. + +### Prose defects found, reported and not applied + +21 items. Most consequential are on `enterprise/poa/`, where the pages describe the module incorrectly rather than merely linking to the wrong line: validator creation is admin-only rather than permissionless and the admin sets initial power; the validators collection is an indexed map keyed by consensus address, not a `(power, consensus_address)` composite requiring re-keying; and the PoA module account is mandatory, because `EndBlocker` panics at block 1 when the ante handler's fee recipient is not the PoA module, where the page calls it recommended with a `fee_collector` fallback. + +Also confirmed false against code: `x/gov` has no `GlobalParams` store, `x/crisis` and `x/mint` params are protobuf-encoded under prefix `0x01` rather than Amino under a `mint/params` key, and `x/feegrant` has three allowance types where the page says two. + +Open editorial questions recorded separately: the upgrade guide's `Migrator` walkthrough is built on `x/bank/migrations/v2` through `v4`, none of which exist at v0.55.x, with `x/staking`'s `migrations/v6` the natural replacement; and `modules/group/README.mdx` duplicates `sdk/latest/enterprise/group/` at 2168 lines against 918. + +## 2026-07-30 + +- Applied the approved PoA prose corrections from the ref sweep to `sdk/latest/enterprise/poa/` and synced to `next/`. Validator creation is now described as admin-only with the admin setting initial power; the validators collection is described as an indexed map keyed by consensus address with operator-address and power indexes, and the Collections Schema table rows for prefixes 1 to 3 were corrected to match `keys.go`; validator updates now say they take effect in the next block, which also removes a self-contradiction with the Gaining Consensus Power section. +- `distribution.mdx`: the PoA module account is documented as required rather than recommended, because `EndBlocker` panics at block 1 when the ante handler's fee recipient is not the PoA module. Removed the false `WithFeeRecipientModule` backwards-compatibility sentence and the `fee_collector` fallback claim. +- `governance.mdx`: corrected the tally function name to `NewPOACalculateVoteResultsAndVotingPowerFn`, the keeper method to `GetValidatorByOperatorAddress`, the error string casing to "active POA validator", and the `AfterProposalSubmission` hook parameter to `proposerAddr`. +- The Collections Schema table was deliberately left at prefixes 0 to 5; prefixes 6 to 8 (`queuedUpdates`, `validatorAllocatedFees`, `lastCommittedPower`) exist in `keys.go` but adding them was out of scope. +- Applied the approved SDK module prose corrections from the ref sweep and synced all nine files to `next/`. `gov/README.mdx`: removed the false `GlobalParams` KVStore sentence, collapsed the three deprecated `DepositParams`/`VotingParams`/`TallyParams` reference blocks into one `Params` block matching the single param set the module actually reads, retargeted the threshold prose from `TallyParams` to `Params`, dropped the Contents entry for a `Software Upgrade` heading that does not exist, and added a missing Contents entry for `Constitution`. +- `crisis/README.mdx` and `mint/README.mdx`: params are documented as protobuf-encoded under prefix `0x01` rather than Amino under a `mint/params` key. The mint page previously contradicted itself, since adjacent prose already gave the correct prefix. +- `feegrant/README.mdx`: three allowance types rather than two, and the `Grant` description no longer names only two of them. +- `staking/README.mdx`: added the missing `MsgRotateConsPubKey` Contents entry, which other pages already link to. `authz/README.mdx`: added the fourth `StakeAuthorization` type, cancelling an unbonding delegation. `distribution/README.mdx`: removed two "Response:" labels sitting above request messages. `slashing/README.mdx`: repointed a `/sdk/v0.47/build/` fee-distribution link at `latest/`. +- `node/run-node.mdx`: two reproduced `app.toml` comments realigned with `server/config/toml.go`, the `minimum-gas-prices` denom separator and the `max_txs` no-op mempool qualifier. +- Not applied: the proposed bump of the remaining non-ADR `main`/`master` links to `release/v0.55.x`. All five are `cometbft/cometbft` URLs, not Cosmos SDK ones, and CometBFT has no `release/v0.55.x` branch, so the bump would have replaced four working links with 404s. Left at `main`/`master`, which resolve. +- Left in place: the `load(GlobalParams, 'TallyingParam')` mention in the gov page's tally pseudocode. It is the last trace of the removed store and was outside the approved scope. diff --git a/work-log/security-release.md b/work-log/security-release.md new file mode 100644 index 000000000..7643af6dd --- /dev/null +++ b/work-log/security-release.md @@ -0,0 +1,569 @@ +# security-release + +## 2026-07-20 + +- Addressing Eric's PR #329 review comments one by one. +- configure-backend: resolved the algorithm-default TODO(ERIC) per Eric's "no defaults" comment; kept the prescriptive "required for all backends" line. Code note for the record: on kms origin/main the file backend does require algorithm (empty → `file: unknown key type`), but pkcs11 and awskms still default an empty value to ed25519 (signing/pkcs11/pkcs11.go, signing/awskms/awskms.go) — page states "required" as guidance per Evan's call. +- configure-backend: resolved the AWS KMS section's ML-DSA TODO(ERIC) at line 20; confirmed against signing/awskms/algo.go (registers ed25519, secp256k1, secp256k1eth, mldsa65 via AWS KeySpecMlDsa65). Added a sentence that the backend signs all four key types incl. mldsa65 (key spec ML_DSA_65). Kept the not-executed-against-real-AWS and hardware-matrix TODOs (mldsa65 has only a unit test, no LocalStack/real-AWS run). Per Evan, switched the worked AWS example from ed25519 to ML-DSA-65 (`--key-spec ML_DSA_65`, `algorithm: mldsa65`); key-spec/algo strings verified against signing/awskms/algo.go and config.go. +- configuration-reference: resolved the gRPC TLS TODO(ERIC) at line 91 per Eric ("mandatory TLS removed, optional now"); verified against validate.go:232-238 (both TLS fields empty → plaintext; one without the other is the error). Table already documented tls_cert/tls_key as Required: no, so deleted the TODO only, no table change. +- configuration-reference: resolved the algorithm-default TODO(ERIC) at line 44 per Eric ("defaults removed, configure explicitly") and Evan's comment-1 call; changed both `algorithm` Required cells (keys block and grpc.keys) from "yes in practice" to "yes". Confirmed the four algorithm strings against config.go:55-58 (ed25519, secp256k1, secp256k1eth, mldsa65) — reference cell correct as written. +- remote-signing: reframed the TMKMS line per Eric (drop "deprecated"/"successor") and Evan's steer — now recommends Cosmos-KMS over previous remote-signing solutions and points TMKMS users to the migrate guide; fixed the missing blank line before the How it works heading. "Deprecated" language on migrate-from-tmkms and run-production still to reconcile when those pages come up. +- remote-signing: resolved the key-types TODO(ERIC) at line 34. Listed privval as ed25519/secp256k1eth/mldsa65 and the gRPC signer service as ed25519/secp256k1eth, per Eric. Code discrepancy on record (Evan's call to match Eric): the signerservice.proto SignatureScheme enum and grpcAlgorithms also include secp256k1 for the signer service; page under-reports it deliberately. +- rotate-key-remote-signer: resolved the mldsa65 matrix TODO(ERIC) at line 44 per Eric ("mldsa supported for file, awskms, pkcs11 for privval"); code-confirmed (supportedPKCS11Algorithms, awskms algos map, file run-verified). Deleted the TODO; hardware-test caveat stays owned by configure-backend's matrix TODO. Also clarified the step-1 PQ Note so it states PKCS#11 and AWS KMS also sign mldsa65 (was vaguely "more info on backends"), per Evan. Fixed the step-2 PQ Note grammar ("a mldsa65 keys" → "a mldsa65 key"). +- create-ml-dsa-account: resolved the account opt-in TODO(ERIC) at line 42 per Eric — no account-side switch; upgrading the whole chain to SDK 0.55 is sufficient, and every node must run it since older binaries can't process ML-DSA signatures. Updated the prerequisite ("on every node") and added a one-line note. +- enable-ml-dsa-keys: deleted the line-71 TODO(MATT) on the --consensus-key-algo Warning (confirmed by Eric ":71 This is true" and by our keygen test's replace-not-append finding). Rewrote the Remove-a-key-type Danger to state the failure halts the chain and that routine delegations trigger it — code-confirmed: validateValidatorUpdates error → ApplyBlock error → consensus/state.go:1820 panic, deterministic across nodes. Resolves the outline page-2 FLAG(Eric) on block-rejection-vs-halt. +- Seed-reuse security callout (Eric migrate-validator-ml-dsa:73), placed on both pages per Evan. Code-confirmed in cosmos-sdk crypto/hd/mldsa65.go: Derive() reuses secp256k1 BIP32 derivation and the 32-byte result is used directly as the ML-DSA keygen seed, so at the same path the secp private key and the ML-DSA seed are the same secret. Added a full Warning to create-ml-dsa-account (step 1) and a short cross-reference Note to migrate-validator-ml-dsa (clarifying consensus keys have no mnemonic). Scoped the Warning to secp256k1 only (dropped eth_secp256k1): the collision is same-path, and secp256k1 and ml_dsa_65 both default to coin type 118 while eth_secp256k1 defaults to coin 60; ed25519 is not an HD account algo (keyring SupportedAlgos = secp256k1 + ml_dsa_65). +- best-practices: resolved the placement-criteria TODO(ERIC) at line 19 per Eric — added a three-way decision list (separate host + firewall default; sentry-fronted → local OK; cost → same-host acceptable tradeoff). Rewrote the one-signer Danger to drop Horcrux (per Evan's remove-entirely call); Evan finalized the Danger wording himself. Restructured the placement section per Evan (Option B): kept the decision bullets, removed the now-redundant "alternative" paragraph, and stated the exposed-machine caveat once after the list. +- Removed the other Horcrux mention from run-production.mdx (the key-sharding HA sentence), completing the remove-Horcrux-entirely directive across sdk/next. +- migrate-from-tmkms: resolved the node-side/addr TODO(ERIC) at line 11 per Eric — tcp://host:port (no node ID) uses CometBFT SecretConnection matching TMKMS so the node listener is unchanged; documented noise://@host:port as the libp2p alternative in the addr table row. Still open: the page has no end-to-end tmkms→kms test run yet (the TODO's "verify before release" concern), tracked in FOU-753. +- migrate-from-tmkms: resolved the backend-mapping TODO(ERIC) at line 14 per Eric — YubiHSM (and other PKCS#11 HSMs like Fortanix) map to the pkcs11 backend keeping the key; Ledger signer is not implemented in Cosmos-KMS and is out of scope, with a feature-request link. Updated the yubihsm/ledgertm table rows; Fortanix folded into the yubihsm note (no tmkms fortanix provider block exists). Prose in the HSM section still to update (next comments :50/:52). +- migrate-from-tmkms: filled the "From YubiHSM or another HSM" prose per Eric :50/:52 — PKCS#11 HSMs (YubiHSM, Fortanix, etc.) use the pkcs11 backend with the same key, no move (non-exportable is the normal case); rotation is the path when changing custodian; Ledger explicitly out of scope with a feature-request link. Trimmed the table notes to pointers to avoid table/prose duplication. Deleted the :52 TODO. +- migrate-from-tmkms: filled the double-sign state section (:58) with Eric's jq translation one-liner (formats differ; stop tmkms, translate to /state/.json, start kms). Concise per Evan (dropped the fallback paragraph). Caveat: Eric-provided command, not yet run; FOU-753 e2e test still needs to confirm kms accepts a three-field state file. +- migrate-from-tmkms: filled the "What can go wrong" error strings (:74) per Eric — the three "if you try X you'll see Y" messages (softsign wrong-length key, node-id left in addr, tmkms state reused as-is) as error-first bullets with fixes and section links. Softsign error is code-verified; deleted the TODO. +- TMKMS "deprecated" framing reconciled across the set per comment 5: migrate-from-tmkms (description + intro) and run-production Note now say Cosmos-KMS is the recommended signer going forward, dropping "deprecated"/"successor". No "deprecated" TMKMS mentions remain in sdk/next. +- Dispatched a subagent to test the tmkms→cosmos-kms migration e2e on a localnet and derive/verify the softsign key-conversion command (findings → tests/tmkms-migration-findings.md). Softsign section (:46) held with its TODO until that returns. +- migrate-from-tmkms: subagent test PASSED (findings in tests/tmkms-migration-findings.md, 2026-07-20; tmkms 0.15.0, simd 46a17713, kms 0007b0d1). Filled the "From softsign" section (:46): softsign key is base64 of the 32-byte seed; documented the two key paths (original priv_validator_key.json direct, or the verified python3/cryptography seed→64-byte conversion, byte-identical to the original) with a shred warning. Added a scoped Manual verification comment (softsign/file-backend path only; HSM and AWS paths NOT run). Added a sentence explaining the state jq's step+1 remap (tmkms 0/1/2 → cometbft 1/2/3). Deleted the last page TODO. +- migrate-from-tmkms: three follow-the-doc clarifications per Evan — the softsign key file is the `path` in the operator's `[[providers.softsign]]` block (tmkms_softsign.key is a placeholder); the conversion command needs `pip install cryptography`; and a line that a standard cosmos-sdk node has no peer-id prefix so both signers use bare tcp://host:port (the "drop the prefix" row assumed one existed). +- Dispatched a second subagent to verify the PKCS#11 (SoftHSM2) and AWS KMS (LocalStack) signing paths, to prove the HSM no-move claim and clear configure-backend's open PKCS#11/AWS test TODOs (findings → tests/kms-backend-verification-findings.md). +- best-practices: deleted the generic line-9 TODO(ERIC) — Eric's review of the page is complete and his two comments were folded in. +- migrate-from-tmkms: restructured into an imperative numbered how-to per Evan (## 1 Translate the config … ## 6 Confirm signing resumed). Added a complete file-backend kms.yaml example under step 1 and clarified the state-file source under step 4 (the `state_file` in `[[chain]]`); moved the Danger to step 3 (stop TMKMS) and put the state translation after the stop, fixing the prior ordering. Updated in-page anchors (state section → #4-translate-the-double-sign-state); links verified consistent with headings. +- configure-backend PKCS#11 section: verified end-to-end via a subagent (SoftHSM2 2.7.0, kms 0007b0d, 2026-07-20 — findings in tests/kms-backend-verification-findings.md; a SoftHSM2-held ed25519 key signed real consensus, stop/restart proof passed). Resolved the PKCS#11 TODO(ERIC) with a Manual-verification comment and added a platform note that the `module` path differs (Linux /usr/lib/softhsm, macOS Homebrew /opt/homebrew/lib/softhsm). This also mechanically proves the migrate-from-tmkms HSM no-move claim. Left the "required for all backends" algorithm wording as-is per Evan (test confirmed pkcs11/awskms default to ed25519, but the prescriptive wording stands). Replaced the mldsa65 per-backend matrix TODO with a status note: matrix is source/owner-confirmed (file/pkcs11/awskms), file run-verified, but pkcs11/awskms mldsa65 not run-verifiable locally (SoftHSM2 has no ML-DSA mechanism; awskms needs real AWS) — the ed25519 pkcs11 run does not cover mldsa65. +- Verbatim follow-through tests (two fresh subagents, docs-as-only-source). TMKMS guide verdict: works as written except one blocker — Step 4's jq redirect fails because `kms init` doesn't create `/state/`; fixed by prepending `mkdir -p /state`. Other TMKMS findings held (signer-first note, node-id error paraphrase, step+1 prose) pending the tutorial re-verify / as cosmetic. KMS tutorial verdict: steps 1-7,9 verbatim-accurate, but flagged Step 8 signer-first "smooth first run" failing with a `can't get pubkey` backoff race and the Step 8 Note misdiagnosing it. Both timing findings were on a contended machine (the two agents collided on ports) and conflict with the clean 2026-07-14 verification, so launching a single clean re-verify of tutorial Step 8 before changing that prose. +- Clean re-verify of tutorial Step 8 (single localnet, 2026-07-20; findings in tests/kms-tutorial-step8-reverify-findings.md) confirmed the failure is REAL, not contention: node within ~5s of the signer passes 5/5, longer gaps hit dead zones and time out with `can't get pubkey`, a rerun recovers. Source-grounded root cause: kms dial backoff (200ms→10s cap) vs the node's single 3s pubkey fetch (cometbft node/setup.go:735). Fixed tutorial Step 8 — added "start the node promptly, within a few seconds" to the body and reworded the Note (dropped the misdiagnosis, kept the rerun recovery); updated the hidden manual-verification comment. Commands unchanged, so the prior verification still stands. migrate-from-tmkms left as-is (node stays running there, no cold-start race). Version-string finding (simd/kms version don't map to "0.55") noted, left for now. +- migrate-from-tmkms cosmetic fixes: corrected the node-id DNS error string to `dial tcp: lookup node-id@host: no such host` (the `:port` never appears — the whole node-id@host is the lookup token; per tmkms-migration and follow-through findings), and expanded the Step 4 jq prose to explain both conversions (round string→number for the int32 error, and step+1 for the 0/1/2→1/2/3 remap). Skipped the version-string note (pre-tag artifact, self-resolves when 0.55 is tagged). + +## 2026-07-20 (Alex review + PoA merge) + +- Alex (aljo242) did a full PR pass. Versions: pages already say CometBFT 0.40 (correct); the post-quantum-keys 0.55-unverifiable TODO is already gone; the stale item is work-log line 92 which records the CometBFT prereq being "corrected to 0.39" — that was later reverted to 0.40 for the release (the 0.39.3 go.mod pin was the pre-bump v0.54 checkout). Confirming the cometbft go.mod bump to 0.40 actually lands is an eng/release item (local checkout still pins v0.39.3). +- PoA rotation PR cosmos/cosmos-sdk#26590 merged (commit c52e67a39d). Verified the merged proto/CLI match the docs exactly (MsgRotateConsPubKey sender/validator_address/new_pub_key; `rotate-cons-pub-key --operator-address --from`, operator-or-admin). Re-verified poa/api.mdx RotateConsPubKey section against merged code and removed its provisional TODO(MATT); rotate-validator-key-poa carries no provisional marker so no change. Updated the migrate PoA-path TODO to point at merged main. +- Dispatched a subagent to localnet-test PoA rotation against merged enterprise/poa (operator self-rotation, admin override, and the ML-DSA PoA path) — the outline's post-merge checklist and the last real PoA test gap. +- migrate-from-tmkms: added Alex's "what you gain" framing to the intro (Cosmos-KMS adds AWS KMS + PKCS#11 backends and post-quantum ML-DSA signing, none in TMKMS) so it reads as an upgrade, not a lateral switch. +- AWS KMS backend verified against REAL AWS (Sandbox account 326804803147, us-east-1; findings in tests/kms-aws-verification-findings.md): both ed25519 and mldsa65 keys signed consensus end to end, stop/restart proofs passed, CloudTrail confirmed GetPublicKey+Sign. Cleanup done (keys scheduled for deletion, aliases removed). Updated configure-backend AWS section: resolved the AWS TODO(ERIC) with a manual-verification comment; added the minimal IAM policy (kms:GetPublicKey + kms:Sign only, no kms:DescribeKey); added a key-import caveat (Ed25519 importable via --origin EXTERNAL, ML_DSA_65 rejected, import forfeits never-leaves-KMS); added the missing-permission error to What can go wrong. Updated the mldsa65 per-backend status note (awskms now real-AWS-verified; only pkcs11-mldsa65 remains, needs an ML-DSA HSM). Added a fresh-key note to the AWS and PKCS#11 sections: a newly created KMS/HSM key is a new consensus key, so an existing validator must rotate to it (or a new validator is registered with its pubkey) — the "only the keys block changes" framing holds only for the file backend. +- configure-backend PKCS#11: added the env-var step the SoftHSM2 test exposed — `export KMS_PIN=` (matching pin_env) and, for the SoftHSM2 rig, `export SOFTHSM2_CONF=` so the module finds the token. Without these the signer can't unlock the token; the page previously showed `pin_env: KMS_PIN` without telling the reader to set it. +- configure-backend full-page verbatim follow-through (findings in tests/configure-backend-followthrough-findings.md): file backend passes as written; PKCS#11 and AWS configure fine but a reader following "only the keys block changes" HALTS the chain (`invalid proposal signature`) because a fresh backend key is a new consensus key. Fixed four gaps: (1) reworded the Prerequisites overclaim (file reuses the existing key; AWS/PKCS#11 need adoption); (2) added an "Adopt the key on a validator" section (existing validator → rotate via rotate-key-remote-signer; new validator → gentx/create-validator --pubkey with the pubkey from /status); (3) corrected the "error names the offending field" claim — file's missing-algorithm error is the cryptic `file: unknown key type`; (4) fixed the AWS error string to `AccessDeniedException`. AWS keys from the run were cleaned up (scheduled for deletion). +- migrate-from-tmkms polish: added the missing step 5 (`kms start`) and step 6 (`/status` height + state-file) commands; reworded the step 3 "safe" line to explain missed blocks are recoverable but a double sign is not (keep the gap short). +- configure-backend polish: removed the redundant "backend problems surface at startup" paragraph from Verify-any-backend (duplicated What-can-go-wrong); linked the tutorial in the File section; tightened the AWS lead-in (create+alias, or reuse via key_id) and the profile/endpoint note (flagged as optional, not shown). +- Reconciled the rotate-validator-key-poa manual-verification comment with the migrate page: was stale ("ML-DSA NOT achievable / fix in flight / Note blocked"), now states ed25519 verified on merged #26590 and ML-DSA verified on the #26614 PR branch (needs --gas auto), re-confirm at #26614 merge. +- Linked the node/simd guide ([Run a node](/sdk/next/node/run-node)) from every keys/kms guide that uses simd in reader-facing commands and lacked it: rotate-validator-key, migrate-validator-ml-dsa, rotate-validator-key-poa, rotate-key-remote-signer, configure-backend, tutorial-file-backend (added to prerequisites). create-ml-dsa-account and enable-ml-dsa-keys already had it. migrate-from-tmkms excluded — it only mentions simd in a hidden verification comment, no reader-facing simd commands. +- PoA rotation test result (findings in tests/poa-rotation-test-findings.md): operator self-rotation and admin override (ed25519) PASS against merged enterprise/poa (origin/main 3d3b901ce5); rejections, power-0, cutover, and fee migration all verified. Added a Manual-verification comment to rotate-validator-key-poa. ML-DSA PoA rotation is BLOCKED by a poa CLI gap (pubkeyFactory has no ml_dsa_65 entry → `unknown pubkey type: ml_dsa_65`; keeper accepts the key) — flagged to eng via Slack (Eric/Matt to take the ~10-line fix). Kept the ml_dsa_65 spots' TODO(MATT) open (migrate PoA path + rotate-poa Note) pending that fix; ed25519 PoA flow left as-is (verified). +- Eric opened the fix: cosmos/cosmos-sdk#26614 (OPEN) adds the mldsa pubkey factory to the poa module + simapp and raises the pubkey-length bound in types/poa.go (ML-DSA pubkeys ~1952B). Updated the migrate PoA-path TODO(MATT) and the rotate-poa manual-verification comment to reference #26614 as the in-flight fix (was "flagged to eng"). Dispatched a subagent to verify #26614 fixes ML-DSA PoA rotation against its PR branch (findings → tests/poa-mldsa-rotation-findings.md) so we can de-provisionalize the moment it merges. +- #26614 verified on its PR branch (head 1ff66f4c5b, 2026-07-20): ML-DSA operator + admin PoA rotation execute, set switches to cometbft/PubKeyMlDsa65, ed25519 regression clean; the fix is WithMlDsa65Support() + simapp wiring + MaxPubKeyLength 128→2048. New verified doc finding folded in now (independent of merge): the ML-DSA PoA rotation runs out of gas at the default 200000 (large pubkey needs ~316000), so added `--gas auto --gas-adjustment 1.5` to the migrate PoA-path command and the rotate-poa PQ Note (same large-pubkey gas issue already noted on create-ml-dsa-account; staking path already uses --gas auto). Updated the TODO(MATT) with the PR URL and the branch-verification note; remove it when #26614 merges. +- Fixed an MDX parse error in sdk/next/modules/staking/README.mdx:1382 (from the 2026-07-14 rotation additions): the KeyRotationFee example value `{"denom":"stake","amount":"1000000"}` was a bare `{...}` that MDX parsed as a JSX expression and failed on. Wrapped it in backticks. `npx mint broken-links` now runs clean site-wide ("no broken links found"), confirming the tmkms restructure's numbered-step anchors and all "see the section" links resolve. + +## 2026-07-14 + +- Added consensus key rotation to the x/staking module reference (sdk/next/modules/staking/README): MsgRotateConsPubKey message, KeyRotationFee parameter row, rotation state stores (0x91 to 0x94), and End-Block lifecycle; links the keys/ guides for procedure +- Added RotateConsPubKey to the PoA API reference with the proto message extracted from the cosmos/cosmos-sdk#26590 diff (sender, validator_address, new_pub_key), the admin rotation capability to poa/governance, and a guide link to poa/overview; provisional until the PR merges +- Added module-reference back-links to the Next steps of key-rotation, rotate-validator-key, and rotate-validator-key-poa (spec lives in module pages, procedure in guides, one link each way) + +- Added sdk/next/kms/configure-backend (per-backend keys blocks for awskms, pkcs11, file, incl. the mldsa65 file-backend variant); fields verified against config.go on kms main; explicit algorithm in every example per the tutorial test finding; AWS and PKCS#11 sections carry test TODOs (not yet executed), plus the mldsa65-vs-ml_dsa_65 naming note +- Added sdk/next/kms/migrate-from-tmkms (deprecation framing, tmkms.toml-to-kms.yaml mapping table grounded in tmkms.toml.example from the tmkms repo, rotate-instead-of-move guidance for non-exportable HSM keys, cutover steps with the one-signer Danger); the softsign key import, state migration, and YubiHSM sections are skeletons with TODO(Eric) comments describing exactly what he should supply +- Added sdk/next/kms/best-practices (signer placement and trust domains, Noise-over-SecretConnection recommendation with the pinning asymmetry explained, key protection levels, the one-signer rule as a Danger, gRPC exposure, custody-vs-availability boundary); one FLAG TODO on placement decision criteria pending Eric +- Added sdk/next/kms/configuration-reference (full kms.yaml field tables transcribed from config.go on main: chains, validators, keys with per-backend sub-tables, grpc block incl. the no-caller-auth warning); two FLAG TODOs carried in-page (algorithm default vs tested behavior; README-vs-config.go on gRPC TLS); resolved the reference TODO links on the tutorial and backend pages + +- Added sdk/next/keys/create-ml-dsa-account (limits-first user account how-to: keyring creation, fund transfer, signing verification); registered in docs.json; chain opt-in mechanism is an MDX TODO pending Eric +- Added sdk/next/kms/tutorial-file-backend (tutorial: local chain signing through cosmos-kms, file backend, with a stop-the-signer proof step); CLI and config template verified against kms source on main, not the README +- Added a Remote Signing group (sdk/next/kms/) to the SDK next nav with its first page, remote-signing (privval explanation, cosmos-kms scope and boundaries, double-sign protection); restructured after review into what-is / how-it-works (five bullets mirroring the upstream README); key types and transport verified against kms source on main (ml_dsa_65 file backend supported), Horcrux and TMKMS linked to their repos; gRPC section cut +- Readability pass on sdk/next/keys/create-ml-dsa-account (split overlong sentences in intro, mnemonic note, and verify step; separate lead-ins for the two verify commands); no technical or structural changes +- Readability pass on sdk/next/kms/tutorial-file-backend (active voice in intro, split chained sentences in steps 7 through 9 and the wrap-up, present tense in wrap-up); commands, snippets, and teaching moments untouched; no technical or structural changes + +- Added sdk/next/keys/rotate-validator-key-poa (PoA rotation: keygen, submit, cutover timing, admin path), grounded in PR cosmos/cosmos-sdk#26590 (branch poa-rotation, enterprise/poa/docs/key-rotation.md); provisional until that PR merges +- Updated key-rotation's Staking and PoA chains section with the PoA differences (admin path; no fee, rate limit, rotation history, or delay) and linked the new page; registered the page in docs.json between rotate-validator-key and migrate-validator-ml-dsa +- Readability pass on sdk/next/keys/migrate-validator-ml-dsa (split overlong sentences, imperative verify step); no technical or structural changes +- Readability pass on sdk/next/keys/rotate-validator-key-poa (split overlong sentences in intro, step 3, and callouts; tightened wording); no technical or structural changes +- Resolved migrate-validator-ml-dsa's PoA link TODO now that rotate-validator-key-poa exists +- Briefly rewrote rotate-validator-key-poa's cutover to the staking-style second-node flow, then reverted to the module runbook's watch-then-swap after weighing the tradeoffs (second-node infra cost, enterprise change control, missed blocks near-free on PoA); open question with Matt whether to add the shadow flow as an alternative and whether PoA tracks liveness +- Readability pass on sdk/next/kms/configure-backend (split semicolon and colon chains across all three backend sections, converted the PKCS#11 field rules to a list, fixed a sentence starting with inline code, made the file-backend lead a full sentence, cut "today" for timelessness); frontmatter, commands, yaml, Note callout, TODOs, and links untouched; no technical or structural changes +- Manually verified sdk/next/kms/tutorial-file-backend end-to-end on a local simapp chain (simd main @ b9a11304cf, kms @ 7932ceb, macOS); all three checkpoints pass; found and fixed two blockers the page as written missed: the file backend needs `algorithm: ed25519` in kms.yaml (otherwise `kms start` fails with "unknown key type"), and the node exits on cold start unless the signer is already dialing (it does not idle-wait); corrected the body (added the algorithm line, reordered steps 7-8 to signer-first with a note that the node cannot start without its signer, bumped the Go prereq to 1.26 since kms requires it), re-verified the signer-first flow, and updated the hidden manual-verification MDX comment to match +- Readability pass on sdk/next/kms/best-practices (split the chained transport and peer-ID sentences, active voice for transport selection and validator maintenance, clarified "the block" to "the service" in the gRPC section, punctuation fixes in the Danger callout and Horcrux closer); frontmatter, commands, callout type, links, and TODO comment untouched; no technical or structural changes + +## 2026-07-10 + +- Added a new Key Management group to the SDK next nav in docs.json, after Run a Node, with four new pages +- Added sdk/next/keys/: post-quantum-keys (key roles and algorithms survey, ML-DSA explanation), enable-ml-dsa-keys (genesis and governance paths; governance flow test-verified on a local simapp chain), key-rotation (concept page for consensus key rotation, ADR-016), rotate-validator-key (zero-downtime rotation procedure; test-verified on a local simapp chain) +- Pages for the rest of the ledger security release (PoA rotation, user accounts, remote signing) are outlined but pending code that lands with the release; links to them are omitted until the pages exist +- Added sdk/next/keys/migrate-validator-ml-dsa (thin how-to composing the enable and rotation pages); step 1 keygen command is an open TODO pending release tooling (comet gen-validator and init are ed25519-only in current checkouts), marked as an MDX comment in the page + +## 2026-07-17 + +- ML-DSA keygen documented from cosmos/cosmos-sdk#26604 (open, approved; all mentions carry provisional re-verify markers): migrate-validator-ml-dsa step 1 is now the --consensus-key-algo ml_dsa_65 flag and both labeled paths add it to their guide's simd init instead of swapping key files; enable-ml-dsa-keys genesis section gained the init/testnet sentence; rotate-key-remote-signer's PQ Note gained the file-backend keygen line; configure-backend's mldsa65 TODO updated from blocked to runnable-at-merge; run-testnet documents the new --consensus-key-algo flag with a PQ localnet example +- Test run of the keygen flows (simd from pr-26604 @ 46a177139a, kms @ 0007b0d, findings in tests/mldsa-keygen-test-findings.md): staking migration path verified end to end (verification comment added to migrate-validator-ml-dsa; PoA path re-flagged to Matt), testnet flag verified (comment on run-testnet), mldsa65 file backend verified (comment on configure-backend) +- Findings folded in: the file-backend-only claim was stale (kms 0007b0d ships mldsa65 for awskms and pkcs11) — corrected on rotate-key-remote-signer, configure-backend, and the remote-signing key-types bullet, marked unverified-on-hardware with the matrix TODO(ERIC) retained; the third ML-DSA spelling (/cosmos.crypto.mldsa65.PubKey proto type) documented in configure-backend's naming Note and as the jq adjustment on rotate-key-remote-signer step 4; the init flag's replace-not-append behavior on genesis pub_key_types documented as a Warning on enable-ml-dsa-keys + +## 2026-07-15 + +- Prerequisites pass over all guide pages (keys/ and kms/): every prerequisites block now lists tools (jq, curl, Go, make, git, per-backend HSM/AWS tooling), running-chain/validator requirements, and funded accounts, each linked to its official source (go.dev, jqlang.org, curl.se, git-scm.com, gnu.org, opendnssec.org, OpenSC, aws.amazon.com) +- Added Prerequisites sections to configure-backend (tutorial link plus per-backend tooling) and migrate-from-tmkms (TMKMS access plus kms installation) +- create-ml-dsa-account: added running-chain and funded-account prerequisites with a Run a node link +- Renamed cosmos-kms to Cosmos-KMS in all prose across kms/ pages (titles, descriptions, headings, body, link text); kept lowercase in the backticked repo reference and all commands +- Replaced bare "kms" prose nouns with "the signer" across kms/ pages (tutorial, best-practices, remote-signing, migrate-from-tmkms); used Cosmos-KMS in the TMKMS mapping table where the product contrast matters; `kms` commands and log output untouched +- post-quantum-keys: reworked the cost example per Eric's PR review (r3600244249) onto a consistent whole-block basis, then moved the figures into a table (ed25519 vs ml_dsa_65: pubkey, signature, per-block signature data 6 KB vs 331 KB, total block with ~4 KB overhead 10 KB vs 335 KB, yearly ~56 GB vs ~1.8 TB); dropped the IBC multiplier figure in favor of "larger" since the 30x-vs-52x number was unresolved +- post-quantum-keys: linked EIP-7702/8051/8141 to eips.ethereum.org and corrected the EVM paragraph (EIP-8141 is Frame Transaction for signature agility; EIP-8051 is the ML-DSA verification precompile; the page had attributed native PQ signatures to 8141) +- Added sdk/next/kms/rotate-key-remote-signer (rotate a consensus key held in Cosmos-KMS; second signer process, shadow node, pubkey from /status RPC); procedure test-verified e2e on a localnet (kms @ 538e5c5, sdk @ b9a11304cf, findings in tests/kms-rotation-findings.md); registered in docs.json Guides subgroup; Danger for the show-validator stray-key trap (jails the validator); resolved the remote-signer TODO link in migrate-validator-ml-dsa; added next-step links from configure-backend and best-practices +- Rewrote migrate-validator-ml-dsa step 2 into three labeled paths (On a staking chain / On a PoA chain / On a remote signer): each names the exact change to the linked guide (key-file swap before node start; PoA also passes ml_dsa_65 instead of ed25519 in the submit command, shown); step 1 keygen TODO untouched (Eric fills it); added a verification TODO since neither path has run with a real ML-DSA key yet +- Added a state-sync pointer to both staking rotation pages (rotate-validator-key step 1, rotate-key-remote-signer step 3): a fresh shadow node takes days from genesis on a real chain; linked run-node#state-sync (which covers state sync, local snapshot restore, and the snapshots commands) +- Release-impact sweep of sdk/next/node/: run-production's remote signer section replaced (full TMKMS walkthrough removed; now points to the Cosmos-KMS overview, tutorial, and best practices, with a TMKMS deprecation Note linking migrate-from-tmkms; only remaining sdk/next TMKMS mentions are the two intentional kms/ pages); keyring's Additional key management corrected against keyring source (SupportedAlgos is secp256k1 + ml_dsa_65, keyring.go:213; flag is --key-type, add.go:94; the old ed25519/--algo claim removed) and linked to post-quantum-keys and create-ml-dsa-account; keyring's broken in-page anchor fixed (#reference:-keyring-backends → #reference-keyring-backends); run-node gained next-step links to key-rotation and remote-signing and its jq links moved from stedolan.github.io to jqlang.org +- Nav restructure: removed the top-level Key Management and Remote Signing groups from the next version and nested three groups under Run a Node instead: Key Rotation (key-rotation, rotate-validator-key, rotate-validator-key-poa), Post-quantum Keys (post-quantum-keys, enable-ml-dsa-keys, migrate-validator-ml-dsa, create-ml-dsa-account), and Remote Signing: Cosmos-KMS (remote-signing, tutorial-file-backend, a Guides subgroup with configure-backend and migrate-from-tmkms, configuration-reference, best-practices). File paths unchanged, no redirects needed (pages unpublished) +- Shortened five page titles: Rotate a consensus key, Staking; Rotate a consensus key, PoA; Enable ML-DSA keys; Create an ML-DSA account; Remote signing tutorial. Updated all internal link text to match +- Readability pass on sdk/next/kms/rotate-key-remote-signer (split semicolon-chained sentences throughout, active voice in the intro, split the three-differences list into sentences, removed the --gas-prices parenthetical, imperative kms.yaml edit); frontmatter, commands, yaml, callouts, hidden verification comment, What can go wrong entries, and links untouched; no technical or structural changes +- Line-by-line source verification sweep of all Ledger security-release pages (fresh reviewer per page per pass, 3-consecutive-clean bar) against cosmos-sdk a2ddd98f43, cometbft 6ac238b, kms 7932ceb, and PoA PR cosmos/cosmos-sdk#26590. post-quantum-keys, enable-ml-dsa-keys, rotate-validator-key, and rotate-key-remote-signer: corrected the CometBFT version prerequisite from "0.40 or later" to "0.39 or later" (cosmos-sdk go.mod pins cometbft v0.39.3; cometbft version.go reports 0.39.0) +- post-quantum-keys: added a TODO(ERIC) flagging the unverifiable "SDK 0.55" release label (no v0.55 tag exists; latest published tag is v0.54.3 and the linked release-notes page is titled v0.54); prose left unchanged pending the owner's call +- kms/remote-signing: corrected the file-backend post-quantum key-type string from `ml_dsa_65` to `mldsa65` (kms config.go:58 `AlgoMLDSA65 = "mldsa65"`; the underscore form `ml_dsa_65` is the chain-side CometBFT type, not the kms config value) +- kms/rotate-key-remote-signer: corrected the two-keys-one-chain startup error string from `app: multiple backends bound to chain` to `app: multiple signers bound to chain` in both occurrences (kms internal/app/build.go:57) +- kms/configuration-reference: corrected the gRPC keys table — `backend` Required no→yes and `algorithm` Required no→"yes in practice" (no default in validate.go; the signer build in internal/app/build.go rejects an unset value), gRPC file `algorithm` value secp256k1→secp256k1eth (build.go:255 requires `AlgoSecp256k1Eth`); completed the consensus per-backend algorithm sentence (added `secp256k1` on awskms and `secp256k1eth` on file, per validate.go:18 and signing/file/file.go); and clarified the file `key_file` raw-key encoding as algorithm-dependent (base64 for ed25519/mldsa65, hex for secp256k1eth, per signing/file/{ed25519,mldsa65,secp256k1}.go). Took 9 passes to converge (distinct latent errors surfaced at passes 1, 2, 5, and 6 — all fixed and source-grounded); passes 7, 8, and 9 were independent exhaustive re-reviews, all clean, giving the required 3 consecutive clean passes +- Two pass-1 reviewer findings triaged as non-defects and left unedited: migrate-validator-ml-dsa line 40 (`tx poa rotate-cons-pub-key ... ml_dsa_65`) is real per PR 26590 (the reviewer had only the checkout, which predates the merge), and configure-backend's "algorithm required for all backends" note is already owned by its adjacent TODO(ERIC). All other pages (13 keys/kms plus the staking-README rotation section, poa/api and poa/governance rotation additions, and the node/keyring, run-node, run-production release-impact edits) passed 3 consecutive clean reviews with no changes needed + +## 2026-07-21 (kms source-verification sweep) + +- Source-verified all three non-runnable kms pages against kms main @ efebc25 (fresh, code-as-truth). best-practices: fully accurate, no changes. remote-signing: no false claims; per Evan, added plain `secp256k1` (AWS KMS backend) to the privval key-type list to match the shipping code + configure-backend + Alex's flag, reversing the earlier match-Eric under-listing for the privval path. +- Deferred (parked pending the kms release-version pin, since main @ efebc25 may be ahead of 2026.1): the gRPC/signer-service details. configuration-reference gRPC section: (1) FIXED — the false "PKCS#11 is not supported over gRPC" claim is gone; the backend cell now reads `file`, `awskms`, or `pkcs11` (code-confirmed). Still parked: (2) under-lists gRPC per-backend algorithms (file also ed25519; awskms also secp256k1eth), (3) omits the gRPC PKCS#11 field set + key-id uniqueness. remote-signing's signer-service list still omits secp256k1. The gRPC SignerService is the deferred interop feature, so these ride until the version is pinned — but note M1 is an outright false statement, not just an omission. +- Repo note (not docs): kms config/default.yaml gRPC example uses backend:file + algorithm:secp256k1, which passes Validate but fails newGRPCSigner — upstream config-template bug. +- tutorial Step 8: the re-verify (kms-tutorial-reverify-findings.md, 2026-07-21) showed the "rerun simd start" recovery is unreliable (~30%; worsened on first boot by a one-time IAVL upgrade delaying the node's listener). Reworded the Step 8 Note to the reliable recovery — restart the signer (resets its backoff to fast dials), then start the node — matching rotate-key-remote-signer step 3, which already had it right. Updated the hidden verification comment. Root cause is a kms/cometbft timing race (kms dial backoff 200ms→10s cap vs the node's single ~3s pubkey fetch, cometbft node/setup.go:736); flagged to eng as a one-line kms fix (lower defaultBackoffMax to ~1s) — no doc wording fully fixes it. +- rotate-key-remote-signer re-verify (kms-rotate-reverify-findings.md): PASS verbatim — fresh reader completes the rotation; rotation code 0, set swapped, no double sign, old signer stopped, chain kept producing; both testable rejections reproduced with exact error text. Minor only: the step-3 Warning's "single ~3s attempt then exits" is the intermittent first-boot timing race (shadow node connected first-try this run); hedge keeps it correct, and the flagged kms backoff fix will moot it. No doc changes needed. +- migrate-from-tmkms re-verify (tmkms-guide-reverify2-findings.md): PASS verbatim — clean handoff (tmkms h118 → kms h119), softsign conversion byte-identical, all three error strings reproduce, step 5/6 commands and mkdir fix confirmed. Applied two gap fixes: (1) note that keys[].algorithm is required with no tmkms.toml equivalent (set ed25519 for softsign) — a reader translating strictly from the config table would miss it; (2) note that the original priv_validator_key.json lives on the node host, so copy it to the signer or use an absolute path (bare key_file resolves against the signer --home). Skipped the minor custom-state_file-naming note. +- kms guide re-test sweep complete: tutorial-file-backend, rotate-key-remote-signer, migrate-from-tmkms all re-verified verbatim PASS this pass; remote-signing/configuration-reference/best-practices source-checked against kms code; configure-backend was already fully tested today. Outstanding externals only: pkcs11-mldsa65 (needs ML-DSA HSM), ML-DSA-PoA rotation (kms #26614), and the kms cold-start backoff fix (flagged to eng). + +## 2026-07-22 (second-reviewer PR comment pass, PR #329) + +- rotate-validator-key-poa: added a Danger at step 2 warning that a validator holding more than 1/3 of voting power halts the chain during the cutover gap (set switch in step 3 until the node key swap in step 4), resuming once the swap completes (gjermundgaraba r3629694479). Not mirrored to the staking guide: that flow is zero-downtime (shadow node already signs the new key at the set switch), so no cutover gap exists. +- Missing `--home` on the outer `tx ... rotate-cons-pub-key` call, fixed in four places (gjermundgaraba r3629708041/r3629709981/r3629711065/r3629713061): rotate-validator-key step 3 (+`--home ~/.node`), rotate-validator-key-poa step 2 and Rotate-as-admin (+`--home ~/.node` on the tx and on the `keys show val -a` sub-call), migrate-validator-ml-dsa PoA submit. Confirmed empirically: without `--home`, the default home `~/.simapp` has no `val` key and the tx fails with `val is not a valid name or address: decoding bech32 failed: invalid bech32 string length 3` (simd @ go/bin, ~/.node keyring-test). +- rotate-validator-key: moved the "never copy the old priv_validator_key.json" Danger up to immediately after the step-1 `simd init` + ML-DSA Note, ahead of the genesis/peer/sync prose, so the destructive warning lands where the fresh key is generated (gjermundgaraba r3629934747). +- kms/remote-signing: extended the double-sign-protection bullet with the HA answer (dianab-cl r3630533701) — protection is per state file, so never run two signers for the same key; use one signer dialing multiple validator nodes for a backup (final wording Evan's). +- kms/tutorial-file-backend: added "This tutorial uses one node, one signer, and one key." to the intro to state the 1:1 topology (dianab-cl r3630574311). +- kms/best-practices: added a "Signer, chain, and key topology" section (dianab-cl r3630645347, r3630649561) — one signer : many chains, each chain : exactly one key (so one signer holds several keys), one signer : many nodes on a chain for redundancy, and one key : one live signer (leads into the existing one-signer section). +- kms/configure-backend: added a least-privilege IAM policy JSON to the AWS section (dianab-cl r3630793050) — `kms:GetPublicKey` + `kms:Sign` scoped to the key ARN (alias resolves server-side). Verified against the real-AWS run: exactly these two actions per source (`signing/awskms/*.go`, no DescribeKey) and CloudTrail; policy matches tests/kms-aws-verification-findings.md:249-258. (Test ran under AdministratorAccess, so the restricted policy is the verified-minimal set, not the literal role attached during the run.) +- dhfang r3627446955 (post-quantum-keys hashing relevance) and gjermundgaraba r3629942048 (PoA admin Warning placement): Evan edited directly. +- Skipped dianab-cl r3630514523 (explain why migrate from TMKMS on remote-signing) per Evan. Eric r3611263290 (whole chain must run 0.55) already covered verbatim at create-ml-dsa-account.mdx:39,42 — resolvable, no edit. +- IBC implications of ML-DSA (Dennis Fang thread, C0BAQAJAS4F): added an "IBC considerations" section to post-quantum-keys covering the two points the thread agreed to document — (1) counterparty chains verifying this chain over an `07-tendermint` light client must upgrade to CometBFT v0.40 before any validator rotates to ML-DSA, since an old client fails as soon as the first ML-DSA validator joins the set (no <1/3 grace period; Eric + Matt); (2) ML-DSA signatures enlarge headers, so IBC client updates grow and CometBFT v0.40 raises max block bytes (Eric). Dropped the relayer/single-tx-size scenario at Dennis's request (hypothetical, not a confirmed constraint). Replaced the old one-line "same growth reaches IBC" note in the cost section. Linked the section from a Warning atop migrate-validator-ml-dsa and a Note in enable-ml-dsa-keys. Final section wording is Evan's. + +## 2026-07-23 (Matt + dianab-cl PR comment pass, PR #329) + +- key-rotation "Security implications" section (Matt's four comments r3634310341/r3634339460/r3634345972/r3634357266, policy: treat referenced PRs as merged as-is): (1) corrected the slashing-history claim — tracking is not "for the length of the unbonding period" but an evidence window computed at rotation time from `MaxAgeNumBlocks` + `MaxAgeDuration` (cosmos-sdk#26616); added the frozen-snapshot risk (the window is fixed at rotation, so extending the evidence params by gov leaves in-flight rotations on the shorter window, opening a gap where a double sign under the old key cannot be slashed); (2) frequent rotations raise light-client update cost (less validator-set overlap → more expensive verification), part of why the once-per-unbonding-period limit exists; (3) rotation resets CometBFT `ProposerPriority`, sending the validator to the back of the proposer order; (4) a set with mixed key types loses CometBFT batch signature verification, falling back to one-by-one and slowing block times (applies whenever the set is non-uniform, not only at rotation). Evan restructured into `###` subsections and added a lead Danger callout; fixed that callout from lowercase `` to `` so it renders. +- Light-client mechanics deep-dive (for the update-cost item): verified against CometBFT source (~/Documents/tests/cometbft) — `DefaultTrustLevel = 1/3` at light/verifier.go:15, 1/3 floor enforced at light/verifier.go:197, and the skip pivot is 9/16 of the gap (light/client.go:31-32,749-750), not the spec's 1/2, for header-cache reuse. The on-chain `07-tendermint` client (ibc-go) carries its own per-client `TrustLevel`; the bisection runs relayer-side via the CometBFT light client library. +- staking/README consensus-key-rotation store list (Matt r3634329631): regenerated for cosmos-sdk#26616 — split the single unbonding-window "maturity" into two concerns and added a fifth store. `ConsKeyRotationQueue` (0x91) now retires only the re-rotation rate limit at unbonding-window end; the old cons address's evidence lock (`RotationLockedConsAddrIndex`, 0x93) now persists until equivocation evidence is no longer admissible; new `ConsKeyEvidenceExpiryQueue` (0x95 | expiryTime | ConsAddress) retires that lock from the evidence time+height windows captured at rotation. Grounded in the PR's keys.go diff. +- kms/best-practices (dianab-cl r3638165488): added a one-line latency note to "Place the signer in its own trust domain" — every layout adds network latency between node and signer, weigh against isolation. +- kms/remote-signing (dianab-cl r3638174921, r3638208406): added a Mermaid sequence diagram of node → signer → backend privval signing in "What is remote signing?" (Mermaid renders in this repo; used across ibc/skip-go/sdk), with a two-sentence explanation; added one sentence to the Cosmos-KMS section that it speaks the CometBFT privval protocol, so it signs for any node implementing privval, not one specific client. +- Verification of this pass: `npx mint broken-links` clean; a subagent verified the Mermaid diagram against kms source (all 5 claims PASS, no corrections); a subagent checked anchor links across keys/kms/staking pages (58 PASS / 2 FAIL, both pre-existing auto-generated staking-README TOC self-links `#queues-1` and `#msgs`, not from this work — the `#msgs` → `## Msg's` apostrophe case is the only possible live break, left for separate cleanup). +- Added a security warning to the top of all four rotation guides (rotate-validator-key, rotate-validator-key-poa, rotate-key-remote-signer, migrate-validator-ml-dsa): "Key rotation can introduce security implications for your chain. Read the Key rotation overview in its entirety before proceeding," linking to /sdk/next/keys/key-rotation. On migrate-validator-ml-dsa it sits above the existing IBC counterparty warning. +- v0.55 upgrade-guide cross-check (v0.55.mdx) against the keys/kms/rotation docs. Fixes applied, all code-confirmed: (1) post-quantum-keys IBC section — corrected "CometBFT v0.40 raises the maximum block size" to "raises its signature-size limits" (cometbft types/signable.go:13 MaxSignatureSize now includes mldsa65.SignatureSize, types/block.go:602 MaxCommitSigBytes derives from it; these are signature caps, not block max_bytes); (2) key-rotation fee param `KeyRotationFee` → `key_rotation_fee` (JSON name per staking.pb.go:934; default 1000000 params.go:41; pool key_rotation_fee_pool pool.go:17; fee denom must equal bond denom); (5) added a downtime-slashing-continuity note to key-rotation Slashing subsection — on rotation the missed-block record and jailed status move to the new consensus key (verified on cosmos-sdk origin/main: x/staking/keeper/rotation.go:524 ApplyConsKeyRotation → AfterValidatorConsKeyUpdated hook → x/slashing/keeper/hooks.go:52 MoveValidatorSigningInfo + MoveMissedBlockBitmap). Verified but not added: (4) rotation event names rotate_cons_pubkey / apply_cons_pubkey_rotation exist (x/staking/types/events.go:13-14) but stay out of the how-to guides (reference-level). Left alone: (3) secp256k1eth (EVM-specific, tangential). Not fixed here per Evan: the upgrade guide's own `--algo ml_dsa_65` slip (actual flag is `--key-type`, confirmed via `simd keys add --help`). +- EVM docs (we own them): evm/next/documentation/concepts/accounts.mdx — the Consensus Nodes entry listed `ed25519` only; added a consolidated note that on SDK v0.55 EVM chains can opt into `secp256k1eth` (Ethereum-style consensus addresses, #26615) and `ml_dsa_65` via `pub_key_types`, that an EVM validator can run any of these consensus signature types incl. post-quantum ML-DSA while EVM user accounts follow Ethereum's account conventions (post-quantum lands upstream), linking the SDK enable/post-quantum pages and the IBC counterparty caveat. Also fixed a pre-existing typo `eth_secp265k1` → `eth_secp256k1`. Eric confirmed EVM will target SDK v0.55 (so evm/next timing is right) and the naming split: validator/consensus type `secp256k1eth` vs account type `eth_secp256k1` (our edit uses this correctly). Evan then edited evm/next: pointed the SDK links at `/sdk/latest/` (publish state), dropped the "on SDK v0.55" version prefix, and reworded the account line to "ML-DSA keys are not currently supported for EVM user accounts." Mirrored the same block + `eth_secp265k1`→`eth_secp256k1` typo fix into evm/latest/documentation/concepts/accounts.mdx (Evan won't run an EVM version migration, so both latest and next carry it directly). +- Known caveat (accepted): the four `/sdk/latest/keys/{enable-ml-dsa-keys,post-quantum-keys}` links from the two EVM accounts pages 404 on the current branch — those SDK pages live in sdk/next and only move to sdk/latest at the SDK v0.55 freeze. Correct for the published state; `npx mint broken-links` will flag them until the SDK side freezes. Left as `/sdk/latest/` per Evan. +- configure-backend AWS KMS (Eric r3646632648): added a Warning about the AWS KMS raw-signing size limit — the backend signs the raw consensus message (`MessageType=RAW`, awskms.go:109; ed25519 is PureEd25519 per algo.go:40) and AWS KMS rejects anything larger than 4096 bytes, so be careful with features that enlarge the signed message, such as vote extensions (Eric's vote-extension case is one example of the general limit; grounded in kms source + cosmos/kms#37). +- FOU-878 review (Dmitry Shlemin) applied. post-quantum-keys: reordered the algorithm lists so curve-based come first and the post-quantum one is last (user account `secp256k1`, `eth_secp256k1`, `ml_dsa_65`; consensus `ed25519`, `secp256k1eth`, `ml_dsa_65` — used the correct consensus EVM name `secp256k1eth`, not the account name Dmitry wrote); promoted the "post-quantum key" definition to a top `` and removed the now-duplicate sentence from "What post-quantum means"; added a highlight line above the cost table calling out the ~3,300-byte signature and block-data growth. enable-ml-dsa-keys: reformatted the consensus-params `jq` one-liner into a readable multiline pipeline, output verified byte-identical to the original (including the 48h0m0s→172800s duration conversion). create-ml-dsa-account: no change (reviewer marked it "All good"). +- PoA rotation LastCommit limitation + vote extensions (Matt, Slack D0ADSEMGCH1): documented that CometBFT's two-height validator-update delay makes `LastCommit` carry the old consensus address for two heights after a rotation. Staking waits out those heights and keeps a historical address mapping so the old address resolves; PoA swaps immediately and keeps no mapping, so a stock PoA chain is unaffected (no x/distribution or x/slashing consuming it) but custom `LastCommit`-address lookups can't resolve a rotating validator for two heights. Because vote-extension signatures are verified by that lookup, vote extensions are not supported on PoA — a rotating validator's precommits are rejected, and if >2/3 of voting power rotates within one window the chain halts. Added to key-rotation "Staking and PoA chains" (full mechanism + Evan's lead Warning), a Warning at the top of enterprise/poa/architecture.mdx linking back to key-rotation, and merged into the lead Warning on the rotate-validator-key-poa how-to (via an "In particular" clause, to avoid stacking callouts). Aligned with Alex + Eric that PoA won't be fixed, just documented. +- v0.55 upgrade guide (sdk/next/upgrade/v0.55.mdx, destined for the cosmos-sdk repo as raw GitHub-rendered markdown, so links use full https://docs.cosmos.network/sdk/latest/ URLs not Mintlify paths): added "for more" doc links to the New Features sections. Validator Consensus Key Rotation → key-rotation, rotate-validator-key, rotate-validator-key-poa. ML-DSA-65 Validator Consensus Keys → post-quantum-keys, enable-ml-dsa-keys, migrate-validator-ml-dsa. ML-DSA-65 Account Keys → create-ml-dsa-account, post-quantum-keys. secp256k1eth → post-quantum-keys. (I also proposed a Cosmos-KMS remote-signing link block on the rotation section; Evan removed it, so the final page links only keys/ pages.) Verification method for pages that aren't live yet: since the freeze promotes sdk/next→sdk/latest verbatim, each /sdk/latest/ URL is valid iff sdk/next/.mdx exists and is registered in docs.json — confirmed all 14 targets pass both checks (they 404 on the live site today because keys/ and kms/ aren't in sdk/latest until the v0.55 freeze, which is correct for the guide's eventual SDK-repo home). Canonical frontmatter target (/sdk/latest/upgrade/upgrade) exists in sdk/latest. + +## 2026-07-28 (accuracy audit fixes, batch 1) + +Four fixes from the 42-page accuracy audit (full findings: ~/Documents/tests/security-release-audit-findings.md). All verified against cosmos/kms `origin/main` @ cbd79ab, since the pinned working tree 538e5c5 predates ML-DSA and secp256k1eth entirely. + +- kms/configuration-reference.mdx:44 — the per-backend algorithm sentence claimed `secp256k1eth` and `mldsa65` were file-backend only. All three backends support `ed25519`, `secp256k1eth`, and `mldsa65`; only `secp256k1` is AWS-KMS-only. Sources: `config/validate.go:13` (`supportedPKCS11Algorithms` = ed25519/secp256k1eth/mldsa65), `:18` (`supportedAWSKMSAlgorithms` = those three plus secp256k1), `signing/file/file.go:20-28` (`Open` switch: ed25519/secp256k1eth/mldsa65). The old text also contradicted configure-backend.mdx:23 on the same site. +- kms/configuration-reference.mdx:95 — the `grpc.keys` `algorithm` row had all three backend mappings wrong or incomplete and omitted `pkcs11`, which the `backend` row directly above lists as valid. Corrected to file/pkcs11 = ed25519 or secp256k1eth, awskms = ed25519/secp256k1/secp256k1eth, per `internal/app/build.go:262-289` (`newGRPCSigner`, whose own doc comment enumerates exactly these three cases). Added that `mldsa65` is privval-only: `config/validate.go:21-24` (`grpcAlgorithms` excludes it because it has no SignatureScheme in the signerservice proto), enforced at `validate.go:262`. +- kms/configuration-reference.mdx:97 — fixed as a consequence of the row above, not separately requested. Once `algorithm` admits `ed25519` on the file backend, "Path to the hex-encoded secp256k1 private key file" becomes self-contradictory. Now matches the `keys[].key_file` row at :52: priv_validator_key.json, or raw base64 for ed25519 / hex for secp256k1eth (`signing/file/ed25519.go:70-89`, `signing/file/secp256k1.go:30-48`). +- kms/rotate-key-remote-signer.mdx:7 — the intro said the signer was "listening on port 26659". The node listens on its privval port and the signer dials it (`internal/manager/dialer.go` dials `validators[].addr`; `cometbft privval/utils.go:63-80`). Inverted the direction the rest of the KMS set is built on (remote-signing.mdx:48, best-practices.mdx:13), and contradicted this page's own step 2 at :63. Line 13's description of the second node is unaffected and was already correct. +- keys/create-ml-dsa-account.mdx:86 — typo `safterwards` → `afterwards`; dropped trailing whitespace. + +Not touched, deliberately: the `grpc.keys` table still omits the PKCS#11 fields (`module`, `token_label`/`slot`, `key_label`/`key_id`, PIN source) that `config/config.go:164-166` embeds inline in `GRPCKey` and `validate.go:258-260` validates. Flagged in the audit as a separate omission; needs its own row set or a pointer to the `backend: pkcs11` section. + +## 2026-07-28 (accuracy audit fixes, batch 2) + +Refreshed all three source checkouts first (they were stale; cometbft by 6 days). cometbft and kms fast-forwarded to origin/main (a54e79b, cbd79ab); cosmos-sdk left on branch pr-26614 and read via origin/main refs @ 8083adfaf1. The refresh materially changed findings: see "upstream drift" below. + +- upgrade/v0.54.mdx:466,486,490-491 — the BlockSTM wiring snippet did not compile. Wrong import path (`baseapp/blockstm`, which has never existed; the package is `baseapp/txnrunner`), the 4th arg mislabeled "debug logging" when it is `estimate`, and `sdk.DefaultBondDenom` passed where the signature wants `func(storetypes.MultiStore) string`. Verified against tag v0.54.0 (7df93688, immutable): `baseapp/txnrunner/blockstm.go:15-23` gives the exact signature. Inherited verbatim from upstream UPGRADING.md, so worth fixing there too. The repo's own experimental/blockstm.mdx:165-171 already had it right. +- cometbft/next/docs/core/configuration.mdx:412 — `max_tx_bytes` 1048576 → 4194304. UPSTREAM DRIFT, not a doc error: correct at 6ac238b, changed by cometbft#5989 (config/config.go:1033). Upstream updated its own copies in the same commit. +- cometbft/next/spec/abci/Requirements-for-the-Application.mdx:644 — default block ceiling "21 MB" → ~53 MiB. UPSTREAM DRIFT via cometbft#5987: `DefaultBlockParams.MaxBytes` is now `22020096 + MaxCommitBytes(MaxVotesCount)`. Computed from source: MaxSignatureSize=3309 (ML-DSA) → MaxCommitSigBytes=3355 → MaxCommitBytes(10000)=33,580,094 → total 55,600,190 B = 53.02 MiB. Kept the 21 MiB data budget in the wording since only the total ceiling moved, and named ml_dsa_65 as the cause, which is this release's own subject. +- genesis.mdx:28 + Requirements-for-the-Application.mdx:552 — `bls12381` → `bls12_381`. The pub_key_types string is `bls12_381` (cometbft crypto/bls12381/const.go:11, consumed at types/params.go:29); `bls12381` is only the Go package and build-tag name, so an operator copying it into genesis hits validation failure at types/params.go:228-231. Added the build-tag and ed25519-default caveats to genesis.mdx. Same edit fixed the grave-accented `ìnt64` typo at :555. +- staking/README.mdx:748,1151,1256 — fallout from #26616 landing (e57b2cb363, now an ancestor of origin/main). :1151 and :1256 were CORRECT when written and are now false: retention is no longer the unbonding period but `max(unbondingTime, MaxAgeDuration)` plus `applyHeight + MaxAgeNumBlocks` (keeper/rotation.go:1171-1184; keys.go:104-107 says "unbonding alone is too short"), and the maturity queue no longer unlocks the old address at all. Post-#26616 there are two queues on two schedules: maturedConsKeyRotationKeys (rotation.go:1071-1095) retires only the 0x91 entry and the 0x92 rate-limit marker; maturedConsKeyEvidenceLockKeys (rotation.go:1100-1147) releases 0x93. Also rewrote :748 (the 0x93 entry carries a value of lock-kind + operator addr, with three kinds at keys.go:127-146, and the pending-target lock is released when the rotation applies, not on the evidence schedule) and corrected :749's key format to include the length-prefixed ValAddress and note the value is the new pubkey. +- key-rotation.mdx:71 — "more than 2/3 of voting power" was wrong on two counts. ValidateVoteExtensions (baseapp/abci_utils.go:107-112) returns on the FIRST unresolvable consensus address, before any power accounting, so one rotating validator fails the whole set; and even under a tally-only reading the threshold at abci_utils.go:144 breaks at 1/3, not 2/3. Reworded to drop the fraction entirely. +- rotate-validator-key.mdx:109 — "slashable ... until the unbonding period ends" superseded by #26616, same as staking/README:1151. Now states the evidence window and links to key-rotation. + +Verified-correct claims whose earlier UNVERIFIED/LIKELY-WRONG verdicts are withdrawn (no edit needed): staking/README:744 "five stores" and :750 ConsKeyEvidenceExpiryQueue 0x95 (keys.go:112, builder at :555-564) are exactly right now that #26616 merged; key-rotation.mdx:43 matches the "Known limitation" comment at rotation.go:1160-1163 almost verbatim; key-rotation.mdx:59 genesis export confirmed; v0.55.mdx:216/:277 event attributes and :246 secp256k1eth description confirmed against #26619/#26615; ML-DSA 1952/3309 confirmed (#26626 was behavior-preserving). + +Correction to the 2026-07-23 entry: the v0.55 guide's `--algo ml_dsa_65` is NOT a slip. `--algo` still resolves via the normalize func at client/keys/add.go:98-104, so both `--algo` and `--key-type` work. No edit needed there. + +Still open and highest-risk: "CometBFT v0.40" is unverifiable in every place it appears. The SDK pins cometbft v0.39.3 with a v0.39.0-rc1 replace, and cometbft's own version/version.go:6 still reads TMCoreSemVer = "0.39.0" (which also makes configuration.mdx:30's `version = "0.40.0"` sample wrong today). Confirm against the CometBFT release plan before publishing. + +## 2026-07-28 (accuracy audit fixes, batch 3) + +Scope note: sdk/next/upgrade/** is out of scope per Evan. The batch-2 edit to v0.54.mdx (BlockSTM snippet) was reverted, so that guide is untouched and its four snippet errors stand unfixed by decision. Upgrade-guide findings are dropped from the audit list. + +- node/run-production.mdx:121,123 — corrected two privval filenames that do not exist: `config/priv_val_key.json` → `config/priv_validator_key.json` (and "directory" → "file"), `data/priv_val_state.json` → `data/priv_validator_state.json`. Confirmed at cosmos-sdk origin/main (tools/systemtests/node_utils.go:18) and consistent with run-node.mdx:53. Pre-existing, but sharper now: the TMKMS section this branch removed was the only place on the page carrying the correct spellings. +- cometbft/next/spec/core/encoding.mdx:62-66 — the Secp256k1 address derivation was a verbatim copy of the Ed25519 block above it, wrong on two counts. Verified firsthand in the refreshed tree (a54e79b): crypto/secp256k1/secp256k1.go:153 doc comment reads "Address returns a Bitcoin style addresses: RIPEMD160(SHA256(pubkey))", implementation at :154-166, and PubKeySize = 33 (:144), not 32. RIPEMD160 emits 20 bytes natively so there is no truncation step. Now states the 33-byte compressed key and `RIPEMD160(SHA256(pubkey))`. Did not port upstream's compressed-point explanation or base64/xxd example (out of scope for a correctness fix). Same file: `it's own` → `its own` (:44), `entires` → `entries` (:22). +- modules/staking/README.mdx:1380-1382 — params table. `KeyMaxEntries` → `MaxEntries`: the field is `MaxEntries` (staking.pb.go:926, proto staking.proto:309), and `KeyMaxEntries` was the legacy x/params store key, which is doubly obsolete since x/params is removed in v0.55. Verified `git grep -c KeyMaxEntries origin/main -- x/` returns zero occurrences. Also corrected all three `uint16` → `uint32` (MaxValidators, MaxEntries, HistoricalEntries), confirmed against both the generated struct (staking.pb.go:924,926,928) and the proto (staking.proto:307,309,311). + Deliberately NOT changed: the HistoricalEntries value 3. Verification turned up `DefaultHistoricalEntries uint32 = 10000` (params.go:31), which looked like a third error, but the table column header is "Example", not "Default" — and the UnbondingTime row is likewise an example (3 days) rather than the 3-week default. So 3 is a legitimate example and the column is internally consistent. Worth a separate decision on whether this table should show defaults instead. +- modules/staking/README.mdx:2758 — `DelegtaorDelegations` → `DelegatorDelegations`. + +Held, not applied: authz/README.mdx stale keeper snippet. Evan approved "trim the block" based on my description of it as lines 511-529 (~19 lines). The block is actually lines 511-978, 467 lines: the whole keeper package dumped inline, mangled by the doc-conversion pipeline, importing github.com/tendermint/tendermint/{abci/types,libs/log} (dead org) and cosmos-sdk/store/types (now cosmossdk.io/store), with `storeKey storetypes.StoreKey` on the Keeper where it is now a store service. A second such block follows at 984-1096. Deleting 467 lines is materially different from what was approved, so this is pending a fresh decision. The specific defect that started it stands: the embedded `DequeueAndDeleteExpiredGrants(ctx sdk.Context)` at :948-951 contradicts the corrected prose at :509, since the real signature is `(ctx context.Context, limit int32) error`. + +## 2026-07-28 (accuracy audit fixes, batch 3 continued) + +- modules/authz/README.mdx — deleted both stale inlined keeper dumps: lines 511-978 (the keeper package: SaveGrant, IterateGrants, removeFromGrantQueue, DequeueAndDeleteExpiredGrants, etc.) and 984-1096 (keys.go: store prefixes and key helpers). 583 lines removed, 1342 -> 759. Evan chose deletion of both after I corrected my earlier mis-measurement of the first block (I had described it as ~19 lines). + Rationale: both blocks were mangled by the doc-conversion pipeline (`func (k Keeper)` split across lines, broken indentation) and imported dead paths, `github.com/tendermint/tendermint/{abci/types,libs/log}` and `cosmos-sdk/store/types`, with `storeKey storetypes.StoreKey` on the Keeper where it is now a store service. The first block also contained `DequeueAndDeleteExpiredGrants(ctx sdk.Context)`, whose real signature is `(ctx context.Context, limit int32) error`, directly contradicting the prose fix this branch made at :509. + Pre-deletion safety checks, all clean: no prose on the page refers to the code ("see above/below", "following code" all absent); no headings sat inside either range, so no anchor could target them; no inbound `modules/authz/README#` anchors anywhere in the repo or docs.json; fence count stayed balanced at 60; section structure intact (Abstract, Contents, Concepts, State, Messages, Events, Client). `npx mint broken-links` unchanged at the 5 known items. Backup of the pre-deletion file kept in the session scratchpad. + The surviving prose carries the information the blocks were meant to illustrate: the GrantQueue key format bullet, the expiration_bytes format note, and the GrantQueueItem description. + +Flagged, not changed: authz/README.mdx now line 511 still reads `-> ProtocalBuffer(GrantQueueItem)` (should be `ProtocolBuffer`; the same page spells it correctly elsewhere). It is an identified WRONG finding but was not in the approved batch, so left alone. + +## 2026-07-28 (KMS guide verification: Tier 1 blocker fixes) + +Source: an execution-based verification run of all seven KMS guides against shipping refs (kms tag v0.1.0 / 4bd8392, cometbft tag v0.40.0 / 0880b4d, simd from cosmos-sdk origin/main 64fd208a11). Report at /private/tmp/.../kms-verify/KMS-DOCS-VERIFICATION-REPORT.md. Result: 4 BLOCKER, 7 MAJOR, 12 MINOR. This entry covers the 4 blockers only. + +Two premises corrected by that run: +- The suspected privval incompatibility between simd and kms v0.1.0 is REFUTED. cosmos-sdk origin/main now pins cometbft v0.40.0 directly in both go.mod and simapp/go.mod with no replace directive; the v0.39.3 + rc1-replace exists only on the older pr-26614 branch. kms v0.1.0 signed for simd across 80+ blocks including a live rotation, so `noise://` is available on the shipping pairing and the "CometBFT 0.40 or later" prerequisite is correct. +- CometBFT v0.40.0 is released (tag 0880b4d). My earlier "v0.40 is UNVERIFIED" verdicts are withdrawn. My cometbft clone had refspec +refs/heads/main only, which is why the tag was invisible; the two config values I changed in batch 2 were re-verified against the tag and both hold. + +Blocker 1 (three pages): kms v0.1.0 fails closed on a missing or empty sign-state file, so a first `kms start` aborts. Shipped by PR #35 (c952fac), which postdates the 538e5c5 commit both tutorials' hidden verification notes cite. Neither `--allow-fresh-state` nor `kms state init` appeared anywhere in the seven pages. +- tutorial-file-backend.mdx step 7: added `--allow-fresh-state kms-demo-1`, the verbatim error it prevents, and the `kms state init` alternative. +- tutorial-file-backend.mdx step 8: the signer no longer creates the state file by signing, so "confirm the signer created" became "confirm ... is in place, written when the signer started". +- tutorial-file-backend.mdx step 9: deliberately left bare, and now says why. This is load-bearing. The flag is inert once the state file exists (chain_signer.go reads it and ignores the flag), so a reader who carries it into every start sees no symptom while losing the protection: a truncated state file would reseed the floor at height 0 instead of refusing to start. Each of the three added Warnings states this scoping. +- rotate-key-remote-signer.mdx step 3: added the flag. This was the worst instance, a brand-new home for a never-used key, which can never have a state file, with no recovery path on the page. +- configure-backend.mdx "Verify any backend": this section defers to the tutorial rather than printing its own `kms start`, so it got a pointer plus the error string rather than a flag. +- configuration-reference.mdx: added the sign-state precondition as a fourth startup constraint, a note that the list is not exhaustive, and a new "First start on a new chain" section documenting both escape hatches with the same scoping warning. + +Blocker 2: best-practices.mdx "Prefer the Noise transport". `kms peer-id` reads /identity.json only, ignores `validators[].identity_key`, and on a miss generates and persists a new key while still exiting 0, so it can print an identity the signer never uses. Pinning that value yields a validator that cannot start (`rejecting non-allowlisted signer peer`, then a pubkey timeout); the verification run confirmed with a positive control that pinning the true ID fixes it. Added a Warning covering the config-ignoring and the key-minting behavior, with the instruction to confirm the printed ID against what the signer reports before pinning. Folded in the D14 gap while there: the page said "pin them" without naming a field, so it now states explicitly that the signer's ID goes in the validator's `priv_validator_laddr` and the validator's goes in the signer's `validators[].addr`, that the two are cross-wired, and what the failure looks like if inverted. Also added that a `noise://` host must be an IP literal (D7's root cause), since it belongs in the same passage. + +Not fixed here: the 7 MAJOR and 12 MINOR divergences. Of those, only D7 (migrate-from-tmkms noise:// hostname) changes a command a reader types; the rest are claim corrections verifiable by source read. + +Re-test required before this ships. The edits change what a reader types in guides 1, 2 and 4, so those need another end-to-end run, plus the noise pinning path for blocker 2. The hidden verification notes in tutorial-file-backend.mdx and rotate-key-remote-signer.mdx still cite kms 538e5c5 / 7932ceb and should be restamped with the new refs once the re-run passes. + +## 2026-07-28 (KMS guide verification: Tier 2, F4 / F5 / F7) + +Three of the six MAJOR divergences. F3 (algorithm not required for pkcs11), F6 (gRPC algorithm list) and F8 (mistyped section silently ignored) remain open by decision. + +- configure-backend.mdx:19 (F4) — the AWS KMS 4096-byte Warning was wrong on both counts. Rewritten to say the cap applies to `ed25519` and `secp256k1` only, that the signer enforces it locally and fails before calling AWS, and that `mldsa65` and `secp256k1eth` are not bound by it. Verified at tag v0.1.0: `kmsRawMessageLimit = 4096` (signing/awskms/signer.go:77) gated on `msgType == types.MessageTypeRaw` at :87, before the `client.Sign` call; per algo.go, ed25519 and secp256k1 are Raw (:71, :79), secp256k1eth is Digest (:87), mldsa65 is ExternalMu (:101). Included the verbatim error string and the 64-byte mu figure, confirmed at algo.go:140 (`mu := make([]byte, 64)` from ShakeSum256). This mattered because the Warning sat directly above a section whose only worked example is ML_DSA_65, the one algorithm it does not apply to, so it steered readers away from the release's headline feature. +- configure-backend.mdx:136 (F5) — the wrong-key symptom was wrong. `invalid proposal signature` never fires: the node fetches its consensus pubkey from the signer at startup, does not find it in the validator set, and demotes itself to non-validator, so it never proposes and nothing is ever rejected for a bad signature. The string exists in cometbft (consensus/errors.go) but only when validating a peer's proposal. Replaced with the real signal, `This node is not a validator` plus a chain that does not advance. The guidance to adopt the key first was already correct and is unchanged. This was the last remaining item that sent a debugging operator looking for a string that does not exist. +- migrate-from-tmkms.mdx:26 (F7) — the mapping table's `noise://` form now says the host must be an IP literal, bracketed for IPv6, and that hostnames are rejected (cometbft privval/noise_listener.go, `net.ParseIP(host) == nil`). Also named whose peer ID it is: the validator's, from `cometbft show-node-id --libp2p`, which is the opposite of the ID that goes in the validator's own `priv_validator_laddr`. The row previously implied `noise://` and `tcp://` accept the same address shape, so a migrating operator copying a working tmkms hostname got a signer that would not start with an error that did not point back at the table. This also resolves an inconsistency introduced in the Tier 1 pass, where best-practices.mdx gained the IP-literal rule while this page still contradicted it. + +F7 is the only one of the three that changes what a reader types, so the re-test scope is unchanged from the Tier 1 entry apart from adding one `noise://` config check. + +## 2026-07-28 (accuracy audit fixes, batch 4) + +Items 2, 3, 4, 5 of the audit queue. Skipped item 1 (cometbft spec/core/genesis.mdx) per Evan. All non-KMS: the seven kms/ pages are under a blind execution test in a separate session and must not shift under it. + +- cometbft/next/spec/abci/Requirements-for-the-Application.mdx (item 2, verified against tag v0.40.0 before editing, as asked). Three fixes: dropped the stale "(as of v0.38.x)" qualifier from a page shipping in the v0.40-targeted tree; added `AuthorityParams.Authority` to the parameter list (renumbered to 9) plus a new `##### AuthorityParams.Authority` subsection; fixed the `####` heading on ABCIParams.VoteExtensionsEnableHeight to `#####` so it nests under "List of Parameters" like its seven siblings rather than rendering as a peer. Source at v0.40.0: types/params.go:55 (`Authority AuthorityParams \`json:"authority"\`` as the sixth member of ConsensusParams), :114-116 (the struct, an opaque string), :174 (DefaultAuthorityParams returns empty), :251 (validated). Anchor unchanged by the heading fix, so no link breakage. +- ADR-050 trio (item 3). Rewrote eight internal links from `/sdk/v0.50/build/architecture/...` to `/sdk/next/reference/architecture/...`: seven to the adr-050 pages and their annexes, one to adr-020. Confirmed `adr-020-protobuf-transaction-encoding.mdx` exists in sdk/next before redirecting. These were stale copy artifacts sending readers of a `next` page into the v0.50 archive. Also `invertability` -> `invertibility` (annex1), and annex2's frontmatter: title was `ADR 050: SIGN_MODE_TEXTUAL: Annex 2 XXX`, an unfilled placeholder rendering in the sidebar and browser tab, now "Annex 2 Device Rendering"; description was the changelog line `Oct 3, 2022: Initial Draft`, now an actual description. + Not touched, flagged: adr-050-sign-mode-textual.mdx:291 has an external buf.build URL beginning `hhttps://` (doubled h). Real broken link, unrelated to the cross-version issue. +- Visible sweep (item 4). guides/abci/app-mempool.mdx:54 had an unclosed markdown link (`in [\`app.go\`:`) rendering as a literal bracket; removed the stray bracket. :74 read "keeps transactions from an sorted by nonce in order to avoid the issues with nonces", missing a noun and circular; rewritten to "keeps each account's transactions sorted by nonce, so they are proposed in the order the account signed them". modules/authz/README.mdx:511 `ProtocalBuffer` -> `ProtocolBuffer`. +- Item 5, both verified against tag v0.55.0 first. + distribution/README.mdx:279 claimed module accounts are blocked "by being added to the distribution keeper's `blockedAddrs` array at initialization". No such field exists; the check is delegated to bank (`k.bankKeeper.BlockedAddr(withdrawAddr)`, x/distribution/keeper/keeper.go:85). Reworded to say the keeper consults bank through `BlockedAddr`. Also deleted the dead pseudo-code block at :289-305 (18 lines with the trailing blank): it was pipeline-mangled, not valid Go ("fail with `ErrSetWithdrawAddrDisabled`"), referenced the nonexistent `k.blockedAddrs`, and misquoted the error as "not allowed to receive external funds" when the real wrap is "is not allowed to receive funds". The prose above it, including the correct paragraph this branch added at :281, covers the behavior more accurately than the snippet did. Backup in the session scratchpad. + staking/README.mdx:1153-1157 failure list had three of seven causes. Added four, all confirmed in x/staking/keeper/msg_server.go at v0.55.0: validator does not exist (ErrNoValidatorFound, :655), validator is jailed (:672), new key already in use by another validator (ErrConsensusPubKeyAlreadyUsedForValidator, :659), new key locked by a rotation (ErrConsensusPubKeyInRotationHistory, :650). + +Pre-existing defect found and NOT fixed: distribution/README.mdx ends with a stray four-backtick fence (````) as its last line, present on main, so not from this branch. It leaves the file's fence count odd (161 before my deletion, 159 after; I removed a balanced pair). Likely a conversion artifact. Worth a separate decision since it may affect rendering of the final code block. + +- Two pre-existing defects flagged in batch 4 and then fixed per Evan: + distribution/README.mdx: removed the stray four-backtick fence that was the file's last line. Confirmed orphaned before deleting: exactly one 4-backtick line in the file, at EOF, with nothing after it, and the other 158 fences pair up. Present on main, so not from this branch. Fence count now balanced at 158. + adr-050-sign-mode-textual.mdx:291: external buf.build link began `hhttps://` (doubled h). Fixed to `https://`. Left the v0.50 path in that URL alone, it is a buf.build module reference for the version the ADR describes, not a docs-site path. + +## 2026-07-28 (accuracy audit fixes, batch 5) + +Items 3 and 5 of the queue. Item 1 (cometbft spec/core/genesis.mdx), item 2's sibling key-rotation.mdx items, and key-rotation.mdx:55 (batch verification) all left as-is per Evan. + +- keys/rotate-validator-key.mdx:113-116 (item 3, kept deliberately brief per Evan). The "What can go wrong" list covered four of the seven causes the message server enforces. Added two compact bullets rather than four verbose ones: the new key being unavailable (already used by another validator, or still locked by a recent rotation) and the validator being jailed. Confirmed at cosmos-sdk tag v0.55.0, x/staking/keeper/msg_server.go: ErrConsensusPubKeyAlreadyUsedForValidator (:659), ErrConsensusPubKeyInRotationHistory (:650), "validator is jailed" (:672). Left out ErrNoValidatorFound (:655) for brevity, since a reader following this guide is rotating a validator they already run. +- keys/rotate-validator-key.mdx:105 — "matches the key shown by `simd comet show-validator`" -> "matches the `key` field shown by". The base64 payloads do match, but `/validators` returns it under `value` while `show-validator` returns it under `key`, so a reader comparing the two JSON blobs field by field is briefly misled. +- modules/staking/README.mdx, both `params` query samples (item 5). Verified at v0.55.0 that `Params` has seven fields (staking.pb.go:922-934); both samples showed five, omitting `min_commission_rate` and `key_rotation_fee`, the latter being the parameter this release adds. Added both to the CLI YAML output (alphabetical, matching the sample's existing order) and to the gRPC JSON output (proto field order 1-7, matching that sample's order). Coin rendering matched to the house style already used at README:1427-1428: quoted `amount`, bare `denom`. min_commission_rate rendered as the quoted LegacyDec string the params table at :1384 already shows. +- modules/staking/README.mdx CLI Transactions section (item 5). Added a `rotate-cons-pub-key` entry after `cancel-unbond`, matching the existing entry format (prose, Usage, Example). Syntax and example taken verbatim from the command's own definition at v0.55.0, x/staking/client/cli/tx.go: `Use: "rotate-cons-pub-key [new-pubkey]"`, `cobra.ExactArgs(1)`, and the Long text's own example with a proto-JSON `Any` and `--from`. The section previously documented six subcommands and omitted the one this release adds. + +## 2026-07-28 (KMS gRPC fix, approved earlier and not applied until now) + +- kms/remote-signing.mdx:49 — the gRPC algorithm list was wrong in both directions. It said the gRPC signer service signs `ed25519` and `secp256k1eth`; it also serves `secp256k1`, and excludes `mldsa65`. Verified at kms tag v0.1.0: `grpcAlgorithms = {AlgoED25519, AlgoSecp256k1, AlgoSecp256k1Eth}` (config/validate.go:23), with the comment at :20-22 stating mldsa65 has no proto scheme and is privval-only. Backed by proto/signerservice/signerservice.proto (no ML-DSA member) and internal/app/build.go:277-278. The earlier execution audit confirmed both halves live: a `secp256k1` gRPC key passes config validation, and `mldsa65` is refused up front with `config: grpc.key[0] algorithm "mldsa65" is not supported over gRPC`. The first half of the same bullet, the privval list, was already correct and is unchanged. +- kms/configuration-reference.mdx, `grpc.keys` table — added a pointer to the `backend: pkcs11` field set. `GRPCKey` embeds `PKCS11Config` inline (config/config.go:164-166) and validates it with the same helper as a privval key (config/validate.go:258-260), so a pkcs11 gRPC key needs `module`, `token_label`/`slot`, `key_label`/`key_id` and a PIN source, none of which the table listed. Pointed at the existing section rather than duplicating seven rows. Note this gap was partly self-inflicted: an earlier fix in this cycle corrected the same table's `algorithm` row to say pkcs11 is valid there, which advertised a backend whose required fields the table did not document. + +Timing caveat: these two pages were under a blind execution audit in a separate session when this edit landed. If that audit reports on remote-signing.mdx:49 or the grpc.keys table, cross-check against this entry before treating it as a live finding. + +## 2026-07-28 (KMS blind execution audit: fixes) + +Source: an independent blind execution audit of all seven KMS guides against shipping refs (kms v0.1.0/4bd8392, cometbft v0.40.0/0880b4d, simd v0.55.0/64fd208a). 88 claims falsification-tested. Report at ~/Documents/tests/kms-docs-audit-findings.md. Applied 10 items; deliberately skipped D12-D19 per Evan (less is more): none of those stops a reader completing a guide. + +REGRESSION NOTE, important. Three earlier fixes in this cycle had been reverted before the audit read the files: configure-backend.mdx:19 (AWS 4096 scoping) and :136 (wrong-key symptom) were back to their original wrong text, and best-practices.mdx was fully reverted, losing the kms peer-id warning, the cross-wiring rule and the IP-literal note. Verified by `git diff --stat`, which no longer listed best-practices.mdx at all. The audit independently confirmed all three corrections were right, then flagged their absence as MAJOR regressions. Cause of the revert unknown; worth understanding before the next round. + +- configure-backend.mdx:19 — restored the 4096-byte scoping, tightened. The cap binds ed25519 and secp256k1 only (MessageTypeRaw); the signer checks size itself and fails before calling AWS; mldsa65 (EXTERNAL_MU, 64-byte mu) and secp256k1eth (Digest) are exempt. The audit confirmed by source and by the repo's own test signing ~5200 bytes on mldsa65. +- configure-backend.mdx:136 — restored the real symptom. `invalid proposal signature` never fires: the node demotes itself to non-validator and never proposes. Audit reproduced it: 0 occurrences of the quoted string, chain height 0, `This node is not a validator` in the log. +- best-practices.mdx, Noise section — restored the peer-id warning, the cross-wiring rule and the IP-literal requirement, all three confirmed empirically by the audit (D10). Also fixed D9 in the same edit: `cometbft show-node-id --libp2p` needs the standalone cometbft binary, since `simd comet show-node-id` has no such flag and no page listed that binary as a prerequisite. Without this the page's recommended transport cannot be configured from the toolchain the docs establish. +- best-practices.mdx:21 (D8) — "In both local layouts the key still never touches disk" was false for the file backend and contradicted three other pages. Now says the key stays off the node's disk, and that only an HSM or cloud KMS backend keeps it out of a file. Also resolves the "both" ambiguity over a three-bullet list. +- tutorial-file-backend.mdx step 8 (D1, the only BLOCKER) — "promptly, within a few seconds" was a hard five-second cliff that never self-heals. Audit measured PASS at 1s and 5s, FAIL at 6s and above, from two fixed constants: kms dial backoff reaching 10s (internal/manager/manager.go:20-21) versus the node's single 3s pubkey fetch (cometbft node/setup.go:736-739). Changed to "within five seconds" and promoted the recovery Note to a Warning that states the cliff, the cause, and that a hand terminal switch usually exceeds it. +- tutorial-file-backend.mdx:83 (D11) — "so no key material remains on the node host" was false: the node regenerates a fresh consensus key file when it finds none. Audit verified with a moved-away key: node ran fine on the remote signer and recreated a private key on disk. Reworded to "reduces what is on the node host rather than leaving it key-free". +- tutorial-file-backend.mdx:127 and configuration-reference.mdx:126 (D6) — my own Warnings argued the danger of a permanent `--allow-fresh-state` is that it is invisible. Inverted: the signer logs `starting with fresh sign state at height 0` on every restart even when the floor is intact (internal/app/build.go:83-85). Dropped the false rationale, kept the real risk. +- configuration-reference.mdx:110 (D4) — my own sentence "Other startup errors are prefixed `config:` and name the field at fault" was false. Only config-level rejections carry that prefix; `app:`, `file:` and the wrapped sign-state error do not. Now distinguishes config-level from open-time errors. +- configuration-reference.mdx constraints (D5) — added that a declared chain with no `validators` entry is not rejected: the signer starts, binds nothing and signs nothing, silently. Audit reproduced (exit 0, no listening socket, no dial attempt). This is the failure mode of a dropped or mistyped `validators` block, and best-practices' accurate "no open ports" claim makes it look correct. +- migrate-from-tmkms.mdx step 4 (D7) — the raw `jq` redirection silently lowered a real floor by 5,345 heights in the audit's test. Added a Warning pointing at `kms state init`, which refuses to lower an existing floor. Also resolves the contradiction where configuration-reference.mdx:124 said `state init` is how a migration seeds a non-zero floor while this page never used it. + +Not applied, per Evan: D12 (privval carries four requests, not three), D13 ("at or below" is wrong for the equal case, which is served from cache), D14 (the two floor-seeding paths write different step values), D15 (quoted errors are correct substrings), D16 (`kms init` prints a bare `.`), D17/D18 (SoftHSM2 module path and SOFTHSM2_CONF sequencing), D19 (leftover `@` is a running-signer symptom, not a startup failure). + +Upstream issues the audit surfaced, for the kms repo rather than the docs: the `kms init` scaffold omits `algorithm` from its file-backend key block, so the generated config will not start; the scaffold's gRPC comments claim `backend` is optional and that `algorithm` defaults to `secp256k1` for file, both wrong; and dial backoff has no jitter, so a restarted signer fleet retries in lockstep (relevant to D1). + +## 2026-07-28 (keys guides blind audit: fixes) + +Source: independent blind execution audit of the seven sdk/next/keys pages against cosmos-sdk v0.55.0 (64fd208a11) + cometbft v0.40.0, PoA via enterprise/poa/simapp. 42 claims falsification-tested. Report at ~/Documents/tests/keys-docs-audit-findings.md. Applied 6 items; skipped D4, D6, D8, D10, D11, D12, D14, D15 per Evan (less is more): none blocks a reader. + +- rotate-validator-key.mdx:71 (D1, the page's blocker) — the co-located-node paragraph called the pprof port collision "a harmless `address already in use` error" and said the node "keeps running". It does not: a pprof listener failure is fatal at startup, the process exits, zero blocks commit, and step 2 then has the reader health-check a dead node. Audit reproduced deliberately, with no pprof substitution in effect. Rewritten to make the distinct pprof port required rather than optional tidying, and switched the remedy from a config.toml edit to the `--rpc.pprof_laddr` flag, which the page never mentioned and which keeps the fix on the same command line as the other three port flags. +- create-ml-dsa-account.mdx:49 (D3, blocker) — step 1 run verbatim wrote the key to the default home ~/.simapp under keyring-backend `os` (the macOS Keychain), while the funded account lives in ~/.node under `test`. Silent: the reader gets a working ML-DSA key in an unrelated keyring, and step 2 fails one step later with a confusing "key not found" for an account they know exists. The `--home` guidance existed only in a hidden comment and at :68 scoped to "both transfers". Added `--home ~/.node --keyring-backend test` to the command plus one sentence saying it applies to every command on the page. +- rotate-validator-key-poa.mdx:41 (D2) — prerequisite demanded "enterprise/poa v1.1 or later". Confirmed against upstream that only enterprise/poa/v1.0.0, v1.0.0-rc.0 and v1.0.0-rc.1 exist, so the gate is unsatisfiable and a careful reader stops before step 1, even though every procedural step then works. Reworded to a build that includes MsgRotateConsPubKey, added after v1.0.0. +- post-quantum-keys.mdx:38 (D7) — the Grover paragraph attributed a 256-to-128-bit reduction to a sentence whose named examples are block transaction hashes and merkle trees. That figure is preimage resistance; the property governing those examples is collision resistance, which was already about 128 bits classically (birthday bound 2^128) and is essentially unaffected by Grover. The conclusion was sound, the reasoning was not, on the page whose purpose is that reasoning. Now attributes the figure to preimage resistance and adds one sentence on collision resistance. +- enable-ml-dsa-keys.mdx:70 (D5) — said the local-testnet path uses "the same flag on `simd testnet`". `simd testnet` accepts no flags but -h; --consensus-key-algo lives on `simd testnet init-files` and `simd testnet start`. Corrected to name both subcommands. +- migrate-validator-ml-dsa.mdx:66 and rotate-validator-key-poa.mdx:37 (D13) — both hidden comments said the PoA ML-DSA rotation was not executable on merged code, failed with `unknown pubkey type: ml_dsa_65`, and needed #26614. It merged as d6a3c6e27a and the audit ran the rotation successfully on v0.55.0. Replaced the stale TODO and the stale ML-DSA clause with the new verification result, including that --gas auto is required and where ml_dsa_65 is registered (enterprise/poa/x/poa/module.go:134). + +Notable verified-correct results worth keeping in mind: enable-ml-dsa-keys is the strongest page in the set, and its Danger about removing an in-use key type is now empirically confirmed (chain froze at h434 with `CONSENSUS FAILURE!!! ... validator is using pubkey ed25519, which is unsupported for consensus`). The staking rotation reproduced exactly, including the two-height swap and the two separate charges (1000000 burned fee plus 2000 gas). The mnemonic-reuse warning at create:58 is literally true: crypto/hd/mldsa65.go returns Secp256k1.Derive() and the 32 bytes are used undigested as the ML-DSA seed, with no hashing or domain separation. + +Left as-is per Evan: key-rotation.mdx:39 (D9), where the time component of the tracking window is max(UnbondingTime, MaxAgeDuration) and the page omits the unbonding floor. Error is in the reader-safe direction (the real window is longer than stated). + +Still outstanding, and now lost three times: the best-practices.mdx fixes for D8 (key never touches disk), D9 (noise:// needs the standalone cometbft binary) and D10 (peer-id warning, cross-wiring rule, IP-literal note) from the KMS audit. Commit 849cad50 contains the reverted version, and `git log -S` confirms my text was never committed at any point, so it is gone from history as well as the working tree. Also outstanding: tutorial-file-backend.mdx now has a duplicated recovery Note at :141 and :155, committed and reader-visible. + +- tutorial-file-backend.mdx — removed the duplicated step-8 recovery Note. The same advice appeared twice, at :141 and :155, with slightly different endings, both committed in 849cad50. Kept the :141 instance: it sits immediately after the `simd start` command that produces the failure, and it carries the "within five seconds" detail that matches the corrected step-8 opening line. Removed the :155 instance, which sat after the state-file check, several steps past where the failure occurs. Fence count still balanced at 26. + +## 2026-07-28 (keys/kms run notes brought current; TODOs removed) + +- Deleted the two `TODO(ERIC): please verify the content of this page and run it for yourself` markers (kms/tutorial-file-backend.mdx, kms/rotate-key-remote-signer.mdx). Both asked for exactly what the 2026-07-28 blind execution audits performed: the tutorial came back PARTIAL with its one blocker since fixed, and rotate-key-remote-signer came back PASS with every command running verbatim on the first attempt. +- Restamped all eight hidden run notes to the shipping refs (kms v0.1.0 4bd83922ee, cometbft v0.40.0 0880b4d378, simd v0.55.0 64fd208a11) with the audit date and each page's verdict. This staleness had real cost: both tutorials carried notes saying "verified" against kms 7932ceb and 538e5c5, commits predating PR #35, which is how the fail-closed sign-state blocker survived to release. The tutorial note now carries an explicit warning that any future re-verification must use a ref at or after v0.1.0. +- Two notes deliberately NOT restamped, because the recent audits did not re-cover them and a later date would overstate the evidence: + configure-backend.mdx awskms — the 2026-07-28 audit skipped AWS (no account, no simulating), so the 2026-07-20 real-AWS run remains the only end-to-end evidence. Marked as such in the note itself so nobody restamps it by reflex. Its claims were re-checked from source at v0.1.0 and hold. + migrate-from-tmkms.mdx — the 2026-07-28 audit completed a full cutover from the page alone, but tmkms was not installed on that host and its softsign key and state file were synthesized in the documented formats. The 2026-07-20 run remains the only evidence involving real tmkms. Marked accordingly. +- Retired two stale re-verification instructions that are now discharged: "Re-verify at #26604 merge" (merged, in v0.55.0) and "Re-confirm when #26614 merges" (merged as d6a3c6e27a; the PoA ML-DSA rotation now runs). +- Earlier refs are kept in each note under "retained for provenance" rather than deleted, so the history of what was tested against what remains readable. + +## 2026-07-28 (release-family page: 2026.1 in-place update) + +Decision source: Slack thread in C0BAQAJAS4F (Dmitry Shlemin opening 11:50 EDT). Dmitry proposed dropping SDK 0.53.x / Comet 0.38.x and spinning up a 2026.2 family. Eric Warehime instead described updating a family in place when the upgrade is low friction: the current family lists only the new versions, and intermediate versions are explicitly abandoned. Dmitry restated it as "2025.1 is v53, 2026.1 is v55 and we explicitly abandon v54"; Eric confirmed "yeah I think so"; Alex Johnson confirmed "it is what Eric said" and "which is the existing plan". So no 2026.2, and 2026.1 absorbs the new versions. + +- sdk/latest/release-family.mdx, Current Release Families: 2026.1 SDK 0.54.x -> 0.55.x and CometBFT 0.39.x -> 0.40.x. This is the whole point of the change: the family previously listed two versions per project, which was Dmitry's original complaint. +- Removed the Examples section entirely, per Evan. It was the blocker for a clean edit: its "Upgrades that would require a new release family" list named SDK 0.54.x to 0.55.0 and CometBFT 0.39.x to 0.40.x, which are precisely the two upgrades the new policy applies in place. Rewriting it accurately is not possible yet because Dmitry's question "what is a trigger to spin up a new release family?" was never answered in the thread (they planned a huddle, Dmitry went afk, it ended on "we could sync later"). Removing a wrong rule is better than guessing at the right one, and the example family table duplicated Current Release Families anyway. Confirmed no inbound links to the #examples anchor before deleting. +- Synced to next/ with scripts/sync-latest-to-next.js. This also resolved a pre-existing divergence between the two copies, in the right direction. Origin of that drift: on 2026-04-29 aljo242 ran 7425522e "Unify release family lifecycle policy source" against latest/ only, then 2de68385 "Restore lifecycle details on canonical release-family page" caught next/ up but never reconciled the overview paragraph. latest/ carried "This page is the canonical source of truth for release family lifecycle..."; next/ instead carried "For lifecycle policy and maintenance windows, see the Security and Maintenance Policy". next/'s version was circular: both security-policy pages say the canonical source is the Release Families page and that they intentionally do not duplicate lifecycle timelines. The sync propagated latest/'s correct sentence. The two files now differ only by the expected /sdk/latest/ vs /sdk/next/ link rewrite. + +Deliberately NOT changed, all unresolved in the thread: +- The 2025.1 family and the End of Life list. Dmitry asked to drop SDK 0.53.x and Comet 0.38.x support, but Eric's roughly-1-year-LTS remark points the other way and nobody settled it. If 2025.1 retires, its versions move to End of Life. +- "Cosmos Labs supports up to two release families at a time" and "Release cadence targets two new release families per year". A 1-year LTS plus families now absorbing minor bumps in place makes the two-per-year target questionable, but that is a policy call. +- Any statement of what triggers a new family. The page now says nothing about it rather than something wrong. + +## 2026-07-28 (release-family page: remaining component versions, and a PoA revert) + +Evan confirmed 2025.1 is still supported, so no change there: it was already listed under Current Release Families, and two supported families satisfies the existing "up to two at a time" stance. The suspected contradiction there does not exist. Declined: a supersession note recording SDK v0.54.x / CometBFT v0.39.x as abandoned, so those two versions currently appear nowhere on the page, neither in a family nor in End of Life. + +- sdk/latest/release-family.mdx, 2026.1: checked all eight component rows against upstream tags, not just the two the Slack thread covered. Three were badly stale and are now updated: Solidity IBC Eureka 0.1.x -> 3.0.x (latest v3.0.2), Relayer 0.1.x -> 1.1.x (latest v1.1.0), Attestor 0.1.x -> 1.0.x (latest v1.0.0). Kept the row's existing major.minor.x convention rather than the enterprise rows' 1.x.y form. Verified still correct as written: SDK 0.55.x (v0.55.0), CometBFT 0.40.x (v0.40.0), Enterprise Groups 1.x.y (v1.0.0), Enterprise PoA 1.x.y (v1.1.0), IBC Go v11.x.y (v11.2.0). Synced to next/. + Caveat recorded at Evan's request being noted here instead: these come from newest upstream tags, which is what is released, not what has been compatibility-tested against the rest of 2026.1. The page's premise is that every version in a family is verified against the others, so someone should confirm this is the tested set. + Incidentally, the Examples section deleted earlier today used "Relayer 0.1.x to 1.0.0" as its example of an upgrade requiring a new release family. The relayer has since crossed 1.0 and then 1.1, inside 2026.1, without a new family. Further evidence that section was wrong rather than merely stale. +- sdk/next/keys/rotate-validator-key-poa.mdx:41 — REVERTED my earlier fix. The keys audit checked upstream and found only enterprise/poa v1.0.0 and two rc tags, so I had replaced the "v1.1 or later" prerequisite with a vaguer capability statement. enterprise/poa/v1.1.0 was then tagged the same day at 13:30 EDT (51cbd30e, mattac21, "chore(poa): remove sdk replace on poa module (#26699)"), roughly two hours after the audit looked. Confirmed it contains both #26590 (c52e67a39d, MsgRotateConsPubKey) and #26614 (d6a3c6e27a, ML-DSA support), so "v1.1 or later" is the correct and more precise gate. The audit was not wrong; the tag simply landed after it. +- Verified all nine GitHub URLs on the release-family page: every one returns 200 with no redirect, so no repo renames and nothing stale. + +Unresolved inconsistency found while checking the links, not fixed: "What a Release Family Contains" lists seven components including Cosmos EVM, but neither family table pins an EVM version, while both tables pin Enterprise Groups and Enterprise PoA, which the list does not mention. The hedge at line 26 about SDK packages arguably covers the two enterprise modules being extra, but nothing covers Cosmos EVM being listed as pinned and then absent from every family. + +Still open from the Slack thread: what triggers a new release family (Dmitry asked, never answered), and whether "Release cadence targets two new release families per year" survives in-place family updates plus a roughly 1-year LTS. + +## 2026-07-28 (pre-release blind runthrough: fixes) + +Source: blind execution run of 13 in-scope pages, all commands verbatim, at shipping refs (kms v0.1.0 4bd83922ee, cometbft v0.40.0 0880b4d378, simd v0.55.0 64fd208a11, PoA from enterprise/poa/simapp). Result 4 PASS / 9 PARTIAL / 0 FAIL, 2 BLOCKER + 10 MAJOR + 17 MINOR. Report at ~/Documents/tests/kms-keys-docs-prerelease-audit-2026-07-28.md. Fixed both blockers, 8 of 10 majors, and 2 minors. best-practices.mdx and the AWS section were out of scope for the run. + +Four of the fixes below are corrections to defects I introduced earlier in this cycle. Noting that explicitly because the pattern matters: each was a fix that was right about the problem and wrong about the remedy, and only execution caught them. + +- keys/rotate-validator-key-poa.mdx prerequisites (BLOCKER-1, BLOCKER-2, MAJOR-8). The page pointed at run-node.mdx to build simd, which builds cosmos-sdk/simapp and has no poa module at all: `simd tx poa --help` prints an empty "Transactions subcommands" with no error explaining why. It also required "a running PoA validator you operate" with no documented path anywhere in sdk/next, since PoA validators come from app_state.poa.validators in genesis rather than gentx, and the only recipe in the repo was inside this page's own hidden MDX comment. And the version prerequisite, which I had reworded earlier the same day, was true but unactionable: MsgRotateConsPubKey is in no released enterprise/poa tag. All three collapsed into one rewritten bullet: build from main via `cd enterprise/poa/simapp && go build ./simd`, verify with `simd tx poa --help` listing rotate-cons-pub-key, and a pointer to the PoA API page for the genesis shape. +- keys/rotate-validator-key-poa.mdx admin section (MAJOR-9). Followed in page order the admin command always failed with code 14, because step 4 has already installed the new key on the live node. Added "Generate a fresh key home for it." Trimmed the explanatory half on review: the page already lists "the new key equals the current one" as a rejection in its own What can go wrong section, so restating the error there was redundant. +- keys/rotate-validator-key-poa.mdx Verify block (MAJOR-10, MINOR-17). The block claimed the rotation emits an event with three addresses and told the reader to confirm it with `simd q poa validators`, which shows no event, no consensus address and no signing status. Split into two commands: the module query for the new key, and `simd q tx ` for the rotate_cons_pubkey event. Added the missing --home to both, which had them falling back to ~/.simapp. +- kms/configuration-reference.mdx "First start on a new chain" (MAJOR-1, MAJOR-2), my section from earlier today, and the audit's highest-value fix. Two defects: `kms state init` fails on a freshly initialized home with `chain "" not declared`, because it loads and fully validates kms.yaml before deriving the state path, so it cannot be the bootstrap step I presented it as; and the two commands I called equivalent write different floors, since `state init` defaults --step to 3 (refuses everything at that height and round) while --allow-fresh-state writes step 0. Rewritten to present --allow-fresh-state as the way to seed a first floor, with state init's config prerequisite and step default stated rather than implied. +- kms/migrate-from-tmkms.mdx:113 (MAJOR-6), also mine from earlier today. My Warning named the real hazard and prescribed a remedy that cannot work: `kms state init` is a bare existence check (internal/signer/state.go:19-21), so it refuses whenever a state file is present, in either direction. Three empirical attempts against a floor at height 51, lower/higher/mid, all failed identically. The real answer, per the source's own comment, is to delete the file first, which the page never said. Rewritten to that. The hazard itself was reproduced: the raw jq silently lowered a floor from 999 to 37, exit 0. +- keys/create-ml-dsa-account.mdx:67 (MAJOR-4), also mine. I added the "use the same --home and --keyring-backend on every command" instruction at :52 without updating step 2's printed command, so the page contradicted itself. Run verbatim it failed twice, once in the nested `simd keys show` substitution and once in the outer send, with the real error buried under a usage dump, and it silently created a stray ~/.simapp. Added the flags to both the outer command and the substitution. +- kms/configure-backend.mdx:141 (MAJOR-5). The new-validator adoption path was circular: it told the reader to read the pubkey from a node's /status, but gentx is pre-genesis and a single-validator chain cannot start before its validator is in genesis. No kms subcommand exports a pubkey either. Replaced with reading the key from the custodian directly, pkcs11-tool --read-object for PKCS#11 and aws kms get-public-key for AWS, and corrected `create-validator --pubkey` to the pubkey field of its validator.json. +- kms/remote-signing.mdx:15 (MAJOR-3). "Three requests" is four: PingRequest is live, the node pings on a ~3.3s timer, kms's read timeout is sized around it, and operators see `SignerListener: Ping timeout`, which the audit observed. I had skipped this twice as low-impact; the log evidence changed that. +- kms/migrate-from-tmkms.mdx:26 (MAJOR-7) and prerequisites (MINOR-6). The noise:// row covered only the signer side; noise is mutual, so the validator must also carry the signer's peer ID from `kms peer-id` or it rejects the connection with `signer peer not allowlisted`. Added that, plus jq to the prerequisites, which steps 4 and 6 both require on the mandatory path. +- keys/enable-ml-dsa-keys.mdx:103 (MINOR-8). The live-chain governance path never mentioned the voting period, two days by default, so the page cannot be completed in a sitting. Also dropped the claim that the submission returns a proposal ID, which it does not; `simd query gov proposals` is the actual route. + +Not fixed, per Evan: the remaining 15 minors, including the -y prompts on the PoA tx commands, the PKCS#11 macOS path note ordering, the truncated sign-state error quotes on three pages, and the precision lists for remote-signing, post-quantum-keys and key-rotation. Also still declined: the post-quantum performance reframe (signing is 4.9x slower, not "slightly", and ML-DSA has no batch verifier) and key-rotation:39's unbonding-time floor. + +Upstream defects the run surfaced, for the kms repo: the `kms init` scaffold omits `algorithm` from its file-backend key block so the generated config will not start, and its gRPC comments claim `algorithm` defaults to secp256k1 for file, which is rejected at runtime. + +Still outstanding on best-practices.mdx, now lost three times and never committed: the D8/D9/D10 fixes from the earlier KMS audit (key-never-touches-disk, noise:// needing the standalone cometbft binary, the peer-id warning and cross-wiring rule). + +- configure-backend.mdx:141 — corrected my own incomplete fix from earlier in this pass. I had written "then base64 the raw key bytes", omitting that the PKCS#11 object comes back as a DER SubjectPublicKeyInfo and the 12-byte prefix 302a300506032b6570032100 must be stripped first, leaving 32 raw bytes. As written a reader would have base64'd the whole DER blob and handed gentx a wrong key: a circular instruction replaced with an incorrect one, which is worse because it looks followable. Now gives the full pipeline and notes AWS KMS returns the same DER wrapper. +- configuration-reference.mdx:121 — restored the step-0 vs step-3 difference at Evan's request after it was trimmed. Both floor-seeding commands remain on the page, so without this a reader can pick either and get materially different protection: `kms state init` defaults --step to 3, which refuses everything at that height and round, while --allow-fresh-state writes step 0. + +## 2026-07-28 (v0.55 release notes and upgrade nav) + +- Renamed sdk/next/upgrade/release.mdx to v0.54-release.mdx; added v0.55-release.mdx, ported from the Notion 2026.1 family draft. +- docs.json: "v0.54 Upgrade" group renamed "v0.55 Upgrade", with a collapsed "Previous versions" subgroup for the v0.54 pages. Moved blockstm into In-depth Guides > ABCI. Redirects for the renamed release/upgrade paths in both next/ and latest/; the latest/ destinations land when the version script runs. + +## 2026-07-28 (3-page blind confirmation run: fixes) + +Source: blind confirmation run of the three pages I had changed, verbatim, at shipping refs. Report at ~/Documents/tests/blind-verification-3page-2026-07-28.md. Result: configure-backend PASS, create-ml-dsa-account PARTIAL, rotate-validator-key-poa PARTIAL. Three of my four fixes from the previous pass were incomplete or wrong; this entry corrects them. + +What held: configure-backend passed outright. The DER-stripping pipeline I had just corrected at :144 was verified byte-exact, raw object `302a...2539` with the wrapper stripped base64ing to the same value `gentx --pubkey` accepted, and `Using slot 0` correctly goes to stderr without polluting the pipe. + +- keys/create-ml-dsa-account.mdx:57, :77, :83 — I had fixed step 2's command to carry `--home ~/.node --keyring-backend test` but left the same defect on three other commands: the `--recover` example, the balance query, and the step-3 send, each also with a nested `simd keys show` substitution that needs the flags too. Fixed one instance of a pattern and left three. All four now carry the flags on both the outer command and the substitution. The failure mode is worse than a plain error, since on a machine with an existing ~/.simapp the commands silently address the wrong keyring instead of failing; the run reproduced that, recreating ~/.simapp mid-session. +- keys/rotate-validator-key-poa.mdx:41 — my build instruction did not work. `cd enterprise/poa/simapp && go build ./simd` fails with `go: build output "simd" already exists and is a directory`, because `./simd` is the package directory. Since this is the page's first prerequisite, the reader was stopped before step 1, so my fix for BLOCKER-1 replaced one blocker with another. Now `go build -o /tmp/poa-simd ./simd`, with a note that `-o` is required and why. +- keys/rotate-validator-key-poa.mdx:43 — my fix for BLOCKER-2 pointed at /sdk/next/enterprise/poa/api for the genesis shape. That page has no genesis content at all: `grep app_state.poa` on it returns nothing, and its headings run Overview through Appendix with no genesis section. Worse, reconstructing genesis from its documented `Validator` type panics at InitGenesis with `unknown field "allocated_fees"`, because that field does not exist on the message at v0.55.0. So the blocker was never cleared. Replaced the pointer with the minimal genesis shape inline: `app_state.poa.params.admin` plus a `validators` entry with pub_key, power and metadata. Ten lines, but a reader cannot stand up the chain this guide requires without them, and no other page carries it. +- keys/rotate-validator-key-poa.mdx:114 — I fixed the admin section's prose to say "generate a fresh key home" but left the command reading `--home ~/.poa-newkey`, which step 4 has by then copied onto the live node. Followed in sequence it still failed `code=14 nothing to rotate`. Added the `simd init` for a fresh home and repointed the rotate command at it. Also: I briefly re-added the explanatory clause Evan had asked me to trim earlier the same day; removed again. + +Upstream doc defect for the PoA batch, outside these three pages: sdk/next/enterprise/poa/api.mdx:37, :45 and :60 document an `allocated_fees` field on `Validator`. At v0.55.0 the message is pub_key, power, metadata only (enterprise/poa/proto/cosmos/poa/v1/poa.proto); fees live in a separate `ValidatorFees` message. Anyone building genesis from that page's type reference hits a panic. Added to the PoA findings doc. + +## 2026-07-28 (final /keys guides blind run: fixes) + +Source: blind verbatim run of the five executable keys guides at cosmos-sdk v0.55.0 (64fd208), PoA binary from enterprise/poa/simapp, six throwaway chains (one per rotation, so the once-per-unbonding-period limit never fired). Result: 4 PASS, 1 PARTIAL. One MAJOR, the rest cosmetic. + +Confirmations worth recording, since these were the fixes from earlier passes: +- Page 5's binary prerequisite now works: `go build -o /tmp/poa-simd ./simd` produced a working binary and `simd tx poa --help` listed rotate-cons-pub-key. +- Page 5's inline genesis JSON was sufficient on its own. The runner explicitly noted needing nothing from the MDX comments, which is what BLOCKER-2 was about. +- create-ml-dsa-account PASS. The flags fix holds on all commands including the nested substitutions, and `gas_used: 257259` against the 200000 default confirms the --gas auto explanation. +- rotate-validator-key PASS. The four port flags at :71 were sufficient for a co-located second node, with no `address already in use`, which is the pprof fix from the previous pass verified. +- migrate-validator-ml-dsa PASS both paths. PoA `gas_used: 233565` is above the default, so the --gas auto requirement at :48 is real; staking `gas_used: 182520` is below it, which is why the page only requires the flag on the PoA path. + +- enable-ml-dsa-keys.mdx:97 (MAJOR, the only non-cosmetic finding) — steps 2 and 3 fail with `mykey.info: key not found` on any node home other than the binary's default. The flag list at :97 enumerated --chain-id, --keyring-backend and --fees but omitted --home, the one that actually fails, and the page never mentioned --home anywhere while pages 1, 3 and 5 all do. Added --home to that enumeration. This was the sole reason page 2 answered No to "could a first-time reader finish". +- enable-ml-dsa-keys.mdx:20 — "Key names and denoms are the only values to adjust, and each appears once" is false: `mykey` appears at both :100 and :106, so a reader trusting the claim changes one and leaves the other. Dropped the clause rather than correcting the count, since it was load-bearing for nothing. + +Not fixed, cosmetic and agreed as skippable: the `-y` omissions at enable:106, migrate:51 and the PoA commands (harmless interactively, and the runner's `echo y |` was a no-TTY artifact); the --home inconsistency between poa:102 and :137 (both work); poa:108 supplying only the `cp` for "stop the node, replace its key, and restart"; create-ml-dsa-account:77's outer query lacking flags (queries need only --node, so it works, though it does sit against the page's own "every command" claim). + +Still open by earlier decision: the voting-period gap at enable-ml-dsa-keys:103-109. The run independently flagged it, noting the default is 172800s and that "When it passes, the new list is live" reads as immediate with nothing indicating a wait. Evan trimmed my earlier note about this; leaving it out pending a call. + +The run also produced a 12-item trimmable list of redundancy across the five pages, which is separate from correctness and has not been actioned. + +## 2026-07-28 (trimming pass, items 1-4 of 12) + +Source: the trimmable list from the final /keys blind run. Editorial, not correctness. Working through it in order. + +- create-ml-dsa-account.mdx step 2 — the trailing paragraph restated `--home` and `--keyring-backend`, already mandated at :52 and now carried explicitly on every command on the page. Trimmed to the only unique content, `--chain-id` and `--fees`/`--gas-prices`. Also dropped the "Both transfers here are ordinary transactions" opener, which referred forward to a transfer that has not appeared yet at that point in the page. +- create-ml-dsa-account.mdx step 3 — deleted the balance query and its lead-in. Step 2's own `code: 0` already establishes the transfer landed, and if it had not, the send in step 3 fails with insufficient funds, which is self-diagnosing. The query also sat under a heading about signing, which is not what it checks. Two commands in that step become one. +- enable-ml-dsa-keys.mdx Warning — cut the sentence describing failure via "a proposal JSON file instead". That is a route the page never instructs, and it was the longer and more alarming half of the Warning, drawing attention from the actionable part (the jq substitution emitting null). Kept the CLI failure and the closing instruction to start from the queried params. + +Evan trimmed the third item himself and went further than I proposed: I argued to keep "The gap between them is your migration progress once validators start rotating" as the only line connecting the two queries to an ongoing operation, and the whole paragraph came out. Not reverted. + +Eight items remain on the list, across rotate-validator-key.mdx, migrate-validator-ml-dsa.mdx and rotate-validator-key-poa.mdx. The largest are the duplicated cross-page content: rotate-validator-key:55's Note reproduces page 4's staking section while linking to it, migrate-validator-ml-dsa:76-81 restates page 3's "What can go wrong" then points at it, and rotate-validator-key-poa carries a third and fourth warning against two simultaneous signers after the Danger at :92. + +## 2026-07-28 (trimming pass, items 5-8 of 12) + +- migrate-validator-ml-dsa.mdx "What can go wrong" — dropped the rotation-limit bullet. It restated page 3's list, and the bullet immediately below it already delegates everything else to page 3, so the pointer covers it. Kept the unsupported-key-type bullet, which is ML-DSA specific and the one failure a reader of this page is most likely to hit. +- migrate-validator-ml-dsa.mdx mnemonic Note — trimmed to the part that belongs on a validator page. The old Note used a true observation (the consensus key has no mnemonic) as a hook to restate page 1's account-mnemonic Warning, on a page that is not about accounts. Kept "raw keypair with no mnemonic behind it, custody the key file, not a seed phrase" and dropped the account cross-reference. +- rotate-validator-key-poa.mdx:90 — reduced the admin forward-reference to a pointer. This was not only redundant: it said "keep the command identical and sign with the admin key", which my own earlier fix made false, since the admin section now generates a fresh key home and points the command at ~/.poa-adminkey rather than ~/.poa-newkey. So the trim doubles as a correctness fix for an inconsistency I introduced. + +Not trimmed, disagreeing with the finding: rotate-validator-key.mdx:55. The report described its Note as duplicating "the whole of page 4's staking section", but the Note is one sentence giving the one-flag delta (`--consensus-key-algo ml_dsa_65`) plus the link. It does not reproduce the section, and removing it would force a reader who wants the post-quantum variant to navigate away to learn that it is a single flag. Kept as signposting. + +## 2026-07-28 (trimming pass, items 9-12 of 12: PoA page) + +- rotate-validator-key-poa.mdx step 3 Danger — cut the last two sentences ("Running two nodes that hold this validator's signing keys at the same time risks a double sign. Never run more than one active signer"). The same hazard is covered more usefully in step 4, which gives the actual sequencing for a redundant standby rather than a prohibition. The Danger now does one job: do not swap before the set switches. +- rotate-validator-key-poa.mdx step 3 — "The value should match the key from `simd comet show-validator --home ~/.poa-newkey`" asked the reader to re-run a command whose output step 1 already had them print; now refers back to step 1. Also dropped the trailing "Confirm the module re-keyed with `simd q poa validators`", which the Verify section already covers. +- rotate-validator-key-poa.mdx prerequisites — scoped the consensus-params bullet to ML-DSA. As written it was vacuous for the ed25519 rotation the page actually walks through, since ed25519 is in pub_key_types by default. Rather than deleting it, which would leave the page's two ML-DSA Notes without their precondition, it now reads "For an ML-DSA rotation, `ml_dsa_65` is in the chain's consensus params." + +The audit counted the two-signer hazard as appearing three or four times; on inspection it was twice (the step 3 Danger and the step 4 standby guidance), with the Danger at :92 being the separate >1/3 halt hazard. Trimmed one of the two. + +REGRESSION, flagged not reverted: the genesis JSON added to the prerequisites to clear BLOCKER-2 has been wrapped in an MDX comment, so it no longer renders. The blind run had confirmed it was exactly what made the page self-sufficient, reporting that the JSON at lines 45-54 was sufficient on its own and that nothing from the comments was needed. With it commented out the prerequisite reads "PoA validators are set in genesis under `app_state.poa`" with no shape, which is the state that stopped the earlier runner before step 1. If the intent was that the genesis recipe belongs elsewhere, the PoA API reference is the natural home, and that page currently has no genesis section at all. + +- rotate-validator-key-poa.mdx:44 — fixed an MDX parse error that broke the page's build. The commented-out genesis block was a multi-line JSX comment containing a fenced code block, sitting unindented between two list items; MDX read its inner lines as lazy continuations and failed with "Unexpected lazy line in expression in container" at 45:1, so `npx mint broken-links` could not parse the file at all. Collapsed the comment to a single indented line with the fences removed, which preserves the intent of keeping it hidden while letting the page parse. Link check now runs clean again, back to the four known /sdk/latest/keys/ items. + Worth noting the fences served no purpose inside a comment, since nothing in there renders. If the genesis shape is ever restored to visible content it should go back as a real ```json block, which also removes this fragility. + +## 2026-07-28 (final grammar pass, /keys and /kms) + +Mechanical sweep plus a line-by-line prose read of all 14 pages. The sub-agent I dispatched for the prose read stalled (139 bytes written, no output for 11 minutes), so I killed it and did the read myself. + +Mechanical: removed trailing whitespace on 5 lines (post-quantum-keys x2, rotate-validator-key-poa, configure-backend, best-practices) and collapsed 9 double blank lines. Verified the bulk regex touched only blank lines, no content: the two files it alone modified show four blank-line deletions and nothing else. Doubled words, it's/its, and mid-sentence double spaces all came back clean across the set. + +18 grammar fixes, all exact-string replacements: +- post-quantum-keys.mdx: "whenever practical quantum hardware arrives" to "when" ("whenever" implies recurrence); "the SHA-256 hash functions that underpin Cosmos SDK" to "hash function that underpins the Cosmos SDK" (SHA-256 is one function, and the article was missing); "so neither has anything to migrate" to "none of them has", since the sentence lists three antecedents, not two; "over 50 times an ed25519 signature" to "over 50 times the size of an ed25519 signature", which was comparing a signature to a multiple; "slower than `ed25519`" to "slower than with `ed25519`", a faulty comparison between operations and an algorithm; "Any counterparty chains that verify" to "Any counterparty chain that verifies". +- key-rotation.mdx: renamed the heading "Mixed key types slow down" to "Slower signature verification". The old form was a verb phrase with no object among three noun-phrase siblings (Slashing window length, Increased IBC light client updates, Proposer priority reset). Confirmed no inbound anchor referenced the old slug. Also fixed an agreement break in the PoA Warning, "custom logic that resolves ... or that use vote extensions" to "or that uses", and backticked `LastCommit` at :71 where the same term is code-formatted everywhere else on the page. +- migrate-validator-ml-dsa.mdx:15 — fixed the garbled prerequisite. "All prerequisites of the rotation procedure, including jq and curl: no rotation in the current unbonding period, and fee funds" read as though the colon defined jq and curl. Restructured so the colon introduces the whole list. +- remote-signing.mdx: "Once replaced, the signer reconnects" was a dangling modifier, since what gets replaced is the node, not the signer; now "Once it is replaced". "validators are recommended to use it" to "validators should prefer it". Added the missing colon in "one file, `kms.yaml`, in three blocks, the chains it signs for, ...", which was a comma splice into a list. +- best-practices.mdx: "In both local layouts" to "In the two local layouts", since the bullets above list three layouts. Added the missing conjunction in "against the same validator key as this can cause double signing". +- migrate-from-tmkms.mdx: "it differs per backend, which step 2 fills in" had an unclear "which"; now "Its fields differ per backend, which step 2 covers". "so it can be evaluated by demand" to "so demand for it can be gauged". "The `jq` makes two needed conversions" to "The `jq` expression makes two conversions". + +Flagged, not changed: key-rotation.mdx:13-15 uses bold for three list labels (Operator key, Consensus key, Node key), which CLAUDE.md prohibits. Left it, since these are parallel definition-list labels where bold does structural rather than emphatic work, and the alternatives lose scannability or read as code. + +Note that best-practices.mdx:21 still carries the unfixed D8 accuracy finding: the sentence claims the key never touches disk in a local layout, which is false for the file backend. The grammar fix corrected only the count. + +Two explicit decisions from Evan at the end of the grammar pass, recorded so they read as choices rather than misses: +- key-rotation.mdx:13-15 bold list labels stay, notwithstanding the CLAUDE.md no-bold rule. +- best-practices.mdx:21 ships with its factual error. The sentence says the key never touches disk in the two local layouts, which is false for the `file` backend, the one a cost-constrained same-host deployment is most likely to choose. Three other pages contradict it (best-practices.mdx:43, tutorial-file-backend.mdx "holds the key in plaintext on disk", configuration-reference.mdx "the key is held in plaintext"). The earlier KMS audit rated it MAJOR on the grounds that it is a security claim wrong in the permissive direction, in the section a reader consults to defend a design in review. Left per decision. + +## 2026-07-28 (Next steps relevance pass, /keys and /kms) + +Reviewed every "Next steps" section across the 14 pages. All links resolved already; the issue was relevance. A consistent pattern showed up: the conceptual and hub pages did not link forward to all of their procedures, while the procedures linked back. Readers navigating forward hit dead ends, readers going backward were fine. + +- post-quantum-keys.mdx — added `create-ml-dsa-account`. The page covers four key types, separates users from validators in "Who can adopt ML-DSA?", and has an EVM accounts section, but every Next step was consensus-key: release notes, enable, rotation. A reader who came for a post-quantum account had nowhere to go. Placed after the upgrade bullet, since accounts need no chain-level change, giving an upgrade / user / chain / validator order. +- key-rotation.mdx — split "Perform a rotation" into staked and PoA. The page has a whole "Staking and PoA chains" section and links the PoA procedure inline, but Next steps pointed every reader at the staking guide. One-directional: the PoA page already linked back here. +- enable-ml-dsa-keys.mdx — retargeted from the generic staking rotation to `migrate-validator-ml-dsa`, which is the page for someone who has just enabled ML-DSA, and which covers both the staking and PoA paths so no separate PoA bullet is needed. Also reversed the two bullets, since the second said "Understand the rotation mechanics first" while appearing second. +- migrate-validator-ml-dsa.mdx — reworded the first bullet. It promised "the allowed-vs-in-use commands", a framing that came from the paragraph on enable-ml-dsa-keys explaining that the gap between the two queries is your migration progress. That paragraph was trimmed earlier today, so the link resolved but the concept no longer existed at the destination. Direct fallout from the trimming pass; now describes what the page actually still shows. +- rotate-validator-key.mdx — added its two variants, `migrate-validator-ml-dsa` and `rotate-key-remote-signer`, both of which already linked to this page. Dropped "Check what key types the chain allows", which is a prerequisite the page already lists at :35 rather than a next step. +- kms/tutorial-file-backend.mdx — added `best-practices`. The tutorial's own closing says the file backend "is not production custody", so hardening is the natural next move, and remote-signing, migrate-from-tmkms and rotate-key-remote-signer all linked to best-practices while the tutorial did not. + +Left as-is: create-ml-dsa-account, best-practices, rotate-validator-key-poa, configure-backend, rotate-key-remote-signer. remote-signing carries six items, which is a lot, but it is the section hub. configuration-reference and migrate-from-tmkms have two each, defensible for a reference page and a one-way migration. + +Net +5 bullets and -1. Slightly against the trimming direction, but these are navigation rather than prose, and each addition covers a place a reader previously stopped. Link count unchanged at the four known /sdk/latest/keys/ items. + +## 2026-07-28 (Mintlify anchor rules: corrected in CLAUDE.md) + +Evan reported a broken anchor on sdk/next/upgrade/v0.55.mdx. Root cause turned out to be CLAUDE.md's anchor rule table, which said `.` is dropped from heading slugs. It is not: it becomes a hyphen. + +Verified empirically by rendering a scratch page with headings varying the punctuation, then clicking both candidate anchors for each. Results: +- `.` becomes `-`. Confirmed on v0.53.x, v0.39, 1.2.3, a.b, v1.0, and e.g. Every dropped-dot form failed; every hyphenated form worked. +- `/` is kept and must be percent-encoded in the fragment: `Removed: x/params` is `#removed-x%2Fparams`. CLAUDE.md was already right that the slash is kept; my hypothesis that it hyphenates was wrong. +- Markdown smart punctuation runs before slugification, so the character in the file is not always the character in the anchor. `Ellipsis... in a heading` becomes `#ellipsis…-in-a-heading`, with three dots collapsed into a single U+2026 that is then kept literally. +- Adjacent replacements collapse to one hyphen: `Use e.g. sparingly` is `#use-e-g-sparingly`, not `#use-e-g--sparingly`. +- Colon confirmed dropped, consistent with the existing rule. + +Fixed the three references to the reported anchor: v0.55.mdx:8 and :25, plus the cross-page one at v0.55-release.mdx:73, all `#upgrading-from-v053x` to `#upgrading-from-v0-53-x`. Nothing else on the upgrade guides touched, per scope. + +Updated CLAUDE.md's Anchor Links section: moved `.` out of the dropped set into its own rule with examples, added the `%2F` requirement for slashes, added the collapse rule, and added a paragraph warning that smart punctuation transforms heading text before slugging, so anchors for headings with punctuation beyond letters, digits, spaces and hyphens must be copied from a render rather than derived. Also recorded that `npx mint broken-links` validates page paths only and never checks fragments, which is why this class of error has always shipped silently. + +Scope check before acting further, read-only: 2,965 headings across 570 pages contain a dot or slash. Of the anchors actually referencing them, 147 are wrong specifically because a dot was dropped, almost all in ibc migration pages replicated across four version directories (`#cosmos-sdk-v050-upgrade`, `#ics20---transfer`, `#ics27---interchain-accounts`). A separate 664 anchors do not resolve for unrelated reasons: renamed or missing headings, and at least one percent-encoded `?` where the rule says `?` is dropped. So roughly 800 anchors in this repo do not resolve and nothing in CI has ever checked. + +Deliberately not swept. None of the 147 touch sdk/next/keys, sdk/next/kms, or any page in the release surface, so this does not block the release. The 664 bucket will contain false positives from smart-punctuation cases I have not characterised, and my slugifier is a model of a system that surprised us twice in one sitting. The durable fix is a linter that checks fragments against the `id` attributes of a rendered page rather than against any written-down rule; that would have caught all ~800 without modelling anything. Filed as post-release work. + +Scratch test page sdk/next/anchor-rule-test.mdx deleted; it was never registered in docs.json and no reference to it remains. + +## 2026-07-29 (SDK v0.55 and CometBFT v0.40 release) + +- Froze both products. SDK: `latest/` archived to `sdk/v0.54/`, `next/` promoted to `latest/` as v0.55 (upstream v0.55.0, released 2026-07-27). CometBFT: `latest/` archived to `cometbft/v0.39/`, `next/` promoted as v0.40 (upstream v0.40.0). Changelogs in `next/` were regenerated first so they carried through the promotion, per the release process. +- Both freeze runs failed at the final nav step, as documented in `scripts/versioning/CLAUDE.md`: `updateNavigation` looks up the dropdown by `SUBDIR.toUpperCase()` ("SDK", "COMETBFT") and the actual labels are "Cosmos SDK" and "CometBFT". The archive and promotion steps had already completed, so only `docs.json` and `versions.json` needed doing by hand. Did both: new `latest` entries cloned from `next` with paths rewritten, old `latest` entries demoted to archive entries with paths rewritten to `v0.54/` and `v0.39/`, order set to latest, next, then archived newest-first. `versions.json` got the new archive dirs and bumped `latestDisplayVersion`. +- Deleted `rfc-template` and `adr-template` from all six SDK version directories, 13 files including a stray nested duplicate at `sdk/v0.50/build/rfc/rfc/rfc-template.mdx`. Added 13 redirects to the parent index pages, removed the 6 `rfc-template` nav entries, and de-linked 13 inbound references to backticked filenames, matching how `PROCESS.mdx` already refers to them. `adr-template` was never registered in SDK nav and had no inbound markdown links. Hub's own `adr-template` pages were left alone per decision; they are a different product and are the only `adr-template` entries in the nav. + +### Changelog generator rewritten to stop dropping content + +Found while reviewing the generated release notes: `manage-changelogs.js` decomposed each release body into `{sections: {name: [bulletString]}}` and then rebuilt it, so anything not expressible as a bullet string inside a named section was silently discarded. Three symptoms, one cause. + +- Wrapped bullets lost everything after their first line. CometBFT wraps heavily, 437 continuation lines in its changelog, so 31 of 44 bullets on the new `latest` page ended mid-sentence with their PR links gone. +- Versions with prose and no `###` section vanished entirely. v0.54.2 was absent from the v0.54 changelog for this reason. +- Nested sub-bullets were flattened to siblings, so the four modules moved to `./contrib` under SDK #25090 read as four independent breaking changes. + +Replaced `parseChangelog` and `generateMintlifyContent` with a passthrough: slice on `##` version boundaries and copy the body verbatim, applying only heading demotion, MDX escaping, bullet-marker normalization, and removal of empty bullets and empty section headers. Only header lines are interpreted now, so the body cannot lose content because it is never rebuilt. + +The escaping is also a real fix rather than the previous whitelist. `sanitizeLine` only escaped `<` and `>` with spaces around them; bare `{` was never handled, and gaia's changelog has an unbacked `{delegatorAddr}` that would have broken the MDX build on its next regeneration. Now `<` and `{` are escaped everywhere outside inline code and fenced blocks. + +Verified by generating every product and target with the old and new code against identical source refs and diffing. `sdk/next` came out byte-identical, so the current SDK release notes did not regress. Recovered content elsewhere: CometBFT bullets with lost text went 31 to 0 on `latest` and 49 to 2 on v0.39, hub 155 to 37 with 16 previously-dropped bullets restored, and v0.54.2 is back. Residual cases are upstream bullets that simply do not end in punctuation. Release labels also improved, since dates on their own italic line are now lifted into the `` instead of being dropped: hub went from 45 entries labelled "Release" to 3. + +Regenerated only the four files in this release. `evm`, `ibc`, and `hub` will pick up the new behavior on their next run, which is worth knowing before the next freeze since hub's output grows by about 400 lines. + +- Re-tagged the four regenerated changelog files, which lost their front matter when rewritten: `noindex` plus `canonical` restored on both `next/` pages, and `tag-archived.js` re-run for the two archive dirs, 1 file each with the rest correctly skipped. Also synced `cometbft/latest/changelog` which the promotion had left holding pre-fix output. + +`writeChangelog` never reads the file it replaces, so a regeneration writes the generator's three hardcoded front matter fields and drops everything else. `noindex` and `canonical` are the obvious casualties, but the subtler one is that a hand-corrected value of a field the template does own is reverted silently: `ibc`'s five archived changelogs are `mode: "center"` against the template's `wide`, and `cometbft/v0.38` reads "Cosmos CometBFT" against the template's uppercased `product.toUpperCase()`. Both revert on their next regeneration. + +Per Evan, left alone deliberately: no preservation logic in the generator and no reordering of the freeze's tagging step. The four files stay tagged, consistent with every other product's changelog page. The cost is that this recurs on each post-freeze regeneration and someone has to re-tag, which is accepted rather than unnoticed. + +One unrelated gap surfaced while checking this and was not touched, since `ibc` is outside this release: `ibc/next/changelog/release-notes.mdx` carries no `noindex`, while every other product's `next/` changelog does and 160 of 185 `ibc/next` pages are tagged. That page is currently indexable against `ibc/latest`. +- `npx mint broken-links` clean. Every `docs.json` nav path resolves to a file, all 52 redirects have live destinations and no cycles, and the six changelog pages scan clean for unescaped JSX characters outside code. + +## 2026-07-29 (post-freeze fixes and version-pinning findings) + +- Bumped the displayed version label from v0.54 to v0.55 on the five SDK pages that carry it in front matter `description`, in both `latest/` and `next/`: `learn.mdx`, `tutorials.mdx`, `reference/spec.mdx`, `reference/rfc.mdx`, `reference/architecture.mdx`. This is what renders as "Version: v0.54" under the page title, and the freeze does not touch it. Archived directories correctly keep their own label. Only the SDK uses this pattern. +- Fixed `learn/concepts/store.mdx`, which documented `TraceKVStore` as a live store wrapper. Store tracing was removed upstream in v0.54 (PR #26061, "Remove store tracing API and all related plumbing"), and `store/tracekv/store.go` is absent from the tree at v0.54.x, v0.55.x and main. Renamed the section from "Gas and trace store wrappers" to "Gas store wrapper", dropped the `TraceKVStore` bullet and its dead link, updated the in-page table of contents anchor, and corrected the storage-stack diagram from "wrapped with gas/trace" to "wrapped with gas metering". Also bumped all 11 remaining GitHub refs on the page from `release/v0.54.x` to `release/v0.55.x` after verifying each path exists at the new ref and that the one line anchor (`store/iavl/store.go#L36`) still lands on the same line. Synced to `next/` with `scripts/sync-latest-to-next.js`, which preserves the destination front matter so `noindex` and `canonical` survived. +- Found this via a dead GitHub link, which is the durable lesson: the 404 was a symptom, and the actual defect was prose describing a removed feature. Bumping the ref would have kept the 404 and left the wrong paragraph in place. +- Left alone deliberately: `reference/architecture/adr-038-state-listening.mdx` still shows `tracekv` in code samples, which is correct because an ADR records what was proposed at the time and should not be rewritten. `modules/genutil/README.mdx:696` still lists a `flagTraceStore` CLI flag inside a copied upstream code block; the flag is gone from both v0.54.x and v0.55.x, but that file is generated content and hand-patching one line of it would drift it further from its source. + +### Version-pinned GitHub links: survey and process change + +Neither the freeze script nor `npx mint broken-links` touches or validates external links. `broken-links` checks internal page paths only, not heading anchors and not external URLs, so nothing in CI has ever looked at these. + +Current state of `latest/` for both products: + +| scope | category | links | anchored | +| ----- | -------- | ----- | -------- | +| sdk | version-tracking (`release/v0..x`) | 125 | 66 | +| sdk | moving (`main`, `master`) | 91 | 24 | +| sdk | pinned tag (64 of them `v0.47.0-rc1`) | 78 | 66 | +| sdk | pinned sha | 29 | 19 | +| cometbft | version-tracking (82 at `v0.38.x`) | 90 | 12 | +| cometbft | moving | 13 | 0 | +| cometbft | pinned sha | 6 | 5 | + +Measured what a blind version bump would do, old ref versus new ref. Paths mostly survive: 93 of 96 SDK, 77 of 82 CometBFT. Line anchors do not. Of 39 anchored SDK links, 29 still landed on the same line, 7 had moved, and 3 needed a human (line deleted, file deleted, ambiguous). CometBFT was worse at two versions of drift, with only 4 of 12 surviving. So roughly a quarter of anchored links would have kept working while pointing at unrelated code, which is worse than leaving them on the old ref. + +Two traps that produced false positives on the first pass, both now documented: paths must be URL-decoded before a tree lookup (`abci%2B%2B_methods.md` resolves fine), and a `/blob/` URL pointing at a directory is valid because GitHub redirects it to `/tree/`. + +Rename detection needs `git diff --find-renames`; matching on basename is useless, giving 23 candidates for `store.go`. Line mapping needs `git diff -U0` hunk offsets rather than content comparison, which cannot distinguish a moved line from a coincidentally identical one and produced six candidates for one `.proto` anchor. + +Process change recorded in both `CLAUDE.md` files: page-content fixes of this kind belong in `next/` **before** the freeze, because the promotion then carries them into `latest/` in one pass. Doing them afterwards means editing `latest/` and syncing every touched file back. The release sequence in the root `CLAUDE.md` gained a new step 2 for this, ahead of the freeze. + +Not done, and the reason: none of the 178 stale version-tracking links were bumped. That is a content change across roughly 110 files, the ambiguous anchors need human judgement, and doing it correctly wants tooling rather than a sed. Also outstanding is the dead `RELEASE_PROCESS.md` link at `modules/modules.mdx:44`, which existed at v0.53.x and was deleted upstream with no replacement, so it needs a decision on where to point. + +Open question, not yet decided: whether to build `scripts/versioning/check-github-refs.js` to implement the checks above, and whether it runs as a freeze step or on a CI schedule. Pinned refs need a drift check rather than a bump, comparing only the anchored region against the current release branch, since a pinned link never 404s but the prose around it can still go stale. The 24 SDK links that combine `main` with a line anchor have no correct maintenance strategy at all and should be repointed at either a pinned sha or the current release branch. diff --git a/work-log/v0.55-updates.md b/work-log/v0.55-updates.md new file mode 100644 index 000000000..1edbbc7a9 --- /dev/null +++ b/work-log/v0.55-updates.md @@ -0,0 +1,20 @@ +# v0.55-updates + +Doc updates for the 2026.1 (Ledger Security) release: Cosmos SDK 0.55 and CometBFT 0.40 deltas. Findings audited and verified against code at `cosmos-sdk@3d3b901ce5` and `cometbft@6ac238b`. + +## 2026-07-23 + +- Removed the `x/protocolpool` module docs (removed from the SDK in 0.55): deleted `sdk/next/modules/protocolpool/README`, dropped its `docs.json` nav entry, added a redirect to the distribution README, and removed the `modules.mdx` bullet. Rewrote the distribution README to drop the fully-removed external community pool apparatus (both `ExternalCommunityPoolKeeper` interface blocks, the "Using an External Community Pool" and usage-warning sections, and the false `FundCommunityPool` warning). Added a `protocolpool` deleted-store step to the upgrade guide. +- Removed `SIGN_MODE_TEXTUAL` (removed from the SDK in 0.55): cut the bullet from the encoding concept page and marked ADR-050 plus annex1 and annex2 as archived. Left ADR-020 and ADR-076 intact (still-valid ADRs that only mention the mode). +- Added the `SigVerifyCostMlDsa65` auth param (default 750) to the auth module params table and YAML example. +- Documented the Block-STM `app.toml` keys (`block-executor`, `block-stm-workers`, `block-stm-pre-estimate`) in the experimental Block-STM guide, including the automatic block-gas-meter disable. +- Added `key_rotation_fee` (field 7) to the reproduced staking `Params` proto block. +- Corrected the authz README pruning description: pruning runs in `BeginBlock`, not `EndBlock`, and is capped at 200 expired grants per block. +- Added a distribution README note on blocked withdraw addresses: automatic block-boundary withdrawals fall back to the owner then the community pool, while user messages return `ErrUnauthorized`. +- Fixed the stale otel telemetry env var in the telemetry guide (`OTEL_EXPERIMENTAL_CONFIG_FILE` -> `OTEL_CONFIG_FILE`). +- Added the `--consensus-key-algo` init flag note and an `app.toml` self-documentation pointer to the run-node guide. +- Added a note distinguishing the SDK in-process app mempool from CometBFT's `mempool.type = "app"` (`InsertTx`/`ReapTxs`) mechanism, with a cross-link. +- CometBFT: documented the `secp256k1eth` and `ml_dsa_65` consensus key types in the encoding spec Key Types section, and updated the `pub_key_types` enumerations in the genesis spec and ABCI requirements. +- CometBFT: regenerated the reproduced `config.toml` in the configuration reference from `cometbft init` at the release commit, restoring full field parity (20 fields had drifted out, including `event_bus_buffer_capacity` and `adaptive_sync`). +- CometBFT: added a validator equivocation-risk warning to the block-sync guide for `adaptive_sync`, worded as an operational double-signing risk so it does not contradict the existing safety-rules statement. +- Post-verification fixes: restored the neutral `moniker = "anonymous"` placeholder in the regenerated `config.toml` (the generator had captured a machine hostname), set the reproduced config version to `0.40.0` to match the release, trimmed an over-specified sentence in the block-sync equivocation warning, and generalized the distribution fallback note from "delegator" to "owner" (delegator for rewards, validator for commission).