Degov original#4
Merged
Merged
Conversation
* fix(indexer): redact rpc urls in logs * fix(indexer): harden rpc url redaction edge cases
…ory (ringecosystem#699) Bumps the npm_and_yarn group with 1 update in the / directory: [dompurify](https://github.com/cure53/DOMPurify). Updates `dompurify` from 3.3.2 to 3.4.0 - [Release notes](https://github.com/cure53/DOMPurify/releases) - [Commits](cure53/DOMPurify@3.3.2...3.4.0) --- updated-dependencies: - dependency-name: dompurify dependency-version: 3.4.0 dependency-type: direct:production dependency-group: npm_and_yarn ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* fix(web): move auth token to http-only cookie * fix(web): hydrate cookie auth state
* Strengthen SIWE context validation * Reject invalid SIWE time bounds * Bind SIWE context to request origin
* Add SIWE login abuse controls * Address SIWE abuse control review feedback * Address SIWE review follow-ups
* Add cached ENS lookup fallback * Address ENS lookup review feedback * docs(web): document ENS environment variables * feat(web): batch ENS record lookups
* fix(web): route ENS lookups through API * fix(web): address ENS review feedback
* fix(web): prefer ENS RPC before DeGov API * fix(web): batch DeGov ENS fallback
* chore(deps): consolidate dependabot upgrades * chore(deps): include next dependabot upgrades
Bumps the npm_and_yarn group with 1 update in the / directory: [postcss](https://github.com/postcss/postcss). Updates `postcss` from 8.5.6 to 8.5.10 - [Release notes](https://github.com/postcss/postcss/releases) - [Changelog](https://github.com/postcss/postcss/blob/main/CHANGELOG.md) - [Commits](postcss/postcss@8.5.6...8.5.10) --- updated-dependencies: - dependency-name: postcss dependency-version: 8.5.10 dependency-type: direct:development dependency-group: npm_and_yarn ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…dates (ringecosystem#720) Bumps the npm_and_yarn group with 4 updates in the / directory: [@protobufjs/utf8](https://github.com/dcodeIO/protobuf.js), [axios](https://github.com/axios/axios), [fast-uri](https://github.com/fastify/fast-uri) and [hono](https://github.com/honojs/hono). Updates `@protobufjs/utf8` from 1.1.0 to 1.1.1 - [Release notes](https://github.com/dcodeIO/protobuf.js/releases) - [Changelog](https://github.com/protobufjs/protobuf.js/blob/master/CHANGELOG.md) - [Commits](protobufjs/protobuf.js@protobufjs-cli-v1.1.0...fetch-v1.1.1) Updates `axios` from 1.15.0 to 1.16.1 - [Release notes](https://github.com/axios/axios/releases) - [Changelog](https://github.com/axios/axios/blob/v1.x/CHANGELOG.md) - [Commits](axios/axios@v1.15.0...v1.16.1) Updates `fast-uri` from 3.1.0 to 3.1.2 - [Release notes](https://github.com/fastify/fast-uri/releases) - [Commits](fastify/fast-uri@v3.1.0...v3.1.2) Updates `hono` from 4.12.12 to 4.12.19 - [Release notes](https://github.com/honojs/hono/releases) - [Commits](honojs/hono@v4.12.12...v4.12.19) --- updated-dependencies: - dependency-name: "@protobufjs/utf8" dependency-version: 1.1.1 dependency-type: indirect dependency-group: npm_and_yarn - dependency-name: axios dependency-version: 1.16.1 dependency-type: indirect dependency-group: npm_and_yarn - dependency-name: fast-uri dependency-version: 3.1.2 dependency-type: indirect dependency-group: npm_and_yarn - dependency-name: hono dependency-version: 4.12.19 dependency-type: indirect dependency-group: npm_and_yarn ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…dates (ringecosystem#721) Bumps the npm_and_yarn group with 2 updates in the / directory: [postcss](https://github.com/postcss/postcss) and [ws](https://github.com/websockets/ws). Updates `postcss` from 8.5.11 to 8.5.12 - [Release notes](https://github.com/postcss/postcss/releases) - [Changelog](https://github.com/postcss/postcss/blob/main/CHANGELOG.md) - [Commits](postcss/postcss@8.5.11...8.5.12) Updates `ws` from 7.5.10 to 8.20.1 - [Release notes](https://github.com/websockets/ws/releases) - [Commits](websockets/ws@7.5.10...8.20.1) --- updated-dependencies: - dependency-name: postcss dependency-version: 8.5.12 dependency-type: direct:development dependency-group: npm_and_yarn - dependency-name: ws dependency-version: 8.20.1 dependency-type: indirect dependency-group: npm_and_yarn ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* feat(indexer): add onchain token power mode * fix(indexer): complete onchain power audit coverage * fix(indexer): report audit power source * fix(indexer): harden onchain reconcile consistency * fix(indexer): correct onchain vote timepoints * fix(indexer): sync onchain transfer delegation power * fix(indexer): regenerate base schema migration * fix(indexer): support compound current vote reads * fix(indexer): fallback on legacy historical vote reverts * feat(indexer): add onchain refresh task worker * fix(indexer): preserve delegation relations with onchain refresh * fix(indexer): defer onchain refresh during historical sync * fix(indexer): allow database isolation override * fix(indexer): keep default database isolation * fix(indexer): preserve locked onchain refresh tasks * fix(indexer): defer historical onchain task seeding * fix(indexer): decouple onchain seed account loading * fix(indexer): retry postgres serialization failures * fix(indexer): restart processor after runtime failures * chore(indexer): add onchain worker compose service * fix(indexer): retry serializable database transactions * fix(indexer): disable hot block tracking by default * fix(indexer): serialize indexer write transactions * fix(indexer): cast onchain task timestamps * fix(indexer): reclaim stale onchain tasks * perf(indexer): lookup reconcile tasks by id * fix(indexer): normalize onchain refresh scope addresses * fix(indexer): allow gateway override * Revert "fix(indexer): allow gateway override" This reverts commit 459335e. * fix(indexer): fallback when archive block is unavailable * fix(indexer): bound archive sync ranges * fix(indexer): seed reconcile tasks during sync lag * fix(indexer): bulk seed reconcile refresh tasks * fix(indexer): requeue stale reconcile refresh tasks * fix(indexer): process reconcile backlog before seeding * fix(indexer): bound reconcile seed scanning * fix(indexer): fallback multicall vote refresh * chore(indexer): document onchain refresh compose env * docs(indexer): explain onchain refresh env
## Summary chore(indexer): remove sqd indexer shell ## Changes - Remove the SQD/Subsquid processor source, handlers, migrations, scripts, tests, Docker image, service startup files, and accuracy audit workflow. - Keep @degov/indexer as a Datalens placeholder package with reference ABI and schema artifacts only. - Remove indexer services from compose and release image publishing, while preserving placeholder just recipes used by repository verification gates. ## Verification - pnpm install --lockfile-only - pnpm install --frozen-lockfile --ignore-scripts - just indexer test-unit - pnpm --filter @degov/indexer build - pnpm --filter @degov/indexer test - docker compose config - pnpm --filter @degov/web build metadata.yaml > schema: conductor/commit/1 repository: degov repository_url: https://github.com/ringecosystem/degov issue: https://plane.kahub.in/endless/browse/HBX-242/ attempt: run-hbx-242-review-repair-1780337068851020405-attempt-1 authority: https://plane.kahub.in/endless/browse/HBX-242/ summary: 'fix(config): restore indexer start block' related: - checkpoint:hbx-242-restore-indexer-start-block
…er. (ringecosystem#730) ## Summary Define Rust technical conventions for the Datalens-native DeGov indexer. ## Changes - Add an authoritative conventions spec covering the Rust stack, Datalens integration boundary, logging, typed errors, ChainTool, database semantics, and implementation-plan requirements. - Route the new conventions document from the docs index. - Add an indexer package convention check that scans Rust workspace files for library tracing macro usage, raw anyhow library APIs, and ethers dependencies while allowing tracing output setup in binary entrypoints. ## Verification - pnpm --filter @degov/indexer test - pnpm --filter @degov/indexer build - just web lint metadata.yaml > schema: conductor/commit/1 repository: degov repository_url: https://github.com/ringecosystem/degov issue: https://plane.kahub.in/endless/browse/HBX-270/ attempt: run-hbx-270-review-repair-1780363798957499108-attempt-1 authority: https://plane.kahub.in/endless/browse/HBX-270/ summary: Review repair for HBX-270. related: - checkpoint:hbx-270-review-repair-instrument-attribute
…tem#729) ## Summary docs(spec): define datalens indexer architecture contract ## Changes - Added docs/spec/datalens-indexer-architecture-contract.md to define the Rust-first external application indexer architecture, Datalens SDK boundary, service endpoint configuration, and secret handling. - Documented that old SQD handlers are reference behavior only, not the new runtime structure. - Covered transaction, checkpoint, retry, idempotency, projection/reconcile, TypeScript compatibility, and observability correctness requirements. ## Verification - Ran whitespace checks for the changed docs files. - Ran just web lint after installing dependencies from the existing lockfile; eslint exited 0 with one unrelated import-order warning. - prettier is not installed in this workspace, so the formatter check is unavailable. metadata.yaml > schema: conductor/commit/1 repository: degov repository_url: https://github.com/ringecosystem/degov issue: https://plane.kahub.in/endless/browse/HBX-245/ attempt: run-hbx-245-review-repair-1780365684961033831-attempt-1 authority: https://plane.kahub.in/endless/browse/HBX-245/ summary: 'docs(spec): avoid local secret path in datalens contract' related: - checkpoint:hbx-245-review-repair-secret-reference
…#736) ## Summary Define Datalens DAO compatibility and preflight policy ## Changes - Added compatibility matrix and fail-fast policy for supported, degraded, and unsupported DAO/governor/token shapes. - Added migration preflight runbook requiring compatibility validation before staging or production migration. - Added compatibility preflight validator and tests for ERC20/ERC721 mismatch, governor method reverts, and degraded vote-read fallbacks. ## Verification - pnpm --filter @degov/indexer test - just web lint metadata.yaml > schema: conductor/commit/1 repository: degov repository_url: https://github.com/ringecosystem/degov issue: https://plane.kahub.in/endless/browse/HBX-269/ attempt: run-hbx-269-review-repair-1780367920484887989-attempt-1 authority: https://plane.kahub.in/endless/browse/HBX-269/ summary: Add isolated COUNTING_MODE degradation regression related: - checkpoint:hbx-269-counting-mode-isolated-regression
…resh initialization path. (ringecosystem#737) ## Summary Define canonical Postgres schema ownership for the Datalens indexer fresh initialization path. ## Changes - Add packages/indexer/schema/postgres.sql as the canonical fresh Postgres schema for checkpoint, reconcile, proposal, vote, delegation, contributor, metric, and timelock tables. - Document reset/recreate database requirements and GraphQL compatibility ownership in the indexer README and docs router. - Add schema ownership and Postgres initialization smoke scripts to package verification commands. ## Verification - pnpm --filter @degov/indexer test - DEGOV_INDEXER_DATABASE_URL=postgresql://postgres:postgres@localhost:32768/indexer pnpm --filter @degov/indexer run smoke:postgres-init - just web lint - git diff --check metadata.yaml > schema: conductor/commit/1 repository: degov repository_url: https://github.com/ringecosystem/degov issue: https://plane.kahub.in/endless/browse/HBX-265/ attempt: run-hbx-265-review-repair-1780368943808547029-attempt-1 authority: https://plane.kahub.in/endless/browse/HBX-265/ summary: Merge main compatibility preflight changes into the HBX-265 Postgres schema branch. related: - checkpoint:hbx-265-base-sync-main-20260602
## Summary feat(indexer): add datalens client foundation ## Changes - Introduce the initial Rust indexer crate with Datalens SDK, figment config loading, typed thiserror errors, and a redacted secret wrapper. - Add a DeGov-owned DatalensNativeReader trait plus SDK adapter and smoke readiness helper for mockable tests. - Add a smoke-datalens CLI entrypoint and wire package build/test scripts to Cargo while preserving existing schema/convention checks. - Add non-secret Datalens environment placeholders using the service base endpoint, with an empty token slot for secret-backed configuration. ## Verification - just indexer build - just indexer test - cargo fmt --manifest-path packages/indexer/Cargo.toml --check - git diff --cached --check - just web lint metadata.yaml > schema: conductor/commit/1 repository: degov repository_url: https://github.com/ringecosystem/degov issue: https://plane.kahub.in/endless/browse/HBX-246/ attempt: run-hbx-246-review-repair-1780373166231987272-attempt-1 authority: https://plane.kahub.in/endless/browse/HBX-246/ summary: 'fix(indexer): remove unused datalens endpoint default' related: - checkpoint:hbx-246-review-repair-remove-unused-endpoint-default
…em#741) * feat(indexer): normalize raw evm logs * ci(indexer): disable unused pnpm cache * fix(indexer): include transaction hash in evm log ids * test(indexer): reject conflicting evm log duplicates * ci(indexer): disable node package manager cache * ci(indexer): disable package manager cache
…ystem#742) * feat(indexer): add datalens evm log planner * fix(indexer): include all dao log topic filters * fix(indexer): reject zero datalens log chunk limit
Implements Datalens ABI event decoding for governor, token, and timelock logs. Adds ERC20/ERC721 Transfer shape validation and case-insensitive token standard parsing for DeGov registry-style configuration.
Adds ChainTool onchain read planning for DeGov Datalens indexing, including execution-key dedupe separated from semantic metadata, typed read result payloads, bounded multicall grouping, and safeguards for invalid batch config.
Adds the Datalens-native onchain power reconcile planner, including compact affected-account dedupe, ChainTool current-power read scheduling, compatible refresh metadata, freshness metrics, and tests for no log-derived final power.
Fixes HBX-423.
Fixes HBX-422.
* perf(indexer): batch onchain refresh RPC reads * fix(indexer): avoid per-read fallback on multicall transport errors * fix(indexer): tighten multicall worker fallback
* fix(indexer): chunk onchain refresh DB apply * fix(indexer): validate onchain refresh apply chunks * test(indexer): cover bounded onchain refresh tick runs * fix(indexer): allow large onchain refresh tick runs
* fix(indexer): correct delegate edge power semantics * fix(indexer): limit delegate balance refresh reads * test(indexer): update delegate refresh read expectations * test(indexer): assert scoped live power overlays * test(indexer): preserve balance refresh conflict semantics * test(indexer): update optimized reconcile read counts
…system#849) * fix(indexer): reconcile delegate relation power from balance * test(indexer): cover delegate relation count refresh
* fix(indexer): align ENS proposal parity * fix(indexer): align warmup selectors with query plans * test(indexer): cover source-specific warmup effectiveness * test(indexer): update postgres runtime source queries * test(indexer): update provisional worker source queries
* perf(indexer): batch ENS token delegate writes * perf(indexer): batch token checkpoint writes * perf(indexer): batch dense delegate mapping writes
Adds a runtime covering partial index for delegate rolling metadata preload and keeps the local combined GraphQL/sync runner from blocking sync on inline onchain refresh by starting the worker separately.
Allows provider-fill chunks below the existing high-query, dense-row, and slow-local safety gates to count toward stable adaptive growth so ENS does not remain stuck at tiny chunk sizes after transient slow ranges.
Reset onchain refresh attempts when requeued by pending_after_lock and allow pending tasks at max attempts to be claimed again.
* fix(indexer): add proposal title refresh command * fix(indexer): satisfy title refresh runtime convention * fix(indexer): bound proposal title refresh updates
Fix the delegate_mapping power-only bulk update SQL builder so VALUES emits one three-column row per update. Add a Postgres runtime regression that persists two delegate mappings before applying a transfer-only batch that exercises the power-only flush path.
Signed-off-by: Sigride <9304947+Sigri44@users.noreply.github.com>
JSON and YAML Validation ResultsJSON Validation Results
Violations: [
{
"file": "degov.yml",
"errors": [
{
"path": "/indexer",
"message": "must NOT have additional properties"
},
{
"path": "/indexer",
"message": "must NOT have additional properties"
}
]
}
] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.