Skip to content

AGENTS.md

Latest commit

 

History

History
386 lines (337 loc) · 20.9 KB

AGENTS.md

File metadata and controls

386 lines (337 loc) · 20.9 KB
Raw

AGENTS.md — mega2

Guidance for AI coding agents working in this repository. Keep changes minimal, follow the patterns already in the codebase, and verify with the commands below before submitting.

Normative task-card rules are defined in docs/plan/plan-template.md, currently v2.5 (effective 2026-10-15): ER-* execution gates, GC-* engineering constraints, and G-01–G-13 granularity/release boundaries. Follow its incremental migration policy for existing plans; this file is a synchronized entry point, not a separate waiver. G-13 permits only a predeclared, reviewed aggregate of 2–3 independent code defects actually blocking strict gates in one unpublished repair frontier; it preserves each card's behavior axis and full final C/D.

Project Overview

  • Name: mega2 (single Cargo package: lib mega2_core + binaries, see Cargo.toml).
  • Edition: Rust 2024.
  • Purpose: Mono‑repo / Git hosting + service engine. Ports and extends several subsystems originally from the Mega project (notably callisto entities and jupiter storage/migration).
  • Entry point: src/main.rs → cli::parse(None).
  • Config: TOML loaded from config/config.toml (override via --config flag or MEGA_CONFIG env var). Loader lives in src/config/loader.rs.

Tech Stack

  • Language: Rust 2024 (stable toolchain).
  • CLI: clap v4 (derive + builder), subcommands registered in src/commands/mod.rs (builtin() / builtin_exec()).
  • Async runtime: tokio (full features).
  • HTTP / API: axum 0.8 + tower-http, OpenAPI via utoipa + Swagger UI.
  • Storage / DB: sea-orm 1.1 (Postgres + SQLite, runtime-tokio-rustls) and sea-orm-migration. Entities are in src/callisto/.
  • Cache / queue: redis (with connection-manager).
  • Auth / policy: cedar-policy (schema in src/contract/policy/mega.cedarschema, policies in src/contract/policy/mega_policies.cedar).
  • Crypto / TLS: rustls, ring, openssl, ed25519-dalek, rsa, secp256k1, pgp. Vault‑style PKI/secret engine via the libvault crate (crates.io 0.3.0, features storage_pg + crypto_adaptor_openssl) and the mega2 integration layer (src/contract/vault/). The RustyVault sources used to be vendored under src/vault/; that module was removed on 2026-08-21 (docs/plan/plan-20260820.md), so import library types from libvault::*.
  • Email: lettre (rustls + tokio).
  • Object storage: inlined src/orbit_api/ (traits/config) and src/orbit/ (object_store backends). Built via crate::orbit::factory::ObjectStorageFactory from src/jupiter/storage/object_storage.rs::build_object_storage.
  • Allocator: jemalloc on non‑Windows, mimalloc on Windows (configured in src/main.rs).
  • Logging: tracing + tracing-subscriber + tracing-appender (hourly rolling file under mega_cache()/logs, or stdout when log.print_std = true).

Commands

Run these from the repo root (the agent's shell already starts there).

Task Command
Build (release-ish) cargo build
Build incl. tests cargo build --tests
Run all tests cargo test
Run one test cargo test --test <name> or cargo test <substring> -- --nocapture
Format cargo fmt --all
Lint cargo clippy --all-targets -- -D warnings (when used)
Run the binary cargo run -p mega2 -- --config config/config.toml <subcommand>
HTTP service example cargo run -p mega2 -- --config config/config.toml service http --host 0.0.0.0 -p 9000

Invariants the build must hold (verified in prior sessions):

  • cargo build MUST produce 0 errors and 0 warnings.
  • cargo build --tests MUST produce 0 errors and 0 warnings.
  • Never silence warnings by adding broad #[allow(...)] on items you just touched without a reason — the crate‑level #![allow(dead_code)] in src/lib.rs is intentional (large pub API surface ported from Mega); do not narrow or remove it without a plan to clean the dead items.

Required Checks Before Submitting Code Changes

Any change to the codebase MUST satisfy all three of the following gates. These are not optional — do not submit a change until each one is green. Use the exact commands below (do not substitute simpler variants such as cargo fmt --all --check or cargo clippy -- -D warnings):

  1. Formatting (nightly, check‑only):

    cargo +nightly fmt --all --check

    Must report no diff. If it does, run cargo +nightly fmt --all to apply formatting and re‑run the check until clean. The nightly toolchain is required because rustfmt.toml may enable unstable options.

  2. Lints (all targets, all features, warnings denied):

    cargo clippy --all-targets --all-features -- -D warnings

    Must exit with 0 warnings, 0 errors. Do not bypass a clippy lint with a blanket #[allow(...)]; prefer fixing the underlying code. When an allow is genuinely required (e.g. a deliberate API name that trips clippy::wrong_self_convention), scope it to the smallest possible item and add a brief rationale.

  3. Tests (with project test env loaded):

    source .env.test && cargo test --all

    Must finish with all tests passing (0 failures, 0 errored). The .env.test file provides DB / cache / service endpoints the test suite expects; do not skip sourcing it. Do not weaken or #[ignore] tests to make this gate pass — fix the root cause.

If .env.test is missing in your environment, stop and ask before submitting; do not silently fall back to running cargo test --all without it.

Project Layout

Cargo.toml                # package `mega2` (lib `mega2_core` + [[bin]])
config/config.toml        # default runtime config (TOML)
src/
├── main.rs               # `mega2` binary entry (allocator + CLI dispatch)
├── lib.rs                # library root; declares top-level modules
├── cli.rs                # clap parsing, log init, ctrlc handler
├── orbit_api/            # object-storage contract (traits, config, errors)
├── orbit/                # object_store backends (adapter, factory)
├── bin/                  # auxiliary binaries (e.g. migrate_local_to_s3)
├── commands/             # subcommand registry (builtin / builtin_exec)
├── common/               # error types (MegaError/MegaResult), utils
├── config/               # config loader, profiles, SecretRef, hot reload
├── api/                  # axum HTTP API surface
├── api_model/            # request/response DTOs (utoipa schemas)
├── server/               # HTTP/SSH/etc. server bootstrap
├── callisto/             # sea-orm entity models (one file per table)
├── jupiter/              # storage, service, migration, redis, utils
│   ├── storage/          # *Storage structs (BaseStorage + per-domain)
│   ├── migration/        # sea-orm-migration migrators
│   ├── service/
│   ├── redis/
│   └── tests.rs          # `pub mod tests` (cfg(test)) — shared test helpers
├── notification/         # email notifications: dispatcher, triggers, storage
├── email/                # Mailer trait + impls (incl. NoopMailer)
├── contract/
│   ├── policy/           # cedar authz: mega.cedarschema, mega_policies.cedar
│   └── vault/            # PKI / KV / secret engine integration layer over the
│       │                 # `libvault` crate (no vendored module since 2026-08-21)
│       └── integration/
│           ├── jupiter_backend.rs
│           └── vault_core.rs # VaultCore, VaultCoreInterface
├── ceres/  context/
tests/                    # process-level integration tests (integration_*.rs)
target/                   # build artifacts (gitignored)

pub use crate::callisto::*; is re‑exported from lib.rs; importing callisto entities elsewhere should use crate::callisto::<table> paths. Object storage public types are available from crate::orbit_api::*; the concrete backend is built through crate::jupiter::storage::object_storage::build_object_storage (which calls crate::orbit::factory::ObjectStorageFactory::build).

Code Conventions

  • Formatting: match cargo +nightly fmt --all output (the same formatter used by the required cargo +nightly fmt --all --check gate). Don't hand‑format around it.
  • Imports: group by std → external crates → crate:: (matches the existing files). Avoid wildcard use crate::*; in library code; wildcards are fine inside mod tests.
  • Errors: use crate::common::errors::{MegaError, MegaResult} for application code paths that already use them; anyhow::Result is used in lower‑level utilities and thiserror for new typed errors. Don't mix the three within the same module.
  • Async: functions returning Result should be async fn -> Result<T, E> using tokio runtime. Don't add block_on inside async contexts.
  • Logging: use tracing::{info, warn, error, debug, trace} macros, not println!. Structured fields preferred (e.g. info!(path = %p, "loaded")).
  • DB access: go through the *Storage types in src/jupiter/storage/ rather than calling sea_orm directly from API/handler code.
  • Comments: sparse, English. Match the surrounding density — do not add comments to files that don't already use them.
  • Files / modules: snake_case filenames, one module per file, mod.rs only for directory module roots.

Documentation Conventions

  • Bilingual guide docs: the user-facing guides under docs/ (quick-start, user-guide, configuration, deployment, architecture, contributing) ship in two languages. English is the default file (<name>.md); Chinese lives in the <name>.zh.md sibling (same convention as README.md / README.zh.md). Write the Chinese version first, then translate; both versions must keep identical structure, carry the language switcher line at the top, and cross-link within the same language (zh → .zh.md, en → .md). When you change one language version of a guide, update the other in the same change.
  • Link, don't copy: facts that have an authoritative home (config/config.toml, docs/user-guide.md for repository and push behavior, docs/manual/monorepo-init.md for initialization layout, docs/deploy-trunk.md for trunk operations, and docs/refactoring/*.md for subsystem contracts) are linked, not duplicated. Every relative link in a doc must resolve to a file in the current checkout.
  • Plan docs under docs/plan/ follow docs/plan/README.md and the plan templates; they are Chinese-first and unchanged by the bilingual convention above.

Common Pitfalls (please read before editing tests or vault)

  1. sea_orm imports inside tests. There is no crate::jupiter::sea_orm re‑export. Import traits from the top‑level crate: 
    use sea_orm::{ActiveModelTrait, ColumnTrait, EntityTrait, QueryFilter, Set};
    Forgetting these traits produces misleading errors such as email_jobs::Entity is not an iterator.
  2. VaultCore path. src/contract/vault/integration/mod.rs does not re‑export VaultCore. Import it directly from its submodule: 
    use crate::contract::vault::integration::vault_core::{VaultCore, VaultCoreInterface};
  3. Glob re‑exports / the vault name. There is no top-level mod vault; any more — the vendored RustyVault module was removed on 2026-08-21 and the library comes from the libvault crate. crate::vault::* is not a valid path; rg 'crate::vault' src bin should stay at zero hits. Three different things still share the name, so keep imports explicit: crate::callisto::vault is the SeaORM entity, crate::contract::vault is the product integration layer, and libvault::* is the library itself.
  4. Crate‑level dead_code allow. #![allow(dead_code)] in main.rs is intentional. If you add a new pub API item, you don't need to add per‑item allows; if you remove the crate‑level allow, expect ~70 warnings.
  5. Test DB helpers. Tests requiring a database use crate::jupiter::tests::test_db_connection(<TempDir path>) followed by crate::jupiter::migration::apply_migrations(&db, true).await. Reuse these helpers instead of constructing connections by hand. Each call creates a mega2_test_<pid>_<n> schema that is dropped with the connection (or when the test thread ends, if something still holds it); test_db_config returns (DbConfig, TestSchemaGuard) and the guard must stay bound for the whole test (_schema, never _), or the schema is dropped under the connection.
  6. Allocator cfg. Don't touch the #[global_allocator] blocks in main.rs unless intentionally changing allocators on a platform.
  7. unwrap() in non‑test code. Avoid introducing new unwrap/expect on fallible operations; return MegaResult/anyhow::Result instead. Existing call sites in vault/pki.rs test helpers are OK because they are test‑only.

Adding a New Subcommand

  1. Implement the command module under src/commands/<name>.rs.
  2. Register it in src/commands/mod.rs via builtin() (clap Command) and wire its executor in builtin_exec() so cli::exec_subcommand finds it.
  3. The executor signature is fn(config: Config, args: &ArgMatches) -> MegaResult.
  4. Add unit tests next to the command and, where useful, a CLI parsing test mirroring the existing ones in src/cli.rs::tests.

Adding a New DB Entity / Migration

  1. Generate or hand‑write the entity file under src/callisto/<table>.rs and add it to src/callisto/mod.rs.
  2. Add a migrator under src/jupiter/migration/ and register it in that module's migrator list.
  3. If a new domain storage is needed, add <domain>_storage.rs under src/jupiter/storage/ and re‑export from storage/mod.rs.
  4. Cover with a #[cfg(test)] mod tests that uses test_db_connection + apply_migrations as shown in notification/dispatcher.rs::tests.

Website-next IT stack reload

When you change ../megaui sources that affect apps/web, rebuild and restart the compose website-next service after the change (do not wait for the user to ask):

./scripts/reload-website-next.sh

The script debounces rapid edits (~3s) and runs docker compose build + recreate in the background. Logs: ${TMPDIR:-/tmp}/mega2-reload-website-next/build.log.

Project hooks in .cursor/hooks.json trigger the same script on megaui file edits and again on agent stop when a reload was requested.

Task card release (plan-20260905 and other docs/plan/ cards)

By default, when an independent plan task card or a release point passes its A/B local acceptance gates and the required review says PASS, proceed through C release close-out without waiting for another request. Do not mark any card Lifecycle=done before its C coverage and all applicable D remote gates are resolved. VCS is Libra (no git).

  1. Check version-face parity. For a card with an actual version increment, bump Cargo.toml version by its Version increment (default patch +1) and let the toolchain refresh Cargo.lock for mega2.
  2. On the final tree (after bump when applicable), run all three gates under Required Checks Before Submitting Code Changes, plus cargo build and cargo build --tests with 0 errors and 0 warnings.
  3. libra add <related paths> and libra commit -s -m "<scope>: <summary>" for that card only; verify the sign-off as required by ER-07.
  4. libra push origin main and verify the remote branch ref. Never --force. If the branch has diverged from origin, stop and report.
  5. For every actual version bump, create and push the matching annotated v<version> tag, then create its GitHub Release from a manually written release note. These actions are part of C.
  6. Determine D from the live workflow event, ref, path filters, and job conditions. Record the actual remote result, or the named ER-04 deferral; then set Acceptance=complete and Lifecycle=done. While an applicable D result is outstanding, use Acceptance=remote-pending and do not report the card complete. Start the next release card only after this card's C actions finish; release remains serial.

Family, plan-release, repair-aggregate children, and no-release cards inherit C/D coverage from the carrier named in the card. See docs/plan/plan-template.md ER-04, ER-07, ER-08, G-12, and G-13 for the full gates, deferral rule, and release order.

For the explicitly declared G-12 plan-wide release group in plan-20260920.md, each child completes its A/B gates, required review, and precise local Libra commit. Until the group's unique final release point, no child bumps the version, pushes the branch or tag, or creates a GitHub Release. At that point, check version parity, bump the patch version exactly once, run the full C checks on the final combined tree after the bump, push the branch and matching annotated tag, create the manually written GitHub Release, and resolve the applicable D gates. Children inherit that final point's C/D coverage and become done/complete only after the coverage is obtained. The unique final tag's Docker D must actually succeed; a named ER-04 D deferral does not complete this plan.

For an eligible G-13 repair aggregate, register 2–3 independent implementation repairs, their actual blocking logs, source cohorts/unpublished baseline, sole publisher, no-push window, recovery/exit rules, and one release point in REL-* before further implementation. Actual, mutually isolated Claude and Codex plan reviews must both explicitly pass its applicability. Each child uses repair aggregate child of REL-<n>, version/R N/A, and names the point for C/D. Establish each code cause and repair, pass independent A/B, fmt/clippy and actual ER-05 review, and commit precisely locally with ER-07 signature checks and sign-off. No earlier branch/tag push, Release, or interleaved release slice is allowed. The sole repair aggregate release point of REL-<n> reviews before its actual patch bump and executes every C step above on the combined final tree; its tag's Docker D must actually succeed before any child is done/complete. Unknown causes, non-reproduction alone, or environment load without a code defect require ER-03 reassessment/new plan review or a return to G-07 independent slices. This is not a G-08 family or an EX waiver, and does not replace the G-12 user directive. Published changes recover forward only; existing ER-09/ER-10 budgets, signature, no-force, and test/ignore boundaries remain unchanged.

A terminal evidence-only release card may set Version increment=N/A and Release write set=N/A only under ER-08: it adds no product behavior and cannot replace an independently released code card's, family release point's, G-12 plan-wide group's, or G-13 repair aggregate's versioned release. It skips the version edit, tag, and GitHub Release while retaining review, the three gates, builds, precise commit, push, and actual D-gate mapping. In the G-12 group above, an optional terminal evidence-only closeout may push evidence, plan-status, and index docs only after the unique patch release point's Docker D succeeds, without another version bump, tag, or GitHub Release.

Boundaries

  • Do not commit secrets, real tokens, or production config.toml values.
  • Do not add new dependencies to Cargo.toml without confirming they pull their weight (compile time / binary size / license). Prefer reusing what's already vendored (e.g. reqwest, rustls, tokio).
  • Do not rewrite working modules to "modernize" them; keep diffs focused on the requested change.
  • Do not disable or weaken tests (#[ignore], --skip, deleted asserts) to make a build pass. Fix the root cause or ask.
  • Do run cargo build and cargo build --tests before submitting any change that touches src/.

Verification Checklist (before submit)

Required gates (see Required Checks Before Submitting Code Changes):

  • cargo +nightly fmt --all --check → no diff.
  • cargo clippy --all-targets --all-features -- -D warnings → 0 warnings, 0 errors.
  • source .env.test && cargo test --all → all tests pass.

Additional sanity checks:

  • cargo build → 0 errors, 0 warnings.
  • cargo build --tests → 0 errors, 0 warnings.
  • No stray debug files (.warnings.log, .output.txt, ad‑hoc scripts) left in the repo root.
  • No new top‑level #[allow(...)] other than what already exists.