Skip to content

feat: fctl v4 isolated implementation#153

Draft
flemzord wants to merge 208 commits into
mainfrom
feat/v4
Draft

feat: fctl v4 isolated implementation#153
flemzord wants to merge 208 commits into
mainfrom
feat/v4

Conversation

@flemzord
Copy link
Copy Markdown
Member

@flemzord flemzord commented May 14, 2026

Draft PR: fctl v4 isolated implementation

Scope

This PR intentionally introduces the fctl v4 rewrite in an isolated top-level v4/ directory. It does not cut over the repository root yet. The goal is to make the v4 architecture, command mapping, compatibility aliases, tests, and migration documentation reviewable before the disruptive root replacement happens.

Important Review Note

Review this PR as the isolated v4 implementation and migration plan, not as the final cutover. The current v3 implementation remains available at the repository root so reviewers can compare old and new behavior side by side.

The follow-up PR should be a dedicated cutover PR that removes/moves the root v3 implementation and promotes v4/ to the repository root once this draft is validated.

What This PR Includes

  • v4 CLI module under v4/ with Cobra kept as a thin parser/router.
  • Context-first runtime model for local, self-hosted, Cloud, and Cloud-stack targets.
  • CLI authentication commands under session login/status/token/logout; auth is reserved for Auth service resources.
  • Auth decoupled from Formance Cloud membership for stack commands.
  • /versions-based API version resolution with compatibility manifests.
  • Product command families for Ledger, Payments, Wallets, Flows, Reconciliation, Auth service, Webhooks, Cloud, and Cloud stacks.
  • Deprecated v3 migration aliases where cheap and useful, with warnings on stderr.
  • v3 command inventory under v4/testdata/v3-command-inventory.json.
  • Migration documentation, command reference, alias policy, testing strategy, cutover plan, and implementation audit.
  • v4 execution todos moved under v4/todos/ so the root stays focused on the current v3 implementation until cutover.

Explicitly Not In This PR

  • No replacement of the repository root with v4 yet.
  • No deletion or move of the v3 implementation yet.
  • No final packaging/goreleaser/CI cutover yet.
  • session login cloud and session login oidc are visible but explicitly deferred until the device/browser flow contracts are defined.
  • Root login and auth login/status/token/logout are intentionally not kept as aliases, to avoid mixing CLI session state with the Auth service.
  • cloud personal-tokens create is documented as blocked by the current Cloud stack/token exchange model.
  • --telemetry and --quiet are intentionally not exposed as no-op flags.
  • A reusable OpenAPI mock server is documented as future work; current tests use targeted httptest servers.

Follow-Up Cutover PR

After this PR is reviewed, create a separate cutover PR that:

  • replaces root CLI implementation with the content of v4/;
  • adapts root go.mod, local replace directives, CI, packaging, and release config;
  • preserves required generated/internal clients;
  • runs root-level go test ./...;
  • removes the old v3 implementation in a dedicated commit.

Verification

Latest local checks run from this branch:

cd v4 && go test ./cmd -run 'TestRootHelp|TestDeferredCloudAndOIDCLoginCommandsReturnActionableErrors|TestSessionLoginTokenStoresCredentialAndUpdatesContext|TestSessionLoginClientCredentialsStoresSecret|TestSessionLoginNoneRequiresConfirmForCloudContext|TestSessionTokenPrintsStoredToken|TestSessionLogoutClearsCredential'
cd v4 && go test ./...
cd v4 && go run . auth --help
cd v4 && go run . session --help
cd v4 && go run . auth login
cd v4 && go run . login
cd v4 && go run . session login cloud --cloud-url https://cloud.example
git diff --check

Previous coverage check from go test ./... -cover:

  • cmd: 75.9%
  • internal/credentials: 83.0%
  • internal/capabilities: 75.4%
  • internal/auth: 72.9%
  • internal/runtime: 72.0%
  • internal/commands/target: 76.2%

Known lower package-level coverage remains documented in the audit because several product behaviors are currently covered through CLI integration tests in v4/cmd/root_test.go.

Key Docs

  • plan.md
  • docs/cli-v4/migration-v3-v4.md
  • docs/cli-v4/command-reference.md
  • docs/cli-v4/compatibility-aliases.md
  • docs/cli-v4/testing-strategy.md
  • docs/cli-v4/cutover-plan.md
  • docs/cli-v4/implementation-audit.md

@flemzord flemzord changed the title Draft: fctl v4 isolated implementation feat: fctl v4 isolated implementation May 14, 2026
@gfyrag
Copy link
Copy Markdown
Contributor

gfyrag commented May 15, 2026

Capabilities system: gap analysis

The capabilities system solves the coarse-grained problem well (which API namespace — v1/v2/v3 — to use for a given operation), but there's a significant gap in sub-operation capability modeling.

What works

  • ComponentCompatibility maps product semver ranges to supported API namespaces.
  • ResolveAPIVersion() intersects server-supported versions with CLI handler versions and picks the highest compatible one.
  • Feature constants are defined per operation and the generated manifest maps them to their API versions with full HTTP metadata.

What doesn't work yet

1. The generated manifest is inert at runtime.

GeneratedManifest is set on Runtime.Manifest but never queried during command execution. The Operations[Feature][APIVersion] map (with method, path, tags) is only used at generation time and in tests. The resolver never checks whether a specific Feature actually exists in the selected API version — it only checks that the Product supports that API version.

2. Feature is ignored in resolution logic.

Feature is passed into VersionResolutionRequest but only used for error messages in UnsupportedFeatureError. The actual resolution logic operates purely at the Product level — all features of a product are treated identically.

3. No sub-operation capability modeling.

There's no way to express "this operation supports parameter X starting from component version Y.Z.0" within the same API namespace. Today this is handled with ad-hoc conditionals scattered across handlers:

// list_transactions.go — manual handler filtering
if (input.StartTime != nil || input.EndTime != nil) && handler.APIVersion != "v1" {
    continue
}

// revert v1 handler — manual error
if input.AtEffectiveDate {
    return ..., fmt.Errorf("--at-effective-date requires ledger API v2+")
}

This pattern doesn't scale. Each new conditional parameter or field requires a hand-coded if in the service layer, with no centralized validation and no way for users to know upfront whether a flag is supported by their stack.

Concrete scenarios that break

Scenario What happens
Ledger 2.5 adds a query param $expand=volumes on v2, CLI sends it to Ledger 2.1 Silent ignore or 400 — CLI can't know in advance
Ledger 2.6 adds GET /v2/.../explain without creating v3 Manifest must be regenerated, but compatibility table still says "v2 supported from 2.0.0" with no per-feature granularity
A response field is added in a minor version CLI compiled with newer SDK may expect the field; older stacks won't return it

Suggestion

The structural pieces are there (Feature type, manifest with per-feature operation metadata, ComponentVersion available at resolution time) but they're not wired together. Two things would close the gap:

  1. Use the manifest at runtime — have the resolver verify that the selected API version actually exposes the requested Feature, not just that the Product supports the API version.
  2. Add feature-level version ranges — extend ComponentRange (or add a parallel structure) to express per-feature availability within an API version, e.g. {Product: "ledger", Feature: "expandVolumes", Range: ">=2.5.0", APIVersion: "v2"}. This would centralize the ad-hoc if checks and make them declarative.

@gfyrag
Copy link
Copy Markdown
Contributor

gfyrag commented May 15, 2026

Follow-up: --help always shows the full superset of flags

Related to the capabilities gap above — since Cobra flags are declared statically at command construction time, --help renders before any connection to the target stack. There's no call to /versions, no capability resolution.

This means --help always displays the superset of all flags across all API versions (v1 + v2 + v3), regardless of what the active profile's stack actually supports.

Example: fctl ledger transactions revert --help shows --at-effective-date even when targeting a Ledger 1.x stack that only supports v1. The user discovers the limitation only at execution time:

--at-effective-date requires ledger API v2+

The help output promises capabilities the stack may not have.

Possible approaches

Minimal (no runtime cost): annotate version-dependent flags in their description string:

command.Flags().BoolVar(&atEffectiveDate, "at-effective-date", false,
    "Revert at the original transaction effective date [ledger v2+]")

This is simple, doesn't require a network call on --help, and at least sets expectations before the user runs the command.

Dynamic (better UX, higher cost): when a profile is active, resolve capabilities on --help and hide or grey out unsupported flags. This gives accurate help but requires a network call to /versions on every --help invocation, which may not be acceptable.

@gfyrag
Copy link
Copy Markdown
Contributor

gfyrag commented May 15, 2026

Long-term concern: CLI bloat from accumulated version handlers

One more thing to flag — with this architecture, every new API version adds a new handler per operation, but old handlers are never removed (they're needed for backward compatibility with older stacks). Over time this means:

  • Binary size grows monotonically. Each handler pulls in its SDK namespace types (shared.V2PostTransaction, shared.V3PostTransaction, ...), request/response structs, and mapping code. The generated manifest also grows with every new operation across every version.
  • Maintenance surface expands. Every existing operation needs N handlers (one per supported API version), N mapping functions (toV1..., toV2..., toV3...), and ad-hoc feature-gating if blocks that multiply with each intra-version difference. A single CLI command like ledger transactions list already has 2 handlers, 2 request mappers, 2 response mappers — with v3 that becomes 3 of each.
  • Test matrix grows combinatorially. Each command × each API version × each version-specific flag combination needs coverage.
  • The compatibility table only grows. compatibility.go accumulates ranges but never sheds them, and the ad-hoc if checks in handlers accumulate with no centralized tracking of which ones exist or whether they're still relevant.

This is the natural consequence of "one CLI binary supports all stack versions," but it's worth deciding upfront what the support window is. Without a policy like "fctl v4 supports stack versions N and N-1 only," the handler/mapping/test accumulation is unbounded.

Something to keep in mind as the v4 architecture solidifies — the current design makes it easy to add version support, but has no mechanism to deprecate or remove it.

@gfyrag
Copy link
Copy Markdown
Contributor

gfyrag commented May 15, 2026

Comparison with #126 (plugin architecture) — focused on versioning strategy

Setting aside authentication and rendering (both can be solved the same way in either approach — plugins can call core rendering methods, and the profile/context model from this PR can be reused), the key differentiator is how product code is organized and how it evolves as API versions change.

Side-by-side

Criteria #153 — monolithic v4 #126 — plugins
1 API version = N handlers in the same binary 1 dedicated binary
Adding an API version Add handlers + mappers + ad-hoc if in existing code Publish a new plugin binary
Removing an API version Impossible without breaking compat — code accumulates Remove version from registry; old binaries remain available
--help reflects target stack No — static superset of all flags across all versions Yes — manifest comes from the installed plugin version
Intra-version capabilities Gap: no modeling, manual if checks scattered in handlers Problem doesn't exist — 1 plugin = 1 version = 1 feature set
SDK dependency One monolithic multi-version SDK (sdk.Ledger.V1, .V2, .V3) Each plugin imports only its target client
Ownership Centralized in the fctl repo Distributed per product team
Runtime overhead Zero — direct call Process spawn + gRPC — real but optimizable
Install friction Zero — single binary Auto-discovery (Cloud) or fctl plugin install (self-hosted/local)

The core issue

The fundamental problem with the monolithic v4 is that complexity is O(products × API versions) in a single binary, and it only grows. Every line in compatibility.go, every v1/v2/v3 handler, every toV1.../toV2... mapper, every ad-hoc feature-gating if is permanent code.

The plugin approach transforms this into O(1) per binary — each plugin knows exactly one API version, so there's a single code path, zero branching, zero feature-gating. Multi-version complexity moves from compilation (code) to deployment (registry), where it's easier to manage.

For reference, handleTransactionsRevert in the plugin ledger (#126) is ~50 lines with a single code path. The equivalent in v4 (#153) is ~400 lines with v1 + v2 handlers, two mappers, and manual feature-gating (if input.AtEffectiveDate { return error }).

Install friction — the only real argument against plugins

The process/gRPC overhead is real but technical and optimizable. The actual question is user experience:

  • Cloud: auto-discovery solves this — fctl knows which modules are active and installs plugins automatically
  • Self-hosted: fctl plugin install ledger once, or a fctl setup that detects services via /versions and installs matching plugins
  • Local dev: fctl plugin install --path ./cmd/fctl-plugin

This is real but one-time friction (at install, not per command), and it's the price for not accumulating multi-version code indefinitely in a monolith.

The three issues identified earlier — how each approach handles them

1. Capabilities / intra-version feature gaps: In #153, the capabilities system doesn't model sub-operation differences (new query params, fields added in minor versions). Each difference requires a hand-coded if in the service layer. In #126, the problem simply doesn't exist — a plugin binary targets exactly one API version, so all features of that version are available by definition.

2. --help showing unsupported flags: In #153, Cobra flags are declared statically at build time — --help always shows the superset. In #126, the manifest comes from the installed plugin, which only declares flags for its target API version.

3. Long-term binary bloat: In #153, handlers/mappers/tests accumulate with no removal mechanism. In #126, each plugin is an independent binary that can be replaced or retired without touching the core.

Bottom line

Once auth and rendering are neutralized, the question is: where does multi-version API complexity live?

The plugin architecture structurally resolves the three problems identified in this review without requiring additional mechanisms — it's a natural side effect of per-binary isolation.

gfyrag added a commit that referenced this pull request May 15, 2026
Proposes a plugin-based architecture for fctl where product CLI commands
ship as independent binaries from product repos. Uses Ledger v3 as the
concrete proof of concept with detailed user journeys covering Cloud,
self-hosted, local dev, debugging, error handling, and version retirement.

Key design decisions:
- Plugins live in product repos at cmd/fctl-plugin/
- Plugin version = service version (enforced)
- Registry derives binary URLs from convention
- Built-in commands remain for Ledger v2, plugin activates for v3+
- Resolution: plugin first, built-in fallback, auto-discovery last
- Rendering centralized in fctl core via display schemas
- Plugin lifecycle fully driven by auto-discovery, no manual updates

Relates to #126 and #153.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants