Skip to content

feat: add managed tool catalog and build environments - #397

Open
JWThewes wants to merge 42 commits into
mainfrom
feat/390-managed-build-environments
Open

feat: add managed tool catalog and build environments#397
JWThewes wants to merge 42 commits into
mainfrom
feat/390-managed-build-environments

Conversation

@JWThewes

@JWThewes JWThewes commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

Summary

  • ship one protected Standard environment with Node.js and Python, plus managed Java, Go, Rust, Maven, and Gradle tool definitions
  • let platform administrators import additional ARM64 tools such as .NET from public HTTPS archives without changing platform source
  • normalize each verified tool version into an immutable OCI artifact with retained provenance, SBOM, size, scan findings, and functional verification evidence
  • compose exact published tool-version digests into immutable environment revisions with dependency resolution, conflict checks, and projected AgentCore image-size enforcement
  • preserve reviewed Critical and High findings through publication after explicit administrator acceptance
  • snapshot the exact environment revision, image digest, runtime version, endpoint, compatibility version, tools, and verification result into every intent
  • add Tools and Environments views to Platform Settings, plus project assignment and repository compatibility warnings

Tool ingestion

  • validate public HTTPS sources and every redirect
  • compute and retain content-addressed source digests
  • distinguish Publisher verified evidence from Platform pinned imports
  • reject unsafe archives, traversal, extraction bombs, unsafe symlinks, credential-bearing URLs, and oversized payloads
  • run advanced installers in a credentialless container without metadata, private-network, Docker-socket, or host access
  • run version and functional verification networklessly as the non-root runtime user
  • support Java, Go, Rust, Maven, Gradle, .NET, generic CLI, and bounded custom verification fixtures

Compatibility

  • Standard is the only seeded environment
  • new and rebuilt environments use catalog-backed schema-v2 recipes
  • historical schema-v1 records remain readable for audit but cannot be built, retried, or published
  • no reset API, Lambda, IAM role, route, or Platform Settings view is shipped
  • the sole development account's legacy non-Standard runtimes, images, lookups, revisions, environments, and migration marker were removed manually

Verification

  • complete repository suite: 140 files, 2,591 tests passed
  • complete frontend suite: 73 files, 482 tests passed
  • managed tool and environment control plane: 10 files, 79 tests passed
  • npm run format:check
  • npm run lint
  • npm run secretlint
  • npm run sdk:check
  • npm --prefix frontend run typecheck
  • npm --prefix frontend run build
  • all remaining environment Lambda handlers bundled successfully
  • terraform fmt -recursive -check terraform
  • Terraform provider-schema validation with a local backend; only the two existing DynamoDB deprecation warnings remain
  • root and frontend production dependency audits report zero vulnerabilities

Terraform was not planned or applied against AWS. The deployed-stack workflow and .NET catalog example are documented in docs/development/testing.md.

Closes #390

@JWThewes
JWThewes marked this pull request as draft August 11, 2026 09:11
Comment thread lambda/environments/recipe.js Fixed
@JWThewes JWThewes changed the title feat: add managed build environments feat: add managed tool catalog and build environments Aug 13, 2026
Comment thread docs/using-the-platform/projects.md
Comment thread lambda/shared/environment-snapshot.js Outdated
Comment thread lambda/environments/status.js Outdated
Comment thread lambda/environments/status.js
Comment thread lambda/intents/index.js
Comment thread lambda/environments/tools-index.js Outdated
Comment thread lambda/projects/index.js Outdated
Comment thread lambda/discussions/services.js Outdated
Comment thread lambda/environments/catalog-recipe.js

@jeromevdl jeromevdl left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This feature is quite complex and there is legitimately complexity in the PR to handle this, but the PR adds 20-30% incidental complexity on top of it because of duplicated subsystems:

  • recipe v1 vs v2
  • 3 different ways to evaluate if environment is ready: readPublishedEnvironment, resolveEnvironmentSnapshot, publishedBase
  • one stack for environments, one for tools very close to the environment (createStore, createHandler, startBuild, createStatusHandler, ... in index/status/store vs tools-index/tools-status/tools-store, both modeled as: a versioned entity with DRAFT/BUILDING/READY/PUBLISHED-style revisions, built via CodeBuild, whose status is reconciled by a polling handler, fronted by a CRUD API handler backed by a DynamoDB store. That shared lifecycle is implemented twice. I've compared the shapes and signatures here, not diffed the two stacks line by line, so I can't put a precise "X% identical" number on it.

Can you check what could be merged (schema v1/v2 ? duplications of env and tools ? one environment resolution function ?) and maybe others to simplify / reduce the amount of code?

Comment thread lambda/discussions/services.js Outdated
@jeromevdl

Copy link
Copy Markdown
Contributor

Also, agents/index.js:79-80,107-108 (fetchRuntimeCapabilities, verifyMcpServers) still hit the core AGENTCORE_RUNTIME_ARN, never a project's managed environment.

@JWThewes

JWThewes commented Sep 1, 2026

Copy link
Copy Markdown
Contributor Author

@jeromevdl Addressed the agents/index.js runtime-routing concern in 170a2a6. Project settings now send projectId for capability discovery, and the agents Lambda authorizes project access, resolves the project’s published environment, and invokes both capabilities and project MCP verification with that runtime ARN + revision qualifier. Global Admin discovery/verification intentionally remains on the protected core runtime. The commit also adds the registry/IAM wiring and focused regression tests.

@JWThewes

JWThewes commented Sep 1, 2026

Copy link
Copy Markdown
Contributor Author

@jeromevdl I worked through the duplicated-subsystems change request:

  • 60ccee6 replaces the misleading recipe v1/v2 naming with fixed-tool (compatibility-only) and catalog (current authoring/build path). Keeping the fixed-tool reader avoids breaking persisted environments; new and rebuilt environments use catalog recipes.
  • 3f1ebfd makes resolvePublishedEnvironment the shared publication/readiness implementation, with resolveEnvironmentSnapshot as its projection. In 656d145, the remaining publishedBase helper was renamed requirePublishedBaseImage because it is specifically a build dependency/image check, not another runtime-readiness resolver.
  • 656d145 consolidates the shared environment/tool lifecycle mechanics: request parsing/auth/error handling, retryable initialization, DynamoDB pagination, S3 build-context upload, ECR retry/finding normalization, and CodeBuild/ECR/poll event dispatch. That commit is net -67 lines and the complete managed-environments suite remains green (89 tests plus all four Lambda bundles).

I kept the two domain state machines explicit because their publication semantics differ materially: environments supersede revisions and provision/verify AgentCore runtimes, while tools maintain version aliases/recommendations/dependency graphs and finish at image readiness. Merging those layers would replace visible domain rules with callback/configuration branching rather than reducing complexity.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[RFC]: Managed build environments for agent verification

3 participants