From 991769eb65e956478674a381018eb8d1a3446603 Mon Sep 17 00:00:00 2001 From: "denizirgin@gmail.com" Date: Sun, 5 Jul 2026 22:22:38 +0300 Subject: [PATCH 01/30] test: complete RIE-free contract coverage and benchmark harness - Add the Iteration 0 planning and research baseline: deep-dive code review findings, performance opportunities, roadmap wave mapping, AOT contract-tier design, execution handoff notes, and the initial Slopwatch baseline for pre-existing findings. - Add contract-test infrastructure for local AWS-style execution, including shared Testcontainers setup, deterministic contract fixtures, HTTP client base URL overrides, and normalized override URL handling for WireMock-backed NuGet/GitHub flows. - Expand active HTTP contract coverage across test-result ingestion, HMAC auth failure paths, test badges, redirects, NuGet package badges, GitHub package badges, upstream unauthorized/forbidden/not-found cases, missing GitHub secrets, routing, HEAD, CORS preflight, and response CORS headers. - Make test-result contract data deterministic so repeated local and CI runs do not collide on run identifiers, timestamps, nonces, or stored DynamoDB state. - Add and harden the k6 performance baseline tooling, including environment-driven configuration, Lambda response projection, RIE-safe validation, memory sampling via mstat export, and local smoke baseline capture. - Replace the active RIE-based contract path with Aspire Testing plus APIGatewayEmulator, and move local benchmark execution to the LocalStack ZIP Lambda path with Function URL fallback where LocalStack Community API Gateway v2 support is insufficient. - Record durable baseline evidence for the mock pre-iteration reference, final local LocalStack smoke, live direct API Gateway smoke, and live CloudFront comparison smoke; document that direct API Gateway is the Lambda/API baseline while CloudFront is only an edge-cache comparison. - Update Aspire/agent handover notes so future work starts from the RIE-free contract and benchmark topology, with CloudWatch Lambda REPORT lines as the source of truth for Lambda duration, memory, and cold-start observations. --- .gitignore | 1 + .../skills/aspire-source-navigation/SKILL.md | 12 + .slopwatch/baseline.json | 206 +++ .slopwatch/config.json.example | 10 + AGENTS.md | 8 + ARCHITECTURE.md | 6 +- Directory.Packages.props | 103 +- .../Constructs/BadgeSmithFunctionConstruct.cs | 66 +- .../Constructs/BadgeSmithHttpApiConstruct.cs | 31 + .../LocalPerformanceStack.cs | 117 ++ .../BadgeSmith.CDK.Shared/ProductionStack.cs | 16 +- build/BadgeSmith.CDK/Program.cs | 81 +- docs/ROADMAP.md | 30 +- docs/agents/README.md | 19 + .../s1-deep-dive-and-iteration0.prompt.md | 200 +++ .../s2-rie-free-iteration0-complete.prompt.md | 109 ++ .../agents/skills/aspire-source-navigation.md | 169 ++ ...-02-iteration0-aot-contract-tier-design.md | 190 +++ ...07-02-iteration0-aot-contract-tier-plan.md | 1501 +++++++++++++++++ ...e-aspire-localstack-implementation-plan.md | 1345 +++++++++++++++ ...-04-rie-free-aspire-localstack-redesign.md | 254 +++ .../2026-07-02-code-review-findings.md | 166 ++ .../2026-07-02-performance-opportunities.md | 219 +++ ...026-07-04-localstack-lambda-image-spike.md | 110 ++ docs/research/baselines/.gitkeep | 0 .../2026-07-04-final-localstack-smoke.json | 28 + .../2026-07-04-live-cloudfront-smoke.json | 100 ++ .../2026-07-04-live-gateway-smoke.json | 85 + .../2026-07-04-localstack-smoke.json | 28 + scripts/k6-perf-test.js | 153 +- scripts/perf-baseline-seed.sh | 89 + scripts/perf-baseline.ps1 | 4 + scripts/perf-baseline.sh | 500 ++++++ .../Core/Http/HttpClientFactory.cs | 24 +- src/BadgeSmith.Api/Core/Routing/ApiRouter.cs | 6 +- .../Core/Routing/Cors/CorsHandler.cs | 4 + .../Security/HmacAuthenticationService.cs | 4 + .../Core/Security/NonceService.cs | 4 + src/BadgeSmith.Api/Dockerfile | 12 + .../Features/GitHub/GitHubPackageService.cs | 4 + .../GitHub/GithubPackagesBadgeHandler.cs | 4 + .../NuGet/NuGetPackageBadgeHandler.cs | 4 + .../Features/NuGet/NuGetPackageService.cs | 4 + .../Handlers/TestResultIngestionHandler.cs | 6 +- .../Handlers/TestResultRedirectionHandler.cs | 4 + .../Handlers/TestResultsBadgeHandler.cs | 6 +- .../TestResults/TestResultsService.cs | 4 + src/BadgeSmith.Api/Program.Telemetry.cs | 4 +- src/BadgeSmith.Api/Program.cs | 4 +- src/BadgeSmith.Host/BadgeSmith.Host.csproj | 16 +- src/BadgeSmith.Host/Program.cs | 15 + src/shared/Constants.cs | 4 + .../BadgeSmith.Api.Tests.csproj | 26 +- .../Functional/HealthContractTests.cs | 22 + .../Functional/PackageBadgeContractTests.cs | 123 ++ .../Functional/RoutingContractTests.cs | 49 + .../Functional/TestResultsContractTests.cs | 208 +++ .../Http/HttpClientFactoryTests.cs | 77 + tests/BadgeSmith.Api.Tests/README.md | 25 + .../CorsHandler/ApplyResponseHeaders.cs | 2 + .../Routing/CorsHandler/CorsOptionsTests.cs | 2 + .../CorsHandler/HandlePreflightTests.cs | 2 + .../Routing/Patterns/ExactPatternTests.cs | 2 + .../Routing/Patterns/RegexPatternTests.cs | 2 + .../Routing/Patterns/TemplatePatternTests.cs | 2 + .../Routing/RouteResolverTests.cs | 2 + .../Routing/RouteValuesTests.cs | 2 + tests/BadgeSmith.Api.Tests/TestBase.cs | 4 +- .../Infrastructure/AspireContractFixture.cs | 101 ++ .../Testing/Infrastructure/AwsTestSeeder.cs | 137 ++ .../Infrastructure/ContractHttpClient.cs | 85 + .../Testing/Infrastructure/HmacTestSigner.cs | 17 + .../Infrastructure/wiremock/__files/.gitkeep | 0 .../__files/nuget-contracttest-index.json | 1 + .../mappings/github-versions-401.json | 4 + .../mappings/github-versions-403.json | 4 + .../mappings/github-versions-404.json | 4 + .../mappings/github-versions-empty.json | 8 + .../wiremock/mappings/github-versions-ok.json | 8 + .../wiremock/mappings/nuget-index-404.json | 4 + .../wiremock/mappings/nuget-index-ok.json | 8 + .../wiremock/mappings/ping.json | 9 + .../Testing/TestCategories.cs | 9 + .../OrgSecretSeeder.cs | 4 +- 84 files changed, 6866 insertions(+), 176 deletions(-) create mode 100644 .opencode/skills/aspire-source-navigation/SKILL.md create mode 100644 .slopwatch/baseline.json create mode 100644 .slopwatch/config.json.example create mode 100644 build/BadgeSmith.CDK.Shared/Constructs/BadgeSmithHttpApiConstruct.cs create mode 100644 build/BadgeSmith.CDK.Shared/LocalPerformanceStack.cs create mode 100644 docs/agents/handover-prompts/s1-deep-dive-and-iteration0.prompt.md create mode 100644 docs/agents/handover-prompts/s2-rie-free-iteration0-complete.prompt.md create mode 100644 docs/agents/skills/aspire-source-navigation.md create mode 100644 docs/plans/2026-07-02-iteration0-aot-contract-tier-design.md create mode 100644 docs/plans/2026-07-02-iteration0-aot-contract-tier-plan.md create mode 100644 docs/plans/2026-07-04-rie-free-aspire-localstack-implementation-plan.md create mode 100644 docs/plans/2026-07-04-rie-free-aspire-localstack-redesign.md create mode 100644 docs/research/2026-07-02-code-review-findings.md create mode 100644 docs/research/2026-07-02-performance-opportunities.md create mode 100644 docs/research/2026-07-04-localstack-lambda-image-spike.md create mode 100644 docs/research/baselines/.gitkeep create mode 100644 docs/research/baselines/2026-07-04-final-localstack-smoke.json create mode 100644 docs/research/baselines/2026-07-04-live-cloudfront-smoke.json create mode 100644 docs/research/baselines/2026-07-04-live-gateway-smoke.json create mode 100644 docs/research/baselines/2026-07-04-localstack-smoke.json create mode 100755 scripts/perf-baseline-seed.sh create mode 100755 scripts/perf-baseline.ps1 create mode 100755 scripts/perf-baseline.sh create mode 100644 tests/BadgeSmith.Api.Tests/Functional/HealthContractTests.cs create mode 100644 tests/BadgeSmith.Api.Tests/Functional/PackageBadgeContractTests.cs create mode 100644 tests/BadgeSmith.Api.Tests/Functional/RoutingContractTests.cs create mode 100644 tests/BadgeSmith.Api.Tests/Functional/TestResultsContractTests.cs create mode 100644 tests/BadgeSmith.Api.Tests/Http/HttpClientFactoryTests.cs create mode 100644 tests/BadgeSmith.Api.Tests/README.md create mode 100644 tests/BadgeSmith.Api.Tests/Testing/Infrastructure/AspireContractFixture.cs create mode 100644 tests/BadgeSmith.Api.Tests/Testing/Infrastructure/AwsTestSeeder.cs create mode 100644 tests/BadgeSmith.Api.Tests/Testing/Infrastructure/ContractHttpClient.cs create mode 100644 tests/BadgeSmith.Api.Tests/Testing/Infrastructure/HmacTestSigner.cs create mode 100644 tests/BadgeSmith.Api.Tests/Testing/Infrastructure/wiremock/__files/.gitkeep create mode 100644 tests/BadgeSmith.Api.Tests/Testing/Infrastructure/wiremock/__files/nuget-contracttest-index.json create mode 100644 tests/BadgeSmith.Api.Tests/Testing/Infrastructure/wiremock/mappings/github-versions-401.json create mode 100644 tests/BadgeSmith.Api.Tests/Testing/Infrastructure/wiremock/mappings/github-versions-403.json create mode 100644 tests/BadgeSmith.Api.Tests/Testing/Infrastructure/wiremock/mappings/github-versions-404.json create mode 100644 tests/BadgeSmith.Api.Tests/Testing/Infrastructure/wiremock/mappings/github-versions-empty.json create mode 100644 tests/BadgeSmith.Api.Tests/Testing/Infrastructure/wiremock/mappings/github-versions-ok.json create mode 100644 tests/BadgeSmith.Api.Tests/Testing/Infrastructure/wiremock/mappings/nuget-index-404.json create mode 100644 tests/BadgeSmith.Api.Tests/Testing/Infrastructure/wiremock/mappings/nuget-index-ok.json create mode 100644 tests/BadgeSmith.Api.Tests/Testing/Infrastructure/wiremock/mappings/ping.json create mode 100644 tests/BadgeSmith.Api.Tests/Testing/TestCategories.cs diff --git a/.gitignore b/.gitignore index 29456b9..f961d26 100644 --- a/.gitignore +++ b/.gitignore @@ -443,3 +443,4 @@ opencode.jsonc # Editor / local harness config (machine-specific; may contain local MCP ports and personal settings) .vscode/ +external/ diff --git a/.opencode/skills/aspire-source-navigation/SKILL.md b/.opencode/skills/aspire-source-navigation/SKILL.md new file mode 100644 index 0000000..11cd52e --- /dev/null +++ b/.opencode/skills/aspire-source-navigation/SKILL.md @@ -0,0 +1,12 @@ +--- +name: aspire-source-navigation +description: Use when compatibility-sensitive Aspire.Hosting.LocalStack work depends on Aspire/AWS/LocalStack upstream source, package-version alignment, AddLocalStack/UseLocalStack/WithReference behavior, endpoint/configuration flow, or AWS SDK/LocalStack.Client wiring. +--- + +# Aspire Source Navigation + +Canonical skill content lives in [docs/agents/skills/aspire-source-navigation.md](../../../docs/agents/skills/aspire-source-navigation.md). + +Read that file and follow it. This file is a native OpenCode discovery relay, not the source of truth. + +OpenCode loads project skills at session start. Restart OpenCode after changing this file if the running UI needs the updated skill. diff --git a/.slopwatch/baseline.json b/.slopwatch/baseline.json new file mode 100644 index 0000000..a5102ae --- /dev/null +++ b/.slopwatch/baseline.json @@ -0,0 +1,206 @@ +{ + "version": 1, + "createdAt": "2026-07-02T07:29:05.6447827+00:00", + "updatedAt": "2026-07-02T07:29:05.6474458+00:00", + "description": "Initial baseline created by 'slopwatch init' on 2026-07-02 07:29:05 UTC", + "entries": [ + { + "hash": "d4ab977cc248de3a", + "ruleId": "SW002", + "filePath": "build/BadgeSmith.CDK.Shared/ProductionStack.cs", + "lineNumber": 1, + "codeSnippet": "#pragma warning disable CA1711, MA0051", + "message": "#pragma warning disable for warnings CA1711, MA0051 without matching restore in same scope", + "baselinedAt": "2026-07-02T07:29:05.6473057+00:00" + }, + { + "hash": "530a245df19ab780", + "ruleId": "SW002", + "filePath": "build/BadgeSmith.CDK.Shared/Constructs/DynamoDbTablesConstruct.cs", + "lineNumber": 1, + "codeSnippet": "#pragma warning disable CA1711, MA0051", + "message": "#pragma warning disable for warnings CA1711, MA0051 without matching restore in same scope", + "baselinedAt": "2026-07-02T07:29:05.6473925+00:00" + }, + { + "hash": "35c40419d0396281", + "ruleId": "SW002", + "filePath": "build/BadgeSmith.CDK.Shared/Constructs/SharedInfrastructureConstruct.cs", + "lineNumber": 1, + "codeSnippet": "#pragma warning disable CA1711, MA0051, MA0056", + "message": "#pragma warning disable for warnings CA1711, MA0051, MA0056 without matching restore in same scope", + "baselinedAt": "2026-07-02T07:29:05.6473966+00:00" + }, + { + "hash": "2575f34a7cb0e2d1", + "ruleId": "SW002", + "filePath": "src/BadgeSmith.Api/Program.cs", + "lineNumber": 2, + "codeSnippet": "#pragma warning disable CA1502", + "message": "#pragma warning disable for warnings CA1502 without matching restore in same scope", + "baselinedAt": "2026-07-02T07:29:05.6473998+00:00" + }, + { + "hash": "32d28b382896d224", + "ruleId": "SW002", + "filePath": "src/BadgeSmith.Api/Program.Telemetry.cs", + "lineNumber": 2, + "codeSnippet": "#pragma warning disable CA1502", + "message": "#pragma warning disable for warnings CA1502 without matching restore in same scope", + "baselinedAt": "2026-07-02T07:29:05.6474042+00:00" + }, + { + "hash": "843c52e2514ea5b6", + "ruleId": "SW002", + "filePath": "src/BadgeSmith.Api/Core/Http/HttpClientFactory.cs", + "lineNumber": 1, + "codeSnippet": "#pragma warning disable S1075", + "message": "#pragma warning disable for warnings S1075 without matching restore in same scope", + "baselinedAt": "2026-07-02T07:29:05.6474065+00:00" + }, + { + "hash": "8e76f6c9847c0e34", + "ruleId": "SW003", + "filePath": "src/BadgeSmith.Api/Core/Http/ResilienceRetryHandler.cs", + "lineNumber": 33, + "codeSnippet": "catch (OperationCanceledException) when (!cancellationToken.IsCancellationRequested && attempt < _maxRetries)\r\n {\r\n // timeout: retry\r\n }", + "message": "Empty catch block swallows exceptions without handling", + "baselinedAt": "2026-07-02T07:29:05.6474133+00:00" + }, + { + "hash": "1dbe6b723437a0c6", + "ruleId": "SW003", + "filePath": "src/BadgeSmith.Api/Core/Http/ResilienceRetryHandler.cs", + "lineNumber": 37, + "codeSnippet": "catch (HttpRequestException) when (attempt < _maxRetries)\r\n {\r\n // transient network error: retry\r\n }", + "message": "Empty catch block swallows exceptions without handling", + "baselinedAt": "2026-07-02T07:29:05.6474167+00:00" + }, + { + "hash": "4047bfeade7908e2", + "ruleId": "SW002", + "filePath": "src/BadgeSmith.Api/Core/Security/HmacAuthenticationService.cs", + "lineNumber": 76, + "codeSnippet": "SuppressMessage(\"Usage\", \"MA0015:Specify the parameter name in ArgumentException\")", + "message": "SuppressMessage attribute suppressing Usage:MA0015:Specify the parameter name in ArgumentException", + "baselinedAt": "2026-07-02T07:29:05.6474191+00:00" + }, + { + "hash": "7a0abdc1638e9dc7", + "ruleId": "SW005", + "filePath": "src/BadgeSmith.Host/BadgeSmith.Host.csproj", + "lineNumber": 9, + "codeSnippet": "$(NoWarn);CS8002", + "message": "Adding warnings to NoWarn: CS8002", + "baselinedAt": "2026-07-02T07:29:05.6474211+00:00" + }, + { + "hash": "a4896e31db66e85f", + "ruleId": "SW002", + "filePath": "src/BadgeSmith.Host/BadgeSmithInfrastructureStack.cs", + "lineNumber": 1, + "codeSnippet": "#pragma warning disable CA1711", + "message": "#pragma warning disable for warnings CA1711 without matching restore in same scope", + "baselinedAt": "2026-07-02T07:29:05.6474234+00:00" + }, + { + "hash": "3c33d74a70342611", + "ruleId": "SW002", + "filePath": "src/BadgeSmith.Host/BadgeSmithInfrastructureStack.cs", + "lineNumber": 17, + "codeSnippet": "#pragma warning disable CA1812", + "message": "#pragma warning disable for warnings CA1812 without matching restore in same scope", + "baselinedAt": "2026-07-02T07:29:05.647425+00:00" + }, + { + "hash": "2c3e661aea7945e1", + "ruleId": "SW002", + "filePath": "src/BadgeSmith.Host/Program.cs", + "lineNumber": 1, + "codeSnippet": "#pragma warning disable CA2252 // Using 'AddAWSLambdaFunction' requires opting into preview features.", + "message": "#pragma warning disable for warnings CA2252 without matching restore in same scope", + "baselinedAt": "2026-07-02T07:29:05.6474274+00:00" + }, + { + "hash": "e64110d1b2254dc1", + "ruleId": "SW005", + "filePath": "tests/BadgeSmith.Api.Performance.Tests/BadgeSmith.Api.Performance.Tests.csproj", + "lineNumber": 6, + "codeSnippet": "$(NoWarn);CA1707;CA1303", + "message": "Adding warnings to NoWarn: CA1707;CA1303", + "baselinedAt": "2026-07-02T07:29:05.6474294+00:00" + }, + { + "hash": "02899ada962d04da", + "ruleId": "SW002", + "filePath": "tests/BadgeSmith.Api.Performance.Tests/BufferAllocationBenchmarks.cs", + "lineNumber": 1, + "codeSnippet": "#pragma warning disable CA1812,CA1852,CA1515", + "message": "#pragma warning disable for warnings CA1812, CA1852, CA1515 without matching restore in same scope", + "baselinedAt": "2026-07-02T07:29:05.6474313+00:00" + }, + { + "hash": "992583dc2c83097e", + "ruleId": "SW002", + "filePath": "tests/BadgeSmith.Api.Performance.Tests/RoutingBenchmarks.cs", + "lineNumber": 1, + "codeSnippet": "#pragma warning disable CA1812,CA1852,CA1515", + "message": "#pragma warning disable for warnings CA1812, CA1852, CA1515 without matching restore in same scope", + "baselinedAt": "2026-07-02T07:29:05.6474332+00:00" + }, + { + "hash": "db67a1c661dc246d", + "ruleId": "SW005", + "filePath": "tests/BadgeSmith.Api.Tests/BadgeSmith.Api.Tests.csproj", + "lineNumber": 6, + "codeSnippet": "$(NoWarn);CA1515;CA1034;CA1707;MA0110;MA0009;CA1062;CA1024", + "message": "Adding warnings to NoWarn: CA1515;CA1034;CA1707;MA0110;MA0009;CA1062;CA1024", + "baselinedAt": "2026-07-02T07:29:05.6474351+00:00" + }, + { + "hash": "1afac4078b2b2d04", + "ruleId": "SW002", + "filePath": "tests/BadgeSmith.Api.Tests/TestBase.cs", + "lineNumber": 1, + "codeSnippet": "#pragma warning disable S2325", + "message": "#pragma warning disable for warnings S2325 without matching restore in same scope", + "baselinedAt": "2026-07-02T07:29:05.6474374+00:00" + }, + { + "hash": "9e9efc33ac987127", + "ruleId": "SW002", + "filePath": "tests/BadgeSmith.Api.Tests/Routing/RouteResolverTests.cs", + "lineNumber": 328, + "codeSnippet": "SuppressMessage(\"Design\", \"MA0051:Method is too long\")", + "message": "SuppressMessage attribute suppressing Design:MA0051:Method is too long", + "baselinedAt": "2026-07-02T07:29:05.6474395+00:00" + }, + { + "hash": "f9453961584da507", + "ruleId": "SW002", + "filePath": "tests/BadgeSmith.Api.Tests/Routing/Patterns/RegexPatternTests.cs", + "lineNumber": 274, + "codeSnippet": "SuppressMessage(\"Design\", \"CA1024:Use properties where appropriate\")", + "message": "SuppressMessage attribute suppressing Design:CA1024:Use properties where appropriate", + "baselinedAt": "2026-07-02T07:29:05.6474414+00:00" + }, + { + "hash": "f808292cb200c5cc", + "ruleId": "SW002", + "filePath": "tests/BadgeSmith.Api.Tests/Routing/Patterns/TemplatePatternTests.cs", + "lineNumber": 249, + "codeSnippet": "SuppressMessage(\"Design\", \"MA0051:Method is too long\")", + "message": "SuppressMessage attribute suppressing Design:MA0051:Method is too long", + "baselinedAt": "2026-07-02T07:29:05.6474434+00:00" + }, + { + "hash": "a683cafea2a944cf", + "ruleId": "SW002", + "filePath": "tests/seeders/BadgeSmith.DynamoDb.Seeders/OrgSecretSeeder.cs", + "lineNumber": 1, + "codeSnippet": "#pragma warning disable CA1812, MA0134, VSTHRD110, CA2000", + "message": "#pragma warning disable for warnings CA1812, MA0134, VSTHRD110, CA2000 without matching restore in same scope", + "baselinedAt": "2026-07-02T07:29:05.6474457+00:00" + } + ] +} diff --git a/.slopwatch/config.json.example b/.slopwatch/config.json.example new file mode 100644 index 0000000..79850ff --- /dev/null +++ b/.slopwatch/config.json.example @@ -0,0 +1,10 @@ +{ + "suppressions": [ + { + "ruleId": "SW002", + "pattern": "**/Generated/**", + "justification": "Generated code from protobuf/gRPC compiler - cannot be modified" + } + ], + "globalSuppressions": [] +} \ No newline at end of file diff --git a/AGENTS.md b/AGENTS.md index 5eb5702..04e7e91 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -221,6 +221,14 @@ its trigger applies, and do not invent an ID. | Performance work / benchmarks | Benchmark and performance-diagnostics capabilities; require measured data | | Package version changes (`Directory.Packages.props`) | Package-management capability (CPM) | +## Aspire Source Compatibility + +For read-only explanation questions, inspect this repository's docs/code first. Invoke `aspire-source-navigation` only when the answer depends on upstream internals, version-specific API shape, or a compatibility conclusion. + +## Aspire MCP Server + +Utilize Aspire MCP server for runtime resource state/logs/traces of CLI-launched AppHosts and LocalStack. And context7 for Aspire related documentation. + ## Semantic Code Navigation When Rider MCP tools are available, prefer semantic tools for C# symbol questions: diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 91dcd90..d53b0ed 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -197,7 +197,8 @@ Package badge endpoints are **unauthenticated** but include: - **`build-lambda.sh/.ps1`**: Multi-platform Docker builds for Lambda deployment - **`test-ingestion.sh/.ps1`**: HMAC authentication testing with real API calls -- **`k6-perf-test.js`**: Load testing with realistic traffic patterns +- **`k6-perf-test.js`**: HTTP load testing with realistic traffic patterns +- **`perf-baseline.sh`**: LocalStack-backed ZIP Lambda benchmark harness. It deploys the local CDK performance stack and uses Lambda Function URL fallback when LocalStack Community cannot create API Gateway v2 through CloudFormation. - **`sample-test-payload.json`**: Example test result payload ## ๐Ÿ—๏ธ **Code Organization** @@ -251,6 +252,9 @@ BadgeSmith uses **OneOf result types** instead of exceptions for predictable err - **LocalStack integration**: AWS service emulation - **Lambda emulation**: Local function execution +- **Contract tests**: Aspire Testing starts `src/BadgeSmith.Host` and calls `APIGatewayEmulator` over HTTP; the test suite does not use Lambda RIE. + +**Local benchmark execution** uses Docker, LocalStack, CDK, and k6. Production keeps API Gateway HTTP v2, but the local performance stack exposes a Lambda Function URL fallback because LocalStack Community 4.6 does not deploy API Gateway v2 CloudFormation resources in this workflow. ## ๐Ÿš€ **Deployment Strategy** diff --git a/Directory.Packages.props b/Directory.Packages.props index aa162e7..3c52b6e 100644 --- a/Directory.Packages.props +++ b/Directory.Packages.props @@ -1,63 +1,66 @@ - - - - - - - - - - - + + + + + + + + + + + + - - + + - - - - - - - - - - + + + + + + + + + + - - + + - - - - + + + + - + - - - - - - - - + + + + + + + + - - - - - - + + + + + + - - - - - - + + + + + + + + diff --git a/build/BadgeSmith.CDK.Shared/Constructs/BadgeSmithFunctionConstruct.cs b/build/BadgeSmith.CDK.Shared/Constructs/BadgeSmithFunctionConstruct.cs index 4452209..57b1553 100644 --- a/build/BadgeSmith.CDK.Shared/Constructs/BadgeSmithFunctionConstruct.cs +++ b/build/BadgeSmith.CDK.Shared/Constructs/BadgeSmithFunctionConstruct.cs @@ -20,34 +20,64 @@ public BadgeSmithFunctionConstruct( ITable nonceTable, ITable orgSecretTable, IRole lambdaExecutionRole, - string id) : base(scope, id) + string id) + : this( + scope, + testResultsTable, + nonceTable, + orgSecretTable, + lambdaExecutionRole, + id, + BadgeSmithFunctionConfiguration.Production) + { + } + + public BadgeSmithFunctionConstruct( + Construct scope, + ITable testResultsTable, + ITable nonceTable, + ITable orgSecretTable, + IRole lambdaExecutionRole, + string id, + BadgeSmithFunctionConfiguration configuration) : base(scope, id) { ArgumentNullException.ThrowIfNull(scope); ArgumentNullException.ThrowIfNull(testResultsTable); ArgumentNullException.ThrowIfNull(nonceTable); ArgumentNullException.ThrowIfNull(orgSecretTable); ArgumentNullException.ThrowIfNull(lambdaExecutionRole); + ArgumentNullException.ThrowIfNull(configuration); + + var environment = new Dictionary(StringComparer.Ordinal) + { + ["DOTNET_ENVIRONMENT"] = "Production", + ["APP_NAME"] = LambdaName, + ["APP_ENABLE_TELEMETRY_FACTORY_PERF_LOGS"] = "true", + ["AWS_RESOURCE_TEST_RESULTS_TABLE"] = testResultsTable.TableName, + ["AWS_RESOURCE_NONCE_TABLE"] = nonceTable.TableName, + ["AWS_RESOURCE_ORG_SECRETS_TABLE"] = orgSecretTable.TableName, + // ["AWS_LAMBDA_EXEC_WRAPPER"] = "/opt/otel-instrument", // For future OpenTelemetry support + }; + + if (configuration.ExtraEnvironment is not null) + { + foreach (var (key, value) in configuration.ExtraEnvironment) + { + environment[key] = value; + } + } BadgeSmithFunction = new Function(this, LambdaId, new FunctionProps { FunctionName = LambdaName, Runtime = Runtime.PROVIDED_AL2023, - Code = Code.FromAsset("../artifacts/badge-lambda-linux-arm64.zip"), + Code = Code.FromAsset(configuration.AssetPath), Handler = "bootstrap", Role = lambdaExecutionRole, Timeout = Duration.Seconds(LambdaTimeoutInSeconds), MemorySize = 512, - Architecture = Architecture.ARM_64, - Environment = new Dictionary(StringComparer.Ordinal) - { - ["DOTNET_ENVIRONMENT"] = "Production", - ["APP_NAME"] = LambdaName, - ["APP_ENABLE_TELEMETRY_FACTORY_PERF_LOGS"] = "true", - ["AWS_RESOURCE_TEST_RESULTS_TABLE"] = testResultsTable.TableName, - ["AWS_RESOURCE_NONCE_TABLE"] = nonceTable.TableName, - ["AWS_RESOURCE_ORG_SECRETS_TABLE"] = orgSecretTable.TableName, - // ["AWS_LAMBDA_EXEC_WRAPPER"] = "/opt/otel-instrument", // For future OpenTelemetry support - }, + Architecture = configuration.Architecture, + Environment = environment, Description = "BadgeSmith Native AOT Lambda function for badge generation", }); @@ -60,3 +90,13 @@ public BadgeSmithFunctionConstruct( public Function BadgeSmithFunction { get; } } + +public sealed record BadgeSmithFunctionConfiguration( + string AssetPath, + Architecture Architecture, + IReadOnlyDictionary? ExtraEnvironment = null) +{ + public static BadgeSmithFunctionConfiguration Production { get; } = new( + "../artifacts/badge-lambda-linux-arm64.zip", + Architecture.ARM_64); +} diff --git a/build/BadgeSmith.CDK.Shared/Constructs/BadgeSmithHttpApiConstruct.cs b/build/BadgeSmith.CDK.Shared/Constructs/BadgeSmithHttpApiConstruct.cs new file mode 100644 index 0000000..0b759ca --- /dev/null +++ b/build/BadgeSmith.CDK.Shared/Constructs/BadgeSmithHttpApiConstruct.cs @@ -0,0 +1,31 @@ +using Amazon.CDK.AWS.Apigatewayv2; +using Amazon.CDK.AWS.Lambda; +using Amazon.CDK.AwsApigatewayv2Integrations; +using Constructs; +using static BadgeSmith.Constants; + +namespace BadgeSmith.CDK.Shared.Constructs; + +/// +/// HTTP API Gateway configured with BadgeSmith's Lambda proxy integration. +/// +public sealed class BadgeSmithHttpApiConstruct : HttpApi +{ + public BadgeSmithHttpApiConstruct(Construct scope, string id, IFunction badgeSmithFunction) + : base(scope, id, new HttpApiProps + { + ApiName = ApiGatewayName, + Description = "BadgeSmith API Gateway for badge endpoints", + DefaultIntegration = CreateLambdaIntegration(badgeSmithFunction), + }) + { + } + + private static HttpLambdaIntegration CreateLambdaIntegration(IFunction badgeSmithFunction) + { + ArgumentNullException.ThrowIfNull(badgeSmithFunction); + return new HttpLambdaIntegration(HttpLambdaIntegrationId, badgeSmithFunction); + } + + public HttpApi ApiGateway => this; +} diff --git a/build/BadgeSmith.CDK.Shared/LocalPerformanceStack.cs b/build/BadgeSmith.CDK.Shared/LocalPerformanceStack.cs new file mode 100644 index 0000000..abd0698 --- /dev/null +++ b/build/BadgeSmith.CDK.Shared/LocalPerformanceStack.cs @@ -0,0 +1,117 @@ +#pragma warning disable CA1711 + +using Amazon.CDK; +using Amazon.CDK.AWS.Apigatewayv2; +using Amazon.CDK.AWS.DynamoDB; +using Amazon.CDK.AWS.Lambda; +using BadgeSmith.CDK.Shared.Constructs; +using Constructs; +using Function = Amazon.CDK.AWS.Lambda.Function; +using static BadgeSmith.Constants; + +namespace BadgeSmith.CDK.Shared; + +/// +/// LocalStack-only stack for running local performance baselines without production edge resources. +/// +public sealed class LocalPerformanceStack : Stack +{ + public LocalPerformanceStack( + Construct scope, + string id, + LocalPerformanceStackSettings settings, + IStackProps? props = null) : base(scope, id, props) + { + ArgumentNullException.ThrowIfNull(settings); + + SharedInfrastructureConstruct = new SharedInfrastructureConstruct(this, SharedInfrastructureConstructId); + + TestResultsTable = SharedInfrastructureConstruct.TestResultsTable; + NonceTable = SharedInfrastructureConstruct.NonceTable; + OrgSecretsTable = SharedInfrastructureConstruct.OrgSecretsTable; + + BadgeSmithFunctionConstruct = new BadgeSmithFunctionConstruct( + this, + TestResultsTable, + NonceTable, + OrgSecretsTable, + SharedInfrastructureConstruct.LambdaExecutionRole, + LambdaConstructId, + new BadgeSmithFunctionConfiguration( + settings.LambdaAssetPath, + settings.LambdaArchitecture, + settings.LambdaEnvironment)); + + BadgeSmithFunction = BadgeSmithFunctionConstruct.BadgeSmithFunction; + BadgeSmithFunctionUrl = BadgeSmithFunction.AddFunctionUrl(new FunctionUrlOptions + { + AuthType = FunctionUrlAuthType.NONE, + }); + + var httpApiConstruct = new BadgeSmithHttpApiConstruct(this, ApiGatewayRoleId, BadgeSmithFunction); + ApiGateway = httpApiConstruct.ApiGateway; + Amazon.CDK.Tags.Of(httpApiConstruct).Add("_custom_id_", ApiGatewayName); + + CreateOutputs(); + + Tags.SetTag("environment", "LocalPerformance"); + Tags.SetTag("stack", "badge-smith-local-performance"); + Tags.SetTag("managed-by", "perf-baseline"); + } + + private void CreateOutputs() + { + _ = new CfnOutput(this, ApiGatewayOutputUrl, new CfnOutputProps + { + Value = ApiGateway.ApiEndpoint, + Description = "API Gateway endpoint URL", + }); + + _ = new CfnOutput(this, LambdaOutputFunctionUrl, new CfnOutputProps + { + Value = BadgeSmithFunctionUrl.Url, + Description = "Lambda Function URL for local performance fallback", + }); + + _ = new CfnOutput(this, TestResultsOutputTableName, new CfnOutputProps + { + Value = TestResultsTable.TableName, + Description = "DynamoDB table name for test results", + }); + + _ = new CfnOutput(this, NonceTableOutputTableName, new CfnOutputProps + { + Value = NonceTable.TableName, + Description = "DynamoDB table name for nonce", + }); + + _ = new CfnOutput(this, OrgSecretsOutputTableName, new CfnOutputProps + { + Value = OrgSecretsTable.TableName, + Description = "DynamoDB table name for GitHub org secrets", + }); + } + + public SharedInfrastructureConstruct SharedInfrastructureConstruct { get; } + + public BadgeSmithFunctionConstruct BadgeSmithFunctionConstruct { get; } + + public Function BadgeSmithFunction { get; } + + public IFunctionUrl BadgeSmithFunctionUrl { get; } + + public Table TestResultsTable { get; } + + public Table NonceTable { get; } + + public Table OrgSecretsTable { get; } + + public HttpApi ApiGateway { get; } +} + +public sealed record LocalPerformanceStackSettings( + string LambdaAssetPath, + Architecture LambdaArchitecture, + IReadOnlyDictionary LambdaEnvironment); + +#pragma warning restore CA1711 diff --git a/build/BadgeSmith.CDK.Shared/ProductionStack.cs b/build/BadgeSmith.CDK.Shared/ProductionStack.cs index ece9d9c..07f9fa5 100644 --- a/build/BadgeSmith.CDK.Shared/ProductionStack.cs +++ b/build/BadgeSmith.CDK.Shared/ProductionStack.cs @@ -9,7 +9,6 @@ using Amazon.CDK.AWS.Logs; using Amazon.CDK.AWS.Route53; using Amazon.CDK.AWS.Route53.Targets; -using Amazon.CDK.AwsApigatewayv2Integrations; using BadgeSmith.CDK.Shared.Constructs; using Constructs; using Function = Amazon.CDK.AWS.Lambda.Function; @@ -42,7 +41,8 @@ public ProductionStack(Construct scope, string id, IStackProps? props = null) : BadgeSmithFunction = BadgeSmithFunctionConstruct.BadgeSmithFunction; - ApiGateway = CreateApiGateway(); + var httpApiConstruct = new BadgeSmithHttpApiConstruct(this, ApiGatewayRoleId, BadgeSmithFunction); + ApiGateway = httpApiConstruct.ApiGateway; var logGroup = new LogGroup(this, "HttpApiAccessLogs", new LogGroupProps { @@ -86,18 +86,6 @@ public ProductionStack(Construct scope, string id, IStackProps? props = null) : private ICertificate ApiLocalStackCertificate => Certificate.FromCertificateArn(this, ApiCertificateId, "arn:aws:acm:us-east-1:377140207735:certificate/227f14fe-92b1-442c-bb80-ae4032e742fe"); - private HttpApi CreateApiGateway() - { - var lambdaIntegration = new HttpLambdaIntegration(HttpLambdaIntegrationId, BadgeSmithFunction); - - return new HttpApi(this, ApiGatewayRoleId, new HttpApiProps - { - ApiName = ApiGatewayName, - Description = "BadgeSmith API Gateway for badge endpoints", - DefaultIntegration = lambdaIntegration, - }); - } - private Distribution CreateCloudFrontDistribution() { var apiGatewayDomain = Fn.Select(2, Fn.Split("/", ApiGateway.ApiEndpoint)); diff --git a/build/BadgeSmith.CDK/Program.cs b/build/BadgeSmith.CDK/Program.cs index b6b192a..67fbac7 100644 --- a/build/BadgeSmith.CDK/Program.cs +++ b/build/BadgeSmith.CDK/Program.cs @@ -4,17 +4,82 @@ var app = new App(); +var stack = app.Node.TryGetContext("stack") as string; + // Get environment from CDK context or CLI -var env = new Amazon.CDK.Environment +var env = stack is "local-performance" + ? CreateLocalPerformanceEnvironment(app) + : new Amazon.CDK.Environment + { + Account = app.Node.TryGetContext("account") as string ?? System.Environment.GetEnvironmentVariable("CDK_DEFAULT_ACCOUNT"), + Region = app.Node.TryGetContext("region") as string ?? System.Environment.GetEnvironmentVariable("CDK_DEFAULT_REGION"), + }; + +if (stack is "local-performance") { - Account = app.Node.TryGetContext("account") as string ?? System.Environment.GetEnvironmentVariable("CDK_DEFAULT_ACCOUNT"), - Region = app.Node.TryGetContext("region") as string ?? System.Environment.GetEnvironmentVariable("CDK_DEFAULT_REGION"), -}; + var localPerformanceSettings = CreateLocalPerformanceSettings(app); -_ = new ProductionStack(app, ProductionStackId, new StackProps + _ = new LocalPerformanceStack(app, LocalPerformanceStackId, localPerformanceSettings, new StackProps + { + Env = env, + Description = "BadgeSmith local performance infrastructure for LocalStack benchmarking", + }); +} +else { - Env = env, - Description = "BadgeSmith production infrastructure with CloudFront and SSL certificate", -}); + _ = new ProductionStack(app, ProductionStackId, new StackProps + { + Env = env, + Description = "BadgeSmith production infrastructure with CloudFront and SSL certificate", + }); +} app.Synth(); + +static Amazon.CDK.Environment CreateLocalPerformanceEnvironment(App app) +{ + return new Amazon.CDK.Environment + { + Account = app.Node.TryGetContext("account") as string + ?? "000000000000", + Region = app.Node.TryGetContext("region") as string + ?? "us-east-1", + }; +} + +static LocalPerformanceStackSettings CreateLocalPerformanceSettings(App app) +{ + var lambdaAssetPath = GetContextValue(app, "lambdaZipPath", "../artifacts/badge-lambda-linux-x64.zip"); + var lambdaArchitecture = GetLambdaArchitecture(GetContextValue(app, "lambdaArchitecture", "x86_64")); + var httpNuGetBaseUrl = GetContextValue(app, "httpNuGetBaseUrl", "https://api.nuget.org/"); + var httpGitHubBaseUrl = GetContextValue(app, "httpGitHubBaseUrl", "https://api.github.com/"); + var localStackEndpoint = GetContextValue(app, "localStackEndpoint", "http://localstack:4566"); + + return new LocalPerformanceStackSettings( + lambdaAssetPath, + lambdaArchitecture, + new Dictionary(StringComparer.Ordinal) + { + ["AWS_ENDPOINT_URL"] = localStackEndpoint, + ["AWS_ENDPOINT_URL_DYNAMODB"] = localStackEndpoint, + ["AWS_ENDPOINT_URL_SECRETS_MANAGER"] = localStackEndpoint, + ["AWS_ENDPOINT_URL_SECRETSMANAGER"] = localStackEndpoint, + ["HTTP_NUGET_BASE_URL"] = httpNuGetBaseUrl, + ["HTTP_GITHUB_BASE_URL"] = httpGitHubBaseUrl, + }); +} + +static string GetContextValue(App app, string key, string defaultValue) +{ + return app.Node.TryGetContext(key) as string ?? defaultValue; +} + +static Amazon.CDK.AWS.Lambda.Architecture GetLambdaArchitecture(string value) +{ + return value switch + { + "x86_64" => Amazon.CDK.AWS.Lambda.Architecture.X86_64, + "arm64" => Amazon.CDK.AWS.Lambda.Architecture.ARM_64, + _ => throw new ArgumentException("lambdaArchitecture must be either 'x86_64' or 'arm64'.", nameof(value)), + }; +} diff --git a/docs/ROADMAP.md b/docs/ROADMAP.md index d361b6e..278b79d 100644 --- a/docs/ROADMAP.md +++ b/docs/ROADMAP.md @@ -1,6 +1,6 @@ # BadgeSmith Roadmap -Date: 2026-07-01 +Date: 2026-07-05 Backlog and progress source of truth for BadgeSmith. Keep this current: the Status & Plan Mapping table is the permanent index, detailed plans live in `docs/plans/`, and @@ -11,16 +11,40 @@ new ideas land in Inbox / Untriaged until they are scoped. | Workstream | Status | Plan | Notes | | --- | --- | --- | --- | | Agent contract adoption | done | โ€” (landed as a single `docs:` commit) | Re-authored `AGENTS.md`, harness relays, and `docs/agents/` for BadgeSmith after they were copied from another repo | +| Iteration 0 โ€” RIE-free contract coverage and local benchmark harness | done | [plans/2026-07-04-rie-free-aspire-localstack-implementation-plan.md](plans/2026-07-04-rie-free-aspire-localstack-implementation-plan.md) | Completed on 2026-07-05 on `feature/iteration0-aot-contract-tier`: RIE was removed from active contract and benchmark paths; Aspire Testing + `APIGatewayEmulator` covers HTTP contracts; LocalStack ZIP Lambda benchmark execution uses a CDK-created Lambda Function URL fallback because LocalStack Community 4.6 blocks API Gateway v2 CloudFormation resources. Baselines: [final local smoke](research/baselines/2026-07-04-final-localstack-smoke.json), [live direct Gateway smoke](research/baselines/2026-07-04-live-gateway-smoke.json), and [live CloudFront comparison smoke](research/baselines/2026-07-04-live-cloudfront-smoke.json). Old RIE plan kept only as historical context: [plans/2026-07-02-iteration0-aot-contract-tier-plan.md](plans/2026-07-02-iteration0-aot-contract-tier-plan.md). | + +## Process Notes + +- For live Lambda/API performance, use the direct API Gateway baseline; CloudFront runs are comparison data because edge caching can reduce Lambda invocations and hide API behavior. +- For Lambda duration, memory, and cold starts, use CloudWatch Lambda `REPORT` lines as the source of truth. k6 client-side cold-start heuristics are only smoke-test hints. ## Backlog Scoped work waiting to start. Promote an item into Status & Plan Mapping (and write a plan under `docs/plans/`) when it becomes active. -- _(none yet)_ +- **Wave 1 โ€” correctness fixes** from the 2026-07-02 deep-dive: HMAC `repoIdentifier` + bug, GSI1PK case bug, build-script RID default vs CDK arm64, seeder `.dist` JSON fix, + nonce-ordering + error-message hygiene, PAT rotation. Details: + [research/2026-07-02-code-review-findings.md](research/2026-07-02-code-review-findings.md) ยง1โ€“2. +- **Wave 2 โ€” test safety net**: HMAC / ResponseHelper / real RouteTable / + NuGetVersionService tests; align resolver tests with production routes. Details: + findings doc ยง4. +- **Wave 3 โ€” hygiene**: DRY refactors (bootstrap, route-param extraction, package + services), dead-code removal, script/docs drift, DynamoDB PITR/removal policy. + Details: findings doc ยง3, ยง5. +- **Logging hygiene โ€” source-generated logging migration** (2026-07-02): Replace + temporary `CA1873` pragmas with `LoggerMessageAttribute` source-generated logging, + then remove the suppressions and keep the zero-warning build contract. ## Inbox / Untriaged Raw capture spot for ideas and requests before they are scoped into the backlog. -- _(empty)_ +- Performance pass (cold start + memory footprint) โ€” measured and decided, ready to + implement: + [research/2026-07-02-performance-opportunities.md](research/2026-07-02-performance-opportunities.md). + Agreed levers: edge-side 404 caching + badge TTL increase, eager INIT warm-up, + `TrimMode=full` + ILC knobs, result caching in package services. Rejected: + provisioned concurrency, keep-warm pings, SnapStart (N/A on provided.al2023). + Folds in GitHub issue #1 (RouteValues buffer guard). diff --git a/docs/agents/README.md b/docs/agents/README.md index f08e37f..5a5a566 100644 --- a/docs/agents/README.md +++ b/docs/agents/README.md @@ -65,6 +65,8 @@ Tier meanings: - **Out of scope** โ€” do not use unless this repo adds that technology or Deniz explicitly asks. +This repo ships exactly one project skill: `aspire-source-navigation`. Its canonical body lives in `docs/agents/skills/aspire-source-navigation.md`, with thin native relays under `.claude/skills/`, `.opencode/skills/`, and `.github/skills/`. + Claude Code uses plugin-qualified names. Copilot CLI exposes the installed skill IDs directly through the running harness's `skill` list. OpenCode exposes skill *frontmatter* names, which depend on the local install; the names below reflect the @@ -94,6 +96,7 @@ manually. | Capability | Claude Code | Copilot CLI | OpenCode | | --- | --- | --- | --- | +| Aspire source compatibility for upstream Aspire/AWS/LocalStack.Client internals | `aspire-source-navigation` | `aspire-source-navigation` | `aspire-source-navigation` | | System.Text.Json AOT source-generation / serialization contracts | `dotnet-skills:serialization` | `serialization` | `serialization` | | Modern C# coding standards | `dotnet-skills:csharp-coding-standards` | `modern-csharp-coding-standards` | `modern-csharp-coding-standards` | | Type design and performance (seal, readonly struct, static pure) | `dotnet-skills:csharp-type-design-performance` | `type-design-performance` | `type-design-performance` | @@ -163,6 +166,22 @@ BadgeSmith still uses VSTest with `dotnet test`. | Working-diff code review (findings-first, severity-ordered) | `code-review` (harness built-in) | `code-review` via `task` | `codex-review` via `task` when present | | Security review of pending changes (HMAC, nonce, secrets, replay protection) | `security-review` (harness built-in) | `security-review` via `task` | N/A | +### Tier 2 โ€” Official Aspire skills and Aspire MCP server + +Official Microsoft Aspire skills and MCP server are local harness setup, not committed project infrastructure. They require Aspire CLI 13.3+ (`aspire agent mcp`). + +| Capability | Claude Code | Copilot CLI | OpenCode | +| --- | --- | --- | --- | +| AppHost lifecycle routing + safety guardrails (`aspire start`, never `dotnet run` on AppHosts) | `aspire:aspire` | `aspire` | `aspire` | +| Start/stop/restart/wait/inspect playground AppHost resources | `aspire:aspire-orchestration` | `aspire-orchestration` | `aspire-orchestration` | +| Resource logs, traces, metrics, dashboard telemetry | `aspire:aspire-monitoring` | `aspire-monitoring` | `aspire-monitoring` | +| Runtime resource state/logs/traces/commands over MCP | `aspire` MCP server (`aspire agent mcp`, stdio; tools surface as `mcp__aspire__*`) | `aspire` MCP server (`aspire agent mcp`, stdio; user `~/.copilot/mcp-config.json`) | `aspire` MCP server (`aspire agent mcp`, stdio; local `opencode.jsonc`) | + +- The MCP server only discovers AppHosts launched with `aspire start` from the workspace directory. In-process `DistributedApplicationTestingBuilder` AppHosts used by integration tests are invisible to it โ€” test debugging stays log/debugger-based. +- These skills/tools are for *consuming* Aspire (running and debugging playground AppHosts). They do not replace `aspire-source-navigation` for upstream source-compatibility work; on conflict, verified package source wins. +- The bundle also ships `aspire-init` and `aspireify` (not for this repo โ€” AppHosts already exist) and `aspire-deployment` (approval-gated and real-AWS targeted; LocalStack playgrounds do not deploy). +- Set up each harness locally and update only that harness's cells after verifying the native skill IDs and MCP status. + ### Tier 3 โ€” Local-only | Capability | Claude Code | Copilot CLI | OpenCode | diff --git a/docs/agents/handover-prompts/s1-deep-dive-and-iteration0.prompt.md b/docs/agents/handover-prompts/s1-deep-dive-and-iteration0.prompt.md new file mode 100644 index 0000000..03b2ef1 --- /dev/null +++ b/docs/agents/handover-prompts/s1-deep-dive-and-iteration0.prompt.md @@ -0,0 +1,200 @@ +--- +name: "S1 Deep-dive audit, perf research, iteration 0 spec+plan" +description: "Superseded priming prompt for the 2026-07-02 BadgeSmith handover after the full-repo deep-dive, measured perf research, and original iteration 0 contract-tier design+plan. Retained for historical context only; use docs/ROADMAP.md and the 2026-07-04 RIE-free plan for current routing." +argument-hint: "Optional focus area, constraints, or reason to override the recommended next step" +agent: "agent" +model: "Claude Fable 5" +--- + +> Superseded note (2026-07-04): This pickup prompt records the 2026-07-02 state only. Do not use it to start Iteration 0. Use `docs/plans/2026-07-04-rie-free-aspire-localstack-implementation-plan.md` and `docs/ROADMAP.md` for current routing. + +You are an engineer entering BadgeSmith right after a research-and-planning wave: +a full-codebase deep-dive audit, a measured performance investigation, and the +iteration 0 (AOT contract-test tier) spec + implementation plan all landed on +2026-07-02 (`6c6f609` + one uncommitted plan file). The single most important state +observation: **no production code has changed** โ€” everything so far is findings, spec, +plan, and baselines. Execution of iteration 0 is approved in principle but has not +started; Deniz had not yet chosen subagent-driven vs inline execution when the session +ended. + +## First Principle + +> Treat every claim here as **current-as-of-authoring (2026-07-02 โ€” S1)** and verify +> against the live repo, git log, `Directory.Packages.props`, `ARCHITECTURE.md`, and +> canonical docs before acting. + +## What Just Happened + +### 1. AWS Health "provided.al2 deprecation" email triage (context for the day) + +The email that started the session does **not** concern this repo: git history proves +BadgeSmith never used `provided.al2` (CDK was born on `PROVIDED_AL2023`; deployed +`badge-smith-function` verified al2023/arm64 via AWS CLI). The flagged functions are +two abandoned 2023 demos in the personal account (`lambda-dotnet-function`, +`profile-service-demo`, eu-central-1, 0 invocations/30d). **Carry-forward, undecided:** +delete vs flip those two; also `couples-threapy-prod/dev` (nodejs8.10, public API +Gateway, ~45k scanner hits/month) awaits a teardown decision. AWS CLI: use +`--profile personal` (account 377140207735). + +### 2. Full-repo deep-dive โ†’ `docs/research/2026-07-02-code-review-findings.md` + +Severity-ordered findings with file:line refs. Headlines the next agent must know: + +| Finding | Where | +| --- | --- | +| HMAC `repoIdentifier` = `Owner/Repo/Repo/Branch` (Repo doubled, Platform missing) | `HmacAuthenticationService.cs:42` | +| Badge query builds GSI1PK from non-normalized case (writes are lowercase) | `TestResultsService.cs:86-93` | +| Build scripts default `linux-x64`, CDK expects arm64 zip | `scripts/build-lambda.*:5` | +| Nonce burned before signature validation; 500s leak `ex.Message` | findings ยง2 | +| Only routing is unit-tested; HMAC/features/ResponseHelper/real RouteTable = zero tests | findings ยง4 | + +Fix waves are defined in the doc (ยง6) and mirrored in `docs/ROADMAP.md` Backlog. +**Rule: contract tests (iteration 0) pin these bugs as current behavior; fixes are +Wave 1, after iteration 0.** + +### 3. Performance research (measured) โ†’ `docs/research/2026-07-02-performance-opportunities.md` + +Prod measurements (30d CloudWatch): init 105โ€“140 ms, **cold invoke 165โ€“680 ms** (the +`Lazy` graph defers AWS/TLS setup into the first billed invoke at ~0.29 vCPU), warm +1โ€“3 ms, memory 32โ€“49 MB/512 MB, 0.07 RPM, max-concurrency 3 (= three README badges +fetched in parallel โ†’ 3 parallel cold starts), 16% cold ratio. CloudFront: 5,683 +req/30d, only ~43% absorbed at edge, **52% are 4xx** and error responses carry no +Cache-Control. Decisions section is authoritative: do edge 404-caching + TTL split, +INIT warm-up, `TrimMode=full` + ILC knobs (each measured), result caching; rejected +provisioned concurrency, keep-warm pings, SnapStart (N/A), hand-rolled SIMD (BCL +already vectorized). Honest scorecard: architecture A, memory A-, cold-start execution +B-. + +### 4. Iteration 0 spec + implementation plan (superseded workstream) + +- Spec (approved): `docs/plans/2026-07-02-iteration0-aot-contract-tier-design.md` +- Plan (approved, **uncommitted at session end**): `docs/plans/2026-07-02-iteration0-aot-contract-tier-plan.md` + +Core idea: the Aspire dev loop runs the Lambda as JIT with `ENABLE_LOCALSTACK` โ€” the +shipped AOT binary is never tested. Iteration 0 builds a Testcontainers-native +contract tier running the real image (`provided:al2023` + RIE) against LocalStack + +WireMock (mock/real upstream switch via new `HTTP_NUGET_BASE_URL`/`HTTP_GITHUB_BASE_URL` +env overrides โ€” the only two production touches), a `perf-baseline.sh` harness +(latency + RSS + zip/binary size + mstat โ†’ dated JSON in `docs/research/baselines/`), +QEMU-free arm64 cross-compilation, and CI gate + nightly on `ubuntu-24.04-arm`. +13 tasks; Task 12 (CI) requires a fresh explicit approval; "test the tester" drill +(inject a missing `JsonSerializable`, prove the suite goes red) is an acceptance +criterion. Baseline ordering rule: infra tasks land โ†’ baseline recorded โ†’ only then +Wave 1+/perf iterations. + +### 5. Slopwatch baseline + +`.slopwatch/baseline.json` committed (22 pre-existing findings: 17ร— SW002, 3ร— SW005, +2ร— SW003 โ€” the SW003s are the intentional retry catches in +`ResilienceRetryHandler.cs:33,37`). Gate command: +`slopwatch analyze --fail-on warning --exclude "artifacts/**,**/bin/**,**/obj/**"`. + +### Verification this session + +Docs/planning only โ€” no build or `dotnet test` run. AWS claims verified live via CLI +(`--profile personal`). Slopwatch analyze verified clean against the new baseline. + +## Session Learnings (read these โ€” they cost real time to earn) + +- **`dotnet-diag:optimizing-dotnet-performance` agent hallucinated an entire report + with 0 tool calls** (fabricated paths/code). Before trusting any subagent output, + check its tool-use count and spot-verify one cited file:line. Prefer Explore agents + for code-grounded work; treat specialist-agent output as hypotheses. +- **Subagents must be told explicitly which skills to load** (Deniz requirement). The + spec's "Required capabilities" section is the per-task list; put it verbatim in + every subagent prompt. +- **Git Bash on Windows mangles AWS CLI args starting with `/`** (log group names): + prefix with `MSYS_NO_PATHCONV=1`. +- **.NET under qemu-user is unsupported** โ€” that's why local arm64 builds failed + historically. The fix is cross-compilation (`--platform=$BUILDPLATFORM` build stage), + not newer QEMU. A one-time timeboxed binfmt retry is in plan Task 11. +- **README badge URLs being lowercase is the only thing masking the GSI case bug** โ€” + don't "clean up" test URLs into mixed case and conclude the service is broken. +- The AWS Health email lists affected resources per **account**, not per repo โ€” check + deployed reality (`aws lambda list-functions`) before assuming the repo is at fault. + +## Current State You Should Assume Until Verified + +- **HEAD (`master`)**: `6c6f609` โ€” "docs: add deep-dive findings, perf research, + iteration 0 design and slopwatch baseline" +- **Worktree**: one untracked file โ€” `docs/plans/2026-07-02-iteration0-aot-contract-tier-plan.md` + (plus this pickup, if not yet committed). Committing them needs Deniz approval. +- **Tests**: not run this session (docs-only). Last known: routing suite green. +- **Active workstream**: Iteration 0 โ€” plan approved, **execution not started**; + open question to Deniz: subagent-driven (recommended) vs inline execution. +- **Local-only state**: Docker Desktop available; `badge-smith:local` image NOT built + yet; slopwatch 0.4.2 installed as a global dotnet tool; AWS `personal` profile has + working static credentials. +- **Undecided AWS cleanup**: two `provided.al2` demo lambdas + couples-threapy stack + (see ยง1) โ€” surface when relevant, don't act without a decision. + +## Historical Recommended Next Step + +These were the next steps as of 2026-07-02. They are superseded by the 2026-07-04 RIE-free plan and are retained only for context. + +1. **Execute iteration 0** (multi-session arc, the default). Pre-flight: read the spec + + plan (grounding list below), confirm Docker is up, then ask Deniz the pending + question โ€” subagent-driven vs inline โ€” and start at Task 1 (RIE spike). Follow the + plan's Global Constraints verbatim (slopwatch per task, CPM via CLI only, pin + current behavior, Task 12 approval stop). Acceptance: plan tasks checked off through + Task 10 (baseline recorded) at minimum; Tasks 11โ€“13 complete the wave. +2. **Wave 1 correctness fixes** (well-scoped, only AFTER iteration 0's suite + baseline + exist) โ€” findings doc ยง1โ€“2; each fix updates the pinned contract assertions in the + same change. +3. **AWS account cleanup decision** (lightweight, optional) โ€” the ยง1 carry-forward; + needs Deniz's pick, then two CLI calls. + +Talk to Deniz before committing to which one. Default to working on `master`; do not +branch unless there is a concrete reason. No commit without explicit +"go / apply / proceed / baลŸla / yap" (AGENTS.md approval gate). + +## Historical Grounding Only + +Do not use this archived prompt as current routing. If you need the historical context, +read the current sources first, then treat the 2026-07-02 RIE documents as superseded +background only: + +1. `AGENTS.md` โ€” canonical contract: approval gate, AOT/Lambda constraints, capability + routing (`CLAUDE.md` is relay-only). +2. `docs/ROADMAP.md` โ€” current backlog and active workstream routing. +3. `docs/plans/2026-07-04-rie-free-aspire-localstack-implementation-plan.md` โ€” current + RIE-free Iteration 0 implementation plan. +4. `docs/plans/2026-07-02-iteration0-aot-contract-tier-design.md` and + `docs/plans/2026-07-02-iteration0-aot-contract-tier-plan.md` โ€” superseded RIE-based + historical design and task list. +5. `docs/research/2026-07-02-code-review-findings.md` + + `docs/research/2026-07-02-performance-opportunities.md` โ€” findings and perf + decisions the original plan built on. +6. `docs/research/baselines/2026-07-04-localstack-smoke.json` โ€” current LocalStack smoke + baseline from the RIE-free benchmark harness. +7. `docs/agents/README.md` (capability mapping โ€” resolve skill names here) + + `docs/agents/KNOWN_ISSUES.md`. +8. `ARCHITECTURE.md` / `README.md` as needed for endpoint behavior. + +## Historical Policy Recap + +This section records the 2026-07-02 session state. Current policy comes from +`AGENTS.md` and the active RIE-free plan. + +- No commit without explicit "go / apply / proceed / baลŸla / yap". Conventional + Commits; **no AI attribution trailers**. +- No feature/refactor/build/CI/CDK mutation without approval. Plan-internal commits + are pre-approved by the plan, **except Task 12 (CI workflows) โ€” fresh approval**. +- Package versions only via `dotnet add package` (CPM); never hand-edit versions. +- `slopwatch analyze --fail-on warning --exclude "artifacts/**,**/bin/**,**/obj/**"` + after every code change (baseline is committed โ€” only NEW slop fails). +- Native AOT discipline: no reflection, JSON types registered in + `LambdaFunctionJsonSerializerContext`, trim/AOT warnings blocking, UTC only. +- Contract tests pin CURRENT behavior (bugs included) โ€” production bug fixes belong to + Wave 1, not iteration 0. +- Subagent prompts must explicitly list the skills to load (spec "Required + capabilities" section). +- Tests are xUnit v3 on VSTest โ€” plain `dotnet test` / `--filter`. + +## Historical Steering Note + +This session converted an inbox scare (a deprecation email that turned out not to be +ours) into the project's first complete map: verified bugs, measured performance +truth, and an approved plan for the safety net that must exist before anyone touches +`TrimMode`. This steering note is superseded by the 2026-07-04 RIE-free plan and is +retained only to explain the earlier handoff state. diff --git a/docs/agents/handover-prompts/s2-rie-free-iteration0-complete.prompt.md b/docs/agents/handover-prompts/s2-rie-free-iteration0-complete.prompt.md new file mode 100644 index 0000000..fc65bf1 --- /dev/null +++ b/docs/agents/handover-prompts/s2-rie-free-iteration0-complete.prompt.md @@ -0,0 +1,109 @@ +--- +name: "S2 RIE-free iteration 0 complete" +description: "Priming prompt for the next agent entering BadgeSmith after RIE-free Iteration 0 landed on feature/iteration0-aot-contract-tier on 2026-07-05. Contract tests, local LocalStack benchmark harness, and live Gateway/CloudFront smoke baselines are recorded. Recommended next: choose and execute the first Wave 1 correctness fix." +argument-hint: "Optional focus area, such as HMAC, GSI case normalization, build-script RID alignment, or perf follow-up" +agent: "agent" +model: "gpt-5.5" +--- + +You are an engineer entering BadgeSmith after the RIE-free Iteration 0 work closed on +`feature/iteration0-aot-contract-tier`. The most important state observation: active +contract and benchmark paths no longer use Lambda RIE; local contract coverage runs +through Aspire Testing and `APIGatewayEmulator`, while the local performance harness uses +LocalStack ZIP Lambda execution with a Function URL fallback. + +## First Principle + +> Treat every claim here as **current-as-of-authoring (2026-07-05 - S2)** and verify +> against the live repo, git log, `Directory.Packages.props`, `ARCHITECTURE.md`, and +> canonical docs before acting. + +## What Just Happened + +### RIE-free Iteration 0 landed + +| Area | Current state | +| --- | --- | +| Contract tests | `tests/BadgeSmith.Api.Tests` starts `src/BadgeSmith.Host` through Aspire Testing and calls `APIGatewayEmulator` over HTTP. | +| RIE removal | `LambdaRieClient` and RIE-based stack fixture paths were removed. RIE references remain only in superseded docs or historical research. | +| Emulator culture fix | `src/BadgeSmith.Host/Program.cs` starts `APIGatewayEmulator` with invariant/C culture so `If-None-Match` does not become Turkish dotless-`i` under culture-sensitive lowercasing. | +| Local benchmark | `scripts/perf-baseline.sh` deploys the LocalStack local performance CDK stack, seeds test data, and runs k6 against a Lambda Function URL fallback when API Gateway v2 CloudFormation is unavailable in LocalStack Community. | +| CDK shape | Shared constructs preserve production API Gateway/CloudFront behavior while adding a LocalStack-only performance stack. | + +Primary commits to inspect: + +- `2827531` - `test: replace RIE harness with Aspire and LocalStack` +- `9ff91ac` - `docs: record live gateway baseline and Aspire agent notes` + +### Verification evidence + +Commands verified during the closing session: + +- `dotnet build --configuration Release` - passed, 0 warnings/errors. +- `dotnet test "tests/BadgeSmith.Api.Tests/BadgeSmith.Api.Tests.csproj" --filter "Category=Unit"` - 292/292 passed. +- `dotnet test "tests/BadgeSmith.Api.Tests/BadgeSmith.Api.Tests.csproj" --filter "Category=Functional"` - 28/28 passed. +- `dotnet test "tests/BadgeSmith.Api.Tests/BadgeSmith.Api.Tests.csproj" --filter "Category!=AotContract"` - 320/320 passed. +- `node --check scripts/k6-perf-test.js` - passed. +- `bash -n scripts/perf-baseline.sh` - passed. +- `slopwatch analyze --fail-on warning --exclude "artifacts/**,**/bin/**,**/obj/**"` - 0 issues. + +### Baselines now recorded + +| Baseline | File | Key result | +| --- | --- | --- | +| LocalStack final smoke | `docs/research/baselines/2026-07-04-final-localstack-smoke.json` | k6 p95 235.46 ms, peak Lambda worker RSS 20.279 MB. | +| Live direct API Gateway smoke | `docs/research/baselines/2026-07-04-live-gateway-smoke.json` | 64 client requests, 64 Lambda REPORT lines, k6 p95 309.14 ms, Lambda p95 229.96 ms, max memory 49 MB, 1 cold start. | +| Live CloudFront comparison smoke | `docs/research/baselines/2026-07-04-live-cloudfront-smoke.json` | 53 client requests, 25 Lambda REPORT lines, k6 p95 253.06 ms, Lambda p95 194.07 ms, max memory 51 MB, 1 cold start. | + +Use the direct API Gateway baseline for Lambda/API measurements. Use the CloudFront +baseline only as an edge-cache comparison because CloudFront can serve requests without +invoking Lambda. CloudWatch Lambda `REPORT` lines are the source of truth for memory, +duration, and cold starts. + +## Current State You Should Assume Until Verified + +- **Branch:** `feature/iteration0-aot-contract-tier`. +- **HEAD before this handover edit:** `9ff91ac` - `docs: record live gateway baseline and Aspire agent notes`. +- **SDK:** `global.json` pins .NET SDK `10.0.100` with `latestFeature` roll-forward. +- **Pinned packages:** `Aspire.Hosting.AppHost` / `LocalStack.Aspire.Hosting` / `Aspire.Hosting.Testing` `13.1.0`; AWS SDK v4 packages; `xunit.v3` `3.2.1` on VSTest. +- **Tests:** last full non-AOT verification was green as listed above; rerun the relevant slice before changing behavior. +- **Local-only artifacts:** k6 summaries live under ignored `artifacts/`; the durable baseline JSON files are under `docs/research/baselines/`. +- **AWS profile for live checks:** `aws --profile personal --region eu-central-1`. + +## Recommended Next Step + +1. **Wave 1 correctness fix - HMAC repo identifier** (well-scoped, high impact). Pre-flight: load the relevant debugging/TDD/.NET skills, read `docs/research/2026-07-02-code-review-findings.md` ยง1, inspect `src/BadgeSmith.Api/Core/Security/HmacAuthenticationService.cs`, and add/update tests that pin the corrected identifier shape. Acceptance: tests fail before the fix, pass after, and contract expectations are updated only for the intended bug fix. +2. **Wave 1 correctness fix - test-result GSI case normalization** (well-scoped, high impact). Pre-flight: inspect `TestResultsService` read/write key construction and existing functional tests. Acceptance: mixed-case public routes resolve the same stored data as lowercase routes, without weakening current HMAC or route tests. +3. **Performance pass follow-up** (multi-session arc). Pre-flight: read `docs/research/2026-07-02-performance-opportunities.md` and the three baseline JSON files. Acceptance: each change has before/after measured data and does not use CloudFront comparison numbers as the Lambda/API baseline. + +Talk to Deniz before committing to which one. Default to the current branch unless there +is a concrete reason to isolate work. No commit without explicit "go / apply / proceed / +baลŸla / yap" and a proposed Conventional Commit message. + +## Mandatory Grounding + +1. `AGENTS.md` - canonical repository contract: approval gate, AOT/Lambda constraints, + Aspire MCP/context7 guidance, and capability routing. +2. `docs/ROADMAP.md` - current backlog and Status & Plan Mapping table. +3. `docs/plans/2026-07-04-rie-free-aspire-localstack-implementation-plan.md` - executed Iteration 0 plan. +4. `tests/BadgeSmith.Api.Tests/README.md` - current test categories and RIE-free contract-test shape. +5. `docs/research/baselines/2026-07-04-live-gateway-smoke.json` and + `docs/research/baselines/2026-07-04-live-cloudfront-smoke.json` - live measurement context. +6. `docs/research/2026-07-02-code-review-findings.md` - Wave 1/2/3 backlog source. +7. `ARCHITECTURE.md` and `README.md` as needed for endpoints and runtime shape. + +## Locked Policy Recap + +- `AGENTS.md` is canonical; harness relays are adapters. +- No production bug fix, feature, refactor, build/CI/CDK mutation, deploy, push, or PR without approval. +- Package versions live in `Directory.Packages.props`; use `dotnet add/remove/list`, not manual package-version edits. +- Native AOT discipline stays active: source-generated JSON, no reflection-heavy shortcuts, no DI container, UTC-only time APIs, warnings as errors. +- Tests are xUnit v3 on VSTest. Use standard `dotnet test --filter`; do not use TUnit or MTP-only filter syntax. +- Run Slopwatch after LLM-authored code/test changes when available. + +## Final Steering Note + +Iteration 0 is now a safety-net foundation, not a feature endpoint. The next useful move +is to take one Wave 1 correctness fix at a time, prove the current behavior with a failing +test, fix the bug minimally, then rerun the direct slices. Keep performance changes +measured and keep CloudFront comparison data separate from Lambda/API baseline data. diff --git a/docs/agents/skills/aspire-source-navigation.md b/docs/agents/skills/aspire-source-navigation.md new file mode 100644 index 0000000..af413d4 --- /dev/null +++ b/docs/agents/skills/aspire-source-navigation.md @@ -0,0 +1,169 @@ +--- +name: aspire-source-navigation +description: Use when compatibility-sensitive Aspire.Hosting.LocalStack work depends on Aspire/AWS/LocalStack upstream source, package-version alignment, AddLocalStack/UseLocalStack/WithReference behavior, endpoint/configuration flow, or AWS SDK/LocalStack.Client wiring. +--- + +# Aspire Source Navigation + +## Overview + +This repository maintains an Aspire hosting integration package. Compatibility-sensitive work depends on this repo's package versions and on matching upstream source checkouts, not on memory or upstream default branches. + +Use source evidence before editing compatibility-sensitive code. Keep this skill version-light: package versions and concrete refs belong in `Directory.Packages.props`, local `external/` checkouts, and the upstream repositories. + +The expected outcome is a short evidence trail: exact package versions, verified upstream refs or an explicit missing-source note, source locations checked, and the compatibility conclusion that drives the change. + +## When To Use + +Use this skill for work involving: + +- `Aspire.Hosting` or `Aspire.Hosting.AWS` internals +- LocalStack.Client behavior +- `Directory.Packages.props` Aspire, AWS integration, or LocalStack client versions +- `AddLocalStack`, `UseLocalStack`, `.WithReference(localstack)`, endpoint/configuration flow, manifest behavior, CloudFormation/CDK, Lambda, or AWS SDK wiring +- String-based references to upstream AWS Aspire integration types +- Reviews of Aspire hosting compatibility or package-version drift + +For read-only explanation questions, use this skill only when the answer depends on upstream source, version-specific API shape, or a compatibility conclusion. Otherwise inspect this repository's docs/code directly. + +Do not use this skill for ordinary Markdown edits, general C# cleanup, or playground-only work that does not depend on Aspire/AWS/LocalStack internals. + +## Required Workflow + +1. Read `Directory.Packages.props` and identify the exact package versions involved. +2. Map the packages to their upstream repositories: Aspire packages to `dotnet/aspire`, `Aspire.Hosting.AWS` to `aws/integrations-on-dotnet-aspire-for-aws`, and LocalStack packages to `localstack-dotnet/localstack-dotnet-client`. +3. Check whether a matching local checkout exists under `external/`. Because `external/` is gitignored, use an ignored-file-aware check such as `Test-Path external`, `git ls-files --others --ignored --exclude-standard external/`, or a direct directory listing. Do not rely on workspace glob/search tools that skip ignored paths. +4. Verify the local checkout's branch/tag/commit against the package version and upstream tags/releases before trusting it. +5. If local source is missing or stale, report that explicitly. Use GitHub MCP only for tag/ref discovery, release verification, or targeted fallback reads. +6. Search upstream source for the exact symbols, annotations, extension methods, and behavior involved in the task. Do not rely on pre-baked search terms. +7. Cross-check this repository's implementation and tests against the verified upstream source. +8. Report evidence with file paths and refs before recommending or making changes. + +## Package-To-Source Map + +Resolve package versions from `Directory.Packages.props` each time. Do not copy versions into this skill. + +| Package or behavior | Upstream source | Local checkout root | +| --- | --- | --- | +| `Aspire.Hosting`, `Aspire.Hosting.AppHost`, `Aspire.Hosting.Testing` | `dotnet/aspire` | `external/aspire/{ref}/` | +| `Aspire.Hosting.AWS`, CloudFormation, CDK, Lambda emulator integration | `aws/integrations-on-dotnet-aspire-for-aws` | `external/aws-integrations/{ref}/` | +| `LocalStack.Client`, `LocalStack.Client.Extensions`, `ILocalStackOptions`, session/config options | `localstack-dotnet/localstack-dotnet-client` | `external/localstack-dotnet-client/{ref}/` | + +When a task spans multiple packages, verify every involved source. Example: `UseLocalStack()` with Lambda SQS event sources usually involves this repository, `Aspire.Hosting.AWS`, and possibly LocalStack client configuration behavior. + +## Local Checkout Layout + +Use this layout when local source is available: + +```text +external/aspire/{ref}/ +external/aws-integrations/{ref}/ +external/localstack-dotnet-client/{ref}/ +``` + +`{ref}` should be derived from the package version and verified against upstream tags/releases. The `external/` tree is ignored by git. Do not commit upstream source checkouts. + +## Ref Verification + +Before trusting local upstream source: + +1. Read the package version from `Directory.Packages.props`. +2. Inspect the local checkout's current ref using git metadata. +3. Verify that ref against upstream tags, release branches, or commits for the package version. +4. If the mapping is not obvious, say so and use a targeted upstream lookup to establish the mapping. + +Acceptable evidence includes a tag name, release branch, commit SHA, or upstream release page that ties the package version to the source. Unacceptable evidence includes repository default branches, approximate version names, or unchecked local folder names. + +### Resolving A Version To A Ref + +Upstream repositories do not all tag releases the same way. Determine the repository's release scheme first, then resolve the package version to a ref: + +- **Semver tags** (e.g. `vX.Y.Z`): match the package version directly to the tag. +- **Non-semver tags** (date-based, build-numbered, or otherwise): the version usually lives in the release notes, not the tag. Do not walk tags one by one. List releases and match the package version string in the release bodies, then take that release's tag and commit SHA. Releases are usually chronological, so a coarse search converges in a few lookups. + +A package's major version may be realigned to track another dependency, so a low major does not imply old source; rely on the resolved ref, not the version's shape. If a repository's scheme is unclear, inspect a couple of recent releases to learn it before resolving. Record each resolved version-to-ref mapping (tag plus SHA) so the lookup is not repeated. + +## Missing Or Stale Source + +If the matching local checkout does not exist after an ignored-file-aware check, do not silently continue with default-branch source. Report the gap before making compatibility-sensitive conclusions. + +Use this wording pattern: + +```text +Upstream source status: +- Aspire.Hosting {version}: no matching local checkout under external/aspire/{ref}; using targeted GitHub fallback for {symbols/files} only. +- Aspire.Hosting.AWS {version}: local checkout {path} verified at {ref-or-sha}. +- LocalStack.Client {version}: not involved in this change. +``` + +Create or refresh `external/` checkouts only when the user has approved that setup work or when the current task explicitly includes source setup. Keep those checkouts uncommitted. + +### Setting Up Checkouts + +When approved, create one checkout per resolved ref. Use a shallow clone to limit size: + +```bash +git clone --depth 1 --branch {ref} {repo-url} external/{name}/{ref} +``` + +- `{name}` is the checkout root from the package-to-source map; `{ref}` is the resolved tag, used verbatim (including non-semver forms). +- `external/` is gitignored, so these clones never enter repository status. Do not commit them. +- After cloning, confirm the checkout's `HEAD` matches the resolved commit SHA before trusting it. + +## Evidence Report + +Before recommending or making changes, provide the evidence in this shape: + +```text +Compatibility evidence: +- Package versions: Aspire.Hosting {version}, Aspire.Hosting.AWS {version}, LocalStack.Client {version-or-not-involved}. +- Upstream refs checked: {repo}@{ref-or-sha}, ... +- Upstream files/symbols checked: {file}:{symbol}, ... +- Repo files/tests checked: {file}:{symbol-or-test}, ... +- Conclusion: {what changed, what is compatible, what is risky, or what remains unverified}. +``` + +Keep the report short, but include enough detail that another agent can reproduce the source lookup. + +## Search Guidance + +Search by the exact behavior under review: + +- Extension methods: `AddLocalStack`, `UseLocalStack`, `WithReference`, `WithEnvironment`, `WaitFor`, `ExcludeFromManifest`. +- Aspire resource model: annotations, `IResourceWithEnvironment`, `IResourceWithWaitSupport`, endpoint references, connection string callbacks, manifest publishing. +- AWS integration: CloudFormation resources, CDK stacks/bootstrap, Lambda emulator resources, SQS event source resources, output/reference annotations. +- LocalStack client: `ILocalStackOptions`, `LocalStackOptions`, `SessionOptions`, `ConfigOptions`, `AddLocalStack`, `AddAwsService`, environment variable binding. + +These are starting points, not a fixed checklist. Add or remove searches based on the concrete task. + +## Official Aspire Skills Cross-Check + +Official Microsoft Aspire skills are useful for Aspire CLI and distributed application workflows, but they may describe newer Aspire versions than this repository uses. If official guidance conflicts with verified package source, prefer verified package source and call out the version mismatch. + +Use official skills when available for: + +| Task | Skill | +| --- | --- | +| AppHost lifecycle | `aspire` or `aspire-orchestration` | +| Logs, dashboard, traces | `aspire-monitoring` | +| AppHost scaffold/resource graph work | `aspireify` | +| New skeleton creation | `aspire-init` | +| Publish/deploy/destroy | `aspire-deployment`, approval-gated | + +Installed local skills can supplement this one: + +| Task | Skill | +| --- | --- | +| Explicit configuration and env vars | `aspire-configuration` | +| Playground ServiceDefaults | `aspire-service-defaults` | +| `DistributedApplicationTestingBuilder` patterns | `aspire-integration-testing`, adapted to this repo's test framework | + +## Common Mistakes + +- Do not use upstream default branches for compatibility-sensitive source checks. +- Do not assume package versions map directly to semver git tags; verify the upstream tag/release scheme. +- Do not copy examples from external testing guidance without adapting them to this repo's test framework. +- Do not commit `external/` source checkouts. +- Do not treat GitHub MCP as the default source-reading path when local source is available. +- Do not claim source compatibility from this repository's tests alone; upstream API shape must be checked for version-sensitive behavior. +- Do not leave the evidence trail implicit in chat history; summarize refs and file paths before the recommendation or edit. diff --git a/docs/plans/2026-07-02-iteration0-aot-contract-tier-design.md b/docs/plans/2026-07-02-iteration0-aot-contract-tier-design.md new file mode 100644 index 0000000..59438cd --- /dev/null +++ b/docs/plans/2026-07-02-iteration0-aot-contract-tier-design.md @@ -0,0 +1,190 @@ +# Iteration 0 Design โ€” AOT Contract-Test Tier, Baseline Harness, Multi-Arch Build + +> Superseded note (2026-07-04): RIE-dependent contract and benchmark paths are superseded by `2026-07-04-rie-free-aspire-localstack-redesign.md`. Keep this document for historical context only. + +Date: 2026-07-02 +Status: approved design, pending implementation plan + +## Context + +The Aspire dev loop runs the Lambda as a JIT/CoreCLR build with `ENABLE_LOCALSTACK` +defined โ€” the trimmed Native AOT binary that actually ships is never exercised by any +test today. Before the planned `TrimMode=full`/ILC iterations (see +[../research/2026-07-02-performance-opportunities.md](../research/2026-07-02-performance-opportunities.md)), +we need a safety net that runs the **real published artifact** and a **recorded +performance baseline** to diff every iteration against. Strategy decisions were agreed +in `docs/ROADMAP.md` (Inbox, 2026-07-02). + +Trim failures historically manifest as silent hangs, not exceptions โ€” the tier must +exercise every route and every response shape, not just happy paths. + +## Goals + +1. A contract-test suite that runs the production AOT artifact in the real Lambda base + image (`public.ecr.aws/lambda/provided:al2023`) via the Runtime Interface Emulator. +2. A repeatable performance/memory baseline harness with dated, committed results. +3. A Dockerfile that cross-compiles arm64 artifacts on an x64 host **without QEMU**. +4. CI wiring: contract suite as a deploy gate + nightly run on native arm64 runners. + +Non-goals (out of scope): any endpoint behavior change (two env-var overrides +excepted), TrimMode/ILC changes, Wave 1 bug fixes, unit-test expansion, Aspire loop +changes. + +## Architecture + +Orchestration choice: **Testcontainers-native** (decided). The xUnit fixture owns the +container lifecycle; the shell harness assembles the same stack with plain `docker` +CLI. The shared contract between the two is (a) the image tag and (b) the env-var +names below โ€” documented in one place (the test project README) to bound drift. + +``` +xUnit collection fixture (Testcontainers, shared Docker network) +โ”œโ”€โ”€ localstack/localstack DynamoDB + Secrets Manager +โ”œโ”€โ”€ wiremock/wiremock:3.x NuGet + GitHub API stubs (checked-in mappings) +โ””โ”€โ”€ badge-smith image prod artifact + RIE, env-pointed at the two above + โ–ฒ + โ”‚ POST /2015-03-31/functions/function/invocations (APIGW v2 event JSON) + test code (LambdaClient helper) โ€” asserts APIGW v2 response JSON +``` + +- The suite **does not build** the image. It consumes a prebuilt tag from + `BADGESMITH_TEST_IMAGE` (default `badge-smith:local`) and fails fast with a clear + message if missing. Building (5โ€“10 min AOT publish) belongs to the build script and + CI steps. +- The Lambda container runs the true production build: `ENABLE_TELEMETRY` and + `ENABLE_LOCALSTACK` off. RIE listens on container port 8080; the host port is + dynamically mapped by Testcontainers. +- Fixture creates the three DynamoDB tables (same names/keys/GSI as + `DynamoDbTablesConstruct`), writes the org-secret mapping row, and puts the HMAC + secret into LocalStack Secrets Manager. Test data is seeded per test class. + +### Environment plumbing (the two production touches) + +1. **AWS endpoints.** Spike first: verify AWS SDK for .NET v4 honors + `AWS_ENDPOINT_URL_DYNAMODB` / `AWS_ENDPOINT_URL_SECRETS_MANAGER` (expected: yes โ€” + zero code change). Fallback if not: add an env-based `ServiceURL` override to the + production path of `AwsClientBuilder` (small, approval-gated edit). +2. **Upstream base URLs.** `HttpClientFactory` hardcodes `https://api.nuget.org/` and + `https://api.github.com/`. Add optional env overrides `HTTP_NUGET_BASE_URL` and + `HTTP_GITHUB_BASE_URL` (fall back to today's constants). This implements the + mock/real switch: contract tests always point at WireMock; the perf harness selects + via `BADGESMITH_UPSTREAM=mock|real` (decided: both modes supported โ€” perf numbers + are wanted against both). + +### WireMock stubs + +Mappings live in `tests/BadgeSmith.Api.ContractTests/wiremock/` and are mounted into +the container. They are **recorded** from the real NuGet/GitHub APIs once (WireMock +recording mode), then sanitized and checked in โ€” not hand-invented. Minimum set: +NuGet flat-container index (large multi-version package, prerelease-only package, +404), GitHub package versions (happy, 401, 403, 404, empty). + +## Coverage matrix + +All six routes through the real binary, asserting status, headers, and full body +shape (any missing `JsonSerializable` registration must surface): + +| Area | Cases | +| --- | --- | +| Health | 200 + no-cache headers | +| NuGet badge | ok, `?version=` range, `?prerelease=`, 404 package, 400 invalid range, ETagโ†’304 | +| GitHub badge | ok, secret missingโ†’401, upstream 401/403 mapping, ETagโ†’304 | +| Test results badge | ok, 404 no data, ETag/304, Last-Modified | +| Ingestion (HMAC) | full signed round-trip (storeโ†’badge read-back), bad signatureโ†’401, stale/future timestampโ†’400, nonce replayโ†’400, malformed hex signature, missing headers | +| Redirect | 302 + Location + cache headers | +| Routing/CORS | unknown route 404, OPTIONS preflight, HEADโ†’GET | + +Contract tests pin **current** behavior, including known bugs โ€” e.g., a malformed hex +`X-Signature` returns 500 today and the test asserts 500 with a comment referencing +findings ยง2. Wave 1 fixes update the pinned assertions in the same change as the fix. +The suite is a trim-regression net, not a bug-fix vehicle. + +**"Test the tester" acceptance criterion:** on a scratch branch, remove one +`JsonSerializable` registration, rebuild the image, and prove the suite goes red. +Iteration 0 is not done until this is demonstrated. + +## Multi-arch build fix + +- Build stage becomes `FROM --platform=$BUILDPLATFORM mcr.microsoft.com/dotnet/sdk:โ€ฆ` + with clang cross toolchain + arm64 sysroot packages; `TARGETARCH` maps to the RID. + ILC cross-compiles natively on the host โ€” no emulation in the build path. +- `scripts/build-lambda.*` default RID: **linux-arm64 for release artifacts** (closes + findings ยง1.3 โ€” aligns with the CDK's hardcoded arm64 expectation); local test image + defaults to host arch (amd64). +- Timeboxed experiment (once, ~30 min): refresh binfmt (`tonistiigi/binfmt`) and try + running the arm64 image locally. If it works, local contract tests may run arm64; + if not, amd64 stands (trim-failure class is arch-independent) and the outcome is + documented. No insistence โ€” decided. + +## Baseline harness + +`scripts/perf-baseline.sh` is the single source of truth; `perf-baseline.ps1` is a +thin wrapper that delegates (avoids the known ps1/sh drift). Prerequisite fix folded +in: `k6-perf-test.js` reads `__ENV.K6_API_URL` (and `K6_DURATION`/`K6_VUS`) as its +README already claims. + +Stages: build image โ†’ record binary/zip sizes + `IlcGenerateMstatFile` output โ†’ boot +the stack (upstream toggle) โ†’ k6 scenario against the RIE/emulated endpoint โ†’ +`docker stats` sampling (idle + under load) โ†’ write +`docs/research/baselines/YYYY-MM-DD-