Add dotnet-aspnetcore REST API authoring skills (controllers + Minimal APIs)#848
Draft
javiercn wants to merge 19 commits into
Draft
Add dotnet-aspnetcore REST API authoring skills (controllers + Minimal APIs)#848javiercn wants to merge 19 commits into
javiercn wants to merge 19 commits into
Conversation
…logic structure Replaces the prior dotnet-aspnetcore skills (dotnet-webapi, minimal-api-file-upload, configuring-opentelemetry-dotnet, convert-blazor-server-to-webapp) with a focused REST API skill set. Adds two skills with Vally evals over a shared e-commerce store fixture (1:1, 1:M, M:N relationships): - author-minimal-api-endpoints: typed Results<...>/TypedResults, correct status codes (201+Location, 204, 404, 400), input validation via ProblemDetails. - structure-api-business-logic: thin endpoints over a service layer that returns a hand-rolled Result<T> with a semantic ErrorType; endpoint translates it to HTTP; service takes DTOs/commands with manual mapping and never returns ASP.NET result types. Both validated baseline-vs-skilled over 3 runs (endpoints 3.17->4.92/5, structure 2.56->5.00/5; no regressions). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Optimistic concurrency + HTTP conditional requests for minimal APIs: concurrency token -> ETag, If-None-Match (304), If-Match (412)/428, and DbUpdateConcurrencyException handling. Baseline 3.89 -> 5.00 over 3 runs; the value concentrates in the If-Match conditional-update gap (1.67 -> 5.00). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Controller-based Web API actions with correct result types and status codes: ControllerBase + [ApiController], ActionResult<T> with typed helpers, CreatedAtAction (201+Location), 204/404, [ProducesResponseType] for discoverability, and data-annotation validation (auto-400). Baseline 3.89 -> 4.67 over 3 runs (+16%), no regressions. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
End-to-end integration testing with WebApplicationFactory<Program>: in-memory host + HttpClient, making Program reachable, replacing the DbContext with an isolated per-factory database, per-test seeding, and asserting status + payload. Baseline 4.33 -> 4.89 over 3 runs (+11%); value concentrated in per-test seeding discipline. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Teaches optimistic concurrency and HTTP conditional requests for controller actions, exposing both validators a resource offers: an ETag from the concurrency token (If-Match/If-None-Match) and Last-Modified from the last-modified timestamp (If-Modified-Since/If-Unmodified-Since), with 304/412/428 handling and DbUpdateConcurrencyException mapping. Natural-prompt eval (MessagingApi fixture, 3 runs/model): sonnet 3.0 -> 5.0 gpt-5.4 2.5 -> 4.5 Both gap criteria (Last-Modified validator; ETag-vs-content distinction) closed on both models with no regression. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Adds the Last-Modified time validator (If-Modified-Since/If-Unmodified-Since) alongside the existing ETag content validator, a ConditionalRequest helper that reads both off the entity, ordinal header comparisons, 304/412/428 handling, and DbUpdateConcurrencyException mapping. Rebuilds the eval on the MessagingApi domain (minimal-API-wired) with natural prompts and dual-validator rubrics. Natural-prompt eval (3 runs/model): sonnet 2.5 -> 5.0 gpt-5.4 2.83 -> 4.33 Last-Modified validator gap (baseline ~1 on both) closed on both models. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Replaces the StoreApi fixture and cued prompts with natural business-logic scenarios (multi-entity rules: create-queue with parent/unique/quota checks; decommission-namespace cascade) and convention-checking rubrics for service extraction, a Result-with-error-kind outcome (no exceptions/tuples across the service boundary), HTTP-agnostic services, and thin endpoints. Natural-prompt eval (3 runs/model): sonnet 3.0 -> 5.0 gpt-5.4 3.5 -> 4.83 Weak baseline criteria (logic-in-service, thin endpoint, result-value) closed on both models, no regression. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Both author-controller-endpoints and author-minimal-api-endpoints cleanly help both models once evaluated on their actual scope (routing, status codes, typed results, operation exposure) rather than bundled domain-modeling rubrics that the models already handle and that caused attention-theft regressions. Skill fixes: author-controller delete example no longer cues a hard delete (soft-delete-aware), and braceless control flow replaced with Allman braces in both skills. Rebuilt evals on the MessagingApi domain (controller-wired and minimal-wired) with natural prompts and endpoint-only rubrics. Natural-prompt eval (3 runs/model): author-controller-endpoints sonnet 3.56 -> 4.67, gpt-5.4 3.78 -> 4.67 author-minimal-api-endpoints sonnet 2.67 -> 5.0, gpt-5.4 3.17 -> 4.33 No criterion regresses on either model. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…oints Both controller-data-access and minimal-api-data-access teach paginating every collection endpoint with a bounded page size, ordering before Skip/Take in the query, reading with AsNoTracking, and projecting to a DTO in the query. The natural-prompt baseline showed pagination is the one real gap (the models already do no-tracking and projection): a list endpoint returns the whole table unless paged, most acutely on sonnet (pagination criterion ~1/5). Natural-prompt eval (3 runs/model): controller-data-access sonnet 3.33 -> 5.0, gpt-5.4 4.83 -> 5.0 minimal-api-data-access sonnet 3.5 -> 5.0, gpt-5.4 4.33 -> 5.0 No criterion regresses on either model. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Each test now owns its WebApplicationFactory (xUnit constructs the test class per method), so every test gets a fresh in-memory database and cannot see other tests' writes; IClassFixture is reserved for read-only or uniquely-keyed sharing. This removes the isolation regression a class-shared factory caused. Rebuilt the eval on the MessagingApi domain with an endpoints-implemented fixture (NamespacesController) and a de-cued natural prompt. Natural-prompt eval (5 runs/model): sonnet 4.4 -> 5.0 (isolation 3.8 -> 5.0) gpt-5.4 4.6 -> 4.8 (DbContext replacement 4.2 -> 5.0) The skill closes the genuine DbContext-replacement and isolation gap. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
Skill Coverage Report
Uncovered:
|
…ve vs imperative) Teaches authorizing API endpoints through the authorization framework instead of inline checks, choosing the mechanism by where the decision data lives: declarative policy + AuthorizationHandler reading context.Resource as HttpContext (route values + endpoint metadata) for route/claim-derivable rules, and imperative IAuthorizationService.AuthorizeAsync with the loaded entity for ownership/state rules. Covers requirement+handler mechanics, OR-semantics (Succeed never Fail), named policies, a fallback policy, 401/403/404, DI lifetimes, and middleware order. The natural-prompt baseline showed both models default to inline if-checks and do not reach for the framework (the declarative-vs-imperative distinction scored ~1). Natural-prompt eval (3 scenarios, 3 runs/model): sonnet 2.33 -> 4.56 gpt-5.4 1.89 -> 3.78 No criterion regresses on either model. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ills Two minimal-API gaps that the de-cued natural-prompt baseline exposed (cued prompts had hidden them by naming the technique): - minimal-api-parameter-binding: bind a custom/value type from the route or query via a static TryParse or BindAsync so parsing and validation happen at the boundary and invalid input is 400 before the handler runs, instead of taking a raw string and validating inline. - minimal-api-endpoint-filters: apply cross-cutting validation and required-header checks as endpoint filters on a route group (short-circuiting with a 400), instead of duplicating the checks in every handler. Natural-prompt eval (3 runs/model): minimal-api-parameter-binding sonnet 2.0 -> 5.0, gpt-5.4 2.0 -> 5.0 minimal-api-endpoint-filters sonnet 2.33 -> 5.0, gpt-5.4 2.33 -> 5.0 No criterion regresses on either model. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Endpoint filters earn their place only when a concern (1) needs the bound, already-validated arguments or the handler result, and (2) is not already covered by a built-in mechanism with the same access. Plain input validation fails test 2 (built-in AddValidation covers it), so the skill no longer teaches a validation filter. The two examples now exercise exactly what nothing else can: rewriting a bound argument before the handler (normalize a value) and changing the returned result (field selection from a query value). Adds a When-not-to-use-a-filter section routing validation to AddValidation, cross-cutting request/response concerns to middleware, authz to policies, exceptions to IExceptionHandler, and response caching to output caching. Eval rebuilt around the arg-rewrite + result-shape scenario. Natural-prompt eval (3 runs/model): sonnet 1.0 -> 5.0 gpt-5.4 2.0 -> 5.0 No criterion regresses on either model. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Two gaps confirmed by de-cued natural-prompt probing on both models: - rate-limiting: throttle with the built-in rate limiter (AddRateLimiter + UseRateLimiter) partitioned per caller (a claim/API key/IP), 429 with Retry-After, a deliberate algorithm. Both models get the concept right but hand-roll a counter instead of reaching for the built-in (that criterion 1/5). - long-running-operations: model slow work as an async operation - accept with 202 and a Location to a distinct operation-status resource the client polls through a running/succeeded/failed lifecycle, rather than blocking the request. Both models do async 202 but omit the separate operation-status resource. Natural-prompt eval (3 runs/model): rate-limiting sonnet 2.0 -> 4.67, gpt-5.4 2.0 -> 4.33 long-running-operations sonnet 3.0 -> 5.0, gpt-5.4 3.67 -> 5.0 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…Link header The shipped pagination guidance was incomplete and its example unstable: it ordered by a non-unique column (Name) with Skip/Take, which repeats or skips rows when the sort field ties or the collection changes between requests, and it returned a page-number envelope. Both controller-data-access and minimal-api-data-access now teach: - a deterministic total order ending in a unique key (ThenBy the id), - keyset/cursor paging (WHERE (sortKey, id) > (lastSortKey, lastId)) instead of Skip/OFFSET, seeking on an immutable key, - the next page delivered as an RFC 8288 Link header (rel=next) with the body as the bare collection. Natural-prompt eval, upgraded to a robust-pagination scenario (3 runs/model): controller-data-access sonnet 3.67 -> 5.0, gpt-5.4 4.0 -> 4.83 minimal-api-data-access sonnet 4.0 -> 5.0, gpt-5.4 3.67 -> 4.83 The Link-header criterion went 1 -> 5 on both models; no criterion regresses. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Teaches an incremental change-tracking (delta) endpoint: after an initial sync a client fetches only what changed - added, updated, removed - since a watermark. Covers a monotonic change marker (a rowversion mapped order-preserving as a comparable number, with an app-maintained timestamp fallback that overlaps and dedups when no rowversion exists), soft-delete tombstones, a total order over (marker, id), the in-flight-commit boundary hazard (MIN_ACTIVE_ROWVERSION), and a multipart/mixed of application/http response (200 present / 410 removed) that keeps entity payloads unchanged, with the change-tracking link in the Link header (rel=deltaLink) and 410 on a stale token. No Atom. Natural-prompt eval, two scenarios (rowversion available / not), 3 runs/model: sonnet 2.17 -> 4.33 gpt-5.4 2.17 -> 4.0 Multipart representation, Link-header token, and no-changes gaps (~1-2 baseline) close to ~5; no criterion regresses. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Two gaps that only surfaced after de-cuing the probe prompts (the cued versions had hidden them): - structured-logging: message templates with named placeholders (not interpolation), a scope (BeginScope) to correlate an operation's lines, levels by severity, exceptions passed as the exception argument, no secrets/PII. - patch-partial-updates: PATCH that changes only the fields sent and distinguishes an explicit null (clear) from an omitted field (leave), via JSON Merge Patch over JsonElement/JsonNode (or JsonPatchDocument / a tri-state Optional), instead of a plain DTO where absent and null collapse. De-cued natural-prompt eval (3 runs/model): structured-logging sonnet 3.0 -> 5.0, gpt-5.4 3.0 -> 4.0 patch-partial-updates sonnet 3.67 -> 5.0, gpt-5.4 3.67 -> 4.67 No criterion regresses on either model. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Switch the tombstone from 410 Gone to a body-less 204 No Content so every multipart part stays in the success range (a DELETE-style outcome) - the client reads body-present as upsert and no-body as remove. 410 is kept only for an expired change-tracking token. Adds a rendered on-the-wire sample (200 upsert / 204 removed + deltaLink) and a note that 201 Created is worthwhile only when a creation marker is tracked separately from the last-change marker. Rubric accepts 204 or 410 for the tombstone. Natural-prompt eval, two scenarios, 3 runs/model: sonnet 2.17 -> 4.5 gpt-5.4 2.0 -> 4.17 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
design-collection-api: greenfield collection-API completeness + data-model reconciliation (stable order, pagination, filtering, delta-readiness, concurrency, status codes) routing to the per-capability skills. Eval: sonnet 2.33->4.67, gpt-5.4 3.00->5.00. filter-and-select: bounded, opt-in filtering (allow-list + bracket operators) and field selection (sparse fieldsets) translated to EF Where/Select, with type coercion and amplification guardrails. Eval: sonnet 3.33->4.33, gpt-5.4 4.00->4.33. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
ASP.NET Core REST API authoring skills (
dotnet-aspnetcore)Adds a disjoint set of eighteen skills that improve how the agent writes ASP.NET Core REST APIs with controllers and Minimal APIs. Each skill targets a genuine, measured gap that both a strong and a mid-tier model miss, and each ships with a convention-checking eval that proves it helps both without regressing either.
How these were built and measured
The skills were developed gap-first: for every candidate, the agent was first measured without the skill on natural, underspecified prompts (the kind a developer would actually write, with no hint of the convention), then with it. A skill is kept only when it helps both models with no meaningful regression; where the model is already strong unaided, no skill is added.
MessagingApi: a messaging namespace aggregate owning queues, topics, subscriptions, authorization rules, with soft delete, row versions, and last-modified timestamps). None of this domain appears in any skill body; skills stay domain-agnostic.Skills added (18) and eval results
Mean overall score (1-5), baseline → with-skill, natural prompts:
controller-concurrencyLast-Modifiedvalidator (If-Modified-Since/If-Unmodified-Since) alongside theETagboth models already didminimal-api-concurrencystructure-api-business-logicResultwith an error kind (no exceptions/tuples across the service boundary); thin endpointsauthor-controller-endpointsActionResult<T>, correct status codes,CreatedAtAction+Location,ProducesResponseType, operation exposureauthor-minimal-api-endpointsResults<>unions,TypedResults,MapGroup, 201+Location,ValidationProblemcontroller-data-accessLinkheader,AsNoTracking+ projectionminimal-api-data-accessLinkheader,AsNoTracking+ projectiontest-apis-with-webapplicationfactoryWebApplicationFactory, replace the registeredDbContext, isolate the database per test, assert status and bodyauthorize-api-endpointscontext.Resource(HttpContext) route/endpoint metadata for route/claim rules; imperativeAuthorizeAsyncwith the loaded entity for ownership; named policies, OR-semantics, 401/403/404, middleware orderminimal-api-parameter-bindingTryParse/BindAsyncso parsing and validation happen at the boundary and invalid input is 400 before the handler runsminimal-api-endpoint-filtersAddValidation, other cross-cutting work to middlewarerate-limitingAddRateLimiter/UseRateLimiter) partitioned per caller (claim/API key/IP), 429 +Retry-After, deliberate algorithm — not a hand-rolled counterlong-running-operations202 Accepted+ a distinct pollable operation-status resource (running/succeeded/failed), not a blocked requestchange-tracking-deltaMIN_ACTIVE_ROWVERSIONboundary,multipart/mixedofapplication/http(200 upsert / 204 removed) keeping payloads unchanged,Link: rel="deltaLink", 410 on stale tokenstructured-loggingBeginScopecorrelation, levels by severity, exception-as-argument, no secrets/PIIpatch-partial-updatesJsonElement/JsonNode(orJsonPatchDocument/ tri-stateOptional)design-collection-apifilter-and-selectWhere/Select:400on unknown field/operator, operands coerced to the field type, the keyset sort key kept in the projection so paging stays stable, and amplification guards (gate leading-wildcardContainsin favor of sargableStartsWith, cap predicate count and page size)No rubric criterion regresses on either model for any kept skill.
Note on the authorization skill
authorize-api-endpointswas added after a second, probe-driven round. Both models default to inlineif (User…)checks and do not reach for the authorization framework at all: the baseline for the framework criteria (requirement + handler, named-policy opt-in, imperativeAuthorizeAsyncfor loaded-entity rules,context.SucceedOR-semantics) scored ~1/5 on both models, making this the largest single gap found. The skill leads with the rule of thumb that decides the mechanism: where does the decision data live? Route values and claims are decided declaratively by a handler that readscontext.Resourceas theHttpContext(and custom endpoint metadata); stored entity state (anOwnerId) is decided imperatively withIAuthorizationService.AuthorizeAsyncafter loading.Note on de-cueing the prompts
minimal-api-parameter-bindingandminimal-api-endpoint-filterswere initially measured as "no gap" because their probe prompts cued the technique (for example "use a strongly typed value in the handler signature" or "apply this across the route group without repeating"). A cued prompt hands the baseline the same guidance the skill provides, so the baseline looked strong (5.0/5.0). Rewriting the prompts to describe only the user-facing goal collapsed the baselines to ~2.0, exposing genuine both-model gaps that the skills then close. Every prompt in these evals describes a goal, never a technique the skill teaches.Skills considered and not built (14)
Every candidate below was measured the same way (natural prompts, both models, LLM-judged rubric). A skill is kept only when it helps both models with no meaningful regression. These were dropped; this list is a record so they are not re-attempted without new evidence.
Already strong on both models — no gap to close
A skill here would only spend context tokens and risk regressing a model that is already at or near ceiling.
design-api-operationsdesign-collection-api, which covers cross-cutting collection completeness and data-model reconciliation, not resource modelingmodel-controller-payloadsmodel-minimal-api-payloadsoptions-validationIOptions<T>withValidateDataAnnotations/ValidateOnStart. Already appliedtransactions-outboxpolymorphic-json[JsonPolymorphic]/[JsonDerivedType]with an explicit discriminator. Already donehealth-checksAddHealthCheckswith liveness/readiness endpoints. Already doneopentelemetryhttp-header-forwardingForwardedHeadersmiddleware when running behind a reverse proxy. Already doneconsume-apis(.NET client)rel=nextto completion, persist the delta link and resync on410, sendIf-Matchand handle412, honorRetry-Afteron429, poll202, useIHttpClientFactory. Strong when the contract is spelled out and when it is only discoverable in an API-reference doc with a goal-only task (de-cued), so the high score is genuine competence, not cueingconsume-apis(JS client)fetchclient (AbortController, backoff,Linkparsing). Already strongAsymmetric or borderline — one skill cannot help both models
A real gap exists on only one model (the other is at ceiling), or the two models are weak on different halves of the problem, so a single skill cannot help both without regressing one.
http-resilienceMicrosoft.Extensions.Http.Resilienceretry, timeout, circuit breaker. A real GPT-5.4 gap, but Sonnet is at ceiling, so a skill risks regressing Sonnetapi-error-handlingProblemDetailsviaIExceptionHandler/UseExceptionHandler. A real Sonnet gap, but GPT-5.4 is already strong. Asymmetricef-bulk-query-shapingExecuteUpdate/ExecuteDeleteandAsSplitQueryto avoid cartesian explosion. Each model is strong on the half the other is weak on, so there is no consistent both-model gapCross-model validation (a third family: MAI-Code-1-Flash)
To check the skills are not overfit to the two frontier models they were tuned on, a spot-check ran a smaller, faster third-family model (MAI-Code-1-Flash) on a representative slate.
The skills lift the weaker model too (baseline → with skill, natural prompts):
authorize-api-endpointsminimal-api-parameter-bindingdesign-collection-apifilter-and-selectThe weaker model starts lower and gains the most where the gap is largest (authorization, parameter binding), so the guidance is not specific to frontier models.
The "no gap" drops are mostly robust across tiers, with one exception. Re-measuring the dropped candidates' baselines on MAI-Code-1-Flash:
polymorphic-jsontransactions-outboxmodel-minimal-api-payloadsdesign-api-operationsconsume-apis(.NET client)health-checksFive of six drops hold: the weaker model also handles those areas well, so they are genuinely no-gap rather than frontier-only. The exception is
health-checks— the frontier models add liveness/readiness checks unprompted, but MAI-Code-1-Flash mostly does not (2.33). It stays dropped for the models these skills target, but is recorded here as a weak-model-tier candidate: ahealth-checksskill would likely help smaller/cheaper models even though it is unnecessary for frontier ones.Notable findings baked into the skills
Last-Modifiedvalidators, the service +Resultstructure, bounded pagination, and per-test database isolation. Endpoint and payload basics are already strong on both models.test-apisprompt inflated the baseline to 5.0/4.67 and hid the real gap; a de-cued prompt exposed a real GPT-5.4 weakness in replacing and isolating theDbContext, which the skill then closes.author-controller-endpoints, and switchingtest-apisfrom a class-shared factory to a per-test factory so each test gets a clean in-memory database.Layout
plugins/dotnet-aspnetcore/skills/<name>/SKILL.mdtests/dotnet-aspnetcore/<name>/(eval.vally.yaml+ afixture/MessagingApiproject)Opening as a draft for review of the skill content and the eval methodology before finalizing.