diff --git a/docs/design/agent-mode.md b/docs/design/agent-mode.md index 5b316b3c..2026ab74 100644 --- a/docs/design/agent-mode.md +++ b/docs/design/agent-mode.md @@ -1,6 +1,6 @@ # Linq2GraphQL Agent Mode Design -**Status:** in progress. Plan steps 1-3 (the groundwork, SDL input and SDL output) are implemented. The AI package is being built in slices: (1) package skeleton and scoped schema, (2) renderer, (3) validator, (4) the four tools, (5) the `AsAgentTools` adapter, (6) sample and regression harness. Slices 1 to 5 are merged; slice 6 (the sample and regression harness) is in review. See [Plan](#plan-and-risks). +**Status:** in progress. Plan steps 1-3 (the groundwork, SDL input and SDL output) are implemented. The AI package is being built in slices: (1) package skeleton and scoped schema, (2) renderer, (3) validator, (4) the four tools, (5) the `AsAgentTools` adapter, (6) sample and regression harness. All six slices are merged, and the package ships as a preview (`-preview`, same version number as the core packages). Next: evaluation with the sample against a real model, then pilots. See [Plan](#plan-and-risks). ## Summary @@ -420,7 +420,7 @@ Subscriptions stay out entirely; they don't fit the request/response shape of to 1. **Core prerequisites** (done): the per-request hook and the schema cache-key fix. 2. **SDL input in the generator** (done): it introduced GraphQL-Parser. 3. **SDL output** (done): the `--agent` flag, `IGraphSchemaSource`, and derived clients for per-user tokens. `[Description]` attributes and the two extra introspection fields shipped with step 2. -4. **Preview and pilots:** the first version as a prerelease package (`-preview`), complete enough to run end to end, with a runnable sample against the test server. Usefulness depends on the use case, so it is evaluated in one or two pilot apps with a real schema and real users, ideally different ones (interactive and background, small and large schema). The evaluation is part of the product: `OnQuery` logs (questions, queries, validation errors, tool calls per answer), simple user feedback per answer in the pilot app, and which questions the team moves to dedicated tools. The pilots can start before step 3 by using an exported `schema.graphql`. The sample (`test/Linq2GraphQL.Agent.Sample`) runs a chat loop against the in-process test server, with Azure AI Foundry (Entra ID, no keys) or any OpenAI-compatible endpoint such as Ollama, and an `eval` mode that runs a question set and reports pass rate, tool calls, rejected queries and tokens; it is the regression harness for prompt and tool changes. CI runs the loop without a model: a scripted chat client replays tool calls through Microsoft.Extensions.AI's function invocation (`AgentLoopTests`). +4. **Preview and pilots:** the first version as a prerelease package (`-preview`; the AI project's csproj adds the suffix to the shared version, so a release publishes the core packages as stable and this one as a preview), complete enough to run end to end, with a runnable sample against the test server. Usefulness depends on the use case, so it is evaluated in one or two pilot apps with a real schema and real users, ideally different ones (interactive and background, small and large schema). The evaluation is part of the product: `OnQuery` logs (questions, queries, validation errors, tool calls per answer), simple user feedback per answer in the pilot app, and which questions the team moves to dedicated tools. The pilots can start before step 3 by using an exported `schema.graphql`. The sample (`test/Linq2GraphQL.Agent.Sample`) runs a chat loop against the in-process test server, with Azure AI Foundry (Entra ID, no keys) or any OpenAI-compatible endpoint such as Ollama, and an `eval` mode that runs a question set and reports pass rate, tool calls, rejected queries and tokens; it is the regression harness for prompt and tool changes. CI runs the loop without a model: a scripted chat client replays tool calls through Microsoft.Extensions.AI's function invocation (`AgentLoopTests`). 5. **Decide from the pilots:** did the teams get a working agent quickly, were users satisfied with the answers, where did it fail (filters, paging, large schemas), and what did they specialise? If it holds up, release `Linq2GraphQL.Client.AI`. If not, the core and generator improvements still stand. **Risks** diff --git a/src/Linq2GraphQL.Client.AI/Linq2GraphQL.Client.AI.csproj b/src/Linq2GraphQL.Client.AI/Linq2GraphQL.Client.AI.csproj index f4d5b88a..5ee28d6f 100644 --- a/src/Linq2GraphQL.Client.AI/Linq2GraphQL.Client.AI.csproj +++ b/src/Linq2GraphQL.Client.AI/Linq2GraphQL.Client.AI.csproj @@ -15,6 +15,18 @@ Linq GraphQL Client AI Agent + + + + $(PackageVersion)-preview + $(PackageVersion) + + + diff --git a/test/Linq2GraphQL.Agent.Sample/AgentSampleApp.cs b/test/Linq2GraphQL.Agent.Sample/AgentSampleApp.cs index 4f570247..30478d5b 100644 --- a/test/Linq2GraphQL.Agent.Sample/AgentSampleApp.cs +++ b/test/Linq2GraphQL.Agent.Sample/AgentSampleApp.cs @@ -3,6 +3,7 @@ using Linq2GraphQL.Client; using Linq2GraphQL.Client.AI; using Linq2GraphQL.TestClient; +using Microsoft.AspNetCore.Hosting; using Microsoft.AspNetCore.Mvc.Testing; using Microsoft.Extensions.AI; using Microsoft.Extensions.Configuration; @@ -174,7 +175,10 @@ private static IDisposable StartServer(string? serverUrl, out HttpClient httpCli return httpClient; } - var application = new WebApplicationFactory(); + // The factory looks for the server's content root next to the solution file, which only works from a + // test project; the test server reads nothing from it, so the sample's own output folder will do. + var application = new WebApplicationFactory() + .WithWebHostBuilder(b => b.UseContentRoot(AppContext.BaseDirectory)); httpClient = application.CreateClient(new() { BaseAddress = new Uri("https://localhost/graphql/") }); return application; } diff --git a/test/Linq2GraphQL.Agent.Sample/ChatClientFactory.cs b/test/Linq2GraphQL.Agent.Sample/ChatClientFactory.cs index 2617fdd3..b2365cc4 100644 --- a/test/Linq2GraphQL.Agent.Sample/ChatClientFactory.cs +++ b/test/Linq2GraphQL.Agent.Sample/ChatClientFactory.cs @@ -76,6 +76,13 @@ internal sealed record AgentSettings(AgentProvider Provider, string Endpoint, st return null; } + endpoint = endpoint.Trim().Trim('"', '\''); + if (!Uri.TryCreate(endpoint, UriKind.Absolute, out var uri) || uri.Scheme is not ("https" or "http")) + { + problem = $"Agent:Endpoint '{endpoint}' is not a valid URL; expected something like https://my-resource.cognitiveservices.azure.com/."; + return null; + } + problem = ""; return new AgentSettings(provider, endpoint, deployment, section["ApiKey"]); }