diff --git a/articles/building-apps/ai/quickstart-guide.adoc b/articles/building-apps/ai/quickstart-guide.adoc index ad88009eb8..b5501b19b3 100644 --- a/articles/building-apps/ai/quickstart-guide.adoc +++ b/articles/building-apps/ai/quickstart-guide.adoc @@ -8,10 +8,10 @@ section-nav: badge-preview badge-flow --- = [since:com.vaadin:vaadin@V25.1]#Quick Start-Guide: Add an AI Chat Bot to a Vaadin + Spring Boot Application# [badge-flow]#Flow# -:preview-banner-content: This guide uses preview AI support features. This means that they are not yet ready for production usage and may have limitations or bugs. We encourage you to try them out and provide feedback to help us improve them. +:preview-banner-content: This guide uses preview AI integration features. This means that they are not yet ready for production usage and may have limitations or bugs. We encourage you to try them out and provide feedback to help us improve them. include::{articles}/_preview-banner.adoc[opts=optional] -This guide shows how to connect a Large Language Model (LLM) into a Vaadin application using Spring AI, Spring Boot, and the <<{articles}/flow/ai-support#,AI support features>>. You'll build a minimal chat UI with **MessageList** and **MessageInput**, stream responses token-by-token, and keep a conversational tone in the dialog with the AI -- all without writing boilerplate wiring code. +This guide shows how to connect a Large Language Model (LLM) into a Vaadin application using Spring AI, Spring Boot, and the <<{articles}/flow/ai-support#,AI integration features>>. You'll build a minimal chat UI with **MessageList** and **MessageInput**, stream responses token-by-token, and keep a conversational tone in the dialog with the AI -- all without writing boilerplate wiring code. image::images/chatbot-image.png[role=text-center] @@ -46,7 +46,7 @@ Add the Spring AI BOM and the OpenAI starter to import the necessary dependencie org.springframework.ai spring-ai-bom - 2.0.0-M2 + 2.0.0-RC2 pom import @@ -178,7 +178,7 @@ Start the application, open the browser, and try your first prompts. == What You Built -* A production-ready **chat bot** using Vaadin's AI support features +* A production-ready **chat bot** using Vaadin's AI integration features * **Token-by-token streaming** with Vaadin Push * **Conversation memory** managed by the LLM provider @@ -188,9 +188,10 @@ Start the application, open the browser, and try your first prompts. * Customize the **system prompt** to steer the assistant (e.g., tone, persona). * Add **file attachments** with `UploadManager` via <<{articles}/flow/ai-support/file-attachments#,`withFileReceiver()`>>. * Support **tool calls** via <<{articles}/flow/ai-support/tool-calling#,`withTools()`>>. -* **Persist conversation history** via <<{articles}/flow/ai-support/conversation-history#,`ResponseCompleteListener`>>. +* **Persist conversation history** via <<{articles}/flow/ai-support/conversation-history#,`ResponseListener`>>. * Let users **populate a Grid** from your database in natural language with <<{articles}/flow/ai-support/ai-powered-grid#, AI-Powered Grid>>. * Let users **build and update Charts** from your database in natural language with <<{articles}/flow/ai-support/ai-powered-chart#, AI-Powered Chart>>. +* Let users **fill a form** from natural-language input or attached files with <<{articles}/flow/ai-support/ai-powered-form#, AI-Powered Form>>. * Log prompts/responses for observability. diff --git a/articles/components/form-layout/ai-powered.adoc b/articles/components/form-layout/ai-powered.adoc new file mode 100644 index 0000000000..90c0cabab0 --- /dev/null +++ b/articles/components/form-layout/ai-powered.adoc @@ -0,0 +1,42 @@ +--- +title: AI Form Filler +page-title: AI Form Filler | Vaadin components +description: Let users fill forms from natural-language input or attached files. +meta-description: Use FormAIController to let an LLM populate the fields of a Vaadin form layout from natural-language prompts or attached files. +order: 60 +section-nav: badge-flow +--- + + += [since:com.vaadin:vaadin@V25.2]#AI-Powered Form# [badge-flow]#Flow# + +AI Form Filler lets your users fill the fields of a Vaadin form by typing in natural language or attaching a document. The [classname]`FormAIController` from the <<{articles}/flow/ai-support#, AI Integration>> module connects a layout to an [classname]`AIOrchestrator` so the LLM can read the current values, look up options for combo boxes and selects, and write new values back. Each write is validated through the [classname]`Binder` or the component's built-in validators, and rejections are reported back so the model can correct them on the same turn. + +[source,java] +---- +TextField name = new TextField("Name"); +EmailField email = new EmailField("Email"); +ComboBox country = new ComboBox<>("Country"); +country.setItems("Finland", "Germany", "United States"); +DatePicker hiredOn = new DatePicker("Hired on"); + +FormLayout form = new FormLayout(name, email, country, hiredOn); +MessageInput messageInput = new MessageInput(); + +FormAIController controller = new FormAIController(form); + +AIOrchestrator.builder(provider, systemPrompt) + .withInput(messageInput) + .withController(controller) + .build(); + +add(messageInput, form); +---- + +Example prompts: + +* "Fill in John Doe, john@acme.com, started in Germany on 2026-03-01." +* "Use the information in the attached resume to fill the form." +* "Clear the country and date fields." + +For the full guide -- including field descriptions, options for selects and multi-selects, [classname]`Binder` integration, and field locking during a fill -- see <<{articles}/flow/ai-support/ai-powered-form#, AI-Powered Form>> in the AI Integration section. The same approach is available for tabular data via <<{articles}/flow/ai-support/ai-powered-grid#, AI-Powered Grid>> and for charts via <<{articles}/flow/ai-support/ai-powered-chart#, AI-Powered Chart>>. diff --git a/articles/components/form-layout/index.adoc b/articles/components/form-layout/index.adoc index 7c3aa69de6..464f58b143 100644 --- a/articles/components/form-layout/index.adoc +++ b/articles/components/form-layout/index.adoc @@ -379,6 +379,11 @@ include::{root}/frontend/demo/component/formlayout/form-layout-colspan.ts[render Column span is capped to the number of columns currently in the layout to prevent overflow. +== AI Form Filler + +Use [classname]`FormAIController` to let users fill a Form Layout from natural-language input or attached files. See <>. + + == Miscellaneous === Form Item Usage diff --git a/articles/flow/ai-support/ai-powered-chart.adoc b/articles/flow/ai-support/ai-powered-chart.adoc index 72c12b39c9..f4dafca886 100644 --- a/articles/flow/ai-support/ai-powered-chart.adoc +++ b/articles/flow/ai-support/ai-powered-chart.adoc @@ -1,5 +1,5 @@ --- -title: AI-Powered Chart +title: AI-Generated Charts description: Use ChartAIController to let users build and update Highcharts visualizations from the application database using natural language. meta-description: Learn how to configure the Vaadin ChartAIController to create and update Charts from a database via natural language prompts and persist the resulting state. order: 70 @@ -70,7 +70,7 @@ if (saved != null) { Listeners do not fire when [methodname]`restoreState()` is called. The current state is also automatically included in session serialization, so no extra save/restore code is needed for in-session persistence. -== Reconnecting After Deserialization +== Reconnecting after Deserialization [classname]`ChartAIController` is not serializable. After session restore, create a new controller, pass it to [methodname]`reconnect()` together with the new provider, and optionally re-apply the saved state: diff --git a/articles/flow/ai-support/ai-powered-form.adoc b/articles/flow/ai-support/ai-powered-form.adoc new file mode 100644 index 0000000000..96ff86f7c1 --- /dev/null +++ b/articles/flow/ai-support/ai-powered-form.adoc @@ -0,0 +1,247 @@ +--- +title: AI Form Filler +description: Use FormAIController to let users fill any Vaadin form from natural-language input or attached files. +meta-description: Configure the Vaadin FormAIController to fill form fields from natural language or attached files, with Binder validation and select-options support. +order: 80 +--- + + += [since:com.vaadin:vaadin@V25.2]#AI-Powered Form# + + +[classname]`FormAIController` populates the fields of a <<{articles}/components/form-layout#,[classname]`FormLayout`>> (Vaadin's responsive multi-column form container) or any other layout, using values an LLM extracts from a user prompt or attached files. The controller traverses the layout, discovers every field, and allows the LLM to read the current values, query the available values for selection components like Combo Box or Radio Button Group, and write new values back. Each write is validated through the [classname]`Binder`, or through the component's built-in validators if [classname]`Binder` is not used. Rejected values are reported back so the model can correct them on the next turn. + +The controller works with any combination of standard Vaadin field components, such as [classname]`TextField`, [classname]`ComboBox`, [classname]`DatePicker`, [classname]`MultiSelectComboBox`, and [classname]`CheckboxGroup`. No extra wiring is needed beyond constructing the controller around the layout and attaching it to the orchestrator. + + +== Basic Usage + +Build the form, construct a [classname]`FormAIController` for it, and attach the controller to an orchestrator: + +[source,java] +---- +TextField name = new TextField("Name"); +EmailField email = new EmailField("Email"); +ComboBox country = new ComboBox<>("Country"); +country.setItems("Finland", "Germany", "United States"); +DatePicker hiredOn = new DatePicker("Hired on"); + +FormLayout form = new FormLayout(name, email, country, hiredOn); +MessageInput messageInput = new MessageInput(); + +FormAIController controller = new FormAIController(form); + +AIOrchestrator.builder(provider, systemPrompt) + .withInput(messageInput) + .withController(controller) + .build(); + +add(messageInput, form); +---- + +Example prompts: + +* "Fill in John Doe, john@acme.com, started in Germany on March 1, 2026." +* "Maria from Helsinki, hired last Monday." (relative dates are interpreted by the LLM) +* "Use the information in the attached resume to fill the form." (when the orchestrator has a <> configured) + +.Built-In Workflow Instructions +[TIP] +The controller already informs the LLM of the workflow it needs. You can focus your own system prompt on application-specific behavior, such as tone, naming conventions, or which fields the user may leave blank. + + +== Field Discovery + +The controller walks the container's component tree on every LLM turn, so fields added or removed between turns are picked up automatically. The container can be any component that implements [classname]`HasComponents`. Any component that implements [classname]`HasValue` is treated as a field, and any nested [classname]`HasComponents` is walked recursively. + +[classname]`PasswordField` is always hidden from the LLM. To hide other fields, for example internal IDs or anything sensitive that the user must fill in manually, call [methodname]`ignore()`: + +[source,java] +---- +controller.ignore(internalIdField); +---- + +Ignored fields are hidden from the LLM and stay editable while a fill is in progress. + + +== Hidden, Disabled, and Read-Only Fields + +The controller checks each field's state on every turn: + +* A field hidden via [methodname]`setVisible(false)`, or one that sits inside a hidden container, is dropped from the LLM surface entirely. The model cannot read its value and cannot write to it. It reappears the moment a value-change listener or other application code makes it visible again. +* A field the application has disabled ([methodname]`setEnabled(false)`) or set read-only ([methodname]`setReadOnly(true)`) stays in the form state as read-only context. The model sees its current value -- useful as context for writes to other fields -- but any write is rejected, and the model receives the rejection reason so it can adjust on the next turn. + +A common case is a conditional field driven by a [interfacename]`ValueChangeListener` -- a "Cost center" enabled only when "Trip type" is set to "Business", or a renewal date enabled by a "Renews automatically" checkbox. The built-in workflow instructions tell the model that a disabled or read-only field is usually waiting on a controlling field, so it sets the controlling field first and writes the dependent one in the same turn. + + +== Field Descriptions + +The LLM sees each field's label, helper text, and component type. When those don't fully capture the field's meaning, for example a numeric field that takes a percentage rather than an absolute amount, or a date that means "renewal date" rather than "purchase date", add an explicit description with [methodname]`describe()`: + +[source,java] +---- +controller.describe(discount, "Discount as a percentage between 0 and 100.") + .describe(renewalDate, "When the subscription renews, not when it started."); +---- + +Later calls for the same field overwrite earlier ones. + +=== Binder Integration + +When the form is backed by a [classname]`Binder`, pass the binder to the controller as well: + +[source,java] +---- +Binder binder = new Binder<>(Employee.class); +binder.bindInstanceFields(this); + +FormAIController controller = new FormAIController(form, binder); +---- + +For every named binding (`bind("propertyName")`, `bindInstanceFields(this)`, or `@PropertyId`), the bean property name is used as a default field description, so when the user mentions a field by its bean-side name, the LLM can match the request to the right field. An explicit [methodname]`describe()` call always overrides the default. Lambda-bound bindings have no property name and contribute no default field description. + + +== Options for Selection Fields + +For [classname]`ComboBox`, [classname]`Select`, and multi-select components like [classname]`CheckboxGroup` whose option set comes from the application rather than a fixed enum on the field, register the options with [classname]`ValueOptions`: + +[source,java] +---- +controller.valueOptions( + ValueOptions.forField(industry) + .options(List.of("Software", "Manufacturing", "Healthcare"))); +---- + +[methodname]`forField()` returns a builder. Use [methodname]`options(Collection)` for a fixed label list, or [methodname]`options(BiFunction)` for a callback the LLM invokes with a filter string and a result-count limit: + +[source,java] +---- +ComboBox projectSelect = new ComboBox<>("Project"); + +controller.valueOptions( + ValueOptions.forField(projectSelect) + .options((filter, limit) -> + projectService.search(filter, limit)), + label -> projectService.findByName(label)); +---- + +The second argument is the label-to-value converter. It is required whenever the field's value type is not [classname]`String`, and a registration that needs one but doesn't provide it fails to compile. For [classname]`String`-valued fields the converter is omitted; the chosen label is already the value. + +`projectService` here is a placeholder for your own data source -- a Spring repository, a REST client, an in-memory list, or whatever your application already uses to look up projects. + +.Eager Items as a Fallback +[NOTE] +Single- and multi-select fields configured with [methodname]`setItems(...)` already share their items with the LLM, so the simple fixed-options case often needs no [methodname]`valueOptions()` call. Use [methodname]`valueOptions()` when items come from a lazy or remote source rather than an in-memory list, or when you want the LLM to fetch options through a filter callback instead of receiving the full set up front. + +=== Multi-Select Fields + +[classname]`MultiSelectComboBox`, [classname]`CheckboxGroup`, and any other field that implements [classname]`MultiSelect` are supported. Use the component's concrete multi-select type for the field reference so the [methodname]`forField(MultiSelect)` overload is selected: + +[source,java] +---- +MultiSelectComboBox projectsField = new MultiSelectComboBox<>("Projects"); + +controller.valueOptions( + ValueOptions.forField(projectsField) + .options(List.of("Apollo", "Vega", "Helios")), + label -> projectService.findByName(label)); +---- + +The converter runs once per chosen label, and the resolved values are written to the field as a set. + +.Multi-Value Fields Must Implement MultiSelect +[NOTE] +A field whose value type is a [classname]`Collection` must implement [classname]`MultiSelect`. The controller rejects two cases at registration time: a [classname]`MultiSelect` field passed through the single-value [methodname]`forField(HasValue)` overload, and a [classname]`Collection`-valued field that doesn't implement [classname]`MultiSelect`. + + +== Validation + +When the model writes back a set of values, the controller commits all of them first and then runs validation once against the resulting form. Each field is checked according to how it is wired: + +* A field that is bound through a [classname]`Binder` is validated through its binding, so the converter and every registered validator run as one unit. +* An unbound field with a default validator (for example the email-format check on [classname]`EmailField`, or the `min` and `max` constraints on [classname]`NumberField` and [classname]`DatePicker`) is validated through that validator. +* Cross-field rules registered with `binder.withValidator((bean, ctx) -> ...)` -- see <<{articles}/building-apps/forms-data/add-form/validation#binder-level-validators,Binder-Level Validators>> -- are evaluated against the post-write form, but only when the binder has a bean set ([methodname]`setBean(bean)`) and every per-field check passes first. A failing rule is reported as a form-level rejection rather than against any single field, so the model can adjust the offending values and try again in the same turn. + +If validation fails for a written field, the value stays in the field, the field's UI error indicator turns on, and the failure is reported back to the model with the field id and the validator's message. The model can supply a corrected value in the same turn, so users typically see only the final, valid state. Fields the current turn did not write are not flagged invalid as a side effect -- a required field the user has not reached yet stays clean. + + +== What the Model Sees + +The controller sends only a defined subset of the form to the LLM. Knowing where that boundary lies matters when the form holds sensitive or domain-specific data. + +The model sees: + +* Each visible field's label, helper text, component type, and any [methodname]`describe()` text or [classname]`Binder` property-name default. +* The current value of every visible, non-ignored field, so it can decide which entries to overwrite. Disabled and application-set read-only fields are included for context, with a flag telling the model not to write to them. +* The eager items of a combo box or select, or the labels returned by a [methodname]`valueOptions()` query callback for the filter the model supplies. + +The model does not see: + +* Any field excluded with [methodname]`ignore()`. Its value, label, and existence are all hidden. +* Any field the application has hidden via [methodname]`setVisible(false)`, or that sits inside a hidden container. +* The contents of [classname]`PasswordField`, which is always excluded. +* Internal data, services, or beans. The model has access only to what the field components themselves show. + +.Visible Field Values Are Sent to the Model +[IMPORTANT] +Every visible field's current value is forwarded to the LLM provider on every turn. Hide fields that carry secrets, identifiers, or personally identifiable information with [methodname]`ignore()`, or keep them out of the layout passed to the controller. + + +== Field Locking During a Fill Turn + +While a fill is in progress, every field the user can currently edit -- visible, enabled, and not already read-only -- is set to read-only so the user cannot type into a field the AI is about to overwrite. Fields the application had already disabled or set read-only stay as they were. Locks are released automatically when the turn ends, whether it succeeded or failed. + +.Read-Only Toggles During a Turn +[NOTE] +If application code changes a field's read-only state during a turn, for example from a [interfacename]`ValueChangeListener` that reacts to one of the model's writes, that change is overridden when the controller releases its own locks at the end of the turn. + + +== Highlighting AI Changes + +When an AI fill changes several fields at once, users benefit from a visual cue that flags which fields the AI wrote. The controller exposes two APIs for this: + +* [methodname]`addFieldValueChangedListener()` registers a listener that fires once per successful turn with the fields whose values changed. +* [methodname]`showHighlight()` and [methodname]`hideHighlight()` toggle a per-field highlight rendered by the `vaadin-field-highlighter` web component. + +A typical pattern flashes every changed field after each fill: + +[source,java] +---- +controller.addFieldValueChangedListener(changes -> + changes.forEach(change -> controller.showHighlight(change.field()))); +---- + +The listener receives a list of [classname]`FieldValueChange` records in document order, each carrying the field, its pre-turn value, and its post-turn value. Fields the application has marked with [methodname]`ignore()` are excluded from the list. The listener is not called when the turn ended in error or when no field's value changed. Multiple listeners can be registered; each is independent, and the returned [classname]`Registration` removes the listener when its [methodname]`remove()` is called. + +A field hidden at turn start that is revealed and written into the same turn is reported with its real pre-turn value rather than `null`, so cascades into conditional fields show up correctly. + +The listener runs on the UI thread with the session lock held, so it can call [methodname]`showHighlight()`, update components, or any other Vaadin API directly -- no [methodname]`ui.access()` wrapper is needed. + +Repeated [methodname]`showHighlight()` calls on the same field are idempotent -- exactly one highlight remains. Each controller marks its highlight with an identifier unique to that instance, so the AI highlight coexists with any other `vaadin-field-highlighter` consumers the application keeps on the field, for example a collaboration session showing other users' edits. The highlight survives detach and re-attach: the controller re-applies it whenever the field returns to the DOM. The field passed to [methodname]`showHighlight()` does not need to belong to the controller's form -- any [classname]`HasValue` [classname]`Component` works. + + +== Reconnecting after Deserialization + +[classname]`FormAIController` is not serialized with the orchestrator. After session restore, create a new controller against the same form (and binder, if any), reapply the same [methodname]`describe()`, [methodname]`valueOptions()`, and [methodname]`ignore()` hints, re-register any change listeners, and pass the controller to [methodname]`reconnect()`: + +[source,java] +---- +FormAIController controller = new FormAIController(form, binder); +controller.describe(discount, "Discount as a percentage between 0 and 100.") + .valueOptions(ValueOptions.forField(industry) + .options(List.of("Software", "Manufacturing", "Healthcare"))) + .ignore(internalIdField); +controller.addFieldValueChangedListener(changes -> + changes.forEach(change -> controller.showHighlight(change.field()))); + +orchestrator.reconnect(provider) + .withController(controller) + .apply(); +---- + +Field ids remain stable across the round-trip because they live on the field components themselves, which Vaadin serializes as part of the UI tree. No separate state object needs saving or restoring; the form fields are the state, and [classname]`VaadinSession` already persists them. + + +== Composing Multiple Forms + +[classname]`FormAIController` manages a single container, but a view can host several. Construct one controller per form section and attach each to its own [classname]`AIOrchestrator`. Each orchestrator also needs a dedicated [classname]`LLMProvider`, [classname]`MessageInput`, and [classname]`MessageList` -- per the <>, none of those instances may be shared across orchestrators. diff --git a/articles/flow/ai-support/ai-powered-grid.adoc b/articles/flow/ai-support/ai-powered-grid.adoc index 302e2e14a5..9e951da3a8 100644 --- a/articles/flow/ai-support/ai-powered-grid.adoc +++ b/articles/flow/ai-support/ai-powered-grid.adoc @@ -1,5 +1,5 @@ --- -title: AI-Powered Grid +title: AI-Generated Grids description: Use GridAIController to let users populate a Grid from the application database using natural language. meta-description: Learn how to configure the Vaadin GridAIController to populate a Grid from a database via natural language prompts and persist the resulting state. order: 60 @@ -38,7 +38,7 @@ A message list is not required -- the grid itself is the output surface. Example * "Show product name and category grouped under Product, and monthly revenue." * "List the top 10 highest-paid employees with name, salary, and hire date." -* "Show me all employees in the Sales department." +* "List all employees in the Sales department." .Built-In Workflow Instructions [TIP] @@ -71,7 +71,7 @@ if (saved != null) { [methodname]`addStateChangeListener()` fires only when the grid is updated by the LLM, not when [methodname]`restoreState()` is called. The current state is also automatically included in session serialization, so no extra save/restore code is needed for in-session persistence. -== Reconnecting After Deserialization +== Reconnecting after Deserialization [classname]`GridAIController` is not serializable. After session restore, create a new controller, pass it to [methodname]`reconnect()` together with the new provider, and optionally re-apply the saved state: diff --git a/articles/flow/ai-support/controllers.adoc b/articles/flow/ai-support/controllers.adoc index adc0370780..3c7e11ec17 100644 --- a/articles/flow/ai-support/controllers.adoc +++ b/articles/flow/ai-support/controllers.adoc @@ -13,12 +13,13 @@ Controllers expose your application's capabilities to the LLM as callable tools If you've registered tool objects via [methodname]`withTools()` elsewhere, controllers are the framework-agnostic equivalent: the same tool calling, but defined through the [classname]`AIController` interface instead of LangChain4j's or Spring AI's [annotationname]`@Tool` annotations. -Vaadin provides two built-in controllers: +Vaadin provides three built-in controllers: * <> -- populates a [classname]`Grid` from a database using natural-language requests. * <> -- creates and updates [classname]`Chart` visualizations from a database using natural-language requests. +* <> -- fills any layout containing input fields, from natural-language input or attached files. -Both rely on a <<#database-provider,[classname]`DatabaseProvider`>> to expose schema information and execute queries on behalf of the LLM. +The grid and chart controllers rely on a <<#database-provider,[classname]`DatabaseProvider`>> to expose schema information and execute queries on behalf of the LLM. The form controller works directly with the form's field components. == Attaching a Controller @@ -111,10 +112,11 @@ Return only the tables, columns, and relationships the LLM needs. A smaller, wel The built-in controllers cover grid and chart data exploration. To expose your own capabilities to the LLM, implement [classname]`AIController` directly. -[classname]`AIController` defines two methods: +[classname]`AIController` defines three methods: * [methodname]`getTools()` -- returns the list of [classname]`LLMProvider.ToolSpec` instances the controller contributes to each LLM request. Tools are collected before every request, so a controller can vary its tool set based on current state. -* [methodname]`onResponseComplete()` -- runs after all tool calls for a user request have finished and the LLM has produced its final response. Controllers use this hook to apply deferred state changes, avoiding partial state and multiple redraws during a multi-tool turn. +* [methodname]`onRequest()` -- runs on the UI thread right before the LLM stream opens. Use it to lock UI surfaces, snapshot state the tool definitions depend on, or otherwise prepare for the turn. +* [methodname]`onResponse(Throwable)` -- runs on the UI thread once the LLM stream has completed, either successfully (`error` is `null`) or with an error. Controllers use this hook to commit deferred state changes on success and release any per-turn state captured in [methodname]`onRequest()` on failure. Each tool is an implementation of [classname]`LLMProvider.ToolSpec`, which has four methods: @@ -163,7 +165,10 @@ public class WeatherController implements AIController { } @Override - public void onResponseComplete() { + public void onResponse(Throwable error) { + if (error != null) { + return; + } // Apply any deferred state changes here. } } diff --git a/articles/flow/ai-support/conversation-history.adoc b/articles/flow/ai-support/conversation-history.adoc index 68d11bf9fc..5dd837ce71 100644 --- a/articles/flow/ai-support/conversation-history.adoc +++ b/articles/flow/ai-support/conversation-history.adoc @@ -15,7 +15,7 @@ The orchestrator tracks conversation history internally. Use [methodname]`getHis == Persisting History -Use [classname]`ResponseCompleteListener` to persist conversation state after each successful exchange: +Use [classname]`ResponseListener` to persist conversation state after each exchange: [source,java] ---- @@ -23,16 +23,19 @@ var orchestrator = AIOrchestrator .builder(provider, systemPrompt) .withMessageList(messageList) .withInput(messageInput) - .withResponseCompleteListener(event -> { + .withResponseListener(event -> { + if (event.getError().isPresent()) { + return; + } var history = orchestrator.getHistory(); saveToDatabase(sessionId, history); }) .build(); ---- -The listener is not called when the response fails, times out, is empty, or when history is restored. +The listener fires once per turn for both successful and failed exchanges -- check [methodname]`event.getError()` to tell them apart. It is not called when history is restored via [methodname]`withHistory()`, and empty responses (turns where the model emitted only tool calls or no visible content) are not appended to history. -.UI Updates from ResponseCompleteListener +.UI Updates from ResponseListener [IMPORTANT] The listener runs on a background thread. It is safe to perform blocking I/O (such as database writes) directly. However, to update Vaadin UI components from this callback, wrap the update in `ui.access()`. @@ -59,7 +62,7 @@ endif::[] This replays messages into the LLM provider's memory, populates the Message List UI, and rebuilds internal mappings for attachment click handling. [NOTE] -[methodname]`getHistory()` returns a point-in-time snapshot. If called while a streaming response is in progress, the snapshot may include the user message without its corresponding assistant response. Use [classname]`ResponseCompleteListener` to capture history at the right time. +[methodname]`getHistory()` returns a point-in-time snapshot. If called while a streaming response is in progress, the snapshot may include the user message without its corresponding assistant response. Use [classname]`ResponseListener` to capture history at the right time. == Data Model @@ -78,7 +81,7 @@ File attachments are represented as [classname]`AIAttachment` records with [meth [classname]`AIOrchestrator` is serializable. Conversation history, UI component bindings, listener registrations, and display names are preserved across serialization. However, the LLM provider and tool objects are transient -- they are not serialized and must be restored after deserialization. -=== Reconnecting After Deserialization +=== Reconnecting after Deserialization After the session is restored, call [methodname]`reconnect(LLMProvider)` to supply a new provider and optionally restore tools and file attachments. The [methodname]`apply()` call replays the existing conversation history onto the new provider so that it has full context for subsequent prompts. The UI is not modified -- the message list, input, and file receiver components retain their state automatically. @@ -98,7 +101,7 @@ orchestrator.reconnect(provider) .Controller Reattachment [NOTE] -[classname]`AIController` instances -- including [classname]`GridAIController` and [classname]`ChartAIController` -- are not serialized with the orchestrator. Create a new controller after session restore and pass it to [methodname]`withController()` on the reconnector. The built-in controllers each have their own state capture and restoration API; see <> and <> for the specifics. +[classname]`AIController` instances, including [classname]`GridAIController`, [classname]`ChartAIController`, and [classname]`FormAIController`, are not serialized with the orchestrator. Create a new controller after session restore and pass it to [methodname]`withController()` on the reconnector. The grid and chart controllers each have their own state capture and restoration API. The form controller has no separate state object, because the form fields themselves are persisted with the [classname]`VaadinSession`. See <>, <>, and <> for the specifics. If [methodname]`prompt()` is called before reconnecting, it throws an [classname]`IllegalStateException`. diff --git a/articles/flow/ai-support/index.adoc b/articles/flow/ai-support/index.adoc index 7af1c51413..a982224567 100644 --- a/articles/flow/ai-support/index.adoc +++ b/articles/flow/ai-support/index.adoc @@ -1,7 +1,7 @@ --- -title: AI Support +title: AI Integration description: Vaadin provides UI interfaces and an AI orchestrator for building AI-powered UIs with Vaadin, handling streaming, history, attachments, and tool calling. -meta-description: Use Vaadin AI support features to build AI-powered interfaces by connecting Message List, Message Input, and Upload components to Spring AI or LangChain4j. +meta-description: Use Vaadin AI integration features to build AI-powered interfaces by connecting Message List, Message Input, and Upload components to Spring AI or LangChain4j. section-nav: badge-preview order: 213 page-links: @@ -10,13 +10,13 @@ page-links: --- -= [since:com.vaadin:vaadin@V25.1]#AI Support# += [since:com.vaadin:vaadin@V25.1]#AI Integration# // tag::description[] Vaadin provides a set of UI interfaces and a coordination engine for building AI-powered UIs. The [classname]`AIOrchestrator` wires UI components to an LLM provider, handling streaming responses, conversation history, file attachments, and tool calling behind a simple builder API. // end::description[] -:preview-feature: AI support features +:preview-feature: AI integration features :feature-flag: com.vaadin.experimental.aiComponents :ticket-url: https://github.com/vaadin/flow-components/issues/new/choose include::{articles}/_preview-banner.adoc[opts=optional] @@ -24,12 +24,12 @@ include::{articles}/_preview-banner.adoc[opts=optional] == Overview -The AI support module consists of four parts: +AI integration consists of four parts: * **Orchestrator** -- [classname]`AIOrchestrator` is a non-visual coordination engine that connects UI components to an LLM provider. It has no DOM element and should not be added to a layout. * **LLM Provider** -- plugs the orchestrator into any LLM framework. Spring AI and LangChain4j are built in via [classname]`SpringAILLMProvider` and [classname]`LangChain4JLLMProvider`; add others by implementing the [classname]`LLMProvider` interface. * **Component interfaces** -- [classname]`AIInput`, [classname]`AIMessageList`, [classname]`AIMessage`, and [classname]`AIFileReceiver` define contracts for UI components that the orchestrator can work with. The builder also accepts standard Vaadin components ([classname]`MessageInput`, [classname]`MessageList`, [classname]`UploadManager`, [classname]`Upload`) directly. -* **Controllers** -- [classname]`AIController` is the framework-agnostic interface for contributing tools and lifecycle hooks to the orchestrator. Built-in controllers such as [classname]`GridAIController` and [classname]`ChartAIController` bring AI-powered data exploration to [classname]`Grid` and [classname]`Chart`, backed by a [classname]`DatabaseProvider`. +* **Controllers** -- [classname]`AIController` is the framework-agnostic interface for contributing tools and lifecycle hooks to the orchestrator. Built-in controllers bring AI-powered data exploration to [classname]`Grid` and [classname]`Chart` through [classname]`GridAIController` and [classname]`ChartAIController` (backed by a [classname]`DatabaseProvider`), and form-filling to any layout containing input fields through [classname]`FormAIController`. Add the UI components to your layout and pass them to the orchestrator through its builder. The orchestrator wires them together and manages the LLM interaction. @@ -113,4 +113,7 @@ section_outline::[] |<<{articles}/components/charts#,Charts>> |Built and updated from the application database via natural-language requests. See <>. +|<<{articles}/components/form-layout#,Form Layout>> +|Fields can be filled in from natural-language input or attached files. See <>. + |=== diff --git a/articles/flow/ai-support/llm-providers.adoc b/articles/flow/ai-support/llm-providers.adoc index 3972cabc5d..cbfcb851f4 100644 --- a/articles/flow/ai-support/llm-providers.adoc +++ b/articles/flow/ai-support/llm-providers.adoc @@ -25,7 +25,7 @@ Both built-in providers maintain a 30-message memory window. Older messages are ---- // From ChatModel - use an implementation of Spring AI ChatModel ChatModel chatModel = OpenAiChatModel.builder() - .openAiApi(...).defaultOptions(...).build(); + .openAiClient(...).options(...).build(); SpringAILLMProvider provider = new SpringAILLMProvider(chatModel); // From ChatClient - use a Spring AI ChatClient diff --git a/articles/flow/ai-support/session-context.adoc b/articles/flow/ai-support/session-context.adoc new file mode 100644 index 0000000000..dba88d3125 --- /dev/null +++ b/articles/flow/ai-support/session-context.adoc @@ -0,0 +1,84 @@ +--- +title: Session Context +description: Pass per-turn ambient information -- date and time, tenant, locale, page state -- to the LLM through the AIOrchestrator. +meta-description: Learn how to send per-turn session context to the LLM using the Vaadin AIOrchestrator withMetadata API, with a built-in current date and time default. +order: 35 +--- + + += [since:com.vaadin:vaadin@V25.2]#Session Context# + + +The AI orchestrator can pass a short, free-form string of session context to the LLM on every turn, such as the current date and time, the active tenant, the user's locale, the page the user is on, or anything else the model might need for each prompt. The string is captured before each request, so it always reflects the state at the time the user sent the prompt. + + +== Default Current Date and Time + +By default, the orchestrator sends a single line with the server's current date and time so the LLM can interpret relative date references like "show sales from the past two months" or "schedule a follow-up for next Friday" without having to guess. No setup is needed beyond constructing the orchestrator as usual; the line the model receives looks like: + +[source] +---- +Current server date and time: 2026-05-28T17:42+03:00 (Friday, Europe/Helsinki) +---- + + +== Custom Context + +To pass anything else -- or to compose several pieces of context together -- register a supplier with [methodname]`withMetadata()`: + +[source,java] +---- +AIOrchestrator.builder(provider, systemPrompt) + .withMessageList(messageList) + .withInput(messageInput) + .withMetadata(() -> { + String tenant = TenantContext.current().getName(); + String locale = UI.getCurrent().getLocale().toLanguageTag(); + return "Tenant: " + tenant + "\nUser locale: " + locale; + }) + .build(); +---- + +The supplier is invoked once per turn, right before the request goes out. The returned string is passed to the model without further interpretation; compose multiple pieces of context with plain string concatenation. Setting a supplier replaces the default current-date-and-time line; include it yourself if you still want it: + +[source,java] +---- +.withMetadata(() -> "Tenant: " + TenantContext.current().getName() + + "\nServer time: " + ZonedDateTime.now()) +---- + +A supplier that returns `null` or a blank string skips the context for that turn, which is convenient for "include this only when X" patterns: + +[source,java] +---- +.withMetadata(() -> { + List open = alertService.openIncidentsFor(currentUser()); + return open.isEmpty() + ? null + : "Open incidents this user owns: " + open; +}) +---- + + +== Disabling Session Context + +Pass `null` to disable session context entirely, including the default date-and-time line: + +[source,java] +---- +AIOrchestrator.builder(provider, systemPrompt) + .withMetadata(null) + .build(); +---- + + +== Threading and Errors + +The supplier runs on the UI thread under the session lock, so it can read [methodname]`UI.getCurrent()` and any session-scoped state. Keep it fast -- anything that touches a database, a remote service, or the file system should be done outside the supplier and cached in session-scoped state. + +A supplier that throws an exception aborts the turn through the normal error path: the assistant placeholder is replaced with a generic error message, the [classname]`ResponseListener` (if registered) fires with the thrown exception, and the exception propagates to the caller of the prompt entry point. + + +== Why Not the System Prompt + +The system prompt is sent verbatim on every turn. Many LLM providers use it as the cache prefix to keep latency and cost down between calls; putting per-turn values such as the current time or the active tenant into the system prompt invalidates that cache on every request. Session context is delivered through a separate per-turn channel that lives outside the system prompt, so the system prompt stays cacheable.