Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 10 additions & 2 deletions articles/flow/ai-support/file-attachments.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ The following MIME type categories are supported: images (`image/*`), text (`tex
[NOTE]
Uploading multiple files with the same name in a single batch throws an [classname]`IllegalArgumentException`. Ensure each file has a unique name.

To persist attachment data externally (since attachments are not stored in the conversation history), use the [methodname]`withAttachmentSubmitListener()` and [methodname]`withAttachmentClickListener()` callbacks:
To persist attachment data externally (since attachments are not stored in the conversation history), use the [since:com.vaadin:vaadin@V25.2]#`withRequestListener()`# and [methodname]`withAttachmentClickListener()` callbacks:

[source,java]
----
Expand All @@ -46,7 +46,7 @@ var orchestrator = AIOrchestrator
.withMessageList(messageList)
.withInput(messageInput)
.withFileReceiver(uploadManager)
.withAttachmentSubmitListener(event -> {
.withRequestListener(event -> {
// Store attachments keyed by message ID
saveAttachments(event.getMessageId(),
event.getAttachments());
Expand All @@ -59,4 +59,12 @@ var orchestrator = AIOrchestrator
.build();
----

The request listener is called on every prompt, just before the LLM request starts -- also for messages without attachments, in which case [methodname]`getAttachments()` returns an empty list. The same message ID is later available from [methodname]`ChatMessage.messageId()` in the conversation history and from the attachment click event, so externally stored attachments can be looked up when the user clicks them or when restoring a saved conversation. See <<index#request-response-listeners,Request & Response Listeners>> for details on the listener lifecycle.


[role="since:com.vaadin:vaadin@V25.2"]
== Sending Attachments Programmatically

Attachments don't have to come from an upload component. The [methodname]`prompt(String, List<AIAttachment>)` overload sends a prompt together with attachments produced in code -- generated files, fetched data, or content from other sources. See <<tool-calling#prompts-with-attachments,Tool Calling & Programmatic Prompts>>.

endif::flow[]
51 changes: 51 additions & 0 deletions articles/flow/ai-support/index.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,57 @@
Synchronous mode does not require push. If push is not available in your environment, disable streaming on the provider. A warning is logged at runtime if push is not enabled when using streaming mode.


[[request-response-listeners]]
[role="since:com.vaadin:vaadin@V25.2"]
== Request & Response Listeners

The builder accepts two lifecycle hooks that fire on every conversation turn:

* [methodname]`withRequestListener()` -- called on every prompt, just before the LLM request starts. The event carries the user message, the assigned message ID, and the attachments included with the message (an empty list when there are none). Use it to persist outbound requests or attachment data keyed by message ID; see <<file-attachments#,File Attachments>>.

Check warning on line 96 in articles/flow/ai-support/index.adoc

View workflow job for this annotation

GitHub Actions / lint

[vale] reported by reviewdog 🐶 [Vaadin.JustSimply] Avoid using 'just'. It may be insensitive. Raw Output: {"message": "[Vaadin.JustSimply] Avoid using 'just'. It may be insensitive.", "location": {"path": "articles/flow/ai-support/index.adoc", "range": {"start": {"line": 96, "column": 66}}}, "severity": "WARNING"}
* [methodname]`withResponseListener()` -- called once per turn when the response has completed, successfully or with an error. The event carries the response text and, on failure, the cause via [methodname]`getError()`. Use it to persist conversation history, trigger follow-up actions, or surface errors; see <<conversation-history#,Conversation History & Session Persistence>>.

[source,java]
----
var orchestrator = AIOrchestrator
.builder(provider, systemPrompt)
.withMessageList(messageList)
.withInput(messageInput)
.withRequestListener(event -> logPrompt(
event.getMessageId(), event.getUserMessage()))
.withResponseListener(event -> event.getError().ifPresentOrElse(
error -> logFailure(error),
() -> saveHistory(orchestrator.getHistory())))
.build();
----

The request listener runs on the UI thread; offload long-running work to a background thread. The response listener runs on a background thread: blocking I/O (such as database writes) is safe directly, but updates to Vaadin UI components must be wrapped in `ui.access()`.

On success, the response text may be empty if the model emitted only tool calls or otherwise stopped without producing visible content. Such turns still trigger the response listener, but empty responses are not appended to the conversation history.

.Replaces Earlier Listener Types
[NOTE]
[classname]`RequestListener` and [classname]`ResponseListener` replace the [classname]`AttachmentSubmitListener` and [classname]`ResponseCompleteListener` types from earlier versions, which have been removed. Replace [methodname]`withAttachmentSubmitListener()` calls with [methodname]`withRequestListener()` -- the new listener also fires for messages without attachments. Replace [methodname]`withResponseCompleteListener()` calls with [methodname]`withResponseListener()` -- the new listener also fires for failed and empty responses, so check [methodname]`getError()` and [methodname]`getResponse()` if the callback should only react to successful, text-bearing responses.


[role="since:com.vaadin:vaadin@V25.2"]
== Session Metadata

The LLM only knows what's in the conversation. Use [methodname]`withMetadata()` to supply ambient session context -- the current date and time, the user's locale, the active tenant, the page the user is on -- to the LLM on every turn. The supplier is invoked once per turn on the UI thread when the request is built, so it can read [methodname]`UI.getCurrent()` and session-scoped state:

[source,java]
----
var orchestrator = AIOrchestrator
.builder(provider, systemPrompt)
.withMessageList(messageList)
.withInput(messageInput)
.withMetadata(() -> "Current time: " + ZonedDateTime.now()
+ "\nUser locale: " + UI.getCurrent().getLocale())
.build();
----

If the supplier returns `null` or a blank string, no context is added for that turn -- useful for patterns where context only applies in certain situations. By default, the orchestrator supplies the current date and time, so the LLM can interpret relative references such as "show me sales from the past two months". Passing `null` to [methodname]`withMetadata()` disables session metadata entirely, including this built-in default.

Check warning on line 138 in articles/flow/ai-support/index.adoc

View workflow job for this annotation

GitHub Actions / lint

[vale] reported by reviewdog 🐶 [Vaadin.FirstPerson] Use first person (such as 'me') sparingly. Raw Output: {"message": "[Vaadin.FirstPerson] Use first person (such as 'me') sparingly.", "location": {"path": "articles/flow/ai-support/index.adoc", "range": {"start": {"line": 138, "column": 282}}}, "severity": "WARNING"}


== Topics

section_outline::[]
Expand Down
20 changes: 20 additions & 0 deletions articles/flow/ai-support/tool-calling.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -65,3 +65,23 @@
.UI Context Required
[IMPORTANT]
[methodname]`prompt()` requires an active UI context. If called from a background thread or outside a Vaadin request, it throws an [classname]`IllegalStateException`. Always call [methodname]`prompt()` from within a UI event handler or wrap the call in `ui.access()`.


[[prompts-with-attachments]]
[role="since:com.vaadin:vaadin@V25.2"]
=== Prompts with Attachments

The [methodname]`prompt(String, List<AIAttachment>)` overload sends a prompt together with caller-supplied attachments. This is useful when attachments are produced server-side -- generated files, fetched data, or content from sources other than an upload component:

[source,java]
----
byte[] report = generateReport();
orchestrator.prompt("Summarize the attached report.",
List.of(new AIAttachment(
"report.pdf", "application/pdf", report)));
----

The message and its attachments appear in the Message List and are forwarded to the LLM provider, just as if the user had uploaded the files through a configured file receiver. Pass an empty list to send no attachments; passing `null` throws a [classname]`NullPointerException`.

Check warning on line 84 in articles/flow/ai-support/tool-calling.adoc

View workflow job for this annotation

GitHub Actions / lint

[vale] reported by reviewdog 🐶 [Vaadin.JustSimply] Avoid using 'just'. It may be insensitive. Raw Output: {"message": "[Vaadin.JustSimply] Avoid using 'just'. It may be insensitive.", "location": {"path": "articles/flow/ai-support/tool-calling.adoc", "range": {"start": {"line": 84, "column": 99}}}, "severity": "WARNING"}

[NOTE]
This overload uses only the supplied list. If a file receiver is configured via [methodname]`withFileReceiver()` (see <<file-attachments#,File Attachments>>), attachments pending in it are not included -- they stay queued for the next [methodname]`prompt(String)` call or the user's next submit through a connected input. To merge uploaded and programmatic attachments, drain the receiver explicitly and pass the combined list.
Loading