Make AI conversation cold-stage reads opt-in

docs/en/changes/changes.md

@@ -12,7 +12,7 @@
 * Add BanyanDB trace tail sampling metrics to the BanyanDB self-observability layer, in a new `otel-rules/banyandb/banyandb-trace-sampling.yaml` rule file. It covers the whole `banyandb_trace_pipeline_*` / `banyandb_trace_tst_pipeline_*` catalog a sampler plugin chain emits — pipeline reconciliation, per-plugin `Decide` execution rate and latency, chain batching, the trace-level evaluated / retained / dropped / immature outcomes, every fail-open guard and bounded-retention counter, drop-set capacity and finalization state, the plugin telemetry-host safety bounds, and the first-party `sw-trace-sampler` / `zipkin-trace-sampler` decision and row metrics. The plugin chain is optional, and the metrics follow it: on a cluster with no sampler configured the wire families are never registered, so every metric here stays absent rather than reading zero. Modeled at Service scope with `group` kept as a metric label rather than at Endpoint scope, so one cluster-wide page can render per-group series and cluster totals alike — OAP does no cross-scope rollup, so an Endpoint-scope metric could not have been aggregated back up to the cluster.
 * Fix a second `CounterWindow` key collision in the v2 MAL engine, this time ACROSS rules. `rate()` / `increase()` / `irate()` resolve their lower bound from a process-wide window keyed on the counter's own name plus its post-`.sum(...)` label set, with nothing identifying the rule doing the evaluation. Two rules that read one wire family, tell their streams apart with `tagEqual(...)`, and then `.sum(...)` away the label they filtered on therefore collapse onto one window slot and difference against each other's values. The queue is ordered by (timestamp, value), so the smaller counter wins the lower-bound lookup and still reads correctly while its partner is inflated by the gap between them — which is why this went unnoticed. A collision needs the discriminating label to be DROPPED by the `.sum(...)`: where it survives, the rules' label values differ and the window keeps them apart. Auditing the shipped rules on that basis gives 10 colliding keys over ~25 rules — `meter_activemq_cluster_gc_parallel_young_collection_count` reported ~9000/min of young-gen collections from a completely idle broker (differencing against the old-gen counter); MySQL `commands_*` / `tps` rate against each other; so do the GenAI gateway input/output token rates, four Envoy `cluster_*` counters, APISIX matched/unmatched instance bandwidth, and BanyanDB's own `network_recv` / `network_sent`, which drop the `kind` label that separates bytes-received from bytes-sent on one interface. Measured against two live scrapes of the demo cluster's FODC proxy, that last pair was wrong on every interface: `network_sent` read a flat 0 B/s and `network_recv` read large negative values (down to -778 MB/s) from differencing against the sent counter, where both now match the byte delta exactly. No rule changes were needed for any of these -- each rule already reduces to the labels it should; only the window key was wrong. The window is now keyed by (owning rule, counter name, labels). This is the complement of the within-rule collision fixed earlier by keying on the counter's own name: neither name alone is sufficient, because the two collisions are independent. `RunningContext.metricName` — written on every rule evaluation and read by nobody since that earlier fix — is what supplies the rule identity, so no code generation or MAL syntax changes. Note the whole-rule-set comparison suite could not have caught this: it resets the shared window before every rule, the one condition under which the collision cannot appear.
 * Fix `meter_rabbitmq_node_outgoing_messages_total` double-counting one of its terms. The rule summed six delivery-rate terms but `rabbitmq_global_messages_delivered_get_auto_ack_total` appeared twice, so auto-ack `basic.get` deliveries were counted once more than the other four delivery paths and the reported outgoing rate ran high whenever polling consumers were in use. The duplicate term is removed, leaving the five distinct families (redelivered, consume auto/manual ack, get auto/manual ack).
-* Add AI agent conversations landed by the AI Sessionizer: the `AI_AGENT` layer, the bundled `lal/ai-agent.yaml` rule with the `ConversationFile` output builder that verifies and stores Session Data and Session Flow files, the `ai_agent_session_data` and `ai_agent_session_flow` models in a new BanyanDB group `recordsAIAgent`, the `ai-agent-conversation` module that folds a conversation into one `asz.view` document, and the `listConversations` (with optional `conversation` and `title` conditions) / `getConversationRawFiles` GraphQL queries and the streamed `GET /ai-agent/conversations/{conversation}/v1/view` route that serves the document. A round from before the list attributes existed lands and lists with zero counts, and the view shows as much as landed: the chain resumes after a missing, unreadable or refused round, and the absent rounds and files are named once as ranges. A file over `maxFileBytes`, 15 MiB by default, is rejected at ingest and counted under the reason `size`, because one file over BanyanDB's 16 MiB gRPC message limit fails the bulk write it travels in and every record behind it; on MySQL the body column is `LONGTEXT`, since a body that size outgrows `MEDIUMTEXT` as Base64. Each window read is capped at `maxResponseBytes`, 100 MiB by default, as a per-call option on the BanyanDB client in place of its 50 MB default, so the module's reads are bounded by its own settings and nothing else's read changes. The document also carries the session's workspace changes: the Sessionizer's Claude Code plugin lands a Session Data file of kind `changes` beside a stream's transcript, `streams/<stream>/changes-<stamp>-<seq>.sd`, with one change record per observed tool call, stored like any other file, and the runtime's own patch for its editing tools travels as the second data part of the call's result record; `workspace_changes` lists every record joined to its step by tool-use id with the record's fields as `changes/1` lists them, `summary.changes` counts them, and a tool step names its records under `changes`. A round's header counts, `changes`, `lines_added`, `lines_removed`, `llm_calls`, `subagents` and `bash_runs`, land on `ai_agent_session_flow` and reach the conversation list as `ConversationRow` fields, absent rather than zero when the round did not carry them. The layer also takes the agent runtime's own metrics, in the `otel-rules/ai-agent/` rule set enabled by default: Claude Code's `claude_code.token.usage`, as its own exporter sends it or as the Sessionizer derives it from the landed transcripts, per service and per sender, by type, model and query source with `session.id` summed away, plus the exporter's cost, active time, sessions, lines of code, commits, pull requests and edit decisions when they arrive.
+* Add AI agent conversations landed by the AI Sessionizer: the `AI_AGENT` layer, the bundled `lal/ai-agent.yaml` rule with the `ConversationFile` output builder that verifies and stores Session Data and Session Flow files, the `ai_agent_session_data` and `ai_agent_session_flow` models in a new BanyanDB group `recordsAIAgent`, the `ai-agent-conversation` module that folds a conversation into one `asz.view` document, and the `listConversations` (with optional `conversation` and `title` conditions) / `getConversationRawFiles` GraphQL queries and the streamed `GET /ai-agent/conversations/{conversation}/v1/view` route that serves the document. The view and raw-file export leave BanyanDB's stage selection unspecified by default and select only the cold stage when the caller explicitly sets `coldStage` to true. A round from before the list attributes existed lands and lists with zero counts, and the view shows as much as landed: the chain resumes after a missing, unreadable or refused round, and the absent rounds and files are named once as ranges. A file over `maxFileBytes`, 15 MiB by default, is rejected at ingest and counted under the reason `size`, because one file over BanyanDB's 16 MiB gRPC message limit fails the bulk write it travels in and every record behind it; on MySQL the body column is `LONGTEXT`, since a body that size outgrows `MEDIUMTEXT` as Base64. Each window read is capped at `maxResponseBytes`, 100 MiB by default, as a per-call option on the BanyanDB client in place of its 50 MB default, so the module's reads are bounded by its own settings and nothing else's read changes. The document also carries the session's workspace changes: the Sessionizer's Claude Code plugin lands a Session Data file of kind `changes` beside a stream's transcript, `streams/<stream>/changes-<stamp>-<seq>.sd`, with one change record per observed tool call, stored like any other file, and the runtime's own patch for its editing tools travels as the second data part of the call's result record; `workspace_changes` lists every record joined to its step by tool-use id with the record's fields as `changes/1` lists them, `summary.changes` counts them, and a tool step names its records under `changes`. A round's header counts, `changes`, `lines_added`, `lines_removed`, `llm_calls`, `subagents` and `bash_runs`, land on `ai_agent_session_flow` and reach the conversation list as `ConversationRow` fields, absent rather than zero when the round did not carry them. The layer also takes the agent runtime's own metrics, in the `otel-rules/ai-agent/` rule set enabled by default: Claude Code's `claude_code.token.usage`, as its own exporter sends it or as the Sessionizer derives it from the landed transcripts, per service and per sender, by type, model and query source with `session.id` summed away, plus the exporter's cost, active time, sessions, lines of code, commits, pull requests and edit decisions when they arrive.
 * Support querying Zipkin traces from the BanyanDB cold stage through the Zipkin HTTP query API. `/api/v2/traces` takes an optional `coldStage` parameter, and `/api/v2/trace/{traceId}` and `/api/v2/traceMany` take optional `coldStage`, `endTs` and `lookback` parameters; the two by-id lookups had no time range at all before. All of them are SkyWalking additions to the Zipkin API and default to the previous behavior, so existing Zipkin clients such as the Lens UI keep working unchanged, and storages other than BanyanDB ignore `coldStage`. The admin debugging endpoints `/debugging/query/zipkin/api/v2/traces` and `/debugging/query/zipkin/api/v2/trace` accept the same parameters. The BanyanDB stages e2e now seeds Zipkin traces into the generated cold data and verifies the three endpoints against it.
 * BanyanDB: a query without a time range now covers everything the group's hot/warm stages retain instead of only the last 24 hours. The hard-coded day was a fallback added with the trace model, which requires a time range; it made `queryTrace(traceId)` without a `duration`, the Zipkin `/api/v2/trace/{traceId}` and `/api/v2/traceMany` lookups, and the TraceQL by-id lookup return "not found" for any trace older than a day even though it was still retained. The unbounded range is now bound instead, and BanyanDB's own retention limits the scan. `queryTraces` now rejects a condition with neither `queryDuration` nor `traceId` instead of silently searching the last day; a `traceId` lookup without `queryDuration` searches everything the hot/warm stages retain and reports a 0-to-now `retrievedTimeRange`.
 * Fix the JDBC storages, MySQL, PostgreSQL and H2, reading only the first day's table of a time range shorter than a day that crosses midnight. `TableHelper` walked the range in 24-hour steps from its first instant, so a range that began late on one day and ended early on the next never reached the second day; every time-ranged read, metrics, traces, logs, alarms, records and the AI agent conversation list, was missing the newer day's rows for the length of the range after each midnight, the last thirty minutes read at 00:10 among them. The range is now walked by calendar day from the start of its first day.

docs/en/setup/backend/ai-agent-conversation.md

@@ -85,14 +85,16 @@ HTTP route on the same server, because its document is as large as the conversat
   most `limit` (default 1000), then folded to one row per conversation. An optional `conversation` narrows the read to
   one conversation by id, and an optional `title` keeps only the rows whose title contains the text,
   case-insensitively — matched after folding, on the newest round's title, so it never widens the rounds read.
+  On BanyanDB, `duration.coldStage: true` selects the cold stage; otherwise the query uses the default hot/warm stages.
 - `getConversationRawFiles(condition, files)` lists every landed file and round of a conversation with its id,
   digest and size; selecting `body` returns the files verbatim, which is the export path. The optional `files`
-  argument narrows the read to named files.
+  argument narrows the read to named files. On BanyanDB, `condition.coldStage: true` selects the cold stage,
+  defaulting to the hot/warm stages when omitted or false.
 
 ### The conversation view route
 
 ```
-GET /ai-agent/conversations/{conversation}/v1/view?service={serviceName}[&instance={instanceName}]
+GET /ai-agent/conversations/{conversation}/v1/view?service={serviceName}[&instance={instanceName}][&coldStage=true]
 ```
 
 It answers with the whole conversation, once, as one `asz.view` version 1.0 document, the document the
@@ -115,6 +117,7 @@ and nothing is cached.
 |---|---|
 | `service` / `serviceId` | the service by name, or by id; one of them is required |
 | `instance` | optional, the sender's instance name from the list row; with it, every storage read is a full series lookup |
+| `coldStage` | optional, false by default. On BanyanDB, true selects only the cold stage; otherwise the read uses the default hot/warm stages. The UI passes its selected stage when opening a conversation. Other storages ignore it. |
 | `Accept` | `application/vnd.skywalking.asz.view+yaml`, or any type naming `yaml`, for YAML; anything else, JSON, as `asz conversation -json` prints it |
 | `Content-Type` | names the document and its version, the HTTP way: `application/vnd.skywalking.asz.view+json; version=1.0` or `application/vnd.skywalking.asz.view+yaml; version=1.0`. The document's own first two keys, `format` and `version`, say the same |
 | `Accept-Encoding` | the body is compressed when the client allows; a document is repetitive text and shrinks several times over |
@@ -205,9 +208,10 @@ dropped for want of a record worker; drop `ai-agent` from `SW_LOG_LAL_FILES` as
   with up to `maxResponseBytes`, 100 MiB by default, as a call option on the shared client in place of its 50 MB
   default, which every other read keeps; the window times `maxFileBytes` must stay under it. Elasticsearch answers
   at most 10,000 hits to one search.
-- A read that is not bound to a duration, the view and the export, covers every retained stage: on BanyanDB the
-  default stages and, when the group keeps one, the cold stage. A conversation the list found in cold storage
-  is served, and one that spans stages is served whole.
+- The view and the export read over the retention window of the caller's selected stages. On BanyanDB, the
+  default is hot/warm; cold is queried only when the caller explicitly sets `coldStage: true`. Every round and
+  file read uses that same selection. A conversation spanning stages can therefore report missing rounds or
+  files that are outside the selected stages.
 - When the caller names no sender, the view and the export read across every sender of the service and keep one
   copy of a file or round two senders both pushed, so a Sessionizer renamed between pushes still yields the
   whole conversation.