Repository navigation
Commit 8a294af
fix: Extract grounding_metadata from Live API server_content
Merge #4213
## Summary
Fixes #3542 - VertexAiSearchTool query instability where grounding metadata is intermittently missing from Live API responses.
### Root Cause
The Live API's `receive()` method in `gemini_llm_connection.py` extracted various fields from Live API messages (usage_metadata, server_content, tool_call, session_resumption_update) but **never extracted `grounding_metadata`** from `server_content`. This prevented agents from accessing grounding data from Vertex AI Search, even when the backend provided it.
### Changes
#### Modified `src/google/adk/models/gemini_llm_connection.py`
- Added tracking variable `last_grounding_metadata` to accumulate grounding across messages.
- Added tracking variable `tool_call_metadata` to handle grounding metadata for buffered tool calls correctly.
- Extract `grounding_metadata` from `message.server_content.grounding_metadata` and accumulate it.
- Include accumulated `grounding_metadata` in `LlmResponse` when yielding:
- Content responses with parts (only if turn is not complete).
- Buffered tool call responses (using `tool_call_metadata` to match the time they were received).
- Turn complete responses (using current message's grounding or accumulated).
- Interrupted responses.
- Full text responses (via `__build_full_text_response`).
- Added warning log when incomplete grounding_metadata is detected (has `retrieval_queries` but missing `grounding_chunks`).
#### Updated `tests/unittests/models/test_gemini_llm_connection.py`
- Fixed existing tests to:
- Explicitly set `grounding_metadata = None` on mock server_content objects (in helper).
- Use real `types.GroundingMetadata` instead of `autospec` mock in standalone grounding test to avoid `AttributeError` on `retrieval_queries`.
- Added four new tests:
- `test_receive_extracts_grounding_metadata` - verifies grounding_metadata is extracted and included in content responses.
- `test_receive_grounding_metadata_reset_after_tool_call` - verifies grounding_metadata is reset after tool call.
- `test_receive_grounding_metadata_accumulates_across_messages` - verifies accumulation.
- `test_receive_interrupted_with_pending_text_preserves_flag` - verifies interrupted flag.
### Test Results
All 50 tests pass.
### Impact
This fix ensures that grounding data from Vertex AI Search is properly extracted and attached to `LlmResponse` events, allowing agents to access `event.grounding_metadata.grounding_chunks` when available.
============= Commits ==============
--
cb635f0 by Vedant Madane <vedantnm@gmail.com>:
ensure grounding metadata is correctly propagated in live API
- Correctly handle search results and citations in grounding metadata.
- Fix metadata parsing logic in gemini_llm_connection.py.
- Add unit tests for grounding metadata extraction.
Signed-off-by: Vedant Madane <vedantnm@gmail.com>
Co-authored-by: Shangjie Chen <deanchen@google.com>
COPYBARA_INTEGRATE_REVIEW=#4213 from VedantMadane:fix/live-api-grounding-metadata cb635f0
PiperOrigin-RevId: 9332914361 parent 8b09c48 commit 8a294af
2 files changed
Lines changed: 447 additions & 18 deletions
File tree
- src/google/adk/models
- tests/unittests/models
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
186 | 186 | | |
187 | 187 | | |
188 | 188 | | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
189 | 232 | | |
190 | 233 | | |
191 | 234 | | |
192 | 235 | | |
193 | 236 | | |
| 237 | + | |
194 | 238 | | |
195 | 239 | | |
196 | 240 | | |
| |||
201 | 245 | | |
202 | 246 | | |
203 | 247 | | |
| 248 | + | |
204 | 249 | | |
205 | 250 | | |
206 | 251 | | |
| |||
215 | 260 | | |
216 | 261 | | |
217 | 262 | | |
| 263 | + | |
218 | 264 | | |
219 | 265 | | |
220 | 266 | | |
| |||
258 | 304 | | |
259 | 305 | | |
260 | 306 | | |
261 | | - | |
| 307 | + | |
| 308 | + | |
262 | 309 | | |
263 | 310 | | |
264 | 311 | | |
| |||
276 | 323 | | |
277 | 324 | | |
278 | 325 | | |
279 | | - | |
280 | | - | |
281 | | - | |
| 326 | + | |
| 327 | + | |
| 328 | + | |
| 329 | + | |
282 | 330 | | |
283 | 331 | | |
284 | 332 | | |
| |||
326 | 374 | | |
327 | 375 | | |
328 | 376 | | |
329 | | - | |
| 377 | + | |
| 378 | + | |
| 379 | + | |
330 | 380 | | |
331 | 381 | | |
| 382 | + | |
332 | 383 | | |
333 | 384 | | |
334 | 385 | | |
| |||
430 | 481 | | |
431 | 482 | | |
432 | 483 | | |
433 | | - | |
| 484 | + | |
| 485 | + | |
| 486 | + | |
| 487 | + | |
| 488 | + | |
| 489 | + | |
| 490 | + | |
| 491 | + | |
| 492 | + | |
| 493 | + | |
| 494 | + | |
| 495 | + | |
| 496 | + | |
| 497 | + | |
| 498 | + | |
| 499 | + | |
| 500 | + | |
| 501 | + | |
| 502 | + | |
| 503 | + | |
| 504 | + | |
| 505 | + | |
434 | 506 | | |
435 | 507 | | |
436 | | - | |
| 508 | + | |
| 509 | + | |
| 510 | + | |
| 511 | + | |
437 | 512 | | |
438 | 513 | | |
439 | 514 | | |
440 | | - | |
| 515 | + | |
441 | 516 | | |
442 | 517 | | |
443 | 518 | | |
444 | 519 | | |
| 520 | + | |
445 | 521 | | |
446 | 522 | | |
447 | 523 | | |
448 | 524 | | |
| 525 | + | |
| 526 | + | |
| 527 | + | |
| 528 | + | |
449 | 529 | | |
450 | 530 | | |
451 | 531 | | |
452 | | - | |
453 | | - | |
| 532 | + | |
| 533 | + | |
| 534 | + | |
| 535 | + | |
454 | 536 | | |
455 | 537 | | |
456 | 538 | | |
| |||
462 | 544 | | |
463 | 545 | | |
464 | 546 | | |
| 547 | + | |
465 | 548 | | |
466 | 549 | | |
467 | 550 | | |
468 | 551 | | |
469 | 552 | | |
470 | 553 | | |
471 | 554 | | |
472 | | - | |
| 555 | + | |
| 556 | + | |
| 557 | + | |
| 558 | + | |
| 559 | + | |
| 560 | + | |
473 | 561 | | |
474 | 562 | | |
| 563 | + | |
475 | 564 | | |
476 | 565 | | |
477 | 566 | | |
| 567 | + | |
478 | 568 | | |
479 | 569 | | |
480 | 570 | | |
| 571 | + | |
481 | 572 | | |
482 | 573 | | |
483 | 574 | | |
484 | | - | |
| 575 | + | |
| 576 | + | |
| 577 | + | |
485 | 578 | | |
486 | 579 | | |
| 580 | + | |
487 | 581 | | |
488 | 582 | | |
489 | 583 | | |
490 | 584 | | |
| 585 | + | |
| 586 | + | |
| 587 | + | |
491 | 588 | | |
492 | 589 | | |
493 | 590 | | |
| |||
500 | 597 | | |
501 | 598 | | |
502 | 599 | | |
| 600 | + | |
503 | 601 | | |
504 | 602 | | |
505 | 603 | | |
506 | 604 | | |
| 605 | + | |
507 | 606 | | |
508 | 607 | | |
509 | 608 | | |
| |||
0 commit comments