Skip to content

feat: add vector-backed memory and Azure AI Search - #1297

Merged
Quim Muntal (qmuntal) merged 5 commits into
mainfrom
feat/vectordata-memory-azure-search
Oct 19, 2026
Merged

Quim Muntal (qmuntal) merged 5 commits into
mainfrom
feat/vectordata-memory-azure-search

Conversation

@qmuntal

Copy link
Copy Markdown
Member

Summary

  • Add vector-backed chat-history memory with automatic retrieval and on-demand search, plus memory examples and feature-usage telemetry.
  • Introduce the reusable VectorData store facade, including Go 1.27-gated typed collection methods while retaining Go 1.26 support.
  • Add Azure AI Search with API-key and Entra ID authentication, schema management, CRUD, portable filters, vector/hybrid search, and embedding generation.
  • Share generic vector representation helpers and default embedding-generator resolution between Azure Search and the in-memory connector, preserving provider-specific validation and ownership.
  • Add regression coverage, an Azure Search example, and updated compatibility documentation.

Builds on the already-merged #1293; this branch is rebased on current main.

Validation

  • 1,153 selected tests passed after rebasing.
  • Affected embedding, VectorData, memory, Azure Search, and telemetry suites and vet passed on Go 1.27.
  • Affected examples build on Go 1.26 and Go 1.27.
  • golangci-lint with --new-from-rev=origin/main: 0 issues.
  • Unfiltered scoped lint reports three gofumpt findings in files unchanged from main: azure.go, demo.go, and telemetry.go. These unrelated files were left untouched.
  • Earlier live Azure validation covered both authentication modes, CRUD, vector/hybrid search, empty collection uploads, and empty-string membership filters. The final generic-helper and rebase checks were offline.
  • No compiled binaries or build artifacts are included.

Add the chat-history memory provider, reusable VectorData store facade, Azure AI Search connector, and shared generic vector helpers with regression tests and examples.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings October 16, 2026 18:44
@qmuntal
Quim Muntal (qmuntal) requested a review from a team as a code owner October 16, 2026 18:44
@github-actions github-actions Bot added area:agent Changes files in the agent area area:examples Changes files in the examples area area:internal Changes files in the internal area area:provider Changes files in the provider area size:xlarge More than 300 changed lines or 10 files labels Oct 16, 2026
@github-actions github-actions Bot added kind:code Changes production behavior or code kind:docs Changes documentation or comments kind:examples Changes examples or example-support metadata kind:tests Changes tests, fixtures, or test infrastructure labels Oct 16, 2026
@github-actions

This comment has been minimized.

@github-actions github-actions Bot added parity-approved Go API consistency review found no parity issues public-api-change Pull Request changes public APIs labels Oct 16, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Shared-handle key inconsistencies can duplicate records, and generated Azure indexes reject accepted key-based ordering.

2 open findings
What changed in this PR

Extends the VectorData foundation from #1293 with vector-backed chat memory, a reusable store facade, and Azure AI Search integration.

Changes:

  • Adds typed and dynamic collection factories with Go 1.26 compatibility and Go 1.27 generic methods.
  • Adds Azure AI Search authentication, schema management, CRUD, filtering, and vector/hybrid search.
  • Adds memory examples, telemetry coverage, and compatibility documentation.
File Description
vectordata/​vectordata.go Moves store contracts into the facade.
vectordata/​store.go Adds provider-independent collection factories.
vectordata/​store_go127.go Adds the generic collection method.
vectordata/​store_go127_test.go Tests generic collection factories.
vectordata/​providerservices/​filter_test.go Migrates schema configuration.
vectordata/​providerservices/​embedding.go Adds shared vector helpers.
vectordata/​providerservices/​embedding_test.go Tests vector representations and generation.
vectordata/​providerservices/​builder.go Resolves runtime types and generator defaults.
vectordata/​inmemory/​vector.go Removes duplicated vector helpers.
vectordata/​inmemory/​store.go Supports differently typed collections.
vectordata/​inmemory/​snapshot.go Adapts snapshots to shared stores.
vectordata/​inmemory/​snapshot_test.go Migrates snapshot regression tests.
vectordata/​inmemory/​search.go Adapts search to shared storage.
vectordata/​inmemory/​search_test.go Migrates search configuration tests.
vectordata/​inmemory/​record.go Adds runtime key and record handling.
vectordata/​inmemory/​embedding_test.go Tests generator precedence and ownership.
vectordata/​inmemory/​collection.go Adapts collection operations to runtime types.
vectordata/​definition.go Documents generator precedence.
vectordata/​collection.go Adds typed collection adapters.
README.md Introduces memory and Azure Search capabilities.
provider/​azureprovider/​search.go Implements filtered, vector, and hybrid retrieval.
provider/​azureprovider/​search_test.go Tests search requests and pagination.
provider/​azureprovider/​schema.go Maps schemas to Azure indexes.
provider/​azureprovider/​schema_test.go Tests schema mappings and validation.
provider/​azureprovider/​record.go Encodes and decodes Azure documents.
provider/​azureprovider/​README.md Documents connector usage and limitations.
provider/​azureprovider/​filter.go Translates portable filters to OData.
provider/​azureprovider/​filter_test.go Tests filter translation.
provider/​azureprovider/​embedding_test.go Tests generation and memory integration.
provider/​azureprovider/​collection.go Implements collection lifecycle and CRUD.
provider/​azureprovider/​client.go Adds authenticated REST transport.
provider/​azureprovider/​client_test.go Tests redirects and cancellation.
provider/​azureprovider/​aisearch.go Adds the Azure Search store.
provider/​azureprovider/​aisearch_test.go Tests authentication and lifecycle operations.
provider/​azureprovider/​aisearch_go127_test.go Tests generic Azure collection access.
provider/​azureprovider/​aisearch_e2e_test.go Adds opt-in live integration tests.
internal/​telemetry/​featureusage.go Registers chat-history memory usage.
internal/​telemetry/​featureusage_activation_test.go Tests memory telemetry activation.
examples/​internal/​demo/​foundry.go Updates the default embedding model.
examples/​02-agents/​rag/​step01_basic_text_rag/​main.go Migrates collection configuration.
examples/​02-agents/​rag/​README.md Updates embedding defaults documentation.
examples/​02-agents/​providers/​azure/​ai_search/​README.md Documents Azure sample setup.
examples/​02-agents/​providers/​azure/​ai_search/​main.go Demonstrates scoped Azure searches.
examples/​02-agents/​memory/​step05_bounded_chat_history/​main.go Demonstrates bounded history with memory.
examples/​02-agents/​memory/​step05_bounded_chat_history/​history.go Archives and recalls overflow messages.
examples/​02-agents/​memory/​step01_chat_history_memory/​main.go Demonstrates cross-session recall.
examples/​02-agents/​memory/​README.md Documents memory examples.
examples/​02-agents/​agents/​step22_foundry_memory/​main.go Updates the default embedding deployment.
embedding/​options_test.go Updates embedding option fixtures.
embedding/​embedding_test.go Updates embedding metadata fixtures.
docs/​feature-usage-telemetry.md Documents memory activation tracking.
docs/​dotnet-go-sdk-feature-comparison.md Documents new capabilities and Go differences.

🧠 Review effort: Balanced


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread vectordata/inmemory/record.go Outdated
Comment thread provider/azureprovider/schema.go Outdated
Quim Muntal (qmuntal) and others added 3 commits October 17, 2026 15:11
Use the declared schema key type for storage and lookup instead of the handle key type. Cover shared typed/dynamic upserts, replacements, deletions, and snapshot merges without changing input records.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Mark generated index keys sortable to support accepted GetFiltered ordering. Assert the schema flag and storage-name translation and exercise ascending and descending UUID key ordering in the live CRUD test.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Assert expected iterator errors and capture the tool-run error channel instead of discarding them. Satisfy golangci-lint 2.15 errcheck diagnostics without weakening the lint configuration.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions github-actions Bot added area:provider/openai Changes files in the provider / openai area area:provider/otel Changes files in the provider / otel area area:workflow Changes files in the workflow area labels Oct 17, 2026
Keep the upstream errcheck test fixes and FileAccess catalog updates. This clears the conflicting merge base so pull-request test and lint workflows can run.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions github-actions Bot removed area:provider/openai Changes files in the provider / openai area area:provider/otel Changes files in the provider / otel area area:workflow Changes files in the workflow area labels Oct 17, 2026
@github-actions

Copy link
Copy Markdown
Contributor

API consistency review

Result Scope
aligned public API, user-visible behavior

Changed Go contract

Re-review of the three commits added since the prior review (2578ecb, 7a5a78b, 0255a18):

  • vectordata/inmemory: Collection[K,R] no longer tracks a separate keyType field internally; Get, Delete, Upsert, and snapshot Import now normalize assignable keys to the schema's declared key type via an unexported normalizeKey helper before storing/looking up records. No exported identifiers changed; this is a bug fix so that typed and dynamic collection handles sharing the same backing map resolve to the same logical key (previously a [2]int-shaped key value supplied through a dynamic handle could silently diverge from the typed handle's key representation).
  • provider/azureprovider: generated Azure AI Search index key fields are now also marked Sortable: true (schema.go), so GetFiltered with vectordata.Sort{Property: "<key field>"} now works against the key field, matching the already-existing, already-reviewed vectordata.Sort/FilteredRecordRetrievalOptions.OrderBy contract. No new exported identifiers.
  • 0255a18: test-only changes (asserting iterator-yielded errors instead of discarding them) to satisfy golangci-lint 2.15 errcheck; out of scope per the review charter (tests are excluded).

Upstream evidence reviewed

No new upstream comparison was required: these commits are bug fixes to already-reviewed functionality (shared in-memory key handling and Azure AI Search index schema generation) from the prior aligned review of this PR (see the earlier review comment), not new features or behavior divergent from the .NET/Python implementations already cited there (Microsoft.Extensions.VectorData store/collection/filter model and the Azure AI Search connector pattern). Making the key field sortable brings Go's Azure AI Search schema generation in line with the general OrderBy/Sort capability that vectordata already exposes for all other fields, which is consistent with upstream's VectorStoreCollection<TKey,TRecord>.GetAsync filtered-search contract allowing ordering by any indexed/sortable field, including the key.

Assessment

No parity issues. These are internal correctness fixes (in-memory key normalization, Azure AI Search key-field sortability) with no exported API changes and no divergence from upstream semantics; they complete behavior already covered by the prior aligned review rather than introducing new public surface or new cross-repo inconsistency.

Generated by Go API Consistency Review Agent for #1297 · copilot · auto · 53.9 AIC · ⌖ 9.26 AIC · ⊞ 13.9K · ◷

@qmuntal

Copy link
Copy Markdown
Member Author

Addressed both review findings and verified the latest head (337b6bf4d):

  • 2578ecbed: normalize shared-handle in-memory keys across storage, lookup, deletion, and snapshot import, without changing input records. Added replacement and import-merge regressions.
  • 7a5a78bac: make generated Azure Search key fields sortable. Added schema/request assertions and live ascending/descending key-ordering coverage.
  • Merged current main to retain its matching errcheck test fixes and resolve the conflicts that prevented pull-request CI from starting. No lint rules were disabled.

Validation: 26 checks passed, 2 skipped, no failures, including root/catalog lint and the complete Go 1.26/1.27 platform test matrix. Affected local tests and vet also pass. The live Azure native CRUD/key-sorting test passed, and its temporary index was deleted.

Both review threads have been answered and resolved.

@qmuntal
Quim Muntal (qmuntal) added this pull request to the merge queue Oct 19, 2026
Merged via the queue into main with commit 1ac53ce Oct 19, 2026
28 checks passed
@qmuntal
Quim Muntal (qmuntal) deleted the feat/vectordata-memory-azure-search branch October 19, 2026 09:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:agent Changes files in the agent area area:examples Changes files in the examples area area:internal Changes files in the internal area area:provider Changes files in the provider area kind:code Changes production behavior or code kind:docs Changes documentation or comments kind:examples Changes examples or example-support metadata kind:tests Changes tests, fixtures, or test infrastructure parity-approved Go API consistency review found no parity issues public-api-change Pull Request changes public APIs size:xlarge More than 300 changed lines or 10 files

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants