Skip to content

feat(azureprovider): add Cosmos DB vector store - #1304

Merged
George Adams (gdams) merged 2 commits into
mainfrom
feat/azure-cosmosdb
Oct 20, 2026
Merged

George Adams (gdams) merged 2 commits into
mainfrom
feat/azure-cosmosdb

Conversation

@qmuntal

Copy link
Copy Markdown
Member

Summary

  • Port the CommunityToolkit Cosmos DB connector to azureprovider.CosmosDB, using stable azcosmos v1.5.0 and a caller-owned native client.
  • Support container lifecycle, typed and dynamic records, CRUD and sequential batches, generated UUIDs, scalar/composite lookup keys, and hierarchical/nullable partition keys.
  • Add float32/uint8/int8 vectors, embedding generation, portable filters, vector search, and full-text/vector hybrid search.
  • Include Go documentation, a runnable example, protocol regression tests, and four opt-in live integration tests.

Compatibility and limits

  • The client and database must already exist. The provider does not close the borrowed client or configure account-level capabilities.
  • Stable azcosmos does not execute cross-partition ranking/ordering. Vector search, hybrid search, and ordered retrieval require equality filters for every partition-key property; unsupported queries fail before service I/O or embedding generation.
  • Unordered filtered retrieval supports cross-partition paging and applies Skip/top while iterating.
  • Generated-vector collections do not support IncludeVectors; sequential batches may partially succeed. These contracts are documented in Go comments.
  • No preview SDK or experimental query engine is introduced.

Validation

  • Focused provider and VectorData tests passed.
  • Cosmos regression tests and all four live integration tests passed on Go 1.26.1 and Go 1.27.0.
  • Live validation used a real Entra ID-only, serverless Cosmos DB account and real 3072-dimensional Foundry embeddings. It covered native container lifecycle, CRUD/batches, all supported vector types, score thresholds, hybrid ranking, dynamic records, hierarchical numeric/null partition keys, named keys, cross-partition paging, and cancellation.
  • The example ran successfully against the real services; the Go 1.27 example build also passed.
  • golangci-lint v2.15.0 reported zero issues.
  • All temporary Cosmos resources were deleted after validation. Live tests remain opt-in and clean up only their own containers.

Port the CommunityToolkit Cosmos DB connector using a caller-owned stable azcosmos client. Add typed and dynamic records, partition-aware vector and hybrid search, embedding generation, Go documentation, an example, and opt-in live integration tests.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@qmuntal
Quim Muntal (qmuntal) requested a review from a team as a code owner October 19, 2026 14:00
Copilot AI balanced review requested due to automatic review settings October 19, 2026 14:00
@github-actions github-actions Bot added area:examples Changes files in the examples area area:provider Changes files in the provider area size:xlarge More than 300 changed lines or 10 files labels Oct 19, 2026
@github-actions github-actions Bot added kind:code Changes production behavior or code kind:dependencies Changes dependencies or manifests kind:examples Changes examples or example-support metadata kind:tests Changes tests, fixtures, or test infrastructure labels Oct 19, 2026
@github-actions

This comment has been minimized.

@github-actions github-actions Bot added parity-approved Go API consistency review found no parity issues public-api-change Pull Request changes public APIs labels Oct 19, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Unresolved schema-path, score-field collision, and timestamp-query defects affect correctness.

4 open issues
What changed in this PR

Adds an Azure Cosmos DB backend to VectorData using a caller-owned native client.

Changes:

  • Supports container lifecycle, typed/dynamic records, CRUD, and partition keys.
  • Adds embedding generation and partition-scoped vector/hybrid search.
  • Includes protocol tests, opt-in live tests, and a runnable example.
File Description
provider/​azureprovider/​cosmosdb.go Store configuration and public contracts.
provider/​azureprovider/​cosmosdb_test.go Protocol regression tests.
provider/​azureprovider/​cosmosdb_query.go Filters, routing, and ranked queries.
provider/​azureprovider/​cosmosdb_e2e_test.go Four opt-in live integration tests.
provider/​azureprovider/​cosmosdb_collection.go Container policies, record mapping, CRUD, and paging.
go.sum Cosmos SDK dependency checksums.
go.mod Adds stable azcosmos dependency.
examples/​02-agents/​providers/​azure/​cosmosdb/​main.go Demonstrates generated embeddings and scoped search.

Review effort: Balanced

case *providerservices.VectorPropertyModel:
c.vectors = append(c.vectors, p)
}
c.properties[propertyName(property)] = property
Comment on lines +290 to +291
value = strings.ReplaceAll(value, "~", "~0")
return strings.ReplaceAll(value, "/", "~1")
return nil, err
}
if found {
document[property.StorageName] = value
yield(zero, cosmosFailure(c.store.databaseName, c.name, "GetFiltered", err))
return
}
query := "SELECT VALUE c FROM c"
Reserve the internal score field, quote Cosmos schema paths, normalize record and filter timestamps to fixed-width UTC, and omit unrequested vectors from filtered queries. Add protocol regressions and document the wire contracts.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions

Copy link
Copy Markdown
Contributor

API consistency review

Result Scope
aligned public API

Changed Go contract

New azureprovider exports: CosmosDB (embeds *vectordata.Store), NewCosmosDB(client *azcosmos.Client, databaseName string, config CosmosDBConfig) (*CosmosDB, error), CosmosDBConfig{PartitionKeyProperties, IndexingMode, Automatic, EmbeddingGenerator}, and CosmosKey{DocumentID, PartitionKey}. Internal cosmosDBStore implements vectordata.StoreProvider, plus a new example under examples/02-agents/providers/azure/cosmosdb/.

Upstream evidence reviewed

Implementation Source Contract checked
C# (CommunityToolkit MEVD, cited in PR description) CosmosCollectionOptions.cs PartitionKeyProperties (null ⇒ document ID is the partition key, string/Guid key type only), IndexingMode (default Consistent), Automatic (default true) — all match CosmosDBConfig defaults and semantics in cosmosdb.go
C# (CommunityToolkit MEVD) CosmosKey.cs Composite document-ID + partition-key struct; Go's CosmosKey{DocumentID, PartitionKey} is the idiomatic Go analog (slice of any vs. hierarchical PartitionKey builder)
C# (CommunityToolkit MEVD) CosmosCollection.cs (lines ~359, 396, 543, 648) IncludeVectors throws NotSupportedException when embedding generation is required — Go's documented "IncludeVectors is unsupported on collections with generated vector fields" matches
C# (CommunityToolkit MEVD) CosmosModelBuilder.cs Key property type restricted to string/Guid; vector types restricted to float/byte/sbyte — matches Go's string/uuid.UUID keys and float32/uint8/int8 vectors
C# (CommunityToolkit MEVD) CosmosCollectionQueryBuilder.cs (RRF ranking, lines ~84-139) Hybrid search uses RANK RRF(...), with ordering/offset-limit constraints — Go's documented reciprocal-rank-fusion hybrid search and cross-partition ordering limitations align

Assessment

This PR is an explicitly documented, scoped port of the CommunityToolkit MEVD Azure Cosmos DB connector, using stable azcosmos instead of the preview SDK used upstream. Defaults (consistent indexing mode, automatic indexing enabled, document-ID-as-partition-key when no partition properties are configured), supported key/data/vector types, and documented constraints (IncludeVectors incompatible with generated vectors, cross-partition ordering requiring equality filters) all match the cited upstream connector. Naming follows existing Go vectordata/azureprovider conventions (e.g., parallels AISearchConfig/NewAISearch). No parity gaps were found; test coverage (26 unit tests plus 4 opt-in live integration tests) and the new example are consistent with the existing aisearch.go provider pattern in this repo.

Generated by Go API Consistency Review Agent for #1304 · copilot · auto · 117.9 AIC · ⌖ 13.8 AIC · ⊞ 14K · ◷

@gdams
George Adams (gdams) added this pull request to the merge queue Oct 20, 2026
Merged via the queue into main with commit c3544c3 Oct 20, 2026
28 checks passed
@gdams
George Adams (gdams) deleted the feat/azure-cosmosdb branch October 20, 2026 07:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:examples Changes files in the examples area area:provider Changes files in the provider area kind:code Changes production behavior or code kind:dependencies Changes dependencies or manifests kind:examples Changes examples or example-support metadata kind:tests Changes tests, fixtures, or test infrastructure parity-approved Go API consistency review found no parity issues public-api-change Pull Request changes public APIs size:xlarge More than 300 changed lines or 10 files

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants