Skip to content

Update storage docs based on gaps identified#231

Merged
prrao87 merged 1 commit into
mainfrom
feat/update-storage
May 6, 2026
Merged

Update storage docs based on gaps identified#231
prrao87 merged 1 commit into
mainfrom
feat/update-storage

Conversation

@prrao87
Copy link
Copy Markdown
Contributor

@prrao87 prrao87 commented May 6, 2026

Address findings from the storage docs-gap audit and clarify OSS vs Enterprise scope.

Architecture page

  • Called out that plain S3 is single-writer-safe and concurrent writers need the DynamoDB commit store, with deeplinks to the relevant configuration sections.
  • Added a "File-format choices that interact with the backend" section explaining how new_table_enable_v2_manifest_paths, new_table_enable_stable_row_ids, and new_table_data_storage_version map onto backend choice.

Configuration page

  • Added an Enterprise scoping note up top: this page is OSS-only; in Enterprise the cluster owns storage credentials, and federated databases use namespace-driven credential vending. Links out to the Enterprise quickstart and Azure deployment guide.
  • Made connection-vs-table storage_options override semantics explicit ("inherit from connection, override per key").
  • Documented the initial_storage_options() / latest_storage_options() accessors on AsyncTable and the deprecation of storage_options().
  • Noted that the new_table_* options are evaluated only at table creation and do not rewrite existing tables.
  • Extended the DynamoDB section with required IAM actions and a dynamodb_endpoint example for LocalStack-style local dev.
  • New "Server-side encryption with KMS" section covering aws_server_side_encryption / aws_sse_kms_key_id and the required KMS IAM actions.
  • Documented the actual Azure SAS-token keys (azure_storage_account_name, azure_storage_sas_token) with a snippet, instead of mentioning "SAS tokens" generically.
  • Removed the StorageOptionsProvider "Python-only" section. Cross-checking against lancedb confirmed storage_options_provider= is not part of the Python public API; the only path is namespace_client=, which belongs in dedicated namespace docs rather than a half-baked replacement here.

Snippets / tests

  • New runnable test snippets for the local DynamoDB endpoint, SSE-KMS, and Azure SAS examples; regenerated MDX exports.
  • Removed the broken provider snippet (and its now-unused mock parameter).

@mintlify
Copy link
Copy Markdown
Contributor

mintlify Bot commented May 6, 2026

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated (UTC)
lancedb-bcbb4faf 🟢 Ready View Preview May 6, 2026, 6:12 PM

💡 Tip: Enable Workflows to automatically generate PRs for you.

@prrao87 prrao87 merged commit cfef13e into main May 6, 2026
2 checks passed
@prrao87 prrao87 deleted the feat/update-storage branch May 6, 2026 18:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant