Skip to content

feat: timed field-index creation; bfb create-field-index / drop-field-index#156

Closed
tellet-q wants to merge 2 commits into
feat/memory-reportingfrom
feat/timed-field-index-creation
Closed

feat: timed field-index creation; bfb create-field-index / drop-field-index#156
tellet-q wants to merge 2 commits into
feat/memory-reportingfrom
feat/timed-field-index-creation

Conversation

@tellet-q

@tellet-q tellet-q commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Note: base is not dev! Only merge after #158

Add a dedicated sub-command to create payload indices and time the build:

bfb upload --file config.yaml -n 1M --skip-field-indices
bfb create-field-index --file config.yaml --json field-index.json

- Field index creation ---
color (keyword): 0.316 s (server 0.277 s)
price (integer): 0.316 s (server 0.287 s)
Total: 0.632 s

Each index is created with wait=true, so elapsed time reflects the build, not the round-trip. Indices build sequentially, so Total is the cross-run number; to attribute cost to one field, build it alone and re-measure on the same data with the companion drop-field-index:

bfb create-field-index --file config.yaml --field color
bfb drop-field-index   --file config.yaml --field color

Per-field timings land under results.create_field_index. config.num_vectors is now Option — omitted for phase-only commands that neither upload nor query.

@tellet-q
tellet-q force-pushed the feat/timed-field-index-creation branch from c768fd5 to 883eef8 Compare July 16, 2026 06:48
@tellet-q
tellet-q requested a review from Copilot July 16, 2026 06:50

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds CLI support to create and drop payload field indices after uploading data, capturing per-field wall-clock/server timings and recording them in the JSON results. It also adjusts the results schema so phase-only commands don’t emit a misleading default config.num_vectors, and refactors field-index creation into reusable “spec” builders for both CLI- and YAML-config-driven collection setup.

Changes:

  • Add bfb create-field-index / bfb drop-field-index commands and emit per-field timings under results.create_field_index.
  • Refactor field-index request construction into pure spec builders and centralize timed creation/drop logic in collection.
  • Make config.num_vectors optional and omitted for phase-only commands; update JSON-related tests and docs accordingly.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
src/results.rs Adds create_field_index results phase + per-field timing structs; makes config.num_vectors optional and updates tests.
src/main.rs Wires new subcommands, runs field-index creation with timing + optional index/memory follow-up, and passes num_vectors into results.
src/collection/mod.rs Implements timed create_field_indices, drop_field_indices, and improves wait_index robustness/progress output.
src/collection/from_config.rs Exposes field_index_specs_from_config (pure) and reuses the shared timed creation function.
src/collection/from_args.rs Builds field-index specs from CLI flags (pure) and reuses the shared timed creation function.
src/args/mod.rs Adds clap subcommands/args for create-field-index and drop-field-index.
src/optimizations.rs Updates comments/tests to reflect field-index-triggered optimizer work attribution.
README.md Documents the new commands and clarifies JSON output semantics (one wording issue noted).
DEVELOPMENT.md Updates module description to include timed field-index creation.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread README.md Outdated
@tellet-q
tellet-q force-pushed the feat/memory-reporting branch from 3412d4a to 13a555e Compare July 16, 2026 08:03
@tellet-q
tellet-q force-pushed the feat/timed-field-index-creation branch from 883eef8 to 3af4fda Compare July 16, 2026 08:05
tellet-q and others added 2 commits July 16, 2026 10:17
…eld-index`

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
@tellet-q
tellet-q force-pushed the feat/memory-reporting branch from 13a555e to f398de7 Compare July 16, 2026 08:18
@tellet-q
tellet-q force-pushed the feat/timed-field-index-creation branch from 9ce4163 to 2e2e98a Compare July 16, 2026 08:18
@tellet-q tellet-q closed this Jul 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants