Skip to content

Add memory reporting#158

Closed
tellet-q wants to merge 2 commits into
devfrom
feat/memory-reporting
Closed

Add memory reporting#158
tellet-q wants to merge 2 commits into
devfrom
feat/memory-reporting

Conversation

@tellet-q

Copy link
Copy Markdown
Contributor

Sample Qdrant's memory/disk telemetry after indexing (REST) and report it under results.memory in --json:

--- Memory & disk ---
total: 154.1 MiB disk, 6.2 MiB ram, 3.9 MiB cached (of 72.8 MiB expected)
vector (default): storage 66.0 MiB disk / 0.0 MiB ram, index 6.8 MiB disk / 0.0 MiB ram
payload: 67.1 MiB disk, 0.0 MiB ram
payload index color: 12.2 MiB disk, 2.7 MiB ram
process: 74.8 MiB resident, 49.0 MiB allocated (server-wide)

cached is what the page cache holds of the on-disk bytes vs. the expected a component wants cached to serve queries without hitting disk — the gap shows how cold the cache is. Runs automatically in bfb upload and legacy runs; gated off by the existing --skip-server-stats.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds post-indexing memory/disk telemetry sampling from Qdrant’s REST API and surfaces it both in stdout (“--- Memory & disk ---”) and in the JSON output under results.memory, with an opt-out flag.

Changes:

  • Introduces src/memory.rs with REST wire types, fetching (/collections/{c}/memory, best-effort /telemetry), and a human-readable printer.
  • Extends results JSON schema (PhaseResults) to include optional memory: Option<MemoryReport>.
  • Adds --skip-server-stats to disable the REST-only sampling and documents the feature in README/DEVELOPMENT.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
tests/fixtures/collection_memory.json Adds a real-world /collections/{c}/memory fixture used by parsing tests.
src/results.rs Adds results.memory to the serialized output structure and updates roundtrip tests.
src/memory.rs Implements REST fetching/parsing + pretty-printed memory/disk report and unit tests.
src/main.rs Calls memory sampling after indexing in upload and legacy runs; handles best-effort failures.
src/args/mod.rs Adds the global --skip-server-stats flag.
README.md Documents memory/disk reporting behavior, JSON field, and port mapping.
DEVELOPMENT.md Adds memory.rs to the source map documentation.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/main.rs Outdated
@tellet-q
tellet-q force-pushed the feat/memory-reporting branch from 13a555e to f398de7 Compare July 16, 2026 08:18
@tellet-q tellet-q closed this Jul 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants