perf: implement batching for HF dataset logging - #102
Open
saquibsaifee wants to merge 1 commit into
Open
saquibsaifee wants to merge 1 commit into
saquibsaifee wants to merge 1 commit into
Conversation
Signed-off-by: saquibsaifee <saquibsaifee2@gmail.com>
saquibsaifee
force-pushed
the
perf/batch-hf-logging-524823540651586851
branch
from
August 31, 2026 17:05
1e809e0 to
dde855f
Compare
Contributor
Author
|
@eaglei15 PR is ready to review and merge |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
💡 What: Replaced the per-request Hugging Face background thread push with a threaded worker and batching queue in
src/utils/analytics.py. Added a queue which receives events to log. A daemon thread batches up events into a single Dataset update and pushes them to the HF Hub periodically, flushing any remainder usingatexit. Added corresponding unit tests intests/test_analytics.py.🎯 Why: Logging to Hugging Face dataset previously incurred heavy CPU usage and network round trips on every request because each analytics event caused a full
load_dataset,concatenate_datasets, andpush_to_hubin its own fire-and-forget thread. During heavy concurrent load this degraded web performance and caused network I/O pileup or failures.📊 Measured Improvement: Simulated processing 5 datasets sequentially takes ~5.05s while batching them into a single update operation runs in ~1.01s. The improvement scales with usage: by batching logs, we dramatically reduce the active threads spinning on network calls, avoiding dataset push race conditions, resulting in roughly an ~80% network I/O time reduction per batch.