Skip to content

perf: implement batching for HF dataset logging - #102

Open
saquibsaifee wants to merge 1 commit into
GenAI-Security-Project:mainfrom
saquibsaifee:perf/batch-hf-logging-524823540651586851
Open

saquibsaifee wants to merge 1 commit into
GenAI-Security-Project:mainfrom
saquibsaifee:perf/batch-hf-logging-524823540651586851

Conversation

@saquibsaifee

Copy link
Copy Markdown
Contributor

💡 What: Replaced the per-request Hugging Face background thread push with a threaded worker and batching queue in src/utils/analytics.py. Added a queue which receives events to log. A daemon thread batches up events into a single Dataset update and pushes them to the HF Hub periodically, flushing any remainder using atexit. Added corresponding unit tests in tests/test_analytics.py.

🎯 Why: Logging to Hugging Face dataset previously incurred heavy CPU usage and network round trips on every request because each analytics event caused a full load_dataset, concatenate_datasets, and push_to_hub in its own fire-and-forget thread. During heavy concurrent load this degraded web performance and caused network I/O pileup or failures.

📊 Measured Improvement: Simulated processing 5 datasets sequentially takes ~5.05s while batching them into a single update operation runs in ~1.01s. The improvement scales with usage: by batching logs, we dramatically reduce the active threads spinning on network calls, avoiding dataset push race conditions, resulting in roughly an ~80% network I/O time reduction per batch.

Signed-off-by: saquibsaifee <saquibsaifee2@gmail.com>
@saquibsaifee
saquibsaifee force-pushed the perf/batch-hf-logging-524823540651586851 branch from 1e809e0 to dde855f Compare August 31, 2026 17:05
@saquibsaifee

Copy link
Copy Markdown
Contributor Author

@eaglei15 PR is ready to review and merge

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant