[None][docs] drop deleted accuracy_eval/gen_server_config from disagg benchmark README - #18969
Conversation
… benchmark README Signed-off-by: ADou <ikun3.1415927@gmail.com>
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Enterprise Run ID: 📒 Files selected for processing (1)
Included review availability: Your plan provides up to 12 included reviews per hour; 7 remain after this review. WalkthroughThe benchmark README now documents ChangesBenchmark documentation
Priority: ⬇️ Low Estimated code review effort: 1 (Trivial) | ~3 minutes Merge Risk: ⚪ Minimal · up to Benchmark documentation now reflects the current accuracy-evaluation workflow and runner scripts, with no remaining merge-readiness risk identified. Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
What
In
examples/disaggregated/slurm/benchmark/README.md:accuracy_eval.shandgen_server_config.py.submit.pyinvokeslm_evaldirectly when accuracy evaluation is enabled.config.yaml/submit.py(per-task nested schema).run_benchmark_aiperf.shalongside the other benchmark runners.Why
accuracy_eval.shwas removed in [None] [feat] Support multiple accuracy tasks for slurm scripts #10500 (accuracy is inlined vialm_evalinsubmit.py).gen_server_config.pywas removed in [None] [feat] Enhancements to slurm scripts #10112.tasks: "gsm8k,..."sample no longer matches the live config schema.How I checked
Verified on main @ 96a25c4:
accuracy_eval.sh/gen_server_config.pyreturn 404 underexamples/disaggregated/andexamples/disaggregated/slurm/benchmark/.run_benchmark_aiperf.sh, which the README omitted.config.yamluses nestedaccuracy.tasks.<name>withmodel/model_args_extra/extra_kwargs.submit.pywrites per-task logs underaccuracy_eval_<task>and invokeslm_evaldirectly.Dev Engineer Review
examples/disaggregated/slurm/benchmark/README.mdto match the current benchmark workflow.accuracy_eval.shandgen_server_config.pyscripts.lm_evalexecution throughsubmit.py.gsm8k.run_benchmark_aiperf.shto the benchmark runner documentation.QA Engineer Review
No test changes.
Per-File QA Perspective
examples/disaggregated/slurm/benchmark/README.md: Documentation-only change. Verify that the documentedlm_evalcommand, nested accuracy configuration, per-task result directories, andrun_benchmark_aiperf.shusage match the current scripts and produce the expected benchmark outputs.