Skip to content

docs(lora): add audited Issue #50 strong-baseline research extension - #184

Open
PinkTulips139 wants to merge 1 commit into
Tencent:mainfrom
PinkTulips139:issue50-research-extension-benchmark
Open

docs(lora): add audited Issue #50 strong-baseline research extension#184
PinkTulips139 wants to merge 1 commit into
Tencent:mainfrom
PinkTulips139:issue50-research-extension-benchmark

Conversation

@PinkTulips139

Copy link
Copy Markdown
Contributor

Summary

Extends the existing Issue #50 LoRA rank-sweep work with an audited research benchmark covering:

  • strong fine-tuning baselines;
  • numerical-stability diagnostics;
  • multi-seed evaluation;
  • formal result-validity semantics;
  • requested-versus-actual batch auditing; and
  • dependency-free result validation.

This PR does not replace historical LoRA rank-sweep results or modify their recorded metrics.

What this adds

  • Head-only, Neck + Head, Last Stage + Neck + Head, Full Fine-tuning, Stable LoRA, and Partial Fine-tuning + LoRA comparisons.
  • Audited formal results containing only valid completed runs.
  • Multi-seed means, sample standard deviations, and best-run values.
  • Separation of requested batch, actual batch, queue execution status, and formal validity.
  • Exclusion of duplicate runs, duplicate result directories, invalid OOM recovery entries, and implementation failures.
  • A dependency-free validation script for CSV schema, unique seeds, sample statistics, and the audited four-objective non-dominated front.

Key results

  • Brain Tumor: Last Stage + Neck + Head reached 0.39172 ± 0.02316 mAP50-95 across seeds 0/1/2, with a best run of 0.40944.
  • VisDrone: Full Fine-tuning with actual batch 4 reached 0.15342 ± 0.00221, with a best run of 0.15527.
  • Stable LoRA did not provide an absolute accuracy advantage under the audited protocols.
  • Head-only remained parameter-efficient on Brain Tumor.

Scope and limitations

Provenance

  • Audited Phase 2 source commit: bb20447
  • Published research-extension commit: 33e62d8e8548b2690e6b1acb9fad9776cd94923a
  • Upstream PR branch commit: 9a33ac3fd1300bd13693c265ef30d6b9810ca1a9

Validation

  • ruff check examples/lora_examples/validate_yolo_master_issue50_research_extension.py
  • python -m py_compile examples/lora_examples/validate_yolo_master_issue50_research_extension.py
  • python examples/lora_examples/validate_yolo_master_issue50_research_extension.py
  • git diff --check

The validator confirms 14 valid formal runs, 9 method summaries, unique configuration/seed combinations, sample-standard-deviation semantics, and the audited four-objective non-dominated front.

@PinkTulips139

Copy link
Copy Markdown
Contributor Author

CI note: the only failing job is the legacy Python 3.8 / PyTorch 1.8.0 test environment. The three failures occur in existing Mixture/MoA/MoT tests because torch.autocast is unavailable in PyTorch 1.8. This PR changes only LoRA research documentation, audited CSV tables, and a dependency-free validation script. The modern tests, benchmarks, isolated exports, MixtureDDP, and Mixture regression checks passed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant