fix(train): do not refuse publication for below-floor labels

[dharrawal]

Summary

• Follow-up to Fix training behavior for thin labels and missing provenance #75: that PR stopped aborting mid-train over a label with fewer than 2 rows, but BELOW_FLOOR remained in BLOCKING_STATUSES, so publication was still refused at the report gate (Training data is structurally incomplete; refusing to publish models for: initialize_defect_info).
• Remove BELOW_FLOOR from the publication blockers. A 1-row command still trains, is reported as unmeasured, and does not fail the run. NO_UTTERANCES and true MISSING (trainer never reached the command) still block.

Test plan

• pytest tests/test_training_report.py tests/test_heldout_evaluation.py (88 passed)
• Confirm a workflow with a single-utterance command (talk_to_ido / initialize_defect_info) publishes after train

Made with Cursor

Summary by Sourcery

Allow below-floor labels to publish while continuing to block runs with missing or empty training data.

Bug Fixes:

• Allow labels with fewer than the evaluation row floor to train and publish without treating them as blocking failures.

Enhancements:

• Update training reports to identify below-floor labels as unmeasured and advisory while retaining publication blocking for missing or empty training data.

Tests:

• Update training report tests to verify below-floor labels are non-blocking and explicitly reported as unmeasured.

[sourcery-ai]

Reviewer's Guide

The report gate now allows publication when labels have fewer rows than the train/evaluate floor: they still train and are reported as unmeasured, while empty or genuinely missing training data continues to block publication.

Flow diagram for training report publication gating

flowchart TD
    A[Training report evaluates label status] --> B{Row status}
    B -->|BELOW_FLOOR| C[Train label]
    C --> D[Report as unmeasured]
    D --> E[Allow publication]
    B -->|NO_UTTERANCES or MISSING| F[Block publication]
    B -->|Other advisory status| G[Report finding]
    G --> E

Loading

File-Level Changes

Change	Details	Files
Makes below-floor labels advisory instead of publication-blocking while preserving their unmeasured classification.	Removes BELOW_FLOOR from BLOCKING_STATUSES, leaving NO_UTTERANCES and MISSING as blockers. Updates report documentation and output to explain that below-floor labels train but lack evaluation coverage and do not block publication. Adds assertions covering non-blocking status, report exclusion, and revised wording.	fastworkflow/train/training_report.py tests/test_training_report.py

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[sourcery-ai]

Hey - I've found 1 issue

Prompt for AI Agents

Please address the comments from this code review:

## Individual Comments

### Comment 1
<location path="fastworkflow/train/training_report.py" line_range="194" />
<code_context>
+#: evaluation row. FELL_BACK is reported loudly but does not fail when rows remain.
 BLOCKING_STATUSES: frozenset[RowStatus] = frozenset(
     {
-        RowStatus.BELOW_FLOOR,
         RowStatus.NO_UTTERANCES,
         RowStatus.MISSING,
</code_context>
<issue_to_address>
**issue (broader_impact):** Removing `BELOW_FLOOR` from the publication blockers does not make a context containing only below-floor labels trainable: the unchanged `split_training_data` raises `TrainingDataError` when no label has enough rows to form an evaluation set. `train_workflow` therefore aborts before reaching the publication gate, so a single-label or otherwise all-one-row workflow still fails instead of training and publishing.

**Triggers:** When a training context has no label with at least two rows, such as a single-command context whose command has one utterance.

**Suggested fix:** Update the training path to support an all-below-floor context with an explicitly unmeasured/no-threshold model, or retain a blocking status for contexts where no evaluable label exists.
</issue_to_address>

Sourcery assessment

Needs a human reviewer. 1 finding to address first, and labels with fewer than the train-and-evaluate floor will now be published despite having no evaluation coverage, so an incorrect floor policy could put poorly validated routing behavior into a published model. Reverting stops future publications, but any already-published model would need rollback or retraining to repair.

Blocking findings: fastworkflow/train/training_report.py:194

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}