Fix training behavior for thin labels and missing provenance

[dharrawal]

Training was failing for commands with a single utterance. Also missing provenance

Summary by Sourcery

Make training resilient to thin intent labels and distinguish absent generation provenance from commands that trained successfully without using a generator.

Bug Fixes:

• Allow training to proceed when individual intent labels have too few rows for evaluation, while keeping those labels unmeasured and warning with readable command names.
• Correct training-report classification so hand-written utterances without generation provenance are not incorrectly reported as missing, while still enforcing row-count floors.

Enhancements:

• Improve label formatting in warnings and errors so decoded names remain readable and stable in console output.
• Retain a fatal error when no labels can produce an evaluation set at all.

Build:

• Bump the package version from 3.2.0 to 3.2.1.

Tests:

• Update training split and report coverage for thin labels, readable warnings, missing provenance, and floor enforcement.

[sourcery-ai]

Reviewer's Guide

The PR makes training resilient to single-utterance labels by training them without held-out evaluation, reports those labels with decoded rich-safe warnings, and distinguishes absent generation provenance from commands proven to have trained through context provenance; the package version is bumped to 3.2.1.

Flow diagram for resilient thin-label training split

flowchart TD
    A[Training dataset] --> B[Count rows per label]
    B --> C{Label below minimum rows?}
    C -- Yes --> D[Warn with decoded label names]
    D --> E[Keep rows in training set]
    C -- No --> F[Stratified train/evaluation split]
    E --> G[Train model]
    F --> G
    B --> H{All labels below floor?}
    H -- Yes --> I[Raise TrainingDataError]

Loading

File-Level Changes

Change	Details	Files
Allow thin labels to train while explicitly tracking when evaluation is impossible.	Partition labels below the shared minimum-row floor out of stratified splitting. Append unmeasured rows to training data and emit stable, decoded warning messages. Retain a fatal error when no label can produce an evaluation row or the remaining data cannot be split. Add regression coverage for training inclusion, evaluation exclusion, warnings, label decoding, and rich-console-safe formatting.	fastworkflow/model_pipeline_training.py tests/test_heldout_evaluation.py tests/test_training_report.py
Refine provenance classification so hand-written utterances are not falsely reported as missing provenance.	Require both absent generation provenance and absent included-context provenance for MISSING status. Judge commands proven to have trained via context provenance using row-count floor rules. Preserve blocking behavior for hand-written commands below the minimum row floor. Add report tests covering non-missing publication and below-floor blocking.	fastworkflow/train/training_report.py tests/test_training_report.py
Publish the bug-fix release as version 3.2.1.	Increment the package version from 3.2.0 to 3.2.1.	pyproject.toml

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[sourcery-ai]

Hey - I've found 1 issue

Prompt for AI Agents

Please address the comments from this code review:

## Individual Comments

### Comment 1
<location path="fastworkflow/train/training_report.py" line_range="698-701" />
<code_context>
         if trains_as_label is False:
             return RowStatus.EXCLUDED
-        return RowStatus.MISSING
+        if not has_included_context:
+            # No generation record AND no sign the trainer ever reached this command:
+            # the run died part-way or provenance was lost. That is what MISSING means.
+            return RowStatus.MISSING
+        # An `UtteranceProvenance` record describes *generation*, and only
+        # `generate_diverse_utterances` writes one. A command whose `generate_utterances`
</code_context>
<issue_to_address>
**issue (broader_impact):** A generated command with missing `UtteranceProvenance` is treated as non-missing whenever any context record has `INCLUDED` status, so the report can return `OK` or `BELOW_FLOOR` and allow publication despite lost generation provenance. `has_included_context` does not distinguish hand-written utterances, which never owed a generation record, from generated utterances whose provenance was lost.

**Triggers:** When generation provenance is absent but the trainer successfully records an included context for the command.

**Suggested fix:** Record whether a command used the generator, or preserve the prior missing-provenance check for generated commands and exempt only commands explicitly known to supply hand-written utterances.
</issue_to_address>

Sourcery assessment

Needs a human reviewer. 1 finding to address first, and if this is wrong, thin-label data or commands without generation provenance could be included in a model and published without the intended evaluation or blocking signal. Reverting stops the behavior for future runs, but already-produced models or reports would need to be regenerated.

Blocking findings: fastworkflow/train/training_report.py:701

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

[drawal1]

the observation is valid; treating INCLUDED as “training happened” is intentional; lost generation metadata is an observability issue, not a publication gate; track a follow-up to record handwritten vs generated explicitly rather than blocking 3.2.1

[cursor]

Post-merge CVE scan (after #75)

Re-ran Trivy, Grype, Syft, osv-scanner, Dockle, and Dive against 734cb1f (v3.2.1). Snyk and Docker Scout were skipped (no auth in this environment).

Python / poetry.lock

• Only finding: diskcache==5.6.3 — CVE-2025-69872 / GHSA-w8v5-vhqr-4h9v / PYSEC-2026-2447 (MEDIUM).
• Easy pyproject.toml bumps: none. osv-scanner: "0 vulnerabilities can be fixed"; PyPI latest still 5.6.3.
• With OpenVEX v7, Trivy/Grype report a clean lockfile.

Human-review recommendations

Finding	Rec	Rationale
diskcache pickle RCE	IGNORE	Not remotely reachable; OpenVEX not_affected
Proxy-image OS High/Critical (python:3.13-slim-bookworm)	IGNORE	No first-party Dockerfile; not shipped
Dockle CIS (root/HEALTHCHECK)	IGNORE	Proxy image only
DSPy cache dir hardening note	FIX (optional docs)	Low-cost operator guidance if cache is shared

Follow-up PR

Report refresh landed in #77 (security/reports/*.html + OpenVEX v7).

View PR

  

_{Sent by Cursor Automation: Untitled}