Skip to content

Julenmendieta/milab 6793 last tweaks - #18

Merged
julenmendieta merged 16 commits into
mainfrom
julenmendieta/MILAB-6793_lastTweaks
Sep 3, 2026
Merged

Julenmendieta/milab 6793 last tweaks#18
julenmendieta merged 16 commits into
mainfrom
julenmendieta/MILAB-6793_lastTweaks

Conversation

@julenmendieta

@julenmendieta julenmendieta commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR adjusts fitted-distribution verdict calling, per-barcode QC status, histogram output, workflow exports, and their UI controls. Important touched terms:

  • Bound probability — the minimum fitted signal-component probability required to call a cell bound; it is now configurable from 0.9 to 1 and recorded with the run.
  • Expected binder fraction — the estimated share of binding cells used to initialize the two-component fit; it is now configurable and defaults to 10%.
  • Fitted background — the per-sample, per-barcode count distribution and mixture fit used by the distribution baseline; its bins now include silent cells, expose a bound-count marker, and can be selected by sample.
  • Undeclared barcode share — one undeclared sequence’s share of all pre-refinement reads in its sample; status now evaluates each sequence independently, warning above 1% and alerting above 5%.
  • Raw tag UMI count — the per-cell, per-tag count before the verdict count floor; it is now emitted, imported, and exported for downstream composition views.
  • Tag order — panel declaration order for barcodes; it is now carried in the histogram payload to keep panel positions stable across samples.

Confidence Score: 4/5

The PR should not merge until the Python execution boundary rejects invalid bound-probability values that otherwise produce silently incorrect verdicts.

The newly configurable cutoff can bypass the Python validation and cause all cells to be classified as not bound while the accompanying fitted-background threshold reports contradictory information.

Files Needing Attention: software/per-cell-metrics/src/emit_verdicts.py, software/per-cell-metrics/src/verdict.py, software/per-cell-metrics/src/tag_distribution.py

Important Files Changed

Filename Overview
software/per-cell-metrics/src/emit_verdicts.py Threads new fit settings through verdict generation and adds QC/raw-count artifacts, but bound-probability validation accepts invalid non-finite and above-one values.
software/per-cell-metrics/src/tag_distribution.py Changes mixture initialization, retains per-pair histogram bins and probability curves, and derives display thresholds from fitted probabilities.
software/per-cell-metrics/src/qc_measures.py Adds vectorized status evaluation and absolute log1p histogram edges while changing undeclared-barcode status thresholds.
model/src/index.ts Extends the persisted-to-workflow settings contract, validation, defaults, and fitted-bin output type.
workflow/src/verdict-args.lib.tengo Adds explicit CLI propagation and defaults for bound probability and expected binder fraction.
workflow/src/verdict-import.tpl.tengo Imports the new pre-floor per-cell/tag count artifact into a dedicated result frame.
ui/src/pages/MainPage.vue Adds controls and explanations for fitted-distribution initialization and calling thresholds.
ui/src/pages/AntigenQcPage.vue Hides unstable tabs during execution and scopes fitted-background plots with a sample selector.
ui/src/components/CountHistogram.vue Adapts histogram rendering to shared absolute log1p edges and pads shorter per-pair distributions with empty trailing bins.
ui/src/components/FittedBackgroundGrid.vue Orders barcodes consistently, supports sample scoping, and displays per-fit bound-count markers and diagnostics.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  UI[Verdict settings UI] --> Model[Validated BlockArgs]
  Model --> Args[Workflow CLI arguments]
  Args --> Fit[Per-sample tag distribution fit]
  Fit --> Calls[Cell binding calls]
  Fit --> Bins[Fitted-background bins and thresholds]
  Calls --> Verdicts[Clonotype verdict outputs]
  Bins --> QC[Run-quality UI]
  Raw[Pre-floor cell/tag counts] --> Export[Downstream composition export]
Loading

Fix all with Greploop Fix All in Claude Code

Prompt To Fix All With AI
### Issue 1
software/per-cell-metrics/src/emit_verdicts.py:304-310
**Invalid bound probabilities pass**

When a direct invocation supplies `NaN`, infinity, or a value above 1, this lower-bound-only validation accepts it. `read_states` then classifies every otherwise reliable cell as not bound, while `NaN` makes `bound_at_count` report the lowest count as the threshold, causing the verdicts and fitted-background marker to disagree silently.

```suggestion
    if not DISTRIBUTION_BOUND_PROBABILITY <= args.bound_probability <= 1.0:
        raise SystemExit(
            f"--bound-probability must be at least {DISTRIBUTION_BOUND_PROBABILITY} and at most 1. "
            f"Below that a cell holding none of a tag could be called bound. Most cells hold none of most tags, "
            f"and the run counts them by arithmetic rather than checking each one, so those cells would be counted "
            f"not-bound in one place and called bound in another. Got {args.bound_probability}."
        )
```

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Reviews (1): Last reviewed commit: "Changeset" | Re-trigger Greptile

Greptile also left 1 inline comment on this PR.

Context used (5)

PaulNewling and others added 14 commits August 31, 2026 15:34
`count_bin_edges` built its edges with `np.geomspace`, which places them between
whole numbers. A UMI count is a whole number, so a bin could fall strictly
between two counts and stand empty at every weight a run could produce.
`np.geomspace(1, 5155, 25)` puts one at [2.039, 2.911); on a 15-tag run it held
nothing on all 15 panels and read as a missing bar.

Edges are now whole and strictly increasing, so every bin holds at least one
count. The step is `max(previous + 1, geometric)`, which draws one count per bar
near a count of 1 and stays geometric above that. The last edge is one past the
top count, making every bin half-open rather than closing the last one and
giving it a count more than its width.

A run now takes at most COUNT_BIN_COUNT bins instead of always that many.
Nothing reads the count: `bin_values`, `per_tag_count_bins` and the chart all
take it from the edge list.
Whole-number edges stop a bin standing empty, but they do not make bar heights
comparable. Bin width in counts rises across the edge set -- one real set spans
1, 1, 2, 3, 4, 6, 9, 14, 21 -- so a bin covering 4 counts stood about four times
a neighbour covering 1 at equal density. That step drew a second hump on tags
whose counts hold one population, and the grid exists to answer whether two
populations separated at all, so a hump the bins invented is the one error this
surface cannot carry.

`CountHistogram` takes a `density` flag, applied in the `log-bins` branch only,
which divides each weight by the whole counts its bin spans. The y axis reads
"Cells per count". The two linear callers, the score spread and the reference
reading, are untouched.

Atom 330 binds the x axis -- "the unit the gate is declared in follows from the
axis" -- and only on the plots a scientist declares from. This grid informs
nothing settable, so its y axis carries no such constraint.

`PlChartHistogram` prints the number it is handed under a fixed `count:` label,
so a hovered bar on this grid now reports the density. Each panel's caption
carries the cell count instead.
VIEW_TABS is derived from the rung the run reports, and that rung is unreported
until the run settles. A strip drawn mid-run therefore offered every plot and
then dropped the ones the served rung cannot draw, so a reader could open a tab
that stopped existing under them.

`isRunning` is the block's own computing signal, the one already driving the
block spinner. The open view's body is untouched and keeps drawing its own
processing placeholder, so the section still shows progress while the strip is
away.
The Status column read `readShare`, the sample's aggregate undeclared share, so
it was one word repeated down every row of a sample. A sample carrying one heavy
undeclared sequence among many light ones said nothing about which sequence to
look at.

It now reads `barcodeShare`, the row's own share of its sample's pre-refine
reads, evaluated per row through the same scalar `status_for` every other status
goes through rather than a polars expression rebuilding the thresholds. The
frame is capped at UNDECLARED_BARCODES_KEPT rows per sample, so the loop cannot
grow with the library.

Warn above 0.01, alert above 0.05, carried in the three places `qcDefaults`
pins together. Operator-set, not inherited: the field publishes 0.50/1.0 for a
sample's AGGREGATE undeclared share and that line does not transfer to one
sequence, since an aggregate reaches 0.50 while no single sequence comes near
it. Admitting a per-barcode line needs an atom on 315, and so does saying what
became of the aggregate one.

The error comparison moved from `alerting-at` to `at-most`. It compared for
equality, which fired only at exactly the error threshold and let every larger
share read warn.

`readShare` keeps its column, relabelled "Sample Undeclared (%)", and carries no
status. Column order is unchanged: the order priorities already produced it.
Every description in the table was rewritten to one instruction per sentence,
active voice, and short sentences.
The undeclared-barcode status was evaluated by calling the scalar `status_for`
once per row over a materialised column. That was justified on the row cap, and
the justification was wrong: `keep` is `int | None` and None keeps every row, so
the loop is a loop over every distinct pre-refine sequence -- 10.2M per sample
and 240.7M over a run, by the figures recorded above the cap. Materialising that
column also undoes the memory work this stage carries.

`status_expr` is the same rule over a column. It reads the SAME `lines` dict and
the SAME `_COMPARISON` table the scalar reads, and `_breaches_expr` mirrors
`_breaches` branch for branch, so only the evaluator differs.

Two evaluators can drift, which is what the loop was avoiding, so
`test_status_expr_agrees_with_status_for` runs them against one another over
every registered measurement: each line's own thresholds, 1e-9 and 0.01 either
side of each, and null, NaN and both infinities. Four mutations were checked
against it and each one fails there: at-most losing its strictness,
alerting-at inverted, warn tested before alert, and the not-a-number guard
removed.
Undeclared-barcode lines judge one sequence's own share, warning above
0.01 and alerting above 0.05. Several baseline tags combine by the
highest count rather than stopping the run. Quality-line tooltips no
longer claim no test asserts the default. Per-sample QC headers for
reads parsed, counts removed, unique counts and sticky cells describe
the one number each column holds. Baseline reading, cells called bound,
seen-in and why-unsettled name the population or values they carry.
Semicolons and em-dashes removed from every tooltip.
… can do

The per-barcode grouping option reads "One identity per barcode". The
combine-column alert no longer tells the reader to pick or clear a
column the form does not offer. Uploading the panel file again is the
one action that clears it, so the alert says that.
- Expose Expected binder fraction (ω), default 10% — the fit's starting split, settable per experiment
- Expose Bound probability, default/floor 0.9
- Adopt the paper's initialisation (split at the ω quantile, was the median)
- Bound-count marker per (sample, tag) on the distribution plots, plus boundAtCount in the fit output
- Sample selector on Fitted background; barcodes in declared panel order via new tagOrder
- New export antigenCellTagCounts [sampleId][cellId][tagId] → pre-floor UMI count, partitioned by sample

Plots
- Equal-width log1p bins (0.2) replacing integer geometric bins; drops the per-bar density division
- Histograms include the zeros the fit was taken over
- Bins computed inside the fit — 2.07M numbers held → 7.8k

Fixes
- Panel column hidden (axis + label) — it was a constant hash
- bg 0 displayed instead of 0.000488 (formatter dropped its own significant digits)
- Run quality page crashed on load (temporal dead zone)
- Absent vs null boundAtCount reported as the same finding
- Resize debounce; JSON indent dropped (~halves payload)

Removals
- count_bin_edges (no caller; reasoning folded into log1p_bin_edges) and the density prop
…in-edges

Milab 6496 integer count bin edges
Comment on lines +304 to +310
if args.bound_probability < DISTRIBUTION_BOUND_PROBABILITY:
raise SystemExit(
f"--bound-probability must be at least {DISTRIBUTION_BOUND_PROBABILITY}. Below that a cell "
f"holding none of a tag could be called bound. Most cells hold none of most tags, and the run "
f"counts them by arithmetic rather than checking each one, so those cells would be counted "
f"not-bound in one place and called bound in another. Got {args.bound_probability}."
)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Invalid bound probabilities pass

When a direct invocation supplies NaN, infinity, or a value above 1, this lower-bound-only validation accepts it. read_states then classifies every otherwise reliable cell as not bound, while NaN makes bound_at_count report the lowest count as the threshold, causing the verdicts and fitted-background marker to disagree silently.

Suggested change
if args.bound_probability < DISTRIBUTION_BOUND_PROBABILITY:
raise SystemExit(
f"--bound-probability must be at least {DISTRIBUTION_BOUND_PROBABILITY}. Below that a cell "
f"holding none of a tag could be called bound. Most cells hold none of most tags, and the run "
f"counts them by arithmetic rather than checking each one, so those cells would be counted "
f"not-bound in one place and called bound in another. Got {args.bound_probability}."
)
if not DISTRIBUTION_BOUND_PROBABILITY <= args.bound_probability <= 1.0:
raise SystemExit(
f"--bound-probability must be at least {DISTRIBUTION_BOUND_PROBABILITY} and at most 1. "
f"Below that a cell holding none of a tag could be called bound. Most cells hold none of most tags, "
f"and the run counts them by arithmetic rather than checking each one, so those cells would be counted "
f"not-bound in one place and called bound in another. Got {args.bound_probability}."
)

Knowledge Base Used:

Prompt To Fix With AI
This is a comment left during a code review.
Path: software/per-cell-metrics/src/emit_verdicts.py
Line: 304-310

Comment:
**Invalid bound probabilities pass**

When a direct invocation supplies `NaN`, infinity, or a value above 1, this lower-bound-only validation accepts it. `read_states` then classifies every otherwise reliable cell as not bound, while `NaN` makes `bound_at_count` report the lowest count as the threshold, causing the verdicts and fitted-background marker to disagree silently.

```suggestion
    if not DISTRIBUTION_BOUND_PROBABILITY <= args.bound_probability <= 1.0:
        raise SystemExit(
            f"--bound-probability must be at least {DISTRIBUTION_BOUND_PROBABILITY} and at most 1. "
            f"Below that a cell holding none of a tag could be called bound. Most cells hold none of most tags, "
            f"and the run counts them by arithmetic rather than checking each one, so those cells would be counted "
            f"not-bound in one place and called bound in another. Got {args.bound_probability}."
        )
```

**Knowledge Base Used:**
- [Integration domain model](https://app.greptile.com/milaboratories/-/custom-context/knowledge-base/platforma-open/feature-integration/-/docs/integration-domain-model.md)
- [Verdict analysis workflow stages](https://app.greptile.com/milaboratories/-/custom-context/knowledge-base/platforma-open/feature-integration/-/docs/workflow-verdict-analysis.md)

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Fix in Claude Code

@julenmendieta
julenmendieta added this pull request to the merge queue Sep 3, 2026
Merged via the queue into main with commit 5d73918 Sep 3, 2026
14 checks passed
@julenmendieta
julenmendieta deleted the julenmendieta/MILAB-6793_lastTweaks branch September 3, 2026 09:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants