Julenmendieta/milab 6793 last tweaks - #18
Merged
Merged
Conversation
`count_bin_edges` built its edges with `np.geomspace`, which places them between whole numbers. A UMI count is a whole number, so a bin could fall strictly between two counts and stand empty at every weight a run could produce. `np.geomspace(1, 5155, 25)` puts one at [2.039, 2.911); on a 15-tag run it held nothing on all 15 panels and read as a missing bar. Edges are now whole and strictly increasing, so every bin holds at least one count. The step is `max(previous + 1, geometric)`, which draws one count per bar near a count of 1 and stays geometric above that. The last edge is one past the top count, making every bin half-open rather than closing the last one and giving it a count more than its width. A run now takes at most COUNT_BIN_COUNT bins instead of always that many. Nothing reads the count: `bin_values`, `per_tag_count_bins` and the chart all take it from the edge list.
Whole-number edges stop a bin standing empty, but they do not make bar heights comparable. Bin width in counts rises across the edge set -- one real set spans 1, 1, 2, 3, 4, 6, 9, 14, 21 -- so a bin covering 4 counts stood about four times a neighbour covering 1 at equal density. That step drew a second hump on tags whose counts hold one population, and the grid exists to answer whether two populations separated at all, so a hump the bins invented is the one error this surface cannot carry. `CountHistogram` takes a `density` flag, applied in the `log-bins` branch only, which divides each weight by the whole counts its bin spans. The y axis reads "Cells per count". The two linear callers, the score spread and the reference reading, are untouched. Atom 330 binds the x axis -- "the unit the gate is declared in follows from the axis" -- and only on the plots a scientist declares from. This grid informs nothing settable, so its y axis carries no such constraint. `PlChartHistogram` prints the number it is handed under a fixed `count:` label, so a hovered bar on this grid now reports the density. Each panel's caption carries the cell count instead.
VIEW_TABS is derived from the rung the run reports, and that rung is unreported until the run settles. A strip drawn mid-run therefore offered every plot and then dropped the ones the served rung cannot draw, so a reader could open a tab that stopped existing under them. `isRunning` is the block's own computing signal, the one already driving the block spinner. The open view's body is untouched and keeps drawing its own processing placeholder, so the section still shows progress while the strip is away.
The Status column read `readShare`, the sample's aggregate undeclared share, so it was one word repeated down every row of a sample. A sample carrying one heavy undeclared sequence among many light ones said nothing about which sequence to look at. It now reads `barcodeShare`, the row's own share of its sample's pre-refine reads, evaluated per row through the same scalar `status_for` every other status goes through rather than a polars expression rebuilding the thresholds. The frame is capped at UNDECLARED_BARCODES_KEPT rows per sample, so the loop cannot grow with the library. Warn above 0.01, alert above 0.05, carried in the three places `qcDefaults` pins together. Operator-set, not inherited: the field publishes 0.50/1.0 for a sample's AGGREGATE undeclared share and that line does not transfer to one sequence, since an aggregate reaches 0.50 while no single sequence comes near it. Admitting a per-barcode line needs an atom on 315, and so does saying what became of the aggregate one. The error comparison moved from `alerting-at` to `at-most`. It compared for equality, which fired only at exactly the error threshold and let every larger share read warn. `readShare` keeps its column, relabelled "Sample Undeclared (%)", and carries no status. Column order is unchanged: the order priorities already produced it. Every description in the table was rewritten to one instruction per sentence, active voice, and short sentences.
The undeclared-barcode status was evaluated by calling the scalar `status_for` once per row over a materialised column. That was justified on the row cap, and the justification was wrong: `keep` is `int | None` and None keeps every row, so the loop is a loop over every distinct pre-refine sequence -- 10.2M per sample and 240.7M over a run, by the figures recorded above the cap. Materialising that column also undoes the memory work this stage carries. `status_expr` is the same rule over a column. It reads the SAME `lines` dict and the SAME `_COMPARISON` table the scalar reads, and `_breaches_expr` mirrors `_breaches` branch for branch, so only the evaluator differs. Two evaluators can drift, which is what the loop was avoiding, so `test_status_expr_agrees_with_status_for` runs them against one another over every registered measurement: each line's own thresholds, 1e-9 and 0.01 either side of each, and null, NaN and both infinities. Four mutations were checked against it and each one fails there: at-most losing its strictness, alerting-at inverted, warn tested before alert, and the not-a-number guard removed.
Undeclared-barcode lines judge one sequence's own share, warning above 0.01 and alerting above 0.05. Several baseline tags combine by the highest count rather than stopping the run. Quality-line tooltips no longer claim no test asserts the default. Per-sample QC headers for reads parsed, counts removed, unique counts and sticky cells describe the one number each column holds. Baseline reading, cells called bound, seen-in and why-unsettled name the population or values they carry. Semicolons and em-dashes removed from every tooltip.
… can do The per-barcode grouping option reads "One identity per barcode". The combine-column alert no longer tells the reader to pick or clear a column the form does not offer. Uploading the panel file again is the one action that clears it, so the alert says that.
- Expose Expected binder fraction (ω), default 10% — the fit's starting split, settable per experiment - Expose Bound probability, default/floor 0.9 - Adopt the paper's initialisation (split at the ω quantile, was the median) - Bound-count marker per (sample, tag) on the distribution plots, plus boundAtCount in the fit output - Sample selector on Fitted background; barcodes in declared panel order via new tagOrder - New export antigenCellTagCounts [sampleId][cellId][tagId] → pre-floor UMI count, partitioned by sample Plots - Equal-width log1p bins (0.2) replacing integer geometric bins; drops the per-bar density division - Histograms include the zeros the fit was taken over - Bins computed inside the fit — 2.07M numbers held → 7.8k Fixes - Panel column hidden (axis + label) — it was a constant hash - bg 0 displayed instead of 0.000488 (formatter dropped its own significant digits) - Run quality page crashed on load (temporal dead zone) - Absent vs null boundAtCount reported as the same finding - Resize debounce; JSON indent dropped (~halves payload) Removals - count_bin_edges (no caller; reasoning folded into log1p_bin_edges) and the density prop
…teger-count-bin-edges
…in-edges Milab 6496 integer count bin edges
Comment on lines
+304
to
+310
| if args.bound_probability < DISTRIBUTION_BOUND_PROBABILITY: | ||
| raise SystemExit( | ||
| f"--bound-probability must be at least {DISTRIBUTION_BOUND_PROBABILITY}. Below that a cell " | ||
| f"holding none of a tag could be called bound. Most cells hold none of most tags, and the run " | ||
| f"counts them by arithmetic rather than checking each one, so those cells would be counted " | ||
| f"not-bound in one place and called bound in another. Got {args.bound_probability}." | ||
| ) |
There was a problem hiding this comment.
Invalid bound probabilities pass
When a direct invocation supplies NaN, infinity, or a value above 1, this lower-bound-only validation accepts it. read_states then classifies every otherwise reliable cell as not bound, while NaN makes bound_at_count report the lowest count as the threshold, causing the verdicts and fitted-background marker to disagree silently.
Suggested change
| if args.bound_probability < DISTRIBUTION_BOUND_PROBABILITY: | |
| raise SystemExit( | |
| f"--bound-probability must be at least {DISTRIBUTION_BOUND_PROBABILITY}. Below that a cell " | |
| f"holding none of a tag could be called bound. Most cells hold none of most tags, and the run " | |
| f"counts them by arithmetic rather than checking each one, so those cells would be counted " | |
| f"not-bound in one place and called bound in another. Got {args.bound_probability}." | |
| ) | |
| if not DISTRIBUTION_BOUND_PROBABILITY <= args.bound_probability <= 1.0: | |
| raise SystemExit( | |
| f"--bound-probability must be at least {DISTRIBUTION_BOUND_PROBABILITY} and at most 1. " | |
| f"Below that a cell holding none of a tag could be called bound. Most cells hold none of most tags, " | |
| f"and the run counts them by arithmetic rather than checking each one, so those cells would be counted " | |
| f"not-bound in one place and called bound in another. Got {args.bound_probability}." | |
| ) |
Knowledge Base Used:
Prompt To Fix With AI
This is a comment left during a code review.
Path: software/per-cell-metrics/src/emit_verdicts.py
Line: 304-310
Comment:
**Invalid bound probabilities pass**
When a direct invocation supplies `NaN`, infinity, or a value above 1, this lower-bound-only validation accepts it. `read_states` then classifies every otherwise reliable cell as not bound, while `NaN` makes `bound_at_count` report the lowest count as the threshold, causing the verdicts and fitted-background marker to disagree silently.
```suggestion
if not DISTRIBUTION_BOUND_PROBABILITY <= args.bound_probability <= 1.0:
raise SystemExit(
f"--bound-probability must be at least {DISTRIBUTION_BOUND_PROBABILITY} and at most 1. "
f"Below that a cell holding none of a tag could be called bound. Most cells hold none of most tags, "
f"and the run counts them by arithmetic rather than checking each one, so those cells would be counted "
f"not-bound in one place and called bound in another. Got {args.bound_probability}."
)
```
**Knowledge Base Used:**
- [Integration domain model](https://app.greptile.com/milaboratories/-/custom-context/knowledge-base/platforma-open/feature-integration/-/docs/integration-domain-model.md)
- [Verdict analysis workflow stages](https://app.greptile.com/milaboratories/-/custom-context/knowledge-base/platforma-open/feature-integration/-/docs/workflow-verdict-analysis.md)
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Greptile Summary
This PR adjusts fitted-distribution verdict calling, per-barcode QC status, histogram output, workflow exports, and their UI controls. Important touched terms:
Confidence Score: 4/5
The PR should not merge until the Python execution boundary rejects invalid bound-probability values that otherwise produce silently incorrect verdicts.
The newly configurable cutoff can bypass the Python validation and cause all cells to be classified as not bound while the accompanying fitted-background threshold reports contradictory information.
Files Needing Attention: software/per-cell-metrics/src/emit_verdicts.py, software/per-cell-metrics/src/verdict.py, software/per-cell-metrics/src/tag_distribution.py
Important Files Changed
Flowchart
%%{init: {'theme': 'neutral'}}%% flowchart LR UI[Verdict settings UI] --> Model[Validated BlockArgs] Model --> Args[Workflow CLI arguments] Args --> Fit[Per-sample tag distribution fit] Fit --> Calls[Cell binding calls] Fit --> Bins[Fitted-background bins and thresholds] Calls --> Verdicts[Clonotype verdict outputs] Bins --> QC[Run-quality UI] Raw[Pre-floor cell/tag counts] --> Export[Downstream composition export]Prompt To Fix All With AI
Reviews (1): Last reviewed commit: "Changeset" | Re-trigger Greptile
Context used (5)