Tracking: PRs for #31, #95, #172, #218, #223, #270, #279, #280

[BenjaminDEMAILLE]

Tracking the batch of PRs opened for the six open issues that had no PR or assignee, plus two bugs found along the way (#279, #280).

PRs

Issue	PR	Status	Closes issue?
#31 NH cap vs STAR	#271	Ready for review. Stacked on #254 (base fix/pair-subset-loci), adds a same chromosome/strand guard	Via #254
#218 bz2/zstd/xz input	#272	Ready for review	Yes
#95 paraseq input	#273	Ready for review	Yes
#172 solo CellRanger flags	#274	Ready for review	Partially (Refs)
#270 hdf5-pure .h5 output	#275	Proposal, design open	No (Refs)
#223 thread scaling	#276	Ready for review (parallel BGZF compression)	No (Refs)
#223 thread scaling	#278	Ready for review. Stacked on #276: SAM/BAM record encoding on the align workers, gated on core saturation	With #276, yes
#279 SE WithinBAM crash	#282	Ready for review. STAR hard-clip rules for SE supplementary records	Yes
Bulk total RNA (unspliced targets)	#283	Ready for review. Opt-in <gene_id>-I targets in TranscriptomeSAM, sp tag, GeneSplicing table; fixes a TranscriptomeSAM soft-clip crash	n/a (Refs COMBINE-lab/salmon#1229)
#280 per-read 10k reservation	#281	Ready for review. Peak RSS ~8x lower, 6-17% faster	Yes

Checklist

• fix(align): a soft-clipped variant of a pair is not a second locus #254 then fix(align): pair-subset dedup requires same chromosome and strand (stacked on #254) #271 (Multi-mapper NH cap differs from STAR (NH up to 20 vs STAR's 7); possibly missing --outFilterMultimapScoreRange #31)
• feat(io): detect input compression by magic bytes, optional bz2/zstd/xz #272 (Read other compressed input formats (bz2 / zstd / xz): evaluate niffler #218)
• feat(io): optional paraseq FASTQ backend #273 (add praseq as an option for input #95), after feat(io): detect input compression by magic bytes, optional bz2/zstd/xz #272
• fix(solo): port STAR's cbMinP, QSmax and oneExact for 1MM_multi barcode correction #274 (solo: two CellRanger-matching flags are inert, and the count matrix is 8.9% off #172)
• feat(solo): optional CellRanger v3 .h5 count matrices via hdf5-pure #275 (Consider hdf5-pure (pure-Rust HDF5) for optional .h5 output #270)
• perf(io): parallel order-preserving BGZF compression for BAM output #276 (Align pipeline stops scaling past ~4-8 threads: decode and write are each a single thread #223, parallel BGZF compression)
• perf(io): encode SAM/BAM records on the align workers (stacked on #276) #278 (Align pipeline stops scaling past ~4-8 threads: decode and write are each a single thread #223, parallel record encoding), after perf(io): parallel order-preserving BGZF compression for BAM output #276
• fix(chimeric): well-formed single-end WithinBAM records with STAR's hard-clip rules #282 (Single-end --chimOutType WithinBAM fails with "read length-sequence length mismatch" #279)
• perf(io): drop the 10k-record per-read reservation in BufferedSamRecords #281 (BufferedSamRecords::new() reserves 10,000 records per read #280)
• feat(quant): unspliced targets in TranscriptomeSAM, splicing-status tag and GeneSplicing counts for bulk total RNA-seq #283 (bulk total RNA, needs maintainer sign-off on non-STAR flags)

Suggested merge order and known conflicts

1. fix(io): decode multi-member gzip input instead of truncating it #219 / feat(io): optional pure-Rust parallel gzip input decoding (rapidgzip-core) #225 vs feat(io): detect input compression by magic bytes, optional bz2/zstd/xz #272: all three touch the gzip branch of src/io/fastq.rs. feat(io): detect input compression by magic bytes, optional bz2/zstd/xz #272 already uses MultiGzDecoder (keep its version over fix(io): decode multi-member gzip input instead of truncating it #219's; fix(io): decode multi-member gzip input instead of truncating it #219's solo fixes are still needed). feat(io): optional pure-Rust parallel gzip input decoding (rapidgzip-core) #225's rapidgzip call moves into the Gzip arm of src/io/compression.rs. feat(io): detect input compression by magic bytes, optional bz2/zstd/xz #272 and feat(io): optional pure-Rust parallel gzip input decoding (rapidgzip-core) #225 both add a [features] table to Cargo.toml.
2. feat(io): optional paraseq FASTQ backend #273 after feat(io): detect input compression by magic bytes, optional bz2/zstd/xz #272: paraseq should read from feat(io): detect input compression by magic bytes, optional bz2/zstd/xz #272's open_decoded so it inherits format detection and multi-member gzip.
3. fix(solo): port STAR's cbMinP, QSmax and oneExact for 1MM_multi barcode correction #274 vs solo: cbMinP posterior threshold, oneExact guard, adapter-anchored geometry (replaces #150) #165: fix(solo): port STAR's cbMinP, QSmax and oneExact for 1MM_multi barcode correction #274 supersedes the cbMinP / oneExact part of solo: cbMinP posterior threshold, oneExact guard, adapter-anchored geometry (replaces #150) #165. Conflicts in resolve_multi_cb and CHANGELOG.md.
4. feat(solo): optional CellRanger v3 .h5 count matrices via hdf5-pure #275 vs feat: output anndata #237: trivial conflict in src/lib.rs next to write_gene_matrix. Different flag and feature (--soloOutH5 / hdf5-out vs --soloOutputFormat / anndata-out).
5. perf(io): parallel order-preserving BGZF compression for BAM output #276: independent of perf(align): cut the align batch size and stop cloning every FASTQ record #222, perf(pipeline): size alignment batches so the result vector stays off the arena path #261, feat(io): optional pure-Rust parallel gzip input decoding (rapidgzip-core) #225, chore(deps): bump noodles 0.113 -> 0.115, noodles-bgzf 0.49 -> 0.51 #211, chore(deps): bump noodles from 0.113.0 to 0.116.0 #259. perf(io): encode SAM/BAM records on the align workers (stacked on #276) #278 (stacked on perf(io): parallel order-preserving BGZF compression for BAM output #276): simulated 3-way merge of src/lib.rs against perf(align): cut the align batch size and stop cloning every FASTQ record #222 and perf(pipeline): size alignment batches so the result vector stays off the arena path #261 gives 0 conflicts.

Open points

• Read other compressed input formats (bz2 / zstd / xz): evaluate niffler #218 / feat(io): detect input compression by magic bytes, optional bz2/zstd/xz #272: behaviour change needs maintainer sign-off. Input compression is detected from content, so a plain *.gz file is read as plain text. The STAR citation in DIVERGENCE.md §4.3 needs a check.
• solo: two CellRanger-matching flags are inert, and the count matrix is 8.9% off #172 / fix(solo): port STAR's cbMinP, QSmax and oneExact for 1MM_multi barcode correction #274: the 8.9% matrix gap has not been re-measured (needs STAR and the 20k-read fixture). Known remaining gaps: CellReads.stats counts reads later dropped by oneExact; Transcript3p records are not filtered by oneExact.
• Consider hdf5-pure (pure-Rust HDF5) for optional .h5 output #270 / feat(solo): optional CellRanger v3 .h5 count matrices via hdf5-pure #275: is there a consumer for .h5 (scanpy, Seurat and CellBender read the MTX directory)? Should it become a --soloOutputFormat value once feat: output anndata #237 lands?
• Multi-mapper NH cap differs from STAR (NH up to 20 vs STAR's 7); possibly missing --outFilterMultimapScoreRange #31 / fix(align): pair-subset dedup requires same chromosome and strand (stacked on #254) #271: the strand guard was not re-benchmarked on the Multi-mapper NH cap differs from STAR (NH up to 20 vs STAR's 7); possibly missing --outFilterMultimapScoreRange #31 fixture.
• Align pipeline stops scaling past ~4-8 threads: decode and write are each a single thread #223 / perf(io): encode SAM/BAM records on the align workers (stacked on #276) #278: pre-encoding is gated (on only when runThreadN nearly covers all cores) because it is slower at low thread counts. RUSTAR_PRE_ENCODE=0|1 overrides.
• Single-end --chimOutType WithinBAM fails with "read length-sequence length mismatch" #279 / fix(chimeric): well-formed single-end WithinBAM records with STAR's hard-clip rules #282: remaining STAR differences in WithinBAM: normal alignment still written alongside the chimera, several chimeras per read possible, PE records lack mate fields and hard clips, QUAL empty.
• perf(io): drop the 10k-record per-read reservation in BufferedSamRecords #281 vs perf(io): encode SAM/BAM records on the align workers (stacked on #276) #278: both touch BufferedSamRecords (perf(io): encode SAM/BAM records on the align workers (stacked on #276) #278 adds fields), minor conflict.
• add praseq as an option for input #95 / feat(io): optional paraseq FASTQ backend #273: paraseq rejects records whose sequence and quality lengths differ (noodles accepts them). Malformed input only.

🤖 Generated with Claude Code