Skip to content

fix(ENGKNOW-3978): refbases on a contig without sequence returns N - #150

Open
gmagnu wants to merge 3 commits into
mainfrom
ENGKNOW-3978-refbases-missing-contig
Open

gmagnu wants to merge 3 commits into
mainfrom
ENGKNOW-3978-refbases-missing-contig

Conversation

@gmagnu

@gmagnu gmagnu commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

Summary

RefSeqFromChromSeq (behind refbases, refbases_with_build, VARNORM, VERIFYVARIANT) keeps lastKey/lastBuff for the last buffer read. getBases warmed the buffer through getBase and then read lastBuff unconditionally. When the requested contig has no chromSeq file, getBase returns 'N' without loading a buffer, so:

  • if an earlier call loaded another contig, getBases returned bases from that contig's buffer — wrong data, no error;
  • if nothing had been loaded yet, it threw a NullPointerException on lastBuff.

The outcome depended on row order.

Fix

  • Buffer loading moved into one getBuffer (LUFO cache or file via readBuffer). It returns null when the contig has no sequence in the build or the read fails, and only sets lastKey/lastBuff when it returns a buffer, so lastBuff always belongs to lastKey.
  • getBase and getBases both use getBuffer. getBases walks the range buffer by buffer: one load per buffer, and a buffer that can not be loaded reads as 'N' for its whole part.
  • Consequences beyond the bug fix:
    • a failed read (e.g. a network IOException wrapped in a RuntimeException) is attempted and logged once per buffer, not once per position;
    • a missing contig costs one path lookup per buffer, not per position;
    • ranges crossing a buffer boundary no longer drop to a per-position getBase loop.

Behaviour unchanged otherwise: positions past the chromosome end and below 1 read as 'N', a direct IOException still raises GorResourceException.

Tests

New tests in UTestRefSeqFromChromSeq:

  • testGetRefbasesMissingContigAfterOtherContig — fails on main: expected:<[NNN]> but was:<[CAG]> (bases from chr1).
  • testGetRefbasesMissingContigFirst — fails on main: NullPointerException on lastBuff.
  • testGetRefbasesFailedReadIsTriedOncePerBuffer — mocked RacFile whose read throws; asserts one read attempt per buffer touched. Failed on the previous revision of this PR (TooManyActualInvocations).
  • testGetRefbasesMissingContigAcrossBuffers — missing contig over a range spanning two buffer boundaries reads as all 'N'.

Also ran, all passing: full :model:test (1554 tests), UTestGenomicFunctions (34), UTestVarNormWithBuild (15), UTestVerifyVariant (8).

Merged main after #149 (ENGKNOW-3981); both fixes compose in getBase/getBases.

Jira: ENGKNOW-3978

🤖 Generated with Claude Code

RefSeqFromChromSeq.getBases read lastBuff after warming via getBase, but
getBase never updated lastKey/lastBuff when the contig had no chromSeq
file. Result: bases from the previous contig's buffer, or an NPE when no
buffer had been loaded yet.

Only use lastBuff when lastKey matches the requested buffer, otherwise
fall back to per-base getBase which returns 'N'. getBase now sets
lastKey/lastBuff on a fresh read, removing the double-call warm-up.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@gmagnu
gmagnu marked this pull request as ready for review October 6, 2026 21:36
…-missing-contig

# Conflicts:
#	model/src/test/java/org/gorpipe/model/gor/iterators/UTestRefSeqFromChromSeq.java
@github-actions

github-actions Bot commented Oct 6, 2026 •

Copy link
Copy Markdown

Junit Tests - Summary

4 916 tests  +4   4 745 ✅ +4   19m 36s ⏱️ -47s
  509 suites ±0     171 💤 ±0 
  509 files   ±0       0 ❌ ±0 

Results for commit a016703. ± Comparison against base commit e2d22eb.

♻️ This comment has been updated with latest results.

Move buffer loading into getBuffer, which returns null for a contig
without sequence or a failed read and only sets lastKey/lastBuff on
success. getBases now walks the range buffer by buffer and fills a
buffer that can not be loaded with 'N' in one step.

Before, a failed read (e.g. a wrapped network error) was retried and
logged once per position, and a missing contig rebuilt the file path
per position.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant