Skip to content

BUG: Locale-independent float parsing and printing in NRRD/DICOM IO - #6695

Open
hjmjohnson wants to merge 2 commits into
InsightSoftwareConsortium:mainfrom
hjmjohnson:nrrdio-locale-strtod
Open

BUG: Locale-independent float parsing and printing in NRRD/DICOM IO#6695
hjmjohnson wants to merge 2 commits into
InsightSoftwareConsortium:mainfrom
hjmjohnson:nrrdio-locale-strtod

Conversation

@hjmjohnson

@hjmjohnson hjmjohnson commented Jul 24, 2026

Copy link
Copy Markdown
Member

Starts from the red NumericLocale.WorksWithDifferentInitialLocale nightly on open.cdash.org (Mac13.x-AppleClang-dbg-Universal). NRRD and DICOM float I/O now parses and prints against a cached "C" locale; the beta-only NumericLocale is removed.

Motivating dashboard failure

This work began from a persistently red nightly on the Insight dashboard at open.cdash.org, not from a code review:

Field Value
Site RogueResearch24
Build Mac13.x-AppleClang-dbg-Universal
Test NumericLocale.WorksWithDifferentInitialLocale

The test asserted that the NumericLocale RAII guard makes numeric parsing locale-independent. It failed only on macOS 13, because that libc's non-_l strtod ignores the thread-local locale that uselocale() installs — so the guard could not honor its contract there, and the test was asserting something the mechanism could not deliver on that platform. Investigating why the guard did not work exposed the larger problem: NrrdIO's read and write paths were locale-dependent underneath it, and on Linux the guard had been masking that from CI. Hence the fix replaces the mechanism rather than adjusting the test.

(Build identifiers are recorded rather than linked: CDash build URLs expire, so a buildId link would rot while this PR's history does not.)

Root cause

Under a decimal-comma LC_NUMERIC (e.g. de_DE.UTF-8), "0.878906" parses as 0 and prints as "0,878906":

  • NRRD read: all float parsing funnels through airSingleSscanf()sscanf() in vendored NrrdIO (spacings, directions, origins, ASCII pixels).
  • NRRD write: all float formatting funnels through airSinglePrintf()printf-family — the writer emitted comma decimals, persisting corrupt files.
  • DICOM DS / every itk::StringTo* IO call site: std::stod/std::stof follow the ambient locale.
  • The previous defense, a NumericLocale RAII guard switching the thread-local locale via uselocale(), is silently ignored by the non-_l strtod/printf on macOS 13-era libc — and on Linux it masked the NrrdIO bug from CI.
Fix (2 commits)

Commit 1 — vendored NrrdIO reads and writes locale-independently.
airSingleSscanf() parses via strtod_l; airSinglePrintf() prints via fprintf_l/snprintf_l (BSD/macOS), _fprintf_l (Windows), or a uselocale() swap (glibc/musl, which lack printf_l but whose printf honors the thread-local locale). The internal round-trip precision probe uses that same "C" locale on every platform arm, so the format it selects never depends on the ambient locale. One cached "C" locale is shared via privateAir.h (air__CLocale()).

A parse fails rather than yielding a wrong value on overflow — ERANGE with ±HUGE_VAL, or a double-range value past FLT_MAX for a "%f" target, which would otherwise narrow silently to infinity — and when the "C" locale is unobtainable. ERANGE underflow is a valid subnormal or zero and is accepted.

On Windows the string path goes through helpers that restore the C99 snprintf contract (always terminate, return the would-be length via _vscprintf_l), which _snprintf_l alone does not honor on truncation; the rest of nrrdSprint[] relies on that contract.

The now-redundant RAII guards in itk::NrrdImageIO are removed.

Commit 2 — canonical helpers fixed, NumericLocale removed.
itk::StringToDouble/StringToFloat (itkStringConvert.cxx, the #6089 itk::ExceptionObject-wrapping helpers) parse via strtod_l/strtof_l, so every existing IO call site is fixed at once. A platform providing neither newlocale nor _configthreadlocale is a #error, so there is no silent fall-back to an ambient-locale parse. DCMTKTransformIO uses itk::StringToDouble for DICOM DS matrix entries. NumericLocale is deleted outright — it appeared only in the v6.0b02 beta, so no deprecation shim is required. The newlocale/_configthreadlocale try_compile checks remain, now serving itkStringConvert.

Upstream teem

The identical NrrdIO change (parse + print + shared locale + native test src/air/test/locale.c) is on the teem branch locale-independent-numeric-parse (SourceForge ssh://git.code.sf.net/p/teem/teem.git, commit 3cbb824ed). This PR carries it in the vendored copy so the fix ships now; when teem merges, UpdateFromUpstream.sh re-vendors cleanly. Teem builds clean and its locale test passes under 4 decimal-comma locales, covering parse, print round trips, and the FLT_MAX overflow case.

Tests

A shared itk::test helper (Modules/Core/TestKernel/include/itkTestNumericLocale.h) carries the candidate locale names in POSIX and Windows spelling plus the "is a comma locale guaranteed here?" policy, so both tests agree on it.

  • itkStringConvertGTest: locale-independence under the comma locales, with the overflow/underflow contract re-asserted inside the comma-locale loop, plus a "0,878906"0.0 check that positively confirms the "C" locale is in effect.
  • itkNrrdLocaleTest: reads verified under each available comma locale, plus a write-under-comma-locale round trip.
  • Finding no comma locale is a hard failure on macOS and Windows, where the OS ships them unconditionally, and a loud warning on Linux, where a lean image legitimately has none until locale-gen runs.
  • NumericLocale tests removed with the class.

Verified locally on macOS (arm64): full ITK build 3730 targets / 0 errors; ITKCommon + ITKIONRRD 166/166; ITKIODCMTK + IOTransformDCMTK 50/50; teem build + locale test green. Windows print paths rely on CI.

@github-actions github-actions Bot added type:Bug Inconsistencies or issues which will cause an incorrect result under some or all circumstances type:Testing Ensure that the purpose of a class is met/the results on a wide set of test cases are correct area:Core Issues affecting the Core module area:ThirdParty Issues affecting the ThirdParty module labels Jul 24, 2026
@hjmjohnson
hjmjohnson force-pushed the nrrdio-locale-strtod branch from 7ffe114 to 2aa5799 Compare July 24, 2026 00:43
@github-actions github-actions Bot added the area:IO Issues affecting the IO module label Jul 24, 2026
@hjmjohnson hjmjohnson changed the title BUG: Parse NRRD floats locale-independently (strtod_l) + guard NumericLocale test BUG: Parse NRRD/DICOM floats locale-independently (strtod_l) Jul 24, 2026
@hjmjohnson
hjmjohnson marked this pull request as ready for review July 24, 2026 00:52
@greptile-apps

This comment was marked as resolved.

Comment thread Modules/Core/Common/src/itkNumericLocale.cxx Outdated
Comment thread Modules/ThirdParty/NrrdIO/src/NrrdIO/parseAir.c Outdated
@hjmjohnson
hjmjohnson marked this pull request as draft July 24, 2026 15:54
@hjmjohnson hjmjohnson changed the title BUG: Parse NRRD/DICOM floats locale-independently (strtod_l) BUG: Locale-independent float parsing and printing in NRRD/DICOM IO Jul 24, 2026
@hjmjohnson
hjmjohnson force-pushed the nrrdio-locale-strtod branch from 2aa5799 to 835c21a Compare July 24, 2026 16:11
@github-actions github-actions Bot added the type:Infrastructure Infrastructure/ecosystem related changes, such as CMake or buildbots label Jul 24, 2026
@hjmjohnson

Copy link
Copy Markdown
Member Author

Force-push 2aa5799835c21a is a redesign, not an increment: it combines a rebase onto current main with a restructure into 2 fresh commits, so the per-push compare is not useful — review the full diff instead. Summary of the redesign is in the updated PR body; it addresses all three greptile P1s (dead __GLIBC__ guard, undeclared pthread, silent conversion errors) and additionally makes the NRRD write path locale-independent.

@hjmjohnson
hjmjohnson marked this pull request as ready for review July 24, 2026 17:01
Comment thread Modules/Core/Common/src/itkStringConvert.cxx Outdated
@hjmjohnson
hjmjohnson marked this pull request as draft July 24, 2026 18:50
NRRD float I/O honored the ambient LC_NUMERIC: under a decimal-comma
locale "0.878906" parsed as 0 and the writer emitted "0,878906",
corrupting spacings, directions, origins, and ASCII pixel values.

Parse, print, and the precision probe share one cached "C" locale
(privateAir.h): strtod_l() and the printf_l() family, or a uselocale()
swap on glibc/musl, which lack printf_l.

A parse fails on overflow (ERANGE with +/-HUGE_VAL, or a value past
FLT_MAX for a "%f" target) or if the "C" locale cannot be created;
ERANGE underflow is a valid subnormal or zero. Mirrors the teem
branch locale-independent-numeric-parse.
itk::StringToDouble and itk::StringToFloat followed the ambient
LC_NUMERIC, so every migrated IO call site parsed "0.878906" as 0
under a decimal-comma locale.

They parse with strtod_l/strtof_l against a cached "C" locale, keeping
the itk::ExceptionObject contract; only overflow throws, as ERANGE
underflow is a valid subnormal or zero. A platform lacking newlocale
and _configthreadlocale fails to compile.

NumericLocale is removed: uselocale() cannot bind the non-_l
std::strtod on macOS 13 libc, and it shipped only in the v6.0b02
beta, so no deprecation shim is required.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:Core Issues affecting the Core module area:IO Issues affecting the IO module area:ThirdParty Issues affecting the ThirdParty module type:Bug Inconsistencies or issues which will cause an incorrect result under some or all circumstances type:Infrastructure Infrastructure/ecosystem related changes, such as CMake or buildbots type:Testing Ensure that the purpose of a class is met/the results on a wide set of test cases are correct

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant