Skip to content

Handle scenario JSON with no point inputs and missing node/baseline data in IWFM output generation - #16

Merged
Gorfaal merged 1 commit into
developfrom
feature/iwfm-missing-point-input-guards
Sep 21, 2026
Merged

Gorfaal merged 1 commit into
developfrom
feature/iwfm-missing-point-input-guards

Conversation

@Gorfaal

@Gorfaal Gorfaal commented Sep 21, 2026

Copy link
Copy Markdown
Contributor

Pushed on behalf of @coosbo-get — Colby is currently blocked by an auth loop between his machine and the ESA GitHub org, so he sent the updated file over and I landed it for him. The commit is authored to him.

What this fixes

CreateGroundwaterLevelPoints assumed three things that don't always hold:

  1. every run's userdata.json carries point inputs (PivotedRunWellInputs)
  2. every closest node has an entry in NodeWaterLevelLayer.csv
  3. baseline head data exists for every node and every timestep

Any of those being absent threw out of output generation for the entire run. From the caller's side that surfaces as a SystemError with no output files and no message — which is what we've been seeing on the Qanat side for Merced runs (17997, 17999).

Each case is now skipped and logged instead:

  • no point inputs → skip the block, log, still write the (empty) TimeSeriesData output
  • node with no layer mapping → warn, skip that point
  • node absent from the head-all output for a timestep → skip that timestep
  • missing baseline data on a differential run → warn, leave the baseline fields null

Also hoists the layer lookup out of the timestep loop and drops a dead else branch that reassigned runValue to the identical expression.

Notes for review

  • Not built locally. Restore needs the private esadatatechnology/sitkatech Azure DevOps feed and I get a 401 from here, so this has not been compiled. It parses clean under Roslyn and every symbol it newly uses checks out against the repo (Logging.GetLogger<T>, Run.RunID, UserDataPoint.Name, GetNodeWaterLevelLayerMapping() → Dictionary<int, int>), but CI is the first real type-check. Please don't merge on my say-so that it compiles.
  • The file Colby sent came through reformatted (blank lines doubled, usings re-sorted, BOM dropped). I applied only the substantive changes onto the current file so the diff stays reviewable — it's +33/-3 ignoring whitespace; the rest is the re-indent from the new if wrapper. The result is line-for-line identical to what he sent apart from the next point.
  • Added readonly to the Logger field to match the other nine engines in this folder. No behavior change.

One open question for @coosbo-get

If userdata.json is empty or missing entirely, DeserializeObject returns null, and the new else path still reaches the SaveFile call at the end — writing the literal string null as the TimeSeriesData output rather than the empty payload the log message describes. Valid JSON that merely omits PivotedRunWellInputs is fine; this is only the degenerate case.

A one-liner above the guard would close it:

userDataObject ??= new UserDataJson { UserDataPointInputs = new List<UserDataPoint>() };

I left it out deliberately since it changes behavior on a path you may have left alone on purpose — you'd know better than me whether the portal side wants an empty object there or no file at all. Happy to add it here or leave it for a follow-up.

🤖 Generated with Claude Code

…ata in IWFM output generation

CreateGroundwaterLevelPoints assumed every run's userdata.json carried point
inputs, that every closest node had a NodeWaterLevelLayer.csv mapping, and that
baseline head data existed for every node and timestep. Any of those being
absent threw and failed output generation for the whole run with no message.

- Skip the point-processing block when there are no point inputs, and log it
- Skip a point whose closest node has no water level layer mapping, with a warning
- Skip timesteps where the node is absent from the head-all output
- Guard the differential baseline lookup, warning instead of throwing
- Hoist the layer lookup out of the timestep loop
- Drop the dead else branch that reassigned runValue to the same expression

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Baseline output is loaded unconditionally, so non-differential runs without baseline data can still fail.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 1 Medium severity

Open (1)
What changed in this PR

Updates IWFM groundwater-level output generation to tolerate missing inputs, node mappings, timestep data, and baseline data without aborting the run.

Changes:

  • Skips processing when point inputs are absent.
  • Skips unmapped nodes and missing timestep data.
  • Logs missing baseline data and preserves null baseline fields.
  • Simplifies layer lookup and removes redundant logic.
File Description
Olsson.GET.Engines/​ModelInputOutputEngines/​IWFMModelInputOutputEngine.cs Adds defensive handling and logging for incomplete IWFM output data.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

});

var parsedHeadAllOutputFile = ParseHeadAllOutputFile(modelFileAccessor);
var baselineHeadAllOutputFile = ParseHeadAllOutputFile(modelFileAccessor, true);

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not addressing in this PR, but the mechanism is correct and worth recording.

GetIWFMHeadAllOutputFile resolves the file with Directory.EnumerateFiles(resultsPath).Single(...), which throws when nothing matches, so a non-differential run with no *HeadAll.Baseline.out in Results/ would throw here before any of the new guards run.

Two qualifications:

  • Pre-existing. The unconditional baseline load is line 77 on develop today; this PR only moved it inside the new if block, which if anything narrows the exposure — a run with no point inputs now skips the baseline parse entirely.
  • Not reachable from the Qanat integration. Qanat hardcodes IsDifferential = true on every run it submits, and GETEngine defaults data.IsDifferential ?? true (Functions.cs:278), so the baseline file is always expected on that path. This is only reachable for a non-differential run submitted from elsewhere.

Leaving it out because it can't be verified from here: this repo has no CI, and restore needs the private esadatatechnology/sitkatech feed, which I can't reach — so a change to a path with no test coverage and no way to exercise it locally isn't one to land on top of a hand-off PR.

The fix, for whoever picks it up:

var baselineHeadAllOutputFile = run.IsDifferential
    ? ParseHeadAllOutputFile(modelFileAccessor, true)
    : new Dictionary<DateTime, Dictionary<int, List<double>>>();

That also avoids parsing a large head-all file on non-differential runs.

@Gorfaal
Gorfaal merged commit f55a08c into develop Sep 21, 2026
1 check passed
@Gorfaal
Gorfaal deleted the feature/iwfm-missing-point-input-guards branch September 21, 2026 18:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants