Skip to content

Add emissions configuration schema (inventories, species maps, sources) to mechanism v1#281

Open
smeechncar wants to merge 5 commits into
mainfrom
feature/emissions-v1
Open

Add emissions configuration schema (inventories, species maps, sources) to mechanism v1#281
smeechncar wants to merge 5 commits into
mainfrom
feature/emissions-v1

Conversation

@smeechncar

@smeechncar smeechncar commented Jun 25, 2026

Copy link
Copy Markdown

Summary

Closes #272.

Adds a new emissions::v1 namespace to MechanismConfiguration that
parses YAML-based emissions configs for consumption by MUSICA/MIEM.

What's new

  • include/mechanism_configuration/emissions/v1/types.hpp — public types
    (EmissionsConfig, SourceDescriptor, Inventory, SpeciesMap, etc.)
  • include/mechanism_configuration/emissions/v1/parse.hpp — public API:
    ParseEmissions(path) and ParseEmissions(string)
  • src/emissions/v1/parser.cpp — full implementation (yaml-cpp private,
    not exposed in headers); supports ${VAR} / ${VAR:-default} env var
    expansion in data root
  • 12 new ErrorCode values in errors.hpp / errors.cpp
  • 14 unit tests covering valid configs, all error paths, and optional-section
    handling

Design notes

Follows the existing v1 chemistry parser pattern: std::expected<T, Errors>
return type, ErrorCode enum errors, yaml-cpp kept private to the library.
Three intentional differences from the MICM parser shape are noted for
reviewers:

  1. No Parse(YAML::Node) overload — emissions configs are always file/string
    entry points
  2. No separate CheckSchema()/Build() split — validation and construction
    are a single pass
  3. Types live in emissions::v1::types (deeper namespace) to avoid collision
    with mechanism_configuration::types

Aside for MC maintainers

While reviewing this I noticed that yaml-cpp headers currently leak into
MC's public headers in a few places (direct YAML::Node parameters on
public APIs). This isn't blocking for MIEM (MIEM only links against MUSICA,
not MC directly), but it could cause issues for downstream consumers. Flagging
here in case the maintainers want to address it in a follow-up.

Test plan

  • All 14 new emissions tests pass
  • All 19 existing chemistry tests still pass (no regressions)
  • Build tested on Derecho with -DMECH_CONFIG_USE_FMT=ON (Intel LLVM,
    no <format> header)

Adds a new emissions configuration schema under
mechanism_configuration::emissions::v1, parallel to the existing
chemistry schema. This is the MC side of the MIEM config pipeline
described in NCAR/miem#272.

New public API:
  ParseEmissions(path) / ParseEmissions(string)
  → std::expected<EmissionsConfig, Errors>

EmissionsConfig holds inventories, species maps, regridding settings,
and a list of SourceDescriptors. Musica's translator will resolve these
into miem::Source structs and drive miem::EmissionsBuilder.

Load-time validation covers all v1 invariants: kind/version dispatch,
cross-reference checks (inventory and species map names), duplicate
source names, duplicate (category, hierarchy) pairs, scaling factor
sums ≤ 1.0, and hard rejections for mode: online, vertical injection:
plume, and regridding type: scrip (none supported in v1).

Note for MC maintainers: yaml-cpp is currently linked PUBLIC in
src/CMakeLists.txt, which causes it to leak into downstream consumers.
Demoting it to PRIVATE (and moving the three public headers that include
yaml-cpp into src/internal/) would be a clean follow-up.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@smeechncar smeechncar requested a review from K20shores June 25, 2026 17:35
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 31.57%. Comparing base (04d2f08) to head (7dd4271).

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #281   +/-   ##
=======================================
  Coverage   31.57%   31.57%           
=======================================
  Files           4        4           
  Lines          19       19           
=======================================
  Hits            6        6           
  Misses         13       13           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@smeechncar

Copy link
Copy Markdown
Author

Next steps (for context)...

This PR is the first of three pieces. Once merged, the plan is:

  1. MUSICA — add a translator (musica::emissions::Translate) that calls ParseEmissions, walks the resulting EmissionsConfig, and constructs MIEM's EmissionsBuilder / Source objects. MUSICA is the only layer that sees both MC and MIEM types.
  2. MIEM integration test — an end-to-end test using a real CAMS-GLOB NetCDF file to verify the full config → translate → runtime pipeline.

Flagging so reviewers know the public types in emissions::v1::types and the two ParseEmissions overloads are the stable interface MUSICA will depend on.

@K20shores K20shores requested a review from boulderdaze June 25, 2026 20:53

@K20shores K20shores left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We want the description of emissions to be agnostic of the miem tool. I think it would be better to include a new section in the mechanism object called Emissions and put all created structs into that.

This would mean our v1.x format would have a new keyword, emissions, which essentially takes lines 5-end of examples/emissions/v1/emissions.yaml and indents it under the emissions key.

I guess I'll think about this a bit longer before adding more specific comments.

@boulderdaze boulderdaze left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would you mind updating the PR title? We include PR titles in the release notes, so we prefer them to be more descriptive and easier to read

Comment thread include/mechanism_configuration/emissions/v1/parse.hpp Outdated
@smeechncar smeechncar changed the title feat: add emissions::v1 config parser for MIEM integration Add emissions configuration schema (inventories, species maps, sources) to mechanism v1 Jun 26, 2026
Aligns with the library convention where all public headers live directly
under include/mechanism_configuration/ without a version subdirectory.
The v1 version is still expressed in the namespace and source layout.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Comment thread examples/emissions/v1/emissions.yaml Outdated

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, we need to rethink this design. We want emissions to be part of the mechanism configuration, not a separate parser. TO do that, emissions, should be part of the mechanism, not its own file. We also need to support both yaml/json so sheel expansion needs to be dropped.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK that was a bit of a rewiring but I think this latest commit will address this. We restructured to fold emissions into the mechanism: types moved into mechanism_configuration::types, EmissionsConfig added as an optional field on Mechanism, and emissions is now parsed as a section of the main Parse() call rather than a standalone parser. Dropped data_root and shell expansion for JSON parity. The separate emissions::v1 namespace and ParseEmissions() API are gone.

#include <filesystem>
#include <string>

namespace mechanism_configuration::emissions::v1

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we do not want a separate namespacde for emissions

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

emissions::v1 namespace is gone. Types are now in mechanism_configuration::types, EmissionsConfig is an optional field on Mechanism, and emissions is parsed as a section of the main Parse() call. Dropped data_root and shell expansion for JSON parity. The standalone ParseEmissions() API and separate example file are removed.

smeechncar and others added 3 commits June 26, 2026 11:21
…er, or file

Addresses K20shores' review of #281:
- Emissions types moved into mechanism_configuration::types (types.hpp)
- EmissionsConfig added as std::optional<types::EmissionsConfig> on Mechanism
- emissions: key parsed as an optional section by the main v1 Parse() call
- Standalone ParseEmissions() API and emissions::v1 namespace removed
- data_root and shell expansion dropped for JSON parity
- Examples updated: emissions.yaml gone; full_configuration.yaml gains an
  emissions: section
- Tests routed through Parse() instead of the old standalone parser

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The public Parse() only accepted a filesystem::path, so passing a
std::string content literal silently converted to a path and failed
with FileNotFound. Dispatches to v1::Parser for version 1 documents.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ser for string tests

ResolveFileConfig only forwarded species/phases/reactions into the
combined node; the emissions: map was silently dropped so file-based
parses never saw it. Added explicit copy of the emissions key alongside
the existing version/name handling.

Reverted the public Parse(std::string) overload added in the previous
commit — it broke existing callers that passed std::string file paths
(they started routing to the content parser instead of the path
parser). String-based emission tests now use v1::Parser{} directly,
consistent with how the existing integration tests work.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add MIEM support (v1.2)

4 participants