Merge stochastic physics (SPPT only) into GSL's develop branch by gsketefian · Pull Request #239 · ufs-community/MPAS-Model

gsketefian · 2026-03-31T18:58:51Z

This PR enables use of the SPPT scheme of stochastic physics into MPAS-Model. This is addressed in Issue #204.

Main changes:

Bring in the stochastic_physics code as a submodule.
Make changes to module files, Makefiles, registry xml files, and fortran files, and GitHub workflow files to allow the SPPT stochastic scheme to be used with MPAS.
Adds the stochastic_physics code as a submodule. There is a companion PR into the authoritative stochastic_physics repo here.

This PR adds a new GitHub Actions CI workflow (run_mpas_stoch.yml) that runs two tests of MPAS-Model on a winter case using GFS ICs/LBCs and with SPPT enabled. The first test is the "convection_permitting" physics suite, and the second is with the "hrrrv5" suite. These two tests use the same grid, static file, ICs, LBCs, etc as the existing CI test(s) for the winter case with GFS ICs/LBCs, etc. These tests require the setup (namelist, streams, etc) files located in the directories

MPAS-Model/testing_and_setup/ufs-community/cases/ufscommunity.convection_permitting.gfs.winter.stoch
MPAS-Model/testing_and_setup/ufs-community/cases/ufscommunity.hrrrv5.gfs.winter.stoch

Mandatory Questions

Does this PR include any additions or changes to external inputs (e.g., microphysics lookup tables, static data for gravity-wave drag -- things like that)?
- No. External data files are not added, but the new CI tests do require the namelist, stream, etc files at MPAS-Model/testing_and_setup/ufs-community/cases/ufscommunity.[convection_permitting|hrrrv5].gfs.winter.stoch, which are under version control.
Does this PR require updating one or more baselines for the CI tests? If so, what?
- No.

Priority Reviewers

Please list the developers/collaborators you'd like to prioritize for review

…he stochastic_physics submodule itself will be added in a separate commit.

…'t updated during latest merge of gsl/develop.

…he gsl/MPAS_stoch_physics in the stochastic_physics repo.

…teger seed workaround.

… smoke) from "output" to "output_smoke" (and file from "history..." to "history_smoke...". This is necessary because the stream name "output" is already taken by the main output stream, and having the same name for two different output streams apparently causes SMIOL I/O errors during the forecast (and incorrect/corrupted history*.nc files).

…) no stale stream_list.atmosphere files exist in the two default_inputs directories (one under core_atmosphere and the other immediately under the MPAS-Model top-level directory), and (2) for the atmosphere core, all files (stream-related as well as namelist) in the top-level directory are backed up before new such files are copied from the default_inputs directory back up a level into the top-level directory. Previously, existing (and thus possibly outdated) stream_list.atmosphere.* files in the top-level directory were not being replaced by newer ones in default_inputs, and that was causing unexpected (and wrong) behavior. Probably a similar fix is needed for the init_atmsophere core and maybe even other ones.

…ll as its output file) so it doesn't conflict with the default output stream.

…tochastic_physics's master branh will gradually happen.

…/MPAS_stoch_physics_try_merge_stoch_master

…tochastic_physics code know that the dycore being used is MPAS.

…ill instead be defined in the Makefile for stochastic_physics only (in a separate commit into the stochastic_physics repo) since it is only needed by the stochastic_physics submodule.

…e that is now added in the Makefile in stochastic_physics).

…h_master

…s the TEMPO tables.

…ysics_try_merge_stoch_master' into gsl/MPAS_stoch_physics_try_merge_stoch_master

…chastic_physics submodule (from mpas_stochastic_physics.F to mpas_stochastic_physics.F90) to reflect change in directory tree.

Change the fetch URL from dustinswales/stochastic_physics to dtcenter/stochastic_physics, which is the authoritative upstream that contains the required commit (063b1897).

guoqing-noaa · 2026-05-27T20:02:01Z

            COMMAND mpas_streams_gen ${CMAKE_CURRENT_BINARY_DIR}/Registry_processed.xml streams.${ARG_CORE} stream_list.${ARG_CORE}. listed
            COMMENT "CORE ${ARG_CORE}: Generate Streams"
-            DEPENDS mpas_streams_gen Registry_processed.xml)
+            DEPENDS mpas_streams_gen ${CMAKE_CURRENT_BINARY_DIR}/Registry_processed.xml)


why we need to add ${CMAKE_CURRENT_BINARY_DIR}?

@gsketefian or @NingWang325, is this one you can answer?

I'm working on remembering the details of why I needed this.

Copilot

Pull request overview

This PR integrates the NOAA-PSL stochastic_physics package (SPPT scheme only) into MPAS-Model as a git submodule, plumbing the perturbation pattern generation and application into the atmosphere core's init/timestep/RK3 paths and the registry. It also adds a dedicated CI workflow that runs convection_permitting and hrrrv5 winter GFS cases with SPPT enabled, plus build-system support (MKL/LAPACK, NetCDF-Fortran) on several HPC modulefiles and CI images.

Changes:

Add stochastic_physics as a submodule with corresponding Make/CMake/Registry wiring and SPPT call sites in atm_core_init, atm_do_timestep, and atm_srk3.
New CI workflow run_mpas_stoch.yml and new test case directories ufscommunity.{convection_permitting,hrrrv5}.gfs.winter.stoch (namelists, streams, stream_lists).
Build environment updates: LAPACK/BLAS/MKL on modulefiles and CI, NetCDF-Fortran setup, MPAS_USE_MPI_F08 always defined, registry-processed-file paths made absolute in CMake.

Reviewed changes

Copilot reviewed 33 out of 33 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
testing_and_setup/ufs-community/cases/ufscommunity.hrrrv5.gfs.winter.stoch/*	New SPPT test case files (namelist, streams, stream lists) for the hrrrv5 suite.
testing_and_setup/ufs-community/cases/ufscommunity.convection_permitting.gfs.winter.stoch/*	New SPPT test case files for the convection_permitting suite.
src/Makefile	Adds `$(LAPACK_LIBS)` to the link line; introduces an `esmf_time_f91` typo in the `-I` path.
src/core_atmosphere/Registry.xml	Adds Gaussian-grid dims (hard-coded 504/248) and new `stoch_pattern_*` variables; includes stochastic_physics registry.
src/core_atmosphere/mpas_atm_core.F	Wires `stochastic_physics_pattern_init` and `_adv` into init and timestep paths.
src/core_atmosphere/Makefile	Builds and links the `stochastic_physics` library; reworks `post_build` to back up existing default_inputs.
src/core_atmosphere/dynamics/mpas_atm_time_integration.F	Calls `stochastic_physics_pattern_apply` for 'phys' and 'prog' tendencies in `atm_srk3`.
src/core_atmosphere/dynamics/Makefile	Adds `-I../stochastic_physics` include path.
src/core_atmosphere/CMakeLists.txt	Defines a `stochastic_physics` CMake target with BLAS/NetCDF deps and links it into core_atmosphere.
modulefiles/mpas/{derecho,gaeac6,hera,orion,ursa}.intel.lua	Loads Intel MKL and exports NetCDF C/Fortran root env vars.
Makefile	Adds `LAPACK_LIBS` to relevant build targets; adds new `intel-mpi-gaeac6` build target.
CMakeLists.txt	Always defines `MPAS_USE_MPI_F08`; adds explicit dependency to serialize init_atmosphere on core_atmosphere builds.
cmake/Functions/MPAS_Functions.cmake	Uses absolute `${CMAKE_CURRENT_BINARY_DIR}` paths for Registry_processed.xml dependencies.
.gitmodules	Adds `src/core_atmosphere/stochastic_physics` submodule pointing at dtcenter fork.
.github/workflows/run_mpas.yml, run_mpas_hrrr.yml, build_mpas.yml, build_mpas_intel.yml	Add LAPACK/BLAS, NetCDF setup, and MKL support; minor commented-out trigger lines.
.github/workflows/run_mpas_stoch.yml	New CI workflow that runs baseline vs feature SPPT-enabled MPAS and compares outputs.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

gsketefian · 2026-06-01T17:33:46Z


 mpas: $(AUTOCLEAN_DEPS) externals frame ops dycore drver
-	$(LINKER) $(LDFLAGS) -o $(EXE_NAME) driver/*.o -L. -ldycore -lops -lframework $(LIBS) -I./external/esmf_time_f90 -L./external/esmf_time_f90 -lesmf_time
+	$(LINKER) $(LDFLAGS) -o $(EXE_NAME) driver/*.o -L. -ldycore -lops -lframework $(LIBS) -I./external/esmf_time_f91 -L./external/esmf_time_f90 -lesmf_time $(LAPACK_LIBS)


I noticed this as well but thought it might be intentional by @NingWang325. Kind of surprising that it works! @NingWang325 can you say if it's intentional or a typo?

joeolson42 · 2026-06-01T11:18:44Z

+                <dim name="lon_for"      definition="504"
+                     description="The number of longitude points for the Gaussian grid"/>
+                <dim name="lat_leg"      definition="248"
+                     description="The number of latitude points for the Gaussian grid"/>


Without full comprehension of what these variables are used for, I think Copilot has some valid points here.

My browser quit on me so I don't know if this will be a duplicate comments or not... Without fully understanding what these variables are for, it seems Copilot has some good points here.

From @NingWang325, these arrays are only used for debugging. These fields aren't being used otherwise. They can be removed.

JeffBeck-NOAA · 2026-06-04T17:50:43Z

+      ! 
+      ! init stochastic pattern generation
+      if (dosppt(domain)) then
+        call stochastic_physics_pattern_init (domain, ierr)
+      endif


From @NingWang325: this code follows procedure from NOAA-PSL/stochastic_physics to ensure initialization is successful. For now, a message is passed to the log saying that stochastic physics could not be initialized.

We need to stop the run in NOAA-PSL/stochastic_physics if initialization of any expect stochastic physics method fails. Check the "ierr" value and issue a stop command.

clark-evans · 2026-05-29T15:52:11Z

@gsketefian @NingWang325 @JeffBeck-NOAA I turned Copilot loose on a review of this PR. It generated five comments, the first few of which (at minimum) look like things that need to be addressed before merge.

joeolson42 · 2026-06-01T11:34:05Z

+&soundings
+    config_sounding_interval = 'none'
+/
+&nam_stochy


just curious, why "nam_stochy"?

@joeolson42 I think this is what the stochastic physics namelist in the FV3 is called, so we just kept it. My guess as to its meaning: "nam" = namelist, "stochy" = "stochastic". Not sure about the "y" in "stochy". I can't find any explanation in the inline comments in the stochastic_physics code. It might have French origins, since I think there were some French developers working on the code at some point. I'm curious too, so I'll ask @pjpegion in the companion PR #97 into the NOAA-PSL/stochastic_physics repo.

Here's the link to my question.

@joeolson42 Sorry, turns out @pjpegion doesn't know the answer regarding the "y" in "stochy".

I’d personally prefer a more straightforward name like stoch_phys or stoch_physics for the namelist, though it isn't a strong preference.

I'd prefer something like "stoch_physics" too, but since it's just a preference, it can't be considered a mandatory change. Maybe let others weigh in.

@joeolson42 I'm still curious about the "y" in "stochy", especially since it's used not just in "nam_stochy" but also all throughout the stochastic_physics code. I had claude look at that code to see if it can come up with an explanation for the "y". It said:

"stochy" reads as an adjectival nickname — like "patchy" or "scratchy" — a natural informal shorthand for "stochastic" that is also more distinctive as an identifier prefix than the bare stem "stoch".

I'm content with that.

… launching of CI tests on push.

dustinswales · 2026-06-01T21:25:48Z

+      ! 
+      ! init stochastic pattern generation
+      if (dosppt(domain)) then
+        call stochastic_physics_pattern_init (domain, ierr)


@gsketefian Should this die and report error if initialization fails?

@dustinswales I would think it should at least write out a warning in the log file(s) if not die altogether, but I think @NingWang325 is best suited to answer this one.

guoqing-noaa · 2026-06-02T00:36:19Z

+if("atmosphere" IN_LIST MPAS_CORES AND "init_atmosphere" IN_LIST MPAS_CORES)
+    add_dependencies(mpas_init_atmosphere core_atmosphere)
+endif()
+


Lines 128-134: Is this a new requirement due to the stochastic physics?
I don't think we need this dependency before.

joeolson42

Conditionally approved, pending any changes needed to address comments.

…multiplication subroutine.

…n building with the stand-alone MPAS-Model, stochastic_physics uses "dgemm" instead of "esmf_dgemm" to perform matrix multiplication.

…mpi-gaeac6" for building the stand-alone MPAS-Model with stochastic_physics on Gaea C6 because it turns out that the existing "ifort_icx" target can be used for this purpose -- as long as a small modification (addition of LAPACK_LIBS) is made to it, which we also do here.

dustinswales · 2026-06-03T16:25:54Z

@gsketefian Can you sync this branch with the noaa/develop branch?

…ch_master

guoqing-noaa · 2026-06-04T20:00:22Z

            COMMAND mpas_streams_gen ${CMAKE_CURRENT_BINARY_DIR}/Registry_processed.xml streams.${ARG_CORE} stream_list.${ARG_CORE}. listed
            COMMENT "CORE ${ARG_CORE}: Generate Streams"
-            DEPENDS mpas_streams_gen Registry_processed.xml)
+            DEPENDS mpas_streams_gen ${CMAKE_CURRENT_BINARY_DIR}/Registry_processed.xml)


Lines 217-234:
Did we get any troubles without these newly added ${CMAKE_CURRENT_BINARY_DIR} etc?

If not, I would suggest removing all the newly added ${CMAKE_CURRENT_BINARY_DIR} etc to keep sync'ed with the upstream NCAR's MPAS-Dev/MPAS-Model as much as possible.

gsketefian and others added 30 commits September 12, 2025 12:36

Add changes to MPAS-Model code necessary to run stochastic physics. T…

6f687ea

…he stochastic_physics submodule itself will be added in a separate commit.

Add stochastic physics as a submodule.

bb262cd

Bug fix: Add missing lines for stochastic physics to Registry.xml.

e5bde4d

Change hash of stochastic_physics.

293e081

Merge branch 'gsl/develop' into gsl/MPAS_stoch_physics

abc034b

Update hashes of GFL and MYNN-EDMF submodules. Not sure why this wasn…

cee787e

…'t updated during latest merge of gsl/develop.

A new version of SPPT merged into the latest gsl/MPAS_stoch_physics.

c93caca

Update hash of stochastic_physics submodule to point to the head of t…

845ff87

…he gsl/MPAS_stoch_physics in the stochastic_physics repo.

Update hash of stochastic_physics to the hash from Ning for 8-byte in…

995a0cf

…teger seed workaround.

Change the name of the output stream in registry.chemistry.xml (as we…

cea828b

…ll as its output file) so it doesn't conflict with the default output stream.

Merge branch 'gsl/develop' into gsl/MPAS_stoch_physics

9d0eb11

Switch hash of stochastic_physics to branch in which the merge with s…

8428af8

…tochastic_physics's master branh will gradually happen.

Merge remote-tracking branch 'origin/gsl/MPAS_stoch_physics' into gsl…

a0700bf

…/MPAS_stoch_physics_try_merge_stoch_master

Fix hash of stochastic_physics.

f8fea43

In the top-level Makefile, add a flag to CPPFLAGS that will let the s…

4060353

…tochastic_physics code know that the dycore being used is MPAS.

Update hash of stochastic_physics.

9bdd0de

Update hash of stochastic_physics.

57f2ccf

Remove definition of macro MPAS_STOCH from main MPAS Makefile. This w…

6f5de0a

…ill instead be defined in the Makefile for stochastic_physics only (in a separate commit into the stochastic_physics repo) since it is only needed by the stochastic_physics submodule.

Update hash of stochastic_physics.

662e9c2

Remove commented-out lines in Makefile that refer to MPAS_STOCH (sinc…

b4c22a1

…e that is now added in the Makefile in stochastic_physics).

Update hash of stochastic_physics.

d2708cf

Update hash of stochastic_physics.

ec08395

Merge branch 'gsl/develop' into gsl/MPAS_stoch_physics_try_merge_stoc…

ae7c0d7

…h_master

Add lines to physics Makefile to also build the executable that build…

a11e46f

…s the TEMPO tables.

Remove commented-out lines in Makefiles.

d5a2107

Clean up comments.

365959c

Get rid of comments in Makefile for brevity.

795dd8c

Update hash of stochastic_physics.

722cac1

gsketefian added 3 commits May 26, 2026 08:52

Update hash of stochastic_physics.

8c5e7e6

Merge remote-tracking branch 'refs/remotes/dtc_fork/gsl/MPAS_stoch_ph…

950c0d1

…ysics_try_merge_stoch_master' into gsl/MPAS_stoch_physics_try_merge_stoch_master

Update name of fortran file name in parent CMakeLists.txt file of sto…

4c45465

…chastic_physics submodule (from mpas_stochastic_physics.F to mpas_stochastic_physics.F90) to reflect change in directory tree.

gsketefian marked this pull request as ready for review May 26, 2026 17:18

Fix stochastic_physics submodule URL to use dtcenter repo.

bd441e1

Change the fetch URL from dustinswales/stochastic_physics to dtcenter/stochastic_physics, which is the authoritative upstream that contains the required commit (063b1897).

clark-evans requested review from AndersJensen-NOAA, XiaSun-Atmos and joeolson42 May 27, 2026 18:12

clark-evans moved this from DRAFT to Needs review in MPAS-A PRs to process May 27, 2026

guoqing-noaa reviewed May 27, 2026

View reviewed changes

clark-evans requested a review from Copilot May 29, 2026 15:45

Copilot started reviewing on behalf of clark-evans May 29, 2026 15:46 View session

Copilot AI reviewed May 29, 2026

View reviewed changes

joeolson42 reviewed Jun 1, 2026

View reviewed changes

gsketefian added 2 commits June 1, 2026 11:30

Remove commented-out line (#on: [workflow_dispatch]) used to turn off…

ec3399f

… launching of CI tests on push.

Fix bug/typo.

6f44536

guoqing-noaa reviewed Jun 1, 2026

View reviewed changes

Comment thread Makefile Outdated

dustinswales reviewed Jun 1, 2026

View reviewed changes

guoqing-noaa reviewed Jun 2, 2026

View reviewed changes

joeolson42 approved these changes Jun 2, 2026

View reviewed changes

gsketefian added 6 commits June 2, 2026 11:03

Update hash of stochastic_physics.

1cdac0f

Enable the build-time macro CESMENABLED to access the "dgemm" matrix …

701ad92

…multiplication subroutine.

Update hash of stochastic_physics.

44b219d

Set the CESMCOUPLED build-time macro in the main Makefile so that whe…

ba6ce23

…n building with the stand-alone MPAS-Model, stochastic_physics uses "dgemm" instead of "esmf_dgemm" to perform matrix multiplication.

Edit comments for clarity.

058d494

Merge branch 'noaa/develop' into gsl/MPAS_stoch_physics_try_merge_sto…

db5705a

…ch_master

guoqing-noaa reviewed Jun 4, 2026

View reviewed changes

Conversation

gsketefian commented Mar 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Mandatory Questions

Priority Reviewers

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

gsketefian Jun 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

JeffBeck-NOAA Jun 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

JeffBeck-NOAA Jun 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

clark-evans commented May 29, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gsketefian Jun 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

dustinswales Jun 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

joeolson42 left a comment

Choose a reason for hiding this comment

Uh oh!

dustinswales commented Jun 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

gsketefian commented Mar 31, 2026 •

edited

Loading

gsketefian Jun 1, 2026 •

edited

Loading

JeffBeck-NOAA Jun 4, 2026 •

edited

Loading

JeffBeck-NOAA Jun 4, 2026 •

edited

Loading

gsketefian Jun 1, 2026 •

edited

Loading

dustinswales Jun 1, 2026 •

edited

Loading

dustinswales commented Jun 3, 2026 •

edited

Loading