#474: LFRic-Nudging: Part-A: Introduce ability to read nudging datafiles #474
#474: LFRic-Nudging: Part-A: Introduce ability to read nudging datafiles #474Mohit Dalvi (mcdalvi) wants to merge 26 commits into
Conversation
… time-axis definitions
…r the new arguments
|
Is there any chance main could be merged into this branch? I'm currently unable to test the changes with JEDI due to the conflicts |
|
Apologies, I was not aware creating a PR triggers all this activity ! I have changed it to draft till all my testing is complete. |
…at are defined in source code; implement upgrade macro
…2bit as done for other azspice runs; remove unused additions to ancil reading code
Matthew Hambley (MatthewHambley)
left a comment
There was a problem hiding this comment.
A few things to consider while we think about the linked ticket.
| type(mesh_type), pointer :: mesh => null() | ||
| type(mesh_type), pointer :: twod_mesh => null() |
There was a problem hiding this comment.
We no longer require initialisation on declaration of pointers. In fact we now prohibit it. Please use nullify(pointer) at the top of the procedure code, if it is not otherwise immediately initialised.
There was a problem hiding this comment.
Done b906772. also updated neighbouring pointer declarations for consistency.
| if (present(twod)) field_spec%twod=twod | ||
| if (present(empty)) field_spec%empty=empty | ||
| if (present(coarse)) field_spec%coarse=coarse | ||
| if (present(coarse_mesh_name)) field_spec%mesh_name=coarse_mesh_name |
There was a problem hiding this comment.
Why is the generic field_spec%mesh_name being assigned from the specific course_mesh_name. If the type member is generic, the initialising variable should be too. If the initialising variable is specific, the type member should also be.
There was a problem hiding this comment.
Code updated to use coarse_mesh_name throughout. b906772
| else | ||
| dim = 1 | ||
| end if | ||
| case ('ecmwf_levels') |
There was a problem hiding this comment.
Does this actually have anything to do with ECMWF or is that just the source of the data which happens to be being used at the moment?
There was a problem hiding this comment.
Changed to nudging_levels in all references now. b906772
|
|
||
| type(field_maker_type) :: creator | ||
|
|
||
| call log_event('GungHo: Creating nudging fields...', LOG_LEVEL_INFO) |
There was a problem hiding this comment.
Does this need to be "info" level. We already suffer the logging being much too talkative. This goes for all other new event logs.
There was a problem hiding this comment.
Changed to LOG_LEVEL_TRACE.
| if (coarse_rad_aerosol) then | ||
| mesh_name = aerosol_mesh_name | ||
| else | ||
| mesh_name = '' | ||
| end if |
There was a problem hiding this comment.
Wasn't this set above, outside the conditional? Do we have code duplication here?
| !> @brief Map moisture array enumerators to moisture array. | ||
| type(time_axis_dict_type), parameter :: time_axis_dict & | ||
| = time_axis_dict_type(525,529,536) | ||
| = time_axis_dict_type(525,529,536,542) |
There was a problem hiding this comment.
Where do these numbers come from? UM code? A comment to explain this would be handy.
There was a problem hiding this comment.
These numbers are part of the field_spec design itself. See original comments lfric_core:#4208, which seem to indicate they have to be unique for each field, within a different range for each field category.
There was a problem hiding this comment.
I see that, in which case, a comment explaining that this is where these numbers are defined and they are opaque identifiers with no intrinsic meaning would help future developers.
A better solution still would be to define them as integer, parameter :: usefule_name = 525, then use them here. If you make these parameters public, they can be used outside the module. If that is their purpose. Even if it isn't, keeping them private but having useful names would help developers of this module.
There was a problem hiding this comment.
Implemented. Note that I am not familiar with the design of the routine so hopefully this is acceptable to the respective section owner.
thomasmelvin
left a comment
There was a problem hiding this comment.
All looks good to me
Matthew Hambley (MatthewHambley)
left a comment
There was a problem hiding this comment.
There are also changes requested for core which may effect this change.
| @@ -0,0 +1,112 @@ | |||
| !----------------------------------------------------------------------------- | |||
| ! (c) Crown copyright 2025 Met Office. All rights reserved. | |||
There was a problem hiding this comment.
Wrong year, although we are not including year in the copyright statement any more.
There was a problem hiding this comment.
Done.
| use external_forcing_config_mod, only : theta_forcing_nudging, & | ||
| theta_forcing, & | ||
| wind_forcing_nudging, & | ||
| wind_forcing |
There was a problem hiding this comment.
Please use the new configuration API. You can find documentation of this at https://metoffice.github.io/lfric_core/how_to_use_it/lfric_datamodel/modeldb.html#accessing-configuration-data but please ask if you have further questions.
There was a problem hiding this comment.
Matthew Hambley (@MatthewHambley)
Re: using modeldb to access configuration values: I have attempted to implement this for process_nudging_fields. However process_nudging_fields is also called in gungho_model_mod/before_context_close, same as the LBC fields. This before_contaxt_close routine name is actually passed as a procedure pointer to init_io -> lfric_core:driver_io and finally called from lfric_xios_context_mod. I cannot see a way to use the modeldb argument at this level in the code hierarchy.
Thomas Bendall (@tommbendall)
There was a problem hiding this comment.
As mntioned above it does not look to be feasible to access modeldb at all places where 'process_nudging_fields' is called (namely from lfric_core:lfric_xios_context_mod'. However, modeldb has been introduced in create_nudging_fields to access clock.
The configuration API has been used in other routines where new configuration variables are being accessed and modeldb is already available e.g. gngo_driver_mod, gungho_init_fields_mod. It does seem that the API does not return a default value (RMDI/ EMDI) if the namelist/ item is not active in a particular model application. The availability has to be explicitly checked before trying to access value of the namelist item, otherwise this causes a invalid memory access error while obtaining the item value.
There was a problem hiding this comment.
While it's not possible to pass ModelDB to the callback due to a circular dependency, it is possible to pass any configuration you need. Either as primitive values, if there are few of them, or as a namelist type if you need lots of configuration.
This is not an ideal solution as it bloats the callback's argument list but it at least makes the problem explicit. The problem still exists when fetching from global scope, it is just obfuscated. It is more important that we move away from global scope configuration and I think we would rather see our problems than not.
As of yesterday the location of the callback has changed so you will have to merge your branch up to head of trunk before making any changes.
Ideally this would be done by creating a new branch from your development branch and merging that up to head of trunk. This leaves a branch based on a release tag, which the GC group can use, should they care to, and one which can actually merge onto trunk. Unfortunately it doesn't seem to be possible to change the source branch of a pull request.
Instead please add a comment to this pull request with the hash of the head-of-branch before you merge up. This can be used to retrieve the version against the release tag.
There was a problem hiding this comment.
Matthew Hambley (@MatthewHambley) I am going to need help in understanding the best way to pass the configuration values to process_nudging_fields. The callback routine gungho_model_mod__before_context_close that calls the former, is now defined in lfric_core:driver_io_mod as a io_configuration_type with only clock as the argument. Since it already receives modeldb%clock, can this be extended to receive modeldb itself by changing the configuration_type to type(modeldb_type), intent(in) :: modeldb, or will that interfere with other locations where 'before_close' variants might be called?
The same issue will arise even if just the required configuration values were to be explicitly passed.
There was a problem hiding this comment.
Do you need modeldb? if you only need the configuration, can you not just pass the config_type object? So you would end up with 2 arguments the config and clock.?
There was a problem hiding this comment.
This has been implemented now as proposed, passing modeldb to process_nudging_fields, and in driver_io under lfric_core.
Tests run overnight complete except for 'nwp_gal9_eda' which seems to have failed in ~umtest/ as well.
There was a problem hiding this comment.
Ricky Wong (@mo-rickywong) , sorry your comment did not appear till I refreshed ! Is there an example of the config_type argument I can look at?
| !> @param[in] mesh The current 3d mesh | ||
| !> @param[in] twod_mesh The current 2d mesh (not used here) | ||
| !> @param[in] mapper Provides access to the field collections | ||
| !> @param[in] clock The model clock | ||
| subroutine create_nudging_fields(mesh, twod_mesh, mapper, clock) |
There was a problem hiding this comment.
If the 2D mesh isn't being used, why pass it in? If it is needed later, then it can be obtained from the mesh collection. Something like mesh_collection%get_mesh(mesh, extrusion=TWOD) but check the API.
Given that you also need the clock and field collections, it might be more straight forward to pass modeldb in. Note that it also contains the configuration so no need to pass that either.
There was a problem hiding this comment.
The comment appears to be left over from earlier version: twod_mesh is passed to tbe 'creator%init' routine.
Clock is now accessed from modeldb'
| !> @brief Map moisture array enumerators to moisture array. | ||
| type(time_axis_dict_type), parameter :: time_axis_dict & | ||
| = time_axis_dict_type(525,529,536) | ||
| = time_axis_dict_type(525,529,536,542) |
There was a problem hiding this comment.
I see that, in which case, a comment explaining that this is where these numbers are defined and they are opaque identifiers with no intrinsic meaning would help future developers.
A better solution still would be to define them as integer, parameter :: usefule_name = 525, then use them here. If you make these parameters public, they can be used outside the module. If that is their purpose. Even if it isn't, keeping them private but having useful names would help developers of this module.
| use multires_coupling_config_mod, only : coarse_nudging, & | ||
| nudging_mesh_name |
There was a problem hiding this comment.
As noted above, we are migrating to configuration objects.
There was a problem hiding this comment.
Introduced in this and other routines where new items are being accessed.
| if (coarse_rad_aerosol) then | ||
| mesh_name = aerosol_mesh_name | ||
| else | ||
| mesh_name = '' | ||
| end if | ||
|
|
There was a problem hiding this comment.
Can you just pass the mesh object rather than messing about with names?
There was a problem hiding this comment.
Possibly, but this will require re-writing all calls to 'make_spec' if not for the aerosol ones with 'coarse' option.
I have opened isue #554 to standardize handling of fields on coarse mesh, although this will be better addressed by a core LFRic developer.
…places where *process_nudging_fields* is called
… additional checks that the namelist is active in current application
Matthew Hambley (MatthewHambley)
left a comment
There was a problem hiding this comment.
One remaining issue, then we can go to trunk.
| use external_forcing_config_mod, only : theta_forcing_nudging, & | ||
| theta_forcing, & | ||
| wind_forcing_nudging, & | ||
| wind_forcing |
There was a problem hiding this comment.
While it's not possible to pass ModelDB to the callback due to a circular dependency, it is possible to pass any configuration you need. Either as primitive values, if there are few of them, or as a namelist type if you need lots of configuration.
This is not an ideal solution as it bloats the callback's argument list but it at least makes the problem explicit. The problem still exists when fetching from global scope, it is just obfuscated. It is more important that we move away from global scope configuration and I think we would rather see our problems than not.
As of yesterday the location of the callback has changed so you will have to merge your branch up to head of trunk before making any changes.
Ideally this would be done by creating a new branch from your development branch and merging that up to head of trunk. This leaves a branch based on a release tag, which the GC group can use, should they care to, and one which can actually merge onto trunk. Unfortunately it doesn't seem to be possible to change the source branch of a pull request.
Instead please add a comment to this pull request with the hash of the head-of-branch before you merge up. This can be used to retrieve the version against the release tag.
|
Latest working commit for dev branch before merging main: 7818d2d |
…ess_nudging_fields*
Matthew Hambley (MatthewHambley)
left a comment
There was a problem hiding this comment.
This change now looks to be ready for trunk.
Thanks Matthew Hambley (@MatthewHambley) . Just adding a reminder here about the test input files to be placed in BIGDATA |
|
#502 has now been committed and I expect this to cause a conflict with this PR. In #502 I split create_model_data into create_model_data and create_physics_model_data. Ideally the changes on this ticket should go into create_physics_model_data and then there should not be any change in the linear_model. |
I will merge with main to resolve conflicts, but I believe the external_forcing with nudging is independant of physics. In fact we have had to move some existing code sections out of #ifdef UM_PHYSICS. |
cjohnson-pi thanks for flagging the conflict! Good question Mohit Dalvi (@mcdalvi) -- I think it could be argued either way. I'd probably lean towards us adding the new fields to the new |
Thanks for the clarification Tom, will merge in the changes accordingly. |
PR Summary
Sci/Tech Reviewer: Thomas Bendall (@tommbendall)
Code Reviewer: Matthew Hambley (@MatthewHambley)
This PR introduces the ability to read in datafiles that will be used for Nudging the Momentum models (Ref: Nudging NWP to AI/ML models - possibly targeting PS49).
The handling of Nudging data is modelled on existing lateral boundary condition handling in LFRic since they are both required to be interpolated from high time resolution file data on every model timestep.
Nudging changes summary:
Ability to read in nudging data for temperature, U-wind, V-wind and Log surface pressure (required for vertical remapping to model levels).
New [namelist:nudging], which will be active when theta or wind are 'forced by nudging' (in namelist:external_forcing). A corresponding upgrade_macro to add the namelist and I/O related items.
Introduce routine create_nudging_fields to create the nudging input fields as '2-D with user-specified (nudge_data) levels'. Corresponding grid_def_nudging.xml and grid_def_nudging_coarse.xml added to lfric_atm/metadata/.
Introduce ability to read in nudging data on native ('dynamics') or a coarser grid, with rationalising of routines that create the model fields ('make_spec', 'create_model_data') to accept both aerosol and nudging coarse mesh definitions.
Note that the above requires a LFRIC_CORE change to allow defining 3-D coarse grid fields as ''multigrid_ll'' x ndata.
Define a nudging_time_axis which will be populated based on the actual number of time records in the specified nudging file.
Introduce two new rose-stem tests lfric_atm_nwp_gal9_nudging-C48 that reads nudging files on the model grid and lfric_atm_nwp_gal9_nudging_coarse-C48 that reads in data at a lower resolution (C12) grid. This currently uses nudging data created from ERA5 source, but aim is to expand this to AIFS and other models.
At this stage the reference fields are only made available to LFRic and do not modify temperature/ winds.
linked #369: Allow coarse-grid fields with ndata levels: vn3.1 read coarse 2d lfric_core#369
Code Quality Checklist
Testing
Test Branch: test4_vn3p1_nudging_data
trac.log
Test Suite Results - lfric_apps - test4_vn3p1_nudging_data/run4
Suite Information
Task Information
❌ failed tasks - 1
⌛ waiting tasks - 1
Performance Impact
AI Assistance and Attribution
VsCode - code completion/ snippets.
Documentation
PSyclone Approval
Sci/Tech Review
(Please alert the code reviewer via a tag when you have approved the SR)
Code Review