Add support for psyclone 3.3, nvfortran and amdflang

[Sergi Siso (sergisiso)]

PR Summary

Sci/Tech Reviewer:
Code Reviewer: Andrew Coughtrie (@andrewcoughtrie)

Add support for amdflang and provide workarounds for the nvfortran limitations discussed with Hacka Fett (@christophermaynard)

Also add support for the upcoming psyclone release (3.3) and update broken psyclone links and oudated names. These are not breaking changes, the code remains compabible with the current psyclone release 3.2.2.

Code Quality Checklist

• I have performed a self-review of my own code
• My code follows the project's style guidelines
• Comments have been included that aid understanding and enhance the readability of the code
• My changes generate no new warnings
• All automated checks in the CI pipeline have completed successfully

Testing

• I have tested this change locally, using the LFRic Core rose-stem suite
• If required (e.g. API changes) I have also run the LFRic Apps test suite using this branch
• If any tests fail (rose-stem or CI) the reason is understood and acceptable (e.g. kgo changes)
• I have added tests to cover new functionality as appropriate (e.g. system tests, unit tests, etc.)
• Any new tests have been assigned an appropriate amount of compute resource and have been allocated to an appropriate testing group (i.e. the developer tests are for jobs which use a small amount of compute resource and complete in a matter of minutes)

trac.log

Security Considerations

• I have reviewed my changes for potential security issues
• Sensitive data is properly handled (if applicable)
• Authentication and authorisation are properly implemented (if applicable)

Performance Impact

• Performance of the code has been considered and, if applicable, suitable performance measurements have been conducted

AI Assistance and Attribution

• Some of the content of this change has been produced with the assistance of Generative AI tool name (e.g., Met Office Github Copilot Enterprise, Github Copilot Personal, ChatGPT GPT-4, etc) and I have followed the Simulation Systems AI policy (including attribution labels)

Documentation

• Where appropriate I have updated documentation related to this change and confirmed that it builds correctly

PSyclone Approval

• If you have edited any PSyclone-related code (e.g. PSyKAl-lite, Kernel interface, optimisation scripts, LFRic data structure code) then please contact the TCD Team

Sci/Tech Review

• I understand this area of code and the changes being added
• The proposed changes correspond to the pull request description
• Documentation is sufficient (do documentation papers need updating)
• Sufficient testing has been completed

(Please alert the code reviewer via a tag when you have approved the SR)

Code Review

• All dependencies have been resolved
• Related Issues have been properly linked and addressed
• CLA compliance has been confirmed
• Code quality standards have been met
• Tests are adequate and have passed
• Documentation is complete and accurate
• Security considerations have been addressed
• Performance impact is acceptable

[Thomas Bendall (tommbendall)]

Looks fine from a code owner perspective. I'm not sure how it was previously compiling without r_def being imported!

[Sergi Siso (sergisiso)]

Thomas Bendall (@tommbendall) r_def only appears in an invoke's kernel argument, and the whole expression is moved to the psy_layer where there is an unqualified constants_mod import. This happens to make the generated code valid, but we are introducing stricter symbol checking in PSyclone, which will report these undefined symbols. There are a few similar cases in lfric_apps that I will also propose fixes for.

[Thomas Bendall (tommbendall)]

Thomas Bendall (Thomas Bendall (@tommbendall)) r_def only appears in an invoke's kernel argument, and the whole expression is moved to the psy_layer where there is an unqualified constants_mod import. This happens to make the generated code valid, but we are introducing stricter symbol checking in PSyclone, which will report these undefined symbols. There are a few similar cases in lfric_apps that I will also propose fixes for.

That makes sense, thanks for explaining!

[Matthew Hambley (MatthewHambley)]

Build system code owner saying "this looks okay."

[Sergi Siso (sergisiso)]

Matthew Hambley (@MatthewHambley) I addressed your comments and merged with main. I don't think I can cover more bullet points form the description as I cannot launch the rose-stem tests myself, but let me know if there is something else needed.

[Matthew Hambley (MatthewHambley)]

Code Owner (Build System) Review: Looks like there was a minor misunderstanding.

[Matthew Hambley (MatthewHambley)]

Code owner (build system) happy with this.

[Iva Kavcic (TeranIvy)]

Testing with PSyclone master on Azure Spice

Changes made to test PSyclone master

diff --git a/rose-stem/site/meto/common/suite_config_azspice.cylc b/rose-stem/site/meto/common/suite_config_azspice.cylc
index 6af6a464e..94403b092 100644

--- a/rose-stem/site/meto/common/suite_config_azspice.cylc
+++ b/rose-stem/site/meto/common/suite_config_azspice.cylc
@@ -7,17 +7,22 @@
 
 {% set azspice_base = 'umask 0022 ; '~
                       'module purge ; '~
-                      'module use /home/users/lfricadmin/lmod ; ' %}
+                      'module use /home/users/lfricadmin/lmod ; '~
+                      'module use /home/users/lfricadmin/test_modules ; ' %}
 
-{% set azspice_compiler_gnu = 'module load lfric/vn3.1' %}
+{#% set azspice_compiler_gnu = 'module load lfric/vn3.1' %#}
+{% set azspice_compiler_gnu = 'module load lfric/vn3.2-rc1 ; '~
+                              'module load py-psyclone/master' %}
 
 {% set azspice_run = 'ulimit -s unlimited' %}
 
 {% set azspice_scitools = 'module load scitools/production-os47-1' %}
 
-{% set azspice_tech = 'module load lfric/vn3.1' %}
+{#% set azspice_tech = 'module load lfric/vn3.1' %#}
+{% set azspice_tech = 'module load lfric/vn3.2-rc1 ; '~
+                      'module load py-psyclone/master' %}
 
-{% set azspice_coupled_gnu = 'ml xios/2701-oasis ; '~
+{% set azspice_coupled_gnu = 'ml xios/2.2701-oasis ; '~
                                   'ml oasis/3-mct5.0' %}

I would recommend making these changes to facilitate testing until the full lfric/vn3.2 environment with PSyclone 3.3 is built. To adopt lfric/vn3.2, the relevant changes in rose-stem/site/meto/common/suite_config_azspice.cylc would be:

-{% set azspice_compiler_gnu = 'module load lfric/vn3.1' %}
+{% set azspice_compiler_gnu = 'module load lfric/vn3.2' %}
 
-{% set azspice_tech = 'module load lfric/vn3.1' %}
+{% set azspice_tech = 'module load lfric/vn3.2' %}
 
-{% set azspice_coupled_gnu = 'ml xios/2701-oasis ; '~
+{% set azspice_coupled_gnu = 'ml xios/2.2701-oasis ; '~
                                   'ml oasis/3-mct5.0' %}

Note that xios/2701-oasis changes to xios/2.2701-oasis to differentiate between XIOS2 r2701 used in LFRic, and an XIOS3 build with Oasis available in test modules.

Testing outcome

Checked out the stfc:psyclone_next fork as test_338_core. Azure Spice tests (cylc vip -z group=azspice -n test_338_core ./rose-stem) are all fine.

P.S. I checked the "PSyclone Approval" tick box as a TCD Team representative.

[Sergi Siso (sergisiso)]

Iva Kavcic (@TeranIvy) Note that the changes in this PR also work with the previous release of PSyclone and the tests should succeed without the rose-stem changes.

I am saying this in case you already want to merge this to keep it out of the way and leave the remaining psyclone update conversation for the LFRic_apps PR (that one is not backwards compatible).

[Sergi Siso (sergisiso)]

Iva Kavcic (@TeranIvy) I have resolved the conflicts with main. Can you test it with rose-stem without moving to psyclone master?

[Iva Kavcic (TeranIvy)]

Iva Kavcic (Iva Kavcic (@TeranIvy)) I have resolved the conflicts with main. Can you test it with rose-stem without moving to psyclone master?

Sergi Siso (@sergisiso), I tested the fork with PSyclone master (on Azure Spice, group=azspice) and with the current lfric/vn3.1 software stack (on both Azure Spice and EX, group=all). I can confirm that the Rose Stem tests pass for both and the changes are indeed backward-compatible.

I also asked the SSD Team about merging this PR without waiting for the Apps PR and they're ok with it. We will need another Core PR for the change of software stack (changes as in my comment above).