diff --git a/doc/.gitignore b/doc/.gitignore index e8797ee56dc..009affbc7a0 100644 --- a/doc/.gitignore +++ b/doc/.gitignore @@ -4,3 +4,4 @@ source/Tools_api source/CIME_api source/_autosummary source/generated +build/ diff --git a/doc/source/ccs/index.rst b/doc/source/ccs/index.rst index e22d59f2221..45177d5e74d 100644 --- a/doc/source/ccs/index.rst +++ b/doc/source/ccs/index.rst @@ -18,7 +18,7 @@ The first thing to do is clone the model repository into ``$SRCROOT``. All examples will be run from ``$CIMEROOT`` which is should exist under ``$SRCROOT`` e.g. (``$CIMEROOT`` would be ``$SRCROOT/cime``). -Next set the ``CIME_MODEL`` evnironment variable for your model, e.g. ``export CIME_MODEL=e3sm``. +Next set the ``CIME_MODEL`` environment variable for your model, e.g. ``export CIME_MODEL=`` (see :ref:`Setting up your environment for CCS ` for valid values). .. note:: @@ -155,16 +155,18 @@ Customizations The CCS allows for a number of customizations to be made to a case. +.. _ccs-env-setup: + Setting up your environment for CCS ``````````````````````````````````` After you've cloned the model repository, you'll need to set up your environment to use the CCS. First you'll need to let CIME know which model configuration to use by setting the ``CIME_MODEL`` environment variable. In bash, use **export** as shown and replace -**** with the appropriate text. Current possibilities are "e3sm", "cesm", or "ufs". +**** with the appropriate text. Current possibilities are "e3sm", "cesm", or "ufs". :: - export CIME_MODEL= + export CIME_MODEL= There are a number of possible ways to set CIME variables. For variables that can be set in more than one way, the order of precedence is: diff --git a/doc/source/ccs/model-configuration/support-a-new-machine.rst b/doc/source/ccs/model-configuration/support-a-new-machine.rst index 4b0b6952d29..319e5a6f341 100644 --- a/doc/source/ccs/model-configuration/support-a-new-machine.rst +++ b/doc/source/ccs/model-configuration/support-a-new-machine.rst @@ -68,24 +68,88 @@ As an example, on a MAC with 2 cores that has mpich with gnu fortran you would i > mpif90 fhello_world_mpi.F90 -o hello_world > mpirun -np 2 ./hello_world -CESM Linux and Mac Support +Linux and Mac Support --------------------------- -The distribution of CESM includes machines called **homebrew** and **centos7-linux** in the file **$CIMEROOT/config/cesm/machines/config_machines.xml**. +Your model distribution may include generic machine definitions in the model's ``config_machines.xml`` file. +The location of this file is model-dependent and is set by the ``MACHINES_SPEC_FILE`` entry in the model's ``config_files.xml`` (see :ref:`MACHINES_SPEC_FILE `); refer to your model repository for the exact path. Please see the instructions in the file to create the directory structure and use these generic machine definitions. +.. _config-machines-schema-versions: + +About config_machines.xml +--------------------------- + +The ``config_machines.xml`` file describes the machines on which CIME-driven models can be built and run. +Both its location and the XML schema(s) it must conform to are **model-dependent**. + +The schema(s) advertised by a model are declared as ```` children of the ``MACHINES_SPEC_FILE`` entry in that model's ``config_files.xml``. +The XSD applied at validation time is then selected from the ``version`` attribute on the root ```` element of the file being loaded. +The two supported schemas are: + +**Version 2.0 (monolithic)** — ``$CIMEROOT/CIME/data/config/xml_schemas/config_machines.xsd`` + A single file contains the full definition for every supported machine. + Each ```` block carries its own ```` (used by CIME to auto-detect the machine), and ```` must appear at least once at the top level. + + .. code-block:: xml + + + + ... + mymachine.* + LINUX + ... + + + ... + + + +**Version 3.0 (per-machine directory)** — ``$CIMEROOT/CIME/data/config/xml_schemas/config_machines_version3.xsd`` + The top-level ``config_machines.xml`` only holds a registry that maps each machine to its node-name regex. + The full definition for each machine lives in its own subdirectory (``//config_machines.xml``) next to that machine's ``config_batch.xml`` and ``*.cmake`` files. + Top-level ```` blocks are optional, ```` is removed from individual ```` blocks, and the v3 schema additionally accepts a ```` element per machine. + + .. code-block:: xml + + + + mymachine.* + other.* + + + + Per-machine file (``/mymachine/config_machines.xml``): + + .. code-block:: xml + + + + ... + LINUX + ... + + + +Not every model registers both schemas. Refer to the ``MACHINES_SPEC_FILE`` entry in your model's ``config_files.xml`` to see which schema versions are available; if no ``version`` attribute is present on the ```` entry, the model accepts a single unversioned schema matching the **2.0** layout. + +See :ref:`MACHINES_SPEC_FILE ` for the full element reference for both schema versions. + Steps for porting --------------------------- Porting CIME involves several steps. The first step is to define your machine. You can do this in one of two ways: -1. You can edit **$CIMEROOT/config/$model/machines/config_machines.xml** and add an appropriate section for your machine. +1. You can edit the model's ``config_machines.xml`` directly (the path is given by ``MACHINES_SPEC_FILE`` in the model's ``config_files.xml``) and add an appropriate section for your machine. + For a **version 3.0** layout, instead add (or update) a ```` entry under the top-level ```` and create a ``//`` subdirectory containing the per-machine ``config_machines.xml``, ``config_batch.xml`` and ``*.cmake`` files. 2. You can use your **$HOME/.cime** directory (see :ref:`customizing-cime`). In particular, you can create a **$HOME/.cime/config_machines.xml** file with the definition for your machine. - A template to create this definition is provided in **$CIMEROOT/config/xml_schemas/config_machines_template.xml**. More details are provided in the template file. + A template to create this definition is provided in **$CIMEROOT/CIME/data/config/xml_schemas/config_machines_template.xml**. More details are provided in the template file. In addition, if you have a batch system, you will also need to add a **config_batch.xml** file to your **$HOME/.cime** directory. - All files in **$HOME/.cime/** are appended to the xml objects that are read into memory from the **$CIME/config/$model**, where **$model** is either ``e3sm`` or ``cesm``. + All files in **$HOME/.cime/** are appended to the xml objects that are read into memory from the directory containing the model's ``MACHINES_SPEC_FILE``. + + .. note:: When the model uses the **version 3.0** schema, the per-machine override is read from **$HOME/.cime//config_machines.xml** (and a top-level **$HOME/.cime/config_machines.xml** may be used to register the ``NODENAME_REGEX``). .. note:: If you use method (2), you can download CIME updates without affecting your machine definitions in **$HOME/.cime**. @@ -97,26 +161,34 @@ In what follows we outline the process for method (2) above: This file contains all the information you must set in order to configure a new machine to be CIME-compliant. - Fill in the contents of **$HOME/.cime/config_machines.xml** that are specific to your machine. For more details see :ref:`the config_machines.xml file `. + Fill in the contents of **$HOME/.cime/config_machines.xml** that are specific to your machine. For more details see :ref:`MACHINES_SPEC_FILE ` and :ref:`the schema overview above `. - Check to ensure that your **config_machines.xml** file conforms to the CIME schema definition by doing the following: + Check to ensure that your **config_machines.xml** file conforms to the CIME schema definition by validating it against the XSD that matches the ``version`` declared on its root element: :: - xmllint --noout --schema $CIME/config/xml_schemas/config_machines.xsd $HOME/.cime/config_machines.xml + # version 2.0 (also used when the model registers only an unversioned schema) + xmllint --noout --schema $CIMEROOT/CIME/data/config/xml_schemas/config_machines.xsd $HOME/.cime/config_machines.xml + + # version 3.0 + xmllint --noout --schema $CIMEROOT/CIME/data/config/xml_schemas/config_machines_version3.xsd $HOME/.cime/config_machines.xml - If you find that you need to introduce compiler settings specific to your machine, create a **$HOME/.cime/*.cmake** file. - The default compiler settings are defined in **$CIME/config/$model/machines/cmake_macros/**. + The default compiler settings are defined in the ``cmake_macros/`` directory that lives alongside the model's ``MACHINES_SPEC_FILE``. - If you have a batch system, you may also need to create a **$HOME/.cime/config_batch.xml** file. - Out-of-the-box batch settings are set in **$CIME/config/$model/machines/config_batch.xml**. + Out-of-the-box batch settings are set in the ``config_batch.xml`` file that lives alongside the model's ``MACHINES_SPEC_FILE`` (its path is given by ``BATCH_SPEC_FILE`` in the model's ``config_files.xml``). - Once you have defined a basic configuration for your machine in your **$HOME/.cime** xml files, run the :ref:`unittest `. After running those steps correctly, you are ready to try a case at your target compset and resolution. -Validating a CESM port with prognostic components +Validating a port with prognostic components ------------------------------------------------- +.. note:: + + This section is specific to CESM (and NorESM). Consult your model's documentation for the equivalent procedure. + The following port validation is recommended for any new machine. Carrying out these steps does not guarantee the model is running properly in all cases nor that the model is scientifically valid on @@ -133,15 +205,15 @@ possible. Users are responsible for their own validation process, especially with respect to science validation. -These are the recommended steps for validating a port for the CESM model: +These are the recommended steps for validating a port: -1. Verify basic functionality of your port by performing the cheyenne "prealpha" tests on your machine. This can be done by issuing the following command: +1. Verify basic functionality of your port by performing the "prealpha" tests on your machine. This can be done by issuing the following command: :: - ./create_test --xml-category prealpha --xml-machine cheyenne --xml-compiler intel --machine --compiler + ./create_test --xml-category prealpha --xml-machine --xml-compiler --machine --compiler - This command will run the prealpha tests *defined* for cheyenne with the intel compiler, but will run them on *your* machine with *your* compiler. + This command will run the prealpha tests *defined* for the reference machine with the reference compiler, but will run them on *your* machine with *your* compiler. These tests will be run in the **$CIME_OUTPUT_ROOT**. To see the results of tests, you need to do the following: :: @@ -153,11 +225,11 @@ These are the recommended steps for validating a port for the CESM model: 2. Carry out ensemble consistency tests: This is described in ``$CIMEROOT/tools/statistical_ensemble_test/README``. - The CESM-ECT (CESM Ensemble Consistency Test) determines whether a new simulation set up (new machine, compiler, etc.) is statistically distinguishable from an accepted ensemble. - The ECT process involves comparing several runs (3) generated with the new scenario to an ensemble built on a trusted machine (currently cheyenne). + The Ensemble Consistency Test (ECT) determines whether a new simulation set up (new machine, compiler, etc.) is statistically distinguishable from an accepted ensemble. + The ECT process involves comparing several runs (3) generated with the new scenario to an ensemble built on a trusted machine. The python ECT tools are located in the pyCECT subdirectory ``$CIMEROOT/tools/statistical_ensemble_test/pyCECT``. - The verification tools in the CESM-ECT suite are: + The verification tools in the ECT suite are: ``CAM-ECT``: detects issues in CAM and CLM (12 month runs) @@ -166,9 +238,8 @@ These are the recommended steps for validating a port for the CESM model: ``POP-ECT``: detects issues in POP and CICE (12 month runs) Follow the instructions in the **README** file to generate three ensemble runs for any of the above tests that are most relevant to your port. - Then please go to the `CESM2 ensemble verification website `_, where you can upload your files and subsequently obtain a quick response as to the success or failure of your verification. -Performance tuning of a CESM port +Performance tuning of a port ------------------------------------------------- Once you have performed the verification that your port is successful, diff --git a/doc/source/ccs/model-configuration/variables/compsets.rst b/doc/source/ccs/model-configuration/variables/compsets.rst index 13d5c82aa3d..5719c9de0c4 100644 --- a/doc/source/ccs/model-configuration/variables/compsets.rst +++ b/doc/source/ccs/model-configuration/variables/compsets.rst @@ -48,7 +48,7 @@ Every file listed in ``COMPSETS_SPEC_FILE`` will be searched to compile possible CIME will note which component's config_compsets.xml had the matching compset name and that component will be treated as the **primary component** As an example, the primary component for a compset that has a prognostic atmosphere, -land and cice (in prescribed mode) and a data ocean is the atmosphere component (for cesm this is CAM) because the compset +land and cice (in prescribed mode) and a data ocean is the atmosphere component (e.g. CAM in CESM) because the compset is defined, using the above example, in ``$SRCROOT/components/cam/cime_config/config_compsets.xml`` In a compset where all components are prognostic, the primary component will be **allactive**. diff --git a/doc/source/ccs/model-configuration/variables/grids.rst b/doc/source/ccs/model-configuration/variables/grids.rst index b35c0cc409a..6bee7517cd4 100644 --- a/doc/source/ccs/model-configuration/variables/grids.rst +++ b/doc/source/ccs/model-configuration/variables/grids.rst @@ -387,14 +387,14 @@ The steps for adding a new component grid to the model system follow. This proce At this time, if you are running with a new ocean or runoff grid, please contact Michael Levy (mlevy_AT_ucar_DOT_edu) for assistance. If you are running with standard ocean and runoff grids, the mapping file should already exist and you do not need to generate it. -6. CESM specific: If you are adding a new atmosphere grid, this means you are also generating a new land grid, and you will need to create a new CLM surface dataset. (Otherwise you can skip this step). - You need to first generate mapping files for CLM surface dataset (since this is a non-standard grid). +6. If you are adding a new atmosphere grid, this means you are also generating a new land grid, and you may need to create a new land surface dataset. (Otherwise you can skip this step). + You need to first generate mapping files for the land surface dataset (since this is a non-standard grid). :: > cd $CIMEROOT/../components/clm/tools/mkmapdata > ./mkmapdata.sh --gridfile --res --gridtype global - These mapping files are then used to generate CLM surface dataset. Below is an example for a current day surface dataset (model year 2000). + These mapping files are then used to generate the land surface dataset. Below is an example for a current day surface dataset (model year 2000). :: @@ -403,7 +403,7 @@ The steps for adding a new component grid to the model system follow. This proce 7. Create grid file needed for create_newcase. The next step is to add the necessary new entries in the appropriate ``config_grids.xml`` file. - You will need to modify ``$CIMEROOT/config/cesm/config_grids.xml`` or ``$CIMEROOT/config/e3sm/config_grids.xml`` depending on the value of ``$CIME_MODEL``. + You will need to modify ``$CIMEROOT/config/$model/config_grids.xml`` depending on the value of ``$CIME_MODEL``. You will need to: - add a single ```` entry diff --git a/doc/source/ccs/model-configuration/variables/pes.rst b/doc/source/ccs/model-configuration/variables/pes.rst index e8be806b939..733739837fb 100644 --- a/doc/source/ccs/model-configuration/variables/pes.rst +++ b/doc/source/ccs/model-configuration/variables/pes.rst @@ -436,8 +436,8 @@ help determine the optimal load balance. Changing the pe layout of the model has NO IMPACT on the scientific results. The basic order of operations and calling sequence are hardwired into the driver and do not change with the pe -layout. However, both CESM and E3SM do impose some contraints in the -tempororal evolution of the components. For example, the prognostic +layout. However, models such as CESM and E3SM may impose some constraints in the +temporal evolution of the components. For example, the prognostic atmosphere model always run sequentially with the ice and land models for scientific reasons. As a result, running the atmosphere concurrently with the ice and land will result in idle processors at diff --git a/doc/source/ccs/running-a-case.rst b/doc/source/ccs/running-a-case.rst index 04c8a709308..f721e94821f 100644 --- a/doc/source/ccs/running-a-case.rst +++ b/doc/source/ccs/running-a-case.rst @@ -19,6 +19,10 @@ Before submitting a case, it is a good idea to preview the run to ensure that th Example output: +.. note:: + + The following is an example output. The specific paths, environment variables, and commands will vary depending on your model and machine configuration. + .. code-block:: bash CASE INFO: @@ -343,6 +347,8 @@ The XML variable ``$GET_REFCASE`` is a flag that if set will automatically pre-s - If ``$GET_REFCASE`` is ``FALSE``, then the data is assumed to already exist in ``$RUNDIR``. +.. _restarting-a-run: + Restarting a Run ````````````````` Active components (and some data components) write restart files @@ -500,7 +506,7 @@ or a hybrid run or to back up to a previous restart date. Long-term Archiving ``````````````````` Users may choose to follow their institution's preferred method for long-term -archiving of model output. Previous releases of CESM provided an external +archiving of model output. Previous releases of some CIME-driven models provided an external long-term archiver tool that supported mass tape storage and HPSS systems. However, with the industry migration away from tape archives, it is no longer feasible for CIME to support all the possible archival schemes available. @@ -516,7 +522,7 @@ Scripts ``````` Variables ``PRERUN_SCRIPT`` and ``POSTRUN_SCRIPT`` can each be used to name a script which should be executed immediately prior starting or -following completion of the CESM executable within the batch +following completion of the model executable within the batch environment. The script is expected to be found in the case directory and will receive one argument which is the full path to that directory. If the script is written in python and contains a diff --git a/doc/source/ccs/setting-up-a-case.rst b/doc/source/ccs/setting-up-a-case.rst index b67f6fffb6e..070af038619 100644 --- a/doc/source/ccs/setting-up-a-case.rst +++ b/doc/source/ccs/setting-up-a-case.rst @@ -28,18 +28,18 @@ To print detailed information about variables, including their description, type ./xmlquery --listall --full -To view a specific group, replace ```` with the desired group name. +To view a specific group, replace ``GROUP`` with the desired group name. .. code-block:: bash - ./xmlquery --subgroup --listall + ./xmlquery --subgroup GROUP --listall -You can search for partial matches using the ``-p`` or ``--partial`` option (can take a regex) and replacing ```` with the string you want to search for. This will return all variables that contain the search string in their name or value. +You can search for partial matches using the ``-p`` or ``--partial`` option (can take a regex) and replacing ``SEARCH_STRING`` with the string you want to search for. This will return all variables that contain the search string in their name or value. .. code-block:: bash # e.g. ./xmlquery --partial STOP, ./xmlquery --partial 'STOP_(N|O)' - ./xmlquery --partial + ./xmlquery --partial SEARCH_STRING Variables can be printed without being resolved by using the ``--no-resolve`` option. @@ -61,19 +61,19 @@ The ``xmlchange`` command is used to modify the configuration of a case. The fol .. code-block:: bash - ./xmlchange = + ./xmlchange VARIABLE=VALUE The `value` can be literal or a reference to another value. When using a reference, it must be prefixed with `$`. .. code-block:: bash - ./xmlchange =$ + ./xmlchange VARIABLE=$VARIABLE The reference can also define the `subgroup`. This is usful when a variable exists under multiple subgroups and a specific one needs to be referenced. The `subgroup` and `variable` are delimited with `::`. .. code-block:: bash - ./xmlchange =$:: + ./xmlchange VARIABLE=$SUBGROUP::VARIABLE Some variables can exist in multiple groups. To change a variable in a specific group, use the ``--subgroup`` option. @@ -95,7 +95,7 @@ For example, if a model's atmosphere model (DATM) was located in the directory ` Users can customize a component models's namelist in two ways: -1. By editing the ``$CASEROOT/user_nl_`` files +1. By editing the ``$CASEROOT/user_nl_COMP`` files These files should be modified via keyword-value pairs that correspond to new namelist or input data settings. They use the syntax of Fortran namelists. @@ -159,9 +159,9 @@ The namelist file for DATM is **datm_in** (or **datm_in_NNN** for multiple insta - To modify the contents of a DATM stream file, first run ``preview_namelists`` to list the *streams.txt* files in the **CaseDocs/** directory. Then, in the same directory: 1. Make a *copy* of the file with the string *"user_"* prepended. - ``> cp datm.streams.txt.[extension] user_datm.streams.txt[extension.`` + ``> cp datm.streams.txt.EXTENSION user_datm.streams.txt.EXTENSION`` 2. **Change the permissions of the file to be writeable.** (chmod 644) - ``chmod 644 user_datm.streams.txt[extension`` + ``chmod 644 user_datm.streams.txt.EXTENSION`` 3. Edit the **user_datm.streams.txt.*** file. **Example** @@ -180,9 +180,9 @@ The namelist file for DOCN is **docn_in** (or **docn_in_NNN** for multiple insta - To modify the contents of a DOCN stream file, first run ``preview_namelists`` to list the *streams.txt* files in the **CaseDocs/** directory. Then, in the same directory: 1. Make a *copy* of the file with the string *"user_"* prepended. - ``> cp docn.streams.txt.[extension] user_docn.streams.txt[extension.`` + ``> cp docn.streams.txt.EXTENSION user_docn.streams.txt.EXTENSION`` 2. **Change the permissions of the file to be writeable.** (chmod 644) - ``chmod 644 user_docn.streams.txt[extension`` + ``chmod 644 user_docn.streams.txt.EXTENSION`` 3. Edit the **user_docn.streams.txt.*** file. **Example** @@ -201,9 +201,9 @@ The namelist file for DICE is ``dice_in`` (or ``dice_in_NNN`` for multiple insta - To modify the contents of a DICE stream file, first run ``preview_namelists`` to list the *streams.txt* files in the **CaseDocs/** directory. Then, in the same directory: 1. Make a *copy* of the file with the string *"user_"* prepended. - ``> cp dice.streams.txt.[extension] user_dice.streams.txt[extension.`` + ``> cp dice.streams.txt.EXTENSION user_dice.streams.txt.EXTENSION`` 2. **Change the permissions of the file to be writeable.** (chmod 644) - ``chmod 644 user_dice.streams.txt[extension`` + ``chmod 644 user_dice.streams.txt.EXTENSION`` 3. Edit the **user_dice.streams.txt.*** file. Data Land (DLND) @@ -217,9 +217,9 @@ The namelist file for DLND is ``dlnd_in`` (or ``dlnd_in_NNN`` for multiple insta - To modify the contents of a DLND stream file, first run ``preview_namelists`` to list the *streams.txt* files in the **CaseDocs/** directory. Then, in the same directory: 1. Make a *copy* of the file with the string *"user_"* prepended. - ``> cp dlnd.streams.txt.[extension] user_dlnd.streams.txt[extension.`` + ``> cp dlnd.streams.txt.EXTENSION user_dlnd.streams.txt.EXTENSION`` 2. **Change the permissions of the file to be writeable.** (chmod 644) - ``chmod 644 user_dlnd.streams.txt[extension`` + ``chmod 644 user_dlnd.streams.txt.EXTENSION`` 3. Edit the **user_dlnd.streams.txt.*** file. Data River (DROF) @@ -233,90 +233,28 @@ The namelist file for DROF is ``drof_in`` (or ``drof_in_NNN`` for multiple insta - To modify the contents of a DROF stream file, first run ``preview_namelists`` to list the *streams.txt* files in the **CaseDocs/** directory. Then, in the same directory: 1. Make a *copy* of the file with the string *"user_"* prepended. - ``> cp drof.streams.txt.[extension] user_drof.streams.txt[extension.`` + ``> cp drof.streams.txt.EXTENSION user_drof.streams.txt.EXTENSION`` 2. **Change the permissions of the file to be writeable.** (chmod 644) - ``chmod 644 user_drof.streams.txt[extension`` + ``chmod 644 user_drof.streams.txt.EXTENSION`` 3. Edit the **user_drof.streams.txt.*** file. -.. TODO:: remove cesm specific docs +Customizing active component-specific namelist settings +------------------------------------------------------- -Customizing CESM active component-specific namelist settings ------------------------------------------------------------- +Active components typically provide a ``buildnml`` script in their ``cime_config`` directory that generates the component's namelist variables. Component-specific CIME xml variables are set in the component's ``config_component.xml`` file and are used by the ``buildnml`` script to generate the namelist. -CAM -``` +To modify an active component's namelist settings, add the appropriate keyword/value pair at the end of the **$CASEROOT/user_nl_COMP** file, where ``COMP`` is the component name. See the documentation at the top of each ``user_nl`` file for details. -CIME calls **$SRCROOT/components/cam/cime_config/buildnml** to generate the CAM's namelist variables. - -CAM-specific CIME xml variables are set in **$SRCROOT/components/cam/cime_config/config_component.xml** and are used by CAM's **buildnml** script to generate the namelist. - -For complete documentation of namelist settings, see `CAM namelist variables `_. - -To modify CAM namelist settings, add the appropriate keyword/value pair at the end of the **$CASEROOT/user_nl_cam** file. (See the documentation for each file at the top of that file.) - -For example, to change the solar constant to 1363.27, modify **user_nl_cam** file to contain the following line at the end: +For example, to modify a namelist variable for an atmosphere component, edit the corresponding ``user_nl`` file: :: - solar_const=1363.27 - -To see the result, call ``preview_namelists`` and verify that the new value appears in **CaseDocs/atm_in**. - -CLM -``` - -CIME calls **$SRCROOT/components/clm/cime_config/buildnml** to generate the CLM namelist variables. - -CLM-specific CIME xml variables are set in **$SRCROOT/components/clm/cime_config/config_component.xml** and are used by CLM's **buildnml** script to generate the namelist. - -For complete documentation of namelist settings, see `CLM namelist variables `_. - -To modify CLM namelist settings, add the appropriate keyword/value pair at the end of the **$CASEROOT/user_nl_clm** file. - -To see the result, call ``preview_namelists`` and verify that the changes appear correctly in **CaseDocs/lnd_in**. - -MOSART -`````` - -CIME calls **$SRCROOT/components/mosart/cime_config/buildnml** to generate the MOSART namelist variables. + VARIABLE_NAME=VALUE -To modify MOSART namelist settings, add the appropriate keyword/value pair at the end of the **$CASEROOT/user_nl_rtm** file. +To see the result, call ``preview_namelists`` and verify that the new value appears in the appropriate file under **CaseDocs/**. -To see the result of your change, call ``preview_namelists`` and verify that the changes appear correctly in **CaseDocs/rof_in**. - -CICE -```` - -CIME calls **$SRCROOT/components/cice/cime_config/buildnml** to generate the CICE namelist variables. - -For complete documentation of namelist settings, see `CICE namelist variables `_. - -To modify CICE namelist settings, add the appropriate keyword/value pair at the end of the **$CASEROOT/user_nl_cice** file. -(See the documentation for each file at the top of that file.) -To see the result of your change, call ``preview_namelists`` and verify that the changes appear correctly in **CaseDocs/ice_in**. - -In addition, ``case.setup`` creates CICE's compile time `block decomposition variables `_ in **env_build.xml**. - -POP2 -```` - -CIME calls **$SRCROOT/components/pop2/cime_config/buildnml** to generate the POP2 namelist variables. - -For complete documentation of namelist settings, see `POP2 namelist variables `_. - -To modify POP2 namelist settings, add the appropriate keyword/value pair at the end of the **$CASEROOT/user_nl_pop2** file. -(See the documentation for each file at the top of that file.) -To see the result of your change, call ``preview_namelists`` and verify that the changes appear correctly in **CaseDocs/ocn_in**. - -CISM -```` - -See `CISM namelist variables `_ for a complete description of the CISM runtime namelist variables. This includes variables that appear both in **cism_in** and in **cism.config**. - -To modify any of these settings, add the appropriate keyword/value pair at the end of the **user_nl_cism** file. (See the documentation for each file at the top of that file.) -Note that there is no distinction between variables that will appear in **cism_in** and those that will appear in **cism.config**: simply add a new variable setting in **user_nl_cism**, and it will be added to the appropriate place in **cism_in** or **cism.config**. -To see the result of your change, call ``preview_namelists`` and verify that the changes appear correctly in **CaseDocs/cism_in** and **CaseDocs/cism.config**. +.. note:: -Some CISM runtime settings are sets via **env_run.xml**, as documented in `CISM runtime variables `_. + Refer to your model's documentation for a complete list of namelist variables and their descriptions for each active component. Setting up the Case ------------------- @@ -340,7 +278,7 @@ If you set the ``CASE_GIT_REPOSITORY`` variable to a valid Git repository URL, t .. code-block:: bash - ./xmlchange CASE_GIT_REPOSITORY= + ./xmlchange CASE_GIT_REPOSITORY=REPOSITORY .. note:: @@ -359,6 +297,6 @@ Depends.* Lists of source code files that need special bui Macros.cmake File containing machine-specific makefile directives for your target platform/compiler. This file is created if it does not already exist. The user can modify the file to change certain aspects of the build, such as compiler flags. Running ``case.setup --clean`` will not remove the file once it has been created. However, if you remove or rename the Macros.make file, running ``case.setup`` recreates it. case.st_archive Script to perform short-term archiving to disk for your case output. Note that this script is run automatically by the normal CIME workflow. cmake_macros/ Directory containing any CMake macros required for the machine/compiler combination. -user_nl_xxx[_NNNN] Files where all user modifications to component namelists are made. **xxx** is any one of the set of components targeted for the case. For example, for a full active CESM compset, **xxx** is cam, clm, or rtm, and so on. NNNN goes from 0001 to the number of instances of that component. (See :ref:`multiple instances`) For a case with 1 instance of each component (default), NNNN will not appear in the user_nl file names. A user_nl file of a given name is created only once. Calling ``case.setup --clean`` will *not remove* any user_nl files. Changing the number of instances in the **env_mach_pes.xml** file will cause only new user_nl files to be added to ``$CASEROOT``. +user_nl_xxx[_NNNN] Files where all user modifications to component namelists are made. **xxx** is any one of the set of components targeted for the case. For example, for a fully active compset, **xxx** could be the atmosphere, land, or river component name, and so on. NNNN goes from 0001 to the number of instances of that component. (See :ref:`multiple instances`) For a case with 1 instance of each component (default), NNNN will not appear in the user_nl file names. A user_nl file of a given name is created only once. Calling ``case.setup --clean`` will *not remove* any user_nl files. Changing the number of instances in the **env_mach_pes.xml** file will cause only new user_nl files to be added to ``$CASEROOT``. software_environment.txt This file records some aspects of the computing system on which the case is built, such as the shell environment. ============================= =============================================================================================================================== diff --git a/doc/source/ccs/timers.rst b/doc/source/ccs/timers.rst index c94d01b29a7..68598cdd494 100644 --- a/doc/source/ccs/timers.rst +++ b/doc/source/ccs/timers.rst @@ -56,6 +56,10 @@ The following describes the most important parts of this timing file: An example timing file of this type is: +.. note:: + + The following is an example timing output. The specific machine, paths, compset, and component names will vary depending on your model and configuration. + .. code-block:: text ---------------- TIMING PROFILE --------------------- diff --git a/doc/source/ccs/troubleshooting.rst b/doc/source/ccs/troubleshooting.rst index cff6a130c5b..fc0100504ab 100644 --- a/doc/source/ccs/troubleshooting.rst +++ b/doc/source/ccs/troubleshooting.rst @@ -37,7 +37,7 @@ Take these steps to check for problems: Runtime problems --------------------------------- -To see if a run completed successfully, check the last several lines of the **cpl.log** file for a string like ``SUCCESSFUL TERMINATION``. A successful job also usually copies the log files to the **$CASEROOT/logs** directory. +To see if a run completed successfully, check the last several lines of the **cpl.log** file for one of the termination markers ``SUCCESSFUL TERMINATION``, ``END OF MODEL RUN``, or ``HAS ENDED``. A successful job also usually copies the log files to the **$CASEROOT/logs** directory. Check these things first when a job fails: @@ -90,6 +90,9 @@ The output looks like this: tStamp_write: model date = 10120 0 wall clock = 2009-09-28 09:10:46 avg dt = 58.58 dt = 58.18 tStamp_write: model date = 10121 0 wall clock = 2009-09-28 09:12:32 avg dt = 60.10 dt = 105.90 +.. note:: + + The ``tStamp_write`` format shown above is produced by MCT/cpl7-based drivers; other drivers may emit different timing diagnostics. Review the run times at the end of each line for each model day. The "avg dt =" is the average time to simulate a model day and "dt = " is the time needed to simulate the latest model day. @@ -103,11 +106,8 @@ For instance, on the last day of every simulated month, the model typically writ Also, some model configurations read data mid-month or run physics intermittently at a timestep longer than one day. In those cases, some variability is expected. The time variation typically is quite erratic and unpredictable if the problem is system performance variability. -Sometimes when a job times out or overflows disk space, the restart files will get mangled. -With the exception of the CAM and CLM history files, all the restart files have consistent sizes. - -Compare the restart files against the sizes of a previous restart. If they don't match, remove them and move the previous restart into place before resubmitting the job. -See `Restarting a run `_. +Sometimes when a job times out or overflows disk space, the restart files will get mangled. Since restart files normally have consistent sizes across runs, compare them against a previous restart -- if they don't match, remove them and move the previous restart into place before resubmitting the job. +See :ref:`restarting-a-run`. It is not uncommon for nodes to fail on HPC systems or for access to large file systems to hang. Before you file a bug report, make sure a case fails consistently in the same place. diff --git a/doc/source/ccs/user-config.rst b/doc/source/ccs/user-config.rst index 8c3f44acdb0..4da800c1277 100644 --- a/doc/source/ccs/user-config.rst +++ b/doc/source/ccs/user-config.rst @@ -19,7 +19,7 @@ Variables can be defined in the **[main]** section of the config file. .. code-block:: ini [main] - CIME_MODEL= + CIME_MODEL= PROJECT= MAIL_TYPE= SRCROOT= diff --git a/doc/source/contributing-guide.rst b/doc/source/contributing-guide.rst index e43b1785f92..7e098e32da5 100644 --- a/doc/source/contributing-guide.rst +++ b/doc/source/contributing-guide.rst @@ -161,16 +161,16 @@ following example assumes the model is checked out in ``$SRC_PATH``. .. code-block:: bash - docker run -it --rm --hostname docker -e CIME_MODEL=e3sm -v ${SRC_PATH}:/root/model -v ./storage:/root/storage -w /root/E3SM/cime ghcr.io/esmci/cime:latest bash + docker run -it --rm --hostname docker -e CIME_MODEL= -v ${SRC_PATH}:/root/model -v ./storage:/root/storage -w /root/model/cime ghcr.io/esmci/cime:latest bash This example will drop into a shell where CIME commands or tests can be run. The options are broken down below. - ``--hostname docker`` is required to tell CIME which machine definition to use. -- ``-e CIME_MODEL=e3sm`` defines the model. -- ``-v ${SRC_PATH}:/root/E3SM`` passes through the model source. +- ``-e CIME_MODEL=`` defines the model. +- ``-v ${SRC_PATH}:/root/model`` passes through the model source. - ``-v ./storage:/root/storage`` persist all data; cases, baselines, archive, inputdata. the bind mounts can be broken out if you only want to persist certain input/outputs. -- ``-w /root/E3SM/cime`` set the current working directory to CIME's root. +- ``-w /root/model/cime`` set the current working directory to CIME's root. - ``ghcr.io/esmci/cime:latest`` container image. - ``bash`` the command to run in the container. @@ -178,9 +178,9 @@ You can even run CIME or testing without a shell. .. code-block:: bash - docker run -it --rm --hostname docker -e CIME_MODEL=e3sm -v ${SRC_PATH}:/root/model -v ./storage:/root/storage -w /root/E3SM/cime ghcr.io/esmci/cime:latest pytest CIME/tests/test_unit* + docker run -it --rm --hostname docker -e CIME_MODEL= -v ${SRC_PATH}:/root/model -v ./storage:/root/storage -w /root/model/cime ghcr.io/esmci/cime:latest pytest CIME/tests/test_unit* .. code-block:: bash - docker run -it --rm --hostname docker -e CIME_MODEL=e3sm -v ${SRC_PATH}:/root/model -v ./storage:/root/storage -w /root/E3SM/cime ghcr.io/esmci/cime:latest ./scripts/create_test SMS.f19_g16.S + docker run -it --rm --hostname docker -e CIME_MODEL= -v ${SRC_PATH}:/root/model -v ./storage:/root/storage -w /root/model/cime ghcr.io/esmci/cime:latest ./scripts/create_test SMS.f19_g16.S diff --git a/doc/source/system_testing.rst b/doc/source/system_testing.rst index 24b08f35e63..a7add4421aa 100644 --- a/doc/source/system_testing.rst +++ b/doc/source/system_testing.rst @@ -81,7 +81,7 @@ The search for test names can be restricted to a single test list using:: Omitting this results in searching all testlists listed in:: - cime/config/{cesm,e3sm}/config_files.xml + cime/config/$model/config_files.xml The ``./scripts/query_testlists`` tool gathers descriptions of the tests and testlists available in the XML format, the components, and projects. @@ -379,7 +379,7 @@ TESTMODS Name of the directory under ``GROUP`` that contains any combination A test mod can contain any combination of ``user_nl_*``, ``shell_commands``, ``user_mods``, or ``params.py``. -For example, the ``ERP`` test for an E3SM ``F-case`` can be modified to use a different radiation scheme by using ``eam-rrtmgp``:: +For example, the ``ERP`` test for an E3SM ``F-case`` can be modified to use a different radiation scheme by using ``eam-rrtmgp``, where ``pm-cpu_intel`` is the E3SM-specific machine/compiler combination (Perlmutter CPU partition with Intel compiler):: ERP_D_Ld3.ne4pg2_oQU480.F2010.pm-cpu_intel.eam-rrtmgp @@ -444,7 +444,7 @@ Lets look at the output from ``./scripts/create_test SMS.f19_f19.A``. Here you c .. code-block:: bash - Creating test directory /home/jgfouca/e3sm/scratch/SMS.f19_f19.A.melvin_gnu.20170504_163152_31aahy + Creating test directory /home/user/scratch/SMS.f19_f19.A.melvin_gnu.20170504_163152_31aahy RUNNING TESTS: SMS.f19_f19.A.melvin_gnu Starting CREATE_NEWCASE for test SMS.f19_f19.A.melvin_gnu with 1 procs @@ -461,7 +461,7 @@ Lets look at the output from ``./scripts/create_test SMS.f19_f19.A``. Here you c Finished RUN for test SMS.f19_f19.A.melvin_gnu in 35.068546 seconds (PASS). [COMPLETED 1 of 1] At test-scheduler close, state is: PASS SMS.f19_f19.A.melvin_gnu RUN - Case dir: /home/jgfouca/e3sm/scratch/SMS.f19_f19.A.melvin_gnu.20170504_163152_31aahy + Case dir: /home/user/scratch/SMS.f19_f19.A.melvin_gnu.20170504_163152_31aahy test-scheduler took 154.780044079 seconds The case is created in ``$CASEDIR`` which can be seen in the output above. The test status is stored in ``$CASEDIR/TestStatus``.