[mpas_aero_v0] feat: initial YAML configuration and modularization

.github/workflows/ci.yml

@@ -3,8 +3,8 @@ name: CI
 on:
   push:
 #    branches: [ main ]
-#  pull_request:
-#    branches: [ main ]
+  pull_request:
+#    branches: [ mpas_aero_v0 ]
 
 jobs:
   lint-and-test:

.gitignore

@@ -12,3 +12,5 @@ __pycache__/
 *.DS_Store
 
 src/regrid_wrapper.egg-info/
+
+/build/

.pre-commit-config.yaml

@@ -43,6 +43,5 @@ repos:
 
 exclude: |
   (?x)^(
-    \.venv/|
-    src/regrid_wrapper/app/chem_regrid/chem_regrid_impl\.py
+    \.venv
   )

README.md

@@ -2,6 +2,39 @@
 
 The `rw` command provides a set of command-line tools for regridding.
 
+## "Chem Regrid" Application
+
+Install via `pip install .` or access via `<regrid-wrapper root dir>/src/regrid-wrapper/app/rw_cli.py`.
+
+```
+usage: rw chem-regrid [-h] [--yaml-path YAML_PATH] [--root-key ROOT_KEY] [--overrides OVERRIDES [OVERRIDES ...]]
+
+options:
+  -h, --help            show this help message and exit
+  --yaml-path YAML_PATH
+                        If provided, path to YAML file containing the configuration's root key
+  --root-key ROOT_KEY   If provided, use this key when extracting the root configuration
+  --overrides OVERRIDES [OVERRIDES ...]
+                        If provided, override arbitrary key+values (e.g. --override key1:nest=val1 key2=val2)
+```
+
+Example:
+
+```shell
+python ${rw_dir}/src/regrid_wrapper/app/rw_cli.py chem-regrid \
+    --overrides workdir=${cr_workdir} \
+                input_dir=${cr_input_dir} \
+                output_dir=${cr_output_dir} \
+                weight_dir=${cr_weight_dir} \
+                scrip_path=${cr_scrip_path} \
+                dst_path=${cr_dst_path} \
+                cycle=${cr_cycle} \
+                mesh_name=${cr_mesh_name} \
+                ebb_dcycle=1 \
+                dataset_name=RAVE \
+                fcst_length=6
+```
+
 ## MPAS to UGRID Conversion
 
 1. `conda env create -f environment-uxarray.yaml`
@@ -44,6 +77,71 @@ cd /opt/project && \
 
 ... or mpi tests:
 
-```
+```bash
 mpirun -n 8 pytest -m mpi src/test
 ```
+
+It is recommended to run `pre-commit` hooks when developing:
+
+```shell
+cd /opt/project && pre-commit run --all-files
+```
+
+# Adding a New Dataset
+
+To add a new dataset to the regridding pipeline, follow these steps:
+
+1.  **Update `DatasetName` Enum**: Add the new dataset key to the `DatasetName` enum in `src/regrid_wrapper/app/chem_regrid/dataset/config/model.py`.
+2.  **Add Configuration**: Add a new entry to `src/regrid_wrapper/app/chem_regrid/dataset/config/datasets.yml` following the schema described above.
+3.  **Create Regrid Context Subclass**: In `src/regrid_wrapper/app/chem_regrid/dataset/context/`, create a new module (e.g., `my_dataset.py`) and a subclass of `AbstractDatasetRegridContext` (e.g., `MY_DATASET_DatasetRegridContext`).
+    *   Implement `iter_file_pairs` to define how source and destination files are paired.
+    *   Override methods as needed for dataset-specific logic.
+4.  **Register the Subclass**: Import and return the new context class in `regrid_wrapper.app.chem_regrid.dataset.context.__init__.py.get_regrid_context_class`.
+5. **Add Test**: Add a new test case for the dataset in `src/test/test_app/test_chem_regrid/conftest.py`.
+
+## Dataset Configuration
+
+Datasets are configured in `src/regrid_wrapper/app/chem_regrid/dataset/config/datasets.yml`. Each entry defines how a specific dataset should be read and regridded.
+
+### Dataset Schema
+
+| Field | Description |
+|---|---|
+| `field_names` | List of variable names to be regridded from the source file. |
+| `x_center` | Variable name for longitude centers. |
+| `y_center` | Variable name for latitude centers. |
+| `x_dim` | Dimension name for the X (longitude) axis. |
+| `y_dim` | Dimension name for the Y (latitude) axis. |
+| `x_corner` | (Optional) Variable name for longitude corners. Set to `null` if not available. |
+| `y_corner` | (Optional) Variable name for latitude corners. Set to `null` if not available. |
+| `x_corner_dim` | (Optional) Dimension name for longitude corners. |
+| `y_corner_dim` | (Optional) Dimension name for latitude corners. |
+| `level_in_name` | (Optional) Name of the vertical level dimension in the source file. |
+| `level_out_name` | Name of the vertical level dimension in the output file. |
+| `level_out_size` | Number of vertical levels in the output. Set to `0` for 2D data. |
+| `time_name` | (Optional) Name of the time dimension in the source file. |
+| `time_size` | Number of time steps. Set to `0` if time dimension is not used. |
+| `InterpMethod` | ESMF interpolation method (e.g., `CONSERVE`, `BILINEAR`, `NEAREST_STOD`). |
+
+### Example Entry
+
+```yaml
+MY_DATASET:
+  field_names:
+    - PM25
+    - SO2
+  x_center: lon
+  y_center: lat
+  x_dim: x
+  y_dim: y
+  x_corner: null
+  y_corner: null
+  x_corner_dim: null
+  y_corner_dim: null
+  level_in_name: null
+  level_out_name: nkanthro
+  level_out_size: 1
+  time_name: time
+  time_size: 1
+  InterpMethod: CONSERVE
+```

src/regrid_wrapper/app/chem_regrid/__init__.py

@@ -0,0 +1,3 @@
+from regrid_wrapper.context.logging import LOGGER
+
+CR_LOGGER = LOGGER.getChild("mpas-regrid")

src/regrid_wrapper/app/chem_regrid/chem_regrid_cli.py

@@ -3,8 +3,8 @@
 
 import yaml
 
+from regrid_wrapper.app.chem_regrid.chem_regrid_context import ChemRegridContext
 from regrid_wrapper.app.chem_regrid.chem_regrid_impl import main
-from regrid_wrapper.app.chem_regrid.context import ChemRegridContext
 from regrid_wrapper.app.override import apply_overrides
 
 

src/regrid_wrapper/app/chem_regrid/chem_regrid_context.py

@@ -0,0 +1,53 @@
+from functools import cached_property
+from pathlib import Path
+
+from pydantic import Field
+
+from regrid_wrapper.app.chem_regrid.dataset.config.model import ChemRegridDataset
+from regrid_wrapper.app.chem_regrid.dataset.context import DatasetName
+from regrid_wrapper.common import RwBaseModel
+
+
+class ChemRegridContext(RwBaseModel):
+    """This is the API class for the regridding implementation. These fields may be customized by
+    users."""
+
+    dataset_name: DatasetName
+    workdir: Path
+    input_dir: Path
+    output_dir: Path
+    weight_dir: Path
+    cycle: str = Field(pattern=r"^\d{10}$")  # Validates YYYYMMDDHH format
+    mesh_name: str
+    input_mesh_path: Path | None
+    dst_path: Path | None
+    ebb_dcycle: int
+    fcst_length: int
+    datasets_yml_path: Path = Path(__file__).parent / "dataset" / "config" / "datasets.yml"
+
+    @cached_property
+    def rw_input_mesh_path(self) -> Path:
+        if self.input_mesh_path is None:
+            return self.workdir / f"mpas_{self.dataset_name.value}-{self.mesh_name}_scrip.nc"
+        return self.input_mesh_path
+
+    @cached_property
+    def rw_dst_path(self) -> Path:
+        if self.dst_path is None:
+            return self.workdir / "init.nc"
+        return self.dst_path
+
+    @cached_property
+    def rw_desc_stats_out(self) -> Path:
+        return self.workdir / f"desc_stats-{self.cycle}.csv"
+
+    @cached_property
+    def rw_dataset(self) -> ChemRegridDataset:
+        return ChemRegridDataset.from_key(self.datasets_yml_path, self.dataset_name)
+
+    @cached_property
+    def rw_weight_path(self) -> Path:
+        weight_path = self.weight_dir / (
+            "weights_" + self.dataset_name.value + "-to-" + "mpas_" + self.mesh_name + "_" + self.rw_dataset.InterpMethod + ".nc"
+        )
+        return weight_path