Update CMORizer CLARA-AVHRR (A3)#4463
Conversation
axel-lauer
changed the title
hb326
left a comment
hb326
left a comment
There was a problem hiding this comment.
Works like a charm. Thanks, @axel-lauer!
I tried the downloader and formatter both.
One little thing: I commented in one of the files that it might be good to add a comment about why the data download end-date is set to 2020. This will most likely be helpful in a few month when we try to update the data until 2025...
hb326
left a comment
hb326
left a comment
There was a problem hiding this comment.
Thanks for adding the notes, @axel-lauer!
Looks good from my side now (I did not look at any Codacy issues...).
schlunma
left a comment
schlunma
left a comment
There was a problem hiding this comment.
2 comments regarding Codacy problems, others can be ignored.
| with gzip.open(file_path, "rb") as f_in: | ||
| with Path(outdir / file_path.stem).open("rb") as f_out: |
There was a problem hiding this comment.
| with gzip.open(file_path, "rb") as f_in: | |
| with Path(outdir / file_path.stem).open("rb") as f_out: | |
| with ( | |
| gzip.open(file_path, "rb") as f_in, | |
| Path(outdir / file_path.stem).open("rb") as f_out, | |
| ): |
| except Exception as ex: | ||
| logger.info("%s: no data downloaded for %s", type(ex), var_name) |
There was a problem hiding this comment.
This will more or less silently skip missing variables. Is this desirable? Can the dataset be used if files are missing? If not, I would remove this try/except block here.
There was a problem hiding this comment.
I agree with Manu - perhaps do a check on the required variables, and if at least one is missing, then raise an exception than mask via try/except
|
@schlunma are you doing a technical review? If not, I can do Axel a solid 🍻 |
No, please go for it 🍻 |
cheers, buds, on it now then 🍻 |
valeriupredoi
left a comment
valeriupredoi
left a comment
There was a problem hiding this comment.
looking good and many thanks for yet another retired NCL script @axel-lauer 🍻 Two very small observations from me, pun intended
| except Exception as ex: | ||
| logger.info("%s: no data downloaded for %s", type(ex), var_name) |
There was a problem hiding this comment.
I agree with Manu - perhaps do a check on the required variables, and if at least one is missing, then raise an exception than mask via try/except
| def _fix_coordinates(cube, definition): | ||
| """Fix coordinates.""" | ||
| axis2def = {"T": "time", "X": "longitude", "Y": "latitude"} | ||
| axes = ["T", "X", "Y"] |
There was a problem hiding this comment.
| axes = ["T", "X", "Y"] | |
| axes = axis2def.keys() |
valeriupredoi
added
approved by technical reviewer
and removed
looking for technical reviewer
labels
4 participants
Description
This PR replaces the old NCL CMORizer for CLARA-AVHRR data with a new Python version. The processed CLARA-AVHRR data (version A3) is obtained from the Copernicus Climate Data Store (CDS) and replaces the old version (A2).
Please note: I will not fix the remaining Codacy issues.
Checklist
It is the responsibility of the author to make sure the pull request is ready to review. The icons indicate whether the item will be subject to the 🛠 Technical or 🧪 Scientific review.
New or updated data reformatting script