Updated examples#446
Closed
browniea wants to merge 87 commits into
Closed
Conversation
Update documentation to use National Laboratory of the Rockies per: https://www.energy.gov/eere/articles/energy-department-renames-nrel-national-lab-rockies At this point in time <https://nrel.gov> is still active and <https://nlr.gov> does not work, so we are keeping the nrel.gov links the same. Also the HSDS api endpoints: `/nrel/US_wave/virtual_buoy/{region}/{region}_virtual_buoy_{year}.h5` `/nrel/wtk/{region.lower()}/{region}_*.h5` `/nlr/wtk/{region.lower()}-5min/{region}_*.h5` In ``` mhkit/wave/io/hindcast/hindcast.py mhkit/wave/io/hindcast/wind_toolkit.py ``` Are not updated yet, but may change in the future.
…ves example, critical PSD bugfix (MHKiT-Software#430) Collection of bugfixes and documentation for Nortek Signature ADCPs. I received a dual-profile datafile from a Nortek Signature250 deployed at PacWave that was collecting both water velocity and wave measurements, and I made some updates to the codebase when things would fail working through the standard ADCP workflow. DOLfYN is currently set up to return individual profiles as individual datasets, so there is one dataset containing the wave-relevant variables, and one containing the water velocity variables. Both datasets contain water velocity information, so to differentiate between the two, the "_avg" tag is added to the dataset containing what Nortek calls the "averaging" profile. However, this dataset no longer comes out of the box "bin-averaged"; individual pings from each duty cycle are now saved in the output file. I've been making updates to dolfyn so that the codebase will recognize "_avg" variables and default to them if untagged variables do not exist. The following updates are the latest: - The "U_mag" and "U_dir" shortcuts will using "vel_avg" to calculate speed and direction if "vel" does not exist in the dataset. - The shear functions in the ADCP turbulence API (`dudz`, `dvdz`, etc) can now utilize "vel_avg" if given as an input. - `calc_declination` will no longer complain if you try to update the magnetic declination in the Nortek-created bin-averaged binary file ("<filename>_avgd.ad2cp") Two, I created an example notebook showing how to calculate wave statistics using DOLfYN's FFT tools. We've gotten questions about this in the past, and now that I have a good dataset, this is a good time to document this. Three, I found a bug in the PSD functions where individual FFTs get a 50% overlap not once, but twice. The core FFT function applies a 50% overlap using a series of "for loops" (the more robust method), while the input FFT function `cpsd` was adding overlap via the "npad" input to the `reshape` function. This latter method pads the first and last FFT with a lot of zeros, which in turn corrupts the first and last spectrum of a timeseries. It appears the latter method was written first and improved upon via the second method and should have been removed. Removing it fixes said bug. --------- Co-authored-by: Adam Keester <72414466+akeeste@users.noreply.github.com>
…e#433) There are three main changes in this pull request: 1. Added the option to determine the number of datapoints in each FFT when creating the sound pressure PSDs. This was currently hardcoded to the maximum FFT length, i.e. the total number of datapoints in each window, which might be more resolution and require more storage space than desired 2. Changed names of bin and windowing related attributes on the PSDs to make them easier to understand 3. Gain was improperly added - the sign has been corrected Minor changes are refactoring some of the argument type checks to clean them up.
There is a bug where something in requiring pyarrow that is likely related to pandas 3.0. Adding pyarrow as a dependency is a reasonable fix, but managing the pyarrow version should be handled by pandas and not mhkit. This pins pandas below 3.0 to see if pyarrow dependencies are caused by including >= 3.0 somewhere.
Previously we had `python>=3.10,<3.13` but this was likely legacy from the conda-forge version of the conda config. This removes the ambiguous python version, which relied on the solver, to a deterministic version with the intent of providing reproducible builds for conda users.
PR to add functionality to read Aquadopp data packets, but this code has only been tested on Aquadopp profilers (not single-point Aquadopps). Note that Aquadopp/AWAC waves functionality has also been added but is untested. I've done a decent bit of cleanup of the nortek.py code in this PR as well. The PR also simplifies the code related to the non-cabled ADV's "orientation_down" parameter. (This parameter exists because the z-axis is flipped on the non-cabled version of the ADV). This PR no longer reads the bit set in each burst ping, which is wont to oscillate for an unknown reason, but instead takes the value from the datafile's header. This means that DOLfYN assumes that a non-cabled ADV is not moved after it starts running. Cabled ADVs, AWACs and Aquadopps also have the "orientation_down" parameter for user information, but it is not needed otherwise.
These changes were a reaction to failing tests, but it is likely that the failing tests were not caused by this code specifically, but pandas 3.0 updates. This reverts these changes back to the develop branch as they are likely unnecessary for the tests to pass when pandas is pinned below 3.0
``` FAILED .github/workflows/test_read_adp.py::io_adp_testcase::test_io_nortek - FileNotFoundError: [Errno 2] No such file or directory: '/home/runner/work/MHKiT-Python/MHKiT-Python/examples/data/dolfyn/AQD_HR.prf' ```
Package / Actions Cleanup - Continuation of MHKiT-Software#435
Updates to `io/d3d.py` code - Added new coordinate names for the latest version of Delft3D - new `calculate_grid_convergence_index` function to calculate the grid convergence index between two grid's results basted on the equation [GCI]( https://www.grc.nasa.gov/WWW/wind/valid/tutorial/spatconv.html) - Allows xarray or netCDF4 input --------- Co-authored-by: akeeste <akeeste@sandia.gov> Co-authored-by: Simms, Andrew <andrew.simms@nlr.gov>
Opening this to solve Issue MHKiT-Software#427 I changed the source code to return degrees instead of radians, which was a simple fix. I also went ahead and updated the polar plots so that zero degrees is at the top of the figure, and positive degrees runs clockwise. We should update this in other polar plots in MHKiT too. I also updated units to pass CF conventions, whereafter I noticed that our tests for this code check that the units are the same as what NDBC outputs. I'm not sure that's necessary. I also saw that we're not actually testing the value output from the source code, which is important. --------- Co-authored-by: Simms, Andrew <andrew.simms@nlr.gov>
## Pandas 3 Changes * Updated pandas dependency to allow versions >=2.2.2 without an upper limit in both `environment-dev.yml` and `pyproject.toml`, enabling support for pandas 3.x. * Added a new function `replace_pandas_missing_values_with_nan` in `mhkit/wave/io/ndbc.py` to replace missing values with `NaN` in a way that is compatible with both pandas 2.x and 3.x. ## Test Updates * Increased `test_get_buoy_metadata` latitude and longitude assertions delta tolerance to allow for a small amount real world buoy drift. * Fix deprecation in `test_request_parse_workflow_multiyear` (`mhkit/tests/wave/io/test_cdip.py`) by using `"D"` instead of `"d"` in the `floor` method. Pandas deprecated `d` in 3.0+ pandas-dev/pandas#58998. period aliases are here: https://pandas.pydata.org/docs/user_guide/timeseries.html#period-aliases
This PR verifies MHKiT-Python supports Python 3.13. No python code changes were required. README was updated to include Python 3.13 Actions were updated to include 3.13 in test matrices and for notebook tests (single version) Action versions for setup-miniconda and download-artifact were update
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
#432