Bugfix OpenMeteo Resource Local Timezone by elenya-grant · Pull Request #814 · NatLabRockies/H2Integrate

elenya-grant · 2026-07-27T17:07:36Z

Bugfix OpenMeteo Resource Local Timezone

Solar resource data pulled from using the OpenMeteo solar resource model would sometimes result in surprisingly low capacity factors of solar PV plants. This underestimate was occurring when data was being downloaded in the local time rather than UTC (default behavior is to use UTC, if the user doesnt explicitly define the timezone in the plant config, then it uses UTC). This hinted that the issue may be related to the timezone and the time stamps. The solar resource data and the corresponding timezone that the data is in are both passed to the PvWatts performance model - so the issue is a mismatch between the data_tz and the actual timezone of the corresponding time information in the solar resource data. Anyway - thats fixed now.

Ready for another review. New questions for reviewers is outlined in Section 2

Section 1: Type of Contribution

Section 2: Draft PR Checklist

Open draft PR
Describe the feature that will be added
Fill out TODO list steps
Describe requested feedback from reviewers on draft PR

TODO:

Update the OpenMeteo Wind Resource model
Update the tests that will fail
Remove resource files deemed unnecessary by reviewers
Add unit tests for the solar resource (upon reviewer response/suggestion)

Type of Reviewer Feedback Requested (on Draft PR)

Structural feedback:

What (if any) additional tests would make sense to add into h2integrate/resource/solar/test/test_resource_models.py? I was thinking that checking that data downloaded in UTC and data downloaded in local timezone are the same. For example, a site with a utc offset of -6 hours should have the same resource data utc_data[6] == local_data[0]. But - I can't test this anymore because we removed the resource files downloaded in UTC. Open to any other ideas on testing.
Should I update any of the doc pages to note some of the differences between:
- OpenMeteo resource data and NLR developer resource datasets?
- OpenMeteo resource data downloaded from the web vs OpenMeteo resource data downloaded using the API call in H2I?

Implementation feedback:

DONE: Should the new method make_time_index() (in OpenMeteoHistoricalSolarResource) be turned into a function that lives in h2integrate/resource/utilities/download_tools.py or a new file in h2integrate/resource/utilities/ since it'll be used by both the OpenMeteo solar resource data model and the OpenMeteo wind resource model?

Other feedback (outdated):

The best way to test this fix (in its full completeness) involves having at least 3-5 resource files downloaded for the same site. This would mean like adding 2-3 more resource files to the repo - which I agree isn't ideal but I'm not sure how else to test that the new method is:

compatible with the previous way of downloading/saving resource data using the API
- (requires file -28.454864_114.551749_2020_openmeteo_archive_solar_60min_local_tz-init.csv and -28.454864_114.551749_2020_openmeteo_archive_solar_60min_local_tz.csv)
compatible with the new way of downloading/saving resource data using the the API
- (requires at least -28.454864_114.551749_2020_openmeteo_archive_solar_60min_local_tz.csv)
compatible with downloading resource data from the web
- (requires at least files open-meteo-28.44S114.55E71m-local_tz_2020.csv and -28.454864_114.551749_2020_openmeteo_archive_solar_60min_local_tz.csv.)

The following files could be removed to remove certain types of testing stuff:

open-meteo-28.44S114.55E71m-local_tz_2020.csv: if removed would not test that openmeteo web download in local time is similar to openmeteo API download in local time
-28.454864_114.551749_2020_openmeteo_archive_solar_60min_local_tz-init.csv: if removed, would not test compatibility with resource files downloaded with the openmeteo API in local time prior to bugfix
-28.454864_114.551749_2020_openmeteo_archive_solar_60min_local_tz.csv: if removed, would not test that results are similar to web download data and/or results using data from the himawari dataset in local time
-28.454864_114.551749_2020_himawari8_v3_60min_local_tz.csv: if removed, does not allow to compare openmeteo results against another dataset

New feedback requested:
Could remove one more resource file (-28.454864_114.551749_2020_openmeteo_archive_solar_60min_local_tz-init.csv) and instead use the existing resource file -28.454864_114.551749_2024_openmeteo_archive_solar_60min_local_tz.csv which has resource data for the same site but a different resource but was downloaded the "old" way. This would mean we aren't able to do a direct year-to-year comparison of solar resource from the old download method to the new download method but the results for each of these years are pretty similar for this site. Thoughts?

Section 3: General PR Checklist

PR description thoroughly describes the new feature, bug fix, etc.
Added tests for new functionality or bug fixes
Tests pass (If not, and this is expected, please elaborate in the Section 6: Test Results)
Documentation
- Docstrings are up-to-date
- Related docs/ files are up-to-date, or added when necessary
- [-] Documentation has been rebuilt successfully
- [-] Examples have been updated (if applicable)
CHANGELOG.md
- At least one complete sentence has been provided to describe the changes made in this PR
- After the above, a hyperlink has been provided to the PR using the following format:
  "A complete thought. [PR XYZ]((https://github.com/NatLabRockies/H2Integrate/pull/XYZ)", where
  XYZ should be replaced with the actual number.

Section 4: Related Issues

May resolve Issue #656

Section 5: Impacted Areas of the Software

Section 5.1: New Files

resource_files/solar/
- open-meteo-28.44S114.55E71m-local_tz_2020.csv: web downloaded data in local timezone
- -28.454864_114.551749_2020_openmeteo_archive_solar_60min_local_tz-init.csv: API download in local timezone before bugfix
- -28.454864_114.551749_2020_openmeteo_archive_solar_60min_local_tz.csv: API download in local timezone after bugfix
- -28.454864_114.551749_2020_himawari8_v3_60min_local_tz.csv: himawari dataset API download in local TZ

Section 5.2: Modified Files

OpenMeteoHistoricalSolarResource in h2integrate/resource/solar/openmeteo_solar.py
- setup():
  - added "is_day" to the hourly_solar_data_to_units attribute (very helpful for debugging because you can see what time the sunrise and sunset is)
- make_time_index: new method that properly gets the time information from a file
- download_data:
  - added utc=True to the pd.date_time calls to make time_data and also added usage of tz_convert if the time_data is downloaded in local timezone
    - added data_tz to the header. Not used by the code at the moment but helpful for seeing which files are downloaded with this bugfix.

Section 6: Additional Supporting Information

Data downloaded from the web has time information formatted as YYYY-MM-DDTHH:mm (ex: 2013-12-31T00:00)

New data downloaded has the time information formatted as YYYY-MM-DD HH:mm:ssZ (ex: 2013-12-30 23:00:00-06:00 - where -06:00 is Z meaning the timezone offset from UTC formatted as HH:mm)

Section 7: Test Results, if applicable

Expected test failures

It's expected that any test or example that uses solar resource data from OpenMeteo and was downloaded in the local timezone will change. Minor changes in values could be attributed to the changes to the OpenMeteo wind resource but the major differences will be due to major increases in solar PV generation due to the bugfix.

h2integrate/resource/solar/test/test_pvwatts_integration.py::test_pvwatts_integration[OpenMeteoHistoricalSolarResource-LeapYear]: expect that AEP will increase (AEP did increase)
Examples that use OpenMeteo resource:
- Example 13 uses OpenMeteo wind and solar resource in local timezone: expected that LCOE will lower, LCOH will lower, and production will increase because of increased solar generation.
  - examples/test/test_all_examples.py::test_electrolyzer_demand
- Example 26 uses OpenMeteo wind resource with local timezone. Small changes in wind generation (and downstream effects) are expected for the distributed wind system.
  - examples/test/test_all_examples.py::test_floris_example
- Example 29: no change expected, wind resource downloaded in UTC
- Example 35 complex profit maximization: no change expected, wind resource downloaded in UTC
Example 13: we can see from solar resource integration test mentioned above that this site has a major increase in solar generation because of the bugfix. This would cause all downstream performance and financials to change - which is what happened.
Example 26: distributed wind capacity factor is slightly smaller (was 22.30 and it now 22.297). This has downstream effects of small decrease in total electricity production from the distributed wind technology and minor increases in distributed LCOE and total LCOE. These changes are expected and make sense

…ed conftest.py files

…nto generalize_api

… and to raise a user warning

johnjasa · 2026-07-27T17:17:30Z

Per your other feedback, I think it is fine to have three resource files for the same place to really test this and make sure it's working. Without a comprehensive test, I fear this would be a challenging issue for folks to understand and debug -- timezones are finnicky and resource data is often assumed as truth without too much digging.

There may be a chance in the future to not have the full 8760s for the resource files, and instead save some space by having just a week of data, but I don't know if that's easily possible given the current codebase and don't want any changes for this PR; more just thinking out loud.

… remove some of these

johnjasa

Thanks for this! Let me try to answer your questions:

Should the new method make_time_index() (in OpenMeteoHistoricalSolarResource) be turned into a function that lives in h2integrate/resource/utilities/download_tools.py or a new file in h2integrate/resource/utilities/ since it'll be used by both the OpenMeteo solar resource data model and the OpenMeteo wind resource model?

Yes, made general

The best way to test this fix (in its full completeness) involves having at least 3-5 resource files downloaded for the same site. This would mean like adding 2-3 more resource files to the repo - which I agree isn't ideal but I'm not sure how else to test that the new method is:

compatible with the previous way of downloading/saving resource data using the API
(requires file -28.454864_114.551749_2020_openmeteo_archive_solar_60min_local_tz-init.csv and -28.454864_114.551749_2020_openmeteo_archive_solar_60min_local_tz.csv)
compatible with the new way of downloading/saving resource data using the the API
(requires at least -28.454864_114.551749_2020_openmeteo_archive_solar_60min_local_tz.csv. Could also use file -28.454864_114.551749_2020_openmeteo_archive_solar_60min_utc_tz.csv)
compatible with downloading resource data from the web
(requires at least files open-meteo-28.44S114.55E71m-local_tz_2020.csv and -28.454864_114.551749_2020_openmeteo_archive_solar_60min_local_tz.csv. Could additionally test with files open-meteo-28.44S114.55E71m-utc_2020.csv and -28.454864_114.551749_2020_openmeteo_archive_solar_60min_utc_tz.csv)

Only Do the "compatible with the new way" imo. If you really want backwards compatibility let's do that one too. Do not worry about the web at all; we don't need to support this, especially because openmeteo doesn't need an API key so the barrier to entry for users should be relatively low.

The following files could be removed to remove certain types of testing stuff:

open-meteo-28.44S114.55E71m-utc_2020.csv: if removed would not test that openmeteo web download in UTC is similar to openmeteo API download when also using UTC
open-meteo-28.44S114.55E71m-local_tz_2020.csv: if removed would not test that openmeteo web download in local time is similar to openmeteo API download in local time
-28.454864_114.551749_2020_openmeteo_archive_solar_60min_local_tz-init.csv: if removed, would not test compatibility with resource files downloaded with the openmeteo API in local time prior to bugfix
-28.454864_114.551749_2020_openmeteo_archive_solar_60min_utc_tz.csv: if removed, would not test that results are similar to web download data and/or results using data from the himawari dataset in UTC
-28.454864_114.551749_2020_openmeteo_archive_solar_60min_local_tz.csv: if removed, would not test that results are similar to web download data and/or results using data from the himawari dataset in local time
-28.454864_114.551749_2020_himawari8_v3_60min_utc_tz.csv: if removed, does not allow to compare openmeteo results against another dataset
-28.454864_114.551749_2020_himawari8_v3_60min_local_tz.csv: if removed, does not allow to compare openmeteo results against another dataset
I think the best ones to remove would be some files in UTC since the problems did not exist with UTC downloads in the past. What do y'all think?

I agree that you could remove the UTC ones since it wasn't a concern. I don't think you should remove others unless you have strong feelings. I'm fine having a few resource files on the repo if it means things are tested well and able to be tracked. I also like some of the cross-resource comparisons you have here. That being said I haven't dug into those individual tests much.

johnjasa · 2026-07-27T21:33:41Z

+    def make_time_index(self, data, timezone, lat, lon):
+        t0 = data["time"].iloc[0]
+        t1 = data["time"].iloc[1]


Docstring please

done in h2integrate/resource/utilities/download_tools.py

johnjasa · 2026-07-27T21:34:32Z

+        # DATA downloaded from the web has time in the timezone corresponding to utc_offset_seconds
+


Should we just say "do NOT download resource data from the web" to avoid any concerns?

no - I just had a lot of comments from trying to keep notes of what was going on. I think it'd be good to keep this compatible with web data downloads.

johnjasa · 2026-07-27T21:34:48Z

            else:
                tz = response.UtcOffsetSeconds() / 3600
                header_data.update({"timezone_abbreviation": f"GMT{tz}"})
+                # header_data["data_tz"] = "local_with_utc_offset"


Remove or keep these commented out header data?

johnjasa · 2026-07-27T21:36:00Z

+        ("OpenMeteoHistoricalSolarResource", "solar", -28.454864, 114.551749, 2020, "openmeteo_archive_solar", 8, 2e-2, 557516.006852063),  # noqa: E501
+        ("OpenMeteoHistoricalSolarResource", "solar", -28.454864, 114.551749, 2020, "openmeteo_archive_solar", 0, 2e-2, 557466.6915506845),  # noqa: E501
+        ("Himawari8SolarAPI", "solar", -28.454864, 114.551749, 2020, "himawari8_v3", 8, 1e-2, 576746.92754748),  # noqa: E501
+        ("Himawari8SolarAPI", "solar", -28.454864, 114.551749, 2020, "himawari8_v3", 0, 1e-2, 576741.41897),  # noqa: E501


If I'm reading this right, it shows that multiple different timezones across a given resource source get approximately the same energy output? This is good!

shows that regardless if you use UTC or local timezone or different resource datasets that the AEP is similar (which is good)

elenya-grant and others added 30 commits May 26, 2026 12:22

added draft of generalizing api key tools

7e52698

generalized api methods properly

de7ddc5

moved base environment var functions to core

6c19042

removed old work

ed387c8

Merge remote-tracking branch 'h2i_upstream/develop' into generalize_api

2a70ff1

minor clean-ups to generalized stuff

9a0b62d

removed commented out code

05f04f3

removed usage of globals()

c965982

started adding in test file

14733b4

Merge branch 'develop' into generalize_api

ec139c7

added another test and bugfix in loading environment variable from file

e48aa67

removed commented out code and added todos

fc84498

updated so that setter method is called after _get_env_with_fallback

3a1376d

Merge remote-tracking branch 'origin/generalize_api' into generalize_api

7ab6dae

added doc strings to setter methods in nlr_developer_api_keys.py

6b9a606

Merge remote-tracking branch 'h2i_upstream/develop' into generalize_api

25ec9b6

refactored environment tools with help from Rob

a158d65

minor changes to nlr_developer_api_keys

ea793df

updated tests for env_tools

7d0b3fd

updated conftest.py files

c05f63b

updated doc page

c995469

updated conftest.py files and get_environment_variables

f71c24f

added doc page for debugging environment variable problems

7c2c723

updated doc page so formatting is nice

7e1a67e

typo fix in model_overview.md

e5dc052

Merge remote-tracking branch 'h2i_upstream/develop' into generalize_api

c278ba5

updates to doc pages

1c979c8

added set_vars as input to get_nlr_developer_api_credential and updat…

dabdc6f

…ed conftest.py files

Merge branch 'develop' into generalize_api

33c3875

updated debugging environment variables doc page

9909098

elenya-grant added 13 commits July 22, 2026 11:26

cleaned up some of test_env_tools.py

e5f442f

finished cleanups to test_env_tools.py

0233866

updated conftest.py files

d2185da

Merge branch 'generalize_api' of github.com:elenya-grant/GreenHEART i…

ac6959d

…nto generalize_api

updated changelog

643340b

Added new method to check for duplicate defined environment variables…

05de2e5

… and to raise a user warning

changed executable code cell to python code block in doc page

0997741

trying to understand problem with openmeteo downloads

44612c7

still trying to understand openmeteo downloads

4c34c20

got openmeteo solar data to work properly

6e3f4a9

minor change to solar_resource_base

53fb785

cleaned up openmeteo solar data

84d2a2e

merged in develop

da442e3

elenya-grant added 4 commits July 27, 2026 11:26

removed print statement

22a0159

minor cleanups to openmeteo solar model

e6521a6

added all resource files that could be used for testing - will likely…

93f1008

… remove some of these

Added big integration test of openmeteo solar models

1d44105

elenya-grant requested a review from johnjasa July 27, 2026 21:31

johnjasa reviewed Jul 27, 2026

View reviewed changes

elenya-grant added 6 commits July 27, 2026 16:24

updated openmeteo wind and moved make_time_index to utility file

5b93a58

removed 3 recently added resource files

6f3de8a

updated solar resource test that was failing

c129365

added doc string and comments to new function in download_tools

fea7c68

removed commented out code from openmeteo solar

038ad86

updated subtest values for example 13 and 26

baf9159

elenya-grant requested a review from johnjasa July 28, 2026 17:45

elenya-grant added the ready for review This PR is ready for input from folks label Jul 28, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Bugfix OpenMeteo Resource Local Timezone - #814

Bugfix OpenMeteo Resource Local Timezone#814
elenya-grant wants to merge 53 commits into
NatLabRockies:developfrom
elenya-grant:openmeteo_local_tz_fix

elenya-grant commented Jul 27, 2026 •

edited

Loading

Uh oh!

johnjasa commented Jul 27, 2026

Uh oh!

johnjasa left a comment

Uh oh!

johnjasa Jul 27, 2026

Uh oh!

elenya-grant Jul 28, 2026

Uh oh!

johnjasa Jul 27, 2026

Uh oh!

elenya-grant Jul 27, 2026

Uh oh!

johnjasa Jul 27, 2026

Uh oh!

elenya-grant Jul 28, 2026

Uh oh!

johnjasa Jul 27, 2026

Uh oh!

elenya-grant Jul 28, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		# DATA downloaded from the web has time in the timezone corresponding to utc_offset_seconds

Uh oh!

Conversation

elenya-grant commented Jul 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Bugfix OpenMeteo Resource Local Timezone

Section 1: Type of Contribution

Section 2: Draft PR Checklist

TODO:

Type of Reviewer Feedback Requested (on Draft PR)

Section 3: General PR Checklist

Section 4: Related Issues

Section 5: Impacted Areas of the Software

Section 5.1: New Files

Section 5.2: Modified Files

Section 6: Additional Supporting Information

Section 7: Test Results, if applicable

Uh oh!

johnjasa commented Jul 27, 2026

Uh oh!

johnjasa left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

elenya-grant commented Jul 27, 2026 •

edited

Loading