diff --git a/DataYears.md b/DataYears.md
index ed74f4d..7222339 100644
--- a/DataYears.md
+++ b/DataYears.md
@@ -30,14 +30,16 @@ The following is a summary of input data and years available for use in building
| Coal mining and transport LCI | | x | | | | | | |
| Geothermal LCI1 | | x | | | | | | |
| Hydro LCI1 | | x | | | | | | |
-| Natural gas LCI | | x | | | | | | |
+| Natural gas LCI2 | | x | | | | x | | |
| Nuclear LCI1 | | x | | | | | | |
-| Solar PV LCI2 | | x | | | | x | | |
-| Solar thermal LCI2 | | x | | | | x | | |
-| Wind farm LCI2 | | x | | | | x | | |
+| Solar PV LCI3 | | x | | | | x | | |
+| Solar thermal LCI3 | | x | | | | x | | |
+| Wind farm LCI3 | | x | | | | x | | |
Notes:
1 The inventories for these technologies are based on data from 2016; however, so long as the same plants exist in other years in the EIA 923 data, that inventory can be re-used and applied to different years.
-2 The inventories for these technologies are based on data from 2016 and 2020 (as defined in the RENEWABLE_VINTAGE parameter in globals.py).
+2 This cradle-to-gate inventory is based on data from 2016 or 2020 (as defined in the `ng_model_year` parameter in the model configuration YAML files).
+
+3 The cradle-to-gate inventories for these technologies are based on data from 2016 or 2020 (as defined in the `renewable_vintage` parameter in the model configuration YAML files).
diff --git a/README.md b/README.md
index c1f220c..f9a1ba7 100644
--- a/README.md
+++ b/README.md
@@ -7,57 +7,59 @@ Pre-configured model specifications are included (in the modelconfig directory o
The created LCI models are exported for use in standard life cycle assessment software (i.e., in JSON-LD format using the openLCA v2.0 schema).
This code was created as part of a collaboration between US EPA Office of Research and Development (USEPA) and the National Energy Technology Laboratory (NETL) with contributions from the National Renewable Energy Laboratory (NREL) and support from Eastern Research Group (ERG).
-More information on this effort can be found in the [Framework for an Open-Source Life Cycle Baseline for Electricity Consumption in the United States](https://netl.doe.gov/energy-analysis/details?id=4004).
+More information on this effort can be found in the [Framework for an Open-Source Life Cycle Baseline for Electricity Consumption in the United States](https://www.osti.gov/biblio/1576767).
## Disclaimer
- This United States Environmental Protection Agency (EPA) and National Energy
+ This United States Department of Energy (DOE) and National Energy
Technology Laboratory (NETL) GitHub project code is provided on an "as is"
- basis and the user assumes responsibility for its use. EPA and NETL have
- relinquished control of the information and no longer has responsibility to
+ basis and the user assumes responsibility for its use. DOE and NETL have
+ relinquished control of the information and no longer have responsibility to
protect the integrity, confidentiality, or availability of the information.
Any reference to specific commercial products, processes, or services by
service mark, trademark, manufacturer, or otherwise, does not constitute or
- imply their endorsement, recommendation or favoring by EPA or NETL.
+ imply their endorsement, recommendation or favoring by DOE or NETL.
# Setup
-A Python virtual environment (recommended v3.12) is required with the following packages installed, which were recorded in February 2025.
+A Python environment (recommended v3.12) is required with the following packages installed, which were recorded in March 2026.
+Dependency versions change.
+Note asterisks beside versions of esupy, fedelemflowlist, and StEWI that were used in the latest model development.
_Note that Python 3.14 is not supported (yet)._
-+ `pip install git+https://github.com/USEPA/Federal-LCA-Commons-Elementary-Flow-List#egg=fedelemflowlist`
++ `pip install pandas==2.2.3`
++ `pip install git+https://github.com/FLCAC-admin/fedelemflowlist`
* Successfully installs:
+ appdirs-1.4.4
- + boto3-1.36.18
- + botocore-1.36.18
- + certifi-2025.1.31
- + charset-normalizer-3.4.1
- + esupy-0.4.0
- + fedelemflowlist-1.3.0
- + idna-3.10
- + jmespath-1.0.1
- + numpy-2.2.2
+ + boto3-1.42.37
+ + botocore-1.42.37
+ + certifi-2026.1.4
+ + charset-normalizer-3.4.4
+ + esupy-0.4.2 (*)
+ + fedelemflowlist-1.3.1 (*)
+ + idna-3.11
+ + jmespath-1.1.0
+ + numpy-2.4.1
+ olca-schema-2.4.0
- + pandas-2.2.3
- + pyarrow-19.0.0
+ + pyarrow-23.0.0
+ python-dateutil-2.9.0
- + pytz-2025.1
- + PyYAML-6.0.2
- + requests-2.32.3
- + s3transfer-0.11.2
+ + PyYAML-6.0.3
+ + requests-2.32.5
+ + s3transfer-0.16.0
+ six-1.17.0
- + tzdata-2025.1
- + urllib3-2.3.0
+ + tzdata-2025.3
+ + urllib3-2.6.3
+ `pip install git+https://github.com/USEPA/standardizedinventories#egg=StEWI`
* Successfully installed:
- + StEWI-1.1.4
- + beautifulsoup4-4.12.3
+ + StEWI-1.2.1 (*)
+ et-xmlfile-2.0.0
+ openpyxl-3.1.5
- + soupsieve-2.5
- + xlrd-2.0.1
+ + xlrd-2.0.2
+ `pip install scipy`
* Successfully installs:
- + scipy-1.15.1
+ + scipy-1.17.0
++ `pip install pytz`
+ * Successfully installs:
+ + pytz-2025.2
# API
In the latest version of ElectricityLCI, there is a dependency on three external datasets that require the use of an application programming interface (API) key.
@@ -74,8 +76,8 @@ Request a free API key by registering at the following address.
- https://www.eia.gov/opendata/.
-NETL's coal transportation inventory update is provided through a public URL on [EDX](https://edx.netl.doe.gov), found within the [Life Cycle Analysis](https://edx.netl.doe.gov/group/life-cycle-analysis) group.
-An automated download of the Excel workbook will trigger a request for an EDX API key.
+NETL's upstream inventory data (e.g., coal transportation and natural gas extraction and processing) are provided through public URLs on [EDX](https://edx.netl.doe.gov), found within the [Life Cycle Analysis](https://edx.netl.doe.gov/group/life-cycle-analysis) group.
+An automated download of the Excel workbooks will trigger a request for an EDX API key.
API keys require registration.
- https://edx.netl.doe.gov/user/register
@@ -116,7 +118,7 @@ The `main()` method has four steps:
1. `build_model_config()`
- Prompts the user to select one of the model configurations.
- The 2016 baseline configurations are:
- * ELCI_1
+ * ELCI_1 (_Fed Commons published version_)
* ELCI_2
* ELCI_3
- Version 2 baselines include:
@@ -141,29 +143,31 @@ The `main()` method has four steps:
- Creates the at-user consumption mix processes (based on calculated transmission and distribution losses)
4. `run_post_processes()`
- Cleans the JSON-LD files (e.g., removing zero-value product flows, removing untracked flows, correcting flow categories, and creating consecutive internal exchange IDs)
+ - Generates residual electricity mix processes (based on public sales data provided by O'Shaughnessy et al., 2025).
- Builds the product systems for balancing authority areas, FERC regions, and US.
# Known Issues
-See Appendix A in [this discussion](https://github.com/USEPA/ElectricityLCI/discussions/288) for an overview of unresolved issues in version 2.
+See Appendix A in [this discussion](https://github.com/NETL-RIC/ElectricityLCI/discussions/288) for an overview of unresolved issues in version 2.
# Troubleshooting
-If you receive a TypeError in `write_jsonld`, got unexpected keyword argument 'zw', then it's likely you have an outdated version of [fedelemflowlist](https://github.com/USEPA/fedelemflowlist).
-Please ensure these USEPA packages are up-to-date:
+If you receive a TypeError in `write_jsonld`, got unexpected keyword argument 'zw', then it's likely you have an outdated version of [fedelemflowlist](https://github.com/FLCAC-admin/fedelemflowlist).
+
+Please ensure these additional USEPA packages are up-to-date:
-- esupy
-- fedelemflowlist
-- stewi
+- [esupy](https://github.com/USEPA/esupy)
+- [stewi](https://github.com/USEPA/standardizedinventories)
-If GitHub-hosted packages fail to clone and install, manually downloading the zip files, extracting them, and running the pip install command within package folder also works (see snippet below for example for older version of fedelemflowlist).
+If GitHub-hosted packages fail to clone and install, manually downloading the zip files, extracting them, and running the `pip install .` command within package folder also works (see snippet below for example for older version of fedelemflowlist).
```bash
# Download the correct version of the repo
-wget https://github.com/USEPA/Federal-LCA-Commons-Elementary-Flow-List/archive/refs/tags/v1.1.2.zip
+wget https://github.com/FLCAC-admin/fedelemflowlist/archive/refs/tags/v1.1.2.zip
unzip v1.1.2.zip
-cd cd Federal-LCA-Commons-Elementary-Flow-List-1.1.2/
+cd cd fedelemflowlist-1.1.2/
pip install .
```
+
# Data Store
This package downloads a significant amount of background and inventory data (>2.5 GB) in order to process electricity baselines.
@@ -190,18 +194,18 @@ The application folder for this package is 'electricitylci'.
Inventory data is provided by USEPA's Standardized Emission and Waste Inventories ([StEWI](https://github.com/USEPA/standardizedinventories)) package (via stewicombo).
The inventory data associated with StEWI are stored in the application folders, 'stewi' and 'stewicombo'.
-Flow mapping is handled using USEPA's Federal Elementary Flow List Python package and is saved in the application folder 'fedelemflowlist'.
+Flow mapping is handled using Federal LCA Commons's Federal Elementary Flow List Python package and is saved in the application folder 'fedelemflowlist'.
-The following is an example of the 181 data files downloaded from running the 2020 configuration file (updated in May 2025).
+The following is an example of the data files downloaded from running the 2023 configuration file (updated in March 2026).
EIA Form 860 Excel workbooks have worksheets that are summarized into CSV files for speed.
Note that once downloaded, these files are referenced (and not downloaded again) unless a different year of data is referenced in a configuration file.
users_data_dir/ <- Folder as defined by appdirs
├── electricitylci/ <- Support data (183 MB) and outputs (80 MB)
│ ├── bulk_data/
- │ │ ├── eia_bulk_demand_2020.json (0.5 MB)
- │ │ ├── eia_bulk_id_2020.json (2.6 MB)
- │ │ └── eia_bulk_netgen_2020.json (0.6 MB)
+ │ │ ├── eia_bulk_demand_2023.json (0.5 MB)
+ │ │ ├── eia_bulk_id_2023.json (2.6 MB)
+ │ │ └── eia_bulk_netgen_2023.json (0.6 MB)
│ │
│ ├── cer_rer/
│ │ └── electricity-trade-summary-resume-echanges-commerciaux-electricite.xlsx (100 KB)
@@ -222,56 +226,56 @@ Note that once downloaded, these files are referenced (and not downloaded again)
│ │ ├── Form EIA-860 Insturctions (2016).pdf (0.4 MB)
│ │ └── LayoutY2016.xlsx (0.2 MB)
│ │
- │ ├── eia860_2019/
- │ │ ├── 1___Utility_Y2019.xlsx (0.4 MB)
- │ │ ├── 2___Plant_Y2019.csv (3.2 MB)
- │ │ ├── 2___Plant_Y2019.xlsx (3.1 MB)
- │ │ ├── 3_1_Generator_Y2019.xlsx (8.7 MB)
- │ │ ├── 3_2_Wind_Y2019.xlsx (0.2 MB)
- │ │ ├── 3_3_Solar_Y2019.xlsx (0.8 MB)
- │ │ ├── 3_4_Energy_Storage_Y2019.xlsx (48 KB)
- │ │ ├── 3_5_Multifuel_Y2019.xlsx (0.7 MB)
- │ │ ├── 4___Owner_Y2019.xlsx (0.4 MB)
- │ │ ├── 6_1_EnviroAssoc_Y2019.xlsx (1.2 MB)
- │ │ ├── 6_2_EnviroEquip_Y2019.xlsx (2.9 MB)
- │ │ ├── EIA-860 Form.xlsx (3.1 MB)
- │ │ ├── EIA-860 instruction.pdf (0.6 MB)
- │ │ └── LayoutY2019.xlsx (0.2 MB)
- │ │
│ ├── eia860_2020/
│ │ ├── 1___Utility_Y2020.xlsx (0.4 MB)
│ │ ├── 2___Plant_Y2020.csv (3.4 MB)
│ │ ├── 2___Plant_Y2020.xlsx (3.3 MB)
│ │ ├── 3_1_Generator_Y2020.xlsx (9.0 MB)
- │ │ ├── 3_1_Generator_Y2020_generator_operable.csv (5.2 MB)
│ │ ├── 3_2_Wind_Y2020.xlsx (0.2 MB)
│ │ ├── 3_3_Solar_Y2020.xlsx (1.0 MB)
│ │ ├── 3_4_Energy_Storage_Y2020.xlsx (60 KB)
│ │ ├── 3_5_Multifuel_Y2020.xlsx (0.7 MB)
│ │ ├── 4___Owner_Y2020.xlsx (0.4 MB)
│ │ ├── 6_1_EnviroAssoc_Y2020.xlsx (1.2 MB)
- │ │ ├── 6_1_EnviroAssoc_Y2020_boiler_nox.csv (0.1 MB)
- │ │ ├── 6_1_EnviroAssoc_Y2020_boiler_so2.csv (64 KB)
│ │ ├── 6_2_EnviroEquip_Y2020.xlsx (2.9 MB)
- │ │ ├── 6_2_EnviroEquip_Y2020_boiler_info.csv (0.6 MB)
│ │ ├── EIA-860 Form.xlsx (3.1 MB)
│ │ ├── EIA-860 Instructions.pdf (0.8 MB)
│ │ └── LayoutY2020.xlsx (0.2 MB)
│ │
+ │ ├── eia860_2023/
+ │ │ ├── 1___Utility_Y2023.xlsx (0.5 MB)
+ │ │ ├── 2___Plant_Y2023.csv (4.0 MB)
+ │ │ ├── 2___Plant_Y2023.xlsx (3.9 MB)
+ │ │ ├── 3_1_Generator_Y2023.xlsx (10.0 MB)
+ │ │ ├── 3_1_Generator_Y2023_generator_operable.csv (5.8 MB)
+ │ │ ├── 3_2_Wind_Y2023.xlsx (0.2 MB)
+ │ │ ├── 3_3_Solar_Y2023.xlsx (1.4 MB)
+ │ │ ├── 3_4_Energy_Storage_Y2023.xlsx (0.3 KB)
+ │ │ ├── 3_5_Multifuel_Y2023.xlsx (0.7 MB)
+ │ │ ├── 4___Owner_Y2023.xlsx (0.5 MB)
+ │ │ ├── 6_1_EnviroAssoc_Y2023.xlsx (1.2 MB)
+ │ │ ├── 6_1_EnviroAssoc_Y2023_boiler_nox.csv (0.1 MB)
+ │ │ ├── 6_1_EnviroAssoc_Y2023_boiler_so2.csv (0.1 MB)
+ │ │ ├── 6_2_EnviroEquip_Y2023.xlsx (2.7 MB)
+ │ │ ├── 6_2_EnviroEquip_Y2023_boiler_info.csv (0.6 MB)
+ │ │ ├── EIA-860 Form.xlsx (0.4 MB)
+ │ │ ├── EIA-860 Instructions.pdf (0.7 MB)
+ │ │ └── LayoutY2023.xlsx (0.1 MB)
+ │ │
│ ├── eia930/
│ │ └── EIA930_Reference_Tables.xlsx (43 KB)
│ │
│ ├── energyfutures/
│ │ └── electricity-generation-2023.csv (1.0 MB)
│ │
- │ ├── epacems2020/ <- 48 lower states + D.C. (0.1 MB)
- │ │ ├── epacems2020al.zip (2 KB)
- │ │ ├── epacems2020ar.zip (2 KB)
+ │ ├── epacems2023/ <- 48 lower states + D.C. (0.1 MB)
+ │ │ ├── epacems2023al.zip (2 KB)
+ │ │ ├── epacems2023ar.zip (2 KB)
│ │ ├── ...
- │ │ └── epacems2020wy.zip (1 KB)
+ │ │ └── epacems2023wy.zip (1 KB)
│ │
- │ ├── f7a_2020/
- │ │ └── coalpublic2020.xls (0.2 MB)
+ │ ├── f7a_2023/
+ │ │ └── coalpublic2023.xls (0.1 MB)
│ │
│ ├── f923_2016/
│ │ ├── EIA923_Schedule_8_Annual_Environmental_Information_\
@@ -282,88 +286,94 @@ Note that once downloaded, these files are referenced (and not downloaded again)
│ │ └── EIA923_Schedules_6_7_NU_SourceNDisposition_\
│ │ 2016_Final_Revision.xlsx (0.7 MB)
│ │
- │ ├── f923_2019/
- │ │ ├── EIA923_Schedule_8_Annual_Environmental_Information_\
- │ │ │ 2019_Final_Revision.xlsx (3.1 MB)
- │ │ ├── EIA923_Schedules_2_3_4_5_M_12_2019_Final_Revision.xlsx (19 MB)
- │ │ ├── EIA923_Schedules_2_3_4_5_M_12_2019_Final_\
- │ │ │ Revisionpage_1.csv (8.1 MB)
- │ │ └── EIA923_Schedules_6_7_NU_SourceNDisposition_\
- │ │ 2019_Final_Revision.xlsx (0.9 MB)
- │ │
│ ├── f923_2020/
│ │ ├── EIA923_Schedule_8_Annual_Environmental_Information_\
│ │ │ 2020_Final_Revision.xlsx (3.0 MB)
- │ │ ├── EIA923_Schedule_8_Annual_Environmental_Information_\
- │ │ │ 2020_Final_Revision_page_8c.csv (0.6 MB)
│ │ ├── EIA923_Schedules_2_3_4_5_M_12_2020_Final_Revision.xlsx (18 MB)
│ │ ├── EIA923_Schedules_2_3_4_5_M_12_\
- │ │ │ 2020_Final_Revision_page_1.csv (8.4 MB)
+ │ │ │ 2020_Final_Revisionpage_1.csv (2.0 MB)
+ │ │ └── EIA923_Schedules_6_7_NU_SourceNDisposition_\
+ │ │ 2020_Final_Revision.xlsx (1.0 MB)
+ │ │
+ │ ├── f923_2023/
+ │ │ ├── EIA923_Schedule_8_Annual_Envir_Info_2023_Final.xlsx (3.0 MB)
+ │ │ ├── EIA923_Schedule_8_Annual_Envir_Info_2023_Final_page_8c.csv (0.5 MB)
+ │ │ ├── EIA923_Schedules_2_3_4_5_M_12_2023_Final_Revision.xlsx (19 MB)
│ │ ├── EIA923_Schedules_2_3_4_5_M_12_\
- │ │ │ 2020_Final_Revision_page_3.csv (3.2 MB)
+ │ │ │ 2023_Final_Revision_page_1.csv (9.2 MB)
│ │ ├── EIA923_Schedules_2_3_4_5_M_12_\
- │ │ │ 2020_Final_Revision_page_5_reduced.csv (2.3 MB)
+ │ │ │ 2023_Final_Revision_page_3.csv (3.1 MB)
+ │ │ ├── EIA923_Schedules_2_3_4_5_M_12_\
+ │ │ │ 2023_Final_Revision_page_5_reduced.csv (2.4 MB)
│ │ └── EIA923_Schedules_6_7_NU_SourceNDisposition_\
- │ │ 2020_Final_Revision.xlsx (1.0 MB)
+ │ │ 2023_Final_Revision.xlsx (1.2 MB)
│ │
│ ├── fedcommons/
│ │ ├── dq_sources.json (0.5 KB)
│ │ ├── dq_systems.json (6 KB)
- │ │ ├── flow_properties.json (12 KB)
- │ │ └── unit_groups.json (36 KB)
+ │ │ ├── flow_properties.json (10 KB)
+ │ │ ├── locations.json (99 KB)
+ │ │ └── unit_groups.json (33 KB)
│ │
│ ├── FRS_bridges/
- │ │ └── NEI_2020_RCRAInfo_2019_TRI_2020_eGRID_2020.csv (0.3 MB)
+ │ │ └── NEI_2020_RCRAInfo_2023_TRI_2023_eGRID_2023.csv (0.3 MB)
│ │
│ ├── netl/
- │ │ └── Transportation_Inventories_02262025.xlsx (0.5 MB)
+ │ │ ├──Transportation_Inventories_02262025.xlsx (0.5 MB)
+ │ │ └── 2020_ng/
+ │ │ ├── ng_lci_2020rev1.csv (43 KB)
+ │ │ └── 2020_ng_model/
+ │ │ ├── Appendix_F_2020_Full_Inventory_Results_Midwest_\
+ │ │ │ ProdThruTrans.xlsx (14 MB)
+ │ │ ├── Appendix_F_2020_Full_Inventory_Results_Northeast_\
+ │ │ │ ProdThruTrans.xlsx (11 MB)
+ │ │ ├── Appendix_F_2020_Full_Inventory_Results_Pacific_\
+ │ │ │ ProdThruTrans.xlsx (3 MB)
+ │ │ ├── Appendix_F_2020_Full_Inventory_Results_Rocky_Mountain_\
+ │ │ │ ProdThruTrans.xlsx (3 MB)
+ │ │ ├── Appendix_F_2020_Full_Inventory_Results_Southeast_\
+ │ │ │ ProdThruTrans.xlsx (13 MB)
+ │ │ └── Appendix_F_2020_Full_Inventory_Results_Southwest_\
+ │ │ ProdThruTrans.xlsx (13 MB)
│ │
│ ├── output/ <- ELCI model results
- │ │ ├── BAA_final_trade_2020.csv (69 KB)
+ │ │ ├── BAA_final_trade_2023.csv (67 KB)
│ │ ├── elci.log (0 KB)
- │ │ ├── elci.log.1 (112 MB)
- │ │ ├── ELCI_2020_jsonld_20250528_142931.zip (23 MB)
- │ │ └── ferc_final_trade_2020.csv (18 KB)
+ │ │ ├── elci.log.1 (106 MB)
+ │ │ ├── ELCI_2023_jsonld_20260326_140109.zip (22 MB)
+ │ │ └── ferc_final_trade_2023.csv (17 KB)
│ │
- │ └── t_and_d_2020/ <- 50 states (4.3 MB)
+ │ └── t_and_d_2023/ <- 50 states (4.3 MB)
│ ├── ak.xlsx (84 KB)
│ ├── al.xlsx (89 KB)
│ ├── ...
│ └── wy.xlsx (83 KB)
│
+ ├── facilitymatcher/ <- Facility mapping data (1.1 GB)
+ │ └── FRS Data Files/
+ │ ├── NATIONAL_ENVIRONMENTAL_INTEREST_FILE_v1.2.1_metadata.json
+ │ └── NATIONAL_ENVIRONMENTAL_INTEREST_FILE.CSV
+ │
├── fedelemflowlist/ <- Flow mapping data (14 MB)
- │ └── FedElemFlowListMaster_v1.2.0_e57a542.parquet (14 MB)
+ │ └── FedElemFlowListMaster_v1.3.0_a79846d.parquet (14 MB)
│
- ├── stewi/ <- Inventory data / metadata (45 MB)
- │ ├── facility/
- │ │ ├── eGRID_2020_v1.1.3_6710a0f.parquet (0.7 MB)
- │ │ ├── NEI_2020_v1.1.0_084a311.parquet (6.0 MB)
- │ │ ├── RCRAInfo_2019_v1.0.5_f40a6aa.parquet (1.3 MB)
- │ │ └── TRI_2020_v1.1.0_084a311.parquet (1.8 MB)
- │ │
- │ ├── flowbyfacility/
- │ │ ├── eGRID_2020_v1.1.3_6710a0f.parquet (0.5 MB)
- │ │ ├── NEI_2020_v1.1.0_084a311.parquet (31 MB)
- │ │ ├── RCRAInfo_2019_v1.0.5_f40a6aa.parquet (2.1 MB)
- │ │ └── TRI_2020_v1.1.0_084a311.parquet (1.2 MB)
- │ │
- │ ├── eGRID_2020_v1.1.3_6710a0f_metadata.json (1 KB)
- │ ├── NEI_2020_v1.1.0_084a311_metadata.json (3 KB)
- │ ├── RCRAInfo_2019_v1.0.5_f40a6aa_metadata.json (1 KB)
- │ └── TRI_2020_v1.1.0_084a311_metadata.json (1 KB)
+ ├── stewi/ <- Inventory data / metadata
+ │ ├── eGRID_2023_v1.2.1_3687292_metadata.json (1 KB)
+ │ └── facility/
+ │ └── eGRID_2023_v1.2.1_3687292.parquet (0.7 MB)
│
└── stewicombo/ <- Data / metadata generated by stewicombo
- ├── ELCI_2020_v1.1.2.parquet (2.2 MB)
- └── ELCI_2020_v1.1.2_metadata.json (7 KB)
+ ├── ELCI_2023_v1.2.1_3687292.parquet (1.9 MB)
+ └── ELCI_2023_v1.2.1_3687292_metadata.json (7 KB)
# Developer's Corner
To install the dependencies for this package without installing the package itself, put the following in a text file, called requirements.txt
- fedelemflowlist @ git+https://github.com/USEPA/Federal-LCA-Commons-Elementary-Flow-List#egg=fedelemflowlist
+ fedelemflowlist @ git+https://github.com/FLCAC-Admin/fedelemflowlist
StEWI @ git+https://github.com/USEPA/standardizedinventories#egg=StEWI
scipy>=1.10
-
+ pytz
To checkout a pull request locally for testing:
diff --git a/electricitylci/__init__.py b/electricitylci/__init__.py
index 1dc64ac..747a2dd 100644
--- a/electricitylci/__init__.py
+++ b/electricitylci/__init__.py
@@ -24,9 +24,12 @@
end user.
Last updated:
- 2025-06-09
+ 2026-03-27
"""
__version__ = elci_version
+__all__ = [
+ "basic_setup",
+]
##############################################################################
@@ -101,6 +104,14 @@ def aggregate_gen(gen_df, subregion="BA"):
return aggregate_df
+def basic_setup(lvl='INFO', elci='ELCI_2023'):
+ """Quickly create a stream logger and define model specs."""
+ from electricitylci.utils import get_logger
+ log = get_logger(True, False, str_lv=lvl)
+ config.model_specs = config.build_model_class(elci)
+ log.info("Setup complete")
+
+
def combine_upstream_and_gen_df(gen_df, upstream_df):
"""
Combine the generation and upstream dataframes into a single dataframe.
@@ -588,7 +599,7 @@ def get_facility_level_inventory(to_save=False, sep_by_fac=True):
upstream_df = None
# NOTE: This method triggers an input request for EPA data API key;
- # see https://github.com/USEPA/ElectricityLCI/issues/207
+ # see https://github.com/NETL-RIC/ElectricityLCI/issues/207
# NOTE: This method runs aggregation and emission uncertainty
# calculations.
logging.info("get aggregated generation process")
@@ -788,9 +799,17 @@ def run_net_trade(generation_mix_dict):
return dist_mix_dicts
-def run_post_processes():
+def run_post_processes(json_path=None):
"""Run post processes on JSON-LD file.
+ Parameters
+ ----------
+ json_path : str, optional
+ The file path to a JSON-LD, defaults to none.
+ If none, uses the current model run's JSON-LD file.
+
+ Notes
+ -----
There is no easy way of editing files archived in a zip and no
easy way of remove files from a zip without just creating a new zip,
so that's what's done here.
@@ -799,21 +818,46 @@ def run_post_processes():
addressed in the new _save_to_json (olca_jsonld_writer.py)
2. Remove flows/*.json that are not found in a process exchange.
See cleanup SQL queries in GitHub issue
- https://github.com/USEPA/ElectricityLCI/issues/216
+ https://github.com/NETL-RIC/ElectricityLCI/issues/216
3. Remove zero flows from quantitative reference exchanges
- https://github.com/USEPA/ElectricityLCI/issues/217
+ https://github.com/NETL-RIC/ElectricityLCI/issues/217
4. DO NOT ADD NETL TRACI 2.1 characterization factors
5. Fix labeling of Heat input to elementary flow
- https://github.com/USEPA/ElectricityLCI/issues/293
- 6. Create product systems for select processes (user, consumption mixes)
+ https://github.com/NETL-RIC/ElectricityLCI/issues/293
+ 6. Create product systems for select processes (user, consumption mixes);
+ now includes residual mixes (if configured in model specifications)
+
+ If you have JSON-LD without residual mixes that you want to add residual
+ mixes (and product systems) to, follow these steps:
+
+ 1. Edit the config.yml that was used to create the original JSON-LD by
+ enabling the 'add_residual_mix' and, optionally,
+ 'add_rem_product_systems', and configure the two settings (i.e.,
+ 'rem_weight_method' and 'neg_rem_method').
+ 2. Quick-start ElectricityLCI with your configuration in Python (e.g.,
+ ``basic_setup("ELCI_2023")``).
+ 3. Define your JSON-LD file path
+ (e.g., ``my_json = "~/ELCI_2023_jsonld_20260327_094053-rem.zip"``).
+ 4. Run post-processes on your JSON-LD. The residual processes will be
+ added to the JSON-LD file and will be cleaned;
+ ``run_post_processes(my_json)``.
"""
from electricitylci.olca_jsonld_writer import build_product_systems
from electricitylci.olca_jsonld_writer import clean_json
+ from electricitylci.residual_grid_mix import add_residual_mixes
+
+ if json_path is None:
+ json_path = config.model_specs.namestr
+ else:
+ logging.info("Running post-processes on %s" % json_path)
- clean_json(config.model_specs.namestr)
+ # HOTFIX: clean JSON after residual mix processes [260107; TWD]
+ add_residual_mixes(json_path) # skipped if not configured
+ clean_json(json_path)
build_product_systems(
- file_path=config.model_specs.namestr,
- elci_config=config.model_specs.model_name
+ file_path=json_path,
+ elci_config=config.model_specs.model_name,
+ add_residuals=config.model_specs.add_rem_product_systems
)
diff --git a/electricitylci/bulk_eia_data.py b/electricitylci/bulk_eia_data.py
index f86206e..86dbe08 100644
--- a/electricitylci/bulk_eia_data.py
+++ b/electricitylci/bulk_eia_data.py
@@ -29,7 +29,7 @@
consumption mix for a given region.
Last updated:
- 2024-08-21
+ 2026-03-25
"""
__all__ = [
"ba_exchange_to_df",
@@ -272,7 +272,9 @@ def ba_exchange_to_df(rows, data_type='ba_to_ba'):
)
continue
data = [x[1] for x in row['data']]
+ # Trim 'EBA.' from series ID
from_region = row['series_id'].split('-')[0][4:]
+ # Trim '.ID.D' from series ID
to_region = row['series_id'].split('-')[1][:-5]
tuple_data = [
x for x in zip(
diff --git a/electricitylci/cems_data.py b/electricitylci/cems_data.py
index 08eba0c..9617a8b 100644
--- a/electricitylci/cems_data.py
+++ b/electricitylci/cems_data.py
@@ -58,7 +58,7 @@
CC-BY-4.0
In the current release, the PUDL methods are replaced with EPA's API:
-https://github.com/USEPA/ElectricityLCI/issues/207#issuecomment-1751075194
+https://github.com/NETL-RIC/ElectricityLCI/issues/207#issuecomment-1751075194
Last edited:
2025-08-13
diff --git a/electricitylci/coal_upstream.py b/electricitylci/coal_upstream.py
index 1c5f7e2..951f819 100644
--- a/electricitylci/coal_upstream.py
+++ b/electricitylci/coal_upstream.py
@@ -16,7 +16,10 @@
from ast import literal_eval
from electricitylci.globals import paths
from electricitylci.globals import data_dir
+from electricitylci.globals import GH_URL
from electricitylci.globals import STATE_ABBREV
+from electricitylci.globals import COAL_BASIN_CODES
+from electricitylci.globals import COAL_TYPE_CODES
from electricitylci.eia860_facilities import eia860_balancing_authority
from electricitylci.eia923_generation import eia923_download
from electricitylci.eia923_generation import eia923_generation_and_fuel
@@ -58,13 +61,10 @@
transportation, but still mainly represents 2016.
Last updated:
- 2025-09-05
+ 2026-03-05
"""
__all__ = [
- "basin_codes", # Globals
- "coal_type_codes",
- "mine_type_codes",
- "transport_dict",
+ "transport_dict", # Globals
"eia_7a_download", # Methods
"fix_coal_mining_lci",
"generate_upstream_coal",
@@ -82,38 +82,6 @@
##############################################################################
# GLOBALS
##############################################################################
-basin_codes = {
- 'Central Appalachia': 'CA',
- 'Central Interior': 'CI',
- 'Gulf Lignite': 'GL',
- 'Illinois Basin': 'IB',
- 'Lignite': 'L',
- 'Northern Appalachia': 'NA',
- 'Powder River Basin': 'PRB',
- 'Rocky Mountain': 'RM',
- 'Southern Appalachia': 'SA',
- 'West/Northwest': 'WNW',
- 'Import': 'IMP',
-}
-'''dict : A map between NETL coal basin names and their abbreviations.'''
-
-coal_type_codes = {
- 'BIT': 'B',
- 'LIG': 'L',
- 'SUB': 'S',
- 'WC': 'W',
- 'RC' : 'RC',
-}
-'''dict : Map between EIA coal fuel source codes and NETL coal codes.'''
-
-mine_type_codes = {
- 'Surface': 'S',
- 'Underground': 'U',
- 'Facility': 'F',
- 'Processing': 'P',
-}
-'''dict : A map between coal mine type and their abbreviation.'''
-
transport_dict = {
'Avg Barge Ton*Miles': 'Barge',
'Avg Lake Vessel Ton*Miles': 'Lake Vessel',
@@ -145,8 +113,8 @@ def _clean_columns(df):
def _coal_code(row):
"""Generate coal basin + energy source + mine type string-based code."""
coal_code_str = (
- f'{basin_codes[row["netl_basin"]]}-'
- f'{coal_type_codes[row["energy_source"]]}-'
+ f'{COAL_BASIN_CODES[row["netl_basin"]]}-'
+ f'{COAL_TYPE_CODES[row["energy_source"]]}-'
f'{row["coalmine_type"]}'
).upper()
@@ -198,7 +166,7 @@ def _make_2023_coal_transport_data(year):
coal_map_df = coal_map_df.drop(columns=['coal_source_code', 'heat_input'])
# Read the 2023 coal model transportation data
- # Source: https://github.com/USEPA/ElectricityLCI/discussions/273
+ # Source: https://github.com/NETL-RIC/ElectricityLCI/discussions/273
coal_dir = os.path.join(data_dir, "coal", "2023")
coal_file = os.path.join(coal_dir, "coal_transport_dist.csv")
if not os.path.isfile(coal_file):
@@ -209,7 +177,7 @@ def _make_2023_coal_transport_data(year):
# NOTE: the 2023 coal model uses a slightly different naming scheme
# for WNW coal basin, so let's fix it.
- basin_codes_new = {k:v for k, v in basin_codes.items()}
+ basin_codes_new = {k:v for k, v in COAL_BASIN_CODES.items()}
del basin_codes_new["West/Northwest"]
basin_codes_new["West/North West"] = "WNW"
@@ -433,7 +401,7 @@ def eia_7a_download(year, save_path):
-----
Some years are provided in XML format and require re-saving to work with
the remainder of the code. If you run into troubles with the download,
- see https://github.com/USEPA/ElectricityLCI/issues/230 for a solution.
+ see https://github.com/NETL-RIC/ElectricityLCI/issues/230 for a solution.
"""
eia7a_base_url = 'http://www.eia.gov/coal/data/public/xls/'
name = ('coalpublic{}.xls'.format(year) if year <= 2022 else
@@ -1070,7 +1038,7 @@ def generate_upstream_coal_map(year):
# Map to NETL coal types --- these should match the coal type found in
# the coal source code.
eia_fuel_receipts_good["coal_type"] = eia_fuel_receipts_good[
- "energy_source"].map(coal_type_codes)
+ "energy_source"].map(COAL_TYPE_CODES)
final_df = eia_fuel_receipts_good.groupby(
['plant_id', 'coal_type', 'coal_source_code'],
@@ -1101,7 +1069,7 @@ def generate_upstream_coal_map(year):
"elec_fuel_consumption_mmbtu"
]].copy()
eia_923_gen_fuel["coal_type"] = eia_923_gen_fuel[
- "reported_fuel_type_code"].map(coal_type_codes)
+ "reported_fuel_type_code"].map(COAL_TYPE_CODES)
eia_923_gen_fuel["plant_id"] = eia_923_gen_fuel["plant_id"].astype(int)
# Effectively filters out NaNs in coal type
# NOTE: quantity is in short tons
@@ -1675,7 +1643,7 @@ def read_eia7a_public_coal(year):
file_pattern_match=['coalpublic'],
return_name=False)
# If you're here, then see the following for hotfix:
- # https://github.com/USEPA/ElectricityLCI/issues/230
+ # https://github.com/NETL-RIC/ElectricityLCI/issues/230
try:
eia7a_df = pd.read_excel(
eia7a_path,
@@ -1684,9 +1652,8 @@ def read_eia7a_public_coal(year):
)
except ValueError:
raise ValueError(
- f'Error reading {eia7a_path}. Please see '
- 'https://github.com/USEPA/ElectricityLCI/issues/230 '
- 'for a solution'
+ f'Error reading {eia7a_path}. '
+ f'Please see {GH_URL}/issues/230 for a solution'
)
eia7a_df = _clean_columns(eia7a_df)
diff --git a/electricitylci/combinator.py b/electricitylci/combinator.py
index 40f285b..44efb7d 100644
--- a/electricitylci/combinator.py
+++ b/electricitylci/combinator.py
@@ -29,7 +29,7 @@
Elementary Flow List in order to provide life cycle inventory.
Last edited:
- 2025-02-05
+ 2025-03-09
"""
__all__ = [
"BA_CODES",
@@ -106,7 +106,7 @@ def add_fuel_inputs(gen_df, upstream_df, upstream_dict):
# Fill in with appropriate fields.
# NOTE: 'quantity' is units of Electricity (MWh) for construction and
# nameplate capacity (MW) for coal, heat input (MJ) for petroleum, tons
- # for coal mining, etc.
+ # for coal mining, net generation (MWh) for nuclear upstream, etc.
fuel_df["Compartment"] = "input"
fuel_df["FlowName"] = expand_fuel_df["q_reference_name"]
fuel_df["stage_code"] = upstream_reduced["stage_code"]
@@ -218,6 +218,9 @@ def concat_map_upstream_databases(eia_gen_year, *arg, **kwargs):
This method sets all data vintages with EIA generation year.
+ Renames upstream 'fuel_type' column to 'FuelCategory' and standardizes
+ fuel category names (i.e., to all caps).
+
Examples
--------
>>> import electricitylci.geothermal as geo
@@ -251,9 +254,7 @@ def concat_map_upstream_databases(eia_gen_year, *arg, **kwargs):
df = map_compartment_path(df)
upstream_df_list.append(df)
upstream_df = pd.concat(upstream_df_list, ignore_index=True, sort=False)
- # Hoping to reduce memory usage or at least make more of it available
- # for the later groupby.
- del(arg)
+
# See https://github.com/FLCAC-admin/fedelemflowlist
# The mapping data includes a conversion factor to convert everything into
# standard units (e.g., kg, MJ, m2*a). Note that 'SourceFlowContext' is
@@ -261,7 +262,7 @@ def concat_map_upstream_databases(eia_gen_year, *arg, **kwargs):
logging.info("Creating flow mapping database")
flow_mapping = fedefl.get_flowmapping('eLCI')
- # as hotfix for https://github.com/USEPA/ElectricityLCI/issues/274
+ # as hotfix for https://github.com/NETL-RIC/ElectricityLCI/issues/274
# append full flowlist to the flow mapping file (dropping duplicates)
# to catch any other mappings of flows that use the same name as already
# in the flow list
@@ -282,6 +283,9 @@ def concat_map_upstream_databases(eia_gen_year, *arg, **kwargs):
)
flow_mapping["SourceFlowName"] = flow_mapping["SourceFlowName"].str.lower()
+ # Memory management [26.03.09; TWD]
+ del flowlist
+
logging.info("Preparing upstream df for merge")
upstream_df["FlowName_orig"] = upstream_df["FlowName"]
upstream_df["Compartment_orig"] = upstream_df["Compartment"]
@@ -313,14 +317,14 @@ def concat_map_upstream_databases(eia_gen_year, *arg, **kwargs):
"Unit_orig",
"Source"
]
- # Addings years to the groupby when data is passed to the function
+ # Adding years to the groupby when data is passed to the function
# that includes years.
if "Year" in upstream_df.columns:
groupby_cols=groupby_cols+["Year"]
# Ensure flow amounts are floats
upstream_df["FlowAmount"] = upstream_df["FlowAmount"].astype(float)
- # Refactoring this a bit due to possibility of some columns not being present.
- # Also gets rid of an if statement!
+ # Refactoring this a bit due to possibility of some columns not being
+ # present. Also gets rid of an if statement!
possible_quant_columns = [
"FlowAmount",
"quantity",
@@ -421,7 +425,7 @@ def concat_map_upstream_databases(eia_gen_year, *arg, **kwargs):
# set conversion factor equal to 1.0.
# Note that all other unmapped flows are lost, which is based on
# only keeping emissions that contribute to TRACI impacts;
- # See eLCI.csv here: https://github.com/USEPA/fedelemflowlist
+ # See eLCI.csv here: https://github.com/NETL-RIC/fedelemflowlist
r_flows = (upstream_mapped_df['TargetFlowContext'].isna()) & (
upstream_mapped_df['input']
)
@@ -477,8 +481,9 @@ def concat_map_upstream_databases(eia_gen_year, *arg, **kwargs):
# defined upstream (renewable O&M). I think in almost all cases year
# will already be defined now. But just in case...
if "Year" not in upstream_mapped_df.columns:
- upstream_mapped_df["Year"]=pd.NA
- upstream_mapped_df.loc[upstream_mapped_df["Year"].isna(),"Year"] = eia_gen_year
+ upstream_mapped_df["Year"] = pd.NA
+ upstream_mapped_df.loc[
+ upstream_mapped_df["Year"].isna(), "Year"] = eia_gen_year
final_columns = [
"plant_id",
"FuelCategory",
@@ -564,6 +569,11 @@ def concat_clean_upstream_and_plant(pl_df, up_df):
Returns
-------
pandas.DataFrame
+
+ Notes
+ -----
+ This process is responsible for creating the 'eGRID_ID' column in the
+ upstream data frame.
"""
# Match location data to the upstream inventory
region_cols = [
diff --git a/electricitylci/data/manual_edits.yml b/electricitylci/data/manual_edits.yml
index e7fcc75..a9e3da1 100644
--- a/electricitylci/data/manual_edits.yml
+++ b/electricitylci/data/manual_edits.yml
@@ -99,8 +99,8 @@ generation.py:
- 2016
# One of these has been an issue since 2016 - Issue #190.
# 60822 is an industrial facility, specifically electronics manufacturing
-# that has signficant SF6 emissions. Since these emissions aren't expected to
-# come from solar power generation in any signficant quantities, we'll just
+# that has significant SF6 emissions. Since these emissions aren't expected to
+# come from solar power generation in any significant quantities, we'll just
# remove for now.
entry_8:
edit_type: "remove"
@@ -137,3 +137,26 @@ generation.py:
Source:
- "TRI"
- "NEI"
+# Heavy-handed method to remove Hawaii balancing authority from generation
+# data; see Issue 276 for current decision [26.02.06; TWD]
+ entry_11:
+ edit_type: "remove"
+ data_source: "yaml"
+ filters:
+ Balancing Authority Code:
+ - "HECO"
+# Entries 12 & 13 drop the elevated Chloromethane air emissions from WI hydro
+# power plants; see Issue 324.
+ entry_12:
+ edit_type: "remove"
+ data_source: "yaml"
+ filters:
+ eGRID_ID:
+ - 3971
+ - 3974
+ FlowName:
+ - "Chloromethane"
+ Source:
+ - "NEI"
+ Year:
+ - 2020
diff --git a/electricitylci/data/process_metadata.yml b/electricitylci/data/process_metadata.yml
index 9cc8f04..6ad109c 100644
--- a/electricitylci/data/process_metadata.yml
+++ b/electricitylci/data/process_metadata.yml
@@ -7,13 +7,17 @@ default_boundary_statement: &boundary_statement
'
-default_datacollection_statement: &datacollection_statement
+default_data_collection_statement: &data_collection_statement
- 'DATA COLLECTION
All inventories were calculated using secondary data from publicly available datasets.
'
+default_data_selection_statement: &data_selection_statement
+ 'Where possible data is sourced from freely available life cycle data, such as NETL unit processes or USLCI.
+ If the material was not available there then ecoinvent processes are used.'
+
default_uncertainty_statement: &uncertainty_statement
- 'UNCERTAINTY ESTIMATION
@@ -23,7 +27,7 @@ default_uncertainty_statement: &uncertainty_statement
Process completeness is calculated by first creating a list of expected flows for each fuel type.
U.S. average profiles for the fuels used in electricity generation are developed for expected flows in the following categories: greenhouse gases (GHG), CAPs, toxic and hazardous air pollutants (HAP), toxic water discharges, toxic soil releases, and hazardous waste.
To determine the expected flows in each category the number of flows reported in the underlying data sources NEI, TRI, eGRID, RCRAInfo are summed across eGRID subregions by fuel type.
- Expected flows by fuel type for other inputs and outputs including intermediate inputs, raw energy demands, and water inputs are determined based on flows reported for the relevant electricity generation fuel type in existing LCI datasets such as ANL''s GREET model, NREL''s USLCI, and ecoinvent.
+ Expected flows by fuel type for other inputs and outputs including intermediate inputs, raw energy demands, and water inputs are determined based on flows reported for the relevant electricity generation fuel type in existing LCI datasets such as ANL''s GREET model, NLR''s USLCI, and ecoinvent.
Expected nutrient discharges to water are derived from a sample of electric facilities reporting to USEPA''s Discharge Monitoring Report (DMR).
The actual flows evaluated for each fuel type by eGRID subregion can then be documented and compared to the U.S. average electrical fuel generation profiles to determine the overall process completeness according to the method described in USEPA LCA Data Quality Assessment guidance.
Air emission flow data collection scores can also draw on these fuel-specific profiles by documenting whether an expected flow is being reported by the fuel type in the relevant eGRID subregion.
@@ -32,33 +36,427 @@ default_uncertainty_statement: &uncertainty_statement
no_uncertainty_statement: &no_uncertainty
- 'UNCERTAINTY ESTIMATION
- This process does not contain any uncertainty.'
-
-source_ingwersen_etal: &ingwersen_etal
- Name: 'Ingwersen, et al. ElectricityLCI: A Python Package for U.S. Regionalized Electricity Life Cycle Inventory Data Creation'
-
- TextReference: 'Ingwersen, et al. (in preparation).
- ElectricityLCI: A Python Package for U.S. Regionalized Electricity Life Cycle Inventory Data Creation.
- Manuscript in preparation.'
-
- Version: '2.0.0'
-
- Category: 'In_Preparation'
-
-source_netl_coal_baseline_2024: &netl_coal_baseline
- Name: 'NETL (2023). Cradle-to-Gate Life Cycle Analysis Baseline for United States Coal Mining and Delivery'
+ This process does not contain any uncertainty.
- Url: 'https://doi.org/10.2172/2370100'
-
- Category: 'NETL'
-
- Year: 2023
+ '
- Version: '1.00.00'
+source_bergesen_etal: &bergesen_etal
+ Name:
+ 'Bergesen et al. (2014). Thin-Film Photovoltaic Power Generation Offers Decreasing Greenhouse Gas Emissions and Increasing Environmental Co-benefits in the Long Term'
+ TextReference:
+ 'Bergesen, J. D., Heath, G. A., Gibon, T., & Suh, S. (2014).
+ Thin-film photovoltaic power generation offers decreasing greenhouse gas emissions and increasing environmental co-benefits in the long term.
+ Environmental Science & Technology, 48(16), 9834–9843.
+ https://doi.org/10.1021/es405539z. '
+ Year:
+ 2014
+ Url:
+ 'https://doi.org/10.1021/es405539z'
+ Category:
+ 'NLR'
+ Version:
+ '2.1.0'
+
+source_cer_energy: &cer_energy
+ Name:
+ 'Canadian Energy Regulator (2023).
+ Canada''s Energy Future 2023: Energy Supply and Demand Projections to 2050'
+ TextReference:
+ 'Canadian Energy Regulator (2023).
+ Canada''s Energy Future 2023: Energy Supply and Demand Projections to 2050 - electricity-generation-2023 [Data set].
+ Online: https://www.cer-rec.gc.ca/open/energy/energyfutures2023/electricity-generation-2023.csv'
+ Year:
+ 2023
+ Version:
+ '2.1.0'
+ Url:
+ 'https://open.canada.ca/data/en/dataset/7643c948-d661-4d90-ab91-e9ac732fc737/resource/1b14f347-8876-4646-b594-955ebb7baafd'
+ Category:
+ ''
+
+source_cooney_etal: &cooney_etal
+ Name:
+ 'Cooney et al. (2016).
+ Updating the U.S. Life Cycle GHG Petroleum Baseline to 2014 with Projections to 2040 Using Open-Source Engineering-Based Models'
+ TextReference:
+ 'Cooney, G., Jamieson, M., Marriott, J., Bergerson, J., Brandt, A., & Skone, T. (2016).
+ Updating the U.S. Life Cycle GHG Petroleum Baseline to 2014 with Projections to 2040 Using Open-Source Engineering-Based Models.
+ Environmental Science & Technology.
+ doi: 10.1021/acs.est.6b02819.'
+ Year:
+ 2016
+ Version:
+ '2.1.0'
+ Url:
+ 'http://doi.org/10.1021/acs.est.6b02819'
+ Category:
+ ''
+
+source_de_chalendar: &de_chalendar_2019
+ Name:
+ 'de Chalendar et al. (2019). Tracking emissions in the US electricity system'
+ TextReference:
+ 'de Chalendar, J. A., Taggart, J., & Benson, S. M. (2019).
+ Tracking emissions in the US electricity system.
+ Proceedings of the National Academy of Sciences, 116(51), 25497-25502.
+ doi: 10.1073/pnas.1912950116.'
+ Year:
+ 2020
+ Url:
+ 'https://doi.org/10.1073/pnas.1912950116'
+ Version:
+ '2.1.0'
+ Category:
+ ''
+
+source_eia_u_2018: &eia_uranium_2018
+ Name:
+ 'U.S. Energy Information Administration (2018). 2018 Uranium Marketing Annual Report'
+ TextReference:
+ 'U.S. Energy Information Administration (2018).
+ 2018 Uranium Marketing Annual Report.
+ U.S. Department of Energy, Washington, D.C. Accessed February 24, 2026.
+ https://www.eia.gov/uranium/marketing/archive/umar2018_2.pdf.'
+ Year:
+ 2018
+ Url:
+ 'https://www.eia.gov/uranium/marketing/archive/umar2018_2.pdf'
+ Version:
+ '2.1.0'
+ Category:
+ ''
+
+source_fingersh_etal: &fingersh_etal
+ Name:
+ 'Fingersh et al. (2006). Wind Turbine Design Cost and Scaling Model'
+ TextReference:
+ 'Fingersh, L., Hand, M., and Laxson, A. (2006).
+ Wind Turbine Design Cost and Scaling Model.
+ NREL/TP-500-40566: National Renewable Energy Laboratory, Golden, CO. Accessed February 13, 2026.
+ https://doi.org/10.2172/897434.'
+ Year:
+ 2006
+ Url:
+ 'https://doi.org/10.2172/897434'
+ Version:
+ '2.1.0'
+ Category:
+ 'NLR'
+
+source_frischknecht: &frischknecht_etal
+ Name:
+ 'Frischknecht et al. (2015). Life cycle inventories and life cycle assessment of photovoltaic systems'
+ TextReference:
+ 'Frischknecht, R., Itten, R., Sinha, P., de Wild-Scholten, M., Zhang, J., Fthenakis, V., Kim, H. C., Raugei, M., & Stucki, M. (2015).
+ Life cycle inventories and life cycle assessment of photovoltaic systems.
+ International Energy Agency (IEA) PVPS Task 12, Report T12, 4, 2015.'
+ Year:
+ 2015
+ Url:
+ 'https://iea-pvps.org/wp-content/uploads/2020/01/IEA-PVPS_Task_12_LCI_LCA.pdf'
+ Version:
+ '2.1.0'
+ Category:
+ 'NLR'
+
+source_hottle_ghosh: &hottle_ghosh_2021
+ Name:
+ 'Hottle & Ghosh (2021). Regional electricity consumption mixes using trade data for representative inventories'
+ TextReference:
+ 'Hottle, T., and Ghosh, T. (2021), Regional electricity consumption mixes using trade data for representative inventories.
+ The International Journal of Life Cycle Assessment 26(6), 1211-1222.
+ doi: 10.1007/s11367-021-01876-3. '
+ Year:
+ 2021
+ Url:
+ 'https://doi.org/10.1007/s11367-021-01876-3'
+ Version:
+ '2.1.0'
+ Category:
+ ''
- TextReference: 'NETL (2023).
- Cradle-to-Gate Life Cycle Analysis Baseline for United States Coal Mining and Delivery, U.S. Department of Energy National Energy Technology Laboratory, Pittsburgh, PA.
- Report DOE/NETL-2024/4846.'
+source_ingwersen_etal: &ingwersen_etal
+ Name:
+ 'ElectricityLCI (2025)'
+ TextReference:
+ 'Davis, T. W., Jamieson, M., Ingwersen, W. W., Schivley, G., Young, B., Ghosh, T., Li, J., Sam, S., Young, D. L., Srocka, M., and Hottle, T. A. (2025).
+ ElectricityLCI.
+ https://doi.org/10.18141/2570075. '
+ Year:
+ 2025
+ Url:
+ 'https://doi.org/10.18141/2570075'
+ Version:
+ '2.1.0'
+ Category:
+ ''
+
+source_mason_etal: &mason_etal
+ Name:
+ 'Mason et al. (2006). Energy payback and life-cycle CO2 emissions of the BOS in an optimized 3.5 MW PV installation'
+ Year:
+ 2015
+ TextReference:
+ 'Mason, J. E., Fthenakis, V. M., Hansen, T., & Kim, H. C. (2006).
+ Energy payback and life-cycle CO2 emissions of the BOS in an optimized 3.5 MW PV installation.
+ Progress in Photovoltaics: Research and Applications, 14(2), 179–190.
+ https://doi.org/10.1002/pip.652. '
+ Url:
+ 'https://doi.org/10.1002/pip.652'
+ Version:
+ '2.1.0'
+ Category:
+ 'Brookhaven'
+
+source_nalukowe_etal: &nalukowe_etal
+ Name:
+ 'Nalukowe et al. (2006). Life Cycle Assessment of a Wind Turbine'
+ TextReference:
+ 'Nalukowe, B. B., Liu, J., Damien, W., Lukawski, T. (2006).
+ Life Cycle Assessment of a Wind Turbine.
+ Unpublished. '
+ Year:
+ 2006
+ Url:
+ 'www.infra.kth.se/fms/utbildning/lca/projects%202006/Group%2007%20(Wind%20turbine).pdf'
+ Version:
+ '2.1.0'
+ Category:
+ 'Unpublished'
+
+source_netl_2011_1502: &netl_2011_1502
+ Name:
+ 'Role of Alternative Energy Sources: Nuclear Technology Assessment'
+ TextReference:
+ 'Skone, Timothy J. (2012).
+ Role of Alternative Energy Sources: Nuclear Technology Assessment.
+ NETL/DOE-2011/1502: National Energy Technology Laboratory, Pittsburgh, PA.
+ https://doi.org/10.2172/1515246. '
+ Year:
+ 2012
+ Url:
+ 'https://www.netl.doe.gov/projects/files/FY13_RoleofAlternativeEnergySourcesNuclearTechnologyAssessment_080113.pdf'
+ Version:
+ '2.1.0'
+ Category:
+ 'NETL'
+
+source_netl_2012_1531: &netl_2012_1531
+ Name:
+ 'Skone et al. (2012). Role of Alternative Energy Sources: Geothermal Technology Assessment'
+ TextReference:
+ 'Skone, Timothy J., Littlefield, James, Eckard, Robert, Cooney, Greg, & Marriott, Joe (2012).
+ Role of Alternative Energy Sources: Geothermal Technology Assessment.
+ DOE/NETL-2012/1531: National Energy Technology Laboratory, Pittsburgh, PA.
+ https://doi.org/10.2172/1515239. '
+ Year:
+ 2012
+ Url:
+ 'https://netl.doe.gov/projects/files/FY12_RoleofAlternativeEnergySourcesGeothermalTechnologyAssessment_090112.pdf'
+ Version:
+ '2.1.0'
+ Category:
+ 'NETL'
+
+source_netl_2012_1532: &netl_2012_1532
+ Name:
+ 'Skone et al. (2012). Role of Alternative Energy Sources: Solar Thermal Technology Assessment'
+ TextReference:
+ 'Skone, Timothy J., Littlefield, J., Eckard, R., Cooney, G., Prica, M., Marriott, J. (2012).
+ Role of Alternative Energy Sources: Solar Thermal Technology Assessment.
+ DOE/NETL-2012/1532: National Energy Technology Laboratory, Pittsburgh, PA.
+ https://doi.org/10.2172/1515242. '
+ Year:
+ 2012
+ Url:
+ 'https://netl.doe.gov/energy-analysis/details?id=62dcb3e0-ad31-4711-8c35-cc1b340054ab'
+ Version:
+ '2.1.0'
+ Category:
+ 'NETL'
+
+source_netl_2015_1723: &netl_2015_1723
+ Name:
+ 'Fout et al. (2015). Cost and Performance Baseline for Fossil Energy Plants
+ Volume 1a: Bituminous Coal (PC) and Natural Gas to Electricity Revision
+ 3'
+ TextReference:
+ 'Fout, T., Zoelle, A., Keairns, D., Turner, M., Woods, M., Kuehn, N., Shah, V., Chou, V., Pinkerton, L. (2015).
+ Cost and Performance Baseline for Fossil Energy Plants Volume 1a: Bituminous Coal (PC) and Natural Gas to Electricity Revision 3.
+ DOE/NETL-2015/1723: National Energy Technology Laboratory, Pittsburgh, PA.
+ Available from https://netl.doe.gov/energy-analysis/details?id=E6586E03-86F0-4D03-BC31-632FD0CE2AEC. '
+ Year:
+ 2015
+ Url:
+ 'https://netl.doe.gov/energy-analysis/details?id=E6586E03-86F0-4D03-BC31-632FD0CE2AEC'
+ Version:
+ '2.1.0'
+ Category:
+ 'NETL'
+
+source_netl_2019_2039: &netl_2019_2039
+ Name:
+ 'Littlefield et al. (2019). Life Cycle Analysis of Natural Gas Extraction and Power Generation'
+ TextReference:
+ 'Littlefield, J., Roman-White, S., Augustine, D., Pegallapati, A., Zaimes, G. G., Rai, S., Cooney, G., and Skone, T. J. (2019).
+ Life Cycle Analysis of Natural Gas Extraction and Power Generation.
+ DOE/NETL-2019/2039: National Energy Technology Laboratory, Pittsburgh, PA.
+ https://doi.org/10.2172/1529553'
+ Year:
+ 2019
+ Url:
+ 'https://doi.org/10.2172/1529553'
+ Version:
+ '2.1.0'
+ Category:
+ 'NETL'
+
+source_netl_2025_4933: &netl_2025_4933
+ Name:
+ 'Khutal et al. (2025). Life Cycle Analysis of Natural Gas Extraction and Power Generation: U.S. 2020 Emissions Profile'
+ TextReference:
+ 'Khutal, H., Kirchner-Ortiz, K., Blackhurst, M., Willems, N., Matthews, H. S., Rai, S., Yanai, G., Chivukula, K., Priyadarshini, Jamieson, M. B., and Skone, T. J. (2025).
+ Life Cycle Analysis of Natural Gas Extraction and Power Generation: U.S. 2020 Emissions Profile.
+ DOE/NETL-2025/4933: National Energy Technology Laboratory, Pittsburgh, PA.
+ https://doi.org/10.2172/2572956'
+ Year:
+ 2025
+ Url:
+ https://doi.org/10.2172/2572956
+ Version:
+ '2.1.0'
+ Category:
+ 'NETL'
+
+source_netl_coal_baseline_2024: &netl_2024_4846
+ Name:
+ 'Cutshaw et al. (2023). Cradle-to-Gate Life Cycle Analysis Baseline for United States Coal Mining and Delivery'
+ TextReference:
+ 'A. Cutshaw, D. Carlson, M. Henriksen, M. Krynock, M. Jamieson, and R. James III.
+ Cradle-to-Gate Life Cycle Analysis Baseline for United States Coal Mining and Delivery.
+ DOE/NETL-2024/4846: National Energy Technology Laboratory, Pittsburgh, PA.
+ https://doi.org/10.2172/2370100. '
+ Year:
+ 2023
+ Url:
+ 'https://doi.org/10.2172/2370100'
+ Version:
+ '2.1.0'
+ Category:
+ 'NETL'
+
+source_qu_etal_2017: &qu_etal_2017
+ Name:
+ 'Qu et al. (2017). A Quasi-Input-Output model to improve the estimation of emission factors for purchased electricity from interconnected grids'
+ TextReference:
+ 'Qu, S., Wang, H., Liang, S., Shapiro, A.M., Suh, S., Sheldon, S., Zik, O., Fang, H., and Xu, M. (2017).
+ A Quasi-Input-Output model to improve the estimation of emission factors for purchased electricity from interconnected grids.
+ Applied Energy, 200, 249-259.
+ https://doi.org/10.1016/j.apenergy.2017.05.046. '
+ Year:
+ 2017
+ Url:
+ 'https://doi.org/10.1016/j.apenergy.2017.05.046'
+ Version:
+ '2.1.0'
+ Category:
+ ''
+
+source_qu_etal_2018: &qu_etal_2018
+ Name:
+ 'Qu et al. (2018). Virtual CO2 Emission Flows in the Global Electricity Trade Network'
+ TextReference:
+ 'Qu, S., Li, Y., Yuan, J., and Xu, M. (2018).
+ Virtual CO2 Emission Flows in the Global Electricity Trade Network.
+ Environmental Science & Technology, 52(11), 6666-6675.
+ https://doi.org/10.1021/acs.est.7b05191. '
+ Year:
+ 2018
+ Url:
+ 'https://doi.org/10.1021/acs.est.7b05191'
+ Version:
+ '2.1.0'
+ Category:
+ ''
+
+source_scherer_etal: &scherer_etal
+ Name:
+ "Scherer & Pfister (2016). Hydropower's Biogenic Carbon Footprint"
+ TextReference:
+ "Scherer L, Pfister S (2016).
+ Hydropower's Biogenic Carbon Footprint.
+ PLoS ONE 11(9): e0161947.
+ https://doi.org/10.1371/journal.pone.0161947"
+ Year:
+ 2016
+ Url:
+ 'https://doi.org/10.1371/journal.pone.0161947'
+ Version:
+ '2.1.0'
+
+source_sengupta_etal: &sengupta_etal
+ Name:
+ 'Sengupta et al. (2018). National Solar Radiation Database'
+ Year:
+ 2018
+ TextReference:
+ 'Sengupta, M., Habte, A., Xie, Y., Lopez, A., & Buster, G. (2018).
+ National Solar Radiation Database (NSRDB) [data set].
+ Open Energy Data Initiative (OEDI).
+ National Renewable Energy Laboratory.
+ https://doi.org/10.25984/1810289.'
+ Url:
+ 'https://doi.org/10.25984/1810289'
+ Version:
+ '2.1.0'
+ Category:
+ 'NLR'
+
+source_vestas: &razdan_garrett
+ Name:
+ 'Razdan & Garrett (2015). Life Cycle Assessment of Electricity Production from an onshore V110-2.0 MW Wind Plant'
+ Year:
+ 2015
+ TextReference:
+ 'Razdan, P. and Garrett, P. (2015).
+ Life Cycle Assessment of Electricity Production from an onshore V110-2.0 MW Wind Plant.
+ Version 1.0.
+ Vestas Wind Systems A/S, Hedeager 42, Aarhus N, 8200, Denmark.'
+ Url:
+ 'https://www.vestas.com/content/dam/vestas-com/global/en/sustainability/reports-and-ratings/lcas/LCAV11020MW181215.pdf.coredownload.inline.pdf'
+ Category: 'Vestas'
+
+source_wind_turbines_db: &wind_turbines_db
+ Name:
+ 'Bauer & Matysik (2018). Wind Turbines Database'
+ TextReference:
+ 'Bauer, L. and Matysik, S. (2018).
+ Wind turbines database.
+ wind-turbine-models.
+ Retrieved from https://en.wind-turbine-models.com/turbines.'
+ Year:
+ 2018
+ Url:
+ 'https://en.wind-turbine-models.com/turbines'
+ Version:
+ '2.1.0'
+
+source_yang_etal: &yang_etal
+ Name:
+ 'Yang et al. (2017). USEEIO: A new and transparent United States environmentally-extended input-output model'
+ TextReference:
+ 'Yang, Y., Ingwersen, W. W., Hawkins, T. R., Srocka, M., & Meyer, D. E. (2017).
+ USEEIO: A new and transparent United States environmentally-extended input-output model.
+ Journal of Cleaner Production, 158, 308-318.
+ doi: 10.1016/j.jclepro.2017.04.150'
+ Year:
+ 2017
+ Url:
+ 'https://doi.org/10.1016/j.jclepro.2017.04.150'
+ Version:
+ '2.1.0'
+ Category:
+ ''
# The default documentation is going to serve as the documentation for
# generation processes wherein the inventory is reported emissions from
@@ -67,53 +465,65 @@ source_netl_coal_baseline_2024: &netl_coal_baseline
# or the defaults will need to apply.
default:
- techno_intro: &default_techno_intro
- - 'This is an electricity generation process that represents the stated fuel type (e.g. coal, gas) aggregated to the stated region (e.g. balancing authority area: California ISO). '
-
techno_process: &default_techno_process
- 'The emissions contained in this process are the reported emissions of all of the facilities in the stated region/fuel type per MWh.
The boundary of these emissions are within the boundaries of the electricity generating facilities.
For the mixed plant category, the emissions can be from any facility within the region that does not match the threshold set in the model configuration file.'
Description:
- - *default_techno_intro
- - *default_techno_process
+ - ''
+ - 'Functional unit: 1 MWh of high-voltage electricity'
+ - 'Co-products: N/A'
+ - 'Default co-product approach: N/A'
+ - 'Data source: ElectricityLCI (2025)'
+ - ''
+ - 'This process is a part of NETL''s electricity baseline and is intended to capture the emissions inventory for electricity generation in the US. '
- TechnologyDescription:
+ TechnologyDescription: *default_techno_process
+
+ TemporalDescription: &gen_proc_temp_desc
+ 'The time frame reflects the period for all background data (e.g., eGRID, TRI, NEI, and RCRAInfo), NETL inventory data (e.g., for renewables and power plant water use), and model generation year (e.g., EIA Form 923).'
Start_date: ''
End_date: ''
SamplingProcedure:
- - *boundary_statement
- - *datacollection_statement
- - *uncertainty_statement
-
- IntendedApplication: 'The intended application for these inventories is to provide high-resolution electricity data for the establishment of robust background data used in LCIs of US systems.
+ - *boundary_statement
+ - ''
+ - *data_collection_statement
+ - ''
+ - *uncertainty_statement
+
+ IntendedApplication:
+ 'The intended application for this inventory is to provide high-resolution electricity data for the establishment of robust background data used in LCIs of US systems.
These data were developed as regionally specific, average electricity generation/consumption LCI for use in the application of LCAs where accurate electricity inventories are needed to assess the system of interest and/or to evaluate the sensitivity to changes in supplied electricity.
A full inventory of environmental flows are included enabling use of a full range of LCIA impact categories.'
DatasetOwner: 'NETL'
- DataGenerator: 'NETL and US EPA and ERG and NREL'
+ DataGenerator: 'NETL'
- DataDocumentor: 'NETL and US EPA and ERG and NREL'
+ DataDocumentor: 'NETL and ERG'
AccessUseRestrictions: 'No Restrictions'
- ProjectDescription: 'US EPA contribution to this research was conducted as part of the USEPA Air, Climate, and Energy national research program.
- US DOE National Energy Technology Laboratory contribution to this research was prepared by MESA for the US DOE NETL, under DOE NETL Contract Number DE-FE0025912.
- Support was provided by the National Renewable Energy Laboratory through interagency agreement DW-89-92448301-1, by Eastern Research Group (ERG) through USEPA contract EP-C-16-015, task order 008, and by the National Energy Technology Laboratory (NETL) through contract numbers DE-FE0025912 and 23CFE000075.'
+ ProjectDescription:
+ 'The current project development is funded by the Department of Energy National Energy Technology Laboratory (DOE/NETL) under the SA Contract (89243323CFE000075).
+ Support for this project is provided by Eastern Research Group (ERG).
+ Previous development was supported by the US Environmental Protection Agency (USEPA) under the USEPA Air, Climate, and Energy national research program and by the National Laboratory of the Rockies (NLR).'
- LCIMethod: 'Attributional'
+ LCIMethod:
+ - 'Attributional'
+ - 'Gate-to-Gate process'
ModelingConstants: ''
DataCompleteness: ''
use_egrid:
- DataTreatment: 'Electricity generation processes are primarily based on facility-reported data.
+ DataTreatment:
+ 'Electricity generation processes are primarily based on facility-reported data.
All the facilities defined in the eGRID ''Plants'' list are the electricity generators used as the pool for creating the LCI.
The net generation quantity reported in the ''Plants'' list was used for
generation amount.
@@ -122,20 +532,22 @@ default:
Emissions and wastes from these facilities are gathered from eGRID as well as the USEPA emissions and waste inventories, NEI, TRI, and RCRAInfo.
Alternative generation data was gathered from EIA-923 when the emissions or waste inventories correspond to a non-egrid year.
The eGRID subregion identifier in the ''Plants'' list was used to associate a facility with a region.
- The fuel and energy types used for the aggregation are the eGRID fuel categories -- oil, natural gas, synthetic gas, nuclear fuel, hydroelectric, biomass, wind, solar, geothermal and other fuel -- except for the coal sources, which are further disaggregated by the eGRID reported primary fuel.
+ The fuel and energy types used for the aggregation are the eGRID fuel categories---oil, natural gas, synthetic gas, nuclear fuel, hydroelectric, biomass, wind, solar, geothermal and other fuel---except for the coal sources, which are further disaggregated by the eGRID reported primary fuel.
The coal fuels are bituminous coal, lignite coal, sub-bituminous coal, refined coal, and waste coal.
Each generation process, defined by fuel type and regions, includes a fuel or energy input, linkage to an infrastructure process, and emissions to air, water, and soil, and hazardous waste reported by facilities in that region in the form of emission and waste factors. '
- DataSelection: 'eGRID is a US EPA compiled inventory of electricity generation and selected greenhouse gas and criteria air pollutant emissions for electric power generators that sell electricity to the grid in the United States.
+ DataSelection:
+ 'eGRID is a US EPA compiled inventory of electricity generation and selected greenhouse gas and criteria air pollutant emissions for electric power generators that sell electricity to the grid in the United States.
eGRID is a primary source along with other USEPA maintained national emission and waste inventories for modeling generation.
These inventories include the Toxic Release Inventory (TRI), the National Emissions Inventory (NEI), and Resources Conservation and Recycling Acts'' Biennial Report, which is stored in system called RCRAInfo.
The Energy Information Administration (EIA) compiles generation and fuel use data in EIA-923 report used for generation data for odd years when eGRID generation data are not available.
- EIA also provides heat-content data on fossil and nuclear fuel inputs which are used to relate heat input data to mass quantities of fuel.
+ EIA also provides heat-content data on fossil and nuclear fuel inputs, which are used to relate heat input data to mass quantities of fuel.
The Federal Energy Regulatory Commission (FERC) receives and reports electricity trading data on the grid in its 714 form.
These data are used for modeling electricity trade across regions.'
replace_egrid:
- DataTreatment: 'Electricity generation processes are primarily based on facility-reported data.
+ DataTreatment:
+ 'Electricity generation processes are primarily based on facility-reported data.
All the facilities defined in the eGRID ''Plants'' list are the electricity generators used as the pool for creating the LCI or if the configuration parameter replace_egrid is True then all facilities reported in EIA 923 data are used.
The net generation quantity reported in the ''Plants'' list was used for generation amount.
The annual heat input was used to estimate the energy in the fuel or directly captured energy.
@@ -143,11 +555,12 @@ default:
Emissions and wastes from these facilities are gathered from eGRID as well as the USEPA emissions and waste inventories, NEI, TRI, and RCRAInfo.
Alternative generation data was gathered from EIA-923 when the emissions or waste inventories correspond to a non-egrid year.
The plants are associated with various regions depending on model configuration with region names coming from eGRID and EIA.
- The fuel and energy types used for the aggregation are the eGRID fuel categories -- oil, natural gas, synthetic gas, nuclear fuel, hydroelectric, biomass, wind, solar, solar thermal, geothermal and other fuel.
+ The fuel and energy types used for the aggregation are the eGRID fuel categories: oil, natural gas, synthetic gas, nuclear fuel, hydroelectric, biomass, wind, solar, solar thermal, geothermal and other fuel.
Coal sources can be further disaggregated by the reported primary fuel: bituminous coal, lignite coal, sub-bituminous coal, refined coal, and waste coal.
Each generation process, defined by fuel type and regions, includes a fuel or energy input, linkage to an infrastructure process, and emissions to air, water, and soil, and hazardous waste reported by facilities in that region in the form of emission and waste factors. '
- DataSelection: 'eGRID is a US EPA compiled inventory of electricity generation and selected greenhouse gas and criteria air pollutant emissions for electric power generators that sell electricity to the grid in the United States.
+ DataSelection:
+ 'eGRID is a US EPA compiled inventory of electricity generation and selected greenhouse gas and criteria air pollutant emissions for electric power generators that sell electricity to the grid in the United States.
eGRID is a primary source along with other USEPA maintained national emission and waste inventories for modeling generation.
These inventories include the Toxic Release Inventory (TRI), the National Emissions Inventory (NEI), and Resources Conservation and Recycling Acts'' Biennial Report, which is stored in system called RCRAInfo.
The Energy Information Administration (EIA) compiles generation and fuel use data in EIA-923 report used for generation data for odd years when eGRID generation data are not available.
@@ -155,34 +568,47 @@ default:
The Federal Energy Regulatory Commission (FERC) receives and reports electricity trading data on the grid in its 714 form.
These data are used for modeling electricity trade across regions.'
- DataCollectionPeriod: 'Generation and emissions data should be from the year(s) specified in the model configuration file.'
+ DataCollectionPeriod:
+ 'The data in this process are representative of efforts that happened between 2018 and 2026.'
- Reviewer: 'NETL and USDA and EPA'
+ Reviewer: 'NETL and ERG'
DatasetOtherEvaluation: ''
Sources:
+
Source1:
<<: *ingwersen_etal
nuclear_upstream:
- techno_intro: &nuclear_techno_intro
- - 'This process represents the cradle-to-gate inventory for the production of electricity from US generation II nuclear reactors, including uranium extraction, enrichment, fuel fabrication, fuel transport, power plant construction, and power plant emissions. '
+ Description:
+ - 'This process represents the cradle-to-gate inventory for the upstream production of electricity from US generation II nuclear reactors, including uranium extraction, enrichment, fuel fabrication, and fuel transport.'
+ - ''
+ - 'Functional Unit: 1 MWh of nuclear fuel, through transportation'
+ - 'Co-products: N/A'
+ - 'Default co-product approach: N/A'
+ - 'Data source: Skone (2012)'
+ - ''
+ - 'This process is a part of NETL''s electricity baseline and is intended to capture the upstream life cycle inventory for nuclear powered electricity generation in the US. '
+
+ LCIMethod:
+ - 'Attributional'
+ - 'Cradle-to-Gate process'
- techno_process: &nuclear_techno_process
- - 'Documentation for the original model is available at http://www.netl.doe.gov/energy-analysis/details?id=620.
+ TechnologyDescription:
+ - 'Documentation for the original model is available in Skone (2012).
This inventory represents an update to some portions of the model to represent 2016 operations, namely the retirement of US gaseous diffusion enrichment operations and to a lesser extent an updated mix of nuclear fuel sources.
The inventory is on the basis of 1 MWh generated electricity and, as a result, is the same for all nuclear power plants.'
- Description:
- - *nuclear_techno_intro
- - *nuclear_techno_process
-
- TechnologyDescription:
-
- ModelingConstants: 'http://www.netl.doe.gov/energy-analysis/details?id=620'
+ ModelingConstants:
+ - 'Uranium production comes from US and EU enrichment chains. '
+ - 'Gaseous diffusion technology was removed from the original enrichment process (now 100% centrifuge enrichment). '
+ - 'Uranium enrichment effluent is based on Urenco USA''s LES Claiborne Enrichment Plant in Eunice, NM. '
+ - 'Uranium production effluent is based on Honeywell Plant in Metropolis, IL. '
+ - 'Fuel fabrication emission quantities are modeled after operational reports from fabrication facilities: Framatome, Inc. in Richland, WA; Global Nuclear Fuel-Americas in Wilmington, NC; and Westinghouse Columbia Fuel Fabrication Facility in Columbia, SC. '
+ - 'Uranium mining is proxied by U.S. coal mining processes. '
DatasetOwner: 'NETL'
@@ -190,54 +616,80 @@ nuclear_upstream:
DataDocumentor: 'NETL'
- DataTreatment: 'The full description of data treatment is available at http://www.netl.doe.gov/energy-analysis/details?id=620.
+ DataTreatment:
+ 'The full description of data treatment is available in Skone (2012).
The data represented here is a modification of that life cycle model - US fuel enrichment is now 100 percent centrifugal and some of the background data was updated with newer versions.'
- DataSelection: 'The full description of data selection is available at http://www.netl.doe.gov/energy-analysis/details?id=620'
+ DataSelection:
+ 'Secondary/tertiary material inputs provided via thinkstep data are from service pack 34 - 2016.
+ The full description of data selection is available in Skone (2012).'
- SamplingProcedure:
- - 'BOUNDARY CONDITIONS
-
- The system boundary includes the extraction of uranium from mining operations, fuel enrichment, fuel fabrication, fuel transportation, power plant construction, and power plant operations.
+ TemporalDescription:
+ 'The inventory was generated with a target year of 2016.
+ It includes EIA uranium marketing data from 2018.'
- '
+ Start_date: '1/1/2016'
- - 'DATA COLLECTION
+ End_date: '12/31/2016'
- The full description of data collection is available at http://www.netl.doe.gov/energy-analysis/details?id=620
+ SamplingProcedure:
+ - 'BOUNDARY CONDITIONS'
+ - 'The system boundary includes the extraction of uranium from mining operations, fuel enrichment, fuel fabrication, fuel transportation, power plant construction, and power plant operations.'
+ - ''
+ - 'DATA COLLECTION'
+ - 'The full description of data collection is available in Skone (2012).'
+ - ''
+ - *no_uncertainty
+ - ''
+ - 'FURTHER DOCUMENTATION'
+ - 'See Skone (2012).'
+
+ DataCollectionPeriod:
+ 'This inventory is representative of efforts that happened from 2011 to 2018.'
- '
+ Sources:
- - *no_uncertainty
+ Source1:
+ <<: *netl_2011_1502
- - 'FURTHER DOCUMENTATION
+ Source2:
+ <<: *eia_uranium_2018
- https://www.netl.doe.gov/energy-analysis/details?id=620'
+all:
- DataCollectionPeriod: '2011 to present. Secondary/tertiary material inputs provided via thinkstep data are from service pack 34 - 2016'
+ Description:
+ - ''
+ - 'Functional Unit: 1 MWh of high voltage electricity'
+ - 'Co-products: N/A'
+ - 'Default co-product approach: N/A'
+ - 'Data source: ElectricityLCI (2025)'
+ - ''
+ - 'This process is a part of NETL''s electricity baseline and is intended to capture the emissions inventory for electricity generation imported from Canada. '
- Sources:
- Source1:
- Name: 'Role of Alternative Energy Sources: Nuclear Technology Assessment'
+ TechnologyDescription:
+ 'The fuels used for electricity generation in this Canadian region are based on the public data published by Canadian Energy Futures and the emissions inventory is proxied using U.S. fuel generation, weighted by the Canadian mix amounts.'
- Year: 2012
+ ModelingConstants:
+ 'Uses Canada''s Energy Future 2023: Energy Supply and Demand Projections to 2050 for the given generation year''s fuel mix and uses weighted U.S. emissions inventories as a proxy.'
- TextReference: 'NETL. (2012).
- Role of Alternative Energy Sources: Nuclear Technology Assessment.
- Document Number: NETL/DOE-2011/1502 National Energy Technology Laboratory, Pittsburgh, PA.
- Available from https://www.netl.doe.gov/energy-analysis/details?id=620 '
+ DataTreatment:
+ 'Balancing authority names are from EIA 930 reference tables.
+ Fuel mixes are based on reported values in CER (2023) based on the ''Current Measures'' scenario.
+ Fuel categories are grouped to proxy those found in the U.S. (e.g., biomass / geothermal is labeled as BIOMASS and hydro / wave /tidal is labeled HYDRO; see import_impacts.py in ElectricityLCI for details).
+ The U.S. generation inventory is summarized by fuel category and used as proxy to create the Canadian inventory (where fuel mix percents are used as weights).'
- Version: '1.0.1'
+ Sources:
- Url: 'https://netl.doe.gov/energy-analysis/details?id=620'
+ Source1:
+ <<: *cer_energy
- Category: 'NETL'
+ Source2:
+ <<: *ingwersen_etal
biomass:
techno_intro: &biomass_techno_intro
- - 'This process represents the operational emissions from power plants categorized as biomass by the electricitylci aggregated to the stated region (e.g. balancing authority area: California ISO).
- These include primary fuels of: black liquor (BLQ), wood/wood waste solids (WDS), wood/wood waste liquids (WDL), biogenic municipal solid waste (MSB), landfill gas (LFG), agricultural byproducts (AB), other biomass liquids (OBL), sludge waste (SLW), other biomass gas (OBG), and other biomass solids (OBS).
+ - 'This process includes the primary fuels of: black liquor (BLQ), wood/wood waste solids (WDS), wood/wood waste liquids (WDL), biogenic municipal solid waste (MSB), landfill gas (LFG), agricultural byproducts (AB), other biomass liquids (OBL), sludge waste (SLW), other biomass gas (OBG), and other biomass solids (OBS).
Currently there are no fuel profiles for these fuel types in part because of the wide array and likely unique characteristics of the fuel used in each plant.
Ideally, the fuel inventories would include the uptake of CO2 (biogenic CO2) that would offset the generally higher emissions of these plants, particular for solid fuels. '
@@ -246,10 +698,17 @@ biomass:
The boundary of these emissions are within the boundaries of the electricity generating facilities.'
Description:
- - *biomass_techno_intro
- - *biomass_techno_process
+ - ''
+ - 'Functional Unit: 1 MWh of high voltage electricity'
+ - 'Co-products: N/A'
+ - 'Default co-product approach: N/A'
+ - 'Data source: ElectricityLCI (2025)'
+ - ''
+ - 'This process is a part of NETL''s electricity baseline and is intended to capture the emissions inventory for biomass powered electricity generation in the US. '
TechnologyDescription:
+ - *biomass_techno_intro
+ - *biomass_techno_process
DatasetOwner: 'NETL'
@@ -258,27 +717,28 @@ biomass:
DataDocumentor: 'NETL'
Sources:
+
Source1:
<<: *ingwersen_etal
geothermal:
- techno_intro: &geothermal_techno_intro
- - 'This process represents the cradle-to-gate inventory for electricity production from US geothermal power plants. '
+ Description:
+ - ''
+ - 'Functional Unit: 1 MWh of high voltage electricity'
+ - 'Co-products: N/A'
+ - 'Default co-product approach: N/A'
+ - 'Data source: Skone et al. (2012)'
+ - ''
+ - 'This process is a part of NETL''s electricity baseline and is intended to capture the emissions inventory for geothermal powered electricity generation in the US. '
- techno_process: &geothermal_techno_process
+ TechnologyDescription:
- 'Power plant construction and operations are included in the boundaries.
Power plant emissions include the release of non-condensible gases in the geofluid product at the plant.
Profiles were developed for different states based on the available data.
Geothermal power plants are based on a variety of technologies.
All technologies are captured within the state-level profiles.
- This inventory is an extension of that described in http://www.netl.doe.gov/energy-analysis/details?id=591.'
-
- Description:
- - *geothermal_techno_intro
- - *geothermal_techno_process
-
- TechnologyDescription:
+ This inventory is an extension of that described in Skone et al. (2012).'
DatasetOwner: 'NETL'
@@ -286,527 +746,499 @@ geothermal:
DataDocumentor: 'NETL'
- DataTreatment: 'http://www.netl.doe.gov/energy-analysis/details?id=591'
+ DataTreatment: 'See Skone et al. (2012).'
- DataSelection: 'Non-condensible gas species are from Argonne National Laboratory Geochemical database.
+ DataSelection:
+ 'Non-condensible gas species are from Argonne National Laboratory Geochemical database.
Details on the types of geothermal power plants in each state are provided by EIA 860 and openei.org.
Electricity generation information is provided by EIA Form 923.'
SamplingProcedure:
- - 'BOUNDARY CONDITIONS
-
- The system boundary includes the construction of geothermal wells, facility installation/deinstallation, and power plant operations.
- http://www.netl.doe.gov/energy-analysis/details?id=591
-
- '
-
- - 'DATA COLLECTION
-
- The full description of data collection is available at http://www.netl.doe.gov/energy-analysis/details?id=591
-
- '
-
- - *no_uncertainty
-
- - 'FURTHER DOCUMENTATION
-
- http://www.netl.doe.gov/energy-analysis/details?id=591'
-
- DataCollectionPeriod: '2016-2018'
+ - 'BOUNDARY CONDITIONS'
+ - 'The system boundary includes the construction of geothermal wells, facility installation/deinstallation, and power plant operations.
+ See Skone et al. (2012).'
+ - ''
+ - 'DATA COLLECTION'
+ - 'The full description of data collection is available in Skone et al. (2012).'
+ - ''
+ - *no_uncertainty
+ - ''
+ - 'FURTHER DOCUMENTATION'
+ - 'See Skone et al. (2012).'
+
+ DataCollectionPeriod:
+ 'This inventory is representative of efforts that happened between 2016 and 2018.'
Sources:
Source1:
- Name: 'Role of Alternative Energy Sources: Geothermal Technology Assessment'
-
- Year: 2012
-
- TextReference: 'NETL. (2012).
- Role of Alternative Energy Sources: Geothermal Technology Assessment.
- Document Number: National Energy Technology Laboratory, Pittsburgh, PA.
- Available from https://netl.doe.gov/energy-analysis/details?id=591'
-
- Version: '1.0.1'
-
- Url: 'https://netl.doe.gov/energy-analysis/details?id=591'
-
- Category: 'NETL'
+ <<: *netl_2012_1531
hydro:
- techno_intro: &hydro_techno_intro
- - 'This process represents the operational emissions from hydro-electric power plants aggregated to the stated region (e.g. balancing authority area: California ISO).'
+ Description:
+ - ''
+ - 'Functional Unit: 1 MWh of high voltage electricity.'
+ - 'Co-products: N/A'
+ - 'Default co-product approach: N/A'
+ - 'Data source: Scherer & Pfister (2016)'
+ - ''
+ - 'This process is a part of NETL''s electricity baseline and is intended to capture the emissions inventory for hydroelectric powered electricity generation in the US. '
- techno_process: &hydro_techno_process
+ TechnologyDescription:
- 'The emissions contained in this process are mostly calculated emissions from measured GHG emissions from reservoirs associated with hydro power plants.
Emissions are then allocated according to share of economic use of the reservoir.
The boundary of these emissions are within the boundaries of the electricity generating facilities and the associated reservoir.
See Data Treatment section for more details.'
- Description:
- - *hydro_techno_intro
- - *hydro_techno_process
-
- TechnologyDescription:
-
DatasetOwner: 'NETL'
DataGenerator: 'NETL'
DataDocumentor: 'NETL'
- # Intent is to use the default entries for DataTreatment and DataSelection
- DataTreatment: 'Emissions from hydrolectric (hydro) power plants are representative of power plants located at reservoirs.
+ DataTreatment:
+ 'Emissions from hydroelectric (hydro) power plants are representative of power plants located at reservoirs.
Run-of-river hydro plants are considered to have minimal GHG emissions.
- Emissions from reservoir power plants are taken from Sherer L. and Pfister S. (2016).
+ Emissions from reservoir power plants are taken from Scherer & Pfister (2016).
The emissions from that study are not exhaustive, so a regression model was used to predict methane and CO2 emissions from reservoirs based on the data within that study and U.S. reservoir environment and characteristics taken from the National Oceanic and Atmospheric Administration (NOAA), national inventory of dams (NID) database, EIA-923/60 with the goal of allocating either directly measured emissions or predicted emissions by major economic uses of the reservoir uses, such as electric power, recreation, irrigation, water supply etc.
In some instances there may also be reported data for the power plant in EPA databases.
The sources of those emissions will be annotated by source/year for the flow in the database.'
- DataSelection: 'NOAA: Pan evaporation, temperature minimum/maximum
+ DataSelection:
+ 'NOAA: Pan evaporation, temperature minimum/maximum
National Inventory of Dams Database: reservoir surface area, date of completion, primary purposes, construction volume, construction material, reservoir lat/long
EIA-8923/860: Power plant lat/long, capacity and net generation, cooling
- Sherer and Pfister: training datasets for regression model.'
+ Scherer & Pfister (2016): training datasets for regression model.'
- DataCollectionPeriod: '2018'
+ DataCollectionPeriod:
+ 'This inventory is representative of efforts that happened in 2018.'
Sources:
+
Source1:
<<: *ingwersen_etal
Source2:
- Name: "Hydropower's Biogenic Carbon Footprint"
-
- TextReference: "Scherer L, Pfister S (2016) Hydropower's Biogenic Carbon
- Footprint. PLoS ONE 11(9): e0161947.
- https://doi.org/10.1371/journal.pone.0161947"
-
- Year: '2016'
-
- Url: 'https://doi.org/10.1371/journal.pone.0161947'
+ <<: *scherer_etal
solar:
- techno_intro: &solar_techno_intro
- - 'Solar inventories are provided in two parts - construction and from operations.
- Together they represent the cradle-to-gate inventories for electricity production from US solar photovoltaic power plants. '
-
- techno_process: &solar_techno_process
- - 'Inventories were developed for each solar PV plant in the 2016 or 2020 EIA 860 data, as specified in the model configuration, using a custom life cycle model.
- Boundaries include plant construction and minor operating emissions from fuel combustion, etc.
- The total inventory for the plant construction is allocated over an assumed 30 year life.
- The resulting yearly construction emissions are divided by the generation for the specified year to provide inventories per MWh (e.g., kg CO2/MWh).'
-
Description:
- - *solar_techno_intro
- - *solar_techno_process
+ - ''
+ - 'Functional Unit: 1 MWh of high voltage electricity'
+ - 'Co-products: N/A'
+ - 'Default co-product approach: N/A'
+ - 'Data source: Various (see sources)'
+ - ''
+ - 'This process is a part of NETL''s electricity baseline and is intended to capture the emissions inventory for solar photovoltaic powered electricity generation in the US. '
- TechnologyDescription:
+ TechnologyDescription: &solar_techno_desc
+ 'The technology is representative of U.S. photovoltaic power plants including, but not limited to, crystalline silicon, amorphous silicon, cadmium telluride (CdTe), and copper indium gallium selenide (CIGS) thin film.'
- DatasetOwner: 'IEA, NREL, JE Mason, VM Fthenakis, T Hansen, HC Kim'
+ DatasetOwner: 'IEA, NLR, JE Mason, VM Fthenakis, T Hansen, HC Kim'
DataGenerator: 'NETL'
DataDocumentor: 'NETL'
- DataTreatment: 'Material requirements to construct the solar PV panels and the balance of system are calculated using plant nameplate capacity and local insolation estimates to calculate the required area of solar panels, which are then used to scale inventories provided by literature.
- The total inventory from construction over the plant life is then divided by the assumed 30 year lifetime to provide emissions per year.
- When these emissions are divided by the generation for the specified year in the electricitylci, the inventory is placed on the basis of 1 MWh.
-
- There also inventories for plant operations that are calculated for each plant and are placed on the basis of a MWh before importing to the electricitylci.
- These emissions are mostly from fossil fuel combustion, hydraulic oil, and external electricity use associated with operations.
- There may also be publicly-reported emissions assigned to the plant.'
-
- DataSelection: 'Where possible data is sourced from freely available life cycle data, such as NETL unit processes or USLCI.
- If the material was not available there then ecoinvent processes are used.'
-
- # Entering the year 2024 for DataCollection Period to show
- # when the most recent data was generated (representing 2020).
- DataCollectionPeriod: '2024'
-
- SamplingProcedure:
- - 'BOUNDARY CONDITIONS
-
- The boundaries include power plant construction with all known material inputs (e.g., steel, glass, concrete) and some minor material requirements and fuel combustion for plant operation.
-
- '
-
- - 'DATA COLLECTION
-
- Data for the material inputs to the plant are from: NETL unit processes, US LCI, and ecoinvent.
-
- '
-
- - *no_uncertainty
+ DataSelection: *data_selection_statement
+
+ LCI_2016:
+
+ DataTreatment:
+ - 'Material requirements to construct the solar PV panels and the balance of system are calculated using plant nameplate capacity and local insolation estimates to calculate the required area of solar panels, which are then used to scale inventories provided by literature.
+ The total inventory from construction over the plant life is then divided by the assumed 30 year lifetime to provide emissions per year.
+ These emissions are divided by the 2016 generation to create the inventory on the basis of 1 MWh.'
+ - 'These are combined with plant operations inventories, which are calculated for each plant and placed on the basis of a MWh.
+ Operations emissions are mostly from fossil fuel combustion, hydraulic oil, and external electricity use associated with operations.
+ There may also be publicly-reported emissions assigned to the plant.'
+
+ DataCollectionPeriod:
+ 'This inventory is representative of efforts that happened between 2018 and 2020.'
+
+ SamplingProcedure:
+ - 'BOUNDARY CONDITIONS'
+ - 'The boundaries include power plant construction with all known material inputs (e.g., steel, glass, concrete) and some minor material requirements and fuel combustion for plant operation.'
+ - ''
+ - 'DATA COLLECTION'
+ - 'Inventories were developed for each solar PV plant identified in the 2016 EIA 860 data using a custom life cycle model.
+ Boundaries include plant construction and minor operating emissions from fuel combustion, etc.
+ The total inventory for the plant construction is allocated over an assumed 30 year life.
+ The resulting yearly construction emissions are divided by the 2016 generation to provide inventories per MWh (e.g., kg CO2/MWh).
+ This is combined with the operations inventory, also on a per MWh basis.'
+ - 'Data for the material inputs to the plant are from: NETL unit processes, US LCI, and ecoinvent.'
+ - ''
+ - *no_uncertainty
+
+ LCI_2020:
+
+ DataTreatment:
+ 'The inventories for plant operations are calculated for each plant and are placed on the basis of a MWh.
+ These emissions are mostly from fossil fuel combustion, hydraulic oil, and external electricity use associated with operations.
+ There may also be publicly-reported emissions assigned to the plant.'
+
+ DataCollectionPeriod:
+ 'This inventory is representative of efforts that happened between 2022 and 2024.'
+
+ ModelingConstants:
+ 'The 2022 model updates include 2020 EIA data, updated aluminum functional unit, new aluminum LCI, and corrected panel efficiency from O&M water and oil.
+ This update was further improved in 2023 with a new capacity method.'
+
+ SamplingProcedure:
+ - 'BOUNDARY CONDITIONS'
+ - 'The boundaries include power plant maintenance some minor material requirements and fuel combustion for plant operation.
+ Power plant construction is separate and provided in its own unit process.'
+ - ''
+ - 'DATA COLLECTION'
+ - 'Inventories were developed for each solar PV plant identified in the 2020 EIA 860 data using a custom life cycle model.
+ Boundaries include minor operating emissions from fuel combustion, etc.'
+ - 'Data for the material inputs to the plant are from: NETL unit processes, US LCI, and ecoinvent.'
+ - ''
+ - *no_uncertainty
Sources:
+
Source1:
<<: *ingwersen_etal
Source2:
- Name: 'Thin-Film Photovoltaic Power Generation Offers Decreasing Greenhouse Gas Emissions and Increasing Environmental Co-benefits in the Long Term'
-
- Year: 2014
-
- TextReference: 'Bergesen, J. D., Heath, G. A., Gibon, T., & Suh, S. (2014).
- Thin-film photovoltaic power generation offers decreasing greenhouse gas emissions and increasing environmental co-benefits in the long term.
- Environmental Science & Technology, 48(16), 9834–9843.'
-
- Url: 'https://doi.org/10.1021/es405539z'
-
- Category: 'NREL'
+ <<: *bergesen_etal
Source3:
- Name: 'Life cycle inventories and life cycle assessment of photovoltaic systems'
-
- Year: 2015
-
- TextReference: 'Frischknecht, R., Itten, R., Sinha, P., de Wild-Scholten, M., Zhang, J., Fthenakis, V., Kim, H. C., Raugei, M., & Stucki, M. (2015).
- Life cycle inventories and life cycle assessment of photovoltaic systems.
- International Energy Agency (IEA) PVPS Task 12, Report T12, 4, 2015.'
-
- Url: 'https://iea-pvps.org/wp-content/uploads/2020/01/IEA-PVPS_Task_12_LCI_LCA.pdf'
-
- Category: 'NREL'
+ <<: *frischknecht_etal
Source4:
- Name: 'Energy payback and life-cycle CO2 emissions of the BOS in an optimized 3.5 MW PV installation'
-
- Year: 2015
-
- TextReference: 'Mason, J. E., Fthenakis, V. M., Hansen, T., & Kim, H. C. (2006).
- Energy payback and life-cycle CO2 emissions of the BOS in an optimized 3.5 MW PV installation.
- Progress in Photovoltaics: Research and Applications, 14(2), 179–190.'
-
- Url: 'https://doi.org/10.1002/pip.652'
-
- Category: 'Brookhaven'
+ <<: *mason_etal
Source5:
- Name: 'National Solar Radiation Database (NSRDB)'
-
- Year: 2018
-
- TextReference: 'National Renewable Energy Laboratory. (2018).
- National Solar Radiation Database (NSRDB) [data set].
- Retrieved from https://dx.doi.org/10.25984/1810289.'
-
- Url: 'https://dx.doi.org/10.25984/1810289'
-
- Category: 'NREL'
+ <<: *sengupta_etal
solarthermal:
- techno_intro: &solarthermal_techno_intro
- - 'This process represents the cradle-to-gate inventory for electricity production from US solar thermal power plants. '
-
- techno_process: &solarthermal_techno_process
- - 'Inventories were developed for each solar thermal plant in the 2016 or 2020 EIA 860 data, as specified in the model configuration, using a custom life cycle model.
- Boundaries include plant construction and minor operating emissions from fuel combustion, etc.
- The inventory for the plant is allocated over an assumed 30 year life.
- The aggregation performed in the electricity LCI code will then divide those inventories by the generation in the selected year to provide results on the basis of MWh.'
-
Description:
- - *solarthermal_techno_intro
- - *solarthermal_techno_process
+ - ''
+ - 'Functional Unit: 1 MWh of high voltage electricity'
+ - 'Co-products: N/A'
+ - 'Default co-product approach: N/A'
+ - 'Data source: Skone et al. (2012))'
+ - ''
+ - 'This process is a part of NETL''s electricity baseline and is intended to capture the emissions inventory for solar thermal powered electricity generation in the US. '
- TechnologyDescription:
+ TechnologyDescription: &solar_therm_techno
+ 'The technology is representative of U.S. parabolic trough and power tower solar plants (see Skone et al., 2012).'
- DatasetOwner: 'NREL'
+ DatasetOwner: 'NLR'
DataGenerator: 'NETL'
DataDocumentor: 'NETL'
- DataTreatment: 'Material requirements to construct the solar thermal power plants are based on the plant configuration (tower vs trough), capacity, and local solar insolation values.
- The total inventory from construction and operations of the plant are then divided by the assumed 30 year lifetime to provide emissions per year.
- When these emissions are divided by the generation for the specified year in the electricity baseline code, the inventory is placed on the basis of 1 MWh.'
-
- DataSelection: 'Where possible data is sourced from freely available life cycle data, such as NETL unit processes or USLCI.
- If the material was not available there then ecoinvent processes are used.'
-
- DataCollectionPeriod: '2024'
-
- SamplingProcedure:
- - 'BOUNDARY CONDITIONS
-
- The boundaries include power plant construction with all known material inputs (e.g., steel, glass, concrete) and some minor material requirements and fuel combustion for plant operation.
-
- '
-
- - 'DATA COLLECTION
-
- Data for the material inputs to the plant are from: NETL unit processes, US LCI, and ecoinvent.
-
- '
-
- - *no_uncertainty
+ DataSelection: *data_selection_statement
+
+ LCI_2016:
+
+ DataTreatment:
+ - 'Material requirements to construct the solar thermal power plants are based on the plant configuration (tower vs trough), capacity, and local solar insolation values.
+ The total inventory from construction over the plant life is then divided by the assumed 30 year lifetime to provide emissions per year.
+ These emissions are divided by the 2016 generation to create the inventory on the basis of 1 MWh.'
+ - 'This is combined with plant operations inventories, which are calculated for each plant and are placed on the basis of a MWh.
+ Operations emissions are mostly from fossil fuel combustion, hydraulic oil, and external electricity use associated with operations.
+ There may also be publicly-reported emissions assigned to the plant.'
+
+ DataCollectionPeriod:
+ 'This inventory is representative of efforts that happened between 2018 and 2020.'
+
+ SamplingProcedure:
+ - 'BOUNDARY CONDITIONS'
+ - 'The boundaries include power plant construction with all known material inputs (e.g., steel, glass, concrete) and some minor material requirements and fuel combustion for plant operation.'
+ - ''
+ - 'DATA COLLECTION'
+ - 'Inventories were developed for each solar thermal plant identified in the 2016 EIA Form 860 using a custom life cycle model.
+ Boundaries include plant construction and minor operating emissions from fuel combustion, etc.
+ The inventory for the plant is allocated over an assumed 30 year life.
+ The aggregation performed in the ElectricityLCI divides those inventories by the 2016 generation to provide results on the basis of MWh.'
+ - 'Data for the material inputs to the plant are from NETL unit processes, US LCI, and ecoinvent.'
+ - ''
+ - *no_uncertainty
+
+ LCI_2020:
+
+ DataTreatment:
+ 'The per-MWh operations inventory is multiplied by the 2020 generation and divided by the modeled generation year''s generation to provide an inventory on the basis of 1 MWh at the facility level before regional aggregation.'
+
+ DataCollectionPeriod:
+ 'This inventory is representative of efforts that happened between 2022 and 2024.'
+
+ ModelingConstants:
+ 'The 2022 model updates include 2020 EIA data and corrections/updates to aluminum LCI.'
+
+ SamplingProcedure:
+ - 'BOUNDARY CONDITIONS'
+ - 'The boundary is for plant operations and maintenance and includes the minor operating emissions from fuel combustion, etc.
+ Power plant construction is separate and provided in its own unit process.'
+ - ''
+ - 'DATA COLLECTION'
+ - 'Inventories were developed for each solar thermal plant identified in the 2020 EIA Form 860 using a custom life cycle model.'
+ - 'Data for the material inputs to the plant are from NETL unit processes, US LCI, and ecoinvent.'
+ - ''
+ - *no_uncertainty
Sources:
+
Source1:
<<: *ingwersen_etal
Source2:
- Name: 'Role of Alternative Energy Sources: Solar Thermal Technology Assessment'
-
- Year: 2012
-
- TextReference: 'National Energy Technology Laboratory (2012).
- Role of Alternative Energy Sources: Solar Thermal Technology Assessment.
- Retrieved from https://netl.doe.gov/energy-analysis/details?id=62dcb3e0-ad31-4711-8c35-cc1b340054ab'
-
- Url: 'https://netl.doe.gov/energy-analysis/details?id=62dcb3e0-ad31-4711-8c35-cc1b340054ab'
-
- Category: 'NETL'
+ <<: *netl_2012_1532
wind:
- techno_intro: &wind_techno_intro
- - 'This process represents the cradle-to-gate inventory for electricity production from US wind farms. '
-
- techno_process: &wind_techno_process
- - 'Inventories were developed for each wind farm in the 2016 or 2020 EIA 860 data, as specified in the model configuration, using a custom life cycle model.
- Boundaries include wind turbine construction and minor operating emissions from fuel combustion, maintenance, etc.
- The inventory for the wind farm is allocated over an assumed 30 year life.
- The aggregation performed in the electricity LCI code will then divide those inventories by the generation in the selected year to provide results on the basis of MWh.'
-
Description:
- - *wind_techno_intro
- - *wind_techno_process
-
- TechnologyDescription:
+ - ''
+ - 'Functional Unit: 1 MWh of high voltage electricity'
+ - 'Co-products: N/A'
+ - 'Default co-product approach: N/A'
+ - 'Data source: Various (see sources)'
+ - ''
+ - 'This process is a part of NETL''s electricity baseline and is intended to capture the emissions inventory for electricity generation from wind farms in the US. '
- DatasetOwner: 'NREL and Vestas'
+ DatasetOwner: 'NLR and Vestas'
DataGenerator: 'NETL'
DataDocumentor: 'NETL'
- DataTreatment: 'Material requirements to construct the wind farm are calculated using wind turbine manufacturer and size information to for each turbine and aggregating all of the turbines in the wind farm.
- The total inventory from construction and operations over the plant life are then divided by the assumed 30 year lifetime to provide emissions per year.
- When these emissions are divided by the generation for the specified year in the electricity baseline code, the inventory is placed on the basis of 1 MWh.'
-
- DataSelection: 'Where possible data is sourced from freely available life cycle data, such as NETL unit processes or USLCI.
- If the material was not available there then ecoinvent processes are used.'
-
- DataCollectionPeriod: '2024'
-
- SamplingProcedure:
- - 'BOUNDARY CONDITIONS
-
- The boundaries include power plant construction with all known material inputs (e.g., steel, glass fiber, concrete) and some minor material requirements and fuel combustion for plant operation.
-
- '
-
- - 'DATA COLLECTION
-
- Data for the material inputs to the plant are from: NETL unit processes, US LCI, and ecoinvent.
-
- '
-
- - *no_uncertainty
+ DataSelection: *data_selection_statement
+
+ TechnologyDescription: &wind_techno_desc
+ "This technology represents onshore V110-2.0 MW wind plants (see Razdan & Garrett, 2015)"
+
+ LCI_2016:
+
+ DataTreatment:
+ - 'Material requirements to construct the wind farm are calculated using wind turbine manufacturer and size information to for each turbine and aggregating all of the turbines in the wind farm.
+ The total inventory from construction over the plant life is then divided by the assumed 30 year lifetime to provide emissions per year.
+ These emissions are divided by the 2016 generation to create the inventory on the basis of 1 MWh.'
+ - 'This is combined with plant operations inventories, which are calculated for each plant and are placed on the basis of a MWh.
+ Operations emissions are mostly from fossil fuel combustion, hydraulic oil, and external electricity use associated with operations.
+ There may also be publicly-reported emissions assigned to the plant.'
+
+ DataCollectionPeriod:
+ 'This inventory is representative of efforts that happened in 2018 through 2020.'
+
+ SamplingProcedure:
+ - 'BOUNDARY CONDITIONS'
+ - 'The boundaries include power plant construction with all known material inputs (e.g., steel, glass fiber, concrete) and some minor material requirements and fuel combustion for plant operation.'
+ - ''
+ - 'DATA COLLECTION'
+ - 'Inventories were developed for each wind farm identified in the 2016 EIA Form 860 using a custom life cycle model.
+ Generation data used in the model are based on 2016 EIA Form 923.
+ Boundaries include wind turbine construction and minor operating emissions from fuel combustion, maintenance, etc.
+ The inventory for the wind farm is allocated over an assumed 30 year life.
+ The inventories are multiplied by the 2016 generation and divided by the model-defined generation year''s generation to provide facility-level results on the basis of MWh before regional aggregation.'
+ - 'Data for the material inputs to the plant are from NETL unit processes, US LCI, and ecoinvent.'
+ - ''
+ - *no_uncertainty
+
+ LCI_2020:
+
+ DataTreatment:
+ 'The emissions inventory is multiplied by the 2020 generation and divided by the model-specified generation at the facility level before regional aggregation.'
+
+ DataCollectionPeriod:
+ 'This inventory is representative of efforts that happened between 2022 and 2024.'
+
+ ModelingConstants:
+ 'The 2022 model updates include 2020 EIA data, corrections/updates to aluminum LCI, and corrections to turbine blade count, trunkline, and switchyard calculations.'
+
+ SamplingProcedure:
+ - 'BOUNDARY CONDITIONS'
+ - 'This process represents the operations and maintenance of U.S. wind farms.
+ Power plant construction is separate and is provided in its own unit process.'
+ - ''
+ - 'DATA COLLECTION'
+ - 'Inventories were developed for each wind farm identified in the 2020 EIA Form 860 using a custom life cycle model.
+ Generation data used in the model are based on 2020 EIA Form 923.
+ Boundaries include wind turbine operating emissions from fuel combustion, maintenance, etc.
+ The inventory for the wind farm is allocated over an assumed 30 year life.
+ The aggregation performed in the ElectricityLCI divides those inventories by the 2020 generation to provide results on the basis of MWh.'
+ - 'Data for the material inputs to the plant are from NETL unit processes, US LCI, and ecoinvent.'
+ - ''
+ - *no_uncertainty
Sources:
+
Source1:
<<: *ingwersen_etal
Source2:
- Name: 'Life Cycle Assessment of Electricity Production from an onshore V110-2.0 MW Wind Plant'
-
- Year: 2015
-
- TextReference: 'Vestas (2015). Life Cycle Assessment of Electricity Production from an onshore V110-2.0 MW Wind Plant
- 18th December 2015, Version 1.0. Vestas Wind Systems A/S, Hedeager 42, Aarhus N, 8200, Denmark.'
-
- Url: 'https://www.vestas.com/content/dam/vestas-com/global/en/sustainability/reports-and-ratings/lcas/LCAV11020MW181215.pdf.coredownload.inline.pdf'
-
- Category: 'Vestas'
+ <<: *razdan_garrett
Source3:
- Name: 'Wind Turbine Design Cost and Scaling Model.'
-
- Year: 2006
-
- Category: 'NREL'
-
- TextReference: 'Fingersh et al., 2006. Wind Turbine Design Cost and Scaling Model,
- Technical Report NREL/TP-500-40566 (Accessed June 15, 2010)'
-
- Url: 'https://docs.nrel.gov/docs/fy07osti/40566.pdf'
+ <<: *fingersh_etal
Source4:
- Name: 'Life Cycle Assessment of a Wind Turbine'
-
- Year: 2006
-
- TextReference: 'Barbara Batumbya Nalukowe, Jianguo Liu, Wiedmer Damien, Tomasz Lukawski, 2006. Life Cycle Assessment of a Wind Turbine. May 22, 2006'
+ <<: *nalukowe_etal
Source5:
- Name: 'Wind Turbines Database'
-
- Year: 2018
-
- Url: 'https://en.wind-turbine-models.com/turbines'
-
- TextReference: '(2018) Wind Turbines Database. Retrieved from https://en.wind-turbine-models.com/turbines'
+ <<: *wind_turbines_db
consumption_mix:
- techno_intro: &cons_mix_techno_intro
- - 'This process provides the electricity inputs from the various balancing authority areas or eGRID subregions that makes up the consumption mix for the stated region.'
-
- techno_process: &cons_mix_techno_process
+ ModelingConstants:
- 'There should exist inputs for all generation mixes that contribute to the consumption mix of the stated region, but in some cases there may not be an available provider.
This likely means that there was no available data for the generation mix for that region.'
- Description:
- - *cons_mix_techno_intro
- - *cons_mix_techno_process
-
TechnologyDescription:
+ TemporalDescription:
+ 'The time frame reflects the modeled trading year.'
+
use_egrid:
- DataTreatment: 'FERC Form 714 and international trading data in conjunction with the updated generation mixes.
+
+ Description:
+ - ''
+ - 'Functional Unit: 1 MWh of high voltage electricity'
+ - 'Co-products: N/A'
+ - 'Default co-product approach: N/A'
+ - 'Data source: Hottle & Ghosh (2021)'
+ - ''
+ - 'This process is a part of NETL''s electricity baseline and is intended to capture the electricity grid consumption mix for the given region in the US. '
+
+ DataTreatment:
+ 'FERC Form 714 and international trading data in conjunction with the updated generation mixes.
There are two different approaches to estimating generation mixes described here: 1) Net Trading and 2) Gross Trading.
Both approaches are based on a trading methodology intended to replicate real-world electricity trading between eGRID subregions within the NERC regions.
The Net Trading approach assumes that the eGRID regions first provide electricity for their own demand and provide any surpluses (generation>demand) to the shared pool for their respective NERC region whereas the Gross Trading approach contributes all trades out to the shared NERC pool then the eGRID regions draw from that shared pool to meet their annual demand. '
- DataCollectionPeriod: 'The FERC Form 714 data used to model the trading for the consumption inventories is based on 2014 reported trades.'
+ DataCollectionPeriod:
+ 'The FERC Form 714 data used to model the trading for the consumption inventories is based on 2014 reported trades.'
DataSelection: 'FERC Form 714'
Sources:
- Source1:
- Name: 'Hottle & Ghosh (2021) Regional electricity consumption mixes using trade data for representative inventories'
-
- Url: 'https://doi.org/10.1007/s11367-021-01876-3'
-
- TextReference: 'Hottle, T., and Ghosh, T. (2021), Regional electricity consumption mixes using trade data for representative inventories.
- The International Journal of Life Cycle Assessment 26(6), 1211-1222.
- doi: 10.1007/s11367-021-01876-3.'
-
- Year: 2021
- Version: '1.0.2'
-
- Category: ''
+ Source1:
+ <<: *hottle_ghosh_2021
replace_egrid:
- DataTreatment: 'US Energy Information Administration (EIA) Form EIA-930 electricity trading data is used to inform an input/output model of electricity trading.
+
+ Description:
+ - ''
+ - 'Functional Unit: 1 MWh of high voltage electricity'
+ - 'Co-products: N/A'
+ - 'Default co-product approach: N/A'
+ - 'Data source: Various (see sources)'
+ - ''
+ - 'This process is a part of NETL''s electricity baseline and is intended to capture the electricity grid consumption mix for the given region in the US. '
+
+ DataTreatment:
+ 'US Energy Information Administration (EIA) Form EIA-930 electricity trading data is used to inform an input/output model of electricity trading.
This model uses generation and trading data at the balancing authority area to calculate the net flow of electricity across all balancing authority areas.
As a result, the consumption mixes for all levels of aggregation (BA, FERC, and US) are linked to balancing authority area generation mixes. '
- DataCollectionPeriod: 'Form EIA-930 data is available from the last half of 2015 through today via the bulk data download that the module uses.
+ DataCollectionPeriod:
+ 'Form EIA-930 data is available from the last half of 2015 through today via the bulk data download that the module uses.
All years between are available for generating consumption mixes.'
DataSelection: 'EIA Form 930 and EIA 923'
Sources:
- Source1:
- Name: 'de Chalendar et al. (2019). Tracking emissions in the US electricity system'
-
- Url: 'https://doi.org/10.1073/pnas.1912950116'
-
- TextReference: 'de Chalendar, J. A., Taggart, J., & Benson, S. M. (2019).
- Tracking emissions in the US electricity system.
- Proceedings of the National Academy of Sciences, 116(51), 25497-25502.
- doi: 10.1073/pnas.1912950116.'
- Year: 2020
-
- Version: '1.0.2'
-
- Category: ''
+ Source1:
+ <<: *de_chalendar_2019
Source2:
- Name: 'Qu et al. (2017). A Quasi-Input-Output model to improve the estimation of emission factors for purchased electricity from interconnected grids.'
-
- Url: 'https://doi.org/10.1016/j.apenergy.2017.05.046'
-
- TextReference: 'Qu, S., Wang, H., Liang, S., Shapiro, A.M., Suh, S., Sheldon, S., Zik, O., Fang, H., and Xu, M. (2017).
- A Quasi-Input-Output model to improve the estimation of emission factors for purchased electricity from interconnected grids.
- Applied Energy, 200, 249-259.'
-
- Year: 2017
-
- Version: '1.0.2'
-
- Category: ''
+ <<: *qu_etal_2017
Source3:
+ <<: *qu_etal_2018
- Name: 'Qu et al. (2018). Virtual CO2 Emission Flows in the Global Electricity Trade Network'
-
- Url: 'https://doi.org/10.1021/acs.est.7b05191'
-
- TextReference: 'Qu, S., Li, Y., Yuan, J., and Xu, M. (2018).
- Virtual CO2 Emission Flows in the Global Electricity Trade Network.
- Environmental Science & Technology, 52(11), 6666-6675.'
-
- Year: 2018
+ Source4:
+ <<: *ingwersen_etal
- Version: '1.0.2'
+distribution_mix:
- Category: ''
+ Description:
+ - ''
+ - 'Functional Unit: 1 MWh of 120 V electricity'
+ - 'Co-products: N/A'
+ - 'Default co-product approach: N/A'
+ - 'Data source: ElectricityLCI (2025)'
+ - ''
+ - 'This process is a part of NETL''s electricity baseline and is intended to capture the electricity at the end user for the given region in the US. '
- Source4:
- <<: *ingwersen_etal
+ TemporalDescription:
+ 'The time frame reflects the modeled generation year, which is used for calculating the transmission and distribution losses.'
generation_mix:
- techno_intro: &gen_mix_techno_intro
- - 'This process provides the electricity inputs from the various generation technologies within the region that make up the regions electricity generation mix.'
-
- techno_process: &gen_mix_techno_process
+ Description:
+ - ''
+ - 'Functional Unit: 1 MWh of high voltage electricity'
+ - 'Co-products: N/A'
+ - 'Default co-product approach: N/A'
+ - 'Data source: ElectricityLCI (2025)'
+ - ''
+ - 'This process is a part of NETL''s electricity baseline and is intended to capture the electricity grid generation mix for the given region in the US. '
+
+ ModelingConstants:
- 'There should exist inputs for all generation technologies that contribute to the generation mix of the stated region, but in some cases there may not be an available provider.
This likely means that there was no available data for the technology type in that region.'
- Description:
- - *gen_mix_techno_intro
- - *gen_mix_techno_process
-
- TechnologyDescription:
-
use_egrid:
- DataTreatment: 'FERC Form 714 and international trading data in conjunction with the updated generation mixes.
+ DataTreatment:
+ 'FERC Form 714 and international trading data in conjunction with the updated generation mixes.
There are two different approaches to estimating generation mixes described here: 1) Net Trading and 2) Gross Trading.
Both approaches are based on a trading methodology intended to replicate real-world electricity trading between eGRID subregions within the NERC regions.
The Net Trading approach assumes that the eGRID regions first provide electricity for their own demand and provide any surpluses (generation>demand) to the shared pool for their respective NERC region whereas the Gross Trading approach contributes all trades out to the shared NERC pool then the eGRID regions draw from that shared pool to meet their annual demand. '
- DataCollectionPeriod: 'The FERC Form 714 data used to model the trading for the consumption inventories is based on 2014 reported trades.'
+ DataCollectionPeriod:
+ 'The FERC Form 714 data used to model the trading for the consumption inventories is based on 2014 reported trades.'
DataSelection: 'FERC Form 714'
replace_egrid:
- DataTreatment: 'US Energy Information Administration (EIA) Form EIA-930 electricity trading data is used to inform an input/output model of electricity trading.
+ TechnologyDescription:
+ 'Uses EIA 923 generation data to calculate the region''s primary fuel mix.'
+
+ DataTreatment:
+ 'US Energy Information Administration (EIA) Form EIA-930 electricity trading data is used to inform an input/output model of electricity trading.
This model uses generation and trading data at the balancing authority area to calculate the net flow of electricity across all balancing authority areas.
As a result, the consumption mixes for all levels of aggregation (BA, FERC, and US) are linked to balancing authority area generation mixes. '
- DataCollectionPeriod: 'Form EIA-930 data is available from the last half of 2015 through today via the bulk data download that the module uses.
+ DataCollectionPeriod:
+ 'Form EIA-930 data is available from the last half of 2015 through today via the bulk data download that the module uses.
All years between are available for generating consumption mixes.'
DataSelection: 'EIA Form 930 and EIA 923'
coal_upstream:
- techno_intro: &coal_upstream_techno_intro
- - 'The cradle-to-gate inventory for production of coal aggregated to basin, mine type, and coal type groups. '
-
- techno_process: &coal_upstream_techno_process
- - 'For coal extraction there are two major processes that form the basis of the coal life cycle model - underground and surface coal mining.
- These are connected to auxiliary processes that provide inventories from things like coal mine methane emissions, water use, water emissions, etc.
- All processes use parameters that allow some differentiation based on region or coal type.
- Details on the coal modeling can be found in the NETL Coal Baseline report published in 2023: https://doi.org/10.2172/2370100.'
-
Description:
- - *coal_upstream_techno_intro
- - *coal_upstream_techno_process
+ - 'This process represents the cradle-to-gate inventory for the production of coal aggregated to basin, mine type, and coal type groups.'
+ - ''
+ - 'Functional unit: 1 short ton of coal, processes, at mine'
+ - 'Co-products: N/A'
+ - 'Default co-product approach: N/A'
+ - 'Data source: Cutshaw et al. (2023)'
+ - ''
+ - 'This process is a part of NETL''s electricity baseline and is intended to capture the emissions inventory US coal production. '
+
+ LCIMethod:
+ - 'Attributional'
+ - 'Cradle-to-Gate process'
TechnologyDescription:
+ 'Technologies represent U.S. surface and underground mining for bituminous, sub-bituminous, and lignite coal.'
+
+ TemporalDescription:
+ 'The time frame reflects the data representativeness within the coal baseline (i.e., 2016).'
Start_date: '1/1/2016'
@@ -818,145 +1250,167 @@ coal_upstream:
DataDocumentor: 'NETL'
- ModelingConstants: 'The model scope is intended to include fossil fuel resource production through delivery to the end user via the vehicle or pipeline transport in the United States.
+ ModelingConstants:
+ 'All processes use parameters that allow some differentiation based on region or coal type.
+ Details on the coal modeling can be found in the NETL Coal Baseline report published in 2023: https://doi.org/10.2172/2370100.
+
+ The model scope is intended to include fossil fuel resource production through delivery to the end user via the vehicle or pipeline transport in the United States.
This scope is broken down into stages for production, mixes of produced fuel, mixes of consumed fuel, and distribution to the final user.
Inputs to fuel production including fuels, capital infrastructure and other purchased goods or extracted resources, and any chemical release and hazardous wastes generated.
One megajoule (MJ) is the declared unit used in processes within upstream life cycle stages. '
- DataSelection: 'USGS data was crucial in contributing data on coal production and blending through their guidelines on coal resource estimation and classification.
+ DataSelection:
+ 'USGS data was crucial in contributing data on coal production and blending through their guidelines on coal resource estimation and classification.
AP-42 is a compilation of air emissions factors for over 200 air pollution source categories across various industries.
AP-42 and EIA also provide heat-content data on fossil and nuclear fuel inputs which are used to relate heat input data to mass quantities of fuel and air emissions factors for fuels.
These data are used for modeling fossil fuel production across regions.
- Further information can be found in the coal baseline report published in 2023: https://doi.org/10.2172/2370100.'
+ Further information can be found in the coal baseline report published in Cutshaw et al. (2023).'
- DataTreatment: 'The coal model divides annual reported and calculated emissions by either the production rate at a mine or the throughput at a facility to provide results on the basis of 1 kg coal.'
+ DataTreatment:
+ 'The coal model divides annual reported and calculated emissions by either the production rate at a mine or the throughput at a facility to provide results on the basis of 1 kg coal.'
SamplingProcedure:
- - 'BOUNDARY CONDITIONS
-
- The coal life cycle model includes emissions from coal mining itself, the manufacturing of coal mining equipment, site construction and commissioning, and coal cleaning as applicable.
-
- '
-
- - 'DATA COLLECTION
-
- All inventories were calculated using secondary data from publicly available datasets.
-
- '
-
- - *no_uncertainty
+ - 'BOUNDARY CONDITIONS'
+ - 'The coal life cycle model includes emissions from coal mining itself, the manufacturing of coal mining equipment, site construction and commissioning, and coal cleaning as applicable.'
+ - ''
+ - 'DATA COLLECTION'
+ - 'For coal extraction there are two major processes that form the basis of the coal life cycle model - underground and surface coal mining.
+ These are connected to auxiliary processes that provide inventories from things like coal mine methane emissions, water use, water emissions, etc.
+ Data from 2016 were used to define the amount, type, and origin of the coal used by each power plant, as well as year-specific parameters such as coal mine methane emissions.
+ All inventories were calculated using secondary data from publicly available datasets.'
+ - ''
+ - *no_uncertainty
- DataCollectionPeriod: 'Data from 2016 were used to define the amount, type, and origin of the coal used by each power plant, as well as year-specific parameters such as coal mine methane emissions.'
+ DataCollectionPeriod:
+ 'This inventory represents efforts conducted between 2018 and 2020.'
Sources:
Source1:
-
- <<: *netl_coal_baseline
+ <<: *netl_2024_4846
gas_upstream:
- techno_intro: &gas_upstream_techno_intro
- - 'The cradle-to-gate inventory for production of gas aggregated to basin or region, depending on the year selected in the model configuration. '
+ NGI_2016:
- techno_process: &gas_upstream_techno_process
- - 'The NETL natural gas life cycle model includes parameters to generate inventories for natural gas extraction based on region or basin and geology which determines the gas extraction type (e.g., Appalachian Shale using hydraulic fracturing).
- 2016 or 2020 natural gas production then informs the amount of each type of technology/region that form the mix in the regions, depending on the year selected in the model configuration.
- These can be further aggregated to a US average.
- More details are in the natural gas upstream report at the following links.
- Link for 2016: https://www.netl.doe.gov/energy-analysis/details?id=4f43cb3f-c0d7-482e-bf01-39995a7c7497
- Link for 2020: https://www.netl.doe.gov/energy-analysis/details?id=546d4009-c43b-43f5-bcc9-64d5e63fc8d5
- '
+ Description:
+ - 'This process represents the cradle-to-gate inventory for the production of gas aggregated to one of 14 production technobasins (i.e., Anadarko, Appalachian, Arkla, Arkoma, East Texas, Fort Worth, Green River, Gulf, Permian, Piceance, San Juan, South Oklahoma, Strawn, and Unita).'
+ - ''
+ - 'Functional unit: 1 MJ of natural gas, through transmission'
+ - 'Co-products: N/A'
+ - 'Default co-product approach: N/A'
+ - 'Data source: Littlefield et al. (2019)'
+ - ''
+ - 'Intended for use as upstream for natural gas power plants.'
- Description:
- - *gas_upstream_techno_intro
- - *gas_upstream_techno_process
-
- TechnologyDescription:
+ TemporalDescription:
+ 'The time frame reflects the data representativeness for the 2016 natural gas baseline.'
- Start_date: '1/1/2016'
-
- End_date: '12/31/2016'
+ Start_date: '1/1/2016'
- DatasetOwner: 'NETL'
+ End_date: '12/31/2016'
- DataGenerator: 'NETL'
+ DataCollectionPeriod:
+ 'This inventory is representative of activities that happened during 2018.'
- DataDocumentor: 'NETL'
+ Sources:
- ModelingConstants: 'The model scope is intended to include fossil fuel resource production through delivery to the end user via the vehicle or pipeline transport in the United States.
- This scope is broken down into stages for production, mixes of produced fuel, mixes of consumed fuel, and distribution to the final user.
- Inputs to fuel production including fuels, capital infrastructure and other purchased goods or extracted resources, and any chemical release and hazardous wastes generated.'
+ Source1:
+ <<: *netl_2019_2039
- DataSelection: 'Data for this system process comes from a variety of sources.
- Main sources for emissions are EPA Greenhouse Gas Reporting Program and drilling info which provides well-level production information.'
+ NGI_2020:
- DataTreatment: 'The natural gas model divides annual reported and calculated emissions by either the production rate at a well or the throughput at a facility to provide results on the basis of 1 MJ natural gas (HHV).'
+ Description:
+ - 'This process represents the cradle-to-gate inventory for the production of gas aggregated to one of six downstream delivery regions (i.e., Pacific, Rocky Mountain, Southwest, Midwest, Southeast, and Northeast).'
+ - ''
+ - 'Functional unit: 1 MJ of natural gas, through transmission'
+ - 'Co-products: N/A'
+ - 'Default co-product approach: N/A'
+ - 'Data source: Khutal et al. (2025)'
+ - ''
+ - 'Intended for use as upstream for natural gas power plants. '
- SamplingProcedure:
- - 'BOUNDARY CONDITIONS
+ TemporalDescription:
+ 'The time frame reflects the data representativeness from the 2020 natural gas baseline.'
- These are system processes that provide cradle-to-gate inventory for the US production of natural gas.
+ Start_date: '1/1/2020'
- '
+ End_date: '12/31/2020'
- - 'DATA COLLECTION
+ DataCollectionPeriod:
+ 'This inventory is representative of activities that happened during 2025.'
- All inventories were calculated using secondary data from publicly available datasets.
+ Sources:
- '
+ Source1:
+ <<: *netl_2025_4933
- - 'UNCERTAINTY
+ Source2:
+ <<: *netl_2019_2039
- Portions (mainly greenhouse gasses) of this inventory provided by this unit process are the average of a bootstrapping approach to find the average based on a relatively small sample of facilities in the natural gas supply chain.
- Other portions of the natural gas life cycle did not consider uncertainty.
- In either case, the unit process used for the inventory does not provide any uncertainty - i.e., all emissions are static values.
+ LCIMethod:
+ - 'Attributional'
+ - 'Cradle-to-Gate process'
- '
+ IntendedApplication:
+ 'This process is intended to serve the upstream inventory to natural gas electricity generation and was created as a part of NETL''s electricity baseline.'
- - 'MORE INFORMATION
+ TechnologyDescription:
+ 'Technologies span production, gathering & boosting, processing, transmission stations, storage, pipelines, and distribution. '
- More information on the inventory is available at https://www.netl.doe.gov/energy-analysis/details?id=3198 '
- Sources:
+ DatasetOwner: 'NETL'
- Source1:
+ DataGenerator: 'NETL'
- Name: 'NETL (2019). Life Cycle Analysis of Natural Gas Extraction and Power Generation'
+ DataDocumentor: 'NETL'
- Url: 'https://netl.doe.gov/energy-analysis/details?id=3198'
+ ModelingConstants:
+ 'The NETL natural gas life cycle model includes parameters to generate inventories for natural gas extraction across all stages of the natural gas supply chain based on scenarios developed in Littlefield et al. (2019).
- Version: '1.0.1'
+ The model scope is intended to include fossil fuel resource production through delivery to the end user via the vehicle or pipeline transport in the United States.
+ This scope is broken down into stages for production, mixes of produced fuel, mixes of consumed fuel, and distribution to the final user.
+ Inputs to fuel production including fuels, capital infrastructure and other purchased goods or extracted resources, and any chemical release and hazardous wastes generated.'
- Year: 2019
+ DataSelection:
+ 'Data for this system process comes from a variety of sources.
+ Main sources for emissions are EPA Greenhouse Gas Reporting Program and drilling info which provides well-level production information.'
- TextReference: 'NETL. (2019).
- Life Cycle Analysis of Natural Gas Extraction and Power Generation.
- DOE/NETL-2019/2039.
- U.S. DOE National Energy Technology Laboratory, Pittsburgh, PA.
- https://netl.doe.gov/energy-analysis/details?id=3198'
+ DataTreatment:
+ 'The natural gas model divides annual reported and calculated emissions by either the production rate at a well or the throughput at a facility to provide results on the basis of 1 MJ natural gas (using higher heating value for conversion).'
- Category: 'NETL'
+ SamplingProcedure:
+ - 'BOUNDARY CONDITIONS'
+ - 'This system process provides cradle-to-gate inventory for the US production of natural gas.'
+ - ''
+ - 'DATA COLLECTION'
+ - 'All inventories were calculated using secondary data from publicly available datasets.'
+ - ''
+ - 'UNCERTAINTY'
+ - 'Portions (mainly greenhouse gasses) of this inventory provided by this unit process are the average of a bootstrapping approach to find the average based on a relatively small sample of facilities in the natural gas supply chain.
+ Other portions of the natural gas life cycle did not consider uncertainty.
+ In either case, the unit process used for the inventory does not provide any uncertainty (i.e., all emissions are static values).'
oil_upstream:
- techno_intro: &oil_upstream_techno_intro
- - 'The cradle-to-gate inventory for production of refinery products (residual fuel oil or diesel fuel oil) for each U.S. Petroleum Administration for Defense District (PADD). '
-
- techno_process: &oil_upstream_techno_process
- - 'The method for generating the inventory largely follows the method detailed in the journal article below, but in this case 2016 data were used for determining crude mixes new versions of the model were used.
- Additionally, OPGEE was used to provide only the amounts of various fuels used in the production of crude, and NETL unit processes were used for combustion of those fuels to provide a more complete inventory.
-
- https://pubs.acs.org/doi/abs/10.1021/acs.est.6b02819
-
- The model is constructed to provide the inventory for diesel fuel oil or residual fuel oil based on PADD.
- This inventory includes crude extraction (foreign and domestic), crude transportation, crude refining, and fuel transportation. '
-
Description:
- - *oil_upstream_techno_intro
- - *oil_upstream_techno_process
+ - 'The cradle-to-gate inventory for production of refinery products (residual fuel oil or diesel fuel oil) for each U.S. Petroleum Administration for Defense District (PADD).'
+ - ''
+ - 'Functional unit: 1 MJ of petroleum fuel through transportation'
+ - 'Co-products: N/A'
+ - 'Default co-product approach: N/A'
+ - 'Data source: Cooney et al. (2016)'
+ - ''
+ - 'This process is a part of NETL''s electricity baseline and is intended to capture the upstream emissions inventory for U.S. refinery products.'
+
+ LCIMethod:
+ - 'Attributional'
+ - 'Cradle-to-Gate process'
TechnologyDescription:
+ TemporalDescription:
+ 'The time frame reflects the data representativeness for the crude oil mixes.'
+
Start_date: '1/1/2016'
End_date: '12/31/2016'
@@ -967,73 +1421,75 @@ oil_upstream:
DataDocumentor: 'Stanford University, University of Calgary, and NETL'
- ModelingConstants: 'The model scope is intended to include fossil fuel resource production through delivery to the end user via the vehicle or pipeline transport in the United States.
+ ModelingConstants:
+ 'The model scope is intended to include fossil fuel resource production through delivery to the end user via the vehicle or pipeline transport in the United States.
This scope is broken down into stages for production, mixes of produced fuel, mixes of consumed fuel, and distribution to the final user.
Inputs to fuel production including fuels, capital infrastructure and other purchased goods or extracted resources, and any chemical release and hazardous wastes generated.
One megajoule (MJ) is the declared unit used in processes within upstream life cycle stages. '
- DataSelection: 'The Oil Production Greenhouse gas Emissions Estimator (OPGEE) and the Petroleum Refinery Life Cycle Inventory Model (PRELIM) were the main tools for modeling petroleum production and refining.
+ DataSelection:
+ 'The Oil Production Greenhouse gas Emissions Estimator (OPGEE) and the Petroleum Refinery Life Cycle Inventory Model (PRELIM) were the main tools for modeling petroleum production and refining.
OPGEE and PRELIM were developed by Stanford University and University of Calgary respectively.
OPGEE is an engineering-based tool for calculating greenhouse gas (GHG) emissions for crude oil from its extraction until it arrives at a refinery, including transport.
PRELIM is mass and energy-based process model that estimates of energy use and GHG emissions with processing crude oil in a refinery.
Further information can be found in the petroleum baseline report.'
- DataTreatment: 'Resource extraction processes are primarily based on PADD-reported data.
+ DataTreatment:
+ 'Resource extraction processes are primarily based on PADD-reported data.
All the fields defined through OPGEE were used as the pool for creating the LCI.
The production quantities reported throughout the baselines list were used for the physical flows and to generate the emissions and wastes values from facilities like refineries.
Locational data were used to associate a field with a region.
Each fuel production process, defined by fuel type and regions, includes emissions to air and water.'
SamplingProcedure:
- - 'BOUNDARY CONDITIONS
-
- All processes are compiled by US region using boundaries based on PADDs defined as ''subregions'' by the Energy Information Administration (EIA).
- Upstream fuel processes are also aggregated by energy source.
- Delivery to the final user is targeted toward an average end user without separation between different user types.
- The time span for all processes is one year, with the goal to represent the current (2018) year.'
-
- - 'DATA COLLECTION
+ - 'BOUNDARY CONDITIONS'
+ - 'All processes are compiled by US region using boundaries based on PADDs defined as ''subregions'' by the Energy Information Administration (EIA).
+ Upstream fuel processes are also aggregated by energy source.
+ Delivery to the final user is targeted toward an average end user without separation between different user types.
+ The time span for all processes is one year.'
+ - ''
+ - 'DATA COLLECTION'
+ - 'Data from 2016 were used to define the mix of crudes and petroleum refinery types.
+
+ The method for generating the inventory largely follows the method detailed in Cooney et al. (2016), but, in this case, 2016 data were used for determining crude mixes and new versions of the model were used.
+ Additionally, OPGEE was used to provide only the amounts of various fuels used in the production of crude, and NETL unit processes were used for combustion of those fuels to provide a more complete inventory.
- All inventories were calculated using secondary data from publicly available datasets.'
+ The model is constructed to provide the inventory for diesel fuel oil or residual fuel oil based on PADD.
+ This inventory includes crude extraction (foreign and domestic), crude transportation, crude refining, and fuel transportation.
- - *no_uncertainty
+ All inventories were calculated using secondary data from publicly available datasets.'
+ - ''
+ - *no_uncertainty
- DataCollectionPeriod: 'Data from 2016 was used to define the mix of crudes and petroleum refinery types.'
+ DataCollectionPeriod:
+ 'This process represents efforts conducted between 2018 and 2020.'
Sources:
- Source1:
- Name: 'Cooney et al. (2016).
- Updating the U.S. Life Cycle GHG Petroleum Baseline to 2014 with Projections to 2040 Using Open-Source Engineering-Based Models.'
-
- TextReference: 'Cooney, G., Jamieson, M., Marriott, J., Bergerson, J., Brandt, A., & Skone, T. (2016).
- Updating the U.S. Life Cycle GHG Petroleum Baseline to 2014 with Projections to 2040 Using Open-Source Engineering-Based Models.
- Environmental Science & Technology.
- doi: 10.1021/acs.est.6b02819.'
- Year: 2016
-
- Version: '1.0.2'
-
- Url: 'http://dx.doi.org/10.1021/acs.est.6b02819'
-
- Category: ''
+ Source1:
+ <<: *cooney_etal
coal_transport_upstream:
- techno_intro: &coal_transport_techno_intro
- - 'The cradle-to-gate inventory for the transportation of coal aggregated to basin, mine type, and coal type groups and also for the various modes of coal transportation. '
-
- techno_process: &coal_transport_techno_process
- - 'This inventory includes impacts from fuel combustion for the various modes of coal transportation to power plants - train, truck, tug and barge, ocean vessel, lake vessel, and conveyer.
- The aggregated inventory is created by using the transportation distances for each mode of transportation for each power plant reporting coal receipts in EIA for the year 2016.
- These inventories are then added to the power plant as upstream impacts similar to upstream coal.
- Details on the coal modeling can be found in the NETL Coal Baseline published in 2023: https://doi.org/10.2172/2370100.'
-
Description:
- - *coal_transport_techno_intro
- - *coal_transport_techno_process
+ - 'This process represents the cradle-to-gate inventory for the transportation of coal aggregated to basin, mine type, and coal type groups via the various modes of coal transportation.'
+ - ''
+ - 'Functional unit: 1 kilogram-kilometer of coal transported'
+ - 'Co-products: N/A'
+ - 'Default co-product approach: N/A'
+ - 'Data source: Cutshaw et al. (2023)'
+ - ''
+ - 'This process is a part of NETL''s electricity baseline and is intended to capture the emissions inventory for coal transportation in the US. '
+
+ LCIMethod:
+ - 'Attributional'
+ - 'Cradle-to-Gate process'
TechnologyDescription:
+ 'This inventory includes impacts from fuel combustion for the various modes of coal transportation to power plants including: train, truck, tug and barge, ocean vessel, lake vessel, and conveyer.'
+
+ TemporalDescription:
+ 'The time frame reflects the data representativeness within the coal baseline (i.e., 2016).'
Start_date: '1/1/2016'
@@ -1045,62 +1501,65 @@ coal_transport_upstream:
DataDocumentor: 'NETL'
- ModelingConstants: 'The model scope is intended to include fossil fuel resource production through delivery to the end user via the vehicle or pipeline transport in the United States.
+ ModelingConstants:
+ 'The model scope is intended to include fossil fuel resource production through delivery to the end user via the vehicle or pipeline transport in the United States.
This scope is broken down into stages for production, mixes of produced fuel, mixes of consumed fuel, and distribution to the final user.
Inputs to fuel production including fuels, capital infrastructure and other purchased goods or extracted resources, and any chemical release and hazardous wastes generated.
One megajoule (MJ) is the declared unit used in processes within upstream life cycle stages. '
- DataSelection: 'AP-42 is a compilation of air emissions factors for over 200 air pollution source categories across various industries.
+ DataSelection:
+ 'AP-42 is a compilation of air emissions factors for over 200 air pollution source categories across various industries.
AP-42 and EIA also provide heat-content data on petroleum fuel inputs which are used to relate heat input data to mass quantities of fuel and air emissions factors for fuels.
These data are used for modeling fossil fuel transport.
Further information can be found in the coal baseline report.
Transportation distances are provided by ABB Velocity Suite.
The source data has been obfuscated by aggregating transportation distances to transportation - the source includes transportation from power plants to individual mines or processing plants.'
- DataTreatment: 'Resource extraction processes are primarily based on transportation-reported data.
+ DataTreatment:
+ 'Resource extraction processes are primarily based on transportation-reported data.
All the transportation defined through NETL were used as the pool for creating the LCI.
The production quantities reported throughout the baselines list were used for the physical flows and to generate the emissions and wastes values from vehicles.
- Each transport type is defined by mode and includes emissions to air and water.'
+ Each transport type is defined by mode and includes emissions to air and water.
+ Details on the coal modeling can be found in Cutshaw et al. (2023).'
SamplingProcedure:
- - 'BOUNDARY CONDITIONS
-
- Coal transportation includes the combustion of fuels to transport coal using the various modes of transportation.
-
- '
-
- - 'DATA COLLECTION
-
- All inventories were calculated using secondary data from publicly available datasets.
-
- '
-
- - *no_uncertainty
+ - 'BOUNDARY CONDITIONS'
+ - 'Coal transportation includes the combustion of fuels to transport coal using the various modes of transportation.'
+ - ''
+ - 'DATA COLLECTION'
+ - 'Data from 2016 were used to define the amount, type, and origin of the coal used by each power plant, as well as year-specific parameters such as coal mine methane emissions.
+ The aggregated inventory is created by using the transportation distances for each mode of transportation for each power plant reporting coal receipts in EIA for the year 2016.
+ These inventories are then added to the power plant as upstream impacts similar to upstream coal.
+ All inventories were calculated using secondary data from publicly available datasets.'
+ - ''
+ - *no_uncertainty
- DataCollectionPeriod: 'Data from 2016 were used to define the amount, type, and origin of the coal used by each power plant, as well as year-specific parameters such as coal mine methane emissions.'
+ DataCollectionPeriod:
+ 'This process represents efforts conducted between 2018 and 2023.'
Sources:
Source1:
-
- <<: *netl_coal_baseline
+ <<: *netl_2024_4846
construction_upstream:
- techno_intro: &construction_techno_intro
- - 'The cradle-to-gate inventory for the construction of the fossil fuel generators (coal, gas, and oil). '
-
- techno_process: &construction_techno_process
- - 'The inventories represent the construction of either natural gas combined cycle or coal power plants using an economic input output life cycle inventory model (https://doi.org/10.1016/j.jclepro.2017.04.150) and the construction costs for these plants as detailed by NETL techno-economics analysis (https://www.netl.doe.gov/energy-analysis/details?id=729).
- These type different types of power plants are then assigned to the various power plants according to the plant they most resemble.
- The inventories are then scaled according to power plant capacity relative to the source power plants and then allocated over an assumed 30 year life.
- The single year of impacts are then divided by the generation of the year selected in the model configuration.'
-
Description:
- - *construction_techno_intro
- - *construction_techno_process
+ - 'The cradle-to-gate inventory for the construction of the fossil fuel generators (coal, gas, and oil).'
+ - ''
+ - 'Functional unit: 1 item of power plant construction'
+ - 'Co-products: N/A'
+ - 'Default co-product approach: N/A'
+ - 'Data source: Various (see sources)'
+ - ''
+ - 'This process is a part of NETL''s electricity baseline and is intended to capture the emissions inventory for the construction of fossil-based power plants in the US.'
+
+ LCIMethod:
+ - 'Attributional'
+ - 'Cradle-to-Gate process'
TechnologyDescription:
+ 'The inventories represent the construction of natural gas combined cycle or coal power plants using an economic input output life cycle inventory model (Yang et al., 2017) and the construction costs for these plants as detailed by NETL techno-economics analysis (Fout et al., 2015).'
DatasetOwner: 'NETL'
@@ -1110,327 +1569,222 @@ construction_upstream:
ModelingConstants: ''
- DataSelection: 'The inventories represent the construction of either natural gas combined cycle or coal power plants using an economic input output life cycle inventory model (https://doi.org/10.1016/j.jclepro.2017.04.150) and the construction costs for these plants as detailed by NETL techno-economics analysis (https://www.netl.doe.gov/energy-analysis/details?id=729).'
+ DataSelection:
+ 'The inventories represent the construction of either natural gas combined cycle or coal power plants using an economic input output life cycle inventory model (Yang et al., 2017) and the construction costs for these plants as detailed by NETL techno-economics analysis (Fout et al., 2015).'
- DataTreatment: 'These type different types of power plants are then assigned to the various power plants according to the plant they most resemble.
- The inventories are then scaled according to power plant capacity relative to the source power plants and then allocated over an assumed 30 year life.
- The single year of impacts are then divided by the generation of the year selected in the model configuration.'
+ DataTreatment:
+ 'These type different types of power plants are then assigned to the various power plants according to the plant they most resemble.
+ The inventories are then scaled according to power plant name plate capacity relative to the source power plants and then allocated over an assumed 30 year life.
+ The single year of impacts are then divided by the generation of the year selected in the model configuration, which is why input amounts are typically less than 1 unit.'
- SamplingProcedure:
- - 'BOUNDARY CONDITIONS
-
- The construction inventories are generated using an economic input output model and so should catch all materials and manufacturing processes that go into creating the power plant.
+ TemporalDescription:
+ 'The construction inventory was built on several years of data.
+ The inventory was generated with a target year of 2016.
+ It includes data from a 2015 techno-economic analysis and is scaled based on the model generation year.'
- '
-
- - 'DATA COLLECTION
-
- All inventories were calculated using secondary data from publicly available datasets.
-
- '
-
- - *no_uncertainty
+ SamplingProcedure:
+ - 'BOUNDARY CONDITIONS'
+ - 'The construction inventories are generated using an economic input output model and so should catch all materials and manufacturing processes that go into creating the power plant.'
+ - ''
+ - 'DATA COLLECTION'
+ - 'All inventories were calculated using secondary data from publicly available datasets.'
+ - ''
+ - *no_uncertainty
- DataCollectionPeriod: 'Data from a 2015 techo-economics analysis were used to define the construction costs for each type of power plant.'
+ DataCollectionPeriod:
+ 'This inventory represents activities conducted between 2018 and 2020.'
Sources:
Source1:
-
- Name: 'Yang et al. (2017) USEEIO: A new and transparent United States environmentally-extended input-output model'
-
- Url: 'https://doi.org/10.1016/j.jclepro.2017.04.150'
-
- TextReference: 'Yang, Y., Ingwersen, W. W., Hawkins, T. R., Srocka, M., & Meyer, D. E. (2017).
- USEEIO: A new and transparent United States environmentally-extended input-output model.
- Journal of Cleaner Production, 158, 308-318.
- doi: 10.1016/j.jclepro.2017.04.150'
-
- Year: 2017
-
- Version: '1.0.2'
-
- Category: ''
+ <<: *yang_etal
Source2:
- Name: 'NETL. (2015). Cost and Performance Baseline for Fossil Energy Plants
- Volume 1a: Bituminous Coal (PC) and Natural Gas to Electricity Revision
- 3.'
-
- Url: 'https://netl.doe.gov/energy-analysis/details?id=729'
-
- TextReference: 'NETL. (2015).
- Cost and Performance Baseline for Fossil Energy Plants Volume 1a: Bituminous Coal (PC) and Natural Gas to Electricity Revision 3.
- Document Number: National Energy Technology Laboratory, Pittsburgh, PA.
- Available from https://netl.doe.gov/energy-analysis/details?id=729. '
-
- Year: 2015
-
- Version: '1.0.1'
-
- Category: 'NETL'
+ <<: *netl_2015_1723
solarpv_construction_upstream:
- # techno_intro: &solar_const_techno_intro
- # - 'This process represents the cradle-to-gate inventory for electricity production from US solar photovoltaic power plants. '
-
- # techno_process: &solar_const_techno_process
- # - 'Inventories were developed for each solar PV plant in the 2016 EIA 860 data.
- # Boundaries include plant construction and minor operating emissions from fuel combustion, etc.
- # The inventory for the plant is allocated over an assumed 30 year life.
- # The aggregation performed in the electricityLCI code then divides those inventories by the generation in the selected year to provide results on the basis of MWh.'
Description:
- - *solar_techno_intro
- - *solar_techno_process
+ - 'This process represents the cradle-to-gate inventory for the construction of US solar photovoltaic (PV) power plants.'
+ - ''
+ - 'Functional unit: 1 item of power plant construction'
+ - 'Co-products: N/A'
+ - 'Default co-product approach: N/A'
+ - 'Data source: Various (see sources)'
+ - ''
+ - 'This process is a part of NETL''s electricity baseline and is intended to capture the emissions inventory for the construction of solar PV power plants in the US.'
- TechnologyDescription:
+ LCIMethod:
+ - 'Attributional'
+ - 'Cradle-to-Gate process'
+
+ TechnologyDescription: *solar_techno_desc
- DatasetOwner: 'IEA, NREL, JE Mason, VM Fthenakis, T Hansen, HC Kim'
+ DatasetOwner: 'IEA, NLR, JE Mason, VM Fthenakis, T Hansen, HC Kim'
DataGenerator: 'NETL'
DataDocumentor: 'NETL'
- DataTreatment: 'Material requirements to construct the solar PV panels and the balance of system are calculated using plant nameplate capacity and local insolation estimates to calculate the required area of solar panels, which are then used to scale inventories provided by literature.
+ DataTreatment:
+ 'Material requirements to construct the solar PV panels and the balance of system are calculated using plant nameplate capacity and local insolation estimates to calculate the required area of solar panels, which are then used to scale inventories provided by literature.
The total inventory from construction over the plant life is then divided by the assumed 30 year lifetime to provide emissions per year.
- When these emissions are divided by the generation for the specified year in the electricitylci, the inventory is placed on the basis of 1 MWh.
-
- There also inventories for plant operations that are calculated for each plant and are placed on the basis of a MWh before importing to the electricitylci.
- These emissions are mostly from fossil fuel combustion, hydraulic oil, and external electricity use associated with operations.
- There may also be publicly-reported emissions assigned to the plant.'
+ These emissions are divided by the 2020 generation to create the inventory on the basis of 1 MWh.'
- DataSelection: 'Where possible data is sourced from freely available life cycle data, such as NETL unit processes or USLCI.
- If the material was not available there then ecoinvent processes are used.'
-
- # Entering the year 2024 for DataCollection Period to show
- # when the most recent data was generated (representing 2020).
- DataCollectionPeriod: '2024'
-
- SamplingProcedure:
- - 'BOUNDARY CONDITIONS
+ DataSelection: *data_selection_statement
- The boundaries include power plant construction with all known material inputs (e.g., steel, glass, concrete) and some minor material requirements and fuel combustion for plant operation.
+ DataCollectionPeriod:
+ 'These data are representative of efforts that happened in 2024.'
- '
+ ModelingConstants:
+ 'The 2022 model updates include 2020 EIA data, updated aluminum functional unit, and new aluminum LCI.'
- - 'DATA COLLECTION
+ TemporalDescription:
+ 'The time frame reflects the target year for the inventory creation.'
- Data for the material inputs to the plant are from: NETL unit processes, US LCI, and ecoinvent.
+ Start_date: '1/1/2020'
- '
+ End_date: '12/31/2020'
- - *no_uncertainty
+ SamplingProcedure: &renewable_const_sp
+ - 'BOUNDARY CONDITIONS'
+ - 'The boundaries include power plant construction with all known material inputs (e.g., steel, glass, concrete).'
+ - ''
+ - 'DATA COLLECTION'
+ - 'Data for the material inputs to the plant are from NETL unit processes, US LCI, and ecoinvent.'
+ - ''
+ - *no_uncertainty
Sources:
+
Source1:
<<: *ingwersen_etal
Source2:
- Name: 'Thin-Film Photovoltaic Power Generation Offers Decreasing Greenhouse Gas Emissions and Increasing Environmental Co-benefits in the Long Term'
-
- Year: 2014
-
- TextReference: 'Bergesen, J. D., Heath, G. A., Gibon, T., & Suh, S. (2014).
- Thin-film photovoltaic power generation offers decreasing greenhouse gas emissions and increasing environmental co-benefits in the long term.
- Environmental Science & Technology, 48(16), 9834–9843.'
-
- Url: 'https://doi.org/10.1021/es405539z'
-
- Category: 'NREL'
+ <<: *bergesen_etal
Source3:
- Name: 'Life cycle inventories and life cycle assessment of photovoltaic systems'
-
- Year: 2015
-
- TextReference: 'Frischknecht, R., Itten, R., Sinha, P., de Wild-Scholten, M., Zhang, J., Fthenakis, V., Kim, H. C., Raugei, M., & Stucki, M. (2015).
- Life cycle inventories and life cycle assessment of photovoltaic systems.
- International Energy Agency (IEA) PVPS Task 12, Report T12, 4, 2015.'
-
- Url: 'https://iea-pvps.org/wp-content/uploads/2020/01/IEA-PVPS_Task_12_LCI_LCA.pdf'
-
- Category: 'NREL'
+ <<: *frischknecht_etal
Source4:
- Name: 'Energy payback and life-cycle CO2 emissions of the BOS in an optimized 3.5 MW PV installation'
-
- Year: 2015
-
- TextReference: 'Mason, J. E., Fthenakis, V. M., Hansen, T., & Kim, H. C. (2006).
- Energy payback and life-cycle CO2 emissions of the BOS in an optimized 3.5 MW PV installation.
- Progress in Photovoltaics: Research and Applications, 14(2), 179–190.'
-
- Url: 'https://doi.org/10.1002/pip.652'
-
- Category: 'Brookhaven'
+ <<: *mason_etal
Source5:
- Name: 'National Solar Radiation Database (NSRDB)'
-
- Year: 2018
-
- TextReference: 'National Renewable Energy Laboratory. (2018).
- National Solar Radiation Database (NSRDB) [data set].
- Retrieved from https://dx.doi.org/10.25984/1810289.'
-
- Url: 'https://dx.doi.org/10.25984/1810289'
-
- Category: 'NREL'
+ <<: *sengupta_etal
solartherm_construction_upstream:
- techno_intro: &solarthermal_const_techno_intro
- - 'This process represents the cradle-to-gate inventory for electricity production from US solar thermal power plants. '
-
- techno_process: &solarthermal_const_techno_process
- - 'Inventories were developed for each solar thermal plant in the 2016 or 2020 EIA 860 data, as specified in the model configuration, using a custom life cycle model.
- Boundaries include plant construction and minor operating emissions from fuel combustion, etc.
- The inventory for the plant is allocated over an assumed 30 year life.
- The aggregation performed in the electricity LCI code will then divide those inventories by the generation in the selected year to provide results on the basis of MWh.'
Description:
- - *solarthermal_const_techno_intro
- - *solarthermal_const_techno_process
+ - 'This process represents the cradle-to-gate inventory for the construction of US solar thermal power plants.'
+ - ''
+ - 'Functional unit: 1 item of power plant construction'
+ - 'Co-products: N/A'
+ - 'Default co-product approach: N/A'
+ - 'Data source: Skone et al. (2012)'
+ - ''
+ - 'This process is a part of NETL''s electricity baseline and is intended to capture the emissions inventory for the construction of solar thermal power plants in the US.'
- TechnologyDescription:
+ LCIMethod:
+ - 'Attributional'
+ - 'Cradle-to-Gate process'
+
+ TechnologyDescription: *solar_therm_techno
- DatasetOwner: 'NREL'
+ DatasetOwner: 'NLR'
DataGenerator: 'NETL'
DataDocumentor: 'NETL'
- DataTreatment: 'Material requirements to construct the solar thermal power plants are based on the plant configuration (tower vs trough), capacity, and local solar insolation values.
- The total inventory from construction and operations of the plant are then divided by the assumed 30 year lifetime to provide emissions per year.
- When these emissions are divided by the generation for the specified year in the electricity baseline code, the inventory is placed on the basis of 1 MWh.'
-
- DataSelection: 'Where possible data is sourced from freely available life cycle data, such as NETL unit processes or USLCI.
- If the material was not available there then ecoinvent processes are used.'
-
- DataCollectionPeriod: '2024'
+ DataTreatment:
+ 'Material requirements to construct the solar thermal power plants are based on the plant configuration (tower vs trough), capacity, and local solar insolation values.
+ The total inventory from construction over the plant life is then divided by the assumed 30 year lifetime to provide emissions per year.
+ These emissions are divided by the 2020 generation to create the inventory on the basis of 1 MWh.'
- SamplingProcedure:
- - 'BOUNDARY CONDITIONS
+ DataSelection: *data_selection_statement
- The boundaries include power plant construction with all known material
- inputs (e.g., steel, glass, concrete) and some minor material requirements
- and fuel combustion for plant operation.
+ DataCollectionPeriod:
+ 'These data are representative of efforts that happened in 2024.'
- '
+ ModelingConstants:
+ 'The 2022 model updates include 2020 EIA data and corrections/updates to aluminum LCI.'
- - 'DATA COLLECTION
+ TemporalDescription:
+ 'The time frame reflects the target year for the inventory creation.'
- Data for the material inputs to the plant are from: NETL unit processes, US
- LCI, and ecoinvent.
+ Start_date: '1/1/2020'
- '
+ End_date: '12/31/2020'
- - *no_uncertainty
+ SamplingProcedure: *renewable_const_sp
Sources:
+
Source1:
<<: *ingwersen_etal
Source2:
- Name: 'Role of Alternative Energy Sources: Solar Thermal Technology Assessment'
-
- Year: 2012
-
- TextReference: 'National Energy Technology Laboratory (2012).
- Role of Alternative Energy Sources: Solar Thermal Technology Assessment.
- Retrieved from https://netl.doe.gov/energy-analysis/details?id=62dcb3e0-ad31-4711-8c35-cc1b340054ab'
-
- Url: 'https://netl.doe.gov/energy-analysis/details?id=62dcb3e0-ad31-4711-8c35-cc1b340054ab'
-
- Category: 'NETL'
+ <<: *netl_2012_1532
wind_construction_upstream:
- techno_intro: &wind_const_techno_intro
- - 'Wind inventories are provided in two parts - construction and from operations.
- Together they represent the cradle-to-gate inventories for electricity production from US wind turbine power plants. '
-
- techno_process: &wind_const_techno_process
- - 'Inventories were developed for each wind farm in the 2016 or 2020 EIA 860 data, as specified in the model configuration, using a custom life cycle model.
- Boundaries include wind turbine construction and minor operating emissions from fuel combustion, maintenance, etc.
- The inventory for the wind farm is allocated over an assumed 30 year life.
- The aggregation performed in the electricity LCI code will then divide those inventories by the generation in the selected year to provide results on the basis of MWh.'
Description:
- - *wind_const_techno_intro
- - *wind_const_techno_process
-
- TechnologyDescription:
+ - 'This process represents the cradle-to-gate inventory for the construction of US wind farms.'
+ - ''
+ - 'Functional unit: 1 item of power plant construction'
+ - 'Co-products: N/A'
+ - 'Default co-product approach: N/A'
+ - 'Data source: Various (see sources)'
+ - ''
+ - 'This process is a part of NETL''s electricity baseline and is intended to capture the emissions inventory for the construction of wind farms in the US.'
- DatasetOwner: 'NREL and Vestas'
+ DatasetOwner: 'NLR and Vestas'
DataGenerator: 'NETL'
DataDocumentor: 'NETL'
- DataTreatment: 'Material requirements to construct the wind farm are calculated using wind turbine manufacturer and size information to for each turbine and aggregating all of the turbines in the wind farm.
- The total inventory from construction and operations over the plant life are then divided by the assumed 30 year lifetime to provide emissions per year.
- When these emissions are divided by the generation for the specified year in the electricity baseline code, the inventory is placed on the basis of 1 MWh.'
+ DataTreatment:
+ 'Material requirements to construct the wind farm are calculated using wind turbine manufacturer and size information to for each turbine and aggregating all of the turbines in the wind farm.
+ The total inventory from construction over the plant life is then divided by the assumed 30 year lifetime to provide emissions per year.
+ These emissions are divided by the 2020 generation to create the inventory on the basis of 1 MWh.'
- DataSelection: 'Where possible data is sourced from freely available life cycle data, such as NETL unit processes or USLCI.
- If the material was not available there then ecoinvent processes are used.'
+ DataSelection: *data_selection_statement
- DataCollectionPeriod: '2024'
+ DataCollectionPeriod:
+ 'These data are representative of efforts that happened in 2024.'
- SamplingProcedure:
- - 'BOUNDARY CONDITIONS
+ LCIMethod:
+ - 'Attributional'
+ - 'Cradle-to-Gate process'
- The boundaries include power plant construction with all known material inputs (e.g., steel, glass fiber, concrete) and some minor material requirements and fuel combustion for plant operation.
+ ModelingConstants:
+ 'The 2022 model updates include 2020 EIA data, corrections/updates to aluminum LCI, and corrections to turbine blade count, trunkline, and switchyard calculations.'
- '
+ TechnologyDescription: *wind_techno_desc
- - 'DATA COLLECTION
+ TemporalDescription:
+ 'The time frame reflects the target year for the inventory creation.'
- Data for the material inputs to the plant are from: NETL unit processes, US LCI, and ecoinvent.
+ Start_date: '1/1/2020'
- '
+ End_date: '12/31/2020'
- - *no_uncertainty
+ SamplingProcedure: *renewable_const_sp
Sources:
+
Source1:
<<: *ingwersen_etal
Source2:
- Name: 'Life Cycle Assessment of Electricity Production from an onshore V110-2.0 MW Wind Plant'
-
- Year: 2015
-
- TextReference: 'Vestas (2015). Life Cycle Assessment of Electricity Production from an onshore V110-2.0 MW Wind Plant
- 18th December 2015, Version 1.0. Vestas Wind Systems A/S, Hedeager 42, Aarhus N, 8200, Denmark.'
-
- Url: 'https://www.vestas.com/content/dam/vestas-com/global/en/sustainability/reports-and-ratings/lcas/LCAV11020MW181215.pdf.coredownload.inline.pdf'
-
- Category: 'Vestas'
+ <<: *razdan_garrett
Source3:
- Name: 'Wind Turbine Design Cost and Scaling Model.'
-
- Year: 2006
-
- Category: 'NREL'
-
- TextReference: 'Fingersh et al., 2006. Wind Turbine Design Cost and Scaling Model,
- Technical Report NREL/TP-500-40566 (Accessed June 15, 2010)'
-
- Url: 'https://docs.nrel.gov/docs/fy07osti/40566.pdf'
+ <<: *fingersh_etal
Source4:
- Name: 'Life Cycle Assessment of a Wind Turbine'
-
- Year: 2006
-
- TextReference: 'Barbara Batumbya Nalukowe, Jianguo Liu, Wiedmer Damien, Tomasz Lukawski, 2006. Life Cycle Assessment of a Wind Turbine. May 22, 2006'
+ <<: *nalukowe_etal
Source5:
- Name: 'Wind Turbines Database'
-
- Year: 2018
-
- Url: 'https://en.wind-turbine-models.com/turbines'
-
- TextReference: '(2018) Wind Turbines Database. Retrieved from https://en.wind-turbine-models.com/turbines'
+ <<: *wind_turbines_db
diff --git a/electricitylci/distribution.py b/electricitylci/distribution.py
index 40202e4..a52fe09 100644
--- a/electricitylci/distribution.py
+++ b/electricitylci/distribution.py
@@ -65,6 +65,6 @@ def distribution_mix_dictionary():
), exchanges_list
)
final = process_table_creation_distribution(reg, exchanges_list)
- distribution_dict['Distribution' + reg] = final;
+ distribution_dict['Distribution' + reg] = final
return distribution_dict
diff --git a/electricitylci/egrid_emissions_and_waste_by_facility.py b/electricitylci/egrid_emissions_and_waste_by_facility.py
index f656e71..54a743e 100644
--- a/electricitylci/egrid_emissions_and_waste_by_facility.py
+++ b/electricitylci/egrid_emissions_and_waste_by_facility.py
@@ -79,7 +79,7 @@ def get_combined_stewicombo_file(model_specs):
# relies on stewicombo, then delete the stewi and stewicombo data store
# folders on your computer (see globals.get_datastore_dir), then run
# try again. This will pull StEWI's pre-processed data. See GitHub
- # issue: https://github.com/USEPA/standardizedinventories/issues/151
+ # issue: https://github.com/NETL-RIC/standardizedinventories/issues/151
df = cbi(
base_inventory = base_inventory,
inventory_dict = model_specs.inventories_of_interest,
diff --git a/electricitylci/egrid_energy.py b/electricitylci/egrid_energy.py
index 540767c..49e6b72 100644
--- a/electricitylci/egrid_energy.py
+++ b/electricitylci/egrid_energy.py
@@ -78,7 +78,7 @@
# NOTE: this data frame is referenced in generation_mix.py
# HOTFIX: add check for missing reference data [2023-11-20; TWD]
-# See https://github.com/USEPA/ElectricityLCI/issues/211
+# See https://github.com/NETL-RIC/ElectricityLCI/issues/211
make_egrid_subregion_ref(model_specs.egrid_year)
path = os.path.join(
data_dir,
diff --git a/electricitylci/egrid_facilities.py b/electricitylci/egrid_facilities.py
index f103590..535385e 100644
--- a/electricitylci/egrid_facilities.py
+++ b/electricitylci/egrid_facilities.py
@@ -154,7 +154,7 @@ def make_egrid_subregion_ref(year):
'''pandas.DataFrame : eGRID facility-level information.'''
# Rename columns. NOTE: missing names resolved
-# (https://github.com/USEPA/standardizedinventories/issues/153)
+# (https://github.com/NETL-RIC/standardizedinventories/issues/153)
egrid_facilities.rename(columns={
'Plant primary fuel category': 'FuelCategory', # added for 2023 STEWI data
'Plant primary coal/oil/gas/ other fossil fuel category': 'FuelCategory',
diff --git a/electricitylci/eia860_facilities.py b/electricitylci/eia860_facilities.py
index 6b36439..cf313cd 100644
--- a/electricitylci/eia860_facilities.py
+++ b/electricitylci/eia860_facilities.py
@@ -17,6 +17,7 @@
from electricitylci.utils import download_unzip
from electricitylci.utils import find_file_in_folder
from electricitylci.utils import create_ba_region_map
+from electricitylci.utils import read_ba_codes
##############################################################################
@@ -29,10 +30,14 @@
For now, this module is using most of the code from eia923_generation.py.
It could be combined and generalized in the future.
+Note that this is one of the few modules in ElectricityLCI that does not
+depend on config.model_specs, which means you can utilize it freely.
+
Last updated:
- 2025-09-02
+ 2026-02-12
"""
__all__ = [
+ "add_balancing_authorities_to_plants",
"eia860_balancing_authority",
"eia860_boiler_info_design",
"eia860_download",
@@ -122,6 +127,93 @@ def _remove_table_note(df):
return df
+# NEW
+def add_balancing_authorities_to_plants(plant_df, plant_col, year):
+ """Helper function to add balancing authority information to a data frame
+ with facility IDs.
+
+ Relies on :func:`read_ba_codes` in utils.py to standardized balancing
+ authority naming.
+
+ Parameters
+ ----------
+ plant_df : pandas.DataFrame
+ A data frame where ``plant_col`` has facility IDs
+ plant_col : str
+ The column name associated with plant IDs.
+ year : int
+ The year to associate with EIA Form 860.
+
+ Returns
+ -------
+ tuple
+ A tuple of length two:
+
+ - pandas.DataFrame, the original data frame returned with additional
+ columns (e.g., 'State', 'NERC Region', 'Balancing Authority Name',
+ and 'Balancing Authority Code').
+ - dict, a dictionary with keys associated with plant IDs (int) and
+ values associated with balancing authority names (str)
+
+ Notes
+ -----
+ This method may be better utilized if it extended to multiple years'
+ worth of EIA 860 to try to capture as many facilities as possible.
+ """
+
+ # Read the EIA Form 860 for the given year
+ eia_df = eia860_balancing_authority(year)
+
+ # EIA has 'No BA' for AK facilities; this can cause problems, so remove it!
+ noba_filter = eia_df['Balancing Authority Name'] == 'No BA'
+ eia_df.loc[noba_filter, 'Balancing Authority Name'] = float('nan')
+
+ # We want BA info, so lose any rows without it.
+ eia_df = eia_df.dropna(
+ subset=['Balancing Authority Name', 'Balancing Authority Code'],
+ how='all'
+ ).copy()
+
+ # Standardized BA names
+ ba_codes = read_ba_codes()
+ eia_df["Balancing Authority Name"] = eia_df["Balancing Authority Code"].map(
+ ba_codes['BA_Name'])
+
+ # Make sure plant ID columns are integers for matching;
+ # NOTE: you can typically get away with 32-bit integer for plant IDs
+ eia_df["Plant Id"] = eia_df["Plant Id"].astype("int32")
+ plant_df[plant_col] = plant_df[plant_col].astype("int32")
+
+ # Get sets of plant IDs
+ eia_plants = set(eia_df['Plant Id'].tolist())
+ user_plants = set(plant_df[plant_col].tolist())
+
+ # Analysis of user plants not found in EIA
+ user_only = user_plants - eia_plants
+ logging.warning(
+ "Found %d plants without balancing authority info!" % len(user_only)
+ )
+
+ # Merge the EIA data to the plant data frame.
+ plant_df = plant_df.merge(
+ eia_df,
+ how="left",
+ left_on=plant_col,
+ right_on="Plant Id"
+ ).copy()
+
+ # Create a helper dictionary to map plant IDs to their BA names.
+ plant_ba_dict = {
+ x[0]: x[1]
+ for x in zip(
+ eia_df["Plant Id"],
+ eia_df["Balancing Authority Name"],
+ )
+ }
+
+ return (plant_df, plant_ba_dict)
+
+
def eia860_balancing_authority(year, regional_aggregation=None):
"""Return a data frame consisting of EIA Plant IDs and other identifying
information, including balancing authority area.
diff --git a/electricitylci/eia923_generation.py b/electricitylci/eia923_generation.py
index 0e32ddb..36a470a 100644
--- a/electricitylci/eia923_generation.py
+++ b/electricitylci/eia923_generation.py
@@ -36,7 +36,7 @@
workbook.
Last edited:
- 2025-09-05
+ 2026-02-26
"""
EIA923_PAGES = {
"1": "Page 1 Generation and Fuel Data",
@@ -94,8 +94,9 @@ def _clean_columns(df):
return df
-def build_generation_data(
- egrid_facilities_to_include=None, generation_years=None):
+def build_generation_data(egrid_facilities_to_include=None,
+ generation_years=None,
+ keep_all_cols=False):
"""Build a dataset of facility-level generation using EIA923.
This function applies filters for positive generation, generation
@@ -114,6 +115,9 @@ def build_generation_data(
Years of generation data to include in the output (default is None,
which builds a list from the inventories of interest and eia_gen_year
parameters).
+ keep_all_cols : bool, optional
+ Whether to keep all data frame columns or filter to just the three
+ listed below. Defaults to false (i.e., filter to three columns).
Returns
-------
@@ -157,6 +161,7 @@ def build_generation_data(
final_gen_df = final_gen_df.loc[f_crit, :]
if model_specs.filter_on_efficiency:
logging.info("Filtering facilities based on their efficiency")
+ # NOTE: in 2023 removes all OTHF plants; see #330 [260318; TWD]
final_gen_df = efficiency_filter(
final_gen_df,
model_specs.egrid_facility_efficiency_filters
@@ -190,9 +195,12 @@ def build_generation_data(
}
)
- all_years_gen = all_years_gen.loc[:, ["FacilityID", "Electricity", "Year"]]
all_years_gen.reset_index(drop=True, inplace=True)
all_years_gen["Year"] = all_years_gen["Year"].astype("int32")
+ if not keep_all_cols:
+ all_years_gen = all_years_gen.loc[
+ :, ["FacilityID", "Electricity", "Year"]
+ ]
return all_years_gen
@@ -237,7 +245,7 @@ def calculate_plant_efficiency(gen_fuel_data):
# HOTFIX: The sum of string columns was to repeat them (e.g., 'ALALAL' for
# three rows of 'AL') [240806;TWD].
# HOTFIX: The NAICS Code filtering must be done here. [240806; TWD]
- # See https://github.com/USEPA/ElectricityLCI/issues/232
+ # See https://github.com/NETL-RIC/ElectricityLCI/issues/232
if model_specs.filter_non_egrid_emission_on_NAICS:
logging.info("Filtering facilities by NAICS code")
row_criteria = (gen_fuel_data['NAICS Code'] == '22') & (
diff --git a/electricitylci/eia_io_trading.py b/electricitylci/eia_io_trading.py
index 5b4a53a..1fadd87 100644
--- a/electricitylci/eia_io_trading.py
+++ b/electricitylci/eia_io_trading.py
@@ -68,7 +68,7 @@
52(11), 6666-6675. https://doi.org/10.1021/acs.est.7b05191
Last updated:
- 2025-06-09
+ 2026-03-25
"""
__all__ = [
"ba_io_trading_model",
@@ -96,7 +96,7 @@ def _check_json(d):
If a JSON entry is missing data, send a critical logging statement.
The consequence of using this data is that consumption mix processes
will not be created in the JSON-LD.
- See https://github.com/USEPA/ElectricityLCI/discussions/254.
+ See https://github.com/NETL-RIC/ElectricityLCI/discussions/254.
Parameters
----------
@@ -252,7 +252,7 @@ def _get_ca_imports(just_read=False):
"New Hampshire": "ISNE",
"Florida": "FPL",
# HOTFIX: missing state maps [2024-03-14; TWD]
- # https://github.com/USEPA/ElectricityLCI/issues/236
+ # https://github.com/NETL-RIC/ElectricityLCI/issues/236
'Illinois': 'MISO',
'Missouri': 'AECI',
'South Dakota': 'SWPP',
@@ -506,6 +506,8 @@ def _make_net_gen(year, ba_cols, ng_json_list):
# Net Generation Data Import
logging.info("Creating net generation data frame with datetime")
df_net_gen = row_to_df(ng_json_list, 'net_gen')
+ # HOTFIX: convert 'net_gen' str values to float [26.02.12;TWD]
+ df_net_gen['net_gen'] = df_net_gen['net_gen'].astype('float')
logging.info("Pivoting")
df_net_gen = df_net_gen.pivot(
@@ -524,7 +526,7 @@ def _make_net_gen(year, ba_cols, ng_json_list):
# Add in missing columns, then sort in alphabetical order
logging.info("Cleaning net_gen data frame")
for i in col_diff:
- df_net_gen[i] = 0
+ df_net_gen[i] = 0.0 # HOTFIX: all cols to floats [26.02.12; TWD]
# Keep only the columns that match the balancing authority names;
# there are several other columns included in the dataset
@@ -568,6 +570,7 @@ def _make_net_gen_sum(net_trade, eia_gen, ca_gen):
# Sum values in each column
# Creates a data frame with one column, rows are BA codes, values are
# annual sums of net generation.
+ # BUG: net_trade is a mix of ints, str, and objects
df_net_gen_sum = net_trade.sum(axis=0).to_frame()
# Add Canadian import data to the net generation dataset,
@@ -757,6 +760,7 @@ def _make_trade_pivot(year, ba_cols, trade_df):
df_ba_trade_pivot = df_ba_trade_pivot.loc[
df_ba_trade_pivot.index.year==year]
+ # Force any columns that are strings to floats (they should all be floats).
cols_to_change = df_ba_trade_pivot.columns[
df_ba_trade_pivot.dtypes.eq('object')]
df_ba_trade_pivot[cols_to_change] = df_ba_trade_pivot[
@@ -786,13 +790,50 @@ def _make_trade_pivot(year, ba_cols, trade_df):
['BAA2', 'BAA1', 'Transacting BAAs'], as_index=False
)[['Exchange']].sum()
+ # Sort by BAAs before merge.
+ df_trade_sum_1_2 = df_trade_sum_1_2.sort_values(by=['BAA1', 'BAA2'])
+ df_trade_sum_2_1 = df_trade_sum_2_1.sort_values(by=['BAA2', 'BAA1'])
+
+ # Rename fields to avoid collisions.
df_trade_sum_1_2.columns = [
'BAA1_1_2', 'BAA2_1_2','Transacting BAAs_1_2', 'Exchange_1_2']
df_trade_sum_2_1.columns = [
'BAA2_2_1', 'BAA1_2_1','Transacting BAAs_2_1', 'Exchange_2_1']
# Combine two grouped tables for comparison for exchange values
- df_concat_trade = pd.concat([df_trade_sum_1_2, df_trade_sum_2_1], axis=1)
+ # BUGFIX: the old concat didn't align BAAs how we thought [26.03.25; TWD]
+ df_concat_trade = pd.merge(
+ left=df_trade_sum_1_2,
+ right=df_trade_sum_2_1,
+ how='outer',
+ left_on=['BAA1_1_2', 'BAA2_1_2'],
+ right_on=['BAA2_2_1', 'BAA1_2_1']
+ )
+
+ # Gap fill the NaNs:
+ # 1. BAA1_1_2 should match BAA2_2_1
+ # 2. BAA2_1_2 should match BAA1_2_1
+ # 3. Exchange_1_2 & Exchange_2_1 NaNs should be zero.
+ df_concat_trade['BAA1_1_2'] = df_concat_trade['BAA1_1_2'].fillna(
+ df_concat_trade['BAA2_2_1'])
+ df_concat_trade['BAA2_1_2'] = df_concat_trade['BAA2_1_2'].fillna(
+ df_concat_trade['BAA1_2_1'])
+ df_concat_trade['BAA1_2_1'] = df_concat_trade['BAA1_2_1'].fillna(
+ df_concat_trade['BAA2_1_2'])
+ df_concat_trade['BAA2_2_1'] = df_concat_trade['BAA2_2_1'].fillna(
+ df_concat_trade['BAA1_1_2'])
+ df_concat_trade['Exchange_1_2'] = df_concat_trade['Exchange_1_2'].fillna(0)
+ df_concat_trade['Exchange_2_1'] = df_concat_trade['Exchange_2_1'].fillna(0)
+
+ # Re-create the transacting BAAs
+ df_concat_trade['Transacting BAAs_1_2'] = (
+ df_concat_trade['BAA1_1_2'] + "-" + df_concat_trade['BAA2_1_2']
+ )
+ df_concat_trade['Transacting BAAs_2_1'] = (
+ df_concat_trade['BAA1_2_1'] + "-" + df_concat_trade['BAA2_2_1']
+ )
+
+ # Create an absolute amount (since exchanges can be +/-)
df_concat_trade['Exchange_1_2_abs'] = df_concat_trade['Exchange_1_2'].abs()
df_concat_trade['Exchange_2_1_abs'] = df_concat_trade['Exchange_2_1'].abs()
@@ -800,6 +841,8 @@ def _make_trade_pivot(year, ba_cols, trade_df):
# both importers or if one of the entities in the transaction reports a
# zero value. Drop combinations where any of these conditions are true,
# keep everything else.
+ # Essentially, we're looking for confirmation that BA1 -> BA2 in both
+ # columns (e.g. positive in Exchange_1_2 and negative in Exchange_2_1).
df_concat_trade['Status_Check'] = np.where(
(
(df_concat_trade['Exchange_1_2'] > 0)
@@ -815,11 +858,9 @@ def _make_trade_pivot(year, ba_cols, trade_df):
'keep'
)
- # Calculate the difference in exchange values
- df_concat_trade['Delta'] = (
- df_concat_trade['Exchange_1_2_abs']
- - df_concat_trade['Exchange_2_1_abs']
- )
+ # Run the filter. For 2023, removes 22 rows.
+ df_concat_trade = df_concat_trade.loc[
+ df_concat_trade['Status_Check'] == 'keep', :].copy()
# Calculate percent diff of exchange_abs values.
# This can be down two ways:
@@ -843,13 +884,9 @@ def _make_trade_pivot(year, ba_cols, trade_df):
df_concat_trade['Exchange_mean'] = df_concat_trade[[
'Exchange_1_2_abs', 'Exchange_2_1_abs']].mean(axis=1)
- # Percent diff equations creates NaN where both values are 0, fill with 0
- df_concat_trade['Percent_Diff_Avg'] = df_concat_trade[
- 'Percent_Diff_Avg'].fillna(0)
-
# Final exchange value based on logic;
# if percent diff is less than 20%, take mean,
- # if not use the value as reported by the exporting BAA.
+ # if not, use the value as reported by the exporting BAA.
# First figure out which BAA is the exporter by checking the value of the
# Exchange_1_2. If that value is positive, it indicates that BAA1 is
# exported to BAA2; if negative, use the value from Exchange_2_1.
@@ -883,7 +920,6 @@ def _make_trade_pivot(year, ba_cols, trade_df):
''
)
)
- df_concat_trade = df_concat_trade[df_concat_trade['Status_Check'] == 'keep']
# Create the final trading matrix; first grab the necessary columns,
# rename the columns and then pivot.
@@ -893,6 +929,16 @@ def _make_trade_pivot(year, ba_cols, trade_df):
df_concat_trade_subset.columns = [
'Exporting_BAA', 'Importing_BAA', 'Amount']
+ # Remove duplicated entries---this is caused by BA1-BA2 and BA2-BA1 existing
+ # in both groupby data frames; however, the same outcome should be found
+ # regardless of the pairing, give the logic above.
+ df_concat_trade_subset = df_concat_trade_subset.drop_duplicates()
+
+ # Sort by exporting BAAs
+ df_concat_trade_subset = df_concat_trade_subset.sort_values(
+ by=['Exporting_BAA', 'Importing_BAA']
+ )
+
trade_pivot = df_concat_trade_subset.pivot_table(
index='Exporting_BAA',
columns='Importing_BAA',
@@ -921,7 +967,7 @@ def _make_us_trade(df):
us_trade = us_trade.groupby(['export BAA'])['value'].sum().reset_index()
us_trade["fraction"] = us_trade["value"]/us_import_grouped_tot
us_trade = us_trade.fillna(value=0)
- us_trade=us_trade.drop(columns=["value"])
+ us_trade = us_trade.drop(columns=["value"])
return us_trade
@@ -984,8 +1030,17 @@ def _read_ba():
- list : U.S. FERC region codes
"""
ba_df = read_ba_codes()
+
+ # HOTFIX: drop Hawaii (HECO) [26.03.19; TWD]
+ # NOTE: Hawaii does not trade with other BA regions and the generation
+ # processes for this BA are currently dropped.
+ ba_df = ba_df.drop("HECO")
+
+ # HOTFIX: remove Canada and Mexico from U.S. BA list [26.03.19; TWD]
+ ca_filt = ba_df['EIA_Region'] != 'Canada'
+ mx_filt = ba_df['EIA_Region'] != 'Mexico'
US_BA_acronyms = sorted(list(
- ba_df.query("EIA_Region != 'Canada'").index.values
+ ba_df.loc[(ca_filt & mx_filt), :].index.values
))
df_BA_NA = ba_df.reset_index()
ferc_list = df_BA_NA['FERC_Region_Abbr'].unique().tolist()
@@ -1040,7 +1095,7 @@ def _read_bulk_api(ba_cols):
-----
For API registration, go to: https://www.eia.gov/opendata/.
- See https://github.com/USEPA/ElectricityLCI/discussions/254 for details.
+ See https://github.com/NETL-RIC/ElectricityLCI/discussions/254 for details.
If you don't want to pass ba_cols, you can find all the respondents
on the API by calling (adding ?api_key=YOUR-KEY at the end):
@@ -1207,7 +1262,7 @@ def _read_bulk_zip():
# All the entries should have a 'series_id' and an 'f' key.
# 'H' for UTC hourly; 'HL' for local hourly; hard-coded to UTC.
- # See https://github.com/USEPA/ElectricityLCI/discussions/254.
+ # See https://github.com/NETL-RIC/ElectricityLCI/discussions/254.
if 'series_id' in f_json.keys() and f_json.get('f', '') == 'H':
series_id = f_json['series_id']
@@ -1663,6 +1718,8 @@ def ba_io_trading_model(year=None, subregion=None, regions_to_keep=None):
# WARNING: Peaks around 11 GB of memory
logging.info("Creating trading data frame")
df_ba_trade = ba_exchange_to_df(BA_TO_BA_ROWS, data_type='ba_to_ba')
+ # HOTFIX: cast string to integer (values range -172k to 316k).
+ df_ba_trade['ba_to_ba'] = df_ba_trade['ba_to_ba'].astype('int')
del(BA_TO_BA_ROWS)
# Make export-import trade pivot table, make it square.
@@ -1745,7 +1802,8 @@ def ba_io_trading_model(year=None, subregion=None, regions_to_keep=None):
return {
'BA': BAA_final_trade,
'FERC': ferc_final_trade,
- 'US': us_final_trade}
+ 'US': us_final_trade
+ }
def olca_schema_consumption_mix(database, gen_dict, subregion="BA"):
@@ -2032,3 +2090,32 @@ def qio_model(net_gen_df, trade_pivot, ba_map, ba_list, roi=None, thresh=1e-5):
)
return df_final_trade_out_filt_melted_merge
+
+
+#
+# MAIN
+#
+if __name__ == "__main__":
+ from electricitylci.eia_io_trading import _check_json
+ from electricitylci.eia_io_trading import _fix_final_trade
+ from electricitylci.eia_io_trading import _get_ca_imports
+ from electricitylci.eia_io_trading import _get_zero_traders
+ from electricitylci.eia_io_trading import _get_zero_traders_w_demand
+ from electricitylci.eia_io_trading import _make_ba_trade
+ from electricitylci.eia_io_trading import _make_ferc_trade
+ from electricitylci.eia_io_trading import _make_net_gen
+ from electricitylci.eia_io_trading import _make_net_gen_sum
+ from electricitylci.eia_io_trading import _make_square_pivot
+ from electricitylci.eia_io_trading import _make_trade_pivot
+ from electricitylci.eia_io_trading import _make_us_trade
+ from electricitylci.eia_io_trading import _match_df_cols
+ from electricitylci.eia_io_trading import _read_ba
+ from electricitylci.eia_io_trading import _read_bulk
+ from electricitylci.eia_io_trading import _read_bulk_api
+ from electricitylci.eia_io_trading import _read_bulk_json
+ from electricitylci.eia_io_trading import _read_bulk_zip
+ from electricitylci.eia_io_trading import _read_ca_imports
+ from electricitylci.eia_io_trading import _read_dng_api
+ from electricitylci.eia_io_trading import _read_eia_gen
+ from electricitylci.eia_io_trading import _read_id_api
+ from electricitylci.eia_io_trading import _write_bulk_api
diff --git a/electricitylci/eia_trans_dist_grid_loss.py b/electricitylci/eia_trans_dist_grid_loss.py
index 35c2e4e..e35d195 100644
--- a/electricitylci/eia_trans_dist_grid_loss.py
+++ b/electricitylci/eia_trans_dist_grid_loss.py
@@ -109,9 +109,9 @@ def eia_trans_dist_download_extract(year):
url += "SEP%20Tables%20for%20" + f"{STATE_ABBREV[key].upper()}.xlsx"
else:
url += f"{STATE_ABBREV[key]}.xlsx"
-
- r = requests.get(url, timeout=20)
- # HOTFIX: https://github.com/USEPA/ElectricityLCI/issues/235
+
+ r = requests.get(url, timeout=20)
+ # HOTFIX: https://github.com/NETL-RIC/ElectricityLCI/issues/235
#adding 20s timeout to avoid long delays due to server issues.
with open (filename, "wb") as f:
f.write(r.content)
diff --git a/electricitylci/elementaryflows.py b/electricitylci/elementaryflows.py
index b5af3bf..896c86c 100644
--- a/electricitylci/elementaryflows.py
+++ b/electricitylci/elementaryflows.py
@@ -21,7 +21,7 @@
Types of flows and compartment information are also determined and indexed.
Last updated:
- 2025-06-09
+ 2026-01-30
"""
__all__ = [
"add_flow_direction",
@@ -89,8 +89,8 @@ def add_flow_direction(df_with_flowtypes):
def correct_netl_flow_names(df, amount_col="FlowAmount"):
"""A helper method that replaces NETL air, water, and ground emissions
with Federal Elementary Flow List equivalents based on a subset of
- flows defined in USEPA's eLCI mapping using the Python package
- `fedelemflowlist `_
+ flows defined in the FLCAC-admin elementary flow mapping Python package,
+ `fedelemflowlist`_.
Parameters
----------
@@ -108,9 +108,29 @@ def correct_netl_flow_names(df, amount_col="FlowAmount"):
are updated based on emissions matches with the FEDEFL. All unmatched
flows are returned 'as is'. If FlowUUID was not in the column list,
it is created; otherwise, the matched UUIDs are updated.
+
+ Notes
+ -----
+ The majority of fixes are with compartment names. For example,
+ "Emission to air/unspecified" -> "emission/air". Also standardizes certain
+ chemical names (e.g., "Toluene, 2,4-dinitro-" -> "2,4-Dinitrotoluene"),
+ and maps chemicals to common names (e.g., "Spent chlorofluorocarbon
+ solvents, unspecified" -> "Chlorofluorocarbons").
+
+ This method is referenced in the following methods:
+
+ - :func:`fix_coal_mining_lci` in coal_upstream.py
+ - :func:`generate_lci` in natural_gas_upstream.py
+
+ The update, which was originally added to the upstream natural gas module
+ and consists of lower-casing and right-stripping the eLCI.csv source flow
+ and source compartment names produces the same 2020 coal results (before
+ and after the modification of this method) [2025-12-30; TWD].
"""
# This data frame has about 4k source flow names and contexts associated
# with NETL unit process models (e.g., petro, nuclear, coal).
+ # NOTE: In versions >1.3.1, the eLCI.csv in fedelemflowlist is updated with
+ # 2020 natural gas mappings.
flow_mapping = fedelemflowlist.get_flowmapping('eLCI')
# Matching occurs on name and compartment; help this along by lowering the
@@ -120,8 +140,19 @@ def correct_netl_flow_names(df, amount_col="FlowAmount"):
df["FlowName"] = df["FlowName"].str.lower().str.rstrip()
df["Compartment"] = df["Compartment"].str.lower().str.rstrip()
- flow_mapping['SourceFlowName'] = flow_mapping['SourceFlowName'].str.lower()
- flow_mapping['SourceFlowContext'] = flow_mapping["SourceFlowContext"].str.lower()
+ # Lower-case names and compartments & remove trailing space from map.
+ # NOTE: this introduces duplicate entries in the map, so remove them.
+ # The duplicates are from entries that include capitalization and trailing
+ # white space; so ignore mapper, verifier and last updated cols when
+ # searching for duplicates. [251230; TWD]
+ flow_mapping['SourceFlowName'] = flow_mapping[
+ 'SourceFlowName'].str.lower().str.rstrip()
+ flow_mapping['SourceFlowContext'] = flow_mapping[
+ "SourceFlowContext"].str.lower().str.rstrip()
+ ignore_cols = ['Mapper', 'Verifier', 'LastUpdated']
+ flow_mapping = flow_mapping.drop_duplicates(
+ subset=[x for x in flow_mapping.columns if x not in ignore_cols]
+ )
# Some compartments in NETL UPs are complex (e.g., 'Emission to water/fresh
# water'), but are listed simply in the FEDEFL eLCI mapper (e.g., 'emission/
diff --git a/electricitylci/emissions_other_sources.py b/electricitylci/emissions_other_sources.py
index 59d109c..6464a9d 100644
--- a/electricitylci/emissions_other_sources.py
+++ b/electricitylci/emissions_other_sources.py
@@ -19,7 +19,7 @@
Program Data (AMPD).
Last updated:
- 2024-08-02
+ 2026-03-24
"""
__all__ = [
"integrate_replace_emissions",
@@ -88,7 +88,7 @@ def integrate_replace_emissions(new_emissions, stewi_emissions):
'eGRID_ID'
]
assert set(required_cols).issubset(set(new_emissions.columns))
- stewi_emissions["eGRID_ID"]=stewi_emissions["eGRID_ID"].astype(int)
+ stewi_emissions["eGRID_ID"] = stewi_emissions["eGRID_ID"].astype(int)
# NEI data sourced from StEWI has different capitalization than eGRID,
# while these are handled in stewicombo, here this issue persists due to
@@ -100,36 +100,37 @@ def integrate_replace_emissions(new_emissions, stewi_emissions):
"Sulfur Dioxide",
"Nitrogen Oxides",
]
- stewi_emissions.loc[
- stewi_emissions['FlowName'].isin(flow_list),
- 'FlowName'
- ] = stewi_emissions['FlowName'].str.capitalize()
+ # HOTFIX: use a filter to match left and right sides [26.03.24; TWD]
+ fl_filter = stewi_emissions['FlowName'].isin(flow_list)
+ stewi_emissions.loc[fl_filter, 'FlowName'] = stewi_emissions.loc[
+ fl_filter, 'FlowName'].str.capitalize()
# Added line below because eGRID_ID got duplicated somewhere causing
# error in concat
- stewi_emissions = stewi_emissions.loc[
- :, ~stewi_emissions.columns.duplicated()].copy()
+ dup_col_filter = stewi_emissions.columns.duplicated()
+ num_dup_cols = dup_col_filter.sum()
+ if num_dup_cols > 0:
+ logging.warning(
+ "Encountered %d duplicate columns in StEWI data; fixing." % (
+ num_dup_cols)
+ )
+ stewi_emissions = stewi_emissions.loc[:, ~dup_col_filter].copy()
updated_emissions = pd.concat([stewi_emissions, new_emissions])
- subset_cols = [
- 'Compartment', 'FlowName',
- 'Unit', 'eGRID_ID'
- ]
# Remove Year from this list, otherwise results in duplicate emissions
# by facility if years don't match in specs
+ subset_cols = [
+ 'Compartment',
+ 'FlowName',
+ 'Unit',
+ 'eGRID_ID'
+ ]
updated_emissions.drop_duplicates(
subset=subset_cols, keep='last', inplace=True)
updated_emissions.reset_index(drop=True, inplace=True)
- # Convert NEI flows back to original case for later flow mapping
- updated_emissions.loc[
- (updated_emissions['Source']=='NEI')
- & (
- updated_emissions['FlowName'].isin(
- [x.capitalize() for x in flow_list])
- ),
- 'FlowName'
- ] = updated_emissions['FlowName'].str.title()
+ # NOTE: all flow mapping is done with lowercase flow names (see
+ # elementaryflows.py); removing unnecessary retitling step. [26.03.24; TWD]
drop_columns = [
'operator_name',
diff --git a/electricitylci/generation.py b/electricitylci/generation.py
index 6af3bcc..2c946f4 100644
--- a/electricitylci/generation.py
+++ b/electricitylci/generation.py
@@ -30,7 +30,6 @@
from electricitylci.eia923_generation import build_generation_data
from electricitylci.eia923_generation import eia923_primary_fuel
import electricitylci.emissions_other_sources as em_other
-from electricitylci.globals import elci_version
from electricitylci.globals import paths
from electricitylci.globals import output_dir
import electricitylci.manual_edits as edits
@@ -39,7 +38,6 @@
from electricitylci.process_dictionary_writer import ref_exchange_creator
from electricitylci.process_dictionary_writer import uncertainty_table_creation
from electricitylci.process_dictionary_writer import unit
-from electricitylci.utils import make_valid_version_num
from electricitylci.utils import check_output_dir
from electricitylci.utils import write_csv_to_output
from electricitylci.egrid_emissions_and_waste_by_facility import (
@@ -57,33 +55,17 @@
the LCA inventories but in python dictionary format) and stores them in
computer memory.
-CHANGELOG
-
-- Remove module logger.
-- Remove unused imports.
-- Add missing documentation to methods.
-- Clean up formatting towards PEP8.
-- Note: the uncertainty calculations in :func:`aggregate_data` are
- questionable (see doc strings of submodules for details).
-- Fix the outdated pd.DataFrame.append call in :func:`turn_data_to_dict`
-- Remove :func:`add_flow_representativeness_data_quality_scores` because
- unused.
-- Replace .values with .squeeze().values when calling a data frame with
- only one column of data in :func:`olcaschema_genprocess`.
-- Fix groupby for source_db in :func:`calculate_electricity_by_source` to
- match the filter used to find multiple source entries.
-- Add empty database check in :func:`calculate_electricity_by_source`
-- Separate replace egrid function
-- Fix zero division error in aggregate data
-- Implement Hawkins-Young uncertainty
-- Add uncertainty switch
-- Drop NaNs in exchange table
-- Move FRS file download to its own function
+CHANGELOG (since v2.0)
+
+- Generalize generation process description (rm 'cradle-to-gate') [260319;TWD]
+- Add Canada generation process descriptions [260311; TWD]
+- Add 2016 to get generation years for NETL water inventory [260305; TWD]
+- Hotfix DQI entry in :func:`turn_data_to_dict` [260109; TWD]
Created:
2019-06-04
Last edited:
- 2025-06-09
+ 2026-03-19
"""
__all__ = [
"add_data_collection_score",
@@ -94,10 +76,13 @@
"calculate_electricity_by_source",
"create_generation_process_df",
"eia_facility_fuel_region",
+ "get_facilities_w_fuel_region",
+ "get_generation_years",
"hawkins_young",
"hawkins_young_sigma",
"hawkins_young_uncertainty",
"olcaschema_genprocess",
+ "read_stewi_frs",
"replace_egrid",
"turn_data_to_dict",
]
@@ -120,15 +105,27 @@ def _calc_sigma(p_series):
float
The fitted sigma for a Hawkins-Young uncertainty method.
Assumes a 90% confidence level (see :param:`alpha`).
+ Returns NaN if the series is empty or all NaNs.
+
+ Notes
+ -----
+ This method is responsible for infrequent ValueErrors, 'zero-size array to
+ reduction operation minimum which has no identity'. Occasionally, pandas
+ performs an inference check to determine the best datatype for a column.
+ In the off chance that numpy is called in to infer a NoneType, you may end
+ up with a 'no identity' error. By changing the return type from None to
+ NaN, the latter being a float, should *potentially* eliminate the problem.
"""
alpha = 0.9
- if model_specs.calculate_uncertainty:
+ # HOTFIX: skip zero-length and all NaN series [26.02.12; TWD]
+ is_empty = p_series.dropna().empty
+ if model_specs.calculate_uncertainty and not is_empty:
(is_error, sigma) = hawkins_young_sigma(p_series.values, alpha)
else:
- return None
+ return np.nan
if is_error:
- return None
+ return np.nan
else:
return sigma
@@ -235,26 +232,23 @@ def _wtd_mean(pdser, total_db):
The flow-amount-weighted average of values.
"""
# HOTFIX: averaging with NaNs in the array.
- non_nans = pdser.values[~np.isnan(pdser.values)]
+ nan_filter = ~np.isnan(pdser)
+ non_nans = pdser[nan_filter].values
if len(non_nans) == 0:
logging.debug("Encountered a NaN array!")
- result = float("nan")
+ return np.nan
else:
- try:
- wts = total_db.loc[
- pdser[~np.isnan(pdser)].index, "FlowAmount"].values
- result = np.average(non_nans, weights=wts)
- except:
- logging.debug(
- f"Error calculating weighted mean for {pdser.name}-"
- f"likely from 0 FlowAmounts"
- )
- try:
- with np.errstate(all='raise'):
- result = np.average(non_nans)
- except (ArithmeticError, ValueError, FloatingPointError):
- result = float("nan")
- return result
+ # Attempt a cleaner weighting approach [26.02.11; TWD]
+ wts = total_db.loc[pdser[nan_filter].index, "FlowAmount"].values
+ # HOTFIX: any NaN flow amount always come back NaN; zero these weights
+ # If all FlowAmounts are NaNs, then perform standard average.
+ wts = np.nan_to_num(wts, nan=0.0)
+ wts_sum = np.sum(wts)
+ if wts_sum != 0:
+ return np.average(non_nans, weights=wts)
+ else:
+ # Triggers if wts is empty or NaN array or weights sum to zero.
+ return np.average(non_nans)
def add_data_collection_score(db, elec_df, subregion="BA"):
@@ -474,9 +468,11 @@ def aggregate_data(total_db, subregion="BA"):
logging.debug("Reduce data from %d to %d rows" % (sz_tdb, len(total_db)))
# Calculate electricity totals by region and source
+ logging.info("Calculating electricity by source")
total_db, electricity_df = calculate_electricity_by_source(
total_db, subregion
)
+ logging.info("Finished calculating electricity by source")
# Assign data score based on percent generation
total_db = add_data_collection_score(total_db, electricity_df, subregion)
@@ -597,7 +593,7 @@ def aggregate_data(total_db, subregion="BA"):
database_f3['electricity_sum'] == fix_val, 'Emission_factor'] = 0
# Calculate the log-normal parameters for uncertainty; see Hawkins-Young
- # https://github.com/USEPA/ElectricityLCI/discussions/240
+ # https://github.com/NETL-RIC/ElectricityLCI/discussions/240
database_f3["GeomMean"], database_f3["GeomSD"] = zip(
*database_f3[["Emission_factor", "uncertaintySigma"]].apply(
_calc_geom_params, axis=1
@@ -907,7 +903,7 @@ def create_generation_process_df():
# NOTE: data pulled from Facility Register Service (FRS) program
# provided by USEPA's FacilityMatcher, now a part of StEWI.
- # https://github.com/USEPA/standardizedinventories
+ # https://github.com/NETL-RIC/standardizedinventories
try:
eia860_FRS = pd.read_csv(inventories_of_interest_csv)
logging.info(
@@ -942,6 +938,7 @@ def create_generation_process_df():
# Effectively removes all non-EIA facilities from StEWICombo inventory.
# drops 909 rows in 2022 inventory
+ # drops 2,545 rows in 2023 inventory
ewf_df.dropna(subset=["PGM_SYS_ID"], inplace=True)
# Drop unused columns; note legacy column names are still here.
@@ -1086,15 +1083,20 @@ def create_generation_process_df():
final_database.rename(columns={"Year_x": "Year"}, inplace=True)
# Use the Federal Elementary Flow List (FEDEFL) to map flow UUIDs
- # NOTE: 10,000 unmatched flows; mostly wastes and product flows
+ # NOTE: 10,000 unmatched flows; mostly wastes and product flows.
+ # For 2023, only 91 flows without a UUID; mainly wastes, but also
+ # includes 'Heat' input, 'Steam' output, and 'Dibenzo(a,h)Anthracene' to
+ # air.
final_database = map_emissions_to_fedelemflows(final_database)
# Sanity check that no duplicated columns exist in the data frame.
+ # 27 columns for 2023
final_database = final_database.loc[
:, ~final_database.columns.duplicated()
]
# Sanity check that no duplicate emission rows are in the data frame.
+ # 2023: 60% of rows were duplicates
dup_cols_check = [
"eGRID_ID",
"FuelCategory",
@@ -1119,7 +1121,7 @@ def create_generation_process_df():
final_database["DataCollection"] = 5
final_database["GeographicalCorrelation"] = 1
- # For surety's sake
+ # For super surety's sake
final_database["eGRID_ID"] = final_database["eGRID_ID"].astype(int)
# Organize database by facility, then by emission compartment
@@ -1146,7 +1148,7 @@ def create_generation_process_df():
# Apply the "manual edits"
# See GitHub issues #212, #160, #121, and #77.
- # https://github.com/USEPA/ElectricityLCI/issues/
+ # https://github.com/NETL-RIC/ElectricityLCI/issues/
final_database = edits.check_for_edits(
final_database, "generation.py", "create_generation_process_df")
@@ -1181,8 +1183,8 @@ def eia_facility_fuel_region(year):
- 'Balancing Authority Name' : str
"""
logging.info(
- "Generating the percent generation from primary fuel category "
- "for each facility")
+ "Calculating the percent generation in %d from primary fuel category "
+ "for each facility" % year)
primary_fuel = eia923_primary_fuel(year=year)
ba_match = eia860_balancing_authority(year)
primary_fuel["Plant Id"] = primary_fuel["Plant Id"].astype(int)
@@ -1273,6 +1275,10 @@ def get_generation_years():
# Check to see if hydro power plant data are used (always 2016)
if model_specs.include_renewable_generation is True:
generation_years += [2016]
+ # Check to see if NETL power plant water use is used (always 2016)
+ if model_specs.include_netl_water:
+ generation_years += [2016]
+
# Add years of inventories of interest; remove duplicates, and
# sort chronologically:
generation_years = sorted(list(set(
@@ -1482,6 +1488,11 @@ def olcaschema_genprocess(database, upstream_dict={}, subregion="BA"):
-------
dict
Dictionary containing openLCA-formatted data.
+
+ Notes
+ -----
+ This process is responsible for naming the generation processes
+ (e.g., 'Electricity - SOLAR - Portland General Electric Company')
"""
region_agg = subregion_col(subregion)
fuel_agg = ["FuelCategory"]
@@ -1535,23 +1546,28 @@ def olcaschema_genprocess(database, upstream_dict={}, subregion="BA"):
x for x in process_df.index.values if x[2] in upstream_dict.keys()]
# HOTFIX: only include stage codes found in process_df [241011; TWD]
sc_list = list(set([x[2] for x in provider_filter]))
+ # Loop over rows with an upstream stage code:
for index, row in process_df.loc(axis=0)[:, :, sc_list].iterrows():
# New Issue #150, try first to match regional construction. Fall back
# is US average.
if "_const" in index[2]:
try:
+ # Extract upstream regional construction process info
+ u_key = index[2] + " - " + index[0]
provider_dict = {
- "name": upstream_dict[index[2] + " - " +index[0]]["name"],
- "categoryPath": upstream_dict[index[2] + " - " +index[0]]["category"],
+ "name": upstream_dict[u_key]["name"],
+ "categoryPath": upstream_dict[u_key]["category"],
"processType": "UNIT_PROCESS",
- "@id": upstream_dict[index[2] + " - " +index[0]]["uuid"],
+ "@id": upstream_dict[u_key]["uuid"],
}
except KeyError:
+ # Fall back on the U.S. average process
+ u_key2 = index[2]
provider_dict = {
- "name": upstream_dict[index[2]]["name"],
- "categoryPath": upstream_dict[index[2]]["category"],
+ "name": upstream_dict[u_key2]["name"],
+ "categoryPath": upstream_dict[u_key2]["category"],
"processType": "UNIT_PROCESS",
- "@id": upstream_dict[index[2]]["uuid"],
+ "@id": upstream_dict[u_key2]["uuid"],
}
else:
provider_dict = {
@@ -1560,15 +1576,18 @@ def olcaschema_genprocess(database, upstream_dict={}, subregion="BA"):
"processType": "UNIT_PROCESS",
"@id": upstream_dict[index[2]]["uuid"],
}
+ # Create the upstream flow and make the upstream process the provider.
row["exchanges"][0]["provider"] = provider_dict
row["exchanges"][0]["unit"] = unit(
upstream_dict[index[2]]["q_reference_unit"]
)
row["exchanges"][0]["FlowType"] = "PRODUCT_FLOW"
+ # Copy the upstream process into the power plant process as an exchange.
process_df.loc[index[0], index[1], "Power plant"]["exchanges"].append(
row["exchanges"][0])
- # These are now only power plant stage codes (and life cycle for CAN)
+ # Remove upstream processes from data frame; they are now provider in the
+ # exchange table of the power plant generation processes.
process_df = process_df.drop(provider_filter)
process_df.reset_index(inplace=True)
@@ -1591,11 +1610,13 @@ def olcaschema_genprocess(database, upstream_dict={}, subregion="BA"):
)
# HOTFIX: construction processes are handled in upstream_dict.py;
- # remove filter and assignment from here.
+ # filter and assignment removed from here.
if region_agg is None:
process_df["location"] = "US"
process_df["description"] = (
- "Electricity from "
+ # Use 'life cycle' in place of 'cradle-to-gate'; see iss328
+ "This process represents the life cycle inventory "
+ + "for the production of electricity from "
+ process_df[fuel_agg].squeeze().values
+ " produced at generating facilities in the US."
)
@@ -1604,38 +1625,48 @@ def olcaschema_genprocess(database, upstream_dict={}, subregion="BA"):
)
else:
# HOTFIX: remove .values, which throws ValueError [2023-11-13; TWD]
- process_df["location"] = process_df[region_agg]
- process_df["description"] = (
- "Electricity from "
- + process_df[fuel_agg].squeeze().values
- + " produced at generating facilities in the "
- + process_df[region_agg].squeeze().values
- + " region."
- )
+ # Update the intro statement for generation processes [26.02.27; TWD]
process_df["name"] = (
"Electricity - "
+ process_df[fuel_agg].squeeze().values
+ " - "
+ process_df[region_agg].squeeze().values
)
+ process_df["location"] = process_df[region_agg]
- # Add model reference and version number
- process_df["description"] += (
- " This process was created with ElectricityLCI "
- + "(https://github.com/USEPA/ElectricityLCI) version " + elci_version
- + " using the " + model_specs.model_name + " configuration."
- )
- process_df["version"] = make_valid_version_num(elci_version)
+ # Try for a better Canadian generation description [26.03.11; TWD]
+ is_canada = process_df[fuel_agg[0]] == "ALL"
+ not_canada = ~(is_canada)
+
+ process_df.loc[not_canada, "description"] = (
+ # Use 'life cycle' in place of 'cradle-to-gate'; see iss328
+ "This process represents the life cycle inventory "
+ + "for the production of "
+ + process_df.loc[not_canada, fuel_agg].squeeze().values
+ + "-powered electricity produced at generating facilities in the "
+ + process_df.loc[not_canada, region_agg].squeeze().values
+ + " region.\n"
+ )
+ process_df.loc[is_canada, "description"] = (
+ # Canadian processes are basically roll-ups; okay to be C2G.
+ "This process represents the cradle-to-gate inventory "
+ + "for the production of electricity in the Canadian region of "
+ + process_df.loc[is_canada, region_agg].squeeze().values
+ + ".\n"
+ )
- # TODO: use `process_description_creation` from process_dictionary_writer to fill in this portion; note that the default text below is captured in the return string from that method.
+ # HOTFIX: remove duplicate eLCI model reference & version number---
+ # this is represented in the 'description' key in processDocumentation.
# Create the dictionaries for process documentation based on fuel type.
- # NOTE: this creates process-level DQI (5;5)
+ # NOTE: this defines the process-level DQI
process_df["processDocumentation"] = [
process_doc_creation(x) for x in list(
process_df["FuelCategory"].str.lower())
]
+ # Append YAML descriptions to generating processes (e.g., biomass,
+ # geothermal, hydro, solar, solarthermal, wind).
process_df["description"] += [
"\n" + x["description"] for x in process_df["processDocumentation"]
]
@@ -1650,10 +1681,12 @@ def olcaschema_genprocess(database, upstream_dict={}, subregion="BA"):
"processDocumentation",
"processType",
"name",
- "version",
+ "version", # NEW--- failed to find this column [26.02.26; TWD]
"category",
"description",
]
+ # HOTFIX: make sure all columns are represented [26.02.26; TWD]
+ process_cols = [x for x in process_cols if x in process_df.columns]
result = process_df[process_cols].to_dict("index")
return result
@@ -1745,10 +1778,9 @@ def turn_data_to_dict(data, upstream_dict):
----------
data : pandas.DataFrame
A multi-row data frame containing aggregated emissions to be turned
- into openLCA unit processes. Columns include the follow (as defined by
- `ng_agg_cols` in :func:`olcaschema_genprocess`):
+ into openLCA unit processes. Columns include the following (as defined
+ by `ng_agg_cols` in :func:`olcaschema_genprocess`):
- - stage_code
- FlowName
- FlowUUID
- Compartment
@@ -1767,6 +1799,9 @@ def turn_data_to_dict(data, upstream_dict):
- GeomMean
- GeomSD
+ The data.name should return the group tuple (e.g., balancing authority
+ name, fuel category, stage code).
+
upstream_dict : dict
Dictionary as created by upstream_dict.py, containing the openLCA
formatted data for all of the fuel inputs.
@@ -1801,12 +1836,13 @@ def turn_data_to_dict(data, upstream_dict):
# HOTFIX: remove exchanges that have NaNs for Emission_factor;
# they crash openLCA. [240813; TWD]
- # https://github.com/USEPA/ElectricityLCI/issues/246
+ # https://github.com/NETL-RIC/ElectricityLCI/issues/246
num_nans = data['Emission_factor'].isna().sum()
if num_nans > 0:
logging.info("Removing %d nans from exchange table" % num_nans)
data = data.dropna(subset='Emission_factor')
+ # Create new columns and fill with relevant info
data["internalId"] = ""
data["@type"] = "Exchange"
data["avoidedProduct"] = False
@@ -1830,8 +1866,8 @@ def turn_data_to_dict(data, upstream_dict):
data.loc[input_filter, "input"] = True
# Define products based on compartment label
- # HOTFIT: input compartment tends to be technosphere flow
- product_filter=(
+ # HOTFIX: input compartment tends to be technosphere flow
+ product_filter = (
(data["Compartment"].str.lower().str.contains("technosphere"))
| (data["Compartment"].str.lower().str.contains("valuable"))
| (data["Compartment"].str.lower().str.contains("input"))
@@ -1844,9 +1880,11 @@ def turn_data_to_dict(data, upstream_dict):
)
data.loc[waste_filter, "FlowType"] = "WASTE_FLOW"
- data["flow"] = ""
- data["uncertainty"] = ""
+ # HOTFIX: pandas 3 has dedicated 'str' datatype; use None for generic object
+ data["flow"] = None
+ data["uncertainty"] = None
for index, row in data.iterrows():
+ # Reminder: use '.at' to assign dictionary as a single row/col's value
data.at[index, "uncertainty"] = uncertainty_table_creation(
data.loc[index:index, :]
)
@@ -1859,17 +1897,15 @@ def turn_data_to_dict(data, upstream_dict):
data["quantitativeReference"] = False
# Pull pedigree matrix values for DQI
+ # HOTFIX: this assumes the first row's DQI value is the same for all
+ # rows; however, each flow has its own DQI values [260109; TWD]
data["dqEntry"] = (
"("
- + str(round(data["DataReliability"].iloc[0], 1))
- + ";"
- + str(round(data["TemporalCorrelation"].iloc[0], 1))
- + ";"
- + str(round(data["GeographicalCorrelation"].iloc[0], 1))
- + ";"
- + str(round(data["TechnologicalCorrelation"].iloc[0], 1))
- + ";"
- + str(round(data["DataCollection"].iloc[0], 1))
+ + data['DataReliability'].round().astype(int).astype(str) + ';'
+ + data['TemporalCorrelation'].round().astype(int).astype(str) + ';'
+ + data['GeographicalCorrelation'].round().astype(int).astype(str) + ';'
+ + data['TechnologicalCorrelation'].round().astype(int).astype(str) + ';'
+ + data['DataCollection'].round().astype(int).astype(str)
+ ")"
)
data["pedigreeUncertainty"] = ""
diff --git a/electricitylci/generation_mix.py b/electricitylci/generation_mix.py
index e016d45..1d8ff9f 100644
--- a/electricitylci/generation_mix.py
+++ b/electricitylci/generation_mix.py
@@ -33,7 +33,7 @@
(either from generation data or straight from eGRID).
Last edited:
- 2025-06-09
+ 2026-03-18
"""
@@ -187,9 +187,18 @@ def create_generation_mix_process_df_from_model_generation_data(
# pulls 2 MWh for every 1MWh generated. For simplicity and because
# NBSO is an imported BA, we'll remove the US-side and assume it's
# covered under the Canadian imports.
+ nbo_filter = (
+ subregion_fuel_gen["Subregion"] != BA_CODES.loc['NBSO', 'BA_Name']
+ )
+
+ # Issue #276---HECO fails when residual grid mixes are generated.
+ # Currently, HECO is hard filtered (see manual_edits.yml); repeat here.
+ heco_filter = (
+ subregion_fuel_gen["Subregion"] != BA_CODES.loc['HECO', 'BA_Name']
+ )
+
subregion_fuel_gen = subregion_fuel_gen.loc[
- subregion_fuel_gen["Subregion"] != "New Brunswick System Operator",
- :
+ (nbo_filter & heco_filter), :
]
canada_list=[]
@@ -361,7 +370,7 @@ def olcaschema_international(database, gen_dict, subregion=None):
def olcaschema_genmix(database, gen_dict, subregion=None):
- """Generate an olca-schema process for each region-fuel pairing.
+ """Create the 'at-grid; generation mix' processes each region-fuel pairing.
Parameters
----------
@@ -376,6 +385,12 @@ def olcaschema_genmix(database, gen_dict, subregion=None):
-------
dict
An olca-schema-formatted process dictionary.
+
+ Notes
+ -----
+ This method is called in :func:`write_generation_mix_database_to_dict`
+ in __init__.py, which is called during :func:`run_distribution` in main.py
+ to create the generation mix dictionary from the generation mix data frame.
"""
if subregion is None:
subregion = model_specs.regional_aggregation
@@ -393,6 +408,8 @@ def olcaschema_genmix(database, gen_dict, subregion=None):
f_list = list(database["FuelCategory"].unique())
for reg in region:
+ # Get the fuel mix associated with the current region
+ # (e.g., GAS, COAL, SOLAR, WIND).
database_reg = database[database["Subregion"] == reg]
exchanges_list = []
@@ -406,41 +423,41 @@ def olcaschema_genmix(database, gen_dict, subregion=None):
database_f1 = database_reg[
database_reg["FuelCategory"] == fuelname
]
- if database_f1.empty != True:
- matching_dict = {
- 'Electricity': None,
- 'Construction': None
- }
- # Iss150, need to search for both electricity and construction
+ if database_f1.empty:
+ continue
+ else:
+ # NOTE: Issue 150 put regionalized construction within the
+ # generation processes (e.g., an resource flow within
+ # 'Electricity - WIND - Avangrid Renewables, LLC');
+ # removing the search for construction here [260213;TWD].
+ matching_dict = {'Electricity': None}
m_str1 = "Electricity - " + fuelname + " - " + reg
- m_str2 = "Construction - " + fuelname + " - " + reg
+
+ # Single search through generation processes for a match
+ # to current fuel and region (e.g., 'Electricity - SOLAR -
+ # Avangrid Renewables, LLC').
for generator in gen_dict:
if gen_dict[generator]["name"] == m_str1:
logging.debug(
"Found matching dictionary for '%s'" % m_str1)
matching_dict['Electricity'] = gen_dict[generator]
- elif gen_dict[generator]["name"] == m_str2:
- logging.debug(
- "Found matching dictionary for '%s'" % m_str2)
- matching_dict['Construction'] = gen_dict[generator]
- # Still allow breaking if we've found both dicts.
- if (matching_dict['Construction'] is not None) and (
- matching_dict['Electricity'] is not None):
- logging.debug("Found both!")
break
- for k, match in matching_dict.items():
- if match is not None:
+ for k, p in matching_dict.items():
+ if p is not None:
+ # Create default exchange table, sets the default
+ # provider based on 'Subregion' field and fuelname,
+ # and exchange amount based on 'Generation_Ratio'.
ra = exchange_table_creation_input_genmix(
database_f1, fuelname
)
+ # These are inputs; set quantitative reference to false.
ra["quantitativeReference"] = False
- # HOTFIX: make category string, not list
- # [2023-11-29; TWD]
+ # HOTFIX: make category string, not list [231129; TWD]
ra["provider"] = {
- "name": match["name"],
- "@id": match["uuid"],
- "category": match["category"],
+ "name": p["name"],
+ "@id": p["uuid"],
+ "category": p["category"],
}
exchanges_list = exchange(ra, exchanges_list)
else:
@@ -450,6 +467,7 @@ def olcaschema_genmix(database, gen_dict, subregion=None):
"Skipping this flow for now"
)
+ # Send the region and exchanges list to create the Process dictionary
final = process_table_creation_genmix(reg, exchanges_list)
generation_mix_dict[reg] = final
@@ -482,7 +500,7 @@ def olcaschema_usaverage(
Notes
-----
- Reference in `run_epa_trade` in \_\_init\_\_.py
+ Reference in `run_epa_trade` in \\_\\_init\\_\\_.py
"""
if subregion is None:
subregion = model_specs.regional_aggregation
diff --git a/electricitylci/geothermal.py b/electricitylci/geothermal.py
index 2d6c8db..a6f5250 100644
--- a/electricitylci/geothermal.py
+++ b/electricitylci/geothermal.py
@@ -13,7 +13,7 @@
from electricitylci.eia923_generation import eia923_download_extract #modelspecs
from electricitylci.globals import data_dir
from electricitylci.globals import output_dir
-
+from electricitylci.utils import filter_out_zero
from electricitylci.solar_upstream import fix_renewable
@@ -28,7 +28,7 @@
Created:
2019-05-31
Last updated:
- 2025-01-22
+ 2026-02-10
"""
@@ -94,7 +94,7 @@ def generate_upstream_geo():
"us_average",
]
- # Read the state-level geothermal LCI file
+ # Read the state-level geothermal LCI file (1740 rows x 12 cols)
# Columns include 'california', 'hawaii', 'idaho', 'new_mexico', 'nevada',
# 'utah', and 'us_average' (see geothermal_states list); as well as
# 'Directionality' (e.g., resource or emission), and 'Compartment'
@@ -114,14 +114,23 @@ def generate_upstream_geo():
)
geo_lci["stage_code"] = geo_lci["stage_code"].map(geothermal_state_dict)
+ # HOTFIX: remove zero-quantity flows from LCI (Issue 314) [26.02.10;TWD]
+ # -> filters 3,072 rows
+ geo_lci = filter_out_zero(geo_lci, "FlowAmount")
+
# Pull generation data and aggregate across prime movers [25.01.22; TWD]
# NOTE: there are 3--5 duplicated facilities for years 2016, 2020-2022.
+ # NOTE: Yes, use 2016 generation to match inventory.
geo_generation_data = get_geo_generation(2016)
geo_generation_data = geo_generation_data.groupby(
by='Plant Id').agg({
'State': 'first',
'EIA Sector Number': 'first',
'Net Generation (Megawatthours)': 'sum'}).reset_index()
+
+ # Merge generation and inventory.
+ # NOTE: 'US' is the only non-matched key from geo_lci.
+ # NOTE: 'quantity' represents net generation (MWh).
geo_merged = pd.merge(
left=geo_generation_data,
right=geo_lci,
@@ -154,7 +163,7 @@ def generate_upstream_geo():
inplace=True
)
- # NOTE
+ # NOTE:
# Compartments are already lowercase (e.g., 'resource', 'water', 'air',
# 'ground'), so no additional mapping needed [25.01.15; TWD].
@@ -162,8 +171,8 @@ def generate_upstream_geo():
geo_merged["Electricity"] = geo_merged["quantity"]
geo_merged["fuel_type"] = "GEOTHERMAL"
geo_merged["stage_code"] = "Power plant"
- geo_merged["Year"]=2016
- # Map directionality to resources (true) or emissions (false)
+ geo_merged["Year"] = 2016
+ # Map directionality to input: resources (true) or emissions (false)
input_dict = {"emission": False, "resource": True}
geo_merged["Directionality"] = geo_merged["Directionality"].map(input_dict)
geo_merged.rename(
diff --git a/electricitylci/globals.py b/electricitylci/globals.py
index acf84a2..890398d 100644
--- a/electricitylci/globals.py
+++ b/electricitylci/globals.py
@@ -20,7 +20,7 @@
modules.
Last updated:
- 2025-08-14
+ 2026-03-27
"""
@@ -33,28 +33,47 @@
except NameError:
modulepath = 'electricitylci/'
-paths=Paths()
+paths = Paths()
+'''Paths : esupy class object to connect user's data directory.'''
paths.local_path = os.path.realpath(str(paths.local_path) + "/electricitylci")
+
# NOTE: output_dir used in a handful of modules (e.g., combinator)
# HOTFIX PosixPath in os.path.join [TWD; 2023-07-27]
output_dir = os.path.join(str(paths.local_path), 'output')
+'''str : The ElectricityLCI local output folder (where models are saved).'''
+
data_dir = os.path.join(modulepath, 'data')
+'''str : The ElectricityLCI Python package's data folder.'''
+elci_version = "0.0.0"
+'''str : The ElectricityLCI Python package version.'''
try:
# HOTFIX: remove dependency on setuptools and its deprecated pkg_resources
elci_version = version("ElectricityLCI")
except:
- elci_version = "2.0.0"
+ elci_version = "3.0.0"
# ref Table 1.1 NERC report
electricity_flow_name_generation_and_distribution = (
'Electricity, AC, 2300-7650 V')
electricity_flow_name_consumption = 'Electricity, AC, 120 V'
-# EIA923 download url - this is just the base, need to add
-# extension and file name
+# GitHub repo URL
+GH_URL = "https://github.com/NETL-RIC/ElectricityLCI"
+'''str : The web address for ElectricityLCI GitHub repository.'''
+
+# EIA base URLs - need to add file name
EIA923_BASE_URL = 'https://www.eia.gov/electricity/data/eia923/'
+'''str : The base URL for EIA Form 923 workbooks.'''
EIA860_BASE_URL = 'https://www.eia.gov/electricity/data/eia860/'
+'''str : The base URL for EIA Form 860 workbooks.'''
+NREL_REC_YEAR = 2024
+'''int : See https://www.nlr.gov/analysis/renewable-power for pub years.'''
+NREL_REC_URL = (
+ "https://www.nlr.gov/"
+ f"docs/libraries/analysis/nrel-green-power-data-v{NREL_REC_YEAR}.xlsx"
+)
+'''str : NLR voluntary renewable power procurement data sheet URL.'''
# EPA Clean Air Markets API URL
# https://www.epa.gov/power-sector/cam-api-portal
@@ -64,51 +83,83 @@
)
'''str : EPA CEMS annual apportioned emissions by facility API URL'''
+# Coal model constants
+COAL_BASIN_CODES = {
+ 'Central Appalachia': 'CA',
+ 'Central Interior': 'CI',
+ 'Gulf Lignite': 'GL',
+ 'Illinois Basin': 'IB',
+ 'Lignite': 'L',
+ 'Northern Appalachia': 'NA',
+ 'Powder River Basin': 'PRB',
+ 'Rocky Mountain': 'RM',
+ 'Southern Appalachia': 'SA',
+ 'West/Northwest': 'WNW',
+ 'Import': 'IMP',
+}
+'''dict : A map between NETL coal basin names and their abbreviations.'''
+
+COAL_TYPE_CODES = {
+ 'BIT': 'B',
+ 'LIG': 'L',
+ 'SUB': 'S',
+ 'WC': 'W',
+ 'RC' : 'RC',
+}
+'''dict : Map between EIA coal fuel source codes and NETL coal codes.'''
+
+COAL_MINE_CODES = {
+ 'Surface': 'S',
+ 'Underground': 'U',
+ 'Facility': 'F',
+ 'Processing': 'P',
+}
+'''dict : A map between coal mine type and their abbreviation.'''
+
# Grouping of Reported fuel codes to EPA categories
FUEL_CAT_CODES = {
- 'BIT': 'COAL',
- 'SUB': 'COAL',
- 'LIG': 'COAL',
- 'RC': 'COAL',
- 'ANT': 'COAL',
- 'SGC': 'COAL',
- 'SC': 'COAL',
- 'NG': 'GAS',
- 'NUC': 'NUCLEAR',
- 'WND': 'WIND',
- 'SUN': 'SOLAR',
- 'DFO': 'OIL',
- 'RFO': 'OIL',
- 'WAT': 'HYDRO',
- # 'HPS': 'OTHF',
- 'GEO': 'GEOTHERMAL',
- 'WO': 'OIL',
- 'KER': 'OIL',
- 'JF': 'OIL',
- 'PG': 'OIL',
- 'BLQ': 'BIOMASS',
- 'WDS': 'BIOMASS',
- 'WDL': 'BIOMASS',
- 'PC': 'OIL',
- 'SGP': 'OIL',
- 'MSB': 'BIOMASS',
- 'MSN': 'OTHF',
- 'LFG': 'BIOMASS',
- 'WOC': 'COAL',
- 'WH': 'OTHF',
- 'MSN': 'OTHF',
- 'OTH': 'OTHF',
- 'TDF': 'OTHF',
- 'PUR': 'OTHF',
- 'MWH': 'OTHF',
- 'AB': 'BIOMASS',
- 'OBL': 'BIOMASS',
- 'SLW': 'BIOMASS',
- 'OBG': 'BIOMASS',
- 'OBS': 'BIOMASS',
- 'OG': 'OFSL',
- 'BFG': 'OFSL',
- 'WC': 'COAL'
+ 'AB': 'BIOMASS', # Agricultural byproducts
+ 'ANT': 'COAL', # Anthracite coal
+ 'BFG': 'OFSL', # Blast furnace gas
+ 'BIT': 'COAL', # Bituminous coal
+ 'BLQ': 'BIOMASS', # Black liquor
+ 'DFO': 'OIL', # Distillate fuel oil (e.g., diesel)
+ 'GEO': 'GEOTHERMAL', # Geothermal
+ # 'H2': 'HYDROGEN' # Hydrogen
+ 'JF': 'OIL', # Jet fuel
+ 'KER': 'OIL', # Kerosene
+ 'LFG': 'BIOMASS', # Landfill gas
+ 'LIG': 'COAL', # Lignite coal
+ 'MSB': 'BIOMASS', # Biogenic municipal solid waste
+ 'MSN': 'OTHF', # Non-biogenic municipal solid waste
+ 'MWH': 'OTHF', # Electricity for energy storage
+ 'NG': 'GAS', # Natural gas
+ 'NUC': 'NUCLEAR', # Nuclear (e.g., uranium, plutonium, thorium)
+ 'OBG': 'BIOMASS', # Other biomass gas (e.g., digester)
+ 'OBL': 'BIOMASS', # Other biomass liquids
+ 'OBS': 'BIOMASS', # Other biomass solids
+ 'OG': 'OFSL', # Other gas
+ 'OTH': 'OTHF', # Other fuel
+ 'PC': 'OIL', # Petroleum coke
+ 'PG': 'OIL', # Gaseous propane
+ 'PUR': 'OTHF', # Purchased steam
+ 'RC': 'COAL', # Refined coal
+ 'RFO': 'OIL', # Residual fuel oil
+ 'SC': 'COAL', # Coal-derived synthesis fuel
+ 'SGC': 'COAL', # Coal-derived synthesis gas
+ 'SGP': 'OIL', # Synthesis gas from petroleum coke
+ 'SLW': 'BIOMASS', # Sludge waste
+ 'SUB': 'COAL', # Subbituminous coal
+ 'SUN': 'SOLAR', # Solar
+ 'TDF': 'OTHF', # Tire-derived fuels
+ 'WAT': 'HYDRO', # Water (e.g., hydroelectric/hydrokinetic)
+ 'WC': 'COAL', # Waste/other coal
+ 'WDL': 'BIOMASS', # Wood waste liquids (excludes black liquor)
+ 'WDS': 'BIOMASS', # Wood/wood waste solids
+ 'WH': 'OTHF', # Waste heat (unattributed)
+ 'WND': 'WIND', # Wind
+ 'WO': 'OIL', # Waste/other oil
+ 'WOC': 'COAL', # Waste coal
}
US_STATES = {
@@ -236,6 +287,25 @@
NG_MODEL_YEARS = [2016, 2020]
'''list : The valid years for natural gas model (i.e., 2016 and 2020).'''
+GREEN_E = ['HYDRO', 'BIOMASS', 'SOLAR', 'SOLARTHERMAL', 'WIND', 'GEOTHERMAL']
+'''list: Green or renewable energy categories for residual mixes.'''
+
+OVERFLOW_E = ['MIXED', 'OTHF']
+'''list: Non-green fuels that can lend overflow electricity for res. mixes.'''
+
+REM_WEIGHT_METHODS = ['count', 'gen']
+'''list : State-level REC sales to Balancing authority weighting methods.'''
+
+NEG_REM_METHODS = ['zero', 'keep']
+'''list : Accounting methods for negative renewable generation for REM.'''
+
+C2G_LCI_METHOD = "Attributional\nCradle-to-Gate process"
+'''str : Metadata text for cradle-to-gate inventory method description'''
+
+G2G_LCI_METHOD = "Attributional\nGate-to-Gate process"
+'''str : Metadata text for gate-to-gate inventory method description'''
+
+
##############################################################################
# FUNCTIONS
##############################################################################
diff --git a/electricitylci/import_impacts.py b/electricitylci/import_impacts.py
index 49451d8..0b766f0 100644
--- a/electricitylci/import_impacts.py
+++ b/electricitylci/import_impacts.py
@@ -30,7 +30,7 @@
-level biomass emissions are multiplied by 0.09. The result is a data frame
that includes balancing authority level inventories for Canadian imports.
-See https://github.com/USEPA/ElectricityLCI/issues/231 for details and
+See https://github.com/NETL-RIC/ElectricityLCI/issues/231 for details and
references.
Last updated:
diff --git a/electricitylci/main.py b/electricitylci/main.py
index 92d9aa0..a3c966e 100644
--- a/electricitylci/main.py
+++ b/electricitylci/main.py
@@ -32,7 +32,7 @@
of this script or it may be passed following the command-line argument, '-c'.
Last updated:
- 2025-03-14
+ 2026-03-05
Changelog:
- Address logging handler import for Python 3.12 compatibility.
@@ -46,6 +46,14 @@
- Test facility-level inventory generation.
- Make use of the post-processing configuration parameter.
- Make main() runnable (add ``is_set`` param)
+
+Quick start. Use the following to create a logger and initialize model
+specifications.
+
+>>> from electricity import basic_setup
+>>> basic_setup("INFO", "ELCI_2023") # 'INFO' and 'ELCI_2023' are defaults
+>>> from electricitylci.main import main
+>>> main()
"""
__all__ = [
"main",
@@ -57,7 +65,7 @@
##############################################################################
# FUNCTIONS
##############################################################################
-def main(is_set=False):
+def main():
"""Generate an openLCA-schema JSON-LD zip file containing the life cycle
inventory for US power plants based on the settings in the user-specified
configuration file. The basic workflow is as follows:
@@ -93,7 +101,8 @@ def main(is_set=False):
>>> config.model_specs = config.build_model_class()
>>> print(config.model_specs.namestr)
"""
- if not is_set or config.model_specs is None:
+ # HOTFIX: replace the is_set with getattr [26.03.05; TWD]
+ if getattr(config, 'model_specs', None) is None:
# Prompt user to select configuration option.
# These are defined as YAML files in the modelconfig/ folder in the
# eLCI package; you might have to search site-packages under lib.
@@ -134,6 +143,7 @@ def run_distribution(generation_process_dict):
generation_mix_df = get_generation_mix_process_df()
# Create the "Electricity; at grid; generation mix" processes
+ # Essentially, calls :func:`olcaschema_genmix`
logging.info("write gen mix to dict")
generation_mix_dict = write_generation_mix_database_to_dict(
generation_mix_df, generation_process_dict,
@@ -148,7 +158,7 @@ def run_distribution(generation_process_dict):
# True for ELCI_1 & ELCI_2 (not ELCI_3)
dist_dict = run_net_trade(generation_mix_dict)
else:
- # ELC1_3
+ # ELCI_3
# NOTE: replace eGRID configuration must be true
# BUG: keyerror in fill_default_provider_uuids in utils.py
dist_dict = run_epa_trade(
@@ -195,10 +205,10 @@ def run_generation():
upstream_dict = {}
upstream_df = None
- # NOTE: This method triggers an input request for EPA data API key;
- # see https://github.com/USEPA/ElectricityLCI/issues/207
+ # NOTE: This method triggers an input request for EPA data via API key;
+ # see https://github.com/NETL-RIC/ElectricityLCI/issues/207
# NOTE: This method runs aggregation and emission uncertainty
- # calculations.
+ # calculations, which may add 15 minutes to run time.
# NOTE: Will import generation.py, which triggers a lot data into memory.
logging.info("get aggregated generation process")
generation_process_df = get_generation_process_df(
@@ -252,10 +262,10 @@ def run_generation():
# Execute main; make is_set true in this block.
try:
- main(True)
+ main()
#get_facility_level_inventory(True, False)
- except Exception as e:
- log.error("Crashed on main!\n%s" % repr(e))
+ except Exception:
+ log.exception("Crashed on main!")
else:
log.info(
"Finished!\n"
diff --git a/electricitylci/manual_edits.py b/electricitylci/manual_edits.py
index f82478b..529672a 100644
--- a/electricitylci/manual_edits.py
+++ b/electricitylci/manual_edits.py
@@ -38,9 +38,9 @@
For details on the issues that these manual edits fix, see the following
on GitHub:
-- https://github.com/USEPA/ElectricityLCI/issues/77
-- https://github.com/USEPA/ElectricityLCI/issues/121
-- https://github.com/USEPA/ElectricityLCI/issues/160
+- https://github.com/NETL-RIC/ElectricityLCI/issues/77
+- https://github.com/NETL-RIC/ElectricityLCI/issues/121
+- https://github.com/NETL-RIC/ElectricityLCI/issues/160
Referenced by 'create_generation_process_df' in generation.py.
diff --git a/electricitylci/model_config.py b/electricitylci/model_config.py
index cb22918..af13b80 100644
--- a/electricitylci/model_config.py
+++ b/electricitylci/model_config.py
@@ -18,6 +18,8 @@
from electricitylci.globals import COAL_MODEL_YEARS
from electricitylci.globals import RENEWABLE_VINTAGES
from electricitylci.globals import NG_MODEL_YEARS
+from electricitylci.globals import REM_WEIGHT_METHODS
+from electricitylci.globals import NEG_REM_METHODS
##############################################################################
@@ -35,7 +37,7 @@
package. To change configuration settings, restart Python.
Last edited:
- 2025-05-13
+ 2025-12-12
"""
__all__ = [
"ConfigurationError",
@@ -144,6 +146,19 @@ class ModelSpecs:
located by default in the output directory (see globals.py).
ng_model_year : int
The natural gas model year (e.g., 2016 or 2020).
+ add_residual_mix : bool
+ Whether to include residual electricity mix processes in JSON-LD.
+ output_residual_mix : bool
+ Whether to save the residual mix data as CSV in output folder.
+ add_rem_product_systems : bool
+ Whether to create "at user; residual consumption mix" product systems.
+ rem_weight_method : str
+ The state-to-balancing authority weighting method (e.g., by facility
+ 'count' or by facility 'gen' weights).
+ neg_rem_method : str
+ The method to deal with negative renewable electricity generation
+ (e.g., if REC sales in a BA are greater than renewable electricity generation); choose either to 'zero' excess or 'keep' excess and
+ attempt to subtract from vague fuel categories (e.g., MIXED or OTHER).
"""
def __init__(self, model_specs, model_name):
"""Class initialization.
@@ -200,6 +215,14 @@ def __init__(self, model_specs, model_name):
self.gen_mix_from_model_generation_data = False
self.calculate_uncertainty = model_specs.get(
"calculate_uncertainty", True)
+ self.add_residual_mix = model_specs.get("add_residual_mix", False)
+ self.add_rem_product_systems = model_specs.get(
+ "add_rem_product_systems", False
+ )
+ self.output_residual_mix = model_specs.get("output_residual_mix", False)
+ # Use empty string rather than crash b/c not implemented in all YAMLs
+ self.rem_weight_method = model_specs.get("rem_weight_method", "")
+ self.neg_rem_method = model_specs.get("neg_rem_method", "")
self.namestr = (
f"{output_dir}/{model_name}_jsonld_"
f"{datetime.datetime.now().strftime('%Y%m%d_%H%M%S')}.zip"
@@ -330,22 +353,46 @@ def check_model_specs(model_specs):
"will not import correctly."
)
if not model_specs['coal_model_year'] in COAL_MODEL_YEARS:
- err_str = "The coal model year must be one of "
+ err_str = "The coal model year must be one of: "
err_str += " or ".join([str(x) for x in COAL_MODEL_YEARS])
- err_str += " not %s!" % model_specs['coal_model_year']
+ err_str += "; not '%s'!" % model_specs['coal_model_year']
raise ConfigurationError(err_str)
-
+
if not model_specs['renewable_vintage'] in RENEWABLE_VINTAGES:
- err_str = "The renewable inventory vintage must be one of "
+ err_str = "The renewable inventory vintage must be one of: "
err_str += " or ".join([str(x) for x in RENEWABLE_VINTAGES])
- err_str += " not %s!" % model_specs['renewable_vintage']
+ err_str += "; not '%s'!" % model_specs['renewable_vintage']
raise ConfigurationError(err_str)
-
+
if not model_specs['ng_model_year'] in NG_MODEL_YEARS:
- err_str = "The natural gas model year must be one of "
+ err_str = "The natural gas model year must be one of: "
err_str += " or ".join([str(x) for x in NG_MODEL_YEARS])
- err_str += " not %s!" % model_specs['ng_model_year']
+ err_str += "; not '%s'!" % model_specs['ng_model_year']
raise ConfigurationError(err_str)
-
- logging.info("Checks passed!")
+ if model_specs['add_rem_product_systems'] and (
+ not model_specs['add_residual_mix']):
+ raise ConfigurationError(
+ "Residual mix product systems cannot be created unless "
+ "`add_residual_mix` is set to true!"
+ )
+ if model_specs['output_residual_mix'] and (
+ not model_specs['add_residual_mix']):
+ raise ConfigurationError(
+ "Residual mix data cannot be generated unless "
+ "`add_residual_mix` is set to true!"
+ )
+ if model_specs['add_residual_mix']:
+ if not model_specs['rem_weight_method'] in REM_WEIGHT_METHODS:
+ err_str = "The residual mix weighting method must be one of: "
+ err_str += " or ".join([x for x in REM_WEIGHT_METHODS])
+ err_str += "; not '%s'!" % model_specs['rem_weight_method']
+ raise ConfigurationError(err_str)
+
+ if not model_specs['neg_rem_method'] in NEG_REM_METHODS:
+ err_str = "The negative renewable allocation method must be one of: "
+ err_str += " or ".join([x for x in NEG_REM_METHODS])
+ err_str += "; not '%s'!" % model_specs['neg_rem_method']
+ raise ConfigurationError(err_str)
+
+ logging.info("Checks passed!")
diff --git a/electricitylci/modelconfig/ELCI_2022_config.yml b/electricitylci/modelconfig/ELCI_2022_config.yml
index 76046f1..161bfb7 100644
--- a/electricitylci/modelconfig/ELCI_2022_config.yml
+++ b/electricitylci/modelconfig/ELCI_2022_config.yml
@@ -68,6 +68,7 @@ inventories_of_interest:
stewicombo_file: 'ELCI_2022_v1.1.4'
# Provide uncertainty estimates for emissions.
+# May add 15 minutes to aggregation step.
calculate_uncertainty: true
@@ -157,3 +158,30 @@ run_post_processes: true
# OTHER PARAMETERS
+# For residual electricity grid mix processes, set `add_residual_mix` option
+# to true. Residual mixes subtract annual voluntary renewable electric
+# certificate sales from generation mixes based on NREL's "Status and Trends
+# in the Voluntary Market" data based on the ``eia_gen_year``. The residual
+# mixes used may be saved in the electricitylci/output folder as a CSV file
+# by setting ``output_residual_mix`` parameter to true. By setting
+# ``add_rem_product_systems`` to true, the "at user; residual consumption mix"
+# processes will be made into a product system.
+add_residual_mix: false
+output_residual_mix: false
+add_rem_product_systems: false
+
+# Option for state-to-BA level aggregation method. Options include:
+# 'count' - uses facility counts that fall within the shared regions
+# of states and balancing authorities.
+# 'gen' - uses facility-level annual generation as weights to the 'count'
+# method.
+rem_weight_method: 'gen'
+
+# Option for handling REC totals that are greater than the renewable
+# energy generated by a balancing authority. Options include:
+# 'keep' - will subtract the excess away from the non-renewable fuel
+# categories (assuming some renewable generation in the 'mix' or
+# 'othf' categories) and maintains the math of generation totals
+# 'zero' - will floor negative renewable energy values to zero; any
+# remainder is not accounted for.
+neg_rem_method: 'zero'
diff --git a/electricitylci/modelconfig/ELCI_2023_config.yml b/electricitylci/modelconfig/ELCI_2023_config.yml
index 17d3eca..183bfc5 100644
--- a/electricitylci/modelconfig/ELCI_2023_config.yml
+++ b/electricitylci/modelconfig/ELCI_2023_config.yml
@@ -8,7 +8,7 @@
# The target year is used to determine the temporal correlation of data with
# the electricity generation processes, which can be used in uncertainty
# calculations.
-electricity_lci_target_year: 2025
+electricity_lci_target_year: 2026
# Select a regional aggregation from "eGRID", "NERC", "BA", "US", "FERC",
# and "EIA". The EPA_eGRID trading method can only be used with "eGRID".
@@ -68,6 +68,7 @@ inventories_of_interest:
stewicombo_file: 'ELCI_2023_v1.2.1_3687292'
# Provide uncertainty estimates for emissions.
+# May add 15 minutes to aggregation step.
calculate_uncertainty: true
@@ -96,12 +97,16 @@ eia_api: ""
# GENERATOR FILTERS
-# These parameters determine if any power plants are filtered out
+# These parameters determine if any power plants are filtered out.
+# First, if true, only facilities with net-zero or greater generation are
+# included (i.e., removes net energy consumers); second, if true, removes
+# outliers if not within the given bounds of efficiency, calculated as the
+# ratio of net generation to total fuel consumption.
include_only_egrid_facilities_with_positive_generation: true
filter_on_efficiency: true
egrid_facility_efficiency_filters:
lower_efficiency: 10
- upper_efficiency: 100
+ upper_efficiency: 100.1
# ELCI creates life cycle processes for each fuel type. If you only want to
# include power plants with a minimum amount of generation from a single fuel,
@@ -155,4 +160,32 @@ NETL_IO_trading_year: 2023
# Product systems for the at-user consumption mixes are also generated.
run_post_processes: true
+
# OTHER PARAMETERS
+# For residual electricity grid mix processes, set `add_residual_mix` option
+# to true. Residual mixes subtract annual voluntary renewable electric
+# certificate sales from generation mixes based on NREL's "Status and Trends
+# in the Voluntary Market" data based on the ``eia_gen_year``. The residual
+# mixes used may be saved in the electricitylci/output folder as a CSV file
+# by setting ``output_residual_mix`` parameter to true. By setting
+# ``add_rem_product_systems`` to true, the "at user; residual consumption mix"
+# processes will be made into a product system.
+add_residual_mix: false
+output_residual_mix: false
+add_rem_product_systems: false
+
+# Option for state-to-BA level aggregation method. Options include:
+# 'count' - uses facility counts that fall within the shared regions
+# of states and balancing authorities.
+# 'gen' - uses facility-level annual generation as weights to the 'count'
+# method.
+rem_weight_method: 'gen'
+
+# Option for handling REC totals that are greater than the renewable
+# energy generated by a balancing authority. Options include:
+# 'keep' - will subtract the excess away from the non-renewable fuel
+# categories (assuming some renewable generation in the 'mix' or
+# 'othf' categories) and maintains the math of generation totals
+# 'zero' - will floor negative renewable energy values to zero; any
+# remainder is not accounted for.
+neg_rem_method: 'zero'
diff --git a/electricitylci/modelconfig/ELCI_2024_config.yml b/electricitylci/modelconfig/ELCI_2024_config.yml
index 4af8286..3bdd339 100644
--- a/electricitylci/modelconfig/ELCI_2024_config.yml
+++ b/electricitylci/modelconfig/ELCI_2024_config.yml
@@ -8,7 +8,7 @@
# The target year is used to determine the temporal correlation of data with
# the electricity generation processes, which can be used in uncertainty
# calculations.
-electricity_lci_target_year: 2025
+electricity_lci_target_year: 2026
# Select a regional aggregation from "eGRID", "NERC", "BA", "US", "FERC",
# and "EIA". The EPA_eGRID trading method can only be used with "eGRID".
@@ -65,9 +65,10 @@ inventories_of_interest:
TRI: 2023
NEI: 2020
RCRAInfo: 2023
-stewicombo_file: 'ELCI_2020_v1.1.4'
+stewicombo_file: 'ELCI_2023_v1.2.1_3687292'
# Provide uncertainty estimates for emissions.
+# May add 15 minutes to aggregation step.
calculate_uncertainty: true
@@ -96,12 +97,16 @@ eia_api: ""
# GENERATOR FILTERS
-# These parameters determine if any power plants are filtered out
+# These parameters determine if any power plants are filtered out.
+# First, if true, only facilities with net-zero or greater generation are
+# included (i.e., removes net energy consumers); second, if true, removes
+# outliers if not within the given bounds of efficiency, calculated as the
+# ratio of net generation to total fuel consumption.
include_only_egrid_facilities_with_positive_generation: true
filter_on_efficiency: true
egrid_facility_efficiency_filters:
lower_efficiency: 10
- upper_efficiency: 100
+ upper_efficiency: 100.1
# ELCI creates life cycle processes for each fuel type. If you only want to
# include power plants with a minimum amount of generation from a single fuel,
@@ -157,3 +162,30 @@ run_post_processes: true
# OTHER PARAMETERS
+# For residual electricity grid mix processes, set `add_residual_mix` option
+# to true. Residual mixes subtract annual voluntary renewable electric
+# certificate sales from generation mixes based on NREL's "Status and Trends
+# in the Voluntary Market" data based on the ``eia_gen_year``. The residual
+# mixes used may be saved in the electricitylci/output folder as a CSV file
+# by setting ``output_residual_mix`` parameter to true. By setting
+# ``add_rem_product_systems`` to true, the "at user; residual consumption mix"
+# processes will be made into a product system.
+add_residual_mix: false
+output_residual_mix: false
+add_rem_product_systems: false
+
+# Option for state-to-BA level aggregation method. Options include:
+# 'count' - uses facility counts that fall within the shared regions
+# of states and balancing authorities.
+# 'gen' - uses facility-level annual generation as weights to the 'count'
+# method.
+rem_weight_method: 'count'
+
+# Option for handling REC totals that are greater than the renewable
+# energy generated by a balancing authority. Options include:
+# 'keep' - will subtract the excess away from the non-renewable fuel
+# categories (assuming some renewable generation in the 'mix' or
+# 'othf' categories) and maintains the math of generation totals
+# 'zero' - will floor negative renewable energy values to zero; any
+# remainder is not accounted for.
+neg_rem_method: 'zero'
diff --git a/electricitylci/natural_gas_upstream.py b/electricitylci/natural_gas_upstream.py
index 1545fcd..4b8e29c 100644
--- a/electricitylci/natural_gas_upstream.py
+++ b/electricitylci/natural_gas_upstream.py
@@ -8,17 +8,17 @@
##############################################################################
import logging
import os
-import sys
import pandas as pd
from electricitylci.globals import data_dir
+from electricitylci.globals import paths
from electricitylci.eia923_generation import eia923_download_extract
import electricitylci.PhysicalQuantities as pq
from electricitylci.generation import add_temporal_correlation_score
from electricitylci.model_config import model_specs
+from electricitylci.elementaryflows import correct_netl_flow_names
from electricitylci.utils import download_edx
-from electricitylci.globals import paths
from electricitylci.utils import check_output_dir
@@ -29,23 +29,28 @@
component of natural gas power plant operation (extraction, processing, and
transportation) for every plant in EIA-923.
-TODO:
-- Make eLCI.csv resource a public submission.
-
Created:
2019-02-18
Last updated:
- 2025-12-04
+ 2026-01-29
"""
__all__ = [
+ "generate_lci",
"generate_upstream_ng",
+ "get_ng_lci",
+ "map_ng_by_basin",
+ "map_ng_by_region",
+ "map_ng_lci_to_plants_by_basin",
+ "map_ng_lci_to_plants_by_region",
+ "read_region_data",
+ "save_ng_lci",
]
##############################################################################
# GLOBALS
##############################################################################
-region_sheets_dict = {
+REGION_SHEETS_DICT = {
'Pacific': 'FI - Pacific Delivery',
'Rocky Mountain': 'FI - Rocky Mountain Delivery',
'Southwest': 'FI - Southwest Delivery',
@@ -55,7 +60,7 @@
}
'''dict : Region names mapped to Excel workbook sheet names.'''
-r_ids_2020 = {
+R_IDS_2020 = {
'Appendix_F_2020_Full_Inventory_Results_Midwest_ProdThruTrans.xlsx':'5665de40-fc2b-4643-b647-ceec226af2bb',
'Appendix_F_2020_Full_Inventory_Results_Northeast_ProdThruTrans.xlsx' :'b396eb50-72ac-45f0-8231-9b613457c6d8',
'Appendix_F_2020_Full_Inventory_Results_Pacific_ProdThruTrans.xlsx' :'347a0cd8-5ff2-4cb3-be0a-f31a56bac9c6',
@@ -65,7 +70,7 @@
}
'''dict : Excel workbook file names mapped to EDX resource IDs.'''
-region_state_mapping = {
+REGION_STATE_MAPPING = {
'WA':'Pacific','CA':'Pacific','OR':'Pacific','MT':'Rocky Mountain','ID':'Rocky Mountain','CO':'Rocky Mountain','NV':'Rocky Mountain','UT':'Rocky Mountain','WY':'Rocky Mountain',
'AZ':'Southwest','NM':'Southwest','OK':'Southwest','TX':'Southwest','MN':'Midwest','ND':'Midwest','IA':'Midwest','KS':'Midwest',
'MO':'Midwest','NE':'Midwest','SD':'Midwest','IL':'Midwest','IN':'Midwest','OH':'Midwest','WI':'Midwest','MI':'Midwest',
@@ -79,122 +84,7 @@
##############################################################################
# FUNCTIONS
##############################################################################
-def correct_netl_flow_names(df, flow_mapping_path, amount_col="FlowAmount"):
- """A helper method that replaces NETL air, water, and ground emissions
- with Federal Elementary Flow List equivalents based on a subset of
- flows defined in USEPA's eLCI mapping using the Python package
- `fedelemflowlist `_
-
- Parameters
- ----------
- df : pandas.DataFrame
- A life cycle inventory data frame with columns, 'FlowName',
- 'Compartment', 'Unit', and ``amount_col``.
- amount_col : str, optional
- The column title representing the flow amount, by default "FlowAmount"
-
- Returns
- -------
- pandas.DataFrame
- A new data frame with the same number of rows and columns as the
- sent data frame. Flow names, compartments, units, and flow amounts
- are updated based on emissions matches with the FEDEFL. All unmatched
- flows are returned 'as is'. If FlowUUID was not in the column list,
- it is created; otherwise, the matched UUIDs are updated.
- """
- # This data frame has about 4k source flow names and contexts associated
- # with NETL unit process models (e.g., petro, nuclear, coal).
- flow_mapping = pd.read_csv(flow_mapping_path, encoding='ISO-8859-1')
-
- # Matching occurs on name, compartment and units; help this along by
- # lowering the case (improves coal UP matches from 10% to 42%).
- df["FlowName_orig"] = df["FlowName"]
- df["Compartment_orig"] = df["Compartment"]
- df["FlowName"] = df["FlowName"].str.lower().str.rstrip()
- df["Compartment"] = df["Compartment"].str.lower().str.rstrip()
-
- # In the map, also lower-case names and compartments and remove trailing
- # space; note this introduces duplicate entries in the map, so remove them.
- # The duplicates are from later entries, so ignore mapper, verifier and
- # last updated cols when searching for duplicates. [250917; TWD]
- flow_mapping['SourceFlowName'] = flow_mapping[
- 'SourceFlowName'].str.lower().str.rstrip()
- flow_mapping['SourceFlowContext'] = flow_mapping[
- "SourceFlowContext"].str.lower().str.rstrip()
- ignore_cols = ['Mapper', 'Verifier', 'LastUpdated']
- flow_mapping = flow_mapping.drop_duplicates(
- subset=[x for x in flow_mapping.columns if x not in ignore_cols]
- )
-
- # Some compartments in NETL UPs are complex (e.g., 'Emission to water/fresh
- # water'), but are listed simply in the FEDEFL eLCI mapper (e.g., 'emission/
- # water'). Improves coal mining UP matches from 42% to 62%.
- is_emission = df['input'] == False
- is_water = df['Compartment'].str.contains('water')
- is_air = df['Compartment'].str.contains('air')
- is_ground = df['Compartment'].str.contains('ground')
-
- df.loc[is_emission * is_water, 'Compartment'] = 'emission/water'
- df.loc[is_emission * is_air, 'Compartment'] = 'emission/air'
- df.loc[is_emission * is_ground, 'Compartment'] = 'emission/ground'
-
- # HOTFIX: Map against source units [250205; TWD]
- # For coal mining, reduces matches from >62% to <62% (about 2k less rows)
- logging.info("Mapping emissions to FEDEFL")
- mapped_df = pd.merge(
- df,
- flow_mapping,
- left_on=["FlowName", "Compartment", "Unit"],
- right_on=["SourceFlowName", "SourceFlowContext", "SourceUnit"],
- how="left",
- )
-
- # If TargetFlowName is present, there was a match.
- is_match = mapped_df["TargetFlowName"].notnull()
- logging.info("Correcting %d NETL flows" % is_match.sum())
-
- # Quality Check (coal_df)
- # Check that target unit matches source unit.
- # No! Hydrogen, Uranium, and Lead-210/kg have mis-matched units.
- # Therefore, unit conversions are necessary.
-
- # Return flow names and compartments back to their original values.
- df["FlowName"] = df["FlowName_orig"]
- df["Compartment"] = df["Compartment_orig"]
- del df['FlowName_orig'] # use this syntax since you're editing
- del df['Compartment_orig'] # a reference object that isn't returned
- mapped_df['FlowName'] = mapped_df['FlowName_orig']
- mapped_df["Compartment"] = mapped_df["Compartment_orig"]
- mapped_df = mapped_df.drop(columns=['FlowName_orig', 'Compartment_orig'])
-
- # Replace FlowName, Unit, and Compartment with new names (where matched)
- mapped_df.loc[is_match, "FlowName"] = mapped_df.loc[
- is_match, "TargetFlowName"]
- mapped_df.loc[is_match, "Compartment"] = mapped_df.loc[
- is_match, "TargetFlowContext"]
- mapped_df.loc[is_match, "Unit"] = mapped_df.loc[is_match, "TargetUnit"]
-
- # Correct values using the conversion factor
- mapped_df.loc[is_match, amount_col] *= mapped_df.loc[
- is_match, 'ConversionFactor']
-
- if 'FlowUUID' in mapped_df.columns:
- # Update existing values with new UUIDs
- mapped_df.loc[is_match, 'FlowUUID'] = mapped_df.loc[
- is_match, 'TargetFlowUUID']
- else:
- # Set UUIDs to target values
- mapped_df = mapped_df.rename(columns={"TargetFlowUUID": "FlowUUID"})
-
- # Drop all unneeded cols
- drop_cols = [x for x in flow_mapping.columns if x in mapped_df.columns]
- mapped_df = mapped_df.drop(columns=drop_cols)
-
- return mapped_df
-
-
def generate_lci(excel_folder_path,
- flow_mapping_path,
destination_path,
final_table_name):
"""
@@ -205,8 +95,6 @@ def generate_lci(excel_folder_path,
excel_folder_path : str
The path to the folder containing the excel files (i.e., NG models and
inventories).
- flow_mapping_path: str
- The path to the flow mapping file.
destination_path : str, optional
The path to the destination folder. If not provided, the function
will save the file in the current working directory.
@@ -224,6 +112,9 @@ def generate_lci(excel_folder_path,
-----
The function is sensitive to the naming convention of the regions in the
Excel file.
+
+ Updated to utilize the :func:`correct_netl_flow_names` found in
+ elementaryflows.py.
"""
final_table = pd.DataFrame()
@@ -234,20 +125,25 @@ def generate_lci(excel_folder_path,
logging.info(f"Reading file: {file_path}")
input_data = pd.ExcelFile(file_path)
sheet_names = input_data.sheet_names
- sheet_name = [
- name for name in sheet_names if name in region_sheets_dict.values()
- ][0]
+ try:
+ sheet_name = [
+ name for name in sheet_names if name in REGION_SHEETS_DICT.values()
+ ][0]
+ except IndexError as e:
+ err_str = (
+ "Failed to find named worksheet in %s. Check your "
+ "workbook against REGION_SHEETS_DICT keys." % filename
+ )
+ logging.error(err_str)
+ raise IndexError(err_str)
# Extract air, water, and ground emissions data for the selected sheet
- # (i.e., technobasin).
air_emissions_data, water_emissions_data, ground_emissions_data = read_region_data(file_path, sheet_name)
# Air emissions
# - Get the correct flow names, compartment, and uuid for each flow
- full_air_emissions_data = correct_netl_flow_names(
- air_emissions_data,
- flow_mapping_path
- )
+ logging.info("Processing natural gas air emissions")
+ full_air_emissions_data = correct_netl_flow_names(air_emissions_data)
# Drop rows with FlowUUID NaN.
full_air_emissions_data = full_air_emissions_data[
full_air_emissions_data['FlowUUID'].notna()
@@ -255,9 +151,9 @@ def generate_lci(excel_folder_path,
# Water emissions
# - get the correct flow names, compartment, and uuid for each flow.
+ logging.info("Processing natural gas water emissions")
full_water_emissions_data = correct_netl_flow_names(
- water_emissions_data,
- flow_mapping_path
+ water_emissions_data
)
# Drop rows with FlowUUID NaN.
full_water_emissions_data = full_water_emissions_data[
@@ -266,9 +162,9 @@ def generate_lci(excel_folder_path,
# Ground emissions
# - get the correct flow names, compartment, and uuid for each flow
+ logging.info("Processing natural gas ground emissions")
full_ground_emissions_data = correct_netl_flow_names(
- ground_emissions_data,
- flow_mapping_path
+ ground_emissions_data
)
full_ground_emissions_data = full_ground_emissions_data[
full_ground_emissions_data['FlowUUID'].notna()
@@ -282,7 +178,7 @@ def generate_lci(excel_folder_path,
])
df1 = df1.sort_values(by='FlowUUID')
region = [
- key for key, v in region_sheets_dict.items() if v == sheet_name
+ key for key, v in REGION_SHEETS_DICT.items() if v == sheet_name
][0]
df1['FlowAmount'] = df1['FlowAmount'].astype(float)
df1['FlowAmount'] = df1['FlowAmount'].fillna(0)
@@ -312,26 +208,30 @@ def generate_lci(excel_folder_path,
'is_input'
]
# Add a column for each basin
- region_columns = list(region_sheets_dict.keys())
+ region_columns = list(REGION_SHEETS_DICT.keys())
for r in region_columns:
final_table[r] = 0
# Add region emissions to final table
try:
logging.info(f"Adding emissions for {region}")
- logging.info(f"df1: {df1['FlowAmount'].head(5)}")
+ logging.debug(f"df1: {df1['FlowAmount'].head(5)}")
final_table[region] = df1['FlowAmount']
except Exception as e:
- sys.exit(
+ err_str = (
"Error reading sheet. "
"Make sure your excel file follows the correct naming "
"convention. For reference, refer to the source code, "
f"lines 70-78. Error: {e}"
)
+ logging.error(err_str)
+ raise IOError(err_str)
# 2. Save final table to excel
save_ng_lci(final_table, final_table_name ,destination_path)
- print(f"Final table saved to {destination_path}/{final_table_name}.xlsx")
+ logging.info(
+ f"Final table saved to {destination_path}/{final_table_name}.xlsx"
+ )
return final_table
@@ -372,7 +272,7 @@ def generate_upstream_ng(year):
# assignment newer data (2020) is available by region plants are connected
# to upstream ng emissions via region assignment
- # 'year' refers to eia_gen_year
+ # NOTE: 'year' parameter refers to eia_gen_year
if model_specs.ng_model_year == 2016:
ng_generation_data_mapped = map_ng_by_basin(year)
else:
@@ -381,11 +281,10 @@ def generate_upstream_ng(year):
# Read the NG LCI file
# If year = 2016
# - this step will directly ready NG_LCI.csv from the data_dir
- # - returns lci (by basin)
+ # - returns LCI (by basin)
# If year = 2020
- # - this step will require edx api, download ng model and mapping
- # - returns lci (by region)
- # Document from edx, and generate lci
+ # - this step requires EDX API (to download ng model) and mapping
+ # - returns LCI (by region)
ng_lci = get_ng_lci(model_specs.ng_model_year)
# merge ng lci and plants based on the common parameter: region or basin
@@ -456,8 +355,8 @@ def get_ng_lci(year):
Get the natural gas life cycle inventory for a given year.
Depending on the year, the natural gas life cycle inventory is either:
- - retrieved from existing data
- - calculated using the natural gas life cycle inventory model
+ - retrieved from existing data (e.g., 2016)
+ - calculated using the natural gas life cycle inventory model (e.g., 2020)
Parameters
----------
@@ -477,7 +376,7 @@ def get_ng_lci(year):
- the NG_LCI CSV file (if the old model is selected in the configuration)
- the EDX API (if the new model is selected in the configuration)
- - the elci flow mapping CSV file (if the new model is selected in the
+ - the eLCI flow mapping CSV file (if the new model is selected in the
configuration)
"""
if isinstance(year, int):
@@ -498,7 +397,7 @@ def get_ng_lci(year):
# Check if the ng_lci_2020rev1.csv already exists
# - if it does then we can skip all the below
if os.path.exists(os.path.join(data_folder, "ng_lci_2020rev1.csv")):
- logging.info(f"NG LCI already exists in your data directory.")
+ logging.info("NG LCI already exists in your data directory.")
ng_lci = pd.read_csv(
os.path.join(data_folder, "ng_lci_2020rev1.csv"),
index_col=[0,1,2,3,4,5]
@@ -514,7 +413,7 @@ def get_ng_lci(year):
# - check if model is data_dir
check_output_dir(os.path.join(data_folder, "2020_ng_model"))
model_folder = os.path.join(data_folder, "2020_ng_model")
- for ngmodel in r_ids_2020.keys():
+ for ngmodel in R_IDS_2020.keys():
if os.path.exists(os.path.join(model_folder, ngmodel)):
logging.info(
f"{ngmodel} already exists in your data directory."
@@ -523,7 +422,7 @@ def get_ng_lci(year):
logging.info(f"Downloading {ngmodel} from EDX.")
try:
download_edx(
- resource_id=r_ids_2020[ngmodel],
+ resource_id=R_IDS_2020[ngmodel],
api_key=model_specs.edx_api_key,
output_dir=model_folder
)
@@ -531,43 +430,11 @@ def get_ng_lci(year):
logging.error(
f"Error downloading {ngmodel} from EDX. Error: {e}"
)
- sys.exit(1)
- # Retrieve flow mapping document from EDX, eLCI.csv, and check if
- # flow mapping CSV exists in data_dir.
- if os.path.exists(os.path.join(data_folder, "eLCI.csv")):
- logging.info(
- "ELCI flow mapping document already exists in your "
- "data directory."
- )
- flow_mapping_path = os.path.join(data_folder, "eLCI.csv")
- else:
- # Download flow mapping document from EDX.
- logging.info(
- "Downloading ELCI flow mapping document from EDX."
- )
- # Resource id of eLCI flow mapping document on EDX
- # NOTE: Currently in Life Cycle Collaborations Workspace
- # ---not public!!!
- r_id_elci = 'e2c8f934-e95e-470a-879b-17ebe4afd39e'
- try:
- download_edx(
- resource_id=r_id_elci,
- api_key=model_specs.edx_api_key,
- output_dir=data_folder
- )
- flow_mapping_path = os.path.join(data_folder, "eLCI.csv")
- except Exception as e:
- logging.error(
- "Error downloading ELCI flow mapping document from "
- f"EDX. Error: {e}"
- )
- sys.exit(1)
# Run the generate_ng_lci function and save it in data_dir.
try:
generate_lci(
excel_folder_path=model_folder,
- flow_mapping_path=flow_mapping_path,
destination_path=data_folder,
final_table_name="ng_lci_2020rev1"
)
@@ -576,11 +443,12 @@ def get_ng_lci(year):
index_col=[0,1,2,3,4,5]
)
except Exception as e:
- logging.error(
+ err_str = (
"Error generating natural gas life cycle inventory. "
f"Error: {e}"
)
- sys.exit(1)
+ logging.error(err_str)
+ raise IOError(err_str)
return ng_lci
@@ -636,10 +504,10 @@ def map_ng_by_basin(year):
# Merge with ng_generation dataframe.
ng_generation_data_basin = pd.merge(
- left = ng_generation_data,
- right = ng_basin_mapping,
- left_on = 'Plant Id',
- right_on = 'Plant Code'
+ left=ng_generation_data,
+ right=ng_basin_mapping,
+ left_on='Plant Id',
+ right_on='Plant Code'
)
ng_generation_data_basin = ng_generation_data_basin.drop(
columns=['Plant Code']
@@ -649,8 +517,8 @@ def map_ng_by_basin(year):
def map_ng_by_region(year):
"""
- Map the natural gas generation data by region.
- This includes 6 regions: Pacific, Rocky Mountain, Southwest, Midwest,
+ Map the natural gas inventory data by downstream delivery region.
+ This includes six regions: Pacific, Rocky Mountain, Southwest, Midwest,
Southeast, and Northeast.
Notes
@@ -660,7 +528,7 @@ def map_ng_by_region(year):
consumption.
- Groups the data by Plant Id and aggregates the fuel consumption by
summing the total fuel consumption.
- - Maps each plant to a region using the region_state_mapping dictionary.
+ - Maps each plant to a region using the REGION_STATE_MAPPING dictionary.
Parameters
----------
@@ -688,14 +556,15 @@ def map_ng_by_region(year):
ng_generation_data_region = ng_generation_data.copy()
- ng_generation_data_region['NG_LCI_Region'] = ng_generation_data['State'].map(region_state_mapping)
+ ng_generation_data_region['NG_LCI_Region'] = ng_generation_data[
+ 'State'].map(REGION_STATE_MAPPING)
return ng_generation_data_region
def map_ng_lci_to_plants_by_basin(ng_lci, ng_generation_data_mapped):
"""
- Map the natural gas generation data by basin.
+ Map the natural gas inventory data by upstream production technobasin.
"""
ng_lci_columns=[
"Compartment",
@@ -708,14 +577,14 @@ def map_ng_lci_to_plants_by_basin(ng_lci, ng_generation_data_mapped):
"FlowAmount"
]
ng_lci_stack = pd.DataFrame(ng_lci.stack()).reset_index()
- ng_lci_stack.columns=ng_lci_columns
+ ng_lci_stack.columns = ng_lci_columns
# Merge basin data with LCI dataset
ng_lci_mapped = pd.merge(
ng_lci_stack,
ng_generation_data_mapped,
- left_on = 'Basin',
- right_on = 'NG_LCI_Name',
+ left_on='Basin',
+ right_on='NG_LCI_Name',
how='left'
)
return ng_lci_mapped
@@ -725,7 +594,7 @@ def map_ng_lci_to_plants_by_region(ng_lci, ng_generation_data_mapped):
"""
Map the natural gas generation data by basin.
"""
- ng_lci_columns=[
+ ng_lci_columns = [
"Compartment",
"FlowName",
"FlowUUID",
@@ -736,19 +605,60 @@ def map_ng_lci_to_plants_by_region(ng_lci, ng_generation_data_mapped):
"FlowAmount"
]
ng_lci_stack = pd.DataFrame(ng_lci.stack()).reset_index()
- ng_lci_stack.columns=ng_lci_columns
+ ng_lci_stack.columns = ng_lci_columns
# Merge basin data with LCI dataset
ng_lci_mapped = pd.merge(
ng_lci_stack,
ng_generation_data_mapped,
- left_on = 'Region',
- right_on = 'NG_LCI_Region',
+ left_on='Region',
+ right_on='NG_LCI_Region',
how='left'
)
return ng_lci_mapped
+def _proc_region_data(df):
+ """Implementation of DRY (don't repeat yourself) for repetitive methods
+ applied to air, water, and ground emissions data frames in
+ :func:`read_region_data`. This method sets the column headers to the
+ LCA stages, drops the LCA stages and stats rows, which are text-based,
+ drops rows that are all NaNs (e.g., with soil and water), then reconstructs
+ the data frame so that the emissions columns are numeric (floats) and the
+ emissions names are strings (or objects), complying with new pandas 3.0.
+
+ Parameters
+ ----------
+ df : pandas.DataFrame
+ A natural gas compartmentalized emissions table (e.g., air, water,
+ or soil), as created within :func:`read_region_data`
+
+ Returns
+ -------
+ pandas.DataFrame
+ The same data frame that was received. See description for what's
+ changed.
+ """
+ # Set columns to unique headers (lca stages)
+ df.columns = df.iloc[0]
+
+ # Drop the textual rows (i.e., the stage names and statistic name).
+ df = df.drop(df.index[0:2])
+
+ # Remove lingering rows with NaNs (thanks, Excel)
+ df = df.dropna(how='all')
+
+ # Reconstruct the data frame with emission names and floating point values
+ df = pd.concat(
+ [
+ df.iloc[:, [0]],
+ df.iloc[:, 1:].astype(float)
+ ],
+ axis=1
+ )
+ return df
+
+
def read_region_data(excel_file_path, sheet_name):
"""
Read Excel file, extract data, and generate a data frame for NG emissions
@@ -771,9 +681,7 @@ def read_region_data(excel_file_path, sheet_name):
- pandas.DataFrame, the water emissions data
- pandas.DataFrame, the ground emissions data
"""
- print(f"Processing sheet: {sheet_name}")
- # create empty database
- df = pd.DataFrame()
+ logging.info(f"Processing sheet: {sheet_name}")
# Extract all the data from the sheet
df = pd.read_excel(
excel_file_path,
@@ -782,7 +690,13 @@ def read_region_data(excel_file_path, sheet_name):
header=None
)
- # Adjustments: 1) changing header, 2) dropping P2.5 and P97.5 columns
+ # HOTFIX: set all column data types to str for pandas 3.0 [26.01.30;TWD]
+ df = df.astype(str)
+
+ # HOTFIX: in pandas 2.3, NaNs become literal 'nan', which won't ffill()!
+ df = df.replace('nan', None)
+
+ # Adjustments: 1) change header, 2) drop all P2.5 and P97.5 columns
df.iloc[0] = df.iloc[0].ffill()
df.iloc[1] = df.iloc[1].ffill()
df.columns = df.iloc[2]
@@ -794,41 +708,54 @@ def read_region_data(excel_file_path, sheet_name):
# FEDEFL elementary flows
# Air emissions
+ # Drop all columns that aren't labeled as 'air' (Cell A1)
air_emissions_data = df.drop(
columns=[col for col in df.columns if col != df.columns[1]]
)
# Drop the last two columns (empty columns from excel)
air_emissions_data = air_emissions_data.iloc[:, :-2]
- # Sum columns 2:11 for each row
- air_emissions_data[f'FlowAmount'] = air_emissions_data.iloc[:, 1:11].sum(
+
+ # At this point, we have air emissions where the first column is the
+ # emission name, and the following columns are the mean amounts across
+ # the life cycle stages (e.g., production, gathering, processing, storage)
+
+ air_emissions_data = _proc_region_data(air_emissions_data)
+
+ # Sum across columns for each row
+ air_emissions_data[f'FlowAmount'] = air_emissions_data.iloc[:, 1:].sum(
axis=1
)
- air_emissions_data = air_emissions_data.iloc[2:]
+
+ # Save only 'Emission' and 'FlowAmount' columns.
air_emissions_data = air_emissions_data.iloc[:, [0,-1]]
+
+ # Add metadata
air_emissions_data['Compartment'] = 'Air'
air_emissions_data.columns.values[0] = 'FlowName' # change header
air_emissions_data['Unit'] = 'kg'
- air_emissions_data ['input'] = False # not an input
+ air_emissions_data['input'] = False # not an input
# Water emissions
+ # Grab two of the last three columns (emissions and water amounts)
water_emissions_data = df.iloc[:, [df.shape[1]-3, df.shape[1]-1]]
+ water_emissions_data = _proc_region_data(water_emissions_data)
+
water_emissions_data.columns.values[0] = "FlowName"
water_emissions_data.columns.values[1] = "FlowAmount"
- water_emissions_data = water_emissions_data.iloc[2:]
- water_emissions_data = water_emissions_data.dropna()
water_emissions_data['Compartment'] = 'Water'
water_emissions_data['Unit'] = 'kg'
- water_emissions_data ['input'] = False
+ water_emissions_data['input'] = False
# Ground emissions
+ # Grab two of the last three columns (emission and ground amounts)
ground_emissions_data = df.iloc[:, [df.shape[1]-3, df.shape[1]-2]]
+ ground_emissions_data = _proc_region_data(ground_emissions_data)
+
ground_emissions_data.columns.values[0] = "FlowName"
ground_emissions_data.columns.values[1] = "FlowAmount"
- ground_emissions_data = ground_emissions_data.dropna()
- ground_emissions_data = ground_emissions_data.iloc[1:]
ground_emissions_data['Compartment'] = 'Ground'
ground_emissions_data['Unit'] = 'kg'
- ground_emissions_data ['input'] = False
+ ground_emissions_data['input'] = False
return air_emissions_data, water_emissions_data, ground_emissions_data
@@ -849,7 +776,12 @@ def save_ng_lci(df, filename, destination_path):
# MAIN
##############################################################################
if __name__=='__main__':
- from electricitylci.globals import output_dir
- year = 2016
+ from electricitylci.utils import get_logger
+ import electricitylci.model_config as config
+
+ log = get_logger(True, False)
+ config.model_specs = config.build_model_class("ELCI_2023")
+
+ from electricitylci.natural_gas_upstream import generate_upstream_ng
+ year = 2023
df = generate_upstream_ng(year)
- df.to_csv(output_dir+'/ng_emissions_{}.csv'.format(year))
diff --git a/electricitylci/nuclear_upstream.py b/electricitylci/nuclear_upstream.py
index 33023c2..d0b2585 100644
--- a/electricitylci/nuclear_upstream.py
+++ b/electricitylci/nuclear_upstream.py
@@ -15,6 +15,9 @@
from electricitylci.eia923_generation import eia923_download_extract
from electricitylci.generation import add_temporal_correlation_score
from electricitylci.model_config import model_specs
+from electricitylci.utils import filter_out_zero
+
+
##############################################################################
# MODULE DOCUMENTATION
##############################################################################
@@ -22,10 +25,19 @@
extraction, processing, and transportation of uranium for each nuclear plant
in EIA-923.
+The upstream nuclear life cycle inventory is based on NETL unit processes,
+which were updated, and ecoinvent inventory (where NETL UPs were not
+available). The cradle-to-grave supply chain was modeled in Sphera's GaBi LCA
+software. Stage 1 (EU and US uranium enrichment chain) and Stage 2 (fuel
+assembly transport) were rolled up into a system process, scaled based on 1 MWh
+of nuclear power generated. This inventory is saved in the nuclear_lci.csv
+provided in the ElectricityLCI repository's data folder and represents
+activities conducted in 2020.
+
Created:
2019-05-31
Last updated:
- 2024-01-10
+ 2024-02-24
"""
__all__ = [
"generate_upstream_nuc",
@@ -43,7 +55,7 @@ def generate_upstream_nuc(year):
Notes
-----
Depends on data file, nuclear_lci.csv, which contains the upstream
- emission impacts of a kilogram of uranium.
+ emission impacts associated with 1 MWh of nuclear fuel, at plant.
Parameters
----------
@@ -56,32 +68,43 @@ def generate_upstream_nuc(year):
"""
# Get the EIA generation data for the specified year, this dataset includes
# the fuel consumption for generating electricity for each facility
- # and fuel type. Filter the data to only include NG facilities and on
- # positive fuel consumption. Group that data by Plant Id as it is possible
- # to have multiple rows for the same facility and fuel based on different
- # prime movers (e.g., gas turbine and combined cycle).
+ # and fuel type.
eia_generation_data = eia923_download_extract(year)
+ # Filter the data on NUC facilities and on positive fuel consumption.
+ # NOTE:
+ # - 62 unique plants in 2016.
+ # - 58 unique plants in 2020.
+ # - 56 unique plants in 2021.
+ # - 55 unique plants in 2022.
+ # - 54 unique plants in 2023.
column_filt = (eia_generation_data["Reported Fuel Type Code"] == "NUC") & (
eia_generation_data["Net Generation (Megawatthours)"] > 0
)
nuc_generation_data = eia_generation_data[column_filt]
- nuc_generation_data = (
- nuc_generation_data.groupby("Plant Id")
- .agg({"Net Generation (Megawatthours)": "sum"})
- .reset_index()
- )
+ # Group data by Plant Id in case multiple rows exist for the same facility
+ # and fuel (e.g., based on different prime movers.
+ # NOTE: This was not identified in years 2016, 2020--2023 [26.02.18; TWD]
+ nuc_generation_data = nuc_generation_data.groupby("Plant Id").agg(
+ {"Net Generation (Megawatthours)": "sum"}).reset_index()
+
nuc_generation_data["Plant Id"] = nuc_generation_data[
"Plant Id"].astype(int)
# Read the nuclear LCI file
+ # NOTE: 2016 LCI has 1772 rows where 421 rows have zero flow amount.
nuc_lci = pd.read_csv(
os.path.join(data_dir, "nuclear_lci.csv"),
index_col=0,
low_memory=False
)
- nuc_lci.dropna(subset=["compartment"],inplace=True)
+
+ # HOTFIX: remove zero-flows from LCI [26.02.24; TWD]
+ nuc_lci = filter_out_zero(nuc_lci, 'FlowAmount')
+
+ # NOTE: the 14 flows without compartment data have no flow amounts
+ nuc_lci.dropna(subset=["compartment"], inplace=True)
# There is no column to merge the inventory and generation data on,
# so we iterate through the plants and make a new column in the lci
@@ -126,8 +149,6 @@ def generate_upstream_nuc(year):
nuc_merged["Source"]="netlnuceiafuel"
# Issue #296 - adding DQI information for upstream processes
- # Setting year to be equal to the year that the costs were generated
- # to develop this USEEIO-based inventory
nuc_merged["Year"] = 2016
nuc_merged["DataReliability"] = 3
nuc_merged["TemporalCorrelation"] = add_temporal_correlation_score(
@@ -136,10 +157,10 @@ def generate_upstream_nuc(year):
nuc_merged["GeographicalCorrelation"] = 3
nuc_merged["TechnologicalCorrelation"] = 3
nuc_merged["DataCollection"] = 4
- #3/20/2025 MBJ - replacing 2016 here so that temporal correlation
- #is based on the year the inventory is based on, but when electricity
- #generation is combined, it needs to be based on the target year for the
- #inventory.
+ # 3/20/2025 MBJ - replacing 2016 here so that temporal correlation
+ # is based on the year the inventory is based on, but when electricity
+ # generation is combined, it needs to be based on the target year for the
+ # inventory.
nuc_merged["Year"] = year
return nuc_merged
diff --git a/electricitylci/olca_jsonld_writer.py b/electricitylci/olca_jsonld_writer.py
index cfb89eb..de4d4bc 100644
--- a/electricitylci/olca_jsonld_writer.py
+++ b/electricitylci/olca_jsonld_writer.py
@@ -16,16 +16,25 @@
import uuid
from zipfile import ZipFile
+from esupy.location import olca_location_meta
import fedelemflowlist
import olca_schema as o
import olca_schema.units as o_units
import olca_schema.zipio as zipio
+import numpy as np
+import pandas as pd
import pytz
import requests
+from electricitylci.globals import COAL_BASIN_CODES
+from electricitylci.globals import C2G_LCI_METHOD
+from electricitylci.globals import G2G_LCI_METHOD
+from electricitylci.globals import GH_URL
+from electricitylci.globals import US_STATES
from electricitylci.globals import paths
from electricitylci.globals import elci_version as VERSION
from electricitylci.utils import check_output_dir
+from electricitylci.utils import read_ba_codes
##############################################################################
@@ -50,14 +59,22 @@
Changelog (since v2.0):
+ - [26.03.27] Skip existing product systems.
+ - [26.03.19] Fix location name and code finder & set IMP to GLO.
+ - [26.03.19] Add LCI_Method description correction to clean_json.
+ - [26.03.05] Add coal basin to location finder.
+ - [26.02.12] Check v3 & v4 UUIDs for locations.
+ - [25.12.19] New update providers helper function.
+ - [25.12.16] New build residual processes method.
- [25.06.11] New method for updating product system description text.
Last edited:
- 2025-06-11
+ 2026-03-27
"""
__all__ = [
"add_to_product_system_description",
"build_product_systems",
+ "build_residual_processes",
"check_exchanges",
"clean_json",
"write",
@@ -111,7 +128,7 @@ def add_to_product_system_description(file_path, description_txt):
_save_to_json(file_path, data)
-def build_product_systems(file_path, elci_config):
+def build_product_systems(file_path, elci_config, add_residuals=False):
"""Generates product systems for electricity at user consumption mixes.
Parameters
@@ -120,6 +137,9 @@ def build_product_systems(file_path, elci_config):
A file path to an existing JSON-LD file with process data saved.
elci_config : str
The model configuration used to make the inventory (e.g., "ELCI_1")
+ add_residuals : bool, optional
+ Whether to create consumption mix product systems for residual mix
+ processes.
Notes
-----
@@ -127,6 +147,8 @@ def build_product_systems(file_path, elci_config):
openLCA to crash.
- This method overwrites the existing JSON-LD with the new product
systems.
+ - This method checks for existing product systems (based on the name
+ of the reference process) and skips any if found.
"""
try:
# Read all JSON-LD data in order to overwrite.
@@ -138,14 +160,33 @@ def build_product_systems(file_path, elci_config):
logging.info("Building product systems in JSON-LD")
# Find all processes for 'at user' consumption mixes
- q1 = re.compile("^Electricity; at user; consumption mix - (.*) - BA$")
- q2 = re.compile("^Electricity; at user; consumption mix - (.*) - FERC$")
- q3 = re.compile("^Electricity; at user; consumption mix - US - US$")
- r1 = _match_process_names(data['Process']['objs'], q1)
- r2 = _match_process_names(data['Process']['objs'], q2)
- r3 = _match_process_names(data['Process']['objs'], q3)
- r = r1 + r2 + r3
- logging.info("Processing %d product systems" % len(r))
+ # NOTE: residual processes could also be converted to product systems.
+ qs1 = "^Electricity; at user; consumption mix - (.*) - BA$"
+ qs2 = "^Electricity; at user; consumption mix - (.*) - FERC$"
+ qs3 = "^Electricity; at user; consumption mix - US - US$"
+
+ q1 = re.compile(qs1)
+ q2 = re.compile(qs2)
+ q3 = re.compile(qs3)
+ q_list = [q1, q2, q3]
+
+ # Provide residual mix support
+ if add_residuals:
+ qr1 = qs1.replace("; consumption", "; residual consumption")
+ qr2 = qs2.replace("; consumption", "; residual consumption")
+ qr3 = qs3.replace("; consumption", "; residual consumption")
+
+ q4 = re.compile(qr1)
+ q5 = re.compile(qr2)
+ q6 = re.compile(qr3)
+
+ q_list += [q4, q5, q6]
+
+ r = []
+ for q in q_list:
+ r += _match_process_names(data['Process']['objs'], q)
+
+ logging.info("Identified %d processes slated for product systems" % len(r))
# Create a common description text
t_now = datetime.datetime.now()
@@ -153,7 +194,7 @@ def build_product_systems(file_path, elci_config):
"This product system was created in openLCA "
"by linking default providers. "
"The processes were generated by ElectricityLCI "
- "(https://github.com/USEPA/ElectricityLCI) "
+ f"({GH_URL}) "
f"version {VERSION} using "
f"the {elci_config} configuration. "
f"Created: {t_now.isoformat()}."
@@ -162,21 +203,146 @@ def build_product_systems(file_path, elci_config):
for pid in r:
p_idx = data['Process']['ids'].index(pid)
p_obj = data['Process']['objs'][p_idx]
- ps_obj = _make_product_system(file_path, p_obj, d_txt)
- # Update master data dictionary
- data['ProductSystem']['objs'].append(ps_obj)
- data['ProductSystem']['ids'].append(ps_obj.id)
- logging.debug("Created %s" % ps_obj.name)
+ # Note product system may already exist; check and skip if found.
+ q = re.compile(p_obj.name)
+ ps_list = _match_process_names(data['ProductSystem']['objs'], q)
+ if len(ps_list) == 1:
+ logging.info(
+ "Product system, '%s', exists! Skipping." % (p_obj.name)
+ )
+ else:
+ logging.info("Creating product system, '%s'" % p_obj.name)
+ ps_obj = _make_product_system(file_path, p_obj, d_txt)
+
+ # Update master data dictionary
+ data['ProductSystem']['objs'].append(ps_obj)
+ data['ProductSystem']['ids'].append(ps_obj.id)
+ logging.debug("Created %s" % ps_obj.name)
# Overwrite JSON-LD
_save_to_json(file_path, data)
+# IN PROGRESS - needs tested
+def build_residual_processes(json_path, rem_ref, rem_txt=""):
+ """Post-processing step that creates residual mix processes for a given
+ JSON-LD.
+
+ Parameters
+ ----------
+ json_path : str
+ File path to JSON-LD.
+ rem_ref : str, pandas.DataFrame
+ Either a file path to CSV file or a pandas data frame containing
+ the balancing authority level residual mixes (e.g., as provided by
+ :func:`get_rem` in residual_grid_mix.py).
+ rem_txt : str, optional
+ Additional residual process description text, by default ""
+
+ Raises
+ ------
+ FileNotFoundError
+ If a non-existent CSV file path was provided for ``rem_ref``.
+ TypeError
+ If ``rem_ref`` was not provided as a file path or data frame.
+
+ Notes
+ -----
+ See reference in :func:`add_residual_mixes` in residual_grid_mix.py.
+ """
+ # 1. READ RESIDUAL MIX DATA
+ # Two options supported here: send pandas DataFrame or CSV file path.
+ if isinstance(rem_ref, pd.DataFrame):
+ logging.info(
+ "Building residual mix processes using provided data frame."
+ )
+ rem = rem_ref
+ elif isinstance(rem_ref, str) and os.path.isfile(rem_ref):
+ # NOTE: the CSV file may round data frame's machine precision
+ # (e.g., 1.0499999999 -> 1.05).
+ logging.info(
+ "Building residual mix processes using provided CSV file."
+ )
+ rem = pd.read_csv(rem_ref)
+ elif isinstance(rem_ref, str) and not os.path.isfile(rem_ref):
+ raise FileNotFoundError(
+ "Failed to find the residual mix CSV located at %s!" % rem_ref
+ )
+ else:
+ raise TypeError(
+ "Residual mixes may be passed as either a pandas DataFrame or "
+ "CSV file path, not %s" % type(rem_ref)
+ )
+
+ # 2. READ JSON-LD DATA
+ try:
+ data = _read_jsonld(json_path, _root_entity_dict())
+ except OSError:
+ logging.warning("Failed to read JSON-LD file, %s" % json_path)
+ raise
+
+ # 3. CREATE RESIDUAL GENERATION MIX PROCESSES
+ logging.info("Creating residual generation mix processes")
+
+ # Initialize process mapping dictionary
+ p_map = {}
+
+ # Find the electricity generation processes in JSON-LD
+ q = re.compile("^Electricity; at grid; generation mix - (.*)$")
+ r = _match_process_names(data['Process']['objs'], q)
+ logging.debug("Found %d generation mix processes." % len(r))
+
+ for pid in r:
+ p_idx = data['Process']['ids'].index(pid)
+ p_obj = data['Process']['objs'][p_idx]
+ ba = q.match(p_obj.name).group(1)
+ rid, data = _make_rem_gen_process(p_obj.id, ba, data, rem_txt, rem)
+ p_map[pid] = rid
+
+ # 4. CREATE RESIDUAL CONSUMPTION MIX PROCESSES
+ logging.info("Creating residual consumption mix processes")
+
+ # Find the electricity consumption processes in JSON-LD
+ q = re.compile(
+ "^Electricity; at (?:user|grid); consumption mix - (.*) - (BA|FERC|US)$"
+ )
+ r = _match_process_names(data['Process']['objs'], q)
+ logging.debug("Found %d consumption mix processes." % len(r))
+
+ for pid in r:
+ rid, data = _make_rem_con_process(pid, data, rem_txt)
+ p_map[pid] = rid
+
+ # 5. UPDATE CONSUMPTION MIX PROCESS PROVIDERS
+ logging.info("Updating residual mix process default providers")
+ for pid in r:
+ # Find the residual process associated with this consumption process.
+ rid = p_map[pid]
+ r_idx = data['Process']['ids'].index(rid)
+ r_obj = data['Process']['objs'][r_idx] # the object being modified
+
+ # The replacement step---
+ # Read through a process's exchanges (r_obj.exchanges).
+ # Examine any/all default providers.
+ # Find replacement UUID in ``p_map`` (note: may throw KeyError)
+ # Replace with Ref object (i.e., search for it in ``data``).
+ r_obj = _update_providers(r_obj, p_map, data)
+
+ # Update the master entity dictionary w/ updated process
+ data['Process']['objs'][r_idx] = r_obj
+
+ # 6. SAVE RESULTS
+ _save_to_json(json_path, data)
+
+
def check_exchanges(p_list):
"""Iterate over process exchanges and log as error when an amount is nan.
This occurrence causes openLCA to crash on upload of JSON-LD.
+ It needs only to be run once; currently implemented in
+ :func:`build_product_systems`, which is the last post-processes
+ step.
Parameters
----------
@@ -210,6 +376,7 @@ def clean_json(file_path):
associated with the resources 'Heat' and 'Water, reclaimed'
6. Fix compartment for two product flows: 'Light fuel oil' and
'Ammonium nitrate' from the coal model.
+ 7. Fix inventory method description for C2G and G2G processes.
Parameters
----------
@@ -225,10 +392,17 @@ def clean_json(file_path):
# Pull flows from each process's exchange list; remove zero product
# flows along the way.
- # https://github.com/USEPA/ElectricityLCI/issues/217
+ # https://github.com/NETL-RIC/ElectricityLCI/issues/217
e_list = []
for p in data["Process"]['objs']:
+ # Issue #328; check if unit processes are C2G or G2G [260319;TWD]
+ has_provider = False
+
for e in p.exchanges:
+ # New tracker for default provider existence [26.03.19;TWD]
+ if e.default_provider:
+ has_provider = True
+
# Get the flow object
fid = data["Flow"]['ids'].index(e.flow.id)
f_obj = data["Flow"]['objs'][fid]
@@ -246,7 +420,7 @@ def clean_json(file_path):
e_list.append(e.flow.id)
# Check if output exchange is labeled as a resource flow
- # https://github.com/USEPA/ElectricityLCI/issues/233
+ # https://github.com/NETL-RIC/ElectricityLCI/issues/233
if not e.is_input and 'resource' in f_obj.category.lower():
logging.warning(
"Fixing resource flow in output exchange! "
@@ -258,7 +432,7 @@ def clean_json(file_path):
p.exchanges.append(e)
# Correct heat resource flows;
- # https://github.com/USEPA/ElectricityLCI/issues/293
+ # https://github.com/NETL-RIC/ElectricityLCI/issues/293
if e.is_input and e.flow.name == 'Heat':
# The new FEDEFL heat resource flow
h_flow = _heat_elem_flow()
@@ -283,7 +457,7 @@ def clean_json(file_path):
p.exchanges.append(e)
# Correct double Elementary Flows category
- # https://github.com/USEPA/ElectricityLCI/issues/149
+ # https://github.com/NETL-RIC/ElectricityLCI/issues/149
if f_obj.category.startswith(
"Elementary flows/Elementary Flows"):
logging.warning(
@@ -293,7 +467,7 @@ def clean_json(file_path):
"/Elementary Flows/", "/")
# Map third-party technosphere flows to NAICS
- # https://github.com/USEPA/ElectricityLCI/issues/149
+ # https://github.com/NETL-RIC/ElectricityLCI/issues/149
# NOTE: this overwrite breaks the reproducibility of the
# UUIDs for these two flows
tech_cat = "Technosphere Flows"
@@ -325,6 +499,23 @@ def clean_json(file_path):
p.last_internal_id += 1
e.internal_id = p.last_internal_id
+ # Update inventory method description
+ if p.process_type == o.ProcessType.LCI_RESULT:
+ # All system processes are currently cradle-to-gate
+ p.process_documentation.inventory_method_description = \
+ C2G_LCI_METHOD
+ elif p.process_type == o.ProcessType.UNIT_PROCESS and has_provider:
+ # Unit process w/ providers are considered gate-to-gate.
+ p.process_documentation.inventory_method_description = \
+ G2G_LCI_METHOD
+ elif (p.process_type == o.ProcessType.UNIT_PROCESS) and (
+ not has_provider):
+ # Unit process w/o providers are considered cradle-to-gate.
+ p.process_documentation.inventory_method_description = \
+ C2G_LCI_METHOD
+ else:
+ logging.warning("You should not be here")
+
# Overwrite
_save_to_json(file_path, data)
@@ -447,10 +638,18 @@ def _actor(name, dict_s):
tuple
olca_schema.Ref : Reference object to an Actor.
dict : The updated root entities dictionary, ``dict_s``.
+
+ Note
+ ----
+ Creates/overwrites Actor UUID using a standard method.
"""
+ # HOTFIX: extract actor names from dictionary; [25.12.16; TWD]
+ if isinstance(name, dict):
+ name = _val(name, 'name', default="")
+
# Skip unnamed or missing actors.
if not isinstance(name, str) or name == '':
- return None
+ return (None, dict_s) # HOTFIX return type; [25.12.16; TWD]
# Generate a standard UUID based on actor name:
uid = _uid(o.ModelType.ACTOR, name)
@@ -532,14 +731,16 @@ def _archive_json(data_list, file_path):
Parameters
----------
data_list : list
- A list of dictionaries.
+ A list of olca schema objects.
file_path : str
A valid filepath to be written to (CAUTION: overwrites existing data)
"""
logging.debug("Writing %d items to %s" % (len(data_list), file_path))
- out_str = ",".join([json.dumps(x.to_dict()) for x in data_list])
+ out_str = ",".join([
+ json.dumps(x.to_dict(), ensure_ascii=False) for x in data_list
+ ])
out_str = "[%s]" % out_str
- with open(file_path, 'w') as f:
+ with open(file_path, 'w', encoding='utf-8') as f:
f.write(out_str)
@@ -569,8 +770,7 @@ def _build_supply_chain(zh, pid, e_list=[], p_list=[]):
1. This method does not apply any standardization for UUID generation.
It is arbitrarily generated by olca_schema class initialization.
2. The methods here are heavily based on those from NETL's NetlOlca class
- currently under development by KeyLogic, here:
- https://github.com/KeyLogicLCA/netlolca
+ currently available online: https://github.com/NETL-RIC/netlolca.
"""
# Pull process object from JSON-LD and add to the processes list.
p_obj = zh.read(o.Process, pid)
@@ -788,6 +988,13 @@ def _exchange(dict_d, dict_s):
tuple
olca_schema.Exchange : Exchange object (or NoneType)
dict : The olca-schema root entity dictionary, updated
+
+ Note
+ ----
+ Secondary dictionary keywords based on olca-schema v2 added in 2025
+ in order to create residual mix processes. The first keywords are based
+ on the eLCI model runs, while the secondary keywords are the standard
+ openLCA keywords.
"""
# Error handle missing data:
if dict_d is None:
@@ -796,12 +1003,26 @@ def _exchange(dict_d, dict_s):
e = o.Exchange.from_dict({
'isQuantitativeReference': _val(
- dict_d, 'quantitativeReference', default=False),
- 'isInput': _val(dict_d, 'input', default=False),
- 'isAvoidedProduct': _val(dict_d, 'avoidedProduct', default=False),
+ dict_d,
+ 'quantitativeReference',
+ 'isQuantitativeReference', # HOTFIX: add 2nd keyword [25.12.16;TWD]
+ default=False),
+ 'isInput': _val(
+ dict_d,
+ 'input',
+ 'isInput', # HOTFIX: add 2nd keyword [25.12.16;TWD]
+ default=False),
+ 'isAvoidedProduct': _val(
+ dict_d,
+ 'avoidedProduct',
+ 'isAvoidedProduct', # HOTFIX: add 2nd keyword [25.12.16;TWD]
+ default=False),
'amount': _val(dict_d, 'amount', default=0.0),
'dqEntry': _format_dq_entry(_val(dict_d, 'dqEntry')),
- 'description': _val(dict_d, 'comment')
+ 'description': _val(
+ dict_d,
+ 'comment',
+ 'description') # HOTFIX: add 2nd keyword [25.12.16;TWD]
})
# Set unit (uses olca unit references)
@@ -820,7 +1041,10 @@ def _exchange(dict_d, dict_s):
# Find the provider process reference (or create one);
# note that this does not update the dict_s entries, but searches them!
# BUG: are you sure this doesn't update dict_s?
- p_ref, dict_s, _ = _process(_val(dict_d, 'provider'), dict_s)
+ # HOTFIX: add secondary keyword for 'defaultProvider' [25.12.16; TWD]
+ p_ref, dict_s, _ = _process(
+ _val(dict_d, 'provider', 'defaultProvider'), dict_s
+ )
if p_ref is not None:
e.default_provider = p_ref.to_ref()
@@ -888,6 +1112,119 @@ def _find_dq(dict_d, dict_key):
return dq
+def _find_location_code_name(loc):
+ """Helper function to find location codes and names amongst
+ balacing authority areas, EIA regions, NERC regions, U.S. states,
+ and openLCA countries & regions.
+
+ Parameters
+ ----------
+ loc : str
+ Location name or code.
+
+ Returns
+ -------
+ tuple
+ A tuple of length two:
+
+ - str, location code
+ - str, location name
+
+ Notes
+ -----
+ This method searches BA names (from ``utils``), US states (from
+ ``globals``) and openLCA locations (from :func:`_get_olca_locations`).
+
+ This method prioritizes U.S. state names over the FERC regions for
+ Hawaii and Alaska.
+ """
+ ba = read_ba_codes()
+ ba = ba.reset_index(drop=False)
+
+ # For any matched column, quickly find the code and name columns
+ code_name_map = {
+ 'BA_Acronym': {'code': 'BA_Acronym', 'name': 'BA_Name'},
+ 'BA_Name': {'code': 'BA_Acronym', 'name': 'BA_Name'},
+ 'EIA_Region_Abbr': {'code': 'EIA_Region_Abbr', 'name': 'EIA_Region'},
+ 'EIA_Region': {'code': 'EIA_Region_Abbr', 'name': 'EIA_Region'},
+ 'FERC_Region': {'code': 'FERC_Region_Abbr', 'name': 'FERC_Region'},
+ 'FERC_Region_Abbr': {'code': 'FERC_Region_Abbr', 'name': 'FERC_Region'}
+ }
+
+ # Don't search these columns for matches (looking at you, Time Zone!)
+ drop_cols = [x for x in ba.columns if x not in code_name_map.keys()]
+ ba = ba.drop(columns=drop_cols)
+
+ # Correct for U.S. states
+ if loc in US_STATES.keys():
+ logging.info("Found U.S. state abbreviation")
+ loc = "US-%s" % loc
+ elif loc in US_STATES.values():
+ logging.info("Found U.S. state name")
+ loc = "United States of America, %s" % loc
+
+ mask = (ba == loc)
+ _, col_indices = np.where(mask)
+
+ # Get the matching column indices
+ match_cols = list(set(col_indices))
+
+ # Initialize string returns
+ name = "%s" % loc
+ code = "%s" % loc
+
+ # Check to see if the location code is a BA, EIA, or FERC region
+ if len(match_cols) == 0:
+ logging.info("Failed first attempt find location for '%s'" % loc)
+
+ # Check upstream coal basins locations [26.03.05; TWD]
+ coal_dict = COAL_BASIN_CODES
+ coal_rev = {v: k for k, v in COAL_BASIN_CODES.items()}
+
+ # Check openLCA standard locations
+ olca_locs = _get_olca_locations()
+ olca_dict = {x.code: x.name for x in olca_locs}
+ rev_dict = {x.name: x.code for x in olca_locs}
+
+ if loc in coal_dict.keys():
+ logging.info("Found name in coal basin locations")
+ code = coal_dict[loc]
+ name = loc
+ elif loc in coal_rev.keys():
+ logging.info("Found code in coal basin locations")
+ code = loc
+ name = coal_rev[loc]
+ elif loc in olca_dict.keys():
+ logging.info("Found code in openLCA locations")
+ code = loc
+ name = olca_dict[loc]
+ elif loc in rev_dict.keys():
+ logging.info("Found name in openLCA locations")
+ code = rev_dict[loc]
+ name = loc
+
+ # HOTFIX: use global for coal import process [26.03.19;TWD]
+ if name == 'Import' or code == 'IMP':
+ logging.info("Setting import location to Global (GLO)")
+ name = "Global"
+ code = "GLO"
+ else:
+ # Assumes hierarchy if multiple columns were matched:
+ # BA first, EIA second, FERC last.
+ match_col = list(ba.columns)[match_cols[0]]
+ row_mask = ba[match_col] == loc
+ num_matches = row_mask.sum()
+
+ if num_matches > 1:
+ logging.warning(
+ "Found multiple matches for '%s' under '%s'" % (loc, match_col)
+ )
+ code = ba.loc[row_mask, code_name_map[match_col]['code']].values[0]
+ name = ba.loc[row_mask, code_name_map[match_col]['name']].values[0]
+
+ return (code, name)
+
+
def _find_ref_exchange(p):
"""Return the exchange class object associated as the quantitative
reference.
@@ -1098,9 +1435,15 @@ def _format_date(entry):
logging.warning("Expected date as string, found %s" % type(entry))
return None
except ValueError:
- logging.warning(
- "Received unexpected date format (M/D/YYYY): '%s'" % entry)
- return None
+ # HOTFIX: add a second-level test for ISO correctness; [25.12.16;TWD]
+ try:
+ d_obj = datetime.datetime.fromisoformat(entry)
+ except ValueError:
+ logging.warning(
+ "Received unexpected date format (M/D/YYYY): '%s'" % entry)
+ return None
+ else:
+ return d_obj.isoformat()
except Exception as e:
logging.warning("Encountered an unexpected error. %s" % str(e))
return None
@@ -1142,6 +1485,77 @@ def _format_dq_entry(entry):
return '(%s)' % ';'.join(nums)
+def _get_olca_locations():
+ """Helper method to create a list of location objects based on
+ Green Delta's openLCA list of locations.
+
+ Returns
+ -------
+ list
+ List of Location objects.
+ """
+ # Putting these in Fed Commons data store; albeit from GreenDelta.
+ data_dir = os.path.join(paths.local_path, "fedcommons")
+
+ loc_file = "locations.json"
+ loc_path = os.path.join(data_dir, loc_file)
+ loc_list = []
+
+ if not os.path.isfile(loc_path) and check_output_dir(data_dir):
+ # esupy's method reads GreenDelta's GitHub.
+ # https://github.com/USEPA/esupy/blob/main/esupy/location.py
+ loc_meta = olca_location_meta()
+
+ # Create a location dictionary.
+ # Based on `build_location_dict` in generate_processes.py
+ # on flac-utils https://github.com/FLCAC-admin/flcac-utils.
+ loc_meta = loc_meta.drop(columns='Category')
+ loc_meta.columns = loc_meta.columns.str.lower()
+ loc_meta['description'] = loc_meta['description'].fillna("")
+ loc_meta = loc_meta.rename(
+ columns={'id': '@id'}
+ ).to_dict(orient='index')
+
+ # Create your location objects
+ for loc_code in loc_meta:
+ loc_list.append(o.Location().from_dict(loc_meta[loc_code]))
+
+ _archive_json(loc_list, loc_path)
+
+ # Only read locally if needed (i.e., if data wasn't just downloaded)
+ if os.path.exists(loc_path) and len(loc_list) == 0:
+ logging.debug("Reading locations from local JSON")
+ with open(loc_path, 'r', encoding='utf-8') as f:
+ my_list = json.load(f)
+ for my_item in my_list:
+ loc_list.append(o.Location.from_dict(my_item))
+
+ return loc_list
+
+
+def _get_location_dict():
+ """Helper method to create a dictionary of location metadata based on
+ GreenDelta's openLCA locations.
+
+ Returns
+ -------
+ dict
+ A dictionary of location metadata where keys are the location codes
+ (e.g., 'CA', 'CA-BC', 'US', 'US-PA').
+
+ Notes
+ -----
+ Used in :func:`_location` to reference against location codes for processes
+ """
+ loc_dict = {}
+ loc_list = _get_olca_locations()
+ for loc_item in loc_list:
+ code = loc_item.code
+ if code not in loc_dict:
+ loc_dict[code] = loc_item.to_dict()
+ return loc_dict
+
+
def _heat_elem_flow():
"""Returns Energy, heat resource from FEDEFL Elementary Flow List
@@ -1263,22 +1677,29 @@ def _location(dict_d, dict_s):
# No code, no location!
# HOTFIX: locations may just be a string [2023-11-14; TWD]
if isinstance(dict_d, str):
- code = dict_d
+ code, name = _find_location_code_name(dict_d)
uid = None
elif isinstance(dict_d, dict):
- code = _val(dict_d, 'name')
+ code, name = _find_location_code_name(_val(dict_d, 'name'))
uid = _val(dict_d, 'id', '@id')
else:
code = ""
+ name = ""
uid = ""
- if not isinstance(code, str):
- return (None, dict_s)
if code == '':
+ logging.debug("No location!")
return (None, dict_s)
+ # HOTFIX: Use GreenDelta's location codes [26.02.13; TWD].
+ gd_loc = _get_location_dict()
+ if code in gd_loc:
+ logging.debug(f"Found location {code} in openLCA default locations")
+ dict_d = gd_loc.get(code)
+ uid = _val(dict_d, 'id', '@id')
+
# Check for valid UUID; otherwise, generate one
- if not _uid_is_valid(uid):
+ if not (_uid_is_valid(uid, 3) or _uid_is_valid(uid, 4)):
uid = _uid(o.ModelType.LOCATION, code)
# Check if location already exists in our records; otherwise, create
@@ -1290,7 +1711,7 @@ def _location(dict_d, dict_s):
else:
logging.debug("Creating new location entry for '%s'" % code)
location = o.Location(id=uid, code=code)
- location.name = code
+ location.name = name # fix name based on new lookup [26.02.26;TWD]
location.latitude = _val(dict_d, 'latitude')
location.longitude = _val(dict_d, 'longitude')
location.description = _val(dict_d, 'description')
@@ -1342,12 +1763,12 @@ def _make_product_system(f_path, process, description=""):
process : olca-schema.Process
A Process object to be converted to a Product System.
description : str, optional
- The product system description text, by default ""
+ The product system description text, by default "".
Returns
-------
olca-schema.ProductSystem
- A product system build on default providers for the given process.
+ A product system built on default providers for the given process.
"""
# Find the reference process
r_ex = _find_ref_exchange(process)
@@ -1411,6 +1832,237 @@ def _make_process_ref(p_obj):
return ref_obj
+# IN PROGRESS: needs tested
+def _make_rem_con_process(pid, e_dict, rem_txt):
+ """Helper function to create residual consumption process.
+
+ Parameters
+ ----------
+ pid : str
+ A universally unique identified associated with a consumption mix
+ process (e.g., 'Electricity; at grid; consumption mix - CAISO - FERC')
+ e_dict : dict
+ The master entity dictionary.
+ rem_txt : str
+ Additional residual process description.
+
+ Returns
+ -------
+ tuple
+ A tuple of length two:
+
+ - str, the new residual consumption process UUID
+ - dict, the updated master entity dictionary
+
+ Notes
+ -----
+ This method adds the new residual consumption mix to the master
+ entity dictionary.
+ """
+ # Extract the original process data & convert to dictionary
+ p_idx = e_dict['Process']['ids'].index(pid)
+ p_obj = e_dict['Process']['objs'][p_idx]
+ p_dict = p_obj.to_dict()
+
+ # Rename to a residual process; try both combinations
+ try:
+ p_dict['name'] = _make_residual_process_name(
+ p_dict['name'], True, False
+ )
+ except ValueError:
+ p_dict['name'] = _make_residual_process_name(
+ p_dict['name'], False, False
+ )
+
+ # Reset UUID (to trigger new UUID generation) and update description.
+ p_dict['@id'] = None
+ if isinstance(p_dict['description'], str):
+ p_dict['description'] += " "
+ p_dict['description'] += rem_txt
+ else:
+ p_dict['description'] = rem_txt
+
+ # Create new residual process
+ rem_obj, e_dict, _ = _process(p_dict, e_dict)
+
+ # Add new process to master entity list
+ if rem_obj.id not in e_dict['Process']['ids']:
+ e_dict['Process']['ids'].append(rem_obj.id)
+ e_dict['Process']['objs'].append(rem_obj)
+ else:
+ # This message should never display.
+ logging.warning(
+ "New residual process UUID already exists! %s" % rem_obj.id
+ )
+
+ return (rem_obj.id, e_dict)
+
+
+def _make_rem_gen_process(pid, ba_name, e_dict, rem_txt, rem_df):
+ """Create a residual mix process for a given at-grid generation mix
+ process.
+
+ Parameters
+ ----------
+ pid : str
+ At-grid generation mix process for a given balancing authority.
+ ba_name : str
+ The name of the given balancing authority.
+ e_dict : dict
+ Master entity dictionary (stored all olca-schema root entities).
+ rem_txt : str
+ Additional process description text.
+ rem_df : pandas.DataFrame
+ Data frame for the given balancing authority with new generation mixes.
+
+ Returns
+ -------
+ tuple
+ A tuple of length two:
+
+ - str, the new residual grid mix process UUID
+ - dict, the updated master entity database
+ """
+ #
+ # Step 1: Create a new Process object.
+ #
+ # Extract (the original process data)
+ p_idx = e_dict['Process']['ids'].index(pid)
+ p_obj = e_dict['Process']['objs'][p_idx]
+ # Convert (to dictionary for editing)
+ p_dict = p_obj.to_dict()
+ # Rename (to a residual process)
+ # NOTE: may want to pass the two booleans along as arguments to this func.
+ p_dict['name'] = _make_residual_process_name(p_dict['name'], True, True)
+ # Reset (to trigger new UUID generation)
+ p_dict['@id'] = None
+ # Update (w/ new residual process description)
+ if isinstance(p_dict['description'], str):
+ p_dict['description'] += " "
+ p_dict['description'] += rem_txt
+ else:
+ p_dict['description'] = rem_txt
+ # Create (new residual process)
+ rem_obj, e_dict, _ = _process(p_dict, e_dict)
+
+ #
+ # Step 2: Update exchange amounts to reflect residuals.
+ #
+ # Define query for searching fuel category from exchange description
+ fq = re.compile("^from (\\w+) - (.*)$")
+ # Extract (residual mix data for current balancing authority)
+ b = rem_df.query("`%s` == '%s'" % ('Subregion', ba_name))
+ # Iterate (over each new process exchange)
+ for p_ex in rem_obj.exchanges:
+ # NOTE: For electricity grid mixes, there are always two or more
+ # exchanges: one output and X inputs. The inputs have the grid mix
+ # values we want.
+ if p_ex.is_input:
+ # Get fuel name (e.g. 'GAS').
+ f_name = ""
+ fc = fq.match(p_ex.description)
+ if fc:
+ f_name = fc.group(1)
+
+ # Query BA data for new mix associated with current fuel.
+ a = b.query("`%s` == '%s'" % ('FuelCategory', f_name))
+
+ # Update mix amounts
+ if len(a) == 1:
+ # Best case scenario; set new mix amount
+ new_mix = a.iloc[0].Gen_Ratio_new
+ logging.debug("Replacing %s with %s for %s in %s" % (
+ p_ex.amount, new_mix, f_name, ba_name))
+ p_ex.amount = new_mix
+ elif len(a) == 0 and len(b) == 0:
+ # Failed to find BA in the data frame.
+ # Could be that there is just no REC data to remove.
+ # For the time being, keep it, because we're none the wiser.
+ logging.info("Failed to find '%s'; skipping" % ba_name)
+ elif len(a) == 0:
+ # Failed to find fuel for a known BA; set to zero.
+ # TODO: consider removing this exchange from exchanges list
+ logging.info("Zeroing mix for '%s' in %s" % (f_name, ba_name))
+ p_ex.amount = 0.0
+ else:
+ # This is a bad place to be.
+ logging.warning(
+ "Found multiple matches of '%s' for '%s'!" % (
+ f_name, ba_name
+ )
+ )
+
+ # NOTE: Consider adding a new Source here, also.
+
+ #
+ # Step 3: Add new residual process to the master entity list
+ #
+ if rem_obj.id not in e_dict['Process']['ids']:
+ e_dict['Process']['ids'].append(rem_obj.id)
+ e_dict['Process']['objs'].append(rem_obj)
+ else:
+ logging.warning(
+ "New residual process UUID already exists! %s" % rem_obj.id
+ )
+
+ return (rem_obj.id, e_dict)
+
+
+def _make_residual_process_name(p_name, at_grid=True, is_gen=True):
+ """Create a new process name for residual generation at grid.
+
+ Parameters
+ ----------
+ p_name : str
+ process name (e.g., Electricity; at grid; generation mix)
+ at_grid : bool, optional
+ Whether name includes "at grid"; otherwise, "at user";
+ defaults to True
+ is_gen : bool, optional
+ Whether names includes "generation"; otherwise, "consumption";
+ defaults to True
+
+ Returns
+ -------
+ str
+ The same electricity generation grid mix process name, but with
+ 'residual' added to the name.
+
+ Raises
+ ------
+ ValueError
+ For a process name that is not 'Electricity; at grid; generation mix'.
+
+ Notes
+ -----
+ This method is taken from Davis et al. (2025) NetlOlca. Online:
+ https://edx.netl.doe.gov/dataset/netlolca, DOI:10.18141/2503973.
+
+ Examples
+ --------
+ >>> orig_name = (
+ ... "Electricity; at grid; generation mix - Arlington Valley, LLC"
+ ... )
+ >>> _make_residual_process_name(orig_name, True, True)
+ 'Electricity; at grid; residual generation mix - Arlington Valley, LLC'
+ """
+ g_txt = "at user"
+ if at_grid:
+ g_txt = "at grid"
+ c_txt = "consumption"
+ if is_gen:
+ c_txt = "generation"
+ q = re.compile("^(Electricity; %s;)( %s mix - .*)$" % (g_txt, c_txt))
+ if q.match(p_name):
+ return q.sub("\\1 residual\\2", p_name)
+ else:
+ raise ValueError(
+ "Expected 'Electricity; %s; %s process', found '%s'" % (
+ g_txt, c_txt, p_name
+ )
+ )
+
+
def _match_process_names(p_list, q):
"""Return a list of process UUIDs that match a given name query.
@@ -1456,9 +2108,17 @@ def _process(dict_d, dict_s):
Returns
-------
tuple
- olca_schema.Process or NoneType : the process object
- dict : the root entity dictionary, updated
- olca_schema.Exchange or NoneType : quantitative reference exchange
+ A tuple of length three.
+
+ - olca_schema.Process or NoneType, the process object
+ - dict, the (potentially) updated root entity dictionary
+ - olca_schema.Exchange or NoneType, quantitative reference exchange
+
+ Notes
+ -----
+ The root entity dictionary does not get the new process by default;
+ however, other entities (e.g., Location, DQSystem, Actor, and Source) are
+ added to the root entity dictionary if encountered for the first time.
"""
if not isinstance(dict_d, dict):
return (None, dict_s, None)
@@ -1468,6 +2128,10 @@ def _process(dict_d, dict_s):
category = _val(dict_d, 'category', default='')
location_code = _val(dict_d, 'location', 'name', default='')
+ # Hotfix location code dictionary [25.12.16; TWD]
+ if isinstance(location_code, dict) and 'name' in location_code:
+ location_code = location_code['name']
+
# Generate the standardized UUID, if absent
if uid is None:
logging.debug("Generating new process UUID for '%s'" % name)
@@ -2047,6 +2711,9 @@ def _save_to_json(json_file, e_dict):
if k == "Flow":
# FEDEFL flows are not added as objects, write them separately
# using fedelmflowlist.write_jsonld() [20240911; BY]
+ # NOTE: calls `Writer._write_flows` in fedelemflowlist.jsonld,
+ # which sets the last updated timestamp to .now() and will
+ # always show up as a change when comparing two JSON-LD files.
flowlist = fedelemflowlist.get_flows()
flows = flowlist[flowlist['Flow UUID'].isin(e_dict[k]['ids'])]
fedelemflowlist.write_jsonld(flows, path=None, zw=writer)
@@ -2301,10 +2968,12 @@ def _unit(unit_name):
unit_name = ""
logging.error(
'dict passed as unit_name but does not contain name key')
- logging.debug("Creating unit, '%s'" % unit_name)
r_obj = o_units.unit_ref(unit_name)
if r_obj is None:
- logging.error("unknown unit, '%s'; no unit reference" % unit_name)
+ logging.error("Unknown unit, '%s'; no unit reference!" % unit_name)
+ else:
+ logging.debug("Returning unit, '%s'" % unit_name)
+
return r_obj
@@ -2345,6 +3014,44 @@ def _update_data(cur_data, new_data):
return new_data
+def _update_providers(p, p_map, e_dict):
+ """Helper function to replace default providers found in a process's
+ exchange table based on a UUID replacement map (i.e., old UUID -> new UUID).
+
+ Parameters
+ ----------
+ p : olca-schema.Process
+ An instance of a Process class to be updated.
+ p_map : dict
+ A dictionary with Process UUIDs and keys (old providers) and Process
+ UUIDs as values (new providers).
+ e_dict : dict
+ The master entity dictionary (e.g., see :func:`_init_root_entities`).
+
+ Returns
+ -------
+ olca-schema.Process
+ A modified version of parameter, ``p``, where default providers
+ are updated based on the UUID map, ``p_map``.
+ """
+ # Read through residual process's exchange table
+ num_ex = len(p.exchanges)
+ for i in range(num_ex):
+ p_ex = p.exchanges[i]
+ # Skip outputs and inputs w/o providers (e.g., elem flows)
+ if p_ex.is_input and p_ex.default_provider is not None:
+ # Read the default provider UUID and find replacement provider
+ dp_id = p_ex.default_provider.id
+ rp_id = p_map[dp_id] # <- throws KeyError when not found
+ # Get the replacement provider's Process object
+ rp_idx = e_dict['Process']['ids'].index(rp_id)
+ rp_obj = e_dict['Process']['objs'][rp_idx]
+ # Update residual process object; link to new provider
+ p.exchanges[i].default_provider = rp_obj.to_ref()
+
+ return p
+
+
def _val(dict_d, *path, **kvargs):
"""Return value from a dictionary.
@@ -2391,3 +3098,26 @@ def _val(dict_d, *path, **kvargs):
if r_val is None and 'default' in kvargs:
r_val = kvargs['default']
return r_val
+
+
+#
+# SANDBOX --- testing grounds for ILCD process renaming.
+#
+if __name__ == '__main__':
+ import os
+ import re
+ from electricitylci.utils import get_logger
+ from electricitylci.globals import output_dir
+ from electricitylci.olca_jsonld_writer import _read_jsonld
+ from electricitylci.olca_jsonld_writer import _root_entity_dict
+
+ log = get_logger(True, False)
+
+ # Read the full JSON-LD into memory
+ my_file = os.path.join(output_dir, "ELCI_2023_jsonld_20260213_091215.zip")
+ my_dict = _read_jsonld(my_file, _root_entity_dict(), False)
+
+ # The focus of ILCD renaming is on Process (and Product System)
+ # Let's start with upstream (2121: Coal Mining)
+ # The groups are (1) region, (2) coal type, (3) mine type / processing
+ p = re.match("^coal extraction and processing - (.*), (.*), (.*)$")
diff --git a/electricitylci/plant_water_use.py b/electricitylci/plant_water_use.py
index ff86556..38a5ce2 100644
--- a/electricitylci/plant_water_use.py
+++ b/electricitylci/plant_water_use.py
@@ -13,6 +13,8 @@
from electricitylci.globals import data_dir
import electricitylci.PhysicalQuantities as pq
from electricitylci.eia923_generation import eia923_download_extract
+from electricitylci.generation import add_temporal_correlation_score
+from electricitylci.model_config import model_specs
##############################################################################
@@ -25,7 +27,7 @@
generation data from the given year.
Last updated:
- 2024-01-10
+ 2026-03-05
"""
__all__ = [
"generate_plant_water_use",
@@ -237,13 +239,19 @@ def generate_plant_water_use(year):
final_water=final_water.drop(columns=["Electricity"])
final_water["plant_id"] = final_water["FacilityID"]
final_water["eGRID_ID"] = final_water["FacilityID"]
- final_water["Year"] = year
final_water["Source"] = "netlwater"
final_water["Unit"] = "kg"
final_water["stage_code"] = "Power plant"
final_water["TechnologicalCorrelation"] = 1
final_water["GeographicalCorrelation"] = 1
- final_water["TemporalCorrelation"] = 1
+
+ # Issue #296 & 328 - updating DQI information for NETL (2016) water
+ final_water["Year"] = 2016
+ final_water["TemporalCorrelation"] = add_temporal_correlation_score(
+ final_water["Year"], model_specs.electricity_lci_target_year
+ )
+ final_water["Year"] = year
+
final_water["DataCollection"] = 5
final_water["DataReliability"] = 1
final_water["input"]=True
diff --git a/electricitylci/process_dictionary_writer.py b/electricitylci/process_dictionary_writer.py
index 161abe0..0a25da3 100644
--- a/electricitylci/process_dictionary_writer.py
+++ b/electricitylci/process_dictionary_writer.py
@@ -14,6 +14,7 @@
import pandas as pd
from electricitylci.globals import (
+ GH_URL,
data_dir,
electricity_flow_name_generation_and_distribution,
electricity_flow_name_consumption,
@@ -38,7 +39,7 @@
Portions of this code were cleaned using ChatGPTv3.5.
Last updated:
- 2025-06-09
+ 2026-03-11
"""
__all__ = [
'con_process_ref',
@@ -83,21 +84,31 @@
# Read in general metadata to be used by all processes
METADATA_FILE = "process_metadata.yml"
+'''str : The process metadata YAML file name.'''
METADATA_PATH = os.path.join(data_dir, METADATA_FILE)
+'''str : The process metadata YAML file path.'''
+metadata = dict()
+'''dict : The dictionary of process metadata.'''
with open(METADATA_PATH, encoding='utf-8') as f:
metadata = yaml.safe_load(f)
# Read in process location uuids
location_UUID = pd.read_csv(os.path.join(data_dir, "location_UUIDs.csv"))
+'''pandas.DataFrame : Columns of NAME and REF_ID for locations.'''
# Read in process name info
process_name = pd.read_csv(os.path.join(data_dir, "processname_1.csv"))
+'''pandas.DataFrame : Contains naming conventions for electricity processes.'''
+
generation_name_parts = process_name[
process_name["Stage"] == "generation"
].iloc[0]
+'''pandas.Series : Naming convention for generation at facility processes.'''
+
generation_mix_name_parts = process_name[
process_name["Stage"] == "generation mix"
].iloc[0]
+'''pandas.Series : Naming convention for at-grid generation mix processes.'''
generation_mix_name = (
generation_mix_name_parts["Base name"]
@@ -106,6 +117,7 @@
+ "; "
+ generation_mix_name_parts["Mix type"]
)
+'''str : At-grid electricity generation mix process name.'''
fuel_mix_name = 'Electricity; at grid; USaverage'
surplus_pool_name = "Electricity; at grid; surplus pool"
@@ -135,7 +147,7 @@
}
OLCA_TO_METADATA = {
- "timeDescription": None,
+ "timeDescription": "TemporalDescription",
"validUntil": "End_date",
"validFrom": "Start_date",
"technologyDescription": "TechnologyDescription",
@@ -163,17 +175,20 @@
"dqSystem": None,
"dqEntry": None
}
+'''dict : Keys are olca-schema fields and values are YAML metadata keys.'''
VALID_FUEL_CATS=[
"default",
- "biomass", # NEW
+ "all", # NEW (Canada generation process metadata)
+ "biomass",
"nuclear_upstream",
"geothermal",
- "hydro", # NEW
+ "hydro",
"solar",
"solarthermal",
"wind",
"consumption_mix",
+ "distribution_mix",
"generation_mix",
"coal_upstream",
"gas_upstream",
@@ -185,6 +200,7 @@
"solartherm_construction_upstream",
"wind_construction_upstream",
]
+'''list : List of valid fuel categories found in the process metadata YAML.'''
##############################################################################
@@ -408,6 +424,10 @@ def exchange_table_creation_input_usaverage(database, fuelname):
A dictionary representing the input exchange table entry for the US
average electricity generation mix.
+ Notes
+ -----
+ Referenced in generation_mix.py and utilized during :func:`run_epa_trade`.
+
Examples
--------
>>> import pandas as pd
@@ -934,19 +954,28 @@ def process_description_creation(process_type="fossil"):
except AssertionError:
process_type = "default"
+ # The subkeys "replace_egrid" and "use_egrid" are relevant to:
+ # - "default" process type (e.g., at-user; consumption mix);
+ # - "consumption_mix" process type (e.g., at-grid; consumption mix)
+ # - "generation_mix" process type (e.g., at-grid, generation mix)
if model_specs.replace_egrid is True:
subkey = "replace_egrid"
else:
subkey = "use_egrid"
- global year
+ # NEW: add subkey for natural gas modeling year. [26.02.18; TWD]
+ if process_type == "gas_upstream":
+ subkey = "NGI_" + str(model_specs.ng_model_year)
+
key = "Description"
try:
desc_string = metadata[process_type][key]
except KeyError:
logging.debug(
- f"Failed first key ({key}), trying subkey: {subkey}")
+ f"Failed first key ({key}) for {process_type}, "
+ f"trying subkey: {subkey}"
+ )
try:
desc_string = metadata[process_type][subkey][key]
logging.debug(
@@ -960,12 +989,16 @@ def process_description_creation(process_type="fossil"):
process_type = "default"
desc_string = metadata[process_type][key]
+ # Append YAML description string with eLCI clause and clean up.
+ # HOTFIX: update GitHub URL [26.02.26; TWD]
desc_string = (
- desc_string
- + " This process was created with ElectricityLCI "
- + "(https://github.com/USEPA/ElectricityLCI) version "
+ desc_string.rstrip()
+ + " This process was created with ElectricityLCI ("
+ + GH_URL
+ + ") version "
+ elci_version
+ " using the " + model_specs.model_name + " configuration.")
+ desc_string = desc_string.strip()
return desc_string
@@ -978,6 +1011,10 @@ def process_doc_creation(process_type="default"):
provided process type. It maps certain keys from OLCA to Metadata and
retrieves values from the metadata based on the process type and keys.
+ This method is called for generation, generation mix, consumption mix, and
+ distribution processes, as well as upstream processes (as found in
+ :func:`_process_table_creation_gen` in upstream_dict.py).
+
Parameters
----------
process_type : str, optional
@@ -996,16 +1033,30 @@ def process_doc_creation(process_type="default"):
"""
from electricitylci.generation import get_generation_years
try:
- assert process_type in VALID_FUEL_CATS, f"Invalid process_type ({process_type}), using default"
+ assert process_type in VALID_FUEL_CATS
except AssertionError:
+ logging.debug(
+ "Invalid process type, '%s', using 'default'" % process_type
+ )
process_type = "default"
+ # The subkeys "replace_egrid" and "use_egrid" are relevant to:
+ # - "default" process type (e.g., at-user; consumption mix);
+ # - "consumption_mix" process type (e.g., at-grid; consumption mix)
+ # - "generation_mix" process type (e.g., at-grid, generation mix)
if model_specs.replace_egrid is True:
subkey = "replace_egrid"
else:
subkey = "use_egrid"
- global year
+ # NEW: add subkey for natural gas modeling year. [26.02.18; TWD]
+ if process_type == "gas_upstream":
+ subkey = "NGI_" + str(model_specs.ng_model_year)
+
+ # NEW: subkey for renewable generation. [26.02.16]
+ if process_type in ["wind", "solarthermal", "solar"]:
+ subkey = "LCI_" + str(model_specs.renewable_vintage)
+
ar = dict()
for kw, key in OLCA_TO_METADATA.items():
@@ -1013,35 +1064,46 @@ def process_doc_creation(process_type="default"):
if key is not None:
try:
# First try the key at the process level
+ logging.debug(f"Looking for metadata {key}")
ar[kw] = metadata[process_type][key]
except KeyError:
- logging.debug(
- f"Failed first key ({kw}), trying subkey: {subkey}")
try:
# Try looking at the subkey level
- ar[kw] = metadata[process_type][subkey][key]
logging.debug(
- "Failed subkey, likely no entry in metadata for "
- f"{process_type}:{kw}")
+ f"Failed key ({key}), trying: {subkey}: {key}"
+ )
+ ar[kw] = metadata[process_type][subkey][key]
except:
try:
+ logging.debug("Failed subkey; looking at default")
# Check to see if default has the key
ar[kw] = metadata["default"][key]
except KeyError:
# Lastly, check to see if the default has the subkey
+ logging.debug("Failed default key; looking at subkey")
ar[kw] = metadata['default'][subkey][key]
except TypeError:
+ # HOTFIX: don't overwrite process_type [26.03.05; TWD]
logging.debug(
"Failed first key, likely no metadata defined for "
f"{process_type}")
- process_type = "default"
- ar[kw] = metadata[process_type][key]
+ ar[kw] = metadata["default"][key]
- ar["timeDescription"] = ""
- # default valid year is the range of generation years
+ # Default valid year is the range of generation years
if not ar["validUntil"]:
- #Hot fix for https://github.com/USEPA/ElectricityLCI/issues/244
+ # Hot fix for https://github.com/NETL-RIC/ElectricityLCI/issues/244
+ # Default is the full scope of background data; should be true for
+ # fuel generation processes (i.e., the StEWI inventory of interest +
+ # NETL renewable data years). NOTE: upstream processes likely have
+ # their validity dates included in the process_metadata.yml.
year_range = get_generation_years()
+ # For at-grid generation, the data for the EIA generation year.
+ # For at-user consumption, the data are EIA gen year (for T&D).
+ if process_type in ['generation_mix', 'distribution_mix']:
+ year_range = [model_specs.eia_gen_year,]
+ # For at-grid consumption, the data are for the trading year.
+ if process_type == 'consumption_mix':
+ year_range = [model_specs.NETL_IO_trading_year,]
ar["validUntil"] = "12/31/" + str(max(year_range))
ar["validFrom"] = "1/1/" + str(min(year_range))
ar["sources"] = [x for x in ar["sources"].values()]
@@ -1052,7 +1114,8 @@ def process_doc_creation(process_type="default"):
ar["exchangeDqSystem"] = exchangeDqsystem()
ar["dqSystem"] = processDqsystem()
# Temp place holder for process DQ scores
- ar["dqEntry"] = "(5;5)"
+ # HOTFIX: replace (5;5) with (2;4); see issue 322. [26.02.24;TWD]
+ ar["dqEntry"] = "(2;4)"
ar["description"] = process_description_creation(process_type)
return ar
@@ -1153,20 +1216,18 @@ def process_table_creation_con_mix(region, exchanges_list):
ar["exchanges"] = exchanges_list
ar["location"] = location(region)
ar["parameters"] = ""
- ar["processDocumentation"] = process_doc_creation(
- process_type="consumption_mix")
+ ar["processDocumentation"] = process_doc_creation("consumption_mix")
ar["processType"] = "UNIT_PROCESS"
ar["name"] = consumption_mix_name + " - " + region
- ar["category"] = (
- "22: Utilities/"
- "2211: Electric Power Generation, Transmission and Distribution")
- ar["description"] = (
- "Electricity consumption mix using power plants in the "
- + str(region) + " region.")
- ar["description"] = (ar["description"]
- + " This process was created with ElectricityLCI "
- + "(https://github.com/USEPA/ElectricityLCI) version " + elci_version
- + " using the " + model_specs.model_name + " configuration."
+ ar["category"] = "%s/%s" % ( # <- same as gen mix
+ generation_mix_name_parts['Category'],
+ generation_mix_name_parts['Subcategory']
+ )
+ ar['description'] = (
+ 'This process provides the electricity inputs from the various '
+ + 'trade regions that make up the electricity consumption '
+ + f'mix for the {region} region.\n\n'
+ + ar['processDocumentation']['description']
)
ar["version"] = make_valid_version_num(elci_version)
@@ -1196,21 +1257,18 @@ def process_table_creation_distribution(region, exchanges_list):
ar["exchanges"] = exchanges_list
ar["location"] = location(region)
ar["parameters"] = ""
- ar["processDocumentation"] = process_doc_creation()
+ ar["processDocumentation"] = process_doc_creation("distribution_mix")
ar["processType"] = "UNIT_PROCESS"
ar["name"] = distribution_to_end_user_name + " - " + region
- ar["category"] = (
- "22: Utilities/"
- "2211: Electric Power Generation, Transmission and Distribution")
- ar["description"] = (
- "Electricity distribution to end user in the "
- + str(region)
- + " region."
+ ar["category"] = "%s/%s" % ( # <- same as gen mix
+ generation_mix_name_parts['Category'],
+ generation_mix_name_parts['Subcategory']
)
- ar["description"]=(ar["description"]
- + " This process was created with ElectricityLCI "
- + "(https://github.com/USEPA/ElectricityLCI) version " + elci_version
- + " using the " + model_specs.model_name + " configuration."
+ ar['description'] = (
+ 'This process provides the electricity inputs from the various '
+ + 'trade regions that make up the electricity consumption '
+ + f'mix for the {region} region.\n\n'
+ + ar['processDocumentation']['description']
)
ar["version"] = make_valid_version_num(elci_version)
@@ -1279,18 +1337,14 @@ def process_table_creation_gen(fuelname, exchanges_list, region):
+ " region"
)
}
- try:
- # Use the software version number as the process version
- ar["version"] = make_valid_version_num(elci_version)
- except:
- # Set to 1 by default
- ar["version"] = 1
+ ar["version"] = make_valid_version_num(elci_version)
+
return ar
def process_table_creation_genmix(region, exchanges_list):
"""
- Create a dictionary representing a process table for a generation mix.
+ Create an olca-schema formatted dictionary a generation mix process.
Parameters
----------
@@ -1307,39 +1361,42 @@ def process_table_creation_genmix(region, exchanges_list):
Notes
-----
- This function creates a dictionary to represent a process table for a
- generation mix. It populates the dictionary with various key-value pairs,
- including region-specific information and exchanges.
+ - This method is responsible for naming generation mix processes (i.e.,
+ "Electricity; at grid; generation mix - REGION").
+ - Note that ``process_doc_creation`` includes the process description
+ field found in the YAML.
+
+ This method is referenced in ``olcaschema_genmix`` in generation_mix.py.
Examples
--------
- >>> region = "ExampleRegion"
+ >>> region = "US"
>>> exchanges = [exchange1, exchange2, exchange3]
>>> process_table = process_table_creation_genmix(region, exchanges)
"""
- process_dict = {
- "@type": "Process",
- "allocationFactors": "",
- "defaultAllocationMethod": "",
- "exchanges": exchanges_list,
- "location": location(region),
- "parameters": "",
- "processDocumentation": process_doc_creation(
- process_type="generation_mix"),
- "processType": "UNIT_PROCESS",
- "name": f"{generation_mix_name} - {region}",
- "category": (
- "22: Utilities/2211: Electric Power Generation, "
- "Transmission and Distribution"),
- "description": (
- f"Electricity generation mix in the {region} region. "
- "This process was created with ElectricityLCI "
- "(https://github.com/USEPA/ElectricityLCI) version "
- f"{elci_version} using the {model_specs.model_name} "
- "configuration."),
- "version": make_valid_version_num(elci_version)
- }
- return process_dict
+ # Update to resemble ``_process_table_creation_gen`` in upstream_dict.py
+ ar = dict()
+ ar["@type"] = "Process"
+ ar["allocationFactors"] = ""
+ ar["defaultAllocationMethod"] = ""
+ ar["exchanges"] = exchanges_list
+ ar["location"] = location(region)
+ ar["parameters"] = ""
+ ar["processDocumentation"] = process_doc_creation("generation_mix")
+ ar["processType"] = "UNIT_PROCESS"
+ ar["name"] = f"{generation_mix_name} - {region}"
+ ar["category"] = "%s/%s" % (
+ generation_mix_name_parts['Category'],
+ generation_mix_name_parts['Subcategory']
+ )
+ ar['description'] = (
+ 'This process provides the electricity inputs from the various '
+ + 'generation technologies that make up the electricity generation '
+ + f'mix for the {region} region.\n\n'
+ + ar['processDocumentation']['description']
+ )
+ ar['version'] = make_valid_version_num(elci_version)
+ return ar
def process_table_creation_surplus(region, exchanges_list):
@@ -1348,6 +1405,7 @@ def process_table_creation_surplus(region, exchanges_list):
This function generates a dictionary representing a process table entry for
a surplus pool process with the provided region and list of exchanges.
+ This method is called when not replacing eGRID.
Parameters
----------
@@ -1382,8 +1440,9 @@ def process_table_creation_surplus(region, exchanges_list):
"2211: Electric Power Generation, Transmission and Distribution")
ar["description"] = "Electricity surplus in the " + str(region) + " region."
ar["description"]=(ar["description"]
- + " This process was created with ElectricityLCI "
- + "(https://github.com/USEPA/ElectricityLCI) version " + elci_version
+ + " This process was created with ElectricityLCI ("
+ + GH_URL
+ + ") version " + elci_version
+ " using the " + model_specs.model_name + " configuration."
)
ar["version"] = make_valid_version_num(elci_version)
@@ -1423,7 +1482,7 @@ def process_table_creation_usaverage(fuel, exchanges_list):
ar["exchanges"] = exchanges_list
ar["location"] = location('US')
ar["parameters"] = ""
- ar["processDocumentation"] = process_doc_creation(process_type="fuel_mix")
+ ar["processDocumentation"] = process_doc_creation("fuel_mix")
ar["processType"] = "UNIT_PROCESS"
ar["name"] = fuel_mix_name + " - " + str(fuel)
ar["category"] = (
@@ -1432,8 +1491,9 @@ def process_table_creation_usaverage(fuel, exchanges_list):
ar["description"] = (
"Electricity fuel US Average mix for the " + str(fuel) + " fuel.")
ar["description"] = (ar["description"]
- + " This process was created with ElectricityLCI "
- + "(https://github.com/USEPA/ElectricityLCI) version " + elci_version
+ + " This process was created with ElectricityLCI ("
+ + GH_URL
+ + ") version " + elci_version
+ " using the " + model_specs.model_name + " configuration."
)
ar["version"] = make_valid_version_num(elci_version)
@@ -1588,7 +1648,6 @@ def unit(unt):
>>> unit_entry = unit(unit_name)
"""
ar = dict()
- ar["internalId"] = ""
ar["@type"] = "Unit"
ar["name"] = unt
return ar
@@ -1598,7 +1657,11 @@ def unit(unt):
# POST-PROCESSING GLOBALS
##############################################################################
for key in metadata.keys():
- metadata[key] = process_metadata(metadata[key])
+ try:
+ metadata[key] = process_metadata(metadata[key])
+ except Exception as e:
+ logging.error("Failed to read metadata key, %s! %s" % (key, e))
+ raise
##############################################################################
diff --git a/electricitylci/residual_grid_mix.py b/electricitylci/residual_grid_mix.py
new file mode 100644
index 0000000..ba11780
--- /dev/null
+++ b/electricitylci/residual_grid_mix.py
@@ -0,0 +1,710 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+#
+# residual_grid_mix.py
+#
+##############################################################################
+# REQUIRED MODULES
+##############################################################################
+import logging
+
+import pandas as pd
+
+from electricitylci.globals import GREEN_E
+from electricitylci.globals import OVERFLOW_E
+from electricitylci.model_config import model_specs
+from electricitylci import get_generation_mix_process_df
+from electricitylci.eia860_facilities import eia860_balancing_authority
+from electricitylci.eia923_generation import build_generation_data
+from electricitylci.olca_jsonld_writer import build_residual_processes
+from electricitylci.utils import get_nrel_rec
+from electricitylci.utils import map_ba_codes
+from electricitylci.utils import write_csv_to_output
+from electricitylci.globals import NREL_REC_URL
+
+
+##############################################################################
+# MODULE DOCUMENTATION
+##############################################################################
+__doc__ = """A module to calculate residual electricity grid mix (REM) based
+on the Energy Information Administration (EIA) Form 923 generation data
+(filtered based on :func:`build_generation_data` found in eia923_generation.py)
+and mix calculations (e.g., generation ratio and fuel category) based on
+:func:`create_generation_mix_process_df_from_model_generation_data` found in
+generation_mix.py. The publicly released Excel workbook, Status and Trends in
+the U.S. Voluntary Green Power Market,[1] published by the National Laboratory
+of the Rockies, provides state-level renewable energy certificate (REC) sales.
+Power plant regional information (based on EIA Form 860) is used to map state
+REC sales to balancing authority generation (e.g., by facility counts and,
+in future releases by facility generation).
+
+To calculate the residual mix, a balancing authority's generation mix is
+categorized as either REC (renewable energy sold as a certificate) or non-REC
+(the desired residual generation). REC generation is based on the above
+aggregation method from the public sales data. Following this division, the
+next step is to determine the fuel-based mix of the residual generation. This
+is accomplished in :func:`update_mix`, where the original mix is divided into
+renewable (REC-compatible generation) and non-renewable (not for REC sale)
+generation. Non-renewable generation is by definition non-REC, thus carries
+over to the non-REC generation mix.
+
+The renewable generation may be labeled under several fuel categories as
+defined in the global variable, ``GREEN_E`` (e.g., hydro, biomass, solar, wind,
+geo). REC sales do not (as of writing) distinguish the fuel type used;
+therefore, the ratio of renewable generation by fuel category and total
+renewable generation is calculated using :func:`calc_relative_ratio`. The
+renewable electricity sold as REC is subtracted from the total renewable
+generation to determine the non-REC renewable generation. The ratio of renewable
+generation by fuel category is used to allocate the non-REC renewable
+generation to each fuel category (i.e., assumes the same relative ratio across
+renewable fuel categories).
+
+Negative non-REC renewable generation amounts are possible due to the inexact
+process of allocating REC sales to balancing authorities. In the case of
+negative non-REC renewable generation, the user may elect one of two options
+(i.e., 'zero' or 'keep' in the YAML configuration). For the 'keep' option, the
+MIXED or OTHER fuel categories are queried in the non-renewable mix fuels. If
+found, the non-renewable non-REC generation is reduced by the overage in REC
+sales to renewable generation (i.e., the assumption that renewable energy
+exists within the MIXED or OTHER fuels); note that this is done
+indiscriminantly across all non-renewable fuel categories using the relative
+ratio method used for renewable fuels. If the user elects to 'zero' the
+overage, the non-REC non-renewable generation remains unchanged. The non-REC
+renewable energy is zeroed regardless of selection.
+
+The non-renewable and renewable non-REC generation amounts are summed to
+determine the non-REC generation total. The fuel-specific generation amounts
+(i.e., based on the renewable and non-renewable non-REC generation amounts
+allocated using the relative ratio method) are divided by the new non-REC
+generation total to determine the new residual mix.
+
+Methods are based on ``elci_to_rem`` Python tool version 2.[2]
+
+1. E. O'Shaughnessy, S. Jena, and D. Salyer. 2025. Status and Trends in the
+ Voluntary Market (2024 Data). Golden, CO: NLR. Online:
+ https://www.nlr.gov/docs/libraries/analysis/nrel-green-power-data-v2024.xlsx
+2. Tyler W. Davis, Matthew Jamieson, Becca Rosen, Joseph Chou, elci_to_rem,
+ 1/21/2025, https://edx.netl.doe.gov/dataset/elci_to_rem,
+ DOI: 10.18141/2503966
+
+Last updated:
+ 2026-03-27
+"""
+__all__ = [
+ "agg_by_count",
+ "calc_relative_ratio",
+ "get_elci_mix",
+ "get_rec_agg",
+ "get_rem",
+ "update_mix",
+]
+
+
+##############################################################################
+# FUNCTIONS
+##############################################################################
+def add_residual_mixes(json_path=None):
+ """Helper function to generate residual mix processes.
+
+ Takes model configuration parameters (``eia_gen_year``,
+ ``rem_weight_method``, and ``neg_rem_method``) to generate residual
+ mix data frame (see :func:`get_rem`), which may be saved to CSV
+ (depending on config parameter, ``output_residual_mix``), and
+ passes the residual mix to olca_jsonld_writer for creating the processes.
+
+ Parameters
+ ----------
+ json_path : str, optional
+ The JSON-LD file path, by default None.
+ If none, the current model run's JSON-LD file is referenced.
+ (_this optional parameter is to permit REM post-processing_)
+
+ Notes
+ -----
+ This method will not run if the configuration parameter,
+ ``add_residual_mix`` is set to false.
+ """
+ # Stop process if configuration is not set up for residual mixes.
+ if not model_specs.add_residual_mix:
+ logging.info("Residual mix processes are not created.")
+ return
+
+ # Create the residual process description text.
+ # NOTE: This is added to all residual process descriptions.
+ rem_text = (
+ "Electricity generation mixes updated to reflect residual grid "
+ "mix based on NLR's Status and Trends in the Voluntary Market "
+ f"for sales in year {model_specs.eia_gen_year} "
+ f"({NREL_REC_URL}). "
+ )
+ if model_specs.rem_weight_method == 'count':
+ rem_text += (
+ "The balancing authority residual mix is based on a facility "
+ "count weighting method of state-level REC sales where excess REC "
+ )
+ elif model_specs.rem_weight_method == 'gen':
+ rem_text += (
+ "The balancing authority residual mix is based on a weighting "
+ "method of facility-level generation using state-level REC sales "
+ "where excess REC "
+ )
+
+ if model_specs.neg_rem_method == 'zero':
+ rem_text += (
+ "generation amounts (MWh) are ignored (i.e., assumed zero; "
+ "accounts for all available renewable generation)."
+ )
+ elif model_specs.neg_rem_method == 'keep':
+ rem_text += (
+ "generation amounts (MWh) are subtracted from non-renewable "
+ "sources, assuming that some renewable energy may be provided from "
+ "a non-renewable fuel category (e.g., mixed/other fuels)."
+ )
+
+ # Create residual mix for BA by fuel category;
+ # let the user decide to save mix as CSV file in outputs
+ df = get_rem(to_save=model_specs.output_residual_mix)
+
+ # Allow user to run this process on an existing baseline JSON-LD;
+ # otherwise, default back to the current output file.
+ if json_path is None:
+ json_path = model_specs.namestr
+ else:
+ logging.info("Adding residual processes to %s" % json_path)
+
+ # Add residual process to JSON-LD
+ build_residual_processes(json_path, df, rem_text)
+
+
+def agg_by_count():
+ """Partition state-based REC electricity generation using the fractional
+ weights of electricity generating facility counts found within the shared
+ boundaries of each state and balancing authority area.
+
+ Notes
+ -----
+ This method assumes the state names reported in EIA Form 860 (i.e., this
+ does not perform any spatial analysis).
+
+ Returns
+ -------
+ pandas.Series
+ A Pandas series where the index is 'BA_CODE' (balancing authority
+ area abbreviation) and the values are 'REC_FRAC' (allocated REC
+ generation from states to their BA areas).
+ """
+ logging.info("Aggregating by state counts")
+ ba_df = eia860_balancing_authority(year=model_specs.eia_gen_year)
+ ba_df.rename(columns={
+ 'Balancing Authority Name': "BA_NAME",
+ 'Balancing Authority Code': "BA_CODE"}, inplace=True)
+
+ # Not every plant has a BA and State, so drop NAs
+ ba_df = ba_df.dropna(subset='BA_CODE')
+
+ # Create plant count tables
+ table_1 = ba_df.value_counts(subset=['State', 'BA_CODE'])
+ table_1.name = "STBA_PLANTS"
+ table_2 = ba_df.value_counts(subset=['State'])
+ table_2.name = "ST_PLANTS"
+
+ # Join these series together
+ t1_df = table_1.reset_index(drop=False)
+ t2_df = table_2.reset_index(drop=False)
+ t1_df = t1_df.merge(t2_df, how='left', on='State')
+
+ # Calculate state-level fractions
+ # these should all add to 1.0 given the NA drop above.
+ t1_df["ST_FRAC"] = t1_df['STBA_PLANTS'] / t1_df['ST_PLANTS']
+
+ # Join REC data
+ rec_df = get_nrel_rec(model_specs.eia_gen_year)
+ rec_df = rec_df[['State', 'Total']].copy()
+ jdf = t1_df.merge(rec_df, how='left', on='State')
+
+ # Calculate BA REC amounts using plant count fractions
+ jdf['REC_FRAC'] = jdf['ST_FRAC'] * jdf['Total']
+
+ # Allocate RECs to their BA areas using the new relative fractions
+ # the sum of REC_FRAC should equal the sum of Total in the REC data frame
+ tot_df = jdf.groupby(by='BA_CODE')['REC_FRAC'].agg("sum")
+ tot_df.index.names = ['BA_CODE']
+ return tot_df
+
+
+def agg_by_gen():
+ """Partition state-based REC electricity generation using the fractional
+ weights of electricity generating facilities found within the shared
+ boundaries of each state and balancing authority area weighted by
+ facility-level annual generation.
+
+ Returns
+ -------
+ pandas.Series
+ A Pandas series where the index is 'BA_CODE' (balancing authority
+ abbreviation) and the values are 'REC_FRAC' (allocated REC
+ generation from states to their BA areas).
+ """
+ logging.info("Aggregating by facility-level generation.")
+
+ # Get state and BA info for each facility.
+ ba_df = eia860_balancing_authority(year=model_specs.eia_gen_year)
+ ba_df.rename(columns={
+ 'Balancing Authority Name': "BA_NAME",
+ 'Balancing Authority Code': "BA_CODE",
+ "Plant Id": "FacilityID"}, inplace=True)
+
+ # Not every plant has a BA and State, so drop NAs
+ ba_df = ba_df.dropna(subset='BA_CODE')
+
+ # Fix column type for merging purposes.
+ ba_df['FacilityID'] = ba_df['FacilityID'].astype("int")
+
+ # Get facility-level generation amounts
+ fac_gen = build_generation_data(
+ generation_years=[model_specs.eia_gen_year,]
+ )
+ fac_gen = fac_gen.drop(columns='Year')
+
+ # Add annual generation to facility data (i.e., 'Electricity' column).
+ ba_df = ba_df.merge(fac_gen, on='FacilityID', how='inner')
+
+ # Create plant generation tables
+ table_a = ba_df.groupby(
+ by=['State', 'BA_CODE']).agg({'Electricity': 'sum'}).Electricity
+ table_b = ba_df.groupby(
+ by='State').agg({'Electricity': 'sum'}).Electricity
+
+ table_a.name = "STBA_GEN"
+ table_b.name = "ST_GEN"
+
+ # Join these series together
+ ta_df = table_a.reset_index(drop=False)
+ tb_df = table_b.reset_index(drop=False)
+ ta_df = ta_df.merge(tb_df, how='left', on='State')
+
+ # Calculate state-level fractions;
+ # these should add to 1.0 for each state given the NA drop above.
+ ta_df["ST_FRAC"] = ta_df['STBA_GEN'] / ta_df['ST_GEN']
+
+ # Join REC data
+ rec_df = get_nrel_rec(model_specs.eia_gen_year)
+ rec_df = rec_df[['State', 'Total']].copy()
+ jdf = ta_df.merge(rec_df, how='left', on='State')
+
+ # Calculate BA REC amounts using facility-level generation weights
+ jdf['REC_FRAC'] = jdf['ST_FRAC'] * jdf['Total']
+
+ # Allocate RECs to their BA areas using the new relative fractions
+ # the sum of REC_FRAC should equal the sum of Total in the REC data frame
+ tot_df = jdf.groupby(by='BA_CODE').agg({'REC_FRAC': 'sum'})
+
+ return tot_df
+
+
+def calc_relative_ratio(df, add_total=False):
+ """Calculate the relative electricity generation fractions in a data frame.
+
+ Parameters
+ ----------
+ df : panda.DataFrame
+ A data frame with 'Electricity' field for electricity per fuel category.
+ For example, taking a subset of electricityLCI's generation mix for
+ a given Balancing Authority area.
+ add_total : bool, optional
+ Switch to include 'Relative_Total' field, by default False
+
+ Returns
+ -------
+ pandas.DataFrame
+ The same as the argument, but with new fields, "Relative_Ratio"
+ and optional "Relative_Total".`
+
+ Raises
+ ------
+ TypeError
+ When argument object is not a data frame.
+ IndexError
+ When data frame does not have the expected 'Electricity' field.
+ """
+ if not isinstance(df, pd.DataFrame):
+ raise TypeError("Expected data frame, received %s" % type(df))
+ if "Electricity" not in df.columns:
+ raise IndexError("Data frame missing required 'Electricity' field!")
+ total_e = df['Electricity'].sum()
+ df['Relative_Ratio'] = df['Electricity'] / total_e
+ if add_total:
+ df['Relative_Total'] = total_e
+ return df
+
+
+def get_elci_mix():
+ """Create data frame of balancing authority electricity generation
+ mix amounts by primary fuel category using EIA Form 860 and generation
+ from EIA Form 923.
+
+ Returns
+ -------
+ pandas.DataFrame
+ A data frame with fields: "Subregion" (i.e, balancing authority
+ names), "FuelCategory" (i.e., primary fuel technology names),
+ "Electricity" (i.e., annual generation, MWh), and "Generation_Ratio"
+ (i.e., fraction of the total generation accounted by the fuel type
+ for the given subregion), and "BA_CODES" with balancing authority
+ abbreviations.
+ """
+ df = get_generation_mix_process_df(regions="BA")
+ return map_ba_codes(df)
+
+
+def get_rec_agg(agg_type, as_series=True):
+ """Return REC generation aggregated from states to balancing authority
+ areas based on the aggregation type.
+
+ Parameters
+ ----------
+ agg_type : str
+ The aggregation type. Valid options are 'area' and 'count'.
+
+ There are two options to determine the fraction of each state's
+ RECs allocated to each balancing authority area.
+
+ - The 'count' option uses facility counts that fall within the
+ shared regions of states and balancing authorities.
+ - The 'gen' option uses facility-level generation as a weighting
+ factor to the 'count' method.
+
+ as_series : bool, optional
+ Switch to return object as either a Pandas series (if true) or as a
+ data frame (if false), by default True.
+
+ Returns
+ -------
+ pandas.Series or pandas.DataFrame
+ A series or data frame with balancing authority codes (BA_CODE) and
+ their REC generation amounts (REC_FRAC).
+
+ Raises
+ ------
+ ValueError
+ If aggregation type is not valid.
+ """
+ if agg_type == 'gen':
+ r = agg_by_gen()
+ elif agg_type == 'count':
+ r = agg_by_count()
+ else:
+ raise ValueError(
+ "Expected agg type to be either 'area' or 'count'; "
+ "found '%s'" % agg_type)
+ if not as_series:
+ r = r.reset_index(drop=False)
+ return r
+
+
+def get_rem(to_save=False):
+ """A short-hand method for creating the residual electricity mix data frame.
+
+ Parameters
+ ----------
+ to_save : bool, optional
+ Switch to save results to CSV file, by default false.
+ If true, output file is written to DATA_DIR in the format,
+ "res-mix_[gen_yr]_rec-[rec_handler]_agg-[agg_handler].csv".
+
+ Returns
+ -------
+ pandas.DataFrame
+ A generation mix data frame with the following columns.
+
+ - 'Subregion', the balancing authority name
+ - 'FuelCategory', the primary fuel category (e.g., SOLAR, COAL)
+ - 'Electricity', the annual generation for fuel category (MWh)
+ - 'Generation_Ratio', the mix fraction for the fuel category
+ - 'BA_CODE', the balancing authority abbreviation
+ - 'Electricity_new', the REC-free annual generation by fuel (MWh)
+ - 'Gen_Ratio_new', the REC-free mix fraction for fuel category
+ """
+ logging.info("Running residual grid mix calculation tool")
+ m_df = get_elci_mix()
+ m_df = update_mix(m_df)
+ logging.info("Complete!")
+
+ if to_save:
+ out_file = (
+ f"res-mix_{model_specs.eia_gen_year}"
+ f"_rec-{model_specs.neg_rem_method}"
+ f"_agg-{model_specs.rem_weight_method}.csv"
+ )
+ logging.info("Writing %s to file" % out_file)
+ write_csv_to_output(out_file, m_df)
+
+ return m_df
+
+
+def update_mix(df):
+ """Update a balancing authority generation mix by removing RECs.
+
+ This methods appends two new columns to the generation mix data frame
+ with REC-free electricity generation (MWh) and new mix fractions for
+ each fuel category under each balancing authority.
+
+ Notes
+ -----
+ - Fixed mis-matched balancing authorities in 'count'-based REC aggregates
+ by merging on BA_CODE field.
+ - Added check for non-green energy categories when dealing with overflow
+ generation in REC data (as compared to the baseline generation).
+ It should be further noted that the presence of one of these overflow
+ fuel categories does not preclude non-renewable fuel categories (e.g.,
+ coal, oil, or gas) from being reduced by overflowing REC-based
+ generation. This would require a third bucket to be introduced to the
+ model, such that overflow only comes from MIXED or OTHF categories.
+ Currently all non-renewable categories are reduced to make up for the
+ difference (when rec_handler is set to 'keep').
+
+ See also
+ --------
+ The README for this repository includes the pseudocode and visual aids
+ regarding this method. Variable names used in this method attempt to
+ match the syntax of the pseudocode provided.
+
+ This method relies on the following two model-configured parameters:
+
+ - rec_handler (str), option for handling REC totals that are greater
+ than the renewable energy generated by a balancing authority.
+ There are two options:
+
+ - 'keep' will subtract the excess away from the non-renewable fuel
+ categories (assuming some renewable generation in the 'mix' or
+ 'othf' categories) and maintains the math of generation totals
+ - 'zero' will floor negative renewable energy values to zero; the
+ remainder is unaccounted for
+
+ - agg_handler (str), Option for state-level to BA REC sales aggregation.
+ There is currently one option, 'count', which aggregates based on the
+ number of shared facilities.
+
+ Parameters
+ ----------
+ df : pandas.DataFrame
+ Generation mix for balancing authorities.
+ Required fields include 'Subregion', 'Electricity', 'Generation_Ratio',
+ and 'FuelCategory'.
+
+ Returns
+ -------
+ pandas.DataFrame
+ The same as the generation mix data frame sent, but with two new
+ fields.
+
+ - 'Electricity_new' is the REC-free electricity generation amount
+ (MWh) for each fuel category under each balancing authority area.
+ - 'Gen_Ratio_new' is the REC-free fractional mix for each fuel
+ category under each balancing authority area.
+
+ Raises
+ ------
+ TypeError
+ If the method does not receive a pandas data frame for df.
+ IndexError
+ If the pandas data frame, df, does not have the required fields.
+ """
+ # Basic error handling
+ if not isinstance(df, pd.DataFrame):
+ raise TypeError("Expected a pandas data frame, found %s" % type(df))
+
+ r_cols = ['Electricity', 'Generation_Ratio', 'Subregion']
+ if not all([i in df.columns for i in r_cols]):
+ raise IndexError("Data frame missing required columns!")
+
+ if model_specs.neg_rem_method == 'keep':
+ logging.info(
+ "Negative renewable energy will be taken from "
+ "non-renewable energy generation amounts.")
+ elif model_specs.neg_rem_method == 'zero':
+ logging.info("Negative renewable energy will be zeroed.")
+
+ # Initialize the new columns with existing electricity amounts and
+ # generation ratios (e.g., for regions with no renewables, these values
+ # shouldn't change).
+ df['Electricity_new'] = df['Electricity']
+ df['Gen_Ratio_new'] = df['Generation_Ratio']
+
+ # Define columns used to merge new electricity and generation mix values
+ # to our data frame (referenced in the for-loop below).
+ m_cols = ['Subregion', 'FuelCategory', 'Electricity_new', 'Gen_Ratio_new']
+
+ # Get the aggregation series and pair to BA area names
+ # NOTE: name corrections for geo BA dataframe should fix any mis-matches
+ logging.info("Using '%s' weighting method" % model_specs.rem_weight_method)
+ agg_df = get_rec_agg(model_specs.rem_weight_method, as_series=False)
+
+ for baa in df['Subregion'].unique():
+ # ~~~~~~~~~~~~~~~~
+ # NON-GREEN ENERGY
+ # ~~~~~~~~~~~~~~~~
+ # Find all non-green energy sources and set the non-green energy
+ # total (ng) and the non-REC non-green energy total (ngx).
+ ng_df = df.query(
+ "(FuelCategory not in @GREEN_E) & (Subregion == @baa)").copy()
+ ng = 0.0
+ ngx = 0.0
+ if len(ng_df) > 0:
+ ng_df = calc_relative_ratio(ng_df, add_total=True)
+ ng_df.drop(
+ ['Electricity', 'Generation_Ratio'], axis=1, inplace=True)
+ ng = ng_df['Relative_Total'].values[0]
+ ngx = ng
+
+ # ~~~~~~~~~~~~
+ # GREEN ENERGY
+ # ~~~~~~~~~~~~
+ # Find only green energy fuels for the given BA area and initialize
+ # the non-REC green energy total (gx), to zero.
+ g_df = df.query(
+ "(FuelCategory in @GREEN_E) & (Subregion == @baa)").copy()
+ gx = 0.0
+
+ # Skip BA areas with no green energy; nothing to do!
+ if len(g_df) > 0:
+ g_df = calc_relative_ratio(g_df, add_total=True)
+ g_df.drop(['Electricity', 'Generation_Ratio'], axis=1, inplace=True)
+
+ # Merge and keep index; thanks to Wouter Overmeire (2012)
+ # https://stackoverflow.com/a/11982843
+ g_df = g_df.reset_index().merge(
+ agg_df, how='left', on='BA_CODE').set_index("index")
+
+ # Pull values from data frame for green energy total (big_g)
+ # and REC energy total (rec_t) and use them to calculate the
+ # non-REC green energy (gx). NOTE: the relative total and
+ # rec frac columns are constants, so it's safe to pull just
+ # one value from the lot.
+ big_g = g_df['Relative_Total'].values[0]
+ rec_t = g_df['REC_FRAC'].values[0]
+ gx = big_g - rec_t
+
+ # NOTE: due to the categorization of "Green energy," there is a
+ # good chance for negative green generation amounts.
+ # 1. If we keep the negative amounts, when these are added back to
+ # the generation totals, we can assume that the "mix" or
+ # "other" fuels compensate ('keep' option); or
+ # 2. We can zero out the negatives ('zero' option).
+
+ # Check for negative green generation
+ if rec_t > big_g:
+ logging.info(
+ "Negative renewable energy for %s (%0.2e MWh)" % (baa, gx))
+ has_ofe = any(
+ [i in OVERFLOW_E for i in ng_df['FuelCategory'].values])
+ if model_specs.neg_rem_method == 'keep' and has_ofe:
+ # Pull the "excess" electricity from non-green
+ gx = 0.0
+ ngx = ng - (rec_t - big_g)
+ # Don't let total generation go negative
+ ngx = max(0.0, ngx)
+ else:
+ gx = 0.0
+
+ # Calculate non-REC
+ non_rec = gx + ngx
+
+ # Calculate non-REC generation amounts and ratios of each fuel type
+ if len(ng_df) > 0:
+ ng_df['Electricity_new'] = ng_df['Relative_Ratio'] * ngx
+ ng_df['Gen_Ratio_new'] = 0.0
+ if non_rec > 0:
+ ng_df['Gen_Ratio_new'] = ng_df['Electricity_new'] / non_rec
+ df.update(ng_df[m_cols], join='left', overwrite=True)
+ if len(g_df) > 0:
+ g_df['Electricity_new'] = g_df['Relative_Ratio'] * gx
+ g_df['Gen_Ratio_new'] = 0.0
+ if non_rec > 0:
+ g_df['Gen_Ratio_new'] = g_df['Electricity_new'] / non_rec
+ df.update(g_df[m_cols], join='left', overwrite=True)
+
+ return df
+
+
+#
+# SANDBOX
+#
+if __name__ == '__main__':
+ # Setup logging
+ from electricitylci.utils import get_logger
+ log = get_logger(stream=True, rfh=False)
+
+ # Setup model
+ import electricitylci.model_config as config
+ config.model_specs = config.build_model_class('ELCI_2023')
+
+ # Test new 'gen' method; six BAs with negative renewable energy.
+ from electricitylci.residual_grid_mix import get_rem
+ df = get_rem(to_save=True)
+
+ #
+ # AGG BY GEN draft
+ # See comparison of facility count and generation weights for ELCI_2023:
+ # https://edx.netl.doe.gov/resource/af61a48f-a372-458f-a0e5-c37cf387f384
+ #
+
+ # Get state and BA info for each facility.
+ ba_df = eia860_balancing_authority(year=config.model_specs.eia_gen_year)
+ ba_df.rename(columns={
+ 'Balancing Authority Name': "BA_NAME",
+ 'Balancing Authority Code': "BA_CODE",
+ "Plant Id": "FacilityID"}, inplace=True)
+
+ # Not every plant has a BA and State, so drop NAs
+ ba_df = ba_df.dropna(subset='BA_CODE')
+
+ # Fix column type for merging purposes.
+ ba_df['FacilityID'] = ba_df['FacilityID'].astype("int")
+
+ # Get facility-level generation amounts
+ from electricitylci.eia923_generation import build_generation_data
+
+ # Decision point:
+ # a) If you send the plant IDs, you get 72% of 2023 plants;
+ # --> includes hugely negative plant generations ;(
+ # b) If you don't specify plant IDs, you get 68% of 2023 plants.
+ # Probably better to go route (b) and avoid negative generation.
+ fac_gen = build_generation_data(
+ # egrid_facilities_to_include=ba_df['Plant Id'].to_list(),
+ generation_years=[config.model_specs.eia_gen_year,]
+ )
+ fac_gen = fac_gen.drop(columns='Year')
+
+ # Add annual generation to facility data (i.e., 'Electricity' column).
+ ba_df = ba_df.merge(fac_gen, on='FacilityID', how='inner')
+
+ # Create plant count tables
+ table_a = ba_df.groupby(
+ by=['State', 'BA_CODE']).agg({'Electricity': 'sum'}).Electricity
+ table_b = ba_df.groupby(
+ by='State').agg({'Electricity': 'sum'}).Electricity
+
+ table_a.name = "STBA_GEN"
+ table_b.name = "ST_GEN"
+
+ # Join these series together
+ ta_df = table_a.reset_index(drop=False)
+ tb_df = table_b.reset_index(drop=False)
+ ta_df = ta_df.merge(tb_df, how='left', on='State')
+
+ # Calculate state-level fractions;
+ # these should add to 1.0 for each state given the NA drop above.
+ ta_df["ST_FRAC"] = ta_df['STBA_GEN'] / ta_df['ST_GEN']
+
+ # Join REC data
+ rec_df = get_nrel_rec(config.model_specs.eia_gen_year)
+ rec_df = rec_df[['State', 'Total']].copy()
+ jdf = ta_df.merge(rec_df, how='left', on='State')
+
+ # Calculate BA REC amounts using facility-level generation weights
+ jdf['REC_FRAC'] = jdf['ST_FRAC'] * jdf['Total']
+
+ # Allocate RECs to their BA areas using the new relative fractions
+ # the sum of REC_FRAC should equal the sum of Total in the REC data frame
+ tot_df = jdf.groupby(by='BA_CODE').agg({'REC_FRAC': 'sum'})
diff --git a/electricitylci/solar_thermal_upstream.py b/electricitylci/solar_thermal_upstream.py
index 3e82b9a..461e05d 100644
--- a/electricitylci/solar_thermal_upstream.py
+++ b/electricitylci/solar_thermal_upstream.py
@@ -27,7 +27,7 @@
are accounted for elsewhere.
Last updated:
- 2025-01-31
+ 2026-02-27
"""
__all__ = [
"generate_upstream_solarthermal",
@@ -73,7 +73,7 @@ def get_solarthermal_construction(year):
header=[0, 1]
)
elif model_specs.renewable_vintage == 2016:
- logging.info(
+ logging.warning(
"The 2016 solar thermal LCI did not have separate construction "
"and O&M. Returning none")
return None
@@ -150,7 +150,7 @@ def get_solarthermal_construction(year):
# Fix/fill construction LCI
solarthermal_upstream = fix_renewable(
solarthermal_upstream, "netlsolarthermal")
- # Issue #296 - adding DQI information for upstream processes
+ # Issue #296 - adding DQI information for upstream processes
solarthermal_upstream["Year"] = model_specs.renewable_vintage
solarthermal_upstream["DataReliability"] = 3
solarthermal_upstream["TemporalCorrelation"] = add_temporal_correlation_score(
diff --git a/electricitylci/solar_upstream.py b/electricitylci/solar_upstream.py
index 440c45f..948f9c4 100644
--- a/electricitylci/solar_upstream.py
+++ b/electricitylci/solar_upstream.py
@@ -26,7 +26,7 @@
accounted for elsewhere.
Last updated:
- 2025-01-31
+ 2026-02-27
"""
__all__ = [
"fix_renewable",
@@ -176,7 +176,7 @@ def get_solar_pv_construction(year):
na_values=["#VALUE!", "#DIV/0!"],
)
elif model_specs.renewable_vintage == 2016:
- logging.info(
+ logging.warning(
"The 2016 solar PV LCI does not separate construction and O&M. "
"Returning none.")
return None
@@ -242,7 +242,7 @@ def get_solar_pv_construction(year):
solar_upstream["input"] = False
solar_upstream = fix_renewable(solar_upstream, "netlnrelsolarpv")
- # Issue #296 - adding DQI information for upstream processes
+ # Issue #296 - adding DQI information for upstream processes
solar_upstream["Year"] = model_specs.renewable_vintage
solar_upstream["DataReliability"] = 3
solar_upstream["TemporalCorrelation"] = add_temporal_correlation_score(
diff --git a/electricitylci/upstream_dict.py b/electricitylci/upstream_dict.py
index 0ce6674..a7f9732 100644
--- a/electricitylci/upstream_dict.py
+++ b/electricitylci/upstream_dict.py
@@ -8,20 +8,19 @@
##############################################################################
import logging
-from electricitylci.coal_upstream import (
- coal_type_codes,
- mine_type_codes,
- basin_codes,
-)
+from electricitylci.globals import COAL_TYPE_CODES
+from electricitylci.globals import COAL_BASIN_CODES
+from electricitylci.globals import COAL_MINE_CODES
from electricitylci import write_process_dicts_to_jsonld
from electricitylci.process_dictionary_writer import (
process_doc_creation,
process_description_creation
)
from electricitylci.utils import make_valid_version_num
+from electricitylci.utils import find_upstream_location
from electricitylci.globals import elci_version
# Issue #150, need Balancing Authority names for regional construction
-from electricitylci.eia860_facilities import eia860_balancing_authority
+from electricitylci.eia860_facilities import add_balancing_authorities_to_plants
import electricitylci.model_config as config
@@ -35,7 +34,7 @@
processing, and transport, and power plant construction.
Last updated:
- 2025-06-09
+ 2026-03-05
"""
__all__ = [
"olcaschema_genupstream_processes",
@@ -94,6 +93,29 @@ def _exchange_table_creation_output(data):
def _exchange_table_creation_ref(fuel_type):
+ """Helper function for defining the quantitative reference flows for
+ upstream processes (e.g., natural gas transmission, processes and
+ transported coal, nuclear fuel production, and power plant construction)
+
+ Parameters
+ ----------
+ fuel_type : str
+ The fuel type (e.g., 'OIL', 'GAS', 'NUCLEAR, 'Coal transport', or 'CONSTRUCTION').
+
+ Returns
+ -------
+ dict
+ An olca-schema formatted dictionary for an Exchange object.
+
+ Notes
+ -----
+ This method is responsible for defining functional units of upstream
+ processes.
+
+ Includes undefined upstream flows for processes not found in the
+ electricity baseline (e.g., geothermal, solar & wind upstream and
+ operation).
+ """
natural_gas_flow = {
"flowType": "PRODUCT_FLOW",
"flowProperties": "",
@@ -155,7 +177,8 @@ def _exchange_table_creation_ref(fuel_type):
"2371: Utility System Construction")
}
- # The following link provides the undefined variables:
+ # The following link provides the undefined variables (i.e, the None
+ # placeholders for GEOTHERMAL, SOLAR, and WIND below):
# https://github.com/KeyLogicLCA/ElectricityLCI/commit/f61d28a3d0cf5b0ef61ca147f870e15a863f8ec3
ar = dict()
ar["internalId"] = ""
@@ -184,17 +207,17 @@ def _exchange_table_creation_ref(fuel_type):
# NOTE: presently, there are no upstream processes for these
elif fuel_type == "GEOTHERMAL":
logging.warning("Undefined geothermal flow")
- ar["flow"] = geothermal_flow
+ ar["flow"] = None # geothermal_flow
ar["unit"] = _unit("MWh")
ar["amount"] = 1
elif fuel_type == "SOLAR":
logging.warning("Undefined solar flow")
- ar["flow"] = solar_flow
+ ar["flow"] = None # solar_flow
ar["unit"] = _unit("Item(s)")
ar["amount"] = 1
elif fuel_type == "WIND":
logging.warning("Undefined wind flow")
- ar["flow"] = wind_flow
+ ar["flow"] = None # wind_flow
ar["unit"] = _unit("Item(s)")
ar["amount"] = 1
# END NOTE
@@ -328,7 +351,8 @@ def _process_table_creation_gen(process_name, exchanges_list, fuel_type):
ar["allocationFactors"] = ""
ar["defaultAllocationMethod"] = ""
ar["exchanges"] = exchanges_list
- ar["location"] = "" # location(region)
+ # Issue 327: apply locations to upstream processes [26.03.05; TWD]
+ ar["location"] = find_upstream_location(process_name, fuel_type)
ar["parameters"] = ""
logging.debug(
@@ -386,17 +410,29 @@ def olcaschema_genupstream_processes(merged):
----------
merged: pandas.DataFrame
Data frame containing the inventory for upstream processes used by
- electricity generation.
+ electricity generation (e.g., ``upstream_df`` produced by
+ :func:`get_upstream_process_df`).
Returns
- ----------
+ -------
dict
Dictionary containing all of the unit processes to be written to
JSON-LD for import to openLCA.
+
+ Notes
+ -----
+ This method is responsible for naming upstream processes such as:
+
+ - "petroleum extraction and processing - PADD"
+ - "coal extraction and processing - basin/coal/mine"
+ - "natural gas extraction, processing, and transport - basin"
+ - "nuclear fuel extraction, processing, and transport"
+ - "coal transport - stage code"
+ - "power plant construction - fuel - US Average/region"
"""
- coal_type_codes_inv = dict(map(reversed, coal_type_codes.items()))
- mine_type_codes_inv = dict(map(reversed, mine_type_codes.items()))
- basin_codes_inv = dict(map(reversed, basin_codes.items()))
+ coal_type_codes_inv = dict(map(reversed, COAL_TYPE_CODES.items()))
+ mine_type_codes_inv = dict(map(reversed, COAL_MINE_CODES.items()))
+ basin_codes_inv = dict(map(reversed, COAL_BASIN_CODES.items()))
# Hotfix: add 'Belt' to coal transport list [12/16/2024;MBJ]
# NOTE: I don't think the belt inventory is actually used anywhere
coal_transport = [
@@ -407,8 +443,15 @@ def olcaschema_genupstream_processes(merged):
"Truck",
"Belt",
]
- # First going to keep plant IDs to account for possible emission repeats
- # for the same compartment, leading to erroneously low emission factors
+
+ # Set data types for flow amount, quantity, plant id [26.02.12;TWD]
+ merged['FlowAmount'] = merged['FlowAmount'].astype("float")
+ merged['quantity'] = merged['quantity'].astype("float")
+ merged['plant_id'] = merged['plant_id'].astype("int32")
+
+ # First keep plant IDs to account for possible emission repeats for the
+ # same compartment, leading to erroneously low emission factors;
+ # NOTE: quantity is averaged here.
merged_summary = merged.groupby(
by=[
"FuelCategory",
@@ -423,26 +466,23 @@ def olcaschema_genupstream_processes(merged):
],
as_index=False,
).agg({"FlowAmount": "sum", "quantity": "mean"})
- # NEW Issue #150, adding regional ability for construction of all types
- plant_region = eia860_balancing_authority(config.model_specs.eia_gen_year)
- plant_region["Plant Id"] = plant_region["Plant Id"].astype("int32")
- merged_summary_regional = (
- merged_summary.loc[
- merged_summary["FuelCategory"].str.contains("CONSTRUCTION"), :
- ]
- .merge(
- plant_region, how="left", left_on="plant_id", right_on="Plant Id"
- )
- .copy().reset_index()
+
+ # Adding regional ability for construction of all types (Issue #150)
+ const_filter = merged_summary['FuelCategory'].str.contains("CONSTRUCTION")
+ merged_summary_regional = merged_summary.loc[const_filter, :].copy()
+
+ # NEW: add balancing authority codes to plants [26.02.12;TWD]
+ merged_summary_regional, plant_dict = add_balancing_authorities_to_plants(
+ merged_summary_regional,
+ "plant_id",
+ config.model_specs.eia_gen_year
)
- plant_region_dict = {
- x[0]: x[1]
- for x in zip(
- plant_region["Plant Id"],
- plant_region["Balancing Authority Name"],
- )
- }
- merged_summary = merged_summary.groupby(
+
+ # Issue #150, adding regional ability for construction of all types
+ # NOTE: Added "Balancing Authority Name" below to enable regionalized
+ # construction processes. Only for balancing authorities now but should
+ # be expanded to eGRID, etc.
+ merged_summary_regional = merged_summary_regional.groupby(
by=[
"FuelCategory",
"stage_code",
@@ -452,15 +492,20 @@ def olcaschema_genupstream_processes(merged):
"ElementaryFlowPrimeContext",
"Unit",
"input",
+ "Balancing Authority Name",
],
as_index=False,
- )[["quantity", "FlowAmount"]].sum()
+ ).agg({"quantity": "sum", "FlowAmount": "sum"})
- # Issue #150, adding regional ability for construction of all types
- # NOTE: Added "Balancing Authority Name" below to enable regionalized
- # construction processes. Only for balancing authorities now but should
- # be expanded to eGRID, etc.
- merged_summary_regional = merged_summary_regional.groupby(
+ # Create the new regional process short names
+ merged_summary_regional["scen_name"] = (
+ merged_summary_regional["stage_code"]
+ + " - "
+ + merged_summary_regional["Balancing Authority Name"]
+ )
+
+ # Next, drop plant ID to get a US-average.
+ merged_summary = merged_summary.groupby(
by=[
"FuelCategory",
"stage_code",
@@ -470,30 +515,30 @@ def olcaschema_genupstream_processes(merged):
"ElementaryFlowPrimeContext",
"Unit",
"input",
- "Balancing Authority Name",
],
as_index=False,
- )[["quantity", "FlowAmount"]].sum()
+ ).agg({"quantity": "sum", "FlowAmount": "sum"})
+
# For natural gas extraction there are extraction and transportation stages
# that will get lumped together in the groupby which will double
# the quantity and erroneously lower emission rates.
# Calculate emission factor (e.g., total emission per total MWh)
- merged_summary["FlowAmount"]=merged_summary["FlowAmount"].astype(float)
- merged_summary["quantity"]=merged_summary["quantity"].astype(float)
-
merged_summary["emission_factor"] = (
merged_summary["FlowAmount"] / merged_summary["quantity"]
)
+
+ # NOTE: In ELCI_2023 all NaNs are from OIL facilities (RFO_4 & RFO_5)
merged_summary.dropna(subset=["emission_factor"], inplace=True)
- # NEW Issue #150, adding regional ability for construction of all types
- merged_summary_regional["FlowAmount"]=merged_summary_regional["FlowAmount"].astype(float)
- merged_summary_regional["quantity"]=merged_summary_regional["quantity"].astype(float)
+ # Adding regional ability for construction of all types (Issue #150)
merged_summary_regional["emission_factor"] = (
- merged_summary_regional["FlowAmount"] / merged_summary_regional["quantity"]
+ merged_summary_regional["FlowAmount"]
+ / merged_summary_regional["quantity"]
)
+ # NOTE: In ELCI_2023, found no NaN emission factors
merged_summary_regional.dropna(subset=["emission_factor"], inplace=True)
+
# Make upstream processes for each stage code and save to a dictionary.
upstream_process_dict = dict()
upstream_list = [x for x in merged_summary["stage_code"].unique()]
@@ -502,21 +547,21 @@ def olcaschema_genupstream_processes(merged):
exchanges_list = list()
upstream_filter = merged_summary["stage_code"] == upstream
- merged_summary_filter = merged_summary.loc[upstream_filter, :].copy()
+ match_filter = merged_summary["FlowName"] != "[no match]"
+ merged_summary_filter = merged_summary.loc[
+ (upstream_filter) & (match_filter), :
+ ].copy()
merged_summary_filter.drop_duplicates(
subset=["FlowName", "Compartment", "FlowAmount"],
inplace=True
)
merged_summary_filter.dropna(subset=["FlowName"], inplace=True)
- # TODO: where does "[no match]" get set? FEDEFL mapper?
- garbage = merged_summary_filter.loc[
- merged_summary_filter["FlowName"] == "[no match]", :].index
- merged_summary_filter.drop(garbage, inplace=True)
- # Issue 296 - changes to get data quality scores into final
- # dictionaries
+ # Issue 296 - changes to get data quality scores for flows into the
+ # final dictionaries.
merged_summary_filter = merged_summary_filter.merge(
- merged.loc[merged["stage_code"]==upstream,
+ merged.loc[
+ merged["stage_code"]==upstream,
[
"FlowUUID",
"DataCollection",
@@ -533,9 +578,10 @@ def olcaschema_genupstream_processes(merged):
_exchange_table_creation_output, axis=1).tolist()
exchanges_list.extend(ra)
+ # Extract fuel category for the current stage code
first_row = min(merged_summary_filter.index)
fuel_type = merged_summary_filter.loc[first_row, "FuelCategory"]
- stage_code = merged_summary_filter.loc[first_row, "stage_code"]
+ stage_code = upstream
if (fuel_type == "COAL") & (stage_code not in coal_transport):
split_name = merged_summary_filter.loc[
@@ -556,8 +602,9 @@ def olcaschema_genupstream_processes(merged):
_exchange_table_creation_ref("Coal transport")
)
elif fuel_type == "GAS":
+ # HOTFIX: address Issue 320 [26.02.06; TWD]
combined_name = (
- "natural gas extraction and processing - "
+ "natural gas extraction, processing, and transport - "
+ merged_summary_filter.loc[first_row, "stage_code"]
)
exchanges_list.append(_exchange_table_creation_ref(fuel_type))
@@ -587,9 +634,12 @@ def olcaschema_genupstream_processes(merged):
# Issue #150, catching multiple types of CONSTRUCTION. Worth noting that
# US average is appended to these in case they're needed (unlikely)
elif "CONSTRUCTION" in fuel_type:
- combined_name= f"power plant construction - {stage_code} - US Average"
+ combined_name = (
+ f"power plant construction - {stage_code} - US Average"
+ )
exchanges_list.append(_exchange_table_creation_ref(fuel_type))
+ # Create the process-level dictionary
process_name = f"{combined_name}"
if (fuel_type == "COAL") & (stage_code in coal_transport):
final = _process_table_creation_gen(
@@ -599,62 +649,63 @@ def olcaschema_genupstream_processes(merged):
final = _process_table_creation_gen(
process_name, exchanges_list, fuel_type
)
- upstream_process_dict[
- merged_summary_filter.loc[first_row, "stage_code"]
- ] = final
- # New, Issue #150 - adding regional construction profiles.
- merged_summary_regional["scen_name"] = (
- merged_summary_regional["stage_code"]
- + " - "
- + merged_summary_regional["Balancing Authority Name"]
- )
+ # Append process dictionary to master process dictionary
+ upstream_process_dict[stage_code] = final
+
+ # REGIONAL PROCESSES
# Issue 296 - changes to get data quality scores into final
- # dictionaries
- small_merged=merged[[
- "FlowUUID",
- "DataCollection",
- "TemporalCorrelation",
- "GeographicalCorrelation",
- "TechnologicalCorrelation",
- "DataReliability",
- "plant_id",
- "stage_code"
- ]
- ].copy()
- small_merged["plant_id"]=small_merged["plant_id"].astype("int32")
+ # dictionaries; start with a copy of merged w/ BA info appended.
+ # NOTE: The US average processes are under their stage code dict key
+ # (e.g., 'coal_const' or 'solar_thermal_const'), whereas the regional
+ # processes have their region name in the dict key
+ # (e.g., 'wind_const - Tuscon Electric Power').
+ small_merged = merged[[
+ "FlowUUID",
+ "DataCollection",
+ "TemporalCorrelation",
+ "GeographicalCorrelation",
+ "TechnologicalCorrelation",
+ "DataReliability",
+ "plant_id",
+ "stage_code"
+ ]].copy()
small_merged["Balancing Authority Name"] = small_merged["plant_id"].map(
- plant_region_dict
+ plant_dict
)
- small_merged["scen_name"]=(
+ # Create the same process short name as was done for regional summary.
+ small_merged["scen_name"] = (
small_merged["stage_code"]
+ " - "
+ small_merged["Balancing Authority Name"]
)
- small_merged=small_merged.drop_duplicates(subset=["scen_name","FlowUUID"])
- upstream_regional_list = [x for x in merged_summary_regional["scen_name"].unique()]
+ small_merged = small_merged.drop_duplicates(
+ subset=["scen_name", "FlowUUID"]
+ )
+
+ upstream_regional_list = merged_summary_regional[
+ "scen_name"].unique().tolist()
+
for upstream in upstream_regional_list:
logging.info(f"Building dictionary for {upstream}")
exchanges_list = list()
upstream_filter = merged_summary_regional["scen_name"] == upstream
- merged_summary_filter = merged_summary_regional.loc[upstream_filter, :].copy()
+ match_filter = merged_summary_regional["FlowName"] != "[no match]"
+ merged_summary_filter = merged_summary_regional.loc[
+ (upstream_filter) & (match_filter), :].copy()
merged_summary_filter.drop_duplicates(
subset=["FlowName", "Compartment", "FlowAmount"],
inplace=True
)
merged_summary_filter.dropna(subset=["FlowName"], inplace=True)
- # TODO: where does "[no match]" get set? FEDEFL mapper?
- garbage = merged_summary_filter.loc[
- merged_summary_filter["FlowName"] == "[no match]", :].index
- merged_summary_filter.drop(garbage, inplace=True)
# Issue 296 - changes to get data quality scores into final
# dictionaries
merged_summary_filter = merged_summary_filter.merge(
- small_merged.loc[small_merged["scen_name"]==upstream,:],
+ small_merged.loc[small_merged["scen_name"] == upstream, :],
on=["FlowUUID"],
how="left",
- suffixes=["","_right"]
+ suffixes=["", "_right"]
)
ra = merged_summary_filter.apply(
_exchange_table_creation_output, axis=1).tolist()
@@ -664,6 +715,7 @@ def olcaschema_genupstream_processes(merged):
fuel_type = merged_summary_filter.loc[first_row, "FuelCategory"]
if "CONSTRUCTION" in fuel_type:
+ # NOTE: This is the new regional construction process name
combined_name = f"power plant construction - {upstream}"
exchanges_list.append(_exchange_table_creation_ref(fuel_type))
diff --git a/electricitylci/utils.py b/electricitylci/utils.py
index ac0f4ce..7cc2e37 100644
--- a/electricitylci/utils.py
+++ b/electricitylci/utils.py
@@ -25,6 +25,8 @@
from electricitylci.globals import output_dir
from electricitylci.globals import API_SLEEP
from electricitylci.globals import CAM_API_URL
+from electricitylci.globals import NREL_REC_URL
+from electricitylci.globals import COAL_BASIN_CODES
##############################################################################
@@ -33,9 +35,14 @@
__doc__ = """Small utility functions for use throughout the repository.
Last updated:
- 2025-08-27
+ 2026-03-26
Changelog:
+ - [26.03.26]: Fix long subfolder paths in archive background data
+ - [26.03.17]: Update stewi inventory years
+ - [26.02.10]: New filter out zero helper function
+ - [26.01.28]: Allow resetting log levels
+ - [25.12.12]: Add NREL REC data handler
- [25.08.27]: Update archive EPA CAMS method
- [25.08.13]: Move check API utility function here
- [25.08.01]: Read line from file helper method
@@ -60,12 +67,18 @@
"download",
"download_unzip",
"fill_default_provider_uuids",
+ "filter_out_zero",
"find_file_in_folder",
+ "find_upstream_location",
+ "find_worksheet_header_row",
+ "get_ba_map",
"get_logger",
+ "get_nrel_rec",
"get_stewi_invent_years",
"join_with_underscore",
"linear_search",
"make_valid_version_num",
+ "map_ba_codes",
"next_month",
"read_line_from_file",
"read_ba_codes",
@@ -361,7 +374,13 @@ def archive_background_data(save_folder="background"):
Parameters
----------
save_folder : str, optional
- The output folder to store the archives, by default "background"
+ The folder name to create in the electricitylci output directory to
+ store the archives, by default "background"
+
+ Examples
+ --------
+ >>> from electricitylci.utils import archive_background_data
+ >>> archive_background()
"""
ds = _init_data_store()
to_skip = ['archive', 'hidden', 'output']
@@ -399,13 +418,24 @@ def archive_background_data(save_folder="background"):
# Archive subfolder files into own ZIP (e.g., stewi.flowbyfacility.zip)
_, sub_folders = _get_non_hidden(cur_path, to_skip)
for sub_folder in sub_folders:
- sub_name = os.path.basename(sub_folder)
+ # This is the folder name from ds.keys().
sub_zip_name = os.path.basename(cur_path)
- sub_zip_name += "."
- sub_zip_name += sub_name
+ # This is the sub-folder node we are archiving.
+ sub_name = os.path.basename(sub_folder)
+
+ # Replace the upstream path with just the current folder
+ sub_zip_name = sub_folder.replace(cur_path, sub_zip_name)
+
+ # Replace folder sep with dot
+ sub_zip_name = sub_zip_name.replace(os.path.sep, ".")
+
+ # Name as .zip file and locate it in outputs
sub_zip_name += ".zip"
sub_zip_path = os.path.join(output_path, sub_zip_name)
+ # BUG: this fails to parse sub-sub and sub-sub-sub folder files;
+ # they all show up in the sub zip file. [26.03.27; TWD]
+ # Consider a loop with root=true until all files are accounted for.
sub_files, _ = _get_non_hidden(sub_folder, to_skip)
with zipfile.ZipFile(sub_zip_path, 'w', zipfile.ZIP_DEFLATED) as z:
for filepath in sub_files:
@@ -1082,6 +1112,44 @@ def fill_default_provider_uuids(dict_to_fill, *args):
return dict_to_fill
+def filter_out_zero(df, col_name):
+ """Filter all rows with zero value in a given column from a pandas data
+ frame.
+
+ Parameters
+ ----------
+ df : pandas.DataFrame
+ A data frame that must have a column, ``col_name`` that is numeric.
+ col_name : str
+ A column name in ``df`` that may have zero values that correspond to
+ rows that are unwanted.
+
+ Returns
+ -------
+ pandas.DataFrame
+ The same data frame that is sent where rows with a zero value in
+ ``col_name`` are dropped.
+
+ Raises
+ ------
+ IndexError
+ If the column name does not exist in the data frame.
+ TypeError
+ If the column's data type is not numeric (must be able to have a zero
+ value).
+ """
+ if col_name not in df.columns:
+ raise IndexError("Column, '%s', not in data frame!" % col_name)
+ if not pd.api.types.is_numeric_dtype(df[col_name]):
+ raise TypeError("Column, '%s', is not numeric!" % col_name)
+
+ zero_filter = df[col_name] != 0
+ logging.debug(
+ "Dropping %d rows from column, '%s'" % (zero_filter.sum(), col_name)
+ )
+ return df.loc[zero_filter, :]
+
+
def find_file_in_folder(folder_path, file_pattern_match, return_name=True):
"""Search a folder for files matching a pattern.
@@ -1134,6 +1202,163 @@ def find_file_in_folder(folder_path, file_pattern_match, return_name=True):
return (file_path, file_name)
+def find_upstream_location(process_name, fuel_type):
+ """Helper function to extract regional names from upstream processes
+ including coal, oil, gas, nuclear, and construction.
+
+ Parameters
+ ----------
+ process_name : str
+ A process name (e.g., 'natural gas extraction, processing, and transport - Northeast')
+ fuel_type : str
+ A fuel type (e.g., 'GAS')
+
+ Returns
+ -------
+ str
+ Location string (e.g., 'US', 'Northeast', or 'RFO PADD 5').
+
+ Notes
+ -----
+ This method is specifically designed to work with eLCI process names,
+ not ILCD process names. At time of writing, ILCD naming is assumed to
+ a post-processing step (configurable in the model specs YAML) that
+ changes process names without changing UUIDs that are built on the
+ original eLCI process names.
+
+ See reference to method in ``_process_table_creation_gen`` in
+ upstream_dict.py.
+
+ Examples
+ --------
+ >>> find_upstream_location(
+ ... 'power plant construction - solar_thermal_const - US Average',
+ ... 'SOLARTHERM_CONSTRUCTION')
+ 'US'
+ >>> find_upstream_location(
+ ... 'natural gas extraction, processing, and transport - Northeast',
+ ... 'GAS')
+ 'Northeast'
+ """
+ match fuel_type:
+ case 'coal_transport' | 'NUCLEAR' | 'OIL':
+ return 'US'
+ case 'COAL':
+ # Search across basin names
+ for basin_name in COAL_BASIN_CODES:
+ if basin_name in process_name:
+ return basin_name
+ case 'GAS':
+ # Take the words after the last hyphen.
+ return process_name.rpartition('-')[-1].strip()
+ case name if name.endswith('CONSTRUCTION'):
+ # Note that some BA names have hyphens, so stop at the first two.
+ loc_parts = process_name.split("-", 2)
+ # Clean up the location name
+ loc_name = loc_parts[2].strip() if len(loc_parts) > 2 else ""
+ # Correct for national average construction
+ if loc_name == 'US Average':
+ loc_name = 'US'
+ return loc_name
+ case _:
+ # All others will be treated thusly.
+ return ""
+
+
+def find_worksheet_header_row(workbook, worksheet, keywords, num_rows_to_scan):
+ """Dynamically scans the top rows of an Excel worksheet to find the
+ header row based on a list of identifying keywords.
+
+ This function reads a limited number of rows (num_rows_to_scan) without a
+ header, iterates through them, and checks if all provided keywords are
+ present in any single row's content. It is designed to handle Excel files
+ where the header row position changes across different vintages.
+
+ Parameters
+ ----------
+ workbook : str
+ The file path to the Excel workbook (.xlsx, .xls, etc.).
+ worksheet : str, int
+ The name (str) or index (int) of the worksheet to scan.
+ keywords : list
+ A list of required column names (or parts of names) that uniquely
+ identify the correct header row. Matching is case-insensitive.
+ num_rows_to_scan : int
+ The maximum number of initial rows to read from the worksheet to
+ search for the header.
+
+ Returns
+ -------
+ int
+ The 0-based row index of the detected header, or -1 if the header
+ was not found within the scanned range or if an error occurred.
+
+ Examples
+ --------
+ >>> wb = 'data.xlsx'
+ >>> ws = 'Sheet1'
+ >>> required_cols = ['Year', 'Emissions', 'Region']
+ >>> idx = find_worksheet_header_row(wb, ws, required_cols, 20)
+
+ Notes
+ -----
+ This method was built using Gemini AI.
+ """
+ header_row_index = -1
+
+ try:
+ # A temporary slice of the worksheet.
+ df_temp = pd.read_excel(
+ workbook,
+ header=None,
+ sheet_name=worksheet,
+ nrows=num_rows_to_scan
+ )
+ for idx, row in df_temp.iterrows():
+ # Convert to single, lowercase string for keyword checking
+ row_content = ' '.join(
+ row.dropna().astype(str).str.lower().tolist()
+ )
+ if all(keyword.lower() in row_content for keyword in keywords):
+ header_row_index = idx
+ break
+
+ if header_row_index == -1:
+ logging.error(
+ "Header row not found in the first %d rows!" % (
+ num_rows_to_scan
+ )
+ )
+ except FileNotFoundError:
+ logging.error("Failed to find Excel workbook!")
+ except Exception as e:
+ logging.error("Unexpected error, %s" % str(e))
+
+ return header_row_index
+
+
+def get_ba_map():
+ """Return a dictionary of balancing authority names and their abbreviations.
+
+ Parameters
+ ----------
+ year : int
+ The year for eLCI generation data.
+
+ Returns
+ -------
+ dict
+ A dictionary with keys of balancing authority names (as per EIA 923)
+ and values of abbreviations.
+ """
+ ba_codes = read_ba_codes()
+ ba_map = {}
+ for idx, row in ba_codes.iterrows():
+ ba_map[row['BA_Name']] = idx
+
+ return ba_map
+
+
def get_logger(stream=True, rfh=True, str_lv='INFO', rfh_lv='DEBUG'):
"""A helper function for creating or retrieving a root logger with
only one instance of stream and/or rotating file handler.
@@ -1175,8 +1400,10 @@ def get_logger(stream=True, rfh=True, str_lv='INFO', rfh_lv='DEBUG'):
for h in log.handlers:
if h.name == 'elci_stream':
has_stream = True
+ h.setLevel(str_lv) # handle level change requests
elif h.name == 'elci_rfh':
has_rfh = True
+ h.setLevel(rfh_lv)
# Create stream handler for info messages
if stream and not has_stream:
@@ -1209,6 +1436,98 @@ def get_logger(stream=True, rfh=True, str_lv='INFO', rfh_lv='DEBUG'):
return log
+def get_nrel_rec(year):
+ """Create state-level voluntary green power generation (MWh) data frame.
+
+ Notes
+ -----
+ Data are based on the NLR Green Power Data by State (2013-2023)[1]_.
+ Estimates are based on green power generated in each state, regardless of
+ where the renewable energy certificate (REC) is retired.
+ Some state-level totals do not add up to market-wide totals because some
+ green power is purchased from Canada.
+
+ There is an estimated 192.1 million MWh sold in 2020.
+
+ [1] E. O'Shaughnessy, S. Jena, and D. Salyer. 2024. Status and Trends in
+ the Voluntary Market (2023 Data). Golden, CO: NLR.
+
+ See also
+ --------
+ 1. https://www.nlr.gov/analysis/renewable-power
+ 2. https://data.nlr.gov/submissions/174
+
+ Parameters
+ ----------
+ year : int
+ The year for REC sales data.
+
+ Returns
+ -------
+ pandas.DataFrame
+ A data frame with state-based green electricity generated (MWh).
+ The "State" column provides the two-letter U.S. state name
+ abbreviations and the "Total" column provides the total green
+ electricity generated and sold as a REC in MWh.
+
+ Other columns include:
+ - "Year" (matches the year provided)
+ - "Utility Green Pricing"
+ - "Utility Renewable Contracts"
+ - "Competitive Suppliers"
+ - "Unbundled RECs"
+ - "CCAs" (community choice aggregations)
+ - "PPAs" (power purchase agreements)
+
+ Examples
+ --------
+ >>> my_df = get_rec(2023)
+ """
+ # Make sure year is an integer and in the valid range
+ if not isinstance(year, int):
+ raise TypeError("Year should be an integer not, %s" % type(year))
+ if year not in range(2016, 2025, 1):
+ raise ValueError("Year should be between 2016-2024, not %d" % year)
+
+ # Define the NLR data store
+ nrel_dir = os.path.join(paths.local_path, "nlr")
+ if check_output_dir(nrel_dir):
+ logging.debug("NLR data store exists")
+
+ # Define the NLR REC Excel workbook file path
+ rec_name = os.path.basename(NREL_REC_URL)
+ rec_path = os.path.join(nrel_dir, rec_name)
+
+ # If file does not exist, download Excel workbook
+ if not os.path.exists(rec_path):
+ download(NREL_REC_URL, rec_path)
+
+ if not os.path.exists(rec_path):
+ raise OSError(
+ "Failed to download NLR Voluntary Renewable Procurement workbook!"
+ )
+
+ # Dynamically find header row (it may change depending on the workbook year)
+ header_row = find_worksheet_header_row(
+ rec_path,
+ "State-Level Generation",
+ ["state", "year", "PPAs", "total"],
+ 20
+ )
+
+ # Sheet name, header, and index are based on examining the file.
+ df = pd.read_excel(
+ rec_path,
+ sheet_name="State-Level Generation",
+ header=header_row,
+ index_col=None
+ )
+ # Ensure year is integer for comparison purposes
+ df["Year"] = df["Year"].astype(int)
+ df = df.loc[df["Year"]==year]
+ return df
+
+
def get_stewi_invent_years(year):
"""Helper function to return inventory years of interest from StEWI.
See https://github.com/USEPA/standardizedinventories for inventory names
@@ -1250,10 +1569,10 @@ def get_stewi_invent_years(year):
STEWI_DATA_VINTAGES = {
# 'DMR': [x for x in range(2011, 2023, 1)],
# 'GHGRP': [x for x in range(2011, 2023, 1)],
- 'eGRID': [2014, 2016, 2018, 2019, 2020, 2021],
+ 'eGRID': [2014, 2016, 2018, 2019, 2020, 2021, 2022, 2023],
'NEI': [2011, 2014, 2017, 2020],
- 'RCRAInfo': [x for x in range(2011, 2023, 2)],
- 'TRI': [x for x in range(2011, 2023, 1)],
+ 'RCRAInfo': [x for x in range(2011, 2024, 2)],
+ 'TRI': [x for x in range(2011, 2024, 1)],
}
r_dict = {}
@@ -1356,6 +1675,32 @@ def make_valid_version_num(foo):
return result
+def map_ba_codes(df):
+ """Map balancing authority abbreviation codes based on EIA Form 930 naming.
+
+ Parameters
+ ----------
+ df : pandas.DataFrame
+ A data frame with column, 'Subregion' or 'BA_NAME' used to match
+ against balancing authority abbreviation map.
+
+ Returns
+ -------
+ pandas.DataFrame
+ The same as the sent data frame with a new column, "BA_CODE".
+ """
+ m_col = 'Subregion'
+ if 'Subregion' not in df.columns and 'BA_NAME' in df.columns:
+ m_col = 'BA_NAME'
+ elif 'Subregion' not in df.columns and 'BA_NAME' not in df.columns:
+ logging.warning("No matching column for BA codes!")
+
+ ba_map = get_ba_map()
+ df['BA_CODE'] = df[m_col].map(ba_map)
+ logging.info("%d mis-matched BA codes" % df['BA_CODE'].isna().sum())
+ return df
+
+
def next_month(dt0):
"""Move a datetime object to the first day of the next month.
@@ -1463,7 +1808,7 @@ def read_ba_codes():
table, which includes a comprehensive list of balancing authorities, see
https://www.eia.gov/electricity/930-content/EIA930_Reference_Tables.xlsx
- Referenced in combinatory.py, eia_io_trading.py, and import_impacts.py
+ Referenced in combinator.py, eia_io_trading.py, and import_impacts.py
and is utilized elsewhere (e.g., via importing `BA_CODES` from combinator).
Returns
@@ -1538,7 +1883,7 @@ def read_ba_codes():
"Alaska": "AK",
"Hawaii": "HI",
}
- logging.info("Reading EIA930 reference table")
+ logging.debug("Reading EIA930 reference table")
df = pd.read_excel(data_path_local)
df = df.rename(columns={
'BA Code': 'BA_Acronym',
@@ -1812,7 +2157,7 @@ def set_dir(directory):
return directory
-def write_csv_to_output(f_name, data):
+def write_csv_to_output(f_name, data, to_zip=False, add_index=False):
"""Write data to CSV file in the outputs directory.
Parameters
@@ -1823,31 +2168,62 @@ def write_csv_to_output(f_name, data):
A data object to be written to file.
A data frame is written using `to_csv` without index.
A string is written to a plain text file.
+ to_zip : bool, optional
+ Whether to compress the output data using ZIP format.
+ Defaults to false.
+ add_index : bool, optional
+ Whether to include a data frame's index in the output CSV.
+ Defaults to false.
Raises
------
TypeError : If the data type is not recognized.
"""
- f_path = os.path.join(output_dir, f_name)
- if os.path.isfile(f_path):
- logging.warning("File exists! Overwriting %s" % f_path)
+ fpath = os.path.join(output_dir, f_name)
+
+ if os.path.isfile(fpath):
+ logging.warning("File exists! Overwriting %s" % fpath)
+
if isinstance(data, pd.DataFrame):
- try:
- data.to_csv(f_path, index=False)
- except:
- logging.error("Failed to write '%s' to file" % f_path)
+ if to_zip:
+ if not fpath.endswith('.zip'):
+ fpath += ".zip"
+ try:
+ data.to_csv(
+ fpath, encoding="utf-8", compression="zip", index=add_index)
+ except:
+ raise
+ else:
+ logging.debug("Saved dataframe to zip.")
else:
- logging.info(
- "Wrote %d lines to file, %s" % (len(data), f_path)
- )
+ try:
+ data.to_csv(fpath, index=add_index, encoding="utf-8")
+ except:
+ raise
+ else:
+ logging.debug("Saved dataframe to CSV.")
elif isinstance(data, str):
- try:
- with open(f_path, 'w') as f:
- f.write(data)
- except:
- logging.error("Failed to write '%s' to file" % f_path)
+ if to_zip:
+ if not fpath.endswith(".zip"):
+ fpath += ".zip"
+ try:
+ with zipfile.ZipFile(fpath,
+ 'w',
+ compression=zipfile.ZIP_DEFLATED,
+ compresslevel=3) as z:
+ z.write(data, arcname=os.path.basename(fpath))
+ except:
+ raise
+ else:
+ logging.debug("Saved data to zip.")
else:
- logging.info("Wrote data to file, %s" % f_path)
+ try:
+ with open(fpath, 'w') as f:
+ f.write(data)
+ except:
+ raise
+ else:
+ logging.debug("Saved data to CSV.")
else:
logging.error("Data type, %s, not recognized!" % type(data))
raise TypeError("Data type, %s, not recognized!" % type(data))
diff --git a/electricitylci/wind_upstream.py b/electricitylci/wind_upstream.py
index 003c0cc..9ed8f93 100644
--- a/electricitylci/wind_upstream.py
+++ b/electricitylci/wind_upstream.py
@@ -26,10 +26,9 @@
contributions.
Last updated:
- 2025-01-31
+ 2026-03-24
"""
__all__ = [
- "aggregate_wind",
"generate_upstream_wind",
"get_wind_construction",
"get_wind_generation",
@@ -76,7 +75,7 @@ def get_wind_construction(year):
low_memory=False,
)
elif model_specs.renewable_vintage == 2016:
- logging.info(
+ logging.warning(
"The 2016 wind LCI does not separate construction and O&M."
"Returning none.")
return None
@@ -117,6 +116,9 @@ def get_wind_construction(year):
'FlowAmount': float,
})
+ # NOTE: wind generation data reflects all wind facilities in the modeling
+ # year; therefore, they will show up in the end LCI; however, this merge
+ # will only return wind facilities from the 2020 generation data.
wind_generation_data = get_wind_generation(year)
wind_upstream = wind_df_t_melt.merge(
right=wind_generation_data,
diff --git a/pyproject.toml b/pyproject.toml
new file mode 100644
index 0000000..f60b369
--- /dev/null
+++ b/pyproject.toml
@@ -0,0 +1,62 @@
+[build-system]
+requires = ["setuptools>=61.0", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "electricitylci"
+version = "3.0.0"
+authors = [
+ { name = "Matthew Jamieson", email = "Mathew.Jamieson@netl.doe.gov" },
+ { name = "Tyler W. Davis" },
+ { name = "Francis Hanna" },
+ { name = "Wesley W. Ingwersen" },
+ { name = "Greg Schivley" },
+ { name = "Ben Young" },
+ { name = "Tapajyoti Ghosh" },
+ { name = "Jing Li" },
+ { name = "Shirley Sam" },
+ { name = "Daniel Lee Young" },
+ { name = "Michael Srocka" },
+ { name = "Troy A. Hottle" }
+]
+description = "A Python package to create regional life cycle inventory models of U.S. electricity generation, consumption, and distribution using standardized facility and generation data for use with open-source LCA software."
+readme = "README.md"
+license = { text = "CC0" }
+classifiers = [
+ "Development Status :: 5 - Production/Stable",
+ "Environment :: Console",
+ "Intended Audience :: Science/Research",
+ "License :: CC0",
+ "Natural Language :: English",
+ "Operating System :: OS Independent",
+ "Programming Language :: Python :: 3",
+ "Topic :: Software Development :: Libraries :: Python Modules",
+ "Topic :: Scientific/Engineering",
+ "Topic :: Utilities",
+]
+dependencies = [
+ "fedelemflowlist @ git+https://github.com/FLCAC-Admin/fedelemflowlist",
+ "StEWI @ git+https://github.com/USEPA/standardizedinventories",
+ "scipy>=1.10",
+ "pandas<3.0",
+ "pytz",
+]
+
+[project.urls]
+Homepage = "https://github.com/NETL-RIC/ElectricityLCI"
+
+[tool.setuptools]
+packages = ["electricitylci"]
+
+[tool.setuptools.package-data]
+electricitylci = [
+ "data/*.*",
+ "data/EFs/*.*",
+ "data/coal/2020/*.*",
+ "data/coal/2023/*.*",
+ "data/petroleum_inventory/*.*",
+ "data/renewables/2016/*.*",
+ "data/renewables/2020/*.*",
+ "modelconfig/*.yml",
+ "output/.gitignore",
+]
diff --git a/setup.py b/setup.py
index 36103c0..cf598d3 100644
--- a/setup.py
+++ b/setup.py
@@ -2,7 +2,7 @@
setup(
name='electricitylci',
- version='2.1.0',
+ version='3.0.0',
packages=['electricitylci'],
package_data={
'electricitylci': ["data/*.*",
@@ -18,13 +18,15 @@
},
url='https://github.com/NETL-RIC/ElectricityLCI',
license='CC0',
- author='Tyler W. Davis, Matthew Jamieson, Wesley W. Ingwersen, Greg Schivley, Ben Young, Tapajyoti Ghosh, Jing Li, Shirley Sam, Daniel Lee Young, Michael Srocka, and Troy A. Hottle',
+ author='Tyler W. Davis, Francis Hanna, Matthew Jamieson, Wesley W. Ingwersen, Greg Schivley, Ben Young, Tapajyoti Ghosh, Jing Li, Shirley Sam, Daniel Lee Young, Michael Srocka, and Troy A. Hottle',
author_email='Mathew.Jamieson@netl.doe.gov',
- description='A Python package to create regionalized life cycle inventory models of U.S. electricity generation, consumption, and distribution using standardized facility and generation data for use with open-source LCA software.',
+ description='A Python package to create regional life cycle inventory models of U.S. electricity generation, consumption, and distribution using standardized facility and generation data for use with open-source LCA software.',
install_requires=[
'fedelemflowlist @ git+https://github.com/FLCAC-Admin/fedelemflowlist',
'StEWI @ git+https://github.com/USEPA/standardizedinventories#egg=StEWI',
'scipy>=1.10',
+ 'pandas<3.0', # see issue 321
+ 'pytz',
],
long_description=open('README.md').read(),
classifiers=[