Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
124 commits
Select commit Hold shift + click to select a range
2e81a51
implement elci_to_rem into eLCI; addresses #276
dt-woods Dec 12, 2025
780d73e
implement new YAML parameters; addresses #276
dt-woods Dec 12, 2025
67a8508
better doc of params in YAMLs; addresses #276
dt-woods Dec 12, 2025
875a54b
hotfix variable name
dt-woods Dec 12, 2025
52c88dd
utilize model_specs for config options; addresses #276
dt-woods Dec 12, 2025
ca1192c
skeleton of build_residual_processes; addresses #276
dt-woods Dec 16, 2025
c3b6863
working draft of build residual processes; addresses #276
dt-woods Dec 16, 2025
c109895
initial draft of add_rem method; addresses #276
dt-woods Dec 16, 2025
6b3e103
add new configurable params; Addresses #276
dt-woods Dec 19, 2025
a6f7759
working draft of residual mix process and product system creation; ad…
dt-woods Dec 19, 2025
f7b5082
fix URLs; addresses #316
dt-woods Dec 22, 2025
7d59989
fix URL in doc string
dt-woods Dec 22, 2025
b841380
update logging statements; addresses #276
dt-woods Dec 22, 2025
df4a80a
hotfix model_specs; update module doc; addresses #276
dt-woods Dec 22, 2025
3fdb0c6
update module doc; addresses #276
dt-woods Dec 23, 2025
4bcc83e
update README w/ REM; addresses #276
dt-woods Dec 23, 2025
70b97af
implement new residual mix aggregation method for facility generation…
dt-woods Dec 23, 2025
c89a20b
Enhance configuration error handling for residual mix
frankhanna94 Dec 29, 2025
f5dfefb
Fix typo in error message for residual mix configuration
frankhanna94 Dec 29, 2025
143b0db
Merge pull request #18 from KeyLogicLCA/rem
frankhanna94 Dec 29, 2025
840283e
remove duplicate NETL flow correction method
dt-woods Dec 30, 2025
6bf5709
Merge pull request #20 from KeyLogicLCA/elemef
frankhanna94 Jan 5, 2026
247113c
Merge pull request #19 from KeyLogicLCA/rem_gen
frankhanna94 Jan 6, 2026
834ff77
fix DQI in turn_data_to_dict
dt-woods Jan 9, 2026
1b882d3
permit logger level redefinition
dt-woods Jan 28, 2026
b2ae1ed
hotfix residual process ordering; addresses #276
dt-woods Jan 30, 2026
60a4cae
remove redundant calls in correct netl flow names
dt-woods Jan 30, 2026
7d67df1
fix NG LCI for pandas 3; addresses #321
dt-woods Jan 30, 2026
186e9e8
fix links; update dependency versions
dt-woods Jan 30, 2026
6e0740c
update NG upstream process name; addresses #320
dt-woods Feb 6, 2026
032a55a
remove unused global year references
dt-woods Feb 6, 2026
cd9ee11
add pytz to requirements
dt-woods Feb 6, 2026
59b5b67
filter HECO
dt-woods Feb 6, 2026
8563cb0
geothermal zero-flow filter; addresses #314
dt-woods Feb 10, 2026
525b39f
remove CHCl emissions from WI hydro plants; addresses #324
dt-woods Feb 11, 2026
363dbda
fix pandas 3 error; addresses #321
dt-woods Feb 11, 2026
d191844
Merge branch 'development' of https://github.com/KeyLogicLCA/Electric…
dt-woods Feb 11, 2026
8f3e3da
upgrade _wtd_mean method
dt-woods Feb 11, 2026
a46aa5c
correct str to float; addresses #321
dt-woods Feb 12, 2026
1a64348
update UUID location check; addresses #327
dt-woods Feb 12, 2026
ca48289
increase utility of write CSV to output
dt-woods Feb 12, 2026
149cc27
bookend with log statements; addresses #321
dt-woods Feb 12, 2026
69ed41f
add QoL improvement in module doc string
dt-woods Feb 12, 2026
9bb6164
standardize BA names in upstream; addresses #326
dt-woods Feb 12, 2026
ba5125a
end of era with undefined vars; addresses #326
dt-woods Feb 12, 2026
b214ac0
empty series check
dt-woods Feb 12, 2026
278b3a6
Merge branch 'development' of https://github.com/KeyLogicLCA/Electric…
dt-woods Feb 12, 2026
b260b67
Merge branch 'development' of https://github.com/KeyLogicLCA/Electric…
dt-woods Feb 12, 2026
4787fa6
another stab at no identity ValueError in _calc_sigma
dt-woods Feb 13, 2026
f7da321
add openLCA locations; addresses #327
dt-woods Feb 13, 2026
10bb409
hotfix logging statement to debug
dt-woods Feb 13, 2026
92c706d
rm internalId from Unit
dt-woods Feb 13, 2026
967a7d5
remove the double check; fixes #311
dt-woods Feb 13, 2026
23f4a95
cast str to int; addresses #321
dt-woods Feb 13, 2026
31bb738
update sources; addresses #325
dt-woods Feb 13, 2026
eb14cc5
correct inline URL refs; addresses #325
dt-woods Feb 13, 2026
5f87418
correct nuclear upstream description; addresses #325
dt-woods Feb 14, 2026
dcde72e
update eLCI ref; addresses #325
dt-woods Feb 14, 2026
065a484
first effort into NGI metadata by model year; addresses #328
dt-woods Feb 18, 2026
5055c3c
remove zero flow amounts from LCI; addresses #261, #325
dt-woods Feb 24, 2026
40100b8
Updated nuclear upstream metadata fields; addresses #328
dt-woods Feb 24, 2026
2ca6817
update documentation; add error handling; addresses #325, #328
dt-woods Feb 24, 2026
20de113
new process DQI score; addresses #322.
dt-woods Feb 24, 2026
d0a5e90
fix process validity range; addresses #328; see also #244
dt-woods Feb 24, 2026
433f96c
fix duplicate process description, update descripton; addresses #328
dt-woods Feb 24, 2026
da6d344
introduce basic setup method
dt-woods Feb 26, 2026
7c4260f
new keep_all_cols parameter in build_generation_data
dt-woods Feb 26, 2026
3453b50
new location code and name finder
dt-woods Feb 26, 2026
18b644a
hotfix inline comment
dt-woods Feb 26, 2026
a0f8262
format process descriptions; addresses #328
dt-woods Feb 26, 2026
49cd720
documentation; addresses #325
dt-woods Feb 26, 2026
e866d02
separate out renewable operations and maintenance metadata for wind, …
dt-woods Feb 26, 2026
cf06607
documentation updates; addresses #325
dt-woods Feb 26, 2026
1636e1a
documentation updates; addresses #325
dt-woods Feb 26, 2026
3834cff
update gen process metadata; addresses #328
dt-woods Feb 27, 2026
3e67c35
make warnings for no construction
dt-woods Feb 27, 2026
544b716
updates to process metadata; addresses #328.
dt-woods Feb 27, 2026
8bc4e57
add temporal description metadata; addresses #328
dt-woods Feb 27, 2026
d0ec4bb
temporal descriptions for construction processes; addresses #328
dt-woods Mar 4, 2026
ea10a5b
punt on pandas 3 for v2.1; addresses #321
dt-woods Mar 4, 2026
bd020b1
update NETL 2016 water temporal correlation; addresses #296 and #328
dt-woods Mar 5, 2026
d3cd93f
update main; addresses #325
dt-woods Mar 5, 2026
f9b2f26
update get generation years
dt-woods Mar 5, 2026
7de11dc
clean up inline comments; addresses #325
dt-woods Mar 5, 2026
997bdbe
update inline comment; addresses #325
dt-woods Mar 5, 2026
eb5b2cf
update validity dates and temporal descriptions for nuclear upstream …
dt-woods Mar 5, 2026
f1fb815
hotfix inline comment
dt-woods Mar 5, 2026
8932aa1
update generation process temporal description; addresses #328
dt-woods Mar 5, 2026
78ed412
correct missing distribution mix; fix overwrite of process_type to de…
dt-woods Mar 5, 2026
20288f0
move generation process description to default; addresses #328
dt-woods Mar 5, 2026
96f829b
initial attempt at upstream locations; addresses #327
dt-woods Mar 5, 2026
a03aa7d
comment improvements and memory management
dt-woods Mar 11, 2026
ccb45bd
hotfix inline comment
dt-woods Mar 11, 2026
e028a7a
New Canada generation process metadata; addresses #317, #325, #328
dt-woods Mar 11, 2026
e292a0f
tick up modeling year to 2026; addresses #328
dt-woods Mar 11, 2026
ef793e9
hotfix file encoding; addresses #327
dt-woods Mar 11, 2026
06adf2e
add traceback to crash on main
dt-woods Mar 11, 2026
b3c4b8b
fix GitHub URL; addresses #316, #328
dt-woods Mar 17, 2026
d8ee19a
Merge branch 'development' of https://github.com/KeyLogicLCA/Electric…
dt-woods Mar 17, 2026
18c7d8c
hotfix GitHub repo URL
dt-woods Mar 17, 2026
ec0236e
update stewi inventory years available in utils.py
dt-woods Mar 17, 2026
fa457a8
NREL -> NRL; addresses #328
dt-woods Mar 17, 2026
29bd3ab
fix NETL (2025) to ElectricityLCI (2025); addresses #328
dt-woods Mar 17, 2026
ebe01ff
Add system boundary to LCI Method metadata field; addresses #328
dt-woods Mar 18, 2026
38fd297
tick up inventory years; addreses #325
dt-woods Mar 18, 2026
aec40a9
add fuel category code names; addresses #325
dt-woods Mar 18, 2026
06fb107
remove HECO from generation mix; addresses #325
dt-woods Mar 18, 2026
2df2881
comment on #330
dt-woods Mar 18, 2026
0e2360b
hotfix doc string
dt-woods Mar 19, 2026
03adc2f
tick-up the upper efficiency bound for OTHF facilities; addresses #330
dt-woods Mar 19, 2026
3fa6f81
Hotfix inventory method description; addresses #328
dt-woods Mar 19, 2026
878adbc
correct location code and name search; addresses #327
dt-woods Mar 19, 2026
f3b917a
remove HI from consumption & distribution; addresses #325
dt-woods Mar 19, 2026
9c5cf17
hotfix all list
dt-woods Mar 24, 2026
6f5a899
bugfix for #332
dt-woods Mar 25, 2026
79fbd98
minor tweaks associated with #331
dt-woods Mar 25, 2026
f173468
note relevant to #331
dt-woods Mar 25, 2026
56ceb6f
README update, new TOML, uptick v3, add author
dt-woods Mar 26, 2026
bab6b90
update archive background data method; addresses #325
dt-woods Mar 26, 2026
51649ad
hotfix archive background data
dt-woods Mar 26, 2026
3f8a669
updates for residual mixes; addresses #276
dt-woods Mar 27, 2026
c04947a
add note that background data archive does not work correctly for sub…
dt-woods Mar 27, 2026
283168f
update data store for 2023
dt-woods Mar 28, 2026
22dd381
tick up version to 3; addresses #325
dt-woods Mar 28, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 7 additions & 5 deletions DataYears.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,14 +30,16 @@ The following is a summary of input data and years available for use in building
| Coal mining and transport LCI | | x | | | | | | |
| Geothermal LCI<sup>1</sup> | | x | | | | | | |
| Hydro LCI<sup>1</sup> | | x | | | | | | |
| Natural gas LCI | | x | | | | | | |
| Natural gas LCI<sup>2</sup> | | x | | | | x | | |
| Nuclear LCI<sup>1</sup> | | x | | | | | | |
| Solar PV LCI<sup>2</sup> | | x | | | | x | | |
| Solar thermal LCI<sup>2</sup> | | x | | | | x | | |
| Wind farm LCI<sup>2</sup> | | x | | | | x | | |
| Solar PV LCI<sup>3</sup> | | x | | | | x | | |
| Solar thermal LCI<sup>3</sup> | | x | | | | x | | |
| Wind farm LCI<sup>3</sup> | | x | | | | x | | |

Notes:

<sup>1</sup> The inventories for these technologies are based on data from 2016; however, so long as the same plants exist in other years in the EIA 923 data, that inventory can be re-used and applied to different years.

<sup>2</sup> The inventories for these technologies are based on data from 2016 and 2020 (as defined in the RENEWABLE_VINTAGE parameter in globals.py).
<sup>2</sup> This cradle-to-gate inventory is based on data from 2016 or 2020 (as defined in the `ng_model_year` parameter in the model configuration YAML files).

<sup>3</sup> The cradle-to-gate inventories for these technologies are based on data from 2016 or 2020 (as defined in the `renewable_vintage` parameter in the model configuration YAML files).
246 changes: 128 additions & 118 deletions README.md

Large diffs are not rendered by default.

64 changes: 54 additions & 10 deletions electricitylci/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,9 +24,12 @@
end user.

Last updated:
2025-06-09
2026-03-27
"""
__version__ = elci_version
__all__ = [
"basic_setup",
]


##############################################################################
Expand Down Expand Up @@ -101,6 +104,14 @@ def aggregate_gen(gen_df, subregion="BA"):
return aggregate_df


def basic_setup(lvl='INFO', elci='ELCI_2023'):
"""Quickly create a stream logger and define model specs."""
from electricitylci.utils import get_logger
log = get_logger(True, False, str_lv=lvl)
config.model_specs = config.build_model_class(elci)
log.info("Setup complete")


def combine_upstream_and_gen_df(gen_df, upstream_df):
"""
Combine the generation and upstream dataframes into a single dataframe.
Expand Down Expand Up @@ -588,7 +599,7 @@ def get_facility_level_inventory(to_save=False, sep_by_fac=True):
upstream_df = None

# NOTE: This method triggers an input request for EPA data API key;
# see https://github.com/USEPA/ElectricityLCI/issues/207
# see https://github.com/NETL-RIC/ElectricityLCI/issues/207
# NOTE: This method runs aggregation and emission uncertainty
# calculations.
logging.info("get aggregated generation process")
Expand Down Expand Up @@ -788,9 +799,17 @@ def run_net_trade(generation_mix_dict):
return dist_mix_dicts


def run_post_processes():
def run_post_processes(json_path=None):
"""Run post processes on JSON-LD file.

Parameters
----------
json_path : str, optional
The file path to a JSON-LD, defaults to none.
If none, uses the current model run's JSON-LD file.

Notes
-----
There is no easy way of editing files archived in a zip and no
easy way of remove files from a zip without just creating a new zip,
so that's what's done here.
Expand All @@ -799,21 +818,46 @@ def run_post_processes():
addressed in the new _save_to_json (olca_jsonld_writer.py)
2. Remove flows/*.json that are not found in a process exchange.
See cleanup SQL queries in GitHub issue
https://github.com/USEPA/ElectricityLCI/issues/216
https://github.com/NETL-RIC/ElectricityLCI/issues/216
3. Remove zero flows from quantitative reference exchanges
https://github.com/USEPA/ElectricityLCI/issues/217
https://github.com/NETL-RIC/ElectricityLCI/issues/217
4. DO NOT ADD NETL TRACI 2.1 characterization factors
5. Fix labeling of Heat input to elementary flow
https://github.com/USEPA/ElectricityLCI/issues/293
6. Create product systems for select processes (user, consumption mixes)
https://github.com/NETL-RIC/ElectricityLCI/issues/293
6. Create product systems for select processes (user, consumption mixes);
now includes residual mixes (if configured in model specifications)

If you have JSON-LD without residual mixes that you want to add residual
mixes (and product systems) to, follow these steps:

1. Edit the config.yml that was used to create the original JSON-LD by
enabling the 'add_residual_mix' and, optionally,
'add_rem_product_systems', and configure the two settings (i.e.,
'rem_weight_method' and 'neg_rem_method').
2. Quick-start ElectricityLCI with your configuration in Python (e.g.,
``basic_setup("ELCI_2023")``).
3. Define your JSON-LD file path
(e.g., ``my_json = "~/ELCI_2023_jsonld_20260327_094053-rem.zip"``).
4. Run post-processes on your JSON-LD. The residual processes will be
added to the JSON-LD file and will be cleaned;
``run_post_processes(my_json)``.
"""
from electricitylci.olca_jsonld_writer import build_product_systems
from electricitylci.olca_jsonld_writer import clean_json
from electricitylci.residual_grid_mix import add_residual_mixes

if json_path is None:
json_path = config.model_specs.namestr
else:
logging.info("Running post-processes on %s" % json_path)

clean_json(config.model_specs.namestr)
# HOTFIX: clean JSON after residual mix processes [260107; TWD]
add_residual_mixes(json_path) # skipped if not configured
clean_json(json_path)
build_product_systems(
file_path=config.model_specs.namestr,
elci_config=config.model_specs.model_name
file_path=json_path,
elci_config=config.model_specs.model_name,
add_residuals=config.model_specs.add_rem_product_systems
)


Expand Down
4 changes: 3 additions & 1 deletion electricitylci/bulk_eia_data.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@
consumption mix for a given region.

Last updated:
2024-08-21
2026-03-25
"""
__all__ = [
"ba_exchange_to_df",
Expand Down Expand Up @@ -272,7 +272,9 @@ def ba_exchange_to_df(rows, data_type='ba_to_ba'):
)
continue
data = [x[1] for x in row['data']]
# Trim 'EBA.' from series ID
from_region = row['series_id'].split('-')[0][4:]
# Trim '.ID.D' from series ID
to_region = row['series_id'].split('-')[1][:-5]
tuple_data = [
x for x in zip(
Expand Down
2 changes: 1 addition & 1 deletion electricitylci/cems_data.py
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@
CC-BY-4.0

In the current release, the PUDL methods are replaced with EPA's API:
https://github.com/USEPA/ElectricityLCI/issues/207#issuecomment-1751075194
https://github.com/NETL-RIC/ElectricityLCI/issues/207#issuecomment-1751075194

Last edited:
2025-08-13
Expand Down
63 changes: 15 additions & 48 deletions electricitylci/coal_upstream.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,10 @@
from ast import literal_eval
from electricitylci.globals import paths
from electricitylci.globals import data_dir
from electricitylci.globals import GH_URL
from electricitylci.globals import STATE_ABBREV
from electricitylci.globals import COAL_BASIN_CODES
from electricitylci.globals import COAL_TYPE_CODES
from electricitylci.eia860_facilities import eia860_balancing_authority
from electricitylci.eia923_generation import eia923_download
from electricitylci.eia923_generation import eia923_generation_and_fuel
Expand Down Expand Up @@ -58,13 +61,10 @@
transportation, but still mainly represents 2016.

Last updated:
2025-09-05
2026-03-05
"""
__all__ = [
"basin_codes", # Globals
"coal_type_codes",
"mine_type_codes",
"transport_dict",
"transport_dict", # Globals
"eia_7a_download", # Methods
"fix_coal_mining_lci",
"generate_upstream_coal",
Expand All @@ -82,38 +82,6 @@
##############################################################################
# GLOBALS
##############################################################################
basin_codes = {
'Central Appalachia': 'CA',
'Central Interior': 'CI',
'Gulf Lignite': 'GL',
'Illinois Basin': 'IB',
'Lignite': 'L',
'Northern Appalachia': 'NA',
'Powder River Basin': 'PRB',
'Rocky Mountain': 'RM',
'Southern Appalachia': 'SA',
'West/Northwest': 'WNW',
'Import': 'IMP',
}
'''dict : A map between NETL coal basin names and their abbreviations.'''

coal_type_codes = {
'BIT': 'B',
'LIG': 'L',
'SUB': 'S',
'WC': 'W',
'RC' : 'RC',
}
'''dict : Map between EIA coal fuel source codes and NETL coal codes.'''

mine_type_codes = {
'Surface': 'S',
'Underground': 'U',
'Facility': 'F',
'Processing': 'P',
}
'''dict : A map between coal mine type and their abbreviation.'''

transport_dict = {
'Avg Barge Ton*Miles': 'Barge',
'Avg Lake Vessel Ton*Miles': 'Lake Vessel',
Expand Down Expand Up @@ -145,8 +113,8 @@ def _clean_columns(df):
def _coal_code(row):
"""Generate coal basin + energy source + mine type string-based code."""
coal_code_str = (
f'{basin_codes[row["netl_basin"]]}-'
f'{coal_type_codes[row["energy_source"]]}-'
f'{COAL_BASIN_CODES[row["netl_basin"]]}-'
f'{COAL_TYPE_CODES[row["energy_source"]]}-'
f'{row["coalmine_type"]}'
).upper()

Expand Down Expand Up @@ -198,7 +166,7 @@ def _make_2023_coal_transport_data(year):
coal_map_df = coal_map_df.drop(columns=['coal_source_code', 'heat_input'])

# Read the 2023 coal model transportation data
# Source: https://github.com/USEPA/ElectricityLCI/discussions/273
# Source: https://github.com/NETL-RIC/ElectricityLCI/discussions/273
coal_dir = os.path.join(data_dir, "coal", "2023")
coal_file = os.path.join(coal_dir, "coal_transport_dist.csv")
if not os.path.isfile(coal_file):
Expand All @@ -209,7 +177,7 @@ def _make_2023_coal_transport_data(year):

# NOTE: the 2023 coal model uses a slightly different naming scheme
# for WNW coal basin, so let's fix it.
basin_codes_new = {k:v for k, v in basin_codes.items()}
basin_codes_new = {k:v for k, v in COAL_BASIN_CODES.items()}
del basin_codes_new["West/Northwest"]
basin_codes_new["West/North West"] = "WNW"

Expand Down Expand Up @@ -433,7 +401,7 @@ def eia_7a_download(year, save_path):
-----
Some years are provided in XML format and require re-saving to work with
the remainder of the code. If you run into troubles with the download,
see https://github.com/USEPA/ElectricityLCI/issues/230 for a solution.
see https://github.com/NETL-RIC/ElectricityLCI/issues/230 for a solution.
"""
eia7a_base_url = 'http://www.eia.gov/coal/data/public/xls/'
name = ('coalpublic{}.xls'.format(year) if year <= 2022 else
Expand Down Expand Up @@ -1070,7 +1038,7 @@ def generate_upstream_coal_map(year):
# Map to NETL coal types --- these should match the coal type found in
# the coal source code.
eia_fuel_receipts_good["coal_type"] = eia_fuel_receipts_good[
"energy_source"].map(coal_type_codes)
"energy_source"].map(COAL_TYPE_CODES)

final_df = eia_fuel_receipts_good.groupby(
['plant_id', 'coal_type', 'coal_source_code'],
Expand Down Expand Up @@ -1101,7 +1069,7 @@ def generate_upstream_coal_map(year):
"elec_fuel_consumption_mmbtu"
]].copy()
eia_923_gen_fuel["coal_type"] = eia_923_gen_fuel[
"reported_fuel_type_code"].map(coal_type_codes)
"reported_fuel_type_code"].map(COAL_TYPE_CODES)
eia_923_gen_fuel["plant_id"] = eia_923_gen_fuel["plant_id"].astype(int)
# Effectively filters out NaNs in coal type
# NOTE: quantity is in short tons
Expand Down Expand Up @@ -1675,7 +1643,7 @@ def read_eia7a_public_coal(year):
file_pattern_match=['coalpublic'],
return_name=False)
# If you're here, then see the following for hotfix:
# https://github.com/USEPA/ElectricityLCI/issues/230
# https://github.com/NETL-RIC/ElectricityLCI/issues/230
try:
eia7a_df = pd.read_excel(
eia7a_path,
Expand All @@ -1684,9 +1652,8 @@ def read_eia7a_public_coal(year):
)
except ValueError:
raise ValueError(
f'Error reading {eia7a_path}. Please see '
'https://github.com/USEPA/ElectricityLCI/issues/230 '
'for a solution'
f'Error reading {eia7a_path}. '
f'Please see {GH_URL}/issues/230 for a solution'
)
eia7a_df = _clean_columns(eia7a_df)

Expand Down
Loading
Loading