Add 2024 ATB Update Pipeline

[Yunzhi-Chen]

Summary

Adds a configuration-driven workflow to convert corrected ATB 2024 data into ReEDS-compatible inputs.

The workflow downloads, formats, plots, and validates ATB data. Historical years are filled from an explicitly configured source, future years can be optionally smoothed, and before/after comparison plots are produced for review.

Technical details

• Uses the ATB flat file for technology parameters and workbook for battery power/energy costs.
• Combines projections with version-controlled historical inputs.
• Preserves ReEDS filenames, schemas, and wind column ordering.
• Every metric selects its historical source explicitly: real (observed project costs), manual (versioned ReEDS history), or broadcast (the first ATB projection value held flat).
• Observed history is scraped from LBNL land-based wind and utility-scale solar, NREL's offshore wind market report, and EIA generator construction costs, then normalized into a single CSV that carries source file, sheet, and dollar year on every row.
• A metric may set one mode per sub-technology, so an observed series is applied only to rows it actually measures. NREL's offshore CapEx series is fixed-bottom, and EIA reports gas plants only as combined cycle or combustion turbine, so the floating, H-Frame, and aeroderivative variants generate their own history rather than borrowing one.
• Future smoothing bridges compact clusters of slope changes. Major transitions of 10% or more, single slope changes, flat stretches, spans longer than four years, and adjustments under 0.5% are left as ATB published them. ATB's published direction is never overridden, so near-term cost increases pass through unchanged.
• Applies to monetary metrics and capacity-factor multipliers; heat rate and round-trip efficiency are never smoothed.
• Keeps raw downloads and generated CSVs out of Git.

Additional changes

• Added atb/config.yaml, a JSON schema for editor completion and validation, and the pipeline scripts.
• Added historical data and documented manual adjustments.
• Added workflow and input documentation.
• Added validation and smoothing-comparison plots.
• Pinned sources to corrected ATB 2024 v3.

Relevant sources or documentation

• ATB workflow documentation
• ATB 2024 v3 flat file
• ATB 2024 v3 workbook
• LBNL Land-Based Wind Market Report
• LBNL Utility-Scale Solar
• NREL Offshore Wind Market Report
• EIA generator construction costs

Additional details

Sanity check: reproducing the current ReEDS inputs

With smoothing disabled and every metric set to manual historical data, the workflow reproduces the inputs ReEDS uses today. This isolates the formatting and dollar-year conversion from any of the new historical-source or smoothing behavior.

Comparing those 32 files against the current ReEDS inputs:

• 21 files reproduce the current inputs exactly.
• 8 differ only by floating-point last-digit noise (1e-9 relative or smaller).
• 3 offshore wind files differ by up to 0.018% in the capacity-factor multiplier, consistent with a different ATB 2024 release than the pinned v3.0.0.

Byte-level identity is not expected: the current ReEDS files were written with 10 significant digits, while this workflow rounds to a fixed number of decimal places, and several files carry the same rows in a different order.

Sanity-check figures — smoothing off, manual historical data, matching current ReEDS:

compare_to_ReEDS.zip

Output verification

The formatter completed successfully and generated 32 ReEDS input files under the proposed configuration. Each plot below shows one technology's final curves against the unsmoothed input, with the historical years coloured by their configured source. The outputs differ from the sanity-check baseline in two intended ways: compact clusters of slope changes are bridged in the projection, and historical years use observed project costs wherever a metric selects real.

The generated CSVs are not attached; they are reproducible by running the workflow with the committed atb/config.yaml.

Before/after smoothing plots:

smoothing_comparison.zip

Documentation

Updated the ATB README files with setup, processing, smoothing, validation, and input-source information.

LLM usage

LLM tools assisted with implementation, documentation, debugging, and validation. Outputs were reviewed and tested against the current ReEDS inputs.