Skip to content

Update REACH Download to use Date-Based API - #27

Merged
Alrobbertz merged 4 commits into
swxsoc:mainfrom
Alrobbertz:update-reach-download
Jun 23, 2026
Merged

Update REACH Download to use Date-Based API#27
Alrobbertz merged 4 commits into
swxsoc:mainfrom
Alrobbertz:update-reach-download

Conversation

@Alrobbertz

Copy link
Copy Markdown
Member
  • Switched the REACH import to the absolute, midnight-aligned UTC window API (download_UDL_reach_window), replacing the seconds-based REACH_DELAY_SECONDS/REACH_WINDOW_SECONDS with day-based REACH_WINDOW_END_DAYS_AGO and REACH_WINDOW_DAYS (defaults preserve the current 54h–30h-ago window) so output files reliably cover a full midnight-to-midnight day.
  • Updated README.rst to document the new env vars and midnight-snapping behavior, and fixed the bucket vars (REACH_DESTINATION_BUCKET_DEV/_PROD).
  • Updated test_import_UDL_REACH_to_s3 to mock download_UDL_reach_window and assert midnight-aligned edges, a one-day-before-midnight end, and a one-day window span.
  • Reworked buildspec.yml: removed the lint/test pre_build phase and parameterized the build to derive ECR account/repo and tag/version dynamically (unrelated to the REACH change).

closes #26

Copilot AI review requested due to automatic review settings June 23, 2026 14:15
@codecov

codecov Bot commented Jun 23, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
⚠️ Please upload report for BASE (main@48c49a0). Learn more about missing BASE report.

Additional details and impacted files
@@           Coverage Diff           @@
##             main      #27   +/-   ##
=======================================
  Coverage        ?   62.54%           
=======================================
  Files           ?        2           
  Lines           ?      251           
  Branches        ?        0           
=======================================
  Hits            ?      157           
  Misses          ?       94           
  Partials        ?        0           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the REACH ingestion flow to use a date-based, UTC-midnight-aligned download window API, replacing the prior seconds-based delay/window configuration to ensure each run covers an exact midnight-to-midnight day.

Changes:

  • Switch REACH download callsite to download_UDL_reach_window(start_time, end_time) and compute midnight-snapped UTC day windows from new env vars.
  • Update unit test to mock download_UDL_reach_window and assert midnight-aligned window edges and one-day span.
  • Rework buildspec.yml to parameterize ECR/account/tagging logic and remove the prior lint/test pre-build steps.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File Description
README.rst Documents the new day-based REACH window env vars and midnight-snapping behavior.
lambda_function/src/executor/executor.py Computes UTC-midnight window boundaries and calls the new REACH window download API.
lambda_function/tests/test_executor.py Updates REACH import test to validate midnight-aligned window semantics with the new API.
buildspec.yml Refactors CodeBuild image build/push logic to use dynamic account/repo/version values.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread README.rst
Comment thread lambda_function/tests/test_executor.py
@Alrobbertz
Alrobbertz merged commit 05cf6c9 into swxsoc:main Jun 23, 2026
4 checks passed
@Alrobbertz
Alrobbertz deleted the update-reach-download branch June 23, 2026 14:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Update REACH UDL Download to use new historical download API.

2 participants