Update REACH Download to use Date-Based API - #27
Merged
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #27 +/- ##
=======================================
Coverage ? 62.54%
=======================================
Files ? 2
Lines ? 251
Branches ? 0
=======================================
Hits ? 157
Misses ? 94
Partials ? 0 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Pull request overview
Updates the REACH ingestion flow to use a date-based, UTC-midnight-aligned download window API, replacing the prior seconds-based delay/window configuration to ensure each run covers an exact midnight-to-midnight day.
Changes:
- Switch REACH download callsite to
download_UDL_reach_window(start_time, end_time)and compute midnight-snapped UTC day windows from new env vars. - Update unit test to mock
download_UDL_reach_windowand assert midnight-aligned window edges and one-day span. - Rework
buildspec.ymlto parameterize ECR/account/tagging logic and remove the prior lint/test pre-build steps.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| README.rst | Documents the new day-based REACH window env vars and midnight-snapping behavior. |
| lambda_function/src/executor/executor.py | Computes UTC-midnight window boundaries and calls the new REACH window download API. |
| lambda_function/tests/test_executor.py | Updates REACH import test to validate midnight-aligned window semantics with the new API. |
| buildspec.yml | Refactors CodeBuild image build/push logic to use dynamic account/repo/version values. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
download_UDL_reach_window), replacing the seconds-basedREACH_DELAY_SECONDS/REACH_WINDOW_SECONDSwith day-basedREACH_WINDOW_END_DAYS_AGOandREACH_WINDOW_DAYS(defaults preserve the current 54h–30h-ago window) so output files reliably cover a full midnight-to-midnight day.REACH_DESTINATION_BUCKET→_DEV/_PROD).test_import_UDL_REACH_to_s3to mockdownload_UDL_reach_windowand assert midnight-aligned edges, a one-day-before-midnight end, and a one-day window span.pre_buildphase and parameterized the build to derive ECR account/repo and tag/version dynamically (unrelated to the REACH change).closes #26