Skip to content

fix(ci): harden weekly ETL schedule - #107

Merged
Sam-24-dev merged 5 commits into
mainfrom
fix/etl-schedule-resilience
Sep 1, 2026
Merged

fix(ci): harden weekly ETL schedule#107
Sam-24-dev merged 5 commits into
mainfrom
fix/etl-schedule-resilience

Conversation

@Sam-24-dev

Copy link
Copy Markdown
Owner

Summary

  • move the weekly ETL schedule to Monday 08:17 UTC to avoid the top-of-hour scheduler hotspot
  • add a pre-publication canonical source freshness guard using dataset_summaries.updated_at_utc
  • document controlled recovery of delayed runs through workflow_dispatch

Validation

  • python -m pytest -q (220 passed)
  • python -m pytest -q tests/test_check_source_freshness.py tests/test_workflow_etl_contract.py (11 passed)
  • python scripts/check_source_freshness.py --project-root . --max-source-age-hours 192
  • static YAML parse and assertions for schedule, workflow_dispatch, and concurrency
  • detect-secrets scan of staged files

Scope

No ETL, deployment, source datasets, frontend, dependencies, or permissions changes.

Copilot AI lite review requested due to automatic review settings August 31, 2026 20:34

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@Sam-24-dev
Sam-24-dev merged commit e0cd5b6 into main Sep 1, 2026
4 checks passed
@Sam-24-dev
Sam-24-dev deleted the fix/etl-schedule-resilience branch September 1, 2026 01:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants