Bucket the runs volume chart on the reader's clock, and click through it - #5191
Merged
Merged
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## tidy-workflow-health #5191 +/- ##
======================================================
Coverage ? 91.1%
======================================================
Files ? 452
Lines ? 22817
Branches ? 0
======================================================
Hits ? 20797
Misses ? 2020
Partials ? 0 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
midigofrank
changed the base branch from
release-2.19.0
to
tidy-workflow-health
September 17, 2026 08:06
midigofrank
force-pushed
the
frank/con-188
branch
from
September 17, 2026 08:38
badd377 to
65ede5b
Compare
midigofrank
marked this pull request as ready for review
September 17, 2026 08:41
Security Review ✅
|
midigofrank
changed the title
The grid was laid on the raw epoch, so a "daily" bar was a UTC day — 03:00 to 03:00 in Nairobi, 05:30 in Delhi — and the day a clock changed split across two bars. Buckets are now cut with date_bin on the local wall clock, in the zone the browser sends as an x-timezone header: a boundary lands on local midnight, London's 25-hour October day is one bar, and every axis and tooltip label is read in that same zone rather than in UTC. Bar widths are unchanged — 2-hourly over a day, half-days over a week, daily over a month — but they are now wall-clock hours, so a daily bar is 23 or 25 hours on the day the clocks move. The response carries bucket_hours and timezone alongside the buckets. The chart used to measure its own bar width by subtracting the first two timestamps, which reads 25 on a clock-change day and mislabels the axis; the server cut the grid, so the server names the width and the clock. The timezone joins the cache key, because it moves every boundary and Cachex keys an in-flight fetch by key alone — without it the first reader to load the page would pin their grid on everyone else's for the TTL. It is validated against the tz database before it gets there, since :workflow_stats has no size limit and an arbitrary header value would let one authenticated reader mint unbounded entries. An unknown or missing zone falls back to UTC rather than 400ing: a bad window cannot be drawn, but a bad zone still has a drawable answer. Configures Tzdata as the time zone database, without which every DateTime.shift_zone/2 by name returns :utc_only_time_zone_database.
Tzdata and pg_timezone_names are versioned apart: :tzdata autoupdates at runtime, while Postgres' zone set is fixed at the server's build. So a zone the controller validated can still be one the binning query has never heard of — America/Coyhaique against PG 15 is exactly that, and it is what Chrome reports for readers in Coyhaique, Chile. Rescue invalid_parameter_value and redraw the window on UTC rather than 500 the endpoint. This matches the policy the controller already sets for unusable timezones: a bad window cannot be drawn, a bad clock still can. The result caches under the requested zone, so the failing query runs once per TTL rather than once per poll.
Each band of each bar links to the work orders behind it: the bar's slot,
and the states that band stacked. The red band folds five run states, so it
carries all five — a column-wide link would land the reader on the answer
they came for mixed back in with the successes.
The history page had no run-level filter to link to. run_date_after,
run_date_before and run_status bound a run's inserted_at and its state,
matched with an EXISTS over the work order's runs. The existing status
filter is the work order's own state, which is its latest run: a failure
since retried to success is a success there, and the band that counted the
failure still has to reach it. run_status takes final states only, since an
unsettled run has no outcome to have been counted.
A bar's slot ends at the next bar's start rather than its own plus a width.
The grid is cut on the reader's clock, so a bar spanning a clock change is
11 or 13 real hours wide, and a run landing on a boundary belongs to the
same bar date_bin counted it in. The newest bar goes over open-ended — it
is still filling.
historyUrl now repeats key[] for a list value, which is what Plug decodes
back into something an {:array, _} field can cast.
The chips read the range in UTC and say so. Nothing in Lightning records
what clock the bars were cut on, and a bare "14:00" under a bar labelled
"16:00" is worse than a suffix.
midigofrank
force-pushed
the
frank/con-188
branch
from
September 18, 2026 09:27
6f7e4f1 to
69f7ce2
Compare
midigofrank
force-pushed
the
frank/con-188
branch
from
September 18, 2026 09:40
69f7ce2 to
75c7452
Compare
date_bin cut the buckets with Postgres' zone rules while bucket_starts and resolve cut the same grid with Tzdata's. The two are versioned apart, so a transition they disagreed about put a tally under a local timestamp absent from the bar list, and Map.get dropped those runs. The bar read zero rather than wrong, which is the kind of wrong nobody reports. width_bucket takes the resolved boundaries instead and returns the slot, so Postgres never sees a zone name and Tzdata alone decides where a bar opens. The Coyhaique case goes with it: nothing asks Postgres about a zone it may not know, so there is nothing to rescue. Those boundaries have to be strictly ascending for width_bucket to mean anything, and Postgres answers an unsorted array with a plausible number rather than an error. Antarctica/Troll shifts two hours, so a transition can be as wide as the narrowest bucket — the property holds because of where the transitions land, not because of any margin, and the boundary test is what holds it. boundaries/3 is public so the grid can be checked without a database. That is the only way to put a fixed clock change inside the window, which is otherwise always relative to now, and it reaches the gap and ambiguous branches of resolve that nothing exercised before. Every run test now asserts the bars sum to the runs in the window.
Three places chose UTC: the controller's default, the rescue that redrew the window when Postgres rejected a zone, and a default argument on Stats.runs/3. The rescue went with the binning change and the default argument goes here, leaving the plug. No header is UTC, because a cached bundle from before the header existed has to keep working through a deploy. Etc/Unknown is UTC too — it is CLDR's sentinel for a host clock it could not map, so a browser sending it is saying it does not know rather than naming a zone we failed to recognise. Anything else the tz database does not know is a 400: the browser picked it, and drawing someone else's clock would hide that rather than fix it. vary, because this is the one action whose body turns on a request header and nothing between the browser and here would guess it. The stats cache stays unbounded. The timezone is in its key, so the key space is wider than it was, but an entry is ~16 KiB against the ~200 ms aggregate it takes to fill one — the database is the scarce resource here, and a size limit would bound the wrong one.
The click-through and the history filters land separately for a reader, so they read as separate entries rather than one. The timezone entry now says what a browser gets when it sends no zone or one the tz database does not know.
stuartc
approved these changes
Sep 18, 2026
stuartc
left a comment
stuartc
left a comment
Member
There was a problem hiding this comment.
Nice work dude. I made a few changes:
- The bucketing now works out the edges with Tzdata, and then passes that to postgres. So we don't have to deal with the mismatch between tzdata and postgres. Since tzdata is updated all the time I picked that as the source of truth.
- Moved the default timezone into one place (in the controller plug). No header means UTC, browsers can say "Etc/Unknown" and we make that UTC. Anything else is a 400; I might regret that hardening but yeah thought I'd rather see an error than quietly have users get UTC when they shouldn't.
- Added
Vary: x-timezoneso that any caching layer we might end up having would know that it's part of the cache keys and it will consider it a different request.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
The grid was laid on the raw epoch, so a "daily" bar was a UTC day — 03:00 to 03:00 in Nairobi, 05:30 in Delhi — and the day a clock changed split across two bars. Buckets are now cut with date_bin on the local wall clock, in the zone the browser sends as an x-timezone header: a boundary lands on local midnight, London's 25-hour October day is one bar, and every axis and tooltip label is read in that same zone rather than in UTC. Bar widths are unchanged — 2-hourly over a day, half-days over a week, daily over a month — but they are now wall-clock hours, so a daily bar is 23 or 25 hours on the day the clocks move.
The response carries bucket_hours and timezone alongside the buckets. The chart used to measure its own bar width by subtracting the first two timestamps, which reads 25 on a clock-change day and mislabels the axis; the server cut the grid, so the server names the width and the clock.
The timezone joins the cache key, because it moves every boundary and Cachex keys an in-flight fetch by key alone — without it the first reader to load the page would pin their grid on everyone else's for the TTL. It is validated against the tz database before it gets there, since :workflow_stats has no size limit and an arbitrary header value would let one authenticated reader mint unbounded entries. An unknown or missing zone falls back to UTC rather than 400ing: a bad window cannot be drawn, but a bad zone still has a drawable answer.
Tzdata and pg_timezone_names are versioned apart — :tzdata autoupdates at runtime, Postgres' zone set is fixed at the server's build — so a zone the controller validated can still be one the binning query has never heard of. America/Coyhaique against PG 15 is exactly that, and it is what Chrome reports for readers in Coyhaique. The query rescues invalid_parameter_value and redraws the window on UTC rather than 500ing, and the result caches under the requested zone, so the failing query runs once per TTL rather than once per poll.
Configures Tzdata as the time zone database, without which every DateTime.shift_zone/2 by name returns :utc_only_time_zone_database.
Click through the chart to the history page
Each band of each bar now links to the work orders behind it: the bar's slot, and the states that band stacked. The red band folds five run states, so it carries all five — a column-wide link would land the reader on the answer they came for mixed back in with the successes.
The history page had no run-level filter to link to. run_date_after, run_date_before and run_status bound a run's inserted_at and its state, matched with an EXISTS over the work order's runs. The existing status filter is the work order's own state, which is its latest run: a failure since retried to success is a success there, and the band that counted the failure still has to reach it. run_status takes final states only, since an unsettled run has no outcome to have been counted.
A bar's slot ends at the next bar's start rather than its own plus a width. The grid is cut on the reader's clock, so a bar spanning a clock change is 11 or 13 real hours wide, and a run landing on a boundary belongs to the same bar date_bin counted it in. The newest bar goes over open-ended — it is still filling. The filter chips read the range in UTC and say so: nothing in Lightning records what clock the bars were cut on, and a bare "14:00" under a bar labelled "16:00" is worse than a suffix.
Closes CON-188
Validation steps
Please disclose whether you've used AI anywhere in this PR (it's cool, we just
want to know!):
You can read more details in our
Responsible AI Policy
Pre-submission checklist
/reviewwith Claude Code)
(e.g.,
:owner,:admin,:editor,:viewer)