fix(logstore): normalize workflow timestamps to naive UTC - #40947
Merged
asukaminato0721 merged 2 commits intoAug 20, 2026
Merged
Conversation
The LogStore repositories converted timestamps with `datetime.fromtimestamp(ts)` and `datetime.now()`, both of which return naive *local* time, while the rest of the codebase stores naive UTC. `datetime.fromisoformat` also returns an aware value when the string carries an offset. Because `created_at` and `finished_at` are matched independently, one record's two timestamps could come from different branches and end up in different frames, so the `elapsed_time` subtraction produced garbage. On a UTC+05:30 host a 30-second run was reported as 19830 seconds, a missing `started_at` yielded a negative `elapsed_time`, and an offset-bearing ISO string raised `TypeError: can't subtract offset-naive and offset-aware datetimes`. Route every branch of both `match` statements through `ensure_naive_utc` / `naive_utc_now` and pass `tz=UTC` to `fromtimestamp`, so all values land in naive UTC regardless of the host timezone.
amogh-nagri-11
requested review from
QuantumGhost and
laipz8200
as code owners
August 18, 2026 17:45
Contributor
Pyrefly Diffbase → PR--- /tmp/pyrefly_base.txt 2026-08-19 03:30:39.634723894 +0000
+++ /tmp/pyrefly_pr.txt 2026-08-19 03:30:26.357618702 +0000
@@ -7975,9 +7975,9 @@
ERROR Object of class `Events` has no attribute `request_start` [missing-attribute]
--> tests/unit_tests/events/test_events_package_compat.py:33:5
ERROR Cannot instantiate `LogstoreAPIWorkflowNodeExecutionRepository` because the following members are abstract: `get_execution_snapshots_by_workflow_run`, `delete_expired_executions`, `count_by_runs`, `delete_by_runs`, `delete_executions_by_app`, `get_expired_executions_batch`, `delete_executions_by_ids`, `get_offloads_by_execution_ids`, `save`, `save_synchronously`, `save_execution_data`, `get_by_workflow_execution` [bad-instantiation]
- --> tests/unit_tests/extensions/logstore/repositories/test_logstore_api_workflow_node_execution_repository.py:12:64
+ --> tests/unit_tests/extensions/logstore/repositories/test_logstore_api_workflow_node_execution_repository.py:18:64
ERROR Cannot instantiate `LogstoreAPIWorkflowNodeExecutionRepository` because the following members are abstract: `get_execution_snapshots_by_workflow_run`, `delete_expired_executions`, `count_by_runs`, `delete_by_runs`, `delete_executions_by_app`, `get_expired_executions_batch`, `delete_executions_by_ids`, `get_offloads_by_execution_ids`, `save`, `save_synchronously`, `save_execution_data`, `get_by_workflow_execution` [bad-instantiation]
- --> tests/unit_tests/extensions/logstore/repositories/test_logstore_api_workflow_node_execution_repository.py:21:64
+ --> tests/unit_tests/extensions/logstore/repositories/test_logstore_api_workflow_node_execution_repository.py:27:64
ERROR Cannot index into `object` [bad-index]
--> tests/unit_tests/extensions/otel/test_celery_sqlcommenter.py:136:20
ERROR Cannot index into `object` [bad-index]
|
Contributor
Pyrefly Type Coverage
|
asukaminato0721
enabled auto-merge
August 19, 2026 09:46
asukaminato0721
approved these changes
Aug 20, 2026
Contributor
Author
|
Thank you @asukaminato0721 for the quick review and merge! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #40943.
The two LogStore repositories deserialize workflow timestamps with calls that resolve against the host's local clock, while the rest of the codebase stores naive UTC:
datetime.fromtimestamp(ts)withouttz→ naive local timedatetime.now()→ naive local timedatetime.fromisoformat(s)→ aware when the string carries an offset, naive otherwiseOn its own that would just be a uniform offset. The real problem is that
created_atandfinished_atare matched independently, so one record's two timestamps can be produced by different branches and land in different frames — and_dict_to_workflow_runthen subtracts them forelapsed_time.Observed on a UTC+05:30 host (reproduction in the issue drives the real functions):
TypeError: can't subtract offset-naive and offset-aware datetimeselapsed_time = 30.0elapsed_time = 30.0started_atmissingelapsed_timenegative (-49875.0)naive_utc_now()On a UTC host all four coincidentally agree, which is why this is easy to miss.
Changes
Every branch of both
matchstatements now terminates in naive UTC, using the helpers that already exist inlibs/datetime_utils:datetime.now()→naive_utc_now()(the same call the Postgres write path uses)datetime.fromtimestamp(ts)→datetime.fromtimestamp(ts, tz=UTC).replace(tzinfo=None)datetime.fromisoformat(s)→ wrapped inensure_naive_utc(...), which normalizes aware values and is a no-op for naive onesdatetimepassthrough branch is wrapped inensure_naive_utctoo, so an aware value from upstream can't reintroduce the mixed-frame subtractionApplied to all six call sites across
logstore_api_workflow_run_repository.pyandlogstore_api_workflow_node_execution_repository.py. One change fixes the offset shift, theTypeError, and both badelapsed_timeresults together.Scope note: this path is only reachable when the Aliyun SLS LogStore backend is enabled, so deployments on the default Postgres path are unaffected. No behavior change on UTC hosts.
Tests
Added regression coverage that pins
TZ=Asia/Kolkataviatime.tzset(), so the assertions are meaningful on the UTC CI host rather than passing by coincidence. It covers all four payload shapes plus the missing-timestamp default, for both repositories.All 10 new tests fail on
mainand pass with this change:ruff check,ruff format --check,mypyandpyreflyare clean on the changed files (the two pre-existingbad-instantiationpyrefly diagnostics in the node-execution test file are on untouched lines).Screenshots
Not applicable — backend-only timezone correctness fix.
Checklist
make lint && make type-check(backend) andvp staged(frontend) to appease the lint gods