Skip to content

improvement(logs): move correlation ids from message tags into metadata and enforce it - #9724

Merged
waleedlatif1 merged 5 commits into
stagingfrom
improvement/log-correlation-fields
Nov 6, 2026
Merged

waleedlatif1 merged 5 commits into
stagingfrom
improvement/log-correlation-fields

Conversation

@waleedlatif1

@waleedlatif1 waleedlatif1 commented Nov 6, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

  • Log messages no longer carry correlation ids in their text. logger.info([${requestId}] Started) becomes logger.info('Started', { requestId }), so a message groups across requests and lines filter by requestId (or documentId, label, …) as a field. 1,937 tagged calls across 294 files moved by an AST codemod; the remaining forms (compound [${label}/phase] tags, ternary and const-built messages, prefix consts, helpers such as logFailureOnce(logger, message, …)) were moved by hand.
  • Inside withRouteHandler, a generateRequestId() id is the request context's id, which the logger already stamps on every line, so those tags are dropped without a field (per CLAUDE.md). Variables that existed only for the tag are deleted.
  • Code that minted its own id only to tag its lines (v1 routes, the poll routes, the table runners) now uses generateRequestId(), so it is the request's id; the poll routes' lock values and response requestId now equal the x-request-id header. Each webhook poll and each emitted Sim event keeps its own id, because executions use it as an idempotency key; each poll logs through a logger bound to its webhookId.
  • @sim/logger now owns requestId: inside a request context the context's id is authoritative, so a requestId passed with a line or bound with withMetadata cannot replace it (it names the request only outside a context), and an undefined field no longer erases a field the line already carries. Before, an explicit field could detach a line from its request: a replay key, a resumed run's original id, or an absent optional id replaced or deleted it. The JSON line now matches the OpenTelemetry export, which already took requestId from the context. v2 chat logs its Mothership request id as simRequestId, and ids that defaulted to 'unknown' default to generateRequestId().
  • check:source-conventions gains log-message-tag: it flags a @sim/logger message that opens with an interpolated tag (also after static tags like [Cascade]) or interpolates a request id anywhere. It covers every apps/* and packages/* module that imports @sim/logger, sees optional calls, computed levels and withMetadata-derived loggers, follows a message through concatenation, ternaries, function-scoped const/let bindings, log* helpers and .call(logger, …), and starts at zero, so the pattern cannot come back. Bracketed lists mid-message (columns [${ids.join(', ')}]) are allowed.
  • CLAUDE.md/AGENTS.md "Logging" and the add-trigger skill example describe the convention.

Type of Change

  • Improvement

Testing

  • check:source-conventions passes with log-message-tag 0; rule tests cover each flagged form and the allowed ones (42 pass).
  • Logger tests: the context id wins over a per-line or bound requestId, a per-line id applies outside a context, and an undefined field erases nothing; the two guard tests fail on the previous logger.
  • Ran the tests that assert on changed log lines plus the touched polling, tools, execution-core, credential, persistence, knowledge and webhook polling suites (about 900 tests); updated assertions to the new message and metadata shape, and rewrote one negative assertion that would otherwise have passed vacuously.
  • biome on all changed files; the full suite and type-check run in CI.

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing (new tests pass the test-audit authoring gate)
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

@vercel

vercel Bot commented Nov 6, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
docs Ready Ready Preview Nov 6, 2026 2:07am UTC

Request Review

@greptile-apps

greptile-apps Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[High impact] The PR appears safe to merge, with non-blocking checker and log-correlation issues remaining.

Findings

  1. P2 `reqId` messages escape the check ▶
  2. P2 Reassigned messages fail the check ▶
  3. P2 Webhook failures become indistinguishable ▶
  4. P2 Unrelated variables fail log checks ▶

Summary

Moves correlation ids from log messages into fields and adds a check against putting them back.

  • Log lines keep the request ID set by their active request.
  • Log messages keep stable text while IDs move into fields.
  • Poll routes use the request ID for work tied to that request.
  • The source check flags interpolated IDs in logger messages.

Reviews (2) · Last reviewed commit: "fix(logs): export call fields to OpenTel..." · Reviewed by Greptile

Comment thread scripts/check-source-conventions.ts Resolved

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 325 files

Note: This PR contains a large number of files. cubic selects up to 200 of the highest-priority eligible files for this review, so some files may not have been reviewed.

Turn on auto-fix | Re-trigger cubic

Comment thread .agents/skills/add-trigger/SKILL.md Resolved
Comment thread apps/sim/lib/table/import-runner.ts Resolved
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 331 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Note: This PR contains a large number of files. cubic selects up to 200 of the highest-priority eligible files for this review, so some files may not have been reviewed.

Turn on auto-fix | Re-trigger cubic

Comment thread scripts/check-source-conventions.ts Resolved
Comment thread scripts/check-source-conventions.ts Resolved
…ta and enforce it

Log messages no longer open with an interpolated [${id}] tag or interpolate a request id. The id moves into the line's metadata, so messages group in queries and lines filter by field. Inside withRouteHandler a generateRequestId() id is already stamped on every line by the request context, so those tags are dropped without a field.

Routes, table runners, the Sim event emitter and the poll orchestrator that minted their own correlation ids now use generateRequestId(), so their lines carry the request's id. Each webhook poll logs through a logger bound to its webhook id, so concurrent polls in one tick stay distinguishable.

check:source-conventions gains log-message-tag: it covers every app and package that imports @sim/logger, follows a message through concatenation, ternaries, local consts and helpers handed the caller's logger, and starts at zero.
…elds from erasing it

An explicit requestId on a line, or one bound with withMetadata, replaced the request context's id in the JSON line, and an undefined one deleted it, so a line could stop correlating with its request. Inside a request context the context's id is now authoritative, matching the OpenTelemetry export; a requestId passed with a line names the request only outside one. Undefined fields no longer erase fields the line already carries.

v2 chat logs its mothership request id as simRequestId. Request ids that defaulted to 'unknown' default to generateRequestId().

log-message-tag now sees optional calls, computed levels, withMetadata-derived loggers and log-named receivers, follows let prefixes, resolves names per function, and only treats an interpolated requestId or reqId, and log* helpers or .call(logger), as message sources.
…event

Executions started by one poll tick, or by one parent request, would otherwise share a request id, and tool code that keys idempotency on it (deployments) could replay one execution's result in another. The logger now owns requestId on the line, so these ids no longer need to match the request to correlate.
…lls apart

Scalar fields passed with a log line now become OpenTelemetry attributes, matching their top-level keys in the JSON line, so ids moved out of message text stay filterable in both sinks. Nested values stay in log.args. The request context still owns requestId.

The polled-webhook processor logs through a logger bound to the webhook id, so lines from concurrent polls in one tick stay distinguishable. Test expectations follow the new metadata shape.
…in log-message-tag

The cheap pre-filter now admits a module whose only request id is reqId, and a let binding reassigned before it is logged is no longer followed to its initializer.
@waleedlatif1
waleedlatif1 force-pushed the improvement/log-correlation-fields branch from 2c48f83 to 5c9823b Compare November 6, 2026 02:01
@waleedlatif1
waleedlatif1 merged commit 7e79983 into staging Nov 6, 2026
91 checks passed
@waleedlatif1
waleedlatif1 deleted the improvement/log-correlation-fields branch November 6, 2026 02:37

This branch was successfully deployed

1 active deployment
Preview — 5c9823b1 Deployed Nov 6, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant