[superlog] Use warn for tool-level errors to prevent false ERROR on succeeded jobs#504
[superlog] Use warn for tool-level errors to prevent false ERROR on succeeded jobs#504superlog-app[bot] wants to merge 1 commit into
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
2 Skipped Deployments
|
|
The latest updates on your projects. Learn more about Unkey Deploy
|
Greptile SummaryThis PR fixes a log-level escalation bug where completed insights generation jobs were emitted at ERROR severity because individual AI tool failures (retried internally by the agent) called
Confidence Score: 4/5Safe to merge for the insights use case; the change is small and the fatal-error path in jobs.ts is untouched. The core fix is correct and well-documented — transient tool failures no longer permanently escalate the job-level log. The only nuance is that the change globally affects the apps/slack and apps/api requestLogger paths too, not just insights. Those contexts share the same createToolLogger package, so their tool-level errors also shift to WARN. This is likely fine given the same agent-recovery reasoning, but it was not explicitly analyzed in the PR. The single changed file packages/ai/src/ai/tools/utils/logger.ts is straightforward. Worth a quick sanity-check on apps/slack and apps/api to confirm they are fine with tool errors surfacing only at WARN on their request loggers. Important Files Changed
Sequence Diagram%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
participant Agent as AI Agent
participant Tool as SQL Tool
participant Logger as createToolLogger
participant RL as RequestLogger (evlog)
participant Job as jobs.ts catch block
Note over Agent,Job: Tool failure that agent recovers from
Agent->>Tool: execute()
Tool->>Tool: chQuery() fails
Tool->>Logger: .error("Query failed", ctx)
rect rgb(255, 220, 220)
Note over Logger,RL: BEFORE (bug)
Logger->>RL: requestLogger.error(err, ctx)
RL-->>RL: "hasError = true (permanent)"
end
rect rgb(220, 255, 220)
Note over Logger,RL: AFTER (fix)
Logger->>RL: requestLogger.warn(message, ctx)
RL-->>RL: hasError stays false
end
Tool-->>Agent: throws error
Agent->>Tool: retry with reformulated query
Tool->>Tool: chQuery() succeeds
Tool-->>Agent: result
Agent-->>Job: job completes successfully
Job->>RL: "emit({ job_status: succeeded })"
Note over RL: BEFORE: emits at ERROR (hasError=true)
Note over RL: AFTER: emits at INFO (hasError=false)
Note over Job: Real failures still log at ERROR
Job->>RL: logger.error(err) in catch block
RL-->>RL: "hasError = true (correct)"
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
participant Agent as AI Agent
participant Tool as SQL Tool
participant Logger as createToolLogger
participant RL as RequestLogger (evlog)
participant Job as jobs.ts catch block
Note over Agent,Job: Tool failure that agent recovers from
Agent->>Tool: execute()
Tool->>Tool: chQuery() fails
Tool->>Logger: .error("Query failed", ctx)
rect rgb(255, 220, 220)
Note over Logger,RL: BEFORE (bug)
Logger->>RL: requestLogger.error(err, ctx)
RL-->>RL: "hasError = true (permanent)"
end
rect rgb(220, 255, 220)
Note over Logger,RL: AFTER (fix)
Logger->>RL: requestLogger.warn(message, ctx)
RL-->>RL: hasError stays false
end
Tool-->>Agent: throws error
Agent->>Tool: retry with reformulated query
Tool->>Tool: chQuery() succeeds
Tool-->>Agent: result
Agent-->>Job: job completes successfully
Job->>RL: "emit({ job_status: succeeded })"
Note over RL: BEFORE: emits at ERROR (hasError=true)
Note over RL: AFTER: emits at INFO (hasError=false)
Note over Job: Real failures still log at ERROR
Job->>RL: logger.error(err) in catch block
RL-->>RL: "hasError = true (correct)"
|
Summary
Insights generation jobs that succeed are being logged at ERROR level because individual SQL query tool failures (which the AI agent internally handles and retries) call
requestLogger.error()on the job-level evlog logger. In evlog, onceerror()is called on aRequestLogger, it setshasError = truepermanently, so the finalemit()at the end of the job uses ERROR severity even whenjob_status: "succeeded".The
setAiRequestLoggerProvider(getActiveInsightsLog)wiring inapps/insights/src/index.tsmeanscreateToolLogger'serror()method writes directly to the job-level wide-event logger. Tool errors that the AI agent recovers from (e.g., a SQL query fails on the first attempt, the agent reformulates and retries successfully) were incorrectly escalating the entire job log to ERROR.The fix changes
createToolLogger.error()to callrequestLogger.warn()instead ofrequestLogger.error(). Job-level failures still correctly log at ERROR becauseapps/insights/src/jobs.ts:180callslogger.error(err)directly in the job's own catch block — that path is unchanged.Alternative: a separate evlog
RequestLoggerscope could be created per tool call so tool errors are isolated from the job-level logger. That would give more granular scoping but requires a larger change across all tool implementations. Thewarn()approach is a minimal, low-risk fix with the same outcome.Incident on Superlog
Was this PR helpful? Leave feedback — goes straight to the Superlog team.
Summary by cubic
Prevents successful insights jobs from being marked as ERROR by downgrading tool-level failures to warnings. Job-level failures still log as errors.
packages/ai/src/ai/tools/utils/logger.ts,createToolLogger.error()now callsrequestLogger.warn()instead ofrequestLogger.error().Written for commit 1a436ea. Summary will update on new commits.