Skip to content

[BUG] [TS SDK OTEL] No workflow-level parent span when a run is triggered from the Hatchet dashboard #3706

@Mdev303

Description

@Mdev303

Summary

When a workflow run is triggered from the Hatchet dashboard (manual event push or manual run), the resulting trace in the observability backend has no workflow-level parent span. The step runs appear as orphaned siblings under the trace root, because the only place the TS SDK emits a workflow-level span is at the trigger site — and the dashboard is not an instrumented trigger site.

Current behavior

Given a workflow find-subprocessors triggered by pushing subprocessor.created from the dashboard:

Image

The step spans share a trace ID (the engine propagates a traceparent via action.additionalMetadata), but the parent span ID they reference was never exported — so no workflow envelope shows up in the trace viewer.

Expected behavior

A workflow-level span should exist around all step runs of a dashboard-triggered run, e.g.:

Trace 76f869...
└── hatchet.workflow_run (find-subprocessors, workflow_run_id=…)
├── hatchet.start_step_run.find-subprocessors:resolve-parent-company
├── hatchet.start_step_run.find-subprocessors:search-company-subprocessors
└── ...

Why this matters

The dashboard is the canonical way to trigger runs outside the SDK — retries, one-off replays, operator debugging, first-run-during-setup, etc. all go through it. Today every one of those runs loses its workflow envelope in tracing, which is exactly when you most want to see it.

Root cause

In sdks/typescript/src/opentelemetry/instrumentor.ts, workflow-level spans (hatchet.push_event, hatchet.run_workflow, hatchet.schedule_workflow) are created only at the trigger site. Worker-side instrumentation only emits per-step hatchet.start_step_run.* spans. The dashboard is not a trigger site that participates in the user's trace export, so the parent span simply doesn't exist.

Suggested fix

Have the dashboard's "push event" / "trigger run" action emit a span (or call through an instrumented path) so the traceparent it injects into additionalMetadata points at a span that actually gets exported. The simplest form: the dashboard backend opens a hatchet.dashboard_trigger (or equivalent) span when the user clicks trigger, and the existing propagation machinery in the TS SDK will chain the step spans under it automatically.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions