Make agent automations tool-driven by default

[sentry-junior]

Scheduled and event tasks still behave like delayed Slack turns: they get Delivery, and the final assistant message posts unless the model emits [[NO_REPLY]]. That works for reminders and digests. It is noisy for silent work such as “when CI fails, fix it.”

Current behavior

• Scheduled and event tasks create a Conversation at an explicit Destination and supply Delivery for assistant output (packages/junior/src/chat/README.md, Simplify Junior around one core Conversation and optional Location #1563).
• Task agent input still uses a prompt-level reply contract and [[NO_REPLY]] for silence (packages/junior/src/chat/task-input.ts, Overhaul Junior automations #1717).
• There is no first-class delivery mode on the stored task; silence is prose, not data (Overhaul Junior automations #1717).
• Event tasks and scheduled tasks share task input framing, but product vocabulary and dashboard still center on “Task.”

Gap

Users need unattended work that can act without publishing a channel update. Prompt-based silence is unreliable, and automatic assistant Delivery makes completion chatter the default.

User stories

Slack

• Reminder: “Remind me Friday at 9am to submit expenses.” Posts one reminder message at the right time. No model work required for fixed text.
• Recurring post: “Post standup time every weekday at 10am in #eng.” Posts the fixed text on cadence.
• Computed digest: “Every Friday, summarize open launch blockers in #launch.” Agent gathers live state, then posts one tool-authored digest.
• Silent maintenance: “Every night, archive stale threads in this channel.” Runs tools only. No completion message in the channel.
• New root message trigger: “Whenever someone posts a new top-level message in #support, triage it and label the thread.” Triggers only on new root messages in a channel, not thread replies; effects run via tools, with Delivery only if the user asked for one.
• Tell-me-when (temporary): “Tell me when this deploy finishes.” Temporary watch/notify in the current conversation, not a durable silent automation unless the user asks for durable behavior.
• Failure handling: If a silent automation fails repeatedly or needs auth, notify the creator privately. Do not dump an assistant explanation into the destination channel.

GitHub

• Fix failed checks: “Whenever checks fail on this PR, fix them.” Agent uses repo tools. No Slack completion message unless the user asked for one.
• Review triage: “When review requests changes on getsentry/junior PRs, summarize the feedback and open a follow-up checklist.” Effects are GitHub artifacts and optional explicit notify; not automatic assistant chatter.
• Label/route issues: “When a new issue is opened with ‘customer’, label it and assign the support rotation.” Tool effects only.
• Release notify: “When a release is published, post the notes in #releases.” Agent or fixed message posts one explicit Slack message.
• Watch vs automation: “Tell me when this PR is approved” stays a temporary watch. “Whenever a release is published, update the changelog PR” is a durable event automation.

Proposal (from product review)

Keep this small. Do not build a workflow graph.

1. Default: agent automations run with Location for tools, without assistant Delivery.
2. Visible output: only explicit tools (for example a Slack post tool with idempotent retries).
3. Simple remind/post fixed text: deterministic message action, no model loop.
4. Computed “post/summarize” jobs: agent action plus a required message effect when communication is the point.
5. Silent action jobs: agent action, zero destination messages on success.
6. Failures/auth: runtime-owned private owner notify; not model-authored channel spam.
7. Product framing: prefer Automation over overloaded Task language; align with Overhaul Junior automations #1717 naming, but ship behavior first if rename is larger.
8. Legacy: existing tasks that rely on final-answer delivery keep current behavior until edited/converted.

Out of scope for the first cut: DAG/node editor, second agent runtime, prompt-only silent flags, or a generic publish-anywhere core tool.

Related

• Overhaul Junior automations #1717 product model / delivery mode as data
• Simplify Junior around one core Conversation and optional Location #1563 Source, Location, Delivery separation
• Prior writeup: https://sentry.slack.com/docs/T024ZCV9U/F0BULRUV5KK

via David Cramer.

--

View Junior Session [Sentry]

[sentry-junior]

Revised scope after review

Narrowed this to one tight reliability slice instead of the full original proposal.

In scope:

1. Flip the default to silent. resolveTaskOutcomes() currently treats omitted outcomes as [{action: "send_message", destination: currentDestination}] (chatty by default). New automations should default to tools-only/silent unless the caller explicitly asks for a send_message outcome. Tool descriptions/schemas need to state this plainly so the model chooses correctly (e.g. "fix CI when it fails" vs "send a weekly digest").
2. Backfill legacy records via migration. Both junior_scheduler_tasks and junior_event_tasks store outcomes in JSONB and currently treat a missing field as legacy-equivalent-to-message at read time (effectiveTaskOutcomes()). Write a migration that backfills every row with missing outcomes to the explicit legacy equivalent ([{action: "send_message", destination: <stored destination>}]), then make outcomes required in the schema/types and remove the compatibility fallback in effectiveTaskOutcomes().
3. Add behavioral eval coverage verifying the model picks silent vs. message outcomes correctly for representative prompts (silent maintenance job vs. computed digest vs. simple reminder).

Deliberately out of scope (for this slice):

• A dedicated idempotent Slack-post outcome tool — existing turn/Delivery path already posts with retryable-error handling; a separate primitive would duplicate machinery.
• Deterministic fixed-text remind/post path that skips the model entirely (avoids hallucinated content in things like reminder text). Real problem, but a separate action-shape change — tracked as a likely follow-up, possibly feeding into Scheduler fabricates mention links in task outputs #554.

Requested by David Cramer.