fix(providers): pass agent system_prompt as native system parameter in Claude provider by hertznsk · Pull Request #293 · microsoft/conductor

hertznsk · 2026-07-13T13:17:27Z

Summary

Fixes #289. The native claude provider (Anthropic Messages API) silently dropped agent.system_prompt. It is now passed as the native top-level system parameter of messages.create() on every API call in the agent execution path: main loop, tool-use iterations, parse recovery, interrupt partial output, and retries. Empty/whitespace system prompts are normalized away and not sent.

Secondary fix: the OutputValidator rubric (synthetic agent with prompt="" and a formatted system_prompt) now reaches the model on the claude provider through the same native system parameter.

Changes

Provider (src/conductor/providers/claude.py): thread a keyword-with-default system_prompt: str | None = None (appended at the end of each private signature, mirroring the existing thinking precedent) through _execute_with_retry → _execute_agentic_loop → _request_partial_output / _execute_with_parse_recovery → _execute_api_call, where kwargs["system"] is set only when non-empty after strip-normalization. Public AgentProvider.execute signature unchanged.
Validator warning (src/conductor/config/validator.py): the system_prompt-without-prompt warning no longer claims Claude ignores system_prompt; rationale is now provider-neutral (empty user message on every provider).
Tests: new tests/test_providers/test_claude_system_prompt.py (8 tests — system kwarg on plain execute / every tool-loop call / parse-recovery calls / interrupt partial-output call; absent for None/empty/whitespace; validator rubric arrives formatted, not raw) + negative assertions in tests/test_config/test_validator.py.
Docs: new "System Prompt" section in docs/providers/claude.md; wording fix in the conductor skill's execution reference.

Testing

New test file: 8 passed.
Full suite: 3803 passed (one deselected integration test hits the live GitHub Copilot API and fails in this sandbox due to org policy — it fails identically on unmodified main).
ruff check clean; conductor validate shows the rewritten warning on a real workflow.

Notes

Behavior change (intended): claude agents that define system_prompt now actually receive those instructions; previously they were silently dropped.

The warning no longer claims that Claude ignores system_prompt entirely. The rationale is now provider-neutral: omitting prompt sends an empty user message to the model on every provider, which almost always indicates a latent authoring mistake. The test pins that the warning still fires for system_prompt without prompt and contains no ignore/entirely claims. Refs microsoft#289

…o Anthropic API The agent execution path now sends the rendered agent.system_prompt as the native top-level system parameter on every messages.create() call: main loop, tool-use iterations, parse recovery, interrupt partial output, and retries. Empty/whitespace system prompts are normalized away and not sent. As a secondary fix, the OutputValidator rubric now reaches the model on the claude provider through the same native system parameter. Refs microsoft#289

…meter in Claude provider Add a System Prompt section to docs/providers/claude.md describing the mapping, the every-call-path behavior, whitespace normalization, and the deferred caching follow-up. Update the stale wording in the conductor skill's execution reference. Refs microsoft#289

jrob5756

Thank you for contributing to Conductor! A couple things that I caught... this fixes the missing Claude system prompt path, but the current normalization changes non-empty prompt text. I also left two wording corrections and two coverage requests for the retry and interrupt branches.

If you can wrap these up, we can merge! Thanks again.

jrob5756 · 2026-07-13T15:50:56Z

        # Done before the per-model max_tokens warning so the warning logic
        # accounts for thinking-aware caps.
        thinking = self._resolve_thinking_for_agent(agent, model)
+        system_prompt = (agent.system_prompt or "").strip() or None


Using .strip() here changes every non-empty system prompt with leading or trailing whitespace. The other providers pass the rendered value through unchanged, so Claude should only use trimming to detect a blank prompt. An exact-equality test with a multiline templated prompt would catch this.

Suggested change

system_prompt = (agent.system_prompt or "").strip() or None

raw_system_prompt = agent.system_prompt

system_prompt = (

raw_system_prompt if raw_system_prompt and raw_system_prompt.strip() else None

)

Fixed in bd0aa63. .strip() is now used only as a blank-prompt predicate — the rendered value is forwarded verbatim:

raw_system_prompt = agent.system_prompt system_prompt = ( raw_system_prompt if raw_system_prompt and raw_system_prompt.strip() else None )

This matches the pass-through behavior of the Copilot, Claude Agent SDK, and Hermes providers. The new retry and in-flight interrupt tests also pin verbatim forwarding with a leading-whitespace / trailing-newline prompt (" Preserve me exactly\n").

jrob5756 · 2026-07-13T15:50:56Z

+                "This sends an empty user message to the model on every provider, "
+                "which almost always indicates a latent authoring mistake. Move "


on every provider isn't true for Copilot, which sends the system and user sections as one non-empty SDK prompt. Describing the missing workflow task prompt keeps the warning accurate across providers.

Suggested change

"This sends an empty user message to the model on every provider, "

"which almost always indicates a latent authoring mistake. Move "

"The user-authored task prompt is empty, which almost always "

"indicates a latent authoring mistake. Move "

Fixed in bd0aa63 — the warning (and the comment above it) now uses your suggested wording:

The user-authored task prompt is empty, which almost always indicates a latent authoring mistake. Move ...

The test in tests/test_config/test_validator.py now also pins the new phrasing and asserts the old "empty user message" / "on every provider" claims are absent.

jrob5756 · 2026-07-13T15:50:56Z

 - For-each `source` format and reserved names
 - Stale agent references and undeclared explicit-mode dependencies in `prompt`, `system_prompt`, `command`, `args`, `working_dir`, `input_mapping`, parallel-group inputs, and workflow `output:` templates
- Warning when an agent defines `system_prompt` but no `prompt:` (unusual — system prompts are only meaningful alongside a user prompt)
+- Warning when an agent defines `system_prompt` but no `prompt:` (unusual — the user message would be empty)


This has the same provider-specific wording problem as the validator warning. Copilot doesn't send an empty SDK prompt; the workflow's task prompt is what is missing.

Suggested change

- Warning when an agent defines `system_prompt` but no `prompt:` (unusual — the user message would be empty)

- Warning when an agent defines `system_prompt` but no `prompt:` (unusual because the user-authored task prompt would be empty)

Fixed in bd0aa63 — applied your suggestion verbatim:

Warning when an agent defines system_prompt but no prompt: (unusual because the user-authored task prompt would be empty)

jrob5756 · 2026-07-13T15:50:56Z

+
+        assert mock_client.messages.create.call_count == 2
+        for call in mock_client.messages.create.call_args_list:
+            assert call.kwargs["system"] == "Rendered tool-loop instructions"


These two calls are turns within one execution, not the outer retry loop. Since the PR promises the system prompt survives retries, please add a retryable failure followed by success and assert both requests receive the same system value.

Added in bd0aa63: test_retry_loop_passes_system_prompt_on_every_attempt. The first messages.create raises a retryable ProviderError(status_code=503), the second succeeds, and the test asserts both requests carry the same verbatim system value (" Preserve me exactly\n"), plus len(provider.get_retry_history()) == 1 to prove the outer _execute_with_retry loop actually re-ran. Backoff is neutralized via RetryConfig(base_delay=0, max_delay=0, jitter=0) and a patched asyncio.sleep.

jrob5756 · 2026-07-13T15:50:56Z

+            return_value=_tool_response("emit_output", {"result": "partial"})
+        )
+        interrupt_signal = asyncio.Event()
+        interrupt_signal.set()


Setting the event before execute() only covers the between-turn branch. The in-flight race takes a different path, so please start a blocked API call, set the event after it begins, and assert the partial-output request keeps the system prompt.

Added in bd0aa63: test_in_flight_interrupt_partial_output_passes_system_prompt. The first messages.create signals call_started and then blocks on an event that is never released; the test sets the interrupt only after the call has begun, so the provider takes the asyncio.wait(FIRST_COMPLETED) race branch, cancels the in-flight task, and issues the partial-output request. The test asserts that request keeps the verbatim system prompt (trailing newline included) and that the appended interrupt message is present. The pre-existing between-turn test is kept as-is.

…ovider Address PR microsoft#293 review feedback: - Use .strip() only as a blank-prompt predicate; forward the rendered system_prompt unchanged, matching Copilot / Claude Agent SDK / Hermes pass-through behavior. Leading/trailing whitespace in templated prompts is now preserved end-to-end. - Reword the system_prompt-without-prompt validator warning: the accurate provider-neutral rationale is an empty user-authored task prompt (Copilot merges system+user into one non-empty SDK prompt, so "empty user message on every provider" was wrong). Same wording fix in the conductor skill execution reference. - Add regression coverage for the two previously untested branches: the outer retry loop (retryable 503 then success — both attempts send the same system kwarg) and the in-flight interrupt race (blocked API call cancelled mid-flight, partial-output request keeps system_prompt). - Pin verbatim forwarding in the OutputValidator test (rubric keeps its trailing newline). Refs microsoft#289

codecov-commenter · 2026-07-13T19:46:58Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
⚠️ Please upload report for BASE (main@7aaa589). Learn more about missing BASE report.

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #293   +/-   ##
=======================================
  Coverage        ?   89.36%           
=======================================
  Files           ?       72           
  Lines           ?    12786           
  Branches        ?        0           
=======================================
  Hits            ?    11426           
  Misses          ?     1360           
  Partials        ?        0

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Genadij Blinov added 3 commits July 13, 2026 16:15

jrob5756 reviewed Jul 13, 2026

View reviewed changes

jrob5756 merged commit 3a1366f into microsoft:main Jul 13, 2026
9 checks passed

jrob5756 mentioned this pull request Jul 13, 2026

chore(release): cut 0.1.21 #295

Merged

		"This sends an empty user message to the model on every provider, "
		"which almost always indicates a latent authoring mistake. Move "

	- Warning when an agent defines `system_prompt` but no `prompt:` (unusual — the user message would be empty)
	- Warning when an agent defines `system_prompt` but no `prompt:` (unusual because the user-authored task prompt would be empty)

Uh oh!

Conversation

hertznsk commented Jul 13, 2026

Summary

Changes

Testing

Notes

Uh oh!

jrob5756 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

codecov-commenter commented Jul 13, 2026

Codecov Report

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants