feat(template): support Jinja include/import/extends for prompts loaded via !file

[hertznsk]

Summary

Closes #287.

Prompt templates loaded via prompt: !file ... (and system_prompt: !file ...) now support loader-dependent Jinja constructs — {% include %}, {% import %}, and {% extends %} — resolved relative to the prompt file's own directory. This enables reusable prompt partials across workflows.

The fix follows the suggested direction from the issue (options 1 + 2 + 3 + 4 combined): preserve the source path of !file-loaded prompts, give the Jinja renderer a FileSystemLoader rooted at that path, document the resolution rules, and produce clear error messages for both missing includes and inline prompts that attempt loader-dependent constructs.

Changes

• src/conductor/file_string.py (new leaf module): FileString(str) subclass carrying source_path: Path; __getnewargs__ keeps it safe across copy.deepcopy / pickle / model_copy(deep=True).
• src/conductor/config/loader.py: !file raw-string content is returned as FileString (dict/list parsing unchanged); _resolve_env_vars_recursive() preserves FileString metadata through ${VAR} resolution.
• src/conductor/config/schema.py: Pydantic wrap-validators on AgentDef.prompt / system_prompt preserve the FileString subclass through validation; public annotations unchanged; model_dump(mode="json") still yields plain strings. Other !file fields (command, stdin, reason, value, ...) intentionally keep coercing to plain str.
• src/conductor/executor/template.py: TemplateRenderer.render() branches on isinstance(FileString) — file-backed prompts render in a _DictSafeEnvironment with FileSystemLoader(source_path.parent) via env.compile(..., filename=...) + from_code(...); inline prompts use the unchanged BaseLoader + from_string path. TemplateNotFound errors now name the searched directory (Template not found: '<name>'. Searched in: <dir>), and inline prompts with {% include %} get an explicit error explaining that loader-dependent constructs require prompt: !file ....
• docs/workflow-syntax.md: new "Jinja Includes in Prompt Files" subsection with example, resolution rules, and the documented limitation that conductor validate does not scan inside included files.
• Tests: 36 new tests — unit (FileString, loader, env preservation, Pydantic, renderer) and end-to-end integration covering prompt, system_prompt, for_each inline agents, and human_gate (shared-renderer side effect), plus missing-include failure path.

Design decisions (scope guardrails)

• Relative includes resolve only against the prompt file's own directory — no workflow-directory fallback, no user-configurable template search path (can be added later if needed).
• Inline prompts behave exactly as before; all changes are gated behind isinstance(FileString).
• human_gate prompts loaded via !file gain include support as a natural side effect of the shared renderer.

Test plan

• uv run pytest -m "not install_scripts" tests/ — 3857 passed (1 pre-existing env-only real_api failure reproduced identically on main, unrelated).
• make check (ruff lint + format + ty) — exit 0.
• Manual QA: extracted the docs example into fixtures and validated with uv run conductor validate (exit 0); verified rendered prompt/system_prompt reach the provider with partials inlined; verified both new error messages.

[hertznsk]

@microsoft-github-policy-service agree

[jrob5756]

Another great PR. Thanks @hertznsk ! I found two behavior issues in the file-backed rendering path. Both are worth fixing before merge because they can leave placeholders unresolved or turn a missing file into a misleading inline-template error.

[jrob5756]

Jinja loads partials after the config loader has finished ${VAR} expansion. Variables in an included, imported, or base template therefore stay literal, while the same text in the main !file prompt is expanded. An unset required variable also stops producing the normal configuration error. Please run each template source through the existing environment resolver before Jinja compiles it, and add an end-to-end partial test.

[hertznsk]

Fixed in b80e95e.

Each partial source now goes through the config loader's own env resolver before Jinja compiles it: _EnvResolvingFileSystemLoader(FileSystemLoader).get_source() runs resolve_env_vars(source) on the text of every {% include %} / {% import %} / {% extends %} target (nested and dynamic includes included), so ${VAR} / ${VAR:-default} in partials behaves identically to the root !file prompt. An unset required variable propagates as the original ConfigurationError (dedicated except ConfigurationError: raise ahead of the template-error handlers), preserving its message and suggestion.

Tests: unit coverage in tests/test_executor/test_template_filestring.py (render-time resolution, default fallback, unset-required ConfigurationError) plus an end-to-end test in tests/test_integration/test_file_prompt_includes.py that sets the env var after config load to prove resolution happens at render time.

[jrob5756]

If the original file is removed or inaccessible after loading, this condition quietly treats the FileString as an inline prompt. Includes then fail with a message telling the user to switch to !file, even though they already did. Please report the unavailable source_path directly instead of changing rendering modes.

[hertznsk]

Fixed in b80e95e.

The file-backed branch no longer degrades silently: TemplateRenderer.render() now checks template.source_path.is_file() for every FileString before entering the try block, and raises immediately:

TemplateError: File-backed prompt source is no longer available: '<path>' (loaded via !file).

with a suggestion to restore the file — no inline fallback, no misleading "convert to prompt: !file" guidance, and no double-wrapping by the generic handler (the tests assert the exact message and __cause__ is None). Note the check uses is_file(), so a directory or broken symlink at that path is reported the same way.

Tests: test_missing_source_path_raises_explicit_error (unit) and test_deleted_prompt_source_fails_with_explicit_error (end-to-end: load config, delete the prompt file, run the engine, assert the error names the missing path).

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
⚠️ Please upload report for BASE (main@7aaa589). Learn more about missing BASE report.

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #291   +/-   ##
=======================================
  Coverage        ?   89.33%           
=======================================
  Files           ?       72           
  Lines           ?    12778           
  Branches        ?        0           
=======================================
  Hits            ?    11415           
  Misses          ?     1363           
  Partials        ?        0           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[jrob5756]

lgtm. Thanks!