Fix explicit skill availability in empty mode

.github/workflows/docs.yml

@@ -20,13 +20,13 @@ on:
   workflow_call:
     inputs:
       version:
-        description: "Version tag (e.g. v1.0.0) to validate and build from"
+        description: "Version tag (e.g. v1.0.1) to validate and build from"
         required: false
         type: string
   workflow_dispatch:
     inputs:
       version:
-        description: "Version tag (e.g. v1.0.0) to validate and build from"
+        description: "Version tag (e.g. v1.0.1) to validate and build from"
         required: false
         type: string
 

CHANGELOG.md

@@ -19,6 +19,26 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
   frozen guidance exists only as a historical experiment fixture. Evaluating
   consumer-provided skills remains supported.
 
+## [1.0.1] - 2026-10-01
+
+### Fixed
+
+- Explicit skill directories now enable native SDK skills in empty mode without
+  enabling ambient discovery. Individual directories and parents are checked
+  before model execution; missing, unreadable, invalid, duplicate-name, or
+  undiscovered requested skills fail clearly without sending the task.
+- Explicit directories combined with `enable_skills=False` are rejected.
+  Named `disabled_skills` remain supported for deliberate exclusions.
+
+### Added
+
+- Public `DiscoveredSkill` and `SkillDiscovery` evidence types and
+  `CopilotResult.skill_discovery`, preserving actual SDK availability and loading
+  diagnostics separately from skill reads. Native schema-4.0 evidence now requires
+  `eval_result.skill_discovery`; regenerate older evidence with the current producer.
+- Unbilled real-SDK discovery and isolation checks, plus a request blocked before
+  network forwarding to verify ambient instructions do not enter model context.
+
 ## [1.0.0]
 
 ### Breaking changes

docs/contributing/evidence-structure.md

@@ -19,6 +19,13 @@ execution configuration, turns, tool arguments and outputs, completion status,
 capture errors, request audits, stop reasons, usage, and missing-pricing models.
 Private SDK/session fields are not serialized.
 
+`eval_result.skill_discovery` saves actual public SDK availability separately
+from requested configuration and observed reads. It preserves metadata, loading
+warnings/errors, and whether discovery completed; `null` means no check was
+attempted. This field is required by the current schema-4.0 producer/reader.
+Regenerate older evidence missing it with the current producer; do not infer
+availability from supplied paths or patch saved JSON.
+
 Preserve missing versus empty output and nullable usage. Do not replace unsupported
 property values with strings or synthesize uncaptured successful assertions.
 The loader rejects old schemas; there is no compatibility reader.

docs/getting-started/skills.md

@@ -96,10 +96,42 @@ from pytest_skill_engineering.copilot import CopilotEval
 
 agent = CopilotEval(
     name="with-skill",
+    client_mode="empty",
     skill_directories=["skills/financial-advisor"],
 )
 ```
 
+Supply either a directory containing `SKILL.md` or its parent containing multiple
+immediate skill directories. Relative paths are resolved from the Python process's
+current directory, not `working_directory`.
+
+Explicit `skill_directories` enable native SDK skill loading automatically,
+including in `client_mode="empty"`. This does **not** enable ambient configuration
+discovery. Empty mode still excludes personal/project skills and custom instructions,
+file hooks, managed settings, and persona-provided instructions/tools. Supply the
+tools needed by your task explicitly; empty mode's default tool list is empty.
+
+Before sending the task, the runner validates every requested skill using the
+existing skill format checks, then checks availability through the public SDK
+`session.rpc.skills.ensure_loaded()`, `reload()`, and `list()` APIs. Missing,
+unreadable, invalid, duplicate-name, or undiscovered skills return
+`success=False`, `stop_reason="execution_error"`, and a setup error without
+sending a model message. A parent with no immediate `SKILL.md` files is an error;
+unrelated child directories without `SKILL.md` are not skill packages.
+SDK loading warnings are logged and saved; loading errors stop execution.
+There is still only one execution attempt.
+
+`extra_config={"enable_skills": False}` conflicts with nonempty explicit
+directories and raises `ValueError` from `build_session_config()`. For a
+no-skill baseline, remove the directories instead. To deliberately disable
+particular discovered skills, use `disabled_skills=["skill-name"]`; these skills
+remain in discovery evidence with `enabled=False`. Passthrough directories follow
+the same enable/check rules and cannot replace a nonempty `skill_directories` field.
+
+`result.skill_discovery` records actual SDK metadata and loading diagnostics,
+not a claim that the agent read or followed a skill. Observed reads remain
+ordinary tool-call evidence. No setup check tells the agent to read a skill.
+
 ## Testing Skill Effectiveness
 
 Compare agents with and without skills:

docs/reference/configuration.md

@@ -46,14 +46,34 @@ agent = CopilotEval(
 | `mcp_servers` | Copilot SDK server config mapping |
 | `allowed_tools` | Optional global tool allow-list |
 | `custom_agents` | Loaded `.agent.md` definitions |
-| `skill_directories` | Skill packages to inject |
+| `skill_directories` | Explicit individual skill directories or parents; enable loading and check SDK availability before sending |
+| `disabled_skills` | Skill names deliberately disabled, retained in discovery evidence |
 | `max_turns` | Advisory top-level turn budget; used for subagent turn caps |
 | `timeout_s` | Hard wall-clock limit for the run |
 
 Each `copilot_eval` call makes one execution attempt. Startup, connection, and
 task failures are recorded without automatically starting another session.
 Rerun a selected pytest case explicitly when appropriate.
 
+### Explicit skills and empty mode
+
+Nonempty `skill_directories` set the public SDK `enable_skills=True` by default.
+`enable_config_discovery` remains false. In empty mode, personal/project discovery,
+custom instructions, file hooks, managed settings, and persona injection stay
+disabled. Explicit tools and system prompt content remain supported.
+
+Explicit directories combined with `extra_config={"enable_skills": False}` are
+rejected, not silently ignored. `None` leaves the explicit-skills default enabled.
+An empty directory list plus `enable_skills=False` is allowed. Use `disabled_skills`
+to deliberately disable named skills within an otherwise loaded set.
+
+Every requested directory must contain a valid `SKILL.md` or immediate child
+skill directories. Relative paths use the Python caller's current directory.
+Validation and public SDK discovery run before the task is sent; setup failure
+is recorded as an execution error with no model send. SDK metadata and diagnostics
+are available as `result.skill_discovery` and in native JSON, separately from reads.
+See [Skills](../getting-started/skills.md) for input and evidence details.
+
 ## MCP server config shape
 
 Attach servers directly to `CopilotEval(mcp_servers={...})`.

docs/reference/result.md

@@ -24,6 +24,7 @@ description: "CopilotResult reference: tool calls, responses, token usage, files
 | `stop_reason` | `completed`, `tool_budget_exceeded`, `timeout`, `request_audit_error`, `execution_error`, or `cleanup_error`; `None` for a result not finalized by the runner |
 | `tool_calls_admitted` | Calls admitted by the framework after caller guards; not proof of execution or application success |
 | `request_audit` | Actual outgoing request records when `audit_requests=True` |
+| `skill_discovery` | Actual pre-execution SDK skill metadata and diagnostics for explicit directories; `None` means no discovery check was attempted |
 
 `success` describes session execution, not whether the application did the right
 thing. A tool can fail and the session can recover. Check application state with
@@ -34,6 +35,25 @@ Missing completion, uncorrelated events, conflicting duplicate events, and
 cancellation make session `success` false. A completed call whose output was not
 captured makes `evidence_complete` false without changing the reported tool outcome.
 
+## Skill availability
+
+For explicit directories, `skill_discovery` contains `skills`, `warnings`,
+`errors`, and `complete`. `complete=True` means discovery finished, not that it
+succeeded: inspect `errors` and the result's `success` too. A failed local or SDK
+check can retain an incomplete record. `skills=[]` means the completed SDK list
+was empty; `skill_discovery=None` means availability was not checked.
+
+Each discovered skill retains SDK `name`, `description`, `source`, `enabled`,
+`user_invocable`, `path`, `plugin_name`, `command_name`, and `argument_hint`.
+Optional metadata remains `None` when absent. `disabled_skills` entries can
+deliberately have `enabled=False`; unexpectedly disabled requested skills block
+the task before a model send.
+
+Availability does not establish that a skill was read, followed, or useful.
+Actual reads remain in the ordinary tool calls; application checks establish
+task correctness. The runner neither forces a read nor changes the task prompt.
+Native JSON saves discovery separately from configuration and tool calls.
+
 ## Captured tool evidence
 
 Each item in `all_tool_calls` has these fields:

pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "pytest-skill-engineering"
-version = "1.0.0"
+version = "1.0.1"
 description = "Reliable Copilot execution and evidence for coding-agent-led testing of MCP servers, CLI tools, skills, and custom agents."
 readme = "README.md"
 license = { text = "MIT" }

src/pytest_skill_engineering/copilot/__init__.py

@@ -22,11 +22,14 @@
 from pytest_skill_engineering.copilot.requests import RequestAudit
 from pytest_skill_engineering.copilot.result import CopilotResult, UsageInfo
 from pytest_skill_engineering.core.evals import load_custom_agent, load_custom_agents
+from pytest_skill_engineering.core.result import DiscoveredSkill, SkillDiscovery
 
 __all__ = [
     "CopilotEval",
     "CopilotResult",
+    "DiscoveredSkill",
     "RequestAudit",
+    "SkillDiscovery",
     "UsageInfo",
     "ClaudeCodePersona",
     "CopilotCLIPersona",

src/pytest_skill_engineering/copilot/eval.py

@@ -139,7 +139,7 @@ class CopilotEval:
     # display_name, tools, mcp_servers, infer)
     custom_agents: list[CopilotCustomAgentConfig] = field(default_factory=list)
 
-    # Skill directories
+    # Explicit skills are enabled and checked before model execution.
     skill_directories: list[str] = field(default_factory=list)
     disabled_skills: list[str] = field(default_factory=list)
 
@@ -235,8 +235,24 @@ def build_session_config(self) -> dict[str, Any]:
             config["hooks"] = self.hooks
 
         # Apply extra_config passthrough
+        if (
+            self.skill_directories
+            and "skill_directories" in self.extra_config
+            and self.extra_config["skill_directories"] != self.skill_directories
+        ):
+            raise ValueError("extra_config must not override explicitly supplied skill_directories")
         config.update(self.extra_config)
 
+        if config.get("skill_directories"):
+            if config.get("enable_skills") is False:
+                raise ValueError(
+                    "Explicit skill_directories conflict with enable_skills=False; "
+                    "remove the directories for a no-skill baseline, or enable skills"
+                )
+            if config.get("enable_skills") is not None and config["enable_skills"] is not True:
+                raise ValueError("enable_skills must be a boolean or None")
+            config["enable_skills"] = True
+
         if self.client_mode == "empty":
             isolated = {
                 "enable_config_discovery": False,

src/pytest_skill_engineering/copilot/fixtures.py

@@ -116,6 +116,7 @@ def _convert_to_aitest(
         stop_reason=result.stop_reason,
         usage=list(result.usage),
         tool_calls_admitted=result.tool_calls_admitted,
+        skill_discovery=deepcopy(result.skill_discovery),
         configuration=deepcopy(
             {
                 "name": agent.name,

src/pytest_skill_engineering/copilot/result.py

@@ -12,7 +12,13 @@
 
 from pytest_skill_engineering.copilot.contracts import CopilotResultAgent, SubagentInvocation
 from pytest_skill_engineering.copilot.requests import RequestAudit
-from pytest_skill_engineering.core.result import StopReason, ToolCall, Turn, UsageInfo
+from pytest_skill_engineering.core.result import (
+    SkillDiscovery,
+    StopReason,
+    ToolCall,
+    Turn,
+    UsageInfo,
+)
 
 __all__ = [
     "CopilotResult",
@@ -68,6 +74,7 @@ class CopilotResult:
     request_audit: list[RequestAudit] = field(default_factory=list)
     stop_reason: StopReason | None = None
     tool_calls_admitted: int = 0
+    skill_discovery: SkillDiscovery | None = None
 
     @property
     def evidence_complete(self) -> bool:

src/pytest_skill_engineering/copilot/runner.py

@@ -14,6 +14,7 @@
 
 import asyncio
 import logging
+from pathlib import Path
 from tempfile import TemporaryDirectory
 from typing import TYPE_CHECKING
 
@@ -27,6 +28,8 @@
 from pytest_skill_engineering.copilot.events import EventMapper
 from pytest_skill_engineering.copilot.requests import RequestAuditHandler
 from pytest_skill_engineering.copilot.result import CopilotResult, StopReason
+from pytest_skill_engineering.copilot.skills import discover_requested_skills, requested_skill_files
+from pytest_skill_engineering.core.result import SkillDiscovery
 
 if TYPE_CHECKING:
     from copilot.client import CopilotClient
@@ -80,8 +83,20 @@ async def run_copilot(agent: CopilotEvalConfig, prompt: str) -> CopilotResult:
     reason: StopReason = "completed"
     error: str | None = None
     cleanup_errors: list[str] = []
+    skill_discovery: SkillDiscovery | None = None
+    send_started = False
     try:
         session_config = agent.build_session_config()
+        skill_directories = session_config.get("skill_directories") or []
+        requested_skills: dict[Path, str] = {}
+        if skill_directories:
+            skill_discovery = SkillDiscovery()
+            if not isinstance(skill_directories, list):
+                raise ValueError("skill_directories must be a list of directory paths")
+            requested_skills = requested_skill_files(skill_directories)
+            session_config["skill_directories"] = [
+                str(Path(directory).resolve()) for directory in skill_directories
+            ]
         caller_hooks = dict(session_config.get("hooks") or {})
         session_config["hooks"] = dict(caller_hooks)
         # Empty mode must not read persona instruction files or inject tools.
@@ -123,12 +138,20 @@ def capture(event: SessionEvent) -> None:
         )
 
         async def execute() -> None:
-            nonlocal session
+            nonlocal session, send_started
             assert client is not None
             async with asyncio.timeout(min(60, agent.timeout_s)):
                 await client.start()
             async with asyncio.timeout(min(30, agent.timeout_s)):
                 session = await client.create_session(**session_config)
+                if skill_discovery is not None:
+                    await discover_requested_skills(
+                        session,
+                        requested_skills,
+                        session_config.get("disabled_skills") or [],
+                        skill_discovery,
+                    )
+            send_started = True
             await session.send_and_wait(prompt, timeout=agent.timeout_s)
 
         execution = asyncio.create_task(execute())
@@ -158,6 +181,9 @@ async def execute() -> None:
     except Exception as exc:
         logger.error("Copilot execution failed: %s", exc)
         reason, error = "execution_error", str(exc)
+        if skill_discovery is not None and not skill_discovery.complete:
+            error = f"Skill setup failed before model execution: {exc}"
+            skill_discovery.errors.append(error)
     finally:
         controls.closed = True
         for watcher in watchers:
@@ -210,12 +236,13 @@ async def execute() -> None:
 
     result = mapper.build()
     result.agent = agent
+    result.skill_discovery = skill_discovery
     result.tool_calls_admitted = controls.admitted
     if result.capture_errors:
         result.capture_errors.extend(controls.incomplete_diagnostics(abort_phase=reason))
     if audit is not None and capture_requests:
         result.request_audit = list(audit.records)
-        if audit.error or (session is not None and not audit.observed_requests):
+        if audit.error or (send_started and not audit.observed_requests):
             audit_error = audit.error or "No outbound model requests captured; audit unsupported"
             result.capture_errors.append(audit_error)
             if reason == "completed" or audit.error: