Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions docs/agent-recipes.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,11 @@ agents-shipgate apply-patches \

Consume the response to decide whether to proceed. Key fields:

- Detection silently skips common fixture corpus directories such as
`fixtures/`, `_fixtures/`, `__fixtures__/`, `testdata/`, `test_data/`,
`test-fixtures/`, `test_fixtures/`, `golden/`, and `goldens/` when they
are below the selected workspace. Point `--workspace` directly at a
fixture project if you intentionally want to classify that fixture itself.
- `is_agent_project` — `true` when at least one Python framework
scored ≥ 2.0 with a strong signal.
- `frameworks[]` — per-framework scores + evidence + candidate file
Expand Down
4 changes: 3 additions & 1 deletion docs/zero-install.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,12 +31,14 @@ The script's output is a **structural subset** of `agents-shipgate detect --json
"codex_plugin_candidates": [{"mode": "package", "path": "..."}],
"next_action": "agents-shipgate init --workspace .",
"workspace_signals": {...},
"script_version": "0.2.0"
"script_version": "0.2.1"
}
```

Like the canonical CLI, the script parse-probes each glob-matched MCP/OpenAPI candidate before suggesting it — a filename match is not a guarantee. A Cursor plugin `mcp.json` is an `mcpServers`-style host config, not an MCP tools-array export; suggesting it would make the next `init --write` → `scan` step fail. Rejected candidates appear under `excluded_sources` (`{type, path, reason}`) instead of `suggested_sources`. The probe is **JSON-only** (stdlib has no YAML parser): a `.json` candidate the adapters would reject is excluded here too, while a `.yaml`/`.yml` OpenAPI spec is always kept as a suggestion (never wrongly dropped). The real-world miss this guards against — `mcpServers`-style host configs — is always JSON, so the probe is exact where it matters.

Like `agents-shipgate detect`, the script silently skips common fixture corpus directories such as `fixtures/`, `_fixtures/`, `__fixtures__/`, `testdata/`, `test_data/`, `test-fixtures/`, `test_fixtures/`, `golden/`, and `goldens/` when they are below the selected workspace. Point `--workspace` directly at a fixture project if you intentionally want to classify that fixture itself.

The script and the canonical CLI are pinned to **structural verdict parity** by [`tests/test_zero_install_detector.py`](https://github.com/ThreeMoonsLab/agents-shipgate/blob/main/tests/test_zero_install_detector.py): same `is_agent_project`, same fired frameworks, same suggested sources, same excluded sources, and same Codex plugin candidates for every sample in `samples/`. Field-by-field byte parity is not pinned and not promised — the script is not a drop-in replacement for the CLI.

**When to use this:** you're a coding agent (Claude Code, Codex, Cursor) deciding *whether* to propose Shipgate. The script tells you in one fetch + one Python invocation. The full flow (`init`, `scan`, `apply-patches`) requires the actual install.
Expand Down
5 changes: 5 additions & 0 deletions llms-full.txt
Original file line number Diff line number Diff line change
Expand Up @@ -717,6 +717,11 @@ agents-shipgate apply-patches \

Consume the response to decide whether to proceed. Key fields:

- Detection silently skips common fixture corpus directories such as
`fixtures/`, `_fixtures/`, `__fixtures__/`, `testdata/`, `test_data/`,
`test-fixtures/`, `test_fixtures/`, `golden/`, and `goldens/` when they
are below the selected workspace. Point `--workspace` directly at a
fixture project if you intentionally want to classify that fixture itself.
- `is_agent_project` — `true` when at least one Python framework
scored ≥ 2.0 with a strong signal.
- `frameworks[]` — per-framework scores + evidence + candidate file
Expand Down
9 changes: 9 additions & 0 deletions src/agents_shipgate/cli/discovery/artifacts.py
Original file line number Diff line number Diff line change
Expand Up @@ -88,8 +88,17 @@
"build",
"dist",
"env",
"fixtures",
"_fixtures",
"__fixtures__",
"golden",
"goldens",
"node_modules",
"target",
"test-fixtures",
"test_fixtures",
"test_data",
"testdata",
"venv",
}
SKIP_DIR_PREFIXES = (".venv",)
Expand Down
8 changes: 4 additions & 4 deletions tests/test_adapter_static_only.py
Original file line number Diff line number Diff line change
Expand Up @@ -193,7 +193,7 @@ class AllowedException:
AllowedException(
relative_path="cli/discovery/artifacts.py",
surface="attr_call:subprocess.run",
line=430,
line=439,
snippet=(
"subprocess.run(['git', '-C', str(workspace), 'rev-parse', "
"'--show-toplevel'], check=False, capture_output=True, "
Expand All @@ -208,7 +208,7 @@ class AllowedException:
AllowedException(
relative_path="cli/discovery/artifacts.py",
surface="attr_call:subprocess.run",
line=446,
line=455,
snippet=(
"subprocess.run(['git', '-C', str(workspace), 'ls-files', "
"'-co', '--exclude-standard', '--full-name', '-z', '--', "
Expand Down Expand Up @@ -1283,7 +1283,7 @@ def test_allowed_exceptions_pin_subprocess_run_per_call_site() -> None:
``subprocess.run`` AllowedException entries (one per call site at
lines 480, 481, 486), not one blanket entry that permits all
occurrences. Same for ``cli/discovery/artifacts.py`` (two call
sites at 361 and 377). Adding a fourth ``subprocess.run`` to
sites at 439 and 455). Adding a fourth ``subprocess.run`` to
``triggers.py`` must require adding a new ALLOWED_EXCEPTIONS entry.

If the test fails because lines drifted, that is the intended
Expand Down Expand Up @@ -1312,7 +1312,7 @@ def test_allowed_exceptions_pin_subprocess_run_per_call_site() -> None:
assert len(artifacts_subprocess_run) == 2, (
f"Expected 2 distinct AllowedException entries for "
f"cli/discovery/artifacts.py subprocess.run calls (one per "
f"call site at lines 361 and 377), got "
f"call site at lines 439 and 455), got "
f"{len(artifacts_subprocess_run)}."
)
verify_subprocess_run = by_file_surface.get(
Expand Down
85 changes: 85 additions & 0 deletions tests/test_detect.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,41 @@
from agents_shipgate.schemas.detect import DetectResult

SAMPLES = Path(__file__).resolve().parent.parent / "samples"
FIXTURE_SKIP_DIR_NAMES = (
"fixtures",
"_fixtures",
"__fixtures__",
"testdata",
"test_data",
"test-fixtures",
"test_fixtures",
"golden",
"goldens",
)


def _write_skipped_fixture_signals(root: Path) -> None:
root.mkdir(parents=True, exist_ok=True)
(root / "agent.py").write_text(
"from langchain.tools import tool\n\n@tool\ndef lookup():\n return 'x'\n",
encoding="utf-8",
)
tools = root / "tools"
tools.mkdir()
(tools / "payments-mcp.json").write_text(
'{"tools": [{"name": "create_payment_link", "description": "Create link."}]}',
encoding="utf-8",
)
(root / "broken-mcp.json").write_text("{not json", encoding="utf-8")
specs = root / "specs"
specs.mkdir()
(specs / "support.openapi.yaml").write_text(
"openapi: 3.1.0\ninfo:\n title: T\n version: '1'\npaths: {}\n",
encoding="utf-8",
)
plugin = root / "plugin" / ".codex-plugin"
plugin.mkdir(parents=True)
(plugin / "plugin.json").write_text("{}", encoding="utf-8")


def test_detects_langchain_sample() -> None:
Expand Down Expand Up @@ -153,6 +188,21 @@ def test_detect_ignores_local_private_and_virtualenv_fixtures(tmp_path: Path) ->
assert result.suggested_sources == []


def test_detect_excludes_common_fixture_dirs_by_default(tmp_path: Path) -> None:
"""Fixture corpora should not make an otherwise empty workspace look agentic."""
for dirname in FIXTURE_SKIP_DIR_NAMES:
_write_skipped_fixture_signals(tmp_path / dirname)

result = detect_workspace(tmp_path)

assert result.is_agent_project is False
assert result.frameworks == []
assert result.suggested_sources == []
assert result.excluded_sources == []
assert result.codex_plugin_candidates == []
assert result.workspace_signals.python_file_count == 0


def test_detect_does_not_skip_workspace_because_parent_is_skipped_name(
tmp_path: Path,
) -> None:
Expand All @@ -170,6 +220,22 @@ def test_detect_does_not_skip_workspace_because_parent_is_skipped_name(
assert langchain.candidate_files == ["agent.py"]


def test_detect_does_not_skip_workspace_named_fixtures(tmp_path: Path) -> None:
workspace = tmp_path / "fixtures"
workspace.mkdir()
(workspace / "agent.py").write_text(
"from langchain.tools import tool\n\n@tool\ndef lookup():\n return 'x'\n",
encoding="utf-8",
)

result = detect_workspace(workspace)

assert result.is_agent_project is True
langchain = next(fw for fw in result.frameworks if fw.type == "langchain")
assert langchain.candidate_files == ["agent.py"]
assert result.workspace_signals.python_file_count == 1


def test_detect_respects_gitignored_nested_agent_artifacts(tmp_path: Path) -> None:
if not shutil.which("git"):
pytest.skip("git is required for git-aware discovery regression coverage")
Expand Down Expand Up @@ -199,6 +265,25 @@ def test_detect_respects_gitignored_nested_agent_artifacts(tmp_path: Path) -> No
assert result.suggested_sources == []


def test_detect_excludes_common_fixture_dirs_with_git_candidates(
tmp_path: Path,
) -> None:
if not shutil.which("git"):
pytest.skip("git is required for git-aware discovery regression coverage")

subprocess.run(["git", "init", "-q"], cwd=tmp_path, check=True)
_write_skipped_fixture_signals(tmp_path / "fixtures")

result = detect_workspace(tmp_path)

assert result.is_agent_project is False
assert result.frameworks == []
assert result.suggested_sources == []
assert result.excluded_sources == []
assert result.codex_plugin_candidates == []
assert result.workspace_signals.python_file_count == 0


def test_pyproject_seeds_project_name_not_agent_name(tmp_path: Path) -> None:
"""pyproject [project].name → project_name_candidates, NOT
agent_name_candidates (post-review correction)."""
Expand Down
52 changes: 52 additions & 0 deletions tests/test_zero_install_detector.py
Original file line number Diff line number Diff line change
Expand Up @@ -134,6 +134,24 @@ def test_script_emits_canonical_top_level_keys(script_module):
SCRIPT_PARITY_GAPS: frozenset[str] = frozenset({"n8n_workflow_agent"})


def _write_skipped_fixture_signals(root: Path) -> None:
root.mkdir(parents=True, exist_ok=True)
(root / "agent.py").write_text(
"from langchain.tools import tool\n\n@tool\ndef lookup():\n return 'x'\n",
encoding="utf-8",
)
tools = root / "tools"
tools.mkdir()
(tools / "payments-mcp.json").write_text(
'{"tools": [{"name": "create_payment_link", "description": "Create link."}]}',
encoding="utf-8",
)
(root / "broken-mcp.json").write_text("{not json", encoding="utf-8")
plugin = root / "plugin" / ".codex-plugin"
plugin.mkdir(parents=True)
(plugin / "plugin.json").write_text("{}", encoding="utf-8")


@pytest.mark.parametrize("sample_dir", _sample_dirs(), ids=_sample_ids())
def test_script_verdict_matches_cli(script_module, sample_dir):
"""Structural parity: for every sample, the zero-install script
Expand Down Expand Up @@ -226,6 +244,40 @@ def test_script_finds_at_least_one_python_file_when_cli_does(
)


def test_script_and_cli_skip_common_fixture_dirs(script_module, tmp_path):
_write_skipped_fixture_signals(tmp_path / "fixtures")

script_result = script_module.detect(tmp_path)
cli_result = detect_workspace(tmp_path.resolve()).model_dump(mode="json")

for result in (script_result, cli_result):
assert result["is_agent_project"] is False
assert result["frameworks"] == []
assert result["suggested_sources"] == []
assert result["excluded_sources"] == []
assert result["codex_plugin_candidates"] == []
assert result["workspace_signals"]["python_file_count"] == 0


def test_script_detects_workspace_named_fixture_dir(script_module, tmp_path):
workspace = tmp_path / "fixtures"
workspace.mkdir()
(workspace / "agent.py").write_text(
"from langchain.tools import tool\n\n@tool\ndef lookup():\n return 'x'\n",
encoding="utf-8",
)

script_result = script_module.detect(workspace)
cli_result = detect_workspace(workspace.resolve()).model_dump(mode="json")

assert script_result["is_agent_project"] is True
assert cli_result["is_agent_project"] is True
assert [fw["type"] for fw in script_result["frameworks"]] == ["langchain"]
assert [fw["type"] for fw in cli_result["frameworks"]] == ["langchain"]
assert script_result["workspace_signals"]["python_file_count"] == 1
assert cli_result["workspace_signals"]["python_file_count"] == 1


def test_script_excludes_mcpservers_config_like_cli(script_module, tmp_path):
"""The load-bearing parse-probe case: a Cursor-style ``mcpServers``
host config matches the ``*mcp*.json`` glob but is not a tools-array
Expand Down
10 changes: 8 additions & 2 deletions tools/shipgate-detect.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,11 @@
every sample in ``samples/``, so the two cannot drift on the load-bearing
fields.

Both this script and the canonical CLI silently skip common fixture corpus
directories (for example ``fixtures/``, ``testdata/``, and ``golden/``) when
those directories are below the selected workspace. Point ``--workspace``
directly at a fixture project to detect that fixture itself.

Like the canonical CLI, glob-matched MCP/OpenAPI candidates are
parse-probed before they are suggested: a filename is a glob match, not a
guarantee. A Cursor plugin ``mcp.json`` is an ``mcpServers``-style host
Expand Down Expand Up @@ -59,7 +64,7 @@
from pathlib import Path
from typing import Any

SCRIPT_VERSION = "0.2.0"
SCRIPT_VERSION = "0.2.1"

# Framework signal vocabulary (mirror of cli/discovery/signals.py).
LANGCHAIN_IMPORTS = {
Expand Down Expand Up @@ -120,7 +125,8 @@
".nox", ".svn", ".mypy_cache", ".next", ".pnpm-store", ".pytest_cache",
".ruff_cache", ".turbo", ".tox", ".venv", "__pycache__",
"agents-shipgate-reports", "build", "dist", "env", "node_modules",
"target", "venv",
"target", "venv", "fixtures", "_fixtures", "__fixtures__", "golden",
"goldens", "test-fixtures", "test_fixtures", "test_data", "testdata",
}
PYPROJECT_NAME_RE = re.compile(r'^\s*name\s*=\s*["\']([^"\']+)["\']', re.MULTILINE)
REQ_TOKEN_RE = re.compile(r"^\s*([A-Za-z0-9_.\-]+)", re.MULTILINE)
Expand Down