From 38a68d6a8723bdcb327a7750bb3ea9d7b1d48271 Mon Sep 17 00:00:00 2001 From: Megan Maton Date: Fri, 31 Jul 2026 15:52:25 +0100 Subject: [PATCH 1/3] docs: add READMEs --- AGENTS.md | 33 ----------------------- multiplex/approach/basic/README.md | 18 +++++++++++++ multiplex/approach/hazop/README.md | 43 ++++++++++++++++++++++++++++++ multiplex/approach/stpa/README.md | 43 ++++++++++++++++++++++++++++++ 4 files changed, 104 insertions(+), 33 deletions(-) delete mode 100644 AGENTS.md create mode 100644 multiplex/approach/basic/README.md create mode 100644 multiplex/approach/hazop/README.md create mode 100644 multiplex/approach/stpa/README.md diff --git a/AGENTS.md b/AGENTS.md deleted file mode 100644 index 1ffa12d..0000000 --- a/AGENTS.md +++ /dev/null @@ -1,33 +0,0 @@ -# AGENTS.md - -Guidance for AI coding agents working in this repository. - -*multiplex* is a modular framework for prototyping LLM-based mutation testing. One run mutates a single method/function — Java or Python, set by `project.language` and located via tree-sitter — generates mutants with an LLM, splices each mutant back into the source file, and runs the target project's tests to see which mutants are killed. - -## Documentation map - -Task-scoped docs live in `docs/` — start at [docs/README.md](docs/README.md) and load only the file relevant to your task instead of reading source: - -- [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) — pipeline, approaches, execution backends, output artifacts. -- [docs/MODULE_REFERENCE.md](docs/MODULE_REFERENCE.md) — every function's signature, behavior, and side effects. -- [docs/CONFIG.md](docs/CONFIG.md) — full `config.yml` schema. -- [docs/EXTENDING.md](docs/EXTENDING.md) — checklists for adding approaches, execution backends, languages, or LLM providers. -- [docs/DEVELOPMENT.md](docs/DEVELOPMENT.md) — setup, conventions, CI, known bugs/gotchas. - -## Commands - -```bash -uv sync -p 3.13 # install deps (pin 3.13: tiktoken fails to build on 3.14) -uv run pytest tests # all tests — must run from repo root (relative resource paths) -uv run pytest tests/checks/test_compiles.py::test_name # one test -uv run ruff check . # lint (matches CI) -uv run ./multiplex ./path/to/config.yml # run the tool -``` - -## Critical rules - -- **Import convention**: the tool runs as a directory (`uv run ./multiplex`), so `multiplex/` itself is on `sys.path`. Modules inside `multiplex/` import each other **without** the package prefix (`from model import Model`); tests import **with** it (`from multiplex.checks... import ...`). Match the file you are editing. Rationale in `docs/DEVELOPMENT.md`. -- Pipeline entry point and dispatch: `multiplex/__main__.py`. Adding an approach touches three places (new `approach//controller.py` package + `APPROACH_PROMPT_KEYS` entry in `multiplex/prompts.py` + dispatch branch). Only the selected approach's `system_prompts` keys are required; `resolve_prompts` validates them (and the approach name) up front, raising `SystemExit` before any destructive step. See `docs/EXTENDING.md`. -- `/output/` is wiped at the start of every run; the target source file is edited in place and restored from a `.orig` backup. -- tree-sitter versions are pinned; do not bump them casually. -- Runnable examples live in `examples/`: `uv run ./multiplex ./examples/config-java.yml` (Java, Maven project — needs `mvn` + a JDK) and `uv run ./multiplex ./examples/config-python.yml` (Python, pytest backend). Both need a running Ollama. See `docs/DEVELOPMENT.md` § Example. diff --git a/multiplex/approach/basic/README.md b/multiplex/approach/basic/README.md new file mode 100644 index 0000000..1208fa6 --- /dev/null +++ b/multiplex/approach/basic/README.md @@ -0,0 +1,18 @@ +# Unguided/Basic Configuration + +The unguided or `basic` configuration requires the `config.yml` file to contain the following: +Update the prompts for your chosen language. + +```yaml +mutation: + approach: basic + +system_prompts: + basic_generate_mutants: | + You are a code generator. + When given a Java method, re-implement it with a single small behavioral defect (a mutant). + Change exactly one thing: an operator, a boundary condition, a constant, or a returned value. + Include a concise code comment describing the injected defect. + Output the defective Java code only and nothing else. + +``` diff --git a/multiplex/approach/hazop/README.md b/multiplex/approach/hazop/README.md new file mode 100644 index 0000000..08da892 --- /dev/null +++ b/multiplex/approach/hazop/README.md @@ -0,0 +1,43 @@ +# HAZOP Configuration + +The HAZOP configuration requires the `config.yml` file to contain the following: +Update the prompts to suite your chosen language. + +```yaml +mutation: + approach: hazop + +system_prompts: + hazop_describe_process: | + When given a Java method, describe the intent of the code in a line-by-line manner, as concisely as possible. + + Explain what the code does, not how it is written. + End each line with a comma and don't use commas anywhere else. + Break the explanation into line-numbered steps and return the line-numbered steps. + hazop_identify_deviations: | + For each numbered description, output one version of the string where one of the rules has changed. + The rule must be changed using one of the "guidewords" from HAZOP. + The guidewords and their interpretation is delimited by ###, in the form guideword - interpretation. + That is, everything before - is the guideword and everything after is the meaning. + + ### + + No (not, none) - None of the design intent is achieved + More (more of, higher) - Quantitative increase in a parameter + Less (less of, lower) - Quantitative decrease in a parameter + As well as (more than) - An additional activity occurs + Part of - Only some of the design intention is achieved + Reverse - Logical opposite of the design intent occurs + Other than (other) - Complete substitution (another activity takes place or an unusual activity occurs or uncommon condition exists) + + ### + + Please output each new rule, as a csv line, without headings in the format: + "number, original_rule, guideword (CAPITALIZED), changed_rule" + hazop_implement_deviations: | + You are a code generator. + When given a description, re-implement the method but with the incorrect behavior/defect as described in the description. + + Include a concise code comment to describe the implemented defect. + Output the defective Java code only and nothing else. +``` diff --git a/multiplex/approach/stpa/README.md b/multiplex/approach/stpa/README.md new file mode 100644 index 0000000..45e3e25 --- /dev/null +++ b/multiplex/approach/stpa/README.md @@ -0,0 +1,43 @@ +# STPA Configuration + +The STPA configuration requires the `config.yml` file to contain the following: +Update the prompts for your chosen language. + +```yaml +mutation: + approach: stpa + +system_prompts: + stpa_describe_control_flow: | + You describe Java methods as control flow diagrams. + When given a Java Method, create a written description in the DOT language used in GraphViz of the control structure. + Use human readable labels and descriptions for each node. + Output the control diagram only, and nothing else. + stpa_identify_ucas: | + Identify 10 different Unsafe Control Actions (UCAs) within the control diagram described in DOT language, referencing the code, using each of the guidewords delimited by ### below: + + ### + * provides + * does not provide + * too early + * too late + * out of order + * stopped too soon + * applied for too long + + ### + + You must describe each UCA in the format: + + An example of this is: + variable x calculated out of order when sorting has not occurred. + + Output the numbered list of UCAs with no empty lines, and nothing else. + stpa_implement_ucas: | + You are a code generator. + When given a description, re-implement the method but with the incorrect behavior/defect as described in the description. + + Include a concise code comment to describe the implemented defect. + Output the defective Java code only and nothing else. + +``` From 5fec93faac87eb289265ab44a7139129702860cb Mon Sep 17 00:00:00 2001 From: Megan Maton Date: Fri, 31 Jul 2026 16:04:18 +0100 Subject: [PATCH 2/3] docs: README.md --- multiplex/approach/llmorpheus/README.md | 14 ++++++++++++++ 1 file changed, 14 insertions(+) create mode 100644 multiplex/approach/llmorpheus/README.md diff --git a/multiplex/approach/llmorpheus/README.md b/multiplex/approach/llmorpheus/README.md new file mode 100644 index 0000000..ee650fd --- /dev/null +++ b/multiplex/approach/llmorpheus/README.md @@ -0,0 +1,14 @@ +# LLMorpheus Configuration + +The LLMorpheus configuration requires the `config.yml` file to contain the following: +Update the prompts for your chosen language. + +```yaml +mutation: + approach: llmorpheus + +system_prompts: + llmorpheus_system: | + You are an expert in mutation testing. Your job is to make small changes to a project's code in order to find weaknesses in its test suite. + If none of the tests fail after you make a change, that indicates that the tests may not be as effective as the developers might have hoped, and provide them with a starting point for improving their test suite. +``` From 92af9e4c9577ae4c12ac2cc588d45f9b3e4f2530 Mon Sep 17 00:00:00 2001 From: Megan Maton Date: Fri, 31 Jul 2026 16:37:01 +0100 Subject: [PATCH 3/3] chore: tidy --- multiplex/prompts.py | 13 ++----------- 1 file changed, 2 insertions(+), 11 deletions(-) diff --git a/multiplex/prompts.py b/multiplex/prompts.py index 52daa62..8feedb6 100644 --- a/multiplex/prompts.py +++ b/multiplex/prompts.py @@ -1,8 +1,5 @@ -"""Resolve and validate system prompts for the selected mutation approach.""" +"""Check prompts for the selected mutation approach.""" -# Each approach uses only its own system prompts. Only the selected approach's -# keys are required for a run; unrelated keys may be omitted from the config. -# Adding an approach means adding its required prompt keys here. APPROACH_PROMPT_KEYS = { "basic": ["basic_generate_mutants"], "hazop": [ @@ -21,13 +18,7 @@ def resolve_prompts(config, approach): - """Return the system prompts required by ``approach``. - - Only the keys the selected approach actually uses are read, so a config need - not define prompts for approaches it will not run. Raises ``SystemExit`` with - an actionable message if the approach is unknown or any required - ``system_prompts`` key is missing or empty. - """ + """Return the system prompts required by ``approach``.""" if approach not in APPROACH_PROMPT_KEYS: valid = ", ".join(sorted(APPROACH_PROMPT_KEYS)) raise SystemExit(f"Invalid approach '{approach}'. Choose one of: {valid}")